Realist Synthesis of PROMs Feedback in Patient Care

School of Sociology and Social Policy Social Policy School of Sociology and A realist synthesis of PROMs feedback to improve the care of individual patients Dr. Joanne Greenhalgh Associate Professor j.greenhalgh@leeds.ac.uk

Acknowledgements Functionality and Feedback: a realist synthesis of the collation, interpretation and use of PROMs data to improve patient care is funded by National Institute for Health Research Health Services and Delivery Research Programme 12/136/31 The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the HS&DR Programme, NIHR, NHS or the Department of Health. Greenhalgh J, Pawson R, Wright J, Black N, Valderas JM, Meads D, Gibbons E, Wood L, Wood C, Mills C, Dalkin S (2014) Functionality and feedback: a protocol for a realist synthesis of the collation, interpretation and utilisation of PROMs data to improve patient care. BMJ Open 2014;4:e005601.doi:10.1136/bmjopen- 2014-005601.

Project Team Joanne Greenhalgh Name Expertise Affiliation Realist synthesis/PROMs University of Leeds Realist methods University of Leeds Sonia Dalkin Realist methods University of Leeds Kate Gooding Realist methods University of Leeds Ray Pawson Health Economics University of Leeds David Meads Information Specialist University of Leeds Judy Wright Nurse/PROMs University of Oxford Elizabeth Gibbons PROMs/policy London School of Hygiene and Tropical Medicine Nick Black GP/PROMs University of Exeter Chema Valderas Consultant Epidemiologist NE Quality Observatory System Liz Lingard Nurse JJ Consulting Jane Jackson PPI representative Leeds Laurence Wood

Overview of presentation What is realist synthesis? Findings to illustrate interplay between theory and evidence in realist synthesis

What is Realist Synthesis Review methodology which seeks to explore what works, for whom, in what circumstances rather than does this intervention work or not Unit of analysis is the programme theory (ideas and assumptions underlying what the intervention is trying to achieve and how), not the intervention itself (what is done) Programme theories expressed as hypotheses to be tested in this situation, the programme works in this way and produces these outcomes Different forms of evidence (qual/quant) make sense of each other The hard slog remains: uncovering, reading and digesting hundreds of primary studies

The Technical Sequence Realist Synthesis Template (simplified) Search to provide overview of programme theory (i.e. theory elicitation ) Question selection to pinpoint key processes for investigation (i.e. theory selection ) Search for studies best placed to test the chosen theory (i.e. theoretical sampling ) Selection of primary studies, not against a hierarchy of evidence, by their theory testing potential Data extraction, not to a standard matrix, but to as confrontation of theory with evidence Synthesis as theory refinement Dissemination as production of abstract middle-range theory (i.e. theory re-articulation)

Identifying programme theories Search for editorials, reviews, think pieces, letters, opinion pieces Personal libraries of project team How is PROMs feedback intended to work? For whom/in what circumstances might it NOT work as intended?

Basic programme theory Within scope of review

PROMs act as a tool to support patients sharing/raising issues with clinicians DEPENDING ON (context) Structure of PROM Nature of patient-clinician relationship Other incentives/ use of PROMs data RESOURCE PROMs completion REASONING Patients reflect on their situation and feel like they have permission to raise issues OUTCOME Patient raises/ discusses issues with the clinician

How we tested these theories Theory: Whether PROM completion support patients to reflect on their health and raise issues within the consultation depends on the structure of the PROM, the existing nature of the clinician-patient relationship and other, potentially conflicting uses of these data Examined different contextual configurations by looking at PROMs use in different settings and using different PROMs: Contextual configuration each setting represents Setting N studies Nature of relationship Structure of PROM Use of incentives/ other use of data Mental health primary care 4 Usually have existing relationship Standardised QOF, indicator of service quality Secondary mental health care Palliative care 4 New but can also be ongoing Standardised Indicator of service quality 9 New Standardised and individualised None

Do PROMs support or constrain the patient- clinician relationship and enable patients to raise issues? We hypothesised this would depend on: The structure of the PROM The existing relationship between patients and clinicians The point in the relationship-building process when the PROM is completed/reviewed Patients and clinicians existing preferences for relationship building

Theory: support or constrain the clinician patient relationship? Patients and clinicians felt patients sharing concerns depended on a trusting relationship: I think you d have to have a trust thing built up first to actually share something with that person (young mental health service user, Stasiak et al (2012) a lot of picking up depression is about rapport and about patients feeling comfortable and establishing a relationship (GP, Leydon et al, 2011) Clinicians preferred to let patients verbalise their concerns, rather than ask patients to complete a PROMs their story and their words tell me what they understand but also what words they use and whether there s any anger there or um.. denial (palliative care nurse, Gamlen and Arber, 2013, palliative care) you can t beat someone you feel confidence in and you feel comfortable with and all these things come out and no amount of bits of paper is going to change that (palliative care patient, Slater and Freeman, 2004)

Structure of the PROM: clinicians views Clinicians in primary and secondary mental health settings perceived standardised PROMs constrained the relationship/trust building process: If you ve had a very loaded consultation the HAD scale can appear to trivialise the depth of emotions (Leydon et al, 2011, primary mental health) I like to let them verbalise their concerns rather than handing them a bit of paper and say tick boxes (palliative care nurse, Gamlen and Arber, 2013, palliative care) They found it hard to fit the PROM into the flow of the consultation: Where do you plonk those great big bombshells in the middle of a normal consultation with somebody (GP, Leydon et al, 2011) they.. break down in tears and tell you how depressed they re feeling . and then oh now I ve got this questionnaire to fill out I just think its so inappropriate sometimes (GP, Mitchell et al, 2011)

Contextual configurations: PROMs structure and incentives Therefore, clinicians either: Avoided using them altogether Used them at the end of the consultation or in later meetings when they had built up a relationship with the patient Adapted how they were used (changed items, how they were administered) may have affected their validity When financial incentives were used, clinicians also engaged in gaming the data: diagnoses of what would be QOF-able depression has probably dropped we realised if we kept labelling people as depressed when they perhaps weren t, then we weren t going to see them again and lose the points . (GP, Mitchell et al, 2011)

Individualised PROMs Clinicians in palliative care and secondary mental health services perceived individualised PROMs supported the relationship building process Therapists felt completion of the SEIQoL helped them to form a better relationship and trust with the service user because it enabled them to get alongside the service user more quickly (counsellor, drug and alcohol services, Cheyne et al, 2001) I feel that the tool really helps the client to tell their story (palliative care, Annells and Koch (2001) Individualised measures acted as a conversation opener However, less useful as an outcome measure

PROMs act as a tool to support patients sharing/raising issues with clinicians CONTEXT Clinicians prefer to develop rapport verbally, are asked to use a standardised PROMs but are not mandated to them (eg palliative care) RESOURCE Standardised PROMs completion REASONING & BEHAVIOUR Clinicians don t use standardised PROMs to avoid upsetting patients OUTCOME Patient has no opportunity to complete PROM or use completion as an opportunity to raise issues

PROMs act as a tool to support patients sharing/raising issues with clinicians CONTEXT Clinicians prefer to develop rapport verbally but are mandated to collect standardised PROMs and lose money if they don t (eg primary care) RESOURCE Standardised PROMs completion OUTCOME PROM used maybe invalid as a measure of depression and prevalence of depression is under-reported REASONING & BEHAVIOUR GPs adapt the PROM to fit into consultation or avoid coding people as depressed to avoid using the PROM without also losing income

PROMs act as a tool to support patients sharing/raising issues with clinicians Context Clinicians prefer to develop rapport verbally and can choose which PROM to use (eg palliative care) RESOURCE Individualised PROMs completion REASONING & BEHAVIOUR Clinicians perceive PROMs enable the patient to tell their story and use the PROM in their patient assessment OUTCOME Patients are enabled to share concerns with clinicians

Summary Realist synthesis is a process of bringing theory to bear on evidence It is a process of identifying, testing and then refining theory Context is complex it is not about institutional settings but about the sets of relationships, norms and culture that exist within those settings These shape the ways in which participants respond to the resources offered by interventions and thus the mechanisms through which interventions work and the resulting outcomes