Enhancing Natural Experiment Research Through Qualitative Work
Qualitative research plays a crucial role in enhancing natural experiment studies by providing insights into the processes of exposure to interventions, identifying variables influencing exposure likelihood, and determining suitable control groups. Complementary methods within the RE-AIM framework aid in assessing the public health impacts of policy interventions by examining key dimensions such as reach, effectiveness, adoption, implementation, and maintenance. Mixed methods incorporating qualitative inquiry help understand the underlying reasons behind results and outcomes across various dimensions.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
How Qualitative Work Can Enhance Natural Experiment Research Beth Nauman, PhD, MPH Managing Director, Louisiana Public Health Institute bnauman@lphi.org
The Common Thread One characteristic of qualitative research is Naturalistic Inquiry that is, studying a phenomenon in the natural setting . Naturalistic inquiry is contrasted to experimental research where the investigator attempts to completely control the condition of the study. Likewise, natural experiments occur when a particular intervention is implemented but the circumstances surrounding the implementation are not under the control of researchers. Thus, qualitative research has the potential to enhance natural experiment studies that evaluate interventions that occur without controlled conditions, such as health policy change.
Qualitative Inquiry in Natural Experiment Research Robust evaluation of natural experiments depends, in part, on the extent that the process by which people fall into treatment and control groups is random. Without being able to manipulate the intervention such that exposure of subjects is completely random, it is often not the case. This is where qualitative research has an important role: To better understand the processes by which people are exposed to an exogenous intervention May help identify measurable and/or unobserved variables that influence likelihood of exposure. May help with determining a suitable control group.
Illustrating Complementary Methods within the RE-AIM Framework RE-AIM is a common framework applied to studying the public health impacts of policy interventions RE-AIM examines key dimensions for the successful and impactful implementation of health policy interventions: Reach, Effectiveness, Adoption, Implementation and Maintenance Mixed methods enable a comprehensive assessment and evaluation of policy interventions encompassing all five components of RE-AIM. Qualitative methods help understand how and why results on individual RE-AIM dimensions occur or how results across dimensions relate.
RE-AIM Framework Dimension Definition Quantitative Assessment Qualitative Inquiry Number/percent of people who participate in or receive the intervention (e.g. utilization data) and their measurable characteristics (e.g. demographics) Quantity and representativeness of individuals who receive the intervention What factors affect whether someone participates in or receives the intervention? Reach Compare changes in important clinical outcomes between treatment and control groups Do stakeholders find the effectiveness meaningful? Reasons for different effectiveness across subgroups Impact of the intervention on outcomes Effectiveness Quantity and representativeness of settings and intervention agents who initiate the program Number/proportion of settings and number/types implementing staff delivering the intervention Why different sites/providers implement or don t; challenges/barriers to successful adoption Adoption Fidelity to elements of an intervention s protocol (i.e. consistency/standardization across settings) Understanding implementers /users priorities in terms of intervention/program components; motivations for/barriers to certain elements of the intervention Assessing measurable components like time, frequency, proportion of elements delivered, etc Implementation Extent to which a program or policy is sustained over time; individuals continuation in the program; or sustained effects on outcomes over time Quantity of encounters with the intervention/program over time (e.g. utilization data); persistence of desired outcomes Why organizations continue or discontinue implementation; why individual benefits continue or fade Maintenance
Qualitative Methods for Health Policy Interventions Key informant interviews: healthcare providers and staff, patients and caregivers, health system leaders/decision-makers, payers Focus groups: care teams, patient subgroups Human-centered design: iterative design process involving patients (or providers), focusing on their needs in each phase of the process Observation or ethnography: document how the intervention is carried out within implementation settings in real time Public deliberation: facilitated deliberation sessions with a citizen panel to assess public perceptions, values, concerns, and expectations pertaining to health policy
Examples of Qualitative Work in NEXT-D2
Non-Face-to-Face Chronic Care Management Year Method(s) Topics Year 1 Key informant interviews with health system leaders, providers delivering NFF CCM, and patients with Diabetes Challenges to setting up the program Motivations for adoption (e.g. perceived advantages of the program) Adoption and implementation logistics (e.g. staffing, billing, IT) Factors associated with patient participation (e.g. copays, dual eligibility) Year 2 Key informant interviews with physicians, health system administrators, and other healthcare staff Barriers and facilitators for implementation of NFF CCM Comparing implementation in different healthcare systems Year 3 Semi-structured interviews with 30 patients age 65+ with Diabetes; thematic analysis; illustrative case study Perceived value of NFF CCM Greatest needs for support through CCM services Case study describing in detail an individual patient's experience with NFF CCM Year 4 Interviews with 16 key informants (physicians, administrators, NFF CCM program staff/providers) from health systems and a third-party vendor Program compliance/fidelity and documentation motivated by revenue generation Quality of services and value to the patients Year 5 Design thinking workshop with patients Generate actionable recommendations for NFF CCM implementation by conducting a patient-centered design workshop to co-create prototypes for NFF CCM services
NFF CCM: Qualitative Findings Dimension Definition Select Findings Quantity and representativeness of individuals who receive the intervention Initially, health systems targeted high utilization patients for the program, citing they re most likely to benefit Reach Higher-need patients cited more value for NFF CCM, while more self-sufficient patients perceived the program to be less beneficial/necessary for them Providers described patients becoming more highly engaged in their care (vs. passive) Effectiveness Impact of the intervention on outcomes Quantity and representativeness of settings and intervention agents who initiate the program Challenges with adoption included staffing the program, setting up required documentation in the EHR and billing procedures Motivation for adoption: opportunity to get reimbursed for what we already do Adoption Fidelity to elements of an intervention s protocol (i.e. consistency/standardization across settings) Providers cited documentation required for billing as an implementation priority Patients prefer phone versus other outreach methods; place more value on live interactions in part due to lack of proficiency with tech Implementation One health system transitioned to a third-party vendor after piloting the program in- house, citing scalability and revenue potential Another health system discontinued the program after piloting, citing barriers to scaling the program for sufficient revenue Administrators identified size of the organization, scale of the NFF CCM program, and cost-effectiveness as key factors for program success/sustainability Extent to which a program or policy is sustained over time; individuals continuation in the program; or sustained effects on outcomes over time Maintenance
Medicaid Health Homes Objective: To understand patients experiences with the health home program Methods: In-depth interviews and focus groups with Medicaid patients Principal finding: Health home program facilitates better access to care Impact on NE research: Examined multiple measures of access as outcomes in the quantitative analyses
Intensive Behavioral Therapy for Obesity Objective: To understand barriers to implementation as well as successes with delivering Medicare s intensive behavioral therapy (IBT) for obesity in primary care settings Methods: Semi-structured interviews with primary care providers who have delivered IBT Identify emergent themes pertaining implementation challenges and successes Describe provider and practice characteristics (including rurality) Study currently underway
Telehealth during the COVID-19 Pandemic Objective: Examine facilitators and barriers to uptake, adoption, and implementation of telehealth services among Medicare patients with diabetes from health systems , providers and patients perspectives Qualitative methods: Semi-structured interviews with patients age 65+ who had at least one telehealth encounter for diabetes since March 6, 2020 Semi-structured interviews with providers of telehealth to patients with diabetes Topics: Patients experiences with telehealth: perceived benefits, challenges, and preferences Providers experiences delivering telehealth: what needs does it meet and what gaps does it leave in patient care for diabetes? Study currently underway
Thank you! Thank you!
