Systematic Assessment of Adaptations in Health System Interventions

This presentation at the 4th Biennial Conference for the Society for Implementation Research Collaboration discusses the systematic multi-method assessment of adaptations in five health system interventions. It covers the definitions of adaptations, balancing fidelity and adaptation, and lessons learned from the process.

• Health System Interventions
• Adaptations
• Program Implementation
• Fidelity

cheyan Follow

Uploaded on Dec 07, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. Systematic multi-method assessment of adaptations in five health system interventions 4thBiennial Conference for the Society for Implementation Research Collaboration Seattle, September 7-9, 2017 Borsika A. Rabin Marina McCreight Russell Glasgow

2. Acknowledgment Implementation Team - Triple Aim QUERI (TAQ), Denver VA (PI: Catherine Battaglia) VA QUERI funding for TAQ Transition Nurse Program Team, Denver VA (PI: Robert Burke) funding from VA Office of Rural Health, N19-FY14Q3-S0-P01240 Cindy Banh and Timothy Cutler from University of California Davis (PCMH Pharmacist Project) funding from UCSF Divine Family Endowment in Clinical Pharmacy Special thanks to Charlotte Nolan for drafting and to Charlotte Nolan and Roman Ayele for piloting the real-time tracking system

3. Overview Defining adaptations and addressing the balance between fidelity and adaptation Conceptualizing and assessing adaptations Adaptations in five health system interventions Lessons learned and next steps

4. Adaptation defined #1: Adaptation can be defined as the deliberate or accidental (i.e., drift) modification of the program #2: Adaptation of programs often occur to improve the fit (or compatibility) of a program to a new setting or to increase the cultural appropriateness of a program. #3: Adaptations to evidence-based interventions are common and (some researchers suggest) inevitable to meet the needs of a specific context. #4: Adaptations might lessen the effectiveness of the program if they compromise the core elements and underlying logic of a program. 2Carvalho et al. J Public Health Manag Pract 2013; 19(4):348-56. 1http://www.csun.edu/sites/default/files/FindingBalance1.pdf

5. Historical view of fidelity and adaptation

6. A mature view of fidelity and adaptation Attention to BOTH program fidelity and adaptation during the complex process of program implementation is critical to successful, sustained implementation of evidence-based programs. .

7. Balancing fidelity and adaptation #1: Identify core components and flexible components of the intervention: theories and frameworks and core component analysis #2: Choose designs that address both internal and external validity #3: Engage stakeholders early and throughout #4: Make adaptations intentional rather than accidental through planning #5: Assess and document fidelity and adaptation throughout the process

8. Goals of documenting adaptations Provide contextual process data to interpret outcomes (i.e., how adaptations contribute to outcomes) Consider refinements to the recommended intervention & implementation strategies based on observed changes Create an organized list of adaptations that future implementers can consider for success To develop and test replicable, easy-to-use documentation methods for recording adaptations

9. Adapting the Stirman framework using the RE-AIM model and clinical experience WHY What is the purpose of the adaptation? Increase reach, participation, access Increase effectiveness Increase adoption by more clinics/settings or make intervention more aligned with organizational goals Increase implementation/ability of staff to deliver intervention successfully Increase maintenance to make intervention more likely to be institutionalized IMPACT What are (subjective) short term results of the adaptation? Are they positive, negative, no real impact? Did the changes impact: Reach/participation/access Effectiveness Adoption Implementation/ability of staff to deliver intervention successfully Maintenance Stirman SW et al. Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implementation Science 2013, 8:65. Hall et al. Understanding adaptations to patient-centered medical home activities: The PCMH adaptations model Translational Behavioral Medicine, 2017, DOI 10.1007/s13142-017-0511-3

10. Methods to assess adaptation #1: Observational techniques #2: Focused interviews #3: Questionnaires, checklists, and logs #4: Content analysis of key documents and curricula #5: Study databases and clinical databases

11. Sample interview questions WHAT component or part of the intervention was changed in this adaptation; in other words, what was the nature of the change? (For instance, was it a change to program content, format, delivery mode, staff delivering it, patients eligible, where, when or how it was delivered, or what?) WHO was responsible for first suggesting or initiating this change? (Was this the person or persons the ones who implemented the change? (If not, who implemented the adaptation?)) WHEN during the ____ program was this adaptation first made? WHY was this adaptation made? (For example, to get more people to participate, to make the program attractive to more settings, to increase its effectiveness, to make it easier to deliver, to make it easier to maintain or reduce costs, etc.?)

12. Example of Tracking form Example of tracking form

13. 5 health systems interventions Adaptation assessment Timing Study name Method Primary Care Network Pharmacist Program Retrospective Interviews Patient Reported Health Status Prospective Interviews Tracking Multi-modal Pain Care Prospective Interviews Tracking Transition of Care Prospective Interviews Tracking Transitions Nurse Program Prospective Interviews Tracking

14. 5 health systems interventions Adaptation assessment Timing Study name Method Primary Care Network Pharmacist Program Retrospective Interviews Patient Reported Health Status Prospective Interviews Tracking Multi-modal Pain Care Prospective Interviews Tracking Transition of Care Prospective Interviews Tracking Transitions Nurse Program Prospective Interviews Tracking

15. Primary Care Network Pharmacist Program Multiple Primary Care Network clinics affiliated with UC Davies Identify and describe barriers and facilitators of, and adaptations and modifications made during the implementation of a clinical pharmacist-based program Semi-structured interviews with multiple stakeholders (n=14) Retrospective assessment of adaptations Major adaptation (sample): Chance in number of clinics and pharmacists Change in program management Adaptations were used to: Improve local program implementation and delivery Inform future scale out within and outside of UC Davis

16. 5 health systems interventions Adaptation assessment Timing Study name Method Primary Care Network Pharmacist Program Retrospective Interviews Patient Reported Health Status Prospective Interviews Tracking Multi-modal Pain Care Prospective Interviews Tracking Transition of Care Prospective Interviews Tracking Transitions Nurse Program Prospective Interviews Tracking

17. Examples of documented adaptations Change in who contacts patient (task shifting) Change in recruitment materials and protocol (change in intervention) Change in feedback procedures (implementation strategy change) QUERI

18. Lessons learned: Strengths Adaptation data can lead to actionable information for program improvement and scale out Rapid and frequent assessment of modifications for QI that can quickly inform other assessment methods Does not require clinical staff time (or minimal) Flexible: can be based on observations, team meetings, conversations or other inputs Can modify the coding categories or options based on results (depending on whether QI or research) to fit your project General system has worked across diverse projects

19. Lessons learned: Limitations Requires in-depth knowledge of intervention and implementation strategies (In our case) initial differentiation between intervention and implementation strategy unclear Coding reliability not assessed thus far Response categories need adjusting so moving assessment and difficult to do some types of comparisons

20. Next steps Rapid methods to assess adaptation so it can inform current project Continue reflecting on best methods to collect data on adaptations (i.e., approach, timing, frequency, participants, etc.) Methods to integrate data from multiple sources Start to separate intentional and unintentional adaptations as well as adaptations to the intervention and implementation strategy Understanding patterns of adaptations across settings/program types through routine documentation/reporting of adaptations

21. QUERI-wide Adaptation, Fidelity, and Tailoring Special Interest Group Started in December 2015 and meets monthly, co-led by Drs. Borsika Rabin and Russ Glasgow 30+ members from a number of QUERI Programs and beyond (open membership contact Borsika if interested in joining) Focuses on best practices on documenting and reporting fidelity, adaptations and modifications for both intervention and implementation fidelity and identification of existing and developing new pragmatic measurement instruments for these areas. Serves as a Work In Progress forum for colleagues to share and discuss ideas.

22. Implementing a program is like constructing a building. An architect draws upon general engineering principles (theory) to design a building that will serve the purposes for which it is designed. However, the specific building that results is strongly influenced by parameters of the building site, such as the lot size, the nature of the site s geological features, the composition of the soil, the incline of the surface, the stability and extremes of climate, zoning regulations, and cost of labor and materials. The architect must combine architectural principles with site parameters to design a specific building for a specific purpose on a specific site....This dynamic is mirrored in the rough-and-tumble world of the human services. Despite excellent plans and experience, ongoing redesign and adjustment may be necessary. Bauman et al. 1991

23. WHAT DO YOU THINK? GET IN TOUCH: borsika.a.rabin@gmail.com