Development and Evaluation of Complex Multicomponent Interventions in Public Health
A key concern in public health initiatives is the lack of emphasis on evaluation during implementation. Rushed introductions of new initiatives without adequate development lead to replacements before effectiveness is determined. Collaborative efforts between researchers and practitioners are crucial. The concept of complex interventions involves interrelated components whose effects cannot be attributed to a single factor, with complexity stemming from various factors including behaviors, targeted groups, outcomes, and intervention flexibility. The issue lies in the settings and research questions, not just the intervention itself. Interventions targeting complex systems should focus on changing the system rather than merely the intervention components.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Development and evaluation of complex multicomponent interventions in public health FACULTY OF MEDICINE DEPARTMENT OF COMMUNITY MEDICINE DR SUDABEH MOHAMADI
Introduction 3 A number of consistent themes have emerged. First is a concern that when new public health initiatives are introduced little thought is given to evaluation.
4 A second is that new initiatives and policy interventions are introduced in too rushed a way with little time for development, piloting, fine tuning, and bedding in before they are replaced by something else.
5 A third theme is the need for researchers and practitioners to work together more closely.
Definition and scope of the concept of a complex intervention 6 A complex intervention is generally held to comprise a number interrelated components where effects cannot be attributed to a single active ingredient . of synergistic
7 Complexity derives from: the number and difficulty of behaviours required by those delivering or receiving the intervention, the number of groups or organizational levels targeted by the intervention, the number and variability of outcomes, and the degree of flexibility or tailoring of the intervention permitted.
8 The issue of complexity is not necessarily to do with the intervention but is about both the complexity of the settings (e.g. family, school, community, country) in which interventions are whether the research question being examined by evaluators intervention requires a simple (e.g. does this work? ) or a complex answer (e.g. for whom does this work and under what circumstances? ). implemented and of the
Complex interventions or complex settings, contexts, or systems 9 Interventions with a number of interrelated parts, may be more helpfully thought of as complicated rather than complex. Complexity is a property of the system that an intervention is trying to change, not an inherent feature of the intervention itself.
10 Complex systems are built up from very large numbers of mutually interacting subunits whose repeated interactions result in rich, collective behaviour.
Standardized or flexible interventions? 11 The development and testing: phases undertaken in drug (1) pre-clinical theoretical phase (2) phase I modelling: defining the components of the intervention
12 (3) phase II exploratory trial: defining the trial and intervention design (4) controlled trial (RCT) phase III definitive randomized (5) phase IV long-term implementation
13 Phase II was specified as the stage at which the intervention could be adapted and changed but it was made clear that once the RCT has begun, the intervention must not evolve, as the RCT results will be unusable if later participants experience a different intervention than earlier ones.
14 Fidelity to treatment protocols or quality standards may effectiveness. be essential to
15 Monitoring will be essential to ensuring that individual styles and evolution in treatment do not render results from different groups of participants excessively reduce the effectiveness of the intervention . incomparable or
16 What should be standardized is the function of the intervention and the steps in the process by which change is to be achieved, not that all the components of an intervention have to be delivered in exactly the same way to each person, or in each setting.
Simple or complex research questions 17 Example: to reduce childhood obesity in a local authority area which includes: a pregnant breastfeeding and healthy weaning health promotion women intervention to encourage for
18 work with nursery schools, child minders, and day care providers to ensure provision of healthy food and sufficient opportunity for physical activity
19 a classroom-based intervention for primary schools aimed at teaching children about good nutrition and being physically active which also involves parents
20 a health promotion intervention via large, local spectator sports clubs to encourage fathers to be physically active with their children and to eat healthily
Intervening at different levels to improve the health of populations 21 The multifaceted interventions was set out in both the 1986 Ottawa Charter for Health Promotion and the later Bangkok Charter. importance of multilevel and
22 Interventions individual, downstream determinants of health such as knowledge and attitudes, as well as community and societal upstream determinants, such as access to health benefiting and health harming goods and services and social inequalities in wealth and power. aim both to address
23 When designing complex public health interventions, the approach to public health but with its origins in ecological systems theory, can provide a useful framework for thinking about the different levels of intervention. socioecological
25 Historical evidence suggests that the greatest improvements in public health have been achieved medical interventions, such as improved sanitation, welfare benefits, and improved workplace safety. through non-
26 Multifaceted and multilevel interventions are more effective. Environmental level interventions are more likely to be cost-effective than individually targeted clinical or non-clinical prevention interventions. published evaluations still tend to focus on one or two levels of intervention.
Health inequalities 27 Public health has the dual goals of improving population health and reducing health inequalities. One approach is to target interventions at disadvantaged groups or those at high risk of morbidity or mortality.
28 This can be problematic, both because it can sometimes be stigmatizing and even harmful, but also effective in reducing these groups risk it may be insufficient in substantial population-level reductions in risk because where disease risks follow a bell-shaped curve (as is the case with almost all non-communicable disease), the majority of cases of a disease will arise from those at low or medium risk (because there are simply more of them). because even if bring about
29 Universalist approaches which do not address the upstream determinants of disease and only address individual behaviour may have the potential for widening inequalities because of the inverse prevention socioeconomically advantaged groups are able to benefit disproportionately from universal interventions. law whereby
30 Progressive advocated as a way of avoiding these difficulties by ensuring that action on health inequalities both addresses the causes of the causes but also by ensuring that the scale and intensity of more downstream interventions proportionate to disadvantage. universalism has been are of the level
31 Stages in the development of complex interventions
Review of existing quantitative and qualitative evidence 32 The first practical step in developing a complex intervention, whether designing something new or modifying an existing approach, is to review the available evidence.
33 If there are already relevant systematic reviews, then it may only be necessary to read these or update them. But if not, you may need to conduct your own review.
34 While quantitative systematic reviews will tell you what is known about the effectiveness of a particular intervention, it is also useful to review relevant qualitative research which may provide important knowledge about the likely acceptability of your proposed intervention and identify potential barriers implementation. and facilitators to
35 It will also be useful to learn about risk and protective factors, both downstream and upstream for the outcome of interest, again preferably drawing on systematic reviews, to consider if existing interventions are addressing these. comprehensive in
Involving policy and practice partners from the outset 36 Consulting or collaborating with key stakeholders at this early stage will also provide useful insights into exactly what is seen to be required from the policy and practice perspective.
Public involvement from the outset 37 Working directly with those whose health will be improved intervention is also vital in ensuring that it is relevant and acceptable. by the planned
38 In the health and social care sectors, pre- existing groups represent particular patient and client groups, which can be engaged as partners in research.
39 INVOLVE is part of the National Institute for Health Research in the United Kingdom and its role is to encourage and support increased public involvement in health research.
Importance of theory 40 All interventions are in essence theories incarnate because they assume that a certain intervention will involve a particular causal process leading to particular outcomes.
41 While interventions formulated without recourse to theory can and do work, there is increasing evidence that they are more likely to be effective when informed by theory. Public dominated by psychological theories. health interventions field is
Development of intervention logic models 42 Intervention theory, that is, how an intervention is intended to produce its effect, is critical to planning a process evaluation.
43 Logic representations of the logical relationships, for example, between an intervention s inputs, activities, outputs, and outcomes. models are diagrammatic
44 Intervention theory and logic models are useful tools in intervention is to be delivered and to what ends, and thereby provide a focus for evaluating process (and outcomes). clarifying how an
Evaluation of complex interventions 45 Piloting and feasibility studies Pilot study: a miniature version of the main trial, designed to test that both the intervention and the evaluation can be implemented as intended.
46 If the pilot is successful and no changes are made then data from it can be incorporated into the main trial (this is termed an internal pilot as opposed to an external one where data cannot be incorporated into the main trial).
47 Feasibility studies: intended to indicate whether or not a definitive trial is feasible and to examine important areas of uncertainty such as the willingness of participants to be randomized, response rates to questionnaires outcome data, or the standard deviation of the primary outcome measure required for the sample size calculation. collecting
48 Such studies can have a much more flexible format and need not necessarily be randomized.
Process evaluation methods 49 As important as outcome evaluation Can examine how interventions are planned, delivered, and received Assess fidelity implementation Clarify causal mechanisms Identify contextual factors associated with variation in outcomes and quality of
Priority areas for process evaluation: fidelity (the intervention is delivered as intended) reach (whether an intervention is received by all those it targeted) dose delivered (the amount or number of units of intervention participants) dose received (the extent of participants active engagement in the scheme) context (local factors which impact implementation) 50 extent to which the offered to
