Understanding Monitoring and Evaluation in Health Policies
Exploring the importance of Monitoring and Evaluation (M&E) in health policies, this content delves into various aspects such as Health Impact Assessment (HIA), Health Lens Analysis, and Health in All Policies (HiAP). It covers reasons for M&E, types of assessment, data sources, and the significance of continuous monitoring in policy cycles for effective health outcomes.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
10 MONITORING & EVALUATION (M&E), HEALTH IN ALL POLICIES (HiAP), HEALTH IMPACT ASSESSMENT (HIA), AND HEALTH LENS ANALYSIS MODULE 10 Measuring progress in health
LEARNING OBJECTIVES 10 1 List reasons for monitoring and evaluation Recognize different types of monitoring and evaluation related to health 2 Distinguish between inputs, outputs, outcomes and impact 3 Explain the purpose and key steps involved in health impact assessment and health lens analysis 4 5 Identify sources of health data and policy advice
MONITORING AND EVALUATION IN THE REVIEW STAGE OF THE POLICY CYCLE 10 POLICY REVIEW AGENDA SETTING Report Identify problem Evaluate Research Monitor Set agenda IMPLEMENTATION FORMULATION Enforce policy Develop options and strategies Implement policy Negotiate Formulate policy Source: modified from http://www.geostrategis.com/images/policycycle.jpg
MONITORING HEALTH DETERMINANTS 10 The ongoing, systematic collection, analysis, interpretation, and dissemination of data regarding a health-related event for use in public health action. Source: https://www.cdc.gov/mmwr/preview/mmwrhtml/rr5013a1.htm
10 EVALUATION Analysis of the effectiveness of a specific intervention or set of interventions in achieving an intended outcome or set of outcomes.
WHY MONITORING AND EVALUATION IN HiAP? 10 HiAP is not an endpoint in itself, but a continuous approach to the promotion of health, health equity, and health systems. Policy-makers usually require: Evidence of progress that HiAP has improved health and well-being and their determinants in the general population or in targeted population groups; and Justification for the investment in establishing, fostering and sustaining HiAP. Acknowledgements: slide adapted from Zeinab Khadr s Powerpoint for HiAP training in the Eastern Mediterranean Region, February 2017.
WHY MONITORING AND EVALUATION IN HiAP? 10 Once a policy has been changed, subsequent monitoring is necessary to evaluate the outcomes of the new policy, and thus monitoring should be an iterative and cyclical process that operates continuously. Built into the planning stage rather than an afterthought . Acknowledgements: slide adapted from Zeinab Khadr s Powerpoint for HiAP training in the Eastern Mediterranean Region, February 2017. in the Eastern Mediterranean Region, February 2017. Acknowledgements: slide adapted from Zeinab Khadr s Powerpoint for HiAP training
BENEFITS OF MONITORING AND EVALUATION (M&E) 10 Holds partners and agencies accountable and also makes them aware of the health gains made Tests the validity and precision of health impact predictions made in assessments Adds to the body of knowledge/evidence base on interventions Provides early warning of unforeseen results/impacts Empowers communities when they are involved in monitoring and evaluation
BENEFITS OF MONITORING AND EVALUATION (M&E) 10 Provides a way to inform stakeholders and the community Strengthens the vision of health in the community and stakeholders Provides a better understanding of the value of HiAP Lessons learned to improve HiAP approaches
STAGES OF THE HEALTH MONITORING CYCLE 10 THERE ARE FIVE STAGES OF THE CYCLE OF HEALTH MONITORING Select relevant indicator Make changes Obtain data Report results Analyse data Source: WHO (2013) Handbook on Health Inequality Monitoring with a Special Focus on Low- and Middle-Income Countries. Geneva, WHO, p. 2.
DIRECT AND INDIRECT INDICATORS FOR MONITORING HEALTH 10 INDICATORS OFTEN CORRESPOND TO DIFFERENT STAGES OF A POLICY INTERVENTION AND CAN BE CATEGORIZED AS: Inputs the resources used to initiate a policy; Outputs processes, products and services that immediately result from a policy or project; Outcomes short- to medium-term results of a policy or project; and Impacts long-term effects produced by a policy or project.
10 WHAT DO WE WANT TO KNOW? HEALTH IN ALL POLICIES IMPACT CHAIN Output Outcome 1 Outcome 2 Impact WHAT Exposure risk factor Attributable risk proportion Interventions on SDH Healthy Public Policy Well-being, morbidity, and mortality (level and distribution) HOW Strategic partnerships Governance of systems of decision-making Determinants Outcome 3
10 WHAT DO WE WANT TO KNOW? 1 Process evaluation How do Health in All Policies mechanisms (e.g. health lens, health impact assessment, policy dialogues) influence the process of policy-making and government business? Source: slide from the South Australian Health in All Policies Summer School Training Manual, Government of South Australia, 2013.
10 WHAT DO WE WANT TO KNOW? 1 Process evaluation Example questions: Is the health lens operating according to the model? Did participants develop a shared understanding of the aims and process? How was this achieved (or not)? Were appropriate decision-makers involved in the process? Were agencies goals and expectations met? Was the process of mutual benefits to partners? What are the barriers and enablers? Source: slide from the South Australian Health in All Policies Summer School Training Manual, Government of South Australia, 2013.
10 WHAT DO WE WANT TO KNOW? 2 Examine Outputs Is Health in All Policies effective in building health and equity considerations into policy and government business? Example questions: Is there evidence of the process informing policy or programmes? Have decision-making processes changed? Have participants gained new knowledge/skills/attitudes? Is there transfer of learning to other areas of work? Source: slide adapted from the South Australian Health in All Policies Summer School Training Manual, Government of South Australia, 2013.
10 WHAT DO WE WANT TO KNOW? 3 Outcome and impact evaluation What impact do the consequent policies and government business have on the determinants of health and their distribution? Source: slide adapted from the South Australian Health in All Policies Summer School Training Manual, Government of South Australia, 2013.
10 WHAT DO WE WANT TO KNOW? 3 Outcome and impact evaluation Here we face some serious challenges e.g. Complexity of determinants and policy interventions Attributing change to the intervention Often long time frames Measurement tools for social health and well-being Measuring prevention (what didn t happen). Source: slide adapted from the South Australian Health in All Policies Summer School Training Manual, Government of South Australia, 2013.
10 CHARACTERIZING M&E FOR HiAP What impact do the consequent policies and government business have on the determinants of health and their distribution? Establishing the health impacts or outcomes for HiAP is methodologically challenging. Development of a programme logic framework or model that outlines a coherent chain of relationships to show how HiAP activities can lead to long-term goals and population health outcomes. Not seeking to establish causality through statistical correlation (not yet).
10 SOUTH AUSTRALIAN CASE SOUTH AUSTRALIAN CONTEXT: SUPPORTIVE HISTORY AND CHANGING POLICY PRIORITIES 2016 Version Impacts on policy environment Outcomes for South Australian Population Activities Policy Understandings about SDoH entrepreneurs Theory of change Mediated by: Investment in social determinants that contributes to future health and equity Strategies Address social determinants of health (SDoH) Intermediaries (champions) Learning Co-benefits Develop relational systems that connect individuals, agencies and sectors Organisational Institutional culture, capacity and priorities Relationship building and maintenance Focus on SDoH outside the Health sector Capacity I M P L E M E N T A T I O N SA a better place to live with increased population health and equity Networks Public Health Partnerships Branch since early 2014 Policy supports health, wellbeing and equity Intersectoral action is required to bring about change in the SDoH Rapid reviews, desktop analyses & partnership agreements Undertake joint problem/opportunity identification and decision-making Power relationships Requires high level political commitment, dedicated resources and skilled personnel to drive change Outputs Central mandate for action Improved performance against sectoral targets HiAP resources Political will Other - but fewer - HiAP initiatives Training HiAP must address core business of partner agencies Utilise governance Systems that connect HiAP work with senior decision- makers Resources Accountability and reporting Build & secure an authorising environment for intersectional partnerships to promote health within the context of reduced support for HP and for HiAP as a dedicated initiative Adapted from: Baum F., Lawless A., Delany T., MacDougall C., Williams C., Broderick D., Wildgoose D., Harris E., McDermott D., Kickbusch I, Popay J. and Marmot M. (2014)
10 BRINGING IT ALL TOGETHER CHARACTERIZING TYPES OF IMPACT ASSESSMENT APPROACHES Intersectoral policy-making and systems / process aspects of HiAP (process, impact, outcome evaluation) 1 Strengthening public health evidence; observe changes in determinants of health or attributed burden of disease 2 3 Strengthen institutions - public health and socio-political
10 BRINGING IT ALL TOGETHER METHODOLOGICAL APPROACHES Organizational learning and critical action research Realistic evaluation Programme logic as a basis for attribution of health and equity outcomes Theories on policy agenda setting and implementation Complexity as a frame for understanding policy-making processes Health, human impact assessment
STRENGTHEN PUBLIC HEALTH EVIDENCE: SUPPORTING EVALUATIVE CAPACITIES 10 Types of assessments to predict health and health equity impacts (predictive evaluations / policy analysis) Health impact assessment Cost-benefit analysis Integrated impact assessment Environmental impact assessment Human health risk assessment Effectiveness data / intervention research / what interventions work?
WHAT IS HEALTH IMPACT ASSESSMENT (HIA)? 10 A HIA is a combination of procedures, methods and tools that assesses the potential effects of a policy or project on the health of a population and the distribution of those effects within the population. HIAs also identify appropriate actions to manage those effects.
10 HIA IS BASED ON FOUR VALUES Democracy Equity Sustainable development Ethical use of evidence
10 APPLYING HIA Quickly establishes health relevance of the policy or project. Is HIA required? Screening Policy and programme development phase for prospective assessments Identifies key health issues and public concerns, establishes ToR, sets boundaries Scoping Rapid or in-depth assessments of health impacts using available evidence - who will be affected, baseline, prediction, significance, mitigation Appraisal Conclusions and recommendations to remove/mitigate negative impacts on health or to enhance positive impacts Reporting Policy implementation phase Action, where appropriate, to monitor actual impacts on health to enhance existing evidence base Monitoring Source: http://www.who.int/hia/tools/en/
WHAT IS HEALTH LENS ANALYSIS (HLA)? 10 The HiAP Health Lens Analysis process builds on traditional Health Impact Assessment methodology to allow the process to deliver both rigour and flexibility that accommodates the operational culture and policy imperatives of the partner agency. Health lens is a model to analyze a policy problem in order to derive benefit for population health while at the same time being able to support and achieve the goals of other sectors. Evaluation, an essential component of the HiAP process, is built into each individual health lens.
APPLYING HLA 10 Health Lens Analysis The South Australian Health Lens Analysis model
HEALTH IMPACT ASSESSMENT AND HiAP HEALTH LENS ANALYSIS MODEL 10 Health in All Policies is an overarching conceptual framework for systematic engagement with sectors outside of health. The Health Lens Analysis model draws on several components of HIA and like HIA, uses general public health methods of investigation and analysis. HIA is applied to existing programmes or policy.
HEALTH IMPACT ASSESSMENT AND HiAP HEALTH LENS ANALYSIS MODEL 10 HiAP is applied to programmes or policy yet to be developed. HiAP does not rigidly apply the steps of HIA and their application is highly context specific.
STRENGTHEN PUBLIC HEALTH AND OTHER SOCIO-POLITICAL INSTITUTIONS 10 1 Strengthen public health institutions: Reports on sustainable development Reporting on health determinants, including HiAP activities Public health legislation Capacities for health impact evaluation Education and training for HiAP New tools for monitoring and evaluation
10 PUBLIC HEALTH REPORTING
STRENGTHEN PUBLIC HEALTH AND OTHER SOCIO-POLITICAL INSTITUTIONS 10 2 Strengthen social and political institutions: Reporting and laws in parliament includes criteria for health Human rights impact assessments Media education Citizen health forums Education Interlinked data on health determinants
End of Module 10 Please continue to Module 11
