RHIS Data Analysis Curriculum Overview
MODULE 5:
RHIS Data Analysis
SESSION 2:
Overview of Steps 1-4 of Data Analysis
ROUTINE HEALTH INFORMATION SYSTEMS
A Curriculum on Basic Concepts and Practice
1
The complete RHIS curriculum is available here:
https://www.measureevaluation.org/our-work/ routine-health-information-systems/rhis-curriculum
Learning Objectives and Topics Covered
Objectives
By the end of this session, participants will be able to:
•
Select appropriate indicators for data analysis
•
Conduct a basic desk review of data quality and adjust data if
necessary
•
Select appropriate denominators
•
Compare findings from routine data with findings from other data sources
•
Analyze routine data to produce information products (tables, graphs,
and maps)
Topics Covered
•
Selection of indicators for analysis
•
Desk review of data completeness and internal consistency
•
Selection of appropriate denominators
•
Comparison of findings from routine data with findings from other data
sources
2
5 Steps of Data Analysis
1.
Select a limited set of core indicators
2.
Review data quality
3.
Select appropriate denominators
4.
Reconcile findings with estimates from
other data sources
5.
Communicate key findings (to be
discussed in Module 5, Session 3)
3
S1: First Step
Select a Limited Set of Core Indicators
4
What Makes an Indicator “SMART”?
(Characteristics of Good Indicators)
•
S
pecific
:
Indicator is concrete, detailed, focused, and
well-defined
•
M
easurable
:
Indicator tells how many or how much
and can be measured with identified measurement
sources
•
A
greed upon
:
Stakeholders vested in a specific M&E
question should agree that indicator is relevant
•
R
elevant
:
Indicator generates data that can answer
the question of interest
•
T
ime-bound
:
Indicator specifies time frame of what it is
measuring
5
S1: Select a Limited Set of Core Indicators
Core indicators should:
•
Be based on M&E framework for the national
health sector strategy
•
Be programmatically relevant, and facilitate
program management
•
Reliably and comprehensively assess the
performance of the health system (whether
national or subnational levels)
•
Be clearly defined
•
Have numerators that are measurable with
routine health data
6
S1: Core Indicator with 2 Data Sources: DTP3
7
S1: Core Indicators Measured Reliably with One
Data Source
8
Some Indicators Are Not Clearly Defined or
Cannot Be Reliably Measured with Routine Data
Some indicators are
not clearly defined
:
•
Percentage of health facilities with a skilled provider
•
Percentage of health facilities with an adequate supply of
drugs
Some health indicators
can be reliably measured with a
household survey or health-facility survey but not with data
reported routinely
by health facilities:
•
Proportion of population using an improved drinking water
source
•
Proportion of health facilities with treatment guidelines
•
Infant mortality rate
Some indicators are
not clearly defined and usually cannot be
reliably measured with data reported routinely
by health facilities:
•
% of health facilities with health equipment
9
Practicing the First Step
Exercise: Part 1
•
Review the list of core health indicators (on the
next slide (Handout 5.2.1)
•
Identify which indicators can/cannot be reliably
measured with routine data
o
Can the numerator be defined with routine
data?
o
Do you need additional data sources to
measure the indicator?
10
List of Health Indicators
1.
Percentage of pregnant women
attending antenatal clinics who
are screened for syphilis
2.
Neonatal mortality rate
3.
Number of neonatal tetanus cases
4.
Cost of one month’s supply of
contraception as a percentage of
monthly wages
5.
Percentage of infants born
protected against neonatal
tetanus in a specified period
6.
Measles vaccine coverage rate
7.
Percentage of registered new and
relapse tuberculosis (TB) patients
with documented HIV status
8.
Percentage of children ages 12–59
months who were dewormed in
the past six months
9.
Percentage of HIV-positive infants
born to HIV-positive women
10.
Maternal mortality ratio
11.
Number of health facilities
providing comprehensive
emergency obstetric care
functions per 500,000 population
12.
Exclusive breastfeeding rate
13.
TB treatment success rate
14.
Percentage of health facilities with
systems that support quality service
delivery
15.
Percentage of districts with current
trend analysis for selected priority
diseases at a given time (e.g.,
month)
11
S1: Practicing the First Step
•
Exercise, Part 2
•
Review the WHO’s Global Reference List of Core
Health Indicators (Handout 5.2.3) or the list of
standard indicators in Handout 5.2.2.
•
Select five indicators from the list.
•
For each indicator:
o
Specify the numerator
o
Specify the denominator
o
What is the data source for the numerator?
o
What is the data source for the denominator?
o
How can the indicator be interpreted?
12
S2: Second Step
13
Review Data Quality
14
•
Completeness and timeliness
o
Completeness of reports
o
Completeness of data
o
Timeliness of reports
•
Internal consistency
o
Accuracy
o
Outliers
o
Trends
o
Consistency among indicators
•
External consistency
o
Data triangulation
o
Comparison with data surveys
o
Consistency of population trends
•
External comparisons (population denominators)
Step 2: Review Data Quality (see also Module 4)
S2: Demonstration, Excel
How to use Excel to:
•
Create a chart
Participants are invited to practice simultaneously.
Here are two websites with 5-minute videos explaining how
to create a chart using Excel:
https://www.youtube.com/watch?v=BcsDnRClzCY
http://excelcentral.com/excel2007/essential/lessons/05010-
create-a-simple-chart-with-two-clicks.html
15
S2: Demonstration DHIS 2
How to use DHIS 2 to:
•
Verify the definition of an indicator
o
Numerator and denominator
•
Create a pivot table
•
Create a chart
Participants are invited to practice simultaneously.
16
S3: Third Step
17
Select Appropriate Denominators
S3: How Do We Get Coverage Denominators?
•
First, estimate the size of the target population.
•
Common target populations for health-facility-based indicators:
o
Total population, children< 5 years, infants, pregnancies,
women of reproductive age, live births at health facilities
•
Size of target populations is often estimated (such as
projections/modeled estimates from national population census).
•
Limitations of estimates:
o
Reliability declines with years since last census
o
Internal migration may make estimates of populations of
regions and districts unreliable
•
Some programs may use their own denominators.
18
How Do We Get Denominators?
•
Document
how the denominator value was obtained:
o
Methods and assumptions used to calculate the
denominator
o
Annual rate of growth if denominators are based on
projections of census figures
o
Present these along with rest of analysis
•
If service coverage >= 95% and data are of high quality, use
ANC1 or DTP1 to estimate the number of surviving infants.
o
Use of service statistics to estimate size of target population
can modify conclusions reached about which districts are
strong performers and which are weak performers.
19
Estimating Denominators
Estimating the number of surviving infants:
T
otal population: 5,500,000
Crude birth rate (CBR): 30/1,000
Infant mortality rate (IMR): 80/1,000
Number of surviving infants
Total population x crude birth rate x (1 - IMR)
= 5,500,000 x 30/1000 x (1 - 0.080)
= 5,500,000 x 0.030 x 0.920
= 151,800
20
Data Quality Check for Denominators
•
Pregnancies = births + pregnancy loss (2% to 10%);
•
Surviving infants = births – infant mortality
Number of pregnancies, deliveries, live births, infants
Tanzania district example, 2014
21
S4: Fourth Step
Reconcile Findings with Estimates from Other Data Sources
22
S4: Reconcile Findings with Estimates from Other
Data Sources
•
Compare data from parallel systems that
routinely report the same health events.
•
Compare estimates from routine health-facility
data with estimates from household surveys at
the national and regional levels.
•
Compare available data with statistics that
have been officially reported to WHO.
23
S4:
Estimates from Two Sources of Routine Health
Facility Data
Administrative estimates of 2014 DTP3 coverage, by district,
DPI/JRF versus HMIS/DHIS data
Note: Red marks are for districts with a negative dropout rate, according to EPI/JRF data.
.
S4: Compare Estimates from Household Surveys
DTP3 immunization coverage, Tanzania, 2009–2012
Sources: Routine EPI facility data, 2010 DHS, and 2011 Immunization Coverage Survey
25
Reconciling with Survey Findings Is Not Always Easy
2014 WHO-UNICEF report on trends in DTP3 coverage in Ethiopia
Administrative/official estimates (red stars and circles) versus surveys
(vertical red lines) versus WHO-UNICEF estimate (blue line)
26
Reconciling with Survey Findings Is Not Always Easy
2014 WHO-UNICEF report on trends in DTP3 coverage in India
Administrative/official estimates (red stars and circles) versus surveys
(vertical red lines) versus WHO-UNICEF estimate (blue line)
27
Small Group Exercise (Steps 1-4 )
•
Distribute Handouts 5.2.4a and b.
•
Form small groups of 4–5 participants.
•
Using Excel and the spreadsheet provided (Penta, 2014 data from
Tanzania, Handout 5.2.4B), calculate indicator values by region for:
o
Penta1 coverage rate
o
Penta3 coverage rate
o
Penta1-Penta3 dropout rate
•
For each region, specify whether access is good or poor.
•
For each region, specify whether utilization is good or poor.
•
Categorize the immunization problem in each region (if any).
•
Brainstorm the differences in coverage between regions.
•
Discuss what action managers can take if coverage and dropout
rates indicate problems.
28
P2: Small Group Exercise, Option B (DHIS 2)
•
Distribute Handout 5.2.6.
•
Form small groups of 4
–
5 participants.
•
Choose either ANC4 or DTP3 as your indicator.
•
Using DHIS 2, verify the definition of the indicator.
•
Create a pivot table with indicator values by district.
•
Graph trends in the completeness of reporting by year
for the past four years by:
o
Facility (hospital versus health center) and
o
Managing authority (public versus private)
•
Discuss how the trend in completeness would affect the
apparent trend in the indicator.
29
ROUTINE HEALTH INFORMATION SYSTEMS
A Curriculum on Basic Concepts and Practice
This presentation was produced with the support of the United States Agency for
International Development (USAID) under the terms of MEASURE Evaluation
cooperative agreement AID-OAA-L-14-00004. MEASURE Evaluation is implemented by
the Carolina Population Center, University of North Carolina at Chapel Hill in
partnership with ICF International; John Snow, Inc.; Management Sciences for Health;
Palladium; and Tulane University. The views expressed in this presentation do not
necessarily reflect the views of USAID or the United States government.
30
This overview provides a glimpse into Module 5 of the Routine Health Information Systems (RHIS) curriculum, focusing on data analysis. Participants will learn how to select indicators, review data quality, choose appropriate denominators, compare findings, and create information products. The session covers key steps such as selecting core indicators and understanding SMART indicator characteristics for effective data analysis.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
ROUTINE HEALTH INFORMATION SYSTEMS A Curriculum on Basic Concepts and Practice MODULE 5: RHIS Data Analysis SESSION 2: Overview of Steps 1-4 of Data Analysis The complete RHIS curriculum is available here: https://www.measureevaluation.org/our-work/ routine-health-information-systems/rhis-curriculum 1
Learning Objectives and Topics Covered Objectives By the end of this session, participants will be able to: Select appropriate indicators for data analysis Conduct a basic desk review of data quality and adjust data if necessary Select appropriate denominators Compare findings from routine data with findings from other data sources Analyze routine data to produce information products (tables, graphs, and maps) Topics Covered Selection of indicators for analysis Desk review of data completeness and internal consistency Selection of appropriate denominators Comparison of findings from routine data with findings from other data sources 2
5 Steps of Data Analysis 1. Select a limited set of core indicators 2. Review data quality 3. Select appropriate denominators 4. Reconcile findings with estimates from other data sources 5. Communicate key findings (to be discussed in Module 5, Session 3) 3
S1: First Step Select a Limited Set of Core Indicators 4
What Makes an Indicator SMART? (Characteristics of Good Indicators) Specific: Indicator is concrete, detailed, focused, and well-defined Measurable: Indicator tells how many or how much and can be measured with identified measurement sources Agreed upon: Stakeholders vested in a specific M&E question should agree that indicator is relevant Relevant: Indicator generates data that can answer the question of interest Time-bound: Indicator specifies time frame of what it is measuring 5
S1: Select a Limited Set of Core Indicators Core indicators should: Be based on M&E framework for the national health sector strategy Be programmatically relevant, and facilitate program management Reliably and comprehensively assess the performance of the health system (whether national or subnational levels) Be clearly defined Have numerators that are measurable with routine health data 6
S1: Core Indicator with 2 Data Sources: DTP3 Routine health information systems Numerator: Number of infants immunized with DTP3 by 12 months of age in a given year Denominator: Total number of surviving infants <12 months of age in same year Population-based survey Numerator: Number of children ages 12 23 months who received three doses of DTP3 vaccine by age 12 months Denominator: Total number of children ages 12 23 months surveyed 7
S1: Core Indicators Measured Reliably with One Data Source Antiretroviral therapy (ART) retention rate Numerator: Number of adults and children with HIV, alive and on antiretroviral therapy (ART) 12, 24, 36 months (etc.) after initiating treatment Denominator: Total number of patients initiating ART during a specific period Source: Routine health information systems Fully-immunized child Numerator: Number of children ages 12 23 months who received 3 doses of OPV, 3 doses of DTP, and 1 dose each of BCG and measles vaccine before age 12 months Denominator: Total number of children ages 12 23 months surveyed Source: Population- based survey 8
Some Indicators Are Not Clearly Defined or Cannot Be Reliably Measured with Routine Data Some indicators are not clearly defined: Percentage of health facilities with a skilled provider Percentage of health facilities with an adequate supply of drugs Some health indicators can be reliably measured with a household survey or health-facility survey but not with data reported routinely by health facilities: Proportion of population using an improved drinking water source Proportion of health facilities with treatment guidelines Infant mortality rate Some indicators are not clearly defined and usually cannot be reliably measured with data reported routinely by health facilities: % of health facilities with health equipment 9
Practicing the First Step Exercise: Part 1 Review the list of core health indicators (on the next slide (Handout 5.2.1) Identify which indicators can/cannot be reliably measured with routine data o Can the numerator be defined with routine data? o Do you need additional data sources to measure the indicator? 10
List of Health Indicators 1. Percentage of pregnant women attending antenatal clinics who are screened for syphilis 9. Percentage of HIV-positive infants born to HIV-positive women 10. Maternal mortality ratio 2. Neonatal mortality rate 11. Number of health facilities providing comprehensive emergency obstetric care functions per 500,000 population 3. Number of neonatal tetanus cases 4. Cost of one month s supply of contraception as a percentage of monthly wages 12. Exclusive breastfeeding rate 5. Percentage of infants born protected against neonatal tetanus in a specified period 13. TB treatment success rate 14. Percentage of health facilities with systems that support quality service delivery 6. Measles vaccine coverage rate 7. Percentage of registered new and relapse tuberculosis (TB) patients with documented HIV status 15. Percentage of districts with current trend analysis for selected priority diseases at a given time (e.g., month) 8. Percentage of children ages 12 59 months who were dewormed in the past six months 11
S1: Practicing the First Step Exercise, Part 2 Review the WHO s Global Reference List of Core Health Indicators (Handout 5.2.3) or the list of standard indicators in Handout 5.2.2. Select five indicators from the list. For each indicator: o Specify the numerator o Specify the denominator o What is the data source for the numerator? o What is the data source for the denominator? o How can the indicator be interpreted? 12
S2: Second Step Review Data Quality 13
Step 2: Review Data Quality (see also Module 4) Completeness and timeliness o Completeness of reports o Completeness of data o Timeliness of reports Internal consistency o Accuracy o Outliers o Trends o Consistency among indicators External consistency o Data triangulation o Comparison with data surveys o Consistency of population trends External comparisons (population denominators) 14
S2: Demonstration, Excel How to use Excel to: Create a chart Participants are invited to practice simultaneously. Here are two websites with 5-minute videos explaining how to create a chart using Excel: https://www.youtube.com/watch?v=BcsDnRClzCY http://excelcentral.com/excel2007/essential/lessons/05010- create-a-simple-chart-with-two-clicks.html 15
S2: Demonstration DHIS 2 How to use DHIS 2 to: Verify the definition of an indicator o Numerator and denominator Create a pivot table Create a chart Participants are invited to practice simultaneously. 16
S3: Third Step Select Appropriate Denominators 17
S3: How Do We Get Coverage Denominators? First, estimate the size of the target population. Common target populations for health-facility-based indicators: o Total population, children< 5 years, infants, pregnancies, women of reproductive age, live births at health facilities Size of target populations is often estimated (such as projections/modeled estimates from national population census). Limitations of estimates: Reliability declines with years since last census o Internal migration may make estimates of populations of regions and districts unreliable o Some programs may use their own denominators. 18
How Do We Get Denominators? Document how the denominator value was obtained: o Methods and assumptions used to calculate the denominator o Annual rate of growth if denominators are based on projections of census figures o Present these along with rest of analysis If service coverage >= 95% and data are of high quality, use ANC1 or DTP1 to estimate the number of surviving infants. o Use of service statistics to estimate size of target population can modify conclusions reached about which districts are strong performers and which are weak performers. 19
Estimating Denominators Estimating the number of surviving infants: Total population: 5,500,000 Crude birth rate (CBR): 30/1,000 Infant mortality rate (IMR): 80/1,000 Number of surviving infants Total population x crude birth rate x (1 - IMR) = 5,500,000 x 30/1000 x (1 - 0.080) = 5,500,000 x 0.030 x 0.920 = 151,800 20
Data Quality Check for Denominators Pregnancies = births + pregnancy loss (2% to 10%); Surviving infants = births infant mortality Number of pregnancies, deliveries, live births, infants Tanzania district example, 2014 14000 12000 12415 11512 11400 11286 11058 10000 8000 6000 4000 2000 0 Pregnancies, early Pregnancies, late Deliveries Live births Infants 21
S4: Fourth Step Reconcile Findings with Estimates from Other Data Sources 22
S4: Reconcile Findings with Estimates from Other Data Sources Compare data from parallel systems that routinely report the same health events. Compare estimates from routine health-facility data with estimates from household surveys at the national and regional levels. Compare available data with statistics that have been officially reported to WHO. 23
S4: Estimates from Two Sources of Routine Health Facility Data Administrative estimates of 2014 DTP3 coverage, by district, DPI/JRF versus HMIS/DHIS data 140% 120% DTP3 -- HMIS (DHIS) 100% Linear (Equal) Linear (Plus 25%) 80% Linear (Minus 25%) 60% . 40% 40% 60% 80% DTP3 - EPI (JRF) 100% 120% 140% Note: Red marks are for districts with a negative dropout rate, according to EPI/JRF data.
S4: Compare Estimates from Household Surveys DTP3 immunization coverage, Tanzania, 2009 2012 Sources: Routine EPI facility data, 2010 DHS, and 2011 Immunization Coverage Survey 25
Reconciling with Survey Findings Is Not Always Easy 2014 WHO-UNICEF report on trends in DTP3 coverage in Ethiopia Administrative/official estimates (red stars and circles) versus surveys (vertical red lines) versus WHO-UNICEF estimate (blue line) 140 WHO-UNICEF estimate i Offic a l g over n m e nt e s tim a t e * Administrative coverage 120 Survey 12-22 months of age, card or history 100 * * * * * * 80 * * * * 60 * * 40 20 0 26 2002 2004 2006 2010 2012 2008
Reconciling with Survey Findings Is Not Always Easy 2014 WHO-UNICEF report on trends in DTP3 coverage in India Administrative/official estimates (red stars and circles) versus surveys (vertical red lines) versus WHO-UNICEF estimate (blue line) 27
Small Group Exercise (Steps 1-4 ) Distribute Handouts 5.2.4a and b. Form small groups of 4 5 participants. Using Excel and the spreadsheet provided (Penta, 2014 data from Tanzania, Handout 5.2.4B), calculate indicator values by region for: Penta1 coverage rate o Penta3 coverage rate o Penta1-Penta3 dropout rate o For each region, specify whether access is good or poor. For each region, specify whether utilization is good or poor. Categorize the immunization problem in each region (if any). Brainstorm the differences in coverage between regions. Discuss what action managers can take if coverage and dropout rates indicate problems. 28
P2: Small Group Exercise, Option B (DHIS 2) Distribute Handout 5.2.6. Form small groups of 4 5 participants. Choose either ANC4 or DTP3 as your indicator. Using DHIS 2, verify the definition of the indicator. Create a pivot table with indicator values by district. Graph trends in the completeness of reporting by year for the past four years by: Facility (hospital versus health center) and Managing authority (public versus private) Discuss how the trend in completeness would affect the apparent trend in the indicator. o o 29
ROUTINE HEALTH INFORMATION SYSTEMS A Curriculum on Basic Concepts and Practice This presentation was produced with the support of the United States Agency for International Development (USAID) under the terms of MEASURE Evaluation cooperative agreement AID-OAA-L-14-00004. MEASURE Evaluation is implemented by the Carolina Population Center, University of North Carolina at Chapel Hill in partnership with ICF International; John Snow, Inc.; Management Sciences for Health; Palladium; and Tulane University. The views expressed in this presentation do not necessarily reflect the views of USAID or the United States government. 30
