Managing Gender-Based Violence Programmes in Emergencies: Monitoring & Evaluation Essentials
Monitoring & Evaluation
Managing Gender-Based Violence Programmes in Emergencies
1
Learning Objectives
1.
Understand the importance of quality M&E to programme design
and management
2.
Identify key results in an M&E logical framework
3.
Identify some of the common risks and challenges in GBV
programme monitoring, collection and use of data
4.
Access additional resources and trainings in programme M&E
Why monitor & evaluate programmes?
3
Purposes of M&E
1: Integral part of project
management to
achieve results
•
Correct mistakes
•
Good planning & design is
NOT enough
•
Necessary for effective
implementation
•
Facilitates results-based
reporting
•
Facilitates evidence-based
decision-making and
planning
2: Enhances
accountability
to
stakeholders
3: Contributes to
learning & KM
4
When do we
monitor & evaluate
programmes?
Project initiation
and design
End of project
transition
PMDPro 1 Guide
Project
identification
M&E activities
M&E across the Programme Cycle
What do our GBV Programmes want to
Achieve?
Real and meaningful changes in lives of women
and girls
7
These changes should be:
•
measurable
•
relevant to needs
•
part of a process of
continuing
improvement
Results-Based Programming
Logic & Definitions
Results-Based Programming:
Uses evidence to answer key questions
What needs to
change?
What is nature and
magnitude of
underlying problem?
What factors contribute
to the problem?
What exactly needs to
change?
What is best way to
achieve results /
change?
What program
strategies have
been effective in
achieving results
(change) in the past
or in other settings?
What else is
needed?
What other
things need to
happen
—aside from
your project—
to achieve
results
(change)?
10
Did the theory
work?
Has my program
achieved the
intended
results?
Why/why not?
Theory of change
Testing
your theory
of change
Definitions: Key
Types of Results
11
Chain of Results
Results of
project
12
S
.
M
.
A
.
R
.
T
.
O
b
j
e
c
t
i
v
e
s
13
Why Logic Models?
To show “If-Then” logic, cause and effect
A
c
t
i
v
i
t
i
e
s
O
u
t
p
u
t
s
O
u
t
c
o
m
e
O
b
j
e
c
t
i
v
e
then
If
If
If
if
if
if
then
then
I
n
p
u
t
s
if
then
14
Results Chain/ Logical Framework
Group Exercise!
Results Chain
Are you missing any logical steps?
17
Activity:
Teach people to spin
their own cloth
Project Goal:
Fall of the British Empire
Sample Results Chain
Objective:
Survivors of rape receive
appropriate, safe, and confidential
healthcare services.
Outcome:
Health service capacity
to respond appropriately to sexual
violence improved.
Output:
Providers who
participate in training have
improved knowledge and skills.
Output:
Confidentiality &
privacy increased (private rooms
created).
Output:
Referral mechanism
has been strengthened.
Activity:
Rewrite policies,
protocols and referral
mechanism.
Activity:
Build walls to replace
curtains.
Activity:
Organize CMR
trainings for select healthcare
providers.
18
Note: an actual
results chain needs
more specificity
Indications of Change
W
h
a
t
i
s
a
n
i
n
d
i
c
a
t
o
r
?
“A
qualitative or quantitative
factor or variable that provides
a
simple and reliable
means to
measure achievement, to
reflect the changes
connected
to an intervention, or to help
assess the performance of a
development actor.”
(OECD/DAC Glossary of Key Terms in
Evaluation and Results-Based Management)
20
Common ways of measuring (metrics)
Counts
•
Number (#) of survivors who called a hotline in the past month
•
# of women participating in skills-building classes
Calculations:
percentages, rates, ratios
•
Percentage (%) of health units with at least one service provider trained to care for
and refer survivors
•
% of people surveyed who can identify GBV service points
Thresholds:
presence/absence
•
Referral pathway developed
•
New law criminalizing marital rape adopted within five years (or not)
21
Quick Exercise!
Output indicator:
SMART measure
of direct output
Outcome indicator
: Higher-level
change, SMART, linked to outcome
C
h
a
r
a
c
t
e
r
i
s
t
i
c
s
o
f
a
g
o
o
d
i
n
d
i
c
a
t
o
r
•
Includes a unit of measurement
:
Does it specify a unit
of measure?
(e.g. Quantitative: #, %, rate, score, etc.;
Qualitative: perspectives, types etc.)
•
Non-directional
:
Does it avoid direction of change
(“
increase”
)?
•
Relevant
:
Does it measure the most important results
you want to achieve among intended beneficiary group
(institution, police, etc.)?
•
Specific
:
Are all terms are clear and can they be clearly
defined?
•
Measurable
:
Do you have resources/methods to
measure it?
•
Valid
:
Does it really measure what you think it does?
•
Reliable
:
Can it measure the same thing consistently
over time?
•
Sensible timeframes and frequency
: Can it be measured
at time intervals that allow you to measure change
(e.g.
before/after project)
?
Sources of
GBV
Indicators
•
GBV Minimum Standards
•
GBV Guidelines
•
Protection Cluster
•
IRC GBV Responders’ website
•
USAID Toolkit for Monitoring & Evaluating GBV
Interventions
•
GWU Global Women’s Institute resources
•
UNFPA ASRH Toolkit for Humanitarian Settings
Recommended that GBV implementing agencies
develop banks of adaptable indicators for
programmes
Back to the purpose…
What changes are most important and how do we measure these?
26
UNIFEM/UNTF A Life Free From Violence is Our Right
Your programme may aim to
influence:
•
Quality of care for survivors of GBV
•
Opportunities available for women
and girls
•
Community norms and practices
•
Laws and policies
Your programme would also like to
decrease the rate of GBV– but this is
NOT what you will measure.
Prevalence Of GBV: Misunderstood and not a
useful indicator of programme success
•
The programme cannot change many conditions that
contribute to rates of GBV.
•
Reporting may increase with awareness and quality services
in place.
•
Prevalence is not measured by service/police data but by
population-based studies.
•
Under-reporting is a universal phenomenon with GBV, even
in population-based studies.
•
Remember that collecting prevalence data is rarely
recommended and not required to take action!
27
INCREASED REPORTS OF GBV DO NOT INDICATE THAT
A GBV PROGRAMME HAS FAILED
Prevalence data vs. Incidence data
All GBV incidents
Reported cases
GBV service
Health
Police
Prevalence data
Prevalence data
Incidence data
Incidence data
Examples: Changes we would like to see
(and can start to measure)
Survivors of violence
•
GBV-focused case management and
emotional support services are available
•
GBV referral pathways have been
established, linking critical service
providers
•
Resources are available to enable
women and girls to access systems of
justice
•
Health care facilities are equipped to
provide clinical care for survivors of
rape, including the availability of trained,
female providers who are linked to
referral pathways
General population
•
Community members recognize gender
imbalances
•
Community leaders do not condone
domestic violence
•
Sexual harassment is not tolerated in
public spaces
•
Girls are empowered to go to school
29
Methods & Tools
Measuring big and little changes….
What changes can we
measure
•
Attitudes
(personal opinion/belief)
•
Knowledge and awareness
•
Behaviors, practices
(usually self-reported)
•
Norms
(perceptions of beliefs/practices)
•
Satisfaction
•
Risks
•
Service availability
•
Referral networks
•
Participation
•
Help-seeking behavior
Challenges &
Limitations
•
“GBV is a sensitive topic here”
•
Access & security
•
Ethical and safety concerns
•
Changes may be slow (and programme
timeline may be short)
•
Limited samples
•
Baseline not established
•
Social acceptability bias
•
Bias if programme staff are involved with
data collection
32
Examples
of Data
Sources/
Methods
33
Tips for Data
Collection
KISS: Keep it simple!
Monitoring should not
eclipse implementation.
Rely on existing systems and tools
Adapt tools to project needs / settings
Review safety and ethics
Limit collection of personally identifiable
information (PII)
Train teams and pre-test tools
34
Safe Storage
&
Information
Sharing
To whom are we most accountable?
•
Programme managers are responsible for ensuring
all measures are taken to keep information secure
•
Programme managers can influence other service
providers to better secure information (health
facilities, police)
•
Programme managers must comply with
requirements to safeguard survivor data and
appropriately respond to requests for information
•
Remember that all service statistics– not only case
management records– can be sensitive. This
includes participation in discussions, arts groups,
etc.
Data Sharing:
Survivor Data
1.
Identifiable, individual-level data
Only shared with consent of survivor,
and
In the context of a service referral
2.
Non-identifiable, aggregate-level data
•
Sharing and applying even anonymized survivor
data requires consent of survivors
•
If multiple service providers are using the same
information management system (GBVIMS),
data can be compiled
•
Information Sharing Protocols should be
established between agencies sharing aggregate
data
Application of Findings
M&E should contribute to better/increased interventions
Information from Quality M&E can be used
to…
•
Adjust activities/services as needed
•
Adjust programme objectives/approaches as needed (consult with
donor)
•
Share findings with other programmes (trainings, dissemination)
•
Share reports to increase resource mobilization
•
Contribute to the evidence base; articles for publication
38
Let’s review our objectives…
1.
Understand the importance of
quality M&E to programme
design and management
2.
Identify key results in an M&E
logical framework
3.
Identify some of the common
risks and challenges in GBV
programme monitoring,
collection and use of data
4.
Access additional resources and
trainings in programme M&E
… and our
Purpose.
1.1
Learn the significance of quality monitoring and evaluation in program design, identify key M&E results, understand risks in GBV program monitoring, and access additional resources. Explore the purposes, timing, and activities of M&E in GBV programs to achieve real changes in the lives of women and girls through results-based programming.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Monitoring & Evaluation Managing Gender-Based Violence Programmes in Emergencies 1
Learning Objectives 1. Understand the importance of quality M&E to programme design and management 2. Identify key results in an M&E logical framework 3. Identify some of the common risks and challenges in GBV programme monitoring, collection and use of data 4. Access additional resources and trainings in programme M&E
Purposes of M&E 1: Integral part of project management to achieve results Correct mistakes Good planning & design is NOT enough Necessary for effective implementation Facilitates results-based reporting Facilitates evidence-based decision-making and planning 2: Enhances accountability to stakeholders 3: Contributes to learning & KM Planning System & Capacity Monitoring Learning & KM Reporting Evaluation 4
When do we monitor & evaluate programmes?
M&E activities M&E across the Programme Cycle Monitoring and Evaluation Project planning Project initiation and design End of project transition Project identification Project implementation PMDPro 1 Guide
What do our GBV Programmes want to Achieve? Real and meaningful changes in lives of women and girls These changes should be: measurable relevant to needs part of a process of continuing improvement 7
Results-Based Programming Not just activities But results! Train staff 30 service providers have improved knowledge and attitudes Provide GBV case management services 90% of surveyed survivors report improved psychosocial wellbeing at case closure
Results-Based Programming: Uses evidence to answer key questions Theory of change Testing your theory of change What is best way to achieve results / change? What else is needed? What needs to change? Did the theory work? What other things need to happen aside from your project to achieve results (change)? What is nature and magnitude of underlying problem? What program strategies have been effective in achieving results (change) in the past or in other settings? Has my program achieved the intended results? Why/why not? What factors contribute to the problem? What exactly needs to change? 10
Definitions: Key Types of Results Term Definition Goal Broad statement of desired changes. Usually looks toward longer-term. Objective More specific goal that your programme can achieve. Outcome Intermediate result of reaching your objective (e.g. changes in health, knowledge, policies). Output Immediate, direct result of project activities. 11
Chain of Results GOAL OBJECTIVES Results of project OUTCOMES OUTPUTS ACTIVITIES 12
S.M.A.R.T. S.M.A.R.T. Objectives Specific: what will change among whom Measurable: result can be measured Achievable: idealistic but realistic Results-oriented: Describes what you want to achieve not what activities you want to complete. Time-bound: achievable by project end Relevant: contributes to chain of results 13
Why Logic Models? To show If-Then logic, cause and effect Objective then then Outcome if If then Outputs if If then Activities if Inputs If if 14
Results Chain/ Logical Framework Goal Objective 1 Objective 2 Outcome 1.1 Outcome 1.2 Outcome 2.1 Outcome 2.2 Output 1.1.1 Output 1.1.2 Output 1.2.1 Output 1.2.2 Output 2.1.1 Output 2.1.2 Output 2.2.1 Output 2.2.2 Activities Activities Activities Activities Acitvities Activities Activities Activities
Group Exercise! Goal: Women and girls are safe in their communities and can freely access a range of services. Objective 1: Provide GBV case management and PSS support services for survivors of GBV and at-risk women and girls. Objective 2: Support women s and girls empowerment through WGSS. Objective 2: Strengthen capacity for health response to GBV. Objective 3: Mitigate risks of GBV within the community.
Results Chain Are you missing any logical steps? Project Goal: Fall of the British Empire Activity: Teach people to spin their own cloth 17
Sample Results Chain Objective: Survivors of rape receive appropriate, safe, and confidential healthcare services. Outcome: Health service capacity to respond appropriately to sexual violence improved. Output: Referral mechanism has been strengthened. Output: Confidentiality & privacy increased (private rooms created). Output: Providers who participate in training have improved knowledge and skills. Activity: Rewrite policies, protocols and referral mechanism. Activity: Build walls to replace curtains. Activity: Organize CMR trainings for select healthcare providers. 18
What is an indicator? What is an indicator? A qualitative or quantitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor. (OECD/DAC Glossary of Key Terms in Evaluation and Results-Based Management) 20
Common ways of measuring (metrics) Counts Number (#) of survivors who called a hotline in the past month # of women participating in skills-building classes Calculations: percentages, rates, ratios Percentage (%) of health units with at least one service provider trained to care for and refer survivors % of people surveyed who can identify GBV service points Thresholds: presence/absence Referral pathway developed New law criminalizing marital rape adopted within five years (or not) 21
Quick Exercise! Output indicator: SMART measure of direct output Outcome indicator: Higher-level change, SMART, linked to outcome
Characteristics Characteristics of a good of a good indicator indicator Includes a unit of measurement: Does it specify a unit of measure? (e.g. Quantitative: #, %, rate, score, etc.; Qualitative: perspectives, types etc.) Non-directional: Does it avoid direction of change ( increase )? Relevant: Does it measure the most important results you want to achieve among intended beneficiary group (institution, police, etc.)? Specific: Are all terms are clear and can they be clearly defined? Measurable: Do you have resources/methods to measure it? Valid: Does it really measure what you think it does? Reliable: Can it measure the same thing consistently over time? Sensible timeframes and frequency: Can it be measured at time intervals that allow you to measure change (e.g. before/after project)?
GBV Minimum Standards GBV Guidelines Protection Cluster IRC GBV Responders website USAID Toolkit for Monitoring & Evaluating GBV Interventions GWU Global Women s Institute resources UNFPA ASRH Toolkit for Humanitarian Settings Sources of GBV Indicators Recommended that GBV implementing agencies develop banks of adaptable indicators for programmes
Back to the purpose What changes are most important and how do we measure these?
Your programme may aim to influence: Quality of care for survivors of GBV Opportunities available for women and girls Community norms and practices Laws and policies Your programme would also like to decrease the rate of GBV but this is NOT what you will measure. 26 UNIFEM/UNTF A Life Free From Violence is Our Right
Prevalence Of GBV: Misunderstood and not a useful indicator of programme success The programme cannot change many conditions that contribute to rates of GBV. Reporting may increase with awareness and quality services in place. Prevalence is not measured by service/police data but by population-based studies. Under-reporting is a universal phenomenon with GBV, even in population-based studies. Remember that collecting prevalence data is rarely recommended and not required to take action! INCREASED REPORTS OF GBV DO NOT INDICATE THAT A GBV PROGRAMME HAS FAILED 27
Prevalence data vs. Incidence data All GBV incidents Prevalence data Reported cases GBV service Health Police Incidence data
Examples: Changes we would like to see (and can start to measure) Survivors of violence GBV-focused case management and emotional support services are available GBV referral pathways have been established, linking critical service providers Resources are available to enable women and girls to access systems of justice Health care facilities are equipped to provide clinical care for survivors of rape, including the availability of trained, female providers who are linked to referral pathways General population Community members recognize gender imbalances Community leaders do not condone domestic violence Sexual harassment is not tolerated in public spaces Girls are empowered to go to school 29
Methods & Tools Measuring big and little changes .
What changes can we measure Attitudes (personal opinion/belief) Knowledge and awareness Behaviors, practices (usually self-reported) Norms (perceptions of beliefs/practices) Satisfaction Risks Service availability Referral networks Participation Help-seeking behavior
Challenges & Limitations GBV is a sensitive topic here Access & security Ethical and safety concerns Changes may be slow (and programme timeline may be short) Limited samples Baseline not established Social acceptability bias Bias if programme staff are involved with data collection 32
Service data (GBVIMS; client records) Participant tracking (WGSS database; tracking instruments) Surveys Examples of Data Sources/ Methods Interviews Pre and post tests among participants / trainees Client satisfaction surveys Institutional checklists Group discussions Policy analysis Participant observations (mapping, diagrams, etc.) 33
Tips for Data Collection KISS: Keep it simple! Monitoring should not eclipse implementation. Rely on existing systems and tools Adapt tools to project needs / settings Review safety and ethics Limit collection of personally identifiable information (PII) Train teams and pre-test tools 34
To whom are we most accountable? Programme managers are responsible for ensuring all measures are taken to keep information secure Programme managers can influence other service providers to better secure information (health facilities, police) Programme managers must comply with requirements to safeguard survivor data and appropriately respond to requests for information Remember that all service statistics not only case management records can be sensitive. This includes participation in discussions, arts groups, etc. Safe Storage & Information Sharing
1. Identifiable, individual-level data Only shared with consent of survivor, and In the context of a service referral 2. Non-identifiable, aggregate-level data Sharing and applying even anonymized survivor data requires consent of survivors If multiple service providers are using the same information management system (GBVIMS), data can be compiled Information Sharing Protocols should be established between agencies sharing aggregate data Data Sharing: Survivor Data
Application of Findings M&E should contribute to better/increased interventions
Information from Quality M&E can be used to Adjust activities/services as needed Adjust programme objectives/approaches as needed (consult with donor) Share findings with other programmes (trainings, dissemination) Share reports to increase resource mobilization Contribute to the evidence base; articles for publication 38
Lets review our objectives 1. Understand the importance of quality M&E to programme design and management 2. Identify key results in an M&E logical framework 3. Identify some of the common risks and challenges in GBV programme monitoring, collection and use of data 4. Access additional resources and trainings in programme M&E
and our Purpose.
