Overview of Monitoring and Evaluation in the GEF

undefined

Carlo Carugi, Senior Evaluation Officer

GEF Independent Evaluation Office

March, 2015

Evaluation in the GEF and

Training Module on

Terminal Evaluations

GEF Expanded Constituency

Workshop

Overview

•

Evaluation in the GEF:

Monitoring and Evaluation in the GEF

GEF Independent Evaluation Office (IEO)

GEF M&E Policy 2010

•

Training Module on Terminal Evaluations

M&E in the GEF

Two overarching objectives

•

Promote

accountability

 for the achievement of GEF objectives

through the assessment of

results, effectiveness, processes

, and

performance

 of the partners involved in GEF activities.

•

Promote

learning

feedback

, and

knowledge sharing

on results

and lessons learned among the GEF and its partners as a basis for

decision making on policies, strategies, program management,

programs, and projects; and to improve

knowledge

and

performance.

Separate reporting lines for Monitoring (through

Secretariat) and Evaluation (through IEO)

GEF Council

Agency evaluation

units

GEF Independent

Evaluation Office

GEF Secretariat

Agency GEF

coordination units

GEF projects and

programs

Annual evaluation reports

Overall Performance Study

(to Assembly)

Annual Work Program

and Budget

Annual Monitoring Report

Evaluation Management Response

Programming documents and indicators

Results Based Management

Project and Program Implementation Reports

Agency Portfolio Reports

Project documents with M&E plans

Project and  Program

evaluations

Corporate evaluations

Project and Program

Independent evaluations

Project and Program Implementation Reports

Project and Program monitoring documentation

Terminal evaluations

GEF Independent Evaluation Office

Mission:

Enhance global environmental benefits through excellence, independence, and

partnership in monitoring and evaluation.

Functions

Independent GEF Evaluation

Normative function

Oversight function

Knowledge sharing and dissemination

Brief history

1996 — Initially established as an M&E unit within the GEF Secretariat

2003 — The M&E unit was made independent of the GEF Secretariat

2005 — The unit was renamed as GEF Evaluation Office

2013 — The office was renamed as GEF Independent Evaluation Office

 GEF IEO Stakeholders

Types of Evaluations

Different TORs, scopes, frequency, audiences , methodologies.

GEF IEO Dissemination and Knowledge

Management

GEF IEO GEF-6 Work Program

•

Implementation of GEF M&E Policy

•

Evaluation Program adjusts to evolving GEF priorities

•

GEF IEO has started consultation to define the GEF 6 evaluation

program

•

Reporting to the GEF Council

•

articipation in

the GEF C

ountry

upport

rogram

•

Overall Performance Studies

—

 replenishment process

(every 4 years)

•

Active participation evaluation communities (UN/MDBs/other)

he GEF M&E Policy

•

Defines the concepts, role, and use of monitoring and evaluation

within the GEF

•

Defines the institutional framework and responsibilities.

•

Indicates the GEF minimum M&E requirements covering:

–

project design,

–

application of M&E at the project level

–

project evaluation, and

–

engagement of Operational Focal Points in M&E.

•

Current M&E Policy: Approved by GEF Council in November 2010

M&E: Minimum Requirement 1

Design of M&E Plans

Concrete and fully budgeted M&E plan by CEO endorsement for FSP and CEO

approval for MSP. Project logical frameworks should align with GEF focal area

results frameworks. M&E Plan should include:

•

SMART indicators

–

Specific

–

measurable,

–

achievable and attributable,

–

relevant and realistic,  (OPS 5 found this is critical)

–

time-bound,

–

trackable and targeted

•

Baseline data for M&E by CEO endorsement

•

Mid Term Reviews (where required or foreseen) and Terminal Evaluations

included in plan

•

Organizational set up and budget for M&E

M&E: Minimum Requirement 2

Implementation of M&E Plans

Project/program monitoring and supervision will include execution

of the M&E plan:

•

Use of SMART indicators for process and implementation

•

Use of SMART indicators for results

•

Baseline for the project is fully established and data are compiled

to review progress

•

Organizational set up for M&E is operational and its budget is

spent as planned

M&E: Minimum Requirement 3

Project/Program Evaluations:

•

All full sized projects and programs will be evaluated at the

end of implementation

•

Evaluations should:

–

Be independent of project management or reviewed by GEF Agency evaluation

unit

–

Apply evaluation norms and standards of the GEF Agency

–

Assess, as a minimum, outputs and outcomes, likelihood of sustainability,

compliance with Minimum Requirements 1 & 2

–

Contain basic project data and lessons on the evaluation itself (including TORs)

M&E: Minimum Requirement 4

Engagement of Operational Focal Points

•

M&E plans should include how OFPs will be engaged

•

OFPs to be informed on M&E activities, including Mid Term

Reviews and Terminal Evaluations, receiving drafts for comments

and final reports

•

OFPs invited to contribute to the management response (where

applicable)

•

GEF Agencies keep track of the application of this requirement in

their GEF financed projects and programs

•

OFPs can if they wish under take monitoring or evaluation of

projects of country portfolio.

undefined

Training Module on

Terminal Evaluations

Introduction

•

Purpose of this module is to:

–

Discuss importance and utility of terminal evaluations

–

Characteristics of a good terminal evaluation

–

Exercise on pre-requisites for preparing a good terminal

evaluation

•

What needs to be done at different stages of preparation and

delivery of a terminal evaluation

–

When a terminal evaluation is planned

–

When a terminal evaluation  is conducted

–

When a draft report is prepared

•

During project preparation

•

During project implementation

Importance and Utility of Terminal

Evaluations

•

Source of information on project

–

Results: Outputs, outcomes and progress to impact

–

Implementation, execution, and project cycle related information

–

Project finances including co-financing

–

Recommendations and Lessons for the future

•

GEF M&E Policy (2010): Minimum Requirement 3

–

Terminal evaluations mandatory since 1995

–

For full size projects required, encouraged for MSPs

Importance and Utility of Terminal

Evaluations contd.

•

Reporting at the project portfolio level (APR, AMR)

•

Input to other evaluations

•

STAR’s performance index

•

About 1000 terminal evaluations completed so far.

•

Terminal evaluation may be accessed at:

–

GEF website:

http://www.thegef.org/gef/gef_projects_funding

–

Through PMIS

Characteristics of a good terminal evaluation

•

GEF IEO criteria for terminal evaluations quality:

–

Outcomes

–

Consistency and comprehensiveness

–

Sustainability

–

Lessons and recommendations

–

Project finances

–

M&E

•

Other characteristics of good terminal evaluations:

–

Transparency and timeliness

–

Candor

–

Balance

–

Utility

•

Question – is good terminal evaluation and good project performance the

same?

•

Terminal Evaluation Guidelines are available at:

http://www.thegef.org/gef/Guidelines%20Terminal%20Evaluations

Examples of good terminal evaluations

•

Links to examples of good terminal evaluations

–

GEF ID #394:

http://www.thegef.org/gef/project_detail?projID=394

–

GEF ID #1599:

http://www.thegef.org/gef/project_detail?projID=1599

–

GEF ID #1188:

http://www.thegef.org/gef/project_detail?projID=1188

–

GEF ID #1348:

http://www.thegef.org/gef/project_detail?projID=1348

•

You will be asked to think about what needs to be done to facilitate

preparation of a good terminal evaluation:

–

At different stages of terminal evaluation preparation.

–

At project preparation and implementation

•

Each group will discuss the requirements at each of these steps and will

record their joint responses as bullets in the response sheet provided to

them

•

After all stages have been discussed by the groups, one of the groups

will present its response for a stage, others may add if their group had

an additional bullets not yet covered. Each stage will be presented by a

different group.

•

We will end with a summary that pulls the discussion together

Exercise: Pre-requisites for preparing a good

terminal evaluation

First Discussion:

What needs to be done at different stages of the

terminal evaluation process?

Consider a situation where a project is about to be

complete within one year. What needs to be done

at different stages of terminal evaluation

preparation to ensure that it leads to a good

quality evaluation:

A. Commissioning of the terminal evaluation

B. Conduct of the terminal evaluation

C. Finalization of the terminal evaluation

Second Discussion:

What needs to be done before the terminal

evaluation process starts?

A. What needs to be done during the project

preparation stage?

–

Development of M&E plan

–

Tracking tools

B. What needs to be done during the project

implementation?

–

Implementation of M&E plan

–

Gathering of data on tracking tools

Short Presentations by stages

•

Presentation on group discussion

•

Summary

undefined

Slide Note

Embed Share

Download

The Evaluation in the GEF and Training Module focuses on promoting accountability and learning within the Global Environment Facility (GEF) through monitoring and evaluation activities. The GEF Independent Evaluation Office plays a crucial role in assessing results, effectiveness, and performance of GEF activities, aiming to enhance global environmental benefits. With separate reporting lines for monitoring and evaluation, the GEF strives to improve knowledge sharing and decision-making processes based on evaluation findings. The history and functions of the GEF Independent Evaluation Office are highlighted, emphasizing excellence, independence, and partnership in monitoring and evaluation efforts.

katel Follow

Uploaded on Sep 10, 2024 | 1 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.

Download Presentation

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.

E N D

Presentation Transcript

Evaluation in the GEF and Training Module on Terminal Evaluations GEF Expanded Constituency Workshop Carlo Carugi, Senior Evaluation Officer GEF Independent Evaluation Office March, 2015

Overview Evaluation in the GEF: Monitoring and Evaluation in the GEF GEF Independent Evaluation Office (IEO) GEF M&E Policy 2010 Training Module on Terminal Evaluations Page 2

M&E in the GEF Two overarching objectives: Promote accountability for the achievement of GEF objectives through the assessment of results, effectiveness, processes, and performance of the partners involved in GEF activities. Promote learning, feedback, and knowledge sharing on results and lessons learned among the GEF and its partners as a basis for decision making on policies, strategies, program management, programs, and projects; and to improve knowledge and performance. Page 3

Separate reporting lines for Monitoring (through Secretariat) and Evaluation (through IEO) GEF Council Annual evaluation reports Overall Performance Study (to Assembly) Annual Work Program and Budget Annual Monitoring Report Evaluation Management Response Programming documents and indicators Results Based Management Agency evaluation units GEF Independent Evaluation Office GEF Secretariat Project and Program Implementation Reports Agency Portfolio Reports Project documents with M&E plans Corporate evaluations Project and Program Independent evaluations Project and Program evaluations Agency GEF coordination units Project and Program Implementation Reports Project and Program monitoring documentation Terminal evaluations GEF projects and programs Page 4

GEF Independent Evaluation Office Mission: Enhance global environmental benefits through excellence, independence, and partnership in monitoring and evaluation. Functions: Independent GEF Evaluation Normative function Oversight function Knowledge sharing and dissemination Brief history: 1996 Initially established as an M&E unit within the GEF Secretariat 2003 The M&E unit was made independent of the GEF Secretariat 2005 The unit was renamed as GEF Evaluation Office 2013 The office was renamed as GEF Independent Evaluation Office Page 5

GEF IEO Stakeholders Page 6

Types of Evaluations Country Level Evaluations Project Evaluations Program Evaluations Process and performance evaluations Cross-cutting and thematic evaluations Impact Evaluations Overall Performance Studies (OPS) Ad-hoc Reviews Special Studies Different TORs, scopes, frequency, audiences , methodologies. Page 7

GEF IEO Dissemination and Knowledge Management Page 8

GEF IEO GEF-6 Work Program Implementation of GEF M&E Policy Evaluation Program adjusts to evolving GEF priorities GEF IEO has started consultation to define the GEF 6 evaluation program Reporting to the GEF Council Participation in the GEF Country Support Program Overall Performance Studies replenishment process (every 4 years) Active participation evaluation communities (UN/MDBs/other) Page 9

The GEF M&E Policy Defines the concepts, role, and use of monitoring and evaluation within the GEF Defines the institutional framework and responsibilities. Indicates the GEF minimum M&E requirements covering: project design, application of M&E at the project level project evaluation, and engagement of Operational Focal Points in M&E. Current M&E Policy: Approved by GEF Council in November 2010 Page 10

M&E: Minimum Requirement 1 Design of M&E Plans Concrete and fully budgeted M&E plan by CEO endorsement for FSP and CEO approval for MSP. Project logical frameworks should align with GEF focal area results frameworks. M&E Plan should include: SMART indicators Specific measurable, achievable and attributable, relevant and realistic, (OPS 5 found this is critical) time-bound, trackable and targeted Baseline data for M&E by CEO endorsement Mid Term Reviews (where required or foreseen) and Terminal Evaluations included in plan Organizational set up and budget for M&E Page 11

M&E: Minimum Requirement 2 Implementation of M&E Plans Project/program monitoring and supervision will include execution of the M&E plan: Use of SMART indicators for process and implementation Use of SMART indicators for results Baseline for the project is fully established and data are compiled to review progress Organizational set up for M&E is operational and its budget is spent as planned Page 12

M&E: Minimum Requirement 3 Project/Program Evaluations: All full sized projects and programs will be evaluated at the end of implementation Evaluations should: Be independent of project management or reviewed by GEF Agency evaluation unit Apply evaluation norms and standards of the GEF Agency Assess, as a minimum, outputs and outcomes, likelihood of sustainability, compliance with Minimum Requirements 1 & 2 Contain basic project data and lessons on the evaluation itself (including TORs) Page 13

M&E: Minimum Requirement 4 Engagement of Operational Focal Points M&E plans should include how OFPs will be engaged OFPs to be informed on M&E activities, including Mid Term Reviews and Terminal Evaluations, receiving drafts for comments and final reports OFPs invited to contribute to the management response (where applicable) GEF Agencies keep track of the application of this requirement in their GEF financed projects and programs OFPs can if they wish under take monitoring or evaluation of projects of country portfolio. Page 14

Training Module on Terminal Evaluations

Introduction Purpose of this module is to: Discuss importance and utility of terminal evaluations Characteristics of a good terminal evaluation Exercise on pre-requisites for preparing a good terminal evaluation What needs to be done at different stages of preparation and delivery of a terminal evaluation When a terminal evaluation is planned When a terminal evaluation is conducted When a draft report is prepared During project preparation During project implementation Page 16

Importance and Utility of Terminal Evaluations Source of information on project Results: Outputs, outcomes and progress to impact Implementation, execution, and project cycle related information Project finances including co-financing Recommendations and Lessons for the future GEF M&E Policy (2010): Minimum Requirement 3 Terminal evaluations mandatory since 1995 For full size projects required, encouraged for MSPs Page 17

Importance and Utility of Terminal Evaluations contd. Reporting at the project portfolio level (APR, AMR) Input to other evaluations STAR s performance index About 1000 terminal evaluations completed so far. Terminal evaluation may be accessed at: GEF website: http://www.thegef.org/gef/gef_projects_funding Through PMIS Page 18

Characteristics of a good terminal evaluation GEF IEO criteria for terminal evaluations quality: Outcomes Consistency and comprehensiveness Sustainability Lessons and recommendations Project finances M&E Other characteristics of good terminal evaluations: Transparency and timeliness Candor Balance Utility Question is good terminal evaluation and good project performance the same? Terminal Evaluation Guidelines are available at: http://www.thegef.org/gef/Guidelines%20Terminal%20Evaluations Page 19

Examples of good terminal evaluations Links to examples of good terminal evaluations GEF ID #394: http://www.thegef.org/gef/project_detail?projID=394 GEF ID #1599: http://www.thegef.org/gef/project_detail?projID=1599 GEF ID #1188: http://www.thegef.org/gef/project_detail?projID=1188 GEF ID #1348: http://www.thegef.org/gef/project_detail?projID=1348 Page 20

Exercise: Pre-requisites for preparing a good terminal evaluation You will be asked to think about what needs to be done to facilitate preparation of a good terminal evaluation: At different stages of terminal evaluation preparation. At project preparation and implementation Each group will discuss the requirements at each of these steps and will record their joint responses as bullets in the response sheet provided to them After all stages have been discussed by the groups, one of the groups will present its response for a stage, others may add if their group had an additional bullets not yet covered. Each stage will be presented by a different group. We will end with a summary that pulls the discussion together Page 21

First Discussion: What needs to be done at different stages of the terminal evaluation process? Consider a situation where a project is about to be complete within one year. What needs to be done at different stages of terminal evaluation preparation to ensure that it leads to a good quality evaluation: A. Commissioning of the terminal evaluation B. Conduct of the terminal evaluation C. Finalization of the terminal evaluation Page 22

Second Discussion: What needs to be done before the terminal evaluation process starts? A. What needs to be done during the project preparation stage? Development of M&E plan Tracking tools B. What needs to be done during the project implementation? Implementation of M&E plan Gathering of data on tracking tools Page 23