Independent Evaluation Department, Asian Development Bank

Website:

http://www.adb.org/evaluation

Headquarters:

Manila, Philippines

Type:

Members

Follow On

https://www.facebook.com/adbevaluation

https://twitter.com/adbevaluation

Work Programme:

AsDB_WPB_2016-18.pdf

Evaluation Policy:

Operations Manual 2013.pdf

Independent Evaluation Department at the Asian Development Bank

The Independent Evaluation Department (IED) independently and systematically evaluates ADB policies, strategies, operations, and special concerns that relate to organizational and operational effectiveness. By doing so, it contributes to achieving development effectiveness of ADB operations by providing evaluation feedback on performance, and generating and disseminating evaluation lessons.

There are two levels of evaluation in ADB:

self-evaluation, conducted by those responsible for designing and implementing a country strategy, program, or project
independent evaluation undertaken by IED

Mandate

IED’s mandate is to undertake evaluation activities to help ADB Management and developing member country stakeholders who are responsible for planning, designing, and implementing projects and programs to understand whether resources have been well spent, and whether the planned outcomes have been achieved.

Evaluation independence

To be credible and to provide an objective performance assessment, the evaluation function must be independent from project or program design and implementation. To fulfill its mission effectively, IED develops and sustains a reputation for excellence by following the principles of independence, impartiality, and integrity. Drawing on the good practice standards of evaluation agencies, IED has recognized and adopted four dimensions of evaluation independence:

Behavioral autonomy
Organizational independence
Insulation from external influence
Avoidance of conflicts of interest

Last update: November 2014

Hidden Group

Type:

1.1 Separate Evaluation Dept

Description:

Yes, Independent Evaluation Department (IED).

Type:

1.2 Location in Organisation Chart and Budget

Description:

IED reports directly to the Board through the Development Effectiveness Committee (DEC).

IED’s work program and budget are approved by the Board. Director General (DG), IED, in consultation with the DEC and AsDB Management, prepares an annual budget proposal that will be subject to review first by the DEC then by the Budget Review Committee. The budget proposal is presented for consideration and approval by the Board separately from AsDB’s overall administrative budget. DG, IED has the authority to reallocate resources across budget items.

Type:

1.3 Seniority of Head of Department

Description:

DG, one level below Vice President (VP), same level as operations department directors general.

Type:

1.4 Participation of the Head of the Evaluation Function in Internal Senior Management Meetings

Description:

DG, IED participates selectively (rarely) at Management Review Meetings, chaired by the President or a VP, to examine new lending, policy or strategy proposals before these are completed and finalized for Board submission.

Type:

1.5 Access to Information by Staff of the Evaluation Department

Description:

The approved policy allows IED full, unfettered access to AsDB records and information related to evaluation work, with the exception of personal information that is typically restricted. Similarly, IED is free to consult with any individual or group, within and outside AsDB, it deems relevant— including government officials, and members of private sector, NGOs, and media in the assisted DMCs.

Type:

2.1 Staffing

Description:

Director General: 1
Division Directors: 2
Advisors: 2
Lead Evaluation Specialist: 1
Principal Evaluation Specialists: 8
Senior Evaluation Specialists: 9
Evaluation Specialists: 7
National Evaluation Officers: 11
Administrative Staff: 13

Type:

2.2 Managing the Staff of the Department

Description:

DG, IED is responsible for the final selection of IED personnel, in accordance with AsDB personnel guidelines. Budget, Personnel, and Management Systems Department of AsDB handles the administrative processes, in accordance with these guidelines. The terms and conditions of the services of IED staff are the same as for other AsDB staff, as provided by staff regulations and administrative orders of AsDB. They include the same performance review process, regulation, disciplinary procedures, and grievance procedures. IED staff may work in other depart-ments/offices of AsDB before and after working in IED. In this context, IED has adopted formal guidelines on avoiding conflicts of interest in independent evaluations. Transfer of IED staff to other departments/offices is similarly governed by AsDB-wide policies, rules, and regulations.

Type:

2.3 Selection and Average Tenure of Head of Evaluation and Other Evaluation Staff

Description:

DG, IED is appointed by the Board, upon the recommendation of the DEC in consultation with the President (i.e., seeking the views and opinions of the President). DG, IED has a 5-year non-renewable term. During this period, DG, IED can only be removed by the Board on the grounds of inefficiency or misconduct. Upon completion of the term or after removal, DG, IED is ineligible for any staff position within AsDB.

Currently, the average tenure of other staff is about 3-5 years. Rotation is encouraged.

Type:

3.1 Consultants: Proportion of Business Covered

Description:

On average, at least 30% of IED dollar costs on evaluation work (funded by internal administrative expense budget and AsDB facility type technical assistance) is for consultants (2010−2013).

Type:

3.2 Internal Secondment

Description:

None so far.

Type:

4.1 Work Programme

Description:

DG, IED proposes a three-year rolling work program, after consultations with the DEC, Management and other departments, taking into account issues of relevance to AsDB developing member countries and the current institutional priorities. The Board approves the IED work program on a no-objection basis following its review and endorsement by the DEC.

Type:

4.2 Percentage of Projects Subject to Evaluation

Description:

Validation of self-evaluation:
Since August 2007, IED independently validates project completion reports (PCRs) and extended annual review reports (XARRs). In 2012, IED adopted new guidelines in validating PCRs and XARRs. From 2013 onwards, it expects to validate at least 75% of project completion reports (PCRs) by operating units for public sector lending and all extended annual review reports (XARRs) for private sector lending.

Independent Evaluation:
Rather than independently evaluating a randomly selected sample of completed programs and projects, IED selects a purposeful sample of about 12 sovereign/non-sovereign operations for in-depth evaluation each year [Project Performance Evaluation Reports-PPERs]

Type:

4.3 Basis of Studies - country, project by project, sector...

Description:

Operations, policies and strategies, and business processes having implications for development effectiveness of AsDB operations, e.g., project, program, non-sovereign operations, technical assistance, regional cooperation, country, sectoral, thematic topics for sector, thematic, corporate evaluations, AsDB processes, policy/strategy reviews, and impact assessments.

Type:

5.1 Costs

Description:

Depends on nature of the study, but the budget assumes that, normally, an individual project evaluation takes 4-6 staff months while broader evaluations take 7-10 staff months of work (which could take 12-18 months of elapsed time, given multitasking).

Type:

5.2 Budget

Description:

IED internal administrative budget is around 1.8% of the equivalent AsDB-wide budget in 2013 (1.8% in 2011 and 1.9% in 2012).

Type:

8.1 Rating Scale Used

Description:

4-point scale

Type:

8.2 Numerical Values Assigned to Ratings

Description:

Yes

Type:

8.3 Criteria Aggregated into the Project Performance Rating

Description:

Relevance
Effectiveness
Efficiency
Sustainability

Type:

8.4 “Below the Line” Criteria

Description:

AsDB performance
Borrower performance
Impact

Type:

7.1 Existence of Specific Disclosure and Comm Policy or Guidelines for Evaluation

Description:

Disclosure of evaluation documents falls within AsDB’s Public Communications Policy (PCP)

ADB routinely produces or requires its borrowers and clients to produce documents which are required to be disclosed throughout the AsDB project cycle—Country Partnership Strategy, Preparation, Appraisal/Approval, Implementation, and Evaluation.

A new PCP took effect on 2 April 2012 (replacing the 2005 policy).

Type:

7.1.1 Web address of disclosure policy

Description:

Public Communications Policy

Type:

7.2 Publication

Description:

All reports available online (see also sections 6.3 and 7.3)

Type:

7.2.1 Publication web address

Description:

Evaluation website

Type:

7.3 Which final evaluation products are not disclosed?

Description:

None; a redacted version of private sector evaluations are disclosed, with commercially confidential parts removed from the report.

Type:

7.4 How long after completion are evaluations disclosed (in practice and according to disclosure policy)?

Description:

Uploading of documents onto the IED’s website takes place within 2 weeks of evaluation completion. An evaluation is considered complete when the report is approved by DG, IED.

Type:

7.5 Allocated Resources (human, financial) for Communication and Dissemination

Description:

One international staff (principal evaluation specialist on part-time basis) and 2 national staff (national officer and administrative staff on full time basis) and a pool of consultants (average of 2.5 person-years per year from second half of 2011−end December 2013) to support knowledge sharing and outreach activities like developmental editing, publications writing, multimedia/media outreach/graphics design, web administration, and evaluation database management.

From 2013 onwards, IED’s administrative expense budget added provisions for knowledge and outreach activities. These covered outreach, media, multimedia etc. and around 1.1% of total administrative expense budget in 2013. IED communications and dissemination is also partly funded from AsDB facility-type technical assistance in support of IED's yearly program of higher-level evaluations.

Type:

6.1 Types of Reports & Timing

Description:

1. Major evaluations:
• Thematic Evaluation Studies
• Corporate Evaluation Studies
• Real-time Evaluation Studies
• Evaluation Knowledge Studies
• Impact Evaluation Studies
• Regional Cooperation/Country Assistance Program Evaluations
• Annual Evaluation Review

2. Intermediate evaluations:
• Project/ Program/Technical Assistance Performance Evaluation Reports
• Validation Reports on Country Partnership Strategy Final Review Reports, Project Completion Reports and Extended Annual Review Reports
• Sector Assistance Program Evaluations/Sector Assessments
• Topical working papers, synthesis of evaluation findings

Type:

6.2 Discussion and Clearing of Reports

Description:

Concept Paper. The draft paper (in cases where it is required) is reviewed and endorsed by evaluation division Directors for approval by Director General (DG), IED.

Evaluation Approach Paper (EAP). The EAP review process begins with internal and external peer reviews of the draft EAP for higher level evaluations and internal peer review for project level evaluations. Following peer review, a one-stop review meeting (OSM) is held for higher level evaluations. The OSM discusses the scope, approach, and budget of the EAP and incorporates comments and suggestions of discussants. After the OSM meeting, the revised draft EAP is submitted for interdepartmental review by relevant departments. It is approved by DG, IED after incorporating comments received from the various departments.

Evaluation Report. Similar to the EAP, the draft report is subject to peer reviews and an OSM before it is sent for interdepartmental review. Draft report will be revised based on comments received and a comments matrix is prepared. Comments are requested from relevant departments and from concerned government or borrower. If necessary, a DG level meeting will be held for select evaluation studies. The draft report is subject to a third party review by IED staff who will read through it and provide a fresh look for inconsistencies and errors. Prior to approval by DG, IED, the draft report is submitted for editing by the AsDB editing unit.

Circulation of Final Evaluation Report. The final report is approved by DG, IED and circulated to the Board and Management (inviting a Management response).

Type:

6.3 Distribution of Evaluation Reports

Description:

Evaluation reports are made available to public online at IED’s website and in hard copies. PPERs maybe circulated to executing and government agencies through relevant AsDB departments to advise them on actions that need to be taken with respect to report’s recommendations. It may also be circulated to AsDB departments with reference to any relevant recommendations. Alternatively, interested parties are informed by email that the report has been uploaded and is available at IED’s website.

All public sector evaluation reports are publicly disclosed on the AsDB website upon circulation to the President and Board of Directors. Redacted versions of private sector evaluations are disclosed, with commercially confidential parts removed from the report. IED's annual evaluation report is publicly available upon discussion by the DEC.

Type:

6.4 Identity of Promoters of Projects

Description:

Public Sector: Not applicable
Private Sector: Not disclosed

search-block

Main menu

Independent Evaluation Department, Asian Development Bank

Contents

Type:

Follow On

Hidden Group

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

search-block

Member Login

Main menu

Independent Evaluation Department, Asian Development Bank

Contents

Type:

Follow On

Hidden Group

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Type:

Related Contents