Independent Evaluation Department at the Asian Development Bank
The Independent Evaluation Department (IED) independently and systematically evaluates ADB policies, strategies, operations, and special concerns that relate to organizational and operational effectiveness. By doing so, it contributes to achieving development effectiveness of ADB operations by providing evaluation feedback on performance, and generating and disseminating evaluation lessons.
There are two levels of evaluation in ADB:
Mandate
IED’s mandate is to undertake evaluation activities to help ADB Management and developing member country stakeholders who are responsible for planning, designing, and implementing projects and programs to understand whether resources have been well spent, and whether the planned outcomes have been achieved.
Evaluation independence
To be credible and to provide an objective performance assessment, the evaluation function must be independent from project or program design and implementation. To fulfill its mission effectively, IED develops and sustains a reputation for excellence by following the principles of independence, impartiality, and integrity. Drawing on the good practice standards of evaluation agencies, IED has recognized and adopted four dimensions of evaluation independence:
Last update: November 2014
Yes, Independent Evaluation Department (IED).
IED reports directly to the Board through the Development Effectiveness Committee (DEC).
IED’s work program and budget are approved by the Board. Director General (DG), IED, in consultation with the DEC and AsDB Management, prepares an annual budget proposal that will be subject to review first by the DEC then by the Budget Review Committee. The budget proposal is presented for consideration and approval by the Board separately from AsDB’s overall administrative budget. DG, IED has the authority to reallocate resources across budget items.
DG, one level below Vice President (VP), same level as operations department directors general.
DG, IED participates selectively (rarely) at Management Review Meetings, chaired by the President or a VP, to examine new lending, policy or strategy proposals before these are completed and finalized for Board submission.
The approved policy allows IED full, unfettered access to AsDB records and information related to evaluation work, with the exception of personal information that is typically restricted. Similarly, IED is free to consult with any individual or group, within and outside AsDB, it deems relevant— including government officials, and members of private sector, NGOs, and media in the assisted DMCs.
Director General: 1
Division Directors: 2
Advisors: 2
Lead Evaluation Specialist: 1
Principal Evaluation Specialists: 8
Senior Evaluation Specialists: 9
Evaluation Specialists: 7
National Evaluation Officers: 11
Administrative Staff: 13
DG, IED is responsible for the final selection of IED personnel, in accordance with AsDB personnel guidelines. Budget, Personnel, and Management Systems Department of AsDB handles the administrative processes, in accordance with these guidelines. The terms and conditions of the services of IED staff are the same as for other AsDB staff, as provided by staff regulations and administrative orders of AsDB. They include the same performance review process, regulation, disciplinary procedures, and grievance procedures. IED staff may work in other depart-ments/offices of AsDB before and after working in IED. In this context, IED has adopted formal guidelines on avoiding conflicts of interest in independent evaluations. Transfer of IED staff to other departments/offices is similarly governed by AsDB-wide policies, rules, and regulations.
DG, IED is appointed by the Board, upon the recommendation of the DEC in consultation with the President (i.e., seeking the views and opinions of the President). DG, IED has a 5-year non-renewable term. During this period, DG, IED can only be removed by the Board on the grounds of inefficiency or misconduct. Upon completion of the term or after removal, DG, IED is ineligible for any staff position within AsDB.
Currently, the average tenure of other staff is about 3-5 years. Rotation is encouraged.
On average, at least 30% of IED dollar costs on evaluation work (funded by internal administrative expense budget and AsDB facility type technical assistance) is for consultants (2010‚àí2013).
None so far.
DG, IED proposes a three-year rolling work program, after consultations with the DEC, Management and other departments, taking into account issues of relevance to AsDB developing member countries and the current institutional priorities. The Board approves the IED work program on a no-objection basis following its review and endorsement by the DEC.
Validation of self-evaluation:
Since August 2007, IED independently validates project completion reports (PCRs) and extended annual review reports (XARRs). In 2012, IED adopted new guidelines in validating PCRs and XARRs. From 2013 onwards, it expects to validate at least 75% of project completion reports (PCRs) by operating units for public sector lending and all extended annual review reports (XARRs) for private sector lending.
Independent Evaluation:
Rather than independently evaluating a randomly selected sample of completed programs and projects, IED selects a purposeful sample of about 12 sovereign/non-sovereign operations for in-depth evaluation each year [Project Performance Evaluation Reports-PPERs]
Operations, policies and strategies, and business processes having implications for development effectiveness of AsDB operations, e.g., project, program, non-sovereign operations, technical assistance, regional cooperation, country, sectoral, thematic topics for sector, thematic, corporate evaluations, AsDB processes, policy/strategy reviews, and impact assessments.
Depends on nature of the study, but the budget assumes that, normally, an individual project evaluation takes 4-6 staff months while broader evaluations take 7-10 staff months of work (which could take 12-18 months of elapsed time, given multitasking).
IED internal administrative budget is around 1.8% of the equivalent AsDB-wide budget in 2013 (1.8% in 2011 and 1.9% in 2012).
4-point scale
Yes
Relevance
Effectiveness
Efficiency
Sustainability
AsDB performance
Borrower performance
Impact
Disclosure of evaluation documents falls within AsDB’s Public Communications Policy (PCP)
ADB routinely produces or requires its borrowers and clients to produce documents which are required to be disclosed throughout the AsDB project cycle—Country Partnership Strategy, Preparation, Appraisal/Approval, Implementation, and Evaluation.
A new PCP took effect on 2 April 2012 (replacing the 2005 policy).
All reports available online (see also sections 6.3 and 7.3)
None; a redacted version of private sector evaluations are disclosed, with commercially confidential parts removed from the report.
Uploading of documents onto the IED’s website takes place within 2 weeks of evaluation completion. An evaluation is considered complete when the report is approved by DG, IED.
One international staff (principal evaluation specialist on part-time basis) and 2 national staff (national officer and administrative staff on full time basis) and a pool of consultants (average of 2.5 person-years per year from second half of 2011‚àíend December 2013) to support knowledge sharing and outreach activities like developmental editing, publications writing, multimedia/media outreach/graphics design, web administration, and evaluation database management.
From 2013 onwards, IED’s administrative expense budget added provisions for knowledge and outreach activities. These covered outreach, media, multimedia etc. and around 1.1% of total administrative expense budget in 2013. IED communications and dissemination is also partly funded from AsDB facility-type technical assistance in support of IED's yearly program of higher-level evaluations.
1. Major evaluations:
• Thematic Evaluation Studies
• Corporate Evaluation Studies
• Real-time Evaluation Studies
• Evaluation Knowledge Studies
• Impact Evaluation Studies
• Regional Cooperation/Country Assistance Program Evaluations
• Annual Evaluation Review
2. Intermediate evaluations:
• Project/ Program/Technical Assistance Performance Evaluation Reports
• Validation Reports on Country Partnership Strategy Final Review Reports, Project Completion Reports and Extended Annual Review Reports
• Sector Assistance Program Evaluations/Sector Assessments
• Topical working papers, synthesis of evaluation findings
Concept Paper. The draft paper (in cases where it is required) is reviewed and endorsed by evaluation division Directors for approval by Director General (DG), IED.
Evaluation Approach Paper (EAP). The EAP review process begins with internal and external peer reviews of the draft EAP for higher level evaluations and internal peer review for project level evaluations. Following peer review, a one-stop review meeting (OSM) is held for higher level evaluations. The OSM discusses the scope, approach, and budget of the EAP and incorporates comments and suggestions of discussants. After the OSM meeting, the revised draft EAP is submitted for interdepartmental review by relevant departments. It is approved by DG, IED after incorporating comments received from the various departments.
Evaluation Report. Similar to the EAP, the draft report is subject to peer reviews and an OSM before it is sent for interdepartmental review. Draft report will be revised based on comments received and a comments matrix is prepared. Comments are requested from relevant departments and from concerned government or borrower. If necessary, a DG level meeting will be held for select evaluation studies. The draft report is subject to a third party review by IED staff who will read through it and provide a fresh look for inconsistencies and errors. Prior to approval by DG, IED, the draft report is submitted for editing by the AsDB editing unit.
Circulation of Final Evaluation Report. The final report is approved by DG, IED and circulated to the Board and Management (inviting a Management response).
Evaluation reports are made available to public online at IED’s website and in hard copies. PPERs maybe circulated to executing and government agencies through relevant AsDB departments to advise them on actions that need to be taken with respect to report’s recommendations. It may also be circulated to AsDB departments with reference to any relevant recommendations. Alternatively, interested parties are informed by email that the report has been uploaded and is available at IED’s website.
All public sector evaluation reports are publicly disclosed on the AsDB website upon circulation to the President and Board of Directors. Redacted versions of private sector evaluations are disclosed, with commercially confidential parts removed from the report. IED's annual evaluation report is publicly available upon discussion by the DEC.
Public Sector: Not applicable
Private Sector: Not disclosed
Copyright © 2014 ECG. All Rights Reserevd.