Evaluation Department, European Bank for Reconstruction and Development
The independent Evaluation Department evaluates the performance of the Bank’s completed projects and programmes relative to objectives.
It systematically analyses the results of both individual projects and wider themes defined in the Bank’s policies.
The core objective of evaluation is to contribute to the Bank’s legitimacy, relevance and to superior institutional performance. To achieve its core objective, the Evaluation Department fulfils two primary functions. It provides a critical instrument of accountability through objective, evidence based performance assessment of outputs and outcomes relative to targets and it contributes to institutional learning for future operations by presenting operationally useful findings.
Last updated: April 2014
Yes, Evaluation Department (EvD)
EvD reports solely and directly to the Board. Based on the approved (currently single-year) work programme, a budget for EvD is prepared by the Chief Evaluator. This is presented separately from the rest of the Bank’s budget. Such budget, which is required to be sufficient to enable EvD to carry out the proposed work programme, is approved by the Board, at the same meeting as the Board decision on the Bank’s general budget, but with a separate vote on EvD’s work programme and budget.
Comparative level not specified.
As per the Evaluation Policy, Management will invite EvD to observe or contribute to Management meetings where this may enhance communication and understanding of issues of common concern. This happens.
As per the Evaluation Policy, EvD has access to all internal information required to execute its Board approved work programme. The fundamental operating presumption is one of full information sharing coupled with full respect for confidentiality.
Chief Evaluator: 1
Senior adviser: 1
Senior evaluation managers: 4
Principal evaluation manager: 2
Evaluation manager: 5
Communication Officer: 1
Evaluation Analyst: 1
Administrative Officer: 2
Total staff: 17
The Chief Evaluator manages EvD staff, to whom the Bank’s human resources and other relevant policies apply. The Chief Evaluator is free to make recruitment decisions, within the limits of the EvD budget, without Management or Board involvement. The salaries of EvD professional staff members are determined by the Chief Evaluator in consultation with the Human Resources Department.
The allocation of salary increases is based on performance and market positioning. In addition to a salary, the remuneration of the EvD’s professional staff may include performance based compensation. In managing EvD financial and human resources, the Chief Evaluator consults and cooperates with relevant departments and units in the Bank, in particular the Budget Unit and the Human Resource Department.
The Chief Evaluator is selected in accordance with a selection procedure established by the Board of Directors. Currently, a Selection Committee for the Chief Evaluator is established consisting of the members of the Audit Committee, the Chair of the Board Steering Group, and the Vice President in charge of Human Resources. The Selection Committee is chaired by the Chair of the Audit Committee. The Selection Committee appoints an executive search firm to conduct an extensive search. Subsequently, the Committee reviews a shortlist of candidates and conducts interviews and proposes a candidate by majority vote. After completion of the selection procedure, the Committee makes a recommendation to the President for the engagement of the Chief Evaluator, without prejudice to the role of the Budget and Administrative Affairs Committee. Before the end of the selection process, a hearing of the proposed candi-date will be organized with interested Board Directors. The President will then consult with the Board in an Executive Session before sending a formal recommendation to the Board for its approval. The Chief Evaluator is appointed for a term of up to four years which may be renewed. Ordinarily, a Chief Evaluator will not hold the position for more than eight years. At the end of his/her term(s), or after removal, the Chief Evaluator is ineligible to perform remunerated services for the EBRD, unless the Board of Directors decides otherwise.
Consulting budget is 16% of total EvD budget or 21% of staff costs.
Possible in and out.
The Evaluation Policy provides that EvD prepares a multi-year work programmes setting out proposed evaluation priorities, products and activities in detail. It also proposes budgets to deliver its work programmes, and results frameworks to enable assessment of its own performance. All are discussed with Management and submitted to the Board for review and approval. Currently, only single-year work programmes and budgets are approved.
• For accountability purposes, all operations are self-evaluated when deemed ready by EvD and Management following agreed guidelines. A sample of such operations, designed to yield statistically sound and credible results, is subject to independent evaluation or validation by EvD.
• Currently, about 10 transactions per year are purposively selected for in-depth evaluation by EvD.
• TC attached to transactions is evaluated as part of the transaction. Standalone TC is evaluated via or as part of special studies. In principle, therefore, 100% of transactional TC should be self-evaluated though in practice TC is not well covered in self-evaluations, with 100% of self-evaluated projects subject to EvD in-depth evaluation, validation or review. 100% of standalone TC should be self-evaluated but these are neither reviewed nor validated by EvD.
For learning purposes, subjects are chosen for evaluation that are likely to yield relevant and timely findings, lessons and recommendations on what was achieved, what worked, what did not and why. Wide consultations are held throughout the Bank to canvass ideas for special studies.
Currently, EvD does not do country programme evaluations although the Evaluation Policy provides for these to be done.
Several new product types have been introduced including syntheses, evaluation briefs and evaluation insights.
Staff time is not allocated to specific products so total costs of each product type are not known. The cost of travel and consulting varies widely depending upon the needs of the particular assignment.
1.18% of EBRD’s net administrative expenses (excl. depreciation).
Under review in 2013
Under review in 2013
Under review in 2013
Under review in 2013
Disclosure of evaluation documents is incorporated in EBRD’s public information policy (PIP).
To be reviewed in 2013
EBRD Public Information Policy
Current provisions of the PIP with respect to external disclosure of EvD reports are as follows:
• Summaries of OPER reports on investment operations (no names of companies and sponsors are disclosed);
• OPER reports on TC operations;
• Special studies;
• Work Programme Final Report;
• Annual Evaluation Overview Report (AEOR);
EvD will argue for greater disclosure of its reports as part of the 2013 review of the PIP.
Self-evaluations and validations of self-evaluations
Under review in 2013
1 Communication officer out of a total staff complement of 16.
1. Operations Assessment (OPA) and Operation Evaluation (OE) evaluate investment operations. Timing: 1. 5 years after full loan disbursement and 2 years after last disbursement of equity investment. After at least one year of commercial operation and in principle one year of audited accounts must be available.
2. Special Studies are timed to be relevant to learning needs/ decision points (e. g. approval of a new strategy)
3. Annual Evaluation Review and Report on Actions taken on Recommendations
1. Project level evaluations:
• Approximately 250 public sector ICR Reviews
• Approximately 200 private sector project validations(of self evaluations) and independent project evaluations; In FY12, IEG validated and evaluated in total 217 projects: 75 IFC investment projects, 131 IFC advisory projects, 11 MIGA guarantees.
2. Country Evaluations
• Country Program Evaluations
• Country Assistance Strategy Completion Report reviews
3. Thematic & Sector Evaluations
4. Corporate & Process Evaluations
5. Impact Evaluations
6. Annual Report: Results and Performance report
7. Global and Regional Partnership Program reviews
8. IEG Work Program and Budget
9. Other reports: Quick turnaround notes, Cluster Evaluations, evaluation briefs, IEG Self-Evaluation, etc.
Draft evaluations are discussed with appropriate operations staff before finalization. IEG reports are issued under DGE signature and approval.
Large scale evaluations (CPE, Sector, Thematic, Corporate evaluations) are submitted to the Board for discussion, typically by CODE, and occasionally by the Board.
CAS Completion Report Reviews are sent to the Board to be discussed together with the new Country Assistance Strategies.
Project Performance Assessment Reports are forwarded to the Board for information.
Other project level evaluations (Implementation Completion Report Reviews, activities supported by IFC and MIGA) are not forwarded to the Board but are shared with relevant operational staff.