About ECG

About the Evaluation Cooperation Group

 

ECG members at the ECG January 2016 Meeting, hosted by the Independent Evaluation Group (IEG) at World Bank Headquarters, Washington DC

The Evaluation Cooperation Group (ECG) was established in 1996 to promote a more harmonized approach to evaluation methodology, following an assessment of the five major MDBs (the African Development Bank, the Asian Development Bank, the European Bank for Reconstruction and Development, the Inter-American Development Bank and the World Bank).
 
In particular, the report of the assessment, Serving a Changing World: Report of the Task Force on Multilateral Development Banks (1996) stated that a ‘determined effort should be made to harmonize performance indicators and evaluation criteria, taking into account the differing circumstances of each institution. The heads of the five MDB evaluation units should meet on a regular basis to exchange experience, with this purpose in mind’. 
 
Promoting evaluation harmonization
To promote evaluation harmonization among MDBs, the ECG has developed and uses Good Practice Standards (GPS) and benchmarking studies to assess the extent to which the GPS are applied by its members. The GPS cover:
  • Country Strategy and Programme Evaluation
  • Policy-Based Lending Operations Evaluation
  • Private Sector Operations Evaluation
  • Public Sector Operations Evaluation
  • Review Framework for the Evaluation Function (Peer Reviews)
  • Technical Assistance/Cooperation Evaluation

In 2012, the ECG Big Book on Good Practice Standards was published as an online document to addresses overlaps noted in the individual GPSs, and resolves differences in terminologies, if any, without losing the original intent of the four GPS reports taken individually. It rearranges the individual GPSs, but there are no material changes or deviations from the original content. 

Members also work together on joint and meta-evaluations and discuss such evaluation issues as the independence of evaluation offices in MDBs, evaluability assessment, the results agenda of MDBs, and lessons learning and dissemination. The ECG has also developed a process for peer review of independent evaluation offices.

Evaluation practice in members’ central evaluation units has improved significantly since 1996, in light of the collaborative effort and information sharing facilitated by participation in the ECG. ECG has promoted not only harmonization but also evaluation independence, professionalism, lessons learning and cooperation. 

Membership

ECG has a three-tier membership structure composed of: members, permanent observers and temporary observers. ECG currently has ten members and three permanant observers. Temporary observers are institutions in the process of applying for full membership. In 2011, ECG members established a formal process for institutions applying for ECG membership. Full details on membership criteria and applications are available in the document ECG Membership Process (see below). 

At the ECG Spring 2014 meeting held in Johannesburg (April 2014), the Black Sea Trade and Development Bank were welcomed as the newest ECG members.

At the ECG Fall 2014 meeting held in Washington DC from 4-5 December, ECG members took note of the CEB’s unilateral decision to withdraw its application for ECG membership. As a result, the Evaluation Department of the CEB will no longer be able to participate in ECG meetings or work streams as a regular observer.

Working practices

ECG members and observers meet twice a year (Spring and Autumn) to discuss and share experiences on current evaluation issues. The meetings are hosted by the ECG Chair, the role of which rotates among ECG members. 

Rotation schedule of the ECG Chair:

2017 - International Fund for Agricultural Development
2016 - European Bank for Reconstruction and Development
2015 - Independent Evaluation Group (IEG), World Bank Group
2014 - African Development Bank
2013 - Inter-American Development Bank
2012 - European Investment Bank
2011 - Asian Development Bank
2010 - European Bank of Reconstruction and Development
2009 - World Bank Group
2008 - African Development Bank
2007 - Inter-American Development Bank

Updated November 2016

MANDATE OF THE EVALUATION COOPERATION GROUP

PURPOSE

ECG contributes to excellence in evaluation by promoting:

  • Mutual learning and sharing of experience on issues such as evaluation governance, tools, methods, products and management.

  • Increased contribution of evaluation to organisational results;

  • Adoption of good practice standards and benchmarking against these; and

  • Facilitating jointly conducted evaluations.

SCOPE

ECG activities to promote the quality, usability and use of evaluation work in the IFI system include:

Improving the Quality of Independent Evaluation by:

  • Developing and periodically updating good practice standards for relevant areas of work.
  • Conducting peer reviews and benchmarking against good practice standards (on member demand).
  • Providing a forum and information sharing service on all aspects of evaluation, including methods, management of the evaluation function, governance and accountability, dissemination, and the interactionsAnchor with management and other stakeholders.
  • Facilititating training in evaluation methodologies and approaches for the staff of central evaluation departments, and those conducting self-evaluations when needed.

Improving the Quality of Self-Evaluation in Member Institutions by:

  • Promoting self-evaluation, developsor encourages development of good practice standards for self-evaluation, and supports their adoption by members’ institutions.

Improving the Usability and Use of Evaluation Findings by:

  • Promoting increased usability and utilisation of evaluation findings within IFIs.

Outreach to the Wider Evaluation and Development Community by:

  • Promoting good practice standards and principles among the wider evaluation community.
  • Participating in meetings of other evaluation networks to both learn from the experience of others, and in turn contribute to their learning.
  • Supporting the dissemination of members’ evaluation reports and related products.
  • Encouraging members who share a common interest to produce joint syntheses of evaluation findings on topical issues, which ECG will help promote.
Year in review

In 2014, ECG was chaired by the African Development Bank. In keeping with tradition, the ECG Chair convened two meetings during the course of the year for members to meet and discuss topics of interest and concern. 

The Spring 2014 meeting was held in April at the AfDB South African Regional Centre in Centurion, South Africa. The agenda included:

  • Evaluation independence, in particular in response to the US Congress legislative directive that the US Treasury push for “independent outside evaluation” for the MDBs;
  • Sharing experiences on self-evaluation, evaluating infrastructure and day-to-day pressures;
  • Ways to both develop and improve ways to develop work programmes;
  • Mainstreaming gender in evaluations; and
  • Evolution in the country strategy evaluation process and their contribution to a comprehensive evaluation of institutional development results.

Professor Ben Turok, former anti-apartheid activist and veteran ANC Member of Parliament, delivered the keynote address and spoke of the need for a special approach to evaluation, especially in South Africa. 

A special open session was also held during which Professor Leonard Wantchekon, Professor at Princeton University gave a keynote address on education and human capital externalities in colonial Africa.

The two day ECG meeting was preceded by a successful meeting with sub-regional development institutions (SRDI) in Africa. They agreed to establish the African evaluation platform for SRDIs, emulating the ECG. The purpose of the platform is to promote an evaluation culture with quality and professionalism for an increased contribution of evaluation to organizational results. Following the meeting a concept note was developed and a core group established to discuss issues related to coordination, planning and budgeting for the operationalization of what is known as the Evaluation Platform for Regional African Development institutions (EPRADI).

Another innovative feature of the Spring meetings was to increase the outreach and dissemination efforts. In addition to two Press conferences, a 30 minute show on CNBC was a highlight of the outreach effort. A few heads of evaluation were on the CNBC show and discussed the role of evaluation in strengthening development outcomes, the elements of independence and the key to success.

Two other outcomes of the Spring meeting were the approval of the full membership to ECG of the Black Sea Trade and Development Bank, and the agreement of the ECG cost-sharing arrangement which shares ECG costs among the 10 members based on the size of the budget of each agency evaluation department. 

The Fall 2014 meeting was held in December in Washington DC. The Chair had hoped to host members at the AfDB headquarters in Abidjan, but the meeting was moved due to the ebola crisis in West Africa. 

The agenda included evaluating credit lines, self-evaluation private public partnership evaluation, use of evaluations, safeguards, independence and evaluation criteria for country strategy papers. 

Rakesh Nangia, ECG Chair, opened the meetings by welcoming Juha Uitto, Director, Independent Evaluation Office at the GEF and Oscar Garcia, Director, Independent Office of Evaluation, IFAD.  Both have been newly appointed to these postions and this was their first meeting in their new capacity.  Rakesh also invited representatives of US Treasury to discuss the recommendations of their review of evaluation standards and practices at the MDBs.

The open session focused on climate change and was moderated by Rakesh Nangia, Evaluator General and Vice President, Independent Development Evaluation, AfDB. Andrew Steer, President and CEO of the World Resources Institute, delivered the keynote presentation during which he emphasized the urgent need to take steps towards combating climate change. The keynote speech was followed by a panel discussion with Naoko Ishii, Chairperson and CEO of the Global Environment Facility and Julie Katzmann, Executive Vice President of the Inter-American Development Bank. 

In terms of other key news for ECG in 2014, the new website and newsletter were launched in November 2014. The new website serves not only as a portal to bring together news from all ECG members, but also hosts a document library and a calendar of evaluation-related events to improve information sharing and communications. 

ECG also published two papers, one on the rationale, challenges and opportunities of evaluating support to SMEs (September 2014) and the second on knowledge management in the IFIs (December 2014). 

The ECG Chair serves for a one year term and, at the December ECG meeting, AfDB handed the Chairpersonship to the Independent Evaluation Group (IEG) of the World Bank. Rakesh thanked ECG members for their collegiality, patience and support throughout the year. Members agreed that 2015 will be a year for ECG to focus on its strategic direction and will begin with a stock taking exercise to assess what the group has achieved in the 20 years since its inception.