OVE has looked at PBL in several contexts, but a full-fledged evaluation of IDB’s policy-based lending has not been undertaken to date. OVE routinely reviews the performance of PBL in the context of its country program evaluations. OVE also reviews and validates IDB’s self-evaluations of completed programs and operations and assigns a performance rating to each completed PBL program or freestanding PBL operation. In addition, OVE undertook a thorough review of the design and use of PBL in 2015.7
This section will briefly discuss the performance ratings of PBL based on OVE’s validations of self-evaluations and then present key findings of OVE’s 2015 review of the design and use of PBL.
IDB’s current self-evaluation system was adopted relatively recently. Project teams are required to prepare a project completion report for a programmatic PBL series when the program has been completed or interrupted or, in the case of freestanding PBL operations, at the time of completion of the operation. These self-evaluations are then validated by OVE which assigns an outcome rating to each program or freestanding PBL operation. In the case of programmatic PBL series, the program as a whole is evaluated against a results matrix for the entire program rather than for each loan. In the case of a freestanding PBL, the operation is assessed against the results matrix for that particular operation. The assessment covers three dimensions: relevance, effectiveness, and sustainability. An overall performance rating is assigned based on a weighted average of the ratings achieved on each of these three dimensions.8
As this system has evolved over time, comparable performance ratings are available for only 4 years; for operations or programs that were validated by OVE in 2017–2020. A total of 26 programs, comprising 48 loans have been rated thus far. Four of these consisted of hybrid operations with a PBL and an investment lending component. Of the 26 validated programs, 15 (58%) achieved an overall outcome rating of partly successful or higher. Excluding the hybrid operations, 14 of the 22 programs (64%) achieved a rating of partly successful or higher (compared with 57% of investment loans).
- 7
IDB Office of Evaluation and Oversight, Design and Use of Policy-Based Loans at the IDB. Document RE-485-6. Washington, DC: IDB https://publications.iadb.org/en/ove-annual-report-2015-technical-note-design-and-use-policy-based-loans-idb - 8
Relevance, effectiveness and sustainability are each rated on a four-point scale. The overall performance rating is a weighted average of the scores on each of these three dimensions, with relevance and sustainability being given a weight of 20% each and effectiveness 60%. The overall performance rating uses a six-point scale ranging from highly successful to highly unsuccessful.