IEG thematic evaluations and learning products (and the World Bank’s DPF retrospective) provide rich insights into various dimensions of DPF performance, especially with respect to results frameworks, policy-based guarantees, use of DPF as an anti-crisis instrument, macroeconomic frameworks, and the performance of DPF in low-income IDA countries. The findings included the following:
- Results frameworks in DPF documents have improved, but shortcomings remain in the relevance of results indicators and prior actions. In particular, prior actions were found to be lacking in many instances in the sense that their completion did not contribute critically to development objectives.13World Bank. 2015. Quality of Results Frameworks in Development Policy Operations. Washington, DC: World Bank.
- IEG’s review of evidence from the early policy-based guarantees (PBGs) during the 2011–2015 period found that borrowers, with World Bank support, could typically meet their financing needs during difficult market conditions. A robust macroeconomic and fiscal policy framework was essential for sustaining benefits from improved access to private finance for deficit financing. The impact of PBGs on borrowers’ credit terms varied from one program to another, but in all of the PBGs that IEG reviewed, the aggregate interest rates were lower than they would have been without guarantees; however, more evidence is needed on the benefits when the implied interest rate is calculated on nonguaranteed terms and takes account of the erosion of the guarantee’s value over time.14World Bank. 2016. Findings from Evaluations of Policy-Based Guarantees. Washington, DC: World Bank.
- The World Bank responded to the global financial crisis, especially in middle-income countries, with 67 crisis response development policy operations focused largely on anti-crisis fiscal management. Policy frameworks focused on the timely provision of budget financing at the time of market turbulence and measures to strengthen fiscal sustainability by improving the effectiveness of public expenditures. They included improvements in the targeting of social entitlements and cuts in unproductive expenditures. At the same time, and because of their counter-cyclical focus, policy frameworks included comparatively few structural measures, which occurred in less than one-third of crisis response operations. Also, tax policy and tax administration reforms to improve revenue collection were notably less frequent.15World Bank. 2017. Crisis Response and Resilience to Systemic Shocks: Lessons from IEG Evaluations. Washington DC: World Bank. See also IEG (Independent Evaluation Group). 2010. The World Bank Group’s Response to the Global Economic Crises, Phase I. Washington, DC: World Bank; and IEG. 2012. The World Bank Group’s Response to the Global Economic Crisis, Phase II. Washington, DC: World Bank.
- An assessment of the degree to which knowledge on public expenditures informed the design of DPF operations found that public expenditure reviews or similar analytics informed most DPF operations in some way, but that the quality of integration of that knowledge into the DPF design varied, in part depending on the quality and length of the policy dialogue and World Bank engagement, and trust between the World Bank and the client government. The main areas that informed DPF operations were public sector governance, social development, and human development. Medium-term expenditure frameworks, budgeting, and public financial management were the most common issues.16World Bank. 2015. How Does Knowledge Integrate with the Design of Development Policy Operations? Washington, DC: World Bank.
- Policy-based lending in the environment sector, which has grown significantly since 2005, was used to pursue broader sectoral and multisector goals related to climate change and the environment. It was most effective “when policy issues are the main barrier to improving environmental outcomes, rather than capacity or other issues.”17World Bank. 2016. Lessons from Environmental Policy Lending. Washington, DC: World Bank. pp. x-xi.Clear theories of change and well-designed results frameworks, analytical work, and technical assistance were identified as important factors influencing design and outcomes, while monitoring and evaluation frameworks were often weak.
- An IEG empirical analysis of success factors in DPF operations in low-income countries found that “improving ‘relevance of design’ is key for achieving better DPF outcomes: it requires congruence between policies supported and project development objectives pursued.” This study also found evidence of analytical underpinnings, macro policies, and government ownership affecting the success of DPF operations. Interestingly, DPF operations with development partners using joint policy assessment frameworks have not been associated with better outcomes than other DPF operations with otherwise similar characteristics.18World Bank. 2018. Maximizing the Impact of Development Policy Financing in IDA Countries: A Stocktaking of Success Factors and Risks. IEG meso-evaluation. Washington, DC: World Bank. p. 7.
- An independent reassessment of the quality of macro-fiscal frameworks in DPF operations found that these frameworks were largely internally consistent and credible, noting some improvement over time. In many cases, the quality appeared to be related to the alignment with the International Monetary Fund and World Bank’s analytical work in the macro-fiscal area. At the same time, the assessment found weaknesses in the following areas: (i) the ambition of macro-fiscal frameworks in some stand-alone operations and in the links between objectives and fiscal measures, (ii) the credibility of the framework in view of the government’s track record, political economy factors, treatment of risks, or institutional fiscal rules, and (iii) the robustness of the debt sustainability analysis.19World Bank. 2015. Quality of Macroeconomic Frameworks in Development Policy Operations. Washington, DC: World Bank.
- 13World Bank. 2015. Quality of Results Frameworks in Development Policy Operations. Washington, DC: World Bank.
- 14World Bank. 2016. Findings from Evaluations of Policy-Based Guarantees. Washington, DC: World Bank.
- 15World Bank. 2017. Crisis Response and Resilience to Systemic Shocks: Lessons from IEG Evaluations. Washington DC: World Bank. See also IEG (Independent Evaluation Group). 2010. The World Bank Group’s Response to the Global Economic Crises, Phase I. Washington, DC: World Bank; and IEG. 2012. The World Bank Group’s Response to the Global Economic Crisis, Phase II. Washington, DC: World Bank.
- 16World Bank. 2015. How Does Knowledge Integrate with the Design of Development Policy Operations? Washington, DC: World Bank.
- 17World Bank. 2016. Lessons from Environmental Policy Lending. Washington, DC: World Bank. pp. x-xi.
- 18World Bank. 2018. Maximizing the Impact of Development Policy Financing in IDA Countries: A Stocktaking of Success Factors and Risks. IEG meso-evaluation. Washington, DC: World Bank. p. 7.
- 19World Bank. 2015. Quality of Macroeconomic Frameworks in Development Policy Operations. Washington, DC: World Bank.