Despite the improvement in PBL performance, assessing results at the country level and linking these with ADB PBL interventions is not straightforward. This section looks at country results related to public sector management, particularly public financial management, which was the primary focus of PBL-supported reforms over the evaluation period. The majority of PSM reforms for which a PVR was available were in Pakistan (7); Indonesia (6); Viet Nam (4); India, Mongolia, Nepal, the Philippines, and Tuvalu (3 each); and Armenia, Georgia, and Samoa (2 each) while 14 further countries had 1 PSM-related PVR each.

Public sector management. ADB PBL interventions in India, Indonesia, Nepal, Pakistan, the Philippines, and Viet Nam aimed to improve public financial management systems at national and subnational levels, on both the expenditure and revenue sides. Reform areas included debt management, strengthening audit capacity, fiscal consolidation, and budget management. The objective was often to build resilience to future economic shocks, improve public finance management, and strengthen overall macroeconomic management. A common feature of PSM-related reforms supported by ADB in India, Indonesia, Nepal, Pakistan, and the Philippines was decentralization and strengthening of state-level public finance management, e.g., in West Bengal and Assam in India, Sindh Province in Pakistan, and broader decentralization processes in Indonesia and the Philippines. ADB also supported initial efforts to strengthen local capacity in Nepal. Decentralization was a common element of reform efforts in the PSM sector in these countries but less so in the countries that had limited PBL and for which only one evaluation report was available, making an assessment of performance in these countries more difficult to make. Further evidence of country results was found in IED country assistance program evaluations (CAPEs), Public Expenditure and Financial Accountability (PEFA) reports, and ADB country performance assessments.39The Public Expenditure and Financial Accountability (PEFA) program was initiated in 2001 by the European Commission, International Monetary Fund, World Bank, and the governments of France, Norway, Switzerland, and the United Kingdom. PEFA established a standard methodology for PFM diagnostic assessments, which report on the strengths and weaknesses of public financial management (PFM). PEFA provides a snapshot of PFM performance at specific points in time using a methodology that can be replicated in successive assessments, giving a summary of changes over time.

India. Project and country field evaluation evidence indicated that PSM support was closely linked to country priorities and strong government ownership. The program aimed to tackle fiscal imbalances, which had led to underinvestment in infrastructure and the social sectors and poor basic services. The India CAPE in 2017 regarded ADB’s support as having been effective, as the major reform objectives and fiscal targets supported by the programs had largely been achieved.40IED. 2017. Country Assistance Program Evaluation: India, 2007–2015. Manila: ADB. However, several years after completion, some indicators, including capital expenditure levels, had regressed. The CAPE concluded that, given the state government’s commitment to achieving fiscal consolidation, the results of the West Bengal PSM program were likely to be sustained. However, it was too early to assess the likelihood of longer-term results, such as the impact that public financial management (PFM) reforms would have on improvements to service delivery. A similar PBL operation in the Indian state of Mizoram did not have the same impact. The PVR for this program noted that the targets were ambitious and the capacity to implement the agreed reforms may have been overestimated.41IED. 2017. Validation Report: Mizoram Public Resource Management Program in India. Manila: ADB.

Philippines. The most recent IED assessment of the program in the Philippines found that ADB had directly contributed to major policy and institutional reforms, improved PSM, and laid the foundations for more business investment.42IED. 2016. Validation Report: Country Partnership Strategy Final Review in the Philippines, 2011–2016. Manila: ADB. PSM support contributed to the government’s policy reform agenda and helped build capacity in revenue and public expenditure management, decentralization, and public–private partnerships (PPPs). The institutional strengthening for PPP was particularly evident, with a PPP center established, 80 improvements made to the legal framework, and a pipeline of PPPs established, which led to the implementation of infrastructure initiatives. ADB also supported a review of the Local Government Code, and at the time of the evaluation two bills focusing on local government revenue generation were before Congress. However, ADB’s support for the judiciary and court administration had not been sustained. This long-standing support had stagnated compared with the early 2000s. 

Indonesia. PBL supported the strengthening of audit functions, decentralization, public financial management, and public service delivery. ADB has been extensively involved in Indonesia’s decentralization reforms, with a focus on financial management systems in regional governments. Although over half of public expenditure in Indonesia is now undertaken at the subnational level, the process of decentralization has produced variable results in terms of increasing citizens’ access to local services. While there has been a general improvement and regional convergence in access to services, the quality remains poor and regional disparity is widening. Further reforms are needed to raise quality.43World Bank. 2017. Indonesia Economic Quarterly: Decentralization that Delivers, December. Jakarta. The World Bank’s 2011 PEFA44World Bank. 2012. Indonesia: Repeat Public Expenditure and Financial Accountability (PEFA) Report and Performance Indicators. Washington, DC. assessment noted steady progress in strengthening the quality of PFM systems in Indonesia between 2007 and 2011, with some reforms in progress supported by ADB, the World Bank and other development partners. 

Viet Nam. ADB’s PSM efforts were largely focused on restructuring state-owned enterprises and improving the business environment. ADB targeted financial restructuring and equitization of several of the country’s general corporations, which required substantial changes to laws, decrees, and regulations. These reform efforts have been largely successful.45IED. 2016. Validation Report: Viet Nam Country Partnership Strategy Final Review Validation, 2012–2015, Linked Document. Assessment of Public Sector Management. Manila: ADB. In 2017, the Ministry of Planning and Investment finalized a list of 375 state-owned enterprises to be wholly or partially divested by 2020.46E. Cook. 2017. Vietnam Gears up to Divest State Owned Enterprises in 2018. The Diplomat. Asia. 

Nepal. Initial PBL support for PSM reforms before 2012 was found to “have had modest results.”47IED. 2013. Validation Report: Country Partnership Strategy Final Review in Nepal, 2010–2012. Manila: ADB. However, a 2014 PEFA assessment suggested that Nepal had subsequently made substantial progress in deepening the structures and processes of PFM, particularly in the use of information technology. Investment efficiency gains had been achieved, despite a political transition period (2006–2010) during which reform was not a high priority.48World Bank. 2015. Public Expenditure and Financial Accountability (PEFA) Assessment Nepal PFM Second Performance Assessment as of FY2013-14 (English). Washington, DC World Bank Group. http//documents.worldbank.org/curated/en/483831467998500044/Public-expenditure-and-financial-accountability-PEFA-assessment-Nepal-PFM-second-performance-assessment-as-of-FY2013-14 This finding demonstrates that reform results may not be immediately visible, so longer-term monitoring outside the PBL timeframe is necessary. 

Pakistan. PBL operations in Pakistan at federal and provincial levels were also weighted toward reforms in public sector management. There were few lasting or major results, however, because of the difficult reform context. The PEFA report showed that there had been positive improvements but only in some areas.49SPBL was used for the first time in 2019, as part of an international financial package to provide balance of payments support for Pakistan. Consultations with government officials revealed that they saw the results of PBL more positively than the ADB evaluations, which had recorded very low success rates for all completed PBL operations over the evaluation period.50Pakistan Federal Government. 2012. Public Financial Management and Accountability Assessment. Islamabad. These officials felt that the incremental nature of the reforms that had arisen from numerous PBL operations was to be expected. The slow progress was at least partly the result of some sector reforms not being supported strongly enough by the government and its development partners. This was particularly the case for civil service reform and anticorruption initiatives, which were not directly tackled by PBL operations, or by national institutions. In this fraught political context, officials regarded incremental progress as sufficient justification for the programs. Overall, the PEFA showed that country systems for public financial management were improving, bearing in mind that improving PFM can take years, and that improvements to systems are not the same as improvements to public service delivery.

Number of PBL operations. ADB country performance assessments (CPAs) provide some evidence to suggest that, where ADB has provided five or more PBL operations, PSM and public finance management in countries eligible for concessional financing is improving. Scores for the country performance assessment (CPA) component (which measures the quality of governance and PSM) had improved in most countries eligible for concessional financing.51A country performance assessment assesses a country’s policy and institutional framework for promoting poverty reduction, sustainable growth, and how effectively it uses concessional assistance. Each country’s performance is based on a composite score constructed from the following: (i) an assessment of the quality of its macroeconomic management, (ii) the coherence of its structural policies, (iii) the degree to which its policies and institutions promote equity and inclusion, (iv) the quality of its governance and public sector management, and (v) performance of its concessional assistance project portfolio. The largest gains were in countries that had started from a relatively low base and received five or more PBL operations over the period, e.g., Cambodia and Lao People’s Democratic Republic. While reforms may be less effective once the quality of PSM has reached a certain level, countries with relatively well-developed systems that have received more PBL, e.g., Bhutan and Viet Nam, also achieved sizable positive changes. However, the evaluation found no relationship between PBL use and the overall CPA score. because there are other factors that influence this result which are exogenous to PBL. 

Capital market development. Countries that received significant support for capital market development had also achieved positive results. ADB has been a major partner of the government of Indonesia in the development of the financial sector that has taken place since the late 1990s. ADB programs contributed to results mainly in government bond markets, the Islamic capital market (sukuk), and the insurance sector. Similar results were found for Bangladesh, where ADB has supported capital market development since the late 1980s. In both cases, technical assistance also played a major role and it is difficult to determine whether reforms would have been undertaken in its absence. Nevertheless, since 2016, the capital market in Bangladesh has not developed to the extent envisaged and more reforms, including strengthening the market regulator, are needed to generate the desired longer-term development impact.

Transport, energy, and water. The PBL evaluation found less progress in PBL operations that supported reforms in transport, energy, and water—areas of significant ADB project investments. PBL supported energy sector reforms in the state of Assam in India,52ADB. 2003. Report and Recommendation of the President to the Board of Directors: Proposed Loans and Technical Assistance Grant to India for the Assam Power Sector Development Program. Manila. Pakistan,53ADB. 2000. Report and Recommendation of the President to the Board of Directors: Proposed Loans to the Islamic Republic of Pakistan for the Energy Sector Restructuring Program. Manila. the Philippines,54ADB. 2006. Report and Recommendation of the President to the Board of Directors: Proposed Program Cluster and Program Loan to the Republic of the Philippines for the Power Sector Development Program. Manila. and Sri Lanka,55ADB. 2002. Report and Recommendation of the President to the Board of Directors: Proposed Loans to Sri Lanka for the Power Sector Development Program. Manila. all of which have closed and been evaluated since 2008. Common issues included energy sector financing and political interference in pricing and supply. For example, the India country assistance program evaluation (CAPE) in 2017 found that, despite success in supporting the national electricity transmission and distribution network through project investments, cost recovery remained a concern (footnote 40). Support for energy sector reforms in Pakistan resulted in incremental improvements in the architecture, roles, and capacity of institutions, but the twin reforms of unbundling and privatization were incomplete.56ADB. 2015. Country Partnership Strategy: Pakistan, 2015–2019. Manila. Substantial progress was made in the Philippines in privatizing power generation and introducing wholesale competition, which reduced unsustainable subsidies to the sector.57IED. 2012. Validation Report: Power Sector Development Program in the Philippines. Manila: ADB.

Macroeconomic stability. Overall, country-level evidence suggests that PBL has contributed to macroeconomic stability and improved public financial management. It has helped reinforce macroeconomic performance and fiscal discipline, especially in crisis-affected countries. Budget support helped focus attention on public finance management and accountability processes, and more generally on broader PSM and governance issues. In most cases, significant progress was made in public financial management, as noted in PEFA assessments. There is also evidence that PBL provides a useful instrument around which development partner support can be better coordinated. However, the evaluation found less evidence to suggest that improvements were being carried through to improvements in human well-being, better service delivery and use of public goods and services, stronger governance, increased business confidence, or higher levels of investment and competitiveness, economic growth, and poverty reduction. This could be related to the weak theories of change that underlay PBL in PSM: ADB has not spelled out the relationship between the interventions and their impacts on the economy and society, nor are such impacts often recorded in PCRs. This could be because development outcomes are long-term and may not be observable at the time of program completion. These complexities apart, the fact is that, overall, assessing the results of PBL in ADB is constrained by design issues, such as a lack of baselines and the absence of counterfactuals (i.e., what would have happened without the policy reform?), as well as by constraints on collecting country data and developing statistical systems. The fact that many policy reforms are now completed before loan approval raises issues about the value added by PBL and how its results can be evaluated. More emphasis needs to be placed on design features, which explains why recent MDB evaluations have focused on this aspect. Strengthening the design of PBL will not only help improve its impact but also support the collection of evidence to document intended and unintended results, enabling ADB’s contribution to be better understood. For instance, PBL design depends on strong analytical foundations, the identification of quantifiable outcomes and indicators, and the selection of policy actions that are critical to the achievement of the development outcome. To measure the results of these policy actions, the PBL design needs to include a clear results framework that links policy actions with their intended results. The findings of this evaluation on design and monitoring issues are presented in the next section.

• 39The Public Expenditure and Financial Accountability (PEFA) program was initiated in 2001 by the European Commission, International Monetary Fund, World Bank, and the governments of France, Norway, Switzerland, and the United Kingdom. PEFA established a standard methodology for PFM diagnostic assessments, which report on the strengths and weaknesses of public financial management (PFM). PEFA provides a snapshot of PFM performance at specific points in time using a methodology that can be replicated in successive assessments, giving a summary of changes over time.
• 40IED. 2017. Country Assistance Program Evaluation: India, 2007–2015. Manila: ADB.
• 41IED. 2017. Validation Report: Mizoram Public Resource Management Program in India. Manila: ADB.
• 42IED. 2016. Validation Report: Country Partnership Strategy Final Review in the Philippines, 2011–2016. Manila: ADB.
• 43World Bank. 2017. Indonesia Economic Quarterly: Decentralization that Delivers, December. Jakarta.
• 44World Bank. 2012. Indonesia: Repeat Public Expenditure and Financial Accountability (PEFA) Report and Performance Indicators. Washington, DC.
• 45IED. 2016. Validation Report: Viet Nam Country Partnership Strategy Final Review Validation, 2012–2015, Linked Document. Assessment of Public Sector Management. Manila: ADB.
• 46E. Cook. 2017. Vietnam Gears up to Divest State Owned Enterprises in 2018. The Diplomat. Asia.
• 47IED. 2013. Validation Report: Country Partnership Strategy Final Review in Nepal, 2010–2012. Manila: ADB.
• 48World Bank. 2015. Public Expenditure and Financial Accountability (PEFA) Assessment Nepal PFM Second Performance Assessment as of FY2013-14 (English). Washington, DC World Bank Group. http//documents.worldbank.org/curated/en/483831467998500044/Public-expenditure-and-financial-accountability-PEFA-assessment-Nepal-PFM-second-performance-assessment-as-of-FY2013-14
• 49SPBL was used for the first time in 2019, as part of an international financial package to provide balance of payments support for Pakistan.
• 50Pakistan Federal Government. 2012. Public Financial Management and Accountability Assessment. Islamabad.
• 51A country performance assessment assesses a country’s policy and institutional framework for promoting poverty reduction, sustainable growth, and how effectively it uses concessional assistance. Each country’s performance is based on a composite score constructed from the following: (i) an assessment of the quality of its macroeconomic management, (ii) the coherence of its structural policies, (iii) the degree to which its policies and institutions promote equity and inclusion, (iv) the quality of its governance and public sector management, and (v) performance of its concessional assistance project portfolio.
• 52ADB. 2003. Report and Recommendation of the President to the Board of Directors: Proposed Loans and Technical Assistance Grant to India for the Assam Power Sector Development Program. Manila.
• 53ADB. 2000. Report and Recommendation of the President to the Board of Directors: Proposed Loans to the Islamic Republic of Pakistan for the Energy Sector Restructuring Program. Manila.
• 54ADB. 2006. Report and Recommendation of the President to the Board of Directors: Proposed Program Cluster and Program Loan to the Republic of the Philippines for the Power Sector Development Program. Manila.
• 55ADB. 2002. Report and Recommendation of the President to the Board of Directors: Proposed Loans to Sri Lanka for the Power Sector Development Program. Manila.
• 56ADB. 2015. Country Partnership Strategy: Pakistan, 2015–2019. Manila.
• 57IED. 2012. Validation Report: Power Sector Development Program in the Philippines. Manila: ADB.