For the envisaged impact to be fully achieved, the government should be encouraged to engage with ADB and other development partners. The IDP housing that the evaluation team visited was intact and in good condition, based on a cursory inspection. Schools, banks, and stores, among other support facilities, have since been established and appear to be functioning well. A more in-depth assessment of the resettlement sites would help to determine the overall success of the resettlement program. It would also confirm the status of the safeguard issues that were not previously considered under the program. A comprehensive analysis of development constraints should also be undertaken to identify binding constraints on economic growth that the government and ADB can focus on, given the active presence of other development partners in the country. This could provide a sound basis for a more strategic support framework moving forward.
The stakeholders met by the evaluation mission indicated that before the 2008 conflict Georgia’s economy was relatively well-managed by the government. The government had a good track record with development partners and this was instrumental in securing substantial support after the twin crises. Development partners interviewed during the evaluation mission noted that negotiations with government for the post-crisis support went smoothly, despite the fact that the government had to deal with multi-donor support delivered through numerous support modalities and channeled through many stand-alone assistance schemes with varying timelines, execution structures and delivery methods. For ADB’s GRSP, the reforms and policy actions stipulated under the loan (and in the JNA) were complied with by the government, more quickly than expected.
The 2008 conflict caused the development partners to engage collectively with the government in designing the crisis response and policy discussions. The NBG said this collective engagement encouraged the government to discuss the crisis response measures and policy actions seriously. As this was an emergency situation, the discussions, negotiations, approvals, and transfers happened quickly. The World Bank and the United Nations Development Programme took the lead in convening the development partners and ADB actively participated, although it did not play a lead role. The collective effort from the development partners helped to alleviate the financial crisis, limited economic contraction, and addressed constraints arising from infrastructure damage, investment climate deterioration, financial difficulties, and worsening social conditions of IDPs.
However, the regional TA was not implemented and the envisaged fiduciary arrangements were no longer monitored under the GRSP. Nevertheless, other development partners have taken related initiatives. The World Bank, Department for International Development of the United Kingdom, Government of the Netherlands, and the Swedish International Development Cooperation Agency provided $15.00 million for public sector financial management reform, the United States Agency for International Development provided $3.29 million to support foreign reserves and payment systems, and the European Union provided €11 million to support a second phase of public financial management reform. The completion report for the World Bank project noted that the MOF had strengthened its strategy and annual budget planning processes and improved arrangements for monitoring and oversight of budget implementation through capacity building provided by the project. Capacity and procedures for strategic expenditure planning, budget preparation, and budget management were also strengthened in line ministries. An internal budget policy and public expenditure review process was introduced and capacity building and training strategy for staff in the MOF Budget Department and line ministry budget units were developed and implemented. The World Bank project also supported an external audit of central government spending. This suggests that the coordination among the development partners under the JNA framework worked well for Georgia.
The absence of consultants to address issues regarding the legal and technical aspects of O&M planning and funding, measures for cost recovery, tariff collection, and the metering system posed a risk to the sustainability of the water supply and sanitation subprojects. Future projects should include technical assistance that would implemented concurrently with the project. This technical assistance would review the tariff, assess its adequacy to generate sufficient revenue to sustain water supply systems, and evaluate the status of metering consumers. In line with this recommendation, ADB is currently supporting UWSCG’s ongoing review of its water tariff policy under a subsequent TA and loan.
The projects were designed as financial intermediation loans largely because the government wanted to disburse funds and execute subprojects quickly. MDF had the capacity and had prior experience of implementing such loans through an earlier World Bank project. However, MDF operated like a regular executing agency rather than as a financial intermediary institution, whose role is limited to transferring or on-lending funds to municipal governments; assessing the financial soundness of the investment; and determining whether the borrowing entity has the technical, safeguard, and financial capacity to implement the project. ADB argued that because MDF was not a commercial financial institution and the borrowers were not commercial entities, it should not be subject to the same assessment criteria as a financial intermediary. This evaluation disagrees with the view. MDF’s status as a government special purpose vehicle should not exempt it from the requirement to build up capacity to mobilize domestic resources and become a true revolving fund. ADB’s approval of the government’s request to make the subproject allocations mainly grant-based, even for more affluent municipalities, prevented MDF from creating a revolving fund, hence it was not consistent with the expectation that MDF would improve its financial capacity. The RRP for the first project contained several statements to indicate this was an ambition, though perhaps not a central objective. Moreover, almost all the funding from MSDP Phase 2 was for secondary and tertiary roads which do not generate revenue. For any future urban projects using financial intermediation loans, institutional and capacity development, particularly with a view to transforming MDF into a sustainable financial institution, should be key objectives.
The evaluation mission’s difficulty in measuring the efficiency of water supply and road subprojects financed under MSDP and MSDP Phase 2 was the result of an inadequate PPMS that focused mainly on monitoring disbursements and outputs. Baseline data were not collected on the market served by water supply subprojects, such as the number of households, persons per household, water sources, amounts of water produced and consumed, and water loss rates. Nor were baseline data collected that would have enabled the efficiency of road subprojects to be calculated. These data were not collected when the subprojects were completed or for several years after completion for evaluation purposes. Water supply and road projects are similar in nature and ADB should develop a computerized template PPMS that records baseline data on implementation. This would provide the information needed to measure impacts and the efficiency of projects after project implementation. The template PPMS should be part of the project RRP and the loan agreement, modified as needed in consultation with the Independent Evaluation Department and the Economic Research and Regional Cooperation Department.
Discussions and interviews with MDF staff and municipal governments during the evaluation mission pointed to the need for further capacity building of MDF and municipal governments. This was one of the objectives of the projects. The projects should not have reallocated funds from the capacity development and training cost category to civil works even if other donors were providing similar assistance; ADB and other development partners could have coordinated closely to ensure that their training programs were complementary and did not duplicate each other. ADB could have used the funds to train local government staff, in addition to technical staff, consultants, and contractors on procurement processes, new road building techniques, and supervision. Arrangements to retain trained staff at least for the project duration (including incentives), could be included in future projects, ADB should continue investing in capacity building for local human resources (consultants, contractors, technical staff) to help ensure that central and local governments will have the skills to maintain the outputs after the project ends.
Private savings through pensions and a more developed capital market will mobilize long-term finance. These reforms, however, are long-term in nature involving legislative changes and institutional strengthening. Capacity building is required to assist government in this endeavor, and to support more knowledge-intensive solutions and analytical work since these sectors are likely to be underdeveloped.
The DMF was reasonably well-designed and the performance indicators—with the exception of the initial program impact indicators—targets, and baseline data were precise and generally easily measurable. However, the PCR assessed the program against the DMF prepared for subprogram 3, which, like subprogram 2 had no indicators to measure impact, although these were included for subprogram 1. The impact indicators under subprogram 1 were not reflective of the impact statement—measuring the reduction in the fiscal deficit, absolute poverty, and income distribution against the impact of higher living standards and more employment opportunities, particularly for women. The impact statement and its measurement need to be more closely aligned to the program and as such, indicated a design deficiency in the DMF. The risks for subprogram 1 related to a deteriorating external environment and delays to reforms, and weak economic growth that impact expenditure efficiency; weak economic growth was not included for subprograms 2 and 3.
This could, for example, be undertaken by a market supervisor or regulator who checks market power.
Also, consideration must be given to include in contracts the benefits from power transit and their associated market risks in neighboring countries and investment. This is in view of GSE being unable to manage the risk if the revenue from power transit is linked only to the power flow. Furthermore, it is suggested that discussions be held with the regulatory commission to identify measures in protecting GSE from market risks associated with power transit.
The effectiveness of public expenditure can be improved by maintaining higher quality standards for services.
The MDF maintained an efficient management and information system throughout the project period, and monitoring system reports were updated every 6 months. This enabled the institution not only to initiate timely management actions but also to comply with ADB reporting and documentation requirements. Environmental monitoring reports were satisfactorily prepared and included in the project progress reports. The MDF promptly submitted to ADB all progress reports, audited financial statements, and other documents in accordance with ADB guidelines and project administration procedures. The management information system created by the MDF can be replicated in other projects in Georgia.
As of March 2013, all the subproject facilities were functioning and being maintained relatively satisfactorily. The O&M of municipal roads is contracted out to local contractors for annual maintenance and repair works under the supervision of municipal governments. However, the budgetary allocations and personnel of municipal governments for road maintenance remained insufficient and needed to be increased. The water supply systems were being satisfactorily operated by the United Water Supply Company of Georgia (UWSCG). However, while it has regional and municipal branch offices, UWSCG capacity for local-level O&M appears limited, with only operators and technicians but no managers, engineers, or specialists. Furthermore, no municipal government has an existing asset inventory and database, which is essential for planning the O&M and management of municipal infrastructure and services. These glaring inadequacies highlight the need for future similar projects to plan the O&M of project facilities more carefully and comprehensively. Legal and technical issues regarding, inter alia, O&M implementation and funding, cost recovery schemes, tariff collection, and metering system should be thoroughly studied as part of project preparation. A comprehensive plan to ensure the adequate O&M and sustainability of project facilities and mitigate risks should be embedded into the loan covenants and other legal agreements, as appropriate.
The project was envisaged to include technical assistance to help participating municipal governments identify their urban priorities; undertake planning and project feasibility studies; and improve charges, accounting, financial management, and reporting systems. It also aimed to provide institutional strengthening and capacity development to the municipalities to ensure that project benefits are sustainable. However, no consultant was engaged to deliver these services. Instead, trained MDF staff visited the participating municipal governments and provided on-the-job trainings, particularly in the preparation of subproject feasibility reports and detailed designs, and in construction supervision. All the participating municipal governments were involved in these on-the-job training activities. However, as these activities consisted of the usual project management tasks, they fell short of delivering the full scope of capacity development envisaged at appraisal. The conduct of full-blown training courses for municipal government staff would have been more beneficial.
This project’s construction supervision activities were mostly carried out by municipal government engineers and technical staff, supported by the MDF engineers who visited the subproject sites at least monthly. While this practice effectively ensured the timely completion of almost all the subprojects, some shortfalls were observed in the engineering design of the road subprojects, for example, with respect to drainage and ditch cover, sidewalk, and traffic safety provision. Poor quality of works was also noted in concreting and curb installation on some subprojects. These weaknesses may be attributable to the limited experience and knowledge of the municipal governments and MDF staff, as well as their consultants and contractors, of the international standards and global trends in road engineering design and technologies. To address these weaknesses, technical trainings on modern road construction and technologies should be incorporated in future road projects in the country. MDB and ADB also need to ensure that similar future projects would benefit from the timely and dedicated services of highly qualified international construction supervision consultants whose terms of reference should include skills transfer and the development of relevant knowledge and guidance materials.
This project was carried out through the Municipal Development Fund (MDF) established by a presidential decree in Georgia to enable municipal governments to avail of soft loans and grants for municipal infrastructure and services development. Proactive consultations by the MDF with municipal governments to ensure prompt selection of subprojects created a momentum that was maintained throughout project implementation and contributed to the completion of physical works ahead of schedule. The MDF and participating municipal governments also ensured that the results of social assessments and consultations with civil society and other stakeholders were considered in identifying investment priorities.
As the newly built 500 kV switchyard at Marneuli substation uses a technology that is relatively new to the country, technicians will require new skills to ensure its proper O&M. Therefore, further onsite training is needed. Extension of the defects liability period to ensure the sustainability of the project assets should also be explored. (Implementation, Completion: energy projects, new technologies energy projects, O&M new technologies energy projects)
The $48 million loan approved by ADB for the project was meant to finance 2 turnkey and 2 consultancy contracts. During implementation, the number of turnkey contracts was increased to 4 and that for consulting services was reduced to 1. By 2014, all five contracts had been awarded, totaling $49.85 million equivalent. However, because of exchange rate fluctuations, the loan’s dollar equivalent fell to $45.52 million. In 2016, the dollar equivalent dropped further to $43.852 million. The financing gap was covered in part by the executing agency, the Georgian State Electrosystem, and the reallocation of the funds originally intended to support the cancelled study on potential hydropower investments. Drawing on this experience, it is important for future cost estimates to include to the extent possible contingencies to meet foreign exchange fluctuations.
The project demonstrated that the GSE is highly capable of acting as executing agency for power projects and has the capacity to undertake major construction projects related to augmentation and rehabilitation of power transmission networks. Nonetheless, ADB should ensure that GSE sufficiently understands its procurement, safeguards, and project administration guidelines and procedures. ADB-sponsored training and workshops on these areas will benefit future projects to be executed by the GSE in terms of timely procurement, better safeguards compliance, and overall improved performance.
Slow-moving and complex reforms such as pension and capital market reforms require simultaneous efforts, including (i) broad stakeholder consultations and public awareness building, (ii) policy formulation and legal drafting, and (iii) institution building, which need to be coordinated under a firm schedule and consistent top-level stewardship. While adherence to the sequence and timing of reforms is essential, flexibility is required to manage bottlenecks. The programmatic approach in policy-based lending, combined with TA, is a suitable modality for such interventions. Policy triggers are necessary to maintain proper sequencing of reforms and intermittent expert support should be provided to enhance the diagnostics and implementation of reforms.
The program covered a wide range of reforms that could only be completed successfully with top government ownership and proper diagnostics. Diagnostics for risk management and public financial management benefited from analytical work prepared by the International Monetary Fund and the World Bank and supplemented by technical assistance (TA) from the Asian Development Bank (ADB). Diagnostics for the pension and capital market reforms were undertaken under the ADB TA.