While the key urban sector and executing agency for this project, the Ministry of Urban Development (MUOD), adopted a structure for GESI mainstreaming with separate budget head, this initiative faced challenges after the country’s administrative restructuring. There were no division offices and GESI units in the districts, and the sociologist’s position that used to support GESI activities was removed from the departments. Nevertheless, using the resource materials developed under this project as guide, the GESI-trained staff in the divisions can push for establishing and institutionalizing GESI in the new structure of the provinces and at the local level. This would require revising MOUD’s GESI operational guidelines 2013 to define the roles and responsibilities in all tiers of government.
Neither the ADB project team nor project coordination office could fully comprehend the financial management requirements and financial covenant issues raised during this project’s review missions. As such, they were unable to follow up on audit opinions and recurrent issues, leading to the recurrence of the same issues and delays in submitting the audited project financial statements This improved only with the inclusion of a financial management staff toward the project’s end.
The initial design of this project was delayed and required modifications during implementation mainly due to the unavailability of information on existing underground utilities such as by plan profile and as-built drawings. Final designs were likewise not always comprehensive, necessitating variations for most contracts, resulting in both startup delays and contract modifications. In future, ADB should ensure that the scope of work of design consultants for urban development projects include an assessment of all existing utilities, including those underground. Also, that the consultants make every effort to meet their deliverables, comply with the agreed schedules and contract obligations, and respond to requests from client governments and ADB.
The financial management arrangements of the borrower and EA were robust. Separate project financial accounts were maintained and audited by statutory auditors. Except for fiscal year (FY) 2018, APFSs were received, albeit with delays up to 3.6 months from the due dates but within the grace period of 6 months. The APFS for FY2018 was rejected because it included a combined audit report for all three projects despite the requirement for separate reports and opinions. Also, the AEFSs for FY end-2017 and prior years were combined with the APFS. All non-compliances could have been mitigated with tighter monitoring from ADB.
The number of overhead water supply tanks built under the project was reduced from 10 to 8 because of poor contractor performance. But the outcome target of augmenting potable water supply by 23.5 million liters per day (MLD) was substantially achieved and reached 20 MLD with the installation of 19 tube wells and by replacing more pumps and other electro-mechanical equipment than targeted (148 actual against the 112 target). The adjustments also resulted in an additional 10,200 people in low-income or poor households (against the target of 3,800) benefiting from the increase in water supply. These accomplishments demonstrate how being outcome-oriented could lead to better results. Nevertheless, the design could have considered incorporating more comprehensive solutions, such as 24x7 water supply with O&M arrangements, into construction contracts. This would have maximized the benefits from the improved water supply systems and enhanced the sustainability of both the project benefits and assets.
The recruitment of a new project management consultant (PMC) and a new design and construction supervision consultant (DSC) for the MFF, although late, worked favorably for this project. The new PMC and DSC performed substantially better than their predecessors. However, the delivery of their services was hampered by site constraints, design changes, and delays in finalizing the drawings. In future, a realistic timeframe for consultant recruitment, detailed design development, and civil works contracts, should be ensured during loan preparation and appraisal.
The financial sustainability analysis conducted at MFF completion showed that there are enough state operating receipts to meet the O&M expenses of the project facilities. Given that the operating institutions did not achieve recovery of the O&M costs as envisaged at appraisal, fiscal transfers from the state and central governments need to continue to ensure the sustainability of the project assets.
The project’s target to have the municipalities adopt the accrual-based accounting system and publish their balance sheets from fiscal year 2015 were only partly achieved. The target to have semi-autonomous water supply entities prepare and adopt organizational development plans was achieved but with delay. So was the target to install a more efficient water billing and collection system that materialized only post-project completion. Therefore, it became evident that the reform targets were rather ambitious considering the state’s limited capacity and the local context, including frequent local unrest, which required a longer implementation period.
The overlapping implementation periods between project 1 and then-ongoing ADB Loan 2151 and other national programs, and concurrent preparation and appraisal of projects 2 and 3 of the MFF imposed a heavy burden on the ERA. Exacerbating this burden was the initially weak capacity of project implementation units (PIUs). ADB providing greater implementation support, particularly for the preparation of subsequent tranches, would help address this challenge in future MFFs. Such support would also help enhance the quality of subsequent tranches of the MFFs and mitigate the risk of implementation delays.
Safeguards implementation arrangement in the executing agency (EA) was adequate. A chief engineer was deputed as director of safeguards and supported by four environmental and resettlement experts. Two officers from the state revenue department were posted as land acquisition officers. These land acquisition officers provided much-needed support to the high-powered committee Divisional Level Committee established by the state government to fast track the implementation of the resettlement plans for subprojects under the project. Creating a land acquisition office in the PMU to manage unavoidable involuntary resettlement may be explored in ongoing and future projects.
Project 2 was completed with a minimal 3-month delay. This was attributable to, among other things, the preparation of better detailed designs concurrent with project 1 implementation.
The executing agency, the Economic Reconstruction Agency (ERA), had a functioning project management unit for another ongoing ADB loan at the time the MFF was approved, and the project loan agreement was signed. Given this, consultant recruitment for project 1 could have proceeded immediately. But for lack of staff and overload from multiple large projects, ERA did not proceed until after the loan became effective. As a result, it took 15 months to recruit the project management consultant (PMC) and one of the design and supervision consultants (DSCs), delaying project 1 startup. Advance action by ERA to facilitate consultant recruitment, detailed designs, and utility sharing would have helped minimize the startup delay and its knock-on effects on subsequent tranches and the whole investment program.
The evaluation mission’s difficulty in measuring the efficiency of water supply and road subprojects financed under MSDP and MSDP Phase 2 was the result of an inadequate PPMS that focused mainly on monitoring disbursements and outputs. Baseline data were not collected on the market served by water supply subprojects, such as the number of households, persons per household, water sources, amounts of water produced and consumed, and water loss rates. Nor were baseline data collected that would have enabled the efficiency of road subprojects to be calculated. These data were not collected when the subprojects were completed or for several years after completion for evaluation purposes. Water supply and road projects are similar in nature and ADB should develop a computerized template PPMS that records baseline data on implementation. This would provide the information needed to measure impacts and the efficiency of projects after project implementation. The template PPMS should be part of the project RRP and the loan agreement, modified as needed in consultation with the Independent Evaluation Department and the Economic Research and Regional Cooperation Department.
Discussions and interviews with MDF staff and municipal governments during the evaluation mission pointed to the need for further capacity building of MDF and municipal governments. This was one of the objectives of the projects. The projects should not have reallocated funds from the capacity development and training cost category to civil works even if other donors were providing similar assistance; ADB and other development partners could have coordinated closely to ensure that their training programs were complementary and did not duplicate each other. ADB could have used the funds to train local government staff, in addition to technical staff, consultants, and contractors on procurement processes, new road building techniques, and supervision. Arrangements to retain trained staff at least for the project duration (including incentives), could be included in future projects, ADB should continue investing in capacity building for local human resources (consultants, contractors, technical staff) to help ensure that central and local governments will have the skills to maintain the outputs after the project ends.
The absence of consultants to address issues regarding the legal and technical aspects of O&M planning and funding, measures for cost recovery, tariff collection, and the metering system posed a risk to the sustainability of the water supply and sanitation subprojects. Future projects should include technical assistance that would implemented concurrently with the project. This technical assistance would review the tariff, assess its adequacy to generate sufficient revenue to sustain water supply systems, and evaluate the status of metering consumers. In line with this recommendation, ADB is currently supporting UWSCG’s ongoing review of its water tariff policy under a subsequent TA and loan.
The projects were designed as financial intermediation loans largely because the government wanted to disburse funds and execute subprojects quickly. MDF had the capacity and had prior experience of implementing such loans through an earlier World Bank project. However, MDF operated like a regular executing agency rather than as a financial intermediary institution, whose role is limited to transferring or on-lending funds to municipal governments; assessing the financial soundness of the investment; and determining whether the borrowing entity has the technical, safeguard, and financial capacity to implement the project. ADB argued that because MDF was not a commercial financial institution and the borrowers were not commercial entities, it should not be subject to the same assessment criteria as a financial intermediary. This evaluation disagrees with the view. MDF’s status as a government special purpose vehicle should not exempt it from the requirement to build up capacity to mobilize domestic resources and become a true revolving fund. ADB’s approval of the government’s request to make the subproject allocations mainly grant-based, even for more affluent municipalities, prevented MDF from creating a revolving fund, hence it was not consistent with the expectation that MDF would improve its financial capacity. The RRP for the first project contained several statements to indicate this was an ambition, though perhaps not a central objective. Moreover, almost all the funding from MSDP Phase 2 was for secondary and tertiary roads which do not generate revenue. For any future urban projects using financial intermediation loans, institutional and capacity development, particularly with a view to transforming MDF into a sustainable financial institution, should be key objectives.
ADB had gained experience in Pakistan’s municipal services sector and these were recorded in past PCRs and country assistance program evaluations. Except for the successful Punjab Community Water Supply and Sanitation Project, available PCRs rated ADB’s interventions less than successful, including the recent experience with the Megacity Development Project in the Sindh province. Lessons from these experiences could have been applied to the design of this MFF.
It had clear, measurable indicators linked to each planned output and outcome that were critical for assessing both physical and institutional progress to provide some prognosis of sustainability.