Decentralized practices among the four states (Chuuk, Kosrae, Pohnpei, and Yap) make coordination and communication on policy issues a challenge. This challenge was compounded by the weak capacity of various implementing agencies and different views and levels of political commitment regarding the reform agenda. These resulted in varying degrees of performance. Through state-specific policy conditionality and project components, program design could have been adjusted for the uneven capacities, needs, and levels of political buy-in within the four states. While it can be argued that the uniform approach adopted by ADB was reasonable in light of resource constraints and efficiency concerns, it also needs to be considered that potential for synergies was limited and not sufficiently identified.
From the start, the IT components for the land management administrations and FSMDB were not properly scoped and resourced. They did not adequately consider the limited capacity of internal IT departments and local IT providers. They also failed to address such related problems as the need to provide implementation support throughout the first year of operation. Although project records are patchy and likely do not reflect the full extent of ADB’s involvement, it appears that ADB relied on short-term consultants to determine IT systems and software requirements and to supervise software and hardware providers. Given the lack of specialized staff expertise in the Regional Department, the complexity of these project components, and the low capacity within the supported agencies, proactive consultation with ADB’s Office of Information Systems and Technology during the project design phase might have resulted in more appropriate IT solutions. Also useful could have been to provide specialized IT expertise within the PIU, more proactive follow-up during implementation to check the functionality of procured hardware and software, and special arrangements for post-implementation support. Furthermore, it would be helpful if the Regional and Sustainable Development Department together with the Office of Information Systems and Technology can collect and disseminate information about experience across ADB with relevant IT applications. While IT projects can be particularly useful for small, remote economies, inherent IT capacity issues need to be proactively addressed.
The effort to reform and divest public enterprises has not been as successful as hoped. One reason is that in such a small and remote lower-income country with a weak private sector, there are few people with the capital or business experience to bid for or successfully manage such enterprises.
The unexpected delays in obtaining national and state legislative authorization delayed the loan approval and loan effectiveness, each by 1 year, respectively. Reform momentum was lost during the long interval between appraisal and program inception.
A large number of policy actions under PSDP were not sufficiently defined to ensure or help ascertain the achievement of meaningful results. Examples include conditionality related to balanced budgets, wage differentials, or the adoption of new laws which did not convey the underlying principles that ADB expected to be realized and contributed to differences in understanding between ADB and the various governments regarding the nature of expected reform actions.
Sufficient resources and time should have been allocated for consultation with implementing agencies in determining program inputs, outputs, and outcomes, as well as in setting and monitoring performance targets. To be effective, ADB must ensure that systems to monitor the achievement of performance targets are in place or are established in time. State government offices could not provide complete sets of foreign investment data. National government offices could not provide data on secured transactions, registered businesses or enterprise performance. Land administrations did not have reliable data on the land titles or leases.
The project was able to deliver benefits to the project area through the inclusion of both the provincial highway upgrade and the rural link roads in the project design. The socioeconomic benefits of these roads were very important, allowing all-weather access to jobs and services (health care, transport). Local tourists use the rural roads to travel to scenic areas, a further source of economic development as well as of support for maintaining the environment. The HPDT advised that it is frequently difficult to gain funding for rural roads because they are of lower priority than the expressways and highways, and ADB support for these projects provided valuable social impacts.
This project was the third ADB road project in Heilongjiang (and a fourth is currently underway). The HPDT has extensive experience in road design and construction, knowledge of ADB practices and requirements, and clear boundaries for its responsibilities. Although international practices can be usefully included in projects, they do not always work well in the local context. With nearly 20 years of experience with the ADB now part of the institutional memory of the agencies involved in these projects, it is likely there are well-qualified local resources available to lead the PPTA studies and produce project designs that are streamlined to deliver the project in the local context and in compliance with ADB requirements.
The difference between the budget and actual cost of supervision and training for the project indicates that the project required substantially more supervision than planned. The HPDT attributes this greater cost, at least in part, to the selection of the low-cost bidders for portions of the civil works. The PCR states “four contracts [of the 26] (procured under international competitive bidding) were awarded on an exceptional basis [due to the very low prices] without determining the combination of bids offering the lowest evaluated cost.” Although these contracts were subjected to higher performance securities, such bids rarely include adequate management and contingency budgets. Therefore, to ensure appropriate quality, greater executing agency supervision and monitoring during construction is required. Comparisons of bids on the basis of established provincial reference prices and the expected quality of higher cost bids should be undertaken before contract awarding to ensure low-cost bidders can deliver civil works of the appropriate quality. The award process should be strictly followed, with no exceptions being made for the low-cost bids. This is likely to result in a better project implementation process within the expected supervisory budget.
These factors ensured that any actual or potential problems were quickly and effectively resolved.
This continuity, together with stability in personnel capacity, was an important factor in the success of the project. While this lesson may be self-evident, it is not easily achieved and is often overlooked.
Knowledge of the project design and implementation that was transferred from ADB to the government was highly valued, and officials expressed the view that this was more important than the loan financing.
The MFF enabled sequential construction and financing requirements that were funded by separate tranches, thus limiting unnecessary duplication of administrative effort and approvals.
Because counterpart government funding was timely, there were no delays and the project was completed early.
ADB’s long-term focus on energy sector development in India has supported all aspects of the improving energy situation in the country. Work done by ADB has contributed to policy changes, new infrastructure, and increased private sector participation in the delivery of electricity to end-users. Established knowledge of the local situation, and particular focus on renewable energy enabled ADB to respond quickly to the policy changes supporting solar power and provide timely support to an important client to deliver the first utility-scale solar PV plant in India. This evaluation recognizes the importance of continued sector focus to allow ADB to continue to contribute its sector and technology knowledge in evolving economic situations.
The hedging policy approved at appraisal did not reflect the foreign exchange hedging instruments available in the local market. When exchange rates deteriorated, the hedging policy and the required sponsor support needed to be renegotiated. It is fairly usual to have rolling foreign exchange hedging requirements, rather than the life-of-loan program that was originally documented, and better investigation of the likely options that would be available to the project could have reduced the work required to adapt to the situation as it arose.
The DSPP entity undertook non-operational decisions relating to the company name and accounting practices without gaining the required approvals from the lenders before implementing the decisions. While these matters have been satisfactorily explained subsequently, they raise concerns for lenders seeking to confirm that the project entity is being managed in accordance with the requirements of the loan documents; significant time is expended to work through these issues as they are discovered. While the plant has performed above expectations, and ESHS matters related to the operations appear to be monitored closely, the nonoperational matters are equally important. This evaluation recognizes that efforts have been made to improve communication on these matters, and seeks to reinforce their importance.
RPL is a strong company with extensive resources, enabling it to quickly identify appropriate technology, supervise quality construction, and be a good corporate partner within the communities it serves. To meet the requirements of CERC’s tariff regulations, RPL needed to develop and construct a project in a short period of time that would withstand the local climate conditions. The project’s performance above expectations is due to RPL’s decisions about technology, design, and construction quality. This evaluation recognizes the importance of financing projects that are implemented by sponsors qualified to manage them.
Effective champions are critical for the success of any new initiative. This is even more true for innovative projects, where strong commitment to the project on the part of both the government and ADB are critical to success. The failure to locate the DEMP within a government agency with strong ownership of the project was one of the factors that led ultimately to its failure. The sustainability of initiatives beyond the lifetime of the project rests heavily on the commitment and support provided by the relevant ministry, in the form of formal institutionalization of systems and processes, further training, and regular follow up by relevant staff. Similarly, the failure of ADB to adequately invest time and resources in project implementation, and the failure to remain involved in ensuring network sustainability in the years after the project ended, when its fate remained in limbo, also contributed to the project’s limited success.
The DEMP failed to expand tertiary education opportunities and increase the employability of graduates to the extent intended. The question of how to achieve both of these objectives remains. A central issue in this regard is how to improve external degrees. One of the lessons of the DEMP is that Sri Lankan students may not yet be ready for fully online courses, at least for the longer degree courses. To raise the quality and range of offerings to external students, a blended learning approach would be most appropriate, combining online learning with a limited amount of face-to-face tutoring. This would allow the introduction of science and engineering as external degrees, currently not offered. Some universities have established their own open and distance learning centers, and these could be considered for future ADB assistance, along with support for further capacity expansion at the OUSL.