A large number of policy actions under PSDP were not sufficiently defined to ensure or help ascertain the achievement of meaningful results. Examples include conditionality related to balanced budgets, wage differentials, or the adoption of new laws which did not convey the underlying principles that ADB expected to be realized and contributed to differences in understanding between ADB and the various governments regarding the nature of expected reform actions.
Sufficient resources and time should have been allocated for consultation with implementing agencies in determining program inputs, outputs, and outcomes, as well as in setting and monitoring performance targets. To be effective, ADB must ensure that systems to monitor the achievement of performance targets are in place or are established in time. State government offices could not provide complete sets of foreign investment data. National government offices could not provide data on secured transactions, registered businesses or enterprise performance. Land administrations did not have reliable data on the land titles or leases.
Decentralized practices among the four states (Chuuk, Kosrae, Pohnpei, and Yap) make coordination and communication on policy issues a challenge. This challenge was compounded by the weak capacity of various implementing agencies and different views and levels of political commitment regarding the reform agenda. These resulted in varying degrees of performance. Through state-specific policy conditionality and project components, program design could have been adjusted for the uneven capacities, needs, and levels of political buy-in within the four states. While it can be argued that the uniform approach adopted by ADB was reasonable in light of resource constraints and efficiency concerns, it also needs to be considered that potential for synergies was limited and not sufficiently identified.
From the start, the IT components for the land management administrations and FSMDB were not properly scoped and resourced. They did not adequately consider the limited capacity of internal IT departments and local IT providers. They also failed to address such related problems as the need to provide implementation support throughout the first year of operation. Although project records are patchy and likely do not reflect the full extent of ADB’s involvement, it appears that ADB relied on short-term consultants to determine IT systems and software requirements and to supervise software and hardware providers. Given the lack of specialized staff expertise in the Regional Department, the complexity of these project components, and the low capacity within the supported agencies, proactive consultation with ADB’s Office of Information Systems and Technology during the project design phase might have resulted in more appropriate IT solutions. Also useful could have been to provide specialized IT expertise within the PIU, more proactive follow-up during implementation to check the functionality of procured hardware and software, and special arrangements for post-implementation support. Furthermore, it would be helpful if the Regional and Sustainable Development Department together with the Office of Information Systems and Technology can collect and disseminate information about experience across ADB with relevant IT applications. While IT projects can be particularly useful for small, remote economies, inherent IT capacity issues need to be proactively addressed.
The effort to reform and divest public enterprises has not been as successful as hoped. One reason is that in such a small and remote lower-income country with a weak private sector, there are few people with the capital or business experience to bid for or successfully manage such enterprises.
The unexpected delays in obtaining national and state legislative authorization delayed the loan approval and loan effectiveness, each by 1 year, respectively. Reform momentum was lost during the long interval between appraisal and program inception.
The Yap State Public Service Corporation (YSPSC), the executing agency for this project, explained that its lack of an internal auditor was due to its small size, and that the role was being performed by the accounting manager. This arrangement does not promote the integrity of internal control processes and points to an inadequacy in YSPSC financial policies. To address these issues in future projects, small utilities should be encouraged to adopt alternative approaches that will not cause them additional overhead. Such approaches can include engaging external auditors to conduct periodic internal audits in addition to annual financial audits.
The project envisaged that the executing agency (EA) would employ a wind turbine maintenance and training contractor for the first 3 years of operation. This was a good feature of the project design, considering the complexity of wind energy system operation and maintenance, and the EA’s lack of experience. While the pertinent covenant was only partially complied with, the experience has highlighted the importance of incorporating system maintenance in the design of future wind projects especially in cases where borrower capacity for this is weak.
The implementation of this project’s GAP was unsuccessful because of unrealistic targets and activities. Limited data were also available on the GAP activities and achievements under the project. Future projects can do better by conducting a gender analysis to inform GAP preparation and ensure that the GAP targets and activities are realistic and feasible. GAP implementation should be monitored, with support provided to the project teams, to ensure the delivery of planned results.
Procurement of the wind, diesel, and integration and control system components of this project used a turnkey approach, while the solar component used a split contract approach that included 1 main goods supply package, 2 balance of plant packages, and the executing agency undertaking the construction activities. It is unclear whether the split contract approach reduced overall costs, but it did delay implementation. The idea of using a split contract approach in future projects should consider the increased risk of delays it poses.
The design and supervision consultant recommended prequalification for the integration and control system and wind packages. Each took about 5 months, but the wind prequalification failed with no applicant meeting the specified criteria and with procurement proceeding through the post-qualification procedure. To avoid a repeat of the wind prequalification experience, which only delayed procurement, the need for prequalification should be assessed during project preparation and considered in the project schedule.
The project preparatory technical assistance report addressed the risks associated with poor wind data by proposing a phased approach to the wind component. During inception, the design and supervision consultant conducted detailed measurements, which firmed up wind data assumptions and revealed the site restrictions in the number of turbines that may be installed. Only 3 of the planned 5 turbines were consequently put in place because of these restrictions. To avoid changes in scope, future projects should ensure the suitability of the projects sites during the preparation of wind energy projects.
The issue of medium-speed versus high-speed diesel generators was addressed at a quantitative level in the report prepared by ADB’s project preparation consultants who recommended the use of medium-speed units. Noting an unreferenced 2010 report by a consultant firm that had recommended high-speed units, the design and supervision consultant (DSC) conducted system modeling and likewise recommended the use of high-speed generators. The project proceeded with high-speed units, which meant that the planned improvement in gross generator efficiencies did not materialize even though net efficiencies, after parasitic load is considered, have improved. The DSC therefore effectively sought to achieve the project objective of increasing energy security by reducing fuel consumption rather than improving efficiency, and the project exceeded fuel reduction targets. Detailed system modelling during project preparation would have identified these issues and enabled a more accurate projection of the outcomes. (Preparation: energy projects, renewable vs. conventional energy projects, high-speed vs. low-speed diesel generators)
This project’s actual solar output was 20% below target even while the projections were made at a conservative 90% confidence level. The reasons for the shortfall are unclear. However, the fact that a concurrent solar project—with a different designer, contractor, equipment, and mounting arrangement—had the same performance level indicates a common issue, possibly linked to solar irradiation data gathering. Possible inaccuracies in solar irradiation data collection and projections need to be handled better in future solar projects financed by ADB.
Recommendations from the project preparatory technical assistance provided by ADB and the ones developed by the design and supervision consultant (DSC) during the inception phase differed on the wind and diesel components. The DSC recommended reducing the total installed wind capacity and increasing the installed capacity and changing the speed of the diesel generators. It also added a new component, an integration and control system. While the project inception report prepared by the DSC included updated costs and performance, there appears to have been no changes on the calculations made during appraisal of the project’s economic and financial returns. Revised calculations would have allowed the executing agency to make more knowledgeable decisions on the costs and benefits of the options available.