The success of the pilot project approach provides a means to improve access to agricultural support services across PNG. More could have been done to capture the learning from the pilot activities and extend them nationwide in PNG. With replication of the pilot project and further devolution to provincial and district DAL's institutions, programs, and budgets, there is potential to improve agricultural quality, quantity, and local economic development. In this regard, the government needs to consider whether to take a strategic approach to private sector involvement in agriculture by adopting the project’s approach in all provinces and districts as was proposed in the NADP, or whether the lessons learned will remain within the pilot provinces.
The project’s use of contracting out as a new approach to the delivery of support services proved successful. It was relevant, effective, efficient, and benefits were mostly sustained. The contracting approach has improved the access of smallholder farmers to agricultural support services and has strengthened the institutional capacities of the national and provincial DALs and support services providers in implementing contractual service delivery. The approach has gained acceptance with and beyond the project stakeholders. Other provinces and development agencies have already replicated the support services provider model.
Prior to the pilot project, agricultural service provision suffered from bureaucratic delays that were not appropriate to the seasonal and market requirements for agricultural development. The trust fund modality allowed an accountable and responsive mechanism that is being continued and valued by the provincial DALs and service providers.
The PRAP process was widely accepted by the targeted population. It was an effective mechanism for identifying required services but also served to identify local development needs. These included the formation of producer groups for higher level production, consolidation, and marketing. In addition, it led to broader economic support by local leaders for other requirements such as infrastructure and access to inputs. The demand-driven approach to planning identified important elements of success that were not originally in the project design; for instance, the demand for training proved to be a mixture of technical and financial training.
This evaluation identified that post-project impact included additional benefits that had emerged from the catalytic effect of the pilot project’s initiatives. The introduction of a contracting mechanism generated capacity within the provincial government to engage more with the private sector. This resulted in other initiatives such as trade fairs for local producers, use of service providers for other agriculture sector activities, and use of improved recruitment processes in other sectors. For service providers, many had not considered use of their skills within the private sector for service provision, e.g., past provincial DAL officers, lead farmers, etc. The opportunities under the pilot project led them to start a new enterprise, marketing their skills both independently and through the SSPA. For individual farmers, the training, particularly in post-harvest and enterprise skills, led to participants expanding and diversifying their economic activities for both agricultural and non-agricultural enterprises.
Strict compliance with loan covenants pertaining to financial management systems, reporting, and project coordination meetings should have been the priority concern after loan effectiveness.
Staff movements to the newly created state-owned enterprises affected the implementation process. NAC had to recruit new supervisors for the civil works contracts due to the institutional changes, which required some period for the new staff to familiarize themselves with their new responsibilities in the PIU.
It is contingent on the resident mission to assist the executing agency in identifying corrective measures and even use its technical resources to assist in resolving issues. The delay due to unresolved resettlement issues took too long to solve resulting in increased project cost. With local knowledge and experience, the resident mission would be in a good position to identify options in resolving such issues.
First, the PPMS should be developed and established prior to the start of project implementation. This would ensure that all implementation activities are effectively monitored and causes of delays and mitigating actions described clearly. Benefit monitoring and evaluation is essential to measuring project impact and outcome indicators so as to measure achievement. This should not be left to the executing or implementing agency and should be undertaken by qualified monitoring and evaluation specialists, including the collection of primary and secondary data.
While specific technical support and management will remain necessary at the central level to guarantee quality and effectiveness of the program interventions, implementation at the local level would yield better results when integrated into PHC, whenever possible.
While the objective of this standardization was to accelerate the signing of MOAs, it had, in fact, an opposite result, i.e., with the MOA not considering the local conditions, constraints and needs at the onset, the MOAs were not always satisfactory, causing delays in signing. Sector level lessons. On the one hand, working with enclave operators and civil society organizations can, under some circumstances, be a cost-effective option to expand and improve health services delivery in rural areas. The project provided useful information on how to engage with the private sector (i.e., economic operators and not-for-profit civil society organizations, especially the churches in PNG). On the other hand, while the NDOH enthusiastically supported a partnership with the economic operators in the rural development enclaves, there appeared to have been a lack of project ownership by the local provincial and district authorities, at least at the beginning of the project. With decentralization, it is important to confirm the local governments’ representatives’ involvement in project preparation and their commitment.
Especially in rural areas, supervision is also required to support often isolated local staff and helps to retain health-workers. Civil works successfully rehabilitated health facilities under the project, but regular supervision of public health facilities by health staff of the economic operators, as agreed in the MOAs, were not sufficiently sustained.
Although the Papua New Guinea Resident Mission (PNRM) was formally established in 2003, which was almost midway in the project’s implementation, it was not indicated in the PCR whether PNRM support was sought for project supervision. PNRM support would have been helpful to expedite the consultant selection approval process and the award of contracts, and to ensure completion of the training program.
While there is evidence that consequent road project processing and implementation by ADB has improved considerably, future ADB assistance to the PNG road sector needs to further improve, incorporating the lessons learned from this project. One such lesson shows the critical importance of delegating project administration to the PNRM as early as possible. Delegation of this project to the PNRM facilitated closer communication and monitoring and supervision, resulting in the steady improvement of the implementation process as well as the capacity of the Department of Works (DOW), the project EA. The monthly review meetings held with the DOW, which proved instrumental in enhancing capacity and performance, could be made mandatory for projects experiencing significant implementation and disbursement delays.
Following the cancellation of the contracting process for the design and supervision consultants, their functions were merged with those of the implementation support consultants. This suggests the lack of a firm basis for the division of responsibility between the two types of consultants, a weakness that should be avoided in future similar projects. When the need for separating these two consultant roles is firmly embedded in the project design, the design and supervision consultants should be recruited first to ensure the delivery of the necessary technical inputs for the finalization and timely implementation of the road subprojects.
The project recorded an economic internal rate of return (EIRR) of 14.9%, above the 12% estimate for the original loan and below the 18.8% estimate for the supplementary loans. The economic benefits arise from: (i) savings in vehicle operating costs, particularly repair costs, because of improved road conditions; (ii) savings in maintenance costs; (iii) time savings for road users; and (iv) benefits to generated traffic. The reappraisal sensitivity analysis indicated that the base case EIRR would be reduced to 12% and the economic present value to zero if the road subprojects benefits were reduced by 15% or the project costs increased by 22%, suggesting that the project is not robust. The most likely cause of the reduced benefits is failure to provide regular maintenance, indicating that project results could be problematic if adequate maintenance activities are not adhered to over a prolonged period. Consequent ADB funding to PNG road projects has increasingly focused on maintenance funding, with limited results. While exploring covenanting the amount of maintenance funding in the loan agreement, ADB should move in concert with development partners to ensure that the government commits to adequate levels of maintenance funding in future donor-financed road projects in the PNG.
Consultant recruitment diverged significantly from the implementation schedule approved for the original project, which was optimistic, given the lengthy procurement process in PNG. Recruitment of the implementation consultants was completed 9 months after loan effectiveness, and the full team was mobilized only around 21 months after loan effectiveness. Recruitment of the design and supervision consultants was worse: it commenced only 29 months after loan effectiveness and 21 months after ADB approved the first road subproject, and was eventually cancelled on the argument that, during that stage, most road subprojects under the original project had already been designed and contracts had been awarded. For civil works, the use of the prequalification procedure in 11 of the 13 ADB-financed contracts under the original project exacerbated the delays caused by the absence of advance action and slow government approval processes. The prequalification of contractors was discontinued under the supplementary loans, and contractors with adequate technical and financial capacity were permitted to bid for several packages and awarded more than one contract based on the least-cost combination of contracts. Besides carefully considering the knowledge and experience of EAs in ADB procurement procedures and the government approval processes, ADB should more strongly assert the need for advance procurement actions to minimize startup and implementation delays in future road projects. The state of the local contracting industry and the capacity of local contractors should also be assessed to determine the ways they can be maximized and assisted under the project.
The original project did not identify the road subprojects, a weakness that may have resulted from the use of the sector loan modality, which requires the economic appraisal only of a single road subproject during loan processing. The weakness was rectified during the appraisal of the two supplementary loans; however, the nearly 7-year lapse in the adoption of corrective measures was a bit too long not to have a significant impact on project implementation. In fact, the project completion review mission averred that, considering the difficulties associated with small highland road subprojects, initial delays to contracting and disbursement may be rooted in the failure to identify and complete the economic appraisal of subprojects during project formulation. To avoid a repeat of this experience, future projects using the sector loan modality should incorporate in the project preparation process a rigorous assessment of EA capacity to evaluate the technical, financial, economic, safeguards and governance aspects of subprojects.
The government’s capacity and willingness to provide counterpart funding was not adequately assessed, and neither were mechanisms established to ensure that counterpart funding commitments were met. Significant shortfalls in counterpart funding, occurring throughout the project period, contributed to excessive implementation and loan disbursement delays. As the provision of counterpart funding could remain an issue in future road projects in PNG, ADB can consider making the availability of counterpart funds a condition for ADB loan disbursement. ADB should also make sure that counterpart funds are kept in a separate project account from that of ADB loan disbursements to ensure the tractability of both, a concern that was never addressed under the project’s financial management system.
The delays encountered by the project during the first 3 years of implementation could not be corrected due to the lack of consistent and timely project review and oversight from ADB. During this time, the project was managed from Manila and only short review missions, once annually, were conducted. As a result, civil works progress fell behind targets and disbursements were constantly below expectations. During the next 3 years, 2004−2007, while the project was still managed from Manila, the ADB review missions increased to 2−3 times annually and were held for longer periods. This resulted in significant improvements in implementation progress and loan disbursement. Although the project was not able to catch up with the planned implementation periods for its 3 loans, transfer of the loan administration to the Papua New Guinea (PNG) Resident Mission in January 2009 accelerated the resolution of key implementation issues. The original loan for the project closed 4 years behind schedule, while the 2 additional financing loans closed 3 years later than envisaged.
The Department of Works, the executing agency for the investment program and the implementing agency for road improvement works, found it hard to find qualified national consultants to supervise and help implement the road improvement component of this project. It thus had to rely on international consultants to do the job. No major issues arose concerning the performance of consultants. However, integrating international consultants, who were also in previous ADB projects, in an advisory role may have worked better for the transfer of skills and capacity development of local staff.
The government’s prolonged procurement approval process, which includes a layer of the National Executive Council, delayed the contract awards for this project. The project procurement-related reviews undertaken by the Asian Development Bank (ADB) revealed weak supervision, and monitoring of procurement processes. Aggravated by poor contract management, weaknesses in civil works procurement produced ineffectual results. In future, ADB and PNG counterparts should coordinate more closely to strengthen compliance to ADB procurement guidelines and ensure timely government approval of procurement requirements of ADB-assisted projects. ADB should take extra effort to assist PNG counterparts in preparing bid documents and facilitating procurement which, although an explicit component of this project, seemed to have been inadequately attended to.