ADB could have satisfied itself about project readiness in the key areas of detailed design, safeguards, procurement, and establishment of the project implementation unit.
A flexible approach could have been adopted with adequate supervision support for sector specialists in capacity development activities. Discussions at the technical level should have led to higherlevel discussion as the proposed reforms were far-reaching.
For significant impact, project investment should not be spread too thinly nor be so ambitious in scope as to hamper implementation. The project interventions, while generating some positive outcomes locally, did not show much impact at the provincial level because they were too small to produce a broader impact. Concentrating investment in a few provinces would have been more effective. Strategic investment choices based on poverty and health needs should have been made when deciding geographical coverage.
Indonesia still has a relatively high maternal mortality ratio. Capacity development in the areas of obstetric and neonatal care is still required, and the evaluation team came to know from the district health departments that they have limited finance to meet this capacity development requirement. According to key informants, basic emergency obstetric and neonatal care centers are often not used to assist with complications due to perceived lack of capability, and that not all districts have a functioning hospital with required emergency capability. Considerable further investment in infrastructure and human resource capacity will be required to meet the maternal mortality MDG target by 2015.
Inclusion of family planning advocacy in future programs could help reduce maternal mortality, as high fertility and maternal mortality are correlated. During key informant interviews, many expressed an opinion that more support for family planning is needed. It will require a coordinated and balanced joint approach from MOH and BKKBN, supported by appropriate resources.
Well-functioning provincial and district systems have proven to contribute to overall coordination and control of local health service delivery. During evaluation, health facility staff frequently cited the development of human resource capacity through longer-term fellowships as a key benefit of the project. Many of the project-supported fellows are now in key positions within the district and provincial health services. Many fellows thought that their careers would not have progressed so well without the project. The skills for networking, problem-solving, and data analysis and presentation that these fellowships help develop are still being used.
This was seen as a key positive feature of the project. Informants thought that by combining better planning and infrastructure and higher quality human resources, access to health services and their quality had improved. Many felt that without a comprehensive approach this might not have happened.
While the potential benefits of a comprehensive approach were noted during the evaluation team’s field visits, there is limited effort to quantify these benefits systematically. A survey of stakeholders was conducted at project exit and documented in a benefit-monitoring report. A range of customer satisfaction questions were posed and scored in the survey. Satisfaction indicators related to benefits associated with (i) training for midwives, (ii) utilization of skills by staff after training, (iii) availability and equipment to support MNCH, (iv) quality of civil works, (v) operations research, (vi) problem-solving ability of district health officers, (vii) staff motivation, (viii) political commitment, (ix) effectiveness of health reforms, (x) level of district funding (xi) health center utilization, (xii) safe delivery, and (xiii) patient satisfaction. However, no baseline survey was conducted with the satisfaction survey, so changes in with-and-without or before-and-after comparisons cannot be assessed. Similarly, no non-project facility appears to have been included in the survey. This would have helped in determining incremental benefits from project intervention compared with non-project situations.
To determine project impact, a focused survey of health status and client attitudes in project and non-project provinces should have been conducted at the start (to establish a baseline) and at project completion. Without this information it is not possible to attribute and quantify shorter-term project benefits.
A large number of staff in health service delivery received training or fellowships under the project. However, there has not been any attempt to document actual benefits stemming from this investment. A proper analysis will help the central, provincial, and district governments to determine skills gaps that can be targeted over the next couple of decades. Periodic evaluation of the value and impact of expenditure on capacity development should have been conducted. Although the number of physicians and the ratio of physicians to population have increased in all provinces and in rural areas, deployment practices and inequitable distribution remain serious concerns. M&E of current deployment and a skills audit are crucial to ascertain whether an effective long-term health workforce strategy is being implemented.
Problem analysis was undertaken to some degree during project preparations. Risks were outlined in the project framework (under Assumptions and Risks) but little detail was given in the main text of the report and recommendation of the President, or in the project design. Greater attention should have been placed on the political economy of decentralization, such as outlining key stakeholders, defining differential impacts of reforms, and identifying risks associated with possible future behavior of key stakeholders, along with capacity assessments across the broad geographic spread of the project. It would have been important to better assess the factors that help or hinder managerial and operational institutionalization of local health systems. The project did not do so. As a result, the standardized approach may not have worked in all districts. Decentralization is an evolving process and requires time to develop adequate capacity.
The health insurance scheme piloted in the Jembrana district of Bali has been recognized as one of the successful initiatives and is now being rolled out across the entire province. A number of lessons have emerged for ADB and the World Bank from piloting of this reform. This view was shared by people interviewed during the evaluation and by the World Bank, who stated that “more lessons could undoubtedly be drawn by undertaking a comparative study of all decentralization experiences, including those supported by development partners like the World Bank and ADB. Such a study could focus on the different ways provinces have established the province-district relationship, the relative costs and benefits of each, and the variety of ways districts and provinces have developed to exercise autonomy in the health sector despite continuing lack of clarity in the policy environment and their dependency on central government funding”.
The Commune Council Development Project has shown that a successful program has to recognize the social, political, administrative, and historical context of the country. In this case, Cambodia was just beginning its post-conflict stage, challenged by various problems such as lack of basic physical resources at the commune level and absence of administrative capacities of elected council officials. The design and implementation of the project took into consideration these local contexts.
The project was among the first ADB-funded projects that supported the construction of commune offices. ADB did not normally fund such activities but in this case supported it based on consultation with the concerned officials. A similar observation can be made for ADB’s support for civil registration. A variety of partnerships have been initiated under the project: (a) between and among the national government offices (MOI-DOLA and MOLMUPC) with the PTC playing a key role; ( b ) between the national government (specifically t h e MOI) and the subnational levels, such as the provinces, districts, and communes, with the national government providing the D&D framework and the communes operationalizing and localizing these, taking into consideration the specific context of the commune; (c) emerging partnerships between the government and nongovernment organizations and civil society organizations, specifically in information and public awareness, including the civil registration with the use of mobile teams by the MOI; and (d) between the government and development partners.
The project experience has shown that decentralization can usefully begin with developing basic capacities (human and physical). Capacity development is, however, a continuous process.
Teachers, community-based volunteers, monks, and even hospitals were tapped to influence more people to register. Also, the use of a private bank for disbursement for the construction of council offices proved efficient.
Agricultural support provided to dehkan farms was limited to the duration of the project. No mechanisms were put in place to ensure sustainability of benefits beyond the project’s life. The activities of demonstrations farms were very helpful to dehkan farms in increasing productivity, but these were stopped after project completion. Further, most of these demonstration farms were distributed to individual and family dehkan farms after completion of the project. Agricultural extension services are in the nature of merit goods, with positive externalities to the society. Hence mechanisms should have been established for the provision of these services on an ongoing basis to ensure sustainability of productivity gains.
Transferring management of tertiary irrigation and drainage systems to beneficiaries as a means for participatory water management was appropriate and in line with IWRM principles. However, the phasing of cost recovery and measures needed for WUAs to become sustainable should have been made clear at the outset. Measures for achieving full cost recovery should have been identified early in the implementation process.
While it was established as part of project management, M&E was discontinued after the PMO closed. Hence, project specific data cannot be provided for evaluating long-term outcomes, impact, and poverty reduction.
High staff turnover among ADB’s project officers and district governments tended to hamper effective communication and rapport-building among stakeholders. Consequently, this disrupted efforts to promote decentralized basic education management. Means of effective transfer and sharing of knowledge through, for instance, continued training are imperative. For ADB’s part, increasing the role of the Indonesia Resident Mission to include project management and monitoring responsibilities could have helped address project implementation issues. As discussed earlier, the project was managed directly from ADB’s headquarters in Manila. Moreover, ADB could have exerted greater efforts in disseminating knowledge products to further promote the sharing of lessons learned and best practices in decentralized basic management to other Indonesian provinces or countries similarly endeavoring to decentralize basic education delivery and management.
To address the structural misalignment between MONE and MORA, the role of provincial governments in decentralized basic education should be strengthened, particularly in monitoring and evaluation. The Indonesian government appears to be moving toward intensifying the delegation of authority to provincial governments in basic education. From 2014, provincial governments have full authority in preparing the materials for the primary school final examination and oversee the district government’s implementation of this examination, as stipulated in Ministerial Regulation No. 102/2013 and another regulation of the head of the MONE Research and Development Center. 101.
MONE has implemented the Dapodik online school database system. Schools are required regularly to submit data to this system. However, many schools do not have access to the internet to accomplish this task. School staff must also be capacitated to utilize this online database effectively. 102.
For the ADB-funded section of the road, the Lao PDR government could keep track of expenditures and verify if timelines and guidelines are followed. This was not the case with the PRC and Thailand financed sections. For example, the PRC used its own design standards for the road without conducting any dialogue with the Lao PDR government.
A mechanism needs to be devised for the costs to be shared in proportion to the benefits received by the stakeholders.
The IEM economic analysis indicates that the net present value of the project for the Lao PDR could be made positive if higher grant elements were used in the financing by the PRC and Thailand.