The combination of programmatic policy-based assistance and TA support provided under this initiative proved to be effective and should be continued. The lack of qualified and experienced local consultants had been a recurring issue, and the Ministry of Health (MOH), the executing agency, continued to face a shortage of sanctioned staff. By giving consultant support to key departments of MOH, the TA contributed to knowledge transfer and capacity building for these departments, and to stronger coordination across the MOH and other ministries.
Effective implementation of system reforms, given their long-term horizon, requires phased support from development partners. The multiple tranche arrangement employed by this program ensured satisfactory progression as a precondition for continued ADB assistance and strengthened the government’s accountability for the reforms.
The policy actions pursued under this program were developed in an open, participatory manner. They were fully aligned with the country’s Health Sector Reform Strategy and Health Sector Development Plan. The participatory development and full alignment of the reform program with the country sector strategy and development plan ensured strong and broad ownership and commitment from the government, and with the assistance of development partners, strengthened policy dialogue and coordination.
Well-functioning provincial and district systems have proven to contribute to overall coordination and control of local health service delivery. During evaluation, health facility staff frequently cited the development of human resource capacity through longer-term fellowships as a key benefit of the project. Many of the project-supported fellows are now in key positions within the district and provincial health services. Many fellows thought that their careers would not have progressed so well without the project. The skills for networking, problem-solving, and data analysis and presentation that these fellowships help develop are still being used.
This was seen as a key positive feature of the project. Informants thought that by combining better planning and infrastructure and higher quality human resources, access to health services and their quality had improved. Many felt that without a comprehensive approach this might not have happened.
While the potential benefits of a comprehensive approach were noted during the evaluation team’s field visits, there is limited effort to quantify these benefits systematically. A survey of stakeholders was conducted at project exit and documented in a benefit-monitoring report. A range of customer satisfaction questions were posed and scored in the survey. Satisfaction indicators related to benefits associated with (i) training for midwives, (ii) utilization of skills by staff after training, (iii) availability and equipment to support MNCH, (iv) quality of civil works, (v) operations research, (vi) problem-solving ability of district health officers, (vii) staff motivation, (viii) political commitment, (ix) effectiveness of health reforms, (x) level of district funding (xi) health center utilization, (xii) safe delivery, and (xiii) patient satisfaction. However, no baseline survey was conducted with the satisfaction survey, so changes in with-and-without or before-and-after comparisons cannot be assessed. Similarly, no non-project facility appears to have been included in the survey. This would have helped in determining incremental benefits from project intervention compared with non-project situations.
To determine project impact, a focused survey of health status and client attitudes in project and non-project provinces should have been conducted at the start (to establish a baseline) and at project completion. Without this information it is not possible to attribute and quantify shorter-term project benefits.
A large number of staff in health service delivery received training or fellowships under the project. However, there has not been any attempt to document actual benefits stemming from this investment. A proper analysis will help the central, provincial, and district governments to determine skills gaps that can be targeted over the next couple of decades. Periodic evaluation of the value and impact of expenditure on capacity development should have been conducted. Although the number of physicians and the ratio of physicians to population have increased in all provinces and in rural areas, deployment practices and inequitable distribution remain serious concerns. M&E of current deployment and a skills audit are crucial to ascertain whether an effective long-term health workforce strategy is being implemented.
Problem analysis was undertaken to some degree during project preparations. Risks were outlined in the project framework (under Assumptions and Risks) but little detail was given in the main text of the report and recommendation of the President, or in the project design. Greater attention should have been placed on the political economy of decentralization, such as outlining key stakeholders, defining differential impacts of reforms, and identifying risks associated with possible future behavior of key stakeholders, along with capacity assessments across the broad geographic spread of the project. It would have been important to better assess the factors that help or hinder managerial and operational institutionalization of local health systems. The project did not do so. As a result, the standardized approach may not have worked in all districts. Decentralization is an evolving process and requires time to develop adequate capacity.
The health insurance scheme piloted in the Jembrana district of Bali has been recognized as one of the successful initiatives and is now being rolled out across the entire province. A number of lessons have emerged for ADB and the World Bank from piloting of this reform. This view was shared by people interviewed during the evaluation and by the World Bank, who stated that “more lessons could undoubtedly be drawn by undertaking a comparative study of all decentralization experiences, including those supported by development partners like the World Bank and ADB. Such a study could focus on the different ways provinces have established the province-district relationship, the relative costs and benefits of each, and the variety of ways districts and provinces have developed to exercise autonomy in the health sector despite continuing lack of clarity in the policy environment and their dependency on central government funding”.
For significant impact, project investment should not be spread too thinly nor be so ambitious in scope as to hamper implementation. The project interventions, while generating some positive outcomes locally, did not show much impact at the provincial level because they were too small to produce a broader impact. Concentrating investment in a few provinces would have been more effective. Strategic investment choices based on poverty and health needs should have been made when deciding geographical coverage.
Indonesia still has a relatively high maternal mortality ratio. Capacity development in the areas of obstetric and neonatal care is still required, and the evaluation team came to know from the district health departments that they have limited finance to meet this capacity development requirement. According to key informants, basic emergency obstetric and neonatal care centers are often not used to assist with complications due to perceived lack of capability, and that not all districts have a functioning hospital with required emergency capability. Considerable further investment in infrastructure and human resource capacity will be required to meet the maternal mortality MDG target by 2015.
Inclusion of family planning advocacy in future programs could help reduce maternal mortality, as high fertility and maternal mortality are correlated. During key informant interviews, many expressed an opinion that more support for family planning is needed. It will require a coordinated and balanced joint approach from MOH and BKKBN, supported by appropriate resources.
The needs of private sector entities and small-scale health providers, however, are different. Easy access to credit by small-scale providers, in itself, is insufficient. Needs and constraints also should be addressed. This covers (i) technical support—referral networks, continuous training, and colleagues’ advice; and (ii) family support—access to schools for children, work opportunities for the spouse, safety, and other needs and constraints. These must be identified and addressed to develop quality health services in rural areas. Among small-scale health providers, midwives are essential to reduce maternal and child mortality in underserved areas, and their specific needs must be identified and met. Innovative solutions are required to facilitate access to small credit without multiplying intermediaries—DBP, learning from experience, has now developed specific credit programs. PPPs, in the health sector must offer value for money. The involvement of the private sector often improves efficiency, the quality of services, and peers’ support. A PPP policy in the health sector must be designed within the context of local conditions. Outsourcing ancillary services is good practice for improving efficiency.