Indonesia: Decentralized Health Services Project
sector: Health | country: Indonesia
Proper problem analysis is needed based on risk and mitigation analysis before rolling out a program under decentralization. Problem analysis was undertaken to some degree during project preparations. Risks were outlined in the project framework but little detail was given in the main text of the report and recommendation of the President, or in the project design. Greater attention should have been placed on the political economy of decentralization, such as outlining key stakeholders, defining differential impacts of reforms, and identifying risks associated with possible future behavior of key stakeholders, along with capacity assessments across the broad geographic spread of the project. It would have been important to better assess the factors that help or hinder managerial and operational institutionalization of local health systems. The project did not do so. As a result, the standardized approach may not have worked in all districts. Decentralization is an evolving process and requires time to develop adequate capacity.
Results from successful pilot initiatives should be disseminated in time and used in scaling up such initiatives to maximize the impact. The health insurance scheme piloted in the Jembrana district of Bali has been recognized as one of the successful initiatives and is now being rolled out across the entire province. A number of lessons have emerged for ADB and the World Bank from piloting of this reform. This view was shared by people interviewed during the evaluation and by the World Bank, who stated that ‘more lessons could undoubtedly be drawn by undertaking a comparative study of all decentralization experiences, including those supported by development partners like the World Bank and ADB. Such a study could focus on the different ways provinces have established the province-district relationship, the relative costs and benefits of each, and the variety of ways districts and provinces have developed to exercise autonomy in the health sector despite continuing lack of clarity in the policy environment and their dependency on central government funding’.
The scope and implementation coverage of projects should be manageable. For significant impact, project investment should not be spread too thinly nor be so ambitious in scope as to hamper implementation. The project interventions, while generating some positive outcomes locally, did not show much impact at the provincial level because they were too small to produce a broader impact. Concentrating investment in a few provinces would have been more effective. Strategic investment choices based on poverty and health needs should have been made when deciding geographical coverage.
Capacity development requires continued commitment. Indonesia still has a relatively high maternal mortality ratio. Capacity development in the areas of obstetric and neonatal care is still required, and the evaluation team came to know from the district health departments that they have limited finance to meet this capacity development requirement. According to key informants, basic emergency obstetric and neonatal care centers are often not used to assist with complications due to perceived lack of capability, and that not all districts have a functioning hospital with required emergency capability. Considerable further investment in infrastructure and human resource capacity will be required to meet the maternal mortality Millennium Development Goal (MDG) target by 2015.
Greater synergy is required between family planning and health-care services. Inclusion of family planning advocacy in future programs could help reduce maternal mortality, as high fertility and maternal mortality are correlated. During key informant interviews, many expressed an opinion that more support for family planning is needed. It will require a coordinated and balanced joint approach from Ministry of Health and National Population and Family Planning Board (Badan KoordinasiKeluarga Berencana Nasional), supported by appropriate resources.
A comprehensive approach is needed for a well-functioning health system. Well-functioning provincial and district systems have proven to contribute to overall coordination and control of local health service delivery. During evaluation, health facility staff frequently cited the development of human resource capacity through longer-term fellowships as a key benefit of the project. Many of the project-supported fellows are now in key positions within the district and provincial health services. Many fellows thought that their careers would not have progressed so well without the project. The skills for networking, problem-solving, and data analysis and presentation that these fellowships help develop are still being used.
ADB has helped improve infrastructure and promote reforms in a comprehensive manner. This was seen as a key positive feature of the project. Informants thought that by combining better planning and infrastructure and higher-quality human resources, access to health services and their quality had improved. Many felt that without a comprehensive approach this might not have happened.
A good monitoring and evaluation system can help develop a better understanding of what works and what does not. While the potential benefits of a comprehensive approach were noted during the evaluation team’s field visits, there is limited effort to quantify these benefits systematically. A survey of stakeholders was conducted at project exit and documented in a benefit-monitoring report. A range of customer satisfaction questions were posed and scored in the survey. Satisfaction indicators related to benefits associated with (i) training for midwives, (ii) utilization of skills by staff after training, (iii) availability and equipment to support maternal, newborn, and child health, (iv) quality of civil works, (v) operations research, (vi) problem-solving ability of district health officers, (vii) staff motivation, (viii) political commitment, (ix) effectiveness of health reforms, (x) level of district funding (xi) health center utilization, (xii) safe delivery, and (xiii) patient satisfaction. However, no baseline survey was conducted with the satisfaction survey, so changes in with-and-without or before-and-after comparisons cannot be assessed. Similarly, no non-project facility appears to have been included in the survey. This would have helped in determining incremental benefits from project intervention compared with non-project situations.
Generally accepted health systems strengthening indicators, such as those outlined in the World Health Organization’s handbook, should be included in a national and project-related monitoring and evaluation (M&E) system. To determine project impact, a focused survey of health status and client attitudes in project and non-project provinces should have been conducted at the start (to establish a baseline) and at project completion. Without this information it is not possible to attribute and quantify shorter-term project benefits.
Proper evaluation of capacity development would help assess the utility of current investment and justify further investment in human resource development. A large number of staff in health service delivery received training or fellowships under the project. However, there has not been any attempt to document actual benefits stemming from this investment. A proper analysis will help the central, provincial, and district governments to determine skills gaps that can be targeted over the next couple of decades. Periodic evaluation of the value and impact of expenditure on capacity development should have been conducted. Although the number of physicians and the ratio of physicians to population have increased in all provinces and in rural areas, deployment practices and inequitable distribution remain serious concerns. M&E of current deployment and a skills audit are crucial to ascertain whether an effective long-term health workforce strategy is being implemented.