In the case of the evaluated project, the decision to delay procurement of an off-the-shelf education sector information system (ESIS) and explore options to build the system resulted in an ESIS that is delivering lasting benefits to Mongolia— above and beyond government and ADB expectations. Although this decision delayed the project and impacted the efficiency of the investment, it contributed to a sound development outcome and a lasting impact.
ADB has been the lead development partner in the Mongolia education sector since 1996, and as such is well placed to support the government to recommence education sector development partner coordination.
ICT equipment, science laboratories, and other physical infrastructure provided under the project were not accompanied by ADB branding (plaques, stickers, etc.). This approach is inconsistent with current practice among development partners and diminishes the visibility of ADB’s positive contributions to development.
Examples of local knowledge innovations from this project that were new for ADB education investments include the translation of seminal English language texts on teacher education into local (Mongolian) language; and the national open source teacher development web portal set up to facilitate sharing of educational materials.
During implementation, the government was negotiating for additional financing. The PRC had committed to finance the Manhan Soum–Khovd road section, but cost increases required additional financing to undertake the Baga Ulaan–Manhan Soum section.
The government’s PCR was of poor quality and had no evaluative use. It contained data and some implementation issues and their resolution. No evaluation of project performance or its stakeholders were provided.
The government reviewed the prefeasibility and detailed design studies and accepted them, but the parameters of the review process were not given. During implementation, significant detailed design variations were made, and the construction cost increased. General price increases were not sufficiently considered in the cost estimates.
A 1-year time horizon was too short. Even though social welfare expenditures increased in the following years, a 5-year time horizon may have been more appropriate to encourage sustainability. Furthermore, a stronger M&E system is required for social welfare programs since output 2 focused on improving their targeting. Both the program and its related TA could have supported efforts to strengthen M&E. This would have enabled the government to better assess whether the improvements in targeting led to lower exclusion and inclusion of beneficiaries and to adjust programs accordingly.
The review could assess whether the targeting mechanisms adopted have ensured that the benefits go to the poor and vulnerable population or misused. Social welfare spillovers could result in suboptimal outcomes. Otherwise, a more sophisticated targeting approach could be undertaken.
The focus of the program was on the poor and vulnerable population and the social welfare assistance provided to them. The assessment of gender impacts of the policy reforms could have been undertaken, taking into account the gender and inclusion issues in the reform and consolidation of social welfare programs.
In situations and certain types of projects where there are expected changes in government and PIU staff, larger consultant inputs is warranted during strategic planning and implementation. Achieving institutional reforms always presents particular challenges and when there is a change of government during implementation, then it should be anticipated that progress is likely to be interrupted.
The PCR suggested that the lack of capacity to prepare bidding documents, evaluate bids, and monitor contract execution delayed project implementation. Technical assistance directed toward building such capacity, along with support in procuring technical equipment and materials (e.g., natural science laboratory equipment, didactic books for TTIs, and ICT learning objects) could have prevented procurement delays due to capacity constraints.
This can be supported, but not led, by an external consulting service. While this may have challenges due to staff capacity, it is ideal from the standpoint of ownership and may perhaps discourage the high staff turnover experienced in the past. Staff accountability can be strengthened through closer monitoring of its performance and the provision of staff development inputs based on needs.