EUROPEAN COMMISSION EUROPEAID CO-OPERATION OFFICE ASIA LATIN AMERICA (ALA)

EVALUATION REPORT

TIMOR-LESTE INTERIM EVALUATION OF THE TRUST FUND FOR TIMOR-LESTE (TFET)

OCTOBER 2004

Chuong N. Phung Jean Franois Bauer

D INVESTMENT DEVELOPMENT CONSULTANCY (I C )

CONTENTS
PAGE

EXECUTIVE SUMMARY CHAPTER I: CONTEXT I II III IV POLITICAL CONTEXT DONORS RESPONSE A. Joint Assessment Mission B. The EC Response FUNDING ARRANGEMENTS OBJECTIVES OF THE EVALUATION CHAPTER II: DESIGN AND OPERATION OF THE TFET I II III IV V VI THE MANDATE LEGAL BASIS FOR WORLD BANK ASSISTANCE ESTABLISHMENT AND DESIGN GOVERNANCE ADMINISTRATION AND PROJECT PROCESSING PROCEDURES RESOURCES AND USES CHAPTER III: EVALUATION I RELEVANCE: WAS TFET THE RIGHT MECHANISM? A. Resource Mobilization B. Fragmentation of Funding Mechanisms C. Relationships Between Main Responsible Institutions UNTAET and the World Bank ADB and the World Bank. D. Donor Coordination E. Donor Involvement F. Responsiveness to Beneficiary Countrys Needs G. Relevance of the TFET Projects H. Alternatives to TFET Bilateral financing World Bank-managed trust fund with the EC as the only donor Financing through UN Agencies EC managed multi-donor trust fund? EFFECTIVENESS: RESULTS ON THE GROUND A. TFET Portfolio B. Project Processing C. Composition of the Portfolio D. Portfolio Performance E. Disbursement Performance F. Implementation Period

5 18 18 20 20 21 24 24 26 26 26 26 27 28 29 32 32 32 34 34 34 36 36 37 39 39 42 42 43 43 43 44 44 46 47 48 49 51

II

-1-

III

IV

G. Procurement Performance Cost Effectiveness of Procurement Sources of Procurement Fraud and Corruption H. Management of the Reconstruction Programme I. Cross-Cutting Issues Credit and Grants Community Involvement Technical Assistance and Consulting Services Gender, Land Acquisition and Environment J. Major Project Achievements EFFICIENCY: ACHIEVEMENTS AND TRADEOFFS A. Importance of Security B. Achievements Macroeconomic Achievements. Establishing a Functioning Administration C. Capacity Building D. Addressing Policy/Sector Issues and Moving from Reconstruction to Development Do no Harm Establishing Sector Institutions and Moving Beyond Reconstruction. SUSTAINABILITY: THE ROAD AHEAD A. Reconstruction to Sustainable Development B. Sustainability of TFET-Financed Projects C. Ownership of the Reconstruction Programme CHAPTER IV: CONCLUSIONS AND RECOMMENDATIONS A. Overall Evaluation B. EC Performance C. TFET as a Mechanism for Post-Conflict Assistance

52 53 53 53 54 56 56 58 60 62 63 65 85 65 65 66 67 69 69 69 71 71 74 75 78 78 80 80

Annex 1: Annex 2: Annex 3: Annex 4: Annex 5: Annex 6: Annex 7:

Case Study: Rural Development DAC Summary Terms of Reference of the Evaluation Methodology Company Presentation Mr. Phungs CV Mr. Bauers CV

-2-

CURRENCY EQUIVALENTS Currency Unit = US Dollar WEIGHTS AND MEASURES: Metric System FISCAL YEAR: July 1 June 30 ABBREVIATIONS AND ACRONYMS ADB ALAT ANAO ARP AusAID CDD CEP CFET CNRT EC ECHO EICBP EIRP ERA ERL ESRP EU ETTA FAO FSQP GDP GEF GoTL ICR IDA ILO IMF IMFTL INTERFET IOM JAM JICA LIL MAFF MDTF NCB NCC NDP NGO Asian Development Bank Agent Local dAssistance Technique Australian National Audit Office Agriculture Rehabilitation Project Australian Aid Agency Community Driven Development Community Empowerment Project Consolidated Fund for Timor-Leste Conselho Nacional da Resistncia Timorense European Commission European Commission Humanitarian Office Economic Institutions Capacity Building Project Emergency Infrastructure Recovery Project Emergency Recovery Assistance Emergency Recovery Loan Emergency School Readiness Project European Union Timor-Leste Transition Authority Food and Agriculture Organization Fundamental School Quality Project Gross Domestic Product Global Environment Fund Government of Timor-Leste Implementation Completion Report International Development Association International Labour Organisation International Monetary Fund Institute for Microfinance of Timor-Leste International Force for Timor-Leste International Organisation for Migration Joint Assessment Mission Japan International Cooperation Agency Learning and Innovative Loan Ministry of Agriculture, Fisheries and Forestry Multi-Donor Trust Fund National Competitive Bidding National Consultative Council National Development Plan Non-Government Organization

-3-

OCHA PAD PCF PCR PID PMU QCBS RVP SEP SIP SHRDP SWAp TA TFET TM TSP UN UNDP UNICEF UNIFEM UNMISET UNOPS UNPKO UNPA UNTAET USAID VLW WSS

Office for the Coordination of Humanitarian Affairs Project Appraisal Documentation Post-Conflict Fund Project Completion Report Project Information Document Project Management Unit Quality Cost Based Selection Regional Vice President Small Enterprise Project Sector Investment Programme Second Health Rehabilitation and Development Project Sectorwide Approaches Technical Assistance Trust Fund for Timor-Leste Task Manager Transition Support Programme United Nations United Nations Development Programme United Nations Childrens Fund United Nations Development Fund for Women United Nations Mission Support in Timor-Leste United Nations Office of Project Services United Nations Peace keeping Operations (Department) United Nations Political Affairs (Department) United Nations Transitional Authority in Timor-Leste United States Agency for International Development Village Livestock Workers Water Supply and Sanitation

-4-

INTERIM EVALUATION OF THE TRUST FUND FOR EAST TIMOR

La critique est aise, et lart est difficile Destouches, 1680-1754

EXECUTIVE SUMMARY

RESOURCES AND FINANCING ARRANGEMENTS 1. Following 25 years of occupation by Indonesia, on August 30th 1999, the Timorese cast their votes in favour of independence in a referendum supervised by the United Nations by a 4-1 margin. The ballot results set off an orchestrated campaign of violence across the territory, which resulted in the destruction of an estimated 70% of social infrastructure and private buildings and the displacement of over two thirds of the population. State and administrative structures in the country suffered a complete collapse. 2. The international communitys response to the violence and destruction of September 1999 was swift. In Tokyo, the donors pledged in December 1999 a total of US$520 million in assistance to Timor-Leste over three years, in addition to about US$700 million per annum reserved for UNTAET from the UN assessed contribution budget. 3. The Tokyo meeting also endorsed the creation of two trust funds: (i) the Consolidated Fund for Timor-Leste (CFET) to be managed by UNTAET for recurrent expenditures, civil service and capacity building; and (ii) the Trust Fund for Timor-Leste (TFET) for economic and reconstruction activities under the trusteeship of the World Bank. The Asian Development Bank was invited to participate as co-implementer and managed projects in infrastructure, water supply and sanitation and microfinance. 4. However, some bilateral donors preferred to finance Timor-Leste reconstruction outside of the TFET framework, while the different UN agencies (UNDP, UNICEF, etc) and NGOs continued to channel their assistance directly to Timor-Leste. 5. There were thus six different aid mechanisms when the reconstruction work started in 2000: (i) the UNTAET-administered CFET mainly covering recurrent expenditures; (ii) the TFET with two implementing agencies (the World Bank and the ADB) financing investments; (iii) the assessed contribution budget of UNTAET financing salaries of UN staff; (iv) projects financed by UN agencies; (v) assistance provided directly by bilateral donors; and (vi) support and work done by or through NGOs. During TFET implementation, a seventh financing mechanism was added in May 2002: TSP (Transition Support Programme) through which the World Bank and other donors helped finance the governments current budget (CFET) after Timor-Leste became independent.

-5-

CONTEXT 6. The context in which the Timor-Leste reconstruction programme has been implemented is particular and not all of the lessons learned here would be applicable to other countries. That context was characterized by (i) a complete collapse of state structures, accompanied by widespread destruction of infrastructure and disruption of livelihoods; (ii) extremely low capacity due to the departure of virtually all middle and higher-level managers (who were mostly Indonesians) following the referendum, as well as 80% of the secondary school teachers and the majority of medical doctors; (iii) rapid and peaceful political transition to a democratic state, with good internal security, based on a broad national consensus; (iv) lack of clarity on Timorese representation in decision-making during the earlier stages of the transition; (v) four different government structures in the span of three years, including UNTAET, which had a dual role as the legal governing authority of TimorLeste and as a major participant in the reconstruction process; and (vi) a complex aid architecture and rigid division of work between the implementing institutions with UNTAET in charge of building up the public administration and strengthening institutional capacity and the two development banks (World Bank and Asian Development Bank) responsible for reconstruction activities. 7. The country did not go through a prolonged and divisive period of civil war like other post-conflict countries. Peace was rapidly restored after the violence and this created a favourable climate for reconstruction, based on internal security that continued throughout the reconstruction period. EUROPEAN COMMISSIONS RESPONSE 8. The EC was present in Timor-Leste throughout the crisis. Beyond large-scale provision of humanitarian aid amounting to 48.94 million, it participated in the JAM (Joint Assessment Mission) and was among the largest contributors to TFET, with 56.4 million, provided through two Financing Decisions (IDN/B7-3040/IB/2000/00 for 40 million in 2000 and ET/AIDCO/2001/0292 for 16.4 million in 2001). 9. The main reasons for the ECs participation in TFET were (i) better donor coordination; (ii) higher efficiency resulting from the use of uniform procedures (particularly in financial matters such as procurement and disbursement) and the elimination of the administrative burden to have a large ground capacity in Timor-Leste; and (iii) to have a voice in the mechanism in which most donors participate. 10. The EC was aware of the drawbacks, particularly the lack of visibility, TFET being seen by all as a World Bank undertaking. There were also concerns as to whether the EC contributions would actually be allocated to the ECs focus sectors and the risk that EC funding would be used for procurement from countries normally not eligible for EC financing. Despite these drawbacks, the EC went ahead with the TFET project because the advantages were thought to by far outweigh the negative aspects.

-6-

OBJECTIVES OD THE EVALUATION 11. As stated in the Terms of Reference, the objectives of the evaluation are two-fold: (i) to assess the progress and achievements of TFET since its inception in fulfilling its mandate of social and economic reconstruction and bringing the benefits to the intended beneficiaries in a post-conflict context; and (ii) to draw lessons in relation to the strategies and coordination mechanisms of TFET, which can be applied in Timor-Leste, as well as in other countries in similar circumstances. 12. As TFET is being phased out, the evaluation should also draw lessons and make recommendations as to the strategy, modalities and approaches of the ECs future assistance to Timor-Leste, taking into account the size of the country and the impracticability for the EC to maintain a large representation in Dili to help design and monitor the implementation of EC-financed projects. METHODOLOGY IMPLEMENTATION OF THE ASSIGNMENT 13. The work was organized into five phases as follows: Phase 1: Familiarization. Briefing in Brussels, interviews of EU staff and review of available documents; Phase 2: Washington and New York. Interviews of World Bank and UN (Department of Political Affairs - DPA and Department of Peacekeeping Operations - DPKO) staff involved in the project; Phase 3: Jakarta, Indonesia and Dili, Timor-Leste. Briefing with the EC Delegation in Jakarta and the EC Correspondence Office in Dili. Interviews with TFET stakeholders, including members of the Timor-Leste leadership, Government officials, donor representatives, PMU staff, and NGOs. The evaluators also made a field trip as part of their case study on projects in the rural sector. Phase 4: Telephone and Written Interviews. The fieldwork was complemented by telephone interviews of key persons involved in the project in the early days of TFET based on a questionnaire previously sent to the interviewees (Annex 4). The questionnaire was also used for interviews in Dili. Phase 5: Report Writing. A draft report was submitted to the Delegation in Jakarta and Dili on the 7th of June 2004, i.e., two weeks following the return of the evaluators to their base. Following a Debriefing in Brussels on the 22nd of June, the draft report was circulated by the Jakarta Delegation to TFET donors and the two implementing agencies for comments. The final report, incorporating comments received, was submitted to the European Commission on the 28th of July 2004.

INFORMATION 14. The evaluators worked with the following sources of information: The ECs TFET files, including internal documents in Brussels, Jakarta and Dili; and Official documents on TFET provided by the World Bank and the Asian Development Bank, such as the reports of the Joint Assessment Mission, appraisal -7-

reports, TFET Grant Agreements, World Bank and ADB implementation supervision missions Aide Mmoires, Implementation Completion Reports, six monthly Reports of the Trustee with Appendices. The evaluators requested authorization to consult the World Bank and ADBs internal files on TFET, but did not receive approval to do so. However, the World Bank offered to provide specific internal information at the evaluators request. We have thus asked the World Bank (and the ADB) to provide data on procurement by nationality of contractors and staff input by project compared with overall and regional averages in order to assess how effective was the system put into place to process TFET-financed projects. We have received procurement and some staff input data from the World Bank. We have not received data from the ADB. It is worth mentioning that in general the ADB has been more restrictive in providing information than the World Bank. We were only allowed to see some recent project Aide Mmoires in the ADB Dili office, but were not given electronic copies. It is also worth noting that no access was given to World Bank and ADB documents relating to phases in the project cycle before appraisal.

AREAS OF FOCUS 15. Following the EC guidelines, the evaluation focused on assessing the relevance, effectiveness, efficiency and sustainability of both TFET as a funding mechanism and the 23 projects it financed. Since it is too early to evaluate the impact of TFET as a whole or the individual effectiveness and efficiency of all the TFET projects, the evaluators focused on particular programme components and reviewed ICRs. As with all EC evaluations, the aspects of ownership, capacity building, sectoral policy formulation and donor participation were given special attention. EVALUATION PERIOD 16. The evaluation covers the period from September 1999 to April 2004. Of particular interest is the period up to May 2002 when Timor-Leste became an independent country. It is the period of programme definition and emergency reconstruction with UNTAET as the governing authority in Timor-Leste and the Timorese gradually assuming larger responsibilities. By studying it, one can assess the usefulness of TFET-like arrangements to fund reconstruction. 17. The period since independence sees a declining UN role with the Timorese Government in charge. It is the time when TFET activities are adapted to the Government of Timor-Leste (GoTL)s priorities as expressed in the National Development Plan (NDP), the NDP Roadmap, Sector Investment Plans (SIPs) and the Transition Support Programme (TSP). It also offers a glimpse of how the performance of projects might evolve in a business as usual environment, i.e., in an environment with a drastically reduced UN presence. ACHIEVEMENTS AND WEAKNESSES 18. Resource Mobilisation. TFET mobilised US$169 million, which together with investment income of over US$8 million brought the financing available for the

-8-

reconstruction of Timor-Leste to over US$177 million. On a per capita basis, it is the highest amount of aid mobilized through a MDTF for reconstruction. 19. The largest contributors were the EC (US$50.7 million)1, followed by Portugal (US$50 million), Japan (US$28 million), Australia (US$12 million), the UK and the World Bank (each with about US$10 million). Together, the European Union (the EC plus Member States) accounts for 69% of contributions received by TFET which, according to TFETs governance rules, gives it a de facto veto power over TFETs decisions, including approval of its budget and work programmes2. 20. Relevance of the Reconstruction Programme. TFET financing of Timor-Lestes reconstruction has been guided by the findings of a Joint Assessment Mission (JAM) undertaken in October 1999 to identify priority reconstruction needs and estimate the financing requirements. The JAM was coordinated by the World Bank and included experts from five donor countries, four UN agencies, the EC and the ADB. The involvement of Timorese experts was critical in orienting the mission to the Timorese context. 21. As of April 2004, TFET has financed 23 projects for a total of US$168 million. Of these 23 projects, 10 have been fully disbursed. The remaining US$8.9 million have been totalling committed. 22. The portfolio is evenly distributed between social sectors, infrastructure reconstruction and economic activities, including rural development. The projects were well selected and are relevant to the reconstruction of Timor-Leste. The sequencing and the intrasectoral allocation of funds are adequate. The Implementing Agencies were diligent in preparing and appraising the projects, which broadly conformed to their policies. Disbursements, which were almost 60% of the total amount mobilised at the end of 2002, have tampered off and full disbursement is expected only for 2007. 23. Management of the Reconstruction Programme. TFET projects are managed according to the World Bank/ADB model in which a Project Management Unit (PMU) is established for every project to help with implementation. Overall coordination of the programme is the responsibility of the office or department in charge of the country. In the Timor-Leste case, it is the World Bank and ADB Country Offices in Dili. 24. PMU staff reports to the Government. In many countries, they are civil servants seconded by the administration. In Timor-Leste, most of the PMU staff was contracted first by UNTAET and then by GoTL. Most were paid by the project. PMU integration into the ministries (with the noteworthy exceptions of agriculture and health) has been difficult as they were seen as being controlled by the Implementing Agencies.

US dollar values are used when discussing the EC contribution in the context of other donors contributions.. The values in Euros are converted into US dollars using prevailing exchange rates. 2 Decisions of the TFET Donors Council are made, to the extent possible, by consensus. In the absence of consensus, each contributor is entitled to one vote for each US dollar equivalent, as determined by the Trustee, of its paid-in contribution to TFET and matters requiring a decision of the TFET Donors Council will be determined by the majority of the votes cast by contributors represented at the meeting of the TFET Donors Council. Resolution Amending the Trust Fund for Timor-Leste, IBRD Resolution No. 99-8/1; IDA Resolution No. 99-5/1, World Bank.

-9-

TFET Reconstruction Programme, 2000-04

Project Name ACTIVE PROJECTS 1. Agriculture Rehabilitation Project II 2. Agriculture Rehabilitation Project III 3. Community Empowerment II 4. Community Empowerment III 5. Fundamental School Quality Project 6. Economic Institutions Capacity Building 7. Health Sector Rehabilitation Project I 8. Health Sector Rehabilitation Project II 9. Petroleum TA Project II 10. Small Enterprise II 11. Emergency Infrastructure Rehabilitation I 12. Emergency Infrastructure rehabilitation II 13. Micro-Finance Project Subtotal CLOSED PROJECTS 14. Agriculture Rehabilitation I 15. Community Empowerment I 16. Dili Employment Generation Project 17. Emergency School Readiness Project 18. Human Resources Survey 19. Hera Port 20. Petroleum TA I 21. Small Enterprise I 22. Water and Sanitation I 23. Water and Sanitation II Subtotal Grand Total
SOURCE: The World Bank

Date Financed 29-Oct-01 20-Jan-04 21-May-01 1-July-02 29-Oct-01 26-Feb-01 7-June-00 15-Oct-01 30-July-03 29-Oct-01 19-April-01 18-May-02 18-Dec-00 29-Oct-01 21-Feb-00 1-April-00 21-June-00 18-Jan-00 27-Nov-01 10-Dec-02 11-April-00 24-Aug-00 27-Aug-01

Amount ($ million) 8.0 3.0 8.5 1.5 13.9 0.6 12.7 12.6 1.7 7.5 29.8 9.0 4.0 122.8 6.8 9.0 0.5 13.9 0.4 1.0 0.2 4.8 4.5 4.5 45.6 168.4

25. Quality of Project Preparation and Implementation. The World Bank put into place special procedures to accelerate the processing of TFET projects. The two Implementing Agencies supervised regularly their projects, which have all been rated satisfactory. They have also satisfactorily fulfilled their fiduciary obligations (i.e., disbursement of funds, procurement of goods, works and services, financial reporting and audit). 26. Project Issues. TFET projects were generally well designed and met Timor-Leste reconstruction needs. There were few issues, with the exception of the following: 1. Provision of Credit. In line with the JAM guidance, TFET (and some NGOs) started micro-credit and four projects channelled loans to rural and urban beneficiaries with only limited success. Loan recovery rate ranges from 30% (CEP) to 40% (SEP). Following strong Government actions, the SEP rate recently increased to 70%3. The credit components of CEP and SEP were adopted against best practices (in the case of CEP by mixing loans with grants; and in the case of SEP the giving out of emergency loans with limited guarantees and without the intermediary bank sharing in the risk). Unlike the
3

Despite this improvement, the credit component of SEP II was cancelled.

-10-

JAM, the evaluators believe that credit is not suitable as a means to inject liquidity in an underdeveloped economy with limited credit experience4 and going through post-conflict reconstruction like Timor-Leste. Grants are more adapted to the situation. Credit should be introduced only at the development stage and when the institutional and policy framework is in place. In Timor Leste micro-finance, there are at present successful NGOs, while the TFET micro-finance project is only barely satisfactory; and 2. Technical Assistance and Consulting Services. Roughly for every $3 spent on TimorLeste reconstruction, $1 was for technical assistance and consulting services. In some projects, the ratio is even higher, sometimes close to 1/1 (Health II). The situation reflects the fact that consultants, often Timorese, had to substitute for the non-existent local administration. Data are not available to make a comparison between TFET projects and projects financed by other aid programmes (UN agencies and bilateral) to see whether the issue is general. 27. Results on the Ground. Much has been achieved in less than four years since the violence and destruction of September 1999. With a few exceptions, most of the infrastructure rehabilitation needs identified by the JAM and the joint donor sector missions had been completed. Basic levels of service in health and education had also been restored. 28. The economy responded to the inflows of external assistance, the high demand for goods and services brought about by the international workers, the improved infrastructure and the countrys better prospects. GDP increased and by Independence in 2002 had reached pre-crisis levels. 29. In addition to stabilizing the economy, the foreign assistance provided through TFET (and other channels) succeeded in re-establishing a functioning administration from a complete collapse of administrative structures in 1999. With the help of foreign assistance and funding of TA projects, the Government has been able to strengthen its fiscal policy and put into place: a functioning treasury and payments office; an administration which ensures basic services throughout the public sector, maintains adequate standards of financial control and compliance; good coordination between the Government and donors and between TFET and CFET; basic health and education services all over the country; and modern procurement rules and practices.

30. Finally, although TFET only represented about half the financing available for TimorLestes reconstruction, its efficient and effective donor coordination and good technical project design and preparation seem to have influenced the allocation of non-TFET resources toward meeting the reconstruction needs identified by the JAM, which have thus been fully funded.
Under the Indonesian occupation, credit schemes introduced by Indonesia had a recovery rate of around 30%. For political reasons, no strong recovery actions were applied against delinquent borrowers. Timorese did not learn the credit concept. Rather, they learned that a loan might not be repaid! An independent review by the World Bank of the SEP cautions: Think twice before offering a line of credit in an immediate post-conflict environment.
4

-11-

31. Speed of Reconstruction and Capacity Building. The pace of reconstruction was brisk, particularly during the first two years, so much that by Independence (May 2002), most of the emergency reconstruction work was completed. As in other postconflict situations, this was achieved at the cost of slower capacity building. However, in the Timor-Leste case, this unavoidable trade-off between speed of reconstruction and capacity building was complicated by the fact that TFET did not have the mandate for capacity building, which was entrusted to UNTAET by the participating donors. EVALUATION TFET PERFORMANCE 32. TFET has been instrumental to the successful reconstruction of Timor-Leste. Its performance (and, by extension, that of the two Implementing Agencies) has been satisfactory. TFET achieved the results the EC expected of it when making the decision to contribute to the trust fund. The mechanism (i) achieved good donor coordination; (ii) succeeded in mobilizing substantial resources for Timor-Leste reconstruction; and (iii) was efficient as it used uniform procedures for project processing and implementation and eliminated the need for the EC to establish and maintain a large office in Dili to monitor implementation of the reconstruction programme. 33. It is a relative success because the mechanism did not significantly improve the countrys low institutional capacity and failed to develop Timorese ownership of the reconstruction programme. It is true that, at the Tokyo meeting, it was to UNTAET that the donors entrusted with the responsibility for capacity building. That division of labour was reconfirmed after Independence when donors helped finance the UNMISET Stability Posts Project and the UNDP Development Posts Project, two operations entirely dedicated to capacity building. 34. However, given the importance of the matter and the fact that capacity building is central to all World Bank and ADB interventions in member countries, the two implementing agencies should have discussed and agreed with UNTAET right at the start on a modus operandi and on a global strategy to jointly address the issue. In fact, with the exception of the health sector, capacity building was tackled in a piecemeal fashion by all the institutions involved, without a clear vision and expected results. 35. The mechanism could also have been more efficient in the utilization of the substantial resources it succeeded in mobilizing for Timor-Lestes reconstruction and development. First, the proliferation of PMUs (one per sector) handling all project implementation functions is a waste of resources and makes coordination of the overall TFET programme more difficult. Very few PMUs were actually integrated in the Timorese nascent administration (agriculture and health were exceptions), as they were perceived as World Bank-controlled entities. The high salary structure in the PMUs compared to that of the Timorese administration also prevented actual integration. 36. While strong PMUs were needed in Timor-Leste because of the extreme weakness of the Timorese administration (non-existent at the beginning), some functions such as procurement and financial management could have been centralised to save costs, make better use of scarce skills and simplify project management. The centralised functions could

-12-

be spun off later to form a service or unit dedicated to procurement or project financial management/monitoring within the Timorese administration. Indeed, in some small countries (e.g., Mauritius, Rwanda), public procurement above a certain threshold is handled by a national agency, instead of the ministries or public enterprises. 37. Second, project processing procedures were not streamlined enough, while no significant simplification of procurement rules took place, despite the statement by World Bank staff presenting the TFET project to the Executive Directors (November 1999) that project preparation would be carried out on an accelerated basis in order to provide near term reconstruction assistance to this post-conflict area and that streamlined procurement and disbursement procedures, consistent with Bank guidelines and used in other post-conflict situations will be utilized. 38. Some sectors, particularly the health sector, did use World Bank accelerated procedures reserved for emergency operations, while project approval was delegated to the RVP. However, staff working on TFET still had to produce full PAD (Project Appraisal Documentation) for all projects, despite the need to move fast, the relatively small size and the repetitive nature of the projects5. This has increased staff time devoted to project preparation/appraisal and slowed down project implementation at the beginning of TFET operation as disbursements were made from Washington and as Timorese (or technical assistants recruited in the PMUs) were struggling with complex procurement procedures. 39. Third, after Independence when the major reconstruction works were completed, there should have been a reorientation of TFET work toward development, instead of continuing on the same reconstruction mode. Adjustments did take place to realign some projects to the NDP vision, but they were done at the project level. By and large, TFET continued its reconstruction business as usual. 40. The shift towards development could have been facilitated had the JAM developed a vision for the sectors in terms of objectives, policies, institutional architecture, staffing and financial needs. Reconstruction activities would have been guided by this vision and would evolve smoothly into development activities. Only health adopted such an approach. 41. At least, there should have been a mid-term evaluation by the World Bank, as the Trustee, of the achievements with a view to addressing emerging problems and, if warranted, moving TFET work toward longer-term development projects6. 42. The evaluators are also concerned about the sustainability of some of the reconstruction work. Some projects, such as the Dili employment project, were not designed to be sustainable, but others, like the credit activities, were but would not. The physical infrastructure, particularly the roads, be it community-built or contractor-built, are not likely to be sustainable either without adequate maintenance and continued investment. On the macroeconomic level, the whole reconstruction effort would be difficult to sustain in the long run if Timor-Leste does not get its fair share of the oil and gas revenues in accordance with

Full PADs were produced even for projects that were just the repetition of the previous ones, such as CEP II and ARP II. 6 It has been pointed out that this mid-term evaluation by the EC could have served the purpose, had it been carried out two years ago as planned. While this is true, it does not absolve the Trustee from making its own evaluation, as its obligations would have required it to do so.

-13-

international laws and substantially improve its capacity to maintain and operate the rehabilitated assets. 43. Finally, it should be noted that the World Bank and the ADB were able to design projects the way they did in Timor-Leste with substantial technical assistance and consulting services (a large part of it foreign) because the funds entrusted to them were grants. With their normal resources, which are loans and credits albeit at preferential interest rates, it would have been financially irresponsible to do so in a poor and non-creditworthy country like Timor-Leste. 44. La critique est aise, et lart est difficile. The evaluators recognize the difficulty of the task at hand when everything had to be done right away from rebuilding schools to repairing roads and establishing a local administration (without mentioning the need to adjust to working with new partners and institutions with different and complex policies and procedures). 45. Timor-Leste was an unusual case and a challenge to all the institutions involved, many of which had to take on new and unaccustomed roles. As the Implementing Agencies pointed out: We learned as we went Mistakes were made, but we learned from them to do a better job. Indeed, at the end, the job (and a good one) got done. 46. In spite of the weaknesses, the evaluators believe TFET was superior to the other alternatives (UN, NGOs and bilateral). Its staff had extensive experience of reconstruction/development financing, was highly competent and motivated and relied the most on Timorese inputs. According to Timorese decision-makers, the mechanism with its efficient donor coordination structure was the least bad instrument for Timor-Leste during this difficult transition period. For the EC, TFET was the best way to provide its assistance to the reconstruction of Timor-Leste. 47. TFET has failed to build up Timorese ownership of the reconstruction efforts, but so have the other aid programmes. The prevailing aid architecture, with UNTAET acting as the legal authority of Timor-Leste for most of the transition period and responsible for capacity building, did not facilitate matters. The failure was also relative, depending on the sectors. In agriculture, the TFET decision-making process was fully integrated into the Timorese administration; such integration was high in education and health but non-existent in CEP and SEP. EC PERFORMANCE 48. The performance of the EC was satisfactory. The Commission was present in TimorLeste right at the beginning. Beyond large-scale provision of humanitarian aid amounting to 51 million, it participated in the JAM (Joint Assessment Mission) and thus contributed to the design and estimate of the cost of the reconstruction programme. 49. The EC was also the largest participant to TFET with contributions totalling 56.4 million. Subsequently, in 2003 and early 2004, the EC made new contributions to co-finance

-14-

projects with TFET. This approach was warranted, as TFET was being phasing out, while sector investment programmes were still being prepared to serve as a basis for SWAps7. 50. The Commission was fortunate to have a competent and dedicated Head of its Dili Representative Office who kept the EC Delegation in Jakarta informed of the reconstruction progress, participated in virtually all TFET related-meetings and made the ECs positions known. The Dili Office needs to be strengthened with specialists from the Delegation in Jakarta or Headquarters when joint sector missions are in Timor-Leste. 51. The idea of an interim evaluation was excellent and would have provided the EC, the other donors, and the Implementing Agencies with the needed information to improve TFET performance and efficiency. It is regrettable that it took place two years later than foreseen. Nevertheless, the evaluation is still relevant and, it is hoped, will help strengthen the ECs participation in future MDTFs MAIN LESSONS LEARNED Donor Coordination 52. TFET was the right mechanism to ensure donor coordination. It is most effective when (i) donors are actively engaged; and (ii) the Implementing Agencies listen to and hear the donors views. Stability/Reconstruction versus Sustainability/Development 53. It is obvious that in a post-conflict situation the main objective is reconstruction and stability. However, a deliberate shift to development and sustainability could and should be made, particularly when, as in the Timor-Leste case, substantial resources have been mobilised (some would say too much) for reconstruction and the period allowed by donors for spending them relatively long. 54. It is impossible to predict in advance when that shift would occur as post-conflict situations are by definition fluid and changing. Reconstruction progress depends on many factors, least of which the security situation. It is therefore imperative to build in the monitoring process periodic assessments of reconstruction progress (not individual project progress) and a mid-term evaluation to decide whether to continue reconstruction efforts or to give more importance to the objectives of sustainability, policy development and capacity
SWAps (Sectorwide approaches) also known as sector programmes or sector investment programmes are mechanisms by which development agencies collaborate to support sector reform programmes that are based on a countrys long-term vision for its development. They have evolved from the traditional investment project as a means to enhance development impact by scaling up, building stronger donor partnerships in support of country-led sector reform programmes, and improving the performance of the beneficiary countrys institutions in managing all sector resources. SWAps were first undertaken in post-adjustment countries which fulfil a number of pre-conditions, including (i) macroeconomic stability; and (ii) transparency and efficiency of budgetary and procurement procedures as such operations generally use the beneficiary countrys procedures and not the donors procedures. Recently, SWAps have been used (particularly by the World Bank) to finance social sectors and the implementation of the PRSP (Poverty Reduction Strategy Papers). The pre-conditions of macroeconomic stability and transparency and efficiency of budgetary and procurement procedures have also become much more flexible since countries such as Cambodia, Ghana and Tanzania have received such assistance.
7

-15-

building. The change of direction, objectives and priorities must be clearly agreed among all the involved parties and appropriate indicators developed to monitor progress. 55. As the AusAID representative put it: there should be a formal mechanism to build in periodic external or joint donor reviews [of] the design and costing of such funding mechanisms in [the] future, as post-conflict situations are often fluid, changing and hecticRegular reflection on the overall strategic direction of such facilities, rather than only on individual activities, is important in this same vein. [Those] regular reviews would highlight the need and right timing for development of exit strategies or strategies to ensure a smooth transition between mechanisms (e.g., the transition from TFET to TSP support was just made without drawing out the linkages between large amounts of not yet disbursed TFET investment and recurrent cost support items and this new form of budget support). Greater synergies might have been achieved earlier and there is still a need - now presumably through the SIP process - to gauge the impact of diminishing TFET funds in certain core areas of government on budget processes. 56. During the reconstruction phase, care must be taken to do no harm, i.e., avoid measures, which could compromise the future, even if they seem to make sense from a stability perspective. In the Timor-Leste case, this means (i) avoid setting high salary levels for low-skilled labour and PMU local staff; (ii) strengthen electricity bill collection; and (iii) avoid credit schemes. 57. Also, during the reconstruction/stability phase, there must be a clear vision as to where the country or sector(s) should be heading. In the Timor-Leste case, the health sector had such a vision and the actions taken during the reconstruction phase avoided harmful measures and had development as the final objective. As the World Bank noted, not surprisingly, it was the first sector to have a broad policy statement adopted by the Council of Ministers after Independence. Ownership/Capacity Building versus Speed of Reconstruction 58. Ownership and capacity building are difficult issues to address in a reconstruction context where the overriding priority is to repair roads, rebuilding schools, etc. to get things going. The lessons learned from the Timor-Leste experience are the following: Develop a global strategy to deal with the issue. Such a strategy should set out clear objectives for individual reconstruction projects and spell out how to deal with other partners if the responsibility for capacity building is divided between several institutions as in the Timor-Leste case (World Bank, UNTAET and ADB); Explicitly accept slower results in physical reconstruction if capacity building is a priority in a particular project. On the other hand, try to do no harm if the objective is only rapid physical reconstruction; Include long-term training, if possible; Finance projects dedicated to capacity building; and Develop a long-term vision.

FUTURE USE OF THE MECHANISM BY THE EC 59. MDTFs are well suited for post-conflict assistance. To increase the value added of their use by the EC, the evaluators suggest the following: -16-

Set clear conditions for participating o Agree at the outset on what the MDTF should address: Rehabilitation and reconstruction or also development; o Set clear overall and sector objectives and expected results; and o Specify the timeframe for the assistance.

Visibility o Request that all the donors be acknowledged on Grant Agreements, advertisements, joint publications, placards, information boards, press releases etc.; and o Be closely involved in the design of the programmes to ensure that focus sectors are adequately addressed.

Monitoring work progress closely o Delegation(s) should be adequately staffed with technical personnel in order to efficiently monitor and contribute to the work. Specialists could also be sent from headquarters. Such a deployment of personnel should be planned and budgeted, including the local recruitment of technical assistants (ALATs); and o Plan mid-term (and a final) evaluations following the procedures of the EC and the Trustee, i.e., including review of internal files.

Tied procurement o Procurement tied to EU countries cannot be managed within MDTFs. However, the ECs assistance is moving toward non-targeted budget support and sector financing (SWAp) and therefore the EC implicitly accepts untied procurement. Many EC countries are also moving toward untying their aid.

Modalities of future EC assistance to Timor-Leste o The EC is considering using SWAps (which could be done either with other donors or directly with the Timorese Government) for its continued assistance in the health and agriculture sectors. This is a logical outcome of the TFET experience. The evaluators strongly support the use of these mechanisms as they incorporate the best features of TFET, i.e., (i) good donor coordination; (ii) strong and continued policy and sector strategy discussions among all stakeholders; and (iii) use of unified project processing and implementation procedures. However, care must be taken to ensure that capacity building is built into these mechanisms, which are not designed to address this fundamental aspect of foreign assistance in Timor-Leste.

-17-

CHAPTER I:
I. POLITICAL CONTEXT8

CONTEXT

1.1. Timor-Leste became the first new state of the 21st century on May 20, 2002. This was almost three years after 78.5% of the registered electorate voted to reject the autonomy offer within the Republic of Indonesia and opted for a transition to independence in an UNsponsored referendum on August 30, 1999. 1.2. The announcement of the ballot results set off an orchestrated campaign of violence across the territory, which resulted in the destruction of an estimated 70% of social infrastructure and private buildings and the displacement of over two thirds of the population. State structures in the country suffered a complete collapse. 1.3. In response to the violence, the Security Council (Resolution 1264) authorized the deployment of an international force. INTERFET entered Timor-Leste on September 20 and restored order across the territory. The military initiative was followed by the establishment of a United Nations civilian administration in Timor-Leste. UNTAET (United Nations Transitory Administration in Timor-Leste) acted as the legal administrative authority until independence on May 20, 2002, under a wide-ranging mandate covering everything from security to humanitarian affairs and the establishment of a functioning civil service. 1.4. Thus, UNTAET was the legal government of Timor-Leste throughout most of the reconstruction period, which saw considerable confusion over who constituted legitimate Timorese representatives for decision-making. CNRT (Conselho Nacional da Resistncia Timorense), the main resistance movement under the presidency of Xanana Gusmo, was not accepted by the UN as the primary legitimate Timorese representative body when the transitional authority was established. Indeed, given its experience in Bosnia and Herzegovina and in Kosovo where it was seen as supporting sectarian interests, the UN wanted to view CNRT only as a political party, not a body representing the Timorese and was wary that close consultation could imply recognition9. 1.5. UNTAETs establishment did not benefit from comprehensive preparatory work. As the 2002 Kings College UNTAET evaluation noted: The planning that took place in the wake of destruction and violence that followed the announcement of the result of the ballot was necessary cursory. The short timeframe influenced the outcome of the planning process in several ways: the plan that was produced focused on structures rather than processes; the planning assumptions adopted were not always adjusted to the new situation; and there was
This section draws heavily on (i) various press releases posted in the website of the External Relations Department of the European Commission; (ii) discussions with Timorese officials and UN and World Bank staff involved in Timor-Leste at that time; (iii) Klaus Rohland and Sarah Cliffe, The Timor-Leste Reconstruction Programme: Successes, Problems and Tradeoffs, Conflict Prevention and Reconstruction Unit, Working Paper No. 2, November 2002, World Bank; (iv) Review of Peace Keeping Operations: A Case for Change, TimorLeste, December 2002, Kings College, London; and (v) The United Nations Transitional Administration in Timor-Leste (UNTAET), Debriefing and Lessons, Report of the 2002 Tokyo Conference, The United Nations Institute for Training and Research (UNITAR), The Institute of Policy Studies of Singapore (IPS) and The Japan Institute of International Affairs (JIIA) 9 There were, however, very good personal relationships and intense informal exchanges of views between the UN Special Representative, Sergio Vieira de Mello and CNRTs President, Xanana Gusmo.
8

-18-

heavy reliance on familiar models, in particular the recently-launched UN Mission in Kosovo (UNMIK). The planning team was also narrowly based and several possible contributors to the process were only tangentially involved, including the Secretariats Department of Political Affairs, the World Bank and the Timor-Leste pro-independence umbrella group, the CNRT10. 1.6. UNTAET established a National Consultative Council (NCC) in January 2000 to advise on governmental matters. Following dissatisfaction amongst the Timorese leadership, NCC was replaced by the Timor-Leste Transitory Authority (ETTA), the first nucleus of a future government, organized around a joint cabinet structure of Timorese and international representatives11. The National Council, the nucleus of a future assembly, was established in October 2000 and consisted of 36 representatives. 1.7. Following the constitutional elections of August 30, 2001, both ETTA and the National Council were abolished and replaced with an 88 member Constituent Assembly and a second Transitional Government with a wholly Timorese cabinet reflecting the composition of the Constituent Assembly. This cabinet operated until May 20, 2002, when a new council of ministers was appointed on the day of independence. 1.8. In March 2002, the Constituent Assembly transformed itself into the National Assembly to sit until 2007. The constitution was drafted and approved rapidly, including the holding of district consultations with enthusiastic popular input. It adopted a parliamentary form of government with a largely symbolic presidency. Xanana Gusmo won the first presidential election held on April 14, 2002 with 88% of the votes. Hand over from UNTAET to the Democratic Republic of Timor-Leste took place on Independence Day May 20, 2002. 1.9. This brief summary of the political events leading to formal independence is important to understand the Timor-Leste case and evaluate TFET as it brings out the context in which reconstruction took place. That context was characterized by: 1. Complete collapse of state structures, accompanied by widespread destruction of infrastructure and disruption of livelihoods; 2. Extremely low capacity due to the departure of virtually all middle and higher-level civil servants (who were mostly Indonesians) following the referendum, as well as 80% of the secondary school teachers and the majority of medical doctors; 3. Rapid and peaceful political transition to a democratic state, with good internal security, based on a broad national consensus. The country did not go through a prolonged and divisive period of civil war like other post-conflict countries; 4. Lack of clarity on the Timorese representation in decision-making during the earlier stages of the transition;

10 11

Review of Peace Keeping Operations: A Case for Change Timor-Leste Op. cit. page 2 Timorese representatives held the portfolios of economic affairs, social affairs (education and health), internal administration and infrastructure. The UN retained the portfolios of finance, justice and political affairs. Security matters were dealt with outside the cabinet structure.

-19-

5. Four different government structures in the span of three years, including UNTAET, which had a dual role as the legal governing authority of Timor-Leste and as a major participant in the reconstruction process; and 6. Complex aid architecture and rigid division of work between the implementing institutions with UNTAET in charge of building up the public administration and strengthening institutional capacity and the multilateral development banks (World Bank and Asian Development Bank) responsible for reconstruction activities. II. A. THE DONORS RESPONSE Joint Assessment Mission (JAM)

1.10. A Joint Assessment Mission (JAM) to assess reconstruction needs in Timor-Leste, to be coordinated by the World Bank, was endorsed at a meeting of donors, UN agencies and Timor-Leste representatives on September 29, 1999. Early deployment of the mission was driven by lessons of other post-conflict situations, where lack of coordination between humanitarian relief and planning for reconstruction/development had delayed the transition from emergency relief to more sustainable development support, and has caused inefficiencies and duplication in the use of external resources. The JAM work was closely coordinate with that of OCHA and other UN agencies working on the budget for humanitarian needs. 1.11. The mission aimed at identifying priority short-term reconstruction initiatives and providing estimates of external financing needs. To foster compatibility with longer-term development objectives, the JAM used a comprehensive development framework approach, covering eight sectors and incorporating international technical expertise from five donor countries, four UN agencies, the European Commission, the Asian Development Bank and the World Bank. A concurrent mission of the IMF accompanied the Joint Assessment Mission. Timorese technical specialists (primarily from the Diaspora) were paired with international experts in each sector, to ensure input of local knowledge and capacity building for local professionals. 1.12. The JAM assessed the situation and needs of eight sectors: Sector Civil service Justice system Economic management Infrastructure Agriculture Health Education Community empowerment Responsible Agency UNTAET UNTAET TFET TFET TFET TFET TFET TFET

1.13. Total needs for the three-year period 2000-2002 were estimated at US$ 307 million, with social sectors accounting for 56%, productive sectors (including economic infrastructure) for 28% and administration and judiciary system for 16%. It is not clear from

-20-

the documents available whether the three-year period was for commitments or disbursements. 1.14. The Timorese were very appreciative of being associated to the exercise and thought that it boded well for the future role of Timorese in TFET. As Klaus Rohland and Sarah Cliffe noted in their Paper12 the involvement of Timorese coordinators and technical specialists was critical in orienting the mission to the Timorese context. 1.15. However, the Timoreses participation in the JAM did not last long because of a lack of financing. Initially, funded by the World Bank through a grant from the Post Conflict Fund (PCF), the Timoreses involvement was supposed to be covered by UNTAET. However, UNTAETs establishment was delayed because of UN internal problems and as no bridge financing was available, most Timorese left at the end of the mission and the work was completed by international experts. 1.16. All the stakeholders share the opinion that the JAM was successful. The inclusion of the Timorese in the exercise was commendable. The lack of continuity and thus ownership by the Timorese could have been avoided with more coordination with the institutions involved and better planning by the UN. 1.17. While the objective was clearly to assess needs, the JAM could not avoid getting involved in policy setting. The actions it took set the direction of the reconstruction. Some are thought to have a positive impact on the future of Timor-Leste, such as (i) the decision to keep civil service recruitment well below the former strength (12,000 vs. 28,000); (ii) the emphasis on community driven reconstruction and private sector recovery; and (iii) the health strategy with its emphasis on district-level care. On the other hand, other actions have been counter-productive, such as the inadvertent setting of local wage at a high level. This still dogs the development of the Timorese economy. The controversial language issue was not resolved (the compromise, i.e., a gradual shift to teaching in the two official languages Portuguese and Tetum, away from Indonesian, was fully articulated and accepted by all concerned only in late 2003) and has adversely affected policy making in the education sector. The Evaluators comments In retrospect, the JAM should have made efforts to define strategies for the sectors in terms of policy objectives and priorities, institutional architecture, staffing plans and expenditure framework. This vision updated as necessary would have been guided reconstruction activities and ensured that they evolve seamlessly into development activities.

B.

The European Commissions Response

1.18. The EC was present in Timor-Leste throughout the crisis. Beyond large-scale provision of humanitarian aid amounting to 51.0 million, it participated in the JAM and was among the largest TFET contributors with 56.4 million provided through two Financing
12

The East Timor Reconstruction Program: Successes, Problems and Trade-off, op. cited.

-21-

Decisions (IDN/B7-3040/IB/2000/00 for 40 million in 2000 and ET/AIDCO/2001/0292 for 16.4 million in 2001). Advantages and drawbacks of participating in TFET for the European Commission13 1.19. The main expected advantages for the EC to participate in TFET were (i) better donor coordination; (ii) use of uniform procedures (particularly in administrative matters such as procurement and financing) and the resulting gains in efficiency; (iii) elimination of the administrative burden on the EC in a country where it does not have a large ground capacity; and (iv) to have a voice in the mechanism in which most donors participate. 1.20. With regard to drawbacks, the Financing Decisions point to concerns regarding (i) lack of visibility, TFET being seen by all as a World Bank undertaking; (ii) assurance that resources will actually be allocated to the ECs priority sectors; and (iii) risk that EC funding would be used for procurement from countries normally not eligible for European financing. Monitoring of Achievements 1.21. The logical framework of the EC Financing Decisions is presented below. The overall objective is to assist in the development of a viable independent nation. Achievement indicators include (i) assumption of administrative and political functions by Timor-Leste; (ii) economic growth; and (iii) existence of sectoral management. The output indicators are defined in terms of physical achievements (number of roads built/opened), use of services and response of the donors. The means of verification consist primarily of the six monthly World Bank reports and donors meetings. 1.22. While adequate at the financing decision phase, this approach needs to be translated in operational terms for effective and substantive monitoring of TFETs contribution to the development of Timor-Leste by the EC (and other donors) staff in the field. The following aspects would need to be developed in the reporting: Quality of inputs (works and technical assistance) and sustainability of outputs; Speed of reconstruction and completeness of reconstruction/rehabilitation work; Progress on sectoral policy development in terms of post-conflict sectoral strategies, including policy objectives and priorities, institutional architecture, staffing and expenditure frameworks; and Exit strategy and steps to move from reconstruction to development.

13

Based on the review of the two Financing Proposals

-22-

Table 1-1 Logical Frame of the EC TFET Financing

Project Structure Indicators of Achievement Means of Verification Assumptions, risks and conditions

General Objective The overall objective of the programme is to contribute to East Timors successful transition to independence and the development of a viable independent nation Specific objectives To respond to the rehabilitation and reconstruction needs over a three-year period supporting agricultural recovery, infrastructure rehabilitation and restoring public health and education services. The EU contribution to the World Bankadministered TFET will focus in particular on health and infrastructure projects. Outputs 1. 2. Basic infrastructure is restored, operating and maintained A functional health care delivery system provides basic essential care to the population. Project activities managed by the World Bank. Inputs Overall project management by World Bank. Timely and effective design and interventions. Six monthly and final reports prepared by World Bank and six monthly consultation meetings with donors. Timely availability of appropriate expertise Effective overall management and coordination by the World Bank. Number of roads etc. built, opened. Health service utilization number of outpatients/inpatients Accelerated/strengthened response by World Bank/other donors to sector needs in number and quality of technical assistance actions undertaken. Management information systems Six monthly and final reports prepared by World Bank and six monthly consultation meetings with donors. Effective coordination between World Bank and recipient. Economic growth Demographic health indicators Sectoral management in place World Bank health and demographic surveys Responsive commitment to reform by recipient country, coupled with appropriate and timely advice and training. Effective consultation with all parties concerned (including civil society), coupled with adequate resource availability on the part of World Bank and recipient country. Administrative and political functions taken over by East Timor authorities National legislation and Government orders. Perpetuation of conflict, political and economic instability Lack of cadres and human resources

-23-

III.

FUNDING ARRANGEMENTS

1.23. The international donors conference held in Tokyo in December 1999 pledged a total of US$520 million in assistance to Timor-Leste over three years, in addition to about US$700 million per annum reserved for UNTAET from the UN assessed contribution budget. 1.24. The Tokyo meeting also endorsed the creation of two trust funds: (i) the Consolidated Fund for Timor-Leste (CFET) to be managed by UNTAET for recurrent expenditures, civil service and capacity building; and (ii) the Trust Fund for Timor-Leste (TFET) for economic and reconstruction activities under the trusteeship of the World Bank. The Asian Development Bank (ADB) was invited to participate as TFET co-implementer and manages projects in the infrastructure, water supply and sanitation and microfinance sectors. This division of labour was confirmed at Independence with the mandate for capacity building entrusted to UNMISET (Stability Posts Programme) and UNDP (Development Post Programme). 1.25. However, some bilateral donors preferred to finance Timor-Leste reconstruction outside of the TFET framework, while the different UN Agencies (UNDP, UNICEF, etc) and NGOs continue to channel their assistance directly to Timor-Leste. 1.26. There were thus six different aid mechanisms when the reconstruction work started in 2000: (i) UNTAET-administered CFET mainly covering recurrent expenditures; (ii) TFET with two implementing agencies (the World Bank and the ADB) financing investments; (iii) UNTAET assessed contribution budget of financing salaries of UN staff and contractual employees; (iv) UN agencies projects; (v) bilateral donors aid and reconstruction programmes; and (vi) NGOs. During TFET implementation, a seventh financing mechanism was added in May 2002: TSP (Transition Support Programme) through which the World Bank and other donors helped finance the governments current budget (CFET) after TimorLeste became independent. 1.27. This fragmentation of funding mechanisms rendered the overall coherence of reconstruction efforts difficult and made coordination crucial for the success of the undertaking. IV. OBJECTIVES OF THE EVALUATION

1.28. As stated in the Terms of Reference, the objectives of the evaluation are two-fold: (i) to assess the progress and achievements of TFET since its inception in fulfilling its mandate of social and economic reconstruction and bringing the benefits to the intended beneficiaries in a post-conflict context; and (ii) to draw lessons in relation to the strategies and coordination mechanisms of TFET, which can be applied in Timor-Leste, as well as in other countries in similar circumstances. 1.29. The evaluation is thus on TFET as a mechanism for financing reconstruction in postconflict situations from the European Commission standpoint as an impact evaluation cannot be undertaken at this stage since about half of the TFET-financed projects are still being implemented. The TORs noted that an evaluation of the individual effectiveness and efficiency of the TFET projects cannot be carried out under the present programme evaluation. However, in assessing the effectiveness and efficiency of TFET as a whole,

-24-

particular programme components will be analysed and Implementing Completion Reports (ICRs) will be reviewed. In this respect, a case study on the rural development sector (a focus sector of EC assistance) will be undertaken. 1.30. The evaluation is intended to benefit the TFET implementation teams, the Government of Timor-Leste, the donor community, and ultimately all the beneficiaries of the programme. 1.31. As TFET is being phased out, the evaluation should also draw lessons and make recommendations as to the strategy, modalities and approaches of the ECs future assistance to Timor-Leste, taking into account the size of the country and the impracticability for the EU to maintain a large representation in Dili to help design and monitor the implementation of EC-financed projects.

-25-

CHAPTER II:
I. THE MANDATE

DESIGN AND OPERATION OF THE TFET

2.1. Unlike some other post-conflict cases, the World Bank was not a party to the political deliberations that shaped Timor-Leste post-conflict arrangements. It did not participate in the discussions that led to the Indonesia-Portugal New York Agreement, or in the deliberations that established UNTAET following the September 1999 violence. 2.2. However, prior to the referendum, the World Bank worked closely with the UN Department of Political Affairs (DPA) to analyse the economic impact of acceptance or rejection of the autonomy offer. Early on, the Bank also met with the Timorese leadership and discussed priorities for reconstruction and development after the ballot. At the request of the Timorese leadership and DPA (the UN Department of Political Affairs), the World Bank convened a meeting of the Friends of Timor-Leste at the time of the 1999 World Bank and International Monetary Fund (IMF) Annual Meetings in Washington, which was attended by most donors and the United Nations. The meeting mandated the World Bank to organize a joint assessment mission to identify reconstruction priorities and costing. Subsequently, at the Tokyo meeting, donors supported the establishment of a multi-donor trust fund with the World Bank as the trustee. II. LEGAL BASIS OF WORLD BANK ASSISTANCE

2.3. According to the World Banks Articles of Agreement, the primary purpose of the bank is to assist in the reconstruction and development of territories of members. The World Bank is also required to use its resources and facilities exclusively for the benefits of members. While it is clear from these provisions that World Bank assistance is to be rendered to or for the benefit of its members, the World Bank has determined that it may provide assistance to non-members and still acts within the Articles of Agreements mandate when such assistance is for the benefit of its members as a whole. 2.4. There have been several such precedents where the World Bank provided assistance to non-member countries or territories via establishment of trust funds that finance emergency reconstruction, technical assistance and economic development. These include the former USSR (1991), Gaza and West Bank (1993), Bosnia and Herzegovina (1995) and Kosovo (1999). In all these cases, the Executive Directors found the assistance would benefit the members. In each case, a transfer from the World Banks surplus to a dedicated trust fund financed the assistance14. III. ESTABLISHMENT AND DESIGN

2.5. TFET was established by a World Bank Board Resolution of December 19, 1999 (Resolution No. IDA 99-5) as a trust fund of the International Development Association (IDA, the World Bank entity dealing with low-income developing countries) with a transfer of US$10 million from the World Banks surplus and was subsequently expanded into a
14

President Memorandum World Bank Assistance to Timor-Leste, World Bank, November 19, 1999.

-26-

Multi-Donor Trust Fund (MDTF) in January 2000 (Resolution No. IDA 99-5/1) to facilitate mobilization of funds from other donors. The Trust is constituted of the contributions from bilateral and other donors as well as from the World Bank surplus and income from investing the contributions. 2.6. As in all other MDTFs, the World Bank (IDA) is the trustee, responsible for managing and using the funds, and for entering into contributing agreements with each donor. However, TFET establishes a precedent in that the ADB is assigned a substantial role as implementing agency alongside the World Bank, albeit without prejudice to the World Banks responsibility as the Trustee. 2.7. Another key characteristic of the Timor-Leste case is the special role of the UN. Elsewhere, the UN role has been primarily to prevent the resurgence of hostilities after the conflict was ended by international military intervention. By contrast, in the case of TimorLeste, as the territory was not yet a country, the UN became the legal governing authority with power to enact new laws and regulations and to amend, suspend and repeal existing ones. It also manages CFET, the trust fund established at the Tokyo meeting to finance the recurrent expenditures of the Timor-Leste administration, rehabilitation of administrative buildings, civil service capacity building and reconstruction in the justice sector mainly with donor contributions. IV. GOVERNANCE

2.8. Each contributor to TFET is entitled to appoint a representative to the TFET Donors Council created to oversee TFET operation. The Council meets at least semiannually15 to discuss and approve the work programmes submitted by the two implementing agencies which indicate projects and programmes proposed to be funded from TFET, the amount of TFET funding proposed for each such project and programme, the identity of the proposed recipient of TFET funding, and the identity of the Implementing Agency proposed to be responsible for such project or programme16. 2.9. The amount of TFET funding proposed in a work programme should not exceed the aggregate amount of contributions provided for in contribution agreements between the Trustee and contributors plus the amount of any contributions from the Bank and any investment income earned thereon, less aggregate amounts retained by the Trustee to administer TFET and to cover the costs incurred by the Implementing Agencies (see below) and of funding approved in prior work programmes. 2.10. TFET work programmes are subject to the approval of the TFET Donors Council. Decisions of the TFET Donors Council are made, to the extent possible, by consensus. In the absence of a consensus, each contributor is entitled to one vote for each US dollar equivalent, as determined by the Trustee, of its paid-in contribution to TFET and matters requiring a decision of the TFET Donors Council will be determined by the majority of the votes cast by contributors represented at the meeting of the TFET Donors Council.
15

In 2002, it was decided to hold the TFET Donors meetings only once a year. The World Bank still produces its six monthly Report of the Trustee, but the donors consultation is virtual, i.e. by e-mail for the December report, while an actual meeting takes place to discuss the May report. 16 From the start, the World Bank tried to involve Timorese in the preparation of the work programmes, including the selection of projects to be processed.

-27-

Contributors represented at the TFET Donors Council holding a majority of all the votes of the contributors shall constitute a quorum for the transaction of business at a meeting of the TFET Donors Council17.18 2.11. The objective of the semi-annual Donors Council meetings is to ensure that TFET is in line with the priorities of the contributors. Broader coordination meetings of donors assisting Timor-Leste (on the Consultative Group format) are usually held the day after the TFET Donors Council meetings. A donor coordinating committee has also been established in Dili. It is chaired by the World Banks Chief of Mission and provides a means of considering requests for TFET assistance between the Donors Council meetings. 2.12. In addition to the semi-annual Donors Council meetings, there are also semi-annual joint donor sectoral missions in which the Implementing Agencies and donors review project implementation progress. V. ADMINISTRATION AND PROJECT PROCESSING PROCEDURES

2.13. The World Bank, as Trustee, has sole responsibility for administering TFET, and receives 1% of the total contributions to cover its costs. As administrator, the World Bank is responsible for keeping appropriate records and accounts to identify TFET resources, the commitments to be financed out of TFET and the receipts as disbursements of funds in TFET. It should, as soon as practicable at the end of each fiscal year, furnish to the Executive Directors of the IDA, the ADB and each of TFET contributors: (i) report on the operations financed from TFET; and (ii) financial statements with respect to TFET, together with a management assertion of the procedures and controls used by the World Bank in administering trust funds. 2.14. The Implementing Agencies have sole responsibility for the use of TFET resources for the projects they implement. To help defray the costs incurred by the Implementing Agencies in the processing and monitoring of projects, they receive an amount up to 5% of the contribution funds calculated over the life of the TFET. 2.15. Eligible operational costs include: Short term consulting fees, travel and subsistence; Project design, preparation, supervision and evaluation costs, including time, travel and subsistence for World Bank staff; Office operational costs in Dili required for advising Timor-Leste and UNTAET, servicing missions and donor coordination; and Cost related to servicing the Trust Fund Donors Committee and reporting to the donors. 2.16. TFET does not cover the capital cost of establishing the Dili office. Similar arrangements were agreed with the ADB. Salaries and other costs already provided under
17

To the missions knowledge, this procedure has never been used. It should be noted that it gives the EC and the Member States a de facto veto power over TFET. 18 Resolution Amending the Trust Fund for East Timor, IBRD Resolution No. 99-8/1; IDA Resolution No. 995/1, World Bank

-28-

other trust funds and the cost of staff of other donor country agencies participating in missions cannot be sought from TFET. Donors Comments The use of administrative fees paid by donors has not been clear and transparent by the banks and there was never a rationale given for the decision to draw down TFET staff from the World Bank Country Office after Independence when over half the funds were still undisbursed. 2.17. Projects processed by the World Bank are administered in accordance with World Bank procedures, including selection of consultants and procurement. Similarly, ADB procedures, including consultant selection and procurement rules, apply to ADB-processed projects. The only exception in both cases is that the definition of member countries for procurement and selection of consultants includes Timor-Leste and all donors and members of international organizations that have provided contribution funds to TFET. 2.18. To accelerate project preparation, the World Bank would use streamlined procedures similar to those for ERA (Emergency Recovery Assistance: Operational Manual BP 8.50, August 1995) and LILs (Learning and Innovative Loans). Projects are initiated on the basis of a PID (Project Information Document), which is cleared by the Country Director for appraisal or pre-appraisal. The identification, preparation and pre-appraisal stages are usually combined. Inclusive of the PID review, these steps should normally be completed within five weeks. 2.19. Within one month of the appraisal missions return, the TM (Task Manager) prepares the PAD (Project Appraisal Document) for management review, while the lawyer drafts the legal documents. Given that TFET projects do not use Banks funds, the approval is delegated to the RVP (Regional Vice President). TFET projects PADs and Grant Agreements are made available to the Board for information only. There is no discussion and the Board cannot oppose a financing approved by the RVP. 2.20. Implementing arrangements consistent with World Bank guidelines and appropriate to each project would be set out in the PADs. Streamlined disbursement and procurement procedures, consistent with World Banks guidelines and used in other post-conflict situations, should be employed. To ensure accountability and transparency in the use of funds, projects should be designed consistent with World Bank financial management requirements and accounts are audited according with World Bank guidelines. Like any other World Bank projects, TFET projects would be monitored through field office and head office supervision and progress regularly reported to the donors19. VI. RESOURCES AND USES

2.21. At the Tokyo meeting, about US$148.0 million were pledged, including the US$10.0 million from the Bank surplus. Eventually, US$169.23 million were contributed, which
19

The Mission had asked the ADB for similar information to assess how it processes TFET projects. We have not received the requested information.

-29-

together with investment income of over US$8.34 million bring the financing available for projects to US$177.57 million20. 2.22. The largest contributors were the European Commission (US$50.67 million), followed by Portugal (US$50 million), Japan (US$28 million), Australia (US$12 million), the UK and the World Bank (each with about US$10 million). Together, the European Union (the EC plus Member States) accounted for 69% of contributions received by TFET (Table 21). Table 2-1: Actual and Pending TFET Pledges and Contributions as of April 9, 2004 (US$ million)
Donor Portugal (a) European Commission Japan TFET Japan (PHRD) (b) Australia(c) United Kingdom World Bank Finland Norway United States of America Ireland IDA-PCF New Zealand Italy Total Commitment Authority Investment income
(d)

Pledges 50.00 50.67 28.00 12.43 10.18 10.00 3.72 2.39 0.50 0.47 0.38 0.36 0.21

Contribution Agreement 50.00 50.67 25.90 2.00 12.43 10.18 10.00 3.72 2.39 0.50 0.47 0.38 0.36 0.21 169.23 8.34

Cash Received or Promissory Notes Lodged 50.00 50.67 25.90 2.00 12.43 10.18 10.00 3.72 2.39 0.50 0.47 0.38 0.36 0.21 169.23 8.34 177.57

Financing Available for Current Work Programme 177.57

(a) (b)

Portugal has contributed US$30 million in cash, with US$20 million remaining against a promissory note. Japans pledge was US$28 million, of which US$2 million was awarded from the Japanese-funded Policy and Human Resource Development (PHRD) Trust Fund prior to the establishment of TFET. (c) Australia has also contributed US$650,000 in cofinancing of the first TFET education programme. (d) Actual income based on an estimated average monthly balance of US$22.4 million and is as of March 31, 2004.

2.23. As of April 9, 2004, 24 grant agreements had been signed, with a value of US$158.4 million. In accordance with the TFET resolution, 6% of total donor contribution funds committed either through cash or promissory notes have been provisioned for project preparation, supervision, and administrative costs, bringing total commitments to US$168.4 million. In addition, supplemental grants for four projects totalling US$8.9 million agreements are under preparation, bringing total project commitments for the current work programme to US$177.3 million.
20

These figures exclude (i) US$1 million given by the World Bank through the Post-Conflict Fund (PCF) for financing the participation of Timor-Leste experts to the Joint Assessment Mission, equipment for Timorese and an HR survey of skilled Timorese technicians; and (ii) about US$9 million provided by the ADB as technical assistance for project preparation.

-30-

2.24. Since inception, TFET has financed 23 projects. Ten have been completed and thirteen are still active. Closing dates have been extended for eight projects. 2.25. As of April 9, 2004, there remains about US$300,000 in the Fund. The Government has proposed to allocate these remaining funds to the Priority Power Investment Project for the urgent repair of the MAK 4 generator and to the Dili power distribution system. 2.26. As of March 31, 2004, US$135.4 million have been disbursed, but given the slowdown of disbursements since December 2002, it is estimated that the remaining US$32.2 million would be totally disbursed by mid-2007 only, at which time TFET would be closed.

-31-

CHAPTER III: EVALUATION

3.1. While financing post-conflict reconstruction is as old as the World Bank, which has been created to finance the reconstruction of Europe and Japan, the use of MDTFs for postconflict assistance is relatively recent, but is growing fast. Recent cases with nation building aspects similar to Timor-Leste include Afghanistan, Bosnia, Cambodia, Kosovo, Sri-Lanka, and West Bank and Gaza. Other cases of interest include El Salvador, Haiti, Guatemala, Mozambique, and Uganda. 3.2. It is said that the mechanism is efficient and facilitates resource mobilization and donor coordination. The use of the same procedures (generally World Bank procedures) for project preparation, disbursement, procurement and monitoring is a major advantage as countries in post-conflict situations typically have limited understanding of aid donors procedures and policies and have low development project implementation capacity. 3.3. In the sections below the major achievements and shortcomings of the mechanism in the Timor-Leste experience are analysed, taking into consideration the ECs interests, development policies and objectives. Originally, the evaluation, which was to take place in 2002, was intended as a mid-term review, with its findings to be used to correct shortcomings and, if needed, reorient TFET work. Coming as it is now two years later, when TFET funds are totally committed, the lessons drawn will serve to improve the mechanism for future use by the European Commission (and other donors) in other post-conflict situations. The evaluation follows the ECs procedures and focuses on the following key evaluation criteria: Relevance, Effectiveness, Efficiency, and Sustainability21. I. A. RELEVANCE: WAS TFET THE RIGHT MECHANISM? Resources Mobilization

3.4. As shown in Table 2-1 in Chapter 2 above, TFET received US$169 million for the reconstruction of Timor-Leste. On a per capita basis, it is the highest amount of aid mobilized through MDTFs for reconstruction (Table 3-1). The sources of aid were diversified with contributions coming from four continents. Table 3-1: Reconstruction Aid Per Capita Mobilized Through MDTF in Selected Post-Conflict Countries
Population (Million) 0.8 2.5 3.7 28.0 GDP per Capita (US$) 430 1,433 987 N/A Aid per Capita (US$) 212 118 41a/ 15

Mostly IDA credits SOURCES: Calculated from (i) S. Schiavo-Campo Financing and Aid Arrangements in Post-Conflict Situations, Social Development Papers No. 6, World Bank, June 2003; and (ii) Klaus Rohland and Sarah Cliffe The Timor-Leste Reconstruction Programme: Successes, Problems and Trade-off, Social Development Papers No. 2, World Bank, November 2002
21

a/

Timor-Leste West Bank and Gaza Bosnia and Herzegovina Afghanistan

As noted in the TORs, Impact is not a criterion in this interim evaluation.

-32-

3.5. However, many bilateral donors chose to help Timor-Leste directly and outside of the TFET mechanism. Even those who were major TFET contributors (e.g., AusAID, JICA and Portugal) also provided substantial project assistance directly to Timor-Leste. On the other hand, USAID, a small TFET contributor, provided most of its assistance outside of TFET. Other important donors but non-TFET contributors include Canada, China and Germany. 3.6. This strategy reflects the use of aid by donor Governments to strengthen bilateral relations, a goal which cannot be achieved through TFET. Recently, as TFET is phasing out, the EC also provided assistance outside of TFET. However, in doing so, it chose to cofinance projects with TFET (see page 42). 3.7. The estimated total resources available for Timor-Leste reconstruction from various sources are shown in Table 3-2 below. Table 3-2: Available Resources for Timor-Leste Reconstruction and Development, 2000-03 (US$ million)
Sector 1. Community Development/Safety Net 2. Education 3. Health 4. Agriculture 5. Infrastructure 6. Private Sector Support/Economic Management 7. Public Administration 8. Land and Property 9. Judiciary and Law Enforcement Grand Total JAM 30.0 57.8 40.2 24.4 93.1 16.2 38.1 --2.9 302.7 TFET 22.0 46.0 28.3 16.7 56.9 16.2 ------186.2 UNTAET ------------22.8 1.6 10.2 34.6 Other1/ 5.3 48.8 10.7 21.3 72.4 1.6 ------161.9 Total 27.3 94.8 39.0 38.0 129.3 17.8 22.8 1.6 10.2 380.7

SOURCE: World Bank estimates 1/ Bilateral and non-TFET multilateral 2/ Amount slightly different from Table 1-1 because some TFET resources were not yet allocated at the time of the estimates.

3.8. The table shows that (i) the resources available for reconstruction exceed the needs estimated by the JAM by more than 25%; (ii) non-TFET resources are almost as large as TFET; and (iii) sector allocation of TFET and non-TFET funds is similar, except for community development. The Evaluators Comments TFET has been successful in mobilizing resources for Timor-Leste reconstruction. However, almost half of the funds available to the country for that purpose during the 2000-03 period was provided and channelled to development projects outside of the TFET framework by donors and aid agencies using their own policies and procedures. This reduces one of the mechanisms main advantages, i.e., rapid, simpler and more efficient project processing and implementation resulting from using the same procedures for project preparation, procurement, disbursement and monitoring; and The similar sector allocation of TFET and non-TFET projects seems to support the suggestion that TFET project work (including the JAM and joint donor sector missions) has influenced the allocation of non-TFET resources.

-33-

B.

Fragmentation of Funding Mechanisms

3.9. As mentioned in the Introduction, there are seven different funding mechanisms: (i) UNTAET-administered CFET; (ii) the TFET; (iii) the assessed contribution budget of UNTAET; (iv) projects financed by UN agencies; (v) assistance provided directly by bilateral donors; (vi) support and work done by or through NGOs; and (vii) TSP through which the World Bank and other donors finance the Government current budget (CFET) after Independence. The Evaluators Comments In all other World Bank-managed MDTFs, there are also parallel sources of development finance. This is unavoidable as aid donors have their own objectives. However, unlike the Timor-Leste case, they are relatively small compared to the funding provided under the MDTF and generally do not strain the governments project management capacity.

C.

Relationships between Main Responsible Institutions

UNTAET and the World Bank 3.10. As mentioned above, the Tokyos donor conference created two trust funds, the CFET managed by UNTAET, the governing authority during the transition period and TFET for which the World Bank is the Trustee. This resulted in a complete separation between recurrent cost financing (CFET) and reconstruction financing (TFET). Despite the production of a combined sources budget based on all available funding sources, this was not sufficient to integrate investment and recurrent expenditure decision-making. 3.11. At the beginning, CFET was also administered in New York and functioned on an annual cycle, while TFET programmes were prioritised on a semi-annual cycle and were implemented by government services with intensive support from the World Bank and ADB. This unduly complicated budget execution. 3.12. Compounding the problem were the tensions between the World Bank and UNTAET during the transition period. As Klaus Rohland and Sarah Cliffe explained in their Paper The Timor-Leste Reconstruction Program: Successes, Problems and Tradeoffs22, cooperation between the Bank and UNTAET was initially hampered by UNTAETs dual role as the United Nations Mission in Timor-Leste and as the Transitional Administration in Timor-Leste. While the former implied mission structures and a role in anchoring and promoting the operations of UN agencies in Timor-Leste, the latter suggested that UNTAET as government should establish full governmental structures and act as a government counterpart to UN agencies, International Financial Institutions (IFIs) and donors. By extension, relations between the Bank and UNTAET would be governed by their roles and mutual obligations as partners within the UN system whenever UNTAET would be seen acting as a UN mission. The usual business model between the Bank and recipient governments would prevail whenever UNTAET was seen as the Government. For the way that the Bank does business, however, to treat UNTAET as Government was crucial, since IDA projects are implemented through
22

Conflict Prevention and Reconstruction Unit, Working Paper No.2, November 2002, World Bank.

-34-

national government counterpart structures23. The lack of understanding of one anothers roles hampered the implementation of the reconstruction programme in the early days. In particular, it: made Timorese involvement in the decision-making process more difficult as UNTAET did not want to rely on CNRT, the only organised Timorese group with some capacity; completely separated reconstruction projects prepared and implemented under TFET from their induced recurrent expenditures administered by CFET; and finally contributed to the failure of capacity building, a major objective, given Timor-Lestes extremely weak capacity. The division of work between the World Bank and UNTAET gave the latter the responsibility for capacity building but, given the relationships between the two institutions in the transition period, no cooperation was established in that crucial area. The problem was exacerbated by UNTAETs decision to recruit government employees from bottom up, which meant that until late 2001, there were no Timorese civil servants in senior or middle-level positions. For its part the World Bank accomplished some capacity building through TFETs PMUs (Project Management Units). However, in the absence of a strategy for building capacity in the central administration, these efforts could not be sustainable.24 Comments by the World Bank Although there was no possibility of setting up the system differently, given that UNTAET was the legal governing authority of Timor-Leste at that time and therefore had to administer the recurrent budget (CFET), earlier efforts to build understanding between UNTAET, the IFIs, and other donors on their respective roles, financing mechanisms and constraints, would have facilitated a smoother transition from early planning to implementation of the reconstruction programmes. Within the Bank, better understanding of the legal mandates, structures and decision-making procedures of the United Nations would have assisted in more coordinated joint planning. Together with formalization by UNTAET of the central role of Timorese representatives in reconstruction, this could have enhanced the speed and sustainability of initial reconstruction efforts As the transition progressed, UNTAET gradually brought Timorese representation into government and moved its structures away from those of a UN mission towards those of a normal governmentcooperation was smooth and efficient 25. Within the Timorese administration, had the same people (technical assistants or civil servants) in the ministries handled both CFET and TFET, many misunderstandings would have been avoided and the problem greatly alleviated.
23

Examples are difficulties in implementing the usual role of the IMF and the Bank in discussing the UNTAET/ETTA budget prior to June 2000; the shape of UNTAET structures during 2000, which resembled a mission more than a normal government and were awkward counterparts for reconstruction programmes; and the question of whether procurement under TFET financed projects should be undertaken by a normal government procurement department under IDA procurement guidelines as mandated by TFETs donors or whether Timor-Leste procurement rules prepared by UNTAET with the support of World Bank-financed consultants and enacted by UNTAET would apply. 24 See Annex on TFET and the rural sector for a discussion of the tensions between UNTAET and the World Bank during the negotiations the of Community Empowerment and Local Governance Project (CEP) 25 Klaus Rohland and Sarah Cliffe, op. cited, page 6

-35-

ADB and the World Bank 3.13. TFET is the first MDTF in which the World Bank shares implementation with another aid agency. ADBs involvement was decided at the Tokyo donors meeting, which also clearly defined the areas of responsibility of the two Implementing Agencies: Infrastructure rehabilitation (including public utilities), WSS (water supply and sanitation) and microfinance for the ADB and agriculture, community development, education, health private sector development and economic management for the World Bank. Donors Comments There was some overlapping of responsibility between the two Implementing Agencies, particularly in the area of micro-credit (World Banks CEP and SEP projects and ADB Microfinance project). The idea of providing credit, sometimes at very high interest rates, in parallel with the provision of grant, confused beneficiaries and may have long-term consequences for the development of a credit market for the poor (see section on credit below). The recurrent cost implications of the ADB infrastructure projects also appear to have been overlooked.

The Evaluators Comments Although there have been occasional tensions, particularly during the first two years when disbursements were still processed from Washington, causing delays in some of the ADB projects, cooperation between the World Bank and the ADB was generally good. The fact that the two Implementing Agencies had different areas of intervention clearly defined and delimitated did help. The use of different procedures for project processing and procurement did not seem to be a problem. D. Donor Coordination

3.14. TFET had a comprehensive donor coordination mechanism. First, the needs assessment was made by a Joint Assessment Mission (JAM) which was truly joint in that it included (i) an equal number of Timorese and international experts; and (ii) five donor countries, four UN agencies, the EU, the ADB and the World Bank. A concurrent IMF accompanied the JAM to assess the macroeconomic framework. 3.15. Second, TFET created the Donors Council, which meets twice a year to discuss and approve work programmes submitted by the Implementing Agencies. At the request of some donors, these meetings were reduced to one per year after Independence in May 2002. 3.16. Third, TFET instituted the six-month joint donor missions in key sectors to assess sector developments and prepare projects to be submitted to the Donors Council for approval. The missions were uneven in their coverage. There were regular missions in health, less regular in education, agriculture, community development, and WSS, and even fewer regular exchanges in infrastructure and private sector development. 3.17. Fourth, there were also monthly field coordination meetings, which were discontinued in 2003 at the request of some donors. -36-

3.18. Reporting to donors is also comprehensive. Donors are provided with bi-annual reports containing summary information on the overall TFET operation and on implementation progress of individual projects. Following comments by some donors, the World Bank enhanced the coverage of its project reporting. Starting August 2000, monthly TFET updates were posted on the web site and, since November 2001, TFET donors also receive a monthly TFET portfolio management report, which presents the performance ratings for every project and outlines salient implementation issues. They are also copied Aide Mmoires of all supervision missions, which give a detailed assessment of the situation of the projects, including their financial management. Since September 2003, at the request of TFET donors, including the EC, the monthly portfolio report has been enhanced by a matrix update, which presents the ratings in twelve categories, including development outcome and implementation progress, as well as salient outcome and management achievements and weaknesses over the preceding month. Views of the Donors In general, donors consider the donor coordination arrangements under TFET to be satisfactory. ANAO (Australian National Audit Office) found that there had been a high and sustained level of communication, consultation and coordination among donors in Timor-Leste. This has contributed to the effectiveness of the international aid effort to assist Timor-Lestes recovery and political transition and also assisted AusAID in the management of Australian aid activities26. ANAO also found the JAM to be innovative and effective.

The Evaluators Comments There is no doubt that TFET has facilitated donor coordination and has helped keep the momentum in reconstruction activities during the transition period. TFET did not make special efforts to reach out to donors outside of the mechanism. However, this did not appear to have a significant impact, as most important outside donors were also TFET contributors. A measure of the achievement of the TFET donor coordination process is the success of the Dili Donors Meeting where pledges sufficient to meet the estimated financing gap in the Governments National Development Plan were made. All the donors supporting Timor-Leste reconstruction remained engaged. E. Donor Involvement

3.19. From discussions with representatives of donors, UNTAET, World Bank and the Timorese authorities involved with TFET until today and review of documents issued by TFET (Reports of the Trustee, Reports/Aide Mmoires of the joint missions, etc.), it appears that donor participation was uneven. Generally, most donors participate in the bi-annual Donors Council meetings. With regard to joint donor sector missions, donors who have a field office such as AusAID, the EC, Portugal, USAID and JICA participate. Some also send
26

Aid to Timor-Leste, Audit Report No. 20 2003-2004, Australian National Audit Office.

-37-

consultants or experts from headquarters: AusAID (most of the time and for most sectors), EC and Portugal (sometimes for health), UK (sometimes for transport) and JICA (sometimes for education). 3.20. However, there is little record of disagreements/divergences with the conclusions of the World Bank-led sector missions or work programmes prepared and presented by the World Bank and the ADB. It also appears that donors participation was more intense and active in the first two years (2000-2001) than at present. 3.21. Donors also influenced project implementation. By refusing further funding (outside of TFET) for the Community Empowerment and Local Government Project (CEP), the EC and AusAID were instrumental in ending the project at its normal closure in early 2004. Conversely, the Government experienced donor interference in the management of some projects. For instance the ICR for the ESRP mentions under the heading Poor handling of pressure from Donors that: 3.22. The following pattern of events occurred more than once: The donors would apply pressure on the Bank regarding slow progress; The Bank would send officers into the ESRP to accelerate progress; The Bank would get involved in micro management of the Project

(i) (ii) (iii)

3.23. The Bank ought to have shielded and defended the Project from the panic of the donors. Government (PMUs)s Comments The World Bank is dominating joint donor missions in practically all sectors. The missions TORs were written by the World Banks Task Managers and generally shared with donors at the last minute, making comments and changes very difficult. Missions Aide Mmoires were also written by the World Banks Task Managers and generally did not reflect divergent point of views among donors.

The Evaluators Comments The usefulness of donor involvement is linked to the ability and readiness of the donors to contribute staff at critical points (e.g., in joint donor missions). The 6% administrative fee paid to the Trustee and Implementing Agencies does not absolve the donors from further intellectual contribution to supervision of project implementation. However, the balance between a useful independent view (CEP) and what can be seen as interference (ESRP) is delicate. Diminished donor interest after 2002 may be attributed to TFET fatigue, as the mechanism is being phased out and as donors are more interested in new initiatives such as TSP (Transition Support Programme) and SIPs (Sector Investment Programmes). The EC is fortunate to have a competent and dedicated Head of the Dili Office who participates in virtually all TFET related-meetings and makes the ECs positions known.

-38-

The Dili Representative Office needs to be strengthened with specialists from Headquarters or from the Jakarta Delegation when joint-donor sector missions are in Timor-Leste. Such support should be planned and budgeted, including for locally recruited staff. EC sector missions should also have been scheduled to coincide with World Banks missions (not only overlap for a few days) in order to minimize duplication and get the ECs views known and taken into account in the work. Finally, the deconcentration process, by transferring responsibility for Timor-Leste to the EC Delegation in Jakarta during TFET implementation, affected continuity and resulted in some institutional memory loss with regard to the Timor-Leste dossier, particularly with regard to the health sector agreement and an agreement with the UN.

F.

Responsiveness to Beneficiary Countrys Needs

3.24. TFETs capacity to mobilize substantial resources, its efforts to minimize fragmentation of funding mechanisms and its comprehensive donor coordination system are very valuable features for countries in post-conflict situation like Timor-Leste. However, the most attractive feature of this type of arrangement is that the beneficiary country has to deal with only one donor. This simplifies dialogue among the parties involved, facilitates the coherence of the reconstruction/development programme and makes adaptation to changing conditions easier. G. Relevance of the projects

3.25. The 23 projects financed to date have been overall relevant to the task of reconstructing Timor-Leste. The almost equal emphasis put on the social, the productive and infrastructure sectors was adequate. As examples of the relevance of TFET projects, we present below the logical framework and the analysis of the relevance for the projects in the case study of CEP and ARP (see Annex 1). Community Empowerment Projects (CEP) 3.26. Relevance Analysis (from Annex 1). The strategic choice of implementing rapidly a scheme of block grants to the local communities was the correct response to the emergency situation in East Timor. It is indeed remarkable that within a few months of the emergency CEP had been able to reach all the communities in the country. There was a need for local bodies to handle the grants. The system of development councils was, however, over designed and removed from the realities on the ground. The relevance of the credit component is questionable in a post-conflict (see below effectiveness).

-39-

Table 3-3 CEP Logical Framework

Hierarchy of objectives Key Performance Indicators Critical assumptions and Risks

Project development objectives (1) Establish development councils (2) Poverty alleviation Year 1: 20 sub districts Year 2: 60 sub districts Transfers to villages US$1,075,000 in first Cycle Cycles 2, 3 and 4: US$2,150,000 in 62 sub districts Credit recovery >80% Procurement and physical distribution capacity is low Community plans may be taken over by elite Poor technical quality of investments Revised local government structure is fully accepted by UNTAET and CNRT UNTAET capacity within District Administration is weak Community planning capacity is low

Agriculture Rehabilitation Projects (ARP) 3.27. The logical framework for the three ARP projects is quite elaborate and has become more so with each new project. Table 3-4 ARP Logical Framework

ARP I

ARP II Development Objectives

ARP III

Improve food security of selected poor households Increasing agricultural production in selected Project areas and promoting rural growth.

Improve food security of selected poor households Increasing agricultural production in selected Project areas and promoting rural growth.

The project aims to strengthen the capacity of the Ministry of Agriculture, Forestry and Fisheries (MAFF) and its development partners to assist rural communities in increasing their production and income in a sustainable way.

-40-

Project Components
Priority productive assets restoration including the financing of cattle vaccination campaigns, the provision of livestock (100 000 chicken and 2000 head of cattle) and tools to poor families who had suffered losses in the violence. Irrigation/rural infrastructure rehabilitation and maintenance including feasibility studies, reconstruction, rehabilitation and maintenance of irrigation schemes covering 7000 ha and the rehabilitation and maintenance through community participation of a minimum of 100km of agricultural access roads. The establishment of up to eight commercially operating Pilot Agriculture Service Centrers (PASC), short term and longterm training of PASC staff design of an agricultural research laboratory, land/soil mapping studies27. (PAD page10) Pilot participatory development and natural resources management. Rapid infrastructure rehabilitation Services to farmers with three sub component: -Information to farmers -Sustainable animal health services - Pilot Agricultural Service Centrers Policy and Strategy Development (PAD page 8) Participatory Development and Natural Resources Management, would support MAFFs main program of assistance to upland and coastal communities, aiming to strengthen their resilience and make them less vulnerable to external shocks. Irrigation Rehabilitation and Management would support MAFFs main program of assistance to rural communities in lowland irrigated areas. Services to Farmers would help institutionalise three essential services to farmers: (a) information; (b) animal health; and (c) agribusiness support. Program Management, would assist MAFF in developing key national policies, strategies, and legislation, strengthen its managerial and technical capacity, and carry out project management. (PAD page 2)

Key Project Outcome Indicators

Nutrition of project beneficiaries improved. Increased availability of eggs, poultry and rice at the house hold level. Off farm employment opportunities increase in Project areas Areas under irrigation and access to main roads increase Yields in rice and maize increase by about 10% at full development. Average rice yields increased from 1.5MT/ha to 3.0MT/ha in five years in irrigated areas rehabilitated by the project. Production of other food crops such as maize cassava and beans at 120% of pre-conflict (1999) levels by the end of the project. About 20% reduction in the number of households with critical low food reserves at the end of the project. All major project activities integrated into MAFF programs and funded by the Government by the end of the project. Average rice yields in rehabilitated irrigated areas increasing from 1.5 to 2.0 metric tons per hectare per crop by the end of the project (EOP). Cropping intensity in Caraulun scheme at 160% by 2008. 70 percent of farmer beneficiaries satisfied with the services received.

27

This component included also the establishment of community radios which was eventually transferred to CEP I

-41-

Major Risks
Equitable distribution of assets Rehabilitated assets are well maintained. Irrigation water is available Prices for major export commodities drop Political disturbances interfere with project implementation Assets rehabilitated are maintained Farmers cooperate in WUA. Farmers are willing to pay for veterinary services. Farmers are willing to pay for input. Maintenance is funded adequately Occurrence of natural disasters Farmers do not have macro economic incentives to increase production. Farmers do not change their current practices following irrigation rehabilitation MAFF staff lacks the capacity to ensure that agricultural information is conveyed in a timely manner, accurately, and effectively Rice seed producers may not be able to produce good quality seed and/or oil mills may become unprofitable Trained counterparts fail to apply new skills effectively or move on to other positions

3.28. Relevance Analysis (from Annex 1). The concept and the content of the ARP projects are overall relevant to the development objectives set out and the emergency situation being addressed. The only exception may be the PASC component. It is not clear how it was to respond to the vital needs of the sector. It was fortunately curtailed at the mid-term review of ARP II. It is also doubtful that the impact of the cattle distribution was worth the effort. 3.29. The inclusion of the National Resource Management component in ARP II and III at a time when the difficulties of CEP were well documented appears questionable. The mission had no access to background documents justifying this component. 3.30. The ARP I PAD refers to a US$2.5 million agricultural credit component, which at the end was included in the project. The evaluators do not know why the agricultural credit was deleted from the TFET programme since they did not find any reference to this component other than the one in the PAD. H. Alternatives to TFET (for the EC)

Bilateral financing 3.31. Some small donors have taken this approach, e.g., Malaysia, Germany and France. Others have adopted the strategy to be both inside and outside (Australia, the US, Portugal, Japan). This allows them to have a seat at the table and at the same time pursue national objectives and have more visibility. One TFET contributor noted that participating in [joint] missions and mission reports have been a valuable mechanism for providing international donors with information on the performance of TFET projects and the aid activities of other donors on a sector-wide basis. More recently, they have also enabled donors to contribute to government planning for key sectors.

-42-

3.32. In view of the large size of its programme for Timor-Leste, to follow that approach, the EC would have had to develop a strong on-the-ground capacity. In the present budgetary circumstances, this does not look like an attractive alternative. World Bank-managed trust fund with the EC as the only donor 3.33. Recently, the EC has used this approach for health28 and agriculture29. Both projects are co-financed with TFET. The approach ensures visibility, while limiting local capacity requirements. However, it remains to be seen whether such an approach would have been feasible in 2000 when TFET was being set up. The argument against the arrangement would have been the unwarranted increased complexity of the World Bank work at a time of emergency. Financing through UN Agencies 3.34. In addition to its co-financing the Agricultural Rehabilitation Project III (ARP III) with TFET, the EC Rural Development Programme in East Timor includes five projects financed through various UN agencies: (i) bridge construction in the East with UNDP/UNOPS (United Nations Office of Project Services); (ii) poverty reduction also with UNDP/UNOPS; (iii) skills training with UNDP/ILO (International Labour Organisation); (iv) womens rights with UNDP/UNIFEM (United Nations Development Fund for Women); and (v) community stabilization with IMO (International Migration Organisation). This allows the EC to benefit from the comparative advantage of the various institutions. With the exception of the first project ( 6.59 million), the amounts allocated are relatively small. It is unlikely that such an approach would credibly replace a World Bank-managed Trust Fund in term of reconstruction impact. EC managed multi-donor trust fund? 3.35. This arrangement has never been tried, but is not unconceivable in areas of strong EC influence, such as Africa and the Middle East and in situations where a political approach may be preferred to the World Banks technocratic approach. However, even under the right circumstances, the EC must show flexibility and strong technical expertise. The Evaluators Comment Among the alternatives discussed above, TFET appears to be the most adapted to the situation.

28

Supplemental Grant to the Second Health Rehabilitation and Development Project (SHRDP) in an amount of 16.5 million. The Administration Agreement between the EU and the World Bank was signed in October 2003. The Grant agreement between the World Bank and the Government signed on January 16, 2004. 29 Supplemental Grant to the Third Agriculture Rehabilitation Project in an amount of 6.9 million. The Administration Agreement between the EU and the World Bank is under preparation at the time of finalization of this report (October 2004).

-43-

II. A. 3.36.

EFFECTIVENESS: TFET Portfolio

RESULTS ON THE GROUND

TFET portfolio to date includes 23 projects, of which 10 have been fully disbursed. Table 3-5: Existing and Expected TFET Grant Agreements as of April 9, 2004
Project Date of Grant Agreement Project Amount (USD m) 1 0.38 9.00 0.49 27.80 4.79 12.70 13.89 6.79 4.50 4.00 0.60 8.50 4.50 12.60 13.90 7.50 8.00 2.00 1.00 9.00 1.50 0.24 1.70 3.00 158.38 6.96 1.38 1.67 168.39 6.70 0.20 0.20 0.15 1.63 8.88 177.27

Human Resources Survey January 18, 2000 Community Empowerment February 21, 2000 Dili Community Employment April 3, 2000 Emergency Infrastructure (ADB) April 5, 2000 Small Enterprise [1] April 13, 2000 Health [1] June 6, 2000 Education [1]1 June 21, 2000 Agriculture [1]1 June 21, 2000 Water and Sanitation [1] (ADB) August 15, 2000 Microfinance (ADB) December 18, 2000 Economic Capacity Building February 26, 2001 Community Empowerment [2] May 21, 2001 Water and Sanitation [2] (ADB) August 27, 2001 Health [2] October 15, 2001 Education [2] October 29, 2001 Small Enterprise [2] October 29, 2001 Agriculture [2] October 29, 2001 Emergency Infrastructure supplemental (ADB) October 31, 2001 Hera Port, part of Agriculture 2 (ADB) November 27, 2001 Infrastructure [2] (ADB) May 18, 2002 Community Empowerment [3] July 1, 2002 Petroleum Technical Assistance December 10, 2002 Petroleum Technical Assistance [2] July 30, 2003 Agriculture [3] January 26, 2004 Total Value of Signed Grant Agreements Project preparation and supervision costs (allocation to date) Outstanding preparation and supervision costs2 Fiduciary and management fee3 Total Commitments to Date Under Preparation/Expected Education [3] National Accounts Updating supplement to Econ. Inst. Capacity Bldg. Small Works Programme in Health supplement to Health [2] Efficient Fuel Acquisition supplement to Econ. Inst. Capacity Bldg. Priority Power Investments Project Total Commitments for Expected Grant Agreements Total Work Programme to Date Notes:
1. 2.

3.

Project Amount equals the total amount of the Grant Agreement (US$) for ongoing projects, or the total disbursed amount (US$) for closed/completed projects. Outstanding preparation and supervision costs eligible under the 5 percent cap over the life of TFET. Calculation is based on total donor contribution funds (cash and promissory notes) into TFET of US$166.85 million and does not include additional World Bank and ADB funds used to support project costs over TFETs life. A one percent fiduciary and management fee is charged on total donor contribution funds received into TFET to cover trust fund administration.

-44-

3.37. Closing dates have been extended for eight projects and grant supplements to four projects are being finalized. One new project is under preparation, the Priority Power Sector Investment Project. 3.38. All projects, whether in the pipeline or active, are presently being aligned with GoTLs Road Map for Implementation of the National Development Plan (NDP), the Stability Priorities and the Sectoral Investment Plans (SIPs). Progress towards SIP alignment is most notable in the Third Agriculture Rehabilitation Project (co-financed with the EC), the restructuring of the Second Small Enterprises Project, the preparation of the supplement to the Fundamental School Quality Project, and the preparation of the Priority Power Investment Project. 3.39. Grant approval took place in three major waves as can be seen in Table 3-6 below: Table 3-6: Grant Approval
Amount approved (US$ million) 84.5 58.6 10.5 153.2 % of TFET resources 48 33 6 100

2000 2001 2002 Total

Note: Excluding the 6% fees paid to the Trustee and the Implementing Agencies SOURCES: Reports of the Trustee

3.40. The first wave in 2000 was for 10 projects for a total of US$84.5 million, equivalent to 48% of the funds in TFET. It included allocations for all the major priorities identified in the JAM, with the largest amounts going to emergency infrastructure, education, health and community empowerment. With the exception of microfinance, all the projects were approved before August 2000, i.e., less than eight months after TFET establishment. 3.41. The second round of approvals (supplemental or second phase in the December 2000 Report of the Trustee) for US$58.6 million, or about 33% of the TFET funds, took place in the second half of 2001. With the exception of the Economic Capacity Building grant, they consisted of extensions of the projects started under the first wave. 3.42. The two waves were necessary because of delays in donors releasing the committed funds and TFET Rules. To a certain extent, the forced phasing was used to move towards sectoral policy development. As reported in the December 2001 TFET Trustee Report: Second generation programmes in health, agriculture and education were agreed by the new Government and signed in September [2001]. These programmes were appraised and negotiated before the [constitutional] elections, and the design ratified with incoming Government ministers. These programmes continue to support physical reconstruction work, but have added components that support longer-term sectoral policy development, in line with the Government priority to develop a medium-term poverty reduction strategy before independence. Sectoral policy studies that have been completed or are scheduled for completion by June 2001 include a national accounts exercise, a hospital assessment, a school mapping exercise, analysis of education financing and the health sector regulatory framework, a demographics study, a fisheries baseline survey and a forestry policy and strategy paper. -45-

3.43. The ICR of ARP I went further and suggested that the financial constraints imposed by the donors could have been used by the Implementing Agencies to improve the quality of the second phase projects. The PAD for ARP I appraised a first grant of US$6.8 million and a second grant for US$11.4 million and bemoaned the lack of funds in TFET, which forced a split approach. One of the findings of the ARP I ICR was that phased programs are appropriate, but avoid overlaps between phases. A phased approach to agriculture rehabilitation is appropriate, provided there is sufficient time between phases to review and apply lessons learned. 3.44. A third wave of approvals took place in mid 2002, i.e., after Independence, for a total amount of US$10.5 million, or 6% of the TFET funds and were primarily for infrastructure rehabilitation. Since then another US$2 million has been approved, while four projects worth about US$ 9 million are under preparation. B. Project Processing

3.45. The two Implementing Agencies were diligent in appraising and negotiating the first projects. The Trustees reports mention that project processing took 3.5 month on average, compared to a 15 months average for the World Bank30. It should be noted that the Education Supplement has been in the TFET work programme since December 2000 and this would indicate a much longer elapsed time. Most likely it is an exception. Table 3-7 summarizes the data obtained from the World Bank on the processing and supervision of its TFET projects. Table 3-7: Average Processing Time and staff inputs of World Bank-TFET Projects31
Appraisal/negotiations Staff weeks 24.5 32.0 28.3 Time elapsed (Months) 2.5 5.1 3.8 Fiscal 2000 13.5 N/A 13.5 Implementation Staff weeks Fiscal Fiscal 2001 2002 26.2 17.0 N/A 7.16 26.2 12.2

TFET Wave I (WB) TFET Wave II (WB) All TFET (WB)

SOURCE: The World Bank

Fiscal 20003 5.1 14.5 9.9

3.46.

The following comments can be made on the basis of the above table:

o The processing time of the first wave projects was indeed fast. To a certain extent, this was due to the fact that TFET projects do not have to be approved by the World Bank Board, being financed with non-World Bank resources;
30

It is worth mentioning that in the evaluators interviews, some donors were not impressed by the World Bank performance. Four or five months may be rapid by World Bank standards, but was not all that impressive in an emergency situation. Their preoccupation went beyond processing time. They were concerned by that the actual disbursement reached beneficiaries on the ground and not only the UNTAET special account. 31 Although we asked we were not given the data from the ADB to compare ADB TFET-managed projects. We also asked the ADB whether, like the World Bank, it had put into place special procedures to process TFET projects. At the time of report finalization, the information was still not received.

-46-

o Processing time and staff input increased by 100% and 33%, respectively for the second wave projects. The production of second wave PAD32 represented a cost, which must have exceeded US$250,000 per project. This amount could have been largely saved if the first wave PAD had been used for second wave projects33; and o Staff time devoted to implementation was quite high in the early years to trend downwards towards the norm (around 8-10 staff weeks per project/year). The Evaluators comments Full PADs were produced for all TFET projects. World Bank Task Managers complained that the appraisal documentation requested for Management approval was too voluminous and should have been simplified for these emergency projects. This observation is particularly valid for the second wave projects where little changes were incorporated in the design compared to the original project. Cases in point are the CEP and ARP projects. The changes could have been recorded in a simple memorandum to management while keeping the original PAD; The evaluators therefore note that the statement in the Memorandum to the Executive Directors on World Bank Assistance to Timor-Leste dated November 19, 1999 that project preparation would be carried out on an accelerated basis in order to provide near term reconstruction assistance to this post-conflict area was not totally followed through and that no documentation simplification ever took place. Some sectors (e.g., health) did used streamlined procedures similar to those for ERL (Emergency Recovery Loans: Operational Manual BP 8.50, August 1995). Project approval were also delegated to the RVP as in LILs (Learning and Innovative Loans); Overall, on the basis of available data, the level of resources devoted to appraisal and implementation appears adequate; and The evaluators remember that GEF (Global Environment Fund) and Montreal Protocol (ozone protection) projects were processed with very simplified documentation, consisting essentially of a Memorandum of less than ten pages to the Country Director who also had the authority to approve the projects and wonders why similar procedures were not adopted for this post-conflict case. C. Composition of the Portfolio

3.47. The portfolio is evenly split three-way between social sectors, infrastructure reconstruction and economic activities (including rural development), as can be seen in Table 3-8 below.
32

The 103 pages PAD for CEP I are followed 15 months later by a 74 pages CEP II PAD and 14 months later by a 60 pages CEP III PAD. The inflation in the size of the PAD over the life of TFET is even more noticeable for the ARP programme: the PAD for ARP I has 75 pages, for ARP II 15 months later it has 137 pages and for ARP III 168 pages. 33 Inversely, the expense would have been well worth, had the projects been reshaped to fit the sector vision.

-47-

Table 3-8: Sectoral Distribution of Projects

Sector Social Sectors Education Health Infrastructure reconstruction Water and sanitation Infrastructure Rural development CEP ARP Economic activities SEP Microfinance Other (TA, etc.) Total
SOURCE: Reports of the Trustee

Amount committed (US$ million) 53.1 27.8 25.3 48.8 9.0 39.8 36.8 19.0 17.8 16.4 12.4 4.0 3.4 158.5

% of total commitments (Excluding fees) 34 18 16 31 6 25 23 12 11 10 8 2 2 100

3.48. The evaluators interviews revealed no major questioning by the donors or the Government regarding the portfolio make up. This may reflect the participative TFET decision-making process. Sequencing 3.49. There is broad agreement that beginning with the Dili Community Employment Project and the CEPs grant component was adequate since the concern at that time was to inject funds in the urban and rural communities to help achieve stability. In view of the problems prevailing in the power sector and the cost to the economy and the budget, one wonders whether TFET should not have looked earlier at the Dili system and at ways of reducing/eliminating the CFET subsidy to the sector. D. Portfolio Performance

3.50. The mission based its review of the portfolio performance on the Trustees semiannual reports and for closed projects on the World Bank ICRs (Implementation Completion Reports). There is no Project Completion Reports (PCRs) by the ADB yet. 3.51. The Trustees reports discuss each project in detail in its appendix. However the Trustees views on the portfolio performance were not included in the report until June 2002. Starting June 2003, the Trustees ratings of development objectives and implementation progress for projects under implementation and outcome and sustainability for closed projects are indicated in the Technical Appendix.

-48-

Donors Comment It is worth noting that the outcome and sustainability ratings and some other enhancements to reporting since late 2002 occurred as a result of being raised as concerns by some of the contributing donors. The World Bank country representative showed a high degree of positive responsiveness when such issues were raised. 3.52. For projects under implementation, the December 2003 Technical Appendix rated all the projects Implementation Progress as satisfactory. The only exception was the Petroleum Technical Assistance Projects rated highly satisfactory. All projects were rated satisfactory for Development Objectives with the exceptions of CEP I and II and FSQP, which were given an unsatisfactory rating. The March 2004 Portfolio Management Report confirmed these ratings, except for FSQP, which is now rated satisfactory for Development Objectives.34 Table 3-9 below presents the ratings of four closed World Bank-managed projects as presented in their ICR. Table 3-9: Project Ratings
Rating category Outcome Sustainability Institutional Dev Bank Performance Borrower Performance Dili Employment S L M S S ARP I S L S S S SEP I S L N S U ESRP S L M S S

S: satisfactory; U: unsatisfactory; L: likely, M: moderate, N: negligible SOURCE: ICRs

The Evaluators Comments All World Bank ratings are subjective: They are given by the Task Team Leader and endorsed by his/her management. This does not mean that the ratings do not tell the story. The conclusions that can be drawn are: (i) project implementation has in general been good; (ii) institutional development is a weak area; (iii) CEP and SEP I are problematic; and (iv) the change in the rating of the FSQP reflects the positive impact of the recent project restructuring. E. Disbursement Performance

3.53. The disbursement performance of the portfolio is a critical criterion of portfolio performance. In post conflict cases, quick infusion of funds into the economy is one of the major objectives of the undertaking. The disbursement pattern is summarized in Table 3-10 below:

34

This improvement in rating results from portfolio restructuring efforts in late 2003 and early 2004.

-49-

Table 3-10: Disbursements (US$ million)

June 2000 3.4 2.9 0.5 Dec. 2000 79.7 17.0 2.4 June 2001 95.5 50.3 5.5 Dec. 2001 142.0 64.8 2.4 June 2002 147.5 83.7 3.2 Dec. 2002 153.8 98.2 2.4 June 2003 155.4 113.3 2.5 Dec. April 2003 2004 155.5 158.5 118.7 126.3 0.9 1.3

Cumulative Approvals Cumulative Disbursements Monthly Disbursements

SOURCE: Trustees reports. Figures exclude the disbursements of the World Bank/ADB fees.

3.54. The best disbursement rate was achieved during the period late 2000-early 2001, when the emergency reconstruction and the CEP grants were going full swing. Since mid2002, the drop in the disbursement rate has been steady, despite an upturn in recent months. Disbursements now average about US$1 million per month. At this rate TFET will not be fully disbursed before mid 2007. The Evaluators Comments The drop in disbursements after Independence seems to result largely from the phasing out of UN personnel and the taking over by the Timorese Government. While the PMUs have remained well equipped and well staffed, decision-making in Government has shifted to staff understandably overwhelmed by their new responsibilities.

The World Banks Comments In order to fully analyse the drop in disbursements after independence and especially after June 2003, it [would have been] useful to provide a more detailed analysis. It would [have been] good to look at the disbursement profiles by project. After independence, SEP and FSQP went through a major restructuring to better adapt to the development objectives of the newly independent country as expressed in the NDP and to take into account lessons learned from the first generation projects. HSRDP I came to an end and the first phase of HSRDP II is focused on hospital design before hospital construction (and major disbursement) will be launched. The ADB managed Water Program was closed on December 31 2003, and had fully disbursed well before that. The ADB managed EIRP encountered major management challenges causing very important delays in the road reconstruction program. CEP was closed on March 301, 2004 and had almost fully disbursed well before that. 3.55. Since mid-2003, when two thirds of the funds (excluding fees) had been disbursed, the disbursement performance has slowed down significantly. It now averages about US$1 million per month. At this rate TFET will not be fully disbursed before mid 2007, four and a half year after the date envisaged in Tokyo35.
35

The Second Health Project and ARP III have a closing date of mid 2007. TFET donors have thus tacitly accepted that the mechanism would continue until then. The termination date of the EC Administration Agreement is 31st of December 2006.

-50-

F.

Implementation period.

3.56. On the basis of information available to them36, the evaluators understands that at the Tokyo donors meeting, it was assumed that TFET would be implemented and disbursed over three years, from 2000 to 2002. As explained above, implementation will last until mid200737. 3.57. The Implementing Agencies interpret the three-year period discussed at the Tokyos donor meeting as the commitment period. World Bank Staff participating in the meeting do not recall that Tokyo ever committed TFET to disburse over three years. The Evaluators Comments There is, indeed, room for interpretation on the expected length of the implementation/disbursement period. The EC Financing decisions make it clear that the intention of the EC was to provide assistance for three years: [The specific objectives of the EC programme] is to respond to the rehabilitation and reconstruction needs over a three-year period supporting agricultural recovery, infrastructure rehabilitation and restoring public health and education services (Logical Framework above). The EC Administration Agreement to TFET terminates in end-December 2006 Many World Bank first generation projects were conceived (closing date) to disburse in less than three years. In any event, even commitments have not been completed four and a half years after the beginning of TFET. The relevant issue here is the proper length of the implementation period and the optimal disbursement rate of a program like TFET. All stakeholders appear to accept the case for a longer (six/seven years) implementation/disbursement period. The evaluators view is that the three-year period was adequate for getting reconstruction underway and that the focus of TFET should have then shifted to development. They would recommend that, at the outset, consideration be given to divide the implementation period be into two. The immediate post conflict period (say two-three year) would be a period of high commitments/disbursements aimed at reconstruction, as was the case for Timor-Leste. The second period (once the emergency has abated and reconstruction is well underway) would need a slower pace of commitments/disbursements. This approach would likely have worked well in Timor-Leste where security was quickly re-established and there was a political consensus. In all post-conflict situations, it would be important to keep in mind that the outcome beyond reconstruction is sustainable development. The smooth transition between the two periods would require (i) a concern with timing of the shift from the outset; and (ii) broad strategies for the sectors (including policy objectives and priorities, institutional architecture, sustainable expenditure framework and staffing plan).

36

The Report to the Executive Directors [of the World Bank] 17 December 1999 contains Summary of external financing requirements 2000-2002 estimating Reconstruction and Development expenditures of US$302 million for the period. The same dates and figures appear on page xxiv of IDAs Transitional Support Strategy of November 3, 2000. 37 The Second Heath Project and ARP III have a closing date of mid 2007. TFET donors have thus already accepted that the mechanism would continue until then.

-51-

G.

Procurement Performance

3.58. World Banks procurement guidelines are used for World Bank-processed projects, while ADB procurement rules were applied for ADB-processed projects. The major methods of procurement used were small works, national and international shopping, national and international competitive bidding. In order to facilitate implementation and spread the wealth locally, the World Bank relaxed its requirements somewhat with the following measures: Accept relatively high thresholds for national shopping and small works (US$ 50,000 and US$100,000 respectively in the early grants, lowered to US$10,000-US$15,000 in the second generation ones); Accept procurement through United Nations Agencies for small items not suitable for local shopping (such as computers, medical supplies and spare parts) Accept splitting of contracts so that they fit under the national competitive bidding (NCB) threshold; and Introduce a clause that NCB contracts for schools and clinics should provide for 80% of unskilled labour from the communities concerned.

Stakeholders Comments Stakeholders commented that these measures did not go far enough. Many government officials and PMU staff complained that World Bank/ADB procurement rules are too complex and not adapted to the post-conflict situations in general and to the Timor-Lestes in particular. The Evaluators Comment Although the Banks Memorandum to the Executive Directors on World Bank Assistance to East Timor of November 19, 1999 stated that streamlined procurement and disbursement procedures, consistent with Bank guidelines and used in other postconflict situations will be utilized, the evaluators did not find any significant simplification of the procurement procedures used in Timor-Leste.

3.59. The review of procurement procedures under the CEP and ARP projects (see Case Study in Annex 1) shows that consulting services were procured mostly through single source or as individuals. For ARP I, these categories account for 33% of the grant amount. The lack of transparency has been noted in several interviews. The World Bank Resident office in Dili advised the evaluators that individual contracts were preferred because they were more cost effective. 3.60. A last point on procurement is the cumbersome situation created by the UNTAET Regulation 2000/10 on Public Procurement for Civil Administration in Timor-Leste, which has been confirmed after independence. As stated in the World Banks Timor Leste Country Procurement Assessment Report in its May 2003 draft (page ii): Regulation 2000/10 is based to a large extent on a draft prepared with technical assistance funded by the World Bank However, there is a need to review and revise the Regulation to take into account the procurement guidelines of the World Bank, the Asian Development Bank and other development partners. -52-

The Evaluators Comment As a result of these differences, TFET appraisal reports go to great length in their procurement annexes to explain why UNTAET procedures cannot be used. In some interviews, stakeholders strongly criticised the World Bank/ADBs refusal to accept the national/UNTAET procedures. The evaluators are puzzled as to why a simple problem could not be resolved in a simple manner. The World Bank Dili Office informed the evaluators that the new procurement rules in preparation would solve this problem Cost effectiveness of procurement 3.61. The situation in East Timor in the immediate post-conflict period was not favourable for cost effective procurement. The small domestic construction industry had suffered in the disturbances and largely left for Indonesia. The size of the market is too small to attract a large number of foreign firms. Indonesian firms, though eligible to participate may have been leery to bid. Finally the large foreign presence linked to UNTAET pushed prices higher. In this situation contractors had no incentives to try to do better then the prices available in the PAD. 3.62. The case of classroom construction shows the progress achieved managing in unit costs. As more experience was gained, designs and procurement methods were adapted. The cost per classroom (52 square meters) has dropped from a high of US$53,000 (or US$1000 per m3) for 16 schools built under NCB to a low of US$15,000 (or US$300 per m3) for 67 schools under small works contracts. Recently contracted schools have seen their average cost per classroom reach US$20,000 or around US$400 per square meter38. Sources of Procurement 3.63. The mission had access to the breakdown by nationality of the largest contracts financed by the World Bank. Out of the US$50.7 million reviewed, European procurement accounts for US$9.7 million (19%), including US$6.7 million for consulting services, US$2.0 million for goods and US$0.8 million for civil works for the reconstruction of Uma Fukuum under CEP. With regard to ADB projects, European procurement accounts for US$ 6 million or 15% of total ADB procurement. Fraud and Corruption39 3.64. As of end January 2004, the total number of fraud and corruption cases reported by the World Bank was fifteen, of which five were open, three substantiated and seven unsubstantiated. One new case opened in February 2004. The situation of the substantiated cases is as follows (February 2004 Portfolio Management Report by the World Bank): Out of the three substantiated cases, for one case of collusive bidding, the debarment40 notice is in process of being issued, for the case of bribery the debarment process is
38 39

Data provided by the World Banks Dili office. The data on fraud and corruption is by necessity skimpy. Following both the World Bank and ADB procedures, information during investigation is only available to staff who need to know. 40 Debarment means barring the guilty bidders from further participation in projects.

-53-

underway and for the case of previous conviction of fraud, the perpetrator was asked to leave the country. 3.65. The ADB is investigating two more cases. The findings on another case are being presented to the Government. 3.66. The World Bank reports only one case of misprocurement under CEP for an amount of about US$10,000. The ADB declared misprocurement for the choice of the concessionaire for the Dili power system. H. Management of the Reconstruction Programme

3.67. The TFET projects are managed according to the World Bank/ADB model in which a PMU (Project Management Unit) is established for every project to ensure its smooth implementation. Overall coordination of the programme is the responsibility of the office or department in charge of the country. In the Timor-Leste case, it is the World Bank and ADB Dili Missions. 3.68. PMUs play a key role in the implementation of World Bank/ADB projects. They are part of the countrys administration and, as the multilateral lending institutions counterparts, they implement the project (including procurement and fund disbursements). Where institutional capacity is weak, it is normal for the banks to specify in their agreements the composition and skills of the PMU staff. If needed, the project can finance the PMU. 3.69. PMU staff reports to the Government. In many countries, they are civil servants seconded by the administration. In Timor-Leste, most of the PMU staff was contracted first by UNTAET and then by GoTL. Most were/are paid by the project. UNTAET offered only contracts in line with its mandate (six months). 3.70. Staff was recruited by the PMU under consulting contracts. Some PMUs, such as the CEP PMU, employed up to 200 people. The PMU consulting contracts were in general for individuals, the argument being that there was not enough time to hire firms, that few, if any, firms had the language capacity to operate in Timor-Leste and that individual contracts were more cost-effective. Given the low capacity in the Timorese administration, the PMUs had to carry the burden of identifying and retaining large number of individuals (national and foreigners). 3.71. PMU integration in the Ministries was variable. An example of poor integration was the CEP PMU which seems to have never achieved integration with the Ministry of local affairs, even though it was the first PMU to be Timorized. The ARP PMU on the other hand is so well integrated in the Ministry of Agriculture, Forestry and Fisheries that it is no longer a distinct entity any more. One of the lessons learned from ARP I is Avoid competition between national consultants and public servants. Where project activities need to be sustained, public servants (and not national consultants) should be given responsibilities for implementation. The hiring of many national consultants under ARP I may have prevented good candidates from joining the public service and hampered future institutionalisation of selected activities. The integration of PMUs in Ministries was also hindered by the differences in salaries between local consultants and Ministry staff. This

-54-

jealousy was mentioned several times regarding the poor integration of CEP PMU and the Ministry. Stakeholders views Many (in Government and among Donors) believe that the PMU staff is hired by and reports to the Implementing Agencies. PMU staff is overpaid by factors of one to three for Timorese consultants compared with Timorese civil servants.

The World Banks Comment The confusion about the reporting lines [of individual international consultants] in many PMUs stems from the fact that we [the World Bank] have tried to be proactive. We have obtained CVs from candidates we know are qualified and circulated them to the PMU. Indeed, at some time all the individual consultants in Washington were asked to work in Timor-Leste (Interview with staff involved in the project since 2000).

The Evaluators comments The proliferation of full-fledged PMUs is a consequence of the World Banks model with each project having its own Task Manager who generally prefers (or even insists) having his/her own fully staffed PMU over which s(he) has full control. Here again, the staff or office responsible for the whole TFET portfolio must have intervened to rationalize the situation. It is a waste of scarce skill resources, particularly in procurement and financial management. The implementation missions aide Mmoires repeatedly report on the difficulty of hiring competent staff in these areas. Integration into ministries would be have been facilitated had PMU staff and CFETfinanced Technical Assistants been working in a unified structure (Technical Assistance to the Ministry) like in agriculture (ARP). This would also ensure the integration of CFET and TFET. Implementing Agencies constant interaction with the PMUs (combined with the requirement for obtaining the non-objection of the Implementing Agencies PMU staff specified in some Grant Agreements) has contributed to the confusion about the status of PMU staff and may have limited their effectiveness. The difficulty in hiring and retaining PMU staff (particularly in financial and procurement functions) may be explained by the fact that UNTAET was offering only short-term contracts (six month renewable) in line with its mandate.

-55-

I.

Cross Cutting Issues

Credit and Grants 3.72. The JAM report states micro-credit is an important element for the development of economic activity, in particular the agricultural sector and small entrepreneurial activities In the case of a new economy like Timor-Leste it would seem particularly desirable to encourage the use of risk sharing and pooling mechanisms, probably supported by NGOs, or credit guarantee schemes backed by donors. For small scale entrepreneurs, microcredit mechanisms are particularly appropriate.41 3.73. In line with the JAM guidance, four TFET projects channelled loans to rural (CEP and Microfinance Project) and urban (SEP I and II and Microfinance Project)42 beneficiaries. All projects have been fraught with problems. 3.74. The CEP had a small component to create a revolving fund (see case study). It was implemented by the PMU and reached about 4,500 borrowers through 400 loans. Only 30% of the amounts lent were repaid. 3.75. The Microfinance Institution (IMFTL) has 7,500 borrowers and has a better recovery rate (92%). However, it is entangled in a difficult institutional problem, which limits its development.43 3.76. The SEP I credit component to about 400 borrowers was managed by a local Portuguese-owned bank, which did not share in the risk. As a result the recovery rate was only 40%. After energetic Government actions, the rate has recently improved to 70%, which is still quite low. 3.77. This is not to say that credit recipients did not do well. For instance the SEP I 400 borrowers created more than 1,300 jobs. There is also evidence that many of the borrowers under the CEP credit scheme are doing well. It is just that without an excellent repayment record and an adequate regulatory framework, credit is not sustainable. 3.78. The World Banks views on the TEFT credit components can be found in the conclusion of the independent review of CEP credit and in the lessons learned from SEP I. The independent review of CEP concludes The experience of credit delivered via CEP offers no new lessons for good practice in micro-credit. Rather it reinforces a number of lessons well understood from experience elsewhere. It does, however, underline some limitations of microcredit in the immediate aftermath of crisis, which bear upon the concerns of donors and the IDA. These include [situations] where: o Decision-making is divorced from risk, credit allocation is inefficient; subsidized credit tends to undercut credit delivered by institutions striving towards sustainability;
41 42

Joint Assessment Mission, Macro Economic Background Paper, pages 3and 4. The PAD for ARP I mentions a US$2.5 million agricultural credit component to be included in SEP II. It has not materialized. 43 It is owned by an ad hoc foundation with the ADB, Portugal and AusAID on its Board. The foundation owns the US$2 million initial capital provided by TFET. The BPA will not issue a full banking license until IMFTL has owners able to stand behind the institution. AusAID recently withdrew from the foundation.

-56-

o Absence of any assurance of follow-up credit is a disincentive to repayment; o Failure to recycle credit promptly reduces the multiplier effect on economic activity of a given quantum of loan capital; o Contamination of non-repayment spreads rapidly if immediate and effective action is not taken; o Microcredit is not well-adapted to the needs of seasonal agriculture; and o Credit-management groups are organic entities forged by training and sustained by mutual obligation, not arbitrary and administratively-convenient assemblages. 3.79. The Banks views are reiterated in the lessons learned from SEP: Think twice before offering a line of credit in an immediate post-conflict environment. Immediately following conflict and a concomitant destruction of property, asset prices often rise to an inflated level. If a line of credit is offered under such conditions, it can result in excessive borrowing, exposing borrowers to unmanageable levels of debt as asset prices deflate post-boom. In a post-conflict environment, borrowers and lenders also face massive information deficienciesmisreading of market conditions and opportunities is more likely than not, thus leading to loan losses, unsuccessful businesses, and worsening of poverty and livelihood levels. For these reasons, and depending on precise operational conditions, support for the creation of a new formal lending sector is immensely difficult and should probably not be undertaken until the reconstruction and development process (2-3 years) is more advanced. Instead of pursuing a credit programme, the explicit objectives of SEP I could have been achieved largely by issuing grants, approved on commercial principles, to entrepreneurs with viable business proposals. This implies a managed transition from intermediated grantgiving to intermediated loan giving. In addition, loan repayment rates under the SEP I line of credit suffered due to competing grants offered under the Community Empowerment Project; such activities under the projects should have been coordinated better at the outset so that only one form (preferably grants) of small enterprise intermediated assistance was offered. Once a decision has been made to move toward intermediated loans, the conditions should be coordinated among all loan programmes with a view to fostering a healthy credit and repayment climate. Stakeholders Views The stakeholders are unanimously critical of these components, which they see as having been made against the Implementing Agencies normal (if not best) practices. The strongest criticism is addressed at the CEP credit component, which was implemented without any banking/credit safeguards. The Microfinance project is seen as having spoiled the market for other microcredit by not promoting group borrowing. The SEP component is criticized for its design, which absolved the lending institution of any credit risk. The World Bank claimed that the SEP has given the commercial bank through which funds were lent to beneficiaries, the knowledge of the market, which has allowed it to become the major lender in Timor-Leste.

-57-

The Evaluators Comments The evaluators agree with most of the criticism on the credit components. They: (i) were premature, being introduced at a time when many agencies were giving grants; (ii) should have been postponed until the financial sector policies were well established; and (iii) were poorly designed in a manner inconsistent with the World Bank and ADB procedures. There should be in place an internal mechanism (to the Implementing Agencies) to ensure consistency between projects and quality control across sectors. In the TimorLeste case, it seems to the evaluators that this function should be the responsibility of the Timor-Lestes Chief of Mission. The idea of using only grants in the immediate post-conflict period is a sound one. However, while it is acceptable to give a grant for building a small food stall, it is more difficult to justify it to buy, say a truck. The selection of grant beneficiaries for significant assets is an extremely difficult issue as has been shown under ARP I . Community Involvement (CDD: Community Driven Development) 3.80. Several TFET projects have community components. The most significant one is the CEP cluster. Other projects included primarily community-based works. 3.81. CEP is discussed in detail in the case study on the rural sector and is only summarized here. It involves three cycles of grants to all the communities in Timor-Leste through local and sub-districts councils. The community councils were established under CEP by direct elections and included an equal number of men and women. They were to decide on, and be responsible for, the use of the CEP grants. Village chiefs were prevented from sitting on the village councils. The stated ambitious goal was to have these councils as a forerunner of a democratically based local administration. 3.82. Bottom up decisions were expected to be better than the top down decisions characteristic of the old Indonesian administration. CEP was largely modelled on the Kecamatan Development Project of Indonesia and aimed to by-pass government to get good government44. In view of its mandate, UNTAET could not agree to what it saw as a parallel system of governance. The CEP PAD claims endorsement of the project by CNRT.45 CEP did not coordinate well with the District Administration (once it had built capacity) and its PMU remained not integrated with the Ministry of Local Affairs. The CEP councils were not used by other TFET projects or by other projects as had been planned. They seem to have vanished with the closing of the CEP projects. 3.83. CEPs success was its ability to reach all the villages in Timor-Leste just a few months after the emergency. Even today, surveys indicate that CEP is the only programme

Lessons learned: Community Empowerment and Local Governance Project (CEP), by The rolling think tank initiative, March 2004, page 29, quoting Judith Edstrom Indonesias Kecamatan Development Project Is it Replicable: Design Considerations in Community Driven Development World Bank, Social Development Paper No.39, March 2002 45 The Prime Minister stated at the May 17, 2004 TFET Donors council that he reluctantly participated in the CEP I negotiations.

44

-58-

since the emergency to have reached so many groups46. contributed to stability.

As such, it has undoubtedly

3.84. Other projects have community participation in construction (education), rehabilitation (agriculture and roads) and maintenance (power and irrigation). Most seem to have been interacting with the communities through traditional chiefs and by consulting with the church, womens and youth groups (what some interlocutors have called the local CNRT structure). It is remarkable that ARP I managed to distribute without too many difficulties scarce and valuable assets (such as cattle) through such an ad hoc scheme47. The Natural Resources Management component of ARP II and III consists also of community grants largely patterned on the CEP scheme (without the parallel government overtones). It is being implemented cautiously as it is now active in only 30 villages versus 498 under CEP48. 3.85. The evaluators believe that these differing approaches had been discussed internally. However, they did not have access to any internal document on this subject. Stakeholders Comments All stakeholders are critical of the CEP village and sub-district council component. It had been reluctantly accepted by UNTAET and CNRT and was only marginally integrated in the local administration. The other community activities are not criticized, which the mission takes as endorsement of the concept. The exception is the power schemes, where community maintenance and operation are considered non-sustainable.

The Evaluators Comments The extensive use of community-based approaches in the situation of Timor-Leste is commendable. CEP was ambitious and dealt with political aspects, such as the role of the traditional chiefs and of the provisional CNRT structures or the interaction with the emerging local administration, which were not the World Banks mandate. Other projects using the existing local structures were equally successful in involving the communities without giving raise to controversy like CEP.

46

See Leene Ostergaard Consultations with Local Government and populations within the framework of preparing the Transition Support Strategy, Final Draft April 2004. The ARP I ICR mentions that Project staff spent many months collecting beneficiary lists and ensuring that they were independently verified by District Agricultural Officers, church members, and representatives of women and youth groups. While this substantially delayed the distribution of assets, it ultimately resulted in smoother delivery, particularly for highly-valued buffalo and cattle, over which conflicts regarding distribution had been expected.

47

48

The consultants in charge of the identification of the ECs rural development program in Timor Leste had misgivings about the NRM component and recommended that it not be financed by the EC. The recommendation was accepted.

-59-

Technical Assistance and Consulting Services 3.86. The main categories financed by TFET are works, equipment and consulting services. On the basis of data available in the technical annexes, the following breakdown of disbursements per category is available for December 2004 (Table 3-10). Table 3-10: Financing by Category under TFET Projects (US$ million)
Project ARP I ARP II CEP I CEP II CEP III DCGP EICBP EIRP I EIRP II ESRP FSQP HSRDP I HSRDP II PTAP I Micro Finance PTAP II SEP I SEP II WSS I WSS II TOTAL Civil Works 1.7 1.7 4.7 5.4 0.2 0.3 --22.0 0 3.0 2.1 5.0 ----2.2 --4.0 0 1.6 2.6 56.5 Equipment 2.7 1.1 1.2 0.5 --0.2 --5.3 1.3 8.7 0.5 2.0 1.6 --0.1 --0.1 0.2 25.5 Consulting services 2.4 3.6 3.1 2.0 1.2 --0.6 2.5 --2.2 2.7 4.7 4.0 0.2 0.8 0.3 0.4 0.5 1.5 0.9 33.6 Total 6.8 6.4 9.0 7.9 1.4 0.5 0.6 29.8 1.3 13.9 5.3 11.7 5.6 0.2 3.1 0.3 4.5 0.7 3.1 3.5 115.6

Notes: Grants under CEP, ARP and the education projects have been included in civil works, as have loans and grants under SEP and microfinance. Training and PMU operating costs have been included in consulting services.

3.87. The table indicates that 29% of the funds have been spent for technical assistance overall. Besides the TA projects, particularly high percentages of TA are recorded for the Health and CEP projects. If the ADB complementary TA is included, the total amount for TA reaches US$41 million, or one third of total TFET expenditures. 3.88. In addition to the absolute amounts spent on TA, there is an interesting situation regarding the procurement methods used for recruiting consultants. The ARP I case is typical. About 33% of its disbursements went for consulting services, operating costs and training. 3.89. Single source procurement was used for two consulting firms whose services are needed during the coming 6 to 8 months (ARP I PAD page 46) for a total of US$0.97 million. The evaluators do not understand why Quality and Cost Based Selection (QCBS) could not be used. The project had been under preparation for more than six months (from -60-

the JAM to the approval). Individual consultants were to be hired for various positions with the PMU for a total of US$ 1.92 million. The World Banks Comments QCBS selection of a consulting firm would not have been a better alternative to individual consultants, particularly in the early days of TFET. In the ARP program, each of the advisors had to be carefully selected according to his/her individual merits and past experience. Hiring individual consultants under fixed term contracts also enabled the Ministry considerable flexibility in replacing advisors which they considered were no longer needed, with whom they did not get along, or who had not met their expectations. Moreover, while there was some lack of coordination amongst advisors, this was solved by hiring a project management advisor, and (under ARP III) by specifying in each advisors' terms of reference that they would report to the management advisor. By contrast, to have opted for a firm would have had several strong disadvantages: first, MAFF did not have the necessary procurement expertise in the early days to distinguish between a good firm and a 'body shop'. Second, MAFF would have likely ended up with a mixed team: some good advisors, some bad. They would not have been able to pick and choose the candidates based on their competence and familiarity. This would have undermined the performance of components overseen by the less competent advisors. Third, it would have been much more difficult for MAFF to be able to replace individual members of the team. Finally, hiring individual consultants was much more cost-efficient than hiring them through a firm where 100 percent overheads are common. Seemingly the only advantage of QCBS would have been for the firms themselves...Quality of individual advisors was and still is paramount to the performance of TFET projects and capacity building within the Ministries. The experience of bilateral donors who have employed consulting firms for complex multi-sectoral assignments in Timor has not been very good. In this context, selection of individual consultants provided MAFF with the flexibility, timeliness and saying in the selection of their advisors. In ARP I, key advisors were mobilized within 12 months of project effectiveness. By contrast, a competitive firm, which MAFF is in the process of selecting is taking 5 months to mobilize. We did not have the luxury of time in the early days of TFET. In order to keep continuity in program implementation, MAFF made the strategic choice to retain most of the advisory team selected during ARP I in ARP II. This was the correct choice even if it implied renewing these contracts under single source procedures. In ARP III, this was no longer deemed justifiable and with sufficient ahead planning, most advisory positions have been re-advertised.

Views of the Stakeholders

There is a general complaint by our Timorese interlocutors, particularly by NGOs and government officials, that too much has gone into technical assistance and consulting services, which are mostly payments to foreigners or expatriate Timorese. There is the perception that foreign technical assistants are overpaid and that the money goes back to the donor countries, instead of being used for Timor-Lestes reconstruction and development. The process of selecting technical assistance is perceived as lacking transparency.

-61-

The Implementing Agencies Comments The capacity of the Timorese administration to implement projects was very low and vastly underestimated. TA was therefore needed to overcome that weakness. Also, a large proportion of TA has gone to Timorese consultants. Finally, it would have been good to compare the level of TA in TFET-financed projects to those financed by bilateral aid. Unfortunately, available data do not allow such a comparison. In health a considerable part of the TA was (and continues to be) devoted to financing international medical staff. At the same time the program also finances the training abroad of Timorese, and the internationals are gradually being replaced by Timorese as they graduate. It is hard to see how medical services could have been delivered over the past four years without these internationals. The Timorese public health system needs many more doctors than were available in Timor at the start of the transition). Indeed, such use of international TA in the context of a systematic (and long term) capacity building program might have been a useful approach for other sectors as well, as opposed to the piecemeal and ad hoc approach that has prevailed and failed.

The Evaluators Comments The proportion of consulting services, particularly in CEP, ARP and Health projects, is indeed high. It is largely justified by the fact that the PMUs and their consultants (often Timorese nationals) have substituted for the nascent yet non-existing local administration. Technical Assistance is more likely to succeed when it is placed in a comprehensive/long term institutional development plan (Health). Over reliance on individual consultants creates an administrative burden and lack of transparency without necessarily providing better services. The consultants expenditures are more in line with non-emergency projects in the infrastructure sectors, which were handled in a more conventional manner.

Gender, Land Acquisition and Environment 3.90. The Implementing Agencies were diligent in applying their operational guidelines on all safeguard areas49 including gender, land acquisition and the environment.

49

World Banks jargon to designate areas of special attention

-62-

The Evaluators Comments The Implementing Agencies reporting covers all three areas. With the exception of gender, they are usually dealt with succinctly. Even gender is dealt with in terms of number of women participants or beneficiaries. It is not apparent that gender considerations were very much taken into account in the design and implementation of projects. Of particular note, however, is the ambitious effort to have 50% female representation on the suco and posto councils elected under CEP. With regard to the environment, most works were classified B. In view of the large erosion caused by an ARP access road observed by the evaluators, this classification may have been one of convenience.

J.

Major Project Achievements

3.91. It is a difficult task to summarize the achievements of 23 projects. This section will thus focus on major physical achievements as well as on policy contributions of the projects, based on data available from the Technical Appendix to the Trustees Report of May 2004. Table 3-11: Major Project Achievements
Project
ARP I (completion)

Major physical achievements

Distribution of productive assets (2500 cattle, 70,000 chicken and 77,000 tools) 37,000 families benefiting) Rehabilitation of infrastructure (7,700 ha of irrigation perimeter and 7,000 families benefiting; 109 km of access roads) Vaccination of 80% of livestock Rehabilitation of infrastructure (3700 ha of irrigation parameter and 100 km of access roads) Over 80% of cattle vaccinated Cycle 1: US$ 1.4 million Cycle 2: US$ 3.4 million Cycle 3: US$ 3.6 million and 290,000 beneficiaries 4600 unskilled labourer employed in Dili; US$ 300,000 spent on wages 57 staff trained

Policy contributions

ARP II (December 2003)

Four policy notes substantially completed (food policy, draft forestry policy, draft fisheries decree law and quarantine and organizational regulations).

CEP (completion)

DCEGP (completion)

EICBP (March 2004)

Macro economic aggregates

-63-

completed Integrated accounting system installed EIRP I (completion) 450 landslips repaired 22000 cubic meters of retaining wall constructed 400 km of roadside reinstalled 900 m of culverts* constructed 50 bridges repaired 1,445 km of roads under community maintenance 16 rural power systems operative; 5,200 consumers connected 535 schools reconstructed 2,780 classrooms reconstructed 72,500 furniture sets for students (for 130,000 students) Over 2 million textbooks distributed One escola basica completed 55,000 sets of books of Portuguese as a foreign language distributed in 4th and 5th grade, plus 5,550 teachers guides Vaccination DPT 3 : 70%, Measles: 59% Access to basic health service: 87% Birth with skilled attendant: 47% Microfinance Development (March 2004) SEP I (completion) Loans disbursed: US$ 1.4 million, 5,100 borrowers Deposits: US$ 960,000, 5,200 depositors Disbursements: US$ 3.99 million 341 loans, average size US$ 11,7000 1,320 jobs created 8 market rehabilitations completed Four business centres established Number of beneficiaries: 96,000 Number of jobs created: 2,100 Number of beneficiaries: 112,000 Number of jobs created: 32,000 Summary of land policy School mapping

ERSP (completion)

FSQP (March 2004))

Education congress Primary school achievement study Draft policy framework Draft education law Options paper on health financing Options paper on role of private sector Health policy framework

HSRDP I and II (June 03)

SEP II (March 2004) WSS I (completion) WSS II (March 2004)

Draft bankruptcy law Sector management and investment framework. Sector legislation 5 year sector development plan

* Small bridge to evacuate rainwater on a road. 3.92. The table above shows the scale and range of TFET projects outputs. It is likely that TFET has touched the life of every Timorese.

-64-

Governments Comments While very appreciative of the international communitys efforts in reconstructing the country and putting the economy on a growth path, some government officials questioned the importance of the achievements, given the substantial amounts of resources which have been poured into Timor-Leste over the past four years (over US$500 million, by some Government estimates). Again, the high levels of (foreign) technical assistance and consulting services in many FTET-supported projects were raised.

The Evaluators Comments TFET only represented less than 50% of resources available for reconstruction during the 2000-early 2004 period. Therefore, TFET cannot claim all the good results achieved in Timor-Leste. However, it is impossible to separate TFET impact from the rest. Furthermore, the evaluators believe that through efficient donor coordination and innovative work, TFET has substantially contributed to minimize the waste, which could have resulted from uncoordinated actions by a dozen aid donors implementing a dozen aid programmes.

III. A.

EFFICIENCY: ACHIEVEMENTS AND TRADEOFFS Importance of Security

3.93. As noted in Chapter I: Context, Timor-Leste did not go through a prolonged and divisive period of insecurity or civil war like many post-conflict situations (Afghanistan, West Bank and Gaza, Kosovo, Sierra Leone and Bosnia). Peace was restored rapidly after a few weeks of violence and the Timorese emerged from Black September united in their determination for independence. The Evaluators Comment Often taken for granted, security was an important factor in all the achievements in the Timor-Leste case. The Timorese Governments patient and conciliatory attitude contributed much to this peaceful situation. The various programmes financed by TFET (e.g., Dili Community Empowerment Project, CEP and SEP) and other bilateral donors also had for objective to stabilise the country and create a favourable environment for reconstruction. B. Achievements

Macroeconomic Achievements 3.94. Much has been achieved in less than four years since the violence and destruction of September 1999. The economy responded to the inflows of external assistance and high demand brought about by the international military and civil administration. GDP increased and by Independence had reached pre-crisis levels. The economy continued to increase in -65-

2002, but stalled in 2003, as a result of the winding down of the international presence and slowdown in reconstruction work, leading to contraction of services that catered to foreigners, transport and construction. Government projections point to a slow recovery to the target growth rate of 6% by 2006. 3.95. Economic data are scarce, but it seems that the rural economy has recovered and that foodstuff production has kept pace with population growth, although the situation remains precarious as a result of recurrent drought. 3.96. Unemployment remains high but has stabilised, thanks to the injection of liquidity into the economy and other pro-employment projects financed by TFET (Dili Community Employment, CEP, SEP) and other bilateral donors. 3.97. With a few exceptions, most of the urgent infrastructure rehabilitation needs identified by the JAM and the joint donor sector missions had been completed. Basic levels of service in health and education had also been restored. 3.98. Table 3-12 below shows a comparison of core development indicators pre-crisis and during the reconstruction period. Table 3-12: Development Indicators, Pre-1999-2003
Indicator GDP ($ million) GDP/Capita ($) Net primary enrolment Access to piped water or pumps Access to potable water Electrification Pre-1999 390 341 65% 30% 66% 35% 2001 390 454 75% 42% 50% 26% 2002 402 459 75% 45% 57% N/A 2003 393 440 N/A N/A N/A N/A

SOURCES: Various World Bank documents, including The Timor-Leste Reconstruction Programme: Successes, Problems and Tradeoffs, Klaus Rohland and Sarah Cliffe and the TSP II; and UNICEF Multiple Indicator Cluster Survey, May 2003

Establishing a Functioning Administration 3.99. In addition to stabilizing the economy, the foreign assistance provided through TFET (and other channels) succeeded in re-establishing a functioning administration from a complete collapse of administrative structures in 1999. With the help of foreign assistance and funding of TA projects, the Government has been able to strengthen its fiscal policy and put into place: C. a functioning treasury and payments office; an administration which ensures basic services throughout the public sector and maintains high standards of financial control and compliance; adequate coordination between Government and donors and between TFET and CFET; basic health and education services all over the country; and workable procurement rules and practices. Capacity Building

-66-

3.100. Although the Tokyo meeting had decided that capacity building initiatives were the responsibility of UNTAET, TFET projects were a potential major source of capacity building in the public and private sectors in a number of ways. It goes much beyond the contribution of the dedicated technical assistance projects (Petroleum and Economic Management) and has taken various forms: overall contribution to Government systems and specific training components of projects, as well as capacity building of PMU local staff, of communities and local contractors 3.101. TFET has played a global role in establishing Government systems. In particular, it has introduced procurement rules, a Treasury system (Free Balance) and audit procedures. All projects include training components for civil servants and other staff. For example SEP has sponsored successful fee-paying entrepreneurship development programmes for the private sector and ARP has trained MAFF staff, farmers and Village Livestock Workers (VLW). Most of the training has been short term and Timor-based. The argument used by the EICBP project is that the World Bank did not want trained staff to leave (or not return) to the country. 3.102. Technical assistants in the PMUs have provided on-the-job training for Timorese counterparts of international staff50. On-the-job training may have been less effective than expected because of the short-term nature of international contracts and the absence of a long-term ministerial staffing/capacity building plan. Some former PMU staff have taken positions in the administration. Thus, five engineers of the EIRP I PMU have moved to Government service, including the Director of Roads. Similarly, in agriculture, the Permanent Secretary comes from the PMU and is now its Director. Most Timorese PMU staff are temporary (consultants) and the chances for them to be absorbed in the administration are slim, not only because of the limit on the number of positions in the Ministries, but also because of the differences in salary. 3.103. Other contributions to capacity building have been through community-based activities and by awarding contracts to local firms. One of the areas of ARP focus is the establishment of Water Users Associations (WUA) in the large irrigation schemes. In general the success in establishing community based maintenance capacity has been elusive. 3.104. The ADB used non-TFET technical assistance funds (for a total of US$8.6 million) to support its TFET projects either for project preparation or addressing specific capacity building and institutional issues. The World Bank has not supported its projects through TA outside of TFET. 3.105. Despite these achievements, capacity has remained low, even now. As S. Schiavo Campo commented in his paper51, TFET succeeded in achieving physical reconstruction of Timor-Leste, but failed to prepare the Timorese to run their own affairs. 3.106. Even in project implementation, the two Implementing Agencies territory, capacity building efforts have been uneven and handled in a piecemeal way, with the exception of the
50

One recent attempt to draw on the skills of PMUs is to second staff from the central Procurement Unit to the PMU for training. 51 Financing and Aid Arrangements in Post-Conflict Situations, Social Development Papers No. 6, The World Bank, June 2003

-67-

health sector, where TFET collaborated closely with UNTAET, UNICEF and WHO in building up the capacity of the Health Ministry. Stakeholders Views The World Bank takes the position that in Timor-Leste as in other post-conflict situations, there is a trade-off between speed of reconstruction and capacity building. It also underlines the fact that UNTAET was responsible for building up the civil service and for capacity building. Government stakeholders point out that TFET did not finance high-level education. They also complained that there were not enough efforts on the part of the Implementing agencies to induct Timorese in positions of responsibility in the PMUs. Finally, expatriate PMU staff should have also been selected for their skills for on-the-job training.

Implementing Agencies Comments In Timor-Leste, the Implementing Agencies, like other institutions, faced a completely new situation and had to take on new and unaccustomed roles. We learned as we went. Mistakes were made, they said, but we recognized them and learned from them to try to do a better job. There was a pool of trained Timorese, but they were not used because of political reasons and because they were trained in Indonesia. In their comment to the draft evaluation report, World Bank staff note: With hindsight, it might indeed have been useful to give TFET and the Implementing Agencies an explicit mandate for capacity building and to make it the most important aspect of that mandate, but that is not what the bilateral donors (including the EC) decided, not early in the transition and not at independence either.

The Evaluators Comments The official division of labour gives UNTAET the responsibility for capacity building. However, because of the importance of this issue in the Implementing Agencies work in general and particularly in Timor-Leste, given the extremely low capacity, they should have discussed and agreed with UNTAET right at the start on a modus operandi and on a global strategy to jointly address the issue. In fact, capacity building was tackled in a piecemeal way by all the involved institutions, without a clear vision and expected results, with the exception of the health sector. It should be noted that, despite the official division of responsibilities, CEP I, the second project processed by the World Bank under TFET, was aimed at building up capacity at the local level, which conflicted with the UNTAET mandate of building up the administration. This was the source of a major conflict resolved at the highest levels of the Bank and the UN. After the closing of the CEP programme, not much remains as a testimony to the CEP capacity building efforts (see the case study).

-68-

The evaluators understand that local training can reach a large number of beneficiaries. However, in light of the need for highly qualified persons in Timor-Leste, it would nevertheless have made sense to fund more high-level education under TFET. The argument that it was unsuitable for emergency funding is not sustainable now that TFET will extend over more than seven years52. The PMUs were rightly seen as the main resource for project implementation. They should also have been more a source for training. It should be recognised that the UN system of hiring consultants for six months renewable contracts (the length of the UN mandate) is not conducive to continuity. Capacity building has suffered as a result.

D. Addressing Policy and Sector Issues and Moving from Reconstruction to Development Do No Harm 3.107. As in all post-conflict cases, the first priorities in Timor-Leste were to re-establish essential services, rebuild the administration and rehabilitate infrastructure. Developing policies and institutions were relegated to a subsequent consolidation phase. 3.108. However, in doing so, care must be taken not to take actions, which are difficult to reverse later on or may make development of sound policies harder. Examples of such actions or inactions include (i) the failure to enforce electricity bills, giving the impression that, like before, electricity will continue to be free; and (ii) the (relatively) high salary levels given to local employees in the transitory administrations, which is unsustainable in the long run. The Evaluators Comments In TFET, this do no harm modus operandi, although not explicitly stated, has been generally followed, with a few exceptions, such as the provision of credit (SEP, CEP and Microfinance) in an unsettling environment without a proper legal and institutional framework and the relatively high salaries paid to Timorese staff in PMUs, which create disparities and make integration into the Timorese administration more difficult. The policy missteps could have been avoided if they had been tested against the sector strategies suggested above. Establishing Sector Policies and Institutions and Moving Beyond Reconstruction 3.109. While reconstruction was the overriding priority, some sectors under TFET have been able to develop good policies and establish strong technical and managerial capacity in their respective ministries. In Health, for example, after the initial phase of re-establishing essential services and alleviating the suffering of the population, mostly done through NGOs,
52

In their comments, World Bank staff note that TFET did and continues to finance high-level training abroad for the health sector, including medical doctors and specialists.

-69-

an Interim Health Authority was established which gave priority to the establishment of a policy framework, medium term planning for the sector and institution-building. It is only when its services were strengthened and adequately staffed that the health authority, now transformed into the Ministry of Health, started to take over the financing and management of health services in the districts from NGOs, who until then were filling the gap53. In other sectors, progress in establishing policies and institutions were slower than physical construction. 3.110. In the education sector, the process started with a focus on reconstructing the schools and getting the kids back to schools. Major policy decisions (such as the language question) were left unattended. It is only in late 2003 that the Education Congress laid the basis for fruitful policy development. Most observers agree that the education sector is now advancing well. The progress in the agriculture sector in policy setting is also deliberate. Major issues have started to be tackled under ARP II, then integrated in the NDP, the Roadmap and the SIP and finally in ARP III. The Evaluators Comments It does seem that the direction taken by the sectors and their success in establishing sound policies and move beyond the reconstruction phase depended mostly on the people in charge, both in Timor-Leste and at the Implementing Agencies. There does not appear to have a deliberate decision to exit TFET from reconstruction and move it to the development mode. By Independence, it was clear that the most urgent reconstruction work was completed and, with over US$80 million still undisbursed and US$20 million uncommitted, it would have been wise to pause, take stock and decide whether to continue the reconstruction work or reorient TFET toward development. The process of moving from reconstruction to development should have been built in the TFET process up front and agreed with the donors at the Tokyo conference. The evaluators noted that the World Bank did not plan any interim evaluation of the TFET. Such an evaluation is generally foreseen in large World Bank-supported projects or programmes. Also, this EU interim evaluation, which was proposed two years ago, could have done that assessment if it had been implemented as planned in 2002. The evaluators understand that there were discussions with the new Government54, which resulted in the World Bank moving toward budget support through the TSP, but no reorientation of TFET took place. Rather, there were adjustments within individual projects to align them to the vision of the NDP and Road Map. The World Banks Views By Independence, the reconstruction was far from finished in some sectors, particularly health (which started by developing sector policies and setting up institutions before embarking on major reconstruction works). For other sectors, such as education and agriculture, after Independence, work was concentrated on developing sector policies and strategies.

53 54

It is worth noting that the Health sector is the only one presenting its achievements as sectoral data. Apparently, the new Government wanted to close TFET and transfer the remaining funds to the budget, which had a large deficit to cover.

-70-

To reorient TFET work after Independence might not have been feasible, even if it would have been desirable, given the absorptive capacity for new initiatives of both the Government and the donors. IV. A. SUSTAINABILITY: THE ROAD AHEAD

From Reconstruction to Sustainable Development

3.111. By early 2004, Timor-Leste has definitely moved beyond the reconstruction phase and well into establishing the basis for sustainable development with the production of the National Development Plan (NDP) and implementation of two annual Action Plans supported by the Transition Support Programme (TSP). 3.112. The Government strategy for FY 2003-07 has two basic goals: to promote rapid, equitable and sustainable economic growth and to reduce poverty. The NDP stresses (i) the importance of a prudent fiscal policy and sound financial practices; (ii) sound governance; (iii) establishment of an enabling environment for private sector development; and (iv) development of economic infrastructure, including transport, power and communications networks. Stakeholders Views All stakeholders agreed that how Timor-Leste got where it is now is largely due to TFET. Despite some shortcomings, the mechanism was better adapted to the transition situation which the country was in than any other alternatives (UN or bilateral programmes). As one stakeholder (World Bank staff) noted it was a stage in Timor-Lestes move to independence and provided Timorese with more inputs in decision-making and better coordination than the UN budget process or bilateral programmes. The Evaluators Comments The evaluators agree that TFET was the best alternative under the circumstances and has contributed substantially to guiding Timor-Leste transition from reconstruction to development. As an aid mechanism, the main lesson of FTET is increased aid efficiency through: (i) good donor coordination; (ii) strong and continued policy and sector strategy discussions among all stakeholders; and (iii) use of unified project processing and implementation procedures. Whatever mechanisms will be used to deliver aid to Timor-Leste in the future, it is hoped that the lessons of TFET will not be forgotten. 3.113. The TSP is conceived as a multi-donor, medium-term programme of balance of payments and budgetary support for the NDP over a three-year period ending in FY 2005. The rationale is to provide bridging finance that will allow GoTL to implement a priority development programme in the lean years before substantial oil and gas revenues come on stream. Balance of payments and budgetary support will address both external and fiscal

-71-

gaps, allowing the country to sustain a level of imports and public expenditures that would not otherwise be possible55. 3.114. As the World Bank explained, based on current GoTL estimates, oil revenues from the Eland Kakatua and Bayu-Undan oil fields (which the Timor Sea Treaty with Australia gave Timor-Leste 90%) should help finance an annual budget of around US$100 million, which is above the current CFET, but below the current level of the combined sources budget. Revenues above the US$100 million level (which are expected toward the end of this decade) will be saved for future generations or for years in which there is a shortfall in oil revenues. If Timor-Lestes claims on the Timor Gap were to be satisfied, an annual budget in the order of US$300 million would be sustainable. However, the Government has never used that assumption as a basis for fiscal scenarios. 3.115. The thrust of the TSP is to establish solid economic structures, good governance and sound financial and budgetary systems and procedures to achieve macro-economic stability and growth, thus allowing the Government to address poverty reduction. Central to the Programme is the setting of an appropriate level of expenditure that can be used productively and is sustainable when taking into account future revenue streams. 3.116. Improving resource allocation is another important theme. In the health and education sectors, for example, the challenge is to mobilize sufficient resources to support on-going operations and maintenance of programmes as external financing decline and operating costs borne by TFET are transferred to the current (CFET) budget. 3.117. Another area of concern is the extremely low level of spending on maintenance, particularly of road maintenance, which accounted for less than 15% of goods and services spending during FY2001-03. This is significantly lower than required to maintain assets rehabilitated during reconstruction. Maintenance allowances must be increased and decentralized management systems put into place to ensure that the funds can be utilized in a timely, efficient and effective manner. 3.118. The TSP has completed its second year and the third one is being prepared. All the donors who had participated in the first and second TSP expressed interest in continuing supporting the process. The Evaluators Comments Budget Unification As discussed above, under the UN administration it was not possible to unify the current (CFET) and the investment (TFET) budgets. However, after independence, when the World Bank was supporting the Government budget with TSP, there was a golden opportunity to deal with the budget unification issue and help GoTL gain more control over its own expenditures. The evaluators understand the difficulty of the task as the amount of project financing outside the budget is at least as large as TFET itself and that bilateral donors may have their own objectives. It is also true that right after Independence, the new Government
55

PAD for the Second TSP, June 19, 2003, The World Bank, page 17

-72-

had so many important tasks to do that budget unification may not be on its priority list. It might also not want to address the issue for fear of losing some bilateral funding. However, we are now into the third budget cycle of an independent and sovereign country (and into the third TSP) and the GoTL still has a separate current budget (CFET), an investment budget (TFET) and a multitude of development projects financed by bilateral aid and implemented outside of the budgetary framework. In addition, these three budgets operate independently from one another, promoting waste and making coordination difficult. Rather than addressing the issue, other initiatives were being launched, such as SIPs, which by themselves are worthwhile, but do complicate the budget unification issue even further. Capacity Building. The evaluators are concerned that by putting all its efforts in TSP (which is probably the World Banks main, if not only, aid mechanism in Timor-Leste over the next few years and which is not designed to address capacity building issues), the World Bank (and the donor community which co-finance the TSP) will again give a low priority in practice to the objective of capacity building, which is the countrys main bottleneck to sustained development and the major failing of the UNTAET/TFET reconstruction effort.

The World Banks Comments The fact that a large part of the development programme is financed outside of CFET is not at all unusual for low-income countries and indeed it is unusual to find a situation in which 40% of the countrys budget is financed by a coordinated consortium of eleven donors. The statement Rather than addressing the issue, other initiatives is somewhat odd because the SIPs were launched precisely with the objective of improving unification or coordination between the budgets. Full unification will unfortunately not be possible because several important donors have made it clear that they cannot bring their assistance under the form of budget support. Bullet 4 (Capacity Building), it is not correct to say: "the World Bank will again give low priority to the objective of capacity building". On the contrary, as announced during the last donors' meeting and despite the continued "division of labour" and the major UNMISET and UNDP capacity building programmes, the World Bank - in coordination with other donors - will launch a capacity building effort for the Ministry of Planning and Finance and the planning/budget/budget execution functions in line agencies. B. Sustainability of TFET-Financed Projects

3.119. The sustainability picture is complex. In the evaluators discussions with donors, the view was expressed several times that TFET was not expected to be sustainable. Its main objective was to help stabilise the country. The point is well taken. Projects like Dili employment were a one-shot affair and no sustainability was expected. Similarly, one could argue that many activities under CEP aimed at stabilisation and therefore should be considered successful. If one takes this argument further, TFET can take some credit for the peaceful transition to the independent Timor-Leste.

-73-

3.120. The evaluators would, however, argue that most TFET projects were also supposed to be sustainable in terms of the quality of construction, sectoral policy formulation and capacity building. 3.121. With regard to quality of construction, the results are uneven. The mission was told by the World Bank engineering staff in Jakarta that the school buildings were of far better quality than anything found in Indonesia and costs have been brought down to comparable levels. The World Bank Office in Dili confirmed the Jakarta-based engineers view and provided data showing decreasing costs per classroom. 3.122. Similar comments have been made for water systems and health facilities. Irrigation works appear satisfactory. 3.123. The big question mark concerns roads, whether community built or contractor built. They may not be sustainable without adequate maintenance and continued investments.56 As discussed earlier, the credit activities do not seem to be sustainable with the exception of the microcredit operation, if its ownership problem can be solved. 3.124. Policy development has succeeded in several sectors, most notably health, education, water supply and sanitation and agriculture. These are also the sectors where the institutional impact has been deepest and ownership is most pronounced. 3.125. From an institutional point of view, CEP development councils have not proven sustainable. Under ARP, the Water User Associations and Village Livestock Workers appear to be on the way to sustainability. The Evaluators Comments The sectors where TFET actions are most likely to be sustainable are those where a longterm vision was developed over the last four years. This argues for sustainability in agriculture, health and education. In spite of the relatively weak performance of its credit component, SEP II stands a good chance of being sustainable. It is not surprising that CEP was not sustainable. Its design called for a quick infusion of funds into the country and the institutional component was not accepted by the recipient. The sustainability of the road works is questionable. The reasons appear to be the difficult terrain and climatic conditions linked to poor design. It is an area worth studying more and understanding better, lest Timor-Leste looses again internal communications. On the macroeconomic level, the whole reconstruction effort would be difficult to sustain in the long run if Timor-Leste does not get its fair share of the oil and gas revenues in accordance with international laws to maintain and operate the rehabilitated assets57.
The ADB is preparing the Project Completion Report for the first Emergency Infrastructure Project and is looking into the quality of the works. 57 Timor-Leste has a very developed physical infrastructure (road network, water and sanitation and power system) for a country at its development level. This was due to the financial assistance and budget support provided by Indonesia during the 25 years of occupation ($100 million/year, or about $2.5 billion over the whole period). A large part of it went to finance the wages and other costs of the Indonesian administration/occupation, but a good part was also used to build up Timor-Lestes infrastructure. The
56

-74-

C.

Ownership of the Reconstruction Programme Better to let them do it imperfectly than to do it perfectly yourself, for it is their country, their way, and your time is short. T.E Lawrence58

3.126. The JAM, with its equal participation of Timorese (mostly from the Diaspora, which was the only group with technical expertise) and international experts, was a good start to establish ownership of the reconstruction programme by the Timorese. The World Bank provided some funds to finance the participation of Timorese experts, but there were no contingency plans to continue covering their participation as it was expected that UNTAET would take over. However, as UNTAETs establishment was delayed, most of the Timorese experts returned to their former jobs after the mission, and the work was completed mostly by international experts. Given the non-existent capacity in Timor-Leste, the reconstruction programme, at least at the beginning, was implemented without significant Timorese participation. 3.127. Three other factors compounded the problem. First, there was UNTAETs position not to involve CNRT, the umbrella movement for pro-independence parties and practically the only organized group with some technical capacity. Second, CNRT, which held all the four ministerial positions in the ETTAs joint cabinet, was disbanded in June 2001 following disagreements between its constituent elements, so that Timorese Cabinet members acted during this period in an individual capacity. There was no Timorese body for policy decisions. Finally, the problem was exacerbated by UNTAETs decision to recruit government employees from the bottom up which meant that until late 2001, there were no Timorese civil servants in senior or middle management positions to take decisions on the reconstruction programme being implemented. 3.128. As noted above, the World Bank made efforts to involve Timorese in the preparation and management of TFET projects. World Bank staff involved in the early period explained that the Bank did not have the power to hire government officials. We made huge efforts to involve Timorese sectoral teams in areas such as health, education and agriculture but could not set up parallel government structures through TFET projects. All the TFET projects made an effort to use local capacities: for example, the agricultural teams early on decided that they did not want FAO as an implementing agency in agriculture, but would prefer a Timorese project implementation unit. We respected that, even though it caused some tensions with FAO. In education, Timorese engineers managed the process and community structures [and] rebuilt schools using Timorese companies In the community empowerment project, a large Timorese team ran the project management and facilitation process. 3.129. However, it was also true that the pressure to deliver and show physical progress was high and pushed expatriate technical assistants to do the work themselves instead of training, empowering, and gradually bringing in local staff.
international community through TFET and bilateral aid outside of TFET provided an estimated $400 million to rehabilitate the physical assets destroyed in September 1999. However, there is no way GoTL can maintain that infrastructure without the oil and gas revenues. 58 Quoted in S. Schiavo-Campos Financing and Aid Arrangements in Post-Conflict Situations, Social Development Papers No. 6, The World Bank, June 2003

-75-

3.130. In conclusion, there have been efforts, but also missed opportunities. The special set up, with UNTAET managing CFET and responsible for capacity building and the World Bank in charge of TFET-financed projects also substantially complicated matters. The World Bank Comments Allocation of TFET funds was always performed and approved by a Timorese Committee before presentation to the Donors Council Meeting (the National Advisory Council in 2000, the Timorese Cabinet members in 2001 and the Timorese Cabinet in 2002). A senior Timorese government official (normally the Prime Minister, occasionally the Foreign Minister or President) always presented the work program to the Donors Council at the sixmonthly donor meetings as the national priorities of the Government or prior to the establishment of the Timorese Cabinet, of CNRT on behalf of the Timorese population. The evaluators note: TFET governance rules (Resolution Amending the Trust Fund for East Timor, IBRD Resolution No. 99-8/1; IDA Resolution No. 99-5/1, The World Bank) stipulate: work programs should be submitted by the two implementing agencies which indicate projects and programs proposed to be funded from TFET" It is commendable that the World Bank got the Timorese involved in the work.

The Governments Comments Following the departure of most Timorese experts from the JAM exercise and the lack of local capacity in the country to help implement the reconstruction programme, more and more Timorese came to view TFET as a World Bank-driven process. There were also misunderstandings. In the early days, the Timorese thought TFET was a coordination mechanism in which they would have a voice on how the resources would be allocated. They did not understand the trust fund concept, with the World Bank managing the process with the participating donors. There were also complaints that there was no mechanism in TFET-financed projects to encourage ownership. PMUs were not thought to be representative of the Timorese Government and were too much influenced by the World Bank. The Timorese agreed that the World Bank consulted them on work programmes and on fund allocation. They were also asked to present the work programmes to the Donors Council. However, they noted that these consultations were often done at the last minute and did not give them enough time to comment on the documents to be presented to the Donors Council and suspected that one of the motives for the consultations (at least during the UNTAET period) was to gain their support against UNTAET (Interview with a senior Timorese official involved in all high level discussions and work related to TFET during the early reconstruction period in Dili in May 2004).

-76-

The World Banks Comments As a Bank staff closely involved in Timor-Leste during the early period recognized: It is undeniable that the Timorese felt a lack of ownership over the process of re-establishing government and rebuilding the country, particularly during the transition period of 200001 and this feeling of disempowerment touched all aid programmes, whether UN, TFET or bilateral.59

The Evaluators Comments The question therefore becomes whether there was a better alternative to TFET. And the answer is NO. TFET could have done better; but it still is the best alternative (the others being the UN and the bilateral). It is also the one that tried to rely the most on Timorese inputs. According to Timorese decision-makers interviewed who were involved with TFET in the early days, the mechanism with its donor coordination structure was the least bad instrument during this difficult transition period. Now that the Timorese has strengthened their government structures, it is no longer adapted to the needs of the country. The evaluators also had the impression that ownership is understood differently in some Government quarters. It does not mean to be in command of ones own investment programme or development/sector strategy, but rather to have control over the use of the funds. Thus, the TSP or SIPs are thought as being owned by the Timorese because the funds are disbursed through the Timorese budget.

A more nuanced remark by the World Bank staff following distribution of the draft report to the donors and implementing agencies for comment: During the early period of reconstruction, TFET tended to have higher ownership than any other funding source of development aid (since there was no institutional mechanism in the early stages for Timorese involvement in prioritisation of the CFET, bilateral or UN programs, whereas TFET had this in place since January 2000). As the Timorese gained more control over the CFET and eventually introduced the TSP, this perception reversed, since budget support clearly offers broader national ownership than ring-fenced projects. As the paper discusses, however, there was little alternative to ring-fenced projects in the early reconstruction period hence TFET probably had the only possible balance of fiduciary control and national ownership for its period.

59

-77-

CHAPTER IV: CONCLUSIONS AND RECOMMENDATIONS

A. Overall Evaluation

4.1. There is no doubt that TFET has been instrumental to the successful reconstruction of Timor-Leste. Its performance (and, by extension, that of the two Implementing Agencies) has been satisfactory. TFET achieved the results the EC expected of it when making the decision to contribute to the trust fund. The mechanism (i) achieved good donor coordination; (ii) succeeded in mobilizing substantial resources for Timor-Leste reconstruction; and (iii) was efficient as it used uniform procedures for project processing and implementation and eliminated the need for the EC to establish and maintain a large office in Dili to monitor implementation of the reconstruction programme. 4.2. It is a relative success because the mechanism did not significantly improve the countrys low implementation capacity and failed to develop Timorese ownership of the reconstruction programme. It is true that at the Tokyo meeting it is to UNTAET that donors entrusted with the responsibility for capacity building. That division of labour was reconfirmed after Independence when donors helped finance the UNMISET Stability Posts Project and the UNDP Development Post Project, two projects totalling dedicated to capacity building. 4.3. However, given the importance of the matter and the fact that capacity building is central to all World Bank and ADB interventions in member countries, the two implementing agencies should have discussed and agreed with UNTAET right at the start on a modus operandi and on a global strategy to jointly address the issue. In fact, capacity building was tackled in a piecemeal fashion by all the institutions involved, without a clear vision and expected results. 4.4. The mechanism could also have been more efficient in the utilization of the substantial resources it succeeded in mobilizing for Timor-Lestes reconstruction and development. First, the proliferation of PMUs is a waste of resources and makes coordination of the overall TFET programme more difficult. Very few PMUs were actually integrated in the Timorese nascent administration (agriculture was an exception), as they were perceived as World Bank-controlled entities. The high salary structure in the PMUs compared to that of the Timorese administration also prevented actual integration. 4.5. While PMUs were needed in Timor-Leste because of the extreme weakness of the Timorese administration (non-existent at the beginning), some functions such as procurement and financial management could have been centralised to save costs and simplify project management. Such an arrangement also makes sense as the centralised functions could be spun off later to form a service or unit dedicated to procurement or project financial management/monitoring within the Timorese administration. Indeed, in small countries (e.g., Mauritius), public procurement above a certain threshold is handled by a national agency, instead of the ministries or public enterprises60.

60

A national procurement agency has also been created in Cameroon at the recommendation of the World Bank to improve efficiency and reduce waste and misprocurement.

-78-

4.6. Second, project processing procedures were not streamlined enough, while no simplification of procurement rules took place, despite the statement by World Bank staff presenting the TFET project to the Executive Directors (November 1999) that project preparation would be carried out on an accelerated basis in order to provide near term reconstruction assistance to this post-conflict area and that streamlined procurement and disbursement procedures, consistent with Bank guidelines and used in other post-conflict situations will be utilized. 4.7. Some sectors, particularly the health sector, did use procedures reserved for emergency operations, while project approval was delegated to the RVP. However, staff working on TFET projects still had to produce full project appraisal documentation, despite the need to move fast and the relatively small size of the projects. This has increased staff time devoted to project preparation/appraisal and slowed down project implementation at the beginning as disbursements were made from Washington and as Timorese (or technical assistants recruited in the PMUs) were struggling with complex procurement procedures. 4.8. Third, after Independence when the major reconstruction works were completed, there should have been a reorientation of TFET work toward development, instead of continuing on the same reconstruction mode. Adjustments did take place to realign some projects to the NDP vision, but they were done at the project level. By and large, TFET continued its reconstruction business as usual. At least, there should have been a mid-term evaluation by the World Bank, the Trustee and Implementing Agency, of the achievements with a view to addressing emerging problems and, if warranted, moving TFET work toward longer-term development projects61. The mechanics of the reorientation should have been agreed among donors at the time of the JAM. 4.9. The evaluators are also concerned about the sustainability of some of the reconstruction work. Some projects, such as the Dili employment project, were not designed to be sustainable, but others, like the credit activities, were but would not. The physical infrastructure, particularly the roads, be it community-built or contractor-built, are not likely to be sustainable either without adequate maintenance and continued investment. On the macroeconomic level, the whole reconstruction effort would be difficult to sustain in the long run if Timor-Leste does not get its fair share of the oil and gas revenues in accordance with international laws and substantially improve its capacity to maintain and operate the rehabilitated assets. 4.10. Finally, it should be noted that the World Bank and the ADB were able to design projects the way they did in Timor-Leste with substantial technical assistance and consulting services (a large part of it foreign) because the funds entrusted to them were grants. With their normal resources, which are loans and credits albeit at low interest rates, it would not have been financially responsible to do so in a poor and non-creditworthy like Timor-Leste. 4.11. La critique est aise, et lart est difficile. The evaluators recognize the difficulty of the task at hand when everything had to be done right away from rebuilding schools to repairing roads and establishing a local administration (without mentioning the need to adjust to working with new partners and institutions with different and complex policies and procedures).
61

It has been pointed out that this mid-term evaluation by the EC could have served the purpose, had it been carried out two years ago as planned. While this is true, it does not absolve the Trustee from making its own evaluation as its obligations would have required it to do so.

-79-

4.12. Timor-Leste was an unusual case and a challenge to all the institutions involved, many of which had to take on new and unaccustomed roles. As the Implementing Agencies pointed out: We learned as we went Mistakes were made, but we learned from them to do a better job. Indeed, at the end, the job (and a good one) got done. 4.13. In spite of the weaknesses, the evaluators believe TFET is superior to the other alternatives (UN agencies, NGOs and bilateral). Its staff has extensive experience of reconstruction and development financing, is highly competent and has relied the most on Timorese inputs. According to Timorese decision-makers, the mechanism with its efficient donor coordination structure was the least bad instrument during this difficult transition period. For the EC, TFET was also the best way to provide its assistance to the reconstruction of Timor-Leste. 4.14. TFET failed to build up Timorese ownership of the reconstruction efforts, but so have the other aid programmes. Also, the failure is relative and depends on the sector. In agriculture, the TFET decision-making process was fully integrated into the Timorese administration; such integration was high in education and health but non-existent in CEP and SEP. B. EC Performance

4.15. The EC was present in Timor-Leste throughout the crisis. Beyond large-scale provision of humanitarian aid amounting to 51.0 million, it participated in the JAM (Joint Assessment Mission) and was among the largest contributors to the TFET, with 56.4 million, provided through two Financing Decisions (IDN/B7-3040/IB/2000/00 for 40 million in 2000 and ET/AIDCO/2001/0292 for 16.4 million in 2001). 4.16. The main reasons for the ECs participation in TFET were (i) better donor coordination; (ii) higher efficiency resulting from the use of uniform procedures (particularly in financial matters such as procurement and disbursement) and the elimination of the administrative burden to have a large ground capacity in Timor-Leste; and (iii) to have a voice in the mechanism in which most donors participate. 4.17. The EC was aware of the drawbacks, particularly the lack of visibility, TFET being seen by all as a World Bank undertaking. There were also concerns as to whether the contributed resources would actually be allocated to the ECs priority sectors and the risk that EC funding would be used for procurement from countries normally not eligible for EC financing. Despite these drawbacks, the EC went ahead with the TFET project because the advantages were thought to by far outweigh the negative aspects. C. TFET as Mechanism for Post-Conflict Assistance: Lessons Learned

4.18. The task facing the international community in late 1999/early 2000 can be stated as how to best deliver the financial and technical assistance to help Timor-Lestes march toward Independence through: (i) reconstructing its destroyed infrastructure; (ii) resuming delivering of social services and; (iii) kick starting the economy. 4.19. The instrument chosen was TFET, a multi-donor trust fund with the World Bank as the Trustee and the ADB and the World Bank as Implementing Agencies.

-80-

1.

Relevance: Was TFET the Right Instrument?

Assessment 4.20. Overall, TFET was the right mechanism to: Mobilize resources; Ensure donor coordination; and Simplify (for the Government) the implementation of a large and complex programme in a post conflict situation

Lessons learned 4.21. 2. Donor co-ordination is most effective when: Donors are actively engaged; and The Implementing Agencies listen to and hear the other donors views. Effectiveness: TFET Portfolio.

Assessment The projects were well chosen, the sequencing and the intra sectoral allocation of funds adequate; The Implementing agencies were diligent in preparing and appraising the portfolio. The projects conformed broadly with the Implementing Agencies policies; Disbursements, which were almost at 60% of the total amount mobilized at the end of 2002, have been tapering off and full disbursement cannot be expected before 2007; The project ratings (per World Bank procedures) have been satisfactory with the exceptions of the CEP cluster, SEP I and II and Education II before their restructuring; The fiduciary aspects (procurement, financial management and audit) have in general been satisfactory; and The projects introduced procurement, disbursement and audit procedures and skills in Timor-Leste.

Lessons learned Credit components (such as those in CEP/SEP and Microfinance), even if they had been well conceived, coordinated and implemented, are not suitable as a means to inject liquidity in an underdeveloped economy with limited credit experience and going through post-conflict reconstruction like Timor-Leste. Grants are more adapted to the situation. Credit components should be introduced only at the development stage and when the institutional and policy framework is in place; The local government component of CEP went beyond the World Banks mandate and, as conceived and implemented, was not suited to the situation in Timor-Leste; and

-81-

Long-term higher education should have been included in all sectors where needed to complement short term and on the job training. Efficiency: Achievements and Tradeoffs

3.

Assessment TFET projects have contributed to: (i) reconstruction of the country; and (ii) economic stability; and The central trade off between speed of reconstruction and (i) quality of works; (ii) capacity building; (iii) policy development; and (iv) ownership of the TFET projects by the Timorese has been solved in favour of the speed of rehabilitation.

Lessons learned The JAM should have developed a vision for the sectors in terms of objectives, policies, institutional architecture, staffing and financial needs to guide reconstruction activities and to help evolve smoothly into development activities; A mechanism should be built into TFET at the outset to ensure that the central tradeoff shift away from the speed of reconstruction in favour of quality, capacity building, policy development and ownership happens in due time. It could include: o Change in the design of the second generation of projects; o A formal mid-term review (held at the proper time). The mid-term review should also look at the best use of any non-committed funds; and o A clear exit policy in place right at inception. Policy making on the run or by default (e.g. salary levels for local personnel or laxity in power bill collection Timor-Leste) should be avoided. It would be useful to have a policy watchdog (perhaps under the Donors Council). Sustainability

4.

Assessment A large portion of the civil works rehabilitated will need to be revisited shortly; Capacity building and policies are lagging; and Long-term sustainability depends to a large extent on Timor-Leste getting its fair share of oil and gas revenues in accordance with international laws.

Lessons learned Give due priority to capacity building and policy development.

-82-

D.

FUTURE USE OF THE MECHANISM BY THE EC

4.22. MDTFs are well suited for post-conflict assistance. To increase the value added of their use by the EC, the Mission suggests the following: Set Clear Conditions for Participating. o Agree at the outset on what the MDTF should address: Rehabilitation and reconstruction or also development; o Ask that initial assessment includes broad sector strategies as well as list of reconstruction needs; o Set clear objectives and expected results; and o Specify the timeframe for the assistance and for the possible shift from reconstruction to development. Visibility o Request that all the donors be acknowledged on Grant Agreements, advertisements, joint publications, placards, information boards, press releases etc.; and o Be closely involved in the design of the programmes to ensure that focus sectors are adequately addressed. Monitoring the Work Progress o Delegation(s) should be adequately staffed with technical personnel in order to efficiently monitor and contribute to the work. If needed, specialists could also be sent from Headquarters to supplement the Delegation (s) technical expertise. Such a deployment of personnel should be planned and budgeted, including the local recruitment of technical assistants (ALATs); and o Plan mid-term (and a final) evaluations following the procedures of the EC and the Trustee, i.e., including review of internal files. Tied Procurement o Procurement tied to EU countries cannot be managed within MDTFs. However, the ECs assistance is toward non-targeted budget support and sector financing and therefore the EC implicitly accepts untied procurement. Many EC countries are also moving toward untying their aid. Modalities of Future EC Assistance to Timor-Leste o The EC is considering SWAps (which could be done either with other donors or directly with the Timorese Government) for its continued assistance in the health and agriculture sectors. This is a logical outcome of the TFET experience. The evaluators strongly support the use of these mechanisms as they incorporate the best features of TFET, i.e., (i) good donor coordination; (ii) strong and continued policy and sector strategy discussions among all stakeholders; and (iii) use of unified project processing and implementation procedures. However, care must be taken to ensure that capacity building is built into these mechanisms, which are not designed to address this fundamental aspect of foreign assistance in Timor-Leste.

-83-