of the PNG DISASTER RISK MANAGEMENT PROGRAM (2010-2014)
These Terms of Reference (TORs) are to conduct a review and prepare an independent report on the implementation of AusAIDs Disaster Risk Management Program (2010-2014) in Papua New Guinea (PNG). In particular, the Review Team will:
1. Assess and rate the progress of the PNG DRM Program against AusAIDs evaluation criteria
2. Provide recommendations that allow AusAID to make strategic decisions on the future direction of the Program itself while also providing recommendations on the continuation of funding and nature of support for specific activities
BACKGROUND:
AusAIDs Papua New Guinea (PNG) Disaster Risk Management (DRM) Program (2010-2014) commenced in September 2010. Its goal is that of PNGs own draft Disaster Risk Reduction and Disaster Management National Framework for Action 2005-2015: To ensure safe and resilient communities in high risk provinces of Papua New Guinea
Its purpose is:
To strengthen the DRM sector in Papua New Guinea, at the national, provincial and sub-provincial levels, to prepare for, mitigate and respond to the effects of disasters
The anticipated outcomes of the DRM Program are:
1. Increased capacity of organisations, both government and non-government, to manage disaster risks and respond effectively to disasters through improved financial and human resources, planning and management systems
2. Increased capacity of selected high risk provinces to manage disaster risks and respond effectively to disasters through improved financial and human resources, planning and management systems
3. Strengthened disaster awareness, disaster preparedness and risk mitigation in targeted, vulnerable communities 2
4. Increased capacity of AusAID Post to effectively respond to disasters in PNG
Through the DRM Program, the following activities have been supported by AusAID: 1. Improved national and provincial disaster response capability: technical support for the National Disaster Centre (NDC) and selected high-risk Provinces (initially West New Britain) to build and improve disaster risk management and emergency response capacity. 2. A better coordinated response capacity: funding to the United Nations Office for the Coordination of Humanitarian Assistance (OCHA) to improve coordination for disaster preparedness activities and during an emergency response. 3. Improved monitoring of major risk factors: continued support for Geoscience Australia, in partnership with PNG technical agencies, to strengthen assessment, monitoring and early warning of natural hazards such as volcanic eruptions, tsunamis and earthquakes. 4. Strengthened community-led disaster awareness and response capacity: continued support to the PNG Red Cross and funding to community groups and NGOs for disaster risk reduction activities. 5. Enhanced AusAID response: continue to train Emergency Response Team members and build the capacity of AusAID PNG Post to respond to emergencies.
PURPOSE OF THE REVIEW:
The Review Team will:
1. Assess and rate the progress of AusAIDs PNG DRM Program against the following evaluation criteria (refer to Annex A): a) Relevance: to assess whether the activity contributes to higher level objectives of the aid program (outlined in country and thematic strategies). b) Effectiveness: to assess whether the activity achieves clearly stated objectives. c) Efficiency: to assess whether the activity is managed to get value for money from our inputs of funds, staff and other resources, and to continually manage risks. d) Impact (where feasible): to assess whether the activity produces positive or negative changes (directly or indirectly, intended or unintended). The degree to which the various aspects of impact can be assessed will vary according to the nature and duration of the activity. Whether impact can be assessed, or the way impact can be assessed will need to be determined by the Review Team. Impact will not be rated. 3 e) Sustainability: to assess whether the activity appropriately addresses sustainability so that the benefits of the activity will continue after funding has ceased, with due account of partner government systems, stakeholder ownership and the phase-out strategy. f) Monitoring & Evaluation: to assess whether the monitoring and evaluation framework effectively measures progress towards meeting objectives. g) Gender Equality: to assess whether the activity advances gender equality and promotes women (considering the four dimensions of gender equality: access, decision-making, womens rights, capacity-building). h) Analysis & Learning: to assess whether the activity is based on sound technical analysis and continuous learning.
The Review Team will be expected to draw out lessons from their assessment of each of the evaluation criterion that may be relevant to ongoing implementation of the DRM Program.
2. Provide recommendations that allow AusAID to make strategic decisions on the future direction of the DRM Program itself while also providing recommendations on the continuation of funding and nature of support for the following activities: a) Rabaul Volcanological Observatory Twinning Program due to end in September 2013 b) Natural Hazard Risk Assessment Mapping Program due to end in September 2013 c) Support to OCHA due to end in December 2012 d) Community based DRR activities due to begin in 2012/13 e) Support to the National Disaster Centre and West New Britain Province initial technical assistance support provided in early 2012 f) Red Cross Disaster Management Program due to end in January 2013
REFERENCE FRAMEWORKS:
The Review Team will base its analysis on the following Reference Frameworks:
1. AusAIDs Humanitarian Action Policy (2011)
2. OECD-DAC and AusAID criteria for evaluating development assistance
3. AusAID Disaster Risk Management Program (2010-2014) Performance Management Framework
4. Other relevant AusAID policies, including the Multilateral Engagement Strategy and Civil Society Engagement Framework
4 SCOPE OF THE REVIEW:
The Review Team will undertake the following:
Stage 1: Finalisation of TORs and Review Plan
1. Review these TORs once selected and if required, the Review Team will revise and update the TORs (to be accepted by AusAID in writing)
2. Develop Review Plan as outlined on page 4, the Team Leader (in consultation with other members of the Review Team) will submit a Review Plan that outlines the methods the Review Team will use to meet the TOR objectives and scope, including the evaluation questions
Approximate timeframe: 3 days
Stage 2: Document Review
1. Document review including AusAID DRM Program documentation (design document, Program Logic/Capacity Building Diagnostic, Performance Management Framework, Quality at Implementation reports, activity design and milestone reports, etc)
Approximate timeframe: 4 days
Stage 3: Field Visits and Consultations
1. Field visits including Port Moresby, Kimbe (West New Britain) and Kokopo/Rabaul (East New Britain)
2. Stakeholder meetings including with Government of PNG agencies (National Disaster Centre, Department of Mineral Policy and Geohazard Management, Rabaul Volcanological Observatory, National Weather Service, Provincial representatives from West and East New Britain, etc), PNG and Australian Red Cross, UN Office for the Coordination of Humanitarian Affairs (OCHA), Geoscience Australia, NGOs, other donors and members of AusAIDs Disaster Management and Emergency Response Teams.
Approximate timeframe: 10 days in-country
Stage 4: Reporting
1. Finalise the Independent Progress Review of the AusAID DRM Program, ensuring that this report addresses the objectives outlined in this TOR
Approximate timeframe: 7 days
5 REPORTING REQUIREMENTS:
1. Review Plan which outlines the methods and timeframe the Review Team will use to meet the TOR objectives and scope, including
a) an evaluation design that describes a logical model for assessing the DRM Program and its activities; b) a process for information collection and analysis (identify specific evaluation questions and how these will be answered, including those identified in Annex A); and c) any substantive challenges to achieving the evaluation objectives that will have to be addressed.
The Review Plan should also provide an outline of the roles and responsibilities of team members and target dates for deliverables. The Review Plan will be no more than three (3) pages and will be submitted to AusAID for approval two (2) weeks before the Review Team undertakes field visits.
2. Aide Memoire presented (and electronically submitted) to AusAID before departure from PNG. The Aide Memoire will briefly (no more than 5 pages) discuss the evaluation criteria and recommendations for future implementation of the DRM Program.
3. Draft Independent Progress Report due within three (3) weeks of completing the field visit to PNG. The Team Leader will be responsible for coordinating inputs from the Review Team and submitting this Report to AusAID. A template will be provided to the Review Team by AusAID.
4. Final Independent Progress Report due within seven (7) days of receiving AusAIDs written comments on the Draft Report. The Team Leader will be responsible for coordinating inputs of the Review Team and submitting this Report to AusAID.
TIMEFRAMES: The Review Team will be available from October December 2012 to complete the requirements of these Terms of Reference. Exact dates will be discussed directly with the Review Team, once selected.
REVIEW TEAM: The Review Team will be comprised of the following two (2) individuals:
6 1. Demonstrated experience and understanding of contemporary M&E strategies and systems, including in a developing country and/or Pacific region context. Experience in PNG is highly desirable.
2. Previous experience in reviewing and evaluating other DRM/DRR activities, including those supported by United Nations agencies, development agencies or other donors, is highly desirable
3. Understanding of AusAIDs M&E policies
4. Extensive experience in the use of participatory and consultative evaluation methods and techniques, including the collection and analysis of qualitative and quantitative data
5. Proven ability in effectively managing small teams and utilising the expertise of each team member in meeting Terms of Reference and contractual obligations (including reporting)
1. Strong familiarity with best practice initiatives, current issues, developments and trends in the humanitarian/disaster management sector (preferably disaster preparedness and risk reduction), including in a developing country and/or Pacific region context. Experience in PNG is highly desirable.
2. Understanding of AusAIDs disaster management/humanitarian policies and the Hyogo Framework for Action
3. Strong understanding of national disaster management organisations, their role, structure and any issues they face
4. A strong understanding of cross-cutting issues, particularly incorporating gender perspectives and ensuring the equal participation of men and women in humanitarian assistance, HIV/AIDS, environment/climate change, inclusion of those with special needs, and how these relate to disaster risk reduction and preparedness initiatives
Review Team members will also possess:
1. Strong liaison, communication, representation and negotiation skills
2. Ability to work effectively in a cross-cultural environment
7 AusAIDs Port Moresby based Disaster Management Team (Second Secretary and/or Program Manager) will accompany the Review Team for the majority of in-country field visits and consultations. It is expected that a Government of PNG representative will also accompany the Review Team.
REMUNERATION:
The Adviser Remuneration Framework is a set of market-based, long and short term remuneration rates applying to AusAID-funded advisers that are commercially contracted by AusAID directly or through a managing contractor. Further information on the Adviser Remuneration Framework can be found on AusAIDs website.
Considering the required skills and experience required to meet the obligations under these Terms of Reference, the following remuneration for these positions are:
As clearly outlined in the Adviser Remuneration Framework, AusAID will provide other costs, such as:
Fixed daily travel rates (or per diems) are prescribed by AusAID for each country, which are in line with equivalent rates paid to AusAID staff undertaking short term travel. Per diems will only be available for travel in PNG.
Adviser support costs are expenses incurred by the adviser that are directly related to their assignment. These costs are reimbursed by AusAID at cost and up to a prescribed limit set by AusAID in the contract. Support costs may include, but not be limited to: o Short-term accommodation o Airfares and other work related transport costs o Compulsory arrival and departure taxes o International communication costs o Medical insurance o Security costs, where applicable
Accommodation (hotel) costs for short-term advisers are prescribed by AusAID for each country, which are in line with equivalent rates paid to AusAID staff undertaking short term travel. Accommodation will only be reimbursed for hotels in PNG.
For individual advisers, management fees are limited to insurance costs that are not solely linked to the current assignment. Refer to the Adviser Remuneration Framework for further details.
8 For advisers engaged through a company, management fees must be clearly documented and justified. Refer to the Adviser Remuneration Framework for further details. 9 ANNEX A
These suggested standard evaluation questions for Independent Progress Reports have been provided as a guide for evaluation teams in developing questions that get the most value from the evaluation.
They are based on evaluation criteria that provide a comprehensive view of aid effectiveness. The evaluation criteria are: relevance to who, effectiveness for who.., efficiency, impact (if feasible) on who? Who benefitted? The aid activity must be rated against these criteria.
The questions can be used as provided, or can be adapted to be more relevant to the aid activity, country context and the size of the evaluation. The independent evaluation team can be asked to adapt the evaluation questions when they develop the methods design in the Evaluation Plan, including developing evaluation questions that assess relevant cross-cutting issues.
Specific questions should be developed to assess compliance with the Paris Declaration on Aid Effectiveness and the Accra Agenda for Action under the relevant criteria that are relevant to the activity. Questions for an Independent Progress Report Relevance Are the objectives relevant to Australian Government and partner government priorities? Are the objectives relevant to the context/needs of beneficiaries? If not, what changes need to be made to the activity or its objectives to ensure continued relevance? Effectiveness Are the objectives on track to being achieved? If not, what changes need to be made to objectives to ensure they can be achieved? To what extent has the activity contributed to achievement of objectives? Efficiency Has the implementation of the activity made effective use of time and resources to achieve the outcomes? Sub-questions: Have there been any financial variations to the activity? If so, was value for money considered in making these amendments? Has management of the activity been responsive to changing needs? If not, why not? Has the activity suffered from delays in implementation? If so, why and what was done about it? 10 Has the activity had sufficient and appropriate staffing resources? Was a risk management approach applied to management of the activity (including anti-corruption)? What are the risks to achievement of objectives? Have the risks been managed appropriately? I mpact (if feasible) Has the activity produced intended or unintended changes in the lives of beneficiaries and their environment, directly or indirectly? Have there been positive or negative impacts from external factors? Sustainability Do beneficiaries and/or partner country stakeholders have sufficient ownership, capacity and resources to maintain the activity outcomes after Australian Government funding has ceased? Are there any actions that can be taken now that will increase the likelihood that the activity will be sustainable? Are there any areas of the activity that are clearly not sustainable? What actions should be taken to address this? Gender Equality Was the activity designed to provide equal participation and benefits for women and men, boys and girls? Is the activity promoting equal participation and benefits for women and men, boys and girls? Sub-questions: Is the activity promoting more equal access by women and men to the benefits of the activity, and more broadly to resources, services and skills? Is the activity promoting equality of decision-making between women and men? Is the initiative helping to promote womens rights? Is the initiative helping to develop capacity (donors, partner government, civil society, etc) to understand and promote gender equality? Monitoring and Evaluation Does evidence exist to show that objectives are on track to being achieved? Is the M&E system collecting the right information to allow judgement to be made about meeting objectives and sustainability at the next evaluation point? Is data gender-disaggregated to measure the outcomes of the activity on men, women, boys and girls? Is the M&E system collecting useful information on cross-cutting issues? Analysis & Learning How well was the design based on previous learning and analysis? 11 How well has learning from implementation and previous reviews (self-assessment and independent) been integrated into the activity? Lessons What lessons from the activity can be applied to (select as appropriate: further implementation/designing the next phase of the activity/applying thematic practices [i.e. working in partner systems/environment/fragile stages] to the rest of the program/designing future activities).
Rating scale: Satisfactory Less that satisfactory 6 Very high quality 3 Less than adequate quality 5 Good quality 2 Poor quality 4 Adequate quality 1 Very poor quality
(GeoJournal Library) Dennis Rumley, Vivian Louis Forbes, Christopher Griffin - Australia's Arc of Instability - The Political and Cultural Dynamics of Regional Security - Springer (2006) PDF