Monitoring, Evaluation and Lesson Learning Guidelines

1. Introduction
These monitoring, evaluation and lesson learning guidelines are for projects supported by DFIDs Development Awareness Fund (DAF). The guidelines explain why it is important to develop good systems for monitoring and evaluation and for learning and sharing lessons. They provide details on the DAF requirements for monitoring and evaluation, with specific guidance on project frameworks, monitoring and evaluation plans and reporting requirements. Good monitoring and evaluation is important because these activities:

support accountability for the effective use of funds, both upwards to DFID, parliament and UK public and downwards to the target group a project seeks to benefit; improve performance by focusing projects on delivering outcomes and impact, provide a system to record lessons learnt which can be shared and used by project managers, DFID and others to improve the design and performance of projects and programmes.

2. The DAF Monitoring & Evaluation Framework

Monitoring is a continuous internal process, conducted by project managers to check the progress of development interventions in this case DAF projects against pre-defined objectives and plans, as set out in the projects planning framework. Evaluations usually take place towards the end of the project. They ask what has actually been achieved, what happened and why, and answer specific questions related to the relevance, effectiveness, efficiency, impact and sustainability of the project. Evaluations make use of the information recorded during the monitoring process. The DAF uses a mixture of monitoring and evaluation to fulfil both lesson learning and accountability functions. The DAF intends that important lessons are learned from and shared so that they can contribute to improvements in development awareness policy and practice. Monitoring and evaluation should be seen as an integrated reflection and reporting system for project managers and others engaged with the project. Monitoring and evaluation must be planned for, managed and resourced. These tasks are essential for good project management and should not be considered as just a statistical task or an external obligation. The main project planning tool used as the basis for DAF project monitoring and evaluation is the project framework document.

3. The Project Framework

The project framework was introduced as a compulsory component of DAF proposals in 2009 to strengthen the design, implementation, monitoring and evaluation of projects. It is an important management tool to describe logically how particular activities are expected to contribute to the achievement of the outcomes or changes that will together produce an overall change as described in the project purpose. The project framework is useful for anyone involved in a DAF project, as it:

Clarifies how the project is expected to work and helps to ensure that there are clear links between activities, outcomes and purpose; Brings together in one place a clear, concise and accessible explanation of all the key components of the project; Enables all those involved with project implementation to develop a common understanding of what the project is aiming to achieve and how, and Provides indicators and targets (qualitative and quantitative) that can be used as the basis for assessing project progress and success.

To ensure these processes work well, DAF projects should:

Ensure that the project framework includes clear purpose and outcome statements for the project ; Establish specific and measurable and time-bound indicators and targets for the project purpose and outcomes;. Establish clear responsibilities for monitoring and evaluation; Make sure staff time and a budget are set aside for data collection, monitoring, evaluation and reporting activities, and Set up monitoring systems early, including the collection of baseline data to be used as a reference point for changes resulting from the project.

At the proposal stage the framework helped to demonstrate the project design and the logical link between activities, outcomes and purpose and the indicators helped to clarify expected, measurable changes. Throughout the project implementation period, the project framework will be used as a key reference document for project management, monitoring and evaluation. The framework will be reported against and reviewed in the Annual Reports and at the end of the project in the Project Completion Report and any project evaluations. As the project framework is such an important tool for both project management and monitoring, at the beginning of the project it is considered necessary to review the document prepared at the proposal stage, and if necessary amend and clarify the content to ensure that it satisfies these requirements. This review process will be undertaken as part of a Project Inception Plan process, to be followed on all DAF projects starting in or after 2009.

4. Project Inception Plans

Within three months of signing the DAF Grant Arrangement, all new projects (from 2009 0nwards) will be required to prepare a Project Inception Plan that comprises:

An updated project framework: including refined and clarified outcomes and indicators, and linked to a clear schedule of activities. A project monitoring and evaluation plan: clarifying the arrangements for collecting and analysing relevant information.

4.1. An updated project framework

At the Project Inception Plan stage, the outcomes and indicators prepared for the proposal may need to be refined to ensure that they are appropriate for use as a basis for project monitoring and evaluation. However, it is important to note that any changes made to the purpose and outcomes must not represent fundamental changes to the project focus as presented in the proposal and all changes to the project frameworks will need to be approved in writing. At the Project Inception plan stage, the key elements of the framework need to be reviewed using the following guidelines: Project Purpose: The project purpose should summarise what overall change the project will achieve and clarify the link between the project outcomes and the selected DAF programme objectives. The purpose should be fully achieved by the time the project is finished. Project Outcomes: The outcomes should describe the main changes in knowledge, understanding, skills and attitudes expected to result from the project activities and contribute to the achievement of the project purpose. For some projects, the expected outcomes may be in the form of changes to the behaviour, policies and practice of individuals or organisations. The outcome statements should clarify the specific target groups and the nature of the anticipated changes. Indicators: DFID defines an indicator as a quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor. The indicators should provide clear and timetabled measures of progress and achievement of targets compared to a measured baseline state. In order to work effectively as a management tool, the project framework must include indicators that are carefully selected to be clear, relevant and practical to measure. Generally, a few strategically selected indicators will be more useful than many. Indicators must also be SMART: i.e. Specific, Measurable, Attainable, Relevant and Timebound. Baseline Data: In order to measure the nature and extent of changes resulting from project interventions it is essential to establish the pre-project, or baseline situation as a reference point. The purpose and outcome indicators included in the project framework and monitoring and evaluation plans must therefore include comparisons to relevant baseline information. This

baseline data may have already been collected as part of the project planning process, but if not, it will need to be established during the project inception phase. Activities: The main actions or tasks that need to be undertaken to achieve the project outcomes. Activities also need to be scheduled over the life of the project and details provided to clarify the quantitative and qualitative targets e.g. the description of training activities should include the number of planned training activities, numbers of people to be trained, the target group and the expected outcome of the training activity. The DAF 2010-11 Project Framework and Activity Schedule form can be used to prepare the refined project frameworks with scheduled activities.

4.2 A project monitoring and evaluation plan

The project monitoring and evaluation plan identifies how progress against the indicators will be measured, by whom, when and how frequently. A suggested format for the monitoring and evaluation plan and an associated checklist is available. This format may be adapted to suit particular project circumstances, but should include all the key components. Most project plans also include provision for an independent external evaluation to be carried out towards the end of the project. This provides an independent assessment of achievements and positive or negative factors affecting performance.

4.3 Link to achievement of DAF programme objectives

All DAF projects are expected to contribute to the achievement of one or more of the four published DAF programme objectives. The project framework document identifies which of the DAF objectives the project is supporting, and the purpose statement should help to clarify the link between the project outcomes and specific DAF programme objectives. The Inception Plan review and appraisal will aim to ensure that project frameworks, monitoring and evaluation plans and activity schedules enable both grant holders and DFID to assess project progress and the projects contribution to the achievement of the DAF programme objectives.

5. DAF Reporting requirements

The inception plan project framework, activity schedules and monitoring and evaluation plans provide the basis for project monitoring and progress reporting to DFID (via Triple Line) through a number of reporting mechanisms. These include expenditure reports (submitted as part of the payment claims process, usually quarterly), annual progress reports, project evaluation reports and project completion reports. The monitoring and evaluation and reporting requirements for DAF projects are listed in the table below:

Requirement of grant holder A Project Inception Plan including a refined project framework, activity schedule and monitoring and evaluation plan. Statements of expenditure confirming actual expenditure against budget.

Timing

DFID/Triple Line responsibility

Three months after signature of the Review the inception plan and DAF grant arrangement provide approval or feedback

With payment claims usually quarterly

Review and query or, if in order, process for payment

An Annual Report based on Due 30th June each year (except Provide feedback and consider regular monitoring and for the final year of the project). any proposed changes to the reflection over the year All annual reports should cover the project framework and activity involving the project team and period of the DFID financial years schedule to ensure that the other people or organizations i.e. from 1st April to 31st March. project remains on track and all engaged in project delivery N.B. If a project covers less than parties have a common and up to check whether the project is 3months of a reporting year, there to date understanding about on track to achieve the project is no need to submit an annual what the project is aiming to purpose and outcomes and to report for that year. Instead it achieve and how. identify factors affecting should be rolled into the previous performance and any changes or subsequent annual report as required to keep the project on appropriate. track. Where undertaken, final evaluations should verify Account for the use of DAF achievements and contributions Towards the end of the project and funds, appraise the contribution towards DAF programme before completing the Project to the DAF programme objectives and note what Completion Report. Submit to objectives and use lessons to happened and why, and identify Triple Line with Project develop knowledge across lessons from the process of Completion Report. DFID and other development implementation that may be of partners. use to others. Complete the Within 4 months of the end of Project Completion Report DFID funding period. Form including a summary of the projects achievements, lessons learnt and the contribution made towards the achievement of the DAF programme objectives. Provide feedback on the reported achievements and identify and further information required to allow closure of the grant.

In addition to the above, grant holders may also be asked to provide material for case studies to be used in DFID internal and external communications to illustrate the impact of our work. All reports require grant holders to complete an Achievement Rating Scale (ARS). The format for this is shown in the templates for each type of report. It is important that the overall score and justification is filled in.

6. Changes to the approved project framework documents

The agreed project framework forms the basis of the contractual relationship between DFID and the project team. DFID expects that the project will achieve what was stated in the purpose by the end of the project in exchange for the funding supplied. However, as the project framework is a prediction, the situation on the ground may change during the course of project implementation, requiring adjustments, particularly to the activities required to achieve the outcomes and purpose. The indicators may also need to be amended, particularly early on in the project when baseline data becomes available. The project purpose and outcomes are not expected to change during the lifetime of the project. Any proposals to change the project framework documents should be communicated to Triple Line as soon as possible via email and no significant changes should be implemented until written approval is received. Once agreed, any revisions to the plans will be used as the basis for further monitoring, evaluation and reporting. By the end of the project, outcomes may not be exactly as planned. However, there will still be some impact and change, both intended and unintended, positive and even in some cases negative changes. This information, provided in Annual Reports, Project Completion Reports and evaluations needs to be recorded so that important lessons can be learned and used to inform future project and programme design.

7. Lesson Learning and Dissemination 7.1 Identifying Lessons

During the lifetime of a project many lessons will be learnt. Sometimes there will be lessons from success and sometimes they will be hard but useful lessons learnt from failure. These lessons may help others to build on what worked well or avoid similar challenges. To tease out lessons, project teams need to understand what works - or has not worked - where, with whom, under what circumstances, and why. This means being able to analyse the context in which the work was done, what work was done and what the outcomes of that work were. Lessons might arise from interesting problems and how they have been tackled, in what circumstances and with what success. There might be other lessons about the assumptions that were made and how these turned out not to be right. Other people may avoid making the same assumptions. Risks may have been poorly identified, or could have been managed better. Lesson learning can only happen when there is time to reflect on practice, identify lessons, publicise them to others and when others have the chance to absorb and apply the lessons. Ideally

project teams should schedule regular meetings for this type of reflection on the project and to record monitoring information. In the first instance monitoring should be giving the project team and stakeholders the chance to learn lessons and improve their practice. This requires openmindedness in reviewing progress so that difficulties can be acknowledged rather than disguised.

7.2 Audience and means of dissemination

Lessons need to be targeted to be shared effectively. It may be helpful when listing lessons to name whom you think the specific audience for the specific lesson might be. This will help others to organise the information in the most accessible way. Different audiences might include DFID, other funding agencies or organisations engaged in development education/development awareness The Annual Report and Project Completion Report provide opportunities for documenting lessons learned. DFID can use these lessons internally to inform future planning, policy and strategy. Projects should also consider how information about the project and lessons that may have widespread value can be disseminated through their own networks, using newsletters, publications, workshops and seminars and by allowing regional staff to meet together to share experiences.