NAME:

HOWARD MWANGI MAINA

INSTITUTION: ICE TRAINING AND CONSULTANCY INSTRUCTOR: DR. CHARLES AVUTIA COURSE: MONITORING AND EVALUATION

ASSIGNMENT: Writing an executive memo to the Vice Chancellor and his top level management staff at the
University of Nairobi (UoN) as an overall response to a set of questions raised by the Vice Chancellor at an internal senior management meeting in reaction to pressure posed by the World Bank to establish an institutional M&E system in the University of Nairobi. Some of his questions include: What is an M&E system? Why do we (UoN) need it? Hasnt our system been running fine without it in the past? What does it take to establish an M&E system? What are the potential benefits and risks associated with developing an M&E system?

ASSIGNMENT BACKGROUND:
University of Nairobi has just reached an agreement with the World Bank for a 30 million dollar loan to launch a 5year Education Improvement Project. Although the project has officially been approved and most of the major components already decided, an institutional M&E system capacity development is not yet finalized. The World Bank insists that the M&E capacity development needs to be part of the project (8% resources to be committed to the M&E development), but UoN does not seem to be convinced yet. However, the Vice Chancellor is genuinely interested in knowing about M&E and its potentials although he has some serious doubts about its benefits. He has two weeks from now to respond to the World Bank officials if the UoN will use some of the loan resources for the M&E system development. APA referencing style used, I have not considered the cover page and reference listing as part of the project so not part of the five pages limit
0

Memorandum
To: From: CC: Date: The Vice Chancellor University of Nairobi Howard Mwangi Monitoring and Evaluation consultancy and Specialist Top Management Staff. University of Nairobi 28th May, 2012

Subject: Monitoring and Evaluation System and Framework

I. PURPOSE

The purpose of this memorandum is to educate, create an understanding and provide deep insight into M&E Systems to the Vice Chancellor and the top level management staff of various schools and entities either affiliated to or part of the University of Nairobi. It is meant to aid in the decision making and implementation of an M&E system and framework with regards to the 5year education improvement project valued at 30million dollars. II. SUMMARY

The absence of an M&E system means lack of a centralized system of overseeing, adjusting and learning from programs or projects thus facilitating ineffectiveness and probably wastage of what economics calls scarce resources. The purpose of this memorandum will be achieved by equipping on what an M&E system is, what its necessity is, what it takes to establish an M&E system, the potential benefits and the risks associated with the development of an M&E system. III. WHAT AN M&E SYSTEM IS

Monitoring is the continuous collection of information and its analysis so as to track progress and performance (it checks if the activities are being implemented as planned) as a basis for decision making especially in the project. It observes, checks, records and keeps an account of day to day decisions and activities and provides data and information for evaluation. Evaluation is the assessing or judging the value of something. It is a periodic process of establishing the extent and the effectiveness to which a program or project has achieved its goals or objectives and also provide a basis for policy

Page 1

making. In other words it judges, values and assesses major decisions and activities and provides information and data for planning. In summary, Monitoring and Evaluation system is that process involving the continuous collection of information and necessary data then subject it to analysis and interpretation with the aim of establishing if things are advancing according to plan and if there is progress towards the pre-specified goals and objectives. Similarly to establish if there are any unintended externalities, either affecting the project or arising from the project and its activities (Memory, 2010, p.3). Monitoring and Evaluation is not a stage in a project cycle but a part of the whole project that starts from the beginning to the end. Monitoring is done continuously but evaluation is done periodically (at intervals especially at project gates). IV. WHY THE NEED FOR M&E IN THE UNIVERSITY OF NAIROBI

The University has numerous programs in its various faculties and departments, therefore the need for an M&E system. The systems importance with additional supportive material from Norman, Martin & Chandavanh (2005, p.6) can be summarized into: Relevance Effectiveness Efficiency Impact Sustainability Accountability

Relevance It is the way in which the M&E system seeks to find the appropriateness of a program/project, prescribed activities and the approaches to project implementation. It seeks answers to questions like: Does/did the project address the focus issues or problems facing the target areas and communities/beneficiaries? Was/is the project/Program consistent with policies of the entity spearheading the project (the University of Nairobi) and the donor/other stakeholders (the World Bank)? Effectiveness The M&E system seeks to find if the right things are being done (monitoring) or have the activities, outputs and outcomes been met and successfully. Efficiency Seeks to find if the prescribed activities, strategies and approaches are being done in the right way such that are/have inputs (human, financial, material, time resources) been used to produce optimal output, if not then could there have

Page 2

been a better way to do things? It at times involves finding if the people executing the project are qualified to do so i.e. expertise and professionalism not leaving out level of experience in the activities in question. Impact M&E captures the periodic and overall contribution of the program/project. It also seeks to find out the effects and externalities (both positive and negative) from the project. E.g. are more people seeking enroll at UON as they seek better education? Sustainability M&E is interested in whether the program or/and its benefits or impact will continue even after the funding is over (the $30million is over) or after the project is terminated. I.e. after 5 years of the education improvement project. Accountability M&E provides an avenue of gathering information about management of resources thus making the institution funded credible thus increasing the confidence of stakeholders whose result is enhancing opportunities for further funding of this project or for different projects of the University. V. WHAT IT TAKES TO ESTABLISH AN M&E SYSTEM

M&E is not a stage or an activity in a project but a process that is integrated in the life cycle of a project from beginning to end (from conception/initiation, planning, execution/implementation and close out/termination. M&E is planned at the project design and implemented in the projects execution, termination and post termination stage. The development of an M&E system is in seven main steps: (Memory, 2010, p.5-11) (Irene, G., Jim, w. 2002, section 4) 1. Assessing the existing readiness and capacity for monitoring and evaluation This involves assessing the existing capacity of the organization (UON) and stakeholders or partners that are or will be involved in the project. This are the resources available for the project more to it for M&E, i.e. the funds, human resource and their skills and level of expertise in their varying fields, time and the quality of data and information systems. Similarly, possible barriers to the M&E have to be considered and identified e.g. bureaucracy, lack of expertise/experience and tribalism/nepotism which has been recently established in the local dailies as a characteristic of majority of public universities. Additionally options such as training may be considered in an attempt to minimize the barriers as much as possible. 2. Establish the purpose and the scope This entails establishing why we need the M&E system (what do we want to accomplish with it?). We define the scope of the M&E. That is; how comprehensive should it be, will it be participatory and similarly define the level of participation in monitoring and evaluation of every stakeholder. The established purpose and scope of the M&E now

Page 3

guides us in the ensuing steps of an M&E system setup which involve; establishing M&Es indicators to track, information need, information type, budget level, tools needed for M&E and the frequency of data collection and dissemination. 3. The identification of indicators, information needs and performance questions This step is based on the projects objectives, goals and expected outputs, outcomes and impact (i.e. the logical framework). An M&E data is best qualitative and quantitative for effectiveness. We need to establish information needs and indicators for which to collect data. A project has complex objectives and goals thus a single or few indicators are not appropriate but rather establish comprehensive indicators that capture all the stages, gates, activities, outputs, outcomes and impacts of a project so as to effectively and efficiently measure project performance and achievements. Each indicator should be specific, measurable, relevant, clear and without ambiguity, consistent (comparable values of indicators data even over time with same methods used), sensitive (they capture even small changes or effects) and attributable (based on evidence e.g. statistics and not only on cause-effect relationship). There are different types of indicators; Input indicators, process indicators, output indicators, outcome indicators, impact indicators and exogenous indicators (covers factors outside the project that may have effect or influence on the project e.g. elections and politics in the university). We also need continuous feedback thus establish on what we need the feedback and from who e.g. the targets who are the students (e.g. is the curriculum enhancing marketability?) 4. Planning information gathering and organization For every information need and indicator we need to establish how to collect data and organize it. Ensure both qualitative and quantitative data, taking into consideration different staff and their expertise, abilities and skills in relation to their various fields. In choosing an appropriate data collection method, perform cost analysis of different methods so as to choose the best (e.g. cost benefit analysis). Examples of data collection methods are observation, interviews, sampling, focus group discussions, ground reports and surveillance among others. 5. Data processing and Plan critical reflection processes and events The collected data needs to be synthesized/analyzed and interpreted. This step involves making sense of the data generated by M&E. The personnel involved in data processing (e.g. data entry and synthesis using statistical software) should be skilled or else trained. Data quality should be considered, good quality data is accurate, with integrity, valid, reliable, complete, precise, timely, confidential if need be, clear (not ambiguous) (USAID, 2011, data quality dimensions, p.2-9). Then there are interim evaluations, critical reflection processes and events e.g. through participatory reviews, meetings and workshops with staff or key stakeholders so as to finally make sense of the information generated by M&E.

Page 4

6. Communication and reporting This is the dissemination of M&E findings to relevant stakeholders, partners, beneficiaries, staff (i.e. project manager), policy makers/the government and possibly the general public as a way of contributing to the existing pool of knowledge. These people do not receive all the information acquired and in the way/form it was after data processing. Instead the information is in different contents, presentation forms and formats based on who the recipient is and their intended use for that information. There needs to be a reporting schedule of when to disseminate given information to a particular project stakeholder e.g. the partner (World Bank), it could be monthly or even annually, to the project manager it could be weekly. Communication may be done in form of journals, articles, publications, trainings, workshops, reports and among others. 7. Necessary conditions and capacities This involves facilitating the sustenance of the M&E system. It entails planning the organizational structure for M&E (this is participatory involving the university and other stakeholders), the necessary human capacity (sufficient staffing levels and types), financial resources, Information management systems and the incentives for implementing M&E developed. The responsibility and requirements of each stakeholder is also to be factored in the M&E system.

VI.

POTENTIAL RISKS AND BENEFITS ASSOCIATED WITH DEVELOPING AN M&E SYSTEM

Most the benefits and risks have been adapted from The Meeting of presidents of technical commissions. (2009). RISKS: M&E success heavily relies on data hence low quality data (unreliable, invalid, incomplete, inaccurate etc.) is a risk that may affect the M&E system and inhibit success. Another risk is the diversion of project resources to M&E such that project operations are weakened by a high cost of data collection and other M&E activities like staff payment. Similarly the resources devoted for M&E should not be little as to impair the quality of data collected and the motivation of M&E staff. The benefits of M&E are difficult to measure in financial terms thus a risk of management neglecting the M&E system which generally cripples the project results as resources are not properly utilized and no information is generated to guide the running of the project and even produce lessons learnt. BENEFITS: An M&E system is beneficial in that it is able to analyze the need of clients as the project starts and develops. It is able to incorporate lessons and best practices learnt from projects into guidelines of operation. It is able to disseminate acquired information and lessons to the general public e.g. through publications. It is able to organize staffing, enhance resource mobilization and improve program delivery & management. It coordinates project staff. It provides a platform for building up experience in programmatic operations and facilitates evidence-based decision making.

Page 5

REFERENCES LISTING Memory, C. (2010). Module 5: Designing Monitoring and Evaluation Systems for NEPAD projects. UNCEA. Norman, W., Martin, S. & Chandavanh, D. (2005). Vientiane: Mekong Wetlands Biodiversity Conservation and Sustainable Use Programme. Irene, G., Jim, w. (2002). A Guide for Project M&E. Geneva: IFAD. Office of Evaluation and Studies. USAID (2011) Data quality, data quality dimensions. Retrieved April 29, 2012 from www.globalhealthlearning.org (data quality) The Meeting of presidents of technical commissions. (2009). World Meteorological Organization, Geneva, 2-4 February 2009.

Page 6