Evaluation Plan

Impact of KLIP activities on governance and policy

making in Kisumu

Date:

21st July 2014

Trim Ref #:

09/

Evaluation Manager:
Contact Person:

Name:
Position title:
Contact Phone:
Business Unit:

Table of Contents
Contents
1.

Background .........................................................................................................................1
1.1

2.

Evaluation Governance........................................................................................................1
2.1
2.2
2.3
2.4
2.5
2.6
2.7

3.

Purpose of the Evaluation ..........................................................................................................1

Core Evaluation Questions .........................................................................................................2
Evaluation Stakeholders .............................................................................................................2
Evaluation Management Structure .............................................................................................3
Evaluation Governance Group Membership ..............................................................................3
Evaluation Governance Group Terms of Reference ..................................................................4
Reporting Requirements .............................................................................................................4

Evaluation Design ................................................................................................................5

3.1
3.2

4.

Project/Program/Initiative Definition ...........................................................................................1

Evaluation Framework ................................................................................................................5

Information and Collection ..........................................................................................................5

Evaluation Resources and Timeframe .................................................................................5

4.1
4.2
4.3

Budget and Staff Resources .......................................................................................................5

Evaluation Staff Roles and Responsibilities ...............................................................................7
Time Schedule ............................................................................................................................7

Appendix 1 Evaluation Plan Checklist ...........................................................................................11

1.

Background

1.1

Project/Program/Initiative Definition

KLIP is part of a consortium of organizations funded by Swedish International

Development Cooperation Agency (Sida) and Mistra Urban Futures (M-UF) with the sole
aim of promoting sustainable urban development. KLIP therefore strives to achieve the
set out goals of Fair, Green and Dense city of Kisumu by incorporating two thematic
pillars of: a) Eco-Tourism and b) Market Places. The two themes aims at addressing
urban challenges like food insecurity, climate change, poverty, urban insecurity among
others within Kisumu city and its environs. The partnership between Kisumu LIP and
participating stakeholders seeks to overcome the threats to food security, urban
insecurity, and skewed distribution of factors of production in a changing environment;
exploring new ways of helping vulnerable urban communities adjust to global changes
and explore other factors of production other than fishing along the adjacent beaches.
This consultancy exercise is a result of the collaboration between KLIP Sida and MistraUrban Futures and its main purpose will be to evaluate the flagship programs of Ecotourism and Market places under the perspective of the various stakeholders, who
include KLIP Trustees, Kisumu Local Government Leaders, Various Private Sector
organizations and the local residents. The whole process will establish the impact of the
activities since their inception and also model future activities based on the results
obtained.

2.

Evaluation Governance

2.1

Purpose of the Evaluation

The purpose is to identify the societal impact that has been achieved so far, or has the
potential to be achieved, through the support of KLIP project and capacity building
activities
Evaluation Plan for KLIP
Evaluation Manager: Dr. Oketch
TRIM Number: 00900

2.2

Core Evaluation Questions

In fulfilling the purposes of the evaluation, the following core questions were addressed:

How has the activities carried out by KLIP, influenced governance and policies of
collaborating public and private sector organizations?

How has KLIP impacted in the society so far, and what can it achieve in the future?

How can the programs in KLIP be further improved for achieving the organizations vision and
mission?

2.3

Evaluation Stakeholders

KLIP TRUSTEES
Under the banner of SIDA and M-UF, KLIP Trustees formulated programs that were targeted at:
1) Supporting Ecotourism and Market Places as the flagship projects
2) Strengthening of the Secretariat through participation in the strategic development of Mistra
Urban Futures; and facilitating the establishment of the KLIP Trust.
3) Promoting Knowledge and Technology Transfer as well as Capacity Building for County
Governments via the.
In this evaluation process, we will try to establish the reasons behind formulation of the programs
by the trustees, their methods of implementation, possible success and failure stories, as well as
the future scope of the projects from the trustees point of view
Kisumu Local Government Leaders
For any project to succeed, participation of local leaders is paramount. And so KLIP partnered
with various local leaders holding various positions, in the implementation of the various projects,
with special focus in Eco-Tourism and Market place.
As a means of evaluation, we will establish the impact of the programs on the governance in
Kisumu. We will find out, how the partnership was carried. Possible success and failures
throughout the partnership and future modifications that will foster better partnership in the
various projects.
Kisumu Private Sector Organizations
Evaluation Plan for KLIP
Evaluation Manager: Dr. Oketch
TRIM Number: 00900

One of the goals of KLIP is to bridge the gap between the private sector and the local
government. Through the various programs, we will evaluate the collaborations created in the
process between the private sector and the local government. We will find out, how these ties
have improved the regions economy. In addition, the evaluation will highlight possible
drawbacks encountered through the partnerships created, and how these can be improved for
future collaborations
Kisumu Residents
KLIP always puts the needs of the locals in the programs and projects it formulates. The goal of
KLIP is cogeneration of knowledge, practical application and dissemination for the purposes of
sustainable urban development. Through the various activities of Eco-tourism and Market
places, the evaluation will establish the following aspects:

Acceptance and attitude of the locals to the various projects

Training involved in the various projects

Impact of the projects on the locals lifestyles

Future proposals for the projects and programs

2.4

Evaluation Management Structure

Position

Name

Assistant-Director General

Dr.

Director

Dr.

Project/Program/Initiative Manager

Mr. Owuor

2.5

Evaluation Governance Group Membership

Position

Name

Research Assistant

Mr.

Evaluation Plan for KLIP

Evaluation Manager: Dr. Oketch
TRIM Number: 00900

Data Analyst

2.6

Mr.

Evaluation Governance Group Terms of Reference

The roles and responsibilities of the Evaluation Governance Group include:

Roles and responsibilities

Project Consultant Overall in charge. Supervises the project.
Project Manager- Supervises day to day activities in the evaluation cycle. Reports to consultant
Research Assistant In charge of data collection
Data Analyst Analyses the data
Field Guide Assists in data collection
Field Support staff provides as need service to the team

2.7

Reporting Requirements

Date

Report type

Writer/s

Audience

08/08/2014

Progress Report

Project Manager

Evaluation Steering
Committee

10/08/2014

Draft Report

Project Consultant

Evaluation Steering
Committee

12/08/2014

Quality Assurance
Review

Contracted Consultant

Evaluation Steering
Committee

13/08/2014

Final Report

Project Consultant

KLIP Director

Evaluation Plan for KLIP

Evaluation Manager: Dr. Oketch
TRIM Number: 00900

3.

Evaluation Design

3.1

Evaluation Framework

The Guide below shows the framework to be applied during the evaluation process

3.2

Information and Collection

Core evaluation
question to
address

Information
required

Information
sources

Data collection and

analysis

What are the reasons

behind setting up the
various programs and
activities in KLIP?

KLIP Programs and

Activities

Literature available
since 2010

Literature review
Interviews

What is the impact of

the KLIP programs on
governance in
Kisumu?

Influence of KLIP
activities on
Governance

Key stakeholder
Local Leaders
Interviews

Questionnaire

What is the impact of

KLIP activities in
fostering private
sector government
collaborations

Influence of KLIP on
fostering private
sector local
government relations

Key stakeholder
Private organization
partnered in the
activities - Interviews

Questionnaire

Key stakeholder
Interviews
Literature reviews

Questionnaire

How has the activities Economy elevation of

impacted on impact
the locals
local residents lifestyle

4.

Evaluation Resources and Timeframe

4.1

Budget and Staff Resources

Evaluation Plan for KLIP

Evaluation Manager: Dr. Oketch
TRIM Number: 00900

Budget for the evaluation study

Internal DET staff
resources

Position

Classification
level

Approximate time
and / costs required

Dr. Oketch

Team Leader

Top level

80 hours Kshs
150, 000

Project Manager

Middle level

50 hours Kshs 75,000

Research Assistant

Middle level

50 hours Kshs 75,000

Data Analyst

Middle level

50 hours Kshs 50,000

External human resources

Approximate time and cost required

Data Entry Technician

40 hours Kshs 35,000

Field Guide

80 hours Kshs 20,000

Field Support Staff

20 hours Kshs 5,000

Other resources required

Cost estimates

Program Logic Modelling session

Accommodation & Travel

Evaluation Plan for KLIP

Evaluation Manager: Dr. Oketch
TRIM Number: 00900

Data tools generation

Kshs 5,000

Questionnaires
booklets

Kshs 2,000

Interview recording
devices (4)

Kshs 15,000

Video recording
devices (1)

Kshs 23,000

Stationary

Kshs 5,000

Totals

Kshs 45,000

Committee
Members

Bus Return Ticket

Kshs
5,000

Kshs25,000

3 Day Hotel bed &

breakfast

Kshs
4,000

Kshs20,000

Total

Kshs 45,000

Allowances for Evaluation Committee

4.2

Food Allowances

Kshs 10,000

Airtime Allowances

Kshs 10,000

Transport Allowances

Kshs 10,000

Miscellaneous

Kshs 15,000

Total allowances

Kshs 45,000

Evaluation Staff Roles and Responsibilities

Internal DET
evaluation team
members

Position

Roles and responsibilities

Dr. Oketch

Project Manager

Overseeing project evaluation process

Mr. Owuor

Research Assistant

Supervision of data collection

Mr. Oloo

Data Analyst

Data cleaning and analysis

External
consultant(s)

Position

Roles and responsibilities of the

consultant(s)

Dr. Okecth

Project Consultant

Consultation on project activities

4.3

Time Schedule

Time/Activities

8:00-10:00AM

11:00-12:00AM

2:00-4:00PM

5:00-7:00PM

31/07/14

Evaluation
Commencement

Approval
process for the
evaluation

Project
Committee 1st
meeting

Project goals
reviewed

01/08/14

Program Logic Model generated

Evaluation Plan for KLIP

Evaluation Manager: Dr. Oketch
TRIM Number: 00900

Specific Objectives for team

members formulated
02/08/14

HOLIDAY

03/08/14

HOLIDAY

04/08/14

Data Tools Generated

Data tools reviewed and revised

05/08/14

Data Collection (interviews)

Data Collection (interviews)

06/08/14

Data Collection (Questionnaires)

Data Collection (Questionnaires)

07/08/14

Data Entry

Draft Project progress Report

08/08/14

Draft Project Report Presented to

Evaluation Governance Team

Feedback from Evaluation

Governance Team

09/08/14

Data Cleaning and Analysis

Quality Assurance consultant

contracted

10/08/14

Data Analysis Complete

Progress Report to evaluation

steering committee

11/08/14

Draft Project Report Presented to

Evaluation Governance Team

Report compilation

12/08/14

Review/Endorsement of report by
the Evaluation Governance Team

Report revision

13/08/14

Final Report approved by

evaluation steering committee

Presentation of final report to KLIP

trustees

Evaluation Plan for KLIP

Evaluation Manager: Dr. Oketch
TRIM Number: 00900

Evaluation Plan for KLIP

Evaluation Manager: Dr. Oketch
TRIM Number: 00900

10

Major milestones

Due date

Consultation with Evaluation Services regarding support and

approval process for the evaluation

31/07/14

Program Logic Model (or evaluation framework) generated

01/08/14

Evaluation Governance Group established with endorsed Terms

of Reference

04/08/14

Draft Evaluation Proposal presented to Evaluation Governance

Group for feedback

05/08/14

Evaluation Proposal endorsed by Evaluation Governance Group

06/08/14

Evaluation Proposal approved by Evaluation Steering Committee

07/08/14

Consultant contracted (if applicable) or Evaluation Team

appointed

08/08/14

Progress Report to Evaluation Steering Committee

08/08/14

Data gathering and analysis complete

10/08/14

Progress Report to Evaluation Steering Committee

11/08/14

Draft Evaluation Report presented to Evaluation Governance

Group for feedback

12/08/14

Draft Evaluation Report endorsed by Evaluation Governance

Group

12/08/14

Final Evaluation Report approved by Evaluation Steering

Committee

13/08/14

Brief with any recommendations to the Policy and Performance

Committee with any recommendations

13/08/14

Evaluation Plan for KLIP

Evaluation Manager: Dr. Oketch
TRIM Number: 00900

11

Appendix 1 Evaluation Plan Checklist

This checklist has been included to assist in the development of the Evaluation Plan.
Socio-political Factors
The team should identify and effectively address affected/concerned groups
Involvement

Whose sanction and support is required?

How will it be secured?

Internal communication

How will key audience needs for information on the

evaluations progress be determined and met?
How will communication be maintained between the
program team, the evaluation team and the Evaluation
Steering Committee (ESC)?

External credibility

Will the evaluation be free of bias?

Realistic expectations

How will the evaluation team make it clear to

stakeholders that realistically only a subset of their
information needs will be addressed?

Security

What provisions will assure security of the data?

Protocol

What communication channels will be honoured and

employed?

Public relations

How will stakeholders be consulted and kept informed

about the intents and results of the program evaluation?

Stakeholder confidence

What checks will be made to ensure that the evaluation

plan and the composition of the evaluation team are
responsive and acceptable to the key stakeholders?

Contractual/Legal Arrangements
The evaluation team and participants should establish clear working agreements to ensure
efficient collaboration and protect involved parties rights
ESC, evaluation team,
and other roles

Who is the sponsor

Who is on the evaluation team
Who are the other audiences
How are they related to the program?

Realistic commitments

What clarifications assure that the evaluation can

proceed while making reasonable efforts to serve a
broad audience but not becoming bogged down in over
identifying and consulting with stakeholders?

Evaluation Plan for KLIP

Evaluation Manager: Dr. Oketch
TRIM Number: 00900

12

Contractual/Legal Arrangements
The evaluation team and participants should establish clear working agreements to ensure
efficient collaboration and protect involved parties rights
Delivery schedule

What is the schedule of evaluation services and

products?

Editing reports

Who has authority for editing evaluation reports?

Access to data

What existing data may the evaluation team use

What new data may they obtain?

Access to stakeholders

Are there sufficient safeguards to assure that the

evaluation team may contact involved stakeholders?

Pre-release reviews

Will the ESC be provided appropriate opportunities to

review the draft report for clarity and fairness prior to
their finalisation and release?

Release of reports

Who will release the reports

What audiences may receive them?

Responsibility and
authority

Have the system personnel and the evaluation team

agreed on what persons and groups have both the
responsibility and authority to perform the various
evaluation tasks?

Finances

What is the schedule of payments for the evaluation

Who will provide the funds?

Technical Design
The evaluation team should convert a general evaluation plan to a detailed, yet flexible technical
plan
Objectives

What is the program intended to achieve/produce

In what terms should it be evaluated?

Variables

What classes of information will be collected, e.g.,

context, inputs, processes, outcomes?

Program description

Will the program be described sufficiently, so that

stakeholders will understand its nature?

Investigatory framework

Under what conditions will the data be gathered, e.g.,

experimental design, case study, survey, site review,
examination, etc.?

Instrumentation

What data gathering instruments and techniques will be

employed
How will the evaluator assure that they address the key
evaluation questions?

Sampling

Evaluation Plan for KLIP

Evaluation Manager: Dr. Oketch
TRIM Number: 00900

What samples will be drawn

How will they be drawn

13

Technical Design
The evaluation team should convert a general evaluation plan to a detailed, yet flexible technical
plan

Will they meet both utility and technical requirements?

Data gathering

How will the data gathering plan be implemented

Who will gather the data?
Have multiple modes of data gathering been planned to
ensure triangulation?

Data storage and

retrieval

What format, procedures, and facilities will be used to

store and retrieve the data?
What strategies are in place to ensure the data complies
with ethics and codes of conduct?

Data analysis

How will the data be analysed?

Sources of interpretation

Who is charged to interpret findings, e.g., the evaluators,

various stakeholders, a regulatory body, etc.?

Bases for interpretation

What bases will be used to interpret findings, e.g.,

objectives, assessed needs, contractual specifications,
laws and regulations, democratic ideals, social norms,
performance by a comparison group, technical
standards or criteria, polls, judgments by reference
groups, etc.?

Methods of interpretation

What methods will be used to assign value meaning to

findings, e.g., focus groups, a Delphi study, advocacy
and adversary reports, etc.?

Reports

What reports will be used to disseminate the evaluation

findings?

Reporting media

Considering the preferences of the audiences, what are

the most appropriate means of reporting findings, e.g.,
detailed technical reports, summaries, press
conferences, study sessions, memos and letters, video
presentations, web assessable documentation, etc.?

Reporting language

Will reports need to be presented in different languages:

technical or non-technical, simple English or other
language(s) to meet the needs of different audiences?

Reporting format

Will reports be carefully formatted to enhance their

readability, e.g., use of white space?

Responsive design

What ongoing evaluation planning process and resource

plan will assure flexibility for adding to or otherwise
revising the evaluation questions and obtaining
unanticipated, pertinent information?

Delimited design

Is there a clear delimitation of the design, including the

Evaluation Plan for KLIP

Evaluation Manager: Dr. Oketch
TRIM Number: 00900

14

Technical Design
The evaluation team should convert a general evaluation plan to a detailed, yet flexible technical
plan
purpose of the evaluation and the questions that will be
answered?
Attention to trade offs

How will the evaluation address trade-offs between

comprehensiveness and selectivity at each stage of the
evaluation: planning; budgeting; and collecting,
organizing, analysing, interpreting, and reporting
information?

Technical adequacy

What are assurances that the findings will be reliable,

valid, and objective?

Moral/Ethical Imperatives
The evaluation team and clients/stakeholders should clarify and confirm the evaluations role in
ethically serving some socially valuable purpose
Philosophical stance

Will the evaluation be value based, value plural, or value

free?

Evaluation teams values

Will the evaluation teams technical standards and

values conflict with the client systems and/or sponsors
values; will the team face any conflict of interest
problems; what will be done about possible conflicts?

Judgments

Will the evaluation team judge the program; or obtain,

analyse, and report the judgments of various reference
groups?

Objectivity

How will the evaluation team avoid being co-opted and

maintain their objectivity?

Equity

How will the evaluation team make sure to address and

honour the needs and rights of all stakeholders
equitably, taking appropriate account of their gender,
ethnicity, and language backgrounds?

Cost effectiveness

Compared to its potential payoff, will the evaluation be

carried out at a reasonable cost?

NB the above listed prices and costs are quoted in their net values.

Evaluation Plan for KLIP

Evaluation Manager: Dr. Oketch
TRIM Number: 00900