Inception Report For The Gfr7 Ete Consultancy - 2 - May - 2014 - 2

CARE International in Kenya
End Term Evaluation for the Global Fund Round 7 HIV Program
(Non
(Non-Government
Component)
Inception Report (May
(
2014)
Prepared by
Table of Contents
Acronyms ......................................................................................................................................... 3
1.0
Introduction and Background ................................................................................................... 5
1.1
Background .......................................................................................................................... 5
1.2
About Global Fund ............................................................................................................... 6
1.3
Global Fund Round 7 Program ........................................................................................... 10
1.4
Our Understanding of the End Term Evaluation ................................................................. 11
2.0
Proposed Methodology of the Evaluation ............................................................................... 17
2.1
Design................................................................................................................................ 17
2.2
Study Population ................................................................................................................ 17
2.3
Area ................................................................................................................................... 17
2.4
Sampling ............................................................................................................................ 18
2.5
Data Collection Methods/Tools .......................................................................................... 25
2.6
Data Collection tools .......................................................................................................... 33
3.0
Data Management .................................................................................................................. 34
3.1
Data Quality Assurance ...................................................................................................... 34
3.2
Ethical Considerations ........................................................................................................ 34
3.3
Data Entry and Analysis ..................................................................................................... 35
3.4
Report Writing ................................................................................................................... 43
4.0
Operational Work Plan ........................................................................................................... 44
5.0
Assignment Technical and Managerial Structure .................................................................... 45
5.1
Team Composition ............................................................................................................. 45
5.2
Roles and Responsibilities .................................................................................................. 45
Appendix 1: Final Report Outline .................................................................................................... 49

Appendix 2: Documents to be reviewed........................................................................................... 50
Appendix 3: Summary of Questions, Methodology and Data Sources for the Project Interventions
Evaluation Component .................................................................................................................... 59
Appendix 4: Summary of Questions, Methodology and Data Sources for the Organizational System
Assessment component ................................................................................................................... 64
List of Tables
Table 1: Global Fund Grants Committed to Kenya as per April 2014 ................................................ 8
Table 2: Sampled SRs for OSA and OCA ....................................................................................... 22
Table 3: List of Key informants for evaluation of program interventions.......................................... 23
Table 4: List of Institutional Capacity Evaluation Assessment Key Informants ................................ 24
Table 5: Organizational Systems Assessment Key Informants ........................................................ 24
Table 6: Summary of number of FGDs and IDIs ............................................................................. 25
Table 7: FGDs with HTC counselors/supervisors ............................................................................ 26
Table 8: FGDs with ART beneficiaries ........................................................................................... 26
Table 9: FGDs with BCC beneficiaries and peer educators .............................................................. 27
Table 10: CSO/NGO capacity ratings ............................................................................................. 30
Table 11: Main competency areas for the organizational capacity assessment .................................. 30
Table 12: Six Competence Areas ..................................................................................................... 32
Table 13: Proposed evaluation tools ............................................................................................... 33
Table 14: Criteria for Rating Risk Levels for SRs ........................................................................... 36
Table 15: OT.1 Cost per person who received testing and counselling services, by mode of provision
who received their results (US$ per person)..................................................................................... 38
Table 16: OT.2Cost per person currently receiving antiretroviral treatment, 1st line regimen (US$ per
person) ............................................................................................................................................ 38
Table 17: OT.3 Cost per person receiving antiretroviral treatment, 2nd line regimen (US$ per person)
....................................................................................................................................................... 38
Table 18: OT.4 Cost per person receiving Community Mobilization (US$ per person)..................... 39
Table 19: OT.5 Cost per person receiving Youth Friendly Activities (US$ per person) .................... 39
Table 20: Data Analysis Framework ................................................................................................ 41
Table 21: Organizational Gender and SRH Evaluation Framework ................................................. 42
Table 22: Programmatic Gender and SRH Evaluation...................................................................... 43
Table 23: Documents to be reviewed ............................................................................................... 50
Acronyms
AIDS
ART
CASCO
CBO
CHW
CIK
CSO
CU
FBO
FGD
FSW
GF
Acquired Immunodeficiency Syndrome

Antiretroviral Therapy
County AIDS and STI Coordinator
Community Based Organizations
Community Health Workers
Civil Society Organization
Community Unit
Faith Based Organizations
Focus group discussion
Female Sex Workers
Global Fund
GFATM
GFR7
HCBC
HCT
HIV
HTC
ICC
IDU
KAIS
KDHS
KII
KMOT
LDT
MARPS
MSM
NACC
NASCOP
PE
PLHIV
PMTCT
PR
PWD
RA
SDA
SR
SW
TA
TB
TOR
UNODC
Global Fund for AIDS, Tuberculosis and Malaria

Global Fund Round 7
Home Community-Based Care
HIV testing and counseling
Human Immunodeficiency Virus
HIV Testing and Counseling
Interagency Coordinating Council
Injecting Drug Users
Kenya AIDS Indicator Survey
Kenya Demographic and Health Survey
Key informant interview
Kenya Modes of Transmission
Long Distance Truckers
Most at Risk Populations
Men having Sex with Men
National AIDS Coordinating Council
National AIDS and STI Control Programme
Peer Educators
Persons Living with HIV
Prevention of Mother to Child Transmission
Principal Recipient
Persons with Disability
Research Assistant
Service Delivery Area
Sub Recipient
Sex Worker
Technical Assistance
Tuberculosis
Terms of reference
United Nations Office on Drugs and Crime
1.0 Introduction and Background

Bon Sant Consulting understands that the End Term Evaluation (ETE) is part of the Monitoring and
Evaluation Plan whose objective is to examine accomplishments over the period of implementation of the
Global Fund Round 7 HIV Programme by CARE Kenya. The evaluation seeks to assess the outcome and
impact of the program to inform ongoing and future implementation of Global Fund grants and other HIV
prorammes.
The inception report therefore presents the Consultants interpretation and understanding of the TOR,
ETE methodological approach and work plan. To this end, the Consultant wishes to state his commitment
to the ETE process and a participatory approach with the main aim of facilitating organizational learning
and stakeholder ownership of the process and application of recommendations in the context of the next
round of GFTAM.
1.1
Background
The Global Fund for AIDS, Tuberculosis and Malaria (GFATM) has over the years channeled a
significant amount of resources to support the most effective prevention and treatment programmes of the
Kenyas HIV/AIDS response.
Kenyas fight against AIDS has generated historic achievements and averted millions of new HIV
infections and AIDS-related deaths through scale-up of HIV treatment, prevention and care. However, the
current adult HIV prevalence rate of 6.2% remains unacceptably high. An estimated 1.6 million Kenyans
are living with HIV, and 111,000 new adult and child infections occur every year (NACC, 2013). As a
result, 29% of adult mortality, 24% of all morbidity, 20% of maternal mortality and 15% of under-5
mortality is HIV-related. This is adversely affecting and consequently threatening gains in life expectancy
and other development and health outcomes achieved over the past decade.
Despite Kenyas domestic spending on HIV/AIDS rising by more than 100 percent between 2006 and
2011, over 75% of the response is finance from international sources including GFTAM. Globally, the
economic architecture is rapidly changing. Many high-income countries are experiencing structural
budgetary constraints that call into question their commitment to international assistance in coming years.
In particular, national populations in most industrialized countries are steadily aging. The proportion of
national populations that will require social security benefits will continue to grow, but the base of
younger workers who will have to pay the bill will likely be stable or decline in many countries. These
pressures have been exacerbated by recent actions by high-income countries to respond to banking and
currency crises, driving national debt to worrisome levels. Partly due to these, the level of international
resources available for HIV and other development issues has flatlined and declined in recent years. Some
analysts have attributed these trends to donor fatigue for HIV programmes (Grepin, 2012).
In light of these global realities and as Kenya continues its progress toward attaining middle-income
status, there is an early indication that the amount of development assistance for HIV will decline, calling
for more prudent use of resources targeted at HIV/AIDS.
Policy provisions, health infrastructure and health seeking behavior demonstrate myriad and diverse
inequalities between men and women. Health promotion, disease prevention, disease treatment,
rehabilitation and support and palliation have different and yet critical implications for womens and
mens health. In most of these dimensions of health provision, the lower status and social value of women
and girls in the household has contributed to their exclusion from appropriate and adequate health
support. Gender inequity is a major obstacle to reaching better family planning, maternal and reproductive
health outcomes, and to preventing and treating HIV/AIDS. It has long been recognized that unequal
power relations between men and women and societal norms of femininity and masculinity are important
influences on HIV epidemics.
Engaging with the gender dimensions of HIV and AIDS policies and programmes is premised on
interrogating and remedying the feminized face of the epidemic in Kenya as it is much of the case in
sub-Saharan Africa. Structural, behavioral and bio-medical factors separately and concertedly precipitate
the vulnerability of women and girls and men and boys to HIV acquisition and hamper mitigation of
degeneration into AIDS related conditions. The latest data on HIV prevalence in Kenya (KAIS, 2012)
indicated that women and girls still bear the disease burden. Accordingly, prevalence among females aged
15 to 64 years is 6.9% which is higher than the national average of 5.6% whereas prevalence among
males within the same age category is 4.4% which is appreciably below the national average (KAIS,
2012). Understanding how gender impacts the roles, rights, attitudes and behaviors of males and females
of all ages and classes, can guide interventions aimed at prevention, treatment and care as well as
advocacy. Risk factors and vulnerability are substantially different for men and for women, as is most
evident in the marked age- and sex-differentiated HIV prevalence rate which has implications for
strategies to reduce overall prevalence. Gender helps to identify how gender differentials and inequalities
make males and females vulnerable (UNIFEM, 2007). Tackling the AIDS pandemic is fundamentally
about behavior change, essentially effecting a transformation of gender roles and relations.
1.2
About Global Fund

6
The Global Fund for AIDS, Tuberculosis and Malaria (GFATM) is the worlds leading financing
mechanism that addresses Malaria, TB and HIV and AIDS in low resource countries. Kenya has for a
number of years been a beneficiary of funds from the Global Fund against these three diseases. Since
2003, Kenya has been awarded the following Global Fund grants:
Round 2 - HIV and AIDS, TB and Malaria;
Round 4 - Malaria;
Round 5 - TB;
Round 6 TB;
Round 7- HIV and AIDS; and
Round 9 TB
Round 10 HIV and AIDS

The Global Fund galvanizes support for the fight against AIDS, TB and Malaria, working with partners to
support the most effective prevention and treatment.
Since its inception in 2002, the Global Fund to Fight AIDS, Tuberculosis and Malaria has been a major
engine driving this remarkable progress. It is the main multilateral funder in global health channeling 82
percent of the international financing for TB, 50 percent for malaria, and 21 percent of the international
financing against AIDS. It also funds health systems strengthening, as inadequate health systems are one
of the main obstacles to scaling up interventions to secure better health outcomes for HIV, TB and
malaria (Global Fund, 2014).
The implementation of Round 2 and 4 grants are completed, while the other grants are at different stages
of implementation. The total active grants committed to Kenya amount to approximately US$ 373
million.
Table 1: Global Fund Grants Committed to Kenya as per April 2014
Grant start
and end date
Grant
Approved Grant (USD)

Phase 1
R2 HIV: Signed
01/04/200331/03/2005
106,786,807
R4 Malaria:
Signed 20/4/05
01/02/200631/01/2011
81,749,756
R5 TB: Signed
3/7/06
01/09/200631/08/2011
R6 TB
Signed 30/10/07
Phase 2
Total
Funds
Disburse
d to PR
by GF
Total
Disbursement
to SRs by the
PR
Balance
held by
the PR
GF Gant
Performa
nce
Rating1
106,786,807
68,006,881
162,173,085
102,535,157
102,443,395
91,761
B2
7,912,684
19,916,156
3,511,242
3,462,972
48,270
B1
01/04/200831/03/3013
4,206,357
9,160,878
2,961,806
2,951,332
10,474
B1
R7 HIV/AIDS
MOF- Signed
7/11/08
01/06/200931/05/2014
30,655,749
11,803,456
11,766,207
37,249
A2
R7 HIV/AIDS
CARE -signed
7/11/08
01/06/200931/05/2014
80,423,329
N/A
130,539,253
16,007,746
12,334,906
R9 TB: Signed
on....
Rd 10 HIV and
AIDS
4,735,494
A1
50,661,908
Rd 10 Malaria
TOTAL
124,524,546
80,423,329
372,451,280
120,811,661
120,623,906
187,755
Source: Global Fund Website
The Global Fund has so far disbursed about 40% of the funds approved for Kenya. Of these, the PR has
disbursed about 99% of the funds to sub-recipients. The average performance of Kenyas grants ranges
from A1, which means exceeding expectations (e.g. for Care International in Kenya), B1 (adequate) to B2
(inadequate), meaning that it is inadequate but potential demonstrated.
Implementation of previous Global Fund rounds in Kenya faced several challenges that have resulted in
low absorption of funds and, in some cases, low achievement of targets. The challenges include:
Inadequate monitoring characterized by delayed reporting by sub recipients to principal recipients
and poor quality reports which, in some cases, are inaccurate and incomplete data.
Bottlenecks in management of procurement which accounts for about 70% of the programme.
Poor oversight by the Country Coordinating Mechanism (CCM), which was tasked to ensuring
accountability for the Global Fund resources committed to Kenya and achievement of targets
agreed with the Global Fund.
The Global Fund rates grants from A to C. A=Exceeding expectations; A2=Meeting Expectations; B1+ Adequate;
B2=Inadequate but potential demonstrated and C= Unacceptable
Up to the start of 2008, Kenya had succeeded in only 30% of its grant applications to the Global Fund.
And as of the end of that year, nearly one-third of all assistance awarded had not been used by the time
the grants expired. But within two years, the situation had improved dramatically (UNAIDS, 2013).
A number of issues with the management of Global Fund grants. Among the key problems identified
were: The Country Coordinating Mechanism (CCM), a multi-sectoral national body thatin the Global
Fund architectureshould manage all Fund-related matters, was isolated and not integrated with any
national structure. Stakeholders had little involvement and the CCM exercised poor oversight of grant
implementation. There were few performance reviews, which meant there was no way to identify and
analyze bottlenecks and take informed decisions. A key structural change was the introduction of the
Kenya Coordination Mechanism (KCM), which houses a National Oversight Committee (NOC) and three
Interagency Coordinating Committees (ICC)one for each disease. The NOC is high-level and chaired
by the Permanent Secretary of the Ministry of Health. The ICCs do the bulk of proposal development
and they are also the first entry point for monitoring and reporting (UNAIDS, 2013).
The establishment of KCM seeks to address weaknesses in governance and implementation
of the Global Fund grants in the following ways (CCM, 2010):
KCM is lean and efficient body focused on oversight of grants.
KCM have strong oversight system, effective decision making processes and ability
to follow up on its actions systematically: ensure PRs reports are presented to the
KCM, reviewed systematically, bottlenecks identified and decisions made on how
to address each bottleneck.
Oversight
enhanced
by
use
of
dashboard tool for reporting to KCM.
Ability
to
hold
responsible
institutions
and
officers
accountable
for
poor performance of Global Fund grants. The institutions and individual
officers responsible for delivering results of the Global Fund grants will be tasked to pres
ent grant reports to the KCM and take responsibility for any shortcomings.
1.3
Global Fund Round 7 Program
The Kenya CCM implemented the principal recipient nomination process laid out in the Governance
Manual to identify the principal recipients for the Round 7 proposal. Key steps in the process include
setting of criteria for selection of the PR, establishing a PR Nomination Committee, Advertising the
request of application for PR from eligible organisations, evaluation of applications and final decision by
CCM appointment of the PR(s). The CCM appointed two principal recipients for this proposal Ministry
of Finance and Care International in Kenya. Care International in Kenya is the Principal Recipient for
Global Fund Round 7 HIV & AIDS Program Non- Government Component (KCCM, 2008). The
Governmental Principal Recipient was mostly to implement the treatment activities while CARE
International in Kenya, the Civil Society Principal Recipient, was to focus on HIV counseling and testing
and prevention activities through Behavior Change Communication (BCC).
CARE International is a humanitarian non -governmental organization (NGO) committed to enhancing
social justice and alleviating poverty. CARE International in Kenya (CIK) also known as CARE Kenya,
has been operational in Kenya since 1968. Since then, CIK has been working in close collaboration with
various stakeholders ranging from government, private sector and civil society organizations including
community based organizations (CBOs), faith based organizations (FBOs) and communities in the fight
against poverty. The priority programmatic interventions for CIK are in areas of health including HIV
and AIDS programming, livelihoods interventions, economic empowerment, refugee and emergency
response.
Global Fund Round 7 is now on its 5th and final year of implementation having started in 2009. The
program has been implemented through 54 sub-recipients (SRs) since its inception in April 2009 and
currently has a total of 29 active SRs. Kenyas country proposal to the Global Fund to fight Malaria, TB
and HIV/AIDS in round 7 for the first time, introduced a more transparent process of selection of sub
recipients to be incorporated in the Countrys round-7 proposal. This was aimed at not only at improving
the quality of submissions but also to enhance integrity of the process. It involved evaluation of
Expression of Interest by an Independent Review Panel (IRP) constituted under the Kenya Country
Coordination Mechanism (CCM) and comprised private consultants (Vakil, et al.).
The overall program was meant to improve the quality of life of People Living with HIV and AIDS and
reduction of HIV infections. The program particularly focused on sustaining ARV treatment and scale up
of prophylaxis for opportunistic infections. It also sought to expand prevention services to most-at-riskpopulations (MARPs) and underserved areas. The Care International in Kenya Program was particularly
focused on extending these services to communities that were not adequately covered at present,
especially rural communities through sub recipients (SRs). This meant that the SRs were meant to
complement the national efforts to decentralize access to ARV treatment and the treatment of
opportunistic infections. The target Group/ Beneficiaries included:
General population
People Living with HIV/AIDS (PLWHA)
Workers
MARPs including but not limited to, youth, Sex Workers and their clients, Men who have sex
with men (MSMs), long distance truckers and prisoners.
10
The strategies that were to be used to meet the objectives of the program included:
Promoting access to HIV Counseling and Testing
Sensitization of youth and MARPs
Increasing access to strategic information to target populations
Behavior Change Communication at community level and at the workplace
Promoting peer education and training
Strengthening institutional capacity of civil society implementers
Program Goal: Improved quality of life for people living with HIV (PLHIVs) in Kenya and reduced HIV
infections. In order to achieve this goal, the program seeks to achieve the following objectives:
To scale up and maintain PLHIVs on ART;
To increase access to HIV testing and counseling (HCT) services;
To increase uptake of HIV prevention and treatment services; and
Strengthening institutional capacity of HIV program implementers.
It is in this regard that CARE Kenya has commissioned the ETE to ascertain the performance of its
programme as a principle recipient of GFATM funding.
The End Term Evaluation (ETE) is part of the Monitoring and Evaluation Plan whose objective is to
examine accomplishments over the period of implementation of the Global Fund Round 7 HIV
Programme by CARE International in Kenya, a grant amounting to US$ 28,342,652 (Grant Number:
KEN-708-G10-H). The program was titled Improvement of the quality of life of People Living with HIV
and AIDS and reduction of HIV infections. The grant was implemented in two phases with Phase 1
starting on 1st April 2009 and ending on 31st March 2011, while Phase 2 started on 1st April 2011 and
ended on 31st March 2014. The grant funds were committed by the Global Fund to Care International in
Kenya (Principal Recipient) in staggered terms involving a first commitment of US$ 24,782,002 and a
second commitment of US$ 3,560,650. The Global Fund stipulated that at the time of signing this
agreement, it shall set aside ("commit") funds up to the amount of the first commitment. A second
Commitment of Grant funds were committed under this Agreement not earlier than 18 months after the
Phase 1 ending date.
The ETE is also expected to facilitate a review of the programme impact in terms of the set goals and
indicators that may lead to evidence based adjustments to the implementation of subsequent GFTAM
HIV programmes under GFTAMs new funding model. The ETE will also inform the implementation of
other HIV programmes undertaken by CARE International in Kenya and other stakeholders. The
inception report therefore presents the Consultants interpretation and understanding of the TOR, ETE
Grants and Finance components methodological approach and work plan. To this end, the Consultant
wishes to state his commitment to the ETE process and a participatory approach with the main aim of
facilitating organizational learning and stakeholder ownership of the process and application of
recommendations in the context of the next round of GFTAM.
1.4
Our Understanding of the End Term Evaluation
11
We understand that the purpose of this assignment is to evaluate the Global Fund Round 7 (GFR7) nongovernment/CSO component which seeks to improve the quality of life for people living with HIV and
reduce HIV infections: majorly through increasing access to HIV testing and counseling services and
increasing uptake of HIV/AIDS prevention and treatment services. The objective of the evaluation is to
analytically examine the overall project to:
i.
Determine the extent to which GFR7 increased knowledge on ART among PLHIV;
ii.
Determine the extent to which GFR7 increased retention on ART among PLHIV;
iii.
Determine the extent to which the program increased access to HIV testing and counseling
services;
iv.
Determine the extent to which the program increased uptake of HIV prevention services in the
target population;
v.
Determine the extent to which the program increased uptake of HIV care and treatment services
in the target population; and
vi.
Establish the extent to which capacity building initiatives strengthened the capacity of partners in
providing HIV services.
As part of gaining better understanding of the task, the following actions have been undertaken:
Interview meeting with CARE Kenya and Bon Sant Consulting at CARE Offices on 25th March
2014;
Meeting to discuss understanding of the task and discussion of the work plan on 27thMarch 2014.
From the above inception meetings, background readings and communications, the Consultant has noted
the following:
The urgency of the ETE process;
The very high expectations of the ETE process and its outcomes;
The forming dimensions of the ETE process including team formation and resource mobilization;
and
The need to have a clear and shared understanding of ETE approach and methodology.
1.4.1
End-term Evaluation Components
We also understand that the review will have four major components, namely: Project interventions
evaluation, Organizational Systems Assessment, Institutional Capacity evaluation and Gender and SRH
integration review.
1.4.1.1
Project Interventions Evaluation
This will involve the evaluation on the impact on the project on the beneficiaries lives. The purpose will
be to determine the extent to which GFR7 increased: knowledge on ART among PLHIV retention on
ART among PLHIV, access to HIV testing and counseling services, uptake of HIV prevention services,
uptake of HIV care and treatment services, knowledge on general HIV and AIDS prevention, care and
treatment in the target population. The component will incorporate various sources of data. Key
informants at national, county, district and SR level will provide an overview of the project
implementation and its impact. Beneficiaries through focus group discussions and in-depth interviews
will give account on what services they have received from the project and how they have benefited
including the changes in their lives as a result. A knowledge, attitudes, practices and behavior will be
12
done and findings compared to national surveys to determine the impact of the project. Desk review will
provide data on, but not limited to, the project coverage, services offered (access, uptake and retention),
effectiveness of the strategies employed, achievements (project and SRs), intended and unintended
results, lessons learnt and best practices. The specific objectives under this component will be:
i.
Determine the extent to which GFR7 increased knowledge on ART among PLHIV.
ii.
Determine the extent to which GFR7 increased retention on ART among PLHIV.
iii.
Determine the extent to which the program increased access to HIV testing and counseling
services.
iv.
Determine the extent to which the program increased uptake of HIV prevention services in the
target population.
v.
Determine the extent to which the program increased uptake of HIV care and treatment services
in the target population.
1.4.1.2
Organizational Systems Assessment
This will involve holistic review of the program systems including grants management system, finance
system that interplayed with the program (backed by CARE finance system and procedure manual),
auditing systems to manage risks and how well compliance was assured and procurement system
including tax systems which informed programmatic achievements. The aim of the Organisational
Systems Assessment is to establish the extent to which the organization system has contributed to the
achievement of the programme objectives. This includes: ascertain if the Grants management system
including the sub- grants management system operated optimally and supported the achievement of the
desired results; determine whether the Finance system that interplayed with the program led to the
achievements reported results; evaluate if the audit systems put in place managed risks and whether there
was compliance by both the PR and the SR and ascertain whether the procurement system including tax
systems led to the programmatic achievement. The specific objectives of this ETE component include:
i.
ii.
iii.
iv.
v.
vi.
vii.
viii.
ix.
x.
Assess the effectiveness of contracting and budgeting procedures between CARE Kenya (PR) and
SRs.
Determine the efficiency and effectiveness of funds disbursements and liquidations involving
GFATM, PR and SRs.
Evaluate the efficiency and effectiveness of operations and systems including financial reporting.
Analyze the utilization of funds to ascertain the drivers and challenges of absorption together
with the actual and normative budget tracking procedures and processes.
Analyze the effectiveness of the financial risk management and audit procedures.
Appraise the effectiveness of technical support/ capacity building to the performance of SRs.
Estimate the unit costs and value for money of interventions undertaken by SRs
Estimate the efficiency of service delivery by SRs.
Evaluate the efficiency of programme close-out by the PR in terms of actual and normative
procedures and practices.
Identify emerging issues and new priorities informed by new evidence generated in GFATM
Round 7 implementation.
1.4.1.3
Institutional Capacity Evaluation
The overall aim of the institutional capacity evaluation is to assess whether project objective 4
(Strengthening Institutional Capacity}objective minimised risk in the resources provided in
13
the overall grant management and to provide a basis for sustainability beyond the grant.
Objective 4 had two Service Delivery Areas namely:
SDA 4.1: Information systems and Research
SDA 4.2 : Strengthening Civil Society and Institutional Capacity Building
The institutional capacity evaluation will focus on the implementation of SDA 4.2 Strengthening
Civil Society and Institution Capacity Building.
The specific objectives of the assessment will be::
How adequate was the technical assistance?,
To what extent the uptake was and how it affected the performance?
What were the changes/improvements by the SR as a result of Technical Assistance
(TA)?
How the SRs have sustained and replicated the technical assistance to other areas of
the organization?
1.4.1.4
Gender and SRH Integration Review
This component of the ETE will focus on gender integration into the institutional and programmatic
dimensions of the GF R7 programme. The evaluation shall be predicated on the assumption that gender
concerns and appropriate responsive interventions informed the design and implementation of the
programme at the Principal Recipient and sub-Recipient levels as anticipated in policy documents such as
the KNASP III (200/10-2012/13), KAIS 2007, Kenya HIV Prevention Response and Modes of
Transmission Analysis (2009) and KDHS (2008-09). In keeping with the broad objectives of the GF R7
programme, this component will seek to establish the design, implementation and effectiveness of gender
specific measures to scale up and maintain PLHIVs on ART, increase access to HIV testing and
counseling (HCT) services and increase uptake of HIV prevention and treatment services. Key questions
will assess strategies employed by the programme to promote equitable participation in project activities
and access to information and services by both male and female community members, evidence of
reduced gender discrimination and enhanced equality as a result of focusing on norms, stereotypes and
practices that promote exclusion and marginalization based on gendered identities. Infusion of sexual and
reproductive health (SRH) content into relevant programme areas will be assessed as a parameter that
vindicates strengthened gender responsiveness. Evidence of positive transformation in health seeking
behaviour by male and female programme beneficiaries will be evaluated to show the success or
otherwise of the gender integration initiatives. Guideline on programme content and monitoring and
evaluation tools will be assessed to ascertain gender responsiveness. Empirical data to validate or negate
some of the questions will be sourced from programme reports that indicate numbers of male and female
beneficiaries reached, dichotomized along age, disability and service delivery categories. Recognizing the
linkage between institutional and programmatic imperatives for gender integration, the gender component
of the ETE will also explore organizational capacity to address gender equality. This will be in
accordance with the GF R7 objective on strengthening institutional capacity of HIV program
implementers which will focus on institutional policies, staff capacity building initiatives, infrastructure
and resources to support gender integration. Gender integration questions will be embedded in relevant
tools under project interventions and institutional capacity strengthening to eliminate the separation of
gender as an autonomous ETE dimension. This means the populations targeted under the two areas (male
and female programme beneficiaries, SR management and staff, PR management and staff and oversight
14
bodies) and methodologies used to generate information (FGDs, KII, structured quantitative data
questionnaire, checklists, observation guides, etc.) will apply to the gender component with requisite
customization. The desk review for gender integration will however engage with specific documents that
show how the programme planned to integrate gender, implementation strategies, successes (for men,
women, boys and girls in access and utilization of services and changes in health seeking behaviour),
challenges and lessons learnt, and best practices. The overarching evaluation questions applicable to the
gender component are listed below:
What were the results of gender integration in the program?
To what extent did the program contribute in addressing gender barriers in HIV programming?
To what extent was gender integrated in the different service delivery areas?
To what extent did the strategies impact positively or negatively in the lives of men, women, boys
and girls?
Appendix 5 outlines how gender assessment will be incorporated into the two components (Program
intervention evaluation and Organizational capacity assessment)
15
1.4.2
Duration of the Assignment
The Consultant understands that requirement of consistent commitment of the consultant and all other
technical assistance (TA) members from the start of the assignment on 2nd April 2014 to the conclusion of
the assignment on 24th July 2014. We also understand that the assignment must be concluded within this
set timelines.
1.4.3
Critical Assumptions for Effective Delivery
To effectively deliver on the above responsibilities, the Consultant assumes that the following conditions
will be present throughout the process:
The SRs, KCM and all other stakeholders will be willing to support and participate in the
consultancy programme according to the agreed schedule.
1.4.4
Deliverables
The Consultant will be responsible for and/ or contribute to the delivery of the following documents:
Inception Report and Draft Tools
Revised tools
Agenda for training of the RAs and the training report
Pre-test report, modified tools and findings from the pre-test
Complete raw data sets (both qualitative and quantitative)
Field report (after completion of data collection)
Preliminary findings
Draft Report
Final Report
Advocacy Report
Power Point Slides
Policy Briefs
County briefs
16
2.0 Proposed Methodology of the Evaluation

2.1
Design
Multiple data collection methods will be employed to collect primary and secondary data and a variety of
quantitative and qualitative information. Various analytical methods will be used to enhance triangulation
of data and increase overall data quality, validity, credibility and robustness and reduce bias. Qualitative
data sources will be given special attention as they are very helpful in identifying the internal and external
factors that affect program achievements. The evaluation is essentially a cross sectional survey.
In order to effectively achieve the objectives and deliverables of the study, an integrated analytical
approach and methodology based on consultation, collaboration and participation of all relevant
stakeholders and key strategic partners will be the focus of involvements throughout the exercise. This
assignment will adopt extensive liaison, consultation and participation of all stakeholders. Therefore our
proposed methodology will provide a framework for:
i.
A significant element of quantitative analysis, evidence based findings and emphasis on making
clear recommendation.
ii.
Adequate participation of key stakeholders within GFATM, CARE Kenya, and selected SRs.
iii.
A clear definition /specification of different SRs to be presented in the analysis.
iv.
Field visit to selected study sites.
v.
Hold an evaluation validation with different stakeholders
vi.
Strong collaboration between the PR and the study team.
2.2
Study Population
The evaluation primary focus will the project implementation. In order to achieve this the following will
be the respondents of the evaluation:
a) Project beneficiaries,
b) Peer educators,
c) Health care workers,
d) SR project staff,
e) PR project staff, and
f) Stakeholders at national, county and district level.
2.3
Area
The evaluation will be undertaken in all regions of Kenya.
17
2.4
Sampling
2.4.1
Sampling of sub recipients
Sampling of sub recipients was done in two phases

1. Sampling of SRs to participate in Project Interventions Evaluation, and
2. Sampling of SRs to participate in Organizational Systems Assessment and Institutional
Capacity Evaluation
Sampling of SRs to participate in project interventions evaluation
A stratified random sampling technique was used to determine the sites and Sub Recipients to be visited.
Sub recipients sites were stratified based on 8 clusters depending on their region of operation and the
main service delivery areas: behavior change communication (BCC), HIV testing and counselling (HTC)
and anti-retroviral therapy (ART). In each region all the SRs undertaking interventions in each of the
SDAs were listed to create a sampling frame. Simple random sampling technique was then used. Half
(50%) of SRs in each region (with a minimum of one) in each service delivery area were drawn randomly
using a lottery system. Within each SDA all the SRs within a specific region had pre-assigned numbers.
Random numbers were drawn until the number of SRs sites sample was attained.
Nairobi Region
SDA
BAR HOSTESS
FHK
FHOK
HWWK
KANCO
KSMH
KUB
NCCK
NOPE
OAHIP
SUPKEM
UDEK
YWCAA
ZINDUKA
BCC
1
1
1
HTC
ART
1
1
1
1
1
1
1
1
1
1
1
1
12
1
1
1
4
Total
1
1
2
1
1
1
1
2
1
1
2
2
2
1
19
Central Region
SDA
ACTION AID
BAR HOSTESS
HWWK
KENWA
KSMH
NCCK
UDEK
BCC
1
1
1
HTC
ART
1
1
1
1
1
5
1
1
3
Total
1
1
2
2
1
2
2
11
18
Rift Valley Region

SDA
CHAT
KANCO
BOH
NCCK
FAVO
FHOK
NIA
Hope Valley Family Institute
ACTION AID
FOVUP
KUB
OAHIP
WORLD RELIEF
BCC
1
1
HTC
1
1
1
1
1
1
1
1
1
1
1
1
11
ART
1
1
1
1
1
1
Total
3
2
1
2
2
2
3
1
1
1
1
1
1
21
North Eastern Province

SDA
PEACENET
WASDA
BCC
1
HTC
1
1
ART
1
1
2
Total
2
2
4
BCC
HTC
ART
Total
1
1
1
1
1
1
1
1
1
1
1
1
3
3
2
1
1
1
1
1
13
Eastern Region
SDA
FHK
HWWK
ZINDUKA
KANCO
KICABA
BAR HOSTESS
OAHIP
NCCK
1
3
Coast Region
SDA
HWWK
KANCO
ZINDUKA
MDM
SUPKEM
KUB
OAHIP
NCCK
KWETU
BCC
1
1
1
1
1
1
1
HTC
1
ART
1
1
1
1
1
1
4
1
4
Total
2
2
2
2
2
1
1
2
1
15
19
Western Region
SDA
BCC
SUPKEM
FASI
FOVUP
KUB
WORLD RELIEF
KANCO
FAVO
HTC
ART
Total
2
1
1
1
1
1
1
8
1
1
1
1
1
4
1
2
HTC
ART
Nyanza Region
SDA
BCC
1
FHOK
WOFAK
KUB
MERLIN
NOPE
YWCAA
NYARAMI
MATATA
Merlin
MSK
GLOBCOM
NCCK
1
1
1
1
1
1
1
1
1
1
1
4
Total
1
1
1
1
1
1
2
1
1
1
1
1
13
Summary of Sampled SRs and Sites

SDA
Nairobi
Central
Rift Valley
North Eastern
Eastern
Coast
Western
Nyanza
BCC
12
5
11
1
7
7
4
5
52
HTC
4
3
5
1
3
4
2
4
26
ART
3
3
5
2
3
4
2
4
26
Total
19
11
21
4
13
15
8
13
104
20
The following is a list of all SRs sampled:

1
2
3
4
5
6
7
8
9
10
11
12
ACTION AID
BAR HOSTESS
BOH
CHAT
FASI
FAVO
FHK
FHOK
FOVUP
GLOBCOM
HVFI
HWWK
13
14
15
16
17
18
19
20
21
22
23
24
KANCO
KENWA
KICABA
KSMH
KUB
KWETU
MATATA
MDM
MERLIN
MSK
NCCK
NIA
25
26
27
28
29
30
31
32
33
34
35
NOPE
NYARAMI
OAHIP
PEACENET
SUPKEM
UDEK
WASDA
WOFAK
WORLD RELIEF
YWCAA
ZINDUKA
Sampling of SRs to participate in Organizational Systems Assessment and Institutional Capacity

Evaluation
Organizational systems assessment and institutional capacity evaluation will be done for all SRs who
participated in the project from phase one. Due to time and logistical constraints 30% of the SRs will be
visited for extensive data collection while the rest will be evaluated via desk review. All the SRs who
participated in the project from were eligible to be visited during the exercise. Stratified random
sampling was adopted to establish those SRs to be visited. SRs were stratified according to the baseline
capacity assessment results (Yellow, blue and green) to form three sampling frames. In each sampling
frame systematic random sampling was employed. In each stratum a sample size (N) was 30% of each
category. An interval (n) was calculated by dividing the total number of SRs in each stratum by the
desired sample size in each. The SRs in each stratum were first ordered alphabetically. A random start
calculator first picked the start SR and the using the interval each nth SR was picked until the desired
number of SRs (N) was achieved. The table below gives an overview of the sampling frames intervals
and the sampled SRs (highlighted in yellow).
21
Table 2: Sampled SRs for OSA and OCA
Total Number
Green rated
Blue rated
Yellow rated
Catholic Diocese of Homa-bay

Centre for Research and Development
Federation of Kenya Employers (FKE)
Goal Ireland*
Kwetu training centre
Action Aid Kenya

Bar Hostess Empowerment & Support Programme
Beacon of Hope
CHAK
Communities Health Africa Trust
Marie Stoppes
Matata Hospital
Family Health Options Kenya (FHOK)

Family Support Institute (FASI)
Merlin
National Council of Churches of Kenya (NCCK)
National Organization of Peer Educators (NOPE)
Peace and Development Network Trust
Self Help Development Africa
Supreme Council of Kenya Muslims (SUPKEM)
Wajir South Development Agency (WASDA)
World Relief
Zinduka Afrika
Food for the Hungry International

Foundation of People Living with HIV/AIDS (FOPHAK)
Friends of AIDS Victims and Orphans
Hope Valley Family Institute
Hope Worldwide Kenya
Kenya AIDS Intervention Prevention Project Group
Kenya AIDS NGOs Consortium (KANCO)
Kenya Network of Women with AIDS (KENWA)
Kenya society for the mentally handicapped
Kenya Union of the Blind
Life Care and Support Centre (LICASU)
Maji na Ufanisi
Musokoto Joint Farmers Self Help Group
Neighbors in Action - Kenya
Nyarami VCT
OAHIP
Purple Images Productions
Sanaa Art Promotions
United Disability Empowerment in Kenya (UDEK)
WOFAK
Women Development Response Agency (WODERA)
Young Professional for Development
Young Women Campaign Against AIDS (YWCAA)
Abha Light Foundation

Global Communications Institute
Health and Water Foundation
KECOFATUMA
Kenya AIDS watch Institute (KAWI)
Kitui Business Integrated Creative Art and
Appraisal (KICABA)
Mothers Delight Moments
National Organization of Volunteers and Charity
Workers in Kenya (NOVOK)
16
30
Desired Sample size
Interval
Random start
22
2.4.2
Sampling of Individual Interviews (IDIs) with beneficiaries
The study will use stratified sampling method, Fisher et al formulae for determining the sample
size will be utilized for the quantitative data:
2
n=Z pqD
2
d
Where, n=desired sample size if target population is greater than 10,000

Z=Standard normal deviate at the required confidence level
p=Proportion in the target population estimated to having the characteristics being
measured.
q=1-p
D=Design effect-usually 1 where there are no replications
d=the level of statistical significance set.
Using the formula, Where p = 0.5, q = 0.5, Z = 1.96, d = 0.05, D =1, the minimum sample size
calculated = is 384. To take into account the anticipated non response rate and substandard filling of
questionnaires a 13.8% (KDHS 2009) of beneficiaries will be added to participate. A sample of 438
individuals will be recruited to participate. All the sampled SRs will be allocated a number of individual
interviews proportionate to their total beneficiaries reached in the implementation of the project. The
beneficiaries will be recruited from the database provided by the PR to eliminate bias in selection by the
SRs.
2.4.3
Sampling of Key Informants
In the assessment of programmatic interventions, the following key informants will be interviewed:
Table 3: List of Key informants for evaluation of program interventions
Designation
NOC Chair/ co-chair
KCM Secretariat Coordinator
CSO representative in KCM
KCM representative of PLHIV
HIV ICC chair
GF focal person at NACC
M&E Head at NACC
NASCOP GF Coordinator
NASCOP MARPS Coordinator
NASCOP HTC Coordinator
Secretary of HTC TWG sub-committee on disability
LFA Team lead
LFA Manager
LFA M&E focal person
Program managers/officers Program
Health program Director
Health Program Director
Senior MANAGER-Program Quality
Grants Manager (both at GF and Sector)
Organization
NOC
KCM
KCM
KCM
HIV ICC
NACC
NACC
NASCOP
NASCOP
NASCOP
PWC
PWC
PWC
Sub recipients
Care
Care
Care
GF/Health Sector
Number
1
1
1
1
1
1
1
1
1
1
1
1
1
1
35
1
1
1
2
23
M&E Coordinator (GF and Health Sector)

Senior Program Manager
Program Manager
M&E Coordinator
GF Country team
DASCOs
GF/Health Sector
Care
Care
Care
Fund Portfolio Manager
Ministry of Health
2
1
1
1
1
8
Total
Sampling of key informants will be purposeful with representation from each
Table 4: List of Institutional Capacity Evaluation Assessment Key Informants
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
Respondent Designation
Organization
Chief Executive Officer or Director

Board Members
Programme Manager
Programme Officers
Finance Manager/Officers/Assistants
Project Accountant/assistants
HR/Administration manager
IT and Operations Manager
IT and Operations Officers
M&E Managers
M&E Officers
Communications Officer/Manager
Total Number
PR/SRs
PR/SRs
PR/SRs
PR/SRs
PR/SRs
PR/SRs
PR/SRs
PR/SRs
PR/SRs
PR/SR
PR/SRs
PR/SR
Minimum number
per institution
1
3
1
1
1
1
2
1
2
1
3
2
19
Number
17
51
17
17
17
17
34
17
34
17
51
34
323
The table below shows the key informants for the organizational systems assessment. This will involve
interviews and discussions with a cross-section of key persons on disbursement and management of
grants in CARE Kenya and SR levels. The interviewees will include:
Table 5: Organizational Systems Assessment Key Informants
Designation
Finance and Grant Director
Procurement Manager
Manager or Overall In-Charge
Finance and Grant Manager
Grant Manager/ Accountant
Records officer
Total
Organization
SR
SRs
SR
SR
Number
1
1
16
16
24
2.5
2.5.1
Data Collection Methods/Tools

Secondary Data Collection/Desk Review
This will involve a thorough review of the relevant studies, research, survey reports and documents
including budget documents, records, and status reports among others. Appendix 2 gives a list of all
documents to be reviewed.
2.5.2
Primary Data Collection Methods
2.5.2.1
Project Interventions Evaluation Methods
Focus group discussions (FGDs) and In-depth Interviews (IDIs) with beneficiaries:
FGDs will be held with various groups of beneficiaries to establish the services they got and the impact it
has had on their lives FGDs will be held with beneficiaries from all SDAs and each SDA will have it
specific tool. In-depth interviews will be held with injecting drug users and long distance drivers. This is
necessitated by the following:
It may not be possible to hold FGDs will IDUs if some of them have at the time of interviews
overly influenced by drugs and due to its illegality may not be easily grouped.
Long distance drivers are highly mobile and at work mostly during the day when the interviews
will be held.
The summary of FGDs for the various SDAs is shown in the table below.
Table 6: Summary of number of FGDs and IDIs
HTC
Code
Number FGDs/IDIs
Number of SRs
FGD with counselors and counselor supervisors
HTC
FGDs with beneficiaries
ART
16
FGDs with youth beneficiaries of outreach events

FGDs with youth beneficiaries of sports events
FGD with Peer educators youth peer education
FGD With beneficiaries youth groups activities
FGD with youth beneficiaries resource centers
FGD with beneficiaries school health clubs
FGD with beneficiaries formal sector
FGD with beneficiaries Informal sector
FGDs with MARPS
Sex workers
MSM
Prisoners
IDIs with IDUs
IDIs with LDTs
FGD PWDs beneficiaries
BCC 1
BCC2
BCC3
BCC4
BCC5
BCC6
BCC7
BCC8
3
3
3
3
3
3
6
3
5
4
3
3
BCC9
BCC10
BCC11
BCC12
BCC13
BCC14
3
3
3
6
6
3
3
3
1
2
1
3
36
12
60
2
ART
BCC
Total FGDs
Total IDIs
25
The tables below show the allocation of the FGDs in each SR (refer to table above for FGD codes)
Table 7: FGDs with HTC counselors/supervisors
HTC
Sub Recipient
Region
SUPKEM
Nairobi
KENWA
Central
CHAT
Rift Valley
WASDA
North Eastern
HWWK
Eastern
KWETU
Coast
SUPKEM
Western
NYARAMI
Nyanza
Number of FGDs
1
1
1
1
1
1
1
1
Total number of FGDs
Table 8: FGDs with ART beneficiaries
Region: Nairobi
Sub Recipient
SDA
UDEK
ART FGD with beneficiaries
YWCAA
Region: Central
Sub Recipient
SDA
KENWA
NCCK
Region: Rift Valley

Sub Recipient
SDA
CHAT
FAVO
Region: North Eastern

Sub Recipient
SDA
PEACENET
WASDA
Number of FGDs
1
1
Number of FGDs
1
1
Number of FGDs
1
1
Number of FGDs
1
1
Region: Eastern
26
Sub Recipient
SDA
FHK
HWWK
Region: Coast
Sub Recipient
SDA
KANCO
MDM
Region: Western
Sub Recipient
SDA
NCCK
SUPKEM
Region: Nyanza
Sub Recipient
SDA
GLOBCOM
WOFAK
Number of FGDs
1
1
Number of FGDs
1
1
Number of FGDs
1
1
Number of FGDs
1
1
Total FGDs
16
Table 9: FGDs with BCC beneficiaries and peer educators
Region: Nairobi
Sub Recipient
KANCO
NOPE
SDA
Number of FGDs
1
1
1
1
1
1
BCC10
BCC7
BCC8
FHK
SUPKEM
KSMH
Region: Central
Sub Recipient
ACTION AID
BAR HOSTESS
HWWK
Region: Rift Valley

Sub Recipient
BCC6
BCC2
BCC14 (IDI-Mentally impaired)
SDA
Number of FGDs
1
1
1
BCC2
BCC3
BCC9
SDA
Number of FGDs
27
FOVUP
KUB
BCC1
WORLD RELIEF
BCC5
1
1
1
1
1
BCC5
BCC6
FHOK
BCC4
Region: North Eastern

Sub Recipient
SDA
PEACENET
WASDA
Number of FGDs
1
1
BCC3
BCC1
Region: Eastern
Sub Recipient
SDA
BAR HOSTESS
BCC9
HWWK
BCC10
Number of FGDs
1
1
1
1
1
1
BCC13
FHK
ZINDUKA
KICABA
BCC4
BCC1
BCC4
Region: Coast
Sub Recipient
SDA
HWWK
BCC10
Number of FGDs
1
1
1
1
1
1
1
BCC12
MDM
BCC9
BCC12
ZINDUKA
BCC1
BCC6
OAHIP
BCC14 (Hearing impaired outreach)
Region: Western
Sub Recipient
SDA
FASI
BCC1
Number of FGDs
1
1
1
1
BCC2
KUB
WORLD RELIEF
Region: Nyanza
Sub Recipient
BCC3
BCC5
SDA
Number of FGDs
28
BCC14 (Blind impaired/resource

centre)
KUB
NOPE
BCC3
BCC3
FHOK
BCC4
YWCAA
BCC1
Total FGDs
1
1
1
1
1
38
Individual Interviews (IDIs) with beneficiaries

Interviews will be held with beneficiaries of the project to establish their knowledge, attitudes, practices
and behaviours. Comparison will be made with national surveys to gauge the impact of the project. A
structured tool will be administered using personal digital assistants (PDAs).
2.5.2.2
Organizational Systems Evaluation
The organizational system assessment will be conducted so as to understand whether the design of the
system led to the successful implementation of Global Fund Round 7 by Care International in Kenya,
which was the Principal Recipient (PRs), disbursing funds to 54 Sub Recipients, who were meant to
deliver services. This is based on the four main areas:
Grants Management system including the sub-grants management system

Finance system that interplayed with the program. This is backed by CARE finance system and
procedure manual
Audit systems to manage risks and how well compliance was assured
Procurement system including tax systems which informed our programmatic achievement
The SRs will be evaluated on the basis of their rating on organizational capacity at baseline, which
depended on:
Legal and contractual status

Governance and risk management
Programme management
Financial management and reporting
Monitoring and evaluation
The three categories into which CSOs were categorized based on their average overall score on all the
five elements listed above are given distinct colour codes as elaborated below:
29
Table 10: CSO/NGO capacity ratings
Colour Code
Explanation
Green
The Green colour code represents or is given to an organization which, from the
comprehensive assessment, has minimal need for capacity building to undertake the
proposed project. The CSOs in category green can be considered for funding by CARE
Kenya without or with very limited need for capacity building.
Blue
The Blue colour code represents or is given to an organization which, from the
comprehensive assessment, has capacity gaps but can implement projects. CARE Kenya can
fund these organisations with pre-conditions that ensure the CSOs address the identified
capacity gaps/weaknesses.
Yellow
The Yellow colour code represents or is given to an organization which, from the
comprehensive assessment, has major capacity gaps that present huge accountability risks to
any disbursed funds. CARE Kenya may not consider these as SRs because of the material
risk involved, unless prudent mechanisms can be found to guarantee both good programme
and finance performance in line with its GF set grant obligations.
Assessment of Finance Management and Systems

Assessment of finance management and systems will focus on the following issues:
Staffing of finance units, including qualification and experience

Financial policies and procedures
Financial accounting system(computerised or not)
2.5.2.3
Organizational Capacity Assessment Tool
The organisational Assessment tool is a comprehensive checklist and guide for conducting a rigorous
diagnostic. The Organisational Assessment will be targeted at different respondents relevant to the
various sections. Generally the following will be targeted for the main competency areas
Table 11: Main competency areas for the organizational capacity assessment
Competency area
Target Audience
1.
Governance
Board Members
2.
Operations & Management Systems
IT, Operations, Administration Staff
3.
Human Resources
Human resource Staff
4.
Financial Resources
Finance and Grant Managers
5.
Service Delivery
Programme Staff
6.
External Relations & Advocacy
Communication Staff
The tool is based on the 7S framework on Structures, Systems, Style (Leadership), Staff, Strategy, Shared
Values and Systems and provides a guide for key informant interviews as well. It works on the premise
that all organizations be they non-profit making or commercial entities must undergo the four phases of
organizational development. The four phases from start-up to mature are described as follows:
30
Start-up: The beginning phase of organizational development. General characteristics

of a start-up are few individuals involved; not much history; generally very small and not
yet stable.
Developing: The second stage of organizational development. Characterized by rapid
growth, high energy and much activity. Much time is spent developing the service
delivery side of the organization. Usually behind in its administrative functions because
of the rapid growth rate. High level of learning necessitated by the rapid growth. A
mature organization can move into this mode when it attempts a new area of focus or
enters a new geographical area.
Consolidating: Third stage of organizational development. Characterized by continued
high quality service delivery and intensified focus on the other five areas of
organizational development. It is a time when the learning of the developing phase are
incorporated into the organization.
Mature: Fourth stage of organizational development. The organization is fully functional
in the competency areas.
Mature
Consolidating
Developing
Start-Up
There are six competency areas that will be assessed according to these four stages of development. All
organizations evolve differently, and the matrix method of examining an organizations components and
competency areas in terms of these four development stages helps to present a fair picture of a very
complex structure. An organization could be in a different stage of development in each of the six
competency areas at any one time. For example, an organization might be in the consolidating stage with
respect to governance, but still in the start-up stage in terms of its resource base. This is to be expected,
and helps an organization define what it needs to focus on in order to move to the next stage of
development. Thus, the scale produces a snapshot of all the pieces of an organization at a given
moment.
Because many organizations must continually attempt to strike a balance between achieving a sufficient
level of professionalism and avoiding becoming too bureaucratic, the method of pinpointing needs by
area of competency and by stage of development can help to highlight necessary development steps over
unnecessary steps.
Outline of areas of competency
Each of the six areas of competency outlined below is divided into sub-categories, which are further
divided into main elements. For example, under Governance, the sub-category of Mission has three
elements within it: 1) statement of purpose, 2) staff understanding of mission, and 3) links between
mission and programs. Each of these elements is described according to its stage: start-up, developing,
consolidating or mature.
31
Table 12: Six Competence Areas

Competency area
Sub Categories
1.
Governance
2.
Operations & Management

Systems
3.
Human Resources
4.
Financial Resources
5.
Service Delivery
6.
External Relations &

Advocacy
Governing Body
Mission
Legal Status
Constituency
Leadership
Information Communication Technology
Administration
Facilities, Property and Equipment Management
Planning
Internal Communications
Program Development and Implementation
Staff Roles
Task Management
Performance Management and Staff Development
Salary Administration
Team Development & Conflict Resolution
Accounting
Budgeting
Financial Controls
Audit/External Financial Review
Resource Base
Sectoral Expertise
Constituency Ownership
Impact Assessment
Public Relations
Regional Collaboration
Government Collaboration
Private Sector Collaboration
NGO Collaboration
Advocacy
Mobilization of Resources
Organizational assessment will be done using Tool OD_A1
2.5.2.4
Key Informant Interviews
Interview guides based on the research protocol objectives have been developed for
interviewing of PR and SR staff as well as key stakeholders including those from NASCOP,
NACC, LFA, NOC, HIV ICC, Ministry of Health and GF.
The purpose of the key informant interviews will be to provide more in-depth information on the
implementation of the project as well as corroborate data collected through desk review and
from beneficiaries of the project. This will be essential in making a triangulation and balanced
judgment on how key interventions were implemented, how this impacted on the PR, SRs and
beneficiaries and achievements of the project goals.
Information from key informants interviews will be essential for all the components under review
and various tools have been developed based on the components under review and the
category of respondents.
32
2.6
Data Collection tools
The table below summarises the tools to be used in the evaluation:

Table 13: Proposed evaluation tools
Tool
Tool Name
Tool P_A1a
Beneficiaries FGD
Tool P_A1b
Beneficiaries FGD
Tool P_A1a
Beneficiaries FGD
Tool P_A1c
Beneficiaries FGD
Tool P_A1d
Beneficiaries FGD
Tool P_A1e
Beneficiaries FGD
Tool P_A1f
Beneficiaries FGD
Tool P_A1g
Beneficiaries FGD
Tool P_A1h
Beneficiaries FGD
Tool P_A1i
Beneficiaries FGD
Tool P_A1j
Beneficiaries FGD
Tool P_A1k
IDU in-depth interview guide
Tool P_A1l
IDU in-depth interview guide
Tool P_A2
Key informant interview guide
Tool P_A3
Tool P_A4
Tool P_A5
Key Informant Interview guide
Tool P_A6
CSO Document Review framework
Respondent
HTC counsellors and supervisors
PLHIV on ART
Youth beneficiaries (outreach events &sport events)
Youth Peer educators
Youth in school health clubs
Youth resource centres & youth group members
formal sector /Informal sector
Sex worker
MSM
Inmate
PWDs
LDT
Injecting drug users
SR Project officers
Care International project staff
Stakeholders (MoH/NACC)
Stakeholders (GF, KCM, NOC)
Project reports and other relevant literature
Tool GF_A1:
Tool GF_A2:
Tool GF_A3
CARE International in Kenya (PR)

Sub Recipient (SR)
Cost and Efficiency Analysis
Finance and Grant Director (To be answered for EACH Phase)

In- Charge/Programme Manager/ Finance and Manager
Accountant/ Records Officer
Tool OD_A1
Organisational Assessment
Framework
Baseline Assessment Capacity
Assessment Tool
PR/SR Interview Guide on SR
Institutional Strengthening
SR Board Members Interview Guide
SR/PR Leadership, Management, Staff and Documentation
Tool OD_A 2
Tool OD_A 3
Tool OD_A 4

Board members from SRs and documentation
33
3.0 Data Management

3.1
Data Quality Assurance
Several measures such as conducting pre-test, training data collectors, ensuring field supervision, and
maintaining confidentiality will be employed to ensure high quality, validity and reliable data is collected.
Qualified Research assistants: Highly qualified research assistants will be used to collect data.
Selection of the research assistants will take into account their academic qualifications, prior experience
in conducting research and data management skills.
Training: Before the field data collection process, research assistants (RAs) will be trained. They will be
introduced to the study objectives and methodological approaches before being trained in qualitative and
quantitative methods. Each question within the tools will be carefully reviewed with the RAs and
language and context issues addressed and resolved.
Pre-test: The tools will be pilot-tested to check for consistency, ease of understanding and later reviewed
to incorporate the feedback from the pilot results. Pretest will be undertaken in SR not participating in the
main evaluation and which have varied service delivery areas (SDAs). The data from the pretest will be
analyzed to generate anticipated results and inform revision of tools and questions if/as necessary.
Supervision: A supervisor will be assigned to each study site and they will be charged with the
responsibility of ensuring data quality issues are adhered to as well as acting as the 1st point of contact in
case of any arising data-related issue. To ensure quality of data collected the lead consultants will make
field checks and validate the data collected.
3.2
Ethical Considerations
The data and information collected all respondents will be confidential and all identifiers linking
responses to any individual will not be obtained unless with express consent of key informants. All
respondents will be informed of the background and purpose of the survey. Participation will be purely
voluntary.
34
3.3
Data Entry and Analysis
For qualitative data, the FGD and KII assistants will have made field notes and under supervision, they
will transcribe the notes and make summaries. These will be handed over to the person in charge of
qualitative data who will triangulate and prepare draft reports per site against set thematic areas. For
quantitative data from the structured interviews, an appropriate database will be created to help with data
capture. The data will be analyzed to address the specific objectives of the end-term evaluation and the
research questions. To address research questions responses from individual interviews will be
triangulated with KIIs and FGDs.
3.3.1
Project Interventions Evaluation
The themes of analysis are:

a) Relevance and quality of design,
b) Efficiency of implementation,
c) Effectiveness of strategies employed,
d) Impact of the interventions (comparison to baseline and midterm),
e) Sustainability of project interventions,
f) Best practices,
g) Lessons learnt,
h) Challenges
For qualitative data, the FGD, IDIs, and KII, assistants will have made field notes and under supervision,
they will transcribe the notes and make summaries. These will be handed over to the person in charge of
qualitative data who will triangulate the field notes, transcribe and summarize notes and prepare draft
reports per category against set thematic areas. For quantitative data, a database will be created using Ms
Access to help with data capture.
The data will be analyzed to address the specific objectives of the evaluation and the research questions.
To address research questions responses from individual interviews will be triangulated with KIIs and
FGDs. Univariate (frequencies and proportions), bivariate (cross tabulations and chi-square) and
multivariate (logistic regression) analysis will be conducted. Other analyses like non-parametric tests may
also be conducted based on the data collected and any other programmatic issues that may arise that may
not have been envisaged during study design and data collection phases.
3.3.2
Organizational Systems Assessment:
Assessment of finance management and systems will focus on the following issues:
Staffing of finance units, including qualification and experience
Financial policies and procedures
Financial accounting system(computerised or not)
35
The themes of analysis are:

a) Contracting & Budgeting:
b) Funds Disbursements & liquidations
c) Financial reporting
d) Operations & Systems
e) funds utilization
f) absorption
g) Financial Risk Management & Audit
h) Technical Support/ Capacity Building
i) Value for money/ impact (performance based funding)
j) Close Out
Assessment of Risk Level
The risk level for SRs in implementation of Phase 1 and 2 of the Global Fund Grant will be rated on a
scale of low, medium and high risk. The rating will based on the characteristics of the strengths and
weaknesses demonstrated by an SR. A weighting will be given for each of the three areas covered under
the assessment based on the risk level. The weighting will correspond to the risk level as follows: Low
risk was weighted 3, Medium risk was Weighted 2, and High risk was weighted 1. The characteristics
considered for each level of risk are shown in Table 6.2 below.
Table 14: Criteria for Rating Risk Levels for SRs
Area
Governance and
Programme
Management
Monitoring and
Evaluation
Financial
Low Risk
Medium Risk
SR board plays its oversight role
SR Board oversight role not
effectively; meets regularly and
adequately developed board
reviews programmatic and
meets regularly, follows up on
financial performance of the
its decisions but does not
organization
receive adequate reports
from management
Strong programme management
met all its targets, clear project
Programme Management
implementation structure that
Met most of its targets, has
reaches to community level, all
implementation structure in
requires staff are in place,
place, some elements of a
performance management system is
performance management
in place and good infrastructure,
system are in place and has
vehicles and computers.
elements of infrastructure
Strong monitoring and evaluation
Monitoring and Evaluation
system (M&E staff, M&E plan and
system (M&E staff in place,
M&E work plan and budget in
M&E plan in place and M&E
Place) Reports to PR according to
work plan and budget
schedule, minimal issues raised on
adequate). Reports submitted
the reports, verifies data presented
to the PR according to
to the PR and has a clear
schedule though with gaps,
mechanisms for monitoring in
monitoring system in place
place
has some gaps but produces
reliable data and SR has not
raised major issues with the
reports submitted.
Strong financial management and
Financial management and
High Risk
SR board is weak does not meet
regularly, does not address issues
arising from project implementation
and not clear whether it makes any
follow ups on decisions it makes;
oversight function conflicted with
management function
Programme management is weak not
met most of the targets;
implementation structure not adequate
reaching up to community level; staff
not adequate and no performance
management system is in place.
Inadequate M&E system (M&E staff,
M&E plan and M&E work plan and
budget largely weak) reports not
submitted according to schedule,
major issues raised by the PR
leading to delay in funds disbursement,
monitoring system not well developed
Inadequate financial management
36
management and
systems
system no major financial issues

raised by the PR, financial policies
and procedures are adequate,
internal controls are enforced and
financial reports are submitted
according to schedule. Has not
been made to refund any money to
the PR
system some issues raised

by PR but resolved within
reasonable duration, financial
policies are adequate, internal
controls are enforced through
they have some gaps and
reporting is according to
schedule. Refund to the PR are
not significant
system PR raised questions on

financial reports; issues not resolved
within reasonable duration; internal
controls not effectively enforced and
has had to refund substantial funds to
the PR
Source: Mid Term Consolidated Sub Recipient Assessment, Care International in Kenya
Cost and Value for Money Analysis
The costs to be included here are the resource costs over the GFATM Round 7s life, from development
to the end of the programme for the 54 SRs. The details of financial costs will be obtained from SR
records and reports submitted to the PR.
Costs to be included are intervention development costs, implementation costs, and post implementation
costs such as follow up feedback and supervision costs. Costs will be presented in local currency (Kenya
shillings) and further converted US dollars, using the purchasing power parity (PPP) rate, providing for
international comparison. The consumer price index (CPI) will be used to deflate each years costs to a
certain base year chosen from the range of costs collected. The base year will be chosen from a year with
less economic upheavals. Capital inputs used in this study will be annualized using a discount rate of 3
percent, which is consistent with the convergence to the long run equilibrium economic growth rate.
Besides, depreciation of the equipment used will be done using the official rates provided by Government
of Kenya. This will be done in consideration to the useful life of the equipment to obtain the salvage
value. Meeting venues will be valued at market prices.
Increasingly, there has been focus on evaluation of Value for Money considerations when using
resources. Value for money refers to an approach to making investments that will thereafter represent the
most efficient (allocative and delivery) use of resources. Defining and estimation of value for money
indicators for HIV at SR level would be done using the specific interventions that the SR were involved
in. Value for money considerations would measure efficiency in the use of resources in the delivery of
services, especially comparing phase 1 and phase 2 of implementation of Global Fund Round 7.
Some of the selected indicators to be estimated for Value for money are listed below.
37
Table 15: OT.1 Cost per person who received testing and counselling services, by mode of provision who received their
results (US$ per person)
S.No
1
2
3
4
5
6
7
Item
Procurement cost (Equipment and supplies)
Service Delivery costs
Transport and distribution cost (Fuel, materials, and per diem etc)
IEC and BCC materials
HTC provision related preparatory costs (studies, consultancies etc)
SR and PR level HTC programme management and overhead allocation for the
period, including training
Output (Total number of HTC conducted in the period)
Table 16: OT.2Cost per person currently receiving antiretroviral treatment, 1st line regimen (US$ per person)
S.No Item
1
2
3
4
5
6
7
8
Procurement cost
Warehousing and security
Service Delivery cost
Tests (all the relevant tests conducted for ART e.g. Liver function tests etc)
ART related preparatory costs (logistics studies, consultants etc)
Programme management and overhead allocation to ARTs for the period, including
training
Output (Number of people provided with ART)
Table 17: OT.3 Cost per person receiving antiretroviral treatment, 2nd line regimen (US$ per person)
S.No
1
2
3
4
5
6
7
8
Item
Procurement cost
Tests (all the relevant tests conducted for ART e.g. Liver function tests etc)
ART related preparatory costs (logistics studies, consultants etc)
Programme management and overhead allocation to ARTs for the period, including
training
Output (Number of people provided with ART)
38
Table 18: OT.4 Cost per person receiving Community Mobilization (US$ per person)
S.No
1
2
3
4
5
6
7
Item
Procurement cost
Community mobilization related preparatory costs (logistics studies, consultants etc)
Programme management and overhead allocation to Community mobilization for the
period, including training
Output (Number of people provided with Community Mobilization)
Table 19: OT.5 Cost per person receiving Youth Friendly Activities (US$ per person)
S.No
1
2
3
4
5
6
7
Item
Procurement cost
Youth Friendly related preparatory costs (logistics studies, consultants etc)
Programme management and overhead allocation to Youth Friendly for the period,
including training
Output (Number of people provided with Youth Friendly activities)
Efficiency Analysis
The measurement of efficiency in the health care sector is complicated by the nature of the production
process. Measurement of the ideal output such as improved health status is difficult, both conceptually
and empirically. Complications arise from the fact that health status is a function of many variables,
which are exogenous to the health sector, for example, household income, education, and intra- household
decisions.
There are different measures of efficiency ranging from average productivity measures (ratios) to bestperformance frontier efficiency measures (DEA and SFA). SRs efficiency is measured relative to
efficiency of other SRs that received grants from CARE International in Kenya. A SR is technically
efficient if it is located on the frontier, whereas allocative efficiency demands that production take place
at the point where costs of production are minimized.
Data Envelopment Analysis
DEA is a functionalist, linear programming methodology for evaluating relative efficiency for each SR
among a set of fairly homogeneous SRs, which can also be referred to as decision making units (DMUs),
since they make decisions on production. Technical efficiency is a measure of the ability of a DMU to
provide maximum quantities of services (outputs) from a given set of health systems resources (inputs).
TE is affected by the size of operations (scale efficiency) and by managerial practices (non- scale
technical efficiency or pure technical efficiency.
DEA plots an efficient frontier using combinations of inputs and output from the best performing DMUs.
DMUs that compose the best practice frontier are assigned an efficiency score of one (or 100%) and are
deemed technically efficient compared to its peers. The efficiency of the DMUs below the efficiency
frontier is measured in terms of their distance from the frontier. The inefficient DMUs are assigned a
score of between zero and 100 percent. The higher the score the more efficient a DMU is.
39
Since DMUs employ multiple inputs to produce multiple outputs, their individual TE can be defined as:
Technical Efficiency Score =
Weighted sum of outputs

Weighted sum of inputs
TE score of each DMU in this study is taken as the maximum of a ratio of weighted outputs (Number of
people provided with a service) to weighted inputs (resources used to provide the services) subject to the
condition that the similar ratios for every DMU be less than one or equal to unity.
Data Type, Sources and Description
Data to be collected will include inputs, outputs and other relevant service provider statistics. However, it
will be necessary to group the SRs on the basis of their size and scope of activity. Recurrent inputs will be
expressed in monetary terms as well as total recurrent expenditure. Outputs are expressed as number of
people provided with services as detailed in the above sections (Section 6.1 to 6.4) such as number of
people provided with HTC, number of people provided with ART, Number of people provided with
community mobilization etc.
Inputs in health production are classified as labour, capital and supplies. Labour input can be
disaggregated into the various professional groups such as doctors, clinical officers, nurses,
administration staff etc. In most studies capital is proxied by the number of beds.
The choice of variables that will be used to estimate the efficiency scores will be agreed upon between the
client and the consultant.
Explaining Efficiency
This study will aim to understand the sources of inefficiency for the selected SRs (DMUs). This will
entail the analysis of collected data together with relating the quantitative results to the qualitative results
obtained from interviews and the interrogation of data and results obtained from the Organisational
Development component.
3.3.3
Institutional Capacity Evaluation Data Analysis Framework
The data collected will be in form of qualitative or through quantitative checklists. The data
analysis will be conducted through context analysis for the qualitative data and summary of
rating for the checklists. The plan for analyzing this information will be placed in the framework
summarized in the table below that generally seeks to answer the following questions:
40
Achievement of
program
goals/lessons
GF Objective 4
Interventions
Achievement of
outcomes
Baseline
findings/programm
e targets
Table 20: Data Analysis Framework

First
Second
Third Reference
reference
Reference
Point End term
Point:
Point (Mid-term Capacity
Baseline
review) End of
Assessment
Assessmen Phase 1,
t for Phase
inception of
1
phase 2
Source of Documentati Mid-term
1)
Organisatio
Informati
on and Key
Capacity
nal Assessment
on
Informant
Assessment
Tools
Interviews
reports,
2)
Key
Baseline
Capacity
Informant
Capacity
Building Plan,
Interviews
Assessment, Other
3)
Review of
and
subsequent
outputs per work
subsequent
plans and
plan (M&E reports,
capacity
documents
Financial reports
reports
among others
before and after
interventions
Were the program

outcomes
adequately met and
did it lead to
achievement of
programme
outcomes?
Did they lead to
improvement
(achievement of
outcomes hence
contributing to the
overall achievement
of goals?
Assessing whether
the outcomes were
achieved and how
they impacted on
delivery of
programme
outcomes
Recommendati
ons and
Lessons learnt
4)
Success
5)
Relevanc
e of the
interventions
6)
Impact of
the Interventions
7)
Efficiency
of the
interventions
8)
What
could have been
done better
Evaluation Design for Institutional Capacity Assessment
41
The evaluation approach will be guided by the research protocol developed by CIK at the
inception meeting. There will be two key reference points namely the baseline capacity
assessment and the mid-term capacity assessment and supporting data. The approach will
accomplish three key activities in conducting the end term evaluation:
1) Conduct a rigorous document review to establish the level of interventions for Objective
4 at both PR and SR level, and the outputs of the same
2) Conduct a comparative analysis of all 54 SRs from baseline to end term review. End
term review of all SRs will be done using the organizational assessment tool (see
Annex) and document reviews on performance held by PR however the mid-term
capacity assessment rating scale will be applied.
3) Conduct a field based analysis of a representative sample of SRs selected for this
evaluation including a thorough organizational assessment checklist and key informant
interviews to assess the effectiveness of phase 2 interventions.
3.3.4
Gender and SRH Integration Review:
The evaluation will focus on both institutional and programmatic aspects of the GF programme that
enabled or constrained gender and SRH integration. The programmatic aspect will encompass the first
three broad objectives of the GF R7
programme (to scale up and maintain PLHIVs on ART, to
increase access to HIV testing and counseling (HCT) services and o increase uptake of HIV prevention
and treatment services) whereas the institutional aspect will address the objective on strengthening
institutional capacity of HIV program implementers. The frameworks for both institutional and
programmatic assessments are as presented below.
Organizational
The structure presented below will be used to assess the extent to which gender and SRH issues have been
integrated in institutional systems for organizations involved in the implementation of the GFR7
programme. The key areas of focus will be organizational policies, mandate, organizational structure and
the organizations human resources.
Table 21: Organizational Gender and SRH Evaluation Framework
Organizational Dimensions
Gender and SRH policies
Human resources
Financial Resources
Organizational Culture
Types of Information Sought

The availability, nature, quality, extent and intensity of support for the
organizations gender and SRH policies.
The level, extent and intensity of gender sensitive human resource
policies, responsibilities, staff development and gender considerations in
hiring and personnel reviews.
The level and extent of organizational resources budgeted to support
gender equity efforts.
The extent and intensity of gender sensitivity in the organizational
norms, structures, systems, processes and relations of power.
42
Programmatic:
The information sought essentially looks at how HIV programmes and projects are designed in an
organization and their responsiveness to gender and SRH issues. The components under this dimension
are listed below.
Table 22: Programmatic Gender and SRH Evaluation
Programming Dimensions
Program Planning & Design
Program Implementation
Monitoring & Evaluation
Partner Organizations
3.4
Types of Information Sought

The extent to which gender and SRH sensitive organizational
procedures and methods were used to conceptualize and design
projects.
The extent and intensity of gender and SRH responsive implementation
of the GF R7 project.
The extent and intensity with which gender disaggregated data and
SRH information is incorporated in the monitoring and evaluation of
the GF R7 programme.
The extent to which gender equity and SRH programming is integrated
in an agencys partner or local NGO affiliate relations.
Report Writing
A draft report will be prepared after completion of field work and data analysis in line with the Terms of
reference (TOR) and evaluation protocol. The draft report will be shared with stakeholders after whom a
final report with clear findings and recommendations for strengthening programming will be prepared.
The report will be presented in such a way that it systematically answers the evaluation objectives and
measuring the project impact, outcome, output and process in place. See appendix 1 for the final report
outline. A popular version of the report will also be prepared based on the final report.
At a later stage the findings of the ETE will be disseminated to the beneficiaries, stakeholders and other
members of the public using various communication channels such as print media (newspapers),
workshops and website. The communication materials to be developed will include policy briefs,
PowerPoint presentations, brochures and reports. During the national and county level dissemination
meetings an information package will be developed for the participants. The contents of the package will
include among others items:
County level brochure highlighting the key findings of the end term evaluation and project;
PowerPoint presentation;
Policy brief.
43
4.0 Operational Work Plan
44
5.0 Assignment Technical and Managerial Structure
5.1
Team Composition
5.2
Roles and Responsibilities
Principal Consultant
Represents Bon Sant in contractual agreements with Care
Represent the research team in any dealing with the Care International,
Attend all planning meetings,
Oversee planning, writing and completion of the research proposal for submission to receive
administrative approval where necessary, including but not limited to:
Development and completion of study protocols.
Development and completion of study tools.
Design of the sampling strategies and selection of sites.
Pretest the questionnaire.
Review data after the pilot study is done and incorporate the necessary changes.
Oversee and where necessary:
Perform data collection
Conduct data analysis and
Oversee the process of report writing ensuring quality and accuracy prevails.
Submit a report detailing activities, achievements, challenges and recommendations in
undertaking this activity.
Co-Consultant
Provide technical lead to all consultants
Attend all planning meetings,
Lead in planning, writing and completion of the research proposal for submission to receive
Design of the sampling strategies and selection of sites.
Lead the process of report writing ensuring quality and accuracy prevails.
Finance and Grants consultant
Provide lead in finance and grants evaluation component,
Attend planning meetings,
Participate in planning, writing and completion of the research proposal for submission to receive
Coordinate RAs engaged in Grants and Finance component
Lead in finance and grants component.
45
Participate in the process of report writing ensuring quality and accuracy prevails. Lead in finance
and grants component.
Organisational Development Consultant
Provide lead in Organizational development evaluation component,
Lead in organizational development component.
Coordinate RAs engaged in organizational development component
Participate in the process of report writing ensuring quality and accuracy prevails. Lead in
organizational component.
Gender Consultant
Provide lead in gender integration evaluation component,
Lead in gender integration component.
Review data after the pilot study is done and incorporate the necessary changes. Oversee and
where necessary conduct data analysis on gender integration and
Participate in the process of report writing ensuring quality and accuracy prevails. Lead in gender
integration component.
Communications Consultant
administrative approval where necessary
Participate in the process of report writing specifically design and layout.
Develop advocacy brief
Develop Research brief,
Develop county briefs.
46
The involvement of the consultant throughout the ETE process is crucial to ensure a proper
understanding of the project is attained and appropriate materials are developed. The role of the
communications consultant in team will be to:
a) Provide advice to the team on the content and format of the materials to be used to
disseminate the findings of the ETE.
b) Undertake the design and layout of the final report (approximately 35 pages).The final
ETE report will be designed to highlight key findings and key messages. Relevant KII
quotes will be highlighted and certain pictures included.
c) Develop a popular version of the end of term evaluation using information extracted from
the final report (approximately 10 pages)
d) Undertake the design and layout of the power point presentations for the validation
workshop and county specific briefing sessions. The PowerPoint presentation for the
validation meeting will summarize the findings from all regions and also give a national
picture.
e) Design and layout county specific brochures highlighting findings from the report
The consultant will guide the research assistants on the pictures which need to be taken during
the data collection process to be incorporated into the final deliverables;
The consultant will ensure layout, language and photography fits within the CARE brand
guidelines;
The consultant will ensure the materials are suitable for the intended target audience County
briefs: administrators in government as well as community members; Policy briefs: government
bodies; Reports: members of the community
Activity Coordinator
Oversee evaluation logistics,
Develop training and pretest report,
Coordinate project interventions evaluation research assistants,
Providerappoteur services to the evaluation team during planning meetings,
Coordinate between Care, Bon Sant and the various consultants.
Administrator
Oversee the use of field work budget,
Prepare financial reports on the use of field work funds and regularly submit to Care for
further disbursements
Prepare final field report on the use of field work funds including submission of receipts
according to Care and Global Fund guidelines
Data analyst
Conduct data analysis
47
Research team lead

Provide supervision of data collection,
Ensure data quality is adhered during data collection,
Conduct data collection
Research Assistants
Conduct data collection
48
Appendix 1: Final Report Outline
49
Appendix 2: Documents to be reviewed

Table 23: Documents to be reviewed
Category
Capacity Assessment
Reports
Sub-Category
FINAL SRS CA REPORTSMidterm
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
Files
ABAHLIGHT FOUNDATION
ACTION AID Revised
BEACON OF HOPE
BHESP Revised
CATHOLIC DIOCESE OF HOMABAY
CHAK
CHAT Revised
CRED
FASI
FAVO
FHK
FHOK
FOPHAK Revised
GLOBOCOM Revised
GOAL Revised
HVFI Revised
HWWK
KAIPPG
KANCO Revised
KAWI
KECOFATUMA
KENWA
KICABA
KSMH Revised
KUB
KWETU TRAINING CENTRE Revised
LICASU Revised
Maji na Ufanisi
MATATA Revised
MDM Revised
MERLIN Revised
MSK Revised
50
Category
EXCHANGE VISITS
Sub-Category
Cross learning visits reports
KCM 2012 Dashboard reports

KCM DashBoard and
Oversight visits reports
KCM 2013 Dashboard reports

Oversight visit reports 2012
Oversight visit reports 2013
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
Files
MSK
MUSOKOTO
NCCK Revised
NIA Revised
NOPE
NOVOK Revised
NYARAMI
OAHIP Revised
PEACENET
SUPKEM Revised
UDEK Revised
WASDA
WODERA
WOFAK
WRK Revised
YPD Revised
YWCAA
ZINDUKA
Baseline GF SR Capacity Assessment ratings -May 09 after feedback - coloured
CBP revised MaY 2013 - Consolidated 13.5.13
Consolidated Mid term Sub Recipient Assessment Report 1 July 2011
REPORT OF THE ASSESSMENT - Basline
SR Capacity stregthening action plan consolidated (3) - reviewed 14th Sept '11
SR LEARNING VISIT REPORT TO WRK
SRs Cross Learning Visit report- Nyarami
SRs Cross Learning Visits Report to UDEK
Exchange Visits Concept 6th June 2012 final July 23rd
CARE GFR7 Phase II Dash Board template for April -June 2012 30 9 12
CARE GFR7 Phase II Dash Board template Oct-Dec 2012 Dated 25. 2.13
Final CARE GFR7 Dash Board template July -Sept 2012 - 30 9 12
CARE GFR7 Phase II Dash Board template for Apr-Jun 2013 dated 15.8.13
CARE GFR7 Phase II Dash Board template for Jan-Mar 2013 17.5.13
CARE GFR7 Phase II Dash Board template for July -Sept 2013 Final
CARE GFR7 Phase II Dash Board template for Oct -Dec 2013 Final 7022014
NOC Oversight Final Report docx
Final Oversight Report Eastern Region June 2013
51
Category
Sub-Category
Key program documents - Phase 2
M&E
M&E Plans
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
Files
Western Region KCM Oversight RecommendationsActionTemplateMay2013-CAREFeedback 1
Final CARE KENYA R7 HIV Phase 2 PF - 6th September 2011
PF REVISED 13.2.2014
PR_Budget Feb_7_2011
RCF GF7 Ph 2 Application Consolidated Budget Mar.8.2011
RCF GF7 Ph 2 SR Budgets Mar.8.2011
CARE MINIMUM STANDARDS DRAFT SUBMISSION WORKING DOC FEBRUARY 6 2013 Draft
BHESP ME Plan Final
BOH ME Plan Final
CHAT ME Plan Final
CIK PME Plan phase 2 _ Final Sep 2011
FASI ME Plan
FHK ME Plan Final
FHOK ME Plan Final
GCI ME PLAN REVISED FINAl
HVFI ME Plan Final
HWWK ph 2 M and E plan-revised Final
KAIPPG ME Plan Final
KANCO ME Plan Final
KENWA ME Plan Feb 9th Final
KICABA ME Plan Feb 9th Final
KUB ME Plan Feb 9 Final
KWETU ME Plan Feb 9th Final
M&E Plans Tally
Matata M E plans final
MDM M&E Plan Final
Merlin ME Plan Final Feb 1st
MNU Final ME Plan Feb 1st 2012
MSK ME Plan Feb 1st Final
NCCK ME Plan Feb 1st Final
NIA ME Plan Feb 1st Final
NOPE M&E Plan final Feb 6th
Nyarami ME Plan Feb 6 Final
OAHIP M&E Pln Feb 6th Final
PEACENET ME PLAN Final
SUPKEM ME PLAN Feb 14th Final
52
Category
Sub-Category
Mid term review

OSDV
Performance Framework
Phase 1 Mgt Letters

MANAGEMENT LETTERS GF
TO CARE
Phase 2 Mgt letters
CARE PF Budget and Work

Plan 5 years
Phase 1
Phase 1 Field Visits Reports
ABL
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
Files
UDEK ME Plan Feb 9th Final
WASDA ME Plan Feb 9th Final
World relief ME Plan Final Feb 14th
YWCAA ME Plan Feb 9 Final
ZAF ME Plan Feb 9th Final
CIK PME Plan phase 2 14 Jan
Mid Term Review Report for Phase 1 of GF Round 7 Program -Final VersionCARE OSDV ML 2012
OSDV 2011 reported results vs verified and 2010 status
PR Responses following the OSDV 2012 Findings
CARE Phase 1 PR PF ME reviewed 21Oct08
Final CARE KENYA R7 HIV Phase 2 PF - 6th September 2011
KEN-708-G10-H_Performance Framework_24sep10
Phase 2 PF REVISED 13.2.2014
Kenya_Global_Fund_Round_7_Proposal
REPORT_OF_THE_INDEPENDENT_REVIEW_PANNEL_EVALUATING_CSO
GF Implementation Letter No 3 - No Cost Extension to June 31 2011 28th March 2011
KEN-708-G09-10_Letter re Budget M+E Plan 8 July 2010
KEN-708-G10-H Fund Implementation Letter 16 June 09
KEN-708-G10-H D02 Management Letter 12th August 2010
KEN-708-G10-H Implementation Ltr 2 30th July 09
Kenya - KEN-708-G10-H - IL 4 - Signed by the Fund 20 March 2011
Kenya - KEN-708-G10-H - PUDR 1 April 2010-30 Sept 2010 -Management Letter 17th Feb2011
CARE Annual Disbursement 1st March 2013
CARE_ML_PUDR_dec 2011 PUDR Jul - Dec 11- 21st Mary 2012
KEN-708-G10-H_D3_Management Letter NCE PUDR 11th Nov 2011
KEN-708-G10-H_D6_ML_7May13 PUDR Jul-Dec 12 7th May 2013
KEN-708-G10-H-D5_Management Letter_26Nov12 PUDR Jan-Jun 12 26th Nov 2012
KEN-708-G10-H-D5_Management Letter_26Nov12 PUDR Jan-Jun 12 26th Nov 2012
CARE PF Budget and Work Plan 5yrs - 30Oct08 overall budget
Workplan
CIK PME Plan 17Apr10
KEN-708-G10-H (CARE) Performance Framework 31 October 2008
KEN-708-G10-H_Performance Framework_24sep10 Revised Sept 2010
ABLF mombasa
ABLFMombasa
53
Category
Sub-Category
BAR HOSTESS
141
142
143
144
145
CDoH
146
CHAK
CHAT
CRED
FASI
FAVO
FHOK
FOPHAK
GLOBCOM
HVFI
HWF
KAIPPG
KANCO
KAWI
KICABA
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
Files
Bar Hostess field supervision and verification Report
BAR HOSTESSES FIELD VISIT REPORT
BHESP FIELD VISIT REPORT MAY 2011, final
PR_field Activity verification_YG_RC and SHC_GFR7_T12 - BHESP, 2011 May 11-13
REPORT OF VISIT WITH BAR HOSTESSES
REPORT OF GF PROGRAM MONITORING VISIT WRK-CHAK-KSMH-PIP 12 TH - 16TH APRIL 2010 Final
CDoHB Verification and Field Visit Report March 2010 final
FIELD VISIT TO CHAK FACILITIES JAN-FEB 2011 a
CHAT field verification visit-10th to 15th May 2010
CRED verification report
CRED VISIT JAN 2011
FASI field visit report 22 1 10 (2)
FAVO Field Visit Report
FHOK Eldoret field visit report_May 2011
FHOK Field verification repoirt_April 2010
FHOK Kisumu field visit report_Feb 2011
FHOK_field data verification_-Kisumu_Feb 2011.
FOPHAK Field Visit Report 27 - 30 1 2010
GLOBCOM FIELD VISIT REPORT MAY 2011 2
GLOBCOM FINAL FIELD VISIT REPORT OCTOBER 2010
HVFI field verification visit-Nanyuki & Nyahururu march 2011
HWF Field Data Verification for ART Sensitization
HWF Field Data Verification Youth Groups _ Manga
HWF Field Data Verification Youth Groups _ Nyamira
HWF Field Visit Report February 2011
HWF Field Visit Report March 2010
KAIPPG Field Visit Report -Mumias9 4 2010
KAIPPG FINAL FIELD VISIT REPORT 22ND FEBRUARY 2011
Copy of KANCO field visit report 21 1 10
KANCO field Vist _12th April
Kanco Verification and Field visit Report
PR_field data verification_YG_RC and SHC
SR Global Fund Field Visit Report to KAWI Arthi river prison
KICABA field supervision and verification Report
TOR FOR FIELD VISIT TO KICABA
54
Category
Sub-Category
KSMH
KUB
KWETU
LICASU
MATATA
MDM
MUSOKOTO
NCCK
NIA
NOVOK
NYARAMI
OAHIP
PEACENET
PIP
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
Files
Copy of KUB field visit report 21 1 10
Field Data Verification YRCs (Aherao and Kakamega)_Feb 2011
Field Data Verification YRCs Feb 2011
KUB field visit report Kisumu and Kakamega_Jan 2011
KUB field visit report_May 2010
PR_field data verification_Sensitization_Trainings and Outreach_Feb 2011
Report of field visit to KUB activities in Mombasa and Kilifi _May 2011
Kwetu Field Data Verification for YRCs Oct 14th 2010
Kwetu Field Data Verification YRCs Oct 13th 2010
Kwetu October 2010 Verification and Field Visit Report
Kwetu Verification and Field visit Report April 2010 Final
LICASU Kenya List of Particpants Called_Oct-Dec 2010
LICASU Verification and Field visit Reporting
LICASU Verification Visit report May 2011
MATATA VISIT JAN 2011
Matata Field Visit Report 30 01 2010
MDM Verifications Visit ReportMusokoto Field Verification Report for December activities_Jan 2011
Musokoto field verification report for January activities_April 2011
Musokoto field visit report_ Finances
PR_field data verification_Sensitization_Trainings and Outreach_Jan 2011
Programme Quality Area
NCCK COAST FIELD VISIT REPORT18TH FEBRUARY 2011 (2)
FIELD VERIFICATION REPORT FOR NEIGHBOURS IN ACTION 13th April 2010
FIELD VERIFICATION REPORT FOR NEIGHBOURS IN ACTION
NOVOK field visit report 21 1 10
NOVOK field visit report 21.1.10
NOVOK field data verification_Feb 2011
NOVOK field visit report Vihiga_May 2011
Nyarami Visit (2)
Nyarami Visit_Oct 2009
OAHIP Verifications Visit Report- (2)
FIELD VISIT TO PEACENET FEB 9-11-2010
PIP FIELD VISIT REPORT
55
Category
Sub-Category
211
212
SANAA
WOFAK
213
214
215
216
217
218
219
220
221
222
223
WRK
224
YPD
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
SHA
SUPKEM
UDEK
WASDA
WODERA
ZINDUKA
Phase 2 field visit reports
Files
PIP report of field visits 22-2-2010
Field visit report for SANAA
FIELD VISIT TO SHA March 2011
SHA Verifications Visit Report
SUPKEM Field Data Verification for Youth Outreach Events March 2011
SUPKEM Field Visit Report March 2011 Final
UDEK Field Visit Report 25 03 2010
WASDA -FIELD VISIT Draft REPORT May 8 - 13 2011
WASDA field visit report FEB 2010
WODERA FEBRUARY 2011 VERIFICATION , VISIT REPORT
WODERA report field visit- 8 1 09
WOFAK GF CA Verifications Reporting
SUMMARY OF YPD RESOURCE CENTRES
Global Fund Field Visit terms of reference Zinduka_April 2010
Zinduka field visit report_May 2011..
FASI report final
Field Visit Kayole KSMH Feb 2012
Field Visit KSMH offices
Field Visit Report - KICABA 15 2 2012
Field Visit to FOVUP - 7th to 8th February 2012 (1)
Field Visit to KENWA - 1st to 2nd Febbruary 2012
FOPHAK Bribery incidence report
GCI report final
HVFI 1-3rd February2012
KAIPPG report final
KANCO, KUB, MSK REPORT JAN 2012
KICABA 2
KWetu- field visit Report Dec 2011
Matata Verification and Field Visit Report February 2011
MERLIN Report, final
Mothers Delight Moments field visit report Dec 2011
Report of field visit to BOH 6 -7 Feb 2012
56
Category
Sub-Category
PR WORKPLAN
PROGRAM REVEIW
MEETINGS
PROGRAM STRUCTURE
Organograms
Phase 1 EFR
PUDRs and EFRs
Phase 1 PUDRs
Phase 2 PUDRs
SRs PERFORMANCE
ANALYSIS
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
Files
Report of field visit to Kwetu 1st - 2nd Feb 2012
Report of field visit to MDM 30th & 31st Jan 2012
Report of field visit to WRK 8 Feb 2012
SUPPORT VISIT TO MATATA NYARAMI -19th to 20th Dec 2011
World Relief 7-8 .2.2012
PR Workplan Consolidated Aug 10- 2011
PR Workplan Review Notes Consolidated July 18 2012 PM Rvd 24Jul
PR Workplan reviewed on 14th Feb 2012 PM Rvd 22 Feb
PR Workplan Schedule of Activities as at 27 Sept 2012 (2)
PR Workplan Schedule of Activities Reviewed as at 7 Nov 2012 Final
PRM - Maanzoni Sept 13 Report
PRM OCTOBER 2012 REPORT DRAFT ONE Jackie
PRW -Hotel Water Buck (March 12)
PRW -Maanzoni may 2013
Report of the 1st GFR7 Implementers meeting June 2010
Report_Programme Review Workshop_Gf Apr 2011
GF Organogram phase 1 Reviewed April 2010
GF Organogram Phase 1 Original at Program Start
GF Organogram Phase 1 Reviewed June 2010
Revised Organogram Phase 2
1ST CIK EFR for the period ended 30th June 2010 Reviewed 28th Sept 2010
2ND CIK EFR 30th Sept 2010 with Reviewed SRs Budgets 25 02 11-Revised
3RD CIK EFR 31st Dec 2010 with Variance Analysis dated 10th Feb 2011
CIK EFR 31st March 2011 25.05.11
CIK 1st PUDR April 09 to Sept 09 Revised Feb 2010
CIK PUDR April 2010- September 2010 revised dated 1st December 2010
CIK PUDR October 2010 - March 2011 Final 13th May 2011 (2)
Final Ph 1 PUDR as at June 2011 - Sept13th September 2011_Updated 29 9 11
Revised 2ND CIK PUDR October 09 - March 2010 25May10 Final
CIK Jan - June 2012 PUDR EFR 29th AUG 2012 with EFR
CIK PH2 Jul 2012 - Dec 2012 PUDR 15th Feb 2013
CIK PH2 PUDR Jan 2013 - June 2013 Final with EFR
CIK PH2 PUDR July-December 2011 20 Feb 2012
CIK PUDR JULY - DECEMBER 2013 14th Feb 2014
Performance Analysis as at End of June 2011 by SDA
SRs Performance Results Phase 1 upto 30 th June 2011_Final
57
Category
Sub-Category
AAIK
ALF
BOH
CHAT
FASI
GLOBCOM
HVFI
HWF
KAIPPG
SRs Quarterly Progress

Feedback Ph1
KANCO
KENWA
KICABA
KUB
MSK
Neighbours
OAHIP
PEACENET
SUPKEM
UDEK
WRK
ZINDUKA
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
Files
AAK program performance feedback
ALF Aug summary rpt
ALF July summary rpt
BOH Program performance feedback _Jan 2011
CHAT Kenya Program performance feedback _Jan 2011
FASI Program Report Review for December 2010
GCI Program performance feedback _Jan 2011
GCI Program performance feedback 2
HVFI Program performance feedback 2
HWF Program Report Review June 2010
KAIPPG Program performance feedback _Jan 2011
KAIPPG Program performance feedback _Jan 2011[1]
KANCO Program performance feedback _Jan 2011
KENWA Program performance feedback _Jan 2011
KICABA Program performance feedback_Jan 2011
July Report Summary KUB thursday
MSK Program performance feedback
July Summary Report for NIA Wafula
NIA Program performance feedback (5)
NIA Program performance feedback
OAHIP Program performance feedback 2 (2)
PEACENET Program performance feedback_Jan 2011
December 2010 programmatic Reports Review for SUPKEM
Supkem program progress feedback_Dec
Supkem program progress feedback_Jan 2011
UDEK Program performance feedback _Jan 2011
World Relief Kenya Program performance feedback 2
ZINDUKA Afrika Program performance feedback _Jan 2011
ZINDUKA Afrika Program performance feedback_Sep 2010
58
Appendix 3: Summary of Questions, Methodology and Data Sources for the Project Interventions Evaluation
Component
Focus Area
1.
2.
Relevance & quality

of design
Efficiency of
implementation
Sample Questions from Evaluation Guidelines

Did the project proposal conform to the goals of the funding agency
Upon what documentation was this project based?
Was the design appropriate for the geographic area?
Was the intervention logic coherent and accurate?
Were recommendations from previous evaluations incorporated in the design?
Were any lessons learned from previous pilot projects in the area?
Were the indicators of progress and of impact in the design of good quality?
How was the quality of the outputs going to be determined?
Were the outputs achievable or overly ambitious?
Were risks properly assessed?
Were the strategies employed under the different service delivery areas
appropriate for the various target populations (youth, PWDs, MARPs, PLHIVs
on ART, persons of the reproductive age, persons in the formal and informal
workplaces)?
To what extent did the strategies impact positively or negatively in the lives of
men, women, boys and girls?
How relevant were program approaches and strategies in contributing to
program quality and results?
To what extent was the program design appropriate and relevant to SD areas?
How realistic were the critical assumptions?
Data Collection Approach

Desk Review
Respondent
Proposal document,
Baseline and midterm evaluation
reports
Key Informant Interview
Project Management (Programme

manager, Government counterpart)
Did the project start on time?
Desk Review
Progress reports
Baseline and midterm evaluations
Project Management
Focus Group Discussion
Was a no cost extension applied for and granted?

Were all key staff in post within 6 months of start up? And
maintained through project life?
Were all inputs delivered on time?
Were inputs of acceptable quality?
Was the methodology of implementation the right one under the
circumstances?
Did the NGO/agency get good cooperation from relevant authorities?
59
Focus Area
Respondent
Desk Review
Facility Checklist

reports
Progress reports
Programme Manager
What was local governments assessment of this intervention?

What was the local leaders assessment of this intervention?
Did the NGO/agency get good cooperation from relevant local
leaders?
Was access to project areas acceptable?
Were most of the outputs achieved to an acceptable standard?
To what extent did the NGO/agency take on board recommendations
from funding agency field visits and feedback on progress reports?
Were the outputs delivered in a timely manner?
How successful was the performance based funding model
implemented?
How adequate were the resources based on the program targets?
What program efficiencies were employed and how effective were
they in:
- Enhancing program delivery
- SR management
- Program quality
- Program monitoring and reporting
To what extent did the PR utilize evidence for decision making?
What mechanisms did the PR use and how effective were they in
engaging program stakeholders.
To what extent did the PR comply with contractual obligations?
Did the program receive sufficient support from key players?
3.
Effectiveness
Did the activities listed in the proposal result in total achievement of the specific
objectives and attainment of outputs?
Were there any non-planned effects and were these good or bad?
Was coordination with other development actors effective?
Were the effects of the Project felt equally across the Project area or were some
60
Focus Area

areas neglected?
Were technical designs effective and appropriate for that environment?
Respondent
Local Leaders
Focus Group Discussions
To what extent did the program contribute to universal access to

HCT?
To what extent has the program targeted hard to reach populations
(key populations and vulnerable) with counseling and testing
services?
To what extent has the program targeted hard to reach districts with
counseling and testing services?
To what extent did the capacity building initiatives strengthen the
provision of HIV services?
To what extent were program outcomes [see performance
framework] achieved?
To what extent did the program contribute in addressing gender
barriers in HIV programming?
To what extent was gender integrated in the different service
delivery areas?
To what extent did the CCM oversight contribute to the program
delivery and results (from application to close out)?
To what extent was quality of services assured during implementation?
To what extent did the program comply with national or internal
program guidelines?
4.
Impact to date
Comparison to baseline
and midterm
To what extent were the program objectives achieved?

What effect has the program had on reducing new HIV infections?
What effect has the program had on the quality of life among PLHIV in Kenya?
To what extent have beneficiaries, including CBOs, benefited from the Project
activities and outputs? Has the Project changed their lives in any meaningful
way?
To what extent have local institutions benefited from the activities and outputs?
To what extent have local leaders benefited from the activities and outputs?
To what extent is the impact sustainable over the longer term?
Desk Review

report
Progress reports
Client Exit Interviews

Peer educators
Focus Group Discussions
Beneficiaries including PLHIV
61
Focus Area
5.
Potential
sustainability

Has the Project increased or decreased dependency on outside intervention?
To what extent did the program contribute to the KNASP objectives?
What are the intended and unintended, positive and negative, long term effects
achieved by the program?
To what extent has the program contributed towards client retention among
PLHIV on ART?
What were the results of gender integration in the program?
To what extent can the outputs be expected to be sustainable over the longer (510 years) term?
What characteristics make the outputs sustainable or unsustainable?
Do the local authorities fully support the initiatives taken by the Project?
Do the local leaders fully support the initiatives taken by the Project?
To what extent has the Project strengthened the capacities of local government
and local leadership structures?
To what extent are the people themselves contributing to the sustainability of
the initiatives?
To what extent has the private sector become involved in the development of
the area as a result of the Project?
Has a special effort been made to educate and train women to assume decisionmaking roles?
Did the NGO/agency formulate a practical exit strategy and is it working?
Respondent
Desk Review
Proposal document
Baseline and other evaluation reports
Progress reports

Project Management
Desk Review
Focus Group Discussion
What mechanisms are in place to enhance sustainability of the

program interventions (Youth groups, YRCS, SHCs, peer educators)?
6.
7.
Reporting
Observations on
donors role and
influence on Project
implementation.
What procedures and tools were used for Project monitoring and reporting?
How was data relayed from local to national levels?
Was monitoring and progress reporting adequate according to the funding
agency requirements?
Desk Review
Were communications with the Contracting Authority satisfactory in terms of

promptness and content?
Was technical / administrative support provided on time and adequately when
requested?
Were Grant Contract administrative procedures and actions timely taken care of
and did these influence Project implementation in any way?
Desk Review
Progress reports
Minutes of meetings
Relevant email correspondence
Project Management from national to

local levels
Key informant interviews
Proposal document
Baseline, other evaluation reports
Progress reports
Project Management
62
Focus Area
8.
Key observations,
overall conclusions,
actions
recommended for
future interventions
by whom and in
order of priority

What are the key lessons learnt?
What best practices were observed?
What are key recommendations for future interventions?

Desk Review
Respondent
Proposal document
reports
Progress reports
Key Informal Interviews
Project Management
63
Appendix 4: Summary of Questions, Methodology and Data Sources for the Organizational System Assessment
component
Processes
Questions
1.
2.
3.
Contracting &
Budgeting:
4.
5.
6.
Funds
Disbursements
& liquidations
1.
Understand the contractual process

between Global Fund, LFA and
CARE in Phase 1 and Phase 2.
Understand the period it took to get
to contracting between Global Fund
CARE. CARE and the Sub Recipient
in Phase 1 and Phase 2.
How did the contractual processes in
Phase 1 and Phase 2 impact on the
performance of the project in terms
of start-up
How did the contract and
addendums processes affect/impact
on performance?
How efficient were the addendum
process in regards to timeliness,
necessity and inclusivity (how
involved were the SRs in
determining the contracts)
Were the cost ceilings costs set for
program and support costs
reasonable. E.g 70/30 ratio, SDA unit
costs, etc
What was the frequency of
disbursements to the SR?
Information
Collection
Methodology
What is source / of
information? Specify
if at SR, PR,
Client/target group,
community, specific
stakeholder level.
Efficiency/
effectiveness
Sub Recipients, and

PR
Effectiveness
Sub Recipients, and

PR
Both
Data Collection Tool
Desk review of
contract and
addendum
documentation
GF_A1
GF_A2
GF_A1
One-on-one
interviews.
Desk review of
disbursements, SR
64
Processes
Questions
2.
3.
1.
Financial
reporting
2.
1.
2.
Operations &
Systems
3.
4.
How timely were the disbursements

and did they affect performance?
How did CAREs review and
approval processes affect SR
liquidations and disbursements?
How did each of the Financial

Reporting Tools e.g. budget tracking
tool, etc impact the performance?
How well did the Financial
Reporting requirements fit into the
organizations financial management
systems?
How did Financial Operations
Manual impact the organizations
control environment, and how did
this affect the performance?
Does your organization have
accounting software? If Yes Answer
Question 3 if not skip it
How did the composition of finance
function and their technical capacity
contribute to GF performance?
How well is the link/cooperation
between Finance and Programs
Functions established and how has
this affected the program
Information
Collection
Methodology
What is source / of
if at SR, PR,
community, specific
stakeholder level.
Efficiency/
effectiveness
budgets to SRs vis-vis program

performance
One-on-one
interviews with
partners, and
CARE staff.
Desk review
Structured
interviews.
Desk review
Finance reporting
tools
GF Round 7
Operating &
Accounting
manual
SR Finance files
GF_A2
GF_A1
GF_A2
GF_A1
GF_A2
Effectiveness
SR Operating &
Accounting
manual
Both
Structured
questionnaires
SR a/c software
65
Processes
Questions
1.
2.
funds
utilization/
absorption
3.
1.
Financial Risk
Management &
Audit
2.
3.
Technical
Support/
Capacity
Building
1.
performance?
What are the contributors of the
variances between the programmatic
achievement and financial burn rate?
What budget tracking mechanisms
do you have in your organization
and did this assist in achieving an
efficient budget tracking process?
Are there other external processes
such as increases in cost of living,
venturing into new program areas,
change of implementation strategy,
political/economic factors, etc that
have affected funds absorption
during implementation? What
strategies were employed?
How and to what extent has the
program mitigated financial risk and
how has this impacted the overall
risk management?
How efficient and cost effective was
the conversion of currencies?
What audit processes were in place
and to what extent was each value
adding?
How adequate was the technical
assistance, to what extent was the
uptake and how has these affected
the performance?
Information
Collection
Methodology
What is source / of
if at SR, PR,
community, specific
stakeholder level.
One-on-one
interviews
PR staff
SR
SR contract and
addendum
documents
Efficiency/
effectiveness
GF_A1
GF_A2
GF_A1
GF_A2
GF_A1
GF_A2
PR staff: Finance,
and management
One-on-one
interview
Risk Committee
Minutes
Effectiveness
Risk Register
Structured
questionnaires
SR staff
PR staff
Training reports
Effectiveness
66
Processes
Questions
2.
3.
Value for
money/ impact
(performance
based funding)
1.
1.
Close Out
2.
Information
Collection
Methodology
What is source / of
if at SR, PR,
community, specific
stakeholder level.
Efficiency/
effectiveness
Did your organization receive TA

from CARE? If Yes, what kind of
support was provided.
How will the SR sustain and
replicate the technical assistance to
other areas of the organization?
How effective were the cost
strategies in achieving program
results?
Did CARE send out close out
guidelines in a timely manner i.e.
three months before the close of the
program?
Did you receive support from CARE
on issues touching on close out to
ensure that there was a smooth
transition process? If Yes, What
support was provided?
Cost benefit
analysis
One-on-one
interviews
Program close out

guidelines
SR Budget
SR Expenditures
PR Budget
Other PR budgets
GF_A3
GF_A1
GF_A2
Both
Efficiency
67

Inception Report For The Gfr7 Ete Consultancy - 2 - May - 2014 - 2

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Inception Report For The Gfr7 Ete Consultancy - 2 - May - 2014 - 2

Uploaded by

Copyright:

Available Formats

CARE International in Kenya

Introduction and Background ................................................................................................... 5

About Global Fund ............................................................................................................... 6

Global Fund Round 7 Program ........................................................................................... 10

Our Understanding of the End Term Evaluation ................................................................. 11

Proposed Methodology of the Evaluation ............................................................................... 17

Study Population ................................................................................................................ 17

Data Collection Methods/Tools .......................................................................................... 25

Data Collection tools .......................................................................................................... 33

Data Management .................................................................................................................. 34

Data Quality Assurance ...................................................................................................... 34

Ethical Considerations ........................................................................................................ 34

Data Entry and Analysis ..................................................................................................... 35

Report Writing ................................................................................................................... 43

Operational Work Plan ........................................................................................................... 44

Assignment Technical and Managerial Structure .................................................................... 45

Team Composition ............................................................................................................. 45

Roles and Responsibilities .................................................................................................. 45

Appendix 1: Final Report Outline .................................................................................................... 49

Acquired Immunodeficiency Syndrome

Global Fund for AIDS, Tuberculosis and Malaria

1.0 Introduction and Background

About Global Fund

Round 2 - HIV and AIDS, TB and Malaria;

Round 7- HIV and AIDS; and

Round 10 HIV and AIDS

Table 1: Global Fund Grants Committed to Kenya as per April 2014

Approved Grant (USD)

Source: Global Fund Website

Global Fund Round 7 Program

To scale up and maintain PLHIVs on ART;

To increase access to HIV testing and counseling (HCT) services;

To increase uptake of HIV prevention and treatment services; and

Strengthening institutional capacity of HIV program implementers.

Our Understanding of the End Term Evaluation

End-term Evaluation Components

Project Interventions Evaluation

Organizational Systems Assessment

Institutional Capacity Evaluation

Gender and SRH Integration Review

Duration of the Assignment

Critical Assumptions for Effective Delivery

2.0 Proposed Methodology of the Evaluation

The evaluation will be undertaken in all regions of Kenya.

Sampling of sub recipients

Sampling of sub recipients was done in two phases

Rift Valley Region

North Eastern Province

Summary of Sampled SRs and Sites

The following is a list of all SRs sampled:

Sampling of SRs to participate in Organizational Systems Assessment and Institutional Capacity

Table 2: Sampled SRs for OSA and OCA

Catholic Diocese of Homa-bay

Action Aid Kenya

Family Health Options Kenya (FHOK)

Food for the Hungry International

Abha Light Foundation

Desired Sample size

Sampling of Individual Interviews (IDIs) with beneficiaries

Where, n=desired sample size if target population is greater than 10,000

Sampling of Key Informants

M&E Coordinator (GF and Health Sector)

Chief Executive Officer or Director