Developing Poverty Targeting and

Social Impact Monitoring Tools for

the Opportunity Network

Brian Beard - Opportunity International

SEEP - PAWG Meeting

May 6, 2004
Background
 Opportunity International Network:
 40 Implementing Partner MFIs in 25
countries
 487,105 clients as of December 31, 2003
 87% women
 Average loan size = $226/ $128 outside EE
 Operational Sustainability = 106%
 

  
  



 


 
 i



Vimitation as an Impact Tool
 Because a monitoring system must be kept
simple and cost-effective, its primary value
is as a management tool ± serves to ³raise
the red flags´ and to let us know which
areas need further research
Does NOT replace:
 Qualitative impact research
 Issue-specific impact research
Poverty Measurement - CIMS
 4 indicators for Poverty Index ± to be
tested/validated by comparing national indicators
     

 Business Income as a proxy for Household Income
 Household income/ poverty line or <$1/day, <$2/day
 HH Income per capita or per # of dependants
 D

 
 Own house/ rental/ family compound/other
 D
  
 
  
 Concrete/block; Wood; Sheet Metal; Mud Brick; Other
 — 


 ! 
 Water/electricity/toilet/telephone/cell phone
Relative weights still to be determined
  "
 Collaborating with CGAP and PVP to test CIMS
poverty indicators in 4 countries.
 In Colombia, the Philippines, Ghana and Albania,
300 clients and non-clients are being interviewed
with a combined tool: CIMS along with the
expenditure and consumption component of the
VSMS or national government survey for
assessing poverty.
 Use data to adjust poverty assessment component
of CIMS, as necessary for each country
(weighting and/or indicators²i.e., EE housing)
  "#  
$
 Implement certified tool, and further test the
validity of the poverty assessment indicators in
other countries as CIMS is implemented
throughout our Network.
 In addition to testing relative weights and
finalizing a numerical poverty scale, will also
compare our indicators with standard poverty
metrics available from MIS (avg. loan size,
median loan size, avg. outstanding loan balance,
etc.)
à

à 
What were the primary reasons behind the
organization choosing to develop and/or use the
tool? For poverty targeting purposes, for
measurement of outreach goals, in order to better
track impact over time, a donor requirement,
market research, internal mission and vision, etc.?
u
 

 





 













 a


 u
 
 



 













 


 

 



 
 à

 ?








 







 
 






















 
?




 




!






 








"

 a
 

 ?


 

#


 



























 ?
  













$% &





!






What type of poverty distinctions are possible
with the tool? Does it only distinguish
between poor and not-poor, or is it able to
help distinguish between very poor, poor,
and not-poor, etc.?

 STAY TUNED«..
àà





 lease provide an estimate of the time
and cost requirements for:
 Staff to implement the tool
 Management to adapt, oversee, and use
data from the tool
 Clients time during implementation
 Estimated Cost w/ 6,800 clients
(
' $ ! 
  $)
 $  
%
,a(   
' './ 
  * a&
' 
-
 .
/00    +
'a,( (
a

a
 
      
a
 
 
 


 _

 
 _   _  _  _  _  _ 



  


 
 
 

!  "    

#  
  
! _ _  _
 _
 _
_"

# _ _  _
_
_ _ 
àf the tool is not administered as part of
routine operations, what is the cost (to the
institution and client) of applying the tool?
Does this significantly increase the cost
(to the organization and the client) of
implementation?
  


 

 





? 





 








Dave you had any challenges finding staff and
assigning a particular staff member with the
right skills to manage, coordinate and
implement the whole process? This refers to
local staff in the organization, not from the
supporting international NGO experts. (Some
WG participants have noted that they have
observed a real shortage of qualified staff at the
local NGO level and find it very hard to spare
some of the qualified staff because they are
literally drawn into every task that is urgent at
the time.)
 A

 
 





!











 
 








 
'

 Vydia Opoku in Ghana
 Mitzi Machado in Colombia











 
(
  
 

 
 

  









!



'

 )
 

 


*






 








 


 

+

 





ààà

 
 

  

-Frequency: Dow often do you carry out data
gathering and what is the length or
duration of one cycle?



 











 ?*,

-








 %









 
 














-Who analyses the data you collect?:
Describe this process and its complexity.
Dave you had to create or adapt an MàS
to deal with this data?
 Currently, by Brian
 Build local capacity
 Framework for Quarterly Reporting in Excel
format with up to 8 graphs per tab
 Template in Epi Info while new software is
being developed.
-Feedback: Dow do you give feedback to the
clients and staff? Dow are they involved
beyond the data gathering?
 +





 .





 
 



 /









-Decision-making: Dow do you integrate the data or
results into decision making? Who takes the
responsibility? What role does the management
play and how are they involved (realistically since
they are busy with day to day operations and
other strategic decisions)?
 Key indices on impact, outreach, satisfaction and
exit rate given equal consideration as portfolio
data at management meetings
 Quarterly Report àncentive
 More in-depth analysis by Network²red flags
à

  

 
    
When you administer the tool, is it done
during a routine visit with the client, either
at her home or when she would routinely
come to the office? àf so, might this timing
affect her answers? Dow so?
 After loan disbursement
 Same location as group meeting
 Extra travel time will undermine efficiency
às your tool susceptible to misreporting by
respondents (e.g., clients might give an
answer they think you want to hear ± in
order to get more funding or services ±
rather than the truthful answer)? lease
provide details on how it is susceptible to
this risk.
 A
*










!


 Vearning from Clients
 V.O. can sometimes tell if client is lying
 Interviewing skill and rapport with client
essential
What does your organization do to overcome a
risk of misreporting? What incentives are in
place to encourage clients to provide accurate
data?
 ?



!

 

 

 
 

/u
# 
 
 


0



 

12

 




What incentives are in place to encourage accurate
collection of data? re those collecting the data
involved with adapting the tool or do they
understand how to effectively use the data?
Dave you made any overt strides towards
achieving µbuy-in¶ of either line staff or
management in implementing this tool? àf so, do
you believe this has helped in using the tool
more effectively or efficiently? àf not, why not?
 

12

 




 3



  


 Experience shows clients willing to give negative
feedback, except about loan officer
Dave you put in place any mechanisms to
double-check the accuracy of the data
collected? For example, do any senior
staff spot check how the data is recorded
or reported?
 As mentioned previously, there will be
several layers of QC, including occasional
spot checks by senior management to look for
inconsistencies.


   
àf your tool is being implemented in more than
one country, please provide an overview of
how you adapted it between countries, and
the time and cost of doing this.
 dapted for Eastern Europe because
responses for housing ownership, materials
and utilities questions don¶t apply there;
 lso, in terms of impact, adapted in frica to
find out what is impact of Dà /àDS
awareness, prevention, testing training given
during weekly meetings;
 Most adaptation done at regional trainings, but
testing of new questions will incur a cost.
Did you make any adjustments to your tool
since you began to use it to make it
cheaper or easier to use? lease provide
details of why and how this was done.
 
"











.



 3





  
  






1

às your tool specific to a microfinance setting

or could it be implemented in any type of
development program?
 Majority of questions could either be used, as is,
or adapted to another type of development
program.
 Poverty level indicators
 Impact, if another program wants the same kinds of
impact
 Exit rate
 Satisfaction questions could be easily adapted
Points to Consider
 Institutionalize the Process ± Buy-in
 Cost-Benefit considerations for
methodology (sampling, interviewers, etc.)
 Standardize vs. Customize
 Minimizing bias vs. maximizing learning
 Quality Control and Incentives
 Vimited effectiveness of measuring
community and spiritual impact
 KISS (time, ease of analysis, utility of data)