Independent Evaluation of Smallholder Agribusiness Development Initiative (SADI) MANAGEMENT RES !NSE " #EARNING and DISSEMINATI!

N #AN

Prepared by: Approved by: !rou"!: Cc: Date Approved:

Rural Productivity Unit Jacqui De Lacy, Minister Counsellor, AusAID Jakarta Petrarca #aret$i, Counsellor, Decentralisation and Poverty Reduction and Rural Develop%ent &ection, AusAID Jakarta Jere%y &trin"er, Unit Mana"er, Decentralisation and Rural Develop%ent, AusAID Jakarta Rani 'oer!ad!ie, &enior Pro"ra% Mana"er, Rural Develop%ent, AusAID Jakarta April ()**

Aid A$tivit% Summar%

Aid A$tivit% Name Aid+orks initiative nu%ber Co%%ence%ent date otal Australian 3 otal ot!er 3 Smallholder Agribusiness Development Initiative (SADI) I',(-. / AIPRD &%all!older A"ribusiness 0&ADI1 I'J()( / &%all!older A"ribusiness Develop%ent Initiative &epte%ber ())2 Co%pletion date -* July ()*)

A3-.45 %illion 6ro% AusAID and A3*47% 6ro% ACIAR i%e and ot!er in8kind inputs 6ro% ,overn%ent o6 Indonesia sta66 in Jakarta and 6our provinces, as 9ell as 6ro% participatin" lead 6ir%s, industry leaders and provincial "overn%ent sta66 &ub8pro"ra% * / Ministry o6 :o%e A66airs under +orld ;ank supervision &ub8pro"ra% ( / International <inance Corporation 0I<C1 &ub8pro"ra% - / Australian Centre 6or International A"ricultural Researc! 0ACIAR1 Indonesian a"encies at national level: 'ational Develop%ent Plannin" A"ency 0;APP='A&1, Ministry o6 :o%e A66airs / Directorate ,eneral o6 Peoples =%po9er%ent 0PMD8M>:A1, Co%%odity <utures radin" Re"ulatory A"ency 0Co< RA1? Indonesian Centre 6or A"riculture ec!nolo"y Assess%ent and Develop%ent 0;;P( P1? as 9ell as in 6our provinces 0+est 'usa en""ara, =ast 'usa en""ara, &out! &ula9esi, &out!east &ula9esi1 Indonesia / 6our eastern provinces 0+est 'usa en""ara, =ast 'usa en""ara, &out! &ula9esi, &out!east &ula9esi1 Rural Develop%ent

Delivery or"anisation0s1

I%ple%entin" Partner0s1

Country@Re"ion Pri%ary &ector

Aid A$tivit% !b&e$tive' !e overall "oal o6 &ADI 9as to ac!ieve a sustained increase in rural "ro9t! and !ouse!old inco%es t!rou"! productivity "ains, better access to %arkets, and on and o6686ar% value8added activities in . tar"et provinces o6 eastern Indonesia, i4e4 +est 'usa en""ara, =ast 'usa en""ara, &out! &ula9esi and &out!east &ula9esi4 !e purpose o6 &ADI 9as to de%onstrate, across a li%ited nu%ber o6 pilot co%%unities in t!e tar"et provinces, a %odel o6 i%proved !ouse!old8level production linkin" e66ectively 9it! i%proved a"ribusiness@&%all Mediu% =nterprise capacity, adequately serviced by decentralised and de%and8driven adaptive researc! capacity4

Independent =valuation Mana"e%ent Response, re"istered A*75 ;usiness Process >9ner: Director, Per6or%ance &yste%s and &upport

U'CLA&&I<I=D pa"e * o6 7 e%plate current to -) June ()**

&ADI 9as desi"ned 9it! t!ree sub8pro"ra%s: &ub8pro"ra% *: =n!anced &%all!older Production and Marketin" 0P'PM8AP1 0i%ple%ented by t!e Ministry o6 :o%e A66airs and overseen by t!e +orld ;ank1, 9!ic! seeks to pilot t!e incorporation o6 block "rant support 6or s%all!older a"ribusiness@liveli!ood activities into P'PM8Rural4 &ub8pro"ra% (: &tren"t!ened Private &ector A"ribusiness and &M= Develop%ent 0i%ple%ented by International <inance Corporation1, 9!ic! seeks to pro%ote s%all!oldersB access to %arkets and 6inance and address proble%s in t!e business environ%ent a66ectin" s%all!older a"ribusiness4 &ub8pro"ra% -: &upport to Market8Driven Adaptive Researc! 0i%ple%ented by Australian Centre 6or International A"ricultural Researc!1, 9!ic! seeks to c!an"e t!e 9ay in 9!ic! t!e a"ricultural researc! syste% 9orks, so t!at it is %ore broad based in nature and responsive to t!e require%ent o6 s%all!older a"ribusiness4

Independent Evaluation Summar%

Evaluation !b&e$tive' o evaluate t!e 9!ole8pro"ra% a"ainst ei"!t criteria: relevance, e66ectiveness, e66iciency, i%pact and sustainability as 9ell as %onitorin" and evaluation, "ender equality and analysis and learnin"4 Lessons learned 9ere structured to in6or% desi"n o6 6uture rural develop%ent in Indonesia4 !e overarc!in" per6or%ance questions used 6or t!e evaluation, and a su%%ary o6 t!e ICR 6indin"s 6or eac!, are as 6ollo9s: () To *hat e+tent has SADI a$hieved its end,of,program out$omes&ADI resulted in outputs t!at de%onstrate its capacity to be e66ective but t!e potential o6 t!e pro"ra% to ac!ieve its end8o68pro"ra% outco%es 9as under%ined by si"ni6icant structural, p!asin" and 6ra"%entation constraints caused by t!e lack o6 inte"ration bet9een t!e t!ree sub8pro"ra%s4 !e return on t!e &ADI invest%ent 9as t!ere6ore si"ni6icantly less t!an its potential4 &o%e o6 &ADIBs key ac!ieve%ents include: &o%e &ADI outputs ac!ieved t!e pro"ra% purpose o6 de%onstratin" %odels o6 i%proved productivity, 9!ic! provides a catalyst 6or scalin" up under ot!er pro"ra%s to !ave a liveli!oods and %arket i%pact at a re"ional scale4 <or instance, 9it! &ADI support C)) 6ar%ers in %ore t!an (7 villa"es adopted ;ali cattle %odels t!at increased productivity by up to 7)D4 ;y reducin" t!e calvin" interval 6ro% *5 to *( %ont!s, increasin" calvin" rates and reducin" cal6 %ortality !ouse!old inco%e "ains 6ro% t!is %odel li6ted adopters out o6 poverty4 !e avera"e annual enterprise inco%e increased 6ro% 5.D o6 t!e ' ; rural poverty line to *EED in *( %ont!s4 &ADI provided a space 6or ,overn%ent o6 Indonesia and lead 6ir%s to take risks and test ne9 ideas, 9!ic! is consistent 9it! &ADIBs purpose to de%onstrate productive %odels supported by an enablin" a"ribusiness environ%ent and adaptive researc!4 <or instance ,oI used &ADI to pilot a rural develop%ent@a"ribusiness co%ponent to t!e national P'PM pro"ra%4 &i%ilarly, 9it! leaders!ip 6ro% Ministry o6 rade 0Co< RA1 &ADI supported t!e "overn%ent to institutionalise 9are!ouse receipts as a %ec!anis% 6or i%provin" 6ar%er access to 6inance / an output o6 nation9ide bene6it and potentially a si"ni6icant avenue 6or linkin" rural 6ar% production to a national co%%odities eFc!an"e4 ;;P( P used &ADI to trial collaborative co%petitive researc! "rants to increase t!e e66iciency o6 researc! invest%ent in Indonesia4 ;y developin" capacity o6 ;;P( P and ;P P in &ADI provinces to better plan and eFecute adaptive researc!, a cultural c!an"e process 9as started t!at 9ill be sustained by ;;P( P institutionalisin" t!e use o6 collaborative co%petitive researc! "rants t!rou"!out its pro"ra% 6ro% ()**4 !ese c!an"es provide a 6oundation 6or e66ective leaders!ip o6 adaptive, %arket8led researc! by ;;P( P and its stake!older or"anisations, allo9in" "reater dependence on Indonesian researc! capacity in 6uture AusAID pro"ra%s4 &ADI started to tar"et causes o6 constraints to a"ribusiness enterprises, particularly in t!e area o6 access to %arkets 0A(M1, access to 6inance 0A(<1 and business enablin" environ%ent 0;==14 A(M activities de%onstrated t!e opportunities available 6ro% reducin" t!e co%pleFity o6 t!e supply c!ain by supportin" 6ar%ers and traders or processors to en"a"e4 <or eFa%ple linkin" cocoa "ro9ers in &out! &ula9esi 9it! Ar%a$aro reduced t!e need 6or local collectors, 9!ic! increased 6ar%8"ate prices by ()D and "ave Ar%a$aro access to traceable supply at no eFtra cost4 &i%ilarly, by supportin" 6ar%ers to sta""er plantin" o6 peanuts in ' ;, ,aruda <oods secured its supply c!ain and 6ar%ers stabilised

Independent =valuation Mana"e%ent Response, re"istered A*75 ;usiness Process >9ner: Director, Per6or%ance &yste%s and &upport

U'CLA&&I<I=D pa"e ( o6 7 e%plate current to -) June ()**

!i"!er prices, 9it! 6ar%8"ate prices on avera"e (7D !i"!er t!an %arket nor%s durin" peak !arvest season4 &ADI enterprise activities de%onstrated positive i%pact on !ouse!old inco%e, 9!ic! contributed to ac!ieve%ent o6 MD, * 0poverty and !un"er1 and MD,- 0"ender equality and e%po9er 9o%en14 <or instance, %ore t!an C7)) 6ar%ers 9ere i%ple%entin" en!anced peanut productivity %odel 9it! ne9 varieties 6or i%proved quality, ne9 6ar%in" practice to increase productivity and sustainability? and sta""ered plantin"s to increase 6ar%8"ate prices, 6ar%erBs increased !ouse!old inco%e by %ore t!an -2D4 >verall *-,C.E %en and *(,..7 9o%en participated in en!anced s%all!older production and %arketin"4 Activities includin" %icro8enterprises in ' ;, pi" production in &out! &ula9esi, sea9eed and cas!e9 processin" in &out!east &ula9esi, and production o6 co66ee po9der in ' 4 Co%%unities 9ere supported to en!ance "ender equity, includin" leaders!ip trainin" 6or local in6or%al 9o%en leaders4

.) /hat lessons $ould be applied to the design of a future rural development program- In parti$ular are there an% lessons relating to management arrangements&o%e o6 t!e lessons learned t!at 9ere derived 6ro% t!e report, and t!at 9ill be considered in a 6uture rural develop%ent pro"ra% desi"n, as 6ollo9s: !e structure o6 &ADI, 9it! t!ree separate sub8pro"ra%s %ana"ed by t!ree independent or"anisations, 9as not e66iciently coordinated and inte"rated4 It is reco%%ended t!at lar"e and relatively co%pleF rural develop%ent pro"ra% s!ould !ave a sin"le %ana"e%ent structure 9it! co!erent %ana"e%ent roles and responsibilities establis!ed t!rou"! consistent contractin" arran"e%ents4 &ADI lacked an overall pro"ra% lo"ic 9!ere t!e absence o6 sin"le "oal and purpose s!ared by all sub8 pro"ra%s create di66iculties in sub8pro"ra% inte"ration4 o avoid 6ra"%entation, any 6uture rural develop%ent pro"ra% s!ould !ave one pro"ra% "oal and purpose 9it! a sin"le pro"ra% lo"ic t!at de6ines clear end8o68pro"ra% outco%es4 In ac!ievin" a clear purpose and "oal, 6leFible delivery is required4 Any ne9 pro"ra% 9ill need to en"a"e 9it! a 9ide ran"e o6 a"riculture stake!olders ran"in" 6ro% very poor subsistence 6ar%ers to success6ul a"ribusiness entrepreneurs4 Pro"ra% s!ould develop 9it! stron" ,oI o9ners!ip4 P!ase delivery o6 %arket and enterprise led activities t!rou"! identi6ication o6 6ar%in" syste%s and inte"rated supply c!ains, en"a"e%ent 9it! tar"et co%%unities to understand needs and lastly to en"a"e 9it! institutional stake!olders to identi6y any capacity develop%ent and adaptive researc! needs4 Use t!e annual plannin" process to incorporate lessons learned 6ro% t!e per6or%ance %onitorin" to %ake any ad$ust%ents needed to ensure pro"ra% e66iciency and e66ectiveness4 Proactively supervise and intervene 9!en necessary durin" pro"ra% i%ple%entation to ensure t!at pro"ra% ob$ectives are %et4 Roles and responsibilities s!ould be articulated in t!e desi"n to ensure appropriate resources are allocated to enable e66icient and e66ective use o6 Australian 6und, as required by t!e <MA Act4 !is is to respond t!e 9eakness o6 &ADIBs %onitorin" and evaluation syste% 9!ere it 9as only %onitored t!e quantitative inputs, activities and output but did not !ave i%pact %onitorin" or assess%ent o6 t!e bene6iciaries4 !ere 9as an opportunity to en"a"e t!e ,overn%ent o6 Indonesia %ore actively includin" to stren"t!en institutions and to use t!e ,oI syste%4 ,oI en"a"e%ent in t!e pro"ra% 9ill increase its relevance and sustainability4 Use output8to8purpose %onitorin" to track per6or%ance %ore re"ularly4 =66ective %ana"e%ent o6 co%pleF pro"ra%s requires re"ular %ana"e%ent %onitorin" o6 variance 6ro% planned inputs, activities and outputs? se%i8annual per6or%ance %onitorin" at 9!ole8o68pro"ra% level usin" output8to8purpose %onitorin" a"ainst a sin"le pro"ra% results 6ra%e9ork and periodic evaluations consistent 9it! ,oI syste%s and t!e AusAID quality reportin" syste%4 >utput8to8purpose %onitorin" in6or%s annual plannin" and provides a basis 6or se%i8annual, $oint8supervision by ,oI and AusAID4

0) Is SADI1s e+it strateg% ade2uate!e uneFpected closure o6 &ADI, poor co%%unication o6 t!e decision and t!e eFpectations raised at sub8 district, district and provincial levels t!rou"! &ADI raise reputational and so%e develop%ent risks 6or AusAID and its stake!olders4

Independent =valuation Mana"e%ent Response, re"istered A*75 ;usiness Process >9ner: Director, Per6or%ance &yste%s and &upport

U'CLA&&I<I=D pa"e - o6 7 e%plate current to -) June ()**

!e ICR identi6ied t!at success6ul a"ribusiness activity i%ple%ented under &ub8pro"ra% ( are likely to be sustained4 AusAID !ad entered into an a"ree%ent 9it! International <inance Corporation to continue so%e o6 &ub8pro"ra% ( activities t!rou"! Pro"ra% 6or =astern Indonesia &M= Assistance II 0P='&A8II1 9!ic! is a %ulti8 donor partners!ip %ana"ed by International <inance Corporation 0I<C1, t!e private sector ar% o6 +orld ;ank4 P='&A8II is ai%ed at creatin" opportunities and i%provin" peoplesB lives in Indonesia t!rou"! sustainable private sector develop%ent, 6ocusin" on t!e ;usiness =nablin" =nviron%ent, Access to <inance, A"ribusiness Linka"es, &ustainable <orestry, and In6rastructure4 <or &ub8pro"ra% - activities, AusAID !ad provided additional 6undin" to t!e Australian Centre 6or International A"ricultural Researc! to sustain adaptive researc! activities 9it! t!e Indonesian Institute 6or A"ricultural ec!nolo"y Assess%ent and Develop%ent 0;;P( P1 and Institute 6or Assess%ent o6 A"ricultural ec!nolo"ies 0;P P14 !ere are, !o9ever, no eFit strate"ies a"reed to 6or &ub8 pro"ra% * activities4 !e ICR reco%%ended t!at t!e institutional arran"e%ents already available t!rou"! P'PM provide a basis 6or e66iciently deliverin" a transitional support 6or &ub8Pro"ra% * until t!e ne9 pro"ra% is i%ple%ented4 Evaluation 3ompletion Date' <inal report sub%itted on *7 July ()*) Evaluation Team' Jo!n <ar"!er 0 ea% Leader 8 A"ricultural &cientist1? Mr Gasuo #onis!i 0 ea% Me%ber 8 A"ribusiness &pecialist1? and Mr =rinc! &a!an 0AusAID Policy >66icer14

Management Response of I3R re$ommendations

!e &ADI Independent Co%pletion Report is 9ell structured, easy to read, !as a "ood %et!odolo"y and contains so%e very use6ul case study in6or%ation and analysis4 !e conclusions presented are clear and based on solid analysis4 !e report also identi6ies constraints and !o9 t!ey could be overco%e in 6uture pro"ra%s4 !e ICR tea% %et 9it! relevant stake!olders and t!e report !as very 9ell addressed t!e topics and questions outlined in t!e er%s o6 Re6erence4 !ou"! t!e report did not provide a clearer analysis o6 eac! subpro"ra% 9it!in eac! o6 t!e ratin" cate"ories, it assessed t!e pro"ra% as one inte"rated pro"ra% and co%bined t!e analysis o6 t!e - subpro"ra%s4 AusAID considers t!e ICR reco%%endations as use6ul 6or t!e desi"n o6 a ne9 rural develop%ent pro"ra%4

Re$ommendation !ne
Re$ommendation' Provide %odest support 6or transition4 =66icient inception o6 a ne9 pro"ra% 9ill build on t!e relations!ips establis!ed by &ADI and ot!er pro"ra%s in eastern Indonesia4 !e uneFpected closure o6 &ADI . years into a *)8year pro"ra%, t!e poor co%%unication o6 t!e decision and t!e eFpectations raised at sub8district, district and provincial levels t!rou"! &ADI raise reputational and so%e develop%ent risks 6or AusAID and its stake!olders4 !e ICR identi6ied t!at success6ul a"ribusiness activities i%ple%ented under &P( are likely to be sustained4 ACIAR !as already obtained additional 6unds and !as its o9n Indonesia pro"ra% to sustain adaptive researc! activities 9it! ;;P( P and ;P Ps4 Activities and relations!ips initiated &P* 9ill 6or% an i%portant entry point in t!ose districts tar"eted 6or any ne9 pro"ra% but are vulnerable 9it!out so%e transitional support4 !e institutional arran"e%ents already available t!rou"! P'PM provide a basis 6or e66iciently deliverin" t!is transitional support4 !e a%ount o6 resources needed is likely to be about t!e sa%e as t!e unspent 6unds to be returned to AusAID by t!e +orld ;ank4 It is t!ere6ore reco%%ended t!at AusAID provide %odest support t!rou"! P'PM to %aintain success6ul activities and relations!ips in t!ose provinces and districts t!at are relevant to any ne9 pro"ra%4 !ese transition arran"e%ents could be %ana"ed by t!e eFistin" &ADI provincial coordinators, reportin" directly to AusAID Jakarta, 6or t!e duration o6 t!e transition4 ,iven lessons learned 6ro% &ADI, t!is transitional support s!ould be inte"rated and %ana"ed by one sin"le or"anisation4 ,iven t!e late start o6 &P*, t!ere is an opportunity to use so%e transitional resources to ensure t!at activities recently aut!orised and i%ple%ented continue to receive t!e required support durin" t!e transition period to !elp %ini%ise reputational risks in districts 9!ere a ne9 pro"ra% %ay operate4 Response' A"ree4

Re$ommendation T*o
Re$ommendation' Use ICR lessons and eFperience to support desi"n process4 =valuation adds value to invest%ents by in6or%in" desi"n and appraisal4 !is is inte"ral to t!e AusAID quality syste%4 !e ICR tea% !as t!e privile"ed position o6 !avin" collected 6ield evidence, and revie9ed &ADI i%ple%entation to in6or% an analysis o6 t!e di66erence &ADI %ade and lessons learned 6ro% its
Independent =valuation Mana"e%ent Response, re"istered A*75 ;usiness Process >9ner: Director, Per6or%ance &yste%s and &upport U'CLA&&I<I=D pa"e . o6 7 e%plate current to -) June ()**

i%ple%entation4 !ose lessons are relevant to peer revie9 and appraisal o6 t!e desi"n and it is reco%%ended t!at t!e ICR report and t!e tea% t!at produced it be used by AusAID Jakarta as a resource to ensure quality at entry 6or t!e proposed ne9 pro"ra%4 Response' A"ree4

Re$ommendation Three
Re$ommendation' &tren"t!en contractual instru%ents 6or use 9it! %ultilaterals4 AusAID needs contractual instru%ents 6or use 9it! %ultilaterals en"a"ed as i%ple%entation service providers to "ive AusAID control and aut!ority 9!ere needed and clear capacity to instruct, cancel activities and retrieve %oneys4 In t!is 9ay AusAID 9ill !ave stren"t!ened capacity to in6luence t!rou"! supervision4 Response' A"ree4

Distribution List:
Sta4eholder AusAID Jakarta ;ranc!, %ainly t!e senior %ana"e%ent, Decentralisation ;y e8%ail and Poverty Reduction section, IRP=, &ection and PHI Unit Relevant AusAID Canberra sta66 0>D=, Rural Develop%ent !e%atic ,roup, Indonesia Desk1 &ADI partners 0+;, I<C, and ACIAR1 'ational ,overn%ent Counterparts: ;APP='A&, M>:A8PMD, ;;P( P Provincial ,overn%ent counterparts: ;APP=DAs and ;P Ps o6 =ast 'usa en""ara, +est 'usa en""ara, &out! &ula9esi and &out!east &ula9esi provinces +>, 0D<A , DA<<, AU& RAD=1 ;y e8%ail ;y e8%ail Provide !ard copy o6 t!e <inal Report Provide !ard copy o6 t!e translated eFecutive su%%ary o6 t!e <inal Report Provide 6inal report by e8%ail Distribution Method

Independent =valuation Mana"e%ent Response, re"istered A*75 ;usiness Process >9ner: Director, Per6or%ance &yste%s and &upport

U'CLA&&I<I=D pa"e 7 o6 7 e%plate current to -) June ()**