doi: 10.

1093/heapol/czh001 HEALTH POLICY AND PLANNING; 19(1): 110

Health Policy and Planning 19(1), Oxford University Press, 2004; all rights reserved.

The effect of Integrated Management of Childhood Illness on observed quality of care of under-ves in rural Tanzania
TANZANIA IMCI MULTI-COUNTRY EVALUATION HEALTH FACILITY SURVEY STUDY GROUP*

Integrated Management of Childhood Illness (IMCI) has been adopted by over 80 countries as a strategy for reducing child mortality and improving child health and development. It includes complementary interventions designed to address the major causes of child mortality at community, health facility, and health system levels. The Multi-Country Evaluation of IMCI Effectiveness, Cost and Impact (IMCI-MCE) is a global evaluation to determine the impact of IMCI on health outcomes and its cost-effectiveness. The MCE is coordinated by the Department of Child and Adolescent Health and Development of the World Health Organization. MCE studies are under way in Bangladesh, Brazil, Peru, Tanzania and Uganda. In Tanzania, the IMCI-MCE study uses a non-randomized observational design comparing four neighbouring districts, two of which have been implementing IMCI in conjunction with evidence-based planning and expenditure mapping at district level since 1997, and two of which began IMCI implementation in 2002. In these four districts, child health and child survival are documented at household level through cross-sectional, beforeand-after surveys and through longitudinal demographic surveillance respectively. Here we present results of a survey conducted in August 2000 in stratied random samples of government health facilities to compare the quality of case-management and health systems support in IMCI and comparison districts. The results indicate that children in IMCI districts received better care than children in comparison districts: their health problems were more thoroughly assessed, they were more likely to be diagnosed and treated correctly as determined through a gold-standard re-examination, and the caretakers of the children were more likely to receive appropriate counselling and reported higher levels of knowledge about how to care for their sick children. There were few differences between IMCI and comparison districts in the level of health system support for child health services at facility level. This study suggests that IMCI, in the presence of a decentralized health system with practical health system planning tools, is feasible for implementation in resourcepoor countries and can lead to rapid gains in the quality of case-management. IMCI is therefore likely to lead to rapid gains in child survival, health and development if adequate coverage levels can be achieved and maintained. Key words: IMCI, child health, policy implementation, Tanzania

Introduction
The Integrated Management of Childhood Illness (IMCI) is a strategy developed by the World Health Organization (WHO), the United Nations Childrens Fund (UNICEF) and other technical partners to address major child health problems in the developing world (Gove 1997). Each year almost 11 million children in low-and middle-income countries die before they reach their fth birthday (Ahmed et al. 2000). Five in 10 of these deaths are due to just ve conditions: malaria, pneumonia, diarrhoea, measles and HIV, often in combination (WHO 2000). Malnutrition contributes to over 60% of these deaths (Black et al. 2002). IMCI seeks to address these problems through three components improved case-management, improved health systems support and improved family and community practices. By December 2001, IMCI was in the early

implementation or expansion phase in 82 developing countries, including the majority of African countries south of the Sahara (WHO 2001). More information is available at www.who.int/child-adolescent-health. With a Gross Domestic Product of US$501 per capita (UNDP 2001) and under-ve mortality of 167 per 1000 (probability of dying before reaching the age of 5 years) (National Bureau of Statistics and Macro International Inc. 1999), Tanzania is undergoing health sector and local government reforms. As districts gain more control over their health budgets, IMCI is one of the strategies recommended by the Ministry of Health (MoH) to address major child health problems. Based on ongoing sentinel demographic surveillance results from the study area, these problems in order of importance are malaria, pneumonia, malnutrition and diarrhoea, which together account for over 83% of

* Joanna Armstrong Schellenberg,1,2,3 Jennifer Bryce,4 Don de Savigny,5,6 Thierry Lambrechts,4 Conrad Mbuya,6 Leslie Mgalula,6,7 and Katarzyna Wilczynska1
1Ifakara Health Research and Development Centre, Ifakara, Tanzania,

School of Hygiene and Tropical Medicine, London, UK, Tropical Institute, Basel, Switzerland, 4WHO Headquarters, Geneva, Switzerland, 5International Development Research Centre, Ottawa, Canada, 6Tanzania Essential Health Interventions Project, Ministry of Health, Dar es Salaam, Tanzania and 7WHO Tanzania
3Swiss

2London

Tanzania IMCI multi-country evaluation study group two of which began implementing IMCI in 1997 (Morogoro Rural and Ruji) and two of which began implementation in 2002 (Kilombero and Ulanga). The District Health Management Teams (DHMTs) decided to adopt IMCI, and to give highest priority to its introduction and implementation, based on evidence available to them from a sentinel burdenof-disease information tool and a district health budget mapping tool developed by the Tanzania Essential Health Interventions Project (TEHIP) (De Savigny et al. 2002). In addition TEHIP provided nancial resources to districts of approximately US$0.92 per capita per year to simulate sector-wide basket funding (Cassels 1997) 3 years in advance of the actual start of basket funding. In the two IMCI implementation districts, DHMTs reported that over 80% of health workers managing children in rst-level facilities had been trained in IMCI by mid-2000, based on an 11day course with approximately 30% of the training time spent in clinical practice. Insecticide-treated nets for the prevention of malaria were available through social marketing in both the intervention and comparison districts. Child health and survival are being monitored in these four districts through before-and-after household surveys and demographic surveillance. As part of this study we carried out a health facility survey in 2000 to compare quality of casemanagement and health systems support indicators in the two districts with IMCI and the two comparison districts that had not yet implemented IMCI. This survey reects process, rather than outcome, indicators related to IMCI. Here we present information on health facility support and on quality of care for children aged 2 months to 4 years from a survey of government health facilities in the four districts.

post-perinatal under-ve deaths (TEHIP 2000). Nationally, per capita expenditure on health was US$11.37 in the year 19992000, including private, out-of-pocket expenses (Ministry of Health 2001). Previous evaluations of child health services delivered by public health facilities in the study area have documented that the quality of care was inadequate prior to the introduction of IMCI. For example, a 1991 evaluation of primary health care services reported poor health worker performance and low community satisfaction (Gilson et al. 1993). Other more recent studies have documented inadequate case-management for malaria: in one study less than half of sick children were physically examined (Nsimba et al. 2002), and in another, 30% of slide-conrmed cases left the facility without antimalarial treatment and 20% of those who were treated for malaria were either under- or over-dosed (Font et al. 2001). The Ministry of Health in Tanzania started implementing IMCI in 1996. As part of health sector and local government reforms, local councils have increased autonomy and control over their own health budgets and plans, and a limited amount of donor-supported basket funding from the health Sector-Wide Approach (SWAp) (Cassels 1997) is available for them to implement these plans.1 (The term council refers to the local government of both rural districts and urban municipalities.) At least 42 of the 117 councils had chosen to introduce IMCI by July 2002. The Ministry of Health in Tanzania had previously adapted the generic IMCI case management guidelines to reect national child health policies (e.g. rst- and second-line treatments for malaria and pneumonia) and local terms for illness symptoms and providers. Operational research was conducted as the basis for developing an IMCI nutrition and counselling card for use by health workers in educating mothers. All materials were translated into Swahili and used as the basis for preparing national and district level trainers. The target audience for the 11-day training was all health workers in rst-level health facilities who provide case-management to children. The majority of these health workers have a 2 or 3 year training in clinical medicine following primary (and sometimes 46 years of secondary) education, although around a quarter have public health training following primary education only. The Multi-Country Evaluation of IMCI (IMCI-MCE) seeks to generate information on the effectiveness, cost and impact of IMCI that can be used to strengthen the delivery of child health interventions and the implementation of the IMCI strategy. Evaluations are ongoing in Bangladesh, Brazil, Peru, Tanzania and Uganda. IMCI-MCE is coordinated by the Department of Child and Adolescent Health and Development of the World Health Organization (WHO/CAH); more information is available at www.who. int/imci-mce. A similar impact evaluation of IMCI is also under way in Benin, in collaboration with the US Centers for Disease Control (Rowe et al. 2001). In Tanzania, the IMCI-MCE study uses a non-randomized observational design comparing four neighbouring districts,

Methods
The survey was carried out in August 2000 in Ruji, Morogoro Rural, Kilombero and Ulanga districts of southeastern Tanzania (see Figure 1), using an early version of the health facility survey developed by WHO/CAH (WHO 2002) and modied to be compatible with the Tanzania adaptation of the IMCI clinical guidelines for rst-level facilities and the MCE requirements. Through observation of case-management, exit interviews with caretakers, re-examination of each child and interviews with health care providers we aimed to collect information on 29 indicators relating to assessment, classication and treatment of the child, counselling and communication with the mother and health systems support (Table 1).2 Three indices were also constructed. One index assessed the availability of eight essential oral treatments through an additive score ranging from 0 (none available) to 1 (all eight available). The essential oral treatments included in this index were oral rehydration solution (ORS), rst-line antibiotics for pneumonia and dysentery, the rst-line antimalarial, vitamin A, iron, mebendazole and either paracetamol or aspirin. The second index assessed the availability of the four vaccines in the Tanzania Expanded Programme on Immunization (BCG, polio, DPT and measles), and ranged from 0 (no vaccines available) to 1 (all essential vaccines available). The third index measured the availability of four pre-referral treatments for severely-ill children.

IMCI in Tanzania

150

300

kilometres

Morogoro Rural
Kilombero

Rufiji

gold standard clinician, facility support and health worker interview, were developed and pre-tested by the IMCI-MCE, translated and back-translated into Swahili and adapted to make them consistent with the Tanzanian adaptation of the IMCI guidelines for rst-level facilities. A full eld test of survey instruments was carried out during the training of the survey teams. A generic English version of the questionnaires is available at www.who.int/imci-mce. Observer reliability for the case-management observation questionnaire was checked at the end of the training period: median intra-observer reliability was 90% (range 75100%) and inter-observer reliability was 88%. All eld work was carried out in August 2000. The survey teams spent one day at each facility: all teams collected data in all four districts. Facility staff were not given advance warning of which facilities would be visited or the exact dates of any visit. Data collection started in a comparison district (Kilombero), moved to the two IMCI districts (Morogoro Rural and Ruji) and nished in a comparison district (Ulanga). Each team consisted of four health workers; observation of case management, exit interviews and re-examination were carried out by team members trained in IMCI. All forms were checked by the team leader on the day of data collection and further checked by the survey co-ordinator each week. The team leader or the survey co-ordinator carried out one or two accompanied interviews of each type each day.

Ulanga

Figure 1. IMCI-MCE intervention and comparison districts, Tanzania. IMCI intervention districts are labelled in bold: comparison districts in italics

A random sample of 20 facilities was selected in each of Morogoro Rural and Ruji districts from government health facilities at dispensary and health-centre levels providing outpatient care for children under ve, of which there are a total of 65 in Morogoro Rural and 49 in Ruji. In Kilombero and Ulanga districts, where there are only 16 and 19 such facilities respectively, all government dispensaries and all three of the government health centres were included. In Morogoro Rural and Ruji the sample was stratied on geographical area and on facility type (health centre/dispensary), with three health centres being chosen from each district. Only government health facilities were included in the survey. Within chosen health facilities, the rst six sick children aged 2 to 59 months attending on the day of the survey for an initial visit for any illness, and whose mother consented to take part, were eligible for inclusion in the case-management observation part of the survey. Where there was more than one health worker managing children, management of the rst child was observed. When the rst childs case management was complete and the child was being re-examined, the observer returned to the entrance of the facility and accompanied the next child to be seen by any health worker. This was repeated until the management of six children was observed. Exit interviews were conducted with the caretaker of each child for whom case management was observed. All caretakers were assured of the complete condentiality of their responses. Six linked questionnaires, covering enrolment, observation of case-management, exit interview, re-examination by a

Data processing and analysis One FoxPro (Microsoft, Seattle, USA) database le was created for each questionnaire. Double data entry was carried out by two different data entry clerks, after which the two versions were compared and any discrepancies resolved with reference to the original forms. Standard range and consistency checks were carried out, including linking each database to the other forms relating to the same facility and/or the same child. A standard MCE analytical plan was adapted (available from authors on request). Data analysis was carried out using STATA version 5. Logistic or ordinary regression methods were used to compare the IMCI and comparison districts, stratifying on facility type (dispensary or health centre). For all indicators relating to individual children, rather than health facilities, the regression models were adjusted for the clustered nature of the data using STATA svy commands, which results in larger condence intervals and less extreme p-values than would be found for unclustered data. For case-management indicators, the analysis was repeated with weighting to make the sample of children observed representative of a typical days attendance in the facility.3 Weighted and unweighted results did not differ in any substantive way, and the unweighted results are presented here for ease of interpretation. Weighted results are available from the authors on request.

Tanzania IMCI multi-country evaluation study group

Table 1. Indicators of health worker skills, health systems support, and assessment and referral of very sick children Category Health workers skills Assessment of sick children Indicators

Correct classication Correct treatment

Counselling and communication

Checked for presence of cough, diarrhoea and fever Weight checked against growth chart Vaccination status checked Assessed for feeding practices if under 2 years Checked for other problems Correctly classied Correctly classied omitting coughs, colds, no dehydration Pneumonia correctly treated Malaria correctly treated Anaemia correctly treated Child needing oral antibiotic and/or oral antimalarial prescribed drug correctly Child not needing antibiotic leaves the facility without antibiotic Child needing vaccinations leaves the facility with all needed vaccinations First dose of treatment given at facility Caretaker advised to give extra uids and continue feeding Caretaker of child prescribed ORS, oral antibiotic and/or oral antimalarial knows how to give the treatment Caretaker of child prescribed oral medication advised on how to administer treatment Caretaker advised when to return immediately Caretaker given or shown a mothers nutrition and counselling card Index of availability of essential oral treatments (mean) Health facility has equipment and supplies to support vaccination services Index of availability of four vaccines (mean) Health facility has essential equipment and materials* Health facility has IMCI chart booklet and mothers nutrition and counselling card Health facility received at least one supervisory visit that included observation of case management during the previous 6 months

Health systems support Availability of drugs Availability of vaccines Availability of supplies Supervision visits Very sick children Assessment

Referral

Child checked for three danger signs: Able to drink or breastfeed Vomits everything History of convulsions Child not visibly awake checked for lethargy Child needing referral is referred Index of availability of injectable drugs for pre-referral treatment (mean)

* Includes accessible, working weighing scales for adults and children, timing device, child health cards, source of clean water, spoons and cups and jugs to mix and administer ORS.

Results
A total of 75 health facilities were visited. Children were observed for case-management in 73 facilities: 20 facilities in Morogoro, 19 in Ruji, 19 in Ulanga and 15 in Kilombero. No children presented for care on the day of the visit in the remaining two facilities. The total number of observed children was 419. The teams were fairly successful in nding the target number of children: six children were observed at 66 facilities (88%). More than one-third of children observed were under 1 year of age (39%; 163/419) and more than two-thirds were under 2 years (69%; 287/419). Approximately half were boys (53%; 220/419), and the mother accompanied 95% (396/419) of the children to the health facility. The youngest child was 2 months old, and the oldest was 56 months. The IMCI and comparison districts were similar in terms of the demographic characteristics of the children observed (data not shown).

All personnel observed managing children in IMCI districts had been trained in IMCI, and none of the personnel observed in the comparison districts were IMCI-trained. In the districts implementing IMCI, all sampled facilities had at least one health worker trained in IMCI, and in 64% had at least 60% of health workers who prescribed treatment for children under ve were trained in IMCI. Thus, although not all staff in IMCI districts were trained in IMCI, we only observed those who had been trained. The personnel observed had been trained in IMCI up to 3 years prior to the survey: 27% (63/232) of children observed were treated by a health worker who had been trained 3 years before the survey, 27% (62/232) by a worker trained 2 years before the survey, 23% (53/232) 1 year before the survey and 23% (53/232) the year of the survey. With respect to the level and gender of the personnel, there were some differences between the districts: Ruji and Kilombero had more female health workers with training as MCH aides or nurse assistants than Ulanga and Morogoro, which had more male health workers trained as clinical ofcers or assistant clinical

IMCI in Tanzania ofcers (data not shown). There was no evidence of differences in case-management by sex of the health worker, nor was there any evidence that case-management practices were worse among those trained 3 years before the survey than among those trained more recently (data not shown). Case management Nearly all the indicators suggested better case-management in areas with IMCI compared with those without IMCI (Table 2). Children in IMCI districts received more thorough assess-

ments and were more likely to be classied correctly than children in non-IMCI districts. In the two districts with IMCI, 95% (95% CI 9099) of children observed were assessed for the presence of cough, diarrhoea and fever while only 36% (2348) of children in comparison districts were assessed (p < 0.0001). Only 5% (012) of children in districts without IMCI had their weight checked against a growth chart and none were checked for feeding problems, while in IMCI districts 77% (6688) and 86% (7795), respectively, were checked. Vaccination status was checked in nearly all children observed from IMCI districts (93%; 8798) but in less than

Table 2. Quality of care for children observed in IMCI and comparison districts IMCI Morogoro and Ruji % (95% CI) Assessment indicators Child checked for the presence of cough, diarrhoea and fever Weight checked against a growth chart Vaccination status checked Feeding practices assessed (<2 yrs only) Child checked for other problems Classication indicators Child correctly classied Child correctly classied omitting coughs and colds and no dehydration Treatment indicators Pneumonia correctly treated Malaria correctly treated Anaemia correctly treated Child needing an oral antibiotic and/or an oral antimalarial is prescribed correctly Child not needing antibiotic leaves the facility without antibiotic Child needing vaccinations leaves the facility with all needed vaccinations Child receives the rst dose of treatment at facility Counselling and communication indicators Caretaker of sick child is advised to give extra uids and continue feeding Child needing an oral medicationc and prescribed one whose caretaker is advised on how to administer the treatment Sick child whose caretaker is advised on when to return immediately Child leaving the facility whose caretaker reports being given or shown a mothers nutrition and counselling card Caretaker of child who received a prescription for an oral medicationc reports correctly at facility exit how to give the treatment
a F-tests

Comparison P-valuea Ulanga and Kilombero % (95% CI)

95% (9099) n = 231 77% (6688) n = 230 93% (8798) n = 229 86% (7795) n = 151 80% (7089) n = 231 63% (5572) n = 219 84% (7891) n = 219 75% (5892) n = 59 88% (8195) n = 169 44% (2960) n = 61 73% (6580) n = 219 86% (8092) n = 150 12% (320) n = 69 84% (7592) n = 179 90% (8496) n = 215 96% (91100) n = 181 66% (5676) n = 219 48% (3461) n = 216 72% (6481) n = 225

36% (2348) n = 188 5% (012) n = 188 24% (1433) n = 188 0% n = 124 21% (1032) n = 187 38% (3144) n = 176 57% (5065) n = 176 40% (2852) n = 52 25% (1436) n = 135 4% (013) n = 23 35% (2545) n = 178 57% (4866) n = 117 0% n = 27 1% (03) n = 145 4% (08) n = 171 18% (629) n = 136 1% (03) n = 178 0% n = 177 56% (4566) n = 179

<0.001 <0.001 <0.001 <0.001 <0.001 <0.001 <0.001

<0.01 <0.001 0.01 <0.001 <0.001 >0.10b <0.001 <0.001 <0.001 <0.001 <0.01 0.02

comparing areas with and without IMCI, unless otherwise stated. exact test. c ORS and/or oral antibiotic and/or antimalarial.
b Fishers

Tanzania IMCI multi-country evaluation study group without having been prescribed one, compared with 57% (4866) in the comparison districts. IMCI was also associated with a major improvement in children receiving their rst dose of treatment at the health facility, with 84% (7592) of observed children starting treatment at the facility in IMCI districts compared with 1% (03) in the comparison districts. All areas showed disappointing results for children leaving the facility with all necessary vaccinations. Although the difference between the districts was in favour of IMCI, it did not reach statistical signicance (12%; 320 vs. 0%). National policy requires opening a vial of vaccine for a single child. In practice, however, health workers often ask the mother to return with the child during a scheduled vaccination session. IMCI-trained health workers showed enhanced counselling and communication skills compared with those in comparison districts (Table 2 and Figure 2). Ninety percent (90%; 8496) of the caretakers of sick children in these districts were advised to give extra uids and continue feeding, while only 4% (08) were advised to do so in the comparison districts. For children who needed an oral medication (ORS and/or an oral antibiotic and/or an oral antimalarial) and

one-quarter of children from comparison districts (24%: 1433). Evidence also suggested that children observed in IMCI districts were more likely to be diagnosed correctly than in the comparison districts (63%; 5572 vs. 38%; 3144). Sick children were more likely to receive appropriate treatment in the IMCI districts than in the comparison districts, as shown in Table 2. Pneumonia was correctly treated for 75% (5892) of the children in IMCI districts but for only 40% (2852) in comparison districts. IMCI was also associated with major improvements in the treatment of malaria, with 88% (8195) of children in IMCI districts correctly treated compared with only 25% (1436) in the comparison areas. Although anaemia was correctly treated in fewer than half of the observed children from IMCI districts (44%; 2960), this was much better than anaemia treatment in the comparison districts (4%; 013). Children needing an oral antibiotic and/or antimalarial received these drugs in 73% (6580) of cases in IMCI districts and in only 35% (2545) of cases in comparison districts. The inappropriate use of antibiotics was less frequent in IMCI districts, where 86% (8092) of children not needing antibiotics left the facility

Advised by health worker about when to return immediately

(n = 397 caretakers)

1 66

No IMCI IMCI

Advised by health worker about how to administer the oral medication* prescribed
(n = 317 caretakers of children prescribed an oral medication)

18 96

Reports correctly at facility exit how to give the oral medication* prescribed
(n = 404 caretakers leaving the facility with a prescription for an oral medication)

56 72

20

40

60

80

100

*prescribed ORS and/or oral antibiotic and/or oral antimalarial

Percentage of caretakers

Figure 2. Selected indicators of counselling and communication, IMCI and comparison districts in rural Tanzania, August 2000

IMCI in Tanzania were prescribed one by the health worker, 96% of caretakers in IMCI districts were advised on how to administer the treatment, compared with only 18% of caretakers in comparison districts. A considerable difference in favour of IMCI was also shown in the proportion of caretakers for whom health workers explained when to return immediately (i.e. urgently) to a facility with a sick child: two-thirds of the caretakers in IMCI districts were told when to return immediately (66%; 5679) compared with only 1% (03) of caretakers in comparison districts. Caretakers in IMCI districts also reported higher levels of correct knowledge about how to care for their sick children (Table 2 and Figure 2): 72% (6481) knew when leaving the facility how to give ORS solution to their child with diarrhoea to prevent dehydration and/or how to give oral antibiotics and/or oral antimalarials to those children who needed them, compared with 56% (4566) in the comparison districts. For all oral medications together, almost all caretakers (96%; 91100) knew how to administer the treatment in IMCI districts but less than one-fth (18%; 629) had this knowledge in comparison districts. In IMCI districts fewer than half of the mothers reported being given or shown a mothers nutrition and counselling card (48%; 3461), but none of the mothers in the comparison districts reported being shown such a card. Health systems support IMCI and comparison districts had similar availability of oral treatments and vaccines (Table 3), with both scoring relatively well on the index of availability but less than perfectly (0.93 vs. 0.95 and 0.76 vs. 0.87, respectively). Availability of essential equipment and materials was poor in all areas, with 16% (631) of health facilities having the complete set of materials in IMCI districts and 6% (120) in comparison

districts. Nearly all facilities with IMCI (97%; 87100) had the required chart booklet and mothers nutrition and counselling cards. Vaccination service support was excellent in the comparison districts, where vaccination equipment and supplies were found in all facilities, and signicantly worse (87%; 7396) in the IMCI districts. IMCI districts fared better than comparison districts in terms of having received supervisory visits that included observation of case management (51%; 3468 vs. 21%; 938). Very sick children Although the survey was not designed to recruit large numbers of seriously-ill children, there was evidence of more frequent assessment for the three danger signs and for lethargy, and better access to pre-referral drugs, in the IMCI districts than in the comparison districts (Table 4). Nevertheless, the rate of referral was low in all areas, with 17% (041) and 20% (046) of children needing referral actually referred in areas with and without IMCI, respectively.

Discussion
We have shown that IMCI training up to 3 years previously is associated with much better case management when compared with districts without IMCI. Our survey represents a quality assessment tool, one example of the rapid evaluation methods developed in the early 1990s (Bryce et al. 1992; Forsbert et al. 1992; Anker et al. 1993). We assessed whether those trained in IMCI actually use it when observed by an outsider. It is clear that the way health workers manage children when they are observed may differ from their usual practices: our survey was designed to assess best-practice at 03 years after training, and the results suggest that this bestpractice case-management is much improved by IMCI training. The study does not reect the impact of this change,

Table 3. Health systems support indicators in IMCI and comparison districts IMCI Morogoro and Ruji (95% CI) Availability of drugs Index of availability of essential oral treatments (mean) Availability of vaccines Health facility has equipment and supplies to support full vaccination services (%) Index of availability of four vaccines (mean) Availability of supplies Health facility has essential equipment and materials (%) Comparison P-valuea Ulanga and Kilombero (95% CI)

0.93 (0.880.97) n = 39 87% (7396) n = 39 0.76 (0.620.89) n = 39

0.95 (0.910.98) n = 35 100% (90100) n = 35 0.87 (0.800.94) n = 35 6% (120) n = 34 0% (010) n = 35 21% (938) n = 34

0.47 0.03 0.15 0.18 <0.001

16% (631) n = 38 Health facility has IMCI chart booklet and mothers nutrition 97% (87100) and counselling cards (%) n = 39 Supervision Health facility received at least one supervisory visit that included observation of -case management during the previous 6 months (%)
a Comparing

51% (3468) n = 37

0.007

IMCI and comparison areas: F-tests.

Tanzania IMCI multi-country evaluation study group

Table 4. Assessment, referral and accessibility of referral care for very sick children in IMCI and comparison districts IMCI Morogoro and Ruji (95% CI) Assessment indicators Child checked for three danger signs (not able to drink or breastfeed, vomiting everything, history of convulsions) Child checked for lethargy Referral indicators Child needing referral is referred Index of availability of injectable drugs for pre-referral treatment (mean)
a Comparing

Comparison P-valuea Ulanga and Kilombero (95% CI)

96% (93100) n = 231 100% n=9 17% (041) n = 12 0.85 (0.780.93) n = 39

0% n = 188 25% (248) n = 16 20% (046) n = 10 0.49 (0.440.54) n = 35

0.0001 0.05 0.85 <0.001

IMCI and comparison areas: F-tests.

with less than universal training coverage, on child health and survival in the community: these measures are being assessed by separate household surveys and ongoing demographic surveillance and will be reported elsewhere. For most health system support indicators, apart from supervision, there were no signicant differences between the intervention and comparison districts. Supervision visits including the observation of case-management were reportedly more common in the districts with IMCI than in the comparison areas. Supervision is commonly considered to be a key part of any health system, and essential to improving the quality of care. Unfortunately there are few rigorous effectiveness evaluations of the contribution of supervision to health worker performance in developing countries, in part because there are few functioning supervisory systems. One controlled eld trial in the Philippines found that systematic supervision of midwives based on observation of health worker performance with immediate feedback was associated with improved performance, but even in this study the clinical signicance of the improved outcomes could be questioned (Loevinsohn et al. 1995). It seems unlikely that good supervision of a weak system would result in high quality of care. However, even sub-optimal supervision of a robust system can result in high quality of care. Our study assesses the effect of a dual intervention IMCI together with a reasonably strong health system empowered by evidence-based planning and expenditure mapping at district level on quality of child health care and health facility support. We are not able to separate the effects of these interventions, and our ndings cannot be generalized to areas with weaker health systems or less managerial capacity at district level than that found in Morogoro and Ruji Districts. Several interrelated factors may explain why health worker performance in these districts in Tanzania was maintained at high levels up to 3 years after IMCI training. First, the presence of relatively strong health system support in both the intervention and comparison districts, reected in reasonably frequent supervision including the observation of

case management and the presence of the basic drugs and equipment required to implement IMCI, may provide a necessary foundation for maintaining adequate performance after training (Bryce et al. 1993). Second, district-level control and accountability for the quality of care as a result of the MoH/TEHIP district strengthening activities may have increased levels of motivation among health personnel in these two districts, despite the fact that levels of nancial compensation remained unchanged and equivalent to those in the comparison districts. Third, utilization rates for public health facilities in these districts were high relative to those reported from other developing countries. In the household surveys that are a part of the MCE, 41% of children ill in the 2 weeks previous to the survey had been taken rst4 to a public health facility for care (Armstrong Schellenberg et al. 2003). The ndings on malaria case management deserve special mention. Eighty-four percent of deaths attributed to malaria occur among children under ve (Murray et al. 2001). Success in reducing malaria mortality will require not only high levels of coverage with effective preventive interventions such as the use of insecticide-treated nets (Armstrong Schellenberg et al. 2001), but also rapid and appropriate treatment with effective antimalarials. Our study has shown that IMCI can lead to rapid improvements in the correct treatment of presumed malaria in health facilities. In the four rural districts of Tanzania studied here, 88% of children with presumed malaria were correctly treated in the IMCI districts while only 25% were correctly treated in districts without IMCI. In addition, P falciparum malaria accounts for much of the anaemia in African children (Menendez et al. 1997; Newton et al. 1997), and this study shows that 44% of children presenting with anaemia received correct treatment in IMCI districts, compared with only 4% in districts not yet implementing IMCI. In conclusion, IMCI has already become policy for primary child care in many countries throughout the world. Among other things, this means that malaria case-management in under-ves at rst-line health facilities in most of subSaharan Africa depends to a large extent upon IMCI.

IMCI in Tanzania Integration, recently dened in the context of primary health care as . . . change to bring together inputs, organization, management and delivery of particular service functions (Briggs et al. 2001) is a logical step for overstretched primary health care workers. Although a recent review of the health service delivery outcomes associated with integrated primary health care services nds little evidence of their effectiveness, only four studies met the inclusion criteria for the review and none of them focused on the integrated packaging of child health interventions (Briggs et al. 2001). Our study suggests that IMCI in the presence of some practical and affordable health system tools is feasible for implementation at high levels of coverage in resource-poor countries, and is likely to lead to rapid gains in child survival, health and development if adequate coverage levels can be achieved and maintained. Further results on the cost-effectiveness and impact of IMCI and the MoH/TEHIP interventions will be forthcoming from this and the other MCE sites over the coming 3 years.

Endnotes
1 The Ministry of Health and partners (the World Bank and the Governments of Denmark, Ireland, the Netherlands, Norway, Switzerland and the UK) pool resources in a common basket from which funds are then directly disbursed to districts through special accounts of the Council Health Management Teams. 2 These indicators include the priority indicators at health facility level for the IMCI-MCE, and are consistent with the core indicators for IMCI at health facility level as dened by an interagency working group including representatives from WHO, UNICEF, the United States Agency for International Development (USAID), the US Centers for Disease Control, and the USAIDfunded BASICS Project. The IMCI-MCE indicators can be found in the description of the IMCI-MCE plan of analysis available at www.who.int/imci-mce; the denitions of the core indicators for IMCI at health facility level can be found at www.who.int/childadolescent-health. 3 The weights had two components, representing (1) the withinfacility and (2) the between-facility probability of inclusion in the sample. These two components were multiplied together. For the within-facility probability of inclusion, the weight was the inverse of the probability of inclusion in the sample within the health facility, which was estimated by the number of children actually seen in the facility in the survey divided by the average daily utilization, estimated from MoH and other reports. For the between-facility probability of inclusion, the weight was the inverse of the probability of the facility being included in the sample, which was the number of facilities of that type in the sample divided by the total number of facilities of that type in the district. 4 Other health care providers include pharmacists, shop keepers selling drugs, traditional healers, family members and neighbours.

References
Ahmed OB, Lopez AD, Inoue M. 2000. The decline in child mortality: a reappraisal. Bulletin of the World Health Organization 78: 117591. Anker M, Guidotti RJ, Orezeszyna S, Sapirie SA, Thuriaux MC. 1993. Rapid evaluation methods (REM) of health services performance: methodological observations. Bulletin of the World Health Organization 71: 1521. Armstrong Schellenberg JRM, Abdulla S, Nathan R et al. 2001. Effect of large-scale social marketing of insecticide-treated nets of child survival in rural Tanzania. The Lancet 357: 12417. Armstrong Schellenberg JRM, Victora CG, Mushi A et al. and the

Tanzania IMCI MCE baseline household survey study group. 2003. Inequities among the very poor: health care for children in rural southern Tanzania. The Lancet 361: 5616. Briggs CJ, Capdegelle P, Garner P. 2001. Strategies for integrating primary health services in middle- and low-income countries: effects on performance, costs and patient outcomes (Cochrane Review). In: The Cochrane Library, Issue 4. Oxford: Update Software. (Last update, 29 July 2001). Black RE, Morris SS, Bryce J. 2003. Where and why are 10 million children dying every year? The Lancet 361: 222634. Bryce J, Toole MJ, Waldman RJ, Voigt A. 1992. Assessing the quality of facility-based child survival services. Health Policy and Planning 7: 15563. Bryce J, Cutts F, Naimoli JF, Beasley M. 1993. What have teachers learnt? The Lancet (letter) 342: 1601. Cassels A. 1997. Guide to sector-wide approaches for health development: concepts, issues and working arrangements. WHO/ARA/97.12. Geneva: World Health Organization. De Savigny D, Kasale H, Mbuya C et al. 2002. TEHIP Interventions: an overview. Tanzania Essential Health Interventions Project, Discussion Paper No 2, February 2002. Dar es Salaam: Ministry of Health. Font F, Alonso Gonzlez M, Nathan R et al. 2001. Diagnostic accuracy and case management of clinical malaria in the primary health services of a rural area in south-eastern Tanzania. Tropical Medicine and International Health 6: 4238. Forsbert BD, Barros FC, Victora CG. 1992. Developing countries need more quality assurance: how health facility surveys can contribute. Health Policy and Planning 7: 1926. Gilson L, Kitange H, Teuscher T. 1993. Assessment of process quality in Tanzanian primary care. Health Policy 26: 11939. Gove S. 1997. Integrated management of childhood illness by outpatient health workers: technical basis and overview. Bulletin of the World Health Organization 75 (Suppl 1): 716. Loevinsohn BP, Guerrero ET, Gregorio SP. 1995. Improving primary health care through systematic supervision: a controlled eld trial. Health Policy and Planning 10: 14453. Menendez C, Kahigwa E, Hirt R et al. 1997. Randomised placebocontrolled trial of iron supplementation and malaria chemoprophylaxis for prevention of severe anaemia and malaria in Tanzanian infants. The Lancet 350: 8449. Ministry of Health. 2001. National Health Accounts, Draft Final Report, May 2001. Dar es Salaam: Ministry of Health. Murray CJL, Lopez A, Mathers C, Stein C. 2001. The Global Burden of Disease 2000 Project: aims, methods and data sources. Global Programme on Evidence for Health Policy Discussion Paper no 36. Geneva: World Health Organization. National Bureau of Statistics and Macro International Inc. 1999. Tanzania Reproductive and Child Health Survey, 1999. Dar es Salaam: Tanzania National Bureau of Statistics. Newton CR, Warn PA, Winstanley PA et al. Severe anaemia in children living in a malaria endemic area of Kenya. Tropical Medicine and International Health 2:16578. Nsimba SED, Massele AY, Eriksen J et al. 2002. Case management of malaria in under-ves at primary health care facilities in a Tanzanian district. Tropical Medicine and International Health 7: 2019. Rowe AK, Onikpo F, Lama M, Cokou F, Deming MS. 2001. Management of childhood illness at health facilities in Benin: problems and their causes. American Journal of Public Health 91: 162535. Tanzania Essential Health Interventions Project (TEHIP). 2000. Ruji Demographic Surveillance System data for 2000. Dar es Salaam. UNDP. 2001. Human Development Report 2001. New York: Oxford University Press. WHO. 2000. Unpublished estimates of under-ve mortality by WHO regions, 19551999. Global Programme on Evidence (GPE), World Health Organization, Geneva. Constructed for special report through abstraction from EIP data bases for

10

Tanzania IMCI multi-country evaluation study group

Coast), Dr Fred Lwilla (District Medical Ofcer, Kilombero), Dr P Mbena (District Medical Ofcer, Ulanga), Dr Harun Machibya (District Medical Ofcer, Morogoro Rural), Dr Saidi Mkikima (District Medical Ofcer, Ruji), Dr P Mongi (IMCI Team Leader at WHO Tanzania), Dr C Drasbek (WHO/PAHO) and Dr A Mbewe (WHO/AFRO). We are also grateful to Professor Cesar Victora for comments on the manuscript. The Tanzania Essential Health Interventions Project is a collaboration co-funded by the Tanzania Ministry of Health and the International Development Research Centre, Canada. The IMCI-MCE is arranged, coordinated and funded by the Department of Child and Adolescent Health and Development of the World Health Organization with the nancial support of the Bill and Melinda Gates Foundation and the US Agency for International Development. This paper is published with the permission of Dr Andrew Kitva, Director-General of the National Institute of Medical Research, for whose support we are grateful. Correspondence: Joanna Armstrong Schellenberg, Gates Malaria Partnership, London School of Hygiene and Tropical Medicine, Keppel Street, London, WC1E 7HT, U.K. E-mail: dajobelo@aol.com

1999 by the Department of Child and Adolescent Health and Development (CAH), with review and clearance by GPE, 2000. WHO. 2001. Global Monitoring of IMCI Implementation, December 2001. Report available at http:/www.who.int/childadolescent-health. Department of Child and Adolescent Health and Development, World Health Organization, Geneva. WHO. 2002. Health Facility Survey for Integrated Child Health Services (IMCI). Final draft, March 2002. Available at http://www.who.int/child-adolescent-health. Department of Child and Adolescent Health and Development, World Health Organization, Geneva.

Acknowledgements
The authors would like to thank those responsible for IMCI implementation in Tanzania, including: Dr A Mzige (Director of Preventive Services, Ministry of Health), Dr Colleta Kibassa (IMCI Co-ordinator at Ministry of Health), Dr F Fupi (Regional Medical Ofcer, Morogoro), Dr M Nyakarungu (Regional Medical Ofcer,