World Development Vol. 40, No. 9, pp.

18101823, 2012
2012 Elsevier Ltd. All rights reserved.
0305-750X/$ - see front matter
www.elsevier.com/locate/worlddev

http://dx.doi.org/10.1016/j.worlddev.2012.04.030

Asking Questions to Understand Rural Livelihoods: Comparing

Disaggregated vs. Aggregated Approaches to Household
Livelihood Questionnaires
PAMELA JAGGER
University of North Carolina at Chapel Hill, USA
Center for International Forestry Research, Bogor, Indonesia
MARTY K. LUCKERT
University of Alberta, Edmonton, Canada
Center for International Forestry Research, Bogor, Indonesia
ABWOLI BANANA and JOSEPH BAHATI *
Makerere University, Kampala, Uganda
Summary. This study tests the hypothesis that both disaggregated and aggregated data collection methods produce similar estimates
of the relative importance of livelihood portfolio activities and expenditures. The results show that dierent methods of data collection
yield substantively dierent estimates of livelihood strategies for two indicators: income and expenditure. We also nd evidence of a seasonal bias in responses to household livelihood questions asked at higher levels of aggregation. Our ndings highlight the challenge of
designing household surveys to elicit accurate and precise information, and demonstrate that dierent methods of data collection inuence our understanding of rural livelihoods.
2012 Elsevier Ltd. All rights reserved.
Key words Africa, livelihoods, forestry, household surveys, methods, Uganda

Collecting data that accurately characterize rural livelihood

portfolios has proven to be a daunting task. The breadth of
livelihood activities presents particular challenges to researchers, as many aspects, for example the role of forest or environmental goods and services that have thin or missing markets,
may be omitted from standardized data collection eorts
(Campbell & Luckert, 2002; Sjaastad, Angelsen, Vedeld, &
Bojo, 2005; Vedeld, Angelsen, Sjaastad, & Berg, 2004).
Characterization of rural livelihoods is dicult even in the
absence of such problems. To be able to fully describe rural

1. INTRODUCTION
Academic researchers, development practitioners, donors,
and others increasingly recognize the importance of understanding micro-level socioeconomic dimensions of the rural
economy. Our interest in livelihood portfolios, often characterized by income, consumption, expenditure, or time use patterns, is motivated by the desire to understand the lives of the
poor, how to best design development interventions, and to
provide information about how development policies and projects aect livelihood outcomes. Demand for high quality
information on rural livelihoods has resulted in the development of a diversity of methods for eliciting information from
local people (e.g., Basic Necessities Survey (Davies, 1997);
Stages of Progress (Krishna, 2005); Sustainable Livelihoods
Framework (Carney, 1998; Ellis & Freeman, 2004); Household Livelihood Security Assessments (CARE, 2002); the
Living Standards Measurement Study Surveys (LSMS) etc.).
Data collection methods vary greatly with respect to: the use
of qualitative and quantitative data collection methods;
whether questions are put to individuals or groups of people;
and the degree to which information is collected by an interviewer vs. generated through joint learning with representative
groups. Discussions regarding the strengths and weaknesses of
alternatives approaches have accompanied the emergence of
these various methods (Campbell, 2002; Davis & Whittington,
1998; Fisher, Reimer, & Carr, 2010; Menton, Lawrence,
Merry, & Brown, 2010; Scoones, 1995; Schrekenberg et al.,
2010).

* We are grateful to the following organizations for funding this research:

the Center for International Forestry Research (CIFOR); the Collective
Action and Property Rights Initiative (CAPRi) of the Consultative Group
on International Agricultural Research; the National Science Foundation
(NSF, Grant No. DDIG 0622392); the Social Sciences and Humanities
Research Council of Canada (SSHRC); the Social Science Research Council (SSRC); and the Sustainable Agriculture and Natural Resource
Management Collaborative Research Support Program (SANREM CRSP, Grant No. EPP-A-00-04-00013-00). Roughly 270 households participated in this study. Their willingness to share information and to host us
for numerous hours in their homes was critical to the success of this
project. We are grateful to colleagues at CIFOR, in particular those who
are part of the Poverty Environment Network (PEN) for comments on
early drafts of this paper. Colleagues with the International Forestry Resources and Institutions (IFRI) research group also provided valuable
comments. We are particularly grateful to Arild Angelsen for substantive
feedback on this research. Any errors or omissions are our own. Final
revision accepted: December 12, 2011.
1810

ASKING QUESTIONS TO UNDERSTAND RURAL LIVELIHOODS: COMPARING DISAGGREGATED

1811

Figure 1. Study area (highlighted in box), Hoima and Kibaale Districts, western Uganda.

livelihoods, one would have to follow and document the production, consumption, expenditure, and/or time allocation
decisions of all household members for an extended period
of time. This approach is both intrusive to the respondent,
and has high nancial and time costs for the person collecting
the data. Further, there is no consensus regarding what set of
variables provides the most accurate information about household welfare. Development economists often favor detailed
consumption and expenditure data (e.g., LSMS, the RePEAT
survey (Research on Poverty, Environment, and Agriculture
in Ethiopia, Kenya, and Uganda)) whereas other social scientists may consider subjective assessments of the relative importance of income sources to more accurately represent
household livelihood portfolios (Chambers, 1994). These
inquiries are informed by a diversity of social science research
methods drawing on a wide range of disciplines including economics, sociology, anthropology, and geography.
Virtually every researcher engaged in social science eld
work has faced decisions regarding the appropriate level of
aggregation to collect data. But these decisions are made largely based on individual experiences and disciplinary training,
as there is little empirical literature on this topic. Our goal in
this paper is to discuss issues regarding the accuracy and precision of data associated with aggregated vs. disaggregated approaches to household livelihood surveys. We hope to inform
further research that addresses the biases inherent in various
approaches to collecting data on livelihood portfolios in developing country settings.
One means of dierentiating between data collection approaches is to consider how specic vs. general the questions
that researchers ask to respondents are. In the extreme, the

research question under investigation could be put directly

to local respondents. But this approach is rarely taken. For
example, researchers would not generally ask respondents
the question: How diversied is your livelihood portfolio?
Instead, researchers tend to approach this more general research question by asking more specic questions about
household activities. But these questions may still vary significantly regarding the specic vs. general approach taken. For
example, specic questions may be asked with income and
expenditure surveys that collect information which is highly
disaggregated, requiring the researcher to aggregate individual
quantities and prices to calculate representations of portfolios
(Campbell et al., 2002; Cavendish, 2000). In contrast, a typical
village level participatory rural appraisal exercise (PRA) may
ask people to rank the relative importance of livelihood
activities for a subset or all households in a village (Krishna,
Kristjansen, Radeny, & Nindo, 2004). Such an approach is
requesting general information from respondents that is
reported in a more aggregated form.
This study asks the question: How does the level of aggregation in data collection aect our understanding of rural livelihoods? We test the hypothesis that both disaggregated and
aggregated data collection methods produce similar estimates
of the relative importance of livelihood portfolio activities. In
practice, whether data are collected in an aggregated or
disaggregated fashion is a matter of degree. In the context of
our study, we dene disaggregated data collection as information that is collected with detailed structured household-level
socioeconomic surveys designed to elicit ne-scale data that
can be aggregated over variables such as income and expenditure, to provide summary information about livelihoods. The

1812

WORLD DEVELOPMENT

aggregated approach, in the context of our study, is a participatory rural appraisal exercise that asks household member
to think in an aggregate way about categories of livelihood
activities. Both methods rely on participation from the diversity of household members in responding to questions, including household heads, spouses, children, and other individuals
that contribute to the economic welfare of the household. 1
We use two indicators in our analysis of livelihood portfolios: income and household expenditures. Our aim is to better
understand how to collect accurate and precise livelihoods
data, to consider the potential biases arising from the use of
various data collection methods, and to contribute to a discussion about methodological choices and their implications for
our ability to design development projects, formulate policies
to improve rural livelihoods, and assess the impact of development programs and policies.
We begin in the next section by identifying potential
strengths and weaknesses associated with disaggregated vs.
aggregated approaches to data collection. We then present
an experiment where we collect data on indicators of rural
livelihood portfolios for two sub-samples of the same population of households in western Uganda using dierent survey
instruments: a highly disaggregated income and expenditure
survey, and a participatory rural appraisal (PRA) survey
instrument that collects more aggregated information. We
compare the results of these two approaches to see whether
and why they are dierent. We conclude with a summary of
our assessment about the appropriateness of the methods for
various research and data collection eorts.
2. ACCURACY AND PRECISION OF
DISAGGREGATED AND AGGREGATED
APPROACHES TO COLLECTING DATA
(a) Data accuracy and precision
In this paper we focus our eorts on issues of data quality
associated with disaggregated vs. aggregated approaches to
collecting household level data that may be used to characterize livelihood portfolios. 2 We focus on two attributes that
contribute to the quality of data: accuracy and precision.
Accuracy implies that the data are representative of the truth,
although the data may be widely distributed around the true
value. For example, a grouping of arrows closer to the
bulls-eye of a target is more accurate than a similarly dispersed grouping that is farther away. 3 The concept of accuracy is parallel to the concept of bias in statistics. More
accurate estimates are less biased. Precision refers to the tightness of the distribution of the data collected, whether or not
the distribution is around the true value. For example, you
could have a tight grouping of arrows that is precise, but it
could be inaccurate if not grouped around the bulls-eye. The
concept of precision is parallel to the concept of variance in
statistics. More precise estimates have smaller variances.
The precision of data can be conceptualized at various levels. At one level, the precision of a single piece of information
collected from a household may be assessed by repeated measures to see if there is measurement error. This type of precision can be dicult to measure in household livelihood
surveys. If one were to visit the same household repeatedly,
to see if the same answers were forthcoming, the conditions
surrounding a households livelihood, such as seasonality,
would likely change between visits, causing answers to change.
Alternatively, if one were to repeatedly ask the same questions
within a short time frame, respondents may get fatigued or

respond dierently after having time to contemplate questions.

Instead, for the purposes of our study, we consider the precision of a measure of a central tendency (i.e., means). Even if
every single data point is precise, a mean value may not be,
if there is a large variance in the data.
Data on rural livelihood portfolios that are more accurate
are desirable as they are closer to being representative of the
true values of local people. Accurate data that also yield a precise measure of a central tendency may also be desirable as
they may better support conclusions related to research questions. For example, accurate data that yield precise measures
of central tendencies could cause a researcher to state with a
high level of condence that forests contribute 20% to household incomes. Data that give us precise measures of central
tendencies, but inaccurate responses, are dangerous, as condence intervals built around mean estimates do not capture
the fact that there is an underlying bias that has caused the
data to vary from the truth. While accurate data are always
desired, the precision of central tendencies may or may not
be a desirable attribute. Though precision in a central tendency can increase the condence in our statement regarding
the contribution of forests to total income, variation in data
can be useful to identify causal relationships. For example, if
we wish to investigate impacts of labor on forest income, then
variation in the data on forest income, labor, and other
explanatory variables are desirable.
Disaggregated approaches may be viewed as sampling exercises, where each bit of information is collected and then combined by the researcher to construct a larger picture.
Disaggregated approaches frequently involve collecting data
representing short time intervals, repetitively and in detail.
Though more aggregated approaches also tend to collect
and combine information to construct a large picture, they
do so with less detailed and frequently fewer questions, thereby counting on the respondent to supply more aggregated
information and leaving the researcher with less information
to aggregate. Therefore in the discussion that follows, a key
underlying question is: Who is in a better position to aggregate information; the researcher or the respondent?
(b) Advantages of disaggregated approaches to collecting
household livelihood data
Disaggregated approaches to data collection may be justied by considering that all respondents are limited, to some
degree, in their cognitive abilities to respond to questions.
Respondents cognitive ability may be limited with respect to
memory, the ability to synthesize or add up activities or
events, and their scope of knowledge. Our ability to recall
events in the medium to long run may make it dicult for
respondents to characterize livelihood activities that occur seasonally or over irregular intervals during the course of a year.
Under such conditions, responses may be biased toward recent
activities that may not be representative of the entire year (e.g.,
Campbell et al., 2002). With repeated seasonal sampling,
respondents are tasked with recalling events in recent memory.
Limitations regarding respondents ability to synthesize
information may similarly be addressed by collecting disaggregated data. For example, asking a respondent the importance of forest products to their household livelihood
requires synthesizing the contribution of many types of
goods and services that forests provide (e.g., fuel wood,
poles, mushrooms, ropes, bamboo, etc.). In contrast, disaggregated information about consumption and expenditures
associated with specic products do not require high levels
of synthesis. Indeed, some of the information collected with

ASKING QUESTIONS TO UNDERSTAND RURAL LIVELIHOODS: COMPARING DISAGGREGATED

disaggregated approaches, such as gendered time use data in

collecting forest products, may be a direct measure of the
behavior that the researcher is trying to understand. Finally,
the respondents scope of knowledge may be better understood with disaggregated rather than aggregated approaches.
For example, if questions are asked about the importance of
forest products in general, then the researcher has little information about the breadth and depth of the respondents
knowledge underlying the response. In contrast, in disaggregated questionnaires, respondents answer detailed questions
about specic forest products such as fuel wood, poles,
mushrooms, and bushmeat. The level of condence and detail of their responses is generally a good indicator of overall
knowledge of the livelihood activity.
Another potential advantage of the disaggregated approach to data collection is that it may help limit strategic
behavior by respondents. When household members respond
to questions, they may answer strategically for a number of
reasons. For example, respondents may purposively portray
an overly negative picture of their situation because they believe it will increase the probability of receiving aid. Alternatively, respondents may purposively overemphasize aspects of
their livelihoods because researchers have told them ahead of
time the objectives of the research (e.g., to value forest products). Whatever the source of bias, strategic answers may be
dicult to formulate with disaggregated questions. With
questions directed toward collecting many discrete pieces of
information, it can be challenging for respondents to consistently answer numerous questions such that a strategic bias
emerges from the data. In contrast, more aggregated approaches may make it easier for respondents to express a
strategic bias. An additional advantage to more disaggregated approaches comes from the greater number of observations that such an approach implies. With repeated visits
and more data comes greater power in the use of statistical
tools for assessing the accuracy and precision of information.
Finally, we note that disaggregated approaches allow for the
estimation of absolute values of income and expenditures
(i.e., vs. relative values) which may be important to donors,
policy makers, and others engaged in understanding not only
the relative importance of rural livelihoods but the magnitude of dierent activities.
The cognitive limitations of respondents and the potential
for strategic behavior suggest that a disaggregated approach
to data collection could increase the accuracy of the information collected. If disaggregated approaches are successful at
collecting information within the bounds of respondents cognition and successful in limiting strategic answers, then, all
other things being equal, the information may be thought of
as being closer to the true values of the respondent. But it is
not clear whether such an approach will lead to greater precision of estimates of central tendencies. Even if disaggregated
questions are posed within the cognitive bounds of respondents, these bounds may be highly variable causing dispersed
distributions of information.
(c) Advantages of aggregated approaches to collecting household
livelihood data
Aggregated approaches may also have advantages in uncovering the truth. Policy makers and development practitioners
are often interested in understanding the livelihood behavior
of local households. If we are collecting information to try
to understand what local people value, and why they do what
they do, then local perceptions are a primary concern. The day

1813

to day livelihood decisions of respondents are made in the context of bounded rationality (Jones, 2001) that recognizes the
limits in cognition discussed above. Therefore, information
which reects the more general perceptions of respondents
may be more relevant to understanding values and behavior
than the disaggregated information compiled by researchers.
Moreover, these general perceptions may, in some ways, be
more holistic indications of respondents values than disaggregated approaches. For example, in disaggregated approaches
that collect data on prices and quantities of sold and consumed forest products, there may be questions regarding the
relevance of prices in reecting local values. Many forest products are harvested for home consumption and not marketed.
Responses about products that have market prices may not
internalize negative environmental eects of activities such as
forest clearing. The market-based information that the researcher is aggregating may be less reective of social values
than the values that respondents are aggregating in their perceptions.
The advantages regarding the relevance and holistic qualities of respondents perceptions suggest that an aggregated approach could increase the accuracy of the information
collected. If aggregated approaches are successful in capturing
these perceptions then, all other things being equal, the information may be thought of as being closer to the true values of
the respondent. But, similar to the situation described above,
it is not clear whether such an approach will lead to greater
precision of estimates of central tendencies. Perceptions
regarding the importance of livelihood activities can vary signicantly within a population even if they are accurately collected.
A nal advantage of aggregated information is the cost of
data collection. The repeated and intensive sampling associated with more disaggregated approaches generally involves
higher costs. Data collection costs frequently involve expenses
to hire, train, and monitor teams of enumerators and supervisors. Once the data have been collected, coding, entering, processing, and analysis of disaggregated data can be costly. In
addition to saving researchers time, aggregated methods also
save respondents time. Disaggregated household surveys can
take a very long time to administer, particularly during the
early rounds of data collection when respondents are asked
to recall very detailed information in a structured format.
Aggregated surveys generally demand far less respondent
time.
(d) Levels of aggregation and data quality
The above discussion points out that, conceptually, there
are potential data quality problems associated with both
researchers and respondents aggregating information. With
little guidance regarding how severe the problems are, or
which problems are likely to be worse, we turn to an empirical approach to attempt to shed light on these issues. With
potential biases inuencing both aggregated and disaggregated approaches, we cannot interpret either approach as being
more accurate than the other. Therefore, in the empirical
analysis that follows, we are restricted to assessing whether
dierent levels of respondent and researcher aggregation
yield dierent results, and then conjecturing about why the
results may be dierent. Moreover, with no a priori expectations about the relative precision of central tendencies of
data collected with aggregated vs. disaggregated approaches,
we can only report results with the hope that they provide
further insights.

1814

WORLD DEVELOPMENT

3. STUDY CONTEXT AND DATA

(a) Study area and overview of data collection
Data are from two separate households surveys conducted
in western Uganda. We investigate dierences in responses
from a survey which takes a disaggregated approach to data
collection (the PEN questionnaire), 4 and one which takes an
aggregated approach to collecting data on household livelihoods (the UFRIC questionnaire) 5. We designed our data collection eort to avoid dierences in our samples of households
that could confound our comparisons. We administered separate questionnaires (i.e., disaggregated and aggregated) to subsets of the same population of rural households. The questionnaires were administered contemporaneously, and designed to
capture data on livelihood portfolios for a years worth of
activities.
The study villages fall within Hoima and Kibaale districts
(Figure 1) in Ugandas western banana, coee, cattle agro-ecological zone. 6 Rainfall is moderate and altitude ranges from
1000 to 1500 m above sea level. Primary crops planted include
banana, coee, maize, sweet potato, and cassava. Smallholders
keep cattle, small ruminants, and poultry in extensively managed pasture systems ( MAAIF, 1995). Landholdings in the
area are relatively small averaging 2.65 hectares per household. Two land tenure systems are common: freehold and
mailo. 7 The area has undergone rapid settlement over the past
10 years largely by Bakiga migrants from land scarce Kabale
District in southwestern Uganda. Tropical high forest is the
dominant land use, though forests are being rapidly cleared
by recent migrants. Degraded forest mosaics are common,
particularly in areas with relatively good market access. Livelihood strategies in the study area fall within ve main categories: agriculture; livestock husbandry, collection of forest and
wild products, wage labor, and self employment (i.e., small
business). The labor force is relatively stationary suggesting
few opportunities for households to receive remittance income.
(b) The disaggregated approach to data collection (PEN)
The disaggregated survey was conducted in six villages
drawn from a stratied random sample of villages throughout
Hoima and Kibaale Districts. Thirty households were randomly selected from each village (N = 170). A list of households residing in each village was compiled, drawing upon
information from village registers, lists provided by village
leaders, and information from key informants. Polygamous
households were listed according to the wifes name; each wife
was considered a separate household unless key informants
indicated that wives undertook activities such as cooking
and cultivating jointly. Each household was visited quarterly.
During each visit, data on the households income portfolio
and expenditures were collected. When applicable, respondents were asked to dierentiate between subsistence income
and cash income. 8 Income and expenditure data were collected according to various recall periods thought to match
the cognitive abilities of respondents (Cavendish, 2002). Additional data were collected on various socioeconomic indicators
including endowments of land, labor, capital, assets, and resource use. The eld work was undertaken between October
2006 and August 2007.
A signicant amount of time is required to administer this
type of survey, both due to the nature and number of questions,
and also due to the need to undertake repeated visits required
to capture seasonal changes in income and expenditures.

Making repeated visits to households is both time and resource

intensive, particularly if the research is undertaken in a remote
area, or includes households scattered across a large geographic area. Detailed data collection of this nature also requires signicant time, patience, and recall abilities on the
part of respondents. The average time to complete the income
and expenditure components of the questionnaire was 45
60 min per quarter. Ideally, both an adult male and adult female from each household participated in each interview, and
as many household members as possible were asked to participate. Analysis of our data reveal that depending on the quarter, between 65% and 79% of households with both an adult
male and female had both present for the duration of the interview. We estimate that each household devoted roughly 4 h
(1 h  4 quarters) to the study, and in many cases, given the
participation of multiple household members, the data collection took eight or more hours of household time over the
course of the year. Gifts of sugar, match boxes, and soap,
approximately equivalent to the value of two adult days of daily wage labor, were given to each household after each quarterly survey to compensate respondents for time away from
productive activities.
(c) The aggregated approach to data collection (UFRIC)
The aggregated survey was undertaken in one village, purposively selected from among the six villages included in the PEN
study. The criterion for village selection was the presence of a
community forest, which was important to the broader objectives of the UFRIC survey. The sample (N = 89) was the population of households in the village, omitting the 30 households
that responded to the PEN questionnaire. We purposefully
avoided interviewing the same households as participation in
the PEN study may have biased responses to the UFRIC questionnaire. Income and expenditure data were collected using
participatory rural appraisal (PRA) methods involving a two
stage process (Chambers, 1994). First, respondents were given
two sets of cards with pictures depicting 10 categories of activities related to income and expenditure, along with verbal
explanations given by enumerators. Respondents were asked
to rank the cards according to their relative importance to
the household. Providing pictures allowed respondents to view
and adjust their ranking decisions; the use of pictures is
thought to enhance cross-cultural communication (Bradley,
1995). Second, after placing the cards in rank order, household
members were asked to distribute 50 beans according to the relative importance or weight of each category of income and
expenditure to the household. The UFRIC questionnaire was
undertaken during the nal 2 weeks of March 2007.
The administration of household level PRA exercises is time
consuming. The household survey took between 40 and
60 min per household. Roughly half of the time was devoted
to the ranking and weighting exercise for the income and
expenditure data collection. But, by visiting each household
only once to collect data retrospectively for a years worth
of activities, the aggregated method signicantly reduces the
burden on respondents. The household level PRA exercise
was one component of a larger household survey focused on
forest management. Data on other socioeconomic variables,
household demographics, household welfare, land, and forest
use were also collected. The UFRIC questionnaire was conducted in tandem with village level focus groups and the collection of plot level biophysical data on forest composition
and condition. Table 1 summarizes the types of questions that
households were asked to respond to regarding activities over
a one year period.

Recall period

Respondents
Disaggregated

Questions

Disaggregated

Aggregated

Income from:
 Unprocessed forest products
 Processed forest products
 Fishing/aquaculture
 Wild products
 Wage labor
 Businesses

30 days

1 year

At least 1 adult;
Preferably 1 adult male
and 1adult female and
any other household
members

At least 1 adult;
Preferably 1 adult male
and 1 adult female and
any other household
members

Aggregated

During the past 30 days,

how much fuel wood did
your household collect (for
subsistence use and sale)?

Disaggregated (e.g.)

Step 1: Rank 10 major sources

of net annual income using
cards according to their relative
importance to your household

Aggregated

Income from:
 Agriculture
 Livestock
 Livestock
 Products
 Other

3 months

1 year

At least 1 adult;
Preferably 1 adult male
and 1adult female and
any other household
members

At least 1 adult;
Preferably 1 adult male
and 1 adult female and
any other household
members

During the past

3 months, how much
fresh cassava has your
household harvested (for
subsistence use and sale)?

Step 2: Using 50 beans,

assign relative weights to
each income category
according to the relative
importance of each category

Expenditures

1 week

1 year

1 adult male and 1

adult female

At least 1 adult;
Preferably 1 adult male
and 1 adult female and
any other household
members

What cash purchases

have you made over the
past 7 days?

Step 1: Rank 10 major

sources of cash expenditures
using cards according to their
relative importance to your
household
Step 2: Using 50 beans,
assign relative weights to each
expenditure category according
to the relative importance of each
category

ASKING QUESTIONS TO UNDERSTAND RURAL LIVELIHOODS: COMPARING DISAGGREGATED

Table 1. Summary of survey questions, recall periods and respondents for disaggregated (PEN) and aggregated (UFRIC) questionnaires

1815

1816

WORLD DEVELOPMENT

(d) Comparing disaggregated and aggregated data

In our analysis, we begin by testing whether the two samples
are considered part of one population. We do this by comparing
dierences of means for a number of demographic and socioeconomic variables. Our logic is that if we can demonstrate that
the samples are not signicantly dierent with respect to basic
socioeconomic and demographic characteristics, then dierences in measures of income and expenditures can be attributed
to the dierent data collection approaches. Our empirical approach is to compare the mean shares and rankings of income
and expenditures for the various livelihood activities for both
data collection approaches. In cases where dierences between
the two approaches are found, we do not conclude which is
most accurate. However, we do discuss potential reasons
for dierences that could be biasing the results. We present
standard deviations to investigate whether one method yields
more precise measures of mean values than the other.
The two approaches produce dierent types of data. The
disaggregated approach collects data on absolute levels of income and expenditures in units of Ugandan Shillings
(UgShs.). The aggregated approach does not involve collecting
data that would allow us to calculate absolute values with specic units, but seeks to measure the relative importance of
alternative activities. Therefore, in order to compare the results of the two approaches we make the data comparable.
We do this by calculating relative measures (i.e., shares and
ranks for each livelihood category) for each data set. We compare the results of the two data collection approaches using
two general methods. One method is to use pair-wise tests of
signicant dierences between means of shares for each type
of activity. However, because shares of each activity within
each data collection approach sum to one, if there is a large
dierence in means for one activity, there is an increased
chance that the remaining activities will also dier signicantly
between the two approaches. 9 Because of this problem, we
also rank the activity shares for each approach, and then compare the rank orders.
Another consideration is that for expenditures, the disaggregated data are collected for one adult male and one adult female in each household, whereas the aggregated approach is
based on expenditure estimates for all household members. 10
Because we do not have gender specic data for the aggregated
approach, we summed the data from the disaggregated approach across the two adults. This implies that expenditures
by children, the elderly, and other adults in the household
are not sampled in the disaggregated data set, though presum-

ably are part of the household responses in the aggregated approach. Given the nature of cash expenditures in this
particular setting, we feel that expenditure data for one adult
female and one adult male are fairly representative of total
expenditures for a household (i.e., most children and elderly
are not making signicant cash expenditures).
4. EMPIRICAL ANALYSIS
(a) How comparable are the disaggregated (PEN) and aggregated (UFRIC) samples?
Before making comparisons between our variables of interest
(i.e., income and expenditure portfolios) we investigate whether
the samples we drew for the disaggregated and aggregated surveys are comprised of comparable households (Table 2). We
conducted t-tests (i.e., two-group mean comparison tests) to
compare means of common demographic and socioeconomic
characteristics of the two groups. 11 The test assumes the variances for the two populations are the same. If two tailed p value
is <0.05 we conclude that the dierence of means between males
and females is dierent from 0. The data demonstrate that there
are few dierences between the samples. The variable minutes to
nearest forest is the only variable where we observe mean values
that are signicantly dierent between the two samples at the
5% level. Though the lack of signicant dierences in means
could also be due to large variances, we note, subjectively, that
neither the variances nor the dierences in the absolute values of
the means appear to be large. From these data we conclude that
we have administered the two separate questionnaires to relatively comparable sub-samples of households, and that if dierent methods yield dierent characterizations of livelihood
portfolios, these dierences are likely to be due to dierences
in survey methods.
(b) Household income portfolios
Estimates of the proportion of annual net household income
from various activities are presented in Table 3. Households
were asked to consider sources of subsistence and cash income, less cash payments for variable inputs and hired labor
(i.e., net income), for both the disaggregated and aggregated
questionnaires. The results indicate the disaggregated and
aggregated methods yield quantitatively dierent pictures of
the relative importance of various sources of income. The
mean values or share of total net income from various

Table 2. Descriptive statistics for disaggregated (PEN) and aggregated (UFRIC) samplesa
Variable
Forest owned by household, hectares
Arable land owned by household, hectares
Female headed household (0/1)
Household size, number
Dependency ratiob
Education of household head, years
Household head is migrant (0/1)
Household owns bike (0/1)
Household owns mobile phone (0/1)
Distance to nearest forest, minutes
a

Disaggregated (PEN) sample (N = 170)

Mean
0.47 (0.78)
1.69 (1.18)
0.17 (0.37)
5.6 (2.6)
145.3 (109.8)
4.3 (3.3)
0.23 (0.42)
0.55 (0.5)
0.13 (0.33)
11.3* (13.2)

Aggregated (UFRIC) sample (N = 89)

Mean
0.68
1.88
0.12
5.2
134.3
4.5
0.31
0.56
0.07
7.4

(1.45)
(3.14)
(0.32)
(2.6)
(115.9)
(3.2)
(0.46)
(0.54)
(0.26)
(7.8)

Values in parentheses are standard deviations.

The dependency ratio is the number of household members under 15 years plus the number of household members over 65 years divided by the number
of members between 15 and 65 years of age. The ratio is then multiplied by 100.
*
Means for the disaggregated approach are signicantly dierent than those from the aggregated approach at the 5% level.
b

ASKING QUESTIONS TO UNDERSTAND RURAL LIVELIHOODS: COMPARING DISAGGREGATED

1817

Table 3. Measures of income, shares and ranks for disaggregated (PEN) and aggregated (UFRIC) samplesa
Sources of income

Mean net income (subsistence + cash)

Disaggregated (N = 170)

Agriculture
Business
Unprocessed forest products
Wild products
Livestock
Wages
Other
Livestock products
Remittances
Processed forest products
a
*

Mean net cash income

Aggregated (N = 88)

Disaggregated (N = 170)

Aggregated (N = 88)

% Share of total

Rank

% Share of total

Rank

% Share of total

Rank

% Share of total

Rank

47.0*
(17.2)
12.7*
(17.2)
10.9*
(6.7)
10.2*
(5.9)
5.7*
(7.4)
5.1*
(9.9)
3.4
(4.5)
2.4
(4.8)
1.3*
(5.6)
1.2*
(3.0)

34.7
(17.6)
6.1
(11.0)
14.4
(6.6)
4.5
(4.7)
9.5
(8.3)
9.5
(13.0)
4.1
(8.6)
2.7
(4.4)
6.7
(13.8)
4.6
(7.1)

27.5*
(23.6)
27.5*
(28.7)
1.1*
(5.0)
0.3*
(1.3)
14.6*
(17.8)
12.5
(20.4)
10.8*
(14.9)
1.1*
(4.1)
3.0*
(11.3)
1.4*
(6.7)

39.5
(21.3)
7.2
(13.8)
4.8
(5.6)
4.9
(5.3)
10.6
(10.0)
10.9
(15.2)
5.2
(11.6)
3.0
(5.2)
8.4
(16.6)
5.5
(9.3)

2
3
4
5
6
7
8
9
10

6
2
8
3
3
9
10
5
7

1
8
10
3
4
5
8
6
7

5
9
8
3
2
7
10
4
6

Values in parentheses are standard deviation.

Means for the disaggregated approach are signicantly dierent than those from the aggregated approach at the 5% level.

livelihood activities are signicantly dierent for the two survey methods for all categories, with the exception of the relative shares of income from livestock products and other
income. 12 The largest dierences are observed for agriculture
(+12.3% for disaggregated approach), business (+6.6% for
disaggregated approach), and wild products (+5.5% for disaggregated approach).
Another way of comparing data on income portfolios is by
ranking the categories used in both surveys and observing
whether there is general consensus between the two methods
on the order of importance of each income category. In
Table 3, agriculture is ranked rst using both survey methods, for both net total and cash income, but beyond this rst
category there is considerable variation. For example, if you
wanted to design a policy intervention around the three most
important sources of net income for rural households in the
study area, the disaggregated method points to focusing
attention on agriculture, business, and the harvesting of
unprocessed forest products. Analysis of the data using the
aggregated method points to agriculture, unprocessed forest
products, and wage income. If we look at the top ve categories, three sources of net income are common to both
survey methods.
To test whether respondents are better at recalling cash (vs.
subsistence) income we present relative shares and rankings
for sources of household cash income. A limitation of collecting income data in rural settings where a considerable share
production is for home use is that it may be cognitively dicult for respondents to aggregate items they have not marketed (e.g., number of cobs of corn harvested and consumed
by all household members, or number of head loads of fuel
wood collected by women and children). With the exception
of wage income all estimates of sources of income are signicantly dierent between the two survey methods. There are
large dierences in the relative importance of agriculture
(+12% for aggregated approach), and business income

(+20.3% for disaggregated approach). Rankings of cash income suggest that there is considerable synergy between the
two methods. Agriculture, business, livestock, and wages are
among the top ve sources of cash income using both methods.
Further to the discussion above, we hypothesize that some
of these dierences may arise because the aggregated approach
is more directed toward subjective measures of importance,
while the disaggregated approach is more directed toward
objective quantitative measures of income. The subjective
measures may include considerations such as the importance
of income sources as safety nets during hard times from events
such as droughts or a household member falling ill. If this
interpretation is correct, we would expect unprocessed forest
products, livestock, wage income, and remittances to be more
important in the subjective, aggregated approach because
these activities play larger safety net functions than agriculture
and business income which are more regularized activities.
This explanation seems to be supported by the results, as local
returns to agriculture and business are likely to be disrupted
by droughts and other shocks, while forests products (processed and unprocessed), remittances, wages, and livestock
may act as safety nets for livelihoods (Dercon, 2002; Pattanayak & Sills, 2001).
Considering measures of net cash income also seems to
support this line of reasoning. For those sectors that have
cash ows that persist during droughts and other shocks
to the household, we would expect subjective measures of
their importance to be higher than the more quantitative
measures. Accordingly, as per above results, we see higher
rankings for unprocessed and processed forest products,
and remittances for aggregated approaches. But the same logic does not hold for agriculture, wild products, or livestock
which have reversed patterns compared to net total income
measures. For agriculture and wild products, higher aggregated values suggest that the cash part of income may be

1818

WORLD DEVELOPMENT
Table 4. Proportion of expenditures for various categories of goods and servicesa

Categories of expenditures

Mean expenditures with other categorya

Disaggregated
(N = 173)
Share of total
*

Other
Agricultural production and food
Livestock
Medical
Entertainment
Transportation
School fees and other education
Formal social occasions
Forest products
Fish
a
*

46.3
(27.7)
19.6
(26.3)
11.0*
(14.1)
10.3*
(17.2)
5.4*
(11.0)
3.6
(11.6)
2.5
(9.3)
1.2*
(3.8)
0.1*
(0.7)
0

Mean expenditures without other categorya

Aggregated (N = 80)

Disaggregated
(N = 173)

Aggregated (N = 80)

Rank

Share of total

Rank

Share of total

Rank

Share of total

Rank

9.2
(11.9)
23.9
(12.0)
5.8
(7.4)
23.2
(14.7)
11.2
(7.5)
3.4
(4.5)
8.9
(9.4)
5.4
(5.0)
8.9
(9.4)
0.1
(1.1)

NA

NA

NA

NA

32.9
(33.0)
23.4*
(26.3)
18.3*
(26.6)
10.9
(20.1)
6.1
(0.36)
4.1*
(14.1)
4.0
(14.4)
0.2*
(1.8)
0

25.8
(12.2)
6.1
(7.7)
26.5
(16.9)
12.5
(8.3)
3.6
(4.8)
9.7
(10.4)
6.3
(6.3)
9.3
(10.4)
0.2
(1.5)

2
3
4
5
6
7
8
9
10

7
2
3
9
5
8
5
10

2
3
4
5
6
7
8
9

7
1
3
8
4
6
5
9

Values in parentheses are standard deviation.

Means for the disaggregated approach are signicantly dierent than those from the aggregated approach at the 5% level.

important during periods of low net income ows, while net

cash income from livestock may be less important during
low income ows.
Comparisons of the standard deviations between the two
approaches suggest that the precision of estimates of means
are similar for most sources of income. Standard deviations
for all values of income shares are generally two times mean
values or smaller, with exceptions largely occurring in cases
where there are small mean values. In sum, neither method
seems to distinguish itself regarding the precision of mean income estimates.
(c) Household expenditure portfolios
The mean share of annual household expenditures attributed to various activities is summarized in Table 4. As with
the income portfolio data, there are considerable dierences
in the expenditure shares observed using the disaggregated
and aggregated data collection methods; six of the 10 categories are signicantly dierent. The most striking nding is the
importance of the other expenditure category. The collection of detailed data on weekly expenditures illustrates the relative importance of many small purchases to overall
expenditures. Data from the disaggregated questionnaire demonstrate that that there is a diverse set of expenditures that
were not accounted for in the categories that were formulated
for the PRA exercise that produced expenditure shares for the
aggregated method. For the disaggregated data we coded over
55 dierent types of expenditures reported by adult males and
adult females.
Because of the overwhelming impact of the other category
on shares, we also present results without the other category
included. For these results, there are signicant dierences in
four of the nine activities; livestock, medical, school fees,
and forest products. For livestock expenses, we suspect the

same bias is occurring as may have been the case for the
other category. Livestock includes frequent and numerous
types of expenditures that may be underestimated if not delineated with disaggregated methods. On the other hand, the
disaggregated approach, which only collected data for expenditures seven days prior to each quarterly visit, may have
missed some irregular, yet relatively large household expenditures such as medical and school fees. The high standard deviations associated with these categories for the disaggregated
approach supports this hypothesis. The rank order of household expenditure categories is also somewhat inconsistent between the two approaches. For the disaggregated approach
without the other category, the three most important expenditure categories are agricultural production and food, livestock,
and medical expenses. The aggregated method tells us that
medical expenses, agricultural production and food, and entertainment are the three most important expenditure categories.
Considering the top ve categories for each method, three categories are common. Comparisons of the standard deviations
between the two approaches suggest that the precision of estimates of means are somewhat greater for the aggregated approach, which generally has standard deviations similar to,
or lower than, mean values. In contrast, the disaggregated approach tends to have standard deviations larger than the
means.
As noted in the discussion of sampling and questionnaire design, the data we present for the disaggregated method reect
expenditures over a seven day time period (for each of four
visits) for one adult male and one adult female from each
household. Given the limited amount of cash income that
households in these communities have, and limited opportunities for spending cash, we feel that our analysis of the disaggregated data are a reasonable proxy of household expenditure
patterns for the entire household. That is, we do not feel that
we missed a large share of expenditures by failing to collect

ASKING QUESTIONS TO UNDERSTAND RURAL LIVELIHOODS: COMPARING DISAGGREGATED

1819

Table 5. Impact of seasonal timing on responses for net income.

Mean shares (%) of total net income (subsistence + cash)a

Source of income

Agriculture
Business income
Unprocessed forest products
Wild products
Livestock
Wage income
Other income
Livestock products
Remittances
Processed forest products

Disaggregated all quarters

(N = 170)

Aggregated
(N = 88)

Disaggregated third quarter

(N = 162)

47.0*
(17.2)
12.7*
(17.2)
10.9*
(6.7)
10.2*
(5.9)
5.7*
(7.4)
5.1*
(9.9)
3.4
(4.5)
2.4
(4.8)
1.3*
(5.6)
1.2*
(3.0)

34.7
(17.6)
6.1
(11.0)
14.4
(6.6)
4.5
(4.7)
9.5
(8.3)
9.5
(13.0)
4.1
(8.6)
2.7
(4.4)
6.7
(13.8)
4.6
(7.1)

37.3
(29.5)
12.9**
(21.6)
13.4
(10.5)
15.2**
(14.1)
6.4**
(12.7)
4.3**
(10.7)
3.6
(6.1)
3.2
(7.9)
1.5**
(8.0)
1.9**
(5.1)

Values in parentheses are standard deviation.

Means for the disaggregated approach are signicantly dierent than those from the aggregated approach at the 5% level.
**
Means for the disaggregated third quarter data are signicantly dierent than those from the aggregated approach at the 5% level.
*

expenditure data for children, elderly, or other household

members. Research eorts that involve expenditures should
experiment with other categories, acknowledging that ranking
and weighting more than 10 categories may be cognitively
challenging for respondents.

nding highlights the importance of recall periods, and demonstrates the potential limitations of collecting household income data using a one year recall period.
5. DIFFERENT METHODS, DIFFERENT STORIES

(d) Does seasonal timing of questionnaires aect responses?

Recall that the disaggregated data were collected over four
quarters in order to capture data through a full calendar year.
The aggregated data were collected once during a 2 week period that overlapped with the third quarter of data collection
for the disaggregated survey. Ideally, and consistent with the
questions asked, we would like the aggregated responses to reect the entire previous year. However, as discussed above,
limited ability to recall information could cause the responses
to be biased by what has happened recently, in this case during
the third quarter. If this is the case we would expect responses
with the aggregated method to be more similar to the third
quarter results of the aggregated method than to the results
for the full year. Table 5 presents the results of this comparison for net income. The results of the comparison between
the disaggregated third quarter data and the aggregated data
suggest that the aggregated results are inuenced by recent
livelihood activities for some categories. Whereas almost all
sources of income from the disaggregated all quarters approach are signicantly dierent to the aggregated results,
we nd that there are no signicant dierences between responses for third quarter income in the disaggregated data collection and the aggregated approach for four categories:
agriculture; unprocessed forest products; livestock products;
and other sources of income. 13 The ndings suggest that the
aggregate approach is biased toward the specic season during
which the data are collected for two very important categories
of income, agriculture and unprocessed forest products. The

Household livelihood data are widely used by development

practitioners, donors, researchers, and other stakeholders for a
diversity of purposes including: characterization of rural livelihoods; targeting development interventions; evaluating the
success of a policy or program; etc. For any of these types
of activities, monitoring or research initiatives having data
that accurately portray rural livelihoods are essential. However, what we have learned through our experiment in using
dierent data collection methods to elicit the same information
from comparable samples is that dierent methods yield dierent stories. We consider the question of the relatively importance of forest products to rural households using both the
disaggregated and aggregated datasets.
(a) How important are forests to rural livelihoods in western
Uganda? (disaggregated)
Rural households in western Uganda derive 10.9% of their
total annual net income from unprocessed forest products
such as fuel wood, bamboo, mushrooms, and other forest
foods, and only 1.2% of income from processed forest products including charcoal, sawn wood, woven products, etc. Income from agriculture and business are both more important
than forest income (47.0% and 12.7% respectively). The majority of forest products are harvested for subsistence use by
households, with only a very small share (approximately
2.5%) of total cash income coming from the sale of forest
products. Households in western Uganda seldom purchase

1820

WORLD DEVELOPMENT
Table 6. Determinants of share of forest incomea,b

Forest owned by household, hectares

Arable land owned by household, hectares
Female headed household (0/1)
Dependency ratioc
Household head is migrant (0/1)
Education of household head, years
Owns bike (0/1)
Owns mobile phone (0/1)
Time to nearest forest, minutes
N
Rsquared

Disaggregated

Aggregated

2.355***R
(0.844)
1.772***R
(0.597)
0.959
(1.788)
0.006
(0.006)
1.900
(1.461)
0.218
(0.202)
1.287
(1.318)
0.030**R+
(0.047)
0.029
(0.048)
167
0.1663

0.992
(1.044)
0.304
(0.481)
0.623
(3.644)
0.031***
(0.009)
6.321***
(2.306)
0.409
(0.339)
1.362
(2.097)
10.752**
(4.161)
0.116
(0.134)
81
0.2815

Values in parentheses are standard errors.

R denotes robust nding for sign of the coecient when comparing the two models; R+ denotes robust nding both in sign and level of signicance
between the two models.
c
The dependency ratio is the number of household members under 15 years plus the number of household members over 65 years divided by the number
of members between 15 and 65 years of age. The ratio is then multiplied by 100.
*
Indicate statistically signicantly at 10% respectively.
**
Indicate statistically signicantly at 5% respectively.
***
Indicate statistically signicantly at 1% respectively.
b

forest products. Data on household expenditures by one adult

male and one adult female from each household suggest that
rural households are on average spending 0.1% of their cash
income on forest products. The largest shares of cash expenditures are for agriculture, livestock, and medical expenses
(19.6%, 11.0% and 10.3% respectively). Rural households in
this region have surprisingly diverse expenditure patterns with
a large number of items falling outside of standard expenditure categories (46.3%).
(b) How important are forests to rural livelihoods in western
Uganda? (aggregated)
Rural households in western Uganda derive 14.4% of their
total annual net income from unprocessed forest products
such as fuel wood, bamboo, mushrooms and other forest
foods, and 4.6% of income from processed forest products
including charcoal, sawn wood, woven products, etc. Income
from agriculture is the only category that is more important
than forest income (34.7%). Livestock and agricultural wage
labor are the third and fourth most important sources of income for rural households. Approximately 10% of annual
cash income for households in the study area comes from
the sale of forest products. Our analysis suggests that households are selling almost half of all harvested forest products
for cash income. In addition to selling forest products,
households in western Uganda purchase forest products.
Nine percent of total cash expenditures are on unprocessed
or processed forest products. After agricultural production
and food, medical expenses, entertainment, and miscellaneous expenditures, forest products are the 5th most important expenditure.

(c) Determinants of the relative importance of forest income

We estimate a simple OLS regression model to explain the
determinants of the share of household income from forest
products. We consider endowments of forested and agricultural land, basic household demographics including indicators
of human capital, ownership of assets, and the distance to the
nearest forest according to the following specication:
Y i b0 b1 land b2 labor b3 capital b4 minforest ei
1
Access to forests is expected to be an important determinant of
the share of income a household derives from forests. We expect
a positive relationship between forest owned and household income derived from forests, and a negative relationship between
the distance from the households to the nearest forest and the
share of income from forests. Households that have to travel
further to collect forest products will gather fuel wood, wild
foods, and other common forest products from other land types
including fallows and bush lands. In many settings in sub-Saharan Africa we nd a correlation between forest dependence
and poverty suggesting that female headed households, households with high dependence ratios, and households with limited
assets should be more dependent on forests. We do not have a
strong hypothesis about the relationship between migration
and forest dependence in the study area. Migrant households
may be more dependent on forests if they have limited land
holdings and portfolios dominated by relatively few income
sources. However, migrant households may have trouble gaining access to forested areas, which would limit the share of income from forests. Regression results are presented in Table 6.

ASKING QUESTIONS TO UNDERSTAND RURAL LIVELIHOODS: COMPARING DISAGGREGATED

Relatively few of the ndings are robust across datasets. We

conclude that forest ownership is a very important determinant of forest income, and that distance to nearest forest is
negatively, but not signicantly correlated with forest dependence. We also draw the conclusion that having a large endowment of arable land reduces forest dependence. Using the
disaggregated data none of the demographic or human capital
variables are signicant. We nd that ownership of a mobile
phone, an important indicator of wealth in the study area, is
negatively associated with forest dependence.
Regression results for the aggregated data highlight a dierent set of relationships. We nd that forest ownership is positively correlated with forest dependence, and that ownership
of arable land is negatively correlated with forest dependence,
however, neither of these ndings is statistically signicant. A
high dependency ratio, which is an indicator of large numbers
of children and elderly in a household relative to the number
of productive adults in the household, is a statistically signicant determinant of forest dependence. In the model using the
aggregated data there is a signicant and negative relationship
between migration and forest dependence, suggesting that
households that do not have longstanding ties to the community are less forest dependent. This could be interpreted as a
limitation on access to forest resources for households that
are relatively new to a community. As with the regression
model using the disaggregated data, mobile phone ownership
is negatively correlated with forest dependence, suggesting
that wealthier households are less reliant on forests.
The major point is that the stories derived from these two
regression models are slightly dierent. In the disaggregated
model we emphasize the importance of ownership of forests,
access to forests, and relative wealth as evidenced in mobile
phone ownership as important determinants of forest dependence. We summarize the story from the aggregated dataset
as one of wealth determining forest dependence, large households with fewer productive adults are dependent on forests,
and households with mobile phones are less dependent on forests. If our interpretation of migrant households having limited access to forests is accurate then we also conclude that
access to forests is important, but we cannot conrm this given
the lack of signicant coecient on the forest ownership variable. Further the positive relationship between time to the
nearest forest and forest dependence muddies the story; we
hypothesize a negative relationship between these variables
that is supported by the analysis of the disaggregated data.
6. DISCUSSION AND CONCLUSIONS
Our goal in designing this study is to understand how two of
the methods commonly used to capture data on rural livelihoods inuence the calibration of indicators frequently used
for understanding the lives of the poor, targeting development
interventions, and monitoring and evaluating the eectiveness
of policies, programs, and projects. Our results demonstrate
that we obtain dierent views of the relative importance of rural livelihood strategies across indicators of income and expen-

1821

diture, depending on the method used to collect the data.

Ranking data are also dierent using disaggregated vs. aggregated methods, though there is considerable overlap in the categories that show up as most important for estimates of the
importance of income sources. We nd evidence of seasonal
bias in the reporting of the relative importance of agriculture
and unprocessed forest products using more aggregated data
collection methods, suggesting that disaggregated methods focused on capturing seasonal variation may be more accurate.
Despite evidence of seasonal bias, the puzzle for us as we seek
to draw lessons from this research is that we cannot say with
condence which approach, disaggregated or aggregated, provides the most accurate representation of the truth. One obvious
solution to the problem of diering ndings on the relative
importance of livelihood strategies is to triangulate data collection eorts. However, the challenge for the use of mixed methods is that disaggregated data need to be processed before we
can begin to identify trends and patterns at the aggregate level.
For accuracy to be veried through triangulation, researchers
would need to be able to process disaggregated data on the spot
and highlight conicting observations immediately. 14 Further,
we are cognizant of the additional time and resources involved
in collecting data at multiple levels of aggregation.
Another important question is what the data are to be used
for. Generating a broad understanding of livelihood strategies
in rural settings is a dierent proposition than developing
behavior models. What we do know is that researchers and
practitioners are employing both methods in eld settings
and that their ndings are frequently drawn upon to inform
policy. Researchers and development practitioners should
strive to collect both accurate and precise data, taking care
to understand and qualify ndings in light of the apparent
strengths and weaknesses of various data collection methods.
All too frequently we are quick to judge the method supported
by our disciplinary training as the most accurate.
There are several opportunities for future research on the topic of the appropriate level of aggregation for collecting data
on livelihood indicators. Our experiment could be tested relatively easily in other settings at the household level to see if the
results are generalizable. In particular, looking across settings
with higher and lower levels of market integration would help
us better understand the importance of level of aggregation in
designing data collection instruments. It may be that households more fully integrated into the market economy provide
disaggregated and aggregated data that are more similar. Finally, another direction for future research is to use innovative
methods for collecting data on true income and expenditures of respondents. As noted in the introduction to this paper, direct observation of a large sample of households for the
duration of a year is not generally feasible. However, technology is rapidly evolving, and the use of devices such as GPS
units, PDAs, and cellular phones may be used to collect and
transmit data in real time. Such approaches may present an
important opportunity to collect data that would allow for triangulation with conventional survey methods to enhance our
understanding of rural livelihoods.

NOTES
1. For a discussion on issues associated with asking questions of dierent
household members, see Fisher et al. (2010).

2. As mentioned above, there are numerous issues in addition to data

quality, such as the role of local participation in generating information,
which have been discussed elsewhere and are beyond the scope of this
paper.

1822

WORLD DEVELOPMENT

3. Explanations that relate the target analogy to concepts such as

accuracy, precision, bias, and variance are widespread (e.g. Wonnacott &
Wonnacott, 1977).
4. The PEN data were collected as part of a Poverty Environment
Network (PEN) study undertaken in western Uganda. PEN is a network
of roughly 35 Ph.D. scholars contributing to a global comparative
database on the relationship between forest use and poverty in the low
income tropics. PEN is administered by the Center for International
Forestry Research (CIFOR). See http://www.cifor.cgiar.org/pen/_ref/
home/index.htm. Modules on household expenditures and time use were
added to the core PEN questionnaire.
5. The UFRIC data were collected as part of the Sustainable Agriculture
and Natural Resource Management Collaborative Research Support
Program (SANREM-CRSP) Long-Term Research Grant Decentralization
Reforms and Property Rights: Potentials and Puzzles for Forest Sustainability and Livelihoods. The project is a collaboration between: Workshop
in Political Theory and Policy Analysis, Indiana University; CGIAR
System Wide Program on Collective Action and Property Rights
(CAPRi); and CIFOR. The Uganda Forestry Institutions and Resource
Center (UFRIC) at the Faculty of Forestry and Nature Conservation,
Makerere University is the local collaborator on the project.
6. The total population of the four sub-counties which the study takes
place in (i.e. Kiziranfumbi (Hoima District)), Kakindo, Kasambya, and
Kiryanga (all in Kibaale District) was estimated to be 84,587 in 2002
(UBOS 2006). We estimate that our sample includes approximately 1,200
individuals. The participants in our study represent approximately 1.4% of
the total population of the four sub-counties in which the study took
place.
7. Under the freehold system landowner property rights are thought to
be secure; landowners hold registered land indenitely. The mailo tenure
system was established by the British colonial government in 1900. Legal
land titles were given to the Buganda royal family. The land was measured
in square miles which is where the term mailo comes from. Land was
partitioned into smaller units and rented out to tenants. Under the 1998

Land Act tenants were granted freehold status for mailo parcels held since
1986. While the 1998 Land Act clearly outlines the provisions of various
land tenure systems and land rights, the act is weakly enforced (Nkonya et
al., 2004).
8. The categories of wage income, business income, and remittances are
assumed to be entirely cash income.
9. Note that the problem of interdependent observations is an inherent
problem in using data that rely on relative rather than absolute measures.
10. In the study area the average number of adults per household is 2.2
(i.e. those over the age of 15).
11. We calculated means and standard deviations for the sub-population
of the disaggregated dataset (N = 28) for the villages where the
aggregated survey was conducted. These households are not statistically
signicantly dierent from the other households in the disaggregated
sample for variables important to forest-based livelihoods including
hectares of forest owned by the household, distance to forest etc. There
were dierences with respect to the number of migrant households (40%),
and asset ownership (bicycle and mobile phone ownership was 64% and
21% respectively).
12. Other income includes: support from the government or NGOs; gifts
including condolences; pension income; payments for forest services;
payments for renting out land/houses; sale of land/standing trees; etc.
13. Recall that we hypothesize above that results for forest products
from the aggregated approach may be biased by the strategic behavior of
households.
14. Recent innovations in tracking household activities (cf. Shirima et al.
(2007) who use PDAs to collect and aggregate data in real time) could help
us understand for a set of representative households what the most
accurate representation of income, expenditures, and time use.

REFERENCES
Bradley, S. (1995). How people use pictures: An annotated bibliography.
London, UK: International Institute for Environment and Development.
CARE. (2002). Household livelihood security assessments: a toolkit for
practitioners. Atlanta, GA: CARE, USA.
Campbell, J. (2002). A critical appraisal of participatory methods in
development research. International Journal of Social Research Methodology, 5(1), 1929.
Campbell, B. M., Jerey, S., Kozanayi, W., Luckert, M. K., Mutamba,
M., & Zindi, C. (2002). Household livelihoods in semi-arid regions:
Options and constraints. Bogor, Indonesia: Center for International
Forestry Research.
Campbell, B. M., & Luckert, M. K. (Eds.) (2002). Uncovering the hidden
harvest: Valuation methods for woodland and forest resources. People
and Plants Conservation Series. London, UK: Earthscan Publications
Inc.
Carney, D. (Ed.) (1998). Sustainable rural livelihoods: What contribution
can we make?. London, UK: DFID.
Cavendish, W. (2000). Empirical regularities in the poverty-environment
relationship of rural households: Evidence from Zimbabwe. World
Development, 28(11), 19792003.
Cavendish, W. (2002). Quantitative methods for estimating the economic
value of resource use to rural households. In B. M. Campbell, & M. K.
Luckert (Eds.), Uncovering the hidden harvest: Valuation methods for
woodland and forest resources. London, UK: Earthscan Publications
Ltd.
Chambers, R. (1994). The origins and practice of participatory rural
appraisal. World Development, 22(7), 953969.

Davies, R. (1997). Beyond wealth ranking: The democratic denition and

measurement of poverty. Available from: <www.mande.co.uk/
docs.democrat.htm>.
Davis, J., & Whittington, D. (1998). Participatory research for
development projects: A comparison of the community meeting and
household survey techniques. Economic Development and Cultural
Change, 47(1), 7394.
Dercon, S. (2002). Income risk, coping strategies, and safety nets. World
Bank Research Observer, 17(2), 141166.
Ellis, F., & Freeman, H. A. (2004). Rural livelihoods and poverty
reduction strategies in four African countries. Journal of Development
Studies, 40(4), 130.
Fisher, M., Reimer, J. J., & Carr, E. R. (2010). Who should be interviewed
in surveys of household income?. World Development, 38(7), 966973.
Jones, B. D. (2001). Politics and the architecture of choice: Bounded
rationality and governance. Chicago, IL: University of Chicago Press.
Krishna, A. (2005). Stages of progress eld manual: a community based
methodology for dening and understanding poverty. Version 2.0.
Available from: <http://sanford.duke.edu/krishna/SoP.pdf>.
Krishna, A., Kristjansen, P., Radeny, M., & Nindo, W. (2004). Escaping
poverty and becoming poor in 20 Kenyan villages. Human Organization, 5(2), 211226.
MAAIF. (1995). Basic facts on agricultural activities in Uganda.
Kampala, Uganda: Ministry of Agriculture, Animal Industry and
Fisheries.
Menton, M., Lawrence, A., Merry, F., & Brown, N. D. (2010). Estimating
natural resource harvests: Conjectures?. Ecological Economics, 69,
13301335.

ASKING QUESTIONS TO UNDERSTAND RURAL LIVELIHOODS: COMPARING DISAGGREGATED

Nkonya, E., Pender, J., Jagger, P., Sserunkuuma, D., Kaizzi, C. K., &
Ssali, H. (2004). Strategies for sustainable land management and poverty
reduction in Uganda. IFPRI Research Report No. 133. Washington,
DC: International Food Policy Research Institute.
Pattanayak, S. K., & Sills, E. O. (2001). Do tropical forests provide
natural insurance? The microeconomics of non-timber forest
product collection in the Brazilian Amazon. Land Economics,
77(4), 595612.
Schrekenberg, K., Camargo, I., Withnall, K., Corrigan, C., Franks, P.,
Roe, D., et al. (2010). Social assessment of protected areas: A review of
rapid methodologies. A report for the Social Assessment of Protected
Areas (SAPA) Intiative. London: International Institute for Environment and Development.
Scoones, I. (1995). Investigating dierence: Applications of wealth ranking
and household survey approaches among farming households in
southern Zimbabwe. Development and Change, 26(1), 6788.

1823

Shirima, K., Mukasa, O., Armstrong Schellenberg, J., Manzi, F., John,
D., Mushi, A., et al. (2007). The use of personal digital assistants for
data entry at the point of collection in a large household survey in
southern Tanzania. Emerging Themes in Epidemiology, 4(5), 18.
Sjaastad, E., Angelsen, A., Vedeld, P., & Bojo, J. (2005). What is
environmental income?. Ecological Economics, 55, 3746.
UBOS. (2006). 2002 Uganda population and housing census. Kampala,
Uganda: Uganda Bureau of Statistics.
Vedeld, P., Angelsen, A., Sjaastad, E., & Berg, G. K. (2004). Counting on
the environment: Forest incomes and the rural poor. In Environmental
Economics Series, No. 98. Washington, DC: World Bank.
Wonnacott, T., & Wonnacott, R. (1977). Introductory statistics for
business and economics (2nd ed.). New York: Wiley.