Professional Documents
Culture Documents
http://wmr.sagepub.com/
Published by:
http://www.sagepublications.com
On behalf of:
Additional services and information for Waste Management & Research can be found at:
Email Alerts: http://wmr.sagepub.com/cgi/alerts
Subscriptions: http://wmr.sagepub.com/subscriptions
Reprints: http://www.sagepub.com/journalsReprints.nav
Permissions: http://www.sagepub.com/journalsPermissions.nav
Citations: http://wmr.sagepub.com/content/28/3/269.refs.html
ISSN 0734242X
Waste Management & Research
2010: 28: 269280
DOI: 10.1177/0734242X10361508
Sara Giorgi
Brook Lyndhurst, London, UK
David C. Wilson
Research Managing Agent, Defra Waste and Resources Evidence Programme, and Department of Civil and Environmental Engineering, Imperial
College London, London, UK
This paper presents one strand of the findings from a comprehensive synthesis review of the policy-relevant evidence on household waste prevention. The focus herein is on how to measure waste prevention: it is always difficult to measure what is not there.
Yet reliable and robust monitoring and evaluation of household waste prevention interventions is essential, to enable policy
makers, local authorities and practitioners to: (a) collect robust and high quality data; (b) ensure robust decisions are made
about where to prioritize resources; and (c) ensure that waste prevention initiatives are being effective and delivering behaviour
change. The evidence reveals a range of methods for monitoring and evaluation, including self-weighing; pre- and post-intervention surveys, focusing on attitudes and behaviours and/or on participation rates; tracking waste arisings via collection data
and/or compositional analysis; and estimation/modelling. There appears to be an emerging consensus that no single approach
is sufficient on its own, rather a hybrid method using a suite of monitoring approaches usually including surveys, waste tonnage data and monitoring of campaigns is recommended. The evidence concurs that there is no benefit in trying to further
collate evidence from past waste prevention projects, other than to establish, in a few selected cases, if waste prevention behaviour has been sustained beyond cessation of the active intervention campaign. A more promising way forward is to ensure that
new intervention campaigns are properly evaluated and that the evidence is captured and collated into a common resource.
Keywords: waste prevention, household waste, behaviour change, monitoring, evaluation, reduction, reuse
Introduction
This paper presents one strand of the findings from a wider
research project, commissioned by the Waste and Resources
Evidence Programme (WREP) of the UK Department for
Environment, Food and Rural Affairs (Defra), to consolidate the policy-relevant evidence base on household waste
prevention. The scope included not only WREP and other
UK governmental evidence work, but also internationally
available, policy-relevant evidence (Brook Lyndhurst, SMP
and RRF 2009). A summary of the broader review is presented in this special issue of the journal (Cox et al. 2010).
Waste prevention is at the top of the waste hierarchy;
however, it is notoriously difficult to measure. The problem
Corresponding author: Veronica Sharp, The Social Marketing Practice, 132 Lydalls Road, Didcot, Oxfordshire OX11 7EA.
E-mail: veronica@socialmarketingpractice.co.uk
Received 9 August 2009; accepted in revised form 29 November 2009
269
Methodology
The same methodology as outlined in Cox et al. 2010 was
adopted. A selection of evidence sources was identified in the
main project, on which a further review was undertaken to
understand the range of approaches used to monitor and evaluate household waste prevention. These sources were selected on
the basis of whether a robust monitoring and evaluation methodology had been adopted and on the depth in which the results
of that evaluation were presented. A total of 25 evidence sources met these criteria, of which 15 had significantly more data
than the others. Many of these sources were also those where
evidence was presented on both the delivery and impact of a
range of waste prevention initiatives (Sharp et al. 2010).
A list of the sources is provided in Table 1 with further citings included in the references. In particular, the WREP
270
portfolio of projects and a handful of others provide instructive examples of methodologies which have been explicitly
tested, and where the approaches used are recorded in detail
notably Project REDUCE (Waste Watch 2006, 2007a, b
[WR0105]), Dorset project (Dorset County Council et al.
2008 [WR0116]), Small Changes Big Difference (Hampshire County Council & Brook Lyndhurst 2008 [WR0117])
and GAP EcoTeams (GAP et al. 2008 [WR0114]). It is the
four WREP sources which provide the most substantive and
up-to-date evidence base on monitoring and evaluating
approaches. Other UK and international examples have been
found and cited in Table 1. However, there were few international examples where the monitoring and evaluation
approach adopted was presented in sufficient detail or in such
a way so as to derive whether it was effective or otherwise.
Context applied
Use of collection round data Uses a mix of monitoring techniques, The Waste Wise Armadale Project (Changeworks 2008, SISTech
to accurately measure waste e.g. tracking waste arisings via waste
2008)
arisings
tonnage/collection round data and
Household Waste Prevention Activity (Dorset County Council et
surveys, upon which to evaluate
al. 2008 [WR0116])
impact of campaigns.
Use of control and pilot
groups
Only two of the source projects undertook pilot & control research
(Dorset County Council et al. 2008 [WR0116], Changeworks 2008,
SISTech 2008); a third considered the option but discounted it as
impractical for the particular delivery model (Hampshire County
Council & Brook Lyndhurst 2008 [WR0117])
Attitude and behaviour surveys including metrics, interviews and focus groups
(outcome focused)
Participation surveys
(or participation monitoring)
including enquiries to help
lines, web statistics, number
of registrants, publications
disseminated, etc.
(output focused)
Compositional analysis
Conversion factors,
estimates and modelling
Using conversion factors, proxies and Potentials for the prevention of municipal solid waste, Vienna
ratio model to derive detailed figures
case study (Salhofer et al. 2008)
on consumption and waste genera- FRN average weights database (FRN 2009)
tion.
North London Watch your Waste Week (NLWA 2009)
Love Food Champions (WRAP and the Womens Institute 2008)
Hybrid a combination of
any one or more of the
above approaches
Uses a mix of monitoring and evalua- North London Watch your Waste Week (NLWA 2009)
tion techniques.
What not to Waste, Western Riverside (Waste Watch 2006
[WR0105])
Small Changes Big Difference (Hampshire County Council &
Brook Lyndhurst 2008 [WR0117])
A Good Practice Guide to Monitoring and Evaluation
(WRAP 2006)
271
272
helpful to report on the reductions in household waste arisings that this type of method has yielded.
GAP has calculated that EcoTeams, achieves a reduction
in total household waste arisings of 0.62 kg household1
week1 (GAP et al. 2008, GAP 2009).
RoWAN (Wickens 2005), a 13-month project monitored
50 households (objective was 100), achieved a 22% reduction of total household waste arisings equivalent to 1.87 kg
household1 week1.
What not to Waste (Waste Watch 2007a [WR0105]), a
6-week challenge, monitoring 14 participants claims a
34% reduction in total household waste arisings equivalent to 4.3 kg household1 week1.
Love Food Champions (WRAP and the Womens Institute
2008), a 4-month project monitoring between 4080 participants, achieved a 50% reduction in their food waste
equivalent to 2.5 kg household1 week1.
North London Watch Your Waste Week (NLWA 2009), a
1-week campaign recruited 124 participants to weigh (or
visually estimate) their waste, achieved a 50% reduction in
household waste arisings equivalent to 5.97 kg household1
(it is not clear whether this included recycling or not).
which was equivalent to 0.98 kg household1 week1 reduction in total household waste arisings.
Dorset County Council et al. (2008 [WR0116]) achieved a
reduction, in its target area of 1577 households of 2%
which was equivalent to 0.5 kg household1 week1 in total
household waste arisings.
A number of challenging considerations had to be taken into
account by both projects, for example, changes to collection
regimes, installation of new bring banks, missed collections
(due to inclement weather), vehicle breakdowns, county-wide
and national campaigns, media, unoccupied new homes.
Disadvantages
Cost effective
Easy to access participants
Usually available to recruit
273
Participation monitoring helps to gauge the impact, of specific intervention campaign activities, for example number of
mail preference service registrations, number of home compost bins sold, number of nappy vouchers given out.
The most extensive participation work was undertaken by
WRAP (2007) to provide detailed estimates of new and
existing home composting households. This work involved a
telephone survey of 20 000 households across Great Britain,
a household questionnaire in Scotland, and a telephone survey of 6000 home composters. This work was supported by
compositional analysis.
Participation monitoring was conducted by Dorset County
Council et al. (2008 [WR0116]) to determine the number of
waste reduction packs, reuse directories and jute bags disseminated. In addition, the individual visits by the doorstep
team and the materials they disseminated were recorded. To
provide an indication of the response by the public to targeted campaign activities, namely participation monitoring,
the number of registrations to the Mail Preference Service
and number of home compost bins sold were also monitored.
Compositional analysis
Compositional analysis is undertaken to understand different waste materials and the impact of an intervention or
campaign on reducing the waste materials. Waste composition analysis was undertaken by WRAP (2007) to provide
detailed estimates of new and existing home composting
households. This work involved two rounds of waste composition analysis and observational studies of compost bin use,
land use in gardens, and follow-up work to assess changes in
behaviour. This work was supported by participation surveys.
Hampshire County Council & Brook Lyndhurst (2008
[WR0117]) conducted a small-scale composition analysis to
investigate whether any changes occurred in types of waste
material as a result of working with participants. A weeks
worth of residual and dry mixed recycling waste (that would
usually be put out for kerbside collection) was taken from
four volunteer participants. This exercise was then repeated
8 months later to see if there had been any changes in the
composition of the waste.
274
guidance (WRAP 2006) and was identified in other literature (see Table 1). Waste Aware Scotlands Waste Prevention
Programme has taken a similar approach to monitoring and
evaluating its programme (Waste Aware Scotland, undated).
Each initiative has a specific monitoring and evaluation
stream including attitudinal surveys, web statistics, composition analysis and particiation surveys.
The North London Watch Your Waste Week (NLWA
2009) also used a suite of monitoring and evaluation methods to assess the impact of its intervention campaign. These
comprised measurement of total quantities of collected
wastes (using waste collection data), changes in uptake of
specific interventions, such as the use of real nappies, and
measurement of declared behaviour (using pre- and postsurveys). Where participants recorded the proportion of
waste they discarded, conversion factors, as outlined above,
were used to report the impact in weight terms.
Small Changes Big Difference (Hampshire County Council & Brook Lyndhurst 2008 [WR0117]) employed a range of
quantitative and qualitative techniques. The methods used
included baseline and follow-up surveys, analysis of diary
entries completed by participants, analysis of data recorded
on weight of residual and recycling waste, qualitative feedback on activities and events, in-depth interviews and focus
groups with participants.
Discussion
Waste prevention is notoriously difficult to measure and yet
reliable and robust monitoring and evaluation methods are
essential to enable policy makers, local authorities and practitioners to ensure that waste prevention initiatives are being
effective and delivering behaviour change. Very little is understood about how to monitor and evaluate waste prevention
particularly among local authority waste managers who are
most likely to implement intervention campaigns. The greatest
value is to them, to other delivery organizations, and to the
government establishments that support them.
The discussion is presented in the following two parts.
1. The key factors to consider when using the monitoring
and evaluation methods discussed.
2. The cross-cutting issues identified from the literature
review.
Key factors
The following provides a broad discussion on each of the
monitoring and evaluation methods. A summary of their relative strengths and weaknesses is provided in Tables 3 and 4.
275
Table 3: Strengths of using household waste prevention monitoring and evaluation methods.
Method
Self-weighing, monitoring or
reporting
Effective in engaging and connecting participants people have been found to value the personal
approach/direct contact puts participants in touch with their waste
Provides visible and immediate tonnage reduction
Provides motivational feedback to participants
Auditing is supported by education and promotion
Observational monitoring can provide alternative to weighing
Diaries tend to work better with regular feedback on performance
The invisibility of waste prevention, particularly the many small, in-home activities, means that participation can only be monitored through self-reports by participants
Hybrid approaches a combina- Provides the context for built-in pre and post surveys with interim self-weighing or observation
tion of any one or more of the
reporting
above approaches
Enables mixed approaches to be used in both short and long-term monitoring and evaluations
Enables longer term studies to build in interim monitoring and evaluation, e.g. surveys and waste
tonnage data
The results from one method can be used as a check on another (e.g. focus groups acting as check
on survey data)
Enables compositional analysis of data to be introduced to the monitoring and evaluation process
Using control and pilot groups requires a number of considerations, as listed here.
Long-term and careful planning is required.
There should be no changes to service provision during
the monitoring period (in the control area).
Population samples should be closely matched.
276
Table 4: Weaknesses of using household waste prevention monitoring and evaluation methods.
Method
Self-weighing, monitoring or
reporting
Inconsistent data can be derived due to different start/stop times, new entrants, incomplete diaries, and
lack of buy-in from participants for weighing waste
Can be in-effective in generating robust statistical evidence of weight-based reductions
Conversion factors are needed to translate measurements (from observational analysis) and this can
reduce the robustness of the results
High drop-out rates are experienced as project progresses (up to 50% in one case)
Is a resource intensive approach, with regular data collection and processing required
There are risks of self-selecting samples
Sample sizes can be too small to be statistically robust
Some people dont want to weigh their waste
The data analysis can be distorted and compounded by considerable fluctuations due to external factors difficult to identify and impossible (in some cases) to quantify
Detailed and careful planning is needed to ensure similar populations/collection systems where
there are no future interventions or changes envisaged in the control. This can be time and resource
intensive
Quality and detailed waste collection data is essential but challenging to derive because this is not held
by many local authorities to the extent that it is needed for this type of approach
Difficulties can be experienced using ACORN to accurately match populations
This approach cannot be used if communities are not geographically defined
Sample sizes can be too small or respondent rates too low to be sufficiently representative or robust
If not carefully managed, evaluators can be swamped by too much survey data with little resource to
evaluate responses effectively
Using a Citizen Panel can bias the sample. However, this route is often the most popular used by local
authorities because it is quick and cost effective to run
Self-completion surveys can give potential for self-selection and bias
Focus groups are not suitable for collecting weight data
Comparative analysis between survey and waste data can be limited if survey questions are not carefully
constructed
Hybrid approaches a combi- Requires careful planning and integration between evaluation of data monitoring, surveys and selfnation of any one or more of
weighing/observation
the above approaches
Use of collection round data cannot be used to monitor the impact of waste prevention interventions
on dispersed communities, i.e. not geographically based
277
The following provides a general discussion on the cross-cutting issues identified from the literature. The discussion considers factors such as determining impact and use of indicators, timeframe, and costs and resources.
Cross-cutting issues
278
Timeframes
Recommendations
Recommendations for improving the monitoring and evaluation of household waste prevention are listed here.
Provide the necessary skills to local authorities to obtain
and manage good quality collection round/tonnage data,
in sufficient detail, and in a consistent manner so that
they can interrogate and use this for measuring household waste prevention effectively. This also applies to
the ability to link compositional analysis and tonnage
data on the one side and attitudinal survey data on the
other.
Develop good practice guidance for local authorities and
other delivery organizations on how to use different monitoring and evaluations approaches that can be tailored to
specific interventions, for example, working closely with
small groups of households will require skills in using selfweighing, observation (for which conversion metrics would
be needed), and waste auditing approaches. The aim
should be to start with a specific type of intervention and
then design in costed options of monitoring and evaluation
(as part of the planning process). Whereas WRAPs guidance (WRAP 2006) does not advocate the use of control
and pilots, Dorset County Council et al. (2008 [WR0116])
recommend that there are lessons to be derived from
using this type of approach more generally; namely how to
derive and use good quality collection round data, and
WRAP in the UK are already addressing some of these recommendations, for example, revising its monitoring and
evaluation guidance which could be adapted to suit different
project designs. Guidance is also being provided on how to
develop strategic targets for waste prevention via the Waste
Prevention Toolkit (WRAP 2009b). The toolkit will be supported by a training programme for local authorities to help
bridge the skills gaps outlined above. In its new guidance,
WRAP has also developed a series of recommended waste
prevention survey questions.
Finally, the review concurs that there is no further benefit
to be obtained in trying to collate evidence from past waste
prevention projects. The quality of data and reporting likely
to be found does not justify the effort. A more promising way
forward is to ensure that new campaigns and initiatives are
being properly evaluated (by providing support if necessary)
and that the evidence is captured and collated into a common resource as it is completed.
279
Acknowledgements
The evidence review on which this paper is based was funded
by the Waste and Resources Evidence Programme (WREP)
of the UK Department for Environment, Food and Rural
Affairs (Defra). Defra is committed to developing policy
from an evidence-based platform; and this project was commissioned to provide such an evidence base for household
waste prevention. The review is not a statement of policy;
and the inclusion of or reference to any given policy should
not be taken to imply that it has, or will be, endorsed by
Defra as an option for England. The views expressed in this
paper are those of the authors, and do not necessarily reflect
those of Defra.
References
Note: Some of the references are to research projects commissioned by
Defras Waste and Resources Evidence Programme (WREP) or other Defra
science programmes. Full copies of the published reports for such projects
(marked in this reference list with an asterisk*) are available via the Defra
Research and Development web portal, http://randd.defra.gov.uk/ click on
search and enter the reference number (e.g. WR1204, EV02004) as the keyword.
Brook Lyndhurst and Waste Watch (2006) WR0504: Establishing the
Behaviour Change Evidence Base to Inform Community-based Waste
Prevention and Recycling. Brook Lyndhurst. A project for Defras
WREP*.
Brook Lyndhurst, SMP (The Social Marketing Practice) and RRF
(Resource Recovery Forum) (2009) WR1204: Household Waste Prevention Evidence Review. A project for Defras WREP*.
Changeworks (2008) (Fletcher, S.I., Tucker, P. & Speirs, D.) The Waste
Wise Armadale Project Final Report. http://www.changeworks.org.
uk/uploads/WWA%20Final%20Report.pdf (accessed 28 July 2009).
Cox, J., Giorgi, I., Sharp, V., Strange, K., Wilson, D.C. & Blakey, N. (2010)
Household waste prevention a review of evidence. Waste Management & Research, 28, 193219.
Cox, J., Wilkins, C., Ledsom, A., Drayson, R. & Kivinen, E (2009)
EV02004. Environmental Action Fund (EAF): A Review of Sustainable Consumption and Production Projects (SCP2.2). A report to the
Department for Environment, Food and Rural Affairs. Defra, London.
Dorset County Council, AEA, The Social Marketing Practice, Mike
Read Associates and The University of Northampton Household
(2008) WR0116: Waste Prevention Activity in Dorset. Summary
Report and Technical Annexes. A project for Defras WREP*.
Fell, D., Cox, J. & Wilson D.C. (2010) Future waste growth, modelling
and decoupling. Waste Management & Research, 28, 281286.
FRN (2009) Average Weights. http://www.frn.org.uk/statistics.asp (accessed
28 July 2009).
GAP (Global Action Plan) et al. (2008) (Nye, M. & Burgess, J.; School of
Environmental Sciences, University of East Anglia). WR0114: Promoting Durable Change in Household Waste and Energy Use Behaviour. A project for Defras WREP*.
GAP (Global Action Plan) (2009). EcoTeams Evaluation Report for Defra
Environmental Action Fund. See Cox et al. (2009), op. cit.
Harder, M.K. & Woodard, R. (2009) Use of home food digesters to
reduce household waste. Waste and Resource Management, 162(2),
6973.
Hines, F., Morley, A., Frater, L., Cartwright, S. & Chandrashekar, S.
(2008) WR0502: Social Enterprises and Sustainable Waste and Resource
Management: Evaluating Impacts, Capacities and Opportunities. A
project for Defras WREP*.
Hampshire County Council & Brook Lyndhurst (2008) WR0117: Small
Changes Big Difference Towards a Materials Resource Authority:
Promoting Practical Waste Prevention and Exploring Options for
Resource Management. A project for Defras WREP*.
280