ESN Evidence Based Social Services ESN

The Social Services
network for Europe
Evidence-based
social services
Toolkit for planning
& evaluating social services
Social
innovation
The European Social Network
is supported by the
European Commission esn-eu.org
The content of this report and its toolkit
consists of a guidance on what type of
knowledge is required for planning and
evaluating services accompanied by two
questionnaires; the first aids practitioners
when planning services, while the second
questionnaire aids practitioners in
their evaluation of local social services.
The toolkit also includes a review of
international evidence-based social
work databases that may be useful for
practitioners when planning services.
The European Social Network (ESN)
has been working on aspects related to
evidence-based policy and practice in
public social services since 2012. This
toolkit was tested with senior practitioners
and applied researchers at a joint meeting
on evidence-based practice in London
in October 2015 and their feedback was
integrated into this final report.
This publication has received financial support from the European Union Programme for
Employment and Social Innovation ‘EaSI’ (2014-2020). For further information please consult:
http://ec.europa.eu/social/easi The information contained in this publication does not
necessarily reflect the official position of the European Commission.
Knowledge
to action
Evidence-based
social services
Toolkit for planning
& evaluating social services
Better
outcomes
for users
Contents
Introductory letter 05
About this toolkit 06
Acknowledgements 06
Background 09
Knowledge required for planning and evaluating services 12
Review of international evidence-based social services databases 27
Appendix 1 Selection of international evidence-based social services databases by country 37
Appendix 2 Questionnaire for planning social services 43
Appendix 3 Questionnaire for the evaluation of a social service 45
Appendix 4 Selection of standardised evaluation measures 51
Bibliography 58
Useful links 60
4 Evidence-based social services

Introductory letter
In 2014, the European Social Network (ESN) launched a new strand of its work focused on
evidence-based practice in social services. This component of ESN’s work programme
consists of a panel of applied researchers and senior practitioners, who meet in a variety of
formats. The input of applied researchers and social service’s directors results in a series of
outputs exploring challenges and opportunities for evidence-based practice.
As a follow up to the panel’s first meeting, which took place in November 2014, we published
a paper that discussed the evidence-based policy and practice phenomenon in social
services in selected European countries. The paper defined the concepts of evidence and
impact and assessed what sort of impact may be relevant and for whom. It reviewed two
case studies from Denmark and the Netherlands, which served as the basis for addressing a
number of challenges and suggesting possible ways to move forward.
Throughout 2015, ESN has continued this work with a review of international databases that
gather evidence-based practice in the field of social work and an assessment of how useful
the information they contain may be for senior social work practitioners. This has helped us
to formulate a proposal around what type of information should be collected to plan and
evaluate social services programmes.
As a result of this work, we are now launching this toolkit consisting of a guidance on what
type of knowledge is required for commissioning services accompanied by two
questionnaires; the first questionnaire includes what questions may be useful to think of
when planning services, the second questionnaire looks at what questions are useful when
evaluating your services. The toolkit also includes a review of international evidence-based
social work databases or evidence hubs that may be useful for practitioners when they
search for evidence-based practice that could guide their service planning and evaluation.
We tested various parts of the toolkit with senior practitioners and applied researchers at a
joint meeting in London in October 2015 and integrated their feedback in the final output,
which we gladly present now.
John Halloran
ESN CEO
Evidence-based social services 5

About this toolkit
This toolkit has been written by Alfonso Lara Montero, Policy Director, from the European
Social Network’s Secretariat and Ann Buchanan, Researcher, from the University of Oxford.
The toolkit consists of a guidance on what type of knowledge is required for planning and
evaluating services accompanied by two questionnaires; the first questionnaire includes
those questions that practitioners may want to think of when planning services, the second
questionnaire looks at what questions are useful when evaluating local social services.
The toolkit also includes a review of international evidence-based social work databases that
may be useful for practitioners when they search for evidence-based practice that could
guide their service planning and evaluation. We tested various parts of the toolkit with senior
practitioners and applied researchers at a joint meeting in London in October 2015 and
integrated their feedback in the final output, which we gladly present now.
Acknowledgements
With thanks to participants at the Evidence panel meetings.
November 2014
Steffen Bohni The National Board of Social Services Denmark
Minna Kivipelto National Institute for Health and Welfare Finland
Martijn Bool Movisie Netherlands
Piotr Nowak Mazovian Social Policy Center Poland
Amanda Edwards Social Care Institute for Excellence United Kingdom
Jenny Billings University of Kent United Kingdom
Stephanie Petrie University of Liverpool United Kingdom
Johanna O'Shea Ulster University United Kingdom
October 2015
Ann Vanden OCMW Gent Belgium

Wyngaerd
Steinar Eggen Municipality of Randers Denmark

Kristensen
Vibeke Normann KORA, Danish Institute for Local and Denmark

Andersen Regional Government Research

Kirstine Werk City of Copenhagen, Department of Social Denmark
Ravnsmed Services
Minna Kivipelto National Institute for Health and Welfare Finland

(THL)
Erkki Penttinen City of Vassa Finland
Emmanuel Social Affairs Department, Haute Saone France

Gagneux Department
Kristin Municipalities of Sandgerði, Garður and Iceland

Thorsteinsdottir Vogar
Milou Dutch Association of Directors of Social Netherlands

Haggenburg Services
Chris Vast KING (Quality Institute of Dutch Netherlands

Municipalities)
Mirosław Janusz Korczak Pedagogical University Poland

Grewiński
Jorge Carvalho Lisbon City Council - Local Housing Program Portugal

Mourão and Social Rights (PLHDS)
Lola Casal Spanish General Council of Social Work Spain

Sánchez
Ana-Isabel Lima- Spanish General Council of Social Work Spain

Fernandez
Kontxi Odriozola Barcelona Provincial Council, Area of Spain

Eizagirre Services for Citizens
Anna Ricart Barcelona Provincial Council, Area of Spain

Services for Citizens
Madalen SIIS Centre of Documentation and Studies Spain

Saizarbitoria
Ledicia Suárez Department of Social Inclusion of the Spain

Rodríguez Regional Goverment of Galicia
Johan Glad National board of health and welfare Sweden
Graham Owen Social Services Department, Municipality of Sweden

Trosa
Anna Lilja Swedish Association of Local Authorities and Sweden

Qvarlander Regions (SALAR)
Christina Practice, Development, Research, Future Sweden

Söderberg (PUFF)

Yvonne Birks Centre for Health Services Studies (CHSS), United Kingdom
University of Kent
Jonathan Nesta Innovation Agency United Kingdom

Breckon
Ann Buchanan Department of Social Policy and Intervention, United Kingdom

University of Oxford
Felicity Elvidge Kent County Council United Kingdom
Chrissie Gale CELSIS United Kingdom
Barbara Janta RAND Europe United Kingdom
Vivien Moffat Institute for Research and Innovation in United Kingdom

Social Services
Georgie Perry- Project Oracle United Kingdom

Crooke
Amy Randall Centre for Health Services Studies (CHSS), United Kingdom
University of Kent
Anne Tavendale Skills for Care United Kingdom
Andrea Wigfield Care-Connect: the Social Care Innovation United Kingdom

Hub (CIRCLE)
With thanks also to Kim Nikolaj Japing (ESN Policy Officer) for his support in the process of
drafting this toolkit.
Published: 2015
Copyright © European Social Network 2015
The overall copyright of this publication is held by the European Social Network (ESN).
For referencing, please use:

Buchanan, A.; Lara Montero, A.; Evidence-based Social Services: Toolkit for planning and
evaluating social services, European Social Network, Brighton, 2015.

Background
Five years after the start of the financial and economic crisis, in February 2013, the
European Commission called on Member States to prioritise social investment in order to
modernise their social welfare states. Within the current budget constraints, the Commission
invited Member States to shift their social policy focus towards investment in human capital
and social cohesion so that they can progress towards the objectives of the Europe 2020
Strategy1.
Background
The EC gave guidance to Member States on more efficient and effective social policies in
Five yearstoafter
response the start ofchallenges
the significant the financial andface.
they economic crisis, in
Recognising theFebruary 2013, the posed
budget constraints
European Commission called on Member States to prioritise social
by the crisis and the demographic challenges posed by population ageing, social investment in policies
order to
modernise
need their adequate
to be both social welfare
and states.
fiscally Within the current
sustainable. Therebudget
is a needconstraints,
to innovatethesocially
Commission
using
invited
the Member
available States to
resources shiftefficiently
more their social policy
and focus towards
effectively investment
and engaging in with
critically humana capital
and social cohesion
combination so that such
of approaches they can progress towards
as universalism, the objectives
targeting of the Europe
and conditionality when 2020
Strategy . policies. Social innovation is essential, especially in times of crisis, and policy
designing
1
approaches need to be tested so that policy-making becomes evidence-based and effective

The EC gave guidance to Member States on more efficient and effective social policies in
approaches are scaled up across Europe.
response to the significant challenges they face. Recognising the budget constraints posed
by the crisisthe
Therefore, and the demographic
modernisation challenges
of public posed by
social services is population
placing an ageing,
increasingsocial policieson
emphasis
need to bewith
evidence bothregard
adequate and fiscally sustainable.
to decision-making, serviceThere is a need
contracting to delivery,
and innovate socially
and using
the availableThe
evaluation. resources
Europeanmore efficiently and
Commission effectively
recognised and2013
in the engaging
Socialcritically with Package
Investment a
combination of approaches such as universalism, targeting and conditionality
(SIP) the need to test social policies and services to favour the most effective. The SIP
2 when
designing
argues effectiveSocial
thatpolicies. innovation
and quality social is essential,
services areespecially in times of hence
a public investment, crisis, and policy
public social
approaches
services playneed
a keytorole
be tested so thatthe
in achieving policy-making
Europe 2020becomes evidence-based
objectives. and effective
In difficult economic times,
approaches are scaled up across Europe.
the imperative to spend public money efficiently is even higher and not withstanding political
choices and demands, the value of informed decision-making drawing on evidence in all its
Therefore, the modernisation of public social services is placing an increasing emphasis on
component parts has gained a new momentum. The European Commission is also
evidence with regard to decision-making, service contracting and delivery, and
increasingly prominent in supporting research and innovation grounded on evidence-based
evaluation. The European Commission recognised in the 2013 Social Investment Package
knowledge sharing3.
(SIP)2 the need to test social policies and services to favour the most effective. The SIP
argues that effective
Programme and
evaluation hasquality social services
thus become are a important.
increasingly public investment, hence public
The European Social social
services (ESN)
Network play a has
key role in achieving
published a number the of
Europe
relevant 2020 objectives.
papers 4
. It hasInbecome
difficult clear
economic
from times,
ESN’s
the imperative to spend public money efficiently is even higher and
work with key personnel across Europe that: “Public authorities struggle to gather accurate not withstanding political
choices
data and demands,
on what works, for the
whatvalue of informed
groups, at what decision-making
cost, and with what drawing onDecision-makers
effects. evidence in all its
component parts has gained a new momentum. The European Commission is also
increasingly prominent in supporting research and innovation grounded on evidence-based
1
European Commission (2015): Europe 2020 in a nutshell. Available at: http://ec.europa.eu/europe2020/europe-2020-in-a-
knowledge sharing3.
nutshell/index_en.htm
2
European Commission (2013): Communication from the Commission to the European Parliament, the Council, the European
Programme
Economic evaluation
and Social Committeehas thus
and the become
Committee increasingly
of the Regions. Towardsimportant. The European
Social Investment Social
for Growth and Cohesion –
including implementing the European Social Fund 2014-2020, Brussels, 20.2.2013. Available at:
Network (ESN) has published a number of relevant papers4. It has become clear from ESN’s
http://ec.europa.eu/social/BlobServlet?docId=9761&langId=en
3
J-Pal Europe (2012): Social experimentation methodological guide. A methodological guide for policy makers. Available at:
work with key personnel across Europe that: “Public authorities struggle to gather accurate
4
data on what
European works,(ESN)
Social Network for what
(2015):groups, at services
Public social what cost, and
in crisis: with what
challenges effects. Available
and responses. Decision-makers
at:
http://www.esn-eu.org/userfiles/Documents/2015/2015_Public_Social_Services_in_Crisis_report_-_FINAL.pdf; European
Social Network (ESN) (2014): Contemporary issues in the public management of social services in Europe. Innovation,
research and evidence-based practice. Available at: http://www.esn-eu.org/raw.php?page=files&id=970. European Social
1
European
Network Commission
(ESN) (2015): Europe
(2015): Evidence in public2020 in services.
social a nutshell.An
Available
overviewat:from
http://ec.europa.eu/europe2020/europe-2020-in-a-
practice and applied research. Available at:
nutshell/index_en.htm
http://www.esn-eu.org/userfiles/Documents/Publications/Practice_reports/Evidence_in_public_social_services.pdf
2
European Commission (2013): Communication from the Commission to the European Parliament, the Council, the European
Economic and Social Committee and the Committee of the Regions. Towards Social Investment for Growth and Cohesion –
including implementing the European Social Fund 2014-2020, Brussels, 20.2.2013. Available at:
3
J-Pal Europe (2012): Social experimentation methodological guide. A methodological guide for policy makers. Available at:
4
European Social Network (ESN) (2015): Public social services in crisis: challenges and responses. Available at:
http://www.esn-eu.org/userfiles/Documents/2015/2015_Public_Social_Services_in_Crisis_report_-_FINAL.pdf; European 9
Social Network (ESN) (2014): Contemporary issues in the public management of social services in Europe. Innovation,
research and evidence-based practice. Available at: http://www.esn-eu.org/raw.php?page=files&id=970. European Social
Network (ESN) (2015): Evidence in public social services. An overview from practice and applied research. Available at:
http://www.esn-eu.org/userfiles/Documents/Publications/Practice_reports/Evidence_in_public_social_services.pdf

and practitioners need data and effective evaluation designs that are flexible enough to be
implemented in a variety of contexts.”
Alongside these pressures, there are also ethical imperatives. Helping strategies for
vulnerable populations should be based on best evidence. Any intervention has three
and practitioners
possible outcomes:need data and
positive effective
change; evaluation
no change; doesdesigns
harm. Atthattheare flexible
very least,enough to be
social workers
implemented
need in that
to ensure a variety of contexts.” do not harm. However, without effective evaluations
their interventions
they can never be sure of the outcomes. Actually, history is replete with well-meaning
Alongside these pressures, there are also ethical imperatives. Helping strategies for
programmes that have caused harm to the recipients.
vulnerable populations should be based on best evidence. Any intervention has three
possible outcomes:
Two well-known positiveofchange;
examples no commonly
harm5 are change; does harm.first,
quoted: At the
thevery least, social workers
Cambridge-Somerville
need
study to ensure
when thatmen
young theiratinterventions do not were
risk of delinquency harm.linked
However,
with awithout effective
supportive evaluations
mentor. In a very
they can never be sure of the outcomes. Actually, history is replete with
thorough longitudinal randomised controlled trial comparing delinquents with non- well-meaning
programmes that have
delinquents, there was caused
a highly harm to thefinding.
significant recipients.
The only problem was that the non-
delinquents had committed more crimes than the delinquents. A second example is the
Two well-known examples of harm5 are commonly quoted: first, the Cambridge-Somerville
outcome from ‘psychological debriefing'. This was a popular therapy for people who had
study when young men at risk of delinquency were linked with a supportive mentor. In a very
experienced severe trauma. The logic was that if a therapist talked through the trauma with
thorough longitudinal randomised controlled trial comparing delinquents with non-
the person this would ease the post-traumatic stress. Studies showed that far from relieving
delinquents, there was a highly significant finding. The only problem was that the non-
the stress, those who had undertaken the therapy had a more prolonged period of mental
delinquents had committed more crimes than the delinquents. A second example is the
health difficulties.
outcome from ‘psychological debriefing'. This was a popular therapy for people who had
experienced
As part of pastsevere
ESN’strauma.
work inThe
the logic was that
framework if a working
of the therapistgroup
talkedon
through the trauma with
leadership,
the person this
performance andwould ease the
innovation, post-traumatic
directors of publicstress.
social Studies
servicesshowed that far
emphasised from
the relieving
importance
themaking
of stress,research
those who had
and undertaken
evidence the therapy
intelligible had a more in
to professionals prolonged period ofits
order to improve mental
use in
health difficulties.
the public sector. Most directors saw evidence as an instrument to achieve a specific aim, for
instance, to improve the provision of services. “As a purchaser of services, I need to ensure
As part of past ESN’s work in the framework of the working group on leadership,
that I buy services that are of quality, efficient and effective”, said John Powell, from the
performance and innovation, directors of public social services emphasised the importance
Association of Directors of Adult Social Services in England (the UK). In other cases,
of making research and evidence intelligible to professionals in order to improve its use in
evidence was seen as an instrument to making the case for a particular investment. “I need
the public sector. Most directors saw evidence as an instrument to achieve a specific aim, for
a strong evidence base in order to convince politicians of the need to invest in a certain
instance, to improve the provision of services. “As a purchaser of services, I need to ensure
service”, argued Karine Lycops, from the Association of Directors of Social Services in
that I buy services that are of quality, efficient and effective”, said John Powell, from the
Belgium6.
Association of Directors of Adult Social Services in England (the UK). In other cases,
evidence
There are,was seen as
however, an instrument
a number to making
of barriers to thethe case for a particular
implementation investment.
of Evidence Based “IPractice
need
a strong evidence base in order to convince politicians of the need to invest
(EBP). Even in the United States, where most EBPs originate, implementation is often poor in a certain
service”,
and argued Karine
programmes come toLycops,
an endfrom
when thefunding
Association
dries of
upDirectors
7 of Socialthat
. It is suggested Services in
this is because
Belgium
there
6
. knowledge about EBP; there is confusion about what constitutes “evidence
is little
based” and concern that, in the real world, there may not be both the human and financial
There are, however, a number of barriers to the implementation of Evidence Based Practice
resources to undertake such programmes.
(EBP). Even in the United States, where most EBPs originate, implementation is often poor
and
In programmes
Europe, come
some of to issues
these an endare
when funding There
magnified. dries up 7
. It understandable
is an is suggested thatreluctance
this is because
to
there is little knowledge about EBP; there is confusion about what constitutes “evidence
import programmes that have been developed in a very different social welfare context (e.g.
based” andStates
the United concern
hasthat,
muchin the realoutcomes
poorer world, there
formay not be
children than both the EU
many human and financial
countries) and
resources to undertake such programmes.
InMcCord,
Europe, some
J. (2007 of these
(originally issues
published are Amagnified.
in 1978)): Thereof is
thirty-year follow-up an understandable
treatment reluctance
effects, crime and family: to
selected essays
5
of Joan McCord. Available at: http://books.google.com/books?id=Upu7qxbvP6IC&pg=PA13#v=onepage&q&f=false; Wessely,

import programmes
S. and Deahl, that havedebriefing
M (2003): Psychological been developed in aAmerican
is a waste of time; very different social
Psychologist, welfare
Vol. 183:1, context
pp. 12-14. (e.g.at:
Available
http://bjp.rcpsych.org/content/183/1/12
the
6 United States has much poorer outcomes for children than many EU countries) and
European Social Network (ESN) (2014): Contemporary issues in public social services: Innovation, research and evidence-
based practice. Available at: http://www.esn-eu.org/raw.php?page=files&id=970
7
Ashford, N.; Elliott, D. and Little, M. (2012): Blueprints for Europe: Promoting evidence based programmes in children’s
5
McCord,Psychosocial
services, interventions,
J. (2007 (originally Vol.
published in 21:2, pp.A205-214.
1978)): Available
thirty-year at:ofhttp://www.redalyc.org/pdf/1798/179824384009.pdf
follow-up treatment effects, crime and family: selected essays
of Joan McCord. Available at: http://books.google.com/books?id=Upu7qxbvP6IC&pg=PA13#v=onepage&q&f=false; Wessely,
S. and Deahl, M (2003): Psychological debriefing is a waste of time; American Psychologist, Vol. 183:1, pp. 12-14. Available 10 at:
http://bjp.rcpsych.org/content/183/1/12
6
European Social Network (ESN) (2014): Contemporary issues in public social services: Innovation, research and evidence-
based practice. Available at: http://www.esn-eu.org/raw.php?page=files&id=970
7
Ashford, N.; Elliott, D. and Little, M. (2012): Blueprints for Europe: Promoting evidence based programmes in children’s
services, Psychosocial interventions, Vol. 21:2, pp. 205-214. Available at: http://www.redalyc.org/pdf/1798/179824384009.pdf

10
indeed some programmes have performed badly when replicated in Europe. For example,
transportation of Multi-Systemic Therapy for young offenders from the United States to
Sweden yielded disappointing results, and researchers have suggested that marked
differences in youth welfare systems and socio-demographic factors may have led to this
outcome8.
indeed some programmes have performed badly when replicated in Europe. For example,
transportation
Perhaps more of Multi-Systemic
important Therapy for
is that, although young
there offenders
are some from
efforts to the United
develop States
EBP to
platforms
Sweden yielded
(for example, thedisappointing results,for
European Platform and researchers
Investing have suggested
in Children, that marked
EPIC, is developing such a
differences
facility for children), there is no pan-European source of information for all socialtowork
in youth welfare systems and socio-demographic factors may have led this
outcome 8
.
target populations, and what is available may not be in all European languages. Added to
this, most evidence based programmes are undertaken in group, schools or community
Perhaps more important is that, although there are some efforts to develop EBP platforms
centres, while most social workers work with individual clients and their task is to assess
(for example, the European Platform for Investing in Children, EPIC, is developing such a
their individual situation and suggest an immediate route forward, but the resource may not
facility for children), there is no pan-European source of information for all social work
be available within the timetable needed.
target populations, and what is available may not be in all European languages. Added to
this, most
Despite evidence
these based
problems, programmes
some imported are undertaken
programmes in group,
have schools or
demonstrated community
good outcomes.
centres, while most social workers work
9 with individual clients and their task is to assess
Among these are The Incredible Years which has been implemented in US, UK, Ireland,
their individual
Norway, situation
Germany, and suggest
Denmark, an immediate
Netherlands, Portugalroute forward, but
and Sweden. theclaim
Their resource
is thatmay
for not
be available
every $1 spentwithin theistimetable
there a return ofneeded.
$1.20. The Nurse Family Partnership that supports
10
mothers at risk of abusing their children, is another such programme. It claims to have a
Despite these problems, some imported programmes have demonstrated good outcomes.
$3.15 return for every $1 spent. In a cash-strapped environment, it is important to know what
Among these are The Incredible Years9 which has been implemented in US, UK, Ireland,
the costs/benefits are. However, at present there is very limited information on the costs
Norway, Germany, Denmark, Netherlands, Portugal and Sweden. Their claim is that for
and benefits of social services programmes in international databases. 10
every $1 spent there is a return of $1.20. The Nurse Family Partnership that supports
mothers at risk of abusing their children, is another such programme. It claims to have a
$3.15 return for every $1 spent. In a cash-strapped environment, it is important to know what
the costs/benefits are. However, at present there is very limited information on the costs
and benefits of social services programmes in international databases.
8
Gardner, F.; Montgomery, P.; and Knerr, W. (2015): Transporting evidence-based parenting programs for child problem
behavior (Age 3-10) between countries: systemic review and meta-analysis, Journal of Clinical Child Psychology and
Psychiatry, 1, pp. 1-14. Available at: http://67.199.123.90/library/wp-paper.asp?nMode=1&nLibraryID=723
9
Gardner, F.; Montgomery, P.; and Knerr, W. (2015): Transporting evidence-based parenting programs for child problem
Psychiatry, Vol. 1, pp. 1-14. Available at: http://67.199.123.90/library/wp-paper.asp?nMode=1&nLibraryID=723
10
Olds, D.; Holmberg, J.; Donelan-McCall, N.; Luckey, D., Knudtson, M.; and Robinson, J. (2014): Effects of home visits by
paraprofessionals
8 and by nurses
Gardner, F.; Montgomery, onKnerr,
P.; and children:
W. follow-up of a randomized
(2015): Transporting trial at agesparenting
evidence-based 6 and 9 years, JAMA
programs forPediatr.; Vol. 168:2,
child problem
pp. 114-21.
behavior Available
(Age at: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4217160/
3-10) between countries: systemic review and meta-analysis, Journal of Clinical Child Psychology and
Psychiatry, 1, pp. 1-14. Available at: http://67.199.123.90/library/wp-paper.asp?nMode=1&nLibraryID=723
9
Gardner, F.; Montgomery, P.; and Knerr, W. (2015): Transporting evidence-based parenting programs for child problem 11
Psychiatry, Vol. 1, pp. 1-14. Available at: http://67.199.123.90/library/wp-paper.asp?nMode=1&nLibraryID=723
10
Olds, D.; Holmberg, J.; Donelan-McCall, N.; Luckey, D., Knudtson, M.; and Robinson, J. (2014): Effects of home visits by
paraprofessionals and by nurses on children: follow-up of a randomized trial at ages 6 and 9 years, JAMA Pediatr.; Vol. 168:2,
pp. 114-21. Available at: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4217160/
11
Monitoring
and
evaluation
Knowledge required
for planning and
evaluating services

Knowledge required for planning and evaluating services
Introducing the Knowledge to Action Cycle
There is a rangerequired
Knowledge of “evidence” foror planning
“knowledge”and needed to develop initiatives
evaluating services or programmes
that work for service users. First, there is a requirement for basic research so that risk and
protective factors can be identified. There is also a need for information about the extent of
Introducing the Knowledge to Action Cycle
the problem, in order to plan ahead and develop resources for particular groups as well as to
focus valuable resources to the problems and areas most in need.
There is a range of “evidence” or “knowledge” needed to develop initiatives or programmes
that
The work for of
process service users.and
identifying First, there isknowledge
applying a requirement for basic
in practice research
is called so that risk
“Knowledge to and
protective factors can be identified. There is also a need for information
Action Cycle” (see figure 1). In the Knowledge to Action Cycle, the starting point is to about the extent of ¿Partimos
the problem,
identify in order to
the problem, plan and
select ahead and which
review develop resourceswe
knowledge for have
particular
aboutgroups as well Itasisto nosotros de
the problem.
focus
at thisvaluable
stage that resources
informationto from
the problems and areas
the reviewed most (see
databases in need.
pages 37-42
36-41) may be most knowledge?
helpful. As can be seen in figure 1, the second stage is to adapt knowledge to the local
The process of identifying and applying knowledge in practice is called “Knowledge to
context. Research suggests that when developing evidence-based practice, in addition to
Action Cycle” (see figure 1). In the Knowledge to Action Cycle, the starting point is to
the various types of evidence, awareness of the social context, needs and wishes of the
identify the problem, select and review which knowledge we have about the problem. It11is
local community, and the stakeholders are a key component of successful programmes .
at this stage that information from the reviewed databases (see pages 36-41) may be most
helpful. Assome
Although can be seen
of the in figure
better known1, the
evidence-based is to adapt knowledge
second stageprogrammes, such as “Theto the local
Incredible
context.and
Years” Research suggests
“Triple P” thatprogrammes
parenting when developing evidence-based
from Australia 12 practice,
have been in addition
trialled in manyto
the varioussome
countries, typesofofthe
evidence, awareness
evidence-based of the social
programmes context,
reported needs
in the and wishes
databases, of the
particularly
11
local community, and the stakeholders are a key component of successful programmes
from the UK and the US, may not travel well. As demonstrated earlier with multi-systemic .
therapy, this may be because of cultural factors; for instance, attitudes to children and
Although some of the better known evidence-based programmes, such as “The Incredible
family may be very different across countries. It may also be12 down to different national
Years” and “Triple P” parenting programmes from Australia have been trialled in many
policies and legislation, but most importantly to the fact that different systems of
countries, some of the evidence-based programmes reported in the databases, particularly
support, such as health and social services have different structures.
from the UK and the US, may not travel well. As demonstrated earlier with multi-systemic
therapy,
Some of this
thesemay
evidence-based cultural factors;
be because ofprogrammes for instance,
may have attitudes
to be tailored to children
to local and The
conditions.
family may be very different across countries. It may also be down to different
dilemma here is that without “implementation fidelity” – that is faithfully following the national
policies and legislation,
original procedures, but most importantly
the programme may not betothe
thesame and different
fact that may havesystems of
very different
support,
results. It such as health
is important and social
therefore that services haveinterventions
the adapted different structures.
are evaluated in their own
context. However, US sites such as SAMHSA13 and Colorado Blueprints14 are increasingly
Some of these evidence-based programmes may have to be tailored to local conditions. The
trialling programmes in international settings, and it would be a mistake to dismiss the
dilemma here is that without “implementation fidelity” – that is faithfully following the
knowledge and support they give just because they come from the USA.
original procedures, the programme may not be the same and may have very different
results. It is important therefore that the adapted interventions are evaluated in their own
context. However, US sites such as SAMHSA13 and Colorado Blueprints14 are increasingly
trialling programmes in international settings, and it would be a mistake to dismiss the
knowledge and support they give just because they come from the USA.
11
Gambrill, E. (2006): Evidence-based social work; in: Buchanan, A.: Evidence-based social policy and practice. A new
ideology or a human rights imperative?, pp. 328-340.
12
The Incredible Years: The Incredible Years Parenting Programme. Parents, teachers, and children training series. Available
at: http://incredibleyears.com/. Triple P International: Triple P - Positive parenting program. Available at: http://www.triplep-
parenting.uk.net/
13
Substance Abuse and Mental Health Services Administration: National registry of evidence-based programs and practices.
Available at: http://www.nrepp.samhsa.gov/Search.aspx
14
11
UniversityE.
Gambrill, of(2006):
Colorado Boulder, Center
Evidence-based for the
social Study
work; and Prevention
in: Buchanan, of Violence: Blueprints
A.: Evidence-based for healthy
social policy development.
and practice. A new
Availableor
ideology at:ahttp://www.blueprintsprograms.com/
human rights imperative?, pp. 328-340.
12
The Incredible Years: The Incredible Years Parenting Programme. Parents, teachers, and children training series. Available
at: http://incredibleyears.com/. Triple P International: Triple P - Positive parenting program. Available at: http://www.triplep- 12
parenting.uk.net/
13
Substance Abuse and Mental Health Services Administration: National registry of evidence-based programs and practices.
14
University of Colorado Boulder, Center for the Study and Prevention of Violence: Blueprints for healthy development.
Available at: http://www.blueprintsprograms.com/
12
Figure 1: The Knowledge-To-Action Cycle 15
Figure 1: The Knowledge-To-Action Cycle 15
Identifying and assessing various types of knowledge
As highlighted in the beginning of this section, there is a range of “evidence” or “knowledge”

needed to develop initiatives or programmes that work for service users. Previous work
undertaken by Nielsen, Grünberger, and Pedersen identified five types of evidence. This part
of the toolkit assesses the extent to which these five types of evidence may be available in
Identifying and assessing
social work databases various
and examines types
each type of
of knowledge
evidence through the development of
questions that decision-makers, social services directors and practitioners may want to ask
As highlighted in the beginning of this section, there is a range of “evidence” or “knowledge”
when they develop evidence-based programmes.
needed to develop initiatives or programmes that work for service users. Previous work
undertaken
The by of
five types Nielsen, Grünberger,
evidence that were and Pedersen
identified identified
to inform five types ofprogrammes
evidence-based evidence. This
are:part
of the toolkit assesses the extent to which these five types of evidence may be available in
1. Evidence
social about the target
work databases population
and examines each type of evidence through the development of
questions that decision-makers, social services directors and practitioners may want to ask
2. Evidence about the potential programme
when they develop evidence-based programmes.
3. Evidence about how the programme should be/is implemented
The five types of evidence that were identified to inform evidence-based programmes are:
4. Evidence about the possible effects
1. Evidence about the target population
5. Evidence about the costs
2. Evidence about the potential programme
3.
15
Evidence about how the programme should be/is implemented
Adapted and used with permission: Graham, I. D., Logan, J.; Harrison; M. B.; Straus, S. E.; Tetroe J.; Caswell, W.; and
Robinson, N. with the Alliance for Continuing Education in the Health Professions, the Society for Academic Continuing Medical
4. Evidence
Education, about
and the the possible
Association for Hospital effects
Medical Education (2006): Lost in knowledge translation: Time for a map?, Journal
of Continuing Education in the Health Professions, Vol. 26:1. Copyright © 2006. Available at:
http://www.ncbi.nlm.nih.gov/pubmed/16557505
13
15
Adapted and used with permission: Graham, I. D., Logan, J.; Harrison; M. B.; Straus, S. E.; Tetroe J.; Caswell, W.; and
Robinson, N. with the Alliance for Continuing Education in the Health Professions, the Society for Academic Continuing Medical
Education, and the Association for Hospital Medical Education (2006): Lost in knowledge translation: Time for a map?, Journal
of Continuing Education in the Health Professions, Vol. 26:1. Copyright © 2006. Available at:
13
Figure 2: Illustration of the logic underlying the social policy evidence base16
Figure 2: Illustration of the logic underlying the social policy evidence base16
Collecting evidence about the target population means looking at who may benefit from a
programme; including volume, demographics and socio-economic characteristics of the
target population. Four questions may help us collect evidence about our target population:
1. What is the problem of concern?
programme;
2. including
What is the extentvolume, demographics and socio-economic characteristics of the
of the problem?
target population. Four questions may help us collect evidence about our target population:
3. What risk and protective factors are associated with the problem?
1. What is the problem of concern?
4. Who should be partners in the programme?
2. What is the extent of the problem?
3. What risk and protective factors are associated with the problem?
What is the problem of concern?
4. Who should be partners in the programme?
To decide what is the social problem that needs addressing, the issue will need to be
contextualised against the norms and values, ethnic and morals and the group identity in
any given community 17. Different societies at different times will have different views and
What is awareness
different
different the problem
awareness of concern?
about
about what
what is or what
is or what isis not
not aa social
social problem.
problem. This information can
This information can only
only
come from those involved in their communities. An example from the UK is the recent
To decide what is the social problem that needs addressing, the issue will need to be
heightened awareness of the large numbers of children involved in sexual abuse.
contextualised against the norms and values, ethnic and morals and the group identity in
16
Nielsen, S.; Grünberger, P; 17
and Pedersen, C. (2013): What types of evidence are needed for decision-makers? Knowledge
any given
What is the
management community
to extent
support . Different
of the socialsocieties
problem?
evidence-based at different
policy and programs timesPaper
in Denmark, will presented
have different viewsEvaluation
at the American and
different
Association
17
awareness
Annual about
Conference. what is or what is not a social problem. This information can only
Buchanan, A. (2000): Present issues and concerns; in: Buchanan, A.; and Hudson, B.: Promoting Children’s Emotional Well-
Knowing
being. the extent of the problem can help in prioritising resources and areas for
intervention. Monitoring changes in incidence (number of new cases in a population in a
given
16 period) can beP;useful in giving a general indication about whether a problem is 14
Nielsen, S.; Grünberger, and Pedersen, C. (2013): What types of evidence are needed for decision-makers? Knowledge
increasing
management toor decreasing
support and whether
evidence-based social policyaand
specific
programsprogramme needs
in Denmark, Paper to beatput
presented in placeEvaluation
the American to meet
Association Annual Conference.
the
17 needsA.of(2000):
Buchanan, a community.
Present issues and concerns; in: Buchanan, A.; and Hudson, B.: Promoting Children’s Emotional Well-
being.
But knowing the extent of the problem can also help in planning ahead and having facilities
14
available
15 forEvidence-based
specific needs. For example, if it can be estimated from current child protection
social services
cases that a number of young mothers will need a programme to prevent the risk of child
come from those involved in their communities. An example from the UK is the recent
What is the extent of the problem?
Knowing the extent of the problem can help in prioritising resources and areas for
given period) can be useful in giving a general indication about whether a problem is
different awareness
increasing aboutand
or decreasing what is or what
whether is not programme
a specific a social problem.
needsThis information
to be put in placecan
to only
meet
come from those involved
the needs of a community. in their communities. An example from the UK is the recent
What is the
available extent of
for specific the problem?
needs. For example, if it can be estimated from current child protection
Knowing the extent of the problem can help in prioritising resources and areas for
abuse/neglect, this programme can be prepared and professionals ready to implement it can
be trained, so that young mothers can be included in to the programme as needed.
given period) can be useful in giving a general indication about whether a problem is
increasing
In or decreasing
Europe, the and whether
cross-European a specificfrom
data available programme
Eurostatneeds
18
can to
bebe put infor
helpful place to meet
comparing
the needs of a community.
countries or for comparing with local data. Information on specific problems may also be
available from national data. For example, the Appeals Board in Denmark19 gathers large
amounts of data from the local level, including local authorities and individual
available for specific needs. For example, if it can be estimated from current child protection
casework. Another database in Germany, INKAR20 provides statistical information on a wide
range of social issues and has thematic maps of all the counties in Germany.
abuse/neglect, this programme can be prepared and professionals ready to implement it can
beEngland,
In trained, so thatdeveloped
maps young mothers
by thecan be included
English IndicesinoftoDeprivation
the programme
21 as needed.
provide a relative
measure of deprivation at small area level. Areas are ranked from least to most deprived on
In Europe, the cross-European data available from Eurostat18 can be helpful for comparing
seven different dimensions of deprivation:
countries or for comparing with local data. Information on specific problems may also be
available from national
 income data. For example, the Appeals Board in Denmark19 gathers large
deprivation;
amounts of data from
 employment the local level, including local authorities and individual
deprivation;
casework. Another database in Germany, INKAR20 provides statistical information on a wide
 health deprivation and disability;
range of social issues and has thematic maps of all the counties in Germany.
 education deprivation;
 crimemaps
In England, deprivation;
developed by the English Indices of Deprivation21 provide a relative
 barriers
measure to housing
of deprivation and services
at small deprivation;
area level. Areas are ranked from least to most deprived on
seven living environment
 different dimensionsdeprivation.
of deprivation:
In addition,
 income two deprivation;
supplementary indexes measure income deprivation: the Income Deprivation
Affecting Children
 employment Index (IDACI) and the Income Deprivation Affecting Older People Index
deprivation;
(IDAOPI).
 healthFrom these indexes,
deprivation neighbourhood maps indicate the most deprived areas. Such
and disability;
maps are used bydeprivation;
education local authorities to place services in the most disadvantaged areas. For
example, local maps of deprivation were used to ensure Sure Start Families Centres22 were
 crime deprivation;
as close as possible
 barriers to the and
to housing communities most in need.
services deprivation;
 living environment deprivation.
InEUROSTAT.
18 addition,Available
two supplementary indexes measure income deprivation: the Income Deprivation
at: http://ec.europa.eu/eurostat/data/database
19
Danish Appeals Board. Available at: https://ast.dk/tal-og-undersogelser
Affecting
20 Children Index (IDACI) and the Income Deprivation Affecting Older People Index
Federal Institute for Construction, Urban and Area Research. Available at: http://www.inkar.de
(IDAOPI). From
21
UK Government, these indexes,
Department neighbourhood
for Communities maps Indices
and Local Government: indicate the mostAvailable
of Deprivation. deprived
at: areas. Such
https://www.gov.uk/government/collections/english-indices-of-deprivation
maps
22 are used
UK Parliament: Theby local authorities
development of Children'sto placeAvailable
Centres. services
at: in the most disadvantaged areas. For
example, local maps of deprivation were used to ensure Sure Start Families Centres22 were
http://www.publications.parliament.uk/pa/cm200910/cmselect/cmchilsch/130/13006.htm
as close as possible to the communities most in need. 15
18
EUROSTAT. Available at: http://ec.europa.eu/eurostat/data/database
19
20
Federal Institute for Construction, Urban and Area Research. Available at: http://www.inkar.de
21
UK Government, Department for Communities and Local Government: Indices of Deprivation. Available at:
22
UK Parliament: The development of Children's Centres. Available at:
http://www.publications.parliament.uk/pa/cm200910/cmselect/cmchilsch/130/13006.htm
15
Figure 3: Atlas of the Indices of Deprivation 2010 for England
Local maps of social deprivation – Sheffield: Index of Multiple Deprivation23
Figure 3: Atlas of the Indices of Deprivation 2010 for England

Local maps of social deprivation – Sheffield: Index of Multiple Deprivation23
On this map of Sheffield in the UK, the most deprived areas are navy blue and the least
deprived in light blue.
What are the risk and protective factors associated with the problem?
On this map of Sheffield in the UK, the most deprived areas are navy blue and the least
Many databases across Europe have sites that give research information/overviews or
deprived
research in light blue.relating to particular problems/conditions. In our review, these sites are
syntheses
highlighted under category ‘Basic Research & Problem Overviews’. Most of these
What are the risk and protective factors associated with the problem?
databases have search engines where you can search a social problem, such as
Many databases across Europe have sites that give research information/overviews or
research
23 syntheses
UK Office for relating
National Statistics: toofparticular
Atlas the Indices ofproblems/conditions. InAvailable
Deprivation 2010 for England. our review,
at: these sites are
http://neighbourhood.statistics.gov.uk/HTMLDocs/AtlasOfDeprivation2010/index.html,
highlighted under category ‘Basic Research & Problem Overviews’. Most of these see local authority “Sheffield”
databases have search engines where you can search a social problem, such as 16
23
UK Office for National Statistics: Atlas of the Indices of Deprivation 2010 for England. Available at:
http://neighbourhood.statistics.gov.uk/HTMLDocs/AtlasOfDeprivation2010/index.html, see local authority “Sheffield”
16
databases have search engines where you can search a social problem, such as
homelessness or domestic violence, and find individual research reports or research
syntheses. These sites provide the information that a manager may need about the target
population; for instance, who is likely to have the social problem; age; gender; ethnic
background; socio-economic status, linked conditions and possible causes.
databases
From haveresearch
the basic search engines
and problem whereoverviews,
you can search
a numbera social problem,
of factors maysuch as
be identified.
homelessness
These factors canor domestic violence,
be classified risk andand protective
find individual research
factors. Riskreports
factorsorareresearch
characteristics
syntheses.
that These
affect the sitesofprovide
chance having thethe information
problem. Risk thatfactors
a manager
may be may need about
individual the target
inherent traits, a
population; for instance, who is likely to have the social problem; age; gender;
factor under a person’s control, or a factor in their family or community. Similarly, protective ethnic
background;
factors socio-economic
are attributes status,family
in the person, linkedorconditions
community, andwhich
possible
helpcauses.
people to deal with the
problem. The advantage of knowing the risk and protective factors is that in your intervention
From the basic research and problem overviews, a number of factors may be identified.
you may directly target, not the people of concern but the risk factor. For example by
These factors can be classified risk and protective factors. Risk factors are characteristics
installing cameras in areas of high vandalism, this may deter delinquents because the
that affect the chance of having the problem. Risk factors may be individual inherent traits, a
cameras will increase the likelihood that they will get caught.
factor under a person’s control, or a factor in their family or community. Similarly, protective
factors
The are attributes
database SAMHSA in the
hasperson,
a usefulfamily
articleorexplaining
community, which
risk help people
and protective to deal
factors 24 with the
. All
problem. The advantage of knowing the risk and protective factors
recommended programmes on the Colorado Blueprints site have a section on risk and is that in your intervention
you may directly
protective factorstarget,
for thenot the people
problem that isofthe
concern
focus ofbutthe
theprogramme
risk factor.25For
. example by
installing cameras in areas of high vandalism, this may deter delinquents because the
Risk & protective
cameras factors
will increase associated
the likelihood thatwith
theydrug usecaught.
will get amongst children and
26
adolescents
The database SAMHSA has a useful article explaining risk and protective factors 24. All
recommended
Risk Factors programmes on the Colorado Blueprints
Domain Protective siteFactors
have a section on risk and
protective factors for the problem that is the focus of the programme 25.
Risk
Early& Aggressive
protective Behaviour
factors associated with drug
Individual use amongst children and
Self-Control
26
adolescents
Risk
LackFactors
of Parental Supervision Domain
Family Protective Factors
Parental Monitoring
Substance Abuse Behaviour

Early Aggressive Peer
Individual Academic Competence
Self-Control
Drug Availability School Anti-drug Use Policies

Lack of Parental Supervision Family Parental Monitoring
Poverty Community Strong Neighbourhood Attachment
Substance Abuse Peer Academic Competence
Drug Availability School Anti-drug Use Policies
Poverty Community Strong Neighbourhood Attachment
24
Substance Abuse and Mental Health Services Administration: Practicing effective prevention. Available at:
https://captus.samhsa.gov/prevention-practice/prevention-and-behavioral-health/key-features-risk-protective-factors/5)
25
University of Colorado Boulder, Centre for the Study and Prevention of Violence: Blueprints for healthy development.
Available at: http://www.blueprintsprograms.com/factSheet.php?pid=ae694b0755cd5eed5886ec4d8e658bde9639331d
26
National Institute on Drug Abuse: Preventing Drug Use among Children and Adolescents. Available at:
http://www.drugabuse.gov/publications/preventing-drug-abuse-among-children-adolescents/chapter-1-risk-factors-protective-
factors/what-are-risk-fact
24
Substance Abuse and Mental Health Services Administration: Practicing effective prevention. Available at: 17
https://captus.samhsa.gov/prevention-practice/prevention-and-behavioral-health/key-features-risk-protective-factors/5)
25
University of Colorado Boulder, Centre for the Study and Prevention of Violence: Blueprints for healthy development.
Available at: http://www.blueprintsprograms.com/factSheet.php?pid=ae694b0755cd5eed5886ec4d8e658bde9639331d
26
National Institute on Drug Abuse: Preventing Drug Use among Children and Adolescents. Available at:
http://www.drugabuse.gov/publications/preventing-drug-abuse-among-children-adolescents/chapter-1-risk-factors-protective-
factors/what-are-risk-fact
17
Risk factors for women having a teenage birth
Risk of having a teenage birth
Risk
Riskfactors
Factorsfor women having a teenage birth
A woman
Risk has a ahigh
of having probability
teenage birthof being teenage mother if her own mother was also a
teenage mother. The probability of being teenage mother is also significantly higher for
women whose parents are unskilled or who grew up with a single parent. Teenage mothers
Risk Factors
are worse off in a number of areas than average, for example, in relation to wages,
education and health.
A woman has a high probability of being teenage mother if her own mother was also a
teenage mother. The probability of being teenage mother is also significantly higher for
Who should
women whosebe parents
partnersare
in unskilled
the programme?
or who grew up with a single parent. Teenage mothers
are worse off in a number of areas than average, for example, in relation to wages,
Involving partners, stakeholders and those who will be the users of the programme at an
education and health.
early stage is key not only for the success of the programme but also for the programme to
obtain wider dissemination. Webster Stratton 27, who has been successful in disseminating
Whoparenting
her should be partners in
programmes the programme?
around the world, suggests that three steps are important in
ensuring success and wider dissemination:
Involving partners, stakeholders and those who will be the users of the programme at an
early
x stage is she
First, key speaks
not onlyoffor the success
developing of thefor
a taste programme but also
change. Those for the in
involved programme
supportingto
27
obtain wider dissemination. Webster Stratton , who has been successful in disseminating
the users under current conditions need to believe that help could be made more
her parenting programmes around the world, suggests that three steps are important in
effective.
ensuring success and wider dissemination:
x Second, she speaks of the need to obtain administrative and financial support for
x First, she speaks of developing a taste for change. Those involved in supporting
the suggested programme.
the users under current conditions need to believe that help could be made more
effective.
x Third, Stratton highlights the need for funding to evaluate the programme and
demonstrate that change has indeed occurred. If evaluation funding is not ring-
x Second, she speaks of the need to obtain administrative and financial support for
fenced at the start, it can be hard to persuade authorities to divert much needed
the suggested programme.
funding away from projects to evaluation later on.
x Third,about
2. Evidence Stratton
thehighlights
programme the need for funding to evaluate the programme and
demonstrate that change has indeed occurred. If evaluation funding is not ring-
fenced programmes
This concerns at the start, it directed
can be hard to persuade
towards the socialauthorities
problems to of
divert much needed
the target population
funding away from projects to evaluation later on.
and includes evidence-based practices, but may also include methods that have not been
subject to rigorous impact evaluations yet, but have promising features. The question a
2. Evidence
social servicesabout the may
director programme
ask her/himself to collect the best possible evidence is: which
level of intervention is likely to have the best outcome?
This concerns programmes directed towards the social problems of the target population
and
Threeincludes evidence-based
main levels of interventionpractices,
may bebut may also include methods that have not been
identified:
subject to rigorous impact evaluations yet, but have promising features. The question a
social
x services
Primarydirector may ask
prevention aimsher/himself
to preventtoa collect the best before
social problem possible evidence
it ever which
is:and
occurs it
level of targets
intervention is likely to have the best outcome?
the whole population. Examples of primary prevention programmes include
media campaigns for HIV/AIDS, smoking, obesity, sex/drug education in schools,
Three main levels
health of intervention
visitor may
services to all be identified:
families of newly born babies.
27 x Primary
Webster Stratton, C.;prevention aimsAdopting
and Taylor, T. (1998): to prevent a social problem
and implementing empirically before
supporteditinterventions.
ever occurs and for
A Recipe it
success; in:targets
Buchanan, the whole population. Examples of primary prevention programmes include
A. and B. Hudson: Parenting, Schooling and Children’s behaviour, pp. 127-160.
media campaigns for HIV/AIDS, smoking, obesity, sex/drug education in schools, 18

health visitor services to all families of newly born babies.
27
Webster Stratton, C.; and Taylor, T. (1998): Adopting and implementing empirically supported interventions. A Recipe for
success; in: Buchanan, A. and B. Hudson: Parenting, Schooling and Children’s behaviour, pp. 127-160.

18
health visitor services to all families of newly born babies.
 Secondary prevention refers to strategies used for those at risk of developing a

problem. A couple of examples illustrative of secondary prevention programmes
include strategies to prevent disaffected young people from offending and Family
Nurse Partnership with families who are at risk of child maltreatment.
 Tertiary prevention refers to the limitation of long term harm for those who have
experienced a social problem. Examples of tertiary prevention programmes include
therapy for children who have been abused, on-going support for people with
depression and training programmes for disabled people to access the labour
market.
Useful information on prevention projects at various levels can be found in most evidence-
based sites. The following example is of both a primary (targeted at the population) and a
secondary prevention programme (targeted at those at risk) to prevent cross border
trafficking in human beings. This comes from a Campbell Collaboration systematic review.
Cross-border trafficking in human beings: Prevention and intervention strategies for

reducing sexual exploitation - A Systematic Review
Policies or interventions to prevent or suppress cross border trafficking for the purpose of
sexual exploitation have not been evaluated rigorously enough to determine their effect.
Using different search strategies and key words in nine different languages, the authors
identified 19.000 studies on trafficking that came out between January 2000 and June
2009. Literature suggests that most interventions focus on prevention through awareness
raising activities. Some of the interventions aim directly at the at-risk population, victims or
perpetrators, others focus more indirectly on populations, trying to create more awareness
among social workers and policy makers.
Initiatives can combine several goals and activities, such as those focusing on legislation,
policy development and awareness-raising. Different methods are used in awareness
raising initiatives. In most cases, awareness-raising initiatives provide skills training to the
targeted population, such as victims and the at-risk population. A popular instrument is the
media (television, radio and posters), mainly used to raise awareness among the public.
Although, the authors were not able to state definitively which strategies worked best, the
document provides useful information about strategies for limiting cross-border trafficking in
human beings.
See more at: http://www.campbellcollaboration.org/lib/project/112/
MOVISIE, the Netherlands Centre for Social Development, led a seventh framework project
for cross-country partnerships to develop secondary prevention programmes for those at
risk of homelessness.
19

The CSEYHP Project - Combating Youth Homelessness
The aim of the CSEYHP project was to gain in-depth knowledge of the life trajectories of
homeless youth and those at risk of homelessness, and to explore the effectiveness of
reinsertion programmes in the four participating countries.
The objectives of the CSEYHP project were fourfold:
1. To understand the life trajectories of different homeless populations in different national

contexts.
2. To develop the concepts of risk and social exclusion in relation to the experience of
young homeless people and to their reinsertion process.
3. To test how different methods of working contribute to the reinsertion process for
young people.
4. To investigate the roles of and relationships between the young person and trusted
adults, lead professionals, peer mentors and family members in the delivery of these
programmes across all four countries.
See more at: https://www.movisie.com/projects/combating-youth-homelessness
At the level of tertiary interventions, there are programmes being developed in Denmark
for people with mental health problems and coexisting substance abuse:
Testing the ACT method for citizens with mental health problems and concurrent
substance abuse
Assertive community treatment (ACT) is an intensive and highly integrated approach for
community mental health service delivery. ACT programmes serve outpatients whose
symptoms of mental illness result in serious functioning difficulties in several major areas
of life, often including work, social relationships, residential independence, and physical
health. In Denmark, ACT has been tested among psychiatric patients and homeless
citizens. The method was implemented in two municipalities and tested among citizens
who suffered from mental disorders as well as substance abuse.
See more at: http://www.sfi.dk/view_all_projects-

4843.aspx?Action=1&NewsId=4747&PID=10047#sthash.flIgOPkt.dpuf
3. Evidence about implementation
This type of evidence refers to the conditions known to drive or inhibit implementation, such
as organisational, inter-personal and individual factors. This type of evidence refers
broadly to the way(s) the programme is implemented; for example, selecting the relevant
programme and means for its delivery, assessing whether the programme has been
implemented elsewhere and therefore, how it should be replicated and finally, whether it
has been evaluated and at which level of evidence.
20

A social services director may want to ask:
x What type of programme is appropriate for the target population?
x What is the theoretical justification for that approach?
x What is the medium by which the programme is delivered?
x What is the length of the programme?
x Has the programme been previously evaluated?
x How should the programme be replicated?
Different databases handle these questions in different ways. Most evidence-based

databases clearly specify the target population and the age group of the target population,
which can be found through the databases’ search engine or library facilities. Other
databases also give some justification for the theoretical approach used usually in their
overview. There are yet more databases that clearly specify the medium and components
used to deliver the programme. In the following example, the Örebro Prevention Programme,
retrieved from the European Platform for Investing in Children (EPIC), parents receive
information by mail as well as attending meetings in schools (group intervention in schools).
Örebro Prevention Programme
Countries that have implemented the practice: Netherlands, Sweden
Age Groups: Teenagers (age 13 to 19)

Target Groups: Children, Parents
Years in Operation: 1995-still operating
Type of Organization Implementing Practice: National Government
Practice Overview
The Örebro Prevention Programme (currently known as EFFEKT) works through parents
and by targeting drinking among 13–16-year-olds. The 2.5-year programme, which was
designed in Örebro County, Sweden, has been funded by the Swedish National Institute
of Public Health. This has been part of its initiative for universal youth alcohol prevention
programmes, which are to be implemented community-wide; the programme targets
youths at ages during which drinking increases; it works in both urban and rural
communities; it makes use of existing community resources and brings together different
agencies and relevant parties.
The programme design and implementation were largely based on empirical findings
demonstrating a negative correlation between levels of youth alcohol drinking and (i) the
strictness levels of parental attitudes against youth alcohol consumption as well as (ii) the
level of youth involvement in structured, adult-led activities. Thus, the core of the
programme has been based on parents receiving information (by mail and during parent
meetings at the schools) encouraging them to maintain strict attitudes against youth
alcohol use as well as encouraging their youth’s involvement in adult-led, organised
activities.
See more at: http://europa.eu/epic/practices-that-work/evidence-based-

practices/practices/orebro-prevention-program__en.htm#chapter_2
21

A key element in implementation is to assess as to whether the programme that is being
implemented may have been evaluated elsewhere and if it was, whether it was evaluated
with a sound level of evidence.
Therefore, the question that a social services director may want to ask is: Has the
A key element in implementation is to assess as to whether the programme that is being
programme been previously evaluated? If so, can it be determined with what level of
implemented may have been evaluated elsewhere and if it was, whether it was evaluated
evidence?
with a sound level of evidence.
There are varying levels of evaluation, which are often called the hierarchies of evidence.
Therefore, the question that a social services director may want to ask is: Has the
These hierarches suggest how much confidence one can have in the findings of the
programme been previously evaluated? If so, can it be determined with what level of
evaluation of a programme according to how the evaluation was undertaken. Databases use
evidence?
different forms of evidence hierarchies. While there is no agreement as to what constitutes
the best
There arehierarchies of evidence,
varying levels figurewhich
of evaluation, 4 presents a synthesis
are often from
called the a variety ofofsources
hierarchies
28
.
evidence.
These hierarches suggest how much confidence one can have in the findings of the
Figure 4: Hierarchy
evaluation of evidence
of a programme according to how the evaluation was undertaken. Databases use
different forms of evidence hierarchies. While there is no agreement as to what constitutes
the best hierarchies of evidence, figure 4 presents a synthesis from a variety of sources28.
Figure 4: Hierarchy of evidence
Some of the databases of evidence-based programmes give a range of ratings broadly

correlating with the above. The section on evidence-based practices of the European
Commission’s EPIC29 uses the following criteria:
Some of the databases of evidence-based programmes give a range of ratings broadly

correlating with the above. The section on evidence-based practices of the European
Commission’s EPIC29 uses the following criteria:
28
As an example on the hierarchy of evidence, see: Social Work Policy Institute: Evidence-based practice. Available at:
http://www.socialworkpolicy.org/research/evidence-based-practice-2.html
29
European Platform for Investing in Children: EPIC Evidence-based practices. Available at: http://europa.eu/epic/practices-
that-work/evidence-based-practices/evidence-criteria_en.htm#section1
22
28
As an example on the hierarchy of evidence, see: Social Work Policy Institute: Evidence-based practice. Available at:
http://www.socialworkpolicy.org/research/evidence-based-practice-2.html
29
European Platform for Investing in Children: EPIC Evidence-based practices. Available at: http://europa.eu/epic/practices-
that-work/evidence-based-practices/evidence-criteria_en.htm#section1
22
x Best practice: Fulfils three categories – ‘evidence of effectiveness’, ‘transferability’
and ‘enduring impact’.
x Promising practice: Fulfils at least two categories: ‘evidence of effectiveness’ and at

least one of the other two categories, ‘transferability’ or ‘enduring impact’.
x Emerging practice: Fulfils at least ‘evidence of effectiveness’.
If you are replicating an evidence based programme a key concern is “implementation

fidelity” or “implementation with integrity”. As mentioned earlier, the programme may need to
be adapted to different cultural contexts and local conditions. However, any adaptation of the
programme means that it is a different programme and outcomes may differ simply because
the crucial elements have not been replicated.
Some of the best known sites such as SAMHSA and Colorado Blueprints have programmes
that have already been trialled in different country settings. It is expected that if designated
procedures are followed, outcomes will be comparable. Such programmes come with an
implementation manual detailing how the programme must be run and often insist on
training for those undertaking the programme. Those replicating the programme are also
expected to follow the length of the programme that is also known as the “dosage” or the
number of individual sessions.
Within our databases review, we have identified a number of databases which give clear
instructions about the implementation requirements and in some cases a help mechanism
for queries.
4. Evidence about the possible effects
This type of evidence concerns the extent to which interventions have proved to be
effective; the effects’ ratio and size. This may also include the standardisation of outcome
measurements across different evaluations/studies. The question that we would need to
answer here is: Does it work?
For more specific questions, please see appendix 2 “How to plan a social service
programme’s evaluation”. Some of the questions that would help social services directors
and practitioners to collect evidence about the effects of a certain programme include:
x What are the intended outcomes?

x What plans have you for evaluating your programme? For instance, are there
any standardised measures that you could use?
x How will you measure the situation at the start of the project? For example in a
project to increase older people’s satisfaction in their residential setting, could you
ask all the participants to complete a simple quality of life questionnaire?
x What are your plans to follow up your project to see whether benefits have
been sustained?
In assessing the effects of a programme, ‘outputs’ are sometimes confused with

‘outcomes’. ‘Outputs’, for example, are the number of people who have been through the
programme. ‘Outcomes’ are what happens to participants having taken part in a programme.
If using a validated programme, there will be details of the outcomes. These need to be
23

examined carefully. For example, in a programme to reduce antisocial drinking, the outcome
may be that the participants believe that they should reduce their drinking, but this does not
mean that they have reduced their drinking. Although changing attitudes may be an
important step on the road to reducing their alcohol consumption, it is not the same as
demonstrating that they have.
If developing your own programme, it is important to include some standardised measures

to monitor possible effects. A list of some of those that are freely available has been
provided in appendix 4. Unfortunately, most of these are in English and will have to be
translated into the language used. However, there are also standardised measures that have
been validated in a number of languages. For example, the Strengths and Difficulties
Questionnaire, which broadly measures child adjustment, is one such measure that has
been translated and validated in numerous languages.
For those developing their own programmes, as demonstrated above, these measures can
be useful in evaluations conducted ‘before and after’ the programme was implemented.
Positive changes may indicate that the programme may deserve a more sophisticated
evaluation, perhaps organised together with a partner in a research facility.
Finally, social services directors should look at whether the benefits of the programme have
actually been sustained and have had a longer term impact; for instance, a follow-up
evaluation has been conducted at least 2 years after the programme was implemented and
a number of positive outcomes were reported.
This type of evidence is concerned with the unit costs per output, outcome, and impact, as
well as cost-benefits of (potentially) implemented programmes. Social services directors
would broadly ask:
How much does this programme cost?
There are two major costs. First, the cost of implementing the programme: administrative
costs, personnel costs, trainers’ costs. If you are using an existing evidence-based
programme, given that implementation fidelity is key (that is implementing the programme
as designed), there a number of costs that cannot be avoided. These include the cost of
buying the programme and manuals as well as a cost for training those implementing the
programme. Some programmes also have a helpline for those implementing it. Most of the
US programmes and some of the UK ones clearly define the costs of using an existing
programme.
The second major cost will be the cost of evaluation. Therefore, you may want to assess
as to whether there is any funding available for evaluating the programme, which could be
an opportunity to partner with a local research facility/university. Funding needs to be
obtained and evaluation planned before the start of the programme.
What are the estimates of costs and benefits of running a programme?
Finally, a social services director may also want to assess what are the estimates of
costs/benefits of running a successful project. Several sites featured in our review also
24

include a cost/cost-benefit analysis. This is an important component when managers are
making judgements about where to focus resources.
If, for example, six older people are helped to stay in their own home with support for an
extra year, the saving might be X compared to the cost of residential care. A director may
look at what would be the cost of their community care versus their cost of residential care
as well as the benefit that most people may want to stay in their own homes. In each of the
listed database, there is a note as to why the site may be useful, including whether
information on costs was included.
Costs of a programme that has been replicated internationally
Although originating from the US, the following programme has been used in a large number
of different settings and countries, and is a good example of the costs involved in its
implementation.
Cognitive Processing Therapy (CPT) for Post-traumatic Stress Disorder
Treatment can be delivered in individual or group format conducted by social workers,

psychologists, psychiatrists, and other mental health therapists. Treatment consists of
typically 12 sessions (range 10-15) conducted once or twice weekly for 60 minutes each
(90 minutes in a group setting). CPT has been implemented in all U.S. states and outside
the United States in Australia, Canada, China, Democratic Republic of Congo, England,
Germany, Iceland, Iraq, Peru, and Switzerland.
Costs
CPT Therapist Manual Free Required
(includes PTSD checklist)
CPT Materials Manual Free Required
CPT Group Manual Free Required
3-day Workshop offered at $1,000–$5,000 per day plus trainer travel Required
VA or non-VA locations expenses. (Variable)
CPT Web Training Free Not

necessarily
required
CPT Phone Consultation $100–$200 per hour. Cost varies Required

depending on length of consultation,
consultant qualifications, and experience
level
CPT Fidelity Checklist Free Required

(included in the CPT
Provider Application)
See more at: http://www.nrepp.samhsa.gov/ViewIntervention.aspx?id=38
25

Cost/benefit
analysis
Review of international
evidence-based social
services databases

Review of international evidence-based social services databases
Introduction
There
Review is a wealth of information available
of international on international
evidence-based websites.
social It was feltdatabases
services it would be
helpful to social service managers and practitioners to bring together in one place, a list of
these sites to assist them to obtain the basic knowledge they need so that they can more
Introduction
easily plan and evaluate their own evidence-based social programmes. A full list of
databases is available online; for a selection, please see appendix 1 (page 37-42 36-41).
There is a wealth of information available on international websites. It was felt it would be
helpful
We to sociala service
conducted review managers and practitioners
of the identified to bring
databases with together
the aim to findinwhat
one place, a list ofis
knowledge
these sites to assist them to obtain the basic knowledge they need so that
included (out of the five types of knowledge described earlier on) and assess what typethey can moreof
easily plan may
knowledge and evaluate
be helpfultheir own evidence-based
in developing social
evidence-based programmes.
social work practice.A full
Welistselected
of
databases is available online; for a selection, please see appendix 1 (page
78 databases; around two thirds of them come from European countries and one third from 36-41).
the United States, Australia and Canada.
We conducted a review of the identified databases with the aim to find what knowledge is
included
Most of the(out
EUofdatabases
the five types
give of knowledge described
research-based earlier
evidence on) and assess
and syntheses of the what type of
factors
knowledge with
associated be helpful
may specific in developing evidence-based
difficulties/conditions. social
However, there arework practice.
few sites We selected
in Europe with
78 databases;
details around two interventions.
of evidence–based thirds of them Most
comedatabases
from European countries and one
with evidence-based third from
programmes
the United
come from States,
the UK Australia
in Europeand andCanada.
from the United States.
Most of the
Although theEU databases
databases give
with research-based
programmes evidence
from the Unitedand syntheses
States of the
should be factors
taken into
associated with specific difficulties/conditions. However, there are few sites in
consideration, as increasingly the best programmes from the United States are now tested Europe with in
details ofit evidence–based
Europe, interventions.
is necessary to develop Most databasespractices
more evidence-based with evidence-based
in Europe thatprogrammes
are
come from the UK in Europe and from the United States.
culturally appropriate. We hope this work will contribute to this process.
Although
How the databases
the sites with programmes from the United States should be taken into
were selected
consideration, as increasingly the best programmes from the United States are now tested in
Europe,
In it is
order to necessary
identify to developlisted,
the databases more four
evidence-based
methods were practices in Europe
used. First, thatinare
we used Google
culturally appropriate. We hope this work will contribute to this process.
the terms ‘evidence-based database’ and ‘evidence-based registries’, which elicited a
number of sites. Second, existing lists of evidence-based sites were accessed, such as the
How the sites were selected
Social Work Policy Institute (US-based, linked to the National Association of Social
Workers)30. Third, we drew on the authors’ knowledge of such sites and information from
In order to identify the databases listed, four methods were used. First, we used in Google
colleagues at the Centre for Evidence Based Research at the University of Oxford; and
the terms ‘evidence-based database’ and ‘evidence-based registries’, which elicited a
finally, we accessed sites across Europe identified by the staff of the European Social
number of sites. Second, existing lists of evidence-based sites were accessed, such as the
Network (ESN).
Social Work Policy Institute (US-based, linked to the National Association of Social
Workers)
A number of
30
. Third, we drew
inclusion and on the authors’
exclusion knowledge
criteria were defined of such assites and information from
follows:
colleagues at the Centre for Evidence Based Research at the University of Oxford; and
finally,
x we accessed sitesbasic
Research-based across Europe identified
knowledge and overviews by theofstaff of thepopulation;
a target European Social
x Sites
Network giving national and local statistical information on the extent of a
(ESN).
problem/condition;
A number
x Sites inclusion
of with specific and exclusion
lists of evidencecriteria
basedwere defined as follows:
programmes;
x Sites in the UK and in the United States which only gave basic research information
x and overviews were
Research-based omitted
basic unlessand
knowledge specifically
overviews related to social
of a target work;
population;
x Sites from givingAustralia
nationaland andCanada were included
local statistical information if it was felt extent
on the they gave
of aa different
perspective to
problem/condition; the existing sites identified.
x Sites with specific lists of evidence based programmes;
x Sites in the UK and in the United States which only gave basic research information
30
Social Work
andPolicy Institute: Evidence-based
overviews were omittedpractice. Available
unless at: http://www.socialworkpolicy.org/research/evidence-
specifically related to social work;
based-practice-2.html#EVP
x Sites from Australia and Canada were included if it was felt they gave a different
perspective to the existing sites identified. 27
30
Social Work Policy Institute: Evidence-based practice. Available at: http://www.socialworkpolicy.org/research/evidence-
based-practice-2.html#EVP
27
 Sites from Australia and Canada were included if it was felt they gave a different
perspective to the existing sites identified.
Initially over 100 sites were identified. After excluding the more medically oriented sites, the
number was reduced to 78. The included databases were then grouped according to the
country they came from, by category and target population.
The following categories were identified:
1. Basic problems and research overviews: Information on research findings relating to

target populations, risk and protective factors, the extent of the problem/condition, possible
causes.
2. Systematic reviews: Sites that follow specific procedures to link together several similar
interventions for a target population and provide an overall rating on the basis of the
outcomes.
3. Multi-focused database with EBP elements: Sites that include information on different
target populations, as well as basic research and evidence-based interventions.
The following target populations were identified: children, youth and families; disabilities;
mental health, as well as older people. These sites include evidence based programmes,
research summaries and best practices for these groups.
We acknowledge that the list does not pretend to be inclusive of all such sites across
Europe. There may be other sites that slipped through our search strategies that are
important. However, the databases included are indicative of the type of material that is
available to inform evidence-based social work policy and practice.
The information we found on the sites
Most databases focused on basic research and research syntheses about different target
populations. With regard to evidence-based programmes, there are some important sites in
Denmark, Germany, Norway and Holland.
In addition to the UK, the United States, Australia and Canada, we identified multi-focussed
sites, which included evaluation of programmes in Belgium, Denmark, Finland, Germany,
Italy, Norway, Holland, Spain, Sweden, Switzerland and a website hosted by the European
Commission. Although these sites all gave details of basic research, and most kept
practice libraries, the standards of evaluation were generally lower than the sites in the
UK and in the United States.
Although some medically oriented sites were later excluded, many sites were health or
nursing-based and contained important information about older people’s care, dementia and
mental health/drugs. Where this was so, they remained in the list, since we felt that these
issues were of relevance for social workers.
Sites with basic problems and research overviews
When it came to sites with basic research and/or overviews of research, there were sites
either specifically for social workers, or more general sites for health and social care
professionals, which gave access to basic research, syntheses of research and statistics in
28

several countries. Such sites were identified in Austria, Finland, France, Germany Norway,
Poland, and the UK and in the United States. Many of these sites also had examples of ‘best
practice’.
From these sites it is possible to find what health practitioners would call “the aetiology” of
several countries. SuchWho
the condition/problem: siteshas
were
it? identified in Austria,
When? Where? TheirFinland,
ages? France,
PossibleGermany
causes? Norway,
What are
Poland, and the UK and in the United States. Many of these sites also had examples
the risk and protective factors associated with the condition/problem? This type of of ‘best
practice’. is the starting point in order to plan interventions effectively.
information
Fromincluded
Also these sites it iscategory
in this possible of
to basic
find what healththere
research, practitioners
are siteswould call “the aetiology”
with statistical of
information
to the various Who
the condition/problem:
relevant targethas it? When?
groups Where?
in social work. InTheir ages?interventions
planning Possible causes? What to
and where are
the risk and protective factors associated with the condition/problem?
place resources it is helpful to understand the extent of the problem/condition and This type of
information
whether it isisincreasing/decreasing
the starting point in order to plan area.
in a certain interventions effectively.
Among these, there are sites generated
by the European Commission, Eurostat and the European Social Surveys31.
Also included in this category of basic research, there are sites with statistical information
relevant
We also to the various
identified target
national groups
sites, suchinassocial work. Insite
the Danish planning interventions
Appeals and gathers
Board32, which where to
place resources it is helpful to understand the extent of the problem/condition
data from local authorities on social issues, children and integration. Another site in and
whether it is
Germany, increasing/decreasing
INKAR 33
provides statisticalin ainformation
certain area. onAmong
a wide these,
range ofthere areissues
social sites generated
and has
by the European
thematic maps ofCommission,
all the regionsEurostat and the
in Germany. TheEuropean Social
UK site on Surveysof .Deprivation34
the Indices
31
would also be included in this category, because it gives information on a range of

We also identified national sites, such as the Danish site Appeals Board32, which gathers
deprivation categories including children, employment and income deprivation at
data from local authorities on social issues, children and integration. Another site in
neighbourhood level.
Germany, INKAR33 provides statistical information on a wide range of social issues and has
thematic
Sites with maps of all thereviews
systematic regions in Germany. The UK site on the Indices of Deprivation34
would also be included in this category, because it gives information on a range of
These reviews
deprivation gather aincluding
categories number children,
of studiesemployment
together on and
a specific
income topic and giveatan opinion
deprivation
as to whether the
neighbourhood level. general approach used by the included studies is effective in changing the
problem/condition of concern. The argument is that the evidence for effectiveness is stronger
Sites with systematic
when several studies are reviews
evaluated together.
These
One of reviews
the more gather
usefula of
number of studies
these sites together
for social on a specific
practitioners is thetopic and give
Campbell an opinion35.
Collaboration
as toiswhether
This the general
an international approach based
organisation used by inthe included
Norway, whichstudies
usesisvery
effective in changing for
strict procedures the
problem/condition
including or excludingof concern.
studies. The argument is that the evidence for effectiveness is stronger
when several studies are evaluated together.
They usually include only Randomised Controlled Trials (RCTs), of which there are few in
One ofwork
social the more useful
as they of these
are very sites fortosocial
expensive practitioners
undertake properly.is Itthe
canCampbell Collaboration
be frustrating, at times, .
35
This is
after an international
reading through the organisation basedgiven
careful analyses in Norway,
by thewhich
author(s)usesofvery
the strict
review,procedures
to find thatfor
including or excluding studies.
they are unable to give a definitive answer as to whether the intervention is effective or
ineffective. This is usually because of methodological failings in the included studies.
They usually include only Randomised Controlled Trials (RCTs), of which there are few in
However it is a useful source to check as they are one of the few sites, which will highlight
social work as they are very expensive to undertake properly. It can be frustrating, at times,
programmes that do not work.
after reading through the careful analyses given by the author(s) of the review, to find that
they are unable to give a definitive answer as to whether the intervention is effective or
ineffective. This is usually because of methodological failings in the included studies.
However
31 it is a useful source to check as they are one of the few sites, which will highlight
European Commission, DG Employment Social Affairs and Inclusion: Publications and documents. Available at:
programmes that do not work.
http://ec.europa.eu/social/keyDocuments.jsp?advSearchKey=elder+care&mode=advancedSubmit&langId=en&policyArea=&typ
e=0&country=0&year=0. EUROSTAT. Available at: http://ec.europa.eu/eurostat/web/main/home. European Social Surveys.
Available at: http://www.europeansocialsurvey.org/about/project_specification.html
32
33
Federal Institute for Construction, Urban and Area Research: INKAR database. Available at: http://www.inkar.de
34
31
European Commission, DG Employment Social Affairs and Inclusion: Publications and documents. Available at:
35
Campbell Collaboration. Available at: http://www.campbellcollaboration.org/
http://ec.europa.eu/social/keyDocuments.jsp?advSearchKey=elder+care&mode=advancedSubmit&langId=en&policyArea=&typ
e=0&country=0&year=0. EUROSTAT. Available at: http://ec.europa.eu/eurostat/web/main/home. European Social Surveys.
Available at: http://www.europeansocialsurvey.org/about/project_specification.html 29
32
33
Federal Institute for Construction, Urban and Area Research: INKAR database. Available at: http://www.inkar.de
34
35
Campbell Collaboration. Available at: http://www.campbellcollaboration.org/

29
In the UK, the EPPI Centre36 at the Institute of Education focuses on educational as well as
social topics and has slightly less strict criteria for including studies and a vast library of
projects. MOVISIE37 from the Netherlands also has a library of systematic reviews.
Multi-focused sites with elements of evidence-based practice

In the UK, the EPPI Centre36 at the Institute of Education focuses on educational as well as
social topics
These and has
databases have slightly
been less stricttogether
grouped criteria for
as including
they includestudies and basic
not only a vastresearch
library ofbut
also information
projects. MOVISIE on evidence-based
37
practices
from the Netherlands for different
also has a library target populations.
of systematic The
reviews.
evaluation of the evidence base is often variable, and some projects have only been
Multi-focused sites
evaluated loosely. with elements
However, of evidence-based
within individual countries, theypractice
are often the most useful sites
for bringing together national/local information about social work practices.
These databases have been grouped together as they include not only basic research but
Below a selectiononofevidence-based
also information some of these sitespractices
across for different target populations. The
Europe:
evaluation of the evidence base is often variable, and some projects have only been
Countryloosely.
evaluated / However, within individual countries, they are often the most useful sites
Name URL
Entity
for bringing together national/local information about social work practices.
Denmark
Below a selectionThe DanishofSocial
of some theseCentre for Research
sites across Europe: www.sfi.dk/about_sfi-2821.aspx
Country / Agence
Name nationale de l'évaluation et de la URL
Entity qualité des établissements et services
http://www.anesm.sante.gouv.fr/spip.php
France sociaux et médico-sociaux: (ANESM) –
Denmark The Danish Social Centre for Research ?page=rubrique&id_rubrique=10
www.sfi.dk/about_sfi-2821.aspx
Recommandations de bonnes pratiques
professionnelles
Agence nationale de l'évaluation et de la
qualité des établissements et services http://www.fskompetenscentret.fi/mathild
Finland Mathilda http://www.anesm.sante.gouv.fr/spip.php
France sociaux etWrede Institute (ANESM) –
médico-sociaux: a_wrede_institutet/in_english
?page=rubrique&id_rubrique=10
Recommandations de bonnes pratiques
professionnelles
Gesis, SOWIPORT. Das Portal für die http://www.gesis.org/en/institute/the-
Germany
Sozialwissenschaften association/mission/
http://www.fskompetenscentret.fi/mathild
Finland Mathilda Wrede Institute
a_wrede_institutet/in_english
http://www.assistentisociali.org/servizio_s
Italy Le Buone Prassi del servizio sociale ociale/buone_prassi_del_servizio_sociale
Gesis, SOWIPORT. Das Portal für die http://www.gesis.org/en/institute/the-
.htm
Germany
Sozialwissenschaften association/mission/
http://www.dobrepraktyki.pl/index.php?p1
Poland Baza Dobrych Praktypk database http://www.assistentisociali.org/servizio_s
=11
Italy Le Buone Prassi del servizio sociale ociale/buone_prassi_del_servizio_sociale
.htm
http://ec.europa.eu/social/keyDocuments.
European Employment Social Affairs and jsp?advSearchKey=elder+care&mode=a
Commission Inclusion, publications http://www.dobrepraktyki.pl/index.php?p1
dvancedSubmit&langId=en&policyArea=
Poland Baza Dobrych Praktypkand documents
database
=11
&type=0&country=0&year=0
Integrated Health Models Observatory http://ec.europa.eu/social/keyDocuments.

Spain
European Employment Social Affairs and http://www.omis-nh.org/
jsp?advSearchKey=elder+care&mode=a
(OMIS)
Commission Inclusion, publications and documents dvancedSubmit&langId=en&policyArea=
&type=0&country=0&year=0
The UK The Social Care Institute for Excellence http://www.scie-socialcareonline.org.uk/
Integrated Health Models Observatory

Spain http://www.omis-nh.org/
(OMIS)
36
EPPI Centre. Available at: http://eppi.ioe.ac.uk/cms
37
MOVISIE. Available at: http://www.movisie.nl/
The UK The Social Care Institute for Excellence http://www.scie-socialcareonline.org.uk/
30
36
EPPI Centre. Available at: http://eppi.ioe.ac.uk/cms
37
MOVISIE. Available at: http://www.movisie.nl/
30
Sites targeted at population groups
Children, families and youth
There were more sites dedicated to children, youth and families than to any other population
Sites
group. targeted
Funded byatthe population groups
European Commission, the European Platform for Investing in
Children is an evidence-based online platform, which reviews practices submitted from
38
Children, families and youth
across Europe according to certain standards of evidence. The website also provides
information about policies that can help children and their families face the challenges that
There were more sites dedicated to children, youth and families than to any other population
exist in the current economic climate.
group. Funded by the European Commission, the European Platform for Investing in
Children
In
38
is an evidence-based
other European countries, there online platform, which
is information of bestreviews
practicepractices submitted
and evaluations of from
across Europe
children’s according
projects but theytohave
certain standards
generally of included
been evidence.inThe
the website also provides
multi-focused sites noted
information about policies that can help children and their families
above. There are four specific sites focusing on children from Germany. Most face the challenges
of the that
exist in thesites
remaining current economic
identified climate.
in the review came from the UK and the United States. The United
States’ sites have been included because of their focus on evidence-based programmes
In other European countries, there is information of best practice and evaluations of
and because the quality of their evaluations is high. Some of these sites include
children’s projects but they have generally been included in the multi-focused sites noted 39
interventions for children that have been tested in Europe, notably Colorado blue prints
above. There are four specific sites focusing on children from Germany. Most of the 40
and the NREPP/SAMHSA’S National registry of evidence programmes and practices .
remaining sites identified in the review came from the UK and the United States. The United
States’ sites have been included because of their focus on evidence-based programmes
People with disabilities
and because the quality of their evaluations is high. Some of these sites include
interventions for children
Despite extensive searches,that havesites
fewer tested
beenwere in Europe,
found that werenotably Colorado
dedicated blue prints
specifically
39
to this
and theThere
group. NREPP/SAMHSA’S
is information onNational registry
this group in theofmulti-focused
evidence programmes
sites and and
on the more .
practices 40
nursing/medically orientated sites. The German site Zentrum für Qualität in der Pflege (ZQP)
People with disabilities
(Centre for Quality in Care41) includes information relating to disabilities.
Despite extensive searches, fewer sites were found that were dedicated specifically to this
In the UK and in the United States, the most informative sites appear to be those run by
group. There is information on this group in the multi-focused sites and on the more
charities. These sites usually have helpful information for carers and those with a
nursing/medically orientated sites. The German site Zentrum für Qualität in der Pflege (ZQP)
disability. It is anticipated that similar charity sites may exist in other European countries,
(Centre for Quality in Care41) includes information relating to disabilities.
but have not been identified in this review, mostly due to language difficulties.
In the UK and in the United States, the most informative sites appear to be those run by
People with mental health problems
charities. These sites usually have helpful information for carers and those with a
disability.
There wereItaisfew
anticipated
more sitesthat
onsimilar charity
this target sites may
population exist
than forinpeople
other European countries,
with disabilities. Many
but have not been identified in this review, mostly due to language difficulties.
of the sites focusing on children, youth and families have programmes for young people with
minor mental health problems. Three sites specifically focusing on mental health were found.
People with mental health problems
One in Germany - Deutsche Institut für angewandte Pflegeforschung (German Institute of
Applied Nursing Research42). Although the main focus here is on nursing, there is helpful
There were a few more sites on this target population than for people with disabilities. Many
information on substance abuse, family counselling, children with mentally ill parents, and
of the sites focusing on children, youth and families have programmes for young people with
partners and family members of schizophrenics.
minor mental health problems. Three sites specifically focusing on mental health were found.
One in Germany - Deutsche Institut für angewandte Pflegeforschung (German Institute of
Applied Nursing Research42). Although the main focus here is on nursing, there is helpful
information on substance abuse, family counselling, children with mentally ill parents, and
partners
38
European and family
Platform members
for Investing of schizophrenics.
in Children. Available at: http://europa.eu/epic/
39
University of Colorado Boulder, Center for the Study and Prevention of Violence: http://www.blueprintsprograms.com
40
Substance Abuse and Mental Health Services Administration: National registry of evidence programs and practices.
Available at: http://www.nrepp.samhsa.gov/Index.aspx
41
Centre for Quality in Care. Available at: http://www.zqp.de/
42
German Institute of Applied Nursing Research. Available at: http://www.dip.de/datenbank-wise/informationen-zu-wise/ueber-
wise/
38
European Platform for Investing in Children. Available at: http://europa.eu/epic/
39
University of Colorado Boulder, Center for the Study and Prevention of Violence: http://www.blueprintsprograms.com
40
Substance Abuse and Mental Health Services Administration: National registry of evidence programs and practices. 31
Available at: http://www.nrepp.samhsa.gov/Index.aspx
41
Centre for Quality in Care. Available at: http://www.zqp.de/
42
German Institute of Applied Nursing Research. Available at: http://www.dip.de/datenbank-wise/informationen-zu-wise/ueber-
wise/
31
In the UK, the Centre for Evidence Based Mental Health43) is a useful source of information.
There is also an online UK journal, Evidence based Mental Health, with excellent reviews of
research evaluations.
Older people
In the UK, the Centre for Evidence Based Mental Health43) is a useful source of information.
Given the increasing prospect of an ageing population across Europe, relatively few sites
There is also an online UK journal, Evidence based Mental Health, with excellent reviews of
were found that were specifically dedicated to this population group. However, four sites
research evaluations.
were found in Germany with important information on practices for older people. The
Zentrum für Qualität in der Pflege (Centre for Quality in Care44), CareLit45, the Deutsches
Older people
Zentrum für Altersfragen (German centre of gerontology46), and the Zukunftswerkstatt
Demenz
Given the(Future workshop
increasing on dementia
prospect
47
). population across Europe, relatively few sites
of an ageing
were found that were specifically dedicated to this population group. However, four sites
The aims of the German Centre of Gerontology are: to increase, collect, evaluate, process
were found in Germany with important information on practices for older people. The
and disseminate knowledge about the living arrangements of44ageing and old people to use
Zentrum für Qualität in der Pflege (Centre for Quality in Care ), CareLit45, the Deutsches
this knowledge for scientifically independent consultation46regarding the challenges of ageing
Zentrum für Altersfragen (German centre of gerontology ), and the Zukunftswerkstatt
for society and social policy. The site Future Workshop on Dementia includes a library facility
Demenz (Future workshop on dementia47).
and information on ways to access funding. Individual sites from France, Sweden,
Switzerland, the German
The aims of the UK and the United
Centre States are also
of Gerontology are:included in this
to increase, category.
collect, evaluate, process
and disseminate knowledge about the living arrangements of ageing and old people to use
The information
this knowledge we extracted
for scientifically from theconsultation
independent sites regarding the challenges of ageing
for society and social policy. The site Future Workshop on Dementia includes a library facility
Under each selected database, we identified the main focus of the site, the type of
and information on ways to access funding. Individual sites from France, Sweden,
information provided (e.g. library or search engine), the availability (whether free or for
Switzerland, the UK and the United States are also included in this category.
pay), the criteria used for evaluating interventions, outcomes and impact, and a note on
the usefulness
The informationor potential value offrom
we extracted the site
thefor those planning evidence-based
sites
programmes.
Under each selected database, we identified the main focus of the site, the type of
In most cases, the focus was clearly given in the opening page of the database. Where the
information provided (e.g. library or search engine), the availability (whether free or for
information on the database covered many topics, a note was made of the main information
pay), the criteria used for evaluating interventions, outcomes and impact, and a note on
of interest for social workers. Similarly where other activities were offered only those of
the usefulness or potential value of the site for those planning evidence-based
interest for social workers and their managers were listed.
programmes.
When it comes to availability, a surprising finding was to see how many sites gave their
In most cases, the focus was clearly given in the opening page of the database. Where the
information freely and without inquiring who was accessing the material. However, some
information on the database covered many topics, a note was made of the main information
sites; for instance, the Italian site Sistema nazionale per le linee guida (SNLG) dell’ Istituto
of interest for social workers. Similarly where other activities were offered only those of
Superiore di Sanità (ISS) do not allow social workers to sign up. It appears that this site may
interest for social workers and their managers were listed.
hold useful information for social workers but access was specifically denied.
When it comes to availability, a surprising finding was to see how many sites gave their
Two excellent UK sites, the Research in Practice site for children and the Research in
information freely and without inquiring who was accessing the material. However, some
Practice site for adults, allow access to some parts of their sites but not to all of them. In
sites; for instance, the Italian site Sistema nazionale per le linee guida (SNLG) dell’ Istituto
other cases, databases are not available unless the inquirer is working in one of the local
Superiore di Sanità (ISS) do not allow social workers to sign up. It appears that this site may
authorities/charities who have signed up and paid for membership, as this is the case with
hold useful information for social workers but access was specifically denied.
RiP (Children), a useful database on what works for troubled children.
Two excellent UK sites, the Research in Practice site for children and the Research in
The standards of evidence for effective programmes given in most European sites was
Practice
Centre forsite for adults,
Evidence allow
Based Mental access
Health. to some
Available parts of their sites but not to all of them. In
at: http://cebmh.warne.ox.ac.uk
43
low.
44
CentreThere were sites that gave information on the quality of the evaluation, but it was
for Quality in Care. Available at: http://www.zqp.de/index.php?pn=project.
other
45
CareLit. Available at: http://www.carelit.de/cont/start/index.php the inquirer is working in one of the local
cases, databases are not available unless
confusing
46
German Centreas for
to Gerology.
what their definition
Available of quality meant. Almost all US sites and most UK sites
at: http://www.dza.de/en/fdz.html
broadly followed a hierarchy of evidence
47
German Federal Ministry of Health: Future Workshop on with top Available
Dementia. quality at:
going to those programmes that
http://www.bmg.bund.de/themen/pflege/demenz/zukunftswerkstatt-demenz/uebersicht-projekte.html
had been evaluated in repeated Randomised Controlled Trials (RCTs) or Systematic
Reviews, with lesser levels of confidence given to evaluations using other methodologies.
43
Centre for Evidence Based Mental Health. Available at: http://cebmh.warne.ox.ac.uk
44
Centre for Quality in Care. Available at: http://www.zqp.de/index.php?pn=project.
32
For
45 instance,
CareLit. AvailableColorado Blueprints has two levels: model programmes and promising
at: http://www.carelit.de/cont/start/index.php
46
German Centre for Gerology. Available at: http://www.dza.de/en/fdz.html
programmes.
47 Promising programmes clearly identify the outcomes; the population for which
German Federal Ministry of Health: Future Workshop on Dementia. Available at:
ithttp://www.bmg.bund.de/themen/pflege/demenz/zukunftswerkstatt-demenz/uebersicht-projekte.html
is intended; have at least one RCT or 2 quasi experimental evaluations; show a significant
positive change that can be attributed to the programme; there must be no harmful effects;
and the programme has to be ready to be disseminated (i.e. manual available, technical
Evidence-based social services
32 33
assistance etc.). Model programmes must have a minimum of 2 RCTs or one high quality
The standards of evidence for effective programmes given in most European sites was
low. There were sites that gave information on the quality of the evaluation, but it was
confusing as to what their definition of quality meant. Almost all US sites and most UK sites
broadly followed a hierarchy of evidence with top quality going to those programmes that
Reviews, with lesser
The standards levels offor
of evidence confidence given to evaluations
effective programmes given inusing
most other methodologies.
European sites was
low.instance,
For There were sites that
Colorado gave information
Blueprints on themodel
has two levels: qualityprogrammes
of the evaluation, but it was
and promising
confusing as to
programmes. what theirprogrammes
Promising definition of clearly
qualityidentify
meant. the
Almost all US sites
outcomes; and most for
the population UK which
sites
broadly followed a hierarchy of evidence with top quality going to those programmes that
it is intended; have at least one RCT or 2 quasi experimental evaluations; show a significant
Reviews, with lesser levels of confidence given to evaluations using other methodologies.
and the programme has to be ready to be disseminated (i.e. manual available, technical
assistance
For instance, etc.). Model Blueprints
Colorado programmes hasmust
two have
levels:a model
minimum of 2 RCTsand
programmes or one high quality
promising
RCT plus one Promising
programmes. high quality quasi experimental
programmes clearly evaluation.
identify Positive impact
the outcomes; must be sustained
the population for which
48
for 12 months after the end of programme .
it is intended; have at least one RCT or 2 quasi experimental evaluations; show a significant
NREPP/SAMHSA’S national registry of evidence programmes and practices 49 also has high
and the programme
evidence standards. has to be
To be ready to
included, thebeprogramme
disseminated
must(i.e. manual
have available,
produced technical
one or more
assistance etc.). Model programmes must have a minimum of 2 RCTs or one high quality
positive behavioural outcomes in mental health or substance abuse among individuals,
RCT plus one high quality quasi experimental evaluation. Positive impact must be sustained
communities, or populations. Positive behavioural outcome(s) must have been demonstrated
forat12least
in months
one after
studythe endan
using programme 48or
of experimental . quasi-experimental design. Studies with a
single group, pre-testnational
NREPP/SAMHSA’S and post-tests
registrydesigns would
of evidence not meet this
programmes requirement.
and practices 49 also has high
Implementation
evidence standards. materials,
To betraining and
included, support
the resources,
programme must and
havequality assurance
produced one or procedures
more
have been developed and are ready for use by the public.
positive behavioural outcomes in mental health or substance abuse among individuals,
communities,
In or populations.
the UK, websites Positive
have a variety behavioural
of definitions in outcome(s) must have
regards to evidence been demonstrated
standards for their
in at least one study using an experimental or quasi-experimental design. Studies with a 50
programmes. In the cases of sites like Project Oracle, Children and Youth Evidence hub ,
single group, pre-test and post-tests designs would not meet this requirement.
which only lists projects that have been tested in London, and the Early Intervention
Implementation
Foundation (EIF)materials,
51 training
, only few and support
programmes meetresources,
the higher and quality EIF
standards. assurance procedures
has also a useful
have been developed and are ready for use by the public.
list comparing different evidence standards of the main national and international
programmes for children.
In the UK, websites have a variety of definitions in regards to evidence standards for their
programmes. In the cases of sites like Project Oracle, Children and Youth Evidence hub 50,
In the EU, the European Platform for investing in Children (EPIC) 52, which focuses mainly on
which only
projects thatlists projects
have that have been
been developed tested defines
in Europe, in London, and the Early
its standards Intervention
as best practice,
51
Foundation (EIF) , only few programmes meet the higher standards.
promising practice and emergent practice. Practices are also assessed as to whatEIF has also extent
a useful
list comparing different evidence standards of the main national and international
they are transferable (i.e. have been replicated elsewhere) and whether they are enduring
programmes for children.
(i.e. there has been a follow up study after 2 years).
52
In the
Our listEU, the European
of evidence Platform
based for investing
databases highlightsinthe
Children
need to(EPIC) , which
have some focuses
common mainly on
agreement
projects that have been developed in Europe, defines its standards as best practice,
(or index of standards) and thereby a common understanding of what each standard of
promising practice and emergent practice. Practices are also assessed as to what extent
evidence means.
they are transferable (i.e. have been replicated elsewhere) and whether they are enduring
(i.e. there has been a follow
Costs/cost-benefit up study after 2 years).
analysis
Our list of evidence based databases highlights the need to have some common agreement
48
(or index ofofColorado
University standards) and
Boulder, thereby
Center a common
for the Study understanding
and Prevention of what
of Violence (CSPV): eachforstandard
Blueprints of
healthy development.
Available at: http://www.blueprintsprograms.com
evidence
49
Substance means.
Abuse and Mental Health Services Administration: National registry of evidence based programs and practices.
50
Project Oracle. Available at: http://project-oracle.com/
Costs/cost-benefit
51
Early Intervention Foundation.analysis
Available at: http://www.eif.org.uk/
52
European Platform for investing in Children (EPIC). Available at: http://europa.eu/epic/
48
University of Colorado Boulder, Center for the Study and Prevention of Violence (CSPV): Blueprints for healthy development.
Available at: http://www.blueprintsprograms.com
49
34
Substance Abuse and Mental Health Services Administration: National registry of evidence based programs and practices.
50
Project Oracle. Available at: http://project-oracle.com/
51
Early Intervention Foundation. Available at: http://www.eif.org.uk/
52
European Platform for investing in Children (EPIC). Available at: http://europa.eu/epic/

34
(or index of standards) and thereby a common understanding of what each standard of
evidence means.
Costs/cost-benefit analysis
A crucial component in deciding which programme to use, is of course the cost. Most of the
US programmes and some of the UK ones clearly define the costs of using an existing
programme. These will include buying the manual or instructions and the evaluation
materials. Often training the trainers is also necessary and some programmes also have a
helpline for those implementing a programme. Given that in replicating a programme,
implementation fidelity is key (that is implementing the programme as designed) these
costs cannot be avoided. Of course, if the programme is adapted to suit the context and
setting, it is not the same programme and similar outcomes seen in the original cannot be
expected.
The other side of cost is estimating the possible financial benefits of a successful
programme. This may be crucial in enabling funding to be released for a project and its
evaluation. Many US and UK sites now include a cost/cost-benefit analysis. In a cost-benefit
analysis the hypothesis may be formulated as follows: if six older people are helped to stay
in their own home with support for an extra year, the saving might be X compared to the cost
of residential care.
This is an important component when managers are making judgements about where to
focus resources, since as we highlighted earlier on in the document, a director may look at
what would be the cost of their community care versus their cost of residential care as well
as the personal benefit that most people may want to stay in their own homes for as long as
possible. In each of the listed databases, there is a note as to why the site may be useful,
including whether information on costs was included.
34

Concluding key messages
There is a range of evidence needed to develop evidence-based social work.
First, there is a requirement for basic research so that risk and protective factors can be
identified.
Second, there is also a need for information about the extent of the problem, in order to
plan ahead and develop resources for particular groups as well as to focus valuable
resources to the most needy problems and areas.
Third, it is key to obtain knowledge about evidence-based programmes that have been
successful. Those responsible for commissioning services may use a programme that has
already proved its effectiveness elsewhere or develop their own version of this programme.
However, this will need to be tested again as it will effectively be a different programme with
different outcomes. Otherwise, they may develop their own innovative programme that is
appropriate to their context and needs, and which they can feel “ownership”.
Fourth, looking into evaluating your programme a final step may be to make a partnership
with a research facility so that the programme can be evaluated.
Reviewing international evidence based social work databases
There are good sites throughout Europe relevant to evidence-based social work, but most
knowledge they provide is at the level of basic information and research syntheses on
different target populations.
There are fewer sites outlining programmes that work, and when they are available, they
mostly come from the United States and the UK. Although there might be a reluctance in
Europe to import these programmes, some of the better known have been tested in Europe
and have demonstrated their effectiveness. However, a common trend across all sites is the
considerable confusion about the various definitions given regarding a programme’s
effectiveness.
A one-stop-shop database
For both, financial (making better use of resources) and ethical reasons (those in need
deserve the most effective interventions), we have identified the need for an EU site that
develops a one-stop-shop database for all social work target populations, which includes
basic knowledge and research, and a register of evidence-based practices that have been
generated in Europe.
Such a site needs to bring together the various definitions of what is an evidence-based
programme, so that those responsible for designing and commissioning services can decide
what is best for their setting. This site should also include a translation facility to ensure that
the information is available in separate EU languages.
Designating funding for this and for quality evaluations, including cost/cost-benefit
analyses of innovative EU projects, will be a necessary step to modernising social work and
making better use of resources.
35

Appendix 1
Selection of international evidence-based social services databases by
country
Denmark
Database category: Multi-focused database with EBP elements
Topics: Active inclusion; Ageing and care; Children, youth and families; Disability; Housing
and homelessness; Mental health; Social services management
Name The Danish National Centre for Social Research
Focus of The main area is in the field of social services and welfare. The Centre
work has an independent department, which contributes to improving the
knowledge base in regards to the effects of initiatives in the social and
welfare sector. Their work includes the labour market, unemployment,
equal opportunities and gender equality, integration policy, societal and
employment-related aspects of health, child and youth policy, family
policy, school and education, disability policy, the social housing sector,
vulnerable groups, volunteer work, care for the elderly, and pensions.
Type of Publications in English from the Danish National Centre for Social
information Research cover mostly working papers and report abstracts, but also
research publications, evaluations, data collections, and studies.
Other types The Centre conducts research and carries out commissioned projects in
of activities the area of public welfare policies. The Centre often works alongside other
research institutions and with private consultancy firms concerning
tenders and successful completion of large evaluation and elucidation
projects. The Centre disseminates its research and commissioned
projects to the public via its website, and by publishing reports, working
papers, a quarterly magazine, press releases, and presentations.
Standards of The Centre is focusing on improving implementation of commissioned

Evidence projects so that they become more useable for customers/users. This is
taking place through close dialogue, solid utilisation of existing knowledge,
international perspective, and targeted communication.
Availability Open access, free, online. A search engine for topics is available.
Outcomes/ Different formats take effects of social programmes into account, such as
Impact systematic reviews.
Is this site The combination of a broad range of topics and high-quality information
useful? makes this suite useful.
Website http://www.sfi.dk
37

European Union
Database category: Multi-focused database with EBP elements
Topics: Children, youth and families
Name European Platform for Investing in Children (EPIC)
Focus of The European Platform for Investing in Children (EPIC), linked to the
work RAND corporation, is an evidence-based online platform that provides
information about policies that can help children and their families facing
socio-economic challenges in Europe. The European Commission funds
EPIC. One of the main aspects of this work consists in a review of
evidence-based practices submitted to EPIC.
Type of The information includes practices, an evidence guide, country profiles,

information statistics, studies and reports, as well as news about events and activities.
Other types EPIC also provides information about recent events, policy initiatives, and
of activities legal changes intended to support families in Europe, upcoming
conferences, seminars, and meetings on policies and practices affecting
children and families.
Standards of Evidence-based practices are assigned one out of three evidence levels.
Evidence Practices are assessed on transferability and endurability:
x Best Practice: A “best practice” needs to have achieved at least a

+ in each of the three evidence categories, including “evidence of
effectiveness”, “transferability” and “enduring impact.”
x Promising Practice: A “promising practice” needs to have achieved
at least a + in “evidence of effectiveness” and a + in at least one of
the other two categories, “transferability” and “enduring impact.”
x Emergent Practice: An “emergent practice” needs to have
achieved at least a + in “evidence of effectiveness.” A graphical
overview on evidence designations is available on the website.
EPIC includes an inventory for practices, which have not yet been
evaluated but share information on ideas.
Availability Open access, free, online. It is possible to browse the platform by names,
policy categories, countries, and by evidence level.
Outcomes/ The evidence-based practices are based on reviews taking into account
Impact outcomes, for which the category “enduring impact” is an example.
Is this site This site presents high-quality content about evidence on interventions for
useful? children, young people, and families.
Website http://europa.eu/epic/practices-that-work/evidence-based-
practices/index_en.htm
38

Germany
Database category: Basic problems and research overview
Topics: Ageing and care; Evidence and innovation; Social services management
Name Gerostat
Focus of The German Centre of Gerontology is a scientific research centre with

work the focus on the living arrangements, life situations, and life-styles of
ageing people. The Research Data Centre provides access to the micro-
data of the German Ageing Survey (DEAS) and of the German Survey
on Volunteering (FWS) to scholars for non-commercial purposes and
advise potential users. These cover the themes employment and
retirement, economy of old age, health and social care, family and social
relations and societal participation.
Type of GeroStat is an electronic information system for gerontological and

information demographic issues in social research, social reporting, and social
policy. It provides a significant collection of statistical data, additional
contextual information, and statistical reports relevant to social
gerontology. The formats come in fact sheets, reports, discussion
papers, and publications.
Other types An important task of the DZA is to report and process information and to
of activities provide advice on social policy. The target groups are public
administrations at the federal and state levels, central public and non-
governmental welfare agencies and those involved in social policy for the
aged.
Standards of The website does not address project evidence, thus no evidence
Evidence standards are applied.
Availability Open access, free, open. Information can be found in the library section.
Outcomes/ This site is more about statistical information and presents structural
Impact developments rather than looking at outcomes and impact.
Is this site A great source of data and further information about older people is
useful? available, which can serve as a reference point.
Website https://www.gerostat.de
38

Norway
Database category: Systematic reviews database
Topics: Learning disability and physical disability; Social services management; Evidence
and innovation
Name Campbell Library
Focus of The Campbell Collaboration is an international research network that

work produces systematic reviews of the effects of social interventions in
crime and justice, education, international development, and social
welfare. Systematic reviews follow guidelines and standards for
summarising international research on the effects of interventions.
Type of Campbell Systematic Reviews is the peer-reviewed online monograph

information series of systematic reviews prepared under the editorial control of the
Campbell Collaboration. Campbell systematic reviews follow structured
guidelines and standards for summarizing the international research
evidence on the effects of interventions in crime and justice, education,
international development, and social welfare.
Other types The Campbell Collaboration provides a meeting place for policy makers,
of activities practitioners and researchers, and stimulates the work of the
organisation. The programme features a variety of presentations of
completed systematic reviews and the opportunity to attend training
sessions on methodology and other issues. Their portfolio includes
training sessions, and they arrange stand-alone training workshops run
by qualified and experienced teachers.
Standards of A systematic review uses transparent procedures to find, evaluate and

Evidence synthesise research. Procedures are defined in advance and studies are
screened for quality. Peer review is a key part of the process, in which
independent researchers control the methods and results. A systematic
review must have inclusion/exclusion criteria, a search strategy,
systematic coding and analysis of included studies, meta-analysis.
Campbell reviews undergo both peer review and editorial review.
Availability Open access, free, online. An advanced search function is available.
Outcomes/ The systematic reviews include effects of interventions. No formalised

Impact levels of evidence are applied.
Is this site This is a very comprehensive site with a great level of insight on
useful? research and interventions.
Website http://www.campbellcollaboration.org/lib/
39

United Kingdom
Database category: Basic problems and research overview
Topics: Mental health; Ageing and care; Children, youth and families
Name Dementia Gateway
Focus of The Dementia Gateway offers a variety of information resources on

work dementia developed by SCIE (Social Care Institute of Excellence), others
developed by external organisations. The topics are early signs and
diagnosis, support after diagnosis, end of life care, getting to know the
person with dementia, communicating well, difficult situations, eating well,
keeping active and occupied, decision-making, partnerships with carers,
young onset dementia, and environment.
Type of The Dementia Gateway contains information and resources, including

information videos, e-learning material and downloads, for care workers, people living
with dementia, and their friends and family.
Other types Links are included to other websites that provide information about
of activities dementia for professionals, overviews of the evidence base behind key
areas regarding dementia.
Standards of Formalised standards of evidence are not applied; however, the key
Evidence messages in the research overviews identify the most relevant findings.
Availability Open access, free, online. All information is in English.
Outcomes/ Impact of dementia at different stages is considered, but is not

Impact systematically analysed.
Is this site The site offers useful information for people suffering dementia and their
useful? family and friends. It also offers more detailed and technical information
and learning resources that can be used by carers and other professionals
in the area of dementia.
Website http://www.scie.org.uk/dementia/resources/index.asp
40

United States of America
Database category: Systematic reviews database
Topics: Children, youth and families; Evidence and innovation
Name Colorado Blueprints
Focus of The Blueprints mission is to identify evidence-based prevention and

work intervention programmes that are effective in reducing antisocial
behaviour and promoting a healthy course of youth development.
Type of Blueprints has reviewed more than 1,250 programmes. One can search
information by key word, or search across categories (age, programmes specifics, risk
and protective factors. The site includes a matrix comparing programmes
recommended by other EBP sites. Blueprints has provided training,
materials, implementation support and fidelity monitoring to school
districts in 16 US states, serving approximately 284,000 students.
Other types The Blueprints Conference brings together researchers, programme

of activities designers, community leaders and advocates, policy-makers, practitioners
and funders to learn about evidence-based youth development programs.
The goal of the conference is to provide information on evidence-based
programmes and guidance and tools to help consumers implement these
programmes successfully.
Standards of There are two levels: model programmes and promising programmes.
Evidence
 Promising programmes: the outcomes and the population for
which it is intended are clearly identified. Features are at least one
randomised controlled trial (RCT) or 2 quasi-experimental
evaluations, a significant positive change that can be attributed to
the intervention, the absence of harmful effects, and the
programme is ready to disseminate (i.e. manual available,
technical assistance).
 Model programmes embrace a minimum of 2 RCTs or 1 high
quality RCT plus high a quality quasi-experimental evaluation. A
positive impact is sustained for 12 months after the intervention.
Availability Open access, free, online.
Outcomes/ The search function includes the possibility to look for outcomes on
Impact specific problems.
Is this site The site presents a high quality evidence site. Many of the projects have
useful? been replicated internationally.
Website http://www.blueprintsprograms.com/
41

Appendix 2
Questionnaire for planning social services
I. General information about the programme

1. Name of programme
2. Country
3. Region
4. Municipality
5. Responsible organisation
6. Contact details of the responsible person(s)
7. Any other partners involved (Y/N)
II. Evidence about the target population
programme: volume, demographics and socio-economic characteristics. Knowing the extent of
a given social problem can help in prioritising areas for intervention but also planning ahead
and having facilities available for specific needs. Monitoring social changes (number of cases in
a population in a given period) can be useful in giving an indication about whether a problem is
increasing or decreasing and whether a specific programme needs to be put in place to meet
the needs of a community.
1. Who is the project for?
2. What is the age of the target population?
3. Gender (m/f/both)
4. Is this project for a particular ethnic group? If yes, please state the group.
5. What will be the level of intervention?
a. Primary (general population)
b. Secondary (those at risk)
c. Tertiary (those with an existing problem)
III. Evidence about the programme context
In deciding what a social problem is, this will need to be contextualised against the norms and
values, ethnic and morals and the group identity in any given community. Different societies at
different times will have different views and different awareness about what is or what is not a
social problem. This information can only come from those involved in their communities.
1. Policy background
a. International
b. National
c. Regional
d. Local
2. What issues in social services are relevant for the programme?
3. What are the Risk and Protective Factors for the problem?
4. What organisation is responsible for the programme area?
a. Are the relevant staff provided with training?
IV. Evidence about the programme implementation
This type of evidence refers to the conditions known to drive or inhibit implementation, such as
organisational, inter-personal and individual factors. When replicating an evidence based
programme a key concern is “implementation fidelity” or “implementation with integrity”. As
mentioned earlier, the programme may need to be adapted to different cultural contexts and
42

local conditions. However, any adaptation of the programme means that it is a different
programme and outcomes may differ simply because the crucial elements have not been
replicated.
1. What services and activities does the programme deliver?
2. Does the programme design follow a model or concept?
3. What is the status of the programme?
a. Is the programme based on an existing programme?
b. If the programme has been evaluated, can we determine with what level of
evidence?
4. What is the scope of the initiative?
5. Who has the leadership and management of the initiative?
6. What is the involvement of users, families, and/or carers? How is their engagement
facilitated?
V. Evidence about the programme’s effects
There are varying levels of evaluation, which are often called ‘the hierarchy of evidence’. These
hierarches suggest how much confidence one can have in the evaluation. In assessing the
effects of a programme, ‘outputs’ are sometimes confused with ‘outcomes’. An example of an
output is how many people participated in the programme. ‘Outcomes’ are what happens to
participants having taken part in a programme. If using a validated programme, there will be
details on outcomes, which need to be examined carefully.
1. What are the evaluation methods of the programme?
a. How will you measure the situation at baseline?
b. Is there a local research facility/academic organisation that would advise in
undertaking an evaluation?
2. What are measurable effects of the programme and what has it achieved?
a. Are there standardised measures which could be used to evaluate their results?
3. What are the anticipated or ‘aspirational’ effects of the programme?
4. Is the programme sustainable?
a. What are plans to follow up on your project to see whether benefits have been
sustained?
5. Could the programme be transferable?
VI. Evidence about the programme’s resources
This type of evidence is concerned with the unit costs per output, outcome, and impact, as well
as cost-benefits of (potentially) implemented programmes. A social services director may want
to assess the estimates of costs/benefits of running a successful project. This is an important
component when managers are making judgments about where to focus resources.
1. What are the costs of running the programme?
2. On which other relevant resources does the programme rely?
3. How is the programme funded?
4. Is there funding available specifically for evaluation?
5. What are the estimates of costs and benefits for running this programme
successfully?
VII. Further information sources / background documents / website
43

Appendix 3
Questionnaire for the evaluation of a social service
I: Initial considerations
It is important to consider how the evaluation of your social service programme will be
undertaken. Money will be wasted and learning lost unless an evaluation plan is built in from
the beginning. The following is a simplified guide which suggests most of the areas you need
to consider when planning the evaluation of your social service programme.
In addition to the name and contact details of the person responsible for the programme, it is
key to assess what the aims of the programme are. It is helpful here to outline the key
question(s) you may want to answer.
For instance:
- Does the programme X reduce the amount of....?
- Perhaps also a few secondary questions, such as for whom, when and at what cost? Are
there any other benefits?’
- When developing the questionnaires and tools, go back to the initial questions to check if
they are able to give the answers you want.
II: What resources are available for evaluation?
As stated in this toolkit, there are various levels of evidence (often called the ‘hierarchy’ of
evidence) that can be collected from evaluation studies. Basically the higher the level, the
more expensive the evaluation.
If developing your own project and not using a programme developed elsewhere, it is
generally better to start with a simple evaluation, such as a before/after design. That is
measuring the extent of the problem before the intervention and then measuring again after
the intervention. If results look promising, a research agency may be then able to undertake
a more complex evaluation.
Below some of the questions you should consider at this stage.
1. What funding/resources are available for running the intervention and for undertaking the
evaluation?
2. What expertise is available to help?

For instance, help from a research agency or an academic department is likely to improve
the quality of the evaluation.
3. What staff are available to run the programme? What training will they need?
4. Where will the intervention take place?
5. Who is available to evaluate the programme?

It is better that this person/agency is independent from those running the programme.
45

6. What other stakeholders are there who could be involved? Are there any people
interested in the outcomes and who may contribute to cover the costs?
III. What is the level of the programme?
If the programme is a primary intervention, it would focus on a large group with the aim of
preventing problems. If it is a secondary intervention, it would target those who are at high
risk of a problem, whilst if it is a tertiary intervention, it would aim at limiting harm for those
who have experienced a social problem.
In the case of a primary intervention; for example, advice to parents to avoid accidents at
home, simple monitoring of the numbers of children presenting at hospital following
accidents may be sufficient. In the case of secondary; for example, parents at risk of
maltreating their children, or tertiary; for example, parents who have maltreated their
children; effective evaluations will need information directly from the parents.
At this stage, you should consider at least the following two questions.
1. Is there a simple primary type intervention, such as the use of media or leaflets, which
would reach all those involved?
2. What would be the cost of doing this?
Though media interventions can be relatively cheap, they may not reach the population that
you want.
IV. What programme are you planning to use?
The advantage of using an existing programme is that much of the thinking will have been
done for you and evaluation tools may well come as part of the package. Therefore, it is
important that you think as to whether you will replicate another programme or develop your
own.
1. Are you planning to replicate an existing evidence based programme?
For this, you will need to check costs of training, and materials, which may be available on
their website.
2. Are you planning to adapt an existing evidence based programme?
Remember any adaptation means that it is a different programme and results may be
different but it is helpful to use their materials as far as possible.
3. Are you planning to develop your own programme?
Make sure you have fully researched the background to the problem and found what may be
the best focus for an intervention. For example, a programme to improve parenting skills and
parental confidence may have also the benefit of reducing child abusive incidents.
46

V. What else do you need to know for your evaluation?
The resources available and the type of intervention will to some extent be dependent on the
answers to the following questions.
1. What numbers are likely to be involved in the programme?
The most common failing in evaluations is that numbers involved rarely live up to
expectation, since people drop out, or fail to turn up. One way to ensure participation is by
looking at providing incentives to encourage participation.
2. What will be the criteria for those who take part?
For example, parents living in district X who have been referred to social services with child
abuse concerns.
3. How will you obtain the names of possible participants?
4. How will you inform the participants about the project?
All participants should sign a form saying that they understand the purpose of the
intervention and agree to take part for the necessary number of sessions and weeks
(informed consent).
5. Will I need ethical approval for the study?
If children or vulnerable adults (i.e. those with dementia or with a disability are involved) it is
important to check this.
6. How will keep the information I obtain confidential and secure?
For instance, anonymously numbering questionnaires rather than putting names on them is
safer, with one person holding a master list.
VI. What type of evaluation?
Depending on the type of programme, you will need to plan the type of evaluation.
For primary interventions, monitoring incidence is all you need. For example, following a
media campaign focussing on domestic violence, a reduction in the number of cases
reported to police for domestic violence, might indicate change.
For secondary and tertiary interventions the following methods are suitable:
a) Before/After study. This takes a measure of the extent of problems before an

intervention and then again at the end. The disadvantage is that you may not know whether
changes are due to other events. For example, with mothers at risk of abuse, the differences
seen may be influenced by the arrival of a new day nursery in the area.
b) A ‘controlled’ study. This measures the difference between two groups before and after.
For example, one group of mothers received the intervention while the other group wait their
turn to take part. This is better than a simple before/after study but you may never be quite
47

sure that the ‘waiting list’ control group is the same as those undertaking the intervention (for
the reasons mentioned above –participants dropping out or not turning up.
c) Randomized controlled trial. This is where the names of all possible participants are put
into a hat and names for those taking part in the intervention are randomly picked out of the
hat. Those not chosen become the control group.
d) Mixed methods. You may decide that you want to know more than just numbers, so you
interview a random sample of those taking part in the interventions and those who have not.
Their comments can be useful in understanding why there are the differences seen.
In order to decide what evaluation is most suitable, you should consider at least two
questions.
1. What type of evaluation is possible with the resources that are available?
2. Within the limits of resources, what type of evaluation will produce the highest quality
evidence?
VII. How will you measure change?

You can develop your own measures. For example, a scale of 1 to 5 where 1 is “I do not
feel that I have the skills” and 5 “I feel very positive about my (parenting) skills”. But much
better is to use ‘standardised measures’ that is questionnaires that have been used and
validated in large studies.
When replicating programmes that have been trialled elsewhere, they will generally have
their own measures which will come as part of the package when buying the programme.
When adapting one of these programmes to your own context, it may be useful to look at the
measures they use, translate and adapt where necessary.
Therefore, you may want to consider the following.
1. Are there standardised measures that you use for your evaluation?
2. If you are developing your own measures, how will you pilot them to test that participants
understand them and they measure what you want to measure?
3. Some measures will already have validated international translations available. Have you
checked whether these are available?
4. If you are translating a standardised measure, it is best first to translate it into your
language and then ask someone else to translate back to the original. This way you can
check if there is any change in meaning.
VIII. What other data will you need to collect?
There are two ways to collect basic data:
1. by questionnaire. In developing a questionnaire, you should remember that participants

have limited patience in filling in forms and their reading age may be limited. As a guide,
questionnaires for most groups should not take more than 20/30 minutes to complete.
47

2. Collecting data by interviews, which is a more labour intensive process. Interviews as a
rule, even in depth interviews should not last more than 40 minutes to 1 hour. Telephone
interviews can be cheaper and quicker.
Sensitive questions need to be worded carefully. For example, participants rarely give
honest answers to questions about their household income. In deprived groups, if your
country has this service, a useful way to access this information, is to ask if their children
have ever received ‘free school meals’.
Information needs to be collected at:
a) baseline, that is the start of the project
b) end of project
c) at any follow up period decided upon to see if the results continue after the end of the
programme.
You need to pilot all questionnaires to see if they provide the answers that you require.
When it comes to collecting data after the intervention, you will repeat the original
standardised measure, and you may want to ask participants a question on how they felt
about the programme and whether it could be improved.
When it comes to collecting additional data, consider the following questions.
1. What other information do you want to collect, about the household, income, number of
children, relationships, and others?
2. What is the simplest and least intrusive way of obtaining this information? If you intend to
use a questionnaire, have you piloted it with a few people to see if they understand the
questions, and whether they feel the questions are acceptable?
3. Have you tested how long it takes to obtain the information?
4. When will you collect the data?
IX. How will you add it all up?

The simplest method is to take the before questionnaires of those who took part in the
intervention and then compare the results with those who did not take part.
If; for example, you were assessing whether an improvement in a residential home had
made any difference, you could:
- Use a quality of life measure (1 = totally satisfied with my life and 5 =totally unsatisfied)
- Measure at the beginning and then repeat it at the end to see if there have been any
changes.
- Interview a few residents to try and understand what changes have made them feel better.
Standardised measures give instructions as to how their questionnaires should be added up.
48

Research institutions may be helpful. If the numbers involved are large enough to do some
statistical tests, it will be possible to see whether the results are better than could have
occurred by chance (that is, your results are statistically significant).
Therefore, you should consider the following questions.

Research institutions may be helpful. If the numbers involved are large enough to do some
statistical
1. Can you tests,
add itup
will
theberesults
possible to see whether the results are better than could have
yourselves?
occurred by chance (that is, your results are statistically significant).
2. Are the number large enough for statistical tests?
Therefore, you should consider the following questions.
Generally such tests need samples of more than 50 both in the group who have had the
1. Can you add
intervention, andupinthe
theresults
controlyourselves?
group.
2.
3. Are the number
Should large
you involve anenough
outsidefor statistical tests?
agency?
Generally such
If a research tests need
agency samples
is involved theyofshould
more than 50 both since
be involved in thethe
group who have had the
start.
intervention, and in the control group.
4. Are the results poorer than expected?
3. Should you involve an outside agency?
Do not despair if that is the case. Much learning comes from having undertaken a project,
Ifbut
a research agency
good results mayisnotinvolved
alwaysthey should beininvolved
be apparent the shortsince
term.the start.
In the famous High scope
Perry Pre School Project the real returns were not seen until 27 and 40 years later.53
4. Are the results poorer than expected?
X. So What?
Do
Younot despair
have if that isthe
undertaken theresearch
case. Much learningoutcomes
to improve comes from having undertaken
for participants. a project,
When planning
but good results may not always be apparent in the short term. In the famous
your intervention you need to think from the beginning what you will do with your results. High scope Too
Perry
often Pre
goodSchool Projectget
evaluations theput
real
upreturns wereshelf
on a dusty not seen untilseen
and not 27 and 40 A
again. years later.
useful
53
strategy is to
ask
X. Sothose responsible for implementing service changes to write a few words in the
What?
evaluation report about
You have undertaken thehow the organisation
research to improveplans to make
outcomes use of the findings
for participants. When and what
planning
actions
your they will actually
intervention you need undertake as a the
to think from result.
beginning what you will do with your results. Too
often good evaluations
Remember get put
to ask yourself the up on a dusty
following shelf and
questions not dissemination
around seen again. A and
useful
usestrategy is to
of findings.
ask those responsible for implementing service changes to write a few words in the
evaluation
1. Who willreport about howfor
be responsible thedisseminating
organisation the
plans to make use of the findings and what
results?
actions they will actually undertake as a result.
2. What is the implication of the evaluation findings for other services both within your
Remember
organisationtoand
askfurther
yourself the following questions around dissemination and use of findings.
afield?
1.
3. Who willoriginal
Are the be responsible for disseminating
stakeholders the results?
able to comment on the findings and suggest next steps?
2.
4. What is the
How can implication
you of thefunding
ensure future evaluation findings
for the for other services both within your
project?
organisation and further afield?
Too often good projects get forgotten about because of lack of funding. The more noise you
3. Are about
make the original stakeholders
the success of yourable to comment
project the moreon the you
likely findings and
are to suggest
receive nextfunding.
further steps?
4. How can you ensure future funding for the project?
Too often good projects get forgotten about because of lack of funding. The more noise you
make about the success of your project the more likely you are to receive further funding.
53
High Scope Educational Research Foundation (2005): Lifetime effects. The high scope perry preschool study through age
40. Available at: http://www.highscope.org/content.asp?contentid=219
49
53
High Scope Educational Research Foundation (2005): Lifetime effects. The high scope perry preschool study through age
40. Available at: http://www.highscope.org/content.asp?contentid=219

49
Appendix 4
Selection of standardised evaluation measures
When social services directors develop their own programme, it is important to include some
Appendix 4
standardised measures to monitor possible effects. A number of standardised measures
have been selected
Selection by Ann Buchanan,
of standardised Centre for
evaluation Evidence Based Research at the
measures
University of Oxford, for the European Social Network.
Whendocument
This social services directors
provides develop their
social services own and
directors programme, it is important
senior practitioners withtoainclude some
selection of
standardised measures to monitor possible effects. A number of standardised
standardised evaluation measures, which could be used as a baseline at the start of a measures
have been
project andselected by Ann Buchanan,
then be reapplied at the endCentre
of the for Evidence
project Based
to monitor Research at the
changes.
University of Oxford, for the European Social Network.
About using standardised evaluation methods
This document provides social services directors and senior practitioners with a selection of
standardised
Standardised evaluation
measures measures, which couldbybepsychologists
have been developed used as a baseline at thesample
using large start oftoa test
project and thenAs
their accuracy. besuch
reapplied at the
they are end
more of the than
reliable project
anytomeasures
monitor changes.
you may develop. They
are used to demonstrate if there has been any change following an intervention.
About using standardised evaluation methods
Sometimes these measures are called ‘scales’; for example, the Strengths and Difficulties
Standardised measures have been developed by psychologists using large sample to test
Questionnaire below, as they have a number of questions and the answers from the
their accuracy. As such they are more reliable than any measures you may develop. They
questions are added up to form a total score. This gives a rating at the start of the
are used to demonstrate if there has been any change following an intervention.
intervention to which we can compare at the end of the intervention.
Sometimes these measures are called ‘scales’; for example, the Strengths and Difficulties
The measures/scales should be completed by the client in privacy and if possible
Questionnaire below, as they have a number of questions and the answers from the
anonymously. This way they record the client’s view without being influenced by others.
questions are added up to form a total score. This gives a rating at the start of the
intervention
Most of thesetomeasures/scales
which we can compare at the
are freely end of on
available thethe
intervention.
Internet.
The measures/scales should be completed by the client in privacy and if possible

The StrengthsThis
anonymously. andway
Difficulties Questionnaire
they record (SDQ)
the client’s view without being influenced by others.
The Strengths and Difficulties (SDQ) is a brief behavioural screening questionnaire for
children
Most aged measures/scales
of these 3 to 16. It exists inare
different versions to
freely available onmeet the needs of researchers, health
the Internet.
and education professionals. All versions of the SDQ ask about 25 attributes, which are also
divided between 5 scales: Emotional symptoms, conduct problems, hyperactivity/inattention,
The Strengths and Difficulties Questionnaire (SDQ)
peer relationship problems, pro-social behaviour.
The Strengths and Difficulties (SDQ) is a brief behavioural screening questionnaire for
children aged 3 to 16. It exists in different versions to meet the needs of researchers, health
25 items are included in questionnaires for completion by the parents or teachers of 4-16
and education professionals. All versions of the SDQ ask about 25 attributes, which are also
year olds.54 There is a slightly modified version for the parents or nursery teachers of 3 (and
divided between 5 scales: Emotional symptoms, conduct problems, hyperactivity/inattention,
4) year olds. Questionnaires for self-completion by adolescents ask about the same 25
peer relationship problems, pro-social behaviour.
attributes, though the wording is slightly different for them.55
25 items are included in questionnaires for completion by the parents or teachers of 4-16
SDQ has been translated (and validated) into many languages. These translated versions
year olds.54 There is a slightly modified version for the parents or nursery teachers of 3 (and
are available for free on the website. This questionnaire has been used in national studies;
4) year olds. Questionnaires for self-completion by adolescents ask about the same 25
for instance, in the UK to assess children at risk of mental 55health problems. The only
attributes, though the wording is slightly different for them.
restriction is that the wording must not be changed in any way.
SDQ has been translated (and validated) into many languages. These translated versions
are
54 available
Goodman, for free
R. (1997): on the and
The strengths website. This
difficulties questionnaire
questionnaire: A research has been
note, used
Journal in national
of Child Psychology studies;
and
Psychiatry, Vol. 38:5, pp. 581–586. Available at: http://onlinelibrary.wiley.com/doi/10.1111/j.1469-7610.1997.tb01545.x/abstract
forGoodman,
55 instance, in the UK to assess children at risk of mental health problems. The only
R.; Meltzer H.; and Bailey, V. (1998): The strengths and difficulties questionnaire: A pilot study on the validity of
restriction
the self-report is that European
version, the wording must
Child and not bePsychiatry,
Adolescent changed Vol.in7:3,
anypp. way.
125-130. Available at:
http://link.springer.com/article/10.1007%2Fs007870050057#page-1 See more at: www.sdqinfo.com.
50
54
Goodman, R. (1997): The strengths and difficulties questionnaire: A research note, Journal of Child Psychology and
Psychiatry, Vol. 38:5, pp. 581–586. Available at: http://onlinelibrary.wiley.com/doi/10.1111/j.1469-7610.1997.tb01545.x/abstract
55
Goodman, R.; Meltzer H.; and Bailey, V. (1998): The strengths and difficulties questionnaire: A pilot study on the validity of
the self-report version, European Child and Adolescent Psychiatry, Vol. 7:3, pp. 125-130. Available at:
http://link.springer.com/article/10.1007%2Fs007870050057#page-1 See more at: www.sdqinfo.com.
50
In Ann Buchanan’s experience, who has used this measure in many of her studies, the
advantage of this tool is that it shows how a specific child, or groups of children sharing
common characteristics, are responding to the particular challenges they face. For example,
children whose parents are disputing in the courts over the child’s care. A second advantage
is
In that
Ann outcomes
Buchanan’s forexperience,
children from small
who hasstudies canmeasure
used this be compared with
in many ofoutcomes forthe
her studies, children
from larger or national studies.
advantage of this tool is that it shows how a specific child, or groups of children sharing
common characteristics, are responding to the particular challenges they face. For example,
Quality of life parents are disputing in the courts over the child’s care. A second advantage
children whose
is that outcomes for children from small studies can be compared with outcomes for children
A standardised
from measure
larger or national to assess life quality: University of Sheffield’s simple
studies.
measure of quality of life 56
A very simple
Quality tool is this quality of life measure. Although very easy to administer, it is widely
of life
respected. A possible scenario when this survey may be used would be as follows: a group
of standardised
A residents; for example,
measureintoaassessfacility for
lifeolder people,
quality: where changes
University are to be
of Sheffield’s made to
simple
improve their quality of
measure of quality of life life,56can be asked to complete it before the changes are made. It is
important
A eachtool
very simple person
is thiscompletes
quality of this independently.
life measure. Several
Although verymonths
easy tolater, after the
administer, it is widely
changes are
respected. A made,
possiblethescenario
exercisewhencan bethisrepeated.
survey mayThisbegives
usedawould
simplebe
before/after
as follows:evaluation
a group
of
of residents; for example, in a facility for older people, where changes are to be madetotothe
the impact of the changes. Of course, an improvement may not be entirely related
changes, as others
improve their qualityfactors
of life, may also
can be havetoancomplete
asked impact, such as athe
it before change of staff,
changes budgetary
are made. It is
issues.
important each person completes this independently. Several months later, after the
changes are made, the exercise can be repeated. This gives a simple before/after evaluation
Thinking about
of the impact ofyour own life and
the changes. personal
Of course, ancircumstances, how not
improvement may satisfied are you
be entirely with your
related to thelife
as a whole?
changes, as others factors may also have an impact, such as a change of staff, budgetary
issues.
Completely Completely
Thinking about your own life and personal circumstances, how satisfied are you with your life
Dissatisfied Satisfied
as a whole?
0 1 2 3 4 5 6 7 8 9 10
Completely Completely
Dissatisfied Satisfied
0 1 2 3 4 5 6 7 8 9 10
Positive Psychology Centre, University of Pennsylvania57

The measures from this Centre are similar to the ‘quality of life’ scale listed above. They are
quick and easy to use and can be administered in a similar fashion. They have a variety of
uses for most adult client groups, such as adults with disabilities, older people.
Positive Psychology Centre, University of Pennsylvania57
Useful
The measures
measures retrieved
from from are
this Centre thissimilar
site include:
to the ‘quality of life’ scale listed above. They are
quick and easy to use and can be administered in a similar fashion. They have a variety of
uses for most adult client groups, such as adults with disabilities, older people.
Useful
56
measures
University of Sheffield:retrieved from this
Wellbeing measures. site include:
Available at:
https://www.shef.ac.uk/polopoly_fs/1.72794!/file/Resources_for_measuring_wellbeing.doc
57
University of Pennsylvania, Positive Psychology Center: Questionnaires for researchers. Available at:
http://ppc.sas.upenn.edu/resources/questionnaires-researchers
51
56
University of Sheffield: Wellbeing measures. Available at:
https://www.shef.ac.uk/polopoly_fs/1.72794!/file/Resources_for_measuring_wellbeing.doc
57
University of Pennsylvania, Positive Psychology Center: Questionnaires for researchers. Available at:
http://ppc.sas.upenn.edu/resources/questionnaires-researchers
51
1. Satisfaction with Life Scale (SWLS)58
2. Psychological Well-being Scales59
3. Subjective Happiness Scale (SHS)60
The Satisfaction with Life Scale gives a longer term view, whereas the subjective Happiness
1. Satisfaction with Life Scale (SWLS)58
Scale may reflect feelings of the moment; for example, happiness following a family visit.
2. Psychological Well-being Scales59
The Psychological well-being scales give more elaborate measurements.
3. Subjective Happiness Scale (SHS)60
Authentic Happiness website61
The Satisfaction with Life Scale gives a longer term view, whereas the subjective Happiness
The following scales are also similar to the above and available for self-completion. Most of
Scale may reflect feelings of the moment; for example, happiness following a family visit.
these have been validated against large populations, so your sample can be compared to
The Psychological well-being scales give more elaborate measurements.
larger samples in other countries. Key references and copy-right information are given.
Scales include those related with emotion, engagement, meaning and life satisfaction.
Authentic Happiness website61
Useful measures available at this site include:
The following scales are also similar to the above and available for self-completion. Most of
these
 have
PANASbeen(Positive
validated against
and largeAffect
Negative populations, so your sample can be compared to
Schedule)
larger
 samples
FordyceinEmotions
other countries. Key references and copy-right information are given.
Questionnaire
Scales include those related with
 Satisfaction with Life Scale emotion, engagement, meaning and life satisfaction.
Useful
 measures
Approaches available at this site
to Happiness Scaleinclude:
 PANAS (Positive and Negative Affect Schedule)

Widely used standardised evaluation methods
 Fordyce Emotions Questionnaire
 Satisfaction with Life Scale
Life satisfaction
Approaches62to Happiness Scale
The
Widelyfollowing
usedthree well-validated
standardised scales aremethods
evaluation available on the web and can be downloaded
for free.
Life satisfaction
Satisfaction 62
with life scale
A five-item scale designed to measure global cognitive judgments of one’s life satisfaction
The
(not following
a measure three well-validated
of either positive orscales are affect).
negative available on the webindicate
Participants and canhowbe much
downloaded
they
for free.
agree or disagree with each of the five items using a seven point scale that ranges from 7
strongly agree to 1 strongly disagree.
Satisfaction with life scale
A five-item scale
Subjective designed
happiness to measure
scale (also knownglobal
as cognitive judgments Scale)
General Happiness of one’s life satisfaction
A four-item scale designed to measure subjective happiness. Each item ishow
(not a measure of either positive or negative affect). Participants indicate much they
completed by
agree
choosingor disagree with each
one of seven optionsof the
thatfive items
finish usingsentence
a given a seven fragment.
point scaleThe
thatoptions
rangesarefrom 7
strongly
different agree to 1ofstrongly
for each the four disagree.
questions (see below for actual items).
Subjective
Bradburn scalehappiness scale (also
of psychologic known as(also
wellbeing General
knownHappiness Scale)
as the Affect Balance Scale)63
A four-item
The scale isscale
madedesigned
up of twotocomponents:
measure subjective happiness.
the positive Each
affect and theitem is completed
negative affect by
choosing oneEach
component. of seven optionshas
component thatfive
finish a given
items. The sentence
scale asksfragment. Theif,options
participants are few
in the past
different for each of the four questions (see below for actual items).
Bradburn
University of scale of psychologic
Pennsylvania, wellbeing
Positive Psychology (also known
Center: Satisfaction with life as theAvailable
scale. Affectat:Balance Scale)63
58
http://ppc.sas.upenn.edu/resources/questionnaires-researchers/satisfaction-life-scale
The
59 scaleofisPennsylvania,
University made up Positive
of twoPsychology
components:
Center: the positivewellbeing
Psychological affect andscale. the negative
Available at: affect
http://ppc.sas.upenn.edu/resources/questionnaires-researchers/psychological-well-being-scales
component.
60 Each component has five items. The scale asks participants if, in the past few
University of Pennsylvania, Positive Psychology Center: Subjective happiness scale. Available at:
http://ppc.sas.upenn.edu/resources/questionnaires-researchers/subjective-happiness-scale /
61
University of Pennsylvania, Authentic Happiness Programme: Questionnaire center. Available at:
https://www.authentichappiness.sas.upenn.edu/testcenter
58
University of Pennsylvania, Positive Psychology Center: Satisfaction with life scale. Available at:
62
The Hong Kong Council of Social Service. Available at: http://www.hkcss.org.hk/uploadfileMgnt/0_201443011362.pdf
http://ppc.sas.upenn.edu/resources/questionnaires-researchers/satisfaction-life-scale
63
59 The Hong Kong
University Council of Social
of Pennsylvania, Service.
Positive Available
Psychology at: http://www.hkcss.org.hk/uploadfileMgnt/0_201443011362.pdf
Center: Psychological wellbeing scale. Available at:
http://ppc.sas.upenn.edu/resources/questionnaires-researchers/psychological-well-being-scales
60
University of Pennsylvania, Positive Psychology Center: Subjective happiness scale. Available at: 52
http://ppc.sas.upenn.edu/resources/questionnaires-researchers/subjective-happiness-scale /
61
University of Pennsylvania, Authentic Happiness Programme: Questionnaire center. Available at:
https://www.authentichappiness.sas.upenn.edu/testcenter
62
63
52
weeks, they have felt certain emotions. The participant answers ‘Yes’ or ‘No’ to each
question. The ‘No’ score is subtracted from the “Yes” score to create a positive/negative
affect difference score.
Mental health
weeks, they have felt certain emotions. The participant answers ‘Yes’ or ‘No’ to each
question.
GHQ-1264The ‘No’ score is subtracted from the “Yes” score to create a positive/negative
affectCHQ-12
The difference score. to be a measure of ‘coping’. Originally consisting of 60 questions
is intended
about mild somatic and psychological symptoms, this questionnaire was later condensed to
Mental
30 and thenhealth
to 12-item questionnaires. Though the 60 question variety is generally too long,
the GHQ-12
GHQ-12 64 is easy to understand and quick to administer. There are various ways to score
the scale
The CHQ-12 isbut based on experience,
intended addingofup
to be a measure the totalOriginally
‘coping’. score from each question
consisting gives a
of 60 questions
more precise
about account.
mild somatic andAs a matter of example,
psychological symptoms, Ann
thisBuchanan suggests
questionnaire was that
lateritcondensed
would be to
good
30 and tothen
assess all parents
to 12-item coming to social
questionnaires. Thoughservices
the 60with this non-intrusive
question scale astoo
variety is generally it long,
indicates in a precise way the level of difficulties/need. The GHQ-12 has
the GHQ-12 is easy to understand and quick to administer. There are various ways to scorebeen used in a
number
the scaleofbutprogrammes, particularlyadding
based on experience, with mothers
up the at riskscore
total of abusing theirquestion
from each children.gives a
65
more precise
Patient Healthaccount. As a matter
Questionnaire of example,
(PHQ-9) 66 Ann Buchanan suggests that it would be
good to assess all parents coming to social services
This easy to use patient questionnaire is a self-administeredwith thisone.
non-intrusive
It is not a scale as it tool
screening
indicates
for in a precise
depression but it isway
usedthe
tolevel of difficulties/need.
monitor The GHQ-12
the severity of depression andhas been used
response in a
to65treatment.
number ofitprogrammes,
However, can be used particularly with mothers
to make a tentative at riskofofdepression
diagnosis abusing their children.
in at-risk populations,
such as Health
Patient those with coronary heart
Questionnaire disease
(PHQ-9) 66 or after stroke.
This easy to use patient questionnaire is a self-administered one. It is not a screening tool
Centre for Epidemiologic
for depression but it is usedStudies Depression
to monitor Scale
the severity (CES-D)67and response to treatment.
of depression
This is a 20-item
However, scale
it can be used questionnaire with four
to make a tentative responseofoptions
diagnosis based
depression in on frequency
at-risk of
populations,
experiencing specified emotional states during the last
such as those with coronary heart disease or after stroke. week. Answers receive a score from
0 to 60.
Centre for Epidemiologic Studies Depression Scale (CES-D)67
Needs
This is abased,
20-itemflourishing and multi-dimensional
scale questionnaire measures
with four response options basedofon
well-being
frequency of
experiencing specified emotional states during the last week. Answers receive a score from
The
0 following are more elaborate versions of the earlier life satisfaction/happiness/well-being
to 60.
scales. The WHO-QOL is interesting as it is aimed to be an international cross-cultural
Quality of
Needs Life measure.
based, It alsoand
flourishing explores various areas or measures
multi-dimensional domains of aofperson’s life. The
well-being
CASP19 is especially developed for use with older people. The particular value of the
European
The Social
following areSurvey well-being
more elaborate measure
versions of is
thethat therelife
earlier are scores from all over Europe,
satisfaction/happiness/well-being
and scores
scales. The from smallerissamples
WHO-QOL canas
interesting beitcompared
is aimed towith
be those from the survey.
an international cross-cultural
Quality of Life measure. It also explores various areas or domains of a person’s life. The
Basic
CASP19 Psychological Needs Scale
is especially developed for (Deci and
use with Ryan)
older
68
people. The particular value of the
This includes
European a family
Social of scales
Survey developed
well-being measurebyisDeci
thatand
thereRyan), one that
are scores fromaddresses needs
all over Europe,
satisfaction in general in one’s life, others address needs satisfaction
and scores from smaller samples can be compared with those from the survey. in specific domains
(e.g. work, inter-personal relationships). The original scale has 21 items concerning needs
Basic Psychological Needs Scale (Deci and Ryan)68
This includes a family of scales developed by Deci and Ryan), one that addresses needs
satisfaction
64 in general
CAMH Knowledge exchange:in one’shealth
General life, questionnaire
others address(GHQ). needs
Available satisfaction
at: in specific domains
(e.g. work, inter-personal relationships). The original scale has 21 items concerning needs
http://knowledgex.camh.net/amhspecialists/Screening_Assessment/screening/screen_CD_youth/Pages/GHQ.aspx
65
The author has used the General Health Questionnaire (GHQ12) version in many studies and it has been very helpful in
assessing change in well-being in adults. There is a Spanish version.
66
Patient Plus: Patient health questionnaire (PHQ-9). Available at: http://www.patient.co.uk/showdoc/40025272/
67
Center for Epidemiologic Studies: Depression scale Available at: https://nts122.chcr.brown.edu/pcoc/cesdscale.pdf
68
University of Rochester: Self-determination theory. Available at:
64
CAMH Knowledge exchange: General health questionnaire (GHQ). Available at:
http://www.psych.rochester.edu/SDT/measures/needs_scl.html
http://knowledgex.camh.net/amhspecialists/Screening_Assessment/screening/screen_CD_youth/Pages/GHQ.aspx
65
The author has used the General Health Questionnaire (GHQ12) version in many studies and it has been very helpful in
assessing change in well-being in adults. There is a Spanish version.
53
66
Patient Plus: Patient health questionnaire (PHQ-9). Available at: http://www.patient.co.uk/showdoc/40025272/
67
Center for Epidemiologic Studies: Depression scale Available at: https://nts122.chcr.brown.edu/pcoc/cesdscale.pdf
68
University of Rochester: Self-determination theory. Available at:
http://www.psych.rochester.edu/SDT/measures/needs_scl.html
53
for three dimensions: competence, autonomy and connexion, but some studies work with
only nine items (three for each dimension).
Approaches to Happiness (OTH) Scale69

This includes
for three three dimensions:
dimensions: meaning
competence, (6 questions),
autonomy pleasure
and connexion, but (6 questions),
some and with
studies work
engagement (6 (three
only nine items questions). Each
for each of these dimensions has the response scale: 1 "Very much
dimension).
unlike me" to 5 "Very much like me". Scoring for each dimension is the average of the 6
questions.
Approaches to Happiness (OTH) Scale69
This includes three dimensions: meaning (6 questions), pleasure (6 questions), and
WHO-QOL 70
engagement (6 questions). Each of these dimensions has the response scale: 1 "Very much
WHO-QOL aimed
unlike me" to to be
5 "Very an international
much cross-culturally
like me". Scoring comparable
for each dimension quality
is the of life
average of the 6
assessment
questions. instrument. It comprises 26 items, which measure the following domains:
physical health, psychological health, social relationships, and environment. The average of
each domain
WHO-QOL 70 is taken, giving a profile of four separate domain scores.
WHO-QOL aimed to be an international cross-culturally comparable quality of life

CASP-19 71
assessment instrument. It comprises 26 items, which measure the following domains:
Quality
physicalofhealth,
life index for older people,
psychological health,developed from a needs
social relationships, andbased perspective.
environment. It includes
The average of
a 19 item Likert scaled index including questions on four domains:
each domain is taken, giving a profile of four separate domain scores. Control, Autonomy, Self-
realisation and Pleasure.
CASP-1971
European
Quality Social
of life indexSurvey
72
for older people, developed from a needs based perspective. It includes
In19
a theitem
module
Likertonscaled
well-being,
index the questionnaire
including includes
questions on fouradomains:
wide range of questions
Control, on Self-
Autonomy,
feelings and functioning.
realisation and Pleasure.
Some standardised
European measures
Social Survey 72 available for free on the Research in Practice (RIP)
website 73
In the module on well-being, the questionnaire includes a wide range of questions on
feelings and functioning.
This website is well recommended by public, private and voluntary social services agencies
in the UK
Some who pay a subscription
standardised measures to available
access some
foroffree
the on
material on their site
the Research in and to attend
Practice (RIP)
training
website events.
73 The following scales, however, can be downloaded free from their website74.
Some of these scales were developed by the English or the Welsh Departments of Health
for
Thisuse by social
website workers.
is well recommended by public, private and voluntary social services agencies
in the UK who pay a subscription to access some of the material on their site and to attend
Alcohol Use Disorders
training events. Identification
The following Test (AUDIT)
scales, however,
75
can be downloaded free from their website74.
The alcohol
Some usescales
of these disorders
wereidentification
developed by test
the(AUDIT)
Englishisorathe
widely used
Welsh measure developed
Departments of Health by
the World Health Organisation
for use by social workers. to help identify people who may be drinking in a manner that
is potentially harmful to their health including hazardous or risk alcohol use, harmful alcohol
Alcohol Use Disorders Identification Test (AUDIT)75
The alcohol use disorders identification test (AUDIT) is a widely used measure developed by
the
69
WorldofHealth
University Organisation
Pennsylvania, to helpCenter:
Positive Psychology identify people
Authentic who programme.
happiness may be drinking in a manner that
Available at:
http://www.authentichappiness.sas.upenn.edu
isWorld
70 potentially harmful to their health including hazardous or risk alcohol
Health Organization: WHO quality of life-BREF (WHOQOL-BREF). Available at: use, harmful alcohol
http://www.who.int/substance_abuse/research_tools/whoqolbref/en/
71
Hyde, M.; Wiggins, R.; Higgs P.; Blane, D. (2003): A measure of quality of life in early old age: the theory, development and
properties of a needs satisfaction model (CASP-19), Aging Ment Health, Vol. 7:3, pp. 186-94. Available at:
72 University of Pennsylvania, Positive Psychology Center: Authentic happiness programme. Available at:
69
University of Cambridge: Wellbeing institute. Available at: http://www.cambridgewellbeing.org/Files/Well-being-
http://www.authentichappiness.sas.upenn.edu
Module_Jun06.pdf
73 World Health Organization: WHO quality of life-BREF (WHOQOL-BREF). Available at:
70
Research in Practice: Standardised measures and guidance. Available at: https://www.rip.org.uk/events-and-online-
http://www.who.int/substance_abuse/research_tools/whoqolbref/en/
learning/change-projects/change-projects-resources/smg-change-project/
74 Hyde, M.; Wiggins, R.; Higgs P.; Blane, D. (2003): A measure of quality of life in early old age: the theory, development and
71
Research
properties ofina Practice: Resources.
needs satisfaction Available
model at: https://www.rip.org.uk/resources/
(CASP-19), Aging Ment Health, Vol. 7:3, pp. 186-94. Available at:
75
Research in Practice: Alcohol use disorders identification test. Available at:
https://www.rip.org.uk/download/254/SMG_change_projectAudit_C.pdf
72
University of Cambridge: Wellbeing institute. Available at: http://www.cambridgewellbeing.org/Files/Well-being-
Module_Jun06.pdf
73
Research in Practice: Standardised measures and guidance. Available at: https://www.rip.org.uk/events-and-online- 54
learning/change-projects/change-projects-resources/smg-change-project/
74
Research in Practice: Resources. Available at: https://www.rip.org.uk/resources/
75
Research in Practice: Alcohol use disorders identification test. Available at:
https://www.rip.org.uk/download/254/SMG_change_projectAudit_C.pdf
54
use, alcohol abuse and alcohol dependence. The AUDIT-C is a pencil and paper, self-report
questionnaire that takes approximately five minutes to complete.
Depression, anxiety and stress scale (DASS)76

The DASS presented here is a short version, the DASS21, which has seven items per scale
use, aalcohol
and total ofabuse and alcohol dependence. The AUDIT-C is a pencil and paper, self-report
21 questions.
questionnaire that takes approximately five minutes to complete.
There are three scales:
Depression, anxiety and stress scale (DASS)76
The1.DASSThepresented
depressionhere is a
scale short version,
assesses the DASS21,
dysphoria, which has
hopelessness, seven items
devaluation per
of life, scale
self-
and a total of 21 questions.
deprecation, lack of interest/involvement, anhedonia, and inertia.
2. The anxiety scale assesses autonomic arousal, skeletal muscle effects, situational
There are three scales:
anxiety, and subjective experience of anxious affect.
3. The stress scale is sensitive to levels of chronic non-specific arousal. It assesses
1. The depression scale assesses dysphoria, hopelessness, devaluation of life, self-
difficulty relaxing, nervous arousal, and being easily upset/agitated, irritable/over-
deprecation, lack of interest/involvement, anhedonia, and inertia.
reactive and impatient.
2. The anxiety scale assesses autonomic arousal, skeletal muscle effects, situational
anxiety, and subjective experience of anxious affect.
The DASS is a pencil and paper, self-report questionnaire that takes approximately five
3. The stress scale is sensitive to levels of chronic non-specific arousal. It assesses
minutes to complete. Participants are asked to use a four point severity/frequency scale to
difficulty relaxing, nervous arousal, and being easily upset/agitated, irritable/over-
rate the extent to which they have experienced each state over the past week.
reactive and impatient.
Emotional Regulation Questionnaire77

The DASS is a pencil and paper, self-report questionnaire that takes approximately five
This is a 10-item scale designed to measure respondents’ tendency to regulate their
minutes to complete. Participants are asked to use a four point severity/frequency scale to
emotions in two ways:
rate the extent to which they have experienced each state over the past week.
1. Cognitive Reappraisal
Emotional 2. Regulation Questionnaire
Expressive Suppression
77
This is a 10-item scale designed to measure respondents’ tendency to regulate their

emotions
The ERQ in is two ways:
a pencil and paper, self-report questionnaire. Respondents answer each item on
a seven-point Likert-type scale ranging from 1 (strongly disagree) to 7 (strongly agree). The
1. Cognitive Reappraisal
higher the scores the greater the use of the emotion regulation strategy.
2. Expressive Suppression
Home conditions assessment78

The ERQ is a pencil and paper, self-report questionnaire. Respondents answer each item on
A tool to assess physical aspects of the home environment. This scale may appear
a seven-point Likert-type scale ranging from 1 (strongly disagree) to 7 (strongly agree). The
judgmental, but workers necessarily make judgements about the safety, order and
higher the scores the greater the use of the emotion regulation strategy.
cleanliness of the place in which the child lives. The use of a list helps the objectivity of
observation. The scale comprises a list of 11 items to be observed during home visits, and
Home conditions assessment78
social presentation, the cleanliness of the children is included. The scale is best used as a
A tool to assess physical aspects of the home environment. This scale may appear
mental checklist to provide a framework for observation.
judgmental, but workers necessarily make judgements about the safety, order and
cleanliness of the place in which the child lives. The use of a list helps the objectivity of
The Home Conditions Assessment was made available as part of The Family Pack of
observation. The scale comprises a list of 11 items to be observed during home visits, and
Questionnaires and Scales from the Department of Health in the UK.
social presentation, the cleanliness of the children is included. The scale is best used as a
mental checklist to provide a framework for observation.
The Home Conditions Assessment was made available as part of The Family Pack of
76
Psychology Foundation of Australia: Depression anxiety stress scales (DASS). Available at:
Questionnaires and Scales from the Department of Health in the UK.
http://www2.psy.unsw.edu.au/dass/
77
University of Berkeley: Emotion regulation questionnaire (ERQ). Available at:
https://www.ocf.berkeley.edu/~johnlab/pdfs/ERQ.pdf
78
UK Department of Health: Home conditions assessment. Available at: http://lrsb.org.uk/uploads/home-conditions-
assessment.pdf
76
Psychology Foundation of Australia: Depression anxiety stress scales (DASS). Available at: 55
http://www2.psy.unsw.edu.au/dass/
77
University of Berkeley: Emotion regulation questionnaire (ERQ). Available at:
https://www.ocf.berkeley.edu/~johnlab/pdfs/ERQ.pdf
78
UK Department of Health: Home conditions assessment. Available at: http://lrsb.org.uk/uploads/home-conditions-
assessment.pdf
55
Maternal and paternal ante-natal attachment scales (MAAS and PAAS)79
The MAAS and PAAS are used to assess maternal/paternal antenatal bonding to the unborn
baby. The MAAS scale includes 19 items and the PAAS 16 items which focus on feelings,
attitudes and behaviours towards the foetus with responses recorded on a five-point Likert
Scale. Many of the questions require the respondent to select their answer based on their
Maternal and paternal ante-natal attachment scales (MAAS and PAAS)79
experience in the previous two weeks.
The MAAS and PAAS are used to assess maternal/paternal antenatal bonding to the unborn
baby. The MAAS scale includes 19 items and the PAAS 16 items which focus on feelings,
Multidimensional scale of perceived social support80
attitudes and behaviours towards the foetus with responses recorded on a five-point Likert
The Multidimensional Scale of Perceived Social Support is a measure of how much support
Scale. Many of the questions require the respondent to select their answer based on their
a parent feels they get from family, friends and significant others. Each of these forms a
experience in the previous two weeks.
separate subscale. A total score can also be calculated.
Multidimensional scale of perceived social support80
Parenting daily hassles81
The Multidimensional Scale of Perceived Social Support is a measure of how much support
This measure aims to assess the frequency and intensity/impact of 20 experiences that can
a parent feels they get from family, friends and significant others. Each of these forms a
be a ‘hassle’ to parents. It has been used in a wide variety of research concerned with
separate subscale. A total score can also be calculated.
children and families. The research suggests that parents/caregivers enjoy completing the
scale, because it touches81on aspects of being a parent that are very familiar and it is seen as
Parenting daily hassles
helping them express what it feels like to be a parent.
This measure aims to assess the frequency and intensity/impact of 20 experiences that can
be a parent/caregiver
The ‘hassle’ to parents. It has to
is asked been
scoreused in aofwide
each variety
the 20 of research
potential hasslesconcerned with ways.
in two different
children
1. Theandfrequency
families. The research
of each type ofsuggests
happening thatprovides
parents/caregivers enjoy
an ‘objective’ completing
marker of how the
often
scale, because
it occurs.it touches on aspects of being a parent that are very familiar and it is seen as
helping themintensity
2. The expressorwhat it feels
impact like
score to be a parent.
indicates the caregiver’s ‘subjective’ appraisal of how
much those events affect or ‘hassle’ them.
The parent/caregiver is asked to score each of the 20 potential hassles in two different ways.
The frequency
The1.Parenting of each
daily hassles type
tool ofmade
was happening provides
available anof‘objective’
as part The Familymarker
Packofofhow often
it occurs.
Questionnaires and Scales from the Department of Health.
2. The intensity or impact score indicates the caregiver’s ‘subjective’ appraisal of how
much those events affect or ‘hassle’ them.
Conclusion
The
MostParenting daily hassles
of the standardised tool waslisted
measures made inavailable as partare
this document of very
The easy
Family
toPack of can give
use and
Questionnaires and Scales from the Department of Health.
practitioners a solid measure as a baseline at the start of a project. They can then be
reapplied at the end of the project to monitor changes.
Conclusion
The main challenge in using these scales is that few of them are published or validated in
Most
other of the standardised
European languagesmeasures listedHowever,
than English. in this document
a simpleare
butvery easy to usemethod
well-accepted and can
of give
practitioners
using them inaanother
solid measure
languageas is
a baseline at the start
first to translate themofinto
a project. They can
the language then be
required and then
reapplied at the end of the project to monitor changes.
ask another translator to translate them back into English. The translated version is then
tested with a pilot group of those with whom the tool is to be used.
The main challenge in using these scales is that few of them are published or validated in
other European languages than English. However, a simple but well-accepted method of
using them in another language is first to translate them into the language required and then
ask another translator to translate them back into English. The translated version is then
tested with a pilot group of those with whom the tool is to be used.
79
Research in Practice: Maternal and paternal ante-natal attachment scales. Available at:
https://www.rip.org.uk/download/263/SMG_change_projectMaternal_antenatal_attachment_scale.pdf
https://www.rip.org.uk/download/264/SMG_change_projectPaternal_antenatal_attachment_scale.pdf
80
University of York: Multidimensional scale of perceived social support. Available at:
http://www.yorku.ca/rokada/psyctest/socsupp.pdf
81
UK Department of Health: Parenting Daily Hassles SCALE. Available at:
https://www.cafcass.gov.uk/media/215160/parenting_daily_hassles_scale.pdf
79
Research in Practice: Maternal and paternal ante-natal attachment scales. Available at:
https://www.rip.org.uk/download/263/SMG_change_projectMaternal_antenatal_attachment_scale.pdf 56
https://www.rip.org.uk/download/264/SMG_change_projectPaternal_antenatal_attachment_scale.pdf
80
University of York: Multidimensional scale of perceived social support. Available at:
http://www.yorku.ca/rokada/psyctest/socsupp.pdf
81
UK Department of Health: Parenting Daily Hassles SCALE. Available at:
https://www.cafcass.gov.uk/media/215160/parenting_daily_hassles_scale.pdf
56
Bibliography
http://www.esn- Journal of Clinical Child
eu.org/userfiles/Documents/20 Psychology and Psychiatry,
Agence nationale de 15/2015_Public_Social_Service Vol. 1, pp. 1-14. Available at:
l'évaluation et de la qualité des s_in_Crisis_report_-_FINAL.pdf http://67.199.123.90/library/wp-
établissements et services paper.asp?nMode=1&nLibraryI
sociaux et médico-sociaux: European Social Network D=723
Recommandations de bonnes (ESN) (2014): Contemporary
pratiques professionnelles. issues in the public Goodman R.; Meltzer, H.; and
Available at: management of social services Bailey, V. (1998): The strengths
http://www.anesm.sante.gouv.fr in Europe. Innovation, research and difficulties questionnaire: a
/spip.php?page=rubrique&id_ru and evidence-based practice. pilot study on the validity of the
brique=10 Available at: http://www.esn- self-report version, European
eu.org/raw.php?page=files&id= Child and Adolescent
Ashford, N.; Elliott, D.; and 970 Psychiatry, Vol. 7:3, pp. 125-
Little, M. (2012): Blueprints for 130. Available at:
Europe: promoting evidence European Social Network http://link.springer.com/article/1
based programmes in (ESN) (2015): Evidence in 0.1007%2Fs007870050057#pa
children’s services, public social services. An ge-1
Psychosocial interventions, Vol. overview from practice and
21:2, pp. 205-214. Available applied research. Available at: Goodman, R. (1997): The
at:http://www.redalyc.org/pdf/17 http://www.esn- strengths and difficulties
98/179824384009.pdf eu.org/userfiles/Documents/Pu questionnaire: a research note,
blications/Practice_reports/Evid Journal of Child Psychology
Buchanan, A. (2000): Present ence_in_public_social_services and Psychiatry, Vol. 38:5, pp.
issues and concerns; in: .pdf 581–586. Available at:
Buchanan, A.; and Hudson, B.: http://onlinelibrary.wiley.com/do
Promoting Children’s Emotional Gambrill, E. (2006): Evidence i/10.1111/j.1469-
Well-being. based Social Work. Paper to 7610.1997.tb01545.x/abstract
the Faculty of Social Science,
European Commission (2015): University of Oxford; in Graham, I. D., Logan, J.;
Europe 2020 in a nutshell. Buchanan, A.: Evidence based Harrison; M. B.; Straus, S. E.;
Available at: Social Policy and Practice. A Tetroe J.; Caswell, W.; and
http://ec.europa.eu/europe2020 new ideology or a human rights Robinson, N. with the Alliance
/europe-2020-in-a- imperative?, pp. 328-340. for Continuing Education in the
nutshell/index_en.htm Health Professions, the Society
Gardner, F.; Montgomery, P.; for Academic Continuing
European Commission, DG and Knerr, W. (2015): Medical Education, and the
Employment Social Affairs and Transporting evidence-based Association for Hospital
Inclusion: Publications and parenting programs for child Medical Education (2006): Lost
documents. Available at: problem behavior (Age 3-10) in knowledge translation: Time
http://ec.europa.eu/social/keyD between countries: systemic for a map?, Journal of
ocuments.jsp?advSearchKey= review and meta-analysis, Continuing Education in the
elder+care&mode=advancedSu Journal of Clinical Child Health Professions, Vol. 26:1,
bmit&langId=en&policyArea=&t Psychology and Psychiatry, pp. 13-24. Available at:
ype=0&country=0&year=0 Vol. 1, pp. 1-14. Available at: http://www.ncbi.nlm.nih.gov/pu
http://67.199.123.90/library/wp- bmed/16557505
European Platform for investing paper.asp?nMode=1&nLibraryI

in Children (EPIC). Available at: D=723 High Scope Educational
http://europa.eu/epic/ Research Foundation (2005):
Gardner, F.; Montgomery, P.; Lifetime Effects. The high
and Knerr, W. (2015): scope perry preschool study
European Social Network
Transporting evidence-based through age 40. Available at:
(ESN) (2015): Public social
parenting programs for child http://www.highscope.org/conte
services in crisis: challenges
problem behavior (Age 3-10) nt.asp?contentid=219
and responses. Available at:
between countries: systemic
review and meta-analysis,
57

Hyde, M.; Wiggins, R.; Higgs Government: Indices of
P.; Blane, D. (2003): A Deprivation. Available at:
measure of quality of life in https://www.gov.uk/government
early old age: the theory, /collections/english-indices-of-
development and properties of deprivation and
a needs satisfaction model http://neighbourhood.statistics.
(CASP-19), Aging Ment Health, gov.uk/HTMLDocs/AtlasOfDepr
Vol. 7:3, pp. 186-94. Available ivation2010/index.html
at:
http://www.ncbi.nlm.nih.gov/pu Webster Stratton, C.; and
bmed/12775399 Taylor, T. (1998): Adopting and
implementing empirically
J-Pal Europe (2012): Social supported interventions. A
experimentation. A Recipe for success; in:
methodological guide for policy Buchanan, A.; and B. Hudson:
makers. Available at: Parenting, Schooling and
http://ec.europa.eu/social/Blob Children’s behaviour, pp. 127-
Servlet?docId=7102&langId=en 160.
McCord, J. (2007, originally Wessely, S. and Deahl, M

published in 1978): A thirty- (2003): Psychological
year follow-up of treatment debriefing is a waste of time; in:
effects, crime and family: American Psychologist, Vol.
selected essays of Joan 183:1, pp. 12-14. Available at:
McCord. Available at: http://bjp.rcpsych.org/content/1
http://books.google.com/books 83/1/12
?id=Upu7qxbvP6IC&pg=PA13#
v=onepage&q&f=false;
Nielsen, S.; Grünberger, P; and

Pedersen, C. (2013): What
types of evidence are needed
for decision-makers?
Knowledge management to
support evidence-based social
policy and programs in
Denmark, Paper presented at
the American Evaluation
Association Annual
Conference.
Olds, D.; Holmberg, J.;

Donelan-McCall, N.; Luckey,
D., Knudtson, M.; and
Robinson, J. (2014): Effects of
home visits by
paraprofessionals and by
nurses on children: follow-up of
a randomized trial at ages 6
and 9 years, JAMA Pediatr.;
Vol. 168:2, pp. 114-21.
Available at:
http://www.ncbi.nlm.nih.gov/pm
c/articles/PMC4217160/
UK Government, Department
for Communities and Local
58

European Social Surveys. Project Oracle. Available at:
Available at: http://project-oracle.com/
Useful links http://www.europeansocialsurv
ey.org/about/project_specificati Samorzad Teritorialny
Federal Institute for on.html Gwarancja Dobrych Uslug.
Construction, Urban and Area Available at:
Research. Available at: German Centre for Gerology. http://www.dobrepraktyki.pl/ind
http://www.inkar.de Available at: ex.php?p1=11
http://www.dza.de/en/fdz.html.
Campbell Collaboration. Social Work Policy Institute.
Available at: German Federal Ministry of Available at:
http://www.campbellcollaboratio Health, future workshop on http://www.socialworkpolicy.org
n.org/ dementia. Available at: /research/evidence-based-
http://www.bmg.bund.de/theme practice-2.html
CareLit. Available at: n/pflege/demenz/zukunftswerks
http://www.carelit.de/cont/start/i tatt-demenz/uebersicht- Substance Abuse and Mental
ndex.php projekte.html Health Services Administration:
National Registry of Evidence-
Centre for Evidence Based German Institute of Applied Based Programs and Practices.
Mental Health. Available at: Nursing Research. Available at: Available at:
http://www.psych.ox.ac.uk/publi http://www.dip.de/datenbank- http://www.nrepp.samhsa.gov/
cations wise/informationen-zu- Search.aspx
wise/ueber-wise/) and
https://captus.samhsa.gov/prev
Centre for Quality in Care.
Gesis/SOWIPORT. Available ention-practice/prevention-and-
Available at:
at: behavioral-health/key-features-
http://www.zqp.de/index.php?p
http://www.gesis.org/en/institut risk-protective-factors /5)
n=project
e/the-association/mission/
The Incredible Years. Available
Danish Appeals Board.
Italian Social Assistants at: http://incredibleyears.com/
Available at: https://ast.dk/tal-
og-undersogelser Association. Available at:
http://www.assistentisociali.org/ Triple P International. Available
servizio_sociale/buone_prassi_ at: http://www.triplep-
Danish Centre for Social
del_servizio_sociale.htm parenting.uk.net/
Resarch. Available at:
http://www.sfi.dk/view_all_proje
cts- Mathilda Wrede Institute. UK Parliament. Available at:
4843.aspx?Action=1&NewsId= Available at: http://www.publications.parliam
4747&PID=10047#sthash.flIgO http://www.fskompetenscentret. ent.uk/pa/cm200910/cmselect/c
Pkt.dpuf fi/mathilda_wrede_institutet/in_ mchilsch/130/13006.htm
english
Early Intervention Foundation. University of Colorado Boulder,
Available at: MOVISIE. Available at: Center for the Study and
http://www.eif.org.uk/ http://www.movisie.nl/ Prevention of Violence.
Available at:
National Institute on Drug http://www.blueprintsprograms.
EPPI centre. Available at:
Abuse. Available at: com/
http://eppi.ioe.ac.uk/cms/
http://www.drugabuse.gov/publi
cations/preventing-drug-abuse-
EUROSTAT. Available at:
among-children-
http://ec.europa.eu/eurostat/we
adolescents/chapter-1-risk-
b/main/home.
factors-protective-factors
59

The content of this report and its toolkit
consists of a guidance on what type
of knowledge is required for planning
and evaluating services accompanied
by two questionnaires; the first aids
practitioners when planning services,
while the second questionnaire aids
practitioners in their evaluation of local
social services. The toolkit also includes
a review of international evidence-based
social work databases that may be useful
for practitioners when planning services.
The European Social Network (ESN)

has been working on aspects related to
evidence-based policy and practice in
public social services since 2012. This
toolkit was tested with senior practitioners
and applied researchers at a joint meeting
on evidence-based practice in London
in October 2015 and their feedback
was integrated into this final report.
European Social Network

Victoria House
125 Queens Road
Brighton BN1 3WB
United Kingdom
Tel +44 (0) 1273 739 039

Fax +44 (0) 1273 739 239
Email info@esn-eu.org
Web www.esn-eu.org

ESN Evidence Based Social Services ESN

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ESN Evidence Based Social Services ESN

Uploaded by

Copyright:

Available Formats

The Social Services

network for Europe

4 Evidence-based social services

Evidence-based social services 5

With thanks to participants at the Evidence panel meetings.

Steffen Bohni The National Board of Social Services Denmark

Minna Kivipelto National Institute for Health and Welfare Finland

Martijn Bool Movisie Netherlands

Piotr Nowak Mazovian Social Policy Center Poland

Amanda Edwards Social Care Institute for Excellence United Kingdom

Jenny Billings University of Kent United Kingdom

Stephanie Petrie University of Liverpool United Kingdom

Johanna O'Shea Ulster University United Kingdom

Ann Vanden OCMW Gent Belgium

Steinar Eggen Municipality of Randers Denmark

Vibeke Normann KORA, Danish Institute for Local and Denmark

6 Evidence-based social services

Minna Kivipelto National Institute for Health and Welfare Finland

Erkki Penttinen City of Vassa Finland

Emmanuel Social Affairs Department, Haute Saone France

Kristin Municipalities of Sandgerði, Garður and Iceland

Milou Dutch Association of Directors of Social Netherlands

Chris Vast KING (Quality Institute of Dutch Netherlands

Mirosław Janusz Korczak Pedagogical University Poland

Jorge Carvalho Lisbon City Council - Local Housing Program Portugal

Lola Casal Spanish General Council of Social Work Spain

Ana-Isabel Lima- Spanish General Council of Social Work Spain

Kontxi Odriozola Barcelona Provincial Council, Area of Spain

Anna Ricart Barcelona Provincial Council, Area of Spain

Madalen SIIS Centre of Documentation and Studies Spain

Ledicia Suárez Department of Social Inclusion of the Spain

Johan Glad National board of health and welfare Sweden

Graham Owen Social Services Department, Municipality of Sweden

Anna Lilja Swedish Association of Local Authorities and Sweden

Christina Practice, Development, Research, Future Sweden

Evidence-based social services 7

Jonathan Nesta Innovation Agency United Kingdom

Ann Buchanan Department of Social Policy and Intervention, United Kingdom

Felicity Elvidge Kent County Council United Kingdom

Chrissie Gale CELSIS United Kingdom

Barbara Janta RAND Europe United Kingdom

Vivien Moffat Institute for Research and Innovation in United Kingdom

Georgie Perry- Project Oracle United Kingdom

Anne Tavendale Skills for Care United Kingdom

Andrea Wigfield Care-Connect: the Social Care Innovation United Kingdom

For referencing, please use:

8 Evidence-based social services

approaches need to be tested so that policy-making becomes evidence-based and effective

Evidence-based social services 9

of Joan McCord. Available at: http://books.google.com/books?id=Upu7qxbvP6IC&pg=PA13#v=onepage&q&f=false; Wessely,

10 Evidence-based social services

Evidence-based social services 12

Introducing the Knowledge to Action Cycle

Figure 1: The Knowledge-To-Action Cycle 15

Identifying and assessing various types of knowledge

As highlighted in the beginning of this section, there is a range of “evidence” or “knowledge”

1. Evidence about the target population

What is the extent of the problem?

as close as possible to the communities most in need. 15

Figure 3: Atlas of the Indices of Deprivation 2010 for England

Substance Abuse Behaviour

Drug Availability School Anti-drug Use Policies

Drug Availability School Anti-drug Use Policies

Age Groups: Teenagers (age 13 to 19)