ARTICLE IN PRESS

Evidence-Based Healthcare & Public Health (2005) 9, 308314

www.elsevier.com/locate/ebhph

CONCEPTS, SKILLS AND RESOURCES

Evidence-based guidelines: The theory and

the practice
Deborah Swinglehurst
GP and Teaching Fellow, Department of Primary Care and Population Sciences,
University College London, UK

Health professionals are inundated with guidelines

local guidelines, national guidelines, clinical
guidelines, organisational guidelines, ethical guidelines the list goes on. The National electronic
Library for Healths Guidelines Finder website links
to no less than 1200 national guidelines from the UK
alone.1 Restricting a search to a defined clinical
area may still yield more than ten guidelines, and it
is not unusual for some of these guidelines to
extend to 50 or more pages of printed text.
The worldwide profusion of guidelines comes at a
time when evidence-based medicine is a key driver
of health policy, at least partly as a result of wide
variation in clinical practice, and the presumption
that some of this variation stems from inappropriate care.2 Policy makers are keen to harness the
potential of guidelines: to increase implementation
of the best scientific evidence, reduce inappropriate variation in practice, improve efficiency and,
ultimately, improve health outcomes for patients.

What is a guideline?
Clinical practice guidelines are defined as systematically developed statements to assist practitioner
and patient decisions about appropriate healthcare
for specific clinical circumstances.3
It is useful to make a distinction between guidelines and protocols as sometimes these terms are
used interchangeably (and erroneously). Protocols
are instructions on what to do in particular

circumstances. They are similar to guidelines but

include less room for individual judgement, are
often produced for less experienced staff, or for use
in situations where eventualities are predictable.
Good quality guidelines can benefit patients,
health professionals and healthcare systems, as
shown in Box 1.2
Unfortunately not all guidelines are systematically developed statements. The term systematic
can refer to both the content of good guidelines
(that is, they are based on a systematic review of
the available literature) and the process of guideline development. Surveys have shown that many
published guidelines are of poor quality, and lack a
rigorous methodological approach to their development.4
The concept of guidelines being of assistance to
patients (as well as practitioners) decisions,
about appropriate healthcare is often overlooked.5
But this definition reminds us that just as the
practice of evidence-based medicine involves the
integration of best evidence with clinical expertise
and attention to a patients choices and preferences,6 so patients should be actively involved in
both the development and application of clinical
practice guidelines.
Because guidelines are developed for use in
specific clinical circumstances, their usefulness
depends on their judicious application in professional practice. Slavishly following inflexible
recommendations and ignoring the unique characteristics of a patient encounter may well be

1744-2249/$ - see front matter & 2005 Elsevier Ltd. All rights reserved.
doi:10.1016/j.ehbc.2005.05.012

ARTICLE IN PRESS
Evidence-based guidelines: the theory and the practice

Box 1

309

Potential benefits of clinical practice guidelines (adapted from Woolf SH et al.2)

Potential benefits for

patients









Better quality of care

Improve health outcomes
Improve consistency of care
Inform patients about what clinicians should be doing
Empower patients to make more informed choices
Influence public policy
Promote distributive justice

Potential benefits for

healthcare professionals








Better quality of clinical decisions

Reassure clinicians that practice is appropriate
Provide explicit recommendations to guide care
Reduce outdated, ineffective or wasteful practice
Support quality improvement initiatives
Inform the research agenda by highlighting gaps in evidence

Potential benefits for

healthcare systems

 Improve efficiency
 Optimise value for money
 Demonstrating adherence to guidelines may improve public image

harmful. For example, guidelines recommend

warfarin for elderly patients with atrial fibrillation
(AF), who are at high risk of ischaemic stroke.
However, prescribing warfarin to an 80-year-old
man who satisfies these criteria, without taking
into account his previous numerous hospital admissions with confusion and falls, may well do more
harm than good.

What makes a guideline evidence-based?

At the heart of an evidence-based guideline is
usually a systematic review of original research
trials. A systematic review can be defined as a
review that has been prepared using a systematic
approach to minimise biases and random errors,
which is documented in a materials and methods
section.7 Regarded as the gold standard in clinical
evidence, a guideline which does not document
how the evidence was identified, evaluated,
synthesised and summarised must be viewed with
caution. The steps involved in conducting a
systematic review are shown in Box 2.
The Cochrane Collaboration8 and the NHS Centre
for Reviews and Dissemination9 are two examples
of organisations dedicated to preparing, disseminating and updating systematic reviews of healthcare interventions.

Box 2 Steps in conducting a systematic

review (adapted from Egger M and Davey
Smith G30)
1. Formulate review question
2. Define inclusion and exclusion criteria
a. Participants
b. Interventions and comparisons
c. Outcomes
d. Study design
3. Locate studies, using a comprehensive
search strategy
4. Select studies (more than one investigator
should check eligibility)
5. Assess study quality (the method used
should be explicit)
6. Extract data
7. Analyse and present results
8. Interpret results

Some systematic reviews include a meta-analysis. This is a statistical synthesis of the results of
the included studies, with the aim of producing a
single estimate of a treatment effect. Metaanalysis is only appropriate in certain circumstances and if carried out inappropriately can lead
to misleading, erroneous results. Readers keen to

ARTICLE IN PRESS
310
master the ability to critically appraise a systematic review and decide whether its results can be
applied to particular clinical circumstances are
directed elsewhere.10,11
Systematically reviewing the literature poses
several challenges for guideline developers. One
fundamental problem is that scientific evidence
about what to recommend is often lacking only a
small fraction of what is done in medicine has been
tested in appropriate, well-designed studies.2 Even
when well conducted systematic reviews already
exist, the leap from interpreting the results of a
systematic review to producing recommendations is
a complex one, which involves subjective value
judgements and an understanding of whether the
results obtained in the research setting might be
appropriate to extrapolate to everyday practice.
Producers of evidence-based guidelines should
make explicit links between the recommendations
contained in the guideline and the evidence that
has been used to inform the recommendations.
Important limitations identified in the retrieved
evidence (relating to the circumstances in which
a trial was conducted or the specific details of
the research participants or intervention, for
example) or absence of evidence should be made
explicit.
Although there are large databases of systematic
reviews and randomised controlled trials of therapy, it is much more difficult to find high quality
studies on diagnostic procedures, prognosis, and
aspects of healthcare whose study requires a
qualitative approach. Many published reports of
diagnostic tests suffer from poor methods and small
samples which make them prone to bias, and poor
quality of reporting which makes them difficult to
interpret.12
Guideline developers often refer to a hierarchy
of evidence and associated recommendations,
using numbers and letters to rank levels of
evidence and grades of recommendation. For
example, recommendations may be Grade A, if
based on evidence from meta-analysis of randomised controlled trials (Level 1a evidence) or
evidence from at least one randomised controlled
trial (Level 1b evidence). Unfortunately there has
been a proliferation of such hierarchies, with
different guideline developers using different
scales and assigning different weightings to different types of evidence. This causes confusion and
undermines the usefulness of such hierarchies.13
The same intervention might be assigned different
grades of recommendation in different international guidelines, thus failing to communicate a
message quickly and concisely in a way that is
universally understood.14

D. Swinglehurst
The GRADE Working Group has recently published
a system designed to overcome some of these
limitations, based on consideration of four key
elements: study design, study quality, consistency
(similarity of estimates of effect across studies) and
directness (the extent to which the people,
interventions and outcome measures are similar
to those of interest).15 These four components are
combined to make a judgement on the overall
quality of evidence, which is rated as high,
moderate, low or very low. Evidence is initially
categorised according to the study design (for
example RCT is high and an observational study
is low) but additional consideration of the quality,
consistency and directness results in shifts upwards
or downwards until the final evidence grading is
determined. The lowest quality of evidence for any
outcome that is critical to decision making then
provides the basis for the overall rating of the
evidence.15 Recommendations are then based on a
consideration of:






the trade-offs (between benefits and harms);

the quality of the evidence;
translation of the evidence into practice in a
specific setting; and
uncertainty about the baseline risk for the
population of interest.

The authors of the GRADE recommendations

suggest this system strikes a balance between
simplicity and clarity, although it is too early to
know whether the GRADE approach will be widely
adopted. Unfortunately, like previous grading systems, the contribution of qualitative research is
overlooked and it is difficult to see how even an
exemplary piece of qualitative research could be
assigned a high rating.
Although drug treatments can be tested in RCTs
and subjected to conventional processes of systematic review, such as those described in Cochrane Handbook,16 this is not necessarily true of
more complex health and social interventions.
Establishing the evidence base for such recommendations is a much more complicated affair. Complex
interventions are those which consist of several
interconnecting parts, where the active ingredients are difficult to specify, and the way in which
different parts of the intervention relate to each
other is hard to define the whole is more than the
sum of the parts. Examples might be the provision
of nurse-led smoking cessation education to groups
of adolescents, or the effectiveness of a rehabilitation service in a spinal injuries department. Such
interventions are highly sensitive to the context,
the agents and management structures involved,

ARTICLE IN PRESS
Evidence-based guidelines: the theory and the practice
and other social and organisational variables.
Producing systematic reviews of this type of
intervention is particularly challenging; the interested reader might like to look out some specialist
articles in this area.17,18

How are evidence-based guidelines

developed?
Attention to the rigour of the process of guideline
development is at least as important as the
evidence base itself. The aim is to produce simple,
patient-specific, user-friendly guidelines that bring
together the relevant, valid evidence needed by
clinicians to assist decision making, and to make
these available in a concise and accessible format.19
Shekelle and colleagues identify five steps in the
guideline development process20







identifying and refining the subject area of a

guideline
running guideline development groups
identifying and assessing the evidence
translating evidence into a clinical practice
guideline
reviewing and updating guidelines.

Guideline development is a costly process, and

the identification of the scope and subject area of a
guideline necessarily involves setting priorities and
weighing up the potential benefits to policy makers
and clinicians of having guidelines available in
competing clinical areas. The scope of the guideline needs to be clarified and this may be done by
consultation with clinicians, patients and potential
users or evaluators of a guideline.20 The clinical
questions and outcomes of importance must be
established, and a target group of guideline users

Box 3

311

should be identified. There are systematic methods

for doing this, for example the Delphi method.
The composition of a guideline development
group is important. Studies have shown that
conclusions reached by a multidisciplinary group
differ from those reached by a specialist group,
even when provided with the same evidence.21,22 It
is important that all relevant stakeholders are
included the likely professional users of the
guideline and also patients and carers whose lives
will be affected by them.5 Multidisciplinary groups
may include representation from clinical and nonclinical (e.g. service management) professionals,
primary and secondary care, patient or carer
participation groups, for example. Skilful leadership by a chairperson is needed to ensure effective
group facilitation. It is important that potential
conflicts of interest are identified and made
explicit guideline developers should be independent of the interests of government, industry
or any special interest groups which may detract
from the evidence and introduce bias into the
recommendations.5
Judgements involved in guideline development
range from prioritisation of guideline content to
interpretations of the evidence, to decisions
regarding generalisability of research findings and
feasibility of interventions in certain settings.
These might all be regarded as scientific judgements. Equally important, and less frequently
discussed within the body of a guideline, are the
value judgements that underpin the recommendations. Box 3 shows some examples.
The National Institute for Health and Clinical
Excellence (NICE) has just published draft guidelines on social value judgements as part of a
consultation exercise.23 The document explains
that social value judgements take account of the
ethical principles, preferences, culture and aspirations that should underpin the nature and extent of
care provided by the NHS, and focuses on the
social value judgements involved in developing

Examples of value judgements that underpin guideline development

 What are the guiding principles regarding how effective an intervention must be (compared to its
potential harms) before its recommendation is considered?
 What values underpin decisions about which guideline developments to prioritise?
 What is the ethical framework to which guideline developers are working in particular relating to
matters of distributive justice/rationing of limited resources?
 Where disagreement between guideline developers exists, what explicit processes are being used to
resolve such disagreements?

ARTICLE IN PRESS
312

D. Swinglehurst

conclusions about cost effectiveness, priority setting and rationing. Whilst the outcome of this
process and the details of the final guidelines
remain uncertain, aiming to define and make
transparent the value judgements that underpin
NICE guidelines should be applauded.
When evidence-based guidelines are prepared,
they should be subjected to external review in
order to check them for validity, clarity and
applicability.20 Shekelle suggests that external
reviews should include people with clinical expertise in the relevant area and experts in systematic
review or guideline development, as well as
potential users of the guideline.

Box 4

Despite well-described methods of guideline

development, the quality of many guidelines
produced internationally is variable. The AGREE
collaboration (Appraisal of Guidelines Research and
Evaluation) is an international collaboration of
researchers and policy makers who work together
to improve the quality and effectiveness of clinical
practice guidelines by establishing a shared framework for their development, reporting and assessment.24 One of its major contributions to this
field has been the development of the AGREE
instrument, a guideline appraisal instrument that
has been developed using rigorous methods and
validated internationally. This can be used by

The AGREE Instrument

Domain 1: Scope and purpose

1. The overall objective(s) of the guideline is (are) specifically described
2. The clinical question(s) covered by the guideline is (are) specifically described
3. The patients to whom the guideline is meant to apply are specifically described
Domain 2: Stakeholder involvement
1. The guideline development group includes individuals from all the relevant professional groups
2. The patients views and preferences have been sought
3. The target users of the guideline are clearly defined
4. The guideline has been piloted among end users
Domain 3: Rigour of development
1. Systematic methods were used to search for evidence
2. The criteria for selecting the evidence are clearly described
3. The methods used for formulating the recommendations are clearly described
4. The health benefits, side effects and risks have been considered in formulating the
recommendations
5. There is an explicit link between the recommendations and the supporting evidence
6. The guideline has been externally reviewed by experts prior to its publication
7. A procedure for updating the guideline is provided
Domain 4: Clarity and presentation
1. The recommendations are specific and unambiguous
2. The different options for management of the condition are clearly presented
3. Key recommendations are easily identifiable
4. The guideline is supported with tools for application
Domain 5: Applicability
1. The potential organisational barriers in applying the recommendations have been discussed
2. The potential cost implications of applying the recommendations have been considered
3. The guideline presents key review criteria for monitoring and/or audit purposes
Domain 6: Editorial independence
1. The guideline is editorially independent from the funding body
2. Conflicts of interest of guideline development members have been recorded

ARTICLE IN PRESS
Evidence-based guidelines: the theory and the practice
practitioners and policy makers to help them assess
the process of guideline development and the
quality of the reporting of this process (as distinct
from the quality of the evidence underpinning the
recommendations).25 Like other appraisal instruments, the AGREE instrument (which can be downloaded from the AGREE website26) uses a checklist
that covers 23 items across six different quality
domains. The developers of the instrument recommend that four individuals use the instrument to
appraise a guideline, as this has been shown to
increase the reliability of the domains and yield a
more robust assessment.25 Many international
bodies are now using the AGREE instrument in the
adoption of standards for guideline development.
A summary of the AGREE instrument is shown in
Box 4.

Conclusion
No discussion of guidelines would be complete
without at least some mention of their implementation. Given the volume and complexity of guidelines, it is perhaps not surprising that major
difficulties arise when introducing clinical guidelines into routine daily practice.27 The evidence
base to guide decisions about which strategy of
guideline dissemination and implementation to use
is poor.28 The journey from the publication of
scientific evidence to its incorporation into guidelines and then to implementation in the real world
of professional practice is a long and complex one
that continues to perplex researchers and policy
developers worldwide. The evidence that guidelines result in improved health outcomes for
patients is far from clear cut, although a quality
evidence-based guideline has the potential to
achieve this, if as much attention is paid to the
dissemination and implementation phase as to its
original development. The implementation of
guidelines involves a complex interplay of sociocultural and organisational factors which extend
way beyond the strength of the evidence conveyed
in the guideline itself. Indeed, the implementation
of guidelines is a very good example of a complex
healthcare intervention.
Evidence-based guidelines are only one dimension in the processes that go on in a clinicians mind
as decisions are made in day-to-day practice.
Ethnographic research has shown that primary care
practitioners (the target audience for many national guidelines) rarely access guidelines when
observed in routine practice, but that their
decisions are informed by what have been de-

313

scribed as mindlines collectively reinforced,

internalised, tacit guidelines, which are informed
by brief reading, their own and colleagues
experiences, their interactions with each other
and with opinion leaders, patients, pharmaceutical
representatives and other sources of largely tacit
knowledge.29 The authors of this research conclude
that practitioners have a collective professional
responsibility to ensure that mindlines are based on
research evidence wherever possible, and issue a
further challenge to those who disseminate research findings to use the knowledge sources that
practitioners actually use in practice, harnessing
the potential of networking and communities of
practice.
With this in mind, practitioners and policy
makers will be relieved to know that guidelines
are no replacement for clinical expertise or
complex clinical decision making processes. When
used wisely, however, they do have the potential to
benefit healthcare systems, healthcare professionals, and, most importantlyy patients.

Acknowledgements
I would like to thank Professor Trisha Greenhalgh
for providing comments on a first draft of this
article.

References
1. National electronic Library for Health. Guidelines Finder.
2005. http://libraries.nelh.nhs.uk/guidelinesFinder/
2. Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J.
Potential benefits, limitations, and harms of clinical guidelines. BMJ 1999;318:52730.
3. Clinical Practice Guidelines: directions for a new program.
Washington DC: National Academy Press, 1990
4. Grilli RG, Magrini N, Penna A, Mura G, Liberati A. Practice
guidelines developed by specialty societies: the need for a
critical appraisal. Lancet 2000;355:1036.
5. Keeley P. Clinical guidelines. Palliat Med 2003;17:36874.
6. Sackett DL, Rosenberg MC, Muir Gray JA, Haynes RB,
Richardson WS. Evidence based medicine: what it is and
what it isnt. BMJ 1996;312:712.
7. Systematic Reviews. London, UK: BMJ Publishing Group,
1995
8. The Cochrane Collaboration. The reliable source of evidence
in healthcare. http://www.cochrane.org/index0.htm
9. The Centre for Reviews and Dissemination, University of
York. http://www.york.ac.uk/inst/crd/
10. Greenhalgh T. How to read a paper: papers that summarise
other papers (systematic reviews and meta-analyses). BMJ
1997;315:6725.
11. Greenhalgh T, Donald A. Papers that summarise other papers
(systematic review and meta-analysis). In: Greenhalgh T,
Donald A, editors. Evidence Based Health Care Workbook.
London, UK: BMJ Publishing Group; 2000. p. 11129.

ARTICLE IN PRESS
314
12. Bandolier Extra. Evidence and Diagnostics. Bandolier.
http://www.jr2.ox.ac.uk/bandolier/Extraforbando/Diagnostic.
pdf
13. Upshur REG. Are all evidence-based practices alike? Problems in the ranking of evidence. CMAJ 2003;169:6723.
14. Schunemann HJ, Best D, Vist G, Oxman AD. Letters,
numbers, symbols and words: how to communicate grades
of evidence and recommendations. CMAJ 2003;169:67780.
15. GRADE Working Group. Grading quality of evidence and
strength of recommendations. BMJ 2004;328:14907.
16. The Cochrane Collaboration. The Cochrane Handbook for
Systematic Reviews of Interventions. http://www.cochrane.org/resources/handbook/
17. Pawson R, Greenhalgh T, Harvey G, Walshe, K. Realist
synthesis: an introduction. 4. ESRC Research Methods
Programme. http://www.ccsr.ac.uk/methods/publications/
documents/RMPmethods2.pdf
18. Medical Research Council. A framework for development
and evaluation of RCTs for complex interventions to improve health. London, UK: MRC; 2000 http://www.mrc.
ac.uk/prn/pdf-mrc_cpr.pdf.
19. Jackson R, Feder G. Guidelines for guidelines. BMJ 1998;
317:4278.
20. Shekelle PG, Woolf SH, Eccles M, Grimshaw J. Developing
guidelines. BMJ 1999;318:5936.
21. Herrin J, Etchason JA, Kahan JP, Brook RH, Ballard DJ. Effect
of panel composition on physician ratings of appropriateness
of abdominal aortic aneurysm surgery: elucidating differences between multidisciplinary panel results and specialty
recommendations. Health Policy 1997;42:6781.

D. Swinglehurst
22. Coulter I, Adams A, Shekelle P. Impact of varying panel
membership on ratings of appropriateness in consensus
panels: a comparison of a multi- and single disciplinary
panel. Health Serv Res 1995;30:57791.
23. National Institute for Health and Clinical Excellence. Social
Value Judgements: guidelines for the institute and its
advisory bodies (draft for consultation). 2005. http://
www.nice.org.uk/pdf/CC_SVJ_0405.pdf
24. AGREE collaboration. 2002. http://www.agreecollaboration.
org/
25. The AGREE Collaboration. Development and validation of an
international appraisal instrument for assessing the quality
of clinical practice guideline: the AGREE project. Qual Saf
Health Care 2003;12:1823.
26. The AGREE Collaboration. AGREE instrument. AGREE . 2005.
http://www.agreecollaboration.org/instrument/
27. Grol R, Grimshaw J. From best evidence to best practice:
effective implementation of change in patients care. The
Lancet 2003;362:122530.
28. Grimshaw J, Thomas RE, MacLennan G, Fraser C, Ramsay CR,
Vale L, et al. Effectiveness and efficiency of guideline
dissemination and implementation strategies. Health Technology Assessment 2004;8.
29. Gabbay J, le May A. Evidence based guidelines or collectively contructed mindlines? Ethnographic study of
knowledge management in primary care. BMJ 2004;
329:10137.
30. Egger M, Davey Smith G. Principles of and procedures for
systematic reviews. In: Egger M, Davey Smith G, Altman DG,
editors. Systematic Reviews in Health Care, pp 23-42. 2001.