Graduate Medical Education

What Are the Barriers to Residents Practicing

Evidence-Based Medicine? A Systematic
Review
Nynke van Dijk, MD, PhD, Lotty Hooft, PhD, and Margreet Wieringa-de Waard, MD, PhD

Abstract
Purpose
Insufficient time and lack of skills are
important barriers to the practice of
evidence-based medicine (EBM).
Residents could have additional barriers
because their practice can be strongly
influenced by the educational system and
clinical supervisors. The purpose of this
study, therefore, was to systematically
appraise and summarize the literature on
the barriers that residents experience in
the application of EBM in daily practice.
Method
The authors searched MEDLINE,
EMBASE, the Cochrane Library, CINAHL,
and ERIC for publications preceding
January 2008. Additionally, they
manually screened the abstracts of

Medical specialists, both practicing

physicians and those in training, are

expected to work according to the
principles of evidence-based medicine
(EBM). We follow the literature in
defining evidence-based practice as
decisions about health care that are
based on information regarding the
best available, current, valid, and
relevant evidence; the patients clinical
and physical circumstances; and the
patients preferences and likely
actions.1,2 These decisions should be
made by those receiving care, informed

Dr. van Dijk is assistant professor, Department of

General Practice/Family Medicine, Academic Medical
CenterUniversity of Amsterdam, Amsterdam, The
Netherlands.
Dr. Hooft is assistant professor, Dutch Cochrane
Center, Academic Medical CenterUniversity of
Amsterdam, Amsterdam, The Netherlands.
Dr. Wieringa-de Waard is professor, Department
of General Practice/Family Medicine, Academic
Medical CenterUniversity of Amsterdam,
Amsterdam, The Netherlands.
Correspondence should be addressed to Dr. van Dijk,
Department of General Practice, Academic Medical
Center, PO Box 22700, 1100 DE Amsterdam, The
Netherlands; telephone: (*31) 20-5668975; fax:
(*31) 20-6912683; e-mail: n.vandijk@amc.uva.nl.

Academic Medicine, Vol. 85, No. 7 / July 2010

relevant conferences (Association for

Medical Education in Europe, Society of
General Internal Medicine, Society of
Medical Decision Making, Ottawa, and
Evidence-Based Health Care Teachers &
Developers) from January 2001 until
January 2008. The search was extended
by contacting experts in the field.
Original studies on barriers to applying
EBM in daily practice were included.
Methodological quality was assessed and
results were extracted by two reviewers
using prespecified forms.
Results
The search resulted in 511 titles, 84
abstracts, and 3 studies suggested by
experts, of which 9 were included in this
review. The quality of the included

by the tacit and explicit knowledge of

those providing care, within the context
of the available resources.1 Yet various
studies showed that medical specialists
find it difficult to use new evidence in
their daily practice. Insufficient time is
the most important barrier mentioned
to implementing EBM.35 Other
limiting factors are the recognition of
questions, formulating search
questions, performing literature
searches, formulating answers to
questions from the available literature,
and translation of those answers to the
individual patient.3,6 Even with the
rising availability of aggregated
evidence in clinical practice guidelines
or evidence-based textbooks (e.g.,
Clinical Evidence, Pier, UpToDate),
implementing new evidence into
practice remains adifficult task.4
Changing a physicians existing ideas
and habits can make implementation
difficult as well.4
Residents may well have different
barriers to practicing EBM than
established physicians. Changing their
own practice might be easier because
their habits have not formed as strongly

studies was high. The most frequently

mentioned barriers for residents were
limited available time (28% 85%),
attitude, and knowledge and skills. In
four studies, specific barriers related to
the position of residents, such as
influences from staff members, lack of
experience in EBM, and low possibilities
to change conditions, were described.

Conclusions
Residents experience specific barriers to
practice EBM. These barriers should be
recognized and integrated into EBM
training programs for residents.
Acad Med. 2010; 85:11631170.

yet. However, despite the best evidence

and the preferences of both the resident
and the patient, the educational system
and preferences of the clinical
supervisors may also influence the
practice of the resident.
In this study, therefore, we aimed to
systematically review the literature on the
barriers residents experience in the
application of EBM in their daily
practice.
Method

Identification of studies
We performed a search for studies in
MEDLINE (PubMed), EMBASE
(Ovid), the Cochrane Library, CINAHL
(EBSCOhost), and ERIC (EBSCOhost)
from the earliest available date for each
database until the end of January 2008.
The search terms used are presented in
List 1. There was no language
restriction on the literature search. We
screened the reference lists of retrieved
studies for relevant publications.
Additionally, online available abstract
reports from the annual or biannual
meetings of the Society of Medical

1163

Graduate Medical Education

List 1
The Authors Search Terms for
Identifying and Including Studies of
Residents Implementation of
Evidence-Based Medicine in Their
Clinical Practice
MEDLINE (PubMed) (1950January
2008):
[Internship and Residency (MeSH) OR
Clinical Clerkship (MeSH) OR
resident (text word)]
AND
[Evidence-Based Medicine (MeSH) OR
evidence based medicine (text word) OR
ebm (tiab)]
EMBASE (Ovid) (1980January
2008):
[Residency Education OR
Resident OR Internship]
AND
[evidence based medicine.mp. OR
ebm.mp. OR Evidence Based Medicine]
CINAHL and ERIC (EBSCOhost) (until
January 2008):
[evidence based medicine OR
ebm OR
MM Medical Practice, Evidence-Based OR
MM Professional Practice, Evidence-Based
OR MM Medical Practice, ResearchBased]
AND
[resident OR
MM Interns and Residents OR
MM Internship and Residency] (without
expanders)
The Cochrane Library (until January
2008):
[Internship and Residency (MeSH, explode
all trees) OR
Clinical Clerkship (MeSH, explode all trees)
OR resident (tiab) OR
Residency NEXT Education OR
Internship]
AND
[Evidence-Based Medicine (MeSH, explode
all trees) OR
evidence NEXT based NEXT
medicine (tiab) OR
ebm]

Decision Making, Society of General

Internal Medicine, Association for
Medical Education in Europe,
Evidence-Based Health Care Teachers
& Developers, and Ottawa conferences
between January 2001 and January 2008
were manually searched for relevant
abstracts. To complete our search for
relevant studies, we contacted experts
in the field by e-mail, asking them if

1164

Figure 1 Flowchart showing the authors searches and selection of papers for inclusion in the
critical review of research on residents application of evidence-based medicine reported in this
article. Adapted from QUORUM (Quality or Reporting Meta-analysis) flowchart.

they were aware of any additional

studies for our review.
We used the following criteria to screen
studies for inclusion in our review:
1. Study design: Original studies, both
qualitative and quantitative
(observational and experimental),
focusing on the barriers that residents
experience in applying EBM in their
daily practice.
2. Population: Residents or those
working on a similar level (MDs at
postgraduate, but premedical
specialist level, working under the
supervision of a medical specialist).
3. Outcomes: Pertaining to barriers
toward the practice of EBM,
prevalence of different barriers toward
the practice of EBM, or perceived/
measured importance of factors
mentioned to be barriers toward the
practice of EBM.
Studies not describing residents, not
reporting on original studies, or not
related to the practice of EBM were
excluded.
Two of us (N.v.D. and M.W.d.W.)
independently reviewed the titles and

abstracts of the retrieved studies for

potentially eligible studies (Figure 1
describes this process). For each potentially
eligible study, the full paper was read by
two reviewers (N.v.D. and M.W.d.W.)
independently. They both assessed whether
the study fulfilled the inclusion criteria.
Studies about which both reviewers had
doubt about inclusion, and studies about
which the reviewers disagreed about
inclusion, were discussed in a consensus
meeting. In any case of persisting
disagreement, the third author (L.H.) was
consulted and a decision was made by
consensus of all authors.
Assessment of study quality
Included studies were assessed for
methodological quality independently by
two reviewers (N.v.D. and M.W.d.W.).
We assessed the quality of quantitative
studies using a modified version of the
recommendations described by the
Strengthening the Reporting of
Observational Studies in Epidemiology
initiative7,8 (List 2). Quality assessment of
qualitative studies was performed using
the modified criteria proposed by
Giacomini and Cook9 (List 2). Finally, we
assessed the quality of randomized
controlled trials (RCTs) according to the

Academic Medicine, Vol. 85, No. 7 / July 2010

Graduate Medical Education

Results

List 2
The Authors Criteria for Assessing the Quality of Studies Included in Their
Review
Qualitative studies9
Clear objectives
Proper selection of study participants
Participants relevant to research question (entire range/purposive sampling)
Selection of participants well reasoned
Adequate methods used to generate data
Appropriate method(s) for question
Appropriate setting
Clear and appropriate rationale for method
Methods clearly described
Comprehensive data collection
Number, type, duration, diversity, and depth of operations
Data auditable
Thorough classification
Authors self-conscious of personal experience and bias
Analysis of data and corroboration of findings
Clear theoretical basis for analysis
Iteration of data collection and analysis until theoretical saturation/informational redundancy
Triangulation (multiple sources) of key findings (interpretation by 1 researcher from
different disciplines; member checking; theory triangulation
Explicit search for conflicting arguments
Clear presentation of results
Display of convincing empirical material quotes
Quantitative study7,8
Specific objectives, including pre-specified hypotheses
Inclusion of participants
Appropriate eligibility and selection criteria of participants
Sound methods of follow-up
Definition of outcomes
All important possible outcomes included
Clear description of methods of measurement
Description of design and quality of instrument used
Low risk of collection bias
Limited time between event and measurement
Prospective or retrospective evaluation
Participants
Cohort: adequate proportion and duration of follow-up
Observational survey: number of responders
Reasons for not participating/loss to follow-up
Clear presentation of results
Numbers of outcomes/summary measures

methods recommended by the Cochrane

Collaboration.10 In studies in which it
was unclear whether the quality criteria
had been met, the reviewers contacted
the first author of the publication to
obtain additional information.
Data abstraction
The results of the included studies were
extracted by two reviewers (M.W.d.W.
and N.v.D.). A prespecified data

Academic Medicine, Vol. 85, No. 7 / July 2010

extraction form was used to extract and

collect information from the included
studies on (1) study design, (2)
characteristics of residents (specialty,
gender, age, number of responders), and
(3) outcomes as published by authors.
We resolved disagreements about data
extraction using a consensus meeting
with all reviewers. Because we expected to
find no homogeneous studies in this area,
no pooling of data was attempted.

Search results and selection of studies

for inclusion
After duplicate studies were removed,
the searches for articles resulted in 511
titles. From these, 37 were considered
potentially eligible for inclusion in the
study on the basis of title and abstract.
We obtained no additional studies by
screening the references of the retrieved
publications. Of the 37 potential studies,
8 were included. The other 29 were
excluded because they did not report
about residents (n 5), barriers (n
21), or both (n 3). The searches of the
meeting abstracts resulted in 84 potentially
relevant titles, of which 4 fulfilled the
inclusion criteria. Three of the included
abstracts1113 were already identified and
included as full papers,14 16 resulting in 1
new included study.17 None of the 3
studies suggested by the experts fulfilled
all inclusion criteria. The final review
therefore included 9 studies (8 full
articles and 1 abstract) evaluating the
attitudes of residents toward EBM and
the barriers toward practicing EBM.14 21
Of these studies, 5 were identified
through MEDLINE,15,19 22 and the others
were identified through MEDLINE and
CINAHL,14 MEDLINE and EMBASE,18
the Cochrane Library,16 and the abstracts
of scientific meetings.17 Agreement
between both reviewers on the inclusion
of studies was 92% after individual
review of the papers and 100% after
discussion. Agreement on data
abstraction was 100%. See Figure 1 for a
graphic representation of the selection
process.
Quality of included studies
Our criteria for the quality of the
individual studies are shown in Table 1.
Most studies we found were of high
quality. The qualitative studies14,15,18
scored high on all criteria, such as
proper selection of study participants,
appropriate methods, and comprehensive
data collection. The quantitative studies
varied in quality, with some unclear
aspects in the study (abstract only) of Ho
et al,17 but none had such poor quality as
to be excluded from the study.
Study characteristics
Of the nine included studies, four
described the results of quantitative
surveys,20 22 one described the results of
quantitative interviews,19 one described

1165

1166

Clear objectives

Montori, 200214

Bhandari, 200318

Analysis of data

Data collection

Quality rating*
Generation of
data

Presentation of results

Ho, 200617 (abstract)

Definition of
outcomes

Methods

Risk recollection
bias

Participants

Presentation of
results

Concealment of
allocation

Randomization

//

Blinding (participants/
teachers/outcome
assessors)

* The following symbols indicate how each criterion was rated: fulfilled all subcriteria; fulfilled most
subcriteria; fulfilled 50% of subcriteria; ? not reported.

Letelier,
200716

Randomized
controlled
trial

Amin, 200720

Complete
follow-up

Groups comparable
at start of study

Intention-to-treat
analysis

Similar
treatment
in both groups
(except
intervention)

......................................................................................................................................................................................................................................................................................................................................................................................................................

Allan, 200721

......................................................................................................................................................................................................................................................................................................................................................................................................................

......................................................................................................................................................................................................................................................................................................................................................................................................................

......................................................................................................................................................................................................................................................................................................................................................................................................................

Green, 2000

19

Participants

Objectives/
hypotheses

Burneo, 200622

Quantitative

Green, 2005

15

......................................................................................................................................................................................................................................................................................................................................................................................................................

......................................................................................................................................................................................................................................................................................................................................................................................................................

Qualitative

Selection of study
participants

Type of study

Quality of Studies Included in the Review of Research on Residents

Implementation of Evidence-Based Medicine in Their Clinical Practice, 2008

Table 1

Graduate Medical Education

the results of an RCT,16 and three

described the results of qualitative
studies. Of these, one used focus
groups,15 and two used a combination of
individual interviews and focus
groups.14,18 A more detailed description
of the nine studies is displayed in Table 2.
Most studies were performed in countries
where English is the primary language
(four in the United States, three in
Canada, one in Ireland, and one in
Chile). Most studies included between 12
and 97 residents from various medical
specialties as subjects. Only three
studies14,19,22 describe whether the
residents had received any training in
EBM before participating in the study.
Green and colleagues19 report that 52%
of the residents had participated in an
EBM curriculum for one or more years;
the neurology residents assessed by
Burneo et al22 participated in biweekly,
90-minute evidence-based clinical
practice sessions, and the internal
medicine residents from the Mayo Clinic
study participated in a clinical decisionmaking conference.14 The other studies
did not report on the presence or absence
of EBM training of the residents.1518,20
Findings

Barriers to implementing all steps of the

evidence-based decision-making process
were reported by various studies. The
main barriers reported were time,
attitude, knowledge and skills, and
resident-specific barriers.

Time. The most often mentioned and

primary barrier for residents was limited
available time. In all qualitative
studies,14,15,18 time was spontaneously
mentioned as a barrier. The neurology
residents responding to the survey of
Burneo and colleagues22 all responded
that time constraints were the main
reason why they do not use EBM all the
time. Three studies17,20,22 showed that
28% to 85% of the residents report time
as an important limiting factor for using
EBM concepts. In contrast, Allan et al21
found that time was not a substantial
barrier. Other studies15,18 suggest that
a lack of priority and competing
responsibilities, due to a lack of allocated
time, constitute a barrier.

The two central themes in the study of

Montori et al14 were time and expertise.
These two themes influenced the
decision-making style of the residents
and the information sources used. The

Academic Medicine, Vol. 85, No. 7 / July 2010

2000

Green19

Number of participants
64

Design

Follow-up on reasons for

not pursuing clinical
questions (n 280)
identified during clinic
sessions

Country
U.S.

Medical specialty
Internal medicine

No.
men/women
29/35

Level of study of
participants
25 first-year; 21 second-year

Academic Medicine, Vol. 85, No. 7 / July 2010

2002

Interviews and focus

group

17 (interviews); 6 (focus group)

Internal medicine

U.S.

First-year

Interviews: 14/3

2003

Focus group and

interviews

8 (focus group); 20 (interviews)

Surgery (all disciplines)

Canada

Focus group: 6/2;

interviews: 18/2

17/17

Focus group: 4 of 8 junior;

interviews: 10 of 20 junior

2005

Focus groups

34

Primary care internal

medicine

U.S.

10 of 34 interns

2006

2006

17

12

NR*

Quantitative survey

Quantitative survey
pre- and post intervention

Canada

U.S.

Neurology

Internal medicine

NR*

NR*

2007

2007

Amin20

92 (62 responders)

20 (19 responders)

Quantitative survey

Quantitative survey

Canada

Ireland

Family medicine

Higher surgical trainees

in otolaryngology

17/3

2007

* NR indicates not reported.

Letelier16

Randomized controlled
trial

97 (100% attendants)

All disciplines

Chile

First-year

NR*

......................................................................................................................................................................................................................................................................................................................................................................................................................

17 first- through fourth-years

......................................................................................................................................................................................................................................................................................................................................................................................................................

Allan21

......................................................................................................................................................................................................................................................................................................................................................................................................................

Ho

......................................................................................................................................................................................................................................................................................................................................................................................................................

Burneo22

......................................................................................................................................................................................................................................................................................................................................................................................................................

Green15

......................................................................................................................................................................................................................................................................................................................................................................................................................

Bhandari18

......................................................................................................................................................................................................................................................................................................................................................................................................................

Montori14

......................................................................................................................................................................................................................................................................................................................................................................................................................

Year

Author

Description of Studies Included in the Review of Research on Residents

Implementation of Evidence-Based Medicine in their Clinical Practice, 2008

Table 2

Graduate Medical Education

preferred style of the residents was to

discuss the evidence with the patients and
incorporate patient preferences into the
decision-making process. However, when
time was limited or residents felt insecure
about their expertise, they were more
likely to consult an expert as an
information source instead. In the first
study by Green et al,19 residents reported
identifying two new questions for every
three patients. Questions were not
answered in 81% of the cases. Lack of
time (60%) and forgetting the question
(29%) were the most frequently
mentioned reasons for not searching for
the answer.

Attitude. Both Bhandari et al18 and

Green and Ruff15 describe factors such as
personal initiative, lack of motivation,
and interest as barriers toward residents
use of EBM. In other studies, the attitude
of residents toward EBM is described as
moderately positive (53% of residents)20
or positive (70% of residents).21
However, Green and colleagues19
mention lack of interest in only a
minority of the cases (4%) as the reason
for not researching a clinical question
using the principles of EBM.

Knowledge and skills. In the qualitative

study by Green and Ruff15 residents
reported experiencing barriers on all
steps involved in the EBM process, such
as forgetting clinical questions, low
searching skills, and difficulty knowing
when to stop searching. Also reported by
Bhandari et al,18 a lack of education and a
lack of critical appraisal skills were
considered an important barrier. Ho and
colleagues17 found that only 20% of the
residents noted that the practice of EBM
was easy, although it was never reported
as being very easy. Allan et al21 described
lack of basic computer skills as a barrier,
especially for immigrant residents
(compared with U.S. and Canadian
residents).

Residency-related barriers. In four

studies, specific barriers related to the
position of residents were mentioned.
Green and Ruff15 found that the learning
climate influenced the residents
motivation for practicing EBM. Bhandari
et al,18 for surgical residents, identified
staff-surgeons-based barriers,
institutional and health care system
factors, and resident barriers (knowledge)
influencing the integration of EBM in
day-to-day routines. The main staff-

1167

Graduate Medical Education

surgeons-based barrier was staff

disapproval.18 Several residents reported
fear of repercussions from staff members
when confronting them with new
evidence, thereby indirectly telling them
their current practices are outdated.
Other staff factors were a lack of interest
and low motivation for implementing
EBM.18 However, Burneo et al22 reported
that only 1 of 12 residents reported the
disinterest of some attending physicians
as a reason for using EBM concepts only
sometimes.
Resident shortage, lack of funding of
health care, and inadequate information
resources were mentioned as examples of
institutional and health care system
factors.18 This may make institutional
factors or learning environment an
important barrier.15 Unfortunately, this
was not examined by most studies. On
the other hand, Green and colleagues19
found the perception of inadequate
resources to be the reason for not
pursuing clinical questions in only 2% of
the cases. In a later study, Green and
Ruff15 describe access to electronic
information resources and available
information technology as varying widely
between hospitals.
Other barriers. Only the findings of
Letelier et al16 focused on language as a
barrier toward the practice of EBM. That
study found that Chilean residents
reading a paper in English used longer
time (12.6 versus 11.8 minutes) and had
more difficulty interpreting (low score
34.7% versus 16.7%) a Cochrane abstract
than residents reading in Spanish.
Potential solutions. Three of the studies
suggested potential solutions to
overcome important perceived barriers.
Bhandari et al18 provide a long list of
strategies to improve the practice of
EBM, such as EBM training, preappraisal
of resources, and journal clubs. Residents
studied by Green and Ruff15 suggested
the use of handheld devices to overcome
barriers in access problems and time and
specially designed Web sites assisting
residents in their searches. Letelier et al16
suggest formal training in English
language and translation of papers as
solutions to overcome language barriers.
Discussion

Although many studies report on the

barriers of medical specialists toward the

1168

application of EBM, only a few studies

report on the specific barriers that
residents experience. Residents encounter
similar problems to practicing physicians
regarding time and searching skills, but
they also encounter the additional and
potentially negative influences of clinical
supervisors, the lack of role models, and
practical and institutional barriers, which
make the use of EBM even more difficult.
This in combination with less experience
in EBM and the high workload of
residents23 could reduce their motivation
to practice EBM to a minimum. Because
residents could be important motivators
for the propagation of EBM, educators
should be made aware that EBM is now
considered a principal component of
good clinical practice,1 and residentspecific barriers should be identified and
integrated into their EBM-training
programs.
Limitations
The few studies we found that focused on
the barriers residents experience to
applying EBM in daily practice were of
high quality and recent date. Although
the overall quality of the studies is high,
most quantitative studies did not fulfill
all subcriteria regarding the definition of
the outcomes. Most studies focused
on one or two requirements (i.e.,
knowledge) for EBM and not on the EBM
process until the application of the results
to the patient. To obtain an overall view
on the process of implementing EBM, a
measurement of the barriers in all five
steps of EBM would be required.
Another limitation of this review is the
location in which the studies took place.
All but two studies are of North
American origin. Differences in
educational and health care systems
between North America and other
regions might limit the generalizability of
the results.
All studies included in this review are
single-center studies. The culture (EBM
and attending supervision in general)
might vary significantly per center and
medical specialty. The resources for,
quality, and amount of available evidence
differ significantly per medical specialty,
and new evidence might be difficult to
implement. Learning a new surgical
technique requires more efforts than
prescribing another type of medication.
Also, the influence of supervisors might
differ per medical specialty. Some

surgeons dismiss clinical studies on the

basis of their perceived quality of the
study surgeons (judging that they have
poor skills and that, therefore, the results
are poor).18 Residents can retrieve new
evidence (techniques), but if they are not
allowed to practice it on patients by their
supervisors, this new evidence will less
likely be applied in later practice.18 Also,
barriers could differ per resident. In a
study by Doran et al24 of preregistration
house officers, 41% of the residents
agreed with the statement that
educational supervisors support critical
appraisal skills in everyday practice, but
this was significantly less so for female
than for male house officers. Differences
between groups of residents have not
been studied thus far.
It is unclear from these studies what
definition of EBM the participants had in
mind. Most of the included studies seem
limited to the search and use of original
studies and do not cover other aspects of
EBM. A broader definition of what
research evidence is (not only original
studies, but also and more easily retrieved
in synopses, summaries, and systems25)
and the practice of integrating research
evidence with the circumstances of the
patient and patient preferences by using
clinical expertise2 might lead to a shift in
perceived barriers.
Implications for medical education
The main resident-specific barriers
toward implementing EBM were a lack of
interest by the staff or even staff
disapproval of EBM. A positive learning
environment, with staff members as EBM
role models, might therefore be one of
the most important factors influencing
the behavior of residents.
Besides reserving specific time for
teaching about EBM in the training
programs of residents, another step in
reducing the time-related barrier could
be to provide a clear definition and
corresponding expectations of the
practice of EBM.26 Finding and
appraising all relevant original papers on
all clinical questions that rise during a
consultation is of course not useful or
attainable for residents.26 Studies
assessing the amount of time needed to
find all relevant original studies indicate
that even skilled searchers like clinical
librarians need 45 minutes.27 One
important solution is the use of
preappraised papers or evidence-based

Academic Medicine, Vol. 85, No. 7 / July 2010

Graduate Medical Education

synopses, summaries (systematic reviews

and evidence-based clinical practice
guidelines), or evidence-based decision
support systems when available.25,28
These resources need to be approached
critically as well,29 but using them can
save a lot of time without reducing the
quality of the retrieved evidence. Also,
evidence preappraised by the residents
own institution and stored in CAT
databases or computer-based clinical
support systems25,30,31 could save time
while still encouraging the practice of
EBM. A study by Sackett and Straus28
reported that an EBM cart, consisting
of secondary sources from their
own department, textbooks, and
computerized (summary) sources such
the Cochrane Library and MEDLINE,
was used during rounds. They found that
the presence of the cart raised the
number of searches for evidence and that
90% of the searches were completed in
the time available on rounds (90
seconds). Also, multiple other
interventions are available and have been
tested. Various Web-based tools,32,33
EBM curricula,34 teach-the-teacher
programs,35 active involvement of clinical
librarians,36 EBM resident rotations,37
and many other educational
interventions have been shown to be
effective in reducing specific barriers to
practicing EBM. Only clinically
integrated teaching, however, has been
reported to improve not only knowledge
but also skills, attitudes, and behavior
toward EBM.38

willingness to practice EBM and the

recognition of answerable clinical
questions should be present before time
is necessary to find the answers.39 Barriers
that are located earlier in the process of
EBM could therefore be more important,
and should be overcome, before
interventions reducing barriers, such as
lack of time or computer facilities, are
most useful. Additionally, the importance
of different barriers for specific (groups
of) residents should be studied. Methods
for identifying the barriers for individual
residents are required to identify their
specific learning needs. These needs can
then be used as a basis for optimizing the
learning process of the residents and
creating a learning environment in which
residents are stimulated to overcome
their own barriers.

Implications for research

References

To create a complete overview of the

barriers of residents in the application of
EBM in daily practice, studies on the
barriers experienced when combining the
evidence with the patients clinical and
personal circumstances and preferences
are required. In these studies, a clear
definition of EBM should be used,
including these patient-factors and all the
evidence-based information sources that
can be applied in clinical practice.25
Measuring the implementation of EBM
as a decision process, however, remains
difficult.
The next step in overcoming barriers is to
quantify the effects of the various barriers
on residents application of EBM.
Currently, the relative importance of
each barrier in the process of EBM is
unknown. Although time is mentioned
often as an important barrier, the

Academic Medicine, Vol. 85, No. 7 / July 2010

10

11

12
13

Besides the more general barriers for the

practice of EBM that practicing
physicians are subject to, residents
experience the additional and potentially
negative influences of clinical supervisors,
the lack of role models, and practical and
institutional barriers. These barriers
should be dealt with when attempting to
integrate evidence-based practice into the
training programs of residents.

14

15

Funding/Support: Department of General

Practice, Academic Medical Center, University of
Amsterdam; Dutch Cochrane Centre.
16
Other disclosures: None.
Ethical approval: Not applicable.

1 Dawes M, Summerskill W, Glasziou P, et al.

Sicily statement on evidence-based practice.
BMC Med Educ. 2005;5:1.
2 Haynes RB, Devereaux PJ, Guyatt GH.
Physicians and patients choices in evidence
based practice. BMJ. 2002;324:1350.
3 Ely JW, Osheroff JA, Ebell MH, et al.
Obstacles to answering doctors questions
about patient care with evidence: Qualitative
study. BMJ. 2002;324:710.
4 Grol R, Grimshaw J. From best evidence to
best practice: Effective implementation of
change in patients care. Lancet. 2003;362:
12251230.
5 Freeman AC, Sweeney K. Why general
practitioners do not implement evidence:
Qualitative study. BMJ. 2001;323:1100 1102.
6 Schaafsma F, Hulshof C, van Dijk F, Verbeek
J. Information demands of occupational
health physicians and their attitude towards
evidence-based medicine. Scand J Work
Environ Health. 2004;30:327330.
7 Von Elm E, Altman DG, Egger M, Pocock SJ,
Gtzsche PC, Vandenbroucke JP. STROBE
Initiative. The Strengthening the Reporting of
Observational Studies in Epidemiology
(STROBE) statement: Guidelines for reporting

17
18

19
20

21
22

23

observational studies. Epidemiology. 2007;18:

800 804.
Vandenbroucke JP, von Elm E, Altman DG,
et al. Strengthening the Reporting of
Observational Studies in Epidemiology
(STROBE): Explanation and elaboration.
Epidemiology. 2007;18:805835.
Giacomini MK, Cook DJ. Users guides to the
medical literature: XXIII. Qualitative research
in health care A. Are the results of the study
valid? Evidence-Based Medicine Working
Group. JAMA. 2000;284:357362.
The Cochrane Collaboration. Cochrane
Handbook for Systematic Reviews of
Interventions. Available at: http://
www.cochrane-handbook.org. Accessed
January 6, 2010.
Montori VM, Tabini CC, Ebbert JO. The
clinical decision-making conference: A
qualitative assessment of learners experience.
J Gen Intern Med. 2000;15(suppl):40.
Ruff TR, Green ML. Why residents fail to
answer their clinical questions: A qualitative
study. J Gen Intern Med. 2004;19(suppl):240.
Letelier LM, Zamarin N, Andrade M, et al.
Exploring language barriers to evidence based
medicine: Is it different to read a Cochrane
review abstract in English or in Spanish for
Chilean post graduate medical students?
Paper presented at: International Conference
of Evidence-Based Health Care Teachers &
Developers; Taormina, Italy; Nov 26, 2005.
Montori VM, Tabini CC, Ebbert JO. A
qualitative assessment of 1st-year internal
medicine residents perceptions of
evidence-based clinical decision making.
Teach Learn Med. 2002;14:114 118.
Green ML, Ruff TR. Why do residents fail to
answer their clinical questions? A qualitative
study of barriers to practicing evidence-based
medicine. Acad Med. 2005;80:176 182.
Letelier LM, Zamarin N, Andrade M, et al.
Exploring language barriers to Evidencebased Health Care (EBHC) in post-graduate
medical students: A randomised trial. Educ
Health (Abingdon). 2007;20:82.
Ho P, Kim P, Hong JJ, Fontelo P. Virtual
evidence cart and resident use of EBM. J Gen
Intern Med. 2006;21(suppl 4):161.
Bhandari M, Montori V, Devereaux PJ,
Dosanjh S, Sprague S, Guyatt GH. Challenges
to the practice of evidence-based medicine
during residents surgical training: A
qualitative study using grounded theory.
Acad Med. 2003;78:11831190.
Green ML, Ciampi MA, Ellis PJ. Residents
medical information needs in clinic: Are they
being met? Am J Med. 2000;109:218 223.
Amin M, Saunders JA, Fenton JE. Pilot study
of the knowledge and attitude towards
evidence based medicine of otolaryngology
higher surgical trainees. Clin Otolaryngol.
2007;32:133135.
Allan GM, Manca D, Szafran O, Korownyk C.
EBM: A challenge for international medical
graduates. Fam Med. 2007;39:160.
Burneo JG, Jenkins ME, Bussiere M.
Evaluating a formal evidence-based clinical
practice curriculum in a neurology residency
program. J Neurol Sci. 2006;250:10 19.
Fletcher KE, Underwood W III, Davis SQ,
Mangrulkar RS, McMahon LF Jr, Saint S.
Effects of work hour reduction on residents
lives: A systematic review. JAMA. 2005;294:
1088 1100.

1169

Graduate Medical Education

24 Doran T, Maudsley G, Zakhour H. Time
to think? Questionnaire survey of preregistration house officers experiences of
critical appraisal in the Mersey Deanery. Med
Educ. 2007;41:487494.
25 Haynes RB. Of studies, syntheses, synopses,
summaries, and systems: The 5S evolution
of information services for evidence-based
healthcare decisions. Evid Based Med. 2006;
11:162164.
26 Akl EA, Maroun N, Neagoe G, Guyatt G,
Schunemann HJ. EBM user and practitioner
models for graduate medical education: What
do residents prefer? Med Teach. 2006;28:192
194.
27 Gorman PN, Ash J, Wykoff L. Can primary
care physicians questions be answered using
the medical journal literature? Bull Med Libr
Assoc. 1994;82:140 146.
28 Sackett DL, Straus SE. Finding and applying
evidence during clinical rounds: The
evidence cart. JAMA. 1998;280:1336 1338.
29 AGREE collaboration. Development and
validation of an international appraisal
instrument for assessing the quality of clinical

30

31

32

33

34

practice guidelines: The AGREE project. Qual

Saf Health Care. 2003;12:18 23.
Patterson ES, Doebbeling BN, Fung CH,
Militello L, Anders S, Asch SM. Identifying
barriers to the effective use of clinical
reminders: Bootstrapping multiple methods.
J Biomed Inform. 2005;38:189 199.
Hunt DL, Haynes RB, Hanna SE, Smith K.
Effects of computer-based clinical decision
support systems on physician performance
and patient outcomes: A systematic review.
JAMA. 1998;280:1339 1346.
Belda TE, Gajic O, Rabatin JT, Harrison BA.
Practice variability in management of acute
respiratory distress syndrome: Bringing
evidence and clinician education to the
bedside using a Web-based teaching tool.
Respir Care. 2004;49:10151021.
Bell DS, Fonarow GC, Hays RD, Mangione
CM. Self-study from Web-based and printed
guideline materials. A randomized, controlled
trial among resident physicians. Ann Intern
Med. 2000;132:938 946.
Green ML, Ellis PJ. Impact of an evidencebased medicine curriculum based on adult

35

36

37

38

39

learning theory. J Gen Intern Med. 1997;12:

742750.
Cartwright CA, Korsen N, Urbach LE.
Teaching the teachers: Helping faculty in a
family practice residency improve their
informatics skills. Acad Med. 2002;77:385
391.
Earl MF, Neutens JA. Evidence-based
medicine training for residents and students
at a teaching hospital: The librarys role in
turning evidence into action. Bull Med Libr
Assoc. 1999;87:211214.
Akl EA, Izuchukwu IS, El-Dika S, Fritsche L,
Kunz R, Schunemann HJ. Integrating an
evidence-based medicine rotation into an
internal medicine residency program. Acad
Med. 2004;79:897904.
Coomarasamy A, Khan KS. What is the
evidence that postgraduate teaching in
evidence based medicine changes anything? A
systematic review. BMJ. 2004;329:1017.
Cabana MD, Rand CS, Powe NR, et al. Why
dont physicians follow clinical practice
guidelines? A framework for improvement.
JAMA. 1999;282:1458 1465.

Cover Art
Artists Statement: Back Again
By definition, parts are no more than
pieces of a whole. But when parts stand
alone, what are they? They become
whole unto themselves. In this spirit, I
created the parts of Back Again to be
seen in unison but also to be seen
staggered or even completely
separately.
On the journey through medical
school, one of the first stops is a
dark basement filled with dead
bodies corpses to be dissected and
analyzed in layers. Their layers are
peeled away, so students like me can
understand how to treat the living,
who are every bit as layered as these
first patients of ours. Art, for me,
exploded after anatomy class. After
my initial disgust with death abated,
my thoughts became clearer and
my hands more steady. I felt
exhilarated to touch the sinew and
soul I would imitate with brush and
canvas. In my painting, the man sits
with the same bones and anatomy
piercing through his skin, visible as if
by X-ray eyes. How magnificent to

1170

creation, of evolution, of God, of

perfection.
The human body is divine, oscillating
in harmony with itself. Think of your
layers: vellus hair, keratinized
squamous skin, muscle, and adipose
all working in concert. Now think
deeper: cells, third space, electrolytes,
adenosine, guanine. How deep can we
go before we must really submit to not
understanding? What or who is the
great organizer that instructs the
minute proteins to align just so? My art
imitates the perfection and flaw found
through observing the complexity of
human form. I am not afraid to show
mistakes in the painting. They add
character; they tell the story of the
evolution of the workthe story of
creation. Our bodies are miracles;
medical school has taught me that
much. Treat yours well.
Back Again

Alan Alfonso Lazzara, Jr

understand the gross form, to see how
the muscles pull perfectly on porous
bone, and to feel the uniqueness of

Mr. Lazzara is a fourth-year medical student,

Loyola University Chicago Stritch School of
Medicine, Maywood, Illinois.

Academic Medicine, Vol. 85, No. 7 / July 2010