The introduction of evidencesbased medicine as a

component of daily practice

By Gaetane C. Michaud, B.Sc.
Medical Student (year IV)
Jessie L. McGowan, M.L.I.S.
Richard H. van der Jagt, M.D., F.R.C.P.(C)
Alison K. Dugan, M.D., F.R.C.P.(C)
Peter Tugwell, M.D., F.R.C.P.(C)
Ottawa General Hospital (O.G.H.)
University of Ottawa
501 Smyth Road
Ottawa, Ontario
Canada K1H 8L6

Evidence-based medicine is an increasingly important concept in

continuing medical education and medical school curricula. To cope
with the rapid evolution of medicine, physicians need to remain
abreast of the many new therapies and diagnostic tools that affect
their practices. Unfortunately, along with the many changes there is
also a surplus of relevant written material. Physicians are unable to
read all of this information due to time constraints. Instead, they
must choose information efficiently. Tools are needed to facilitate this
process.
Over a two-month period, a demonstration model was carried out
at the Ottawa General Hospital to encourage faculty, residents, and
students to incorporate evidence-based medicine into their daily
practice. A study was conducted to investigate the level and type of
information required by these individuals in a clinical setting. A
literature searching service was introduced six months after the
formal introduction of evidence-based medicine in the Department of
Medicine. The logistics of and recommendations for providing such a
service are presented in this paper.

INTRODUCTION lum to a problem-based curriculum. The impact of

The Ottawa General Hospital is an acute-care and
teaching hospital, located in the Health Sciences
Complex of the University of Ottawa, which consists
of the hospital, the Children's Hospital of Eastern
Ontario, the Royal Ottawa Rehabilitation Centre, and
the Health Sciences Building of the University of
Ottawa.
In the 1991/92 academic year, the University of
Ottawa switched from a traditional medical curricu-
this conversion was felt by the staff physicians of the
Ottawa General Hospital, as most of these physicians
also hold a university appointment. Greater emphasis
is now being placed on lifelong learning skills. Stu-
dents in their clerkships rely more heavily on the
medical literature than before and are being taught
valuable searching skills [1].
The incorporation of evidence-based medicine into
the educational setting was felt to be essential. Ac-
cording to the Evidence-based Medicine Working

478 Bull Med Libr Assoc 84(4) October 1996


Group at McMaster University, there has been a par-
adigm shift in the assumptions guiding clinical prac-
tice. In the past, physicians relied upon sound knowl-
edge of pathophysiology, observations from clinical
practice, and content expertise to guide their reso-
lution of patient problems. The physician of the new
paradigm relies much more heavily on a sound
knowledge of the rules of evidence and the ability
to access, select, and interpret useful references from
the medical literature related to specific patient prob-
lems [2].
Although there have not yet been any studies quan-
tifying the impact of literature searching on patient
care, there have been several qualitative studies, such
as those described by K.B. Langton et al., M.E. John-
ston et al., and J.G. Marshall [3-5]. These studies sug-
gested that patient care improved when physicians
used computer-based clinical decision support sys-
tems.
METHODS
The prospective study evaluating the impact of the
demonstration model was undertaken by the depart-
ment of Internal Medicine at the Ottawa General Hos-
pital over a two-month period. Descriptive statistical
analysis was performed on the results. The popula-
tion studied consisted mainly of house staff from the
clinical teaching unit (CTU). The CTU consisted of
two general medicine teams, each composed of a fac-
ulty physician, a senior resident, one or two junior
residents, and one or more clinical clerks.
A literature searching service was made available
to members of the department. A trained literature
searcher followed the teams and was present at group
meetings to ensure that any question raised was sub-
ject to a literature search. This service was promoted
by the department staff.
The study instrument consisted of a series of three
one-page questionnaires. The first document was a
search request form. Its purpose was to determine
demographic information about the user, such as sub-
specialty and level of medical experience. It also ex-
amined the setting in which the clinical question
arose, the publication type desired, and the aim of
the search. Then the search was conducted by using
CD PLUS (now OVID) MEDLINE on CD-ROM and
relevant articles were retrieved by the requester. The
user then was asked to use the second questionnaire
to evaluate the relevance of the articles to the clinical
question. This document also examined the physi-
cian's perception of the impact of the citation on pa-
tient care, patient outcomes, and their own medical
practices. The first two questionnaires were based on
a combination of questions developed by Haynes et
al. [6] and Lindberg et al. [7].
A third document was distributed to those who
Bull Med Libr Assoc 84(4) October 1996
The introduction of evidence-based medicine
participated upon completion of the study. This eval-
uation form was intended to assess the usefulness of
the search service and determine the physicians' and
students' search habits.
RESULTS
Demographics
Table 1 consists of the raw data collected from the
three questionnaires. The majority of physicians and
students who requested searches were enrolled in
training within general medicine. All subspecialties
and other fields of study, such as family medicine,
were grouped together, as none of these subgroups
consisted of a significant number of participants. Most
(90%) of the clinical questions that prompted the
searches involved patient care, and were based on
cases from the emergency room, wards, clinics, and
handover rounds (daily meeting of students and staff
in which those on call the previous night present
interesting cases admitted that night).
From the choice of publication types listed on the
survey, physicians selected a preference for random-
ized controlled trials in 45% of cases, review articles
in 33% of cases, and any available reference in 22%
of cases. Although other publication types were list-
ed, none of these was requested specifically. A cor-
relation was also noted between the level of experi-
ence of the requester and the level of information
sought. Residents requested more clinical trial re-
ports, whereas third- and fourth-year medical stu-
dents requested review articles.
Aim of search and impact on practice
Sixty-three searches were analyzed. The aim of each
search was examined. Perhaps because of the academ-
ic nature of the Ottawa General Hospital, 36% of the
searches were performed to enhance knowledge of a
particular subject. The other significant reason for
searching was therapy, which accounted for 32% of
searches performed.
Regarding the physicians' impressions of the im-
pact of searches on patient care, the main effects were
on the selection of the most appropriate diagnostic
tests and treatments, reported in 31% and 30% of all
searches, respectively.
Regarding the impact on patient outcomes, 52% of
physicians said that searches reduced the use of un-
necessary diagnostic tests and reduced medical costs.
Beyond these major impacts on patient outcomes, the
other two outcomes selected quite frequently were
the choice of alternate therapies and that of alternate
diagnoses, which were identified by 31% of physi-
cians.
Finally, the impact on clinical practice was assessed.
In the majority of cases physicians stated that they
479
Michaud et al.

Table 1
Data collected from the questionnaires
Physician demographics
Level of experience Specialty
Clinical clerk 13 General medicine
PGY-1 8 Other 39
Resident 28 24
Staff 14
Setting from which search was derived: Publication types requested:
Wards 19 Clinical trials
Emergency 15 Review articles 28
Handover 16 All available information 21
Clinic 7 14
Other 6
Aim of search: Perception of impact on patient care:
Update knowledge 12 New treatment 11
Increase knowledge 23 Different treatment 19
Decrease symptoms 7 Most appropriate diagnosis 14
Prognosis 8 Most appropriate test 19
Choice of therapy 13
Impact on patient outcomes: Impact on current practice:
Decrease costs 17 Will change 26
Different treatment 9 Will oonsider changing 18
Different diagnosis 11 Awaiting more data 8
Decrease unnecessary therapy 15 No impact 11
Other 11

would either alter their current practices or at least
consider alternate therapies after seeing a quality ref-
erence from the medical literature. There were three
circumstances in which the physician was most likely
to alter current practices: when dealing with a newly
identified treatment, when dealing with a newly
identified diagnostic test, and when attempting to
reduce costs by cutting back on unnecessary thera-
pies.
Utility of the service
The last phase of the study was an evaluation of the
overall worth of the project. Twenty physicians on
staff during the study chose to provide anonymous
feedback concerning the project. All but one had used
the service provided. Those who had used it said that
the service was useful and that they would continue
to use such a service if it were extended. All these
respondents had read at least some articles retrieved
from the medical literature, which they perceived to
have had some impact on their clinical practice.
DISCUSSION
Although the study was quite small, consisting of
only sixty-three literature searches, significant trends
were evident. Many of the observed trends have been
reported in larger studies.
As noted earlier, most participants requested lit-
erature on clinical trials. This emphasis may reflect
the heavy participation of residents, who requested
more searches than did any other group, and the
strong correlation between the level of information
requested and the level of the user's medical expe-
rience. The latter relationship was studied in depth
by Wildemuth et al., who concluded that medical
students, because they are still developing their
knowledge base, request review articles more fre-
quently than other literature [8]. Residents, due to
their increasing clinical expertise, tend to read more
sophisticated medical literature, such as reports on
randomized controlled trials.
The results of the present study are also supported
by the Rochester study, which examined the impact
of hospital library services on patient care. The Roch-
ester study indicated that the choice of tests and the
choice of drugs were influenced by the medical lit-
erature in 51% and 45% of reported cases, respectively
[5]. Although a slightly lower proportion (32%) of
literature searches performed in the Ottawa study was
aimed at determining the most appropriate therapy,
the percentage is still significant.
House staff clearly appreciate the utility of evi-
dence-based medicine. They are receptive to the in-
formation obtained from the medical literature and
are willing to base clinical decisions on relevant find-
ings from the references. Although all the physicians
surveyed enjoyed having a search service offered to
them, few performed more than one or two searches
per week.
Feedback from the house staff suggests that a lack
of resources is a significant obstacle to searching. Ac-
cessibility is the key. House staff have hectic sched-
ules and are frequently occupied with patients during

480 Bull Med Libr Assoc 84(4) October 1996


regular office hours. A searching system should there-
fore be accessible twenty-four hours per day, seven
days per week. Furthermore, numerous stations must
be available at all times to accommodate multiple users.
As patient care is the thrust of most searches, it is
crucial that the physician be able to access references
immediately, so the literature can be used in diag-
nosing and treating patients.
Accessibility does not mean just computer access,
but also database and computer knowledge. Many
physicians may be willing to incorporate relevant
written information into their clinical practices but
unable to obtain such references. Physicians must be
taught how to operate their hospital's particular med-
ical information system and to formulate the com-
ponents of a clinical question and perform critical
appraisal. In the Ottawa study, the quality of clinical
questions, not to mention the literature searches
themselves, were vastly improved by the five evi-
dence-based medicine information sessions offered to
the participants [9].
Once the search has been performed, the reference
will be useful only if the journal is available. Due to
the size and limited budgets of most medical libraries,
journal selection can be quite limited. If physicians
are truly using evidence-based medicine to reduce
costs by choosing the one diagnostic test that is the
most sensitive and specific, then perhaps additional
hospital funding should be provided to expand li-
brary resources. If increasing the number of available
journals is not feasible, then an effort must be made
to ensure that interlibrary loans are very efficient.
Turn-around times must be such that the impact of
the literature on patient care is not obviated. When
dealing with patient care references, turn-around
times of more than a couple of days make the search
worthless, as clinical decisions concerning diagnosis
and treatment typically have to be made in a shorter
time period.
In short, physicians are receptive to the concept of
evidence-based medicine if literature is easily acces-
sible and user friendly. They are influenced by rel-
evant information from the medical literature, par-
ticularly when dealing with new therapies and di-
agnostic tests. If evidence-based medicine is to be user
friendly, then available resources must be optimized
and accessible. Additional resources should also be
made available.
CONCLUSIONS AND RECOMMENDATIONS
The experience at Ottawa General indicated that a
literature searching service would be beneficial for
patient care and that its role should be expanded. In
addition, it was evident that literature needs differ
depending on the level of medical training. Follow-
Bull Med Libr Assoc 84(4) October 1996
The introduction of evidence-based medicine
ing are recommendations regarding the resources
necessary to provide an environment encouraging
the use of evidence-based medicine in the delivery
of quality, cost-efficient patient care:
1. The system must be accessible whenever a clinical
question is formulated, meaning twenty-four hours
per day, every day of the year.
2. The system must have many user stations available
at strategic points within the hospital.
3. Physicians must have a foundation of knowledge
in the use of evidence-based medicine. They must be
familiar with the components of a clinical question,
critical appraisal of the medical literature, and the use
of specific medical databases.
4. The required reference must be available imme-
diately if dealing with a question on patient care, and
within forty-eight hours in all other circumstances.
Journal selection must be maximized and turn-around
time on interlibrary loan requests must be rapid.
5. Senior staff members should be encouraged to pose
clinical questions to their students and to expect a
solution to the problem, if students' learning expe-
rience is to be complete.
REFERENCES
1. SHIN JH, HAYNES RB, JOHNSON ME. Effect of problem-
based, self directed undergraduate education on life-long
learning. CMAJ 1993 Mar 15;148(6):969-76.
2. EvIDENCE-BASED MEDICINE WORKING GROUP. Evidence-
based medicine, a new approach to teaching the practice
of medicine. JAMA 1992 Nov 4;268(17):2420-5.
3. LANGTON KB, JOHNSTON ME, HAYNES RB, MATHIEU A. A
critical appraisal of the literature on the effects of computer-
based clinical decision support systems on clinician per-
formance and patient outcomes. Proc Annu Symp Comput
Appl Med Care 1992:626-30.
4. JOHNSTON ME, LANGTON KB, HAYNES RB, MATHIEU A.
Effects of computer-based clinical decision support systems
on clinician performance and patient outcome. A critical
appraisal of research. Ann Intern Med 1994 Jan 15;120(2):
135-42.
5. MARSHALL JG. The impact of hospital library on clinical
decision making: the Rochester study. Bull Med Libr Assoc
1992 Apr 80(2):169-78.
6. HAYNEs RB, McKIBBON KA, WALKER CJ, RYAN N, ET AL.
Online access to MEDLINE in clinical settings. A study of
use and usefulness. Ann Inter Med 1990 Jan 1;112(1)78.
7. LINDBERG DA, SIEGAL ER, RAPP BA, WALLINGFORD KT, ET
AL. Use of MEDLINE by physicians for clinical problem
solving. JAMA 1993 Jun 23-30;269(24):3124-9.
8. WILDEMUTH BM, DE BLIEK R, FRIEDMAN CP, MIYA TS. In-
formation-seeking behaviors of medical students: a classi-
fication of questions asked of librarians and physicians. Bull
Med Libr Assoc 1994 Jul;82(3):295-304.
9. GOLDMAN F. An introduction to evidence-based medi-
cine: a self-instructional module. 1994 (unpublished).
Received September 1995; accepted February 1996
481