Information Mastery:

Evidence-Based Medicine in
Everyday Practice

David C. Slawson, MD
Allen F. Shaughnessy, PharmD
The Medical Information Business
Producti
on

Original Research
Clinical
experience

Shaughnessy AF, Slawson DC. Are we providing doctors with the training and tools for lifelong learning?
British Medical Journal 1999 (13 Nov): www.bmj.com. (http://bmj.com/cgi/reprint/319/7220/1280.pdf)
The Medical Information Business
Productio
n

Systematic
reviews
Refinement
POEM Alert
System
The Medical Information Business
Production

Clinician-centered
Just-in-time
information
Hunting/foraging Refineme
tools nt

Distributi
on
The Medical Information Business
Production

Information
Sales & Mastery:
Marketing The Applied
Refineme
Science of nt
Clinical Medicine

Distributio
n
 How do we learn?
Adults learn by solving problems
Our problems = clinical questions
CME can highlight advances and make us
aware of our deficits
120 studies, short term benefit to testing, but no
patient benefit to traditional lecture
Answering clinical questions at the point of
care is how we learn and improve outcomes
 Clinical questions
Theyre common: Average 1 question for every other
patient
Theyre important
Only 1 in 3 questions pursued
Found answer 80% of time
Guess at 70-80% of information needs!
Journals only used to answer 2 of 1101 questions
Books, colleagues used most often
Average search time: 1 minute or less.

Ely JE. Analysis of questions asked by family doctors regarding patient care. BMJ
1999;319:358-61
 Clinical Questions
Internal Medicine Residents
2 for every 3 patients
29% pursued
textbook (31%); journals (21%); attendings (17%)
Patient expectation, fear of malpractice
associated with seeking answer
Lack of time (60%), forgot (29%).
Am J Med 2000;109:218-33.
 Information Sources for the
Point of Care

Everything is based on the usefulness equation:

Usefulness = Relevance x Validity

Work

Slawson DC, Shaughnessy AF, Bennett JH. Becoming a Medical

Information Master:Feeling Good About Not Knowing Everything. The
Journal of Family Practice 1994;38:505-13.
 Validity
The hard part of Information Mastery
Technique: EBM working group
Guyatt G, Rennie D, ed. Users guides to the medical literature. A manual for
evidence-based clinical practice. Chicago: AMA, 2002.

Did the researchers find what they

think they found?
Do the results apply to your patients?
Self vs delegation- Take responsibility
 Work
Not all information sources are
created equal
Two type of information sources
Just-in-case sources: high work
Just-in-time sources: low work
 Relevance: Type of Evidence

POE: Patient-oriented evidence

mortality, morbidity, quality of life
Live longer and/or better
DOE: Disease-oriented evidence
pathophysiology, pharmacology,
etiology
Shaughnessy AF, Slawson DC, Bennett JH. Becoming an Information Master: A Guidebook to the Medical
Information Jungle. The Journal of Family Practice 1994;39(5):489-99.
 POEM
Patient-Oriented
Evidence
that Matters
matters to you, the clinician,
because if valid, will require you
to change your practice
Shaughnessy AF, Slawson DC, Bennett JH. Becoming an Information Master: A Guidebook to the Medical
Information Jungle. The Journal of Family Practice 1994;39(5):489-99.
POEMs: The Change Factor
Pen G prevents rheumatic fever
Would a study showing this require a change?

ALLHAT study (ALLHAT Officers and Coordinators. Major outcomes in high-

risk hypertensive patients randomized to angiotensin-converting enzyme inhibitor or calcium
channel blocker vs diuretic. The Antihypertensive and Lipid-Lowering Treatment to prevent
Heart Attack Trial (ALLHAT). JAMA 2002;288:2981-97).

Diuretics > effective than other treatments to

prevent mortality and morbidity associated with
hypertension, including type 2 diabetes
Would this study require a change in behavior?
 Determining Validity
Levels of Evidence (LOE):
1a, b, c; 2a, b, c; etc., 5- expert opinion
A, B, C, D,
Therapy, diagnosis, prognosis, reviews, etc.
A moving target
The best way to ascertain trust in the
process
 Was allocation assignment
concealed?
Did investigators
know to which
group the potential
subject would be
assigned before
enrolling them?
 Importance of concealed
allocation

Trials with unconcealed allocation

consistently overestimate benefit by
~40%

Schulz KF, Chalmers I, Hayes RJ, et al. JAMA

1995;273:408-12
Schulz KF, Grimes DA. Lancet 2002;359:614-18.
 Was allocation assignment
concealed?

Concealed allocation blinding

Blinding can occur without concealed
allocation
UVA example- surfactant in the NICU
Allocation can be concealed in an
unblinded study
PT vs surgery for knee DJD
Moseley JB, O'Malley K, Petersen NJ, et al. N Engl J Med 2002; 347:81-8.
 Importance of concealed
assignment
Meta-analysis of trials evaluating screening
mammography
In studies in which allocation wasnt concealed
Higher SE status, education level in screened group
Age disparity (average 6 mo older in the unscreened
group)
Richer, smarter, younger = live longer!
Trials with concealed allocation = screening
harmful!
No effect or increased mortality
20% more mastectomies
Lancet Jan 8, 2000; Oct 20, 2001
 Should we use a new test?
Levels of Valid POEMness
1. Sensitivity & specificity
2. Does it change diagnoses?
3. Does it change treatment?
4. Does it change outcomes?
5. Is it worthwhile (to patients and/or society)?
(examples: T4/PKU in newborns, BNP for CHF,
HbA1C for DM, electron beam tomography
for CAD, CRP, BMD)
Fryback DG, Thornbury JR. The efficacy of diagnostic imaging. Med
Decis Making 1991; 11:88-94
Mundus Vult Decipi-
The world wishes to
be deceived
People would rather be deceived
that have the truth cause anxiety

Caleb Carr, Killing Time

 Two Tools Needed to Master
Information- BMJ 1999
A method of being alerted to new
information (a foraging tool)
A tool for finding the information again
when you need it. (a hunting tool)
Without both:
You dont know that new info. is available
You cant find it when you do
Clinical example- Riboflavin for migraines
Shaughnessy AF, Slawson DC. Are we providing doctors with the training and tools for
lifelong learning? British Medical Journal 1999 (13 Nov): www.bmj.com.
(http://bmj.com/cgi/reprint/319/7220/1280.pdf)
 Can we trust reviews:
(Hunting Tools)?
Study of 36 review articles using 10 criteria for determining rigor
Overall rating of rigor:
Experts correlation = 0.23
Non-experts correlation = 0.78
expertise of writer = stronger prior opinion less time spent
on review lower quality
Study of quality of 35 review articles on type 2 DM
Ave. score 1/15; best score 5/15
Experts should do research
Non-experts should write due to less bias
Oxman AD, Guyatt GH. The science or reviewing research. Ann N Y Acad Sci 1993;703:125-33. Shaughnessy AF, Slawson DC.
What happened to the valid POEMs? A survey of review articles on the treatment of type 2 diabetes. British Medical Journal
2003;327:266-9.
Typical Flow of Information
in Medicine
Medical Research Published

Results Summarized by Experts

Review Articles Continuing Medical Education

Clinicians

Patient Care
 Flow of Information from
the UKPDS
Results of UKPDS Published

Results Selectively Presented by Experts

Flawed Review Articles ? Continuing Medical Education

Clinicians

Patient Care Based on Wrong/Incomplete Information

 Translational Validity
Can We Trust Review Articles?
Reporting of the UKPDS by 35 review articles
85% of reviews: readers not told that good
glucose control alone doesnt decrease
mortality (NNR = 7) Important Non-Valid
POEM
All reported that good control decreased
complications
None reported that almost all (84%) benefit due to
decreased rates of retinal photocoagulation (no
change in blindness rate, the Important Non-Valid
POEM)
Valid POEM Only 18% (NNR = 6) reported that
metformin decreased mortality, independent of
BS control
 Translational Validity
Can We Trust Review Articles?
None reported lack of any benefit (micro- or
macrovascular) of insulin/ sulfonylureas in obese
diabetics- Important Non-Valid POEM
Valid POEM Only 13% (NNR = 8) reported that blood
pressure control is more important than BG control
None of the reviews reported all Valid and
Important Non-Valid POEMs (NNR > 35).
Including significant EB databases

Shaughnessy AF, Slawson DC. What happened to the valid

POEMs? A survey of review articles on the treatment of type 2
diabetes. BMJ 2003;327:266-271.
 Lending a Hand to Patients with Type 2 Diabetes
SA
?A

re
/

su
in

es

l
ro
r m

pr
?

ste
tfo l
ro

od

e
e t

ol
n

o
M o

Ch
Bl
s ec
c o
u
ng Gl
i
ok
Sm

Vijan S. Treatment of hypertension in type 2 diabetes mellitus: blood pressure goals, choice
of agents, and setting priorities in diabetes care. Ann Intern Med 2003; 138:593-602.
Drilling for the Best Information

Cochrane Library
EB Practice Guideline
Clinical Evidence
Clinical Inquiries

Specialty-specific
POEMs
Usefulness

Best Evidence
Textbooks, Up-to-
Date, 5-Minute
Clinical Consult

Medline
 InfoRetriever 2005
Windows 95/98/NT/ME/2000/XP, PocketPC, Palm and Web
1900 short research
Cochrane Database
synopses (400 added
of Systematic Reviews:
per year)
over 1300 abstracts
5 Minute Clinical
Consult

150 clinical Bayesian

prediction diagnostic
rules test / H&P
calculator

Basic drug
info by class Key evidence-based
and cost for treatment guidelines
1200 drugs
www.InfoPOEMs.com
 Characteristics of an Ideal
Clinical Awareness System
Specialty-specific
Comprehensive
Coordinated hunting and foraging tools
Specific and reproducible criteria for
relevance and validity
Available at the point-of-care
All backed up by levels of evidence
 The Clinician of the Future
I know a lot, therefore I am
Replaceable by a computer
I think, therefore I am
Never replaceable by computer
Travel agent should they memorize
schedules?
Would you trust them?
How do you know?
Hand held computer = stethoscope of the
future
 Take-Home Points
Confidence through information
Hunting & foraging tools providing
relevant and valid information when
needed
Focus on valid POEMs Patient-
Oriented Evidence that Matters
 Effect on Patient-Oriented
Outcomes
Symptoms SORT SORT
Functioning
Quality of Life B A
Lifespan
Relevance of Outcome

Effect on Disease Markers

Diabetes
Arthritis
Peptic Ulcer SORT
C
Effect on Risk Factors for
Disease
Improvement in markers
(blood pressure, cholesterol)

Uncontrolled Observations Physiologic Research Highly Controlled Research

& Preliminary Clinical Randomized Controlled
Conjecture Research Trials
Case reports Systematic Reviews
Observational studies

Validity of Evidence
 Take-Home Points
Clinicians will be/are valued by how
they think and not by what they know

The information age is about

information management, not
information acquisition