Professional Documents
Culture Documents
Journal Club
Toolbox
Arjun Rajagopalan
Critical reading of “original” medical articles
CONTENTS
Interpreting interventional
STEP 9A: 20
studies
Interpreting studies on
STEP 9B: 21
value of diagnostic tests
Interpreting studies on
STEP 9C: 23
risk/ association/ causality
© Dr Arjun Rajagopalan - 1
Critical reading of “original” medical articles
© Dr Arjun Rajagopalan - 2
Critical reading of “original” medical articles
T
here's way too much stuff out there waiting to be read. The
repeated chanting of the mantra of “evidence based
medicine” leaves you with the nagging feeling that you
should at least try to read some of the stuff, but, if you are like me,
you have never received any formal instruction on how to go about
making sense of arcane stuff like “p” values and 95% confidence
intervals. Your memory of journal clubs is that of the potato chips
and snacks that were provided.
There are 2 kinds of journal readers:
2. The hard core researchers and nerds who will wade through
a hundred and thirty-six references to answer a simple
question and who, somehow, don't strike us as being
capable of taking care of patients in the real world.
STEP 1
© Dr Arjun Rajagopalan - 3
Critical reading of “original” medical articles
STEP 2
A proposition or statement
Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Cras suscipit, diam ultricies consectetuer
elementum, purus tellus dignissim odio, vitae porttitor odio orci id quam. Aenean interdum nunc id elit.
Integer aliquet tempus augue. Nam erat. Praesent malesuada quam. Morbi id mauris. Fusce porta
justo ut diam.
STEP 3
© Dr Arjun Rajagopalan - 4
Critical reading of “original” medical articles
A necessary first step is to tag the study as one of the three types
listed. The title of the paper will be sufficient in most cases.
© Dr Arjun Rajagopalan - 5
Critical reading of “original” medical articles
STEP 4
Example:
In normotensive, type I diabetics (population), does a low dose of 1.25 mg of the ACE inhibitor
ramipril (indicator variable), prevent progression of incipient diabetic nephropathy as measured by
microalbuminuria (outcome variable), as compared to standard, 5 mg doses of ramipril or placebo.
The paper's conclusion needs to be measured against the stated question and assessed for the
degree of completeness and lack of evasiveness in addressing the issues raised. In this study,
the authors have concluded:
Microalbuminuria was reduced significantly by ramipril treatment in type 1 diabetic patients without
hypertension, as compared to placebo. Although the magnitude of the response was greater, there
was no significant difference between responses to 1.25 or 5 mg ramipril.
(Note: The paper is reported without any attempt at examining the details of the study)
© Dr Arjun Rajagopalan - 6
Critical reading of “original” medical articles
Once you have completed this task, identify the 4-part clinical
question as:
3. Fuzzy.
Assume it is everywhere
© Dr Arjun Rajagopalan - 7
Critical reading of “original” medical articles
STEP 5
N o n -p r o b a b ility s a m p le
© Dr Arjun Rajagopalan - 8
Critical reading of “original” medical articles
© Dr Arjun Rajagopalan - 9
Critical reading of “original” medical articles
This example will clarify: (Ref: The risk of cesarean delivery with
neuraxial analgesia given early versus late in labor. NEJM 2005; 352:655-
65.) “The study was designed to have 80 percent power to detect a
difference of 50% in the rate of cesarean delivery, with a two-sided
alpha level of 0.05. The sample size required to detect this
difference was 350 subjects per group.”
© Dr Arjun Rajagopalan - 10
Critical reading of “original” medical articles
STEP 6
Precise and Precise but not Accurate but not Neither precise
accurate accurate precise nor accurate
Observer bias
Good studies will attempt to reduce observer bias through various
strategies. Look for the following:
© Dr Arjun Rajagopalan - 11
Critical reading of “original” medical articles
Measurement bias
Strategies for reducing measurement bias include:
© Dr Arjun Rajagopalan - 12
Critical reading of “original” medical articles
STEP 7
It is therefore mandatory in all comparative trials that there be a reference group (control) against
which the comparison is made. A large number of studies are invalid as evidence because of
inadequacies in the control and the compared groups. Sources of error include:
1.No controls. However large the numbers and however rigorously designed the study, an
uncontrolled study is unacceptable and ranks only as anecdotal evidence. (It is like the sound of one
hand clapping!).
2.Historical controls. Quite commonly, the lack of controls in the study will be addressed by
comparing the results of the study with other studies on the similar topic, done elsewhere, at
different times in the past. Historical controls are not fair comparisons and are unacceptable.
3.Poorly matched controls. Controls may be present, but are not comparable with the study
group. As an example, those is the study group may be younger and with less comorbidties than the
controls and therefore, be associated with better outcomes - comparing apples with oranges.
© Dr Arjun Rajagopalan - 13
Critical reading of “original” medical articles
© Dr Arjun Rajagopalan - 14
Critical reading of “original” medical articles
© Dr Arjun Rajagopalan - 15
Critical reading of “original” medical articles
'p' value
The ' p' value is a simple estimate of the probability of error or chance in producing the observed
difference. By convention, it is expressed as a decimal fraction, for eg. p = 0.03. The table shows some
examples.
In biostatistical usage, a p value of 0.05 or less is taken as a signficant difference; i.e. less than
5% probability that the difference observed is due to chance alone.
© Dr Arjun Rajagopalan - 16
Critical reading of “original” medical articles
2. The 'p' value tells us nothing about the size of the difference
or its direction. It is not a quantitative measure.
All studies rely on samples that are hopefully representative of the truth. The confidence interval is a
quantitative estimate of the range of values within which the truth is likely to lie, with a specified
degree of confidence. The 95% confidence interval is the range of values within which we can be
95% sure that the truth regarding the population lies. CIs can be calculated for any continuous
variable. CIs may be expressed for any value, 90%, 99% etc, but, like the 'p' value, the sweet spot is
the 95% cut-off. The table below compares the 95% CI with 'p' values.
© Dr Arjun Rajagopalan - 17
Critical reading of “original” medical articles
The data summarized in the table shown below is from a RCT testing
the efficacy of pertussis vaccine. Patients were randomly assigned to
receive either pertussis vaccine (study group) or a placebo (control).
The table shows the outcome of the study as measured by the
number of study subjects who developed pertussis in the follow up
period.
Vaccine Placebo
(1670) (1665)
Developed pertussis 72 240
(4.3%) (14.4%)
Ref: Trollfors B, Taranger J, Lagergard T, et al. NEJM 1995: 333: 1045-50
Sensitivity 95% CI
n = 24 95.8% 75 – 100%
n = 240 95.8% 92.5 – 98.0%
© Dr Arjun Rajagopalan - 18
Critical reading of “original” medical articles
STEP 8
© Dr Arjun Rajagopalan - 19
Critical reading of “original” medical articles
STEP 9 A
© Dr Arjun Rajagopalan - 20
Critical reading of “original” medical articles
STEP 9B
© Dr Arjun Rajagopalan - 21
Critical reading of “original” medical articles
Sensitivity Specificity LR + LR -
95% 95% 19 0.05
95% 80% 4.75 0.06
80% 95% 16 0.21
85% 75% 3.4 0.20
75% 85% 5 0.29
75% 65% 2.1 0.38
65% 75% 2.6 0.47
95% 65% 2.7 0.07
65% 95% 13 0.37
70% 70% 2.33 0.43
Sensitivity Specificity LR + LR -
Ultrasound 21% 85% 1.4 5.26
Mammogram 22% 86% 1.57 5.57
MRI 67% 64% 1.86 0.52
Now see what happens when Likelihood Ratios are calculated (which
the authors did not).
• All three have LR+ < 2 - that puts them in the "should we
bother getting it?" category. In simple words, this means
that there will be one false positive test for every two true
positives.
"Indispensable?"
© Dr Arjun Rajagopalan - 22
Critical reading of “original” medical articles
STEP 9C
Risk assessment
Have measurements been made on
more than one occasion?
NO YES
Evidence from these studies is acceptable only if they fall into one of
the two types that are shown above. The longitudinal cohort study is
the better choice but may not always be practical, in which event, a
well done, case-control study is an option.
From the numbers in this simple 2x2 table the following results will
be stated:
© Dr Arjun Rajagopalan - 23
Critical reading of “original” medical articles
Association of obesity and cancer risk in Canada. The use of estrogens and progestins and the risk
Pan SY, Johnson KC, Ugnat AM et al. Am J Epidemiol. of breast cancer in postmenopausal women. Colditz
2004; 159:259-68. GA, Hankinson SE, Hunter DJ et al. NEJM, 1995;
332:1589-93
This is a population-based, case-control study of
21,022 incident cases of 19 types of cancer and This is data from a prospective, cohort study
5,039 controls aged 20-76 years during 1994-1997 During 725,550 person-years of follow-up, 1935
to examine the association between obesity and cases of invasive breast cancer were newly
the risks of various cancers. The study compared
diagnosed. When compared with women who had
people with a body mass index of less than 25
never used hormones, the data was as shown.
kg/m2, with (body mass index of > or = 30 kg/m2.
Odds ratio 95% CI
Relative risk 95% CI
Overall 1.34 1.22 – 1.48
Estrogen alone 1.32 1.14 – 1.54
Colon 1.93 1.61 – 2.31
Estrogen + 1.41 1.15 – 1.74
Pancreas 1.51 1.19 – 1.92 progestin
Breast 1.66 1.33 – 2.06 5-9 yrs users 1.46 1.22 – 1.74
Ovary 1.95 1.44 – 2.64
Prostate 1.27 1.09 – 1.47
© Dr Arjun Rajagopalan - 24
Critical reading of “original” medical articles
Proving causality
© Dr Arjun Rajagopalan - 25
Critical reading of “original” medical articles
STEP 10
INFER, where
© Dr Arjun Rajagopalan - 26
4-part research question Inclusion criteria R ? T
Population:
Predictor variable:
Authors:
Comparison:
BACKGROUND:
Sampling
Probablility sample | |
Simple random| |Stratified random| | Cluster| |
Non-probability sample | |
Consecutive| |Convenience| |Judgmental| |
Sampling scorecard
Target population
Accessible population
Intended population
EBM Dashboard (after inclusion/ exclusion)
Evidence hierarchy
Summing up
Double blind RCT
Prospective cohort
Case control
Case series
Sampling
Measurement
Comparison
Applicability
1 2 3 4 5
Interesting
Novel
Feasible
Ethical
Relevant
/ /200
EBM 4 dummies – - 1 of 2
© Dr Arjun Rajagopalan
Measurement
Devices used
Authors:
Journal:
Affiliation:
Comparison
Controls
Randomised Measurement error
Case controlled Device error Observer error
Non-random
Protocols
Repetition
Gold standard
Training
Scoring
Blinding
Device suited to task
Device used
Historical R ? T
None
1.
Controls - details 2.
Randomisation method 3.
4.
5.
6.
7.
Details
8.
Comparability
Disparity
EBM 4 dummies – - 2 of 2
© Dr Arjun Rajagopalan