Critical Appraisal for Therapy Articles THERAPY STUDY: Are the results of the trial valid?

(Internal Validity) What question did the study ask? Patients Intervention Co!parison "utco!e(s) #a$ % Was the assi&n!ent of patients to treat!ents rando!ised? What is best? Where do I find the information? Centralised computer randomisation is ideal and often The 'ethods should tell you ho( patients (ere used in multi-centred trials. Smaller trials may use an allocated to &roups and (hether or not rando!isation independent person (e.g, the hospital pharmacy) to (as concealed$ police the randomization. This paper) *es ( +o ( ,nclear ( Co!!ent) #-$ % Were the &roups si!ilar at the start of the trial? What is best? Where do I find the information? If the randomisation process worked (that is, achie ed The %esults should have a ta-le of ./aseline compara!le groups) the groups should !e similar. "he Characteristics. co!parin& the rando!i0ed &roups more similar the groups the !etter it is. on a nu!-er of varia-les that could affect the "here should !e some indication of whether differences outco!e (ie$ a&e1 risk factors etc)$ If not1 there !ay !etween groups are statistically significant (ie. p alues). -e a description of &roup si!ilarity in the first para&raphs of the %esults section$ This paper) *es ( +o ( ,nclear ( Co!!ent) 2a$ A Aside fro! the allocated treat!ent1 (ere &roups treated equally? What is best? Where do I find the information? #part from the inter ention the patients in the different $ook in the Methods section for the follow-up schedule, groups should !e treated the same, eg., additional and permitted additional treatments, etc and in Results for treatments or tests. actual use. This paper) *es ( +o ( ,nclear ( Co!!ent) 2-$ A Were all patients (ho entered the trial accounted for? and (ere they analysed in the &roups to (hich they (ere rando!ised? What is best? Where do I find the information? $osses to follow-up should !e minimal % prefera!ly less "he Results section should say how many patients were than &'(. )owe er, if few patients ha e the outcome of interest, then e en small losses to follow-up can !ias the results. *atients should also !e analysed in the groups to which they were randomised % +intention-to-treat analysis. This paper) *es ( +o ( ,nclear ( Co!!ent) 3$ ' Were !easures o-4ective or (ere the patients and clinicians kept 5-lind6 to (hich treat!ent (as -ein& received? What is best? Where do I find the information? It is ideal if the study is +dou!le-!linded, % that is, !oth -irst, look in the Methods section to see if there is some patients and in estigators are unaware of treatment mention of masking of treatments, eg., place!os with the allocation. If the outcome is objective (eg., death) then same appearance or sham therapy. Second, the !linding is less critical. If the outcome is subjective (eg., Methods section should descri!e how the outcome was symptoms or function) then !linding of the outcome assessed and whether the assessor.s were aware of the assessor is critical. patients/ treatment. This paper) *es ( +o ( ,nclear ( Co!!ent)

,niversity of "7ford1 2889

Critical Appraisal for Therapy Articles What (ere the results? #$ :o( lar&e (as the treat!ent effect? 0ost often results are presented as dichotomous outcomes (yes or not outcomes that happen or don/t happen) and can include such outcomes as cancer recurrence, myocardial infarction and death. 1onsider a study in which 23( ('.23) of the control group died and 2'( ('.2') of the treatment group died after & years of treatment. "he results can !e e4pressed in many ways as shown !elow. What is the !easure? Relative Risk (RR) 5 risk of the outcome in the treatment group . risk of the outcome in the control group. What does it !ean? "he relati e risk tells us how many times more likely it is that an e ent will occur in the treatment group relati e to the control group. An %% of # !eans that there is no difference -et(een the t(o &roups thus1 the treat!ent had no effect$ An %% ; # !eans that the treat!ent decreases the risk of the outco!e$ An %% < # !eans that the treat!ent increased the risk of the outco!e$ Since the 66 7 2, the treatment decreases the risk of death. "he a!solute risk reduction tells us the a!solute difference in the rates of e ents !etween the two groups and gi es an indication of the !aseline risk and treatment effect. #n ARR of 0 !eans that there is no difference -et(een the t(o &roups thus1 the treat!ent had no effect$ "he a!solute !enefit of treatment is a 3( reduction in the death rate. "he relati e risk reduction is the complement of the 66 and is pro!a!ly the most commonly reported measure of treatment effects. It tells us the reduction in the rate of the outcome in the treatment group relati e to that in the control group. "he treatment reduced the risk of death !y 88( relati e to that occurring in the control group. "he num!er needed to treat represents the num!er of patients we need to treat with the e4perimental therapy in order to pre ent 2 !ad outcome and incorporates the duration of treatment. 1linical significance can !e determined to some e4tent !y looking at the <<"s, !ut also !y weighing the <<"s against any harms or ad erse effects (<<)s) of therapy.

In our e7a!ple1 the %% = 8$#8>8$#9 = 8$?@ Absolute Risk Reduction (ARR) 5 risk of the outcome in the control group - risk of the outcome in the treatment group. "his is also known as the absolute risk difference. In our e4ample, the #66 5 '.23 - '.2' 5 '.'3 or 3( Relative Risk Reduction (RRR) 5 a!solute risk reduction . risk of the outcome in the control group. #n alternati e way to calculate the 666 is to su!tract the 66 from 2 (eg. 666 5 2 - 66) In our e4ample, the 666 5 '.'3.'.23 5 '.88 or 88( 9r 666 5 2 - '.:; 5 '.88 or 88( Number Needed to Treat (NNT) 5 in erse of the #66 and is calculated as 2 . #66.

In our e4ample, the <<" 5 2. '.'3 5 &'

=e would need to treat &' people for & years in order to pre ent 2 death. 2$ :o( precise (as the esti!ate of the treat!ent effect? "he true risk of the outcome in the population is not known and the !est we can do is estimate the true risk !ased on the sample of patients in the trial. "his estimate is called the point estimate. =e can gauge how close this estimate is to the true alue !y looking at the confidence inter als (1I) for each estimate. If the confidence inter al is fairly narrow then we can !e confident that our point estimate is a precise reflection of the population alue. "he confidence inter al also pro ides us with information a!out the statistical significance of the result. If the alue corresponding to no effect falls outside the >3( confidence inter al then the result is statistically significant at the '.'3 le el. If the confidence inter al includes the alue corresponding to no effect then the results are not statistically significant. Will the results help !e in carin& for !y patient? (A7ternalValidity>Applica-ility) "he ?uestions that you should ask !efore you decide to apply the results of the study to your patient are@ Is my patient so different to those in the study that the results cannot applyA Is the treatment feasi!le in my settingA =ill the potential !enefits of treatment outweigh the potential harms of treatment for my patientA

,niversity of "7ford1 2889