Evidence-based practice in EPA

What is evidence based practice?

Making sure there is a valid rationale for a treatment

Is there evidence that the treatment works and that it is safe?

Why is this important?

It is unethical to give treatments when

we dont know if they really work

fraudulent to charge patients for placebo effects (sugar pills are 5

cents!)

effective treatments may not be used

wrong to waste their time and our government salaries

we dont know if they could cause harm

How do we know if something works?

Theoretical studies suggest a physiological effect

Experimental animal studies show an effect

An RCT in patients says it works better than a placebo

A systematic review of RCTs says it works

How valid is valid?

Some valid evidence is better than others.

The quality of the evidence will determine how much belief we have that
we are using an appropriate treatment

We will NEVER have complete belief that a treatment really works

So there is always going to be a spectrum of belief from some to a lot

Where do you draw the line?

RCT evidence?

Randomised Controlled trials

Problems with a single group
Hawthorne effect:

the alteration of behaviour

by the subjects of a study
due to their awareness of
being observed.
As well as intervening variable
o hypothetical variable used to explain causal links between
other variables.
o Intervening variables cannot be observed in an experiment (that's
why they are hypothetical)
We have no idea how much of this improvement is due to treatment

Internal validity: Extent to which we are sure that the results are solely
due to the treatment

If internal validity high treatment CAUSES improvement in pain

If internal validity low treatment MAY cause improvement in pain but we
dont really know.

How do we eliminate these intervening variables from the picture?

Control group theoretically should be exposed to the SAME intervening
variables

Control group will only be exposed to the same intervening variables if it is

very similar to the intervention group
o For example.
intervention group are aged 25-40
control group are aged 65 to 80
we might expect the intervention group to have more of a natural
recovery effect
o Apparent treatment effect Real treatment effect
So how do we get comparable groups?
o By randomisation

The randomised trial

Randomised (clinical) trials are clinical trials in which participants are

randomly allocated to one of two or more treatments

Randomisation should lead to a good random mixing of all characteristics

This should allow elimination of intervening variables and allow us to make

a useful causal interpretation of effects

With a well-conducted RCT, the difference in the outcome (pain) between

groups can be taken as the difference in effects between treatments

Because the groups were SO SIMILAR that they had the same placebo
effects, natural recovery effects and Hawthorne effects. Thus all these
factors cancelled each other out, leaving just the difference in treatment
effect

So we can say that the treatment was the ONLY factor that could explain
the different levels of pain after treatment because it was the ONLY factor
that was different between the groups

Drawbacks of RCTs (provided they have been carried out correctly)

In terms of external validity there may be limitations

RCTs may tend to keep characteristics of patients in narrow ranges to

prevent any large chance differences between groups

External validity is the extent to which the findings can be extended

to wide contexts (like patients living in a different area, or various
age groups)

This may limit external validity

They may also have to have small sample sizes as it is difficult to recruit
subjects for RCTs

Again this can limit the range of people included and so limits
external validity

Can cause false negative findings

These problems may be reduced if we look at several RCTs together

Systematic reviews

These look at ALL the RCTs comparing a treatment to another group.

They synthesise the findings from studies in slightly different populations

this increases external validity

we can start to see differences even if individual studies dont show clear
differences

Reduces chance of a false negative

They are therefore often considered the highest form of evidence

A meta-analysis in a systematic review

Other evidence

Non-randomised comparisons may be very biased because the groups

may be very different

Often called quasi-randomised

But how biased depends on how the people are allocated to groups

Based on patient preference

Based on the hospital someone goes to

Based on time maybe one treatment before 2007 and then the
new treatment after 2007

Based on clinician preference

Based on severity

Theoretical studies can be a useful secondary source of evidence if there

is an absence of evidence from the RCTs/SRs

CAUTION!!!

If a study suggests no effect of a treatment this does not necessarily mean

it does not work

Doses may have been wrong in studies

Very few studies that are small

Flawed studies

Need to consider the methodology and decide if it is a case of:

EVIDENCE OF ABSENCE

ABSENCE OF EVIDENCE

Application of theoretical evidence

SR of US published in 2001 suggested that US does not work
But the doses used were at odds with the theoretical evidence

US biophysical and physiological effects

US causes heating of the tissues by mechanical agitation biophysical

The depth of heating varies with the frequency and the intensity
biophysical

Heating of tissues has physiological effects when the temperature

reaches 41-45 degrees

But the doses used in the studies would not have caused enough heating
in the target tissues to lead to any physiological effect

So no wonder US didnt have a clear clinical effect

Conclusion

So the SR results were not really very valid

Thus no evidence of absence: evidence of any kind that suggests

something is missing or that it does not exist.

Absence of evidence: evidence is missing

The theoretical research does suggest that certain doses WILL cause
heating in target tissues that could lead to effects on collagen
extensibility, blood flow, and healing rates.

So we have some indirect evidence for those doses

Nevertheless still no strong clinical evidence that US at those doses

actually helps patients

The CLINICAL research needs to be done

So do we use US?

We have a theoretical basis for US.

There is an absence of clinical evidence at present but more RCTs using

proper doses may provide some clinical evidence.

Wrong to discount US

BUT also wrong to assume it works, based on the weak evidence.

Until the clinical evidence using better dosage emerges we still

dont really know

In the meantime we should use it when there is no viable alternative and

explain the evidence to the patient.

Its all about being honest about the evidence: Some evidence it may
possibly work, but its not very strong.

Interpretation of evidence

It is never black and white

Good RCT /SR evidence USE

No good RCT / SR evidence DONT USE

We need to weigh up the evidence for each modality in a fluid and

intelligent way.