Dr Samira Alsenany

Dr SA 2012

Purpose of this seminar

The purpose of this seminar is to provide

a brief overview of the critical appraisal

process.
A critical review must identify the
strengths and weaknesses in a piece of
research and this should be carried out in
a systematic manner (Eachus, 2003)

Dr SA 2012

Critical Appraisal
Critical appraisal is the process of

systematically examining research evidence

to assess its validity, results, and relevance
before using it to inform a decision (Hill and
Spittlehouse, 2001, p.1).

Dr SA 2012

Critical Appraisal
Critical appraisal is an essential step in the
process of putting research into practice.
Asking questions about an articles research
methodology, scrutinizing its data collection
and analysis methods, and evaluating how its
findings are presented will help you to
determine whether that articles conclusions
should influence practical decision-making.

Dr SA 2012

When should you Critically

?Appraise

Conducting literature reviews for grant proposals.

Evaluating the effectiveness, costs, and benefits
of health programs.
Establishing new health programs.
Implementing policies; and Public health decision
making, especially at the senior management
level.
Dr SA 2012

What is critical
?appraisal
Critical appraisal is an essential part of evidence-

based.
Critical appraisal allows us to make sense of

research evidence and thus begins to close the gap

between research and practice.
The most appropriate design for studying the

effectiveness of a specific intervention or treatment.

Dr SA 2012

Appraising systematic
reviews
Once you have conducted a literature search

and obtained full text articles, you can begin

the critical appraisal process.
Consider the following questions to assess

the quality of the article

Dr SA 2012

Questions to Ask
What kind of reports do I want?
Is this article relevant to my issue and setting?
How much detail do I need?
How comprehensive do I need to be?
How far back should I search?
Dr SA 2012

Questions to Ask
Is it of interest?
Why was it done?
How was it done?
What has been found?
What are the implications?
What else is of interest?

Dr SA 2012

Critically Appraise What You

.Read
Simple checklists enable the useful information
.to be identified

Dr SA 2012

Checklists
Checklists for particular types of literature

are a quick and easy way of learning

critical appraisal.

They all have 3 stages:

Basic questions.
Essential appraisal.
Detailed appraisal

Dr SA 2012

Available at:
http://www.sign.ac.uk/methodology/checklists.html
Scottish Guidelines Network (SIGN) has developed a
number of critical appraisal tools in support of their
creation of evidence-based guidelines. These
appraisal tools used by SIGN to conduct systematic
reviews have been made available for public use on
Critical
Appraisal: Notes and Checklists
their website, including methodology evaluation
checklists for each of the following 6 types of studies:
systematic reviews/meta analyses, RCTs, cohort
studies, case-control studies, diagnostic studies, and
economic evaluations.
Dr SA 2012

Questions to Ask
The abstract
Use the information found in the abstract to answer the
questions below:
Are your issues discussed there?
What are the main findings of the research?
Do you want to know more after reading the abstract?
Was the research done in a similar setting to yours?
Does it address a related question? (Even research
that covers your issue indirectly can be useful.)
Are there reasons to doubt the findings without
reading the whole article?
Dr SA 2012

Questions to Ask
You may conclude that the study is not

reliable merely by reading the abstract.

If this is the case, move on to another article!

Critical

Dr SA 2012

Questions to Ask
Why was it done?

Introduction
Should a clear statement of the purpose of the
study.
Focus on the time, place and circumstances of the population

and setting studied.

How similar or different is the study population or setting to
yours? Is a difference likely to matter for the issue at hand?
The absence of such statements can imply that the authors
had no clear idea of what they were trying to find out.

Or they didnt find anything but wanted

to publish!
Dr SA 2012

How was it done?

Brief

Methods

but should include enough detail to enable one

to judge quality.
Must include who was studied and how they were
recruited.
Basic demographics must be there.
How the data was collected (interviews, focus
groups, questionnaires, surveys, observations, etc.).

Dr SA 2012

Questions to Ask
Results
Three broad issues need to be considered the

results of the study :

A Are the results of the study valid?

B What are the results?
C Will the results help locally

The major challenge.

What do I think this really means?
Dr SA 2012

Questions to Ask
Results
What has it found?

The data should be there statistics.

Is the sample large enough to produce significant

results? Is the effect size clinically or operationally

relevant?

Are the aims in the introduction

addressed in the results?

Look for illogical sequences, bland
statements of results.
Flaws and inconsistencies.
All research has some flaws , the
impact of the flaws need to assessed.
Dr SA 2012

Questions to Ask
How good are the measures?
Do the measures accurately reflect what the
researcher was trying to measure
(validity)?
How clear and appropriate are these
measures? Too broad? Too narrow?
Are they actual measures or proxy measures?
Are the measures well established in either
prior research or through pilot testing?
Dr SA 2012

Questions to Ask
Discussion
The discussion section is more detailed and

precise than the abstract, and will explain:

A review of how these results compare or
contrast with prior research? If this report
found something different from.
The limitations of the research and possible
implications which are not mentioned in
the abstract.

Dr SA 2012

Questions to Ask
implications
What are the implications?
Abstract / discussion.
The whole use of research is how far the results can
be generalised.
All authors will tend to think their work is more
important than the rest of us!
What is new here?
What does it mean for health care?
Is it relevant to my speciality?

Dr SA 2012

 The following 3 reading series were published in core

medical journals and contain in-depth

information on critical appraisal and evidence-based
practice.
From the British Medical Journal (BMJ):
How to Read a Paper: The Basics of Evidence-Based
Medicine
Available at: http://www.bmj.com/collections/read.dtl
Evidence-Based Nursing Notebook
Available at: http://ebn.bmj.com/cgi/collection/notebook
From the Journal of the American Medical Association (JAMA):
User Guides to Medical Literature
Available at: http://www.shef.ac.uk/scharr/ir/userg.html

Recommended Reading
Dr SA 2012

Online Critical Appraisal

Tools

The Centre for Methods and Tools (http://www.nccmt.ca/) is currently developing a critical
appraisal framework that is specifically tailored to
public health research. This guide will be the definitive critical appraisal resource for
public health practitioners in Canada.
AGREE Instrument
Available at: http://www.agreetrust.org/instrument.htm
The purpose of the Appraisal of Guidelines Research & Evaluation (AGREE) Instrument
is to provide a framework for assessing the quality of clinical practice guidelines. It
originates from an international collaboration of researchers and policy makers who work
together to improve the quality and effectiveness of clinical practice guidelines by
establishing a shared framework for their development, reporting and assessment.
A Beginners Guide to Judging Research Studies
Available at http://www.cihr-irsc.gc.ca/e/34192.html
This is a succinct guide to critical appraisal written by John Frank, Scientific Director at
the Canadian Institutes of Health Research. Although this article focuses on making
sense of media reports of new health research, the astute questions Frank poses can be
applied directly to research studies themselves.

Dr SA 2012

Online Critical Appraisal

Tools

CASP Critical Appraisal Tools

Available at: http://www.phru.nhs.uk/Pages/PHD/resources.htm
The UK-based Critical Appraisal Skills Programme (CASP), a division of
the NHSs Public Health Resources Unit, has developed a
customized critical appraisal checklist for each common type of
research study. Direct links to each of the 6 appraisal tools are
listed below:
Qualitative Research
http://www.chsrf.ca/kte_docs/casp_qualitative_tool.pdf
Review Articles (including Systematic Reviews)
http://www.phru.nhs.uk/Doc_Links/S.Reviews%20Appraisal
%20Tool.pdf
Case Control Studies
http://www.phru.nhs.uk/Doc_Links/Case%20Control
%2011%20Questions.pdf

Dr SA 2012

Thanks

Dr SA 2012