RESEARCH DESIGN

What is Research Design?

A detailed outline of how an investigation

will take place.
A research design will typically include
how data is to be collected, what
instruments will be employed, how the
instruments will be used and the intended
means for analyzing data collected.

Reliability

Is the degree to which an assessment

tool produces stable and consistent results.
Reliability, like validity, is a way of
assessing the quality of the measurement
procedure used to collect data in a
dissertation.
In order for the results from a study to
be considered valid, the measurement
procedure must first be reliable.

Validity

Refers to how well a test measures what it is

purported to measure. In general, VALIDITY is
an indication of how sound your research is.
More specifically, validity applies to both the
design and the methods of your research.
Validity in data collection means that your
findings truly represent the phenomenon you
are claiming to measure.
Valid claims are solid claims

FACTORS JEOPARDIZING
INTERNAL AND
EXTERNAL VALIDITY

Internal validity

Refers specifically to whether an

experimental treatment/condition makes a
difference or not, and whether there is
sufficient evidence to support the claim.

Factors which jeopardize internal validity

History--the specific events which occur between the first

and second measurement.
Maturation--the processes within subjects which act as a
function of the passage of time. i.e. if the project lasts a few
years, most participants may improve their performance
regardless of treatment.
Testing--the effects of taking a test on the outcomes of
taking a second test.
Instrumentation--the changes in the instrument,
observers, or scorers which may produce changes in
outcomes.
Statistical regression--It is also known as regression to the
mean. This threat is caused by the selection of subjects on
the basis of extreme scores or characteristics. Give me forty
worst students and I guarantee that they will show
immediate improvement right after my treatment.

Selection of subjects--the biases which may result in selection

of comparison groups. Randomization (Random assignment)
of group membership is a counter-attack against this threat.
However, when the sample size is small, randomization may
lead to Simpson Paradox, which has been discussed in an
earlier lesson.
Experimental mortality--the loss of subjects. For example, in
a Web-based instruction project entitled Eruditio, it started with
161 subjects and only 95 of them completed the entire module.
Those who stayed in the project all the way to end may be more
motivated to learn and thus achieved higher performance.
Selection-maturation interaction--the selection of comparison
groups and maturation interacting which may lead to
confounding outcomes, and erroneous interpretation that the
treatment caused the effect.
John Henry effect--John Henry was a worker who
outperformed a machine under an experimental setting
because he was aware that his performance was compared with
that of a machine.

External validity

refers to the generalizability of the

treatment/condition outcomes.

Factors which jeopardize external validity

Reactive or interaction effect of testing--a pretest
might increase or decrease a subject's sensitivity or
responsiveness to the experimental variable. Indeed,
the effect of pretest to subsequent tests has been
empirically substantiated
Interaction effects of selection biases and the
experimental variable
Reactive effects of experimental arrangements--it
is difficult to generalize to non-experimental settings
if the effect was attributable to the experimental
arrangement of the research.
Multiple treatment interference--as multiple
treatments are given to the same subjects, it is
difficult to control for the effects of prior treatments.

FORMS OF RESEARCH

Basic Research - This research is

descriptive in nature and is used to
understand and explain a phenomenon.
This type of research is often conducted
for the sake of increasing and advancing
a knowledge base.
Applied Research - The purpose of this
research is to provide information that
can be used and applied in an effort to
help people understand and control their
environment. This type of research is
more prescriptive in nature and seeks to
offer potential solutions to problems.

Evaluation Research - The purpose of

evaluation research is to examine the
processes and outcomes associated with
a particular solution to a problem. The
research may be formative in that it
attempts to improve the intervention or
solution or it may be summative and
attempt to evaluate the effectiveness of
solution or program.
Action Research - This research is often
conducted within a program, organization
or community and the researchers are
involved in gathering data and studying
themselves.

PRE-EXPERIMENTAL
DESIGNS

Pre-experiments

are the simplest form of research design.

In a pre-experiment either a single group or
multiple groups are observed subsequent to
some agent or treatment presumed to
cause change.
Types of Pre-Experimental Design
One-shot case study design
One-group pretest-posttest design
Static-group comparison

TRUE EXPERIMENTAL
DESIGNS

A true experiment

is a type of experimental design and is

used to establish cause and effect
relationships. There are three criteria that
must be met in order for an experiment to
be determined as a true experiment:
At least one experimental and control
group. Research-manipulated variable

QUASI-EXPERIMENTAL
DESIGNS

A quasi-experiment
is an empirical study used to estimate
the causal impact of an intervention on its
target population.
Quasi-experimental research shares
similarities with the traditional experimental
design or randomized controlled trial, but
they specifically lack the element of
random assignment to treatment or control.
Instead, quasi-experimental designs
typically allow the researcher to control the
assignment to the treatment condition, but
using some criterion other than random
assignment

SELECTING A STUDY
DESIGNS

Abstract
Various epidemiological study designs are
available to investigate illness and injury risks
related to workplace exposures. The choice of
study design to address a particular research
question will be guided by the nature of the health
outcome under study, its presumed relation to
workplace exposures, and feasibility constraints.
This review summarizes the relative advantages
and limitations of conventional study designs
including cohort studies, crosssectional studies,
repeated measures studies, casecontrol (industry
and communitybased) studies, and more recently
developed variants of the nested casecontrol
design: casecohort and casecrossover studies.

EVALUATION

Evaluation research

can be defined as a type of study that

uses standard social research methods for
evaluative purposes, as a specific research
methodology, and as an assessment
process that employs special techniques
unique to the evaluation of social programs.