CHAPTER 2

Research Designs
Chapter Summary
The research design process starts with determining the research questions. The
chapter opens with an excerpt from the Carrolls Alices Adventures in Wonderland
story. We see that some direction is required to start a journey. Whether it be an
actual destination or to simply explore and experience the journey, both are
directed in some way. Conducting social research can very much be seen as a
journey.
The intent of research may be to try to describe connections between variables,
exploring a social phenomenon, how it changes over time, the meanings that people
attach to things in their lives, to develop generalizations that apply to broad
categories of people. Further you need to consider the type of explanation you
would like to use.
Quantitative research relies on cause and effect based in laws and principles that
can be extrapolated to people both inside and outside of the study. This is referred
to as nomothetic explanation. To be acceptable it requires the presence of three causes
or conditions.
1. Correlation (An affected variable changes with the variable that causes it)
2. Time Order( The cause must occur prior to the effect)
3. Non-spurious (There is no alternate explanation for the correlation)
Qualitative research is typically ideographic. It tries to get deep understanding of the
perceptions and feelings of research subjects. Typically it only applies to localized
groups/activities in what is called an idiographic explanation. It addresses only the
activity of those involved in the research. The data is not meant to be extrapolated
to a large external social group.
Methods used in qualitative research range from non-invasive self-completed questionnaires through to participant observations which may entail infiltration of social
groups. Case studies for example examine a specific place, time, event, person,
community, organization. The researcher must decide how best to get access to data
that will explain the particular case.

Social Research Methods, Third Canadian Edition

Oxford University Press Canada, 2012

Causality
Causality in social research is directly related to variables. Variables quite simply are
characteristics or attributes of people or things that are different or change (vary)
within the population. Examples may include determined physiological attributes
such as gender, athletic ability, and/or subjective characteristics such as interest in a
particular topic, beliefs.
There are three key criteria for evaluating a research project:
1. Reliability Would the same results occur if the research was repeated again
with the same research subjects? This speaks to the impact of differing
locations, timing and the researcher on the outcome.
2. Replicability Would the same results occur if the research was repeated by
other researchers? This speaks to good research procedures.
3. Validity Is there integrity in the outcomes and conclusions? There are
three types of validity.
i) Measurement or construct validity
Are you actually measuring what you what to measure?
ii) Internal Validity
How confident can you be that the variables are co-related? Is the
secondary or dependent variable actually impacted by what you think
is the primary or independent variable.
Independent variable proposed cause (occurs first)
Dependent variable proposed effect (occurs second as a result of the
specified independent variable that occurred/existed before it)
iii) External Validity
Is the research design or setting too artificial? Do the study findings
relate to the real world that exists beyond the particular research
environment? Qualitative and quantitative methodologies use different
methods to overcome this concern.
Qualitative methodology incorporates naturalistic observations.
Quantitative methodology uses representative sampling to overcome
this concern and generalize the findings to extrapolate them to a larger
population.
Lincoln and Guba (1995) argue for different standards to judge qualitative methods.
They argue for the inclusion of trustworthiness. Trustworthiness, however, simply
includes four principles that mirror the measuring criterion for quantitative
methods.
Trustworthiness
Quantitative Criterion
Credibility
Internal Validity
Transferability
External validity
Dependability
Reliability
Confirmability
Replicability
Social Research Methods, Third Canadian Edition
Oxford University Press Canada, 2012

Experimental Design
Experimental design is common in psychology and organizational studies because of
the ability to manipulate independent variables to determine the influence on the
dependent variable. In sociology and political science the process of variable
manipulation is much more difficult, if not impossible to do. Many variables (i.e.,
poverty, war) have long-term issues that cannot be set up as experiments and ethical
concerns prevent experiments. See the Stanford Prison Experiment listed below.
Even where experimental design may seem operationally sound for qualitative
research, it will not get at the perceptions and feelings of the research subjects that
is a key component of qualitative research. The one may lead to the other. Further
there is a difference between laboratory and field experiments. Laboratory experiments occur in artificial settings (e.g., candid camera) whereas field experiments
take place in real-life surroundings. Because laboratory experiments take place in
artificial environments it is easier to control the research environment, easier to
randomly assign research subjects, which enhances internal validity, and it is easier
to replicate.
Experimental design requires some specific components to be valid:
A group that is essentially left alone to function as it did prior to the test is
required. It is called the control group. It provides a baseline to which
researchers will compare the experimental, or treatment, group.
The experimental (treatment) group receives some sort of treatment or manipulation.
Research subjects must be divided between the control and experimental
groups in a random fashion. This allows the researcher to
o presume the two groups are the same, relative to the independent
variable; and
o accept any change in the treatment group as an occurrence that is related
to the treatment received.
In the classic experimental design the independent and dependent variables are
identified. The dependent variable is observed or measured (pre-test) in each of the
control and treatment groups and recorded as T1 (time 1). The treatment group
receives the treatment/manipulation while the control group is left alone. The
dependent variable is observed or measured (post-test) in each of the control and
treatment groups and recoded as occurring at T2 (time 2). Any changes in each
group are noted. Ideally change will only occur in the treatment group. See
Rosenthal and Jacobson below.
True experimental evidence would eliminate all other possible (rival) explanations
for the change in the dependent variable in the treatment group. But most social
experience involves several and complex issues. So the use of a control group, and
the random assignment of research subjects between the control and treatment
groups, is aimed at increasing internal validity.
Social Research Methods, Third Canadian Edition
Oxford University Press Canada, 2012

Cook and Campbell (1979) identify six threats to internal validity:

1. History Events that occur other than manipulation of the independent
variable. The presence and measuring of the control group reduces the impact
of this. Both the control group and treatment group should be affected
equally.
2. Testing The experience of the pre-test increase the research subjects ability
when faced with a post-test. The presence and measuring of the control group
reduces the impact of this. Both the control group and treatment group
should be affected equally.
3. Instrumentation There is a change in the way the pre-test and post-test
are administered. The presence and measuring of the control group reduces
the impact of this. Both the control group and treatment group should be
affected equally.
4. Mortality Research subjects leave the experiment early. The presence and
measuring of the control group reduces the impact of this. Both the control
group and treatment group should be affected equally.
5. Maturation Research subjects change over time. The presence and
measuring of the control group reduces the impact of this. Both the control
group and treatment group should be affected equally.
6. Selection There may be pre-existing differences between the control group
and the treatment group. The use of random assignment to each of the
groups reduces the impact of this. However, this may not work with small
groups.
Further, Cook and Campbell (1979) identify five threats to external validity.
1. Interaction of selection and treatment The findings may be specific to
the research subject and therefore not be generalizable to a wide variety of
people who were not in the experiment. See Rosenthal and Jacobson below.
2. Interaction of setting and treatment The findings may not apply to
settings and environments that differ from those of the experiment. Is it
reasonable to have the same expectations of people who exist in different
settings? Would Rosenthal and Jacobsons findings apply outside of a school
setting?
3. Interaction of history and treatment The findings may not apply to
other time periods, either in the past or in the future.
4. Interaction effect of pretesting The findings may not apply to people
who were not pretested, and few people in society are pre-tested.
5. Reactive effects of experimental arrangements The findings may be
invalid because they were caused by subjects behaving atypically due to the
fact that they were in an experimental situation.

Social Research Methods, Third Canadian Edition

Oxford University Press Canada, 2012

Quasi-experimental Design
Quasi-experiments have some characteristics of the experimental model but lack
one or more of the internal validity requirements. Natural experiments for example
occur when there some change to a group by people who are not researchers, but
there is data available for researchers to collect and analyse for its effect.
Cross-sectional Design
Cross-sectional design is the gathering of data at a specific point in time. There is no
manipulation of the independent variable or treatment group. The data gathered is
meaningful because two or more measurable variables are compared for patterns
of association. Methods used may include questionnaires, structured interviewing,
and structured observation. This design tends to be used in quantitative studies
because the large sample size provides confidence that the suggested correlation
between identified variables occurs regardless of other potential influences and
because statistical techniques typically need large sample sizes.
Because there is no before and after timeline to assess cause and effect, the crosssectional design tends only toward providing patterns of association rather than
causality. Did one variable cause the other to occur, or is the impact of one variable
to another reciprocal?
The issues of reliability, replicability, and validity remain the same for crosssectional design. Are the measure tools appropriate? Are the procedures described
fully? How are the perceived associations substantiated? Perhaps causation is
substantiated through the use of a secondary research method. Was the sample used
a random one?
There is often no ability to manipulate variables in social research, e.g., gender, age,
racial background, ethnicity. That does not mean that causal inferences are not
possible. Rather, the comparison of a large sample size of people of different ethnic
backgrounds may lead to a strong indicator that the dependent variable is impacted
by ethnicity. Likewise the cross-sectional research approach may allow researchers
to rule out the impact of a given pre-existing variable.
The cross-sectional design is used in both quantitative and qualitative inquiry. In
qualitative research the researcher collects in-depth data on what influences
peoples activities. In-depth data on what people believe influences their behaviour
is a discussion of patterns of association. The sense of influence as perceived by
the research subject, by its very nature, implies causality.

Social Research Methods, Third Canadian Edition

Oxford University Press Canada, 2012

Longitudinal Design
Data is collected at a particular time (T1) just as in the cross-sectional research
design. But it is also gathered again at a later time, and perhaps at even further
times (T2, T3, . . .). This process may occur several times. Although there is no
manipulation of an independent variable, the longitudinal design overcomes the
direction of causation problem found in cross-sectional design. It provides insight
into the time order of change in variables to determine, to some degree, which
variables are dependent and which are independent. The additional time and cost
required to conduct longitudinal design projects hinders the use of this methodology.
There are two types of longitudinal design: the panel study and the cohort study.
The panel study relies on the same set of research subjects. They are studied at time
one (T1) and again at a later time, and perhaps at even further times (T2, T3, . . .).
The cohort study examines an experience that people have and then again at a later
time, studies the same experience, although the research later subjects may be, and
most often are, different from the first group. Both study designs share the same
weaknesses:
Attrition (Panel study) Some research subjects may not be available for
subsequent study. This reduces the representativeness of your sample and
thereby the reliability of your data somewhat. The cause for their unavailability may be directly related to a variable not just a random occurrence.
The researcher simply doesnt know.
Timing When is the appropriate time to do further data collection?
Panel Conditioning Being engaged in a panel study (observed) over an
extended period of time may become a variable in the activity of the research
subjects.
Case Study Design
A case study is quite simply the in-depth study of a single case. The case may be an
individual, a family, an organization, an event, a geographic location, a specific
time, etc. The case study is not aimed at achieving external validity. It provides an
in-depth description of a particular set of circumstances.
The term case study may be seen as an overarching research style. It identifies the
individual incidence of activity to be studied as an object of interest in its own right.
The particular method used to study that particular object of interest is determined
by assessing what methodology will be most appropriate given the case scenario.
The methods used may be quantitative or qualitative. Qualitative methods used for
a case study tend to take an inductive approach whereas quantitative methods tend
to take a deductive approach.
Social Research Methods, Third Canadian Edition
Oxford University Press Canada, 2012

There are three types of case study:

1. Critical Case A hypothesis is established and the critical case study is used
to understand the conditions that support or detract from that hypothesis.
2. Extreme (unique) Case Exposes the unusual case which provides comparison data to help understand the more common ones.
3. Revelatory Case An exposition of data that has not be observed or studied
previously
There are inherent strengths and weaknesses to the use of a multiple-case study
approach. Comparison of similarities and differences is similar to the experimental
and quasi-experimental models. Highlighting those similarities or differences may
assist in assess or generating theories.
The key difficulties in using a multiple-case approach rests in the need to focus on
ways to contrast the cases. Narrowing the research in order to compare a set of two
or more cases is necessary, to establish what facets of the cases are comparable.
This subordinates the focus on the specific details of any one specific case.
Ultimately, the multiple-case study approach establishes a focus of study right at the
beginning. Establishing a focus reduces the ability to utilize an open-ended approach
which one would hope for in a case study. The open-ended approach is valuable
because it provides for a more intensive examination and broader contextualization
of the collected data.
Multiple Choice Questions
1. Developing generalizations (laws and principles) that apply to broad categories of
people is typically part of ________.
a) quantitative research methodology
b) qualitative research methodology
2. Nomothetic (cause and effect) explanation of social activity is ________.
a) typically used in a qualitative research methodology
b) is not concerned with the relationship between when one activity occurs
and another occurs
c) incorporates the possibility of alternate explanations
d) requires correlation
3. Self-completed questionnaires may be used in ________.
a) qualitative research methods
b) quantitative research methods
c) both qualitative and quantitative methods
d) They do not provide valid data for either type of research

Social Research Methods, Third Canadian Edition

Oxford University Press Canada, 2012

4. Ideolographic explanation ________.

a) is qualitative in nature
b) applies to broad populations outside of the research group
c) is concerned with proving correlations between data
d) requires a social group of five or more people to be valid
5. Which of the following is NOT usually considered a qualitative research method?
a) case studies
b) participant observations
c) surveys
d) statistical analysis
6. Qualitative research ________.
a) is aimed at extrapolation of data to a broad population
b) relies only on case studies for collecting data
c) is the best form of social research
d) focuses on collection of in-depth data related to particular social activity
7. Causality in social research ________.
a) can only be determined through statistical analysis
b) is directly related to analysis of variables
c) is not address in any qualitative approach to research
d) does not include determined physiological attributes such as gender
8. Reliability ________.
a) is a key criteria for evaluating research projects
b) is a measure of causality
c) confirms that a research project is quantitative in design
d) is the reason one would choose to use the classic experimental design
9. With regard to causality, ________.
a) the dependent variable comes before the independent variable
b) it is the only measure of correlation between two variables
c) the independent variable comes before the dependent variable
d) it is the primary reason for conducting qualitative research
10. Understanding the perceptions and feelings of research subjects is a key
component of ________.
a) quantitative methods
b) qualitative methods
11. With regard to quantitative and qualitative research, ________.
a) they use different research methods
b) they are mutually exclusive categories of research design
c) they both require a control group
d) they have different criteria for evaluation
Social Research Methods, Third Canadian Edition
Oxford University Press Canada, 2012

12. Which of the following is NOT considered a threat to the internal validity of
research data?
a) history
b) pre-testing may increase the research subjects skills for the post-test
c) a change in the way a test is conducted
d) random assignment of the research subject to control and treatment
groups
e) research subjects get older
13. Cross-sectional design ________.
a) is the gathering of data at three
b) proves a direct link of causality
c) involves the manipulation of variables at one specific time
d) provides patterns of association
14. Longitudinal design ________.
a) overcomes the direction of causation problem found in cross-sectional
design
b) is considered one of the easiest forms of research design
c) requires data to be collected at least three times
d) may suffer from research subject attrition
15. A case study ________.
a) could be used to support an hypothesis
b) could be used to critique the conclusion drawn from research gathered
through a classic experimental design
c) may provide comparative data from an extreme set of circumstances to
better understand common cases
d) all of the above
True or False Questions
1. Although some say that good research requires some sort of predetermined
direction, it is not always necessary.
True ____
False ____
2. Testing for correlation of variables is the only valid qualitative reason for
conducting social research.
True ____
False ____
3. Internal validity is a confirmation that the dependent variable and the
independent variable are co-related.
True ____
False ____
Social Research Methods, Third Canadian Edition
Oxford University Press Canada, 2012

4. The measuring standard of trustworthiness used in qualitative methods mirrors

the measuring criterion for quantitative methods.
True ____
False ____
5. Experimental design is commonly used in qualitative research.
True ____
False ____
6. Experimental design involves the manipulation of variables
True ____
False ____
7. The experimental design presumes the sample groups are the same, relative to
the independent variable.
True ____
False ____
8. The classic experimental design requires a pre-test and post-test.
True ____
False ____
9. External validity may be negatively impacted if the selection of research subjects
or the setting of the experiment differs from that of the larger population.
True ____
False ____
10. Quasi-experimental design is when researchers set up an experiment but it is
lacking a key component for the pre-test.
True ____
False ____
11. The issues of reliability, replicability, and validity are the same for cross-sectional
design as for experimental design.
True ____
False ____
12. Random sampling is a key component of validating cross-sectional research
methodology.
True ____
False ____
13. Cross-sectional design can be used in either qualitative or quantitative research.
True ____
False ____
14. Research into the population of a country could be a case study.
True ____
False ____
15. A case study research project may include data collection from multiple cases.
True ____
False ____

Social Research Methods, Third Canadian Edition

Oxford University Press Canada, 2012

Media Resources

For a short report of several research projects using different research designs on
the correlation between media and violence, and a comparison of their various
strengths and weaknesses, see
http://psychlotron.org.uk/resources/social/AQA_A2_aggression_mediaantinotes.pd
f
o Does the stated aim of the research affect the conclusion?
o How do you account for the difference in conclusions?

For a report of a medical science quasi-experimental design, see

http://bjp.rcpsych.org/content/178/40/s41.full
For a summary of research into the link between genetics and political
orientation, see http://ts-si.org/content/view/2937/994/
and
http://jayjoseph.net/yahoo_site_admin/assets/docs/Claims_and_Refutations_July_2
010_pdf.205163035.pdf
o What were the primary methods used in each study?
o Is the study a quantitative method?
o What is the impact of connecting biological research with social science
research?
o Do they fit together?
o What is the impact of the differences in methodology?
For a critique of this methodology, see
http://sites.lafayette.edu/suhaye/files/2010/01/plugin-critique_of_twin_studies__suhay__kalmoe__mcdermott_101007.pdf
and
http://jayjoseph.net/yahoo_site_admin/assets/docs/Claims_and_Refutations_July_2
010_pdf.205163035.pdf
o What premises do the critics rely to refute the findings of the earlier study?

See the Stanford Prison Experiment for ethical considerations.

http://www.prisonexp.org/
o Would this study be permissible today?
o Do the social gains outweigh the individual costs?

Social Research Methods, Third Canadian Edition

Oxford University Press Canada, 2012

See Jane Elliott A Classroom Divided

http://video.google.com/videoplay?docid=6189991712636113875
o Is there potential harm when applying experimental or quasi-experimental
design on children?
o If there is actual harm, do the gains outweigh that harm?

See Rosenthal and Jacobson

http://fcis.oise.utoronto.ca/~daniel_schugurensky/assignment1/1968rosenjacob.ht
ml
o What is the impact of economics and social class on the research subjects and
the research?
o Would Rosenthal and Jacobsons findings apply outside of a school setting?
o What is the impact of expectation on the part of the researcher?
o What is the impact of expectation on the part of the research subject
(teachers and students)?

Social Research Methods, Third Canadian Edition

Oxford University Press Canada, 2012

Answer Key
Multiple Choice Questions
1. a (p. 23)
2. d (p. 23)
3. c (p. 24)
4. a (p. 24)
5. d (p. 24)
6. d (p. 24)
7. b (p. 24)
8. a (p. 24)
9. c (p. 25)
10. b (p. 24)
11. d (p. 25)
12. d (p. 28)
13. d (p. 33)
14. a (p. 36)
15. d (p. 39)

True or False Questions

1. F (p. 23)
2. F (p. 23)
3. T (p. 25)
4. T (p. 25)
5. F (p. 26)
6. T (p. 26)
7. T (p. 27)
8. T (p. 27)
9. T (p. 29)
10. F (p. 30)
11. T (p. 35)
12. T (p. 35)
13. T (p. 36)
14. T (p. 38)
15. T (p. 39)

Social Research Methods, Third Canadian Edition

Oxford University Press Canada, 2012