PSY202 Advanced Qualitative

Research and Mixed-Methods

Research
Lecture 11: Proposal
Templates;
Sampling and
Analyses in MixedMethods Research

Lecture Overview
Readings: Creswell & Clark Ch. 3 and 6
Creswell, J.W., Shope, R., Plano Clark, V.L. & Green, D.O. (2006).
How interpretive qualitative research extends mixed methods research.
Research in Schools, 13, 1-11.
Onweugbuzie, A.J. & Collins, K.M.T. (2007). A typology of mixed
methods sampling designs in social science research. The Qualitative
Report, 12, 281-316.
Onweugbuzie, A.J. & Leech, N.L. (2004). Enhancing the
interpretation of significant findings: The role of mixed-methods
research. The Qualitative Report, 9, 770-792.

Framing your mixed-methods study how you write and present a

mixed-methods study sets the stage for your proposal.
Title, phrasing, presentation and write-up of mixed-methods
research proposals different from that of single-method approaches.
Sampling and analyses for your studies.

Titles, Research Questions, Hypotheses

/ Qualitative Interview Questions
Research
Title

Research
Question
s

Hypothe
ses,
Qualitati
ve
Question
s

Bro
Speci
ad
Be careful not to confuse all of thesefic
this
problem is widespread at the thesis level
as well (especially confusing research
questions with hypotheses)

Example Quantitative and Qualitative Titles

Quantitative Titles
The primary
variables
The participants
The site
Quantitative signal
words

Self-esteem, academic
performance, parental
support
High school students
Malaysian public
school
Relationship,
interaction
(moderating effect)

A study of the relationship between Malaysian

high school students self-esteem and academic
performance: The moderating role of parental
support.

Example Quantitative and Qualitative Titles

Qualitative Titles
The central
phenomenon
The participants
The site
The type of
qualitative research

Refugee experiences
Vietnamese refugees
Refugee camp in
Malaysia
Phenomenology

A phenomenological study of the lived

experiences of Vietnamese refugees in a
Malaysian refugee camp.

Good Mixed Methods Titles

Mixed-Method
Titles
Short and succinct
Includes major topic,
participants and site
Include the term
mixed methods
Neutral (neither too
quantitative or too
qualitative)

Experiences of stress /
burnout
Clinicians
Private hospital, Malaysia
Mixed-methods /
multimethod / qualitative
and quantitative

A mixed-methods study of the experiences of

stress and burnout in clinicians from a Malaysian
private hospital

Describing a Design in a Written Report

It is important that in your introduction/overview
(see Assignment 2 marking scheme) that you
clearly highlight your mixed-methods
design.
Include in the introduction/overview:
The type of mixed methods design.
Defining characteristics of this design.
The rationale for using the design.
Important! Justify, justify, justify!

References on the design from the mixed methods

literature.
See references posted up on eLearning.

Mixed Methods Purpose Statement

Mixed-methods
Purpose
Statements
The overall
intent of the
study
Type of design
and its definition
Specific
qualitative and
quantitative
purpose
statements
The reason for

The overview explains the

design of the mixedmethods study, justifies it
well, and explains why it is
the optimal approach to
answering research
compared with singlemethod approaches.

Example Purpose Statement

Script
This mixed methods study will address
[overall content aim]. A [mixed-methods
design] mixed methods design will be used,
and it is a type of design in which the
collection and analysis of
[qualitative/quantitative] data is
conducted before the collection and analysis
of the [quantitative/qualitative] data
before they are then merged.
If sequential exploratory: In this study,
[qualitative data] will be used to first
[purpose of qualitative study]. This will
involve [participants, site, method] The
second quantitative study will be used to
[purpose of quantitative study]. This will
involve [participants, site, method].

The intent
The design

The
qualitative
purpose
statement,
The
details
quantitative
purpose
statement,
details

Framing the Overview: A

Template
Design
A Need Exists in the Research to
Explanat
ory

not only obtain quantitative results, but to

explain such results in more detail to
accommodate for the subjective differences in
the topic area of interest and thus add to a more
holistic, nuanced understanding of the
phenomena of interest.

Explorat
ory

explore a topic because variables are

unknown and to (later) assess the extent to
which such exploratory results generalize to a
wider population / to statistically validate the
implied relationships/associations between
concepts from the qualitative study.

Quantitative Study: Example Quantitative Purpose Statement

Quantitative
Purpose
Statements
Variables
Participants
Research site
Quantitative
method
Words that
connect
variables
Directional
language

The central research question for this

quantitative study is to

Examine the relationship between

student self-esteem and academic
performance in a Malaysian school.

Investigate whether parental support

impacts the effects of student self-esteem
on their academic performance in a
Malaysian school.

If phrased as a question, it might sound

like: What is the relationship
between student self-esteem and
their academic performance?

Qualitative Study: Example Qualitative Purpose Statement

Qualitative
Purpose
Statements
Central
phenomenon
Participants
Research site
Qualitative
design
Action verbs
Nondirectional
stance

The central research question for

this qualitative study is to

Explore/understand the lived

experiences of Vietnamese
refugees living in a refugee camp in
Malaysia.

If phrased as a question, it might

sound like: How do Vietnamesee
refugees experience and make
sense of, their lives as refugees
in a Malaysian refugee camp?

Try it Now: Write Your Overview Using

the Template and Suggestions Provided

This mixed methods study will address [overall

content aim]. A [sequential exploratory]
mixed methods design will be used, and it is a
type of design in which the collection and
analysis of qualitative precedes the collection
and analysis of quantitative data before they are
then merged. In this study, [qualitative data]
will be used to first [purpose of qualitative
study]. This will involve [participants, site,
method]. The second quantitative study will be
used to [purpose of quantitative study]. This
will involve [participants, site, method].

Quantitative Study:
Hypotheses State clearly and explicitly the
Quantitative
Hypotheses
Relate variables in
a specific, detailed
manner.
Make predictions
when warranted by
the literature.
Include
references to
nature of
relationships in
statistical terms.

relationship between your

variables.
Be specific explain how one
variable is related to the other.
Use statistical terms such as:
Difference between one group
and another
Positively / negatively correlated
Positive / negative relationship
Mediate
Moderate

Too general: There is a relationship

between self-esteem and academic
achievement.
Better: There is a positive

Qualitative Study: Interview

Questions
Qualitative Research
Questions

Interview and
focus-group
questions
Open-ended
Begin with what
and how
Focus on a single
concept (i.e.
singular focus)

Refer to notes on structuring effective

interview/focus-group questions.
Questions to ask, questions to avoid.

Questions should be as open-ended as

possible, aimed at allowing respondents
to give broad descriptions of an
experience.
Too specific/quantitative: How does living as a
refugee affect your relationship with your
family?
Better: Tell me about your experiences as
a refugee here in Malaysia; Tell me about
your familys experiences here in
Malaysia.

Important: For Sequential Exploratory Designs

Remember for sequential exploratory designs

(qual-quant), you may not have hypotheses before you
conduct and analyze your qualitative data.

If this is the case, then broadly suggest that:

The hypotheses for the quantitative study will be

based on /derived from the analysis of the
qualitative data. Categories/themes from the
analysis will be used to construct hypotheses in
the quantitative study.

You can provide some (justified and reasonable) expectations of what

categories/themes may emerge from your qualitative study.
These categories/themes may subsequently be measurable
variables in the quantitative study.

Mixing and Merging

The challenge with this assignment is to be able to

propose both quantitative and qualitative
studies that draw on the strengths of both
approaches, but at the same time, merging them in
a complementary program of research.

Explanatory In what way does the qualitative

data help in explaining the quantitative results?

Exploratory In what way does the quantitative

data help in generalizing the qualitative
results?

Mixed Methods Research Questions

and Justifications Can Be Written In
Different Ways
Method Focus
Follows
pragmatic
underpinning
What is the
value of using
both
quantitative
and qualitative
methods in this
research
program?
What is the
strength and
limitation of
the quantitative
and qualitative
components of
the study?

Content Focus

How does the

mixed-methods
study answer
the research
question and
add to the
understanding of
the phenomena
of interest?

Combination
Mixed-methods
as justified in
use because it
answers both
the method
and content.
Try and justify
your mixedmethods
proposal using
this
combination.

Break

Sampling Procedures:
Quantitative
Quantitative Data
Collection
Identify the site.
Identify the
participants.
Select a large
sample based on
power analyses.
Identify the
probabilistic or
non-probabilistic
sampling strategy
Discuss

Aim is to enhance the

representativeness (either
statistical or analytical) of
the research question from
the sample.
Simple Random, Stratified
Random, Cluster,
Systematic Random
(every kth participant).
Recruitment may involve
attaining gatekeeper
permissions.

Sampling Procedures:
Qualitative
Qualitative Data
Collection

Identify the site(s).

Identify the
participants.
Select a small
sample.
Identify the
purposeful
sampling strategy
and why it was
chosen.
Discuss
recruitment

Aim is to enhance the

representativeness
(analytical) of the
research question
from the sample.
Purposeful,
convenience,
intensity (esp. for
phenomenology),
extreme cases.

A Recommendation for Sample Size

Recommendations
Research
Design

Minimum Sample Suggestion

Reference

Correlational

64 participants for one-tailed

hypotheses; 82 for two-tailed
hypotheses

Onweugbuzie
et al., 2004

Experimental

21 participants per group for onetailed hypotheses

Onweugbuzie,
et al., 2004

Case Study

3-5 participants

Creswell, 2002

Phenomenolo
gy

10 interviews, 6 interviews

Creswell,
1998; Morse,
1994

Grounded
Theory

20-30

Creswell, 2007

Interview

Guest, Bunce
& Johnson,
12 participants; 20 for approaching
2006;
saturation.
Alexandersson

Sampling Considerations: Sequential Explanatory (Quant-Qual)

Designs

Decision
Considerations

Individuals in
Samples

Which Quant
results to
follow up?

Recommendations

Include individuals
from the Quant study
in the Qual Study
(Qual participants
can be a subset of
the earlier quant
study)
Significant/nonsignificant results;
outliers and
deviant cases.

Qua
l
Quant

Writing Up the Sampling Strategies in a Mixed-Methods Study

If you are planning to conduct a sequential

explanatory (Quant-qual) mixed-methods
study
What do you expect to uncover from the quant
study?
Because you have hypotheses, you can more freely
expect the pattern of results to go a certain way.
Who will your sample be? Are you expecting to
generalize statistically or analytically from your sample?
Who will participate in your follow-up qualitative
study? How will you recruit them?

Sampling Considerations: Sequential Exploratory (Qual-Quant)

Designs

Decision
Considerations

Individuals in
Samples

Which Qual
results to
follow up?

Recommendations

Large sample of
different individuals.
Quant participants
may not
necessarily be
from the qual
study.
Use themes,
codes, and quotes
to help design
instrument or

Qua
l

Quant

Writing Up the Sampling Strategies in a

Mixed-Methods Study: Sequential
Explanatory
If you are planning to conduct a sequential
exploratory (Qual-quant) mixed-methods study
What do you intend to do with the results from your
qualitative study?
Can the categories and themes you uncover from the
quantitative study be approximated to an established
psychological construct?
Who will participate in your follow-up quantitative
study? How will you recruit them?
A follow-up quant study may not necessarily use a random sample.

Collecting Information in Quantitative

and Qualitative Studies: Instruments
and Procedures
Quantitative Data
Scales and Measures
Close-ended questions
Instruments, measures,
scales.
Report on the validity
and reliabilities of the
measures you intend to
use (alpha values)

Qualitative Data
Questions
Open-ended
questions.
Interview questions,
procedures (protocol).
Report on the validity
of the qualitative
questions you used
perhaps run a pilot/trial
interview and adjust
before actual data
collection.

Writing Up Quantitative Data Analysis

Quantitative Data
Analysis
Choose statistical test.
Analyze the data to
answer the research
questions or test
hypotheses.
Report inferential tests,
effect sizes, and
confidence intervals.
Explain how the results
address the research
questions or
hypotheses.
Compare the results with
past literature,
theories, or prior

Given the [nature of

data, way in which
data was measured],
the data will be
measured by a series of
[statistical test]. This
is because
[justification]. The data
will first be [data
screening/cleaning
approaches].
To test the hypotheses, a
series of [statistical
test(s)] will be

Writing Up Qualitative Data Analysis

Qualitative Data
Code the
data.
Analysis
Assign labels to text
units.
Group codes into
themes (or categories).
Interrelate themes (or
categories) or abstract
to smaller set of
themes.
Assess how the
research questions
were answered
Reflect on the personal
meaning of the
findings.

Data from the

[qualitative method]
will be analyzed using
[data analysis
method]. This method
involves [explanation].
Data analysis will be
conducted during and
after the data collection
phases. This is because
[justification]. To
guard against
researcher bias, certain
approaches will be taken
to ensure the validity
and reliability of the

Validating the Data and

Results
Quantitative
Data Analysis
External
standards.
Check validity
and reliability of
measures from
past instrument
use.
Guard from
internal threats
to validity in
current study.

Qualitative Data
Use Analysis
researcher
and participant
standards
(internal
standards).
Use multiple
validation
strategies.
Enhance
reliability
through inter-

Discussion
Get into your tutorial groups, and use this time to ask any
final questions you may have about your mixed-methods
proposals. Please group yourselves together with your
respective tutors.
Next Week: No lecture. However, I will be in class as usual for
a final review/discussion with anyone needing further help
with their mixed-methods assignment.
Next Next week: Three minute research proposal
competition.
Sign up now if you wish to take part
1st place 5 bonus marks
2nd place 3 bonus marks
3rd place 2 bonus marks
Participation 1 bonus mark