1

METHODOLOGICAL CRITIQUE

Quantitative Versus Qualitative Research Methodologies:

A Methodological Critique

Alexandra Iggulden

87536141

ETEC 500 Section 65C

University of British Columbia

Dr. Janet McCracken

March 15, 2015

 2
METHODOLOGICAL CRITIQUE

Abstract

This methodological critique will analyze the quantitative methods used within Hamre &
Pianta’s (2005) article, “Can Instructional and Emotional Support in the First-Grade Classroom
Make a Difference for Children at Risk of School Failure?”, and the qualitative methods used
within Sleeter’s (2009) “Developing Teacher Epistemological Sophistication About
Multicultural Curriculum: A Case Study”. This paper will include a critique and comparison of
the effectiveness of the methods, procedures, and reporting styles used​ ​for each of the studies. I
will synthesize my understanding of the qualitative and quantitative methods in order to form my
own research design.

Part 1: Descriptive Analysis and Critique

Article Summaries

Hamre & Pianta’s (2005) causal-comparative study examines ways in which high-quality
emotional and instructional support from first-grade teachers can affect the risk of school failure
in 910 5-6 year old students. The authors state that at-risk children who are placed in highly
supportive classrooms show similar achievement scores and teacher-student relationships as
those of their low-risk peers.

Sleeter’s (2009) heuristic case study explores the effectiveness of the researcher’s teaching
methods in a Multicultural Curriculum Design graduate course in relation to a 2nd year graduate
student’s understanding and planning of curriculum within her own classroom. The authors
findings show that in order to increase student learning educators must provide, opportunities for
reflective discussions, safe environments for support of uncertainty, and inquiry style group
settings.

Analyses of Methods used in Quantitative Article

Hamre & Pianta’s (2005) study uses a causal-comparative research design. Participants were
recruited from hospitals and each had already taken part in the NICHD Study of Early Child
Care. Of the original 1364 children eligible for the study, 910 were said to have complete data
and were used as participants. An increased sample size and a participant group that is specific to
this study would greatly increase the reliability and validity of the findings. The authors
acknowledge this limitation within their study.

Grouping in Hamre & Pianta’s (2005) study was predetermined by demographic and functional
risk factors, as well as, the quality of classroom environments. Classroom environments were
coded in a single 3-hour observation completed by a trained observer using a 7-point scale.
However, extraneous variables that are uncontrollable within each of the classrooms may have
have affected the coding of the classroom during the single observation. In order to increase
reliability and validity researchers must increase the number of observations within classrooms.
 3
METHODOLOGICAL CRITIQUE

Two major outcomes within this study were examined. “Performance on an individually
administered, standardized achievement battery, and first-grade teacher ratings of conflict with
the student” (​Hamre & Pianta, 2005, p.37​). The use of a standardized test is adequate for this
study due to the large number of participants. The researchers increase the reliability of their
results by using ANCOVA models to determine whether extraneous variables from the
classroom environment has an effect of the results.

Analyses of Methods used in Qualitative Article

Sleeter’s (2009) research follows a heuristic case study design. Her objective is structured around
an explanatory research question that focuses on determining how her own teaching strategies
affect the students enrolled in her graduate level Multicultural Curriculum Design course. Sleeter
(2009) uses a purposeful qualitative sampling strategy to select a single participant, Ann, a
novice teacher who is a student in Sleeter’s graduate course. Choosing a single participant for her
case study allows Sleeter to focus on obtaining an information-rich effect of the phenomenon on
Ann. Although a single participant will provide Sleeter (2009) with a basic understanding of the
effectiveness of her teaching, if she chose to study multiple sub-cases she would gain a broader
understanding of the phenomenon that she is investigating. Similarly, multiple sub-cases would
have provided Sleeter with the opportunity to triangulate her results in this study against the
results seen in her sub-cases.

Over the 15 weeks of the study, Sleeter (2009) uses multiple qualitative data collection
techniques including, a personal journal, a record of Ann’s growth over time which is assessed
using a heuristic rubric, group discussions, unit development observations, and two observations
in Ann’s classroom. Multiple data collection tools enable Sleeter (2009) to build a detailed
picture of the effect that her teaching has on Ann. Multiple observations for a greater length of
time will provide Sleeter (2009) with a reliable picture of whether real learning has occurred or
not. Similarly, 15 weeks is not enough time to gain an understanding about whether the results
are directly related to her own instruction in the graduate course, Ann’s own intrinsic motivation
to learn about the subject matter, or simply Ann’s completion of necessary requirements for the
course. A study completed over a longer period of time would increase the reliability and validity
in Sleeter’s (2009) results. Sleeter (209) also conducted a 40-minute recorded interview at the
end of her study. However, in order for her findings from this interview to be reliable, Sleeter
(2009) must include direct quotations from the interview within her paper rather than just her
own interpretations of Ann’s responses.

Effective Comparison Synopsis

Hamre & Pianta’s (2005) and Sleeter’s (2009) studies have many similarities since both explore
the effectiveness of teaching strategies and environments on student learning. However, their
research methods and reporting styles differ. Aside from the differences described in the above
sections of this critique, both studies vary in how they report on literature and conclusions.
 4
METHODOLOGICAL CRITIQUE

Reporting of Literature

Hamre & Pianta’s (2005) includes a relevant literature review on multiple articles that help to
support both their hypothesis and findings. In contrast, Sleeter (2009) utilizes scholarly works to
support the tools and methodologies that she employs, but does not include consistent bodies of
work to support her findings. An in-depth literature review of similar studies would greatly
increase the reliability and validity of Sleeter’s (2009) findings.

Reporting of Conclusions

Both researchers state that their studies can be used to guide improvements in teaching
practices that lead to positive student development and learning. A significant difference in the
reporting of conclusions between Hamre & Pianta’s (2005) and Sleeter’s (2009) articles is that
Hamre & Pianta (2005) use data tables and visual supports to communicate their results and
conclusions while Sleeter (2009) does not. Sleeter (2009) also concludes her study by making
statements that generalize her results to her other students, however she did not facilitate a case
study that is generalizable to a greater population.

Part 2: Methodologies for Personal Research

These studies have provided me with many examples of what I must take into account when
designing my own educational research. It is clear that a strong central question is necessary for
both quantitative and qualitative approaches. How participants are selected, the accurate
reporting of data, and a clearly written literature review, are all crucial aspects of a quantitative
approach that can easily hinder the validity and reliability of a study. A qualitative approach
identifies a need for multiple data collection tools, including, detailed descriptions of multiple
observations, a participant structure that reflects the goal of the study, and a literature review
used to support the findings.

A qualitative approach with few quantitative aspects will be the most fitting design for my
research. I will have to include multiple observations of student work in order to determine the
effect that blogging has on the metacognitive processes in middle school students. I will also
conduct observations, interviews, and questionnaires in order to gain insight into student
motivation and psychological change in self throughout the blogging process.
 5
METHODOLOGICAL CRITIQUE

References

Hamre, B.K. , & Pianta, R.C. (2005). Can instructional and emotional support in the first-grade

classroom make a difference for children at risk of school failure?. ​Child Development​, ​76​,

pp. 949-967. In L.R. Gay, G.E. Mills, & P.W. Arisian, Educational research: Competencies

for analysis and application (10​th​ ed.) (pp. 33-50). Upper Saddle River, NJ: Merrill.

Sleeter, C. (2009). Developing teacher epistemological sophistication about multicultural

curriculum: A case study. ​Action in Teacher Education, 31​(1), pp. 3-13. In L.R. Gay, G.E.

Mills, & P.W. Arisian, Educational research: Competencies for analysis and application

(10​th​ ed.) (pp. 51-59). Upper Saddle River, NJ: Merrill.