Evaluating Research Articles: Qualitative

Designs
Sarah Hanson
In this assignment you will have the opportunity to read a qualitative
study and practice your skills at evaluating its methodological aspects.
Directions:
Read the article A Qualitative Study of Leadership Characteristics
Among Women Who Catalyze Positive Community Change, focusing
on the methods, results, and discussion sections, which you will be
asked to evaluate below.
For each evaluation question, rate the degree to which the information
provided in the research report satisfactorily addresses the question on
a 5-point Likert-type scale. In addition, justify each of your ratings by
referring to specific things
mentioned or not mentioned in the research report that formed the
basis of your rating.
Scale:

1
3
5

Poorly addressed
Partially addressed
Extremely well addressed

1. Do you think the research design used in this study was well suited
for addressing the type of question(s) being asked?
Rating:

Justification: The research design for this study is a grounded theory

approach. This is the most appropriate approach to use to answer the
questions being asked because the researchers are trying to connect
leadership skills with successful organizations. The definition of
grounded theory according to Understanding Research: A Consumers
Guide is an explanation that is developed about a process, action, or
interaction through the collection and analysis of qualitative data so
that the explanation is built from (or grounded in) the experiences
and perspectives of participants (p. 297).

2. Was the type of sampling used appropriate for the questions

investigated and the type of research design used?
Rating:

Justification: The researchers used purposeful sampling for this study.

The participants that were chosen were the best candidates to explore
the central phenomenon. All candidates were female, over the age of
21, and a leader in an organization related to public health.
Throughout the research paper, there were comments alluding to
male versus female leadership roles. In the limitations and
implications section of the paper, the researchers said, this research
was designed to elucidate womens leadership characteristics, but not
to compare and contrast them with mens (p. 11). However, I
question why the comparisons were made throughout the paper. I feel
that these comments confused the purpose of the paper.
3. Did the authors convince you that the type of analyses performed
was an appropriate approach to take?
Rating:

Justification: The researchers followed the appropriate protocol when

analyzing data: prepared the data with transcription verbatim,
exploring the data, and coding the data to develop themes along with
the NVivo qualitative data analysis software. However, the
researchers did not use a method to validate their findings. No one
outside the study audited the analysis and participants were not
asked to review the results. The researchers could have elicited more
steps to ensure that the results of the research were valid.
4. Do the researchers convince you that the study is credible,
according to the criteria below?
Do the researchers indicate their background, interests, and
possible biases?
Is there evidence of triangulation?
Were field notes collected?
Were multiple types of data and/or data sources used?
Was member checking employed?
Rating:

Justification: The researchers did not provide background information

or interests in this study. There is mention of a possible bias in the
study based on the fact that the authors are women and this could
have potentially skewed how the participants responded to their
questions. There is evidence of triangulation in this study. The
researchers connect leadership their results on leadership
characteristics with existing literature. There was no mention of field
notes being collected outside of the interviews. The only source of
data were the sixteen participants all interviewed in person or via
telephone. Member checking was not employed after the analysis;
only before the interviews took place.
5. Did the researchers convince you that their conclusions follow from
their data?
Rating:

Justification: The only part of the conclusion that follows from their
data is including a leadership component may greatly enhance future
interventions (p. 11). Although I am sure the women interviewed for
this study are making a difference for so many people with improving
public health, this was not the main focus or their data. The data
focused on leadership characteristics, motivation, success, and
mentors.