Vol 16 No 8 - August 2017

IJLTER.
ORG
International Journal
of
Learning, Teaching
And
Educational Research
p-ISSN: 1694-2493
e-ISSN: 1694-2116
Vol.16 No.8
PUBLISHER International Journal of Learning, Teaching and
London Consulting Ltd Educational Research
District of Flacq
Republic of Mauritius
www.ijlter.org The International Journal of Learning, Teaching
and Educational Research is an open-access
Chief Editor journal which has been established for the dis-
Dr. Antonio Silva Sprock, Universidad Central de semination of state-of-the-art knowledge in the
Venezuela, Venezuela, Bolivarian Republic of field of education, learning and teaching. IJLTER
welcomes research articles from academics, ed-
Editorial Board
Prof. Cecilia Junio Sabio ucators, teachers, trainers and other practition-
Prof. Judith Serah K. Achoka ers on all aspects of education to publish high
Prof. Mojeed Kolawole Akinsola quality peer-reviewed papers. Papers for publi-
Dr Jonathan Glazzard cation in the International Journal of Learning,
Dr Marius Costel Esi Teaching and Educational Research are selected
Dr Katarzyna Peoples through precise peer-review to ensure quality,
Dr Christopher David Thompson
originality, appropriateness, significance and
Dr Arif Sikander
Dr Jelena Zascerinska readability. Authors are solicited to contribute
Dr Gabor Kiss to this journal by submitting articles that illus-
Dr Trish Julie Rooney trate research results, projects, original surveys
Dr Esteban Vzquez-Cano and case studies that describe significant ad-
Dr Barry Chametzky vances in the fields of education, training, e-
Dr Giorgio Poletti learning, etc. Authors are invited to submit pa-
Dr Chi Man Tsui
pers to this journal through the ONLINE submis-
Dr Alexander Franco
Dr Habil Beata Stachowiak sion system. Submissions must be original and
Dr Afsaneh Sharif should not have been published previously or
Dr Ronel Callaghan be under consideration for publication while
Dr Haim Shaked being evaluated by IJLTER.
Dr Edith Uzoma Umeh
Dr Amel Thafer Alshehry
Dr Gail Dianna Caruth
Dr Menelaos Emmanouel Sarris
Dr Anabelie Villa Valdez
Dr zcan zyurt
Assistant Professor Dr Selma Kara
Associate Professor Dr Habila Elisha Zuya
VOLUME 16 NUMBER 8 August 2017
Table of Contents
Learning from each other: Dialogical Argumentation in an Online Environment ........................................................ 1
Anita Chadha and Rene B. Van Vechten
A Qualitative Examination of Factors for Success in a Content-Based English Language Learner Classroom ....... 18
Janet Delgado, Ed.D and Lorraine T. Benuto , Ph.D.
Radio Wave Errors: Students Mistaking Radio Transverse Electromagnetic Light Waves as Longitudinal Sound
Waves ..................................................................................................................................................................................... 37
A. E. Tabor-Morris, Ph.D., T. M. Briles, Ed.D. and R. Schiele, B.S.
Impact of Teaching Attitudes and Behaviors for Learning on the Reading Achievement of Students Falling
Behind .................................................................................................................................................................................... 51
Michael E. Bernard
Balancing Reflection and Validity in Health Profession Students Self-Assessment ................................................... 65

Sherri Melrose
Innovative Teaching with Use of an Art Work ................................................................................................................. 77

Marios Koutsoukos and Iosif Fragoulis
1
International Journal of Learning, Teaching and Educational Research

Vol. 16, No. 8, pp. 1-17, August 2017
Learning from each other: Dialogical

Argumentation in an Online Environment
Anita Chadha
University of Houston
Downtown, Houston, Texas, U.S.
Rene B. Van Vechten

University of Redlands
Redlands, California, U.S.
Abstract. This research builds upon past work exploring how an online
academic website can provide a learning environment in which students
engage in dialogic argumentation by voicing their diverse perspectives,
challenging their peers through counterarguments, and articulating
their positional differences. Drawing from two semesters of data from
an academic website populated by three classes, we analyze 375 peer-to-
peer responses for their argumentative interactions. Using a mixed
methods approach, we find statistically significant evidence that
argumentative interactions lead to deeper engagement across the classes.
This study concludes that online discussionsa form of computer
mediated communication (CMC)are an innovative means to advance
e-learning, a concern for educators across disciplines.
Keywords: Argumentation; Online deliberation; Online teaching;

Dialogic argumentation; Computer-mediated communication
Introduction: Learning from each other: Dialogical Argumentation in

an Online Environment
Online learning environments now proliferate in our digital age and researchers
have observed that in online, networked environments, learning can occur
through an egalitarian process in which participants generate, challenge, reflect
upon, and defend ideas, thereby constructing meaning through a social process
(Rowntree, 1995; see also Chu et al., 2017; Cooper, 2001; Gordon & Connor, 2001;
Wilson 2001). Also known as computer-mediated communication (CMC), web-
based, interactive technologies are particularly well-suited to creative
collaboration among active participants (Lee & McLoughlin, 2007). The CMC
environment influences interaction due in part to visual anonymity and the
absence of nonverbal cues. As Herring (1993) argues they provide for the
possibility that individuals can participate on the same terms as others, that is,
more or less anonymously, with the emphasis being on the content, rather than
on the form of the message or the identity of the sender (p. 1). With greater
focus on the written message produced through asynchronous means, students
2017 The authors and IJLTER.ORG. All rights reserved.

2
can participate in discussions on the same terms as others, without respect to

geographic distances (Lane, 1994) or due to personal disabilities (Collins, 2014;
Lane, 1994). Relative anonymity can also encourage users to be more expressive
and form relationships with others rapidly (Schouten et al., 2009).
Researchers have demonstrated that learning through CMC transpires through
an individual process of critical reflection, a process of testing ones ideas while
being challenged, reconsidering ones experiences and ideas in light of new
information, and then reconciling differences. The argumentative process also
involves synthesizing information and anticipating and responding to
opposition, all of which are particularly conducive to learning (Jacoby, 2009;
Blount, 2006; Bloom et al., 1956). In short, communicators learn through arguing
with each other (Dehler & Porras-Hernandez, 1998), and dialoguing in ways that
contains elements of argumentation also represents an opportunity to learn
actively (Bender, 2003). As Socrates might have asserted, active argumentation
channels learning.
In this paper, we investigate online discussion forums created for and by
undergraduate students enrolled in American Politics courses from three
campuses, assessing their interactions for patterns of dialogical argumentation.
In the experiment, students were given a weekly prompt about a contemporary
issue in American politics, and participants created individual statements that,
inevitably, reflected various levels of intellectual engagement with the material.
From generalizations to fairly thoughtful and well-constructed essay-like
answers that evidence deep, critical reflection, the content of those discussion
posts provided the data for our study. First we identify various forms of
interaction, and present a model for analyzing the content of those website
discussion posts, testing whether students engage the learning process when
they argue with each other in online discussion forums.
Literature Review
Argumentation, according to Toulmin (1958), is a process whereby an individual
or group, wanting to be taken seriously, tries to convince the others that the
assertions being made are acceptable, meritorious, or valid, and there is
abundant evidence that this argumentative process has great worth as a learning
tool (Clark & Sampson 2008; Schwarz & DeGroot, 2007; Clark & Sampson 2007).
Through it, students understanding of challenging concepts can increase
(Andriessen et al., 2013) and their ability to reason productively also can
improve (Kuhn et al., 1997; Kuhn & Udell, 2003; Bell, 2004).
Argumentation sets the scene for changes in peoples views because of the
knowledge building and transformation of ideas that can occur through this
process, leading to learning. This is a process of critical reasoning, and at its core
is the idea of change in thinking. Change occurs because the arguer convinces, or
a respondent critically reflects upon an idea and updates or refines an existing
concept or belief. In any case, argumentation involves opposition, a process that
some have characterized as occurring within a dialectic, whereby a position
statement is made and justification is given, a counterargument is made and claims
are questioned or examined, and a reply or rebuttal to the counterargument is
supplied the dispute may ultimately be resolved into a conclusion (Toulmin 1958;
Clark & Sampson 2007; 2008).

3
Arguments include supports such as warrants, backing, and qualifiers. In an

online setting where students confront a discussion question and puzzle through
its implications, the process generally parallels basic argumentative phases
described by Toulmin (1958): online, they might might raise questions, challenge
a premise, add new information, and/or anticipate responses. In essence,
through dialoguing with each other via e-collaboration, students have the
opportunity to engage in a process of argumentation that enables learning.
In dialogical argumentation, these being arguments carried out through written
or verbal dialogue, participants negotiate their divergences and reconstruct their
perspectives in a social context. In other words, arguments by participants are
sensitive, as Leito (2000) argues, to specific demands of argumentative
situations (p. 336). In this way, macro meets micro, as macro-level factors
influence what is essentially a micro-level process of decision making. Hakkinen
(2013) points out this interactive relationship: these processes are
intertwinedin a way that is not reducible to one level only (p. 550). For
example, a person might respond in a certain way because of how s/he has
internalized shared norms about proper conduct, or the collective understanding
about the purposes of an argument (Resnick et al., 1993). Likewise, personal
attitudes such as openness to change or expectations about compromise (Coirier
& Golder, 1993), as well as personal characteristics such as race, can influence
how arguments unfold. Measurable change, therefore, takes place at the
individual level as well as the social or group level. These changes might be of
any magnitude; wholesale change is not required for an argument to be
successful. As Leito (2000) points out, in a discussion in which opposing views
are justified and recognized, shifts in perspective occur across a continuum,
ranging from subtle (qualifying a position) to complete reversal in stance.
Much research on online argumentation draws on Toulmins initial work on
argumentation (1958). For example, Clark & Sampson (2007) note that analytic
frameworks focus on many different aspects of argumentation including
argument structure, epistemic types of reasoning, conceptual normativity,
quality of warrants, number of warrants, logical coherence of claims with
warrants, argumentation sequences, patterns of participation, conceptual
trajectories, and the process of consensus building which can be applied across
disciplines. (p. 275).1 They examine how students engage six major components
of arguments: claims (assertions about what exists or what values people hold);
data (facts or statements used as evidence to support the claim); warrants
(statements that explain the relationship of the data to the claim); qualifiers
(special conditions under which the claim holds true); backings (underlying
assumptions); and rebuttals (exceptional conditions capable of defeating or
rebutting the warranted conclusion. The context, combined with the type of
project, often determine which components are necessary for a successful
argument.
1 As Clark et al. point out that the pedagogical goal of an online project, class, or environment
determine it use for students to learn from argumentation (e.g., develop a more in-depth
understanding of the content that is being discussed), whereas the hierarchical analytic
framework is better suited for analyzing online environments where students are learning how to
engage in argumentation (e.g., proposing, justifying, challenging ideas) (2007: 352).

4
Several scholars argue that these components can be combined into more
parsimonious models with fewer categories (Stegmann et al., 2012; Kollar et al.,
2007; Means & Voss, 1996; Stegmann et al, 2007). Thus, the quality of each
component depends on the validity and content of the argumentative claims, but
how they are ultimately judged is discipline- or domain-specific. In order to test
how students are in fact engaging in academic argumentation for the purposes
of learning in online discussion forums, we turn to Clark & Sampson (2007) and
(Erduran et al., 2004), who incorporate Toulmins framework to evaluate the
presence, type, and quality of each element within online group dialogue.
As Clark and Sampson (2007) explain, argumentative phrases are categorized
based on their operational purpose: (a) opposing a claim, (b) elaborating on a
claim, (c) reinforcing a claim with additional data and/or warrants, (d)
advancing claims, and (e) adding qualifications (p. 255). In our study, we
combine and then organize these categories into progressively complex
combinations in order to create a rubric by which to the judge the quality of an
argument, whereby quality refers to the structure rather than the normative
content of the argument. This approach allows coded phrases to be aggregated
and evaluated for their argumentative strength, and we adapt this method in the
first part of our analysis.
We also turn to scholars who have developed a variety of analytical approaches,
tools, and frameworks for evaluating qualitative argumentative dialogues
generated in pursuit of different educational goals in different subjects (physics,
mathematics, linguistics, social sciences), and through various modalities (face-
to-face, online chatting). These methods for analyzing online dialogues include
qualitative, quantitative, and mixed methods approaches. Likewise, we use a
mixed methods approach for our analysis, first by coding the discussion forums
and then testing the content analysis quantitatively. These methods have been
used successfully in past research (Chadha, 2017a; Chadha, 2017b; Chadha,
2017c; Van Vechten and Chadha 2013). Before we elaborate upon this model,
however, we first describe the nature and source of our data: a website designed
around discussion threads.
The Collaborative Website Overview

Data are drawn from a collaborative, cross-campus website project that involved
students enrolled during the two spring semesters of 2012 and 2013 as shown in
Table 1. In spring 2012, students from two campuses participated in the website,
for a total of 79 students. In spring 2013, a total of 81 students from three
campuses participated, including 21 from an upper level class, and 60 from two
introductory American Politics courses on other campuses. Except for the upper
level course, courses contained mostly freshmen and sophomores, and virtually
all were unfamiliar with the use of online courses requiring argumentation.

5
Table 1
Courses and Participants
Total # of
Spring 2012 Spring 2013 peer-to-
peer
responses
Campus A B C D E
Course American American Political American American N=375
Title Politics Politics Science Politics Politics entries
Capstone
Number 48 31 21 34 26
of
Students
% of 15% 13% 15% 13% 10%
Course
Grade
Methods
The collaborative website was organized around asynchronous discussion
forums that students developed through their online participation. Our research
focused on the discussion forum entries recorded by the 160 students during the
two spring semesters, and also questions that our students answered on pre- and
post- surveys. Our approach included both qualitative and quantitative elements.
First, we performed a content analysis of the 375 postings produced by the
students, and then tested the data through linear regression.
Comparability across classes. To minimize differences among courses, the
professors agreed to three syllabi requirements that were distributed to all
students. First, the students were required to respond to a minimum of eight
instructor-posed questions and respond to their peers a minimum of eight times,
for a total of 16 posted responses over the course of the semester. Second, they
were required to use a minimum of 75 words in each response. Third, each
professor assigned a grade for these activities that represented between 10 to 15
percent of the course grade for this collaborative activity. Participation was
voluntary, and students could opt for an alternative assignment.
During each semester a total of 14 weekly discussion questions were posed,
covering variety of contemporary and enduring issues in American Politics. The
number of responses varied with the type of question, whereby hot button,
controversial issues received the most attention. For this analysis, we selected
discussion questions to represent a cross-section of the type of questions asked,
as shown in Table 2. With the exception of laying the ground rules for civil
discourse in the general guidelines that were distributed by each professor at the
start of the semester, it should be noted that the professors did not intervene in
the forums. Typically the students had one week to think about and post their
replies.
Data collection. Our data collection began with the selection of discussion
question forums for analysis. In the past nine years of work in this area we have
found that controversial civil rights subjects with a moral dimension often elicit
the strongest responses and provoke the liveliest arguments; whether to site a
Muslim mosque near Ground Zero in Manhattan or to allow a fundamentalist
Christian group to protest against gay rights at a soldiers funeral were two that

6
elicited heavy back-and-forth dialogue, for example. Questions that contain links
to articles also seem to draw more thoughtful responses. Alternatively, when
students are asked to consider slightly more abstract or theoretical issues, or are
asked to supply a personal judgment to questions such as, What is presidential
greatness? they offer assertions but rarely engage in vigorous debate or
challenge each other. Students seem more unwilling to challenge each other
when opinions prevail over argumentative elements (most seem to assume a
judge-not-lest-ye-be-judged position). Peer-to-peer interaction is also biased in
favor of agreement (Chadha, 2017b; Chadha, 2017c; Van Vechten & Chadha
2013; Van Vechten, 2013).
We chose discussion question forums (DQs) from two semesters that would
represent different types of queries, both controversial and theoretical, and
include high numbers of posts. For the sake of comparison, we also included one
question that was nearly identical in both semesters (gay marriage). The selected
topics are included in Table 2 where the Responses constitute peer-to-peer
responses and the Posts refers to responses to the discussion question.
Table 2: Discussion Question Forums Selected for Analysis

Spring 2012 and Spring 2013
Semester Week Discussion Questions Peer
Posted Responses /
Posts*
Spring 2 Relevance of a presidential candidates personal
2012, n=79 life 44/ 100
3 Federal government support for colleges & 30/ 76
universities
4 Free speech and right to privacy 34/ 79
5 Gay marriage 58/ 119
7 Right to lie 39/ 91
Spring 2 Gun control 26/ 76
2013, n=81
4 Governments role 34/ 72
10 Regulating food 37/ 88
11 Political representation 32/ 69
12 Gay marriage 41/ 91
TOTAL: 375/ 861
As Table 2 shows, there were a total of 375 responses and 861 posts during the
spring semesters of 2012 and 2013. It is important to note that not every student
is represented in a given forum; because students are required to respond to a
question plus post a reply to another student, the total number of replies reflects
about two posts per student. A typical discussion forum includes responses
from roughly two-thirds of the websites student population. To ensure
consistency and reliability of interpretation, only one author coded the data.
Operationalizing the variables with the framework. The analytic framework
that Clark and Sampson (2007, 2008) developed to evaluate dialogic interactions
in the hard sciences forms the basis for our analysis of argumentative quality, as
shown in Table 3. We focus on the type of interaction, not content, to determine

7
quality. Clark and Sampsons model depicts six levels that represent
progressively more sophisticated forms of argumentation typically presented in
the hard sciences. In their model, each higher step represents higher-level
reasoning that involves more intellectually demanding components of the
process, such as providing backing for claims in rebuttals. At bottom is an
absence of argumentation, and at the highest level are extended arguments that
include at least one rebuttal.
Our analysis focuses on interactive argumentative dialogue in the social sciences
rather than on factual claim/counterclaim exchanges that typify hard sciences
discourse. In our adaptation, we propose that the quality of the dialogue should
be judged on the range, type, nature, and frequency of argumentative elements
contained in peer-to-peer responses, as shown in Table 3. For our model we
created a more parsimonious hierarchy of four types (instead of six), whereby
each type represents progressively more sophisticated levels of argumentation
as shown alongside the Clark & Sampson model. Coding each phrase within a
posting for argumentative elements, or variables, within each online response
allowed us to distinguish four levels of dialogical argumentation. It should be
noted that a complete statement or posting could contain any number of these
different elements.
Table 3
Dialogical Argumentation Typology
Clark & Sampson (2007) Our (2013) Model
Levels Characteristics of Type Characteristics/elements of dialogical
Argumentation argumentation
5 Rebuttals and at least 3 Rebuttals that Challenge and Dispute peers
one rebuttal that claims on the grounds used to support
challenges the grounds those claims, using warrants, claims and
used to support a claim counterclaims
4 Rebuttals that Rebuttals that Correct and Clarify a Position
challenge the thesis of a 2 with peers on the grounds used to support
claim but does not a claim, using qualifiers, claims, or
include a rebuttal that counterclaims
challenges the grounds
used to support a claim
3 Claims or counter-
claims with grounds
but only a single
rebuttal that challenges
the claim
2 Claims or counter- 1 Agreement/Disagreement With and/or
claims with grounds Repetition of peers argument, but Adds to
but no rebuttals Argument by providing more information,
1 Simple claim versus such as facts or backing of claims; no
counter-claim with no grounds or rebuttals
grounds or rebuttals
0 Non-oppositional 0 Contains unsupported generalizations
In our model, Type 0 would include a response consisting mainly of

unsupported generalizations: sweeping statements or opinions offered without any

8
supporting logic. Virtually no substantive or meaningful information was

offered. Type 1 responses mostly contained echoes or repetition of a peers
claims, but the argument was advanced minimally through the inclusion of a
new perspective, angle, or information. We coded for whether they added new,
non-normative information that expanded the discussion (as opposed to
providing emotionally-charged, normative, should directives or claims),
entries that also might have taken the form of teaching a new angle or offering
a new perspective. We also looked at whether a student simply agreed with a
peer, disagreed (a more challenging position), or did both in their responses. Type
2 responses encompassed clarifications, meaning that positions were clarified
through qualifiers and/or counterclaims are rebutted. The arguer may have
offered an analogy, considered new angles, sharpened the position, and so forth;
in essence, the aim was to rebut a counterclaim by adding new information or
adding qualifiers. In Type 2 responses the author might also have corrected a
peer by adding new information, or pressed a peer to reconsider a claim.
However, no direct challenges to opposing claims were offered. At the highest
level of argumentation, Type 3, the arguer offered direct rebuttals or challenges to
peers that included warrants or qualifiers intended to push deeper thinking
about a point that was made. There was also an attempt to dispute or argue, by
disputing a claim and questioning its validity or veracity. Each of these levels
evidences progression of thought that promotes learning. Descriptive measures
for these interactive components are presented in Table 4.
Table 4
Elements of Dialogic Argumentation (Spring 2012 and Spring 2013) N= 375
Combin AGREED CORRECT ADD CLARIFY CHAL- DISPUTE
-ations and/or INFO LENGE
DISAGREE
D
No 39.5% 22.4% 47.7% 52.3% 23.5% 25.6%
Yes 60.5% 77.6% 52.3 % 47.7 % 76.5 % 74.4 %
Table 4 shows that over half (60.5%) of peer-to-peer responses contained direct
engagement with a peer, which took the form of agreement and/or
disagreement. Well over half of the posts (77.6%) included corrections, meaning
that they provided factual information in an attempt to clear up a misconception.
Another 52.3% added or provided additional information to support their
responses, and 47.7% clarified their responses with specific information or by
articulating a different perspective. A similarly high percentage (76.5%)
challenged each other, and another 74.4% disputed (or directly argued) by
supplying supporting evidence or reasoning for their claims.
We were also interested in measuring whether students could use these different
elements in combination, which would be a sign that students were more deeply
engaged and on the path to actually learning through their interactions. In our
view, generalized replies that required little thought, expressed emotional
reactions, contained unsupported generalizations, and contributed nothing new
to the discussion could be distinguished from those in which students were
pushing themselves to consider new angles and reconsider their own issue
stances.

9
To measure depth of academic engagement, we created an index based on five

elements. First, we scored student entries for assertions that were reflective,
deliberative or critical (reflective), the functional equivalent of claims, and/or
qualifiers, and/or rebuttals. Second, we coded for whether the post included an
honest question that created further deliberation among students (rather than a
rhetorical one), such as when a student asked a peer to think about another
aspect of an issue (honestq). Third, we looked backings to claims that took
shape in two forms: in references to authorities, such as an assigned text or the
professors teachings (classtext); or in links or references to outside media or
sources such as an article, video clip, or other online materials supporting any
assertions the student is making (media). Fifth, we coded for length (short,
medium, or long based on the number of words), as a proxy for effort to
articulate an argument. Students who wrote virtual essays, for example, clearly
achieved a different level of critical thought than those who merely offered an
opinion that was expressed in a few lines.
Our composite variable, depth of academic engagement (or more simply,
depth of engagement) represents a sum of the scores for these five elements.
Therefore, a post that evidences deep engagement would incorporate all five
elements: reflective + honestq + class text + media materials + length. These
results are presented below.
Research Questions and Hypotheses. We were interested in how seriously

students engage with each other in online discussions, and whether they argued
with each other and wrestled with the material in gently provocative ways that
could change a persons mind or produce a new position. More specifically, we
wanted to know whether the computer-mediated communication process of
dialogical argumentation could foster academic learning. Building on Clark and
Sampsons work (2007; 2008), we hypothesized (H1) that the most sophisticated
levels of argumentation would be least common, in that students would
challenge and dispute each other (Type 3 responses) less often than they would
correct and clarify their positions to each other (Type 2), and that the majority of
students would reach a basic level of engagement by agreeing and/or disagreeing
with each other (Type 1). We also hypothesized (H2) that Type 0 responses
would be less prevalent than Type 1 responses, given our clear guidelines about
length of posts and our expectations that they would reflect on their answers
before recording their thoughts. Thus, we expected the greatest number of posts
to be Type 1, representing entry-level engagement with the learning process.
Furthermore, we hypothesized that students who dove deeply into the
process by incorporating links to other materials or producing lengthy posts
would also be more likely to argue at higher levels of sophistication.
Findings and Discussion

Type of Arguments. Noting first that a students post could contain different
argumentative elements of argument, we mined the responses for progressively
more sophisticated combinations that would allow us to categorize them by type.
We found that almost one-third (30.1%) of posts included the most advanced
Type 3 combination of arguments: these incorporated challenges and
disputations, and pushed ahead the discussion with new, engaging points or
questions. A larger percentage (56.5%) included Type 2 combinations, which

10
encompassed corrections and clarifications. The largest percentage (68.5%)

included Type 1 interactions where the students agreed and/or disagreed with
each other, or added a new point or information. A much larger portion, 77.4%,
contained unsupported generalizations, as reported in Table 5.
Table 5
Dialogical Argumentation Types: Spring 2012 & Spring 2013 (N=375)
Percent of Interactive Combined Elements of
Posts Argumentation
Type 3 30.1% Challenge + Dispute
Type 2 56.5% Correct + Clarify
Type 1 68.5% Agreement/Disagreement + Offer
Info
Type 0 77.4% Information + Unsupported
generalizations
The results in Table 5 support our main hypothesis (H1), such that fewer than a
third of all peer-to-peer responses contained the most sophisticated arguments,
while non-oppositional statements of opinion were among the most common
types found among the responses. More difficult arguments were indeed less
common. Contrary to our expectations (H2), however, Type 0 responses were
more common than Type 1, which provides some evidence that students were
contributing to the discussions without investing much thought.
At the lowest level of engagement, Type 0, students typically made sweeping
claims or generalizations lodged in common wisdom, yet remained civil. This
example of a Type 0 response comes from a spring 2013 dialogue about the
utility of banning sugary drinks and taxing fatty foods:
I do think its a nice thought however ultimately I just feel that people should
just do a better job of taking care of themselves and be better role models for the
youth. If you set good examples kids will look up to you and what you do.
In this example, the student backs her opinion by a broad generalization. This
exmplifies Type 0 responses in which information relevant to the thread might
be included, but unsupported generalizations render it unhelpful for advancing an
argument from which students can learn, either through practice or the act of
considering their peers arguments.
In Type 1 interactions, students disagree at least mildly with their peers (often
they combine disagreement with agreement), and they continue to advance an
argument by offering a new perspective, angle, or information, even if only
briefly. No direct challenges are made. In the following excerpt from a
discussion forum about government regulation of food from the Spring 2013
semester, Student Y responds to Student X by not only repeating Xs claim, but
also by supplying his own reasoning, which effectively adds a new point
(childhood diseases) to the dialogue:
Student X initial post: Bloomberg's attempt to ban the large sugary drink is a
good idea because most of America is unhealthy and obese. Banning the large
sugary drink is a good health decision. Now the question, Is it really a debate?
No, this should not be a debate and the large sugary drink should not be banned.
Banning the large drink will not stop people from drinking large amount of soda,
it will only have them purchase two drinks instead of one which will equal to or

11
more to the size of one large drink. People should be allowed to purchase their
own size of drink. If customers are interested in their health, restaurants have
posters up on the wall which has the amount of calories on items sold
Student Ys response to Student X: I agree because these days we now have
children with diabetes and obesity. The educational system has already
attempted to help the obesity problem by offering healthier food options. Maybe
allowing the government to help with the obesity problem will aid the obesity
and diabetes issue that we have present.
Here, Student Y adds to the discussion with this concise point about
governments responses to childhood obesity, but does so without challenging
his peer directly and without providing data or qualifiers for the assertion about
obesity. Type 1 responses thus contribute in some small way to the general
argument, and over two-thirds, 68.5%, did so.
Over half the interactions (56.5%) were of Type 2: a student would clarify his or
her position, and/or rebut a counterclaim, sometimes correcting a peer by adding
new information. Warrants and backings in the form of reasoning and examples
were common, indicating that the author was engaging the learning process in a
more rigorous way. Type 2 is exemplified by this thoughtful reply to another
students post, which the author does not question:
Even though i am proud of Bloomberg for trying to help make New York's
citizen's healthier, i do not think potentially banning soda size is the main health
problem. There are many reason why. For example, just banning the soda size
alone will not stop people from drinking more soda. In fact, this will have the
complete opposite reaction. Once people hear their soda size is being cut down, it
will only make them want to buy more soda to make up for the loss in size,
meaning they will buy more cans or bottles of sodas until they are satisfied. I
agree with [Margo] that restaurants should make the public more aware of the
ingredients rather than the calorie count of food items. If people know about
what is in their food, then they will have a better idea of how to control what
they eat, making healthier food choices. Let us take the fast food chain "Subway"
for example. They give healthy food choices and make their customers aware of
the calorie and sugar content which helps us all to make better food choices.
When it comes to decisions about food, at the end of the day each person is
responsible for their own choices and will have to bear the consequences or gain
the benefits according to how they choose.
Type 2 posts, therefore, include a correction or counterclaim (information that
corrects a peers claim) and a clarification of ones own position, usually through
additional data, warrants, or qualifiers. We interpret this type of post as
moderate engagement of the learning process.
Type 3 responses include a direct challenge to a peers statement or premise, and
key parts of a dispute are present as real dialogical argumentation unfolds. Again,
30.1% fell into this category, as they combined elements of argumentation
(warrants, claims, counterclaims) that enabled the author to clarify, challenge,
and argue thoughtfully. This kind of argument is demonstrated here:
I don't think that you managed to capture the entirety of my argument. All you
managed to do was call me bigoted; and you support your argument by saying
that the beliefs of many don't mean anything (as in your case for Religion) when
your opinion on gay marriage is at its foundation just a belief. The phrase of
separate but equal was meant to be in respect to the Church and State. I'll admit

12
that it was a poor choice of words given the civil rights history of the U.S. I
respect your point of view here, but you completely overlooked mine.
We regard this type of response as paving the way for others to reconsider their
views, including the target of the post, as well as other students who might read
the exchange. Students at this level are fully engaged in argument, trying to
convince others that his claims are meritorious and valid.
Depth of Engagement. We also assessed depth of academic engagement
quantitatively. First we created an index for depth of engagement by scoring the
responses for the presence of five various elements (as described in the methods
section): overall reflectiveness, asking honest questions, including references to
the class or textbook, inserting links to outside media or materials, and length
(scored one to three)2. A response containing none of these elements would be
scored zero; a response reflecting deep engagement would incorporate all five
elements. Actual scores ranged from zero to five, and most of the 375 responses
clustered around the mid-rangewhat we might call moderately engaged, as
Table 6 shows.
Table 6
Frequency and Percent of Academic Engagement Scores
Score Frequency Percent Cumulative Percent
0 <1 .03% .03%
1 43 11.47% 11.5%
2 145 38.7% 50.2%
3 98 26.1% 76.3%
4 66 17.6% 93.9%
5 23 6.1% 100%
N 375 100%
According to Table 6, about one out of ten responses demonstrated
engagement at the lowest level; its unlikely that the author learned anything
new or that peers gleaned meaningful information from these posts. Over two-
thirds (64.8%) were moderately engaged, having scored at least a two or a three
on our scale. At levels four and five, students are now invested in the learning
process, sharing materials and new ideas, prodding each other to question
further, or providing links to interesting articles that could shed further light on
the issue at hand. Almost a quarter (23.7%) appeared to be deeply engaged.
Finally, we wanted to know if a students use of argumentative elements could
predict how reflective his or her response was. We scored each response for
overall reflectiveness: did the student generally seem to be thoughtful, or was
the response a knee-jerk, lets-get-this-over-with response? Using a dummy
2 Considered as a single variable (it is otherwise included in the depth score), length is another
indicator of students engagement through discussions with each other leading to in dialogical
argumentation. Students were required to post at least 75 words, and found that posts on average
exceeded the minimum at a mean of 96 words, but with a rather large standard deviation (56
words). Viewed another way, in both semesters most students (73%) posted what we coded as
medium length posts, meaning 50-150 words. At the lower end, 15% of all students posted far
less than the required minimum (0-50 words), and the remainder (11.5%) far exceeded the
minimum by posting at least double what was required (151+ words).

13
variable for reflective, we analyzed the data through a series of simple linear
regressions to estimate the relative weight of each element of argumentation.
Interestingly, each term was found to be statistically significant (p<.001). A
summary of results is displayed in Table 7.
Table 7
Summary of Regression Analyses
Unstandardized Standardized
Model Coefficients Coefficients
B Std. Error Beta
(Constant) 1.603 .108
Correct -.226 .130 -.085*
Clarify my position .819 .098 .367*
Challenge views .214 .159 .082*
Dispute .002 .157 .001*
Adding information .716 .108 .321*
Agreeing and
.194 .067 .154*
disagreeing
*p< .001.
This finding tells us that students who employ argumentative elements also
tend to be more reflective in their answers; they invest more in their learning
when they argue. Overall, the regression analysis confirms what we found
through our content analysis, providing further evidence that dialogical
argumentation occurs across three different types of argumentthat is, Types 1
to 3. This is the strongest evidence that websites designed for academic purposes
can produce virtual learning environments.
Related Variables
While we found statistically significant evidence that students engage each other
in the process of argumentation, we continued to explore other questions that
formed during this process. Would a question about gay marriage that was
posed a year apart produce noticeable differences in argumentation? We found
this not to be the case. The student populations from two semesters took similar
approaches to answering questions, a conclusion confirmed by the depth of
engagement patterns, which were roughly parallel across two semesters as
shown in Figure 1.

14
Gay Marriage (2013)
Gay Marriage (2012)
0 10 20 30 40
Percent
L-5 L-4 L-3 L-2 L-1
Figure 1: Depth of Engagement Scores for Discussion Questions about Gay Marriage
In Spring 2012 and Spring 2013
We also explored whether the wording of the discussion question was correlated
with the sophistication of argumentation. We found that question prompts that
attract the most replies are current-events-oriented and contain links to outside
materials such as news articles. Theoretical questions attract the fewest
responses (even when they are required), as well as the most limited branching
among students; there is plenty of room for students to think critically and post
reflective statements, but they argue less with each other over definitions or
applications of abstract concepts when these are the actual topic of discussion. In
spring 2012 and spring 2013 the discussion questions that attracted the fewest
replies were about governments role (n=72) and the nature of political
representation (n=69), compared to 119 and 91 replies about gay marriage.
A final note concerns the way that students interacted asynchronously to
create back-and-forth dialogues. Most of the peer-to-peer responses (84%)
involved one single reply rather than a sustained series of responses; 11.5%
extended to two responses; 4.5% involved three or more responses. However,
some of those exchanges involved several persons, and branching was common.
The spring 2012 forum on gay marriage provides good examples of this. Almost
half (47%) of the posts in this forum were actual interchanges between or among
students. Similarly, a spring 2012 forum that asked students to weigh privacy
against the governments need to collect private information attracted 79 replies,
44% of which were branches that included three or more people. Clearly
students are engaging each other through this format, though just under half are
participating in actual dialogues involving more than two people.
Conclusions
The purpose of our shared academic website was to provide a space for
undergraduate students from different campuses to interact and to promote
thoughtful discussion and learning through asynchronous discussion-based
forums. We hoped our students would learn about the issues and their own
positions through dialogical argumentation. This inquiry into the nature of
online student dialogue uncovered statistically significant evidence that students
did just that: they engaged the learning process through arguing with each other,
asynchronously through discussion forums, in the spring 2012 and 2013
semesters. Concrete elements of argumentation were visible in students

15
responses to each other, namely in the way that they countered each others
claims, clarified their own positions, offered new perspectives and information,
questioned each other, and challenged one other to account for their views. It
should be noted that the process does not encompass all students equally; not
everyone argued, and not every student was invested in the site. However,
based on the totality of evidence, we conclude that the process was a valuable
learning tool for those who did engage.
Students are more likely to engage in activities when they feel their contribution
is valued by others. As students reported in open-ended survey responses at the
end of the semester, the iterative nature of the online exchanges tended to foster
an online community (59% in spring 2012 agreed that the website made them
feel as if they were part of a larger political community), which begins with
following basic rules for civil discourse. The overwhelming majority (84%) also
felt that the discussion on the site increased their interest in political issues and
prompted them (84%) to find more information about these issues. Specifically
designed educational portals such as ours can simultaneously promote engaged
e-learning and a sense of community. Definitively, CMC is an effective means to
engage students in meaningful academic exchanges, regardless of discipline.
In using a digital portal designed to support interactive e-learning and by
concentrating on students interaction, we have shown that argumentation
involving students across geographic boundaries can lead to productive
conversations that prod students into thinking reflectively in an environment
conditioned by academic instruction. Its clear that online educational portals
possess great potential to encourage critical thinking and learning. The
ingredients for knowledge construction and cognitive development are threaded
into discussion forums, and when enlivened through argumentation, learning
can take place.
References
Andriessen, J., Baker, M., & Suthers, D. (Eds.). (2013). Arguing to learn:
Confronting cognitions in computer-supported collaborative learning
environments. Springer Science & Business Media 1. 33-56.
Bell, P. (2004). Promoting students argument construction and collaborative
debate in the science classroom. Internet environments for science education,
3. 115-143.
Bender, T. (2012). Discussion-based online teaching to enhance student learning:
Theory, practice and assessment. Stylus Publishing, LLC.
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956).
Taxonomy of educational objectives, handbook I: The cognitive domain.
Blount, A. G. (2006). Critical reflection for public life: How reflective practice
helps students become politically engaged. Journal of Political Science
Education, 2(3), 271-283.
Chadha, A. (2017a). Learning to Learn: Lessons from a Collaboration. Journal of
the Scholarship of Teaching and Learning, 17(3), 34-47.
Chadha, A. (2017b). E-Learning Opens Doors to the Online Community: Lessons
from a Longitudinal Study. National social science association, 45.

16
Chadha, A. (2017c). Comparing Student Reflectiveness in Online Discussion

Forums across Modes of Instruction and Levels of Courses. Journal of
educators online 14 (2), 1-19.
Chu, H. C., Chen, J. M., & Tsai, C. L. (2017). Effects of an online formative peer-
tutoring approach on students learning behaviors, performance and
cognitive load in mathematics. Interactive Learning Environments, 25(2),
203219. doi:10.1080/10494820.2016.1276085
Clark, D. B., & Sampson, V. D. (2007). Personally seeded discussions to scaffold
online argumentation. International Journal of Science Education, 29(3), 253-
277.
Clark, D. B., & Sampson, V. D. (2008). Assessing dialogic argumentation in
online environments to relate structure, grounds, and conceptual quality.
Journal of Research in Science Teaching, 45: 293-321.
Coirier, P., & Golder, C. (1993). Writing argumentative text: A developmental
study of the acquisition of supporting structures. European journal of
psychology of education, 8(2), 169-181.
Collins, L. (2014). How online learning enables personalized learning. Getting Smart.
Retrieved fromhttp://gettingsmart.com/2014/03/online-learning-
enables-personalized-learninginterview-dr-lisa-collins/
Cooper, J. (2001). Peer learning in law: using a group journal. In D. Boud., R.
Cohen, & J. Sampson (Eds.), Peer learning in higher education: Learning from
& with each other (pp. 115127). London, UK: Stylus Publishing Inc.
Dehler, C., & Porras-Hernandez, L. H. (1998). Using Computer Mediated
Communication (CMC) to Promote Experiential Learning in Graduate
Studies. Educational Technology, 38(3), 52-55.
Erduran, S., Simon, S., & Osborne, J. (2004). Tapping into argumentation:
Developments in the application of Toulmin's argument pattern for
studying science discourse. Science education, 88(6), 915-933.
Gordon, R. & Connor, R. (2011). Team based learning in management education.
In D. Boud, R. Cohen. R., J. & Sampson (Eds.), Peer learning in higher
education: Learning from & with each other (pp. 8598). London, UK: Stylus
Publishing Inc.
Hakkinen, P. (2013). Multiphase method for analyzing online discussions.
Journal of Computer Assisted Learning, 29(6), 547-555.
Herring, Susan. (1993). Gender and democracy in computer-mediated
communication. Electronic Journal of Communication, 3(2), 331.
doi:10.1016/B978-0-12-415040-9.50121-4
Jacoby, B. (2009). Civic engagement in higher education. Concepts and practices.
Jossey-Bass, San Francisco.
Kollar, I., Fischer, F., & Slotta, J. D. (2007). Internal and external scripts in
computer-supported collaborative inquiry learning. Learning and
Instruction, 17(6), 708-721.
Kuhn, D., Shaw, V., & Felton, M. (1997). Effects of dyadic interaction on
argumentative reasoning. Cognition and instruction, 15(3), 287-315.

17
Kuhn, D., & Udell, W. (2003). The development of argument skills. Child
development, 74(5), 1245-1260.
Lane, D. R. (1994). Computer-mediated communication in the classroom: Asset
or liability. In Workshop presented.
Lee, M. J., & McLoughlin, C. (2007). Teaching and learning in the Web 2.0 era:
Empowering students through learner-generated content. International
journal of instructional technology and distance learning, 4(10), 21-34.
Leito, S. (2000). The potential of argument in knowledge building. Human
development, 43(6), 332-360.
Means, M. L., & Voss, J. F. (1996). Who reasons well? Two studies of informal
reasoning among children of different grade, ability, and knowledge
levels. Cognition and instruction, 14(2), 139-178.
Resnick, L. B., Salmon, M., Zeitz, C. M., Wathen, S. H., & Holowchak, M. (1993).
Reasoning in conversation. Cognition and instruction, 11(3-4), 347-364.
Rowntree, D. (1995). Teaching and learning online: a correspondence education
for the 21st century? British Journal of Educational Technology, 26(3), 205-
215.
Schouten, A. P., Valkenburg, P. M., & Peter, J. (2009). An experimental test of
processes underlying self-disclosure in computer-mediated
communication. Cyberpsychology: Journal of Psychosocial Research on
Cyberspace, 3(2).
Schwarz, B. B., & De Groot, R. (2007). Argumentation in a changing world.
International Journal of Computer-Supported Collaborative Learning, 2(2), 297-
313.
Stegmann, K., Weinberger, A., & Fischer, F. (2007). Facilitating argumentative
knowledge construction with computer-supported collaboration scripts.
International journal of computer-supported collaborative learning, 2(4), 421-
447
Stegmann, K., Wecker, C., Weinberger, A., & Fischer, F. (2012). Collaborative
argumentation and cognitive elaboration in a computer-supported
collaborative learning environment. Instructional Science, 40(2), 297-323.
Toulmin, S. (1958). The uses of argument. Cambridge: Cambridge University Press.
Van Vechten, R. B. (2013). He Said/She Said: Tracking Gender Differences in
Online Academic Discussions. Paper presented at the APSA teaching and
learning conference.
Van Vechten, R. B. and Chadha, A. (2013). How students talk to each other:
Findings from an academic social networking project. Teaching civic
engagement: From student to active citizen, 167-188.
Wilson, J. T. (2001) Project management teams: A model of best practice in
design. In D. Boud., R. Cohen, & J. Sampson (Eds.), Peer learning in higher
education: Learning from & with each other (pp. 99114). London, UK: Stylus
Publishing Inc.

18

Vol. 16, No. 8, pp. 18-36, August 2017
A Qualitative Examination of Factors for Success

in a Content-Based English Language
Learner Classroom
Janet Delgado, Ed.D.

The School District of the City of York
Lorraine T. Benuto , Ph.D.

University of Nevada, Reno
Department of Psychology
Abstract. The dramatic increase of English Language Learners (ELLs)

created a sense of urgency across school districts who struggle with
efficiently educating students in a manner that facilitates the acquisition
of English for ELLs and ensures that testing standards across content
areas are met. Content-based sheltered instruction can provide quality
education while maintaining the integrity of effective English language
practices to a large number of ELLs simultaneously. The purpose of this
qualitative multiple-case study was to identify and understand the
essential attributes and relationships that contribute to the
successfulness of content-based sheltered instruction elementary
classrooms for English learners in an urban school district in southern
Pennsylvania (this school has successfully and effectively accelerated
their ELLs' levels of second language acquisition). Utilizing an
Appreciative Inquiry Approach, the attributes that maximize the
instruction of ELLs across four content-based, sheltered instruction ELL
classrooms within one K-8 school were identified. Results revealed that
these classrooms were successful when strong interrelationships were
evident between language and content learning, efficient organizational
structures, as well as a focus on the celebration of culture.
Introduction
Many public school districts across the country require additional
resources to educate students who arrive daily from other U.S. cities and
countries around the world (Gndara, Maxwell-Jolly & Rumberger, 2008). The
vast differences that exist between students across socioeconomic, cultural, and
linguistic dynamics contribute to students' levels of knowledge upon entering
school (Fayden, 2011). These factors create a sense of urgency to accelerate
English mastery so as to narrow the achievement gap, meet testing standards
(Cosentino de Cohen & Chu Clewell, 2007), and avoid the ramifications of the
Elementary and Secondary Education Act (U.S. Department of Education, 2009).

19
Unfortunately, this exacerbates an already impoverished learning environment

for many (Fayden, 2011).
Culturally and linguistically diverse (CLD), or English language learners
(ELLs), are among the largest and most disadvantaged subgroups in many
school districts (Fortuny, Capps, Simms & Chaudry, 2009). Approximately 17
million culturally and linguistically diverse children live in the U.S. (Migration
Policy Institute, 2012). These learners come from home environments where a
language other than English is spoken and they are acquiring language and
literacy skills in English, regardless of their birth origin (Herrera, 2010).
Studying language proficiency is important although other factors (e.g.,
socioeconomic circumstances) can impact how students perform on high-stakes
testing (Fortuny et al., 2009). It is common for members of school organizations
to select pedagogical approaches that can directly affect change within students'
rate of second language acquisition (Ramos & Krashen, 2011; Starnes, 2010).
Unfortunately many ELLs receive content instruction from educators who are
not prepared to address their second language development needs (Echevarra,
Short & Powers, 2008; Gndara, et al., 2008). Content-based, sheltered
classrooms can provide a superior alternative to traditional English-only
methods (Thomas & Collier, 2002) as these classrooms are led by dually certified
ELL and content educators who deliver grade-level material and focus on
English acquisition simultaneously (Genzuk, 2011). In most cases, they
incorporate the Sheltered Instruction Observation Protocol (SIOP) model
(Echevarra, Vogt & Short, 2017) as they demonstrate cultural responsiveness
and allow native language and English dialogue in a low-anxiety language
environment (Herrera, 2010; Taylor, 2010). This pedogogical design was
influenced by Thomas and Collier's seminal work, known as the Prism Model
(1997), which facilitates the simultaneous development in the academic,
cognitive, linguistic and sociocultural domains of ELLs in their native and target
languages within their academic environments.
Thomas and Collier (1997) created the Prism model for language
acquisition from a large-scale study and the prism model considers multiple
areas of linguistic, academic, cognitive and sociocultural development (Thomas
& Collier, 1997; 2002). When fostered simultaneously, these areas can determine
the academic success for English language learners. In particular, facilitators
within schools must encourage the development of language and culture in both
the English learners native and target languages to provide supportive
sociocultural environments for students (Thomas & Collier, 1997). Once all the
learners' domains are addressed within the school, second language acquisition
and acculturation can take place and students can prosper academically (Jang &
Jimenez, 2011). Thomas and Collier (2002) were able to reaffirm their positive
longitudinal results several years later, supporting the importance of developing
the domains presented in the Prism Model for language acquisition within
schools.
Although Thomas and Collier (1997; 2002) stressed the importance of
bilingual education, they proposed models of language instruction in English
only that correlated closely to the areas of the Prism Model of second language
acquisition. Specifically content-based ESL programs, featuring sheltered
instruction, proved to be the most effective alternative to bilingual education

20
(Clark, 2009) when compared to other forms of subtractive schooling (Garza &
Crawford, 2005), such as English immersion and/or pull-out services. Content-
based, sheltered instruction ELL classrooms operate through an instructional
delivery approach that is focused on teaching English through content via
explicit and scaffolded language experiences to assist students in reaching
grade-level content expectations in English (Echevarra, Richards-Tutor, Canges
& Francis, 2011).
Many school districts experience rapid increases in the enrollment of
English learners and are required to implement English-only instructional
practices to educate them (Caldern, Slavin & Sanchez, 2011). However, school
districts must offer English language instruction that amalgamates the learners
academic needs with appropriate personnel and resources without
compromising the second language development of English language learners in
the school setting (Herrera, 2016; Thomas & Collier, 1997). One such means by
which school districts can achieve this is via the implementation of content-
based sheltered instruction ELL classrooms (Short, Fidelman & Louguit, 2012).
Content-based sheltered instruction classrooms provide learning environments
that allow learners to acquire English as they benefit from grade-level content
and language instruction designed specifically for ELLs. Not only do they
accelerate the learners rate of English acquisition, they make it possible for
educators to differentiate their instruction as their goal is to narrow the
achievement gap between English language learners and their monolingual
peers within their school districts as quickly as possible (Genzuk, 2011).
However, this sheltered instruction ELL classroom model is underutilized
because of the negative perceptions that allude to the segregation of students
(Gndara & Orfield, 2010), as well as the perceived overwhelming concentration
on English language skills in these classrooms (Clark, 2009). Thus the problem
addressed in this study was that the increasing need for educating large
populations of ELLs efficiently continues to be a national challenge (Garca,
Jensen & Scribner, 2009) and while content-based sheltered instruction was
effective, it was unclear what factors contribute to the successfulness of content-
based sheltered instruction elementary classrooms for English The purpose of
this qualitative multiple-case study was to identify the essential attributes and
relationships (Yin, 2014) that contributed to the successfulness of content-based,
sheltered instruction elementary classrooms for English language learners in an
urban school district in southern Pennsylvania. Because data was collected from
multiple sources in multiple forms (and for the purpose of organization and
clarity), the information about the materials and participants is consolidated
here under this section.
Methodology
Research tools
Individual Interviews. Semi-structured individual interviews were
conducted with administrators. The interview questions were pilot tested with
one administrator and one district support personnel. The interview questions
were revised based on the feedback provided. Participants in the individual
interview portion of this study consisted of three administrators, including one

21
ELL reading specialist. They were selected because of their administrative roles,
expertise and support (Lauckner, Patterson & Krupa, 2012) at the onset of the
schools implementation of sheltered ELL classrooms for a duration of at least
five years. Additional study participants included seven educators certified in
both elementary education and ELL. A variety of viewpoints based on teaching
experiences were represented as three of the seven educators had taught as ELL
pull-out teachers, as well as grade-level content teachers.
Research tools for Observations. Four classroom observations were
conducted and data was collected using the Danielsons A Framework for Teaching
protocol (2007) and the SIOP Protocol (Echevarra, Vogt, et al., 2017). See Table 1
for information about participants.
Table 1
Classroom Observation Characteristics
Classrooms Grade Content Group ELL Service ELLs in
Area Class
Classroom 1 6th math full class integrated 57%
Classroom 2 3rd science full class integrated 68%
Classroom 3 1st language full class integrated 63%
arts
Classroom 4 3rd language full class integrated 68%
arts
A Framework for Teaching protocol (Danielson, 2007). The classroom

environment scale from the Danielsons A Framework for Teaching protocol (2007)
is a professional practice performance scale utilized by administrators to
evaluate teachers' performance. It was used to examine sociocultural attributes
within each of the ELL classrooms to provide a context for the observations.
Specifically, Doman 2 focused on classroom culture and climate, procedures and
physical environment.
SIOP Protocol (Echevarra, Vogt, et al., 2017). The instructional content
of each classroom was observed and data was measured accessing a tool
specifically created for ELL sheltered instruction (Short, Echevarra, et al., 2011).
The SIOP protocol evaluates the categories of lesson preparation, building
background, comprehensible input, learning strategies, interaction
opportunities, application experiences and assessment options (Echevarra,
Vogt, et al., 2017) within sheltered instruction lessons. The SIOP observation
tool (Echevarra, Vogt, et al., 2017) provided rich data that was already present
during content instruction within each elementary ELL classroom. The
observation tool was utilized during English language arts or a specific content
area lesson of the classroom teachers choice as it increased the opportunity for
the educators to demonstrate a variety of practices implemented across content
areas. This valid and reliable instrument was organized as a 5-point Likert scale
(Trochim & Donnelly, 2008) and demonstrated the degree to which educators
followed the lesson preparation and delivery protocol with fidelity. At the

22
conclusion of each observation, a brief post-observation interview with the

teacher was conducted to clarify and/or extend conversation regarding what
was observed during the session.
Focus Groups. Six students from grades three or six were also included
via focus groups (see Table 2 for demographic details), so that the data collected
represented varied perspectives (Yin, 2014) of content-based, sheltered
instruction ELL classrooms (see Table 3 for a breakdown of demographics). At
the outset of the focus group interview, the purpose and protocol was
introduced after initial questions about educator experiences were discussed in a
whole group session format. The protocol established the procedures for
creating a visual representation of their group's conversations, an Affinity
diagram. The Affinity diagram is created by members of a group (Abilla, 2010),
and provides a visual structure that organizes a large amount of data into
themes based on their connections and relationships (National School Reform
Faculty, 2012).
Table 2
Student Focus Group Demographics
Student Sex Grade Yrs. at Home Family
School Language(s) Origin
Student 1 male three 2 Spanish Puerto Rico/
Dominican
Republic
Student 2 female three 3 Spanish/English Puerto Rico
Student 3 male three 3 Khmer/English Cambodia
Student 4 female three 3 Spanish Puerto Rico
Student 5 male six 6 Spanish/English Puerto Rico
Student 6 male six 3 Spanish Puerto Rico
Data Collection
An Appreciative Inquiry (AI) approach is a method of inquiry that
focuses on the positive attributes of an organization (Gaddis & Williams, 2009).
The main components included in an AI approach are discovery, dream, design
and delivery/destiny (Bushe, 2011) and these provided a procedural focus for
this study inquiry (Bushe, 2011; Cantore & Cooperrider, 2013; Stevenson, 2011).
Data Processing and Analysis
A case study database was necessary to capture large amounts of
language data across multiple settings that visually represented commonalities
in the data and assisted in the compilation of each case record (Patton, 2002)
utilizing Atlas.ti7. Adhering to an inductive process (Schutt, 2012); code
identification, revision, as well as data reduction (Johnson, Christensen &
Turner, 2014) was necessary to focus a large amount of language data into
manageable components (Miles, Huberman & Saldana, 2013). To determine
elements of significance and relationships across the multiple sources of data,

23
additional analysis led to the identification of the context of each assigned code
frequency (Miles et al., 2013). Table 3 demonstrates the distribution of the
frequency of codes by data source and was utilized as the first layer of analysis
(Friese, 2012) as to arrive at themes that exemplify the successfulness of content-
based, sheltered instruction ELL classrooms at Wilson K-8 School.
Table 3
Code Families by Data Source
Code Family Individual Classroom Focus Total
Interviews Observations Groups
Academic Language 26 90 7 123
Class Community 19 46 24 89
School Characteristics 43 15 33 91
School Culture and Climate 36 17 30 83
School Supports and Resources 32 9 25 66
Student Achievement 23 6 5 34
Instructional Practices 76 27 212 315
Teacher Behavior and Beliefs 41 16 25 82
Totals 296 411 176 883
Results
RQ1. The first research question was: How are content-based, sheltered
instruction classrooms maximizing the instruction of ELLs in an elementary setting
with large populations of ELLs? . The data from the four content-based,
sheltered classroom observations revealed that multiple explicit instructional
practices within positive and engaging classroom communities, in conjunction,
maximized the instruction of ELLs. These classrooms consistently provided safe
and supportive conditions for learning by all students, including English
language learners, so as to accelerate their progress academically and
linguistically (Jang & Jimenez, 2011). Evidence of successful student talk and
active engagement were described within each classroom observation.
Academic/language instruction was infused within all of the described grade-
level content lessons. However, additional data was needed to identify specific
practices and strategies as evident within the lessons, as well as from the
perspectives of the students themselves. The following two sub-questions led to
more precise information as the triangulation of the data (Shutt, 2012; Yin, 2014)
from multiple sources i.e., the SIOP observation protocol (Echevarra, Vogt, et
al., 2017) and the student focus group interviews.
SQ1. The first sub-question derived from research question 1 was: What
instructional practices take place in content-based, sheltered instruction ELL classroom
that makes them effective for English language learners? Two current and retired
administrators were asked what positive attributes were evident, relating to
instructional practices, when they entered and/or observed content-based,

24
sheltered instruction ELL classrooms on a regular basis. Although a minimum

of eight specific examples were provided by each participant, they discussed
topics and referred to examples of instructional practices throughout the
interviews. When the language data was compiled from the individual
interviews, the overarching categories of instructional practices most identified
are displayed in Table 4.
Table 4
Categories of Instructional Practices Witnessed in ELL Classrooms
Instructional Practices Frequency
Explicit academic/language instruction 23
Bilingual support 12
Differentiated instruction and participation 23
Levels of support throughout lessons 14
Multisensory teaching and learning 13
Explicit academic/language instruction, bilingual support, differentiated

instruction and participation, levels of support throughout lessons and
multisensory teaching and learning were the instructional practices identified as
effective for the English language learners in the ELL classrooms. These
observation teaching behaviors were documented on the SIOP to indicate the
elements of effective sheltered instruction for each observed classroom and were
reported. Although all categories were identified within each lesson, the
following, in particular supported the instructional practices mentioned above:
comprehensible input, strategies, interaction, practice/application and delivery.
These categories measured the effectiveness in content delivery, the variety of
multisensory experiences, explicit strategies, as well as student/peer
engagement and support and were noted on the Likert scale (1-not evident to 4-
highly evident) as 3-evident or 4-highly evident within each lesson observed
at Wilson K-8 School. The data triangulated, or analyzed from multiple sources,
was necessary to ensure validity of the results (Yin, 2014) throughout the
qualitative study. Particularly to respond to this sub-question, it was
accomplished via the classroom observations and the results from the SIOP
protocol. The results support the effective practices of explicit
academic/language instruction, bilingual support, differentiated instruction and
participation, levels of support throughout lessons and multisensory teaching
and learning as essential to the success of content-based, sheltered instruction
ELL classrooms.
SQ2. From the perspectives of learners, what strategies increase their
learning and their use of academic language?
The English language learner participants at Wilson K-8 identified
several practices that increase their learning and use of academic language
within content-based, sheltered instruction ELL classrooms (see Tables 5 and 6
for details). Students confirmed that working collaboratively within peer
configurations (partners or groups) naturally elevates the levels of student

25
engagement and support during explicit content and strategy instruction.

Additionally, the classroom observation data supports the ELLs' statements and
expanded on the examples of academic language techniques used in the
classroom, such as cognate instruction and teacher elaboration of vocabulary
and student responses. In fact, within the classroom observation data, the
instructional practices and academic language code families displayed two
strongest occurrences: teacher practices and academic language. Table 6
indicates the specific codes within the academic language family displayed
during the four non-participatory classroom observations of the ELL classrooms.
The trend was reaffirmed that in order to increase content learning, as well as
academic language, intentional opportunities for student engagement,
experimentation and discussion among peers were essential in the successful
operation of content-based, sheltered instruction ELL classrooms.
Table 5
Instructional Practices that Increase Learning and Academic Language
Instructional Practices Student
Response
Frequency
Collaborative peer groups (partners included) 9
Explicit content and strategy instruction 6
Multisensory teaching and learning 6
Student engagement in lessons 4
Students receive sufficient support 4
Table 6
Academic Language Code List
Components of Academic Language Frequency within
Classroom
Observations
Explicit academic and language Instruction (cognates 11
included)
English language practice and experimentation 52
Teacher elaboration of vocabulary and student 37
responses
Vocabulary instruction 34
Q2. How are successful content-based, sheltered instruction ELL

classrooms supported within their school? Data, specific to this question, was
analyzed from two sources within the multiple-case study. Data was collected
from individual interviews and the teacher focus group interview. Both sources
were necessary as to reveal the evidence derived from multifaceted perspectives
through triangulation (Lauckner et al., 2012). Within the individual interviews,

26
as well as the teacher focus groups, the participants were asked to reflect on the
unique attributes of the school and in what ways are the ELL classrooms
supported from the school/district level. Ultimately, they both affirmed the
necessity of support available of varying degrees within the school structure.
The individual interview participants who consisted of past and current
personnel within administrative positions at Wilson K-8 School, as well as the
teachers involved in the teacher focus group interview, presented similar
elements considered as essential academic support systems within the school.
Flexibility within scheduling and curriculum allows for additional personnel to
assist with small group and/or explicit instruction based on the students'
individual needs, as well as the administration encouraging teacher discretion
on the depth and breadth of content/language delivery necessary for each
classroom within a flexible schedule and curriculum. Both sources reported that
the self-contained student configuration created an entire system of support
within content-based, sheltered instruction ELL classrooms. Although the
specific school's climate and climate was mentioned as a support system that
overlapped the essential components, this area will be discussed in greater detail
within its corresponding sub-question.
SQ3. How are educators maintaining a culturally sensitive
environment that promotes the efforts of ELLs? The language data compiled
from interviews, classroom observations and focus groups offered insight on
how educators maintain an environment that facilitates the development of their
ELLs among their peers within content-based, sheltered instruction classrooms.
The triangulation of the data (Yin, 2014) indicated that the grade-level ELL
teachers and their students consistently work on ensuring a positive, classroom
community as to encourage all learners to feel they are part of a unit and that
they are capable of excelling in academics similar to their peers. Although the
code frequencies across the language data were mostly found in the classroom
community and teachers behaviors and beliefs code families, instructional
considerations also contributed to their successful environments.
Data collected from classroom observations and from their teachers, in
particular, indicated that a sense of community was created when the students
realized that collaborative peer groups were expected. Encouragement and the
respect of differences in individuals cultures, personality traits and language
skill sets were essential to build the class environment. We are a family.
Helping out comes naturally to many of our students.
From the evidence analyzed from the language data from the multiple
sources, as well as from the Danielson's classroom observation tool (2007),
educators maintain a culturally sensitive environment that promotes the efforts
of ELLs within content-based, sheltered instruction ELL classrooms. They
achieve this by delivering sound, effective instructional opportunities that
intentionally demonstrate to the English language learners that they are capable
of academic achievement similar to that of their peers. These opportunities must
also be provided within the context of positive classroom environments for all
students.
SQ4. In what ways can school climate and culture support content-
based, sheltered instruction classrooms? The schools climate and culture is

27
categorized as caring, supportive and knowledgeable, according to the

individual interview participants. Although 10 out of the 21 faculty members
worked at the school for at least seven years, the current administrator observed
the schools unique qualities within her first year working at the school. We
have a nurturing culture here at Wilson K-8. Teachers here go above and
beyond the four walls to support our students. I would say it is way more than
what I have seen at other buildings that I have worked in. Teachers support the
students who have language needs. The teachers are experienced in reading
students where they are and move them forward. There is compassion for what
students are going through in their everyday lives. Table 7 demonstrates the
holistic distribution of occurrences within the school culture family code across
the three sources of data. One interview participant noted that as a result of the
school community, the relationships between teachers, students and their
families were why many ELL families request building transfers to remain or to
enter this school specifically from within the school district.
Table 7
Components of the School Community
School Culture Primary Codes Overall
Frequency
Community among staff 9
Described as family 9
Parent participation and events 8
School community 33
Teacher and student academic advancement 20
The belief that all individuals can learn and work toward academic
advancement among students and teachers contributed to the successful climate
and culture of Wilson K-8. Not only was bilingualism viewed as a positive
attribute within students to accelerate students along academically, the teachers
at Wilson K-8, were also viewed as professional learners. The teachers were
knowledgeable as over 50% were ELL certified, as well as within their content
area. However, the teachers were learning effective strategies from each other.
One administrator explained professional development opportunities that
occurred among colleagues. The ELL classrooms often, many of them, became
models I could refer teachers to go visit. If they wanted to see how a particular
teaching/learning strategy was used, I could send them to observe a
classroom.We had a lot of strong teaching models, and teachers were learners.
They didnt stop learning. They learned from each other in professional learning
communities (PLCs). In their PLCs, they could learn from each other, they could
observe each other, and that was one of the strongest benefits that I think we had
in our building because there was such a wealth of good teaching in that
building and people were willing to try new things.
Data from this multiple-case study provides the manners in which school
climate and culture support content-based, sheltered instruction ELL

28
classrooms. As a result of their close professional relationships, the teachers and

other personnel at the school assist each other so that they can support the
academic advancement of their students. Because they are knowledgeable and
compassionate about the students' family and cultural biographies, they
problem-solve together to provide what ELLs and their families need to achieve
personally and academically.
Inductive reasoning and analysis was conducted to identify the themes
present across the various sources of data within the multiple-case study. Six
themes emerged through memo-ing and examining the frequency and
correlations between codes and within code families. These themes were
identified below (see Table 8) and are presented as equally vital to the success of
content-based, sheltered instruction ELL classrooms. Although the following
themes are representative and validated throughout all of the questions posed
during the multiple-case study, the themes are discussed in order of the main
research questions and their corresponding sub-questions. The overarching
concepts all relate to the members of the organization understanding their
population and addressing their students needs within their academic,
linguistic and sociocultural development as suggested by Thomas & Collier
(1997; 2002).
Table 8
Emerging Themes
Classroom configurations in language and learning
Explicit instruction by knowledgeable personnel
Student engagement for students of all language levels
Academic and structural flexibility
Celebration of culture, language, and learning
Functionality of its members as a support unit
Discussion
This descriptive multiple-case study was an analysis of factors present in
successful content-based, sheltered instruction ELL classrooms within an urban
K-8 school educating high populations of English language learners at the
elementary level. Data collected from individual interviews, classroom
observations and focus groups addressed the research questions. The findings
contribute to the affirmations presented within the theoretical framework of the
Prism model of language acquisition within schools (Thomas & Collier, 1997;
2002). Although the selected school did not demonstrate effective bilingual
education practices as recommended in the Prism model; the content-based,
sheltered instruction ELL classrooms were established in efforts to move
students academically within similar language communities. They exhibit
components that support students academically, linguistically and cognitively
within socioculturally supportive environments (Herrera, 2016; Thomas &
Collier, 1997; 2002). The implications from the findings of this study are
discussed below.

29
Classroom configurations in language and learning. The results form

this study support the notion that content-based, sheltered instruction ELL
classrooms maximize instruction by creating specially designed classroom
environments in which educators can provide explicit instructional practices
that unify academic and language objectives throughout all grade appropriate
content delivery necessary for English language learners (Clark, 2009; Short &
Echevarra, 2016; Genzuk, 2011). They address the importance of solid core
instruction within Tier 1 of the Response to Intervention (RTI) framework
(Echevarra & Hasbrouck, 2009) through the implementation of explicit
instruction utilizing the SIOP protocol designed specifically for these learners
(Echevarra, Vogt, et al., 2017). Findings from this study indicate that classroom
configurations within content-based, sheltered instruction ELL classrooms may
reduce the amount of ELLs in need of intervention at a more intense level as
their core instruction is designed specifically for their learner populations and
not of the traditional structure intended for the majority culture. These findings
are consistent with the extant research (Chang, 2008; Gndara et al., 2008).
Explicit instruction by knowledgeable personnel. Explicit
academic/language instruction, bilingual support, differentiated instruction and
participation, levels of support throughout lessons (Echevarra, Frey & Fisher,
2015) and multisensory teaching and learning were identified as the effective
instructional practices provided by ELL certified grade-level teachers across
content-based, sheltered instruction ELL classrooms (Chang, 2008) observed
during the study. Administrators stated there was a specific advantage to the
high number of certified grade-level ELL teachers with these skills so as to
provide consistency of effective instruction from year to year from kindergarten
to sixth grade within one school. Multiple years within the program was viewed
as vital to accelerating students academically and linguistically (Cosentino de
Cohen & Chu Clewell, 2007) as language proficiency is multi-directional and
unique for each student (Herrera, 2016).
Student engagement for students of all language levels. From the
learners' perspectives, students confirmed that working with partners and/or
groups engages and provides natural support during explicit content, strategy,
and/or language instruction (Swain, 2000). Regardless of English language
levels, all students benefit from additional time to negotiate for meaning (Short,
Echevarra, et al., 2011), experimenting with language within their own zones of
learning (Min, 2006), as well as working through new academic content with
others. By facilitating total student participation within lessons (which is
recommended by Herrera [2016] and Himmele and Himmele [2012]) ELLs feel
empowered and a sense of belonging within their learning communities
(Washburn, 2008).
Academic and structural flexibility. Flexibility within the school
organization became the overarching theme for how the classrooms in the
current study operate successfully within their school communities. Successful
content-based, sheltered instruction ELL classrooms are supported within the
school organization by facilitating academic and structural flexibility within all
aspects so as to tailor their instruction to the specific needs of their ELLs. Not
only is there educational autonomy with regards to essential content and
scheduling, educators make decisions to create appropriate matches between the

30
ELL service types available and the linguistic, academic and acculturation
experiences of the students. Echevarra and Hasbrouck (2009) placed a high
priority on the ability for schools to be able to make decisions based on their
ELLs individual cultural biographies (Herrera, 2016) within the core structure of
the school.
Celebration of culture, language, and learning. The student
participants embraced their sense of home within their content-based, sheltered
instruction ELL classrooms and among the school. In order to achieve this,
educators must maintain culturally sensitive environments that promote the
efforts of ELLs by facilitating instructional opportunities that reassure ELLs that
they can see themselves in their learning (Taylor, 2010). Particularly within the
contexts of positive classroom environments, English language learners levels of
language proficiency in L1 and L2 and their own life experiences (Dong, 2013)
are showcased and utilized intentionally throughout content lessons. Although
sociocultural theory remains prevalent within this theme, Cummins contextual
interaction theory (1996) justified the connection between the relationships of
successful language input, a supportive affective environment, as well as the
status associated within culture and language among peers and teachers
(Lavadenz, 2011). Content-based, sheltered instruction ELL classrooms become
a center for individual motivation through the celebration of culture, language
and learning.
Functionality of its members as a support unit. The school climate and
culture at Wilson K-8 supports the success within content-based, sheltered
instruction ELL classrooms by establishing a cohesiveness that is understood
and accepted by all (Rodriguez, Ringler, ONeal & Bunn, 2009). A high level of
support was noted as one of the key elements of success for these classrooms as
the strong-knit relationships between among administrators, teachers, students
and their families maintain a positive learning environment that flexes and
problem-solves to understand their learners (Washburn, 2008)). Social capital
theory substantiates the benefits of human relationships and the power to
achieve more through collective actions (Bourdieu, 1985). Essentially, the
functionality of the schools' members as a support unit provides the context and
conditions for excellence among elementary level English language learners at
this school.
Future Directions
The first recommendation for educational application is for schools to
expand on the variety of ELL service options available within a school within
increasing student populations of ELLs. This is necessary so that
knowledgeable, certified educators can appropriately match students of
different cultural biographies with optimal instructional and school climate
conditions specific to their needs (Pray & Monhardt, 2009). Content-based,
sheltered instruction self-contained and ELL integrated classrooms can provide
classroom environments that balance the academic, linguistic and sociocultural
needs of specific students as supported by the Prism model (Thomas & Collier
1997; 2002). Many of the administrators and teachers who participated in this
study stated the importance of placing elementary students in classrooms that

31
allow for explicit instruction and classroom engagement experiences that match
the specific academic, language and sociocultural needs of the students (Jang &
Jimenez, 2011; Murry, 2012). However, continuous professional development
opportunities that include the most recent research-based strategies within
explicit instruction, second language acquisition and instruction and the SIOP
model (Echevarra, Short, et al., 2008 must remain at the forefront as schools in
need must establish an effective educational framework for the influx of English
language learners (Genzuk, 2011; Reeves, 2009; Taylor, 2010 ).
The second recommendation is that a team, consisting of a school
administrator and ELL personnel could be assembled within the schools. Their
focus would be to assist with the academic, structural and sociocultural
flexibility necessary to appropriately screen and monitor the large populations
of ELLs (Brown & Sanford, 2011) through collaboration, instructional planning,
and/or direct student support in efforts to continue to make measurable gains in
English language proficiency (Anderson & Dufford-Melendez, 2011).
Ultimately, the ELL support team could work alongside a school's RTI team
(Brown & Sanford, 2011; Caldern, Slavin & Sanchez, 2011; Echevarra &
Hasbrouck, 2009) to maintain solid core instruction while increasing the
sociocultural experiences by way of strong interrelationships between teachers,
students and their families (Good et al., 2011).
Recommendations for further research. Future studies could be
performed to expand on the factors for success within content-based, sheltered
instruction ELL classrooms. The first recommendation could be to replicate this
study, but data from observations could be collected throughout a longer period
of time during one school year as it would provide longitudinal data that could
represent a more accurate depiction (Cozby, 2014) of the effective instructional
practices and classroom environments of each classroom. Additionally, an
increase in the number of ELL classrooms observed could also contribute data in
the same manner within the multiple-case study.
The second recommendation for further research could add an additional
student focus group interview session of current students who have participated
in content-based, sheltered instruction ELL classrooms, but have provisionally
exited out of ELL services as they have accomplished academic and linguistic
proficiency at their grade-level. This could increase the depth into the students'
perspectives of their experiences (regarding instructional practices and their
classroom environments within their previous content-based, sheltered
instruction ELL classrooms at Wilson K-8 School). Most importantly, they could
offer additional insight into the program's success overall as the students
participated since the beginning of their journey towards English language
proficiency within content-based, sheltered instruction ELL classrooms within
the schoolwide model (Kang, 2010; Rodriguez et al., 2009).
Finally, the last recommendation could examine the difference between
the mission of this school's content-based, sheltered instruction ELL classroom
model and their classroom configurations to include integrated and self-
contained and establish clear definitions of the expectations and achievements
when comparing the existing English-only models. Further research analysis of
existing ELL classrooms to the classrooms highlighted within this study may

32
lead to additional contributing factors for success that may have been
overlooked in this study.
Conclusions
A qualitative multiple-case study was conducted to identify the essential
attributes and relationships (Yin, 2014) that contributed to the successfulness of
content-based, sheltered instruction elementary classrooms. Minimal research
exists of ELL classroom models with similar characteristics addressing the needs
of large populations of ELLs efficiently and utilizing English as the language of
instruction within one school. In addition to the urging pressures of upcoming
achievement standards (Alberti, 2013; Schmidt & Burroughs, 2013) and new
teacher evaluation tools (Danielson, 2012; Marzano, 2012), school districts across
the nation struggle to identify effective solutions to educate new, large
populations of ELLs (Guccione, 2011) effectively in the quickest amount of time
without providing bilingual education. The findings of this study establish
attributes of successful content-based, sheltered instruction ELL classrooms that
can serve as a viable method for many school districts in need of solutions.
Six themes emerged from the triangulation of the data collected from
interviews, observations and focus groups which answered the research
questions: 1) classroom configurations in language and learning; 2) explicit
instruction by knowledgeable personnel; 3) student engagement for students of
all language levels; 4) academic and structural flexibility; 5) celebration of
culture, language and learning; and 6) functionality of its members as a support
unit. The implications were organized and presented by the themes that arose
from the findings of the multiple-case study. However, these relationships
indicated within the study mirror the areas of development necessary for
English language proficiency in schools as indicated by Thomas and Collier
(1997; 2002). Thus, the fundamental implication of this qualitative, multiple-case
study is that all attributes and relationships must interact and depend on each
other so as to create successful content-based, sheltered instruction ELL
classrooms for its students.
The two recommendations for application based on the findings of the
study have been discussed through the perspective of improving the quality of
services English language learners receive in school districts with rising ELL
populations, in particular. School districts must develop their ELL programs so
several models are available within each school with a large population of ELLs
in the elementary grades. Because ELLs obtain their own unique cultural
biographies (Murry, 2012) that include academic experiences in their native
language, the type of instructional service and classroom environment must also
vary so as to meet these learners' needs efficiently. An ELL team of
professionals within each building can assist with these student cases and offer
professional development, instructional strategies, as well as hands-on support
in the classroom.
Further investigation of successful content-based, sheltered instruction
ELL classrooms is suggested to increase the validity and reliability of the results
of this study. It is recommended that the study be conducted on a larger scale to
include classroom observation within content-based, sheltered instruction ELL

33
classrooms multiple times throughout the school year. Longitudinal research

could provide a deeper holistic perspective of the instructional practices and
positive classroom elements. Future researchers may wish to include students
who have provisionally exited the ELL program and attend classes in
mainstream grade-level classrooms in an additional student focus group
interview session. Finally, a study examining the attributes and expectations of
other ELL classrooms within the existing literature compared to this studys
findings would allow school districts to select content-based, sheltered
instruction classrooms as described in the study as the ideal ELL service model
when bilingual education is not available to address many ELLs at once.
Ultimately, school districts may invest additional resources and ethical support
for the learners on the path to academic and English language success within
their school organizations.
References
Abilla, P. (2010). Affinity diagram. Retrieved from http://www.shmula.com
Alberti, S. (2013). Making the shifts. Educational Leadership, 70(4), 24-27.
Anderson, K.S. & Dufford-Melendez, K. (2011). Title III accountability policies and
outcomes for K-12: Annual measureable achievement objectives for English
language learners in Southeast Region states. (REL-No. 105). Washington,
D.C.: U.S. Department of Education, Institute of Education Services,
National Center for Education Evaluation and Regional Assistance.
Retrieved from http://ies.ed.gov/ncee/edlabs.
Bourdieu, P. (1985). The forms of capital. In J.G. Richardson, Handbook of Theory
and Research for the Sociology of Education, (pp. 241-258). New York:
Greenwood.
Brown, J.E. & Sanford, A. (2011). RTI for English language learners: Appropriately
using screening and progress monitoring tools to improve instructional
outcomes. Washington, D.C.: National Center on Response to
Intervention.
Bushe, G.R. (2011). Appreciative inquiry: Theory and critique. In Boje, D. Burnes,
B. and Hassard, J. The Routledge Companion to Organizational Change, 87-
103, Oxford, UK: Routledge.
Caldern, M., Slavin, R. & Sanchez, M. (2011). Effective instruction for English
learners. The Future of Children, 21(1), 103-127.
Cantore, S. P. & Cooperrider, D.L. (2013). Positive psychology and appreciative
inquiry. The Wiley-Blackwell Handbook of the Psychology of Leadership,
Change, and Organizational Development, 267-287. doi:
10.1002/9781118326404.ch13
Chang, M. (2008). Teacher instructional practices and language minority
students: A longitudinal model. The Journal of Educational Research, 102(2),
83-97, 160.
Clark, K. (2009). The case for structured English immersion. Educational
Leadership, 66(7), 42-46.
Cosentino de Cohen, C. & Chu Clewell, B. (2007). Putting English language
learners on the educational map: The No Child Left Behind Act

34
implemented. Education in Focus. Washington, D.C.: The Urban Institute.

Cozby, P.C. (2014). Methods in Behavioral Research. New York: McGraw Hill.
Danielson, C. (2012). Observing classroom practice. Educational Leadership, 70(3),
32-37.
Danielson, C. (2007). Enhancing professional practice: A framework for teaching.
Alexandria, VA: Association for Supervision and Curriculum
Development (ASCD).
Dong, Y.R. (2013). The Bridge of Knowledge. Educational Leadership, 71(4), 30-36.
Echevarra, J., Frey, N.S. & Fisher, D. (2015). What it takes for English learners to
succeed. Educational Leadership, 72(6), 22-26.
Echevarra, J. & Hasbrouck, J. (2009). Response to intervention and English
learners. CREATE: Center for Research on the Educational Achievement and
Teaching of English Language Learners Washington, D.C.: Institute of
Education Services.
Echevarra, J., Richards-Tutor, C., Canges, R. & Francis, D. (2011). Using the
SIOP model to promote the acquisition of language and science concepts
with English learners. Bilingual Research Journal, 34(3), 334-351.
Echevarra, J., Short, D. and Powers, K. (2008). Making content comprehensible
for non-native speakers of English: The SIOP model. The International
Journal of Learning, 14(11), 41-49.
Echevarra, J., Vogt, M. & Short, D.J. (2017). Making content comprehensible for
English learners: The SIOP Model. New York: Pearson Education.
Fayden, T. (2011). Forming the forms of higher intelligence: Further than
standards. NABE News, 33(1), 16-20.
Fortuny, K., Capps, R., Simms, M. & Chaudry, A. (2009). Children of
immigrants: National and State Characteristics. Perspectives on low-income
working families. Washington, D.C.: The Urban Institute.
Friese, S. (2012). Qualitative data analysis with ATLAS.ti. London: Sage.
Gaddis, S.E. & Williams, C. (2009). See yourself in 4-D: How to use appreciative
inquiry to ignite positive change. Retrieved from
http://www.communicationsdoctor.com/articles/seeyourselfin4-D.pdf
Gndara, P. & Orfield, G. (2010). A return to the Mexican room: The segregation of
Arizonas English learners. Civil Rights Project, University of California,
Los Angeles, CA. Retrieved from ERIC database. (ED 511322)
Gndara, P., Maxwell-Jolly, J. & Rumberger, R. (2008). Resource Needs for English
Learners: Getting Down to Policy Recommendations. UC Santa Barbara:
Linguistic Minority Research Institute.
Garza, A. V., & Crawford, L. (2005). Hegemonic multiculturalism: English
immersion, ideology, and subtractive schooling. Bilingual Research
Journal, 29(3), 599-619,728.
Genzuk, M. (2011). Specially designed academic instruction in English (SDAIE) for
language minority students. Center for Multilingual, Multicultural
Research Digital Papers Series. Center for Multilingual, Multicultural
Research, University of Southern California. Retrieved from
http://www.usc.edu/dept/education/CMMR/DigitalPapers/SDAIE_
Genzuk.pdf

35
Good et al., 2011).

Guccione, L.M. (2011). Integrating literacy and inquiry for English learners. The
Reading Teacher, 64(8), 567-577.
Herrera, S.G. (2016). Biography-driven culturally responsive teaching. New York:
Teachers College Press.
Himmele, P. & Himmele, W. (2012). How to know what students knows.
Educational Leadership,70(1), 58-62.
Jang, E.Y. & Jimenez, R.T. (2011). A sociocultural perspective on second
language learner strategies: Focus on the impact of social context. Theory
into Practice, 50, 141-148. doi: 10.1080/00405841.2011.558443
Johnson, R.B. Christensen, L.B. & Turner, L.A. (2014). Research methods, design and
analysis. Saddle River, NJ: Pearson.
Kang, A. (2010). The contribution of L1 CALP and L2 proficiency to successful
CBI experience. English Teaching, 65(4), 3-39.
Lauckner, H., Patterson, M., & Krupa, T. (2012). Using constructivist case study
methodology to understand common developmental process: Proposed
methodological questions to guide the research process. Qualitative
Report, 17, 1-22.
Lavadenz, M. (2011). From theory to practice for teachers of English learners.
CATESOL Journal, 22(1), 18-47.
Marzano, R.J. (2012). The two purposes of teacher evaluation. Educational
Leadership, 70(3), 14-19.
Miles, M.B., Huberman, A.M. & Saldana, J. (2013). Qualitative data analysis: A
methods sourcebook. Thousand Oaks, CA: Sage.
Min, G. (2006). Vygotsky's sociocultural theory and the role of input and output
in second language acquisition. CELEA Journal, 29(4), 87-92.
Murry, K. (2012). Cognitive development, global learning, and academic
progress: Promoting teacher readiness for CLD students and families.
Journal of Curriculum and Instruction, 6(1), 11-24.
National School Reform Faculty (2012). Appreciative inquiry: A protocol to support
professional visitation. Retrieved from http://www.nsrfharmony.org
Patton, M.Q. (2015). Qualitative research and evaluation methods: Integratinge theory
and practice. Thousand Oaks, CA: Sage.
Pray, L. & Monhardt, R. (2009). Sheltered instruction techniques for ELLs: Ways
to adapt lessons to meet the academic needs of English language
learners. Science and Children, 46(7), 34-38.
Ramos, F. & Krashen, S. (2011). Success without bilingual education?:
Acquisition of academic language in English through comprehensible
input. NABE News, 33(4), 13-14.
Reeves, D.B. (2009). The value of culture. Educational Leadership. 66(7), 87-89.
Rodriguez, D., Ringler, M., O'Neal, D. & Bunn, K. (2009). English language
learners' perceptions of school environment. Journal of Research in
Childhood Education, 23(4), 513-526.
Schmidt, W.H. & Burroughs, N.A. (2013). How the Common Core boosts quality
and equality. Educational Leadership, 70(4), 54-58.
Short, D.J. & Echevarra, J. (2016). Developing academic language with the SIOP

36
Model. Boston: Pearson.

Short, D. J., Echevarra, J., & Richards-Tutor, C. (2011). Research on academic
literacy development in sheltered instruction classrooms. Language
Teaching Research, 15(3), 363-380. doi:10.1177/1362168811401155
Short, D.J., Fidelman, C.G., & Lougit, M. (2012). Developing academic language
in English language learners through sheltered instruction. TESOL
Quarterly, 46(2), 334-361.
Shutt, R.K. (2012). Investigating the social world: The process and practice of research.
Thousand Oaks, CA: Sage.
Starnes, B.A. (2010). Rethinking diversity. Kappan, 92(1), 74-75.
Stevenson, H. (2011). Appreciative inquiry: Tapping into the river of positive
possibilities. Cleveland Consulting Group. Novelty, OH. Retrieved from
http://www.clevelandconsultinggroup.com/articles/appreciative-
inquiry.php
Swain, M. (2000). The output hypothesis and beyond: Mediating acquisition
through collaborative dialogue. In J. Lantolf, Sociocultural theory and
second language learning, (pp. 97-114). Oxford: Oxford University United
Press.
Taylor, R.W. (2010). The role of teacher education programs in creating
culturally competent teachers: A moral imperative for ensuring the
academic success of diverse student populations. Multicultural Education,
17(3), 24-28.
Thomas, W.P. & Collier, V. (2002). A national study of school effectiveness for
language minority students long-term academic achievement. Center for
Research on Education, Diversity and Excellence, Santa Cruz, CA.
Retrieved from ERIC database. (ED 475048)
Thomas, W.P. & Collier, V. (1997). School effectiveness for language minority
students. National Clearinghouse for Bilingual Education, Washington,
D.C. Retrieved from ERIC database. (ED 436087)
Trochim, W.M.K. & Donnelly, J.P. (2008). The research methods knowledge base.
Mason, OH: Atomic Dog.
Washburn, G.N. (2008). Alone, confused and frustrated: Developing empathy
and strategies for working with English language learners. The Clearing
House, 81(6), 247-250. Retrieved from
http://search./proquest.com/docview/196844858.accountid=28180
Yin, R.K. (2014). Case study research: Design and methods. Los Angeles: Sage.

37

Vol. 16, No. 8, pp. 37-50, August 2017
Radio Wave Errors: Students Mistaking

Radio Transverse Electromagnetic Light Waves
as Longitudinal Sound Waves
A. E. Tabor-Morris, Ph.D., T. M. Briles, Ed.D. and R. Schiele, B.S.

Georgian Court University
Lakewood, New Jersey, USA
Abstract. Commonly anecdotally noted among physics instructors is

that students often misidentify radio-waves as sound waves, not as part
of the electromagnetic light-energy spectrum. To highlight the
prevalence of this error, a pilot survey, whose results are presented here,
was made of a total of 225 high school physics students from four high
schools in New Jersey in the USA, taken immediately after students had
covered both sound and electromagnetic radiation. Note that although
the study is made in one locality, there is likelihood that the same data
would be obtained in any introductory physics classroom and future
studies are suggested. This survey suggests that a majority of students
appear to still incorrectly conclude that 'radio waves' are sound, even
after instruction otherwise. This is perhaps reinforced by students'
sensory illusion interpretation which might be articulated as: "I hear a
radio, I experience 'radio broadcasts' as sound, so if 'radio signals' are
'radio waves', they are hence sound waves". The survey results were
also sought to see if students who responded that "radio waves are
sound" in this study, more consistently answered other related questions
that used that assumption - that is, once they made that decision, did
they stick with it, even when not consecutively asked the questions in
the survey? Or is it possible that aspects of questioning or syllabi can
mislead students? To help teachers assist students in properly
identifying radio waves some ideas are suggested, particularly directly
challenging students to realize that this is a frequent misunderstanding.
Keywords: Radio Waves; Physics Education; Astronomy Education.
Introduction
Are radio waves sound waves or part of the electromagnetic spectrum?
and are they hence longitudinal or transverse waves? These are common
questions from physics teachers to their students when studying radio waves.
Yet, anecdotally, students often get these questions wrong. Why do physics
teachers on both the secondary level and in higher education often refer to
difficulties in teaching the electromagnetic spectrum with a sense of frustration,

38
especially regarding students frequent misidentification of radio waves as

longitudinal sound waves and not as transverse electromagnetic waves after
being taught otherwise? Are there conceptualizations that students are making
regarding the physical phenomenon of radio transmissions that make it difficult
to communicate the correct physics of radio signals to students? Are more
stringent warnings to students of difficulties needed when teaching sound
waves and electromagnetism?
The purpose here is to document the prevalence of the existence of this
difficulty and possible implications of students misidentification of radio waves
as sound instead of as electromagnetic waves in the radio frequency range, an
invisible portion of the light spectrum. For this pilot study, the students
completed a paper-pencil instrument to show their understanding of this
material. Also, how strong is the possibility that students may carry through
their conclusions that radio waves as sound or light to other questions? It is also
the first published study to attempt to quantify the prevalence of this error, and
also to point out the difficulty of producing testing questions that do not mislead
students via pointing out the weakness of the survey tool used.
Note that due to its fundamental nature, we use the term light (as is done
in many physics textbooks) to mean both the visible portion of the
electromagnetic spectrum and also invisible portions of the electromagnetic
spectrum such as radio waves, microwaves, infrared, ultraviolet rays, X-rays,
and gamma rays. The choice to confine the definition of light to just visible light
appears to be increasingly viewed as abstruse and unusual, hence the
terminology choice made here.
Background
The need for, and difficulties of, students to understand the nature of
radiation, radio waves, sound waves and radio signals is noted in both the
physics and astronomy communities (Berger, 2015; Barder et al., 2005; Landt,
2015; McGuinnes & Oliver, 1998; Plotz & Hopf, 2016; Newmann & Hopf, 2012;
Rego & Peralta, 2006). Radio waves are introduced in most physics textbooks
during the discussion of waves and then again after sound when
electromagnetic radiation is introduced. Astronomy textbooks do so when they
introduce the concept of light. Physics textbook author Paul Hewitt generated a
thought problem on this important misunderstanding in his Figuring Physics
series available as a Next Time Question (Hewitt, 2007). Other suggestions and
helps in addressing teaching radio waves are available in Perkins et al. (2006),
Wise (2006), DeVries (2001), and Finkelstein et al. (2006). The question remains
of why would students think of radio waves, transmissions from one radio to
another radio tower as sound waves?
It should be noted that when one thinks of a radio, one thinks of the
sound it produces such as listening to a car radio or a radio station on ones
cell phone. Students might then easily misinterpret radio waves as being the
same as radio signals. A radio signal does change forms: from sound to electrical
signal inputted to a transmitting antenna, to electromagnetic (EM) radio waves
that travel to a detecting radio antenna, which in turn produces electrical signals
that are converted to sound output that is heard, where the receiving radio
itself is acting like a translator box and producing a sound wave. If one is simply

39
considering the beginning and end product, it only makes sense to consider
radio signals as sound waves. In that sense, physics learning that asks the
question does the physics make sense in everyday life based on what is sensed
and are students navely putting in a situation of observation to be deceived by
sensory illusions and correlation to everyday experiences (Tabor-Morris, 2015;
Caramazza et al., 1981) which can mislead if all aspects of the radio signal are
not considered? Also the intricacy of signal changing forms may be only briefly
addressed in class, perhaps because physics teachers tend to try to stick to the
basics such as saying radio waves transmit signal and ignore the other
aspects of radio signals such as signals through electronics before and after the
radio waves are sent, and the fact that the original signal was a sound wave and
the final product is a sound wave, such as music listened to on a radio
(Lazebnik, 2002).
In addition, other examples given in class may lend to the misconception.
For example, some radio telescope dishes, even in current astronomy, have been
construed to resemble ears and are sometimes even referred to as listening to
outer-space. Others, perhaps unfortunately, have been identified by the
colloquial term of horns, such as the horn antenna used by Penzias and
Wilson in the 1960s to map radio signals from the Milky Way leading to the
discovery of the Cosmic Microwave Background.(APS website online).
However, is it possible that some other explanations and demonstrations are
reinforcing this misinterpretation rather than clarifying it?
Education research indicates also that errors are often actualized not only
from direct input but by organizing and reorganizing which initially may be
fragmented (knowledge in pieces) and re-evaluating misconceptions such as
nave theories (Tuminaro & Redish, 2007; Bao & Redish, 2006; Disessa &
Sherin, 1998, Carmazza et al., 1981; Etkina & Ruibal-Villasenor, 2008). Wave
types might be considered in class in different contexts (such as a discussion of
sound and then later light) that are never fully connected for students.
Purpose and methodology of survey

The source, nature, and consequence of the described errors is explored
in this article, which reports on a pilot study (via a paper survey) to physics
students in several high schools. A short seven question multiple choice survey
of students was prepared and distributed in the classroom by the high school
physics teacher. The reasoning for the questions in the pilot survey was based on
frequent multiple-choice questions types similar to those often asked/tested on
light and sound to high school physics students with the goal to obtain data on
student responses on this subject, following physics education research models
(Ding & Liu, 2012). The objective was to test the idea that students may have
trends in how they answer these multiple-choice questions.
The survey was reviewed and approved by the Universitys six-scientist
Institutional Research Review Board (IRRB) prior to dissemination and was also
evaluated by an outside evaluator from the Social Work Department and one
from the School of Education (with pre-college teaching experience) at our
University for face and content validity in addition to bias. This study was
intended as a first look at this problem and outside expertise was not sought in
this initial study but future survey tools will be subject to more rigorous

40
scrutiny. Methodology in the selection of the high school students taking physics
was based on availability of the test subjects via the cooperation of local high
schools which included parental permission as all test subjects were minors.
The high schools were surveyed after students had covered
electromagnetic radiation in their spring semester (April), as verified by a survey
to their teachers. Data was collected with a total of 225 students (that is, N=225,
where N is a variable equal to the number of subjects) participating from 4 high
schools (mix of public and private institutions) with a total of 7 different classes
who participated in this survey. No record of who the teachers of these classes
was made, nor if any of these teachers taught multiple courses that were
surveyed. All the students in this study were from the same age group strata
(high school juniors and seniors of ages 16-18 years old) with the data taken in
only one particular year, in 2012; hence, this study is only representative of a
single cohort based on age, year, and demographic location. Distribution to
other cohorts was never initiated due to inherent concerns in the study as will be
highlighted.
The demographics of the students in the study were such that all were
from central/southern New Jersey and were in basically the same economic
strata: the average student was from a middle-class family not under financial
stress. The students were taking a physics course from one of all levels of
physics including general, college preparatory, honors and advanced placement
(AP) and all were juniors or seniors in high school; no Physics First (freshman)
classes were surveyed as these schools did not have that program. Students were
not asked personal questions regarding family, wealth, gender identity,
ethnicity, minority status or whether or not their parents were college graduates.
Each student completed a paper survey in their physics classroom. The
survey was administered in the selected high school physics classrooms by the
high school physics teachers. These surveys were returned to the researchers
and tabulated. This study could not be further stratified since the surveys were
delivered back from the schools bundled without differentiation between
classes; hence, the experiences due to a variety of physics class levels or teachers
(who might teach various levels) could not be segregated for further analysis,
although individual class sizes were small; and hence, statistically insignificant
as stand-alone units.
This surveys results are presented mostly in a qualitatively-descriptive
manner and the discussion of the survey study is meant to serve as a pilot for a
possible larger group sampling, although the sample size was adequate, within
common and appropriate apriori parameters (Apriori Calculator online). Even
given the limitations of this study, some interesting trends can be noted. Details
of the survey instrument are given in the next section along with the results.
Survey and results

Table 1 is a summary presentation of all the results obtained in this
study. Note that consideration of whether high school students would take such
a survey seriously should be made (Kalat, 2010). We did make this
consideration. Firstly, the survey was distributed in students physics class by
their physics teacher, a place where students would be expected to answer
questions thoughtfully, being in the classroom or laboratory where they had

41
learned the material. Secondly, all students answered all the questions in the
survey, indicating that they took it seriously. Thirdly, Question #5, an easy
stand-alone question, had a high percentage of correct answers. It was a
question embedded deep in the survey so unlikely to be prompted by a teacher,
and had an overall 92% correct percentage and only one class (High School 3)
with only one notable lower percentage (72%), possibly not having covered this
or emphasizing it less. Conclusions likely can be drawn that students took the
survey seriously, noting that distractor answers in that question an energon is
a make-believe particle, a pion likely would not (yet) have been discussed in a
high school physics class, nor would a phonon.
Table 1: Physics questions asked high school students (with percentages normalized
to N=225)
Note: numbers may not add to 100% as numbers were rounded.
Overall High High High High
Question Text Percentage School 1 School 2 School 3 School 4
# (correct answer is Correct Percentage Percentage Percentage Percentage
underlined) Correct Correct Correct Correct
(N=225) (N=104) (N=43) (N=11) (N=67)
1 Light is a ___ wave.
a.) longitudinal 33% 41% 41% 45% 12%
b.) transverse 67% 59% 59% 55% 88%
2 Radio waves are a

form of sound
waves.
a.) true 60% 57% 48% 64% 70%
b.) false 40% 43% 51% 36% 30%
3 Electromagnetic
waves have zero __.
a.) mass 83% 77% 88% 63% 91%
b.) wavelength 4% 5% 5% 0 3%
c.) energy 3% 5% 2% 9% 0
d.) frequency 10% 13% 5% 27% 6%
e.) velocity 0 1% 0 0 0
4 The electromagnetic
spectrum, in order
from lowest energy
to highest.
a.) x-ray,microwave, 14% 18% 16% 9% 8%
infrared, visible,
ultraviolet
b.) visible, infrared, 21% 25% 30% 27% 9%
ultraviolet,
microwave, x-ray
c.) x-ray, infrared, 11% 14% 16% 18% 0
microwave,

42
ultraviolet,
visible
d.) microwave, 46% 37% 25% 9% 80%
infrared, visible,
ultraviolet, x-ray
e.) microwave, 7% 6% 9% 36% 3%
x-ray, ultraviolet,
visible, infrared
5 Another name for a

bundle of light is a
a.) phonon 1% 0 2% 0% 0
b.) photon 92% 92% 93% 72% 95%
c.) proton 3% 2% 2% 9% 5%
d.) pion 2% 2% 2% 9% 0
e.) energon 2% 4% 0 0 0
6 Radio waves travel

in air at a speed that
is __ the speed of
SOUND.
a.) slower than 20% 22% 25% 27% 13%
b.) the same as 44% 39% 35% 36% 57%
c.) faster than 36% 39% 40% 36% 30%
7 Radio waves travel

in air at a speed that
is __ the speed of
LIGHT.
a.) slower than 71% 71% 65% 72% 70%
b.) the same as 21% 18% 25% 18% 22%
c.) faster than 8% 11% 5% 9% 8%
Note that a glaring 8% of the population surveyed said incorrectly that

radio waves could travel faster than the speed of light, when in most high school
classes, the speed of light is expected to be proclaimed to new learners as the
universal speed limit. However, it should be noted that no collection of syllabi of
the students was collected and hence no correlation made to what students were
presented with which may have missing items from those usually expected, at
least in this study.
The introductory survey question and implications

It was desirable that the survey be short since the aim was to sample
students knowledge and not have them realize that their answers were incorrect
and to go back and change them. The survey was meant as an expression of
student knowledge, and not a learning experience. The first question was

43
considered a control question, to see if students remembered light was a

transverse wave, and to determine what the overall rate of student correct
answers would be to a technical course-based question.
1.) Light is a ________ wave.
A) longitudinal INCORRECT
B) transverse - CORRECT
Overall average results in student responses was tabulated with 32%

incorrect and 68% answering correctly. However, before proceeding, let us also
take a look at the performance of individual schools (See Fig. 1).
Figure 1: Question #1 results (Light is a longitudinal/transverse wave):

Percentage correct vs. high schools surveyed.
Except for one school, the spread of answers was similar. The glaringly
large percentage correct (at nearly 90%) on Question #1 for High School 4 (likely
two classes but maybe more, at N=67), but lower scores on later questions may
indicate that possibilities:
1.) topic of light had possibly just recently been covered; but possibly
radio waves were not covered or at least not in much depth,
2.) possibly that the teacher(s) at this school is (are) doing a better job
overall in achieving better results in student learning on that
topic, and/or
3.) teacher(s) may have prompted the answer to Question #1 to the
students, for example, such as by saying doing this survey

44
remember how we covered, or some other similar, even more

outright, prompting. However, if the answer had just been
given, the return rate would likely have been 100%. It is a topic
for future studies.
Having an initial question to buffer was not intentional but in retrospect
seemed good practice, especially as teachers would be unlikely to prompt more
than one question. Note that it might be difficult to get a third party to
administer the survey.
Excluding the results from High School 4, the percentage rate of correct
answers was a more moderate 58% correct and more consistent, within 3
percentage points. Still this is a discouraging statistic, considering that all
students in this study would have recently covered this topic. However, the
unfamiliarity of the words (longitudinal and transverse) and students only
recently being introduced to them, and not seeing them often might lend to less
accuracy in answering the question.
Rate of students incorrectly identifying radio waves as sound

The questions of most interest for this article in which physics students
incorrectly identify radio waves as sound are Questions 2, 6, and 7. Logically, a
percentage of students answering Questions 2 incorrectly might also tend to
answer Question #6, and Question #7 incorrectly but consistently (that radio
waves were sound). Figure 2 highlights these answers per high school.
Question #2: Radio waves are a form of sound waves.
True: N=134 (60%) incorrect

False: N=91 (40%) correct

45
Figure 2: Percentage correct VS. Questions regarding Radio Waves (as light)
Averaged results showed 60% of these high school physics students were
identifying radio waves as sound. However, there also exists the possibility that
students are mistaking radio waves for radio signal, as mentioned
previously, which would, as heard from a radio, indeed be sound (Lazebnik,
2002).
Also, out of those 134 incorrect answers, 82 also answered Question #6
that radio waves travel at the same speed as sound a percentage rate of
82/134X100= 61% of those students who specifically answered Question #2 that
radio waves were sound who were sticking with that idea that radio waves
travel at the speed of sound in Question #6, higher than the overall percentage at
44%.
Of the 82 who answered that radio waves were sound in Question #2
and Question #6, there were 73 who answered in Question #7 that radio waves
travelled less than the speed of light. Hence 89% of those seemed to be sticking
to the idea that radio is sound. That is 32% of the entire population (N=225)
surveyed, a possible indication that this portion of the students were again
following through on the idea that radio is sound. This seems to be defined as a

46
nave theories as discussed in the background section (Tuminaro & Redish,

2007; Bao & Redish, 2006; Disessa & Sherin, 1998, Carmazza et al., 1981; Etkina &
Ruibal-Villasenor, 2008). Students may, once committed, not realize that they are
being deceived by what could be termed a sensory illusion (Tabor-Morris, 2015)
and/or be over-correlating their experience of everyday experiences in a way
that is not physical (Caramazza et al., 1981).
Students correctly answering and following though that radio waves

are light
Of the 91 students who answered Question #2 correctly that radio waves were
light, 34 also correctly answered Question #7 that radio waves travel at the
speed equal to the speed of light. So, it appears 34/91X100= 37% of those
students followed through with their idea that radio waves are light into that
question.
The index of refraction effect on the speed of light would be negligible,
something some students might (or might not) have considered, in which then,
assuming students follow through, would raise the percentage of correct
answers to that is, including a and b as correct from 37% to (89 answering
either a or b) a total of 97%. However, it is unclear that students would be
thinking along those lines. This is clearly an area for future research and survey
modification.
Correlating this back to Question #6, if students were following the logic
that radio is light, they would have answered that radio waves travel faster than
sound. Of the students who answered that radio waves travel at the speed of
light (only), 28 or 82% indicated that radio waves travel faster than sound
(assumedly at the speed of light, as they answer in the next question). This is
12% of the entire student population surveyed, a lower overall percentage of
those who appeared to follow through on the logic for radio waves as sound.
Possible changes to questions in survey in future research

Question 6 and 7 might have the in air portion of the phrase also put at the
end of the sentence in both cases so that any confusion regarding the effects of
index of refraction could be eliminated. There also exists the unexplored
possibility that some students may be identifying radio waves, instead of
electromagnetic waves, but instead as AC-type electrical signals. Question #2
could be expanded to include (c.) electrical signal and/or a Question #8 could be
added that distinguishes students thoughts about which is fastest could be
added:
Which is fastest? Speed of a.) sound in air, b.) light in vacuum, c.)
electricity in metal, expounding on past problems seeking to be addressed.
Noting that electricity travels faster than sound but slower than light in
vacuum (Halliday, 2007), this question might allow students to better qualify
radio waves or at least distinguish them from other phenomenon. Student
attitudes among a particular cohort, with participants representing a single
strata and age but inspires the research question whether the results could be
applied more broadly to physics, such as on the undergraduate level. Giving the
same tests to the same class twice within a short period of time is used to judge
the stability of the testing (Engelhardt, 2009). This was not completed in this case

47
as this was always intended to be a pilot study and questions were not Leichardt
gray-scale, and therefore less likely to be answered differently if asked
consecutively to the same audience, with the assumption being that learning is
stable, which would be a separate study. Cohort and longitudinal studies with
pre- and post-tests are also suggested for the future.
In addition, the responses of students in southern New Jersey (mostly
suburban) may or may not be representative of all students nationally or
internationally. There is also less racial and/or economic diversity than in many
other populations in the United States and elsewhere. However, given that the
survey was taken in a region of the country that was somewhat culturally
homogenous and overall non-stressed financially, these stressors and
differentiators most likely were not any significant factor in, for example,
student-to-student interactions.
For this study, a cross-sectional sampling across many age groups could
not be made since most people take a physics class in a short period/time of
their life only, although college students could also be studied in the future and
may be of interest due to their further maturity and possibility of having taken a
physics course in high school. A longitudinal study was also not possible for this
group to investigate if their knowledge and attitudes carried though to future
classwork. This was due to the fact that the students were not identified as
individuals and are in high school only a short period of time before graduation.
Hence after high school graduation students and their responses would not be
traceable as either individuals or a unit, since limited communication is
maintained by high school graduates with their alumni schools. In the future, a
larger sampling of students could be made and several years of data from
students who are juniors/seniors in those years. Also, students on the college
level were not tested but would be of interest.
Conclusion and Suggestions

Many students have difficulty correctly identifying radio waves as
transverse light waves. The primary purpose of this study was to highlight that
the problem exists on a statistically significant level and is worthy of further
study and active remediation by teachers. Based on the results of the survey-tool
in this pilot study, a large percentage of students appear to conclude that, since
they experience radio broadcasts as sound, then sound waves are the actual
transmission of radio signals, at least more so than that for light. That things are
not always what they seem is something scientists are used to analyzing. How
can teachers prepare students for conditions when what seems initially obvious
is more subtle?
Teachers awareness and addressing this problem directly may be
essential to assisting student learning. Note that it could be easy to see that
students might interpret the term "radio waves" as waves emitted by a (hand-
held) radio - which would be sound, but would be more concerning is if the
misconception extended to students believing that radio broadcasts (from base)
are transmitted as sound (and somehow amplified by a radio). A future survey
could address this issue. For the purposes here, it is suggested that teachers alert
students to this often made error is one possible method to help remediate
student confusion.

48
Can teachers better identify to students that a radio is a translator box

that translates radio signals into sound? Questioning students on the steps for
radio communication might also be helpful in sorting out their understanding of
the process. Also helpful may be identifying to students that radio waves are
electromagnetic radiation in a range invisible to the eye, in a sense, invisible
light with better distinction between radio signals and radio waves? Radio
stations are continually broadcasting radio waves on many frequencies (We are
now awash in radio waves but do you hear anything? No.). Cosmic sources
such as the sun create what is interpreted by radios as static. People simply
cannot sense the radio waves using our bodily senses.
In addition, there are also indications that students need to take in
physics terms multiple times for physics jargon to be absorbed into the students
vocabulary and understanding. In addition, repetition of phrasing may be
necessary for students to be able to distinguish scientific and everyday meanings
for the same words. For example, the survey results suggest that terms such as
longitudinal and transverse need to be reviewed multiple times by teachers
for students, but that term-sticking is achievable as seen in the survey with the
term photon, not a term used in everyday language. Fragmenting of
knowledge, as addressed in the background (Tuminaro & Redish, 2007; Bao &
Redish, 2006; Disessa & Sherin, 1998, Carmazza et al., 1981; Etkina & Ruibal-
Villasenor, 2008), may also be occurring from the time students are first
presented with wave types until they study light. Frequent quizzes that keep
students on track might be very valuable during the study of sound and light,
including questions that tie past concepts with present instruction. Future
studies beyond this pilot study are suggested. This would entail revaluating the
questions for possible bias as well as content. Another item for future studies
would be to check syllabi between classes data was taken for to assure
consistency of topics.
Another issue may lie in the fact that students may not even be able to
identify radio transmission towers. Why? Many may not even experience
radio except streaming over the internet. While examples of transmission of
cell phones and cell phone towers, might be more effective for this current
younger generation, as many are cellular phone device users, most of these
transmissions fall into the microwave region. Microwaves constitute another
invisible portion of lights electromagnetic spectrum and are perhaps difficult to
address without discussing microwave ovens, another common appliance to
students. Confusion can ensue regarding applications of these as well: relaying
communications versus heating of food. This would merit another study.
Confusions in astronomy and physics with sound and light, such as the use of
terms horn telescopes mentioned in the background should also be considered
with care. Consequences of not understanding the nature of radio waves would
also include students being unable to ascertain secondary effects such as
absorption and polarization as well as applications of communications such as
AM and FM. While some important questions were addressed in this study, a
number of others are still unanswered, such as the construction of questions that
will not lead students to answers, but also will not confuse them.

49
Acknowledgments
Gratefully acknowledged is the opportunity afforded at the American Physical
Society (APS) National Meeting, Baltimore, Maryland March 2013 National
Meeting to present preliminary analysis of the data in this study.
References
Bao, L. & Redish, E. F. (2006). Model analysis: Representing and assessing the dynamics
of student learning, Physical Review Special Topics-Physics Education Research,
2(1), 010103.
Bardar, E., Prather, E., Brecher K., Slater T. (2005). The Need for a Light and
Spectroscopy Concept Inventory for Assessing Innovations in Introductory
Astronomy Survey Courses, Astron. Educ. Rev. 4 (2) 20, doi:
http://dx.doi.org/10.3847/AER2005018
Berger, A., (2015). Interviews with Upper-Level Undergraduates about Representations
of Electromagnetic Plane Waves, presented at the Physics Education Research
Conference 2015, College Park, MD, doi:
http://www.compadre.org/Repository/document/ServeFile.cfm?ID=13836&D
ocID=4254
Caramazza A., McCloskey M. & Green B. (1981). Nave beliefs in sophisticated
subjects: Misconceptions about trajectories of objects, Cognition, 9(2), 117-123.
DeVries, M. (2001). The history of industrial research laboratories as a resource for
teaching about science-technology relationships, Research in Science Education,
31(1), 15-28.
Ding L. & Liu X. (2012). Getting Started with Quantitative Methods in Physics Education
Research online at:
<http://www.compadre.org/per/items/detail.cfm?ID=12601> [Accessed 25
July 2017]
Disessa A. A. & Sherin BL (1998). What changes in conceptual change? International
journal of science education, 20(10), 1155-1191.
Engelhardt P. V. (2009). An Introduction to Classical Test Theory as Applied to
Conceptual Multiple-choice Tests in Getting Started in PER, edited by C.
Henderson and K. A. Harper, American Association of Physics Teachers,
College Park, MD Reviews in PER Vol. 2, doi: http://www.per-
central.org/items/detail.cfm?ID=8807
Etkina, E., Karelina, A. & Ruibal-Villasenor M (2008). How long does it take? A study of
student acquisition of scientific abilities, Physical Review Special Topics-Physics
Education Research, 4(2), 020108.
Finkelstein, N., Adams, W., Keller, C., Perkins, K. & Wieman, C., (2006). High-tech tools
for teaching physics: The physics education technology project Journal of Online
Learning and Teaching 2(3)
Halliday, S. (2007). Introduction. In Science and Technology in the Age of Hawthorne,
Melville, Twain, and James, Palgrave Macmillan US., 1-15.
Hewitt, P., (2007) Next Time Questions- Radio Waves, online (originally published in
The Physics Teacher journal), doi: http://www.arborsci.com/next-time-
questions
Kalat, J. (2010). Introduction to psychology, Independence, KY: Cengage Learning, 37-38.
Landt, J., (2005). The history of RFID IEEE potentials, 24(4), 8-11.
Lazebnik, Y. (2002). Can a biologist fix a radio?Or, what I learned while studying
apoptosis, Cancer Cell, 2(3), 179-182.
McGinnis, J. & Oliver, J. S. (1998). Teaching about Sound: A Select Historical
Examination of Research, Science & Education, 7(4), 381-401.

50
Neumann, S. & Hopf, M. (2012). Students conceptions about radiation: Results from an
explorative interview study of 9th grade students, Journal of Science Education
and Technology 21(6) 826-834.
Perkins, K., Adams, W., Dubson, M., Finkelstein, N., Reid, S., Wieman, C. & LeMaster, R.
(2006). PhET: Interactive simulations for teaching and learning physics, The
Physics Teacher, 44(1) 18-23.
Plotz, T. & Hopf, M. (2016). Students misconceptions about invisible radiation. In
Electronic Proceedings of the ESERA 2015 Conference, Science Education
Research: Engaging learners for a sustainable future, 95-100.
Rego, F. & Peralta, L. (2006). Portuguese students' knowledge of radiation physics,
Physics Education, 41(3), 259.
Tabor-Morris, A. E. (2015). Thinking in terms of sensors: personification of self as an
object in physics problem solving, Physics Education, 50(2), 203-209.
Tuminaro, J. and Redish, E. F. (2007). Elements of a cognitive model of physics problem
solving: Epistemic games. Physical Review Special Topics-Physics Education
Research 3.2 020101
Wise, K. C. (2006). Can You Hear Them Now?: Investigating Radio Waves Science
Activities. Classroom Projects and Curriculum Ideas, 43(3), 23-30.
APS Website: The Large Horn Antenna and the Discovery of Cosmic Microwave
Background Radiation, doi:
www.aps.org/programs/outreach/history/historicsites/penziaswilson.cfm
A Priori Sample Size Calculator, doi:
http://www.danielsoper.com/statcalc/calculator.aspx?id=1

51

Vol. 16, No. 8, pp. 51-64, August 2017
Impact of Teaching Attitudes and

Behaviors for Learning on the Reading
Achievement of Students Falling Behind
Michael E. Bernard
Melbourne Graduate School of Education
University of Melbourne
Abstract. This research evaluates the impact of a teacher professional

development program, Attitudes and Behaviors for Learning (AB4L)
on the achievement of students struggling with reading. Two primary
school teachers from two economically disadvantaged schools received
three half-day training sessions in practices to teach students positive
attitudes and behaviors for learning, which they implemented during
literacy classes. 98 students were taught the AB4L program while 86
students were not. Results include: (a) Student- and teacher-rated
learning behaviors were positive correlated with objective reading
performance; (b) A significant benefit of AB4L on the reading
performance of those students who scored in the lower 50 percent of their
class on a reading comprehension survey; (c) Students in classes where
AB4L was implemented who showed improvements in reading
comprehension also showed increases in behaviors for learning. A
recommendation is that teacher professional development programs
should incorporate positive attitudes and behaviors for learning.
Keywords: Reading Achievement; Reading Improvement; Reading

Teaching; High Risk Students; Student Learning Behavior.
Introduction
This research seeks to illuminate the extent to which primary-age
students who have fallen behind their classmates in reading display delays in
the development of attitudes and behaviours for learning that have been found
to contribute to student engagement and achievement. A second question
addressed in the present research is whether a professional development teacher
training program that up-skills teachers in ways to present and strengthen
student attitudes and behaviours for learning during classroom literacy lessons
results in an improvement in the achievement of students struggling with
reading.
As exemplified by the Department of Education and Training investment
in the National Partnerships for Low SES Schools. Literacy and Numeracy and
Improving Teacher Quality (Australian Department of Education and Training,
2014), educational policy continues to explore innovative and effective ways to
2017 The author and IJLTER.ORG. All rights reserved.

52
assist schools in helping students meet basic literacy standards and to close the
achievement gap for the disadvantaged. This study adds to the best practice
literature on the teaching of reading and the link between students attitudes
and social and emotional learning skills which are vital for student engagement
and the development of their literacy skills (Bernard, 2011).
Student characteristics fundamental to engagement and achievement
(e.g., Lee, 2014) have been termed learning-to-learn skills (Barnett, et. al., 1996),
academic self-regulation skills (Zimmerman, 1990), learning behaviors
(McDermott, 1999), academic enablers (DiPerna & Elliott, 1999), approaches to
learning (Rock & Pollack, 2002), social and emotional learning competencies
(Collaborative for Social and Emotional Learning) and student dispositions
(Hattie, 2013). Specific learning behaviors include goal setting, self-monitoring,
time planning, social skills including seeking help when needed, engagement,
confidence, persistence, self-talk for managing frustration, flexible methods for
learning as well as positive attitudes towards learning including high self-
efficacy beliefs and intrinsic interest in learning (McDermott, 1999).
Research into the characteristics of students with difficulties in reading
reveals delays in the development of self-regulatory learning behaviors such as
setting and achieving learning goals, monitoring success, the failure to use self-
talk to manage anxiety and frustration of completing difficult learning tasks as
well as a range of negative attitudes towards themselves and learning (e.g.,
Vaughn, & Broadman, 2007).
Research has examined gender differences in students use of behaviors
for learning with the advantage being demonstrated by girls. For example,
Duckworth and Seligman (2006) discovered gender differences in favour of girls
in self-discipline and self-control. Gender differences in behaviors for learning may
depend on the academic domain (e.g., Pokay & Blumenfeld, 1990).
Research continues to accumulate demonstrating the effects of non-
cognitive and linguistic competencies on student achievement (e.g., Durlak, et.
al., 2011). Of particular relevance to this research is a study (Ashdown &
Bernard, 2012) that investigated the effect on reading achievement of a social
and emotional learning skills curriculum designed to teach positive attitudes
and behaviors for learning and well-being. The lessons were designed to teach
young children confidence, persistence, organization and resilience including a
range of positive attitudes (e.g., optimism, self-acceptance, internal locus of
control for learning). The lessons were taught three times a week, supported by
a variety of additional social and emotional teaching practices. The results
indicated that the program increased reading achievement for the lower
achieving grade 1 students. Bernard (2006) proposed that it is time that we teach
social-emotional competence for learning as well as we teach academic
competence.
Five teaching practices are contained in the Attitudes and Behaviours for
Learning (AB4L) (Bernard & Milne, 2016) professional development program
that was evaluated in this research. Practice 1. Prepare Students to Begin
Literacy Lesson with a Positive Mindset. For many years (e.g., see Bloom, 1976
Human Characteristics and School Learning) researchers have identified student
attitudinal dispositions towards school and specific classes as they begin a
learning task as a major factor in their achievement. Hatties (2013) meta-

53
analysis of over 800 studies on student achievement reported the effect size of
students disposition to learn as .61. Practice 2. Share with Students the Goals of
the Literacy Lesson, Have Them Set Goals, Monitor Progress and Revise
Learning Methods and Behavior and Practice 3. Communicate Behavior-Specific
Feedback for Learning. Hatties (2013) meta-analysis also revealed a large effect
size of teachers helping students set goals for learning and providing positive
and negative feedback to students on their achievement as well as on their use of
learning strategies (also, see Locke & Latham, 2002; Schunk, 2003). Practice 4.
Identify and Discuss Behaviors for Learning. Research shows student learning
behaviors contribute to school readiness, literacy and mathematics outcomes
(e.g., DiPerna, Volpe & Elliott, 2002; Fantuzzo, Perry, & McDermott, 2004; Green
& Francis, 1988; McDermott, 1984; McWayne, Fantuzzo, & McDermott, 2004).
Practice 5. Discuss Positive (and Negative) Self-Talk for Learning. The self-
regulatory nature of inner speech and self-talk (Vygotsky, 1934/86) has been
found to assist students in guiding their thinking and learning (Kross, et. al.,
2014; Winsler & Naglieri, 2003; Winsler, Manfra & Diaz, 2006; see Hardys 2006
review of self-talk literature). Self-talk is a major component of cognitive-
behavior therapy, a best practice intervention for young people with emotional
and behavioral problems that interfere with their learning (Bernard, 2006).
Moreover, students who acquire self-regulatory skills experience improved
academic achievement and increased self-efficacy (e.g., Zimmerman & Schunk,
2001).
The present project posed the following research questions.
1. Are students behaviors for learning associated with their reading
performance?
2. Are there gender differences in behaviors for learning?
3. Will the AB4L program have a positive effect on students behaviors for
learning?
4. Will the AB4L program have a positive impact on students reading
performance?
5. Will students in the bottom 50 percent of their class in reading
performance who receive the AB4L program show greater improvement in
their reading than students in the bottom 50 percent of their class in
reading performance who do not receive the program?
6. Will the AB4L program have a different impact on the behaviors for
learning and reading performance of boys versus girls?
7. Do students who show improvements in their behaviors for learning
show concomitant changes in their reading performance?
Method
Participants
The study used a pre-post treatment-control quasi-experimental design
to evaluate the effectiveness of AB4L. Two school principals located in a rural
community in Victoria agreed to have the program implemented in their
schools. In School A, the students in two composite grade 3/4 classes were
chosen to receive the AB4L program while in School B, the students in two
composite grade 5/6 classes received the AB4L program. For purposes of
comparison, in School A, the students in two composite grade 5/6 classes did

54
not receive the AB4L program while in School B, students in two composite
grade 3/4 classes did not receive the AB4L program while students in two
composite grade 5/6 classes received the AB4L program. The total number of
students receiving the program was 98 (51% female) while the total number of
students not receiving the program was 86 (45% female).
Measures
Teacher Ratings: Learning Behaviors Scale (LBS). The LBS (McDermott et
al., 1999) is a standardized 29-item teacher-completed rating scale. Items are
rated on a 3-point scale (0 = doesn't apply, 1 = sometimes applies, 2 = most often
applies) indicating the presence of the behavior over the past two months. Scales
were scored as the mean of the items. The measure included an overall score and
four subscales measuring motivation, attitude, persistence, and strategy. Internal
consistency coefficients are high for the overall (.89-.92) and subscale scores (.70-
.87) both overall and for age, gender, and ethnic subsamples, and stability
coefficients across a 2-week interval were strong (.91-.94; McDermott, 1999).
Student Ratings: Student Learning Behaviors Survey (SLBS). A new
rating scale, the Student Learning Behaviors Survey, was developed for this
study and was designed to measure student self-perceptions of attitudes and
behaviors associated with their engagement during literacy instruction (reading
and writing). Items were rated on a 2-point scale (0 = disagree, 1 = agree).
Questions were developed by the first author of this study that examined
student self-perception of their confidence, persistence, goal orientation,
teamwork, disorganization and worry associated with literacy instruction. The
initial survey of 18 items asked students to agree or disagree with a series of
questions; for example, I get easily tired when I read or write, I distract
others during reading or writing time. See Table 2 for complete item text. The
SLBS had a mix of positively and negatively worded items.
A maximum likelihood factor analysis was conducted on the pre-
intervention responses of all participating students to the initial set of 18 items.
Examination of the scree plot suggested one main factor, and a second smaller
factor; variance explained by the first 8 unrotated components was 23.6, 10.9, 7.3,
6.6, 6.2, 5.7, 5.3, 4.9. The two items that loaded highly on the second factor (17. I
could do a lot better in my reading; and 18. I could do a lot better in my
writing) were removed as they appeared to combine both perceptions of low
ability with a perceived ability to do better. After removal, the scree plot
showed clear support for a large first factor with variance explained by the first
8 unrotated components of 23.6, 8.4, 7.3, 6.4, 5.9, 5.3, 5.1, and 4.2. While it would
be possible to attempt to further explore subscales on this measure, the scale
composed of the first factor reflected the most reliable and systematic source of
variance, and provides a parsimonious representation of overall positive
attitudes and behaviors regarding reading and writing. The retained items and
their factor loadings are shown in Table 1. The test was scored as the mean of the
16 items after item reversal. Cronbachs alpha reliability for the Student SLBS
was .79 at Time 1 and .73 at Time 2. The correlation between pre and post
intervention scores was r = .61.

55
Table 1: Item Loadings for 16 Retained Items of the Social Learning and
Behavior Scale
_______________________________________________________________________
Item Loading
_______________________________________________________________________
1. I get easily tired when I read or write. (R) -.36
2. I sometimes forget to bring to class things I need to learn (pencils, paper,
book). (R) -.33
3. I distract others during reading or writing time (R) -.43
4. At the beginning of a lesson (reading, writing), I set a goal for what I want to
learn. .20
5. I put up my hand to answer a difficult question (reading, writing). .33
6. I like to read. .47
7. I like to write. .33
8. I worry a lot about my schoolwork (reading, writing). (R) -.38
9. When reading or writing gets really hard, I can give up before getting it done
properly. (R) -.73
10. I can do schoolwork that is hard to do (reading, writing). .41
11. When I do not understand something (reading, writing), I give up easily.
(R) -.74
12. I am a good listener when working in my reading or writing groups. .49
13. I help others when they do not understand something (reading, writing). .43
14. I get distracted when I am doing my reading and writing. (R) -.43
15. It takes a long time for me to settle down to do my reading and writing.
(R) -.47
16. I lose confidence when reading or writing. (R) -.62
_______________________________________________________________________
Note. Reversed items are indicated by (R). Loadings are based on a one factor
maximum likelihood factor analysis.
Objective Reading Performance. The Victorian Curriculum and
Assessment Authoritys On Demand Computer Adaptive Reading Test (2006, 2010)
was used to assess students level of reading comprehension and, specifically,
the extent of development of reading comprehension competence over the three
and a half month period of this evaluation project. This 30-item test presents 10
sub-test packets of three reading items to students. Each item is designed to test
a specific skill associated with reading comprehension. Some examples of
reading comprehension skills assessed include analyze imagery in a text,
analyze plot in a text, analyze point of view in a text and analyze setting in a
text. The test provides a standard score that corresponds to grade-level
performance relative to AusVELS standards. Across the whole sample reading
performance scores were correlated r = .79 across the two time points.
Attitudes and Behaviors for Learning Program
The AB4L program provides teachers with explicit instruction in the use
of five practices that can be employed throughout a reading lesson to teach
students various attitudes and behaviors for learning (see Table 2). Teachers
were trained to integrate the five AB4L practices throughout the different
components of a literacy lesson (before the lesson begins, during whole class,
teacher-led instruction, during small group/dyadic/individual work, at end of
literacy session reflection on learning, assignment of literacy homework).

56
Table 2: Teaching Practices for Integrating Attitudes and Behaviors for

Learning into Different Components of Literacy Instruction
_______________________________________________________________________
1. Prepare Students to Begin Literacy Lesson with a Positive Mindset. At
the beginning of a literacy lesson, help students maintain a positive focus
by reviewing different positive attitudes.
2. Share with Students the Goals of the Literacy Lesson, Have Them Set
Goals, Monitor Progress and Revise Learning Methods and Behavior.
Regularly, ask students to set goals (what they want to learn; mark they
will receive). Spell out the different concepts and skills/strategies that
will be taught in the literacy lesson. At the end of the class, have students
reflect on goal attainment. Based on this feedback, encourage students to
modify their approach to learning.
3. Communicate Behavior-Specific Feedback for Learning. Acknowledge
individual and groups of students who display different behavior for
learning by providing feedback that names/describes the behavior and
attitude they have demonstrated in a literacy activity
4. Identify and Discuss Behaviors for Learning. Discuss different behaviors
that students should practice/use that can help them to be self-managing
and engaged during a literacy activity.
5. Discuss Positive and Negative Self-Talk for Learning. Describe and
model positive and negative self-talk that that students can use to remain
calm when feeling frustrated or overwhelmed by a learning activity.
_______________________________________________________________________
Procedure
During the two weeks before the first of three teacher-training sessions,
collection of evaluation data occurred. All teachers (those to receive training in
the AB4L program; those who did not receive training) completed the Learning
Behaviors Scale for each of their students. Teachers had all students complete
the Student Learning Behaviors Survey. All students also completed the On
Demand Computer Adaptive Reading Test.
The teacher training sessions took place over a three- and a half-month
period. The sessions were conducted by an experienced classroom
teacher/literacy coordinator. Each session took approximately three hours.
Evaluation Data Collected (2 weeks before commencement)
Week 1. Teacher training Session1.
Week 3. Classroom observation of teachers by trainer.
Week 6. Teacher training Session 2.
Week 11. Teacher training Session 3.
Evaluation Data Collected (2 weeks after commencement)
During weeks 3, 8, and 13, the trainer conducted a classroom observation
of each participating teacher as the teacher taught a literacy lesson. The purpose
of the observation was for the trainer to determine the extent to which the
teacher was implementing AB4L. After each observation, the trainer would

57
summarize in an email the findings in terms of each teachers strengths and

areas for improvement.
To address the issue of potential bias of having the developer of the
AB4L program involved in its evaluation, the program was implemented and
surveys administered by a literacy coordinator of a nearby school whose
livelihood did not depend on finding significant effects of the AB4L program.
Additionally, the second author of this research paper who conducted all aspects
of data entry and statistical analysis had no prior familiarity with the program
and has no vested interest in the results.
Results
Correlations at Pre-Intervention
Pearson correlations using pre-intervention measures indicated that
objective reading performance was positively correlated with both student-rated
learning behavior (r = .34, p < .01) and teacher-rated learning behavior (r = .45, p
< .01). All subscales of teacher-rated learning behavior were correlated with
objective reading performance (correlations ranged from r = .32 to r = .53).
Finally, student-rated learning behavior was positively correlated with teacher
ratings (r = .53, p < .01).
With regards to gender differences, an independent groups t-tests
indicated that teachers rated the learning behavior of girls more highly than
boys (d = 0.46, p = .003). However, no significant differences were obtained for
student-rated behavior (d = 0.24, p = .14) or objective reading performance (d =
0.10, p = .53).
Effect of the Teaching Intervention (AB4L)

Means and standard deviations for pre- and post-intervention scores are
shown in Table 3. Before assessing the effect of the intervention, we first
examined assumptions. There were no significant differences between the
control group and intervention group at pre-intervention on any of the outcome
measures. However, although not a significant difference, the intervention
group did score about a third to a half standard deviation lower on learning
behaviors at baseline. Standard deviations were similar across time points and
groups. In terms of normality, objective reading performance (skew at pre-
intervention = 0.30) had minimal skew, whereas student-rated learning
behaviors (skew at pre-intervention = -0.67) and teacher-rated learning
behaviors (skew at pre-intervention = -0.87) were negatively skewed reflecting a
tail of particularly poor performers. Given the moderate to large sample size, the
inferential tests used are robust to the presence of this mild skewness.

58
Table 3. Means and Standard Deviations for Students Who Did and Did Not
Receive the AB4L program at Pre-Test and Post-Test on Main Outcome
Variables
Pre-Intervention Post-Intervention
Control Interven. Control IntInterven.
Variable (scale range) M (SD) M (SD) M (SD) M (SD)
SLBS (0 - 1) 0.75 (0.21) 0.70 (0.22) 0.75 (0.18) 0.83 (0.17)
Reading (1 - 10) 3.12 (1.19) 3.18 (1.11) 3.26 (1.26) 3.44 (1.06)
LBS Total (0 - 2) 1.58 (0.39) 1.43(0.49) 1.75 (0.29) 1.73 (0.39)
LBS Strategy (0 = 2) 1.74 (0.36) 1.62 (0.45) 1.82 (0.28) 1.80 (0.38)
LBS Motivation (0 - 2) 1.55 (0.47) 1.26 (0.57) 1.71 (0.34) 1.70 (0.44)
LBS Attitude (0 - 2) 1.63 (0.42) 1.51 (0.54) 1.83 (0.26) 1.79 (0.38)
LBS Persistence (0 - 2) 1.48 (0.51) 1.27 (0.65) 1.62 (0.43) 1.66 (0.51)
Note. SLBS = Student Learning Behavior Scale (Student-Rated); LBS = Learning

Behavior Scale (Teacher Rated). Control group did not receive the AB4L
intervention, whereas the Intervention group received the AB4L intervention.
To assess the effect of the teaching intervention, a gain-score approach
was adopted. This involved first computing change scores for each outcome
measure calculated as post-intervention score minus pre-intervention score.
Then, independent groups t-tests were performed on these change scores with
intervention group as the independent variable. This approach is statistically
equivalent to examining the interaction effect in 2 by 2 mixed ANOVA (Knapp
& Schafer, 2009). To quantify the size of the difference in improvement, the effect
size measure discussed in Morris (2008) was used. This is the pre-post
equivalent of standardized mean difference (i.e., Cohen's d) and represents the
standardized mean difference in change scores. Students who received the
program showed significantly greater increases compared to the control group
in teacher-rated learning behaviors (d = 0.30, p = .03), student-rated learning
behaviors (d = 0.55, p < .001), but no significantly greater increase in objective
reading performance (d = 0.11, p = .32).
Because the research uses a quasi-experimental design, we also examined
whether the effects were maintained using an ANCOVA approach to assessing
the effect of the intervention (for a discussion of this issue, see Knapp & Schafer,
2009). This involved running a linear model predicting time 2 outcome scores
from condition (intervention or control) and covarying for time 1 outcome
scores. Using this alternate approach to assessing the effect of the intervention,
the effect of student rated learning behaviors was still highly significant (p <
.001), and the effect on objective performance was still non-significant (p = .26).
However, the effect on teacher-rated learning behaviors changed somewhat
whereby the effect was only statistically significant for persistence (p = .03), and
was non significant for total (p = .25), strategy (p = .09), motive (p = .12), and
attitude (p = .58). This difference in results between the ANCOVA and change

59
score approach is an instance of Lord's paradox. It presumably arises for teacher-

rated behavior, because scores were slightly lower, albeit non-significantly, in
the intervention group at pre-intervention. This makes it less likely that the
intervention group will have larger covariate adjusted scores, thus making it
harder to get a significant ANCOVA. Alternatively, under some mechanisms of
change, it makes it somewhat easier to get a significant change score.
To assess whether the intervention was particularly effective for lower
performing students, a further analysis was conducted of students in the lower
half of their class in their reading scores at pre-intervention. Using the above
mentioned t-test on change scores, for these students, there was significantly
greater increases in objective reading performance in the intervention group
(Pre-Test, M = 2.34, SD = 0.74; Post-Test, M = 2.88, SD = 0.65) relative to the
control group (Pre-Test, M = 2.07, SD = 0.51; Post-Test, M = 2.29, SD = 0.62), d =
0.42, p = .03.
To assess gender differences in the effect of the AB4L program, an
ANOVA was performed examining the gender by intervention group
interaction on change scores for teacher-rated behavior, student-rated behavior,
and objective reading performance. There was no statistically significant
evidence for differential effects of the AB4L program for boys and girls (all p's >
.05).
Correlation of Improvements in Rated Behavior and Objective Performance

To assess whether students who show improvements in their learning
behaviors show concomitant changes in objective reading performance,
correlations between change scores separately for the two conditions were
calculated (see Table 4). Change scores were calculated as post-intervention
minus pre-intervention scores, such that a positive change score indicates that
the student showed higher scores after the intervention (e.g., better learning
behaviors or improved objective reading performance). Correlations of change
scores of teacher-rated behavior and student-rated behavior with change scores
for objective reading performance were positive and significant in the
intervention group but not the control group. Thus, it can be seen that for
students receiving the AB4L Program, those who showed increases in their
student-rated and teacher-rated learning behaviors tended to show
improvements in objective reading performance. However, a test of significant
differences between independent correlations using Fishers r to z
transformation was performed (for formulas, see Cohen et. al., 2003) indicated
that the differences between control and intervention group correlations were
not statistically significant.

60
Table 4. Correlation of Change Scores on Teacher and Student Rated Learning

Behavior with Change Scores on Reading Performance for Intervention and
Control Groups
Change in Objective Reading Performance

Control Intervention
Change in SLBS .03 .22
Change in LBS Total .10 .27
Change in LBS Strategy .06 .24
Change in LBS Motivation .07 .27
Change in LBS Attitude .10 .24
Change in LBS Persist .03 .13
Note. Values in the table are Pearson's correlations between change scores (post-
intervention minus pre-intervention) calculated separately for control and
reading intervention groups. For example, a positive value indicates that
increases in self- or teacher-rated behaviors are correlated with increases in
objective reading performance. SLBS = Student Learning Behavior Scale
(Student-Rated); LBS = Learning Behavior Scale (Teacher Rated). Correlations
larger than .22 are statistically significant at the .05 level and are shown in bold.
Discussion
This investigation examined the extent to which schools, especially those
with high proportions of students from socially and economically
disadvantaged backgrounds, should concern themselves with ensuring that the
set of student characteristics referred to as attitudes and behaviors for learning
should be an essential aspect of literacy teaching practice.
Baseline Correlations and Gender Differences
Consistent with previous research (e.g., Rock & Pollack, 2002), objective
reading comprehension was correlated with teacher and student ratings of
learning behaviors. These correlations may reflect both the benefits of the
learning behaviors as well as a general sense of efficacy in performance that may
be induced by the teaching environment that included opportunities for
feedback and peer comparison.
Teachers rated girls higher in behaviors for learning than they did boys
confirming previous findings of gender differences (e.g., Duckworth &
Seligman, 2006; Schaeffer, 2004). Of interest is that when student self-perceptions
were examined, no gender differences were found. This inconsistency with
teacher ratings may be due to a tendency of boys to provide unrealistic ratings
of their learning behaviors resulting in the elevation of scores on the Student
Learning Behaviors Scale. One implication of this finding is that teachers may
need to be more explicit in providing boys with feedback concerning their use
and non-use of various attitudes and behaviors for learning.
Effect of Intervention
The AB4L program had a positive impact on the behaviors for learning of
students who received the program. This finding that behaviors for learning are
teachable is supported by extensive previous research (McDermott, Leigh, &
Perry, 2002; McDermott, Mordell & Stoltzfus, 2001). A novel aspect of AB4L is

61
that it is not a stand-alone program taught where students are taught

foundational positive attitudes and learning behaviors apart from academic
instruction. Integrating the teaching of attitudes and behaviors for learning as a
part of literacy instruction is likely to produce a much stronger effect than a
program taught on its own.
While the AB4L program benefitted students behaviors for learning, it
did not show the same overall effects on reading performance for all students.
There are multiple influences on students reading competences and
achievement and while the enhancement of student learning behavior places
students in a better position to profit from instruction, we also know that
prerequisite, background knowledge is a major factor in predicting and
explaining levels of achievement (e.g., Wang, Haertel &Wahlberg, 1993).
Perhaps, participating students pre-requisite reading comprehension skills were
so under-developed that improvements in a reading comprehension test
(Reading in Demand) was not possible in such a short period of time.
Alternatively, it is the case that there are students in the two low SES
schools in the sample who are reading near grade level expectations. It may be
that these students have reasonably well-developed learning behaviors despite
the low SES index of the school. For these students, it may be the case that the
benefits of AB4L on reading performance may be seen in the long-term.
Results reveal a significant benefit of AB4L on the reading performance
of those students who scored in the lower 50 percent of their class on the reading
comprehension survey used in this project. Comments from teachers indicated a
shift in focus of class concern from reducing negative behavior to advancing
positive behavior and an increase in whole-class student interest wanting to be
successful. It may be the case that this shift in classroom culture along with the
explicit teaching of positive attitudes and behaviors for learning had the most
impact on the disengaged, under-achieving students.
The finding of equal benefit of impact of AB4L on boys and girls is an
important finding especially for the education of boys. It appears that the
explicit teaching practices employed in AB4L where students are asked to
practice ways of thinking and learning behaviors to use during classroom
instruction combined with behavior-specific feedback equally suits the learning
styles of boys and girls.
Correlated Changes
Of some significance is the finding that students in classes where AB4L
was implemented who showed improvements in their reading comprehension
also showed increases in their behaviors for learning. This would suggest that as
many have argued that behaviors for learning are, indeed, mediating factors in
the chain of influence leading to academic competence and achievement (e.g.,
McDermott, et. al., 2001).
Additionally, this evidence of correlated changes in reading achievement
and behaviors for learning suggests that the positive impact of AB4L was a
specific effect of explicit teaching of attitudes and behaviors for learning and the
teaching practices employed rather than solely a general effect of teachers being
more positive.

62
Limitations
First, the study used a quasi-experimental design that made use of
existing classes. Thus, baseline differences between groups and differences
between teacher effectiveness may have influenced the results. Nonetheless, the
use of a pre-testing in the current study and the focus on change scores provides
some control. Second, the AB4L program only had a significant effect on
objective reading performance of students in the lower 50% of reading
performance. Benefits of the AB4L program for students competent in reading as
well as attitudes and behaviors for learning cannot be ascertained from the
present analyses. These finding cannot be generalized to students who have
reading challenges but do not attend economically disadvantaged schools. A
third limitation of the study is that it is impossible to discern which of the
different teaching practices focused on positive attitudes and behaviors for
learning were the most powerful.
Conclusion
Based on these findings and previous research, student characteristics
and their role in academic development and achievement needs to be in the
center of the radar screen of education reform efforts to improve reading of
students falling behind. Teacher preparation and professional development
programs as well as the coaching and mentoring of principals and teachers
should incorporate positive attitudes and behaviors for learning, especially as
additional instructional support students who are most likely to be at risk for
educational failure as well as those who are under-achieving in literacy and
numeracy.
References
Ashdown, D.M, & Bernard, M.E. (2012). Can explicit instruction in social and emotional
learning skills benefit the social-emotional development, well-being, and
academic achievement of young children? Early Childhood Education Journal, 11,
397-405.
Australian Government Department of Education and Training (2014). National
Partnerships for Low SES Schools. Literacy and Numeracy and Improving Teacher
Quality. https://www.education.gov.au/national-partnerships-low-ses-schools-
literacy-and-numeracy.
Barnett, D. W., Bauer, A. M., Ehrhardt, K. E., Lentz, F. E., & Stollar, S. A. (1996). Keystone
targets for changes: Planning for widespread positive consequences. School
Psychology Quarterly, 11, 95-117.
Bernard, M.E. (2006). Its time we teach social-emotional competence as well as we teach
academic competence. Reading and Writing Quarterly, 22, 103-119.
Bernard, M.E. (2011). The link between students social and emotional characteristics
and the development of literacy and numeracy skills. Research Monograph #08.
The Victorian Literacy and Numeracy Secretariat, State Government of Victoria.
Bernard, M.E., & Milne, M. (2016). Attitudes and behaviours for learning: A teacher
professional development program for raising student engagement and achievement in
Literacy and numeracy (2nd. Ed.). pp. 70. Manuscript submitted for publication.
Bloom, B.S. (1976). Human characteristics and school learning. New York: McGraw-Hill.
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple
regression/correlation analysis for the behavioral sciences (3rd edition). Mahwah, NJ:
Erlbaum.
Collaborative for Social and Emotional Learning, www.casel.org.

63
DiPerna, J.C., & Elliott, S.N. (1999). The development and validation of the Academic
Competence Evaluation Scales. Journal of Psychoeducational Assessment, 17, 207
225.
DiPerna, J.C., Volpe, R.J., & Elliott, S. (2002). A model of academic enablers and
elementary reading/language arts achievement. School Psychology Review, 31,
298-312.
Duckworth, A., & Seligman, M. (2006). Self-discipline gives girls the edge: Gender in
self-discipline, grades and achievement test scores. Journal of Educational
Psychology, 98, 198-208.
Durlak, J., Weissberg, R., Dymnicki, A., Taylor, R. & Schellinger, K. (2011). The impact of
enhancing students social and emotional learning: A meta- analysis of achool-
based universal interventions. Child Development, 82, 474- 501.
Fantuzzo, J., Perry, M.A., & McDermott, P. (2004). Preschool approaches to learning and
their relationship to other relevant classroom competencies for low-income
children. School Psychology Quarterly, 19, 212-230.
Green, L. F., & Francis, J. M. (1988). Children's learning skills at the infant and junior
stages: A follow-on study. British Journal of Educational Psychology, 58, 120-126.
Hardy. J. (2006). Speaking clearly: A critical review of the self-talk literature. Psychology
of Sport and Exercise, 7, 81-97.
Hattie, J. (2013). Visible learning: A synthesis of over 800 meta-analyses relating to
achievement. London: Routledge.
Knapp, T. R., & Schafer, W. D. (2009). From Gain Score t to ANCOVA F (and vice
versa). Practical Assessment, Research & Evaluation, 14(6), 1-7.
Kross, E., Bruehlman-Senecal, E., Park, J., Burson, A., Dougherty, A., Shablack, H.,
Bremner, R., Moser, J., & Ayduk, O. (2014). Self-talk as a regulatory mechanism:
How you do it matters. Journal of Personality and Social Psychology, 106, 304-24.
Lee, J. (2014). The relationship between student engagement and academic performance:
Is it a myth or reality? Journal of Educational Research, 107, 177-185.
http://dx.doi.org/10.1080/00220671.2013.807491
Locke, E.A., & Latham, G.P. (2002). Building a practically useful theory of goal setting
and task motivation: A 35-year odyssey. American Psychologist, 57, 705-717.
McDermott, P. A. (1984). Comparative functions of preschool learning style and IQ in
predicting future academic performance. Contemporary Educational Psychology. 9.
38-47.
McDermott, P. A. (1999). National scales of differential learning behaviors among
American children and adolescents. School Psychology Review, 28, 280-291.
McDermott, P. A., Leigh, N. M., & Perry, M. A. (2002). Development and validation of
the Preschool Learning Behaviors Scale. Psychology in the Schools. 39, 353-365.
McDermott, P. A., Green, L. F., Francis, J. M., & Stott, D. H. (1999). Learning Behaviors
Scale. Philadelphia: Edumetric and Clinical Science.
McDermott, P.A., Mordell, M., & Stoltzfus, J.C. (2001). The organization of school
performance in American school discipline, motivation, verbal learning and
nonverbal learning. Journal of Educational Psychology, 93, 65-76.
McWayne, C. M., Fantuzzo, J. W., & McDermott, P, A. (2004). Preschool competency in
context: An investigation of the unique contribution of child competencies to
early academic success. Developmental Psychology, 40, 633-645.
Morris, S. B. (2008). Estimating effect sizes from pretest-posttest-control group designs.
Organizational Research Methods, 11, 364-386.
Pokay, P., & Blumenfeld, P.C. (1990). Predicting achievement early and late in the
semester: The role of motivation and use of learning strategies. Journal of
Educational Psychology, 82, 41-50. http://dx.doi.org/10.1037/0022-
0663.82.1.41

64
Rock, D.A., & Pollack, J.M. (2002). Early childhood longitudinal study kindergarten
class of 1998-99 (ECLS-K), psychometric report for kindergarten through first
grade. Washington, DC: U.S. Department of Education, Office of Educational
Research and Improvement, National Center for Educational Statistics.
Schaefer, B.A. (2004). A demographic survey of learning behaviors among American
students. School Psychology Review, 33, 481-497.
Schunk, D.H. (2003). Self-efficacy for reading and writing: Influence of modeling, goal
setting, and self-evaluation. Reading and Writing Quarterly, 19, 159-172.
The Victorian Curriculum and Assessment Authoritys On Demand Computer Adaptive
Reading Test (2006, 2010).
www.vcaa.vic.edu.au/Pages/prep10/ondemand/index.aspx
Vaughn, S., & Broadman, A. (2007) Teaching reading comprehension to students with
learning difficulties. New York: Guilford Press.
Vygotsky, L. (1934/1986). Thought and language. Cambridge, MA: MIT Press.
Wang, M.C., Haertel, G.D., & Walberg, H.J. (1993). Toward a knowledge base for school
learning. Review of Educational Research, 63, 249-294.
Winsler, A., Manfra, L., & Diaz, R. M. (2006). Should I Let Them Talk? : Private speech
and task performance among preschool children with and without behavior
problems. Early Childhood Research Quarterly, 22, 215- 231.
Winsler, A., & Naglieri, J. (2003). Overt and covert verbal problem-solving strategies:
Developmental trends in use, awareness, and relations with task performance in
children Aged 5-17. Child Development, 74, 659-678.
Zimmerman, B.J. (1990). Self-regulated learning and academic achievement: An
overview. Educational Psychologist, 25, 3-17.
Zimmerman, B.J., & Schunk, D.H. (Eds.) (2001). Self-regulated learning and academic
achievement. Mahwah, NJ: Erlbaum.

65

Vol. 16, No. 8, pp. 65-76, August 2017
Balancing Reflection and Validity in Health

Profession Students Self-Assessment
Sherri Melrose
Athabasca University
Alberta, Canada
Abstract. Students and practitioners in self-regulating health professions

are expected to engage in reflective, valid self-assessment activities.
However, self-assessment processes can be flawed. People may have a
limited understanding of the critical thinking needed to reflect on their
performance and they may over-estimate or under-estimate their
abilities. This article highlights educational approaches that can help
students achieve a balance of reflecting critically and developing more
accurate self-assessments. Considerations involved in defining self-
assessment are identified. Explanations of how integrating reflection
requires critical thinking; information from both internal and external
sources; and incidental learning are provided. Suggestions for
addressing validity by recognizing that inaccuracies exist; knowing that
peoples history with academic success can impact their self-
assessments; and creating links to affective outcomes are offered.
Emphasis is placed on viewing self-assessment as a formative learning
activity that is introduced early and consistently in health education
programs.
Keywords: health profession students self-assessment; reflection in self-

assessment; validity in self-assessment
Introduction
Self-assessment, a necessary skill for lifelong learning, requires people to
identify standards to apply to their work, and then to make judgements about
the extent to which they have met these standards (Boud, 1991; 1995). For
practitioners in self-regulating health professions, self-assessment activities are
an integral aspect of both their pre-service programs and their ongoing in-
service professional development (Eva & Regehr, 2005). Novice practitioners
enter their profession with a stronger ability to assess and develop the
competencies they need when they have become familiar with assessing their
own progress during their education (Boud, & Falchikov, 2006; Kajander-
Unkuri, Meretoja, Katajisto, Saarikoski, Salminen et al., 2013; Linn, Arostegui &
Zeppa, 1975; Passi & Southgate, 2016). Supporting learners towards developing

66
their capacity to self-assess has been identified as the missing link needed to
ensure that future health professionals are truly reflective, self-regulating
practitioners (Redwood, Winning & Townsend, 2010).
However, the self-assessment process can be flawed (Melrose, Park & Perry,
2015). For the most part, people overrate themselves and assess their progress as
above average (Davis, Mazmanian, Fordis, Van Harrison, Thorpe, et al., 2006;
Dunning, Heath & Suls, 2004; Mort & Hansen, 2010; Pisklakov, Rimal &
McGuirt, 2014). They often identify areas of weakness inaccurately (Regehr &
Eva, 2006). People can overestimate their performance and misjudge the skills
they believe they have mastered (Baxter & Norman, 2011; Galbraith, Hawkins &
Holmboe, 2008). Students who are least able to self-assess accurately often also
demonstrate limited abilities in other areas of study (Austin & Gregory, 2007;
Colthart et al., 2008).
The phenomenon of less able people over-assessing their ability and more able
people underestimating themselves is known as the Dunning-Kruger effect
(Ehrlinger, Johnson, Banner, Dunning & Kruger, 2008; Kruger & Dunning, 1999).
Consequently, the validity of self-assessment as an accurate measurement of
student learning has been questioned (Falchikov & Boud, 1989; Gordon, 1991;
Lundquist, Shogbon, Momary & Rogers, 2013; Ward, Gruppen & Regehr, 2002).
Balancing the merit of reflection in self-assessment with questions about the

validity of health profession students self-assessment is not straightforward.
Existing research has focused on evaluation studies and most of this work has
been directed to physicians learning. However, health professionals from a
variety of different settings are expected to engage in self-assessment in their
learning and in their practice. Increasing understanding of self-assessment
among all members of health care teams can make an important difference in
helping learners grow into self-regulated professionals. Geared to a
multidisciplinary audience, this article provides an overview of how self-
assessment can be defined, how reflection can be integrated into the process and
how issues of validity can be addressed.
Toward a Definition of Self-Assessment

Assessment provides information about how people are progressing in relation
to objectives, goals and outcomes. In health professions programs, assessments
usually include standardized measurement tools as well as inferences about
what individuals do in relation to what they know (Melrose, Park & Perry, 2015).
In clinical practice settings, specific times at both mid-term (formative) and end
of course (summative) are designated for discussing student progress. Self, peer
and educator assessment may be included in these discussions. Formative
evaluations are diagnostic, ongoing and focused on both what students are
currently doing well and areas where they need to improve in future. A final
grade is seldom included in formative evaluations as the goal of the activity is to
improve student performance (Melrose, Park & Perry, 2015).

67
Self-assessment, particularly when integrated into formative evaluations, can be

construed as a learning activity and not merely a grading activity. When viewed
as a learning activity, self-assessment invites students to actively participate in
and reflect on their own learning (Boud & Falchikov, 1989). It helps students
recognize desired goals, gather evidence about their present position and come
to an understanding about ways they can close the gap between the two (Black
& William, 1998).
Instead of simply relying on teachers to evaluate their progress, opportunities

for self-assessment encourage students to think critically about the quality of
their studies (Andrade & Valtcheva, 2009). Self-assessment assists students to
create links among tasks they are presently engaged in, outcomes expected by
their profession, outcomes they expect for themselves and future tasks they will
engage in (Bourke, 2016). In essence, self-assessment can be conceptualized as a
formative, educational, developmental, self-monitoring activity that draws upon
both internal and external data, standards, and resources to inform and judge
ones performance (Sargeant 2008).
Integrating Reflection
Reflection has a dynamic relationship with self-assessment. As Mann (2010) so
eloquently stated: To be effective at self-assessment requires skills in critical
reflection; to be effective in reflection, self-assessment skills are required
(p.311). However, the purposes and goals of reflection are different from those of
self-assessment. Reflection is a process of personal self-understanding that can
lead to significant discoveries and insights, while self-assessment involves using
predetermined performance criteria to determine insights, strengths and needed
improvements (Desjarlais & Smith, 2011).
Reflective processes are often retrospective; they do not necessarily involve

others or externally imposed performance criteria; and they may not include
expectations of improvement. It is important to acknowledge that self-
assessment skills are not limited to engaging in reflective activities. However,
reflection, particularly critical reflection, plays a foundational role in health
profession students self-assessment.
Critical Reflection Reflection that can be considered critical and therefore of most
use in self-assessment goes beyond simply looking back on experiences.
Theorists have extended our understanding of the complex reasoning that is
involved. Advocating for the use of reflection as an active and deliberate
problem-solving process, John Dewey (1933) believed reflection should include
recalling an event and then questioning why things happened as they did.
Donald Schn (1983), theorized that reflective practice includes both reflection-
in-action (intuitively drawing on previous experiences to resolve situations
while they are occurring) and reflection-on-action (thinking about an event that
has taken place and considering what could be changed in future). Steven
Brookfield (1995; 1988) asserted that reflective practice also requires people to
become aware of and question their assumptions and their ways of interpreting

68
information. Jack Mezirow (1998) explained how critical reflection requires

people to examine the way they perceive events and then transform their
thinking in order to find new ways of making meaning. In nursing, Christopher
Johns (2017) proposed a structured model calling for practitioners to reflect on
experiences by both "looking in" to examine their thoughts and emotions and
"looking out" to understand external factors influencing the situation.
The complexities of thinking critically and engaging in reflective practice may

seem overwhelming to health profession students, particularly those at a
beginning stage of their program. One approach that can help students
strengthen their self-refection skills and to grow as reflective practitioners is to
introduce reflective activities early (Falchikov & Boud, 1989; Kanthan & Senger,
2011; Mann, Gordon & MacLeod, 2009).
Tools such as reflective journals can provide opportunities for developing

reflective practice skills (Constantinou & Kuys, 2013; Koh, Wong & Lee, 2014;
Lew & Schmidt, 2011). While reflective journals are usually written products,
Tulgar (2017) notes how reflections can also be captured through Smartphone
audio or video self-recordings. Similarly, students can use social media
applications to create reflective journals (Dabbagh & Kitsantas, 2012). In clinical
practice settings, educators can intentionally invite students to begin any
discussion of their performance with self-reflection and self-analysis (Melrose,
Park & Perry, 2015). When students are consistently required to engage in
critical reflection throughout their programs, the process becomes increasingly
familiar.
Extending students critical reflection skills to strengthen their self-assessment

skills involves building in opportunities to cast students own thinking against
predetermined outcomes. When students are performing new clinical tasks, it is
not unexpected that their capacity to self-assess is also less accurate. However,
later in their programs, self-assessment accuracy improves (Blanch-Hartigan,
2011; Fitzgerald, White & Gruppen, 2003). Therefore, just as providing
supplemental opportunities to practice clinical skills can be helpful, providing
opportunities to practice self-assessment can also be helpful. The climate within
these practice opportunities should be supportive and non-punitive (Asadoorian
& Batty, 2005).
Integrating information from external sources Self-assessment skills also involve

integrating information from external sources. For health profession students,
the educators who evaluate them (faculty, instructors, tutors, mentors,
preceptors and practitioners) are key external sources. Given the power and
influence these educators have over students progress in their chosen
profession, feedback from educators is a critical element that undergirds the self-
assessment process. Explicit, formative feedback lets students know how their
educators perceive their performance. In turn, these perceptions can clarify
criteria expected for good performance; they can stimulate learners to identify
strengths and weaknesses; and they can help learners focus their efforts
productively (Sitzmann, Ely, Brown & Bauer, 2010). Self-assessment that does

69
not integrate educator feedback is incomplete (Motycka, Rose, Ried & Brazeau,
2010).
In many instances, feedback can be difficult to hear, and can leave students
feeling distressed; doubtful about their abilities; unmotivated; and reluctant to
persevere with their studies (Mann, 2010). Students may view even the most
well intended educator comments as a potential intolerance for their mistakes
and an indication that they lack knowledge, leaving them reluctant to seek out
and act on feedback (Mann, 2010).
Efforts to ameliorate these difficulties can include regular meeting times;

educators sharing their anecdotal notes or ongoing records of student progress;
and providing specific time-limited strategies for task improvement (Melrose,
Park & Perry, 2015). Further, opportunities where students routinely exchange
assessment feedback with their peers can help make the process less
intimidating. Feedback exchanges, where students apply the same assessment
criteria as educators, can be organized as pair-share and small group activities.
When feasible, these sessions could involve students in decisions about the
assessment criteria being used; the origin and relevance of the assessment
criteria; and practice priorities that may impact the criteria.
Incidental learning Affirming learning that students have achieved which does
not relate to predetermined goals is a valuable but often neglected aspect of
reflection. Incidental learning, also called surprise, unexpected or unintended
learning, is learning that occurs as a by-product of doing something else
(Marsick & Watkins, 1990; 2001). Incidental learning can emerge from observing
others; from discussions with people in the environment; as a consequence of
making mistakes; and from being required to adapt to or accept situations
(Kerka, 2000). Creating space for students to share and celebrate incidental
learning within their self-assessments can highlight accomplishments that may
otherwise go unnoticed. To draw out incidental learning, educators can pose
questions such as What surprised you when ? or Talk about what
happened that you didnt expect when (Melrose, Park & Perry, 2015).
In sum, integrating reflection into self-assessment can begin by simply reflecting

and seeking to gain new personal insights. Developing the skill further can
include critical reflection, which involves thinking deeply about ways of solving
problems that are occurring or have occurred. Critical reflection requires people
to change their thinking and consider new ideas. When the process of reflection
becomes especially valuable to self-assessment is when these internal processes
are coupled with the integration of information from external sources. For health
profession students, feedback from educators and peers is a primary external
source. A balance of internal, external and incidental information is needed
when students seek to assess their performance in relation to the standards,
criteria and competencies required by their profession. In the next section,
common concerns related to the validity of students self-assessments are
discussed.

70
Addressing Validity
As previously mentioned, the Dunning-Kruger effect (Dunning & Kruger, 1999)
where less able people over-state their ability and more able people under-state
their ability, has influenced peoples views about the validity of self-assessment.
Questions are often posed about whether self-assessment activities provide
accurate, dependable and truthful representations of students abilities.
Recognize that inaccuracies exist It is important to recognize that inaccuracies in

students self-assessments exist in many health professions. Research evidence
indicates that students self-assessments frequently differ from educator
assessments. Comparing classroom test scores, Brown and Harris (2013) found
only weakly positive correlations between educator ratings and students self-
assessed ratings; between actual test scores and self-estimates of performance;
and between educator and student judgments when the same rubric was used.
In simulated emergency situations, Baxter and Norman (2011) found nursing
students self-assessments were significantly inaccurate in comparison with
educators observations of their performance. Similarly, in peer simulation
situations, Sanderson, Kearney Kissell and Salisbury (2016) found dental
hygiene students self-assessments were also significantly inaccurate in
comparison to those of their educators. Measuring communication skills, Gude,
Finset, Anvik, Brheim, Fasmer et al. (2017) also reported a lack of concordance
between medical students own and their educators assessment.
Clearly, consistently achieving congruence between student and educator

assessment may not always be possible. In these instances, conceptualizing
student self-assessment as a formative developmental learning activity can be
helpful. Approaches such as video and verbal feedback have been found to
enhance the accuracy of students self-assessments (Colthart et al., 2008;
Hulsman & van der Vloodt, 2015; Volino & Das, 2014). Providing easy online
access to self-administered tests with answers has the potential to provide
students with accurate information about their level of knowledge (Miller, 2008).
Reviewing a collection of work, such as a portfolio, capstone project or reflection
summary, rather than just single instances of student performance can provide a
wider view of how students are meeting competencies (Gadbury-Amyot, Woldt
& Siruta-Austin, 2015). Implementing self-assessment activities in contexts
where the emphasis is on mastery goals (achieving competence in practice)
rather than performance goals (achieving immediate competence completing a
task) can also contribute to more accurate self-assessment (Butler, 2011).
Know the impact of a history with academic success Students accepted into health
profession programs often have a strong history of academic success. If students
are used to performing well in learning situations and have consistently received
positive feedback, they are likely to feel confident in their abilities. In turn, they
may have a view of themselves as above average. When asked to self-assess,
their thinking may be based on potential or ideal performance more than their
actual performance (Evans, McKenna & Oliver, 2001).

71
From this perspective, the lack of congruence between student and educator
assessments can be viewed as an opportunity to support students positive self-
concept and self-worth. Rather than emphasizing inaccuracy, educators can
prompt students to identify steps they have taken in the past to achieve success,
and then encourage them to apply these steps to their present learning situation.
On the other hand, students who do not have a strong history of academic
success may be unaware of inaccuracies in their self-assessment or they may be
reluctant to disclose them. If students perceive their learning environment as
overwhelming, they may not know where to begin identifying what they do not
know. In health care environments, where professionals are accountable to the
public for providing safe competent patient care, students may not feel that it is
acceptable to admit weakness. In these instances, once again, rather than
emphasizing inaccuracy, educators can highlight the links between accurate
practitioner self-assessment (which includes admitting to not knowing and then
seeking out needed information) and patient safety (Sujata, Oliveras & Edson,
2001).
Create links to affective outcomes A further consideration influencing the validity of

self-assessment is the distinction between cognitive and affective learning
outcomes. Cognitive learning outcomes are more factually based, may relate to a
particular course of study, and are associated with external sources such an
exam grade or educator rating (Sitzmann, Ely, Brown & Bauer, 2010). Affective
learning outcomes are related to internal sources, extend beyond a specific
course or learning event and they include feeling satisfied, motivated, able to
carry out tasks and willing to apply and use knowledge gained (Sitzmann et.al.).
A meta-analysis of evaluation studies revealed that construct validity of self-
assessment was strongly correlated with affective outcomes (particularly
satisfaction and motivation) and only weakly correlated with cognitive
outcomes (Sitzmann et. al.). Given this correlation, self-assessment activities
linked to affective outcomes have a greater chance of yielding a more accurate
measurement result.
Practitioners from different disciplines and practice areas all need self-
assessment skills that help develop their thinking beyond the boundaries of a
single course or learning event (He & Canty, 2013; Mann, 2010). Therefore, when
addressing validity in self-assessment, connections between self-assessment
activities and the nature of the outcomes being measured is an important
consideration. Knowing the inherent difficulty in quantifying success with
affective achievements, addressing validity in self-assessment must be grounded
in a commitment to designing activities that are suitable for measuring broad
outcomes, mastery goals and critically reflective thinking.

72
Conclusion
Self-regulating health professionals must be able to assess what they know; what
they dont know in relation to what they are expected to know; and what they
need to learn in order to provide safe competent care. Self-assessment is a
learned skill and one that can be best developed through early, consistent and
supportive activities during pre-service educational programs. Conceptualizing
self-assessment as a formative learning activity offers a perspective where
students and educators focus on improving performance rather than simply
grading competencies.
A balance of reflection that taps into critical thinking and accurate

representations of students abilities is needed for self-assessment to be viewed
as valid. Achieving this balance between reflection and validity is complex. In
order to integrate reflection into their self-assessments, students must think
critically and find new ways to solve problems and find meaning. They must
analyze their performance in relation to pre-determined outcomes. They must
also extend their own thinking to include feedback received from educators,
peers and other external sources. Further, they must take incidental or surprise
learning into account. Journals, either written or audio/video recorded are
useful tools for developing reflective thinking. Inviting students to self-assess at
the beginning of educator-student conversations; in pair-share discussions; and
in small group conferences can provide valuable practice opportunities.
The validity of student self-assessment is often questioned because students

views of their abilities can be very different from those of their educators. Less
able students over-estimate their ability and more able students underestimate
their ability. Questions about validity can begin to be addressed by first
recognizing that inaccuracies exist. Providing video feedback; self-administered
online tests with answers; and reviewing a collection of work instead of a single
instance can help students self-assess more accurately.
A history of either success or limited academic success impacts congruence

between student and educator assessments. Reminding students of how they
achieved success in the past provides useful guidance. Emphasizing how
disclosing areas of weakness can lead to increased patient safety offers
meaningful rationale for moving forward.
Finally, stronger validity can be achieved when links are created between self-
assessment activities and broad affective outcomes related to feeling satisfied
with knowledge that has been gained and feeling motivated to apply and use
that knowledge. Continuing to find ways to balance reflection and validity in
self-assessment is both a challenge and an opportunity for students and
educators in the health professions.

73
References
Andrade, H., & Valtcheva, A. (2009). Promoting learning and achievement through self-
assessment. Theory into Practice, 48(1), 12-19.
http://dx.doi.org/10.1080/00405840802577544.
Asadoorian, J. & Batty, H. (2005). An evidence-based model of effective self-assessment
for directing professional learning. Journal of Dental Education, 69(12), 1315-1323.
http://www.jdentaled.org/content/69/12/1315.full
Austin, Z. & Gregory, P. (2007). Evaluating the accuracy of pharmacy students self-
assessment skills. American Journal of Pharmaceutical Education, 71(5), 18.
Baxter, P., & Norman, G. (2011). Self-Assessment or self-deception? A negative
association between self-assessment and performance. Journal of Advanced
Nursing, 67(11), 24062413. doi:10.1111/j.1365-2648.2011.05658.x
Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom
assessment. Phi Delta Kappan, 80(2), 139-148.
Blanch-Hartigan, D. (2011). Medical students self-assessment of performance: Results
from three meta-analyses. Patient Education and Counseling, 84(1), 3 9. DOI:
http://dx.doi.org/10.1016/j.pec.2010.06.037
Boud, D. (1991). HERDSA Green Guide No 5. Implementing student self-assessment
(Second ed.). Campbelltown: The Higher Education Research and Development
Society of Australasia (HERDSA).
Boud, D. (1995). Enhancing learning through self-assessment. London. Routledge Falmer.
Boud, D. & Falchikov, N. (1989). Quantitative studies of self-assessment in higher
education: A critical analysis of findings. Higher Education, 18, 529-549.
Boud, D. & Falchikov, N. (2006). Aligning assessment with long-term learning.
Assessment and Evaluation in Higher Education, 31, 399413.
DOI:10.1080/02602930600679050.
http://www.jhsph.edu/departments/population-family-and-reproductive-
health/_docs/teaching-resources/cla-01-aligning-assessment-with-long-term-
learning.pdf
Bourke, R. (2016). Liberating the learner through self-assessment. Cambridge Journal of
Education, 46(1), 97-111. DOI: 10.1080/0305764X.2015.1015963
Brookfield, S. (1995). Becoming a critically reflective teacher. San Francisco: Jossey Bass.
Brookfield, S. (1998). Critically reflective practice. Journal of Continuing Education in the
Health Professions, 18, 197205. DOI:10.1002/chp.1340180402
Brown, G. T. L., & Harris, L. R. (2013). Student self-assessment. In J. H. McMillan (Ed.).
The SAGE handbook of research on classroom assessment (pp. 367-393). Thousand
Oaks, CA: Sage.
Butler, R. (2011). Are positive illusions about academic competence always adaptive,
under all circumstances: New results and future directions. International Journal
of Educational Research, 50(4), 251-256. DOI: 10.1016/j.ijer.2011.08.006
Colthart, I., Bagnall, G., Evans, A., Allbutt, H., Haig, A., Illing, J. & McKinstry, B. (2008).
The effectiveness of self-assessment on the identification of learning needs,
learner activity, and impact on clinical practice: BEME Guide No. 10. Medical
Teacher, 30,124145.
Constantinou, M., & Kuys, S. (2013). Physiotherapy students find guided journals useful
to develop reflective thinking and practice during their first clinical placement:
A qualitative study. Physiotherapy, 99(1), 4955. DOI:10.1016/j.physio.2011.12.002
Dabbagh, N., & Kitsantas, A. (2012). Personal learning environments, social media, and
self-regulated learning: A natural formula for connecting formal and informal
learning. Internet and Higher Education, 15(1), 3-8.
DOI:10.1016/j.iheduc.2011.06.002

74
Davis, D., Mazmanian, P., Fordis, M., Van Harrison, R., Thorpe, K. et al. (2006). Accuracy
of physician self-assessment compared with observed measures of competence:
A systematic review. The Journal of the American Medical Association, 296(9), 1094
1102.
Desjarlais, M. & Smith, P. (2011). A comparative analysis of reflection and self-
assessment. International Journal of Process Education, 3(1), 3-18.
http://www.processeducation.org/ijpe/2011/reflectionh.pdf
Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the
educative process. New York: D.C. Heath and Company.
Dunning, D., Heath, C. & Suls, J. M. (2004). Flawed self-assessment: Implications for
health, education, and the workplace. Psychological Science in the Public Interest,
5(3), 69-106. https://faculty-gsb.stanford.edu/heath/documents/PSPI%20-
%20Biased%20Self%20Views.pdf
Ehrlinger, J., Johnson, K., Banner, M., Dunning, D. & Kruger, J. (2008). Why the unskilled
are unaware: Further explorations of (absent) self-insight among the
incompetent. Organizational Behavior and Human Decision Processes, 105 (1), 98
121.doi:10.1016/j.obhdp.2007.05.002
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2702783/
Eva, A. & Regehr, G. (2005). Self-assessment in the health professions: a reformulation
and research agenda. Academic Medicine, 80(10 Suppl):S46-54.
Evans, A., McKenna, C. & Oliver, M. (2001). Towards better understanding of self-
assessment in oral and maxillofacial surgery. Medical Education, 35(11),1077.
Falchikov, N. & Boud, D. (1989). Student self-assessment in higher education: A meta-
analysis. Review of Educational Research, 59, 395430.
Fitzgerald, J., White, C., & Gruppen, L. (2003). A longitudinal study of self-assessment
accuracy. Medical Education, 37(7), 645-649.
Gadbury-Amyot, C., Woldt, J. Siruta-Austin, K. (2015). Self-assessment: A review of the
literature and pedagogical strategies for its promotion in dental education.
Journal of Dental Hygiene, 89(6), 357-364
Galbraith R., Hawkins, R. & Holmboe, E. (2008). Making self-assessment more effective.
Journal of Continuing Education in the Health Professions, 28(1), 2024.
Gordon, M.J. (1991). A review of the validity and accuracy of self-assessments in health
professions training. Academic Medicine, 66, 762769.
Gude, T., Finset, A. Anvik, T., Brheim, A., Fasmer, O. et al. (2017). BMC Medical
Education, 17,107.DOI 10.1186/s12909-017-0943-y
He, X., & Canty, A. (2013). A comparison of the efficacy of test-driven learning versus
self-assessment learning. The Journal of Chiropractic Education, 27(2), 110115.
http://doi.org/10.7899/JCE-13-6
Hulsman, R. & van der Vloodt, J. (2015). Self-evaluation and peer-feedback of medical
students communication skills using a web-based video annotation system.
Exploring content and specificity, Patient Education and Counseling, 98(3), 356-363
http://dx.doi.org/10.1016/j.pec.2014.11.007.
Johns, C. (2017). Becoming a reflective practitioner (5th Ed.). Oxford, UK: Wiley-Blackwell
Kajander-Unkuri, S., Meretoja, R., Katajisto, J., Saarikoski, M., Salminen, L., et al. (2013).
Self-assessed level of competence of graduating nursing students and factors
related to it. Nurse Education Today, 34(5), 795 801.
Kanthan, R. & Senger, J. (2011). An appraisal of students' awareness of "self-reflection" in
a first-year pathology course of undergraduate medical/dental education. BMC
Medical Education, 11, 67. https://doi.org/10.1186/1472-6920-11-67
Kerka, S. (2000). Incidental learning. Trends and Issues Alert 18. Columbus OH: ERIC
Clearinghouse on Adult, Career and Vocational Education. http://www.calpro-
online.org/eric/docs/tia00086.pdf

75
Koh, Y., Wong, M. & Lee, J. (2014). Medical students reflective writing about a task-
based learning experience on public health communication. Medical Teacher,
36(2), 1219. DOI: 10.3109/0142159X.2013.849329.
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in
recognizing ones own incompetence lead to inflated self-assessments. Journal of
Personality and Social Psychology, 77(6), 1121-1134.
https://www.ncbi.nlm.nih.gov/pubmed/10626367
Lew, M. & Schmidt, H. (2011). Self-reflection and academic performance: Is there a
relationship? Advances in Health Sciences Education, 16(4), 529545.
http://doi.org/10.1007/s10459-011-9298-z
Linn, B., Arostegui, M. & Zeppa, R. (1975). Performance rating scale for peer and self
assessment. British Journal of Medical Education, 9(2), 98101.
Lundquist, L. M., Shogbon, A. O., Momary, K. M. & Rogers, H. K. (2013). A comparison
of students self-assessments with faculty evaluations of their communication
skills. American Journal of Pharmaceutical Education, 77(4), 72.
http://doi.org/10.5688/ajpe77472
Mann, K. (2010). Self-assessment: The complex process of determining how we are
doingA perspective from medical education. Academy of Management Learning
& Education, 9(2), 305313.
Mann, K., Gordon, J. & MacLeod, A. (2009). Reflection and reflective practice in health
professions education: A systematic review. Advances in Health Science Education,
14, 595621. DOI 10.1007/s10459-007-9090-2
Marsick, V. & Watkins, K. (1990). Informal and incidental learning in the workplace. London
& New York: Routledge.
Marsick, V. & Watkins, K. (2001). Informal and incidental learning. New Directions for
Adult and Continuing Education, 89, 2534.
Melrose, S., Park, C. & Perry, B. (2015). Creative clinical teaching in the health professions.
Retrieved from http://epub-fhd.athabascau.ca/clinical-teaching/
Mezirow, J. (1998). On critical reflection. Adult Education Quarterly, 48, 185-198.
http://dx.doi.org/10.1177/074171369804800305
Miller, P. (2008). Self-assessment: The disconnect between research and rhetoric.
Physiotherapy Canada 60(2), 117124. DOI: 10.3138/physio.60.2.117
Mort, J. & Hansen, D. (2010). First-year pharmacy students self-assessment of
communication skills and the impact of video review. American Journal of
Pharmaceutical Education, 74(5), 17.
Motycka, C., Rose, R., Ried, D. & Brazeau, G. (2010). Self-assessment in pharmacy and
health science education and professional practice. American Journal of
Pharmaceutical Education, 74 (5) Article 85.
Passi, V. & Southgate, L. (2016). Revisiting 'Performance rating scale for peer and self
assessment'. Medical Education, 50(3),267-270. DOI: 10.1111/medu.12798
Pisklakov, S., Rimal, J. & McGuirt, S. (2014). Role of self-evaluation and self-assessment
in medical student and resident education. British Journal of Education, Society and
Behavioral Science, 4(1), 19.
Redwood, C., Winning, T. & Townsend, G. (2010). The missing link: Self-assessment and
continuing professional development. Australian Dental Journal, 55, 1519.
DOI:10.1111/j.1834-7819.2009.01177.x
http://onlinelibrary.wiley.com/doi/10.1111/j.1834-7819.2009.01177.x/full
Regehr, G. & Eva, K. (2006). Self-assessment, self-direction, and the self-regulating
professional Clinical Orthopaedics and Related Research, 449, 3438.
http://innovationlabs.com/r3p_public/rtr3/pre/pre-read/Self-
assessment.Regher.Eva.2006.pdf

76
Sanderson, T., Kearney, R., Kissell, D. & Salisbury, J. (2016). Evaluating student self-
assessment through video-recorded patient simulations. Journal of Dental
Hygiene, 90(4), 257-262.
Sargeant J. (2008). Toward a common understanding of self-assessment. Journal of
Continuing Education in Health Professions. 28, 14. DOI: 10.1002/chp.148.
https://www.ncbi.nlm.nih.gov/pubmed/18366124
Schn, D. (1983). The reflective practitioner: How professionals think in action. London:
Temple Smith.
Sitzmann, T., Ely, K., Brown, K. & Bauer, K. (2010). Self-assessment of knowledge: A
cognitive learning or affective measure? Academy of Management Learning &
Education, 9(2), 169-191.
Sujata, B., Oliveras, E. & Edson, W. (2001). How can self-assessment improve the quality
of healthcare? Operations Research Issue Paper 2(4). Published for the U.S. Agency
for International Development(USAID) by the Quality Assurance (QA)Project,
Chevy Chase, MD and JHPIEGO Corporation, Baltimore, MD.
Tulgar, A. (2017). Selfie@ssessment as an alternative form of self-assessment at
undergraduate level in higher education. Journal of Language and Linguistic
Studies, 13(1), 321-335.
Volino, L. & Das, R. (2014). Video review in self-assessment of pharmacy students
communication skills. Journal of Education and Learning, 3(4), 26-36.
doi:10.5539/jel.v3n4p26. http://dx.doi.org/10.5539/jel.v3n4p26
Ward, M., Gruppen, L. & Regehr, G. (2002). Measuring self-assessment: Current state of
the art. Advances in Health Sciences Education Theory and Practice, 7, 6380.

77

Vol. 16, No. 8, pp. 77-84, August 2017
Innovative Teaching with Use of an Art Work
Marios Koutsoukos
ASPETE,
School of Pedagogical and Technological Education,
Kozani, Greece
Iosif Fragoulis
ASPETE,
School of Pedagogical and Technological Education,
Patra, Greece
Abstract. This study presents the use of an art work in the teaching of a
course entitled Environment and Agriculture in the last grade of a
Vocational Lyceum. More specifically, the painting of Dutch artist Jan
van Goyen entitled A windmill by a river was used as a teaching tool
in an innovative approach of teaching the topic of wind energy as a
renewable natural resource. The purpose of this approach was to
enhance students critical judgment concerning renewable sources of
energy, such as wind energy, fostering at the same time transformative
learning, as students become aware and critical of their own initial
assumptions and develop new perspectives. Taking into consideration
and applying David Perkins theory about using art works in teaching,
students were able to strengthen their creative thinking, getting to know
the various aspects of wind energy through emotional observation and
thorough study of an art work.
Keywords: Teaching; Use of art; Observation; Critical judgement
Introduction
Several academics and scholars, such as Gardner, Eisner, Perkins, and Kokkos,
have occasionally expressed the view that education through the employment of
artwork contributes to the learning process, and it also helps students develop a
variety of skills, including critical thinking, cognitive development, creativity,
discovery of new knowledge and expression of emotions (Gardner, 1990;
Perkins, 1994; Eisner, 2002; Kokkos, 2011; Phillips & Fragoulis, 2012). Therefore,
artworks can prove to be a precious treasure that can be utilised appropriately in
educational practices, serving a wide range of objectives (Barnes, 2015).
In recent years, the use of visual arts in education has also been extended to the
teaching of environmental sciences. According to research in the international

78
bibliography, this practice can result in impressive learning outcomes (Inwood

& Taylor, 2012; Rosenthal, 2003; Neperud, 1997). For instance, a painting
depicting a natural landscape can function as a stimulus for introducing an
interesting teaching approach to an environmental education course. More
specifically, a painting portraying a river or a lake can serve as the starting point
for teaching water resource management.
Use of art in education

Over the past few years, both formal and non-formal educational approaches
have been developing based on the use of art for educational purposes.
Education through the use of art involves the introduction of artworks in the
teaching process the ones related to the subject of teaching in the context of
processing a learning objective. Through the processing of these artworks, the
meaning they carry gets revealed, and then it can be used as a pre-text for
undertaking a deeper approach towards the subjects under consideration
(Barnes, 2015; Kokkos, 2011; Efland, 2002).
In Greece, the first systematic efforts to train the teachers of formal education,
regarding the use of art in education, were implemented during the school year
2011-2012, as part of the implementation of the Major Edification Program
project. The Hellenic Open University has also made a substantial contribution
to this field, since the Academic year 2012-2013, in the framework of the
Postgraduate Programs Studies in Education and Adult Education. Similar
initiatives have been assumed by the Adult Education Scientific Association for
the edification of teachers and adult educators in the utilization of works of art
in the teaching practice.
Models of approaching works of art in the educational process

Artwork approach models provide a theoretical and methodological framework,
according to which the student, gradually approaching a work of art, enters
areas of deeper understanding by activating his critical and reflective spirit
(Kokkos, 2011). Four models are described in the relevant literature. Those are
the models of Feldman, Broudy, Anderson and Perkins. According to Feldman
(1967), the approach of an artwork is realized through four interrelated phases:
description, analysis, interpretation and evaluation. The Broudy (1972) model,
which is related to that of Feldman, emphasizes the field of activating the
aesthetic observations of an artwork (e.g., searching for shapes, volumes, ways
to connect them), as well as the identification of the expressiveness that the
artwork radiates.
Andersons (1993) model includes the following reflective processes: initial

reaction, description, interpretation, evaluation. It presents several similarities to
the Feldman model, but differs from that in the first phase, which in this model
is described as the initial reaction. Within the context of this model, and
particularly during the first phase, the student proceeds to formulate an
instinctive reaction to the observed artwork. In the aforementioned models, the
analysis of an artwork is based on the identification and observation of a subject
that is gradually approached, starting from its instantly perceived features and,

79
finally, unveiling its inner-deeper characteristics, that is, the deeper meaning
of the work.
The Perkins artwork analysis model

According to Perkins, the approach of works of art is done in such a way so that
the observer is able to distinguish: a) the key actions that trigger the reflective
spirit, b) the possible questions that can cause the reflective effect, c) the teaching
approaches one needs to implement in order to provoke reflection. These three
elements form a comprehensive methodology for the reflective observation of
works of art.
Perkins model is considered more comprehensive in comparison to other

models that have been proposed from time to time, as it can apply to the
approach of various artworks, requiring only minimal modifications (Kokkos,
2011; Phillips & Fragoulis, 2012). As part of the artwork observation process
suggested by Perkins, the observer adopts a documented view at the
observation. Thus, students are not confined only to value judgments, such as I
like it or I dislike it but, by observing specific elements of the artwork, they
methodically develop an argument that strengthens their opinion on the art
work (Kokkos, 2011). In addition, thoughtful looking at art appears to have an
instrumental value as it provides an excellent setting for the development of
better thinking (Perkins, 1994).
Perkins argues that, through this process, students are not only able to enrich
their aesthetic experience, but also, at the same time, cultivate a critical-reflective
approach towards the events taking place within and outside the educational
framework (Perkins, 1994). Perkins' model consists of four phases that are
correlated and can be summarized as:
a) giving looking time,
b) making looking broad and adventurous,
c) making looking clear and deep and
d) making looking organized.
More specifically, in the first phase students are given time to observe carefully
an artwork, at first without attempting to understand or evaluate it. Next,
during the second phase, students try to approach the artwork bearing in mind
the question what is that the artist would like us to observe (Phillips &
Fragoulis, 2012). Proceeding to the third phase, analytical and deeper
observation takes place, as students in collaboration with their teacher attempt
to answer to several questions which arose during previous observation, as well
as to interpret meanings and draw conclusions. Finally, during the fourth phase,
students holistically approach the art work by taking advantage the
observational experience of previous stages.

80
Teaching Scenario
The main and general purpose of this educational approach was to develop
students' critical thinking and reflection on issues related to the use of
alternative, environmentally friendly types of energy, as well as to transform
their initial assumptions regarding their importance. With regard to the
expected learning outcomes deriving from the use of this specific educational
practice, the students, after processing the artwork, had to be able to:
express awareness on issues related to the exploitation of mild and
environmentally friendly forms of energy,
develop a holistic approach to individual environmental issues,
particularly regarding the use of wind energy,
demonstrate critical thinking on environmental issues,
apply the stages of the Perkins model to the artwork approach,
adopt a positive attitude towards the creative exploitation of important
works of art for the purpose of approaching environmental issues.
The painting A windmill by a river, by the Dutch painter Jan Van Goyen, was
created in 1642 and is currently kept at the National Gallery of London. This
depicts a landscape of the Dutch province, with a gray and hazy sky covering
of the painting, while in the lower right side of the composition, emerges a
windmill at the rivers edge (Beck, 1977). The windmills, a trademark of the
Dutch countryside, were largely used for pumping and supplying water from
rivers and lakes, as well as for the grinding of grains, at the time the artwork was
created (Beck, 1977). Since the days of Van Goyen and up to the present day,
wind energy, generated by the exploitation of the winds, constitutes a mild and
environmentally friendly form of energy, as it does not burden the natural
resources.
Within this context, and in the teaching of the Renewable Natural Resources
thematic sector of the course Environment and Agriculture, the use of this
particular artwork as a teaching tool was attempted for the development of
critical thinking. Perkins artwork observation model, as described in the
relevant literature, (Perkins, 1994; Kokkos, 2011) was used as a teaching tool.
This particular teaching approach was implemented over the course of 3
teaching hours, including 2 intermediate breaks.
More specifically, the phases used were as follows:

Phase 1: Time for observation,
Phase 2: Open and adventurous observation,
Phase 3: Analytical and deeper observation,
Phase 4: Overview of the process.
More specifically, in the first phase, students were given the necessary time to
observe the artwork and express their initial spontaneous estimates. This way,
the teacher invited students to observe the artwork, focusing on the colors, the

81
figures, the landscape, and the windmill. Afterwards, following the expression
of the initial spontaneous remarks regarding the artwork, the educator used the
collaborative method and invited the students to split into groups and write in
captions what they noticed during their initial contact with the work of art.
Then, using the brainstorming technique, the students were asked to capture
and write down concepts, ideas and views, regarding their first visual contact
with the artwork.
Moreover, through the use of visual intelligence, the teacher invited the students
to make use of the previous knowledge they might have on this artwork, in
order to stimulate their perceptiveness, while outlining the natural landscape
depicted in the painting. Subsequently, the students were asked to distance
themselves from the artwork for a little while, averting their gaze off it for a
short period of time. After a while, returning to the observation of the painting,
the teacher posed a question regarding the position of the windmill in the
depicted landscape, and the students, divided into working groups, were
invited to circle the interesting features of the artwork and formulate further
questions regarding the painting.
Moving on to second phase, that of open and adventurous observation, the

teacher attempted to stimulate the students attention, asking them why is the
sky illustrated cloudy and full of winds, as well as why is the windmill
positioned next to the river. The teacher went on with the pursuit of the spirit,
the symbols, and the meanings, exploring whether the view of this specific
artwork stimulates some feelings to the students and whether there are certain
messages behind the artworks symbolisms.
Another important activity of the second phase was the alteration of the
observation scale, where the teacher focused on the sky, the positioning of the
windmill, as well as the river, and asked of the students to circle and study a
specific area in the artwork, effectively inviting them to identify the central
theme in relation to the regional one. At this point, the space-time placement of
the artwork was carried out, as the teacher provided the historical details of the
painting, which was created by Van Goyen in 1642 and illustrates a typical
landscape of the Dutch countryside during that time, where windmills were
used for pumping and supplying water from rivers and lakes as well as for grain
milling.
Entering the third phase of the analytical observation, the teacher posed the
question What message does the creator of the artwork wish to convey, by
placing the windmill in this particular landscape?. At the same time, the
students, working in workgroups, were invited to respond to key concerns, for
example: Why is the cloudy and windy sky prevailing in of the painting?
How is the windmill near the river connected with the rest of the landscape?
How is the wind energy, generated by the windmill, being utilized?. Moreover,
through the use of active and participatory techniques, such as discussion and
brainstorming, the teacher, in the context of focused intellectual change activity,
asked from the students to isolate parts of the artwork or modify them

82
accordingly. At this point, questions, such as the following, arose: If the

windmill was absent from the landscape, would that alter the structure of the
painting? and Were the sky not cloudy and full of winds, would the message
of the artwork be altered?.
Delving deeper into the third phase of the analytical observation, the teacher
invited the students to cover some elements of the artwork, using their hands,
such as the windmill or the river. Then the teacher asked them to observe what
exactly would happen, in terms of the artworks quality, if these elements were,
in fact, absent. This way, the students were led to the discovery of the
significance of a specific object while, simultaneously, answering the following
questions: Is the presence of the windmill near the river important? and How
is the windmill connected to the use of wind energy and the pumping of water
from the adjacent river?.
Subsequently, the teacher, making use of additional sources (e.g., internet,

encyclopedias, or related books), invited the students to compare this particular
painting to other artworks by the same artist, created during the same time
period and utilizing a similar subject, and encouraged them to identify the
similarities and the differences between them and reflect on why these
similarities and differences exist. Upon completion of the 3rd Phase, arises the
critical question: How can environmentally friendly forms of energy, such as
wind energy, be exploited?.
The fourth and final phase of the Perkins artwork observation model concerned
the review of the whole process. More specifically, the teacher invited the
students to split into workgroups of 4-5 people and answer the question: What
is the link between the artwork and the critical question?. At first, the students
were asked to record the thoughts they had before approaching the visual
artwork in question, and then to write down the thoughts they have now, after
the process of approaching the artwork.
Table 1: The art work used as a teaching tool.

Painting title A windmill by a river shown in figure 1 below
Artist Jan van Goyen (1596 1656)
Characteristics 29,4cm 36,3 cm
Year created 1642
http://www.nationalgallery.org.uk/paintings/ja
Web
n-van-goyen-a-windmill-by-a-river
Museum National Gallery, London

83
Figure 1: A windmill by a river
Conclusion
The evaluation process carried out after the implementation of the method
revealed that, through the systematic observation of the work of art, the
students: a) acquired knowledge and expressed awareness regarding issues
related to the exploitation of mild forms of energy, b) developed a contemplative
attitude towards environmental issues, c) acknowledged the possibilities offered
by the utilization of works of art to the approach of environmental issues, d)
became familiar with the methodology of approaching artwork as a tool for
developing critical thinking, e) adopted a positive attitude towards the creative
exploitation of important works of art in the approach of environmental
education issues, and, f) developed their creative thinking and broadened their
perception by using their imagination and ingenuity.
In conclusion, through the use of an artwork, activation of the students was
attempted, in order to actively engage them in the learning process and the
examination of data and parameters of a subject. Jan Van Goyens painting, A
windmill by a river, has been a useful teaching tool, utilized in the context of an
effective and innovative approach to the environmental issue of renewable
natural resources and, more specifically, wind energy. This process has, on the
one hand, intrigued the interest of the students, who participated with warmth
and excitement, and, on the other hand, made the teaching approach of these
specific cognitive objects more experiential, interactive and lively for the teacher.
As far as practical implications are concerned, the case study presented in this
article can offer interesting insights for teachers, educators and researchers
interfering with use of art works in education. The teaching approach suggested
in this article could be adopted by educators who wish to teach individual
modules of environmental education in an experiential and innovative way. In

84
this direction, the present study enriches the relevant literature and at the same
time, provides ideas for the use of art works in the teaching of environmental
sciences.
Acknowledgements
The authors would like to thank the students who participated to the present
research, giving a useful feedback to this innovative teaching procedure. In
addition, the authors would like to thank the editor and the reviewers of this
journal for their valuable comments on an earlier version of this paper.
References
Anderson, T. (1993). Defining and structuring art criticism for education. Studies in Art
Education, A Journal of Issues and Research, 34, 199-208.
Barnes, R. (2015). Teaching art to young children. Third Edition. New York: Routledge
Beck, H.U. (1977). Jan Van Goyen, 1596-1656: Poet of the Dutch Landscape. London: Alan
Jacobs Gallery.
Broudy, S. (1972). The Enlightened Cherishing: An Essay on Aesthetic Education. Urbana, IL:
University of Illinois Press and Kappa Delta Pi.
Burnham, R. & Kai-Kee, E. (2011). Teaching in the Art Museum. Interpretation as Experience.
Los Angeles: The J.Paul Getty Museum
Efland, A. (2002). Art and Cognition: Integrating the Visual Arts in the Curriculum. New
York: Teachers College Columbia University.
Eisner, E. (2002). The Arts and the Creation of Mind. New Haven and London: Yale
University Press.
Feldman, E. (1967). Arts as Image and Idea. New Jersey: Prentice Hall.
Fragoulis, I. (2003). Life Long Learning and Environmental Education. Teaching Approaches.
Athens: Papazisis (in Greek)
Fragoulis, I. (2014). Local History-Educational Material for Life Long Learning Centers.
Athens: Ministry of Education (in Greek)
Gardner, H. (1990). Art Education and Human Development. California, Los Angeles: The
Getty Education Institute for the Arts.
Inwood, H. & Taylor, R. (2012). Creative Approaches to Environmental Learning: Two
Perspectives on Teaching Environmental Art Education, International Electronic
Journal of Environmental Education, 2(1), 65-75
Perkins, D. (1994). The Intelligent Eye: Learning to Think by Looking at Art. California,
Los Angeles: The Getty Education Institute for the Arts.
Kleiner, F.S. (2015). Gardners art through the ages. 15th Edition. The Western
Perpesctive. Boston: Cengage Learning
Kokkos, A. (2011). Transformative Learning through Aesthetic Experience Towards a
Comprehensive Method. Journal of Transformative Education, 8(3), 155-177
Neperud, R. (1997). Art, ecology and art education: Practices and linkages. Art Education,
50(6), 14-20.
Phillips, N. & Fragoulis, I. (2012). The use of art in the Teaching Practice for Developing
Communication Skills in adult, International Education Studies, (5)2, 132-138.
Rosenthal, A. (2003). Teaching systems theory and practice through environmental art.
Ethics and the Environment, 8(1), 153-168.

Vol 16 No 8 - August 2017

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Vol 16 No 8 - August 2017

Uploaded by

Copyright:

Available Formats

IJLTER.

Balancing Reflection and Validity in Health Profession Students Self-Assessment ................................................... 65

Innovative Teaching with Use of an Art Work ................................................................................................................. 77

International Journal of Learning, Teaching and Educational Research

Learning from each other: Dialogical

Rene B. Van Vechten

Keywords: Argumentation; Online deliberation; Online teaching;

Introduction: Learning from each other: Dialogical Argumentation in

2017 The authors and IJLTER.ORG. All rights reserved.

can participate in discussions on the same terms as others, without respect to

2017 The authors and IJLTER.ORG. All rights reserved.

Arguments include supports such as warrants, backing, and qualifiers. In an

2017 The authors and IJLTER.ORG. All rights reserved.

The Collaborative Website Overview

2017 The authors and IJLTER.ORG. All rights reserved.

2017 The authors and IJLTER.ORG. All rights reserved.

Table 2: Discussion Question Forums Selected for Analysis

2017 The authors and IJLTER.ORG. All rights reserved.

In our model, Type 0 would include a response consisting mainly of

2017 The authors and IJLTER.ORG. All rights reserved.

supporting logic. Virtually no substantive or meaningful information was

2017 The authors and IJLTER.ORG. All rights reserved.

To measure depth of academic engagement, we created an index based on five

Research Questions and Hypotheses. We were interested in how seriously

Findings and Discussion

2017 The authors and IJLTER.ORG. All rights reserved.

encompassed corrections and clarifications. The largest percentage (68.5%)

2017 The authors and IJLTER.ORG. All rights reserved.

2017 The authors and IJLTER.ORG. All rights reserved.

2017 The authors and IJLTER.ORG. All rights reserved.

2017 The authors and IJLTER.ORG. All rights reserved.

Gay Marriage (2013)

Gay Marriage (2012)

L-5 L-4 L-3 L-2 L-1

2017 The authors and IJLTER.ORG. All rights reserved.

2017 The authors and IJLTER.ORG. All rights reserved.

Chadha, A. (2017c). Comparing Student Reflectiveness in Online Discussion

2017 The authors and IJLTER.ORG. All rights reserved.

2017 The authors and IJLTER.ORG. All rights reserved.

International Journal of Learning, Teaching and Educational Research

A Qualitative Examination of Factors for Success

Janet Delgado, Ed.D.

Lorraine T. Benuto , Ph.D.

Abstract. The dramatic increase of English Language Learners (ELLs)

2017 The authors and IJLTER.ORG. All rights reserved.

Unfortunately, this exacerbates an already impoverished learning environment

2017 The authors and IJLTER.ORG. All rights reserved.

2017 The authors and IJLTER.ORG. All rights reserved.

A Framework for Teaching protocol (Danielson, 2007). The classroom

2017 The authors and IJLTER.ORG. All rights reserved.

conclusion of each observation, a brief post-observation interview with the

2017 The authors and IJLTER.ORG. All rights reserved.

Totals 296 411 176 883

2017 The authors and IJLTER.ORG. All rights reserved.

sheltered instruction ELL classrooms on a regular basis. Although a minimum

Explicit academic/language instruction, bilingual support, differentiated

2017 The authors and IJLTER.ORG. All rights reserved.

engagement and support during explicit content and strategy instruction.

Q2. How are successful content-based, sheltered instruction ELL

2017 The authors and IJLTER.ORG. All rights reserved.

2017 The authors and IJLTER.ORG. All rights reserved.

categorized as caring, supportive and knowledgeable, according to the

2017 The authors and IJLTER.ORG. All rights reserved.

classrooms. As a result of their close professional relationships, the teachers and