Vol 4 No 1 - April 2014

IJLTER.
ORG
International Journal
of
Learning, Teaching
And
Educational Research
p-ISSN: 1694-2493
e-ISSN: 1694-2116
Vol.4 No.1
PUBLISHER International Journal of Learning, Teaching and
London Consulting Ltd Educational Research
District of Flacq
Republic of Mauritius
www.ijlter.org The International Journal of Learning, Teaching
and Educational Research is an open-access
Chief Editor journal which has been established for the dis-
Dr. Antonio Silva Sprock, Universidad Central de semination of state-of-the-art knowledge in the
Venezuela, Venezuela, Bolivarian Republic of field of education, learning and teaching. IJLTER
welcomes research articles from academics, ed-
Editorial Board
Prof. Cecilia Junio Sabio ucators, teachers, trainers and other practition-
Prof. Judith Serah K. Achoka ers on all aspects of education to publish high
Prof. Mojeed Kolawole Akinsola quality peer-reviewed papers. Papers for publi-
Dr Jonathan Glazzard cation in the International Journal of Learning,
Dr Marius Costel Esi Teaching and Educational Research are selected
Dr Katarzyna Peoples through precise peer-review to ensure quality,
Dr Christopher David Thompson
originality, appropriateness, significance and
Dr Arif Sikander
Dr Jelena Zascerinska readability. Authors are solicited to contribute
Dr Gabor Kiss to this journal by submitting articles that illus-
Dr Trish Julie Rooney trate research results, projects, original surveys
Dr Esteban Vzquez-Cano and case studies that describe significant ad-
Dr Barry Chametzky vances in the fields of education, training, e-
Dr Giorgio Poletti learning, etc. Authors are invited to submit pa-
Dr Chi Man Tsui
Dr Alexander Franco
pers to this journal through the ONLINE submis-
Dr Habil Beata Stachowiak sion system. Submissions must be original and
Dr Afsaneh Sharif should not have been published previously or
Dr Ronel Callaghan be under consideration for publication while
Dr Haim Shaked being evaluated by IJLTER.
Dr Edith Uzoma Umeh
Dr Amel Thafer Alshehry
Dr Gail Dianna Caruth
Dr Menelaos Emmanouel Sarris
Dr Anabelie Villa Valdez
Dr zcan zyurt
Assistant Professor Dr Selma Kara
Associate Professor Dr Habila Elisha Zuya
VOLUME 4 NUMBER 1 April 2014
Table of Contents
Angovian Methods for Standard Setting in Medical Education: Can They Ever Be Criterion Referenced? ............. 1
Brian Chapman
Development Model of Learning Objects Based on the Instructional Techniques Recommendation ....................... 27
Antonio Silva Sprock, Julio Cesar Ponce Gallegos and Mara Dolores Villalpando Caldern
Influential Factors in Modelling SPARK Science Learning System ............................................................................... 36

Marie Paz E. Morales
Investigating Reliability and Validity for the Construct of Inferential Statistics ......................................................... 51
Saras Krishnan and Noraini Idris
Influence of Head Teachers Management Styles on Teacher Motivation in Selected Senior High Schools in the
Sunyani Municipality of Ghana ......................................................................................................................................... 61
Magdalene Brown Anthony Akwesi Owusu
Comparison and Properties of Correlational and Agreement Methods for Determining Whether or Not to Report
Subtest Scores ....................................................................................................................................................................... 61
Oksana Babenko, PhD. and W. Todd Rogers, PhD
Analysis of Achievement Tests in Secondary Chemistry and Biology ......................................................................... 75

Allen A. Espinosa, Maria Michelle V. Junio, May C. Manla, Vivian Mary S. Palma, John Lou S. Lucenari and Amelia E.
Punzalan
Towards Developing a Proposed Model of TeachingLearning Process Based on the Best Practices in Chemistry
Laboratory Instruction ......................................................................................................................................................... 83
Paz B. Reyes, Rebecca C. Nueva Espaa and Rene R. Belecina
1
International Journal of Learning, Teaching and Educational Research

Vol. 4, No.1, pp. 1-26, April 2014
Angovian Methods for Standard Setting in

Medical Education:
Can They Ever Be Criterion Referenced?
Brian Chapman
School of Rural Health (Churchill)1
Faculty of Medicine, Nursing and Health Sciences
Monash University
Churchill, Victoria, Australia
Abstract. This paper presents a discussion of Angovian methods of

standard setting methods which are widely used with the intent of
defining criterion-referenced absolute standards for tests in medical
education. Most practitioners, although purporting to pursue absolute,
criterion-referenced standards, have unwittingly slipped into focussing
on norm-referenced concepts of borderline students and their
predicted ability to answer assessment items in a test. This slippage has
been facilitated by a shift in language from the original concept of
minimally acceptable persons to the modern concept of borderline
persons. The inability of university academics to predict accurately the
performance of borderline graduate-entry medical students is
illustrated by presentation of data obtained from three successive
cohorts of a small regional medical school during the years 2010-2012.
Other data are presented to show how student performance, both
borderline and general, can be significantly altered by switching from
didactic lectures to tutorials preceded by task-based active learning.
A protocol, based on a stricter interpretation of what is meant by a

minimally acceptable person, is suggested for moving towards a more
criterion-referenced standard for a test based on the curriculums
learning objectives. Nonetheless, the fallibility of criterion-referenced
standard-setting processes means that norm-referenced relative
standards may need to be brought into play to deal with anomalous
grade results should they arise. The ideal of defining an absolute
criterion-referenced standard for a test, using the most commonly
implemented Angovian method, is probably as least as unattainable for
graduate-entry medicine as it has been previously shown to be for
secondary school science.
Keywords: standard setting; Angoff method; medical education; norm-

referenced standard; criterion-referenced standard
1 Formerly Gippsland Medical School (2007-2013).
2014 The author and IJLTER.ORG. All rights reserved.

2
1. Introduction
Medical schools are required to define standards of quality assurance in the
assessment of medical trainees such that society can have confidence in the
professional competence of medical graduates once they are registered to
practice. To this end, the quality of a medical curriculum is defined by the
clarity and comprehensiveness of its stated learning objectives, and the efficacy
of the teaching and learning processes directed towards the attainment of those
objectives is assessed using a variety of measuring instruments. These
instruments may include written examinations comprising multiple-choice
questions (MCQs), extended matching questions (EMQs) and short-answer
questions (SAQs), viva voce examinations such as the Objective Structured
Clinical Examination (OSCE), or a variety of essays and assignments. Quality
assurance then focuses on defining a minimally acceptable standard of
competence for each assessment question or task encountered by the students as
they progress through the course. In common with most licensing and certifying
operations, it is desired to define an absolute standard of competence for
assessing the quality of a medical graduate rather than a relative standard
expressed by comparison either with other candidates in a given cohort or with
the performance of preceding cohorts. The definition of an absolute standard is
called criterion referencing while the use of a relative standard is called norm
referencing; application of criterion referencing is intended to establish
minimum standards of competence and this is widely held to be preferable to
norm referencing (Searle, 2000; Norcini, 2003; Downing, Tekian, & Yudkowsky,
2006).
A cursory glance at the literature on assessment in medical education will reveal

the widespread use of methods for standard setting attributed to William H.
Angoff (1919-1993), a researcher at the Educational Testing Service in the United
States for 43 years, whose main contributions to educational research and
practice were focussed on the measurements used in testing and scoring. The
key reference cited for this attribution is Chapter 15 Scales, Norms, and
Equivalent Scores in Educational Measurement, Second Edition, edited by Robert L.
Thorndike (Angoff, 1971). Yet, within this 93-page chapter, as many people have
noted over the years (e.g., Zieky, 1995, pp.8-9; Cizek & Bunch, 2007, p.81),
Angoffs original description is very short, comprising no more than nine
sentences of text distributed between two paragraphs and an associated
footnote.
The purpose of this paper is to present a critical discussion of the rationale,

intent and implementation of the most widely used of the several variants of
Angoffs (1971) original suggestion that have emerged. Preparation for this
discussion reveals that very little can be said today in criticism of Angovian
procedures that has not been said before. This suggests that much current
practice is based on pragmatism, allowing standard setting to proceed for any
number of reasons, including ignorance or wilful disregard of the many
objections and concerns that have been raised in the past. It is not the aim of this
discussion to review this literature or to rehearse old arguments. Rather, the
original contribution sought here is to illuminate the discussion by identifying
the problems of linguistic imprecision and conceptual vagueness that have
3
interacted and confounded the most well-intentioned quests for defining

absolute assessment standards in medical education. Samples of original
assessment data are included to illuminate the discussion further.
The detailed analysis and discussion will be usefully facilitated by

distinguishing two methods of standard-setting contained in Angoffs (1971)
original description: Angoffs Text Method (ATM), and Angoffs Footnote
Method (AFM).
2. Angoffs Text Method (ATM)

The context for Angoffs original description is the standard-setting problem of
finding a suitable pass mark and honours mark on a scale applied to a test,
such cut scores being decided on the basis of careful review and scrutiny of the
items themselves (Angoff, 1971, p.514). The aim was to establish standards that
would be independent of normative data relating to actual performance as it
exists (p.514). This raises immediately the problem of determining whether a
test is criterion referenced or norm referenced. Although Angoff doesnt express
the issue in these words, it seems that he is striving to define a criterion-
referenced standard that will stand immutable in the face of actual performance
data. To this end, Angoffs (1971, pp.514-515) two paragraphs specify ATM as
follows:
A systematic procedure for deciding on the minimum raw scores for

passing and honors might be developed as follows: keeping the
hypothetical minimally acceptable person in mind, one could go through
the test item by item and decide whether such a person could answer
correctly each item under consideration. If a score of one is given for each
item [p.515] answered correctly by the hypothetical person and a score of
zero is given for each item answered incorrectly by that person, the sum of
the item scores will equal the raw score carried by the minimally
acceptable person. A similar procedure could be followed for the
hypothetical lowest honors person.
With a number of judges independently making these judgments it would

be possible to decide by consensus on the nature of the scaled score
conversion without actually administering the test. If desired, the results of this
consensus could later be compared with the number and percentage of
examinees who actually earned passing and honors grades [emphasis
added].
Looking back over the ensuing four decades or more since the above words
were written, it may be safely concluded that the thinking of Angoff in this
specific instance, and of all those who have subsequently used Angovian
methods of standard setting, has been dominated by the view that assessment
should be objective, measurable and certain (and therefore that assessment
can be made reliable and valid) (Williams, 2008, p.402). This is implicit in the
notion that an absolute standard for a test can be set without actually
administering the test and that such a standard might be compared with
performance as it exists if desired.

4
However, Angoffs (1971) prescription is far from being a robust example of

criterion referencing in action. This is because of the different constructions that
might reasonably be placed upon the first paragraph.
2.1 Angoffs First Paragraph

The wording of this paragraph, as quoted above, is insufficiently precise to yield
a single, unambiguous reading. As Zieky (1995, p.9 footnote 4) has noted,
Angoffs (1971) use of the word could is frustrating in its lack of precision and
its uncertain distinction from alternatives such as should or would.2 On this
matter Impara and Plake (1997, p.363 note 1) also observe that should is
typically interpreted as a higher target than would. In the same footnote 4 of
Zieky (1995, p.9), we find that, when Zieky personally asked Angoff in the early
1980s which of could or would was correct, Angoff replied that he did not
think it mattered very much. This is very illuminating and it suggests that
Angoff did not think it important to develop a full appreciation of the
importance of linguistic precision in defining an unambiguous method for
establishing a criterion-referenced standard.3 In that sense, therefore, Angoff
(1971) did not set his method on a sufficiently firm foundation.
However, let us choose the least vague of the three alternatives would and see
where that leads us. The criterion-referenced standard-setting prescription of
ATM then becomes:
Keeping the hypothetical minimally acceptable person in mind, one could

go through the test item by item and decide whether such a person would
answer correctly each item under consideration.
There remains a problem with the dual focus of the procedure. Does the focus
lie on the concept of the minimally acceptable person or on the content of each item?
The crucial linguistic watershed here is Angoffs (1971) concept of the
minimally acceptable person. It is possible to construct this concept in two
ways, one lending itself more readily to criterion referencing than the other.
A. Firstly, it may be given a more criterion-referenced construction by

implicit reverse engineering of the text. In short, by definition, a
minimally acceptable person will answer correctly every item that the
assessors have identified as embracing the minimally acceptable
performance criteria for the test. So the pass mark becomes the sum of
all the marks deriving from such minimally acceptable performance
items. The procedure under this construction would be to ask of the item,
not whether a minimally acceptable person could answer it correctly,
but whether it encapsulates an element of minimally acceptable
2Angoff (1971) uses would in the slight variation of ATM represented by AFM.
3As suggested later (see Footnote 8), there is no reason why Angoff (1971) should have
given these matters any more thought or space than he did within the context of his
original article. This is the responsibility of contemporary users of Angovian methods.
5
performance. It seems that this construction has been attempted rarely,

if at all.
B. Alternatively, it may be given a less criterion-referenced construction by

allowing the difficulty of the item (as distinct from its criterion-referenced
content) to weigh in the assessors estimates as to whether or not a
minimally acceptable person would answer the item correctly. This
question cannot be answered with any certainty because the focus has
moved away from the items content to the ability of a minimally
acceptable person to answer the item successfully. Any estimate of this
ability must necessarily take into account a margin for error that such
guesswork entails. This construction inevitably tends towards norm
referencing, where the norm is a person or group of persons of
indeterminate worthiness of passing a test or progressing to the next
level.
We shall return to the more criterion-referenced option later in the discussion

but, for now, we must deal with the fact that the overwhelming majority of
practitioners have not placed such an interpretation on the prescription of ATM.
Two factors seem to have generated this situation: one deriving directly from
Angoffs footnote to his first paragraph; the other deriving from, and perhaps
concealed by, the subtle shift in language from that used by Angoff (1971) to that
used widely today. Let us deal with the footnote first.
3. Angoffs Footnote Method (AFM)

As an exemplar of criterion referencing, Angoffs prescription stumbles at the
first hurdle through the barely perceptible sleight-of-hand that occurs as we
switch from Angoffs first paragraph to its associated footnote.
Angoff (1971, p.515) offers an alternative to the procedure outlined in the first
paragraph of ATM by specifying AFM as follows:
A slight variation of this procedure is to ask each judge to state the

probability that the minimally acceptable person would answer each item
correctly. In effect, the judges would think of a number of minimally
acceptable persons, instead of only one such person, and would estimate
the proportion of minimally acceptable persons who could answer each
item correctly. The sum of these probabilities, or proportions, would then
represent the minimally acceptable score. A parallel procedure, of course,
would be followed for the lowest honors score.
This footnote has achieved a prominence far outweighing its casual inclusion in
the original article, specifying the most widely-used procedure for
implementation of a method purported to produce a criterion-referenced
standard that seeks to establish competence (Mills & Melican, 1988; Norcini,
2003; Amin, Chong, & Khoo, 2006; Lypson, Downing, Gruppen, & Yudkowsky,
2013). Nonetheless, AFM cannot be regarded as a slight variation of ATM if
ATM is implemented according to the strategy given in Section 2.1 A above.
According to a strictly criterion-referenced view of minimally acceptable for the

6
establishment of a pass mark, each item in a test must be given a binary value of
0 or 1 as a multiplier of the respective mark attached to the item (1 for all must
know items, 0 for all other items). But this view cannot accommodate a
compromise notion of probability where the value of the probability may take
non-binary values other than 0 or 1. The fact that Angoff (1971) regarded AFM
as a slight variation of ATM suggests that he may not have appreciated the
extent to which he lost sight of his assumed criterion-referenced goal almost as
soon as he tried to illustrate how it might be achieved.
While most applications of Angoffs methods have adopted the AFM approach
of assigning probabilities over a continuous range between 0 or 1, judges have
clearly found the procedure difficult (Norcini, 2003, p.466). Such difficulties
apparently led to the re-discovery of ATM by Impara and Plake (1997),
mistakenly reported by Jalili, Hejri, and Norcini (2011) as being proposed in 1997
as another variation of the Angoff method, now called the Yes/No Angoff
method. In turn, difficulties with applying the Yes/No method to test items
then led to the emergence of a Three Layered Angoff (TLA) method in which
the ratings are Yes = 1, No = 0 and Maybe = 0.5 (Yudkowsky, Downing, &
Popescu, 2008; Jalili et al., 2011). The introduction of the Maybe category
shows plainly that the focus has shifted from strict criterion referencing to some
kind of norm referencing, the norm in this case being the examiner-conjured
virtual image of a minimally acceptable examinee. Maybe is not a category
to which examiners should be in the habit of consigning significant chunks of
their curriculum or batches of their questions for criterion referencing, but it is
certainly a category that would be heavily populated by examiners attempting
to predict the performance of students who cannot be judged with confidence as
being of either pass-grade or fail-grade quality.
The intended criterion-referenced goal has been obscured by the attention given
to the probability that an ill-defined subset of candidates could or would answer
each item on a test successfully. Thus, it emerges that the AFM approach cannot
be regarded as a slight variation of the ATM approach, as presented by Angoff
(1971), unless the ATM approach is also interpreted as a norm-referenced
method, whereupon the ATM approach emerges as a particular extreme case of
the more general AFM approach. Somewhere between the extreme binary
approach of ATM and the widely-used continuum approach of AFM lies the
more recent variation, the Yes/No/Maybe approach (Yudkowsky et al., 2008;
Jalili et al., 2011). But, whichever of these three approaches has been used, it
would appear that the original goal of achieving an absolute criterion-referenced
standard has been obscured by the aforementioned subtle change in language to
which we now turn.
3.1 Softening the language of ATM and AFM

While a central feature of Angoffs methods of standard setting is the definition
of minimally acceptable persons, contemporary literature speaks, instead, of
borderline persons. The difference between these labels reflects a shift from a
sterner view of what is minimally acceptable to a more nebulous concept of
what characterises a borderline student. This shift has blurred the conceptual

7
distinction between minimally acceptable (more attuned to criterion

referencing) and borderline (more attuned to norm referencing).
As discussed earlier, it is possible to put a criterion-referenced construction on

the usage minimally acceptable person by defining such a person in terms of
what knowledge, understanding and skills are required. By contrast, it does not
seem possible to put such a construction on the concept of a borderline person.
On the contrary, the concept would appear from the outset to be norm referenced.
Thus, the modern linguistic trend of substituting borderline for minimally
acceptable has facilitated and completed the confounding of norm referencing
and criterion referencing in the standard-setting process.
4. Angoffs Second Paragraph

4.1 The Quest for Consensus
Angoffs (1971) second paragraph, reproduced above, describes how a
consensus about the standards might be achieved among several independent
judges. This raises a number of procedural possibilities and issues according
to the nature of the judging panel. Let us consider two extreme situations: A: a
panel comprising judges having equal expertise in relation to all n test items; B: a
panel comprising judges having expertise limited to different subsets of the n
test items, such expertise showing overlap between individual judges and
ranging from total overlap to zero overlap. In most medical schools running an
integrated curriculum, it would be reasonable to expect that any given panel of
judges will lie somewhere between these two extremes and, in practically all
cases, much closer to the latter extreme.
A. Consensus among totally independent experts

In this idealised extreme case, each judge would be equally expert both on the
entire content of the test and on the matter of assigning to each item an estimate
for the proportion of borderline persons who would answer the item correctly
(hereinafter called the BL value or, simply, the BL).
However, real-world variability between judges might be expected to yield some

slight variation in the BL estimates, resulting in some items being assigned
different BL values by different judges, i.e., the creation of a subset of
inconsistently estimated items m. This would indicate a need to achieve a
consensus by sacrificing post hoc the absolute independence of the judges by
averaging their estimates for each of the m items for which their estimates
differed.
B. Consensus among inter-dependent partial experts

This situation is fraught with difficulty because the partial expertise is rarely
manifest as complete expertise on a subset of n and zero expertise on the
remainder. Rather, there will be a gradient of expertise for each judge, ranging
from complete to zero among the n items. This difficulty could be overcome if
each judge self-disqualified on each item for which less than complete expertise
was possessed. However, in practice, consensus in these cases can only be
reached among non-independent judges by having the less expert judges yield

8
to the opinions expressed by the more expert judges on each item. While the
intrusion of human nature will be an inescapable complication of this process, it
is perhaps to be preferred over an alternative procedure in which each item
would be assigned a BL according to an average of independently derived
expert and inexpert inputs.
4.2 What is purported to happen

Nowadays, much emphasis is placed on training members of standard-setting
panels in the art of grasping the concept of a borderline student. For example,
a typical description of the Angoff standard-setting process by Norcini (2003,
p.465), under the heading Angoffs method, reads as follows:
Judges are asked to first define the characteristics of a borderline group of

examinees (a group with a 50% chance of passing). They then consider the
difficulty and importance of the first item on the test. Each judge estimates
what percentage of the hypothetical borderline examinees will respond
correctly to the item. This judgement is often informed by data on the
performance of the examinees. The judges discuss their estimates and are
free to change them, and then proceed in the same manner through the
remainder of the items on the test. The judges estimates are averaged for
each item and the cutpoint is set at the sum of these averages.
Despite the fact that this description is not without its problems, both logical and
logistical, it is a fair description of what participants in contemporary standard-
setting sessions in medical education imagine that they are doing. The problems
are sufficiently important to warrant detailed dissection of this description,
sentence by sentence.
Judges are asked to first define the characteristics of a borderline group of

examinees (a group with a 50% chance of passing).
This process of definition is quite illusory. The word borderline embodies the
uncertainty attaching to persons who cannot be characterised as being clearly
acceptable (deserving to pass) or clearly unacceptable (deserving to fail). Given
this uncertainty, on what basis can such a group be said to have a 50% chance of
passing? Only if the uncertainty of the examiners is symmetrical, i.e., if every
conceivable type of person who is neither clearly acceptable nor clearly
unacceptable, is equally likely to have been wrongly excluded from either of
these two categories.
Nonetheless, regardless of these issues, the focus is clearly on a subset of the

cohort of examinees (whether actual, anticipated or imaginary), i.e., a norm-
referenced focus, not a criterion-referenced focus. Moreover, given that judges
are asked to conceive of such students as a hypothetical abstraction, based on
their prior teaching and assessment experience, they can do nothing other than
conjure up a norm-referenced concept (the borderline person) and apply to it
their own subjective guesswork. Given the difficulty in establishing a criterion-
referenced absolute standard derived from such norm-referenced guesswork, it
is not surprising that a desire has arisen among examiners to seek the protection

9
and reassurance of group consensus among as large a collection of judges as can

be mustered to define a standard for any given test.
They then consider the difficulty and importance of the first item on the
test.
This sentence directly and unnecessarily confounds the concepts of difficulty

and importance. In fact, it is possible that these two concepts, in relation to
individual test items, might be either essentially identical or totally unrelated.
For example, the difficulty of an item might be judged to be a direct reflection of
its importance, i.e., it is important for the item to be difficult as a property of its
defining a minimally acceptable criterion of coping with difficulty. On the other
hand, an item might be extremely important yet trivially easy for a correctly
trained candidate, so that importance and difficulty are totally unrelated.
Moreover, items that are unlikely to be answered successfully by any but the
most exceptional candidates may possibly be very difficult yet essentially
unimportant.
Each judge estimates what percentage of the hypothetical borderline

examinees will respond correctly to the item.
This is clearly a norm-referenced judgment, with no necessary link to any sense of

importance of the item (criterion referencing). As already noted, the formation of
such estimates is reported to be difficult (Lorge & Kruglov, 1953; Bejar, 1983;
Impara & Plake, 1998; Norcini, 2003) although claims have been made that
judges benefit from an iterative process whereby they can learn from the
estimates of their fellow judges formed during previous standardisation sessions
dealing with the same assessment test (Cizek & Bunch, 2007, p.84).
However, except where special funding for educational research projects is

available, iterative standard setting is beyond the resources and the practical
exigencies of most medical schools. Moreover, when the test is a major
examination of an integrated curriculum (reflecting current trends in medical
education), it is not always possible to convene judging panels in which all
specialities within the curriculum are adequately represented, let alone having
multiple experts on each area capable of working out a consensus on the
respective estimates.
At this point it is worth commending to the readers attention the salutary study
of AFM by Impara and Plake (1998) in which they tested the ability of 26
classroom teachers to estimate item performance for two groups of their
students on a locally developed district-wide science test. They found that
teachers estimates of the average proportion correct for borderline students
were for the most part quite inaccurate, with only 23% of 1300 estimates
focussed on these students being accurate (defined as within .10 of actual item
performance. Their concluding paragraph is worth quoting in full (Impara &
Plake, 1998, p.80):

10
The most salient conclusion we can draw from this study is that the use of a
judgmental standard setting procedure that requires judges to estimate
proportion-correct values, such as that proposed by Angoff (1971), may be
questionable. The teachers in this study performed the estimation task in
such a way that if their performance estimates were used to set a standard,
the validity of the standard used to identify borderline students would be in
question. If teachers who have been with their students for most of the school year
are unable to estimate student performance accurately using a test that is familiar
to them, how can we expect other judges who may be less familiar with examinees
to estimate item performance on a test those judges may never have seen before?
(emphasis added)
Returning to our dissection of Norcinis (2003, p.465) description, we find:
This judgement is often informed by data on the performance of the

examinees.
In our experience of standard setting, the practice of setting borderlines by

reference to item statistics pertaining to past examinee cohorts ranges between
two extremes:
A. During a standard-setting session, this practice is usually frowned upon as

being norm referenced, the purpose of the standard setting being to
establish a criterion-referenced pass mark. It is difficult at such sessions to
establish acceptance of the fact that the participants are, in fact, being
asked to predict a number that should be highly correlated with, and
reasonably close to, the actual statistically derived proportion of LOW4
students (as determined by post-test item analysis) that will be
discovered to answer the item correctly. Attempts to establish the truth
of this identity before the test is administered will often be contradicted
by remarks such as, No, thats norm referencing. We are criterion
referencing. Such remarks are untrue, but they frequently win the day,
presumably because the abovementioned confounding of difficulty and
importance is fairly pervasive.
B. During results review, this practice might be encouraged if, as could be the
case, reversion to norm referencing for a difficult item would lower the
pass mark. For the record, this procedure has not been used at
Gippsland Medical School.
While these extremes are mutually incompatible, they illustrate ways in which
the standard-setting process can become compromised in practice.
4 The LOW subset of a cohort of examinees is the bottom 27% (approximately) as

measured over the whole examination, including all multiple-choice questions and
extended matching questions, but excluding short-answer questions (SAQs). The LOW
success rate for a given question is the proportion of the LOW subset who answer the
question correctly.
11
The judges discuss their estimates and are free to change them, and then
proceed in the same manner through the remainder of the items on the test.
This part of the process has already been considered above under section 4.1 B.
The judges estimates are averaged for each item and the cutpoint is set at
the sum of these averages.
As already noted, the cutpoint may be subject to alteration when the results of
the examination are reviewed.
4.2.1 What actually happens

The description here is suggested to be typical of what happens routinely in
medical schools that apply the Angoff Footnote Method (AFM) of standard
setting. It may not be typical of what happens in standard-setting sessions that
form part of specially resourced educational research projects.
A standard-setting session is usually held several days after a draft of the

examination paper has been pre-circulated among all academics involved in
teaching and/or examining the unit. For most items, each BL has already been
supplied by the respective items author. In many cases, prior statistics deriving
from item analysis are also attached to individual items that have been used in
previous examinations.
Academics are encouraged to review the examination paper thoroughly,

focussing particularly on their own areas of expertise, advising of any errors or
recommendations for improvement, reviewing the BLs supplied and, where not
supplied, suggesting BL scores. Academics who are unable to attend the
standard-setting session are encouraged to submit their comments in writing so
that they may be considered at the meeting.
At the meeting, attention is focussed only on those items that have been flagged
for attention in the period prior to the meeting. It is noteworthy that concord
between the BLs and their respective LOW success rates (where item statistics
are available from prior examinations) is frequently absent. Where the disparity
is severe, there may be a consensus at the meeting to alter the BL, but we have
rarely seen any BL set lower than 0.3 prior to the examination, despite many
LOW success rates being 0.2 or less (see Figure 1, lower right panel).
This disparity between BL predictions and actual LOW success rates finds
resonance in the following quote from Norcini (2003, pp.465-466):
Angoffs method is relatively easy to use, there is a sizeable body of

research to support it, and it is frequently applied in licensing and certifying
settings. It also has the virtue of focusing attention on each of the questions
and thus can be very helpful from a test development perspective. This
method produces absolute standards, so it is best suited to tests that seek to
establish competence. However, judges sometimes feel as though there is
no firm basis for their estimates and application of the method can be
tiresome for longer tests.
12
While the standard-setting process (regardless of the method used) is certainly

most helpful from a test development perspective, it cannot be supported that
AFM produces absolute standards. Indeed, there is no firm basis for the
estimates, as will now be discussed.
5. Angoffs Footnote Method in Action in a Small Medical School

Let us now turn to some assessment data obtained in three successive years from
examinations set for first-year graduate-entry students at Monash Universitys
Gippsland Medical School in 2010-2012. Prior to each examination, a standard-
setting session was held in which a panel of interdependent partial experts was
asked to reach a consensus, item by item, on estimating the proportion of
borderline candidates who would answer each item correctly for multiple-choice
questions (MCQs) and extended-matching questions (EMQs).5
5.1 Relation of BL estimates to different subsets of examinees

Figure 1 shows data plots relating the BL estimate associated with each MCQ or
EMQ and the respective performance (P) success rate for the entire student
cohort (PALL) and for the three subsets of candidates identified by statistical
analysis of the results (PHIGH, PMID and PLOW; see Footnote 4). All the data
represented in Figure 1 were obtained from the 2010 cohort of 76 examinees
answering 83 questions in the mid-semester 1 examination. Thus, although each
data plot contains 83 data points, far fewer than this number are visible owing to
superposition of many data points.
Figure 1 also shows the results of analysing the data using linear regression of P
values on respective BL estimates. While there is no reason to expect
uncomplicated linear correlations, it is reasonable to expect that the values of
both the slope and R2 of the linear regression would be greatest for the LOW
subset and least for the HIGH subset. It is also to be expected, as found, that the
ordinate intercept of the regression should be close to zero for the LOW subset
and significantly greater than zero for the HIGH subset. Consistent with these
expectations are the intermediate values of slope, R2 and ordinate intercept
observed for the MID subset and for the whole cohort (ALL).
Despite the fulfilment of these expectations, it is clear that the BL estimates

arrived at through a consensual implementation of AFM are almost randomly
and widely inaccurate; the performance success rates span much more than half
the available range between 0 and 1 for almost all the BL estimate levels
provided. The overall impression gleaned from these data is that academics BL
estimates are extremely poor predictors of examinee performance, and this
impression is at its strongest in relation to the performance of the LOW subset.
5They were also asked to estimate the average mark likely to be obtained by a borderline
candidate on each item for Short Answer Questions (SAQs), but this category of
estimates is not included in the present analysis.
13
This impression of gross inaccuracy is maintained as we refine the focus to the

Borderline6 subset as will be shown in the next subsection.
Figure 1: Data plots of performance rates, P, vs BL estimates for students answering

83 questions in a mid-semester 1 examination in 2010. The plots are for the entire
cohort of 76 students (ALL, upper left), for the top 20 students (HIGH, lower left), for
the 35 midrange students (MID, upper right) and the bottom 21 students (LOW, lower
right).The linear trend line, linear regression equation and R2 value are included on
each respective plot.
5.2 Accuracy of BL estimates for the Borderline subset of examinees

This 2010 examination was given again in 2011 and 2012 with essentially the
same group of academics producing very similar styles of questions. The data
plots relating the BL estimate associated with each MCQ or EMQ and the
respective performance (P) success rate for the Borderline subsets are shown in
Figure 2 for eleven examinees in 2010 (upper left panel), three examinees in 2011
(upper right panel) and for nine examinees in 2012 (lower panel). For these
particular data plots, it is highly appropriate to perform linear regression
analysis on the relations between BL estimate and performance because the
estimate is explicitly purported to predict the actual performance of the
identified Borderline students.
The persistently poor correlations already observed in the data plots of Figure 1
are also observed in the data shown in Figure 2, indicating that the academics
predictive ability did not improve when only the Borderline data were
included, and nor did it improve with experience. In all three years it was often
6Borderline students are defined as examinees whose results fall within a range from
one SEM above to two SEMs below the overall BL determined for the test, where the
SEM is the standard error of measurement of examinees scores.
14
found that Borderline performance values spanning the entire possible range
between 0 and 1 could be returned for groups of items assigned any given BL
value by the examiners. Moreover, the highly respectable slope and ordinate
intercept seen from the 2010 Borderline subset were not seen in the two
following years.
Figure 2: Data plots of performance rates, P vs BL estimates for Borderline students

answering questions in a mid-semester 1 examination in 2010 (83 questions, 11
examinees, upper left), in 2011 (82 questions, 3 examinees, upper right) and in 2012 (95
questions, 9 examinees, lower). The linear trend line, linear regression equation and
R2 value are included on each respective plot.
Figure 3 shows analysis of data gleaned from the same cohort as presented for
the mid-semester 1 examination in 2012 (Figure 2 lower), showing analyses of
data from the end-semester 1 (left) and mid-semester 2 (right) examinations
from the same year. Interestingly, this cohort provided no data for such analysis
in the 2012 end-semester 2 examination because all the students passed. That is,
for the 2012 cohort, the numbers of Borderline students identified by using
Angoffs Footnote Method for the four successive mid-semester and end-
semester examinations were 9, 9, 15 and 0, respectively. Any discussion or
further exploration of this interesting finding would stray too far from the focus
of the present critique and so will not be pursued here.
It might be objected that the small numbers of identified Borderline examinees

in the examination data reported here (ranging from 3 to 15 per examination,
with one instance of zero) militate against drawing firm conclusions. However,
the conclusions pertain to the accuracy of predictions of BL values for large
numbers of questions, ranging from 75 to 95 per examination. Almost by
definition, one hopes, the numbers of Borderline candidates identified among
cohorts of graduate-entry medical students should be small. The critique
concerns the accuracy of the large number of predictions about individual

15
questions that are made in relation to the actual performance on those questions
by the small numbers of identified Borderline candidates.
Figure 3: Data plots of performance rates, P vs BL estimates for Borderline students

answering questions in an end-semester 1 examination (87 questions, 9 examinees,
left) and a mid-semester 2 examination (75 questions, 15 examinees, right) in 2012.
The linear trend line, linear regression equation and R 2 value are included on each
respective plot.
5.3 A want of feasibility and credibility for Angoffs Footnote Method

While this direct comparison of BL values with Borderline performance values
has been held to be a way of providing useful checks on the passing score that
is being chosen (e.g., Kane, 1994, p.447), it seems likely that the poor predictive
powers of academics makes the use of AFM an exercise in futility. The only
comfort that can be taken from the data shown in Figures 2 and 3 is that the
large errors of BL estimation are so randomly distributed that there may be no
systematic error in the passing scores actually obtained by using AFM in this
way. Thus the attempt to make accurate predictions, though futile and
unsuccessful, may actually do little harm.
This record of poor prediction of BL values by academics involved in delivering

an integrated medical curriculum should come as no surprise to anyone who has
absorbed the results and conclusions of the study by Impara and Plake (1998)
cited earlier. The least that can be suggested is that these BL predictions cannot
be held up as exemplars of the goal of defining criterion-referenced absolute
standards that are reliable and valid. On the contrary, the BL estimates of
Borderline examinees performances shown in Figures 2 and 3 are visibly
unreliable and invalid. In fact, the data shown in Figures 1 to 3 would seem, at
worst, to vindicate Glasss (1978, p.259) forlorn conclusion that setting
performance standards on tests and exercises by known methods is a waste of
time or worse and, at best, to resonate with Impara and Plakes (1998) finding
that even schoolteachers who are deeply familiar with a long-established test
and their successive student cohorts are unable to predict the performance of
borderline students in such a way as to produce a reliable criterion-based
standard.
As already noted, this study is not the first to question the feasibility and
credibility of the Angoff process; nonetheless there seems to have been a well-
established historical tendency for educators to proceed blithely ahead with their

16
theories and methods without due acknowledgement of precedence or

criticism.7
The data plotted in Figures 1 to 3 cast grave doubt on Norcinis (2003, p.466)
claim that the Angoff (Footnote) Method produces absolute standards. The
standards set for individual items do not seem to correlate in any reassuring
way with the degree of difficulty of the items, as encountered by either the LOW
subset of examinees or the TOTAL cohort. As for correlation with the importance
of the items (cf. Norcini, 2003, p.465 and earlier discussion in Section 4.2), that
would seem to be a matter totally beyond analysis.
The data reported here are consistent neither with the findings of Shepard
(1994), who found that judges tend to overestimate low performers and
underestimate high performers, nor with the opposite finding reported by
Impara and Plake (1998, p.75) who found that teachers systematically
underestimated the performance of borderline students. It seems that
examiners are as likely to underestimate as to overestimate performance success
rates over a wide range of BL estimate values. These disparate findings would
appear to provide further evidence that absolute standard setting by such
prediction is unattainable.
6. A Possible Criterion-Referenced Implementation of ATM

Let us now consider how ATM might be interpreted and implemented using the
more criterion-referenced construction of the concept of the minimally
acceptable person suggested in Section 2.1 A. In such an approach there must
be a strictly criterion-referenced focus on the content of the assessment items and
not a norm-referenced focus on the performance probabilities of examinees. Let
us consider a test comprising 100 items, each item carrying 1 mark, with no
possibility of scoring fractional marks on any of the items. That is, for each item,
a correct answer scores 1 while an incorrect answer scores zero. The method is
to go through the test item by item and decide whether a minimally
acceptable person ... could answer correctly each item under consideration.
7Zieky (1995, p.10) records a disturbing fact about the landmark publications of Angoff
(1971), Hills (1971) and Glaser and Nitko (1971), all appearing in the same 2nd Edition of
Educational Measurement. They all failed to recognise the pioneering work of Nedelsky
(1954) published in Educational and Psychological Measurement. Zieky (1995, p.6) notes
that concepts described by Nedelsky are found in more recent descriptions of methods
of setting standards, and he finds that Nedelskys concept of a borderline student
corresponds to Angoffs (1971) minimally acceptable person, to Ebels (1972)
minimally qualified (barely passing) applicant, and to the members of the borderline
group described by Zieky and Livingston (1977). Despite the importance of
Nedelskys (1954) work, Zieky (1995, p.10) notes with interest that neither Angoff nor
Hills nor Glaser and Nitko referenced Nedelskys article Absolute Grading Standards
for Objective Tests. The article was clearly relevant to the problems addressed in their
chapters and had been printed in a major journal about 17 years earlier.
17
6.1 Unambiguous criterion referencing: focus on the test

By this more criterion-referenced definition, a minimally acceptable person
must know, understand or accomplish certain well-defined facts, concepts or
procedures, respectively. In other words, a minimally acceptable person must
demonstrate mastery of certain well-defined criteria. Once the criteria have been
defined according to the objectives of the curriculum, the determination of a
criterion-referenced pass mark (or honours mark) becomes straightforward and
unambiguous.
By thus substituting must for could in ATM, the implication is that, of all the n
items in a test, a given item, i, encapsulates required material if no person failing
to show mastery of this material (i.e., failing to score 1 for item i) should be
allowed to pass the test. The sum of marks attaching to all such required items, p,
is therefore the pass mark defining the mastery requirement of the minimally
acceptable person.
To follow Angoffs suggestion that a similar procedure could be followed for the
hypothetical lowest honours person, all that is required is that, in addition to the
p must know items already identified as required material for the minimally
acceptable person, a further h should know items be identified as required
material for the lowest honours person. It then follows that, for a test containing
a total of n items, there will be a number of items, x = n p h, that will be
answered correctly only by exceptional candidates (nice to know). This
proposed distribution of the n test items is summarised in Table 1.
These considerations of minimally acceptable or exceptional performance can be

applied equally to test items whether they encapsulate required knowledge,
required understanding, required skills, or some combination of these attributes.
It is important to note, therefore, that this method of standard setting is focussed
entirely upon the test items insofar as they are identified as encapsulating
required material at whatever level; the focus is not upon the examinee.
Table 1
Item type Description
Must know/understand/accomplish all items for which mastery is
p
required by a minimally acceptable person.
Should know/understand/accomplish all further items for which
h
mastery is required by the lowest honours person.
Nice to know/understand/accomplish these items are unlikely to
x
be answered correctly by any but exceptional persons.
n =p+h+x Total number of items (marks)
To construct such a test in which the pass mark is 50% and the honours mark is
85%, it is sufficient to ensure that p items carry 50% of the marks and h items
carry 35% of the marks. In our example test, comprising 100 equally weighted
items (1 mark per item), such a standard would be set by setting p = 50 and h =
35. But note that this implies that 50 p-category items, 35 h-category items and
18
15 x-category items have been pre-identified independently and brought together to

produce such a combination of 100 items for the test so that such a standard
obtains for the test. Other combinations of test items could be put together prior
to standardisation, following which the standard setting would determine
different values of p, h and x. These could then be assigned different marking
weights so as to produce, if so desired, cutpoints at the 50% and 85% boundaries.
The above model is simply offered as a suggestion as to how a more consciously

criterion-referenced approach to standard setting might be developed. It is not
seen as a pure model; such a thing would seem to be unattainable. For example,
it is possible for a candidate to score p marks while answering some of the p-
category questions incorrectly, the balance of the marks coming from correct
answers to questions in other categories. When one allows for the intrusion of
guesswork into candidates answers, including the unknowable proportions of
informed and uninformed guesswork, the criterion-referenced goal of an
absolute standard becomes even more illusory. Rather, this method of standard
setting is offered as an approach to the development of tests that are explicitly
tied to curriculum objectives and that allow for the capture of information about
ranking of candidates in relation to the attainment of those objectives.
6.2 Teach to the objectives and test the objectives

Let us now turn to a simple observation, drawn from experience, that highlights
the difficulty in believing that AFM can produce an absolute standard of any
kind.
At Gippsland Medical School in the years 2008, 2009 and 2010, the
electrophysiological material on propagation of the nerve action potential was
delivered as a didactic lecture to first-year graduate-entry medical students.
Among other things, the lecture dealt with the passive electrical properties
(resistances, capacitances) of long cylindrical nerve axons and the effect of
myelination on those properties. This topic is clinically important because of the
disease state of multiple sclerosis in which nerve axons lose their myelin sheaths,
leading to motor disability and death.
The lecture was always supported by comprehensive, detailed lecture notes

made available online in two formats: as a linear text document and as an
interactive 3-layered hypertext application. However, it was consistently found
that students had difficulty in assimilating and applying the concepts
underlying the effects of myelination or demyelination on electrical signalling in
nerves. A typical multiple-choice question was set in 2010 as follows (correct
answer in bold type):
What is the effect of myelination on a nerve fibre?

A. It increases the membrane resistance while reducing the membrane
capacitance
B. It reduces the membrane resistance while increasing the membrane
capacitance
C. It reduces both the membrane resistance and the membrane capacitance
D. It increases both the membrane resistance and the membrane capacitance

19
E. It has no effect on either the membrane resistance or the membrane

capacitance, provided the influence of the electrogenic pump is ignored
The statistical item analysis for this question in 2010 was as follows:
ITEM 51: DIF=0.513, RPB= 0.478, CRPB= 0.427 (95% CON= 0.223, 0.595)
RBIS= 0.599, CRBIS= 0.535, IRI=0.239
GROUP N INV NF OMIT A* B C D
TOTAL 76 0 0 0 0.51 0.36 0.08 0.05
HIGH 20 0 0.75 0.10 0.10 0.05
MID 35 0 0.60 0.37 0.00 0.03
LOW 21 0 0.14 0.57 0.19 0.10
TEST SCORE MEAN %: 76 65 66 65
DISCRIMINATING POWER 0.61 -0.47 -0.09 -0.05
STANDARD ERROR OF D.P. 0.16 0.15 0.11 0.08
While it was regarded as disappointing that only 51% of the class answered the
question correctly, this was consistent with observations in the preceding two
years where students found this topic quite difficult. Note, however, that the
question showed a high discriminating power of 0.61, with 75% of the HIGH
group, 60% of the MID group and only 14% of the LOW group answering
correctly. The underlined part of the analysis shows the performance of the
LOW group of students (the 21 lowest performers on the overall examination
out of a total cohort of 76 students).
In 2011 it was decided to replace the respective didactic lecture with a

compulsory Tutorial for which students had to prepare answers to six set tasks.
The six tasks were assigned for presentation among the cohorts six Problem-
Based Learning (PBL) groups for cooperative preparation of a presentation to be
posted online a few days before the tutorial. All students were required to
prepare for the tutorial by studying all six of the online presentations. At the
tutorial, students were selected at random to present their findings to the
tutorial group (Presenters) or discuss the findings of other students
(Discussants). In short, active learning was enforced in 2011, unlike in previous
years where a didactic lecture was used. The same linear text document and
interactive 3-layered hypertext application were used to support the tutorial as
for the previous years lecture. The relevant tutorial task was given as follows:
The Effect of Myelination on Passive Electrical Properties of Nerve Axons

a. Explain how myelin is formed in the central and peripheral
nervous systems.
b. What are the principal features of a node of Ranvier?
c. How does myelination influence:
membrane resistance?
membrane capacitance?
d. Therefore, how does myelination affect the conduction velocity
of a nerve fibre?
When the same MCQ was set in the 2011 examination it was given a BL score of
0.2 using AFM, based on the 2010 item analysis (i.e., LOW = 0.14). In the event,
the item analysis in 2011 was as follows:
20
ITEM 11: DIF=0.841, RPB= 0.222, CRPB= 0.176 (95% CON= -0.034, 0.372)
RBIS= 0.334, CRBIS= 0.266, IRI=0.081
GROUP N INV NF OMIT A* B C D
TOTAL 88 0 0 0 0.84 0.09 0.02 0.05
HIGH 25 0 1.00 0.00 0.00 0.00
MID 40 0 0.77 0.10 0.05 0.08
LOW 23 0 0.78 0.17 0.00 0.04
TEST SCORE MEAN %: 70 62 66 67
DISCRIMINATING POWER 0.22 -0.17 0.00 -0.04
STANDARD ERROR OF D.P. 0.09 0.08 0.00 0.04
Thus, for this particular question, there was a very large improvement in
performance in 2011 relative to 2010 across the entire cohort, with 84% of the
class answering the question correctly. Moreover, the questions discriminating
power became much lower (0.22), with 100% of the HIGH group, 77% of the
MID group and 78% of the LOW group answering correctly. The underlined
part of the analysis shows the performance of the LOW group of students (the 23
lowest performers on the overall examination out of a total cohort of 88
students).
When the same question was run in the corresponding 2012 examination after
delivering the material using the same 2011 active learning model, its
performance statistics were as follows:
ITEM 12: DIF=0.721, RPB= 0.216, CRPB= 0.161 (95% CON= -0.053, 0.360)
RBIS= 0.289, CRBIS= 0.214, IRI=0.097
GROUP N INV NF OMIT A* B C D E
TOTAL 86 0 0 0 0.72 0.14 0.03 0.09 0.01
HIGH 24 0 0.83 0.13 0.00 0.04 0.00
MID 38 0 0.74 0.13 0.00 0.13 0.00
LOW 24 0 0.58 0.17 0.13 0.08 0.04
TEST SCORE MEAN %: 69 66 58 68 60
DISCRIMINATING POWER 0.25 -0.04 -0.13 -0.04 -0.04
STANDARD ERROR OF D.P. 0.13 0.10 0.07 0.07 0.04
This represents a slight drop in performance in 2012 relative to 2011, but still
much improved relative to 2010 across the entire cohort, with 72% of the class
answering the question correctly. The questions discriminating power
increased slightly (0.25), with 83% of the HIGH group, 74% of the MID group
and 58% of the LOW group answering correctly. The underlined part of the
analysis shows the performance of the LOW group of students (the 24 lowest
performers on the overall examination out of a total cohort of 86 students).
These observations on the performance statistics of a single question show that

the BL estimations cannot possibly produce the claimed absolute standard of
criterion referencing, and this remains true however the observations are
interpreted. They could have been due in part to having superior cohorts of
students in 2011 and 2012 relative to previous years, and it must be
acknowledged that the level of competition and the cut-off scores associated

21
with gaining admission as graduate-entry medical students are increasing year

by year. However, it is strongly suggested here that most of the differences are
due to the substitution of active learning for what, in the past, had been largely
passive learning; this is hardly a surprising result. Whatever the relative
contributions of these two causes to these observations might be, the fact
remains that the claim (Norcini, 2003, pp.465-466) of setting absolute standards
in BL predictions using the Angoff Footnote Method is entirely without
foundation. The performance of borderline students is more dependent on the
methods of teaching and learning applied than it is on the intrinsic difficulty of
the content.
This result certainly accords with the conclusion of Glass (1978, p.239), who
wrote:
The vagaries of teaching and measurement are so poorly understood that

the a priori statement of performance standards is foolhardy.
However, the suggested remedy for these problems is not to seek some
unattainable absolute standard, but to apply criterion-referenced (i.e.,
curriculum-determined) standards of relative importance to test items, ensuring
that all items test the objectives, and then teach to the objectives.
6.3 Constructing a criterion-referenced standard for a test

The following checklist of requirements is suggested for producing a
curriculum-determined standard according to the criterion-referenced
interpretation of ATM in which a test comprises a mixture of items in the p, h,
and x categories described above in Section 6.1 and Table 1:
Avoid undue dependence on centralised question banks; there can be no

certainty that the questions have been composed in relation to the
currently operative learning objectives, or with a view to accommodating
the problems of standard setting addressed in this paper. Some such
questions may prove useful, but only after they have been subjected to
careful scrutiny to decide how they might be assigned among the p, h and
x categories.
Before presenting the formal teaching/learning opportunity (lecture,

tutorial, practical class and any supporting handouts, online
documentation, etc.), identify the respective learning objectives and
construct at least one item in each category to address each learning
objective as follows:
o Category p Must know or Must understand or Must accomplish A

person who fails to answer such questions correctly is not yet
ready to proceed to the next year level. These questions cover
essential information, understanding and skills. Such material is
not necessarily easy, although in many cases it may be.

22
o Category h Should know or Should understand or Should

accomplish A person who fails to answer such questions
correctly is unworthy of attaining a Credit, but such failure is not
a significant impediment to progression to the next year level.
These questions may be more challenging than those in the
preceding category, but should not venture beyond material that
has been presented in teaching/learning sessions or documented
in online course materials.
o Category x Nice to know or Nice to understand or Nice to

accomplish A person who answers one quarter of such questions
correctly is worthy of attaining a Distinction; a person who
answers half of such questions correctly is worthy of attaining a
High Distinction. These questions should be relevant to the
material covered in the respective session but not necessarily
covered in detail or even directly mentioned at all. They could
explore material that a good student might be expected to pick up
through further self-directed study.
When presenting the formal teaching/learning opportunity and

preparing any supporting online documentation, it is important to teach
to the objectives with respect to the p-category and h-category items. This
underlines the importance of identifying the learning objectives and
preparing the targeted test items before preparing the associated teaching
and learning resources.
As with conventional application of the various Angovian methods,

examiners should have the opportunity to submit their questions to
colleagues for feedback on their individual standard-setting judgments
(in this case, the distribution into p, h and x categories). In particular,
there may be need for review of the assignment of questions among the
three categories both prior to, and following, the administration of the
test. This would provide essential information regarding the accuracy
and feasibility of such attempts to set relative standards unambiguously.
When the test results are known, review the allocation of students among
the various grades and, if the results of applying the implicit ATM-
derived criterion-referenced standards are unacceptable, then let the
allocation of grades be influenced by norm-referenced considerations. If
too much norm-referenced adjustment has to be made to the ATM-
derived standard, then use that information to guide an inquiry into the
construction and allocation of questions among the three categories, the
teaching and learning resources provided, and the communication of
information to students.
7 Borderline Students in Graduate-Entry Medical Schools

Competition to gain admission to graduate-entry medical courses is very strong
in Australia. Those who succeed do so by obtaining increasingly high marks in

23
the Graduate Australian Medical School Admissions Test (GAMSAT)

examination. Given that such students have already demonstrated academic
success at tertiary level, that they remain sufficiently motivated to study
medicine and that, at Gippsland Medical School and many other schools, they
are further assessed at interview, there is good reason to suppose that every
student gaining admission to graduate-entry medicine should be able to proceed
to graduation in the minimum time. That is, we should not expect to find more
than a handful of borderline students in each cohort and we should not be
surprised occasionally to find none at all. On the contrary, such expectations
flow naturally from the confidence we place in the selection process for
graduate-entry medicine. The reasonable default expectation is that all
graduate-entry medical students should pass.
It follows that the existence of borderline students in graduate-entry medical

cohorts simply demonstrates that the selection process is imperfect and unable
to guarantee an absolute standard. Norm referencing will find such students
out, as it always has. It would seem unrealistic and unproductive to search for
an absolute objective standard such as would relieve examiners of the need to
take responsibility for exercising subjective judgments, when required, to deal
with borderline students. As Kane (1994, p.427) observed:
We create the standard; there is no gold standard for us to find, and the choices
we make about where to set the standard are matters of judgment.
8 Conclusion
As this discussion has been more critical than supportive of existing applications
of Angoffs methods and their derivatives, it is important to clarify that this was
never intended to be a direct criticism of Angoff (1971)8. Rather, it has been a
discussion of the possibility that succeeding generations of educationists may
have been too uncritical in their application of a method that was originally
offered quite casually as a brief incidental insertion within a very large chapter
devoted to other assessment issues.
It is suggested that:
Generations of educators who have sought to implement Angovian

methods for defining criterion-referenced standards have, whether
consciously or not, been guided by a belief that an objective absolute
standard is achievable.
This belief has been undermined from the outset by:
8Angoff is even reported to have claimed in the early 1980s that the true originator of the
method attributed to him was the American mathematician Ledyard Tucker, as recorded
by Jaeger (1989, p.493) and Zieky (1995, p. 9 footnote 3).

24
o blurring the focus on criterion referencing with an insufficiently

precise definition of a minimally acceptable person, and
o replacement of the already ill-defined concept of a minimally

acceptable person with the norm-referenced concept of a
borderline person.
Rather than Angoffs Footnote Method (AFM) being a slight variation

of Angoffs Text Method (ATM), the dominant interpretation and
implementation of Angovian methods of the past forty years reveal ATM
to be an extreme, binary example of the more generally used probability
continuum of AFM, all such applications being norm referenced by
focussing on the examinee rather than on the curriculum.
Unless standard setting takes place in the context of a generously funded

educational research project, the search for consensus among panels of
independent experts is not feasible in the routine management of
assessment of an integrated curriculum in a graduate-entry medical
school.
Academics predictions of borderline student performance are

manifestly inaccurate in a small rural medical school and are unlikely
anywhere to be more accurate than the documented inaccuracy reported
among secondary school science educators (Impara and Plake, 1998).
However, the prediction errors, though wide-ranging, are possibly
sufficiently random to generate no serious systematic error in the
performance estimates for borderline students averaged over a test
comprising many assessment items. Thus, the inaccuracy of the
predictions may not do significant harm, even though the expenditure of
resources in generating them may not be justified.
A more rigorously prescriptive interpretation of ATM raises the

possibility of applying criterion-referenced (i.e., curriculum-determined)
standards directly to test items, with items distributed into must know,
should know and nice to know categories.
While standard setting should be done as objectively as possible, the

intrusion of imperfection in student selection procedures and assessment
procedures will always require examiners to take responsibility for
exercising subjective judgments.
Acknowledgements
Thanks are due to Professor William Hart and Associate Professor Elmer
Villanueva for their helpful comments on earlier drafts of this paper.

25
References
Amin, Z., Chong, Y.S., & Khoo H.E. (2006). Practical Guide to Medical Student Assessment.
London, England: World Scientific Publishing Co.
Angoff, W. H. (1971). Scales, Norms, and Equivalent Scores. In R. L. Thorndike (Ed.),
Educational Measurement, 2nd edn, (pp. 508-600). Washington, DC: American
Council on Education.
Bejar, I. I. (1983). Subject matter experts assessment of item statistics. Applied
Psychological Measurement, 7(3), 303-310.
Cizek, G .J., & Bunch, M.B. (2007). Standard Setting: A Guide to Establishing and Evaluating
Performance Standards for Tests. Thousand Oaks, CA: Sage Publications.
Downing, S. M., Tekian, A., & Yudkowsky, R. (2006). Procedures for establishing
defensible absolute passing scores on performance examinations in health
professions education. Teaching and Learning in Medicine, 18(1), 50-57.
Ebel, R. L. (1972). Essentials of educational measurement, 2nd edn. Englewood Cliffs, NJ:
Prentice-Hall.
Glaser, R., & Nitko, A. J. (1971). Measurement in learning and instruction. In R. L.
Thorndike (Ed.), Educational Measurement, 2nd edn, (pp. 625-670). Washington,
DC: American Council on Education.
Glass, G. V. (1978). Standards and criteria. Journal of Educational Measurement, 15(4), 237-
261.
Hills, J. R. (1971). Use of measurement in selection and placement. In R. L. Thorndike
(Ed.), Educational Measurement, 2nd edn, (pp. 680-732). Washington, DC:
American Council on Education.
Impara, J. C., & Plake, B. S. (1997). Standard Setting: An Alternative Approach. Journal of
Educational Measurement, 34(4), 353-366.
Impara, J. C. & Plake, B. S. (1998). Teachers Ability to Estimate Item Difficulty: A Test of
the Assumptions in the Angoff Standard Setting Method. Journal of Educational
Measurement, 35(1), 69-81.
Jaeger, R. M. (1989). Certification of student competence. In R. L. Linn (Ed.), Educational
measurement, 3rdedn, (pp. 485-514). New York, NY: American Council on
Education/Macmillan.
Jalili, M., Hejri, S. M., & Norcini, J. J. (2011). Comparison of two methods of standard
setting: the performance of the three-level Angoff method. Medical Education,
45(12), 1199-1208.
Kane, M. T. (1994). Validating the performance standards associated with passing
scores. Review of Educational Research, 64(3), 425-461.
Lorge, I., & Kruglov, L.K. (1953). The improvement of the estimates of test difficulty.
Educational and Psychological Measurement, 13(1), 34-46.
Lypson, M. L., Downing, S. M., Gruppen, L. D., & Yudkowsky, R. (2013). Applying the
Bookmark method to medical education: Standard setting for an aseptic
technique station. Medical Teacher, 35, 581-585.
Mills, C. N., & Melican, G. J. (1988). Estimating and adjusting cut off scores: Features of
selected methods. Applied Measurement in Education, 1(3), 261-275.
Nedelsky, L. (1954). Absolute grading standards for objective tests. Educational and
Psychological Measurement, 14(1), 3-19.
Norcini, J. J. (2003). Setting standards on educational tests. Medical Education, 37(5), 464-
469.
Searle, J. (2000). Defining competency the role of standard setting. Medical Education,
34(5), 363-366.
Shepard, L. A. (1994, October). Implications for standard setting of the NAE evaluation
of NAEP achievement levels. Paper presented at the Joint Conference on
Standard Setting for Large Scale Assessments, National Assessment Governing
Board. Washington, DC: National Center for Educational Statistics.

26
Williams, P. (2008). Assessing context-based learning: not only rigorous but also
relevant. Assessment & Evaluation in Higher Education, 33(4), 395-408.
Yudkowsky, R., Downing, S. M., & Popescu, M. (2008). Setting standards for
performance tests: a pilot study of a three-level Angoff method. Academic
Medicine, 83(10), S13-S16.
Zieky, M. J. (1995). A historical perspective on setting standards. In Proceedings of the
Joint Conference on Standard Setting for Large Scale Assessments (pp. 1-38).
Washington, DC: National Assessment Governing Board and National Center
for Educational Statistics.
Zieky, M. J. and Livingston, S. A. (1977). Manual for setting standards on the Basic Skills
Assessment tests. Princeton, NJ: Educational Testing Service.

27

Vol. 4, No.1, pp. 27-35, April 2014
Development Model of Learning Objects Based on

the Instructional Techniques Recommendation
Antonio Silva Sprock

Universidad Central de Venezuela
Caracas, 1043, Venezuela
Julio Cesar Ponce Gallegos and Mara Dolores Villalpando Caldern

Universidad Autnoma de Aguascalientes
Aguascalientes, Ags, C.P.20131, Mxico
Abstract. This paper presents the progress of the proposal for a model of
support in the development of Learning Objects. It incorporates the
most appropriate instructional techniques to the cognitive processes
involved in the student learning objectives proposed by the teacher, and
learning styles of students in order to create the Learning Object. The
proposed model is based on Felder-Silverman learning style model
(Felder & Silverman, 1988) and the cognitive processes proposed by
Margarita de Sanchez (1991). The paper presents the proposed model,
the cognitive processes studied, learning styles, instructional techniques
included in the study and the relationship of the techniques with
cognitive processes and cognitive styles of learning. Finally, it shows the
mathematical model and prototype implementation of the mathematical
model.
Keywords: Learning Objects; Cognitive Processes; Learning Styles;

Instructional Techniques.
1. Introduction
Learning Objects (LO) are considered as the design paradigm of digital
educational resources that can be updated, reused and maintained over time
(Hernndez & Silva, 2001). It should be noted that there is no single LO
definition. One important definition is given by David Wiley (2000) who
describes the LO like elements of a new type of computer-based instruction and
based on the object orientation paradigm, so that the LO can be used in different
contexts of study. Polsani (2003) indicates that it is a self-contained unit and
learning, predisposed for reuse. The LO are interactive and educational
resources in digital format, developed with the purpose of being reused in
different educational contexts, with the same instructional need, this being its
main feature, for promoting learning.
2014 The authors and IJLTER.ORG. All rights reserved.

28
The reuse of LO is achieved by the introduction of self-descriptive information

expressed into metadata, these are a set of attributes or elements necessary to
describe the object, with the metadata, you have a first approach to the LO,
knowing its main features, such as name, location, author, language, keywords,
etc. However, because a LO is a software product for educational purposes, it is
feasible to consider pedagogical, technology and Human Computer Interaction
(HCI) aspects in its design.
1.1. Cognitive Learning Process

These processes operate in the mental processes of acquiring new information,
organization, retrieval or activation in memory. Thus they are related to
regulatory processes that govern and control the mental processes involved in
learning and thinking in general, affecting several activities of information
processing, with special emphasis on learning complex (Rivas, 2008). The
cognitive psychological processes are essential for the implementation of
complex academic tasks (Daz-Barriga & Hernndez, 2010).
The basic psychological processes mentioned by Margarita Amestoy de Sanchez

(1991), are: Observation, Comparison and Relationship, Simple Classification,
Sorting, Hierarchical Classification, Analysis, Synthesis and Evaluation. These
psychological processes are closely related to the instructional learning objective
to be achieved in the design of teaching and learning process and can associate
certain verbs used when generating the objectives. Every psychological process
defined by Margarita de Sanchez (1991, 1991a, 1993) is described below:
1. Observation: to identify, to name, to describe, to discuss, to list, to locate,

to characterize, to observe, to define, to label, to collect.
2. Comparison and Relationships: to interpret, to summarize, to associate,
to differentiate, to distinguish, to compare, to relate, to merge.
3. Simple Classification: to categorize, to sort, to group, to sort, to select, to
divide, to tabular.
4. Sort: to sequence, to serialize, to sort
5. Hierarchical Classification: to rank, to structure, to combine, to integrate.
6. Analysis: to connect, to predict, to extend, to interpret, to discuss, to
display, to report, to experiment, to discover, to solve, to calculate, to
analyze, to discriminate, to induce.
7. Synthesis: to estimate, to summarize, to apply, to demonstrate, to plan, to
generalize, to complete, to illustrate, to explain, to show, to build, to
infer, to create, to design, to invent, to develop, to modify, to formulate,
to rewrite, to replace, to integrate, to use, to form, to deduct.
8. Evaluation: to test, to measure, to recommend, to judge, to explain, to
evaluate, to criticize, to justify, to support, to persuade, to conclude, to
predict, to argue, to feed back.
1.2. Learning Styles

Learning Styles are a sort of personal variables that lay somewhere between
intelligence and personality and explain the individual different ways of
approaching, planning, and answering to the learning challenges (Kolb, 1984).

29
The Learning Styles included cognitive and affective features. Cognitive features
are related to how students structure the content, form and use concepts,
interpret information, and solve problems. The affective features are related to
the motivations and expectations that influence learning, while physiological
features are related to gender and bio rhythms, such as the sleep-wake of the
student (Woolfolk, 2006).
There are many classification models of learning styles, such as David Kolb
model (1976), model of Ned Herrmann Brain Quadrants (Herrmann, 1982, 1990)
model of NLP Bandler and Grinder (1982), model Multiple Intelligences Howard
Gardner (1983), model of the cerebral hemispheres of Bernice McCarthy (1987)
and the model of learning styles Felder and Silverman (1988), among others. In
this work we used the model of Felder and Silverman, as a model currently
working in the area of the LO (Capuano et all, 2005), (Graf, 2005), (Mustaro &
Frango, 2006), (Graf and Kinshuk, 2006, 2009), (Chang et all, 2009), (Popescu,
Badica and Moraret, 2010), (Alharbi et all, 2011).
1. The model of Felder and Silverman (1988) classifies learning styles based
on five dimensions:
2. Sensitive-Intuitive: the sensitive student prefers to learn by studying facts
that deal with aspects of daily life and the intuitive student through the
study of abstract concepts.
3. Visual-Verbal: the visual student prefers to learn using visual teaching
aids while the verbal student prefers to do it by listening or written form.
4. Inductive-Deductive: The best form for understanding the information
for the inductive student is when he sees facts and observes and then
infer the principles or generalizations, and the deductive student prefers
to deduce consequences and applications.
5. Sequential-Global: the sequential student prefers to learn by following a
sequential order and the global student prefers to follow a general
schema that allows to visualize a whole instead of its compounding parts
6. Active-Reflective: the active student prefers to learn by doing activities
and the reflective student through reasoning on things.
1.3. Instructional Techniques

Instructional or teaching techniques are procedures structured logically and
psychologically for directing student learning, but in a limited or in a phase of
the study of a topic, such as presentation, elaboration, synthesis or critique of it
(Nrici, 1992). The technique is less extensive than that of an instructional
method and strategy. It is related to the form of immediate presentation of
content. It corresponds to the mode of action, objectively, to achieve a goal and
fulfill a definite purpose of teaching. It is part of the method in the learning
implementation (Nrici, 1992). For example, a case study, projects.
2. The Problem
Students, depending on their learning style, use in a conscious form, controlled
and deliberate, procedures (sets of steps, operations, or skills) to learn and solve
problems, i.e. structure their learning strategy (Daz-Barriga & Hernndez,

30
2010). The effectiveness thereof depends largely on the instructional strategy

used (Ossandn & Castillo, 2006), in fact instructional strategies do not work in
all situations to develop with any content.
The LO are computer and educational resources at the same time, and often in
their design the Pedagogical Dimension issues are not considered. People
consider models and technical standards that ensure interoperability
characteristics, accessibility, reusability, adaptability and durability. For this
reason, we must also consider the pedagogical characteristics in the LO
(Hernndez, 2009), this means, the LO must serve to different types of users,
considering the individual characteristics of each and adapting instructional
activities according to the learning styles (Arias, Moreno & Ovalle, 2009).
The instructional activities are implemented following instructional techniques;

these techniques are part of the instructional strategies. You could say that the
strategy is realized and made effective through the methods and teaching
techniques (Nrici, 1992). Each instructional technique is assigned different
degrees of adequacy and effectiveness in the teaching and learning, according to
each learning style. Therefore, learning styles are very important in the teaching
and learning process (Paredes, 2008). Felder and Silverman (1988) for example,
argue that students with a strong preference for a learning style may have
difficulties in the process if the learning environment does not suit their learning
style.
Similarly, the Pedagogical Dimension of the LOs considers the proposed

objectives, which are closely related to the cognitive processes that must operate
in the mental processes of acquisition of new information, for their organization,
recovery or activation in memory. Like learning styles, cognitive processes are
also crucial in the selection of instructional techniques, because this has different
degrees of effectiveness for each cognitive process.
From these perspectives, the LO design is a challenge for a teacher, who must
also choose the content, use instructional techniques, based on the student
characteristics from the standpoint of the learning style of the user (Ossandn &
Castillo, 2006) , and cognitive processes related to learning objective of the
student, defined at the beginning of the design of LO. For all the above, what can
be recommended to the LO developers in terms of the most appropriate
instructional techniques to learning styles and cognitive processes involved in
the learning objective?.
3. The Model
In response to the above question, a model for LO development is proposed,
based on the assessment of instructional techniques (Figure. 1). The teacher,
through a learning platform, defines learning objectives, and then this platform
selects cognitive processes involved in the objectives set by the teacher, also the
teacher defines students learning style to whom the LO is directed and finally
from a platform selects from a population of 36 instructional techniques, the
techniques that best suit to the cognitive processes and learning styles selected.

31
The teacher can structure instructional strategies, using the techniques indicated
and then include the activities in the LO, according to the techniques. The
technology platform uses a mathematical model to select the most appropriate
techniques to learning styles and cognitive processes involved.
Learning Cognitive
Objetives Process Technology
Teacher
defines Platform
Learning Mathematical
Styles Model
Instructional
Techniques
Learning
Instructional Activities,
Teacher structure
Strategies Evaluation
Activities
Learning Objects
Figure 1: Development Model of LO, Based on the Instructional

Techniques Recommendation.
4. The Model
As noted in the previous section, the selection of instructional techniques is
performed using a mathematical model, which assigns a value to each technique
according to the sum of adequacy factors of each technique to each selected
cognitive process and learning style indicated by the teacher. The adjustment
factor for each instructional technique to each learning style and each cognitive
process is in the range of [2,10].
Equation (1) presents the mathematical model which calculates the value and
shows the first three instructional techniques most suitable to each cognitive
process (taking only the technical adjustment factor which is greater than 8), in a
descending order.
8 4
ti (ti, pj ) (ti, ek ) (1)
j 1 k 1
Where:
ti T , i 1, , 36
pj P , j 1, , 8
ek E, k 1, , 4
(t , p) Adjustment Factor of the Instructional Techniques t
respect to Cognitive Process p
(t , k ) Adjustment Factor of the Instructional Techniques t
respect to Learning Style e
5. Results
Below it is shown the screen where the teacher indicates the instructional
objectives (see Figure.2), then the technology platform shows the cognitive

32
processes (De Snchez, 1991), associated with the instructional objectives given
by the teacher (see Figure.3) and the teacher selects the learning style according
to Felder and Silverman model (Felder & Silverman, 1988).
Figure. 2. Instructional objectives indication.
Figure. 3: Cognitive Processes associated with the instructional objectives and

learning styles selection.
Process Instructional Technique Total valor of Technique

Simple Classification
workshop 79
study conducted 78
management notes 65
Hierarchical Classification
workshop 79
study conducted 65
pre questions 63
Analysis
workshop 79
study conducted 78
management notes 65
Figure. 4. Results of the evaluation of instructional techniques.
The latter figure shows to the left the cognitive processes associated with
instructional objectives defined by the teacher, in this case the processes: Simple
Classification, Hierarchical Classification and Analysis. For each process, it
shows the three instructional techniques rated to each cognitive process. The
assessment of each instructional technique, as mentioned, is associated with
adjustment factors in each dimension of learning style and the factors chosen for
adaptation to the cognitive processes involved. The results show the technical
factors in the dimensions of learning style. It is observed that the most valued
technique for the Simple classification is "workshop", whose total value is 79,
which means that it fits within the value of 40 in the dimensions of the selected
learning styles and a value of 39 to cognitive processes.

33
At the end the cognitive processes show the valuation of all instructional
techniques included in the model. Note that the technique "Workshop" is, in
general, the most valued, and properly applied in each cognitive process
involved. However, the technique "addressed Study", the second highest score
among all techniques, is not suitable for the hierarchical classification process.
Similarly, valuation techniques which put them in third place, "prior organizers"
and "Underline", respectively, are not suitable to any of the cognitive processes
involved.
6. Conclusion
Once the teacher selects the learning styles and verifies the cognitive processes
associated with the learning objectives for students, he activates the evaluation
of techniques, obtaining the best instructional techniques to be used in the
development of LO (see Figure. 4). The article presents the evaluation of
instructional techniques according to the valuation calculated by its relevance to
the learning styles according to Felder and Silverman model (Felder &
Silverman, 1988) and cognitive processes proposed by Margarita De Snchez
(1991).
The Felder and Silverman model has been widely used to determine the LO
suitability and of teaching resources in general. Similarly, cognitive processes
defined by Margarita Sanchez is adapted to cognitive theory, emphasizing the
internal forms of assimilation and processing of information.The evaluation of
instructional techniques is based on the implementation of the proposed
mathematical model, using the stored factors of each technique with respect to
its suitability for cognitive process and learning style, these factors can be
modified and better adjust by expert teachers.
The proposed model may be incorporated into a LO generator, which permits

the use of predesigned templates for each specific instructional technique and
directed to the teacher, for the design and construction of LO.
References
Alharbi, A., Paul, D., Henskens, F. and Hannaford, M. (2011). An Investigation into the
Learning Styles and Self-Regulated Learning Strategies for Computer Science
Students. Proceedings ASCILITE 2011, Hobart, Tasmania, Australia. Accessed
May 20, 2012, from:
http://www.ascilite.org.au/conferences/hobart11/downloads/papers/Alharbi
-full.pdf.
Arias, F., Moreno, J. and Ovalle, D. (2009). Modelo para la Seleccin de Objetos de
Aprendizaje Adaptados a los Estilos de los Estudiantes. Revista Avances en
Sistemas e Informtica. Vol.6 N1, junio 2009. Medelln, Colombia. ISSN: 1657-
7663. Accessed February 2, 2011, from:
http://www.revista.unal.edu.co/index.php/avances/article/viewFile/14445/1
5360.
Bandler, R. and Grinder, J. (1982). De sapos a prncipes. Editorial Cuatro Vientos.
Capuano, N., Gaeta, M., Micarelli, A. and Sangineto, E. (2005). Automatic student
personalization in preferred learning categories. In: 3rd International Conference
on Universal Access in Human-Computer Interaction. Las Vegas, Nevada, USA.
Accessed May 10, 2012, from:

34
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.70.8877&rep=rep1
&type=pdf.
Chang, YC., Kao, WY., Chu, CP. and Chiu, CH. (2009). A learning style classification
mechanism for e-learning. Computers & Education. Volume 53, Issue 2,
September 2009, Pages 273285. Accessed April 2, 2012, from:
http://www.sciencedirect.com/science/article/pii/S036013150900044X.
De Snchez, M. (1991). Procesos Bsicos del pensamiento. Mxico, Trillas.
De Snchez, M. (1991a). Procesos directivos, ejecutivos y de adquisicin de
conocimiento. Gua del instructor. Mxico, Trillas.
De Snchez, M. (1993). Planifica y decide. Mxico, Trillas.
Daz-Barriga, F. and Hernndez, G. (2010). Estrategias docentes para un aprendizaje
significativo. 3 Edicin. McGraw-HILL, Mxico.
Felder, R. and Silverman, L. (1988). Learning and Teaching Styles in Engineering
Education," Engr. Education, 78(7), 674-681. Accessed September 18, 2011, from:
http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Papers/LS-
1988.pdf.
Garca, J. (2006). Estilos de Aprendizaje. Web de Jose Luis Garca Cue. Accessed
December 12, 2010, from: http://www.jlgcue.es.
Gardner, H. (1983). Frames of Mind: The Theory of Multiple Intelligences. New York:
Basic Books, Division of Harper Collins Publishers.
Graf, S. (2005). Fostering Adaptivity in E-Learning Platforms: A Meta-Model Supporting
Adaptive Courses. CELDA 2005, pp.440-443, Portugal. Accessed May 13, 2012,
from: http://sgraf.athabascau.ca/publications/graf_ CELDA05.pdf.
Graf, S. and Kinshuk, K. (2006). Considering Learning Styles in Learning Management
Systems: Investigating the Behavior of Students in an Online Course. Semantic
Media Adaptation and Personalization, 2006. SMAP '06. First International
Workshop on, vol., no., pp.25-30. Accessed May 12, 2012, from:
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4041954&isnumbe
r=4041941.
Graf, S. and Kinshuk, K. (2009). Advanced Adaptivity in Learning Management Systems
by Considering Learning Styles. Proceedings of the 2009 IEEE/WIC/ACM
International Joint Conference on Web Intelligence and Intelligent Agent
Technology, Volume 03, Pages 235-238. Accessed May 12, 2012, from:
http://dl.acm.org/citation.cfm?id=1632300.
Hernndez, Y. and Silva, A. (2011). Una Experiencia Tecnopedaggica en la
Construccin de Objetos de Aprendizaje Web para la Enseanza de la
Matemtica Bsica. Revista de Tecnologa de Informacin y Comunicacin en
Educacin Eduweb. Vol 5 No 1. Junio 2011. Accessed November 18, 2011, from:
http://servicio.bc.uc.edu.ve/educacion/eduweb/vol5n1/art4.pdf.
Herrmann, N. (1982). The Creative brain. NASSP Bulletin, 31-45.
Herrmann, N. (1990). The Creative Brain. Brain Books, Lake Lure, North Carolina.
Kolb, D. (1976). The Learning Style Inventory: Technical Manual, Boston, Ma.: McBer.
McCarthy, B. (1987). The 4MAT system: Teaching to learning styles with right/left mode
techniques. Barrington, IL: Excel, Inc.
Mustaro, P. and Frango, I. (2006). Learning Objects: Adaptive Retrieval through
Learning Styles. Interdisciplinary Journal of Knowledge and Learning Objects,
Volume 2. Accessed January 20, 2012, from:
http://www.ijello.org/Volume2/v2p035-046Mustaro.pdf.
Nrici, I. (1992). Hacia una didctica general dinmica. 3 Edicin. Kapelusz, Argentina.
Ossandn, Y. and Castillo, P. (2006). Propuesta para el Diseo de Objetos de
Aprendizaje. Revista Facultad de Ingeniera. Universidad del Tarapac, vol. 14
N 1, pp. 36-48. Accessed July 13, 2011, from:

35
http://www.scielo.cl/scielo.php?pid=S071813372006000100005&script=sci_artte
xt.
Paredes, P. (2008). Una Propuesta de Incorporacin de los Estilos de Aprendizaje a los
Modelos de Usuario en Sistemas de Enseanza Adaptativos. Tesis Doctoral.
Universidad Autnoma de Madrid. Departamento de Ingeniera Informtica.
Madrid, Espaa. Accessed March 20, 2011, from:
http://arantxa.ii.uam.es/~pparedes/tesis.pdf.
Polsani, P. (2003). Use and Abuse of Reusable Learning Journal of Digital Information,
Volume 3 Issue 4, Article No. 164. Accessed March 23, 2011, from:
http://journals.tdl.org/jodi/article/viewArticle/89/88.
Popescu, E., Badica, C. and Moraret, L. (2010). Accommodating Learning Styles in an
Adaptive Educational System. Informatica, International Journal of Computing
and Infomatics, 34 (2010). 451462. Accessed April 2, 2012, from:
http://www. informatica.si/vol34.htm #No1
Rivas, M. (2008). Procesos Cognitivos y Aprendizaje Significativo. Madrid. Comunidad
Autnoma. Servicio de Documentacin y Publicaciones.
Wiley, D. (2000). Connecting learning objects to instructional design theory: A definition,
a metaphor, and a taxonomy. In D. A. Wiley (Ed.), The Instructional Use of
Learning Objects. Accessed March 03, 2011, from:
http://reusability.org/read/chapters/wiley.doc.
Woolfolk, A. (2006). Psicologa Educativa. 9 Edicin. Pearson Educacin, Mxico.

36

Vol. 4, No.1, pp. 36-50, April 2014
Influential Factors in Modelling SPARK Science

Learning System
Marie Paz E. Morales

Educational Policy Research and Development Center
Philippine Normal University
1000 Manila, Philippines
Abstract. The study is focused on the exploration of influential factors in

modelling PASCO-designed technology in science classes. Mixed
method was employed to critically explore how the SPARK Science
Learning System is meaningfully integrated into the teaching of selected
topics in Earth and Environmental Science. The SPARK Science learning
system is an all-in-one mobile device that integrates the power of probe
ware with inquiry-based content and assessment. It is a device that
includes a large, full-color display, finger-touch navigation and data
collection and analysis capabilities designed to become a discovery-
based science learning environment. It provides both the teacher and the
students the embedded support for exploring science concepts. Results
show that there is a significant gain in student achievement with the
integration of SPARK Science learning system. Significant positive
correlation is observed between post-test and intrinsic motivation.
Correlation between post-test and evaluation and correlation between
intrinsic motivation and evaluation, however, posit non-statistically
significant correlation. Mapped advantages and disadvantages of using
the technology resulted to recurring themes for framework design of
using the SPARK Science Learning System to further institute its effect
in the curriculum as a precursor towards envisioning the 21 st century
learning.
Keywords: Environmental Science; Technology Integration, Pedagogy
Introduction
Dramatic technological revolution ushered the new millennium. Focus on
digitization and technology use has been the subject of several researchers
because of this trend. In many countries, todays students are referred to as
digital natives and todays educators as digital immigrants. Thus, there is a
need for teachers to work closely with students whose entire lives have been
immersed in the 21st century media culture. This enculturation of students as
digital natives is described as P21 or better known as Partnership for 21st
Century Skills (Kellner 2002).

37
It is well-established by researches that integrating technology into the

curriculum and instruction will bring about significant student achievement and
therefore deepunderstanding of concepts (Clark, 2010). He claimed, however,
that technology has to be integrated meaningfully into the curriculum and
instruction, for probable positive impact on student learning and achievement.
Meaningful integration of technology refers to the process of matching the
most effective tool with the most effective pedagogy to achieve the learning
goals of a particular lesson. Each tool brings different opportunities to the
learning environment and involves a different set of skills on the part of teachers
and students. Each can play a unique role in the learning process when used at
the appropriate time, under the most suitable learning conditions. It is simply
the degree to which a particular technologys capabilities are matched with the
expected learning outcomes and supported by fitting pedagogy that will
determine the impact that technology has on learning and achievement (Clark,
2010).
This match of the technological tool with pedagogy and curriculum is the main
focus of the study. Further, the research would want to establish that this match
is feasibly achieved by the attributes of the teachers as the digital immigrants
working collaboratively with the students as the digital natives to help foster
the intended partnership and be confluent with the P21 flow.
The purpose of this research is to investigate the use of SPARK Science learning
system in Earth and Environmental Science classes. The specific research
objectives are to:
1. Determine the effect of using the SPARK devices on student motivation;
2. Establish the effect of using SPARK on student achievement;
3. Identify the influential factors in modelling integration of technology
(SPARK Science learning system) in science classes; and
4. Design a framework to integrate technology in science classes and adopt
them to the 21st century learning.
Framework and Literature

In the 21st century framework, the definition shifts to learning towards learning
technologies and on how instructional technologies can best serve learning. The
Association for Educational Communications and Technology (AECT 2003)
defines educational technology as "the study and ethical practice of facilitating
learning and improving performance by creating, using and managing
appropriate technological processes and resources (Richey, Silber, & Ely, 2008).
A revisit of important attributes of learning such as motivation and preliminary
attempts of technology integration can explore the initial results of the benefits
of technology in the curriculum.
On motivation
One of the many aspects that can help foster better achievement by students in
the classroom, according to Slavin (2003), is motivation. He defined motivation
as what gets you going, keeps you going, and determines where you want to
go. Many researchers (Brookhart et al. 2006; Palmer 2005; and Mazer, Murphy

38
& Simonds, 2009) provide an impression that motivation is the key component
in reaching a high level of student achievement.
In a study conducted by Martin (2006), he argued that if students set meaningful

goals that are attainable, they will progressively achieve higher results. There is
a need to provide students with a distinct set of goals that can help them be
motivated. He further suggested that if students have predetermined goals they
will strive for personal bests with a higher level of motivation. Teachers can play
a large role in determining the motivation level of the students in the class.
Studies on the effects of teacher self-disclosure on student motivation using
Facebook web-based software as medium for disclosure conclude that students
were more motivated when their teacher shared some personal information
about themselves. However, some disadvantages of this self-disclosure surfaced
with too much self-disclosure which led to non-elicitation of same motivation
(Mazer, Murphy & Simonds, 2009).
On technology integration and learning

Educational technology has been defined in numerous ways. It usually
highlights the teacher and the pedagogies that might be employed on the
learner. In the 20th century, four paradigm shifts are characterized as the
physical science or media view; the communications and systems concept; the
behavioral science-based view; and the cognitive science perspective. Each of
these shifts has different philosophical and theoretical orientations that affected
theory, practice and definitions of educational technology (Saetller, 2004).
Several studies have been conducted on the goodness and effectiveness of

technology as integrated into the curriculum or instruction. According to Floyd
et al. (2008), integration of technological advances should be a major part in
designing the most effective and innovative emergent technology literacy
intervention. Successful technology integration, according to Mishra & Kohler
(2006), requires that educators blend strong content knowledge with appropriate
pedagogical strategy. From which they were able to come up with Technology-
Pedagogy-andContent Knowledge or TPACK framework. This highlights P21
or known as Partnership for the 21st Century Skills, which focused on
meaningful integration of technology. As expressed by Clark (2010),
integrating technology in meaningful ways involves matching instructional tools
with curricular goals, desired student outcomes and instructional practice.
Choosing the right tool for a learning task requires not only familiarity with
the kinds of tools available, but also depends upon an understanding of how
those tools can support the development of desired knowledge and skills. As
with any tool selected for any purpose, the choice of what technology to use and
how to use it must be guided by a set of beliefs-a vision-for how learning is best
supported. Though technology integration is foreseen as a way of attaining
meaningful learning on the part of the digital natives, there were several studies
marking the disadvantages of technology integration. One of which was noted
by Schmidt in Veitch (2010) who voiced a concern that people might be losing
deep-reading skills, as they spend less time reading long-form literature
passages. This probably has an effect on cognition and reading, although no one
really knows what that does. Gasser and Palfrey (2009) identified multitasking

39
as a skill developed when students are engaged in technology integration. They

claimed multitasking does not render learning impossible. It does not even
necessarily make it more difficult to accomplish tasks. Multitasking is likely to
change learning qualitatively by making the learner rely on different memory
systems that vary in flexibility when it comes to the use of knowledge. However,
they also mentioned that the loss of attention and the time spent switching from
task to task is likely to have an adverse effect on digital natives' ability to learn
complex new facts and concepts. Some of these issues and concerns of
technology integration into the curriculum were addressed by Siemens (2005a)
theory of connectivism, where he claimed that technology has also contributed
to a rise in informal learning where the majority of education no longer occurs in
formal settings but through learning communities of practice, personal networks
and through completion of work related task. In contrast to established theories
of learning, the essence of connectivism is that learning is viewed as a
connections/network-forming process (Siemens, 2005b).
Meaningful technology integration touches ground on motivation and

appropriate use of tools to match the learners and pedagogy at hand. The
information provided by this research is of value to science teachers working on
similar objectives. This also allows science teachers to explore and improve their
motivation techniques which may later lead to a deep conceptual understanding
of the subject matter. Further, the results would help establish effectiveness of
technology-inspired science classroom in trying to be at par with the 21st century
learning.
Methodology
The study used mixed methods in order to gather data and pertinent
observations regarding the use of technology in science classroom. Presented
below is the summary of the study including different stages, data gathering
procedures, participants and statistical analysis. Qualitative method was used to
validate quantitative results derived from the investigation.
Table 1: Summary of the Methodology

Stages of the
Data Collection Instrument/Tool Data Analysis
Study
Literacy and Average
Technology ratings and
SPARK Science Checklist Aikens content
Preparation Learning System Intrinsic validity
and Pre- Orientation Motivation coefficient
Implementation Administration Inventory Averages of
of pre-tests Achievement Test technology
Evaluation Form literacy
constructs
Lesson Sessions
During using SPARK SPARK Science
Instruction Science Learning Learning System
System
Post-
Administration Intrinsic Paired sample
Implementation

40
of post-tests, Motivation t-test

post- Inventory Correlation
implementation Achievement Test Interview
interviews Evaluation Form transcriptions
Interview protocol
Participants
The participants of this study included one intact class of tertiary students who
were specializing in physics and were enrolled in both Computer Literacy 1 and
Earth and Environmental Science classes. These are pre-service physics students
who qualified as Philippine government scholars in science teaching. They enjoy
the consortium benefits with the De La Salle University, Manila. As government
scholars, these students were nationally selected from different science oriented
and non-science oriented high schools all over the Philippines. They also enjoy
the benefits of the grant and are envisioned to be the future Physics teachers.
Materials and Instruments

SPARK Learning System - This is an all-in-one mobile device that integrates the
power of probe ware with inquiry-based content and assessment. The device
includes a large, full-color display, finger-touch navigation and data collection
and analysis capabilities. It is designed to become a discovery-based science
learning tool, providing both the teacher and the students the embedded
support for exploring science concepts. It has more than 60 free pre-installed
SPARK-labs which are standard-based guided inquiry labs in a unique
electronic notebook format that integrates background content, data collection,
analysis, and assessment. (PASCO Scientific, 2008).
Literacy and Technology Checklist - This instrument established the students

knowledge and know how in technology, literacy and web expertise, which is
requisite to the use of SPARK Science Learning system. The three major parts of
the instrument are background information, technology component, and literacy
& web expertise. The second part highlights technology component using a four-
point Likert scale system. In addition, the other components are in open-
approach.
Intrinsic Motivation Inventory - This is a multidimensional instrument intended to

appraise participants subjective experience related to a target experiment in
laboratory sessions. It has been used in several experiments related to intrinsic
motivation and self-regulation (Gottfried, 1985). There are several versions of
this inventory. The two versions were used in the study are the full 45-item tool
that completes the 7 subscales, and the 25-item version that was used in the
internalization study, including the three subscales of value/usefulness,
interest/enjoyment, and perceived choice.
Achievement Test - The achievement test is a 19-item test, which has undergone
content validation by three science experts and science educators. Item analysis
procedures have reduced the number of questions of the set from 25-items to 19-
items. The test covered topics on radiation and insolation which are the major
topics on which the SPARK Science Learning System were integrated.

41
Evaluation Form - This is a 13-item survey in Likert scale intended to identify the
insights of the students on the use of SPARK Science Learning System as a
technology in the teaching and learning of science concepts. This post-
implementation tool was administered to students where they were asked to tick
on the appropriate cell. Part of the tool included questions related to the
advantages and disadvantages of using SPARK Science Learning System in
open-ended format.
Procedure
Preparation and pre-implementation

Pre-implementation commenced with the preparation of the equipment and the
instruments needed for the study. Correspondence with De La Salle University,
physics laboratory technicians and computer literacy instructor of the
participants was done prior to implementation of the technology integration. As
pre-intervention procedure, Literacy and Technology Checklist, Intrinsic
Motivation Inventory, and Pre-Test (Achievement Test) were administered.
Profiling of students was conducted to determine their background information

and their technological literacy. Since every participant is a government scholar
in physics teaching, these students are highly motivated to study Physics. Thus,
SPARK Science Learning System was integrated to Earth Science lessons instead
of lessons in Physics.
Table 1. Technology Literacy Checklist

Experience Computer
Access on
Respondent Gender High School with Literacy and
Technology
Technology Web Expertise
R1 Male Marikina Science High 0.7 0.9 0.8

School
R2 Male Ramon Magsaysay Cubao 0.7 0.9 0.6
High School
R3 Female Tala High School 0.7 0.8 0.5
R4 Female LPNHS (main) 1.0 0.9 0.8
R5 Female DARSSTHS 1.0 0.8 0.6
R6 Female Patoc National High 1.0 0.8 0.4
School
R7 Female Ramon Magsaysay Cubao 1.0 0.8 0.5
High School
R8 Female Sorsogon national high 1.0 0.8 0.7
school
R9 Male Jonu Rural School 1.0 0.7 0.4
R10 Female Muntinlupa Science High 1.0 0.7 0.7
School
R11 Female Rizal National High 1.0 0.8 0.8
School
R12 Female Lagro High School 1.0 1.0 0.8
R13 Female NOHS 0.7 0.6 0.5
R14 Female Jose P. Laurel High School 1.0 1.0 1.0
R15 Female Rosario National High 0.7 0.8 0.7

42
School
R16 Female San Jose National High 1.0 0.8 0.5
School
R17 Male Pasay City South High 1.0 0.9 0.8
School
R18 Male Ramon Magsaysay Cubao 1.0 0.8 0.7
High School
R19 Female Cavite National High 1.0 0.9 0.8
School
R20 Female Paranaque national High 1.0 0.8 0.7
School-Lahuerta
R21 Female Cavite National High 1.0 0.9 0.7
School
R22 Female MORMS 1.0 0.8 0.4
R23 Female Mount Carmel School Of 1.0 0.8 0.7
Infanta
R24 Male Binan National High 1.0 0.9 0.7
school
R25 Male Baclaran high School 1.0 0.9 0.7
R26 Male DARSSTHS 1.0 0.8 0.7
R27 Male Paranaque National High 1.0 0.8 0.6
School-Lahuerta
AVERAGE 0.9 0.8 0.7
Table 1 shows the background information and the summary of the technology
literacy of the participants. The indices were computed as ratios of the averages
of student ratings based on a four-point Likert scale and the theoretical average
in each of the constructs: Access with technology, experience with technology,
and computer literacy and web expertise. All values are close to 1 which
connotes that all students are technologically literate enough ready to use the
SPARK Science Learning System.
These students were products of public high schools directly administered and
monitored by the department of education. Everyone graduated either from
science oriented schools, science high schools or department of science and
technology-science education institute node schools. These participants can be
said to be at par with one another in terms of learning experiences. Further, it
can be inferred that majority of these students have access to computers with
internet capabilities. This may be through the Learning Resource Center
provided by the department of science and technology, the Philippine Normal
University and the consortium benefits with the De La Salle University, Manila.
The majority of the participants use computers and other computer related
technology for personal interest and lesson-related activities which make their
technology usage a part of their daily routine. This means that they are well-
informed in manipulation of devices and technology which has the same
features as that of a computer. They can be considered ready users of the SPARK
Science Learning System.

43
During instructions
The succeeding sessions were focused on the integration of the SPARK Science
learning system to two major topics in Science 3 (Earth and Environmental
Science). The two major topics: radiation and insolation, in the course syllabus of
Science 3 (Earth and Environmental Science) were selected for the purpose of the
study. Session plans were prepared to map out the integration and instruction of
the selected topics.
The implementation of the integration of the SPARK Science Learning System in

selected topics was conducted in several sessions. The first session highlighted
the orientation on the SPARK Science Learning System. This orientation was
conducted at the Philippine Normal University. In this session the researcher
presented the visual reference, the users guide, and the quick start guide to the
participants. Discussions on how to use the instruments and some comparison
with the classical laboratory procedure were also presented and discussed with
the students. The first impression of the students was that the instrument maybe
very expensive. They expressed some anxiety on the use for reasons that they
may damage the said instrument. Further discussions on the said instrument
was done by comparing SPARK Science Learning System with some common
and familiar technology these students are adapted to like the touch screen
mobile phones and PSPs which helped them concretely visualize the introduced
technology (SPARK Science Learning System).
The succeeding sessions were hands-on orientation on the instrument and

integration of the SPARK Science Learning System on selected topics in Science 3
(Earth and Environmental Science) - Radiation and Insolation. The integration
procedure followed pedagogically accepted process as presented in the session
plans prepared by the researcher and content validated by experts including the
researchers consultant. Within the short span of time students were able to
come up with good results using the SPARK Science Learning System.
Post-Implementation
To determine the effect of the SPARK Science Learning System, an achievement
test was administered to the participants after implementation of SPARK
integration. Post-test results were statistically compared to the results obtained
in the pretest to determine gains if any in student achievement. Intact group pre-
test-post-test design was used in the study. One limitation of the study is
identifying comparable set of participants. Thus, only anecdotal comparison of
the student achievement using SPARK Science Learning System with student
achievement without the integration was done to validate the significant
statistical difference in the pre- and post-test results on student content
knowledge. The intrinsic motivation inventory and evaluation tool were
administered to determine whether the students were intrinsically motivated by
the SPARK Science Learning System integration. Correlation of post-test with
intrinsic motivation, post-test with evaluation, and intrinsic motivation with
evaluation was done to identify the factors that may have influenced the gains in

44
student achievement. Transcriptions of interviews and annotation of verbatim

answers on the open-ended questionnaire part of the evaluation were used to
further identify influential factors in the design of framework on technology
integration.
Results
The primary goals of this study are to establish the effect of using SPARK
Science Learning system on student achievement; to determine the effect of
using the SPARK devices on student motivation; to identify the influential
factors in modelling SPARK Science learning system in science classes; and
design framework to integrate technology in science classes and adopt them to
the 21st century learning. Results of the study are presented according to these
major goals.
On the effect of SPARK Science Learning System on student achievement

To determine if there was a significant gain in students content knowledge,
statistical comparison of the pre-test and post-test of the participants through
paired sample t-test was done as presented in Table 2.
Table 2. Paired Sample Statistics

Pre Test Post Test
Pair N p-value
Mean Mean
Pre-Test and Post-Test 25 9.00 13.60 0.00*
(*)Significant at 0.05
The participants performed better in the post test as compared to the pre-test
with the implementation of the SPARK Science Learning System. The difference
in the pre-test mean and the post-test mean was statistically significant with a p-
value of less than 0.05 (p-value = 0.00 < 0.05). As targeted, the integration of
SPARK Science Learning System has brought about significant gains in the
student achievement. This implies that the integration of SPARK Science
Learning System in selected topics in Earth and Environmental Science is highly
effective. Anecdotal comparison of student achievement using SPARK Science
Learning System with student achievement without the integration was also
done to validate the significant statistical difference in the pre- and post-test.
Students from other classes encountered difficulty in meaning making when it
comes to learning the concepts of Earth and environmental science. They usually
scored lower in examinations given to them. They are not that active during
class discussions and more often they encountered erroneous sets of data when
performing comparable experiments with those done by the participants.

45
On the effect of SPARK Science learning system on student motivation
Table 3. Correlation of SPARK Evaluation, Post-Test and Intrinsic Motivation

Categories Post-Test Evaluation Intrinsic Motivation
Mean 13.64 4.75 5.68
Pearson
Post-Test 1.00 -0.063 0.618*
Evaluation -0.063 1.00 -0.353
Intrinsic Motivation 0.618* -0.353 1.00
Model Summary** 0.464**
(*) Significant at 0.05 ** Predictors: Post-Test, Evaluation & Intrinsic Motivation
Table 3 presents post-test mean value of 13.64 out of the 19-item test of the
participants. This means that the participants were able to correctly answer more
than 70% of the items about radiation and insolation through the integration of
SPARK Science Learning System. Evaluation of the SPARK Science Learning
System has a high mean value (4.75 out of 5). This connotes that participants
express positive attitude towards the use of SPARK Science Learning System in
learning science concepts. Intrinsic motivation has moderate mean value of 5.68
out of 7.
Significant positive correlation is observed between post-test and intrinsic

motivation. The other pairs: post-test & evaluation and intrinsic motivation &
evaluation posit non-statistically significant correlation. Low positive
correlation of three variables: post-test, evaluation, and intrinsic motivation
presented in the model summary was observed with an R-value of 0.464. This
is lower than the usually accepted value of 0.5. This implies that there may be
other constructs of learning that are better predictors of student achievement
other than the evaluation of the technology (SPARK) and the post-experimental
intrinsic motivation.
Discussions
The high mean value of the evaluation of the SPARK Science Learning System is
complemented by the student answers in the open-ended portion of the
evaluation. They positively identified several advantages of using the device as
follows:
Learners will now find it easy and fun to do experiment. The results
will be no doubt accurate.
The SPARK is very useful during the experiments; students can
easily record data accurately while doing the graphs and tables at the
same time.
Besides from being handy, it is also good in understanding a concept
because the background gave the information about the concept and
after this is a follow up question that will help the student think.
It gives background concepts on the activity to be performed and asks
questions to tests our knowledge on the topic.
Results are readily seencontinuous to record data and can be
saved.

46
The device can be easily manipulated. It provides learners with

necessary guide questions that directly lead to further understanding
of the lesson and its concepts.
The concepts are already stated in the activities.
Its accurate, innovative, safe.
Similar answers were provided by selected students during the post-

implementation interview. They pointed out how the SPARK Science Learning
System was helpful and engaging to students. They attested that SPARK Science
Learning System is novel to them and is very visual in perspective, which
matches their learning needs and style.
Na-amaze ako mam sa nagagawa ng instrument or device.

(I was amazed with what the instrument can do.)
Yes mam, the SPARK Learning System helped a lot. I was able to
answer the follow up questions with ease and also the evaluation
questions.
Mam sometimes its hard to learn using books alone because they are
not that much available or engaging, unlike the SPARK, it has a way
of making interactions work out.
Yes mam, sa tulad ko po na madali makaintindi pag may illustrations
mas maganda para sa amin ang mga ganitong device para mas
maintindihan and concepts.
(Yes Mam, for student like me who hardly understands concepts
in science but can possibly do so with good visuals.)
I would recommend the use of SPARK Learning system but in
partnership with written outputs, written graphs and computations.
However, students have also identified several areas of weaknesses and

improvement in integrating SPARK Science Learning System in science lessons
to make learning much more meaningful and appreciated by them. In the post-
instruction interview, these pre-service students believe that the full potential of
SPARK Science Learning System may be achieved in combination with other
written curriculum materials. The positive correlation of post-test and intrinsic
motivation could mean that they were already highly motivated in the subject
area as they are science-oriented students but this intrinsic motivation is hardly
identified with the integration of the SPARK Science Learning System. This
result is complemented by students answers when asked about some
disadvantages of using the SPARK Science Learning System as follows:
Graphing skills of the students and manipulating data may be

affected negatively.
Less interaction or cooperation among students since it can be done
individually.
The students will be lazy and always depends on the SPARK.
The students might just rely on the tool in graphing and not do it
manually.
There will be little interaction between the teachers and the learners.
Learners will only depend on the approaches.

47
From the transcriptions, students still hold on to the ideology of learning by

doing. Graphical skills may not be developed if graphs are automatically done
by the instrument. Since everything becomes automated, students seem to exert
less effort and they perceive this as being lazy or not being able to give their best
shot in an activity. They further claim that the tool may just develop dependency
of the students to equipment rather than on their own skills. Although they were
working in groups and the nature of the course is collaborative and inquiry-
based, they feel that interactions within the group for them to nurture
relationships and build their socialization skills are fewer with the tool at hand.
They also experience less interaction with the teacher since all answers
regarding the topics presented can be understood using the SPARK Science
Learning System.
In terms of student achievement, integration implementing SPARK Science

Learning System was a success. The integration brought about significant and
meaningful learning on the part of the participants. Motivation, on the other
hand, did not positively correlate with student perceptions on implementing the
technology. The same non-correlation result was found between student
achievement and student perception on the integration of SPARK Science
learning system. This suggests that some other factors were able to influence the
motivation of students to learn such as other pedagogical techniques, teaching
and learning of other important science process skills that the technology is
incapable of doing, and learning environment. These factors were identified by
the participants in their verbatim answers in the open-response part of the
evaluation tool.
Implementing the SPARK Science Learning System could touch grounds on

learning and innovation skills, which focus on creativity, critical thinking,
communication and collaboration. This is a good foundation in preparation for
the shift towards P21 or 21st Century Learning. Embedded in the learning system
are activities that could promote the needed attributes of students to attain
learning and innovation skills. With the technology, students could be able to
exhibit a range of functional and critical thinking skills related to information,
media and technology. The use of the SPARK Science learning system gives
students more opportunities to develop skills related to information, media and
technology. Life and career skills are also needed for students to navigate the
complex life and work environments in the globally competitive information
age. This can be achieved by combining the SPARK Science learning system with
other curriculum materials that may develop the latter identified skills. These
are the needed skills of a new generation student to be able to adapt and be a
successful citizen. These are the bases of identifying the influential factors
needed in modelling the SPARK Science learning system in science classrooms.

48
Pedagogical Techniques
Collaborative
Self-Directed
Culture and Cross-Culture Based
Content-Based
Inquiry-Based
Other Curriculum 21st Century Skills Assessment

Material Learning and SPARK Integrated
Graphical Analysis Innovation Skills Assessment
Written Assessment Information, Media Performance-Based
Computations and Technology Assessment
Use of indigenous Skills
tools Life and Career Skills
Learning Environment
Balance of dependence and
independence
Interdependence
Socialization
Manipulative and hands-on
learning
Figure 1: Framework of Implementing Technology Integration
The framework presented in Figure 1 shows all the influential factors of

modelling the SPARK Science Learning System. It was identified that integration
of the SPARK Science Learning System was effective to a certain extent. The low
correlation observed between evaluation of the technology & intrinsic
motivation and evaluation of the technology with post-test result led to the idea
that positive results may not be solely attributed to the integration of the SPARK
Science Learning System in the pedagogy. It was noted that probable
combination of other curriculum materials, proper learning environment, proper
planning of integration process, and other forms of assessment could lead to
much more meaningful integration of the SPARK Science Learning System. As
claimed by Mishra & Kohler (2006), successful technology integration requires
that educators blend strong content knowledge with appropriate pedagogical
strategy. From the study, it can be gleaned that factors that influence the
significant effect of integrating technology are clustered into four. These are
pedagogical techniques, other curriculum materials, assessment procedures, and
learning environment. To achieve full meaning of technology integration,
combinatorial presentation of the four constructs with the integration would
achieve meaningful learning. This is known as Technology-Pedagogy-and
Content Knowledge or TPACK on which the designed model is aligned.

49
Conclusions and Recommendations

The foci of this study were to establish the effectiveness of the integration of the
SPARK Science Learning System on selected topics in Earth and Environmental
Science. It is intended to determine whether or not integration of SPARK
Science Learning system positively affects student motivation eventually leading
to student achievement; to identify influential factors on modelling technology
and to design framework of technology integration. The intervention
administered was effective because it led to a significant gain in the pre-test and
post-test mean difference on student knowledge of the content. This implied a
meaningful integration of the SPARK Science Learning System on Earth and
Environmental Science. The integration of the SPARK Learning System also had
positive effects on student post-experimental intrinsic motivation and was
evaluated positively by the respondents. These were separately manifested in
the means or averages of the data sets. However, it was noted, that two of the
three variables: post-test, evaluation, and post-experimental intrinsic motivation
had low positive correlation. It can be inferred that although the integration was
effective, constructs other than student motivation and evaluation of the
integrations contributed to the mean gain in the pre-test and post-test difference.
Post-instruction interviews with the students provided other details of the low
correlation. Further, influential factors that are needed in the much more
meaningful integration of the SPARK Science learning system were deduced
from the post-implementation interviews and open-responses of the students in
the evaluation. These factors were noted as inputs to the design of framework
which captured all study results for meaningful integration of technology
(SPARK) leading to development of 21st century skills as preparation to P21
learning. Integrating technology is not just using the technology. It is a special
skill of combining the technology with other learning constructs such as
curriculum materials, pedagogy, assessment and learning environment to
achieve the full potential of the technology to induce learning to the students.
Though the study is able to enhance student achievement and has provided
framework for meaningful integration of technology, there are still some
limitations. The identified participants were government scholars in the field of
sciences thus they are already science enthusiasts. Large gains in terms of
student achievement and learning motivation may be deduced if the study is
replicated to a group of non-science students. An experimental design with a
control group may be adopted to compare student achievement with the
integration to those without technology integration. The focus of the
experimental study would be cognition, process skills and affective domains of
learning. Pre-and post-implementation interviews may also be conducted
highlighting motivation constructs and not only focused on perception of the
students on the use of the technology. Classroom observations can be done to
determine other significant observations which may not be provided by
interviews and test results.
Replication of the study is needed to fully establish effectiveness of meaningful

integration of technology in learning science. A study to test the designed model
may help launch meaningful integration of technology that leads to the

50
development of the 21st century skills. Teacher education curriculum designers

may look into the feasibility of the model or framework in developing pre-
service students TPACK that would greatly support the development of
teachers competencies that would help mold the 21st century learners.
References
Brookhart, S.M., Walsh, J.M., & Zientarski, W.A. (2006). The Dynamics of Motivation and
Efforts for Classroom Assessments in Middle School Science and Social Studies.
Applied Measurement in Education, 19(2), 151-184.
Clark, J. (2010). Best Practices Research Summary. Sun Associates 2010. Retrieved
November 1, 2012 from www.sun-associates.com
Floyd, K. et.al. (2008). Assistive Technology and Emergent Literacy for Preschoolers: A
Literature Review. Assistive Technology Outcomes and Benefits, 5(1), 92-102.
Gasser, U., & Palfrey, J. (2009). Mastering Multitasking. Educational Leadership, 66(6),
15-19.
Gottfried, A. E. (1985). Academic Intrinsic Motivation in Elementary and Junior High
School Students. Journal of Educational Psychology, 77, 631-635.
International Technology Education Association. ( 2003). Advancing Excellence in
Technology Literacy: Student Assessment, Professional Development, and Program
Standards. Retrieved October 15, 2011from www.iteawww.org\
Kellner, D. (2002). New Media and New Literacies: Restructuring Education for the New
Millennium. Retrieved March 4, 2012 from http://pages.gseis-
ucla.edu/faculty/kellner.
Martin, A. J. (2006). The Relation between Teachers Perceptions of Student Motivation
and Engagement and Teachers Enjoyment of and Confidence in Teaching. Asia-
Pacific Journal of Teacher Education, 34, 73-93.
Mazer, J., Murphy, R., & Simonds, C. (2009). The Effects of Teacher Self-disclosure via
Facebook on Teacher Credibility. Learning, Media and Technology, 34(2), 175-183.
Mishra, P., & Koehler, M. J. (2006). Technological Pedagogical Content Knowledge: A
new framework for teacher knowledge. Teachers College Record. 108(6), 1017-1054.
Palmer, D. (2005). A Motivational View of Constructivist-Informed Teaching.
International Journal of Science Education, 27(1), 1853-1881.
_______(2008). PASCO Scientific. Retrieved December 15, 2011 from
http://www.pasco.com/prodCatalog/PS/PS-2008_spark-science-learning-
system/index.cfm
Richey, R. C., Silber, K. H., & Ely, D. P. (2008). Reflections on the 2008 AECT definitions
of the field. TechTrends, 52(1), 24-25.
Saettler, P. (2004). The Evolution of American Educational Technology. Greenwich, CT:
Information Age Publishing.
Siemens, G. (2005a, January). Connectivism: A learning theory for the digital age.
International Journal of Instructional Technology and Distance Learning, 2(1).
Retrieved December 30, 2011 from http://www.itdl.org/
Journal/Jan_05/index.htm.
Siemens, G. (2005b). Learning Development Model: Bridging Learning Design and
Modern Knowledge Needs. Elearnspace. Retrieved October 25, 2011 from
http://www.elearnspace.org/Articles/ldc.htm
Slavin, R. (2003). Educational Psychology, Theory and Practice (7 thed.). Boston, MA:
Allyn and Bacon.
Veitch, M. (2010). Google's 'Deep Reading' Fears Lost in Shallows. CIO Insider. Retrieved
December 20, 2011 from http://www.cio.co.uk/blogs/cio-news-view/googles-
deep-reading-fears-lost-in-shallows/

51

Vol. 4, No.1, pp. 51-60, April 2014
Investigating Reliability and Validity for the

Construct of Inferential Statistics
Saras Krishnan and Noraini Idris

University of Malaya
Kuala Lumpur, Malaysia
Abstract.A hierarchical construct to assess and describe students

learning of inferential statistics has been previously developed using the
Rasch analysis. In particular, the Rasch Partial Credit Model was
instrumental in identifying the number of strata in the construct and in
establishing the reliability and validity of the instrument used. In this
paper, the analysis is replicated with a different sample of students to
investigate if the reliability and validity still hold.
Keywords: assessment; Partial Credit Model; Rasch analysis
Introduction
Past studies in various aspects of inferential statistics provide evidence of
students continual difficulties in learning the many aspects and concepts of
inferential statistics (e.g., Francis, Kokonis,& Lipson, 2007; Weinberg, Wiesner,&
Pfaff, 2010). This situation is worrying since inferential statistics is taught in a
majority of courses,and the knowledge and skills of inferential statistics will be
required at one time or another by the students. Despite the many studies of
students learning of various topics of inferential statistics, at present there is
need for more research in this area(Smith, 2008; Sotos, Vanhoof, Noortgate,&
Onghena, 2009).
A construct of learning to describe students understanding of inferential

statistics in hierarchical levels has been developed as part of the main authors
postgraduate research. The developmental process of this construct is discussed
in Krishnan and Idris (2013a). Discussion included the use of Rasch analysis in
establishing the reliability and the validity of the results, and in determining the
number of levels in the construct. Further, another paper discussed the use of
thehierarchical construct to investigate students learning of inferential statistics
(Krishnan & Idris, 2013b). In this paper, we investigate the reliability and
validity of the results using a different sample of students with the same sample
size. The purpose of this investigation is to determine if the conditions of
reliability and validity are still fulfilled when a different sample of students is
used.

52
Literature Review
Issues of concernin the assessments in statistics education included the
assessment of various statistics topics and the assessment of aspects of statistics
that are indicative of students varying levels of understanding (Bude, 2006).
Another concern is that assessment of students statistical learning is yet to be
adequately addressed (Smith, 2008). With regards to these concerns, several
attempts have been made to assess and describe students learning of statistics in
hierarchical stages of understanding.
The statistical literacy construct developed by Watson and Callingham (2003)

describes students understanding of statistics involving average and chance,
sampling and inference, representation of data, and variation. The two
frameworks for the statistical literacy construct are Biggs and Collis (1982, 1991)
SOLO Taxonomy and Watsons (1997) three tier statistical literacy model. Other
constructs that assessed students learning of statistics have basically evolved
from Watson and Callinghams construct (e.g., Callingham, 2009; Kaplan &
Thorpe, 2010; Watson, Kelly,& Izard, 2005). On the other hand, Kataoka, da
Silva, Vendramini, and Cazorla (n.d.) used the SOLO Taxonomy to categorize
students responses to a statistics questionnaire but did not offer any learning
construct in their study.
The Construct of Inferential Statistics (Krishnan & Idris, 2013b)contains six

hierarchical levels that describe students understanding of inferential statistics
in increasing complexity as we ascend the levels. For instance, the first level
involves students ability to identify inferential terminologies and symbols when
presented in contextual form while the fifth level involves understanding of
sampling, students ability to infer in different contexts, and knowledge and
understanding of inferential procedures and concepts.
The different constructs of statistics have been developed using the Rasch model
for analysis of data. Rasch model has been particularly useful in statistics
assessments in determining students levels of understanding of various
statistics concepts.Apart from that, Rasch analysis has also been used to
investigate students understanding of basic statistical concepts (Kassim, Ismail,
Mahmud,& Zainol, 2010),and to investigate attitude and knowledge of statistics
among postgraduate students (Mahmud, 2011).
There are two important reasons for using the Rasch analysis in our studies.
First, Rasch analysis is used to determine the number of strata or levels of the
construct in describing the stages of students understanding of inferential
statistics. Second, Rasch analysis is used to establish the reliability and validity
of the instrument and the sample of students. The first reason is facilitated by
the use of the item separation reliability and the item-person map. The second
reason is facilitated by the use of the fit analysis primarily the table of summary
statistics and the table of misfit order of items. Explanation on these can be
found in Krishnan and Idris (2013a, 2013b). The item-person maps are not
included in this paper due to the irrelevancy to the discussion here.

53
Among the different types of Rasch models, the Partial Credit Model (Masters,
1982) is especially instrumental in our research because it accommodates items
that have different hierarchical scoring categories. In other words, the Rasch
Partial Credit Model allows the dichotomous and polytomous items to be put
together in the same instrument (Bond & Fox, 2007). Thus, it is a model
particularly practical and instrumental in education assessments because it is
common for students to provide partly correct answers to any questions in a
written assessment.
Methodology
Research design
Descriptive research design, in particular the cross-sectional survey method was
used to collect quantitative data. Descriptive research primarily describes a
current state of affairs usually with the use of visual aids (Knupfer & McLellan,
2001). Our studies employed the descriptive research design because we want to
describe categories of information relating to students understanding of
inferential statistics with the aid of the item-person map in the Rasch analysis.
Instrumentation
The instrument used to collect data in this study is a task-based questionnaire on
inferential statistics. Three progressive sets of pilot studies were conducted in
developing the instrument. At each stage the instrument was further improved
to meet the criteria of Rasch analysis particularly in terms of the reliability and
validity of the instrument. In addition, the language and the structure of the
questions were also modified to be able to elicit more valid responses from the
students.
The purpose of the first pilot study was to collect baseline data to get an idea of
the possible responses to the questionnaire and possible problems in coding
these responses. The second and third pilot studies had a more definite purpose
of investigating the quality of the instrument whereby items that do not meet the
conditions of reliability and validity are either removed from the instrument or
are restructured. The results of these pilot studies are reported in Krishnan and
Idris (2013a).
The final instrument named as the Questionnaire for the Construct of Inferential
Statistics contained 10 main items and 21 items altogether and is a task-based
questionnaire that allows students to give open-ended responses. As such, we
are able to gather a multitude of different responses and can perceive a greater
variability of students learning of inferential statistics in the higher education.
As of now, we are not able to furnish the questionnaire due to the unpublished
status of the first authors thesis.
Data collection
The actual data collection process was carried out over a period of 6
months.Two factors contributing to the duration is the availability of the
students and authorization from the higher education institutions in concern.

54
Each data collection required 40 minutes where in the first 10 minutes the
students were briefed about the purpose of the data collection and were given
the necessary instructions. Then, students had 30 minutes to respond to the
items in the questionnaire individually. Data collection involved 150 students in
each sample. In using Rasch analysis, there are no specific requirements for the
sample size. In general, the sample size is large enough if the item reliability is
not less than 0.90.
Samples of study
Malaysia is a country in the South East Asia with a population of various ethnic,
cultural and lingual backgrounds. The many ethnic groups predominantly
consist of the Malay, Chinese and Indian races. The national language is the
Malay language while English is widely used as the second language. The two
main higher education providers in Malaysia are the government (60%) and the
private sector (40%). Notwithstanding, the number of students opting for a
private education has been increasing over the years (Krishnan & Idris, 2013c).
Purposive sampling has been used to identify the samples of students from the
different higher education institutions. Sample 1 is made up of students from
one private and one semi-private higher education institutions from two
different states in the central region of the country. The private higher education
institution was founded more than a quarter century ago and at present offers a
range of programs from pre-university studies to postgraduate courses. The
students for this study are taken from one pre-university program and two
different degree programs from this private higher education institution.
The semi-private higher education institution has been in operation longer than
the private higher education institution, having evolved from a training centre to
a full fledge higher education provider. Some of the courses available at this
institution are architecture, communication studies and dentistry. The students
for this study are taken from an external pre-university program at this semi-
private higher education institution, which is a different pre-university program
than the one from the private higher education institution.
Sample 2 consists of students from a public higher education institution in the

northern region that offers a range of undergraduate and postgraduate
programs in pedagogy in the different faculties it houses. The 150 students
sampled from this higher education institution belong to the same diploma
program and is taught by the same instructor in four separate classes. Although
the official medium of instruction at this institution is English, the Malay
language was often used because the students are largely from the Malay
language speaking background and thus have limited English speaking and
writing capabilities. The teaching materials too are sometimes provided in dual
languages to compensate students English language inadequacy.
The defining differences between these two samples are: (i) gender, (ii) ethnicity,
and (iii) English language capability. Table 1 shows the composition of students
in the samples according to this segregation. In comparison, both samples have

55
more female students than the male students. The largest ethnic group for
Sample 1 is Chinese while the largest ethnic group for Sample 2 is Malay. On the
other hand, the smallest ethnic group for Sample 1 is other ethnicity while the
smallest ethnic group for Sample 2 is Indian. Further, a small percentage of the
students in Sample 1 maintained that they have good English speaking and
writing capabilities whereas for Sample 2 the students English capability ranged
from moderate to poor. None of the students in Sample 2 have good English
speaking or writing capability. In fact, for both samples, the largest percentages
of students have moderate speaking and writing capabilities of the English
language.
Table 1: Composition of students in the samples

Sample 1 Sample 2
Gender Male 42.7% 25.3%
Female 57.3% 74.7%
Ethnicity Malay 29.3% 88.7%
Chinese 56% 2%
Indian 8% 0.7%
Others 6.7% 8.7%
Spoken English Good 14.7% 0%
Moderate 67.3% 81.3%
Poor 18% 18.7%
Written English Good 15.3% 0%
Moderate 66% 87.3%
Poor 18.7% 12.7%
Analysis of Results
Table 2 shows the reliability and fit indices for Sample 1. These results have been
discussed in earlier paper that described the development of the hierarchical
construct (Krishnan & Idris, 2013b). The purpose of this study is to investigate
the results of these indices for a different sample, Sample 2. The item separation
reliability determines the breadth of the items whereby a value more than 1.00
indicates that the items have enough breadth as with the case of Sample 1. In a
similar manner, the person separation reliability must be more than 1.00 to
warrant that the students are measured across the continuum. This condition has
been met by Sample 1.
Table 2: Reliability and fit indices for Sample 1

Item separation reliability 6.48
Item infit mean square 1.00 (s.d. 0.08)
Item reliability 0.98
Person separation reliability 1.77
Person infit mean square 1.03 (s.d. 0.33)
Person reliability 0.76
Cronbachs alpha 0.75

56
The item infit mean square and the person infit mean square must be in the
range of 1.00 to 1.20 to be reckoned as acceptable. Value less than 1.00 means
that the responses are too predictable. It also suggests the presence of redundant
items. On the other hand, value more than 1.20 suggests unpredictable
responses or inappropriate response patterns. Meanwhile, the standard
deviation must be smaller than 2.00 to indicate little misfit.Both the item infit
mean square and the person infit mean square for Sample 1 as well as their
standard deviation values met the required conditions.
As mentioned in Krishnan and Idris (2013b) there is no hard and fast rule on the
acceptable range of the fit statistics and different researchers have complied with
different ranges of these values. Discussion on the possible different values of
the fit statistics can be found in Green and Frantom (2002), and Linacre (2002).
In addition, the item reliability, the person reliability and Cronbachs alpha in
Table 1 are more than 0.70. The item reliability and the person reliability values
are equivalent to the value of Cronbachs alpha, said Green and Frantom (2002).
In this study, Cronbachs alpha of 0.70 is used as an acceptable reliability
coefficient (Nunnaly, 1978; Santos, 1999).
For Sample 2, some of the aforementioned conditions were met whereas others
were not. First, the item separation reliability of 3.98 and the person separation
reliability of 1.03 both satisfy the condition that these values must be more than
1.00. However, they are lower than the values for Sample 1. This observation
suggests that the spread of the items and students in Sample 2 is smaller
compared to Sample 1. The item infit mean square for Sample 2 is in the
stipulated range of between 1.00 and 1.20 but the person infit mean square does
not fulfil this condition. Likewise, the item reliability is more than 0.70 but the
person reliability is not. The Cronbachs alpha too does not meet the condition of
reliability.
Table 3: Reliability and fit indices for Sample 2

Item separation reliability 3.98
Item infit mean square 1.00 (s.d. 0.10)
Item reliability 0.94
Person separation reliability 1.03
Person infit mean square 0.98 (s.d. 0.45)
Person reliability 0.51
Cronbachs alpha 0.58
The misfit order of items for analysis of both samples is displayed in Table 4.
The infit mean square values (denoted by MNSQ) and the infit z-standardized
values (denoted by ZSTD) are investigated to establish the validity of an
instrument whereby the conditions for validity are:
(i) MNSQvalues between 0.70 and 1.33 (Watson & Callingham, 2003), and
(ii) ZSTD values between -2.00 and +2.00 for samples of sizes between n = 30
and n = 300 (Bond & Fox, 2007).

57
Table 4 shows that both conditions have been met for Sample 1 and Sample 2.
For Sample 1 the MNSQ values ranged from 0.86 to 1.17 and the ZSTD values
ranged from -1.60 to 1.40 (Krishnan & Idris, 2103b). Meanwhile, for Sample 2 the
MNSQ values ranged from 0.79 to 1.27 and the ZSTD valuesranged from -0.80 to
1.40.
Table 4: Misfit order of items

Sample 1 Sample 2
MNSQ ZSTD MNSQ ZSTD
1.15 1.40 0.92 -0.20
1.12 1.30 1.27 1.00
1.17 1.20 1.07 0.90
1.07 0.60 1.14 1.40
1.13 1.10 1.09 0.60
0.94 -0.60 1.00 0.10
0.94 -0.60 1.07 0.30
1.00 0.00 1.04 0.70
0.92 -0.40 1.04 0.40
1.03 0.40 1.01 0.20
1.01 0.20 1.02 0.20
0.96 -0.40 1.01 0.10
0.99 -0.10 1.00 0.30
0.99 -0.10 1.00 0.30
0.98 -0.10 0.99 -0.10
0.98 -0.10 0.92 -0.80
0.97 -0.20 0.98 0.00
0.94 -1.20 0.96 -0.10
0.91 -0.90 0.89 -1.30
0.88 -1.10 0.89 -0.40
0.86 -1.60 0.79 -0.60
Overall, the analyses from Sample 1 and Sample 2 reveal that the reliability and
validity of the instrument has been established regardless of the sample
diversity. Especially the results of analysis of Sample 2 corroborate the quality of
the Questionnaire for the Construct of Inferential Statistics because the
conditions of reliability and validity have been met by the instrument despite
the students in Sample 2 not fulfilling the conditions of reliability.
Conclusion
Statistics assessment has evolved in the past 40 years (Jolliffe, 2007) from
assessing students knowledge of statistical formulas to assessing students
understanding of statistical concepts. The various existing constructs to assess
students learning of statistics are largely concerned with students
understanding of the descriptive statistics. We have developed a construct to
assess students learning of the inferential statistics in the higher education
contexts and have discussed the development of this construct in earlier papers
(Krishnan & Idris, 2013a, 2013b).

58
The development of the construct of inferential statistics basically supports the

requirement to increase the number of literature in the area of students learning
and understanding of inferential statistics because studies in this area are still
scarce at present (Smith, 2008).The construct of inferential statistics can be
utilized by statistics educators to improve students understanding of the logic
of statistical investigations and the need to infer from samples to populations. It
can also aid in developing students deep and connected understanding of
inferential statistics. By identifying the different levels of students
understanding of inferential statistics, instructors can focus on the development
and improvement of students understanding of the levels in concern.
In this paper, we investigated if the reliability and validity achieved earlier is

maintained if a different sample of students were used. It was found that
although the sample of students in this study did not meet some conditions of
reliability, the reliability and validity of the instrument was fulfilled. This served
to verify the validation of the instrument and subsequently the validation of the
construct. However, this study did not investigate if gender, ethnicity or
language capability or a combination of them affected the results. We suspect
language or rather lack of it could have played a major role because a number of
questionnaires from Sample 2 were barely attempted and some students
conceded that it was because they did not know how to explain their reasoning
in English.
Lesser (2010) believes that student diversity interacts with the learning of
statistics and it is important for instructors to use student diversity as an
opportunity instead of obstacle. Lesser and Winsor (2009) also believe that
language is an important factor in students performance but found that at
present there is lack of research on statistics learning involving English
articulateness. Future possible work with respect to this paper is to investigate in
detail how students different language capabilities affect the reliability and
validity of the construct.
References
Biggs, J.B.,& Collis, K.F. (1982). Evaluating the quality of learning: The SOLO taxonomy.
New York, NY: Academic Press.
Biggs, J.B.,& Collis, K.F. (1991). Multimodal learning and the quality of intelligent
behaviour. Intelligence: Reconceptualization and measurement, 57-76.
Bond, T.G.,& Fox, C.M. (2007). Applying The Rasch Model: Fundamental Measurement in
the Human Science. Mahwah, NJ: Lawrence Erlbaum Associates.
Bude, L. (2006). Assessing students understanding of statistics. In Proceedings of the 7th
Annual Meeting of the International Conference on Teaching Statistics. Brazil: ISI.
Retrieved April 16, 2010 from
www.stat.auckland.ac.nz/~iase/publications/17/6G3_BUDE.pdf
Callingham, R. (2009). Using Rasch Measurement to Identify Cross-cultural Aspects of
Statistical Literacy.Changing Climates: Education for Sustainable Futures, 1, 1-10.
Francis, G., Kokonis, S.,& Lipson, K. (2007). Enhancing Student Understanding in
Statistical Inference-Assessing the Effectiveness of a Computer Interaction.
IASE/ISI Satellite. Retrieved February 20, 2011 from
http://iase-web.org/documents/papers/sat2007/Francis_et_al.pdf

59
Green, K. E., & Frantom, C. G. (2002). Survey Development and validation with the
Rasch Model. Paper presented at theInternational Conference on Questionnaire
Development, Evaluation, and Testing. Charleston, SC.Retrieved February 6, 2013
from http://www.jpsm.umd.edu/gdet/final_pdf_papers/green.pdf
Jolliffe, F. (2007). The Changing Brave New World Of Statistics Assessment. IASE/ISI
Satellite. Retrieved February 20, 2011 from
http://www.stat.auckland.ac.nz/~iase/publications/sat07/Jolliffe.pdf
Kaplan, J.J.,& Thorpe, J. (2010). Post Secondary and Adult Statistical Literacy: Assessing
Beyond the Classroom. In Proceedings of the Eighth International Conference on
Teaching Statistics. Netherlands: ISI.Retrieved September 2, 2011 from
http://iase-web.org/documents/papers/icots8/ICOTS8_5E3_KAPLAN.pdf
Kassim, N.A., Ismail, N.Z., Mahmud, Z., & Zainol, M.S. (2010). Measuring Students
Understanding of Statistical Concepts using Rasch Measurement. International
Journal of Innovation, Management and Technology, 1(1), 13-19.
Kataoka, V.Y., da Silva, C.B., Vendramini, C.,& Cazorla, I. (n.d.). Using Rasch Partial
Credit Model to analyze the Responses of Brazilian Undergraduate students to a
Statistics Questionnaire.
Retrieved June 1, 2014 from http://www.cerme7.univ.rzeszow.pl
Knupfer, N.N.,& McLellan, H. (2001). Descriptive Research Methodologies. In D.H.
Jonassen (Ed.), Handbook of Research for Educational Communications and
Technology (pp. 1196-1213). Mahwah, NJ: Lawrence Erlbaum Associates.
Krishnan, S., & Idris, N. (2013a).The Development of an Assessment Construct for
Inferential Statistics. Paper presented at the International Conference on Assessment
for Higher Education Across Domains and Skills (AHEADS2013). Kuala Lumpur,
Malaysia.
Krishnan, S., & Idris, N. (2013b). The Use of a Hierarchical Construct to Investigate
Students Learning of Inferential Statistics. In Proceedings of the Joint IASE/IAOS
Satellite Conference(pp. 1-8).Macao, China. August 2013.
Krishnan, S., & Idris, N. (2013c). The Use of Graphics Calculator in a Matriculation
Statistics Classroom: A Malaysian Perspective. Technology Innovations in Statistics
Education, 7(2), 1-13.
Lesser, L.M. (2010). Equity and the Increasingly Diverse Tertiary Student Population:
Challenges and Opportunities in Statistics Education. In Proceedings of the Eighth
International Conference on Teaching Statistics (pp. 1-6).Netherlands: ISI. Retrieved
December 10, 2013 from
http://iase-web.org/documents/papers/icots8/ICOTS8_3G3_LESSER.pdf
Lesser, L.M., & Winsor, M.S. (2009). English language learners in introductory statistics:
Lessons learned from an exploratory case study of two pre-service teachers.
Statistics Education Research Journal, 8(2), 5-32.
Linacre, J.M. (2002). What do Infit and Outfit, Mean-Square and Standardized Mean.
Rasch Measurement Transactions, 16(2).
Mahmud, Z. (2011). Diagnosis of Perceived Attitude, Importance, and Knowledge in
Statistics Based on Rasch Probabilistic Model. International Journal of Applied
Mathematics and Informatics, 5, 291-298.
Masters, G.N. (1982). A Rasch model for partial credit scoring. Pschometrika, 47, 149-174.
Nunnaly, J. (1978). Psychometric theory. New York, NY: McGraw Hill.
Santos, J.R.A. (1999). Cronbachs alpha: A tool for assessing the reliability of scales.
Journal of extension, 37(2), 1-5.
Smith, T.M. (2008). An Investigation into Student Understanding of Statistical Hypothesis
Testing. Doctoral dissertation. University of Maryland.
Sotos, C., Vanhoof, S., Noortgate, W.,& Onghena, P. (2009). How confident are students
in their Misconceptions about Hypothesis Tests?. Journal of Statistics Education,
17(2).

60
Watson, J.M. (1997). Assessing statistical literacy using the media. In I. Gal & J.B.
Garfield (Eds.),The Assessment Challenge in Statistics Education (pp.107-121).
Netherlands: IOS Press.
Watson, J.M.,& Callingham, R.A. (2003). Statistical literacy: A complex hierarchical
construct. Statistics Education Research Journal, 2(2), 3-46.
Watson, J.M., Kelly, B.A.,& Izard, J.F. (2005). Statistical Literacy over a Decade. Building
connections: Theory, research and practice, 1, 775-782.
Weinberg, A., Wiesner, E.,& Pfaff, T.J. (2010). Using Informal Inferential Reasoning to
Develop Formal Concepts : Analyzing an Activity. Journal of Statistics Education,
18(2), 1-23.

61

Vol. 4, No.1, pp. 61-75, April 2014
Influence of Head Teachers Management Styles

on Teacher Motivation in Selected Senior High
Schools in the Sunyani Municipality of Ghana
Magdalene Brown
Anthony Akwesi Owusu
Department of Arts and Social Sciences Education
University of Cape Coast
Ghana
Abstract. Senior High School administration in Ghana is bedeviled

with a lot of problems most of which emanate from deficiencies in
management styles of heads of these institutions. There have been times
when teachers had to stage open protests to register their displeasure
about head teachers management styles which they often described as
administrative and managerial incompetency. This situation has often
led to low morale among teachers and therefore this study focuses on
finding out the effects of head teachers management styles on teachers
motivation in SHSs in Ghana. The study also attempts to look at and
explain how head teachers informal relationship with teachers serves as
a motivation for teachers to work to achieve institutional goals. The
study sampled 100 senior high school teachers and 10 head teachers in
the Sunyani Metropolis. These were sampled using the quota and
simple random sampling procedures. The study, among other things,
revealed that most teachers see their heads as bosses and not as friends.
Again, it was revealed that though teachers were involved in decision
making, the actual setting of objectives for a school were left in the
hands of the heads and the school management team. It was therefore
recommended for head teachers to adopt the Management by Walking
About and Management by Objective so as to get closer to their teachers
in the running of schools.
Key words: Management by Objective; Management by Walking About;

Motivation
Introduction
In every organisation for which the school is not an exception, management is
expected to produce results. These results do not just happen overnight. They
demand great efforts by the leaders who in turn, are to spearhead the affairs of
the organisation. A high degree of workplace spirituality and spiritual
leadership, as a driver of organizational commitment and productivity, is
important to enhancing organizational performance (Fry & Matherly, 2006).The
person at the helm of affairs is usually the manager (Ekeland, 2005). The
2014 The authors and IJLTER.ORG. All rights reserved

62
managers of various organisations were seen as not sociable and distant people
who are seen once in a while either at a meeting or for specific programmes.
Also, the norm was that the communication process was a downward one
whereby decisions are taken by managers and pushed down to the subordinates.
However, for the objective of an organisation to be achieved, it demands the
collective efforts of both managers and employees. This case is not different
when it comes to school management. If the management of any school will
achieve results, the efforts of the teachers must be appreciated. For managers to
achieve their organizational targets, services of people are imperative which in
the school setting will be teachers (Thomson, 1998). Managers can only use this
tool (people) effectively when they instill in them a sense of commitment and the
desire to accomplish organizational goals. Again, if this tool can be used well,
their efforts should be controlled and coordinated toward goal accomplishment.
The manager in all these should give subordinates the opportunity to increase
their skills and abilities in contributing to achieving the organizations
aspirations. The individuals style will be use based on a combination of their
beliefs, values and preferences, as well as the organizational culture and norms
which will encourage some styles and discourage others (Almansour, 2012).
Teacher Motivation
Aacha, (2010) in a study alluded to the fact that teacher motivation has become
an important issue since teachers preoccupation is to transmit skills, knowledge
and attitudes to learners. Teachers who are satisfied with their job tend to give
off their best and can go a long way to influence students performance (Mertler,
1992). Motivation guide peoples actions and behaviours toward achievement of
some goals (Analoui, 2000). In the world work and the school setting, motivation
can be perceived in two distinct contexts: intrinsic and extrinsic motivation
(Sansone & Harackiewicz, 2000). The former emanates from within the
individual and positively influences their behavior, achievement (Ryan & Deci,
2000). The latter on the contrary, comes as result of influences from the external
environment which acts as stimulus. Thousands of studies have been conducted
following Thorndikes (1911). On such study was the Emery Air Freight study
carried out by Hamner and Hamner (1976) and Komaki (1982) on how behaviors
change by the manipulation of extrinsic factors.
With intrinsic motivation, one performs an act for its own sake rather than being
urged by an external factor. The issue of intrinsic motivation is a vital concept
which has been amply dealt by White (1959), Maslow (1943) and Alderfer (1969).
A research paper on extrinsic and intrinsic motivation of teachers that was
measured in terms of teaching work satisfaction alluded to a number of intrinsic
motivational variables. Some of these included satisfaction derived from
teaching, setbacks in teaching, the competition in teaching, recognition, career
advancement and, control over others. The paper also viewed the extrinsic
motivation of teachers in terms of salaries, free accommodation, free meals,
weekly duty allowances and extra teaching load allowances, advance payments
in case of financial problems, leave of absence and free medical care among
others.

63
Teachers, per their job make or unmake a society and this is why their work
output is of great interest to everybody in society. Oxford Advanced Learners
Dictionary defines performance as - the act or process of carrying out something
or execution of an action or a repetitive act or fulfillment or implementation
(Hornby, 2000). In this respect, teacher performance connotes the teachers role
of teaching students in class and outside the class. The key aspects of teaching
involve the use of instructional materials, teaching methods, regular assessment
of students, making lesson plans, assessment of pupils, conduct of fieldwork,
teachers participation in sports, attending school assembly and guidance and
counseling. Therefore, teacher job performance is the teachers ability to
integrate the experience, teaching methods, instructional materials, knowledge
and skills in delivering subject matter to students in and outside the classroom.
Teacher performance in this study was measured by regular and early reporting
at school, participation in extra-curricular activities, supervision of school
activities, adequate teaching preparation (schemes of work, lesson plans),
marking and general punctuality among others.
Teaching is a mass occupation, which accounts for one-half to two-thirds of

public sector employment in most developing countries (Bennell, 2004). While
other professions (engineers, medical doctors and lawyers) enjoy a high degree
of self-regulation and are successful in maintaining high barriers to entry in
terms of qualification requirements and registration, teachers on the other hand,
tend to have weak, state-dominated professional organizations with several
trade unions. In addition, public sector recruitment freezes in many countries
during the 1990s have seriously constrained the growth of the number of
teachers in government or government-aided schools. Although such events are
expected to have affected the morale of teachers to perform, detailed studies
examining the effect of motivation on performance of teachers are still scanty.
Available evidence, however, reveals that the teaching force has expanded
rapidly in only a relatively few countries (most notably Bangladesh, Malawi,
Ethiopia, Eritrea, Mozambique and Uganda). Bennell (2004) indicates that
teaching has become employment of the last resort among university graduates
and secondary school leavers. About one-half of junior secondary school leavers
in Malawi and Tanzania who finished school in 1990 were employed as teachers
in 2001. The corresponding figure for Uganda is a staggering 81 per cent (Al-
Samarrai & Bennell, 2003). Consequently, teachers often lack a strong, long term
commitment to teaching as a vocation. Besides, the status and pay of primary
school teachers compared to secondary school teachers is generally much lower
in developing countries. Thus, in the absence of alternative employment
opportunities, becoming a secondary school teacher is the main avenue for social
and economic advancement for the most able primary school teachers. This has
had important implications for intrinsic motivation of primary school teachers.
Literature have reviewed various management styles adopted by management

but this study sought to look at how head teachers as managers of schools are
able to adopt the management by objective and management by walking about
style in their management practices and its effects on teacher motivation.
Management by walking/wandering about, according to Peters and Waterman

64
(1982) is an unstructured approach to hands on, direct participation by the

managers in the work-related affairs of their subordinates, in contrast to rigid
and distant management.
For teachers to be motivated extrinsically to give off their best, the heads
management style contribute immensely (Holten, Dent & Rabbett, 2009; Aacha,
2010). The issue of management style and employee performance has gained so
much ground especially the western world. Many researches conducted in the
field of Business world have proven that the management styles of managers
greatly influence the motivation of subordinates. However, when it comes to
the teaching field it remains unclear how the styles of management of heads
have effects on teacher motivation. The output of the Ghanaian teacher is said to
be going down in recent years. Most of these teachers are now performing
below expectation just because people claim that their incomes are low but this
problem could be due to so many factors and one such factor could be the
management styles adopted by their heads. Management styles which are
adopted by head teachers usually affect the performance of teachers in that they
either positively or negatively on teacher motivation to give off their best. Also
most research work on management styles based on the popular management
styles as the democratic, autocractic, laissez-faire etc. but much has not been
done when it comes to the management by objective and management by
walking about.
If head teachers as school managers, sit jointly together with their teacher to set
specific institutional objectives to be accomplished within a certain specific time
frame, all players buy into the vision, hands are put on deck, objectives are
attained. The emphasis of this management style is that goals jointly set by
teachers and heads within the school boosts teacher morale. Also, the informal
relationship of head teachers informal towards their teachers serves as a
motivation for teachers to work to achieve institutional goals. The normal style
of the heads is seeing them in their offices performing their role. The teachers
see their heads only on formal grounds like meetings or being summoned to see
the head in his or her office. Again, what normally happens is that objectives are
set by the management board and the heads normally bring them to staff
meetings for discussion. The question therefore is are the heads aware there is
a management technique like these and are the head teachers actually using the
management style of walking/wandering around or management by walking
about and what effects have these styles on teacher motivation?. Also do the
head teachers actually involve their teachers in setting objectives? To help
unfold these issues this research is aimed at finding the effects of the heads
management style on teacher motivation. The purpose of the study was to
identify the management styles of head teachers and their effects on teacher
motivation.
The research will therefore bring to light the importance of these two vital
management styles that can be adopted by managers of schools to increase
teacher motivation. The study focused only Senior High Schools in Brong Ahafo
and not the entire country; hence the generalization of the result might not be

65
easy since the condition pertaining to one region might not be the same for
another region. The study was confined to only Brong Ahafo Region and not the
entire country. Again, the study did not address other management styles
adopted by managers in their day-to-day activities.
Research Questions
To what extent does head teachers management style (MBO and MBWA)
influence teachers motivation to perform?
Research Hypotheses
H0: Head teachers management style has no significant effect on teacher
motivation to perform.
Overview of Literature
This chapter sought to review literature on the two management styles

(Management by Objective and Management by walking/wandering around
and motivation.
Management by objective (MBO)

Management By Objective theory was introduced to the business science in the
mid of 1950s by Peter Drucker, a prominent management scientist. Druckers
(1954) concept of the MBOs introduction was the apparent importance of
businesses clear objectives to their profitability, productivity, share market
increase and concurring reputation. According to Drucker (1954), the procedure
of objectives setting and progress monitoring are determining factors towards
the function of organizations, thus these factor should permeate the entire
organization, from top to bottom. To achieve this, it is important for top level
management to jointly set organizational targets with their subordinates.
Management by objective is a system whereby the superior and subordinates of
an organisation jointly identify its common goals, define individuals
responsibility in terms of expected results. It makes a demand on every manager
to set targets to be achieved in the future and inspires subordinates to
persistently ask what can be done (Thomson, 1998). He went further to assert
that organizations do not exist for their own sake but for a purpose. And to
achieve this purpose, higher level managers should set attainable and specific
objectives by involving subordinates at all levels of decision making in attaining
targets. Organizations which do not adopt the MBO strategy normally risk
having a downward decision-making approach. That is, the goals are set by top
management and are handed down to subordinates for implementation. The
lack of participation by subordinates which is usually manifested by non-
commitment in the decision-making process hampers the full realization of
organizational goals and objects. According to Thomson this does not encourage
commitment on the part of the subordinates. MBO gained attention because it
focuses on objectives and results which a manager wants to achieve in specific
time as well as focusing on participative management

66
MBO is a result centered managerial approach for the effective utilization of

material, physical and human resources of the organization. MBO tries to
combine long term goals of organization with short term goals. MBO does not
only focus on goals but also on effective performance. It also focuses of the
participation of employees in goal setting process. In the school Management by
Objective could be applied. The head can adopt the participative management
approach where formulation of objective becomes a shared activity between the
head and the teachers. This could be exhibited in the nature of decision making
in the schools. Staff members involved decision making generally develop a
high level of satisfaction. Teachers must be recognised and respected by the
administration for their expertise (Conley, Schmidle, & Shedd, 1988). Teachers
are empowered to act as leaders in concert with the principal. Liontos (1993)
suggested that a shared decision-making strategy has the potency to improve
the quality of decisions thereby increasing decision acceptance rates, boost staff
morale , increase staff efficiency, staff commitment and teamwork; build trust
among staff; help staff acquire new skills and increase overall school
effectiveness.
Management by Wandering About/ walking Around

The term Management by Walking (MBWA) was defined by Peters and
Waterman (1982) as a style of management whereby managers wander around
in an unstructured manner at the workplace randomly with a view to checking
with employees, equipment, about the status of ongoing work. This definition
emphasizes wandering as a random movement in a workplace rather than a
carefully plan visits by managers to employees at a more systematic and pre-
arranged times and venue. The advantage with this style of management is that
a manager is more likely to increase productivity and total quality in the
management of the organization in contrast to staying glued in a specific office
and waiting for employees, or the delivery of feedback reports on what goes on
in the organization.
This term was first used by executives of Hewlett-Packard company, in the 1970s
(Mears, 2009). MBWA was used by the above company executives in boosting
morale within the company. Following success in its usage, the style was
embedded into the culture of the organization. However, Hewlett-Packard
company was not the only company in those days that adopted this style of
management where managers made unstructured visits to subordinates in the
workplace. For instance, Tom Peters and Robert H. Waterman have earlier used
this term in their book: In Search of Excellence: lessons from America's best-run
companies in 1982. According to Hinners (2009) management by walking about
connotes a sense of purposeful random visit by a manager to learn at first hand,
the working conditions which have beneficial and fruitful consequences for
organizational growth. He explained how he adopted the MBWA in his
managerial role when he was appointed the director for Smithsonian National
Air and Space Museum. He commented on how MBWA has served as an eye-
opener in his career. He explained that by walking around the working sites
and asking questions he was able to get vital information he believes would
never had come at a formal meeting. Roaming about at the work site helps the

67
manager to learn on a daily basis procedures and processes in the organization.

He explained that subordinates see you (manager) as a superior; they will be
more willing to open up to tell you (the manager) the happenings and problems
within the organization before some of them become problems. Furthermore, as
subordinates gets to know their boss better, they tend trust him/her more.
Another advantage of adopting MBWA is that subordinates will be ready to
share more vital information with the manager and this can help break down
barriers to communication. Again, visits by the manager to different
departments can help him/her discover and fix problems earlier in efficient
manner. According to Hinners (2009), communication in MBWA thus becomes
spontaneous and symbiotic and also gives room for on-the-job mentoring. These
features described earlier differentiate MBWA from other management styles.
This approach guarantees a fantastic multiplier effect as far as information
diffusion is concerned. Word gets around rapidly and potentially makes people
(workers or subordinates) feel comfortable coming unannounced to offices of
their bosses (heads) to get some tidbit off their chest. Another dimension of this
approach is the use of norm, a most unusual leadership style which makes use of
profound technical skills, and an ability to analyze audience using appropriate
humor and presentation skills. It enables subordinates to put on a can do spirit
attitude: after all, nothing is impossible and challenging what we do.
It is a wander why many managers do not adopt this style (ie MBWA).
Nonetheless, MBWA is not a panacea to all managerial problems. To make
MBWA successful, other management tools and styles must be adopted
concurrently. As is true of any individual management tool, it must be
augmented and complemented by a host of other proven techniques.
Management by Walking About can easily be adopted by heads of schools
where they could pay random visit to classrooms to interact with the teachers
and be able to get hands on information about the activities that go on in the
classrooms. The two approaches are expected, when used properly to motivate
teachers to function effectively and efficiently as teachers. Figure 1 gives a
diagrammatic representation of the two styles opined by Oates (1977) and
Hinners (2009).
Figure 1. The process and impact of MBO and MBWA in an organization (Author
derived)

68
Methodology
Research Design
The cross sectional survey design of the descriptive design was adopted for this
study. The cross sectional survey design was employed in carrying out the
study. This design was adopted because it affords the researcher the
opportunity to observe, assess and describe the extent to which the management
styles of head teachers in the Senior High Schools in the Brong Ahafo Region
motivate the teachers to perform.
Population of the Study

The population comprised 10 senior high school head teachers and 633 teachers
in the Sunyani Municipality. The sample for this study comprised 100 teachers
who were randomly selected from all the public senior high schools in the
Sunyani Municipality. The census survey was used to select 10 head teachers
from all the selected senior high schools in the Sunyani municipality.
Instrument for Data Collection

According to Gay (1992), all research studies involve data collection with the
help of research instruments. For this reason the researchers decided to use
questionnaire for both the head teachers and the teachers. The questionnaire was
divided into three sections. Section A was the introductory section containing an
item on teachers education zones. Section B comprised 7 items on the relevance
of MBO and teacher motivation, while section C had 12 items on the benefits of
MBO to school management. The items in the questionnaire were both closed
ended and open ended questions.
Validity and Reliability of Instrument

The instrument was given to expert to ascertain its content and face validity.
Changes were effected to improve upon the instrument. The instrument was
pilot tested thereafter. The purpose of the pilot testing according to Brown
(2012) was to gain an insight into the relative strengths and weakness of the
research instrument in order to make possible improvement prior to the main
study. One hundred and ten (110) questionnaires were administered to both
head teachers and teachers in the Sunyani Municipality. This area was chosen
because the researchers believe that the head teachers and teachers in this
district bore similar characteristics in terms of qualification and experience with
other teachers. The data gathered were analysed and the Cronbachs alpha was
established for each of the items. Both questionnaires had a Cronbachs alpha
value of 0.80 which was respectably reliable in every sense.
Results and Discussion
Data Analysis
The outcome of the study has been presented and discussed in the following
section. The study examined the extent to which head teachers management
style (MBO and MBWA) influenced teachers motivation to perform and
whether head teachers management style had any significant effect on teacher
motivation to perform.

69
Table 1: Management by Objective
Statement Strongly Agree Disagree Strongly Mean

Agree No. (%) No. (%) Disagree
No. (%) No. (%)
My head teacher brings specific 13(13) 72(72) 15(15) 0(0) 2.98
goals and measures for discussion
during staff meetings and this
motivates me to give off my best
I am liable for the 55(55) 17(17) 15(15) 13(13) 3.22

accomplishment of goals
My head rewards for the 0(0) 17(17) 28(28) 55(55) 1.62

accomplishment of goals and this
serves as a motivation for me
I have been involved in 0(0) 13(13) 87(87) 0(0) 2.13

establishing disciplinary policies
and so I am motivated
I have been involved in planning 0(0) 13(13) 32(32) 55(55) 1.58

structural facilities for the school
and this is an incentive to work
hard.
I have been involved in planning 27(27) 13(13) 32(32) 28(28) 2.39

new projects for the school and so
I give off my best
I have been involved in resolving 44(44) 41(41) 15(15) 0(0) 3.29

staff disputes and this encourages
me to give off my best
Total 2.46
It could be deduced from table 1 that the heads bring specific goals to staff
meeting for discussion because majority, 72(72%) agreed to that statements.
Again it could be observed that 55(55%) of the responded agreed that their
heads hold them responsible for the accomplishment of those goals brought for
discussion. However, issues concerned with disciplinary policies, a clear
majority disagreed that they are involved in such decisions and this clearly
discouraged a lot of them. Also, on the issues on planning of structural facilities
and new projects, a greater number of the respondents disagreed with those
issues. This is a clear indication that most of the decisions taking in the schools
are solely taken by the head teachers. Again, on the issue of whether the
respondents were involved in planning disciplinary policies, majority, 87(87%)
indicated that they were not involved. Druckers (1954) who introduced the
concept of Management By Objective was of the view that for an organisation
such as the school to be able to achieve its objectives successfully it is important
that top level management (head teachers) jointly set organisational objective

70
with their subordinates (teachers). Liontos (1993) stated that a shared decision-
making strategy has the potency to improve the quality of decisions thereby
increasing decision acceptance rates, boost staff morale , increase staff efficiency,
staff commitment and teamwork; build trust among staff; help staff acquire new
skills and increase overall school effectiveness.
Table 2: Management by walking about

Statement Strongly Agree Disagree Strongly Mean
Agree No. (%) No. (%) Disagree
No. (%) No. (%)
My head frequently visits my 44(44) 13(13) 43(43) 0(0) 3.01
class
I feel intimidated when I see my 0(0) 31(31) 41(41) 28(28) 2.03

head around my class
I feel relaxed when my head 30(30) 42(42) 0(0) 28(28) 3.02

comes to my class to visit me
I am able to share problems with 27(27) 30(30) 43(43) 0(0) 2.84

my head more when he/she
comes to visit me in my class
My head is often curious to know 0(0) 30(30) 70(70) 0(0) 2.30

how some topics are taught when
he/she visits my class.
My head often uses informal visits 0(0) 44(44) 56(56) 0(0) 2.88
to discuss formal issues with me
I am able to share personal issues 45(45) 26(26) 29(29) 0(0) 3.16

during informal discussions with
my head
My heads informal visits 24(24) 41(41) 29(29) 6(6) 2.83

approach reduces the
cumbersome bureaucratic
procedures.
Total 2.75
Out of the 100 teachers employed in this study only 44(44%) indicated that their
heads frequently visit them in their class. Again, on the issue of whether the
head is often curious to know how some topics are taught when they visit their
classes, 70(70%) respondents disagreed. However, on the issue of whether the
heads informal visits reduces the cumbersome bureaucratic procedures,
majority, 65 (65%) of the respondents agreed. The result in Table 2 indicates that
heads were not so much involved in management by walking about.
Management by walking about involves the manager visiting work sites
frequently, discussing issues with subordinates. According to Hinners (2009),
visits by the manager to different departments can help him/her discover and

71
fix problems earlier in efficient manner. According to Hinners (2009) is of the

view that communication in MBWA thus becomes spontaneous and symbiotic
and also gives room for on-the-job mentoring. This means that if head teachers
frequently visits their teachers in their classroom they will be able identify
problems and help to give coaching when it becomes necessary.
From the results, it is clear that study examined the extent to which head
teachers management style (MBO and MBWA) influenced teachers motivation
to perform and whether head teachers management style had any significant
effect on teacher motivation to perform.
Table 3: Correlation between head teachers management style and teacher motivation
to perform
Variables N Mean SD R P- value

Management style 100 15.92 3.01 .263 0.008
of head teachers
Teacher 100 21.77 4.97

motivation to
perform
*P<0.05
This test was conducted to determine the relationship between head

teachers management style and teacher motivation to perform. Testing was
done at 0.05 level of significance (95% confidence level). The hypothesis was a
non-directional one (2-tailed). From Table 3, management style of head
teachers, r (100) = .263, P = .008, given that = 0.05 and P = 0.23. Since P < ,
the result is statistically significant. The null hypothesis is therefore rejected
and a conclusion made that, there is s significant relationship between
management style of head teachers and teacher motivation to perform.
This outcome confirms the suggestion by Liontos (1993) that the management
style of head teachers has the ability to improve the quality of decisions;
increase the decisions acceptance rate and provide avenues for learning new
skills.; inspire staff, instill commitment and teamwork; build trust among staff;
and increase overall school effectiveness.
Head teachers
Five head teachers were involved in this study. On the issue of whether the
heads bring specific goals to staff meeting for discussion, all the five heads
responded in the affirmative. Again, on the issue of whether they hold teachers
responsible for the accomplishment of goal, 3 out of the 5 respondents indicated
that they do hold the teachers responsible because in their view if one is
involved in the decision making process then that person should be held
responsible for its accomplishment. When asked whether they involve their
teachers in the planning of structural facilities, and designing of new
disciplinary policies, 2 indicated that those areas are managerial in nature and
has got nothing to do with teachers, 2 also indicated that disciplinary policies are
mostly designed by the central government and not at the school level.

72
On the issue of whether the heads frequently visits their heads, 3 indicated that
they sometimes visit their teachers in their classes. The 2 indicated that they had
more office work to perform and this makes it difficult for them to be visiting
their teachers when they are teaching.
Conclusions
There is no doubt that the management style adopted by head teachers either
motivates teachers to perform or demotivate them in the discharge of their
duties. One cannot imagine how chaotic society would have been if there were
no leaders to manage affairs of organisations. People would have been without
mission and direction and development would not have been thought of. In the
school setting, head teachers are seen as managers and their management style
can greatly influence the performance of their teachers. The two management
styles: management by objective and management by walking about are often
adopted by some heads educational institutions even though these heads are not
in a position to christen them as this study has done through review of literature.
Head teachers do not use management by walking about as a style in the
discharge of their duties. The question now is: Are the head teachers even aware
there is a style of that nature and its effective implementation could yield
positive outcome in the school setting? The conclusions drawn from this study
indicated that management by objective as a management style was adopted by
the headmasters.
Irrespective of the form the MBO takes within the school set up, it is basically a
style that helps to direct head teachers attention toward results and force
teachers of a school to commit themselves to achieving specific goals and
facilitating their thinking of future needs and the objectives to set. Again, the
MBO approach can provide the head teachers with greater measures of the tools
they need to make in the best interest of the schools progress. The head teacher,
as a manager can gain maximum cooperation and desire to contribute from
subordinates (teachers) by making them to feel that the objectives they are
working toward were not taking by some people and just handed to them but
are really part because they played a part in setting them, and also giving
teachers a sense of belonging in the school setting by making clear how their
objectives fit into the overall goal of the school. And, the approach helps to inject
life into the school that comes with the energy produced as stakeholders strive to
achieve its goals. Finally, it helps heads in the school setting gain better control
and coordination toward goal attainment by having a clearer understanding of
who is doing what and how the parts all fit together. It also helps when the
heads have teachers who are more likely to control and coordinate their own
activities because they know what will help and what will hinder their goal
achievement.
MBO easily is often misused in that what is supposed to be a system that allows
for dialogue and growth between head teachers and teachers with a view to
achieving results often reduces into a school system in which the heads
sometimes put too much pressure on the teachers to produce results forgetting

73
that school success comes as a result of a lot of factors. Sometimes, heads with
good intentions are prone to misuse MBO due to their lack of human skills or
knowledge. Finally, many heads fail to see that MBO is one out of thousand
management styles that could be adopted in solving management problems in
our schools. Heads of SHS must in their attempt to use this approach guard
against some of the pitfalls that have created rather ravaging effects in the use of
MBO. This explains why this study advocates for a blend of the styles in order to
tap their full positive effects in our schools.
Recommendations
1. It is recommended that heads of schools be given in-service training to
educate them on how to blend the use of the two management styles to
motivate their teachers to perform. In this regard, the situational
leadership style based on the interaction among the dimensions of
relationship behavior and task behavior, as well as follower readiness or
maturity for performing a certain task is strongly recommended during
such in-service training sessions. Potential managers of schools must be
made aware that their teachers are the most critical factor in ensuring
leadership success in their schools.
2. Again, heads of the various schools should be able to delegate some of
their duties to other assistant heads and other teachers to give them
greater opportunities to walk around the schools to see for themselves
what really goes on in the classrooms. Heads must first see themselves as
first among their equals and fully give teachers the respect they
deserve. By definition, delegation is the transfer of authority to make
decisions and complete specific tasks. Learning how to delegate is one of
the most important skills for managers and leaders to possess. Strong
delegation techniques can help managers save time, motivate people,
and train people, as well as these techniques can enable managers to take
on new opportunities. However, the lack of delegation practices often
leaves people frustrated, unmotivated, and under-trained, while the
manager remains overworked. Delegation is a skill that enables
managers to achieve more without burning themselves out. This is one
single surest way of motivating or driving a person to do something.
Much of the driven are the thought of a potential reward, or a
consequence of not doing something.
3. The main concepts in delegation such as authority, responsibility, and
accountability for the task changes should be hammered when a head
decides to delegate a task. Before delegating a task, it is important to
understand how it affects these three concepts. Authority is the power
given to a person or group of people to act and make decisions within
designated boundaries. When delegating a task, the authority is shared
between the head teacher and the teacher receiving the delegation. Also,
responsibility refers to the act of carrying out the task. When delegating a
task, the head teacher and the teacher receiving the delegation share the
responsibility of completing the work. The head teacher has the
responsibility of providing instructions on what work needs to be done,

74
while the teacher receiving the delegation is responsible for figuring out
how the task should be completed. Accountability is the act of being
liable for a persons actions and decisions. During delegation of a task,
the accountability of the task transfers from the head teacher to the
teacher receiving the delegation and actually completing the work. Any
positive or negative consequences associated with the teachers
performance are ultimately the responsibility of the head teacher.
Understanding these basic concepts would equip heads with some skills
in delegating tasks in the school effectively.
4. Efforts should be made to continuously involve the teachers in the
decision making process in the school to boost their motivation to
perform. Heads must consciously tell their teachers after making the
decision and announce it to the staff with a clear direction. Once that has
been ensured, the head should attempt to gain commitment of the staff
by "selling" the positive aspects of the decision. The heads should then
invite input into the decision while retaining authority to make the final
decision themselves. Finally, invite teachers to make inputs into the
decision with. In the process, the head should consider themselves as
having a voice equal to those of the subordinates.
References
Aacha, M. (2010). Motivation and the performance of primary school teachers in Uganda:
A case of Kimaanya-Kyabakuza division, Makasa district. Unpublished dissertation,
Makerere university, Kampala.
Alderfer, C. P. (1969). An empirical test of a new theory of human needs. Organisational
Behavior and Human Performance, 4, 143-175.
Almansour, Y. M. (2012). The relationship between leadership styles and motivation of
managers conceptual framework. International Refereed Research Journal, 3(1), 1
6.
Al-Samarrai, S. & Bennell, P. (2003). Where Has All The Education Gone On Africa?
Employment Outcomes Among Secondary School And University Leavers. Brighton:
IDS/KSD.
Analoui, F (2000). What motivates senior managers? The case of Romania. Journal of
Managerial Psychology, 15(4), 324-340.
Armstrong, M. (2004). How to be an even better manager. London: Kogan Page Press.
Bennell, P (2004). Teacher motivation and incentives in Sub-Saharan Africa and Asia.
Brighton: Knowledge and Skills for Development.
Brown, M. (2012). Motivational factors as determinant of performance among business
Students in the SHSs in the Cape Coast metropolis, Unpublished masters thesis,
University of Cape Coast, Cape Coast.
Conley, S. C., Schmidle, T., & Shedd, J. B. (1988). Teacher participation in the
Management of school systems. Teachers College Record, 90, 259-280.
Drucker, P. (1954). The practice of management. New York: Harper & Row.
Ekeland, T., P. (2005). The relationship among affective organisational commitment,
transformational leadership style, and unit organizational effectiveness within
the corps of cadets at Texas A & M University. Unpublished Doctorate
Dissertation, Texas A & M University, Texas.
Fry, L. & Matherly, L. (2006). Workplace spirituality, spiritual leadership, and
performance excellence. In S. Roglberg & C. Reeve (Eds.), The Encyclopedia of

75
Industrial and Organisational Psychology. San Francisco: Sage Publishing.

Gay, L. R. (1992). Educational research. New York, NY: Maxwell Macmillan International.
Hamner, W. C., & Hamner, E. P. (1976). Behavior modification and the bottom line.
Organizational Dynamics, 4(4), 3-21.
Hertzberg, F. (1966). Work and the nature of man. Cleveland: World Publishing.
Hornby, A., S. (2000). Oxford Advanced Learners Dictionary of Current English. Oxford
University Press.
Hinners, N. W. (2009). Management by wandering around. NASAs Mars Exploration
Programme Analysis Group, University of Colorado Aerospace Engineering
Sciences Department. Retrieved November 16, 2013 from
http://www.pmhut.com/
Holton, V., Dent, F., & Rabbetts, J. (2009). Motivation and employee engagement in the 21st
century: A survey of management views. An Ashridge Report.
Humble, J. (1968). Improving business results. New York: McGraw-Hill.
Humble, J. (1970). Management by objectives in action. New York: McGraw-Hill.
Komaki, J. (1982). Managerial effectiveness: Potential contributions of the behavioral
approach. Journal of Organizational Behavior Management, 3, 71-83.
Liontos, L. B. (1993). Shared decision-making. Eugene, OR: Oregon School Study
Council, University of Oregon, OSSC Bulletin Series.
Maslow, A. H. (1943). A theory of human motivation. Psychological Review, 50, 370-396.
McGregor, D. (1966). Leadership and motivation. Cambridge, MA: M.I.T. Press.
Mears, M. (2009). Leadership Elements: A guide to building trust , p.51, Google Books link:
BooksG-TOC-51
Mertler, H. (1992), Value make the Company. Harvard Business Review.
Odiorne, G. (1970). Management by objectives. New York: Pitman.
Peters, T. & Waterman, R. (1982). In Search of Excellence. New York: Random House.
Reddin, W.J. (1971). Managerial effectiveness. New York: McGraw-Hill.
Ryan, R. M., & Deci, E. L. (2000). When rewards compete with nature: the undermining
of intrinsic motivation and self-regulation. In C. Sansone & J. M. Harackiewicz
(Eds.), Intrinsic and extrinsic motivation: The search for optimal motivation and
performance (pp. 14-54). San Francisco: Academic Press.
Sansone, C., & Harackiewicz, J. M. (2000). Intrinsic and extrinsic motivation: The search
for optimal motivation and performance. San Diego: Academic Press.
Thorndike, E. L. (1911). Animal intelligence. New York: Macmillan.
White, R. (1959). Motivation reconsidered: The concept of competence. Psychological
Review, 66, 297-333.

61

Vol. 4, No.1, pp. 61-74, April 2014
Comparison and Properties of Correlational and

Agreement Methods for Determining Whether or Not to
Report Subtest Scores
Oksana Babenko, PhD
W. Todd Rogers, PhD
University of Alberta
Edmonton, Canada
Abstract. Large-scale testing agencies often report subtest scores in addition to reporting
the total test score. But is there evidence that subtests reveal differences in student
performances? Three methods for determining whether subscore reporting is warranted
were examined and evaluated using large-scale data as well as samples of various sizes
for Reading and Mathematics assessments. Results revealed that subtests did not differ
among themselves and added no value over the total test. The method statistics were
determined to be accurate and precise estimators of the population parameters.
Implications for subscore reporting are discussed.
Keywords: subscore reporting; accuracy; precision; large-scale assessment
Introduction
Results from a large-scale achievement test can be reported in the form of the total test score
and, if justified, as a series of subtest scores together with the total test score. The common
practice is to report the total score as a summary of achievement of the total domain tested.
However, large-scale testing agencies have increasingly adopted the practice of reporting
subtest scores in addition to reporting the total test score because of the potential diagnostic
value of subtest scores (Wainer, Sheehan, & Wang, 2000; Tate, 2004; Sinharay, Haberman, &
Puhan, 2007; Yao & Boughton, 2007; Sinharay, Puhan, & Haberman, 2009; Sinharay, 2010). Part
of the argument put forward to support subscore reporting is based on the fact that the items
comprising a test are referenced to a curriculum that is multidimensional in nature, with each
dimension characterized by specific content and/or cognitive skills. For example, items on a
Mathematics achievement test can be referenced to (a) content areas, such as number sense and
numeration, measurement, geometry and spatial sense, patterning and algebra, and data
management and probability, and/or (b) cognitive skills, such as knowledge and
understanding, application, and problem solving. In test development, the table of
specifications serves to ensure that the test reflects the multidimensionality of the curriculum.
However, what needs to be recognized is that there must be evidence that the variables or skills
62
measured by the subtests are indeed sufficiently distinct to warrant reporting scores from the
subtests. Additionally, while the number of test items typically reflects the proportional
weighting given to each cell within the table of specifications, the total number of items
included in the test is limited by the amount of available test administration time.
Consequently, more often than not the number of items for each dimension in the table of
specifications is not sufficient to achieve a high degree of reliability or a low error of
measurement.
Despite these cautions, officials at large-scale assessment agencies still want to report subtest
scores even though no deliberate attempt was made to ensure that (a) the variables (e.g.,
number sense and numeration) assessed by subtests are distinct and not highly related, and (b)
there is a sufficient number of items for each subtest to ensure high reliability. The evidence that
is usually used to determine if the variables are sufficiently distinct is the correlation among the
subtest scores whereas the internal consistency of the items in each subtest provides evidence of
subscore reliability. What is desired are low subtest correlations and high subtest internal
consistencies (American Educational Research Association, American Psychological
Association, & National Council on Measurement and Evaluation, 1999; Wainer, Vevea,
Camacho, Reeve, et al., 2001; Tate, 2004).
Various methods for determining whether subtest scores are distinct and/or add value over
and above the total test score have been developed. These methods include the agreement
method (Kelley, 1923; see also Gulliksen, 1951; Lord & Novick, 1968; Ryan, 2003; Haladyna &
Kramer, 2004), correlations corrected for attenuation due to unreliability of the measures
(McPeek, Altman, Wallmark, & Wingersky, 1976; Harris & Hanson, 1991; Haladyna & Kramer,
2004), factor analytic method (McPeek et al., 1976; Grandy, 1992), statistical model fit (Harris &
Hanson, 1991), and, in the case of determining only whether a subtest has value over the total
test, the proportional reduction of the mean squared error (Haberman, 2005, 2008; Sinharay,
Haberman, & Puhan, 2007). Three of these methods were considered in the present study:
Kelleys agreement method (KR; Kelley, 1923), correlations corrected for attenuation ( c jk ;
McPeek, et al., 1976), and the proportional reduction of the mean squared error (PRMSE;
Haberman, 2005; Sinharay, Haberman, & Puhan, 2007).
The agreement method takes into account the actual differences between observed scores on
subtests j and k expressed in the same score metric (Kelley, 1923; Gulliksen, 1951; Lord &
Novick, 1968; Ryan, 2003; Haladyna & Kramer, 2004). Working with z-scores ( 0 ; 1 ) or
scores in some other standardized metric to remove the effects of different means and standard
deviations of subtests j and k, the difference, d i , between two standard scores for student i is
given by:
di zij zik ,
where zij is the observed standard score of student i on subtest j, and zik is the observed
standard score of student i on subtest k. If the estimated standard error of the difference for a
given student, 2 jj kk , where jj and kk are the reliabilities of subtests j and k, and the

63
estimated standard deviation of the obtained differences for the group of students, 2 2 jk ,
where jk is the correlation between the scores on subtests j and k, are close in value, then the
obtained differences are no greater than what would be expected by chance (Kelley, 1923, p.
329). In order to determine directly the percentages of students with differences beyond what
would be expected by chance, Kelley computed the proportion of cases in excess of the chance
as a function of the ratio:
2 jj kk
KR (Kelley, 1923, p. 330).
2 2 jk
Kelley (1923) illustrated his agreement method with the eight subtests of the Stanford
Achievement Test Battery and found 10% to 44% of the students had differences beyond chance
for 36 pairs of subtests (p. 331). Values of KR closer to one led to small proportions of students
with differences beyond chance; values of KR further from one (i.e., closer to zero) led to larger
proportions of students with differences beyond chance.
The correlation corrected for attenuation due to unreliability, c jk , is given by:

jk
jk ,
jj kk
c
where jk is the uncorrected correlation between the scores on subtests j and k, and jj and kk
are the internal consistency estimates (Cronbach, 1951) for subtests j and k, respectively. If c jk
is less than 0.90, then it is concluded that student performances on subtests j and k differ and
that reporting of subtest scores is warranted (McPeek et al., 1976; Haladyna & Kramer, 2004).
For example, Haladyna and Kramer (2004) used the c jk method to determine whether subtest
scores on a basic biomedical science test revealed any differences in examinees performances.
They found that the corrected correlations were higher than 0.90, suggesting a high degree of
similarity in examinees performances on the subtests of the test.
The proportional reduction of the mean squared error method involves predicting the true
scores on subtest j from the observed scores on subtest j and from the total test score:
ij j jj (sij j ) (1)
and

iX j s x j
( xi X ) , (2)
t
X
where ij and iX are, respectively, the true score for student i on subtest j when predicted from
the observed subtest score and the true score of student i on subtest j when predicted from the
total test X score for student i; j and X are the means of subtest j and the total test X;
sij and xi are the observed scores on subtest j and the total test X, respectively, for student i;
jj and XX are the internal consistencies of subtest j and the total test X, respectively;
64
X is the standard deviation of the total test scores, and j jj , where j is the standard
j
deviation of the scores on subtest j; and s x s2 x XX , where

t
s2 x is computed as outlined in
Haberman (2005). The corresponding mean squared errors (MSE) are given by:
and
,
where 2j is the true score variance for subtest j. The MSE when j is simply predicted from
s j is the subtest score error variance, e2 2j (1 jj ) .
j
The proportional reduction of the mean squared errors when the true score is predicted from a
subtest score using equation (1) is given for each subtest by:
e2 - MSE / s
PRMSE =
e2
/s
The PRMSE when the true score is predicted from the total test score is computed in the same
way but using the MSE / x as the base. If PRMSE / s > PRMSE/x , then reporting the scores for
subtest j adds value over reporting only the total test scores (Haberman, 2005, 2008; Sinharay et
al., 2007, 2009; Lyren, 2009; Sinharay, 2010). Haberman (2008) used the PRMSE method to
determine whether or not the subtest scores on SAT I had added value over and above the
value of the total score and found that none of the section scores of SAT I math or SAT I
verbal provide any appreciable information concerning an examinee that is not already
provided by the math or verbal total score (p. 221). Using the PRMSE method, Sinhary (2010)
examined 25 operational tests to see if the subtests within each test had added value over the
full test. He found that 16 of the 25 tests had no subtest scores with added value even though
subtest scores were reported in many cases. Of the remaining nine tests, some but not all of the
subtests had added value. However, it should be noted that in contrast to correlations corrected
for attenuation, the PRMSE does not compare subtest scores to determine if they are distinct
from one another.
In contrast to correlations corrected for attenuation and proportional reduction of the mean
squared error methods, which do not specifically look at the agreement between two observed
scores obtained from two subtests, the agreement method takes into account the actual
differences between observed scores on subtests j and k expressed in the same score metric. In
the case of the c jk method, if differences among the subtests are revealed, then the agreement
method will need to be used to determine which students have pairs of scores that differ. In the
case of the PRMSE method, if a subtest is found to have value over the total test, then the
agreement method will need to be used to determine which students have subtest scores that
differ from the total test. Thus, it seems reasonable to use the Kelleys agreement method alone.
65
Hence, one purpose of the present study was to determine whether Kelleys agreement and
correlations corrected for attenuation methods would lead to the same or different decision
regarding the identification of pairs of distinct subtests and whether Kelleys agreement and
proportional reduction of the mean squared error methods would lead to the same or different
decision about subtests having added value over the total test. If the decisions were the same in
both, then the agreement method could simply be used.
A second purpose of the study was to examine the accuracy and precision of the statistics used
in the KR, c jk , and PRMSE methods. No studies were found in the published literature that
comparatively examined accuracy and precision of the statistics used in these methods. If one
method produced biased or imprecise estimates, then different decisions could be made when
using samples rather than the population. However, if the method produced unbiased and
precise estimates, then the decisions made would not be due to bias or impreciseness.
Method
The two data sets used in the study were population data sets for the Junior (Grade 6) English-
language Reading and Mathematics assessments conducted by the Education Quality and
Accountability Office (EQAO) in Ontario, Canada (www.eqao.com). EQAO conducts annual
province-wide assessments in both of Canadas official languages (English and French) at the
Primary (Grade 3) and Junior (Grade 6) levels in the areas of Reading, Writing, and
Mathematics, at the Grade 9 level in Academic Mathematics and Applied Mathematics, and at
the Grade 10 level in Literacy (Reading and Writing). Results are reported at the provincial,
district, school, and student levels and are publically available on the EQAO website, with
emphasis on progress from the previous year. EQAO requested that the present study be
conducted to determine if reporting subtest scores was justified given no explicit attempt was
made to develop subtests with psychometric characteristics that allowed subtest score
reporting.
Description of the Reading Test

The English-language Reading test items are referenced to three knowledge and skills
categories as specified in the curriculum for the province: Explicit Information, Implicit
Information, and Connections. The items in the Explicit Information subtest require students to
detect and understand information and ideas stated explicitly in a variety of text types
identified in the provincial curriculum. The items in the Implicit Information subtest probe
students understanding of implicitly stated information and ideas. The items in the
Connections subtest require students to demonstrate their understanding of text passages by
connecting, comparing, and contrasting the ideas presented in the passages and drawing upon
their own knowledge, experience and insights, other texts, and the world around them. Thus,
the three subtests can be ordered in terms of complexity, with the Explicit Information and
Connections subtests at the lowest and the highest levels of complexity, respectively. The
Explicit Information subtest contains six multiple-choice items, the Implicit Information subtest
contains 14 multiple-choice items and four open-response items, and the Connections subtest

66
contains six multiple-choice items and six open-response items. The 10 open-response items are
scored using four-point scoring rubrics.
Description of the Mathematics Test

In contrast to the Reading assessment, the items on the Mathematics assessment are referenced
by content areas (i.e., strands) and by cognitive skills as specified in the mathematics
curriculum. The five content areas include: Number Sense and Numeration (8 items involving
estimation, rate, ratio, and use of fractions), Measurement (8 items involving the use of area
relationships, understanding of the dimensions of the shapes needed to calculate their areas,
and the conversion of metric area units), Geometry and Spatial Sense (6 items dealing with the
identification, performance and description of transformations, the identification of angles, and
accurate use of rulers and protractors), Patterning and Algebra (7 items dealing with growing
patterns, use of diagrams, tables and number sequences to represent the stages of patterns), and
Data Management and Probability (7 items involving concepts of probability, predicting and
representing the probability of an outcome, comparing probabilities using common
representations (e.g., common denominators, percents or decimals), and interpreting graphs).
The five content areas are not ordered in terms of complexity.
Cognitive skills are divided into three categories: Knowledge and Understanding (8 items),
Application (15 items), and Problem Solving (13 items). The items referenced to the Knowledge
and Understanding category require students to demonstrate subject specific content
(knowledge) and the comprehension of its meaning and significance (understanding). The
Application items require students to select and fit an appropriate mathematical tool or get the
necessary information. The Problem Solving items require students to select and sequence a
variety of tools to solve a problem and demonstrate a critical-thinking process. That is, to
answer Problem Solving items, students need to make a plan. In contrast to the content subtests,
the cognitive subtests can be ordered in terms of complexity, with the Knowledge and
Understanding subtest and the Problem Solving subtest being at the lowest and the highest
levels of complexity, respectively. The total number of items on the Mathematics assessment is
36, including 8 open-response items scored using a four-point scoring rubric and distributed
such that each content subtest has at least one open-response item.
Analyses
The analyses were conducted in two main stages corresponding to the two purposes of the
study. First, the responses of the population of students were analysed to obtain the population
value of each test statistic for each of the three detection methods. Following this, the analyses
were repeated for 1,000 replicated independent samples of five different sizes 250, 500, 1,000,
2,000, and 5,000 randomly drawn from the population with replacement to (1) determine the
effect of sample size on the accuracy and precision of the estimators, and then to (2) assess the
consistency of the decisions made using the three detection methods in light of the findings
about accuracy and precision. At the first stage, means and standard deviations of the
distributions of sample statistics were used to evaluate the three detection methods with respect
to their accuracy and precision. At the second stage, the KR and c jk methods were applied,
first, at the population level to determine if the subtests were distinct, and then applied to each
67
of 1,000 replicated samples for each of the five sample sizes to see if the same decision was
made. The PRMSE method was applied at the population level and then for each replicated
sample to see if the subtests added value over the total test. The consistency of the decisions
made was assessed using the percentage of samples that led to the same decision that was made
at the population level.
Psychometric Properties of the Reading and Mathematics Tests

The psychometric properties of the Junior Reading and Mathematics subtests and the total tests
are provided in Table 1 for the population of students. The means and standard deviations are
reported in the observed score units and as percentages (in parentheses).
Table 1. Means, Standard Deviations, and Internal Consistencies for Reading and Mathematics Tests
Subtest/Total Test k/msa X. sX Correlations
Reading, N = 128,089 EI II C TT
Explicit Information (EI) 6/6 4.59 (76.5) 1.28 (21.3) 0.47b 0.59 0.52 0.69
Implicit Information (II) 18/30 20.92 (69.7) 4.42 (14.7) 0.76 0.74 0.93
Connections (C) 12/30 17.16 (57.2) 4.33 (14.4) 0.74 0.92
Total Test (TT) 36/60 42.68 (64.7) 8.97 (13.6) 0.87
Mathematics, N = 127,596
Content Area N M A P G TT
Numeration (N) 8/14 8.44 (60.3) 3.02 (21.6) 0.63 0.66 0.62 0.67 0.63 0.87
Measurement (M) 8/11 6.34 (57.6) 2.67 (24.3) 0.63 0.59 0.63 0.63 0.84
Algebra (A) 7/10 6.63 (66.3) 2.14 (21.4) 0.58 0.62 0.59 0.80
Probability (P) 7/13 7.20 (55.4) 2.71 (20.8) 0.61 0.62 0.85
Geometry (G) 6/12 7.18 (59.8) 2.80 (23.3) 0.60 0.83
Total Test (TT) 36/60 35.79 (59.7) 11.20 (18.7) 0.89
Cognitive Skill K/U A PS TT

Know/Understand (K/U) 8/8 5.45 (68.0) 1.87 (23.4) 0.60 0.70 0.67 0.80
Application (A) 15/24 14.98 (62.4) 4.91 (20.5) 0.75 0.79 0.94
Problem Solving (PS) 13/28 15.36 (54.8) 5.42 (19.4) 0.78 0.94
Total Test (TT) 36/60 35.79 (59.7) 11.20 (18.7) 0.89
ak is number of items in a subscale or the total test and ms is the maximum score given the use of
dichotomously scored multiple-choice items and polytomously scored open-response items.
b Internal consistencies of the subtests and the total test are shown in italics along the principal diagonal
of each correlation panel.
Reading. The mean percentages revealed that students performance declined on the original
three subtests as the complexity of the constructs increased. The standard deviations
(percentages) were essentially the same for the Implicit Information and Connections subtests,
which are at the two higher levels of complexity, but smaller than for the Explicit Information
subtest, likely because of the smaller number of items and, therefore, total points for this
subtest. As shown along the main diagonal of the correlation matrix on the right side of Table 1,

68
the internal consistency (alpha; Cronbach, 1951) of the Explicit Information subtest was much
lower than the internal consistencies of the Implicit Information and Connections subtests,
which were essentially the same. The low reliability of the Explicit Information is due to the
relatively small number of items (6) in this subtest in comparison to the other subtests (18 and
12, respectively). The estimate of the internal consistency of the total test was 0.87, reflecting the
typical practice mentioned above of ensuring that the total test reliability is at an acceptable
high level. The values of the correlations were either greater than the corresponding reliabilities
or close in value, which suggests that the three procedures examined in this study will show
that the subtests are not distinct and the subtests do not add value over and above the total test.
Mathematics content area. The mean percentages revealed that the mean for the Algebra subtest
was the highest, the mean on the Probability subtest was the lowest, and the means of the other
three subtests were between and essentially the same. The standard deviations were somewhat
larger for the Measurement and Geometry subtests than the standard deviations for the
Numeration, Algebra, and Probability subtests, which were essentially the same. Given the
numbers of items in each subtest did not differ much as they did in the case of Reading and
Mathematics cognitive skills, the internal consistencies of the five content subtests were
essentially the same, ranging from 0.58 to 0.63. However, as with Reading, the values of the
correlations were close to the values of the reliabilities, suggesting again that the three
procedures examined in this study will show that the subtests are not distinct and the subtests
do not add value over and above the total test. Further, some of the values of KR and c jk will
exceed 1.00, which theoretically should not happen.
Mathematics cognitive skills. Similar to Reading, the students performance on the three
mathematics cognitive subtests declined as the level of required thinking increased from
knowledge and understanding to application to problem solving. The standard deviations were
essentially the same for the Application and Problem Solving subtests, which are of higher
complexity, but smaller than the standard deviation for the knowledge and understanding
subtest, again likely because of the smaller number of items in the latter subtest. The internal
consistency of the knowledge and understanding subtest, 0.60, was lower than the internal
consistencies of the Application and Problem Solving subtests, which were more alike, 0.75 and
0.78, respectively. The somewhat low value of reliability for the knowledge and understanding
subtest was likely due to the relatively smaller number of items (8) in this subtest as compared
to the numbers of items in the other two subtests (15 and 13, respectively). The estimate of the
internal consistency of the total test was 0.89, again reflecting the typical practice mentioned
above of ensuring that the total test reliability is at an acceptable high level. Again, we see as for
the Reading and Mathematics content areas that the values of the correlations are close to the
values of the reliabilities, suggesting that the three procedures examined in this study will show
that the subtests are not distinct and the subtests do not add value over and above the total test,
with some of the values of KR and c jk will exceed 1.00.

69
Accuracy and Precision of the Estimators

The means and standard deviations of the 1,000 replications for each sample size are reported in
Table 2 for the KR method, Table 3 for the c jk method, and Table 4 for the PRMSE method. As
shown in Table 2, all but two of the means of the sampling distributions of 1,000 replications
across the pairs and sample sizes were within 0.01 of the corresponding population value of KR
for each subtest pair and sample size (the difference is 0.02 for the Numeration and Probability
subtest pair, with n = 250 and n = 500). The standard error of KR decreased as the sample size
increased. For example, for n = 250 the standard errors were between 0.043 and 0.049, whereas
for n = 5,000 the standard errors were between 0.009 and 0.011.
Table 2. Accuracy and Precision: Kelleys Ratio (KR)

Sample Size
Subtest Pairs 250 500 1,000 2,000 5,000 Population
Reading
Exp InfoImp Info 0.97 (0.049)a 0.97 (0.035) 0.97 (0.024) 0.97 (0.017) 0.97 (0.011) 0.97
Exp InfoCon 0.92 (0.046) 0.91 (0.032) 0.91 (0.022) 0.91 (0.015) 0.91 (0.010) 0.91
Imp InfoCon 0.96 (0.043) 0.95 (0.032) 0.95 (0.021) 0.95 (0.015) 0.95 (0.009) 0.96
Mathematics Content Area
NumMea 1.04 (0.051) 1.04 (0.035) 1.04 (0.025) 1.04 (0.017) 1.04 (0.011) 1.04
NumAlg 1.03 (0.050) 1.03 (0.034) 1.03 (0.025) 1.03 (0.016) 1.03 (0.011) 1.02
NumProb 1.09 (0.050) 1.09 (0.037) 1.08 (0.026) 1.08 (0.026) 1.08 (0.011) 1.07
NumGeo 1.03 (0.049) 1.03 (0.037) 1.03 (0.025) 1.02 (0.017) 1.02 (0.011) 1.02
MeaAlg 0.98 (0.045) 0.98 (0.033) 0.98 (0.023) 0.98 (0.016) 0.98 (0.010) 0.98
MeaProb 1.02 (0.046) 1.02 (0.032) 1.02 (0.024) 1.02 (0.017) 1.02 (0.010) 1.01
MeaGeo 1.02 (0.052) 1.03 (0.035) 1.02 (0.026) 1.02 (0.017) 1.02 (0.011) 1.02
AlgProb 1.03 (0.049) 1.03 (0.033) 1.03 (0.024) 1.03 (0.017) 1.03 (0.010) 1.03
AlgGeo 1.00 (0.047) 1.00 (0.033) 1.00 (0.023) 1.00 (0.017) 1.00 (0.010) 1.00
ProbGeo 1.03 (0.048) 1.03 (0.034) 1.03 (0.023) 1.03 (0.017) 1.03 (0.011) 1.02
Mathematics Cognitive Skill
Kno/UndApp 1.04 (0.052) 1.04 (0.035) 1.04 (0.026) 1.03 (0.017) 1.04 (0.011) 1.04
Kno/UndProb Sol 0.98 (0.049) 0.98 (0.031) 0.98 (0.024) 0.97 (0.018) 0.97 (0.011) 0.97
AppProb Sol 1.05 (0.048) 1.05 (0.035) 1.05 (0.024) 1.05 (0.018) 1.05 (0.011) 1.05
a The first value is the mean and the value in parentheses is the standard deviation of the sampling
distribution (i.e., standard error) of 1,000 replications.
The means of the sampling distributions of c jk were within 0.01 of the corresponding
population values of c jk for all the pairs of subtests and sample sizes (see Table 3). The
standard errors of sample estimators decreased as the sample size increased. For n = 250, the
standard errors ranged between 0.029 and 0.081, whereas for n = 5,000, the standard errors were
as low as 0.007 and as high as 0.017. Given the low reliability of the Explicit Information subtest
in the Reading assessment, the standard errors for the pairs involving this subtest were
consistently higher than the standard errors for the remaining pairs of subtests.

70
Similar to KR and c jk , the means of the distributions of sample estimators of PRMSE / s and
PRMSE/x were within 0.01 of the corresponding population values for all four subtests (Table
4). The standard errors of sample estimators were the largest when the Explicit Information
subtest was considered (e.g., the standard error PRMSEE 0.100 for n = 250) but decreased as
the sample size increased, ranging between 0.003 and 0.020 for n = 5,000. Taken together, the
results provided in Tables 2, 3, and 4 reveal that sample estimates of KR, c jk , and PRMSE / s
and PRMSE/x are accurate and precise. Therefore, any differences among the three detection
methods used for the detection of subtest differences or subtest-total test differences are not
confounded by presence of biased or imprecise estimators.
Table 3. Accuracy and Precision: Correlation Corrected for Attenuation ( c jk )

Sample size Popu-
Subtest Pairs 250 500 1,000 2,000 5,000 lation
Reading
ExpImp Info 0.99 (0.078)a 0.99 (0.051) 0.99 (0.037) 0.98 (0.026) 0.98(0.016) 0.98
Exp InfoCon 0.90 (0.081) 0.89 (0.056) 0.89 (0.039) 0.89 (0.017) 0.89 (0.017) 0.89
Imp InfoCon 0.97 (0.029) 0.97 (0.021) 0.97 (0.015) 0.97 (0.011) 0.97 (0.007) 0.97
NumMea 1.05 (0.054) 1.05 (0.037) 1.05 (0.026) 1.05 (0.018) 1.04 (0.012) 1.04
NumAlg 1.04 (0.062) 1.04 (0.042) 1.04 (0.031) 1.03 (0.020) 1.03 (0.014) 1.03
NumProb 1.09 (0.051) 1.09 (0.038) 1.09 (0.027) 1.09 (0.018) 1.09 (0.012) 1.09
NumGeo 1.03 (0.058) 1.03 (0.040) 1.03 (0.029) 1.03 (0.021) 1.03 (0.013) 1.03
MeaAlg 0.98 (0.062) 0.97 (0.045) 0.97 (0.032) 0.97 (0.022) 0.97 (0.014) 0.97
MeaProb 1.02 (0.055) 1.02 (0.038) 1.02 (0.028) 1.02 (0.020) 1.02 (0.013) 1.02
MeaGeo 1.02 (0.062) 1.03 (0.041) 1.03 (0.031) 1.03 (0.020) 1.03 (0.013) 1.03
AlgProb 1.04 (0.064) 1.04 (0.043) 1.04 (0.030) 1.04 (0.022) 1.04 (0.013) 1.04
AlgGeo 0.99 (0.066) 1.00 (0.046) 0.99 (0.033) 0.99 (0.023) 0.99 (0.014) 0.99
ProbGeo 1.04 (0.059) 1.03 (0.042) 1.03 (0.029) 1.03 (0.021) 1.03 (0.014) 1.03
Kno/UndApp 1.04 (0.047) 1.04 (0.031) 1.04 (0.023) 1.04 (0.016) 1.04 (0.010) 1.04
Kno/UndProb Sol 0.99 (0.048) 0.99 (0.031) 0.98 (0.023) 0.98 (0.016) 0.98 (0.010) 0.98
AppProb Sol 1.03 (0.026) 1.03 (0.018) 1.03 (0.013) 1.03 (0.009) 1.03 (0.006) 1.03
aThe first value is the mean and the value in parentheses is the standard deviation of the sampling
Detection of Performance Differences and Consistency of Decisions

As foreshadowed in the presentation of the psychometric properties of the subtests and total
test and as revealed by the results in Tables 2, 3, and 4, the subtests were determined to be not
distinct nor did the subtests add value over the total test. The values of KR were close to one
with one exception (Reading, Explicit Information and Connections subtests; Table 2). Further,
11 of the 16 KR values exceeded one, which theoretically should not happen. For the agreement
procedure to work, the sum of the reliabilities of the two subtests has to be greater than two
times the correlation between the two subtests being compared. This was not the case with the

71
subtests considered in the present study, with the sum of the reliabilities in the 11 cases being
less than two times the corresponding correlations.
The decision rule for the method of correlations corrected for attenuation is a value less than
0.90 indicates that the two subtests being correlated are sufficiently different to warrant
reporting the scores on each (McPeek et al., 1976). With one possible exception (Reading,
Explicit Information and Connections subtests; Table 3), the values of c jk exceeded 0.95, with
10 of the 16 values being greater than 1.00, which theoretically should not happen. The decision
rule for the PRMSE method is: if PRMSE / s > PRMSE/x , then the subtest has added value over
and above the total test and, therefore, the score on the subtest should be reported. As shown in
Table 4, for all subtests, PRMSE / s < PRMSE/x . For both the c jk and PRMSE methods, the
reliabilities of the subtests must be high, which was not the case in the present study.
In the case of Reading, with perhaps one exception, the decisions made using population values
of KR and c jk were that the subtests did not differ, and the population values of PRMSE / s
and PRMSE/x indicated that the three subtests did not add value over the total test. For the
Explicit Information and Connections pair of subtests, KR suggested that there was a difference
beyond chance for 5% of the students, and that the value of c jk , 0.89, was just less than 0.90.
The sample data revealed that with exception of two subtest pairs, Explicit Information and
Connections and Explicit Information and Implicit Information with n = 250 and n = 500, the
same decision was made using sample data for at least 91% of the replications using the KR,
c jk , and PRMSE methods across the different sample sizes. In the case of the Explicit
Information and Connections pair, the decision consistency for c jk varied from 51.4% to 78.8%
across the five sample sizes (i.e., 514 of the 1,000 replications led to the same decision made at
the population level). This finding is attributable to the low reliability of the Explicit
Information subtest, 0.47, and the observation that the value of c jk was only 0.01 below the
decision value of 0.90. In the case of the Explicit and Implicit Information pairs, the decision
consistency for n = 250 was 90.5% and for n = 500, 95.1%, while for n 1, 000 the decision
consistencies were 99.1%, 100%, and 100%.
In the case of Mathematics, KR and c jk indicated that there were no distinct subtests and
PRMSE / s and PRMSE/x indicated that no subtest added value over the total test. Further, the
majority of values for KR were greater than 1.00 due to the fact that the sum of the reliabilities
was greater than two times the uncorrected correlation. Similarly, the majority of the values for
c jk were greater than 1.00 due to the fact that the square root of the product of the reliabilities

was less than the uncorrected correlation between the pairs of subtests. The sample data
revealed that, with three exceptions, Measurement and Algebra with n = 250 and n = 500 and
Algebra and Geometry with n = 250, the same decision was made using sample data for at least
97% of replications using the KR and c jk methods and, in the case of the subtest-total test
pairs, the PRMSE method. The exceptions included the Algebra subtest, which had the lowest
72
reliability out of the all Mathematics subtests. Again, as for Reading, the sample values of the
correlations were close to the sample values of the reliabilities, and in the majority of cases the
two times the correlation exceeded the sum of the reliabilities, leading to sample estimates
greater than 1.00.
Table 4. Accuracy and Precision: Proportional Reduction of the Mean Squared Error (PRMSE)
Sample Size Popu-
Subtest/PRMSE 250 500 1,000 2,000 5,000 lation
Reading
Exp PRMSE / s 0.47 (0.054)a 0.47 (0.038) 0.47 (0.027) 0.47 (0.020) 0.47 (0.012) 0.47
Info
PRMSE/x 0.80 (0.100) 0.79 (0.065) 0.79 (0.047) 0.79 (0.033) 0.79 (0.020) 0.79
Imp PRMSE / s 0.76 (0.023) 0.76 (0.016) 0.77 (0.011) 0.77 (0.008) 0.77 (0.005) 0.77
Info
PRMSE/x 0.87 (0.017) 0.87 (0.013) 0.87 (0.008) 0.87 (0.006) 0.87 (0.004) 0.87
Con PRMSE / s 0.73 (0.023) 0.73 (0.016) 0.73 (0.011) 0.74 (0.008) 0.74 (0.005) 0.74
PRMSE/x 0.85 (0.020) 0.85 (0.014) 0.85 (0.010) 0.85 (0.007) 0.85 (0.004) 0.85

Num PRMSE / s 0.63 (0.028) 0.63 (0.019) 0.63 (0.015) 0.63 (0.010) 0.63 (0.006) 0.63
PRMSE/x 0.94 (0.037) 0.94 (0.025) 0.94 (0.019) 0.94 (0.012) 0.94 (0.008) 0.94
Mea PRMSE / s 0.63 (0.029) 0.63 (0.024) 0.63 (0.015) 0.63 (0.010) 0.63 (0.006) 0.63
PRMSE/x 0.90 (0.042) 0.90 (0.031) 0.90 (0.021) 0.89 (0.014) 0.89 (0.009) 0.89
Alg PRMSE / s 0.58 (0.038) 0.58 (0.025) 0.58 (0.019) 0.58 (0.013) 0.58 (0.008) 0.58
PRMSE/x 0.89 (0.055) 0.88 (0.039) 0.88 (0.027) 0.88 (0.019) 0.88 (0.012) 0.88
Prob PRMSE / s 0.61 (0.032) 0.61 (0.022) 0.61 (0.019) 0.61 (0.011) 0.61 (0.007) 0.61
PRMSE/x 0.94 (0.042) 0.93 (0.029) 0.93 (0.020) 0.93 (0.015) 0.93 (0.009) 0.93
Geo PRMSE / s 0.60 (0.033) 0.60 (0.024) 0.60 (0.017) 0.60 (0.011) 0.60 (0.007) 0.60
PRMSE/x 0.90 (0.045) 0.90 (0.031) 0.90 (0.023) 0.90 (0.016) 0.90 (0.010) 0.90

Kno/ PRMSE / s 0.60 (0.038) 0.60 (0.026) 0.60 (0.019) 0.60 (0.013) 0.60 (0.008) 0.60
Und
PRMSE/x 0.90 (0.055) 0.89 (0.036) 0.89 (0.027) 0.89 (0.018) 0.89 (0.012) 0.89
App PRMSE / s 0.75 (0.019) 0.75 (0.013) 0.75 (0.010) 0.75 (0.007) 0.75 (0.004) 0.75
PRMSE/x 0.91 (0.016) 0.91 (0.011) 0.91 (0.008) 0.91 (0.005) 0.91 (0.004) 0.91
Prob PRMSE / s 0.78 (0.017) 0.78 (0.012) 0.78 (0.009) 0.78 (0.006) 0.78 (0.004) 0.78
Sol
PRMSE/x 0.89 (0.014) 0.90 (0.010) 0.90 (0.007) 0.90 (0.005) 0.90 (0.003) 0.90
aThe first value is the mean and the value in parentheses is the standard deviation of the sampling

73
Taken together, the results provided in Tables 2, 3, and 4 revealed that there were no differences
among the abilities of the three detection methods to detect subtest differences or subtest-total
test differences. Kelleys agreement and correlations corrected for attenuation methods led to
the same decisions regarding the identification of pairs of distinct subtests. Likewise, Kelleys
agreement and proportional reduction of the mean squared error methods led to the same
decisions about subtests having added value over the total test. Specifically, the decisions were
that the subtests did not differ among themselves and the subtests did not add value over the
total test.
Conclusion and Implications for Practice
Whether or not to report subtest results is an important topic that has immediate practical
implications. Given a profile of subtest scores, teachers and school counsellors can identify areas
of strength and areas that need to be addressed for individual students. Similarly, changes in
curriculum and instruction designed to maintain strength and address issues at the school and
class levels can be made to improve student learning and achievement.
Subscore reporting will most likely be enhanced if subtests are specifically developed to
measure a multidimensional construct or domain. The subdomains to be assessed must be
clearly defined and, if supportable, weakly to moderately correlated. The number of items used
to assess each dimension or subdomain must be large enough to ensure an adequate level of
reliability. The correlations between the subtests examined in the present study were moderate
to moderately strong and the reliabilities of the subtests were not high, resulting in reliabilities
and correlations being similar in value.
But it seems reasonable to assume that the values of the correlations for the pairs of subtests in
the present study are likely to be found in other large-scale assessments of achievement in the
school system. Consequently, given this assumption, it is necessary to increase the reliabilities
of the subtests. For example, assuming the median observed correlation among Mathematics
content subtests in the present study, 0.63, the percentage of students who would be identified
with subtest differences beyond chance using the agreement method would be approximately
5% if the reliability of the two subtests was 0.70, 9% if the reliability of the two subtests was
0.75, 15% if the reliability of the two subtests was 0.80, and 20% if the reliability of the two
subtests was 0.85. Likewise, for the correlations corrected for attenuation and the proportional
reduction of the mean squared error methods, pairs of subtests are most likely to be found
distinct and subtests are most likely to have value over and above the total test if the subtests
have relatively high reliabilities and the true subtest scores and the true total scores have only
moderate correlations. The results for replicated random samples (n = 250, 500, 1,000, 2,000, and
5,000) revealed that the statistics for the three detection methods were accurate and precise
estimators of the corresponding population parameters.

74
References
American Educational Research Association, American Psychological Association, & National Council on
Measurement in Education. (1999). Standards for educational and psychological testing.
Washington, DC: American Educational Research Association.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297334.
Grandy, J. (1992). Construct validity study of the NTE core battery using confirmatory factor analysis.
(ETS Research Report No. RR-92-03). Princeton, NJ: Educational Testing Service.
Gulliksen, H. (1950, 1967). Theory of mental tests. New York: John Wiley & Sons, Inc.
Haberman, S. J. (2005). When can subscores have value? (ETS Research Report No. RR-05-08). Princeton,
NJ: Educational Testing Service.
Haberman, S. J. (2008). Subscores and validity. (ETS Research Report No. RR-08-64). Princeton, NJ:
Educational Testing Service.
Haladyna, T. M. & Kramer, G. A. (2004). The validity of subscores for a credentialing test. Evaluation and
the Health Professions, 27, 349368.
Harris, D. J. & Hanson, B. A. (1991, April). Methods of examining the usefulness of subscores. Paper
presented at the annual meeting of the National Council on Measurement in Education, Chicago.
Kelley, T. L. (1923). A new method for determining the significance of differences in intelligence and
achievement scores. Journal of Educational Psychology, 14, 300303.
Lord, F. M. & Novick, M. R. (1968). Statistical theories of mental test scores. New York: AddisonWesley.
Lyrn, P. E. (2009). Reporting subscores from college admission tests. Practical Assessment, Research and
Evaluation, 14(4), 110.
McPeek, M., Altman, R., Wallmark, M., & Wingersky, B. C. (1976). An investigation of the feasibility of
obtaining additional subscores on the GRE Advanced Psychology Test (GRE Board Professional
Report No. 74 - 4P). Princeton, NJ: Educational Testing Service. (ERIC Document No. ED163090)
Ryan, J. (2003). An analysis of item mapping and test reporting strategies. Greensboro, NC: South
Carolina Department of Education.
Sinharay, S. (2010). How often do subscores have added value? Results from operational and simulated
data. Journal of Educational Measurement, 47, 150174.
Sinharay, S., Haberman, S. J., & Puhan, G. (2007). Subscores based on classical test theory: To report or not
to report. Educational Measurement: Issues and Practice, 26, 2128.
Sinharay, S., Puhan, G., & Haberman, S. (2009). Reporting diagnostic scores: Temptations, pitfalls, and
some solutions. Paper presented at the National Council on Measurement in Education, San
Diego, CA, USA.
Tate, R. L. (2004). Implications of multidimensionality for total score and subscore performance. Applied
Measurement in Education, 17, 89112.
Wainer, H., Sheehan, K. M., & Wang, X. (2000). Some paths toward making Praxis scores more useful.
Journal of Educational Measurement, 37, 113140.
Wainer, H., Vevea, J. L., Camacho, F., Reeve, B. B., Rosa, K., Nelson, L., Swygert, K. A., & Thissen, D.
(2001). Augmented scores borrowing strength to compute scores based on small numbers of
items. In Test Scoring (pp. 343387). Mahwah, NJ: Lawrence Erlbaum Associates.
Yao, L. & Boughton, K. A. (2007). A multidimensional item response modeling approach for improving
subtest proficiency estimation and classification. Applied Psychological Measurement, 31, 83105.

75

Vol. 4, No. 1, pp. 75-82, April 2014
Analysis of Achievement Tests in Secondary

Chemistry and Biology
Allen A. Espinosa
Faculty of Science, Technology and Mathematics, College of Teacher Development,
Philippine Normal University, Manila, Philippines
Maria Michelle V. Junio

Makati Science High School, Osias St., Poblacion Makati City, Philippines
May C. Manla
St. Louise de Marillac School of Tabaco, Ziga Avenue, Tabaco City, Albay,
Philippines
Vivian Mary S. Palma

NOH-School for Crippled Children, Banawe St., Quezon City, Philippines
John Lou S. Lucenari

San Jose National High School, Rodriguez Rizal, Philippines
Amelia E. Punzalan
University of the Philippines National Institute for Science and Mathematics
Education Development, Diliman, Quezon City, Philippines
Abstract. The study analyzes the performances of students in a regular high school,
a special education school and a special science high school of different regions in
the division achievement tests in Chemistry for school year 2009-2010. It also
identifies the content of the division achievement tests in chemistry as well as in
biology. The biology achievement test serves as a benchmark on how sophomores
are being prepared, prior to taking a chemistry course. Test items in the division
achievement tests are classified according to the level of thinking being developed.
Difficulty indices for each item in the Chemistry achievement tests are also
determined. The study found out that both Chemistry and Biology achievement
tests focus on factual knowledge which promotes lower order thinking skills. It also
found out that the special school and regular high schools performances in the
achievement test still fit a bell-shaped normal curve while a special science high
school has a skewed to the right curve. Performances of students are also affected
by face validity.
Keywords: achievement test in chemistry; achievement test in biology

76
Background of the Study
Teachers use different assessment tools in assessing their students. One of the
assessment tools that teachers use is the achievement test. Tatum (2010) defined
achievement tests as examinations that are designed to determine the degree of
knowledge and proficiency exhibited by an individual in a specific area or set of
areas. In the Philippines, the achievement test is given towards the end of the school
year. This is usually given in February or March when all the expected learning
competencies given by the Department of Education have been tackled already.
Achievement tests in the public schools are usually provided by the Department of
Education, at both division and national levels. The National Educational Testing
and Research Center of the Department of Education is the office in charge of
administering the achievement test. In the Division of City Schools in Quezon City,
the division achievement is made by master teachers in the field and by the Division
Supervisor of the area (Division of City Schools Quezon City, 2010). These reasons
have prompted the researchers to conduct a study regarding division achievement
tests in different regions.
This study aims to determine and analyze the performances of students in a regular
high school, a special school and a special science high school of different regions in
the division achievement tests in Chemistry for school year 2009-2010. Moreover, it
also identifies the content of the division achievement tests in Chemistry. It also
attempts to compare the content of the division achievement test in Biology on how
teachers in the second year level prepare students for the Chemistry course that
they will be taking after Biology. After the analysis, the study also aims to propose a
plan of action in increasing the performance of students in division achievement
tests.
Methodology
The researchers gathered Chemistry division achievement tests of different regions

where they are teaching. These regions include the National Capital Region
(represented by Makati and Quezon City) and Region IV-A (represented by Rizal).
One of the researchers teaches Biology and was able to gather a Biology division
achievement test in Region V (represented by Albay).
In National Capital Region and Region IV-A, the researchers were able to gather
actual data of students responses on the said Chemistry division achievement tests
while in Region V, the researchers were not able to gather actual student responses.
The researchers would only use the Biology division achievement test to test the
level of thinking that it is promoting in relation to preparing the students to a
Chemistry course.

77
After gathering the needed data, the researchers made an item analysis (see
appendices) wherein each question in the division achievement test was categorized
into three. These are factual knowledge; conceptual understanding; and reasoning
and analysis. After classifying each item, the researchers counted the number of
students who were able to give a correct response. From this, they computed for the
index of difficulty of each item by dividing the total number of students who were
able to give a correct response by the total number of examinees. From the item
analysis, the researchers made a table summarizing the levels of thinking and a
table summarizing the difficulty index.
The following data were collected from the item analysis done.
Table 1. Summary of Level of Thinking

Number of Items (Percentage)
per Region
Thinking
NCR
Levels
Quezon Makati IV-A V
City City
Factual 16 18 22 31
Knowledge (32%) (36%) (37%) (62%)
Conceptual 19 15 21 11
Understanding (38%) (30%) (35%) (22%)
Reasoning and 15 17 17 8
Analysis (30%) (34%) (28%) (16%)
Total Number 60
50 50 50
of Items
Table 1 shows the summary of levels of thinking in the division achievement tests in
Chemistry in Regions IV-A and the National Capital Region. It also shows the
summary of level of thinking in the division achievement in Biology for region V.
From the table, it is noticeable that in general, division achievement tests give much
emphasis on factual knowledge, thereby focusing only on lower order thinking
skills. The skills being developed under this category are recalling or recognizing,
defining, describing, and using tools and procedures (Mullis, et. al., 2003). The next
skill that is given emphasis is conceptual understanding which focuses on
illustrating with examples, comparing or contrasting or classifying, representing or
modeling, relating, extracting or applying information, finding solutions and
explaining (Mullis, et. al., 2003).
The thinking level that is given least emphasis is reasoning and analysis which
develop the higher order thinking skills of students like analyzing, interpreting or

78
solving problems, integrating or synthesizing, hypothesizing or predicting data,

drawing conclusions, generalizing, evaluating and justifying (Mullis, et. al., 2003).
One will also notice that the Biology division achievement test (as shown by Region
V) is similar with the Chemistry division achievement test (as shown by NCR and
Region IV-A) in terms of thinking level content. Both give much emphasis on
factual knowledge and less on reasoning and analysis.
It is also very evident that the Biology achievement test gives a very big emphasis
on factual knowledge having more that 50% of the test items. The 1986 Constitution
of the Republic of the Philippines mandates all schools to encourage critical and
creative thinking (Constitution of the Republic of the Philippines, 2005, p. 55)
among all Filipino students. Moreover, the 2002 Basic Education Curriculum
prescribes the use of inquiry in teaching science to promote higher order thinking
skills, such as critical and creative thinking (Department of Education, 2002).
Table 2. Summary of Difficulty Index

Number of Items (Percentage) per
Difficulty Region
Index NCR
Range Quezon Makati IV-A
City City
0.00 4 (8%) 2 (4%)
1 (2%)
0.04
0.05 0 0
2 (3%)
0.09
0.10 0 2 (4%)
0
0.14
0.15 9 (18%) 0
1 (2%)
0.19
0.20 0 0
3 (5%)
0.24
0.25 0 1 (2%)
2 (3%)
0.29
0.30 13 (26%) 0
2 (3%)
0.34
0.35 0 0
5 (8%)
0.39
0.40 0 1 (2%)
6 (10%)
0.44
0.45 0 0
6 (10%)
0.49

79
0.50 7 (14%) 1 (2%)

6 (10%)
0.54
0.55 0 0
5 (8%)
0.59
0.60 0 1 (2%)
3 (5%)
0.64
0.65 7 (14%) 2 (4%)
3 (5%)
0.69
0.70 0 2 (4%)
5 (8%)
0.74
0.75 0 1 (2%)
2 (4%)
0.79
0.80 7 (14%) 1 (2%)
4 (8%)
0.84
0.85 0 5 (10%)
2 (3%)
0.89
0.90 0 8 (16%)
2 (3%)
0.94
0.95 0 6 (12%)
0
0.99
1.00 3 (6%) 17 (34%) 0
Table 2 shows the summary of index of difficulty across two regions.
0.4
Frequency of Occurence
0.35
0.3
0.25 Quezon City
0.2 Makati City
0.15 Rizal
0.1
0.05
0
1 3 5 7 9 11 13 15 17
Difficulty Index Interval
Figure 1.
Difficulty Index Across Regions
You will notice that if you graph (see figure 1) the index of difficulty for Quezon
City (NCR) and Region IV-A, you will be able to get a bell-shaped normal curve,

80
meaning, the scores are equally distributed. The representative school from Quezon
City (NCR) is a special school while the representative school from Region IV-A is a
regular high school. But if you will graph the index of difficulty for Makati City
(NCR), you will not be able to get a bell-shaped normal curve since the scores are
not equally distributed. The curve will be skewed to the right, meaning, more
students were able to get a correct response to each item in the achievement test.
This is because the representative school from Makati City (NCR) is a special
science high school.
The items manifesting a very high difficulty index show that the each item is easy
and that it needs to be replaced or restructured. On the other hand, the items
manifesting a very low difficulty index indicate that the said item is difficult and
that it needs to be replaced or restructured, as well. A rough "rule-of-thumb" is that
if the item difficulty is more than .75, it is an easy item; if the difficulty is below .25,
it is a difficult item (Classroom Assessment, 2010). In other cases, a very low
difficulty index would mean that the topic was not tackled in the class.
The result shows that even if it is the governments primary battle cry to produce
Filipino citizens with higher order thinking skills, the present classroom instruction
proves to be unsuccessful in developing higher order thinking skills among Filipino
students. Florencio Abad, the then- secretary of the Department of Education,
mentioned that the mastery levels in Science, Mathematics and English are
devastating (Abad, 2005, p.8). Abad is referring to the performance of Filipino high
school students in various competency- based examinations in 2004.
The bad performance of the Philippines in the Trends in International Mathematics

and Science Study (TIMSS) in 2003, ranking near the bottom also shows that Filipino
students are weak in terms of higher order thinking abilities because the test
required more skills in reasoning and analysis rather than conceptual
understanding and factual knowledge (Martin, et. al., 2004).
Table 3. Inconsistencies in the 2009-2010 Division Achievement Tests in Chemistry

Item Numbers by Regions
(Total Items =Percentage)
Category NCR
Quezon Makati IV-A
City City
6, 9, 11,
14, 15, 16,
Improper
22, 23, 25,
arrangement 0 0
27, 35, 37,
of choices
40, 42, 46
(15=30%)
Improper 13
0 0
positioning of (1=2%)

81
a figure in a
test question
2, 3, 10,
21, 30, 41,
Incomplete 11, 12, 13, 6, 12
43, 52
question 30 (4%)
(5=8%)
(7=14%)
Misleading 35, 45, 48 35, 45, 48 56
question (6%) (6%) (1=2%)
Misleading 6, 45, 53
0 0
choices (3=5%)
Parallelism in
49 2 6
construction
(2%) (1=2%) (1=2%)
of choices
2, 3, 11, 9,
5, 23, 38, 6, 30, 41,
Typographical 30, 32, 37,
46 54, 52
Error 45, 46
(4=8%) (8%)
(18%)
13
Unclear figure 0 0
(1=2%)
Another factor affecting student achievement in the division achievement test in

Chemistry is the face validity of the test itself. Face validity is concerned with how a
measure or procedure appears (Colorado State University, 2010). Table Three
shows inconsistencies in the 2009-2010 division achievement tests in chemistry.
These could be factors that affect the scores of students who took the achievement
tests.
Conclusion and Recommendations
As seen in the summary table for levels of thinking, division achievement tests in
Chemistry and Biology give much emphasis on factual knowledge, which focuses
on developing lower order thinking skills only. The least given emphasis is
reasoning and analysis, which focuses on higher order thinking skills. As a result,
Filipino high schools students manifested a poor performance in international
competency- based examinations, as well as in division and national achievement
tests. There are test items that are not face validated and content validated. This also
affects the performance of the students in the division achievement tests. Results of
the Chemistry achievement test vary from one type of school to another. The special
science high schools performance is far better than both the special school and
regular high school. Against that, the graphs of the difficulty index for the special
school and regular high school are bell-shaped normal curve, meaning, the scores
are still equally distributed. To improve the performance of Filipino high school
students in international competency- based examinations, the following are
recommended:

82
a. Test items in the division and national achievement tests should be face and
content validated;
b.The number of test items in the division and national achievement tests for
reasoning and analysis should be augmented;
c. Classroom instruction should encourage creative and critical thinking; and
d. Teachers should be trained in using classroom instruction that encourages
creative and critical thinking.
References
Abad, F. (2005). Why the crisis in education. BizNews Asia, 3(26), pp. 8-12.
Colorado State University (2010). Retrieved May 18, 2010 from

http://writing.colostate.edu/guides/research/relval/com2b2.cfm
Constitutions of the Philippines (2005). Manila: Anvil Publishing.
Department of Education (2002). Basic Education Curriculum Operations Handbook.

Manila, Philippines: Department of Education.
Division of City Schools Quezon City (2010). National Achievement Test. Retrieved
May 17, 2010 from http://www.cityschoolsqc.ph/innovation.asp
Martin, M.O., Mullis, I. V.S., Gonzales, E.J., Gregory, K.D., Smith, T.A., &
Chrostowski, S.J. (2004). TIMSS 2003: International science report; findings from IEAs
report of the Trends in International Mathematics and Science Study. Chestnut Hill, MA:
The International Study Center, Lynch School of Education, Boston College.
Mullis, I.V.S., Martin, M.O., Smith, T.A., Garden, R.A., Gregory, K.D., Gonzales, E.J.,
Chrostowski, S.J. (2003). TIMSS Assessment Frameworks and Specifications 2nd Ed.
Boston: TIMSS International Study Center, pp. 63-68.
Ramirez, Rachel Patricia and Mildred Ganaden (2008). Creative Activities and
Students Higher Order Thinking Skills. Quezon City: Education Quarterly, Volume 66
(1), pp. 22-23.
Tatum, Malcolm (2010). What is an Achievement Test. Retrieved May 16, 2010 from
http://www.wisegeek.com/what-is-an-achievement-test.htm

83

Vol. 4, No. 1, pp. 83-166, April 2014
Towards Developing a Proposed Model of Teaching-

Learning Process Based on the Best Practices in Chemistry
Laboratory Instruction
Paz B. Reyes
Lyceum of the Philippines University, Batangas City, Philippines
Rebecca C. Nueva Espaa

College of Graduate Studies and Teacher Education Research, Philippine Normal University,
Manila, Philippines
Rene R. Belecina
College of Graduate Studies and Teacher Education Research, Philippine Normal University,
Manila, Philippines
Abstract. This study investigated the teaching practices employed by the faculty of the
Lyceum University System in teaching chemistry laboratory in order to attain the seven
goals of laboratory instruction: (2) mastery of subject matter; (b) scientific reasoning; (c)
understanding complexity and ambiguity of empirical work; (d) practical skills; (e)
understanding the nature of science; (f) interest in science and in learning science; and (g)
teamwork skills. It also determined the extent by which the attainment of the goals of
science laboratory instruction was manifested in the students (a) attitude and
motivation; (b) laboratory skills; and (c) achievement. Finally, a proposed model of
teaching-learning process in chemistry laboratory instruction was developed based on
the identified best teaching practices. The qualitative-quantitative methods of research
particularly the descriptive design were used. To gather data, interview was conducted
to separate groups of students and faculty. Further, classroom observations and
questionnaires were conducted and administered to gather other pertinent data. The
subjects of the study were eighty students enrolled in General Chemistry during the
second semester of the school year 2011-2012 and 4 chemistry instructors. The chemistry
instructors were chosen from each of the four universities included in the Lyceum
University System. With the aim of determining the best teaching practices employed by
the faculty in teaching chemistry laboratory, five instruments were developed and
validated by experts: Focus Group Interview Questionnaire for faculty and for Students;
Observation Checklist; Attitude/Motivation Instrument; Practical Test; and the
Achievement Test. The data analysis made use of frequency, percentage, mean and
standard deviation. The results of the study revealed that the teaching practices of the
chemistry faculty of the Lyceum University System were based on the university vision,
mission goals and objectives and therefore attained the seven goals of the science
laboratory instruction. Likewise, the students acquired a positive attitude towards
chemistry, high competency in laboratory skills and average level of achievement in the
subject. It can be deduced from the findings that indeed the best practices of the faculty

84
in teaching chemistry laboratory are those practices where students engaged in

experiential learning, active learning, meaningful learning, and cooperative learning. It
was manifested in the students attitude/motivation, laboratory skills and achievement
as shown by their interest in chemistry and in learning chemistry, their cognitive and
manipulative skills and their understanding of the concept. The use of the proposed
model of constructivist teaching-learning process is recommended for an effective
chemistry laboratory instruction.
Keywords: best practices in chemistry laboratory instruction, teaching-learning model in

chemistry
Introduction
Science and technology play a major role in mans quest for quality of life which subsequently
causes a great impact to society. Science is relatively an experimental field, and most of the time
learning its concepts and skills happen in the laboratory. Investigating scientific phenomena
and testing hypotheses begin with making observations and gaining reasons for or describing
observed situations. As such, it is the supreme art of the science teacher to awaken a childs
curiosity and enkindle the eagerness to explore, to search for knowledge, truth and harmony.
To Petrucci, Herring, Madura & Bissonnette (2002), inculcating scientific discipline among
learners reflect a response to a higher goal of learning. With different governments in the world
trying to redefine and fine tune education, it is therefore imperative to develop more capacity
building in science and technology. In the Philippines, for instance the 1986 Constitution
provides support for science and technology. Article XIV reads: Science and Technology are
essential for national development and progress. The state shall give priority to research and
development, invention, innovation and their utilization; and to science and technology education,
training and services. It shall support indigeneous, appropriate, and self reliant scientific and
technological capabilities, and their application to the countrys productive system and national life.
Among the different scientific discipline, Chemistry is regarded as an active and continually
growing science and is focussed on realms of both nature and society. Chang (2009) posited that
Chemistry is every bit a modern science, an experimental science that rests on a foundation of
precise vocabulary and established methods. Following this line of thinking, chemistry must
be concerned not only with the teaching of concepts, but also of laboratory skills. Therefore, to
study chemistry is to understand how concepts are translated into application in a laboratory
setting.
Among the scientific changes in the concept of chemistry teaching in the last decade is the
emphasis of laboratory work as an efficient and meaningful technique in learning science
concepts. This change is patterned on Salandanans (2002) definition of experiment which is a
mean of illustrating the basic concepts of science and giving a clear view of the topics studied in
class. Wink, Gislason, & Kuehn (2000) further validated that through experimentation,
knowledge can be formed and in some instances erroneous beliefs that have been passed down
by authority can be discarded.

85
Still, chemistry is not complete without laboratory works because it is where students can
discover things for themselves, where they can be actively involved in identifying and using
varied chemicals
With the onset of modern learning styles and modes of education, it is imperative to consider
the different learning tasks in teaching college chemistry to make the system more responsive to
the demands of the 21st century. In fact, at the collegiate level, all students should have
opportunities to experience more meaningful science laboratory investigations. Such laboratory
experiences should aim to address how students should be taught how to work independently
and collaboratively as well as incorporate and critique scientific studies published. Moreover,
laboratory experiences must teach students how to develop scientific reasoning and appropriate
laboratory techniques to define and solve problems, and finally, to draw and evaluate
conclusions based on quantitative evidences. Laboratories should correlate closely with
lectures and should not be separate activities. This fact reflects that exposure to rigorous,
inquiry based laboratories at the college level duplicates the same experience the science
teachers had when they took their undergraduate studies.
On the part of the educators, it is not enough for chemistry teachers to simply give facts, figures,
concepts, theories, laws and other data, but they should be concerned with incorporating new
teaching methods into their laboratory activities and development of courses with more realistic
expectations of student involvement in experimental designs, data analyses and data
interpretations. Linking laboratory activities which the students really enjoy provides a wider
span of meaningful learning and development for both teachers and students (National Science
Teachers Association, 2007). To achieve such meaningful learning, laboratory instruction must
be designed in a way that it will develop not only the cognitive but also the manipulative skills
of students. Again, the argument remains the same, it is not enough for students to learn the
concept, it must be taught together with the process.
As active participants in science laboratories, students gain a deeper sense of understanding

and a greater confidence in their learning. With the acknowledged importance of a laboratory
experience for all students, it is necessary for instructors to think clearly about the elements that
could help achieve an effective laboratory experience. For instance, it is of great importance to
know what techniques can be utilized to encourage students to confidently contribute to their
laboratory groups.
Corollary to this, which scientific skills and procedures must be practiced and mastered by
students to achieve that level of confidence during laboratory works. Still another important
point to emphasize are the kinds of instruments in a science laboratory which students should
be familiar with. Similarly, understanding the importance of the other laboratory skills such
as communication (written and oral), teamwork, ethics, fairness, and responsibility should be
taught to maximize students participation in performing experiments.
In a study conducted by Narayan (2005), she suggested that for students to be engaged in
science, they needed to be involved in learning to use language, think and act in ways that
enable one to be identified as a member of the scientific literate community and participate in

86
the activities of that community. She added that learning occurs more effectively when the
student is socialized into a community of practice that he is immersed in.
Most science educators encourage fellow teachers to provide students with access to more
authentic science activities. Queries on possible steps to take on how to improve the delivery of
science lessons and skills remains to be the primary objective in redevelopment and
restructuring of pedagogical practices in science classrooms
The Lyceum of the Philippines University, one of the countrys premier institutions, set the
standards of commitment in pursuing excellence in education. Guided by its vision, mission
and core values, the university offers various science programs which include laboratories as an
integral part of the curriculum.
The researcher, as a science educator, is concerned with the meaningful learning of students in
chemistry. With her several years of teaching chemistry laboratory, the issue on what strategies
to use to help students develop positive attitude towards chemistry and become independent
learners who are ready to face the challenges of the 21st century science education, has been her
problem. Thus, the researcher attempted to investigate the best teaching practices that will
focus on the attainment of the goals of chemistry laboratory instruction which aims to develop
positive attitude and high motivation of students as well as competency on their laboratory
skills which could lead to high probability of achievement.
Conceptual Framework
This study is anchored on Piagets Theory of Constructivism which encourages learning

through collaboration and interchange among the students themselves. Piaget (Muijs and
Reynolds, 2011) suggested that students construct new knowledge from their experiences
through accomodation and assimilation. Constructivism as a learning theory views learning
as a process in which students actively construct or build new ideas and concepts based upon
prior knowledge and new information. Further, it suggests that instruction should follow
some basic principles such as; (1) children should be allowed to make mistakes and correct
these on their own thereby enabling them to accommodate, assimilate and reconstruct
knowledge on their own; discovery learning is emphasized; (2) the process of experimentation
at all stages is important; and (3) knowledge is always a construction by the learner which
involves operative processes that lead to transformation of reality, either in action or thought
therefore experimentation should be done continually. The constructivist teacher encourages
students to discover principles and construct knowledge within a given framework or structure
by helping students connect with prior knowledge and experiences while new information is
being presented. Through constructivism students can dispense their misconceptions and build
a correct understanding.
Constructivism is a conceptual basis of this study because practices in chemistry laboratory

instruction if tailored on the elements of constructivism will lead to the attainment of the seven
goals of science laboratory instruction and will develop positive attitude, competency in
laboratory skills and high achievement of students

87
In Shulmans view as cited by Rowan, Schilling, Ball & Miller ( 2011), the trend in education is
one that addresses the pedagogical content knowledge (PCK). PCK is a form of practical
knowledge that entails, among other things: (a) knowledge of how to structure and represent
academic content for direct teaching to students; (b) knowledge of the common conceptions,
misconceptions, and difficulties that students encounter when learning particular content; and
(c) knowledge of the specific teaching strategies that can be used to address students learning
needs in particular classroom circumstances. PCK is concerned with the representation and
formulation of concepts, pedagogical techniques, knowledge of what makes concepts difficult
or easy to learn, knowledge of students prior knowledge and theories of epistemology. It
further views the knowledge of what the students bring to the learning situation, knowledge
that might be either facilitative or dysfunctional for a particular learning task at hand. This
knowledge of students includes their strategies, prior conceptions (both "nave" and
instructionally produced); misconceptions students are likely to have about a particular domain
and potential misapplications of prior knowledge. PCK represents the blending of content and
pedagogy into an understanding of how particular aspects of subject matter are organized,
adapted, and represented for instruction. Finally, Rowan, et al. (2011) argued that "pedagogical
content knowledge" reflects the content knowledge that deals with the teaching process,
including "the ways of representing and formulating the subject that make it comprehensible to
others. In a larger vantage and scope, therefore, a constructivist chemistry teacher implement
the best teaching practices in chemistry laboratory if he/she has a knowledge of both content
and pedagogy. This idea makes not only constructivism but also pedagogical content
knowledge as the conceptual bases of this study.
Studies have revealed that there are seven goals of laboratory instruction in Science Education
(Singer, Hilton, & Schweingruber, 2005 and Jona, Adsit & Powell, 2008). These goals include a)
enhancing mastery of subject matter b) developing scientific reasoning c) understanding the
complexity and ambiguity of empirical work d) developing practical skills e) understanding
the nature of science f) cultivating interest in science and interest in learning science and g)
developing teamwork skills. These goals were achieved in the classroom, according to Jona, et
al. (2008) if a student shows mastery of subject matter by readily remembering and
understanding the concepts taught. On the same level, if a student manifests the ability to
apply the knowledge acquired, then the student has mastery of subject matter. Scientific
reasoning is manifested in the students ability to explain, predict and control the occurrence of
events. Students understand the complexity and ambiguity of empirical work if they can
address the challenges inherent in directly observing and manipulating the material world.
Practical skills are developed if students can use scientific equipment correctly and safely, make
observations, take measurements and carry out well-defined scientific procedures. The nature
of science is being understood if students can interpret data from the material world and they
can discover that different people may interpret the same data differently. Interest in science
and interest in learning science may be reflected from the positive attitude and high motivation
of students. Teamwork skills are developed if students have the ability to collaborate effectively
with others. To achieve all these goals, therefore, teachers must implement practices to address
positive attitude, high motivation, competency in laboratory skills and high achievement in the
subject.

88
Figure 1 shows the conceptual paradigm of the study. The study focused on the investigation
of the best practices in teaching chemistry laboratory with the purpose of attaining the seven
goals of laboratory instruction in science education.
Chemistry Laboratory Instruction must be studentcentered which is focused on how students

are motivated to acquire positive attitudes towards chemistry, competencies in lab skills and
high achievement. To achieve this goal, teachers must integrate the seven goals of science lab
instruction in their practices to address a student-centered classroom setting.
Teaching practices to attain:
Mastery of
subject matter
A Model
Scientific
reasoning Of
Understanding Students:
complexity and Teaching-Learning
Attitude/
ambiguity of
motivation Process
empirical work
Laboratory skills
Practical skills
Achievement In
Understanding
nature of science Chemistry Laboratory
Interest in science
and learning Instruction
science
Teamwork skills
Figure 1. Conceptual Paradigm of the Best Teaching-Learning Process in Chemistry Laboratory

Instruction
The first box in the paradigm reflects the teaching practices of chemistry laboratory teachers
intended to attain the seven goals of science laboratory instruction. These goals include
enhancing mastery of subject matter, developing scientific reasoning, understanding the
89
complexity and ambiguity of empirical work, developing practical skills, understanding the
nature of science, cultivating interest in science and interest in learning science, and developing
teamwork skills. The second box on the other hand, shows the goals of science laboratory
instruction are achieved by the correct pedagogical practices of the faculty. Towards the end, a
model of teaching-learning process in chemistry laboratory instruction was developed from the
investigated best teaching practices leading to the attainment of the seven goals which are then
manifested in the attitude, motivation, laboratory skills and achievement of students.
Statement of the Problem
The main objective of the study is to propose a model of a teaching-learning process based on
the identified best practices in chemistry laboratory instruction.
Specifically, the study sought answers to the following questions:

1. What are the teaching practices employed by the faculty in teaching chemistry
laboratory in order to attain the seven goals of science laboratory instruction?
1.1 mastery of subject matter
1.2 scientific reasoning
1.3 understanding of the complexity and ambiguity of empirical work
1.4 practical skills
1.5 understanding of the nature of science
1.6 interest in science and interest in learning science
1.7 teamwork skills
2. To what extent do students manifest the attainment of the goals of science laboratory
instruction in some student-related parameters?
2.1 attitude and motivation
2.2 laboratory skills
2.3 achievement
3. Based from the findings of the study, what model of teaching-learning process in
chemistry laboratory instruction may be proposed to attain the goals of science
laboratory instruction?
Goals of Laboratory Instruction in Science Education.
A better science program is said to be that which includes laboratories and other forms of
scientific investigations. Scientific investigations must be conducted in accordance with the
goals of laboratory instruction in science education. Singer, et al. (2005) reported in the
Americas Lab Report of the National Research Council the seven goals for scientific
investigation as the desired targets of a comprehensive science program. These goals, he added
should be taught in every science laboratory and these include the attainment of a) mastery of
subject matter b) scientific reasoning c) the understanding of the complexity and ambiguity of
empirical work d) practical skills e) the understanding of the nature of science f) interest in
science and interest in learning science g) teamwork skills. Jona, et al. (2008) discussed further
each of the seven goals of scientific investigation.

90
Mastery of subject matter. Similar to other disciplines, science aims to teach both established
facts and concepts (content) and the skills used by professionals in that field (process). The
National Research Council (2005) found that in typical programs, content and process are
taught separately. However, modern educational theorists view them as related educational
goals because according to Newmann and Wehlage as cited by Jona, et al. (2008), kinesthetic
activities and other active learning experience help students in learning the content of the
subject matter. Therefore, mastery of subject matter could be attained if concept and process are
taught simultaneously. This goes exactly the opposite of typical laboratory experiences where
students perform a process without a clear understanding of the relation of that process to
content. For example, when students perform titration they do not understand completely why
they are doing it or they cannot explain the results in terms of scientific concepts.
Furthermore, typical laboratories attempt to demonstrate scientific concepts by making students

follow set processes in a recipe type just to confirm something that has been already taught.
Inquiry activities, on the other hand, that include manipulation of ideas rather than materials
and procedures enhance student understanding of facts and concepts.
Corollary to this, an integrated learning program was proposed by the National Research
Council (2005). This program makes use of a constructivist approach which according to
Teachnology (2007) attempts to make students observe and draw conclusions about concepts
prior to receiving explicit instruction. The program consists of instructional design which will
improve student mastery of subject matter. This includes the close integration of investigative
activities into content, a merging of content instruction and process instruction, and reflection
on the meaning of the learning activity once it is completed.
Scientific reasoning. Students should be taught the various kinds of scientific processes and
valid reasoning principles and at the same time must be given the opportunity to practice these
reasoning skills. To achieve this, laboratory instructions must be planned so that students can
be encouraged to participate in designing the process of investigation, making them draw and
support conclusions. In a direct contrast, a typical science course, students do not develop
scientific reasoning skills because they were not given time for planning investigation or
interpreting results. Experts take such scenario due to the focus of instruction only on learning
content and laboratory experiences focuses upon following specified procedures.
With the integrated learning program, a variety of skills associated with scientific reasoning can
be developed among students. According to the National Research Council (2005), these include
the ability to identify questions and concepts leading to scientific investigations, design and
conduct scientific investigations, develop and revise scientific explanations and models;
recognize and analyze alternative explanations and models, and make and defend a scientific
argument, including writing, reviewing information, using scientific language appropriately,
constructing a reasoned argument, and responding to critical comments.
It is very well expected that a well designed science course should consider a core scientific
process which deal with the ability of students to construct scientific arguments. In this process,
students must be taught to design experiments, make predictions, interpret and explain data,
recognize discrepancies between predicted and observed outcomes, design good experiments.

91
Instructional practice is effective only when students learned how to relate theoretical claims
with evidences gathered from their laboratory investigations.
Understanding of the complexity and ambiguity of empirical work. To Singer, et al. (2005),
scientific investigations should be properly designed in such a manner that students are able to
expect outcomes or experimental results contradicting widely accepted scientific principle.
Students must understand that even the same experiments may lead to different results if
performed at different times or by different people. Similarly, students should not be confused
with the misconceptions that science is a collection of clearly defined laboratory procedures
whose outcome firmly support received instruction, instead students must know how to deal
with these complexities and ambiguities of empirical work as one aspect of the nature of
science.
Researchers believe that a well designed scientific investigation program must include
opportunities for students to be involved in activities like troubleshooting of laboratory
equipment, rechecking data observations and analysis, examining the parameters, assumption
and study definitions in contradictory studies, and generally performing the kind of follow-up
investigations done within the scientific community. Further, such program must allow
students to understand measurement error and interpret and aggregate the resulting data.
One technique for an instructional designer is to allow students who are working in a team to
perform activities independently, compare results and then discuss and account for
discrepancies. Students must be allowed to make mistakes and correct them on their own.
Experimental errors are not hindrances to learning, but they are opportunities for greater
learning. So instead of working hard to remove complexities and ambiguities, laboratory
instructors should include the expectation of experimental errors in their instruction.
Practical skills. Although practical skills refer to the proper use of scientific equipment and the
conventions of science such as measuring, observing and other science processes, it is not
enough for students to know how to use tools and follow correct procedures in scientific
investigations. Rather, what is important is they know how to apply effectively the appropriate
scientific processes to a new investigation so that they can make accurate observations and
follow accepted procedures to ensure valid results.
Understanding the nature of science. In a typical science course, students do not realize that
science is a human endeavor that seeks to understand the material world and that scientific
theories, models and explanations change over time on the basis of new evidence. They simply
see that science is a collection of laws and facts without really understanding how existing
concepts came into being, how existing ideas are reshaped with new discoveries, how accepted
theory differs from wild guess or from firm facts, and how new concepts and theories emerge
through investigations.
In an integrated learning program, laboratory teachers should explicitly teach concepts in the
instructional phase and aim to reinforce the understanding of the concepts through an
investigative process. To achieve this, various instructional strategies, such as constructivist

92
activities and activities that allow students to create their own scientific investigations to solve
problems, must be implemented in their instruction. Students should be given metacognitive
assignments which will allow them to reflect on their learning and to relate their experiences to
scientific principles and procedures.
Interest in science and interest in learning science. Instruction is considered effective if it

cultivates an interest in the subject and motivates students to continue learning more about the
subject. This is made possible in science laboratory instruction by applying the five principles
of authentic instruction. Scientific investigations and other laboratory activities are applications
of authentic instruction which are consistent with the integrated learning process. The five
principles of authentic instruction in science include: a) higher order thinking skills b) depth
of knowledge c) connectedness to the world beyond the classroom d) substantive conversation
and e) social support for student achievement. According to Fraser as cited by Jona, et al.
(2008) an extensive study involving multiple countries (including the United States) indicated
that positive student attitudes toward science are strongly associated with cohesiveness (the
extent to which students know, help, and are supportive of one another) and integration (the
extent to which laboratory activities are integrated with non-laboratory and theory classes).
Teamwork skills. To National Research Council (2005), scientific investigations promote a

students ability to collaborate effectively with others in carrying out complex tasks, to share the
work of the task, to assume different roles at different times, and to contribute and respond to
ideas. It is important in a learning community to have teamwork and collaboration among
members. A well-designed collaborative authentic instruction can enhance student learning in
contrast to a poorly designed collaborative process that undermine instruction and student
achievement. Teamwork skills therefore, must be integrated as part of the instructional process
by introducing investigate processes early in the course.
Still in the same argument, teamwork is a part of authentic instruction where substantive
conversation requires interaction among members. There is high level of substantive
conversation if 1) there is considerable interaction about the ideas of a topic (the talk is about
disciplined subject matter and includes indicators of higher-order thinking such as making
distinctions, applying ideas, forming generalizations, raising questions, and not just reporting
experiences, facts, definitions, or procedures) 2) there is sharing of ideas which is evident in
exchanges that are not completely scripted or controlled (as in a teacher-led recitation). Sharing
is best illustrated when participants explain themselves or ask questions in complete sentences
and when they respond directly to comments of previous speakers 3) The dialogue builds
coherently on participants ideas to promote improved collective understanding of a theme or
topic.
In a true collaborative work, students share ideas about hypotheses, procedures and
conclusions, directly contradicting how students in a typical laboratory experience work in
group to divide limited laboratory equipment and space among a large number of students.
The Role of Laboratory Instruction in Science Education

The National Science Teachers Association (NSTA, 2007) defined school laboratory
investigations as an experience in the laboratory, classroom, or the field that provides students
93
with opportunities to interact directly with natural phenomena or with data collected by using
tools, materials, data collection techniques, and models. From these investigations, evidences
are collected through observations which become the basis in generating scientific theories and
scientific laws. In the entire process of investigations, students are expected to acquire skills
and knowledge such as the ability to design investigations, engage in scientific reasoning,
manipulate equipment, record data, analyze results, and discuss their findings. To achieve this
purpose of instruction, National Science Teachers Association (2007) recommend the inquiry-
based laboratory investigation which is the process of asking questions and conducting
experiments as a way to understand the natural world.
Inquiry-based laboratory investigations provide instruction with a priority on making

observations and gathering evidence, much of which students experience in the lab or the field,
to help students develop a deep understanding of the science content, as well as an
understanding of the nature of science, the attitudes of science, and the skills of scientific
reasoning. To address this, integration of inquiry-based laboratory investigations in the science
lesson in every level of education starting from preschool to higher education should be
achieved. As students move up to higher grades, the level of complexity of laboratory
investigations should also increase. In the preschool and elementary level, students should be
given opportunities to investigate appropriate questions, analyze the results of laboratory
investigations, debate what the evidence means, construct an understanding of science
concepts, and apply these concepts to the world around them.
As the students move up to the high school level, students should develop a growing
understanding of the complexity and ambiguity of empirical work, as well as the skills to
calibrate and troubleshoot equipment, understand measurement error; and have the skills to
aggregate, interpret, and present the resulting data. They should also improve their ability to
collaborate effectively with others in carrying out complex tasks, share the work of the task,
assume different roles at different times, and contribute and respond to ideas.
At the tertiary level, students must learn how to work independently and collaboratively,
incorporate and critique the published work of others in their communications, use scientific
reasoning and appropriate laboratory techniques to define and solve problems, and draw and
evaluate conclusions based on quantitative evidence.
To Domin (2009), all the expected outcomes are possible in science education. Learning about
the methods and processes of scientific research (science process) and the knowledge derived
through this process (science content) are expected as a well developed science education
curriculum.. Science process involves direct interactions with the natural world in order to
explain natural phenomena. Further, science education should include opportunities for
students to learn about both the process and content of science and this is possible only through
laboratory experiences.
Laboratory Experiences and Student Learning
The Committee on High School Laboratory, as cited by Singer, et al. (2005) in Americas Lab
Report, pointed out the importance of laboratory experiences of students in attaining the seven
94
goals of laboratory instruction in science education. In typical laboratory experiences, students

are engaged in one or two experiments followed by assessment to determine whether their
understanding of science concept had increased. The Committee on High School Laboratory
recommended integration of laboratory experiences into instructional sequences in order to
help students progress toward science learning goals. In this way, students learning about the
concepts and processes of science are also integrated. This integration is referred to by the
committee as the integrated instructional units which is designed to be learner-centered.
Integrated instructional units believed that effective instruction begin with what learners bring
to the setting including cultural practices and beliefs, as well as knowledge of academic content.
Students based their preconceptions of the natural phenomena on their everyday experiences in
the world. These preconceptions are often reasonable and can provide satisfactory everyday
explanations to students, but they do not always match scientific explanations thereby
considered as intuitive ideas. Teachers are challenged with these intuitive ideas of students,
they are challenged to help students move towards a more scientific understanding through
change in and not merely an addition to what students notice and understand about the world.
The principle behind the integrated instructional units is that learning is enhanced when the
environment is knowledge-centered. In the knowledge-centered environment, students learn
with understanding rather than simply acquiring sets of disconnected facts and skills. There are
two bodies of knowledge in science with which students must be engaged to one is
knowledge of accepted scientific ideas about natural phenomena and the other is understanding
of what it means to do science. These two aspects of science are reflected in the goals of
laboratory experiences, which include mastery of subject matter (accepted scientific ideas about
phenomena) and several goals related to the processes of science (understanding the complexity
of empirical work, development of scientific reasoning). Student thinking about science shows
a progression of ideas about scientific knowledge and how it is justified. At the first stage,
students perceive scientific knowledge as right or wrong. Later, students characterize
discrepant ideas and evidence as mere opinion. Eventually, students recognize scientific
knowledge as being justified by evidence derived through rigorous research.
Metacognitive strategies when implemented in a knowledge-centered environment will enable

students to reflect on their own learning progress, to identify, monitor and regulate their own
thinking and learning which in turn will facilitate their learning. To be effective problem
solvers and learners, students need to determine what they already know and what else they
need to know in any given situation, including when things are not going as expected. The basic
metacognitive strategies include: (1) connecting new information to former knowledge, (2)
selecting thinking strategies deliberately, and (3) monitoring ones progress during problem
solving.
Furthermore, in a knowledge-centered learning, the practices and activities in which people

engaged in while learning, shape what they learn. Transfer (the ability to apply learning in
varying situations) is made possible to the extent that knowledge and learning are grounded in
multiple contexts such as what transpires in the laboratory. Through multiple contexts, students
can develop a deeper understanding of the concept and its use aside from they can acquire the
ability to transfer what has been learned in one context to others.
95
Learning is enhanced in a community setting, when students and teachers share norms that
value knowledge and participation (Cobb, Stephan, Clain & Gravemeijer, 2001). Such norms,
Cobb further argued, increase peoples opportunities and motivation to interact, receive
feedback, and learn. It is said learning is enhanced when students have multiple opportunities
to articulate their ideas to peers and to hear and discuss others ideas. Such scenario can be
achieved in integrated instructional units which are combined laboratory experiences and other
types of science learning activities which may include lectures, reading, and discussion. If a
classroom addresses quality norms, students are given more opportunities to frame their own
research questions, design and execute experiments, gather and analyze data, and construct
arguments and conclusions as they carry out investigations thereby making them independent
learners. On the other hand, diagnostic and formative assessments are embedded into the
instructional sequences and can be used to gauge students developing understanding and to
promote their self-reflection on their thinking.
The National Research Council (2005), in its report considered four principles of instructional
design that can help laboratory experiences achieve their intended learning goals. These
principles are (1) instructions must be design with clear learning outcomes in mind, (2) they
must be thoughtfully sequenced into the flow of classroom science instruction, (3) they must
integrate learning of science content with learning about the processes of science, and (4) they
must incorporate ongoing student reflection and discussion. Combined with the seven goals,
these principles offer better chances for students to experience worthwhile laboratory
experiences. They provide a framework for curriculum developers, administrators, and teachers
to use in reconsidering how laboratory experiences can be successfully incorporated into science
courses.
Laboratory Instruction in Chemistry
As an experimental science, Chemistry depends heavily on experimental work as a strategy for

teaching scientific principles and concepts and its development and application demand a high
standard of performance of laboratory activities. Other than the expected results in student
learning, laboratory activities allow students to appreciate and experience the constraints,
potential and tensions of an investigative process which can only be experienced in the
laboratory.
The laboratory is the most attractive place for the students to develop and show applications for
general principles and techniques, says Wink, et al. (2000). It allows students to experience how
the solution of real problems by people in all walks of life requires a thorough understanding of
general chemistry principles. Bishop, Bishop & Whitten (2000) pointed out that the best way to
acquire a deep, clear understanding of the nature of chemistry is in hands-on laboratory
experiments with the real chemicals and real equipment which chemists use.
There are four reasons, according to Zulueta and Guimbatan (2002) for using the laboratory as a
method of instruction in science, in particular, in teaching chemistry. Laboratory instruction in
chemistry gives opportunities to students to manipulate concrete objects; participate actively;
develop scientific competencies and motivation. Science involves the learning of highly
complex and abstract subject matter. By allowing the students to have hands-on experience,
they understand and use scientific principles learned from the opportunity to manipulate actual
96
objects and materials. Further, participating in a laboratory exercise gives students an

appreciation of the methods of science and promotes problem-solving and other analytic
competencies that can be generalized and applied to other areas. Students enjoy goal-oriented
activities and practical work where they can see the relevance of abstract concepts and
principles; and consequently, become interested in sciences and are motivated to learn more
about discipline.
In addition, Walton as cited by Corpuz, Rimas, Galangco & Bautista (2003), gives the aims of
laboratory method as to give firsthand experience in the laboratory which may increase student
interest; to provide student participation in original research and to develop skills in the use of
laboratory equipment and instruments.
Best Practices in Science Teaching

The National Research Council (2005) identified pedagogical practices which are considered
truly best practices or authentic best practices in science teaching. Used in the science
classrooms, there are significant evidences that show how these practices help students in
learning better. These authentic best practices include: 1) Engaging resilient preconceptions.
Students upon coming into the classroom have already initial understanding and
preconceptions about the topic to be discuss. These preconceptions often limit what a student
can learn so that it is important for a teacher to identify, confront and resolve this initial
understanding. 2) Organizing knowledge around core concepts. All these best practices aim to
increase understanding and retention of concepts among students by carefully and scientifically
organizing information. For instance, students can readily remember a concept if they are
taught how to recognize a certain pattern. In this way, teachers provide a foundation of factual
knowledge and conceptual understanding to students. 3) Supporting metacognition and
student self-regulation. This is making students assess themselves as to what they know and
what they dont know. This could be done by requiring students to make a reflection that
summarizes what they have learned or by administering a pre-test. In this way, students can
take control of their learning.
On the other hand, Minstrell and Kraus (2005) enumerated the so-called best practices in science
teaching which are based on ideology rather than on the findings of empirical research. These
ideal practices which are often closely associated with students success are:
Establishing and maintaining classroom environments that are:
o learner centered -- identifying, confronting, and resolving preconceptions, and

beginning instruction with what students know.
o knowledge centered -- focus on how something is know as much as what is
known, and provide examples of what mastery looks like.
o assessment centered -- make frequent attempts to make students' thinking and
learning visible as a guide for further instruction.
o community centered -- encourages a culture of questioning, including a bit of
risk taking and respect for others
97
Using an empirical approach

Regularly employ active learning strategies
Employ inquiry labs
Talk about the nature of science
Provide meaningful, engaged learning for all students.
Provide an active approach to learning that includes a strong emphasis on student
interaction with phenomena.
Clear and explicit linkage between representations and phenomena represented.
Engage students in challenging, authentic, interdisciplinary tasks.
Provide opportunities for students to observe, explore, and test hypotheses.
Eliminate discipline boundaries when natural, logical, and appropriate.
Encourage the students' imagination, logic, and open-mindedness.
Incorporate the content and processes of science giving due regard to science teaching
standards.
Give due regard to affective as well as cognitive domain.
Link scientific concepts and processes with prior learning in science and other
disciplines.
Using a constructivist approach.
Depth and breadth of coverage are reasonably balanced.
Goals of tasks are conceptual and conceptual means are required to accomplish them.
Assigning manageable tasks
Setting high expectations
Engage all learners in meaningful scientific tasks involving high-order thinking skills.
Providing and receiving feedback
Accommodating student learning styles
Teaching in a way that is consistent with student development
Including real-world applications in the learning process
Using individual and group motivation
Moving from concrete to abstract
Requiring practice of learned skills
Employing learning cycles - observation, generalization, verification, application
Making use of multiple intelligences
Establishing conducive learning environments
Encouraging student evaluation of alternative hypotheses
Addressing conceptual goals and means
Eliciting and addressing misconceptions
Promoting critical thinking
Creating, sharing, and using scoring rubrics
Aligning objectives, instruction, and assessment
Focusing on depth in addition to breadth of coverage
Placing strong emphasis on interaction with phenomena
Making clear and explicit linkage of representations to phenomena
Using multiple representations of physical phenomena
Employing Socratic dialogues
Common Strategies in Chemistry Laboratory Instruction

98
Different approaches are employed by laboratory instructors in teaching chemistry laboratory,

but all these approaches have an end goal of making students enjoy, understand, participate
and develop skills. Aquino (2003) enumerated the three objectives of science teaching as the
development of science process skills, scientific attitudes, and literacy. On the other hand,
Salandanan (2002) stated that students should be able to achieve the different goals of science
teaching such as the development of scientific attitudes and values; enhancement of skills in
employing a systematic and scientific methodology; gaining an understanding of functional
knowledge; arousing further interest in science-based pursuits; and development of desirable
social attitudes.
In any teaching strategy in the laboratory, students must be given the opportunity to develop
critical and analytical mind. Teachers must engage in practices that will arouse the curiosity of
students by encouraging them to ask questions. Creativity and resourcefulness could be
inculcated into the students by stimulating them to generate new ideas and original ways of
doing things. Other wholesome attitudes should be develop by students through awakening
their interest and keeping them highly motivated to inquire about occurrence in the natural
environment. They must learn to make fair and unbiased decisions, accept evidences,
suggestions, and alternatives in the light of new discoveries. They must relentlessly pursue an
investigation and be responsible enough to complete an assigned task despite constraints. They
must have constant practice in experiencing step-by-step procedure to find answers to their
endless questions.
Encouraging students to participate actively in planned experiments might be daunting for the
students, however it will definitely enable them to acquire functional knowledge which can be
applied in solving problem situations in the environment instead of knowledge which is merely
memorized and easily forgotten. Another best practice in the teaching of science is the
treatment of life experiences as necessary tools to motivate students to participate in classroom
activities. These learning experiences might include joining movements in science promotions,
protection and care of the environment and natural resources, as well as helping decide on
issues of nationwide interest. Experts are saying that given the correct avenue to express and
explore these life experiences, there is a strong chance that students will decide to pursue a
science profession in the future and will develop a feeling of gratitude and appreciation for the
advances in science and technology that continue to raise the present quality of life. Further,
when students are allowed to experience more group investigations, positive attitudes could be
developed among students such as tolerance, respect for the opinions of others and willingness
to accept criticisms and suggestions, learning to cooperate with others, willingness to share
findings and resources and the readiness to extend expertise.
Laboratory Method. The laboratory method according to Acero, et al. (2000) deals with
experimentation, observation or application by individuals or small groups dealing with actual
materials. There are two types of laboratory method; the experimental and the observational
method. They differ in aims and emphasis in the sense that experimental method aims to train
pupils in problem-solving with incidental acquisition of information and motor skill while
observational method aims on the acquisition of facts. As to emphasis, experimental method is
focussed on discovery, original procedure, analysis, and solution of problems while in
99
observational the emphasis is on the acquisition of facts through activities such as visits to
museums, exhibits, and art galleries, watching demonstration, listening to lectures, viewing
films, and going on field trips.
Laboratory method consists of three steps: the introductory step, work period and the
culminating activities. In the process, introductory step aimed at orientation and motivation
where determination of work to be done is presented. The work period is defined as a
supervised period where students gain experience in the scientific procedure, handling raw
materials, and using tools while working on the same problem or on different problems on their
own. In the culminating activities, students discuss and organize their individual findings after
completion of the work. They present the results by a) explaining the nature and importance of
the problem the group had worked on; b) reporting data gathered on other findings; c)
presenting illustrative materials or special contributions; d) special reporting and exhibition of
work by those with individual projects; and finally, e) exhibiting various projects and
explanations by their sponsors.
To Hidalgo (2000), classroom strategy which uses laboratory method has an advantage over
other methods because students are learning by doing since actual experience is vivid. When
this happens, the learning gained is retained longer; reality is more vivid than any symbol; and
it is a direct preparation for a new way of life. Hidalgo (2000) gave suggestions to the
laboratory teachers on the better way of handling laboratory classes. For an efficient delivery of
lesson, Hidalgo (2000) opines that teachers should adapt laboratory exercises to the needs,
interest and capacities of students. To address reflected thinking, laboratory exercise must
grow out of problems so that a recipe-type activity is not recommended. Another good practice
for a laboratory teacher is to require students to keep a laboratory notebook where they can
record not only the results of their investigations but also the learning they got from the
experiment.
For his part, Domin (2009) described four types of laboratory instructions: expository
(traditional), problem based, discovery and inquiry-based. Domin (2009) explains that they
differ in outcome, approach and procedure. Expository, problem based and discovery
laboratories all have predetermined outcomes because he emphasized, the expected results are
already known. However, the limitation is pegged on the fact that it is only the teacher who
knows the outcome in the discovery and problem based unlike in the expository where both the
teacher and students know the expected result before doing any activity. In the inquiry-based
instructions, meanwhile, neither the teacher nor the students know the outcome of the
experiment. Given this, therefore, the choice of instruction is crucial to achieve life-long
learning among the students.
The study conducted by the National Science Teachers Association (2007) reveals that
laboratory investigations must be adapted to the age and ability levels of students. They should
not be the recipe-type activity that is somewhat related to the instructional sequence of the
topics discussed in the lecture. Well designed laboratory investigations are those in which the
objectives of the activity are clearly communicated to students and which focus on science
processes and integrate student reflection and discussion. Finally, the designed objectives give

100
the students the opportunities to develop safe and conscientious laboratory habits and
procedures.
Experimenting. An experiment (Salandanan, 2002) is described as a learning activity wherein

a student investigates a problem by manipulating a variable. Salandanan (2002) listed the
reasons why experiments are done: a) to develop basic science process skills; b) to cultivate an
inquiry mind; c) to acquire higher-order thinking skills like critical thinking, creativity and
inventiveness; d) learning by doing forms part of the students attitudes, habits, and ways of
reacting; e) to internalize and instantly apply in solving problem situations f) to replace hearsay,
superstitions, and unfounded beliefs by more objective assessment and evaluation; g) to make
students appreciate and be grateful for the achievements of scientists; and h) to make students
responsible for their own learning by completing the assigned tasks.
For an experiment to be successful, it is argued that the students must clearly understand the
problem; variables must be tested one at a time; students must participate actively in
manipulating tools, materials and equipment. Any absent component would mean that the
classroom will not be successful. On the part of the teacher, the teacher must be a keen
observant who can easily spot incorrect steps and procedure to be able to encourage the use of
improvised materials to promote resourcefulness and creativity and to underscore the
important elements of classroom setting.
Demonstration Method. Still, Hidalgo (2000) defined demonstration method as the planned
manipulation of materials and equipment to the end that students are able to observe all or at
least some of the manifestations of one or more scientific principles operating within a
phenomenon. Demonstration method differs from experimenting because it is content oriented
while experimenting is process oriented. It simply reinforces the previous learning and aims
toward a summary of ideas while experimenting aims to solve problems and gain new learning.
Therefore, Hidalgo (2000) summarizes, demonstration method is used when time and
equipment are limited and the process can be described as complicated or difficult while
experimenting is used when the purpose of the classroom is to develop resourcefulness among
students.
Demonstration method is considered as an excellent method to motivate and arouse the
interests of students in introducing any new lesson. On the other hand, Garcia as cited by
Acero, et al. (2000) called demonstration method as imitative method where learning a skill is
faster and more effective since the students are shown how the job is done by using the actual
tools, machines, and materials.
Discovery Method. The discovery method is a teaching strategy in which objectives help the
student to learn through self-discovery (Corpuz, et al., 2003). In this type of classroom, the
teacher prepares a class situation where students are led to find answers or solutions to a
problem on their own. Further, discovery method employs the inductive approach wherein the
teacher asks thought-provoking questions before performing an experiment to allow for self
introspection and analysis.

101
The discovery classroom, says Hidalgo (2000) demands for the teacher to cultivate among the
students an attitude of trying to solve problems on their own even if it would result in failure
than not trying at all. Hidalgo (2000) further suggested that teachers allow for independent
learning to allow for more learning.
With all these benefits an educator gets from the discovery method, Corpuz, et al. (2003)
opined that it is an extremely effective method. The most important gain it gives to the learners
is the feeling of satisfaction and joy for the students in discovering new learning and concepts.
Inquiry Approach. The goal of inquiry teaching according to Salandanan (2002) is to make
children learn how scientists learn, and in the process, learn science. Further, it aims to
encourage students to rely to a greater extent on their own resources. In fact, in using the
inquiry approach, learning leads to the attainment of one of the most significant outcomes of
science teaching- that is, developing a scientific mind while not undermining the desirable
social values. Students develop traits such as critical-mindedness, objectivity, and rationality
while engaged in an inquiry approach. They become more cooperative, tolerant, and
considerate in dealing with others because of constant involvement in group activities, thereby
making them highly motivated.
In his study, Straatman (2006) made use of inquiry methods in laboratory activities and
demonstrations along with traditional teacher-focused methods. A variety of data collection
methods were used to investigate changes that occurred in his approach to teaching chemistry
especially in relation to questioning strategies. The study revealed effects of inquiry techniques
on students problem-solving and logical thinking skills. His study enabled him to take a more
in depth look at how teaching methods affect student learning.
Process Method. Hidalgo (2000) defined process as a method of doing something; a

systematic and interdependent action of things related to a discovery approach where at the
end things are attained. In the science classroom, scientific activities are processes or methods
and the scientific information is the product of the process. Laboratory investigation, for
example, is a process of making a discovery where students become more actively involved.
Small Group Instruction. This method of instruction enables the teachers to give more
individual attention to each students learning needs. In small groups, the students become
more actively involved in their own learning and participate more freely in discussions.
(Hidalgo, 2000). In this method, students leadership is develop aside from they learn the skills
of discussion and group processes. This method enhances cooperation, team work, and group
motivation among students.
Principles of Best Practice Learning
Zemelman, Daniels & Hyde (2005) claims that there are classroom practices which need to be
enhanced to be implemented more often while there are those which should be implemented in
a lesser frequency. In a bigger concept, therefore, there must be less implementation of teacher-
directed instruction like lecturing; student passivity like sitting, listening, receiving, and
absorbing information; presentational or one-way transmission of information from teacher to
student; prizing and rewarding of silence in the classroom; classroom time devoted to fill-in-
102
the-blank worksheets, dittos, workbooks, and other seatwork; student time spent reading
textbooks and basal readers; attempts by teachers to thinly cover large amounts of material in
every subject area; rote memorization of facts and details; emphasis on the competition and
grades in school; tracking or leveling students into ability groups; use of pull-out special
programs; and use of and reliance on standardized tests.
In contrast, he recommended the frequent or more implementation of experiential, inductive,

hands-on learning; active learning, with all the attendant noise and movement of students
doing, talking, and collaborating; diverse roles for teachers, including coaching, demonstrating,
and modeling; emphasis on higher-order thinking; learning a fields key concepts and
principles; deep study of a smaller number of topics, so that students internalize the fields way
of inquiry; reading of real texts: whole books, primary sources, and nonfiction materials;
responsibility transferred to students for their work like goal setting, record keeping,
monitoring, sharing, exhibiting, and evaluating; choice for students (e.g., choosing their own
books, writing topics, team partners, and research projects); enacting and modeling of the
principles of democracy in school; attention to affective needs and varying cognitive styles of
individual students; cooperative, collaborative activity; developing the classroom as an
interdependent community; heterogeneous classrooms where individual needs are met through
individualized activities, not segregation of bodies; delivery of special help to students in
regular classrooms; varied and cooperative roles for teachers, parents, and administrators;
reliance on descriptive evaluations of student growth, including observational/anecdotal
records, conference notes, and performance assessment rubrics.
From those practices which he recommended to be implemented more and those which need to
be implemented less, he then identified thirteen principles characterizing a model education.
These principles are interrelated and are actually influencing each other. These practices, he
pointed out are student-centered, experiential holistic, authentic, challenging, cognitive,
developmental, constructivist, expressive, reflective, social, collaborative and democratic.
In a student-centered learning environment, teachers must consider the real interest of students,
taking their own questions into precedence over other selected content. At some point, it
involves building on the natural curiosity of students and asking them what they want to learn.
Teachers must guide their students in solving their own questions by structuring for them
widening circles of experiences and investigations. At this point, the teachers serve as
facilitators understanding deeply the needs and experiences of their students in order to design
enjoyable and engaging activities.
In an experiential learning, students are given the opportunities to experience the most
powerful and natural form of learning which is acquired through doing instead of just hearing.
Students must be engaged in active, hands-on, and concrete experiences such as conducting
experiments, going on field trips to investigate natural settings, pollution problems, and
laboratories at nearby factories, universities or hospitals.
If planned properly, holistic learning is possible when students encounter whole ideas, events,
and materials in purposeful contexts, instead of studying isolated subparts from actual use.
Information and ideas must not be presented to students in small building blocks because this

103
part-to-whole approach undercuts motivation for learning since students dont perceive why
they are doing such things. Students should not be deprived of an essential condition for
learning, that is encountering material in its full, lifelike context.
Linking learning to real life concepts is considered authentic teaching which integrates real, rich
complex ideas and materials in contrast to the lessons or textbooks that disempower students.
Experts all agree that learning becomes meaningful when students are faced with genuine
challenges, choices and responsibility while learning independently. There is cognitive learning
when students acquire true understanding of concepts using higher order thinking associated
with various fields of inquiry and through self monitoring of their cognition and mental
processes. Learning activities should fit the developmental level of students, therefore they
must be taught in a constructivist approach by making them recreate and reinvent every
cognitive system they encounter. Students must also be trained to employ regularly the whole
range of communicative media to fully express ideas, construct meaning and remember
information. Opportunities to reflect, debrief, and abstract from students experiences what
they have felt and thought and learned should also be provided for a more effective learning
environment. Teachers need to create classroom interactions which are socially constructed to
show that learning can be achieved through collaboration to eliminate competition and
individualistic approaches. Democratic learning makes the classroom a model community
where students learn what they live as citizens of the school.
Principles of Effective Laboratory Experiences
In 2008, Jona, et al. Emphasized the four curriculum standards that were identified as principles
of effective laboratory experiences by the National Research Council (2005). These are clearly
communicated purposes; sequenced into the flow of instruction; integrated learning of science
concepts and processes; and ongoing discussion and reflection.
In National Research Councils landmark study (2005), laboratory experiences are considered
effective if they have clear learning goals that guide the design of the learning experience. The
teacher must communicate clearly the purpose of the activity so that students can successfully
carry it out and achieve the desired goals set. It is recommended to design an inquiry activity
where students learn specific concepts which are clearly communicated to them throughout the
learning and discovery process. At the end, they will be assessed on their ability to achieve the
instructional purpose of the activity.
Still in the results of the study, laboratory experiences are said to be sequenced into the flow of
classroom science instruction if they are explicitly linked to what has come before and what will
come after. Scientific investigations when integrated into a well-designed sequence of
instruction will serve as an instructional purpose that is consistent with the objectives of the
learning unit. To achieve this, laboratory teachers must give their students ample time to
discuss the activities they are engaged in during laboratory period and reflect on the meaning
they can make out of them. They must also be given opportunities to formulate hypotheses
before experimentation so that they can reflect on their ideas after the complex process of
experimentation. A knowledge-centered environment is created when students reflect on their
own learning progress, when they identify, monitor and regulate their own thinking and

104
learning. Metacognitive strategies must be implemented by laboratory teachers in a way that

students can determine what they already know and what else they need to know in any given
situation including when things are not going as expected.
Teaching of Nature of Science
Crowther, Lederman & Lederman (2005) suggested some teaching strategies that will highlight
the teaching of nature of science. Teachers can design lessons around science topics or concepts
that have changed over time and the instruction must be explicit on how knowledge has
changed and why. Through this, students will learn that scientific knowledge in and of itself is
not static and that with new information, scientific theories can change. However, students
must be taught also that some laws in science have stood the test of time. In teaching scientific
laws, teachers must emphasize how these laws describe nature and how things act under
certain conditions. Also, the manner a teacher poses questions inside the classroom lead to
investigation and experiments then will eventually lead to conclusions - but still there are many
different pathways that scientists take. In conclusion, it is incorrect to assume that all scientific
investigations follow the same set and sequence of steps. One of the reasons why knowledge is
subject to change is that these different types of investigations provide different information
and evidence concerning the natural world, hence, different learning outcomes.
Motivation and Attitude Toward Science
Students in the tertiary level of education are required to take an introductory science course as
general education subject. It is not surprising that some students do not succeed because they
just enroll the subject for requirement purposes rather than taking the subject because they have
a passionate interest for learning it. It is the concern of the science instructor to shape the
attitudes of students toward science so that they leave their classes with positive views of the
discipline.
A possible reason why students seem not to learn much concepts on science subjects is that
most instructors focus primarily on content of the subject instead of helping learners cultivate a
holistic attitude towards the subject. The pedagogic strategies of the teacher also play an
important role in how students will appreciate science subjects.
Still, there are other factors which may result in negative attitudes towards science: lack of
needed skills to learn and apply scientific concepts, lack of motivation to work hard in science
classes, home backgrounds, school and classroom environments, biases of peer groups, the
media's portrayal of scientists, and students' perceptions of rewards associated with learning,
science anxiety, the fear of science learning, and apprehension toward scientists and science-
related activities.
Previous experiences of students also affect the attitude of students toward science as a subject
and as a body of knowledge. It has been argued that to build motivation and positive attitude
among students, there must be a good understanding of the content being taught, therefore the
teacher must find ways to probe knowledge which the students have previously constructed. If
105
the prior knowledge of the student is insufficient, inaccurate and in conflict with what is being
taught, the teacher, then must guide the students in reconstructing their knowledge. Learning
that will give the opportunity to student to reconstruct their own conceptual knowledge and
understanding leads to a lasting improvement in students' attitudes greater chances of success
in their studies and lives.
The attitude of a student toward a subject has something to do with the motivation made by the
teacher in introducing a lesson. Positive attitudes develop if a student is highly motivated and
this can be done by the teacher through improving the teaching practices and by showing to the
students the relevance of the topic to their everyday lives. Teachers must create a learning
environment that will encourage and inspire the students not only to come to class regularly
but also to have a desire to learn and enjoy learning.
Movahedzadeh (2011) suggested some teaching practices that will motivate the students and
will lead to their positive attitude towards the subject. Teachers must consider the
preconceptions of students regarding the topic by asking them of their personal views so that
diversity of views in the class lead to a deeper discussion about the process of doing science, the
application of scientific discoveries, and the impact of science on society.
The relevance of science can be further emphasized to students by mobilizing the scientific and
engineering research community. When students are given access to practicing scientist and
engineers who can provide them with valuable information on careers and studies, students
would increase their interest and enthusiasm in learning science concepts. For the students,
inviting experts in the class would help to put the subject into context and make classroom
activities more exciting. It is not only school visits of professionals but also visit of students to
the workplaces of these professionals that will help them to learn about and understand specific
professions.
Brodie (2006) even proposed some projects that will increase student participation, motivation
and success by involving the whole scientific community. These are the Researchers in
Residence Project, Express Yourself Conferences and creating Centre for Science Education.
In the Researchers in Residence Project, creative research talents such as PhD students and
post-doctoral researchers in science, technology, engineering and mathematics share their
passion for their field of specialization to the students for the purpose of igniting a fresh interest
for science among young people. Through the involvement of these research talents who act as
positive role models, significant change occur in how people view scientific researches,
scientists, and technical aspects of science.
The project dubbed as Express Yourself Conferences hopes to enable students to present the
findings of their own science investigations. In these designed conferences, students are given
opportunities to communicate and share their ideas with other students, teachers and
researchers; present research papers in seminars chaired by researchers in residence; present
and host displays of their investigations; and participate in other activities, such as discussing
their work with experienced researchers, attending keynote lectures and demonstrations, and
participating in practical workshops. On the other hand, the Centre for Science Education is
created for the purpose of inspiring and capturing the imagination of young people in science
106
through the development of creativity-rich resources and activities. In this way, students will
be motivated to pursue science courses and to be successful in their chosen career.
With the same purpose and intention, Wilson, Cordry & Uline (2004) said that participation of
students in science fairs will promote positive attitude towards science because in doing science
fair projects, students find enjoyment in applying scientific method thereby promoting their
interest in science. It also develop students sense of personal capabilities and qualities and
appreciation for nature and the relevance of science in daily life.
A study was conducted by Hall (2006) on the techniques to encourage students to confidently
contribute to their lab groups in science classrooms. She observed her students during
laboratory and analyzed how they conduct themselves. A new system was implemented where
students were assigned specific roles during laboratory and a grade was given based on their
level of contribution to the lab group. As a generalization, her data showed a positive
relationship between the implemented treatment and more active science lab participation.
Another study, that of Washtak in 2006, examined the use of technology in both the lecture and
laboratory settings of a high school chemistry class. The focus of her study was on the student
motivation and ability to learn from technology. As variables of the study, PowerPoint, SMART
Board and computer animations were used in the lecture setting. Logger Pro software and
individual laptops were used in the laboratory setting. The assessment techniques included
pre- and post-tests, surveys, teacher journal, analysis of specific test and laboratory questions,
student interviews and comparison of test scores, results of which found students motivated by
technology and were able to learn from them.
In an action research spearheaded by Nordick (2006), he introduced a unit plan that included
detailed lecture guides. Each guide contained objectives, key terms, and important topics for
students to follow during lectures. Such lecture guides were organized into unit plans and
presented to the students prior to the beginning of each unit.
In another study, the effect of having older students teach science concepts to younger students
was given focus. Muchmore (2006) looked into the argument whether students reach higher
levels of achievement when they take on the teacher role versus that of student roles. After
observing a dramatic rise in student participation in cooperative groups, Muchmore (2006)
recognized the importance of active student engagement and responsibility for
learning. Evidence indicated that peer assisted learning did, in fact, increase student interest,
however, a thorough investigation on better ways for students to retain learning was
recommended.
Mentzer (2006) also explored on how student motivation, as defined by validity and self-
efficacy, was affected by journaling, and if that motivation affected the time it took students to
get ready for class. His study revealed that a short period intended for journal writing before
classes begin, spell a big difference in students learning.

107
Eastwell and Rennie (2002) for their part used a quasi-longitudinal case study to determine the
effects on secondary students of participation in a program of enrichment and extracurricular
science activities in terms of their interest and enjoyment in being involved in science activities,
their motivation to continue to participate in science, and their perceptions about scientists and
about the role of science in society. A strong positive relationship was found between changes
in students interest and enjoyment and changes in their motivation, and both these variables
increased, in an overall sense for the combined student population, during the study period. All
students generally held a high perception of both the normality of scientists and the importance
of science in society throughout the study period. Participation in science activities impacted
overall positively, but to varying extents for different activities, on all four dependent variables.
Suggestions for the structure and/or conduct of competitions, excursions, and practical work,
including the design of museum exhibits, and implications for further research are presented.
A study with focus on integrating graphic organizers in the attitude, perception and
achievement of students in chemistry was done by Torres (2009). The results of the study
proved that students prior knowledge of outlining format allowed them to more easily utilize
and organize information. In addition, the sequencing and planning of instruction by the
teacher in an outline format allowed students to extract and synthesis information in an
organized manner.
Science Laboratory Skills
A well-designed laboratory activity has the potential to motivate students, support meaningful
learning of concepts, and develop manipulative competencies among students. According to
Moni, Hryeiw, Poronnik, Lluka & Moni (2007), students must be taught of the differences
among knowing about a topic, knowing how to complete a skill, showing how to
complete a skill, and finally doing the skill. This is possible through integration of skills
development with conceptual learning. Skills were considered as embedded elements of the
more complex laboratory practices of problem-based or case-based inquiry learning tasks.
(Moni, et al., 2007). Skills can be further differentiated from practices by saying that skills
represent hands-on or doing while practices represent the combination of hands-on and
minds-on. With this difference, teachers must teach skills to students with the expectation
that competencies in skills would support open-ended, student-driven explorations.
Kanli and Yagbasan (2008) identified laboratory principles for teachers which cover laboratory
teaching approaches to develop science process skills and conceptual achievement among
students. According to them, teachers must practice the principles of knowing how to excite,
explore, explain, elaborate, extend, exchange, and evaluate students. They enumerated
teaching practices under each principle. To excite students, teachers must provide the students
with the thoughts of the first scientist and make them feel like them; intrigue to ensure students
participation (a simulation may be watched about experiments); make a spark about the subject;
try to discover what students know about new concept or subject; ask questions that may
confuse minds (create unbalance); and ask questions about misconception.
In the field, science teachers explore when they provide environment for concrete, tangible
activities that include skills and concepts; ask probing question; listen and observe students; just
108
play the role of a good adviser or coach in students journey to cognitive balance; create a rubric
that will evaluate the skills of students about determining variables and establishing
hypothesis; and ensure that the students save the data they acquired correctly.
Another laboratory principle students must learn from the classroom interaction is the ability to
explain. Teachers must encourage students to explain and determine concepts; demand
explanations and proofs from students; emphasize that students use the data they acquired to
make reasonable explanations; and bring forward new concepts by taking students prior
experiences and making explanations and definitions.
This is possible, Kanli and Yagbasan (2008) noted if the teacher explores on the elaboration of
ideas and topics in the classroom. A teacher can elaborate the topic if she encourages students
to apply concepts and skills into new situations; demand from students to use concepts,
explanations and definitions with the previously acquired ones. A teacher values exploration in
her classroom if the students become aware that proofs and data are necessary in proving ones
opinion or point. The classroom, as a main source of knowledge, must ensure that the students
extend their learning by guiding them in associating present concepts with other fields and/or
other concepts/subjects. Further, a teacher may also ask research questions to help them in
associating the current lesson to concepts/subjects of other fields. Teachers must also teach to
their students the principle of exchange by preparing proper environment for students to
discuss their ideas with their friends; observing and listening to the students who are sharing
their knowledge; and ensuring the interaction within student groups, competing student ideas.
Lastly, teachers must know how to evaluate their students learning by observing students that
apply new concepts and skills; evaluating knowledge and skills of students; searching the
reasons of students changes of attitudes and ideas; letting the students evaluate their
knowledge and group process skills; and asking the open-end questions such as Why did you
think like that?, What is your proof for this?, What do you know about ....?, How do you
explain...?
Achievement in Science
Successful teaching and learning of science is the product of the correct use of an appropriate
teaching method whose objectives focus on the high achievement of the learners. One of the
challenges of a science teacher is how to facilitate learning which will address the difficulties of
the learners in assimilating concepts. Wachanga and Mwangi (2004) stated that knowledge
about teaching methods affect students learning may help educators in selecting methods that
will improve teaching quality, effectiveness, and accountability to learners and the public. To
be an effective teacher, students must be given opportunities to learn and technically manage
instruction. Effective learning and students achievement will be enhanced if students are
allowed to use their hands, eyes, ears and their mind.
In the Cooperative Class Experiment Teaching Method, Wachanga and Mwangi (2004) reflected
on how students acquire greater mastery of the subject matter because of peer teaching. Aside
from generating better intergroup relations, the shared responsibility and interaction in this
method result in better self-images for students with histories of poor achievement (Wachanga

109
and Mwangi, 2004). Slow learners, moreover, benefit in this cooperative learning method
because fast learners share their ideas so that the others learn the ideas in depth and remember
them longer.
In a science classroom, on the other hand, well-managed laboratory activities in enhancing

students learning of science concepts result in an enhanced interest in science (Claveria, 2002).
Students prefer experimenting, demonstrating, film showing using instructional media, making
diagrams, drawings, painting and sketches, gathering clippings of inventions and significant
science events classifying plants and animals through pictures and observing prepared slides
and specimens to enhance students interest in science, collecting rocks, taking photographs of
nature, collecting variations of plants and animals and preparing botanical gardens.
Furthermore, she recommended that students interest in science should be maintained by
inviting young scientists from whom they can draw further inspiration. The teachers should be
updated on the new techniques and trends in teaching science subjects, so they could be more
creative and resourceful. In addition, the teachers should teach their students to improvise
science materials that they can use in the laboratory activities.
To prove such theory, Ricardo (2008) conducted a study on how chemistry is taught in the
public secondary schools to determine the factors affecting students performance. To gather
enough data, Ricardo (2008) did actual observations, supplemented by interviews with the
teacher, head teacher, and principal. In his observation, the predominant teaching strategy
used by most teachers is still lecture-discussion, a traditional method. A significant correlation
existed between the performance of students in the achievement test and the method by which
chemistry is taught.
The effects of Task-Oriented Learning Approach (TOLA) on chemistry achievement among self-
handicapping students were analyzed by Reyes in 2002. TOLA consisted of several
components like task-on-activity, developing collaborative skills, classroom management
strategies and self-assessment. Self-handicapping students are those who are complaining
about the subject, avoiding seeking help, avoiding taking risk in difficult task, withdrawing
effort, reducing performance setting, lacking in preparation, procrastinating and making
excuses. The results showed that TOLA consistently improved the achievement of self-
handicapping students in chemistry.
Roble-Estrella (2009) suggested some teaching strategies in chemistry laboratory that will
enhance the performance of students in the laboratory in her research. For her, the teacher
must state the objectives clearly in every laboratory activity; emphasize major ideas as they are
presented; provide step-by-step directions when necessary; check for understanding at intervals
before proceeding to the next major idea or concept; provide concrete examples to explain and
reinforce information; use appropriate scientific vocabulary; must be specific and precise by
referring to concrete objects and events; ask questions or obtain work samples before
proceeding to the next procedure; call on slower students and non-volunteers and print out
necessary parts of the activity. Her study supports other researches on best practices to enhance
the learning capabilities of students enrolled in science subjects.

110
In the study by Palada (2002), the level of performance of students in chemistry is found to be
significantly influenced by their perception about chemistry and by the teachers most preferred
teaching practice as perceived by the students. She further concluded that teaching practice
influences the academic performance outcome of the teaching process. Knowledge of content
and understanding of students strengths and weaknesses along with appropriate teaching
practices can improve teaching and result to higher students achievement (Palada, 2002).
For her part, Leonor (2007) used the Scientific Inquiry Method to determine the extent of
academic performance in chemistry of students. She enumerated techniques which are
considered to be included in the scientific inquiry method. These include the limitation in the
use of lecture and direct instruction in presenting the lesson; use of students prior knowledge
as basis for introducing new concepts; exploring students interest to make learning relevant
and meaningful; using inquiries and investigations to anchor new information to previously
held knowledge; initiating classroom dialogue and discourse by posing essential or starter
questions; asking questions that require higher order thinking skills and critical skills; using
wait time techniques appropriately and not interrupting students in the middle of their
questions or answer; rephrasing students questions and answers; establishing everyday
routines for group interaction; arranging students desks for collaborative work in small groups;
focusing the lesson on engaging and relevant problem-solving situations; encouraging students
to design and carry out their own investigations; integrating science content with process skills
and problem solving strategies; valuing students responses and viewing wrong answers as an
open door to their misconceptions; encouraging students to use concept maps; graphic
organizers; and drawings of models to explain and demonstrate newly acquired knowledge.
Synthesis
The present study attempted to explore the different practices in chemistry laboratory
instruction that were expected to attain positive attitudes of students towards learning
chemistry, competencies in the laboratory skills and high achievement in the subject. In
particular, this present study bears similar concepts and focus to that of Hall (2006); Washtak
(2006); Mentzer (2006); Eastwell and Rennie (2002); and Claveria (2002) in that they looked into
how teachers deliver their lessons for the enhancement of the interest of students in learning
science.
There is a similarity between the present study and the studies of Palada (2002); Straatman
(2006); Nordick (2006); Ricardo (2008); and Roble-Estrella (2009) because they focussed on the
teaching strategies for the development of various skills of students in science subjects.
Lastly, Reyes (2002); Muchmore (2006); Leonor (2007) and Torres (2009) focussed their studies
on how teaching practices influence the achievement of the students in science subject similar to
the present study.
Research Design
This study employed the descriptive design, particularly the qualitative-quantitative method of
research which according to Alasuutari (2004) is that type of research which involves checking
of data collected via one method with data collected using another. The main objective of the
111
study was to propose a model of a teaching-learning process based on the best practices in
chemistry laboratory instruction. The descriptive design was the most appropriate design to be
used in determining the best practices in chemistry laboratory instruction that will attain the
goals of science lab instruction. These were investigated through qualitative method in which a
focus group interview of faculty and students followed by class observations were conducted.
Quantitative method was used in determining the students manifestation of the attainment of
the seven goals of science laboratory instruction through administration of instruments such as
Attitude/Motivation Instrument, Practical Test and Achievement Test.
Research Locale
This study was conducted at Lyceum of the Philippines University in Batangas, Laguna, Cavite
and Manila. LPU, an institution of higher learning, inspired by the ideals of former Philippine
President Jose P. Laurel, is committed to the advancement of his philosophy and values
Veritas et Fortitudo (truth and fortitude) and Pro Deo et Patria (for God and Country).
Guided by its vision and mission, it aims to provide quality education through its three-fold
function of instruction, research and community extension. It offers various programs in science
where General Chemistry is one of the basic subjects. General Chemistry is a five unit subject
comprising of three hours lecture and three hours laboratory in a week.
Participants of the Study
This study involved teachers and students from the four universities of the Lyceum University
System (LPU in Batangas, Laguna, Cavite and Manila). The participants were chemistry faculty
and their students enrolled in General Chemistry during the second semester of the school year
2011 - 2012. The chosen faculty have been teaching chemistry for a minimum of three years and
were either chemical engineers or chemists by profession. Most of them have Masters degrees.
The students, on the other hand, belong to degree programs such as B.S. Physical Therapy, B.S.
Psychology, B.S. Engineering, and A.B. Mass Communication. Majority of the student-
respondents were first time takers of the subject, however, some of them were repeaters.
Two to six chemistry faculty from each university and a group of three to nine chemistry
students participated in the focus group interview. The profile of faculty was secured from the
Human Resource Office of the university to find out who among them are science majors. The
faculty-respondents were selected from the faculty of sciences. This faculty is responsible for
teaching chemistry in the university. The students who were interviewed were selected from
the students who were enrolled in General Chemistry during the second semester. About thirty
percent of the class or three to nine students were selected randomly from the class based on the
total number of students in each chemistry class.
Class observations were also done to further validate data gathered from interviews and focus
group discussions. A total of four faculty from the four universities were observed and a total
of 80 students responded to the administered questionnaires. The Achievement Test, Practical
Test and Attitude/Motivation Questionnaire were administered to the intact class of the
observed faculty at the end of the semester.
112
Data Gathering Instrument
To approximate the best teaching practices employed by the faculty in teaching chemistry
laboratory, five instruments were developed by the researcher and were content validated by
experts. They were developed after a thorough review of conceptual literature about best
teaching practices.
Focus Group Interview Questionnaire for Faculty and for Students. This instrument was
prepared to approximate if the chemistry faculty is implementing practices in teaching
chemistry laboratory which will lead to the attainment of the seven goals of science laboratory
instruction (See Appendix H- Appendix I ). It was a structured questionnaire consisting of
open-ended questions which were answered by a group of chemistry faculty and a group of
students. The items were classified according to the goals of science lab instruction. The items
were intended to get information on the enhancement of mastery of subject matter, the
development of scientific reasoning, the development of the students understanding of the
complexity and ambiguity of empirical work, the development of practical skills, the students
understanding of the nature of science, the cultivation of interest in science and in interest in
learning science, and the development of teamwork skills. The questionnaire was presented to
the advisers for comments and suggestions and then to three chemistry experts for face and
content validity.
Observation Checklist. This checklist was used in conducting unannounced class observation
to validate if the faculty concerned is practicing the good instruction in conducting chemistry
laboratory classes mentioned in focus group discussions and to see if the seven goals of science
instruction are manifested in their teaching practices (See Appendix J). The items in the
observation checklist were classified according to the goals of science laboratory instruction.
The content of the checklist was similar to the content of the focus group interview
questionnaire. The items were enumerated so that the observer can take note of the practices
implemented by the faculty after thirty minutes, after two hours and at the end of the class. Like
that of the previous instrument, the questionnaire was presented to the advisers for comments
and suggestions and to three chemistry experts for face and content validity. The following
scale range was used to interpret the data gathered by the questionnaire.
Scale Range Verbal Interpretation
2.28 3.00 always practiced
1.52 2.27 often practiced
0.76 1.51 sometimes practiced
0 0.75 never practiced
Note: A mean of 1.52 to 3.00 is an indication that the faculty is implementing the best teaching practices
that will lead to the attainment of the seven goals of science lab instruction.
Attitude/Motivation Instrument. To approximate the extent by which students manifest the

attainment of the goals of science laboratory instruction in their attitude and motivation, an
attitude/motivation instrument was constructed by the researcher. This Likert Scale Instrument
(See Appendix L) consisted of 15 items which were categorized as to the views of students
about learning chemistry (attitude) or interest in learning chemistry (motivation). Seven items
113
of the questionnaire determined the students attitude towards chemistry and eight items
determined their motivation. The instrument also approximates the students understanding of
the nature of science. The students were asked to chose from a five-scale option such as strongly
disagree, disagree, neither agree nor disagree, agree and strongly agree. Equally, face and
content validity were established for this instrument. Interpretation of the data yielded by this
attitude/motivation questionnaire was based on the following scale range:

4.51 5.00 Very positive attitude
Very highly motivated
Very much understood
3.51 4.50 Positive attitude

Highly motivated
Much understood
2.51 3.50 Moderately positive attitude

Moderately motivated
Moderately understood
1.51 2.50 Negative attitude

Lowly motivated
Not so understood
1.00 1.50 Very negative attitude

Very lowly motivated
Not understood
Note: A mean of 3.51 to 5.00 is an indication of a best teaching practice of a faculty that will lead to the
attainment of interest in science and in learning science and understanding of the nature of science.
Practical Test. To measure the extent by which students manifest the attainment of the goals of
science laboratory instruction in their laboratory skills, a Practical Test was constructed (See
Appendix K). This Practical Test approximates not only the practical skills of the students but
also their teamwork skills and understanding of the complexity and ambiguity of empirical
work in a chemistry classroom. The instrument used in this test was a checklist which included
items on common laboratory techniques such as handling liquids and measuring volume,
handling solids and weighing, bunsen burner manipulation, heating substances in a test tube,
doing evaporation, and doing filtration. It also included items that indicate if students consider
safety precautions in performing experiments, if they exhibit teamwork skills and understand
complexity and ambiguity of empirical work. Attached to the checklist was the list of materials
and the procedure followed by the students in taking the Practical Test. The questionnaire was
presented to the advisers for comments and suggestions. To ensure that the questionnaire was
valid three chemistry experts were consulted for face and content validity.

114
The practical skills, teamwork skills, and the students understanding of the complexity and
ambiguity of empirical work was categorized and verbally interpreted as recommended by
experts as:

76 % - 100 % Highly competent / Highly understood
51 % - 75 % Competent / Understood
26 % - 50 % Moderately competent/Moderately Understood
0 25 % Not competent / not understood
Note: A scale range of 51% to 100% is an indication that the faculty is implementing best teaching
practices that will attain practical skills, teamwork skills and understanding complexity and ambiguity of
empirical work.
Achievement Test. This was a Concept-Application-Procedural (CAP) Test developed to

approximate whether the students had attain the mastery of subject matter and scientific
reasoning (See Appendix M). Twenty one (21) items of this multiple choice test were
categorized under mastery of subject matter while thirty eight items on scientific reasoning.
Each question consisted of four options of which students encircled the correct answer. The
questions were taken from the seven experiments performed during the semester such as
Measurement; Changes in Matter; Classifications of Matter; Laws of Chemical Change; Types of
Chemical Reactions; Solutions; and Classes of Compounds. Similarly, this questionnaire was
subjected to face and content validity.The original 100 items were then reduced to fifty nine
items after item analysis was done. The researcher also deleted the items which were not
relevant to the topics. The data gathered by this instrument was interpreted using the following
scale:

76 % - 100 % high level of mastery / high level of
scientific reasoning
51 % - 75 % average level of mastery / average level of
26 % - 50 % low level of mastery / low level of
0 25 % no mastery / no scientific reasoning
Note: A scale range of 51% to 100% is an indication that the faculty is implementing the best teaching
practices that will lead to attainment of mastery of subject matter and scientific reasoning.
Data Gathering Procedure
The study was conducted in successive phases. The details of activities can be referred
to the Gantt Chart of Activities.
Phase I - Planning Stage. This stage involved the review of literature on the study and
development of the instruments used in the study. A thorough reading of books, journals,
theses and dissertations together with internet resources was made to gather theories and
115
concepts related to best teaching practices. From the constructs gathered, five instruments were
developed such as Focus Group Interview Questionnaire for Students and Faculty, Observation
Checklist, Attitude/ Motivation Instrument, Practical Test and Achievement Test. The
prepared instruments were presented to the advisers for comments and suggestions. To ensure
that the instruments were valid, chemistry experts were consulted for face and content validity.
Comments, suggestions and recommendations were considered to refine the instruments. It
took the whole of the first semester to develop and validate the instruments. Phase I also
entailed the securing of approval from Lyceum of the Phil. University in Batangas, Laguna,
Cavite and Manila to conduct the study in their respective locuses.
Phase II - Gathering of Qualitative Data on Best Teaching Practices. This phase consisted of
the focus group interview and class observations. Separate focus group interview for the faculty
and students were conducted at the beginning of the second semester. Since the classes for the
second semester started on the second week of November 2011, the focus group interview for
faculty in the four universities was conducted on the last week of November of the same year.
Focus group interview for students was conducted on the first week of December, 2011. Two to
six chemistry faculty in each university were interviewed while a group of three to nine
chemistry students were included in the focus group interview for the students. The students
interviewed were the current students of the teachers who were observed for the study. The
interview lasted for one hour. It was video-taped and the responses gathered were analyzed
and interpreted qualitatively. The interview and focus group discussions were done to identify
the teaching practices implemented by the faculty in order to attain the seven goals of science
lab instruction.
To validate that the faculty was implementing the teaching practices that will attain the goals of
science laboratory instruction, unannounced class observations were also conducted. Only one
faculty per university was observed for this purpose. Class observation in each university was
conducted for three different experiments performed by the class. Each observation lasted for
three hours and was documented by photographs and video tapes. During the observation, the
observable items from the instruction made by the faculty were checked. Each faculty got a
score of one every time the item was observed and a perfect score of three if during the three
observations made, the faculty always demonstrated such item. The faculty got zero if the item
was never observed. The mean of the scores for each item of the four observed faculty was
computed and was verbally interpreted according to the scale range recommended by experts.
Class observation started on the second week of December 2011 and lasted until the last week of
February 2012.
Phase III - Gathering of Quantitative Data on Best Teaching Practices. This phase consisted
of determining whether the observed faculty implemented the goals of science laboratory
instruction as manifested in the students attitude, motivation, laboratory skills and
achievement. The students were asked to answer Attitude/Motivation Test, did Practical Test
and answered the Achievement Test at the end of the semester. The Practical Test was given on
the first and second week of March 2012 while the Attitude/Motivation Test was administered
together with the Achievement Test on the third week of March of the same year.

116
The Practical Test was conducted on the intact class of the observed faculty with the assistance
of their teacher. The class was divided into groups and each group was given an allotted time of
thirty minutes to finish the test including instructions on how they will do the test. Each
student was given a copy of the procedure before the test. All materials such as laboratory
equipments and reagents were made available already on the work table before the start of the
test. During the test, the students were rated by checking the items which were observed from
the group. The number of groups who demonstrated a particular skill was counted and the
corresponding percentage out of the 18 groups of students was computed.
The Attitude/Motivation instrument was administered to the intact class of the observed
faculty in each university at the end of the semester for a period of thirty minutes including
instructions on how students will answer the instrument. For scoring purposes, a score of 5 was
given if the respondent strongly agree to the item, 4 if agree, 3 if neither agree nor disagree, 2 if
disagree and 1 if strongly disagree. The mean of the scores for each item of the 80 student
respondents was computed and was verbally interpreted according to the scale range
recommended by experts. The Achievement Test was administered to the intact class of the
observed faculty at the end of the semester for a period of ninety minutes including the
instructions on how students will answer the test. The perfect score for mastery of subject
matter was 21 and for scientific reasoning was 38. The mean of the scores of the 80 student
respondents and the corresponding percentage were computed and verbally interpreted
according to the scale range recommended by experts.
Data Analysis Procedure
Content analysis of the responses of students and faculty in the FGI were done by deduction
and induction. From here, it was determined whether the teaching practices implemented by
the faculty conformed with the seven goals of science lab instruction.
To analyze the teaching practices observed among the faculty, the researcher made use of
statistical mean. Frequency, percent, mean and standard deviation were used to describe the
extent by which students manifest the attainment of the goals of science lab instruction in their
attitude and motivation, lab skills and achievement. These statistical treatments were used to
analyze the responses and performance of students in the Attitude/Motivation Test, Practical
Test, and CAP test.
I. Teaching Practices Employed by the Faculty in Teaching Chemistry Laboratory in Order to

Attain:
A. Mastery of Subject Matter
Table 1. Practices Employed by Faculty in Designing their Laboratory Instruction

Group of Respondents Laboratory Practices of Faculty
Basing on the syllabus; patterned with the sequencing of
Faculty of LPU 1 topics discussed in the lecture; making it simultaneous
with the topic in the lecture; following the recipe-type of
procedure

117
Requiring students to do experiments which are similar to

Students of LPU 1 the topics discussed in the lecture; simply following the
step-by-step procedure in the lab manual
Doing experiments after discussing the concept in the
Faculty of LPU 2 lecture; performing experiments which are related and
simultaneous with the topics discussed in the lecture;
following the recipe type
Discussing first the concept in the lecture and then
Students of LPU 2 applying it in the experiment; performing all the
experiments related to the topics in the lecture; following
the given procedures in the lab manual
Performing experiments while discussion of the topics in
Faculty of LPU 3 the lecture is going on; designing it in a way that theres
close supervision of students learning simultaneously in
lecture and lab; sometimes implementing discovery
approach with investigative type of procedure
Discussing the topic while performing experiment that is
Students of LPU 3 related to it; sometimes asking students to perform first
the procedure given in the lab manual and then asking
students what they learned from the result.
Applying deductive approach; providing procedures in
Faculty of LPU 4 the laboratory manual; simply following the cook book
style in the manual; doing experiments simultaneously
with the discussed topics in lecture
Requiring students to have individual laboratory manual;
Students of LPU 4 assigning students to read the experiment corresponding
to the topic in the previous lecture before coming to class
so that students can readily perform the written
procedures while doing the experiment
Table 1 presents the practices employed by the faculty in designing their laboratory instruction.
It shows that all of the four groups of faculty design their laboratory instruction in relation to
the topics discussed in the lecture. They plan the science experiment according to how the
topics in the classroom lectures were designed. Three out of the four groups make the
experiment procedure in recipe type or cook book style, while only one group of faculty said
that they sometimes implement the discovery approach where the procedures are following the
investigative approach. They require their students to use laboratory manual where the list of
experiments are conforming to the sequence of lecture topics indicated in the syllabus. One
group of faculty discusses the topic during the lecture sessions of the class so that the students
will be guided properly come laboratory periods.
All of the faculty-respondents plan the experiment to be done alongside the lectures because to
them, the laboratory manuals were developed in relation to the topics discussed in the lecture.
The presence of laboratory manuals, according to the teacher-respondents, allows them to
follow the recipe type of experiment procedure. Some teachers are planning to shift to the
investigative type, while some others are already implementing the discovery approach but still
with close supervision of students. The group of faculty who are implementing the discovery
approach are those with fewer students. As part of their strategy, the teachers conduct both
118
lecture and laboratory classes in the laboratory room for the chance to add to the discussion
while experiments are on-going. To them such style of handling laboratory instruction is aimed
to have a close supervision of students learning both in theory and in practice.
The data gathered indicate that the teachers relate laboratory activities closely with lectures in
order to help students progress toward science learning goals. This conforms with the
statement of the National Science Teachers Association (2005) that laboratories and lectures are
not separate activities. In the same scope, it also complies with the recommendation of the
Committee on High School Laboratory to integrate laboratory experiences into instructional
sequences (Singer, et al., 2005).
On the other hand, the reason for most of the teachers following the recipe-type of experiment
procedure is because of lack of time to venture into investigative type of experiment. To the
teacher-respondents, the pressure of trying to finish the syllabus for one semester marks the
biggest hurdle for them. This finding is similar to the description of a typical science course
given by Jona, et al. (2008) that students do not have enough time for planning investigation or
interpreting results because teachers bombard them with so many lessons from the syllabus.
As reflected in the FGI with students, all the chemistry laboratory teachers from the four
universities require their students to perform experiments in relation to the topics being
discussed in their lecture. The lecturers are all following the given procedures included in their
laboratory manual. Three out of the four groups of students have discussion of the concept in
their lecture period prior to the performance of the experiment for their laboratory session.
Only one group of students was required to perform experiment while the discussion of the
topic is going on. In some cases, this same group of students sometimes perform the given
procedure in their laboratory manual first before their teacher ask them on the learnings they
get from the results of the experiment.
Still, table 1 indicates that most of the times teachers provide their students with prior
knowledge of the concept of the experiment either by discussing it in the lecture before
conducting the laboratory class or by assigning the students to read about the experiment
before coming to the laboratory class. It could be that these teachers want that the learning
gained by their students in the lecture could be retain longer through hands-on activities, or it
could be that from the learning the students acquired in their lecture, the teacher can enable the
students to reconstruct their previous knowledge by doing experiments.
These findings conform with Hidalgos (2000) statement that in the laboratory method, the
learning gained by students is retained longer because the students are learning by doing.
Table 2 shows the practices of faculty in designing experiment if the necessary material and
equipment is not enough for the entire class.
Table 2. Practices of Faculty in Designing the Experiment if the Necessary Material is Only
Limited to One or Two Groups

119

Demonstration of experiment by the teacher; replacing
Faculty of LPU 1 materials; improvising instruments; not skipping any
experiment in the manual for the reasons that
instruments/materials are not available
By rotation, by sharing, the teacher does not skip
Students of LPU 1 experiments when instruments/materials are not
available; there is an alternative material/instrument for
the unavailable one
Not skipping experiment because materials and
Faculty of LPU 2 instruments are enough for the students
The teacher divide the class two batches so that materials
Students of LPU 2 will be accommodated by all the students; unavailability
of materials happen rarely; providing all
materials/instruments so that no experiment is omitted
Encouraging students to be resourceful; not skipping
Faculty of LPU 3 experiment due to unavailability of
materials/instruments
Does not happen that the materials /instruments are
Students of LPU 3 limited or not available because there are very few
students in the class; doing all experiments in the lab
manual
Performing all experiments because all materials and
Faculty of LPU 4 instruments are available; delegating a representative
student to demonstrate the procedure or sharing the
materials to each group if the material is limited
The teacher ask the most intelligent student to do the
Students of LPU 4 experiment in front of the class; the students are
required to perform all the experiments in the lab
The table shows that all of the four groups of faculty do not skip experiments even though the
instruments/materials needed are not available. Two groups of faculty do the experiment
through demonstration of the procedure either by the teacher or by a representative student
from the class. On the other hand, two other groups of faculty experienced having enough
materials and instruments and are available to their students. One group of faculty pushed
their students to be resourceful while one group improvised the unavailable instrument and
replaced the unavailable or limited materials. Another group of faculty simply shared whatever
material is available to the students.
All the faculty-respondents performed the experiments in the laboratory even if there is
unavailability of instruments/materials because in their university the faculty themselves
develop the laboratory manual so that all the needed instruments/materials were requested
and were provided at the start of the semester. In case of limited number of equipment, the
faculty concerned preferred to use the demonstration method of laboratory instruction to bridge
120
the learning of students. This finding is similar to what Acero, et al. (2000) cited that
demonstration method is an imitative method where learning a skill is faster and more effective
if students are shown how the job is done by using the actual tools, machines and materials.
One of the results of the focus group interview, students point out that, the teachers require
them to do all the experiments included in their laboratory manual and the unavailability of
materials/instruments is not a reason for them to forego any experiment. If ever the necessary
material is only limited to one or two groups, their teacher finds ways to address the problem.
One group of students cited that if the materials/instruments are unavailable, their teacher
makes the class to share the limited material or use the limited instrument by rotation. In their
classrooms, there is always an alternative material/instrument for the unavailable one.
Another group of students pointed out that they rarely encounter unavailability of
materials/instruments; if ever the material is limited, their teacher divides the class into two
batches so that the limited material can be used by all members of the class. Another group of
students pointed out that they never experienced limited or unavailable materials/instruments
because they are very few in the class. On the other hand, the teacher of one group of students
asks the most intelligent student among them to do the experiment in front of the class.
Table 2 also reveals that the teachers implement various techniques in handling problems such
as unavailability or limited materials/instruments because these teachers have knowledge of
the specific teaching strategies that can be used to address students learning needs given
particular classroom circumstances like lack of materials or equipment. These teachers do not
skip experiments when materials/instruments are not available because they want to inculcate
to their students the value of resourcefulness which is one of the scientific attitudes a student
needs to develop. This is one of the forms of practical knowledge in the pedagogical content
knowledge (PCK) of Shulman as cited by Rowan, et al. (2011).
The practices of the faculty on how they begin their laboratory class on the other hand, are
presented in Table 3. The table shows that all the four groups of faculty begin their laboratory
class by presenting the objectives of the experiment to their students. This is done when
teachers state the goals and objectives of the experiment themselves or letting students read the
objectives stated in the laboratory manual, or asking the students to think of more objectives
aside from those stated in the laboratory manual.
Table 3. Practices of Faculty on How they Start their Laboratory Class

Giving a brief description of what the experiment is all
Faculty of LPU 1 about, how to do it, and what to expect from it; stating
clearly the goal of the experiment
Giving a brief summary of the experiment for the day is;
Students of LPU 1 the teacher states the objectives of the experiment; asking
questions for students to discover the possible result of
experiment
Describing the procedure of the experiment;
121
Faculty of LPU 2 emphasizing the safety precautions; checking materials

and lab gowns; letting students read the objective of the
experiment since it is already in the manual
The teacher introduces the experiment; making students
Students of LPU 2 read the manual since the objectives are already in the
lab manual and then explaining the procedure
Asking a series of questions so that students will begin
Faculty of LPU 3 thinking about the possible outcome of the experiment;
telling the students the purpose of doing the experiment
Students of LPU 3 Allowing students to discover the outcome of the

experiment; allowing students to expound on the
objective of the experiment
Reminding the students about laboratory policies such
Faculty of LPU 4 as proper arrangement of chairs and bags; asking
students to think of more objectives of the experiment
aside from those stated in the lab manual; asking
students on how to apply the concept they learned from
their lecture on the experiment
Students of LPU 4 Reviewing the topics discussed in the lecture and then
asking students to relate it to experiment
Table 3 shows that two groups of faculty give descriptions of the procedure of the experiment to
the students while two other groups remind the students on safety precautions and laboratory
policies before any laboratory work is done. Two groups of faculty begin their laboratory class
by asking questions to make students think of the possible outcome of the experiment or to let
them connect the concept learned from their lecture to the experiment they will perform.
The responses of the faculty as shown in the table indicate that these teachers employ teaching
practices which aim for the success of an experiment because they make their students
understand the problem clearly before doing the experiment. This is in consonance to NSTAs
(2007) description of a well- designed laboratory instruction where the objective of the activity
is clearly communicated to students and gives opportunities to students to develop safe and
conscientious laboratory habits and procedures.
As observed by the students, the teachers of the three groups of students clarify the objectives to
the entire class at the beginning of the class. For the purpose of clarity, teachers of the two
groups of students introduce the experiment or give a brief summary of the experiment. One
group of students was allowed by their teacher to discover the outcome of the experiment on
their own while one group of students was asked by their teacher to relate the topics discussed
in their lecture to the experiment performed.
The data prove that the teachers implement a properly designed laboratory instruction because
the students are allowed to expect outcomes or experimental results which may or may not be
122
contradictory to the concept learned from their lecture. This conforms to one of the four
principles of instructional design which according to NRC (2005) can help laboratory
experiences achieve their intended learning goal. According to this principle, instructions must
be design with clear learning outcomes in mind in order to attain the desired learning objective.
The practices of the faculty in conducting pre-lab discussion are shown in Table 4. It can be
noted from the table that all the four groups of faculty list down vocabulary or terms related to
the experiment, especially those important terms or concepts, the title of the experiment,
materials needed, and even the set-up of the procedure is drawn on the board. Two groups of
faculty use their students prior knowledge and previous concepts learned as a basis for
introducing new concepts. It is done by asking students of their ideas about the experiment
before introducing it or by reviewing or recalling the previous concepts learned so that students
can connect the gap between prior knowledge and the topic to be introduced while two groups
of faculty allow their students to draw hypothesis about concepts by demonstrating selected
procedures or by basing the hypothesis from their prior knowledge.
The responses of the faculty prove that they are implementing good practices in conducting
pre-lab discussion because they provide a foundation of factual knowledge and conceptual
understanding to students. It is responsive to the pedagogical practices identified by NRC
(2005) which are considered authentic best practices in science teaching.
Table 4. Practices of Faculty in Conducting Pre-lab Discussion
Faculty of LPU 1 Asking students of their ideas about the experiment

before introducing it; writing important terms or
concepts on the board
The teacher stimulate the discussion by allowing
Students of LPU 1 students ask questions; allowing students to state their
previous knowledge and are elaborated by the teacher;
writing definitions of terms and keywords on the board
Faculty of LPU 2 Demonstrating selected procedures so that students can

draw hypothesis from it; writing the title of the
experiment and the materials needed on the board
Students of LPU 2 Demonstrating delicate procedures while the rest of the

procedures are done by students at their own ; writing
concepts on the board
Faculty of LPU 3 Writing vocabulary or terms related to the experiment

on the board; asking students to formulate hypothesis
about the experiment based on their prior knowledge
123
Students of LPU 3 Allowing students to make a guess on the result of the

experiment before doing it by themselves; listing down One
vocabulary or terms related to the experiment on the of
board thes
Reviewing or recalling the previous concepts learned to e
Faculty of LPU 4 link previous topic to new topic or to have continuity of aut
their previous knowledge; letting students connect the hen
gap between prior knowledge and the topic to be tic
introduced; writing meaning of terms on the experiment best
together with drawing of set-up on the board prac
tice
Students of LPU 4 Asking students to recall and relate the concepts learned s is
from the lecture to the experiment; drawing the set-up of eng
apparatuses on the board agin
g
resilient preconceptions where teachers identify, confront and resolve this initial understanding
of the students. Another authentic best practice is organizing knowledge around core concepts
which aims to increase understanding and retention of concepts among students.
Cited in the responses of students, the teachers of the four groups of students conduct their pre-
lab discussion by first writing on the board the keywords, definitions of terms, concepts and set-
up of the apparatuses to be used in their experiment. Two groups of students were asked to
recall and state their previous knowledge before the conduct of the experiment. The teacher of
one group of students demonstrates the varied procedures of their experiment while one group
was asked by their teacher to infer on the result of the experiment before doing the actual
experiment.
The data revealed that the teachers practice a properly designed pre-lab discussion because they
help students move towards a more scientific understanding of what students understand
about the prior concepts they learned. This finding affirms Singers, et al. (2005) idea of an
integrated instructional units which she believes is an effective instruction because it begins
with what learners bring to the setting such as knowledge of academic content.
The practices of the faculty-respondents in supervising or guiding students in the process of

performing the experiment are shown in Table 5. From the table, it is clear that all of the four
groups of faculty supervise and guide their students in the process of performing the
experiment by moving around the laboratory room to check if each group of students does the
experiment properly. Two groups of faculty never allow their students to manipulate the
procedures and materials while one group of faculty allows students to manipulate the
procedure and materials if they have their approval. Another group of faculty allows their
students to manipulate procedures and materials depending on the situation such as when
materials are not available so that the students are allowed to use replacement.
Table 5. Practices of Faculty in Supervising or Guiding Students in the Process of Performing

Experiment

124
Faculty of LPU 1 Moving around the room, checking each group if they
follow the correct procedure; entertaining questions
from students; not allowing students to manipulate
procedures and materials
Asking questions upon seeing that students are not
Students of LPU 1 following the procedure correctly; roaming around the
room; entertaining questions; consent is being asked
from the teacher if students want to manipulate
procedure or materials
Faculty of LPU 2 Going from one group of students to another group;

seeing if they are doing the experiment properly; never
allowing students to manipulate procedures and
materials
Students of LPU 2 The teacher is moving around and checking each group
if they are doing the experiment correctly; not allowing
any manipulation of procedure or materials
Faculty of LPU 3 Going around and checking each group as to how they
do the experiment; not allowing students to manipulate
the procedure and the materials without their approval
Students of LPU 3 Moving around; not allowing students to do

manipulation of procedure nor materials
Faculty of LPU 4 Making rounds; discussing with each group one at a

time; allowing students to manipulate procedures and
materials depending on the situation; using replacement
if materials are not available
Students of LPU 4 Going from one group to another finding out if students
are having problem with the experiment; allowing
students to improvised unavailable instrument or use
substitute materials
Based on the responses presented, it seems that these teachers employ the method of small
group instruction by moving from one group to another because they want to give attention to
each students learning needs. This is in response to Hidalgos (2000) idea of small group
instruction where students become more actively involved in their own learning and participate
more freely in discussions.

125
As observed by students, the teachers of the four groups of students supervise students in
doing experiment by moving around the room to check whether each group is doing the
procedure properly. Two out of the four groups of students are not allowed to manipulate
procedure or materials, while one group ask consent from their teacher when they want to do
some manipulations. Only one group of students was allowed to improvised instrument or use
substitute materials.
Most of the practices of faculty, observed by the students, reveal that they make close
supervision of the students while doing experiment because of the high demand for success.
This is in consonance to what Salandanan (2002) emphasized when she said that for an
experiment to be successful, the teacher must be a keen observant who can easily spot incorrect
steps and procedure. Table 6 shows the practices of faculty in conducting a post lab discussion.
Table 6. Practices of Faculty in Conducting a Post-lab Discussion

Asking the students to relate the result of the experiment
Faculty of LPU 1 to their previous discussion; giving the positive result
which may or may not be contrary to the result obtained
by students; allowing students to compare their results
to find out their mistakes; giving post-lab quiz in the
next lab period
Discussing partial result before the expt; discussing the
Students of LPU 1 result of experiment after the expt if theres enough time;
giving correct result; allowing students to compare
result; giving post- lab quiz the following meeting
Calling all members of the group to discuss their result;
Faculty of LPU 2 discussing only questions in the manuals; allowing
students to compare results per procedure with each
group; giving post-lab quiz the following meeting
Students of LPU 2 Asking students about their ideas on the result of the
expt; allowing students compare ideas; telling which is
the correct result
Faculty of LPU 3 Conducting post-lab discussion during lecture time;

requiring students to report the result; allowing
students to compare results but not letting them change
their result; giving post-lab quiz during lecture
Students of LPU 3 Allowing students to compare results with other groups

and if there are discrepancies making students retest or
repeat the procedure; giving post-lab quiz during lecture
time
Conducting a post-lab discussion a meeting after;
126
Faculty of LPU 4 providing more questions to students if the result

deviate from expected result; allowing students to
compare results to identify why they are wrong and how
they can correct their mistakes; giving post-lab quiz after
post-lab discussion
Students of LPU 4 Allowing comparison of result to know their mistakes;

giving quiz after discussion of result
All of the four groups of faculty require their students to do the discussion by asking them to
relate the result to their previous discussion, by reporting, or by providing more questions for
students to answer. All of them allowed their students to compare result with other groups of
students and all of them give a post lab quiz after the post-lab discussion.
It is reflected in the table that the teacher-respondents conduct post- lab discussions to clarify
the results in relation to the lecture. They require their students to do the discussion with the
aim of teaching their students communicative skills. This is similar to the laboratory principle
identified by Kanli and Yagbasan (2008) which states that teachers must encourage students to
explain by demanding proofs or making students use the data they acquired in making
reasonable explanations.
As observed by students, all the four groups of students were allowed by their teachers to
compare the results of experiment with those of their classmates. Three groups of students
were given a post-lab quiz after the post-lab discussion. Only two groups were given the correct
result of the experiment by their teacher while the other two groups of students were allowed
by their teacher to find their own mistakes.
It is also evident from the table that the teachers conduct a post-lab discussion in order to
provide the necessary connection between the result of the experiment and the appropriate
science concepts. Students were allowed to compare results with their classmates so that they
can identify and correct their own mistakes. This conforms with Piagets Theory of
Constructivism where children are allowed to make mistakes and correct these on their own
thereby enabling them to accommodate, assimilate and reconstruct knowledge on their own
(Muijs and Reynolds, 2011). Table 7 presents the data gathered from class observations to
determine
Table 7. Practices Observed Among the Faculty that will Enhance the Mastery of Subject
Matter
Practices Mean Verbal
Interpretation
1. Giving a brief description of the experiment 3.00 Always
for the day
2. Stating the goals and objectives of the 3.00 Always
experiment
3. Letting the students think of the objectives in 0 Never
127
doing the experiment

4. Asking a series of questions for the students to 2.75 Always
begin thinking about the topic
5. Presenting challenging questions to draw out 2.00 Often
the preconceptions of the students
6. Conducting a pre-lab discussion 3.00 Always
7. Using the students prior knowledge as a basis 2.75 Always
for introducing new concepts
8. Allowing the students make observations and 2.50 Always
draw conclusion about concepts prior to
giving explicit instruction
9. Listing down vocabulary or terms related to 3.00 Always
the experiment
10. Supervising/guiding the students in the 3.00 Always
process of performing the experiment
11. Allowing the students explore ideas rather 1.00 Sometimes
than manipulate material and procedures
12. Making the experiment procedure in recipe 3.00 Always
type
13. Making the experiment procedure in an open- 0 Never
ended or investigative type
14. Conducting a post-lab discussion 3.00 Always
15. Discussing the results of the experiment in 3.00 Always
relation to lecture content
16. Administering a post-lab quiz right after the 1.50 Sometimes
experiment
17. Providing guide questions for students to 3.00 Always
answer
18. Allowing students to compare results with
other groups and making sense of the
collective data of the class
19. Providing the necessary connection with the 2.75 Always
results of the experiment and the appropriate
science concepts 3.00 Always
OVERALL MEAN 2.38 ALWAYS
The practices implemented by the faculty in enhancing mastery of subject matter. The data
reveal that teachers always implement the given practices with a mean of 3.00. Practices like
giving a brief description of the experiment for the day, stating the goals and objectives of the
experiment, conducting a pre-lab discussion, listing down vocabulary or terms related to the
experiment, supervising/guiding the students in the process of performing the experiment,
making the experiment procedure in a recipe type, conducting a post-lab discussion, discussing
the results of the experiment in relation to lecture content, providing guide questions for
students to answer and providing the necessary connection with the results of the experiment
128
and the appropriate science concepts are marked as common practices among the teacher-
respondents.
Practices with a mean of 2.75 on the other hand, are always implemented by the teachers. These
practices are asking a series of questions for the students to begin thinking about the topic,
using the students prior knowledge as a basis for introducing new concepts, and allowing
students to compare results with other groups and making sense of the collective data of the
class. The teachers sometimes implement those practices with a mean of 1.50 and 1.00. These
are administering a post-lab quiz right after the experiment and allowing students to explore
ideas rather than manipulating materials and procedures. Practices with a mean of 0 and a
standard deviation of 0 are never implemented by the teachers. These are letting the students
think of the objectives in doing the experiment and making the experiment procedure in an
open-ended or investigative type.
In general, the overall mean of 2.38 was an indication that the teachers always implement
practices that enhance the mastery of subject matter among students. It could mean that these
teachers exert efforts in teaching the content and process as related educational goals because
they want their students to readily understand and apply the concept they acquired. This
finding is similar to what Jona, et al. (2008) stated that mastery of subject matter could be
attained if concepts and processes are taught simultaneously so that students perform a process
with a clear understanding of the relation of that process to content.
B. Scientific Reasoning
The practices employed by faculty in developing scientific reasoning among their students are
revealed in Table 8. The table reveals that all the four groups of faculty require scientific
explanations for the result of experiment. Two groups of faculty give on-the-spot questions
while performing experiment. One group of faculty require students to submit a reflection
paper while the other group of faculty allow their students to reflect by sharing with other
students what they have learned from the experiment. One group of faculty let their students
check their data, analyze and repeat the procedure when they got a wrong result while another
group of faculty ask their students to trace all the errors.
It appears from the table that the teachers implement practices that develop scientific reasoning
of students because they require students to make scientific explanations of the occurrence of
events as in the result of the experiment. They teach their students valid reasoning principles
and at the same time give opportunities to their students to practice these reasoning skills. This
is in consonance with the integrated learning program of NRC (2005) which is considered an
effective instructional practice because students can relate theoretical claims with evidences
gathered from laboratory investigation.
Table 8 presents the practices of faculty in developing scientific reasoning among students. As
observed by students, all the teachers of the four groups of students require them to analyze
and explain the data, their observations and discuss the results of the laboratory activity. Two
groups of students are given on- the- spot questions asking them the reason for doing such
procedure. Two groups of students are asked to do error analysis and to find out the sources of

129
error while two groups of students are required to submit reflective essay or narrative reflection
of what they learned.
The practices of the faculty as observed by the students revealed that their teachers implement
practices that develop scientific reasoning among them because they were taught how to
explain and give reasons for what they are doing. Some of their teachers allow them to reflect
on their own learning so that these teachers practice metacognitive strategies. Metacognitive
strategies according to Singer, et al. (2005) when implemented in a knowledge-centered
environment will enable students to reflect on their own learning progress, to identify, monitor
and regulate their own thinking and learning which in turn will facilitate their learning.
Table 8.
Practices of Faculty in Developing Scientific Reasoning Among Students

Allowing students to develop scientific explanations for
Faculty of LPU 1 the result of the experiment; asking on-the-spot
questions while performing experiment; asking students
to defend why they got such a result; requiring students
to submit a reflection paper on what they learned
Students of LPU 1 Asking students why they are doing such procedure and
what they observe; asking the class to explain how they
arrived to the result; requiring students to pass a
reflective essay for the whole semester
Giving on-the-spot questions while doing the
Faculty of LPU 2 experiment to check if they are following procedure
correctly; asking only those who are not following
correct procedure; tracing all the errors before allowing
students to develop their scientific explanation
Students of LPU 2 Asking students to do error analysis; asking students to

explain and analyze the data obtained; on-the-spot
questions are addressed to the idle member of the group
Faculty of LPU 3 Asking the students to explain why they were not able to
produce the result; letting students check their data,
analyze and repeat the procedure
Students of LPU 3 Requiring students to analyze and explain the data and
graphs obtained and to discuss the results; asking
students to find out the sources of error

130
Faculty of LPU 4 Making students defend their result based on the laws
and principles studied; allowing students to make
reflection by sharing with other groups what they
learned from the experiment
Students of LPU 4 Asking students to submit a report sheet for every

experiment which is a narrative reflection of what
students learned from the experiment; asking students to
discuss the observations
In addition to this, Kanli and Yagbasan (2008) said that teachers must demand explanations and
proofs from students and at the same time emphasize that students should use the data they
acquired to make reasonable explanations.
Table 9 presents the data gathered from class observations to determine the practices employed
by the faculty in their instruction to develop scientific reasoning among their students.
Table 9. Practices Observed among the Faculty that will Develop Scientific Reasoning of
Students
Laboratory Practices Mean

Verbal
Interpretation
1. Encouraging the students to design and 0.25 Never

conduct scientific investigations
2. Requiring students to identify questions and 0.25 Never
concepts that guide scientific investigation
3. Giving on-the-spot questions to check the 3.00 Always
understanding of students of why they are
doing such procedure
4. Allowing students to develop and revise 0.50 Never
scientific explanations and models or
recognize and analyze alternative
explanations and models
5. Allowing students make and defend a 0.75 Never
scientific argument by reviewing
information, using scientific language
appropriately, constructing a reasoned
argument, and responding to critical
comments
6. Requiring students to explain/analyze their 3.00 Always
data, discuss the results including graphs
and do error analysis
7. Requiring students to make reflection where 0.50 Never
131
they will defend their conclusions based on

data and analysis of data, compare results
with other sources and explain differences
OVERALL MEAN 1.18 Sometimes
As shown in the table, the practices such as giving on- the-spot questions to check on the
understanding of the students of why they are doing such procedure, and requiring students to
explain/analyze their data, discuss the results including graphs and do error analysis received a
mean of 3.00 which shows that the teachers always practice them. Allowing students to make
and defend a scientific argument by reviewing information, using scientific language
appropriately, constructing a reasoned argument, and responding to critical comments got a
mean of 0.75 indicating that it is never implemented by the faculty. Practices with a mean of
0.50 and 0.25 are never implemented by the faculty. These refer to allowing students to develop
and revise scientific explanations and models and recognize and analyze alternative
explanations and models; requiring students to make reflection where they will defend their
conclusions based on data and analysis of data, compare results with other sources and explain
differences; encouraging students to design and conduct scientific investigations; and requiring
students to identify questions and concepts that guide scientific investigation.
In general, the overall mean of 1.18 was an indication that the teachers sometimes implement
practices that develop scientific reasoning of their students. It could mean that these teachers
cannot employ the inquiry method of laboratory instruction which allow their students to make
their own investigation because they lack enough time to conduct such investigation since they
are pressured to finish the syllabus before the end of the semester. This is contrary to the idea
of Salandanan (2002) that in the inquiry approach, the reasoning skills of the students are
improved upon learning how to investigate and discover new information.
C. Understanding Complexity and Ambiguity of Empirical Work
Table 10 shows the practices of faculty in developing the students understanding of the
complexity and ambiguity of empirical work. Two groups of faculty avoid errors due to
equipment failure by allowing the technician do the checking and troubleshooting of equipment
before an experiment, whereas only one group of faculty teach students how to troubleshoot
equipment.
Table 10. Practices of Faculty in Developing Students Understanding ofComplexity and

Ambiguity of Empirical Work
Faculty of LPU 1 Explaining the cause of error; teaching students in

troubleshooting equipments in order to avoid errors;
asking students to make several trials to check precision
and accuracy

132
Students of LPU 1 Teaching students on the maintenance and proper care

of equipments; requiring students to repeat the
procedure if the data is not precise or accurate;
explaining that errors cannot be avoided
Faculty of LPU 2 Asking the technician to do the calibration and

troubleshooting of equipments; giving hints to students
to account for discrepancies like for example in the unit
conversion
Students of LPU 2 Replacing equipments which are not functioning; giving

the students clues when there are deviations from
expected values
Faculty of LPU 3 Checking the equipment before using; doing a dry-run

of the experiment before asking students to perform;
letting students compare their data with the standard
Students of LPU 3 Making alternative equipments available if there is

malfunctioning of equipments; giving already the
expected value before the experiment
Explaining to students that there are conditions which
Faculty of LPU 4 may affect the result; asking the technician to check the
equipments before lending to students and do the
troubleshooting of malfunctioning equipments;
requiring students to make three trials for every
measurement
Students of LPU 4 Asking the students to replace the malfunctioning

equipments; explaining the factors which may cause
error
The table also shows that two groups of faculty explain the cause of error and clarified that
there are conditions which may affect the result. Two groups of faculty require their students to
make several trials in order to check for precision and accuracy of data while only one group of
faculty gives hints to students to account for discrepancies. Still another group of faculty allows
students compare their data with the standard.
The data indicate that the faculty design their laboratory instruction in such a way that students
are able to expect outcomes or experimental results which are contradictory to the accepted
scientific principle. This conforms to Jona, et al. (2008) statement that experimental errors are
not hindrances to learning, but they are opportunities for greater learning. So instead of
working hard to remove complexities and ambiguities, laboratory instructors should include
the expectation of experimental errors in their instruction.

133
As observed by students, the teachers of the three groups of students replace the
malfunctioning equipment while only one group of students were taught how to maintain and
give proper care to equipment. The teacher of one group of students explains that errors cannot
be avoided while the teacher of another group explains the factors which may cause errors.
One group of student said that they are given the expected value before doing the experiment
while another group said that they are required to repeat the procedure if the data is not precise
or accurate and still another group said that their teacher give clues when there are deviations
from expected values.
The data confirm that the teachers implement practices which enable the students to find
solutions to problems encountered while performing experiments. They make their students
understand that even the same experiment may lead to different results if performed at
different times or by different people. According to NRC (2005), a well designed scientific
investigation program must include opportunities for students to be involved in activities like
rechecking data observations and analysis and performing the kind of follow-up investigations
that will validate the result of the investigation.
The teaching practices observed among faculty to attain the understanding of complexity and
ambiguity of empirical work of students are shown in Table 11. Teaching practices with a mean
of 2.75 and 2.50 show that they are always implemented by the faculty-respondents. These
practices are giving some clues to students to account for the discrepancy; making the students
take notice of serious experimental errors due to equipment failure; emphasizing the need to
compare data from standards or controls; making students take notice of precision issues and
accuracy issues where accuracy depends on the standardized calibration; and letting students
notice deviations from expected values.
The faculty-respondents often implement those practices that got a mean of 2.25 and 2.00.
These practices are helping students learn to address the challenges inherent in directly
observing and manipulating the material world, including troubleshooting equipments used to
make observations, understanding measurement error, and interpreting and aggregating the
resulting data; emphasizing to students that random error is a normal part of the data and the
data must have random error that cannot be eliminated through careful data collection; and
allowing students to check repeatability of data they gathered.
Table 11. Practices Observed among the Faculty to Develop the Students Understanding of
Complexity and Ambiguity of Empirical Work
Laboratory Practices Mean Verbal

Interpretation
1. Helping students learn to address the 2.25 Often

challenges inherent in directly observing and
manipulating the material world, including
troubleshooting equipment used to make
observations, understanding measurement
error, and interpreting and aggregating the
134
resulting data
2. Making students take notice of precision 2.50 Always
issues and accuracy issues where accuracy
depends on the standardized calibration
3. Emphasizing to students that random error is 2.00 Often
a normal part of the data and data must have
random error that cannot be eliminated
through careful data collection
4. Giving some clues to students to account for 2.75 Always
the discrepancy
5. Making the students take notice of serious 2.75 Always
experimental errors due to equipment failure
6. Letting students notice deviations from 2.50 Always
expected values
7. Allowing students to check repeatability of 2.00 Often
data they gathered
8. Emphasizing the need to compare data from 2.75 Always
standards or controls
OVERALL MEAN 2.44 Always
The overall mean of 2.44 was an indication that the teachers always implement practices that
will attain the understanding of the complexity and ambiguity of empirical work of students.
This shows that the teachers help their students find solutions to problems encountered while
performing experiments because they have the pedagogical content knowledge which
according to Shulmans view as cited by Rowan, et al. (2011) is the knowledge of a teacher of the
difficulties that students encounter when learning particular content.
D. Practical Skills
Table 12 shows the practices of faculty in developing the practical skills of students. All the
four groups of faculty teach their students on the proper use of laboratory equipment and then
check if they acquired the skills on its use by means of practical tests. All of them are particular
with safety precautions such as the use of laboratory gowns. Two groups of faculty check the
data and observations recorded by their students. Two groups of faculty check whether their
students read the procedure before coming to class by giving a pre-lab quiz or by looking at the
amount of reagents they are getting and the sequence of steps they are following.
From their practices, it is clear that the faculty implement teaching practices which develop the
practical skills of their students because they are concerned not only with the proper use of
equipments but also with the safety of the students in following correct procedure. With this
concern, students can apply effectively the appropriate practical skills acquired to a new
investigation similar to the inquiry-based laboratory investigation recommended by NSTA
(2007) where students learn appropriate laboratory techniques to define and solve problems.

135
Table 12. Practices of Faculty in Developing Practical Skills of Students

Training students on the proper use of lab equipments
Faculty of LPU 1 such as measuring devices; checking whether students
are recording their observations correctly; checking the
use of lab gowns and reminding of other safety
precautions
Students of LPU 1 Explaining to students the use of an instrument

demonstrating how it is used and its proper care and
maintenance; putting deductions on those who are not
wearing lab gowns
Demonstrating to students the proper operation of lab
Faculty of LPU 2 equipments per group; checking the data of students in
their manual; checking whether they read the procedure
before the experiment by looking at the amount or
reagents they are getting and the sequence of steps they
are following; constantly reminding them of safety
precautions
Explaining the procedure at the same time teaching
Students of LPU 2 students how to use the equipment; checking whether
students acquire the skill in using equipment during
practical exams; implements wearing of lab gown is a
must; moving around to check the data recorded in the
manual
Faculty of LPU 3 Giving precautionary measures for every experiment;

giving practical test to determine if students learned the
skills in using lab equipment
Students of LPU 3 Emphasizing the precautions written on the manual and

the use of lab gowns; checking the data sheet to find out
if students got the correct observation; teaching students
the proper use of equipments
Faculty of LPU 4 Giving practical exam; giving a pre-lab quiz to

determine whether they read the procedure before
coming to class; not accepting students if they are not in
their lab gown
Students of LPU 4 Not allowing students enter the lab room if not in lab
gown; keep on saying As long as you handle reagents
and instruments properly, no accident will happen

136
As observed by students, all the teachers of the four groups of students require their students to
wear laboratory gowns. Three groups of students are taught by their teachers on the proper use
of laboratory equipment. The teachers of two groups of students check their data while the
teacher of one group of students check their skill in using equipment during practical exams.
Most of the practices of faculty, as observed by students, indicate that the teacher is concerned
with the safety of the students in doing experiment like wearing of laboratory gown because for
them this is one of the skills they must learn in chemistry. This conforms with the belief of
NSTA (2007) that laboratory investigation is well-designed if it gives opportunities to students
develop safe and conscientious laboratory habits and procedures.
The teaching practices observed among the faculty to develop the practical skills of students are
shown in Table 13. As indicated in the table, the faculty always implement the practices with a
mean of 3.00. These practices are requiring students to read and understand procedures before
carrying them out and adapt them as required; checking whether students know how to operate
laboratory equipment and understanding exactly how equipment works before physically
approaching it; reminding the class about safety precautions and checking whether the students
observe the precautions; helping students develop skills in using scientific equipment correctly
and safely, making observations, taking measurements, and carrying out well-defined scientific
procedures; requiring students make and record observations of their experiment; teaching the
students to use measurement devices and to record data with correct precision; and checking
the student response to in the lab report data table for correct accuracy and precision.
Table 13. Practices Observed among the Faculty to Develop Practical Skills of Students

Interpretation
1. Requiring students to read and understand 3.00 Always

procedures before carrying them out and
adapt them as required
2. Providing the students with hints and 0.25 Never
suggestions on possible experimental design
and encouraged students to try their own
ideas
3. Checking whether students know how to 3.00 Always
operate laboratory equipment and
understanding exactly how equipment works
before physically approaching it
4. Reminding the class about safety precautions 3.00 Always
and checking whether the students observe
the precautions
5. Allowing students to visually describe the 2.25 Often
procedures of the experiment before actually
doing it
137
6. Helping students develop skills in using 3.00 Always

scientific equipment correctly and safely,
making observations, taking measurements,
and carrying out well-defined scientific
procedures
7. Requiring students make and record 3.00 Always
observations of their experiments
8. Teaching the students to use measurement 3.00 Always
devices and to record data with correct
precision
9. Providing opportunities for students to take 2.75 Always
readings from equipments
10. Checking the student response in the lab 3.00 Always
report data table for correct accuracy and
precision
11. Encouraging students to deviate from given 0.25 Never
procedures if they know what they are doing
12. Encouraging students to consider alternative 0 Never
procedures and providing them with sufficient
instructions to succeed
13. Checking whether students really acquired the 2.75 Always
necessary skills in the experiment
OVERALL MEAN 2.25 Often
Similarly, providing opportunities for students to take readings from equipment and checking
whether students really acquired the necessary skills in the experiment, are also implemented
always by the faculty because they got a mean of 2.75. Allowing students to visually describe
the procedures of the experiment before actually doing it got a mean of 2.25 which means that
this is often implemented by the faculty. However, practices with a mean of 0.25 and 0 are
never implemented by the faculty. These practices are providing the students with hints and
suggestions on possible experimental design and encouraged students to try their own ideas;
encouraging students to deviate from given procedures if they know what they are doing; and
encouraging students to consider alternative procedures and provide them with sufficient
instructions to succeed.
In general, the overall mean of 2.25 was an indication that the teachers often implement
practices that develop the practical skills of their students. It reflects that they are doing these
practices because they believe that with those teaching practices they can enhance the skills of
students in employing a systematic and scientific methodology which in turn according to
Salandanan (2002) will enable their students to experience step-by-step procedure in finding
answers to their endless questions. This is supported by Monis, et al. (2007) statement that
teachers must teach skills to students with the expectation that competencies in skills would
support open-ended, student-driven explorations.

138
E. Understanding the Nature of Science
Table 14 presents the practices of the faculty in developing students understanding of the
nature of science.
Table 14. Practices of Faculty in Developing Students Understanding of the Nature of Science

Telling the students that their previous knowledge may
Faculty of LPU 1 or may not affect their conclusion; helping students to
overcome errors by following procedures correctly;
allowing students to interpret their data based on their
own understanding
asking students own interpretation of the data, however
Students of LPU 1 at the end she will give the actual interpretation by
relating it to the theory; explaining the source of error
and then giving students another chance to correct their
error; correcting students misconceptions
Asking students at the start of the experiment to give
Faculty of LPU 2 their knowledge about the concept and telling them that
their knowledge may or may not affect their conclusion;
giving tips to students on how to overcome errors
Students of LPU 2 Advising students to overcome error; asking students

about their ideas, then compare our ideas and she will be
the one to tell which is correct
Gathering preconceptions of students before the
Faculty of LPU 3 experiment; making explanations if after the experiment
the result is contradictory to their preconception; telling
students that errors are normally encountered but it can
be overcome
Students of LPU 3 Asking students to trace the cause of error so that next
time they can avoid it; always tell the students that they
cannot simply rely on what they previously know, they
have to discover more
Faculty of LPU 4 Emphasizing to students that their ideas might be pure

misconceptions; telling students that errors may be
overcome only by correct techniques
Students of LPU 4 Often telling students that different persons have

different interpretation of the result but whatever the
interpretation is, it depend on their previous
understanding of the concept

139
All the four groups of faculty advise their students to overcome their errors and also
emphasizing to their students that each persons preconceptions or prior knowledge may or
may not affect the final conclusion. Only one group of faculty allow their students to interpret
their data based on their own understanding.
The practices of the faculty indicate that the teachers are implementing practices that make
students understand that science is a way of knowing aside from being a human endeavour. It
is a way of knowing because the prior knowledge the students have may or may not be
contrary to the result of their experiment. If the prior knowledge is a misconception then the
teacher must find ways to correct it, however if the previous knowledge conforms with the
result of the experiment, then the teacher must give opportunities to student to construct or
build new ideas. This conforms well with Piagets Theory of Constructivism (Muijs and
Reynolds, 2011) which states that knowledge is always a construction by the learner where the
student actively construct new concepts based upon prior knowledge and new information. On
the same level, this reflects Singers, et al. (2005) statement that teachers must be challenged
with the intuitive ideas of students by helping them move towards a more scientific
understanding through change in and not merely an addition to what students notice and
understand about the world.
To the students, the teachers of the three groups of students advised them to overcome error, to
trace the cause or source of error and to correct their error. Two groups of students were asked
by their teacher to make their own interpretation of the data or result of the experiment and
telling that different persons have different interpretation. The teacher of one group of students
corrects the previous knowledge of students if it is a misconception, another group of students
were asked to discover more and not simply rely on their previous knowledge, while the
teacher of another group of students tells the students that their interpretation depends on their
previous knowledge.
The data showed that the teachers design laboratory instruction that develop the students
understanding of the nature of science because they are trying to emphasize that knowledge is
subject to change. This supports Crowthers, et al. (2005) suggestion to teachers to design
lessons around science topics or concepts that have changed over time and the instruction must
be explicit on how knowledge has changed and why.
Table 15 presents the practices observed among the faculty to develop the students
understanding of the nature of science. As shown in the table, the faculty always emphasize to
the students that each persons preconceptions may or may not affect the final conclusion. It
has a mean of 2.75. They often allow the students discover that different people may interpret
the same data differently. This got a mean of 1.75. Advising the students to help them
overcome errors, and discover that science is not as simple or as black and white as they may
have thought got a mean of 1.75. This is also often implemented by the faculty.
Table 15. Practices Observed Among the Faculty to Develop Students Understanding of the
Nature of Science

140
Interpretation
1. Allowing the students discover that 1.75 Often

different people may interpret the same
data differently
2. Emphasizing to the students that each 2.75 Always
persons preconceptions may or may not
affect the final conclusion
3. Advising the students to help them 1.75 Often
overcome their errors, and discover that
science is not as simple or as black and
white as they may have thought
As a general picture, the overall mean of 2.08 was an indication that the teachers often
implement practices that develop the students understanding of the nature of science. It could
reflect that these teachers want to emphasize in their instruction that knowledge is subject to
change because of the different types of investigations that provide different information and
evidence concerning the natural world. Crowther, et al. (2005) reflect the same argument when
he said that scientific knowledge in and of itself is not static and that with new information,
scientific theories can change.
F. Interest in Science and Interest in Learning Science
The practices of the faculty in cultivating students interest in science and interest in learning
science are shown in Table 16.
Table 16. Practices of Faculty in Cultivating Students Interest in Science and Interest in
Learning Science
Faculty of LPU 1 Making students realize how important their lesson is to

their daily life situations; challenging them to find out
for themselves the possible result if a certain situation
may happen
Students of LPU 1 Asking students to relate the experiment to real life

situation; at the end of our lesson, the teacher often
leaves a question for the student to answer
Faculty of LPU 2 By emphasizing the relevance of what they are studying

to their future job; by showing some magic in chemistry
141
during pre-lab discussion
Students of LPU 2 Often telling us stories on how she apply the knowledge
in chemistry in her on-the-job training; the teacher
trigger students interest in the topic at the start of the
lesson
Faculty of LPU 3
Telling simple jokes about chemistry to motivate
students; conducting plant visits to chemical industries
Students of LPU 3 making the lesson not boring by telling funny stories
about chemistry; taking students on field trips
Faculty of LPU 4 Stating the relevance of the lesson; connecting the lesson
to real world experiences
Students of LPU 4 Asking students to give practical applications of what

they learned in chemistry
All of the four groups of faculty provide practical and real life situations where the experiment
is applicable by making students realize how important their lesson is to their daily life
situation, emphasizing the relevance of what they are studying to their future job, conducting
plant visits to chemical industries, stating the relevance of the lesson and connecting the lesson
to real world experiences. Two out of the four groups of faculty motivate their students at the
beginning of the lesson by showing some magic in chemistry and telling simple jokes about
chemistry. Only one group of faculty said that they challenged their students to find out for
themselves the possible result if a certain situation happens.
The practices of the faculty proved that these teachers have a great desire to develop positive
attitudes among their students towards chemistry and make them highly motivated to continue
learning chemistry. Positive attitude is developed if a student is highly motivated and this can
be done by the teacher through improving the teaching practices and by showing to the
students the relevance of the topic to their everyday lives. This finding is similar with that of
Movahedzadehs (2011) findings that students lose interest in science when the teaching of its
context seems irrelevant to their lives or even to their future jobs.
As observed by students, the teachers of the four groups of students emphasize the application
of their learning in chemistry by asking them to relate the experiment to real life situation,
telling stories on its application to on-the-job training, going on field trips, asking them to give
its practical applications. The teachers of two groups of students motivate them or trigger their
interest in the topic at the beginning of the lesson and make their lesson not boring by telling
funny stories about chemistry. The teacher of one group of students often leaves a question for
the student to answer.

142
The table clearly shows that the teachers engage themselves in a more interesting approach that
will make the students see the value of chemistry and will motivate them to develop a positive
attitude towards the subject. The practices implemented by the teachers are in conformity to
the teaching principle of Kanli and Yagbasan (2008) of exciting students by making a spark
about the subject.
The practices observed among the faculty that will cultivate students interest in science and
interest in learning science are presented in Table 17.
Table 17. Practices Observed Among the Faculty to Cultivate StudentsInterest in Science and
Interest in Learning Science
Laboratory Practices Mean

Verbal
Interpretation
1. Providing avenue where interest of students 2.50 Always

are triggered making them more eager to find
out the answer through experimentation
2. Illustrating how alive science can become if 1.75 Often
lab experiences are not limited to routine
classroom laboratory
3. Providing practical and real life situations 2.50 Always
where the experimental set-up is applicable
4. Providing thought-provoking questions that 2.00 Often
compels students to find out things by
themselves
As shown in the table, the teachers always implement practices such as providing avenue
where interest of students are triggered making them more eager to find out the answer
through experimentation and providing practical and real life situations where the
experimental set-up is applicable. These practices got a mean of 2.50. Practices with a mean of
2.00 and 1.75 are often implemented by the faculty: providing thought-provoking questions
that compel students to find out things by themselves; and illustrating how alive science can
become if lab experiences are not limited to routine classroom laboratory.
In general, the overall mean of 2.19 was an indication that the teachers often implement
practices to cultivate the interest of students in science and their interest in learning science. It
could be generalized that these teachers are committed to their desire of creating a learning
environment that will encourage and inspire students to have a desire to learn and enjoy
learning. These teachers have a vision in mind of making their students become future scientist
who will contribute to the progress of the nation, similar to what Salandanan (2002) had said.
According to her, with high motivations, students will decide to pursue a science profession in
143
the future and will develop a feeling of gratitude and appreciation for the advances in science
and technology that continue to raise the present quality of life.
G. Teamwork Skills
The practices of the faculty in developing teamwork skills among students are presented in
Table 18.
Table 18. Practices of Faculty in Developing Teamwork Skills of Students

Grouping students alphabetically; grouping is
Faculty of LPU 1 permanent throughout the entire semester; assigning of
leader in a group who is responsible for dividing the
task among members; rotational leadership is
implemented
Students of LPU 1 Grouping the class into 5 members each; assigning a

different group leader for every experiment; never
regrouping students instead making them work with
their group mates the whole semester
Faculty of LPU 2 Division of labor among the members of each group;

there is rotational leadership; not allowing regrouping
Students of LPU 2 Encouraging students to participate with their group

mates; giving chance to those who are willing to be the
leader to lead the group; not regrouping the members of
the group
Giving tasks to each member of the; asking a member of
Faculty of LPU 3 the group to help his group mate if he finished earlier;
making students work with their group mates all
throughout the semester; assigning anyone to be the
leader of the group
Students of LPU 3 Requiring each member of the group to work on

different procedure so as to finish the experiment at
once; assigning leader by rotation
Faculty of LPU 4 Assigning leader for every experiment who monitors the
performance of others; grouping is permanent in the
whole semester; there is division of labor among
members
Students of LPU 4 Asking the group leader to assign specific task for each
144
member; giving everybody the chance to be the leader;

not allowing students to transfer to another group
As shown from the table, all the four groups of faculty group their students into smaller groups.
A leader is assign for every group and there is rotational leadership. However, these four
groups of faculty never allow regrouping of students; instead the students work with their
permanent group mates all throughout the semester. To compensate, all the four groups
employ division of tasks among the members of the group.
The table also indicates that the teachers implement practices that develop teamwork skills
among students. They group students into smaller groups possibly to enable students
collaborate effectively with others in carrying out complex tasks. They also divide the tasks
among the members of the group maybe to make students contribute and respond to ideas of
others. Leadership is on a project basis so that students will assume different roles at different
times. They do not allow regrouping of students but make them work with their permanent
group all throughout the semester possibly because they want their students to establish
harmonious relationship with their group mates and such relationship will lead to the success
of the experiment.
Such findings are similar to those of Halls (2006) study where a system was implemented in a
way that specific roles were assigned to students during laboratory and a grade was given
based on their level of contribution to the group. As observed by students, the teachers of the
four groups of students group the class into smaller groups and assign a group leader for every
experiment performed. All of them were not allowed by their teacher to regroup or transfer to
another group during the whole semester.
The data proved that the teachers are implementing practices that develop the teamwork skills
of students because these teachers want their students to interact with each other so that they
can share their knowledge through performing specific tasks. This conforms with Kanli and
Yagbasan (2008) laboratory principle of exchange where teachers prepare proper environment
for students to discuss their ideas with their friends, observe and listen to students who are
sharing their knowledge and ensure the interaction within student groups. The practices
observed among the faculty that will develop teamwork skills of students are presented in
Table 19.
Table 19. Practices Observed Among Faculty to Develop Teamwork Skills Among Students

Interpretation
1. Making students collaborate effectively with 3.00 Always

others in carrying out complex tasks, share the
work of the task, assume different roles at
different times, and contribute and respond to
ideas
2. Making students work in the same group 2.75 Always
145
throughout the entire semester

3. Allowing students to regroup during the 0.50 Never
semester
4. Allowing students to take rotational and 3.00 Always
specific active roles in the group
OVERALL MEAN 2.31 Always
As shown from the table, the teachers always implement those practices that got a mean of 3.00.
Such practices are making students collaborate effectively with others in carrying out complex
tasks, share the work of the tasks, assume different roles at different times, and contribute and
respond to ideas; and allowing students to take rotational and specific active roles in the group.
The practice making students work in the same group throughout the entire semester got a
mean of 2.75 and is always implemented by the teachers. However, the teachers never allow
students to regroup during the semester garnering a mean of 0.50.
As a whole, the overall mean of 2.31 was an indication that the teachers always implement
practices that will develop students teamwork skill. It could mean that these teachers are
familiar with small group instruction which according to Hidalgo (2000) is an effective strategy
that enhances cooperation, teamwork, leadership and group motivation among students.
II. Extent by which Students Manifest the Attainment of the Goals of Science
Laboratory Instruction in their:
A. Attitude and Motivation
The extent by which students manifest the attainment of their interest in chemistry is reflected
in Table 20. It can be gleaned from the table that the 80 students have positive attitude towards
chemistry as indicated by the overall mean of 4.14 and a standard deviation of 0.42. The mean
of their attitudes toward chemistry ranged from 3.54 to 4.41.
Table 20. Extent by which Students Manifest the Attainment of Interest in Chemistry
Item Mean Standard Verbal

Deviation Interpretation
1. Learning chemistry requires a serious effort 4.41 0.71 Positive

and special talent
2. Reasoning skills that are taught in chemistry 4.14 0.74 Positive
can be helpful to me in my everyday life
3. For me doing well in chemistry courses
depends on how well the teacher explains 4.39 0.75 Positive
things in class
4. Understanding chemistry gives me a sense of

146
accomplishment 4.31 0.63 Positive

5. Theories and scientific laws in chemistry are
difficult to understand 3.54 0.93 Positive
6. How well I do in chemistry exams depends on
how well I can recall material in the way it 4.13 0.6 Positive
was presented in class
7. Learning chemistry has helped me to
understand situations in my everyday life 4.08 0.85 Positive
OVERALL MEAN
4.14 0.42 Positive
It could mean that these students view chemistry as a subject which requires serious efforts and
special talents because the theories and scientific laws are not easy to be understand. Further,
the student could harbor the feeling that their achievement in chemistry depends not only on
how they can recall materials but also on how their teacher presented or explain it in class.
However, they believe that chemistry can help them in their everyday life and understanding
the subject gives them a sense of accomplishment. This finding is similar to the idea of Zulueta
and Guimbatan (2002) that students enjoy goal-oriented activities and practical work where
they can see the relevance of abstract concepts and principles and consequently become
interested in sciences. Table 21 presents the extent by which students manifest the attainment of
their interest in learning chemistry.
Table 21. Extent by which Students Manifest the Attainment of Interest in Learning Chemistry

1. If I am having trouble learning chemistry, I 4.06 0.64 Highly

try to figure out why motivated
2. I have a real desire to learn chemistry 4.08 0.71 Highly

motivated
3. The subject has created a knowledge-base 4.19 0.86 Highly

which will help me in my career motivated
4. I put enough effort into learning chemistry 3.95 0.78 Highly

motivated
5. I use my imagination and creativity in doing

scientific investigations 4.08 0.58 Highly
motivated
6. The chemistry lecturers have made me feel
147
that I have the ability to pursue my study in 4.12 0.64 Highly

chemistry motivated
7. I am willing to master the knowledge and

skills in chemistry course 3.74 0.85 Highly
motivated
8. When learning chemistry, I prefer to put
concepts/ideas in my own words 3.94 0.83 Highly
motivated
OVERALL MEAN
4.02 0.48 Highly
motivated
It appears from the table that 80 students are highly motivated by their teachers to learn
chemistry as revealed by the overall mean of 4.02 and a standard deviation of 0.48. The mean of
their motivations ranged from 3.74 to 4.19.
The results mean that these students have willingness to learn chemistry because they believe
that chemistry has created a knowledge-base which will help them in their career as a result of
the encouragement their teachers have given them to pursue their study in the subject. Their
great desire to learn chemistry is reflected in them using their imaginations and creativity in
doing scientific investigations, figuring out why they are having trouble learning chemistry,
putting enough effort to learn chemistry by putting concepts/ideas in their own words and
willingness to master the knowledge and skills in chemistry course.
The findings conform to Salandanans (2002) statement saying that wholesome attitudes of
students may be developed by awakening their interest and keeping them highly motivated to
inquire about occurrence in the natural environment. She added that students must relentlessly
pursue a scientific investigation and be responsible enough to complete an assigned task despite
constraints.
The extent by which the attainment of the understanding of the nature of science is manifested
by students is presented in Table 22. The students have much understanding of the nature of
science as indicated by the overall mean of 3.91. The mean of their understanding ranged from
3.54 to 4.08. No matter how difficult the theories and scientific laws in chemistry are, still these
students can understand situations in everyday life. It suggests that they have the ability to
interpret data from the material world because they put concepts/ideas in their own words and
use their imaginations and creativity to do scientific investigations.
Table 22. Extent by which Students Manifest their Understanding of the Nature of Science


148
1. Theories and scientific laws in chemistry are 3.54 0.93 Much

difficult to understand understood
2. I use my imagination and creativity in doing 4.08 0.58 Much

scientific investigations understood
3. When learning chemistry, I prefer to put 3.93 0.83 Much

concepts/ideas in my own words understood
4. Learning chemistry has helped me to 4.07 0.85 Much

understand situations in my everyday life understood
OVERALL MEAN 3.91 Much

understood
These findings are in consonance with the statement of Crowther, et al. (2005) that in teaching
scientific laws, teachers must emphasize how these laws describe nature and how things act
under certain conditions. It should be taught also that questions lead to investigation and
experiments then lead to conclusions - but still there are many different pathways that scientists
take.
B. Laboratory Skills
The extent by which the students manifest the attainment of practical skills is shown in Table
23.
Table 23. Extent by which Students Manifest the Attainment of Practical Skills
Skills Number Percent Verbal
of Interpretation
Groups
of
Students
A. Handling Liquids and Measuring Volume
1. Places the cover of the reagent bottle on the 18 100 Highly competent
table in an upside down position

2. Uses a pipette correctly in getting liquid 16 89 Highly competent
chemicals from the reagent bottle
3. Reads the volume of liquids precisely using a 12 67 Competent
graduated cylinder
4. Measures exact volume of liquids with a 13 72 Competent
pipette
Average 15 82 Highly competent
B. Handling Solids and Weighing

Moderately
149
1. Sets the scale to zero before starting to weigh 6 33 competent

Highly competent
2. Places the object on the left pan and the set of 18 100
masses on the right pan
Highly competent
3. Uses a dry spatula in getting solids from the 14 79
reagent bottle Highly competent
4. Avoids using bare hands in handling 16 89
chemicals Highly competent
5. Obtains accurate weight using platform 14 79
balance Moderately
competent
6. Uses a paper lining in introducing solids into 6 33
test tube
Competent
7. Avoids returning unused reagents to the 13 72
reagent bottle Competent
8. Discards solid wastes into an appropriate 11 61
waste container Competent
12 67
Average
Moderately
C. Bunsen Burner Manipulation competent
1. Lights the Bunsen burner properly by 7 39
closing the air inlet then lighting the burner
from the side of the barrel going up Highly competent
2. Regulates the amount or flow of gas properly 15 83
so as to get an ideal height of the flame Highly competent
3. Produces a non-luminous flame by opening 18 100
the air inlet
Competent
Average 13 74
D. Heating Substances in a Test tube and Doing Highly competent

Evaporation 18 100
1. Heats the test tube in an inclined position (45
angle) moving it back and forth while heating Highly competent
2. Not pointing the mouth of the test tube to 16 89

anybody while heating Highly competent
3. Follows the proper set-up for evaporation 16 89
Highly competent
Average 17 93
E. Doing Filtration Highly competent

1. Folds the filter paper correctly 18 100 Highly competent
2. Follows the proper set-up for filtration 16 89

Highly competent
Average 17 95
F. Safety Considerations Highly competent

150
1. Wears lab gown properly 18 100

Not competent
2. Wears appropriate goggles all the time 3 17 Highly competent
3. Wears appropriate clothes and footwear 18 100
Competent
Average 13 72
Highly competent
Overall Percent 81
It can be noted from the table, that as to handling liquids and measuring volume, 18 groups of
students or 100 % placed the cover of the reagent bottle on the table in an upside down position;
16 groups or 89 % used a pipette correctly in getting liquid chemicals from the reagent bottle; 13
groups or 72 % measured exact volume of liquids with a pipette; and 12 groups or 67 % read the
volume of the liquids precisely using a graduated cylinder.
As a general view, about 15 groups or 82 % of the students are highly competent in handling
liquids and measuring volume. It could be that these students had acquired skills in handling
liquids and measuring volume even when they were still in high school because they have
already performed similar laboratory activity before. This finding affirms Marines (2003) idea
that if an experiment is repeated it can greatly help students to understand or improve their
laboratory techniques.
As to handling solids and weighing, 18 groups or 100% placed the object on the left pan and the
set of masses on the right pan; 16 groups or 89 % avoid using their bare hands in handling
chemicals; 14 groups or 79 % used a dry spatula in getting solids from the reagent bottle and
obtained accurate weight using platform balance; 13 groups or 72 % avoid returning unused
reagents to the reagent bottle; 11 groups or 61 % discarded solid wastes into an appropriate
waste container; and 6 groups or 33 % only set the scale to zero before starting to weigh and
used paper lining in introducing solids into test tube.
In sum, only 67 % or 12 groups of students are competent in handling solids and weighing. It
seems that not all students have acquired the necessary skills in handling solids and in
weighing as justified by the number of students who were able to set the scale to zero before
weighing and those who used paper lining in introducing solids into test tube. This can mean
that these students are careless in following the correct techniques that they always work in a
hurry for the purpose of finishing the experiment at once without considering the accuracy of
what they are doing. They do not understand measurement error which may affect the result of
their experiment. This finding conforms with Singers, et al. (2005) recommendation to teachers
to help students learn to address the challenges inherent in directly observing and manipulating
the material world including the understanding of measurement error.
In terms of Bunsen burner manipulation, 18 groups or 100 % were able to produce a non-
luminous flame by opening the air inlet; 15 groups or 83 % were able to regulate the amount or
flow of gas properly so as to get an ideal height of the flame; while only 7 groups or 39 % light

151
the Bunsen burner properly by closing the air inlet then lighting it from the side of the barrel
going up.
In general, only 13 groups or 74 % of the students are competent in manipulating the Bunsen
burner. This can be a clear indication that although students are familiar with the use of a
Bunsen burner, still not all of them know how to light it properly. It could be that these
students were not listening nor watching the demonstration made by their teacher during pre-
lab discussion on the proper way of lighting the Bunsen burner. Demonstration method
according to Garcia as cited by Acero, et al. (2000), is an imitative method where learning a skill
is faster and more effective since students are shown how the job is done by using actual tools,
machines and materials.
As regards the skill heating substances in a test tube and doing evaporation, 18 groups or 100 %
of students heated the test tube in an inclined position moving it back and forth while heating;
16 groups or 89 % were not pointing the mouth of the test tube to anybody while heating and
followed the proper set-up for evaporation. Almost 17 groups or 93 % of the students are
highly competent in heating substances in a test tube and doing evaporation. It appears that
these students had encountered minor accidents on heating substances in a test tube and
evaporation during their high school chemistry that they now developed the proper techniques
from their previous mistakes. This affirms Jonas (2008) statement that mistakes encountered by
students during experiments are not hindrances but they are opportunities for greater learning.
On the other hand, doing filtration is easy for 18 groups or 100 % of the students who were able
to fold the filter paper correctly while 16 groups or 89 % followed the proper set-up for
filtration. Thus, 17 groups or 95 % of the students are highly competent in doing filtration. It
suggests that the students have acquired the skills in filtration because they were taught about
the principles of filtration and how to do filtration. These findings affirm Monis, et al. (2007)
idea that students must be taught of the differences among knowing about a topic, knowing
how to complete a skill, showing how to complete a skill and doing the skill. This is done
through integration of skills development with conceptual learning.
In terms of safety considerations, 18 groups or 100% of the students wore laboratory gown
properly and appropriate clothes and footwear while 3 groups or 17 % only wore appropriate
goggles all the time. As a whole, only 13 groups or 72 % of the students are competent with
regards to safety considerations. It could mean that those students found it inconvenient to
wear goggles while doing experiments since they are not used to it aside from they were not
given strict implementation on its use. This is in contrary to NSTAs (2007) suggestion to
teachers of giving the students opportunities to develop safe and conscientious laboratory
habits and procedures. In general, the students are highly competent in their practical skills as
justified by an over-all percentage of 81 %.
Table 24. Extent by which Students Manifest the Attainment of Teamwork Skills
Number
Skills of Percent Verbal
Groups Interpretation
of

152
Students
1. Tries to get other team members involved 16 89 Highly

Competent
2. Presents ideas about how to work on the task 8 44 Moderately

Competent
3. Enjoys working on the team 16 89 Highly

Competent
4. Questions others task ideas constructively 8 44 Moderately

Competent
5. Tries to get other team members to voice their 8 44 Moderately

opinions about ideas on the table Competent
6. Responds calmly to others 16 89 Highly

Competent
7. Tries to raise alternatives that werent on the table 6 33 Moderately

Competent
8. Helps explain other ideas 14 78 Highly

Competent
9. Integrates ideas of different members 12 67 Competent
10. Responds appropriately to any questions 16 89 Highly

presented in the group Competent
Overall Percent 67 Competent
The extent by which the students manifest the attainment of teamwork skills is shown in Table
24. It can be gleaned from the table that out of the 18 groups of students, only 16 groups or 89
% tries to get other team members involved; enjoys working on the team; responds calmly to
others; and responds appropriately to any questions presented in the group. Fourteen groups
or 78 % help explain other ideas while twelve groups or 67 % integrate ideas of different
members. Eight groups or 44 % present ideas about how to work on the task; question others
task ideas constructively; and try to get other team members to voice their opinions about ideas
on the table. Only 6 groups or 33 % try to raise alternatives that werent on the table.

153
To sum up, only 12 groups or 67 % of the students are competent in their teamwork skills. It
seems that not all students attained teamwork skills because they did not demonstrate a true
collaborative work. It reflects that they work in group for the purpose of dividing limited
laboratory equipment and space among a large number of students. This is contrary to NRCs
(2005) idea of teamwork that requires high level of substantive conversation. There is high level
of substantive conversation if there is considerable interaction about the ideas of a topic and if
there is sharing of ideas.
Table 25 presents the extent by which students manifest the attainment of the understanding of
complexity and ambiguity of empirical work. It can be noted from the table that 18 groups or
100 % of students have knowledge on troubleshooting equipment. Seventeen groups or 94 %
take notice of deviations from expected values. Fifteen groups or 83 % take notice of
experimental errors due to equipment failure while only 7 groups or 39 % take notice of
precision issues and accuracy issues. It appears that almost all of the 13 groups or 71% of the
students understood the complexity and ambiguity of empirical work because they can find
solutions to problems encountered while doing experiments as in troubleshooting equipment.
It could be that they were given proper instructions on the proper use and maintenance of
equipments. Most of them can take notice
Table 25. Extent by which Students Manifest the Attainment of the Understanding of
Complexity and Ambiguity of Empirical Work
Number
Skills of Percent
Groups Verbal
of Interpretation
Students
1. Has knowledge on troubleshooting equipment 18 100 Highly

Understood
2. Takes notice of precision issues 7 39 Much

Understood
3. Takes notice of accuracy issues 7 39 Much

Understood
4. Takes notice of experimental errors due to 15 83 Highly

equipment failure Understood
17 94 Highly
5. Takes notice of deviations from expected value Understood
Overall Percent 71 Understood

154
of deviations from expected result and can take notice of experimental errors due to equipment
failure possibly because their teachers have emphasized to them that errors cannot be avoided
when performing experiment. Instead students must know how to deal with these
experimental errors. This conforms with Jonas, et al. (2008) statement that a well designed
scientific investigation must allow students to understand measurement error.
C. Achievement in Chemistry
Table 26 shows the extent by which the students manifest the attainment of mastery of subject
matter and of scientific reasoning.
Table 26. Extent by which Students Manifest the Attainment of Mastery of Subject Matter and
Scientific Reasoning
Mean
LPU 1 LPU 2 LPU 3 LPU 4 Average Percent

Goal of Instruction Mean
Enhancing Mastery 11.91 11.84 12.54 14.23 12.63 60.14

of Subject Matter
Developing 28.78 23.33 22.18 22.15 24.11 63.45

Scientific Reasoning
It can be noted from the table that enhancing mastery of subject matter got an average mean of
12.63. Out of the 21 questions about mastery of subject matter, the highest score obtained by the
students is 18 and the lowest score is 7. Students from LPU 4 got the highest mean of 14.23
while students from LPU 3, LPU 1 and LPU 2 got a mean of 12.54, 11.91 and 11.84 respectively.
It appears that the students have attained an average level of mastery of subject matter as
justified by the mean of 12.63 which is about 60.14 %.
Developing scientific reasoning got a mean of 24.11. Out of the 38 questions about scientific
reasoning, the highest score obtained by the students was 36 and the lowest score was 10.
Students from LPU 1 got the highest mean of 28.78 while students from LPU 2, LPU 3 and LPU
4 got a mean of 23.33, 22.18 and 22.15 respectively. It appears that the students had attained an
average level of scientific reasoning because the mean is 24.11 which is about 63.45 %.
The students attained an average level of mastery of subject matter because they can readily
understand and apply the concepts they have learned. It could be that their teachers taught
content and process simultaneously. This affirms Jonas, et al. (2008) idea that mastery of
subject matter could be attained if concept and processes are taught simultaneously so that in
performing a process the student has clear understanding of the relation of that process to
content.

155
The students attained an average level of scientific reasoning possibly because they were
trained how to construct scientific arguments where they will use their reasoning skills. This is
in consonance to Jonas, et al. (2008) statement that students should be taught of the various
scientific processes and valid reasoning principles and at the same time must be given
opportunities to practice those reasoning skills.
III. Model of Teaching Practices in Chemistry Laboratory to Attain the Goals of Science
Laboratory Instruction
Laboratory experiences in chemistry are important for students to gain a deeper sense of
understanding and a greater confidence in learning. With the acknowledged importance of a
laboratory experience for all students, it is necessary for chemistry teachers to conceptualize
clearly the elements that make up an effective and well-designed laboratory instruction.
The chemistry faculty of the Lyceum of the Philippines University strive to provide their
students with access to a more authentic laboratory experience by complying with the
universitys vision, mission, goal and objective (VMGO). The design of their instruction is
tailored in accordance with what is stated in the purposes of the university. The university
VMGO was simplified using the acronym S-E-R-V-E to make it more realistic for all
stakeholders. Each letter in the acronym has a corresponding meaning on which the faculty
patterned their teaching practices and strategies.
Based from the findings of the present study, the LPU faculty, although did not fully implement
the ideal practices as stated in the seven goals of science laboratory instruction, were still able to
develop students positive attitude, laboratory skills and high achievement in chemistry. This
could be due to the uniqueness of their teaching practices that were anchored on the
universitys VMGO which are parallel to the seven goals of science laboratory instruction.
A model of teaching practices in chemistry laboratory was proposed from the identified best
teaching practices of chemistry faculty of LPU with the hope of contributing to the body of
knowledge in science education. The identified best teaching practices of chemistry faculty of
LPU are shown in Figure 3. The figure consists of four rectangular boxes. The first box shows
the purposes of LPU which were embodied in its VMGO.

156
Letter S means to seek excellence in all the things that we do; E means to exert efforts to teach
and motivate; R means to respect the opinion and efforts of others; V means to vigorously
pursue the virtue of humility; and E means to enjoy the challenges that are in your hands. The
second rectangular box contains the seven goals of science laboratory instruction. Each purpose
in the first box is connected by a line to a goal or goals in the second box to indicate that the
purposes of LPU are parallel with the seven goals of science laboratory instruction. The third
rectangular box represents the students manifestation of the attainment of the goals of science
laboratory instruction in their achievement, attitude/ motivation, and laboratory skills. Arrows
connect the goals in the second box to the third box to indicate that the goals of science
laboratory instruction are attained if they are manifested in the high achievement, positive
attitude, high motivation and competencies in laboratory skills of students. A fourth and final
rectangular box contains the identified best teaching practices of the chemistry faculty of LPU
which were anchored on the purposes of the university.

157
Twenty best teaching practices of LPU chemistry faculty were identified and were categorized
under each purpose. Since each purpose is parallel to the goals of science laboratory
instruction, this means that although the LPU faculty did not fully implement the ideal teaching
practices stated in the goals of science laboratory instruction, they were still able to attain those
goals as manifested in the achievement, attitude, motivation and laboratory skills of their
students.
In order to seek excellence in all the things being done, the LPU chemistry faculty, in designing
their laboratory instruction, integrate teaching practices that will attain the mastery of subject
matter and scientific reasoning and will lead to the high achievement of students in chemistry.
Enhancement of mastery of subject matter may be attained by giving a pre-lab quiz to students
before conducting a pre-lab discussion; requiring students to submit a flow diagram of the
procedure of the experiment prior to its actual performance; outlining the procedure and
drawing the experimental set-up on the board; and requiring students to make a logbook of the
experiments. By simply giving a short pre-lab quiz at the start of the class where students will
be asked to give the title of the experiment, the reagents and equipments to be used and even
the objectives of the experiment, the teacher assesses if the students read the experiment and
already have an idea about the experiment. Diagnostic, formative assessments when embedded
into the instructional sequences can be used to gauge students understanding. Requiring
students to submit a flow diagram of the procedure prior to the actual performance of the
experiment will enable the student to organize information that will increase the students
retention of concepts. A chemistry laboratory teacher must outline the procedure and draw the
experimental set-up on the board in order to make students understand the process and make
the learning outcomes clear to the learners. Students must be required to make a logbook of the
experiment where they can record not only the results of their experiment but also the learning
they got from the experiment. With this logbook, students can make multiple representations
that will show the correlation between the results of the experiment and the concepts previously
learned.
Teaching practices implemented by LPU chemistry faculty that will attain the development of
students scientific reasoning are asking students the purpose of doing a certain procedure
while performing an experiment; and making students reflect on their own learning by sharing
their experiences while doing the experiment to the other groups of students during post-lab
discussion. Students are trained to use their reasoning skills when they were asked on the
purpose of doing such a procedure. The use of student reflection and discussion signify that the
faculty supports metacognition and student self-regulation where they can control their own
learning.
In exerting efforts to teach and motivate, the LPU chemistry faculty implement teaching
practices to attain the interest of students in science and their interest in learning science and
also the students understanding of the nature of science. This in turn will lead to a positive
attitude and high motivation of students in chemistry. The students interest in learning science
can be developed if the faculty post the list of the top 10 or top performers on the door outside
the laboratory room after each major examination or after the midterm grade had been released;
and the teacher uses technical terms related or appropriate for the course of the student during
pre-lab and post-lab discussion. Students will be inspired and motivated to strive harder in
158
learning chemistry if his name is included in the list of top performers. Using technical terms
appropriate to the course of the students will enable the students see the relevance of the subject
to their future job thereby motivating them to pursue their study.
A teaching practice implemented by LPU chemistry faculty that will attain the students
understanding of the nature of science is the faculty accepts all the interpretations made by each
group regarding the experimental data obtained without telling the students that they are
wrong. In this way, the teacher can emphasize to the students that different people may
interpret the same data differently depending on the steps they followed in making their
scientific investigations.
Respecting the opinion and efforts of others so that they will also respect yours is also very
important in developing teamwork skills among students. The LPU chemistry faculty inculcate
this virtue of respecting one another leading to the development of teamwork skills of students
by implementing practices such as properly arranging the seats so that students can join their
group mates not only during the actual performance of the experiment but also during pre-lab
and post-lab discussion; asking the bright student in each group to guide his group mates and
do peer tutoring; allowing students to discuss among themselves the result of the experiment
before recording it on the data sheet; and assisting students in getting and returning materials
from the stock room. If students are seated together with their group mates from the start until
the end of the class, they will become close to each other so that there will be harmonious
relationship in the group and they will enjoy working as a team. Peer tutoring is necessary to
help explain other ideas. Students try to get other team members to voice their opinions and
integrate ideas of different members when they are allowed to discuss among themselves the
results of the experiment. The respect for the opinions of others can be observed when students
respond calmly to their team mates and question others opinions or ideas constructively. If the
teacher assists the students in getting and returning materials from the stockroom, she
demonstrates the value of cooperation which is very important in developing team work skills.
To vigorously pursue the virtue of humility is a purpose of LPU chemistry faculty where
humility is considered as the very foundation of leadership of a teacher so that a teacher can
influence the behavior or attitude of students. In influencing the behavior of students, the LPU
chemistry faculty employs teaching practices that lead to the interest or positive attitudes of
students in science. These teaching practices of LPU chemistry faculty are assigning only one
group instead of the whole class to get the reagents from the stockroom; and giving a
borrowers slip to each group for them to list the needed equipment that will be borrowed from
the stockroom. Only one group from the class was assigned to get the reagents and this group
is responsible for distributing the reagents to the other groups. In this way, the students are
taught the positive attitude of responsibility and willingness to share the resources to others.
Proper enlisting of equipments in the borrowers slip is one way of training the students to be
organized and systematic in what they are doing.
Finally, LPU chemistry faculty also gear to make students enjoy the challenges that are in their
hands because they will make them a better and complete person. In so doing, the LPU
chemistry faculty implement teaching practices that lead to the development of practical skills
of students and their understanding of the complexity and ambiguity of empirical work. The
159
teaching practices that lead to the development of practical skills include providing posters
and signages on safety precautions in the laboratory room such as No Lab gown, no entry,
Place your bags in the shelves or tables provided for at the back of the room, and Check the
gas supply before and after using; placing all the needed reagents in the demonstration table
for the teacher to see the amount of reagents the students will get; and giving practical test on
the proper use of equipment such as Bunsen burner, pipette and platform balance. Posting
signages on safety precautions will help students develop safe and conscientious laboratory
habits which is one of the practical skills a student must acquire in science education classes.
Another important practical skill is for the student to use the exact amount of reagents as
indicated in the procedure to avoid wastage, contamination and inaccuracy of result. Practical
test is given to check if students acquired the skill in using simple equipment such as proper
lighting of Bunsen burner, measuring accurate volumes of liquids with a pipette and getting
exact weight of reagents in a platform balance. These are fundamental skills which are
necessary for the success of an experiment.
Teaching practices of LPU chemistry faculty that lead to the students understanding of the
complexity and ambiguity of empirical work include making students trace the source of their
error in the result of the experiment and requiring students to repeat the procedure if they got
a result which is far beyond the actual value. By allowing students to trace the source of error,
they will recheck data observations which will enable them to expect and understand
experimental error in every scientific investigation. Requiring students to repeat the procedure
makes them correct their own mistakes. In this way, a teacher can emphasize to students that
experimental errors are not hindrances to learning but they are opportunities for greater
learning.
Figure 4 is a proposed model of constructivist teaching-learning approach based on the
identified best teaching practices in chemistry laboratory. It consists of three rectangular boxes
connected by arrows. The first box represents the constructivist teaching approach containing
the four elements of constructivism such as interweaving, scaffolding, modeling and coaching.
The second box shows the four elements of constructivism which are included in the
constructivist learning environment such as collaboration, articulation, reflection and
exploration. The seven goals of science laboratory instruction which include mastery of subject
matter, scientific reasoning, understanding the complexity and ambiguity of empirical work,
practical skills, understanding the nature of science, interest in science and interest in learning
science and teamwork skills are contained in the third box.

160
An arrow linking the first box to the second box shows that a constructivist teaching approach
creates a constructivist learning environment. Another arrow linking the second box to the
third box shows that a constructivist learning environment manifests the seven goals of science
laboratory instruction. It means that if a constructivist teacher implements teaching practices
based on the seven goals of science laboratory instruction then the students will manifest the
attainment of the seven goals of science laboratory through constructivist learning.
Based on the findings of the study, the best teaching practices of the chemistry faculty of LPU
which are based on the universitys VMGO are parallel to the ideal practices which on the
other hand are based on the seven goals of science laboratory instruction. This means that
constructivist teaching could be attained if the ideal practices and the best teaching practices
from LPU are both implemented. Both practices lead to the enhancement of mastery of subject
matter, developing of scientific reasoning, interest in science and interest in learning science,
understanding the nature of science, developing teamwork skills, practical skills and
understanding the complexity and ambiguity of empirical work.
Interweaving is connecting of new ideas to prior knowledge in order to make learning

meaningful. The best practices of the LPU faculty such as giving of pre-lab quiz and
submission of flow diagram by their students together with the ideal practices based on the
seven goals such as making the topic in the lecture simultaneous with the laboratory and
161
conducting of pre-lab discussion, all of which lead to the enhancement of the mastery of
subject matter. The practice of giving a pre-lab quiz will enable a teacher to diagnose what is
already known by the student so that they can relate new knowledge (concepts and
propositions). Submission of flow diagram of the procedure make students organizes
information into a meaningful whole. When the topic in the lecture is simultaneous in the
laboratory, the knowledge acquired is applied in the experiment so that concept and process
are taught simultaneously. Conducting a pre-lab discussion will enable the teacher to find out
what the pupils know about the topic before doing the experiment. Such practices lead to the
enhancement of mastery of subject matter.
Scaffolding is accomplished by giving assistance to students in achieving tasks that they cannot
yet master on their own and then gradually withdraws the teachers support. Best practices of
LPU faculty such as assisting students in getting and returning materials and asking students
of the purpose of doing such a procedure are examples by which scaffolding is done. Assisting
students in getting and returning materials makes students develop teamwork skills while
asking students of the purpose of doing such a procedure develop their scientific reasoning.
Other best teaching practices of LPU faculty such as posting of signages about safety
precautions, and placing needed reagents in the demo table are also implemented during
scaffolding which lead to the development of practical skills of students. On the other hand,
ideal practices based on the seven goals of science laboratory instruction were also
implemented by LPU faculty, and these are checking and troubleshooting of equipment before
the experiment, not skipping experiments simply because the materials are not available and
supervising and guiding students in performing experiment.
During modeling, the teacher performs a complex task to show the students the processes
needed in carrying out the experiment. The best practice of LPU faculty of outlining the
procedure and drawing of set-up of the experiment on the board make students learn large
amounts of meaningful material from textual representations thus enhancing mastery of
subject matter. Developing practical skills of students such as checking whether students know
how to operate lab equipment and understand exactly how equipment works before physically
approaching it, may also be implemented during modeling and these conform to the ideal
practices based on the seven goals of science lab instruction.
Coaching is a process of motivating learners, analyzing their performance, and providing

feedback on their performance. The best practices of LPU faculty which are included under
this process are posting of top performers, using technical terms related to the course of the
students during discussion, accepting all interpretations made by each group about the result
of the experiment, giving of practical test and requiring students repeat procedures if they got
wrong result. Posting of top performers and using technical terms related to the course of the
students during discussion will develop students interest in science and interest in learning
science. Ideal practices based on the seven goals of science laboratory instruction include
emphasizing the relevance of the lesson to the students future job, making the objective of the
experiment clear to the students before doing the experiment, listing down keywords on the
board, advising students to overcome their errors, emphasizing to students that each persons
preconception may or may not affect conclusion and giving of post-lab quiz. A post lab quiz

162
and practical test are given to analyze and provide feedback for the performance of the
students.
In collaboration, students learn from each other such as when their seats are arranged by group
and when they are allowed to do peer tutoring. When students are grouped into smaller
groups, a leader is assigned for every group, everyone is given specific active roles and not
allowing students to regroup are examples of collaboration. In so doing, students develop
teamwork skills. Assigning only one group of students to get the reagents and share with the
rest of the class, and listing equipments needed in the borrowers slip may be implemented
during collaboration. These practices lead to the development of practical skills. Among the
best practices of LPU faculty are arranging the students by group, allowing them to do peer
tutoring, assigning only one group of students to get the reagents, and making them list the
equipments needed in the borrowers slip. On the other hand the ideal practices based on the
seven goals of science lab instruction include grouping students into smaller groups, assigning
a leader for every group, allowing students to do rotational and specific active roles, and not
allowing students to regroup.
During articulation, students are encouraged to articulate their ideas, thoughts and solutions.
They are allowed to think about the method they use in doing the procedure and whether this
method arrived to the correct result. Best practices of LPU faculty such as requiring students
to submit logbook, allowing them trace error and repeat the procedure if they got wrong result
may be implemented during articulation. Submission of logbook enhances mastery of subject
matter, while tracing error and repeating procedure if students got wrong result makes them
understand the complexity and ambiguity of empirical work.
During reflection students compare their results with other students. Practices such as making
students do reflection by sharing and making them discuss results may be implemented
during reflection. Making reflection by sharing develops scientific reasoning. Developing
students understanding of the nature of science as in accepting all their interpretations of
experimental result may also be implemented during reflection. During post-lab discussion
students can compare results, give scientific explanations for their result, analyze and discuss
the data and observations. Among the best practices of LPU faculty is making students do
reflection, accepting all interpretations made by students about experimental result, and
discussing among themselves experimental result. On the other hand the ideal practices based
on the seven goals of science lab instruction include conducting post-lab discussion, allowing
students compare result, making students give scientific explanations for their result and
making students analyze and discuss the data and observation.
In exploration, students develop novel combinations of ideas and thinking about hypothetical
outcomes of imagined situations and events. In the logbook that students are required to
submit, they must include the possible application of the experiment to other situations and
they can propose investigatory studies related to the experiment. That is one of the best
practices of LPU faculty which lead to the enhancement of mastery of subject matter. Ideal
practices such as going on field trips, and connecting the lesson to real world experiences are
also exploration.

163
Conclusions
Based on the findings of the study, the following conclusions are drawn:
1. The teaching practices employed by the faculty in teaching chemistry laboratory that attain
the seven goals of science laboratory instruction are those practices where students engaged in:
a. experiential learning where experience is translated through reflection into concepts,

which are used as guides for active experimentation
b. active learning or learning by doing where learners use their learning in realistic and
useful ways, seeing its importance and relevance
c. meaningful learning where learners organize information through integrating new and
previous knowledge and
d. cooperative learning where students work as self-directed in small collaborative groups
2. Students enjoy goal-oriented activities and practical work, have willingness and great
desire to learn chemistry and can understand situations in everyday life no matter how difficult
the theories and scientific laws in chemistry are.
Students can recognize the differences among knowing about a topic, knowing how to
complete a skill, showing how to complete a skill and doing the skill. They demonstrate
true collaborative work and interaction through sharing of ideas. They know how to deal with
experimental errors and can find solutions to problems encountered while doing experiments
Students can readily understand and apply the concepts they have learned. They are aware of
valid reasoning principles and can practice those reasoning skills.
Recommendations
In the light of the findings of the study, the following recommendations are endorsed:
1. The model of teaching practices be used by chemistry faculty in designing their laboratory
instruction to develop students positive attitude towards chemistry, laboratory skills and high
achievement in the subject.
2. The findings of this study could be an avenue for chemistry faculty in maximizing the active
participation of students in the laboratory by incorporating the significant findings of the study
in training and seminars of chemistry faculty.
3. An in-depth study can be conducted in other science subjects that will determine the best
practices in its laboratory instruction.
4. The significant findings of this study may be integrated as a guide in developing

instructional materials in chemistry.

164
5. Curriculum planners may include the salient findings of this study as a concrete basis in
determining the objectives and methods in the design of chemistry laboratory instruction.
6. Policy makers may utilize the findings of this study as a guideline in considering the
educational purposes that science education can best provide to students.
References
Acero, V., Javier, E. & Castro, H. (2000). Principles and strategies of teaching. Manila, Philippines:
Rex Bookstore.
Alasuutari, P. (2004). The globalization of qualitative research. Qualitative research practice. London:
Sage.
Aquino, L. (2003, March). Literature-based Science Instructions for Preschools. Educators
Journal. Volume 22, No.10, p.7.
Avilla, R. (2009). Students Images and Views of Chemistry; Its Implication to Student Achievement.
(Masteral Thesis). Phil. Normal University. Manila, Phil.
Bishop,C., Bishop M. & Whitten, K. (2000). Standard and microscale experiments in General
Chemistry. U.S.A: Saunders College Publishing.
Brodie, M. (2006, July 26). Promoting science and motivating students in the 21st century.
Scientix. Retrieved March 29, 2012 from http://scienceinschool.org/2006/issue2/rir
Chang, R. (2009). Chemistry. New Jersey: Mc Graw Hill Inc.
Claveria, G. (2002). The Utility Value of a Well-Managed Laboratory Activities in
Students Learning of Science Concepts. (Masteral Thesis). Batangas State University,
Batangas City, Phil.
Cobb, P., Stephan,M., Clain K. & Gravemeijer, K. (2001). Participating in Classroom
Mathematical Practices. Retrieved April 27, 2010 from
www.nap.edu/catalog.php?record_id=11311#description.
Corpuz, B., Rimas, A., Galangco, N.R. & Bautista, A. (2003). Student teachers manual for
observation and participation. Quezon City, Philippines: Katha Publishing Co., Inc.
Crowther, D., Lederman, N. & Lederman, J. (2005). Understanding the True Meaning of the
Nature of Science. Retrieved March 12, 2012 from
caee.naaee.org/...science.../katzenberger-methods-and-strategies_ ...
Domin, D. (2009). Considering Laboratory Instruction through Kuhns View of the Nature of
Science. Retrieved April 27, 2010, from
www.nap.edu/catalog.php?record_id=11311#description.
Eastwell, P. & Rennie,L. (2002). Using Enrichment and Extracurricular Activities to Influence
Secondary Students Interest and Participation in Science. Retrieved November 29, 2009
from www.scienceeducationreview.com/open_access.
Glynn, S. & Koballa, T. (2006). Science Motivation Questionnaire. Retrieved July 25, 2011 from
http:// www. Coe.uga.edu/smq/
Hall, T. (2006). Encouraging Active Participation of Students during Science Laboratories.
Retrieved January 8, 2010 from http://www.montana.edu/msse/capstone.2006.htm
Hidalgo, F. (2000). Tips on how to teach effectively. Manila, Philippines: Rex Book Store.
Isnian, S. (2009). Definition of Achievement. Retrieved May 30, 2012 from
http://hmsofyanisnianspd.blogspot.com/2009/08/definition-of-achievement.html

165
Jona, K., Adsit, J. & Powell, A. (2008). Goals, Guidelines, and Standards for Student Scientific
Investigations. Retrieved April 29, 2010 from
www.inacol.org/research/docs/NACOL_ScienceStandards_hub-pdf.
Kanli, U. & Yagbasan, R. (2008). The Effects of a Laboratory Approaches on the Development of
University StudentsScience Process Skills and Conceptual Achievement. Essays in
Education Special Edition. Retrieved April 15, 2012 from
http://www.usca.edu/essays/specialedition/UKanl%c3%acandRYagbasan.pdf
Leonor, J. (2007). The Effects of Scientific Inquiry Method on the Academic Performance in Chemistry
of High School Students in the University of St. La Salle Integrated School. (Masteral Thesis).
University of San Agustin, Iloilo City.
Marine, S. (2003, April). Building Skills with Reiterative Laboratory Projects. Journal of Chemical
Education. Vol. 80, No. 4. P. 366.
Mentzer R. (2006). The Effect of Self-Reflective Journaling on Student Motivation. Retrieved
January 8, 2010 from http://www.montana.edu/msse/capstone.2006.htm
Minstrell, J. & Kraus, P. (2005). Best Practices of Science Teaching. Retrieved July 3, 2011 from
http://www.phy.ilstu.edu/pte/311content/effective/best_practice.html
Moni, R.W., Hryeiw, D.H., Poronnik, P., Lluka, L.J. & Moni, K.B. (2007, September). Assessing
Core Manipulative Skills in a Large, First-year Laboratory. Advances in Physiology
Education . vol. 31 no. 3 pp.266-269 .Movahedzadeh, F. (2011, Summer). Improving
Students Attitude Toward
Science Through Blended Learning. Science Education and Civic Engagement Journal.
Retrieved March 29, 2012 from
http://seceij.net/seceij/summer11/movahedzadeh_im.html
Morehead, L. (2006). Integrating a Science Club into an Alternative High School Setting in
order to Enhance Students' Learning. Retrieved January 8, 2010 from
http://www.montana.edu/msse/capstone.2006.html
Muchmore, L.A. (2006). Does Cooperative Learning and Peer Assisted Learning Increase
Middle School Student Achievement and Involvement? Retrieved January 8, 2010 from
http://www.montana.edu/msse/capstone.2006.htm
Muijs, D. & Reynolds, D. (2011). Effective Teaching Evidence and Practice. 3rd edition. London.
Sage Publications.
Narayan, R. (2005). An Interpretative Study of the Discourse Practices. Retrieved January 9,
2010 from http://www.coe.uga.edu/prism/Ratna_Narayan_Dissertation_2005.pdf.
National Research Council (2005). Americas Lab Report: Investigation in High school Science.
Retrieved April 27, 2010 from www.nap.edu/catalog.php?record_id=11311#description.
Nordick, T. (2006). Why do Students Fail to Turn in Work? Retrieved January 8, 2010 from
http://www.montana.edu/msse/capstone.2006.htm
National Science Teachers Association (2007). Laboratory Science Instruction Statement - NSTA
Position Statement. Urban Science Education Leadership Academy. Retrieved April 29, 2010
from www.nsta.org/about/position/laboratory.aspx
Ogle, R. (2005). Increasing Student Motivation. Retrieved April 5, 2010 from
http;//www.uncw.edu/cte/events/spring 05/html
Palada, G. (2002). Perception about Chemistry and Teachers Teaching Practices; their Influence on
Students Performance. (Masteral Thesis). University of San Agustin, Iloilo City, Phil.

166
Petrucci, R., Herring, G., Madura, J. & Bissonnette, C. (2002). General Chemistry. Principles
and Modern Application (8th ed). New Jersey: Prentice Hall Inc.
Reyes, C. (2002). Effects of Task-Oriented Learning Approach on Selected Affective Variables and
Chemistry Achievement among Self-Handicapping Students. (Dissertation). University of the
Phil.Diliman, Quezon City, Phil.
Reyes, P. (2005). Laboratory Competencies in General Chemistry I of College Students of the Lyceum of
Batangas. (Masteral Thesis). Batangas State University. Batangas City, Phil.
Ricardo, L. (2008). Chemistry Teaching in the Public Secondary Schools: Basis for Policy Formulation.
(Dissertation). Mariano Marcos State University. Laoag City, Phil.
Roble-Estrella, D.R. (2009). Predictors of Students Performance in Chemistry Laboratory of Selected
Private Schools. (Masteral Thesis). Bataan Peninsula State University. Balanga City,
Bataan, Phil.
Rowan, B., Schilling, S., Ball, D. & Miller, R. (2011). Measuring teachers pedagogical content
knowledge in surveys: An exploratory study. Consortium for Policy Research in
Education. Retrieved March 20, 2012 from
http://www.sii.soe.umich.edu/documents/pck%20final%20report%20revised%20BR10
0901.pdf
Salandanan, G. (2002). Teaching children science. Quezon City, Philippines:
Katha Publishing Co. Inc.
Singer, S., Hilton, M. & Schweingruber, H. (2005). Americas Lab Report: Investigation in
Highschool Science by Committee on Highschool Lab: Role and Vision. Retrieved April
27, 2010, www.nap.edu/catalog.php?record_id=11311#description.
Straatman, C. (2006). Implementing Inquiry into Chemistry Classroom. Retrieved January 8,
2010 from http://www.montana.edu/msse/capstone.2006.htm
Teachnology (2007). Current Trends in Education: Constructivism. Retrieved April 29, 2010
from www.inacol.org/research/docs/NACOL_ScienceStandards_hub-pdf
Torres, M. (2009). Integrating Graphic Organizers in Facilitating Learning Chemistry.
(Masteral Thesis). Phil. Normal University, Manila, Phil.
Wachanga S. & Mwangi, J.G. (2004). Effects of the Cooperative Class Experiment Teaching
Method on Secondary School StudentsChemistry Achievement in Kenyas Nakuru
District. International Education Journal Vol 5, No 1, 2004. Retrieved March 28, 2012 from
http://iej.cjb.net
Washtak, A. (2006). Evaluating Chemistry Students Motivation and Learning with Technology.
Retrieved January 8, 2010 from http://www.montana.edu/msse/capstone.2006.htm
Wilson, J.D., Cordry, S. & Uline, C. (2004). Science Fairs: Promoting Positive Attitudes towards
Science from Student Participation. College Student Journal, Vol. 38, 2004. Retrieved
March 31, 2012 from http://www.questia.com/googleScholar.qst?docId=5006016959
Wink, D., Gislason, S. & Kuehn, J. (2000). Working with Chemistry (A Lab Inquiry Program). New
York: W.H. Freeman and Company.
.Zulueta, F. & Guimbatan, K. (2002). Teaching strategies and educational alternatives.
Manila,Philippines: Academic Publishing Corporation.
Zemelman, S., Daniels, H. & Hyde, A. (2005). Best practice. Todays standards for teaching and
learning in Americas schools. Third Edition: Portsmouth, New Hampshire HEINEMANN

Vol 4 No 1 - April 2014

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Vol 4 No 1 - April 2014

Uploaded by

Copyright:

Available Formats

IJLTER.

Influential Factors in Modelling SPARK Science Learning System ............................................................................... 36

Analysis of Achievement Tests in Secondary Chemistry and Biology ......................................................................... 75

International Journal of Learning, Teaching and Educational Research

Angovian Methods for Standard Setting in

Abstract. This paper presents a discussion of Angovian methods of

A protocol, based on a stricter interpretation of what is meant by a

Keywords: standard setting; Angoff method; medical education; norm-

1 Formerly Gippsland Medical School (2007-2013).

2014 The author and IJLTER.ORG. All rights reserved.

A cursory glance at the literature on assessment in medical education will reveal

The purpose of this paper is to present a critical discussion of the rationale,

interacted and confounded the most well-intentioned quests for defining

The detailed analysis and discussion will be usefully facilitated by

2. Angoffs Text Method (ATM)

A systematic procedure for deciding on the minimum raw scores for

With a number of judges independently making these judgments it would

2014 The author and IJLTER.ORG. All rights reserved.

However, Angoffs (1971) prescription is far from being a robust example of

2.1 Angoffs First Paragraph

Keeping the hypothetical minimally acceptable person in mind, one could

A. Firstly, it may be given a more criterion-referenced construction by

performance. It seems that this construction has been attempted rarely,

B. Alternatively, it may be given a less criterion-referenced construction by

We shall return to the more criterion-referenced option later in the discussion

3. Angoffs Footnote Method (AFM)

A slight variation of this procedure is to ask each judge to state the

2014 The author and IJLTER.ORG. All rights reserved.

3.1 Softening the language of ATM and AFM

2014 The author and IJLTER.ORG. All rights reserved.

distinction between minimally acceptable (more attuned to criterion

As discussed earlier, it is possible to put a criterion-referenced construction on

4. Angoffs Second Paragraph

A. Consensus among totally independent experts

However, real-world variability between judges might be expected to yield some

B. Consensus among inter-dependent partial experts

2014 The author and IJLTER.ORG. All rights reserved.

4.2 What is purported to happen

Judges are asked to first define the characteristics of a borderline group of

Judges are asked to first define the characteristics of a borderline group of

Nonetheless, regardless of these issues, the focus is clearly on a subset of the

2014 The author and IJLTER.ORG. All rights reserved.

and reassurance of group consensus among as large a collection of judges as can

This sentence directly and unnecessarily confounds the concepts of difficulty

Each judge estimates what percentage of the hypothetical borderline

This is clearly a norm-referenced judgment, with no necessary link to any sense of

However, except where special funding for educational research projects is

2014 The author and IJLTER.ORG. All rights reserved.

Returning to our dissection of Norcinis (2003, p.465) description, we find:

This judgement is often informed by data on the performance of the

In our experience of standard setting, the practice of setting borderlines by

A. During a standard-setting session, this practice is usually frowned upon as

4 The LOW subset of a cohort of examinees is the bottom 27% (approximately) as

4.2.1 What actually happens

A standard-setting session is usually held several days after a draft of the

Academics are encouraged to review the examination paper thoroughly,

Angoffs method is relatively easy to use, there is a sizeable body of

While the standard-setting process (regardless of the method used) is certainly

5. Angoffs Footnote Method in Action in a Small Medical School

5.1 Relation of BL estimates to different subsets of examinees

Despite the fulfilment of these expectations, it is clear that the BL estimates

This impression of gross inaccuracy is maintained as we refine the focus to the

Figure 1: Data plots of performance rates, P, vs BL estimates for students answering

5.2 Accuracy of BL estimates for the Borderline subset of examinees