A CASE STUDY DESCRIBING STUDENT EXPERIENCE OF

LEARNING SCIENCE IN A WEB BASED EDUCATION

ENVIRONMENT
Semiral nc, Ph.D.
Uluda University, Faculty of Education
Department of Computer Education and Instructional Technologies, Bursa / Turkey
semiral@uludag.edu.tr

Erhan engel, Ph.D.

Uluda University, Faculty of Education
Department of Computer Education and Instructional Technologies, Bursa / Turkey
erhansengel@uludag.edu.tr

mer Delialiolu, Ph.D.

Middle East Technical University, Faculty of Education
Department of Computer Education and Instructional Technologies, Ankara / Turkey
omerd@metu.edu.tr

ABSTRACT
Many freshmen show no interest in computer literacy classes. They do not participate or actively seek information. It is hypothesized in this
study that this tendency results from prior computer knowledge. Students who already mastered the content do not find it useful, so they get
bored, leading to the hypotheses that (1) students who are computer literate will have low student engagement and have high achievement.
Moreover (2) students who are not computer literate will have high engagement, and have exam scores that vary randomly depending on how
hard they study. MANOVA results from 267 freshmen show that different departments have significantly different engagement and achievement
scores. However, the prior knowledge can predict achievement but not engagement. Moreover, students with little or no prior content
knowledge score significantly less than their counterparts. Overall students are not actively involved in schoolwork which is a call for action to
improve engagement.
Keywords: Prior knowledge, student engagement, computer literacy, achievement

INTRODUCTION

significant predictor of subsequent exam performance, even

with other influences on student achievement, for example
retention, controlled. When all this research is taken
together, prior knowledge appears to have a facilitating,
rather than hindering, influence on new learning.

When undergraduates take an introductory class, what

determines whether they succeed or fail? Or what
determines the level of success? Although courses are
carefully designed to promote student learning through
lectures and discussions in class, assignments, projects,
and performance evaluations to motivate student effort,
there are some other influences on student achievement
and engagement. This study focuses on the effect of prior
knowledge on student engagement and achievement in
undergraduate level computer literacy classes.

Student Engagement
For decades, educational researchers have been interested
in the effects of students attitudes about school and their
experiences in school on achievement and engagement
(National Center for School Engagement, 2006)
Student engagement has been used to depict students
willingness to participate in routine school activities, such as
attending classes, submitting required work, and following
teachers directions in class (Chapman, 2003). For example,
Natriello (1984) defined student engagement as
participating in the activities offered as part of the school
program (p.14). Negative indicators of engagement in this
study included unexcused absences from classes, cheating
on tests, and damaging school property. According to
Skinner & Belmont (1993), engagement versus disaffection
in school refers to the intensity and emotional quality of
childrens involvement in initiating and carrying out learning
activities (p. 572).

The potential influence of prior knowledge is especially

important when students enroll in courses like computer
literacy because of widespread interest in computer
concepts and their applications to everyday life. Students
are thus likely to begin an introductory course in computer
literacy with considerable prior knowledge derived from
many sources, including earlier coursework in high school,
information from the media, and daily applications.
Researchers from around the world have shown how prior
knowledge and expectations contribute to new learning and
influence how people respond to new situations (Spires,
Donley, & Penrose, 1990). Prior knowledge can assist or
hinder new learning; individuals with greater preexisting
knowledge of a topic understand and remember more than
those with more limited prior knowledge (Thompson &
Zamboanga, 2004).

Fredricks, Blumenfeld, & Paris (2004) classify engagement

into three types: behavioral, cognitive and emotional.
Behavioral engagement is commonly defined as following
rules, and accepting and applying classroom norms as well
as being involved in academic tasks. Emotional
engagement refers to students' reactions to events around

In a recent study, Thompson and Zamboanga (2003) found

that prior content knowledge was a uniquely positive and

196

the classroom, including interest, boredom and emotions.

Cognitive Engagement is going beyond the requirements of
the classroom and doing more than what is expected from
the students. Cognitively engaged students are intrinsically
motivated and willing to do more. The main focus of this
study fits into behavioral engagement, which includes
behaviors such as asking questions, persistence, effort, and
taking active role in class discussions.

Question 1: Is there a tendency for students who are

computer literate to have low student engagement scores
and have high exam scores?

As educators should we care about engagement? Yes, we

should, because the literature indicates a relationship
between student engagement and both their persistence in
academic programs and their achievement or learning
outcomes as measured by test scores (Fredricks,
Blumenfeld, & Paris, 2004; National Research Council &
Institute of Medicine, 2004; Pascarella & Terenzini, 2005).
The same literature also shows that engaging students in
learning is something that instructors can facilitate through
the pedagogical approach they use. A generalization that
could be drawn from these research is that, if the instructor
successfully engages students to actively participate in the
learning process (e.g., through asking questions in class,
participating in projects, participating in collaborative groups,
and discussing the subject matter with other students, the
instructor, and others outside of class), then learning and
persistence can be expected to increase.

METHOD

Question 2: Is there a tendency for students who are not

computer literate to have high student engagement scores,
and have exam scores that vary randomly depending on
how hard they study?

Design
The basic university level computer courses teach general
information about computers, Microsoft (MS) Windows XP
operating system, and common editing programs. In the
context of this study, the editing programs included MS
Word software. A quasi-experimental research design was
used to investigate the research questions, which is detailed
next.
Sample
This study involved students from 11 different sections
(classes). They were from 4 different departments in the
school of education. A total of 356 students were invited to
participate in the study. In the end, there were a total of 267
students responding to the both (pre and post explained
next) surveys from the four departments. Figure 1
summarizes the distribution of the students according to
their departments. The majority of the students (N=155)
came from the Language Education department mastering
the English Language. Counseling and Educational
Psychology department had the least number of students
(N=28).

One of the studies investigating relationship between

student success and engagement was carried out by, Carini,
Kuh & Klein (2006). They examined (1) the extent to which
student engagement is associated with experimental and
traditional measures of academic performance, (2) whether
the relationships between engagement and academic
performance are conditional. Their sample consisted of
1058 students at 14 four-year colleges and universities that
completed several instruments. The findings indicate that
many measures of student engagement were linked
positively with such desirable learning outcomes as critical
thinking and grades, although most of the relationships were
weak in strength. The results of the study suggest that the
lowest-ability students benefit more from engagement than
their classmates. The study also reveals that first-year
students and seniors convert different forms of engagement
into academic achievement.
Freshman students of all departments are required to take
basic computers classes as covered in the national
curriculum till 2007. The new curriculum after 2007 requires
sophomores, instead, to take these classes. Todays
technology driven school environments as well as the daily
routines already make students technology literate. Many
students must take these classes even though they already
know how to use computers well. This fact results in
undesired situations, one of which is the result of lack of
student engagement, as hypothesized in this study. Many
students show no interest in the lessons; they do not
participate; and they do not try to respond to questions; they
do not actively seek information.

Figure 1. Distribution of the students to departments of

school of education.
Data Collection and Analysis
Data were collected during fall 2007 semester. Students
were given an entry survey. This survey consisted of
questions determining their level of knowledge about
computers. This study incorporated one item from that
survey, which grouped students according to their level of
computer knowledge based on a four-point Likert scale (1 =
not knowledgeable to 4 = very knowledgeable). Later,
these four points were collapsed into two groups identified
as low (little or no perceived prior knowledge) and high
(medium or higher perceived prior knowledge). These two
groups formed one of the independent variables.

Purpose of the Study

It is the authors expectation that this pattern of action could
have especially one critical factor in common, which is the
prior computer knowledge.
The possibility that the students who already mastered most
of the intended curriculum in introductory technology
classes do not find the newly presented information useful
and get bored needs to be empirically tested. It is the aim of
this study to search for a possible relation between prior
knowledge and engagement in introductory technology
classes. The research questions guiding the current study
are:

All students were given a midterm exam and at the end of

the semester, students were also given a final exam
consisting of 25 multiple choice items. All students took the
same exams. The purpose of the exams was to evaluate

197

how much the students mastered the courses and therefore

determine their level of achievement.

Table 1. Results of the study

The student engagement items were attached to the final

exam sheet and were adapted from the National Survey of
Student Engagement (NSSE) 2006. They consisted of 13
items with questions such as asked questions in class or
contributed to class discussions and discussed grades or
assignments with an instructor measuring behavioral
engagement with the schoolwork. The questions had a fourpoint Likert scale where 1 = never to 4 = very often. Out
of these 13 questions, 12 were grouped together to form an
engagement construct which had a .801 Coefficient alpha
reliability (Cronbach, 1951). This construct was used in the
MANOVA. The remaining item was reported individually.

Religious
Studies

Language
Ed. A

Language
Ed. B

The MANOVA technique was used to investigate the

hypotheses. A mean score of student engagement items
and the average exam scores formed the dependent
variables. The prior computer knowledge and the section
where the student took the course were the independent
variables.

Language
Ed. C

Language
Ed. D

RESULTS
The descriptive results of the study were summarized in
Table 1. There were a total of six Language Education
classes, and two Elementary Education classes. There was
one class of students per department for the rest. Pseudo
names were used to describe the classes to preserve
privacy. The number of students (N) was relatively balanced
across the classrooms ranging from 19 to 31.

Language
Ed. E

Language
Ed. F

The MANOVA results indicate that there is a significant

difference between students of different departments (Wilks
Lambda=.788, F=3.095, p <.01) as wells as between
students of different prior knowledge (Wilks Lambda=.945,
F=7.153, p <.01) in terms of student engagement and/or
student achievement; whereas there were no interaction
effect of department and prior knowledge (Wilks
Lambda=.935, F=0.837, p >.05) on those two dependent
variables.

Language
Ed. G

Engagement
Achievement
M/(SD)
M/(SD)
N
L/(T) L
H
T
L
H
T
10 1.62 1.77 1.72 69.70 73.42 72.09
(28) (.38 (.37 (.37 (8.04) (6.31) (7.06)
)
)
)
9

1.93 1.88 1.90 71.72

74.15

73.00

(19) (.48 (.42 (.44 (10.46 (10.50 (10.26

)
)
)
)
)
)
15

2.14 2.01 2.09 64.30

66.75

65.15

(23) (.48 (.34 (.43 (10.20 (10.70 (10.20

)
)
)
)
)
)
18

1.80 1.89 1.83 68.61

70.56

69.21

(26) (.31 (.66 (.43 (7.48) (12.65 (9.15)

)
)
)
)
10

1.57 1.55 1.56 65.05

73.15

69.10

(20) (.37 (.44 (.39 (6.98) (8.37) (8.57)

)
)
)
12

1.73 1.82 1.77 70.63

82.31

75.30

(20) (.34 (.45 (.38 (8.57) (9.15) (10.39

)
)
)
)
18

1.69 2.07 1.80 66.64

72.43

68.26

(25) (.33 (.82 (.53 (7.44) (5.79) (7.39)

)
)
)
9

1.81 1.54 1.65 65.72

69.69

68.07

(22) (.35 (.28 (.33 (8.73) (7.53) (8.09)

)
)
)

Coun. & Ed. 17 1.75 1.81 1.78 70.24 73.04 71.50

Psyc.
(31) (.32 (.35 (.33 (6.81) (7.71) (7.24)
)
)
)

As seen in Table 1, the classroom whose engagement

scores were the lowest (Language Ed. D) had the average
engagement score of 1.56 where 1 = not engaged, 4 =
highly engaged. The classroom with the highest average
student engagement score (Language Ed. B) had the value
of 2.09. It is important to note that these two extremes were
reported from the same department (Language Ed.).
Overall, the average of all students shows that the level of
student engagement (1.80) is extremely low in the computer
literacy classes, where 2.50 is the midpoint.

Elem. Ed. A

17

1.68 1.96 1.80 60.50

65.23

62.55

(30) (.31 (.65 (.49 (17.04 (7.38) (13.72

)
)
)
)
)
Elem. Ed. B

11

1.81 1.93 1.87 68.09

68.13

68.11

(23) (.28 (.50 (.41 (10.04 (4.75) (7.56)

)
)
)
)
Total

The classroom with the lowest average achievement score

was Elementary Education A (62.55 out of 100). Overall, the
Elementary Education classes had relatively low
achievement scores, but average engagement levels. The
highest average achievement belonged to the Language
Education E classroom (75.30 out of 100). The overall
average (69.12) is typical of this sort of computer classes.

146 1.78 1.82 1.80 67.14

71.51

69.12

(267) (.38 (.48 (.43 (10.04 (8.84) (9.74)

)
)
)
)
L = Low little or no perceived prior computer knowledge.
H = High medium level or higher perceived prior computer
knowledge.
T = Total (overall)
M = Mean
SD = Standard Deviation

Tests of between-subjects effects revealed that, students

from different departments differed on both how the
students rated their level of engagement (F=2.241, df=10,
p<.01) and the average exam scores they received at the
end of the course (F=3.771, df=10, p<.01). How the scores
from different departments significantly vary is shown in
Table 2 and Table 3 based on Tukey B post-hoc tests.

In Table 2 and Table 3, mean engagement and achievement

scores are organized into subsets where p<.05. The means
(and therefore the classrooms) that are in the same subset
do not significantly differ from each other. Only the
classrooms that are in different subsets differ significantly.
According to this, for example, Elementary Education A
classroom had a mean achievement score of 62.55, which
is significantly different from the Language Education B
classroom with the mean score of 65.15.

198

While the students with different perceived prior knowledge

did not differ on how they rate their engagement scores
(F=1.472, df=1, p>.05), they differed in terms of their
average exam scores at the end of the course (F=13.805,
df=1, p<.01).

199

Table 2. Homogenous distribution of the departments based

on their effect on the student engagement.
Department
Language Ed. D

N
20

Subset
1
2
1.56

Language Ed. G

22

1.65

Religious Studies

28

1.72

1.72

Language Ed. E

20

1.77

1.77

Coun. & Ed. Psyc.

31

1.78

1.78

Elem. Ed. A

30

1.80

1.80

Language Ed. F

25

1.80

1.80

Language Ed. C

26

1.83

1.83

Elem. Ed. B

23

1.87

1.87

Language Ed. A

19

1.90

1.90

Language Ed. B

23

the latter. These results were significant at the p<.01 level

based on an ANOVA (F=21.563, df=1).

Figure 2. Response rates to the question whether students

had to study more than they expected to meet their
teachers standards.

2.09

Table 3. Homogenous distribution of the departments based

on their effect on the student achievement.
Subset
2

DISCUSSION AND CONCLUSION

The researchers expected to find that the students with little
or no prior knowledge would have high engagement in
schoolwork towards the computer literacy classes in
contrast to students with medium or higher level prior
knowledge. Then, the latter students would be more
successful whereas the success rates for the former could
vary. These hypotheses were depicted in Table 4 along with
what was actually found in this study.

Department
Elem. Ed. A

N
30

1
62.55

Language Ed. B

23

65.15

65.15

Language Ed. G

22

68.07

68.07

Elem. Ed. B

23

68.11

68.11

68.11

Language Ed. F

25

68.26

68.26

68.26

Language Ed. D

20

69.10

69.10

69.10

Language Ed. C

26

69.21

69.21

69.21

Coun. & Ed. Psyc.

31

71.50

71.50

Prior
Computer Knowledge

Expected
Eng. Ach.

Religious Studies

28

72.09

72.09

Little or no

Language Ed. A

19

73.00

73.00

Medium or higher

Language Ed. E

20

75.30

Eng. = Engagement; Ach. = Achievement

= higher score; = lower score; ~ = randomly varying score

68.07

Table 4. Research hypotheses and results are compared.

Found
Eng. Ach.

~
~

As seen in the last row of Table 1, students with little or no

prior knowledge rated an average engagement score of
1.78 across the classrooms, which did not turn out to be
significantly different from the engagement rates of students
with medium or higher prior knowledge.

What was found was not in support of the research

hypotheses. Even though there were some indications
supporting the hypotheses as described in Figure 2, the
levels of engagement as a construct could not be
significantly predicted from the students prior knowledge.

On the other hand, students with little or no prior knowledge

had average exam scores (67.14) that were significantly
lower than the scores of the students with higher perceived
prior knowledge (71.51).

The researchers also further investigated whether being a

student of different department could affect these dependent
variables in order to control for the outcomes. Although
students from different departments have varying degrees of
engagement, even this variation have no interaction effect
with student prior knowledge on student engagement.
Therefore, a linkage cannot be derived specifically for prior
knowledge, engagement, and achievement.

Although the students with little or no prior knowledge were

not more engaged than their counterparts in school work,
there were indicators that those students thought they spent
more effort to meet their teachers expectations (see Figure
2). When the students were asked whether they had to
study more than they expected to meet their teachers
standards, the students with lower prior knowledge marked
often and very often more frequently than the others.
Moreover, the former marked never less frequently than

This study examined the mediating affect of engagement.

Regardless of engagement, however, the students with little
or no prior knowledge received exam scores that are
significantly less than their more knowledgeable

200

counterparts. Independent from the focus of this study,

these results align with the literature.

Skinner & Belmont (1993). Motivation in the classroom:

Reciprocal effects of teacher behaviour and
student engagement across the school year.
Journal of Educational Psychology, 85(4), 571581.

These findings can be beneficial to inform whether such

computer literacy courses should be required for all
freshmen. It is the authors recommendation that there
should be an entrance exam, which determines the level of
student knowledge in terms of computer literacy. Students
who are already computer literate should be exempt from
such courses. Such an intervention would provide many
benefits including but not limited to: (1) advanced students
would make better use of the extra time they gain; (2)
resources would be used more effectively especially
thinking that todays computer labs are not able to meet the
demand; (3) government would save a substantial amount
of budget; (4) instructors would better be able to spare their
time for the needy.

Spires, H. A., Donley, J. and Penrose, A. M. (1990). Prior

knowledge activation: Inducing text engagement
in reading to learn. Paper presented at the annual
meeting of the American Educational Research
Association, Boston, MA.
Thompson, R. A., & Zamboanga, B. L. (2003). Prior
knowledge and its relevance to student
achievement in Introduction to Psychology.
Teaching of Psychology, 30, 96101.
Thompson, R. A., & Zamboanga, B. L. (2004). Academic
Aptitude and Prior Knowledge as Predictors of
Student
Achievement
in
Introduction
to
Psychology. Journal of Educational Psychology
Vol. 96, No. 4, 778784.

However, these findings alone cannot differentiate whether

knowledgeable or less knowledgeable students would be
more active when taking classes. It is not in the scope of
this study to determine what controllable factors can affect
engagement so that a positive change can be made to
improve engagement; but the scope of this study indicates
that from the classroom standpoint, students of computer
literacy classes characteristically have low engagement
rates. Actions should be taken to increase levels of
engagement in the classroom.
REFERENCES
Carini, R. M., Kuh, G. D., & Klein S. P., (2006). Student
Engagement and Student Learning: Testing the
Linkages. Research in Higher Education, 47 (1),
1-32.
Chapman, E. (2003). Alternative approaches to assessing
student engagement rates. Practical Assessment,
Research & Evaluation, 8(13). Retrieved March
31, 2008 from http://PAREonline.net/getvn.asp?
v=8&n=13
Cronbach, L. J. (1951). Coefficient alpha and the internal
structure of tests. Psychometrika, 16(3), 297-334.
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004).
School engagement: Potential of the concept,
state of evidence. Review of Educational
Research, 74, 1, 59-109.
Marks, H. M. (2000), Student engagement in instructional
activity: Patterns in the elementary, middle, and
high school years. American Education Research
Journal, 37, 1, 153-184
National Center for School Engagement, (2006). Quantifying
School Engagement: Research Report, Colorado
Foundation for Families and Children, Denver
National Research Council and the Institute of Medicine.
(2004). Engaging schools: Fostering high school
students' motivation to learn. Committee on
Increasing High School Students' Engagement
and Motivation to Learn. Board on Children,
Youth, and Families, Division of Behavioral and
Social Sciences and Education. Washington, DC:
The National Academies Press.
Natriello, G. (1984). Problems in the evaluation of students
and student disengagement from secondary
schools. Journal of Research and Development
in Education, 17, 14-24.
Pascarella, E., & Terenzini, P. (2005). How college affects
students (Vol. 2): A third decade of research. San
Francisco: Jossey-Bass.

201