ISSN 2336 International Journal of Teaching and Education Vol. II (No.

4)

Use of i-clickers to enhance learning outcomes assessment in

classroom: A Case Study in King Saud University

Fatiha Bousbahi

Fatiha Bousbahi IT Department, CCIS, King Saud University, Riyadh, Saudi Arabia
Email:fbousbahi@ksu.edu.sa

Abstract

Educators in King Saud University understand that helping students learn is a continual
process, requiring a commitment to assessment and evaluation. Nowadays, formative
assessment is a crucial component of teaching and learning. Hence, teachers are in need of
tools to assess students’ understanding and skills into classroom to improve students’
academic achievement. i-clickers are simple technology allowing formative assessments due
to the immediate information received by the teacher concerning the students’ mastery of a
given concept. Thus teachers can adjust their teaching methods accordingly on time. This
paper reports an experience conducted in the Department of Information Technology (IT) in
King Saud University (KSU). i-clickers served as an assessment tool of Course Learning
Outcomes (CLOs) in classroom. Findings revealed that firstly students were much better in
performing course projects and secondly teacher was able to improve her course strategies
more efficiently and identify students’ misunderstanding on time.

Keywords: Formative Assessment; Course Learning Outcomes; Student; i-clicker; Feedback.

17
 ISSN 2336 International Journal of Teaching and Education Vol. II (No. 4)

1. Introduction
The usage of Technology Enhance Learning (TEL) in education has evolved significantly
over the last decades and government spending for education in Kingdom of Saudi Arabia is
growing abundantly. A fund of SR 204 billion was allocated to the educational sector in 2013
(Irfan, 2013). The Department of Information Technology (IT) is one of the five departments
of the College of Computer and Information Sciences (CCIS) at KSU. It is the first
department in this specialization for female students in Saudi Arabia. Convinced by the
important role of women in Saudi society, KSU has undertaken the task of developing the
skills of women in order to ensure qualified graduates to work in the field of Information
Technology. From this standpoint, IT department considers linking academic excellence to
completion is essential and must be done. To fulfill its core mission, essential and ongoing
assessments are done to structure an environment of student success and completion in IT
department.

Literature reviewer
Even e-learning exists everywhere nowadays, traditional face to face delivery of courses is
still the main way to give and receive learning. In IT department at KSU, a blended learning
(Osguthorpe, 2003) is offered where lectures are done in traditionally way and other parts of
academic delivery as: materials, homework, announcements, calendar occur using online
platform via a Learning Management System (LMS). Consequently, motivation, engagement
and assessment play a fundamental role to improve learning.
According a study conducted by Hartley and Davies (1978) the amount of information
retained for the last ten minutes of a lecture decreased to 20%. Thus the traditional teaching
dilemma is how to get students involved in classroom.
To enhance students’ retention, motivation, participation, and assessment during learning
process (Thomas, 1972), KSU launched an experience in Spring 2014 to adopt Student
Response Systems (SRS) in classroom. The experience was launched in attempt to offer
active learning rather than passive learning and formative assessment rather than only
summative assessment. Passive learning is a teacher-centered approach to learning in which
student’s role is limited to listening and taking notes (Blumberg, 2008) while active learning
is focusing not only on teacher but also on student. The student is required to be involved in
classroom (Biggs, 2011). Summative assessments are administrated for grading purposes.
They lead students to focus much more on grades then knowledge. They are not suited for
assessing higher-level understanding and skills (Wirth, 2005). Formative assessments are a
pro-active approach to education due to the immediate information received by the teacher
concerning the students’ mastery of a given concept (Hayes, 2012). Performed regularly, they
allow students to situate their knowledge throughout a semester, especially before exams.
There is a real need of simple tools as SRS in classroom that: 1) increase motivation and
retention during lecture; 2) provide formative assessments of Course Learning Objectives
(CLOs) immediately following the lecture; 3) turn assessment in knowledge by supporting
immediate feedback; 4) can be easily used by students to reflect on and monitor their

18
 ISSN 2336 International Journal of Teaching and Education Vol. II (No. 4)

progress; 5) can be used by teachers to identify students’ misunderstanding and adjust their
teaching methods accordingly on time.
As instructors are always seeking to elevate students’ performance and engagement in
classroom, the SRS offer a potentially helpful teaching and assessment tool.

Student Response Systems: i-clicker

i-Clickers are simple technological devices that enable instructors and students to interact
dynamically in minutes. They are used to promote active learning by increasing the
opportunity for the student to participate in classroom. They allow students to actively take
part in lectures by anonymously respond to interactive questions such as multiple choice,
true/false, or opinion questions previously selected by the teacher. Current technology i-
clicker software provides histograms of the aggregated responses and includes time for
students to compare their viewpoints and possibly revise their answers. Prompt feedback on
understanding is an essential contributor to student learning (Nelson & Pearson, 1999) and
many studies proved that SRS enabled professors to gauge student understanding and respond
accordingly. The most important benefit of SRS is that, they can engage learners and
instantly provide feedback for teachers to validate student achievement (Ducan, 2005;
Weerts, Miller, & Altice, 2009; Kimo, 2010). According to Siew, both teachers and students
would benefit from an assessment tool such as SRS that enables formative assessments or
feedback to be provided at different points during the learning process (Siew, 2003).
Furthermore, the use of TEL as i-clickers into (CLOs) assessment will enable faculties to
reconsider learning and teaching strategies and improve their efficiency in managing
assessment.

Course Assessment Methodology at IT Department

IT Department makes Learning Outcomes (LOs) assessment an explicit part of department
culture. With 2010, IT department have become increasingly engaged in articulating and
assessing learning outcomes to account for and ensure quality in its educational programs.
The KSU academic calendar is divided into two semesters and a semester is an average of 15
weeks lectures. The IT Department offers generally, courses that contain three semester
credit hours of content, of which two contact hours are dedicated to laboratory practice. The
assessment is based on the assessment of Student Outcomes (SOs) covered by each course.
The assessment is mandatory for all courses in each semester. Each faculty member teaching
a course must produce a Course Assessment Report (CAR) as part of the course portfolio. If a
course has several sections and each section is taught by a different instructor, assessment
must be done for each section separately. The CAR must contain both direct assessment and
indirect assessment. The direct assessment considers the point of view of the instructor
through exams, quizzes, assignments and projects. The indirect assessment considers the
point of view of students through surveys. Indirect assessment evaluates the attainment of
specific learning outcomes of the course as well as student outcomes covered by the course.
The CLOs survey is posted for students at the end of the academic semester to assess their
learning outcomes from the course. The questionnaire is answered by the enrolled students in
the course. Table 1 gives the list of CLOs to be achieved in Intelligent Systems (IS) course,

19
 ISSN 2336 International Journal of Teaching and Education Vol. II (No. 4)

Fall 2013-2014. Direct assessment evaluates the attainment of CLOs covered by the course.
For both direct and indirect assessment, faculties use the percentage (%) of students
achieving the satisfactory or exemplary levels. Each faculty member must keep her data at
the most detailed level. Results of this assessment are collected at the end of each semester in
a CAR. Faculty members compare performance levels of current students with those of
students in previous offerings of the same course. Based on the results, the teacher relates
learning barriers and issues expressed by the student while filling the survey. The teacher is
asked to propose a plan to overcome these barriers next semester. Quality unit staff
intervenes to highlight problems but unfortunately this intervention is not on time. The results
are carried too late to undertake any action for the actual students whom from the feedback
about teaching was taken. Therefore we intend to do the process along the semester in a
timely manner where it will be most beneficial and giving students a sense of their progress.

Table 1. Students Outcomes addressed by the course IS.

# Student Outcome addressed by the course

An ability to acquire knowledge of best practices and their
(o)
applications;
An ability to acquire knowledge of standards and their
(m)
application;
An ability to analyze a problem, and identify the computing
(b)
requirements appropriate to its solution;
An ability to apply knowledge of computing and
(a)
mathematics appropriate to the discipline;
An ability to analyze the local and global impact of
(g)
computing on individuals, organizations and society.

2. Methodology
We (the author) taught Information Systems (IS) course three semesters without SRS. At the
end of these semesters, while gathering data to assess and evaluate the course, we found a gap
between direct assessment and indirect assessment especially for the student outcomes (g, m,
o). We observed also that students have difficulties to apply the concepts and techniques
learned in real world problems while performing mini-projects of the course. Figure 1
illustrates the results of course assessment and comparison between direct assessment and
indirect assessment. It shows that the attainment of SOs: g, m and o is under expectation in
Spring 2013. In attempt to address these gaps, this study aims to explore the effects of
lectures that incorporated i-clickers as formative assessment on students’ achievement
immediately following the lecture.
This paper reports on the use of i-clickers as a formative assessment tool in undergraduate
course. The purpose of this assessment was to elevate engagement in classroom and increase
students’ performance. No grades were given to this assessment.

20
 ISSN 2336 International Journal of Teaching and Education Vol. II (No. 4)

2.1. Study setting

The experience launched by KSU was conducted in many departments such as IT
Department in CCIS. The participants in IT Department were volunteers among faculty
members in many courses, such Data mining, Software Engineering, IS, Data Structure and
Architecture. There was not a uniform use for i-clickers in the classroom. The participants
were free to choose in which manner to use these devices in classroom. As mentioned earlier,
the experience had as aims but not limited to: first, evaluate the effectiveness of SRS in
increasing students’ academic performance, compare passive learning (lecture without i-
clickers) versus active learning (lecture with i-clickers) on students’ motivation and
participation in class and second, use i-clickers as CLOs assessment tools. This paper study
concerns the introduction of the SRS in classroom by the students (n=47) enrolled in IS
course. It was conducted with two sections of this course. The intent of this paper was to
evaluate the effect of using SRS to assess LO and the effectiveness of these devices on
student’s academic achievement.

Fig. 1. SOs/CLOs assessment - Fall 2013

2.2. Description of the course

The course IS introduces students to the wide field of Artificial Intelligence (AI) with
emphasis on its use to solve real world problems. During the academic semester, the course
IS covers three hours of lectures and two hours of lab per week. During lectures, various
fundamental topics of AI including methods and algorithms are explained and demonstrated
to be understood and put in practice by students during the lab work through exercises. This
course is also an opportunity for students to discover important applications of AI
technology. At the end of the course, students practice programming skills by achieving a
mini project where they have to find a problem’s solution by applying AI algorithms and
techniques. On completion of this module, students should relate in a mini research project
what they have learned to what impact AI is making to society. The course addresses five
Student Outcomes (SOs) shown in Table 1. The course was split into six units. Each one has
a CLO shown in Table 2.

21
 ISSN 2336 International Journal of Teaching and Education Vol. II (No. 4)

Table 2. CLOs of IS course and their mapping with the SOs

# Course Learning Outcomes

1 Solve a problem and provide the suitable formulation. (b)
2 Apply the different search techniques: uninformed search, informed search and
constrained search Select the suitable. (a)
3 Select the suitable AI technique to solve a particular problem. (m)
4 Apply the different ways to represent knowledge: propositional logic and first order
logic. (b)
5 Describe and apply the inference rules and algorithms.(o)
6 Analyze the impact of AI based technologies. (g)

2.3 Procedure
We incorporated questions about CLOs in lectures at the end of each unit. Questions were in
three forms: multiple choice, true/false and opinion questions. Students answered questions
anonymously. Results’ data were displayed immediately upon completion of each interactive
question to serve as immediate feedback to the teacher and the student. This encouraged
discussions between teacher and students and also between peers. A survey questionnaire
(part of indirect assessment) in form of opinion question was given to students after each unit
i.e. after achieving each CLO. Table 3 shows the survey results for all the CLOs in Spring
2014.
At the end of the semester, data was collected as usual, on student grades based on common
test questions on the quizzes, midterm, mini-projects and final exams for both sections as
direct assessment process. Data of the questionnaire survey was gathered from i-clickers
report to perform the indirect assessment.

3. Findings
The purpose of this section was to examine the effects of clickers as assessment of CLOs to
improve students’ learning and course evaluation. The results show that the SRS are useful
tool to conduct formative assessment throughout the course to provide immediate feedback to
teacher and students to improve students’ achievement. Indeed results show that students did
a progress while performing their mini-projects mentioned in the description of the course
and increase their grades in the summative assessment. The attainment of CLOs exceeds
expectations in both direct and indirect assessments. Table 3 shows the percentage of
Satisfactory. Results have shown also that the gap between direct and indirect assessments
was significantly reduced.
The findings revealed that the use of i-clickers has a positive impact on overall student
learning.

22
 ISSN 2336 International Journal of Teaching and Education Vol. II (No. 4)

Fig. 2. SOs/CLOs assessment in Fall 2013

Figure 2 above shows that attainment exceeds expectations for CLO1 and CLO2 and meets
expectation for the remaining CLOs for both direct and indirect assessment. Findings have
shown and confirm that by using i-clickers students are more attentive and encouraged to
share accountability with teacher into classroom.

Table 3. CLOs assesment - Spring 2014

I am able to SA A N D SD
1. Solve a problem and provide the suitable 19% 72% 8% 1% 0%
formulation.
2. Apply the different search techniques: 34% 51% 14% 1% 0%
uninformed search, informed search and
constrained search Select the suitable.
3. Select the suitable AI technique to solve a 34% 51% 14% 0% 0%
particular problem.
4. Apply the different ways to represent 18% 51% 25% 2% 2%
knowledge: propositional logic and first order
logic.
5. Describe and apply the inference rules and 22% 51% 24% 2% 1%
algorithms.
6. Analyze the impact of AI based technologies. 15% 63% 19% 3% 1%

4. Conclusion and Limitations

The Student Response Systems as i-clickers are a useful tool for enhancing learning. The
study shows an increase of level overall understanding in class. There is also a significant
increase of students’ performance comparing the results of course evaluation before adopting
SRS and after. For the student, i-clickers provide an explicit guide to the concepts, techniques
and learning objectives of a course. They are an efficient formative assessment tool and can
be used to help students develop self-assessment skills. Instructors can use i-clickers’ results
to assess student understanding and the effectiveness of new course strategies. SRS results
indicate they are correlated with other assessment tools as exams and course grades. These

23
 ISSN 2336 International Journal of Teaching and Education Vol. II (No. 4)

results are largely consistent with case studies in other countries cited above in the literature
review.
We recognize that this pilot study is limited by the small size of students. We intend to
expand our experience further in other courses with large classes for 2 semesters.

Acknowledgements

This research project was supported by a grant from the Research Center of the Center for
Female Scientific and Medical Colleges in King Saud University.

Reference
Biggs, J. & Tang, C. (2011). Teaching for Quality Learning at University. The Society for
Research in Higher Education. Fourth Edition. UK.
Blumberg, P. (2008). Developing Learner-Centered Teaching. San Francisco: Jossey-Bass.
Duncan, D. (2005). Clickers in the classroom: How to enhance science teaching using student
response systems. San Francisco: Pearson.
Fies, C. & Marshall, J. (2006) Classroom Response Systems: A Review of the Literature.
Journal of Science Education and Technology, 15, 101-109.
Gauici, S., Dantas, A., Williams, D. & Kemm, R. (2009). Promoting Student-Centered Active
Learning in Lectures with a Personal Response System. Advances in Physiology
Education, 33, 60-71.
Hartley, J., & Davies, I. (1978). Note Taking: A Critical Review. Programmed Learning and
Educational Technology, 15, 207–224.
Hayes, J. A. (2012). A comparison of lecture and interactive lecture using student response
systems in an inclusive and non-inclusive classroom. A Dissertation Submitted in Partial
Fulfillment of the Requirements for the Degree of Doctor of Education. The University of
Memphis.
Irfan, M. (2013). Kingdom tops world in education spending. Arabnews. 4th January.
Available from: http://www.arabnews.com/saudi-arabia/kingdom-tops-world-education-
spending. [Accessed: 10 October 2014].
Kimo, A. Y. (2010). The effects of Clickers on Student Learning. Academic Exchange
Quarterly. Vol.14, Issue 1.
Nelson, P. E. & Pearson, J. C. (1999). Large lecture classes, in: A. L. Vangelisti, J. A. Daly &
G. W. Friedrich (Eds) Teaching communication: theory, research, and methods (Mahwah,
NJ, Lawrence Erlbaum), 347–358.
Osguthorpe, R. T. & Graham, C. R. (2003). Blended learning environments: definitions and
directions. Quarterly Review of Distance Education, 4(3), 227–233.
Thomas, J. (1972). The variation of memory with time for information appearing during
lecture. Studies in Adult Education, 4, 57-62.
Trees, A. R. & Jackson, M. H. (2007). The learning environment in clicker classrooms:
student processes of learning and involvement in large university-level courses using
student response systems. Learning, Media and Technology, Vol. 32, No. 1, March 2007,
pp. 21–40.

24
 ISSN 2336 International Journal of Teaching and Education Vol. II (No. 4)

Siew, P. (2003). Flexible on-line assessment and feedback for teaching linear algebra.
International Journal of Mathematical Education in Science & Technology, 34(1), 43.
Weerts, S., Miller, D., & Altice, A. (2009). “Clicker” Technology Promotes Interactivity in
an Undergraduate Course. Journal of Nutrition Education & Behavior, 41, 227-228.
Wirth, K. R. & Perkins, D. (2005). Knowledge Surveys: An Indispensable Course Design and
Assessment Tool. Innovations in the Scholarship of Teaching and Learning.

25