You are on page 1of 16

ABSTRACT

E-LEARNING COURSE ON
WASTE MANAGEMENT
Course Evaluation Report

This report consists the evaluation of


the E-learning course on Waste
Management by Group E. The
evaluation
evaluates
the
effectiveness of the implemented
module of the course as per the
guidelines provided in IS 3018-Elearning & Instructional Design and
the wiki framework provided by the
Learning Management System. It also
contains the brief overview of the
course and UPDATED learning
objectives as per the Blooms
taxonomy.

Group E
IS 3018 E Learning & Instructional
Design

Contents
1.

Team members and their roles ............................................................................................................ 2

2.

Introduction to the Course................................................................................................................... 3

3.

Goals and Learning objectives of completed module ......................................................................... 4

4.

Evaluation procedure (method, personnel & criteria) ....................................................................... 5

5.

Results of Evaluation ............................................................................................................................ 7

6.

5.1

Accessibility ................................................................................................................................... 7

5.2

Content ......................................................................................................................................... 7

5.3

Goals and objectives ..................................................................................................................... 8

5.4

Structure ....................................................................................................................................... 8

5.5

Visual design ................................................................................................................................. 8

5.6

Text ............................................................................................................................................... 9

5.7

Timing............................................................................................................................................ 9

5.8

eLearning resources. ..................................................................................................................... 9

5.9

Interactivity ................................................................................................................................. 10

5.10

Multimedia .................................................................................................................................. 10

5.11

Assessments ................................................................................................................................ 10

5.12

eLearning professional contribution ........................................................................................... 10

5.13

Social interactions availability ..................................................................................................... 11

5.14

eLearning course expectations ................................................................................................... 11

5.15

Overall eLearning experience ..................................................................................................... 11

Discussion ........................................................................................................................................... 12
6.1 Learning Theories used .................................................................................................................... 12
6.2 Design guidelines & principles ......................................................................................................... 13

7.

Conclusion and future work ............................................................................................................... 14


7.1 Conclusion ........................................................................................................................................ 14
7.2 Future work ...................................................................................................................................... 15

1|Page

1. Team members and their roles

Index No

Reg Number

12020196

2012IS019

M. R. Faideena

Project
Manager,
Developer, Animator

12020082

2012IS008

Sajeeha Begum

Designer, Content Developer

12020374

2012IS037

Minura Jinadasa

Editor,
Narrator,
Developer

Content

12020201

2012/IS/020

M. S. M. Fathih

Developer, Graphic
Content Developer

Designer,

12020323

2012IS032

Fareed Isfan

Graphic
Designer,
Developer

Content

12020684

2012is068

Roshana Fathima

Designer, Content Developer

2|Page

Name

Role
Content

2. Introduction to the Course


Technological advancement, improved healthcare and increased standard of life has
enabled human beings to overcome natural population controls unlike other species and
maintain in a continuing exponential increase in global population. Increase in
population together with the quest for better and better living standards have increased
the production of waste as well as posing a serious strain on the existing scarce resources
which are depleting. The world is struggling with problems of high waste volume,
disposal costs, disposal technologies and methodologies, impact on the environment and
resource depletion.
One of the possible solutions for this is effective waste management. However, awareness
in a mass scale is impractical to attain through traditional awareness methodologies. As
such, public awareness e-learning course has been developed for waste management.
This Elearning course has been developed by third year students of The University of
Colombo School of Computing under the guidance of Dr. T. A. Weerasinghe (Ph.D.),
Senior Lecturer of UCSC as a part of course IS 3018 E-learning and Instructional Design.
The course is to offer accessible, practical & sustainable waste and resource management
skills. It improves understanding of how & why waste should be managed efficiently and
offer advice and guidance that will save money and aid regulatory compliance.
The course outline below provides an overall idea of the entire course
1. Introduction (Course description, audience and course delivery method)
2. Environment issues (pollution, env issues & need for waste management)
3. 3R system
(The Reduce, Reuse, Recycle approach concept, benefits & application with
examples) This was the implemented module as it is the module that would
provide the highest benefit independently of others when implemented
standalone. Also, implementing a module that can be taught standalone helps
us assess the module, design, content effectiveness since we know that any
mistakes will not be due to dependencies with other modules but drawbacks in
implementation. (Implemented Module)
4. 3D system (Colour coded dustbins at home/city/country concept & application)
5. How to minimize creation of waste at home, in city & country
6. Organic waste management
7. Health and safety
8. Innovative solution

3|Page

3. Goals and Learning objectives of completed module


The goal of the module is to develop a system that enables the individual to understand
the 3R concept to the extent that he can teach it to another and implement the system in
practice in day to say life.
On completion of the module, course participants should be able to;

explain the concept of 3R to a person who has no idea of the 3R concept

outline the importance of reducing, reusing and recycling to a person who has no idea
of their importance

distinguish between conventional reuse & new life reuse

apply the concept to manage waste at home by reducing and reusing

identify recyclable material at home

list at least 5 examples of applications of each of the 3R at home

4|Page

4. Evaluation procedure (method, personnel & criteria)


In evaluating, the system, we initially focused on the user friendliness on the system
more but with later online research, we identified more areas that we should consider
in evaluating a system developed for e-learning. Multiple web sites were used for this
purpose but we found www.elearningindustry.com to be the most useful with a
comprehensive framework which we utilized and enhances using other sites.
In evaluation, we felt it essential to use external participants who are new to the course
(which excludes people who earlier provided feedback such as for storyboards).
Although a large sample from multiple backgrounds should have been used, time
availability and schedule constraints meant we could only use three people. We
however ran each of those three through the entire course and did a proper evaluation
although we could not involve a large number of people. All three had reasonable
English communication ability. (This course does not support visually impaired or
physically handicapped users who cannot use a mouse and keyboard which is a
limitation).
The three people were A=An IS student of UCSC, B=A CS student of UCSC, C=An
employed individual, a former student of Royal College CIMA foundation complete
but currently having no degree.
The conducting of the test on subjects A and B were carried out by us which was in
effect, an alpa evaluation while we allowed another IS student to carry out the
conducting of the beta evaluation, having first provided him with the material.
However, individual modules have been designed from inception to be used fully
automated except at the stage of giving one-to-one human feedback through fora as
the intention of this course is to serve as a public awareness course for the masses.

5|Page

Following key areas were identified by us in performing the evaluation


1. Accessibility
2. Content
3. Goals and objectives
4. Structure
5. Visual design
6. Text
7. Timing
8. eLearning resources
9. Interactivity
10. Multimedia
11. Assessments
12. eLearning professional contribution
13. Social interactions availability
14. eLearning course expectations
15. Overall eLearning experience

6|Page

5. Results of Evaluation
5.1 Accessibility
Accessibility refers to the ease of accessing the course. The course is hosted online and
is available free of charge for any interested party and as such, accessibility is high.
We have also followed the guidelines in IS3018 to increase this. To give a few
examples we have tried it in Firefox and Linux on Chrome as well as Firefox, Chrome,
Safari and Microsoft Internet Explorer in Windows (Chrome was on a dual core
laptop). Except for an inherent slowness of internet explorer, it worked reasonably
fast, each page loading well above the 10 second threshold that frustrates users
including in Internet Explorer.
However, it was not possible to find the course when searched by common keywords.
This is an inherent problem when using free domains, however we did attempt a level
of SEO when it didnt turn up on the first search page, although it still did not appear
in the first page.
Navigation was considered effective by all three and this was understandable
considering the site followed at all times possible, the standard web layouts and
buttons including in the content pages. All links and buttons had been tested by us
and worked well as we had tested them earlier.

5.2 Content
We understood that text and the narration seemed to contradict at 2 points when a
reader reads while listening. We reduced parts of the narration to remove this. (This
was pointed out by participant C; a fact missed by both A & B. We also adjusted the
narration speed in 2 slides which had more text so that an average reader (reading at
approx. 200 words per minute) will not get left behind during narration.
Content on slides were approved by all three as clear. The content was adjusted
several times throughout the development process to ensure there was not too much
text but also not too many slides with little amount of text that would discourage
participants due to slide numbers; by reaching a balance. The images selected were
used to complement the text consistently and this was commented by participants A
and B both on their own accord while studying examples. When queries, they
affirmed that the examples were found to be relevant by them.

7|Page

5.3 Goals and objectives


We had clearly defined goals and objectives with the audience, behavior, condition
and degree stated and as such, it was easy to evaluate the achievement. Both A and B
were able to demonstrate the required knowledge but we did not consider the
response of C for objective 1 as he stated that he was familiar with the 3R system.
Considering that the system was developed to solve a very real problem in real life
and the objectives were centered on this, they do provide real world benefits. We got
a suggestion that we might include an objective that focused on implementing a 3R
system by participant C but we are not in a position to execute this one since as an
eLearning system, we cannot evaluate projects by individuals.

5.4 Structure
The content sequence tuned by us during preparation of course maps and story
boards. All three participants affirmed that they found the flow of the content to be
logical and well-structured with consistency across how each of the three Rs were
presented.

5.5 Visual design


As trackable from earlier submissions, we changed the images and the placement until
final stage to make them more streamlined. In doing so, aesthetic balance as well as
consistency was kept in mind, a reason for us to not add certain images even though
they were arguably explanative in comparison to some current ones. While we do not
claim the aesthetic aspect to be outstanding in this system, we have attempted to
maintain a simple, yet light and pleasing style. This was a reason to comply with the
recommendation of C, removing the original green background in the design for the
pages during implementation. C pointed out that while green portrayed
environmentally friendliness, it is distracting and reduced visibility.
As claimed by both B and C, we too found that narrative needs control. Therefore, we
cut back automatic narratives and instead, added an icon to play and pause narrative
while also allowing to move to some extent to different points of the narrative with a
track bar.

8|Page

The font was changed from the original design when making the revised design. We
maintained the formatting in the final revised design and this appeared appealing to
our test subjects as well.

5.6 Text
As this is a public awareness course, from inception, special attention was paid to
ensure that the sentence structures, words and ideas were simple to extent possible
without compromising the desired learning outcomes. Spelling and grammar was
tested by the team to comply with design.
We had included a significant amount of text in each slide in the first design. In order
to highlight key points as well as enhance the readability of the slides by reducing
cluttered up text. We transferred majority of the information in to the narrations.

5.7 Timing
For our participants A, B and C, the entire module took less than 2 hours (participant
C taking 1 hour 45minutes which was the longest); to the level to complete the quiz
with 80% accuracy. While C commented that he could have finished faster if required,
all three agreed that what they learnt was sufficient within the time period (also taking
in to account the fact that they were able to show sufficient recall to meet the learning
outcomes after an hour). The time allocated for module 3 in syllabus is 3 hrs.

5.8 eLearning resources.


The main resource we focused on beyond the interactive page made by us are the two
videos. One video was embedded in the lecture notes while the other was provided
separately made from flash. There were also three links to pages which provided
additional examples without lengthening or cluttering the main notes. All three
participants had checked the links but they mentioned that they did not spend much
time on those external pages except to get an overall idea. We acknowledge therefore
that the actual total time spent might vary to some extent with participant attention to
resources. It was clearly acknowledged by them that the link and its purpose was
visible. We removed the actual link and embedded it in the word Link which was in
the standard link format (blue, underlined) to make it appear consistent.
9|Page

5.9 Interactivity
Considering this should be a system that was to be used by a diverse range of
individuals, we the interactivity was not excessive but at a suitable and very standard
level and we also paid attention to the loading time and client machine/bandwidth
constraints in deciding on interactive features. Considering this, we cut back on the
special effects such as when clicking a button but maintained aesthetic appearance.
This was balanced to ensure that response did not exceed the maximum allowable 1.0
second for interactive systems although once pages were loaded, response was
practically instantaneous.

5.10

Multimedia

This was considered together with interactivity. The pages of the system contains
narrative and two videos as stated to improve the effectiveness of the learning
process. They have been evaluated by us and also affirmed by the participants A, B,
C to be useful as well as usable in systems they used while increasing attractiveness.

5.11

Assessments

Variety of the quizzes to cover the breadth was considered sufficient by the
participants. It was recommended by participant C to consider developing a method
to track the progress of multiple participants and we noted this as a future
development. Both MCQ and drag-and-drop quizzes were approved by participants.
As the course was not fully developed, being restricted to one module, the formative
evaluation of the module was, in effect, also the summative evaluation at the current
stage of development though grading facility is not provided at this stage.

5.12

eLearning professional contribution

This focused on availability and support of the professional, that is, the lecturer. We
could not evaluate this since it varies from person to person, however, in developing
this course, we constantly focused this to be a system that could be used fully
independent of a lecturer and this was evaluated as such a system with positive
feedback.

10 | P a g e

5.13

Social interactions availability

This was facilitated by the forum on the learning site. We tested it for functionality,
but we could not extensively test it with multiple participants and we launched the
system with current functionality.
A limitation that we are currently working to improve is the inability to add files in
postings for other participants to see.

5.14

eLearning course expectations

Considering the module that we implemented, we can reasonably affirm that the
learning outcomes were met when we evaluated performance from the quizzes as well
as random questioning by us to further ensure the meeting of the objectives. The
reason for further questions were not due to insufficiency of the quiz; the quizzes were
developed for the candidate to test him/herself while in this case we needed to also
evaluate the system which was not a primary function of the quiz.

5.15

Overall eLearning experience

In conclusion, all three participants were satisfied with the module and also provided
some constructive feedback. We were also able to evaluate this from our questions on
their knowledge of the course.

11 | P a g e

6. Discussion
6.1 Learning Theories used
The main theory that guided us is cognitivism. We focused on how people learn using
schema where we intended to maintain a logical flow of information, giving information
in chunks that make sense. This is clearly evident when considering the consistent
manner that each slide has just the right amount of related info that can be processed and
assimilated in one or to chunks and the separating/spacing of paragraphs and points too
are expected to contribute to this, for eg, each R concept explanation slide is consistent
with the others and each R concept slide is followed by an example slide; these example
slides are all consistent. The contents list clearly outlines the course contents so that the
learner understand the flow and knows what to expect at which point ones the structure
is understood. To increase effectiveness, we have used consistent and related images with
text which is also backed up by the narration.
We have made of secondary use of constructivism to a less extent when familiarizing the
applications of concepts by using real life examples that participants directly come to
contact with so that knowledge can be created by relating the new information with the
past experience.
Social constructivism is also used
Where the quiz is concerned, in addition to the above, operant condition of behaviourism
was used by perceptually rewarding the user with sounds of approval and the colour
green with a tick as well as negative sounds/warnings (familiar due to their use in
Windows) associated with red and also inability to make the move (in drag and drop)
In effect, while cognitivism forms the base, we were guided all four theories to various
extents in developing the course

12 | P a g e

6.2 Design guidelines & principles


The process of design was initiated by identifying the learning objectives. They were
later revised as per the Blooms taxonomy as instructed. Following analysis of the Task,
Need, User and the environment, the required content was listed and adjusted with an
external expert, a contact of a team mate.
The course maps, first high level followed by the detailed level ones were developed
in the format taught and this was approved by all team members. The method of
content presentation was decided as videos and interactive pages followed by quizzes
at each stage.
Following this, story boards were developed by us individually according to the format
directed to us, in order to prepare the plans for the pages as well as to provide a base
for discussion and improvement as well as communication of the layout of the course.
(Story boards were developed for only Chapter 3 The 3R system as were instructed
to develop only one of the planned modules in the syllabus). Checking the story boards
for consistency was done through the checklist in lecture 4 and the suggestions for
improvement were from individuals outside the team.
We had initially decided to add a background music with the narration, however, we
later decided against it as the intended audience is highly diverse and we want this to
fit all audience of all strata with different understanding capability & we felt music,
while soothing, would be distracting and compete with the narration while also using
bandwidth.
Audio scripts were developed keeping all guidelines in mind to ensure clarity and clear
communication, with simple words and guidelines as to pauses, emphasis etc, no page
turns were allowed inside a single narration.

13 | P a g e

7. Conclusion and future work


7.1 Conclusion
Considering the feedback received and our view considering the final output of
considerable refinement to the end results, we feel that we can state conclude with
reasonable certainty that the system developed is successful. The system when evaluated
against the criteria above by us and also independent participants provided satisfactory
results meeting the objectives defined while also making the learning process and
interesting experience accessible to a wide range of users.
It is also our belief that despite the extremely high workload involved, we did make
continuous improvements in our capability in designing and developing effective
eLearning systems through an effective and high standard process as we understood
from improvements we were able to make from initial stage to the final development of
the system through hands on practical experience, making us confident that we are now
able to develop an eLearning system to fulfill education and training requirements given
the objectives and the required tools.

14 | P a g e

7.2 Future work


Due to time constraints, only one module of the multiple module system has yet been
developed and as such, the first priority would be to develop the rest of the modules to
prepare a complete waste management course using the same methodology used for this
module (and the learning gained during the process).
In considering further improvements, we are considering features improvements which
will enable participants to be evaluated by the facilitator, results analyzed in a range of
ways to identify the effectiveness of the course material and delivery, that is, general
results of all students in a particular areas; and also the result of each student in multiple
areas (evaluation of the module and evaluation of the student)
A further improvement which is already being considered is the ability to upload files in
posts to share with colleagues in building successful discussions for effective
collaborative learning.
Integrating an open source badge system & automated certificate creation is another
possibility where one can receive a certificate with the grade and also post the badge as a
link in for eg LinkedIn which, when clicked, gives course details.

15 | P a g e

You might also like