Evaluating Learning

Introduction to Evaluating
Evaluation is a continuous process in instructional designs.

Fig.Design Phase in the ADDIE Model

 Definition, Purpose and Types
Evaluation is the collection of, analysis and interpretation of information about
any aspect of a program of education or training as part of a recognized process of
judging its effectiveness, its efficiency and any other outcomes it may have.
Assessment
Assessment is therefore, a part of evaluation, which is a larger term. Difference
Between Assessment and Evaluation are as follows:
Dimension of Difference Assessment Evaluation
Assessment Evaluation
Content: timing, primary Formative: ongoing, to Summative: final, to gauge
purpose improve learning quality
Orientation

Orientation: focus of Process-oriented: how Product-oriented: whats

measurement learning is going been learned

Findings: uses thereof Diagnostic: identify areas for Judgmental: arrive at an

improvement overall grade/score
 Types of Evaluation
Based on the stage of the course at which it is
conducted:
1.Formative
2.Summative
3.Confirmative

Based on the purpose of evaluation:

1.Criteria-based
2.Norm-Based
 Formative Evaluation
When the cook tastes the soup, thats formative; when the guests
taste the soup,thats summative.

What It Is
1. Any evaluation conducted before the course to identify the
strengths and weakness of the course is called formative evaluation.
2. It provides feedback to the instructional designer about the
likelihood of the course achieving its objectives. The instruments
used here are SME reviews, dry runs with a sample group of
learners ,expert observation of pilots or drafts and field trials of
prototype
 Summative Evaluation
1.Evaluation conducted at the end of course to measure the
degree to which the course has achieved its objectives is
called summative evaluation.
2.In other words, summative evaluation checks whether
learners have acquired the skills and knowledge that the
course aimed to develop.
3.The instruments used in this are either written tests or case
studies.
4.Summative evaluation forms the basis for deciding whether
the course should be used again on a large scale.
 Confirmative Evaluation
What It Is:

Evaluation conducted a long time after implementation of the

course, to assess whether the course has been able to make
learners perform and retain knowledge and skills in the long run
is confirmative evaluation,
 Criteria-Based and Norm-Based
Evaluation
Parameter Criteria-Based evaluation
Purpose Whether learners have learned the
required skills and what they can do
after the course.
Score P to some performance of the all
Interpretation learners is compared external, specific,
and predefined benchmark. Learners
either achieve that standards or not.
Scope Tests for microskill areas,for example ,a
programmers ability to create error-
free modules or function
Norm-Based evaluation
Differentiate between learners
and separate the high
performers from the low
performer.
Score of such learners
compared with the scores of
other learners.How well a
learner does in the course
depends on the performance
of other learners in the same
course.
Tests for macroskill areas,for
example ,a programmers
proficiency in C++
programming
Output Scores are absolute
numbers
Questions Questions for one
particular skill are all at the
same level of difficulty
Scores are relative, in the
form of a grade or
percentile. Merit based
listing takes place.
Questions for the same skill
can be at different levels of
difficulty

Common Forms An Aptitude test or ,English The CGPA System,

vocabulary Test classroom test,GRE,SAT
 Key Decisions for Designing Tests
Each type of evaluation requires the instructional designer make
the following decisions before the actual test can be designed:

1.Purpose of the test

2.What the test should measure
3.Test Frequency
4.Time available to learners for taking the tests
5. Use of optional question
6.Allowing the repeat attempts
7.Hamdling technical problems
8.Validity and reliability of Tests
 Types of Tests Items
Some common Types of Tests are:

1.Multiple-choice
2.True/False
3.Short-answer or Text-Input
4.Matching-list
5.Simulation
6.Fill-in-the-blank
7.Eassay-type
8.Sequencing
 Parts of a Multiple Choice Question
1.Question Item
A question item is the problem statement or the body of the question that poses the
problem to the learner. The stem may be a complete sentence or a sentence
completed by the question options. For Example:

Which of the following is the largest continent in the world?

i)Asia ii)Africa iii)Australia iv)Europe

2.Keys
The test item can have one or more correct answers; these are called the keys.
They may be numbers ,full sentences, incomplete sentences, phrases or, terms.
As Given above example key or the correct answer is:
Asia

3.Distractors
The incorrect option for the question are the distractors, As given above example
distractors or the incorrect options are:
Australia,Africa,Europe
The keys and distractors are together called the option of a question.
 Variation of Forms
Multiple-choice questions can be presented in the following forms
as well:
1.Common contents as base
2.Common set of options
3.Items in analogy form
 True/False Questions
These questions present learners with a fact or statement in the
question stem.

For Example:
The colour black is a good conductor of Heat.
i)True ii)False

An average female requires 1,500 calories per day.

i)True ii)False

Variation of Forms
True/False questions need not always have true and false as options. Sometimes,the in
structional designers uses two opposite as the two possible answers to a question:,one of
which would be correct and other incorrect:
Lions are:
1.Herbivorous 2.Carnivorous
 Matching-List Questions
Some situation in which matching-List questions would be useful
are:
Matching terms with definitions
Matching words with opposites
Matching Questions with answers
Matching Symbols with meanings

Variations of Forms
1.Click-in-picture
The description for the items are given in the text form and learners
are asked to select a pictorial option by clicking it.
2.Drag-and-drop question
Drag-and-drop question ask learners to select visuals and drop them
into specific location.
 Fill-in-the-Blank Questions
These questions require learners to complete a statement or a table by supplying a
missing word or a item.

Example:
The__________ option of word 2000 aloows you to change the case of selected text.
 Sequencing Questions
Sequencing questions have an activity or a concept described in steps,and learners are
required to arrange the option in logical sequence.
Example:
Sequence the various planets in order of their distance from the sun.
1.Mercury
2.Uranus
3.Jupitor
4.Earth
5.Pluto
Variation of Forms
Sequencing questions can be given as a variation of matching-list question. Example:
Column A Column B

Step 1 Create a key frame in the Over frame

Step2 Create a new button symbol for the menu item

Step3 Fill the rectangle with a different colour for the highlight colour of
menu items
Step4 Draw a filled rectangle in the Up frame of the button symbols timeline.
 Essay-Type Questions
Questions that are requires learners to apply their knowledge of a particular concept are
called essay-type Questions.
Example:
What were the major causes of the French revolution?
 Guideline for Writing Test Items
1.Multiple-choice Question
Follow the Guideline for writing the different parts of multiple-choice question:
Writing stems
-Use Standard terminology
-Keep the stem concise
-Avoid Giveaways
-Do not make the stem Negative

Writing Options
-Avoid Giveaway
-Keep Options parallel in form
-Keep Options Concise
-Have an optimum Number of options
 2.True/False Questions
The guidelines for creating true/false questions are given as follows:
1.Prevent Guessing:
Ask more than one true/false question on a subject to check whether learners
have understood the key concepts taught in the course. Present a series of
True/False items to reduce guesswork by learners and add complexity to the test.

3.Matching List Questions

Guidelines:
1.Make the question easy to read and to attempt.
2.Avoid Giveaways
3.Guidelines for other forms of Matching-List Questions
4.Click-in-picture Questions
5.Drag and Drop Question.
4.Simulation Questions
Guidelines:
Make it easy to interpret.
Keep optimum Number of steps

5.Fill-in-the-Blank Questions
Guidelines:
Blank should preferably be at the end of the stem, instead of at
the beginning or in the middle.
 Writing Feedback and Grading
Feedback-
Feedback is the mechanism that informs learners whether their response to aparticular
question was correct or in correct.

Types of Feedback:
1.Prescriptive
2.Diagnostic

Grading-
Assigning scores or categories to learners based on their responses to
questions in the tests is called Grading.
Advantages-
1.Scores/Grades can be provided immediately to the learner.
2.The instructional designer can track learners performance and provide immediate
feedback .
3.The procedure of grading is objective and unbiased since a computer evaluates
responses.
 Key Decisions: A Case Study
Table presents the decisions and rationale for the decisions:
 Test Items
Summative Evaluation:
Sample Questions for Summative Evaluation
Thank You!