This document contains both information and form fields.

To read information, use the Down Arrow from a form field.

Ontario Adult Literacy Curriculum Framework

How To Use the

Checklist for Evaluating
Learning Materials
Ontario Ministry of Training, Colleges and Universities
March 2011

O A L C F | How to Use the Checklist for Evaluating Learning Materials

Contents
What is this document about? ...................................................................................................................1
Introducing the Checklist for Evaluating Learning Materials .................................................................1
How was the Checklist for Evaluating Learning Materials developed? .....................................................1
How is the Checklist organized? ...............................................................................................................2
Examining the Checklist in detail ...............................................................................................................3
Describing the Quality of Content section............................................................................................3
Describing the Potential Effectiveness as a Teaching-Learning Tool section .....................................3
Describing the Ease of Use (for Practitioners and Learners) section ..................................................4
Describing the Comments section .......................................................................................................4
The Checklist in Action: Kitchen Math Example ......................................................................................5
Filling out the Content section of the checklist ........................................................................................5
Answering the evaluative questions and providing comments ..................................................................5
Checklist for Evaluating Learning Materials .......................................................................................... 19

O A L C F | How to Use the Checklist for Evaluating Learning Materials

What is this document about?

How to Use the Checklist for Evaluating Learning Materials

introduces you to the Checklist for Evaluating Learning Materials by

o

providing information on its purpose and development

detailing the six Ontario Adult Literacy Curriculum Framework (OALCF) principles on
which it is based

describing the checklists five sections

illustrates how to use the checklist by evaluating an actual learning material

provides a blank checklist for evaluating your own learning materials

Introducing the Checklist for Evaluating Learning

Materials
With the implementation of the new Ontario Adult Literacy Curriculum (OALCF), practitioners in the
Literacy and Basic Skills (LBS) Program will need to evaluate the extent to which their program practices
support adult learners in a task-based and goal-directed learning environment. Specifically, as a volunteer
tutor, instructor, or program coordinator, you will need to evaluate your learning materials to determine
their effectiveness within the OALCF.
To help you in your work, the Checklist for Evaluating Learning Materials has been developed. The
checklist provides a set of standard questions to evaluate learning materialsensuring greater quality
and consistency of programming and a competency- and goal-directed program approach. You can use
the checklist to evaluate both print and non-print learning materials, such as books, workbooks, video
collections, software, and websites.

How was the Checklist for Evaluating Learning Materials developed?

The following six principles guided the development of the OALCF:
1. Address learner needs first and foremost. The Checklist for Evaluating Learning Materials
identifies materials according to the learners culture, language, and goal path. Using the
checklist will enable you to assess the match between the learning material and learners needs,
interests, goal, and culture helping to ensure that your programming is learner-centered,
relevant, and meaningful.
2. Make practitioners jobs easier. The Checklist for Evaluating Learning Materials will display the
results of your evaluation in an easy-to-read format. Then, both you and the learners in your
program can easily find learning materials that best match their needs.
3. Be appropriate for, and accessible to, learners and practitioners with diverse cultural needs. The
Checklist for Evaluating Learning Materials asks you to identify the cultural group of the target
audience for the learning material. In this way, you can easily find resources reflecting the cultural
values and language of learners.

O A L C F | How to Use the Checklist for Evaluating Learning Materials

4. Be accountable to the public. The Checklist for Evaluating Learning Materials helps you evaluate
a learning material that may already be in your delivery agencys collection or may be a resource
that your agency intends to buy.
5. Be built on a specified, strong theoretical foundation consistent with andragogy. The Checklist for
Evaluating Learning Materials uses evaluative questions that directly reflect adult learning
principles. Specifically, the questions take into account that learners

are self-directed

want to be engaged in decisions affecting them

have life experiences and knowledge

are goal-oriented

must see the reason for learning something

are practical

need to be treated with respect

6. Be clear to learners, practitioners, the Ministry of Training, Colleges and Universities (MTCU),
and other stakeholders. The Checklist for Evaluating Learning Materials makes explicit the
connection between a particular learning material and the OALCFs competency- and task-based
program approach.
The development of the checklist was informed by various educational and institutional sources.
Specifically, the evaluative questions and the categories into which they are grouped were adapted from
MERLOT-Multimedia Education Resource for Learning and Online Teaching. You can access this
resource at http://taste.merlot.org/evaluationcriteria.html.
To further understand the key concepts and rationale that informed the development of the Checklist for
Evaluating Learning Materials, please read Foundations of Learning Materials
(http://www.tcu.gov.on.ca/eng/eopg/publications/OALCF_Foundations_of_Learning_Materials_March_20
11.pdf), available from the Linking Learning Materials to the OALCF web page of the OALCF website.

How is the Checklist organized?

The Checklist for Evaluating Learning Materials is comprised of the following five sections:
1. Content
2. Quality of Content
3. Potential Effectiveness as a Teaching-Learning Tool
4. Ease of Use (for Practitioners and Learners)
5. Comments
The blank checklist at the back of this document shows the five sections.

O A L C F | How to Use the Checklist for Evaluating Learning Materials

Examining the Checklist in detail

The first part of the checklist is the Content section. Here you identify the name of the learning material
and its cultural orientation, the goal path to which it relates, and the predominant and secondary
competencies it supports. You also indicate the date of your evaluation.
The Content section is followed by three sections, Quality of Content, Potential Effectiveness as a
Learning-Teaching Tool, and Ease of Use (for Practitioners and Learners). These three categories
organize the 13 evaluative questions that further determine the fit of a learning material within the
OALCF. Taken as a whole, the questions help to ensure that the learning material matches the OALCFs
learner-centred, goal-directed, competency-based, and task-based approach to programming.
To answer each evaluative question, you will use the scale shown in Figure 1 to rate the learning material
from low to high. Low indicates a not at all response to an evaluative question and high indicates a
most definitely response.

Low

High

Figure 1: Rating scale for use with the evaluative questions

Describing the Quality of Content section

In the Quality of Content section of the checklist, you answer four evaluative questions to determine the
extent to which the learning material is adult appropriate, has clear objectives, includes current, relevant,
and accurate content, and is appropriate for the targeted cultural group or delivery sector. You then use
the boxes on the right side of the checklist to rate your response to each question from low to high.
For example, to answer the question Is the material appropriate for adults?, some qualities that would
inform a high rating are as follows:

The subject is presented in a mature format

Topics are of interest to adults

Materials are designed so that learners can immediately apply the skills, knowledge, and
behaviours they have acquired in class

Conversely, some qualities that would inform a low rating are as follows:

The material is presented with childrens needs and interests in mind

The activities consist of drill and kill exercises without providing any meaningful and relevant
context

Topics do not reflect an adult lifestyle

Describing the Potential Effectiveness as a Teaching-Learning Tool section

In the Potential Effectiveness as a Teaching-Learning Tool section, you answer five evaluative
questions to determine the extent to which the learning material can be adapted, satisfies different
teaching and learning styles, reflects current, accepted methodology, and enables you to deliver task-

O A L C F | How to Use the Checklist for Evaluating Learning Materials

based learning that meets individual needs. You use the boxes on the right side of the checklist to rate
your response to each.
For example, to answer the question Does the material present options for meeting individual needs?,
some qualities that would inform a high rating are as follows:

The material can be adapted to respond to the needs, interests, and goal of the learner

Suggested out-of-class activities enable learners to apply what they have learned in authentic
situations

Conversely, some qualities that would inform a low rating are as follows:

The material provides no opportunity for learners to apply their newly acquired abilities in real-life
situations

Adaptation of the material is challenging

Describing the Ease of Use (for Practitioners and Learners) section

In the Ease of Use (for Practitioners and Learners) section, you answer four evaluative questions to
determine the extent to which the learning material allows you to easily identify OALCF competencies and
task groups, presents the information attractively, and supports flexibility and self-directed learning.
Again, you rate your response to each question from low to high.
For example, to answer the question Does the material present information is appealing ways?, some
qualities that would inform a high rating are as follows:

The layout of the material looks uncluttered; therefore it is not intimidating

The use of visuals breaks up the text and helps the reader to understand it

Conversely, some qualities that would inform a low rating are as follows:

The text is very dense, making the materials look cluttered

The type is very small and difficult to read for some learners

Describing the Comments section

In the Comments section, you have an opportunity to provide more detailed information about the
learning materialinformation that might help other practitioners or learners evaluate and select
appropriate learning materials.

O A L C F | How to Use the Checklist for Evaluating Learning Materials

The Checklist in Action: Kitchen Math Example

To help you understand how to use the Checklist for Evaluating Learning Materials, an evaluator at an
LBS delivery agency has used it to evaluate Kitchen Math, an actual learning material. Developed by Lisa
Campbell of the Northwest Territories Literacy Council in 2009, the Kitchen Math workbook is one of three
in the Everyday Math Skills series. It is available for download at NALD (http://library.nald.ca/item/8508).
We will take you step-by-step through the evaluation process.

Filling out the Content section of the checklist

Designed for anglophone and Native learners in Northern Ontario, Kitchen Math is used in communitybased delivery agencies. Included in the workbook are activities to develop learners skills, knowledge,
and behaviours to plan, select, purchase, and prepare food for meals replicating the real-life tasks of
feeding a family. These tasks relate primarily to competency C. Understand and Use Numbers. However,
the tasks also relate to the two competencies Find and Use Information and Communicate Ideas and
Information. The workbook uses authentic examples to provide a basis for learning and practicing basic
math skills and includes activities outside the classroom to further contextualize learning.

Answering the evaluative questions and providing comments

The following pages illustrate how the evaluator answered the 13 evaluative questions for Kitchen Math.
For each question, we have provided the evaluators rating as well as descriptions to anchor the low to
high rating scale for the response. The Evaluators comments have also been included.

O A L C F | How to Use the Checklist for Evaluating Learning Materials

Quality of Content

Is the material appropriate for adults?

You would give the learning material a high rating if

the subject was of interest to adults

the topic was relevant to activities in which adults would participate

basic skills were presented in a mature format; no child-like images or concepts were present

learners could immediately apply the skills they had learned

You would give the learning material a low rating if

the subject matter was presented with childrens interests, needs, and level of understanding in
mind

the topics were presented without context. For example, they contained isolated multiplication
drills and spelling lists.

the material indicated a grade level directly, such as a Grade 3 level

Kitchen Math

Is the material appropriate for adults?

Low

High

Evaluators Comments

Adults who would use this material are probably interested in nutrition and in
learning the skills necessary to plan meals.

The topic is relevant to adults since they need to prepare food for themselves
and their families.

Lessons and activities generally pertain to a typical adult lifestyle and most can
be applied immediately.

O A L C F | How to Use the Checklist for Evaluating Learning Materials

Quality of Content

Are the learning objectives clear?

You would give the learning material a high rating if

the learning goals were clearly stated before each unit or lesson, as in By the end of this lesson,
learners will be able to . . .

an obvious connection existed between the material being taught, the reasons for teaching it, and
the gains learners could make

learners could understand and state what they were expected to learn

You would give the learning material a low rating if

learners would be unclear about what they were supposed to be learning and why

the materials had no reasonable pattern or flow

Kitchen Math

Are the learning objectives clear?

Low

High

Evaluators Comments

Adults who would use this material are probably interested in nutrition and in
learning the skills necessary to plan meals.

The topic is relevant to adults since they need to prepare food for themselves
and their families.

Lessons and activities generally pertain to a typical adult lifestyle and most can
be applied immediately.

O A L C F | How to Use the Checklist for Evaluating Learning Materials

Quality of Content

Is the content current, relevant, balanced, and accurate?

You would give the learning material a high rating if

the themes, messages, and concepts accurately reflected current social and economic realities

learners could see themselves, their values, and their realities in the themes, messages, and
concepts that supported learning

ideas were presented without bias

factual material was correct

what was being taught was something learners could use

You would give the learning material a low rating if

the themes, topics, and messages were outdated and did not reflect current beliefs, social
structures, or realities

obvious biases existed

the material had mistakes

Kitchen Math

Is the content current, relevant, balanced, and accurate?

Low

High

Evaluators Comments

Nutrition is an important consideration in this workbook, reflecting current

global health concerns.

Learners are introduced to modern information gathering and sorting tools for
good health, such as the Body Mass Index (BMI) and food labels.

The worksheet topics are timely, accurate, and reflect the interests and
realities of individuals who live in the North.

Prices listed accurately reflect food prices in the North.

Recipes are provided and learners are encouraged to use them at home.

The material has more drills than task-oriented lessons.

O A L C F | How to Use the Checklist for Evaluating Learning Materials

Quality of Content

Is the content appropriate to the needs of the cultural

group or delivery sector?
You would give the learning material a high rating if

the subject matter and the way it was presented were sensitive to the socio-economic, cultural,
and linguistic differences of learners

concepts and messages were addressed with specific traditions or beliefs in mind

learners could relate to what was being taught and the vehicle for teaching it

You would give the learning material a low rating if

learners could not relate to or understand the subject matter

obvious cultural biases existed

learners traditions and cultural norms were misrepresented

Kitchen Math

Is the content appropriate to the needs of the cultural group or

delivery sector?

Low

High

Evaluators Comments

The material introduces the concept of nutrition, an important consideration for

all individuals, regardless of their cultural or linguistic backgrounds.

Marginalized groups are at greater risk for poor health, so information to

improve their health is advantageous.

Recipes make reference to a typical Northern diet.

Much of the content in the first section was adapted from an anglophone
resource; therefore, concepts and references within it reflect the anglophone
culture.

O A L C F | How to Use the Checklist for Evaluating Learning Materials

Potential Effectiveness as a Teaching-Learning Tool

Does the material present opportunities for task-based

learning?
You would give the learning material a high rating if

skills were developed and taught as they related to the tasks the learner was required to
complete

the material emphasized completing tasks rather than on completing skills and drills or
completing worksheets

the tasks that learners completed reflected their immediate needs

You would give the learning material a low rating if

the material developed skills, but indicated no purpose for using them

skill development was mostly in the form of drills, repetitive worksheets, or cloze exercises

learners could not apply the skills they were learning in their lives

Kitchen Math

Does the material present opportunities for task-based learning?

Low

High

Evaluators Comments

Kitchen Math develops basic math skills by applying them to the tasks involved
in meal preparationtasks that learners completed outside the class.

Learners are presented with opportunities to learn about related tasks that
involve more than just math skills, such as making a shopping list, planning a
workshop, or searching for food information on the Internet.

Kitchen Math is a workbook that requires a great deal of pen- and-paper work;
however, the author includes extended learning activities that are task-based.

10

O A L C F | How to Use the Checklist for Evaluating Learning Materials

Potential Effectiveness as a Teaching-Learning Tool

Does the material present options for meeting individual

needs?
You would give the learning material a high rating if

the material provided suggestions for alternate activities or approaches

lessons were set up so learners could advance at their own pace, even if they skipped ahead or
spent more time on a particular unit

You would give the learning material a low rating if

the material was difficult to modify

no opportunities existed for learners to apply what they had learned in authentic situations

Kitchen Math

Does the material present options for meeting individual needs?

Low

High

Evaluators Comments

Kitchen Math is predominantly a pen-and-paper activity workbook; however,

opportunities exist for learners to apply the skills they are learning in practical
situations that may be more suited to their individual needs.

The material is somewhat personalized and could be altered by the

practitioner.

The material provides opportunities for discussion, as the topics are timely.

Practitioners can pick and choose activities or worksheets that best suit the
needs and skills of the individual learner.

11

O A L C F | How to Use the Checklist for Evaluating Learning Materials

Potential Effectiveness as a Teaching-Learning Tool

Does the material satisfy the various teaching and learning

styles?
You would give the learning material a high rating if

lessons or tasks could be modified to address learners individual needs. For example, the
practitioner or another learner could read the directions aloud.

lessons used multiple approaches to accommodate various learning styles

You would give the learning material a low rating if

lessons were presented in only one way, for example, with only written instructions

the material did not include a visual or an audio component

Kitchen Math

Does the material satisfy the various teaching and learning styles?

Low

High

Evaluators Comments

The material is presented in only a couple of formats; it uses only text and
graphics.

A practitioner or classmate can act as a scribe and help a learner complete the
written portions orally.

12

O A L C F | How to Use the Checklist for Evaluating Learning Materials

Potential Effectiveness as a Teaching-Learning Tool

Can the material be adapted to meet the needs of the

targeted cultural group or delivery sector?
You would give the learning material a high rating if

lessons or tasks could be modified to address the social and economic differences of learners

lessons or tasks could be changed to respond to the cultural and linguistic differences of learners

You would give the learning material a low rating if

practitioners would have difficulty modifying lessons or tasks to respond to the social and
economic differences of learners

lessons or tasks could not be changed easily to respond to the cultural and linguistic differences
of learners

Kitchen Math

Can the material be adapted to meet the needs of the targeted

cultural group or delivery sector?

Low

High

Evaluators Comments

Kitchen Math is predominantly a pen-and-paper workbook; therefore,

alterations to content are limited to the amount of changes the practitioner can
or wants to make.

A learner who is not comfortable with pen-and-paper activities would be

challenged by this material.

13

O A L C F | How to Use the Checklist for Evaluating Learning Materials

Potential Effectiveness as a Teaching-Learning Tool

Does the content reflect current and accepted

methodology?
You would give the learning material a high rating if

lessons reflected the various notions about how adults learned

lessons were developed with learner input or were learner-directed

lessons avoided methods, such as cloze exercises, that did not match activities that would be
carried out beyond the classroom

lessons relied on the learners prior knowledge for building new skills

You would give the learning material a low rating if

the material did not reflect current practices and concepts associated with adult learning

lessons did not take into consideration the learners prior knowledge or skills

Kitchen Math

Does the content reflect current and accepted methodology?

Low

High

Evaluators Comments

Kitchen Math is mainly a pen-and-paper workbook, a format that is usually less

effective for adult learners.

The workbook author has not recognized the importance of setting up activities
outside the classroom to practice newly learned skills and to allow learners to
take charge of their learning.

14

O A L C F | How to Use the Checklist for Evaluating Learning Materials

Ease of Use (for Practitioners and Learners)

Are OALCF competencies and task groups easily

identified?
You would give the learning material a high rating if

OALCF competencies and task groups could be clearly identified

the competencies and task groups of the OALCF were stated

You would give the learning material a low rating if

it was unclear what competencies and task groups could be aligned with the learning materials

the competencies and task groups of the OALCF were not stated

Kitchen Math

Does the content reflect current and accepted methodology?

Low

High

Evaluators Comments

The title of the book clearly indicates that it will be dealing with numbers, so it
helps learner work with OALCF competency C. Understand and Use Numbers
and its task groups.

After reviewing the material, it is obvious that the learning material helps to
develop other competencies.

15

O A L C F | How to Use the Checklist for Evaluating Learning Materials

Ease of Use (for Practitioners and Learners)

Does the material present information in appealing ways?

You would give the learning material a high rating if

the material was attractive and uncluttered

the material looked easy to read and understand

topics were current and of interest to adults

materials included a variety of activities that challenged the learner

You would give the learning material a low rating if

adults did not enjoy working with the material since it was unattractive and cluttered

the text did not look easy to read and understand, making it appear intimidating

Kitchen Math

Does the material present information in appealing ways?

Low

High

Evaluators Comments

Kitchen Math comes across visually as a typical workbook with lots of

instructions and drills.

The variety of ideas for activities outside the classroom is interesting and likely
to appeal to an adult audience.

16

O A L C F | How to Use the Checklist for Evaluating Learning Materials

Ease of Use (for Practitioners and Learners)

Does the material provide flexibility in its use?

You would give the learning material a high rating if

it did not prescribe an order for teaching

the material allowed the practitioner to use different media with the tasks

the material allowed the practitioner to adapt it to meet learners needs, interests, and goals

You would give the learning material a low rating if

the learner had to use the material from start to finish

the material could only be used in one format, such as paper

the context of the task could not be changed to meet the needs and goals of learners

Kitchen Math

Does the material provide flexibility in its use?

Low

High

Evaluators Comments

The pen-and-paper approach may limit some learners.

The materials provide learners with an opportunity for outside class activities to
practice what they have learned in class.

Learners or practitioners can pick and choose worksheets according to the

learners prior knowledge and skill levels.

The practitioner can use the worksheets as a guide to help create more
suitable, task-based activities, based on the learners needs and goal.

17

O A L C F | How to Use the Checklist for Evaluating Learning Materials

Ease of Use (for Practitioners and Learners)

Does the material support self-directed learning?

You would give the learning material a high rating if

the material supported learners working on their own

it provided a lot of opportunity for learner input

You would give the learning material a low rating if

learners had to rely on the instructor to complete the lessons

the material did not allow for learner input

Kitchen Math

Does the material support self-directed learning?

Low

High

Evaluators Comments

The Math Projects pages encourage learners to practice what they have
learned by completing activities outside the classroom.

Answer pages in the back of the workbook support learners working on their
own.

Learners do not have to rely on practitioner support to get through the

activities.

18

O A L C F | How to Use the Checklist for Evaluating Learning Materials

Checklist for Evaluating Learning Materials

Name of Learning Material

Date of Evaluation

1. Content
Anglophone

Deaf

Francophone

Native

Cultural Group

Employment

Apprenticeship

Secondary School
Credit

Postsecondary

Independence

Goal Path

A. Find and Use

Information

B. Communicate
Ideas and
Information

C. Understand
and Use
Numbers

D. Use Digital
Technology

E. Manage
Learning

F. Engage with
Others

Predominant
Competency
Secondary
Competencies

2. Quality of Content

Low

High

Is the material appropriate for adults?

Are the learning objectives clear?
Is the content current, relevant, and accurate?
Is the content appropriate to the needs of the targeted cultural group or
delivery sector?

19

O A L C F | How to Use the Checklist for Evaluating Learning Materials

3. Potential Effectiveness as a Teaching-Learning Tool

Low

High

Low

High

Does the material present opportunities for task-based learning?

Does the material present options for meeting individual needs?
Does the material satisfy the various teaching and learning styles?
Can the material be adapted to meet the needs of the targeted cultural
group or delivery sector?
Does the content reflect current and accepted methodology?

4. Ease of Use (for Practitioners and Learners)

Are OALCF competencies and task groups easily identified?
Does the material present information in appealing ways?
Does the material provide flexibility in its use?
Does the material support self-directed learning?

5. Comments

20