Welcome to the Open ECBCheck

(Version 2011/12)

The Open ECBCheck Quality Assessment Toolkit for Programmes and Courses allows to evalua
organization's eLearning offerings against internationally agreed quality standards. This toolkit is
sections, which are described below.

0. Basic Information

1. Instructions
2. Criteria Overview

3. Quality Toolkit

4. Results

5. Glossary of terms

The Quality criteria are updated regularly. Please send you feedback and suggestions to: info@ecbYour organization can join the Open ECBCheck Charta at www.ecb-check.org!

me to the Open ECBCheck

(Version 2011/12)

ment Toolkit for Programmes and Courses allows to evaluate your

ainst internationally agreed quality standards. This toolkit is composed of 5

Basic information on the programme/course that is

assessed and guidance to determine whether the criteria is
applicable to a course or a programme.
How to use this toolkit?
Overview of the criteria framework, which presents the
areas assessed, followed by explanations in each one of
them, and determines whether they represent "Minimum"
or "Excellence" criteria.
The practical working area to asses and rate courses
and/or programmes
This spreadsheet provides the overview with the results the
programme/course assessed has obtained.
Glossary of terms and concepts used in this toolkit

ed regularly. Please send you feedback and suggestions to: info@ecb-check.org

on can join the Open ECBCheck Charta at www.ecb-check.org!

0. Basic Information on Programme or Course

Please fill in the following information on the programme or course to be assessed with the Open ECBCheck.

This information will help you to understand whether the criteria for programmes or courses are appropriate (see 2. C
also be useful to identify the programme/course which has been assessed and who has assessed it.

Programme: refers to a training/learning offer that consists of several modules, which may be composed of a numbe
comprises a workload of at least 100 hours. The workload includes participation in tasks/activities; self study; prepara
whenever applicable; and assessments. At least 20% of the programme must be technology enhanced for it to fall wi
Open ECBCheck.

Course: refers to a short learning unit (less than 100 hours workload). Usually it is composed of only one unit, which
and covers one complete topic. At least 20% of the course must be technology enhanced for it to fall within the scope
ECBCheck.

1. Organization providing the programme/course

2. Title of the programme/course
3. Duration/Workload (h) and number of modules/lessons
4. Amount of learning time which is technology enhanced (in % of learning time)
5. Contact person(s) and contact detail (email/Tel/Fax)
6. Date of self-assessment
7. Staff involved in the self-assessment

The Quality criteria are updated annually. Please send you feedback and suggestions to: info@ecb-ch
Your organization can join the Open ECBCheck Charta at www.ecb-check.org!

on Programme or Course

ramme or course to be assessed with the Open ECBCheck.

her the criteria for programmes or courses are appropriate (see 2. Criteria Overview). It will
which has been assessed and who has assessed it.

at consists of several modules, which may be composed of a number of lessons, and

workload includes participation in tasks/activities; self study; preparation for assessments,
20% of the programme must be technology enhanced for it to fall within the scope of the

100 hours workload). Usually it is composed of only one unit, which takes several hours
e course must be technology enhanced for it to fall within the scope of the Open

y enhanced (in % of learning time)

nnually. Please send you feedback and suggestions to: info@ecb-check.org

an join the Open ECBCheck Charta at www.ecb-check.org!

Instructions - How to use the Open ECBCheck Quality Assessment Toolkit

The Open ECBCheck Quality Assessment Toolkit is a new accreditation and quality improvement scheme for eLearning programmes and institutions
in international Capacity Building. It supports you and your organization to measure how successful eLearning programmes/courses are and fosters
continuous improvement.
This toolkit allows to evaluate the quality of programs and/or courses (see Glossary for definition) using a predefined set of criteria. To evaluate the
quality of institutions a second toolkit is available.
The evaluation workspace can be found under section "3. Quality Toolkit". The criteria are divided into "Minimum" and "Excellence" Criteria (see
definition in Glossary). Minimum criteria are those which are absolutely necessary for a positive quality evaluation of your course and/or program.
Excellence criteria are those which characterize high achievements in the field of e-learning quality. (How were the "minimum" and "excellence" criteria
weighted and calibrated? The toolkit includes a predefined set of criteria selected by international experts - however, you can modify your own quality
profile by adjusting the Minimum and Excellence criteria according to your own requirements. Please note that if you would like to obtain the official
Open ECBCheck label the standard configuration of Minimum/Excellence criteria provided has to be used).
The evaluation is conducted in three phases:
Phase 1 - Evaluation Minimum Criteria: In the first phase only the minimum criteria are used to evaluate
the quality. If a criterion is met it is marked with "M" in the "Evaluation Results" Colum. If one or more
Minimum Criteria are not met, the program/course has to be improved.
Phase 2 - Evaluation Excellence Criteria: If all Minimum Criteria are met, the second phase is to evaluate the
Excellence Criteria. Excellence Criteria are rated with a scale from 0 = not realized up to 3 = realized
excellently in the column "Evaluation Results" as well.
Phase 3 - Results: The last phase is to analyze the results that the programme/course has achieved in the area
"Results". This overview allows you to identify areas for potential quality improvement for your programme/course
in the future.

Calibration
(Optional)

Phase 1

Use the standard weighting of Minimum/Excellence Criteria provided or change Minimum/Excellence rating according to your requirements.
The use of the standard weighting provided is required for obtaining the Open ECBCheck label.
1. Rate Minimum Criteria with "M" in the column "Criterion Level"
2. Rate Excellence Criteria with "E" in the column "Criterion Level"
Evaluation of Minimum Criteria
If the Minimum Criteria are met by a programme/course, indicate "YES" in the column "Evaluation Results", otherwise leave the field blank. In case
one or more of the Minimum Criteria are not met, the programme/course needs to be improved in the particular area before proceeding to the second
phase.

Phase 2

Evaluation of Excellence Criteria (only when ALL Minimum Criteria are met)
Rating of Excellence Criteria on a scale of 0 to 3
0 = not met
1 = partly met
2 = met adequately
3 = met excellently

Phase 3

Quality overview/Results - Section 4

In this section you will get an overview of the results the programme/course has obtained and how achievement of the Minimum and Excellence
criteria are distributed. In case the Minimum Criteria are met all totals are shown in green; if the Minimum Criteria are not met the totals are shown in
red. For the Excellence Criteria a sum of the overall rating is shown and a percentage of the maximum score is provided.

The Quality criteria are updated regularly. Please send you feedback and suggestions to: info@ecb-check.org
Your organization can join the Open ECBCheck Charta at www.ecb-check.org!

Criteria Overview
This section gives you an overview on the structure of the Open ECBCheck Quality Evaluation Toolkit for programmes/courses.
Areas
A Information about and organization of the programme
A.1 General description, objectives and programme organization
A.2 Technical and organizational requirements

Sum Minimum Excellence

4
2
6

4
2
6

0
0
0

4
4

3
3

1
1

6
1
4
4
2
5
1
Sum 23

4
1
2
3
2
3
0
15

2
0
2
1
0
2
1
8

4
4

4
4

0
0

Sum
B Target Audience Orientation
B Target Audience Orientation
Sum
C. Quality of Contents
C. Quality of Content

Sum
D. Programme/Course Design
D.1 Learning Design and Methodology
D.2 Motivation
D.3 Learning Materials
D.4 eTutoring
D.5 Collaborative Learning
D.6 Assignments & Learning Progress
D.7 Assessment & Tests
E Media Design
E. Media Design
Sum
F Technology

F. Technology

4
4

4
4

0
0

4
Sum 4
Total 51

4
4
38

0
0
13

Sum
G Evaluation & Review

G.Evaluation & Review

The Quality criteria are updated regularly. Please send you feedback and suggestions to: info@ecb-check.org
Your organization can join the Open ECBCheck Charta at www.ecb-check.org!

oolkit for programmes/courses.

Explanation of the Areas
A Information about and organization of the eLearning programme
The area "Information about and organization of the eLearning program" is
concerned about the basic information provided to learners about the
programme. It also considers the general organization of the programme and
the
qualification
of staff.
B Target
Audience
Orientation
"Target Audience Orientation" highlights the importance to include the learners
perspective at all times, while developing, running and evaluating/ improving a
programme.
C Quality of the Content
The area "Quality of the Content" is concerned with the quality of the materials
provided. It also considers aspects of gender and cultural diversity.

D. Programme/Course Design
Programme/Course Design focuses on the quality of the learning experience
based on: the overall course layout and presentation; instructional design
(learning strategies, learning objectives); learning materials to support learners;
the quality of eTutoring (learner support) as well as how assignments and
knowledge assessment tests are designed and used.

E Media Design
"Media Design" focuses on accessibility and usability of the media provided to
learners
F Technology

The area "Technology: Equipment & Infrastructure" considers how technology is

supporting an optimal learning experience. It also considers the reliability of all
systems.
G Evaluation & Review
This section includes the elements that need to be considered in a programme
evaluation in order to ensure continuous improvement. The programme
evaluation has to take into consideration a number of factors such as
programme design, programme management, learning materials, methodology,
e-tutoring and performance evaluation.

This colum tells you if a criterion is

applicable for Programs (P) or
Programs and Courses (P/C). If you
are evaluating a Course, you do not
need to cover the criteria that are
only marked by P.

In this colum you will find all

Open ECBCheck Quality
Criteria sorted in different
categories

Criterion

programme (P)
and/or Course
(C) related

Description

This colum provides a more

detailed description for each
criterion to make the criteria
more clear

This colum should assist

you in evaluating the
criteria. It provides
statements, hints and
questions to help you to
find relevant information
for the evaluation of
each criterion.

P/C

A.1.3 The methodological approach is clearly

explained. It presents how the
methods/approaches selected lead to the
achievement of the learning objectives. Learners
are able to understand the added value of
technology enhanced learning.

P/C

A.1.4 Information on contact persons is available,

mentioning name, area of responsibility, and
contact details.

P/C

A.2 Organizational and technical requirements

For Minimum Criteria: Please put an "M"

into the field if a criterion is met. If not,
please leave the field blank.

Evaluation Results

Minimum criteria:
"YES" if criterion is met. It not
met leave the field blank
Guidance on how to evaluate

A Information About and Organization of the programme

A.1 General Description, Objectives and Programme Organization
A.1.1 All information potential learners need to
Objective and clear information is available to Does the description of the programme/course
orient their decision about taking the course is
orient/facilitate decision making of interested
(brochure/flyer/webpage etc.) provide all
available.
candidates: Course/programme goals and
relevant information for potential learners?
overall learning objectives; methodological
approach; target group; prerequisites for
participation; selection process (if applicable);
technical requirements; and fees . Information
on knowledge assessment tests, timetable and
workload is also available. A calendar/timetable
details duration of learning phases (or
P/C
modules) and dates of assessments.
Whenever applicable such information may be
divided to be provided before (information that
orients candidates decision-making) and after
the registration (information that further details
course/programme organization).

A.1.2 Learning objectives state clearly the

expected improved competency. All learning
objectives are measurable (e.g. stated according
to the SMART criteria).

This colum is where the evaluation

results can be documented.

Learning objectives are statements about the

outcomes of a training course. They provide
information about the knowledge and skills the
learners will acquire upon completion of the
course. Learning objectives are formulated
from the learners' perspective; verbs used
should reflect the level of performance learners
will achieve. They correspond to specific
competencies. They must be verifiable and
measurable, thus are stated with action verbs.

Do the learning objectives clearly state the

purpose of the course with measurable
elements? Are the objectives defined on the
learners' perspectives? In case the
programme/course is divided into several
modules/sections do these modules/sections
have specific learning objectives? Are these
objectives related to the overall
programme/course learning objectives? Is the
course/programme content in line with the
learning objectives? Are the learning objectives
stated in a clear and objective manner, allowing
adequate understanding by learners?

Learning methods and tools and their

contribution to learning objectives achievement
are explained. Potential learners are informed
about the contribution technology-enhanced
learning provides for achieving the learning
objectives. The forms/types of technology
enhanced learning utilized are made clear to
learners.

The description of the programme/course

(flyer/brochure/webpage etc.) contains
information on:
- methodological approach, including eventual
utilization of collaborative elements
- type of media used (text, animation, sound,
videos, etc.)
- brief explanation contextualizing the use of
technology-enhanced learning.

Learners know how to reach contact persons to

obtain information and/or assistance about
technical issues, methodological advice, etc.
Contact information (phone, email) is clearly
informed.

The programme/course documentation

(syllabus, flyer, leaflet) contains contact details
of the person responsible for learner advice.
Throughout the programme/course learners
have access to people responsible for:
- student advice on methodological/pedagogical
issues
- technical matters
- eTutors
- complaints

Criterion
Level

Excellence Criteria:
0 = not met
1 = partly met
2 = met adequately
3 = met excellently

For Excellence Criteria, please indicate

each criterion on the scale from 0 = not
realised up to 3, realised excellently

Comments

Please fill in any comments

regarding the self-assessment
of the criteria that may help
the peer-reviewer to
understand your rating.

Documentation

Programme/course description (printed

or digital)

Programme description (printed or

digital)
List of learning objectives

Programme description (printed or

digital)

Programme description, brochures,

webpages or other documents where
contact persons are listed.

A.2.1 People responsible for the analysis, design,

development, implementation, and evaluation of
the programme (internal staff/external experts)
are adequately qualified.

There are clear defined roles related to

administration, teaching responsibilities,
monitoring and evaluation of the programme.

Is there a document that identifies individuals

roles and specifies responsibilities of each
individual? Is there a particular document
stating the relevant expertise of the subject
Possible roles may include: programme
matter experts, e-tutors/facilitators, regarding
manager, learning designer, subject matter
their relevant professional competence and
expert (authors), technical support, e-tutors, e- experience to participate in programme?
moderators, quality assurance managers.

Document that defines the roles,

responsibilities and skill requirements
of staff and describes the roles,
qualifications and experience of staff
involved.
M

The professional profile of individuals involved

takes into account roles and responsibilities to
assure the quality and value of the
programme/course.
A.2.2 Technical requirements necessary to allow
for adequate participation in the
programme/course are clearly described.
P/C

B Target group Orientation

B.1 Target group's learning needs are taken into
account and considered in the design of the
programme/course (curriculum, methodology).

P/C

B.2 The programme workload and schedule are

consistent with the target groups requirements.

Technical requirements that should be indicated Does the general information about the
as a minimum: basic computer configuration; programme/course general inform learners of
operating system; internet access; browser;
the minimum technical requirements so as they
plug-ins; any particular software; firewall
are able to assess whether they can fulfill those
access, etc.
requirements? Is the minimum Internet
connection speed specified? Are the firewall
access rules (if applicable) described?

The programme/course is aligned with the

Is there is a document describing the
learning needs of the specific target group. It
characteristics of the target group? Are these
takes into account professional and working
characteristics (prior learning , previous
context, previous experience or prior learning (if experience, computer literacy) taken into
applicable). It is designed in accordance with
account in the programme/course?
the learning characteristics of the target group. Does the programme/course provide any
It is presented on a flexible manner, allowing for instrument to personalize learning paths
self directed learning paths, analysis of learning according to individual needs/interests?
progress, and ability to identify learning gaps. Has a learning needs analysis been
conducted? Were the results considered in the
programme design?

Workload and schedule are consistent and

realistic regarding the curriculum and learning
objectives. Each learning objective is
contemplated with sufficient amount of
workload, without causing conflicts in the
programme' s overall schedule.

The programme should reflect learners'

Is there a process that enables stakeholders to
learning needs and respond to their specific
include their perspective in the programme
characteristics. Learners, teachers, tutors, and development process?
other concerned stakeholders should be
involved in the programme/course development
process.

B.3 Key stakeholders are involved in the

programme design through participatory process

B.4 Learners have access to counseling services

and advice both prior to the start of the
programme and during its implementation.

Learners benefit from tailored counseling and

advice, supporting their learning process
throughout the duration of the course.

Has the workload been planned and tested

according to the characteristics of the target
group, including full time job occupation?

B.6 A mechanism or system for complaints and

appeals is available to learners.
P

C Quality of the Content

Documentation of target group analysis

in combination with documentation on
learning design and methodology

Document providing an overview on

the workload/schedule/tasks.
M

Is three a person responsible for learners

counseling/advice? Is the learner aware these
options are available? Is there a fixed amount
of time allotted to provide advice to learners or
indication of office hours for this?

B.5 The programme foresees processes to bridge

learning deficits of low achievers

Programme description (printed or

digital)

Preparatory classes, additional learning

materials, recommended reading as well as
pedagogical guidance/advice and other forms
of support are available for learners facing
particular difficulties during the programme.

Do learners facing difficulties receive support

during the programme? Do learners receive
adequate orientation related to the
methodological approach to guarantee a
successful participation in the programme?
Does the programme monitor low achievers,
assisting/guiding them to facilitate attainment
of learning objectives?

There is explicit process to manage complaints

and appeals. The process is documented. This
criterion deals with the management of
complaints and appeals. It does not relate to
the mere existence of a possibility to complain.

Is there a certain process or institutional

arrangement to deal with complaints? Is there
a person responsible to deal with learners'
complaints related to the programme,
assessments, tutoring?

Here adjustments were made in the

description and questions to avoid
repetition with questions that were
already asked in the section
"Information". Thus, this criterion would
focus stress the aspects of counseling
and advice rather than general
information related to decision-making.
This is all covered in Section A.

Description of the services that are

offered

Document describing the appeal

process. If applicable, an example of a
complaint with explanations on how it
has been resolved.

C.1 The content of the course/ programme is

coherently presented and subdivided in logical
sequences of modules and/or lessons/sections,
organized in such a way that enables
comprehension and retention.

The content is developed in accordance with

the learning objectives. It is contextualized with
the objective of developing the competences of
the learner rather than dispensing new
information.
P/C

C.2 The contents are provided in a flexible

manner, allowing for different learning paths.

The content is provided in a manner which

allows also non linear ways of navigating
through the content. Learners can freely
navigate through the content, choosing their
learning path.

Does content provide the needed elements

(information and knowledge) that enable the
achievement of each of the learning objectives?
Is the amount of content provided adequately
balanced as regards cognitive load? Is the
content focused on developing the indicated
expected competencies or is it packed to
dispense information?

Is there more than one way to go through the

learning materials? Do learners have the
possibility to navigate freely in the course
according to their needs (e.g. review of past
units, advance to future ones, access to
additional learning resources)?

P/C

C.3 Content is gender sensitive. It takes into

account cultural diversity.

P/C

C.4 Media rich content is utilized exclusively with

a fixed and definite purpose.

The content is presented using a gender

Is the content of the programme/course gender
inclusive approach; there is no sexist language. inclusive? Does it consider and respect cultural
The programme also considers cultural
diversity?
diversity in the way the content is presented
and contextualized. Visual materials
(illustrations, images) reflect cultural diversity.

The utilization of media rich elements (audio,

Is the media utilized effectively supporting
video, hypertext, images, graphics) has the only learners' comprehension/understanding of the
purpose of enhancing and/or facilitating
content? Is the media used in a coherent and
learners' comprehension of the contents.
balanced way as regards the content they
support? Are images, graphics, illustrations
copyright free?
P/C

D Programme/ Course Design

D.1 Learning Design and Methodology

D.1.1 The blending of learning methods (online,
face-to-face, self-learning, tutor-facilitated,
asynchronous, synchronous) is adequate and
meets the needs of the learners.

D.1.2 A learner centered learning design

facilitates the development of the desired skills
and expected competencies described in the
learning objectives.
P/C

D.1.3 Social and collaborative activities are

included in the programme methodologies and
contribute to the achievement of the learning
objectives.

P/C

The utilization of blended approaches is

explained and incorporated in the programme
methodological approach. The utilization of
blended approaches takes into account
learners' needs and characteristics.

Is there a document describing how blended

learning methods are utilized and how they
support the pedagogical approach of the
programme?

Learning tasks and activities are coherent with

the learning objectives. When learners are
performing them, they are already practicing
the desired skills and expected competencies
as indicated in the learning objectives. Learning
methodology guides learners to achieve
expected learning objectives.

Are the learning design methodologies

adequate to the development of prescribed
skills and competencies? Are they adequately
described? Is there a clear alignment between
the learning tasks and activities and the kind of
learning objectives expected to be achieved?

Activities which promote interaction and

Are the activities promoting interaction and
collaborative work among learners are planned collaborative work among learners contributing
and organized as part of the programme/course to the achievement of learning objectives? Is
design. The learning design (learning
there a document describing how these social
objectives, methodologies, content, and
activities contribute to reach the learning
assessments) is adequate to meet learners'
objectives?
requirements.

Documentation on the learning design

and methodology of the programme
E

Documentation on the learning design

and methodology of the programme

Documentation on the learning design

and methodology of the programme
M

D.1.4 The learning methodologies offer

opportunities for learners to determine their
learning pace.
P/C

D.1.5 Through the learning activities learners are

encouraged to consider and use systems thinking
and to view issues from different perspectives.

D.1.6 The content of the programme is organized

in a logical sequence from the simpler to the most
complex concepts. The modules/lessons/units
build progressively on each other.

P/C

Deadlines are established, but learners can self Do learners have the possibility to navigate
control their path through the
freely through the course? Do they have access
programme/course. Learners can access the
to the learning materials at any time? Are
various resources in the sequence they choose learners motivated or guided to manage the
and complete the required tasks managing their learning activities according to their own
time individually.
learning pace?
The learning activities open up possibilities for Are there activities that foster systems thinking,
learners to appreciate things from different
to allow the analysis of topics under different
perspectives and to establish synergies and/or perspectives?
connections among different elements.
The learning units are progressively presented
in a logical sequence from the simplest and
most basic concepts.to the most complex ones.
Therefore, each unit builds upon the previous
one.

Are the basic concepts explained in the first

units? Do the basic concepts help the learners
understand the more complex concepts? Is
there a gradual level of complexity in the way
the content is presented?

The learning methodology contemplates

activities where learners are required to actively
contribute, for example, reflection, peer
feedback, group work, discussion boards.

Are there specific tasks and activities designed

to motivate learners to actively participate? Are
there collaborative activities that require
participation?
Do the collaborative or participatory tasks
designed take into account the characteristics
of the target group?

P/C

The programme offers a learning experience

which is relevant to professional practice
through including, amongst others, materials
such as case studies, practical examples, good
practices, and real-life examples.

Have realistic or real life case studies been

used?
Have practical examples been included in the
development/review of content and tasks?
Have external experts been involved in the
development of the programme to include
practical examples?

P/C

D.2 Motivation/ Participation

D.2.1 The learning methodologies motivate
learners to actively participate in the learning
process.

Documentation on the learning design

and methodology of the programme
M

Documentation on the learning design

and methodology of the programme
E

Documentation on the learning design

and methodology of the programme
M

Documentation on the learning design

and methodology of the programme
M

D.3 Learning Materials

D.3.1 The programme offers a learning
experience which is relevant to professional
practice.

D.3.2 Each learning unit is presented with the

elements needed to guide learners in achieving
the learning objectives.

P/C

D.3.3 Modules/lessons/units provide a glossary of

terms associated to the learning materials.
D.3.4 A distinction is made between compulsory
and recommended study/ reading materials. The
bibliography is commented.

P/C

P/C

Modules/lessons are introduced with their

Do the modules/lessons/units present the
specific learning objectives. Introductory
required elements for learners to have an
elements should serve both to motivate
overview of the particular module/lesson? Are
learners and to provide them an overview of the learners able to know from the start the specific
module/lesson. Introductory elements may be: learning objectives and the way they will be
- brief description of the module/lesson/unit
assessed regarding the achievement of the
- specific learning objectives for the
objectives?
module/lesson/unit
- estimate of the amount of time required to
complete the module/lesson/unit
- Outline of eventual knowledge assessment
tests/exams that will enable evaluation of the
achievement of the learning objectives

Learners have access to a glossary from any

part of the programme and/or through direct
hyperlinks from the content materials.

Is a glossary available? Do learners have easy

access to it?

The bibliography and the list of links is

commented. A distinction is made between
compulsory, and recommended readings and
additional resources/materials.

Are there comments on the list of links and

bibliography?
Is there a distinction between compulsory and
recommended materials?

Glossary
E
Bibliography
E

D.4 eTutoring
D.4.1 A tutor provides guidance and accompanies
the learners throughout the learning process.
P

D.4.2 A tutor provides feedback to learners on

tasks/activities learners are required to perform.
P

D.4.3 Tutors have specific knowledge and

competences to facilitate online
courses/programmes.

A tutor is available to guide learners through

Is there a tutor available during the
the learning process, helping them and
programme?
providing assistance with programme related
Is the tutor available to all learners, providing
questions/problems. Tutors can sometimes also support and guidance throughout the learning
provide technical support or this service can be process?
offered by another individual.
Is the tutor easily reachable by learners (e-mail,
forum, chat, phone)?
The feedback provided by the tutor is timely,
accurate, helpful for future learning and
targeted.

Does the tutor provide regular feedback to

learners? Does the tutor regularly
react/comment on learners tasks such as forum
posts, reflections, etc? Is the feedback provided
in a timely manner?

Tutors should be specifically trained on online Do tutors have specific e-tutoring/online

facilitation skills in order to adequately support facilitation skills? Are the tutors required to
learners in an online environment.
attend tutor training?
Do tutors have previous experience in etutoring tasks?

Evidence on the expertise and

experience of the tutors.
M

D.4.4 A set of predefined tutoring skills are being

used as a standard within the program.
P

Tutors are equipped with suitable skills for

tutoring which are documented as a skill
standard for tutoring.

Are tutors sensitized to specific aspects related

to e-tutoring responsibilities and tasks?
Has tutors' agreement on the basic e-tutors
skills been documented?

Documentation on applicable
standards.
E

D.5 Collaborative Learning

D.5.1 Collaborative learning is explicitly supported

D.5.2 Search functions are available for forums,

discussion boards or blogs, whenever such tools
are utilized.
D.6 Assignments & Learning Progress
D.6.1 Assignments are clearly formulated and
adequately explained to learners. Learners have
a clear understanding of what they are expected
to perform and how their performance will be
measured.

D.6.2 Assignments and knowledge assessment

tests are aligned with the learning objectives,
enabling adequate measurement of their
achievement.

P/C

P/C

D.6.3 Learners progress and achievements are

monitored and evaluated.

D.6.4 Assignments and/or knowledge

assessment tests and tasks are designed using
different approaches, including self-assessment
and peer-review.
P/C

D.6.5 Individual feedback is provided to learners

whenever a task requires problem solving skills.
The feedback has an analytical approach and
considers the way the solution was provided.

D.7 Assessment & Tests

D.7.1 A specific timeframe is foreseen for tutor's
to provide learners with feedback on assignments
and knowledge assessments tests. Learners are
informed of the timeframe.
P

E Media Design
E.1 Accessibility standards have been applied.
P/C

Collaborative learning is part of the pedagogical Does the programme contains specifically
model. It includes, for example, peer review,
designed group assignments, which require
group work, discussion boards, etc. Learners collaboration among learners?
are provided with adequate environment and
Does the learning environment offer
tools to take part in the collaborative activities collaboration tools such as forums, chats,
and tasks. Tutors/facilitators provide guidance blogs, to support collaborative tasks? Is the
to learners while performing such
learning environment supportive to collaborative
activities/tasks and feedback once they are
activities/tasks? Are collaborative tools such as
concluded.
forum,
chat,function
collaborative
working
spaces,
The
learning environment provides search
Is
a search
available?
Does
it allow for
document
sharing
available
functions to the collaborative tools utilized in
full text search
in tools
such to
aslearners?
blogs, forums,
the programme/course.

discussion boards?

Learners are provided with

guidelines/instructions that explain the task in
detail (what has to be done, timeframe and
what are the expected outputs). They are also
provided with instructions/guidelines on the way
they will be assessed.

Are there guidelines detailing the conditions of

the assignments? Are the assignments clearly
explained? Do learners understand what is
expected from them when performing the
assignments? Is the way learners will be
assessed clear for them?

Assignments and knowledge assessment tests

are designed in accordance with the learning
objectives. Assignments and/or assessments
have measurable elements and can
demonstrate whether or not learning objectives
have been met.

Do the assignments and knowledge

assessment tests provide elements to measure
whether or not learning objectives have been
met?

Learners progress is regularly monitored with

the objective to identify learners that might
need special attention or need special support.
Monitoring of the learning process also allows
specifically the identification of learners who
may require particular attention to attain
learning objectives.

Is there a mechanism to identify learner with

low performance?
Are there possibilities for self-assessment and
assessment of learners' performance during the
programme?
Is there a process or mechanism to tailor
assignments according to learners'
performance?

Self-assessment or practice exercises are

available for learners to know their level of
knowledge. Modules/lessons/units have
knowledge assessment tests. There is an
adequate number of assignments and/or
knowledge assessment tests that are evaluated
by tutors and/or peers during the
programme/course (formative assessment) in
order to monitor the progress of learners after
each module/unit.

Does each module/unit or major section have

an assignment? Do they present opportunities
for learners to self-assess their knowledge? Do
tutors review assignments and knowledge
assessments tests during the course to monitor
progress in learning?

Assignments requiring a problem solving

approach are not necessarily assessed as right
or wrong. Learners are provided with analytical
feedback, which takes into consideration their
reasoning and contributes to a better
understanding of the solution provided

Do tutors provide individual feedback to

learners whenever problem solving tasks are
required? Is the feedback provided
comprehensive enough to help learners better
understand the assessment made of the
solution they have chosen?

Learners receive clear information on the

Do learner receive clear information on the
timeframe foreseen for the tutor's feedback on timeframe required for the tutor to provide them
assignments and knowledge assessment tests. back with feedback on their assignments and
knowledge assessment tests?
Is there a process to check if the tutor has met
his responsibilities related to feedback
provision?

Accessibility is the degree to which a product

Were the accessibility standards considered in
(service, environment, device, etc.) is available the design of the programme/course?
to as many people as possible. It is often
referred to with a focus on specially abled
people.

Documentation on the learning design

and methodology of the programme
M

Examples of assignments and

guidance provided.
M

Documentation on the learning design

and methodology of the programme
M

Documentation provided to learners

informing on the time frame for
feedback provision on
assignment/task/exam.
E

Document describing how accessibility

is taken into account in the design of
the programme.

E.2 Usability standards are met.

E.3 The navigation (through the mandatory

learning materials) allows learners to know about
their progress and position in relation to the
overall content
E.4 Screens, table of content, and learning
materials, including additional resources are
printable.

P/C

Usability focuses on the easiness of the

learning environment and its components,
including the way materials and content are
presented.

Is the layout clear and free of unnecessary

elements?
Is the size and type of the font utilized
comfortable for reading?
Are images, illustrations, tables and other visual
aids (static or dynamic) easy to read?

P/C

Learners have at all times the possibility to

identify their position within the programme
(e.g. unit, module, lesson, part of a unit etc.).

Does the navigation allow learners to always

understand their position within the
programme?

P/C

Learners can easily print out any material

available.

Is a print function available for all the content of

the programme/course?

The file formats and size are in accordance

with the infrastructure available to the target
group. Local conditions regarding bandwidth
limitation are considered whenever the
programme/course targets group in developing
countries.

Has the download speed of the materials been

tested on the basis of the minimum internet
connection required? Are the file formats
utilized accessible to learners?

F Technology
F.1 The downloadable learning materials have
common formats and acceptable size.
P/C

F.2 The virtual learning environment runs on an

adequate server, which guarantees its stability.

F.3 The virtual learning environment is accessible

through different browsers and operating
systems.
F.4 The technology adequately supports the
learning strategies utilized. In particular, the
technology is in accordance with the IT
infrastructure available to the target group.

P/C

P/C

Wrong entries made by learners do not lead to Has the programme/course been tested on
constant crashes or interruptions.
various browsers/operating systems before
launching?

The programme/course runs smoothly

independently of the browser or operating
system utilized by the learners.
The technological solution offered is the result
of a comprehensive analysis, which takes into
account target group equipment and
connectivity, learners' skills and needs, staff
teaching skills, learning objectives,
assignments and other activities.

Has the programme/course been tested on

different browsers/operating systems? Does it
run adequately without compromising any
configuration/functionality?
Does the technology and the functionalities
offered by the learning platform support the
learning design and methodology and the
learning strategies used? Is the technological
solution and platform suitable to the target
group as regards IT infrastructure available?

Document describing how usability is

taken into account in the design of the
programme.

M
Document describing how technology
supports the learning approaches.
M

G Evaluation & Review

G.1 A comprehensive evaluation process is

foreseen at the end of the course/ programme to
evaluate its quality and overall coherence, and
contribute to its further improvement for
subsequent deliveries.
P/ C

Clear and comprehensive evaluation processes

are foreseen and described. A comprehensive
evaluation process includes: 1) learners
feedback collection through questionnaires or
other means, 2) preparation of an evaluation
report, in which feedback is analyzed and
converted into clear recommendations for
improvements, and 3) incorporation of the
recommendations in future programme design
to improve subsequent deliveries.

Is there a document describing how the

programme evaluation is conducted? Does the
evaluation process include: collection of
learners' feedback and an evaluation report
with recommendations for programme
improvement?.

Information on evaluation procedures

for the programme and, if applicable, a
report of the last evaluation.

G.2 Learners' feedback on the programme/

course delivery are collected through a
questionnaire or other means.

P/C

G.3 A report, based on the feedback collected by

learners, is prepared and includes clear
recommendations for further improvement of the
programme for subsequent deliveries.
P

G.4 A process for integrating the

recommendations for improvement is foreseen
and is part of programme/ course design
activities, in order to ensure continuous
programme/ course improvement.

P/C

A comprehensive questionnaire specifically

developed for the programme/ course is used,
which assesses overall quality and
appropriateness of: 1) Course Design
(methodology used, pedagogical approach,
navigation, structure of the course), 2) Course
Management (schedule and workload, grading
policy, e-tutor performance, ability to engage
learners, accuracy and timeliness of feedback,
guidance and advice provided, collaborative
activities versus individual, assignments), 3)
Course/Programme Content (accuracy and
relevance of learning content, learning
objectives, knowledge assessments tests, case
studies examples, relevance of discussions in
forum, additional resources, course
documentation: user's guide, course syllabus,
flyer and information provided to learners prior
to the delivery) and 4) Course Media and
Technical Support (registration process, access
and user friendliness of learning platform,
download time)

Is there a document or a questionnaire

specifically designed for the
programme/course, which aims at collecting
feedback, comments and suggestions from
learners on: 1) Course Design, 2) Course
Management, 3) Course/Programme Content
and 4) Course Media and Technical Support.

A comprehensive report, based on the

feedback provided by learners, includes clear
recommendations for further improvement of
the programme for subsequent deliveries. The
feedback collected from the learners should be
interpreted, analyzed and converted into clear
recommendations for further improvement
related to : 1) Course Design, 2) Course
Management, 3) Course/Programme Content
and 4) Course Media and Technical Support.

Is there a document/report describing how to

improve the programme for future deliveries?
Are there clear recommendations covering
course design, management, content and
technological aspects?

There is an established process for systematic

programme review and update and it is
documented. Once the learners' feedback is
collected, analyzed and converted into clear
recommendations, these are then part of
programme design activities and integrated to
ensure continuous programme improvement.

Are the recommendations systematically used

during subsequent programme design
activities? Is there a document describing how
the programme is continuously reviewed and
updated after each delivery? Is the process fully
documented?

Documentation informing about the last

review or a reviewplan for the
programme.

Information on evaluation procedures

for the programme and, if applicable, a
report of the last evaluation.

Documentation informing about the last

review or a reviewplan for the
programme.
M

Scoreboard
How to read this Scoreboard:
The column "Sum Minimum" shows the number of Minimum criteria to be met in each area.

The column "Minimum Met" shows the number of Minimum criteria the programme/course met in the evaluation. Th
minimum criteria are met for an area; otherwise it is shown in red.
The column "Sum Excellence" indicates the number of criteria defined as excellence criteria.

The column "Excellence Score" displays the sum of the ratings (between 0 and 3) that has been achieved by the p

The column "% of Maximum Score" demonstrates the performance of the programme/course vis--vis to the theore
criteria rated with 3). In case the course/programme has three Excellence Criteria, each rated with 2, the programm
have achieved 66,66% in this area.

Minimum Criteria
Possible max.
Score

Areas
A Information about and organization of the eLearning program
A.1 General description, objectives and program organization
A.2 Technical and organizational requirements
Sum

4
2
6

Sum

2
2

Sum

3
3

Sum

4
1
2
3
2
3
0
15

Sum

4
4

Sum

4
4

Sum
Total

4
4
38

B Target group Orientation

B.1 Target group Orientation
C Quality of the Contents
C.1 Quality of the Contents
D Programme and Course Design
D.1 Learning Design & Methodology
D.2 Motivation/ Participation
D.3 Learning Materials
D.4 eTutoring
D.5 Collaborative Learning
D.6 Assignments & Learning Progress
D.7 Assessment & Tests
E Media Design
E.1 Media Design
F Technology
F.1 Technology
G Evaluation & Review
G.1 Evaluation & Review

each area.

me/course met in the evaluation. The sum is shown in green when all

nce criteria.

) that has been achieved by the programme/course.

mme/course vis--vis to the theoretical maximum (all excellence

a, each rated with 2, the programme/course score will be 6 and it will

Minimum Criteria
Your Score

Excellence Criteria
Possible max.
% of Max
Your Score
Score
Score

0
0
0
0
0

12
12

0
0

0%
0%

0
0

3
3

0
0

0%
0%

0
0
0
0
0
0
0
0

0%

6
3

0
0

0%
0%

6
3
24

0
0
0

0%
0%
0%

39

0
0
0
0
0
0
0

Glossary of Terms
Assessment

Capacity Building

Course

Excellence Criteria
(marked with 'E')
Learning Design
Learning Methodology

Learning Objectives

Minimum Criteria
(marked with 'M')
Pedagogical Model

Programme

SMART Criteria

y of Terms
Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our
expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically
gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and
standards; and using the resulting information to document, explain, and improve performance.
Capacity Building means further education for individuals to empower them to shape and take part in their living
spaces as well as giving them the chance to develop competencies that are needed at their workplace. It refers
likewise to the empowerment of organizational learning, the development of management structures as well as the
design of legal and political frameworks, to strengthen the capacity for independent advancement
Refers to a short learning unit (less than 100 hours workload). Usually it is composed of only one unit, which takes
several hours and covers one complete topic. At least 20% of the course must be technology enhanced for it to fall
within the scope of the Open ECBCheck.
It reflects the level of excellence of the learning programme. The Excellence Criteria do not need to be met for
obtaining the Open ECB Check Label.
The planning, shaping and organization of learning activities including all learning methodologies, activities, and
strategies to support the learner's progress and competence development within a learning scenario.
Learning Methodology relates to all activities which are chose in order to support the student's learning process. The
term is often related to certain pedagogical models, e.g. "Anchored Instruction", "Exploratory learning", or others.
Learning objectives are statements about the outcomes of a training course. It informs what the learner will be able
to do upon completion of the course. They are presented on the learners' perspective, expressing as a target the
improvement of particular competencies. They set the targets that are to be reached by the end of the course, clearly
stating
what the
learner
willbasic
be able
to do
or must
knowbe
when
is completed. They
must the
be verifiable
and
The Minimum
Criteria
are
criteria
that
metthe
for course
the programme/course
to obtain
Open ECB
Check
measurable,
stated
with
action
verbs
and
following
the
SMART
criteria.
Quality Label.
The term pedagogical model relates to approaches which are suggestion certain forms of teaching/ learning
activities. Usually they are based on one or more learning theories and contain an array of learning activities. They
are often formulated in principles rather than in concrete instructions.
Refers to a training/learning offer that consists of several modules, which may be composed of a number of lessons,
and comprises a workload of at least 100 hours. The workload includes participation in tasks/activities; self study;
preparation for assessments, whenever applicable; and assessments. At least 20% of the programme must be
technology enhanced for it to fall within the scope of the Open ECBCheck.
SMART is a term used in project management at the project objective setting stage. It is a way of evaluating the
objectives or goals for an individual project - here used to validate learning objectives. The term is also in common
usage in performance management, whereby goals and targets set for employees must fulfill the criteria.
S: Specific & Significant
M: Measurable & Meaningful
A: Attainable & Appropriate
R: Relevant & Realistic
T: Time-bound & Traceable
(Literature: Doran, George T. "There's a S.M.A.R.T. way to write management's goals and objectives." and Miller,
Arthur F. & Cunningham, James A "How to avoid costly job mismatches" Management Review, Nov 1981, Volume 70
Issue 11.)