PGCE (PCE)

Curriculum Evaluation

Derek Wise February 2008

Content
Assignment 6b
Definitions of Syllabus, Curriculum and Evaluation
Curriculum Models
HOW to Evaluate: A Simple Model for Evaluating Specifics
Success Criteria
WHAT to Evaluate: the CIPP Model of Programme Evaluation
WHAT to Evaluate: the OFSTED of Course Evaluation
Curriculum Evaluation The Stakeholders and Interested Parties
The 4 levels of training evaluation
Curriculum evaluation an exam board perspective
Competence a semantic differential
Curriculum Evaluation - Less Reputable Evaluations
Bennets Seven Curriculum Issues
Curriculum Evaluation Gathering Data
Change - the five categories of product adopters:
Reading List

3
4
5
7
8
9
12
16
17
19
20
21
23
24
27
27

Progress & Presentation of assignment 6b

Carry out an evaluation of curriculum, teaching and learning issues in the teaching of your
subject specialism; measure the effectiveness of teaching and learning and make informed
conclusions and recommendations as to the future development of the programme
Feb 7: brief informal discussions with tutor on proposed curriculum evaluation
Feb 28: max 30 minute presentation of your proposed curriculum evaluation in 3 or 4 groups
of 4/5 course members
Presenters should provide a seminar paper (max 2 A4 sides) to support their presentation
this will probably be bullet points responses to the 6 seminar issues above. Make 3 copies - 1
copy per 2 members and 1copy for tutor. OHTs should NOT be used. NB presenters will be
stopped after 30 minutes to ensure equal time for all members
June 5 : max 30 minute final presentations of your completed curriculum evaluation.
In 3 or 4 groups of 4/5 course members.
Presenters should provide a seminar paper (max 2 A4 sides) to support their presentation
-this will probably be bullet points responses to the 8 seminar issues above (which could of
course be an updated version of the earlier seminar paper). Make 3 copies - 1 copy per 2
members and 1copy for tutor. OHTs should NOT be used. NB presenters will be stopped
after 30 minutes to ensure equal time for all members
(Curriculum Evaluation (approx 2500words) in a plastic project folder with name clearly
indicated should be handed to Derek at the end of the session for marking). A copy of the
seminar papers for February and June should be included in the folder
Assignments will be returned within 1 week and any required developments (hopefully there
wont be) can be emailed to Derek. Thoughtful and carefully presented seminars will be
positively taken into account in the light of any weaknesses in the final assignments!

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 2

Assignment 6b Evaluation of a Curriculum

Aims and Learning Outcomes
By the end of this assignment you will be able to:
select and describe a course programme in your specialist subject
analyse specific issues raised by the programme specification, course design,
teaching, learning or assessment methods in relation to the needs of typical learners
of this subject and level
use evaluation techniques to identify and analyse the responses of learners and
colleagues to aspects of the curriculum and its delivery
explain and apply models of curriculum to their evaluation
identify strengths and weaknesses of the current programme,
propose and justify improvements to the design or delivery of the programme
justify your analysis and recommendations with reference to current research into
teaching and learning, with specific reference to you specialist subject
present your evaluation in a concise, well-written and referenced report
Task
Candidates are required to:
1. carry out an evaluation of curriculum, teaching and learning issues in the teaching of your
subject specialism
2. measure the effectiveness of teaching and learning
3. make informed conclusions and recommendations as to the future development of the
programme.
Suggested activities could include:

identifying your current weaknesses in and updating your subject knowledge

gathering information about syllabuses, standards and national developments in your
subject
reviewing the resources available for the teaching of your subject
critically reviewing current issues in assessment related to your subject area
undertaking a comparative examination of the teaching of your subject in another
establishment
discussion with experienced practitioners on proven effective strategies for planning
and delivering teaching and learning.

Word count 2500 maximum, excluding bibliography and appendices

Assessment Criteria
This assignment is marked on a pass/refer basis.
In completing this assignment, you should be able to demonstrate evidence of Standards
AS1-7, AK1.1-5.1, 6.1-7.3, BK1.1-2.7, 3.3, 4.1-5.2, CS1-4, CK1.1-4.2, DS1-3, DK1.1, 1.3, 2.1,
3.1-3.2, ES1, 5, EK1.1-2.1, 2.3, 3.1-3.2, 5.1-5.2, FS3,

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 3

Definitions of Syllabus, Curriculum and Evaluation

Evaluating Education- A Warning!
With the institutionalisation of new wave evaluation, I think we have produced a situation in
which there is some danger that evaluators create their own establishment and glamorise it
as an elite. Lets keep hold of the idea that it is mostly a matter of common sense and
learning from experience. That is not entirely true but it keeps us from going technical or
theological; and a little modest oversimplification is better than a lapse into jargon or
pretentiousness
Stenhouse, L. (1975) An Introduction to Curriculum Research and Development Heinemann
p116
Syllabus
A syllabus is a list of indicative content that is usually provided by the awarding body. This
may come in forms such as competence statements, programmes of study, directives,
learning outcomes etc. It is essentially static and in this sense prescribed and is only brought
alive through description by the teacher and by the students where it becomes transformed to
a curriculum.
Curriculum
A curriculum is dynamic. It incorporates the whole of the learners experience, both formal
and informal. Curriculum is not the intention or prescription but what happens in real
situations. It is about perceiving, understanding and describing what happens
.. a course; especially a regular course of study at a school or university
(Shorter Oxford Dictionary)
In 1984 Sir Keith Joseph (then Secretary of State for Education) gave the annual North of
England speech on the curriculum stating that The curriculum must have relevance to the
world of work ... it must be balanced, there must be breadth and differentiation
21 years later these principles are still fundamental to any curriculum
Evaluation
To judge or assess the worth of..
Collins Concise English Dictionary
The process of evaluation is essentially the process of determining to what extent the
educational objectives are actually being realised by the program of curriculum and
instruction. However, since educational objectives are essentially changes in human
beings, that is, the objectives aimed at are to produce certain desirable changes in
the behaviour patterns of the student, then evaluation is the process of determining
the degree to which these changes in behaviour are actually taking place.
Tyler (1949) Basic Principles of Curriculum and Instruction Chicago: University of
Chicago (pp. 105-106)
Evaluation is the process of conceiving, obtaining and communicating information for
the guidance of educational decision-making with regard to a specified programme
(MacDonald, 1975)
One of the important considerations in any systematic evaluation is to make clear what the
criteria are for the evaluation.
It is the insistence on the explicitness about criteria which distinguishes evaluation
from the everyday use of the term. (Harlen)

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 4

Curriculum Models
There are a number of 'models' of curriculum or instructional systems. Remember a model is
an abstraction from reality - it doesnt exist in itself but can provide a useful often visual basis for analysis. Consider a curriculum with which you are currently engaged does it
conform to one or more of the following models?
1

linear curriculum:

Useful for instruction related to sequential topic areas containing objectives of roughly equal
weighting.
Knowledge
1a
Skill 1a

Knowledge
1b
Skill 1b

Knowledge
1c
Skill 1c

Required
Knowledge
& Skill for
topic 1

Knowledge
2a
Skill 2a

Knowledge
2b
Skill 2b

Required
knowledge
& Skill for
topic area 2

Example of use:
particular value:
Any inherent problems:
Style(s) of teaching/learning best suited by this model:
2

Spiral Curriculum

Topics are covered in enough depth to provide basic understanding and prerequisite knowledge
for 2nd time round and then represented in greater depth for 3rd time etc etc

Example of use:
particular value:
Any inherent problems:
Style(s) of teaching/learning best suited by this model:
3

Pyramid Curriculum

Clear move from general (broad base) to particular

Year 3 - options
Year 2 - some specialism
Year 1 - broad introduction

Example of use:
particular value
Any inherent problems:
Style(s) of teaching/learning best suited by this model:

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 5

Common Core

The common core is often linked to other models (eg pyramid with common core + options) It
requires ensuring that the common element is necessary & sufficient

Example of use:
particular value
Any Inherent problems
Style(s) of teaching/learning best suited by this model:

Project centred curriculum

Leads out from an initial central theme and provides network of topics as a map to show where
the learner should get to over a period of time - can be accredited as accumulated/transferable
units. Very popular in primary education and syllabi for adult learners with particular needs

Example of use:
particular value
Any inherent problems:
Style(s) of teaching/learning best suited by this model:

Research/Inquiry centred

Centred around substantive issue(s) or question(s) which the student has to inquire into in order
to provide a coherent, in depth and analytical research report or dissertation or thesis

Example of use:
particular value
Any inherent problems:
Style(s) of teaching/learning best suited by this model:

If the curriculum with which you engaged does not fit to any of the above models how would
you define it?

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 6

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 7

Curriculum Evaluation HOW to Evaluate: A Simple Model for Evaluating Specifics

Evaluation is EXTREMELY important to the teacher as it provides the opportunity to make an
informed opinion about something - and helps reflection. Evaluation is concerned with
looking at the strengths & weaknesses of something; the efficiency & effectiveness of
how it is undertaken and the value to those who are involved. This can be remembered
by the acronym SWEET (nb the acronym SWEET was created by Derek Wise and is
not published).
Evaluation should be based on evidence rather than just gut feelings.
This evidence can be quantitative (ie quantifiable as assessment results, numbers or physical
evidence such as retention on courses). You can observe learner participation and interest,
for example the time they spent on-task (doing the work they should be doing) as against
being off-task over a period of time and recording your results. Your evaluation can also be
qualitative (ie about the quality or feeling or perception of something for example learners
feelings, tutors feelings or the reaction to the learners to the lesson).

Strengths
What were the good things about it how can they be used in the future?
What worked well? why? can this be applied to other lessons?
What were the good results? can they be replicated in the future?

Weaknesses
What were the weak things about it? how can they be overcome in the future?
What did not work well? - why? How could they be developed for the future?
What were the poor results? how can they be improved?

Efficiency
Efficiency is concerned with competence; the ability to do something well or achieve a desired
result without wasted energy or effort, or the degree to which this ability is used
What things showed that the lesson was efficient eg (re-usable learning resources,
recap of previous work to ensure all learners were ready to start etc). how can this
efficiency be applied to other situations?
What things suggested that the lesson was not efficient? how can this be improved?

Effectiveness
Effectiveness is concerned with producing or causing a result; especially the desired or
intended result. To be truly effective the result should be striking especially in producing a
strong or favourable impression on people
What things showed that the lesson was effective? eg (learners were involved and
appeared to be learning; appropriate learning resources, recap of work at the end of
the session which showed learners had learnt) what lessons can I learn for my
teaching?
What things suggested that the lesson was not effective?

The value to those involved (value added)

Sometimes social / personal things like teaching cannot be easily evaluated in terms
of efficiency and effectiveness or based on qualitative or quantitative data. The
extent to which the lesson (or more probably whole programme of study) has
contributed to the growth of the learner although this may not appear in examination
results is very important. This is sometimes called value added
Evaluation often requires comparison with similar events eg another lesson it is
essential that this evaluation is based on like events.

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 8

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 9

Curriculum Evaluation HOW to Evaluate: Success Criteria

For a valid and reliable evaluation (particularly a course evaluation) it is essential to have a
clear idea of the criteria for what would constitute a successful outcomes (ie the success
criteria).
Success criteria are statements of what should reasonably be expected as a result of the
action taken
For example if the student achievement on a particular course is 70% is this good or bad? If
the success criteria established at the outset of the course was that the course should have
an 80% student achievement rate (with grade stated if necessary) based on previous years,
or on national benchmark figures from other providers, then clearly 70% is unsatisfactory.
Writing success Criteria
It is essential to ensure that the success criteria fair and honestly capture the true situation
not an adaptation to suit the evaluation!
A course evaluation of a GCSE course states that one success criteria will be the % of
students who gain a grade B qualification. Results shows that 60% of the course members
who took the exam gained a grade B. It appears pretty good!
But
The course started with 24 students and only 12 completed and took the exam - there was a
50% drop-out! Of these 12 students 60% - ie 7 students - gained a grade B.
So
in fact only 30% of the students who started the programme gained a grade B
The success criteria did not make it clear whether it was using the start number or the number
taking the exam.
For each of the issues considered in the CIPP or OFSTED models it is important to establish
a success criteria based on previous years, or national benchmark figures or on what the
course team feels is appropriate. It is important to state how the success criteria has been
reached

Success criteria should be observable and measurable

Success criteria should be based on what the objective reasonable and legitimately
requires

Success criteria should be attainable they should be designed to attain success

rather than prevent it.

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 10

Curriculum Evaluation WHAT to Evaluate: the CIPP Model of Programme Evaluation

A useful model for teaching programme evaluation is the Context-Input-Process-Product (CIPP)
model (Daniel Stufflebeam 1966, 1967, 1971,1972,2000) which proposes a four part evaluation
for any programme and suggests a number of questions which should be asked within those
headings
In general, these four parts of an evaluation respectively ask:
1
2
3
4

What needs to be done? (context)

How should it be done? (Input)
Is it being done? (process)
Did it succeed? (product)

Context - The setting of the course or subject ie the aims. This can be seen in terms of the
validity of what is being taught External factors which impinge on the programme and which
may affect its outcomes might be considered here (eg course is requirement for Government
New Deal) .
Input - The input element relates to the factors which contribute to the course, ie the tutor(s),
the students, the resources, the environment
Process - The process element concerns the appropriateness of what happens on the course,
how the input elements are used to achieve the aims and objectives.
Product - The product concerns the outcomes - what has been gained (ie value added) and
what has been achieved, (ie qualifications)
Product element sub divisions
More recently the Did it succeed? or product evaluation part is itself divided into
Impact

Were the right beneficiaries reached? Was there a possible audience

who were not reached if so why (nb LLUK values of open access)

Effectiveness -

Were their needs met? Was there an in-effective prioritization of one

aspect of the curriculum over another. Could the programme have
been delivered in a more cost efficient way with equal effectiveness?
Do the results of the effectiveness evaluation suggest that changes
should be made in specific areas?

Sustainability -

Were the gains for the beneficiaries sustained? Was there a short
term improvement which was not evidence some time after the
programme. Possible interview of past course members to ascertain
the long-term programme value

Transportability - .

Did the processes that produced the gains prove transportable and
adaptable for effective use in other settings? How will early adopters
be reached? How will the more reticent early and late adopters be
persuaded? (see note on Everett Rogers at back of handbook)

Thus the model can be remembered by the acronym CIPPIEST!

For the purposes of general curriculum evaluation the initial CIPP model should suffice although
evaluators could consider the 4 product aspects for greater insight.
CIPP in course evaluation
The four CIPP main headings provide a range of questions to be asked (for each one consider
how do I know this (data)?)

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 11

Context (The setting of the course or subject)

Meeting Needs
Are the aims consistent with the needs of the learner, industry or society?
Does the curriculum develop links with the world of work?
Does the curriculum meet current Government initiative eg basic skills/key skills?
Proposed Course Structure
Is the course structure the best way of achieving the desired outcomes?
Are the aims updated regularly/has the course been updated in the light of national
developments (eg basic skills/key skills, vocational developments
Encouraging Progression
Do the aims encourage progression in further or higher education, or transferability to other
programmes?

Input

(The input factors which contribute to the course)

Students
Do the students possess the entry ability required by the course content
Did the students original wish to do this course?
Tutors
Are tutors qualified (an area of current interest in post compulsory education)
Are the tutors competent in the subject; in current teaching methods and where necessary, in
current industrial requirements?
Are there adequate tutors for all aspects
Do tutors have adequate time for all the requirements of the course eg IV/double
marking/tutorials
Delivery
Is there an appropriate amount of time in line with awarding body specifications
Are resources appropriate, adequate and available?
Is the environment conducive to learning and appropriate to the world of work?

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 12

Process (The appropriateness of what happens on the course,)

Students
Do students have a voice (eg staff/student forums)
Is there appropriate course group support and interactivity facilities
Is there an effective student guidance/counselling system
Tutors
Do tutors receive/access, understand and use, all available documents related to
the curriculum, ie information from examining bodies regarding syllabus and assessment
requirements
Are there regular and useful course team meetings
Do team members have access to necessary professional development?
Delivery
Are teaching strategies appropriate?
Are appropriate teaching aids and learning resources used?
Is there appropriate/adequate general access to information technology?
Is there appropriate/adequate general access to library and academic support ?
Are course accommodation & timetabling facilities maintained (eg no constant room or
timetable changes)
Monitoring & Assessment
Is the monitoring/tutoring/support system appropriate and adequate?
Is the assessment system valid and reliable?
Is there an effective system of feedback to learners ?

Product (what has been gained (ie value added) and achieved)
Students
Attendance rate (and possible comparison to achievement)
What is the staying-on rate as a trend (this year compared to the last 2or 3)?
What is the qualification achievement rate as a percentage of starters?
Has the course contributed to the students ability to meeting the needs of work (eg
presentation skills?
What is the progression rate to further industry/FE/HE?
What was the students response to the course (eg results of student perception
questionnaires
Tutors
What staff development was identified and satisfied at individual and course team level?
Are tutors motivated by their involvement in the course?
Delivery
To what extent has the course been modified in the light on on-going feedback?
If appropriate, what do employers feel about the course?
Is there an effective system for evaluating and gaining feedback about the programme?
Developed by Derek Wise from Stufflebeam D. (1971). Educational Evaluation and Decision
Making. In Stufflebeam, D.L. Madaus, G. F. & Kellaghan, T. (Eds.), (2000) Evaluation models (2nd ed.).
(Chapter 16). Boston: Kluwer Academic Publishers.

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 13

(note: the CIPP model was developed over many years with specific aspects added at each
review and development.)

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 14

Curriculum Evaluation WHAT to Evaluate: the OFSTED of Course Evaluation

The Office for Standards in Education (Ofsted) is a non-ministerial government
department, established in 1993 under the Education (Schools) Act 1992. It is responsible for
inspecting the standards of independent and state schools, local education authorities, child
day care and childminding in England; it also monitors the work of the Independent Schools
Inspectorate. Ofsted is also required to provide independent advice to the United Kingdom
government and parliament on matters of policy and to publish an annual report to parliament
on the quality of educational provision in England. The legal name is the "Office of Her
Majesty's Chief Inspector", reflecting the fact that Ofsted's powers are vested in its Chief
Inspector, appointed by the Queen-Council. In September 2005 a new system of short notice
inspections came into being. From April 2007 OFSTED will merge with the Adult Learning
Inspectorate (ALI) to provide an inspection service which includes all post 16 government
funded education
A college of course Evaluation undertaken by Ofsted
A college inspection is a process of evidence gathering to evaluate the effectiveness of the
college in supporting the learning and achievement of students. The emphasis is on judging
the degree to which rigorous self-assessment leads to quality improvement in the college
The inspectors carry out inspections using the Common Inspection Framework, details of the
criteria can be found on the Ofsted website. www.ofsted.gov.uk . Once the inspection has
been conducted a report will be published, this will list the names of the qualification reviewed
(including retention and achievement data).
OFSTEDs Common Inspection Framework as a model for course evaluation

What are the school's main strengths?

What are you working on to improve?
How successful are your policies and practice?
Does what you offer help children to reach the highest standards?
Does what you do help to make the children safe and healthy?

Ofsted considers five issues

Effectiveness of provision
Capacity to improve
Achievements and standards
Quality of provision
Leadership and management

When observing courses and classes they ask 4 questions

How well do learners achieve?
How effective are teaching and learning?
Do activities meet the needs of learners?
How well are learners guided and supported?
(A curriculum evaluation could be based on these four questions)
These are assessed against four standards
Outstanding
Good

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 15

Satisfactory
Inadequate

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 16

OFSTED SCHOOL SELF-EVALUATION

Schools now receive an on-line self evaluation form which is used by Inspectors. This form
can be a useful basis for a curriculum evaluation
1. CHARACTERISTICS OF YOUR SCHOOL
1a Please outline the main characteristics of the learners, including:
- their attainment on entry and how you know this;
- their social and economic backgrounds, indicating the level of prosperity or deprivation.
1b Please summarise briefly your distinctive aims and describe any special features of
your school, for example:
- whether you intend to become a specialist school, or school with special status, and if
it is one already, the main changes that have occurred because of this;
- whether you are a school with a religious character;
- any special units, additional community services or extended provision;
- significant partnerships with other providers or agencies (such as shared arrangements
for the curriculum, federal arrangements, or partnerships with employers).
1c Please outline specific contextual or other issues that act as aids or barriers to
raising performance, for example:
- any difficulties in recruiting and retaining staff, or governors;
- recent or impending reorganisation;
- mobility of learners;
- particularly important facts in your recent history, such as change of leadership.
1e Please outline briefly the main priorities in your improvement/development
plan, and how they reflect the context in which you work.
2. VIEWS OF LEARNERS, PARENTS/CARERS AND OTHER STAKEHOLDERS
What are the views of learners, parents/carers and other stakeholders and how do you
know?
2a How do you gather the views of learners, parents/carers and other stakeholders,
such as those accessing additional services, how often do you do this, and how do
you ensure the impartiality of the information?
2b What do the views of learners, parents/carers and other stakeholders tell you about
the learners standards, personal development and well-being, and the quality of your
provision?
2c How do you share with parents/carers and other stakeholders the collated findings
about their views?
2d Can you give examples of action you have taken based on the views of learners,
parents/carers or other stakeholders, with an evaluation of the effectiveness of what
you did?
- Are there examples of actions you decided not to take (with the reasons for this)?
- Are there examples of ways in which your stakeholders have influenced the priorities
noted in section 1e?

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 17

3. ACHIEVEMENT AND STANDARDS

How well do learners achieve?
In answering the following questions, please make clear the main evidence, such as
performance data, assessments and records of learners progress, on which your evaluation
is based (but please use data selectively, avoiding the copying out of tables of descriptive
information).
3a What are learners achievement and standards in their work?
- the standards learners reach as indicated by their test and examination results, taking
account of: any significant variations between groups of learners, subjects, courses and key
stages; trends over time; comparisons with other schools; whether learners reach challenging
targets
- the standards of learners current work in relation to their learning goals (noting any
significant differences between current work and recent results)
- learners progress relative to their starting points and capabilities, with any significant
variations between groups of learners (making clear whether there are any groups that are
underachieving and could be doing better)
3c On the basis of your evaluation, what are your key priorities for development?
4. PERSONAL DEVELOPMENT AND WELL-BEING
How good is the overall personal development and well-being of the learners? In answering
the following questions, please make clear the main evidence on which your evaluation is
based.
4a To what extent do learners adopt healthy lifestyles?
- whether learners take adequate physical exercise, and eat and drink healthily
- learners growing understanding of how to live a healthy lifestyle
4b To what extent do learners feel safe and adopt safe practices?
- whether learners feel safe from bullying and racist incidents
- the extent to which learners have confidence to talk to staff and others when they feel at risk
4c How much do learners enjoy their education?
- take account of learners attitudes, behaviour and attendance
- learners spiritual, moral, social, emotional and cultural development
4d How well do learners make a positive contribution to the community?
- learners growing understanding of their rights and responsibilities, and of those of others
- how well learners express their views and take part in communal activities
4e How well do learners prepare for their future economic well-being?
- how well learners develop skills and personal qualities that will enable them to achieve
future economic well-being
- learners understanding of career options, and the acquisition of workplace skills
4g On the basis of your evaluation, what are your key priorities for development?

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 18

5. THE QUALITY OF PROVISION

Your evaluation of the quality of provision should take account of the impact on the standards
achieved and the personal development and well-being of learners. In answering the
following questions, please make clear the main evidence, such as monitoring of teaching, on
which your evaluation is based.
5a How good is the quality of teaching and learning?
- how well teaching meets the needs of the full range of learners and course requirements
- the suitability and rigour of assessment in planning learning and monitoring learners
progress
- the diagnosis of, and provision for, individual learning needs
- the involvement of parents and carers in their childrens learning and development
5b How well do the curriculum and other activities meet the range of needs and
interests of learners?
- the extent to which the curriculum or activities match learners needs, aspirations and
capabilities, building on prior attainment and experience
- how far the curriculum meets external requirements and is responsive to local
circumstances
- the extent to which the provision enables and encourages learners to be healthy and stay
safe
- the extent to which learners have opportunities to develop enterprise, financial skills and
work in teams
- the extent to which enrichment activities and, where appropriate, extended services
contribute to learners enjoyment and achievement
- where appropriate, the extent to which employers needs are met through developing workrelated skills
5c How well are learners guided and supported?
- the care, including as appropriate integrated day care, advice, guidance and other support
provided to safeguard welfare, promote personal development and make good progress in
their work
- the quality and accessibility of information, advice and guidance to learners in relation to
courses and programmes, and, where applicable, career progression
- the extent to which the school and any additional services contribute to the learners
capacity to be healthy, including vulnerable groups, such as looked after children
5e On the basis of your evaluation, what are your key priorities for development?
Quality of teaching and learning
Quality of the curriculum and other activities
Quality of care, guidance and support for learners

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 19

Curriculum Evaluation The Stakeholders and Interested Parties

Formal & Informal Evaluations

The learner (primarily in terms of value to self)

informal but may be involved in formal (through personal reflection and value judgements
about the worthwhileness of the course). Primarily qualitative

The tutor - (for monitoring programme benefits)

formative informal but may be involved in summative formal. Probably primarily
qualitative

The curriculum/programme team - (through the internal verifier/moderator)

Primarily formal and qualitative (standardisation of assessment and possibly quantitative (eg
pass rates)

The College / Educational Organisation (through the quality unit)

Primarily formal and quantitative (eg pass rates) possibly qualitative (student perceptions of
course)

The Awarding Body (through the external verifier/ moderator)

Eg 3 main groups (OCR; Edexel; AQA/CGLI) plus ABC Awards; OCN
Primarily formal and quantitative (eg pass rates; standards of assessment; portfolios);
possibly qualitative (student perceptions of course)

Learning & Skills Council (LSC) / local LSC (LLSC)

(Through OFSTED and ALI (ALI ceasing 2006)
Primarily formal and quantitative (eg completion/achievement rates; retention rates); possibly
qualitative (student perceptions of course gained during inspections)

DfES
Primarily formal and quantitative gained from OfStEd reports

The General Public

Primarily formal and quantitative gained from OfStEd reports available on-line (open.gov.uk)

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 20

Kirkpatrick's Four Levels of Evaluation in Training

In Kirkpatrick's (1998) four-level model, each successive evaluation level is built on

information provided by the lower level. According to this model, evaluation should always
begin with level one, and then, as time and budget allows, should move sequentially through
levels two, three, and four. Information from each prior level serves as a base for the next
level's evaluation. Thus, each successive level represents a more precise measure of the
effectiveness of the training programme, but at the same time requires a more rigorous and
time-consuming analysis.

Level 1 Evaluation - Reactions

Evaluation at this level measures how participants in a
training programme react to it. It attempts to answer
questions regarding the participants' perceptions - Did they
like it? Was the material relevant to their work? This type of
evaluation is often called a smilesheet. According to
Kirkpatrick, every programme should at least be evaluated
at this level to provide for the improvement of a training
programme. In addition, the participants' reactions have important consequences for learning
(level two). Although a positive reaction does not guarantee learning, a negative reaction
almost certainly reduces its possibility.
Level 2 Evaluation Learning
To assess the amount of learning that has occurred due to a
training programme, level two evaluations often use tests
conducted before training (pretest) and after training (post
test).
Assessing at this level moves the evaluation beyond learner
satisfaction and attempts to assess the extent students have
advanced in skills, knowledge, or attitude. Measurement at
this level is more difficult and laborious than level one.
Methods range from formal to informal testing to team
assessment and self-assessment. If possible, participants
take the test or assessment before the training (pretest) and after training (post test) to
determine the amount of learning that has occurred.

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 21

Level 3 Evaluation - Transfer

This level measures the transfer that has occurred in
learners' behaviour due to the training programme.
Evaluating at this level attempts to answer the question - Are
the newly acquired skills, knowledge, or attitude being used
in the everyday environment of the learner? For many
trainers this level represents the truest assessment of a
programme's effectiveness. However, measuring at this level
is difficult as it is often impossible to predict when the change
in behaviour will occur, and thus requires important decisions
in terms of when to evaluate, how often to evaluate, and how
to evaluate.

Level 4 Evaluation- Results

Level four evaluation attempts to assess training in terms of
business results. In this case, sales transactions improved
steadily after training for sales staff occurred in April 1997.
Frequently thought of as the bottom line, this level measures
the success of the programme in terms that managers and
executives can understand -increased production, improved
quality, decreased costs, reduced frequency of accidents,
increased sales, and even higher profits or return on
investment. From a business and organizational perspective,
this is the overall reason for a training programme, yet level
four results are not typically addressed. Methods for Long-Term Evaluation include:

Send post-training surveys

Offer ongoing, sequenced training and coaching over a period of time
Conduct follow-up needs assessment
Check metrics (e.g., scrap, re-work, errors, etc.) to measure if participants achieved
training objectives
Interview trainees and their managers, or their customer groups (e.g., patients, other
departmental staff)

Kirkpatrick, D. (1998) Evaluating Training Programs: The Four Levels.

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 22

Curriculum Evaluation
Comparison of the roles of the internal and external moderators/verifiers as a part of
curriculum/programme quality control
Internal Verification
Job Specifications

External Verification
Job Specifications Appointed by awarding body

Occupational competence
Need `V1' unit for NVQs

Occupational competence
Need V2 unit issued by awarding body

Can be marker/assessors, but not IV their

own work

Not in their familiar area, conflicts of interest must

be declared

Need knowledge of using current IV

systems and procedures and needs of the
awarding body documents and
syllabus/standards of award

Attend regular meetings for currency Suitably

qualified and experienced in the area

Responsibilities
Provide support, feedback and guidance
for marker/assessors

Responsibilities
Be familiar with the syllabus/standards of awards

Identify and support training needs for

assessors

Reports to and receives advice from the awarding

body (principal verifier or head office)

Set up and complete an IV system in

accordance with the awarding body
regulations
Check that assessor/markers are working
to a common standard

Provide support, feedback and guidance for

marker/assessors

Sample marked work (All learners/ units/

methods) to check standard and to give
feedback to the assessors
Hold standardisation meetings to share
ideas of good practice with assessor team,
information from the awarding body

Make visits to approved centres who wish to offer

further qualifications

Make approval visits to organisations wishing to

become an accredited centre

Check that the recording of all results and

enrolments are completed

Make routine verifier visits to centres to check that

all aspects of the centre delivery is `safe' (H/S,
equal ops, procedures, paperwork, recording
systems etc)
Interview candidates, assessors, verifiers, (observe
assessment or IV taking place in NVQs)

Check that any action plans from the EV

are implemented in the timescale agreed.

Discuss ideas for improvement and make SMART

action plans for the centre to work through

Ensure that the equal ops, antidiscriminatory policies are in place and
upheld.
Maintain safe and secure storage of the
students' work

Maintain records for each centre as required by

awarding body.

Record and deal with any appeals from the

learners using centre systems

Check dates for enrolments, certification and

timescale of the, awards

Check that formative and summative assessor

verification is taking place

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 23

Competence assessment
1 2 3 4 5
Once a person is competent in an
area they are always competent in
that area
Competence indicates the ability to
perform a skill with basic levels of
safety and achieving a reasonable
outcome (if you pass your driving test
you are therefore a competent driver)
There are different levels of
competence - you can be more or
less competent
A competent performance does not
require a large amount of
underpinning knowledge
A true standard of performance
which could be called competent only
exists in the mind of the person
assessing the performance and
cannot really be communicated to
the learner
It is possible to teach a person to be
totally competent at a job
Competence can be accurately
assessed by a few observations
To accurately assess competence
you have to be extremely skilled in
the activity yourself
A performance without thorough
understanding of all of the possible
implications cannot be seen as
competent
True competence requires significant
individual creativity
Within an occupational area you can
be extremely competent in one job
and totally incompetent in another,
although the jobs may be related
True competence cannot really be
assessed since it requires accurate
performance in all possible
conditions
True competence can be
demonstrated in simulated conditions
if they are appropriately testing

Competence is very short lived and

needs constant practice - you can be
competent one minute and not
competent the next
True competence indicates a high level
of creative skill which only occurs after
years of constantly refined practice
Competence is competence - you are
either competent or you are not there
are no half measures
A competent performance requires a
large amount of underpinning
knowledge
A true standard of competent
performance can be accurately written
down and communicated to the learner

True competence can be only be

acquired by the individual practising
the activity for themselves
Competence can be accurately
assessed by a large number of
observations
To accurately assess competence you
really only need to know what the
performer has to achieve
A performance can be competently
carried out even if the performer
doesn't really know all of the
implications
True competence requires the
performer to repeat the actions which
they have been taught accurately
To be truly competent you must be
able to perform all related jobs to a
high level
True competence can be easily
assessed if performance criteria are
determined
True competence can only be
demonstrated in a realistic work
environment with all of the pressures
that that entails

Based on The Semantic differential Osgood C & Tannenbaum P (1957) The Measurement of
Meaning University of illinois press

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 24

Curriculum Evaluation - Less Reputable Evaluations

Consider the following less reputable types of evaluation identified by Patton (1981) - do you
recognise any of them? Where might these be used in education or training?
Less reputable types of
evaluation

Where they might be used - and prevented!!

Quick-and-Dirty Evaluation

Weighty Evaluation

Guestimmate Evaluation

Personality-Focused
Evaluation

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 25

Pseudo-evaluation identified by Suchman (1967)

Pseudo-evaluation

Where they might be used - and prevented!!

Eyewash Evaluation

Whitewash Evaluation

Submarine Evaluation

Posture Evaluation

Postponement Evaluation

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 26

Bennets Seven Curriculum Issues

Thinking about evaluation
There is a saying in education if it moves teach it; if it doesnt move evaluate it. There is a
danger of evaluating for evaluation sake. Always consider the following:
1

What is the evaluation trying to achieve what am I doing it for?

What would be appropriate indicators of success? If you are evaluating something

then you must have some idea of a standard or benchmark which means that the
thing evaluated is successful or not. If you evaluated a new car you would have
success criteria eg speed, comfort, in-car facilities, petrol consumption, visibility,
price, look etc (NB consider these factors related to a car which ones are efficiency
factors and which ones are effectiveness factors)

How should data be gathered and processed? What evidence will I gather to
evaluate a lesson -observation; discussion with teacher, discussion with learners,
homework, later class work etc

With what can the results be validly compared? You may not expect the results of
say an A level session with students who have struggled to gain a couple of GCSEs
grades C/D to be the same as the results of an A level undertaken with learners who
have gained a number of grade A GCSEs.

What other information is necessary to put the results into context?

What conclusions can legitimately be drawn?

What action follows?

Reference Bennet N et al (1994) Improving Educational Management through Research and

Consultancy Chapman
Task
What do you understand by Bennets 7 points? Make notes on each point and think about
these when you come to your programme evaluation.

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 27

Curriculum Evaluation Gathering Data

What evidence can we use in evaluating courses and curricula?
QUANTITATIVE EVIDENCE [ so-called robust evidence]

enrolments & patterns of enrolments (gender, ethnicity, widening participation etc.)

attendance & partial attendance
course completion and retention
achievements and public outcomes
evidence of progression to further courses
evidence of progression to employment
three year trends for any of the above
institutional comparisons for any of the above (e.g. attendance)
national benchmarks with other institutional sectors (6 th-form, tertiary, general FE etc)

How many of these would be useful to you in your course evaluations? Would you be able to
get access to this information?
QUALITATIVE QUANTITATIVE EVIDENCE [evidence from human sources that you
can turn into quantities if you chose to do so]

evidence from the learners themselves

evidence from employers
evidence from parents
evidence from assessors [markers, examination boards, accreditation bodies]
evidence from moderators, internal and external verifiers
evidence from observers of teaching: formal teaching observations [including those
by inspectors, consultants and critical friends] and more informal peer observations
and team-teaching
evidence from the course team and the teachers and themselves

How many of these would be useful to you in your course evaluations? Do you use any of
these at present? Which, for your purposes, would you consider the most valuable?
Nevos 10 Major Issues in Curriculum Evaluation
David Nevo (1986) has attempted to clarify the meaning of evaluation by identifying 10
questions that represent the 'major issues addressed by the most prominent evaluation
approaches in education'
1
How is evaluation defined?
Educational evaluation is a systematic description of educational objects and/or an
assessment of their merit or worth.
2
What are the functions of evaluation?
Educational evaluation can serve four different functions:
(a)
(b)
(c)
(d)

formative (for improvement);

summative (for selection and accountability);
political (to motivate and gain public support); and
administrative (to exercise authority).

3
What are the objects of evaluation?
Any entity can be an evaluation object. Typical evaluation objects in education are students,
educational and administrative personnel, curriculum, instructional materials, programs,
projects, and institutions.

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 28

4
What kinds of information should be collected regarding each object?
Four groups of variables should be considered regarding each object. They focus on
(a)
(b)
(c)
(d)

the goals of the object;

its strategies and plans;
its process of implementation; and
its outcomes and impacts.

5
What criteria should be used to judge the merit of an object?
The following criteria should be considered in judging the merit or worth of an educational
object:
(a)
(b)
(c)
(d)
(e)

responding to identified needs of actual and potential clients;

achieving national goals, ideals, or social values;
meeting agreed-upon standards and norms;
outdoing alternative objects; and
achieving important stated goals of the objects.

Multiple criteria should be used for any object.

6
Who should be served by an evaluation?
Evaluation should serve the information needs of all actual and potential parties interested in
the evaluation object ('stakeholders'). It is the responsibility of the evaluator(s) to delineate the
stakeholders of an evaluation and to identify or project their information needs.
7
What is the process of doing an evaluation?
Regardless of its method of enquiry, an evaluation process should include the following three
activities:
(a)
(b)
(c)

focusing the evaluation problem;

collecting and analysing empirical data; and
communicating findings to evaluation audiences.

There is more than one appropriate sequence for implementing these activities, and any such
sequence can (and sometimes should) be repeated several times during the life span of an
evaluation study.
8
What methods of enquiry should be used in evaluation?
Being a complex task, evaluation needs to mobilise many alternative methods of enquiry from
the behavioural sciences and related fields of study and utilise them according to the nature
of a specific evaluation problem. At the present state of the art, an a priori preference for any
specific method of enquiry is not warranted.
9
Who should do evaluation?
Evaluation should be conducted by individuals or teams possessing
(a)
(b)
(c)
(d)

extensive competencies in research methodology and other data analysis techniques;

understanding of the social context and the unique substance of the evaluation
object;
the ability to maintain correct human relations and to develop rapport with individuals
and groups involved in the evaluation; and
a conceptual framework to integrate the above-mentioned capabilities.

10
By what standards should evaluation be judged?
Evaluation should strike for an optimal balance in meeting standards of
(a)
(b)

utility (to be useful and practical);

accuracy (to be technically adequate);

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 29

(c)
(d)

feasibility (to be realistic and prudent); and

propriety (to be conducted legally and ethically).

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 30

Empiricist v Rationalist approaches to the curriculum

For centuries and throughout the world, education has been dominated by either the
Empiricist tradition (ie physically or empirically - observing by the senses) or the Rationalist
tradition(recognising humanities intellectual abilities which may not by observable).
England & the USA have had a strong empiricist tradition (eg Hume & Dewey) and curricula
have tended towards the empiricist idea with evidence for achievement being extremely
important (for examples portfolios of evidence in NVQs
Europe has tended towards the rationalist tradition (eg Voltaire) and therefore baccalaureate
type curricula with value placed upon reasoned argument have been strong
Consider a curriculum with which you are involved. Plot it against each of the empiricistrationalist criteria by ticking a box.
1

The Empiricist Tradition

The Rationalist Tradition

Proof by evidence of the

senses eg portfolios of
evidence!

Proof by logic &

understanding eg rationally
argued essays!

Behaviourist Principles
[lessons based on objectives
and outcomes!]

Gestalt Principles linking

ideas to form patterns
[experiential reflective
learning]
Stimulus to gain Insight
Eureka
[intrinsic motivations!]

Constant Reinforcement
[Extrinsic motivations!]
Mastery Learning
[Competence assessment]

Conceptual Learning
[assessment of
understanding]

Ability demonstrated by
performance

Ability demonstrated by
explanation/ critical evaluation

Motivation assumed to be
External (Extrinsic)

Motivation assumed to be
Internal (Intrinsic)

Lots of direct instruction (the

teacher as expert informing the
unknowing learners)

student centred learning

(learners creating their own
understanding

NVQ & GNVQs /key skills type

programmes

Academic/recreational type
programmes

Lots of Internal and external

verification of assessors
marking

Internal and external

examinations of students
understanding

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 31

Change - the five categories of product adopters:

Innovators venturesome, educated, multiple info sources;
Early adopters social leaders, popular, educated;
Early majority deliberate, many informal social contacts;
Late majority skeptical, traditional, lower socio-economic status;
Laggards neighbours and friends are main info sources, fear of debt.
Rogers E (1962) Diffusion of innovations
Rogers stated that adopters of any new innovation or idea could be categorized as
innovators (2.5%),
early adopters (13.5%),
early majority (34%),
late majority (34%)
laggards (16%)
Reading List
Bennet N, Glatter R and Levacic R (1994) Improving Educational Management through
Research and Consultancy Chapman
Hopkins, D. (1993) A Teachers Guide to classroom Research OUPress
Kirkpatrick, D. (1998) Evaluating Training Programs: The Four Levels. 2nd ed.
Osgood, C. & Tannenbaum, P. (1957) The Measurement of Meaning University of illinois
press
Stenhouse, L. (1975) An Introduction to Curriculum Research and Development Heinemann
Stufflebeam, D. (1971). Educational Evaluation and Decision Making
Stufflebeam, D.L. Madaus, G. F. & Kellaghan, T. (Eds.), (2000) Evaluation models (2nd ed.).
Boston: Kluwer Academic Publishers.

Exeter PGCE (PCE) - Curriculum Evaluation Handbook: Page 32