Internal Quality Assurance from a HEI Perspective Uppsala University

2nd Conference on Internal quality assurance at higher education institutions, 1 Dec. 2006, Bern
1 Background of the quality system at UU and the national demands 2 Main features of the quality system at UU

3 Some examples, outcomes and benefits of the system at UU

4 Some conclusions
Annika.Lundmark@uadm.uu.se

Main University Building

Uppsala University some facts

Oldest university in the Nordic countries - founded in 1477 6 000 employees - including 4, 000 teachers/researchers Education and research across nine faculties

Three disciplinary domains:

Arts and Social Sciences Science and Technology Medicine and Pharmacy

Tradition-drenched student activities

Interdisciplinary campus areas

Education
40,000 undergraduate students More than 40 programs of study 1,800 single-subject courses 2,500 graduate students Student exchange programs with 400 universities in 40 countries

Organisation

Background of the actual quality system at UU

1995 1993-98
National Agency for Higher Education established

1st central quality group at UU

1996
1999-2000

1st quality audit at UU

2nd central quality group & 2nd quality audit at UU

1999-2001 2001-2006

SAUNA an internal quality evaluation The National Agency policy changes from quality audit to quality assessment of all subjects and programs
A new quality and evaluation unit 3d central quality group The actual quality enhancement policy

From 2001 From 2002

Some demands from the National Agency

1. Quality audits at all institutions between 1995 and 2002

2. Quality assessments of all subjects & programs between 2001 and 2006 The six year plan
3. Evaluations with focus on specific aspects, e.g. - students influence, internationalization 4. Both quality audits and quality assessments between 2007 and 2013 A new six year plan will be decided later this year Perhaps including the possibility to apply for honour for an excellent educational environment

Our view of the National Agencys six year plan between 2001 and 2006
Positive effects and conclusions:

UU has been successful The NA has focused on important national problems,

e.g. the situation for small subjects, lack of resources Negative effects and conclusions:

Sometimes (often?) a heavy burden for departments Lack of clarity of the aims and focus
The usefulness for departments and programs?

the self evaluation the site visit

To discuss..
What are your own experiences of quality audits and/or quality assessment concerning the effects on quality, quality assurance and quality enhancement? Are your experiences the same as the experiences at Uppsala university? Or are your experiences quite different? If YES: In what ways?

The concept of quality

You know it when you see it
Quality is a concept with different potential meanings 1. The standard of something when it is compared to other things like it; how good or bad something is High standard a thing that is a part om a persons character, especially something good

2. 3.

4.

a feature of something, especially one that makes it different from something else

Oxford Advanced Learners Dictionary

Quality
depends on the eye of the beholder
develops and changes over time cannot be described in a subtle way using quantitative data

can focus different aspects: prerequisites, processes, results and effects

is a relative concept - fitness for purpose - value for money will have different concrete meaning depending upon what you focus, e.g. studentsview of a certain course or the development of a new vocational program

The actual quality assessment system at Uppsala university

* * * * * *
The nine faculties are responsible for their quality and quality assurance in education and research Decisions in the University Board concerning infrastructural matters and resources A long term policy for quality enhancement in education and research An action plan for central quality enhancement has been decided each year Central unit for quality and evaluation and a quality committe 2006: - evaluation of research at UU - work on a new quality enhancement policy - quality assurance will be more integrated with the operational planning & annual reporting

Some values and conditions of quality assessment at UU

Quality is created and shaped
in faculties and departments by teachers and students

The central strategies and actions must

reinforce a culture of quality enhancement

The Vice Chancellors engagement is essential

for a well-functioning quality system

An active student union enhances work with

quality issues

Some external demands are good for

enhancing the internal quality work

Internal resources and quality work

Central level A quality committe Units for quality & evaluation, for teaching & interactive learning, for management training etc. Policies, e.g. Quality Enhancement Program Educational policy Projects, e.g. Teaching portfolios Training, e.g. teacher training ACTION ORIENTED EVALUATIONS Different activities on quality issues, e.g. analyses of resource allocations, quality enhancement etc.

Faculty level

Departmental Different projects level Program- & subject- course evaluations

Aims and objectives for the unit for quality and evaluation
Policy work together with the quality committ,
e.g. this year work on Bologna issues

Implementation of the actual action plan;

among others action oriented evaluations, e.g. this year reporting on an PhD-alumni evaluation

Support and consultative work

to faculties, departments and programs, e.g. this year - the faculty of social science webbased course evaluations - the faculty of pharmacy a webbased evaluation of the students views of each semester in programs - together with the Student Union: how to prevent dropouts

Covering the national and international discussion

The National Agencys Quality Assurance of Subjects and Programmes every year 20012006 Audit of UU

2001

2002

2003

2004

2005

2006

Unit for Quality & Evaluation

- Alumni evaluations of all subjects & programs

evaluation of PhD-studies PhD-alumni evaluation

- Consultations & ACTION ORIENTED evaluations

The National Agencys Quality Audit & Quality Assessments 2007-2013 Audit of UU

2007

2008

2009

2010

2011

2012

2013

Unit for Quality & Evaluation

- Continued Alumni evaluations and probably a second set of PhD-evaluations

Action oriented evaluations

Faculty Responsible for the quality level Demands for quality enhancement in the operational plan for all faculties a certain year
Reporting on quality enhancement in the annual report

Undergraduate alumni evaluations

1. Selection of approx. 6000 students who have graduated at least two years earlier in subjects or programmes assessed by the National agency a certain year, e.g. 2006: nursing, pharmacy, law, history of art am.others Questionnaires are sent by mail Separat more than 80 reports to all departments of the specific subject/program A final report on the results of the different faculties

2. 3.

4.

Undergraduate alumni evaluations: the Questionnaire

Background questions
- Sex, age, degree

Position in the labour market

- How long time until employment? Working position, tasks, salary etc.

Evaluation of the education

- Attainment of goals set for higher education in the the Higher Education Act and goals set by the Faculty Board - Attainment of skills, e.g. communication skills - What the students were especially satisfied and dissatisfied with

The demands of the present employment

- To what extent do the respondents use certain skills? - The usefulness of the education

Forms of teaching, learning and study work

Independent work Written individual PMs Lectures Group work Satisfactory extent Less of Oral presentations Seminars Study visits etc. No experience

More of

Groupdiscussions Laboratory work Problembased learning 0% 20% 40% 60% 80% 100%

Part of working time in the relevant educational field

Total

All the time

Nursing/medicine Pharmacy Law Theology Engineer Civil engineer Science & technology Social sciences Arts Languages

More than half the time Less than half the time Not at all

0%

20%

40%

60%

80%

100%

Undergraduate alumni evaluations cont.

Some more examples of the results:

nine out of of ten students would have chosen UU again many students satisfied with teaching & learning in
problemsolving & written work

about six out of ten students dissatisfied with the training

in oral communication Some outcomes and benefits:

faculties, departments, programs and teachers get

more interested and find evaluations useful

quality enhancement in different areas

The value of the alumni evaluations?

an interview study with responsible professors/lecturers at twelve different departments and programs
the most interesting parts of the reports, e.g. the most interesting results, are - the employment situation for the alumni - how the alumni value the outcomes and their teaching and learning experiences - the appendixes in which the answers from every alumni on all open ended questions are described, e.g. current employment especially valuable in the education what was missing in the education

The value of the alumni evaluations? cont.

The usefulness of the report:

very useful when

- doing the self assessment - working with the Bologna process

the report actualizes both new and old questions on the

agenda for further discussion

gives arguments for changes gives suggestions for concrete action confirmation that the quality enhancement and/or
course/program planning is working well

gives much information to use in study counselling

To discuss
The alumni evaluations we have implemented are ACTION ORIENTED: Which are your own experiences of the value of alumni evaluations? Can you see clear advantages and disadvantages with these centrally implemented alumni evaluations?

Evaluation of PhD-studies
1. Questionnaires to all PhD-students
importance of and satisfaction with working & research conditions * importance of and satisfaction with supervision & dissertation work * courses * pedagogical experience & training
*

2. Separate reports of the results to each department

Evaluation of PhD-studies cont.

Support of the supervisor (percent)
Very important resp. very satisfied (5) 4 3 2 Unimportant resp. very dissatisfied (1) Importance Satisfaction

20

40

60

80

3.

Departments: self-assessment

4.
5.

Analysis of results and summary report for all faculties at institutional level Faculties: analysis and suggestions for enhancement in each faculty

Evaluation of PhD-studies: some outcomes and benefits

Some results:

big differences between departments most PhD-students satisfied with physical aspects of
their working environment

some students dissatisfied with some aspects of

supervision & support of their own lecturing Some outcomes and benefits:

Department management and teachers

are interested and find the report useful

Quality enhancement in departments and

faculties

Evaluation of PhD-alumni
1. Questionnaires to all PhD-alumni who graduated between 1997 to 2001
Current relation to Uppsala university * Labour market placement * Main working assignments * To which extent is research included in the assignment? * Usefulness of the PhD-education * Satisfaction and demands with the PhD-education in different aspects, e.g. critical scientific thinking, working in a project group (a total of 18 aspects)
*

2. Separate reports of the results to each faculty

Evaluation of PhD-alumni cont.

Demands and satisfaction: generic skills (Faculty of Medicine)
Satisfied
Written presentations Written presentations

Demands

OralOral presentations presentations

Giving arguments Giving arguments

Workingin inproject projectgroups groups Working

Some conclusions
Necessary with some external demands, especially in the
initial work of building an internal quality system

Important that as many teachers and students as possible

are involved in some elements of the quality system (incl. pedagogical training) and find it useful

The University management & University Board must be

involved both in discussions and decisions and find this work useful

It is important to make the quality work visible Quality enhancement should be evidence based
(Action oriented evaluations)

The quality system must be well known and it is very

important to have quality in the quality work to strengthen the legitimacy of the system

Thank you for your attention

www.uadm.uu.se/kvalitet Annika.Lundmark@uadm.uu.se