Quality Assurance In Higher Education

Program Assessment and Continuous Improvement of Quality Day 2

Iram Sohail

What is Quality?

Defining Quality
A survey of literature reveals quality as one of the widely defined terms:
High degree of goodness (Longman Dictionary of Contemporary English) A degree or level of excellence (The Oxford Large Print Dictionary) The standard of excellence of something often a high standard (Cambridge International Dictionary of English) Doing the right things right (W. Edwards Deming Pioneer of the quality movement in industry)

What is Quality?
It has to do something with a certain degree of effectiveness or excellence and satisfaction In academic scenario it means satisfaction of all the stakeholders
Whether the satisfaction of all the stakeholders is relevant to your programme or not........

Four fold typology to help understand the term quality as applied in Higher Education Sector Quality as excellence Quality as fitness for purpose Quality as value for money Quality as transformation

What is Quality Assurance?

A means by which managers satisfy themselves that control mechanisms are working to maintain and enhance standards Refers generally to all planned and systematic actions or processes necessary for providing adequate confidence to managers that a product or a service will satisfy the specified requirements for quality

Key Components and Specific Terms of Quality Assurance

Quality Design Quality Control Quality Improvement Quality Enhancement
Quality Management

Quality Audit

Quality Assessment

Quality Transformation

A sustainable quality assurance programme enhances employment opportunities Improves the education and training of future employees harnesses future leaders facilitates an enabling learning environment enriches the academic and intellectual landscape

 QECs serve as focal points for quality assurance in the institutions in order to improve and uphold the quality of higher education Capacity building of academia in quality assurance is one of the key functions of QAA and subsequently of QEC

What is Assessment?
Assessment is a systematic process of gathering, reviewing and using important quantitative and qualitative data and information from multiple and diverse sources about educational programs, for the purpose of improving student learning, and evaluating whether academic and learning standards are being met.

Self Assessment
Self-assessment is an important tool for quality assurance and provides feedback for the management to initiate action plans for improvement The objectives of self assessment are to:
Maintain and continuously enhance performance Enhance effectiveness of the output Verify that the existing activities meet their objectives and institutional goals Provide feedback for quality assurance

Faculty Involvement
Academic Program Assessment, to be most effective requires the participation of all faculty members

Each department appoints a Program Team (PT) which will prepare a Self Assessment Report (SAR) demonstrating the conformance of the program to all the relevant criteria and standards as stipulated in the Self Assessment Manual(SAM), in coordination with the members of Quality Enhancement Cell (QEC)

Size of PT can be determined according to the size of the department

Sr. # Number of Students in a Department Up to 300 300 - 1000 1000 and above PT Members

01 02 03

3 PTM + 1 Chairperson 5 PTM + 1 Chairperson 6 PTM + 1 Chairperson

Desired Qualities of PT
Commitment to the principles of quality in higher education and the policies set by the Higher Education Commission of Pakistan, provided in the form of SAM. Ability to work in teams. An enquiring disposition, power of analysis and judgment. Experience of organization and management, particularly in relation to teaching and learning matters, time management skills. High standards of oral and written communication skills.

 To be proactive than reactive Initiate improvements to achieve academic excellence Systematize the process of self assessment To be current and take a leadership role

Criteria for Self Assessment

The self assessment of an institution / organization is based on several criteria To meet each criterion a number of standards must be satisfied The following part of this presentation proposes criteria and standards for self assessment by Programme Team The scheme is based on the HEC self assessment criteria

The Elements of A Successful Assessment

Purpose identification
Outcomes identification Measurements and evaluation design Data collection Analysis and evaluation

Decision-making regarding actions to be taken

Process of Generating SAR

1
2 3 4 5

QEC initiates Self Assessment exercise

Department forms Program teams, who will be responsible for preparing and generating Self Assessment Report (SAR)

QEC reviews the Final SAR in one month

If SAR is declared complete by QEC it moves to next step, other wise it goes back to step 2

Vice Chancellor forms Assessment Team in consultation with and recommendation of QEC

6 7 8

QEC plans and fixes AT visits

AT conducts assessment and presents its findings to QEC

QEC submits the Executive Summary to the Vice Chancellor along with AT findings, SAR and corrective measures for the department (Table A2)

QEC performs follow-up of corrective measures

QCs
Assessment Team joins here

This all is given in SAM. Therefore reading SAM with an understanding and application approach will make the SAR generating process, easy and smooth.

Corrective Measures

Components of Self Assessment Process

Eight Criteria for Self assessment QC1: Programme Mission Statements, Objectives and Outcomes QC2:Curriculum Design and Organization (4) (7)

QC3:Laboratories and Computing Facilities

QC4:Student Support and Advising QC5: Process Control

(3)
(3) (5)

QC6:Faculty
QC7: Institutional Facilities QC8:Institutional Support

(3)
(3) (3)

Benefit of an Academic Program Assessment Process

Identify program weakness Inform financial decisions based on academic priorities Provide information to constituents on the quality of the education students receive

Ensure continuous improvement of programs and curricula

Process Improvement
Steps involved in process improvement are
Defining mission statements Defining program objectives Defining program outcomes Collecting data

Analyzing/interpreting data
Taking corrective action with a view to remove deficiencies.

Why do we need a Program Mission Statement?

Mission statement is the guiding philosophy of all activities.

Provide the foundation which supports all other aspects of program assessment. Clarify the program to all stakeholders (faculty, staff, student, alumni, employers, potential donors, etc.), allowing programs to focus their resources and efforts on issues that are critical to the mission.

What is Mission Statement????

It is a brief description of an institutions fundamental purpose and articulates the rationale of its existence to the stakeholders. At the very least the mission statement must convey the institutions purpose in a way that inspires commitment, innovation and courage.

The Programme Mission Statement should

be in line with Universitys Mission Statement express your faculty/department/programmes purpose in a way that inspires support and ongoing commitment motivate the stakeholders be convincing and easy to grasp use proactive verbs to describe what, why and how of being or its existence be free of jargon be short enough so that anyone related to the faculty/department/programmes can repeat it

How to Write/Revist a Mission Statement

Step 1 (Self Assessment) Does the mission statement communicate your purpose?

Is it short and concise?

Is it specific enough to be distinctive? Does it give you guidance to determine your programmes? Is it inclusive of all your activities? If not, do you need to broaden your mission or narrow your programmes? Does it inspire you?

Step 2 (Process to create or review mission statement)

To ensure relevance, it is important to review the mission statement periodically Review every three years or whenever there is significant change Engage faculty members, programme team members or key volunteers in the process At onset, clarify roles Typically the Head will play a leadership role and is responsible for approving/adopting the final mission statement

Step 2 contd.
(Process to create or review mission statement)

Consultative Working Group meeting is effective to develop or update a mission statement, but final fine tuning is best done by an individual or smaller sub committee The mission statement cannot be developed in a single meeting A neutral facilitator/moderator can be helpful to bring the discussion to resolution

Group Activity # 1:
Instructions:

45 minutes

1. Arrange yourselves in five groups (3 members in each group) 2. Develop a department and programme mission statement keeping into account the steps mentioned earlier 3. The groups should select from the following disciplines Science and Technology, Arts and Social Sciences, Health Sciences, Engineering, Languages etc 4. The group members will identify the group leader and present the mission statements

Sample Mission Statements

Our mission is to provide a platform to talented youth of the nation where they can interact with internationally leading scientists, to acquire firsthand knowledge and explore new horizons of useful research (ASSMS).
The mission of IIB is to endeavour for the development of human resource and sustainable bioprocesses in the integrated fields of Microbiology and Biotechnology. (IIB)

Sample Mission Statements (contd.)

SDSC is committed to mobilize social sciences, and science & technology for managing boundaries between knowledge and action and to address sustainable issues in Pakistan. (SDSC) To raise the quality of teaching and training of professional Clinical Psychology to the highest level and develop into an indigenous discipline through empirical research. (Clinical Psychology) To develop effective player-managers and leaders, well rounded in modern management concepts and techniques contributing to the organizations they serve, and to the societies in which they live (MSD)

Criterion 1: Program Objectives and Outcomes

Thinking About Outcomes The Process: Planning with the end in mind Design down .. Deliver up

Program outcomes
Course outcomes Unit outcomes Lesson outcomes

Objectives of the Session

Participants will be able to:
Understand what a learning objective is and why they increase the effectiveness of training. Compare and contrast learning goals and learning objectives. List the parts of the ideal learning objective. Understand about learning domains, Blooms Taxonomy and their levels of learning. Connect the verbs associated with each level in Blooms taxonomy.

Meeting Standard 1-1

Document institution, mission statements. State program objectives. departmental and program

Describe how each objective is aligned with program, departmental and institution mission statements.
Outline the main elements of the strategic plan to achieve the program mission and objectives. Provide for each objective how it was measured, when it was measured and improvements identified and made

Why Use Learning Objectives?

By defining where you intend to go, you increase the likelihood that the learner ends up in the intended destination.

Guides the learner, helps his/her focus on what needs to be learned, and sets priorities.
Shows the learner what behaviours are valued. Focuses and organizes the instructor. Creates the learners basis for self-assessment.

Effective learning objectives

Are learner-focused (not instructor or content focused) Focused on the intended learning that results from an activity, course, or program Focuses on skills and abilities central to the discipline and based on professional standards

Learning Objectives
1. A measurable verb

(Mager,1975)

Learning objectives include:

One task or behavior per verb Choose the verb that best describes the type of behavior or task the learner must display after training How the task or behavior will be performed Under what conditions will the task be performed How well the task or behavior must be performed to meet the standard

2.

The condition

3.

The standard for acceptable performance

The ABCD Method

A = Audience (resident students, students in a class, members of a
club, participants in a workshop) What population are you assessing?

B = Behavior (remember, observable and measurable demonstrate, select, categorize, participate)

What is expected of the participant?

C = Conditions (upon completion of the program, at the end of the semester, given a calculator)
Under what circumstances is the behavior to be performed?

D = Degree (this is how you know if the student succeeded)

How well must the behavior be performed? To what level?

Writing Learning Objectives

Understand your Audience

Consider the characteristics of your audience when writing learning objectives.
What they will be expected to do after the training Their physical, mental and thinking abilities Their educational level, skills and abilities How they best learn What they need to know to do the job What they already know about the job Have they been trained before

Learning Objective Categories or Domains

Types or Domains of Learning Objectives:
Cognitive (knowing) Psychomotor (doing) Affective (feeling)

Levels of Learning Objectives

(Blooms Taxonomy)

Benjamin Blooms Taxonomy has been around since the mid-1950s. His structure for thinking behaviors provides a nice stair-step approach to thinking about the levels of learning.

Blooms Taxonomy Levels

Level Knowledge Comprehension Application Analysis Description Focuses on memorization and recall Focuses on understanding the information memorized Focuses on being able to apply what is understood Focuses on being able to take apart and use critical thinking skills to understand what was applied

Synthesis

Taking what is known and has been applied and using it in different ways.
Assessing what has been applied and providing feedback on how the task is completed.

Evaluation

Writing Learning Objectives

A well-stated objective provides a clear picture of the outcome or performance you expect as a result of the lesson. It should be specific, concise, and most importantly, observable or measurable.

48

Relating the Measurable Verb to Blooms Levels

Verbs Taxonomy

Evaluate Design Distinguish Apply Explain Define

Evaluation Synthesis Analysis Application Comprehension Knowledge

49

Start All Objectives with Verbs

Level Appropriate Verb for Level
Define Memorize List Recall Recognize Repeat Related Record Name Identify Acquire Underline Label State Relate Order Restate Discuss Describe Identify Locate Report Explain Express Recognize Review Transform Represent Select Tell Indicate Translate Interpret Apply Practice Illustrate Operate Convert Explain Demonstrate Dramatize Sketch Employ Schedule Use Sequence Prepare Predict Generalize Implement Plan Show Solve Complete Distinguish Differentiate Appraise Analyze Calculate Criticize Estimate Discover Order Compare Contrast Examine Test Relate Experiment Investigate Question Detect Break down Contrast Diagram Debate Examine Classify Categorize Determine Inspect Inventory Compose Plan Propose Design Assemble Create Write Prepare Formulate Organize Manage Construct Set-up Systemize Arrange Collect Construct Organize Systematize Argue Conclude Create Integrate Theorize Combine Improvise Manage Specify Derive Set up Judge Appraise Measure Value Estimate Choose Compute Assess Test Evaluate Revise Score Select Rank Check Defend Verify Justify Criticize Rate Support Weigh

Knowledge
Comprehension Application Analysis

Synthesis

Evaluation

Group Activity # 2
Instructions:

Time: 1 hour

1. Arrange yourselves in five groups (3 members in each group) 2. Develop a programme objectives keeping into account the Blooms Taxonomy and the level of Cognitive domain. 3. The groups should select from the following disciplines Science and Technology, Arts and Social Sciences, Health Sciences, Engineering, Languages etc 4. The group members will identify the group leader and present the programme objectives

Sample Programme Objectives BSc (Hons) in Environmental Science

By the end of this programme the students will be able to 1. Understand the intricate linkages within and between biophysical and socioeconomic systems, and appreciate the principles and requirements that facilitate the transition to sustainability within these systems. 2. Apply theoretical understanding, professional judgement and skills in mitigation of environmental problems 3. Formulate and implement solutions to problems of sustainable development through the use of analytical skills and theoretical knowledge. 4. Make meaningful contributions to improving legal and administrative structures and processes relevant to sustainable development and environmental management.

Program Outcome
A program outcome is a specific, measurable, statement of what a student should know, be able to do, or value when they complete a program, course or sequence of courses / experiences / activities.

Characteristics
Learning outcomes are what result from a learning process. They are specific measurable achievements and are stated as achievements of students These should specify the minimum acceptable standard for a student to be able to pass a programme or course

Clearly stated outcomes can help Students understand what is expected of them Teachers focus on precisely what they want the students to achieve

How to Write Program Learning outcomes

Well written learning outcomes include the following characteristics
They specify what the student must be able to do They are achievable within the time and resource limitation of the programme The specified action is assessable (i.e. Measurable) The greater the synergy between learning outcomes, teaching strategies and assessment techniques the more successful learning process is likely to be.

Guidelines for writing learning outcomes

Try to use verbs
Ensure that each learning outcome is acquirable Avoid complicated sentences. If necessary, use more than one sentence for clarity. When writing learning outcomes in cognitive domain, avoid overuse of knowledge or comprehension based verbs. Try to include some outcomes based on application, analysis synthesis and evaluation

Benefits of Learning outcomes

Program improvement Evaluation of instruction Course design and revision

Curriculum assessment and change

Improved Communication Advising Tools Improving promotional materials Targets for assessment and accreditation

Group Activity # 3
Instructions:

Time: 1 hour

1. Arrange yourselves in five groups (3 members in each group) 2. Develop a programme learning outcomes keeping into account the Blooms Taxonomy, learning process and the level of Cognitive domain. 3. The groups should select from the following disciplines Science and Technology, Arts and Social Sciences, Health Sciences, Engineering, Languages etc 4. The group members will identify the group leader and present the programme learning outcomes.

Sample Learning outcomes

On successful completion of this programme the students should be able to Apply multidisciplinary knowledge of environmental science to solve environmental problems. Use modern tools, techniques and skills necessary for working as an environmentalist in various capacities. To communicate effectively and professionally in written, oral and graphical. independently design and carry out experimental research and evaluate the results critically.

Any Questions

For Quality Practitioners:

Be patient but consistent

you can do it !!!

Thank You & Good Luck!!!!