You are on page 1of 31

This article was downloaded by: [University of Wyoming Libraries]

On: 30 September 2013, At: 22:01


Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Total Quality Management & Business


Excellence
Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/ctqm20

Relationships between the EFQM model


criteria: a study in Spanish universities
a a a
Arturo Calvo-mora , Antonio Leal & Jos L. Roldn
a
Department of Business Administration & Marketing, Research
Group on Innovation, Quality and Change Management, Faculty of
Business and Economic Science, University of Seville, Spain
Published online: 24 Jan 2007.

To cite this article: Arturo Calvo-mora , Antonio Leal & Jos L. Roldn (2005) Relationships
between the EFQM model criteria: a study in Spanish universities, Total Quality Management &
Business Excellence, 16:6, 741-770, DOI: 10.1080/14783360500077708

To link to this article: http://dx.doi.org/10.1080/14783360500077708

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the
Content) contained in the publications on our platform. However, Taylor & Francis,
our agents, and our licensors make no representations or warranties whatsoever as to
the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources
of information. Taylor and Francis shall not be liable for any losses, actions, claims,
proceedings, demands, costs, expenses, damages, and other liabilities whatsoever
or howsoever caused arising directly or indirectly in connection with, in relation to or
arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Conditions of access and use can be found at http://www.tandfonline.com/page/terms-
and-conditions
Total Quality Management
Vol. 16, No. 6, 741 770, August 2005

Relationships between the EFQM Model


Criteria: a Study in Spanish Universities

N
ARTURO CALVO-MORA, ANTONIO LEAL & JOSE L. ROLDA
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

Department of Business Administration & Marketing, Research Group on Innovation, Quality and Change
Management, Faculty of Business and Economic Science, University of Seville, Spain

ABSTRACT In the literature on quality management it is necessary to undertake empirical studies


that go further into the knowledge and understanding of the relationships between the key
implementation factors and the results. This work falls within that context and its reference is the
EFQM Excellence Model and the field of higher education. The structural equations method is
used and, more specifically, the Partial Least Squares (PLS) technique. The results from the
analysis of the measurement and structural model support the reliability and validity of the
European Excellence Model as a reference framework for the implementation, evaluation and
improvement of quality in the area of higher education. The need to consider the agents together
as the determinants of the results is also revealed, as well as there being an internal logic that
connects the models results. Moreover, the analysis has allowed us to conduct an in-depth study
of the causal structure of the EFQM Model. An understanding of this structure may allow centres
of higher education to direct their management towards the achievement of excellent results.

KEY WORDS : EFQM model, higher education, universities, partial least squares

Introduction
It is inevitable for quality management processes, which have helped to transform
businesses and overcome many of their problems, to be transferred to the field of education
(Peak, 1995). Moreover, the rapid and continuous changes taking place in the economic
and social environment require the educational and management models of this kind of
institution to be modified (Salmon, 1993; Schargel, 1995).
Different approaches have been adopted for the introduction of quality management in
universities, such as self-assessment and external assessment of the institutions, accredita-
tion systems or models of total quality management. The latter represent a strategic option
and an integrated management philosophy for organizations, which allow them to reach
their objectives effectively and efficiently and to achieve a sustainable competitive

Correspondence Address: Arturo Calvo-Mora, Departamento de Administracion de Empresas y Marketing,


Facultad de Ciencias Economicas y Empresariales, Av. Ramon y Cajal n81, 41018, Sevilla, Spain. Email:
schmidt@us.es

1478-3363 Print=1478-3371 Online=05=06074130 # 2005 Taylor & Francis


DOI: 10.1080=14783360500077708
742 A. Calvo-Mora et al.

advantage (Goldberg & Cole, 2002). Their implementation has taken the European Excel-
lence Model of the European Foundation for Quality Management (EFQM) as a European
context point of reference.
In Spain, the First National Plan for the Assessment of Quality in Universities, in 1995,
and the Second Plan, in 2002, aimed to establish a series of indicators that give infor-
mation about the quality level of processes, products and services in centres of higher
education. Three assessment guides, known as protocols, emerge from these plans for
teaching, research and management (the latter clearly inspired by the EFQM Model).
This study has the following three aims. First, to obtain a model that will serve as a
reference for centres of higher education in order to evaluate and improve the quality of
the management and provision of services. Second, to analyse the validity and predictive
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

power of the EFQM Model adapted to the university sphere. Third, to test the relationships
implicit in this model since, as Black & Porter (1996) or Westlund (2001) point out, it is
necessary to perform more empirical research to explore more deeply the links between
the agents that compose the EFQM Model and the results.
To achieve these objectives, this study begins with a description of the EFQM Excel-
lence Model and the empirical work that has tried to verify the connection between the
key implementation factors and the results. This analysis, along with the EFQM Model,
will serve as the basis for establishing our research model and hypotheses. Finally, the
research methodology followed and the principal results and conclusions arising from
the study are presented.

The EFQM Excellence Model


The EFQM is a non-profit organization created in 1988 by 14 large European companies.
Its mission is to promote excellence in European companies in a sustainable manner, its
vision being a world in which companies stand out for their excellence.
The EFQM Model is a non-prescriptive framework, which recognizes that excellence
may be achieved in a sustainable manner through the adoption of different approaches
(Rusell, 2000). Within this framework, there are certain fundamental concepts that are
expressed and specified in nine dimensions or criteria, which serve as a guide for imple-
menting total quality management (hereafter TQM), and for measuring the results that are
being achieved by the organization (Lau & Anderson, 1998). These nine dimensions are,
in turn, divided into five key implementation factors or enablers and four result types in
order to measure excellence (Figure 1).
The outline above is based on the assumption that there are interrelationships between
agents and results, although the model does not explicitly establish what they are. Thus,
there are studies that have tried to probe and verify these relationships, but research in
this direction is at an early stage and more exhaustive empirical studies are required
(Eskildsen et al., 2001).
The European Excellence Model is a useful work tool that serves as a stimulus to guide
organizations towards both quality and the customer. Whats more, it is about an approach
that goes further than a simple self-assessment tool (Ghobadian & Gallear, 1997;
Ghobadian & Woo, 1996; Hewitt, 1997), since, by presupposing the existence of inter-
relationships between agents and the results of the model, somewhat more than an additive
relationship is proposed. This focus may also be applied to any field and type of
Relationships between the EFQM Model Criteria 743
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

Figure 1. The EFQM excellence model

organization, that is to say, not only to private companies (Zink & Schmidt, 1995; Osseo-
Asare & Longbottom, 2001).

Literature Review
Much of the literature on TQM is of a descriptive nature and lacking in consistent research
that addresses the relationships that may exist between quality practices and between the
latter and the results. Only recently have there been attempts to identify these relationships
and to advance the development of a better knowledge and understanding of quality
management (Kannan et al., 1999: 35).
Specifically, up to the beginning of the 1990s, the literature contains numerous real
cases of companies who have been successful in implementing these management prac-
tices, especially in Japan and North America (Porter & Parker, 1993). Other studies
describe the basic concepts and the management practices that must guide this phenom-
enon and propose particular processes for quality implementation and improvement, as
well as the results obtained or which should be obtained. On this point, practically all
of the great authorities on the subject have declared their position, such as Deming,
Crosby and Juran.
However, prior to the study by Saraph et al. (1989), there was no systematic and scien-
tific attempt to organize and synthesize the large amount of material that has been written
about quality. Therefore, it was important to develop and advance knowledge of the oper-
ation of quality management via empirical research. Such studies would allow the deter-
mining of possible relationships between the critical implementation factors and, at the
same time, identify those that are the most important in order to achieve better results.
744 A. Calvo-Mora et al.

Pursuing this theme, some studies, such as those by Ahire et al. (1996), Anderson et al.
(1994; 1995), Curkovic et al. (2000), Dow et al. (1999), Flynn et al. (1994; 1995), Kanji
(2000), Powell (1995) and Ravichandran & Rai (2000), start from the identification of the
key TQM dimensions after an exhaustive analysis of the literature. Next, they construct
valid and reliable scales to measure them. Finally, the possible relationships between
the dimensions and the measurements of results are analysed. Another group of studies
is formed by those whose objective is to verify the relationships established in excellence
models in different contexts such as Black & Porter (1996), Eskildsen & Kanji (1998),
Flynn & Saladin (2001), Pannirselvan & Ferguson (2001), Wilson & Collier (2000) or
Winn & Cameron (1998). Our research belongs, therefore, to this last line or tendency
in TQM research.
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

Development of research model and hypotheses


Our initial research model is based on an adaptation of the EFQM Excellence Model. As
can be noted in Figure 2, the models enablers have been kept, with only a small alteration
made to the four types of results that it puts forward (compare Figures 1 and 2).
Thus, our model (Figure 2) proposes that excellent results with respect to students,
people, the centre and society are achieved through a leadership that directs and promotes
policy and strategy, people, partnerships and resources, and the key processes. In short, it
is accepted that the different criteria in the EFQM Model are interrelated, and that being
successful in some individual areas is not enough to achieve excellence (Naylor, 1999).

The Relationship between Enablers of the EFQM Model


Agents define what organizations do in order to achieve excellence. Specifically, it is to do
with activities related to the leadership of the directors, the management of human and
material resources, as well as process management. Moreover, these activities are not
independent: they must be implemented together and in a coordinated fashion in order
to achieve excellent results (Tamimi, 1998). Next, the variables that comprise the

Figure 2. Research model and hypotheses


Relationships between the EFQM Model Criteria 745

model are defined, the relationship (two by two) between these variables is established by
proposing hypotheses, and these hypotheses are supported by the theoretical and empirical
references provided by the TQM literature.

Leadership & Commitment: Excellent leaders develop and facilitate the achieve-
ment of the mission and vision. They develop organizational values and systems
required for sustainable success and implement these via their actions and beha-
viours. (EFQM, 1999: 12)

The managements commitment and leadership in quality must be visible, permanent


and present at all management levels (Dean & Bowen, 1994), since it acts as the guide
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

and promoter of the TQM implementation process. But to be successful one has to
move into action. Here, investment in human, material and financial resources supporting
the achievement of the objectives and the development of policies and strategies becomes
indispensable (Ahire et al., 1996). Moreover, the involvement of the entire workforce
must be fostered and their efforts directed towards improvement recognized (Yusof &
Aspinwall, 1999).
The above aspects are confirmed by empirical studies performed by some authors such
as Ahire et al. (1996), Anderson et al. (1995), Eskildsen & Dahlgaard (2000) or Flynn
et al. (1994), which demonstrate the significant positive correlation between leadership
and the other key TQM implementation factors. Wilson & Collier (2000), taking
Malcolm Baldriges model as a reference point, prove the relationship that exists
between leadership, human resource management, strategic planning and information
and analysis. Pannirselvam & Ferguson (2001) perform a similar empirical analysis on
a sample of companies and observe a significant relationship between leadership and
human resource management. In their analysis of the European Excellence Model,
Eskildsen & Dahlgaard (2000) find significant positive relationships between leadership,
people management, policy and strategy, and alliances and resources. Davies et al. (2001)
and Detert & Jenni (2000) undertake their study in the field of higher education and reach
this conclusion: the leadership must clearly establish and maintain the focus on stake-
holders, give priority to long term strategic planning, ensure an appropriate structure
and system for continuous improvement, motivate individuals and teams, and monitor
success. In short, leadership is the engine, the driver and soul of all aspects of the
quality system.
From the above, we can extract the following research hypotheses:

H1a: The leadership and commitment of the management have a positive influence on
people management.
H1b: The leadership and commitment of the management have a positive influence on
policy and strategy.
H1c: The leadership and commitment of the management have a positive influence on
partnerships and resources.

Policy & Strategy: Excellent organizations implement their mission and vision by
developing a stakeholder-focused strategy that takes account of the market and
sector in which it operates. Policies, plans, objectives, and processes are developed
and deployed to deliver the strategy. (EFQM, 1999: 14)
746 A. Calvo-Mora et al.

Policy and strategy must be put into practice through the deployment of the key pro-
cesses, suitable policy and staff management, and through the establishment of partner-
ships (Winn & Cameron, 1998).
The theoretical and empirical literature analysed focuses on the development
and implementation of specific quality policies and strategies (Quazi et al., 1998;
Ravichandran & Rai, 2000; Saraph et al., 1989), and on how these should be integrated in
the organizations policies and strategies (Black & Porter, 1996; Porter & Parker, 1993;
Wilson & Collier, 2000). Other studies make no explicit reference to policy and strategy
but analyse it as a fundamental aspect within other dimensions such as the shared vision
(Dow et al., 1999). This includes aspects such as the establishment in the organization of stra-
tegic planning processes, which encompass the points of view and requirements of internal and
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

external customers. For his part, Powell (1995) speaks of the adoption and communication of
the philosophy, which implies the introduction of TQM principles into the companys
mission, vision, strategies and policies. That is to say, it involves moving from ideas to
deeds, and is achieved via the appropriate execution of the organizations key processes.
Detert & Jenni (2000) speak of a system thinking, which requires all members of the
organization to take into account how their actions affect those of other people in the uni-
versity institution. In a centre of higher education, this overall vision may be demonstrated
using clear goals shared by all: professors, students and managers. These goals must take
shape in all the activities of the university via the strategic planning process (Zink &
Schmidt, 1995).
From the above, we can extract the following research hypotheses:

H2a: Policy and strategy have a positive influence on people management.


H2b: Policy and strategy have a positive influence on partnerships and resources.
H2c: Policy and strategy have a positive influence on process management.

People Management: Excellent organizations manage, develop and release the full
potential of their people at an individual, team-based and organizational level. They
promote fairness and equality and involve and empower their people. They care for,
communicate, reward and recognise, in a way that motivates staff and builds com-
mitment to using their skills and knowledge for the benefit of the organization
(EFQM, 1999: 16)

The importance of this yardstick is emphasized by Ahmad & Schroeder (2002) or


Mukherjee et al. (1998), who note that human resource management is the cornerstone
on which an important part of the success of TQM rests, since the quality improvement
process is one of organizational learning based on people. Essential activities for people
management include appropriate selection, reward and professional development (Flynn
et al., 1994), the establishment of training plans (Gatewood & Riordan, 1997), the com-
mitment to and involvement with quality (Ahire et al., 1996; Anderson et al., 1994), or
the establishment of an effective communication system (Zink & Schmidt, 1995). In the
education field, Detert & Jenni (2000) and Osseo-Asare & Longbottom (2002) emphasize
the role of training as a key factor and imply the continuous acquisition of new knowledge
and skills by all employees.
As far as the relationship between people management and the other key TQM
implementation factors are concerned, Ahire et al. (1996) find a positive relationship
Relationships between the EFQM Model Criteria 747

between practices related to people management (empowerment, commitment and train-


ing) and other quality strategies (product design, process management or the utilization of
internal and external information for quality), demonstrating the importance of this factor
for the successful implementation of TQM. On the other hand, Flynn et al. (1995) and
Wilson & Collier (2000) show how people or human resource management is related to
process management in a significant, positive manner.
In short, an appropriate personnel recruitment and selection policy, along with a work-
force that is trained, involved and committed to quality and to the improvement of the
activities of the organization must have an effect on the correct performance and the
improvement of the organizations key processes. This leads to the achievement of
better results.
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

From the above, we can extract the following research hypothesis:

H3: People management has a positive influence on process management.

Partnerships & Resources: Excellent organizations plan and manage external part-
nerships, suppliers and internal resources in order to support policy and strategy
and the effective operation of processes. During planning and whilst managing part-
nerships and resources they balance the current and future needs of the organiz-
ation, the community and the environment. (EFQM, 1999: 18)

The importance of relationships with suppliers and the management of tangible


and intangible resources is an aspect frequently addressed in the literature on quality
management (Curkovic et al., 2000; Dow et al., 1999; Eskildsen & Dahlgaard, 2000;
Ravichandran & Rai, 2000). In the field of higher education, resource and partnership
management it is not a subject that is particularly discussed in the literature. In spite of
this, universities, like any other organization, must optimize the scarce resources they
have, and appropriately manage the suppliers of specific inputs, which represent a signifi-
cant cost in budgetary terms (Osseo-Asare & Longbottom, 2002; Ward & Chandler, 1999;
Zink & Schmidt, 1995). This will lead to better and more efficient management of their
processes or key activities. Eskildsen & Dahlgaard (2000), in an empirical analysis of
the EFQM Model, find a significant positive relationship between partnership manage-
ment and key process management.
From the above, we can extract the following research hypothesis:

H4: Partnership and resources have a positive influence on process management.

Process Management: Excellent organizations design, manage and improve pro-


cesses in order to fully satisfy, and generate increasing value for, customers and
other stakeholders. (EFQM, 1999: 20)

Analysing the structure of the relationships in the EFQM Model, process management
appears to be the link between the other agents and the results. In the industrial sector,
process management means the performance of activities such as management, control
and improvement of design and manufacture, preventive maintenance of equipment,
the statistical monitoring of processes, reduction in the inspection and variability of pro-
cesses, etc. These aspects are positively related to particular measurements of the
748 A. Calvo-Mora et al.

organizations results, people and customers (Curkovic et al., 2000; Eskildsen &
Dahlgaard, 2000; Pannirselvam & Ferguson, 2001; Wilson & Collier, 2000). From the
university point of view, it is fundamental for these institutions to make efforts to ident-
ify their key processes and establish priorities for improvement in terms of their relative
impact on results. The key processes of a higher education centre will not be the same as
those in other organizations, but once they have been identified, there should be no
differences in terms of their management and improvement (Zink & Schmidt, 1995).
The key processes are considered to be those that have a significant effect on the critical
results for a given organization (Kanji & Tambi, 1999; Osseo-Asare & Longbottom,
2002). In the universities, these processes are identified by Zink & Schmidt (1995) as
the processes of administration and service, teaching and research. For Hubbard
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

(1994) and Zink & Schmidt (1995), the critical results for any university are customer
satisfaction with the services provided, the value which the market attributes to its gradu-
ates, the volume of scientific production and its impact on the different knowledge areas,
or the efficiency of resource management.
From the above, we can extract the following research hypotheses:

H5a: Process management has a positive influence on student results.


H5b: Process management has a positive influence on people results.
H5c: Process management has a positive influence on the centre results.

The EFQM Models Results


The quality of excellence consists not only in the achievement of key business results but
also in the satisfaction of internal customers (workers) and external customers (consu-
mers and/or users), and of the society in which the organization performs its activity
(Nabitz et al., 2001: 70). In this respect, the underlying idea in the EFQM Model is
that customer satisfaction, employee satisfaction and a beneficial impact on society
together produce excellent overall results (Westlund, 2001: 126). These relationships
are demonstrated empirically by Reiner (2002). However, given the characteristics of
university centres and the social function of education, the results achieved by students
and people will determine the centres overall results. This will ultimately be related to
the perception that society has of a university. In the case of public institutions, such as
university centres, considering this type of customer becomes not just a requirement but
also a must.
With respect to external customers, organizations working to achieve excellent results
in the customer satisfaction will confirm the improvement in their final results. Thus, the
high level of importance attributed by the EFQM Excellence Model to client results is due
to the need to consider their expectations and satisfaction in order to achieve organiz-
ational success.
On the other hand, a satisfied, motivated and properly trained workforce will perform
their jobs better and more efficiently, with favourable results for customer satisfaction
and for the organizations results (Koys, 2001). In the field of university education, it
seems logical to think that a satisfied, motivated and properly trained workforce will
have a favourable impact on the students academic results and in their satisfaction
levels (Osseo-Asare & Longbottom, 2002).
Relationships between the EFQM Model Criteria 749

From the above, we can extract the following research hypotheses:

H6a: People results have a positive influence on the centre results.


H6b: People results have a positive influence on the student results.
H7: Student results have a positive influence on the centre results.
H8: The centre results have a positive influence on social results.

Methodology
The data were obtained using a questionnaire following the self-assessment philosophy of
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

the EFQM Model of 1999 and with the objective of covering the criteria, sub-criteria, and
areas of the model. The questionnaires were addressed to the senior staff at the univer-
sities. The unit of analysis was the centre since it groups together administrative and man-
agerial tasks, along with other aspects related to teaching and the provision of services to
the university community.

Pilot Study
A first version of the questionnaire was sent to 10 professors with experience in the quality
area and to a Dean and Deputy-Dean who had participated in quality assessments of their
centre. Our objective was to achieve the content validity of the measurement instrument.
One week after the questionnaire had been sent, interviews were conducted to get these
individuals comments. These were later incorporated into the final questionnaire
(Table 2; Appendix A).

Sample
The population studied was formed by 346 centres of Spanish public universities evaluated
under the National Plan for the Assessment of Quality in Universities in the implemen-
tations in 1996, 1998, 1999 or 2000. A random stratified model was developed, with
the allocation being proportional to the number of centres in each university so as to
obtain representative samples of each stratum. The study considered a confidence level
of 95%, a sample error of 5% and p q 0.5. From this, the optimal sample size
ought to be 186 university centres. The number of returned questionnaires was 119, of
which eight had to be eliminated because they had not been completed correctly. As a
result, the number of valid questionnaires was 111, representing a response rate of a
little over 32% of the population. The sample error obtained with the final sample is
7.76%. The characteristics of the sample are set out in Table 1.

Measures
In order to measure the key implementation factors and the results, we took as a reference
the EFQM Model of 1999, and an adaptation of this model for the field of higher education
developed by the Club Gestion de Calidad (Quality Management Club), an EFQM partner
in Spain. In this respect, the information envisaged by the EFQM Model is appropriate for
developing measurement scales (Eskildsen & Kanji, 1998).
750 A. Calvo-Mora et al.

Table 1. Samples characteristic

Frequency Percentage

Profile of the surveyed individuals


Dean 53 47.7
Deputy-Dean 29 26.1
Director of University School 15 13.5
Other senior positions 14 12.6
Total 111 100.0
Type of study
Bio-health or Health Sciences 11 9.9
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

Experimental Sciences 16 14.4


Social and Legal Sciences 32 28.8
Humanities 20 18.0
Technical Studies 31 27.9
Non declared 1 0.9
Total 111 100.0
Experience in quality
1 3 years (Low Experience) 42 37.8
4 6 years (Medium Experience) 46 41.4
.6 years (High Experience) 23 20.7
Total 111 100.0

To measure the key implementation factors, seven-point Likert scales were used
(Table 2). Three types of measurement scales were used for the results (Appendix A).
The former takes into account the change that has taken place in particular indicators of
results in the past three years (people and student negative critical incident reduction,

Table 2. Measures

Key Constructs Indicators

Leadership & Commitment (LC) lc1 to lc8


Policy & Strategy (PS) ps1 to ps10
People Management (PM) pm1 to pm8
Partnership & Resources (PR) pr1 to pr8
Process Management (PC) Educational Processes (EP) ep1 to ep3
Research Processes (RP) rp1 to rp3
Administrative Processes (AP) ap1 to ap7
Results Indicators
People Results (PPR) People NCI Reduction (PNCI) pnci1 to pnci4
People Satisfaction (PPS) pps1 to pps4
People Skill & Knowledge (PSK) psk1 to psk3
Student Results (SR) Student NCI Reduction (SNCI) snci1 to snci2
Student Satisfaction (SS) ss1 to ss 4
Centre Results (CR) cr1 to cr8
Society Results (SCR) Society Satisfaction (SOS) sos1 to sos 4
Environmental Protection epa1 to epa4
Activities (EPA)
Relationships between the EFQM Model Criteria 751

and environmental protection activities variables). Although covering other fields, this
type of measure is used by Anderson et al. (1995), Powell (1995) or Saraph et al.
(1989). The second type is the level of student, staff and society satisfaction. The satisfac-
tion level is an indicator of quality management much used in the literature (Larson &
Sinha, 1995; Saraph et al., 1989; Terziovski & Samson, 1999; Wilson & Collier 2000),
basically in relation to internal and external customer satisfaction.
The third type of measurement of results refers to the tendency and evolution of particu-
lar indicators in an interval of time (those related to the results of the university, student
failure and dropout rate, image of the university, or support for cultural and sporting activi-
ties). This way of measuring results is used in the EFQM Model of 1999 and in other
research studies such as those by Powell (1995) or Wilson & Collier (2000).
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

Each scale was reviewed using factor analysis in order to establish its unidimensional-
ity, with the exception of the Centre Results measure, whose indicators were considered as
formative, i.e. cause or give rise to the unobserved theoretical construct. Therefore, the
manifest variables produce or contribute to the latent variable (LV) (Fornell, 1982).
These indicators are not necessarily correlated. Rather, each indicator may occur indepen-
dently of the others (Chin & Gopal, 1995).
Based on the constitutive definition of the EFQM variables applied to a university
context presented earlier, we mapped a group of factors or dimensions with respect to
their higher level construct, namely Process Management, People Results, Student
Results, and Society Results. Factor scores, computed by averaging the item scores for
each dimension, were used as indicators of the constructs in the research model. This
was necessary since the tool applied to assess the model, Partial Least Squares (PLS),
does not directly support second-order factors.

Data Analysis and Results


Partial Least Squares
The research model presented in Figure 2 was tested using Partial Least Squares (PLS),
a multivariate analysis technique for testing structural models (Wold, 1985). PLS is a
general method for the estimation of path models involving latent constructs indirectly
measured by multiple indicators (Wold, 1982). This tool is primarily intended for
causal-predictive analysis in which the problems explored are complex and theoretical
knowledge is scarce. PLS is an appropriate technique to use in a theory development situ-
ation (Wold, 1979), such as this research. This technique uses a component-based
approach to estimation. Because of this, it places minimal demands on sample size and
residual distributions (Lohmoller, 1989) and allows for the use of both formative and
reflective measures, something not generally achievable with covariance-based structural
equation modelling techniques such as LISREL or AMOS (Chin, 1998a). We have used
PLS-Graph software version 3.00 (Chin & Frye, 2003).
In PLS, indicators may be modelled as reflective or formative (Fornell, 1982). Reflec-
tive indicators are determined by the construct and, hence, covary the level of that con-
struct (Chin & Gopal, 1995). A latent variable with formative indicators implies that
the construct is expressed as a function of the manifest variables; the observed variables
form, cause, or precede the construct. Because the LV is viewed as an effect rather than a
cause of the indicator responses, these indicators are not necessarily correlated. Rather,
752 A. Calvo-Mora et al.

each indicator may occur independently of the others (Chin & Gopal, 1995). Conse-
quently, traditional reliability and validity assessment have been argued as inappropriate
and illogical (Bagozzi, 1994; Bollen, 1989). In our research model, all first-order factors
are constructs specified with reflective indicators, with the exception of indicators of the
Centre Results variable that was modelled formatively.
On the other hand, we have represented process management, people results, student
results, and society results variables as second-order factors. Following this approach, we
have had to make a choice between a molar and molecular1 approach to analysis. The
choice depends primarily on whether the first-order factors or dimensions are viewed as
causes or as indicators of the second-order factors (Chin, 1998b). If a change in one of the
dimensions or beliefs necessarily results in similar changes in other beliefs, then a molecular
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

model is appropriate. Otherwise, a molar model is suitable (Chin & Gopal, 1995). In our
research, we have decided to model process management and society results as molar
second-order factors. For example, increasing the quality of the educational processes
does not imply an increase of the quality of the administrative processes. People results
and student results have been considered as molecular second order factors. Besides, the
items for each dimension were optimally weighted and combined using the PLS algorithm
to create a latent variable score. The resulting score reflects the underlying construct more
accurately than any of the individual items by accounting for the unique factors and error
measurements that may also affect each item (Chin & Gopal, 1995). As a result, the dimen-
sions or first-order factors become the observed indicators of second-order factors.
A PLS model is analysed and interpreted in two stages: (1) the assessment of the
reliability and validity of the measurement model, and (2) the assessment of the structural
model. This sequence ensures that the constructs measures are valid and reliable before
attempting to draw conclusions regarding relationships among constructs (Barclay et al.,
1995).

Measurement Model
The measurement model in PLS is assessed in terms of individual item reliability,
construct reliability, convergent validity, and discriminant validity. As we explained
earlier, we would like to point out that this is appropriate only for constructs with reflective
indicators and molecular second order factors.

Individual item reliability


Individual item reliability is considered adequate when an item has a factor loading that is
greater than 0.7 on its respective construct (Carmines & Zeller, 1979). This is applicable to
variables with reflective indicators and high order factors with molecular orientation. Not-
withstanding, in the case of constructs measured by formative indicators and molar
second-order factors, the loadings are misleading because the intraset correlations for
each block were never taken into account in the estimation process. Therefore, it makes
no sense to compare loadings among indicators within a block. The interpretation of
LVs with formative indicators should be based on the weights (Chin, 1998a). Like the
canonical correlation analysis, the weights allow us to understand the make-up of each
LV. That is to say, these provide information about how each indicator contributes to
the respective construct. However, a concern related to using formative measures deals
with the potential multicollinearity among the items (Diamantopolulos & Winklhofer,
Relationships between the EFQM Model Criteria 753

2001; Mathieson et al., 2001). High collinearity among indicators may exist and would
produce unstable estimates and would make it difficult to separate the distinct effect of
the individual indicators on the construct. Therefore, we performed a collinearity test
using the SPSS program. The results showed minimal collinearity with the variance
inflation factor (VIF) of all items ranging between 1.070 and 1.822, both for formative
indicators and first-order factors linked to molar second-order constructs (Appendix B).
This is far below the common cut-off threshold of 5 to 10 (Kleinbaum et al., 1988).
Appendix B shows the individual item weights and loadings of the variables. Loadings
are generally above 0.7 both for indicators and for first-order factors (dimensions) related
to molecular higher order constructs. Some variables do not reach this level; nevertheless
we decided to keep them because this rule of thumb should not be so rigid at early stages of
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

scale development (Chin, 1998a), such as this study.

Construct reliability and convergent validity


The measures for construct reliability and convergent validity represent measures of
internal consistency and, as discussed earlier, these are only applicable for LVs with
reflective indicators (and also for molecular second-order factors). Construct reliability
is assessed using a measure of internal consistency: composite reliability (rc) (Werts
et al., 1974). We interpret this value using the guidelines offered by Nunnally (1978)
who suggests 0.7 as a benchmark for a modest reliability applicable in the early
stages of research. In our research, all of the constructs are reliable (Appendix B). They
all have measures of internal consistency that exceed 0.7970 (rc). To assess convergent
validity we examine the average variance extracted (AVE) measure, created by Fornell
& Larcker (1981). AVE values should be greater than 0.50. Consistent with this sugges-
tion, AVE measures for all LV surpass 0.5035 (Appendix B).

Discriminant validity
To assess discriminant validity, AVE should be greater than the variance shared between
the construct and other constructs in the model (i.e. the squared correlation between two
constructs). For adequate discriminant validity, the diagonal elements should be signifi-
cantly greater than the off-diagonal elements in the corresponding rows and columns
(Barclay et al., 1995). This condition is satisfied for reflective variables and molecular
second-order constructs in relation to the rest of the variables (Table 3). For the variable
with formative indicators (process management) and molar higher order factors, we
cannot analyse their situation because of the non-availability of AVE values.

Structural Model
Figure 3 shows the variance explained (R2) in the dependent constructs and the path
coefficients (b) for the model. Consistent with Chin (1998a), bootstrapping (500 resam-
ples) is used to generate standard errors and t-statistics. This allows us to assess the stat-
istical significance of the path coefficients.
Fifteen of 16 hypotheses were supported (Table 4). H1a c were proved, although H1c
was only marginally confirmed. This shows the leadership & commitment variable
exerts a significant positive influence on subsequent variables, both directly and indirectly
through the policy & strategy. In its turn, this latter plays an outstanding role as the proven
hypothesis H2a c demonstrates. Moreover, H3 and H4 have also been supported, which
proves the influence of the antecedent variables (policy & strategy, people, and
754
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

A. Calvo-Mora et al.
Table 3. Discriminant validity coefficients

Leadership & Policy & People Partnerships & Process People Student Centre Society
Commitment Strategy Management Resources Management Results Results Results Results

Leadership & Commitment (0.769)


Policy & Strategy 0.754 (0.768)
People Management 0.579 0.613 (0.822)
Partnerships & Resources 0.485 0.507 0.624 (0.779)
Process Management 0.611 0.568 0.606 0.637 (n.a.)
People Results 0.493 0.422 0.421 0.560 0.611 (0.786)
Student Results 0.270 0.254 0.311 0.264 0.348 0.518 (0.814)
Centre Results 0.513 0.488 0.430 0.436 0.557 0.586 0.524 (n.a.)
Society Results 0.407 0.462 0.470 0.538 0.583 0.530 0.431 0.548 (n.a.)

Diagonal elements (values in parentheses) are the square root of the variance shared between the constructs and their measures (AVE). Off-diagonal elements are the corre-
lations among constructs. For discriminant validity, diagonal elements should be larger than off-diagonal elements. (n.a.) Non-applicable
Relationships between the EFQM Model Criteria 755
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

Figure 3. Structural model results

partnerships & resources) on the process management variable. Of the three hypotheses
whose origin was process management (H5a2c), only H5a has been rejected. However,
this variable has an indirect effect on student results through the people results variable.
Indeed, we should underline the important impact of process management on people
results (H5b, b 0.611, p , 0.001). This study also proves hypotheses H6a b and H7,
which support all links between centre results and explanatory variables. Finally, we
found a strong relationship between centre results and the society results variable.
The research model seems to have an appropriate predictive power for most of the
dependent variables. The mean of the explained variance of the implied variables is
39.85%.

Discussion
Tables 4 and 5 summarize the results obtained in the testing of hypotheses as well as the
different effects between variables and the percentage of the total variation of the depen-
dent variable accounted for by each independent variable. Additionally, they will help us
with the discussion of the results.
First, it is confirmed that the leadership and commitment of the senior officers of the
centres act as the driving-force of the whole of the quality management and quality
improvement process. This can be seen from the effect this variable has on policy & strat-
egy, partnerships & resources, and people and process management. These relationships
756
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

A. Calvo-Mora et al.
Table 4. Structural model results

Suggested Path t-value


Hypothesis effect coefficients () (bootstrap) Support

H1a: Leadership & Commitment ! People Management 0.271 2.0725 Yes


H1b: Leadership & Commitment ! Policy & Strategy 0.754 18.6947 Yes
H1c: Leadership & Commitment ! Partnerships & Resources 0.238 1.8118 Yes
H2a: Policy & Strategy ! People Management 0.409  3.4117 Yes
H2b: Policy & Strategy ! Partnerships & Resources 0.328 2.4517 Yes
H2c: Policy & Strategy ! Process Management 0.240 2.5995 Yes
H3: People Management ! Process Management 0.226 2.0338 Yes
H4: Partnerships & Resources ! Process Management 0.374 3.0753 Yes
H5a: Process Management ! Student Results 0.051 0.4381 No
H5b: Process Management ! People Results 0.611 8.9417 Yes
H5c: Process Management ! Centre Results 0.303 2.8447 Yes
H6a: People Results ! Centre Results 0.251 2.5610 Yes
H6b: People Results ! Student Results 0.487 3.6855 Yes
H7: Student Results ! Centre Results 0.289 3.2693 Yes
H8: Centre Results ! Society Results 0.548 8.2837 Yes

p , 0.001,  p , 0.01,  p , 0.05, p , 0.1 (based on t(499), two-tailed test).
t(0.001; 499) 3.310124157; t(0.01; 499) 2.585711627; t(0.05; 499) 1.964726835; t(0.1; 499) 1.64791345.
Relationships between the EFQM Model Criteria 757

Table 5. Effects on endogenous variables

Direct Indirect Total Explained


Effects on endogenous variables effect effect effect Variance

Policy & Strategy (R2 0.5680)


H1b: Leadership & Commitment 0.754 0.7540 0.5680
People (R2 0.4075)
H1a: Leadership & Commitment 0.271 0.3083 0.5793 0.1569
H2a: Policy & Strategy 0.409 0.4090 0.2506
Partnerships & Resources (R2 0.2816)
H1c: Leadership & Commitment 0.238 0.2473 0.4853 0.1154
H2b: Policy & Strategy 0.328 - 0.3280 0.1663
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

Processes (R2 0.5112)


H2c: Policy & Strategy 0.24 0.2400 0.1363
H3: People Management 0.226 0.2260 0.1370
H4: Partnerships & Resources 0.374 0.3740 0.2382
People Results (R2 0.3734)
H5b: Process Management 0.611 0.6110 0.3733
Student Results (R2 0.2700)
H5a: Process Management 0.051 0.2976 0.3486 0.0177
H6b: People Results 0.487 0.4870 0.2523
Centre Results (R2 0.4676)
H5c: Process Management 0.303 0.2394 0.5424 0.1688
H6a: People Results 0.251 0.1407 0.3917 0.1471
H7: Student Results 0.289 0.2890 0.1514
Society Results (R2 0.3007)
H8: Centre Results 0.548 0.5480 0.3003

are confirmed by the work of Ahire et al. (1996), Eskildsen & Dahlgaard (2000), Flynn
et al. (1995) or Wilson & Collier (2000), although in a field different from education.
In the latter field, and using the Malcolm Baldrige Model as a reference, Winn &
Cameron (1998) confirm the relationship between leadership, policy & strategy, and
people and process management.
Secondly, the policy and strategy must act as a reference for the setting of staff policy
and the management of resources and processes. Reiner (2002) confirms how policy and
strategy play a central role in the block of agents. This aspect is also highlighted by Ahire
et al. (1996) and Eskildsen & Dahlgaard (2000), confirming the relationship between
policy & strategy and the management of people and resources. Wilson & Collier
(2000) also verify positively the relationship of policy & strategy with the management
of people and resources. Winn & Cameron (1998) positively confirm this relationship
in the field of education.
Thirdly, appropriate management of people is key to the management of processes. In
this respect, Eskildsen & Kanji (1998) show that those organizations which make no
efforts in motivating or training their employees do not manage to involve them in
process improvement. In the empirical literature, there are several studies that corroborate
the positive relationship between the management of people and resources (Flynn &
Saladin, 2001; Wilson & Collier, 2000; Winn & Cameron, 1998). In the field of education,
Detert & Jenni (2000) emphasize the role of motivation and training as key factors affect-
ing the continuous acquisition of new knowledge, skills and abilities by all employees,
758 A. Calvo-Mora et al.

thus leading them to perform their job better. Moreover, as is confirmed by our study, suit-
able process development does not depend exclusively on people and the organizations
policy and strategy but also on appropriate management of material and financial
resources and the involvement of suppliers in these processes. This aspect is corroborated
by the work of Eskildsen & Dahlgaard (2000).
Fourthly, the results of our research confirm the effect of process management on
the results of people, students (as external customers), and the centre. Nonetheless, the
relationship between processes and the student results takes place indirectly through the
results of people (teaching, administration and services staff), reaching a significant value
(0.2976). These results are in keeping with those of Anderson et al. (1995) about the fact
that the principal determinant of peoples and customers results is the form in which the
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

organization manages its key activities and continuously improves them. Winn &
Cameron (1998) and Flynn & Saladin (2001) also verify this relationship as well as the influ-
ence of management of key processes on the overall results of the organization.
With regards to the type of process that affects the results to the greatest extent, and with a
focus on the formative measures, special attention must be paid to the weightings (Mathie-
son et al., 2001: 105). Our study reveals that administrative and educational processes are
those with the greatest weight and, therefore, those that most influence the results.
It is worth noting that not only are the results affected by the key processes, but the
achievement of excellent results will be produced by a coordinated effort of all activity
areas of the organization or agents of the model (Ghobadian & Woo, 1996). In our
study, this aspect can be verified by analysing the indirect relationships that exist
between the agents of the model and the different measurements of results.
Finally, the results of our research also confirm the existence of an internal logic con-
necting the results of the EFQM Model (Ghobadian & Woo, 1996), although, due to the
adaptation of the model to the field of higher education, these relationships do not coincide
exactly with those of the reference model. Our study confirms how people results influence
student results, and how both people and student results affect the centre results. Lastly,
the manner in which society perceives the work performed by universities is directly deter-
mined by the overall results of the institution and indirectly by the results of people and the
centre.

Conclusions and Limitations


The results we obtain from the analysis of the structural model and its forecasting ability
support the validity of the European Excellence Model as a reference framework for the
implementation, evaluation and improvement of quality in the field of higher education.
Also revealed is the systemic nature of the model, that is to say the need to consider
the agents, on a whole, as the determinants of the results as well as the relationship that
exists between the different measures of them.
On the other hand, the analysis of the structural model has allowed us to study the causal
structure of the EFQM Model in depth. Knowledge of this structure may allow universities
to lead their management towards excellent results.
In this respect, the principal implications for the management of the universities may be
the following. First, for the implementation of any quality improvement initiative, it is
necessary to have the leadership and commitment of the senior management of the
centres. They must create and disseminate the values of this management philosophy,
Relationships between the EFQM Model Criteria 759

set goals and objectives that are consistent with these values and create an appropriate
organization and system to achieve them.
This definite management commitment must go hand in hand with a well-defined policy
and strategy, implemented and communicated at all levels of the institution. The absence
of this prevents measurement of the effectiveness and efficiency of universities or any of
their subsystems, in addition to creating disequilibrium in the allocation of resources
within the system. Policy and strategy must be based on the needs and the current and
future expectations of the stakeholders, and supported by the mission, vision and values
established by the institution.
Subsequently, the planning must be put into practice through concrete actions aimed at
continuous improvement and generating more and more value for the stakeholders. That is
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

to say, via a suitable design, management and improvement of key processes. The man-
agement of key processes links the other variables defining the system with the measure-
ments of the results. With this in mind, each university centre must try to identify its key
processes, document them and assign people to be in charge of monitoring and improving
them. For these monitoring and improvement activities, it is essential to have a system of
indicators to evaluate the effectiveness and the efficiency of the processes, and, in addition,
provide transparent external information (students, society, etc). For public universities, it
is essential the authorities and other stakeholders understand the actions of the university
centres. Within these processes, the importance of teaching and administration stand out
due to the effect they have on the results of students and people, and, indirectly, on the
results of the centre and of society.
Human resource management is a basic pillar of successful quality management. Here,
the effect of this variable on processes, the results of students, the workforce and the
overall results of the institution is noteworthy. Improvement is a process of organizational
learning, which is largely based on people. Therefore, the participation of the entire work-
force in improvement activities must be encouraged, and the efforts made must be
rewarded and recognized.
Moreover, the universities and their centres, as with any other organization, must try to
optimize the scarce resources they have (monetary, information, infrastructure or techno-
logical) and execute adequate control and management of suppliers of specific inputs that
represent a significant budgetary cost. In this way, appropriate management of partner-
ships and resources will have a positive effect on the development of processes and will
indirectly affect results. As far as the results are concerned, in order to achieve better
results of people and students, the development and improvement of the teaching and
administrative processes must be strengthened, supported and promoted. To achieve
this, the university should deploy the human and material means necessary for the appro-
priate development of these processes. In addition, the improvement of these results will
have a positive effect on the results of the university and on how customers and society in
general perceive the role of the university.

Limitations
The study has certain limitations that must be considered when it comes to interpreting the
results and conclusions arising from it.
A first limitation is related to the notion of causality. Although evidence is provided on
the causality of the model, this has not really been tested. Our study has considered a
760 A. Calvo-Mora et al.

flexible modelling directed more towards prediction than causality. A second limitation is
determined by the technique used for the proposed model: the structural equations, which
assume the linearity of relationships between the latent variables. Thirdly, the study has
relied on measurements based on the perceptions of the individuals who participated in
it. But it has to allow for the possibility that the perceptions of those surveyed do not
provide a completely accurate picture of reality. Finally, the design of the research
employed was cross-sectional instead of longitudinal. In this respect, the quality assess-
ment and improvement is a process that develops over time and whose effects are
really appreciated only in the long term. It would, therefore, be appropriate to follow a
longitudinal approach for future research.
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

Note
1. Whereas the molar attitude represents an emergent construct that is formed from the first-order factors, in
the molecular approach it is hypothesized that an overall latent construct exists and is indicated and
reflected by the first-order factors (Chin & Gopal, 1995).

References
Ahire, S.L., Golhar, D.Y. & Waller, M.A. (1996) Development and validation of TQM implementations
construct, Decision Sciences, 27(1), pp. 2356.
Ahmad, S. & Schroeder, G. (2002) The importance of recruitment and selection process for sustainability of total
quality management, International Journal of Quality and Reliability Management, 19(5), pp. 540550.
Anderson, J.C., Rungtusanatham, M. & Schroeder, R.G. (1994) A theory of quality management underlying the
Deming management method, Academy of Management Review, 19(3), pp. 472509.
Anderson, J.C., Rungtusanatham, M., Schroeder, R.G. & Devaraj, S. (1995) A path analytic model of a theory of
quality management method: preliminary empirical findings, Decision Sciences, 26(5), pp. 637658.
Bagozzi, R.P. (1994) Structural equation models in marketing research: basic principles. In: R. Bagozzi (Ed.)
Principles of Marketing Research, pp. 317 385 (Oxford: Blackwell).
Barclay, D., Higgins, C. & Thompson, R. (1995) The partial least squares (PLS) approach to causal modelling:
personal computer adoption and use as an illustration, Technology Studies, Special Issue on Research Meth-
odology, 2, pp. 285309.
Black, S.A. & Porter, L.J. (1996) Identification of the critical factors of TQM, Decision Sciences, 27(1), pp. 121.
Bollen, K.A. (1989) Structural Equations with Latent Variables (New York: Wiley).
Carmines, E.G. & Zeller, R.A. (1979) Reliability and Validity Assessment, Sage University paper series on quan-
titative applications in the social sciences, N. 07-017, (Beverly Hills, CA: Sage).
Chin, W.W. (1998a) The partial least squares approach to structural equation modelling. In: G.A. Marcoulides
(Ed.) Modern Methods for Business Research, pp. 295 336 (Mahwah, NJ, Lawrence Erlbaum Associates,
Publisher).
Chin, W.W. (1998b) Issues and opinion on structural equation modeling, MIS Quarterly, 22, pp. viixv.
Chin, W.W. & Frye, T. (2003) PLS-Graph, Version 3.00 (Build 1017) (University of Houston).
Chin, W.W. & Gopal, A. (1995) Adoption intention in GSS: relative importance of beliefs, Database, 26,
pp. 42 64.
Club Gestion de Calidad (1999) Metodo para mejorar la calidad en la educacion superior basado en el modelo de
la EFQM [Method for quality improvement in the higher education based in the EFQM Model] (Madrid,
Club Gestion de Calidad).
Curkovic, S., Vickery, S. & Droge, C. (2000) Quality-related action programs: their impact on quality perform-
ance and firm performance, Decision Sciences, 31(4), pp. 885905.
Davies, J., Hides, M.T. & Casey, S. (2001) Leadership in higher education, Total Quality Management, 12(7&8),
December, pp. 10251030.
Dean, J.W. & Bowen, D.E. (1994) Management theory and total quality: improving research and practice through
theory development, Academy of Management Review, 19(3), pp. 392 418.
Relationships between the EFQM Model Criteria 761

Detert, J.R. & Jenni, R. (2000) An instrument for measuring quality practice in education, Quality Management
Journal, 7(3), pp. 20 37.
Diamantopolulos, A. & Winklhofer, H.M. (2001) Index construction with formative indicators: an alternative to
scale development, Journal of Marketing Research, 38, pp. 269 277.
Dow, D., Samson, D. & Ford, S. (1999) Exploding the myth: do all quality management practices contribute to
superior quality performance?, Production and Operations Management, 8(1), Spring, pp. 1 27.
Eskildsen, J.K. & Dahlgaard, J.J. (2000) A causal model for employee satisfaction, Total Quality Management,
11, pp. 10811094.
Eskildsen, J.K. & Kanji, G.K. (1998) Identifying the vital few using the European foundation for quality manage-
ment model, Total Quality Management, 9, pp. S92S95.
Eskildsen, J.K., Kristensen, K. & Juhl, H.J. (2001) The criterion weights of the EFQM Excellence Model,
International Journal of Quality and Reliability Management, 18, pp. 783795.
European Foundation for Quality Management (1999) Modelo EFQM de Excelencia, (Brussels, European
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

Foundation for Quality Management).


Flynn, B.B. & Saladin, B. (2001) Further evidence on the validity of the theoretical models underlying the
Baldrige criteria, Journal of Operations Management, No.19, pp. 617652.
Flynn, B., Schroeder, R.G. & Sakakibara, S. (1994) A framework for quality management research and an associ-
ated measurement instrument, Journal of Operations Management, 11(4), pp. 339366.
Flynn, B., Schroeder, R.G. & Sakakibara, S. (1995) The impact of quality management practice on performance
and competitive advantage, Decision Sciences, 26(5), pp. 659 691.
Fornell, C. (1982) A second generation of multivariate analysis: an overview. In C. Fornell (Ed.) A Second
Generation of Multivariate Analysis, Vol. 1, pp. 121 (New York: Praeger Publishers).
Fornell, C. & Larcker, D.F. (1981) Evaluating structural equation models with unobservable variables and
measurement error, Journal of Marketing Research, 18, pp. 3950.
Gatewood, R.D. & Riordan, C.M. (1997) The development and test of a model of total quality: organizational
practices, TQ principles, employee attitudes and customer satisfaction, Journal of Quality Management,
2(1), pp. 41 65.
Ghobadian, A. & Gallear, D. (1997) TQM and organization size, International Journal of Operations &
Production Management, 17, pp. 121163.
Ghobadian, A. & Woo, H.S. (1996) Characteristic, benefits and shortcomings of four major quality awards,
International Journal of Operations & Production Management, 13, pp. 1044.
Goldberg, J.S. & Cole, B.R. (2002) Quality management in education: building excellence and equity in student
performance, Quality Management Journal, 9(4), pp. 822.
Hewitt, S. (1997) Business excellence: does it work for small companies?, The TQM Magazine, No.9, pp. 7682.
Hubbard, D.L. (1994) Can higher education learn from factories?, Quality Progress, May, pp. 9397.
Kanji, G.K. (2000) A European application of the business excellence index, Quality Progress, December,
pp. 109 114.
Kanji, G.K. & Tambi, A.M. (1999) Total quality management in UK higher education institutions, Total Quality
Management, 10(1), pp. 129153.
Kannan, V.R., Tan, K.C., Handfield, R.B. & Ghosh, S. (1999) Tools and techniques of quality management: an
empirical investigation of their impact on performance, Quality Management Journal, 6(3), pp. 3449.
Kleinbaum, D.G., Kupper, L.L. & Muller, K.E. (1988) Applied Regression Analysis and other Multivariate
Analysis Methods (Boston, PWS-Kent Publishing Company).
Koys, D.J. (2001) The effects of employee satisfaction, organizational citizenship behavior, and turnover on
organizational effectiveness: a unit-level, longitudinal study, Personnel Psychology, No. 54, pp. 101 113.
Larson, P. D. & Sinha, A. (1995) The TQM impact: a study of quality managers perceptions, Quality Manage-
ment Journal, Spring, pp. 5366.
Lau, R.S.M. & Anderson, C.A. (1998) A three-dimensional perspective of total quality management,
International Journal of Quality & Reliability Management, 15(1), pp. 8598.
Lohmoller, J.B. (1989) Latent Variable Path Modeling with Partial Least Squares (New York: Springer-Verlag).
Mathieson, K., Peacock, E. & Chin, W.W. (2001) Extending the technology acceptance model: the influence of
perceived user resources, The Data Base for Advances in Information Systems, 32, pp. 86112.
Mukherjee, A.S., Lapre, M.A. & Van Wassenhove, L.N. (1998) Knowledge driven quality improvement,
Management Science, 44, November, pp. 3549.
Nabitz, U., Severens, P., Van der Brink, W. & Jansen, P. (2001) Improving the EFQM model: an empirical study on
model development and theory building using concept mapping, Total Quality Management, 12(1), pp. 6981.
762 A. Calvo-Mora et al.

Naylor, G. (1999) Using the business excellence model to develop a strategy for a healthcare organization,
International Journal of Health Care Quality Assurance, 12, pp. 3744.
Nunnally, J.C. (1978) Psychometric Theory (New York: McGraw-Hill).
Osseo-Asare, A.E. & Longbottom, D. (2002) The need for education and training in the EFQM Model for quality
management in UK higher education institutions, Quality Assurance in Education, 10(1), pp. 2636.
Pannirselvam, G.P. & Ferguson, L.A. (2001) A study of the relationship between the Baldrige categories,
International Journal of Quality & Reliability Management, 18(1), pp. 1434.
Peak, M.H. (1995) TQM transforms the classroom, Management Review, September, pp. 1318.
Porter, L.J. & Parker, A.J. (1993) Total quality management-the critical success factors, Total Quality Manage-
ment, 4(1), pp. 1322.
Powell, T.C. (1995) Total quality management as competitive advantage: a review and empirical study, Strategic
Management Journal, 16, pp. 1537.
Quazi, H.A., Jemangin, J., Wai Kit, L. & Lee Kian, C. (1998) Critical factors in quality management and
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

guidelines for self-assessment: the case of Singapore, Total Quality Management, 9(1), pp. 3555.
Ravichandran, T. & Rai, A. (2000) Total quality management in information systems development: key
constructs and relationships, Journal of Management Information Systems, 16(3), Winter, pp. 119155.
Reiner, G. (2002) Analysis of critical factor of company success based on the EFQM Excellence Model, Proceed-
ing of the 7th world Congress for Total Quality Management, Verona (Italy), 2, pp. 361366.
Rusell, S. (2000) ISO 9000: 2000 and the EFQM Excellence Model: competition or co-operation?, Total Quality
Management, 11, pp. 657665.
Salmon, V.R. (1993) Quality in American schools, Quality Progress, October, pp. 73 75.
Saraph, J.V., Benson, P.G. & Schroeder, R.G. (1989) An instrument for measurement the critical factors of
quality management, Decision Sciences, 20(4), pp. 810829.
Schargel, F.P. (1995) Salvaging our public schools, Management Review, September, pp. 2225.
Tamimi, N. (1998) A second-order factor analysis of critical TQM factors, International Journal of Quality
Science, 3, pp. 7179.
Terziovski, M. & Samson, D. (1999) The link between total quality management practice and organisational
performance, International Journal of Quality and Reliability Management, 16(3), pp. 226237.
Ward, B. & Chandler, W.D. (1999) Applying quality management concepts to managing business schools, SAM
Advance Management Journal, Autumn, pp. 2124.
Werts, C.E., Linn, R.L. & Joreskog, K.G. (1974) Interclass reliability estimates: testing structural assumptions,
Educational and Psychological Measurement, 34, pp. 2533.
Westlund, A.H. (2001) Measuring environmental impact on society in the EFQM systems, Total Quality Manage-
ment, 12(1), pp. 125135.
Wilson, D.D. & Collier, D.A. (2000) An empirical investigation of the Malcolm Baldrige National Quality Award
causal model, Decision Sciences, 31(2), pp. 361 390.
Winn, B. A. & Cameron, K. S. (1998) Organizational quality: an examination of the Baldrige national quality
framework, Research in Higher Education, 39(5), pp. 491 512.
Wold, H. (1979) Model Construction and Evaluation when Theoretical Knowledge is Scarce: an Example of
the Use of Partial Least Squares. Cahiers du Departement DEconometrie (Gene`ve: Faculte des Sciences
Economiques et Sociales, Universite de Gene`ve).
Wold, H. (1982) Systems under indirect observation using PLC. In C. Fornell (Ed.) A Second Generation of
Multivariate Analysis, Vol. 1, pp. 325 347 (New York: Praeger Publishers).
Wold, H. (1985) Systems analysis by partial least squares. In P. Nijkamp, H. Leitner & N. Wrigley (Ed.)
Measuring the Unmeasurable, pp. 221251 (Dordrecht: Martinus Nijhoff Publishers).
Yusof, S.M. & Aspinwall, E. (1999) Critical success factors for total quality management implementation in
small and medium enterprise, Total Quality Management, 10(45), pp. 803809.
Zink, Z.L. & Schmidt, A. (1995) Measuring universities against the European quality award criteria, Total
Quality Management, 6(5/6), pp. 547562.

Appendix A: Research Constructs and Measures


Leadership & Commitment: Rank the following statements, taking as a reference the
actions of the Centres managers and using the following scale: 1 (totally disagree);
4 (neither agree nor disagree); 7 (totally agree).
Relationships between the EFQM Model Criteria 763

lc1: They develop the centres mission, vision and values


lc2: They communicate the mission, vision and values to all levels of the Centre
lc3: They improve their actions, making them fit in with the Centres present and future
needs
lc4: They design an organisational structure (neither a teaching nor a research one) suit-
able for the Centres policies and strategies
lc5: They implement a system of key processes or activities supporting the Centres
policy and strategy, and its goals
lc6: They keep in touch with the different stakeholders in order to know their expec-
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

tations and opinions


lc7: They encourage students and staffs involvement in the improvement actions
lc8: They publicly acknowledge the successes of people and groups in quality improve-
ment actions

Policy & Strategy: Indicate the degree of agreement or disagreement with the following
statements using the following scale: 1 (totally disagree); 4 (neither agree nor disagree);
7 (totally agree).

ps1: The Centres policies and strategies are in line with its mission, vision and values
ps2: The Centres policies and strategies are clearly formulated in writing
ps3: All the areas in the Centre are involved in the process of formulating and commu-
nicating the policies and strategies
ps4: There is a formal process of reviewing and updating policies and strategies
ps5: The Centres policies and strategies are structured in a Strategic Plan
ps6: The Centres goals are set out in writing and in a clear and quantifiable manner
ps7: The goals are communicated at all levels of the organisation
ps8: The principles of quality are incorporated into all of the Centres policies, strat-
egies and goals
ps9: There is a procedure allowing for the deployment of the policies and strategies and
for their being turned into short term plans
ps10: The formulation and revision of policies and strategies include the needs and
expectations of the stakeholders

People Management: Indicate to which extent the Centre has a formal system to carry out
the following questions. Use the following scale: 1 (no system); 4 (the system exists but it
is not a formal one); 7 (the system is formal and subject to revisions).

pm1: Identifying the staffs present and future needs regarding knowledge, competen-
cies and skills
pm2: Developing training plans for the improvement of the staffs knowledge, compe-
tencies and skills
pm3: Promoting actions which support the staffs commitment and involvement in the
improvement actions
pm4: Encouraging the staffs assumption of responsibilities and empowerment to carry
out improvement actions
764 A. Calvo-Mora et al.

pm5: Developing suitable channels for sharing and communicating better practices,
knowledge and experiences
pm6: Recognising quality improvement related efforts, either at a personal or group level
pm7: Establishing social benefits and improvement of the staffs services and facilities
pm8: Encouraging the staffs involvement in topics related to health and safety, the
environment, and social and ethic responsibility
Partnerships & Resources: Indicate the degree of emphasis the Centre places on each of
the following questions, using the following scale: 1 (no emphasis); 4 (moderate empha-
sis); 7 (strong emphasis).
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

pr1: Establishment of partnerships with suppliers to generate value and mutual benefits
pr2: Development of agreements guaranteeing the exchange of knowledge and experi-
ences with suppliers
pr3: Making appropriate investments for the development of the Centres policy, strat-
egy and continuous improvement
pr4: Suitable use of the stock, the materials and the energy
pr5: Identification and evaluation of the impact of new technologies on the Centre
pr6: Implementation of mechanisms for the collection and use of data supporting the
Centres policy and strategy
pr7: Implementation of mechanisms for the identification of the information needs of
the stakeholders
pr8: Use of information for the continuous improvement of the management system and
the services

Process Management: Indicate the degree of agreement or disagreement with the follow-
ing questions using the following scale: 1 (totally disagree); 4 (neither agree nor disagree);
7 (totally agree).
Educational Process
ep1: The teaching activity envisages the students needs and expectations
ep2: The teaching activity envisages the companies needs and expectations
ep3: The teaching activity envisages the needs and expectations of the community or the
society in general
Research Process
rp1: The research activity envisages the students needs and expectations
rp2: The research activity envisages the companies needs and expectations
rp3: The research activity envisages the needs and expectations of the community or the
society as a whole
Administrative Process
ap1: The Centre makes efforts addressed to identifying and analysing key processes and
actions
ap2: There is documentary support for processes (field of action, the actions they are
made of, validity, etc.)
ap3: Responsibilities are allocated for the periodic monitoring and review of the processes
ap4: Data are collected about claims and suggestions of the stakeholders, then used to
improve the processes
Relationships between the EFQM Model Criteria 765

ap5: Creativity and innovation are applied to the development of new processes and
services
ap6: Procedures are developed aimed at guaranteeing the adequate provision of services
to the stakeholders
ap7: Specific services are developed to support people and students

People Results
People Negative Critical Incident (NCI) Reduction
Based on the last three years, indicate the change undertaken on each of the following indi-
cators, according to the following scale: 1 (very significant increase); 4 (same as in pre-
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

vious years); 7 (very significant decrease).


pnci1: Number of claims and complaints by the teaching staff
pnci2: Number of claims and complaints by the administration and service staff
pnci3: Average time needed to respond or solve claims and complaints
pnci4: Absenteeism and off-work rates

People Satisfaction
According to the following scale, rank the staffs present level of satisfaction: 1 (totally
dissatisfied); 4 (neither satisfied nor dissatisfied); 7 (totally satisfied)
pps1: Teaching and research staff
pps2: Administration and service staff

Based on the last three years, indicate the tendency and evolution of the following aspects
according to the following scale: 1 (very significant decrease); 4 (unchanged); 7 (very sig-
nificant increase)
pps3: Teaching and research staff satisfaction
pps4: Administration and service staff satisfaction

People Skill & Knowledge


Based on the last three years, indicate the tendency and evolution of the following aspects
according to the following scale: 1 (very significant decrease); 4 (unchanged); 7 (very sig-
nificant increase)
psk1: Involvement in improvement actions and suggestions made
psk2: Involvement in actions regarding training and retraining skills and knowledge
psk3: Degree of achievement of training plans, and promotion and development plans

Student Results
Student Negative Critical Incident (NCI) Reduction
Based on the last three years, indicate the change undergone by each of the following indi-
cators according to the following scale: 1 (very significant increase); 4 (same as in pre-
vious years); 7 (very significant decrease).
snci1: Average time needed to respond or solve claims and/or complaints
766 A. Calvo-Mora et al.

snci2: Number of claims and/or complaints submitted by students

Student Satisfaction
Based on the last three years, indicate the change undergone by each of the following indi-
cators according to the following scale: 1 (very significant increase); 4 (same as in pre-
vious years); 7 (very significant decrease).

ss1: Student failure rate


ss2: Student dropout rate
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

Based on the last three years, indicate the tendency and evolution of the following aspects
according to the following scale: 1 (very significant decrease); 4 (unchanged); 7 (very sig-
nificant increase).

ss3: Student satisfaction


ss4: Graduation rate in the theoretical time

Centre Results
Based on the last three years, and according to the following scale, indicate the tendency
and evolution of the following parameters of Centre results: 1 (very significant decrease);
4 (unchanged); 7 (very significant increase)

cr1: Percentage of students being able to have work-experience contracts in companies


or other institutions
cr2: Improvement on the times needed for service provision and process performance
(registration, issuing of certificates, internal mail, library, economic management
of payment orders, etc.)
cr3: Employment rate for graduate students from the Centre
cr4: Number of postgraduate (Doctorate) theses
cr5: Number of research projects obtained from public institutions
cr6: Number of registered patents and utility models
cr7: Degree of performance of the costs and revenues budget
cr8: Ratio of own/third-party resources

Society Results
Society Satisfaction
Based on the last three years, indicate the change undergone in the following questions
according to the following scale: 1 (very significant worsening or decrease);
4 (unchanged); 7 (very significant increase or improvement).

sos1: Centre image in the community or society


sos2: Support of cultural or sport activities
sos3: According to the following scale, rank the present level of satisfaction of the sur-
rounding community or society: 1 (totally unsatisfied); 4 (neither satisfied nor dis-
satisfied); 7 (totally satisfied)
Relationships between the EFQM Model Criteria 767

sos4: Based on the last three years, indicate the tendency and evolution of the surround-
ing community or society with regards to the Centre according to the following
scale: 1 (very significant decrease); 4 (unchanged); 7 (very significant increase)

Environmental Protection Activities


Based on the last three years, indicate the change undergone in the following questions,
according to the following scale: 1 (very significant decrease or worsening); 4
(unchanged); 7 (very significant increase or improvement).

epa1: Risk prevention


Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

epa2: Environment protection and preservation by reducing waste and pollutant


emissions
epa3: Degree of refuse recycling (paper, cartons, toner, etc.)

Appendix B. Measurement Model Evaluation

Variance Average
CONSTRUCT/ inflation Composite variance
Dimension/ factor reliability extracted
Indicator (VIF) Weight Loading (rc) (AVE)
LEADERSHIP & 0.9200 0.5916
COMMITMENT
(Reflective)
lc1 0.8354
lc2 0.7439
lc3 0.7954
lc4 0.7774
lc5 0.7524
lc6 0.7142
lc7 0.7238
lc8 0.8026
POLICY & STRATEGY 0.9350 0.5895
(Reflective)
ps1 0.7353
ps2 0.7957
ps3 0.7271
ps4 0.7896
ps5 0.7202
ps6 0.7939
ps7 0.8176
ps8 0.8084
ps9 0.7371
ps10 0.7447

(continued)
768 A. Calvo-Mora et al.

Appendix B. Continued

Variance Average
CONSTRUCT/ inflation Composite variance
Dimension/ factor reliability extracted
Indicator (VIF) Weight Loading (rc) (AVE)

PEOPLE MANAGEMENT 0.9430 0.6760


(Reflective)
pm1 0.8334
pm2 0.8238
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

pm3 0.8967
pm4 0.8953
pm5 0.8746
pm6 0.7990
pm7 0.7289
pm8 0.7032
PARTNERSHIP & 0.9250 0.6067
RESOURCES (Reflective)
pr1 0.7214
pr2 0.7606
pr3 0.7694
pr4 0.7246
pr5 0.7591
pr6 0.8142
pr7 0.8514
pr8 0.8206
PROCESS MANAGEMENT n.a. n.a.
(Molar 2nd order factor)
Educational Processes 1.803 0.5166 0.8892 0.7281
ep1 0.8365
ep2 0.8270
ep3 0.8948
Research Processes 1.455 0.1554 0.8210 0.6048
rp1 0.7466
rp2 0.7756
rp3 0.8096
Administrative Processes 1.395 0.5359 0.9154 0.6076
ap1 0.7784
ap2 0.7349
ap3 0.7898
ap4 0.7668
ap5 0.8133
ap6 0.8192
ap7 0.7505

(continued)
Relationships between the EFQM Model Criteria 769

Appendix B. Continued

Variance Average
CONSTRUCT/ inflation Composite variance
Dimension/ factor reliability extracted
Indicator (VIF) Weight Loading (rc) (AVE)

PEOPLE RESULTS 0.8290 0.6184


(Molecular 2nd order factor)
People NCI reduction 0.7797 0.8560 0.5996
pnci1 0.8526
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

pnci2 0.8331
pnci3 0.7470
pnci4 0.6475
People satisfaction 0.8386 0.8590 0.6065
pps1 0.6102
pps2 0.8113
pps3 0.8620
pps4 0.8078
People Skills & Knowledge 0.7376 0.9170 0.7873
psk1 0.8680
psk2 0.8885
psk3 0.9051
STUDENT RESULTS 0.7970 0.6628
(Molecular 2nd order factor)
Student NCI Reduction 0.8251 0.8010 0.6694
snci1 0.8719
snci2 0.7607
Student Satisfaction 0.8030 0.7990 0.5035
ss1 0.6385
ss2 0.6037
ss3 0.8450
ss4 0.7265
CENTRE RESULTS n.a. n.a.
(Formative)
cr1 1.368 0.3596
cr2 1.253 0.2897
cr3 1.190 0.2130
cr4 1.587 0.5069
cr5 1.822 -0.4607
cr6 1.587 0.2964
cr7 1.176 0.3237
cr8 1.536 -0.1605

(continued)
770 A. Calvo-Mora et al.

Appendix B. Continued

Variance Average
CONSTRUCT/ inflation Composite variance
Dimension/ factor reliability extracted
Indicator (VIF) Weight Loading (rc) (AVE)

SOCIETY RESULTS n.a. n.a.


(Molar 2nd order factor)
Society satisfaction 1.070 0.8405 0.8820 0.6523
sos1 0.8143
Downloaded by [University of Wyoming Libraries] at 22:01 30 September 2013

sos2 0.6967
sos3 0.8438
sos4 0.8651
Environmental protection 1.070 0.3679 0.8420 0.6403
activities
epa1 0.8596
epa2 0.7832
epa3 0.7541

You might also like