Professional Documents
Culture Documents
the nature of the relationship among client organizations and external training providers,
the training needs assessment, and the nature of the training program. In addition, it is to
needs assessment, and the nature of the training program. Two survey instruments were
developed to measure the variables. One survey asked HRD managers about training
program characteristics, and another survey asked senior managers about the perception
companies who received training funds from the Ohio Investment in Training Programs
from 2002 to 2004. Forty five out of 125 companies completed both surveys, and thus,
the response rate was 36 percent. The collected data was merged with the some
ii
The results showed that most participant organizations are privately owned,
manufacturing companies. The major external training providers are private organizations.
Few companies engage in partnership training with educational institutions. The results
showed that the operational margin of the programs where private training providers were
involved increased more than the programs that did not involved private training
providers. In addition, if external training providers were involved in more stages of the
training process, operational margin increased. The results also showed that senior
managers perceived the entirely developed training program was more effective than the
relationship between training needs assessment and training program effectiveness. This
study provides several implications on future study in HRD area as well as practitioners
higher education.
iii
Dedicated to my sisters,
iv
ACKNOWLEDGMENTS
I would like to express sincere gratitude to my adviser, Dr. Joshua D. Hawley, for
his endless encouragement and consistent support. His encouragement and considerate
assistance made my journey of doctoral study complete, though in many times I lacked
patience and endurance to pursue the study. His generous understanding and support
helped me be more confident in my abilities to do this study. I also appreciate the way he
shared his knowledge and experience in Workforce Development & Education Policy.
committee in Human Resource Development, as well as, my boss for the last four years.
He always challenged the logic and ideas that I brought to him. I cannot help admitting
his continuous challenges improved the quality of this work, but also the quality of who I
am as a young scholar. I was blessed to work with him as a GRA from the beginning of
learned fundamental, essential research methodologies through classes and meetings with
A special thank goes to Dr. Ada Demb, who served on my general exam
committee in Higher Education. I appreciate the role model she provided as a woman
v
scholar and learning from her about international view points and acceptance of a
diversity of culture.
I would like to thank Ms. Jamie Klinger and Ms. Carla Wood at Ohio Department
of Development for their proactive support in data collection. Although collecting data
required tremendous effort, their help greatly relieved my burden. I also appreciate their
Ewha Womans University. She has been my lifetime mentor for the last 15 years and has
guided not only my academic and professional life but also my personal values. I deeply
I also owe gratitude to so many of my friends at The Ohio State University. They
shared their wisdom and experiences about how to be successful in this academic
program. I want to thank them for their friendship and for our mutual learning. I also
want to thank sisters and brothers at the Korean Church of Columbus for their prayer in
the Lord.
I do not know how to express my gratitude to my parents, Mrs. Eun-Suk Lee and
Mr. Ju-Hyun Paek. Without their support and trust, I might not have been able to start
this study. They have taught me how to follow our Lord through everyday their lives.
Above all, I would like to praise the Lord who prepares my way and completes
His good will through my life. He has expressed His great love and support to me
through every day, intimate living with Him. I joyfully appreciate everything He has
provided me in this work, and hope to glorify Him through this work.
vi
VITA
PUBLICATIONS
Research Publication
vii
2. Paek, J. (2002, October). Women workforce development in Korea: Issues and
environmental forces. Proceedings of the SOM Conference on Globalization, Innovation
and HRD for Competitive Advantages (pp.229-236). Bangkok, Thailand.
3. Hawley, J. D. & Paek, J. (2005, March). Developing human resources for the
technical workforce: A comparative study of Korea and Thailand. International Journal
of Training and Development, 9(1), 79-96.
4. Hawley, J. D. & Paek, J. (2004, November). Developing human resources for the
technical workforce: A comparative study of Korea and Thailand. In Y. Moon, A. M.
Osaman-Gani, S. Kim, G. L. Roth, & H. Oh (Eds.), Proceedings of the Third Asian
AHRD Conference, (pp. 328-335). Seoul, Korea.
5. Jacobs, R. L., Wanstreet, C. E., & Paek, J. (2004, May). Using system theory to
evaluate organizational change in an Indian software firm: A case study. Proceedings of
the Fifth Conference on HRD Research and Practice across Europe. Dublin, Ireland.
6. Lee, C. & Paek, J. (2003, May). Exploring strategic staff development in higher
education institutions. Proceedings of the Fourth Conference on HRD Research and
Practice across Europe. Toulouse, France.
FIELDS OF STUDY
viii
TABLE OF CONTENTS
Page
Abstract …………………………………………………………………………………..ii
Dedication ……………………………………………………………………………….iv
Acknowledgments ………………………………………………………………………..v
Vita ……………………………………………………………………………………...vii
List of Tables …………………………………………………………………………….xi
List of Figures …………………………………………………………………………..xiv
Chapters:
1. Introduction …………………………………………………………..……… 1
4. Results ……………………………………………………………………….. 83
x
LIST OF TABLES
Table Page
3.1 Cronbach’s Alpha Coefficients for the Survey Responses (Senior manager survey,
n=44)……………………………………………………………..…….…...……76
4.2 Company Size, Number of Participants in Training, and Base Wage of Training
Participants among Respondent Companies, Non-respondent Companies, and
Not-in-Sample Frame Companies (OITP database)………………………..……87
4.11 Independent Samples t-test on In-house Training Staff and Training Program
Effectiveness (Manager survey and Senior manager survey)…………………..101
xi
4.12 Independent Samples t-test on Educational Institution Providers and Training
Program Effectiveness (Manager survey and Senior manager
survey)…………………………………………………………………..………102
4.13 Independent Samples t-test on Private Training Providers and Training Program
Effectiveness (Manager survey and Senior manager survey)…………………..103
4.20 Independent Samples t-test on Training Needs Assessment and Training Program
Effectiveness (Manager survey and Senior manager survey)……………….….112
4.21 Correlation Matrix between the Quality of Training Needs Assessment and
Training Program Effectiveness (Manager survey and Senior manager
survey)…………………………………………………………………….…….114
4.23 Distribution on Training Participants’ Portion among Total Employees and Level
of Customizations (Manager survey, n=45)……………………………………117
xii
4.24 Summary Data: Regression of Perception of the Specific Training Program
Effectiveness on Selected Variables in Training Program Characteristics
(Manager survey and Senior manager survey)………………………………....121
xiii
LIST OF FIGURES
Figure Page
2.1 Conceptual Framework for the Study of Training Program Characteristics and
Training Effectiveness among Organizations Receiving Services from External
Training Providers…………………………………………………….…………62
4.1 Number of Survey Participant Companies in each County in Ohio (OITP database,
n=45)………………………………………………………………………..……95
xiv
CHAPTER 1
INTRODUCTION
diversified customers have also impacted the business environment. Those constant
1
Training as an HRD intervention has played a significant role in improving
create job performance outcomes as well as to enhance employee’s knowledge and skills
(Lewis, 1996). Although business organizations are able to generate, develop, and
training has been utilized as a major means for sustaining current employee development
and partnerships with other organizations (Osterman, 1995; Knoke, 1997; Stewart, 1999).
In fact, in this fast changing environment, it is very difficult for business organizations,
provide all the necessary training programs to meet their organization’s training needs
internally (Knoke & Janowiec-Kurle, 1999). Business organizations may hire outside
training program for their needs, or only hire instructors. Also, business organizations
may select outside training providers for their entire training program design and
for U.S. companies sharply increased from 9.9 billion dollars in 1994 to 19.3 billion
2
One of the most important sources of training among external training providers
are higher education institutions (Carnevale, 1998; Hagen 2002; Johnston, 2001;
benefits from partnerships, partnerships between industry and university have expanded
(Campbell & Slaughter, 1999; Hagen, 2002; Normile, 1996). Government agencies also
need for continuous professional development, for flexibility and for continuous
adaptability to change (Mavin & Bryans, 2000). However, it is very challenging for
higher education to help increase the effectiveness of the workforce (Hanna, 2001;
Lumby, 1999).
business and industry in comparison to other higher education institutions in the U.S.
programs by community colleges (Johnstone, 1994; Lynch, 1991). Because one of the
major goals of community colleges is to train and educate the current and future
workforce for their communities, community colleges have been involved in training and
increased, the types of training programs also changed to meet the various training needs
of organizations. The training of subjects ranges from basic writing and math skills to
high tech skill training like computer software and managerial types of training (Cappelli
et al., 1997).
3
Statement of the Problem
training and made great efforts to improve training quality in order to sustain
competitiveness and improve performance (Jacobs & Washington, 2003). However, due
for identifying training needs, and for developing and implementing training programs
1997; Sole 1999). Among outside training providers, community colleges have actively
between external training providers and client organizations has become one of the major
As partnership training has become more common, the nature of the relationships
training, varies (Hawley et al., 2005). Some partnership training programs with
community colleges have formal contracts while others may not. Some participants in the
partnership training complete the training program with or without academic credentials
and their history of the relationships might vary. In addition, different types of training
4
providers add to the diversity of the relationship among client organizations and external
training providers.
assessments vary. Business organizations have different training needs and resources and
have their own training design process. At the same time, external training providers also
have different resources and experience in training design process and implementation.
Thus, some training design processes in partnership training programs heavily involve
needs assessment while others may not. Therefore, partnership training is widely
organizations (Ellis & Moon, 1998; Gold et al., 1998; Hall & Scott, 2001; Hardingham,
1996; Ryan & Heim, 1997; Mavin & Bryans, 2000; Roever, 2000). Business
organizations often have partnership training programs simply because of the need to
develop a full range of training programs to meet organization’s needs. Previous research
on partnership training has found that training programs serve various needs including
organizational development and employee development. (Ryan & Heim, 1997; Roever,
It is also believed that the outcomes of training for organizations are dependent
not only on the quality of the training needs assessment and the nature of the training
program such as the type of training and extent of customization, but also on the nature of
the relationships with external training providers. Although the partnership training
provided by external training providers has unique characteristics in the nature of the
relationship among client organizations and external training providers, training needs
5
assessment, and the nature of the training program, there is little information about
whether those unique relationships, training needs assessment, and the nature of the
training program relate to any outcomes of training. In other words, literature provides
little information whether the training effectiveness differ if the nature of the provider-
client relationships, training needs assessment, and the nature of the training programs
Therefore, the purpose of this study is to examine the impact of training program
the nature of the relationships among client organizations and external training providers,
the training needs assessment, and the nature of the training programs. In addition, it is to
Research Questions
2. Does the degree of training program effectiveness differ based on the nature of
6
3. Does the degree of training program effectiveness differ based on the quality of
and expected outcome? How do these differences in the nature of the training
Definition of Terms
The major terms for this study have been operationally defined as follows:
or universities and organizations (Rohdes, 2001; Rowley, et al, 1998). There are a wide
range of formats including research projects, training and education courses, placement,
educational institutions such as colleges and universities for client organizations in both
7
the private and public sectors. The ultimate goal of the training is to improve current
institutions such as colleges and universities, vocational schools, and public supported
training centers for organizations both in the private and public sectors (Allen, 2002;
Aslanian, 1988)
to maintain their presence. These organizations are current users of training programs or
potential training program users. They can be private companies, public or government
training services. Training services include training needs assessment, training program
design, instructional materials, training courses, training instructor, or the delivery of the
of the entire training program. Training providers can be educational institutions such as
8
Training design process. A training provider’s process to develop training
instructional strategy development, delivery methods and media selection, and evaluation.
Training needs assessment. Process used by the organization to identify the needs
components in the relationship among client organizations and external training providers.
One example of the nature of the relationship is how to form a relationship between client
organizations and training providers through different types of contracts. Other examples
follow-up contact, and 4) the external training providers’ knowledge about the business
of client organizations.
the training programs. The extent of customization, training program’s relationship to job,
the type of training, expected outcome are example of components in the training
programs.
HRD manager. Persons who are mainly responsible for HRD tasks such as work
analysis, training and education programs, and career development for employees. For
the purpose of the study, a training coordinator or training manager can be referred to as a
HRD manager.
9
Training effectiveness. Degree to which the training reaches the intended
Senior manager. Persons those who are in charge of overall operation in business
organizations and who persistently pursue operational efficiency via their management
role.
Training impact. The outcome and consequence of training result on the client
& Ramos, 2003; Lupton et al., 1999; Robinson & Robinson, 1989; Warr & Bunce, 1995).
institutions, and governmental agencies, that provide employment and training services
Limitations
1. Any generalizations from the results of this study are limited to the population of
10
3. The study is limited by training programs for currently employed workers, not
Significance
The results of this study will add knowledge to the field in three theoretical and
partnership projects, few studies conducted empirical research that evaluated the impact
studies based (Keithley & Redman, 1997; Lynch et al., 1991, McMurtrie, 2001; Normile,
1996; Otala, 1994; Powers & Powers, 1988b; Roessner et al., 1998’ Santoro & Betts,
2002). By examining the effectiveness of partnership training programs, the study will
client organizations.
This study will bring valid research to workforce development policy analysts as
well as policy makers. Many government agencies tried to stimulate their local economy
through workforce development policy. Partnership training programs are one area to be
encouraged by policy to train and educate the current, as well as the future workforce, to
11
assist the local economic development. Specifically, the result of the study will assist
more effectively and efficiently by utilizing all the current and potential resources in the
local areas. The policy can be developed toward creating synergy among local economy
policy analysts will also gain a research-based analysis tool to evaluate their workforce
development policy in terms of not only outcome level, but also process level.
Numerous studies have been conducted about training needs assessment and
customization and their relationship to the outcome of HRD training (Brown, 2002;
Goldstein, 1993; Kaufman et al., 1993; McClelland, 1994a, 1994b, 1994c, 1994d; Wright
& Geroy, 1992). However, this study will detail the difference between training programs
conducted through partnership projects. This study not only will add to the research of
training needs assessment and customization in HRD but also to the research on
The results of the study will improve current HRD practice. Organizations are
engaging with numerous types of external training providers and receive various forms of
training services. The data gathered from this study will help HRD practitioners to make
decisions about not only who will be selected as an external training provider, but also
how to engage in the outsourcing process including training needs assessment and
partnership training programs and can determine the level of involvement in the training
needs assessment process while considering expected results from their decisions.
12
Ultimately, they can gain assistance in developing the most appropriate training programs
In addition, the study will enhance the understanding of the evaluation of training
by measuring training results at the individual level including learning and change
behavior, but little research has been conducted to measure the impact of an
organizational level. On the other hand, labor economists have focused on measuring the
However, there has been little research linkage between training effectiveness and
training impact on organizational performance. The results of this study will provide
and whether one can be a good indicator of another. The information about this
The findings of this study will reveal senior managers’ awareness about training
impact and how they perceive training links with their organizations’ mission. Thus, it
will provide business leaders with critical knowledge about their significant role in
training evaluation, and provide organizations with sufficient rationale to assist their
One of the most common functions of community colleges is to train and educate
current and future workforces. Many higher education scholars have studied the
economic function of community colleges and asserted that partnerships with business
industries are inevitable for their existence (Bowie, 1994; Campbell & Slaughter, 1999;
Osterman, 1995). Because the result of this study will describe how community college
13
training services impact their local business, higher education scholars can make a strong
useful knowledge from this study. The findings of the study will assist workforce
business and industries and providing training services will be provided to workforce
Overall, the results of the study will provide critical information about the
HRD practitioners, business leaders, educational scholars and leaders, as well as policy
14
CHAPTER 2
This chapter discusses review of literature in four areas. They are evaluation of
training, partnership training, training needs assessment, and nature of the training
literature review.
Evaluation of Training
This section is divided into four parts of review of literature on training evaluation.
The first part reviews HRD literature on training evaluation, and the second part of
review discusses economic literature on training evaluation. The third part discusses
evaluation of state-funded, employer bases training. The last part reviews business
The evaluation of training has been studied by labor economists and by human
resource development (HRD) scholars. From the HRD perspective, evaluation of training
15
and organizational levels (Brinkerhoff & Gill, 1994). HRD literature views the
One of the best known and most widely used frameworks for classifying
evaluation is the Kirkpatrick model. He originally proposed the model as steps in 1959
and described the model as levels in 1996 (Kirkpatrick, 1996). Kirkpatrick’s four levels
are:
Methods for measuring results include measuring costs, quality and return on
investment (ROI).
The strengths of the Kirkpatrick model lie in its simplicity, usefulness, and
comprehended and makes sense to organizations and has become the most commonly
adopted model or framework on training evaluation (Alliger & Janak, 1989). Although
16
there have been criticisms of the Kirkpatrick model, numerous studies in evaluation
Since the Kirkpatrick model was introduced, several modifications of this model
model from organizational perspective. For example, Phillips (1996) added a fifth level—
ROI level—to separate the assessment of the monetary benefits of the training compared
to its costs.
elements model by Kaufman and Keller (1994). They argued that the Kirkpatrick model
was useful for only evaluating training, and that the model needed to be modified since
organizations wanted to evaluate other types of development events. They expanded the
Although HRD research has put great effort on developing a model to aid
development perspective, few empirical studies were found due to limits of inability of
models to distinguish the actual impact of training from other variables on benefits (Gray
1997 and 1998 American Society for Training and Development (ASTD) databases based
on the Kirkpatrick model. In order to identify training impact for lower-wage workers
from employer’s providing training programs, they distinguished the courses designed for
lower-wage workers by participant characteristics. They divided the participants into two
levels: participants had fewer than 12 years of formal education, or participants in the
17
course who earned equal to or less than ten dollars per hour. In the total database of 831
courses, less than ten percent of the training courses were oriented to lower-wage workers.
newly earned knowledge or skills are applicable to their current job. It was found that the
reaction level of the participants in the lower-wage worker oriented courses was less
favorable than those in other courses. In the follow-up evaluation, usually three to six
months after completion of the course, participants’ performance change was measured
those workers who participated in the lower-wage worker oriented courses was much
higher than those in other courses, though it was not statistically significant (Ahlstrand et
al., 2003).
Economists have also studied training evaluation from a slightly different point of
view. Although both HRD scholars and labor economists applied the human capital
theory to their research, the economists’ approach is more empirical and focuses more on
return on investment of human capital. The human capital approach assumes that workers
their lifetime earning so that workers pay the full cost of general training and earn the full
One of the key distinguishing features in the labor economics’ training evaluation
is the concept of present value. The training cost is paid in current dollars while the
18
training ROI is expected to be gained in the future. Hence, the expected return should be
calculated at the present value, which is discounted by the long-term interest rate when
Another key feature is that economists have conducted a number of empirical and
less clearly defined than in the HRD area. It is generally understood as all the training
programs that are related with training receivers’ current job and that are provided by
employers for their employees including in-class training and on-site training (Barron et
Economists used experimental data to find out how this training works. For
example, many research questions are related to the effect of training on earnings or
employment rate or duration, which require empirical data to provide answers (Acemoglu
& Pischke, 1999). Even though there are concerns regarding using only simple estimators
to assess training returns, and even though they admit limits of inability to assess non-
Barron, Berger, and Black (1999) studied the relationships among on-the-job
training, starting wages, wage growth and productivity growth. Two survey data sets
were used: (1) an Employment Opportunity Pilot Program surveyed 5,700 employers and
gained a working sample of 756 workers in 1980 and 1982 and (2) a survey of 3,600
firms in the Small Business Administration in 1992 gained a working sample of 1,323.
Training was measured by time spent in formal training, time spent in informal training
with supervisors or coworkers, and time spent watching others perform. The study used
19
several dummy variables such as years of education, industry, occupation, and unionized
However, the study’s findings were different from the hypotheses based from
human capital theory. The study found that workers who required less training are more
likely to earn a higher starting wage but those who required more training than the typical
worker did not receive a lower starting wage. It means that companies bear the major
portion of training, not workers. Based on the human capital theory, firms do not pay for
the general training, but the survey represented more than 60 percent of training was
general training, and only eight percent of training was pure firm specific. General
training is that new knowledge and skills learned by new employees in this training are
useful outside of the company while specific training is that new knowledge and skills
learned are too firm-specific to be used in the outside of the company (Borjas, 2000). The
study also found that wage growth was weakly correlated with training but productivity
growth was highly correlated with training. Thus, the impact of training on productivity
growth is much larger than the impact of training on wage growth (Barron et al., 1999).
Every state government has various forms of training programs. For the last
business and industry from other regions, (2) retention, revitalization, or expansion of
existing business and industry, and (3) development of new business and industry (GAO,
20
2004; Grubb & Stern, 1989; Hodson et al., 1992; Moore et al., 2003; Regional
Technology Strategies, 1999). Not only in the United States, but in many other
developing countries in Asia, South America, and Africa, local and federal governments
have developed community colleges as workforce training agencies with the cooperation
programs have been evaluated by various local and federal governments and independent
researchers. Their evaluations also have been conducted based on legislation and policy
need more in-depth of evaluation in terms of their impact at the firm level, the employer’s
reason for selecting training providers, and linkage with other state and federal higher
education programs (GAO, 2004; Regional Technology Strategies, 1999). Although most
the results of the study found that there were weak linkages between state-funded,
institutions, though states’ interests were increasing to connect both entities (Regional
21
One state with the best practices, California, showed that major training providers
are community colleges. While few states require independent, outside evaluations, the
evaluation is designed to measure pre-12 months of wages and post-12 months of wages
This report also concluded similar results with a previous survey in 1999. The report
stated that all 23 states assessed their training programs in 2002, but none have
terms employee’s wages or company’s earnings. In 41 percent of the states, internal state
government staff evaluates the programs, while only four percent of the states hire
sample purposefully in order to capture a wide range of business types and sizes, and
used both qualitative and quantitative evaluation methods. They measured the quality of
training through assessing the quality of instructors, training materials, and customization.
They applied Kirkpatrick’s four levels of evaluation to assess learning from training.
developed a very interesting framework for measuring the value of training impact. The
22
framework represents that the value of potential gains realized from training is a function
wages. The growth in the number of employees was calculated compared to growth on
average of the same industry, and the growth in total wages per employees was also
calculated compared to average wage growth in the same industry. One of the rationales
for measuring them is because they are only uniform data available cross companies
measure learning pre and post training intervention. Self-reporting methodology has
model with four levels has been one of the most popular models in training evaluation.
Hence, many researchers have developed measures to assess training outcomes based on
the Kirkpatrick’s evaluation model (Alliger et al., 1997; Warr & Bunce, 1995). A recent
reported that 78 percent of companies use reaction measures, 32 percent use learning
23
measures, and less than 10 percent of companies assess measures of behavioral change or
Trainees’ reactions, the level one in Kirkpatrick’s model, have been measured by
perceived usefulness should be differentiated and that both should be used to measure
trainees’ reaction. Warr and Allan (1999) further developed measurement in this reaction
level and found that differentiated measures are more closely related to learning
outcomes than traditional measures. They used trainee’s enjoyment, perceived usefulness,
trainees’ pre-motivation and confidence level. One other recent study added two more
measurements such as post-transfer utility reactions and transfer climate reactions to the
knowledge is generally used. The amount of learning from the training usually is
measured immediately after the program. More recent studies asserted trainees’ perceived
investigated how an individual’s self efficacy is critically associated with higher learning
efficiency. Learning outcomes are separately measured based on types of learning such as
cognitive learning, skill-based learning, and affective (or attitudinal) learning outcomes
(Sekowski, 2002).
24
Behavior change or transfer, the third level in the Kirkpatrick’s model, is usually
measured based on supervisor’s observation or trainee’s self-reporting at the pre and post
training points of time. Level two and level three of the Kirkpatrick’s model have been
Measures in all three levels are believed to measure training impacts consistently.
Warr and Allan (1999) studied 23 two-day training courses attended by motor-vehicle
technicians. All participants completed tests and questionnaires pre-training and post-
training. The results of the study found that the measures of the first level have
statistically significant relationships with the measures in the second level and in the third
level. For example, for those who enjoyed the course and perceived utility, presented
Leach and Liu (2003) also found that there are positive relationships among
reactions, knowledge acquisition, and behavior change. They asserted that those who had
constructive reactions to training have some potential to learn materials, and those who
with a higher level of knowledge retention were more likely to apply it in their work. The
result of their study found that measuring level two is most related to the results of
outcomes, assessing knowledge transfer is the best indicator among Kirkpatrick’s level of
evaluations.
The results or business impacts has been considered to most difficult measure
since not a single event affects them, though level four outcomes are considered most
tangible (Goldberg & Ramos, 2003; Lupton et al., 1999; Warr & Bunce, 1995). However,
25
researchers have made efforts to identify relatively reliable measurements such as sales,
productivity, cost, quality, and turnover rate. The following section will describe what
measures have been identified to assess the fourth level of organizational outcomes.
Financial Measures
Business and economic scholars have frequently used financial measures to assess
training impact. However, most studies on returns from training have focused on
individual returns to training (Jacobs & Washington, 2003). Thus, an individual’s wage
has been one of the most popular measures in assessing training impact, in particular,
training outcome at the organizational level. Moore, Blake, Phillips, and McConaughy
measured change of total wages per employee for all the employees which included
influenced increased organizational performance, total wages of all the employees should
increase overall.
measures such as growth, earnings, return of investments, or unit costs, though few
26
Growth is one of the conventional measures to indicate organizational
Although growth itself does not promise increased income, it is set as many
scale in many industries, growth in revenue and/or in production directly relate to lower
unit costs and higher return. In addition, if companies’ short-term goal is to increase
revenue, growth becomes the major measure for organizational performance (Kalleberg
only operational efficiency but also efficiency of financial decisions are to be measured,
net income is more appropriate. Operational earnings are equal to subtracting operating
expenses, depreciation and amortization from the total revenue. Operating expenses
include production costs and overhead expenses such as energy, maintenance, human
resources, marketing, and any other supporting costs. Net income is equal to subtracting
financial expenses (loss or earnings) from operating earnings. Financial expenses include
Non-financial Measures
employee satisfaction, market share, product quality, retention, and turn-over rate. These
27
Since there are relatively few reliable measures available to assess performance of
which have a positive relationship with their performance. They include job satisfaction,
turnover rate, absenteeism, and organizational commitment (Leach & Liu, 2003; Lincoln
seven biopharmaceutical companies. Although her work does not assess results of
commitment. The flexibility was measured by three levels: formal, informal, and usable
organizational commitment. The results of the study found that usable flexibility policy
small companies. She used an organizational scale drawn from Lincoln and Kalleberg
(1990).
customer relations. Because salespeople work on a commission base, they are believed to
be relatively sensitive in time spending and time-value oriented. Thus, the authors
colleges, a couple of state government’s departments, public training center, vendors, and
28
local chamber of commerce for client organizations. In this study, Pauley assessed
(Anderson et al., 1994; Banker et al., 2000; Ittner & Larcker, 1998). However, the results
of those studies are not consistent. For example, one survey study analyzed customer and
business unit data for two service firms. It found that customer satisfaction measures are
positively related to future financial returns. However, the results of their cross-sectional
data analysis did not show any consistent associations between customer satisfaction and
market returns (Ittner & Larcker, 1998). Another study analyzed 77 Swedish firms from
various industries and showed that customer satisfaction is positively related with
concurrent returns on investment. But it showed that there are negative or weak positive
However, a more recent study asserted that those inconsistencies are due to an
inappropriate time lag between customer satisfaction measured and financial returns
hotel chain data to determine the relationship between customer satisfaction and long
term financial performance. The results of the study found a significant association
revenue and operating profit. They described that a single measure of customer
satisfaction is just as effective as a combined complex measure. They asserted that the
29
time lag should be shorter if the study is applied in service sector, and that the time lag
One very extensive training evaluation study used change in the number of total
However, the authors asserted that the increased number of employees compared to the
average increase at the same industry could represent company’s viable growth and
increased performance. They believed that greater growth in employee size could be a
proxy for company success. They used the percentage change to the total number of
industry partnership and partnership training. The first part reviews literature on
employee levels of impact. The third part reviews literature on rationale of partnership
University-Industry Partnerships
industry and university in various areas. That is the reason why university-industry
partnerships have been implemented in a wide range of communities, and why industry’s
30
participation in partnership programs has increased. Before looking at the current practice
be discussed.
During the 1920s and 1930s, private foundations were the dominant external
source of funds for university research. At the time, those funds went into basic research
with goal of benefiting mankind. After World War II, university research partnership
arose out of need for intensive research requested by the federal government. The
Department of Defense, the Atomic Energy Commission, and NASA were major
supporters of research at universities. By that time, the value from business interests was
also realized among universities, though industrial supports grew gradually (Bowie,
1994).
In the late 1970s and early 1980s financial support to universities—usually for
research & development (R&D)—from the federal and state governments started to
decrease, and universities began to look for other financial sources from business, which
increasingly common and intensively implemented (Bowie, 1994; Campbell & Slaughter,
1999).
Besides financial challenges, there were several other pressures that encouraged
important to achieve competitive advantages for industry. This shift motivated business
31
and industry to consider universities as their source of knowledge. Thus, industry’s
sponsorship to universities has taken the form of partnerships with universities (Santoro
survey sponsored by the National Science Foundation showed that the reasons companies
(1) to access to manpower, (2) to obtain technology and scientific knowledge, (3)
(1) to access a new source of money and to diversify university’s funding base,
facilities and research data, and (5) to provide better training for increasing
However, the ultimate goal beyond these motives is to improve practice. Thus,
practice” (Jacobs, 1999; Starbuck, 2001). When university and industry conduct
discussed above. Hence, a university-industry partnership might have dual goals: industry
32
wants to solve immediate organizational problems or issues while faculty (university)
Two major questions are why industry wants to solve their organizational
problems and why researchers want to generate new knowledge. Industry wants to
building new knowledge to apply practice. Jacobs (1999) defined HRD partnership
research as “the process of improving HRD practice through research” (p.874). The
through research, training, or services. Starbuck (2001) also defined the university-
industry partnership as “working together toward a common goal” (p.2), and the common
intensity of collaboration and scale of intervention. The most conventional and common
to act as a consultants and analysts, sponsoring and facilitation applied research programs
33
in management development, and developing a ‘learning organization’ through
partnership (Bailey, 1995; Ellis & Moon, 1998; Keithley & Redman, 1997; Pearce, 1999;
Roever, 2000; Thacker, 2002; Wells, 1999). Recently the traditional ‘customer-supplier
involving a mixture of learning, consultancy, and research (Keithley & Redman, 1997).
level are discussed in a review of evidence from the following case studies.
in continuous learning and gain new technology and knowledge for their continuous
34
universities to industry contributed 38 billion dollars to the economy, creating over
300,000 jobs and forming hundreds of new companies (Hall & Scott, 2001). Another
study found that technology and knowledge are transferred from universities to industries
not only from the research partnership projects but also from the training and education
appropriately trained workforce who can increase technology transfer (Ryan & Heim,
1997).
participants to view organizations from a more objective point of view and to initiate
organizational change as change agents. For example, all the Company Associate
partnership between British Small and Medium Enterprises (SME), recent graduates,
academic support from a local college and Leeds Metropolitan University in 1993.
During the program, even those who were resistors of change could understand the need
of and advantages of change, and those who supported change understood the resistor’s
fear and minds. Thus, their graduates could help SMEs to change their organization and
Public sector scholars also thought that universities were uniquely positioned to
play a role which encourages individuals and their organizations to critically challenge
their ways of working and thinking. Thus, in the 1990s when the public sectors faced
requests to become more effective organizations, Mavin and Bryans (2000) described
35
the development of continuous learning, culture change, organizational development
workforce and develop their participating employees’ skills and knowledge not only from
training partnership programs but also even from research partnership projects (Ellis &
Private sector companies were not the only entities to value external training
training and development, reflecting the need for continuous professional development,
for flexibility, and for continuous adaptability to change (Mavin & Bryans, 2000). Even
assistance of Hong Kong five star Hotels—and educated international mangers with
development and organizational performance (Jacobs & Washington, 2003). Even though
there are not many studies that have proved a clear linkage between employee
36
Compared to traditional education, participants in university-industry partnership
gives participants
“more convenient time and places to study, less worry about finances, clearer
with other students who have similar career objectives, more chances to practice
classroom skills on the job, less concern about jobs after graduation, and greater
during the training period, their placement rates are very high. For example, placement
rates in the Ben Franklin Programs, technology training education programs through the
partnership between the Behrend College and a consortium of several small companies
with common interests in plastic and material technology, were almost at 100 percent
East Tennessee State University and Sprint adopted the open system approach to
discipline knowledge with a focus on solving problems (Yasin et al., 2000). Education of
performance, job satisfaction, and their employment status at their workplace (Johnstone,
1994).
37
Although there are a few empirical studies found to examine impact of university-
organizations.
New York Telephone Company and Empire State College. In 1991, the State
University of New York’s Empire State College and New York Telephone Company
short-course training and tailored programs. This partnership occurred because the New
competitive market and customers’ needs required excellent service using state-of-the-art
technology. With the assistance of the New York City public schools, Empire State
College and New York Telephone recruited one hundred senior high school graduates
who were willing to go to college but could not afford it because 92 percent of them
came from minority groups. In addition, 30 current employees and 20 from outside the
College’s program flexibility and capacity for partnership program. Nine months before
38
seminar’ for young students and ‘intense personal mentoring’ program. While the
average attrition rate in the community colleges in New York City was 50 percent at the
end of first semester and 80 percent by the end of one year, 90 percent of the
corporate/college program students completed the first semester and over 70 percent of
It was too early to measure the long-term financial return from this partnership
project to New York Telephone, but a positive indicator for productivity appeared. The
younger students, those who were hired right after high-school graduation and
participated in the partnership program were among the best salespeople in every office
and often outperformed their senior colleagues. From the evaluation, students expressed
further education also increased due to this program (Johnstone, 1994). This case study
organizational performance.
DuPont Corporation and Penn State University. DuPont and Penn State have a
(TQM). DuPont has adopted TQM as the cornerstone to restructuring the company
following downsizing and reengineering. The partnership developed a TQM model for
DuPont and facilitated adaptation of the model at DuPont. From this partnership project,
DuPont and Penn State established an additional communication channel that enabled
direct linkage between university and corporate officials in various strategic areas such as
39
human resource development and continuing education, technology transfer, and the
technology from sources outside of the company. Since the director of technology
acquisition at DuPont and the director of the Industrial Research Office at Penn State
DuPont and Penn State were able to successfully launch research and technology
to the DuPont’s training needs. DuPont team members were also involved in an
& Heim, 1997). This case study showed that initially developed partnering relationship
Cummins Engine Company and Teesside Business School in UK. Cummins, one
of the major players in diesel-engine manufacturing in the world, faced severe global
Cummins production systems (CPS). CPS led to Cummins re-evaluating its management
activities and found that new management skills would be required to support CPS at all
levels.
40
The partnership program designed several part-time, integrated programs with a
Most courses were offered as a work-based learning approach and were comprised of
monthly residential and open-learning sessions for 24 months. Most managers were
encouraged to develop their own personal development plans to implement CPS, and
as Teesside Business School faculty members. Although programs were validated by the
Following the success of the initial programs for supervisory staff, another
partnership program for middle managers was introduced with a broader range of
development, and so on. This middle manager program was also linked to the university
validated qualifications.
This case showed the impact of workforce development outcomes for both the
identified the underpinning knowledge and skills needed by middle managers and
training program. Second, Cummins were able to train their managers to change
managerial attitudes and behavior that the new culture required under CPS. Third,
because participants were able to gain some recognition in terms of nationally validated
earned not only professional knowledge for their career but also an academic credential
(Keithley & Redman, 1997). Overall, these three cases showed that university-industry
41
partnerships assisted companies’ employee development, organizational development,
Partnership Training
internal labor markets that supported a higher quality workforce via their own trainings.
However, as investment in training was getting costly, only larger companies can retain
comprehensive training centers to assess, train, and evaluate their workforce, and more
companies face with the decisions about ‘make or buy’ training. Although moving
one of the major trends that arose in organizations in the 1990s was to outsource training
training providers for their employee training. A1991 national survey showed indicated
that only about five percent of organizations developed and implemented their training
functions with their own staff and/or parent organization staff. Twenty-three percent of
organizations utilized only external training providers, but this percentage was higher in
smaller organizations. Thirty-five percent of organizations have their own training staff
have their own training staff, a parent organization’s training staff, and outside training
providers. This survey showed that most organizations collaborate with external training
providers for their employee training, though organizations maintain their own training
42
Knoke and Janowiec-Kurle (1999) described three kinds of training modes
utilizing external training providers: (1) the company asks employees to obtain training,
individual employees choose and attend training programs from external training
providers, and the company pays the cost, (2) the company staff set training
goals/objectives, choose external training providers, and ask them to conduct the entire
training program, or (3) the company staff and external training providers collaborate to
Why Outsource?
It is evident that the choice between internal and external training providers
depends on the budget and costs. That is true in many cases not only for private
companies but also for public sectors as well. For instance, after their intensive study on
outsourcing training, the Center for Naval Analysis recommended that the Navy engage
However, there are much more complex issues beyond the cost and benefit issue.
There are tradeoffs to using internal or external staff. The key advantages of using
internal training staff as internal trainers are following. Internal trainers: (1) understand
the company’s culture and organizational characteristics, (2) are perceived as colleagues
by the trainees, and (3) are seen as having more influence with the company management.
In contrary, external training providers: (1) bring knowledge of other organizations, (2)
are perceived as more expensive so their opinions are perceived as more valuable, and (3)
sophisticated, specialized training to generic, basic skill trainings. Some of the training
43
providers are non-profit entities such as voc-tech schools, and community-based learning
centers, while others are commercial training providers. Community colleges and
on partnership training assert that partnership training between internal staff and external
training providers brings far more benefits to the company than either utilizing only
internal training staff or external training providers (Hardingham, 1996). On the other
hand, Moore (2003) asserted that internal staff implemented more effective training
relationships with outside training providers. A company may have a formal relationship
with a formal contract and exchange companies’ strategy with outside partners or may
have just informal relationship without a contract. However, formal partnership contracts
have usually involved a contract to provide training between community colleges and
business organizations. Many researchers asserted that a formal contract is necessary for
training programs to produce a good quality of training outcomes (Hawley, 2003b; Kenis
involvement and commitment. Because many studies have proved that management’s
44
support is one of the strongest factors for success of training programs, a formal
For example, O’Rear (2002) found from her study that business leaders’ support for
training in the training design and evaluation process produced significant relationship
This section is divided into three parts of review of literature on needs assessment
and training needs assessment. The first part describes definition of needs assessment and
training needs assessment. The second part reviews research on ways of conducting
training needs assessment. The last part reviews the roles of needs assessment
It is widely believed that needs assessment is one of the key phases in the training
performance, and to prioritize current resources to reduce those gaps (Altschuld, 2003).
Needs assessment can be defined as “a systematic set of procedures undertaken for the
improvement and allocation of resources. The priorities are based on identified needs”
45
developing need-based training programs either to meet organizational needs and/or
individual needs for training. Therefore, Brown (2002) asserted that training needs
Although numerous studies exist regarding the ways to do needs assessment, not
many studies are can be found to test the impact of needs assessment on training outcome
or the relationship between quality of needs assessment and training outcomes. In this
One of the major topics discussed most frequently in needs assessment are data
documents, and work samples (Goldstein, 1993). Witkin and Altschuld (1995)
categorized needs assessment data gathering methods into three types based on data
sources such as archival data, noninteractive and interactive communication, and analytic
data. Archival data includes records, logs, demographic data, census data,
sources come from written questionnaires, key information interviews, or critical incident
technique. Interactive communication data comes from nominal group techniques, focus
group interviews, or futures scenarios. Analytical data can be gathered from fishboning,
success mapping, task analysis, fault tree analysis, risk analysis, or trend analysis.
46
1994c), and on-site observation (McClelland, 1994d). His articles provided weaknesses
factors influence the quality of data gathering practice, many techniques have been
introduced to identify the most appropriate techniques given limited resources. Many
studies described the advantages and disadvantages of different techniques for data
Archival data can be used to identify training needs include production records,
defect records, accident logs, social indicators, and other data related to the training area.
The archival data are mainly quantitative which assists in determining the current status
of a problem or training needs. It is relatively easy to gather this type of data, but many
times provides only limited information related to training needs (Witkin & Altschuld,
1995).
Survey methods are the most frequently used methods in training needs
Surveys can collect a large amount of information from many people and can minimize
some forms of bias. It also has many statistical inference methods to analyze these
collected data (McClelland, 1994a). Because surveys are very structured, identified needs
are uniformly collected but mainly the data is mostly qualitative such as perceptions,
1995).
On the other hand, surveys also have some disadvantages. Surveys might take a
longer period of time to collect the data, even though it may be administered through
47
internet or email. Surveys may provide unclear, often subjective results, but it is difficult
to ask follow-up questions to participants (Gray et al., 1997). In addition, people can
easily provide an expected or socially accepted answer rather than their true opinion
Interviewing includes key consultation with those who understand the training needs for a
group or organization, individual interviews with persons who will participate in training
sessions, and focus group interviews with individuals who are knowledgeable regarding
training needs (Miller & Hustedde, 1987). Interviews provide ample opportunities for
respondents to express their opinions and feelings more completely than other methods.
In particular, those opinions and judgments can be from experts if they interview
Unlike other methods focus group interviews allow the group to reach a certain
level of consensus for potential solution generation. Focus group interviews and various
group procedures also uniquely allow follow-up discussion to inquire further regarding
the rationale for suggestions or the genesis of ideas in order to refine the expressions
through discussion with others (Eastomond, 1994). Group techniques additionally play a
function of informing people about what taking place, thereby encouraging their support
Because various group techniques are also able to gain unexpected results or ideas
regarding training needs, and because they serve various purpose of training needs
assessment, they, like focus group interviews, are one of the most widely used techniques
48
disadvantages of individual interviews and focus group interviews include the amount of
time required and the skills necessary as an interviewer or facilitator of the group
(Goldstein, 1993).
Training needs assessment can be conducted through observation at the work site
by subject matter experts. Although the observation method requires a great amount of
time and the observers should have content and process knowledge, it can provide highly
relevant information to the work setting (Goldstein, 1993). Because relatively few
subjects can be observed, this technique is usually limited to a specific job level
(McClelland, 1994d).
currently, people use a specific model or analytical framework for training needs
assessment. They are fishboning, success mapping, fault tree analysis, and human
resource competency model specific for training needs (Gorman et al., 2003). Using these
frameworks helps identify root causes of the problems or causal relationship for the
purposes of the needs assessment. Obviously the first purpose of needs assessment is a
gap analysis between desired performance and current measured performance to identify
training needs (Kaufman et al., 1993). However, that is not the only purpose expected.
Many studies found there are more roles of training needs assessment that are beneficial.
49
A study of public organizations stated that the top three roles of training needs
assessment are (1) to introduce new programs, (2) to address performance and
productivity problems, and (3) to link employee performance with organizational goals
Rossett (1987) described six main purposes of training needs assessment. (1) to
seek optimal and actual status to determine detailed discrepancies, (2) to understand
perceptions of real people and in the real world, (3) to look for causes of a problem, (4) to
seek priorities, (5) to make significant parties involved, and (6) to train management in
not a process of searching for the right answer to the organizational problems. It is a
Brown (2002) also described four main roles of training needs assessment. The
first role is to identify specific organizational problems which will guide training
HRD department, no one really knows which kind of specific training program solves
specific problems. Thus, training needs assessment should first identify specific problems
in an organization.
The second role is to obtain management support for the training program. In
general, it is not easy for managers to see the actual impact of training programs on their
50
see potential results of the training program. The third role is to generate data for
evaluation. Because training needs assessment sets specific goals of the training programs,
those goals can be the criteria when the results of training programs are evaluated. Thus,
training needs assessment also can serve as a basis for evaluating the effectiveness of the
training program.
According to Brown, the fourth role of training needs assessment is to analyze the
costs and benefits of training. A training needs assessment is able to identify potential
benefits from specific training by discrepancy analysis in performance and potential costs
to conduct training. The training costs and benefits analysis assist top management in
In summary, few studies were found that address the actual impact of needs
assessment on training. Most existing studies on training needs assessment discuss the
ways to conduct needs assessment, the level or areas to be assessed, and role and purpose
and appropriate levels and areas assessed is also important, but most of all, how carefully
needs assessment is planned and implemented might be the most critical piece of
Several characteristics determine the nature of the training program. For example,
whether the training program is job-specific or job-related can determine one aspect of
the nature of the program. Types of the training program and employee levels also
determine other aspect of the nature of the program. This section of the literature review
51
discusses some of those characteristics which determine the nature of the training
Customization
The extent of customization is one of the greatest determinants of the nature of the
program developers may customize some standard instructional programs to fit their
organization’s needs, the origin of customization began with outside training providers,
Jacobs, 1993).
and Stern (1989) defined customized training as “relatively firm-specific skill training for
individual firms, and therefore, a form of training which is more specifically responsive
to a firm’s requirements than are general vocational programs”(p.31), it does not give
refers to the middle level of customization, in which programs customize some contents,
examples, sequence, or contexts with a standard program. The other extreme is a tailored
training program, which is newly developed and uniquely structured for the client’s
52
However, because other researchers (Sole, 1999) use terms such as “customized”
For the purpose of this study, all degrees of customized or tailored training programs will
course pace (Bragg & Jacobs, 1993). Moore, Blake, Phillips, and McConaughy (2003)
described three customization areas in their training evaluation study. Adjusting training
pace and contents to the level of trainees is one method of customization. Using examples,
tools, and/or materials pulled from the company context is also another method of
customization, and to align with the company’s culture can be considered another method
of customization.
courses to highly specific non-credit technical and managerial training (Bragg & Jacobs,
1993). However, their condition for customized training requires different components in
contract. The Ohio State Council on Vocational Education (1987) described customized
training as “enter into contractual arrangements with a particular business (or group of
business) for specialized training services” (p.11). Contracts, either written or verbal,
between community colleges and client organizations set development and delivery of
customized training.
53
Motives of Customization
customization, and another approach is to assess real impacts or perceived benefits from
customized training.
technology, rapid change in high tech, and spread of innovation require continuing
training, and this increased demand has caused tremendous growth in customized training
opportunities (Hodson et al, 1992; Parrish, 1998). Parrish (1998) reported that the
organizational issues. For example, one article reported that customized training is
necessary for diversity management training for mangers and training for women and
ethnic minority groups to enter senior positions (Rajan & Harris, 2003). Another article
54
training (Dare, 1999). Many business schools also experienced dramatic growth in
their involvement in economic development in the region (Bragg & Jacobs, 1993). One
training by the late 1980s (Lynch et al., 1991). Community colleges have undergone
consistent reform in their education and training programs to meet business industry and
community needs. One of the new approaches that community colleges implemented is
training. One of the greatest advantages is that customized training assists companies in
Based on social learning theory, people can learn better from the familiar
materials and easily can apply training when learning materials are similar their actual
materials. Warren (2000) introduced one case study of customized training using
college and a large business corporation. Her data collection had three phases: (1)
reviewing written course evaluations, (2) interviewing with participants and participants’
55
three phases of evaluation, she concluded that the customized training was successful
because a new partnership between the community college and the company was formed
for a new four-year degree program. Some courses are now broadcast over interactive
television, and because many follow-up partnership activities appeared after this
customized training program. Most of all, the evaluation indicated that the original
purposes of this customized training program to equip middle managers without college
degree with more knowledge of the company’s long-term missions and plans and to
create more confidence as leaders of other employees was reached. As a bottom line,
Warren asserted that community colleges need to realize that training must be customized
Brown (1999) conducted a survey with directors and managers in training and
corporations. His survey results showed that a high level of customized training programs
with high level of measurement of learning (credit-based course) offer several advantages
“(1) to induce higher participant energy and motivation level, (2) to encourage
more rigorous learning, (3) to offer higher relevance, (4) to offer opportunities for
team building and culture change, (5) to provide more opportunities for the
application of learning and better support for participants from the employers, and
(6) to make it easy for employers to monitor the quality of training program and
participant’s performance.”
56
Several studies also have reported that the relevance of instructional content as
well as the relevance of instructional design as critical and necessary factors to support
asserted that training programs should increase similarity of training content (Baldwin &
For disadvantages, the study described fewer opportunities to exchange ideas with
managers from other companies and less open debates because power relations from the
workplace may be influential on learning environment. It also expressed one concern that
prescribed answers or solutions by the client company’s training developers might limit
Hodson, Hooks, and Rieble (1992) also conducted a study on customized training
with in-depth interviews with 65 human resource directors, trainers, and training
participants in 20 manufacturing plants. Their study specified three different settings: (1)
large, unionized monopoly sector companies that have developed intensive training
program, (2) smaller, relatively marginal sector companies that use state-supports for
training, and (3) new start-up companies that use training in communication skills and
group processes for specific job skills. The study measured the customized training by
autonomy and task complexity. The study results showed that the first group of large
companies implemented the most intensive training programs and that customized
training helped bringing specific usable skills, network, norms, and trust. However, a
significant limitation of customized training found in this study is that they developed
transferable skills but generally without a credential to identify possession of those skills.
57
Another benefit of customized training is that it may be more cost effective. If the
pure training costs of fully-customized training programs are calculated, it might be more
expensive than standard training or moderated customized training. In one study, training
providers asserted that customized training is more cost effective. Companies might pay
for an employee’s training time for one- or two-days of standard training when they can
teach exactly what they want to teach via a half day of customized training (Sole, 1999).
Relationship to Job
There are two ways to describe the relationship of training programs to jobs. First,
the training programs which are implemented at the company can be categorized as either
related to current tasks. It has been found that job specific training results in higher
earnings to low-wage workers versus training which is not firm specific (Ahlstrand,
2003).
programs to jobs. The type of training is categorized as: 1) technical, 2) managerial, and 3)
non-job related (or awareness) trainings (Jacobs, 2001a). According to Spitzer (1984) and
transfer of training programs versus far transfer of training programs. Because near
transfer can be successful when training place and content reflects the workplace, the
58
On the other hand according to Laker (1990), far transfer would be the objective
programs might be more effective when they are implemented in far transfer of training
design.
Conceptual Framework
Training program effectiveness refers to whether training achieves its intended purpose
data are often reported as training effectiveness (Brinkerhoff & Gill, 1994; Kirkpatirck,
1996; Ahlstrand et al., 2003). The first hypothesis is, if perceptions of training
client organizations.
The outcomes of partnership training for client organizations are dependent not
only on the quality of training needs assessment and the nature of the training program,
but on the nature of how business organizations have developed relationships with
training through three main factors: the nature of the provider-client relationship,
provider's training needs assessment, and the nature of the training program. Based on the
literature review, the following hypotheses are developed to conduct this study.
59
Formal partnership contracts are hypothesized to create benefits for firms. By
having formal contracts, several training conditions including training programs, training
content, expected outcomes, and trainers’ or trainees’ pre-requisites can be written more
involvement and commitment. Because many studies have proved that management’s
support is one of the strongest factors for success of training programs, formal
hypothesized that training programs with external training providers who have a formal
contract and exchange a company’s strategy are more relevant to training program
the external training providers are involved in more stages of training process, have more
effectiveness is higher.
implement some form of training needs assessment, needs assessment does not
automatically guarantee training effectiveness. The level of and quality of actual needs
performance. Thus, we can hypothesize that more levels of focus (among individual,
work process, and function) of provider's training needs assessment increases perceptions
identifies performance levels before and after training, the needs assessment results in
greater perceptions of training program effectiveness and higher than when the needs
60
The nature of the training program is also hypothesized to have a relationship
the company’s data, format, materials, and/or equipments for instructional materials,
trainees can learn more and can easily apply this learning to their work place (Brown,
1999). Thus, we can hypothesize that the higher level of customization provided among
partnership training programs, the more relevant they are to training program
effectiveness. The level of customization can be defined three ways: (1) generic/standard
training program with little customization, (2) moderate level of revision, and (3) newly
developed training programs. The areas to be customized are components of a lesson plan.
They are training purpose, trainee’s pre-requisites, content, delivery methods, and
evaluation. Thus, it can be hypothesized that more customized training programs increase
programs are hypothesized to have higher perceptions of training effectiveness than the
61
Types of Training Provider
• Internal staff
• Community college
• Four-year university
Nature of the Provider-Client • Vocational center
Relationship • Adult learning center
• Level of involvement • Private training vendor
• Follow-up contact
• Previous contract history
• Knowledge of the business
• Formality
Training Program Effectiveness
• Level of training evaluation
Provider’s Training Needs • Relative perception
62
Figure 2.1: Conceptual Framework for the Study of Training Program Characteristics and Training Effectiveness
among Organizations Receiving Services from External Training Providers
CHAPTER 3
METHODOLOGY
This chapter describes the methodology that was used to answer the proposed
research questions. The first section describes the research type, and the second section
describes the research background, study setting, and sample. The third section explains
development, and instrument validity and reliability. The fourth section illustrates data
Research Type
Ex post facto research was mainly used in this study with descriptive analysis to
investigate the impact of the nature of the relationships among client organizations and
external training providers, training needs assessment, and the nature of the program on
studies, ex post facto research belongs to descriptive analysis. The main purpose of the
descriptive research is to illustrate “what is.” Thus, while quasi experimental study is
variables, ex post facto research cannot determine direct cause and effect relationship
63
Ex post facto research is conducted after the variance of the variable occurred in
the natural occurrence, and it means that the variables can be manipulated but are not
manipulated in this research. Since ex post facto research can describe potential
research. Ex post facto research enables us to predict one variable or variables from the
knowledge of the other variable or variables (Ary et al., 2002). Therefore, in this research,
the independent variables occur naturally and are observed before the dependent
variables are observed. With the knowledge of independent variables, the researcher is
able to predict dependent variables while controlling for potential attribute variables.
(Hair, 1998)
training, training needs assessment, and nature of the training program. The researcher
proposes to investigate relationship among four variables. They are the nature of the
relationships among client organizations and external training providers, training needs
assessment, the nature of the program, and training program effectiveness. Three major
needs assessment and the nature of the program, occurred naturally preceding the
dependent variable, training program effectiveness. Therefore, ex post facto research was
64
Research Setting and Sample
Research Setting
There are four criteria to consider for a suitable research setting in this study as
following:
- Because the unit of analysis is a training program, the subject is a company not
an individual.
training funds from the state government. The Ohio Investment in Training Program
companies who received training dollars were used for this study.
development programs to attract, create and retain jobs for Ohio businesses. The OITP is
administered by ODOD and the OITP regional training coordinators, who are based at 12
and financial resources for employer-based training to attract new business and to
maintain and upgrade the skills of Ohio’s workforce. The OITP is one of the few
programs in Ohio that provides direct financial support to employers for training.
65
The mission of OITP is “to support the mission of the State of Ohio, the Ohio
financial and technical assistance to companies and collaborative efforts that strengthen
Employers can request training grant funds through a Request for OITP
grant is determined by the training activities and costs associated and identified with
training needs. Employers can choose the training provider whether it is a company
Although most companies can request the OITP fund for their training, there are
some restrictions. Media and similar businesses in such industries as publishing, motion
services and data processing services, are not eligible to obtain the government funds due
The OITP recipient companies have implemented and completed the training
consistently made an effort to monitor performance through training. Also, because they
have right to select training providers, there is enough variance among types of training
providers to use this database. Thus, the researcher selected the OITP funds recipient
66
Sample
This study used primary data and secondary data for analysis. The primary data
was gained through survey to the OITP fund recipient companies. The secondary data
Primary data
The population of this study are all companies that received training fund through
the OITP and spent the training fund on their employees’ training. The sampling frame
was obtained from the current OITP database, which was updated the 1st of October,
2004. Since the OITP first provided their training fund for companies under the name of
the OITP in 1999, 432 companies have participated and are currently listed in the OITP
database.
The researcher conducted a census study, which studied all accessible populations.
The researcher chose all the companies that implemented and completed training
programs between January 2002 and June 2004. The reason to set the selecting criteria in
terms of time is to measure dependent variables accurately and appropriately with fewer
intervening variables. If the training program was completed more than three years ago,
senior managers’ perception on a specific training effectiveness might not be reliable due
to the length of time between the training and the survey. By shortening the time between
the training and completing the survey the impact of history may be somewhat controlled.
In addition, since the training impact may have been diminished over time, the researcher
will only include training programs that were completed within the past three years
(Jacobs, 2003).
67
On the other hand, it is necessary to have at least a six month interval between
performance. Researchers on training result measurement assert that generally a one year
interval is adequate for the service industry, and six months is appropriate for the
manufacturing industry (Banker et al., 2000). Consequently, the researcher will require at
least a six month interval between completion of a certain training program and
measurement of the training impact on the client’s organization. Based on these criteria,
189 companies implemented and completed training programs between January 2002 and
June 2004.
Several efforts were made to control errors. Descriptive studies have four kinds of
errors in survey. They are sampling error, selection error, frame error, and non-response
using random sampling. Selection error can occur when some sampling units have a
greater chance to be selected than others. Because this study is not an inferential study,
sampling error and selection error do not need to be controlled. However, it was possible
that some population units might have a greater chance to be selected than others.
Because the researcher reviewed the current OITP database list and eliminated three
repetitive data, this error was controlled. However, any omitted companies cannot be
Frame error refers to a discrepancy between the intended target population and the
actual population frame. The researcher controlled frame error by obtaining an up-to-date
database from the OITP. In addition, the researcher made sure that every company has
completed contact information for survey either telephone numbers or email address.
68
Non-response error occurs when a subject fails to respond, refuses to respond or
does not return the mailed questionnaire. The researcher made great efforts to increase
response rate by calling and by sending out emails to non-respondents before the deadline.
After the deadline, the researcher controlled non-response error by comparing data on
respondents.
The survey asked information regarding most independent variables and one
dependent variable data, operational margin. The survey asked whether a training needs
assessment was conducted or not, the extent of customization, nature of the contract with
training providers and participant’s perception on training effectiveness, and the changes
in operational margin.
Secondary data
maintains and updates the OITP database which contains generic information about OITP
participant companies and their training programs. The analysis of the OITP database
provided a sampling frame to draw a survey list and basic participant’s demographic
number and email address, company name, company's geographic location, company size,
industry sector with SIC code number, company ownership, the type of training, and
69
Operationalization of Variables
This section describes how to measure four constructs of this study: 1) the nature
business of the client organizations, formality of the contract, and follow-up contact.
Level of involvement
client organization’s training process. This variable was measured by the number of
stages of the training process in which the external training providers participated. The
tages of the training process are planning, developing, delivering, and evaluating the
training programs.
Contract history
This variable refers to how many previous experiences that the client organization
worked with the external training providers. It was measured by the number of the
previous contract that the client organization had with the external training providers.
knowledge about client organizations’ business. This variable was measured through
training managers’ perception and external training providers' previous experience with
70
Formality of the contract
This variable refers to whether client organizations have a formal contract with
their training providers in terms of training services provided. The variable of formality
was measured by written or verbal. Thus, it was assessed by four ordinal scales: 1) a
written contract at every time; 2) an initial written contract and later an oral contract; 3)
Follow-up contact
This variable refers to whether needs assessment for training was conducted
before implementing the actual training program. Training needs assessment was
assessed by the level of focus, timing of need to measure, and prioritizing of training
Level of focus
performance information in individual level, work process level, and/or function level.
Timing of need
performance level before training, and whether training needs assessment project
71
Prioritizing
prioritizes training issues, and if current training programs reflected the result of training
needs assessment.
Extent of customization
This variable refers to the extent of customized level of a training program. The
Customization was measured by the total score of 10 items whether the item was
lesson plan. They are training objectives, trainee’s prerequisites, instructional materials in
Relationship to job
This variable refers to how the training is closed related to main tasks of training
Employee level
This variable refers to the level of employee who participated in the training. It
executive.
Types of training
This variable refers to the types of training contents. It was measured by technical
72
Expected outcome
This variable refers to the intended goal of the training program, which is planned
ahead. This variable was assessed by selecting one of ten items, and the ten items was
developed based on the Kirkpatrick’s model. They are: 1) to have trainees gain
Training program effectiveness refers to the degree to which the training reaches
the intended objective(s) or immediately expected outcome. This construct was measured
effectively one specific training program reached the intended goal of the training
program. Perceived training effectiveness was measured by the level of area and the
relativity. The level of area refers to whether training is effective in increasing such areas
items. The relativity refers to how much the specific training program is more or less
effective than the training in general in client’s organizations. This was measured from
73
Financial Performance
reducing costs, and earnings. This construct was measured by the increases of operational
margin between at the training starting point of time and six months after completing the
training program. Operational margin refers to earnings per total revenue. Operational
margin and increase of operational margin were calculated based on the following
equations.
training
Instrument Development
Instruments for two surveys were developed by the researcher with assistance of
academic advisors and committee members at the Ohio State University This section
describes how we established instrument validity and reliability and designed the
instruments.
Instrument Validity
74
A panel of experts reviewed the questionnaires for content validity. The panel of
experts was composed of scholars, practitioners, and government officials. They are two
research method professors, one research method scholar, one HRD professor, one
Workforce Development and Education professor, four HRD practitioners who are in
industry, and two government officials in the economic development area. The experts
were asked to identify the clarity of the questionnaires. If the experts found any unclear
wording, they were asked to mark it and to propose more appropriate wording. The
experts were also asked to review the appropriateness of items, and, if there were any
inappropriate item, they were asked to mark it. They were also asked to provide any
Additionally, the research method experts were asked to review the format of the survey
Instrument Reliability
consistently cross samples what it should measure. For ensuring the reliability of the
instruments, the pilot study can be implemented. However, because this study is a census,
it was not possible to draw pilot samples, and the researcher did not pretest and posttest
shows the degree of internal consistency across items to measure one underlying
construct. It ranges from 0 to 1, and over .7 is accepted as a reliable level. Table 3.1
75
Variables Cronbach’s Alpha
General training effectiveness on employee development .878
General training effectiveness on organizational performance .910
Overall general training effectiveness .926
Specific training program effectiveness on employee
development .908
Specific training program effectiveness on organizational
performance .867
Overall specific training program effectiveness .945
Table 3.1: Cronbach’s Alpha Coefficients for the Survey Responses (n=44, Senior
Manager Survey)
This survey was composed of four parts. Part One asks general background
information about the training program. This information included training start date, end
date, estimation of the number of training participants, estimation of the total number of
employees when the training began, duration of the training programs, training
participants’ employee level, how closely the training is related to main job, who
developed the training program, who delivered it, and intended goal of the training
program. In questions on who developed and delivered the training program, respondents
could select multiple answers among in-house training staff, community college, 4 year
university, vocational center, adult learning center, private training vendor, and non-for-
Questions in Part Two asked about training needs assessment, customization, and
nature of the relationship with external training providers, if they hired outside training
respondents were asked to answer either YES or No on every activity in each area. In
76
questions regarding relationships with training providers, respondents were asked to
select the status of their contract, and answer questions regarding the level of external
providers’ knowledge in the business, and follow-up contact. Part three asked
respondents to provide a senior manager’s name and contact information for a matching
survey. Questions in the Part Four asked respondents’ demographic information such as
HRD manger, HR manager, Supervisor or other), and year of the current position.
The second survey was composed of two parts. Items in the Part One asked the
general on their organization. Those items consist of a six point Likert-type scale
(strongly disagree, disagree, slightly disagree, slightly agree, agree, and strongly agree)
and not applicable. The same 16 statements were given to respondents to answer their
perception on effectiveness of the specific training programs funded by the OITP. They
were also asked to provide operational income and total revenue information.
77
Research Procedures
Data Collection
The primary data was collected by mainly web-based survey to HRD manager or
managers. Four surveys were collected through faxed survey questionnaires, and a phone
survey was also attempted. The following section describes how these two surveys were
implemented.
The first survey was distributed to training coordinators, training managers, HRD
The email asked recipients to participate in the survey which is sponsored by the
Department of Development, and the cover letter from the Assistant Manager at the OTIP
in the Department letterhead with the State government’s logo was attached to the email.
To ensure that the rights and welfare of the participants’ activities in research was
protected, the survey questionnaires obtained an approval from the Ohio State University
To increase response rate, an initial call was made to all contact people who had a
phone number. Through this phone call, the researcher could correct potential recipients'
email address. The email recipients could participate in the survey by clicking the link
address to the survey questionnaire website. After the participants completed and
submitted the questionnaire, their responses were sent out to the researcher’s email
account. Five business days after the first survey email was sent, a second email was sent
to encourage survey participation and to remind those who did not yet reply to participate.
78
Several additional phone calls were made and emails were sent out to encourage survey
After responses from the first survey were returned, the second survey was
conducted based on the first survey information. The second survey was designed to
company due to the specific training program information, customized and individualized
web-survey questionnaires was sent out. The email asked recipients to participate in the
survey which is sponsored by the Department of Development, and the cover letter from
the Assistant Manager at the OTIP in the Department letterhead with the State
government’s logo was attached to the email. To ensure that the rights and welfare of the
approval from an Ohio State University of Institutional Review Board (See Appendix B
The email recipients could participate in the survey by clicking the link address to
the survey questionnaire website. After the participants completed and submitted the
questionnaire, their responses were sent out to the researcher’s email account. Five
business days after the first survey email was sent, a second email was sent to encourage
survey participation and to remind those who did not yet reply to participate. Several
follow up calls were made and emails were sent out to encourage senior managers to
79
Data Analysis
After the data was collected, it was automatically coded and could be easily
transferred to the spreadsheet database. The data from the secondary data sources was
merged in one database and analyzed by using the Statistical Package for the Social
and respondents was analyzed to determine descriptive statistics such as frequencies and
percentages. The demographic characteristics of companies were size, industry type, and
the number of years in business. The size of the company was presented based on the
were the program duration and the participant’s proportion of the whole employees. The
demographic characteristics of respondents are their job title, the number of years in the
current position, and the number of years in the company. Next, the proposed research
Research Question One: Does training program effectiveness differ based on types of
training providers? Does the training provided by community colleges and/or four-year
calculated. Next, one-way ANOVA was applied to examine group differences in the
The F value was calculated to see the difference. Third, t-test was performed to examine
80
only two group differences in perceived training effectiveness between higher
Research Question Two: Does the degree of training program effectiveness differ based
on the nature of the relationship among client organizations and external training
providers?
between perceived training effectiveness and the nature of the relationship. The model
regressed the degree of the perceived training effectiveness on a vector of five sub
training providers’ knowledge about the client organization’s business, and formality of
the contract. The squared multiple coefficient of correlation was presented to describe the
‘goodness of fit’ of the model by indicating the percentage of the variance in perceived
variables. Because the formality of the contract was measured by four ordinal scales, they
Research Question Three: Does the degree of training program effectiveness differ based
perceived training effectiveness and the quality of training needs assessment. The value
of a correlation coefficient represents the extent to which two variables are related to each
other, with results ranging from a perfect positive relationship (1.00) through no
81
Research Question Four: How do training programs vary in terms of the extent of
employees, and expected outcome? How do these differences in the nature of the training
First, descriptive statistics were generated to present how the nature of the
program varies regarding the five sub variables. Frequencies, percentages, mean, median,
range, and standard deviations were calculated. Next, standard multiple regression
effectiveness and the nature of the program. The model regressed the degree of the
perceived training effectiveness on a vector of five sub variables. The squared multiple
coefficient of correlation was presented to describe the ‘goodness of fit’ of the model by
the linear combination of five sub independent variables. Dummy coding was assigned to
employee level, types of training, and expected outcome since they are measured by
ordinal scales.
Research Question Five: Are perceived training program effectiveness and client
perceived training effectiveness and the increase of the operational margin at client’s
organization. The value of a correlation coefficient represents the extent to which two
variables are related to each other, with results ranging from a perfect positive
82
CHAPTER 4
RESULTS
This chapter presents the results of the study. The first section presents the
descriptive statistics of the study. The second section reports the results for each research
question.
Descriptive Statistics
The first part of this section contains a description of companies that are in the
The OITP database shows that 202 companies received training funds and
implemented training programs from January 2002 to June 2004. However, in attempting
in the sample frame. Seventeen out of 77 cases were omitted because the person in charge
of OITP is no longer employed and the position has not been reassigned. Eleven out of 77
cases were omitted due to incorrect contact information. Three cases reported that the
company was under different management. The remaining 46 cases could not be reached
83
due to incorrect email addresses or telephone numbers. Thus, the total number in the
Fifty seven out of 125 companies replied to the first survey, the manager survey. Forty
five companies out of fifty seven replied to the second survey, the senior manager survey.
As a result, 45 companies out of 125 completed both surveys, resulting in a response rate
of 36 percent.
(n=45) are located in 22 counties, non-respondents (n=80) are located in 42 counties, and
not-in-sample frame companies (n=77) are located in 38 counties. The ratio between
county verses company respondents is 1:2.05, the ratio of non-respondents is 1:1.90, and
the ratio of not-in-sample frame is 1:2.03. Table 4.1 presents frequencies and
percentages of type of industry of each group with the majority of companies found in
three groups in the manufacturing industry. Table 4.2 presents average company size in
terms of number of employees among the three groups. The average size of not-in-sample
frame group is 341.5 while the average of respondent group is 264.9 and the average of
No significant differences were found when comparing the three groups in terms
of the number of training participants and the average of base wage of training
84
slightly lower than non-respondents by 1.7 percent, and lower than non-in-sample frame
companies by 7.7 percent. Table 4.2 also shows the average base wage of training
participants. The average base hourly wage in respondents group is slightly higher than in
85
Non- Not-in-
Respondent Respondent Sample Frame
company company company
Table 4.1: Frequency and Percentage of Type of Industry among Respondent Companies,
Non-respondent Companies, and Not-in-Sample Frame Companies (OITP database)
86
n Mean SD
Company Size
Table 4.2: Company size, Number of Participants in Training, and Base Wage of
Training Participants among Respondent Companies, Non-respondent Companies, and
Not-in-Sample Frame Companies (OITP database)
87
Respondents
From this part, the researcher analyzed 45 completed cases from both manager
and senior manager surveys. This part presents the demographic information of the 45
respondents. The first two sets represent demographic information from two respondents
of each company. The other two sets represent demographic information from
years with the company, current title, years in current title, and sex. The frequencies and
percentages are presented in tables 4.3 and table 4.4. Results show that the average length
of service for the manager survey respondents’ is 10.91 year (SD=9.47), and the average
length of service for the senior manager survey respondents’ is 12.08 year (SD=9.15).
The manager survey respondents consist of an equal number of females and males, and
Current job titles for the manager survey respondents were reported as follows:
(17.8%). Only one respondent reported his/her title as HRD manager. Those who
Current job titles for the second group of respondents were reported as follows:
Only one respondent reported his/her title as HRD VP or manager. Those who reported
88
transportation, director finance & information services, IT manager, material coordinator,
Company size (number of employees) varies from 5 employees to over 1,700 employees.
The average company size is 389 employees (SD=399). Over 40 percent of companies
range from 100 and 299 employees. The number of years doing business in Ohio ranges
from 4 years to 120 years. The average length of time doing business in Ohio is 43.3
years (SD=33.7).
to SIC code. The remaining companies represent the service and trade industry (Table
4.3). Seven participant companies (15.6%) are either publicly owned or subsidies of
publicly owned companies, and the majority (84.4%) are either privately owned or
Ohio. Five companies (11.1%) are located in Cuyahoga county, and 4 companies (8.9%)
are located in Summit county. The majority of companies are located throughout the state
in 21 counties ranging from one, two, or three from each county. Figure 4.1 shows survey
Table 4.6 describes training program characteristics. Forty four percent of the
training programs (20 programs) include technical training and managerial training.
programs ranges from short to long with 33 percent being longer than 8 weeks, and 27
89
percent being shorter then 2 weeks. The length of training programs shape is bimodal.
SD=210). Less than 30 people participate in half of the training programs (53.3%). All
training programs (11.1%). Only 12 training programs (26.7%) have one level of
participant, and the remaining programs have more than one level of employee
participating.
90
n %
Number of Years at the Company
0 – 5 years 15 33.3
6 – 10 years 11 24.4
11 – 15 years 5 11.1
16 – 20 years 1 2.2
More than 20 years 10 22.2
Missing 3 6.7
Job Position
Training Coordinator or Manager 8 17.8
HRD Manager 1 2.2
HR Manager 12 26.7
Supervisor 1 2.2
Executive 14 31.1
Other Manager 2 2.2
Other 5 11.1
Missing 3 6.7
Number of Years at the Current Job
0 – 5 years 20 44.4
6 – 10 years 15 33.3
11 – 15 years 3 6.7
16 – 20 years 2 4.4
More than 20 years 2 4.4
Missing 3 6.7
Sex
Male 26 57.8
Female 19 42.2
Table 4.3: Demographic Information for Manager Survey Respondents (Manager survey,
n=45)
91
n %
Number of Years at the Company
0 – 5 years 13 28.9
6 – 10 years 10 22.2
11 – 15 years 6 13.3
16 – 20 years 8 17.8
More than 20 years 6 13.3
Missing 2 4.4
Job Position
Managing Director 6 13.3
Operational VP 8 17.8
HRD VP or Manager 1 2.2
HR VP or Manager 9 20.0
Quality Manager 1 2.2
Safety Manager 1 2.2
Plant Manager 4 8.9
President/Partner/Owner 3 6.7
Production/Operational Manager 1 2.2
Other 9 20.0
Missing 2 4.4
Number of Years at the Current Job
0 – 5 years 23 51.1
6 – 10 years 13 28.9
11 – 15 years 3 6.7
16 – 20 years 2 4.4
more than 20 years 2 4.4
Missing 2 4.4
Sex
Male 32 71.1
Female 13 28.9
Table 4.4: Demographic Information for Senior Manager Survey Respondents (Senior
manager survey, n=45)
92
n %
The Size (Number of Employees)
1–9 3 6.7
10 – 49 8 17.8
50 – 99 6 13.3
100 – 299 19 42.2
300 – 999 6 13.3
Over 1,000 3 6.7
Number of Years at the Business in Ohio
0 – 10 years 9 20.0
11 – 20 years 7 15.6
21 – 30 years 1 2.2
31 – 40 years 5 11.1
41 – 50 years 2 4.4
Over 50 years 17 37.8
Missing 4 8.9
Industry Type*
Manufacturing 33 73.3
Transportation & public utilities 3 6.7
Wholesale trade 2 4.4
Retail trade 3 6.7
Finance, insurance & real estate 1 2.2
Services 1 2.2
Missing 4 8.9
Ownership
Privately owned 38 84.4
Publicly owned 7 15.6
* Two companies have two industry types so the total sample is 47.
93
n %
Types of Training
Technical 20 44.4
Managerial 10 22.2
Awareness 9 20.0
Missing 6 13.3
Duration
0 – 2 weeks 12 26.7
3 – 4 weeks 3 6.7
5 – 8 weeks 5 11.1
More than 8 weeks 15 33.3
Missing 10 22.2
Number of Participants
1 – 9 employees 7 15.6
10 – 19 employees 8 17.8
20 – 29 employees 4 8.9
30 – 39 employees 6 13.3
40 – 49 employees 4 8.9
50 – 99 employees 8 17.8
100 – 999 employees 7 15.6
Over 1,000 employees 1 2.2
Trainees’ Employment Level in Company*
Frontline employee 40 88.9
Supervisor 30 66.7
Manager 24 53.3
Executive 10 22.2
Number of Employment Level Participated in
Training
1 Level 12 26.7
2 Levels 12 26.7
3 Levels 16 35.6
4 Levels 5 11.1
* Companies can answer more than one category as their trainees’ level in companies.
Table 4.6: Demographic Information of the Training Programs (Manager survey, n=45)
94
Figure 4.1. Number of Survey Participant Companies in each County in Ohio (OITP
database, n=45)
95
Results for Each Research Question
Research Question One: Does training program effectiveness differ based on types of
training providers? Does the training provided by community colleges and/or four-year
Training programs were developed and delivered by various entities. Table 4.7
(66.7%), and delivered by private companies (55.6%), however, some were developed
and delivered by cooperation with in-house training staff within client companies.
development and delivery is relatively small (11.1%). Private training providers include
96
Types of Training Provider n %
Table 4.7: Frequencies and Percentages of Type of Training Providers (Manager survey,
n=45)
97
One-way analysis of variance was applied to examine group differences among
the degree of training effectiveness by different types of training providers. Table 4.8
presents group differences in the degree of perception on the specific training program
effectiveness. Table 4.8 shows the value of F (2, 41) is .037 and is not significant (p>.10),
which implies that there is no significant difference among training developer. Another
value of F (2, 41) is .901 and is not significant (p>.10), which implies that there is no
Total 17.261 43
Total 17.261 43
effectiveness of the training program. Table 4.9 shows the value of F (2, 41) is .205 and
is not significant (p>.10), which implies that there is no significant difference among
98
training developer. Another value of F (2, 41) is .360 and is not significant (p>.10),
Total 7.225 43
Total 7.225 43
As can be seen Table 4.8, the value of F (2, 11) is 1.941 and is not significant (p>.10),
which implies that there is no significant difference among training developer. Another
value of F (2, 11) is 2.391 and is not significant (p>.10), which implies that there is no
99
Source Sum of df Mean F Sig
Squares Square
Training developer 11.153 2 5.577 1.941 .190
Total 42.763 13
Total 42.763 13
training program effectiveness between in-house training staff involvement and no in-
house training staff involvement. The result of t-tests in Table 4.11 shows there is no
significant difference between the two groups. The t value in terms of perception on this
specific training program effectiveness is .239 and is not significant (p>.10). The t value
significant (p>.10). The t value in terms of increase in operational margin is .475 and is
100
Training t df Sig. Mean SE
Effectiveness Difference
Perception of this
specific training .239 42 .812 .047 .196
program (n=43)
Perception of
relative .889 25.065 .382 .123 .138
effectiveness
(n=43)
Increase in
operational .475 12 .643 .480 1.010
margin (n=13)
Table 4.11: Independent Samples t-test on In-house Training Staff and Training Program
Effectiveness (Manager survey and Senior manager survey)
program effectiveness is -.245 and is not significant (p>.10). The t value in terms of
(p>.10). The t value in terms of increase in operational margin is -1.160 and is not
significant (p>.10).
101
Training t Df Sig. Mean SE
Effectiveness Difference
Perception of this
training program -.245 42 .808 -.075 .304
(n=43)
Perception of
relative .497 42 .622 .098 .196
effectiveness
(n=43)
Increase in
operational -1.160 12 .269 -2.155 1.858
margin (n=13)
Table 4.12: Independent Samples t-test on Educational Institution Providers and Training
Program Effectiveness (Manager survey and Senior manager survey)
Table 4.13 presents the results of independent samples t-test regarding private
training provider involvement. The t value in terms of perception on this specific training
program effectiveness is .052 and is not significant (p>.10). The t value in terms of
(p>.10). However, the t value in terms of increase in operational margin is 2.888 and is
significant (p>.05). This implies that there is a statistical difference in the increase in
Thus, the results show that client organization’s perceived training effectiveness
does not differ based on the type of training providers and that training provided by
community colleges and/or four-year universities does not result in a different degree of
providers. However, the results show that there is a statistical difference in the increase in
102
operational margin between the training programs where private training providers were
involved versus the programs that did not involved private training providers.
Table 4.13: Independent Samples t-test on Private Training Providers and Training
Program Effectiveness (Manager survey and Senior manager survey)
Research Question Two: Does the degree of training program effectiveness differ based
on the nature of the relationship among client organizations and external training
providers?
Standard multiple regression analysis was performed on both manager and senior
manager surveys to examine the relationship between training program effectiveness and
the nature of the relationship with external training providers. The model regressed the
external training provider’s level of involvement in training process, contract history with
this particular training provider, whether the providers initiated a follow-up contact, the
103
external training provider’s knowledge and experience in client organization’s business,
goodness of fit of the model by indicating the percentage of the variance in perceived
Because the formality of the contract is measured by four ordinal scales, they were coded
Table 4.14, table 4.16, and table 4.18 present the results of the correlation
statistics among the variables. Table 4.15, table 4.17, and table 4.19 present the results of
standard multiple regression analysis. For perception of this specific training program
a significant relationship with this perception at an alpha level of .05 (Table 4.15). The
this perception at an alpha level of .05 (Table 4.17). The squared multiple coefficient of
correlation is 0.
with increase in operational margin (Beta = .376, p<.05). This implies that operational
margin of program where the external training providers are involved in additional stages
104
of the training process increases more than program where the external training providers
are not involved in additional stages. External training provider’s knowledge and
operational margin (Beta =.655, p<.05). This implies that programs having external
training providers with more knowledge and experience in the client organization’s
business have greater increases in operational margin when compared with programs
where the external training provider has less knowledge and experience.
whether the contract was written every time (Beta = -.685, p<.05). It implies that the
program operational margin increases when clients write down every contract in
comparison to those that do not. There is also negative relationship between increase in
operational margin and whether the contract was first written and later verbal (Beta = -
.398, p<.01). The squared multiple coefficient of correlation is .889, which implies that
the group of independent variables explains 88.9 percent of the total variance of increased
operational margin.
Thus, the results show that the degree of training program effectiveness differs
based on the nature of the relationship among client organizations and external training
providers. The increase in operational margin differs based on the external training
knowledge and experience in the client organization’s business, and the formality of the
contract
105
1 2 3 4 5 6 7 8 Mean SD
1. Provider’s level of 1.000 .531** .834** .410* -.476** .004 -.164 -.301* 1.98 1.602
involvement
2. Contract history with providers 1.000 .639** -.056 -.227 .194 -.193 -.105 2.11 2.622
3. Provider’s knowledge & 1.000 .252 -.728** .136 .050 -.314* 2.53 1.727
experience
4. Provider’s follow-up 1.000 -.415* -.135 .224 -.102 .84 .374
Table 4.14: Summary Data: Regression of Perception of the Specific Training Program Effectiveness on Selected Variables in
the Relationship with External Training Providers (Manager survey and Senior manager survey)
Variables R2 R2 change b Beta
Provider’s Level of Involvement .017 .017 -.113 -.187
Contract History with Providers .036 .019 .080 .315
Provider’s Knowledge & Experience .107 .071 -.664 -.676
Provider’s Follow-up .108 .000 -.035 -.020
Contract .221 .113
Contract I – neither written nor verbal -1.839 -.513
Contract II– first written, later verbal .157 .107
Contract III– written contract in every time -.100 -.062
107
(Constant) 7.253
Table 4.15: Standard Multiple Regression of Perception of the Specific Training Program Effectiveness on Selected Variables in
the Relationship with External Training Providers (n = 28) (Manager survey and Senior manager survey)
1 2 3 4 5 6 7 8 Mean SD
1. Provider’s level of 1.000 .531** .834** .410* -.476** .004 -.164 .032 1.98 1.602
involvement
2. Contract history with providers 1.000 .639** -.056 -.227 .194 -.193 -.105 2.11 2.622
3. Provider’s knowledge & 1.000 .252 -.728** .136 .050 -.029 2.53 1.727
experience
4. Provider’s follow-up 1.000 -.415* -.135 .224 .059 .84 .374
Table 4.16: Summary Data: Regression of Perception of the Relative Training Program Effectiveness on Selected Variables in
the Relationship with External Training Providers (Manager survey and Senior manager survey)
Variables R2 R2 change b Beta
Provider’s Level of Involvement .016 .016 .013 .032
Contract History with Providers .026 .010 -.021 -.122
Provider’s Knowledge & Experience .032 .005 -.163 -.249
Provider’s Follow-up .032 .000 .093 .081
Contract .202 .171
Contract I – neither written nor verbal -.591 -.247
Contract II– first written, later verbal .296 .303
Contract III– written contract in every time -.205 -.191
109
(Constant) .441
Table 4.17: Standard Multiple Regression of Perception of the Relative Training Program Effectiveness on Selected Variables in
the Relationship with External Training Providers (n = 28) (Manager survey and Senior manager survey)
1 2 3 4 5 6 7 8 Mean SD
1. Provider’s level of 1.000 .531** .834** .410* -.476** .004 -.164 -.018 1.98 1.602
involvement
2. Contract history with providers 1.000 .639** -.056 -.227 .194 -.193 -.255 2.11 2.622
3. Provider’s knowledge & 1.000 .252 -.728** .136 .050 -.329 2.53 1.727
experience
d
4. Provider’s follow-up 1.000 -.415* -.135 .224 .84 .374
Table 4.18: Summary Data: Regression of Increase in Operational Margin on Selected Variables in the Relationship with
External Training Providers (Manager survey and Senior manager survey)
Variables R2 R2 change b Beta
Provider’s Level of Involvement .459 .459 .696 .376*
Contract History with Providers .513 .054 -.176 -.201
Provider’s Knowledge & Experience .622 .108 4.114 .655*
Contract .951 .329
Contract II– first written, later verbal -1.877 -.398*
Contract III– written contract in every time -3.226 -.685**
(Constant) -17.664
111
Table 4.19: Standard Multiple Regression of Increase in Operational Margin on Selected Variables in the Relationship with
External Training Providers (n = 10) (Manager survey and Senior manager survey)
Research Question Three: Does the degree of training program effectiveness differ based
no training needs assessment. The result of the t-test in Table 4.20 shows there is no
significant difference between the two groups. The t value in terms of perception on this
specific training program effectiveness is -.769 and is not significant (p>.10). The t value
in terms of perception on relative effectiveness of the training program is -.751 and is not
significant (p>.10). The t value in terms of increasing operational margin is 1.203 and is
not significant (p>.10). This implies that there is no statistical difference between the two
effectiveness.
Table 4.20: Independent Samples t-test on Training Needs Assessment and Training
Program Effectiveness (Manager survey and Senior manager survey)
112
Correlational statistics were used to determine the relationship between
perceived training effectiveness and the quality of the training needs assessment. The
value of a correlation coefficient represents the extent to which two variables are related
to each other, with results ranging from a perfect positive relationship (1.00) through no
Table 4.21 presents the results of the correlation statistics. The correlation
coefficient between the perception of the specific training program effectiveness and the
quality of training needs assessment is .175, and is not statistically significant (p<.10).
The correlation coefficient between the perception of the relative training effectiveness
and the quality of training needs assessment is -.036, and is not statistically significant
(p<.10). The correlation coefficient between the increase in operational margin and the
quality of training needs assessment is -.287 and is not significant (p<.10). These imply
that there are no significant relationships between training program effectiveness and the
Thus, the results show that training program effectiveness does not differ based
113
1 2 3 4
*p<.01
Table 4.21: Correlation Matrix between the Quality of Training Needs Assessment and
Training Program Effectiveness (Manager survey and Senior manager survey)
Research Question Four: How do training programs vary in terms of the extent of
employees, and expected outcome? How do these differences in the nature of the training
First, descriptive statistics are presented in table 4.6, table 4.22, and table 4.23 to
show how the training program varies regarding the five sub variables. As table 4.6
shows, the type of training varies from technical training (44.4%), to managerial training
(22.2%) and awareness training (20%). The proportion of training participants among
total employees also varies. Table 4.23 shows that 11 training programs (24.3%) have
less than 10 percent of participants versus total employees, and that all employees
114
Table 4.22 shows that most training programs are job-specific (n=32, 71.1%)
rather than job-related (n=13, 28.9%). Almost half (44.4%) of the training program’s
intended goal is related to “to increase knowledge and skills”, and the other half (42.2%)
training program goal. Other specified training goals are “ISO certificate” and “to
In terms of customization, only one training program has never customized any
part of the training program, and the other 44 training programs customized at least more
than one item of the training program with client companies materials or to meet
participant’s needs. The average number of customized items per training program is 5.67
(SD=2.67) among 10 items. Table 4.23 presents distribution of the number of customized
items. Among customized items, instructional materials are most frequently customized
(n=39, 86.7%), and training objectives or trainees’ prerequisites are customized next
frequently (n=36, 80%). Table 4.22 shows that 35 training programs (77.8%) were
115
N %
Other 2 4.4
Relationship to Job
Job-specific 32 71.1
Job-related 13 28.9
Extent of Customization
No customization 1 2.2
Customized 9 20.0
116
N %
Participants’ Portion among Total Employees
0 – 9.99 % 11 24.3
10 – 19.99 % 4 8.9
20 – 29.99 % 2 4.4
30 – 39.99 % 5 11.1
40 – 49.99 % 1 2.2
50 – 59.99 % 5 11.1
60 – 69.99 % 0 0.0
70 – 79.99 % 1 2.2
80 – 89.99 % 2 4.4
90 – 100.00 % 14 31.1
Number of Items to be Customized
0 – 3 items 8 17.8
4 – 6 items 18 40.0
7 – 10 items 19 42.2
Customized Training Item
Objectives or Trainees’ Prerequisites 36 80.0
Instructional Materials 39 86.7
Timetable 23 51.1
Delivery method 24 53.3
Evaluation or Test Forms 25 55.6
Table 4.23: Distribution on Training Participants’ Portion among Total Employees and
Level of Customization (Manager survey, n=45)
117
Next, standard multiple regression analysis was employed to examine the
relationship between training program effectiveness and the nature of the training
program on manager and senior manager surveys. The model regressed the degree of the
training program effectiveness on a vector of five sub variables. The five variables
training and expected outcome were dummy coded since they are measured by ordinal
scales. Relationship to job was coded as a dichotomy variable since a training program is
Table 4.24, table 4.26, and table 4.28 present the results of the correlation
statistics among the variables. Table 4.25, table 4.27, and table 4.29 present the results of
standard multiple regression analysis. For perception of this specific training program
significant relationship with this perception at the alpha level of .05 (Table 4.25). First,
the beta of portion of trainees among employees is -.344 (p<.05), which means that if the
program effectiveness can be predicted to decrease. Second, the beta of intended goal to
increase learning is -1.230 (p<.05), which means that if the intended goal is to increase
decrease. The beta of intended goal to change behavior is -1.178 (p<.01), which also
means that if the intended goal is to change behavior, perception of this specific training
118
program effectiveness can be predicted to decrease. The beta of intended goal to increase
The beta of entirely development of the program is .469 (p<.01), which implies
that if the training program is entirely developed, perception of this specific training
customization is -.393 (p<.05), which means that if the number of customized training
decrease. The squared multiple coefficient of correlation is .376, which means that the
group of independent variables explains 37.6 percent of the total variance of the
perception of relative training program effectiveness (Beta = -.414, p<.05). The beta of
relationship to job is .369 (p<.10), which implies that if the training contents are job-
The beta of entirely development of the program is also positive and very
strongly significant. It is .486 (p<.01), which means that if the training program is
increase. The squared multiple coefficient of correlation is .264, which means that the
119
group of independent variables explains 26.4 percent of the total variance of perception
negative relationship (Table 4.29). The beta of entirely developed program is -3.504
(p<.10), which means the operational margin of the program which is entirely developed
increases less than the program which is not entirely developed. The beta of technical
type of training is -2.237 (p<.10). The squared multiple coefficient of correlation is .367.
Thus, the results show that training program effectiveness differs based on the
120
1 2 3 4 5 6 7 8 9 10 Mean SD
1. Proportion of trainees 1.000 -.242 -.165 .044 .078 .076 .066 -.366* .083 -.317* .50 .396
among employees
2. Job specifica 1.000 .274 -.145 -.150 -.105 -.006 .573** -.372* .291 .71 .458
3. Intended goal Ib 1.000 -.279 -.765* .048 .181 .243 .019 .072 .44 .503
c
4. Intended goal II 1.000 -.267 .167 -.109 -.009 .189 -.346* .09 .288
5. Intended goal IIId 1.000 -.084 -.148 -.127 -.190 .045 .42 .499
6. Entirely developede 1.000 .318* .133 .076 .092 .78 .420
7. Level of 1.000 .127 -.124 -.199 5.67 2.663
customization
8. Type of training If 1.000 -.602** .133 .51 .506
9. Type of training IIg 1.000 -.142 .26 .442
10. Perception of the 1.000 4.85 .634
121
specific training
program effectiveness
a
0 = Job-related training program; 1 = Job-specific training program
b
0 = Intended goal is not to increase learning; 1 = Intended goal is to increase learning
c
0 = Intended goal is not to change behavior; 1 = Intended goal is to change behavior
d
0 = Intended goal is not to increase organizational performance; 1 = Intended goal is to increase organizational performance
e
0 = A training program was not entirely developed; 1 = A training program was entirely developed
f
0 = Type of training is not managerial; 1 = Type of training is managerial
g
0 = Type of training is not technical; 1 = Type of training is technical
Table 4.24: Summary Data: Regression of Perception of the Specific Training Program Effectiveness on Selected Variables in
Training Program Characteristics (Manager survey and Senior manager survey)
Variables R2 R2 change b Beta
Proportion of trainees among employees .124 .124 -.592 -.344*
Job specific .162 .038 .209 .143
Intended goal .332 .170
Intended goal I -1.665 -1.230*
Intended goal II -2.541 -1.178**
Intended goal III -1.649 -1.244*
Entirely developed .408 .076 .704 .469**
Level of customization .525 .117 -.095 -.393*
122
Table 4.25: Standard Multiple Regression of Perception of the Specific Training Program Effectiveness on Selected Variables in
Training Program Characteristics (n = 37) (Manager survey and Senior manager survey)
1 2 3 4 5 6 7 8 9 10 Mean SD
1. Proportion of trainees 1.000 -.242 -.165 .044 .078 .076 .066 -.366* .083 -.411** .50 .396
among employees
2. Job specifica 1.000 .274 -.145 -.150 -.105 -.006 .573** -.372* .300* .71 .458
3. Intended goal Ib 1.000 -.279 -.765* .048 .181 .243 .019 -.014 .44 .503
4. Intended goal IIc 1.000 -.267 .167 -.109 -.009 .189 .060 .09 .288
5. Intended goal IIId 1.000 -.084 -.148 -.127 -.190 -.028 .42 .499
6. Entirely developede 1.000 .318* .133 .076 .293 .78 .420
7. Level of 1.000 .127 -.124 -.093 5.67 2.663
customization
8. Type of training If 1.000 -.602** .272 .51 .506
9. Type of training IIg 1.000 -.193 .26 .442
10. Perception of the 1.000 -.06 .410
123
specific training
program effectiveness
a
0 = Job-related training program; 1 = Job-specific training program
b
0 = Intended goal is not to increase learning; 1 = Intended goal is to increase learning
c
0 = Intended goal is not to change behavior; 1 = Intended goal is to change behavior
d
0 = Intended goal is not to increase organizational performance; 1 = Intended goal is to increase organizational performance
e
0 = A training program was not entirely developed; 1 = A training program was entirely developed
f
0 = Type of training is not managerial; 1 = Type of training is managerial
g
0 = Type of training is not technical; 1 = Type of training is technical
Table 4.26: Summary Data: Regression of Perception of the Relative Training Program Effectiveness on Selected Variables in
Training Program Characteristics (Manager survey and Senior manager survey)
Variables R2 R2 change b Beta
Proportion of trainees among employees .159 .159 -.453 -.414**
Job specific .230 .071 .342 .369*
Intended goal .259 .029
Intended goal I -.222 -.258
Intended goal II -.087 -.063
Intended goal III -.227 -.270
Entirely developed .372 .113 .465 .486***
Level of customization .402 .030 -.033 -.213
124
Table 4.27: Standard Multiple Regression of Perception of the Relative Training Program Effectiveness on Selected Variables in
Training Program Characteristics (n = 37) (Manager survey and Senior manager survey)
1 2 3 4 5 6 7 8 9 10 Mean SD
1. Proportion of trainees 1.000 -.242 -.165 .044 .078 .076 .066 -.366* .083 -.282 .50 .396
among employees
2. Job specifica 1.000 .274 -.145 -.150 -.105 -.006 .573** -.372* .194 .71 .458
3. Intended goal Ib 1.000 -.279 -.765* .048 .181 .243 .019 .238 .44 .503
c
4. Intended goal II 1.000 -.267 .167 -.109 -.009 .189 -.227 .09 .288
5. Intended goal IIId 1.000 -.084 -.148 -.127 -.190 -.112 .42 .499
6. Entirely developede 1.000 .318* .133 .076 -.324 .78 .420
7. Level of 1.000 .127 -.124 -.075 5.67 2.663
customization
8. Type of training If 1.000 -.602** -.068 .51 .506
9. Type of training IIg 1.000 -.194 .26 .442
10. Perception of the 1.000 -.655 1.814
125
specific training
program effectiveness
a
0 = Job-related training program; 1 = Job-specific training program
b
0 = Intended goal is not to increase learning; 1 = Intended goal is to increase learning
c
0 = Intended goal is not to change behavior; 1 = Intended goal is to change behavior
d
0 = Intended goal is not to increase organizational performance; 1 = Intended goal is to increase organizational performance
e
0 = A training program was not entirely developed; 1 = A training program was entirely developed
f
0 = Type of training is not managerial; 1 = Type of training is managerial
g
0 = Type of training is not technical; 1 = Type of training is technical
Table 4.28: Summary Data: Regression of Increase in Operational Margin on Selected Variables in Training Program
Characteristics (Manager survey and Senior manager survey)
Variables R2 R2 change b Beta
Proportion of trainees among employees .024 .024 -6.307 -1.376
Job specific .048 .024 -13.283 -3.591
Intended goal .120 .072
Intended goal I -9.841 -2.380
Intended goal II -.932 -.145
Entirely developed .255 .135 -14.493 -3.504*
Level of customization .333 .078 1.384 2.072
Type of training .873 .540
126
Table 4.29: Standard Multiple Regression of Increase in Operational Margin on Selected Variables in Training Program
Characteristics (n = 10) (Manager survey and Senior manager survey)
Research Question Five: Are perceived training program effectiveness and client
perceived training program effectiveness and the increase in operational margin for the
client’s organization. Table 4.30 presents the results of the correlation statistics. The
statistically significant (p<.05). It means that perception of this specific training program
training effectiveness and the increase in operational margin is -.283, and is not
significant (p<.10). The correlation coefficient between the perception of the relative
training effectiveness and the increase in operational margin is -.074, and is not
significant (p<.10). These imply that there are no significant relationships between
Thus, the results show that there is no relationship between perceived training
127
1 2 3
**p<.01
128
CHAPTER 5
This chapter is composed of three sections. The first section summarizes the
results of the data analysis and describes findings of five research questions. The second
section explores to further discussion based on the findings. The last section proposes a
Summary of Findings
The purpose of this study was to describe the relationship between training
from external training providers. The study was limited to 202 organizations that received
training funds from the OITP and implemented training program from January 2002 to
June 2004. Because 77 organizations were reclassified not in sample, the total number of
organizations in the sample frame was 125. The number of organizations that completed
both manager and senior manager surveys was 45, and thus, the response rate is 36
percent. The results showed that the three groups, responses, non-responses, and not-in-
the sample frame groups, are not different in terms of organization’s size, type of industry,
129
geographic locations, number of training participants, and average of base hourly wage of
trainees.
owned companies. Most companies are manufacturing but vary having less than 10
Research Question One: Does training program effectiveness differ based on types of
training providers? Does the training provided by community colleges and/or four-year
The study identified various entities involved in training process among client
organizations receiving funds from the OITP. Less than a half of business organizations
assistance. More than a half of business organizations received training related services
from external training entities. The majority of external training providers are private
training vendors and private equipment manufacturers. It is found that there are a few
The results of the study showed that training program effectiveness does not differ
based on the type of training provider. Client organizations do not perceive that who
found that client organizations do not perceive that who delivered training programs
made training program effectiveness varied. Who developed training programs and who
delivered training programs were found to not impact the increase in operational margin.
130
It was found that the training provided by community colleges or four-year
universities does not result in a higher degree of perceived training program effectiveness
in comparison to other external and internal training providers. It shows that client
organizations perceived training program effectiveness does not vary based on different
training providers.
However, the study found that client organizations who received training services
from private training providers experienced a greater increase in operational margin than
client organizations who did not received training services from private training providers.
It cannot be concluded that private training provider’s involvement increases the increase
of operational margin of client organizations, but it can be concluded that the operational
margin of companies who received training services from private training providers
increases more than companies who do not received training services from private
training providers.
Research Question Two: Does the degree of training program effectiveness differ based
on the nature of the relationship among client organizations and external training
providers?
The results showed that the degree of training program effectiveness relates to
the fact that clients organizations agreed on training programs and formally contracted
with external training providers. Although the external training provider’s level of
involvement in the training program was not shown to relate to perceived training
effectiveness, it showed that the more external training providers were involved in the
training process, the more operational margin increase. In addition, the results showed
131
that the more external training providers have business knowledge about client
Research Question Three: Does the degree of training program effectiveness differ based
The results of the study showed that training needs assessment has no
addition, it is found that the quality of training needs assessment does not relate to
Research Question Four: How do training programs vary in terms of the extent of
employees, and expected outcome? How do these differences in the nature of the training
The results showed that training programs vary in terms of the extent of
employees, expected outcome. Almost half of training is technical training, almost one
quarter of training is managerial training, and the other quarter is awareness training. The
proportion of training participants verses total employees also varies from less than 10
percent to 100 percent. Most training programs are job-specific. Almost half of the
training program’s expected outcomes are related to employee development, and another
132
In terms of customization, all training programs have customized at least one part
of the training program except one program. The majority of training programs were
The results showed that the fewer employees participated in training programs,
senior managers perceived the training program was more effective. It also showed that
when the training program is specific to jobs, senior managers perceived the training
program was more effective than when the training is simply related to jobs. It is found
that when the training program was entirely developed, senior managers perceived that
the training program was more effective than when the program was not newly developed.
The training program that was entirely developed also predicts more increase of
operational margin than the program that was not newly developed.
Research Question Five: Are perceived training program effectiveness and client
The result of the study showed that perceived training program effectiveness
Discussion
As described in the previous section, some of the study results presented different
results from the proposed conceptual framework, while some of the results partially
133
supported the conceptual framework. This section discusses some possible interpretation
The result showed that senior managers in client organizations perceive training
interpreted in a couple of ways. First, senior managers in client organizations may not be
aware of financial performance of their organizations or they may not evaluate training
performance. Since most organizations are manufacturing in the study sample, and since
many manufacturing companies fail these days, organization’s existence might mean a
expected to grow business, they might hire more employees to supply market demands. It
Another possible explanation is that senior managers may be less obligated from a
financial perspective from OITP funded training since this training is publicly funded.
134
External Training Providers and Training Effectiveness
training might be able to be more encouraged due to the synergy of public funds. The
OITP fund is public fund, so the organizations spent public funds for their training. At the
same time, many community colleges, four-year universities, and vocational training
centers also run based on public support. Therefore, research (GAO, 2004) on state-
funds utility. However, the result of this study showed that only a couple of organizations
engaged with educational institutions for their training. Therefore, it is difficult to test
institutions.
institutions, partnership training with private training providers produced different results.
client organizations, though the number of sample case is small. Because it is very hard
to conclude that the increase of operational margin can be a reliable measurement for
135
cannot be concluded that partnership training with private training providers directly
training with educational institutions. Rather, from this result, it can be interpreted that
Another interpretation is that private training providers may focus more on client’s
training relationship, like training provider’s level of involvement in training process and
studies, it is not surprising that training provider’s level of involvement and knowledge
contracting was negatively correlated with training effectiveness. This result may be
explained by the small number of case engaging with external training providers in the
sample.
The result of this study showed that training needs assessment did not relate to
training effectiveness of the companies that received training funds from the OITP,
contrary to the results of previous research on needs assessment. Since the study
represents one setting, it’s difficult to generalize the results to other contexts. There are
136
two possible interpretations to explain this fact. First, whether to implement training
needs assessment or not might not impact on training effectiveness. However, the quality
of training needs assessment might impact on training effectiveness, and the study
instrument might have failed to measure this quality in an extensive level enough to
about the training programs for the company might be already aware of training needs for
the company. Therefore, the training needs assessment did not influence on decision
making about training, and thus, training needs assessment did not impact on training
effectiveness. Because, in this case, training needs are assessed and recognized
informally, formal training needs assessment may not be more effective than informal
might not performed with a good quality. It might assess training needs perfunctorily.
Although the number of items customized was identified not to relate to training
effectiveness, the result of the study showed that the training programs that were entirely
most training programs customized at least some instructional materials such as company
data, format, materials, and equipment, the extent of customization may not be related to
training effectiveness.
In terms of training duration, the distribution was bi-modal with the average of
13.48 weeks. Since the results of study do not provide any information regarding
intensity of training program—how many hours per week—, the amount of training
137
duration has limits to be tested for training effectiveness. Another limit is that the
absolute number of case is too small in certain variables to apply statistical analysis. This
study also cannot eliminate other training program’s impact besides the OITP funded one.
Implications
The first part of this section presents implications for future research including a
revised conceptual framework. The second part provides several implications for practice
using senior managers’ perception about training effectiveness and financial measures. It
focused on the increase in operational margin that reflects operational efficiency within
program, they should have had correlations. However, since they did not have any
expect the increase of organizational performance through training. Thus, the research on
training should emphasize identifying current managers’ knowledge and attitude toward
138
From this study, it was found the need to develop varied measurements to assess
training effectiveness based on type of industry and/or type of training. From this respect,
the study on training evaluation should be conducted further toward contributing to assess
This study also identified needs to further study in partnership training. First, it
cases where partnership trainings with educational institutions were used. Second, it
needs to study practice of partnership training with private training providers. This study
study further about why private training providers are preferred. It is also important to
training process and the provider’s knowledge about client organization’s business.
However, the study failed to clearly identify how the contractual relationship between
139
identify other contractual components. It might be more appropriate if qualitative analysis
is emerged to investigate the relationship. In addition, this study attempted to examine the
study the quality of interaction in addition to the quantity of interaction, to more clearly
understand the impact of the relationship between external training providers and training
effectiveness.
In terms of training needs assessment, this study showed that the fact to
implement training needs assessment does not relate to training effectiveness. Thus,
more meaningful to study the quality of training needs assessment. Qualitative research
methods can be used to identify determining factors of quality. Since the quality of
current training needs assessment is suspicious poor, it is important to study the quality of
needs assessment. In addition, the study on training needs assessment should investigate
the process to reflect the results of training needs assessment on training decision and
whether the results of training needs assessment change or impact on training decision in
reality.
The revised conceptual framework will guide future research. Based on the
findings from the data analysis, discussion, and literature review, the proposed conceptual
140
Types of Training Provider
• Internal staff
• Educational institution
• Private training vendor
Nature of the Provider-Client
Relationship
• Level of involvement
• Knowledge of the business
• Formality
Assessment
• Formality
• Quality
One of the contributions of this study to the business is to call attention to the
relationship between training and business performance. The results of this study implied
that managers in charge of training might not be aware of the impact of training on
organization’s financial performance or might not know the way to evaluate training
ineffective training. It is very much critical to develop, deliver, and evaluate training
fundamental intention for training. By doing this, appropriate training programs will be
able to obtain the rational of training, not as cost but as investment. Organizations also
returns on investment.
This study also draws attention of business to training needs assessment. The
results of this study showed that it is very much possible for training needs assessment to
be poorly performed in the real world. Thus, organizations should realize that the
possibility of poor quality of results in training needs assessment. They should carefully
conduct training needs assessment or control the quality of training needs assessment
This study is one of the few studies to investigate state-funded, employer based
training, and the first attempt to examine the impact of the OITP funds on businesses in
Ohio. This study presents implication on assessing state-funded, employer based training
142
effectiveness, but also on assisting businesses have more effective training practice. The
results of this study provide several considerations that the state government should think
The state government might need to carefully examine the current OITP process
to utilize the funds more effectively. The study showed that the OITP funds were mainly
granted to private owned manufacturing companies with less than 300 employees. Is this
group of companies what the state government originally targeted? The state government
first can develop state economic development strategies and then can review the
characteristics of the OITP fund recipient companies to see whether her original intention
was met or not. Based on the results of this study, the state government is able to target a
In addition, the study showed that there is very weak linkage between state-
supported educational institutions and state-funded programs. The results of this study
raise the attention of policy makers to develop policy toward creating synergy among
local economy needs, capability of workforce, and higher educational functions. Thus,
This study also showed that the OITP recipient organizations do not evaluate
training effectiveness with financial measures. Based on OITP documents, the OITP do
not evaluate the impact of OITP from a financial performance perspective but did gain
data on the projected increase in the number of employees after training. It may be
concluded that the impact of OITP and OITP evaluation outcome are not monitored. The
OITP needs to establish evaluation criteria to monitor OITP impact to ensure training
143
Lesson for Higher Education Practice
higher education on their practice of partnership training. Community colleges and four-
year universities might have developed their own market to provide training and
vocational education services. However, the results of this study presented there are
another potential market that higher education can penetrate. By enforcing partnership
training with private business organizations, higher educations will be able to obtain
several benefits for their institutions such as developing up-to-date curriculum to meet
current business and utilizing existing training staff, facilities, and curriculum
continuously.
144
REFERENCES
Ahlstrand, A. L., Bassi, L. J., & McMurrer, D. P. (2003). Workplace Education for Low-
Wage Workers. Kalamazoo, MI: W. E. Upjohn Institute for Employment
Research.
Alliger, G. M., Tannebaum, S. I., Bennett, W., Traver, H., & Shotland, A. (1997). A
meta-analysis of the relations among training criteria. Personnel Psychology, 50,
341-358.
Anderson, E., Fornell, C., & Lehmann, D. (1994). Customer satisfaction, market share,
and profitability: Findings from Sweden. Journal of Marketing, July, 53-66.
Ary, D., Jacobs, L. C., & Razavieh, A. (2002). Introduction to Research in Education
(Vol. Sixth Edition). Belmont, CA: Wadsworth/Thomson Learning.
Barron, J. M., Berger, M. C., & Black, D. A. (1999). Do workers pay for on-the-job
training? Journal of Human Resources, 34, 235-252.
145
Borjas, G. (2000). Human Capital. Labor Economics, 226-261.
Brinkerhoff, R. O., & Gill, S. J. (1994). the learning alliance: Systems thinking in human
resource development. San Francisco: Jossey-Bass.
Brown, J. (2002). Training needs assessment: A must for developing an effective training
program. Public Personnel Management, 31(4), 569-580.
Cappelli, P., Bassi, L. J., Katz, H., Knoke, D., Osterman, P., & Useem, M. (1997).
Change at Work. New York: Oxford University Press.
Carnevale, A. P. (1998). Education and Training for America's Future. Washington D.C.:
The Manufacturing Institute.
Damodaran, A. (1999). Applied corporate finance. New York: John Wiley & Sons
Dillman, D. A. (2000). Mail and internet surveys: The tailored design method (Second
ed.). New York: John Wiley & Sons.
146
Eaton, S. C. (2003). If you can use them: Flexibility policies, organizational commitment,
and perceived performance. Industrial Relations, 42(2), 145-167.
Ellis, N. & Moon, S. (1998). Business and HE links: The research for meaningful
relationships in the placement marketplace-part two. Education & Training, 40(9),
390-397.
GAO. (2004). Workforce training: Almost half of state fund employment placement and
training through employer taxes and most coordinate with federally funded
programs (No. GAO-04-282). Washington, D.C.: United State of General
Accounting Office.
Gold, J., Whitehouse, N., & Hill, M. (1998). "If the CAPS fit...": Learning to manage in
SMEs. Education & Training, 40(6/7), 321-327.
Goldberg, M. & Ramos, L. (2003). Trainer's task: Do more with less. Pharmaceutical
Executive, 110-118.
Golfin, P. A., White, J. D., & Curtin, L. A. (1998). A role for community colleges in Navy
training (No. CRM-97-97). Alexandria, VA.: Center for Naval Analysis.
Gorman, P., McDonald, B., Moore, R., Glassman, A., Takeuchi, L., & Henry, M. (2003).
Custom Needs Assessment for Strategic HR Training: The Lost Angeles County
Experience. Public Personnel Management, 32(4), 475-496.
Gray, G. R., Hall, M. E., Miller, M., & Shasky, C. (1997). Training practices in state
government agencies. Public Personnel Management, 26(2), 187-202.
Gray, K. C. & Herr, E. L. (1998). Workforce Education: The Basics. Boston, MA: Allyn
and Bacon.
Grubb, W. N. & Stern, D. (1989). Separating the wheat from the chaff: The role of
vocational education in economic development. Berkeley, CA: National Center
for Research in Vocational Education.
Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (1998). Multivariate data
analysis (5th ed.). Upper Saddle River, NJ: Prentice Hall.
147
Hall, Z. W. & Scott, C. (2001). University-industry partnership. Science(291), 553-555.
Ham, J. C. (1994). Looking into the black box: Using experimental data to find out how
training works. International Journal of Manpower, 15(5), 32-37.
Hardingham, A. (1996). Improve an inside job wih an outside edge. People Management,
2(12), 45-47.
Hawley, J. D., Sommers, D., & Melendez, E. (2003a). The Earnings Impact of Adult
Workforce Education in Ohio. Paper presented at the Networking and Best
Practices in Workforce Development, The Ford Foundation.
Hawley, J. D., Sommers, D., & Melendez, E. (2005). The impact of institutional
collaborations on the achievement of workforce development performance
measures in Ohio. Adult Education Quarterly (forthcoming).
Hodson, R., Hooks, G., & Rieble, S. (1992). Customized training in the workplace. Work
and Occupations, 19(3), 272-292.
Ittner, C. & Larcker, D. (1998). Are nonfinancial measures leading indicators of financial
performance? An analysis of customer satisfaction. Journal of Accounting
Research, 36, 1-35.
Jacobs, R. L. (2003). Structured on-the-job training (2nd ed.). San Francisco: Berrett-
Koehler.
148
Jamrog, J. (2004). The perfect storm: The future of retention and engagement. Human
Resource Planning, 27(3), 26-34.
Kaufman, R., Rojas, A. M., & Mayer, H. (1993). Needs Assessment: A User's Guide.
Englewood Cliffs, NJ: Educational Technologies Publications.
Knoke, D. (1997). Job training programs and practices. In P. Cappelli, L. Bassi, H. Katz,
D. Knoke, P. Osterman & M. Useem (Eds.), Change at Work (pp. 122-153). New
York: Oxford University Press.
Knoke, D., & Janowiec-Kurle, L. (1999). Make or buy? The externalization of company
job training. Research in the Sociology of Organizations, 16, 85-106.
Kraiger, K., Ford, J. K., & Salas, E. (1993). Application of cognitive, skill-based, and
affective theories of learning outcomes to new methods of training evaluation.
Journal of Applied Psychology, 78, 311-328.
149
Lincoln, J. R. & Kalleberg, A. L. (1990). Culture, control, and commitment: A study of
work organization and attitudes in the United States and Japan. New York:
Cambridge University press.
Lumby, J. (1998). Restraining the further education market: Closing Pandora's box.
Education & Training, 44(2/3), 57-62.
Lupton, R. A., Weiss, J. E., & Peterson, R. T. (1999). Sales training evaluation model
(STEM): A conceptual framework. Industrial Marketing Management, 28(1), 73-
86.
Lynch, L. (1992). Private sector training and the earnings of young workers. American
Economic Review, 82, 299-312.
Lynch, R., Palmer, J., & Grubb, W. N. (1991). Community College Involvement in
Contract Training and Other Economic Development. Berkeley, CA: National
Ceter for Research in Vocational Education, University of California.
Mavin, S. & Bryans, P. (2000). Management development in the public sector: what roles
can universities play? The International Journal of Public Sector Management,
13(2/3), 142-152.
150
Moore, D. E., Christenson, J. A., & Ishler, A. S. (1987). Large-Scale Surveys. In D. E.
Johnson, L. R. Meiller, L. C. Miller & G. F. Summers (Eds.), Needs Assessment:
Theory and Methods (pp. 142-155). Ames, Iowa: Iowa State University Press.
Moore, R. W., Blake, D. R., Phillips, G. M., & McConaughy, D. (2003). Training that
works: Lessons from California's employment training panel program.
Kalamazoo, Michigan: Upjohn Institute for Employment Research.
Parrish, D. A. (1998). The new boom in client training. VARBusiness, 14(14), 91-94.
Phillips, J. J. (1996). ROI: The search for best practices. Training & Development, 50(2),
42-27.
151
Prince, C. (2002). Developments in the market for client-based management education.
Journal of European Industrial Training, 26(7), 353-359.
Rajan, A. & Harris, S. (2003, September 9, 2003). What works, What doesn't, and why.
Personnel Today, 19-21.
Rhodes, F. H. T. (2001). The creation of the future: The role of the American University.
Ithaca, NY: Cornell University Press.
Roessner, D., Ailes, C. P., Feller, I., & Parker, L. (1998). How industry benefits from
NSF's engineering research centers. Research Technology Management, 41(5),
40-44.
Rowley, D. J., Lujan, H. D., & Dolence, M. G. (1998). Strategic choices for the academy:
How demand for lifelong learning will re-create higher education. San Francisco:
Jossey-Bass.
152
Swanson, R. A. & Holton, E. f. (2001). Foundations of human resource development.
San Francisco: Berrett-Koehler.
Van Buren, M. E. & Erskine, W. (2002). The 2002 ASTD state of the industry report.
Alexandria, VA: American Society of Training and Development.
Warr, P. B. & Bunce, D. (1995). Trainee characteristics and the outcomes of open
learning. Personnel Psychology, 48(2), 347-375.
Weinberg, S. L. (2002). Data analysis for the behavioral sciences using SPSS.
Cambridge, UK: Cambridge University Press.
Wright, P. C. & Geroy, G. D. (1992). Needs analysis theory and the effectiveness of
large-scale government sponsored training programs: A case study. Journal of
Management Development(1), 1-27.
Yasin, M. M., Czuchry, A. J., Martin, J., & Feagins, R. (2000). An open system approach
to higher learning: The role of joint ventures with business. Industrial
Management + Data Systems, 100(5), 227-233.
153
APPENDIX A
TRAINING PROGRAM AND A RELATIONSHIP WITH TRAINIG PROVIDER
SURVEY QUESTIONNAIRE
154
155
156
157
158
159
160
161
162
163
164
APPENDIX B
TRAINING EFFECTIVENESS SURVEY QUESTIONNAIRE
165
166
167
168
169
170
171
APPENDIX C
COVER EMAIL SCRIPTS FOR BOTH SURVEYS
172
Cover Email Scripts for Manager Survey
Dear Mr._________
We are pleased that you will respond to this survey regarding the training programs
funded by the Ohio Investment of Training Programs (OITP). The survey is conducted
by the Ohio State University and supported by the OITP, Ohio Department of
Development. You will find the support letter from the OITP as an attachment.
It should take 15 minutes to respond to items in the questionnaire. Questions ask about
the training program characteristics funded by OITP. All responses to this survey will be
kept confidential. Only aggregate data will be reported in the study results. Your
participation is voluntary and you can skip any questions that you do not wish to
answer.
Please complete the survey and submit it by March 28, 2005.
If you have any questions, please contact Jeeyon Paek, the Ohio State University, via
email paek.7@osu.edu or phone 614‐783‐7890. You may also contact Dr. Joshua Hawley,
The Ohio State University (email: Hawley.32@osu.edu, phone: 614‐247‐6226)
Thank you again in advance for your time and effort.
Survey begins by clicking the following link
http://.paes.osu.edu/wde/~hrdmanagersurvey.html
Sincerely,
Jeeyon Paek
173
Cover Email Scripts for Senior Manager Survey
Dear Mr._________
We are pleased that you will respond to the survey regarding the training programs
funded by the Ohio Investment of Training Programs (OITP). This survey is conducted
by the Ohio State University and supported by the OITP, Ohio Department of
Development.
It should take 10 minutes to respond to items in the questionnaire. Questions ask about
the training effectiveness at your company. All responses to this survey will be kept
confidential. Only aggregate data will be reported in the study results. Your
participation is voluntary and you may skip any questions that you do not wish to
answer.
Please complete the survey by March 30, 2005.
If you have any questions, please contact Jeeyon Paek, The Ohio State University, via
email paek.7@osu.edu or phone 614‐273‐0642. You may also contact Dr. Joshua Hawley,
The Ohio State University (email: Hawley.32@osu.edu, phone: 614‐247‐6226)
Thank you again in advance for your time and assistance.
Survey begins by clicking the following link
http://.paes.osu.edu/wde/~hrdmanagersurvey.html
Sincerely,
Jeeyon Paek
174
APPENDIX D
SUPPORTING LETTERS FROM OITP FOR EACH SURVEY
175
176
177