Innov Manag - ProQuest - Diss19 PDF

ONLINE BUSINESS SIMULATIONS: A SUSTAINABLE OR DISRUPTIVE
INNOVATION IN MANAGEMENT EDUCATION?

by
Jason Scott Earl
MARY F. WHITMAN, DBA, Faculty Mentor and Chair

MAUDIE GALLOP HOLM, PhD, Committee Member
CLARK GILBERT, DBA, Committee Member
William A. Reed, PhD, Dean, School of Business and Technology
A Dissertation Presented in Partial Fulfillment

Of the Requirements for the Degree
Doctor of Philosophy
Capella University
June 2012
UMI Number: 3517084
All rights reserved

INFORMATION TO ALL USERS
The quality of this reproduction is dependent on the quality of the copy submitted.
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.
UMI 3517084
Copyright 2012 by ProQuest LLC.
All rights reserved. This edition of the work is protected against
unauthorized copying under Title 17, United States Code.
ProQuest LLC.
789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, MI 48106 - 1346
UMI Number: 3517084
All rights reserved

INFORMATION TO ALL USERS
The quality of this reproduction is dependent on the quality of the copy submitted.
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.
UMI 3517084
Copyright 2012 by ProQuest LLC.
All rights reserved. This edition of the work is protected against
unauthorized copying under Title 17, United States Code.
ProQuest LLC.
789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, MI 48106 - 1346
Jason Scott Earl, 2012
Abstract
The focal goal of this research was to extend the empirical effort on business
simulations as a form of experiential learning by providing the first empirical analysis of
business acumen and knowledge application skills. Disruptions in technology are
providing more opportunities to improve the simulation gaming learning experience and
a number of pedagogical innovations are beginning to emerge which will drive the way in
which business simulations are used in the future. The purpose of this quantitative,
experimentally-based research study was to investigate the use of online business
simulations as a disruptive technology by measuring the change in participants business
knowledge and business acumen compared to traditional corporate training. A sample of
65 participants was randomly selected from a company population of 720 employees and
managers. This quantitative based research study demonstrated the disruptive nature of
online business simulations when it comes to gains in business knowledge by measuring
a 2.55 standard deviation difference in the normalized gains between traditional training
and business simulation training. Baseline tests against a control group and traditional
training group using MANCOVA to account for multiple variables and covariates imply
that online business simulations enhance both business knowledge and business acumen
on a staggering scale and over a very short period of time.
iv
Dedication
This work is dedicated to my wonderful wife and mother of our five young
children. Thank you, Natalie for all of your love and support over these last 15 years. I
would have never left the private equity world and gone into academia without your faith
in me and our purpose in life together. If nothing else, I have learned from this long
journey that true teaching is leadership and most teaching is bad leadership. Thank you
for your example and personal leadership in our own family.
iii
Acknowledgments
Someone once said that defeat is bitter, but only if you swallow. I would like to
acknowledge the support of Dr. Mary Whitman who believed in me and on more than
one occasion, kept me from swallowing that bitter pill. I would also like to acknowledge
my dissertation committee members who have served as great examples to me and as a
source of personal inspiration. I hope that my life can be a small reflection to my own
students of the great principles that they have taught and, more importantly, lived.
iv
Table of Contents
Acknowledgments
iv
List of Tables
ix
List of Figures
CHAPTER 1. INTRODUCTION
Background to the Study
Introduction to the Problem
Statement of the Problem
Purpose of the Study
Specific Variables
Rationale
Research Questions
10
Null Hypotheses
11
Significance of the Study
12
Sustaining vs. Disruptive Innovation
14
Definition of Terms
15
Assumptions and Limitations
18
Nature of the Study
19
Organization of the Study
21
CHAPTER 2. LITERATURE REVIEW

Background on Experiential Learning
24
Nature of Business Simulations
28
Benefits of Experiential Learning
30
Educational Effectiveness of Simulations
31
Simulations and Corporate Training
33
Knowledge Application
35
Business Simulations as Knowledge Application Tools
36
Knowledge Application & Simulation Performance
38
Reflection
41
Reflection in Experiential Learning
43
Reflection Using Business Simulations
46
Debriefs Within Business Simulation
48
Business Simulations as a Disruptive Innovation
50
Business Simulations as a Disruptive Force Today
55
Andragogical Support for Business Simulations
58
CHAPTER 3. METHODOLOGY
Philosophy and Justification
65
Primary Research Question
66
General Linear Model for the Primary Research Question
68
Null and Alternative Hypotheses
69
vi
Research Design
70
Justification and Methodology
70
Research Design Strategy
71
Sample
75
Setting
80
Instrumentation and Measures
79
The Business Simulation
82
Dwyer and Gansters (1991) Autonomy/ Work Control
84
Field Testing
85
Business Simulation Performance Scores
85
Data Collection
86
Variables
87
Data Analysis
87
Validity and Reliability
92
Ethical Considerations
97
CHAPTER 4. RESULTS
Training for a Semiconductor Company
101
Participant Demographics
102
Normality Testing
103
ANOVA Analysis
109
MANCOVA Analysis
111
vii
Correlation Analysis
112
Comparison of Means
114
Hypothesis Testing
115
Alternative Hypotheses
117
118
Secondary Research Questions
118
Supporting Research Questions
119
Summary
121
CHAPTER 5. DISCUSSION, IMPLICATIONS, RECOMMENDATIONS RESULTS

Summary and Discussion of Results
124
Research Questions
124
Disruptive Innovation
129
General Discussion &Theoretical Implications
130
Practical Implications
131
Limitations and Recommendations for Future Research
135
Conclusion
137
REFERENCES
140
APPENDIX A. PRE-EXPERIMENT SURVEY & ASSESSMENT
152
APPENDIX B. POST-EXPERIMENT SURVEY & ASSESSMENT
157
viii
List of Tables
Table 1. Summary of Variables Used in This Study
21
Table 2. Instructional Methods Used by U.S. Organizations
33
Table 3. Null and Alternative Hypotheses
69
Table 4. Descriptive Statistics for Survey Items (N = 65)
104
Table 5. Descriptive Statistics for Dependent Variables (N = 65)
105
Table 6. Test of Homogeneity of Variances for Dependent Variables
106
Table 7. Test of Normality for Change in Business Knowledge
107
Table 8. Test of Normality for Change in Business Acumen
108
Table 9. ANOVA for Pre/Post-Test Results
109
Table 10. Tests of Between Subject Effects for Experimental Treatment
112
Table 11. Pearson Correlation Coefficients
113
Table 12. Group Statistics for Dependent Variables
114
Table 13. Independent Samples Test for Equality of Means
115
Table 14. Analysis of Variance Between Groups
120
Table 15. Summary of Hypothesis Testing
123
ix
List of Figures
Figure 1. Conceptual Framework
20
Figure 2. Venn Diagram Depicting the Focus Area of this Study
23
Figure 3. Impact of Disruptive Innovation
56
Figure 4. Impact of Disruptive Innovation on Military Training Industry
57
Figure 5. General Linear Model for Primary Research Question
68
Figure 6. Research Design Schematic
72
Figure 7. Overview of the Research Design
74
81
86
Figure 8. Comp-XM Comparative Standings

Figure 9. Foundation Business Simulation Performance Score.
Figure 10. Demographic Variable: Level of Autonomy
103
Figure 11. Normal Q-Q Plots and Boxplot for Change in Business Knowledge
107
Figure 12. Post-Test on Business Knowledge vs. Experimental Treatment
110
Figure 13. Post-Test on Business Acumen vs. Experimental Treatment
111
CHAPTER 1. INTRODUCTION
There has been a great divide between the gaming community and business
educators over the past twenty five years (Anderson & Lawton, 2009; Gosen &
Washbush, 2005; Wolfe, 1985). The gamers have dismissed educational simulations as
boring and irrelevant while business management educators have dismissed gaming and
simulations as trivial and pedagogically unproven (Aldrich, 2009a, xxi). Both appear to
be right and yet both may have missed an opportunity which lies within an engaging
business simulation and its potential impact on the world of management education
(Anderson & Lawton, 2009). Many business professionals have argued that the K-12 and
higher education systems are failing, myopically trapped in a nineteenth-century world of
learning by knowing, while the twenty-first-century world requires the judgment and
skill of learning by doing (Aldrich, 2009b, p. 12). Disruptions in technology are
which business simulations are used in the future (Faria, Hutchison, & Wellington., 2009,
p. 485). One of the major challenges with research in this field is that nobody has shown
definitively that simulation training works in the business world any better than
traditional instruction through workbooks or lectures (Davies, 2003, p. 36). The purpose
of this quantitative, experimentally-based research study was to investigate the use of
online business simulations as a disruptive technology by measuring the change in
participants business knowledge and business acumen compared to traditional corporate
training.
1
Background to the Study

In many respects, Blooms Taxonomy has been the anchor for assessing whether
learning occurs in business simulations (Bloom, Englehart, Furst, Hill, & Krathwohl,
1956). A substantial share of the early research on simulations focused on the attitudes
(affective domain) of participants exposed to the pedagogy. Much of this initial research
focused on comparing the general perceptions of students regarding cases, lectures, and
simulations (Anderson & Woodhouse, 1984; Blythe & Gosenpud, 1981). Subsequent
research expanded into attempting to assess what is learned from participating in a
simulation and almost all of these studies relied on perceptions and self-reports of
learning, rather than more objective measures (Anderson & Lawton, 2009, p.211). When
focused only on those studies that aim to examine participants affective reaction to
simulations, it is evident that students like simulation exercises and view them more
positively than either lectures or case discussions (Burns, Gentry, & Wolfe, 1990; Faria,
2001; Gosen & Washbush, 2004). It is worth noting that these relative comparisons have
been made by students experiencing different pedagogies within a course. There is a
dearth of studies employing experimental designs with control groups or where
comparisons are made between participants attitudes in one section of a course that is
solely lecture based versus those in a class that is solely case discussion based or solely
simulation based. Most business classes are taught today with one main pedagogy (e.g.,
lecture, case study or simulation) and almost no experimental studies exist that compare
learning outcomes under alternative pedagogies (Anderson & Lawton, 2009, p. 208).
Wolfe (1990) identified this problem over 20 years ago, yet the gap still exists.
Researchers continue to use self-assessments rather than more suitable tools because selfassessments are much easier to employ. As a consequence, studies on the educational
merits of simulations often are measuring the affective domain, not the cognitive domain
they purport to measure (Anderson & Lawton, 2009, p. 197). Using perceptions tends to
be advantageous to those who wish to claim the superiority of simulations over
alternative pedagogies because simulations almost invariably are rated positively by
students. The downside of using perceptions is that evidence based on perceptions often
is dismissed by scholars because it lacks suitable rigor. However, studies that attempt to
go beyond perceptions to more objective measures of learning more often than not use
tools best suited for measuring lower levels of learning on Blooms taxonomy (Anderson
& Lawton, 2009, p. 209). The decisions required to effectively run a business simulation
often tap analytical, synthesis, and application skills of Blooms taxonomy (Bloom,
1956). This has led some researchers to believe that using simulations is a powerful and
disruptive form of learning because it is taking place at the higher levels of Blooms
taxonomy (Smith, 2006).
Disruptions often bring significant changes to an industry and these disruptions
create opportunities for those organizations willing to adopt and champion disruptive
technology (Smith, 2006). Christensen (1997) has highlighted the sometimes devastating
impact in the corporate environment of what he refers to as disruptive innovations (p. 41).
Successful, well managed firms that dominate their markets have sometimes gone into a
sharp decline or even collapsed when a new technology disrupts the pattern of their
market segment. Other firms and organizations, however, have handled such transitions
3
smoothly, maintaining their position of dominance in the market by employing specific

techniques to integrate the new and disruptive technologies into their operations
(Christensen & Raynor, 2003, p. 57). Traditional research universities enjoy a dominant
position in the higher education market, but they are beginning to feel the impact of
disruptive innovations such as online universities, distance education, and continuing
education units as semiautonomous incubators (Archer, Garrison, & Anderson, 1999,
p.137).
According to Smith (2006), changes in the underlying technology for online
simulations have improved to the point that they are now more powerful than many of the
established pedagogical tools in the field of management education (p. 8). This
disruption in the field of management education is very similar to the innovation model
that Christensen first proposed in his dissertation and built upon in The Innovators
Dilemma (Christensen, 1997). This theory of disruptive innovation allows for a detailed
lens to be focused on business simulations in an attempt to identify them as sustaining,
complementary, or disruptive innovations. Gren-Yanoff and Weirich (2010) argues that
these technological and economical forces which are at work in the education industry
will create a tsunami of change throughout management education. Thus, they predict
this change will allow for the spread of more cost-effective, more powerful, and more
accessible simulations across the field of corporate training and management education
(Gren-Yanoff & Weirich, 2010, p. 45).
Introduction to the Problem

Of great concern to management education scholars is the identification of factors
which will contribute to rapid changes in business and how business leaders in the future
will be taught (Smith, 2006). Although many educators do not see simulations as a threat
to traditional education programs today, there are signs showing a significant impact
from simulations already (Andersen & Lawton, 2007). The advent of flight simulators
and computer games has finally introduced a technology and learning media which is
interactive, low-cost, and scalable. Today, authors are creating virtual velds where
participants can repeatedly practice skills, instead of just hearing about them (Aldrich,
2009b). As individual managers and employees gain more autonomy or work control
over day-to-day business decisions, the ability to take advantage of these disruptive
technologies is expected to increase dramatically. An empirical study performed at a
manufacturing facility involving corporate training demonstrated that experimental
interventions that aim to augment worker control over their tasks and work environment
increased productivity and innovation (Dwyer & Ganster, 1991). It may be that
education reformers are signaling the end of the age of learning how to know rather
than how to do or how to be in a complex, interactive world (Aldrich, 2009a).
Davies stated that part of the problem is that nobody has shown definitively that
simulation training works in the business world any better than workbooks or lectures
(2003). The challenge is determining whether these simulations are truly disruptive
innovations to management education or simply sustaining innovations.
According to Christensen (1997), the main reason why so many successful and
dominant institutions fail due to innovation within their industry is their inability to
5
recognize the difference between sustaining and disruptive innovations. Sustaining

technologies are typically sought after by these successful institutions because they
improve the performance of established products for their most lucrative clients. This
strikes many innovative professionals as paradoxical, because the excellent business
practice of listening closely to their customers does not lead to disruptive innovation,
only incremental sustaining innovations. Technologies, in the sense that Christensen uses
the word, may refer to either hard technologies that result in new types of physical
goods or soft technologies that result in new ways of organizing work or providing a
service. Interactive simulations are referred to as veld technologies based on the
learning-to-do skills. It is possible that these technologies will successfully challenge the
institutions of management and corporate education in the near future (Aldrich, 2009a,
p.212).
Statement of the Problem

D. Goleman, R. E. Boyatzis, and A. McKee (2004) suggest that one of the largest
mistakes in management education is to assume that simply acquiring more information
(e.g., business knowledge) will automatically lead to becoming a more effective manager
or leader. They state that the development of competencies in cognitive or intellectual
ability (e.g. business acumen) leads to outstanding performance. Goleman et al also
suggests that self-directed learning is an effective method of achieving sustainable
changes in both business knowledge and business acumen (2004, p. 274). Online
business simulations are quickly becoming an effective tool for self-directed learning
while enhancing these management competencies (Segon & Booth, 2009, p. 112). The
6
problem thus becomes how do business management educators significantly increase the
level of both business knowledge and business acumen for learners today?
Because of the open-ended content of most business simulations, participants can
have a hard time articulating what they have learned during their game-play experience
(Anderson & Lawton, 2007). For example, there are no well-conducted studies that
actually investigate the learning effects of business simulations on both learners
knowledge application skill and business acumen (Anderson & Lawton, 2009, p. 209). In
fact, almost all studies have focused on the students attitudes and perceptions of their
experience with the simulation (Aldrich, 2009, p. 220). The real gap in the research and
literature is the use of business simulations in the world of adult learning or andragogical
approach to learning (Anderson & Lawton, 2009). There is a need to determine whether
simulations are a disruptive innovation because the holy grail of research within
management education is to empirically demonstrate that andragogical techniques lead to
better learning outcomes (Knowles, Holton & Swanson, 2005, p. 235). Recent advances
in computer simulations allow potential business leaders to practice and rehearse these
business management skills in a context rich environment removed from real life, thus
allowing a participant the opportunity to strengthen their skills in an atmosphere of safety
(Sidor, 2008). Using computer based simulations, participants have the ability to review
business management skills and potentially modify their real world application skills.
This forms the following studys primary research question: Do online business
simulations provide an increase in knowledge or business acumen for participants, which
is on the order of a disruptive innovation?
Purpose of the Study

The purpose of this quantitative study was to investigate the use of online
business simulations as a disruptive technology by measuring the change in participants
business knowledge and business acumen compared to traditional corporate training.
This research was undertaken to prevent management educators from continuing to miss
the opportunity for creative learning by adopting inappropriate educational strategies that
are of little relevance to practicing managers (Burns, 1995, p. 284). Gosling and
Mintzberg (2004) have stated that business management is neither a science nor a
profession, neither a function nor combination of functions. Business management is a
practice and this research has a high level of relevance at this time because it allows
participants to appreciate the experience of making business decisions within a given
context. Management may use science, but it is an art that is combined with science
through craft (Gosling & Mintzberg, 2004, p. 19). This topic has been of repeated
interest in both simulation and gaming journals as well as business journals focused on
policy, research, and management education (Segon & Booth, 2009).
Specific Variables
Participants in the simulation group ran an online business simulation, titled
Foundation by Management Simulations Inc. which simulates five years of running a
company in a high-tech industry (Foundation , 2012). This simulated industry is very

similar to the semi-conductor industry where all participants work as managers and/ or
employees. A sample of 65 participants were randomly selected from a semiconductor
company population of 720 employees and managers for this research. The two
8
experimental groups (case study and business simulation) responded to prompt questions
during the three day training period. In an attempt to investigate these research questions,
this study also looked into which of the following variables contributed most to
participants business simulation performance:
(a) Number of years of industry experience
(b) Level of education (i.e., undergraduate, graduate)
(c) Area of expertise in the company
(d) Level of autonomy/ work control at the company
(e) Previous business simulation experience
Rationale
The following topics have been addressed by some scholars as fertile areas for
research: Do participants improve their grasp of interrelationships among the various
functions of business (marketing, finance, production, etc.) as a result of participating in a
simulation? Are the interpersonal skills of participants improved through participating in
a simulation? Do participants in simulations really develop a greater appreciation for the
difficulty of implementing what may, on the surface, appear to be rather straightforward
business concepts? Are business simulations really effective devices for integrating
participants into business programs, and are they effective at improving retention rates?
(Anderson & Lawton, 2009, p. 212).
Based on these research questions, both practical and theoretical reasons were
present for conducting this study. From a practical perspective, knowing the benefits of
engaging students in reflection activities may be very helpful in order to improve
9
business acumen and knowledge application skills for the participants. Understanding
the impact that this experiential activity can have on knowledge application and
simulation performance will provide information that is valuable to both designers and
administrators of future simulations. Furthermore, if the types of prompt questions used
in this study; (a) strategy questions for business acumen and (b) financial and/or
accounting questions for knowledge application, turn out to be effective in promoting
participants knowledge and simulation performance, then simulation designers and
business instructors will have a basis from which to make informed decisions when it
comes to designing a better learning experience. On a theoretical level, this study
integrated a number of the precepts of experiential learning, reflective learning, and
online simulations in order to provide information on the factors that promote
participants knowledge. This combination of instructional techniques clearly
demonstrated the impact of this teaching innovation. Past research has shown that a two
standard deviation gain in a pre/post-test of interactive-engagement compared to
traditional instruction may place this innovation on the order of disruptive scale for
education (Hake, 1998).
Research Questions
This research attempted to address the following research questions:
Primary Research Question:
1. Do online business simulations provide an increase in business knowledge or
business acumen for participants, which is on the order of a disruptive
innovation?
Secondary Research Questions:
10
2. Are knowledge application skills in business positively correlated with online

business simulation performance scores?
3. How does the change in knowledge application skills with traditional
corporate training compare with online business simulations?
Supporting Research Questions:
4. Do participants who are taught using an online business simulation gain a
significant increase in business knowledge and business acumen?
5. Do participants who are taught using traditional corporate training gain a
6. Is the level of education or industry experience positively correlated with
online business simulation performance scores?
7. Is the level of participant autonomy/ work control positively correlated with
This study used a MANCOVA technique for data analysis, based on three
experimental groups. The full general linear model and associated hypotheses are
discussed in Chapter 3. The null hypotheses based on these research questions are listed
below:
Null Hypotheses
Ho1. There is NO difference across experimental groups for business knowledge
after adjusting for previous simulation experience and autonomy/ work control.
Ho2. There is NO difference across experimental groups for business acumen
after adjusting for previous simulation experience and autonomy/ work control.
Ho3. NO correlation exists between participants business acumen and business
knowledge.
Ho4. Participants who engage in an online business simulation will NOT
demonstrate higher business knowledge after experiencing the business
simulation.
11
Ho5. Participants who engage in an online business simulation will NOT

demonstrate higher business acumen after experiencing the business simulation.
Ho6. There is NO difference between the simulation group, the case study group,
and the control group based on their change in business knowledge.
Ho7. There is NO difference between the simulation group, the case study group,
and the control group based on their change in business acumen.
Ho8. NO correlation exists between participants business simulation
performance and their level of autonomy/ work control in the company.
performance and their years of industry experience.
performance and their level of education.
Significance of the Study

This study provided key insight into what happens to participants in a business
simulation by measuring the overall increase in business knowledge and business
acumen. Several researchers have documented the benefits of using business simulations
in management education (Chapman & Sorege, 1999; Lefebvre, 1997; Segon, 2009).
The most common reported benefits included practice in an environment without risk,
increased creativity, more focused competitive analysis, increased cross-functional
understanding, and increased subject content knowledge. Although case studies and
practitioner experience support these benefits, little empirical evidence is offered on the
change process that an individual participant experiences (Scherpereel, 2005, p. 389).
Specifically, research has not studied adequately the effects of engaging students in a
business simulation and measuring the change in business acumen or knowledge
application skills. Anderson and Lawton (2009) refer to this as a dearth of studies
12
employing experimental designs ( p. 196). In addition to the possibility of simulations

operating on the order of a disruptive innovation, there is virtually no existing literature
investigating the order of magnitude impact that business simulations have on
participants when it comes to quickly and effectively mastering business management
skills (Anderson & Lawton, 2009, p. 211).
This study added to the body of knowledge on the nature and impact of business
simulations by employing five methods of measurement: (1) a pretest and posttest
designed to assess the participants knowledge application skills, (2) a pretest and posttest
designed to assess the participants business acumen, (3) participants responses to
different prompt questions, (4) business simulation performance scores, and (5) survey
questionnaires. The survey questionnaires were designed to investigate the participants
academic and work-related background as well as their individual level of autonomy or
work control within the organization. A sampling of 65 managers and employees at a
semiconductor manufacturing facility were randomly assigned to three different groups.
These groups consisted of (a) the traditional instruction (case study), (b) the online
business simulation group, and (c) the control group. Participants in both the traditional
instruction group and online business simulation group were prompted to respond to
strategy questions which evaluated their business acumen. These same participants were
also asked to answer knowledge application questions which evaluated their business
knowledge. Participants in the control group were not prompted to respond to either
strategy questions or knowledge application questions.
13
Sustaining vs. Disruptive Innovation

Christensens (1997) theory of disruptive innovation was built upon the concept
of radical and incremental innovation initially proposed by Dewar and Dutton (1986).
Christensen proposed that disruptive innovations are different than radical innovations in
that they have the value-destroying characteristics of radical innovation; however, these
innovations work much more slowly and methodically up through an industrys value
chain. Christensens theory states that disruptive innovations have a beginning point that
is actually much lower on the performance scale than similar existing technologies in the
same market. According to Smith (2006), the reason why low performance innovations
are so disruptive is due to their ability to meet the needs of a niche market that is
unaddressed by the current leading products and technologies (p. 4). Christensen (1997)
states that these disruptive innovations often grow in underserved markets because of
their low cost and consequently, are perceived as insignificant and less profitable to the
industry leaders due to much lower margins. This disdain held by industry leaders is
typically due to the small size of these niche markets and the small profits that are
available from them. Consequently, disruptive innovations move slowly up the value
chain and often take time to destroy the value of established products and technologies
(Christensen, 1997). In fact, almost all growth from these innovations is in completely
new markets. These disruptive innovations erode the value of formerly successful
institutions by systematically stealing away customers from the bottom of the value chain
and gaining more and more market share over time. As these disruptive innovations
improve the quality of their products or services to customers, they eventually meet the
expectations of a large customer base which is being served by the industry leaders and
14
suddenly become a real threat. While business simulations have been recognized as a
separate pedagogy for instruction, there is little empirical evidence in the research
literature on the magnitude of this innovation when it comes to measuring the increase in
business knowledge or business acumen (Faria et al., 2009, p. 485).
Accordingly, this research contributed to both the management education and
disruptive innovation literature. By investigating the order of magnitude impact that the
business simulation had on participants based on the pre-test and post-test analysis, it was
possible to determine whether online business simulations are simply a sustaining
innovation for educators or truly a disruptive technology that will change how managers
and business leaders learn in the future.
Definition of Terms
The following definitions were used in this study:
Assessment
The art and science of testing individuals to determine what they have learned or,
as is more often the case, what they have not learned (Grne-Yanoff & Weirich, 2010).
Business Simulation
Computer-based role-playing game which makes use of high fidelity simulated
environments and involves decision-making in the research & development (R&D),
marketing, operations, human resources (HR), and finance departments of a company
(Grne-Yanoff & Weirich, 2010).
15
Business Strategy
This is a particular game-play-strategy or group of unique business decisions
which are in complete alignment in order for participants to achieve the highest possible
score on simulation performance (Grne-Yanoff & Weirich, 2010).
A term coined by Clayton Christensen and used in business and technology
literature to describe innovations which improve a product or service for non-consumers
in ways that the market does not expect. This is typically done by lowering price or
designing for a different set of consumers. Few technologies are intrinsically disruptive
or sustaining in character; however, it is the strategy or business model behind the
technology that it enables, which creates the disruptive impact (Christensen & Raynor,
2003).
Experiential Learning
A learning model which begins with the experience, followed by reflection,
discussion, analysis, and evaluation of the experience (Albert, 1970).
Game Based Learning
A learning method which combines educational content and elements of computer
games (Aldrich, 2009a).
The process of selecting appropriate business knowledge suitable to the challenge
at hand, and making connections between selected business knowledge and specific
strategies (Sarin & McDermott, 2003).
16
Online Business Simulation

An instructional method based on a representation of a physical or social business
reality in which participants compete for certain outcomes according to an established set
of rules or constraints. The competition can be (1) among themselves as individuals or
groups, or (2) against some specified standard, working as individuals or cooperating as a
group (Szczurek, 1982).
Prompt Questions
Questions that prompt participants reflection and guide the process of their
knowledge application during the course of the simulation. Two different types of
prompt questions will be used for this study the Strategy Question and the Knowledge
Application Question.
Reflection
An important human ability, in which a person recaptures his or her experience,
thinks about it, mulls over it and evaluates it (Boud et al., 1985). Reflection typically
takes place after an experiential learning experience. These reflection periods are often
referred to as debriefs with an online business simulation.
Simulation Performance
Measured at the end of each round (or year) and defined by the companys
Balanced Scorecard which is a combination of metrics for the Company involving
financial ratios (i.e., return-on-equity and stock price), learning and growth of people,
customer satisfaction, and internal business processes.
17
Sustaining Innovation
A sustaining innovation allows for increases in performance at typically higher
costs in the same market without effecting non-consumers from other markets.
Sustaining innovations allow a given technology to continue to improve in their own
market, but they do not directly impact other markets (Christensen & Raynor, 2003).
Assumptions and Limitations

In order for this study to proceed, certain assumptions were required. First, the
study assumed that valid and reliable data existed and could be obtained. Second, the
study assumed that all managers at the semiconductor manufacturing facility would
participate in the Capstone (Management Simulations, Inc.) business simulation. Third,
participants would accurately complete the survey questionnaire. Fourth, all three groups
of participants would have a similar level of knowledge of business management related
to managing a growing business in a highly competitive industry.
As with most empirical studies, a number of limitations began to emerge. Some
of these limitations had an effect on the validity and reliability of results, as described
below.
1. This study used a specific type of business simulation which is in the domain
of managing a growing business in the semi-conductor or electronic sensor
industry; hence it may be difficult to generalize results to other types of
business simulations.
2. This study used a relatively small sample size of 65 business managers in the
semi-conductor or electronic sensor industry, both limiting the kinds of
18
quantitative analyses that can be conducted and reducing the generalizability

of results.
3. Analyses were limited to mainly quantitative methods. While inferential
statistical procedures showed significance with numerical data, there was a
need for some qualitative analysis to shed light on the participants learning
experience with a business simulation. This study allowed for some limited
qualitative analysis in a short-answer post-test survey.
4. This study focused on a group of managers and employees at the same
electronic sensor manufacturing company. Consequently, it will be difficult
to generalize results to other managers or employees working in other
industries.
Nature of the Study
A true experimental study was used to investigate the possible cause-and-effect
relationships by exposing two experimental groups (business simulation and traditional
instruction) to one or more treatment conditions and comparing the results to a control
group not receiving the treatment.
Figure 1 highlights the conceptual framework that was used in this study, showing
how each of the variables relates to the Research Questions and Hypotheses.
The conceptual framework reiterates the problem that was used for this study.
There are no clear empirical studies on whether online business simulations are a
sustaining or disruptive innovation when compared with traditional instruction. The
problem established the framework for the study and allowed for both business acumen
and business knowledge to be measured in a true experimental study.
19
PROBLEM
Are Online Business Simulations a Sustaining
or Disruptive Innovation?
DEPENDENT VARIABLES
- Business Simulation Score
RQ4
H1-H4
RQ1
in Business
Acumen
in Business
Knowledge
INDEPENDENT VARIABLES
- Online Business Simulation
- Traditional Instruction
RQ3
RQ5
H6
H7
Level of Autonomy in
the Company
H8
RQ2
H3
RQ4
H2-H5
DEMOGRAPHIC VARIABLES
(Impact Performance)
WHY?
RQ6
RQ7
Reflection Activity:
1. Level of Engagement
2. Ability to Apply Learning
3. Peer Instruction
4. Blooms Taxonomy
Simulation
Experience
H1-H2
Autonomy/
W-Control
H8
Industry
Experience
Education
Level
H9
H10
Figure 1. Conceptual Framework. The research questions and hypotheses in this figure
tie out with the dependent, independent, and demographic variables in order to define the
problem in this study.
Characteristics of this experimental design required rigorous management of
experimental variables and conditions by direct control/manipulation or through
randomization (Isaac & Michael, 1997). The independent variables in this study were
participation in an online business simulation or participation in traditional instruction.
There was also a control group which did not participate in either. The primary
dependent variables were the change in business knowledge and change in business
20
acumen based on the pre/post-test for each group. The intent of this study was to make
ten predictions about the relationships among the variables identified in the null
Hypotheses. A summary of all variables is found in the table below (Table 1).
Table 1.
Summary of Variables Used in This Study
Variable
Pedagogical Nature of Training
Change in Business Knowledge
Change in Business Acumen
Knowledge Application Score
Level of Autonomy at Company
Simulation Experience
Area of Expertise
Industry Experience
Level of Education
Type of Variable
Experimental Treatment
Dependent
Dependent
Dependent
Independent Mediating
(Continuous Covariate)
Independent Moderator
(Dichotomous Covariate)
Descriptive
Descriptive
Descriptive
Measurement
Independent
Comp-XM
Comp-XM
Foundation
Work Control (Dwyer)
Level
Ordinal
Ratio
Ratio
Ratio
Ratio
Survey Item
Nominal
Survey Item
Survey Item
Survey Item
Nominal
Ordinal
Ordinal
Organization of the Study

Following this introduction, the next chapter includes a thorough review of
relevant literature in the areas of experiential learning, online business simulations, and
disruptive innovation. This review supported the conceptual framework used in this
study and helped illuminate the gaps in existing works which represents opportunities for
future research. In chapter three, the research methodology is described
comprehensively, including the research design, experimental approach, review of
measurement instruments, and the data analysis procedures. Reliability and validity
concerns are also discussed. Chapter four reports results from the experiment as well as
from the subsequent data analysis which was conducted according to the procedures
outlined in chapter three. Finally, chapter five includes a discussion of results and their
21
implications. The study concludes with recommendations for future research based on
the results from the experiment.
22
CHAPTER 2. LITERATURE REVIEW
Corporations, management education institutions, development consulting firms,

and collegiate business programs use business simulations to train and teach (Summers,
2004). Many educators assume that business simulations will help participants learn how
too apply their knowledge to solve real
real-world
world problems based on their interactive,
integrative, and iterative nature (Anderson & Lawton, 1997). However, there are no
empirical studies that actually investigate the learning effects of business simulations on
participants knowledge application or increase in business acumen (Miller, 1998). In
fact, almost all studies have focused on the students attitudes and perception of their
experience with the simulation (Anderson & Lawton, 2006). The research
esearch in this
th study
helps fill an important research gap by examining the relationship between online
business simulations, experiential learning, and disruptive innovation.
Figure 2. Venn diagram depicting the focus area of this study

23
Overall, the educational merits of business simulations have been subject to

considerable debate (W. Biggs, 1990). While many universities use simulations, only a
small fraction of the faculty members within those institutions integrate simulations into
their coursework (Summers, 2004). There are studies which indicate that other forms of
pedagogy are just as effective or more effective than business simulations. Anderson and
Lawton (1990) argued that only a very weak link between participation in a simulation
and learning has been shown. These researchers also pointed out that valid, reliable
instruments to assess mastery are rare, and valid measures of higher level learning of
objectives are almost nonexistent. Kayes (2002) noted that while countless management
scholars and practitioners see experience as central to management learning, the notion
of experience has received critical attention. Kayes explains that criticisms of so called
experience-based learning arise for both empirical and theoretical reasons (p. 137).
Based on the gaps identified in the existing literature, this research study seeks a
causal relationship between online business simulations and experiential learning as a
possible source of disruption when compared to more traditional methods of corporate
training. This study therefore connects experiential learning theory from psychology
(Lewin, 1935) with the developing field of disruptive innovation (Christensen, 2008) in
order to support or refute the business case for significantly higher levels of adult
learning through the use of online business simulations.
Background on Experiential Learning

The use of business simulations as a form of experiential learning and disruptive
innovation has evolved significantly over the last six decades. The evolution and
24
development of experiential philosophy has been significantly influenced by five

individual researchers: John Dewey, Kurt Lewin, Jean Piaget, , David Kolb, and Clayton
Christensen.
Dewey's (1938) Experience and Education focuses on the conflict between
traditional and progressive education. The essence of Deweys work was that truth and
knowledge are not absolute but rather continuously evolving over time. Dewey
explained that experiences directly influence knowledge and what is come to be known
as truth. He further argued that experience should be incorporated into the education
process and that all education should be participatory in order to be experience-based.
Several of Deweys ideas have made their way into traditional educational programs over
the last sixty years, particularly at the primary and elementary levels. As individuals
grow, they find that new experiences conflict with earlier learning and knowledge.
Experiential educators in higher education are addressing these challenges by using an
innovative approach that incorporates the best of traditional and experiential
methodologies (Ruben, 1999).
Kurt Lewin is often referred to as the founder of American social psychology and
his work has laid much of the foundation for modern educational and organizational
development work. Lewins (1935) research on experiential learning and group
dynamics had a profound influence on the discipline of social psychology and
organizational behavior. Lewins studies on group dynamics and the methodology of
action research led to laboratory-training. This training is now considered one of the
most significant educational innovations of this century when it comes to the process of
learning and change (Kolb, 1984). Lewin described the change that takes place during
25
learning as a three-stage process; (1) unfreezing or overcoming inertia and dismantling

the existing mindset, (2) a period of confusing and transition where old ways are being
challenged but there is no clear picture of the new mindset that will take place, and (3)
the third and final stage he called freezing. Lewin (1947) describes this new mindset as
crystallizing and former comfort levels are returned to the learning participant.
Jean Piaget, a renowned French psychologist and epistemologist, is another major
contributor to experiential learning. The essence of Piaget's (1973) work is based on the
description of how intelligence is shaped by experience. Piaget stated that learning is the
product of interaction between the person and their environment. The growth and shape
of intelligence is impacted by decisions and the realization of consequences for each
decision. In this interaction, the ability for the person to act and experience consequences
is key. In Piagets studies of children, from infants to teenagers, this research
demonstrated the importance of abstract reasoning and interaction with the environment.
The ability of the child to manipulate symbols comes directly from the infants actions in
exploring and coping with the immediate concrete environment.
David Kolb (1984), an American educational theorist, laid the foundations of
modern experiential education theory based on the idea that knowledge is gained through
both personal and environmental experiences. Based on Lewins earlier work, Kolb
developed the experiential learning cycle which has been widely reproduced based on a
four-stage model of learning. Initially called The Lewinian Experiential Learning
Model, this model is now primarily recognized as Kolbs (1984). Kolb states that in
order to gain genuine knowledge from an experience, certain abilities are required; (1) the
learner must be willing to be actively involved in the experience, (2), the learner must be
26
able to reflect on the experience, (3) the learner must possess and use analytical skills to
conceptualize the experience, and (4) the learner must possess decision-making and
problem solving skills in order to use the new ideas gained from the experience. This
theory offers a fundamentally different view of the learning process from that of the
behavioral theories of learning based on an empirical epistemology or the more implicit
theories of learning that underlie traditional educational methods (Kolb, 1984).
Christensens (1997) The Innovators Dilemma laid the foundational research
which helped to explain why successful companies and institutions often fail to invest
aggressively in disruptive technologies. When applying this theory to higher education,
Christensen (2008) states that the more student-centric classrooms become, the more
demand there will be for new technologies. Christensens research also suggests ways to
identify innovations which are about to disrupt entire industries. The primary method for
identifying these innovations is that they begin as products or services which are much
simpler and cheaper than the existing competition. Also, these innovations generally
promise much lower margins and very little profit. The secondary method for
recognizing disruptive technologies is based on the fact that they are typically
commercialized in new and/ or insignificant markets (Archer, 1999). The third way that
disruptive innovations are recognized is based on the perspective of the leading firms
most profitable customers. Typically, these high-end customers, who are willing to pay
much higher prices, dont want or cant use such inferior products or services. In almost
every case, a disruptive innovation is initially embraced by customers who mean very
little to the industry leaders in that particular market. The great irony of this theory of
disruptive innovation is that those companies which are the best at listening to and
27
serving their most profitable and most successful customers are rarely able to build a case
for investing in disruptive technologies. When they finally do, it is often too late .
Christensens (1997) research highlights the fact that disruptive innovations in the
past were technologically straightforward and did not rely on a specific business model.
In fact, Christensen argues that many of the early disruptions in the computer industry
consisted of off-the-shelf components put together in a product architecture that was
simpler than prior approaches. These innovations typically offered less of what
customers in established markets wanted because they were much lower in product
performance. These disruptive innovations also began in emerging markets and
consequently, offered a different package of attributes which were considered
unimportant to the industry leaders. Christensen (2008) argues that through experiential
learning in a virtual environment, assessmentthe art and science of testing individuals to
determine what they have learnedcan be revolutionized . One of the potential areas for
this revolution in the virtual learning environment is online business simulations.
Nature of Business Simulations

Clearly distinguishing simulations from simulation games is quite difficult and
debatable. While a simulation imitates reality and is often used to predict what would
happen in a given scenario, the word game suggests playfulness and competition.
Simulation games combine all of these characteristics. Many researchers now use the
term simulation game (Jacobs & Dempsey, 1993) to describe a new class of games that
make use of high fidelity environments. Simulation games have also been defined
variously as a combination of simulations and games with competition (Heyman, 1982)
28
and as a subset of games (McGrenere, 1996). One of the more complex definitions is
proposed by Szczurek (1982). Szczurek referred to such educational tools as an
instructional method based on a simplified model or representation of a physical or social
reality in which participants compete for certain outcomes according to an established set
of rules or constraints. The competition in simulations is often against some specified
standard, where participants can work as individuals or cooperate as a team.
The origin of the business simulation dates back to 1955. In that year, the Rand
Corporation developed an exercise called Monopologs (Jackson, 1959). Monopologs
required its participants to perform as inventory managers in a simulated Air Force
supply system, thus providing decision-making experience without the risks associated
with the consequences of a wrong decision. The Air Force continued the use of
Monopologs for many years and reported it to be a highly successful training device.
One of the first practical and most successful business simulations for the masses was
Top Management by the American Management Association (AMA) in 1956. It was
used in numerous management seminars (Meier, Newell, & Pazer, 1969). Additionally,
the consulting firm of McKinsey and Company developed the Business Management
Game in 1957 for use in its management seminars (Andlinger, 1958) and the University
of Washington became the first university classroom user of a business game when a
simulation developed by Scheiber was used in a business policy course in 1957 (Watson,
1981).
The increased usage of business simulations can be measured in several different
ways. First, the number of simulations available in the market has increased dramatically
over the last ten years. Second, the number of organizations and journals devoted to
29
business games and simulations has also risen significantly (Faria, 2009). An email
survey of 14,497 business faculty members across all disciplines at American Association
of Collegiate Schools of Business member schools led to only 1,085 respondents. Of
those who responded, 30.6% were current business simulation users, 17.1% were former
simulation users and 52.3% had never used a business simulation in their coursework. In
earlier work, Faria (2004) had estimated that 95% of AACSB schools (The Association to
Advance College Schools of Business) and 86% of all business schools in the United
States were using business simulation games. Surprisingly, business simulations showed
the highest use in business policy and marketing areas.
Benefits of Experiential Learning

Experiential exercises, including role-plays and simulations, have been widely
used for educational purposes. Learning objectives are thought to be accomplished by
providing realistic, but controlled, environments in which students are guided only by
implicit rules. Although there are distinctions between different modes of simulation, the
design of most simulations allows students to be exposed to stimuli that encourage them
to acquire the key concepts of the subject area being taught (Druckman & Ebner, 2008).
Cherryholmes completed the earliest evaluations of simulation learning outcomes (1966).
He evaluated five hypothesized topicsinterest, learning, retention, critical thinking, and
attitudeswith six studies using complex simulations conducted over periods of time
ranging from one day to 12 weeks. The results were clear: Only interest in the material
being learned by the simulation participants improved significantly (compared to case
study and other conventional classroom approaches); negligible changes occurred on
30
learning and attitudinal variables. However, there were convergent findings when the
task of designing a simulation before playing it, either re-designing an existing game or
constructing one of their own (Cherryholmes, 1966, p. 7). Based on finding from
Cherryholmes research, a design opportunity will be provided for in this study, where
participants will be able to choose the financial outcomes and ratios that are most
important to their chosen strategy while running the business simulation.
Educational Effectiveness of Simulations

Pierfy (1977) reviewed the findings obtained from comparative evaluation studies
reported during the 1960s and 1970s. With regard to learning, 15 of 21 studies showed
no difference between simulations and other instructional techniques. With regard to
retaining the information learned, 8 of 11 studies showed that the students participating in
simulations retained the information longer than those exposed to other instructional
techniques. On interest, 7 of 8 studies reported that students showed more interest in the
simulation activities than in more conventional classroom tasks. In their update of
Pierfys review, Bredemeier and Greenblat (1981) concluded that simulations are as
effective as, but not better than, other instructional methods on learning the subject. They
stated that simulations are more effective when used only as aids to retaining the learned
material and in instilling a positive attitude toward the subject matter. Also supporting
these results is research completed by Ellington, Fowlie, and Gordon (1998) who found
that simulations have an advantage over traditional methods in motivation, participant
involvement, and commitment. A question suggested, but not answered, by the studies is
why the learning impacts are modest. Wolfe and Crookall (1998) have also asked why
31
the field of experiential gaming has made little progress. Regarding educational
effectiveness, a variety of suggestions for significantly improving the contribution of
simulations has been given. Examples include clarifying learning objectives (Bredemeier
& Greenblat, 1981), providing more conceptual background on the subject prior to the
simulation activity (Druckman & Robinson, 1998), creating time for reflection on the
events and getting feedback (Mclaughlan & Kircpatrick, 2005), and providing
participants with conceptual maps and graphics that reflect the simulations purpose
(Druckman & Ebner, 2008).
In contrast to just game playing, actual simulation design contributes to analysis
by identifying critical elements (roles, goals, resources, and rules) leading to new
analytical questions (Ebner & Efron, 2005). Attention to the design process remains a
strong focus, as evidenced by the simulation-building exercise featured at the 2007
International Simulation and Gaming Association conference at Nijmegen in the
Netherlands (Durckman & Ebner, 2008). These observations suggest the hypothesis that
simulation designers learn more about the concepts being simulated than do simulation
role-players. Crookal expounded on this hypothesis when he stated the key features of
the design process: (a) Design is concrete you can touch the results; (b) it is creative
you develop an object, and (c) it is involving you develop understanding in a passionate
and intimate way (1995, p.161). When participants have the ability to make changes
within the simulation and design their own experience, the learning about relations
between different concepts goes up significantly and approaches synthesis (Greenblat,
1998). In the case of an online business simulation, the participants in this research study
will have the opportunity to design their own performance metrics for their own company
32
and specific industry of competitors. This interactive engagement with the simulation
may lead to higher levels of motivation, participant involvement and commitment.
Simulations and Corporate Training

Organizations have a wide variety of methods available for training their
employees and business simulations have begun to enter this market. Table 2 reports
U.S. organizations usage of selected instructional materials and methods from Training
Magazines 2003 survey (Galvin, 2003). Computer-based games and simulations have
low usage rates, with 1% of respondents always using them, 9% often using them, 47%
seldom using them, and 44% never using them.
Table 2
Instructional Methods Used by U.S. Organizations (Galvin, 2003)
Percentage of Respondents
Often or Always
Never or Seldom
Instructional Method
Instructor-led classroom
Self-study, Web based
Performance Support
Public seminars
Case studies
Role-play
Non-computer-based games, simulations
Self-study, non-computer based
Virtual classroom with instructor
91
44
44
42
40
35
25
23
21
9
6
3
Computer-based games, simulations

Experiential programs
Virtual reality programs
SOURCE: Industry Report (2003, p. 31)
33
9
56
56
58
60
65
75
77
79
91
94
97
Despite the low percentage of business simulations used in 2003, Summers (2004)
outlines three specific advantages to using computer-based simulations in corporate
training which will allow it to grow exponentially in the future. These three advantages
are; (a) specific knowledge, (b) learning on demand and (c) lower costs (Summers, 2004,
p. 226). Summers also argued that many new technology companies are introducing
business simulations to corporations based on the benefits of learning on demand (p.
228). This research poses the question: Will the new technology companies come to
dominate or even replace the old? If so, this would be a case of creative destruction, a
concept posed by Schumpeter (1911/1989).
Alternatively, the new technology companies which introduce these simulations
to corporate training programs may simply increase the number and variety of products.
When the theory of disruptive innovation (Christensen, 1997) is cast on this dilemma, the
outcome depends on whether the new technology is superior and whether the new
technology companies can consolidate the industry (Summers, 2004, p. 232). Online
business simulations have rapidly penetrated business schools; however, the superiority
of such experiential learning methods has not been proven in the corporate training arena.
A simple increase in the number and variety of training methods for management
education would indicate a sustaining innovation for the industry, while a significant
consolidation of the corporate training industry would indicate a disruptive innovation
(Christensen, 2008).
34
Application of theory is an on-going issue in higher education (Falkenberg,
Russell and Ricker, 2000). The fact that most of what participants learn is intended for
application to problem situations in real life is indicative of the importance of knowledge
application as a learning objective, especially in the field of management education.
Bloom, Engleheart, Furst, Hill, and Drathwohl (1959) developed a system that classified
learning into six levels. These levels arranged in a hierarchical order to reflect
progressively higher levels of learning. They are, in ascending order: basic knowledge,
comprehension, application, analysis, objective synthesis, and objective evaluation.
Bloom (1956) showed the components of knowledge application in the problemsolving process of answering questions. The process involves six steps, in ascending
order: restructuring and classifying situations, selection of abstraction suitable to problem
type, the use of abstractions to solve a problem, and solution to a problem. This process
shows that in order to solve a problem through knowledge application, there are certain
steps to be followed. Learning how to apply knowledge means learning how to follow
these steps effectively. Bloom (1956) distinguished knowledge application from
knowledge comprehension by saying that a demonstration of comprehension shows that a
student can use the abstraction when its use is specified; while a demonstration of
application shows that he/she will use it correctly, given an appropriate situation in which
no mode of solution is specified.
Bloom (1956) stated that comprehending an abstraction does not certify that the
individual will be able to apply it correctly. Thus, participants need practice in applying
their knowledge to real-world problems in order to make their knowledge more useful for
35
real-world decision making rather than remain inert. In this study, knowledge
application in a business simulation refers to the process of selecting appropriate
business knowledge suitable to the problem at hand, and making connections between
selected business knowledge and business strategies to solve a complex problem. Agyris
and Shon (1974) argued that the only way for organizational effectiveness to increase
over time was through individuals learning from experience. Online business simulations
build upon this concept because the application of learning is based almost completely on
the participants experiences. Agryis and Shon (1974) stated that these types of
experiences provide insights during the learning process and allow for specific
knowledge application tools to be developed leading to consistent learning outcomes .
Business Simulations as Knowledge Application Tools

One major problem that comes from learning with business simulations is that it
is not always clear that learners will leave with exactly the same conclusions, mental
models, and learning outcomes. In fact, it is not clear that learners will be able to apply
what they have learned in future real-world situations. From their experiment with
business school students, Mandl, Gruber, and Renkl (1992) confirmed that students using
a computer-based simulation had serious deficits in knowledge application and problem
solving using their previous knowledge. The results of the experiment clearly show that
business school students have considerable deficits in using their own declarative
knowledge that they acquired in business school, and they are not able to use their
knowledge as a tool in the real world. This is largely due to the fact that those students
gained their business knowledge mostly through traditional methods, like lectures, case
36
studies, and textbooks, and had not had enough opportunities to practice applying their
knowledge to real-world problem-solving either in a specific situation or in a simulated
environment. As a result, their knowledge was not sufficiently conditioned to relevant
application conditions, and remained inert.
Training students to apply their knowledge requires very different methods of
instruction than training them to memorize information or understand relationships within
a business context (Reigeluth & Moore, 1999). In business education, the most
commonly used instructional methods are those of linear formats, such as
lecture/textbook format and case analysis. These linear formats can be more efficient
than the experiential learning method for communicating a large number of concepts to a
large number of students. However, these formats do not do enough to encourage
creativity, problem solving, decision making, risk taking, and knowledge application. In
addition, most knowledge that business school students acquire from their lectures and
textbooks is what Anderson (1985) called the declarative knowledge: various business
concepts and principles are taught in a declarative form rather than an experiential form.
Therefore, when students learn business knowledge, they often treat new information as
facts to be learned rather than knowledge to be used. As a result, many business school
students have difficulties in using their knowledge as a tool in their business decision
making (Mandl et al., 1992). Since their knowledge is not based on actual experience, it
often remains inert.
Not all business knowledge needs to be taught in a procedural form. However,
most of those management-related concepts and principles consist of knowledge about
how to do things, which is what Anderson (1985) called the procedural knowledge.
37
Procedural knowledge is goal-oriented performance knowledge that can be executed

efficiently. Therefore, when learning this kind of management-related knowledge,
students need opportunities in which they can transform their declarative knowledge into
procedural knowledge so that their knowledge can be more readily available for realworld problem solving rather than remaining inert. One important instructional goal of
business simulations is to help students transform their declarative knowledge into
procedural knowledge. Business simulations are designed to help students practice in
applying their knowledge, which is mostly in a declarative form, into specific actionoriented problems in a relatively safe, controlled, and simulated environment.
Knowledge Application & Simulation Performance

Business instructors have promoted simulations as a means of accomplishing a
wide range of learning objectives, including improving interpersonal skills
(VirtuaLeader), improving general decision-making skills (Capsim), and helping
individuals to understand themselves (Second Life). Anderson and Lawton (1997)
outlined learning objectives common to simulation exercises. These include outcomes
such as increased knowledge of facts and concepts of the business discipline; improved
analytical skills, critical thinking, decision making, and interpersonal relations; enhanced
ability to simultaneously manage interrelationships; and a better understanding of
business dynamics.
While business simulations are believed to have the potential to stimulate learning
at all levels of learning objectives, many scholars argue that simulations are best suited to
facilitate learning at the higher level of Blooms Taxonomy of learning objectives:
38
application, analysis, and synthesis of knowledge (Anderson & Lawton, 1997). In fact,
researchers in business education have frequently used Blooms Taxonomy as a
framework for guiding their thinking on the areas where simulations are likely to have the
greatest impact on learning (Bloom et al., 1959). Since business simulations require
participants to act in the role of managers, it would seem likely that, if business
simulations excel in any area they would be strong in application (Anderson & Lawton,
2002). However, objective evidence for business simulation effectiveness at these higher
levels of Blooms taxonomy has been lacking.
Measuring the higher levels of Blooms taxonomy of learning objectives has
proven to be a difficult task. A lack of reliable and valid instruments has hindered
attempts to measure the learning occurring at the higher levels of Blooms taxonomy
(Anderson & Lawton, 1995). Thus, while Blooms taxonomy provides a useful
framework for the purpose of establishing learning objectives, the framework has not
been as helpful for assessing student learning, especially in the context of business
simulations. However, Anderson and Lawton (2002) conducted a study that has been
directed exclusively at the efficacy of simulations as a pedagogy for learning about the
application of business concepts. The premise underlying their research was that if
simulation performance does reflect learning associated with analysis and application,
those students who apply the concepts that are critical to the discipline should outperform
those who do not apply the knowledge presented in the course. The results of their study
showed a significant relationship between the application of concepts presented in a basic
marketing course and performance on a marketing simulation game. The greater the
39
number of concepts that students utilized in the management of their simulation

company, the higher their performance scores.
These findings provide a powerful validation for simulations as learning and
teaching tools. In answer to the question of whether applying the principles and concepts
of a discipline results in positive results in a simulation, the answer appears to be an
emphatic yes (Anderson & Lawton, 2002). Their study demonstrated that simulations
are useful tools for operating at the application level of Blooms hierarchy, the level at
which traditional classroom lectures are thought to be weak. De Jong and Van Joolingen
(1998) point out that an appropriate design theory for instructional simulations may arise
based on this higher form of learning. These researchers also defend that discovery
learning with simulations can take its place in learning and instruction as a new line of
learning environments based on technology where more emphasis is being placed on the
learners own responsibility ( pp. 19).
Other research that investigated the link between knowledge application and
simulation performance is found in a study conducted by Wolfe and Luethge (2003).
Wolfe and Luethge investigated whether students, who were involved more in the
simulation and applied more business knowledge over time, performed better than
students who were less involved and applied less business knowledge. They examined
the following in this study; (a) the degree to which simulated companies in a business
simulation need to be run by active, knowledgeable, and engaged players and (b) the
extent to which strategic involvement affects performance. The results of this study
found that less involved students, who applied less business concepts, performed poorly
relative to more engaged students. The results of their study indicated that good
40
performance in a business simulation is not the result of luck or random guesses and that
a business simulation rewards intelligent, planned decision-making practices.
Reflection
The word reflection appears frequently in the literature relating to experiential
learning. Boud et al. (1985) defined reflection as an important human ability in which
people recapture their experience, think about it and evaluate it. Dewey (1910) pointed
out that all genuine education comes through experience and that reflection can assist in
this process. The importance of reflection goes back to Deweys early writing, but there
has been increased interest in researching and using reflective processes in adult teaching
in the last twenty years (Salmon, 2001).
Bruce (2001) stated that reflection is described as contemplating the results of a
given experience within the overall context of the impact on the individual. Boud, Keogh
and Walker (1985) argued that only when this reflective process leads to a significant
change in behavior, can it be called reflective learning. Costa and Garmston (1994)
stated that reflective learning is the ability to mentally wander through a recent personal
experience. This mental process of reflection includes the following; (a) drawing forth
cognitive and emotional information from visual, auditory, kinesthetic, and tactile
sources, (b) linking information to previous learning, (c) comparing the results that were
anticipated and intended with the results that were achieved, (d) searching for effects and
finding connections among causal factors, (e) acting on and processing the information
by analyzing, synthesizing, and evaluating, (f) applying learning to contexts beyond the
41
one in which it was learned and making commitments to plans of action, and (g) the
metacognitive process of thinking about thinking.
Some researchers argue that reflection is essentially an independent activity. An
online business simulation allows for ideal period of reflection in between rounds of
company decisions. This period of reflection or debrief allows participants to ask critical
questions about their company performance and challenge their own basic assumptions
about strategy and execution (Davies, 2003; Foreman, 2004). Other researchers stress the
importance of collaboration with others in terms of the reflection process (Rose, 1992).
Lin (1999) argued that students reflection can be enhanced in a reflective social
discourse, and defined reflective thinking as actively monitoring, evaluating, and
modifying ones thinking and comparing it to both expert models and peers.
Reflection on experience is based upon the metacognitive theory developed by
Flavell (1987), who argued that becoming aware of oneself as a learner allows the student
to reflect, monitor, and revise the process and products of his own learning. The term
metacognition itself emerged from the early work of Flavell who referred to it as
knowledge concerning ones own cognitive process and products or anything related to
them (Flavell, 1976).
J. Biggs (1985) discussed the role of metacognition in learning, utilizing the term
metalearning to define the application of metacognition to student learning. More
particularly, he also defined metalearning as students awareness of their learning and
control over their strategy selection and employment. According to J. Biggs (1988), a
metalearner is one who is aware of their motives, task demands and personal cognitive
resources and exerts control over strategies used. J. Biggs (1988) also stated that these
42
reflections invite thinking about thinking which helps participants to make meaning out
of recent events and experiences. In this way, helping students develop abilities to
monitor and revise their own strategies and uses of resources may enable them to
improve general learning expertise that can be used in a wide variety of settings (Brown,
Campione, & Day, 1981; Scardamalia & Bereiter, 1991). Furthermore, by monitoring
effectiveness of ones own learning and uses of resources, participants may be able to see
the need to pursue a new level of learning and understanding (Bransford & Nitsch, 1978;
Chi, Bassok, Lews, Reimann, & Glaser, 1989). Online business simulations allow for
this self-awareness where participants can escape the nineteenth-century world of
learning by knowing and begin to compete in the twenty-first-century world which
requires the judgement and skill of learning by doing (Aldrich, 2009b, p. 12).
Disruptions in technology are providing more opportunities to improve the simulation
gaming learning experience and a number of pedagogical innovations are beginning to
emerge which will drive the way in which business simulations are used in the future
(Faria et al., 2009, p. 485). The underlying idea of metacognitive reflection is to enable
students to develop the ability to learn how to learn (Vye, Schwartz, Bransford, Barron,
and Zech, 1998).
Reflection in Experiential Learning

In experiential learning, simply to experience is not enough (Boud et al., 1985).
Often learners are so deeply involved in the experience itself that they are unable, or do
not have the opportunity, to step back from it and reflect upon what they are doing in a
critical way. In his Meta-model for learning from experience, Montgomery (1993)
43
emphasized the importance of reflection in experiential learning. He claimed that

reflection is what enables the learners to generalize, as they create mental models from
their experience. It is, in many ways, the process of learning from experience, and it is
essential that the learners reflect on their learning in order to give meaning and
significance to their experience. Thus, in order to maximize meaning from experiences,
there should be a phase of observation and reflection activities, whereby participants
discuss their experiences and different perspectives so they become more understandable.
The ultimate purpose of reflection and one of the great outcomes of business simulations
which allow for reflection is that it gets participants into the habit of thinking about their
experiences and relating that experience to future possible events.
In addition to Montgomery, many authors have argued the importance of
reflection in experiential learning. Albert (1970) argued that the assumption of
experiential learning is that we seldom learn from experience unless we assess the
experience, assigning our own meaning in terms of our own goals, aims, ambitions, and
expectations, and from these processes come insights, discoveries, and understanding.
Kolb (1984) explained that participants involved in an experiential learning
exercise must be able to (a) involve themselves fully, openly, and without bias in new
experiences, (b) observe and reflect on the experiences from many perspectives, (c)
create concepts that integrate their observations into logically sound theories, and (d) use
these theories to make decisions and solve problems.
In addition, Schon (1983) claimed that there are two different kinds of reflection:
reflection on action and reflection in action. Reflection-on-action consists of
reflection after the event, or out of the experience. It is a deliberate, conscious, and
44
public activity principally designed to improve future action. However, reflection-inaction is the contemplation of the experience itself, and is supposed to guide further
action for the rest of the experience (Bruce, 2001). Bruce also argued that this type of
reflection involves the necessary gathering and analysis of the data which is very actionoriented in that learners because they must use various tools to convert the data gathered
into useful information (2001). For example, in learning with a business simulation,
reflection-on-action involves stepping back from running the simulation and reflecting on
the different strategies employed after the simulation is over. When the simulation is
over, the student may take time to reflect on what strategies were employed, how they
worked, and their outcome.
In contrast, reflection-in-action, as the term suggests, occurs in the midst of
action. It is described as the consideration of the action being undertaken, and based on a
rapid interpretation of the situation, where rapid decisions are required. Bruce (2001)
stated that this type of reflection involves incorporating the data and information gathered
from the recent experience into immediate action. For example, in learning with a
business simulation, reflection-in-action involves stepping back from playing the game
and reflecting on ones current strategies while a student is still running the simulation.
Thus, in the midst of playing the simulation, a student may take a moment to reflect on
what strategies are being employed, how they are working, and their potential outcome.
Ertmer and Newby (1996) extended Schons (1983, 1987) notions of reflectionon-action and reflection-in-action to include reflection-for-action, which involves
empowering reflective thinking skills to evaluate the results of ones own learning
efforts. Unlike reflection-on-action, this type of reflection directs attention towards a
45
future orientation and considers a more directive global perspective than simply taking
immediate action. Bruce (2001) states that the learner in this realm seeks a mental
correlation between the long-term desired goal and the intermediate action steps required
to meet that goal. For example, in learning with a business simulation, reflection-foraction involves crosschecking current strategies with the desired goal, and then revising
these strategies based on the results for reflection-on-action. Thus, reflection-for-action
is more future-oriented than reflection-on-action. Reflection-on-action is simply stepping
back from a round of the simulation and reflecting on current business results. However,
reflection-for-action goes one step further and involves revising the current strategies
employed in order to achieve the desired goal more effectively.
Reflection Using Business Simulations

Many authors have tried to pinpoint the problems of business simulations. Quinn
(1994) noted that there was insufficient pedagogical support in instructional computer
games; the pedagogical support that will ensure that learners learn what they are
supposed to, empower them to become more independent in their approach to learning,
and help them justify and monitor their uses of knowledge and strategies. The challenge
with business simulations as learning tools often results from a lack of pedagogical
support. Prensky (2001) pointed out that one of the major problems of business
simulations is that students often become very non-reflective and non-analytical about
what they are doing during the game play. However, the effects of reflection on a
business simulation are not completely understood. In the last 10 years, several studies
have been directed exclusively at the relationship between reflection and the outcomes of
46
business simulation performance (Adkins, 2004; Bruce, 2001; Davies, 2003). These
intelligent or open simulations present the user with a complex and dynamic business
problem that a learner cannot master simply by trial and error (Foreman, 2004). The
format of the simulation based on ethics, teamwork, innovation, leadership, motivation,
and conflict management require a period of reflection for the learner to understand what
has just happened (Adkins, 2004; Foreman, 2004).
One research study conducted an experiment which attempted to measure
participants reflection-in-action, reflection-on-action, and reflection-for-action during an
undergraduate business course while the participants were using a business simulation
(Bruce, 2001). The design of this particular experiment focused on capturing the
reflective behaviors demonstrated by the participants. Measurement of these behaviors
involved assessing the quality and quantity of time spent by participants of the business
simulation in reflecting on the different aspects of the experience. Participants involved
in the simulation were asked to record the amount of time and the quality of that time
spent on different types of decision-making activities. This experiment made several
measurements of the identified constructs and data values were collected as the
participants made quarterly marketing decisions within the simulation. The other source
of data for this research study came from the financial and operational reports which are
generated based on each round of the marketing simulation (Bruce, 2001). Three
hypotheses were tested in this study: (a) reflection-in-action time will decrease over the
length of the simulation experience, (b) reflection-on-action will decrease over the
simulation experience, and (c) reflection-for-action will remain constant during the
simulation experience.
47
Bruce (2001) also stated that based on the specific observations from this study,
the teams which outperformed their peers demonstrated improved decision making as
evidenced by decreased time spent in both reflection-in-action and reflection-on-action.
The teams, which did not perform well, were able to improve their financial measures
(i.e., net income) but at a much slower rate. For these teams which did not perform as
well, a pattern of decreased reflection time was not evident. Bruce (2001) argued that
based on these findings there was a strong correlation between reflection and actual
learning with a business simulation. However, further study is required to investigate
what kind of role reflection plays in learning with simulations and how to utilize
participants reflection to make learning with business simulations more effective.
Debriefs Within Business Simulations

Debriefing is the most frequently used reflection method in learning with business
simulations. Lederman (1984) defined a debrief as an oral discussion in which
participants and instructors engage in a question and answer session designed to guide
participants through a reflective process about what they have just experienced in the
simulation. Debriefing has also been defined as learning through reflection on a
simulation experience (Pearson & Smith, 1986; Raths, 1987; Thatcher, 1990); emotional
recovery based on recent critical incidents (Bergemann & Queen, 1987); appraisal and
synthesis of work-related groups (DeNichola, 1990) or analysis of job performance in
order to build managerial strengths within a team (Bailey, 1990; Lederman, 1992).
In a debriefing, the learners who have just completed a round of the simulation
experience are given the opportunity to speak, and they describe what they have learned
48
including their emotions during the game. Special emphasis is placed upon situations
which occurred during the simulation that had a significant effect upon the results and
outcome, which the participants failed to detect in their own analysis. Lederman (1992)
stated the primary purpose and rational for debriefing was to (1) help participants gain an
understanding of what has happened, (2) find out what the participants have learned, and
(3) to test these outcomes against the instructors learning objective. Lederman makes it
clear that the purpose of the debrief is NOT to tell the participants that they have learned
what the instructor wanted them to learn. The process is much more discovery based as
the instructor wants to find out what was learned and determine what the implications are
for both the participants and the instructor (Lederman, 1992).
Over forty years ago, Harry (1971) stated that debriefing is a sine qua non for
learning in a simulation, and stated that a post-game discussion is necessary for
maximum effectiveness of any simulation. A few years later Gillespie (1973) observed
that simulations were not self-teaching and needed a good debriefing session to assist
students in reflecting on their behavior and the purpose of the simulation. More recently,
Thatcher (1990) stated that debriefing helps participants to reflect upon their learning and
in doing so come to recognize what they have learned.
In the military, debriefing is known as the after action review, when the players
climb out of their simulated tanks, cockpits, or command tents and discuss the battle
(Presnsky, 2001). The role of such an after-simulation reflective processing is to help the
participants highlight and generalize the various lessons learned so that they can later
apply them to future rounds of the simulation.
49
A great deal of work on educational simulations has stressed the importance of

debriefing, but it is a practice which is often given much less attention than its proponents
claim that it should warrant (Boud et al., 1985). Debriefing is a very effective reflection
activity that can be, and is being, used in many course settings. However, it requires
someone who can lead participants discussion, along with participants who can share
their experiences and thoughts. One of the big questions revolves around whether such a
simulation can be used out of a classroom environment and go completely online.
Harvard Business School currently runs a hybrid course in their executive education
program which takes advantage of online meetings to debrief simulation rounds.
Business Simulations as a Disruptive Innovation

According to Rafii and Kampas (2002), most new technologies that come to the
attention of executives will never amount to a real competitive threat and consequently,
deserve to be ignored. As a result, disruptive innovations are usually not taken seriously
by large and/ or successful institution until they become obviouswhen it is too late. A
disruptive innovation is a technology, product, service or process that creeps up from
below an existing business with the potential to displace it. This new innovation
typically offers much lower performance and less functionality at a much lower price.
While the product or process is not good enough to satisfy most customers needs; other
customers welcome the disruptions simplicity. Gradually, the innovation improves to
the point where it has the ability to displace the incumbent by satisfying the markets
need with an acceptable product or service at a much lower price point. According to
Gilbert and Bower (2002), when a company or industry faces a major disruption, the
50
perception of senior management will influence how they respond. For example, if
senior management views the disruption as a threat, managers tend to overreact and
commit too many resources too quickly. However, if senior management sees the
disruption as an opportunity, they are likely to commit insufficient resources to its
development which does not allow for a successful launch. Gilbert and Bower (2002)
argued that thinking of disruptive innovations in such definitive terms threat or
opportunity is dangerous. The authors claimed that the most successful companies
frame the challenge differently at different times: When resources are being allocated,
senior management should see the disruptive innovation as a threat. However, when the
hard strategic work of discovering and responding to new markets begins, the disruptive
innovation should be treated as an opportunity (Gilbert & Bower, 2002).
Christensen (2003) stated that disruptive technologies are not always disruptive to
customers and the process can occur over a long period of time. These types of
innovations are often difficult to recognize and it is often entirely rational for existing
organizations to completely ignore disruptive innovations. The tendency of managers
and leaders to ignore these disruptions is based on how poorly theses new innovations
compare against existing technologies or products. Also, the niche markets available for
a disruptive innovation are often very small compared to the market for the established
technology (Christensen & Raynor, 2003). In the case of business simulations, it is clear
that many educators have dismissed this innovation as irrelevant (Aldrich, 2009b).
It is common for disruptive technologies to be very subtle when they first enter
the market and show very little threat to competing technologies (Christensen & Raynor,
2003). Examples include digital photography, online banking, IP/Internet telephony,
51
mini-mill steel plants, and distance education. The subtle nature of these technologies
showed them initially penetrating a very different market than the competing firms;
however, over time, the technology became better and better, eventually replacing
existing technologies (Christensen, 1997). In a similar way, it appears that business
simulations have been slowly integrated into some University curriculum and corporate
training programs over time with some disruption; however, it is not clear that business
simulations on their own are a disruptive technology.
For the last 100 years, corporate and university education has been delivered via
the lecture method. This method has been modernized recently with the use of both hard
and soft technologies while still relying heavily on lecture. Archer (1999) states that the
hard aspect of this technology consists of a physical classroom while the soft aspect
of the lecture method consists of the learning model framework devised by each
individual instructor. Various sustaining technologies or innovations have refined and
improved lectures with better presentation software and effective methods of collecting
feedback from participants. Online business simulations are a dramatic contrast to this
standard product in that they eliminate the same place feature of the lecture method,
and in many forms of business simulations the same time feature as well.
Demonstrating one of the key features of a disruptive innovation, these online business
simulations may also eliminate the lecturer.
This function could be readily divided among subject matter experts, instructional
designers, and course section tutors (Archer, Garrison & Anderson, 1999). Clearly,
business simulations are a discontinuity in corporate education. These same authors
stated that discontinuity with previous practice is not the defining feature of disruptive
52
innovations (1999, p. 19). Some sustaining technologies have also been discontinuous;
however, there was no major disruption to the industry. Thus, it is necessary to consider
whether online business simulations meet the description of a disruptive technology
based on the following: (1) the technology results in a product or service that has lower
performance relative to the competition and is less profitable to the producer, (2) the
technology is first commercialized in emerging or insignificant markets, and (3) the
leading institutions most profitable customers do not want these products or services due
to lower performance characteristics and consequently these products or services are
initially embraced by the least profitable customers in a market (Christensen, 1997).
1. Are online business simulations easier to implement and lower cost than the
lecture method, and do they promise lower margins? Yes, it is cheaper, if not
necessarily simpler. It is possible for corporate employees to enroll in a business
simulation without ever stepping foot onto a training campus or entering into an
expensive seminar. Also, the cost to a run a business simulation is only a fraction of
normal seminar instruction. For example, a three credit hour business class will typically
cost around $500 per credit hour for a total of $1,500, while access to a robust business
simulation is less than $250. In terms of the non-monetary reward system, creating and
teaching a business simulation is unlikely to earn a corporate trainer the prestige that the
same amount of time invested in research and publication would garner.
2.
Was the technology first commercialized in emerging or insignificant
markets? Yes, online business simulations have addressed small groups of learners
peripheral to the central concern of corporate consulting firms and large universities. The
53
biggest foothold market for business simulations has been found in small junior level
colleges and distance education companies, which focus on nontraditional adult learners.
3. Is it the case that corporate consulting firms most profitable customers
generally do not want, and indeed initially cannot use, products based on business
simulations? Yes, few consulting firms profitable customers (i.e., Fortune 100
Companies) initially accessed online business simulations. Kirby and Garrison (1990)
stated that when it comes to corporate training, many corporate policies which allow for
new types of training are enforced by such gatekeepers as human resource executives.
The initial lack of business simulations at the corporate training level may be due to
limited knowledge of human resource managers, rather than these most profitable
customers not wanting them. Whatever the cause of the initial lack of penetration of
business simulations into this segment of the market, the situation is changing very
rapidly with the recent introduction of new online and distance training programs.
Considering the criteria listed above, online business simulations seem to
correspond relatively well to Christensens description of disruptive technology
(Christensen, 1997). The challenge is determining whether these simulations are truly
disruptive innovations by measuring the effect of these business simulations on
participants today. As business simulations gain a foothold market and evolve over time,
there appears to be three areas of emphasis in order for this technology to become truly
disruptive; (a) there must be stronger pedagogical support for business simulations, (b)
the use of reflection and observation within these simulations needs to significantly
increase, and (c) empirical data on the impact of business simulations needs to be
collected as a form of experiential learning (i.e., increase in business acumen and
54
knowledge application skills). This research study was designed to help evaluate these
areas of emphasis and answer the main question: Do online business simulations provide
an increase in business knowledge or business acumen, which is on the order of a
disruptive innovation?
Business Simulations as a Disruptive Force Today

Business simulations moved first into the management education industry from
the gaming industry about forty years ago. Smith (2007) states there has always been a
strong tie between the technological innovations behind the gaming industry and the
management education industries. At the same time, there has also been a growing
divide between the gaming community and university educators (Anderson & Lawton,
2009; Gosen & Washbush, 2005; Wolfe, 1985). The gamers have dismissed educational
simulations as boring and irrelevant while business management educators have
dismissed gaming and simulations as trivial and pedagogically unproven (Aldrich, 2009a,
xxi). Both appear to be right and yet both may have missed an opportunity which lies
within an engaging business simulation and its potential impact on the world of
management education (Anderson & Lawton, 2009). Disruptions in technology are
which business simulations are used in the future (Faria et al., 2009, p. 485). One of the
major challenges with research in this field is that nobody has shown definitively that
simulation training works in the business world any better than traditional instruction
through workbooks or lectures (Davies, 2003, p. 36).
55
There is strong evidence to support the idea that business simulations have been
slowly moving up the value chain of management education. Davies (2003) argued that
computer simulations provide numerous advantages which can be applied across multiple
industries. Modern day business simulations provide very innovative ways of using
existing technology to foster communication and group collaboration. Improvements in
the delivery of these technologies will allow business simulations to move across
industries by providing significant gains in learning that are lower cost, higher
performance, and eventually more accessible than existing products (Davies, 2003). The
impact of disruptive technological change on an industry can be shown with both the
Product Performance
upper and lower band of performance for the market (see Figure 3).
Market-Years
Figure 3. The Impact of Disruptive Innovation. This graph indicates the difference
between sustaining technologies and disruptive technologies relative to performance at
the high end and low end of the market (Christensen, 1997).
When disruptive technologies enter a market, they not only offer a value
proposition that is attractive to low-end consumers, but these disruptions also allow for
56
growth in new markets. If they are simply sustaining or complementary innovations,

they may offer a value proposition, but they will not cause growth in new markets
(Christensen & Raynor, 2003). As shown in Figure 3, disruptive innovations allow new
customers to move to new products and/ or solutions which change the balance of the
industry. Christensen (1997) also argued that market forces do not operate to maintain
the dominance of existing players, but rather they move to meet the demands of the
maximum number of customers. Disruptive changes in the management education
industry are becoming more and more apparent. Smith (2007) stated that when it comes
to military simulations, research shows that companies, researchers, system developers,
and service providers can choose to adopt the innovation or dismiss it; however, they
Product Performance
cannot dissipate the force of the innovation as it disrupts the entire industry (p. 10).
OneSAF
CCIT
OLIVE
Americas
Army
Spearhead
SIMNET
Market-Years
Figure 4. The Impact of Disruptive Innovation on Military Training. This graph

indicates the difference between sustaining technologies and disruptive technologies
relative to military training (Smith, 2007).
57
Online business simulations provide a technologically powerful and appealing

alternative to current corporate training. The low cost nature of simulations offer a value
proposition that is persistent and one that grows larger as companies look for ways to cut
back on corporate training. One example of this ability for disruption is in the military
training industry where Virtual Trainer and Personal Computer Games started at the lowend of customer demand and increased in quality and market penetration over time
(Smith, 2007). What was initially considered low-end on performance and only a
game grew to become the main source of training all military personnel as shown in
Figure 4. Specific lessons from the military simulation industry which have undergone
tremendous innovation include; poor initial performance, increased visual appeal,
customer pull, quick dismissal, and build capabilities.
If business simulations are entering an industry, leaders must determine whether
to see this innovation as a threat or an opportunity (Gilbert and Bower, 2002). This will
allow management education leaders to either create their own in-house expertise or
develop relationships with smaller business simulation companies (Smith, 2007). There
are a number of business simulation companies (i.e., Vertical Learning Curve, Cesim,
SimuLearn) that have been successful in selling software to Universities, but lack the
skills necessary to apply these simulations to corporate training.
58
Andragocial Support for Business Simulations

One of the primary reasons that stronger empirical research on business
simulations has not emerged is that there is no psychometrically valid instrument to
measure the andragogical constructs (Knowles, Holton & Swanson, 2005). Knowles
actually developed an instrument himself, but it has not been empirically validated.
Knowles, Holton and Swanson explain that this is simply because no instrument to
measure either the assumptions or process elements of andragogy. This explains why
business simulations are often measured by quantitative surveys of faculty regarding how
they feel about the learning gains rather than actually measuring these gains. Empirical
research on the theory cannot advance until there is an instrument that can reliably and
validly measure these constructs in a learning situation (Knowles, Holton & Swanson,
2005). Knowles (1995) primarily advocated that the learning assessment be; (1) mutually
agreed to by the learners and facilitator and (2) performance based rather than traditional
schooling oriented. In this form, it is entirely possible to construct valid research-quality
measures of learning outcomes to conduct a strong test of management education.
Rachal (2002) suggests seven criteria for andragogical empirical studies, including: (1)
voluntary participation, (2) adult status, (3) collaboratively determined objectives, (4)
performance-based assessment of achievement, (5) measuring satisfaction, (6)
appropriate adult learning environment, and (7) random assignment of participants within
the research survey.
The first survey research method involves voluntary participation. The researcher
should examine or design learning situations in which the learner wants to participate for
her own personal fulfillment or some other internal motivator. In this case, business
59
simulations are a potential answer based on the game feel and sense of achievement
obtained from such simulations. Under no circumstances should externally imposed
negative consequences follow for nonparticipation. Knowles himself advocated:
Although it acknowledges that adults will respond to some external motivators a better
job, salary increase, and the like-the andragogical model predicates that the more potent
motivators are internal-self-esteem, recognition, better quality of life, greater selfconfidence, self-actualization, and the like (1984, p. 12). Future researchers in
management education should examine situations such as noncredit continuing education
programs where the great majority of the learners who want to be there, are motivated to
learn the material because it is intrinsically useful to them and are inclined to see the
learning activity as inherently valuable and not as a means to an end.
The second strategy for this kind of research focuses on adult status. Rather than
mixing adult and traditional students as Clark (1991), McMasters (1996), and
Strawbridge (1995) do, future management education studies should avoid college
settings if the various groups being compared are partly comprised of traditional college
students. In keeping with Knowles (1980) view that andragogy is for adults, traditional
college students and adult students should not be mixed within a comparison group. If a
college setting is used and traditional students are part of the study, it is very desirable to
have four groups, including an adult andragogy and an adult pedagogy group. For future
management education research, adult should refer to learners who have assumed the
social and culturally-defined roles characteristic of adulthood and who perceive
themselves to be adult. Rachal suggests that if those qualities are not ascertainable,
learners who are age 25 or older would be regarded as adult irrelevant of social
60
circumstances (2002, p. 220). According to Merriam and Brocketts definition of adult

education, it is activities intentionally engaged in for the purpose of bringing about
learning among those whose age, social roles, or self-perception define them as adults
(1997, p. 8).
The third strategy should focus on collaboratively-determined objectives. When
designing the survey and focusing on management education, it appears to be critical that
the researcher design learning situations in which the learner plays a significant role in
the determination of the learning objectives. For example, a business simulation focused
on How to run a technology business would naturally attract those adult learners whose
objectives are congruent with the course objectives. Where competence is less the goal
than learner satisfaction, the objectives may be entirely defined by the learner(s).
Management education researchers should seek settings in which the learner has a
substantive role in either (a) the planning the activity or (b) the method by which they
will be surveyed in the research.
The fourth strategy should focus on performance-based assessment. When the
purpose of the management education experience is primarily proficiency in a content
area, the research should examine the actual achievement. Based on Knowles (1980)
theory of adult learners, this measure of achievement should be as unlike traditional
schooling and as low-threat as possible (1980). The desirable assessment measure is
demonstration of the ability to perform the learned material through a direct means, such
as actually taking a photograph or entering decisions into a business simulation. Of those
studies examining achievement, both Clark (1991) and Cross (1989) utilized some
performance activity as a measure of achievement. The ideal would be a performance
61
assessment in which the performance is either clearly successful or not successful.

Although Knowles (1980) does not seem to comment on performance tests, he is very
wary of standardized tests and specifically states: tests often smack of childhood
schooling to adult learners, and so should be used with caution and preferably with the
participants full participation in the decision, administration and analysis (p. 213). It
appears that Knowles would rather have an assessment used for the students own
edification regarding relative gains and having the students construct their own test,
either for themselves or their fellow students, either individually or in teams (p. 215).
The researcher may possibly use a test where a large pool of questions is determined by
the learners or the test is taken anonymously, using only group data, and only for the
purpose of examining achievement as it relates to experimentally testing the learners
knowledge of management education.
This would help to diminish the schooling and
anxiety factors Knowles finds incompatible with andragogy (1989).

The fifth strategy should focus on measuring satisfaction. According to Rachal
(2005), many adult education activities do not have as their objective mastery of some
content or acquisition of a skill, but rather the inherent pleasure of participating in a
learning activity. In such settings, the measurement of satisfaction, as part of the survey,
is critical to the researcher. Achievement needs to be measured in those settings where
achievement is the primary objective and satisfaction with the learning experience should
be measured in all settings. Based on a survey by Faria (2004), there is some evidence
which suggests that the variable of satisfaction can be effectively measured along with
the educational activity and other variables of interest.
62
The sixth strategy for future survey work involves appropriate adult learning
environments. According to Knowles (1984), future andragogy studies involving
management education should make every attempt to insure that both the physical and
psychological environments are congruent as possible for adult learning settings.
Physical logistics are critical to the room arrangement as it allows for break out
sessions with smaller groups and creature comforts where a sense of collaboration can
be nurtured. Two memorable and symbolic gestures from Malcolm Knowles were that
(a) his learners call him by his first name and (b) his commentary on papers was done
with a green (never a red) pen (Knowles & Hulda, 1973). From the researchers
perspective, these environmental qualities are perhaps the most difficult to quantify.
Such atmospheric elements are often the result of some constellation of unique
characteristics of the facilitator, for example, friendliness, confidence, content
knowledge, charisma, empathy, humor, expressiveness, enthusiasm, body language,
fairness, respect, kindness, and understanding (Rachal, 1995). Such characteristics are
the Mona Lisas Smile of the instructor of management education. However, the
andragogy researcher should be able to recognize the right atmosphere when he or she
sees it.
The final survey research method recommended from this literature review
involves random assignment of participants. The reality that a group of management
education learners will fall within the range of college undergraduates to senior
executives involved in corporate training is unlikely. Saying that, the idea of in situ
groups where we find learners with similar backgrounds and experiences are the norm
and should be considered acceptable. Secondly, the issue of whether a single instructor
63
conducts both treatments (Anaemena, 1986) or separate instructors each conduct one
treatment (Clark, 1991) is not nearly as clear. One facilitator for both treatments (i.e.,
traditional corporate seminar versus online business simulation) helps assure that the
personality variables do not confuse the outcome based on the assumption that the
instructor does not present himself as Dr. Jekyll for one group of students and Mr. Hyde
for another group of students (Rachal, 2002). However, a single facilitator for both
treatments may exhibit some bias in favor of one treatment, or be inexperienced in one
treatment simply based on their background and experience. By contrast, the use of two
instructors, one for each treatment, invites inevitable differences in personality, rigor, or
experience which might easily be more important than the instructors teaching methods
when it comes to outcome measures such as achievement and especially satisfaction
(Rachal, 2002). Consequently, it is recommended that only one facilitator be used for
both treatments. Rachal (2002) points out that two facilitators may be used in the
research study, but it is critical that both instructors be matched as closely as possible in
terms of such qualities as background, experience, content knowledge and former
teaching evaluations. Other criteria appropriate to experimental and quasi-experimental
research should also be followed: an adequate number of participants (based on statistical
bias), equal and appropriate treatment duration, informed consent, and comparability of
groups (Donavant, 2009).
64
CHAPTER 3. METHODOLOGY
In this chapter, the research design for this study is explained in detail, including a
full justification for each of its elements. After a review of the studys research
questions, this process will begin with a defense of the choice of a quantitative approach,
including its underlying philosophical assumptions. The experimental approach itself
will be comprehensively described. The sample will next be defined, along with the
process of randomly selecting from the sample, the characteristics of the sample, and the
impact on validity and reliability. Next, this chapter will review the measurement
instruments which will be used to collect data. The data collection process will be
described in detail, followed by a stepwise description of the planned data analysis
procedure. With the research design then fully defined, this chapter will next turn to a
discussion of related validity and reliability concerns. Finally, this chapter will end with
a full consideration of ethical concerns related to the study, including steps which will be
taken to minimize specific risks and protect the participants.
Based on the gaps identified in the existing literature in chapter two, the purpose
of this quantitative, experimentally-based research study is to investigate the use of
training. This experimental research allowed for a direct comparison between three
groups and identified a causal relationship between online business simulations and
experiential learning as a possible source of disruption. This study therefore connects
experiential learning theory from psychology with the developing field of disruptive
65
innovation, in order to support the business case for significantly higher levels of adult
learning through the use of online business simulations.
In order to examine the effectiveness of engaging adult learners in an online
business simulation, this experimental design study focused on the change in business
knowledge application skills of participants based on exposure to the simulation. This
research study evaluated the gains in business knowledge and business acumen to help
determine whether online business simulations are a sustaining or disruptive innovation
in comparison to more traditional corporate training methods. This study further
attempted to identify and understand the impact of online business simulations on worker
knowledge based on individual backgrounds, work experience, level of education and
tenure with the company. These research priorities were considered with respect to two
potential moderating variables, including level of autonomy in the company and previous
simulation experience. Previous simulation experience was a dichotomous covariate
variable in this studys multivariate analysis of covariance (MANCOVA) analysis, while
perceived autonomy (due to different levels of management) will be controlled for as a
continuous covariate.
This forms the studys primary research question:
Do online business simulations provide an increase in business knowledge

or business acumen for participants, which is on the order of a disruptive
innovation?
In order to provide supporting evidence that the business simulation and

traditional training methods are impacting knowledge application skills, the level of
business knowledge and business acumen of each participant was measured before and
66
after the intervention. This not only served to further validate the experimental design,
but also allowed investigation of the relationship between business knowledge and
business acumen. While the primary research question deals with the potential level of
disruption that comes from experiential learning, the secondary research questions are
relational:
Are knowledge application skills in business positively correlated with

How does the change in knowledge application skills with traditional

Two of the dependent variables considered in the primary research question for this
studychange in business knowledge and change in business acumenwere analyzed
using multivariate analysis of covariance (MANCOVA) analysis. MANCOVA is
defined as an extension of analysis of covariance (ANCOVA) methods which are
designed to cover cases where there is more than one dependent variable and where these
two dependent variables cannot be combined (Swanson & Holton, 2005, p. 135). This is
similar to multiple analysis of variance (MANOVA); however, MANCOVA also allows
for control of the covariates (Cooper, 2008). Covariates are usually the variables in
experimental design which are not controlled by the experiment. However, these
covariates still have an effect on the dependent variables. In this research study, the
covariates include previous simulation experience (dichotomous covariate) and work
control (continuous covariate). The relationship between business knowledge application
skills and online business simulation performance, the second research question, was
investigated using correlation analysis. Finally, the differences between experimental
groups (case study, business simulation, and control) were analyzed using a simple means
67
test. The details of the research design and data analysis procedure will be discussed in
later sections of this chapter. Based on the use of these statistical procedures, the
research hypotheses are formed. Figure 5 displays the form of the linear model which
was used to investigate the studys primary research question.
General Linear Model for Primary Research Question

Equation:
Y = mX + b
Where:
Y = Vector of dependent variables

Business Acumen
Business Knowledge
X = Design matrix of explanatory variables
Pedagogical Nature of Training (Experimental)
Previous Simulation Experience (Dichotomous Covariate)
Autonomy/ Work Control (Continuous covariate)
m = Slope
b = Error term
Figure 5. General Linear Model for Primary Research Question. This equation
describes the general linear model where Y is the vector composed of dependent
variables and X is the design matrix of explanatory variables.
The specific hypotheses examined in this study are presented in Table 3 as both
null and alternative hypotheses. Hypothesis one reflects the primary research question as
to whether online business simulations provide gains in business knowledge which are on
the order of a disruptive innovation.
68
Table 3
Null and Alternative Hypotheses
Null Hypothesis
There is NO difference across
experimental groups for business
knowledge after adjusting for previous
simulation experience and autonomy/
work control.
H2 There is NO difference across
experimental groups for business
acumen after adjusting for previous
simulation experience and
autonomy/work control.
H3 NO correlation exists between
participants business acumen and
business knowledge.
H4 Participants who engage in an online
business simulation will NOT
demonstrate higher business
knowledge after experiencing the
online business simulation.
H5 Participants who engage in an online
business simulation will NOT
demonstrate higher business acumen
after experiencing online the business
simulation.
H6 There is NO difference between the
simulation group, the case study
group, and the control group based on
their business knowledge.
H7 There is NO difference between the
simulation group, the case study
group, and the control group based on
their business acumen.
participants business simulation
performance and their level of
autonomy in the company.
performance and their years of
industry experience.
performance and their level of
education.
H1
69
Alternative Hypothesis
There is a difference across experimental
groups for business knowledge after
adjusting for previous simulation
experience and autonomy/ work control.
There is a difference across experimental
groups for business acumen after
adjusting for previous simulation
experience and autonomy/work control.
A correlation exists between participants
business acumen and business
knowledge.
Participants who engage in an online
business simulation will demonstrate
higher business knowledge after
experiencing the online business
simulation.
Participants who engage in an online
business simulation will demonstrate
higher business acumen after
experiencing the online business
simulation.
There is a difference between the
simulation group, the case study group,
and the control group based on their
business knowledge.
There is a difference between the
simulation group, the case study group,
and the control group based on their
business acumen.
business simulation performance and
their level of autonomy/work control in
the company.
their years of industry experience.
their level of education.
Research Design
In Chapter 2, a need was identified for empirical research into the potential
disruptive nature of online business simulations when compared with more traditional
instruction in the field of management education (Segon & Booth, 2009). Based on this
research, this study employed an experimental design and pre/post-experiment survey.
Justification of Methodology
This study utilized a quantitative experimental research methodology.
Quantitative techniques are particularly strong at studying large groups of people and
making generalizations from the ample being studied to broader groups beyond the
sample (Swanson & Holton, 2005). Cohen (1994) asserted that quantitative techniques
must be used sensibly and be heavily informed by judgment (p. 1002). A quantitative
approach is typically employed when the research goal is to study relationships between
established variables in an objective manner (Creswell, 2009). This is the case for this
research, which will examine relationships between variables. Research has shown that
participants perceive online business simulations as an effective training tool (Burns et
al., 1990; Faria, 2004; Gosen et al., 2004). However, all of these studies relied upon
perceptions and self-reports of learning rather than more objective measures (Anderson &
Lawton, 2009, p. 211). There is a dearth of studies employing experimental designs with
control groups and almost no experimental studies exist that compare learning outcomes
under alternative pedagogies.
70
Wolfe (1990) identified this problem over 20 years ago, yet the gap still exists.
Researchers continue to use self-assessments rather than more suitable tools because they
are much easier to employ. As a consequence, studies on the educational merits of
simulations often are measuring the affective domain, not the cognitive domain they
purport to measure (Anderson & Lawton, 2009, p. 197). This has led some researchers to
believe that using simulations is a powerful and disruptive form of learning because it is
taking place at the higher levels of Blooms taxonomy (Smith, 2006). While continued
exploration of online business simulations as an abstract phenomenon is needed, the goal
of this research work is not explorative, but rather consequential. Quantitative
approaches are most appropriate when the target population is large and there is a desire
for generalization beyond the studys sample, the ability to predict future results, or the
capacity to credibly establish causation (Swanson & Holton, 2005). Quantitative
techniques are particular well suited for experimental methods where the environment is
well controlled against outside influences. While this limits generalizability, it also
maximizes internal validity (Creswell, 2009).
Research Design Strategy

A true experimental study was used to investigate the change in business
knowledge and business acumen by exposing two experimental groups (business
simulation and traditional instruction) to one or more treatment conditions and comparing
the results to a control group not receiving the treatment.
71
Population
Random
Sample
Treatment
Measure
Traditional Instruction
Online Business Simulation

Gender
Experience
Autonomy
Education
Tenure
Control Group
Change in
Business Knowledge
(Theory)
Change in
Business Acumen
(Application)
Figure 6. Research Design Schematic. This schematic identifies the random sampling
from a company population for three experimental groups (traditional instruction, online
business simulation instruction, and the control group.
A sample of 65 participants was randomly selected from a semiconductor

company population of 720 employees and managers. These participants were then
randomly organized into three experimental groups as shown in Figure 6. Each
experimental group had a total of 20 to 23 participants and each participant was asked to
take the pre-test and survey before the training began.
The Traditional Instruction group was given a copy of a Harvard Business Review
article on the subject of strategy and execution to read on the first day of corporate
training. These participants were also given a Harvard case study dealing with strategy
which was discussed during a training class on the second day. Each participant was
asked to read the business article and come to class prepared to discuss the case. During
the case discussion, specific prompt questions were asked regarding strategy and
72
execution. On the third day, each participant was allowed to take post-test assessment in
order to evaluate the change in business knowledge and business acumen.
The Online Business Simulation group was introduced to the simulation and then
broken into smaller teams of 5 members each. These individual teams were allowed
several rehearsal rounds of the simulation where they could practice making decisions
for the company. Over the next two days, these teams competed with each other for
market share and highest level of stock price as they implemented specific strategies and
tactics that they had decided upon as a team. Each team ran a total of 5 rounds,
simulating 5 years of running a company. In between each round, a debrief meeting was
held where participants could ask questions and learn more about the competitive nature
of the industry. The first two days of training allowed participants to make decisions in
five different rounds, simulating five years of running the company. On the third day,
each participant in the online business simulation training group was allowed to take the
Comp-XM assessment in order to evaluate the change in business knowledge and

business acumen as shown in Figure 7.
73
Prompt Questions
Strategy Only
Traditional
Instruction
HBS Review
Article
Discussion
Case Study
Discussion
Prompt Questions
Strategy & Application
Business
Simulation
Pre
- Simulation
Survey &
Knowledge Test
Practice
Rounds
Round 3
Round 4
Round 5
Post -Simulation
Survey &
Knowledge Test
No Reflection
Activity
No Intervention for the Control Group

Control
Group
A pre/post -test control group design will be used as the choice of research design for this study.
Participants will be randomly assigned to the Case Study, Business Simulation, and Control Group
Figure 7. Overview of the Research Design. A pre/post-test control group was used as the choice of experimental research
design for the study. Participants were randomly assigned to the Traditional Instruction (Case Study) Group, the Online Business
Simulation Group, and the Control Group.
74
The Control Group did not have any intervention and was simply allowed to take
the pre-test on the first day of training and the assessment on the last day of training.
This research design is unique in the business simulation literature. A significant
amount of quantitative and empirical research was collected which can be analyzed for
both comparisons across experimental groups and measured to determine the significance
in the change of business knowledge and business acumen based on the type of training
given in a corporate setting. Previously to this study, all empirical research had relied on
the participants perception of their learning experience (Williams, 2011). Utilizing an
experimental method which allowed for the clear establishment of increased learning
through different pedagogies, this research design clearly demonstrated the gains in
business knowledge and business acumen based on modern day corporate training. This
empirically based research study is a significant contribution to the understanding of the
magnitude of disruption that online business simulations can have on management
education today.
Sample
The most common method to calculate appropriate sample size is based on
calculating the necessary sample in order to achieve a desired power, effect size, and
level of statistical significance (Lenth, 2001). Power can be simply described as the
ability of a statistical test to detect a particular effect size. Lewis-Beck (1994) also
describes power as the probability of correctly rejecting a false null hypothesis given that
an effect size is present in the sample (p. 1080). Effect size is defined as an estimate of
the relationship between two variables. It can take a number of forms, but it is frequently
75
expressed as a standardized ratio between the magnitude of an observed effect and the
standard deviation (Fields, 2009). Selecting the appropriate effect size of interest prior to
conducting a scientific study can be challenging. The best method is to use prior research
as a guide (Lenth, 2001). Using the G3 Power software tool, (Faul et al., 2009) a power
analysis showed that a sample as small as 14 for three treatment groups will be able to
detect a MANCOVA main effect size of one standard deviation, with a power of 80%
and a statistical significance of 95%. At 90% power, a sample size of 17 is required
while 80% power is commonly accepted in social science research (Fields, 2009). These
calculations indicate that this research study with 20 to 23 participants per experimental
group was sufficient. It was determined that even if the effect size had fallen as low as
0.5 standard deviations, a sample size of 20 would be able to reject the null hypothesis at
95% significance with 90% power.
The population of interest for this study was managers and employees for a large
semiconductor manufacturing company. This experiment sought out a specific causeand-effect relationship between the type of pedagogical instruction offered and resulting
change in business knowledge and business acumen from the training. A more than two
standard deviation increase in the mean scores indicates a source of disruptive
innovation. However, this study is careful to state that such results are not able to make
statistical generalizations beyond the sample (Cooper & Schindler, 2008). The primary
goal of this study was to investigate the use of online business simulations as a disruptive
technology by measuring the change in business knowledge and business acumen
compared to traditional corporate training. Focusing on this critical population insured
that any conclusions, while limited in generalizability, will have the greatest possible
76
value. The focus on a large manufacturing company with random sampling allowed for a
population consisting of employed professionals, knowledge workers, and managers who
are typical of corporations seeking quality operations training today.
In order to access the targeted population, this researcher contacted the Vice
President of Operations for a large semiconductor company and requested permission to
provide corporate training for approximately 75 members of the senior management
team. Preliminary approval for this research study was received from the company in
advance. Once approval was received through the IRB approval process, a meeting was
held with senior management at the company to discuss the purpose and structure of the
research study. The corporate headquarters for this semiconductor company were
provided with an executive summary of the results as well as a presentation of
information and resources that could aid strategic plans for training senior management in
the future. Based on an initial target of 25 participants per group, a total of 65
participants were randomly selected and found available time to participate in the study.
This total of 65 is out of a population of 720 employees and managers.
This study employed a randomized cluster sampling technique in order to select
participants from the organization. Cluster sampling may result in less statistical
precision, but it is more economical than simple random sampling for large populations
(Cooper & Schindler, 2008, p. 392). Within the population of interest for this study,
departments within the semiconductor company form natural clusters (e.g., research &
development, production, finance, and marketing). Subgroups were created based on the
background and experience of each participant and participants were then randomly
selected for one of three experimental groups. Based on the parameters for each
77
participant it was possible to ensure that all three experimental groups were similar in
background, experience, and autonomy. Employing three groups with 20 to 23
participants per group was sufficient to achieve the desired level of statistical power.
The population of clusters for this study is all of the employees and managers of
the semiconductor company. According to its business profile, there are 14,307
personnel employed by this company around the world. From this population, a list of
potential participants were be randomly generated by the human resource department at
one manufacturing site. Random assignment of these participants to each experimental
group was necessary in order to insure the sample group was as homogenous as possible,
and therefore comparable with regard to the studys dependent variables. Issac and
Michael (1997) stated that the best solution for true experimental design is random
assignment of subjects to groups and random assignment of experimental treatments to
groups. Utilizing a randomization technique with a control group forms the classic R-OX-O true experimental design, maximizing internal validity and relative efficiency
(Creswell, 2009). All three groups were administered both a pre-test and post-test, but
the treatment was only provided to two of experimental groups (e.g., traditional
instruction and online business simulation).
Setting
A number of studies exist which have looked at the educational effectiveness of
business simulations in corporate settings (Burns, Gentry, & Wolfe, 1990; Faria, 2001;
Gosen & Washbush, 2004). However, none of these studies attempted to empirically
measure the change in business acumen and knowledge application skills of the
78
participants across different experimental groups (Sidor, 2008). This study was unique in
that it investigated the effects of engaging participants in different pedagogies of learning
and different methods of reflection as participants step back from the case study or
business simulation and consider what they have just learned in order to apply this new
knowledge to the post-test assessment. When it comes to investigating the magnitude of
learning based on different pedagogies as a possible disruptive innovation to management
education, this study was the first in its field.
While this experimental study added to the body of knowledge on experiential
learning, online business simulations, and disruptive innovation, there were also benefits
for the semiconductor company and participants involved in this study. The use of online
business simulations in the corporate training and management education is quickly
growing (Williams, 2011). However, this type of training is typically very expensive.
Fortune 500 companies are willing to pay as much as $50,000 per training session for top
managers and employees to attend. In the case of this research study, the semiconductor
company paid a small fraction of the normal cost. Participants in this study gained
significant insights into their business and how to better manage their manufacturing
facilities. This study confirmed that adults working in a corporate environment prefer
engagement and the opportunity to experience concrete action as the basis for learning,
rather than being lectured in a classroom (Segon & Booth, 2009). The senior
management team of the semiconductor company perceived this as a great learning
opportunity for their employees and the company as a whole.
79
Instrumentation and Measures

Quantitative techniques necessitate narrowly defined hypotheses and the ability to
collect data which can be reasonably used to test these hypotheses. This requires valid
and reliable measurement instruments (Creswell, 2009). This section will fully describe
each of the instruments which was used to facilitate this research investigation. The
validity and reliability implications of the research design selected for this study will also
be reviewed in the next section.
Both business knowledge and business acumen have proven very difficult to
measure (Segon & Booth, 2009). Gosling and Mintzberg (2004) state that management
is neither a science nor a profession, neither a function nor combination of functions (p.
10). Burns (1995) identified simulations as a mock up of real situations that are generally
far more complex than role-plays; however, methods of measuring the impact of these
simulations has been elusive (p. 284). Consequently, this study will employ three
methods of measurement; (1) a pre-test and post-test designed to assess the participants
business knowledge and business acumen, (2) participants responses to their level of
autonomy or work control within the company, and (3) business simulation performance
scores based on the Foundation business simulation.
Capsims Comp-XM assessment tool was used to measure both business

knowledge and business acumen as the pre-test and post-test for the simulation. All
decisions and scores were recorded in Capsims online database and can be shown in a
scatter plot for each individual participant including standard deviations from the mean
for this population as well as other participant scores (see Figure 8). The Comp-XM
80
assessment tool has been used by over 50,000 MBA students and is used as an instrument
for online computer usage, locus of control, and literacy (Prince, 2004).
Figure 8. Comp-XM Comparative Standings. This scatter-plot shows participant

scores based on business acumen and business knowledge relative to 12,604 peers who
have taken the same assessment exam.
With the exception of two demographic factors, all of the variables included in
this study were measured by survey instruments designed for their specific uses.
Previous simulation experience was considered a potential moderator in the casual
relationship between business knowledge and business acumen. Simulation experience
was measured by a single question on the post-experiment survey, with a dichotomous
choice: (1) yes or (2) no. In addition, data was collected on industry experience, purely
as a descriptive variable to help understand the composition of the sample. Industry
experience was divided into ten ordinal ranked categories, each representing five years:
81
(1) 1 to 4, (2) 5 to 9, (3) 10 to 14, (4) 15 to 19, (5) 20 to 24, (6) 25 to 30, (7) 31 to 34, (8)
35 to 40, (9) 41 to 44, and (10) 45 years or more. These variables and survey items are
common and have been used in previous studies dealing with simulations (Segon &
Booth, 2009; Williams, 2011).
The Business Simulation
An online computer-based simulation, titled Foundation (Management

Simulation Inc., 2012) was chosen to explore whether online business simulations are a
sustaining or disruptive technology in management education. Foundation is a

moderately complex business simulation, where participants are allowed to run a
company in a high tech manufacturing industry. A larger company was broken up by the
Security and Exchange Commission (SEC) resulting in six smaller companies which are
each generating approximately $50 million in sales. The companies are poorly
positioned with outdated products, over capacity, and unsatisfied customers. Participants
in the simulation are allowed to make changes in research and development (R&D),
marketing, operations, human resources, and finances; however, there are very limited
resources based on available cash. The simulation compresses time, allowing
participants to quickly see the results of their actions and better understand how
management decisions impact the company on an operational, market, and financial
basis. While most simulations offer a single perspective of a business which is engaging
for a few hours, this particular simulation can be run for hundreds of hours and still never
quite mastered, based on the multiple strategies available and the very real impact of
competitive strategies from other participants who are also involved in the same
82
simulation. As the CEO of a simulated company, participants make decisions in an

increasingly complex environment as products are launched in two different markets
which have constantly changing requirements based on the customers expectations over
time. The decisions required in this simulation are very similar to what the management
and employees at the semiconductor company experience in their everyday jobs.
The decisions that participants make during each year (or round) of the simulation
center around the prior years operational results and inputting allocations to R&D,
changes to existing products, creation of new products, promotional marketing, sales
budgets, production capacity, automation, overtime, staffing, training of personnel,
recruiting budget, initiates around total quality management (TQM), dividend policy,
issuing or retiring of stock, and determining how to fund the company. Each decision-set
in the simulation requires approximately 50 smaller decisions in each of the functional
areas of the company.
Participants were able to see the long-term implications of their day-to-day
operating choices and receive various types of feedback from the Courier or
operational report which is released at the end of each year (or round). At pre-determined
times, the simulation offers participants the opportunity to review the Courier and reflect
on the impact of their decisions and plan out future decisions. Resources in the
simulation include articles, video clips, planning spreadsheets, and templates which offer
a wealth of insight around the areas critical to the success of a company which is
attempting to change business models and become profitable again in a very competitive
industry. The financial statements which are included in the Courier each year (or round)
of the simulation allow the participants to see the cash move through the Income
83
Statement and to the Balance Sheet. Specific questions from the Board of Directors are
asked at the end of each year, which gives an opportunity for the participant to study the
financial statement and provide answers based on their knowledge (i.e., knowledge
application skills). The simulation also tracks the overall quality of the decisions
throughout the simulation which completes a Balanced Scorecard at the end of each year.
It is possible for a participant to score well on the balanced scorecard once or twice;
however, it is very difficult to maintain a good score throughout the duration of the
simulation without adhering to solid business strategy (i.e., business acumen).
Dwyer and Gansters (1991) Autonomy/ Work Control

This measure, developed by Dwyer and Ganster (1991), describes the extent to
which workers perceive they have control over numerous aspects of their day to day
decisions. These aspects include control over the variety of tasks performed, the order of
task performance, the pace of tasks, task scheduling, task procedures, and the
arrangement of their work environment. Musselwhite (2006) highlights that simulations
allow people to learn by doing, consistent with adult learning principles; however, these
individuals must have a certain level of autonomy to apply what they have learned. He
cites Agyris (1974) discovery learning that promotes double loop learning that in turn
promotes behavioral changes. Musslewhite refers to the a-ha or gestalt moment that is
produced during debriefing sessions that make simulations so effective. One of the key
purposes for measuring work control was to identify and separate those employees who
have limited autonomy from managers who have much more control over their work,
scheduling and environment.
84
Dwyer and Gansters (1991) instrument consists of twenty-two Likert-scaled

items where each participant is asked to indicate the extent to which each item is an
accurate or inaccurate description of their job by circling a number in front of each
statement. The response options are 1 = very little, 2 = little, 3 = a moderate amount, 4 =
much, and 5 = very much. Control over aspects of a job in this instrument correlate
positively with job workload, work satisfaction, and level of responsibility within the
company. Based on a sample of 191 white-collar workers in a manufacturing facility,
these researchers calculated a standardized Cronbachs alpha of 0.87, well above the
accepted limits (Rubin & Babbie, 2005).
Field Testing
Because this studys survey instruments were derived from previously validated
items, a distinct field test was not included in this research study.
Business Simulation Performance Scores
Foundation , the Capsim Business Simulation, is an assessment tool used to

determine what participants have learned through the business simulation experience
(Smith, 2011). The unique two-part Comp-XM examination process uses a business
simulation with a built-in balanced scorecard, followed by a series of board query
questions that are specific to the results of the answers provided in the prior simulation
round. The overall score for performance on the simulation is based on four components;
financial, internal business process, customer, learning & growth of the company. Each
of these areas are evaluated and totaled for a final score (see Figure 9).
85
Figure 9. Foundation Business Simulation Performance Score. This table lists the
four primary areas of the balanced scorecard based on the participants decisions in
finance, internal business process, customer, and learning/growth for the workforce.
Data Collection
This research was conducted at a large semiconductor company which has
multiple manufacturing facilities around the world. One site was selected with a total of
720 employees and managers and randomly selected participants met in a conference
room at the facility where corporate training takes place. Data was collected from
participants in two ways. The first was through management decision software based on
a pre/post-test using Capsims Foundation and Comp-XM software. The second

method for data collection was through an online survey which was distributed to each
participant immediately following the experiment. This survey captured information
demographic variables including, background, previous simulation experience, years of
industry experience, level of education, and area of expertise. This survey was also used
to measure the level of autonomy/ work control for each participant.
86
Variables
Characteristics of this experimental design required rigorous management of
experimental variables and conditions by direct control/manipulation or through
randomization (Isaac & Michael, 1997). The independent variables are participation in
an online business simulation or participation in traditional instruction. There was also a
control group which did not participate in either. The dependent variables are the change
in business knowledge and change in business acumen based on the pre/post-test for each
group. The level of autonomy of each participant is an independent mediating variable
which helps to identify employees from managers with more responsibility. The intent of
this study was to make 10 predictions about the relationships among the variables
identified in the null Hypotheses. A summary of all variables is found in Table 1 (see
page 21).
As participants completed the Capsim Comp-XM assessment they were asked to

complete a short survey. This allowed survey results to be matched with corresponding
results from Comp-XM which measure both business knowledge and business acumen.
Each survey was later marked with the number identifying individual assessment results,
along with the type of pedagogical instruction that was given over the three day period.
These survey results were then be transferred into a secure database manually.
Data Analysis
This studys primary research question and objective was focused on two
dependent variables: business knowledge and business acumen. These dependent
87
variables are assumed to have continuous distributions. The studys experimental

independent variable is categorical (e.g., case study, business simulation, and control),
while the level of autonomy for each participant is independent mediating. In addition to
these variables there are also descriptive variables for each participant including:
previous simulation experience, area of business expertise, years of industry experience,
and level of education. Given multiple continuous dependent variables, a categorical
experimental independent variable, plus the existence of potential covariates, the
appropriate statistical technique is multivariate analysis of covariance or MANCOVA
(Swanson & Holton, 2005, p. 135). MANCOVA analyzes multiple dependent variables
as a vector, after the effects of covariates are statistically removed. Analysis of variance
or ANOVA can be used as a follow-up technique to determine the extent means differ
between groups. A two standard deviation change in the mean may indicate a disruptive
innovation (Hake, 1998).
For ease of manipulation, the raw data from the pre/post-test experiment and
survey was first entered into a Microsoft Excel 2011 spreadsheet. This also allowed the
survey results to be quickly tabulated according to the developers instructions for each
scale. For the Comp-XM
assessment tool, the business knowledge and business
acumen scores were entered. For Dwyer and Gansters (1991) measure on the level of
autonomy or work control for each participant, a simple arithmetic mean is calculated.
The final performance simulation score based on the balanced scorecard for the virtual
company will also be entered for the online business simulation experimental group.
Data analysis was conducted with version 20.0 of the SPSS software. After the
data was tabulated, the data analysis process began with a thorough graphical analysis.
88
This allowed for the quick identification of any spurious data, yield information about the
distribution of the data set, and gave some guidance as to how the investigation should
proceed (Norusis, 2008). The next phase of data analysis was descriptive analysis,
starting with the calculation of univariate statistics for each variable. The distribution of
each continuous variable was then examined by calculating skewnees and kurtosis. This
analysis also served to highlight any outliers required further investigation (Norusis,
2008). Cronbachs alpha values were calculated for all scaled factors in the survey as a
check of acceptable internal consistency and reliability.
Analysis of variance methods, such as MANCOVA, are often considered an
extension of regression techniques which can be conceptualized as a multiple linear
regression with multiple dependent variables. Consequently, MANCOVA shares many
of the same assumptions as other general linear models (Norusis, 2008). These
assumptions were examined for the following; (1) independence of observations, (2)
random sampling from the population, (3) independence of the covariates and treatment
effects, (4) multivariate normality, (5) homogeneity of covariance matrices, and (6)
homogeneity of covariate regression slopes (Fields, 2009).
Independence of observations, random sampling, and independence of the
covariates across treatments are a function of the experimental procedure (Norusis,
2008). These assumptions can be ensured by random selection of experimental groups
and by utilizing a meticulously consistent experimental process. To check the
assumption of the independence of covariates across treatments, a simple analysis of
variance was run across the experimental groups in which the covariates act as the
dependent outcome variables. Because the main effects of these ANOVA analysis were
89
not significant, it was assumed that the covariates are roughly equal across the
experimental groups (Fields, 2009).
Univariate normality was verified through normality plots of the variables and by
conducting normality tests (e.g., Kolmogorov-Smirnov and Shaprio-Wilk). Fortunately,
analysis of variance is not very sensitive to normality. Because the data are not
extremely non-normally distributed and the sample sizes were not particularly small,
more than 17 each, it was considered safe to proceed with the analysis (Norisus, 2008).
The assumption of homogeneity of covariance matrices means that not only
should the variances between each experimental group be roughly equal for each
dependent variable, but also that the correlation between dependent variables is equal
from group to group. To check this assumption, first Levenes test was conducted on
each of the dependent variables in order to insure there is univariate equality of variances
between groups. Second, Boxs test was used to compare the variance-covariance
matrices for each group (Fields, 2009).
Finally, the assumption of homogeneity of covariate regression slopes was
investigated. MANCOVA relies on the assumption that the relationship between each
dependent variable and the covariates is consistent across groups. The assumption was
tested by calculating the significance of the interaction effects between the covariates and
the primary independent variable. If this interaction is statistically significant, the
assumption of homogeneity of covariate regression slopes is suspect (Fields, 2009).
Once the assumptions were verified, the MANCOVA analysis began. This
research study was tested by examining the significance of the F statistics for the
independent variable and for each covariate in the ANOVA tables. A significance level
90
below the critical value of 0.05 allowed for the null hypothesis to be rejected.
Confidence intervals and contrast matrices were also built to further evaluate statistical
results. The original research method allowed for other statistical methods to be
employed. If the assumptions of MANCOVA were substantially violated, techniques to
transform the data were available (Fields, 2009). In addition, robust methods which are
less susceptible to the requirements of traditional MANCOVA analysis, such as the
Munzel-Brunner method, were also available (Wilcox, 2005).
The second research question involving the degree of the relationship between
knowledge application skills and online business simulation performance was considered
using correlation analysis. The magnitude and direction of the Pearson correlation
coefficients served to describe the relationship between the variables. If the Pearson
correlation coefficients were found to be statistically significant, with critical values
below 0.05, then the null hypothesis was rejected.
The third research question involved the change in knowledge application skills
when comparing traditional corporate training with the use of online business
simulations. This investigation was intended to provide additional evidence that the
change in business knowledge and business acumen over the same period of time is
significantly different based on the type of pedagogical training that is given to adult
learners. This portion of the analysis was accomplished with simple comparisons of
means and T-tests. Business knowledge and business acumen levels were compared
across the three different experimental groups. Statistically significant results, with a
critical value below 0.05, indicated manipulation of the independent treatment variable
was related to changes in participants overall level of business knowledge and acumen.
91
Validity and Reliability

A central concern of any scientific effort is validity and reliability. Validity has to
do with whether a study measures the construct intended. Reliability is the ability to
maintain consistent results. Reliability is a necessary condition for validity; however, it
is not the only condition. A measurement can be reliable without being valid, but validity
cannot be obtained without reliability (Holton & Burnett, 2005). Validity is commonly
subdivided into internal and external validity. Internal validity is the ability to correctly
draw conclusions from the research data. External validity is the ability to draw
inferences in order to generalize research results to the population or to other settings
(Cooper & Schindler, 2008).
The use of an experimental design with randomized treatment and control groups
maximizes internal validity (Rubin & Babbie, 2005). However, Creswell (2009) lists
eleven different types of threats to internal validity which must be addressed by any
experimental research design: (1) History, (2) Maturation, (3) Regression, (4) Selection,
(5) Mortality, (6) Treatment diffusion, (7) Compensatory demoralization, (8)
Compensatory rivalry, (9) Testing, and (10) Instrumentation.
History and maturation threats arise due to the passage of time and events which
might occur during the experiment. Similarly, mortality threats arise when the
composition of study groups changes over time (Cooper & Schindler, 2008). In this
study these threats were abated by tightly controlling the experimental settings, and by
conducting the experiment at a single point in time. The threat of statistical regression
arises when participants with extreme scores are intentionally selected for study, which
will not be the case in this research. As discussed previously, the threat of selection bias
92
was addressed by randomizing which participants were selected for the control and
experimental groups (Creswell, 2009). The use of a classic R-O-X-O experimental
design with random assignment of groups is an effective method to control the risks of
history, maturation, selection, and statistical regression (Cooper & Schindler, 2008).
Treatment diffusion occurs when participants in the control and treatment groups
speak with one another. While it was impossible to prevent participants from
communicating, this risk was minimized by conducting the experiments over a short time
frame (3 days), and by asking the participants to refrain from discussing their experiences
until the results are revealed. Compensatory demoralization or rivalry is risked when the
treatment and control groups receive different benefits. After the study was complete, the
researcher returned to the participants classrooms to reveal and discuss the results,
reestablishing equality. Pre-testing was not a threat to this study since participants were
allowed to participate in the experiment once, and the survey was administered at the end
of the experimental activity (Creswell, 2009).
The final threat to internal validity cited by Creswell (2009) is instrumentation.
The Capsim Comp-XM instrument was used to assess business knowledge and business
acumen. Quantitative methods were confined to only that which is measurable and
controllable (Malterud, 2001), which has been an area of much debate in business
simulation literature (Sidor, 2008). Engineered quantitative approaches may strip context
and meaning from phenomena in an attempt to study them objectively. It is important,
particularly with abstract subject matter, to insure the research design does not impose
meaning on observations rather than simply observe them (Swanson, 2005). Steps which
93
have been taken by the developers of the measurement instruments which were used in
this research to insure validity and reliability were previously discussed in this chapter.
The second class of validity threats is dangers to external validity. Laboratory
experiments maximize internal validity because they have the best chance of controlling
extraneous variables. However, the price of maximum internal validity is
generalizability (Echambadi et al., 2006). The artificiality of experimental settings risks
outcomes which may not exist in natural surroundings (Cooper & Schindler, 2008). The
simulated workplace which is being created for the purposes of the experiment is very
similar to the actual workplace for these managers and employees in the semiconductor
industry. Any claims made about the results of the research should be restricted to
groups which have similar characteristics to the experimental sample (Creswell, 2009).
Since the researcher had to directly interact with the participants during the experimental
activity in order to give instructions, another risk to external validity is experimenter
effects. Conscious or unconscious acts by the researcher could influence results (Gay &
Airasian, 2002). In order to minimize this risk, instructions were carefully scripted and
contact with the participants was limited.
Reliability is a matter of consistency or repeatability of measurements (Creswell,
2009). Low reliability can dramatically impact a researchers ability to uncover theorized
relationships between variables. The most commonly used indicator of reliability is
Cronbachs alpha value, which indicates the degree of internal consistency between
measurements. With survey research, internal consistency means items draw on the same
construct and have homogeneous results. In social science, Cronbach alpha values
exceeding 0.70 are generally accepted as sufficient for research (Norusis, 2008). The
94
Capsim instruments used in this research study were analyzed using psychometric
measures which the computing environment has used for many years (Prince, 2004).
Kay (1990) developed a 10-item instrument to assess computer locus of control that was
used by Prince to evaluate the Capstone business simulation. In both studies, the
Chronbach alpha was above 0.8 with Kays analysis reporting an alpha = 0.87 and
Princes study reporting an alpha = 0.88 (Prince, 2004).
There are several sources of reliability and validity threats specific to surveys
which must be considered. Survey methods rely on a communications process to capture
participant feelings and attitudes. A breakdown in this process could occur due to the
interviewer, the measurement instrument, or the participant (Creswell, 2009). With selfadministered surveys, such as in this study, the potential influence of the interviewer is
minimized. The validity and reliability of each of the measurement instruments which
was used by this study has been previously established. Error generated by the
participant, however, can come from either a lack of willingness to respond or from
mistakes in how a participant responds (Cooper & Schindler, 2008).
The reliability and validity of survey instruments is dependent on participants
who are willing to provide accurate and complete responses to the questions posed to
them. This can be a challenge, particularly for certain sensitive subject matters such as
religion, finances, or health. Participants may choose not to respond to a particular
questionnaire item at all, creating non-response bias (Cooper & Schindler, 2008). To
moderate this risk, the survey which was used in this study intentionally avoided such
sensitive topics as income, even though this could be theorized as a potential moderator
variable. In addition, the participants were assured of the strict confidentiality of their
95
responses in order to help allay any fears. Steps were taken to handle any non-responses
which may occur were outlined in the data analysis section of this chapter.
Unfortunately, surveys similar to Dwyer and Gansters (1991) work control which
utilize Likert-type scales do not allow for interpretation and offer very limited richness of
data in comparison to interviews or observations in natural settings (Creswell, 2009).
This makes errors from participants, or response bias, more difficult to detect. Response
bias can result from lack of personal experience with the subject matter, poor
instructions, or even social desirability bias, where participants modify responses in a
deliberate attempt to provide more socially acceptable results (Cooper & Schindler,
2008). In this study, the participants were asked to report on their experiences during the
experimental activity, so there is little risk of response bias due to lack of personal
knowledge. Participants were all educated professionals, but instructions were carefully
crafted for both the experiment and the survey in order to minimize any confusion. When
any confusion came up, the researcher was careful to share the same clarifying instruction
with all participants. Social desirability bias was controlled in part because the
participants did not initially know the true purposes of the study.
In addition, assurances
of strict confidentiality should diminish participants desires to provide less than truthful
responses in order to preserve their reputation or standing.
In this study, the reliability of the experimental productivity measurements also
has to be considered. Common method variance is variation that can be attributed to the
measurement method instead of the experimental treatment (Creswell, 2009).
Researchers must be concerned with both the stability and equivalence of the
measurement techniques. Stability refers to the ability to secure consistent results when
96
measuring the same person repeatedly. Equivalence has to do with error introduced by
different observers or different samples (Cooper & Schindler, 2008).
As experimental data was collected, several steps were necessary in order to
maximize reliability. To help ensure measurement stability, instructions were carefully
scripted and all interaction with the researcher were limited. This ensured consistency of
delivery from group to group and minimizes potential sources of bias. Uniformity
between the groups was strictly maintained in terms of experimental setting, timing, and
even tone. The experimental activity during the online business simulation involved
group work, so factors such as competence may have affected productivity and
performance.
Ethical Considerations
This study did not contain any inherent level of risks above the minimum for
participants. Ethical standards were maintained through the organizations ethics
committee and normal University IRB approval. Permissions to access the targeted
participants were sought by respective human resource and ethics committees, which
function as corporate versions of institutional research boards. All participants were
informed that their participation in the study was voluntary and that they could opt out
any time. Participants were asked to indicate their informed consent at the beginning of
the survey and pre-test. Participants were also informed that any results published or
shared with their employers would only include aggregated data that would not permit
the identification of specific individuals, workgroups, or organizations.
97
One item of particular concern that may arise from a research design with
different experimental groups is the use of deception, or at least the temporary
withholding of information. While some scholars believe deception should never occur
during a research study, others suggest it may be necessary in order to prevent bias or to
protect confidential information. Based on the research objective in this research study,
there was no need to disguise or conceal the true purpose of the study. All three
experimental groups were trained separately and the goodwill of the participants was
maintained at all times.
Another item of potential concern in this research design was the use of a sample
comprised of full-time employees and managers who were being assessed on strategy and
execution skills that they use in their day to day operations of running a semiconductor
manufacturing facility. These employees may be considered a vulnerable population due
to the fact that management has a position of authority over them and because they fear
that lack of participation may have some impact on their employment (Cooper &
Schindler, 2008). However, to help relieve these concerns, employees and managers at
the site were clearly informed that participation was completely voluntary and would not
impact their employment.
Participant confidentiality was protected through such measures as not collecting
personal identification information, such as names, but rather identifying each respondent
with an assigned code. All responses were further matched to the participant code only.
All data was collected by the online simulation database and solely accessed by this
researcher. Data will be electronically safeguarded with a username and password. Data
will be stored, through password protection, for the requisite seven years, at which time
98
they will be destroyed by electronic wiping of the hard drive and external devices.
Approval letters for both the simulation and semiconductor companies has been attached
in the appendix. While risks were minimal, they were balanced through research gains
and by fully debriefing participants after the studys completion.
99
CHAPTER 4. RESULTS
This chapter presents the results of the statistical analysis employed in this study.
The purpose of this experimentally-based research study was to investigate the use of
training. The primary research question focuses on whether online business simulations
provide gains in business knowledge or business acumen, which is on the order of a
disruptive innovation. Initially, the training provided to the participants is discussed as
well as the online survey which was used to gather participant demographics. Next, the
reliability of individual variables used in the research study is analysed. After these
preliminary analyses are discussed, this chapter next turns to the testing of the alternative
hypotheses of this research study. Hypotheses were tested by examining the significance
of the F statistics for the independent variable and for each covariate in the ANOVA
tables. A significance level below the critical value of 0.05 allowed for the null
hypothesis to be rejected. Confidence intervals and contrast matrices were also built to
further evaluate the statistical results.
Supporting research questions involving the degree of the relationship between
knowledge application skills and online business simulation performance was evaluated
using Pearson correlation analysis. If the Pearson correlation coefficients were found to
be statistically significant, with critical values below 0.05, then the null hypothesis was
rejected. The hypothesis testing related to each of the supporting research questions is
presented and this chapter ends with a summary of the results.
100
Training for a Semiconductor Company

As described in the previous chapter, a semiconductor company was invited to
participate in corporate training meant to help increase business acumen for their
employees and managers. The Human Resource Department selected participants from a
total of 720 employees at a $600MM manufacturing facility. A total of 75 participants
were randomly selected and then placed in one of three experimental groups based on
their background and experience in the company. Due to work schedules and production
delays, only 65 of the employees and managers were available to participate in the three
groups; control (23), traditional (22), simulation (20). The ideal number of participants
was estimated to be between 20 and 25 for each group, in order to detect a MANCOVA
main effect size of one standard deviation with at least 95% significance and 90% power.
According to Faul (2009) a sample size as small as 14 for three treatment groups would
be able to detect a MANCOVA main effect size of one standard deviation, with a power
of 80% and a statistical significance of 95%. In this case, the number of participants for
each group was considered acceptable because they were within the range of participants
initially targeted for each experimental group.
Within the population of interest for this study, departments form natural clusters
and this study employed randomized cluster sampling in order to get representation of
these departments in each group. The main departments located at the manufacturing
facility include; production, finance, marketing, and human resources. Within the
production department there are many other divisions (i.e., manufacturing engineering,
process engineering, facilities, and maintenance); however, all positions were rolled up
into the four major departments. This worked out well because within the online
101
business simulation training, there are four major decision areas which tie out with the
actual departments within the semiconductor company.
Participant Demographics
An online survey was used to capture information on the background, experience
and education for each participant just prior to the training. The information from these
survey items was used as descriptive variables in the analysis and included the following;
gender, level of education, years of industry experience, area of expertise, prior
simulation experience and level of autonomy or work control within the company. For
the demographic item of gender, 86.2% of the participants were male (N = 56), 58.5%
had earned a Bachelors degree (N = 38), 50.8% had more than 15 years of industry
experience (N = 33), 43.1% were in the production department (N = 28), 50.8% had no
prior simulation experience (N = 33), and 52.3% self-reported a moderate level of
autonomy in their current employment position (N = 34). This demographic information
tied out with the overall population having a majority of male employees and managers
with 15-20 years of industry experience who are involved in manufacturing operations.
The mean number of level of education was 16.3 years (including high school)
while the mean level of industry experience for participants was 16.5 years. Out of the
four major departments located at the manufacturing facility, 43.1% were from
operations (N = 28), 10.8% were from human resources (N = 7), 16.9% were from
marketing (N = 11), and 29.2% were from finance (N = 19). The continuous covariate,
level of autonomy, was based on a survey given to all participants using an instrument
developed by Dwyer and Ganster (1991). This survey instrument consists of twenty-two
102
Likert-scaled items where each participant is asked to indicate their level of individual
responsibility within the organization. On a scale of one to five the mean was 3.12 with a
standard deviation of 0.839 (N = 65) as shown in Figure 10.
\
Figure 10. Demographic Variable: Level of Autonomy. This normality plot shows the
level of autonomy for all participants in their current position within the company. This
demographic variable was treated as a continuous covariate in the MANCOVA analysis.
Normality Testing
Several different statistical tests were performed to determine the normality of the
aggregate data set. First, Shapiro-Wilk statistics were generated on each of the variable
scales. Next, histograms with superimposed normal curves and Q-Q plots were generated
for each scale. Skewness and kurtosis scores were also generated for each scale. Table 4
103
shows descriptive statistics for each of the demographic variables: years of education,
years of industry experience, and autonomy.
Table 4
Descriptive Statistics for Survey Items (N = 65)
Valid
Missing
Mean
Std. Deviation
Skewness
Kurtosis
Minimum
Maximum
Years of
Education
Years of Industry
Experience
Prior Simulation
Experience
Autonomy
65
0
16.25
1.36
1.20
3.89
14.00
22.00
65
0
16.49
6.25
0.38
-0.07
6.00
35.00
65
0
0.49
0.50
0.03
-2.06
0.00
1.00
65
0
3.12
0.84
-0.40
0.64
1.00
5.00
The histogram and Q-Q plot examination indicated a normal distribution for most
variables. This visual examination was confirmed by the skewness scores for these
variables: years of education = 1.20, years of industry experience = 0.38, prior
simulation experience = 0.03, and autonomy =-0.40. Both a visual examination of the
histograms and the generated kurtosis scores reveal minimal kurtosis for years of industry
experience (-0.07) but significant kurtosis for years of education (3.89).
A similar statistical test was performed for the independent variable, type of
pedagogical training, and for all dependent variables; change in business knowledge,
change in business acumen, and final knowledge application score. The histogram and
Q-Q plot examination indicated a normal distribution for all dependent variables. This
visual examination was confirmed by the skewness scores: type of training = 0.09,
104
change in business knowledge = 1.08, change in business acumen = 1.83, and knowledge
application = -0.96 as shown in Table 5. Both a visual examination of the histograms and
the generated kurtosis scores reveal minimal kurtosis for change in business knowledge
(0.00), but significant kurtosis for change in business acumen (3.99).
Table 5
Descriptive Statistics for Dependent Variables (N = 65)
Pedagogical
Training
N
Valid
Missing
Mean
Std. Deviation
Skewness
Kurtosis
Minimum
Maximum
65
0
1.95
0.82
0.09
-1.50
1.00
3.00
Change in
Business
Knowledge
65
0
18.15
26.50
1.08
0.00
-14.70
80.00
Change in
Business Acumen
Knowledge
Application
65
0
9.92
19.14
1.83
3.99
-21.40
80.00
65
0
739.97
126.83
-0.96
1.33
325.00
954.00
Normality Test for Dependent Variables

The importance of the assumption of equal variances in all of the groups depends
on whether they all have roughly equal sample sizes (Norusis, 2008). In this research
study, each experimental group had between 20 to 23 participants. A test of homogeneity
of variance between the dependent variables; (1) change in business knowledge, (2)
change in business acumen and (3) knowledge application score was performed. This
assumption of homogeneity of covariance matrices was checked using Levenes test to
insure there is univariate equality of variances between groups. The Levene Statistic is
shown below for each of the dependent variables. A high level of significance (p =
105
0.000) was achieved for change in business knowledge and change in business acumen;
however, the knowledge application score was not considered significant (p = 0.625).
Table 6
Test of Homogeneity of Variances for Dependent Variables
Levene
Statistic
df1
df2
Sig.
Change in Business
Knowledge
13.218
62
0.000
Change in Business
Acumen
15.019
62
0.000
Knowledge
Application Score
0.474
62
0.625
Dependent Variables
Normality Test for Change in Business Knowledge

Univariate normality was verified through normality plots of the variables and by
conducting the Komogorov-Smirnov and Shapiro-Wilk normality tests. Small values for
the observed significance level lead to rejection of the null hypothesis that the data come
from normal populations. This test of normality for the dependent variable, change in
business knowledge, did not allow for rejection of the null hypothesis due to the high
level of observed significance for Shapiro-Wilk, as show in Table 7. Univariate
normality was also visible in the Q-Q plots and boxplot for change in business knowledge
based on the type of experimental treatment for all three groups (see Figure 11).
106
Table 7.
Tests of Normality for Change in Business Knowledge

Dependent
Variable
Change in
Business
Knowledge
Kolmogorov-Smirnova
Shapiro-Wilk
Experimental
Treatment
Statistic
df
Sig.
Statistic
df
Sig.
0.111
23
.200*
0.969
23
0.675
0.146
22
.200*
0.927
22
0.106
0.142
20
.200*
0.925
20
0.124
*. This is a lower bound of the true significance.
Figure 11. Normal Q-Q Plots and Boxplot for Change in Business Knowledge.
Univariate normality was identified in the Q-Q plots and Boxplot for change in business
knowledge based on the type of experimental treatment for each group.
107
Normality Test for Change in Business Acumen

Univariate normality for business acumen was also verified by conducting the
Komogorov-Smirnov and Shapiro-Wilk normality tests. A relatively low value (p =
0.006) was reported in the Shapiro-Wilk analysis for this dependent variable when
looking at the traditional training method; however, this was not determined to be
significant enough to lead to rejection of the null hypothesis that the data come from
normal populations as shown in Table 8.
Table 8
Tests of Normality for Change in Business Acumen

Dependent
Variable
Change in
Business
Acumen
Kolmogorov-Smirnova
Shapiro-Wilk
Experimental
Treatment
Statistic
df
Sig.
Statistic
df
Sig.
0.177
23
0.060
0.920
23
0.067
0.213
22
0.010
0.864
22
0.006
0.117
20
.200*
0.961
20
0.560
*. This is a lower bound of the true significance.
Test of Equality of Covariance Matrices

The assumption for the multivariate approach is that the vector of the dependent
variables follows a multivariate normal distribution, and the variance-covariance matrices
are equal across the cells formed by the between-subject effects (Norusis, 2008). Boxs
M test gave a result of 93.3 with significance of 0.000. This indicates that the null
hypothesis that the observed variance-covariance matrices are equal across the three
different experimental groups is strongly rejected. MANCOVA relies on the assumption
that the relationship between each dependent variable and the covariates is consistent
108
across groups. The assumption can be tested by calculating the significance of the
interaction effects between the covariates and the primary independent variable. If this
interaction is statistically significant, the assumption of homogeneity of covariate
regression slopes is suspect (Fields, 2009).
This statistical analysis was performed using analysis of covariance (ANCOVA).
The extraneous variables, or covariates, identified in this study were the participants
prior experience with business simulations and autonomy or level of work control within
the company. The ANCOVA results show the continuous covariate (level of autonomy)
with no real significance (p = 0.710); however, a very high significance (p = 0.001) was
calculated for the dichotomous covariate (previous simulation experience). The
interaction is statistically significant for prior simulation experience which may indicate
that the assumption of homogeneity for this regression slope is suspect.
ANOVA Analysis
An analysis of variance (ANOVA) was performed for the pre/post-test for each
experimental group in order to determine the level of statistical significance between
groups. Based on this analysis, the post-test results for both business knowledge and
business acumen were determined to be statistically significant (p = 0.000).
109
Table 9
ANOVA for Pre/Post-Test Results

Dependent
Variable
Experimental
Treatment
Sum of
Squares
df
Mean
Square
Mean
Square
Sig.
Pre-Test on
Business
Knowledge
Between Groups
262
131.1
0.854
0.431
Within Groups
9512
62
153.4
Total
9775
64
Post-Test on
Business
Knowledge
Between Groups
30389
Within Groups
4188
62
Total
34577
64
Pre-Test on
Business
Acumen
Between Groups
1491
745.3
Within Groups
12908
62
208.2
Total
14398
64
Post-Test on
Business
Acumen
Between Groups
18237
9118.4
Within Groups
6498
62
104.8
Total
24735
64
15194.4 224.938
0.000
67.5
3.580
0.034
86.996
0.000
The pre and post-test mean for business knowledge and business acumen were also
calculated for all three experimental treatments, resulting in 30.4 for the control group,
37.4 for traditional training and 80.2 for simulation training as shown in Figure 12.
Figure 12. Post-Test on Business Knowledge vs Experimental Treatment.

110
The mean post-test score for business acumen was 36.7 for the control group, 40.7 for
traditional training and 74.8 for simulation training as shown below in Figure 13.
Figure 13. Post-Test on Business Acumen vs Experimental Treatment
MANCOVA Analysis
The primary research question for this study was tested by examining the
significance of the F statistics for the independent variable and for each of the covariates
in the ANOVA tables. A significance level below the critical value of 0.05 allowed for
the null hypothesis to be rejected. Confidence intervals and contrast matrices were also
built to further evaluate the statistical results. This analysis showed a high level of
significance for experimental treatment (p = 0.000) and prior simulation experience (p =
0.002). Level of autonomy for each participant does not appear to be significant with a
significance of 0.511 and an observed power of 0.207. The tests of between subject
111
effects for experimental treatment was shown to have a significance of p = 0.000 and an
observed power of 1.000, 0.999, and 0.983 for change in knowledge, change in acumen
and knowledge application, respectively. Autonomy does not appear to have a high
significance; however simulation experience is significant for knowledge application as
shown in Table 10.
Table 10
Tests of Between Subject Effects for Experimental Treatment

Source
Training
Type III Sum

of Squares
Mean
Square
Sig.
Observed
Power
27,978
13988.8
95.458
0.000
1.000
7,125
3562.3
15.383
0.000
0.999
229,773
114886.5
10.303
0.000
0.983
247
247.3
1.688
0.199
0.248
0.6
0.003
0.958
0.050
10,170
10170.1
0.912
0.343
0.156
14
13.6
0.093
0.761
0.060
64
63.6
0.274
0.602
0.081
156,161
156160.8
14.004
0.000
0.957
Dependent Variables
Autonomy Change in Business Acumen
Prior
Simulation Change in Business Acumen
Experience Knowledge Application
Correlation Analysis
The secondary research questions involving the degree of the relationship
between knowledge application skills and online business simulation performance was
considered using correlation analysis. The magnitude and direction of the Pearson
correlation coefficients served to describe the relationship between variables. This
analysis was performed by initially looking at the scatterplots based on experimental
112
treatment. In order to calculate a more objective measure, a Pearson correlation

coefficient was calculated for each dependent and descriptive variables as shown in Table
11.
Table 11
Pearson Correlation Coefficients

Correlation
Sig. (2-tailed)
N
Change in
Business
Knowledge
Change in
Business
Acumen
Knowledge
Application
Score
Experimental
Treatment
Area of
Expertise
Industry
Experience
(Years)
Education
(Years)
Change in Change in Knowledge

Business
Business Application
Knowledge Acumen
Score
.560**
0.239
1
0.055
0.000
65
65
65
.560**
1
.330**
0.000
0.007
65
65
65
0.239
.330**
1
0.055
0.007
65
65
65
.811**
.596**
.347**
0.000
0.000
0.005
65
65
65
-.372**
-0.233
0.016
0.062
0.898
0.002
65
65
65
.302*
.283*
0.088
0.014
0.022
0.487
65
65
65
-0.090
-0.085
0.099
0.477
0.502
0.434
65
65
65
Type of
Training
Area of
Expertise
.811**
0.000
65
.596**
0.000
65
.347**
0.005
65
1
-.372**
0.002
65
-0.233
0.062
65
0.016
0.898
65
-.426**
0.000
65
1
65
-.426**
0.000
65
.252*
0.043
65
-0.046
0.717
65
65
-0.120
0.341
65
0.105
0.407
65
Industry
Level of
Experience Education
(Years)
(Years)
.302*
-0.090
0.014
0.477
65
65
.283*
-0.085
0.022
0.502
65
65
0.088
0.099
0.487
0.434
65
65
.252*
-0.046
0.043
0.717
65
65
-0.120
0.105
0.341
0.407
65
65
-0.140
1
0.267
65
65
-0.140
1
0.267
65
65
**. Correlation is significant at the 0.01 level (2-tailed).

*. Correlation is significant at the 0.05 level (2-tailed).
A significant correlation was shown between change in business knowledge, change in

business acumen (p = 0.000), knowledge application score (p = 0.007), type of training or
experimental treatment (p = 0.000), and area of expertise (p = 0.002).
113
Comparison of Means
The supporting research questions involve measuring the change in learning when
comparing the control group (group 1) directly against traditional corporate training
(group 2) and online business simulations (group 3). This investigation was intended to
determine whether there was any additional evidence that the change in business
knowledge and change in business acumen was significantly different based on the type
of pedagogical training that was given to adult learners. This portion of the analysis was
accomplished with simple comparison of means and T-tests. The mean values and
standard deviation for each of these variables is shown in Table 12.
Table 12
Group Statistics for Dependent Variables based on Experimental Treatment
Dependent
Variable
Change in
Business
Knowledge
Change in
Business
Acumen
Knowledge
Application
Experimental
Treatment
1
2
3
1
2
3
1
2
3
N
23
22
20
23
22
20
23
22
20
Mean
-0.290
5.550
53.210
-0.530
4.623
27.76
670.1
781.3
774.9
Std.
Deviation
8.910
6.502
18.362
4.664
9.129
24.85
139.9
104.6
101.4
The change in business knowledge and business acumen for the control group was
relatively small at -0.290 and -0.530, respectively. However, change in business
knowledge and business acumen was slightly higher for the traditional group at 5.550 and
114
4.623, respectively. Overall change in business knowledge and business acumen for
simulation training was significant at 53.210 and 27.76, respectively.
An independent samples test for equality of means was also performed in order to
determine the level of significance for each of these dependent variables where equal
variances are not assumed. The level of significance, mean difference, and standard error
of difference was calculated for change in business knowledge, change in business
acumen and knowledge application. Change in business knowledge showed a significant
value (p = 0.000) for both Levenes test and the t-test. Change in business acumen also
showed a significant alue (p = 0.001) for both tests; however, knowledge application was
not considered significant Levenes Test (p = 0.935) or the t-test (p = 0.841).
Table 13
Independent Samples Test for Equality of Means
Dependent
Variable
Change in
Business
Knowledge
Change in
Business
Acumen
Knowledge
Application
Levene's Test for Equality of Variance

Equal
Variances
F
Sig.
Assumed
19.477
0.000
Not Assumed
Assumed
12.612
0.001
Not Assumed
Assumed
0.007
0.935
Not Assumed
df
t-test for Equality of Means

Sig.
Mean
Std. Error
(2-tailed) Difference Difference
-11.423 40.0
0.000
-47.660
4.172
-10.998 23.3
0.000
-47.660
4.334
-4.080
40.0
0.000
-23.137
5.672
-3.930
23.6
0.001
-23.137
5.887
0.202
40.0
0.841
6.423
31.852
0.202
39.8
0.841
6.423
31.805
Hypothesis Testing
Given the preceding discussion of the underlying characteristics of the data for
this study, the next step of analysis is the testing of the hypotheses. These hypotheses
115
were constructed via predicted relationships between one independent variable, three
dependent variables, two covariates, and three descriptive variables. The one
independent variable is type of pedagogical training. The three dependent variables are
change in business knowledge, change in business acumen, and knowledge application
skill based on the training that was introduced to each participant. The two covariates are
level autonomy within the company (continuous covariate) and previous simulation
experience (dichotomous covariate). The three descriptive variables are area of expertise,
industry experience, and level of education for each participant.
The three dependent variables were based on a pre/post-test which was given to
all participants. The change in business knowledge and change in business acumen was
based on the difference between the pre-test and post-tests (see Appendix A), while the
knowledge application score was based on the final Comp-XM score for each
participant. The continuous covariate, Level of Autonomy, was based on a survey given
to all participants using an instrument developed by Dwyer and Ganster (1991) as shown
in Appendix B. This survey instrument consists of twenty-two Likert-scaled items where
each participant is asked to indicate the extent to which each item is an accurate or
inaccurate description of their job by circling a number in front of each statement. The
dichotomous covariate, Simulation Experience, was also based on a survey where each
participant simply identified whether they had ever used a business simulation before this
training. The three descriptive variables regarding expertise, industry experience and
level of education were also captured through an online survey prior to the training.
Level of education was based on number of years of education including high school.
Industry experience was also self-reported by each participant as total number of years of
116
work experience in the semiconductor industry. Finally, expertise was based on their job
position and organized into four major departments; operations, marketing, finance, and
human resources.
Alternative Hypotheses
The alternative hypotheses are reiterated as follows:
H1a. There is a difference across experimental groups for business knowledge
after adjusting for previous simulation experience and autonomy/work control.
H2a. There is a difference across experimental groups for business acumen after
adjusting for previous simulation experience and autonomy/work control.
H3a. A correlation exists between participants business acumen and business
knowledge.
H4a. Participants who engage in an online business simulation will demonstrate
higher business knowledge after experiencing the business simulation.
H5a. Participants who engage in an online business simulation will demonstrate
higher business acumen after experiencing the business simulation.
H6a. There is a difference between the simulation group, the case study group,
and the control group based on their change in business knowledge.
H7a. There is a difference between the simulation group, the case study group,
and the control group based on their change in business acumen.
H8a. A correlation exists between participants business simulation performance
and their level of autonomy/ work control in the company.
and their years of industry experience.
and their level of education.
117

The intent of this research question was to explore whether business simulations
provide an increase in business knowledge or business acumen, which is on the order of a
disruptive innovation. The hypotheses related to this question (H1 and H2) were
evaluated by comparing the average normalized gain <g>. This is defined as the ratio of
the actual average gain (%<post> - %<pre>) to the maximum possible average gain (100
- %<pre>). The traditional training group (N = 22) which was based on the case method
and lecture achieved an average gain in business knowledge <g>T-BK = 5.55 6.50 (std
dev). In sharp contrast, the online business simulation group (N = 20) achieved an
average gain in business knowledge <g>S-BK = 53.21 18.36 (std dev.). This is a 2.55
standard deviation difference in the normalized gains between traditional and simulation
training.
In the case of business acumen, the gains were also significant at <g>T-BA =
4.62 9.13 (std dev.) while the online business simulation group achieved an average
gain of <g>S-BA = 27.76 24.85 (std dev.) which is almost one full standard deviation
difference in the normalized gains between traditional and simulation training. These
results demonstrate that there is a significant difference across experimental groups for
business acumen and business knowledge, allowing for both null hypotheses to be
rejected. Conclusions from these findings are discussed fully in Chapter 5.

The intent of these research questions were to explore whether knowledge
application skills (business knowledge) are positively correlated with online business
118
simulation scores (H3) and if so, to determine how this change in knowledge application
skills compares across experimental groups (H6 and H7). Based on the analysis of
variance (ANOVA), the post-test results for both business knowledge and business
acumen were determined to be statistically significant (p = 0.000). Also, a strong
correlation was found between business knowledge and business acumen for all three
experimental groups based on the Pearson correlation coefficients. This analysis allowed
for the null hypothesis to be rejected for all three hypotheses. Conclusions from these
findings are also discussed in Chapter 5.

The intent of these supporting research questions was to investigate whether
traditional training or online business simulations lead to a significant increase in
business knowledge or business acumen (H4 and H5) and also to investigate whether any
correlation exists between simulation performance and autonomy (H8), industry
experience (H9), or level of education (H10). As discussed earlier, the comparison of
means for gains in business knowledge and business acumen across experimental groups
shows over a two standard deviation difference in the normalized gains for business
knowledge and almost one full standard deviation difference in normalized gains for
business acumen, allowing for the null hypothesis to be rejected for H4 and H5.
MANCOVA analyzes multiple dependent variables as a vector, after the effects of
covariates are statistically removed. When evaluating the individual hypotheses and
looking at the correlation between variables and covariates, analysis of variance or
119
ANOVA can be used as a follow-up technique to determine the extent means differ
between groups and determine the level of significance for each variable.
Table 14
Analysis of Variance Between Groups
Variable
Between Groups
Within Groups
Total
Previous
Between Groups
Simulation
Within Groups
Experience Total
Change in
Between Groups
Business
Within Groups
Knowledge Total
Change in
Between Groups
Business
Within Groups
Acumen
Total
Knowledge Between Groups
Application Within Groups
Score
Total
Between Groups
Area of
Within Groups
Expertise
Total
Industry
Between Groups
Experience Within Groups
(Years)
Total
Between Groups
Education
Within Groups
Level (Years)
Total
Between Groups
Gender
Within Groups
Total
Level of
Autonomy
Sum of
Squares
0.0
45.0
45.0
3.7
12.6
16.2
35895.0
9040.8
44935.7
9492.4
13957.9
23450.3
174046.4
855515.5
1029561.9
21.1
87.1
108.2
158.9
2339.3
2498.2
0.4
117.7
118.1
0.2
7.5
7.8
120
df
2
62
64
2
62
64
2
62
64
2
62
64
2
62
64
2
62
64
2
62
64
2
62
64
2
62
64
Mean
Square
0.02
0.73
1.84
0.20
F
0.033
Sig.
0.967
9.079
0.000
17947.49 123.081
145.82
0.000
4746.22
225.13
21.082
0.000
87023.21
13798.64
6.307
0.003
10.55
1.40
7.511
0.001
79.46
37.73
2.106
0.130
0.19
1.90
0.098
0.907
0.11
0.12
0.916
0.405
This analysis shows a high level of statistical significance between groups for the
dichotomous covariate, previous simulation experience: F65 = 9.08, p = 0.000; change in
business knowledge: F65 = 123.08, p = 0.000; change in business acumen: F65 = 21.08, p
= 0.000, knowledge application score: F65 = 6.31, p = 0.003 and area of expertise: F65 =
7.51, p = 0.001. However, years of industry experience showed only slight significance:
F65 = 2.10, p = 0.130; while level of education: F65 = 0.098, p = 0.907 and level of
autonomy within the organization: F65 = 0.333, p = 0.967 showed very little significance
in this analysis. Based on the fact that these last three variables failed to achieve
statistical significance (at p = < 0.05), the null hypothesis was accepted for H8, H9, and
H10. Conclusions from these findings are discussed fully in Chapter 5.
Summary
This chapter presented the statistical analysis and interpretation of the quantitative
data collected during this study. A sample population of employees and managers at a
semiconductor manufacturing facility were exposed to different types of corporate
training in order to investigate the use of online business simulations by measuring the
change in participants business knowledge and business acumen. Descriptive statistics
of the demographics of this sample were discussed. The reliability of individual variables
used in the research were analyzed and univariate normality was verified. The
assumption of homogeneity of covariance matrices was also checked using Levenes and
Boxs tests. Finally, the assumption of homogeneity of covariate regression slopes was
investigated so that a multiple analysis of covariance (MANCOVA) could be completed.
121
An analysis of the data indicated that seven of the ten alternative hypotheses were
fully supported. No significant relationships were found between the dependent variable
of business simulation performance and participants individual level of autonomy, years
of industry experience or level of education. The results of the hypotheses test and the
previously mentioned descriptive statistics for the sample demographics are used in
Chapter 5 to answer the primary and secondary research questions as well as the
supporting research questions.
Exploratory analysis of the data indicated that some of the dependent variables
(e.g., change in business knowledge and knowledge application) are significantly
impacted by the type of pedagogical training given to participants. The results of this
exploratory analysis provide rich material as a basis for examining the research questions
and insight into the use of online business simulations as a disruptive technology.
122
Table 15
Summary of Hypothesis Testing
Null Hypothesis
Result
H01
There is NO difference across experimental groups for business knowledge after

adjusting for previous simulation experience and autonomy/ work control.
Rejected
H02
There is NO difference across experimental groups for business acumen after

adjusting for previous simulation experience and autonomy/ work control.
Rejected
H03
NO correlation exists between participants business acumen and business

knowledge.
Rejected
H04
Participants who engage in an online business simulation will NOT demonstrate

higher business knowledge after experiencing the online business simulation.
Rejected
H05
Participants who engage in an online business simulation will NOT demonstrate

higher business acumen after experiencing online the business simulation.
Rejected
H06
There is NO difference between the simulation group, the case study group and
the control group based on their business knowledge.
Rejected
H07
There is NO difference between the simulation group, the case study group and
the control group based on their business acumen.
Rejected
H08
NO correlation exists between participants business simulation performance

and their level of autonomy/ work control in the company.
Accepted
H09

and their years of industry experience.
Accepted
H10

and their level of education.
Accepted
123
CHAPTER 5. DISCUSSION, IMPLICATIONS, RECOMMENDATIONS
Summary and Discussion of Results

The purpose of this experimentally-based research study was to investigate the
use of online business simulations as a disruptive technology by measuring the change in
training. This quantitative study explored whether there was a relationship between
knowledge application skills and online business simulation performance. Further, it
explored the difference in overall change in knowledge application skills between two
experimental groups while taking into account numerous demographic variables and
covariates. This study used a MANCOVA technique for data analysis in order to address
the research questions associated with this study.
Research Questions
The primary research question was as follows:
1. Do online business simulations provide an increase in business knowledge or
business acumen for participants, which is on the order of a disruptive
innovation?
Two alternative hypotheses were tested in this study in order to answer this first
research question. Change in business knowledge and change in business acumen were
hypothesized to show a difference across experimental groups after adjusting for three
descriptive variables and two covariates. The three dependent variables were change in
business knowledge, change in business acumen, and knowledge application skill based
124
on the type of training that was introduced to each participant. The three descriptive
variables were area of expertise, industry experience, and level of education. The two
covariates were level of autonomy within the organization (continuous covariate) and
previous simulation experience (dichotomous covariate).

A significant difference in business knowledge gains was discovered when
comparing traditional corporate training to online business simulation training. The
traditional training group (N =22) which was based on lecture and the case method
achieved an average gain in business knowledge <g>T-BK = 5.55 6.50 (std dev). In
sharp contrast, the online business simulation group (N = 20) achieved an average gain in
business knowledge <g>S-BK = 53.21 18.36 (std dev.). This is a 2.55 standard deviation
difference in the normalized gains between traditional and simulation training. Past
research from Hake (1998) has shown that anything close to a two standard deviation
gain in a pre/post-test of interactive-engagement compared to traditional instruction may
classify this pedagogy as a disruptive innovation. This comparison of instructional
techniques when measuring the change in business knowledge clearly demonstrates the
impact of this teaching innovation.

A difference in business acumen gains was discovered when comparing
traditional corporate training to online business simulation training; however, it was not
nearly as significant as the change in business knowledge. The traditional training group
125
(N = 22) which was based on lecture and the case method achieved an average gain in
business acumen <g>T-BA = 4.62 9.13 (std dev.) while the online business simulation
group achieved an average gain of <g>S-BA = 27.76 24.85 (std dev.). This gain is
almost one full standard deviation difference in the normalized gains between traditional
and simulation training. These results demonstrate that there is a difference across
experimental groups for business acumen and business knowledge allowing for both null
hypotheses to be rejected. However, it is not clear that this can be classified as a
disruptive innovation for change in business acumen, because the gain was not near a
2.00 standard deviation difference in the normalized gains between teaching pedagogies.
Part of this may be due to the wide spread standard deviation at 24.85% for business
acumen compared to smaller spread of 18.36% for business knowledge. Further research
could be done in this area to determine if a focus on change in business acumen through
application is what actually leads to such a dramatic increase in business knowledge.

The secondary research questions asked by this study were as follows:
2. Are knowledge application skills in business positively correlated with online
business simulation scores?
3. How does the change in knowledge application skills with traditional
Three alternative hypotheses were tested in this study in order to answer these
secondary research questions. The intent of these research questions were to explore
whether a correlation exists between participants business simulations and if so,
126
determine how this change in knowledge application skills compares across experimental
groups. Based on the analysis of variance (ANOVA), the post-test results for both
business knowledge and business acumen were determined to be statistically significant
(p = 0.000). These results indicate a strong correlation between a participants individual
level of business knowledge (what they know) and their individual level of business
acumen (what they do). Also, a strong correlation was found between the change in
business knowledge and change in business acumen for all three experimental groups
based on the Pearson correlation coefficients. This analysis allowed for the null
hypothesis to be rejected for all three hypotheses and these conclusions shed light on
theoretical debate about the usefulness of online business simulations. These results are
in line with researchers who defend the disruptive nature of business simulations and the
positive influence of promoting training benefits provided by new technologies
(Anderson & Lawton, 2006; Cantoni & McLoughlin, 2004; Hernandez et al., 2010).

The supporting research questions asked by this study were as follows:
4. Do participants who are taught using an online business simulation gain a
5. Do participants who are taught using traditional corporate training gain a
6. Is the level of education or industry experience positively correlated with
7. Is the level of participant autonomy/work control positively correlated with
127
Five alternative hypotheses were tested in this study in order to answer these
supporting research questions. The intent of these supporting research questions was to
investigate whether traditional training or online business simulations lead to a significant
increase in business knowledge or business acumen and also investigate whether any
correlation exists between simulation performance and autonomy, industry experience, or
level of education. The comparison of means for gains in business knowledge and
business acumen across experimental groups shows over a two standard deviation
difference in the normalized gains for business knowledge and almost one full standard
deviation difference in normalized gains for business acumen, allowing for two of these
five null hypothesis to be rejected.
This analysis showed a high level of statistical significance between groups for
previous simulation experience: F65 = 9.08, p = 0.000; change in business knowledge: F65
= 123.08, p = 0.000; change in business acumen: F65 = 21.08, p = 0.000, knowledge
application score: F65 = 6.31, p = 0.003 and area of expertise: F65 = 7.51, p = 0.001.
However, years of industry experience showed only slight significance: F65 = 2.10, p =
0.130; while level of education: F65 = 0.098, p = 0.907 and level of autonomy within the
organization: F65 = 0.333, p = 0.967 showed very little significance in this analysis.
Based on the fact that these last three variables failed to achieve statistical significance (at
p = < 0.05), the null hypothesis was accepted that no correlation exists between
participants business simulation performance and their level of autonomy, years of
industry experience or level of education. In summary, the background or work
experience of the participant is not relevant to their gains in learning.
128
This research supports the theory of disruptive innovation (Christensen, 1997)
that a new technology within management education, such as online business
simulations, can come into non-consumer markets where successful implementation is
not dependent on the participants background or work experience. This study also
indicates that the use of online business simulations could be used with success in other
markets of non-consumption where expensive simulation training is typically limited.
The gains in business knowledge and business acumen which resulted from this research
study were based on managers and employees at relatively low positions of autonomy
within the organization. One of the key identifying characteristics of disruptive
innovations is that they begin in markets which are considered insignificant or emerging
relative to competing technologies. In the case of online business simulations today,
most are used at very high levels of executive management within a corporation or at
elite business schools.
The results from this research study indicate that significant gains in business
knowledge can be gained for employees and managers at all levels within an
organization. This study therefore connects experiential learning theory from psychology
(Lewin, 1935) with the developing field of disruptive innovation (Christensen, 2008), in
order to support the business case for significantly higher levels of adult learning through
the use of online business simulations. The latest research today on business simulation
games still focuses on faculty perception of simulations instead of actual empirical data
(Tanner et al., 2012). Based on these perceptions, there are still arguments over whether
online business simulations are effective teaching tools or simply window dressing.
129
In order to examine the effectiveness of engaging adult learners in an online

business simulation, this experimental design study focused on the change in business
knowledge application skills of participants based on exposure to the business simulation.
This research study evaluated the gains in business knowledge and business acumen to
help determine that online business simulations are a disruptive innovation in comparison
to more traditional corporate training methods. This study further identified that the
dramatic impact of online business simulations is independent of current worker
knowledge, background, work experience, level of education, or tenure within the
organization.
General Discussion and Theoretical Implications

Results from this study support the findings by Summers (2004), who described
how corporations, management education institutions, development consulting firms, and
collegiate business programs use business simulations to train and teach. Many educators
have assumed that business simulations will help participants learn how to apply their
knowledge to solve real-world problems based on their interactive, integrative, and
iterative nature (Anderson & Lawton, 1997). However, this research is the first
empirically based study to investigate the learning effects of business simulations on
participants knowledge application or increase in business acumen. Up until this time,
all studies regarding business simulations have focused on the students attitudes and
perception of their experience (Anderson & Lawton, 2006). The research in this study
helps fill an important research gap by examining the relationship between online
business simulations, experiential learning, and disruptive innovation.
130
Practical Implications
The future of online business simulations shows promise as a disruptive
innovation versus a sustaining innovation. Based on the characteristics of a disruptive
technology, this type of teaching pedagogy has a bright future in emerging or
insignificant markets. One of these emerging markets today is online education. In fact,
the use of business simulations in online education and the impact on business knowledge
seem to correspond relatively well to Christensens (1997) description of disruptive
technology. The challenge in the past has been determining whether these simulations
are truly disruptive innovations by measuring the effect of these business simulations on
participants today. Based on this research and the gains shown in business knowledge
and business acumen relative to traditional education, a strong case can be made for the
use of online business simulations as a disruptive innovation.
As online business simulations gain a foothold market and evolve over time, there
appear to be three areas of emphasis in order for this technology to become truly
disruptive; (a) there must be stronger pedagogical support for business simulations, (b)
the use of reflection and observation within these simulations needs to significantly
increase, and (c) empirical data on the impact of business simulations needs to be
collected as a form of experiential learning. This empirical-based research study helps
move management education forward in at least two areas of emphasis. Empirical data
from this study can be used to encourage stronger pedagogical support for online
business simulations and encourage focus on current non-consumer markets.
This research study helped to fill an important gap in the literature based on the
fact that most business classes are taught today with one main pedagogy and almost no
131
experimental studies exist that compare learning outcomes under alternative pedagogies
(Anderson & Lawton, 2009, p. 208). Todays learning environments in both corporate
training programs and business management classes mandate greater levels of participant
engagement, due mostly to shorter attention spans and the need for increased interaction
and stimulation within a structured environment (Tanner et al., 2012). Unfortunately,
most research today on interactive engagement tools, such as online business simulations,
is based on qualitative analysis via the perceptionof faculty and instructors. For
example, when evaluating knowledge application, the following Likert Statements from
the Clute Institute (2012) are often being used:
The simulation helps participants understand and integrate previous business
concepts in ways that enable them to apply the concepts in the future because they
1. are helpful in applying concepts that are taught in my discipline.
2. are effective in creating a learning context where students are willing to
open their minds to the course subject matter.
3. are effective in getting students to apply lessons learned in my course.
4. make learning more enjoyable.
5. make learning the material in my course easier to understand.
6. make the material in my course easier to understand.
7. provide an educational experience where students can learn about interfunctional coordination within a business.
8. enable students to experience competition within a marketplace.
These long-winded questions with ambiguous ratings on a scale of 1 to 5 do not get to the
heart of what is happening with actual learning gains inside of the simulation. At best,
132
this type of research gives a sense of how faculty and participants feel about their
learning; however, there is no empirical evidence to support this analysis!
Based on a recent survey of 1,586 marketing management faculty in the United
States (out of which only 107 responded) less than half of the faculty felt that simulations
inspired them to be more creative or helped them to connect with their students (Tanner
et al., 2012). With respect to creativity, only 33.9% of faculty agreed or strongly agreed
that simulations inspire them to be more creative in class. Similar results were found in
the area of connecting with students, where only 45.9% of faculty agreed or strongly
agreed that simulations inspired them to connect with their students. This focus on
inspiration and creativity on the part of faculty has very little to do with learning gains,
which is what business simulations are designed to do. This same study went on to great
lengths in order to draw on the most recent survey research for business simulations
(Buzzetoo-More et al., 2009; Goorha & Mohan, 2010; Wilson & Gerber, 2008) which
were also faculty perception based in order to point out that while simulations are
engaging, faculty do not believe that they are holistically more effective than cases,
service learning, or in-class discussions (Tanner et al., 2012).
When considering the disruptive nature of this technology, it is not surprising that
faculty and instructors specifically say that they feel less valued and less creative. This is
because the innate nature of the business simulation and the ability of the participant to
reflect on what they are learning actually replace the instructor as the source of
knowledge. Here it becomes clear that not only are researchers using the wrong tools to
measure the impact of business simulations, they are surveying the wrong people with the
wrong instruments and methods!
133
As business simulations begin to migrate from the classroom to online learning

platforms, it will be important to consider how this technology is viewed by organizations
willing to offer this technology to non-consumers. As a disruptive technology, these
simulations will initially offer much lower performance and less functionality at a much
lower price. Gradually, the online delivery of these business simulations will improve to
the point where they have the ability to displace the incumbent (high priced professors
and business consultants) by satisfying the markets need with an acceptable product or
service at a much lower price point. As discussed in Chapter 2, when an industry faces a
major disruption, the perception of leaders regarding that disruption will dramatically
influence how they respond (Gilbert & Bower, 2002). For example, if leaders of industry
view the disruption as a threat, senior management tends to overreact and commit too
many resources too quickly. However, if senior management sees the disruption as an
opportunity, theyre likely to commit insufficient resources which will allow them to
compete in the future and use the disruption to their advantage.
Thinking of business simulations in such definitive terms threat or opportunity
is dangerous. Gilbert and Bower (2002) have stated that when confronted with disruptive
technologies, the most successful companies frame the challenge differently at different
times. When resources are being allocated, leaders in this industry may see online
business simulations as a threat. However, when the actual work of responding to these
new markets begins, they should change their perception. If this theory holds true for
online business simulations, these industry leaders in management education should see
tremendous opportunity for new non-consumer markets in the near future as online
education technologies continue to improve and become more interactive.
134
Limitations and Recommendations for Future Research

As with most empirical studies, a number of limitations began to emerge during
this research. Some of these limitations had an effect on the validity and reliability of
results. First, this study used a specific type of business simulation which is in the
domain of managing a growing business in the semiconductor industry; hence, it may be
difficult to generalize results to other types of business simulations. Consequently, it will
be important to run a similar study with three experimental groups within an organization
that is not related to the semiconductor industry.
Second, this study used a relatively small sample size of 65 employees and
managers in the semiconductor industry, both limiting the kinds of quantitative analyses
that could be conducted and reducing the generalizability of results. Further research in
this area should include a broader study with approximately 50 participants per
experimental group in order to ensure the maximum desired power, effect size, and level
of statistical significance (Lenth, 2001).
Third, analyses were strictly limited to quantitative methods. While inferential
statistical procedures showed significance with numerical data, there was a need for some
qualitative analysis to shed light on the participants learning experience with a business
simulation. Future research in this area should allow a mixed methods approach with
some limited qualitative analysis in a short-answer post-test survey. A combined
research approach using both qualitative and quantitative data will most likely provide
the best picture for how online business simulations can be used in the future.
Future research identified by others in this field (Tanner et. al., 2012) have
suggested a focus on the perceptions of simulation novelty or sexiness in todays
135
classrooms. Based on the findings in this research and the importance of real empirical
evidence, it is the opinion of this researcher that the last thing that needs further analysis
is additional perception of business simulations. Other future research that has been
identified is the possible recency effect associated with simulations. It has been argued
that methodologies like case studies have existed much longer than simulations and it
would be interesting to determine if there is a recency effect in order to explain the
positive perception of simulations among students and participants. There is some
suspicion here that these same researchers are not interested in empirical data that would
allow simulations to stand on their own as an effective learning tool.
In light of these recommendations by other researchers in this field and the
valuable data gained from this research study, it is recommended that future research with
business simulations be conducted in the area where business management education is
beginning to show the biggest impact on transformational learning online education
(Garrison, 2004).
Online learning is pervading higher education and compelling
educators to confront existing assumptions of teaching and learning in higher education

(Christensen, 2008). Online learning is also disrupting higher education and corporate
education with the ability to replace the faculty in the classroom and the sage on the
stage of corporate training events. Given the increasing evidence that Internet
information and communication technologies are transforming much of society, there is
little reason to believe that online education will not be the defining transformative
innovation for education in the 21st century (Garrison, 2004). As shown by the dramatic
increase in business knowledge in this research study, online business simulations have
the potential to be a driving force behind this transformative innovation.
136
Conclusion
This study set out to investigate online business simulations as a form of
experiential learning by providing the first empirical analysis of business acumen and
knowledge application skills. This quantitative based research study demonstrated the
disruptive nature of online business simulations when it comes to gains in business
knowledge by measuring a 2.55 standard deviation difference in the normalized gains
between traditional training and business simulation training. Baseline tests against a
control group and traditional training group using MANCOVA to account for multiple
variables and covariates imply that online business simulations enhance both business
knowledge and business acumen on a staggering scale and in a very short period of time.
Previously to this study, almost all empirical research regarding business
simulations had relied on the participants perception of their learning experience (Tanner
et al., 2012 & Williams, 2011). This research study was unique in the business
simulation literature in that a significant amount of quantitative data was collected and
analyzed in order to measure the change of participants business knowledge and
business acumen based on the type of training given in a corporate setting. These
research results strongly suggest that the adoption of online business simulations in
corporate training environments can (1) increase business knowledge of employees and
managers at all levels within a company and (2) provide a rate of increase in business
knowledge and acumen which is well beyond that obtained by traditional practice.
As a potential disruptive technology, online business simulations have not
received the empirical research they deserve. Recent studies focused on the effectiveness
of business simulations have been based on perceptions of business management faculty.
137
Not surprisingly, these same faculty memberswho could quickly be replaced by such
technologyhave responded in surveys that simulations limit their creativity and ability
to connect with participants. It is important that the correct research methods be used to
measure the individuals who are actually impacted by the simulation and capture the
actual learning gains. One recent study of simulations (Xu & Yang, 2010) which actually
used empirical research methods focused on mental models as learning outcomes for
simulation participants instead of changes in business knowledge or business acumen.
This study hypothesized that the social interaction among group members and the level of
psychological safety led to higher learning gains, not necessarily the pedagogical nature
of the business simulation instruction. When it came to measures in this study, a Likerttype scale was used and once again the measurement was perception of the participants
learning experience instead of actual gains in learning.
Clearly, the disruptive nature of online business simulations will not be found in
the current consumer market of elite business schools or executive management programs
for large corporations. This technology has the low-cost and scalable structure to impact
non-consumer markets where successful implementation is not dependent on the
participants background or work experience. It is very possible that someday in the near
future, large numbers of employees at all levels within an organization will be able to
benefit from the use of online business simulations. The self-paced and interactive nature
of such simulations will allow participants to teach one another and reflect on what they
have learned through learning by doing rather than learning by knowing. This
disruptive form of learning may continue to be called gaming or it may translate into
something much more appealing for the common knowledge worker, actually becoming.
138
The great divide between the gaming community and management educators is
quickly coming to a close. In the past, gamers have dismissed educational simulations as
boring and irrelevant, while business management educators have dismissed gaming and
simulations as trivial and pedagogically unproven (Aldrich, 2009a). This research study
points out a significant opportunity which lies within an engaging business simulation
and its potential impact on the world of management education. As a truly disruptive
technology, online business simulations have the potential to dramatically impact
emerging markets such as online education with new and more powerful ways of
teaching and learning.
139
REFERENCES
Adkins, S. S. (2003). Radical learning technology happening now. Training and
Development, 57(11), 65-73.
Albert, W. (1970). Participative education and the inevitable revolution. Journal of
Creative Behavior, 4(4), 234-282.
Aldrich, C. (2009a). The complete guide to simulations and serious games: How the most
valuable content will be created in the age beyond Gutenberg to Google. San
Francisco, CA: Pfeiffer.
Aldrich, C. (2009b). Virtual worlds, simulations, and games for education: A unifying
view. Innovate 5 (5). Retrieved August 1, 2010 from http://faculty.chasshttp:
//www.innovateonline.info/index.php?view=article&id=727.
Anderson, J. R. (1985). Cognitive psychology and its implications (2nd ed.). New York:
Freeman.
Anderson, P. H., & Lawton, L. (1990). The relationship between financial performance
and other measures of learning on a simulation exercise. Developments in
Business Simulations and Experiential Exercises, 17(1), 6-10.
Anderson, P. H., & Lawton, L. (1995). The problem of determining and individualized
simulations validity as an assessment tool. Developments in Business Simulations
and Experiential Exercises, 22(1), 43-48.
Anderson, P. H., & Lawton, L. (1997). Demonstrating the learning effectiveness of
simulations: Where we are and where we need to go. Developments in Business
Simulations and Experiential Exercises, 34(1), 43-50.
Anderson, P. H., & Lawton, L. (2002). Is simulation performance related to application?
An exploratory study. Developments in Business Simulations and Experiential
Exercises, 29(1), 108-113.
Anderson, P. H., & Lawton, L. (2006). The relationship between students success on a
simulation exercise and their perception of its effectiveness as a pbl problem.
Developments in Business Simulations and Experiential Exercises, 33(1), 41-47.
Anderson, P. H., & Lawton, L. (2007). Simulation performance and its effectiveness as a
pbl problem: A follow-up study. Developments in Business Simulations and
Experiential Learning, 34(1), 43-50.
140
Anderson, P. H., & Lawton, L. (2009). Business simulations and cognitive learning:
Developments, desires, and future directions. Simulation & Gaming, 41(6), 193216.
Anderson, P. H., & Woodhouse, R. H (1984). The perceived relationship between
pedagogies and attaining objectives in the business policy course. Developments
in Business Simulations and Experiential Exercises, 11(1), 152-156.
Andlinger, G. R. (1958). Business games play-one. Harvard Business Review, 28(3),
115-125.
Archer, W., Garrison, R., & Anderson, T. (1999). Adopting disruptive technologies in
traditional universities: Continuing education as an incubator for innovation.
Canadian Journal of University Continuing Education, 25(1), 13-44. Retrieved
from Business Source Complete database.
Argyris, C., & Schon, D. A. (1974). Organizational learning: A theory of action
perspective, reading. Oxford, England: Addison-Wesley.
Bailey, D. A. (1990). Developing self-awareness through simulation and gaming. Journal
of Management Development, 9(2), 38-42.
Bergemann, L. H., & Queen, T. R. (1987). The aftermath: Treating traumatic stress is
crucial. Corrections Today, 49(5), 100-104.
Biggs, J. (1985). The role of metalearning in study processes. British Journal of
Educational Psychology, 55(1), 185-212.
Biggs, J. (1988). The role of metaleraning in enhancing learning. Australian Journal of
Education, 32(2), 127-138.
Biggs, W. (1990). Introduction to computerized business management simulations. Guide
to Business Gaming and Experiential Learning. (pp. 23-35). London, England:
Nichols Publishing.
Bloom, B. S. (1956). Taxonomy of educational objectives: Handbook/cognitive domain.
New York: McKay.
Bloom, B. S., Englehart, M. D., Furst, E. J., Hill, W. H., & Drathwohl, D. R. (1956).
Taxonomy of educational goals. Handbook 1: Cognitive domain. New York:
McKay.
Blythe, S. E., & Gosenpud, J. J. (1981). A relative evaluation of experiential and
simulation learning in terms of perceptions of effected changes in students.
Developments in Business Simulations and Experiential Exercises, 8(1), 145-148.
141
Boud, D., Keogh, R., & Walker, D. (1985). Reflection, turning experience into learning.
London, England: Nichols Publishing.
Bransford, J. D., & Nitsch, K. E. (1978). Coming to understand things we could not
previously understand. Speech and Language in the Laboratory, School and
Clinic (pp. 267-307). Cambridge, MA: MIT Press.
Bredemeier, M. E., & Greenblat, C. S. (1981). The educational effectiveness of
simulation games: A synthesis of findings. Simulation & Games, 12(1), 307-332.
Brown, A., Campione, J., and Day, J. (1981). Learning to learn: On training students to
learn from texts. Educational Researcher, 10, 14-21.
Bruce, M. (2001). Reflecting on reflection: An examination of learning and assessing
outcomes. Simulation and Gaming, 28, 8-12.
Burns, A. C., Gentry, J. W., & Wolfe, J. (1990). A cornucopia of considerations in
evaluating the effectiveness of experiential pedagogies. In J. W. Gentry (Ed.),
Guide to business gaming and experiential learning (pp. 253-278). East
Brunswick, NJ: Nichols/GP Publishing.
Buzzetto-More, N., & Mitchell, B. (2009). Student performance and perceptions in a
web-based competitive computer simulation. Interdisciplinary journal of elearning and learning objects, 5(1), 73-89.
Cantoni, L., & McLoughlin, C. (2004). Different thinking hats: The continuouslyevolving role of the instructor in E-problem based learning. Proceedings of EdMedia 2004: World Conference on Educational Multimedia, Hypermedia &
Telecommunications, (Chesapeake, VA: AACE), 2864-2870.
Chapman, K. J., & Sorge, C. L. (1999). Can a simulation help achieve course objectives?
An exploratory study investigating the differences among instructional tools.
Journal of Education of Business, 74(1), 225-230.
Cherryholmes, C. (1966). Some current research on effectiveness of educational
simulations: Implications for alternative strategies. American Behavioral
Scientist, 10(1), 4-7.
Chi, M. T., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Selfexplanations: How students study and use examples in learning to solve problems.
Cognitive Science, 13, 145-182.
Christensen, C. M. (1997). The innovators dilemma: When new technologies cause great
firms to fail. Boston: Harvard Business School Press.
142
Christensen, C. M. (2008). Disrupting class: How disruptive innovation will change the
way the world learns. New York: McGraw Hill.
Christensen, C. M., & Raynor, M. E. (2003). The innovators solution: Creating and
sustaining successful growth. Boston: Harvard Business School Press.
Clark, J. A. (1991). Self-directed learning skills and clinical performance: A comparison

of traditionally taught and learning contract-taught nursing students (Doctoral
dissertation, University of Southern Mississippi, 1990). Dissertation Abstracts
International, 51(107), 2236A.
Comp-XM (2012). Competency exam [Computer software]. Chicago, IL: Capsim

Management Simulations, Inc.
Cooper, D. R., & Schindler, P. S. (2008). Business research methods (10th ed.). New
York: McGraw Hill/Irwin.
Costa, A., & Garmston, R. (1994). Cognitive coaching: A foundation for the renaissance
school. Norwood, MA: Christopher Gordon.
Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods
approaches (2nd ed.). Thousand Oaks, CA: Sage Publications.
Crookal, D. (1995). A guide to the literature on simulation/gaming. Simulation and
gaming across disciplines and cultures (pp. 151-177). Thousand Oaks, CA: Sage.
Cross, K. P. (1989). The influence of self-directed learning contracts versus a passive
therapy treatment in a back rehabilitation program (Doctoral dissertation, Indiana
University, 1988). Dissertation Abstracts International, 50(5), 1837B.
Davies, P. (2003). Simulation: Brining e-learning to a new level. Computersuser.com, 3137. Retrieved July 25, 2011, from
http://www.computeruser.com/articles/2207,1,1,1,0701,03.html
De Jong, T., & Van Joolingen, W. R. (1998). Scientific discovery learning with computer
simulations of conceptual domains. Review of Educational Research, 68(2), 179201.
De Nicola, N. (1990). Debriefing sessions: The missing link in focus groups. Marketing
News, 24(1), 20-22.
Dewar, R. D., & Dutton, J. E. (1986). The adoption of radical and incremental
innovations: an empirical analysis. Management Science, 32(11), 1422-1434.
143
Dewey, J. (1910). How we think. Boston: D.C. Heath.

Dewey, J. (1938). Experience and education. Kappa Delta and Collier, New York.
Donovant, B. W. (2009). The new, modern practice of adult education: Online instruction
in a continuing professional education setting. Adult Education Quarterly, 50(3),
227-245.
Druckman, D., & Ebner, N. (2008). Onstage or behind the scenes? Relative learning
benefits of simulation role-play and design. Simulation and Gaming, 39(4), 465497.
Druckman, D., & Robinson, V. (1998). From research to application: Utilizing research
findings in training programs. International Negotiation, 3(1), 7-38.
Dwyer, J. D., & Ganster, D. C. (1991). The effects of job demands and control on
employee attendance and satisfaction. Journal of Organizational Behavior, 12(7),
595-608.
Ebner, N., & Efron, Y. (2005). Using tomorrows headlines for todays training: Creating
pseudo-reality in conflict resolution simulation games. Negotiation Journal,
21(1), 377-394.
Ellington, H., Fowlie, J., & Gordon, M. (1998). Using games and simulations in the
classrooms. London, England: Routledge.
Ertmer, P. A., & Newby, T. J. (1996). The expert learner: Strategic, self-regulated and
reflective. Instructional Science, 24, 1-24.
Falkenberg, L., Russel, R., & Ricker, L. (2000). Linking theory with practice:
Undergraduate project management with school-age children. Journal of
Management Education, 24(6), 745-762.
Faria, A. J. (2004). A survey of simulation game users, former-users, and never-users.
Simulation & Gaming, 35(2), 178-207.
Faria, A. J., Hutchinson, D., & Wellington, W. J. (2009). Developments in business
gaming: A review of the past 40 years. Simulation & Gaming, 40(4), 464-487.
Echambadi, R., Campbell, B., & Agarwal, R. (2006). Encouraging best practice in
quantitative management research: An incomplete list of opportunities. Journal of
Management Studies, 4(8), 1801-1820.
144
Fields, A. (2009). Discovering statistics using SPSS (3rd ed.). Thousand Oaks, CA: Sage
Publications.
Flavell, J. H. (1976). Metacognitive aspects of problem solving. The Nature of
Intelligence (pp. 231-235). Hillsdale, NJ: Erlbaum.
Flavell, J. H. (1987). Speculations about the nature and development of meta-cognition.
Meta-cognition, Motivation and Understanding (pp. 21-29). Hillsdale, NJ:
Erlbaum.
Foreman, J. (2004). Game based learning: How to delight and instruct in the 21st century.
Educause Review, 39(5), 50-66.
Foundation (2012). Business simulation [Computer software]. Chicago, IL: Capsim

Management Simulations, Inc.
Galvin, T. (2003). 2003 Industry report: Training magazines 22nd annual comprehensive
analysis of employer-sponsored training in the united states. Training, 40(9), 1945.
Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative
potential in higher education. Internet and Higher Education, 1(7), 95-105.
Gilbert, C. J., & Bower, J. L. (2002). Disruptive change: When trying harder is part of the
problem. Harvard Business Review, 80(5), 94-101.
Gillespie, P. H. (1973). Learning through simulation games. Paramus, NJ: Paulist Press.
Goleman, D., Boyatzis, R. E., & McKee, A. (2004). Primal leadership: Learning to lead
with emotional intelligence. Boston, MA: Harvard Business School Press.
Goorha, P., & Mohan, V. (2010). Understanding learning preferences in the business
school curriculum. Journal of education for business, 85(3), 145-152.
Gosen, J. J., & Washbush, J. (2004). A review of scholarship on assessing experiential
learning effectiveness. Simulation & Gaming, 35, 270-293.
Gosling, J. &Mintzberg, H. (2004). The education of practicing managers. MIT Sloan
Management Review, 45(4), 19-22.
Greenblat, C. S. (1998). Designing games and simulations: An illustrated handbook.
Newbury Park, CA: Sage.
Gren-Yanoff, T. & Weirich, P. (2010). The philosophy and epistemology of simulation:
A review. Simulation & Gaming, 41(1), 20-50.
145
Hake, R. R. (1998). Interactive engagement versus traditional methods: A six-thousandstudent survey of mechanics test data for introductory physics courses. American
Journal of Physics, 66(1), 64-74.
Harry, L. (1971). Using simulation in the classroom. Feedback on Instructional
Simulation Systems, 127-134.
Hernandez, A. B., Gorjup, M. T., & Cascon, R. (2010). The role of the instructor in
business games: A comparison of face-to-face and online instruction.
International Journal of Training & Development, 14(3), 169-179.
Heyman, M. (1982). What are simulation games? Environmental Education Report,
10(5), 2-5.
Isaac, S. & Michael, W. (1997). Handbook in research and evaluation. San Diego, CA:
EDITS.
Jackson, J. R. (1959). UCLA executive decision game. Proceedings of the National
Symposium on Management. University of Kansas, VI-9-VI-15.
Jacobs, J. W., & Dempsey, J. V. (1993). Simulation and gaming: Fidelity, feedback, and
motivation. Interactive Instruction and Feedback (pp. 192-227). Englewood Hills,
New Jersey: Educational Technology Publications.
Kayes, D. C. (2002). Experiential learning and its critics: Preserving the role of
experience in management learning and education. Academy of Management
Learning and Education, 1(2), 137-149.
Kirby, D., & Garrison, R. D. (1990). Graduate distance education: A question of
communication? Canadian Journal of University Continuing Education, 16(1),
25-36.
Knowles, M.S. (1978). The adult learner: A neglected species, 2nd ed. Houston: Gulf.
Knowles, M. S. (1980). The modern practice of adult education: From pedagogy to
andragogy. Englewood Cliffs, NJ: Cambridge.
Knowles, M.S. (1984). Andragogy in action. San Francisco: Jossey-Bass.
Knowles, M. S. (1989). The making of an adult educator. San Francisco: Jossey-Bass.
146
Knowles, M.S., Holton, E.F., & Swanson, R.A. (2005). The adult learner: the definitive
classic in adult education and human resource development. Sixth edition.
Boston: Elsevier.
Knowles, M. S. & Hulda, F. (1973). Introduction to group dynamics. Chicago: Follett.
Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and
development. New Jersey: Prentice Hall.
Lederman, L. C. (1984). Debriefing: A critical re-examination of the post experience
analytic process with implications for its effective use. Simulation and Games, 15,
415-431.
Lederman, L. C. (1992). Debriefing: Toward a systematic assessment of theory and
practice. Simulation and Gaming, 23, 145-159.
Lefebvre, J. R. (1997). The battle for change. Mortgage Banking, 57(4), 1-7.
Lenth, R. V. (2001). Some practical guidelines for effective sample size determination.
The American Statistician, 55(3), 187-193.
Lewin, K. (1935). A dynamic theory of personality. New York, NY: McGraw-Hill.
Lewin, K. (1947). Frontiers in group dynamics. Human Relations, 1(2), 143-153.
Lewis-Beck, M. S., Bryan, A., & Lia, T. F. (2004). The Sage encyclopedia of social
science research methods. Thousand Oaks, CA: Sage Publications.
Lin, X. D., Hmelo, C., Kinzer, C., & Secules, T. (1999). Designing technology to support
reflection. Educational Technology Research and Development, 47(3), 43-62.
Mandl, H., Gruber, H., & Renkl, A. (1992). Problems of knowledge utilization in the
development of expertise. Flexibility in Training and Vocational Education (pp.
291-305).
McGrenere, J. L. (1996). Design: educational electronic multi-player games: A Literature
review. Technical Report 96-12. Department of Computer Science, University of
British Columbia.
Mclaughlan, R. G., & Kirkpatrick, D. (2005). Online text-based role play-simulation: The
challenges ahead. Simtect, Sydney: Simulation Industry Association of Australia.
147
McMasters, B. L. (1996). Effects of andragogical method of instruction on engineering

student performance and retention in the first year (Doctoral dissertation,
University of Arkansas, 1995). Dissertation Abstracts International, 56(11),
4294A.
Meier, R. C., Newell, W. T., & Pazer, H. L. (1969). Simulation in Business and
Economics. New Jersey: Prentice-Hall.
Merriam, S. B., & Brockett, R. G. (1997). The profession and practice of adult education:
An introduction. San Francisco: Jossey-Bass.
Miller, H. E., Schumann, P. L., Anderson, P. H., & Scott, T. W. (1998). Maximizing
learning gains in simulations: Lessons from the training literature. Developments
in Business Simulations and Experiential Exercises, 25(1), 217-223.
Montgomery, J. (1993). Reflection, meta-model of learning, and a proposal to improve
the quality of university teaching. Paper presented at the faculty of seminar on
Reflection and Improving the Quality of University Teaching and Learning.
Musselwhite, C. (2006). University education gets real: Business simulations, a learn-bydoing approach. American Society for Training and Development, 5(1), 57-59.
Norusis, M. J. (2008). SPSS 16.0 guide to data analysis. Upper Saddle River, NJ:
Prentice Hall.
Pearson, M., & Smith, D. (1986). Debriefing in experience-based learning.
Simulation/Games for Learning, 16(4), 155-172.
Piaget, J. (1973). Psychology and epistemology. New York: The Viking Press.
Pierfy, D. A. (1977). Comparative simulation game research, stumbling blocks, and
steppingstones. Simulation & Games, 8(1), 255-268.
Prensky, M. (2001). Digital Game Based Learning. New York: McGraw-Hill.
Quinn, C. N. (1994). Designing educational computer games. Interactive multimedia in
University Education: Designing for change in teaching and learning (pp. 45-57).
Amsterdam: Elsevier Science.
Rachal, J. R. (1995). Adult reading achievement comparing computer-assisted instruction
and traditional approaches: A comprehensive review of the experimental
literature. Reading, Research, and Instruction, 34(1), 239-258.
Rachal, J. R. (2002). Andragogys detectives: A critique of the present and proposal for
the future. Adult Education Quarterly, 52(3), 210-227.
148
Rafii, F., & Kampas, P. J. (2002). How to identify your enemies before they destroy you.
Harvard Business Review, 80(11), 115-123.
Raths, J. (1987). Enhancing understanding through debriefing. Educational Leadership,
45(2), 24-27.
Reigeluth, C. M., & Moore, J. A. (1999). Cognitive education and the cognitive domain.
Instructional-design theories and models (2nd ed). Hillsdale, NJ: Eribaum.
Rose, A. (1992). Framing our experience: Research notes on reflective practice. Adult
Learning, 3(4), 5.
Ruben, B. D. (1999). Simulations, games, and experience-based learning: The quest for a
new paradigm for teaching and learning. Simulation & Gaming, 30(4), 498-505.
Rubin, A., & Babbie, E. (2005). Research methods for social work (5th ed.). Belomont,
CA: Thomson.
Salmon, W. (2001). Reflections of a bashful Bayesian: A reply to peter lipton. In
explanation: theoretical approaches and applications. Kluwer Academic
Publishers.
Sandefer, J. (2009). Foreword to: the complete guide to simulations and serious games:
How the most valuable content will be created in the age beyond Gutenberg to
Google. San Francisco, CA: Pfeiffer.
Sandefer, J. (2007). Yales Taliban man and other tales. Academic Questions, 20(2),
113-121. doi: 10.1007/s12129-007-9003-0.
Sarin, S., & McDermott, C. (2003). The effect of team leader characteristics on learning,
knowledge application, and performance of cross-functional new product
development teams. Decision Science, 34(4), 707-739.
Scardamalia, M., & Beretier, C. (1991). Higher levels of agency for children in
knowledge building: A challenge for the design of new knowledge medial.
Journal of the Learning Sciences, 1, 37-68.
Scherpereel, C. M. (2005). Changing mental models: Business simulation exercises.
Simulation & Gaming, 36(5), 388-403.
Schon, D. (1983). The reflective practitioner: How professional think in action. New
York: Basic Books.
Schon, D. (1987). Educating the reflective practitioner: Toward a new design for
teaching and learning in the professions. San Francisco: Jossey Bass.
149
Schumpeter, J. A. (1989). The theory of economic development (R. Opie, Trans.), New
Brunswick, NJ: Translation. (Original work published 1911).
Segon, M., & Booth, C. (2009). Enhancing management competence through business
simulations. Journal of Business & Policy Research, 4(2), 111-123.
Sidor, S. M. (2008). The impact of computer based simulation training on leadership
development (Doctoral dissertation). Retrieved from ProQuest Dissertations and
Theses database. (AAI 3277804)
Smith, R. (2006). Technology disruption in the simulation industry. Journal of Defense
Modeling and Simulation, 1(3), 3-10.
Strawbridge, W. G. (1995). The effectiveness of andragogical instruction as compared
with traditional instruction in introductory philosophy courses (Doctoral
dissertation, University of Southern Mississippi, 1994). Dissertation Abstracts
International, 55(11), 3387A.
Summers, G. J. (2004). Todays business simulation industry. Skillful product
development. Simulation & Gaming, 35(2), 208-241.
Swanson, R. A. & Holton, E. F. III. (2005). Research in organizations: Foundations and
methods of inquiry. San Francisco, CA: Berrett-Koehler.
Szczurek, M. (1982). Meta-analysis of simulation games effectiveness for cognitive
learning. Dissertations Abstracts International, 43(4), 1031A.
Tanner, J.R., Stewart, G., Totaro, M.W., & Hargrave, M. (2012). Business simulation
games: Effective teaching tools or window dressing? American Journal of
Business Education, 5(2), 115-128.
Thatcher, C. (1990). Promoting learning through games and simulation. Simulation and
Gaming. 21(3), 262-273.
Vye, N., Schwartz, D. L., Bransford, J. D., Barron, B. J., & Zech, L. (1998). SMART
environments that support monitoring, reflection and revision. Metacognition in
Educational Theory and Practice, 305-347. Hillsdale, NJ: Erlbaum Associates.
Watson, H. J. (1981). Computer Simulation in Business. New York: John Wiley & Sons.
Wilcox, R. R. (2005). Introduction to robust estimation and hypothesis testing (2nd ed.).
Burlington, MA: Elsevier.
150
Williams, D. (2011). Impact of business simulation games in enterprise education. In:

Paper presentations of the 2010 University of Huddersfield Annual Learning and
Teaching Conference. University of Huddersfield, Huddersfield, 11-20.
Wilson, M., & Gerber, L. (2008). How generational theory can improve teaching:
Strategies for working with the millenials. Currents in teaching and learning,
1(1), 29-44.
Wolfe, J. (1985). The teaching effectiveness of games in collegiate business courses: A

1973-1983 Update. Simulation & Gaming, 16, 251-288.
Wolfe, J., & Crookall, D. (1998). Developing a scientific knowledge of
simulation/gaming. Simulation & Gaming, 29(1), 7-19.
Wolfe, J., & Luethge, D. (2003). The impact of involvement on performance in business
simulations: An examination of Goosens know little decision-making thesis.
Journal of Education for Business, 11, 69-74.
Xu, Y., & Yang, Y. (2010). Student learning in business simulation: An empirical
investigation. Journal of Education for Business, 85(4), 223-228.
151
APPENDIX A. PRE-EXPERIMENT SURVEY & ASSESSMENT
From Comp-XM Competency exam [Computer software], Capsim. Copyright

2012 by Capsim Management Simulations, Inc. Reprinted with permission.
1.
Which description best fits the Andrews company?

(For clarity:
- A differentiator competes through good designs, high awareness, and easy
accessibility.
- A cost leader competes on price by reducing costs and passing the savings
to customers.
- A broad player competes in all parts of the market.
- A niche player competes in selected parts of the market.)
A. Andrews is a broad differentiator
B. Andrews is a broad cost leader
C. Andrews is a niche differentiator
D. Andrews is a niche cost leader
2.
Review the FastTrack to determine Ferriss current strategy. Where will they
seek a competitive advantage?
From the following list, select the top five sources of competitive advantage
that Ferris would be most likely to pursue.
[ ] Seek high automation levels
[ ] Reduce labor costs through training and recruitment
[ ] Seek high plant utilization, even if it risks occasional small stockouts
[ ] Seek the lowest price in their target market while maintaining a
competitive contribution margin
[ ] Reduce cost of goods through TQM initiatives
[ ] Accept lower plant utilization and higher capacities to insure sufficient
capacity is available to meet demand
[ ] Increase demand through TQM initiatives
[ ] Seek excellent product designs, high awareness, and high accessibility
[ ] Offer attractive credit terms
[ ] Add additional products
152
3.
This year Baldwin achieved an ROE of 53.3%. Suppose the Board of

Directors of Baldwin mandates that management take measures to increase
financial Leverage (= Assets/Equity) next year. Assuming Sales, Profits, and
Assets remain the same next year, what effect would you expect this new
Leverage policy will have on Baldwin ROE?
A. Baldwin ROE will decrease.
B. Baldwin ROE will increase.
C. Baldwin ROE will remain the same.
4.
Chester currently has $1,027 (000) in cash and management has decided to
issue stocks and bonds worth an additional $20,000 (000). Assuming that
cash from operations will be the same for each of the following activities,
which activity exposes this company to the most risk of being issued an
emergency loan?
A.
B.
C.
D.
5.
Purchasing $22,000 (000) worth of plant and equipment

Liquidate the entire inventory
A $5 dividend
Retiring the oldest bond
Your Competitive Intelligence team reports that a wave of product liability

lawsuits is likely to cause Andrews to pull the product Able entirely off the
market this year. Assume Andrews scraps all capacity and inventory this
round, completely writing off those assets and escrowing the proceeds to a
settlement fund, and assume these lawsuits will have no effect on any other
products of Andrews or other companies. Without Andrews product Able,
how much can the industry currently produce in the Low Tech segment this
next year? Consider only products primarily in the Low Tech segment last
year. Ignore current inventories. Figures in thousands (000).
A.
B.
C.
D.
E.
8,400
9,130
6,350
14,000
18,260
153
6.
Americ is a product of the Andrews Company. The sales forecast for Americ
is 1,500 units. Americ wants to have an extra 10% of units on hand above
and beyond their forecast in case sales are better than expected. (They would
risk the possibility of excess inventory carrying charges rather than risk lost
profits on a stock out.) Taking current inventory into account, what will
Andrews Production After Adjustment have to be in order to have a 10%
reserve of units available for sale?
A.
B.
C.
D.
7.
1,650 units
1,620 units
1,630 units
1,480 units
Which description best describes the strategy for Baldwin Company?

(For clarity:
- A differentiator competes through good designs, high awareness, and easy
accessibility.
- A cost leader competes on price by reducing costs and passing the savings
to customers.
- A broad player competes in all parts of the market.
- A niche player competes in selected parts of the market.)
A. Baldwin is a niche differentiator
B. Baldwin is a broad differentiator with a product lifecycle focus
C. Baldwin is a broad cost leader
D. Baldwin is a niche cost leader
8.
Demand is created through meeting customer buying criteria, credit terms,

awareness (promotion) and accessibility (distribution). According to the High
Tech segment's customers, which of these products was the most competitive
at the end of last year?
A.
B.
C.
D.
Baker
Feast
Eyring
Fast
154
9.
Chester Corporation is considering adding capacity to their Custar product,

currently automated to 7.0. Assume:
- They will use the new capacity next year to make and sell 200 additional
units (000).
- Each unit of capacity will cost $26.00.
- Cedars price will be unchanged at $32.00.
- Material costs will remain $8.16 next year.
- Labor costs will remain $6.17 on first shift, and $9.25 on second shift.
- Bond interest will remain 10.7% next year.
- Depreciation will be straight line over 15 years.
- SG&A costs can be ignored because they would be the same with or without
the new capacity.
Which of the following tactics will yield the highest ROI in their first year of
production?
A. Buy 200 units of capacity. Finance the $5,200 purchase entirely with a
new bond.
B. Buy 200 units of capacity. Finance the $5,200 purchase entirely with a
stock issue.
C. Buy 100 units of capacity. Finance the $2,600 purchase entirely with a
new bond.
D. Buy 100 units of capacity. Finance the $2,600 purchase entirely with a
stock issue.
10.
What is the Working Capital of Ferris?

A.
B.
C.
D.
$20,510
$37,910
$43,679
$23,169
155
11.
Digby has a leverage of 1.6 This means that:

(Assume leverage is calculated as Assets/Equity)
A.
B.
C.
D.
12.
$1.80 of assets is funded with $1.00 of equity and $0.60 of debt.

$1.80 of assets is funded with $1.00 of debt and $0.60 of equity.
Assets are funded with 60% debt.
Assets are funded with 60% equity.
Last year the Chester company increased their equity. In 2014 their equity
was $28,363. Last year (2015) it increased to $37,673.
What are causes of change in equity? Check all that apply.
[
[
[
[
[
[
[
[
]
]
]
]
]
]
]
]
Profits of $13,455
Depreciation of -$4,185
Issue and retirement of stock
Dividend payment of $1,562.
Change in inventory of -$832.
An accounts payable change of $392.
An investment in plant and equipment of $5,050.
A change in cash of -$1,873.
156
APPENDIX B. POST-EXPERIMENT SURVEY (Dwyer & Ganster, 1991)

From The effects of job demands and control on employee attendance and
satisfaction, by Dwyer & Ganster, Journal of Organizational Behavior.
Copyright 1991 by John Wiley and Sons, Ltd. Reprinted with permission.
For the purpose of this research, when answering the following questions, you should
consider your work or current position at the company. Below are listed a number of
statements which could be used to describe a job. Please read each statement carefully
and indicate the extent to which each is an accurate or an inaccurate description of your
job by circling a number below each statement.
1. How much control do you have over the variety of methods you use in
completing your work?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
2. How much can you choose among a variety of tasks or projects to do?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
3. How much control do you have personally over the quality of your work?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
4. How much can you generally predict the amount of work you will have to do
on a given day?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
5. How much control do you personally have over how much work you get
done?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
6. How much control do you have over how quickly or slowly you have to
work?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
157
7. How much control do you have over scheduling and duration of your rest
breaks?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
8. How much control do you have over when you come to work and leave?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
9. How much control do you have over when you take vacation or days off?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
10. How much are you able to predict what the results of your decisions you
make on the job will be?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
11. How much are you able to decorate, rearrange, or personalize your work
area?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
12. How much can you control the physical condition of your work station
(lighting, temperature)?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
13. How much control do you have over how you do your work?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
14. How much can you control when and how much you have to interact with
others at work?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
158

Innov Manag - ProQuest - Diss19 PDF

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Innov Manag - ProQuest - Diss19 PDF

Uploaded by

Copyright:

Available Formats

ONLINE BUSINESS SIMULATIONS: A SUSTAINABLE OR DISRUPTIVE

INNOVATION IN MANAGEMENT EDUCATION?

MARY F. WHITMAN, DBA, Faculty Mentor and Chair

William A. Reed, PhD, Dean, School of Business and Technology

A Dissertation Presented in Partial Fulfillment

UMI Number: 3517084

All rights reserved

UMI Number: 3517084

All rights reserved

Jason Scott Earl, 2012

Introduction to the Problem

Statement of the Problem

Purpose of the Study

Significance of the Study

Sustaining vs. Disruptive Innovation

Assumptions and Limitations

Nature of the Study

Organization of the Study

CHAPTER 2. LITERATURE REVIEW

Nature of Business Simulations

Benefits of Experiential Learning

Educational Effectiveness of Simulations

Simulations and Corporate Training

Business Simulations as Knowledge Application Tools

Knowledge Application & Simulation Performance

Reflection in Experiential Learning

Reflection Using Business Simulations

Debriefs Within Business Simulation

Business Simulations as a Disruptive Innovation

Business Simulations as a Disruptive Force Today

Andragogical Support for Business Simulations

Primary Research Question

General Linear Model for the Primary Research Question

Null and Alternative Hypotheses

Justification and Methodology

Research Design Strategy

Instrumentation and Measures

The Business Simulation

Dwyer and Gansters (1991) Autonomy/ Work Control

Business Simulation Performance Scores

Validity and Reliability

Primary Research Question

Secondary Research Questions

Supporting Research Questions

CHAPTER 5. DISCUSSION, IMPLICATIONS, RECOMMENDATIONS RESULTS

General Discussion &Theoretical Implications

Limitations and Recommendations for Future Research

APPENDIX A. PRE-EXPERIMENT SURVEY & ASSESSMENT

APPENDIX B. POST-EXPERIMENT SURVEY & ASSESSMENT

Table 2. Instructional Methods Used by U.S. Organizations

Table 3. Null and Alternative Hypotheses

Table 4. Descriptive Statistics for Survey Items (N = 65)

Table 5. Descriptive Statistics for Dependent Variables (N = 65)

Table 6. Test of Homogeneity of Variances for Dependent Variables

Table 7. Test of Normality for Change in Business Knowledge

Table 8. Test of Normality for Change in Business Acumen

Table 9. ANOVA for Pre/Post-Test Results

Table 10. Tests of Between Subject Effects for Experimental Treatment

Table 11. Pearson Correlation Coefficients

Table 12. Group Statistics for Dependent Variables

Table 13. Independent Samples Test for Equality of Means

Table 14. Analysis of Variance Between Groups