Principles and Practices of Outcomes Based Training & Education

Riccio, G., & Darwin, M. (2010). Principles and Practices of Outcomes Based Training & Education.
In: Riccio, G., Diedrich, F., & Cortes, M.

(Eds.). An Initiative in Outcomes-Based Training and Education: Implications for an Integrated Approach to Values-Based Requirements
(Chapter 3). Fort Meade, MD: U.S. Army Asymmetric Warfare Group. [Cover art by Wordle.net represents word frequency in text.]
Principles and Practices of OBTE 50
Asymmetric Warfare Group
Chapter 3. Principles and Practices of Outcomes Based Training & Education

Gary Riccio and Morgan Darwin
The Wexford Group International

The approach of the U.S. Army Asymmetric Warfare Group (AWG) to Outcomes-Based Training
and Education (OBTE) assumes that teaching in the information age must be different from
predominant approaches in the industrial age (Darwin, 2008a, b). Whether or not education takes
place in an institutional setting, the responsibility for both teaching and learning must become
considerably more decentralized than in the 20
th
century. OBTE further assumes that educational
practice in the Army must accommodate the fact that teaching and learning already is more
decentralized, and is becoming increasingly so, because of the social and semantic web
(Mikroyannidis, 2007; Paavola & Hakkarainen, 2005; Scardamalia & Bereiter, 2006).
Accordingly, the Institutional Army should be proactive in assessing its preparedness for the
demographic fact that the next generation of Soldiers will be comprised of millennials for
whom the ubiquitous decentralization offered by the World Wide Web is a basic fact of life
(Carlson, 2005; Howe, Strauss, & Matson, 2000). The initiative of the AWG in OBTE is a
response to this need for responsible decentralization of teaching and learning.

Responsible decentralization implies personal autonomy that must be bounded or grounded in
some way. In Army training and education, the appropriate grounding is in Army Values and
other cultural imperatives. OBTE helps identify how Army training and education can be
grounded in this organizational culture. Partially for this reason, OBTE is what many would
consider to be good training; however, such a characterization begs all the important questions.
Thus, the intent of this chapter is to elucidate specifically what good means in the context of
Army training and education, and how it relates to the emerging understanding of Full Spectrum
Operations. The result of this review is a better understanding of the established and emerging
instructional practices that should be sustained and those that should be improved to meet the
needs of the future. It provides the basic facts for a deeper scientific inquiry into the social and
psychological foundations of OBTE as an approach to teaching and learning.

3.1 Multifaceted Inquiry

Over the course of the investigation, a variety of methodologies were employed iteratively and
concurrently to help achieve a deeper and multifaceted understanding of the principles and
practices of OBTE (CMMI Product Team, 2009; Cresswell, 1998; D. Kirkpatrick, 1994; Miles &
Huberman, 1994; Msse, Moser, Stokols, et al., 2008; Tashakkori & Teddlie, 1998; Trochim,
2004; Trochim, Marcus, Msse, Moser, Weld, 2008). In addition to the COMPASS process
(Chapter 2), we analyzed surveys of participants in the AWGs Combat Applications Training
Course (CATC) (Chapter 6), observed CATC-trained instructors and their students in rifle
marksmanship training (Chapters 8 and 9), surveyed alumni of CATC about its downstream
impact in theater (Chapter 10); observed potential applications of OBTE in a classroom-based
instructor education course (Chapter 13), and identified critical issues pertaining to organizational
change (Chapter 14) and leader development (Chapter 15).

Our development of a grounded theory for OBTE also has been informed by several other
concurrent activities. In particular, we interviewed progenitors and practitioners of OBTE and
discussed relevant scientific research with them, attended AWG workshops and seminars on
OBTE, reviewed documents from the AWG, observed AWG instructors in CATC, participated in
CATC, interviewed stakeholders in the Army Training and Doctrine Command (TRADOC), and
51 Riccio & Darwin

participated in various non-AWG conferences and workshops in which OBTE was addressed.
These sources of evidence are briefly summarized below. The COMPASS process and its
products guided and facilitated qualitative crystallization across these multiple methodologies
(Camic et al., 2003; Denzin & Lincoln, 2003; Ellingson, 2009; Wise, 1980). The crystallization
involved more than constant comparison among different kinds of evidence. There also was a
balance and tension among the perspectives of a variety of experts with substantially different
backgrounds in a dialectic that extended over more than a year (Figure 1). The descriptions of
OBTE provided in this chapter reflect the claims, insights, and clarifications that emerged from
this multifaceted inquiry.

Figure 1. Balance and tension of different perspectives in the collaborative inquiry.

3.1.1 Interaction with Progenitors of OBTE

The military experts involved in our collaborative inquiry included both active duty and retired
Army noncommissioned officers (NCO) and commissioned officers associated with the AWG
who had been involved in the development of their initiative in OBTE. The most striking aspect
of this collaboration was that it did not involve a common division of labor across topics based on
assumed relevance of background. Instead, all perspectives were brought to every aspect of the
inquiry on a level playing field in which there was no place for assumptions of hierarchy or
dominance relations among different types of experience or expertise. The role of the technical
experts was not merely to transcribe the opinions of the operational experts but, instead, to
interpret their perspectives in terms of familiar scientific assumptions and paradigms. Similarly,
the operational experts interpreted the putatively relevant science in terms of exigencies of the
Full Spectrum Operations and preparation of Soldiers for it.

In additional to seeking common ground, the intent was to influence the sense making within the
multidisciplinary team in ways that were visible and traceable over an extended period of time.
The elaboration of the principles, practices and measures of OBTE summarized in this chapter
Officer
Perspective
NCO
Perspective
Engineering
Perspective
Scientific
Perspective
SOF
Perspective
Perspective of
Conventional
Forces
(see also Chapters 2, 4, and 5) are the result of these extended and iterative deliberations. This
process of crystallization is typical of the approach we have used in other research on Army
training and education (e.g., Riccio, Sullivan, Klein, Salter, & Kinnison, 2004; Sidman, Riccio,
Semmens, et al., 2009). Moreover, this kind of collaboration is representative of the
organizational environment of the AWG from which the initiative in OBTE emerged.

3.1.2 AWG Documents on OBTE

Various members of the AWG had prepared documents for internal coordination and for
presentation to stakeholders outside the AWG. Four documents were particularly influential in
our identification of assumptions and claims about OBTE (Darwin, 2008a, b; Cornell-dEchert,
2009a, b). These documents have been widely distributed among stakeholders in Army training
and education. Over a period of two years, one or more of these documents has been presented in
numerous forums in which there was opportunity for feedback and discussion of the meaning and
implications of OBTE at particular sites and in particular programs. While there were many
contributors to these documents, there was a clear first author and main contributor for each set of
two documents. Both of these authors are progenitors of OBTE.

One of the authors of this chapter, a retired Command Sergeant Major from the U.S. Army
Special Operations community, took the lead in preparing and maintaining a working document
comprised of approximately 100 PowerPoint slides. Darwin (2008a) is an early version, and
Darwin (2008b) is a more recent version of this document (the document is updated periodically).
The intended audience is primarily senior leaders associated with specific TRADOC units and
programs of instruction. The early version (Darwin, 2008a) laid out the purpose, vision, and
philosophical underpinnings of CATC. The later version (Darwin, 2008b) extended the concepts
more explicitly beyond marksmanship training to OBTE in general and revealed the relationship
between OBTE and doctrinal requirements for adaptability in Full Spectrum Operations. A retired
U.S. Army Lieutenant Colonel took the lead in preparing documents intended primarily for
personnel who are responsible for design, development, and implementation of specific Army
courses. One of these documents (Cornell-dEchert, 2009a) presents twenty-six articles for
OBTE that describe its intent and attempts to clarify misconceptions about the place of OBTE in
broader Army culture and practices. The other document (Cornell-dEchert, 2009b) is an
implementation guide that begins to translate the twenty-six articles into practices and
considerations for adapting the approach for existing programs of instruction. It is intended as a
living document to guide peer-to-peer sharing of best practices and lessons learned.

3.1.3 Collaborative Reflection on Participant Observation in CATC

Four members of the research team participated in the one-week CATC as students. One pair of
researchers took the course one week, and another pair took the course another week. Each pair
included one scientist and one person with military experience. The researchers in each pair were
encouraged to discuss their experiences with each other as the week unfolded and to document
key experiences and insights to the extent possible after each day of instruction. After completion
of the course, another member of the research team interviewed the participant-observers. The
interviews were conducted with two participant observers at a time, one from each of the prior
pairings; the two scientists were interviewed together, and the two retired military officers
together (one had been an Army Infantry Captain, and the other was a retired U.S. Marine Major).
The primary role of the interviewer was to stimulate dialog between the interviewees in which
they would compare and contrast their respective experiences and the meanings they made of it.
This dialog facilitated subsequent discussions with progenitors of OBTE, instructors in CATC,
and participants in CATC.
53 Riccio & Darwin


3.1.4 Interaction with Stakeholders

There were two main contexts in which there were opportunities for substantial and extended
interaction with OBTE stakeholders in Army training and education (See Chapter 1, section
1.3.2). One context involved the AWG seminars and briefings to individuals at sites associated
with particular programs of instruction. Members of the research team attended several of these
briefings. Another context was our preparation for field-based observations of the implementation
of OBTE at Fort Benning, Fort Jackson, and Fort Sill (see, e.g., Chapters 8, 9, and 13). We also
have interacted with many other stakeholders through various AWG events (e.g., AWG, 2009;
see Chapter 11, section 7.2.3).

3.2 Essential Characteristics of OBTE

The most important contributions of the AWGs initiative in Outcomes-Based Training &
Education (OBTE) and the associated research are the claims and evidence that (a) the gap
between Army instruction and Full Spectrum Operations can be viewed in terms of the
relationship between a singular focus on tangible, near-term objectives and the resulting effect on
enabling intangible attributes valued by Army leaders, (b) everything that is done in Army
training and education has an impact on this relationship whether one addresses it or not, (c) this
impact can be understood and described thus allowing leadership to make informed decisions
based on verifiable observations and valid scientific reasoning, (d) OBTE is an approach that can
be utilized to inform leaders at all levels on the direction their unit is moving in the development
of leadership traits, problem solving skills and intangible attributes, (e) the approach can be
verified and validated, and (f) it can influence without necessarily replacing any extant
instructional methodology (M. Darwin, personal communication, June 30, 2009).

3.2.1 The Meaning of Developmental is a Critical Difference

OBTE is an approach to training and education that focuses on development of the individual in
relation to basic needs of the individual as well as associated cultural values and objectives of the
individuals occupation (Darwin, 2008a, b; Cornell-dEchert, 2009a, b). The meaning of
developmental thus is fundamentally different from its typical connotations in training and
education, which emphasize individual differences and, in many cases, refer to remedial learning
in specific subject matter. This is not to say that OBTE is unconcerned with Soldiers mastery of
specific knowledge and skills. An assumption of OBTE is that longer-term developmental
outcomes can be pursued while Soldiers are being prepared for specific tasks (Cornell-dEchert,
2009b). In OBTE, immediate and long-term objectives are not considered fundamentally
inconsistent. There may be tradeoffs about nested instructional objectives to be made on the scale
of hours in particular learning events to the scale of months or years in curriculum design, but this
more likely would be a question of return on investment rather than incompatibility (see Chapter
5 and Epilogue). More generally, OBTE is an approach in which persistent awareness of long-
term outcomes, including but not limited to the explicit learning objectives for an event,
influences the planning and execution of instruction in important ways.

Why consider Soldier development in training and education when it seems to be the purview of
other programs, the responsibility of other people, or a focus for time periods outside of formal
learning events? The reason is simple and inescapable. Every interaction with a student influences
long-term development whether or not it is specifically considered in planning or execution of
instruction. The profundity of this human factor in training and education has been recognized
from antiquity to modern times (e.g., Aristotle, trans. 1925; Augustine, trans. 1953; Bandura,
1977; Dewey, 1915/2008; Guay, Ratelle, & Chanal, 2008; James, 1890/1907; Keatinge, 1896;
Lave & Wenger, 1991). OBTE takes into consideration that an individual can be changed by the
learning experience in ways that transcend the particular knowledge or skills acquired during a
learning event.

On this philosophical foundation in learning and development, OBTE stridently rejects any
separation of learning and doing. This is not merely a superficial claim about experiential
learning (i.e., that one learns by doing). OBTE claims that, while applying knowledge or skill in
practice, individuals also can learn about other things that can be done or that can be known.
Individuals weave performatory behavior with exploratory behavior to achieve a more
differentiated and more refined understanding of their capabilities and of the environment in
which they are situated (J. Gibson & E. Gibson, 1955; E. Gibson, 1988, 1991). This weave of
performance and discovery essentially is an inter-temporal compromise between immediate and
long-term goals of behavior (cf., Glaser, 2002; Riccio, 1993a). In general, inter-temporally
directed behavior often involves minimal tradeoffs to the extent that exploratory behavior can be
nested within performatory behavior with little or no interference (cf., Feldbaum, 1965; Filatov &
Unbehauen, 2004; Riccio, 1993a, b; Riccio & McDonald, 1998). This is addressed in Chapters 4
and 5 in the context of individual adaptability and collective agility.

OBTE claims that there are ubiquitous opportunities for instructors and students to engage in
thinking or activity beyond the task at hand, amid the activities required by the task at hand.
These opportunities exist on the scale of seconds to minutes. Moreover, OBTE claims that
instructors and students often are engaged in such apparently incidental behavior, that is, behavior
not prescribed by the task at hand. The less esoteric of version of this claim is that human beings
are not inanimate objects or machines even when they are implicitly or explicitly treated as such
in the development or execution of instruction. Our informal observations in the field revealed
that, when students are engaged motivationally in a learning event, their thinking and behavior
outside the task at hand relates to the task at hand. For example, under favorable conditions,
students will spontaneously talk with each other about performance on a task even if they are not
required to do so. Such ad hoc engagement in learning does not introduce any delays in the
learning event. Under less favorable conditions, something unrelated to the learning event will
capture the attention of students in the time periods between behavioral scripts. In both cases,
habits of learning and engagement are nurtured, for good or ill.

Rarely if ever, is each student involved in scripted behavior at every second of every learning
event, especially when the student-to-instructor ratio is high. An assumption of OBTE is that,
during waiting time (e.g., in the ubiquitous queues), students are making sense of their experience
in a learning event. This does not necessarily involve overt or even explicit intentional analysis of
the event. In the extreme, it may involve nothing more elaborate than the conclusion that there is
little sense to be made of the event but the consequences for human development are no less
profound. OBTE seeks to engage such metacognitive activity to have a more purposeful influence
on development. As discussed below, this can be achieved in the approach to planning and
execution of instruction without necessarily adding tasks for the instructor or the student. At a
high level, OBTE influences metacognitive activity by addressing the reasons behind the
structure of a learning event and the constraints on its validity. It also does this by providing
models (i.e., real-time examples) of collaborative reflection and reinforcing it wherever
appropriate. This does not require an instructor to be everywhere at all times or to help every
student exploit every opportunity for useful reflection. It merely requires some examples of
collaborative reflection that students can model whether they participated in the examples or
experienced them vicariously.

55 Riccio & Darwin

Even occasional engagement of students on a metacognitive level is powerful because this is
where ownership of their own learning and development resides. This should not be interpreted in
ways that imply an infinite regress in analysis of ones own thinking and behavior.
Fundamentally, it is about the deeply personal struggle between the known and the unknown,
between the controllable and the uncontrollable. It is where passion and faith intermingle with
reason and belief (Figure 2) (Kierkegaard, 1843). Through development, it is where individuals
internalize previously external sources of motivation and where they make meaning of the
balance between what they can control and what they are either unwilling or unable to control,
that is, with respect to which they yield or cede power (cf., Aristotle, trans. 1925; Augustine,
trans. 1991; Hoffman, Yang, Kaklauskas, & Chan, 2009; Idel & McGinn, 1999; James,
1902/1982; Maslow, 1964; Plato, trans. 1892a; Solomon, 2004). Engaging Soldiers on this level
helps them find a balance between choice and responsibility that is personally sustainable and
more likely to help them be successful in the ambiguous and changing conditions of Full
Spectrum Operations. Students will find their own levels but it is in the interest of future
commanders, peers, and subordinates for Soldiers as students to understand and continually
develop their relationships with various sources of power that influence their decisions and
actions, beyond the formal or explicit lines of authority within their units. For this reason, as
discussed below, OBTE motivates the development and use of formative measures that engage
students and instructors on this level (see Chapter 2).

Figure 2. Intangible metacognitive and metaphysical influences on capabilities for thinking,
decision-making, action, and their development.

With respect to development and assessment, OBTE is related to approaches in education that
focus on progress maps, especially those that consider multidimensional constructs including
latent variables (Pellegrino, Chudowsky, & R. Glaser, 2001). The latent variables of OBTE,
however, are not mathematical constructs or other abstractions derived computationally from
quantitative data. They are based on assumptions about shared values and qualitative evidence
about an individuals values that manifest in both mundane and extraordinary situations (cf.,
Aspinwall, & Staudinger, 2003; Keyes & Haidt, 2003; Peterson & Seligman, 2003; Riccio et al,
2004; Webster, 2001). OBTE considers Soldier development to be the most powerful focus for
any approach to Army training and education insofar as it addresses attitudes and values that
guide life-long learning and self-development. More than anything else, this focus engages
instructors as leaders and leader developers. In this sense, OBTE essentially is both values-based
and leadership-based training and education. It thus helps bring what the Army does best into the
realm of training and education (United States Army Training and Doctrine Command
(TRADOC], 2009). It seeks to give training and education a higher status and sense of calling
(Riccio et al., 2004).

3.2.2 The Definition of Outcomes is a Critical Difference

The current revision of TRADOC Regulation 350-70 tries to provide guidance on the selection
and use of a variety of instructional methodologies and associated theories of education. In this
respect, it is important to note that OBTE is more general and inclusive than a particular method
of instruction or a prescriptive set of instructional techniques. At the same time, from the
perspective of OBTE, not all methods are equal. OBTE is not, for example, consistent with
outcomes based methods of instruction that focus exclusively on course-specific objectives.
OBTE should not be confused with constructs that are based on a definition of outcomes that is
more like the definition of common learning objectives and that implies a different developmental
approach. While the outcomes based education movement of the 1990s (see e.g., Spady &
Marshall, 1991; Spady, 1994, 2009), for example, stressed more of a focus on the objective
output than on an a priori script or input focus for an instructional event (Figure 3), it is
inconsistent with OBTE on a fundamental level in that it rejects values, attitudes, internal mental
processes or psychological states of mind as outcomes (Spady, 1994, 2009).

While reverse planning is a common aspect of different approaches to outcomes-based
instruction, OBTE starts with different kinds of outcomes, ones that relate to attitudes and
attributes developed in any learning event whether or not one explicitly recognizes it. Another
fundamental difference is that OBTE is developed for situations, as in the Army, in which there
are practical constraints on time and resources. In the Army, there is not the luxury to allow
students to achieve a pre-specified level of performance at their own pace. OBTE requires an
inter-temporal focus that balances immediate learning objectives of the task at hand with longer-
term habits of thinking and action that transfer to different situations. In this respect, it is not wise
to give high performers short shrift so that relatively low performers can be given more assistance
with their performance. All Soldiers can benefit from being engaged on the level of
metacognitive development (section 3.2.1) irrespective of their level of knowledge or skill.

Currently there is considerable confusion in the use of the terms objectives and outcomes.
Sometimes they are used synonymously, sometimes not, and there is a conspicuous lack of
consistency in the use of each of these terms. An outcome in OBTE is not the same as a terminal
learning objective that typically is the focus of Army training and education. We equate
objectives with intended results of a well-defined component of training and education, whether
short-term or long-term. We refer to outcomes as all results of a well-defined component of
training and education, whether intended or not. In general, intentions are specified explicitly
only with respect to short-term results, at least in any way that is verifiable or measurable; thus
use of the term objectives often connotes short-term results. When verifiability of objectives is
considered, the focus is almost exclusively on tangible results. The use of the term outcomes in
OBTE emphasizes long-term results that, of necessity, are mostly intangible but that also can
become verifiable and measurable, in conjunction with traditional short-term results.

One way to clarify the use of the terms, objectives and outcomes, in OBTE is to begin a rigorous
and traceable inquiry into the relationship between verifiable intangible long-term results
(outcomes) and verifiable tangible short-term results (a priori objectives and ensuing outcomes).
The first step in this inquiry is to identify ways to talk about and measure both tangible and
intangible results in which they are commensurable. Then one can begin to identify the
antecedents of long-term results that can be observed at the same time short-term results are
57 Riccio & Darwin

observed (e.g., Deci & Ryan, 2008; Peterson & Seligman, 2004; Rick & Loewenstein, 2008;
Riccio et al., 2004). This would enable identification of conflicts and synergies in the means to
achieving both short-term and long-term results. We believe that a promising line of inquiry for
OBTE is the role of motivation and emotion in tradeoffs involving intangible long-term outcomes
that influence perception and action (Camber, Loewenstein, & Prelec, 2005; Loewenstein &
Lerner, 2002; Loewenstein, Weber, Hsee, & Welch, 2001; cf., Deci & Ryan, 2008; Ryan & Deci,
2008; Peterson & Seligman, 2004). As demonstrated by the measures of Chapter 2, intangibles
should not necessarily be interpreted as fuzzy concepts. They are only intangible in terms of their
manifestations under the uncertain conditions of the future (cf., Rick & Loewenstein, 2008).

Figure 3. Highly stylized depiction of three approaches to training and education. Dotted
horizontal line represents target performance associated with the terminal learning
objective for a learning event. Green lines represent relatively fast learners; red lines
represent relatively slow learners. Gray band indicates uncertainty about retention of
skills or knowledge or transfer of learning to a new situation.

It may be helpful to consider the antecedents of long-term outcomes as analogous to enabling
learning objectives typically associated with terminal learning objectives in Army training and
education. This suggests a potential synthesis by recognizing that objectives are nested (cf.,
Feldbaum, 1965; Filatov & Unbehauen, 2004; Glaser, 2002; Riccio, 1993a,b; Riccio &
McDonald, 1998; Riccio & Stoffregen, 1991), perhaps in ways that are directly analogous to
nested intent statements in mission command (Cornell-dEchert, 2009b; cf., Appendix C;
Freeman et al., 2008). Longer-term attitudes and values can be addressed, along with immediate
learning objectives, in the context of the same learning event. This approach seems timely given
the exigency in the most recent version of Army leader development strategy to integrate leader
development with training and education (TRADOC, 2009). Toward this end, we believe it is
essential to make more progress in identifying scientific underpinnings for OBTE. The
development of measures for instructors helped us recognize some of these connections (see
Chapter 2). In this chapter, we begin to describe the research that followed the development of
measures, that helped us refine and organize them, and thus that helped us realize a theoretical
commitment for a rigorous line of inquiry (see Chapters 4 and 5).

3.2.3 The Emphasis on Values and Causally Potent Intangibles is a Critical Difference

The principles of OBTE (e.g., Darwin, 2008a, b) reveal the kinds of outcomes that can be
addressed during any learning event. The principles also suggest why these outcomes are
important with respect to retention and transfer of learning to operational environments (i.e., sub-
bullets in list of principles below).

Training to Grow Problem Solving
o teach Soldiers to learn for themselves the skills necessary to the success of
their mission, within an established framework of knowledge
Training to Increase Intangibles
o develop intangible attributes like confidence, accountability, initiative, judgment
and awareness; reinforce positive character traits
Training to Increase Understanding and Awareness
o teach through contextual understanding of the task and its mission application
Training to Increase Deliberate Thought
o condition Soldiers to exercise always a deliberate thought process (evaluation,
judgment and decision) while under stress
Training to Improve Combat Performance
o condition Soldiers to overcome the psychological and physiological effects, and
the physical requirements of combat
A focus for our interactions with the progenitors of OBTE was to develop a better understanding
of the causal potency of these principles. The most significant discussions revolved around the
so-called intangibles of OBTE. This collaborative inquiry involved constant comparison with
diverse lines of research in psychology and related disciplines that have a long history of
scholarship with respect to intangible attributes and latent causal variables. While such
interdisciplinary connections can be as complex as one would like them to be, we chose to
identify a simple framework for sustained trans-disciplinary communication and collaboration
that is well grounded in a variety of disciplines across the physical, biological, behavioral, and
social sciences. In particular, we chose a triadic framework within which a balance and
equilibrium of two causal factors is manifested with respect to a third factor (see e.g., Burke,
1985; Lorentz, Einstein, Minkowski, & Weyl, 1923/1952; Glass & Mackey, 1988; Goldberg,
1986; Heisenberg, 1958; Lanczos, 1970; Wolfram, 2002). The point here is not to equate our
current scientific understanding of OBTE with these well-developed branches of science. Instead,
the point is that considerable explanatory power with respect to complex phenomena can be
achieved from relatively simple concatenation operations if one starts with primitives and
59 Riccio & Darwin

principles that are trustworthy. Such confidence typically derives from a history of careful
observation of phenomena from which primitives and principles are abstracted.

We believe a triadic framework is the simplest possible expression of the essential features of
OBTE and thus is most useful in translating theory into practice. The primitives and principles in
this framework are based on decades of experience of a variety of people who have approached
Army training and education from different albeit converging perspectives (Figure 4). The
nascent concatenation operations of balance and tension between primitives are based on this
experience as well as on themes from a variety of philosophical perspectives that have stimulated
thought and dialogue for millennia. Some of the important implications of a triadic framework are
discussed further in Chapters 4 and 5 and in the Epilogue.

Figure 4. Triadic balance and equilibrium among intangible personal attributes
and the associated scaffolding for capabilities of the human dimension in Full
Spectrum Operations and competence in trans-extremis environments (e.g.,
transitions between strategically significant combat and noncombat operations).

Early on in the collaborative inquiry with progenitors of OBTE, we identified three intangible
attributes as foundational for OBTE. Confidence, initiative, and accountability became the
organizing framework for outcomes that are given special attention in OBTE. We believe that any
two of these attributes has an impact on the third attribute but, for power and simplicity, we
emphasize the balance between initiative and accountability. The power of this dyad can be
understood in terms of the interplay between choice and responsibility (Aristotle, trans. 1925;
Augustine, trans. 1991; Hickman, L.A. & Alexander, 1998; James, 1890; Solomon, 2004) or
between agility and the variable contextual constraints of Full Spectrum Operations (HQDA,
2003; 2005). This triad emphasizes the urgency of choice within contextual understanding of
military situations, with particular attention to the powerful aspect of initiative coupled to moral
accountability, as compared to ethical responsibility (cf., Lvi-Valensi, 2006; Clausewitz, 1976
trans.; Kierkegaard, 1843). There are any number of motivational contrivances and tricks that can
be employed to make individuals more flexible or to display more initiative but this is not what
the Army or the Nation needs of its Soldiers. Soldiers need to be adaptable in initially ambiguous
and in changing conditions. Adaptability implies that their decisions and actions are appropriate
given the context. At the same time, the Army needs Soldiers to exhibit cunning and not merely
to be driven by context. There is a need for units to be agile, proactive rather than reactive, and to
recognize the functional potency of human agency in social structures.

The balance of agility and context is not merely about recognition of the limits of power. It also is
about an appreciation of the consequences of exercising power (cf., Clausewitz, 1976 trans.).
Consequences include longer-term effects, including unintended effects, as well as impact on the
task at hand. OBTE assumes that habits of such inter-temporal awareness and thinking can be
developed in any and all learning events, in formal learning events or informal life-long learning.
This is an important determinant of the confidence the Army needs in its Soldiers, not a false
confidence bred from ignorance, invalid assumptions, or unexamined misinterpretations. This is
not to imply that Soldiers must know everything at all times. To the contrary, OBTE assumes that
habitual experience with the interplay between initiative and accountability helps Soldiers come
to terms with uncertainty and the application of knowledge and skill amid uncertainty. While this
is a deeply personal experience, in OBTE, it is not a solitary endeavor. OBTE builds habits of
collective struggling with uncertainty and collaborative experience even in the context of
ostensibly individual tasks.

There are other intangibles emphasized in OBTE. To maintain simplicity, such intangibles are
envisioned in terms of a triadic infrastructure (Figure 4). This conceptual nesting should be
viewed as analogous to the grouping of causal factors in a factor analysis. The nesting is not
intended to represent all the relationships or to explain all the variance. It is intended to be the
least arbitrary grouping that achieves the simplest possible description of a myriad of concepts.
By simplest, we mean the easiest to put into practice. In this sense, every attribute can be
addressed through the interplay of two other attributes.

We explored a triadic infrastructure in another project to simplify inquiry into the inculcation of
Warrior Ethos in Initial Entry Training (Riccio et al., 2004; Bruny, Riccio, Sidman, Darowski, &
Diedrich, 2006; Appendices D and E). We found that this framework also helped achieve a useful
mapping among a variety of values-based concepts in current Army doctrine. Again, while such
relationships are not intended to suggest one-to-one mappings or a full understanding of each
concept, they do alleviate unnecessary complexity in important sets of Army requirements and
thus suggest ways in which the requirements have actionable implications for Army training and
education. Along those lines, the mapping has recently been extended to yet another set of values-
based requirements in Army training and education (Sidman, Riccio, Semmens, et al., 2009). The
composite triadic mapping is indicated in Table 1.

Table 1. Triadic mapping among plethora of concepts in Army training and education.
OBTE Attributes of Tenets of Army Pentathlete
Intangibles Warrior Ethos
Warrior Ethos Values Characteristics
Confidence Perseverance
Courage Warrior Leader
Initiative Adaptability Respect
Judgment Prioritization Mission First Duty Critical Thinker
Deliberate thought Tradeoffs
Never Accept Defeat Honor Creative Thinker
Accountability Sense of Calling Integrity Leader Developer
Discipline Responsibility Never Leave Fallen Selfless Service Resource Manager
Awareness Dependence
Never Quit Loyalty Ambassador

61 Riccio & Darwin

The triadic mapping between the intangibles of OBTE and the attributes of Warrior Ethos is most
striking. This is important because the focus of the project that generated the attributes of Warrior
Ethos was to provide valid and verifiable ways to assess progress in inculcating Warrior Ethos in
Soldiers during Initial Entry Training. A promising line of inquiry is the adaptation and utilization
of the attributes of Warrior Ethos as measures of long-term development, perhaps even standards,
that relate to values-based requirements in current Army doctrine (see Epilogue).

3.2.4 The Meaning of Experience is a Critical Difference

In OBTE, experience refers to more than just a personal history of participation in learning; that
is learning by doing. To be sure, participation in ones own learning is central to OBTE; however,
participation also is the best way to experience learning and to find meaning in it. Experiencing
the conditions, process, and consequences of learning is important. In other words, consistent
with the philosophy of John Dewey, it matters what the student experiences during learning
(Hickman & Alexander, 1998). In OBTE, it is particularly valuable if the learning experience
includes reflection on the dynamic interplay of attributes such as initiative and accountability, the
confidence one derives from the balancing of these attributes, the resulting capacity to navigate
the unknown with what is known, and negotiation between internal and external sources of
power. While this may seem like an existentially significant or cognitively intense undertaking, it
need not be. That it can be more mundane is, of course, critically important in practice. That it
should become somewhat routine or ordinary is the point of OBTE (cf. Riccio et al., 2004;
Appendices D and E).

Figure 5. Development of thinking in action and thinking about action in
outcomes-based training and education [from Darwin, 2008a].

An essential feature of experience in OBTE is that it is inter-temporal (see e.g., Figure 5). There
is the students experience during participation in the designed learning event, and there is the
students experience outside the context of this required participation. The latter typically is
neglected. It can be neglected either unintentionally or even by design based on priorities or
expectations that do not associate intangible attributes with the performance or learning of a task
or subject. One challenge is that instructors typically are not involved with every student during a
scripted learning event, thus it may appear that instructors cannot have an influence on it. Another
challenge is that discipline may preclude overt activity of students when they are not involved in
a scripted learning event (e.g., during waiting time), or the layout of the range or the classroom
may make the informal activity of students difficult to observe. In neither case would it be
accurate to assume that no thinking or communication is occurring. If neglected, the thinking that
occurs during such time periods may not be favorable to learning and retention of the knowledge
or skills addressed in the formal learning event. In any case, there is an opportunity to stimulate
reflection during these frequent periods of time, however brief, to help students make values-
based connections with experience in the learning event, and thus to influence longer-term
outcomes. OBTE aggressively and systematically seeks to exploit such micro-experiences to
build habits of inter-temporal thinking. We believe that micro-experiences are a significant
untapped or misused resource in most learning events (M. Darwin, personal communication, May
5, 2009).

3.2.5 The Emphasis on Instructor-Student Interactions is a Critical Difference

The most concrete findings from our inquiry into OBTE are best practices associated with
interactions between instructors and students (Chapter 2, Appendix A). These practices are not
expressed as immutable scripts but as guidelines for the ongoing pursuit of mastery in the art of
teaching (cf., Augustine, trans. 1953; James, 1890/1907; Keatinge, 1896). The essential aspects of
this art in OBTE are depicted in Figure 6.

Figure 6. Three critical roles of an instructor as a leader, leader developer, and
influencer of human development [From Darwin, 2008a].

Instructor-student interactions seem to be relatively neglected in the science of education,
presumably because it is considered to be an art and something that can only be addressed in the
field, or perhaps because it is deeply and broadly embedded in apparently esoteric research of
scientific psychology. We agree that acumen in working with students is something that can only
be learned in the field but we reject any notion of a strict separation between the art of teaching
and the science of psychology (cf., James, 1890/1907). Part of the intent of OBTE, and of
chapters 3 through 5 in particular, is to reorient Army training and education to theory and data
from relevant subdisciplines in scientific psychology. Our position is that the design,
63 Riccio & Darwin

development, and implementation of instruction can get considerable practical guidance from a
deeper understanding of recent research and historical inquiry in scientific psychology that
addresses the reciprocal influence between individuals and their environment and the impact
these interactions have on the development of motivation, personal identity, and values.

3.2.6 The Emphasis on Learning to Learn is a Critical Difference

The emphasis of OBTE on instructor-student interactions begs questions about how to enable this
through staff and faculty development (see Epilogue). We dont believe that this implies a
necessity for additional instructor education in established theories of education with which
OBTE seems to have most in common, such as experiential education (e.g., Hickman &
Alexander, 1998; Kolb, 1984), constructivist education (e.g., Hmelo-Silver, 2004; Tobias &
Duffy, 2009), or adult education (e.g., Jarvis, 2004; Knowles, Holton, Swanson, 2005). We
believe it is important to return to the foundations for educational theory in psychological science
both historically and in terms of significant trends in psychological science over the last thirty
years (see Chapters 4 and 5). Partly for this reason, and to avoid premature conflation of OBTE
with extant theories of education, the progenitors of OBTE have chosen to orient their principles
and practices with respect to a simple framework for education that has endured for millennia
(Figure 7). Interestingly, this framework suggests an approach to Army training and education
that is reminiscent of a liberal arts education rather than a pre-professional education.

Figure 7. Conceptualization of outcomes-based training and education in terms of
the Trivium and the spiral development of individual competence based on
continuing experiences [from Darwin, 2008a].

It could be argued that one of the most important objectives of a liberal arts education is to help
students acquire learning skills inside the classroom that will help them learn outside the
classroom. OBTE assumes that, in the Army, such skills center around learning collaboratively
with others (cf., Lave & Wenger, 1991; Wenger, 1998). Collaborative experience and inter-
subjectivity thus is important in OBTE, not just in transactions between instructor and student but
also in interactions between peers. Peer-to-peer dialog is valuable for several reasons. It provides
students with opportunities to learn with, through, and from another person who is not necessarily
in a position of authority. Thus the student can learn in a formal learning event how to learn
outside the context of formal learning events, that is, in situations in which an authority figure is
not available to tell individuals what to do. It also provides students with the experience of
transitioning between momentary context-specific roles of leader and subordinate. This skill is
invaluable in task organization of groups in ambiguous or changing conditions.

Learning to learn is directly relevant to notions of adaptability (see Chapter 4). There seems to be
different meanings of adaptability in the institutional and operational Army. In an attempt to
bridge this gap, we offered a more axiomatic formulation of the problem of adaptability in a
workshop held by the University of Southern California and Institute for Creative Technologies,
in April 2009.

1) The vast majority of learning in the Army occurs on the job or in ad hoc instructional events.
a) Instructional events designed and delivered by leaders at the company level and below.
b) Instructional objectives driven by exigencies of the day and influenced by priorities
implicit in intent of higher commanders.
c) The implication is that most learning happens outside the context of curricula and courses
that result from the Army's formal instruction system design (ISD) process.
d) Need: What can be done in the institutional Army to develop curricula and courses that
improve the quality of instruction and learning outside this formal context?

2) The amount of operationally relevant learning occurring in formal curricula and courses in
the Army has been increasing.
a) This is partly because of greater and more prevalent experience of instructors in the full
spectrum of operations (i.e., offense, defense, stability, support)
b) This is partly because of the intent of leadership in Army schools to utilize experience of
instructors and close the gap with the operational Army.
c) It is accepted that lessons learned from experience in Full Spectrum Operations can be
valuable in the classroom.
d) Need: How can we determine what kind of experience is useful in the classroom and how
it should influence curriculum development and instructional activities?

3) The most fundamental unit of analysis in the Army is the group not the individual.
a) Adaptability and the associated agility of groups in the Army is exemplified and fostered
by task direction and organization (i.e., ad hoc task-specific division of labor that takes
into account what is known about group members).
b) Adaptability and the associated agility of groups in the Army is exemplified and fostered
by ad hoc prioritization and tradeoffs (e.g., typically there is more to be done than can be
done given time and resource limitations) that take into account understanding of one's
responsibility to others in the group and one's dependence on them.
c) Need: What are the implications for curriculum development (e.g., nature and extent of
instructional guidance) if we assume that Soldiers must become better at utilizing and
eliciting guidance outside the classroom to achieve collective agility rather that assuming
that they need to become less dependent on guidance in moving from the classroom to
the job?

In OBTE, micro-experiences typically are inter-subjective as well as inter-temporal. The
experiences create a habitual orientation to shared values and long-term outcomes. This has
implications about the consideration of individual differences in personality or learning style. In
particular, it suggests that there is some value to acquainting students with the effects of
individual differences in collaborative work (see e.g., Allik & McCrae, 2004; de Raad &
65 Riccio & Darwin

Perugini, 2002; Heine & Noranzayan, 2006; Hofstede, Pedersen, & Hofstede 2002; Matsumoto
&Yoo, 2006) insofar as it could help them with task organization and the collaborative agility it
enables. Interestingly, the assumptions about adaptability in OBTE also suggest that too much
accommodation for individual differences during a learning event may be counterproductive. The
actors and trans-extremis conditions in Full Spectrum Operations are not likely to accommodate
individual differences in ways that instructors can, in principle. Thus, careful thought about
downstream consequences should be given to the treatment of individual differences in Army
training and education.

3.2.7 The Emphasis on Collaborative Design and Development is a Critical Difference

In Army training and education, historically, there has been a rather strict division of labor
between analysis, design, development, implementation, and evaluation (ADDIE). OBTE argues
for less separation and more collaboration across these curricular activities and responsibilities
(see e.g., Chapters 14 and 15; Cornell-dEchert, 2009a, b). In particular, we believe that instructor
cadre should be included in local decisions involving prioritization and tradeoffs about the extent
to which various skills and knowledge can be addressed in a program of instruction as well as
when and how they should be addressed. This helps establish a command climate in which
mutual trust is both highly valued and salient (cf., Bandura, 1995; Edmonson, 2003; Vaughan,
1997; Rasmussen, 1997). It helps an instructor take ownership of the instructional environment;
and it helps them take responsibility for instructional effectiveness rather than merely nominal
efficiency. The instructor is able to prioritize on the task set forth by the Army, to the standard set
forth by the Army, but in a manner that permits adaptation of the instructional environment to the
uniqueness of the instructors situation. OBTE maintains that this results in a positive, long-term
outcome for instructor and student.

Figure 8 depicts a process that has emerged in the adoption of OBTE in various programs and at
various sites in the Army (e.g., Appendix C; Perry & McEnery, 2009). Commanders in
instructional programs have established a command climate and communicated an intent that
fosters collaborative discussion and decision-making by instructional cadre and their chain of
command to achieve a balance of near-term learning objectives and the long-term developmental
outcomes emphasized by OBTE. These commanders are influenced heavily and self-consciously
by a belief that there is a gap between the historical priorities of institutional training and
emerging operational needs such as preparation for the ambiguous and changing trans-extremis
conditions of Full Spectrum Operations. With the participation of their instructional cadre, they
establish priorities based on emerging needs of Full Spectrum Operations as defined by the Army,
and make tradeoffs based on limited time and resources while still achieving the intent of higher
commanders.

An important aspect of the decentralized programmatic decision-making occurring in the
adoption of OBTE is the identification of outcomes that are observable during training and that
have a plausible causal relationship with downstream outcomes in the operational environment
(see section 3.2.2). The principles and intangibles of OBTE have been helpful to commanders and
their instructional cadre in their collaborative identification of outcomes. The intangibles guide
thinking about plausible relationships between near-term outcomes and long-term development
outcomes (see section 3.2.3). The principles of OBTE have been helpful in identifying outcomes
that are most important in Full Spectrum Operations and in identification of instructional
practices that help develop attributes in Soldiers on which such outcomes depend. Verification
and validation is given special attention in such programs. Commanders and their instructional
cadre are diligent in observing and reflecting on the developing instructional practices and
outcomes in their programs (see sections 3.2.4-3.2.6). In this context, there is considerable
interest in formative measures such as those discussed in this chapter (see also Chapter 2), and
there is increasing interest in measures that more directly relate to outcomes of interest in Full
Spectrum Operations (see Epilogue).

Figure 8. Collaborative design and development for OBTE in a local
command. Solid lines represent direct programmatic influences. Dotted lines
connote informal or indirect programmatic influences. Heavy solid lines
connote direct psychological causes and effects of instruction on students. For
simplicity of depiction, Command Intent & Collaborative Decision-Making
includes all activities involved in planning, executing, and assessing
instruction. Compare with Figure 1 in the Epilogue.

3.3 Toward a Grounded Theory for OBTE

3.3.1 Need for an Integrated Interdisciplinary Framework

As an approach, OBTE is multifaceted and multidisciplinary because it addresses (a) the various
ways that an organization invests in or otherwise influences the development of individuals and
(b) the long-term return on this investment for the individual and the organization. It recognizes
that instruction requires careful consideration of the student, the instructor, and the nested
structure of the instructional institution and the culture it serves (Bransford, et al., 2000; Dewey,
1915/2008; Hmelo-Silver, 2004; James, 1890/1907; Keatinge, 1896; Lewin & Grabbe, 1945;
Magolda, 1999; Riccio, et al., 2004; Wenger, 1998). Effective instruction should have
components that have an impact on all these elements and levels of an instructional system. For
these reasons, OBTE includes interventions that address individual motivation and well-being,
interpersonal interaction and influence, the ecology and development of perception and action,
and reciprocal means-end relations within adaptable organizations.

We use Bransford et al. (2000) as a framework within which to organize and describe the
multidisciplinary considerations implicit in the set of formative measures for OBTE (Chapter 2;
67 Riccio & Darwin

Appendix A). This framework addresses learner-centered, knowledge-centered, and assessment-
centered needs of a learning environment that are grounded in a deep understanding of
community-centered needs (Figure 9). Bransford et al. (2000) argue that learning systems will be
more effective to the extent that they meet these needs.

Figure 9. Exigencies for design of a learning environment that is effective with respect to
human development and far transfer [after Bransford et al., 2000].

We believe that this framework facilitates understanding of the capability gaps to which OBTE is
a response as well as the way in which it addresses those gaps. It is a broad and integrated view of
educational programs that also has implications for program evaluation (see Chapters 1 and 11).
In particular, we believe Bransfords four essential characteristics of a good learning environment
ground the development of measures in a way that helps achieve an integration of Kirkpatricks
four levels of evaluation (D. Kirkpatrick, 1994), integration envisioned by Kirkpatrick but that
has proved to be elusive in most implementations of his system (J. Kirkpatrick & W. Kirkpatrick,
2009). At the same time, a concrete integrated approach to program evaluation helps those
involved in various phases of the ADDIE process understand specifically what they can do to
address the needs of a learning environment as articulated by Bransford et al.

One of the most important results of the methodological crystallization in our inquiry about
OBTE was an appreciation of the actionable implications of principles and practices of OBTE
(Darwin, 2008a, b; Cornell-dEchert, 2009a, b) for a learning environment consistent with the
framework of Bransford et al. The following sections address these opportunities in the learning
environment in OBTE.

3.3.2 Formative Measures of Instructor Behavior as Evolving Best Practices of OBTE

The product of our COMPASS effort with instructional designers and cadre of OBTE yielded a
set of 65 operationally relevant and observable measures (see Chapter 2; Appendix A). These
measures define OBTE at a level sufficient for rigorous verification of instructional practices of
OBTE and, thus, they are useful for both program evaluation and formative feedback to
instructors. It is important to note that the set of 65 measures are intended as a menu from which
one can chose a manageably small subset for a particular learning event. The measures codify the
best practices of OBTE that, in essence, are perspectives on how to translate theory into practice.

The measures are not specific to one application, such as marksmanship training, and they can be
employed to assess instructor behavior in virtually any training domain for quality assurance or
improvement. In this respect, there is a value to measures such as these beyond the particular
relevance they have to OBTE. It is critically important in quality assurance to have some method
of verifying what instructors actually are doing. Absent that, it would be difficult if not
impossible to justify conclusions about what to sustain or what to improve based merely on
measures of learning or student performance. Validation is useless without verification (CMMI
Product Team, 2009); dependent variables are problematic without independent variables. A
fundamental premise of OBTE is that instructor-student interactions are the most important factor
affecting the learning and development of Soldiers. An enhanced capability for verification of this
aspect of instruction is the most important contribution of the formative measures developed in
our research on OBTE.

The essential characteristics of a good learning environment for OBTE were used to identify
categories for the measures of instructor behavior that facilitate selection and use of the measures
(see Appendix A). This was accomplished by ranking the principles in terms of their relevance to
each of the 65 measures. The measures were then grouped into categories of five to nine items
based on similarity (cf., Tversky, 1977) with respect to the four characteristics of a good learning
environment. Ten high-level categories emerged including three for planning, six for execution,
and one for collaborative reflection. Collaborative reflection currently has three small
subcategories that we expect to expand as experience is gained with OBTE in a wider range of
applications.

3.4 Emerging Best Practices in OBTE for a Community-Centered Environment

To be community-centered, Bransford, et al. (2000) argue that learning systems must encourage
exploration in an environment that is relevant to and reflects the community in which learning
will be applied. Consistent with this, OBTE strives to emphasize the relationship between what is
learned early in training and education and what will be necessary in order to function
appropriately in Full Spectrum Operations. Also consistent with Bransfords framework, OBTE
strives to create an open and supportive learning environment that encourages the manifestation
and development of leadership. This is important for the community that OBTE serves, given that
these are characteristics of a learning environment that reflects Army Values. We believe that the
characteristics of a community-centered environment relate directly to leadership and
enculturation of Soldiers (see Table 2).

3.4.1 Leadership and Enculturation of Soldiers

What can be done to develop curricula and courses that improve the quality of life-long learning
and development such as in an operational context where significant and critical learning occurs?
One answer to this question follows from the realization that a collective, not an individual, is the
most fundamental unit of analysis in the Army. Task organization and division of labor in the
Army both requires and provides opportunities for momentary leadership that is not necessarily
formal as designated by rank, position or specialty. Leadership and guidance that individuals
provide to each other is not limited to formal learning events. The goal should not necessarily be
to wean Soldier-students from guidance so that they are competent on their own when they leave
formal instruction. It should be to make them more competent in eliciting, utilizing, and
69 Riccio & Darwin

providing guidance within a task-organized unit. Learning events can be designed to develop this
competency in individuals and thus to prepare Soldiers for success in Full Spectrum Operations.

Table 2: Exigencies of the learning environment in OBTE
Leadership and enculturation of Soldiers
Inculcation of Warrior Ethos and Army Values
Instructor is a leader, facilitator, advisor, mentor, and role model
Instructor gives purpose & vision, allows passion & risk, fosters growth
What is taught and how it is taught reflects necessity of Full Spectrum Operations
Community-centered learning environment

Integrated understanding of basic Soldier skills in Full Spectrum Operations
Condition Soldiers to overcome the psychological and physiological effects of combat
Condition Soldiers always to exercise deliberate thought under stress
Demonstrate the linking of tasks in a military situation
Understand relationships between what is taught and why, when, and how it is taught
Knowledge-centered learning environment

Collaborative reflection and problem solving
Training to grow problem solving
Teach Soldiers to learn for themselves within an established framework of knowledge
Teach through contextual understanding of the task such as its mission application
Draw out of the Soldier a critique of performance during the process
Assessment-centered learning environment

Soldier motivation and development of intangibles
Training to develop intangibles such as confidence, initiative, and accountability.
Assist the Soldier to understand the situation and desired result
Assist the Soldier in identifying obstacles to the desired result
Allow the Soldier to work towards a solution within defined principles
Learner-centered learning environment

Given that there is an abundance of Soldiers with recent and relevant experience in Full Spectrum
Operations, how can we determine what kind of experience is useful in the classroom? OBTE
assumes that the most valuable experience is in the development of individuals. The outcomes
emphasized by OBTE are long-term developmental trajectories and milestones (B. Glaser, 2002;
Rick & Loewenstein, 2008; Peterson & Seligman, 2004; Deci & Ryan, 1985, 2008) more
commonly associated with leader development (TRADOC, 2009). In this sense, OBTE
essentially is a leadership-based approach to training, education, and self-development. OBTE is
based on the recognition that Soldiers, who are both confident and competent, are developed
when leaders allow subordinates reasonable autonomy to exercise individual and small-unit
initiative. This requires a climate of accountability that is fostered by mutual trust among
superiors and subordinates (Riccio et al., 2004). It also requires that leaders at every level
recognize their influence as role models (Bandura, 1977, 1997; Lave & Wenger, 1991) by
thinking and acting flexibly based on constant awareness and adjustment to deviations from
optimal or expected conditions in any situation.

An instructional strategy associated with this principle is to create opportunities for Soldiers to
assume responsibility for their actions, and hence, to develop a sense of accountability (Darwin,
2008a, b; Cornell-dEchert, 2009a, b). The priority is to encourage mutual awareness among
peers. Awareness of the consequences of ones behavior with respect to ones peers also leads to
a heightened awareness of the behavior of those peers (Bandura, 1977; Lewin, 1951). This can
lead to ad hoc peer coaching that can be a valuable instructional multiplier. For example, consider
what happens when a Soldier is taught the importance of pointing his weapon in a safe direction
at all times. As Soldiers move around during training, they become aware of where their weapon
is pointing, but also where other Soldiers weapons are pointing, so that they do not intentionally
walk into a potential line of fire, and call out other Soldiers for a lack of such awareness. Contrast
this with Soldiers who are told to always point their weapon up and down range when on the
firing line. They follow those instructions to the letter, and ignore the purpose of pointing the
weapon up and down range, for safety, and may perseverate on this rule even in situations when it
results in their weapon pointing at someone who may be standing on an elevated terrain. This
example illustrates that reciprocal influence and accountability need not be a complex
undertaking. The foundations for collective agility and success can be developed early in a
Soldiers training and education.

3.4.2 Robust and Adaptable Plan

There are eight measures in this category (Appendix A). A representative measure is Do the
instructors adapt outcome expectations to account for Soldiers who learn slower/faster than
anticipated? This measure includes the following anchors for novice, intermediate, and expert
behavior respectively: (a) cancels training or maintains focus on specific outcomes for original
task; (b) chooses different outcomes but does not achieve effective training; or (c) chooses
outcomes to achieve training effectiveness, leverages condition as another training opportunity.

Outcomes and training events should focus on the mission relevance of knowledge, skills, and
attitudes rather than strict adherence to prescribed (e.g., expository) details of a certain task. In
OBTE, instructors plan to emphasize effectiveness over efficiency; that is, they focus on progress
with respect to development of the Soldier as a student and development of the student as a
Soldier, not merely task completion or throughput. In OBTE, an instructor is prepared with a
frameworka mindset, intent, and nested objectivesthat helps prioritize an inexhaustible array
of prior considerations and opportunities that may arise during a learning event (Cornell-dEchert,
2009b; Haskins, 2009a, b; see Appendix C).

To ensure that training is effective, instructors should plan to frequently assess how the training is
progressing. During this assessment, instructors can determine if they are achieving their intent
for the day and making changes as necessary. The selection of a manageable number of formative
measures for a particular learning event can facilitate assessments that can be conducted during or
after a learning event (see Appendix A). They provide a theme for a learning event that is robust
to uncontrolled variation in conditions. Planning flexibility is important in dealing with variables
that may be out of the instructors control. A mature instructor should have a contingency plan or
expectation to facilitate in-stride adjustments, for example, to inclement weather or limited
resources. In every situation, the instructor should anticipate leveraging unforeseen conditions as
a training opportunity rather than a constraint. For example, a rainy day can be leveraged to build
Soldier confidence in the capabilities of their weapon to operate in inclement weather.

3.4.3 Instructors as Role Models

There are six measures in this category (Appendix A). A representative measure is Do the
instructors demonstrate openness in changing their training progression? This measure includes
the following anchors for novice, intermediate, and expert behavior respectively: (a) training will
not deviate from a set schedule; (b) instructors determine when it is time to move on to a new
task; or (c) Soldiers have input into the progression of training.
71 Riccio & Darwin


Instructors can monitor whether students are getting the training they need and decide when
students should advance to the next event or revisit a learning objective within the parameters of
established time resources. Instructors, as mentors, exhibit behavior associated with the same
attributes that they are trying to grow in their students throughout training (e.g., confidence,
initiative, accountability). In OBTE, this means that instructors demonstrate that they are open to
deviations from the plan and that they are open to insights and guidance from others such as their
peers in the cadre. A central premise of OBTE is that all instructors are influential as role models,
all the time, whether or not they intend to be role models and irrespective of the extent to which
they are prepared to act as role models (Bandura, 1977, 1997; Lave & Wenger, 1991). Instructors
may have admired status, or otherwise may have high credibility, whether or not they deserve it.
The vastly important implication of this assumption is that instructors may actually do more harm
than good if they are not educated about the potential sources and manifestations of such
influence. The measures for this category are a simple mechanism to help instructors achieve
awareness of their influence and a framework for instructor education.

3.4.4 Collaborative Identification of Outcomes and Measures

There currently are no measures in this category. The need for this category is emphasized by the
importance of collaborative discussion among instructional cadre and their chain of command to
achieve a balance of near-term learning objectives and the long-term developmental outcomes
emphasized by OBTE. Such collaborative decision-making is verifiable and best practices can be
identified (Appendix C; Haskins, 2009a, b; Perry & McEnery, 2009). Development of formative
measures for this activity of an instructional organization is one way to focus attention on a
change initiative and to track its progress within the organization (cf., Chapter 14). Measures of
this kind have not yet been developed for OBTE because addressing organizational change and
associated frictions was not part of the original intent of the investigation. Nevertheless,
organizational change has emerged as a critical element of OBTE.

Interestingly, the process used to develop measures for OBTE (e.g., Chapters 2, 7, and 12) is a
model for how outcomes and measures can be developed collaboratively by an instructional cadre
and their chain of command in a way that is scientifically rigorous and traceable (see section
3.2.7). In principle, one also can observe and measure implementation of this model as a critical
element of organizational behavior and come to a better understanding of the factors affecting its
implementation (cf., Bandura, 1995; Edmonson, 2003; Vaughan, 1997; Rasmussen, 1997).
Formative measures should facilitate post hoc internal or external review of the collaborative
conduct and performance of instructional cadre and their chain of command in identifying: (a)
the psychological and physiological effects and the physical requirements of combat; (b)
outcomes associated with adaptability amid the trans-extremis conditions of Full Spectrum
operations; (c) outcomes associated with inculcation of Warrior Ethos and Army Values; (d)
observable instructor behavior associated with being a leader, facilitator, advisor, mentor, and
role model; and (e) instructional strategies that introduce and control passion and risk.

3.5 Emerging Best Practices in OBTE for a Knowledge-Centered Environment

Knowledge-centered environments foster learning in ways that lead to an integrated
understanding of a discipline and to subsequent transfer (Bransford et al., 2000). There is an
emphasis on a network of connections within learning events, between learning events, and with
both prior and subsequent learning. A student becomes oriented within a curriculum-level
landscape instead of being lost in a disconnected complex of knowledge and skills acquired over
an extended period of time. We believe that the characteristics of a knowledge-centered
environment relate most directly to integrated understanding of basic Soldier skills in Full
Spectrum Operations (see Table 2).

3.5.1 Integrated Understanding of Basic Soldier Skills in Full Spectrum Operations

OBTE emphasizes that instruction is most effective when it reflects and responds to the most
urgent requirements of Full Spectrum Operations. These requirements are considered in terms of
basic Soldier skills such as move, shoot, and communicate but at a layer or two deeper than what
those words commonly connote (Flanagan, 2005). These operational requirements can be
addressed at a level that relates to individual adaptability and collective agility and that addresses
the why, what, when, where, and how of actions that can have irrevocable consequences in
operational settings and missions (Darwin, 2008a, b). Thus, they can be addressed at a level that
benefits all Soldiers and their units.

Many of these requirements relate to stress, psychological and physiological effects of stress,
performance given these effects, and strategies that are resistant to stress (Lerner & Keltner,
2001; Lerner & Tiedens, 2006; Lukey & Tepe, 2008). Experience with these relationships and
effects across disparate learning events helps a Soldier overcome stress by avoiding distraction,
maintaining an outward orientation, and always exercising deliberate thought (Darwin, 2008a,b;
cf., Kolditz, 2007; Lukey & Tepe, 2008). It helps students make sense of the relationships
between what is taught and why, when, and how it is taught. It provides them with a path toward
ever deepening understanding of their capabilities in context (Hmelo-Silver, 2004; Magolda,
1999; Riccio et al., 2004).

An instructional strategy within the framework of OBTE is for instructors to communicate with
Soldiers and guide them to discover the purpose and intent of the training. Purpose generally
illuminates a broader context within which tasks are interrelated and the role of an individual is
related to the mission of the unit (J. Gibson, 1977; Lewin, 1951). A focus on purpose reveals the
relevance of the instructional task to an operational mission. More deeply, it reveals the aspects of
the surroundings about which a Soldier should be aware in a collective task, whether in training
or in an operational mission. This lays the foundation for education of the attention, habitual
orientation to the surroundings in any and all situations, and an integrated understanding of a
situation (Endsley, 1995; E. Gibson, 1988; J. Gibson, 1977).

3.5.2 Task Relevance of Planned Instructional Events

There are five measures in this category (Appendix A). A representative measure is Do the
instructors build time for this discussion [about tactical relevance of the task] into the schedule?
This measure includes the following anchors for novice, intermediate, and expert behavior
respectively: (a) does not build in time for discussion into the schedule; (b) builds in time for
short discussions after the day is complete; or (c) builds in time for short discussions between
tasks.

It cannot be assumed that the students will understand why a training exercise is relevant to their
mission. To ensure a knowledge-centered environment, it is important to design exercises that aid
the student in making these connections. For example, understanding the design of a learning
event and the reasons behind it will contextualize students learning, and presumably lead to
better skill retention and transfer to the operational environment. A learning environment that is
both community-centered and knowledge-centered should reflect and develop values of the Army
culture (see Chapters 1-5 and the Epilogue; Riccio et al., 2004). The training environment does
not have to look like the operational environment to develop values, basic Soldier skills, and
73 Riccio & Darwin

resilience to stress. Operational fidelity of a training event can be considered, instead, in the
context of the conditions that foster development of intangible attributes that are critically
important in an operational context or mission.

In OBTE, instructors focus on setting up conditions that foster fidelity of interpersonal
interactions and task linkage that, in turn, provide students with an appreciation of the context in
which knowledge, skills, and attitudes make a difference (cf., Neisser, 1976; Shaw & Bransford,
1977; Winograd, Fivush, & Hirst, 1999). In this approach, variation in conditions is an enabler for
the development of competencies that can transfer to ambiguous or changing environments. The
most robust competencies are those defined in terms of the relations or interactions between the
individual and the environment (Bandura, 1977; E. Gibson, 1988; J. Gibson, 1977; Lave &
Wenger, 1991; Lewin, 1951).

3.5.3 Reveal Operational Relevance of Training

There are seven measures in this category (Appendix A). A representative measure is Do the
instructors group tasks into collective behavior? This measure includes the following anchors for
novice, intermediate, and expert behavior respectively: (a) does not group tasks into collective
behavior, results in incorrect performance of linked tasks; (b) does not group tasks into collective
behavior (no observable negative consequences); or (c) groups task in a way that simulates the
combat application and reinforces correct performance of linked tasks.

OBTE seeks to lay the foundation for training, education, and self-development (Darwin, 2008a,
b). Soldiers, for example, should come to understand the reasons behind a training event,
especially if its operational relevance is not readily apparent. The purpose is not for the instructor
to seek the approval or buy-in of students. Instead, it is to help students understand that the
operational relevance of learning events is not necessarily manifested in tangible ways or through
superficial similarities to an operational scenario. In addition to the interpersonal interactions and
task linkages mentioned above in the context of planning for operationally relevant instruction,
students should gain experience in collaborative problem solving and the collective agility it
enables. They should gain experience dealing with the unknown and the learning in real time that
it necessitates. They should be given a rich variety of experiences that gives them practice in
making connections among disparate events and in extrapolating to new situations (Tobias &
Duffy, 2009; Van Merrienboer & Kirschner, 2007). Finally, they should understand that, in these
ways, training is like the operational environment.

3.5.4 Incorporate Stress into Instructional Events

There are six measures in this category (Appendix A). A representative measure is Do the
instructors encourage deliberate thought in stressful situations? This measure includes the
following anchors for novice, intermediate, and expert behavior respectively: (a) rushes or
encourages Soldier to complete the task without deliberate thought, focuses on speed; (b)
encourages Soldier to think but fails to establish effective training conditions to reinforce
thinking; or (c) focuses Soldier on thinking through situation regardless of the stress level,
focuses on accountability.

As indicated above, stress is not inconsistent with a positive learning environment if the source
and manifestations of stress are representative of demanding operational environments such as
combat (see, e.g., Lukey & Tepe, 2008). Complexity introduced through scenarios is one way to
create stress that is relevant and that potentially allows Soldiers to develop strategies that reduce
stress or mitigate the effects of stress. Training scenarios need not look exactly like operational
scenarios to be useful in this way. In general, scenarios can reveal complexity in linkage among
tasks, for example, and the related implications for interdependence of individuals in a task-
organized unit. Systematic manipulation of such complexity has instructional utility because it is
relevant to the understanding and capabilities for action that Soldiers acquire during training.
Experience and development of self-efficacy in such situations helps Soldiers build habits of
outward orientation amid a growing repertoire of personal capabilities. OBTE assumes that an
understanding of what one can do, ones self-efficacy, in various situations can help one avoid
distractions from the physiological or psychological effects of stress (cf., Kolditz, 2007; Lerner &
Keltner, 2001; Lerner & Tiedens, 2006; Lukey & Tepe, 2008).

3.5.5 Identify General Lessons Learned and Extrapolate to New Situations

There currently are two measures in this evolving category (Appendix A). A representative
measure is Does the instructor ensure Soldiers can articulate how to apply concepts to new
situations? This measure includes the following anchors for novice, intermediate, and expert
behavior respectively: (a) discussion focuses on the specific task only, does not discuss combat or
mission application; (b) facilitates a discussion of the specific tasks and how they apply to
missions; or (c) facilitates a discussion of the intangible attributes underling the task and how it
applies to ambiguous combat/mission situations.

This category of measures, generally utilized after a learning event, addresses the role of inter-
subjectivity and collaboration in learning and development. It is relevant to acquisition of
knowledge and skill but at multiple levels. Thus it is involves refection on more than the
knowledge or skill per se. Some of the measures also should reveal the development of
metacognitive understanding about how the knowledge or skill is utilized in particular situations
and how this is influenced by the situation. An important consideration in further development of
these measures should be to help Soldiers reflect on the role of emotion in thinking and
performance (Lerner & Keltner, 2001; Lerner & Tiedens, 2006; Lukey & Tepe, 2008). The
assumption is that such reflection will develop metacognitive awareness can help Soldiers (a)
exercise deliberate thought under stress; (b) be better prepared for transitions into, out of, and
between stressful situations characteristic of Full Spectrum Operations; and (c) learn lessons from
their experience in particular stressful situations and trans-extremis conditions that can be applied
to different situations.

3.6 Emerging Best Practices in OBTE for an Assessment-Centered Environment

Assessment-centered environments emphasize formative feedback that promotes understanding
(Bransford et al., 2000). Opportunities for feedback should be ubiquitous and should be driven by
noteworthy events in individual or group activities. Feedback should help a student make
connections between current learning events and other parts of the curriculum and the students
life. It should help students appreciate and become more skillful at self-assessment and peer
assessment (Bransford et al., 2000). Consistent with this characteristic of a good learning
environment, OBTE extensively utilizes after-action reviews and ad hoc instructor-student
interactions to draw out of the Soldiers a critique of their own performance throughout the
learning cycle. We believe that the characteristics of an assessment-centered environment relate
most directly to collaborative reflection and problem solving (see Table 2).

75 Riccio & Darwin

3.6.1 Collaborative Reflection and Problem Solving

OBTE seeks to establish a mindset of collaborative reflection and problem solving in which there
is continuous vigilance for noteworthy events and lessons learned worth discussing among
instructors, students, and peers. Development of mastery in assessing oneself and ones unit is a
key outcome for OBTE. It is important to understand that collaborative reflection is a ubiquitous
opportunity that exists in any program of instruction. It can be done on an as-needed basis when
the lessons learned justify the investment of time and resources. It can exploit naturally occurring
events and utilize whatever method is appropriate at the time, such as after action reviews, hot
washes, and informal conversation. This does not mean that constant discussion is a goal of
OBTE or that discussion itself is a reflection that OBTE is occurring. It is as important to decide
not to engage in collaborative reflection if there has not been a noteworthy event or if there is a
more valuable use of time for learning. This mindset, thus, also continually reinforces the
importance of prioritization and tradeoffs required in most military scenarios (Riccio et al., 2004).

If a decision is made to engage in collaborative reflection (e.g., to execute a planned after-action
review), the discussion should be conducted in an open and supportive manner regarding lessons
learned. The purpose is not to summarize merely what happened in the learning event but to
facilitate a discussion that allows students to evaluate their own performance in terms of the
consequences of their actions for the rest of the team and with respect to other linked tasks
(Neisser, 1996; Neisser & Jopling, 1997). Collective self-examination leads to an understanding
in which the whole is greater than the sum of the parts (cf., Barnes, 1959; Cotkin, 2003; Solomon,
2004).

A core strategy of OBTE is to guide Soldiers to the discovery of solutions, rather than merely
telling them what to do (e.g., telling the solution prior to introduction of the problem) (Darwin,
2008a, b; Savory & Duffy, 1995; Vandergriff, 2006, 2007; see also Tobias & Duffy, 2009). This
prepares Soldiers to take initiative in addressing unique problems that may be faced in a mission
environment. OBTE recognizes that training design cannot predict all of the problems that
Soldiers will face. Therefore, rather than try to condition a repertoire of scripted responses, OBTE
attempts to develop skills that can support adaptive behavior even in unforeseen or novel
environments. One of these skill sets is the ability to make decisions about collaborative
reflection and to engage effectively in it. This problem solving approach helps to develop a sense
of initiative that is grounded in awareness of contextual relationships and vigilance about the
attendant consequences of ones own behavior (J. Gibson, 1977; Shaw & Kinsella-Shaw, 2007).
Guidance provided by instructors includes stimulating and facilitating awareness of this dynamic
context rather than simply telling the Soldier about the best course of action given these
considerations.

3.6.2 Communication

There currently are six measures in this evolving category (Appendix A). A representative
measure is Does the instructor encourage inter-trainee communication and discussion? This
measure includes the following anchors for novice, intermediate, and expert behavior
respectively: (a) Soldiers do not have the opportunity to discuss; (b) training design provides
opportunities for unguided inter-trainee communication; or (c) instructor facilitates discussions
and encourages follow-on inter-trainee discussion, training design provides opportunities.

This category includes measures that initially were developed outside of the COMPASS process
using an approach employed in previous research on adaptive communications analysis (e.g.,
Entin, Diedrich, & Rubineau, 2003). The purpose of these measures was to conduct a quantitative
assessment of communication between instructors, students, and peer coaches that is emphasized
in OBTE (see Chapter 2). Some of the measures (e.g., guided learning or self-discovery and
constructive or diagnostic communication) also can be useful in formative assessment; thus, we
expect these measures to be a priority in continuing development of the comprehensive set of
measures for OBTE.

3.6.3 Nature and Extent of Guidance

There are seven measures in this category (Appendix A). A representative measure of how an
instructor might gauge the need for guidance to a Soldier is Does the instructor recognize when a
Soldier is too withdrawn/distracted to effectively participate in training? This measure includes
the following anchors for novice, intermediate, and expert behavior respectively: (a) does not
recognize a deeper individual problem when Soldier is struggling with a task; (b) sees that there is
a deeper problem, but doesnt do anything (doesnt realize he can do something); or (c)
recognizes there is a deeper problem, takes appropriate action to help Soldier get back in the
game.

OBTE occurs when instructors make principled decisions about when and how to adapt the
instructional environment to achieve a positive outcome. Such adaptation includes utilization of
whatever instructional methods are appropriate for the problem at hand, whether that is direct
instruction, problem-centered instruction, or some other method (see e.g., Hmelo-Silver et al.,
2007; Tobias & Duffy, 2009; Van Merrienboer & Kirschner, 2007). The goal is to provide
learners with the right amount of guidance at the right time to achieve both the learning objective
for the event and longer-term developmental outcomes such as the growth of confidence,
initiative and accountability.

3.6.4 Establish a Pervasive Mindset of Collaborative Reflection

There currently are two measures in this evolving category (Appendix A). A representative
measure is Does instructor conduct AAR or hot-washes as needed? This measure includes the
following anchors for novice, intermediate, and expert behavior respectively: (a) does not conduct
AAR or hot-wash, or conducts regardless of need; (b) Tells or lectures Soldiers the lessons
learned in a timely manner; or (c) facilitates discussion on lessons learned amongst the Soldiers in
a timely manner.

This category of measures, generally utilized after a learning event, addresses the practices of
inter-subjectivity and collaboration in learning and development. Further development of these
measures will parallel the development of measures outlined in section 3.6.2 that generally are
utilized during a learning event. The difference is that these measures should foster collaborative
metacognitive reflection. Engaging others in reflection, after joint participation in an event, on the
characteristics and effectiveness on communication and collaboration that had occurred during
the event can help metacognitive reflection from becoming an infinite regress into the abstract. It
keeps it inter-subjectively valid and grounded in a shared reality. The prior intent and
interpretations of other participants can be made explicit, thus, the interpersonal causes for
miscommunication and misinterpretation are more readily revealed. It is important to know
whether problems were due to differences in prior experience, prior knowledge, and associated
assumptions or whether they were due to personality characteristics or even momentary
conditions (e.g., stressors). Differentiating these potential causes, and identifying the critical ones,
can improve collaboration and performance of the group but it also potentially can improve an
individuals understanding and awareness of interpersonal factors in different groups.

77 Riccio & Darwin

3.7 Emerging Best Practices in OBTE for a Learner-Centered Environment

To be learner-centered, an educational system must address what learners know, what they do not
know, how they learn, and what they are motivated to learn (Bransford et al., 2000). In OBTE,
the strategy is to provide learners with right amount of guidance at the right time to achieve both
learning and development. As a mentor and facilitator, the instructor assists the Soldier to
understand the situation, the desired result, and the obstacles to the desired result. The instructor
creates an environment that encourages initiative, a broad appreciation of accountability, and the
development of confidence that is grounded in demonstrable competence. We believe that the
characteristics of a learner-centered environment relate most directly to Soldier motivation and
development of intangibles (see Table 2).

3.7.1 Soldier Motivation and Development of Intangibles

OBTE strives to provide learners with guidance that helps them achieve both the learning
objective for the event and longer-term developmental outcomes such as the growth of
confidence, initiative and accountability (Deci & Ryan, 1985; Guay, Ratelle, & Chanal, 2008;
Peterson & Seligman, 2004). The intent is to leverage internal motivation and to internalize
sources of motivation by balancing initiative with a sense of accountability based on discipline
and awareness of ones relationship to others, and by building confidence based on enhanced self-
efficacy and commitment to the pursuit of mastery (Bandura, 1997; Deci & Ryan, 2008; Ryan &
Deci, 2008).

OBTE occurs when instructors make principled decisions about when and how to adapt the
instructional environment to achieve a positive outcome for particular students in particular
situations, that is, when they exemplify leadership. Such adaptation includes utilization of
whatever instructional methods are appropriate for the problem at hand (Tobias & Duffy, 2009).
The ways in which OBTE can benefit from methods motivated by direct instruction (e.g.,
Kirschner, Sweller, & Clark, 2006), constructivist instruction (e.g., Hmelo-Silver, Duncan, &
Chinn, 2007), and experiential instruction (e.g., Hickman & Alexander, 1998) remain to be seen
but the exigencies of OBTE can inform decisions about the nature and extent of guidance
provided to students or instructors in any of these methods. For example, even in situations in
which students do a "worked example" or solve a well-defined problem given by the instructor, it
is beneficial for the student to understand that the assumptions of the given problem are valid and
why the assumptions are necessary, even if they are not ready to be thrown into a situation in
which assumptions are violated. This allows them to become ready, to become prepared to deal
with such a situation. The foundation for collective agility is thus reinforced in all training and
educational situations, even ones that ostensibly do not require individual adaptability.

A key strategy for the continuing development of intangibles is an emphasis on mastery (Darwin,
2008a, b). From OBTE perspective, pursuit of mastery is essential for confidence that is grounded
in the development of competence. In addition, mastery enables Soldiers to see beyond the
instructional situation and beyond the associated standards as end states. Competence as a pursuit
should motivate an interest in further learning and in obtaining enhanced depth or breadth of
competence. An instructional consideration in this strategy is setting up conditions in which
Soldiers are allowed to experience improvement amid awareness that they can do better (Cornell-
dEchert, 2009a, b; Bandura, 1997; Cole, John-Steiner, Scribner, & Souberman, 1978). For each
of these strategies, the cornerstone of OBTE is for the instructor to set the conditions that create
opportunities for learning (e.g., Dewey, 1915/2008; Bransford et al., 2000; Tobias & Duffy,
2009).

3.7.2 Plan for Development of the Individual

There are nine measures in this category (Appendix A). A representative measure is Is the
training designed to develop intangible attributes (Judgment, Adaptability, Accountability,
Problem Solving, Confidence, Initiative, Awareness, Thinking Skills)? This measure includes
the following anchors for novice, intermediate, and expert behavior respectively: (a) focus of
training is solely to accomplish the task; (b) instructor discusses intangibles directly related to
task being trained, does not focus plan on development; or (c) instructor discusses intangibles
directly related to the task being trained, plan focuses on development.

Instructors should prepare training that develops Soldier understanding of a concept, and supports
adaptive thinking and problem solving that is robust to changes in conditions. Instructors should
not be the proximate source of stress through techniques such as intimidation and belittlement
that have a counterproductive effect on Soldier confidence, initiative, and accountability. This
does not imply that instructors should coddle Soldiers. There are times for a strident authoritarian
style but it should be used judiciously. To utilize stress in training, instructors should prepare
challenges that are more likely to carry over to realities of the operating environment. This frees
the instructor to be a source of guidance and insight about how to manage stress (Darwin, 2008a,
b; Cornell-dEchert, 2009a, b).

Students learn best in a relatively comfortable, low stress environment, but this does not ensure
that they will learn to deal with stress. To address this tradeoff, in OBTE, instructors plan for
systematic increases in stress across learning events (Darwin, 2008a, b; cf., Hmelo-Silver et al.,
2007; Van Merrienboer & Kirschner, 2007). Stress does not necessarily have negative
connotations in this context. One source of increasing stress or challenge is to provide students
with opportunities to take ownership of their own learning. This is a powerful motivator and
source of initiative (Deci & Ryan, 2008; Ryan & Deci, 2008) but it also requires increased
discipline, awareness, and accountability that bounds ones own initiative and ensures that it is
suitably robust to the inevitable mistakes that are critical in learning to learn.

Time should be built into the schedule for students to engage in self-discovery and collaborative
reflection. Though it may be quicker to simply tell a student what the answer is, it is operationally
more relevant if they are provided the tools to arrive at the solution themselves (cf., Hmelo-
Silver, 2004; Tobias & Duffy, 2009). This helps students acquire the skills and internal sources of
motivation that enable self-development in and outside of formal learning environments (Darwin,
2008a, b; dEchert, 2009a, b; Guay, Ratelle, & Chanal, 2008).

3.7.3 Get Students to Take Ownership

There are eight measures in this category (Appendix A). A representative measure is Does the
instructor foster self-development amongst the Soldiers? This measure includes the following
anchors for novice, intermediate, and expert behavior respectively: (a) tells them how they did
without giving Soldiers a chance to reflect; (b) asks the Soldiers how they think they did, tells
them what they did wrong; or (c) asks the Soldiers how they think they did, facilitates self-
discovery of mistakes through targeted questioning.

A balance of power and leadership is critical in OBTE. Achieving this balance is complicated for
many reasons, not the least of which is that students may be unfamiliar with this instructional
style. The hurdle can be overcome more easily if the instructor acts as an advisor and mentor with
a clear interest in the students development. In OBTE, instructors allow passion and measured
risk (Darwin, 2008a, b). They encourage the development of individual initiative that is
79 Riccio & Darwin

constrained by the collective accountability implicit in interdependence and by confidence
grounded in demonstrable competence. Thus, they foster self-determination in Soldiers that is
consistent with military order.

3.7.4 Collaborative Reflection as a Means to Develop Self Efficacy

There currently are five measures in this evolving category (Appendix A). A representative
measure is Does the instructor ask Soldiers to assess their own performance? This measure
includes the following anchors for novice, intermediate, and expert behavior respectively: (a)
does not ask Soldiers to assess their own performance or is overly negative in assessment; (b)
asks Soldiers to assess their own performance but does not recognize accurate self-assessments;
or (c) asks Soldiers to assess their own performance but monitors to ensure they are realistic.

This category of measures, generally utilized after a learning event, addresses the effects of inter-
subjectivity and collaboration in learning and development. The important concept of self-
efficacy, ones perception of ones own capabilities, is most noteworthy when there is an apparent
discrepancy (Bandura, 1977, 1997). Inter-subjective influences can either increase or decrease
discrepancies. Collaborative reflection is useful in this regard to the extent that others are willing
and able to help inform one about ones own capabilities, that is, to foster the veridical perception
that is critical in engagements with the real world. Such guidance from more astute observers,
from less biased observers, or simply from a different perspective, is even more critical amid the
uncertainty of inferences about ones zone of proximal development (cf., Cole et al., 1978). The
acumen or perspective of others can help one identify paths in the pursuit of mastery that may not
be obvious or that may be obscured by habits, assumptions, or unknowns about strategies for
cognition and action. Collaborative reflection in groups fosters intersubjective convergence on
reality (cf., Chapters 4 and 5).

Collaborative reflection and inter-subjectivity also are central in the development of values
implicit in the balance of choice and responsibility (cf., Aristotle, trans. 1925; Augustine, trans.
1991; Hoffman et al., 2009; Maslow, 1964; Neisser & Jopling, 1997; Solomon, 2004). Others can
help reveal constraints of shared values on ones cognition and action, they can help reveal non-
obvious ways in which ones cognition and action is situated in a shared context, and the impact
of the mere existence of others in a shared context on ones cognition and action. It both directs
and educates the attention. In the context of the group, which is the fundamental unit of analysis
in the Army, inter-subjectivity is a source of coupling among individuals (i.e., their respective
cognition and action) that renders the notion of individual adaptability almost irrelevant.
Individual adaptability is of consequence and is admissible only in the context of collective
agility, in a context that bounds unplanned activity and that gives it purpose.

The principles and practices of OBTE, summarized in this chapter, provide basic facts from
which to build an appropriate scientific foundation for OBTE. Such scientific foundations are
explored in the Chapters 4 and 5.

3.8 References

Allik, J., & McCrae, R. R. (2004). Towards a geography of personality traits: Patterns of profiles
across 36 cultures. Journal of Cross-Cultural Psychology, 35, 13-28.
Augustine (1953). The teacher. In: J. Burleigh (Trans. & Ed). Augustine: earlier writings.
Philadelphia, PA: The Westminster Press. (Original work written ca. 389)
Augustine (1991). Confessions. H. Chadwick (Trans). New York: Oxford University Press.
(Original work written ca. 400)
Aristotle (1977). Nicomachean ethics. In: R.M. Hutchins et al. (Eds.). Aristotle II. Chicago, IL:
Encyclopedia Britannica. (Reprinted from The Works of Aristotle, by W.D. Ross Ed. &
Trans., 1925, Oxford, UK: Oxford University.) (Original work written ca. 350 BCE)
Aspinwall, L. & Staudinger, U. (Eds.) (2003). A psychology of human strengths: Fundamental
questions and future directions for a positive psychology. Washington, DC: American
Psychological Association.
Bandura, A. (1977). Social learning theory. New York: General Learning Press.
Bandura, A. (Ed.) (1995). Self efficacy in changing societies. Cambridge, UK: Cambridge
University Press.
Bandura, A (1997). Self-efficacy: The exercise of control. New York, NY: Freeman and
Company.
Barnes, H. (1959). Humanistic existentialism: The literature of possibility. Lincoln, NE:
University of Nebraska.
Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn: Brain, mind,
experience, & school. Washington, DC: National Academy Press.
Bruny, T., Riccio, G., Sidman, J., Darowski, A., & Diedrich, F. (2006). Enhancing warrior ethos
in initial entry training. Proceedings of the 50th Annual Meeting of the Human Factors and
Ergonomics Society, San Francisco, CA.
Burke, W. (1985). Applied differential geometry. Cambridge, UK: Cambridge University.
Camber, C., Loewenstein, G., & Prelec, D. (2005). Neuroeconomics: How neuroscience can
inform economics. Journal of Economic Literature, 43(1), 9-64.
Camic, P.M., Rhodes, J.E., & Yardley (Eds.) (2003). Qualitative research in psychology:
Expanding perspectives in methodology and design. Washington DC: APA.
Carlson, S. (2005). The net generation goes to college. The Chronicle of Higher Education,
Volume 52 (7), A34.
Clausewitz, C. von (1976). On war. In: M. Howard. & P. Paret (Trans. and Eds.). On war.
Princeton: NJ: Princeton University. (Original work written 1832)
CMMI Product Team (2009). CMMI for services, version 1.2. (CMU/SEI-TR-2009-001; ESC-
TR-2009-001). Pittsburgh, PA: Carnegie Mellon University.
Cole, M., John-Steiner, V., Scribner, S., & Souberman, E. (1978). Mind in society: The
development f higher psychological processes (L.S. Vygotsky). Cambridge, MA: Harvard
University Press.
Cornell-dEchert, B. (2009a). An introduction to outcomes-based training and education. Fort
Meade, MD: Asymmetric Warfare Group.
Cornell-dEchert, B. (2009b). Outcomes-based training and education: Implementation guide.
Fort Meade, MD: Asymmetric Warfare Group.
Cotkin, G. (2003). American existentialism. Baltimore, MD: Johns Hopkins University.
Cresswell, J. (1998). Qualitative inquiry and research design: Choosing among five traditions.
Thousand Oaks, CA: Sage.
Darwin, M. (2008a). Asymmetric Warfare Group combat applications training course (CATC)
senior leader discussion (Briefing, January 23, 2008). Fort Meade, MD: Asymmetric Warfare
Group.
Darwin, M. (2008b). Outcomes-based training and education: fostering adaptability in full
spectrum operations (Briefing, December 2008). Fort Meade, MD: Asymmetric Warfare
Group.
Deci, E. & Ryan, R. (1985). Intrinsic motivation and self-determination in human behavior. New
York: Plenum.
Deci. E. & Ryan, R. (2008). Self determination theory: a macrotheory of human motivation,
development, and health. Canadian Psychology, 49, 182-185.
81 Riccio & Darwin

Denzin, N.K, & Lincoln Y.S. (Eds.) (2003). Strategies for qualitative inquiry. Thousand Oaks,
CA: Sage.
De Raad, B., & Perugini, M. (2002). Big five assessment. Gottingen, NL: Hogrefe & Huber.
Dewey, J. (2008). The school and society, the child and the curriculum. Chicago, IL: Centennial
Publications of The University of Chicago Press. (Original work published 1915)
Ellingson, L. (2009). Engaging crystallization in qualitative research: An introduction. Los
Angeles, CA: Sage.
Endsley, M.R. (1995). Toward a theory of situation awareness in dynamic systems. Human
Factors, 37, 32-64.
Entin, E. E., Diedrich, F. J. & Rubineau, B. (2003). Adaptive communication patterns in different
organizational structures. Proceedings of the Human Factors and Ergonomics Society Annual
Meeting, Denver, CO.
Feldbaum, A. (1965). Optimal control systems. New York: Academic Press.
Filatov, N.M. & Unbehauen, H. (2004). Adaptive dual control: Theory and applications. New
York: Springer.
Flanagan, S. (2005). Combat skills. Unpublished manuscript (Nov. 16, 2005).
Freeman, J., Jason, J., Aten, T., Diedrich, F., Cooke, N., Winner, J., Rowe. L., & Riccio, G.
(2008). Shared Interpretation of Commander's Intent (SICI). Final Report to the Army
Research Institute for the Behavior and Social Sciences, contract number W74V8H-06-C-
0004.
Gibson, E. (1988). Exploratory behavior in the development of perceiving, acting, and the
acquiring of knowledge. Annual Review of Psychology, 39, Pages 1-42.
Gibson, E. (1991). An odyssey in learning and perception. Cambridge, MA: MIT Press.
Gibson, J. (1977). The theory of affordances. In R. Shaw & J. Bransford (eds.), Perceiving, acting
and knowing: toward an ecological psychology. Hillsdale, NJ: Erlbaum.
Gibson, J. & Gibson, E. (1955). Perceptual learning: Differentiation or enrichment. Psychological
Review, 62, 32-51.
Glaser, B. (2002). Efficiency versus sustainability in dynamic decision making: Advances in inter-
temporal compromising. New York: Springer.
Glass, L. & Mackey, M. (1988). From clocks to chaos: The rhythms of life. Princeton, NJ:
Princeton University.
Goldberg, S. (1986). Introduction to difference equations: With illustrative examples from
economics, psychology, and sociology.
Guay, F., Ratelle, C. F., & Chanal, J. (2008). Optimal learning in optimal contexts: The role of
self-determination in education. Canadian Psychology, 49, 233240.
Haskins, C. (2009a, March). Development of outcomes based training. Presentation at the US
Army Asymmetric Warfare Group workshop on Outcomes-based Training and Education,
Applied Physics Laboratory, Johns Hopkins University, Laurel MD.
Haskins, C. (2009b). Outcomes-based training and education. White Paper solicited by
TRADOC Headquarters, August, 2009.
Headquarters Department of the Army (2003). Mission command: Command and control of
Army forces. FM 6-0. Washington, DC: Department of the Army.
Headquarters Department of the Army (2005). The Army. FM 1. Washington, DC: Department of
the Army.
Heine, S. J. & Norenzayan, A. (2006). Toward a psychological science for a cultural species.
Perspectives on Psychological Science, 1(3), 251-269.
Heisenberg, W. (1958). Physics and philosophy: The revolution in modern science. New York:
Harper.
Hickman, L.A. & Alexander, T.M. (1998). The essential Dewey, Volume 1: pragmatism,
education, and democracy. Bloomington, IN: University of Indiana Press.
Hmelo-Silver, C. (2004). Problem-based learning: What and how do students learn? Educational
Psychology Review, Vol. 16(3), 235-266.
Hmelo-Silver, C. Duncan, R.G., & Chinn, C. (2007). Scaffolding and Achievement in Problem-
Based and Inquiry Learning: A Response to Kirschner, Sweller, and Clark (2006).
Educational Psychologist, 42(2), 99107.
Hoffman, L., Yang, M., Kaklauskas, F., Chan. (2009). Existential psychology East-West.
University of. Rockies Press. Colorado Spring.
Hofstede, G. J., Pedersen, P., & Hofstede, G. (2002). Exploring culture: exercises, stories, and
synthetic cultures. Boston, MA: Intercultural Press.
Howe, N., Strauss, W., & Matson. R.J. (2000). Millennials rising: the next great generation. New
York, NY: Vintage.
Idel, M., & McGinn, B. (Eds.) (1999). Mystical union in Judaism, Christianity, and Islam: An
ecumenical dialogue. New York: Continuum.
James, W. (1890). Principles of psychology. New York, NY: Henry Holt and Company.
James, W. (1907). Talks to teachers on psychology: and to students on some of life's ideals. New
York, NY: Henry Holt and Company. (Original work published 1899)
James, W. (1982). The varieties of religious experience: A study in human nature. New York:
Penguin. (Original work published 1902)
Jarvis, P. (2004). Adult education and lifelong learning: Theory and practice. London, UK:
Routledge-Falmer.
Keatinge, M. (1896). The great didactic of John Amos Comenius. Whitefish, MT: Kessinger
Publishing.
Keyes, C. & Haidt, J. (2003). Flourishing: Positive psychology and the life well-lived.
Washington, DC: American Psychological Association.
Kierkegaard, Soren (1843) under the pseudonym Silentio, Johannes. Fear and Trembling, edited
by Alastair Hannay. (1985): Penguin Classics
Kirkpatrick, D., (1994). Evaluating training programs. San Francisco, CA: Berrett-Koehler
Publishers, Inc.
Kirkpatrick, J., & Kirkpatrick, W. (2009). The Kirkpatrick four levels: A fresh look after 50 years,
1959 2009. Retrieved May, 2009, from http://kirkpatrickpartners.com.
Kirschner, Sweller, and Clark (2006). Why minimal guidance during instruction does not work:
An analysis of the failure of constructivist, discovery, problem-based, experiential, and
inquiry based teaching. Educational Psychologist, 41, 75-86.
Knowles, M., Holton, E., Swanson, R. (2005). The adult learner. Burlington, MA: Elsevier.
Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development.
New Jersey: Prentice-Hall.
Kolditz, T. (2007). In extremis leadership: Leading as if your life depended on it. New York:
Wiley.
Lanczos, C. (1970). The variational principles of mechanics. New York: Dover.
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation.
Cambridge, UK: Cambridge University Press.
Lerner, J., & Keltner, D., (2001). Fear, anger, and risk. Journal of Personality and Social
Psychology, 81(1), 146-159.
Lerner, J., & Tiedens, L. (2006). Portrait of the angry decision maker: How appraisal tendencies
shape angers influence on cognition. Journal of Behavioral Decision Making (Special Issue
on Emotion and Decision Making), 19, 115-137.
Lvi-Valensi, J. (Ed.) (2006). Camus at combat: Writing 1944-1947. Princeton, NJ: Princeton
University. (Original work written 1944-1947).
Lewin, K. (1951). Field theory in the social sciences. New York, NY: Harper.
Lewin, K & Grabbe, P. (1945). Conduct, knowledge, and acceptance of new values. The Journal
of Social Issues, 1 (3), 53-64.
83 Riccio & Darwin

Loewenstein, G., & Lerner, J.S. (2002). The role of affect in decision making. In: R. Davidson,
K. Scherer, & H. Goldsmith (Eds.), Handbook of affective science (pp. 619642). New York:
Oxford University Press.
Loewenstein, G., Weber, E., Hsee, C., & Welch, N. (2001). Risk as feelings. Psychological
Bulletin, 127 (2), 267286.
Lorentz, H., Einstein, A., Minkowski, H., & Weyl, H. (1952). The principle of relativity: A
collection of papers on the special and general theory of relativity. W. Perrett & G. Jefferey
(Trans.). New York: Dover. (Original work published 1923)
Lukey, B. & Tepe, V. (Eds.) (2008). Biobehavioral resilience to stress. Boca Raton, FL: CRC
Press.
Magolda, M.B.B. (1999). Creating contexts for learning and self-authorship: constructive-
developmental pedagogy. Nashville, TN: Vanderbilt Press.
Maslow, A. (1964). Religions, values, and peak-experiences. New York: Penguin.
Msse, L. C., Moser, R. P., Stokols, D., Taylor, B. K., Marcus, S. E., Morgan, G. D., Hall, K.L.,
Croyle, R.T., Trochim, W. (2008). Measuring Collaboration and Transdisciplinary
Integration in Team Science. American Journal of Preventive Medicine, 35(2, Supplement 1),
S151-S160.
Matsumoto, D., & Yoo, S. H. (2006). Toward a new generation of cross-cultural research.
Perspectives on Psychological Science, 1(3), 234-250.
Mikroyannidis, A. (2007). Toward a social semantic web. Computer, November 2007, 113-115.
Los Alamitos, CA: IEEE Computer Society.
Miles, M.B. & Huberman, A.M. (1994). Qualitative data analysis. Thousand Oaks, CA: Sage.
Neisser, U. (1976). Cognition and reality. San Francisco: W.H. Freeman.
Neisser, U. (Ed.) (1996). The perceived self: ecological and interpersonal sources of self
knowledge. Cambridge, UK: Cambridge University Press.
Neisser, U., & Jopling, D.A. (Eds.) (1997). The conceptual self in context. New York: Cambridge
University Press.
Paavola, S. & Hakkarainen, K. (2005). The knowledge creation metaphor An emergent
epistemological approach to learning. Science and Education, 14(6), 535-557.
Pellegrino, J., Chudowsky, N., & Glaser, R. (Eds.) (2001). Knowing what students know: The
science and design of educational assessment. Washington, DC: National Academy Press.
Perry, R. & McEnery, K. (2009). Army reconnaissance course: Defining the aim point for
reconnaissance leader training. Armor, July-August, 14-20.
Peterson, C., & Seligman, E. P. (2004). Character, strength, and virtues. Oxford, UK: Oxford
University Press.
Plato (1977a). Phaedo. In: R.M. Hutchins et al. (Eds.). Plato. Chicago, IL: Encyclopedia
Britannica. (Reprinted from The Dialogues of Plato, by B. Jowett, Trans., 1892a, Oxford,
UK: Oxford University.) (Original work written ca. 360 BCE).
Rasmussen, J. (1997). Risk management in a dynamic society: a modeling problem. Safety
Science, 27, 183-213.
Riccio, G. (1993a). Information in movement variability about the qualitative dynamics of
posture and orientation. In: K. Newell (Ed.). Variability and Motor Control (pp. 317-357),
Champaign, IL: Human Kinetics.
Riccio, G. (1993b). Multimodal perception and multicriterion control of nested systems: Self
motion in real and virtual environments. (UIUC-BIHPP-93-02). University of Illinois at
Urbana-Champaign: Beckman Institute for Advanced Science & Technology (also in NASA
Technical Paper series 3703).
Riccio, G., & McDonald, P. (1998). Characteristics of EVA mass handling skill. Society of
Automotive Engineers Paper No. 981625 (also NASA Technical Paper 3684).
Riccio, G., & Stoffregen, T. (1991). An ecological theory of motion sickness and postural
instability. Ecological Psychology, 3, 195-240.
Riccio, G., Sullivan, R., Klein, G., Salter, M., & Kinnison, H. (2004). Warrior ethos: Analysis of
the concept and initial development of applications. ARI Research Report 1827. Arlington,
VA: US Army Research Institute for the Behavioral and Social Sciences.
Rick, S. and Loewenstein, G. (2008). Intangibility in intertemporal choice. Philosophical
Transactions of the Royal Society B, 363, 38133824.
Ryan, R. & Deci. E. (2008). From ego depletion to vitality: Theory and findings concerning the
facilitation of energy available to the self. Social and Personality Psychology Compass 2(2),
702717.
Savory, J.R. & Duffy, T.M. (1995). Problem based learning: An instructional model and its
constructivist framework Educational Technology, 35, 31-38.
Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and technology.
In K. Sawyer (Ed.), Cambridge Handbook of the Learning Sciences (pp. 97-118). New York:
Cambridge University Press.
Shaw, R. & J. Bransford, J. (eds.) (1977). Perceiving, acting and knowing: toward an ecological
psychology. Hillsdale, NJ: Erlbaum.
Shaw, R., & Kinsella-Shaw, J. (2007). The survival value of informed awareness. Journal of
Consciousness Studies, 14(1), 137-154.
Sidman, J., Riccio, G., Semmens, R., Geyer, A., Dean, C., & Frederick Diedrich, F. (2009).
Reshaping Army institutional training: Current training. Final Report to the Army Research
Institute for the Behavior and Social Sciences, contract number W74V8H-04-D-0047 DO
0010.
Solomon, R. (Ed.) (2004). Existentialism. Oxford, UK: Oxford University Press.
Spady, W. (1994). Outcome based education: Critical issues and answers. Arlington, VA:
American Association of School Administrators.
Spady, W. (2009). Outcome based education. Florence, KY: Cengage Learning.
Spady, W., & Marshall, K. (1991). Beyond traditional outcome-based education. Educational
Leadership 49 (2):6772.
Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining qualitative and
quantitative approaches. Thousand Oaks, CA: Sage.
Tobias, S. & Duffy, T. (eds.) (2009). Constructivist instruction: Success or failure. New York:
Routledge.
Trochim, W. (2004). Concept mapping. In Matheson, S. (Ed.) Encyclopedia of Evaluation.
Thousand Oaks, CA: Sage.
Trochim. W., Marcus, S.E., Msse, L.C., Moser, R.P., Weld, P. (2008). The evaluation of large
research initiatives: A participatory integrative mixed-methods approach, American Journal
of Evaluation, 29, 1, 8-28.
Tversky, A. (1977). Features of similarity. Psychological Review, 84(4), 327-352.
United States Army Training and Doctrine Command (2009). Army Leader Development Strategy
(Draft, May 21, 2009), Ft. Monroe, VA: Training and Doctrine Command.
Vandergriff, D. (2006). Adaptive leaders course, Part 1: Old dogs teaching new tricks. Army,
November 2006, 59-66.
Vandergriff, D. (2007). From swift to Swiss: Tactical decision games and their place in military
education and performance improvement. Performance Improvement, 45(2), 30-39.
Van Merrienboer, J., & Kirschner, P. (2007). Ten steps to complex learning: A systematic
approach to four-component instructional design. New York: Routledge.
Vaughan, D. (1997). The Challenger Launch Decision: Risky Technology, Culture, and Deviance
at NASA. Chicago, IL: University Of Chicago Press.
Webster, D. S. (2001). In defense of the 'Natural Attitude': charting a return from the madness of
interpretation. Assemblage 6, Available at:
http://www.shef.ac.uk/~assem/issue6/webster_web.html
85 Riccio & Darwin

Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. New York :
Cambridge University.
Winograd, E., Fivush, R., & Hirst, W. (1999). Ecological approaches to cognition: Essays in
honor of Ulric Neisser. Mahwah, NJ: Lawrence Erlbaum.
Wise, G. (1980). American historical explanations: A strategy for grounded inquiry.
Minneapolis, MN: University of Minnesota.
Wolfram, S. (2002). A new kind of science. Champaign, IL: Wolfram Media.

ix Riccio, Diedrich, & Cortes (Eds.)
TABLE OF CONTENTS
.
page

Prologue: A Programmatic View of the Inquiry into Outcomes-Based Training & Education.......1
Historicity of our Research on OBTE..........................................................................................1
The Approach and Lessons Learned from the Research..............................................................3
Documentation of the Research ...................................................................................................4

Section I. Development of Stakeholder Requirements for OBTE..............................................6

Chapter 1. Preparation for Full Spectrum Operations ......................................................................7
1.1 Requirements of Full Spectrum Operations...........................................................................8
1.2 Outcomes-Based Training and Education (OBTE)..............................................................10
1.2.1 Exemplar of OBTE: Combat Applications Training Course........................................11
1.2.2 OBTE as a Multifaceted Instructional System .............................................................12
1.3 An Appraisal of Instruction with Respect to OBTE ............................................................13
1.3.1 A Systems Engineering Framework for Integration and Development of OBTE........13
1.3.2 Preparation for Validation and Verification .................................................................14
1.4 References ............................................................................................................................17

Chapter 2. Formative Measures for Instructors ..............................................................................20
2.1 Development of Formative Measures ..................................................................................20
2.1.1 The COMPASS Methodology......................................................................................20
2.1.2 Development of Measures for OBTE...........................................................................21
2.2 Description of Formative Measures .....................................................................................21
2.2.1 Results of the COMPASS Process................................................................................21
2.2.2 Elaboration on the Description of Measures.................................................................23
2.3 OBTE Performance Measures: Planning for Training.........................................................23
2.3.1 Define Outcomes ..........................................................................................................23
2.3.2 Create a Positive Learning Environment ......................................................................25
2.3.3 Create the Parameters of Learning................................................................................27
2.4 OBTE Performance Indicators: Training Execution............................................................28
2.4.1 Communicate the Parameters of Learning....................................................................28
2.4.2 Training Emphasizes Broad Combat or Mission Success ............................................29
2.4.3 Customize Instruction When Possible Based on Constraints/Conditions ....................31
2.4.4 Facilitates Learning of Concepts ..................................................................................32
2.4.5 Creates a positive learning environment.......................................................................34
2.4.6 Instructors Utilize Measures of Effectiveness & Self-Evaluation................................36
2.4.7 Uses scenarios to facilitate learning..............................................................................38
2.4.8 Instructors exhibit intangible attributes in own actions ................................................40
2.4.9 Hotwashes and Mini-AAR............................................................................................42
2.5 Uses of the Measures ...........................................................................................................43
2.5.1 Formative Measures for Instructors..............................................................................44
2.5.2 Quality Assurance and Instructor Education ................................................................44
2.5.3 Continuous Improvement of Assessments....................................................................45
2.5.4 Program Evaluation and Organizational Change..........................................................46
2.6 References ............................................................................................................................46

Table of Contents x
Chapter 3. Principles and Practices of Outcomes Based Training & Education............................50
3.1 Multifaceted Inquiry.............................................................................................................50
3.1.1 Interaction with Progenitors of OBTE..........................................................................51
3.1.2 AWG Documents on OBTE .........................................................................................52
3.1.3 Collaborative Reflection on Participant Observation in CATC ...................................52
3.1.4 Interaction with Stakeholders .......................................................................................53
3.2 Essential Characteristics of OBTE.......................................................................................53
3.2.1 The Meaning of Developmental is a Critical Difference..............................................53
3.2.2 The Definition of Outcomes is a Critical Difference....................................................56
3.2.3 The Emphasis on Values and Causally Potent Intangibles is a Critical Difference .....58
3.2.4 The Meaning of Experience is a Critical Difference ....................................................61
3.2.5 The Emphasis on Instructor-Student Interactions is a Critical Difference ...................62
3.2.6 The Emphasis on Learning to Learn is a Critical Difference .......................................63
3.2.7 The Emphasis on Collaborative Design and Development is a Critical Difference.....65
3.3 Toward a Grounded Theory for OBTE................................................................................66
3.3.1 Need for an Integrated Interdisciplinary Framework ...................................................66
3.3.2 Formative Measures of Instructor Behavior as Evolving Best Practices of OBTE......67
3.4 Emerging Best Practices in OBTE for a Community-Centered Environment.....................68
3.4.1 Leadership and Enculturation of Soldiers.....................................................................68
3.4.2 Robust and Adaptable Plan...........................................................................................70
3.4.3 Instructors as Role Models ...........................................................................................70
3.4.4 Collaborative Identification of Outcomes and Measures .............................................71
3.5 Emerging Best Practices in OBTE for a Knowledge-Centered Environment .....................71
3.5.1 Integrated Understanding of Basic Soldier Skills in Full Spectrum Operations ..........72
3.5.2 Task Relevance of Planned Instructional Events..........................................................72
3.5.3 Reveal Operational Relevance of Training...................................................................73
3.5.4 Incorporate Stress into Instructional Events .................................................................73
3.5.5 Identify General Lessons Learned and Extrapolate to New Situations ........................74
3.6 Emerging Best Practices in OBTE for an Assessment-Centered Environment ...................74
3.6.1 Collaborative Reflection and Problem Solving ............................................................75
3.6.2 Communication.............................................................................................................75
3.6.3 Nature and Extent of Guidance.....................................................................................76
3.6.4 Establish a Pervasive Mindset of Collaborative Reflection..........................................76
3.7 Emerging Best Practices in OBTE for a Learner-Centered Environment ...........................77
3.7.1 Soldier Motivation and Development of Intangibles....................................................77
3.7.2 Plan for Development of the Individual .......................................................................78
3.7.3 Get Students to Take Ownership ..................................................................................78
3.7.4 Collaborative Reflection as a Means to Develop Self Efficacy....................................79
3.8 References ............................................................................................................................79

Chapter 4. Grounded Theory for Values-Based Training & Education.........................................86
4.1 Exploration of Holistic and Functionalistic Underpinnings for OBTE ...............................86
4.1.1 Fundamental Units of Analysis.....................................................................................87
4.1.2 Nested Time Scales and Adaptability...........................................................................88
4.1.3 Adaptability and Ambiguity .........................................................................................90
4.1.4 Mechanistic Analogies and Predominant Experimental Paradigms .............................92
4.2 Three Pillars for the Scientific Foundation of OBTE ..........................................................93
4.2.1 Ecological Psychology..................................................................................................93
4.2.2 Self-Efficacy Theory.....................................................................................................97
4.2.3 Positive psychology ......................................................................................................98
4.3 A More Integrated Scientific Infrastructure.......................................................................101
xi Riccio, Diedrich, & Cortes (Eds.)
4.3.1 Self Determination Theory .........................................................................................101
4.3.2 Situated Learning Theory ...........................................................................................103
4.3.3 Existential Psychology................................................................................................105
4.4 Building on the Scientific Infrastructure for OBTE...........................................................109
4.4.1 Triadic Frameworks....................................................................................................109
4.4.2 Further Development ..................................................................................................112
4.5 References ..........................................................................................................................112

Chapter 5. Passion and Reason in Values-Based Learning & Development ...............................118
5.1 The Nested Self ..................................................................................................................118
5.1.1 An Alternative to Individual versus Collective ..........................................................118
5.1.2 Cognition and Reality .................................................................................................119
5.2 Conscious Experience and the Dynamics of Thinking ......................................................122
5.3 Emotion, Information, and Engagement ............................................................................125
5.3.1 Ecological Perspective on Emotion ............................................................................125
5.3.2 Emotion as Engagement .............................................................................................126
5.3.3 Implications for Training and Education....................................................................129
5.4 Emotion, Decision-Making, and Inter-Temporal Choice...................................................129
5.4.1 Toward a More Integrated Theory..............................................................................129
5.4.2 Emotion and Decision-Making...................................................................................130
5.4.3 Emotion and Nested Time Scales ...............................................................................131
5.4.4 Neuroeconomics and Inter-Temporal Reasoning .......................................................132
5.5.5 Inter-Temporal Reasoning and Adaptive Dynamical Systems...................................133
5.5 Beyond Science..................................................................................................................134
5.5.1 Existentialism..............................................................................................................134
5.5.2 The Soldier-Scholar as an Emergent Property of a Collective Pursuit.......................135
5.6 References ..........................................................................................................................137

Section II. Verification and Validation of OBTE as a Service System..................................142

Chapter 6. Initial Impressions of Participation in CATC.............................................................143
6.1 Methods..............................................................................................................................143
6.1.1 Participants..................................................................................................................143
6.1.2 Procedure ....................................................................................................................143
6.1.3 Analyses......................................................................................................................144
6.2 Results ................................................................................................................................144
6.3 Implications for Service System Development: Peer Review ...........................................146
6.4 References ..........................................................................................................................147

Chapter 7. Local Development of Measures of Effectiveness .....................................................149
7.1 What do Instructors Believe Soldiers Should Learn in Initial Entry Training?.................149
7.2 Measure Development Process ..........................................................................................150
7.3 What do OBTE-Trained DS Believe is Important to Assess in BRM/ARM? ...................151
7.4 Implications........................................................................................................................156
7.5 Conclusions ........................................................................................................................158
7.6 References ..........................................................................................................................159

Table of Contents xii
Chapter 8. Observations of Behavior and Communication in Rifle Marksmanship Training .....160
8.1 Methods..............................................................................................................................160
8.1.1 Participants..................................................................................................................160
8.1.2 Procedure ....................................................................................................................160
8.1.3 Analyses......................................................................................................................161
8.2 Results ................................................................................................................................163
8.2.1 Behavior of DS ...........................................................................................................163
8.2.2 Behavior and Performance of Privates .......................................................................165
8.2.3 Patterns of Communication ........................................................................................168
8.2.4 Potential Influence of Instructor Behavior on Performance of Privates.....................170
8.3 Implications for Service System Development..................................................................171
8.3.1 Verification of OBTE .................................................................................................171
8.3.2 Validation of OBTE....................................................................................................172
8.4 References ..........................................................................................................................173

Chapter 9. Impact on Rifle Marksmanship Training....................................................................174
9.1 Behavioral Data Collection During Basic Rifle Marksmanship........................................174
9.1.1 Method........................................................................................................................174
9.1.2 Assessment..................................................................................................................175
9.1.3 Results An Overview...............................................................................................177
9.1.4 Evidence for Influence of OBTE................................................................................178
9.1.5 Behavior of Drill Sergeants after Exposure to OBTE ................................................180
9.1.6 Behavior of Privates....................................................................................................182
9.1.7 Patterns of Communication ........................................................................................186
9.1.8 Summary.....................................................................................................................186
9.2 Attitudes Toward an OBTE in Basic Training...................................................................187
9.2.1 Method........................................................................................................................187
9.2.2 Results.........................................................................................................................187
9.4 References ..........................................................................................................................191

Chapter 10. Influence of CATC in an Operational Setting ..........................................................192
10.1 Methods............................................................................................................................192
10.1.1 Participants................................................................................................................192
10.1.2 Procedure ..................................................................................................................192
10.1.3 Analyses....................................................................................................................193
10.2 Results ..............................................................................................................................193
10.2.1 Downstream Impact on Marksmanship ....................................................................193
10.2.2 Downstream Impact on Training in the Units ..........................................................194
10.2.3 Downstream Impact on Self Efficacy.......................................................................195
10.3 Implications for Service System Development: Validation.............................................196
10.4 References ........................................................................................................................197

Chapter 11. Implications for Service System Development.........................................................198
11.1 Lessons Learned about Transfer of OBTE.......................................................................198
11.2 Implications for Service System Development................................................................199
11.2.1 Further Development and Analysis of Stakeholder Requirements for OBTE..........199
11.2.2 Further Development of OBTE as a Service System ...............................................199
11.2.3 Further Verification and Validation of OBTE..........................................................201
11.3 References ........................................................................................................................203

xiii Riccio, Diedrich, & Cortes (Eds.)
Section III. Further Development of OBTE as a Service System ..........................................206

Chapter 12. Development of General Measures for Students ......................................................207
12.1 Intent ................................................................................................................................207
12.2 Performance Measure Development Process...................................................................207
12.2.1 Phase One: Define Performance Indicators (PI).......................................................207
12.2.2 Phase Two: Translate PI into performance measures...............................................208
12.2.3 Phase Three: Measure refinement.............................................................................208
12.2.4 Phase Four: Retranslation of Measures ....................................................................208
12.3 Product of Measure Development....................................................................................209
12.3.1 Learner Perception of the Instructor and Course ......................................................209
12.3.2 Learner Engagement .................................................................................................211
12.3.3 Student Relationship with Teacher ...........................................................................212
12.3.4 Student Results .........................................................................................................214
12.3.5 Self-Report Measures ...............................................................................................216
12.4 Conclusion........................................................................................................................217
12.5 References ........................................................................................................................217

Chapter 13. Adapting OBTE in a Classroom Environment .........................................................219
13.1 Intent ................................................................................................................................219
13.2 Observing OBTE in the Classroom Environment............................................................219
13.2.1. Participants...............................................................................................................219
13.2.2. Procedure .................................................................................................................220
13.2.3. Measures ..................................................................................................................220
13.3 Utility of OBTE Measures in a Classroom Environment ................................................220
13.3.1 Generality of Measures.............................................................................................220
13.3.2. Implications for Improvement of Measures.............................................................221
13.3.3 Implications for improvement of course design .......................................................222
13.4 Use of 360 Reviews for Collaborative Reflection..........................................................223
13.4.1 The Role of a 360 Review in OBTE.......................................................................223
13.4.2 Narrative of a Participant Observer ..........................................................................225
13.5 Learning, cognitive load and motivation..........................................................................228
13.5.1 The NASA Task Load Index as a subjective measure of student workload.............228
13.5.2 Results.......................................................................................................................229
13.5.3 Implications ..............................................................................................................230
13.6 Conclusions ......................................................................................................................230
13.7 References ........................................................................................................................231

Chapter 14. Organizational Climate and Creation of Durable Change ........................................233
14.1 The Need ..........................................................................................................................233
14.2 Initial Indications of Possible Resistance to Change .......................................................234
14.3 Models and Considerations for Sustainable Change........................................................235
14.3.1 The Change Transition Period ..................................................................................235
14.3.2 Organizational Culture..............................................................................................237
14.3.3 Clarity of Mission and Shared Understanding..........................................................237
14.3.4 Relevant Observations During the Current Investigation.........................................238
14.3.5 Organizational Support and Incentives.....................................................................238
14.4 Conclusions ......................................................................................................................239
14.5 References ........................................................................................................................239
Table of Contents xiv
Chapter 15. Five ways OBTE can enable the Army Leader Development Strategy....................242
15.1 Background ......................................................................................................................242
15.2 An Emerging Consensus ..................................................................................................244
15.2.1 What Part to Balance?...............................................................................................244
15.2.2 Improving Training, by Design ................................................................................245
15.2.3 Increased Use of dL and Dependence on Self-Development ...................................246
15.2.4 Future Orientation, Unknown Requirements............................................................247
15.2.5 The Quality Instructor Challenge .............................................................................247
15.2.6 Purpose and Design are Key.....................................................................................248
15.2.7 A Natural Advantage ................................................................................................249
15.2.8 Task Specialization or Generalized Competency .....................................................249
15.3 Conclusion........................................................................................................................251
15.4 References ........................................................................................................................252

Epilogue. Integration of Leader Development, Education, Training, and Self-Development .....254
Toward Values-Based Standards for Army Doctrinal Requirements ......................................254
Nested Standards and Quality Assurance.................................................................................256
Needs and Opportunities for Staff & Faculty Development ....................................................259
A Role for Science and Measurement .................................................................................259
Toward Best Practices in Instructor Education....................................................................260
Critical Considerations for Further Scientific Investigation ....................................................263
The Necessity of Long-Term Studies ..................................................................................263
False Dichotomy of Objective-Subjective...........................................................................264
Clarity About What Is Evaluated.........................................................................................265
Next Steps ............................................................................................................................266
References ................................................................................................................................268

Section IV. Appendices...............................................................................................................270

Appendix A. OBTE Principles & Practices: Instructor Measures................................................271
A.1 Genesis of Formative Measures for Instructors ................................................................271
A.2 Principles of Outcomes-Based Training & Education ......................................................272
A.3 Guide to Using Measures of Instructor Behavior..............................................................276
A.4 Complete Menu of Instructor Measures............................................................................279

Appendix B. OBTE Principles & Practices: Student Measures ...................................................318
B.1 Guide to Using Measures of Student Behavior .................................................................318
B.2 Complete Menu of Student Measures ...............................................................................319

Appendix C: A Commanders View of Outcomes-Based Training and Education.....................340
Summary ..................................................................................................................................340
Definition.............................................................................................................................340
Description...........................................................................................................................340
Elements of OBTE. ..................................................................................................................341
Developing the Outcomes....................................................................................................341
Developing the Training Plan..............................................................................................341
Conducting Training............................................................................................................342
How Training is Assessed....................................................................................................344
Conclusion................................................................................................................................344
xv Riccio, Diedrich, & Cortes (Eds.)
Appendix D: Warrior Ethos..........................................................................................................345
Analysis of the Concept and Initial Development of Applications..........................................345
Current Understanding of Warrior Ethos.............................................................................345
Purpose.................................................................................................................................348
Approach..............................................................................................................................348
Expansion of the Definition of Warrior Ethos.....................................................................348
The Tenets of Warrior Ethos ...............................................................................................349
Clarifying the Definition of Warrior Ethos..........................................................................351
Warrior Attributes Derived from the Tenets of Warrior Ethos ...........................................353
References ................................................................................................................................355
Supplementary Work Product from Warrior Ethos Project .....................................................355

Appendix E: Indicators of Warrior Ethos.....................................................................................356
Methods....................................................................................................................................356
Participants...........................................................................................................................356
Instruments and Facilities ....................................................................................................356
Procedure .............................................................................................................................356
Results ......................................................................................................................................358
Qualitative Findings.............................................................................................................358
Quantitative Findings...........................................................................................................358
Discussion ................................................................................................................................359

!"#$%&'#( #* &+, '(",-&'./&'#( /- 0,*$,1&,2 '( &+, 1+/3&,0- #* &+'- 4#(#.0/3+5

Principles and Practices of Outcomes Based Training & Education

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Principles and Practices of Outcomes Based Training & Education

Uploaded by

Copyright:

Available Formats

Riccio, G., & Darwin, M. (2010). Principles and Practices of Outcomes Based Training & Education.

In: Riccio, G., Diedrich, F., & Cortes, M.

You might also like

Principles and Practices of Outcomes Based Training &amp; Education

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Principles and Practices of Outcomes Based Training &amp; Education

Uploaded by

Copyright:

Available Formats

Riccio, G., & Darwin, M. (2010). Principles and Practices of Outcomes Based Training & Education.

In: Riccio, G., Diedrich, F., & Cortes, M.

You might also like

Principles and Practices of Outcomes Based Training & Education

Principles and Practices of Outcomes Based Training & Education