Cooperation Begins Encouraging Critical Thinking Skills

Computers & Education 97 (2016) 97e115
Contents lists available at ScienceDirect
Computers & Education

journal homepage: www.elsevier.com/locate/compedu
Cooperation begins: Encouraging critical thinking skills

through cooperative reciprocity using a mobile learning game
Hyunjeong Lee a, David Parsons b, Gyuhyun Kwon a, Jieun Kim a,
Krassie Petrova c, Eunju Jeong a, Hokyoung Ryu a, *
a
Hanyang University, Seoul, South Korea
b
The Mind Lab. Unitech, Auckland, New Zealand
c
AUT University, Auckland, New Zealand
a r t i c l e i n f o a b s t r a c t
Article history: Mobile learning has the advantage of being able to be used within and between contexts
Received 4 March 2015 and can also be seamlessly integrated into broader learning experiences that include other
Received in revised form 29 February 2016 forms of learning. Such experiences can assist in the development of cognitive and
Accepted 11 March 2016
collaborative skills by encouraging learners to work together to solve problems, see others'
Available online 16 March 2016
perspectives and cooperatively nd creative and critical solutions. This paper describes a
serious mobile learning game designed to allow participants to play the role of business
Keywords:
consultants to an organisation facing some serious challenges. It uses mixed reality re-
Mobile learning
Cooperative/collaborative learning
sources to lead the players through a realistic scenario, providing them with physical,
Critical thinking skills cognitive and collaborative challenges. Following mobile learning, the learners demon-
Collective interest strate their critical insights into the learning content by creating a consulting presentation
21st century learner in the classroom. Our study contrasts group cooperation where each learner is given
asymmetric learning contents, with a cooperative group with two single learners given
symmetric (identical) learning contents. We present the results of an experiment designed
to measure the effectiveness of asymmetric learning content in fostering cooperative
critical thinking, as examined by content and conversation analysis whilst preparing the
consulting presentation. We found that the implicit cooperation condition e cooperative
reciprocity, triggered by the asymmetric learning contents - was important for maximising
critical thinking skills.
2016 Elsevier Ltd. All rights reserved.
Competition has been shown to be useful up to a certain point and no further, but cooperation, which is the thing we must
strive for today, begins where competition leaves off. d Franklin D. Roosevelt
Ruin is the destination toward which all men rush, each pursuing his own best interest in a society that believes in the
freedom of the commons. Freedom in a commons brings ruin to all. (Hardin, 1968)
* Corresponding author.
E-mail address: hryu@hanyang.ac.kr (H. Ryu).
http://dx.doi.org/10.1016/j.compedu.2016.03.006
0360-1315/ 2016 Elsevier Ltd. All rights reserved.
98 H. Lee et al. / Computers & Education 97 (2016) 97e115
1. Introduction
Hardin's Tragedy of the Commons (1968), although dealing with the depletion of natural resources rather than the gaining
of knowledge, captures the difculty of human cooperation and the power of self-interest over collective interest. We know
that cooperation is hard, sometimes it is arguably an impossibility (Greene, 2014), yet without it we are left only with
competition. Cooperation can be fostered, but only if self-interest and collective interest are well aligned. In this article, we
endeavour to show that learning motivation can involve more than self-interest by aligning cooperation with simultaneous
benet (i.e., cooperative reciprocity e Trivers, 1971). We report on a learning activity using a mobile mixed-reality serious
game, in which collective interest was deliberately engaged in one of the learning conditions. We also examine to what extent
the collective interest shared by the learners made them more creative and critical in constructing their knowledge.
1.1. Critical thinking skills from cooperative learning: a mobile learning approach
In Dickens' Hard Times (1854), the Member of Parliament for Coketown, Thomas Gradgrind, insists that facts are the
fundamental and essential building blocks of every educational activity. In the 21st century, facts alone do not have any
advantages when they can be easily looked up on the Web. Thus contemporary education is not so much about factual in-
formation, but what we can do with it. This also has implications for the nature of remembering; remembering factual
knowledge is less important in an information rich environment, but other forms (e.g., meta-cognitive knowledge, critical
thinking skills, creative and innovative ideas) become more important. Therefore, we have to try to help students develop a
capacity and appetite for learning and divergent thinking that will stand them in good stead their whole lives. The Partnership
for 21st Century Skills (2011) suggests the 4Cs (critical thinking, communication, collaboration, creativity) as the contem-
porary core skills.
There is little question that the nature of education must change to meet the challenges of the 21st century, now that
knowledge production and dissemination is a distributed and interactive activity, mediated by Information and Communi-
cation Technologies, entailing a shift from knowledge production to knowledge conguration (Gibbons, 1998). Whilst a
greater level of inclusion in education, particularly higher education, has been seen in many countries, simply producing more
graduates is not itself a solution to the problem of educating for a changing world (Chang, 2010). Rather, there is a widely
perceived need to provide an education that is somewhat more pertinent to the needs of both individuals and society.
Much has been written about what types of skills a 21st century learner needs to develop. In general, the core skills are
taken from the higher levels of Bloom's taxonomy; analysis, synthesis and evaluation in the original taxonomy; analyse,
evaluate and create in the revised taxonomy (Krathwohl, 2002), increasingly supplemented by newly required skills such as
the ability to lter out meaning from the mass of information only recently made available to us (Dede, 2007). Often these
skills are summarised as aspects of critical thinking. Willingham (2008) referred to critical thinking as seeing both sides of an
issue, being open to new evidence that conrms your ideas, reasoning dispassionately, deducing and inferring conclusions
from available facts and solving problems.
A number of methods have been suggested to assist the development of critical thinking. Cooperative learning, where two
or more people learn something together, is one such example (Dillenbourg, 1999). Abrami, Bernard, Borokhovski, Wade,
Surkes, Tamim, et al. (2008) note in their review of the literature that collaboration has a positive effect on critical
thinking skills (though other factors in the learning context are also important). We might contrast this with the view that
self-interest (or at least lack of interest in cooperation) kills creativity (Bechtoldt, Dreu, Nijstad, & Choi, 2010). Gokhale (1995)
suggested that cooperative learning could enhance the development of critical thinking through an alternating speaker-
listener process of evaluating, discussing and clarifying a partner's thinking. A further question is the extent to which coop-
erative learning is encouraged by mutual benet (i.e., reciprocity e Salomon & Perkins, 1998; Donahue, Bowyer, & Rosenberg,
2003). We are more likely to offer help when we feel that others might help us. In contrast, uncooperative behaviour triggered
by self-interest will lose the potential benets of future cooperation. Palmer and Steadman (1997), Brosnan, Freeman, and De
Waal (2006), Simon (1990) and others also claimed that mechanisms of cooperative reciprocity can account for various types
of cooperation that spread through many different cultural settings. In this article, therefore, we question whether an
expectation of another learner's cooperation might encourage one's own cooperative learning behaviour, and in turn whether
such acts would foster more critical thinking skills.
A primary research theme addressed in this article is how we might leverage a mobile learning activity to trigger such
cooperative learning. Using a mobile serious mixed-reality learning game, designed to allow two learners to play together the
role of business consultants to an organisation, we describe the cooperative reciprocity provided in the learning activity, and
explore the impact of this on critical thinking.
1.2. Cooperation begins with mobile mixed-reality learning
Game-based learning has become an increasing focus for educational researchers (Prensky, 2005; Dondlinger, 2007). Its
focus on both learning content and motivation (through gamication concepts such as rewards and rankings) is seen as
effective (Prensky, 2005). Further, situated and contextual game-based problem-solving in a physical environment can
present direct learning experiences (Tan & Soh, 2010). The assumption is that the pairing of instructional content with certain
H. Lee et al. / Computers & Education 97 (2016) 97e115 99
gaming features (rewards, feedback etc.), combined with meaningful learning contexts, will be more engaging for learners
and thus achieve the desired instructional goals (Annetta, 2008).
A popular domain for serious games is the business curriculum (Faria, Hutchinson, Wellington, & Gold, 2009), where they
can be used to simulate various organisational issues, e.g., understanding multi-dimensional perspectives from different
departments in an organisation. Examples include a longitudinal web based game that simulates a business consulting ex-
ercise (Bos & Gordon, 2005), and an MMS (Multimedia Message Service)-based game in which students reect on the new
digital economy (Kittl, Edegger, & Petrovic, 2008). Yang (2015) describes how the use of commercial games that simulate
business contexts can increase both collaboration and higher-level skills development. The game described in this paper also
addresses the business domain, and involves the simulation of a business consultancy to a ctitious mobile phone
manufacturing company.
Mobile serious games in general have been shown to increase collaborative problem solving skills among learners
nchez & Olivares, 2011.) Nam and Zellner (2011) discuss how games that are only competitive promote negative inter-
(Sa
dependence, while games that are also cooperative promote positive interdependence. Mobile augmented reality (AR) games
in particular have frequently been used to support cooperative learning. Environmental Detectives (Klopfer, Squire, & Jenkins,
2002; see Fig. 1(top)), for instance, put players in the role of environment experts in charge of investigating a toxic leak at a
real geographical location. The players carried location-aware devices that allowed them to collect virtual eld evidence and
interview virtual characters on the mobile device to achieve their goals. In Premierloytnant Bielke (Wake & Baggetun, 2009)
players used a mobile game to learn about the local history of their city by reproducing the steps of a soldier trying to build
Fig. 1. Several mobile games in cooperative learning e Environmental Detectives (top), Savannah (middle), Alien Contact! (bottom) (copyrights to the original
authors).
gunboats in the same city in the 1800s. In some cases these collaborative AR games are in fact mixed-reality games, which use
a combination of physical and virtual artefacts (Chang, Lee, Wang, & Chen, 2010). The game described in this article is also a
mixed-reality game.
Several studies have previously addressed the question of how such collaborative games might support critical thinking.
For example, Schrier (2006) designed a mixed-reality game, Reliving the Revolution to explicitly teach critical thinking skills
with cooperative learning. The game used a combination of physical and virtual objects in a real location, and allowed
learners to relive the Battle of Lexington and try to work out who red the rst shot. The game was nonlinear, with time
constraints, and players had distinct roles with different perspectives. Playing in pairs, cooperation was essential and they
became more open to express diverse opinions and understood the necessity for applying multiple perspectives to suggest
critical ideas around current global and community issues. However, the study did not discuss why the cooperative learning
process would be triggered, or what conditions might foster an appropriate level of cooperation for critical thinking.
In this regard, the Savannah project (Facer et al., 2004; see Fig. 1(middle)) suggests how to promote cooperation by a
special instructional design. In this augmented reality activity, students role-played a pride of lionesses trying to survive in a
virtual African environment. They played individually as lionesses, but needed to work together to hunt down the prey, role-
played by other students. Each lioness had partial information about the location of prey, thus they were implicitly
encouraged to work together. More interesting, in a subsequent classroom session, the students who played together revealed
a more cooperative learning attitude in sharing their understandings from the game, and highly appreciated other students'
perspectives.
Our own mobile game follows a similar path to a number of these previous examples by putting players into specic roles
within a business domain while exploring a geo-tagged environment, and exposing them to different perspectives on the
same context, while seeking to address a given complex problem. In this respect, the game used in this article is similar to
Alien Contact! (Dede, 2007; see Fig. 1(bottom)), in which players were presented with a scenario where aliens have landed
on Earth and seem to be preparing for a number of possible actions. In teams of two, in different roles, players interviewed
virtual characters, collected different digital items, and solved puzzles to nd out why the aliens had landed. At the end,
players orally presented their ndings to the class. The potential for mixing the real, the augmented and the virtual to provide
new types of contextual learning experience, along with the seamless blending of mobile and class-based activity, is the major
learning setting behind the activity described in this paper.
1.3. Cooperative reciprocity and critical thinking skills
Biologically, humans are, to a great extent, designed for self-interest. Therefore a cooperative mindset cannot automati-
cally arise unless they can see the future benets of cooperation. It can thus be seen that the tension between self- and
collective interest might impede effective teamwork. For instance, in Savannah (Facer et al., 2004), each learning participant
had their own solo learning activity within the game, and the kind of cooperation that puts collective interest before self-
interest was quite implicit at the time of learning. It was not until the participants were back in the classroom, to do some
knowledge-sharing exercises, that they were asked to explicitly do some teamwork-based learning activities. This approach to
blended learning design, a mobile activity followed by a classroom activity, has proved popular (Kukulska-Hulme, Sharples,
Milrad, Arnedillo-Sanchez, & Vavoula, 2009), and is also the basis of the learning design described in this article.
1.3.1. Cooperative knowledge creation and sharing

Even in a learning activity designed to be cooperative, learners would be very reluctant to share their knowledge if the
effort for cooperation was too high. Conversely, if the future benet was seen to be high, learners would be very keen to
cooperate; simply, a cost-benet analysis determines levels of cooperation. A full specication of the costs and benets
relevant to cooperative learning is dependent on learning contexts, and beyond the scope of this article. Our focus is rather on
how to establish the motivation to cooperate (i.e., cooperative reciprocity), and if such cooperative reciprocity would trigger
more critical and novel solutions during knowledge-externalising in pairs. Thus we intended to empirically test, through a
mobile and blended learning activity, if this form of instruction would benet from allowing individuals to tap into each
other's creative powers.
Fig. 2 shows the cooperative knowledge construction process. The knowledge reconstructor (listener) who does not have a
particular item of information, needs to take it from the knowledge owner. In return, the knowledge owner might have more
opportunity to develop subsequent ideas from the listener's interpretation. Knowledge exchange, for the purpose of
knowledge creation, thus probably benets most from allowing each individual to build upon and tap into the creative
powers of another. To be successful, as shown in Fig. 2, cooperative learning should involve a certain level of give-and-take,
which is a protective measure in order to avoid one party doing too much taking and not enough giving, to resolve the tension
between self- and collective interest. To model how to form cooperative reciprocity for knowledge construction between the
two parties Hendriks (1999) suggests two sub-processes: i) an act of knowledge externalisation by those who have knowledge,
and ii) an act of knowledge internalisation by those seeking to acquire knowledge. As shown in Fig. 2, the rst party, the
speaker, should communicate their knowledge, consciously and willingly or not, in some form or other (by speech, in writing,
etc.) The other party, the listener, should be able to perceive these expressions of knowledge, and make sense of them.
However, the knowledge owner should gain some benet from initiating the knowledge externalisation; otherwise, the
motivation to externalise knowledge will quickly disappear.
Fig. 2. The cooperative knowledge construction process.
1.3.2. Motivations for knowledge sharing

Two psychological factors are relevant to preserve the motivation for presenting knowledge to others. One possibility is
that the knowledge owner will share knowledge because they expect or hope for recognition and appreciation of their
knowledge. This reputation-seeking mindset can enhance cooperation in a more direct way, and serve as a strong motivator
to cooperate with the knowledge reconstructor. Haidt (2012) argues that this mindset might have evolved to facilitate more
cooperation in, especially, large groups, by giving people incentives to demonstrate their cooperativeness and their intol-
erance of non-cooperativeness (Greene, 2014). However, if the group is small, the reputational benets from cooperation
quickly disappear, and then a more immediate benet is required. In this sense, Heffernan (2014, p. 90e91) claims that the
knowledge owner readily shares their knowledge because they expect or hope that others too will share knowledge that may
be useful to them in return. Feeling that they form a single cognitive system is thus important, when the group is particularly
small (such as the cases that we are concerned with in this study), so that their thinking styles are merged in order for each
individual to build upon and tap into the creative powers of others. That is, cooperative reciprocity seems to be essential for
establishing a cooperative mindset in a small group, and further, trust arising from the single cognitive system within the
small group ensures future cooperation.
At the centre of this psychological process is empathy. To encourage empathy in cooperative learning, three steps are
needed (Greene, 2014). Firstly, the knowledge owner has to share attention with the knowledge reconstructor, to look at or
talk about the same thing. Once they are both focused on the same thing, their thinking starts to merge, because they can react
to the same event. When they are evaluating the same event or experience, they are setting the stage for thinking and feeling
the same way as well. Second, once their minds are attending to the same event or information they feel that they form a
cooperative group or a unied cognitive system. The challenge is to get such a group started and to prevent it from falling
apart. In recent years, cognitive neuroscientists have studied the neural basis of empathy and found that watching another
person experience pain, for example, engages the same emotion-related neural circuits that are engaged when one expe-
riences pain oneself, and the brains of the people who report having high levels of empathy towards others exhibit this effect
more strongly (Singer & Lamm, 2009). Finally, thinking styles are accordingly merged, once empathy is formed. Thoughts and
feelings come from what they are sharing, so they are likely to be feeling and thinking like a single cognitive system (i.e.,
distributed cognition - Hutchins, 1995).
1.3.3. Cooperative reciprocity and critical thinking

Cooperative reciprocity seems a necessary condition for effective cooperation, but it is not sufcient for developing critical
thinking skills. For teamwork-based learning design, it is common that learners do not have all the items of knowledge.
Instead, they are deliberately asked to learn specialized knowledge, and then later cooperate to nd solutions together. In this
kind of learning design, if person A does not cooperate with person B, person B can punish person A for not being cooperative
so that next time person A will be more cooperative. Conversely, if person A does cooperate with person B, person B can
reciprocally help person A, so that their teamwork-based learning will be further consolidated. It is thus believed that team-
based learning with cooperative reciprocity might readily present multiple perspectives, and develop more critical thinking
skills.
A philosophical framework suggested by Dennett (1989) - intentional stance - is relevant here. It is dened as the human
ability to understand another individual's state of mind, and particularly their intentions towards externalising knowledge.
He claims that the level of intentional stance is a hierarchical metric that we might think of as a series of reexive mind states,
and this conception could be used for explaining the levels of one's cooperational intentionality. For instance, I know the fact
(X) is equated with rst order intentional stance. By comparison, one's own beliefs about the beliefs of another (e.g., I think
you know the fact (X); You think I know the fact (X); I don't think you know the fact (X); You don't think I know the fact (X)) are
various cases of second order intentional stance. This higher order intentional stance asks what it is that someone else is trying
to convey, which is a critical thinking skill for determining the levels of cooperation with others. Hence, if someone believes
that another already knows all the facts (as shown at the top of Fig. 3), they would be less likely cooperate to learn. In contrast,
Fig. 3. Two distinct cooperative learning settings: a) Two people have exactly the same items of knowledge; b) Two people need to intentionally share their own
knowledge. The dotted lines in (a) represent implicit knowledge sharing, and the solid lines in (b) represent explicit knowledge sharing.
if they were aware that they did not know something (as shown at the bottom of Fig. 3), they would be more likely to form a
strong level of cooperative reciprocity.
The levels of critical thinking are neither easily observable, nor are they the primary concern of this article. Our aim in this
study is to examine if the collective interest shared by the participating cooperative learners triggers critical thinking skills in
constructing their learning. Ennis (1987) dened four categories of critical thinking skills: clarication, assessing evidence,
making and judging inferences, and using appropriate strategies and tactics. Newman, Johnson, Webb, and Cochrane (1997)
suggested several critical thinking skills (see Table 1) based on the research of Garrison (1992) and Henri (1992). Relating
to both cooperative reciprocity and critical thinking skills, a key theme of this article is how these observable critical thinking
skills might be made evident by our learning design.
Our discussion in this section suggests that if someone thinks they already know everything, there is no reason to be more
cooperative in order to learn something from others. Conversely, if they believe there is something they need to know from
others, they would follow a rather different knowledge externalisation process. That is, if one member of a pair thinks the
other member has the same knowledge (e.g., I think you already know what I know), then the focus of intentional stance
would simply rest on just conrming knowledge rather than trading knowledge. On the other hand, if one thinks the other
has different knowledge (e.g., I think you know something that I don't know), then it is assumed that a stronger cooperative
reciprocity is formed and critical ideas and solutions would be likely to emerge in the knowledge externalisation process.
In this sense, asymmetric information is often used in many learning settings (e.g., Schmitz, Schuffelen, Kreijns, Klemke, &
Specht, 2015). This approach usually examines if cooperative learners with asymmetric information would be better off than
individual learners, in terms of how much factual knowledge they share together (this is commonly measured by the total
amount of shared information). Knowledge exchange for the purpose of knowledge creation, however, probably benets
cooperative learners by allowing each individual to build upon and tap into the creative powers of another, not simply
counting how many facts they can recall. Our aim in this study is thus to examine if the collective interest shared by the
participating cooperative learners with asymmetric information would be stronger than those who have symmetric infor-
mation in performing their subsequent learning activities, which is strikingly different from the previous approach.
The rest of this paper describes a learning activity that blends mobile and classroom based learning, in which students are
visiting and consulting an imaginary organisation facing some serious managerial issues. Here, as discussed above, in
externalising one's knowledge to foster cooperative learning, cooperative reciprocity is assumed to be essential, by which we
would hope to see more creative and critical solutions.
2. Mobile & blended learning design: Kiwi Mobile
In this section we briey describe the mobile learning game that was used in the experiment, then describe the experi-
mental conditions used in our study.
Table 1
Critical thinking skills (extended by Newman et al. (1997), Henri (1992) and Garrison (1992)).
Henri (1992) Garrison (1992) Newman et al. (1997)

Elementary clarication Problem identication Ambiguity
e.g., identifying relevant elements, reformulating e.g., desire to understand; e.g., discussing relevant statements or ambiguities to clear up
problems, asking relevant questions aware of issues
In-depth clarication Problem denition Importance, Novelty
e.g., establishing referential criteria, seeking out e.g., clarifying subjecte.g., discussing important issues, suggesting new ideas for discussion,
specialized information seeking new problem-related information
Problem exploration Linking ideas (outside knowledge, width of knowledge)
e.g., exploring new ideas, e.g., linking facts, ideas and notions, generating new data from information
and disentangling ideas collected, drawing on personal experience, using relevant outside
materials, discussing within a larger perspective
Judgement Problem applicability Justication
e.g., judging the relevance of solutions, value e.g., Critical assessment, e.g., justifying solutions or judgements, providing proof of examples
judgements and judging solutions
Inferences Problem integration Critical assessments
e.g., making generalizations, drawing conclusions e.g., Testing solutions, and e.g., critical assessment/evaluation of own or others' contributions
Strategies relating to other Practical utility
e.g., deciding on the actions to be taken, situations e.g., relating possible solutions to familiar situations, discussing practical
proposing more solutions, interacting with utility of new ideas
those concerned
2.1. Mobile learning game design
Our mobile learning game was developed for the Android platform and uses GPS positioning and the Google Maps API for
contextual awareness. It is a mixed-reality outdoor game designed for two players. It follows a narrative learning path, re-
ected by a physical path through a series of geo-located waypoints, as players move around a physical location that rep-
resents a virtual set of destinations. Important to the concept of the mobile learning game is that the physical locations
represent virtual aspects of a ctional organisation. For example, when conguring the game, various physical buildings on
the site can be chosen to represent the various departments in the organisation. In our experiment, different buildings on the
university campus were used to play the roles of the following departments: Management Ofce, Quality Assurance, R&D,
Marketing, and Assembly Production. Details of the game's design have been published elsewhere (Parsons, Ryu & Petrova,
2012), so coverage in this section is limited to a brief summary, though it should be noted that the game is being continually
updated.
When playing the game, the players, in the role of business consultants, navigate to locations and investigate a problem
presented to them at the beginning of the learning session, which is in the context of a ctional mobile phone manufacturing
company (Kiwi Mobile) that is facing some problems. Each physical location represents a part of the company, and has both
virtual and physical artefacts (i.e., information about the departments, interview videos with department managers, and
relevant questions) that mimic the work context of an organisation. Each of these artefacts implicitly presents a series of
departmental perspectives on the problems faced by the company. Hence, as they engage in the learning process, players have
to gather, analyse, infer and reect on various (and sometimes conicting) pieces of information. The mobile learning activity
aims to exercise several cognitive skills, in particular, critical thinking (Why am I getting different stories from different
people? How can these different perspectives within the organisation be reconciled?) In addition, the game is designed to
support cooperative learning though pair-play. This cooperative learning is non-zero-sum; one player does not win at the
expense of the other, rather both players win together, because the decisions jointly made by the players determine the
outcome of the game. This outcome is decided in a classroom session following the game play, involving creating a summary
report and presentation for the CEO.
The game was originally developed for English language users, but many of the materials in the game can easily be
translated into other languages using XML conguration les. Following a pilot experiment where the English language
version of the game was used with Korean students, it was recognised that working in English made it difcult for the
students to fully understand the game's content and learning goals. Therefore, for the experiment reported in this article, the
game was congured to use the Korean language. Some small elements of English text remained in the game. Fig. 4 shows
examples of the three main component types in the game; navigation (map view shown), virtual artefacts (video shown) and
physical artefacts (newspaper article shown). For international readers, English language versions of these game components
are shown here.
It is important that a serious mobile learning game has some kind of narrative ow that leads the player through the
activity. The core narrative that we have adopted in our game is taken from Larsen's model of standard linear ction as
described in Smith (2000). This model draws the learner through a series of stages: teaser, elaboration, conict escalation,
climax and resolution, as shown in Fig. 5. The concept of conict escalation in the context of a cooperative learning activity
is deepened by having branching learning paths. Each node in the learning path relates to a physical location, a virtual
location, an interview with a character in the game, and an artefact. In Fig. 5 we show the virtual locations, mapped to the
elements of the narrative ow described above. The teaser lays out the initial problem to the players and leads them to the
Fig. 4. (Left to Right) Map view with current position and waypoint (1), video interview with the CEO (2), the newspaper article artefact (3) from the rst
waypoint in the game.
Fig. 5. The learning paths. In the single player mode, the learner visits ve departments. In the pair-player mode each learner follows a slightly different path to
visit four departments each.
elaboration. After this stage the players are sent to different departments in the company and are given conicting infor-
mation at different locations. The climax gives them the nal piece of the jigsaw that should help them to the point of
resolution. At the resolution stage the players provide feedback about the issues they have identied as causing the problem
under investigation, by creating a presentation. Although the game is designed to be played in pairs, it can also be congured
in single player mode, where an individual player visits all the departments (as shown in Fig. 5.) There were a total of ve
locations that players had to visit when playing the game. In the single player mode, the player individually visits all the
locations. In the pair-player mode, each player visits only four departments; one player does not receive information about
the Marketing Department, and the other one does not receive information about the Research and Development (R&D)
Department. This design is deliberate in order to encourage collaboration.
2.2. Experimental method
Given the prior discussions about the need for higher-level skills development, the use of cooperative learning to foster
critical thinking skills, and the potential for mobile, mixed-reality learning to support such processes, we designed a mobile
and blended learning activity to examine the development of critical thinking skills. The main research questions addressed
in the empirical study were: i) is there a difference between solo learning and cooperative learning with respect to
demonstrating critical thinking skills; ii) is the cooperative learning effect on critical thinking skills enhanced when a strong
cooperative reciprocity is formed?
In order to address these research questions, we used the mobile learning game described above to contrast the outcomes
from both solo and cooperative learning. In one learning context (pair-player mode) we deliberately provided asymmetric
learning contents, as shown in Fig. 5 (i.e., Person A receives information from the Management Ofce, Quality Assurance,
Marketing, and Production, while Person B receives the same information, but visits the R&D department instead of Mar-
keting). In other words, the two learners in a pair have a knowledge gap, or different perspectives, of the Marketing and R&D
departments. Because of this knowledge gap between the two learners in a pair, it is expected that they will have to cooperate
to maximise their understanding in the classroom session that follows on from the mobile learning game. In contrast, in the
other learning context, two single players are working together in the classroom after solo game play (i.e., both Persons A and
B individually visit all ve departments). We also provided a solo learning condition where there is no collaboration, as a
control condition. Thus these three conditions; the pair-playing group (asymmetric knowledge), the group with two single
players (symmetric knowledge), and a solo learning group (control condition), are the main manipulations in this article. Note
that in the two single players mode, both players visit all the locations individually. It is assumed that they would have a
weaker cooperative reciprocity in the classroom activity from being aware that they have few knowledge gaps.
2.2.1. Administration of the experiment

Twenty-ve undergraduate students majoring in Industrial Engineering (IE) at Hanyang University, Korea, participated in
the experiment. They had similar prior knowledge levels on manufacturing industry consultancy (third-year IE undergrad-
uate students, mean GPA 3.5, s.d. 0.4) and age group (mean 23.92 yrs, s.d. 1.52 yrs), being recruited from the course
Introduction to Operation Management. The participants were randomly assigned to one of three groups:
1. [Group I e Control condition - 5 people] Solo learning mode: Single players must visit the ve departments and indi-
vidually learn how each department sees the organisational problems;
2. [Group II e Knowledge Asymmetry - 10 people for 5 pair-playing groups] Pair-player learning mode: Although these
players had to present in pairs, they each visited four of the departments alone. One visited all except the R&D department,
while the other visited all except the Marketing department. They did not work as a pair until after all the departments had
been visited separately. The intention of this learning mode was to see how those who have different knowledge gaps
worked together at the point where they were creating their presentations;
3. [Group III e Knowledge Symmetry - 10 people for 5 pair-playing groups] Two single players learning mode: Two players
visit all ve departments separately, then work together later.
These manipulations e solo learning, pair-player learning with knowledge asymmetry, and two player learning indi-
vidually with knowledge symmetry, were designed to answer our research questions, in particular to see if a cooperative link
arising from a strong cooperative reciprocity supports the development of critical thinking skills.
Prior to the activity, players were briefed on the objectives of the consulting exercise, and how the game play works. Then
the participants were asked to play the mobile learning game (described in the previous section.) They were allowed to freely
navigate the site with the mobile phone, and there were no time limits on visiting the departments. At each geo-tagged
location, they had to view the virtual and physical artefacts, then answer a quiz about that location before moving on, to
ensure that they were following the correct learning path.
After the game play, each learner was guided to a classroom, and asked to prepare a summary report and presentation for
the CEO (the role of CEO was taken by the course lecturer). In this blended learning session, the solo learners had to make
their report and presentation alone, while the pair-players and the two single players worked together in their pairs. This
blended learning session consisted of two phases: i) the sharing phase (15 min) and ii) the solution phase (15 min). We
strongly encouraged (but did not force) the participants to share the knowledge they had gained in the sharing session; in
contrast, in the solution session, they were encouraged to nd creative and critical solutions. Of course, the solo players did
not need to share knowledge, so they were all given a 30 min solution session. Finally, another 5 min were allowed for
presenting their ndings about the problems of Kiwi Mobile to the CEO. This learning design was intended to mimic a natural
setting for the roles of business consultants.
2.3. Learning procedure
The learning process within the game is based on an initial story about the ctional company. In the video interviews, as
well as information being provided about that person's perceptions of the company's difculties, each character also refers to
a physical artefact that the player must locate and look at, and makes statements about other departments and/or characters
in the game. The artefacts introduced by the virtual interviewees are physical, rather than virtual, and are positioned close to
the waypoints (on the walls of nearby buildings). In the rst video interview, the CEO describes a negative news story about a
battery in one of the company's phones that apparently caught re. The CEO expresses the opinion that the story is false, and
that the main problem facing the company is a failure of public relations and marketing. This is associated with an artefact, a
copy of the newspaper article, which is located close to the geo-tagged location that triggers the interview. During the
interview, the CEO directs the players to go and nd the article and read it (the article is shown to the right of Fig. 4), and their
main objective is to nd who should be blamed for the news story about the re.
In terms of team play, the most important point is the conict escalation (see the learning path in Fig. 5). In the pair-
players mode (Group II), each player in the pair is directed to a different waypoint as part of their learning path. Player A
is directed to the Marketing department, while Player B is directed to the R&D department. The idea at this point is that each
of the virtual interviewees blames the other for some of the problems that the company is facing. Players in the pair-player
mode re-join at the following waypoint (Production), at which point they each possess only one perspective of the conicting
information between the Marketing and the R&D departments. In contrast to the learning path taken by the pair-player
mode, the two single-players group (Group III) follow a single learning path through the waypoints, so each of them has
rst-hand experience of the conicting information between the Marketing and R&D departments.
The penultimate waypoint in the mobile learning game (i.e., the Production department) provides a kind of resolution, in
that it answers the original question posed by the newspaper article, namely, why did the phone battery catch re? The nal
virtual character explains that there is a problem with the layout of the assembly hall that has led to batteries being damaged
while in storage, and that this has led to the battery re. Whilst this answers the question initially posed in the game, it is
important that, in the process of getting to this point, the players have also been given a range of other information about the
company that points to other serious problems that need to be addressed. At the nal waypoint, the players are taken back to
their original point of departure, the CEO's ofce, and asked what they have found out. The learning activity does not nish
with the conclusion of the mobile game. Rather, the nal question posed by the CEO leads into the second part of the learning
process in the classroom, where the teams are given the opportunity to report and present their ndings back to the CEO (and
the rest of the class) in a similar manner to Alien Contact! (Dede, 2007).
2.4. Learning outcomes, and assessing critical thinking skills
In the empirical study, the learning outcomes from the mobile and blended learning activity were measured in various
formats: detailed descriptions of the information obtained, recall of the information (from interviewees or artefacts) at each
location, and so forth. One limitation of such measures is that they do not address creative learning. Hence, the creation of a
report & presentation to the CEO as the nal phase of the learning activity is intended to move seamlessly from the contextual
learning part of the process to a situation where a number of additional skills can be exercised. Requiring the players to create
a report and presentation exercises this skill.
In this phase of the activity, the participants were required to create presentations to evaluate both their memorization
and critical thinking levels. When evaluating memorization, the main experimental task consisted of writing down infor-
mation about each department's situation in detail, while the critical thinking part consisted of analysing the relationships
between departments and providing reasons for such thinking. As shown in Fig. 6, we provided a presentation format for the
participants to ll in; If you think there is a relationship between certain departments, then link them and write down specic
reasons why you link them
For the initial part of the presentation, the participants were asked to write down the information that they got from each
department, followed by another 30 min to prepare the presentation document, in pairs for (for Groups II and III) or alone (for
Group I). The rst 15 min were for sharing their knowledge), the next 15 min for nding critical and creative solutions for the
organisation. Whilst sharing and discussing their views to prepare the presentation report, they were asked to think aloud,
and all their conversations were recorded.
Critical thinking skills were additionally assessed using verbal protocols. It is believed that when cooperative reciprocity is
formed, a small cooperative action could produce a sense of obligation to return a larger favour (i.e., the cooperative
knowledge construction process as shown in Fig. 2). This feeling of obligation in teams of two fosters more cooperative
knowledge exchanges. Applying this concept for the purpose of creating new knowledge, knowledge presented by this
cooperative reciprocity is primarily relevant as the starting point and touchstone for subsequent ideas, so it might have more
Fig. 6. Examples of CEO reports & presentations.
chance of developing new and critical thoughts. To analyse the verbal protocols, we applied two coding schemes adopted in
the literature of critical thinking skills development: Newman (1997; see Table 2) and Pena-Shaff and Nicholls (2004; see
Table 3).
3. Results
When evaluating memorization, i.e., writing down information about each department's situation, it was found that all
three groups (solo learning group e Group I, the pair-player group with knowledge asymmetry e Group II, the two single-
players group with knowledge symmetry e Group III) had no differences in terms of the number of situations described in
the summary report (i.e., on average, Group I reported 7.81 problems; 8.03 in Group II; 7.94 in Group 3). This means that the
serious mobile learning game itself did not have a different learning effect on the single players who visited all the ve
departments compared to the pair-players who each missed one of the two departments (Marketing or R&D). A non-
parametric statistical analysis (Kruskal-Wallis Test at the signicance level 0.05) was applied and no statistically signi-
cant difference was found.
Apart from identifying each department's situation, another learning effect can be measured in how our participants see
the conicts between (or among) the departments. Table 4 shows the key statements in the summary reports presented to the
CEO, which were re-coded by the authors. Note that the conicting issue between the Marketing and R&D departments was
already made explicit in the mobile learning game, so all of the learners clearly identied this problem (i.e., the lack of
communication between these departments). Apart from this issue, we were interested in seeing if the participants were able
to identify further conicts arising from other departments, since these were not overtly mentioned in the interview videos
and/or physical artefacts in the game.
The novel discoveries about conicting departments (i.e., QA e Assembly Production/R&D e QA/Marketing e Assembly
Production/R&D e Assembly Production) are thus of a greater interest for a further analysis of the learning outcomes. Looking
at Table 4, Group I only provided ve novel links (three for QA-Assembly, 1 for R&D-QA, 1 for Marketing-Assembly, and 0 for
R&D-Assembly), which might be compared with 8 for Group II (2 for QA-Assembly, 3 for R&D-QA, 1 for Marketing-Assembly,
and 2 for R&D-Assembly), and 9 for Group III (5 for QA-Assembly, 2 for R&D-QA, 1 for Marketing-Assembly, and 1 for R&D-
Assembly). However, these frequency data are not amenable to statistical analysis thanks to the nature of the difference in the
sample population (i.e., Groups II and III have ve pairs with ten participants respectively, while Group I has only ve
subjects).
Instead, to assess critical thinking skills development, we applied the coding scheme proposed by Newman (1997), as
shown in Table 5. For the negative indicators of critical thinking skills, learners tend to simply use ideas or concepts that have
already been presented to them when proposing their own solutions. This is revealed by; 1) using given learning materials
without advancing one's own (or others learners) ideas, 2) repeating what has been said or learned without adding new
elements, or 3) proposing several solutions but refraining from deciding on the most suitable one. In comparison, learners
demonstrating the positive indicators of critical thinking skills are more likely to create new information from collected data,
and develop new strategies with wide and diverse interpretations. In the present study, when evaluating critical thinking
skills, we employed the verbal protocol data whilst the learners were preparing their summary reports and presentations.
Table 5 shows examples of the verbal protocols by Groups II and III. The Critical thinking indicator column represents
whether the participant's utterance is positive or negative with respect to the six dimensions of critical thinking skills
proposed by Newman's coding scheme (1997) (clarication, assessment, novelty, justication, linking ideas, and importance,
see Table 2 for the denitions of the coding scheme). The next column (Utterances) contains the excerpts from the
Table 2
Coding schemes for critical thinking skills from verbal protocols (adopted from Newman et al., 1997).
Category Indicator Descriptions

Critical thinking Clarication Focusing on a question/analysing arguments/asking and answering questions of clarication/dening terms and
: Positive judging denitions
indicator Focusing on a question unrelated to the problem/analysing arguments inappropriately/asking inappropriate or
: Negative irrelevant questions/incorrectly answering questions of clarication and incorrectly dening terms and
indicator inappropriately judging denition
Assessment Judging the credibility of a source/making and judging observations/Critical assessment of other's contribution or
situation course material (in our experiment)
Judgments and observations based on inappropriate criteria/Uncritical acceptance or unreasoned rejection
Novelty New ideas for discussion
Repeating what has been said
Justication Providing proof of examples/Justifying solutions or judgments
Irrelevant or obscuring or examples/Offering judgments or solutions without explanations or justication
Linking Idea Linking facts, ideas and notions
Repeating information without making inferences or offering an interpretation/Stating that one shares the ideas or
opinions stated, without taking these further or adding any personal comments
Importance Important points/issues
Unimportant, trivial points/issues
Table 3
Knowledge construction categories (modied from Pena-Shaff & Nicholls, 2004).
Category Indicator Descriptions

Sharing information Questions Information seeking or knowledge conrming questions
Reply/Explain Direct responses to information-seeking questions or knowledge conrming questions
Discussing information Questions Discussion or reective questions
Reply/Explain Elaborated responses that include information sharing, clarication and elaboration, and interpretation
conversations, and the next column counts the number of such utterances (this is the same as the number of indicators). Note
that Group I (i.e., solo learners) was asked to think aloud, but most of the solo learners were very reluctant to talk aloud, so the
analysis of Group I was omitted.
Looking at Table 5, it seems that Group II had a higher number of positive utterances in almost every dimension of critical
thinking skills than Group III (33 cases vs. 30 for Clarication, 25 vs. 16 for Assessment, 29 vs. 21 for Novelty, 8 vs. 4 for
Justication, 16 vs. 2 for Linking ideas, and 23 vs. 14 for Importance). In contrast, Group II coherently revealed a smaller number
of negative utterances in every dimension of critical thinking skills (3 cases vs. 9 for Clarication, 4 vs. 6 for Assessment, 9 vs. 11
for Novelty, 1 vs. 3 for Justication, 12 vs. 19 for Linking ideas, and 3 vs. 8 for Importance). Notable in Table 5 is that more linking
ideas were created from Group II (the pair-player group with knowledge asymmetry) than Group III (the two single-players
group with knowledge symmetry). Pairwise Mann-Whitney U tests supported these interpretations (i.e., only signicant for
Linking ideas at the level of 0.05, but n.s. for other dimension).
Newman (1997) further proposed a mathematical expression to represent the critical thinking ratio during the knowledge
construction process (i.e., the conversations between the two people in a cooperative group), as Eq. (1). This equation can
summate each of the six dimensions of critical thinking skills (using the number of positive and negative indicators in the
column No. of utterance of Table 5) into a representative single value. The critical thinking ratio of each dimension for Groups
II and III is also shown in the last column of Table 5.
x x
Critical Thinking Ratio (1)
x x
x The number of positive indicators
x The number of negative indicators
1 uncritical & surface Critical Thinking Ratio 1critical & deep
To ensure the quality of the verbal protocol analysis, two independent researchers (who have more than 3 years' expe-
rience) were employed, and the positivity and negativity of each utterance were reviewed twice. The initial inter-rater
reliability was 0.75 (Cohen's Kappa), and this value increased to 0.87 in the second round. The critical thinking ratios of
both Groups II and III are also shown in Fig. 7, as a radar chart. One dimension of critical thinking skills e Linking ideas -
seems to starkly contrast between Groups II and III (0.81 for Group III vs. 0.14 for Group II), which means that the two single
players with knowledge symmetry either repeat information without making inferences or persevere with their initial ideas
or opinions without taking these further. This can arguably imply that Group III has limited knowledge exchange for the
purpose of knowledge creation so that neither learner builds upon or taps into the creative powers of the other.
Table 4
The summary report. Our participants suggested novel ideas regarding the conicting departments in Kiwi Mobile. Note that the issues between the
Marketing and R&D departments were known from the interview video (the column of Marketing e R&D).
Conicting Marketing e R&D QA e Assembly R&D e QA Marketing e R&D e Assembly

links (This issue is on production Assembly production
between the the interview video) production
departments
Group I #1 There is a lack of communication The QA department should No issue No issue No issue
between the marketing and R&D check the battery quality.
departments;
The marketing department must
consider the circumstances of the
R&D department.
#2 There is a lack of communication No issue No issue The reason for the No issue
between marketing and R&D. cost increase
proportional to the
sales is probably
the old production
line for various
products.
#3 The Marketing and R&D dept. No issue No issue No issue No issue
blame each other
#4 The Marketing didn't listen to the Assembly line with the New technology No issue No issue
requests of R&D. latest technology and a development in the
storage expansion will be R&D dept. could
required for the high enhance the sales
quality product. by multiple- and
high quality
products.
#5 There is a serious communication Assembly line should No issue No issue No issue
problem in between marketing manage the high quality
and R&D. control
Group II #1 Due to the absence of No issue No issue Too many types of Too many types of
communication between the the products led to the products led to
departments, too many types of the complexity of the complexity of
products were designed the production line the production line
(note that this (note that this
group suggested group suggested
the conicting the conicting
issues between issues between
Marketing-R&D- Marketing-R&D-
Assembly) Assembly)
#2 The lack of communication It will be better to produce They should No issue No issue
between the departments made the battery in-house for minimize the
Kiwi Mobile develop too many high quality. number of product
new products without market portfolios to make
research better quality
products
#3 There is a serious problem in No issue No issue No issue No issue
between the Marketing and R&D
#4 The Marketing and R&D blame No issue Due to many types No issue No issue
each other so there is a of mobile phone
communication problem products, it is hard
between them. to manage the
quality
#5 There is serious problem in The QA department has to Accidents in No issue Old production line
between the Marketing and R&D. nally manage products products could could limit the
Marketing strategy could redirect that arrive from the affect the following development of
the R&D's strategical focal point. Assembly dept. R&D roadmap new product
Group III #1 R&D department should follow Efcient assembly line is Quality No issue No issue
the analysis of the Marketing necessary for the quality management
dept. maintenance. should be
coordinated the
R&D department
when developing
new products.
#2 There is a serious problem in The QA was not informed The reliability tests Improvement in Research for the
between the marketing and R&D. about the accident in the in the product the old production efcient assembly
assembly line development is line could lower line is essential for
very important for cost product
product quality management.
assurance
(continued on next page)
Table 4 (continued )
Conicting Marketing e R&D QA e Assembly R&D e QA Marketing e R&D e Assembly

links (This issue is on production Assembly production
between the the interview video) production
departments
#3 A research on the marketing QA should manage the No issue No issue No issue
strategy is essential for increasing assembly line for higher
revenues quality
#4 The Marketing and R&D need to Management in the No issue No issue No issue
communicate each other assembly line is essential
for high quality
#5 There is a serious problem Cheap battery production No issue No issue No issue
between marketing and R&D. from the third-party
vendors could lower the
battery quality
Yet, analyses of the learning outcomes above are not able to show how such learning outcomes were triggered. This article
assumed that critical thinking skills can be developed if the participants are under a certain cooperative condition, a certain
level of give-and-take, by which knowledge sharing is fostered by collective interest. This can be examined from two per-
spectives: i) if the linking ideas suggested by the cooperators are triggered by such a give-and-take knowledge exchange
process, and ii) if they both have attended to the innovative solutions development process.
The rst part of the conversation analysis in Table 6 (timed at 14:21 to 14:27) shows that Person B simply rejected Person
A's idea (i.e., the conict between R&D and Assembly) by saying I have no idea of the R&D department (at 14:27). In fact,
Person B did not visit the R&D department, so she was not able to link the R&D department and the Assembly department. It
can be seen that knowledge exchange for further knowledge creation was blocked by Person B, so no linking idea was
generated in this conversation. However, the same group made a new linking idea between 14:29 and 14: 53. Here, two
utterances of clarication were followed by a linking idea between Person A (14:37) and Person B (14:40) e The R&D and
Assembly department should work closely. This part of the conversation reveals that a strong cooperative reciprocity between
the two persons was formed, because the knowledge reconstructor (listener) who did not have a particular item of infor-
mation, needed to take it from the knowledge owner, and in return, the knowledge owner had an opportunity to develop
subsequent ideas from the listener's interpretation. In the same vein, Table 7 shows a part of the conversation analysis for
Group III. Of course we cannot ensure that linking ideas only arise from the cooperative reciprocity (give-and-take). A lower
number of linking ideas from Group III than Group II (2 vs. 16) is also evidence that a weaker cooperative reciprocity between
the two participating learners was formed.
A weaker cooperative reciprocity in Group III can be further explained as when one learner begins to suggest an idea to the
other but the other does not adjust or modify the idea to create more critical solutions. Clark and Schaefer (1989) claimed that
this pattern of conversation is why some people could not develop creative knowledge. If person A assumes their contribution
was successful and adds what they said to the common ground, they may continue to build upon what they believe was
established, but if person B has not registered the contribution correctly, then they will fail to make a single cognitive system
together.
Again, to develop linking ideas, we claimed in the introduction that the knowledge owner has to share attention with the
knowledge reconstructor, to talk about the same thing. Then, once their minds are attending to the same event or infor-
mation, they feel that they form a cooperative group or a unied cognitive system (Hutchins, 1995). Of course it is not possible
to measure by observation if two people share attention on the same event or information; we instead examined if there is
any difference in the number of their knowledge externalisation items (or topics), as revealed in the conversations while they
were making the summary reports and presentations (see Table 8). In the classroom, in fact, there are two types of knowledge
externalisation. By the coding scheme proposed by Pena-Shaff and Nicholls (2004; see Table 3 above), knowledge sharing can
be revealed by i) information seeking or conrming questions, ii) direct response to questions or knowledge. In contrast,
discussing knowledge can be seen by i) reective questions and ii) elaborated responses that include clarications and in-
terpretations. Hence, any difference in these two knowledge externalisation types might reveal how they had learned.
Note that in the classroom session, we strongly recommended (but did not force) the participants to share knowledge in
the rst 15 min (Phase I e Sharing phase), and then for the next 15 min (Phase II e Solution phase) they were encouraged to
nd creative and critical solutions. We analysed all the conversations in the classroom. Notable in Table 8 is that Group II
seemed to discuss their solutions even in Phase I, attending more to the conicting issues of the organisation. This implies
that Group II had more chances to develop their solutions. A Fisher's exact test (d.f 1, 20.68, p < 0.05) of Phase I revealed the
signicant difference between Groups II and III, but this was not the case for Phase II (d.f 1, 0.78, n.s).
4. Conclusions and discussion
Many blended learning activities that integrate mobile learning (e.g., ME-Learning Experience: De Crom & De Jager, 2005)
include a component where an individual learns something independently, and collaborates later to discover something new
Table 5
Excerpts from the conversations in preparing the summary report and presentation.
Critical thinking Utterances (Examples) No. of utterance Critical

indicator ( no. of thinking
(positive/negative) indicators) ratio
Clarication Group II 33 Group II:
Twelve items? Isn't that too many? (Person A in pair #2) 0.83
Don't you think that the forklift truck worker might be the problem? (Person Group III:
B in pair #2) 0.54
Should the R&D or Marketing dept. be blamed for the high production cost?
(Person A in pair #5)
Group III 30
Is that what the CEO thinks? (Person B in pair #2)
What? More than 10 cases? (Person A in pair #5)
Do you think that the CEO meets up the R&D dept. through the request from
the Marketing dept.? (Person A in pair #4)
Group II 3
Hmm.. Sales? I thinking something is relevant to sales (Person A in pair #3)
Nope! I mentioned other things!!! (Person B in pair #3)
Group III 9
No ideas (Person A/B in pair #4)
How about something like diverse target users? (Person B in pair #5)
Assessment Group II 25 Group II:
Yes. They have some revenues, so no loss at all (Person B in pair #5) 0.72
Yes. You are right. There are too many items in the product portfolios, and no Group III:
communications (Person B in pair #4) 0.45
Group III 16
Yes. It is nothing to do with marketing (Person A in pair #5)
If that is true, we have to link them up for the conicting issues (Person B in
pair #1)
No..No.. Put the CEO in here and then link them up in here (R&D) and here
(Assembly) (Person A in pair #2)
Group II 4
No idea.. (Person A in pair #3)
Okay.. you just said. (Person A in pair #4)
Group III 6
Okay..pass.. (Person A in pair #4)
Just pass no more idea (Person B in pair #5)
Novelty Group II 29 Group II:
I think there is some issue of battery power? (Person A in pair #5) 0.34
Do you think that QA and Marketing is relevant? (Person A in pair #1) Group III:
What artefact did you see? (Person A in pair #2) 0.24
Marketing is problematic. Too wide a range of target users are considered
(Person B in pair #4)
Assembly is not able to produce new items because it has an obsolete line. If
so, marketing should wait until new product is introduced. (Person A in pair
#1)
Group III 21
The battery module is being made by third parties (Person A in pair #2)
Assembly? Assembly is relevant to R&D (Person B in pair #4)
Group II 9
There are workstation problems The forklift truck is also a problem
Group III 11
The graphs of sales and unit production costs are in the artefact. (Person A in
pair #5)
The marketing dept. blamed the R&D dept. (Person A in pair #3)
Justication Group II 8 Group II:
Yes. There are some gains. No loss at all. (Person A in pair #3) 0.78
Should be every department be linked? They all work for the company Group III:
Why do they think they are all alone? (Person A in pair #2) 0.14
Because there are a lot of R&D activities, it is necessary to set up an assembly
line to make a large variety of products for small orders. (Person B in pair #5)
Group III 4
So... I think that makes this problem in this department (Person A in pair #2)
Too many unreliable products are purchased by the Assembly line (Person B
in pair #4)
Group II 1
Nope It is not an issue of the company. Costs are nothing to do with the
company. (Person B in pair #3)
Group III 3
No I don't think so. (Person A in pair #4)
(continued on next page)
Table 5 (continued )
Critical thinking Utterances (Examples) No. of utterance Critical

indicator ( no. of thinking
(positive/negative) indicators) ratio
Linking Idea Group II 16 Group II:
Multiple portfolios? Wow.. It will cost a lot Don't you think? (Person B in 0.14
pair #2) Group III:
Now it has too many types of products, so it is difcult to develop, and then 0.81
no further communication is made from other departments. So they all face a
quite difcult time (Person A in pair #1)
For a petty order, the Assembly line could not make the due date the old
factory layout does not make this petty order in this short production
schedule (Person A in pair #2)
Group III 2
More commercials and advertisement are needed. (Persons A/B in pair #4)
Group II 12
I have no idea of the R&D department (Person A in pair #4)
Group III 19
The marketing dept. keeps introducing new products (Person A in pair #5)
They already have existing product portfolios, why do they think of another
new item? (Person B in pair #3)
Importance Group II 23 Group II:
There are too many product items, and no communications between the 0.77
departments, and the R&D dept. is seriously suffering from this lack of Group III:
communication. (Person B in pair #1) 0.27
The marketing strategy will dominate the following research roadmap?
Group III 14
Revenues against sales in last two years were getting worse, this might come
from the unit production cost which increased a lot. (Person A in pair #3)
Group II 3
The CEO is the professor? (Person A in pair #4)
Group III 8
If I were near the phone, I might get burnt (Person A in pair #3)
Fig. 7. Critical thinking ratios e Group II vs. Group III. The dimension of linking ideas is in stark contrast (0.81 for Group III vs. 0.14 for Group II). The other
dimensions are more or less comparable between Group II and Group III.
in a classroom context. In the same vein, in our learning setting, the players explore the problem space with a mobile learning
game, and later discuss their ndings in the classroom. What is interesting in this approach is how mobile learning and
traditional classroom learning can be blended and, in this type of integrated learning context, to see how each learner relates
their own ndings to those of others.
A prominent way of working in the mobile learning context is through communication and cooperation. This study
empirically demonstrated how mobile learning can foster cooperative learning activities by quickly engaging with the mind
of another person, and hinted how cooperative reciprocity could be formed through a cooperative mobile learning game and a
subsequent classroom communication. However, it should be also noted that this article does not intend to revisit the benets
Table 6
A part of the conversation analysis of how learners develop their resolution in Group II (Pair #1).
Time Person Critical thinking Utterances Cooperative

indicator reciprocity

14:21 A Hmm The R&D dept. and Assembly might be associated? X
14:25 A Linking ideas () Perhaps, the R&D needs to study a product design of how Assembly can make productions easier?
14:27 B Linking ideas () I have no idea of the R&D.
14:29 A Clarication () It is the R&D department that develops new products O
14:33 B Clarication () Then, the products developed in the R&D department are all considered by the Assembly department
for mass production?
14:37 A Linking ideas () Hmm.. Then, the R&D department should consider easy assembling for the department?
14:40 B Linking ideas () Well I think a new production line should be considered unless the old one meets the new products
by the R&D.
14:53 A Assessment () Yes.. That's cool idea

17:34 B Novelty () Does the Marketing and the Assembly relate to each other? X
17:47 A Linking ideas () I have no idea
18:06 B Linking ideas () Does the Marketing and the QA relate to each other? O
Novelty ()
18:21 B Clarication () The Marketing department mentioned that there were no quality problems
18:36 A Importance () Then, Assembly and QA might be related?
18:38 B Assessment () Yes, I think it is.

19:08 B Clarication () R&D is just for the products? O
Novelty ()
19:15 A Clarication () New product development
19:29 B Linking ideas () Aha!! Then, the Assembly department needs to consider if its current production line can cope with
Novelty () this new product developed by the R&D department.
19:45 A Importance () Yes. The marketing strategy will guide the subsequent R&D activities.

of mobile learning. Instead, it explored what situations or learning settings are likely to help the development of critical
thinking skills.
Our rst research goal was to see how critical thinking skills developed in cooperative mobile learning groups. Addi-
tionally, the second research goal revolved around how decient knowledge motivates the cooperative learning processes. To
examine this we constructed two cooperative learning groups, each with different game playing modes, along with a third
(control) group of solo learners. When evaluating their critical thinking skills, as revealed by conversation analysis, there were
signicant differences among the three groups. We found that the solo learners usually identied only simple organisational
structures such as single links with other departments, using ideas or proposing solutions based on the given learning
materials. Further, the cooperative groups where the experimental condition encouraged cooperative reciprocity performed
better than the cooperative groups operating without this condition. This was particularly evident in our analysis of their
ability to link ideas. Of course, some existing learning activities (e.g., group projects as adopted in many universities) are able
to rmly build upon this cognitive skill. However, it has not been extensively studied in contextual, mixed-reality mobile
learning, where the learning design is intended to increase engagement and motivation in the process of cooperative
learning.
In conclusion, it can be seen that cooperative knowledge construction requires that learners recognise the value of the
knowledge to be shared, both to share their own knowledge with others, and to gain knowledge in return. The key to success
in interactive cooperative learning is thus that self-interest should quickly align with collective interest. Therefore learning
design, such as the mobile and blended approach described in this article, must consider how this alignment can be created
(for instance, our learning design - a mobile learning game blended with subsequent classroom activities - sufciently
motivated the learners to align with a collective interest). Knowledge asymmetry that allows more communication and
cooperation might be a way forward in designing such cooperative mobile learning games.
In this sense, we need to acknowledge that many ICT-based educational technologies can support knowledge sharing, but
lack mechanisms to effectively establish cooperation (note that many electronic meeting systems and online-learning
platforms allow people to connect remotely, but lack explicit means to externalise or internalise their knowledge). Re-
ports, therefore, show that all too often the introduction of these systems does not result in signicant improvements in
critical thinking skills (Frohberg, Go th, & Schwabe, 2009). A question for future research is how other computer assisted
learning activities might advance this cooperative reciprocity condition for critical thinking skills development.
There are many limitations of the work reported this article. The causality between the given learning activity and
cooperative reciprocity is not examined, which might be mixed with other exogenous factors (e.g., learning environment and
student's previous knowledge). To explore this, a rather different behavioural data set would need to be collected and more
sophisticated statistical analyses (e.g., structural equation method) could be employed. Since we acknowledge that these
Table 7
A part of the conversation analysis of how learners develop their resolution in Group III (Pair #2).
Time Person Critical thinking Utterances Cooperative

indicator reciprocity

17:25 B Novelty () There was a graph that shows the sales and the unit cost. X
17:29 B Importance () High sales and high unit production costs
17:36 A Novelty () The marketing and R&D blamed each other, and then no further new product development would
follow.
17:50 B Linking ideas () Advertisements on the existing products should be made.

18:32 B Linking ideas () The Marketing keeps developing new products O
18:35 A Linking ideas () New products should be based on the target users, and the R&D department thinks that new
products .
18:45 B Importance () No No No No more new products That is what the R&D department is saying
18:48 A Linking ideas () Existing product portfolios Not new development They have to introduce new commercials
from the existing portfolios.
Table 8
The number of knowledge externalisation items (topics) presented in the classroom.
Phase I (Sharing phase) Frequency Phase II (Solution phase) Frequency Total
Sharing ideas Discussing solutions Sharing ideas Discussing solutions

Group II 67 33 5 32 137
Sum 100 37
Group III 68 3 10 51 132
Sum 71 61
Italics represents that Group II had more discussed to develop their solutions in Phase I.
factors cannot be easily controlled in any learning studies, we instead took a reductionist approach in suggesting a naturalistic
business consulting exercise using the blended and mobile learning approach. Further, this approach has not been system-
atically tested for other educational contexts, so our conclusions may be inuenced by the cultural or academic context of our
experiments thus far.
Nonetheless, this study noted that critical thinking skills arising from cooperation adjust our behaviour in such a way as to
allow for others' interests as well as ours. Cooperative learning skills development, where decit information was inten-
tionally given to encourage collaborative thought, is an effective way to foster critical thinking skills. This can be seen as the
utility of ICT-based educational technologies (e.g., mobile learning, ipped learning), which cannot be comparable with fact-
based pedagogy.
In summary, while fact-based solo learning alone may have some learning benets, cooperative learning offers greater
support to critical thinking skills. Here, it is noted that cooperative relationships may break down when one party does too
much taking and not enough giving. In learning too, a group of learners can only avert the Tragedy of the Commons when they
form a willingness to put Us ahead of Me. This article argues that a richer version of oneself can only be formed by
cooperation under a certain condition; cooperative reciprocity. The task for learning design is to foster such conditions.
Acknowledgement
We thank the anonymous reviewers of the article for their detailed comments and helpful suggestions. This work was
supported by the National Research Foundation of Korea (NRF-2014R1A2A2A01002583).
References
Abrami, P., Bernard, R., Borokhovski, E., Wade, A., Surkes, M., Tamim, R., et al. (2008). Instructional interventions affecting critical thinking skills and
dispositions: a stage 1 meta-analysis. Review of Educational Research, 78(4), 1102e1134.
Annetta, L. A. (2008). Video games in education: why they should be used and how they are being used. Theory into Practice, 47(3), 229e239.
Bechtoldt, M. N., De Dreu, C. K., Nijstad, B. A., & Choi, H. S. (2010). Motivated information processing, social tuning, and group creativity. Journal of Personality
and Social Psychology, 99(4), 622.
Bos, N. D., & Gordon, M. D. (2005). A simulated consulting project with a deregulating utility company. Simulation & Gaming, 36(1), 91e113.
Brosnan, S. F., Freeman, C., & De Waal, F. (2006). Partner's behavior, not reward distribution, determines success in an unequal cooperative task in capuchin
monkeys. American Journal of Primatology, 68(7), 713e724.
Chang, H. J. (2010). 23 things they don't tell you about capitalism. London: Bloomsbury Press.
Chang, C.-W., Lee, J.-H., Wang, C.-Y., & Chen, G.-D. (2010). Improving the authentic learning experience by integrating robots into the mixed-reality
environment. Computers & Education, 55(4), 1572e1578.
Clark, H. H., & Schaefer, E. F. (1989). Contributing to discourse. Cognitive Science, 13, 259e294.
De Crom, E. P., & De Jager, A. (2005). The ME-learning experience: PDA technology and e-learning in ecotourism at the tshwane university of technology (TUT)
Accessed 16.02.20 http://www.mlearn.org.za/CD/papers/De%20Crom.pdf.
Dede, C. (2007). Transforming education for the 21st century: New pedagogies that help all students attain sophisticated learning outcomes Accessed 16.02.20
http://citeseerx.ist.psu.edu/viewdoc/download?doi10.1.1.462.7531&reprep1&typepdf.
Dennett, D. C. (1989). The intentional stance. Cambridge: MIT press.
Dickens, C. (1854). Hard times: A novel. London: Harper & Brothers.
Dillenbourg, P. (1999). What do you mean by collaborative learning? In P. Dillenbourg (Ed.), Collaborative-learning: Cognitive and computational approaches
(pp. 1e19). Oxford: Elsevier.
Donahue, D. M., Bowyer, J., & Rosenberg, D. (2003). Learning with and learning from: reciprocity in service learning in teacher education. Equity & Excellence
in Education, 36(1), 15e27.
Dondlinger, M. J. (2007). Educational video game design: a review of the literature. Journal of Applied Educational Technology, 4(1), 21e31.
Ennis, R. H. (1987). A taxonomy of critical thinking dispositions and abilities.
Facer, K., Joiner, R., Stanton, D., Reid, J., Hull, R., & Kirk, D. (2004). Savannah: mobile gaming and learning? Journal of Computer Assisted Learning, 20(6),
399e409.
Faria, A. J., Hutchinson, D., Wellington, W. J., & Gold, S. (2009). Developments in business gaming: a review of the past 40 years. Simulation & Gaming, 40(4),
464e487.
Frohberg, D., Go th, C., & Schwabe, G. (2009). Mobile learning projectsea critical analysis of the state of the art. Journal of Computer Assisted Learning, 25(4),
307e331.
Garrison, D. R. (1992). Critical thinking and self-directed learning in adult education: an analysis of responsibility and control issues. Adult Education
Quarterly, 42(3), 136e148.
Gibbons, M. (1998). Higher education relevance in the 21st Century Accessed 16.02.20 http://www-wds.worldbank.org/servlet/WDSContentServer/WDSP/IB/
2000/07/19/000094946_9912220532351/Rendered/PDF/multi_page.pdf.
Gokhale, A. A. (1995). Collaborative learning enhances critical thinking. Journal of Technology Education, 7(1).
Greene, J. (2014). Moral tribes: Emotion, reason and the gap between us and them. New York: Penguin Press.
Haidt, J. (2012). The righteous mind: Why good people are divided by politics and religion. New York: Vintage.
Hardin, G. (1968). The tragedy of the commons. Science, 162(3859), 1243e1248.
Heffernan, M. (2014). A bigger prize: How we can do better than the competition. New York: PublicAffairs.
Hendriks, P. (1999). Why share knowledge? the inuence of ICT on the motivation for knowledge sharing. Knowledge and Process Management, 6(2), 91e100.
Henri, F. (1992). Computer conferencing and content analysis. In A. R. Kaye (Ed.), Collaborative learning through computer conferencing (pp. 117e136).
Springer Berlin Heidelberg.
Hutchins, E. (1995). Cognition in the wild. Cambridge: MIT press.
Kittl, C., Edegger, F., & Petrovic, O. (2008). Learning by pervasive gaming: an empirical study. In H. Ryu, & D. P. Parsons (Eds.), Innovative Mobile Learning:
Techniques and technologies (pp. 145e171). Hershey: IGI Global.
Klopfer, E., Squire, K., & Jenkins, H. (2002). Environmental detectives: PDAs as a window into a virtual simulated world. In Proceedings IEEE international
workshop on wireless and Mobile technologies in education, 2002 (pp. 95e98). IEEE.
Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: an overview. Theory into Practice, 41(4), 212e218.
Kukulska-Hulme, A., Sharples, M., Milrad, M., Arnedillo-S anchez, I., & Vavoula, G. (2009). Innovation in mobile learning: a European perspective. Inter-
national Journal of Mobile and Blended Learning, 1(1), 13e35.
Nam, C. W., & Zellner, R. D. (2011). The relative effects of positive interdependence and group processing on student achievement and attitude in online
cooperative learning. Computers & Education, 56(3), 680e688.
Newman, D. R., Johnson, C., Webb, B., & Cochrane, C. (1997). Evaluating the quality of learning in computer supported co-operative learning. Journal of the
American Society for Information Science, 48(6), 484e495.
Palmer, C. T., & Steadman, L. B. (1997). Human kinship as a descendant-leaving strategy: a solution to an evolutionary puzzle. Journal of Social and
Evolutionary Systems, 20(1), 39e51.
Parsons, D., Petrova, K., & Ryu, H. (2012). Designing mobile games for engagement and learning. Journal of Information Technology and Applications, 41e46,
2011 ICITA.
Partnership for 21st Century Skills. (2011). Framework for 21st Century learning Accessed 16.02.20 http://www.p21.org/storage/documents/1.__p21_
framework_2-pager.pdf.
Pena-Shaff, J. B., & Nicholls, C. (2004). Analyzing student interactions and meaning construction in computer bulletin board discussions. Computers &
Education, 42(3), 243e265.
Prensky, M. (2005). Computer games and learning: digital game-based learning. In J. Raessens, & J. Goldstein (Eds.), Handbook of computer game studies (pp.
97e122). Cambridge: MIT Press.
Salomon, G., & Perkins, D. N. (1998). Individual and social aspects of learning. Review of Research in Education, 23, 1e24.
nchez, J., & Olivares, R. (2011). Problem solving and collaboration using mobile serious games. Computers & Education, 57(3), 1943e1952.
Sa
Schmitz, B., Schuffelen, P., Kreijns, K., Klemke, R., & Specht, M. (2015). Putting yourself in someone else's shoes: the impact of a location-based, collaborative
role-playing game on behaviour. Computers & Education, 85, 160e169.
Schrier, K.. (2006, July). Using augmented reality games to teach 21st century skills. In ACM SIGGRAPH 2006 Educators program (p. 15). ACM.
Simon, H. A. (1990). A mechanism for social selection and successful altruism. Science, 250(4988), 1665e1668.
Singer, T., & Lamm, C. (2009). The social neuroscience of empathy. Annals of the New York Academy of Sciences, 1156(1), 81e96.
Smith, J. H. (2000). The road not takenethe How's and Why's of interactive ction Accessed 16.02.20 http://game-research.com/index.php/articles/the-road-
not-taken-the-hows-and-whys-of-interactive-ction/.
Tan, C. T., & Soh, D. (2010). Augmented reality games: a review. In In proceedings of GAMEON-ARABIA, EUROSIS.
Trivers, R. L. (1971). The evolution of reciprocal altruism. Quarterly Review of Biology, 35e57.
Wake, J. D., & Baggetun, R. (2009). premierlytnant Bielke: a Mobile game for teaching and learning history. International Journal of Mobile and Blended
Learning, 1(4), 12e28.
Willingham, D. T. (2008). Critical thinking: why is it so hard to teach? Arts Education Policy Review, 109(4), 21e32.
Yang, Y.-T. (2015). Virtual CEOs: a blended approach to digital gaming for enhancing higher order thinking and academic achievement among vocational
high school students. Computers & Education, 81, 281e295.

Cooperation Begins Encouraging Critical Thinking Skills

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cooperation Begins Encouraging Critical Thinking Skills

Uploaded by

Copyright:

Available Formats

Computers & Education 97 (2016) 97e115

Contents lists available at ScienceDirect

Computers & Education

Cooperation begins: Encouraging critical thinking skills

1.2. Cooperation begins with mobile mixed-reality learning

1.3. Cooperative reciprocity and critical thinking skills

1.3.1. Cooperative knowledge creation and sharing

Fig. 2. The cooperative knowledge construction process.

1.3.2. Motivations for knowledge sharing

1.3.3. Cooperative reciprocity and critical thinking

2. Mobile & blended learning design: Kiwi Mobile

Henri (1992) Garrison (1992) Newman et al. (1997)

2.1. Mobile learning game design

2.2. Experimental method

2.2.1. Administration of the experiment

2.3. Learning procedure

2.4. Learning outcomes, and assessing critical thinking skills

Fig. 6. Examples of CEO reports & presentations.

Category Indicator Descriptions

Category Indicator Descriptions

x The number of positive indicators

x The number of negative indicators

1 uncritical & surface Critical Thinking Ratio 1critical & deep

Conicting Marketing e R&D QA e Assembly R&D e QA Marketing e R&D e Assembly

Conicting Marketing e R&D QA e Assembly R&D e QA Marketing e R&D e Assembly

4. Conclusions and discussion

Critical thinking Utterances (Examples) No. of utterance Critical

Critical thinking Utterances (Examples) No. of utterance Critical

Time Person Critical thinking Utterances Cooperative

Time Person Critical thinking Utterances Cooperative

Phase I (Sharing phase) Frequency Phase II (Solution phase) Frequency Total

Sharing ideas Discussing solutions Sharing ideas Discussing solutions

You might also like