Educ Psychol Rev (2012) 24:251269 DOI 10.

1007/s10648-012-9192-0 REVIEW

Learning from Comparing Multiple Examples: On the Dilemma of Similar or Different

Jian-Peng Guo & Ming Fai Pang & Ling-Yan Yang & Yi Ding

Published online: 22 February 2012 # Springer Science+Business Media, LLC 2012

Abstract Although researchers have demonstrated that studying multiple examples is more effective than studying one example to facilitate learning, the principles found in the literature for designing multiple examples remain ambiguous. This paper reviews variation theory research on example design which sheds light on unclear issues regarding the effects of example variability. First, the distinction of surface/structural should be replaced by critical/uncritical in example study. Aspects and features that are critical to students understanding should be identified and compared in example design. Second, variation as well as similarity among examples should be taken into consideration in example design. Certain patterns of variation and invariance should be adopted to systematically determine the variability of examples. Third, students with different levels of prior knowledge perceive different aspects of examples that are critical for their learning. Examples should be designed according to aspects that are critical to specific students. Keywords Multiple examples . Variability . Comparison . Critical aspects . Variation theory

J.-P. Guo (*) Institute of Education, Xiamen University, Xiamen, China e-mail: guojp@xmu.edu.cn

M. F. Pang Faculty of Education, The University of Hong Kong, Hong Kong S.A.R., China e-mail: pangmf@hkucc.hku.hk L.-Y. Yang College of Education, University of Iowa, Iowa, USA e-mail: ling-yan-yang@uiowa.edu Y. Ding Graduate School of Education, Fordham University, New York, USA e-mail: yding4@fordham.edu

252

Educ Psychol Rev (2012) 24:251269

Examples can be broadly defined as illustrations of concepts and principles (Sowder 1980) and can include anything used as raw material for generalizing (Watson and Mason 2002). For instance, we can construct different triangles as examples of the concept of triangle. Even for principles, there are examples that help us elaborate. For instance, the problem 3x + 6 0 12 together with its solution is called a worked example of a linear equation. The pedagogical values of examples and their effects in facilitating student learning have long been recognized by educators. Learners primarily use examples as information sources and apply previously stored examples to classify a newly encountered instance or to solve a problem in a new domain (Gentner et al. 2003; Kolodner 1997; Renkl et al. 1998; Ross 1984, 1987, 1989b; Ross and Kennedy 1990; Ross and Kilbane 1997). Learning from examples has attracted considerable attention from educational psychologists and has been a major topic in educational research for at least the past four decades (Atkinson et al. 2000). An abundance of cognitive science literature has examined the importance of examples and provided empirical support for principles of example design to promote learning (e.g., Gentner et al. 2003; Gentner and Namy 1999; Gick and Holyoak 1983; Holyoak 1985; Merrill and Tennyson 1978; Namy and Gentner 2002; Oakes and Ribar 2005; Reed and Bolstad 1991; Schwartz and Bransford 1998; Sweller and Cooper 1985; Sweller et al. 1998). One issue that has been intensively investigated is the quantity of examples, i.e., how many examples are needed for learning? Multiple examples are recommended over one example for learning because the comparison evoked by multiple examples facilitates schema construction and is thus beneficial for learning (Catrambone and Holyoak 1989; Cooper and Sweller 1987; Gibson and Gibson 1955; Oakes 2001; Reed 1993; Rittle-Johnson and Star 2007, 2009; Schwartz and Bransford 1998; Schwartz and Martin 2004; Silver et al. 2005; Wolff and Gentner 2000). As Gentner (2005) noted, Comparison is a general learning process that can promote deep relational learning and the development of theory-level explanations (p. 251). Not all comparison, however, may be equally effective (Gick and Paterson 1992; Quilici and Mayer 1996; Rittle-Johnson et al. 2009). The effectiveness of multiple examples depends on the variability of the examples being compared (Gentner and Namy 1999; Quilici and Mayer 1996; Renkl et al. 1998; Rittle-Johnson and Star 2007, 2009; Rittle-Johnson et al. 2009), the prior knowledge of students who compare the examples (Albro et al. 2007; Clarke et al. 2005; Gentner et al. 2007; Schwartz and Bransford 1998), and other factors such as instructional supports within the examples (Catrambone 1994; Kurtz et al. 2001; Richland and McDonough 2010; Schwartz and Bransford 1998; Wittwer and Renkl 2010). Learning from examples is complex, and so it is not surprising that the factors contributing to example-based learning success or failure are multiple. Given the fact that there is substantial research on example-based learning and several reviews have already been published (see the reviews by Atkinson et al. 2000; Tennyson and Park 1980; Wittwer and Renkl 2010), our purpose in this manuscript is to (a) review the evidence regarding one factor in example-based learning known as example variability, (b) summarize inconclusive findings and their explanations regarding example variability, and (c) derive a set of instructional principles for designing multiple examples to facilitate successful learning. In this review, we first summarize ambiguous findings in the cognitive science literature regarding the effects of example variability on comparison and learning. We then review variation theory studies on example-based learning that examine the effect of example variability from a variation theory perspective. Next, we compare the variation theory approach to other approaches with respect to key issues in example-based learning. Finally, we conclude with a set of instructional principles for designing multiple examples and present our recommendations for future research.

Educ Psychol Rev (2012) 24:251269

253

Variability of Multiple Examples: Similar or Different? The impact of comparison on learning depends on the variability of multiple examples, i.e., the extent of similarity or difference among examples (Gentner and Namy 1999; Quilici and Mayer 1996; Renkl et al. 1998; Rittle-Johnson and Star 2009; Rittle-Johnson et al. 2009). Two aspects of examples are typically analyzed in the literature: surface (irrelevant) features and structural (relevant) features (Gick and Holyoak 1980, 1983; Reed 1989; Ross 1989a, 1997; Ross and Kennedy 1990; VanderStoep and Seifert 1993). Example variability consists of differences in the surface and structural aspects among the examples. Holyoak and Koh (1987) used the relevance to goal attainment to distinguish the types of features: surface features, such as names, objects, numbers, and story lines, are irrelevant to goal attainment; structural features, such as underlying mathematical procedures, rules, solutions, and principles, are relevant to goal attainment (Quilici and Mayer 1996; Ross and Kilbane 1997). For instance, an apple is an example of the concept of fruit. The shape, size, and color are surface features of the apple, and the attribution that the apple is edible is a structural feature. An unresolved question is to what extent the variability of multiple examples affects learning and transfer, i.e., how similar multiple examples should be in terms of surface and structural features (Renkl et al. 1998; Rittle-Johnson and Star 2009). A considerable number of studies on concept learning and procedural learning have shown that positive examples with different surface features should be given to help learners focus on structural features rather than relying on surface features, which might promote construction of structure-based schema and develop a more accurate understanding of the concept (Hammer et al. 2008; Merrill and Tennyson 1978; Ranzijn 1991; Tennyson 1973) or the procedure (Paas and Van Merrienboer 1994; Quilici and Mayer 1996; Rittle-Johnson and Star 2009). Given superficially similar examples, the learner might consider surface features as relevant, which might interfere with schema induction and future problem solving (Quilici and Mayer 1996; Reed 1989). In contrast, Gentner and her colleagues (Gentner and Namy 1999; Namy and Clepper 2010; Namy and Gentner 2002) did not suggest that positive examples of a concept should be as different as possible in terms of surface features to facilitate learning. Instead, they argued that superficial similarity among examples facilitates structure mapping and helps the learner to notice deeper structural commonalities. Researchers in the field of procedural learning have also suggested that superficially similar worked examples should be used during instruction because they do not overtax the students and can help them to pay attention to the structural aspects (Renkl et al. 1998; Ross 1989b). When given examples that have very dissimilar surface features, students often find it more difficult to discover the underlying common structure (Gick and Holyoak 1980, 1983; Richland et al. 2004; Ross 1989b; Ross and Kennedy 1990). Researchers have investigated the role of student prior knowledge in learning from comparing examples to account for the ambiguous effect of example variability. Findings, however, are still contradictory (Rittle-Johnson et al. 2009). Some researchers argued that students with low prior knowledge can hardly benefit from comparing multiple examples, especially the complex and unfamiliar examples (e.g., Holmqvist et al. 2007; Schwartz and Bransford 1998); some researchers found that more able students can benefit from comparing any kind of examples, while less able students can only benefit from comparing very different examples (e.g., Quilici and Mayer 1996); some researchers found that students with higher prior knowledge benefited more from comparing high-variability examples while students with lower prior knowledge benefited more from comparing low-variability examples (e.g., Groe and Renkl 2006, 2007; Rittle-Johnson et al. 2009); and some researchers did not find any interaction effect between students prior knowledge and the variability of

254

Educ Psychol Rev (2012) 24:251269

examples being compared (e.g., Renkl et al. 1998). In short, it remains unclear what type of example variability should be provided to students with different levels of prior knowledge. The unclear effects of example variability might be due to the fact that researchers describe examples as high- or low-variable in terms of surface and structural features and examine their relative effectiveness for the learning of students with different levels of prior knowledge. What matters might be the features that the learners need to know, instead of the variability associated with surface and structural features. In the following review, we summarize the inconclusive findings and derive a set of instructional principles for designing multiple examples on the basis of our research paradigm, variation theory.

Variation Theory Studies on Learning from Comparing Examples Regarding the unclear effects of example variability revealed by cognitive research, variation theory studies on learning from comparing examples have examined these issues from a different perspective and provided implications for example-based learning. Variation theory holds variation to be epistemologically fundamental for all learning to happen (see Marton and Booth 1997; Marton and Tsui 2004 for details). Different from cognitive theories that consider learning as formation of mental representations, variation theory interprets learning from a relational and perceptual perspective. As Marton and Booth (1997) argued, An experience is of its essence nondualistic (p. 122); that is, the human and the world are internally related. When seeing something, we create individual-world relations through our experiences. For example, if a person experiences an object as a bird, then the meaning of the bird is not in the object, neither is it in the subjects head. Instead, as Svensson (1984) explained, it is constituted as the relation between the object to which awareness is directed and the person as the subject. This view of learning is similar to situated cognition theory claiming that learning is a subject-world relation (Lave 1996, p. 156), as opposed to the idea of understanding the learner s mental representations of the outside world. But variation theory does not consider the social setting and cultural environment outside the individual as the fabric of knowledge. Instead, variation theory draws on the notion of intentionality to characterize a nondualist model of experience (Linder and Marshall 2003, p. 272). According to variation theory, learning takes place by a change in something in the world as experienced by a person, i.e., a new way of seeing something (Marton and Booth 1997). The new way of seeing something is constituted in the relationship between the individual and the phenomenon, and amounts to discerning certain critical features of that phenomenon and focusing on them simultaneously (Marton 1999). In particular, the notions of critical aspects/features and patterns of variation and invariance from variation theory provide direct guidelines to address the unsolved issues mentioned above. Critical aspects/features of the object of learning Variation theory is content-oriented and focuses on what is experienced and how it is experienced (Marton and Booth 1997, p. 114). The something to be experienced, i.e., the object of learning, refers to a particular insight, skill, or capability to be developed in students after instruction (Marton and Pang 2006). For a particular object of learning to be learned, critical aspects of the object of learning should be first identified. According to variation theory, aspects and features of a phenomenon and its examples are analyzed as critical or uncritical to students understanding and learning, rather than surface or structural

Educ Psychol Rev (2012) 24:251269

255

to the objective disciplinary knowledge. Critical aspects are aspects that cause difficulty for students in the process of learning; they might be superficial or structural. For example, if a child believes that fruit can only be round, then the surface aspect (shape) and feature (round) become critical for the child to learn the concept of fruit. To help the child realize that fruit could be in different shapes, we should show him/her different examples (e.g., an apple, a banana, and a carambola) that present different shapes. Here, the surface aspect of shape is the critical aspect for learning the concept of fruit. As Marton and Pang (2008) argued, both the disciplinary knowledge and the students understanding should be taken into account when identifying the critical aspects of an object of learning. Patterns of variation and invariance According to variation theory, to learn a phenomenon means to simultaneously discern and focus on the critical aspects/features of the phenomenon. To discern a particular feature, the learner must experience variation in that dimension. When an aspect of a phenomenon varies while other aspects remain invariant, the aspect that varies would be discerned (Pang and Marton 2005). In particular, Marton and his colleagues (e.g., Marton and Pang 2006; Marton and Tsui 2004) defined four patterns of variation and invariance (contrast, separation, fusion, and generalization) to facilitate the discernment of critical aspects. Contrast occurs when a learner experiences variation of different values or features in an aspect of a phenomenon. To experience something, the learner must experience something else to make a comparison. For example, to experience what round is, the learner must experience other shapes such as rectangle or triangle. Only after having experienced other values (rectangle, triangle etc.) of the aspect (shape) can the learner discern the specific shape of round. The pattern of contrast focuses on a particular value or feature of an aspect. Separation happens when a learner focuses on an aspect of a phenomenon. To experience a certain aspect of something separately from other aspects, it must vary while other aspects remain invariant. In this pattern, the aspect is discerned by the learner. For example, to discern the aspect of the shape of an object, other aspects (e.g., size, color, and height) must be kept invariant while varying the aspect of shape. In this way, the aspect of shape can be separated from other aspects. Contrast and separation occur when two or more objects have a varying aspect while other aspects remain invariant. Fusion takes place when a learner wants to discern several aspects of a phenomenon that vary simultaneously. To experience a phenomenon, the learner must discern all critical aspects at the same time when different critical aspects vary simultaneously. For example, if a teacher wants to teach students what a robin is, he/she should expose the students to simultaneous variation in all critical aspects of a robin (e.g., feathers, size, and sound). The students will grasp a concept if they can simultaneously discern all critical aspects of the concept. Generalization occurs when a learner wants to apply his/her previous discernment to various contexts. To fully understand an object of learning, the learner must experience many other examples to generalize the meaning. The idea of roundness can only be achieved after the learner has experienced various round objects, such as a round ball, a round plate, a round table, and so on. With respect to the sequence of using the four patterns of variation and invariance, researchers suggested that contrast and separation should be first used to help students discern each critical aspect separately, followed by fusion that simultaneously varies all critical aspects (Ki 2007; Marton and Tsui 2004; Pang 2002). Generalization could be used after students have simultaneously discerned all critical aspects to generalize the discernment to other contexts.

256

Educ Psychol Rev (2012) 24:251269

Guo and Pang (2011) confirmed the effectiveness of the principles of variation theory in designing multiple examples for learning the concept of the altitude of a triangle. In this study, Guo and Pang identified six critical aspects for students to learn the altitude of a triangle: vertex, perpendicularity, opposition, orientation, location, and altitude-base-correspondence. The four patterns of variation and invariance were used to design multiple examples in the experimental group. First, contrast and separation were used to help students separately discern each critical aspect. For example, Fig. 1 shows how students could separately discern the critical aspect of vertex by comparing one pair of positive and negative examples. Within this pair, the only difference between the positive and negative examples is in the critical aspect of vertex: segment AD in the positive example passes through a vertex, whereas the corresponding segment ED in the negative example does not pass through a vertex. By keeping all of the other critical aspects the same, the varied aspect of vertex and the value of passing through a vertex should be discerned by students. These techniques are known in variation theory as separation and contrast. In other words, after comparing the positive and negative examples in the first pair, students should understand that the altitude of a triangle must pass through a vertex. After having separately discerned each critical aspect by comparing multiple examples, students were shown examples that simultaneously varied all critical aspects, which made use of the pattern of fusion. As shown in Fig. 2, altitudes AD and BE pass through different vertexes (A and B), are perpendicular to different opposite sides (BC and AC), and are at different orientations. Experiencing simultaneous variation of all critical aspects is considered a necessary stage for developing a complete understanding of the object of learning. Subsequently, different types of triangles (acute, right-angled, and obtuse triangles) could be used to generalize students discernment to other examples, thus applying application of the pattern of generalization. Empirical findings on variation theory about example-based learning Researchers have carried out a number of experimental studies to apply principles of variation theory to the teaching of different academic subjects (e.g., Cheung 2005; Fraser et al. 2006; Kwong 2005; Lam 2005; Linder et al. 2006; Lo 2002; Lo et al. 2004, 2005;

Fig. 1 One pair of positive and negative examples designed by the use of contrast and separation

Educ Psychol Rev (2012) 24:251269

257

Fig. 2 Two positive examples designed by the use of fusion

Ng and Lai 2002; Pang 2002; Pang et al. 2006; Pang and Marton 2003, 2005; Pong 2000). Results from these studies have consistently shown that (a) multiple examples with the same surface story should be used and (b) the examples used should help students discern critical aspects first separately and then simultaneously. Pang (2002) conducted a study to investigate the effect of using variation theory in instructional design on secondary school students learning of an economic principlethe incidence of sales tax. The critical aspects identified for learning this principle are to recognize the relations between the elasticity of demand and supply and the change in sales tax burden: (a) the consumers have to bear more (less) of the sales tax burden if the elasticity of demand reduces (increases) and the elasticity of supply remains invariant, (b) the consumers have to bear more (less) of the tax burden if the elasticity of supply increases (reduces) and the elasticity of demand remains invariant, and (c) the change in tax burden is determined by the relative elasticity of demand and supply given the elasticity of demand and supply changes at the same time. Teachers in the experimental group first varied the elasticity of demand, then the elasticity of supply, and finally the two aspects simultaneously in a same context of red wine sales; teachers in the control group separately varied the elasticity of demand and supply in different contexts but did not vary the two aspects simultaneously. Pang (2002) found that the students in the experimental group better mastered the principle of sales tax incidence after the instruction than did the students in the control group. The results showed that simultaneous variation of critical aspects was conducive to learning. In addition, the study suggested using a constant context (surface story) in example-based teaching. If examples from different contexts are used, students may consider the surface stories as separate and independent. A relatively more effective method is to vary

258

Educ Psychol Rev (2012) 24:251269

critical aspects within a particular context (Pong 2000). Keeping the context constant helps students focus on the varied critical aspects that need to be discerned. Lo et al. (2006) conducted a study to help third-grade students acquire a scientific understanding of rainbows. Two critical aspects were identified in the study: (a) the prism is only a tool to split up sunlight and does not create a rainbow and (b) there is a part-whole relationship between sunlight and the colors of a rainbow: sunlight can be split up into its component color lights and these color lights can be recombined to form sunlight. To help students discern the first critical aspect, the teacher used different tools (e.g., prism, soap bubbles, or water droplets) to split up sunlight. To help students discern the second critical aspect, the teacher introduced two analogous examples that had a similar part-whole relationship with the different surface stories. Students performance on the posttest indicated that showing students different tools helped them discern the first critical aspect. By keeping other aspects invariant (i.e., the coexistence of sunlight and the rainbow), the teacher helped students discern the varied critical aspects of the tools. In other words, students could discern that the prism was only one of those tools and that the rainbow came from sunlight. However, results from the posttest showed that students did not fully understand the second critical aspect of the part-whole relationship of sunlight and the rainbow, by simply comparing two analogies. Lo et al. (2006) thus suggested that future research should be conducted in which students could experience simultaneous variation in the light source and the spectrum formed (whole to part), and simultaneous variation in the spectrum and the color of the spot of light formed by focusing on the spectrum (part to whole). As the authors argued, this would help students discern the part-whole relationship between sunlight and the rainbow within the same context. In mathematics, Choy (2006) conducted an experimental study on the calculation of slope to examine the effectiveness of separate variation of critical aspects. The slope of a straight line is determined by two aspects: the horizontal distance and the vertical distance of a line segment. The teacher in the experimental group used a series of examples of a staircase to help secondary school students discern the two critical aspects first separately and then simultaneously. In contrast, the teacher in the control group began the lesson by using examples that simultaneously varied the two critical aspects that had not been separately discerned by students. Then the teacher used examples to separately vary the two critical aspects. Results from the posttest showed that students in the experimental group significantly outperformed those in the control group in answering questions that required separate discernment of the two critical aspects. A possible explanation is that students in the experimental group had the opportunity to separately discern each critical aspect before simultaneously experiencing them, whereas students in the control group experienced the simultaneous variation on the two critical aspects at the beginning of learning without having separately discerned them. The findings of Choys (2006) study are complementary to Pangs (2002) study: Pang confirmed the benefits of simultaneous variation of critical aspects whereas Choy showed the effectiveness of separate variation in learning. In contrast to these teacher-led whole-class verbal instructional studies described above, Ki (2007) conducted a series of experiments to investigate the effectiveness of variation theory in training college non-tonal language speakers to learn Cantonese via self-learning computer training programs. Results from this study confirmed Choys (2006) finding that separately experiencing variation of critical aspects is beneficial for later experiencing simultaneous variation of these aspects. Ki thus suggested the learning process of separation then fusion to help students first separately discern each critical aspect and then simultaneously attend to these aspects for better learning. He concluded that the

Educ Psychol Rev (2012) 24:251269

259

separation-then-fusion approach resulted in a more analytic/differentiated conception of the whole and thus was more effective than the fusion-only approach, which tended to result in more global/undifferentiated conception of the whole. Guo and Pang (2011) conducted a study to investigate the role of example variability and students prior knowledge in learning a geometry conceptthe altitude of a triangle. Results confirmed the effectiveness of separate variation in learning. In addition, Guo and Pang found an interaction effect between student prior knowledge and patterns of variation and invariance. Based on the findings, the authors concluded that (a) critical aspects and features of the object of learning should be the focus when designing examples, (b) aspects that are critical for learning should be varied first separately and then simultaneously, and (c) students with different levels of prior knowledge may perceive different aspects that are critical for their learning and thus benefit differently from identical instruction.

Comparison of Variation Theory to Cognitive Approaches on Principles of Designing Multiple Examples Based on the cognitive and variation theory literature on example-based learning reviewed above, this section compares variation theory and cognitive approaches on three unclear issues in this area: What aspect of examples should be focused on?, How similar or different should examples be?, and When is comparing examples effective? It is concluded that aspects and features critical to student understanding should be identified and compared in example design, certain patterns of variation and invariance should be used to systematically determine the variability of examples, and examples should be designed according to aspects that are critical to specific students who have different levels of prior knowledge. What aspects or dimensions of examples should be focused on: critical aspects of the object of learning Multiple examples are considered beneficial for learning because comparing multiple examples can promote schema induction and formation (Gick and Holyoak 1983; Reed and Bolstad 1991; Sweller and Cooper 1985). A learner first compares multiple examples of a concept or a rule and forms a mental structure (schema) in his/her mind which represents the concept or rule in the outside world, by abstracting the similarity of the underlying structure of the examples. The aim in this phase is to help the learner discover the underlying structure of the examples and form a structure-based schema that contains only the structural features of the concept or rule and discards irrelevant surface features. As a result, the schema would be stored in the learners mind for later use. Afterwards, when the learner encounters a new similar problem, he/she would retrieve the previously stored schema by mapping the schema and the target problem through the similarity between the schema and the problem and generate a solution to the target. Based on this view of learning from comparing examples, aspects of an example are analyzed as surface or structural according to the knowledge to be learned: structural aspects are underlying principles or rules that define the knowledge (e.g., the aspect of edibility in the concept of fruit), whereas surface aspects are aspects that are irrelevant and unimportant to the knowledge (e.g., the aspect of shape in the concept of fruit). Most cognitive research previously reviewed has adopted the structural/surface or relevant/irrelevant distinction and has focused on comparing examples that have the same structural relevant features but different surface irrelevant features (i.e., isomorphic examples). For instance, in Paas and Van Merrienboer s (1994) study, the xy coordinate system, basic CNC-programming code,

260

Educ Psychol Rev (2012) 24:251269

Pythagorean theory, right-angled triangles, and the trigonometric function were the structural aspects that students were supposed to master; values and problem formats (i.e., problem goal and localization in the cross of axes) were the surface aspects. In Quilici and Mayer s (1996) study, statistical principles (i.e., t test of means, chi-square test of proportions, and test of correlation) were the structural aspects while the surface stories and values were the surface aspects. The structural/surface distinction, however, is found to be problematic, as some surface aspects might be important for learning while some structural aspects might have been grasped by students and thus would be unimportant for learning. According to variation theory, experiencing multiple examples can help the learner discern aspects that are critical for learning a phenomenon. As Emanuelsson et al. (2002) argued, at least two examples are necessary for the learner to distinguish the example from the general principle embedded. By experiencing examples of a phenomenon, the learner directs his/her awareness to the phenomenon; some critical features of the phenomenon come to the forefront of the attention and are discerned whereas others recede into the background and are taken for granted. The discerned features and those that are taken for granted constitute a figureground relationship, which defines the structure of the phenomenon in the learner s awareness and provides the learner with a meaning of the phenomenon. Multiple examples can thus help to form the figure-ground relationship and develop a new way of experiencing the phenomenon. This view suggests that aspects of an example are not analyzed as superficial or structural to the objective knowledge, but as critical or uncritical to students understanding and learning. Critical aspects are aspects that cause difficulty for students in the process of learning; the aspects might be superficial or structural. For students with different levels of prior knowledge, some surface features might become relevant and critical to learning, or some structural features might become irrelevant and uncritical to learning, i.e., students learning should be the criterion for determining the important and relevant features (Guo and Pang 2011). In conclusion, critical aspects rather than structural aspects should be the focus during example design. Students should have the opportunity to compare and identify critical aspects of examples. For instance, Rittle-Johnson et al. (2009) recommended that for novices in a domain to learn, examples should be presented in a highly similar way in which only one surface feature varies, because too many different surface features might cause difficulty for structure alignment. In view of this, inexperienced students could learn the same solution to 3(x +4) 0 12 and 3(y +4) 0 12, and then 8(h +2) 0 30, and then 7(a +2) 0 (a +2)+13, and finally compare two different solutions for the same problem. This is due to the fact that some novices might have difficulty with identifying the different variable letters in the equation, or letters on two sides of the equation. The surface features of letters or sides are thus relevant to learning and should be taken into account when determining the variability of multiple examples. In Guo and Pangs (2011) study described above the authors found that surface aspects of orientation and location were very critical for students with low prior knowledge to learn the concept of the altitude of a triangle, whereas structural aspects of vertex and perpendicularity were not critical for students with high prior knowledge who had already discerned these aspects. It should be noted that traditional variation theory research with whole-class instruction focuses on identifying students collective critical aspects for learning and facilitating the discernment of these critical aspects as we previously described. This is because whole-class instruction is the most common teaching practice in school education. Nevertheless, as indicated by Guo and Pang (2011), different students perceive different aspects that are

Educ Psychol Rev (2012) 24:251269

261

critical for their learning. Identifying aspects that are critical for individual students learning and designing individualized learner-tailored learning programs accordingly should be one potential area to apply variation theory research. How similar or different should examples be: patterns of variation and invariance Previous studies adopted the surface/structural distinction and argued whether to present students with high- or low-variability examples (e.g., high- and low-variability examples in Paas and Van Merrienboer 1994; surface- and structure-emphasizing example sets in Quilici and Mayer 1996; multiple and uniform examples in Renkl et al. 1998; equivalent, problem types, and solution methods in Rittle-Johnson and Star 2009). As previously reviewed, it remains unclear how similar or different examples should be, according to existing studies. Sweller (2010) recently used element interactivity to account for the effect of example variability. As he claimed, high-variability examples have substantially higher element activity and cause more intrinsic cognitive load than low-variability examples. Thus, in a learning situation with low extraneous load, high-variability examples would increase the available working memory resources (i.e., germane cognitive load) to deal with the element interactivity, which is associated with intrinsic cognitive load as well as with learning. However, the effectiveness of highly variable examples reverses when the learners working memory is overtaxed (Sweller et al. 1998). Under this view, the high-variability examples in Paas and Van Merrienboer (1994), the structure-emphasizing example sets in Quilici and Mayer (1996), and the solution methods in Rittle-Johnson and Star (2009) were examples that increased the learners germane cognitive load without overtaxing their working memory. In contrast, the multiple examples in Renkl et al. (1998) might overburden the learners working memory and thus were ineffective. Gentner and her colleagues (Gentner and Namy 1999; Namy and Gentner 2002) proposed the structure-mapping theory of comparison and argued that common surface features among examples can initially prompt the structure mapping and alignment process and may invite the child to notice further, more abstract, commonalities (Gentner and Namy 1999, p. 507). According to this theory, comparison is the process of structural alignment. When multiple examples are compared, common structure of the examples that may not have been explicitly assessed is aligned and highlighted, which is crucial for deeper learning. Examples with similar surface features could make the common structural features more apparent and lead the child to comparisons that yield deeper structural commonalities. In addition to comparison that highlights commonalities among perceptually similar examples of the same kind, Namy and Clepper (2010) also discussed the role of contrast that highlights structural differences among positive examples and out-of-category counterexamples. As they argued, contrast is one kind of alignment process that yields different information for children to identify structural features, and can thus facilitate conceptual organization. However, Namy and Clepper (2010) found that contrast could only be effective when comparison preceded, rather than followed. A number of studies have shown that positive examples of a concept presented to students should be different in surface features to help learners focus on structural features, prevent misconception errors, and develop a more accurate understanding of the concept (Hammer et al. 2008; Merrill and Tennyson 1978; Ranzijin 1991; Tennyson 1973). In addition, researchers argued that positive and negative examples having the same surface features and different structural features help learners recognize structural features of concepts or principles than positive examples only (Curry 2004; Gick and Paterson 1992; Siegler 2002; Merrill and Tennyson 1978; van Dooren et al. 2004).

262

Educ Psychol Rev (2012) 24:251269

It is, however, problematic to analyze example variability as high or low in terms of surface and structural aspects as revealed by the inconsistent findings reviewed. This is due to the difficulty to define what high- and low-variability examples actually are; highvariability examples can become low-variability when being compared by different learners. Furthermore, when designing multiple examples for classroom instruction, it is very difficult for teachers to decide to what extent the example should vary and whether they should vary the surface or structural aspects. As Sweller (2010) pointed out, determining the appropriate level of example variability is a challenge that all teachers encounter during their routine instruction. In view of this, we adopted the critical/uncritical distinction and suggested that four patterns of variation and invariance (contrast, separation, fusion, and generalization) could provide a systematic way to determine the variability of multiple examples, as elaborated above. According to variation theory, learning means the development of a new way of seeing a phenomenon and is characterized by critical aspects of that phenomenon discerned and focused on simultaneously (Marton 1999). The simultaneous discernment of critical aspects determines a particular way of understanding a phenomenon, but the critical aspects should first be separately discerned (Marton & Tsui 2004). For a certain aspect to be separately discerned, this aspect should vary while other aspects remain invariant (Bowden & Marton 1998). If the learner does not discern each critical aspect, he/she would not be able to subsequently discern and focus on all critical aspects at the same time, or understand the phenomenon. This is in line with Schwartz and Bransfords (1998) findings that comparing multiple examples varying one or a few important dimensions can help students develop differentiated knowledge and notice relevant distinctive features of examples. Although we recommend the effectiveness of patterns of variation and invariance in promoting learning, we are not suggesting that these patterns can always benefit all students. It often happens that some students are not able to discern what they are expected to (Marton and Morris 2002). The patterns of variation and invariance are conditions created for discernment, and there are other conditions that might influence learning as well (Marton 2006). What we suggest is that pedagogical methods that do not take into consideration the variation of examples during example design are likely to fall short of potential gains. In contrast to leaving the variation of examples to chance, we have emphasized the necessary conditions in terms of what is varied and what is kept invariant, which creates the opportunity for learning to take place. Besides the methods of determining the variability of examples based on variation theory, there are other ways that might also promote discernment and learning and that deserve to be further studied. In this review, we also differentiate the relative role of variation and similarity in learning from examples. As previously noted, similarity is important to the process of learning from comparison (Gentner and Medina 1998; Gick and Holyoak 1983; Kotovsky and Gentner 1996; Paik and Mix 2006); variation is fundamental as well (see Marton 2006 for detailed discussion). According to variation theory, variation is a fundamental learning mechanism (Bowden and Marton 1998; Marton and Booth 1997; Marton and Tsui 2004). Learning and transfer are a function of perceived differences within and between learning situations (Marton 2006). To discern a phenomenon and its aspects, the learner must experience variation. When comparing multiple examples, learners cannot discern the similarities among examples if they have not experienced variation in aspects of the examples. Schwartz and colleagues (Bransford and Schwartz 1999; Schwartz and Bransford 1998; Schwartz and Martin 2004) also demonstrated the effectiveness of contrasting multiple examples from the discernment of differentiated knowledge. In Schwartz and Bransfords (1998) study, students who had the opportunity to compare cases of psychological concepts

Educ Psychol Rev (2012) 24:251269

263

before reading a text or hearing a lecture performed well on predicting outcomes for a hypothetical-related experiment. Schwartz and Martin (2004) compared the effects of taking part in an innovative learning condition using contrasting cases of a given topic and subsequently being exposed to formal teaching on the same topic on the one hand, with being exposed to formal teaching on a topic followed by formal teaching of a different kind on the same topic on the other hand. They found that the first combination was more powerful than the second from the point of view of learning. There were two types of contrasts in the first combination of Schwartz and Martins (2004) study: differences between the cases provided the students with one kind of contrast and differences between conditions of contrasting cases and being told provided them with another contrast (Marton 2006). Contrasting cases, as Schwartz and Bransford (1998) stated, can help students notice specific features and dimensions that distinguish the examples and generate the differentiated knowledge structures, which provides the basis for future learning. For instance, a student who perceives the difference between the positive and negative examples in Fig. 3 would discern that the aspect of vertex is critical for the concept of the altitude of a triangle. After having discerned this critical aspect, the student may discover a similarity between the two positive examples in Fig. 4 in that the two altitudes both pass through a vertex. If the student has not experienced the variation of the aspect of vertex and therefore not discerned it, he/she would probably have difficulty in finding the similarity between the two examples in Fig. 4. In this case, learning would not occur. If variation is not taken into consideration for designing examples, students might have difficulty finding similarity as indicated by the literature reviewed above: if examples were given with too much variation, students might be unable to find the similarity; if examples were given with too little variation, students might detect irrelevant similarity. As Ki (2007) argued, it is the variations that necessitate the awareness to the dimensions, and it is also the variations that teach or convince the person to attend to such dimensions of variation (p. 1118). When is comparing examples effective: the role of prior knowledge in learning from comparing examples Although the effectiveness of examples variability has been found to be altered by students prior knowledge, it is unclear what level of example variability should be provided to students with what level of prior knowledge. First, it is ambiguous to define examples with high or low variability and to differentiate students with high or low prior knowledge. For instance, in Quilici and Mayer s (1996) study, high-variability examples were superficially different isomorphic examples; low-variability examples were superficially similar isomorphic examples. An SAT score of 575 was used to differentiate high-achieving students from low-achieving students. In contrast, the high-variability examples in Quilici and Mayer
Fig. 3 A positive example and a negative example of altitude of a triangle differing in the aspect of vertex

264 Fig. 4 Two positive examples of altitude of a triangle

Educ Psychol Rev (2012) 24:251269

(1996) were low-variability examples in studies conducted by Rittle-Johnson and colleagues (Rittle-Johnson and Star 2007, 2009; Rittle-Johnson et al. 2009). High-variability examples in these studies were examples that had similar surface features but different structural features. Rittle-Johnson et al. (2009) classified students who used an algebraic approach to solve an equation at pretest as higher prior knowledge students, and those who never used algebra at pretest as lower prior knowledge students. Second, multiple examples are effective only if they do not overload the learner s working memory capacity. Quilici and Mayer (1996) found that comparing the highvariability examples can help lower ability students to focus on and recognize the structural features, and thus to construct more structure-based schema and demonstrate more structurebased performance. Rittle-Johnson et al. (2009) argued that students with lower prior knowledge benefited more from comparing low-variability examples because these examples required fewer resources to compare problem features and only examining the first line of the example (p. 850). Students with lower prior knowledge might not have sufficient prior knowledge to benefit from comparing high-variability examples. When given superficially similar examples that had different solution methods, students with lower prior knowledge were likely to focus on the different solution methods or structural features; however, as they did not have sufficient knowledge, those features would cause cognitive overload, even if students noticed important features of the examples, they would need sufficient knowledge to make sense of those features (p. 850). Rittle-Johnson et al. (2009) finally concluded that high-variability examples can be harmful for students with low prior knowledge of solution methods; neutral for students who are attempting to master one of the to-be-compared methods; and beneficial for students who accurately use one of the to-becompared methods (p. 849). High example variability is a way to increase intrinsic cognitive load to include interacting elements important to a task (Sweller 2010, p.134). However, as Sweller (2010) suggested, for a specific instructional procedure to be effective, intrinsic cognitive load (or example variability, element interactivity) must be increased as long as the total cognitive load does not exceed the learners working memory capacity but should be decreased if it is too overwhelming to be handled by the learners working memory. Different types of example variability might overtax different learners. It appears complicated to determine an optimal level of example variability and intrinsic cognitive load for different learners. Simply arguing that the effectiveness of example variability is altered by student prior knowledge has little pragmatic implication for teachers to determine which type of example variability should be given to their students with different levels. In our view, analyzing comparison of examples from the critical aspects of examples can also address the ambiguous effect of prior knowledge on learning from examples. Because critical aspects are identified according to students understanding, students who have

Educ Psychol Rev (2012) 24:251269

265

different levels of prior knowledge should perceive different aspects that are critical for their learning. Aspects that are critical for students with less prior knowledge might not be critical for students with more prior knowledge, as far as a particular learning task is concerned. For instance, Guo and Pang (2011) found that the aspects of vertex and perpendicular were critical for fourth-grade students to learn altitude of a triangle but were uncritical for sixthgrade students who had more prior knowledge. Consequently, students who have different levels of prior knowledge may perceive different aspects that are critical for their learning and thus benefit differently from the identical instruction. For a particular set of examples to be effective, critical aspects should be first identified for specific learners and certain patterns of variation and invariance should be used for discerning these critical aspects. On the one hand, if students do not have an opportunity to compare aspects that are critical for their learning, they would encounter difficulty; on the other hand, instruction or examples would be ineffective and inefficient if students are asked to compare and study aspects that have already been mastered and are therefore not critical to their learning.

Conclusions Variation theory research on example design contributes to the literature on learning from examples. It not only gives systematic guidance on how to design multiple examples to enhance student learning, but also provides theoretical justifications to account for the results. Findings of variation theory research shed light on ambiguous issues regarding example design in the literature. First, the distinction of surface/structural should be replaced by critical/uncritical in example study. Aspects and features that are critical to students understanding should be identified and compared in example design. Second, variation as well as similarity among examples should be taken into consideration in example design. Certain patterns of variation and invariance should be adopted to systematically determine the variability of examples. Third, students with different levels of prior knowledge perceive different aspects of examples that are critical for their learning. Examples should be designed according to aspects that are critical to specific students. These conclusions are complementary and provide instructional implications for teachers who are constructing examples in their instruction. The first step of instruction should be to ascertain aspects that are critical for students learning. Critical aspects are those that have not yet been learned and thus cause difficulty for students in the process of learning. Effective and efficient instruction should be tailored to individual students learning, even though making decisions about the critical aspects of an object of learning for different students is not an easy task. On the one hand, if the instruction does not provide sufficient information on the necessary critical aspects, then students with lower prior knowledge may have difficulty with learning. On the other hand, instruction that focuses on aspects that are uncritical to learning may narrow the space of variation for students with greater prior knowledge and thus is inefficient. After the critical aspects are identified for individual students to help them discern these aspects, examples should be designed with controlled variation to ensure that critical aspects are separately discerned before being simultaneously varied. Except for the conceptual and procedural learning in which variation theory has shown its effectiveness, it would be interesting to explore whether variation theory is effective in facilitating more complex learning (e.g., project learning) which differs in many aspects. Before confirming and advocating the effectiveness of variation theory in example design,

266

Educ Psychol Rev (2012) 24:251269

future research should be conducted to examine the effectiveness of different patterns of variation and invariance in promoting learning from examples, with a variety of topics and with a wider range of ages and disciplinary prior knowledge.
Acknowledgments This research was based on the project An Investigation of Creating Effective Problem Context in Teaching Mathematics supported by Key Project of Ministry of Education, Plan of National Science of Education of China (GIA117009), and a research grant from the Hong Kong Research Grants Council. We thank the anonymous reviewers for their constructive comments and suggestions.

References
Albro, E., Uttal, D., De Loache, J., Kaminski, J. A., Sloutsky, V. M., Heckler, A. F., et al. (2007). Fostering transfer of knowledge in education settings. In D. S. McNamara & G. Trafton (Eds.), Proceedings of the 29th meeting of the Cognitive Science Society (pp. 2122). Austin, TX: Cognitive Science Society. Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. (2000). Learning from examples: Instructional principles from the worked examples research. Review of Educational Research, 70(2), 181214. Bowden, J., & Marton, F. (1998). The university of learning: Beyond quality and competence in higher education. London: Kogan Page. Bransford, J. D., & Schwartz, D. L. (1999). Rethinking transfer: A simple proposal with multiple implications. In A. Iran-Nejad & P. D. Pearson (Eds.), Review of research in education, 24 (pp. 61101). Washington, DC: American Educational Research Association. Catrambone, R. (1994). Improving examples to improve transfer to novel problems. Memory and Cognition, 22(5), 606615. Catrambone, R., & Holyoak, K. J. (1989). Overcoming contextual limitations on problem-solving transfer. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15(6), 11471156. Cheung, W. M. (2005). Describing and enhancing creativity in Chinese writing. Unpublished Ph.D. Dissertation, University of Hong Kong, Hong Kong. Choy, C. K. (2006). The use of variation theory to improve secondary three students learning of the mathematical concept of slope. Unpublished Dissertation, University of Hong Kong, Hong Kong. Clarke, T., Ayres, P., & Sweller, J. (2005). The impact of sequencing and prior knowledge on learning mathematics through spreadsheet applications. Educational Technology Research and Development, 53(3), 1524. Cooper, G., & Sweller, J. (1987). Effects of schema acquisition and rule automation on mathematical problemsolving transfer. Journal of Educational Psychology, 79, 347362. Curry, L. A. (2004). The effects of self-explanations of correct and incorrect solutions on algebra problemsolving performance. In K. Forbus, D. Gentner, & T. Regier (Eds.), Proceedings of the 26th Annual Conference of the Cognitive Science Society (p. 1548). Mahwah, NJ: Erlbaum. Emanuelsson, J., Haggblom, J. et al. (2002). The way we do things in Sweden: On school cultures and math lessons from different perspectives. Paper presented at the Learner s Perspective Study symposium, Melbourne, Australia. Fraser, D., Allison, S., Coombes, H., Case, J., & Linder, C. (2006). Using variation to enhance learning in engineering. International Journal of Engineering Education, 22, 102108. Gentner, D. (2005). The development of relational category knowledge. In D. H. Rakison & L. GershkoffStowe (Eds.), Building object categories in developmental time (pp. 245275). Mahwah, NH: Erlbaum. Gentner, D., & Medina, J. (1998). Similarity and the development of rules. Cognition, 65, 263297. Gentner, D., & Namy, L. L. (1999). Comparison in the development of categories. Cognitive Development, 14 (4), 487513. Gentner, D., Loewenstein, J., & Thompson, L. (2003). Learning and transfer: A general role for analogical encoding. Journal of Educational Psychology, 95(2), 393405. Gentner, D., Loewenstein, J., & Hung, B. (2007). Comparison facilitates childrens learning of names for parts. Journal of Cognition and Development, 8(3), 285307. Gibson, J. J., & Gibson, E. J. (1955). Perceptual learning: Differentiation or enrichment? Psychological Review, 62, 3241. Gick, M. L., & Holyoak, K. J. (1980). Analogical problem solving. Cognitive Psychology, 12, 306355. Gick, M. L., & Holyoak, K. J. (1983). Schema induction and analogical transfer. Cognitive Psychology, 15, 138. Gick, M. L., & Paterson, K. (1992). Do contrasting examples facilitate schema acquisition and analogical transfer? Canadian Journal of Psychology, 46, 539550.

Educ Psychol Rev (2012) 24:251269

267

Groe, C. S., & Renkl, A. (2006). Effects of multiple solution methods in mathematics learning. Learning and Instruction, 16, 122138. Groe, C. S., & Renkl, A. (2007). Finding and fixing errors in worked examples: Can this foster learning outcomes? Learning and Instruction, 17, 612634. Guo, J. P., & Pang, M. F. (2011). Learning a mathematical concept from comparing examples: The importance of variation and prior knowledge. European Journal of Psychology of Education, 26, 495525. Hammer, R., Bar-Hillel, A., Hertz, T., Weinshall, D., & Hochstein, S. (2008). Comparison processes in category learning: From theory to behavior. Brain Research, 1225, 102118. Holmqvist, M., Gustavsson, L., & Wernberg, A. (2007). Generative learning: Learning beyond the learning situation. Educational Action Research, 15(2), 181208. Holyoak, K. J. (1985). The pragmatics of analogical transfer. In G. H. Bower (Ed.), The psychology of learning and motivation (pp. 5987). New York: Academic Press. Holyoak, K. J., & Koh, K. (1987). Surface and structural similarity in analogical transfer. Memory and Cognition, 15(4), 332340. Ki, W. W. (2007). The enigma of Cantonese tones: How intonation language speakers can be assisted to discern them. Unpublished Ph.D. dissertation. University of Hong Kong, Hong Kong. Kolodner, J. L. (1997). Educational implications of analogy. American Psychologist, 52, 5766. Kotovsky, L., & Gentner, D. (1996). Comparison and categorization in the development of relational similarity. Child Development, 67, 27972822. Kurtz, K. J., Miao, C., & Gentner, D. (2001). Learning by analogical bootstrapping. The Journal of the Learning Sciences, 10, 417446. Kwong, S. P. E. (2005). The use of variation theory in developing students critical thinking skills. Unpublished Dissertation, University of Hong Kong, Hong Kong. Lam, Y. S. (2005). The use of variation theory to improve the teaching and learning of international trade. Unpublished Dissertation, University of Hong Kong, Hong Kong. Lave, J. (1996). Teaching, as learning, in practice. Mind, Culture, and Activity, 3(3), 149164. Linder, C., Fraser, D., & Pang, M. F. (2006). Using a variation approach to enhance physics learning in a college classroom. Physics Teacher, 44(9), 589592. Linder, C., & Marshall, D. (2003). Reflection and phenomenography: Towards theoretical and educational development possibilities. Learning and Instruction, 13(3), 271284. Lo, M. L. (2002). Catering for individual differences: Building on variation: The first findings. Hong Kong: INSTEP. Lo, M. L., Chik, P., & Pang, M. F. (2006). Patterns of variation in teaching the colour of light to Primary 3 students. Instructional Science, 34(1), 119. Lo, M. L., Marton, F., Pang, M. F., & Pong, W. Y. (2004). Towards a pedagogy of learning. In F. Marton & A. B.M. Tsui (Eds.), Classroom discourse and the space of learning (pp. 189-225). Mahwah, NJ: Lawrence Erlbaum Associates. Lo, M. L., Pong, W. Y., & Chik, P. M. P. (2005). For each and everyone: Catering for individual differences through learning studies. Hong Kong: Hong Kong University Press. Marton, F. (1999). Variatio est mater Studiorum. In: Opening address delivered to the 8th European Association for Research on Learning and Instruction Biennial Conference, Goteborg, Sweden, August 2428. Marton, F. (2006). Sameness and difference in transfer. The Journal of the Learning Sciences, 15(4), 499535. Marton, F., & Booth, S. (1997). Learning and awareness. Mahwah, NJ: Lawrence Erlbaum Associates. Marton, F., & Morris, P. (2002). What matters? Discovering critical conditions of classroom learning. Kompendiet, Goteborg, Sweden: Acta Universitatis Gothoburgensis. Marton, F., & Pang, M. F. (2006). On some necessary conditions of learning. Journal of the Learning Sciences, 15(2), 193220. Marton, F., & Pang, M. F. (2008). The idea of phenomenography and the pedagogy for conceptual change. In S. Vosniadou (Ed.), International handbook of research on conceptual change (pp. 553559). London: Routledge. Marton, F., & Tsui, A. B. M. (2004). Classroom discourse and the space of learning. Mahwah, N.J.: Lawrence Erlbaum Associates. Merrill, M. D., & Tennyson, R. D. (1978). Concept classification and classification errors as a function of relationships between examples and nonexamples. Improving Human Performance Quarterly, 7, 351 364. Namy, L. L., & Clepper, L. E. (2010). The differing roles of comparison and contrast in childrens categorization. Journal of Experimental Child Psychology, 107, 291305. Namy, L. L., & Gentner, D. (2002). Making a silk purse out of two sows ears: Young childrens use of comparison in category learning. Journal of Experimental Psychology. General: 131(1), 515.

268

Educ Psychol Rev (2012) 24:251269

Ng, F. P., & Lai, M. C. (2002). Learning study 6: P.4 Chinese language lesson on writing of modern poetry. In M. L. Lo (Ed.), Catering for individual differences: Building on variation: The first findings (pp. 7485). Hong Kong: INSTEP. Oakes, L. M. (2001). The role of comparison in category formation in infancy. Paper presented at the 68th Anniversary Meeting of the Society for Research in Child Development, Minneapolis, MN. Oakes, L. M., & Ribar, R. J. (2005). A comparison of infants categorization in paired and successive presentation familiarization tasks. Infancy, 7, 8598. Paas, F., & van Merrienboer, J. (1994). Variability of worked examples and transfer of geometrical problemsolving skills: A cognitive-load approach. Journal of Educational Psychology, 86, 122133. Paik, J. H., & Mix, K. S. (2006). Preschoolers use of surface similarity in object comparisons: Taking context into account. Journal of Experimental Child Psychology, 95, 194214. Pang, M. F. (2002). Making learning possible: The use of variation in the teaching of school economics. Unpublished Ph.D. dissertation, University of Hong Kong. Pang, M. F., Linder, C., & Fraser, D. (2006). Beyond lesson studies and design experiments: Using theoretical tools in practice and finding out how they work. International Review of Economics Education, 5(1), 28 45. Pang, M. F., & Marton, F. (2003). Beyond lesson study: Comparing two ways of facilitating the grasp of some economic concepts. Instructional Science, 31(3), 175194. Pang, M. F., & Marton, F. (2005). Learning theory as teaching resource: Enhancing students understanding of economic concepts. Instructional Science, 33(2), 159191. Pong, W.Y. (2000). Widening the space of variationInter-contextual and intra-contextual shifts in pupils understanding of two economic concepts. Unpublished Ph.D. dissertation, University of Hong Kong, Hong Kong. Quilici, J. L., & Mayer, R. E. (1996). Role of examples in how students learn to categorize statistics word problems. Journal of Educational Psychology, 88, 144161. Ranzijn, F. J. A. (1991). The number of video examples and the dispersion of examples as instructional design variables in teaching concepts. The Journal of Experimental Education, 59(4), 320330. Reed, S. K. (1989). Constraints on the abstraction of solutions. Journal of Educational Psychology, 81, 532 540. Reed, S. K. (1993). A schema-based theory of transfer. In D. K. Detterman & R. J. Sternberg (Eds.), Transfer on trial: Intelligence, cognition, and instruction (pp. 3967). Norwood, NJ: Ablex. Reed, S. K., & Bolstad, C. A. (1991). Use of examples and procedures in problem solving. Journal of Experimental Psychology: Learning, Memory, and Cognition, 17, 753766. Renkl, A., Stark, R., Gruber, H., & Mandl, H. (1998). Learning from worked-out examples: The effects of example variability and elicited self-explanations. Contemporary Educational Psychology, 23, 90108. Richland, L. E., & McDonough, I. M. (2010). Learning by analogy: Discriminating between potential analogs. Contemporary Educational Psychology, 35, 2843. Richland, L. E., Holyoak, K. J., & Stigler, J. W. (2004). Analogy use in eighth-grade mathematics classrooms. Cognition and Instruction, 22, 3760. Rittle-Johnson, B., & Star, J. R. (2007). Does comparing solution methods facilitate conceptual and procedural knowledge? An experimental study on learning to solve equations. Journal of Educational Psychology, 99(3), 561574. Rittle-Johnson, B., & Star, J. R. (2009). Compared with what? The effects of different comparisons on conceptual knowledge and procedural flexibility for equation solving. Journal of Educational Psychology, 101(3), 529544. Rittle-Johnson, B., Star, J. R., & Durkin, K. (2009). The importance of prior knowledge when comparing examples: Influences on conceptual and procedural knowledge of equation solving. Journal of Educational Psychology, 101(4), 836852. Ross, B. H. (1984). Remindings and their effects in learning a cognitive skill. Cognitive Psychology, 16, 371 416. Ross, B. H. (1987). This is like that: The use of earlier problems and the separation of similarity effects. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13, 629639. Ross, B. H. (1989a). Distinguishing types of superficial similarities: Different effects on the access and use of earlier problems. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15, 456468. Ross, B. H. (1989b). Remindings in learning and instruction. In S. Vosniadou & A. Ortony (Eds.), Similarity and analogical reasoning (pp. 438469). Cambridge: Cambridge University Press. Ross, B. H. (1997). The use of categories affects classification. Journal of Memory and Language, 37(2), 240267.

Educ Psychol Rev (2012) 24:251269

269

Ross, B., & Kennedy, P. (1990). Generalizing from the use of earlier examples in problem solving. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16(1), 4255. Ross, B. H., & Kilbane, M. C. (1997). Effects of principle explanation and superficial similarity on analogical mapping in problem solving. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23 (2), 427440. Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and Instruction, 16, 475522. Schwartz, D. L., & Martin, T. (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22(2), 129 184. Siegler, R. S. (2002). Microgenetic studies of self-explanation. In N. Granott & J. Parziale (Eds.), Microdevelopment: Transition processes in development and learning (pp. 3158). New York: Cambridge University Press. Silver, E. A., Ghousseini, H., Gosen, D., Charalambous, C., & Strawhun, B. (2005). Moving from rhetoric to praxis: Issues faced by teachers in having students consider multiple solutions for problems in the mathematics classroom. The Journal of Mathematical Behavior, 24, 287301. Sowder, L. (1980). Concept and principle learning. In R. Shumway (Ed.), Research in mathematics education (pp. 244285). Reston, VA: NCTM. Svensson, L. (1984). Mnniskobilden i INOM-gruppens forskning: Den lrande mnniskan. [The view of man in the research of the INOM-group: The learning man]. Goteborg: Institutionen fr pedagogik, Gteborgs universitet. Sweller, J. (2010). Element interactivity and intrinsic, extraneous, and germane cognitive load. Educational Psychology Review, 22, 123138. Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and Instruction, 2, 5989. Sweller, J., van Merrienboer, J. J. G., & Paas, F. G. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251296. Tennyson, R. D. (1973). Effect of negative instances in concept acquisition using a verbal learning task. Journal of Educational Psychology, 64, 247260. Tennyson, R. D., & Park, O. C. (1980). The teaching of concepts: A review of instructional design research literature. Review of Educational Research, 50(1), 5570. Van Dooren, W., de Bock, D., Hessels, A., Janssens, D., & Verschaffel, L. (2004). Remedying secondary school students illusion of linearity: A teaching experiment aiming at conceptual change. Learning and Instruction, 14, 485501. VanderStoep, S. W., & Seifert, C. M. (1993). Learning how versus learning when: Improving transfer of problem-solving principles. The Journal of the Learning Sciences, 3, 93111. Watson, A., & Mason, J. (2002). Student-generated examples in the learning of mathematics. Canadian Journal of Science Mathematics and Technology, 2(2), 237249. Wittwer, J., & Renkl, A. (2010). How effective are instructional explanations in example-based learning? A meta-analytic review. Educational Psychology Review, 22, 393409. Wolff, P., & Gentner, D. (2000). Evidence for role-neutral initial processing of metaphors. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26, 529541.