CHAPTER EIGHTEEN

.. ~~~.a~=-as-=~&* ..

=-

-=c=~DU~

...

PROSPECTS AND LIMITS OF THE EMPIRICAL STUDY OF EXPERTISE: AN INTRODUCTION

K.A~1)ERS ERICSSON AND JACQUI SMIm

Research on expertise may be one of the most ra'Jidl;-. expanding areas within cognitive psychology and cognitive science. Typically. "hen a topic becomes popular ill psychology. the research approach and the methodology associated with it are also accepted, and the pressure 10 demonstrate the utility and feasibility of the approach diminishes. Ufforts are directed instead toward the theoretical integration of research findings. furthermore, popularity of a new approach nearly always means that many investigators will adopt it. An even larger number of investigators, however, will adopt only the terminology and will attempt to modi Iy other research approaches to encompass the IIC\'I.' concepts. That. in rum, leads to diffusion of the defining characteristics of the "new" approach, making. straightforward attempts to integrate published research findings difficult. Because of thi-, P"l>CeSS of diffusion, often the new approach will no longer be readily disringuishabie from previOllS alternative research approaches. In this chapter we attempt to provide a conceptual framework for &Lin~ishing irnporJanl characteristics of the original expert ise approach. Our chapter consists of three sections. The first section attempts to characterize the study of expe+ise in the most general and domain-independent manner so that we can compare tae expertise approach with a number of alternative approaches mat nad similar objectives. The focu IIf this section is on briefly reviewing some of the outcomes and failures 0:( e earlier approaches, Our goal is to show that the expertise approach can account for these failures at the expense of greater empirical and theoretical complexity. In the second section we specify the nature of the original

From K. Anders Ericsson, Jacqui Smidt. Toward u Ganerai Theory nfF:AI".r/j~t, "Prospects and:inL.~ "j!he empirical smdy of expertise: un mtroduction, pp. 1-;0.

393

394

SECTION VIII

EXPERTISE

expertise approach and methodology. Here the pioneering work 01: chess expertise by de Groot (1978) and Chase and Simon (1973) is used to exemplify the sequence of research steps that cnsracterized the original expertise approach. Tnthe final section we elaborate criterie :::0: :hese steps and usc these criteria to discuss and review the prospects for. and limits Jr. more recent research on expertise.

1R.l

DEFTNITION or OUTSTANDING PERFORMANCE AND EXPERTISJ:: A COMPARISON

On the most general level, the study ()r expertise seeks to understand and account for what distinguishes outstanding individuals in a domain from less outstanding individuals in that domain, as well as from people in general. We deliberately usc the vague term "outstanding" because by not spcci fying more detailed criteria we are able TO point to a number of distinctly different scientific approaches that have addressed the same problem. In nearly all human endeavors there always appear to he some people who perform at a higher level than others, people who for some reason stand out from the majority, Depending on the historical period and the particular activity involved, such individuals have been labeled exceptional, superior, gifted, talented, specialist, expert, or even lucky. The label used to characterize them reflects an attribution of the major factor responsible for their outstandmg reZr:..:'L nilether '.; icreaded 10 or not. Scientific efforts to understand the sources of such o~~ c.e:z.-:,): ,.= ~ been _Ldee t-;. similar conceptions and attributions. We limit 01..r ~._~".. to rhn-e ca.-.c .. i~",hie" -'_e cutstaoding beaavior can be attributed to relatively STablecnaractensncs c: the re i"'''ln: mC!\'jaz-k n:e believe that ~ilit) ofrhe individual characteristics 1S a necessary condiuon fur all} empirical approach seeking to account tor the behavior with reference to characteristics of me individual. This constraint does not distinguish whether the characteristics are inherited or acquired, It docs. however, eliminate a large number of achievements due to unique immediate environmental circumstances, The most obvious achievements to be excluded by the stability constraint are those that involve events of fair games chance, 'Ouchas winning a large amount of money in a single lottery. More interestingly, the same criterion rules out achievements that occur only once in a lifetime, such as a single scientific discovery, a major artistic creation, a historically significant decision Of prediction, or a single victory in a sport This, of course, does not mean that we reject the possibility of defining criteria tor outstanding performances in rhe arts, sciences, and sports arenas. It does mean, however, that a single achievement in a unique situation docs not allow ux to infer thal the achievement was solely due to the particular indi vidual's characteristi cs. In o'"Clerto support an attribution to the stable characteristics of a person, ideally one would require a series outstanding achievements under different circumstances. Furthermore, one 'Wouldlike to have a larger group of other individuals (a "control" group of sorts) who have e.... rperienced similar opportunities to make contributions or to achieve. In the case in which man) other indiv iduals would he equally likely tll achieve in similar situations, there is no need to anribute the achievement to special personal characteristics. Almost by definition the numbers of individuals given opportunities in some life realms to achieve and

or

or

C:HAPTF.R FilOHTEl:::-I

PROSPECTS AND

LIMITS or

rue EMPIRICIIL S'I UDY OF EXPERTISE

395

to stand out from die majority arc ~'TT1a II (e.g., heads of state, army generals, people with vast economic resources). In such cases, even a stable series of achievements cannot unambiguously be linked to stable personal characteristics. because of the confounding iaflucnce of a unique stable situation. Examination of our simple stable-characteristic constraint indicates that many achievements popularly acknowledged as evidence for expertise must be questioned and carefully scrutinized. Another important consequence ofthis constraint is more indirect and concerns the validity of social evaluation and perception o outstanding performance or ability. One would expect social evaluation to he greatly influenced by observations or previous performances (not all h) the same individual) occurring under ....mque circumstances. A social judgment, Then. mighr not be tbe most precise cvalearioz 0: an individuals current ability to perform. Ideally. one needs 1.0determine the nniqce sr:"'...:mon0: the individual and to observe performance- 'J! standardized situatious that allow i:::l!.1!Tllkh. idual comparisons (e.g., laboratory tasks or tests). Once it is possible to measure superior performance under standardized conditions. there is no need to rely 011 social indicators. Attuned to some of the difficulties of ciefil'Jilio~and assessment, let us now proceed to discuss some scientific approaches that have been directed toward accounting for outstanding or superior performance.

lR.2

SCTRNTTFlCAPPROACHESTOACCOL FOR OUTSTANDING PERFORMANCE

Several different scientific approaches

L'G

IH1\'e been used HI imc~~e uj ~...s=..i 1$ performance. The constructs thai have he en investigated haw' priman.y rcaec:=.: ,:..o:;.l!;:.r attributions regarding the source of the outstanding behavior. These coe:ep~jOll.5. in turn. have directly influenced what empirica. eo ..idence has been considered and collected. Table 18.1 summarizes the di fferenl ~0' ('-"stable persona, charac.enst cs that have been hypothesized to underlie outstanding :;:;er.orm?ll-:e and :_m_s those artribution: to associated theoretical constructs and researca rnethod-. --e am-=-~l:""ed :-CTsonal characteristics noted in table 18.1 reflecr a basic oelief mar bez:r.nor either is predominantly influenced by inherited qualities or is a function Of:e:r...Uilg and acquisition. Further, outstanding performance is attributed ci[;"e~ ro some general characteristic of the indix idual or to a specific aspect. The assocrated taeoretical constructs and methodologies reflect these dimensions: inherited "e-rSIIS acquired, general versus specific. So, for example, the researcher will focus either on the effects of general traits (e.g., intelligence. personality), specific abilities (e.g., musical ability, spatial ability), and general Ii fc ann educational experience (e.g., language, study strategies) or on domainspecific training and practice. One's conception of the likely origins of outstanding performances will greatly influence the group of people selected for study, as well as the type of information sougar concerning these individuals. For example, investigators pursuing an account in terms of general inherited capacities would be likely ttl consider individuals regardles- <If their domains and would be particularly interested in information allowing assessment of the genetic contribution. A longitudinal study of individuals identified as having exceptionally

396
TABLE J8.1

SECTION VIII

EX1'I:.IUTSF.

Different Approaches to Aceounuug for Outstanding I'erformance

COJliSTRUCT

ATIRlBUTlOK

RESEARCII APPROAl H

Primarily inherited General abiaties Specific abilities Primarily acquired General learning and experience Domain-specific training .::nJ practice General knowledge and cognitive strategies Domain- or task-specific knowledge Investigation of common processing strategies Anatysis of (ask performance. i.c., the expertise approach Intelligence. personality Lg , music ability, artistic ability. body build Correlation witn personality profile, general intelligence Correlation with measures of specific ability

high intelligence, by Terman and his associates (Odcn, 1968; Stanley, George, & Solano, 1977; Terman & Oden, 1947), illustrates this approach. A focus on domain-specific acquired characteristics would lead investigators 10 constrain themselves to one domain or task and to try t.oassess what was acquired (e.g .. specific memory strategies), as well as the process of acquisition. OL. a priori grounds one cal" Z1'5-..! tha: the most ~[J1on;(\~ theoretical account of outstanding performance is in terms of general. piedominant.y mhenred characteristics. Indeed. In the history of scientific research on superior performance, that approach was initially preferred. It was primarily because of inability to explain certain empirical observations that accounts based on more specific abilities and acquired characteristics came to be seriously considered. Wr.:, shall briefly consider some of those failures before turning to a consideration of the expertise approach that exemplifies the belief that specific acquired characteristics underlie outstanding performance.

18.2.1 Accounts in Terms of General and SpecificInherited Characteristics

-0 attribute outstanding performance to general inherited characteristics, it is reasonable to rely on readily available criteria to identify instances of outstanding behavior ami of individua.s who exhibit that behavior, criteria such as social evaluation and recognition by onc-, peers. In the first major study in that area, Ga lton (1869) used social recognition to identify eminent individuals in a wide range of fields and then studied their familial and genetic origins. Galton argued that individuals gained eminence in the eyes of others because of a long-term history of achievement. Such achievement. he suggested, was the product of a blend of intellectual (natural) ability and pcrsonal motivation. He reported strong evidence for eminence's being limited to a relatively small number of families stemming from common ancestors, and he inferred that eminence was genetically determined.

If one wants

CHAPTER F1GIITFJ::N

PROSPE('1SA~D

Ll~1lTI' OF THE E:-'1PflUCAJ. STUDY OF tXl'ERnc;,e

39

Contemporary work in Galton's time and suhsequcnr studies were directed at uncov ering the loci of indi vidual differences in genera, ability. 1 he genetic nature of those general capacities led investigators to search for di'ferences in basic characteristics o processes, such as the speed of mental processes as re ~ected by reaction time. In sub sequent studies, however, individual difference" in pc-forrmace simple tasks showed disappoinnngly low correlations, both among tasks ana bcrweer performance and indice of ability, such a" grade in school (twIford, 1967). More recent effort to uncover general basic cognitive proce es ilia: could accoun for individual differences have been inconclusive (Baron, 1978. C2.~\J = - -~: Cooper & Regan. 1982; Hunt. 19RO). For example, research on individual dH--~ __ es n genera memory ability has found low correlations of memory performance across .:o-c:"re~t type of material and methods of testing, leading investigators to reject the ,~_ - _ ;eneral memory ability (Kelley. :964). More direct evidence against stable ~. : ........ mor processes comes from repeated demonstration" that memory performance f~- spec fK types of material can be drastically improved even after short periods of practice _1935; Kliegl, Smith. & Baltes. 1989}. Moreover, as Cooper and Regan noted (19 :.:inadequacies ill the definition and design of both cognitive tasks and intelligence rneascre create serious problems for interpreting correlations between measures of basic co~~ tive processes and ability. Tests measuring general intelligence have been extremely useful for prediction an diagnosis in a wide range of siruations, although there is considerable controversy abou what they actually measure (Resnick, 1976; Sternberg, 19R2). IQ tests, however, have bee remarkably unsuccessful in accounting tor individual differences in levels of performance ill the aITS and sciences and advanced professions, as measured by social indicators (e.g., money earned, status) and judgments (c.g., prizes, awards) (Tyler. 1965). There were other lines of research that examined subjects with reliably ~ ~ forrnanccs and compared them with control groups, Much of that researc" '--:,s '_""..l.I motivated by the belief that exceptionally high levels of performance wouk, ~"'ct ~ exceptional ability involving, attention (power or concentration). mere ~ ... _ ~ szeec reaction, or command of logic. Some investigators, however, ...... ~ ~ individual characteristics, such as features of personality. mo.-:(e.g., Cacell, 1963: Roe. 1953). In ~e I o~Os.. wee Russian professors examined dl:: -~- .... :1_'- of eight gran CU5'~ "..o.-ld-.::.1...~ chess players) on a wide range ~--hl -- ,~.s ."i.lrbasic cognitive and ~Ice;:~ ~ 1:.0;:-- de Groot. ; 946 'I 9781. S~~ ;,,:;: grand masters did not di fer from coarrol subjects ill IDOsebasic abilities, r-_i ~"'!' lO.~ ... dea-rly superior 111 memor tests involving c~... " po- -0-...5. In the C2Se "': :.a-""' pe-ilO"...-::l-:.:e. '"-=t:ro r spatial ability often is ass:::r;. to be essential (Cha.-.e .&. Simon, {0-3: 2n!~ . " 5 . Doll and Mayr (19R7) co~ the performances of about u.m" or 1:4_ ~ ebcss players ill what was then West C-e-.::-~ with those of almost ninery :- ~~ ec ...... of similar ages, using an TQ test ~_;.::_ ~-:-subscales, Only three of the subseales showed reliable differences, and somewtm !-'T'ri.s ingly the largest difference bet'>'cen the two groups concerned higher scores fo :::~-ccacularion for the chess masters. noll and Mayr (1987) found no evidence that d:e:-., player were selectively better on spatial tasks. In accounting for the unexpected superiority of th

0:

-=

~_=-

.<

398

SECTION \1111 FXPl:.lU lSI::

chess players on two of the subscales, Doll and Mayr (J 987) argued that one reason could be that elite chess players had prior experience in coping with time pressure because of their past chess competitions. When the analysis was restricted to the group of elite chess player s, none of the subscales of the lQ test was found to have a reliable correlation with chess-

r:a~ mg performance.
Of the research that has focused not on intelligence hut on other relatively stable characteristics of individuals. that by Cattell (1963; Cattell & Drevdahl, 1955) is probably the best example. Cattell ..ought to determine whether the personality profiles for eminent researchers in physics. biology. and psychology could he distinguished from those of teachers and administrators in the same fields and from those of the general population. Compared with all other groups. rop researchers were found to exhibit a consistent profile, being more self-sufficient. dominant, emotionally unstable, introverted. and reflective. Such a profile supports Galton's earher opuuon that eminence and outstanding achievement in a field are products DOton:!) u: abllt.) but also of aspects of personal motivation. ;"lotivation and striving for excellence often are focused em a small number domains or even a single domain, suggesnng that aspects of motivation may well be acquired. Despite these hints at possible personality patterns, the research approach of accounting for outstanding and superior performance in terms of general inherited characteristics has been largely unsuccessful in identifying strong and replicable relations. The search for links to specific inherited abilities has been similarly inconclusive. Indeed as the specific characteristics proposed to account for the superior performance become integral to that performance, it becomes difficult to rule out the possibility that such characteristics have not been acquired as a result of many years of extensive training and practice. Investigators have therefore focused their attention on characteristics that appear in children and that reflect basic capacities tor which a generic origin is plausible. We shall briefly consider two examples of such ba..... ic capabilities, namely, absolute pitch among :nusicians and physiological differences among elite athlete ... A recent review of the research on absolute pitch shows mar most ofthe empirical cvideoce fuors an account in terms of acquired skill (Ericsson & Faivre, 1989). The ability to recognize mzsical pitch is not an all-ot-nunc skill, and many musicians have it to various degrees. r."1e~display the best performance on their own instruments, and their performance decreases as ar.ificial tones from a toile generator are presented (Bachem, 1937). The ability to name pitches correcny is closely related to the amount of one's formal musical training (Oakes, 19551. Furthermore, pitch recognition can be dramatically improved with training, ami one musician has documented how he acquired absolute pitch through longterm training (Brady. 19-0). Similarly, a recent review shows that many anatomical characteristics of elite athletes, such as larger hearts. more capillaries for muscles, and the proportions of different types of muscle fibers. are acquired during years of practice (Ericsson, 19(0). Such findings showing the far-reaching effects of training do not, however, rule out possible genetic constraints. An individual's height and overall physique are determined by generic factors (Wilson, 1986). Height and physique, for example, impose important constraints in many physical and sports domains, such as basketball, highjumping, gymnastics, ballet, and professional riding. Tl is also conceivable that genetic factors might influence the rate of improvement due to training. .cvcrthclcss, training and preparation appear to be

or

18.2.2

Account s in Terms of Specific Acquired Charaetcri lies: The Fxpertise Approach

Inthis brief review we have seen that the more parsimoniou- L~corncal approaches relying on stable inherited characteristics seem inadequate to ace ULI fur ctsranding and superior performance. It is therefore necessary 10 consider accounts cased on xqmred characteristics. Here we need to identify not Oldy what the acquired charaererisrics .... re ;:n;: also the process by which they arc acquired. How long is the acquisition period, and over what time frame un ..... -e neec ID o...bserve and monitor changes in performance? Simon and Chase (197.~) were the a> :n..: that 10 years or more of full-time preparation are required to attain an internar'onar le e{ of performance in chess. Studies by Hayes (I % I) and 1I100m ( 1985) revealed Thata decade of intensive preparation is necessary to become an international performer in sports 01 in the arts or sciences. In a recent review, Ericsson and Crutcher (1990) found consistent support for the requirement of 10 years of intensive preparation in a wide range of studies of international levels of performance. Furthermore, Ericsson and Crutcher (1990) found for many domains that most international-level performers had been seriously involved in their domains before the age of (> years, The period of preparation for snperior performance appears to cover a major proportion of these individuals' development during adolescence and early adulthood. :\ detailed analysis of acquisition processes extending over decades under widely differen. crl' 1..""\: nmemal circumstances 1S extraordinarily difficult TO conduct. Without a thcorencal rramework 00 outline 'ne relevant aspects, the number of possible factors thaI. coold be critical W :lr:.:cin, penor oerformance b ..!.~~ rme cae, (I:: X'_:.e, f:!ZlP' -:!.e""::..:l or ~ range of fa, IF:::::' ;-~g :....J;;I~lhje,anc .rr.a.!:."SeS or ~-"cl ...... --s - et:nI:L~':e5 :_ The lives of outst:mdine 'C1:eI:: sts 3M arnsts Alberu, ~~ i. ,<.r:e .:.ct;.. 38.3). ill. i!> .. nhkefy, though, that de -cnpave stedie :-.. -cking correianoas rern.~"C ulttma~ pezforraance ofindividuals and informr.ion aboo- their 00 eklP=I~:' ci5::.uri_- w: ~. er be able to yield conclusive results. A ro.udl ;:n.cri'e promising :approach is offered ;)j' a carer ul analysis of the

::.cst on

attained performance. ~ i.., lOCcrux of rhe expertise approach. 'I he expertise app1'!"J3t differs trom the :l~ro::u::ht:~ discussed earlier in some important respects. The other approaches were anempe- to measure independently the constructs hypothesized to be the sources "" ,~ of omsrandicg performance. In contrast, the expertise approach is an arternpt tn describe me cnncal performance under standardized conditions. to analyze it, and to identify the components of the performance that make it superior. Two features distinguish the expertise approach from other approaches: first, the ''''~'renee that it is necessary to identify or design a collection of representative tasks to ..... pnrre the relevant aspects of superior performance in a domain and TO elicit superior performance under laboratory conditions; second. the proposal that systematic empirical anal) - of the processes leading to the superior performance will allow assessment of cnnca; mediating mechanisms. Moreover, it is possible to analyze the types of learning or adaptation processes

400

1:..XPI:RTlSE

~ 'Io,~ch these mechanisms can be acquired and to study their acquisition in real lite or u;a1..- ::..J>omlory conditions. The expertise approach is more limited in its application than the other approaches renewed earlier. Whereas the other approaches can use social indicators as criterion variables of outstanding performance, the expertise approach requires the design of a set of standardized tasks wherein the superior performance can be demonstrated and reliability reproduced. With this importaru limiration in mind we now turn to a closer examination of the original expertise approach.

1M.2.3 The Original Expertise Approach: The Pioneering Work on Chess

ne experase a;p:oach should be characterized. Tfone rakes se :::. ce Groor 19-8) anti Chase and Simon (1973), however, ir I" r: ' o!'"':tr':''';-ll ib!'~ l:' eneral chrractcri-rics. First. the focus is Oll producing and obse-vzig ~ =- j1';:GilWf"'lTl,7elC. me laoorarory under relati- ely standardized condi- s.t~ ~ ~ ..;1zaeorerical concern to analyze and describe the cognitive processes L1 ~ If:e;:-......... cnor; of an outstanding performance on such tasks. Finally, the critical cogarcve processes are examined, and explicit learning mechanisms arc proposed to account for their acquisition. If one is interested in reproducing superior performance under standardized conditions, one should give preference to domains in which there are accepted measures of perforrnance. Chess provides such a domain. It is possible to measure an individual's chess-playing ability from the results of matches against different opponents in di ffcrcnt tournaments (Elo, 1978).11 is easy to select groups of chess players who differ sufficiently in chess ability that the probability of one ofthe weaker players heating one of the stronger players ill a particular game is remote. A critical issue ill the expertise approach is how to identify standardized tasks that will allow the real-life outstanding performance to be reproduced in the laboratory. Because of the interactive nature of chess games and tilt; vast number of possible sequences of moves, the same sequences of chess moves are hardly ever observed in two different chess games. Belter chess players will consistently win over weaker chess players employing a wide variety of chess-playing styles. One could therefore argue that the better chess players consistently select IllOVC~ as good as, or better than, the moves selected by weaker players. De Groot (1978) argued that it is possible to develop a collection of well-defined tasks capturing chess expertise by having chess players select the "best next move" for a number of different chess positions. 'vleasuremenr of performance in this task requires that it be possible : ._!\ aluate qualitatively, on a priori grounds, the dependent variable, that is, the next chess -m"C <elected for a given chess position. II is nnt currently possible to evaluate the quality or caess mows for an arbitrary chess position. In fact, one international chess master claims 10 have spent a great part of his lite unsuccessfully seeking tu determine the best move for one p.articula chess position (Saariluoma, 19114). De Groot 19-8) collected think-aloud protocols from chess players of widely di Hering Ie- els of expertise while they selected their best next moves for several chess positions. A [lcr extended analysis of these classic positions, however, he found that only one ofthem etess ~

CHAPTCR F.HjHTEEK PROSPECTSJ\h'D LIMITS OFTHE L\fPl.RIC ...l.' STUDY UF EXI'1::RTISt;,

401

differentiated between grand masters and other Ch.eS5 experts who differed greatly in chess ability: All of the very best chess players selected bener moves than did any o f the comparatively weak players (nonovcrlappingj. Hence. he [v-"e;n:u that Lhetask of selecting moves for that chess position must elicit cognitive processes ~ differentiate chess players at di ffercnt levels of expertise. Another pioneering aspect of de Groot's study was r..15 u-e of verbal protocols. He was able to localize differences in cognitive processes berveec ~e grand masters and the other class expert....by analyzing think-aloud protocols froc, ro. -s ess-next-movc Task. He found that both masters and experts spent about 10 minutes betore ~nCi'lg 00 a move. In the beginning, [he players familiarized themselves with the ch ..... S J:!Pro. .;o'21ualed the position for strength ...~~ weaknesses, and identified a range of pr- ~ esc c. Later they explored in greaie, &:~uthe consequences of a few of those moves, 'b :!,<'e""3gte_ boch masters and experts C\,rs Ct:r-eC more than thirty move possibilities in\'OJVIn5 iJ ~'lB-Ia.:,,: and White and considered (:h-ee or four distinctly different first moves. De Groot (I 9n 11:st!~~d the possibility that, compared with chess experts .. N~ grand masters were able :v ~ re , :::ge; move combinations and thereby uncover the )l('S1 move. He found. hw C'. .e' _ I<.il.U 1h!II:C;l.'WI'Wll depth of the search (i.e., the length of move combinations) \Hi:> nnucl._ ~ "~. oe t groups. When de Groot then focused hi:analysis on n()ll.\the p_.1!':" ....:::-~. _ ':cr tt.rf:rcrn moves for the position, he did find differences. few or 6.: cces- experts -;u, J -~d ;.be best move, whereas most of the grand masters had noticed the 1x.cosmo e dbri _ 11:~ ~~..a:JOn with the position. More generally, de Groot argued, on the basis D.' __~. tb= pD':c~ols. that the grand Hli-lSters perceived and recognized the characicrisncs C ~ ~~~~. - ~ 1Ddevaluated possible moves by relying on their extensive experience rather (_,'''In b-v u::;:.::(JlI.: ~ t."cse cha-acteristics by calculation and evaluation of move possibilities. [0 ~.:;;..:ceses .,,_ "'"-"'--C- ery of promising chess moves was linked to the verbal report of a localized ",~i.>:..~ )- aae 0Jlil)0nent's chess position. Other grand masters discovered the same move \na.\ r ~ "er1~~ report of a mediating step (de Groot, 1978, p, 298). The superior chess-playing ability ofmcs, experienced chess players, according to de Groot, is attributable to their extensive expertence, allowing retrieval (If direct associations in memory between characteristics of chess positions and appropriate methods and moves. De Groot (1978, p. 3l6) argued that mastery in "the field of shoernaking, painting, building, [or] coufectionary' is due to a similar accumulation of experiential linkings. To examine the critical perceptual processing occurring at the initial presentation of a chess position. de Groot (1978) briefly showed subjects a middle-game chess position (2-10 seconds). Shortly after the end of the presentation the chess players gave retrospective reports 011 their thoughts and perception" during the brief presentation and also recalled the presented chess position as best they could. From the verbal reports, de Groot found that the position was perceived in large complexes (e.g .. a pawn structure. a castled position and that unusual characteristics ofthe position (such as an exposed piece or a far-advanced pawn) were noticed. Within this brief time, the chess masters were found to integral': a._;he characteristics of the position into a single whole, whereas the less experienced players .. ere not able to do so. The chess masters also often perceived the best move within that short exposun; time. The analysis of the amount recalled from the various chess positions was consistent with the evidence derived from the verbal reports. Chess masters were able TO recall the

or

402

SEl:TIONVUJ

F.XPERTISI:

positions of all the 20-30 chess pieces virtually perfectly; whereas the positions recalled by the less experienced chess expert ranged from 50 to 70 percent, The classic study of Chase and Simon (1973) followed up on this superior memory perforrrance by chess masters for briefly presented chess positions. They designed a standardized memory task in which subjects were presented with a chess position for 5 seconds with the sole task of tbe subjects being to recall the locations of as many chess pieces as possible. We shall later review more carefully to what extent this new task call be viewed as capturing the cognirn . processes underlying superior chess-playing performance. With that memory rask, Chase ani Simon (1973) were able to corroborate de Groot's earlier finding that chess players \\ ith higher levels of expertise recalled the correct locations of many more pieces forrcpresenrarive chess positions .' I'hey also went a significant step farther and expenmentalry varied the characteristics of the presented configurations of the chess pieces . 0: chessboards with randomly placed pieces, the memory performances of the chess - .asters 'were no better than those of novice chess players, showing that the superior memory performance of the master depends OIl the presence of meaningful relations between the chess pieces, the land ...of relations seen in actual chess games. Chase and Simon (1973) found that a player's ability to reproduce from memory the previously presented chess position proceeded in burst" in which chess piece were rapidly placed, with pauses of a couple of seconds between bursts. TIle pieces bclongmg to a burst were shown to reflect meaningfully related configurations of pieces (i.e., chunks) that corresponded well to the complexes discovered by de Groot (1978). The ches ...masters were found to eli ITerfrom other chess players primarily in me number of pieces oelouging to a chunk, that is, the size of the chunk. In support of the hypothesis that memory and perception of chess positions rely on the same encoding processes, Chase and Simer l ~r:} demonsirared that the recall process had it srrucrnrc similar to thai of the process ...~!"~iJfOduclTJg perceptually a a.::lb~edie .., posmons. Rather than discuss the large number of additional empirical studies I'lJ Chase ar.! Simon (1973), we shall change the focus and consider their theoretical effort :o specify the detailed processes underlying superior lDem:. performance and the relation of'these processes ro general constraints on human i:r~Onn,1;]1hiprocessing. One of The most ses ere ... onstraints ou all aCCOIlTlt that is based on aa.:,-u....~IOlm\ ledge and skill involves ex-plicating "'hal has been acquired and showir g m:l! the acquired characteristics arc sufficient TO account [<IT the superior performance wim-oUtviolating the limitations ofthe general capacities of human information processing fNe'\\el & Srnon, 19i2). The superior recall of 15-30 ches ...pieces by chess masters would ar firs; glance seem ro be inconsistent wi th the limited capacity of short-tern! memory in h'.JID3:lS, wnicb allows storage of around 7 chunks (Miller, 1956). Chase and SilllOlllllr3) Iound that the number of chunks recalled by chess players at all skill levels was well within the limit of around 7 :.:.2. They attributed the difference in memory performance between strong and weak players to the fact that the mOT(; expert chess players were able to recogmze more complex chunks, that is, chunks with a larger number of chess pieces per chunk. On the basis of computer simulations of the encoding and recall of middle-game chess positions. Simon and Gilmartin (1973) were able to show that 1,000 chunks were sufficient to reproduce the memory performance or a chess expert. They estimated {hal simulation of the performance of a chess master would require between 10.000 and 100,000 chunks. Assuming that the superior performance of the expert depends on the recognition or familiar patterns that

CHAPTER ElGlITEE'I

PROS!'ECTS AND LI).IITS OF rut EJ\.lPIRIC/I.L S" UDY OF EXPIRTISE

403

index previously stored relevant knowledge of successful methods (actions), the timeconsuming process ofbccoming an expert would consist in acquiring those patterns and the associated knowledge. Simon and Chase ( (973) estimated thar around 3,000 hours are required to become an expert, and around 30.000 hours to become a chess master, They also commented rhar "The organization uf the .\IIa."tcr" elaborate repertoire 01' information takes thousands of hours to build up, and the same IS true of any skilled ta,,~, e.g., football, music). That is why practice is the major independent variable in the acquisinoz or sk2:." p. 279). Whether or not one agrees with the Chase and Simon theory of expertise. :. . _ oe znwise to confound the methodology of their research with the theoretical as-umpriozs o; De:;specific theory. Indeed, Chase and Simon (1973) were rather \.datlOll' when they proposed Ii:;:::i:- --;"e._--::. describing it as simply a rough first approximation

18.3

TIlE THREl?; STF.PS OF THE ORlGI:,\'.-\L EXPERTIS.E APPROACH

from our review of the pioneering research on hess expertise we have extracted three steps. The first step involves capturing the eSSCIH.. "C r ':.!f"-"rperf0n11anCe under standardized laboratory conditions by identifying representarrve ...:. ....5 IL the following sections we try to distinguish between collections of tasks that caprare ~ sepenor performance and collections of tasks that measure a related function or ability. la om lre\."iev; of the initial work on chess we argued that only the task that required rhat 51 ll.'ects :Otli.Slsnentlyselect the "best moves" meets the criterion of capturing the nature of super-Of rertfarnor-wee. Two other tasks, one involving perception and the other measuring m .. "IIIut} fi chess positions, assess related functions hut do not directly represent CE:t'S5-p~ ililL The second step involves a detailed anal) sis of the ~_;}I--"D:lrf':~."':, -ance. The pioneering research on chess nicely illustrates the usc 0; refined r.:~"5C:5 c;-:;~~R:lCeS of verbal reports and placement of chess pieces to infer ::'e ~JyEg .:o;n:-i\ e processes mediating the superior performance, as well as the use ~ i!X~~ - ...."';;" 'aj('~ stimulus materials. The third and final step involves efforts to aca:om;;:--_ tae ::.;:~ !--;; -- ~i- ._c ;:!.;a.racteristics and cognitive structures and processes rtai cz e Y.e"_ ...;;;.J: ~ -,c~'::~~ :-"e superior performances of experts. A persistent fail..:.: ~~ . ..:.r.:.,:'_~ uncer which the critical characteristics could be acquired or ir-'r-;.y.tC .. c _.:. ~Q\ide strong evidence thai those characteristics are unmodifiable and hecce ~~C' and most likely inherited. OUT explication of the original expertise ::!~roa..:h imposes clear limits for its successful application. Unless the essence of the ,aperior performance of the expert can be captured ill the laboratory (satisfying the cnrenon tor the first step), there will not be a perIorrnancc to be further analyzed in terms of its mediating processes. Similarly, failure to identity mediating processes that can account for The superior performance during the second step will leave the investigator witb only the original di Ifcrcnces in overall perfo:mance and will make the third step essentially superfluous, At the same lime, our explication of the expertise approach is applicable tD.ll:y phenomenon involving reliably superior performance that can be captured in i.~ Izbcratory. We believe that an attempt to encompass phenomena normally labeled 35 perceptual

~-=~, ;~:~

404

SF.CTIO~ \1111 EXPERTISE

(e.g .. chicken sexing), motoric (e.g .. typing), or knowledge-based (e.g., physics) within the same overall approach will allow us to identify common methodclogical and theoretical issues and to consider a common and more differentiated ct of learning mechanisms in accounting for achievement of superior performance in an~ 0De of these different domains. Such an approach will have the additional advantage of allowing us to consider the many different perceptual, memo'). motoric. and knowledge-hasel! a-pects of superior performance ia domains like chess Caarness. 1991), physics (Anzai. : QC - medicine (Patel & Groen, 1991), performing art- ...a... sports ( Allard & Starkes, 1991 .~;: znusic (Sloboda, 1991 ).

18.:U Capturing Superior Performance: The Fir~tStep

The first Sie;" --~ ~r-emse approach involves finding or de=ig -,,; a collection of tasks to caprcre tle -o..?.:riOl pertormance in the appropriate dorrzsa .t ":ie is able to identify suca a cc ec; -- oJ: Ta! k s, the following important advantages accrue: first. the per:'l.-..:;.:m~~ o::;.~ designed tasks will reflect the stable characteristirs ,,':~e superior real-life p~~ce. \ron: important, the availability of such a collection ...::.sAS will allow us to performance of the experts extensively in order to aCCl!IL~": sufficient in formar: -l" on the mediating processes 1'0 make a detailed assessment ar..i :Iir;L." .... During these extensive observations uf performance, we should not expect SigrrlR.,,:.:IC" ;;~ges due to learning and practice, as we shall be monitoring stable processes IiJ..:j l;:.;r~ been adapted and perfected over a long period of time. The period during whi;;:l;a fl"I!'IM1IlI...J!lCe will be observed will be negligible in comparison. Finally, these collections Ill' tasks \\111provide us with all exce" ~ e-itllC,; g""o..nd for studying how rapidly the various irlcn ti fied characteristics can he :!,-.p:u:-e: ;:x.. -g.h practice. In fact, one could argue that with an adequate collection of tasks, _1-", -.tte:t .,:-"-acquisilion should be comparable for practice with the collection of tasks a::.c ia rea, c.!f. on the other hand, tile collection of designed tasks does not elicit the mec~r,K;;h::: ~1 mediate superior real-life performance, OT does so only partially, then we arc h~ t:) ~ substantial learning and changes in the processes as a result of further practice, .. Oj d5 of tasks that lead to rapid rises in levels of performance by experts with further are unlikely to yield an adequate representation of superior performance. Even r<"'~: devasrenng evideuce against the claim that such a collection can capture superior ~c::- ance comes from situations in which novices have matched or surpassed the perforriance Ie- els of experts after only a few weeks or mouths of practice. For some types of expertise i: is easy to identify such a COLecCOJ: of tasks, but ill most cases it is the most difficult step. Wc shall first describe some simple cases and then turn to rte i!:it'Th..-ul. i~e:, invc 1\ ed in designing a collection of tasks to characterize real-life experrise, \\~ sW:!....'(: cc .;; .lcr .he advantages and problems of designing a collection of mem''''-: u:s:i:s W 5:uC) scoenor memory performance by experts, as opposed to studying directly the ill[C'""c;- pcnlmnam.e of experts.

~=

me

r-~~

Tasks Capturing Real-Life Expertise. There arc few instances of real-life expertise in which superio-performancc CUll be demonstrated under relatively standardized conditions. Mental calculators and memory experts provide such instances. They often exhibit their

r:HAPTF.R F.1(lHTFFN

I'ROS?E(.'TS AND L11...tlTS Of THE Hfl'TRTCAl. STUDY OF F"\"Pl:IUlSf

405

performance under conditions similar LO tho. C used in rraditional experiments. Tn hoth of these cases it is easy to define a large pool 0: ~erent stimuli (e.g .. 10 ,,; , "0 possible multiplications ofn ... o 5-digit numbers, or l00::ril:. ~ cL,:-it sequences of 14 digns Drawing on this pool of items, the experimenter can observ ~ the :--crformance in a large nr.. :ri!:-erof different trials and accumulate information x. -~~ eogr.mve processes underlying _;'e expertise. Similarly, some types of psychomot t ce ..... ~~. such as typing, and some sporting events can easily he imported into the _.. Apart from The preceding cases, the I:Q1;:: r $~...!Z' __ task,...to capture real-life expert performance is difficult. The problem is so... e\\b:rt~.. - C- .. of isolating phenomena in the natural anti biological sciences. 3_ ea tefu' a::.a;.::,15 ~ ~e:. oerforrnancc in real life, we try to identify recurrent activities ~.a" C3l:. be repro,........ ~..... ~_ -"?n'led conditions. In Those domains in which expertise can be measured, ~-::1:t the focus to those activities that arc involved in :-nduci"'g "he re.evant ~ annaace es:'''':;;ing product, One should search for goal-directed aztrvities that result in ()~.:T'!h~-h2\im irult can be reproduced by presentation of the appropnaze stimuli, A nice illustration of this procedure co+es ~mn The previously described research on chess, in which de Groot (19n) designee ~ "j;~ 0:- selecting. the best next rnnvc for a given middle-game position. It should be possr e to culled a large number of such positions with 'Which even top-level chess players w fd:e unfamiliar. in extracting out a single chess position from a chess game, one 15 ~~ ~ 1::'-: a problem that is common in research on expertise, namely, the determination of~e;: "rei:': .e5POll.'C. or the reliable evaluation of selected mo ... es. Given that currently tt>CTC ._~ rccrhod available that could have provided Thatinformation objectively, de Groce -:.,: ~t an extended period carefully analyzing the selected chess position to evaluare zae ~-e .l,cnL...of di ffcrcnr rr-oves. A different method of dealing with this problem was ,,=c;:rl ;_ a ~eC'entsrucy 0) Saariluoma (1984). who selected chess positions that had de.:.:-.: .:. sceraible best next mov es. Both C)::.hese methods are oriented toward finding or des:_.,..._.,_ ... SIIt:.!l se, '-......,.k......... ~ _"V _!'ill:lOl easily he extended into specifying a large pop~ - ta~ r ~ "'" ~ ~ The chess expertise. III most other complex task domains, s:...- 2S ~ _ gaurrs lend LO select a small number of tasks those tasks were chosen to be a representanve task analysis of even a single complex pro ;0.::. ... consuming, MOT(; important, our knowledge 0""" ;:x -5 of e-xpertise is incomplete, and it would not at this time be possible to ~L -:: _ popu.anon of tasks to capture such expertise. Many scientists, however, ace work.=l".. bu.ld.ng expert systems in which the tasks and prerequisite knowledge must be ~ia~ and other researchers are working on describing the formal characteristics of variozs ~ environments. (see Charness, 1991 L In many domains. experts produce con.p.ex products such as tcxix on a given topic or performances of a given piece of music. Alzhough judges can reliably assess the StJ;c.o ~ ial-ry of the product, it is difficult to analyze such products in order to identify ~:! C~surab e aspects capturing the superior quality of the product. lIence. in their r . v~ of e ... pertise ic writing, ScardamaJia and Bereuer (1991) focus on systematic C~..1.~"~~~CS of ""'Ie ... _;niuYe processes involved in designing and writing a text in an effort to :"=~!entiate expert i:o-n novice writers.

i, ,,~ - ~.

406

~EcnON VUJ EXPER'HSE

!~ 15. of course, possible to give up the hope of designing a collection of tasks that could capture the full extent of the superior performance and focus instead on one 01' more well-defined activities involved in the expertise or measuring knowledge about the task domain. In adopting such an approach. one no longer can be certain that one is examining _ gninve structures and processes essential to the superior performance, Occasionally, expected differences between the performance of novices and that of experts in component activities arc not found. For example. Lewis (19!H) found no reliable differences in performancc on algebra problems berweer; expert mathematician" and the top third of a group of college students. The most frequellil} 5'ooied activity related to expert performance is memory for meaningful stimuli fru'11 we ~~ domain. Tasks Focusing lin Domain-Specifie
'\{CD;.OT} Performance. Tn. the context of the difficulties ofidentl~1r.g .. co ~~::: f"r-sk~ilia; can capture the expertise. it is easy to sec the attractiveness f "r.;d~.::_ eL:'. ::re"':~rm:m.:..:.It is possible to evaluate memory performance fc ~5.e1h-"'Ci iJ; i r;::;urtL; ::Jf:' r::~~ 0"recognition and reproduction of literal detail- .:.; oorr:::c'" - ....... ~ 0; cl;r."s ~ ~!'SL. u1:m:hdoes not involve any in-depth analysis the gne-:l tl mam, L.rge samples of different meaningful _ ~ .. ce exrracreo rrom a given do-nam even though no formal Ces..1! >-oc.di::;~ ~ -. Iation of S-;lL':Illl~ given. Similarly, it is relatively ~J ltD a$;S.~'e C!d~_e:s...'"'il;.:uj'\~.Y even me:umgn:,,- stimuli by recombining stimulus e C.ec:ltrs n at: arb rra~ or random mauner In a wide range of'different domains. experts have been shown to display superior memory performance tor representative stimuli from their domains of expertise \\ hen adaptations of Chase and Simon's ( I973) original procedure have been used: chess (for a review, S~~ Charness. 1(91); bridge (Chamess, 1979; Engle & Bukstel, 1978): go (Reitman, 1976); music notation (Sloboda, 1976); electronic circuit diagrams (Egan & Schwartz, 1979); computer programming (McKeithen, Reitman, Rueter, & Hirtle, 1981); dance, basketball, and field hockey (Allard & Starkes, 1991). Other studies have shewn superior retention of domainrelated intorrnarion as a function of the subject's amount o f knowl edge of the domain ..such as baseball (Chiesi. Spilich, & Voss. I97Y; Spilich, vesonder, Chiesi, & Voss, 1979; voss, Vesonder, & Spilich, 1980) or soccer (Morris, Gruneberg, Sykes, & Merrick, 1981; Morris, Tweedy, & Gruneberg, 1985), Hence, many studies have found evidence supporting a monotonic relation between recall performance for a domain and expertise III that domain. There arc, however, several lines of research that have questioned the generality of that relation. Sloboda (J 991) points out the striking similarity in accuracy and structure of recall ofprescnred melodies between musicians and non-musicians, which he attributes to shared extensive experience with music. Allard and Starkes (1991) show that superior recall of briefly presented game situations by elite players, as compared with intramural players, is not always found in sports with s. ~-crlstress, such as volleyball. finally, Patel and Groen (199 I) demonstrate that levels of :nec_::: expertise have nonmonotonic relations to the amounts of information recalled from preserced medical cases. which they attribute in part to the ability of experts to efficiently idee f.: ~ r-~onnarion relevant to the medical diagnosis. These findings show Lhatsuperior memory perrormance is nOT an inevitable consequence of "training expertise. Ii is ili~ "'.u~sti"rable that a collection of tasks to measure the superior memory of experts can he claimed to really capture the expertise ill question. With the exception of

-~~~~==~~~~--~~=-~----- --- -

C.HAPTFR F'Grt Il:El\

PROSP;::CTS A~D I "'ITr

01- IH.l:.E~1I'IRJCAL

srunv (lr I:.XPIoRTISE

407

experts on memory task's, superior performance bl experts in many domains does not include explicit tests of memory performance. ~Ion... "'m'cr taere is 110 reason to believe that experts explicitly train 'with the goal of increasing tne~met::'..:r: performance. it is therefore unlikely that their memory performance would have re_tlr;;c J. ~':l'e maximum. Wi.:. shall later discuss in lIIOTedetail the cognitive processes n::...ting~: rerformance and expertise. An Issue shared by studies of superior rnemon ~"'Ce and studies of superior performance ill other realms is the problem of de,crr.--"'" _ .-==:":us characteristics necessary LO evoke performance in the laboratory analogoes so r!!"_- -,:;:--ertise.

Finding the Appropriate Stimuli to Evoke Superior Performance,

Ia

level performance, one attempts to create a situanon that is m..,."'_: _ ~ -; ~ 3."d yet sufficiently similar to the real-life situation to allow the reproducrion...- ~ ~er:::. e _ der laboratory conditions. The mere demonstration that an expert-level pelf ~ reproduced under controlled laboratory conditions reveals something imporran; a:m:rl tze mechanisms underlying the corresponding expertise. It reduces the number of "",'s.i :X~ stimulus variables that are critical to performance, and it can also eliminate a number .systematic covariations that would make the real-life performance much easier than it would initially appear. Despite the critical importance of the process of :finding appropriate stimuli to evoke superior performance, that procc .ss has rarely been documented. Ericsson and Pol ;;011 (Iygga, 1988b) investigated the ability of expert waiters ami waitresses to match meal orders to customers. They reproduced under laboratory conditions the superior memory performance related to dinner orders by simulating actual CUSLOlllCTS wi lh pholos of faces. Similarly. Bennett (1983) reproduced the superior memory performance related to drink orders by cocktail waitresses in a simulated situation with dolls representing customers. Hence, highly schematic stimuli Me sufficient to elicit the perceptual and representational mechanisms that mediate superior memory performance. Similarly Chase and Simon 19-3) found that the memory performance of two chess exrer.s ti1.: ~~: :! -"'er; for che"""'~rds with real pieces and schematic ~iagrams or chess ':'" 5.. .. ~:! beginner 31 chess :.l:u\\~cl poorer recall """lUi ->e!l<elI:.Ztl.. c-__r~ __ IJ5e cl. 1)[

familiaruy \\ lh

-;.<!"

:?Jm

;:_m--

\'lo~"'el) ~ ~:v.:xr~~
---: -~n::r::=:;re was only ':-..4 -., tti~ performance in<;: performance with the real e eerrect research using different visual

type ofle~er;:_g:rao

_ es:

halfas good as his p..~

~ I

I ,-

with the unfamiLrrdu;-~ ~ board. Charness (1991 ~:.!c representations of c:n..,,, - , There is some evla~~ tll::_ tI:et-:: er e ~.b ill the extent to which stimuli can be a bstracted Gilhooly, Wood. KilD,."lI1", anti GIU..'ll ( 1988) demonstrated that the lack of superior memory performance by exper: map users, as compared with the novices studied try Thorndyke and Stasz t 19!5O. _ :1 d be attributed to their lise of schematic maps mamJ.:. used by tourists) as stimuli. B~ lId_ ing recall of both schematic maps and more ... .1 ~eC contour maps by expert and nonce map users, Gilhooly cl al. (1988) found, as expected seperior rncmory of contour maps b) the experts. hut no differences between expensand W-rlces for the commonly available schematic maps. The tact that superior pcrfr--mances can he reproduced ill the laboratory with schematic stimuli is important not only for practical purposes but also for theoretical analyses \)1' the mediating mechanisms.

408

SF.I.TlO1\ vm .t.XPERTISF.

The !SSUl::\ of how to design representative laboratory tasks are discussed in many chapters in this volume. For example, Patel and Groen (1991) consider the differences Q;;l"~T' medical diagnoses based OJ) written texts presenting medical cases and diagnoses basec on interview'S with real patients. Dorner and Scholkopf (chapter 9, this volume) report 0'"' the management of simulations of very complex systems.

Summary. The essential first step of the study of expert performance involves identifying a collection of standardized tasks that can capture the superior performance under controlled conditions. it is a llCCC'S3rycondition for further anal} "i ...;tat superior performance by experts be reliably "~11 for 61.: designed tasks. III co-nplex domains it is often especially difficult to iJcn:i.~ a pop.rlarion of tasks to capture the expertise: it may be possible to identify instead :! .. II~. -urmber of reprc ienranve tasks to ehcir superior performance. Nonetheless. : ma be useful to think of expertise in terms of a corresponding population of tasks. \.:!.._. ~,experts may. however, require different popula.i 1"'.5 or'rasks. [ ... ] Similar sp~~-icl.~ IS :0 be expected in mOSI complex domains. To capture specialized expertise 3(1_ ~~~ r- ..5 necessary to design special populations of tasks ~:,-O?na~e for a small gr: _p or exocns or even individual experts (case studies). Superior memoi y performance ~'.~_"Tli a :egitimatc subject tor study as long as we keep in IIU.::Cmal the processes ..;: ".:-yng the superior memory performance may only partially overlap with those that generauy underlie the superior performance of expcrrs. The fact that it is possible to reproduce expert performance in 3 laboratory Task has important theoretical impl ications, It reduces the significance of large numbers of factors that influence complex real-life situations. Furthermore, it indicates a .air degree of generalizability. especially concerning the detailed stimulus representation. Let us now turn to further analysis of'thc processes that mediate superior performance.

c~_

18.3.2 Analysisof Expert Performance: "he Second Step

After identifying collection of tasks that can capture the superior performance of experts, one call apply rhe full range of methods of analysis in cognitive psychology to examine the phenomena associated with a particular type of expertise. In the following sections we present a brief outline of the wide range of observations that can be made ro infer information about the processes mediating superior performance. We then discuss different research paradigms. such as comparisons of performance by experts and novices in a small number of tasks, and extended analysis of individual experts, Finally. we report on analyses of paracular types of superior performance, such as superior memory performance.

ca::..::..JI! .:::-e..:: -: observe

=t is clear that one mediating cognitive processes. but what can be observed co .!Z'~~ '\':m cogn+ve processes can be related to tbc underlying cognitive processes \\It" .., e iZImrnzricr.:-proce!-sirg theory of cogninon. figure I R.I shows a nurnher of different ~~ .: cservsrio-is that can be collected on all. eogrutrve process, At the top of rigor,-" cr>g me ;:rocessing is represented schematically a.'; a series of internal processing steps, ,,~:- :~ ~ --e mformation-p-ocessing theory of human cognition. These interne p;ocessJ!.; ~~ cannot, of course. be observed directly, but it is possible to speci Iy hypothese cou; the relations oer-veen .r e mrernal processing steps and
Perl rmanee Analysis: Methods of Inferring Mediating Processes.

CHAPTER FIClHllS

PROSPF.('T='

\N9 U\fITS Of TIlE F\fI'WC.-'\L

STUDY OF cXI'eRTISr:

409

Start of press-itatlon of lntormanon ;or specltlc task (Ie)

,ocleSSin~
SAp

<rocel

S Ar

SSin~> /

__

<

..
Reaction lime

= 11-

10

P~~:';G::';"~.:

OC~~.~ orIS

------- Ek----~
(Eye Ilxutlons during solutions)

MAmory tor tas infor rnauon nelrospActlvc rAport PostexpArlmental

----<0-

InlArview

I'IGURF.Uti

An overview of different types of

b:;,.er.:lt.lOIJSOO _ 1.-

:e

on a task, adapted from a figure in Ericsson and () rvee I

observable behavior. For example, when a subject tix~ t.:~ or ft:t" '=~ on a specm .. Item in a visually presented table nf information. we car; mf:r tha the corresconding internal steps involve processing that information. On tbc ha,_, , " '. enc.cal recall of the presented information after the task has been completed and cne presented information is no longer available to the subject, one can infer that that .nformation was processed during the completion of the task, In research on transcription typing, it is possible to determine what port of the text the typist is looking at and what parr otthe text is simultaneously being typed. TII(.;general finding is that the higher the skill level of the typist, the farther ahead in the text the typist looks during typing. Being able lo look ahead in the text appears to be crrac., to the superior typing speeds of expert typists, because when their freedom to look :ilield L'I experimentally restricted, their typing speeds arc reduced to levels appro-d. ... !> 1'1; -e
.:'.)~ novice typists [ ... J. it.ls possible to extend our analysis beyond the processing of pc~ Iilrlformn..:liun :;we c n i,jer one's access of preexisting knowledge and procedures . a a zask andrsis fite particular task should be performed before the d.;:.... _ 0Ii" .l .. s..:analy0;;.5 .nr..,,*.es specifying a number of different sequences of 1Mf."I; l::~:.rrg5".:rS LD.J.Icould gcua"L..! Ill:: l. ""1Xt answer for a specific task gh en t1h:, cx:t:: f;'"'"!I:':'XlS:_~~owledge. In \\eL:--.!l.:'~ ~ d "lai:1s. such as mental mulrip"i .!In" \ ing in logic, it is

410

SCTIO:-l vui

EXPERTISE

;:a.,,: to speci fy nearly exhaustively the different sequences of processing steps leading -" a correct answer in all efficient manner. In more complex domains, the a priori task l.:'.aIYS1S makes explicit the pool of hypothesized processing sequences that i..; being l.vd;, cered, On the h(t.~!s of the think-aloud verba.izaricns of subjects, one can determine only ilia tbc verbalized mrormation \\0115 accessed .. -\ ta... k analysis is cnncal for relating the verr-ahzed mformanor, \ ) .,..: underlying cogni+ e processes leading to its access or gCllcrntion t Ericsson & Simon Q 4). Analysis o'-'nmk-aloud verbalizat.or,s is iirnc-consurrung. and therefore researchers in expertise usi '_ 11.: c type." of data tend :0 collect data on mall) 5Jb. ccts for a "rna11number of tasks (t:" ...... -or \ ice cornpansons) or to C( llect data OT' indiv idual subjects tor a large number or ':!.~ , case studies).
rclativ el Kf' Comparisons. Comparisor ft! ink-aloud verbalizations by experts and ne ~,'-r,llOwn method (If a";sc'';I- .: d ~erences in the mediating processes as ~ U::_ ,J "'Jeers' levels of expense _..:b ects at different levels of expernse are n aloud while carrying out a srr _11 wl1~T of representatrve tasks. TIle number u.:.--mJ1) is not xufficicm for assessing sta e characteristics of individual subjects: _ 15 011 comparing the groups or subjects ee .deniify salient differences in regard to [!:!:OJ;lm;~bO'.\ ledge and processes . .. rypes ct differences found in a wid; "'m_", 0 f domains ofexpertise are remar kably consistent ,..ith !llI ,1: !)l'1gmallynoted by de Gr" 19-~1in :be domain of chess, Fxpert per- rr1~f' l.:"d '0 retrieve a solution method te _ ~:\l moves for a chess position) as part of the Immediate comprehension of the task. ,\ hcxa~ less experienced subjects have to construct 0 representation of the task deliberately JLd ~enernte a step-bv-stcp solution, as shown by research on physics problems (Anzai, 1991. Ct. . G~T_ & R(..'Cs,1982; Larkin, McUennott, Simon, & Simon, 1980, Simon & Simon, 19-' ,'_;ebra-"-ord problems (Hinsley, Hayes. & Simon, 1977). Vfedical experts generate ~l: diagnoses by studying the symptoms (forward reasoning), whereas less experienced medical ~lUdI!IlL" tend tel check: the correcrness of a diagnosis by inspecting relevant xyrnptorn- (i .~~ -rd reasorung) (Patel &. Groen. 1991)_ 011the same theme, expert performers bave a 000)' ofknowledge that 110t 01 I) i... more extensive than that for nonexpcrts but abo "IT''' C accessible (Feltovtch. Johnson, Moller, & Swanson, 1984: Johnson et al., 198_: ""-s;,. tJreene, Post. &.. Penner, :9 3). Whenever knowledge is relevant, experts appear to access II efficiently (Jeffries, Tumer, Polson, & ~m ood, 1981). TIle experts an: therefore ahlc to nonce incoastsrencres rapidly, and thus iccorsistenr hypotheses 'Ire rejected rapl.:4) .n favor of the correct diagnosis (Fcltovich ct al, I . -J: Johnson et al.. 1981). On presenratioa, informatics in the problem 15 integrated \\i the re evant domain knowledge (Pate] & Groen 19'-:0.1 1 St..--;';llaf characteristics of expert performance are ro:.:.n..1 ...cross different domains of expeiise. The -rcdies cited earlier suggest several in [Xlft.mt ... ba-acteristics that can be more cffccr '". ~ed In relation to task" parricularly de5in~:':; ehcit them in a more controlled manner. \\! shiU am .. der SJch research sbortl:. Exten"iH Case ~rudics of Singl!! ubjecrs.. cvnrrasr to the group stuuies uiseu~sd earlier, In \vhich s....,! m.mbers of~ ,',ere t:..d.o elicit the cognitive processes of experts, we shall briell} C"'I'l.:,lde:' ~'O e:umpl-e-s ::: :-near ...L \!lTort" thai have used detailed case studies in order to descnbe :he rogciLi\.: pmce ... e5 underlYl1lg superior performanl:c. F.xflert-'m

Oi.'U'!ER EIGllTEEN

l:'lWS?ECTS A!'D t.C(1T5 OF THt e.\1PTruCAL :'TUllY OF EXPERTISE

411

The first example draw' on several C'"'_~studies of calendar calculations. Calendar calculation is the rather astounding ability to llZ1r.C L'le day of the week on which a given date falls. For example, when asked on v;hat tla::; of!tt:~ -;oee.cAugust 5, 1934, felt. such a subject would he able to say, correctly. that i:\~:1S,;, .5:::u":"__ +ajor interest in this curious ability derives from the fact that several individuals . it -.:l :;bC ha\ c been severely mentally retarded, and little is known about how the abihr; e:.:le:~_ - ','35 acquired. Analysis of this performance is further complicated by the 10" .n.; ~ ....... ee ~':"e "..bjects. On the basis of a task analysis, where 110knowledge about calculation caa be lll.,~Ad for these mentally retarded subjects, one IS led to assume that the subjects must - e o._~:z..""d .pe information for all dales. Investigators have examined a fairly large number of indix till;z.. . -: ~ it)' of calendar calculation has been substantiated (for reviews. see Ericss :;:..)::: ~, and Howe &. Smith, 1988). \1m:it calendar calculators can demonstrate that at ~ .-: ~ a limited range of years. All such subjects examined have been unable TOexplain zc: . ~ know the correct answers. Some j nvcstigators, however, have been able to assess media '""':li; step.'> by analyzing these subjects' rnumblings prior to reporting an 311S\ver. Other investigators have been able to obtain informative retrospective reports on mediating steps. The most reasonable conclusion seems to be that the derailed structures of these subjects' processes dif .. fer from subject to subject and rely on a combination of memory for specific dates and some limited specialized calculation (J lowe & Smith, 19R8). The rare calendar calculators whose abilities extend from A.D. 0 to A.D. 999,999 appear to 'Use a version of known algorithms that can he mastered by a graduate student within a couple of weeks to reach a comparable level of performance (Addis & Parson, described in Ericsson & Faivre, 1%8). A second example of single-subject research to analyze expert performance draws on the many case studies of memory experts. Studies of expert memory performances an: par .. ticularly suited for the laboratory and can capitalize on the long tradition of experimental research 00 memo-y. J he same research tradition has pnmarily ased s ....,~. -'- . ""! :.--;: selected to b:: i1e.J.llI11g1ess, or (iL least ha.- rr.inm::1Zed the roie : ,~~ 10 ""t!<!:7'~ capture bas c ~~r:~n;,,~.....es.. I~. :.._ ......... ;!'\"!'I. cee'JldiI __ ,' i!I."L~ ~ .. .,.st _ ->_tx-nil[ meC10~ ~~....= v...=.....= a....:. rrx.r;)fJC.ttC \::_. iii.7:.~~ ~ ... ~es<ed that such exccpnrrva t:ld.. ..."Dd-.ds ~ .:n&:.v.:e: .:DJtA.: ~'".ir.ill. CE.ff:- ~or: systems (Luria, 1968; Wecnsler, I 5~ _:t .. e"Iqler.: - _ . pr-"-......:m_= 15 d.fficult even in rhc information-procc-stcg ::: J:;,:l(oo.. ~ ... DI ,., ~ c.;ossib\e to conduct an a priori task analysis specifying il:e ~ ~ prces.. ;. s:.."'Ps ;cd :be relex ant knowledge used to store information efficieruly l~ -~ -:. One of the rnetaods .:!'.:J.J.lab.-e is to use tbir'k ..aloud and retrospective verbal reports to identify the knowledge use ....~ ID'I i""~i,\.,dolalmemory expert and experimentally evaluate hypotheses about the mcdiazag role of that knowledge, For each individual expert it is i"""sible to hypothesize which stimuli 'Could and could not be successfully encoded usicg tl:.= uncovered mediating know ledg; qy comparing rnemor y performances tor comoarble end incompatible stimuli, it is possible :0 validate hypotheses about the mediating ,'JIO'T, ledge using the general method developed by Chase and Simon (1979). In a s;-,--:. cfa longdistance runner who acquired an excepnonal digit span through extended zra.n.ng, Chase and Ericsson (1981) found That the runner encoded sequences of'three di~it:. 51 ~) as familiar running times (5 minutes and 13 seconds in a mile race) whenever possible. When the runner was presented with experimentally prepared sequences of triplets of digits that COlild

412

~a:-IIOl\

vm

CXPF.R11~t::

not be encoded as running times (-kG would be 4 minutes and ~n seconds), his digit-span perforrnance was dramatically reduced. I!IIdfor prepared sequences oftriplets all of which could be encoded as running times, ~, performance W(lS reliably improved over his performance with random digit :-cqUC1lCe5 Similarly, Sloboda (1991) shows that superior mcmt r: performance for classical nUS1': =: .. .0.,; -avants is mediated by knowledge of that type of music and cannot be gen.:~""'lLlDr::~rn atonal music. Case studies of merr "": rr:'e7..5 J.:!\;! re- ealed that the knowledge used to encode the presented inforruatioo ~ rres ;-~tI~ I! ill I.: per. to expert. Similarly, the details of the acquired cognitive srrucn zes ......... '"1..-:al ~.lTesllu store information in retrieval form in long-term memory ~}. ":-_ Chaseacd Ericsson ( 1982: Ericsson, 1985), however. found three principles c.f, ~Or) tha: describec rt.e general characteristics of essentially all memory evperc- .,' ~.e been :.: '~ec:.a:l'CaE: studied. Studies wf ~ studies . ~t'lt ~~e'cts of Expert P\:w[ rmance, Up to this point
WI.;

have discussed

J.

~.:<$

p,::::::m......J

-~- rmance using tasks ~A"'eC.:....... zapture the essence of that performance. ocr tt..l: in many cases p3Iti:~;.Wt.!'O!:"'1 -i\'e activities associated with expertise

-e.i t..'h:rr could be more .:,1-;_'CO'\.e~ examined in tasks designed 1:0 focus on hj)~.c-acnx.:..:r eognnrve activities. For exarrp e, ic their study of experts in physics. Chi eca., IQ: _-cnsed on the initial encocmg o:pt:..- pcoblems to account for these experts' .... 'lli~'e\.hatcavauability of plans for complete s Imion.; [,J those problems. They asked experts ard novices to sort a large number of physics pre -!ems L." 11 categories of similar problems. Consistent with the hypothesis that experts" mcc..finF' would incorporate information about solution methods, the experts' categories -P'O)= ems reflected the physical principles underlying the solutions, whereas the nO\"iC6' cnegories were based on the situations and objects mentioned in the problem text. ln this case. the knowledge uncovered stands in close correspondence to the knowledge evoked during ibe solution of the physics problems. Severa I OTherinvestigators have used similar snrtil1g methods :0 assess the immediate cncodings of mathematical problems (Berger &. Wilde. ) '%I. 1981), as well as encodings of pictures of situations in team sports (Allard & Starkes. 1()91). Tt is, of course, possible to examine tbe knowledge of experts more generally. Tntheir study of representation of expert knowledge, OLson and Biolsi (1991) discuss a wide range of methods. Attempts to measure knowledge about chess directly with psychometric rests have been quite successful, and scores on these tests show a clear correlation with rated chess performance (Charness. :99 I ,. During a study of the selection of the best wove for an unfamil iar chess position, de Groot (1978) also found that the critical differences in cognitive processes relating I.() chess expertise occurred within the initial perception ofthe chess position. After a brief exposure all unfamiliar chess position, the chess masters could give very ieformarive verbal reports aooa: the perceived characteristics ofthe presented chess position, along with virtually peri;.. ~~ 11:1.... ': of the locations of all chess pieces. In subsequent research. superior memory performacce and superior perceptual performance of experts have been studied in specially designed ~4s. -\, reponed earlier, Chase and Simon (1973) accounted for the superior memory perforrnance 0:' chess masters in terms of their storage of chess positions in short-term rncrnory using complex independent chunks of chess pieces. The assumptions of storage i.n short-term memory and of independence of chunks have been seriously questioned by more recent investigators. Carefully designed studies of superior memory performance for chess

,HAPITR

EIGHTEl:N

PROSPECTS.'u'.'DLP,llTS OfWF

nll'UUl'AJ..

STUDY OF EXPERTISF

4J 3

positions, as reviewed by Charness ~r.aFe-:!.. ... : . showed thar chess experts store infermarion about chess positions in long-term ~_ ~ i solely ill short-term memory as Chase and Simon (1973) originally prvp- -cd, Subsequent researcher" have questioned (1J..:I.:> .;:r.C Simon's (1973) assumption that chunks of chess pieces were distinct and that a ;r., ea doc:M or. ece could therefore "dong to only a single chunk, Chi (1978) showed that occas~ ~ _ ess piece can belong to more than one chunk, a finding that suggests relations be~~.'l d:::;-.;3 from a given chess position, On the basis of retrospective verbal reports I .: :;Ct! ers and masters after brief exposures to chess positions, de Groot (1978) founc c !ax... .-~~cepnon of chess pieces in chunks, or complexes, as well as of encodings I'l.~ ; dI:::r:ks~'" '''e another to form a global encoding of the position. It appears necessary to a~.e tiUt:;L:b.:L _ ~ .ntcgrm:ing encodings account for the ability of chess experts to recall uCC'.:.rac:d, 11: Ij' ome brietiy presented chess position at a single trial (Frey & Adesman, 1976" In analyses of superior memory performance ill domains other than chess. ~ . ..'! ce of global inregranon of the presented information has also been found (Egan & Sch '7_ 1979; Reitman, 19;61 Snxiies ia other domains, however, have als revealed differences from the findings regarding chess experts, In domains with complex stimuli, such as medicine (Patel &. Groen, 1991) and computer programming (Adelson, 19l>4), it is clear t!tal part of the integration of the presented infoml:t.tiflTl;nvolves identification of the relevant and critical information, and any anarysis 07 scbsecuent recall must distinguish between relevant and irrelevant information. For ili~c.;-;:--t ' .. ~?Crtise, the processes of encoding prescntcd information will be quire 6~'~.Oepo!""._: O::~::' de-:...nUs of the particular type of expertise (Allard & Starkes. 199: E Je::-t:.:......c~....:: ~..;o: memory tor presented dance sequences, whereas :ki'l...d ' 1 <._::::r. ~ ~-. C!.et:!\.l u:e ~..;:...., "r me volleyball with superior speed. Supe-rior percepr.u pre .:;t::i_ '" bs:ills tt:.1re drr!:lO!!btr:!a .:J (IS C1function of chess expertise for tasks im-ohin-.,..:i.im:llp! e p.."::i.\.:plru;ln_~~.,...~ , ieal aspects otpresentcd chess positions {("h::!."71.. ~,- 10;)

~-=.;

General Comments on the Analysis of Expert ~ r_ Once the expert performance can be el ici leu hy a collcc.ion , ; t:!., b ~~::l:O~. tae full range of methods in cognitive science can be applied to .3.SSCS5 t.;.~ mem~tlng cognitive structures and processes. The mediating mechanism for an expen per ~)nnance should be stable and not much influenced by the additional experience - - e laboratory, as the laboratory experience will constitute only a minor fraction tee experts' total experiem:t: of task ill their domains. In fact. an absence of'further improvement during extended laboratory testing should provide a nice index fur evaluating our ahilir:-- to capture the mechanisms underlying the real-life expertise, On the basis of this argument, one immediately realizes some potential dangers I): studying aspects of "real" expert performance with tasks not encountered in the normal environments of the experrs.Jf we provide an expert with unfamiliar tasks, we n._",-J _.10sider the possibility that the expert may resort 10 nonoptimal and unstable straregres mat can be rapidly improved even during just a couple of sessions. With respect :__ eluOr) for briefly presented chess positions, Ericsson and Oliver (Ericsson & SIa.'>ZC" ,' .._ '~~9) found substantial improvement in the memory performance of a candidate chess master during a few months of'testing. They found no evidence of changes in the medicring processes, howeyer. only a marked speedup of the processes.

or

41 4

SEeTIO,",vru

EXPRRTlSE

We have been unable to find u uch evidence concerning the effects of extended testing of experts. Ericsson (1985) rcpo-ted sex eral instances of marked improvements ill the perf ..'Mlance of memory experts when they were observed on several test occas... ions, In sev\.~I cases the tests were scparared bv several years, find one cannot distinguish between the effects of testing and rhe impro. emen: due LO accumulated experience outside the labora:.... :y. Ericsson and Polson 19S~~) 1 t:J1Lcontinual improvements in their expert waiter's performance of their standard n.sk during about two years of weekly testing. It is likely that part of the observed speedup resuhed from the particular constraints of'tbe dinner orders studied. A more important d~J;;"!'C1in:l!r ofthe speedup, however, was The tact that the real-Life ta"k of memorizing LIr::r oecers v..L' not constrained by "peed, because the customers required more rime decIde ~ dinner selections than the waiter needed 1(.1 memorize them. Only in the la!:: S!O:!!lOO ",i'h preselected dinner orders diu the time required for memoriza . '" (;;'1" eal, II, ' -~ I::fft!'ri ces between rea -I :.: -rruations lind analogous laboratory tasks with r':~JX..1- ~ tmum speed nc me presented perceptual information arc likely aa f!'~ : e . 'V experts. til ring extended resting. But as lone <IS the p.'"'3tt:~ effects f.. e ~'\PC:I"_' remain comparatively small and the performance of the ~ t'~:llr:;, -eH~bl:) superior to those for 110\ ices even after extended practice, we ...._UTI that such d collection of rasks cal! successfully capture the superior expert t=-e:-:::... rmaJle.:. The effects of extended practice for novices will provide a 1TllJO: source of empirical evidence as we now turn to a review of theoretical accounts of how the superior performance of ex-pens can be acquired through extensive training
18.3.3 Accounting for Superior Performance hy Experts: The Third Step
In all the studies discussed earlier, the assessed mechanisms mediating superior performance implicated cogninve structures that '..ere specific tn the relevant task domains. The nature of the mediating cognitions allows U~ to infer that they reflect acquired knowledge and previous experiences in the domain. In order to account for those aspects of superior performance that arc acquired, il is critical to understand the role of knowledge acquisition and the important effects of practice and training tor their acquisition. Wheu we restrict ourselves to those task domains in which superior performance has been adequately captured. The empirical findings can be summarized relatively easily. The superior performance consists of faster response limes for the tasks in the domain. such as the supcrior.,~ of expert typists, pianist", and Morse code operators In addition, chess experts at.:;;..: 5"_;",";0:- ability to piau ahead while selecting a move (Charuesx, 19l11).IIIa wide range or nsk domains . experts have been found TO exhib.t superior memory pel formance. " 33.i: 1" acquired by experts? Superior performance it: different domains reflects processes end snc edge specific to the particular domain. The challenge is to account tor the \\ ides: .~-;e ; empirical phenomena \\ ith tne smallest of learning mechanisms and processe- .e<>p~..:\-:. e tor changes as a Iuncrion of long-term practice. Because it is not possible ro 005.:[\ ~ subiects dunng a decade Cl: .ntcnsive practice, most of the empire ical evidence is based Oil C' rrapolauon 0: changes in performance found as a result of practice at laboratory .a sks OVer mucn snorter Terms. A norher important constraint is that

CHAl'I'cR

EIGIlTIiE:-.I

PRO":PECTS ...... ''''0 LIMITS OF H-_EH1I'TRlCAT ~TLOY OF E.XPLRTlSF.

415

the proposed descriptions cannot pcsrr perrormarce capacities thar would v iolatc the known limits of human information process -ltg. In this section we shall consider \'a."10US zccccnts concerning the processes and knowledge that experts have acquired. \\e 5~1 fils:~. describe the Chase and Simon theory of expertise. Then we shall briefly re e -; ... e empirical evidence concerning speedup of performance, superior memory perro :a::~ superior ability to plan. with lite intent of pointing to issues requiring fur.her _._ _~aboranon.

i_~.,... ~-

The Chase and Simon Theory. Chase and Simon (19-3 _~~among masters, experts, and novices in a wide range of (i~~ ~ .;:OC to their immediate access to relevant knowledge. Chase and Simon's ; ~~~ e = _. - -:-..-etical account of'chess expertise provided an account of how the ma-sers rap~_ ~"'T move possibilities nom long-term memory. The recognized configurauoa.:oC.a.."'SS;:: eces (chunks) served as cue ...so elicit the best move possibilities, which had been stored ia or-tnJcy at an earlier time. ~e caess masters' richer vocabulary of chunks thus played a critical rok in the xtorage anc retr,e a: \ ~superior chess moves. Within the same i.':'..~o:etical framework, the speedup in selecting moves can be accounted for In terms or recoenition ; chess configurations and direct retrieval of know ledge about appropriate move -clecriorrs Siznlarly, Chase and Simon (1973) proposed that rbe superior memorv pcrfoznarce i.:r ::-'~7 prc-enred chess positions was due to recognition of familiar configurations of ..!'~ - ~::: tae masters. The near-perfect recall by the chess masters, involving 1"10~ 4-~ -e-,,: -55 .s assumed to be mediated by approximately seven chunks or conn;-~memory. Finally, with respect to planning. C'~ !:l- S whereby the experts' chess knowledge could be a:::-=c """ moves in the mind's eye. Given that no cv i.1..~ = -_._'-_'1..,' planning increased with a rise in the level 0:~ ~,-'ba:rn::S5... sider the acquisition of such a mechanism.

~=

Accounts Focusing on Practice and Leamfnz " 2 wide range of tasks, an improvement i.nperformance is a direct functioe c ", ~0:' practice, and [his relation can be remarkably accurately described by a pc ...er m;:.~:.O!1 (~cwel1 & Rosenbloom, 19~ I). This consistent relation between performcaee and practice has been given a theoretical account hy Newell and Rosenbloom Q'i; using a uniform mechanism of learning chunks, which they explicitly relate to C:U5e and Simons (1973) analysis of chess expertise. It 15 possible to describe skill acquisition in a- roader range of tasks ami domains ;., which the subject at the outset docs not have the prerequisite knowledge to produce errorfree performance. In systematizing a large body of data on the acquisition of ~", ?:,,_, (1964) proposed three different acquisition stages: The "cognitive stage" is characteazec oy an effort to understand the task ami irs demands and to learn to what information o:..e must attend, The "associative stage' involves making the cognitive processes ~~ allow rapid retrieval and perception of required in formation. During the "auumomons stage." performance is automatic, and conscious cognition is minimal. More recenny; \r.dersou (1')82) provided a theoretical model with three different learning mechanisms. each co-respcnding

410

SECTION VlU

F.Xl'l:XI1SE

to a stage of the Fitt model. Anderson was able to derive a power law for relating performance 10 the amount ofpractice, J: is dear that the learning -nechanisms that mediate increasing improvements from Jepe ...cd practice trial). must plr- important roles in the acquisition of expertise. Ltmay even be useful to consider such mechanisms with all eye TO identifying some limits to their applicability. First, It is important : <tsor:;::::'1 between practice and mere exposure or experience. Ir is well known that learn __ ft:q'-"''"C5 feedback in order to be effective. Hence, ill environments wiLhpoor or ever. ';::b1;ed feedback. kd"1ing may be slow or even nonexistent. Making predictions and J- "1."'C2S ' ror complex Cl'1\ ronmems .har are dynamically changing call present difficult inr-..:moo~..\_d('n problems, wioch DUy. at least in part, account for the poor perforrrarve - e:!.p:!rt consultants and deci-, on-makers Camerer & Johnson, IY91). Tn addition. !"::_ _ -n;..::; a tas], ,hes nc: !:::sU!".:,~ sacsec: ... eru performance wi II he improved. F e'\-er;.-.b .... vncrience anyone can cite ccuntle .... e: amples of individuals . 'I:'":: __ _ -::"" ~~ars to improve m spne of more man 10 year- of daily activThese ':!tJOll5 d ..~CTVCto be considered in more Ll.wi'."'ur we shall limit I:SS'3e ele .Jr to research on expertise: On the basis ot the foregoing con:... "'e particularly careful about accepting one's number of years of ~T. -=> ... a ccurate measure of one'S level of expertise. "'vnd the learning mechanisms discussed can account onlv for making Ih\! initial gmnve processes more efficient and ultimately automatic. In real-life perceptual mOIOI' skills, there eXISt a wide range of motor movements that can al ow realization of a given goal. There is good evidence from sports that the beginner 5 spontaneously adopted baseline strokes in tennis or basic strokes In swimming arc nonoptimal and that It is impos.;ihle to improve their efficiency by Iterative refinement, Lence, the first thing a coach will do when beginners start training is to have them relearn the.r basic strokes to achieve correct f011l1. Only then can the basic motor patterns be pel fL"t:.cdthrough fur ther training. 11i!> thus possible that the final performance levels IIId)' rcrecl differences in the initial representations used by different subjects. Thud, once we are willing to consider the effects that result from weeks. months, ami years of daily practice, it is likely that ".! ear-nor limit the consideration to purely cognitive effects on the central nervous xystem. Research on sports performance shows that extensive and intensive training is associated wi:..) fu'l range of change I> related to the blood supply and the efficiency of muscles (Ericsson ." Q<JJ I. Such changes will influence the speed of performance. It i:. possible that the correzaaons concerning speed of movements, as measured by maximum rate of tapping and speed of typewriting (Keele & Hawkins, 1982; Salrhouse, i9~ . 3.hnuJa oe considered not oul, d.' reflections of inherited characteristics but also as ~n.:r.:s of;,. '" motor system c111~g years of practice. fi.ri.a!h. acd ~ ... TT" .... .an~.these typc of learning mechanisms focus only on how performance car. Oe marle faster lJJd more efficient; they do not take into account the acqui:'1110"1 "': ;-...-V ogn -e sm... ~t.!:'-: ..... processes that are prerequisites for the unique ability of experts to pb aDd reason a lit problem situations.

Accounts Focus.ng on ~Icmo r y Functiouiug, The Chase-Simon hypothc ... is that the superior memory of tae expert reflects storage of more complex independent chunks in short-term memory has been seriously questioned, ami most ofthe empirical evidence also

rH \ PTh~ EIGH lEEI\

PROSPECTS A.>"-O LTMIT" OFn-rE EMPlRJCAl STUDY OF F.XFF.:Kl~E

417

suggests storage of interrelated informanoc III 'eng-term memory, as mentioned earlier. Even without the constraints of independer-ce of chunks and storage in a limited-capacity short-term memory, human information-pr It:t:',,il1b theory suggests a number of lirmts and processing constraints that. must be taken into CO::iS:.I;;!emriOD in any acceptable account. BuL let us lirst review some orthe empirical charactertsric "u-;ne superior memory of experts. Over a broad range of domains. experts ha- ~ 5:--;~ I" - _ r, 'I) restricted to intonnation in their domains of expertise. Furthermore. de Groce .,_ C~ and Simon (1973) found that chess skill among a small number of suhjecrs -;\~ ~..caU~ related TO their memory performance, which would suggest a high correlanon ];1.!t<,(;,.:."ID ...,;:"1 'evel and memory performance. Subsequent studies with representative sainpl, IIlt'>~ilil'; :..::qe- numbers of subjects found reliable correlations, but the strength of the assOC'I3.U;)!Il \1\;:1.:511[ .~....._;.; would have been expected from the Chase-Simon theory (Charness, 199L H....ll:r?- ~... .;; Although experts with decades of experience nearly always exhioi: ~~;;. errorIIIalice superior In that of subjects lacking expertise, there is at least one mtn,; ..... E ~ !::"rerexarnple: E\e:J though experts in mental calculation show far beuer memory performacce tor numbers than do n ,rmal subjects, their performance is far inferior to that of subjects who have practiced mC'lOf17:..ng digits over extended periods (Chase & Ericsson, 1982; Ericsson, 1985). Whereas me menea,.. ralculation experts rely predominantly on their vast rnathcmatreal knowledge of 11 urrbcr-; dIe trained subjects draw on a variety of knowledge essentially unrelated to rnathcrmncs .. Ii~ $.05< :u:npo~.m: difference between mental calculators and memory experts is that mental czk~ :q ''''L":'L''a.3 ana decades of practice to achieve memory performance cornparab' e "0 -::;(1- ~.~ oe :;..:.::;e-.,eC b:. normal subjects after 50-100 hours of practice in a memory task, Hen e. .:...:i pcss-: e th:n tl.: superior memory performance of experts has only a weak associ ..ti In v..LhIIh..-i:- nlr"''''Oi '-'il~. edge, Similarly, superior memory for" ner"y presemeo czess p~ can be traim .. xl, Ericsson and Harris (198~) found that after 5G no..:."S<):-;:ro=" ,~~;;n playing experience was able to recall chess posuions ......Ii ~ ~: ~.tU= ~ -; that of some chess masters; Tn similarity to the digir-spas ~~.::....:!' _.Ct5< e:w:~ lite mediating processes revealed that the subject's pertormanee ,,= ;m:U"dl.:J h~ perceptually salient configurations of chess pieces. w itbuul imp iM"!Ons for playing chess. Hence, it appears llial hy means of practice directed toward Improving memory of perforrnancc, subjects without expertise can, after a couple of mouths of daily practice: match or surpass the superior memory performance of experts. To account for the results concerning memory experts and long-term training studies, Chase and Ericsson (1981. 1982; Ericsson, 1985, 1988; Ericsson & Staszewski, 1989) proposed a skilled-memory theory to account for how memory performance can be improved within the known lirnitx of human information processing. Chase and Ericsson proposed that experts can develop skilled memory to rapidly store and retrieve information using' long-term memory for information in their domains of expertise. Building on the <llst!1:.:lion between a limited short-term memory and a vast long-term memory, this thevr sees lhe key problem to be selecnve access to information stored in long-term me- i)! S!..; .edroemory theory postulates that a.t the lime of encoding, experts acquire :! se: cr retrieval cues that are associated in a meaningful way with the information to be stored, At a later lime, :he desired information can be retrieved from long-term IUt:fIIUr) ~ <.Lilllg [he appropriate -ctricval cue. After extensive practice using a stable set of rerreul cues. with meaningful mfocmarion in the domain, one's speed of encoding and retrieval rs cssumed to approach that

=_t: ...::::-

418

SEL'1I0'lVlTI EXPERTISE

for short-term memory. The best empirical evidence regarding the structure ami operation of skilled memory comes from studies of subjects who achieved exceptional levels of perforrnance OD the digit-span task (Chase & Ericsson, 19RI, 1982: Staszewski, 1yg7). The retrieval cues used for rapid storage of meaningful encodings of three- and four-digit groups (up 10 a total of more than a hundred digits) can be used to access digits in presented matrices in a manner earlier believed to requite a raw visual image (ericsson & Chase, 1982). Studies of other types or expertise have given clear evidence tor retrieval cues indexing content. (c.g .. spcci fie intermediate products ill mental calculation) (Eric .sson & Staszewski, I Y~9;
Staszewski, I n~). The most direct t\ idencc -uggcsting the use of retrieval structures in chess comes from a series of studies with a candidate chess master by Ericsson and Oliver (uricsson 8::: Staszewski, I~~N They found that the chess master could read the description of the sequence of chess. _ i~ c garrc and mentally generate the sequence of intermediate chess positions _ - '" as fasr 1S be could play out similar chess games by actually moving the pieces on a chessboard, During the process of mentally playing out the chess games somemces Lh.:: IJ.nuld mtermpr him and test h.is ability to name [he piece on a :;i e ',"f..:::tre f - ~e current chess position, which he could do within a few seconds. In .~- \."I..pc..;:....."'n~$hIS speed of access to different types of information Tor a brietly preseared mrdcie-game chess position was examined. The chess master could name the piece located on a given square within a second, and within seconds he could rc.;port the number of his opponent's pieces that were attacking a given square, which suggests remarkable availability of many di [fcrent types of information about the presented chess position. Ericsson and Oliver (Ericsson & Staszewski, 1989) found ev idencc lilT rapid and flexible retrieval using a retrieval structure. This research misc." the possibility that acquisition of expert-level chess skill involves the development of skilled memory tor chess positions. Once it is accepted that mediating mechanisms are acquired, that raises a number of challenging issues. One can no longer assume that superior performance is automatically achieved merely as a function of practice. lhe history of expert memory performance provide" a number of cases III which individuals who have had extensive practice and experience have settled for suboptimal methods. Crutcher and Ericsson (Ericsson & Polson, I988b) found that several waiters and waitresses who on a daily basis memorized dinner orders relied on less effective encoding methods than did the expert waiter Jc. who exhibited vastly superior performance. Chase ami Ericsson (1981, 1982) documented extended problemsolving efforts by digit-span ex-perts to identify strategies and encoding methods to increase their digit-span performance, as well as similar efforts by other subjects, whose performance -~e-lInproved or did not improve beyond a certain level. When that evidence is considered together with studies of other memory experts (Ericsson, 1985. 19RR)past and present, it appears thal til" memory experts rely on the same limited set of mechanisms (Chase & Eric scr '9&2. Given thai most memory experts have not been instructed but have themselves disco' "ered Lhc structures necessary for their memory skills over extended periods, the importance of problem solving for their ultimate performance can hardly be overestimated. Similarly, studies of the development of a number of perceptual motor skills suggest the importance of cfu.;m c-ed methods and strategies for performing tasks such as juggling (Norman, 1976). There appears to exist a wealth of phenomena such that successful performance in the future cannot be predicted on the basis of current performance. Similarly,

CIL-\PTtR tIGIITIC:-I

PRmPF(TS .,.:-..UU\Ul:S or r i-iE EMPTRICAl Sn.;DY OF F.XPR

I!)_t

419

there is no reason to believe that such problem development of expert performance.

vlving is limited to the early stages in the

Accounts Focusing on the Ahilit_,. til Plan and Ressen, Analyses in several di ffcrent domains of expertise have revealed that experts en~~ ~ :. zianbcr of complex mental activities involving reasoning that relies OIl mental IT': .k ;;a::ri. !.=emal representations. The most frequently studied activity has been rhe p.1n==_; ~:: ~:. moves. Charness (19& I) round that lie depth to which a possible move sequence ._ a ;"'ess position was explored was c.osely related TO tbe level of chess skill, 2I '~i - c es rayers ai or below the level of chess experts. Menral planning and evalucr.cc of pess ~'.; move sequences place greater demands on memory as the depth increase , a -t ' , a~ "";_', e activity will be particularly tractable using acquired skilled memory to re,p.:es!'Ot d!ie-ss positions. As noLI;J earlier, de G:OOt (l978) found 110reliable differences ir regard to jc:;Jln of search among acvanced chess players with differing levels of chess ability. Holding (1985) suggested tha, the dj-=e~ces were too small to be detected, because of the small number of suhjccrs. Chamess I~S9. hoe ever. presented a case study suggesting that the depth of search may increase whh .:h5S ;,~I.: onlv .IP to some level of chess skill and then level off. One should also k:ql ill mind -...I.l C'l~ fHl.:"0': searching for a move for a middlegame chcs:o,poxition is no: .ies:g::el! tn i:".e3...q..-e me capacity to make deep searches and hence may well reflect pragmatic .: "lcriz oF. ,_ ~ ~ -. ~ of exploration to evaluate a prospective move. In support of the findings of :eiUolkl:b ~:m-!5. o.p -.: dc,,, ~sitions mentally, it is well known that chess i :lY'~-~ ~ .:t::o le\c i:""- ,..~ .. -J~o.ded with only a minor reduction in chess !:':!p.a(Holding. 19~5). In the absence of a SIDe;:uree limit to the depth to which a C1CS" ma ... tc- czt:: _ -_ 550':;' and C er (Ericsson &. Staszewski. 1989) found thai 3 C all the mformanon about a mentally generated _ '!5S pus'au rapidl . ...nd accurately, and they showed that the memory Tcprc'ocro.l1u.m r _ _'r:~" pcsirion was consistent with the characteristics of skilled-memory tneorj Caase J.: Ericsson, 1982; Ericsson & Staszewski, 1989). The IlCI!U 10 represent and integrate large al7'~otpresented information internally is common to a wide range of different types 0; expertise. Charness (1989) showed that expertise ar the game of bridge was closely :in.\"ed with the capacity III generate successful plans for playing thc cards in the optimum order. In medical diagnosis, the medical expert has to integrate many different pieces of mformarion that are not simultaneocsly available perceptually. The internal representation of the presented medical informaz; -;: must be sufficiently precise TO allow extensive reasoning and evaluanou of conscsrency, but also must be sufficiently flexible to allow reinterpretation as new informat.on becomes available (Lesgold et al., 1985; Patel &. Groen. 1991). Anzai (1991) reviews +e critical role of effective representations in solving physics problems and how metbods of generating such representations can be developed through practice. In order In account for CXPl;T' tise, it is essential to describe emerging skills for managing extended memory demands, as well as their efficient processing and manipulation.

420

SEC I ION VIII

l::Xl'ERT ISE

Comments on the Problem of Accounting tor Expert Performance. Chase and Simon (1973) may have been correct in their claim that access 10 aggregated pa.s i experience is the single most important factor accounting for the development of expertise. More recent research. however,shows that to describe the structure of expertise accurately, several other factors must he considered, ranging from acquired skill allowing for an extended working memory to increased physiological efficiency of tile motor system due to adaptation to intensive practice. We believe that rbe research on superior expert performance is benefited more by the development of a taxonomy of different types of mechanisms acquired through different types of learning and adaptation processes than by restrienng the definition of expertise to a specific l. 'pe of acquisition through learning.

SUMMARY
In this chapter -e ~tial:y contrasted the study of expertise with a number of other approachc... S:-:L.i_.lr.g outstanding and superior performance, and we Wo.!l}d that one distin~ltS; -~ t,:.. ...-;.rewas the claim that the superior performance wa predeminantly acquired. :Jrn-;'I. m; on the pioneering work on chess, we identified three nnponam steps In the study l)f expertise: first, identi fication of a collection of representative rash 'M:_' zaeans of which the superior performance of experts CUll he reproduced; second u'>,:.S[S of the cognitive processes mediating that performance, followed by design of e~lllt:rtlal tasks to elicit the critical aspects of such performance in a purer form: liurd. theorencal and empirical accounts of how the identified mechanisms can be acquired tar '-='~ training and practice. The most effective approach to organizing the results across di fferent domains of expertise is to propose a small number of learning mecaamsms that can account tor the development (If similar performance characteristics m J:'erent domains within the limill> of human information capabilities. There i" now 0- erwhelming empirical support for the tbeory of acquisition of skill with mechanisms "::n: to those originally proposed by Chase and Simon (1973). They proposed their aCL: :rot as "simply a rough first approximation" (p. 252), and it would therefore make sense to seek a fuller account, both looking for the conditions limiting those principles and . npplying other principles that can account for the complete range of performance capacities. "Nextwe looked at SOHle of those additional mechanisms. It would seem tl-ar one of the strengths of a generalized study of superior performances lies in a .rreful consideration of learning mechanisms and associated acquired characterisnes uncovered across different domains, \\e believe that both the excellcrv prospects aud the clear-cut lirnitarions of the esperase approach lie in its exacting methodological criteria, particularly the criterion tha; s.zpcricr performance should be demonstrated as well as captured by a collection of Iaborazo: .;. :35.>., 10 me extent that we are studying mechanisms and phenomena that have emerged ~ a result of imensrve preparation during years or decades, we can be certain that tens or lur-dreds of hours of laboratory testing are not likely to alter their structure seriously.This affords excellent opportunities to examine and to describe carefully the mechanisms mediating the observed superior performance. In this regard, the superior expert performance is a phenomenon that is particularly well suited for laboratory study and experimental analysis-

CllAl'TER lliGIITECN

PRO'!.I'I!(."TS.o>Jo-l) LIMITS OF7HF F.Ml'UUCAL STUDY Of F.XPF-{USl::

421

A major limitation of the approach IS tze ract Gal many types of expel tisc have not yet been adequately captured In some cases. the ::.:1.. of success in capturing the essence of an expertise is so well documented that [here rna, -xu oe a legitimate phenomenon :0 study. Perhaps the most important limitation concerns (he cilfficu::y of studying the development of supenor performance in real-life expertise. To t:OC..~ l"'e zaany factors underlying why some individuals attain the highest levels of'pcrfor-zazce It=e:-e3$ others do not, we need to broaden our approach. Indeed, in many cases we roa) \\~l: be t; r-ed to rely 011 correlational methods. As uur ability to describe the structurc-, of ..;.-:cent uf expert performance improves. we shall be able to focus IITI the csserrtal 3S?O=-.'" ~ ;:'d~ ee monitored in longitudinal studies. On the most general level, the srudy of ex pert performance !JfO\ i\b- J.- VI a range of capacities and associated character-sties rhat can be acquired. A .:~re-:UJ '. ~..::"-z-.' .1 of those should allow us til map out the porential fur human performance thet ~ through experience,

-:.?eS

ACK~O\~1 ,EDGMI:.]\'1 S
The thoughtful suggestions and conuuents on car.icr dram of th " chapter by Ral f Krampe, Natalie SachsEI icsson, Herbert Simon. ann Clemens T~b-Ri.iC1.:r arc grarerully acknowledged.

REFERENCF.S
Adelson. B. (1984). When novices SUrp3~"experts The difficult} of lite task U11o~ Increase with expertise Jour /f(I! vI Esperuuentul Psycholoev. 1.l'tv";"8 . Memor, unci Cognitio, 10. 4S3-4Y5. Albert, R. S, (198~) Family p(lsil OIl!> and me auan-ment of eminence In R. S ~Ihtrt [Ed, . (j~'tll..c and enunenre (pp. 111-15.1 . 0, ford Pergurncn l're~'. (Origil1al work published IQS).) Allard, r., & Rurnett.::-J (19851 Skill U:. ,~:-~ Car..;11iJm Journal ofPsychology, .19. 29+ .} Allard, r.. &. Starkes .. J L. (1091), ~Ioiur-" ,:"Q{:-' sports, dance. ami other domains. 111 K.. -\. & J. Smith (Eds.), Toward n General Throry (1 Expertise (pp, 126-152). '-Jew Yo.k. C;;mbnd;: Univcrstry I'rc3S. Anderson, J. R. (!Y~2). Acquisition of cognitive ~".1I. Psychological Review, 89, 369-40<1. AnZO',i. Y. (1991). Learning and use of represea.ations for physics expertise. In K. A. Ericssoi, &. J Smith (Eds.), Toward a General Theory of Espcrttse (pp. 64-92) .. New YMk: Cambridge University Press. Bachem .. \. 1193"'). Various types of absolute pitch. Journal uf th Acoustical Society o( America, 9.

BarouJ.

(1971)1. lntelligence and genera I suaiegies. In G. Underwood (Ed.), Straregics in informutron processing (pp. 403-450). 1ondo:Ap.-!r=c Pless. Bennett, H. L. (1983). Remcrnbcaag ~ .. or~t:~ ~ I11('mo~ skil' of cv..:l-......J ...:.~ H.;.~ Leam-

z.

=.~

B.001 n. S.

(ril J f 1985). Dcveioptng talent in young peoole. 'ewY'1rk: Ballantine Books. t!rJd). P. T. (.19~O).The genesis of absolute pitch, Journal ot the 4rmwirol SOCil!l.1' 0/.1 merica. 48. 883-887 Camerer, C. F., & Johnson, E. 1. (1991). 1u K. A. Ericsson & J. Smith (Eds.), Toward II General The......... of Expertise (pp. 195-'-17). New Yorl. C...wbric::""
Unrvcrsiry Press. Carroll. J. B. (19ili). How shall we stud} iuh.iJwI diffcrences in cognitive abilities? ~1",,':.~,,-,,;cgic;al and
theoretical perspec;v~ !n;eJ:.;<: ...~ :.8--1 is. Cutlell, R. B. (1%3). The personalir; and n.ot.vatiou of the

mg -

146-151.

researcher from measure-neces of contempora ries