1

Technical Papers 

Deductive and Inductive Ways of Investigation on Human Problem Solving

Kazuhisa Miwa3
3 Graduate School of Human Informatics, Nagoya University, Nagoya 464-8601, Japan.

Received MM DD, 19YY.

Keywords: problem solving, production system, cognitive science, computer simulation, protocol analysis Summary
In this paper, we discuss the relationship between the computational approach and the psychological approach for investigating human problem solving. We call the former approach Model Oriented Approach (MOA), and the latter Behavior Oriented Approach (BOA). We compare both approaches based on the framework of production system models. First, we demonstrate MOA through constructions of simple models, where we try to match the rule modications and additions with certain representative aspects of the individual dierences of each subject. Based on this correspondence, we can produce some simulations of typical behaviors. Second, we also demonstrate BOA through constructions of well tting models, where initially we have to focus on the t between the model's behavior and the human behavior. After obtaining satisfactory correspondence then we move on to analysis of rule modications and additions needed for the t. Last, we dene MOA as a deductive way of investigation, and BOA as an inductive way of investigation, both of which provide us with strong methodologies for exploring human minds.

1.

Introduction

In the past three decades the methodology for studying cognitive science has been well developed and established. In thinking and reasoning studies, many researchers have focused on constructing process models that explain human problem solving. Every study has its own specic method for constructing such models. We can classify the methods into the following two basic categories: the psychological approach and the computational approach. In the psychological approach, one of the best known methods for studying thinking and reasoning is the think aloud method [Ericsson 80], in which researchers use a coding schema as a model of human thinking processes. As subjects solve a problem, they are asked to verbalize their thoughts. The researchers then label each segment of the subNov. 1999

jects' problem solving process based on the schema. We have learned much about human thinking processes through, for example, analyzing the occurrence of each segment and focusing on interesting patterns of a series of segments. The other approach is the computational approach. In this approach, a model of human problem solving is represented as a computer program. For process models, one of the most popular and successful frameworks is the production system architecture [Anderson 93, Newell 91, Neches 87]. The set of production rules constituting a computer program is considered successful if it produces| through computer simulations|the same behavior as humans. One program can only represent a prototype model of many human subjects. Individual dierences can be accounted for by modifying the models, for example, by removing some rules or adding new ones [Simon 75, Simon 78, Young 81].
1

Deductive and Inductive Ways of Investigation on Human Problem Solving

2 Both the psychological and the computational approach have their own advantages in investigating the human mind. However, since the two approaches are strongly related; one compensates for the other, a well-designed combination of the two approaches enables us to understand the human mind better than either approach alone. Actually processes of solving a single task that provided a common opportunity for discussing a certain research topic was often analyzed based on both two approaches: for example, Tower of Hanoi problem [Simon 75, Anzai 79], the 2-4-6 task [Klayman 89, Miwa 96], nding the Kepler's third law [Qin 90, Langley 87]. In this paper, we would like to consider the relationship between these two basically dierent, but mutually compensating, approaches to studying cognitive science. Brie y we will summarize the contents of this paper. In this paper, we use a model representation based on production systems for the comparison of the two approaches; the representation will be introduced in 2. For demonstrating and verifying our discussions, we use the cryptarithmetic task and construct computational models to solve it. We chose this task because it has already been analyzed in detail|and we can use that analysis for comparison|and because its complete protocol data are readily available [Newell 72, Waterman 71]. The computational approach is called Model Oriented Approach, on the other hand, the psychological approach is called Behavior Oriented Approach. In sections 3 and 4, we demonstrate an analysis of the Model and Behavior Oriented Approach based on the production system model representation introduced in section 2. Finally in section 5, we compare the two types of analysis, and discuss the relationship between the two approaches as methodologies for exploring the thinking and reasoning processes of humans.
2. Model Representation
Production system for Subject 1 = Primary rules Parameters for Subject 1 + Additional rules for Subject 1

Production system for Subject 2 ... Production system for Subject n

Primary rules

Parameters for Subject 2 ...

Additional rules for Subject 2 ... Additional rules for Subject n

Primary rules

Parameters for Subject n

Fig. 1

Basic model representation.

21 Basic Model Representation

First, we will introduce our model representation

2

using the framework of production system models (see Figure 1). In this representation, the production systems for Subject 1, Subject 2, through Subject n, consist of two kinds of rule sets: primary production rules and additional production rules. Rules in the primary set may have parameters, each of which can take on either a default value or another specic value. These parameters may appear in the condition or the action clause of the primary rules, prescribing the ring conditions or actions of the rules. An important feature of this representation is the ability to divide each production system into three subsystems based on the diagram shown in Figure 1. We can distinguish a common part from an individual part of the systems. The individual part can be separated further into two categories: a rule modication part and a rule addition part. The dierence between the two individual parts is that the rule modication part is based on the primary production rules (that is, modications of the primary rules by parameter changes); whereas the rule addition part requires additional rules for the explanation. The three parts are described as follows:  Common part: This part consists of the rings of the primary rules, using the default values of the parameters.  Rule modication part: This part consists of the ring of the primary rules, using nondefault values of the parameters.  Rule addition part: This part consists of the rings of the additional production rules. In the description above, note the dierence between rules themselves and rings of the rules. We associate rule rings, not rules themselves, with each of the three parts. Based on the three types of
Vol.14 No.6

3 rule rings, we can separate the systems' behavior into three categories: behavior using common skills of the systems, behavior involving slight modications of the common skills, and behavior involving completely dierent procedures. Our representation is a computational simulation model which consists of three data sets for the primary production rules, their parameters, and the additional production rules. When executing simulations, the set of primary rules is xed for all subjects. The latter two data sets, namely storing the parameter settings and the additional rules, can be independently varied for each simulation.
22 Cryptarithmetic Task
Test Digit Assignment AN begin SC Select Column end TD Find Column FC Test Digit PC Process Column TD Rejection RJ Coordination CD

Initial Stage of An Episode

Basic Cycle

Fig. 2

Goal transition pattern within an episode.

Next, we will construct a model for solving the cryptarithmetic task. The following is an example problem.
DONALD +GERALD ROBERT D=5

The problem is to assign digits (0, 1, 2,...,9) to the letters, so that when the letters are replaced by their corresponding digits, the sum is satised. Here the information D=5 is given in the initial statement of the problem.
23 Summary of Model

The behavior of subjects in this task can be organized as episodes. Each episode consists of a sequence of inferences. An episode starts either with assigning one of the digits to one of the letters, or with selecting a column to be processed. For example, for all subjects the rst episode begins with an assignment, in which the digit 5 is assigned to the letter D. (This assignment is a special case because the information D=5 is given by the experimenter. ) The experimenter does not give the subjects any further information, so they try to test assignments systematically or arbitrarily, which sometimes produces trial and error behavior. Another way an episode can begin is with selecting a column. For example, the fth column, O+E=O, can be worked on independently without the other columns. Subjects often select this column in the initial stage of an episode,
Nov. 1999

and can directly infer from it E=0 or E=9, without any arbitrary assignments or other information obtained from previous episodes. Either type of episode continues until all obvious information has been inferred based on the assignment of digits to letters or the selection and examination of a column. After the episode ends, the next episode begins with another assignment or column selection. Figure 2 shows the goal transition pattern within an episode. As noted earlier, an episode starts with an Assignment (AN) or a Select Column (SC) operation. Then, some basic cycles follow. Each basic cycle processes one column, and generates information about the letters and carries associated with that column. For example, in the representative case, the inferences in the rst episode develop as follows: (1) D=5, (2) T=0 & c2=1 (the carry into the second column equals 1), (3) R=odd, (4) R>5 & G<5, and (5) R=7 or R=9. At rst, D=5 is initially obtained by the assignment in the initial stage. Then three basic cycles follow; in the rst cycle, T=0 & c2=1 are obtained by processing the rst column, in the second cycle R=odd is obtained from the second column, and in the third cycle R>5 & G<5 are obtained from the sixth column, then R=7 or R=9 is inferred from R=odd and R>5. Each basic cycle starts with the Find Column (FC) operation, which selects a column from columns on the column stack. The column stack stores the columns that will be processed in basic cycles of that episode. For example, in the three basic cycles of the rst episode, the rst, second, and sixth columns are selected one after another. The selected column is processed by the following Process Column (PC) operation. In this example,
3

Deductive and Inductive Ways of Investigation on Human Problem Solving

4 in the three basic cycles, T=0 & c2=1, R=odd, and R>5 & G<5 are obtained by three PC operations. Then in the next Test Digit (TD) operation, information produced by the PC operation is checked with other information stored in memory. If there is a contradiction between two pieces of information, control is moved to the Rejection (RJ) operation, in which information responsible for the contradiction is rejected. If it's possible that new inferences may be derived from the checking, control is moved to the Coordination (CD) operation. For example, when R>5 is inferred in the third cycle, R=odd was already obtained in the second cycle; so the new information R=7 or R=9 can be inferred by this CD operation. Through these TD and CD operations, some additional columns may be deposited into the column stack. The next basic cycle starts, in which the FC operation selects a column from the columns stored on the column stack. When there are no columns on the column stack, the episode ends.
24 Primary Production Rules and Parameters

the column stack, and this column is processed in the following PC (process column) operation. Depending upon an appropriate value of t4 , rule R2 can pick up any column on the column stack and send it to the Current Column, but cannot deposit any columns that are not on the stack.
3. Model Oriented Approach

In this section, we will demonstrate the Model Oriented Approach.

31 Examples of simple models

A set of primary production rules consists of twenty-two rules: R1 to R22. Some of them have parameters that prescribe ring conditions or control the actions of the rules. There are nine parameters: t1 to t9 . For example, Rule R2 is the following. This rule controls column deposits in the FC (nd column) operation. Goal = FC, There are columns on Column stack ==> Deposit one of the columns on Column stack into Current Column, Delete the deposited column from Column stack, Set Goal to PC. This rule involves a parameter, t4 . This parameter can take on one of four values: mostconstrained, right, left, and (particular n). The default value is most-constrained. With the default value, R2 deposits the most constrained column on
4

 Rule R2

We can produce a wide variety of models based on our model representation by modifying, in a well described sense, the most basic model that consists of the common part only. We can use one of two strategies: we can construct models as simple as possible, or we can construct well tting models, in which well tting means that the models trace as closely as possible the solution paths of subjects. First we would like to describe some model constructions based on the simple model strategy. Simple models are constructed based on the Model Oriented Approach. The relation between the simple model strategy and the Model Oriented Approach will be described in detail in section 5. In our model representation, simplicity can be measured (1) by the ratio of ring of individual productions to ring of total productions, or (2) by the consistency of ring of individual productions. First, in order to reduce the ratio of individual parts, we permit only a small number of parameter changes (rule modications), and few or no additions of secondary production rules (rule additions). Second, in order ensure consistency, we control the parameters systematically, so that some parameters are consistently assigned to one of the non-default values and the others are assigned to the default values. We provide eight examples of the simple models below. Figure 3 shows the behavior of each model. Model S1 consists of the common part only. No secondary rules are added, and all values of all parameters are default. The behavior does not inVol.14 No.6

 Model

S1: The most basic version:

5
PC E1 D=5 PC PC T=0 c2=1 R=odd PC G<5 R>5 CDa R=7or9 E1 A=4 PC L=8 c3=1 c2=1 c4=0 D=5 PC T=0 PC G<5 c2=1 R>5 c5=1 PC A=4 PC c3=1 c6=1 c4=0 N>3 B<7 CDe N=6 B=2or3 PC L>5 PC G<4 R>5

E2

select column

CDc CDb PC PC G=1 PC c5=1 E=9 R=7 c7=0 c6=1 CDe PC PC N=6 B=3 N>3 B=2or3 c5=1 B<7 E=0or9

E2

CDc PC select PC E=0or9 E=9 column G=1 PC R=7 PC L=8 c2=1 c7=0 PC

E3

B=3 c5=1

E3

O=2

(a) Model S1: The most basic model.

PC E1 D=5 G<5 R>5 PC T=0 c2=1 PC R=odd CDa R=7or9 PC PC

E4

O=2

(f) Model S6: No parity problem space.

E1 D=5 PC T=0 c2=1 PC PC R=odd c5=1 c6=1 E7 E8 L=7 L=8 PC R=5 c3=1 Reject E7

E2

CDc CDb PC PC PC PC select c5=1 G=1or2 E=9 E=0or9 R=7 column c6=1 c7=0 CDe PC PC PC A=4 PC N>3 CDe N=6or8 B=odd B=3 c3=1 B<7 c4=0 O=2

G=1 c7=0 N=6 c5=1

N>3 B<7 L=8 c2=1

E2

E3

E3

CDc PC select PC E=0or9 E=9 column PC A=4 PC G=odd c3=1 c4=0 PC R=3 L=1 Reject E3 c3=0 L=2 PC R=5 c3=0 PC R=7 c3=0 PC Reject E4

PC G=1 R=7 c7=0 c3=1 Reject E9

E9

N=2

PC B=9 c5=0 PC B=0 c5=1

E10

N=3

Reject E10

(b) Model S2: Controlling FC operation (left to right).

E4 PC E1 D=5 PC T=0 c2=1 R=odd PC G<5 R>5 CDa R=7or9 E5 L=3

E11 Reject E5

N=6

PC B=3 c5=1

E12

O=2

E2

CDc CDb PC PC A=4 PC PC c5=1 select PC L=8 L=3or8 E=9 E=0or9 R=7 c3=1 column c6=1 c2=1 c2=1 c4=0 PC PC G=1 PC N>3 CDe B=3 N=6 c7=0 B<7 O=2

E6

L=6

PC G=7 R=3 c7=1 c3=1 Reject E6

(g) Model S7: No inequality problem space.

E3

(c) Model S3: Controlling FC operation (right to left).

E1 E2 E3 E4 E5 E6 D=5 PC T=0 c2=1 PC PC R=odd G<5 R>5 CDa R=7or9

E1

D=5

E2 E3 E4 E5

R=7 L=3 G=1 A=2 N=6 N=8 N=9 O=8 O=nil

PC G=1or2PC L=3or8 R=7 c2=1 c7=0 c2=1 PC E=even c3=0 CDc select PC E=0or9 E=9 column L=3 PC c2=1 PC E=odd L=8 c3=1 PC CDc select PC E=0or9 E=9 column CDe N=6 PC B=3 B=2or3 c5=1 O=2 c5=1 PC c6=1 G=1 PC A=4 PC c3=1 c7=0 c4=0 B<7 N>3 PC

reject E3, E4

E6 E7 E8 E9 E10

PC T=0 PC PCG<5 R=odd R>5 CDa c2=1 R=7or9 PC G=1or2 PCL=3or8 c7=0 c2=1 PC c2=1 PC E=even c3=0 PC c6=1 c7=0 PC E=4 c4=0 PC B=3 Reject E6 c5=1 PC B=5 Reject E7 c5=1 PC B=6 c5=1 Reject E9 Reject E5

E21 E22 E23 E24

N=2 PC B=0 c5=1 N=4 PC B=2 c5=1 O=9 O=nil Reject E23 Reject E22 Reject E24

Reject E21

N=9

PC

B=7 c5=1 E=8 c4=1

E25 E26 E27 E28 E29 E30 E31

N=nil N=2 N=4 O=6 O=nil O=2 O=nil PC PC

Reject E20 B=0 c5=1 B=2 c5=1 Reject E28 Reject E27 Reject E30 Reject E29

A=9

PC

Reject E26

N=6

PC

B=4 c5=1 Reject E25

E7

(d) Model S4: Assign if two alternatives.

E1 E2 D=5 PC T=0 c2=1 PC PC CDa G<5 R=7or9 R=odd R>5 E=even

E11 E12 E13

Reject E8 N=nil E=8 A=4 PC c4=0 B=9 N=2 PC c5=1 O=6 O=nil Reject E12 Reject E11 Reject E13

N=nil G=2

A=nil

Reject E4

...
N=6 PC B=3 c5=1

PC c6=0 c7=0 Reject Reject N=nil E44 E47 Reject G=nil Reject A=nil E3 E31 PC c2=1 PC L=8 E=odd c3=1 PC c6=1 G=1 c7=0 PC E=5 Reject E51 A=2 c4=0 PC E=7 A=3 Reject E52 c4=0 PC E=9 A=4 c4=0 PC B=9 Reject E54 N=2 c5=1 PC B=0 Reject E55 N=3 c5=1 PC B=3 N=6 c5=1 O=2

E49 O=nil

PC G=1or2PC L=3 PC R=7 c3=0 c7=0 CDc select PC E=0or9 E=9 column PC L=4 R=9 c2=1 c3=0 R=7

E14 reject E2, E3 c6=1 c7=0 PC E=9 Reject E4 E15 E16

N=9 O=2 O=nil

PC B=6 c5=1 Reject E15 E50 Reject E51 E10 E52 E53 E54 E55 E56 E57

E3

E4

PC G=3or4 CDc PC G=3 c7=0

E5

PC G=1or2PC L=3or8 CDd PC c2=1 PC L=8 E=odd c3=1 c2=1 c7=0

E17 E18 E19 E20

N=nil Reject E14 PC E=2 A=6 c4=1 B=2 N=4 PC Reject E17 c5=1 N=8 PC B=6 c5=1 N=9 PC B=7 c5=1 N=nil Reject E16 Reject E18 Reject E19

E6

E7

CDc PC c5=1 PC PC A=4 PC N>3 select PC G=1 E=0or9 E=9 c3=1 column B<7 c6=1 c7=0 c4=0 CDe N=6 PC B=3 B=2or3 c5=1 O=2

PC E=6 A=8 c4=1

(e) Model S5: c3=0 error.

Fig. 3

(h) Model S8: No processing of fifth column.

Behavior of simple models

volve any trial and error processes nor any drawn

Nov. 1999

out searches in the solution path. The most basic

5

Deductive and Inductive Ways of Investigation on Human Problem Solving

6 model deserves, in some sense, to be regarded as the most complete model.

 Model

Model S2 processes the columns from left to right, whereas Model S3 processes from right to left. These processes are governed by the consistent value change of parameter t4 , which controls the Find Column operation indicated in section 2.4. If two alternatives for an assignment are obtained, then one of them is assigned (an AN operation is applied), though in the default situation the model begins with the SC operation. At the three points, the beginning of Episode 2, Episode 3, and Episode 5, the values of parameter t1 |determining how to begin each episode|are changed from the default value. In the former four models, S1 to S4, no secondary rules were added to the primary production rules. That is, the rule addition part of the models was empty. Model S5 has one secondary production rule, whose ring corresponds to the rule addition part. The additional rule produces an erroneous inference. It processes the second column, and infers L=3 and c3=0 from L+L=R in the second column and R=7 (see E2 of the behavior). The correct conclusion should be L=3 or L=8 and c2=1. This type of erroneous inference is one of the most typical bugs observed in the behavior of subjects.
 Model  Model  Model

S2 and S3: Controlling the Find Column operation:

S4: Assign if two alternatives:

S5: c3=0 error:

In our model, three types of information: digit, parity, and inequality are available. If one of them is absent from a subject's problem space, to what degree is the performance reduced by its absence? To simulate this situation, we consistently change two parameters, t5 and t6 , making it impossible for the model to process parity and inequality information in the PC and the TD operation. As a result, Model S6 does not have a parity problem space, and Model S7 does not have an inequality space. As noted in section 2.3, the fth column, O+E=O, is very important for solving the prob6

S6 and S7: Restricting the problem space:

lem. If this fth column can be processed correctly, E=0 or E=9 is obtained without any other information. Like the two previous models, S6 and S7, this model, S8 can not process the fth column by changing the t5 and t6 parameters. In addition to the parameter change, two secondary rules are added to the primary rules. Both rules provide additional mechanisms for rejection operation. Each rule red nine times and twenty times respectively. Based on the solution paths presented by these simple models, we can obtain interesting information about the relation between individual dierences in mental processing and behaviors. For example, the lack of the inequality expression in a subject's problem space in uences the solution path more than the lack of the parity expression does. Moreover, the ability to process the fth column is decisively important. The absence of this ability not only makes the solution path articially longer, but, in order to solve the problem completely, it also requires two additional mechanisms (rules) for systematically rejecting episodes that are not needed when the ability is available. In spite of the simplicity of these models, we can obtain a wide variety of behavior from them. This suggests that small or consistent dierences in mental procedures sometimes produce large dierences in behavior. Thus, widely variable human behavior, which seems not to have any commonality at rst glance, may be explained by only a few basic individual dierences in mental processing.
32 Di erences between Rule Modi cations and Rule Additions

 Model S8:

No processing of fth column:

In our model representation, two essentially different individual dierences are divided into two categories: rule modications and rule additions. The Model Oriented Approach enables us to understand the distinction more clearly. Reconsider one of the primary rules R2, and a parameter t4 shown in section 2.4. Rule R2 selects a column from the column stack in the FC (nd column) operation, and parameter t4 controls which column gets deposited. Through parameter changes, the model can pick up other columns on the column stack and deposit them on the current column, but
Vol.14 No.6

7 it cannot deposit any columns that do not exist on the stack. We can create two kinds of models: models in which only uctuations within the rule modications are permitted, and models in which uctuations extending over the rule additions, beyond the rule modications, are also allowed. These two situations can be produced by replacing R2 with either of the following rules, R2' or R2".
Fig. 4
160

Total Number of Rule Firings

140

: Rule modifications : Rule additions

120

100

80

60 20 0 20 40 60 80 100 120

Percentage of random choices of column

Goal = FC, There are columns on Column stack ==> Deposit one of the columns on Column stack randomly into Current Column, Delete the deposited column from Column stack, Set Goal to PC.
 Rule R2"

 Rule R2'

Increase in average length of solution paths with rule modications and with rule additions.

Goal = FC, There are columns on Column stack ==> Deposit any column (1 to 6) randomly into Current Column, Delete the deposited column from Column stack (if the column is on Column Stack), Set Goal to PC. The point is that with R2", the mechanism of the column stack, one of the most important concepts characterizing our model, disappears because the column deposits are not restricted by the column stack. We can examine how each type of uctuation affects our model's behavior. We now introduce a percentage of randomness from 0% through 100%. For example, in using rule R2', when the ratio is 25%, in one-fourth of the occurrences one of the columns on the column stack is selected at random and deposited into the current column, and in the remaining cases the most constrained column is deposited. Thus, 0% randomness means that no individual parts appear when the most constrained column is always selected. Figure 4 shows to what degree the randomness within each individual part aects the length of solution paths. The horizontal axis indicates the
Nov. 1999

percentage of random choices and the vertical axis indicates the average number of rule rings, corresponding to the length of the solution path, averaged over 50 trials. Figure 4 shows that as the randomness increases, uctuations caused by the rule additions make the solution paths much longer. On the other hand, the rule modications do not seriously increase the length of solution paths even when the degree of randomness approaches 100%. We can nd not only these quantitative differences, but also qualitative dierences. When only rule modications were permitted, the solution paths were very similar to each other. For example, each of the solution paths obtained through the three hundred simulations (6 randomness levels * 50 trials) consisted of three episodes, E1, E2, and E3, just as the solution path of the three basic models, S1, S2, and S3 did (see Figure. 3). On the other hand, when rule additions were also permitted, substantial dierences in the solution paths were observed. In twenty of the three hundred simulations, the two initial episodes, E1 and E2, were combined and, in all, only two episodes appeared through the solution paths. In twenty-two cases, trial and error behavior appeared when the solution paths consisted of ve or eight episodes. In only two cases, the model could not obtain the correct solution when the solution was D=5, T=0, E=9, A=4, L=2, R=7, G=1, B=3, N=6, and O=8 (the italic means incorrect assignments). The Model Oriented Approach shows that the eect of rule additions on behavior is often quantitatively and qualitatively dierent from the eect of rule modications.
7

Deductive and Inductive Ways of Investigation on Human Problem Solving

8
PC E1 E2 E3 E4 E5 E6 E7 E8 E9 E10 E11 E12 E13 E14 E15 E16 D=5 L=1 PC CDc PC R=1or3 T=0 G=even R=odd or7or9 c2=1 PC R=3 PC c7=1 reject c3=0 E2 reject G (B1-B39) (B40-B57) (B58-B60) (B61-B77) (B78-B90) PC c5=0 PC N<3 c6=0 A=4 reject c3=1 E, c5, c6 c4=0 PC PC G=3or4 c5=1 c7=0 reject E9 A=5 c3=0 c4=1 reject E6 (B91-B143) (B144-B168) E=9 c6=1 reject E8 (B169-B203) except R=9 (B204-B209) (B210-B211) (B212-B216) (B217-B224) (B225-B238) reject PC L, c2, c3 L>5 in E5 (B239-B267) (B268)

select PC R>5 CDa R=7or9 column PC L=3 PC G=1or2 R=7 c2=1 c7=0 c3=0 select PC E=0or9 column PC PC CDa select E=even E=0 column PC PC PC G=1 c5=1 E=9 c7=0 c6=1 PC L=4 PC hold E4- R=9 c2=1 E7 c3=0 PC E=0 PC A=5 c5=0 c3=0 c6=0 c4=1 PC c3=1 L>5 L=6 PC R=3 reject E10, E11

Next, we demonstrate the analysis of subjects' mental procedures based on the Behavior Oriented Approach, using the well tting models. We choose (B301) E17 the PC (process column) operation for the analysis, Fig. 5 Behavior of a well tting model. because this PC is the most complex and presumably the most important operation of our model. More concretely we discuss individual dierences in 4. Behavior Oriented Approach processing the fth column of the problem. As we noted, the fth column, O+E=O, is a very unique We presented some examples of simple models in one. If this column is processed correctly, E=0 or E=9 can be inferred without any other information. the previous section. In this section, we will disA single rule, R4 is implemented with this PC opcuss well tting models. Well tting models are eration. Rule R4 is accompanied by a set of twelve constructed based on the Behavior Oriented Apsub-procedures constructing a type of production proach. In the previous models, special emphasis system. For example, a sub-procedure for Digitwas placed on simplicity. However, in well tting 1 is "IF two letters and a carry in a column are models, the focus is on the exact t between the behavior generated by the models and the psychoknown, THEN the other letter in the column and logical data on human behavior. a carry in the next left column are inferred." For example, in the fourth column, N+R=B, N=6, and 4 1 Well Fitting Models for S3, S4, S5,a carry of the fth column, c5=1, are inferred from known information, R=7, B=3, and c4=0. and S8 Parameter t5 is involved in this R4 and controls The protocol data of four subjects: S3, S4, S5, these sub-procedures, for example, by deciding the and S8 are reproduced in [Newell 72]. Based on this applied priority of each sub-procedure. data, we constructed four well tting models, W3, Two of the twelve procedures, Fifth-1 and FifthW4, W5, and W8, with each providing a specic 2, are exclusively used for processing the fth colproduction system for one of the subjects: S3, S4, umn. So we can obtain information on individual S5 and S8. As an example, we show the behavior of dierences in processing the fth column through Model W3 in Figure 5, in which the corresponding examining the value change of t5 and the usage of protocol segments reproduced from [Newell 72], are the sub-procedures controlled by the parameter. also shown. In Model W3, Fifth-1 and Fifth-2 are used eight In these well tting models we lose simplicity. times. In Model W4, these two procedures are The complexity of the models increases as we strive
N=6 PC B=3 c5=1 (B269-B300)

CDc select PC reject E=0or9 E=0 column E12 CDc PC reject select c5=2 E=0or9 E13 column PC PC c5=1 recall E=9 c3=1 c6=1 E4-E7 PC R=3 Reject E15 L=6 c3=1 PC PC A=4 PC CDe R=7 L=8 N>3 c3=1 c3=1 c4=0 O=2

for more accurate ts between the models' behavior and human behavior. To increase accuracy we have to allow a number of arbitrary changes to parameters and the addition of secondary rules. For example, in Model W3 the default value (mostconstrained) of a t4 parameter that controls the FC (nd column) operation is used 29 times, and the three specic values, left, right, and (particular n), are used three times, once, and once, respectively. Thus, there is a tradeo between simplicity and accuracy. This will be discussed in detail in section 5.
42 Analysis of Process Column

Vol.14 No.6

9 used three times although this model has the shortest solution path. Hence the two models, W3 and W4, process the fth column correctly by adopting Fifth-1 and Fifth-2. On the other hand, in Model W8, each of the procedures, Fifth-1 and Fifth-2, is used only once, even though there are more chances to apply them because the solution path is the longest. Model W8 could not adopt these specic procedures to process the fth column until near the end of the solution path, and the absence of the applications of Fifth-1 and Fifth-2 in the early stage of the solution path causes the length of the solution path to increase. The most interesting case is Model W5, in which Fifth-2 is used only once, and Fifth-1 is not used at all. However, when we look at the behavior of W5, we notice that the fth column is processed many times, because the information E=0 or E=9 appears repeatedly throughout the solution path. This repetition happens because the column is processed by another procedure, Digit-2, not by using Fifth-1. Digit-2 is similar to Digit-1, but the Digit-2 procedure is applied only when two letters are known and a carry in the column is unknown. Again in the fourth column, N+R=B, Digit-2 infers N=5 or 6, and c5=1 from known information, R=7, B=3. For processing the fth column, Digit-2 is used instead of Fifth-1. This happens because W5 makes an erroneous inference, O=1, in an early stage of the solution path. This buggy information, O=1, is stored in the following episodes, and places the fth column, O+E=O, in the condition, 1+E=1. In this situation, the more general procedure, Digit-2, is applied to the fth column prior to the specic procedure, Fifth-1, and infers E=0 or E=9, just as Fifth-1 does.
5. Two Approaches for Analyzing Human Problem Solving

In sections 3 and 4 we described two approaches for constructing process models of human problem solving. In section 1, we noted that the Model Oriented Approach corresponded to the computational apNov. 1999

proach and the Behavior Oriented Approach to the psychological approach. This claim can be better understood by comparing the two ways models are constructed, as described in sections 3 and 4. First let us see the tradeo relationship between simplicity and accuracy. Simple models have the advantage of parsimony, whereas well tting models often sacrice simplicity for accuracy. Table 1 shows the tradeo between simplicity and accuracy. Models S1, S4, and S5 have very few individual parts. That is, only small changes in the parameters or restricted rule additions are introduced. The occurrence of the individual parts in models S2, S3, S6, S7, and S8 are substantially more frequent than those of the former three. Especially, in models S6, S7, and S8, in which the ratios approach .5. This relatively large number of individual parts can be attributed to the consistent changes of a few parameters, and to several additional rules, each of which res many times. This means that in these models, the individual parts can be characterized as consistent; so simplicity is actually preserved in all of these models. On the other hand, the well tting models W3, W4, W5, and W8|even though their behavior corresponds closely with human behavior|have prominent individual parts. As we noted in the previous section, these parameters had to be changed arbitrarily to t and many specic rules were added, almost all of which re only once to match specic situations. In the well tting models, simplicity disappears. The trade o relationship of simplicity and accuracy is well known as \Occam's Razor". Recently Domingos pointed out two aspects of this principle [Domingos 98]. Each characteristic of the Model Oriented and Behavior Oriented Approaches is derived from a methodological dierence between the two strategies. Figure 6 illustrates the relationship between the two approaches. In simple models, we try to match the parameter changes and the rule additions with certain representative aspects of the individual dierences of each subject. Based on this correspondence, we
9

Deductive and Inductive Ways of Investigation on Human Problem Solving

10
No S1 S2 S3 S4 S5 S6 S7 S8 W3 W4 W5 W8 Percentages of common part, rule modication part, and rule addition part. Version # of Rule Firings # of Protocol Common Rule Modication Part Rule Addition Part Total Segments Part Total Inconsistent Total Specic The most basic 70 0 0 0 0 70 (100%) (0%) (0%) (0%) (0%) Controlling FC 62 11 0 0 0 73 left to right (84.9%) (15.1%) (0%) (0%) (0%) Controlling FC 71 13 0 0 0 84 right to left (84.5%) (15.5%) (0%) (0%) (0%) Assign if two 108 3 3 0 0 111 alternatives (97.3%) (2.7%) (2.7%) (0%) (0%) c3=0 error 136 4 4 1 1 141 (96.5%) (2.8%) (2.8%) (0.7%) (0.7%) No parity 41 32 0 0 0 73 (56.2%) (43.8%) (0%) (0%) (0%) No inequality 89 45 0 0 0 134 (66.4%) (33.6%) (0%) (0%) (0%) No processing of 303 204 0 29 0 536 fth column (56.5%) (38.1%) (0%) (5.4%) (0%) Well tting for 178 51 51 13 13 242 321 Subject S3 (73.6%) (21.1%) (21.1%) (5.4%) (5.4%) Well tting for 74 32 32 2 2 108 105 Subject S4 (68.5%) (29.6%) (29.6%) (1.9%) (1.9%) Well tting for 334 70 70 7 7 411 548 Subject S5 (81.3%) (17.0%) (17.0%) (1.7%) (1.7%) Well tting for 343 76 76 25 20 444 661 Subject S8 (77.3%) (17.1%) (17.1%) (5.6%) (4.5%)
Table 1
Theory Model

INDUCTIVE Behaior Oriented Approach Psychological fit (trace) behavior then identify the model specification through model tuning accuracy large and inconsistent indivicual parts DEDUCTIVE Model Oriented Approach Computational model specification then test behavior produced by the model parsimony small or consistent individual parts

Behavior Data

Fig. 6

Comparison of model oriented and behavior oriented approaches.

can produce some simulations of typical behaviors. Thus, in this strategy, as demonstrated in section 3, specications of the models are selected by the parameter changes and the rule additions, and then behavior generated by the models is observed and discussed. Accordingly, the analysis is developed from model to behavior, which gives a model-driven avor to this strategy. This direction of model-tobehavior can be regarded as a process from theory to data. So, the Model Oriented Approach provides us with a deductive way of conducting an analysis. This kind of design is a general way of conducting
10

analyses in computer simulations whose characteristics are shown in the right side of Figure 6. On the other hand, in well tting models, human behavior represented as psychological data takes a primary role. The individual parts of models are added one by one to the common part; in other words, parameter changes and rule additions are tuned, while checking the models' behavior with the human behavior. In this case, initially we have to focus on the t between the model's behavior and the human behavior. After obtaining satisfactory correspondence in this way, then we move on to analysis of the parameter changes and the rule additions, which tell us what modications were needed for the t. Therefore this approach can be summarized as follows: rst trace behavior then identify the procedures needed for the trace, and analyze, for example, the occurrence frequency of each procedure and important patterns of usage of each procedure. Thus the analysis is developed from behavior to model, which contrasts with the model-to-behavior order of the computational approach. In contrast to simple models, this strategy has the avor of behavior-driven analysis, and can be regarded as a
Vol.14 No.6

11 process from data to theory. So, the Behavior Oriented Approach provides us with an inductive way of conducting an analysis. This approach is what we usually do when analyzing processes of human problem solving based on a coding schema, which are obtained in psychological experiments with the think aloud method. This approach is shown in the left side of Figure 6. The indication that the Model Oriented Approach is a deductive way of investigation and the Behavior Oriented Approach is an inductive way of investigation might be not an essential argument because those two concepts are logically identical: that is, deduction and induction are naturally dened by concepts of theory (model) driven and data (behavior) driven respectively. However by indicating the correspondence of the two parallel concepts, we conrm another aspect of the two methodologies. The interplay between deduction and induction in natural sciences, as shown in Figure 6, have been indicated by many philosophers through the long history of the philosophy of science (for example, see [Popper 59]). We can understand the relationship between the Model Oriented approach and the Behavior Oriented approach based on the traditional viewpoints of scientic investigation.
6. Conclusions

these viewpoints. However, we believe it is essentially important that these major two methodologies of cognitive science, the computational approach and the psychological approach, were clearly placed as the Model Oriented Approach and the Behavior Oriented Approach, based on the two principle concepts, Theory and Data, in natural sciences. In addition, this meta-level discussion was empirically supported by computer simulations, constructing a computational model that actually worked as a computer program. By conducting simulations, we veried high reliability of our discussion in this paper. Generally, natural sciences, such as physics, chemistry, biology, have been developed through the interaction of deductive and inductive ways of investigation. That is, a theory predicts future phenomena, and data obtained in experiments verify the predictions. We can approach the truth based on these two bridges: the deductive (theory to data) and the inductive (data to theory) bridges. The methodology discussed in this paper is one example of these kinds of investigation. Both the computational and psychological approaches provide us with strong methodologies for investigating the human mind.

References

In this paper, we discussed the relationship between the psychological and computational approaches, which are two basic methodologies used to construct process models of human problem solving. We regarded the computational approach as the Model Oriented Approach, and the psychological approach as the Behavior Oriented Approach. The Model Oriented Approach provides us with deductive ways of conducting analyses, whereas the Behavior Oriented Approach provides inductive ways of conducting analyses. We also demonstrated the two approaches using the production system architecture, which has been widely used for constructing process models of human problem solving. Many cognitive scientists have already recognized
Nov. 1999

[Anderson 93] J. Anderson, Rules of the Mind, Lawrence Erlbaum Associates, 1993. [Anzai 79] Y. Anzai and H. Simon, The theory of learning by doing, Psychological Review, vol. 86, pp. 124-140, 1979. [Domingos 98] P. Domingos, Occam's two razors: the sharp and the blunt, Proceedings of the fourth international conference on knowledge discovery and data maining, pp. 37-43, 1998 [Ericsson 80] K. Ericsson and H. Simon, Verbal reports as data, /it Psychological Review, vol. 87, pp. 215-251, 1980. [Klayman 89] J. Klayman and Y. Ha, Hypothesis testing in rule discovery: Strategy, structure, and content, /it Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 15, pp. 596-604, 1989. [Langley 87] P. Langley, H. Simon, G. Bradhsaw and J. Zytkow, Scientic discovery, Cambridge, MA: MIT Press, 1987. [Miwa 96] K. Miwa and T. Okada, Eective Heuristics for Hypothesis Testing: An Empirical Discussion using a Computer Simulation Method, Journal of Japanese Society for Articial Intelligence, vol. 11, pp. 877-887, 1996. [Neches 87] R. Neches, P.Langley, and D. Klahr, Learn11

Deductive and Inductive Ways of Investigation on Human Problem Solving

12
ing, development, and production systems, In D. Klahr, P. Langley, and R. Neches (Eds.), Productions Systems of Learning Development, Cambridge, MA: MIT Press, 1987. [Newell 72] A. Newell and H. Simon, Human Problem Solving, Englewood Clis, N.J.:Prentice-Hall, 1972. [Newell 91] A. Newell, Unied Theories of Cognition, Cambridge University Press, 1991. [Popper 59] K. Popper, The logic of scientic discovery, Harper & Row, 1959. [Simon 75] H. Simon, The functional equivalence of problem solving skills, Cognitive Psychology, vol. 7, pp. 268288, 1975. [Simon 78] D. Simon and H. Simon, Individual dierences in solving physics problems, In R. Siegler (Ed.), Children's Thinking: What Develops? Hillsdale, J.J.: Erlbaum, 1978. [Qin 90] Y. Qin and H. Simon, Laboratory replication of scientic discovery processes, Cognitive Science, vol. 14, pp. 281-312, 1990. [Waterman 71] D. Waterman, and A. Newell, Protocol analysis as a task for articial intelligence, Articial Intelligence, vol. 2, pp. 285-318, 1971. [Young 81] R. Young, and T. O'Shea, Errors in children's subtraction, Cognitive Science, vol. 5, pp. 153177, 1981.

Author's Prole
1989, Dr. of Eng. Department of Information Engineering, Nagoya University. 1989-1993, Assistant Professor of Education Center for Information Processing, Nagoya University. 1991-1992, Visiting Assistant Professor of Department of Psychology, Carnegie Mellon University. 1993-, Associate Professor of Graduate School of Human Informatics, Nagoya University. Research Interests: Computational Models for Human Problem Solving, Learning, Scientic Discovery. Aliations: Japanese Society for Articial Intelligence; The Japanese Cognitive Science Society; The Institute of Electronics, Information and Communication Engineers. miwa@cog.human.nagoya-u.ac.jp

Kazuhisa Miwa (member)

12

Vol.14 No.6