APPENDIX.

Qualifying Questionnaire
This section provides a more in-depth discussion of how respondents ranking
of the tasks for Mario might be interpreted differently. As explained, the
questionnaire was designed to uncover respondents natural stance towards
critical thinking: by asking respondents to value-rank tasks which require
different levels of critical thinking, the questionnaire covertly evaluated how
much each respondent valued a given set of critical thinking abilities, thus
providing a window onto whether and how much a respondent is naturally a
critical thinker. The assumption is that the more automatic a respondent
values abilities involving critical thinking, the more likely the respondent will
make efforts to cultivate critical thinking in their classrooms. On a more
conservative note, one could hypothesize that respondents who place a high
value on mnemonic abilities would probably be more prone to test for details
rather than conceptual understandings (CITE). However, since the
questionnaire did not explicitly ask respondents to attend to critical thinking
per se, there are alternative interpretations to the rank-values obtained in
Section 1. This is what this section discusses.
At first glance, the findings reported in Section 1 for the sample of 505
respondents was rather positive since the categorization of the five abilities
partially confirmed the a priori assumptions regarding the level of critical
thinking involved with each of the abilities: PHOTO would require 8-year-old
Mario to not only observe, describe and compare and contrast two photos, but
to be able to assume his grandmothers position of information recipient and
thus provide the necessary information for his grandmother to see the photos
over the phone. The complexity of perspective taking requires an objective
stance and a high level of critical thinking for 8-year-old Mario. In addition, the
fact that CAPITAL and FLOWER received the least important ranking most
frequently and thus had the lowest average rank-values confirmed the a priori
assumption that educators should not favor these more mnemonic abilities
which require the least amount of critical thinking.
Interestingly, whether respondents would have distinguished between
CLASSMATE and TEACHER had not been established a priori as both require
cognitive processes which engage 8-year-old Mario in a reflection upon his
sentiments. However, CLASSMATES, is an easier task than TEACHER as
CLASSMATES stops as Marios own sentiments on a classmates way of being
while TEACHER would require Mario to objectify his own subjective sentiments
as he considers the BEST TEACHERS way of doing that makes him/her the
best teacher. Since the best TEACHER may not necessarily be the nicest
teacher, valuing TEACHER means that respondents feel it important for Mario
to be able to overlook subjective sentiments in favor of objective evaluations.
In addition, the ability to identify the BEST TEACHER means that 8-year-old
Mario is aware of how the best teacher makes learning easier and is therefore
aware of how he learns best, a highly meta-cognitive process.
Therefore, TEACHER actually calls upon much more complex cognitive
processes than both CLASSMATE and PHOTO. Had the respondents been
explicitly queried and invited to reflect more deeply on the degree of critical

thinking needed for each task, TEACHER may have emerged as a more
important ability than CLASSMATE and PHOTO. However, since the
questionnaire was designed to reveal respondents tacit stances towards
critical thinking, it was interesting that TEACHER was always the most
ambiguous ability.
Without additional data, it is possible to only deduce why 76 and 151
respondents had assigned TEACHER rank-values of 5 and 4, respectively. At
least two main reasons come to mind. The first is that these respondents
actually were aware that TEACHER requires that Mario engages in the
aforementioned highly meta-cognitive processes while the second reason may
be that respondents who rated TEACHER as the most important ability value
interpersonal rapport and thus consider it important for 8-year-old Mario to
reflect on his own sentiments, even while judging authority.
How these respondents classified the next most important ability can be used
to distinguish between these two reasons. Of the 76 respondents who
classified TEACHER as the most important ability, the ability CLASSMATE
received the most rank-value of 4: of the 76 respondents assigning 5 to
TEACHER, 49 respondents (ca 65%) considered CLASSMATE the next most
important ability. PHOTO received a rank-value of 4 from only ca. 21% (n=16)
of this group of 76 respondents (figure XX1). For these respondents, PHOTO
received the most value-3 assignment. Not surprisingly, those that valued
TEACHER as 5 considered FLOWER and CAPITAL to be equally unimportant
(n=28 and 32, respectively). The average rank-values these 76 respondents
assigned to each of the abilities is shown in figure XX1B.
Figure XX1.

figure XX1B

TEACHER = 5

Ph
ot
o

at
e

Cl
as
sm

2.8

1.9

1.9

Ca
pi
ta
l

3.5

Fl
ow
er

5.0

Te
ac
he
r

6
5
4
3
2
1
0

Consistent with this are the findings for the 131 respondents who assigned a
value of 5 to CLASSMATE: for these respondents, TEACHER received the most
frequent value-4 assignment from ca. 52% of the 131 respondents (figure X1).
However, since PHOTO also received value-4 from ca. 36% of these
respondents, together with the fact that PHOTO also received the highest
number of value-3 assignment (ca. 46%), among the 131 respondents
assigning 5 to CLASSMATE, both TEACHER and PHOTO received an average
rank-value of 3.1 (figure X1B).
Figure X1.

Figure X1B.

CLASSMATE = 5
6
4

5.0
3.1

3.1

2.0

1.8

Capital

Flower

0
Classmate Teacher

Photo

For these 131 respondents who had most valued CLASSMATE, the least valued
skills was, not surprisingly, FLOWER, assigned a rank-value of 1 by ca 42%
(n=55) of these respondents and CAPITAL which was assigned a value of 1 by
ca. 34% (n=45) of these 131 respondents.

Together, these data which collocate TEACHER with CLASSMATE indicate that
when respondents valued these abilities, it was probably due to the fact that
they valued interpersonal skills for 8-year-old Mario. Although some
respondents may have indeed recognized that TEACHER requires highly metacognitive processes such as the objectification of subjective sentiments or an
awareness of how I learn best. However given the questionnaire design, it
was not possible to determine if this was the case for those that did value
TEACHER or the case for those who, realizing this, may have considered such
cognitive processes too difficult for 8-year-old Mario.
Finally, given that the questionnaire did not explicitly address critical
thinking, the ability TEACHER appeared to be the most ambiguous ability
probably because, while it was clearly better than mnemonic abilities,
respondents were probably intuitively aware that TEACHER requires more
complex processes than CLASSMATE or PHOTO. Not aware that the abilities
were designed for evaluating the complex construct of critical thinking, it is not
surprising that PHOTO was most favoured as it was easily considered the
most important ability for 8-year-old Mario with CLASSMATE also rather
easily considered an important ability for Mario. In fact, the cognitive
processes surrounding PHOTO remain at a very objective level as the task does
not call upon Mario to be aware of his own sentiments and nor does Mario need
to manage his sentiments to describe the photos to his grandmother.
Therefore, PHOTO was probably chosen by most of the respondents because
the online questionnaire was designed to reveal respondents immediate,
default appreciation of the five abilities and was not designed to seek well
thought-through reasons for ranking the five abilities under the context of
Critical Thinking.
Future teacher training sessions which explicitly address critical thinking will
make it possible to see if, given an explicit classification scheme based on
critical thinking, teacher-trainees will be able to identify TEACHER as the ability
requiring deeper critical thinking processes than PHOTO and CLASSMATE.
In fact, figures X1 and X2 show that respondents who assigned a rank-value of
5 to TEACHER gave a very high rank-value to CLASSMATE, and interestingly,
visa versa.
. For these 47 respondents who assigned a rank-value of 5 to FLOWER,
CLASSMATE was consistently considered the least important ability (figure
F3) since it received the least number of rank-value-4 (only four (8.5%)) from
these respondents and likewise, received the most rank-value of 1 (n=16
(34%)) from this group of respondents (figure F4).

FLOWER_5 x other skills

25
20
15 value 1

value 2

value 3

value 4

10
5
0

photo

classmate

capital

teacher

figure F2
FLOWER = 5 (n=47): assignment of 4
21
12

10

figure F3

figure F4

m
at
e
cl
as
s

te
ac
he
r

ph
ot
o

ca
pi
ta
l

25
20
15
10
5
0

FLOWER = 5
6
5
4
3
2
1
0

5.0
3.0

Flower

Capital

2.7

2.3

Photo

2.0

Teacher Classmate

Interestingly, of the 27 who considered CAPITAL the most important ability for
8-year-old Mario, 12 actually considered TEACHER to be the next most
important ability with a rank-value of 4 (figure C1). Only two of the 27
respondents had assigned 4 to PHOTO. However, along with the fact that
FLOWER and CLASSMATE both received ca. 22% of the rank-value 4
assignments, the averaged rank-values was rather comparable between the
remaining four abilities (fig C2). The most notable data which confirms the
emergent clustering of the abilities is the fact that of these 27 respondents who
considered CAPITAL the most important ability, 16 (59%) considered PHOTO
the least important ability for 8-year-old Mario. Therefore, albeit a small
sample of only 27 respondents favouring CAPITAL, these results nonetheless
confirm the fact that the skills clustered into two distinct groupings: when
respondents considered the skills related to memorization and detail-learning
more important, the skills requiring Mario to engage in more complex critical
thinking processes were considered less important.
Figure C1.
CAPITAL_5 x other skills
20
15
10

value 1

value 2

value 3

value 4

5
0

photo

flower

classmate teacher

Figure C2.
CAPITAL = 5
6
5
4
3

5.0
2.9

2.9

2.5

1.8

1
0
Capital Classmate Teacher

Flower

Photo

Again, consistency in evaluation.

It should be noted that the gradation was consistent: if preference was for
memorization skills, then the move towards more interpersonal skills went
through PHOTOS, which is more to do with non-emotional than CLASSMATES
and TEACHER.

GIOVANNI, look at the following section isnt the information on the left weak without
an r/R or something that quantifies the distributions for each ability? What can we use
if not R2?

The distribution of rank-values of

each of the five abilities is shown
below, in the column on the left.

shown in the graphs on the right and

confirm that the respondents
considered CAPITAL and FLOWER of
similar less importance 8-year-old
Mario (negative R2) while PHOTO
and CLASSMATE were considered
important abilities for Mario (positive
R2). Again, the ability which showed
almost no value-distinction was
TEACHER with a correlation R2 of
0.05.

The correlations (R2) derived from

the ranking of each ability are
photo

250

200
150
100
50

photo

227

250

200

122
48

150

77

100

34

50
0

value 1 value 2 value 3 value 4 value 5

value 1 value 2 value 3 value 4 value 5

flower

flower
200

250
200
150
100
50

R = 0.66

162

150

152
85

R = 0.93

100

62

47

50
0

0
value 1 value 2 value 3 value 4 value 5

value 1 value 2 value 3 value 4 value 5

classmate

classmate

250

200

200

159

150

100

89

100

131

R = 0.78

100

29

50

150

50

value 1 value 2 value 3 value 4 value 5

0
value 1 value 2 value 3 value 4 value 5

capital

capital

250

200

200

181

173

150

R = 0.86

150
100

69

100

58

27

50
0

value 1 value 2 value 3 value 4 value 5

50
0
value 1 value 2 value 3 value 4 value 5

teacher
250
200
150
100
50
0

teacher
200

152

132

96

150

76

52

100

R = 0.05

5
e

4
lu
va

va

lu

3
va

lu

2
e
lu
va

va

lu

50
0
value 1 value 2 value 3 value 4 value 5

As seen in figure X101X, above, a comparable number of respondents assigned

value-4 to CLASSMATE and TEACHER (n=158 and 151, respectively). Since
these, alongside PHOTO, were hypothesized to involve more critical thinking
than mnemonics and since each rank-value could only be used once, to discern
whether CLASSMATE or TEACHER was more important, we summed the
frequency each ability received the rank of both 5 and 4. Figure x2x shows
that, when considering the two most important abilities, the abilities of
PHOTO and CLASSMATE emerged as comparably important, and more so than
TEACHER. The abilities of CAPITAL and FLOWER remained the least important
abilities, i.e. receiving value-assignments of 4 or 5 from only, respectively, ca.
17% (n=85) and 21% (n=108) of the 505 respondents (see Section VV for a
more detailed discussion on this last point).
Figure x2x.

Another source of data which confirm the clustering of PHOTO with CLASSMATE
and FLOWER with CAPITAL is seen by how respondents who valued one type of
ability de-valued the other. For example, figure x1x shows the distribution of
the rank-values of 1 to 4 among the remaining 4 skills for the 224 respondents
who had assigned PHOTO a value of 5. For this group of respondents,
CLASSMATE was the second most valued ability, receiving a rank-value of 4
from ca. 44% (n=99) of the 224 respondents with an average rank-value of 3.1,
followed by TEACHER (n=61) with an average rank-value of 2.7 (see fig. X1x2
and Appendix XX2 for details). In line with the emergent indicators, of the 224
respondents who considered PHOTO the most important ability, the least
valued skill was CAPITAL which received a rank-value of 1 from 38% (n=86) of
this group of respondents, with an average rank-value of 1.9. The second-least
important ability for this group of respondents was FLOWER which received a
rank-value of 1 from 33% (n=74) of these 224 respondents, with an average
rank-value of 2.3.
Figure x1x.

PHOTO-5 x other skills

120
100
80
60

value 1

value 2

value 3

value 4

40
20
0
flower

classmate

capital

teacher

Figure x1x2.

PHOTO = 5
6
5
4
3
2
1
0

5.0
3.1

2.7

Photo Classmate Teacher

2.3

1.9

Flower

Capital

Therefore, respondents favouring abilities associated with more critical thinking

processes, i.e. PHOTO, CLASSMATE and TEACHER, tended to not favour
CAPITAL and FLOWER which call upon more mnemonic abilities, and vice versa.
This clustering of abilities can be seen in figure F1 which shows that, of the 47
respondents who assigned value-5 to FLOWER, the second-most valued ability
receiving the most rank-value of 4 was CAPITAL (figure F2), with an average
rank-value of 3.0 (see figure F2 and Chapter 2 Qualifying Questionnaire for
details). In line with the emergent clustering of abilities, of the 47 respondents
considering FLOWER the most important ability for 8-year-old Mario, the least
valued ability was CLASSMATE which received a rank-value of 1 from 16 (34%))
respondents from this group and an average lowest rank-value of 2.0.
figure F1

FLOWER_5 x other skills

25
20
15 value 1

value 2

value 3

value 4

10
5
0

photo

classmate

capital

teacher

figure F2

FLOWER = 5
6
5
4
3
2
1
0

5.0
3.0

Flower

Capital

2.7

Photo

2.3

2.0

Teacher Classmate

Interestingly, of the 27 who considered CAPITAL the most important ability for
8-year-old Mario, 12 actually considered TEACHER to be the next most
important ability with a rank-value of 4 (figure C1). Only two of the 27
respondents had assigned 4 to PHOTO. However, along with the fact that
FLOWER and CLASSMATE both received ca. 22% of the rank-value 4
assignments, the averaged rank-values was rather comparable between the
remaining four abilities (fig C2). The most notable data which confirms the
emergent clustering of the abilities is the fact that of these 27 respondents who
considered CAPITAL the most important ability, 16 (59%) considered PHOTO
the least important ability for 8-year-old Mario. Therefore, albeit a small
sample of only 27 respondents favouring CAPITAL, these results nonetheless
confirm the fact that the skills clustered into two distinct groupings: when
respondents considered the skills related to memorization and detail-learning
more important, the skills requiring Mario to engage in more complex critical
thinking processes were considered less important.
Figure C1.

CAPITAL_5 x other skills

20
15
10

value 1

value 2

value 3

value 4

5
0

photo

flower

classmate teacher

Figure C2.
CAPITAL = 5
6
5
4
3

5.0
2.9

2.9

2.5

1.8

1
0
Capital Classmate Teacher

Flower

Photo