Cognitive Development of Informal Inferential Reasoning

Chris Reading
The National Centre of Science, information and Communication Technology, and Mathematics for Rural and Regional
Australia, University of New England, Armidale, NSW, Australia, 2351.
E-mail: creading@une.edu.au


Introduction
Research into cognitive development and proposing models to explain such development is invaluable for
assisting educators in planning teaching sequences and assessing learning. In statistics education, research
into models of cognitive development has expanded from dealing with general instructional models to
models for assessing the development of reasoning and then, more recently, to models of reasoning about
key statistical concepts such as variation and distribution. Interest amongst an international group of statistics
education researchers has now focused on reasoning about statistical inference, in particular informal
inference. This focus provided the theme for the Fifth International Research Forum on Statistical
Reasoning, Thinking and Literacy, SRTL5, held in 2007. The eleven detailed research reports provided a
broad basis for discussions at the forum. Subsequently, the ideas from five of these research reports were
further developed and published in a special issue of the Statistics Education Research Journal, SERJ7(2).
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
1 of 27

This paper presents a synthesis of this research. The range of research was based on participants from aged
eight to adult. This paper is more than a literature review because it comes from a synthesis of the authors
experiences during the detailed discussions at SRTL5, as well as the written documentation and later
interpretation of results based on the discussions. First, informal inferential reasoning (IIR) is defined.
Second, the foundation, or underlying, concepts most appropriate as a basis for IIR are explained. Third,
necessary components for an IIR conceptual framework are shared. Fourth, a structure for an IIR cognitive
development framework is proposed. Fifth, scaffolds are explained that support the development of IIR.
Finally, challenges are set for future research into the cognitive development of IIR.

Informal Inferential Reasoning
The recent increase in interest in researching IIR has necessitated some agreement on the meaning of the
term IIR. There is general agreement that statistical inference involves drawing, from data, conclusions that
have a degree of uncertainty (see, e.g., Bakker, Kent, Derry, Hoyles & Richards, 2007; Makar & Rubin,
2007), which involves parameter estimation and/or hypothesis testing (Ben-Zvi, Gil & Apel, 2007). Many
statistics educators approach defining IIR by stating that it is inferential reasoning that does not involve
sophisticated statistical testing but not all researchers hold that view. For example, Makar and Rubin (2007)
defined IIR by providing three essential principles (i) generalization that goes beyond the data, (ii) use of
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
2 of 27

data as evidence for the generalizations, and (iii) use of probabilistic language to describe the generalisations.
These guiding principles have been utilised by others (e.g., Johnson-Wilder, Ainley & Pratt, 2007) but some
researchers widen the scope of the definition by including the cognitive processes that are necessary for
achieving these various inferences (Ben-Zvi et al., 2007). As a culmination of SRTL5 discussions, a useful
definition that includes how inferences are reached describes IIR as the way in which students use their
informal statistical knowledge to make arguments to support inferences about unknown populations based on
observed samples (Zieffler, Garfield, delMas & Reading, 2008, p. 44).

So, how does informal inferential reasoning differ from its formal counterpart? Zieffler et al. (2008)
describe IIR by identifying what the reasoning can be about: (i) population characteristics (e.g., shape) based
on a sample; (ii) possible population differences based on observed sample difference; or (iii) the likelihood
of a data sample given a particular expectation or claim. However, this does not help to distinguish
informal because such reasoning is also necessary for formal inference. Apart for the lack of formality in
the processes, one important difference identified by Bakker et al. (2007) is that context is always important
in IIR, whereas in formal inferential reasoning context can be suppressed during certain stages of the
reasoning process.

I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
3 of 27

Thus far, the focus has implied that the reasoners are progressing from the less formal to the more formal
inferential reasoning but there are situations where practitioners need to move forward with decision-making
but without the time or the data to do formal tests (e.g., the Statistical Process Control reported by Bakker,
Kent, Derry, Noss & Hoyles, 2008). This form of informal inference differs from formal inference in that the
focus is on the need for action and the context cannot be suppressed. Bakker et al. (2008) made it clear that
there are many who need to make evidence-based decisions without using formal statistical tests. For a more
detailed discussion of the similarities and differences of informal versus formal inferential reasoning in the
workplace see Bakker et al. (2008, pp. 138-139).

A necessary step for statistically literate citizens is to move from simply making observations about shapes
in data, to using this information to make decisions. Watson (2007) explains that students need assistance
with the early stages of making this step. For some, a first step is intuitive informal inference where
experience can assist in assessing the strength of evidence for or against a claim (Rossman, 2008) but
intuition will not always be able to provide what is expected to occur. Previously, access to more formal
inferential reasoning has been restricted for younger students because of the difficulties in understanding the
concepts. This means that they were only provided with experiences involving descriptive statistics but the
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
4 of 27

use of technology now allows data-driven activities to be used to lay conceptual foundations even for young
students (Paparistodemou & Meletiou-Mavrotheris, 2007, p. 110).

Ben-Zvi et al. (2007) proposed that IIR can form an important bridge to guide reasoners from exploratory
data analysis to formal inference. However, research (Zieffler, Garfield, delMas & Gould, 2007) has shown
that the transition from IIR to formal inferential reasoning is not as seamless as hoped, even with specially
designed learning experiences. Thus, it is essential that more research focus on the cognitive processes
involved in IIR to better inform supporting the development of formal inferential reasoning. Next,
foundation concepts that are necessary components of an IIR conceptual framework are proposed.

Foundation Concepts
There are critical underlying concepts that need to be understood to provide a sound foundation on which to
build the necessary reasoning involved with making statistical inferences. The SRTL5 research reported a
variety of foundation concepts that fall into two categories: features and actions. The basic statistical features
that need to be understood to engage effectively in IIR include: variation, distribution, mean, spread and
graphs. Although Bakker et al. (2007) identified such concepts as necessary to engage in informal inference
they found that it was difficult to identify exactly what concepts were needed for employees to engage in
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
5 of 27

particular IIR activities in the workplace. Apart from these statistical features there were four statistical
actions identified by the researchers as necessary building blocks to form a basis for IIR.

Viewing Data as an Aggregate
If data are viewed as individual cases then drawing informal inferences will be difficult and the necessary
reasoning will not ensue. Once data are viewed as an aggregate, the reasoning necessary for IIR is possible.
When working with 10 to 11 year olds using InferenceMaker, Pratt, Johnston-Wilder, Ainley and Mason
(2008) found that students tended to have a local perspective on data focusing on local points, thus attending
to the here-and-now rather than the aggregate picture in the data. They rightly point out that such a view of
the data should not be considered a misconception but should be viewed as a starting point on which to build.
Konold, Kazak, Lehrer & Kim (2007) found that a short intervention activity involving comparison of
groups helped young students to see data as an aggregate.

Focusing on Proportions Rather Than Absolutes
An understanding of proportionality is critical to any interpretation of data. If raw frequencies are the focus
of data interpretation then incorrect inferences will be drawn. Rossman (2008) explained the increasing
conceptual difficulty from understanding sample proportions to appreciating the role of sample size.
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
6 of 27

Working with proportions needs to be nurtured in younger students who find it difficult to express
proportional relationships (Johnston-Wilder et al., 2007). They struggle with both the notion of
proportionality and the necessary language. Eight year olds interpreting Tinkerplots pie charts made additive
comparisons rather than using proportional reasoning when comparing groups but Paparistodemou and
Meletiou-Mavrotheris (2008) found that some students started using terms like part of whole. When
describing data students may describe relative sizes but they do not quantify the differences and rarely use
correct proportional terminology (Pratt et al., 2008). This may be related to the tendency to take a local,
rather than global, view of data. When Watson (2008) gave students an interpretation problem with data
presented in boxplots, the lack of data values meant that the students were pressured into recalling
proportional reasoning. Ways to encourage students to develop proportional reasoning are: provide a variety
of possibilities to visualise data (Paparistodemou & Meletiou-Mavrotheris, 2008), divide graphical
representations into simple and clear proportions (Pratt et al., 2008) and develop tasks that use proportions to
summarise data (Zieffler et al., 2008). Teacher discourse focused on a soup metaphor helped them to
consider proportions and thus understand that precision is related to sample size and not population size
(Rubin & Hammerman, 2007).

I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
7 of 27

Appreciating Variability in Samples
Reasoning that necessitates inferences from data to a model requires an understanding of the variability that
occurs in samples. Rubin and Hammerman (2007) say that to make inferences from samples it is necessary
to know about both representativeness and variability. This appreciation of variability includes the ability to
distinguish between systematic and random variation. To help students develop a statistical view of data
distributions as resulting from processes including both signal and noise, Konold et al. (2007) provided
opportunities to create spinner models that allowed students to view data behaving as if it were real.
However, developing an appreciation of the variability of samples is not easy. Despite students undertaking a
course that focused on developing fundamental statistical ideas, Zieffler et al. (2007) still found the students
were not good at considering variability within samples. In conjunction with this appreciation of the
variability of samples, those drawing inferences also need to appreciate the representativeness of samples
(Rubin & Hammerman, 2007). This leads to necessary discussions about sample size.

Appreciating Randomisation as a Process
Reasoning that necessitates inferences from model to data necessitates an understanding of the notion of
chance variability. Once 10 and 11 year olds had an aggregated view of data, they began to talk about long-
term randomness proportion of results stabilised when more trials were executed (Pratt et al., 2008, p.
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
8 of 27

110). Deliberately introducing randomness into the data collection process is fundamental in statistical
inference (Rossman, 2008).

Necessary Components for an IIR Conceptual Framework
A solid understanding of the foundation concepts is necessary before embarking on IIR. What follows is a
clarification of the components of IIR, as recommended by the SRTL5 researchers. This is not shared as a
definitive framework but as a summary of the different views of the SRTL5 researchers. Some researchers
presented their own frameworks for IIR while others elaborated on specific components.

Three basic components of IIR proposed by Zieffler et al. (2008) provide a useful starting point for a
framework. These are: (i) making judgements, claims or predictions about populations based on samples; (ii)
drawing on, utilizing and integrating prior knowledge; and (iii) articulating evidence-based arguments for
judgements, claims or predictions about populations based on samples. Explanation of these three
components is provided in the context of three different types of tasks (Zieffler et al., 2008) but more
detailed explanations are needed of the capabilities needed to achieve these. The essence of such capabilities
is what needs to be sought as necessary components for an IIR conceptual framework.

I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
9 of 27

Some IIR frameworks, or descriptions, presented at SRTL5 were developed for particular situations. For
example, Watson (2008) proposed a beginning IIR framework to guide teachers in implementing learning
sequences based on Tinkerplots hat plots. This framework had eight elements of which five (summary, shift,
signal, spread and sampling) are comparable to the previously articulated foundation concepts. The three
remaining elements (i.e., finding reasons for trends, showing understanding of context and considering
interesting individual cases) provide an important contribution to a possible IIR conceptual framework.

Another example of a development for specific situations was Ben-Zvi et al.s (2007) emerging vision of a
framework designed to support learning design for 11 to 12 year olds working in an open-ended EDA
learning environment facilitated by Tinkerplots. An important contribution from this proposal was the
identification of a need to consider both cognitive and socio-cultural aspects of IIR. Of the nine cognitive
aspects proposed, six have already been addressed in the foundation concepts in the previous section. This
leaves contextual reasoning, reasoning about comparing groups and inferential reasoning which all describe
specific types of inference that might be attempted. The socio-cultural aspects described include necessary
components as well as scaffolds to support IIR.

I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
10 of 27

Apart from these frameworks for specific situations, other researchers focused on one or two specific
components believed to be essential in an IIR framework. Six of these components are now shared.

Arguing
To engage in IIR persuasive arguments need to be developed. Research by Paparistodemou and Meletiou-
Mavrotheris (2007) indicated that young students, working in groups, could engage in argumentation about
multivariate data. This critical component of their IIR should not be developed in isolation from the data
(evidence). In an in-depth analysis of argumentation by 11 to 12 year olds engaged in statistical reasoning,
Ben-Zvi et al. (2007) stressed the importance of this arguing being based on relevant data. After analysing
inferences made by young students, Paparistodemou and Meletiou-Mavrotheris (2008, p. 100) identified
three different ways the students expressed their inferences: data-based argumentation; data-based
argumentation and generalization; and data argumentation and chance. These provide a springboard to the
next three components for IIR.

Basing on Evidence
To engage in IIR the persuasive arguments need to be based on relevant evidence (data). It is not sufficient to
be able to develop arguments, the arguments must be seen to be data-supported. A strong focus in Makar and
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
11 of 27

Rubins (2007) view of IIR was on the use of data as evidence, which assisted primary school teachers to
learn how to develop statistical inquiry sequences for students. One interesting aspect of basing inference on
data is the student notion of what makes evidence stronger. When Pratt et al. (2008) studied students making
informal inferences based on small amounts of data, they found that the students believed that by collecting
more data they would get more of the same and hence provide stronger evidence of their inference.
Argumentation based on evidence was particularly relevant in Bakker et al.s (2007) work with adults
engaged in IIR in the workplace, because critical decisions needed to be made in a timely manner based on
available evidence.

Looking Beyond the Data
To engage in IIR it is necessary to be able to look beyond that data. Rossman (2008) identified two important
ways that inference may need to go beyond the data: either by generalising results beyond the group from
which data was collected or by drawing a more profound conclusion about relationship(s) between variables.
Makar and Rubin (2007) believed that the ability to look beyond the data actually marks a fundamental
difference distinguishing inferential statistics from descriptive statistics. Their research indicated that some
teachers found it difficult to work beyond the data, and in fact could not even see the relevance of doing this.
A similar view of the importance of looking beyond the data was held by Ben Zvi et al. (2007) who provided
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
12 of 27

11 and 12 year olds with technology supported Exploratory Data Analysis situations and scaffolded activities
to encourage informal inference in the form of generalisations.

Expressing Uncertainty
To engage in IIR the degree of uncertainly for argumentation needs to be expressed. A language needs to be
developed that facilitates discussion about chance. Expressions such as more likely, might be, more
possible to were used to articulate uncertainty when third graders were drawing inferences about unknown
populations (Paparistodemou & Meletiou-Mavrotheris, 2008). The use of probabilistic language was one of
three principles of IIR included by Makar and Rubin (2007) to frame their work with teachers engaged in
informal inference. The other two principles were basing on evidence and looking beyond data as described
previously. Even Bakker et al.s (2007) research with IIR in the workplace emphasised the necessity of being
able to express uncertainty when making generalisations.

Connecting Samples and Populations
To engage in IIR what is happening in the sample(s) must be connected back to the population. This forges a
close connection between IIR and the concept of distribution, with the necessity of being able to connect
data-centric and modelling perspectives of distribution. Johnston-Wilder et al. (2007) claimed that the ability
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
13 of 27

to connect samples and populations was actually at the heart of IIR. One stumbling block in making informal
inferences for 10 to 11 year olds was their tendency to focus on the sample as if it actually were the whole
population. The difficulty of this component of IIR was demonstrated when Watson (2007) found that the
reasoning demonstrated when students were connecting samples and population was inconsistent. The
importance of connecting what is observed in samples back to inferences about populations is emphasised in
the working definition of IIR developed by Zieffler et al. (2008).

Connecting Results Back to Context
To engage in IIR, arguments must be connected back to the context in which the inference is set. Contexts
can have an important impact on the ability to engage in IIR. This component, along with a good knowledge
of the context, was identified as crucial to IIR in the workplace (Bakker et al., 2008). In the process of
connecting back to the context two important issues arise. Firstly, data-based claims are often mixed with
context-based claims (Ben-Zvi et al., 2007). Secondly, although reasoning processes can be followed in
familiar circumstances, trouble arises when applying the reasoning in less familiar contexts (Rossman,
2008).

I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
14 of 27

Two other components that emerged in the SRTL5 discussions were developing models and looking for
causes. While these two components were not as universally applicable to all IIR situations, they were
considered important. The need to base IIR on probability models was reported by Rossman (2008), while
Watson (2008) expected reasons to be found for trends after data investigations in Tinkerplots. All the IIR
components described were recurring themes in the discussions. However, there was no general agreement
on one definitive set of components to form an IIR framework.

For a different approach to framing these components of IIR, consideration should be given to the space of
reasons formulated by Bakker et al. (2008) to describe the variety of issues involved in making inferences.
This space of reasons encompasses both contextual and statistical reasons, and includes implications,
evidence, conclusions, goals, purposes, utility and knowledge of cause and effect. They proposed that either
contextual reasons or statistical reasons could be prioritised in IIR depending on what was required to reach a
particular inference goal. The space of reasons was developed to make the point that IIR could also include
more complex components such as understanding the relationships between relevant variables and the causes
and effects of changing those variables.

I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
15 of 27

While the author has not sought to propose a specific IIR conceptual framework, what has been presented is
a collection of components identified by researchers as critical. This provides an important way forward in
determining what is involved in IIR but it is not enough. The design of learning experiences and assessment
of IIR needs to be informed by the articulation of the cognitive development of IIR to be more effective.
Next, SRTL5 research about the development of cognition involved in IIR is presented.

IIR Cognitive Development Framework
What is needed is a description of the path of cognitive development involved in IIR and how a students
progress on this path might be measured. Although, there was little research at SRTL5 that focused directly
on this need, the following proposes a starting point for the quest.

A framework was proposed by Zieffler et al. (2007, pp. 19-20) to analyse the nature and development of
students IIR. While the description of the four dimensions of the framework, i.e., fluency, consistency, prior
knowledge, and use of evidence, were cognitively relevant there was no indication of levels of cognition
within each of the dimensions. Descriptions of the dimensions included words such as extent and degree
but it was not clear how these were determined.

I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
16 of 27

A number of researchers shared specific examples of components of IIR that were conceptually more
difficult than other components. For example, Rossman (2008) described the increasing conceptual difficulty
in moving from dealing with sample proportions to understanding the role of sample size. As these specific
examples unfolded, the notion of a continuum of cognition pervaded the SRTL5 discussion. This developed
from the piecing together of discussions across a number of individual research reports that provided
snapshots of cognition at particular stage(s) in the developmental path of IIR. As the progressive path
evolved, it soon became apparent that the SRTL5 researchers believed that there were critical points in this
movement. These critical hurdles for students included the jumps: (i) from describing to arguing; (ii) from
descriptive statistics to data analysis; (iii) from being able to make conclusions to being able to make
conclusions under uncertainty; (iv) from addressing variability between groups to addressing variability
within groups and then both; and (v) from working with what is known to working with what is unknown.

One interesting complication identified amongst these developments was that students were jumping
backwards and forwards between using numbers and using proportions. Biehlers (2007) proposed learning
trajectory (simulations, then emphasizing interpretations, then stepwise introduction into hypothesis testing)
implies levels of cognition but is not explicit about these levels or how they could be used to assess the level
of cognitive development.
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
17 of 27

The only SRTL5 research that reported cognitive levels of IIR was Watsons (2008) work with Grade 7
students, which used the Structure of Observed Learning Outcome (SOLO) Taxonomy (Pegg, 2002) to
underpin descriptions of the levels of cognition in a beginning framework of IIR. Eight elements of IIR were
chosen (Watson, 2008) and the increasing levels of cognition (U-unistructural, M-multistructural and R-
relational) were determined by the sophistication and complexity of the manner in which these elements
were combined during IIR. The more desirable relational level of cognition was not achieved until the
various elements could be combined into a coherent argument.

The author now proposes a SOLO-based model that could be used to analyse the level of cognition in any
IIR situation. The model has two cycles of U-M-R levels, each with a different set of elements on which to
base the decision about the sophistication and complexity of the combination of elements. The first cycle is
based on the IIR foundation concepts (described earlier), e.g., viewing data as an aggregate or focusing on
proportions rather than absolutes. The level of cognition, U or M or R, is decided based on the sophistication
and complexity of the combination of these IIR components. Relational cognition is demonstrated when
these foundation concepts can be related well enough to allow successful engagement in the components of
IIR. At this stage nave inference can be attempted but it is generally not related to chance. The relational
level of Watsons (2008) framework is similar to the relational level of this first cycle of cognitive
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
18 of 27

development. The second cycle of the proposed SOLO-based model has the components of IIR (described
earlier) as the elements on which it is based. This cycle involves IIR that is based more on chance and
involves contextual reasoning.

The SRTL5 researchers concurred that measuring cognitive development of IIR is going to be difficult.
While the group were still coming to grips with exactly what it is that needs to be measured, they did agree
that the research methodology needed to include pre and post assessment, and implementation across a range
of tasks, all ages and at different times. Both Ben Zvi et al. (2007) and Zieffler et al. (2008) made it clear that
longitudinal studies are needed to be able to develop a cognitive development framework for IIR and to be
able to track the cognitive development of students. Technology will be useful to aid in the investigation of
how reasoning develops, as well as being a tool for supporting the reasoning.

Scaffolding IIR
Although researchers have only begun to describe the cognitive development of IIR, they have detailed
important scaffolds to support the development of IIR in specific learning situations. Instruction (teaching)
has been demonstrated to promote IIR (Paparistodemou & Meletiou-Mavrotheris, 2008) and four different
types of scaffolds were found to be effective.
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
19 of 27


The first type of scaffold involved the design of the task or experience being undertaken. Two previously
developed approaches were advocated; growing samples (Paparistodemou & Meletiou-Mavrotheris, 2008),
and the 3R Process, i.e., re-randomize data, repeat many times, reject model that puts observed in the tail
(Rossman, 2008). Two other approaches were also shown to have been successful. First, placing students in
a situation where they had to work backwards by mending an electronically simulated gadget before
making an inference helped the students to understand the connection from the data (samples) to the
population (Pratt et al., 2008). Second, inventing measures and then seeing how well they worked was a
useful step for students as part of the learning progression for data modelling, with further revisions of the
invented measures helping to bridge the conceptual gap between measurement and natural variation (Konold
et al. (2007). Importantly, Paparistodemou and Meletiou-Mavrotheris (2007) reminded us that emphasising
and revisiting helps students.

The second type of scaffold is the context in which the task or experience is undertaken. More powerful
experiences appear to occur in more authentic contexts. This applies to both the research and the IIR
experiences. Ben-Zvi (2007), like many others, advocated that teachers and researchers work in the
classroom. Biehler (2007) wanted students to be able to participate in random processes, so long as
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
20 of 27

interpretations are emphasised. Paparistodemou and Meletiou-Mavrotheris (2008) remind us that the interest
is high when students are allowed to use real data, as the investigative process (collecting and exploring)
helps students develop the big ideas.

The third type of scaffold relates to the teaching techniques used to support student work. Two critical
aspects of this are the language used and questioning techniques. The language needed for IIR should be
used early and students need to be engaged regularly in discourse that allows them to use that language
(Zieffler et al., 2007). Students should be allowed to use their own language initially when reasoning but as
soon as they are ready they should be introduced to the correct terminology, especially when it comes to
discussing uncertainty (Ben-Zvi et al., 2007). Even, young students are capable of using terms like more
likely and might be (Paparistodemou & Meletiou-Mavrotheris, 2008). Questioning is critical, both the
questions asked by the teacher and the questions the students need to ask (see, e.g., Makar & Rubin, 2007).
Part of the fear of encouraging questions is the chance that the dialogue will go into the realm of the
unknown. Despite this, teachers need to strategically structure questions to encourage a higher level of
cognition. Also, the use of metaphors was found to be very useful to assist teachers in developing their IIR.
One teacher proposed a soup metaphor, in relation to taking relative amounts (sampling), and this became,
along with a comparable stew metaphor, the focus of rich discussions with other teachers (Rubin &
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
21 of 27

Hammermann, 2007). Researchers also indicated that interactions and collaborations are important in
supporting the development of IIR (see, e.g., Paparistodemou & Meletiou-Mavrotheris, 2008).

The final type of scaffold is the technology used to support engagement with the task or experience.
Technology has greatly assisted by providing simulations (see, e.g., Zieffler et al., 2007) and real data sets to
test assumptions (Biehler, 2007), and in encouraging students to build/refine/reorganize their statistical
understanding (Paparistodemou & Meletiou-Mavrotheris, 2008). Purpose built, InferenceMaker (Johnston-
Wilder et al., 2007) provides a wealth of options to plan learning experiences with the Workings Box able
to be hidden from the user who is then required to predict. However, much of the SRTL5 research work
focused on how IIR was supported by specific software packages, in particular Tinkerplots and Fathom.
Tinkerplots has been shown to be useful in facilitating beginning inference (Ben-Zvi et al., 2007; Watson,
2007). The constructivist approach facilitated by Tinkerplots even allows students as young as eight to
explore inferential ideas (Paparistodemou & Meletiou-Mavrotheris, 2007). The flexibility in representations
of data in Tinkerplots allowed the young students to focus on their conceptual understanding. This is
facilitated because the software: (i) operates quickly and accurately; (ii) dynamically links multiple
representations; (iii) provides immediate feedback; and (iv) transforms entire representations into a
manipulable object (Paparistodemou & Meletiou-Mavrotheris, 2008). When exploring teachers thinking,
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
22 of 27

Rubin and Hammermann (2007) made use of both Tinkerplots and Fathom to allow teachers to explore data
sets. Sampler, a recent addition to Tinkerplots, improves the scope for sampling experiences by allowing
students to build models of a variety of probabilistic situations by filling a generic mixer (e.g., a spinner)
with labelled elements and then being able to specify how they want to sample from it (Konold et al., 2007).

This work has provided many ideas to help teachers to better support students development of IIR. Now
statistics education researchers need to focus on developing an IIR cognitive development framework.

Conclusion
Whilst it is not possible to ever replicate the richness of the experience of being part of the STRL Research
Forum discussions, the author has attempted to present a synthesis of the ideas that evolved as the
researchers discussed their research. The author has left it up to the reader to link the SRTL5 research to
other IIR research and/or to the readers own research. The wealth of research presented at SRTL5 is evident
in the above-described foundation concepts, IIR conceptual framework components, starting point for an IIR
cognitive development framework and scaffolds to support IIR.

I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
23 of 27

This provides useful information for educators planning learning sequences involving IIR but there is still a
need for better ways to assess the cognitive development of IIR. The big challenge now is for statistics
education researchers to study cognitive development when IIR is undertaken by a group of students in a
variety of tasks and at a number of different times. The two-cycle, SOLO-based, IIR cognitive development
framework proposed in this paper should be considered as a starting point when researchers need a
framework on which to build their own framework for assessing students IIR. Researchers are encouraged
to continue to share their investigations into the cognitive development of IIR so that better measurement
tools can be developed to more easily capture the level of cognition.

The author would like to thank all researchers who presented at SRTL5 for contributing their findings as a
stimulus for the engaging discussion and for substantially increasing the amount of valuable research
available in the field of IIR.

REFERENCES
[1] Bakker, A., Kent, P., Derry, J., Hoyles, C., & Noss, R. (2007). Making non-formal statistical inferences
to monitor production processes. Paper presented at the Fifth International Research Forum on
Statistical Reasoning, Thinking and Literacy, University of Warwick, United Kingdom. Online:
srtl.stat.aukland.ac.nz/srtlt5/
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
24 of 27

[2] Bakker, A., Kent, P., Derry, J., Noss, R., & Hoyles, C. (2008) Statistical inference at work: Statistical
process control as an example. Statistics Education Research Journal, 7(2), 130-145.
[3] Biehler, R. (2007). Challenging students informal inferential reasoning by means of smoothly
introducing p-value based hypothesis testing. Paper presented at the Fifth International Research
Forum on Statistical Reasoning, Thinking and Literacy, University of Warwick, United Kingdom.
Online: srtl.stat.aukland.ac.nz/srtlt5/
[4] Ben-Zvi, D., Gil, E., & Apel, N. (2007). What is hidden beyond the data? Helping young students to
reason and argue about some wider universe. Paper presented at the Fifth International Research
Forum on Statistical Reasoning, Thinking and Literacy, University of Warwick, United Kingdom.
Online: srtl.stat.aukland.ac.nz/srtlt5/
[5] Johnston-Wilder, P., Ainley, J., & Pratt, D. (2007). Thinking-in-change about informal inference.
Paper presented at the Fifth International Research Forum on Statistical Reasoning, Thinking and
Literacy, University of Warwick, United Kingdom. Online: srtl.stat.aukland.ac.nz/srtlt5/
[6] Konold, C., Kazak, S., Lehrer, R., & Kim, M. J. (2007). To understand a distribution, try building it
from scratch. Paper presented at the Fifth International Research Forum on Statistical Reasoning,
Thinking and Literacy, University of Warwick, United Kingdom. Online:
www.srtl.stat.aukland.ac.nz/srtlt5/
[7] Makar, K., & Rubin, A. (2007). Beyond the Bar graph: Teaching informal statistical inference in
primary school. Paper presented at the Fifth International Research Forum on Statistical Reasoning,
Thinking and Literacy, University of Warwick, United Kingdom. Online:
www.srtl.stat.aukland.ac.nz/srtlt5/
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
25 of 27

[8] Paparistodemou, E., & Meletiou-Mavrotheris, M. (2007). Enhancing reasoning about statistical
inference in 8 year-old students. Paper presented at the Fifth International Research Forum on
Statistical Reasoning, Thinking and Literacy, University of Warwick, United Kingdom. Online:
www.srtl.stat.aukland.ac.nz/srtlt5/
[9] Paparistodemou, E., & Meletiou-Mavrotheris, M. (2008). Developing young students informal
inference skills in data analysis. Statistics Education Research Journal, 7(2), 83-106.
[10] Pegg, J. (2002). Assessment in mathematics: A developmental approach. In J. M. Royer (Ed,).
Mathematical cognition (pp. 227-259). Greenwich, C.T.: Information Age Publishing.
[11] Pratt, D., Johnston-Wilder, P., Ainley, J., & Mason, J. (2008). Local and global thinking in statistical
inference. Statistics Education Research Journal, 7(2), 107-129.
[12] Rossman, A.J. (2008). Reasoning about informal statistical inference: A statisticians view. Statistics
Education Research Journal, 7(2), 5-19.
[13] Rubin, A. & Hammerman, J. (2007). Soup or stew: Metaphors for the relationship between samples and
populations. Paper presented at the Fifth International Research Forum on Statistical Reasoning,
Thinking and Literacy, University of Warwick, United Kingdom. Online: srtl.stat.aukland.ac.nz/srtlt5/
[14] Watson, J.M. (2007). Facilitating beginning inference with TinkerPlots for novice grade 7 students.
Paper presented at the Fifth International Research Forum on Statistical Reasoning, Thinking and
Literacy, University of Warwick, United Kingdom. Online: srtl.stat.aukland.ac.nz/srtlt5/
[15] Watson, J.M. (2008). Exploring beginning inference with novice grade 7 students. Statistics Education
Research Journal, 7(2), 59-82.
I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
26 of 27

[16] Zieffler, A. S., Garfield, J., delMas, R., & Gould, R. (2007). Studying the development of college
students informal reasoning about statistical inference. Paper presented at the Fifth International
Research Forum on Statistical Reasoning, Thinking and Literacy, University of Warwick, United
Kingdom. Online: srtl.stat.aukland.ac.nz/srtlt5/
[17] Zieffler, A., Garfield, J., delMas, R., & Reading, C. (2008). A framework to support research on
informal inferential reasoning. Statistics Education Research Journal, 7(2), 40-58.

I
P
M
s
S
e
s
s
i
o
n
s
B
I
P
M
s
S
e
s
s
i
o
n
s
T
Contents
Print
Page
27 of 27