This article was downloaded by: [University of Malaya] On: 25 November 2013, At: 01:28 Publisher: Routledge Informa

Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

The Journal of Educational Research

Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/vjer20

Effects of Calculators on Mathematics Achievement and Attitudes of Ninth-Grade Students

Sean Close , Elizabeth Oldham , Gerry Shiel , Therese Dooley & Michael OLeary
a b a b a a a

St. Patrick's College , Ireland

Trinity College , Ireland Published online: 10 Sep 2012.

To cite this article: Sean Close , Elizabeth Oldham , Gerry Shiel , Therese Dooley & Michael OLeary (2012) Effects of Calculators on Mathematics Achievement and Attitudes of Ninth-Grade Students, The Journal of Educational Research, 105:6, 377-390, DOI: 10.1080/00220671.2011.629857 To link to this article: http://dx.doi.org/10.1080/00220671.2011.629857

PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the Content) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions

The Journal of Educational Research, 105:377390, 2012 Copyright C Taylor & Francis Group, LLC ISSN: 0022-0671 print / 1940-0675 online DOI:10.1080/00220671.2011.629857

Effects of Calculators on Mathematics Achievement and Attitudes of Ninth-Grade Students

SEAN CLOSE
St. Patricks College, Ireland

ELIZABETH OLDHAM

GERRY SHIEL THERESE DOOLEY MICHAEL OLEARY

St. Patricks College, Ireland

Downloaded by [University of Malaya] at 01:28 25 November 2013

Trinity College, Ireland

ABSTRACT. Three calculator tests were administered to a national sample of 1,469 Irish students in Grade 9the last cohort to study mathematics without calculators (Phase 1). Three years later, the same tests were administered to a similar sample with calculators (Phase 2). Scores on a test of calculator-inappropriate items showed no signicant change over the 3 years. For a test of calculator optional items, students were divided randomly into 2 groups, 1 with calculator access and the other without. In both phases, the students with calculators achieved signicantly better than the students without calculators. Achievement on a test of calculator appropriate items showed signicant improvement over the 3 years. Students attitudes toward calculators also improved over the time. Keywords: achievement, attitudes, calculators, mathematics

two main sources. One source is large-scale survey research (usually done at a national or international level and often addressing other issues as well as calculator use) in which representative samples of students and/or teachers are asked, by means of questionnaire or interview instruments, about their attitudes to and uses of calculators in mathematics. The resulting data may be correlated with the students achievement in mathematics as well as with other characteristics. The other source is the experimental-type study, in which groups of students participate in controlled conditions in which calculators form a special part of the teaching, learning, and assessment involved. In these studies, students processes and outcomes may be monitored, and often compared with those of similar students following programs that do not involve calculators or involve different uses of them. Large-scale research surveys. A number of recent international and national surveys of mathematics education in schools provide us with information on calculator use within and across countries and its relation to achievement. The Third International Study of Mathematics and Science (TIMSS 95) involved over 40 countries at Grades 7 and 8 (Beaton et al., 1996). The study found that mathematics teachers in three quarters of the countries reported a high frequency of calculator use in their classes, often for checking answers, doing routine computation, and solving complex problems. However, in a fth of the countries little use was made of calculators in mathematics classes, including Korea, which scored highly on the mathematics tests, and Ireland. In the TIMSS1 surveys in 2003 and 2007, there was a wide variation in the reported degree of usage

lthough calculators have been widely and cheaply available for the past 30 years or so, there is still considerable debate about the appropriateness of their use in school mathematics, particularly in primary school and in the early years of secondary school. The issue is further complicated by developments in the technology. Different kinds of handheld calculators and computational devices have become available, ranging from simple arithmetic calculators, through scientic calculators and graphing calculators, to symbolic calculators and palm-top computers with a variety of calculating modes and functions including algebra systems and spreadsheets. The purpose of the present study was to examine the effects of the use of scientic calculators in mathematics in the rst 3 years of secondary school (i.e., in the Irish Junior Certicate mathematics curriculum and accompanying examinations), with a particular focus on Grade 9. Review of Research Much of the research data relating to the use of calculators in the context of school mathematics comes from

Address correspondence to Sean Close, Educational Research Centre, St. Patricks College, Drumcondra, Dublin, 9 Ireland. (E-mail: sean.close@erc.ie)

378

The Journal of Educational Research

of calculators in Grade 8 across countries and no clear-cut relationship between achievement in mathematics and degree of calculator usage in mathematics classes (Mullis, Martin, & Foy, 2008; Mullis, Martin, Gonzalez, & Chrostowski, 2004). The Organisation for Economic Co-operation and Developments Programme for International Student Assessment (PISA) involved carrying out international surveys of the mathematical knowledge of 15-year-olds in 2000, 2003, 2006, and 2009 that were designed to test mathematical skills in realistic problem contexts or situations, and were not constrained by the need to reect the mathematics curricula of the participating countries. PISA permits calculator usage throughout the mathematics test. In PISA 2000 and PISA 2003, it was found that the percentages of students who used calculators in mathematics classes differed across countries, and differed between survey dates (Organisation for Economic Co-operation and Development, 2001, 2004). An example of the latter nding was Ireland. Whereas only 24% of the students in the Irish PISA 2000 survey said they used calculators frequently in mathematics classes (Shiel, Cosgrove, Sofroniou, & Kelly, 2001), 78% of the students in the 2003 survey said that they did so (Cosgrove, Shiel, Sofroniou, Zastrutzki, & Shortt, 2005). This reects the revision, in the interim, of the Junior Certicate mathematics curriculum and examination to include calculators. It is worth noting that this dramatic increase in calculator availability and experience with their use between 2000 and 2003 was not accompanied by any signicant improvement or deterioration in average scores on PISA mathematics for Irish 15-year-olds, either in absolute terms or relative to other countries. It should also be reiterated that the items included in the TIMSS and PISA surveys do not require usage of a calculator (i.e., they could be expected to be done by most students using pen-and-paper and mental methods and are referred to hereafter in this study as calculator optional items). In the 2009 National Assessment of Educational Progress in the United States, 66% of fourth-grade students said they never used a calculator for tests or quizzes, 33% said that they used one sometimes, and 2% said they used one always (National Center of Educational Statistics [NCES], 2009). In the same survey, 28% of eighth-grade students said they never used a calculator, 51% that they used one sometimes, and 21% that they used one always (NCES, 2009). The NAEP test itself allows calculators for one third of items at Grades 4 and 8, and these are referred to as calculator active items. In the United Kingdom, use of calculators is recommended from upper primary school onwards. In reviewing research in the United Kingdom on the effects of calculators on attainment in numeracy Ruthven (1997) concluded that calculator use remains modest in most UK schools and by most pupils and that the evidence suggests that calculators cannot be blamed for the disappointing mathematical performance at primary level in particular. In recent years students taking the Key Stage 3 (Year Nine) national test in

mathematics can use calculators for half of the test, though the test is no longer mandatory. Experimental studies. With the advent of calculators in schools across most countries in the 1980s and 1990s, many experimental studies were carried out to examine the effects of calculators on mathematics achievement and attitude. These studies, too numerous to report here, varied considerably in terms of size of samples, grade levels included, and type and range of tasks or test items involved; however, all involved some form of control of the use of calculators in the teaching/learning or testing of mathematics. Most of these were supportive of (nongraphing) calculator use in mathematics classrooms in primary and secondary schools (Dunham, 2000). Hembree and Dessart (1986, 1992) and Smith (1997) carried out meta-analyses involving many of these studies that met specied statistical criteria. Based on analyses of effect sizes, they concluded that students who used calculators possessed more positive attitudes and had stronger self-concepts in mathematics than non-calculator users, and that testing with calculators produced higher achievement scores at all grades and ability levels. A later meta-analysis of 54 studies of the effects of calculators in Grades K12 by Ellington (2003) showed that operational and problem-solving skills improved when calculators were an integral part of teaching and testing and that students using calculators had better attitudes to mathematics than those not using them. In general, the body of research involving controlled studies says that students who use calculators perform better on pen-and-paper tests of basic skills and problem solving than students who do not have access to calculators when doing such tests.

Downloaded by [University of Malaya] at 01:28 25 November 2013

Context and Goals of the Study A revised Junior Certicate mathematics syllabus, with provision for using calculators in curriculum and assessment, was introduced in 2000 for rst examination in 2003 (Department of Education and Science/National Council for Curriculum and Assessment, 2000, 2001, 2002). The types of calculators sanctioned for use in the state mathematics examinations at the end of Grade 9 are four-function and scientic (nonprogrammable) ones, though there is no embargo on the use of other calculators, including graphing calculators, as teaching and learning tools. The introduction of calculators into the Junior Certicate mathematics curriculum provided opportunities for developments in teaching and learning, and for improvements in mathematics performance, particularly in conceptual knowledge and in problem solving and applications. However, it also raised concerns about the maintenance of computational skills for which calculator use is not appropriate. It was as a result of these concerns that the present study took place. The intent of the study was to assess students levels of performance in calculator-related areas of the mathematics

The Journal of Educational Research

379

curriculum, both before and after the introduction of the revised Junior Certicate syllabus and examinations. Specic goals. Greenes and Rigol (1992) classied the items used in U.S. College Board standardized tests into three main types based on calculator relevance: calculator inactive items in which there is no advantage to using a calculator, calculator neutral items that can be solved without a calculator but for which a calculator may be used, and calculator active items in which access to a calculator might be expected to convey an advantage. For the purposes of the present study, the rst category of item was not considered relevant and the second category, calculator neutral items, was seen as being composed of two types of item: (a) that which ninth-grade students should be able to do without a calculator, and therefore for which a calculator is inappropriate (e.g., 7 5); and (b) that for which use of a calculator might be advantageous, though not necessary, and so could be considered optional (e.g., 1.25 8). The calculator optional items were thus seen as the items on which the effects of calculator access could be directly measured by assigning, in a random manner, one half of the students to do them with a calculator and the other half to do them without a calculator. Three tests were developed to assess different aspects of students mathematical skills in the presence or absence of a calculator. These are described as the following: A calculator inappropriate test taken without access to a calculator by all students in the sample in each phase, and containing items that students should be able to do without a calculator: for example, (3 4)/2. A calculator optional test taken by half the sample, in each phase, with access to a calculator, and by the other half without such access, and containing items that students should be able to do with or without a calculator: for example, (3.1 25)/2. A calculator appropriate test taken by all students in each phase, with access to a calculator, and containing items that would normally require a calculator: for example, (3.12 24.75)/0.2052.2 Questionnaires were also prepared to assess students attitudes to and use of calculators before and after their introduction into the curriculum. Arising from the previous considerations, the following research objectives were formulated: 1. To examine, before the introduction of calculators in the Junior Certicate mathematics curriculum (Phase 1) and after their introduction (Phase 2), the performances of Grade 9 cohorts on the three calculator tests. 2. To examine, in both phases, the effects of calculator access versus nonaccess during the testing of student performance on the calculator optional test, and to compare the results. 3. To examine the attitudes of students in both phases toward calculator usage and the relationships between atti-

tudes and performance of the students on the three calculator tests. 4. To examine the relationship between the performance of students on the three calculator tests in both phases and their performance on the corresponding Junior Certicate mathematics examinations.

Method The study was carried out in two phases. The rst phase involved the administration of three types of calculator-related mathematics tests (calculator inappropriate, calculator optional, and calculator appropriate) and questionnaires to a nationally representative sample of the nal cohort of ninthgrade students to experience a mathematics curriculum in which calculators were not included. The second phase involved the administration of the same three tests and revised questionnaires to a nationally representative sample of the third cohort of ninth-grade students to experience a mathematics curriculum in which calculators were included. In this way the effects of calculator availability on mathematics performance and attitudes could be studied in a quasicontrolled study in which the performance and attitudes of a sample of the students who experienced a noncalculator mathematics curriculumthe preintervention group (Phase 1)could be compared with an equivalent group who subsequently experienced a calculator-friendly mathematics curriculumthe postintervention group (Phase 2). Concerns about the validity of these comparisons between phases included (a) possible confounding effects of any content changes in the revised curriculum (though these were minor); (b) changes between phases in the numbers taking the three course levels at which Junior Certicate mathematics is available (higher, ordinary, and foundation); (c) modest school-level response rates of 73% in both phases; and (d) possible differential effects, across the two phases, of demographic or cultural trends. Steps were taken to allow for the second and third of these factors, including statistical weighting of data, but results should be viewed in the light of these possible limitations. Within this larger study, a second more specic study was implemented, in each phase, in which the students taking the calculator optional test were randomly assigned to two test conditions, one without calculators, and the other with calculators, to examine further the effects of calculators on items for which calculators may or may not be used. Figure 1 summarizes the design of the study. Development of the Calculator Tests As described previously, the design for the test specied three measures: a calculator inappropriate test (in which calculators were not available to any students), a calculator optional test (in which calculators were available to half of

Downloaded by [University of Malaya] at 01:28 25 November 2013

380

The Journal of Educational Research

Study Phase

Calculator Tests Taken

Calculator

Test Condition

No access to calculators for test

Phase 1

inappropriate test

Preintervention group Experienced a mathematics curriculum in Grades 7 to 9

Calculator optional test

Half of sample No access to calculators for test

Half of sample Access to calculators for test

Downloaded by [University of Malaya] at 01:28 25 November 2013

that did not include calculators

Calculator appropriate test Access to calculators for test

Revised calculator-friendly curriculum introduced in 2000 and examined in 2003 for first time

Calculator Phase 2 inappropriate test Calculator Postintervention group Experienced a mathematics curriculum in Grades 7 to 9 that included calculators Calculator appropriate test optional test

No access to calculators for test Half of sample No access to calculators for test Half of sample Access to calculators for test

Access to calculators for test

FIGURE 1. Overall design of the study.

the cohort and not available to the other half), and a calculator appropriate test. Test items fell into two cognitive process categories specied in the curriculum: those that assessed knowledge of mathematical facts, procedures, and concepts, and those that assessed knowledge of applications to real-life contexts. They included multiple-choice (one third) and short constructed-response items (two thirds), and displayed an overall gradient of difculty; that is, overall, the calculator inappropriate test was intended to be easier than the calculator optional test, which, in turn, was

intended to be easier than the calculator appropriate test. Test items focused chiey on assessing the Junior Certicate syllabus content area applied arithmetic and measure (because of its relevance for the use of real-life data), followed by number systems and statistics (as these are the most calculator sensitive topics accessible to all ninth-grade students), and to a limited extent on algebra (mainly on the solution of simple equations; Table 1). Items were located in textbooks, tests, and Junior Certicate examination papers (foundation, ordinary and higher

The Journal of Educational Research

381

TABLE 1. Number of Items in Each Calculator Test, by Mathematics Content Area Number systems 13 12 7 Applied arithmetic and measure 10 15 15

Test Calculator inappropriate Calculator optional Calculator appropriate

Algebra 1 4 0

Statistics 1 2 8

Total 25 33 30

Note. These data refer to items used in Phase 2.

Downloaded by [University of Malaya] at 01:28 25 November 2013

levels), or written when necessary, and were then assembled into tests. Two pilot studies were conducted: one in a convenience sample of seven schools, and a second in a more representative sample of 15 schools. Items were selected for the main study tests on the basis of item facilities and discrimination coefcients. Reliability coefcients (Cronbachs alpha) for the three tests ranged between .85 and .88. A high proportion of the items on the calculator inappropriate test emphasized basic numerical skills, while the calculator appropriate test contained items predominantly of the applied type (emphasizing the use of the calculator in practical contexts). The calculator optional test included a combination of basic and applied-type items. Within tests, consideration was also given to the placement of items by content area. Where appropriate, items testing a given content area were grouped together, to avoid arbitrary shifts of focus from one topic to another. Sample items from the tests, along with estimates of their difculty levels, are given in Figure 2. Development of the Student Questionnaire A student questionnaire was prepared for Phase 1 and developed further for Phase 2. It was designed to investigate variables that might be associated with student performance on the tests, and to provide background data on participating students. As well as background information, the questionnaire sought information on students calculator usage at home and at school in other subjects as well as mathematics, and asked about attitudes to mathematics in general and toward calculator usage in particular. In Phase 2, additional questions were asked to investigate students experience of calculator use in the revised curriculum.

full-time special (resource) classes in ordinary schools (under 3% of students), were excluded. The departments database provided a listing of schools and of the numbers of male and female students in Grade 9 in each school. All schools were considered for inclusion in the study. Schools were stratied by type (secondary, vocational, community/comprehensive) and size. Within each stratum, schools were sorted by the percentage of female students in Grade 9 and by school size. Schools were then selected using probability proportional to size systematic sampling. Replacement schools were also selected during this procedure. One class in each school was then selected at random to participate in the study. Of the 90 schools selected for Phase I (including replacements), 60 agreed to participate, and of the 100 schools selected for Phase II, 73 agreed to participate. Given the tight time frame within which the study was conducted, it was not possible to recruit additional replacement schools in Phase 2. This meant that 1,453 students in Phase 1 and 1,459 students in Phase 2 completed the calculator tests. Weights were computed and applied to compensate for the somewhat unequal distribution of students in different strata in the sample, using procedures applied in the 1995 TIMSS study, which also involved sampling of intact classes in schools (see Foy, 1997).

Implementation in Schools Participating schools appointed coordinators to oversee the administration of the tests. In all cases, the 30-min calculator inappropriate test was administered rst. Students did not have access to a calculator for this test. After a short break, the 40-min calculator optional test was administered to students. The calculator and noncalculator versions of the test (only the directions with respect to calculator usage were different) were distributed to alternate students to ensure random assignment to the two calculator conditions.3 After another short break, the 25-min calculator appropriate test (which had been divided into two parallel forms) was administered. Following another short break, the student questionnaire was administered.

The Sample of Schools and Students Both phases of the calculator study were implemented at the same time of year (November) so that valid comparisons could be drawn between the two samples. The target population consisted of students in Grade 9 in schools on the Department of Education and Sciences postprimary schools database. Students in special schools, or in

382

The Journal of Educational Research

Calculator Inappropriate Items Jane bought a CD for 5 and sold it for 7. What was her percentage profit? (A) 2% (B) 4% (C) 20% (D) 40% A class has 25 students. The ratio of boys to girls is 3:2. How many girls are in the class? Answer_______________

Applied Arithmetic and Measure Moderately Difficult* (45%) Number Systems Average Difficulty (51%)

Calculator Optional Items A pack of 120 identical cards is 3 cm thick. How thick is one card? (A) 0.0025 cm (C) 0.25 cm (B) 0.025 cm (D) 0.4 cm

Applied Arithmetic and Measure With CalculatorAverage Difficulty (65%) Without Calculator Moderately Difficult (40%)

Downloaded by [University of Malaya] at 01:28 25 November 2013

Multiply:

6.4 2.5

Answer______________

Number Systems With CalculatorVery Easy (94%) Without CalculatorAverage Difficulty (64%)

Calculator Appropriate Items Evaluate: (9.8)3 (29.2)2 0.0025 Answer______________ How many 700 millilitre bottles of water can be filled from a 350 litre barrel? Answer______________ Number Systems Highly Difficult (26%)

Applied Arithmetic and Measure Highly Difficult (28%)

FIGURE 2. Sample items for each calculator test. Difculty level was calculated as the percentage of students responding correctly to the item. Percentage correct scores are weighted. The following descriptors are used to interpret item difculties: Very Easy (80%+); Moderately Easy (70%79%); Average (50%69%); Moderately Difcult (40%49%); Highly Difcult (below 40%). These percentages are based on Phase 2 results.

Scaling the Calculator Tests In Phase 1, item response theory (IRT) methodology was used to derive scores for each examinee on the calculator inappropriate, calculator optional, and calculator appropriate tests. A modied three-parameter (3PL) model was used to scale the calculator inappropriate test, as it consisted of multiple-choice (16) and short-answer (nine) items. The modication was achieved by setting strong prior distributions on the guessing parameter for short-answer items. A 2PL model was implemented for the calculator optional test, as more items were of the short answer variety, and the 3PL model revealed a poorer t to the data. A 2PL model was also used for the calculator appropriate test A maximum likelihood approach was used in item calibration, with not-reached items being treated as not attempted. For the calculator optional test, item parameter estimates were

based on the responses of those who took the test without a calculator. In assessing item t, item characteristic curves (ICCs) were examined visually and a small number of poortting items omitted from scaling. In calculating examinee estimates, not-reached items were treated as incorrect. The Phase 1 student logit scores for each of the tests were transformed to a scale with a mean of 250 and a standard deviation of 50 to facilitate reporting. The scaling parameters derived in Phase 1 were applied to the Phase 2 data. In this way scores derived from both administrations of the tests can be considered comparable. Analysis of the Data Mean raw scores, scale scores,4 percentage correct scores, and percentages of students, which are used to report results in the next section, are weighted population estimates that

The Journal of Educational Research

383

TABLE 2. Mean Scale Scores, Standard Errors, and Difference Scores of Calculator Group Versus Noncalculator Group on the Calculator Inappropriate and Calculator Appropriate Tests Calculator inappropriate Phase 1 Calculator optional Calculator access No calculator access Difference (SE) Score 249.6 250.4 0.89 SE 3.97 4.84 1.94 Phase 2 Score 241.6 244.8 3.21 SE 3.97 3.59 1.64 Calculator appropriate Phase 1 Score 250.5 249.5 0.95 SE 4.49 4.62 1.94 Phase 2 Score 262.4 264.5 2.11 SE 3.98 3.97 2.33

Note. For calculator inappropriate, 95% condence intervals were [4.83, 3.06] and [6.54, 0.12] for Phases 1 and 2, respectively. For calculator appropriate, 95% condence intervals were [3.00, 4.90] and [6.82, 2.61] for Phases 1 and 2, respectively.

Downloaded by [University of Malaya] at 01:28 25 November 2013

take into account the unequal representation of students from different schools and school types in the sample. They were obtained by applying weights to students scores during analysis. Mean and percentage scores in this report are accompanied by corresponding standard errors of measurement, which were computed using the WesVar statistical package (Westat, 2007) to take into account the nested nature of the sample (i.e., pupils were clustered within classes and schools). Differences between mean scores were also tested for signicance in WesVar. Factor analysis was used to analyze questionnaire responses on attitudes toward calculators and to generate factor scores. The signicance of correlation coefcients was evaluated using multiple regression in WesVar. In both phases, prior to examining the effects of calculators on performance of the calculator group relative to the noncalculator group on the calculator optional test, student scores on the calculator inappropriate and calculator appropriate tests were used as measures of mathematics ability to investigate the extent to which students in the two groups were of equal overall ability in mathematics. In both phases, the small, insignicant differences between the mean scale scores on the calculator inappropriate and calculator appropriate tests, of those who had access to calculators for the

calculator optional test, compared with those who did not, indicated that the two groups were well matched in terms of overall ability in mathematics (Table 2). Results Here we give a brief description of the results of the study over its two phases along with comments where appropriate. As described previously, the three calculator tests were scaled separately in Phase 1, and the mean scale score and standard deviation for each test was set at 250 and 50, respectively, while scores on the Phase 2 tests were linked to the Phase 1 scales. Results are presented rst by overall performance on each test and then by content areas and key items. Results are also reported for the student questionnaire, and associations between performance on the calculator tests and on the Junior Certicate mathematics examination are considered. Performance on the Three Calculator Tests for Phase 1 and Phase 2 The mean scale scores and mean percentage correct scores for the three tests for Phase 1 and Phase 2, along with the results of tests of the signicance of the differences and effect sizes, are given in Table 3.

TABLE 3. Mean Scale Scores, Standard Errors, and Difference Scores on the Calculator Inappropriate, Calculator Optional, and Calculator Appropriate Tests (Phase 1 and Phase 2) Phase 1 Test Calculator inappropriate Calculator optional (access) Calculator optional (no access) Calculator appropriate Score 250.0 266.2 235.8 250.0 SD 50.00 42.84 51.39 50.00 SE Score 4.54 3.36 4.68 4.45 243.2 271.5 227.1 263.4 Phase 2 SD 48.98 48.30 46.56 50.48 Phase 2 Phase 1 SE Difference SE 95% condence interval d (Effect size) 4.48 4.59 5.35 2.39 6.80 5.27 8.70 13.4 6.32 4.87 7.11 5.05 [5.79, 19.45] [15.00, 4.46] [5.50, 22.90] [23.50, 3.34] 0.14 0.12 0.18 0.26

Note. Condence intervals around statistically signicant differences are in bold.

384

The Journal of Educational Research

Downloaded by [University of Malaya] at 01:28 25 November 2013

Performance on both the calculator inappropriate test, and on the calculator optional test for the no calculator condition, declined slightly between Phase 1 and Phase 2by 6.8 scale points in the case of the former and by 8.7 in the case of the latter. However, these differences were not statistically signicant. On the other hand, performance improved by 13.4 points on the calculator appropriate test, and by 5.3 on the calculator optional test with calculator access, with the larger difference being statistically signicant. Altogether, therefore, performance without calculators declined slightly but not signicantly, and performance with calculators increased. The nding of no signicant difference between the two phases on the calculator inappropriate test suggests that students in Phase 2 did not lose out signicantly on basic mathematical skills while following the revised Junior Cycle mathematics curriculum, compared with Phase 1 students who followed the previous curriculum. The improvement in performance on the calculator appropriate test, which includes the items most likely to bring calculators into play, suggests that students ability to make use of the calculator in solving problems improved over the 3 years. While, as described previously, items on this test were intended to be somewhat challenging, the overall low performance in Phase 1 and Phase 2 (with percentage correct scores of 35.0% and 42.6%, respectively) raises concerns about how well students can use their mathematical knowledge to solve practical problems. Effects of Access to Calculators on Performance on the Calculator Optional Test for Phase 1 and Phase 2 In Phase 1, the difference in overall achievement between students with and without access to a calculator on the calculator optional test was 30.4 scale points, and in Phase 2 it was 44.4 (Table 4).5 On both occasions the difference was in favor of the calculator access group. Both of these differences are statistically signicant, showing clearly the advantage conferred on students with access to calculators compared with those with no access. The effect size for Phase 1 is medium (.64) and for Phase 2 is large (.93; Cohen, 1988). The nding that the difference increased over time is in line with results from other studies showing that students

test scores in mathematics improve to a signicant extent when calculators are made available to them.

Performance on Mathematics Content Areas in Phase 1 and Phase 2 As described previously, each item on the calculator tests was categorized according to the mathematics content area of the syllabus it addressednumber systems, applied arithmetic and measure, algebra, and statistics. Percentage correct scores on subsets of items on the calculator inappropriate test in Phase 1 were 64.5% for number systems and 53.1% for applied arithmetic and measure (Table 5). The corresponding gures for Phase 2 were 62.1% (number systems) and 49.8% (applied arithmetic and measure). The differences between Phase 1 and Phase 2 were not statistically signicant for either of these content areas. This supports the earlier nding that in mathematics curricula with calculator availability there is no signicant loss in ability to do important basic mathematical tasks without a calculator. It should be noted that, on this test, there was just one item each in algebra and statistics, so, for these areas, the data represent item rather than subtest statistics. On the calculator optional test, the Phase 2 group who had access to calculators did signicantly better than the Phase 1 group with access on number systems and algebra, and slightly better (though not signicantly so) on applied arithmetic and measure, and statistics (Table 6). This nding indicates that students were able to use calculators with somewhat greater effect on calculator optional tasks in number systems and algebra in Phase 2 than in Phase 1. Students without access to a calculator for this test did signicantly less well on number systems in Phase 2 than in Phase 1 (Table 7). In Phase 1 and Phase 2, items on the calculator appropriate test were more difcult for students than items on the other tests, giving rise to a steeper gradient of difculty than had been intended in the design of the study. Items that caused most difculty were mainly in the area of applied arithmetic and measure. However, there was a signicant

TABLE 4. Mean Scale Scores, Standard Errors, and Difference Scores for Phase 1 and Phase 2 on the Calculator Optional Test, by Access to Calculator Phase 1 Calculator optional Calculator access No calculator access n 731 732 M 266.2 235.8 SD 42.84 51.39 SE 3.36 4.68 n 742 718 M 271.5 227.1 Phase 2 SD 48.30 46.56 SE 4.59 5.35

Note. For Phase 1, Difference = 30.4, SE = 5.76, 95% condence interval = [19.11, 41.69] (signicant), effect size = 0.64. For Phase 2, Difference = 44.4, SE = 7.05, 95% condence interval = [38.44, 58.21] (signicant), effect size = 0.93.

The Journal of Educational Research

385

TABLE 5. Mean Percentage Correct Scores, Standard Errors, and Differences Scores on the Calculator Inappropriate Test, by Mathematics Content Area (Phase 1 and Phase 2) Phase 1 Content area (no. of items) Number systems (13 items) Arithmetic/Measure (10 items) Algebra (1 item) Statistics (1 item) % 64.5 53.1 63.7 67.6 SD 23.8 27.0 48.1 46.8 SE 2.1 2.1 3.1 2.7 % 62.1 49.8 56.2 64.1 Phase 2 SD 23.7 25.5 49.6 48.0 SE 1.5 2.2 0.2 3.2 Difference 2.4 3.3 7.5 3.4 SE 2.6 3.0 3.1 4.2 Phase 2 Phase 1 95% condence interval [2.7, 7.5] [2.8, 9.4] [1.3, 13.7] [4.9, 11.8] d (Effect size) 0.10 0.13 0.15 0.07

Note. Alpha level adjusted for multiple comparisons. Condence intervals around statistically signicant differences are in bold.

Downloaded by [University of Malaya] at 01:28 25 November 2013

increase in percentage correct scores on number systems between Phase 1 and Phase 2 reecting, perhaps, increasing facility in using the calculator on items in this area (Table 8). There were no algebra items on the calculator appropriate test. Student Variables and Performance The student questionnaire was administered to students participating in each phase of the study. A brief account of the results is presented here. Students use of calculators. In Phase 1, 55.8% of students reported that they had access to a calculator at school, whereas practically all students in Phase 2 had access to one. By Phase 2, scientic calculators were most frequently owned, and fewer than 1% of students used or had access to a graphing calculator. Students were asked about the frequency of calculator use in four subjects: mathematics, business studies, science, and technology. In Phase 2, 81% of students reported using a calculator often in mathematics classes (compared with just 1% in Phase 1) and 62% said they did so with the same frequency in business studies (45% in Phase 1). Fewer than 3% in Phase 2 reported using a calculator often in

science (the same as in Phase 1), and three quarters of the respondents studying technology in Phase 2 reported that they never used one in that subject, whereas 65% in Phase 1 said they never did. When asked about calculator usage in elementary school, 72% of students in Phase 2 reported that they had never used a calculator in their primary mathematics classes, whereas 3% reported having used one often. However, most students in the Phase 2 study would not have been taught under the revised 1999 Primary School Mathematics Curriculum, which permits calculators in senior primary school (Grades 46), as implementation did not begin in earnest until 20022003. With regard to the frequency of calculator usage in mathematics classes at postprimary level, just over 10% of students in Phase 2 reported that they never used a calculator in their Grade 7 mathematics classes, though for Grade 8 this gure dropped to 5%. The frequency of reported calculator use in some mathematics content areas is displayed in Table 9. In general, students who reported using a calculator a lot in a particular area tended to achieve higher scores on the calculator tests than students who did not. For example, on the calculator inappropriate test, those who reported using calculators a lot on length, area, volume, and time had a mean score (246.4) that was signicantly higher than those

TABLE 6. Mean Percentage Correct Scores, Standard Errors, and Difference Scores on Mathematics Content Areas on the Calculator Optional Test Calculators Available (Phase 1 and Phase 2) Phase 1 Content area (no. of items) Number systems (12 items) Arithmetic/Measure (15 items) Algebra (4 items) Statistics (2 items) % 74.1 52.4 45.4 55.5 SD 16.8 23.5 34.4 34.7 SE 1.1 1.9 2.9 2.5 % 78.2 54.5 56.8 56.9 Phase 2 SD 16.5 22.9 33.0 31.8 SE 1.1 2.0 1.8 3.0 Difference 4.1 2.1 10.5 1.4 SE 1.6 2.7 3.4 3.9 Phase 2 Phase 1 95% condence interval [7.2, 1.0] [7.6, 3.3] [17.2, 3.7] [9.2, 6.3] d (Effect size) 0.34 0.09 0.34 0.04

Note. Alpha level adjusted for multiple comparisons. Condence intervals around statistically signicant differences are in bold.

386

The Journal of Educational Research

TABLE 7. Mean Percentage Correct Scores, Standard Deviations, Standard Errors, and Difference Scores on Mathematics Content Areas on the Calculator Optional Test No Calculators Available (Phase 1 and Phase 2) Phase 1 Content area (no. of items) Number systems (12 items) Arithmetic/Measure (15 items) Algebra (4 items) Statistics (2 items) % 53.2 45.6 38.4 48.0 SD 23.0 22.2 34.9 32.7 SE 2.0 1.9 3.2 2.6 % 45.4 41.8 40.8 46.6 Phase 2 SD 22.7 20.9 33.0 31.8 SE 1.7 2.9 1.4 2.2 Difference 7.8 3.7 2.4 1.3 SE 2.6 3.5 3.4 3.4 Phase 2 Phase 1 95% condence interval [2.5, 13.1] [3.2, 10.7] [9.3, 4.4] [5.5, 8.1] d (Effect size) 0.34 0.18 0.07 0.06

Note. Alpha level adjusted for multiple comparisons. Condence intervals around statistically signicant differences are in bold.

Downloaded by [University of Malaya] at 01:28 25 November 2013

reporting that they never used one (228.8; difference = 17.8; SE = 3.56; 95% condence interval [CI; 2.45, 33.10]). Students attitudes to calculators. The proportions of students who agreed or strongly agreed that a calculator can help with performance and that its use should be allowed for classwork and homework in mathematics increased from around two thirds in Phase 1 to over 90% in Phase 2 (Table 10). About three quarters of students in Phase 2 believed that calculators were not only for students experiencing a lot of difculty in mathematics, and that a calculator should not replace the need to learn pen-and-paper calculations. The items in Table 10 were subjected to (maximum likelihood) factor analysis in both phases. A decision on how many factors to retain was taken using parallel analysis (OConnor, 2000).6 Following direct oblimin rotation, the pattern and structure matrices (see the Appendix) were examined to better understand each retained factor. The three factors identied in both phases were support for calculator use in general, negative effects of calculator use, and positive effects of calculator use on mathematics. Together, these explained 57.8% of the variance in Phase 2 (Table 11). Scale scores were computed using the regression method. Respective Cronbachs alpha reliabilities in this phase were .83, .58, and .79. The relatively low reliability of the second factor

scale may have resulted from the inability of some students to interpret the negatively worded items as intended. Phase 2 students who reported higher levels of support for the availability of calculators across subjects, including mathematics, tended to perform better on the calculator appropriate test than those reporting lower levels of support (r = .10, p < .01). On the other hand, students who were more negatively disposed to calculator usage, and felt it could make them lazy at school mathematics, tended to do less well on the calculator inappropriate (r = .15, p < .01), calculator optional (without calculators; r = .16, p < .001), and calculator appropriate (r = .08, p < .01) tests than students who did not hold such views. Students supporting positive effects of calculator usage in mathematics tended to do less well on all of the calculator tests (Table 11) than students not expecting such effects. Aspects of calculator use that Phase 2 students enjoyed most were the ease and speed of computation and the convenience for basic operations, algebra, and fractions. The main perceived disadvantages of calculator use were the greater potential for making mistakes, difculty in using the calculator, and a fear that calculators do not engage the brain. The proportion of students who believed a calculator could make them lazy at school mathematics decreased from 55% in Phase 1 to 41% in Phase 2, perhaps owing to their greater

TABLE 8. Mean Percentage Correct Scores, Standard Deviations, Standard Errors, and Difference Scores on Mathematics Content Areas on the Calculator Appropriate Test (Phase 1 and Phase 2) Phase 1 Content area (no. of items) % SD SE % Phase 2 SD SE Difference 13.05 1.29 2.99 SE 2.14 2.14 2.27 Phase 2 Phase 1 95% condence interval d (Effect size) [17.33, 8.77] [5.56, 2.98] [7.52, 1.54] 0.41 0.06 0.08

Number systems (7 items) 39.7 33.0 2.11 52.8 30.2 0.34 Arithmetic/measure (15 items) 33.3 21.4 1.67 34.6 19.9 1.34 Algebra (0 items) Statistics (8 items) 43.3 40.5 2.14 46.3 35.3 0.75

Note. Alpha level adjusted for multiple comparisons. Condence intervals around statistically signicant differences are in bold.

The Journal of Educational Research

387

TABLE 9. Mean Percentage of Students Indicating Various Levels of Calculator Usage in Different Mathematics Content Areas (Phase 2) Content area Fractions, decimals, and percentages Length, area, volume, and time Algebra Statistics Geometry Trigonometry Graphs n 1,409 1,402 1,407 1,394 1,361 1,332 1,398 A lot 52.0 54.2 25.0 30.6 23.1 42.3 13.9 SE 1.86 2.08 1.79 2.03 1.66 2.21 1.33 To some extent 42.6 40.9 49.6 56.8 53.0 42.9 50.2 SE 1.57 1.84 2.04 1.93 1.86 2.07 2.17 Never 5.4 4.9 25.5 12.7 24.0 14.8 36.0 SE 0.73 0.90 2.00 1.13 2.06 1.39 1.95

Note. For the content area topics, these topics, rather than the content areas referred to in the syllabus, were used in the student questionnaire.

Downloaded by [University of Malaya] at 01:28 25 November 2013

familiarity with calculators and their better appreciation of the benets of calculators in doing mathematics. Students Performance on the Calculator Tests and on the Junior Certicate Mathematics Examination As part of the study, the scores of students in the Phase 1 and Phase 2 samples on the three calculator tests were related to their performance on the state (Junior Certicate) examination in mathematics taken some 6 months after the calculator tests. Junior Certicate performance was examined with respect to examination level (higher, ordinary, or foundation) and performance score. In computing a students performance score, a students grade on the examination was converted to a points score, allowing for the placement of students taking different levels of the test on the same underlying scale. Points were on a scale of 1 (Grade F on foundation level) to 6 (Grade D on ordinary level) to 12 (Grade A on higher level). These scores were then correlated with student performance on the calculator tests. As in Phase 1, strong positive correlations were found between Phase 2

students Junior Certicate mathematics performance scores and their scores on the calculator inappropriate (r = .70, p < .001), calculator optional (r = .66, p < .001), and calculator appropriate (r = .69, p < .001) tests. The correlation between performance on the calculator optional test, with calculator access, and Junior Certicate performance in mathematics was .80 (p < .001), whereas, without calculator access, it was .69 (p < .001). This suggests that the items on the calculator optional test (with calculator available) resemble more closely items on the Junior Certicate mathematics examination (where a calculator is also available) than is the case for the other tests. In Phase 2, the mean scale score of ordinary level students on the calculator optional test with calculator access (243.40, SE = 2.59) approaches the mean score of higher level students without access (250.90, SE = 2.71; SE of difference = 3.84; 95% CI [0.20 to 15.40]). Because the difference is not statistically signicant, it may be inferred that calculator access enables students taking ordinary level to perform at a level higher than they would otherwise attain on the types of tasks assessed by the test.

TABLE 10. Percentage of Students Expressing Strong Agreement or Agreement on Selected Attitude Toward Calculator Items (Phase 1 and Phase 2) Statement I think a calculator can help me to get better marks in school maths. I think I should be allowed to use a calculator in maths class. I think I should be allowed to use a calculator for maths homework. I think I should be allowed to use a calculator for classwork in other subjects. I can solve problems better when I have a calculator to help me with the arithmetic. I think a calculator could help me get better at maths. You dont have to think much when using a calculator. I think a calculator could make me lazy at school maths. Since I have a calculator I do not need to learn to do calculations with pen and paper. A calculator should be used only by a student who has a lot of difculty with school maths.
a

Phase 1 74.3 65.7 71.5 73.6

SE 1.84 1.89 1.77 1.45 2.35 2.17 1.71

Phase 2 92.3 94.0 93.6 89.2 86.1 70.2 46.8 41.1 26.0 18.8

SE 1.10 0.78 0.84 1.01 1.13 1.89 2.06 1.65 1.61 1.61

a a a

49.5 54.8

40.0

Question not included in Phase 1 Student Questionnaire.

388

The Journal of Educational Research

TABLE 11. Results of Factor Analysis of Data on Student Attitudes to Calculators and Correlations Between the Factors and Calculator Test Scores (Phase 2) Correlation with performance on calculator optional test With calculators .07 .05 .02 Without calculators .05 .16 .16

Factor 1. Support for calculator use in general 2. Negative effects of calculator use 3. Positive effects of calculator use in math

Percentage of explained variance 34.0 14.5 9.3

Correlation with performance on calculator inappropriate test .01 .14 .15

Correlation with performance on calculator appropriate test .10 .03 .08

Downloaded by [University of Malaya] at 01:28 25 November 2013

< .01.

Discussion We set out to examine the effects of introducing calculators into the Irish Junior Certicate mathematics curriculum and examinations. This was achieved by carrying out the study in two phases. Phase 1 was designed to assess ninthgrade students performance on key areas of numeracy in the mathematics curriculum that was in place at the time, in which calculators were not a feature of the syllabus or the Junior Certicate Mathematics examination. Phase 2 was carried out in 2004, following implementation of a curriculum in which calculator usage was actively promoted, to obtain data in the same key areas using the same instruments, for comparison with the data from Phase 1. Major outcomes with regard to student performance are summarized here. In Phase 2, the average performance of students who did not have access to a calculator (hence, on the calculator inappropriate test and on the calculator optional test for the no access condition) was lower than that in Phase 1, though not to a signicant extent. However, the average performance of students who did have calculator access (hence, on the calculator optional test for the access condition and on the calculator appropriate test) was higher for the Phase 2 cohortsignicantly so in the case of the calculator appropriate test. In general, students overall performance on the major mathematical content areas assessed did not change signicantly between Phase 1 and Phase 2, although similar patterns emerged with regard to somewhat higher performance with calculators and somewhat lower performance without them. Where Junior Certicate mathematics examination level is concerned, the benet of calculator access is evident. On the calculator optional test in Phase 2, the mean scale score of ordinary level students who had access to a calculator approached, and was not signicantly different from, the mean score of higher level students without access to a calculator.

The nonsignicant decrease in performance when students do not have calculator access should be monitored over time, but should not be cause for concern, unless this trend is shown to continue. There are many factors besides calculator availability that should be taken into consideration in evaluating this. As noted previously, changes in the curriculum were not limited to those involving calculator usage. In addition, perceptions that the revised curriculum is shorter than its predecessor may have led to reduced time allocations for mathematics by schools in an increasingly overloaded Junior Cycle. Furthermore, professional development for teachers with regard to the revised curriculum was still ongoing, and much work remains to be done with regard to promoting good use of calculators as learning tools as well as for computational purposes. In any case, as indicated previously, it is important that the situation is monitored over the coming years, so that good practice in the teaching and learning of mathematics can be appropriately developed. Between Phase 1 and Phase 2, there was a substantial increase in the proportion of students who believed a calculator could help mathematics performance and that it should be used in class and at home. Students in Phase 2 who held more positive attitudes toward calculators in general tended to perform better on the calculator appropriate test than students who held less positive or even negative attitudes. Students in Phase 2 reported enjoying the ease and speed of computations when using a calculator, though some expressed concern over the potential for making mistakes when using one. As noted previously, performance on the calculator appropriate test was disappointing in both phases. This is an issue that may be addressed in the context of Project Maths (National Council for Curriculum and Assessment, 2009), a new national initiative whose aims include providing students at Junior and Senior cycles with opportunities to engage in more realistic problem solving in the mathematics

The Journal of Educational Research

389
Mullis, I. V. S., Martin, M. O., Gonzalez, E. J., & Chrostowski, S. J. (2004). Findings from IEAs Trends in International Mathematics and Science Study at fourth and eighth grades. Chestnut Hill, MA: TIMSS and PIRLS International Study Center. Mullis, I. V. S., Martin, M. O., Ruddock, G. J., OSullivan, C. Y., Arora, A., & Erberber, E. (2005). TIMSS 2007 assessment frameworks. Chestnut Hill, MA: Boston College. National Center for Educational Statistics. (2009) The nations report card. Retrieved from http://nces.ed.gov/nationsreportcard/mathematics/ whatmeasure.asp National Council for Curriculum and Assessment. (2009). Project maths: Developing post-primary mathematics education. Retrieved from http://www.ncca.ie/eng/Curriculum and Assessment/Post-Primary Edu cation/Review of Mathematics/Project Maths OConnor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicers MAP test. Behaviour Research Methods, Instruments and Computers, 32, 396402. Organisation for Economic Co-operation and Development. (2001). Knowledge and skills for life: First results of PISA 2000. Paris, France: Author. Organisation for Economic Co-operation and Development. (2004). Learning for tomorrows world: First results from PISA 2003. Paris, France: Author. Ruthven, K. (1997). The use of calculators at key stages 13 (Discussion Paper No. 9). London, UK: School Curriculum and Assessment Authority. Shiel, G., Cosgrove, J., Sofroniou, N., & Kelly, A. (2001). Ready for life: The literacy achievements of Irish 15-year olds with comparative international data. Dublin, Ireland: Educational Research Centre. Smith, B. A. (1997). A meta-analysis of outcomes from the use of calculators in mathematics education. (Texas A&M University, 1996). Dissertation Abstracts International, 58, 787A. Westat. (2007). WesVar 5.1. Users guide. Rockville, MD: Author.

curriculum. This could involve enhanced use of scientic calculators as well as more systematic use of graphics calculators in secondary school classrooms.
NOTES 1. Renamed Trends in International Mathematics and Science in 1999. In TIMSS 2003, calculators were permitted for part of the TIMSS mathematics test, and in TIMSS 2007, for the entire test (Mullis et al., 2005). 2. The type of item used here to illustrate the three types of calculator sensitive tasks are all context free, whereas the items in the tests were a combination of context-free and context-embedded items. 3. In Phase 1, 731 students took the calculator optional test with a calculator and 722 without a calculator. In Phase 2, the corresponding numbers were 741 and 718, respectively. 4. Scale scores were used to report total test score by type of test and test condition, whereas mean percentage correct scores were used to report performance by mathematics content area. 5. Table 6 also includes the condence intervals around these mean score differences generated in WesVar. 6. In parallel analysis the factor was retained when the associated eigenvalue exceeded the corresponding eigenvalue at the 95th percentile of the distribution of eigenvalues derived from random data. REFERENCES Beaton, A. E., Mullis, V. S., Martin, M. O., Gonzalez, E. J., Kelly, D. L., & Smith, T. A. (1996). Mathematics achievement in the middle-school years: IEAs Third International Mathematics and Science Study. Chestnut Hill, MA: TIMSS International Study Center, Boston College. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Cosgrove, J., Shiel, G., Sofroniou, N., Zastrutzki, S., & Shortt, F. (2005). Education for life: The achievements of 15-year-olds in Ireland in the second cycle of PISA. Dublin, Ireland: Educational Research Centre. Department of Education and Science/National Council for Curriculum and Assessment. (2000). Mathematics syllabus: Higher, ordinary and foundation level. Dublin, Ireland: Stationery Ofce. Department of Education and Science/National Council for Curriculum and Assessment. (2001). Calculators: Guidelines for second level schools. Dublin, Ireland: Stationery Ofce. Department of Education and Science/National Council for Curriculum and Assessment. (2002). MathematicsJunior Certicate: Guidelines for teachers. Dublin, Ireland: Stationery Ofce. Dunham, P. H. (2000). Hand-held calculators in mathematics education: A research perspective. In E. D. Laughbaum (Ed.), Hand-held technology in mathematics and science education: A collection of papers (pp. 3947). Columbus: The Ohio State University. Ellington A. J. (2003). A meta analysis of the effects of calculators on students achievement and attitude levels in pre-college mathematics classes. Journal for Research in Mathematics Education, 34, 433463. Foy, P. (1997). Calculation of sampling weights. In M. Martin & D. Kelly (Eds.), Third International Mathematics and Science Study, technical report: Vol. 2. Implementation and analysisprimary and middle-school years. Chestnut Hill, MA: TIMSS International Study Centre, Boston College. Greenes, C., & Rigol, G. (1992). The use of calculators on College Board standardized tests. In J. T. Fey & C. R. Hirsch (Eds.), Calculators in mathematics education: 1992 yearbook. (pp. 186194). Reston, VA: National Council of Teachers of Mathematics. Hembree, R., & Dessart, J. D. (1986). Effects of hand held calculators in pre-college mathematics education: A meta-analysis. Journal for Research in Mathematics Education, 17(2), 8399. Hembree, R., & Dessart, J. D. (1992). Research on calculators in mathematics education. In J. T. Fey & C. R. Hirsch (Eds.), Calculators in mathematics education: 1992 yearbook (pp. 2332). Reston, VA: National Council of Teachers of Mathematics. Mullis, I. V. S., Martin, M. O., & Foy, P. (2008). Findings from IEAs Trends in International Mathematics and Science Study at fourth and eighth grades. Chestnut Hill, MA: TIMSS and PIRLS International Study Center.

Downloaded by [University of Malaya] at 01:28 25 November 2013

AUTHORS NOTE Sean Close is a Research Associate at the Educational Research Centre, St. Patricks College, Dublin, and was formerly a Lecturer in Mathematics Education in St. Patricks College from 1976 to 2007. His current research focuses on the assessment of mathematics achievement and attitudes in national and international studies. Elizabeth Oldham was a Senior Lecturer in Mathematics Education in the School of Education at Trinity College, Dublin; since retiring in 2010, she lectures part-time in the School of Mathematics. Her research interests include IT in mathematics education and national and international studies of mathematics curriculum and achievement. Gerry Shiel is a Research Fellow at the Educational Research Centre, St. Patricks College, Dublin. His research interests include factors associated with performance in mathematics. He has worked extensively on PISA mathematics in Ireland. Therese Dooley is a Lecturer in Mathematics Education at St. Patricks College, Dublin. Her research interests include problem solving, creativity, affect, and inclusion in mathematics education. Michael OLeary is the Director of Postgraduate Studies in Education and a Principal Lecturer in Assessment and Testing and Research Methodology at St. Patricks College, Dublin. His research interests include assessment at the classroom, national, and international levels and teachers lives and careers.

390

The Journal of Educational Research

APPENDIX Pattern and Structure Matrices for Identication of Factors in Student Attitudes to Calculators (Phase 2) Factors (pattern matrix) Factors (structure matrix) Negative effects of calc. use .08 .40 .16 .36 .18 .05 Positive effects of calc. on mathematics .70 .30 .64 .27 .62 .39

Support for Negative Positive effects Support for calculator use in effects of calc. of calc. on calculator use general use mathematics in general STQ16a: A calculator helps me get better marks in school maths. STQ16b: A calculator could make me lazy at school maths. STQ16c: A calculator could help me get better at maths. STQ16d: A calculator should only be used by student with difculty. STQ16e: I should be allowed use a calculator for maths homework. STQ16f: I should be allowed use a calculator for homework in other subjects. STQ16g: I should be allowed use a calculator in maths class. STQ16h: I should be allowed use a calculator for classwork in other subjects. STQ16i: I can solve problems better with a calculator. STQ16j: You dont have to think much using a calculator. STQ16k: Maths is more fun with a calculator. STQ16l: Since I have calculator, I dont need to do calculations with paper and pen. .09 .03 .02 .16 .49 .81 .01 .37 .10 .33 .10 .02 .64 .28 .62 .15 .33 .07 .46 .17 .38 .27 .69 .77

Downloaded by [University of Malaya] at 01:28 25 November 2013

.50 .85 .29 .03 .01 .02

.08 .02 .06 .77 .17 .52

.36 .05 .40 .09 .51 .16

.72 .82 .52 .00 .28 .06

.17 .05 .01 .76 .12 .50

.65 .44 .56 .03 .50 .12

Note. Extraction method: maximum likelihood; rotation method: oblimin with Kaiser normalization.