Reliability and Validity of the Evaluation Tool of Children’s
Handwriting–Cursive (ETCH–C) Using the General Scoring
Criteria
KEY WORDS
• education
• educational measurement
• handwriting
• psychomotor performance
• reproducibility of results
Sharon Duff, BAppSc (OT), is Clinical Specialist
Occupational Therapist, Rehabilitation Department, Royal
Alexandra Hospital for Children, Westmead, Sydney, New
South Wales, Australia.
Traci-Anne Goyen, BAPPSc (OT), PhD, is Clinical
Specialist Occupational Therapist, Centre for Newborn
Care, Westmead Hospital, Westmead, Sydney, New South
Wales 2145 Australia; tagoyen@optushome.com.au
The American Journal of Occupational Therapy
Downloaded From: http://ajot.aota.org/ on 02/02/2018 Terms of Use: http://AOTA.org/terms
Sharon Duff, Traci-Anne Goyen
OBJECTIVES. To determine the reliability and aspects of validity of the Evaluation Tool of Children’s
Handwriting–Cursive (ETCH–C; Amundson, 1995), using the general scoring criteria, when assessing children
who use alternative writing scripts.
METHOD. Children in Years 5 and 6 with handwriting problems and a group of matched control participants
from their respective classrooms were assessed with the ETCH–C twice, 4 weeks apart.
RESULTS. Total Letter scores were most reliable; more variability should be expected for Total Word scores.
Total Numeral scores showed unacceptable reliability levels and are not recommended. We found good dis-
criminant validity for Letter and Word scores and established cutoff scores to distinguish children with and
without handwriting dysfunction (Total Letter <90%, Total Word <85%).
CONCLUSION. The ETCH–C, using the general scoring criteria, is a reliable and valid test of handwriting
for children using alternative scripts.
Duff, S., & Goyen, T.-A. (2010). Reliability and validity of the Evaluation Tool of Children’s Handwriting–Cursive (ETCH–C)
using the general scoring criteria. American Journal of Occupational Therapy, 64, 37–46.
S
tudentswithhandwritingdysfunctionarefrequentlyreferredtooccupational
therapyandformaconsiderableproportionofthecommunityorschool-based
occupationaltherapist’scaseload.Performanceonwrittentasksatschool,including
assignmentsandexaminations,canbeinfluencedbyachild’spoorlegibilityand
speed(Graham,Weintraub,&Berninger,2001;Tseng&Cermak,1993).
Assessmentformsanintegralpartofmanagementwiththisclinicalpopulation.
Relativelyfewstandardizedinstrumentsareavailableforclinicianstospecifically
evaluatehandwritingperformance(Amundson,1995;Ziviani&Elkins,1984).
Thoseavailablearerarelyusedbyoccupationaltherapistsinclinicalpractice(Feder,
Majnemer,&Synnes,2000).Rather,teststhatassessunderlyingabilities,including
theDevelopmentalTestofVisualMotorIntegration(Beery,1997),theBruininks–
OseretskyTestofMotorProficiency(Bruininks,1978),andtheTestofVisual
Perceptual Skills (Gardner, 1996), are reported to be commonly used to assess
childrenwithhandwritingdysfunction(Federetal.,2000).
Reliableandvalidstandardizedassessmentsofhandwritingprovideanobjective
measureofactualhandwritingperformance.Inassessinghandwriting,theseassess-
ments are preferable to tests of underlying abilities (Feder & Majnemer, 2003;
Goyen&Duff,2005).Standardizedassessmentofhandwritingallowscomparison
betweenpeersandbetweenpre-andposttreatmentscores;theycandetermineeli-
gibilityforservicesandcanbeusedinresearch.Intheirreviewofhandwriting
37
 assessments,FederandMajnemer(2003)recommendedthat
therapists use a comprehensive approach to handwriting
evaluationthatincludesarangeofhandwritingtasksneces-
saryforday-to-dayfunctioningintheclass.Theyconcluded
thattheassessmenttoolsavailablearenotwidelyusedand
thatvalidationstudiesforthesetoolsarelacking.
The type of handwriting scripts that are taught by
schools,particularlycursive,canvaryacrosscountriesand
states and even within districts. As a result, handwriting
assessmentshaveuseddifferentwritingscripts,whichmay
limittheutilityofhandwritingassessmentsthatoccupational
therapistscanselectforclinicalandresearchpurposesintheir
own contexts. The Evaluation Tool of Children’s
Handwriting(ETCH;Amundson,1995)isoneassessment
thatincludeswritingmodelsandscoringcriteriathatcould
potentially accommodate differing scripts. The ETCH is
usedtoexaminelegibilityacrossavarietyoffunctionalwrit-
tencommunicationtaskscommonlyperformedintheclass-
room, including writing from memory, copying from a
model, and self-generated writing. Instead of examining
detailedandmorespecificscoringcriteriathoughttocon-
tributetolegibility—suchasletterformation,adherenceto
lines,andspacingofwordsandletters—theETCHincor-
poratesglobalscoringcriteriathatarebasedonoverallread-
abilityofthewriting(Rosenblum,Weiss,&Parush,2003).
Forinstance,awordisconsideredillegibleifitisnotquickly,
easily,andcorrectlyreadastheintendedword;isconfused
foranotherword;orcontainsextraneousforms.
TheETCHoffersscoringcriteriatoassesschildrenon
both manuscript (ETCH–M) and cursive (ETCH–C)
scripts.Scoringencompassesbothgeneralscoringcriteriafor
letters,words,andnumbersandspecificcriteriawithexam-
plestoassistthescorer.AlthoughtheETCHisbasedonthe
D’Nealian script, the general scoring criteria can be used
withchildrenwhousealternatewritingstyles.Theauthor
oftheETCH,Amundson(1995),hascommentedthatchil-
drenwhowereunfamiliarwithD’Nealianscriptwerenot
confusedbythemodelscriptusedintheETCH.Dennis
andSwinth(2001)usedthegeneralletterandwordlegibility
scoring criteria of both the ETCH–M and ETCH–C to
investigatetheassociationbetweenpencilgraspandlength
of writing task. Their findings suggested that the ETCH
generalscoringcriteriacanbesuccessfullyappliedtoscoring
writingtasksotherthanthoseincludedintheETCH.This
assumption,however,requiresfurtherinvestigation.
Limited published information exists regarding the
ETCH’spsychometricproperties,particularlythoseofthe
ETCH–C.Moreover,reliabilityandvalidityoftheETCH–
C’sgeneralscoringcriteriawhenusedwithalternatewriting
scriptshasnotbeenexplored.Thisinformationisneeded
beforetheETCHcanbeconsideredausefultoolinfuture
38
Downloaded From: http://ajot.aota.org/ on 02/02/2018 Terms of Use: http://AOTA.org/terms
researchusingthegeneralscoringcriteriaorforclinicians
whoassesschildrenwhousedifferentwritingscripts.
Our purpose in this study was to determine the reli-
ability and aspects of validity of the ETCH–C using the
generalscoringcriteria.Wedesignedthestudytoexamine
thefollowinginrelationtotheETCH–C:intraraterreli-
ability,interraterreliability,test–retestreliability,discrimi-
nantvalidity,concurrentvaliditywiththeTestofLegible
Handwriting(TOLH;Larsen&Hammill,1989),andrela-
tionshipwithteacher’sratingsofhandwriting.
Method
Participants
ParticipantswerechildreninYears5and6(6thand7thyear
offormalschooling)attendingmainstreampublicprimary
schoolsinNewSouthWales,Australia.Approvalfromthe
Children’s Hospital Ethics Committee was granted.
Permission to conduct the study in schools was obtained
fromtheDepartmentofEducationandTrainingandthe
CatholicEducationOffices.
First,werandomlyselected10publicprimaryschools
from within a 20-km radius of Westmead Hospital and
obtainedtheirconsenttoparticipate.WeaskedYear5and
Year 6 teachers to select two groups of participants. One
groupofstudents—casestudents—hadhandwritingdifficul-
tiesandwereselectedbyteachersashavingdifficultywith
handwritinglegibilityorslownesswhenwritingthatinter-
feredwiththeirabilitytoperforminclass.Teachersthen
selectedmatchedcontrolstudentsfromthesameclasswho
didnothavehandwritingproblems,wereofthesamegen-
der, and had the closest birth dates to the participants.
Writteninformedconsenttoparticipateintheresearchwas
obtainedbytheteacherfromtheparentsofallstudents.We
permittedamaximumof2caseparticipants(and2control
participants)fromanyoneclasstoensurethatnoparticular
class was overrepresented. Students identified as having a
disability,havingrepeatedaclassyear,havingepilepsy,or
havingbeenbornprematurelywereexcluded.Caseandcon-
trolparticipantswerethereforematchedformajorconfound-
ingvariablesdeemedtoinfluencehandwritingperformance,
including gender (Ziviani & Watson-Will, 1998), age
(Ziviani,1995),presenceofdisability,schoolclass(Graham
etal.,2001),andtypeandamountofhandwritinginstruction
(Edwards,2003;Graham,Berninger,&Weintraub,1998).
Instruments
Evaluation Tool of Children’s Handwriting. The ETCH
(Amundson, 1995) evaluates the legibility and speed of
handwritingofchildreninYears2through6andhasbeen
January/February 2010, Volume 64, Number 1
 usedtoevaluatetheeffectivenessoftreatment(Case-Smith,
2002;Sudsawad,Trombly,Henderson,&Tickle-Degnan,
2002).Itisacriterion-referenced,standardizedassessment
thatfocusesonthereadabilityofletters,words,andnumbers
ataglanceandoutofcontext.TheETCHtestsmanuscript
(ETCH–M)andcursive(ETCH–C)writingstyles.Tasks
aresimilartothoserequiredofstudentsintheclassroom,
includingwritingthealphabetinlower-anduppercaselet-
tersfrommemory,writingnumbersfrommemory,copying
anonsensesentencefromanear-andafar-pointdistance,
writingdictation,andsentencecomposition.Forthepur-
posesofthisstudy,thenear-pointtasksheetandfar-point
copyingwallchartwerealteredbyagraphicartisttoaccom-
modatetheNewSouthWales(NSW)FoundationScript
thatisingeneraluseinNewSouthWalespublicschools.A
globalscoringmethodisusedtoassessreadabilityofletters,
words,andnumerals.TheETCHyieldsTotalLetter,Total
Word,andTotalNumeralscores,whichareexpressedasa
percentage of total legible letters, words, or numbers.
Examplesoflegibleandillegiblesamplesareprovidedinthe
testmanualtoassistthescoringprocess.Inaddition,legibil-
itycomponents(e.g.,letterformation,size,andspacing)can
beanalyzed.Pencil-and-papermanagementtasksrelatedto
handwriting can also be evaluated. Scoring tutorials and
quizzesareincludedtoincreasescoringcompetency.
The ETCH’s psychometric properties have not been
comprehensivelyestablished.Thetestmanual(Amundson,
1995) reports interrater reliability intraclass coefficients
(ICCs)for14childrenfromaregularclassand15whowere
referredtooccupationaltherapyforhandwritingproblems.
ICCsforthecursiveversionwere.89forTotalLetter,.94for
TotalWord,and.53forTotalNumeralscores.Manyofthe
ICCsforeachofthewritingtaskswerelowerthandesired.
Consequently, Amundson (1995) recommended that the
TotalLetter,TotalWord,andTotalNumeralscoresbeused
ratherthantheindividualtaskscores.DennisandSwinth
(2001),whoexaminedtheassociationbetweenpencilgrasp
andlengthofwriting,reportedverygoodagreementforthe
ETCH’sinterraterreliabilityusingthegeneralscoringcriteria
(letterlegibilitypercentageofagreement=96.9%–99.4%;
wordagreement=86.7%–100%).However,theydidnot
specifically examine the ETCH–C, and participants used
eithermanuscriptorcursiveanddidnotusetheETCHwrit-
ingtasks.Diekema,Deitz,andAmundson(1998)reported
moderatelevelsoftest–retestreliabilityfortheETCH–Mbut
didnotexaminetheETCH–C.Nostudieshaveinvestigated
theETCH–C’sintraraterortest–retestreliability.
KoziatekandPowell(2002)examinedtheETCH–C’s
concurrentvaliditywithteachers’gradesonthebasisofper-
sonaljudgmentfor101typicallydevelopingstudentsfrom
Grade4.Pearsoncorrelationcoefficientsweremoderatefor
The American Journal of Occupational Therapy
Downloaded From: http://ajot.aota.org/ on 02/02/2018 Terms of Use: http://AOTA.org/terms
ETCH–CTotalLetter(r=.65)andTotalWordscores(r=
.61).Inaddition,theirstudywasdesignedtoidentifyETCH–
Ccutoffscoresthatdiscriminatedsatisfactoryandunsatisfac-
tory handwriting. Using receiver operating characteristic
(ROC)curves,KoziatekandPowell(2002)determinedthat
on the ETCH–C, Total Letter scores of 81% and Total
Wordscoresof75%werethebestcutoffpointstodistinguish
betweensatisfactoryandunsatisfactoryhandwriting.Thiswas
slightly lower than the 85% cutoff score suggested by
Amundson (1995). Construct or criterion-related validity
studiesareneeded(PolenaFeder&Majnemer,2003).
Test of Legible Handwriting. The TOLH (Larsen &
Hammill, 1989) is a standardized and norm-referenced
assessmentofhandwritinglegibilityforchildreninGrades
2through12.Writingsamplesareelicitedfromavarietyof
contexts and compared with graded legibility samples. A
legibilityquotientisobtained(mean=100,standarddevia-
tion=15).Noacceptedgold-standardassessmentofhand-
writing legibility exists for this study’s target age group.
Despitethis,weselectedtheTOLHbecause,liketheETCH,
it is designed to assess readability of handwriting using a
globalapproachtoscoring.
Teacher’s rating of handwriting. Teacherswereaskedto
giveanoverallratingofeachchild’shandwritingona5-
pointscale(very poor, poor, average, good, andvery good)to
provideanoverallimpressionofthechild’swritinginthe
classroom.Test–retestreliabilityofthisscalehasbeenestab-
lishedasgood(weightedκ=.73;Duff&Goyen,2001).
Procedure
ParticipantswereassessedwiththeETCH–CandTOLHat
twopointsintime,4weeksapart.Weassessedparticipants
attheirschoolinasmallgroup.Theorderoftestswasran-
domizedtoeliminateanordereffect,andresponsesheets
werecodedtoensurethatraterswereblindtochild,group
allocation,school,orfirstorsecondtestadministration.After
completionofthedatacollectionphase,onerater,experi-
encedinadministeringandscoringtheETCH–Candthe
TOLH,scoredalltests.Theraterusedthegeneralguidelines
outlinedintheETCHmanual,withthreeminorchanges
madetoaccommodatetheNewSouthWalesFoundation
Script:(1)“manuscriptiswrittenwhencursiveisrequested”
wasnotincluded;wefeltthatthiswouldbetoodifficultto
score because the manuscript and cursive styles are very
similar;(2)loopeddescenderswerepermitted;and(3)for
the letter k, both the D’Nealian and looped Foundation
scriptstyleswerepermitted.
The same rater scored the tests again after a 4-week
periodtoobtainintraraterscores.Asecondrater,anovice
scoreroftheETCH,thenscoredeachinitialtest,allowing
evaluationofinterraterreliability.
39
 Data Analysis initialassessmentday.Therewere46malestudentsand17
femalestudents,with32fromYear5and31fromYear6.
WeanalyzeddatafortheETCH–CusingtheTotalLetter,
Twentyfivewereage10,26wereage11,and12wereage
TotalWord,andTotalNumeralscores.WecalculatedICCs
12. The group consisted of 57 right handers and 6 left
toexamineintrarater,interrater,andtest–retestreliability.
handers.
An ICC of 0.9–1.0 was considered very high reliability;
Fifteen participants were not available for the retest
0.7–0.9,highreliability;0.5–0.7,moderatereliability;and
assessment,leaving24correctlymatchedcase–controlpairs
0.3–0.5,lowreliability(Hinkle,Wiersma,&Jurs,1998).
forthetest–retestandvalidityanalyses.
Weundertookfurtheranalysisoftest–retestreliabilityusing
ameans-versus-differenceplot,asdescribedbyBlandand Reliability
Altman(ascitedinPeat,2002).Thex-axisrepresentsthe
Intrarater, interrater, and test–retest reliability coefficients
average of the test and retest scores. The y-axis plots the
fortheETCH–CarereportedinTable1.Agreementforthe
actualdifferencebetweenthetwoscores.Themeans-versus-
TotalNumeralscoreswasmoderatetolowonallthreemea-
differenceplotallowsonetoobservethespreadoferrorand
suresofreliability.Resultsforthetest–retestreliabilityofthe
determinewhethersystematicerrorhasoccurred.
Total Letter and Total Word legibility scores were lower
Weexamineddiscriminantvalidityusingreceiveroperat-
thanexpected.
ingcharacteristic(ROC)curvestomeasurethetest’saccuracy
We constructed means-versus-difference plots for the
in separating the participants into those with handwriting
TotalLetter(Figure1)andTotalWord(Figure2)scoresto
problems(caseparticipants)andthosewithouthandwriting
determine whether any systematic error occurred and to
problems(controlparticipants).AROCcurveisconstructed
examinespreadoferror.Bothplotsillustratethatnosystem-
fromthesensitivityandspecificitycalculationsofatest.The
atic error occurred; however, more variability existed for
ROCcurveplotsthefalsepositiverateonthex-axisandthe
thoseparticipantswithlowerscores.Wedidnotconstructa
1−falsepositiverateonthey-axis.Itshowsthetrade-off
means-versus-differenceplotfortheTotalNumeralscores
betweenatest’ssensitivityandspecificity.Thecoordinatesof
becauseoftheverylowICCsobtained.TheTotalLetterplot
thecurveclosesttotheupperleftsideindicatethebestcutoff
(Figure1)showedanarrowrangeofscoresfromapproxi-
pointforthetesttoidentifychildrenwithhandwritingprob-
mately80to95,andtheactualdifferencebetweentestand
lems.Theareaunderthecurvecanbeusedtomeasurethe
retestscoreswasasmuchas±10points.TheICCmayhave
test’saccuracy(Peat,2002).IftheareaundertheROCcurve
beenartificiallyloweredbecauseofthegroup’snarrowrange
iscloseto1,thetest’saccuracyinidentifyinghandwriting
ofscores(20points),withbetween80andamaximumof
difficultyisconsideredexcellent.Iftheareaunderthecurve
100 percentage points possible. The larger differences
iscloserto0.5,thetestisconsideredtohavepoordiscrimi-
occurredforparticipantswithlowerscores,suggestingmore
nantability.Anapproximateguideforclassifyingaccuracyis
variability in scores for children with poorer handwriting
1.0–0.9=excellent,0.9–0.8=good,0.8–0.7=fair,0.7–0.6
legibility.TheTotalWordplot(Figure2),however,showed
=poor,and0.6–0.5=fail(Tape,n.d.).
awiderrangeofscores(between50and100),andalthough
WedeterminedconcurrentvalidityusingPearson’scor-
manyparticipantshadlittledifferencebetweentestandretest
relationalanalysistomeasuretheassociationoftheETCH–
scores,thedifferencewasasmuchas±20pointsforseveral
CTotalLetterscoreswiththeTOLHLegibilityQuotient
participants.Wealsofoundlargerdifferencesforparticipants
scores.Finally,weexaminedtherelationshipbetweenthe
whoachievedlowerwordlegibilityscores.
teacher’sratingofthechild’shandwritingandthechild’s
performanceontheETCH–C.Teachers’ratingswerecol- Discriminant Validity
lapsed into three groups: very poor–poor, average, and
WefoundaTotalLetterscoreof92tobethebestcutoff
good–very good. We then compared the differences in
pointtodiscriminatebetweencaseandcontrolparticipants
ETCHscoresamongthethreegroupsusinganalysisofvari-
(sensitivity=.88andspecificity=.83,asnotedinTable2).
anceandTukeyposthocanalysis.Wesetthelevelofsignifi-
canceatp <.05.
Table 1. Reliability (Intraclass Correlation Coefficients) of the
Evaluation Tool of Children’s Handwriting–Cursive (Modified)
Intrarater Interrater Test–Retest
Results Score (n = 63) (n = 63) (n = 48)
Thestudyparticipantswere63childrenfrom10schools.Of Total Letter .80 .84 .61
Total Word .71 .62 .65
these,33werecaseparticipantsand30werecontrolpartici-
Total Numeral .55 .57 .24
pants.Threecontrolparticipantswerenotavailableonthe
40 January/February 2010, Volume 64, Number 1
Downloaded From: http://ajot.aota.org/ on 02/02/2018 Terms of Use: http://AOTA.org/terms
 Difference of Test–Retest Scores

Mean of Test–Retest

Figure 1. Mean-versus-difference plot for Total Letter score.

The ROC curve (Figure 3) indicates good discriminant
validity(areaunderthecurve=.86;95%confidenceinterval
[CI]=.75–.98).
WefoundaTotalWordscoreof85tobethebestcutoff
pointtodiscriminatebetweencaseandcontrolparticipants
(sensitivity=.71andspecificity=.75,asnotedinTable3).
The ROC curve (Figure 4) indicates good discriminant
validity(areaunderthecurve=.85;95%CI=.47–.96).
WefoundaTotalNumeralscoreof95tobethebest
cutoffpointtodiscriminatebetweencaseandcontrolpartici-
pants(sensitivity=.42andspecificity=.88,asnotedinTable
4). The ROC curve (Figure 5) indicates fair discriminant
validity(areaunderthecurve=.76;95%CI=.63–.90).
Difference of Test–Retest Scores
Concurrent Validity
WefoundtheconcurrentvalidityoftheETCH–CTotal
LetterscorewiththeTOLHLegibilityQuotienttobegood
(r=.6,p<.001),usingPearsoncorrelationcoefficients.
Teacher’s Rating and the ETCH–C
Wereceivedteacher’sratingsforonly46childrenandcate-
gorizedtheirratingofoverallhandwritingabilityintothree
groups. Twenty-two children were rated as having very
poor–poor handwriting, 8 were rated as having average
handwriting,and16wereratedashavinggood–verygood
handwriting; two teacher ratings were not returned. We

Mean of Test–Retest

Figure 2. Mean-versus-difference plot for Total Word score.

The American Journal of Occupational Therapy 41

Downloaded From: http://ajot.aota.org/ on 02/02/2018 Terms of Use: http://AOTA.org/terms
 Table 2. Discriminant Validity Using Total Letter Score Table 3. Discriminant Validity Using Total Word Score
Case Participants Control Participants Case Participants Control Participants
(With Handwriting (Without Handwriting (With Handwriting (Without Handwriting
Score Difficulties) Difficulties) Total Score Difficulties) Difficulties) Total
Letters <92 21 4 25 Word <85 17 6 23
Letters ≥92 3 20 23 Word ≥85 7 18 25
Total 24 24 Total 24 24
Note. Sensitivity = .88; specificity = .83; positive predictive value = .84; nega- Note. Sensitivity = .71; specificity = .75; positive predictive value = .74; nega-
tive predictive value = .87. tive predictive value = .72.

foundasignificantdifferencebetweengroupsonalllegibility
scores(refertoTable5).Posthocanalysesindicatedthatthe
very poor–poor group scored significantly lower than the
other two groups, but the average and good–very good
groupsdidnotdiffer.FortheTotalNumeralscore,thevery
poor–poorgroup’sperformancewassignificantlylowerthan
thatofthegood–verygoodgroup.
Discussion
Inthisstudy,weinvestigatedwhetheruseofthegeneralscor-
ingcriteriafortheETCH–Cisareliableandvalidmeasure
of legibility for children who use the New South Wales
FoundationScriptratherthantheD’Nealianscript.Findings
haveimplicationsforcliniciansandresearcherswhoassessthe
handwriting of children who use alternate writing scripts.
ResultsindicatethatTotalLetterscoresarethemostreliable
and are preferable to use when diagnosing and evaluating
handwritingdysfunction. AlthoughTotalWordscores are
useful,morevariabilityshouldbeexpected.Becauseonlya
fewwordsactuallycontributetothetotalpercentageofscores
ontheETCH,anydiscrepancyislikelytocreatealargerdif-
ferenceorerror.Forinstance,iftwowordsarewrittentoo
closetogether,thenbothofthesewordsareconsideredillegi-
ble.Inrelationtointerraterreliability,oneratermayconsider
thesewordstobetoocloseandthereforegiveascoreof15of
apossible17,thusleavingaTotalWordscoreof88%.If
anotherraterconsidersthesewordsnottobetoocloseand
thereforenotillegible,heorshewouldgiveascoreof17,
givingaTotalWordscoreof100%.Thus,onedifferencein
scoring word legibility would yield a real difference of 12
pointsbetweentheratersforthisparticipant.

ROC Curve

1.0

0.8

0.6
Sensitivity

0.4

0.2 Area under the curve = .86

0.0
0.0 0.2 0.4 0.6 0.8 1.0
1 - Specificity

Figure 3. Receiver operating characteristic (ROC) curve for Total Letter score.

42 January/February 2010, Volume 64, Number 1

Downloaded From: http://ajot.aota.org/ on 02/02/2018 Terms of Use: http://AOTA.org/terms
 ROC Curve

1.0

0.8

0.6
Sensitivity

0.4

0.2

0.0
0.0 0.2 0.4 0.6 0.8 1.0
1 – Specificity

Figure 4. Receiver operating characteristic (ROC) curve for Total Word score.

The ETCH–C demonstrated reasonable discriminant Thisstudy’sresultssuggestthatthemostappropriate

andconcurrentvalidity.Usingthegeneralscoringcriteria,we cutoff points to differentiate between children with and
foundthetesttohaveadequatetogoodabilitytodiscriminate withouthandwritingdysfunctionis90%forTotalLetter
betweenchildrenwithandwithouthandwritingdysfunction. scoreand85%forTotalWordscore.Intheirvaliditystudy
Also,weidentifiedcutoffscoresforTotalLetter(<90%)and oftheETCH–C,KoziatekandPowell(2002)reportedthat
TotalWord(<85%)scorestodistinguishbetweenchildren acutoffpercentagescoreof82%forTotalLetterscoreand
withandwithouthandwritingdysfunction. 75%forTotalWordscorediscriminatedbetweensatisfac-
The general scoring criteria were able to distinguish tory and unsatisfactory handwriters. In comparison, our
betweenthosewithandwithouthandwritingdysfunction, cutoffpointsarehigher.KoziatekandPowell’s(2002)study
lendingsupportforuseoftheETCH–Casadiagnostictool. includedonlyfourthgraderswhowereyoungerthanour
Thetestdidnotdiscriminatebetweenaverageandabove- participants and had less practice writing with a cursive
average writers, as rated by teachers. Although this test script.Onewouldanticipatethatourparticipants,whohave
featuremaynotbeneededforclinicalpopulations,itcould hadmorepracticewritingincursive,wouldscorehigher.
limitthetest’sutilityforresearchpurposes.Thesefindings, Moreover,highercutoffpointscouldbeattributedtouseof
however,shouldbevalidatedinalargersample. thegeneralscoringcriteria,whichmaynotbeasprecisea
measurementasthecriteriaspecifiedintheETCHmanual
andusedintheKoziatekandPowell(2002)study.
Table 4. Discriminant Validity Using Total Numeral Score TheTOLHisanotherhandwritingassessmentthatuses
Case Participants Control Participants general scoring criteria and has cutoff points to diagnose
(With Handwriting (Without Handwriting
handwriting dysfunction. Our results revealed that the
Score Difficulties) Difficulties) Total
Numeral <95 10 3 13
ETCH–C has good concurrent validity with the TOLH
Numeral ≥95 14 21 35 whenusingthegeneralscoringcriteria.TheETCH–C,how-
Total 24 24 48 ever,hasadvantagesovertheTOLH.First,theTOLHisno
Note. Sensitivity = .42; specificity = .88; positive predictive value = .77; nega- longerinprint.Also,youngerchildrenfrequentlyproduce
tive predictive value = .6. minimalwrittenworkinresponsetotheTOLHstimulus
The American Journal of Occupational Therapy 43
Downloaded From: http://ajot.aota.org/ on 02/02/2018 Terms of Use: http://AOTA.org/terms
 ROC Curve

1.0

0.8

0.6
Sensitivity

0.4

Area under the curve = .76

0.2

0.0
0.0 0.2 0.4 0.6 0.8 1.0
1 – Specificity

Figure 5. Receiver operating characteristic (ROC) curve for Total Numeral score.

picture, which makes scoring difficult. The ETCH–C is Numerallegibilityiscrucialformathematicsandcan

designedtoexaminecommonhandwritingerrors,whichcan meanthedifferencebetweencorrectorincorrectanswers.
behelpfulintreatmentplanning,buttheTOLHcontains TotalNumeralscoresontheETCH,however,showedunac-
noscopeforthis.Finally,inadditiontoexaminingerrors ceptablelevelsofreliabilityandvalidityandshouldbeused
thatcaninfluencelegibility,theETCHgivesgoodinforma- withcaution.
tion regarding functional written communication in the InrelationtousingtheETCH–C’sgeneralscoringcri-
class,whichisusefulinplanningintervention.Forinstance, teriatoevaluatetreatmentoutcomes,ourresultsarenotclear.
therapists can determine whether the child has problems Test–retestreliabilitywaslowerthandesiredbutwithinthe
with all written tasks or only with self-generated writing. moderate range and useful for clinical purposes. This test
Errorpatterns,asidentifiedbytheETCH,canbetargeted characteristicisimportanttodeterminewhetherchangein
forintervention. scoresovertimeismorelikelytheresultoftreatmentpro-
videdthanoftesterrororinstability.AstudybyDiekemaet
al. (1998) found moderate test–retest reliability on the
ETCH–M(LetterICC=.77,WordICC=.71,Number
Table 5. Teacher’s Rating of Handwriting and Evaluation Tool ICC=.63)inagroupofchildrenwithhandwritingdysfunc-
of Children’s Handwriting–Cursive (Modified) tion.InresponsetotheDiekemaetal.(1998)study,Schneck
Very Poor– Good–Very (1998)speculatedthattest–retestreliabilitymayhavebeen
Poor Average Good lowerthanexpectedbecauseperformanceisknowntobe
Score (n = 22) (n = 8) (n = 16) pa F (df)
more inconsistent among children with difficulties. On
Letters (M ± SD) 85.2 ± 5.8b
91.6 ± 7.1 94.5 ± 5.2 <.01 12.5 (2, 43)
Words (M ± SD) 69.7 ± 19.0b 88.8 ± 9.4 90.4 ± 7.2 <.01 11.2 (2, 43)
examiningourtest–retestdataclosely,wedeterminedthat
Numbers (M ± SD) 92.0 ± 5.7c 95.5 ± 3.3 98.0 ± 2.5 <.01 8.6 (2, 43) scores may have been artificially lowered because of their
Note. M = mean; SD = standard deviation.
narrowspread.Whenusingthegeneralscoringcriteria,we
a
Analysis of variance with Tukey’s post hoc test.
foundatest–retesterrorofupto10points.Thisfinding
b
Significantly lower than other groups. could be validated in a subsequent study using a shorter
c
Significantly lower than good–very good group. retestperiod.

44 January/February 2010, Volume 64, Number 1

Downloaded From: http://ajot.aota.org/ on 02/02/2018 Terms of Use: http://AOTA.org/terms
 Thestudy’smajorlimitationistheselectionofpartici-
pantsbasedonteacherjudgment,whichassumesthatteacher
perception of handwriting is accurate and consistent.
Concernshavebeenraisedaboutthereliabilityofteacher’s
judgments of handwriting (Daniel & Froude, 1998;
Sudsawad,Trombly,Henderson,&Tickle-Degnen,2001).
Despitetheseconcerns,otherstudieshavereliedonteacher
judgmenttoidentifyparticipantswithhandwritingdysfunc-
tion(Diekemaetal.,1998;Sudsawadetal.,2001;Wallen
&Mackay,1999).Othershavefoundexperiencedteachers
to be good judges of handwriting legibility (Koziatek &
Powell,2002;Tseng&Murray,1994).
Conclusion
We investigated the ETCH–C’s reliability and aspects of
validity when the general scoring criteria are used. Results
indicatethatTotalLetterscoresaremostreliableandthat
althoughTotalWordscoresareuseful,morevariabilityshould
beexpected.TotalNumeralscoresshowedunacceptablereli-
abilitylevels,andwewouldnotrecommendtheiruse.The
test–retestreliabilitycoefficientswere,however,lowerthan
desired,similartomanyotherpediatrictests.Wefoundthe
ETCH–Ctohaveadequatetogoodabilitytodiscriminate
betweenchildrenwithandwithouthandwritingdysfunction,
withestablishedcutoffscoresforTotalLetterof<90%and
forTotalWordof<85%.Thesecutoffscorescouldbeverified
infutureresearchusingdifferentagegroups.
ThisstudyconfirmstheuseoftheETCH–C’sgeneral
scoringcriteria,particularlythatfortheTotalLetterscore
becauseitismostreliable.Thisconfirmationishelpfulto
clinicianswhentheyareassessingchildrenwhouseanalter-
nate writing script in terms of identifying children with
handwriting dysfunction and in treatment planning. The
ETCH–Cwouldbeausefulresearchtooltoquantifydiffer-
encesbetweengroupsandtodistinguishchildrenwithhand-
writingdysfunctionusingthecutoffscores.
Ourresultsreflectthesubjectivenatureofscoringhand-
writinglegibility,particularlywithassessmentsusingaglobal
scoringmethod.TheETCHhasbeendesignedspecifically
toaccommodateamoregeneralprocessofscoringrather
thanrequiringprecisemeasurementswith,forexample,a
ruler.Scoresshouldalwaysbeinterpretedaspartofacom-
prehensiveevaluationofachild’shandwritingskills. s
Acknowledgments
We would like to acknowledge Margaret Wallen for her
assistancewiththestudydesignandmanuscriptpreparation
andJenniferPeatforherstatisticalsupportandadvice.
The American Journal of Occupational Therapy
Downloaded From: http://ajot.aota.org/ on 02/02/2018 Terms of Use: http://AOTA.org/terms
References
Amundson,S.(1995).Evaluation Tool of Children’s Handwriting.
Homer,AK:O.T.Kids.
Beery, K. (1997). The Beery–Buktenica Developmental Test of
Visual–Motor Integration(4thed.).ParsippanyNJ:Modern
CurriculumPress.
Bruininks,R.H.(1978).Examiner’s manual: Bruininks–Oseretsky
Test of Motor Proficiency.CirclePines,MN:AmericanGuid-
anceService.
Case-Smith,J.(2002).Effectivenessofschool-basedoccupational
therapy intervention on handwriting. American Journal of
Occupational Therapy, 56,17–25.
Daniel,M.E.,&Froude,E.H.(1998).Reliabilityofoccupational
therapistandteacherevaluationsofthehandwritingquality
ofgrade5and6primaryschoolchildren.Australian Occupa-
tional Therapy Journal, 45,48–58.
Dennis,J.L.,&Swinth,Y.(2001).Pencilgraspandchildren’s
handwritinglegibilityduringdifferent-lengthwritingtasks.
American Journal of Occupational Therapy, 55,175–183.
Diekema,S.M.,Deitz,J.,&Amundson,S.J.(1998).Test–retest
reliabilityoftheEvaluationToolofChildren’sHandwriting–
Manuscript.American Journal of Occupational Therapy, 52,
248–255.
Duff,S.,&Goyen,T.-A.(2001,September).Development of a
handwriting rating scale for teachers.Paperpresentedatthe
FirstAustralianPaediatricOccupationalTherapistsConfer-
ence,Sydney,NewSouthWales,Australia.
Edwards,L.(2003).Writinginstructioninkindergarten:Exam-
ininganemergingareaofresearchforchildrenwithwriting
andreadingdifficulties.Journal of Learning Disabilities, 36,
136–148.
Feder,K.,&Majnemer,A.(2003).Children’shandwritingevalu-
ation tools and their psychometric properties. Physical and
Occupational Therapy in Pediatrics, 23, 65–84.
Feder, K., Majnemer, A., & Synnes, A. (2000). Handwriting:
Currenttrendsinoccupationaltherapypractice.Canadian
Journal of Occupational Therapy, 67,197–204.
Gardner, M. F. (1996). Test of Visual–Perceptual Skills (Non-
Motor) revised.Hydesville,CA:Psychological&Educational
Publications.
Goyen, T.-A., & Duff, S. (2005). Discriminant validity of the
DevelopmentalTestofVisual–MotorIntegrationinrelation
tochildrenwithhandwritingdysfunction.Australian Occu-
pational Therapy Journal, 52, 109–115.
Graham,S.,Berninger,V.W.,&Weintraub,N.(1998).Therela-
tionshipbetweenhandwritingstyleandspeedandlegibility.
Journal of Educational Research, 91, 290–296.
Graham, S., Weintraub, N., & Berninger, V. (2001). Which
manuscriptlettersdoprimarygradechildrenwritelegibly?
Journal of Educational Psychology, 93,488–497.
Hinkle,D.R.,Wiersma,W.,&Jurs,S.G.(1998).Applied statis-
tics for the behavioural sciences(4thed.).Boston:Houghton
Mifflin.
Koziatek,S.M.,&Powell,N.J.(2002).Avaliditystudyofthe
EvaluationToolofChildren’sHandwriting–Cursive.Ameri-
can Journal of Occupational Therapy, 56,446–453.
45
 Larsen,S.C.,&Hammill,D.D.(1989).Test of Legible Handwrit-
ing.Austin,TX:Pro-Ed.
Peat,J.(2002).Health science research. London:Sage.
PolenaFeder,K.,&Majnemer,A.(2003).Children’shandwriting
evaluationtoolsandtheirpsychometricproperties.Physical
and Occupational Therapy in Pediatrics, 23,65–84.
Rosenblum,S.,Weiss,P.L.,&Parush,S.(2003).Productand
process evaluation of handwriting difficulties. Educational
Psychology Review, 15,41–81.
Schneck, C. M. (1998). Clinical interpretation of “Test–retest
reliability of the Evaluation Tool of Children’s Handwrit-
ing–Manuscript.”American Journal of Occupational Therapy,
52,256–258.
Sudsawad,P.,Trombly,C.A.,Henderson,A.,&Tickle-Degnen,
L.(2001).TherelationshipbetweentheEvaluationToolof
Children’sHandwritingandteachers’perceptionsofhand-
writinglegibility.American Journal of Occupational Therapy,
55,518–523.
Sudsawad,P.,Trombly,C.A.,Henderson,A.,&Tickle-Degnen,
L.(2002).Testingtheeffectofkinesthetictrainingonhand-
writingperformanceinfirst-gradestudents.American Journal
of Occupational Therapy, 56, 26–33.
46
Downloaded From: http://ajot.aota.org/ on 02/02/2018 Terms of Use: http://AOTA.org/terms
Tape,T.G.(n.d.).The area under an ROC curve.RetrievedSeptem-
ber2,2006,fromhttp://gim.unmc.edu/dxtests/roc3.htm
Tseng,M.H.,&Cermak,S.A.(1993).Theinfluenceofergo-
nomicfactorsandperceptual–motorabilitiesonhandwriting
performance.American Journal of Occupational Therapy, 47,
919–926.
Tseng,M.H.,&Murray,E.A.(1994).Differencesinperceptual–
motor measures in children with good and poor hand-
writing. OTJR: Occupation, Participation and Health, 14,
19–36.
Wallen, M., & Mackay, S. (1999). Test–retest, interrater and
intraraterreliability,andconstructvalidityoftheHandwrit-
ingSpeedTestinyear3andyear6students.Physical and
Occupational Therapy in Pediatrics, 19, 29–42.
Ziviani,J.(1995).Thedevelopmentofgraphomotorskills.InA.
Henderson&C.Pehoski(Eds.),Hand function in the child
(pp.184–193).St.Louis,MO:Mosby.
Ziviani,J.,&Elkins,J.(1984).Effectsofpencilgriponhandwrit-
ingspeedandlegibility.Educational Review, 38,247–257.
Ziviani,J.,&Watson-Will,A.(1998).Writingspeedandlegibility
of7–14-year-oldschoolstudentsusingmoderncursivescript.
Australian Occupational Therapy Journal, 45,59–64.
January/February 2010, Volume 64, Number 1