Dear Author,

Here are the proofs of your article.

• You can submit your corrections online, via e-mail or by fax.

• For online submission please insert your corrections in the online correction form. Always
indicate the line number to which the correction refers.

• You can also insert your corrections in the proof PDF and email the annotated PDF.

• For fax submission, please ensure that your corrections are clearly legible. Use a fine black
pen and write the correction in the margin, not too close to the edge of the page.

• Remember to note the journal title, article number, and your name when sending your
response via e-mail or fax.

• Check the metadata sheet to make sure that the header information, especially author names
and the corresponding affiliations are correctly shown.

• Check the questions that may have arisen during copy editing and insert your answers/
corrections.

• Check that the text is complete and that all figures, tables and their legends are included. Also
check the accuracy of special characters, equations, and electronic supplementary material if
applicable. If necessary refer to the Edited manuscript.

• The publication of inaccurate data such as dosages and units can have serious consequences.
Please take particular care that all such details are correct.

• Please do not make changes that involve only matters of style. We have generally introduced
forms that follow the journal’s style.
Substantial changes in content, e.g., new results, corrected values, title and authorship are not
allowed without the approval of the responsible editor. In such a case, please contact the
Editorial Office and return his/her consent together with the proof.

• If we do not receive your corrections within 48 hours, we will send you a reminder.

• Your article will be published Online First approximately one week after receipt of your
corrected proofs. This is the official first publication citable with the DOI. Further changes
are, therefore, not possible.

• The printed version will follow in a forthcoming issue.

Please note
After online publication, subscribers (personal/institutional) to this journal will have access to the
complete article via the DOI using the URL: http://dx.doi.org/[DOI].
If you would like to know when your article has been published online, take advantage of our free
alert service. For registration and further information go to: http://www.springerlink.com.
Due to the electronic nature of the procedure, the manuscript and the original figures will only be
returned to you on special request. When you return your corrections, please inform us if you would
like to have these documents returned.
Metadata of the article that will be visualized in OnlineFirst

Please note: Images will appear in color online but will be printed in black and white.
ArticleTitle The impact of next and back buttons on time to complete and measurement reliability in computer-based
surveys
Article Sub-Title
Article CopyRight The Author(s)
(This will be the copyright line in the final PDF)
Journal Name Quality of Life Research
Corresponding Author Family Name Hays
Particle
Given Name Ron D.
Suffix
Division
Organization UCLA Department of Medicine
Address 911 Broxton Avenue, Room 110, 90024-2801, Los Angeles, CA, USA
Email drhays@ucla.edu

Author Family Name Bode

Particle
Given Name Rita
Suffix
Division Feinberg School of Medicine
Organization Northwestern University
Address 303 East Chicago Avenue, 60611-3008, Chicago, IL, USA
Email
Author Family Name Rothrock
Particle
Given Name Nan
Suffix
Division Feinberg School of Medicine
Organization Northwestern University
Address 303 East Chicago Avenue, 60611-3008, Chicago, IL, USA
Email
Author Family Name Riley
Particle
Given Name William
Suffix
Division
Organization National Heart, Lung and Blood Institute
Address 6701 Rockledge Dr., MSC 7936, 20892, Bethesda, MD, USA
Email
Author Family Name Cella
Particle
Given Name David
 Suffix
Division Feinberg School of Medicine
Organization Northwestern University
Address 303 East Chicago Avenue, 60611-3008, Chicago, IL, USA
Email
Author Family Name Gershon
Particle
Given Name Richard
Suffix
Division Feinberg School of Medicine
Organization Northwestern University
Address 303 East Chicago Avenue, 60611-3008, Chicago, IL, USA
Email

Received
Schedule Revised
Accepted 21 May 2010
Abstract Purpose:
To assess the impact of including next and back buttons on response burden and measurement reliability of
computer-based surveys.
Methods:
A sample of 807 participants (mean age of 53; 64% women, 83% non-Hispanic white; 81% some college or
college graduates) from the YouGov Polimetrix panel was administered 56 items assessing performance of
social/role activities and 56 items measuring satisfaction with social/role activities. Participants were
randomly assigned to either (1) automatic advance to the next question with no opportunity to go back (auto/
no back); (2) automatic advance to the next questions with an opportunity to go back (auto/back button); (3)
next button to go to the next question with no opportunity to go back (next/no back); or (4) next button to go
to the next question with an opportunity to go back (next/back button).
Results:
We found no difference in missing data, internal consistency reliability, and domain scores by group. Time
to complete the survey was 50% longer when respondents were required to use a next button to go on.
Conclusions:
Given the similarity in missing data, reliability and mean scale scores with or without use of the next button,
we recommend automatic advancement to the next item with the option to go back to the previous item.
Keywords (separated by '-') Computer-based surveys - Social/role activities - PROMIS
Footnote Information
Journal: 11136
Article: 9682

Author Query Form

Please ensure you fill out your response to the queries raised below
and return this form along with your corrections

Dear Author

During the process of typesetting your article, the following queries have arisen. Please
check your typeset proof carefully against the queries listed below and mark the
necessary changes either directly on the proof/online grid or in the ‘Author’s response’
area provided below

Query Details required Author’s response

1. Please check and confirm the authors
and their respective affiliations are
correctly identified.
2. Please confirm the section headings are
correctly identified.
 Qual Life Res
DOI 10.1007/s11136-010-9682-9

2
1

3 The impact of next and back buttons on time to complete

4 and measurement reliability in computer-based surveys
5 Ron D. Hays • Rita Bode • Nan Rothrock •

F
6 William Riley • David Cella • Richard Gershon

OO
Author Proof

7 Accepted: 21 May 2010

8 Ó The Author(s) 2010. This article is published with open access at Springerlink.com

PR
9 Abstract next button, we recommend automatic advancement to the 32
10 Purpose To assess the impact of including next and back next item with the option to go back to the previous item. 33
11 buttons on response burden and measurement reliability of 34
12 computer-based surveys. Keywords Computer-based surveys
35
13 Methods A sample of 807 participants (mean age of 53; Social/role activities
PROMIS 36
14 64% women, 83% non-Hispanic white; 81% some college
ED
15 or college graduates) from the YouGov Polimetrix panel
16 was administered 56 items assessing performance of social/ There are multiple methods of collecting survey data 37
17 role activities and 56 items measuring satisfaction with including mail, phone, fax, and face-to-face interviews. 38
18 social/role activities. Participants were randomly assigned Because access to computers is increasing, there is greater 39
19 to either (1) automatic advance to the next question with no use of computer-based self-administered surveys (e.g., web 40
CT

20 opportunity to go back (auto/no back); (2) automatic surveys) for research and clinical information gathering 41
21 advance to the next questions with an opportunity to go [1, 2]. Responses to self-administered web-based surveys 42
22 back (auto/back button); (3) next button to go to the next have been found to be generally comparable to mail surveys 43
23 question with no opportunity to go back (next/no back); or and have the advantage of eliminating data entry, providing 44
24 (4) next button to go to the next question with an oppor- a basis for estimating response times, and facilitating the 45
E

25 tunity to go back (next/back button). use of complex branching and dynamic assessment [3, 4]. 46
26 Results We found no difference in missing data, internal The presentation and elicitation of answers to survey 47
RR

27 consistency reliability, and domain scores by group. Time questions using the web presents new challenges for 48
28 to complete the survey was 50% longer when respondents researchers. Only a limited number of items can fit on a 49
29 were required to use a next button to go on. computer screen. Rather than marking their answer using a 50
30 Conclusions Given the similarity in missing data, reli- pencil or pen, respondents on web-based surveys click on 51
31 ability and mean scale scores with or without use of the the answer via a mouse or touch-screen technology. Instead 52
of moving down a page manually, the next screen is pre- 53
CO

sented by the computer interface following response to the 54

A1 R. D. Hays (&) previous item. A fundamental question is whether to 55
A2 UCLA Department of Medicine, 911 Broxton Avenue, automatically advance to the next screen as soon as a 56
A3 Room 110, Los Angeles, CA 90024-2801, USA response is provided to the last (perhaps only) question on 57
A4 e-mail: drhays@ucla.edu; hays@rand.org
the screen or to require the respondent to select a ‘‘next’’ 58
UN

A5 R. Bode
N. Rothrock
D. Cella
R. Gershon button. 59

A6 Feinberg School of Medicine, Northwestern University, The automatic advance approach requires less effort and 60
A7 303 East Chicago Avenue, Chicago, IL 60611-3008, USA time than does the next button. However, respondents may 61
initially enter the wrong response. Using a back button 62
A8 W. Riley
A9 National Heart, Lung and Blood Institute, 6701 Rockledge Dr., gives them the opportunity to return and change their 63
A10 MSC 7936, Bethesda, MD 20892, USA response, but respondents may be reluctant to do so once 64

123
Journal : Large 11136 Dispatch : 3-6-2010 Pages : 4
Article No. : 9682 h LE h TYPESET
MS Code : 4 CP
h 4 DISK
h
 Qual Life Res

65 they are presented with the next question. This study

66 evaluates the effects of using next and back buttons on
67 survey completion times, missing data, internal consistency
68 reliability, and mean scale scores.

69 Methods

70 Based on power calculations, we targeted a sample of 800 Fig. 1 Screen shot of sample item

F
71 study participants (200 for each of 4 study arms). With this
72 target sample, we would have 80% power to detect a small the response of the last item displayed. Additionally we 115
73 effect (effect size = 0.28) between two groups. We inclu- reviewed missing data rates, internal consistency reliability 116

OO
74 ded 807 participants from a national, web-based polling [7], and mean scale scores by condition. 117
75 company, Polimetrix (now YouGov Polimetrix, www.
Author Proof

76 polimetrix.com). Over one million adult panel members

77 have provided email addresses, contact information, and Results 118
78 responses to core profile items in order to receive occasional

PR
79 surveys about a variety of subjects [5]. Panel members We identified respondents whose response times were 119
80 generally receive about four surveys per year and earn judged by the study team to be unreasonably fast (i.e.,\2 s 120
81 ‘‘polling points’’ that can be redeemed for rewards. Po- per item): 18 in the auto/no back condition, 10 in the auto/ 121
82 limetrix sent out a study invitation to the panel and the first back condition, 2 in the next/no back condition, and 1 in the 122
83 ones to respond were included. next/back condition. All 18 of these respondents in the auto- 123
84 We administered 56 items assessing performance of no back condition and the 18 respondents in the other three 124
85 social/role activities rated on a 5-category never to always
ED experimental conditions with the fastest times (including 125
86 response scale. Sample items include ‘‘I am limited in those with unreasonably fast times) were excluded from the 126
87 doing my work (include work at home),’’ ‘‘I am able to do analysis. 127
88 all of my regular family activities,’’ and ‘‘I am able to do The remainder of the sample had an average age of 53 128
89 all of my regular leisure activities.’’ We also administered (SD = 16). The majority were women (64%); 83% were 129
90 56 items measuring satisfaction with social/role activities non-Hispanic white, 6% Hispanic, 4% African American, 130
CT

91 using a 5-category not at all to very much response scale. and 7% other. Two percent reported less than a high school 131
92 Sample items include ‘‘I am happy with how much I do for education, 18% were high school graduates, 44% had some 132
93 my family,’’ ‘‘I am satisfied with my ability to work college, and 37% were college graduates. Age, gender, 133
94 (include work at home),’’ ‘‘I am satisfied with my current race, and education did not differ significantly (P [ .05) by 134
95 level of social activity,’’ and ‘‘I am satisfied with my ability experimental condition (see Table 1). 135
E

96 to do leisure activities.’’ These items were preliminary Time to complete differed significantly by study arm: 136
97 versions of item banks created for the Patient-Reported social/role performance (F = 10.36, P \ .001; social/role 137
RR

98 Outcomes Measurement Information System (PROMIS) satisfaction: F = 15.77, P \ .001). The average total 138
99 project [6]. response time for the 56 social/role performance items was 139
100 One item was presented to the study participants on each 4.3 min for the auto-no back group, 4.9 min for the auto- 140
101 screen. Participants were randomized to one of four study back group, 6.5 min for the next-no back group, and 141
102 arms: (1) automatic advance to the next question and no 7.2 min for the next-back group. The average total response 142
103 opportunity to go back (auto/no back); (2) automatic time for the 56 satisfaction items was 4.7 min for the auto- 143
CO

104 advance to the next questions with an opportunity to go no back group, 4.3 min for the auto-back group, 6.5 min for 144
105 back (auto/back button); (3) next button to go to the next the next-no back group, and 7.2 min for the next-back group 145
106 question and no opportunity to go back (next/no back); or (Table 2). Differences between groups with and without a 146
107 (4) next button to go to the next question with an oppor- back button were not statistically significant in Bonferroni- 147
108 tunity to go back (next/back button). The next button was adjusted multiple comparisons. Bonferroni-adjusted differ- 148
UN

109 included for all conditions but using it to advance to the ences between groups with and without a next button were 149
110 next question was only required of persons randomized to statistically significant (P \ .025). The next button 150
111 the 3rd and 4th conditions (see Fig. 1). The next button increased response time by about 2 min overall (2 s per 151
112 could be used by all participants to skip a question. item). 152
113 We evaluate time to complete the items based on web Internal consistency reliability for the 56-item perfor- 153
114 server time elapsed from initiation of first item displayed to mance of social/role activities and 56-item satisfaction with 154

123
Journal : Large 11136 Dispatch : 3-6-2010 Pages : 4
Article No. : 9682 h LE h TYPESET
MS Code : 4 CP
h 4 DISK
h
 Qual Life Res

Table 1 Sample demographic characteristics

Auto/No Back Auto/Back Next/No Back Next/Back Test
(n = 206) (n = 201) (n = 200) (n = 200)

% Female 63 65 64 63 v2(3) = 0.19, P = 0.98

% White 89 92 94 91 v2(3) = 3.04, P = 0.38
% Hispanic 7 6 6 2 v2(3) = 5.13, P = 0.16
% College or college graduate 77 82 83 82 v2(3) = 2.87, P = 0.41
Age (mean) 56 52 57 57 F(3,803) = 0.55, P = 0.65

F
Auto/No Back Automatic advance with no back button; Auto/Back Automatic advance with back button; Next/No Back Next button with no back
button; Next/Back Next with back button

OO
Author Proof

Table 2 Time to complete, missing data, and scale scores by study arm
Auto/No Back Auto/Back Next/No Back Next/Back Test
(n = 206) (n = 201) (n = 200) (n = 200)

PR
Time to complete: Social/Role Performance 4.3 min 4.9 min 6.5 min 7.2 min
Time to complete: Social/Role Satisfaction 4.7 min 4.3 min 6.5 min 6.8 min
Missing data: Social/Role Performance 0.63 1.23 1.60 1.01 F(3, 803) = 1.62,
P = 0.18
Missing data: Social/Role Satisfaction 1.00 0.87 1.51 1.13 F(3, 803) = 0.65,
P = 0.58
Scale score mean and SD: Social/Role 210.30
ED
206.89 211.92 211.98 F(3, 800) = 0.44,
Performance SD = 54.40 SD = 50.37 SD = 47.61 SD = 49.16 P = 0.73
Scale score mean and SD: Social/Role 210.17 210.43 214.54 213.65 F(3, 803) = 0.38,
Satisfaction SD = 53.55 SD = 53.40 SD = 50.03 SD = 51.16 P = 0.77
Auto/No Back Automatic advance with no back button; Auto/Back Automatic advance with back button; Next/No Back Next button with no back
button; Next/Back Next with back button. Scales included 56 items with 5 response choices each scored 1–5. Possible range of scale scores was
CT

56–280 with a higher score representing better social/role performance and satisfaction

155 social/role activities was 0.99 for each of the experimental power to detect interaction effects by educational 173
156 groups. We also found no significant differences in missing attainment. 174
E

157 data rates or scale scores by group (see Table 2). Including a next button is important to allow respon- 175
dents to skip items they prefer not to answer. However, 176
RR

requiring use of the next button after each response 177

158 Discussion increases response time without changing the quality of the 178
data. Hence, we recommend that after item responses are 179
159 Time to complete the survey was 50% longer when made that there be automatic advancement to the next item 180
160 respondents were required to use a next button after with the opportunity to go back to the previous item for 181
161 answering an item to go to the next item. Missing data, computer-based surveys, particularly in more educated and 182
CO

162 reliability, and mean scale scores were similar across the experienced survey respondents. This approach helps to 183
163 randomized groups. However, it is important to note that minimize response burden but retain a safeguard for 184
164 our sample was highly educated and experienced survey respondents who initially enter an incorrect response. 185
165 respondents; therefore, these results may not generalize to We also note that the automatic advancement with no 186
166 other samples. Future research needs to be conducted to back option results in 18 respondents who responded so 187
UN

167 determine if requiring the use of a next button provides rapidly (\2 s per item) that their attention to each item was 188
168 benefit in terms of data quality for less educated or less questionable. In contrast, there were 10 people in the 189
169 computer experienced respondents. It is possible that the automatic advancement with a back option that responded 190
170 effect of including the next button on missing data or this quickly and 1–2 people in the next button conditions. 191
171 reliability depends on level of education. We were unable Therefore, the presence of a back button not only provides a 192
172 to address this issue in our study because we had limited safeguard for respondents who enter an incorrect response 193

123
Journal : Large 11136 Dispatch : 3-6-2010 Pages : 4
Article No. : 9682 h LE h TYPESET
MS Code : 4 CP
h 4 DISK
h
 Qual Life Res

194 to the previous item but was also found to reduce the six Primary Research Sites (Duke University, PI: Kevin Weinfurt, 234
195 number of respondents who spent so little time answering a PhD, U01AR52186; University of North Carolina, PI: Darren DeW- 235
alt, MD, MPH, U01AR52181; University of Pittsburgh, PI: Paul A. 236
196 question that it was unlikely they gave it much thought. The Pilkonis, PhD, U01AR52155; Stanford University, PI: James Fries, 237
197 automatic advance option may be particularly helpful for MD, U01AR52158; Stony Brook University, PI: Arthur Stone, PhD, 238
198 people with physical limitations that make fine motor U01AR52170; and University of Washington, PI: Dagmar Amtmann, 239
199 control of a mouse more difficult or fatiguing (e.g., patients PhD, U01AR52171). NIH Science Officers on this project are 240
Deborah Ader, PhD, Susan Czajkowski, PhD, Lawrence Fine, MD, 241
200 with Parkinson’s disease, spinal cord injury). DrPH, Louis Quatrano, PhD, Bryce Reeve, PhD, William Riley, PhD, 242
201 All of the items tested in this study allowed for only one and Susana Serrate-Sztein, PhD. Ron D. Hays, PhD, was also sup- 243
202 answer. ‘‘Select all that apply’’ or numeric/text entry are ported by grants from the National Institute on Aging (AG020679-01, 244
245

F
203 not compatible with an automatic advance format as the P30AG021684), NCMHD (2P20MD000182), and the UCLA Older
Americans Independence Center (P30-AG028748). See the web site 246
204 computer would not know when a participant had finished at www.nihpromis.org for additional information on the PROMIS 247
205 selecting all desired responses or entering all data. There-

OO
cooperative group. 248
206 fore, future usability testing may address to what extent
207 switching between automatic advance and use of a next Open Access This article is distributed under the terms of the 249
Author Proof

Creative Commons Attribution Noncommercial License which per- 250

208 button in a single survey would be confusing to partici- mits any noncommercial use, distribution, and reproduction in any 251
209 pants. It might also be interesting to evaluate the relative medium, provided the original author(s) and source are credited. 252
210 merits of the single question per screen used in this study 253

PR
211 versus presenting multiple questions per screen.
212 Allowing respondents the option to go back and change
213 the prior response has implications for computer-adaptive References 254
214 testing (CAT). Because the items presented later in the
1. Wyatt, J. C. (2000). When to use web-based surveys. Journal of 255
215 administration are dependent on answers to previous items, 256
the American Medical Informatics Association, 7, 426–429.
216 the next item presented could change when a respondent 2. Schmidt, W. C. (1997). World-wide web survey research: Benefits, 257
217 changes their initial response. Additional work is needed to
ED potential problems, and solutions. Behavior Research Methods, 258
218 document whether this produces any confusion among Instruments, & Computers, 29, 274–279. 259
3. Christian, L. M., Parsons, N. L., & Dillman, D. A. (2009). 260
219 respondents. Similarly, if an item that is answered incor- 261
Designing scalar questions for web surveys. Sociological Methods
220 rectly initially leads to satisfaction of a threshold standard & Research, 37, 393–425. 262
221 error for the CAT, it may be important to delay termination 4. Coons, S. J., Gwaltney, C. J., Hays, R. D., Lundy, J. J., Sloan, J. A., 263
222 of the CAT until the respondent has an opportunity to go Revicki, D. A., et al. (2009). Recommendations on evidence needed 264
CT

to support measurement equivalence between electronic and paper- 265

223 back to that final item. 266
based patient-reported outcome (PRO) measures: ISPOR ePRO
224 In summary, this study provides support for automatic good research practices task force report. Value in Health, 12, 419– 267
225 advance to the next item with the option to go back to the 429. 268
226 previous item. 5. Polimetrix, Inc. (2006). Scientific sampling for online research. 269
Palo Alto, CA: Author. 270
E

227 6. Cella, D., Yount, S., Rothrock, N., Gershon, R., Cook, K., Reeve, B., 271
Acknowledgments The Patient-Reported Outcomes Measurement
228 et al. (2007). The patient reported outcomes measurement informa- 272
Information System (PROMIS) is a US National Institutes of Health
229 tion system (PROMIS): Progress of an NIH roadmap cooperative 273
(NIH) Roadmap initiative to develop a computerized system mea-
RR

230 group during its first two years. Medical Care, 45(Suppl. 1), S3–S11. 274
suring patient-reported outcomes in respondents with a wide range of
231 7. Cronbach, L. (1951). Coefficient alpha and the internal structure of 275
chronic diseases and demographic characteristics. PROMIS was
232 tests. Psychometrika, 16, 297–334. 276
funded by cooperative agreements to a Statistical Coordinating Center
233 (Northwestern University, PI: David Cella, PhD, U01AR52177) and 277
CO
UN

123
Journal : Large 11136 Dispatch : 3-6-2010 Pages : 4
Article No. : 9682 h LE h TYPESET
MS Code : 4 CP
h 4 DISK
h