You are on page 1of 12

Research Report

A Further Step to Develop Patient-


Friendly Implementation Strategies
for Virtual Reality–Based
Rehabilitation in Patients With
Acute Stroke
Minyoung Lee, Sung-Bom Pyun, Jinjoo Chung, Jungjin Kim, Seon-Deok Eun,
BumChul Yoon
M. Lee, PT, BSc, Department of
Physical Therapy, College of
Health Sciences, Korea University,
Background. Virtual reality (VR)– based rehabilitation is gaining attention as a way to
Seoul, Republic of Korea. promote early mobilization in patients with acute stroke. However, given the motor weakness
and cognitive impairment associated with acute stroke, implementation strategies for over-
S-B. Pyun, MD, PhD, Department coming patient-perceived difficulty need to be developed to enhance their motivation for
of Physical Medicine and Rehabil- training.
itation, College of Medicine,
Korea University.
Objective. The purpose of this study was to explore patient-perceived difficulty and
J. Chung, PhD, Institute for Soci- enjoyment during VR-based rehabilitation and the factors affecting those experiences.
ety & Health, Seoul, Republic of
Korea.
Design. An exploratory mixed-method design was used in this study.
J. Kim, PT, MS, Department of
Physical Therapy, College of
Health Sciences, Korea University. Methods. Eight individuals with acute stroke participated in 2 training modes of VR-based
rehabilitation (ie, workout and game modes) 20 to 30 minutes per day for 5 to 8 sessions. A
S-D. Eun, PhD, Korea National visual analog scale was used to assess patient-perceived difficulty and enjoyment at every
Rehabilitation Research Institute, session. Then semistructured interviews were conducted to explore the factors affecting those
Seoul, Republic of Korea.
experiences.
B. Yoon, PT, OT, PhD, Depart-
ment of Physical Therapy, College Results. Levels of difficulty and enjoyment varied depending on the training mode and
of Health Sciences, Korea Univer-
participants’ phases of recovery. Five major factors were identified as affecting those varied
sity 145, Anam-ro, Seongbuk-go,
experiences: (1) ease of following the directions, (2) experience of pain, (3) scores achieved,
Seoul 136-701, Republic of
Korea. Address all correspon-
(4) novelty and immediate feedback, and (5) self-perceived effectiveness.
dence to Professor Yoon at:
yoonbc@korea.ac.kr. Conclusions. Levels of difficulty and enjoyment during VR-based rehabilitation differed
[Lee M, Pyun S-B, Chung J, et al. A
depending on the phases of recovery and training mode. Therefore, graded implementation
further step to develop patient- strategies for VR-based rehabilitation are necessary for overcoming patient-perceived difficulty
friendly implementation strategies and enhancing enjoyment. Ease of following the directions might be best considered in the
for virtual reality– based rehabilita- very early stage, whereas multisensory feedback may be more necessary in the later stage.
tion in patients with acute stroke. Health professionals also should find a way for patients to avoid pain during training. Feedback,
Phys Ther. 2016;96:1554 –1564.] such as knowledge of results and performance, should be used appropriately.
© 2016 American Physical Therapy
Association

Published Ahead of Print:


May 5, 2016
Accepted: April 28, 2016
Submitted: May 10, 2015

Post a Rapid Response to


this article at:
ptjournal.apta.org

1554 f Physical Therapy Volume 96 Number 10 October 2016


Patient-Friendly Strategies for Virtual Reality–Based Rehabilitation and Acute Stroke

B enefits of early mobilization after


stroke have been emphasized in
previous studies. For example, a
minimum of 16 hours of exercise ther-
apy in the early stage has been reported
to overcome such challenges need to be
developed.

Meanwhile, flow experience has re-


cently emerged as a primary factor affect-
ing VR-based rehabilitation based on this
theoretical framework and the factors
that may affect these experiences.

Therefore, in this study, we explored the


to induce a 5% increase in activities of ing patient adherence to VR-based reha- levels of difficulty and enjoyment that
daily living (ADL) performance and to bilitation.15,16 One qualitative study patients experienced during VR-based
improve long-term physical outcomes.1–3 reported that patients with acute stroke rehabilitation to verify whether the
Nevertheless, the actual time that experienced positive feelings, such as enjoyment level could be an indicator of
patients spend on exercise is rather “enjoyment” and “engagement,” while the appropriateness of the level of diffi-
low4 due to practical limitations, such as conducting VR-based rehabilitation, culty for patients with acute stroke who
the time-consuming and labor-intensive causing them to think of rehabilitation as had differing physical and cognitive sta-
nature of conventional therapy and diffi- a meaningful occupation.15 Another tus, as Flow Theory hypothesizes. We
culty traveling to special facilities.5,6 study indicated that patients frequently also explored the possible factors affect-
reported that they were so engaged in ing such difficulty and enjoyment in
Virtual reality (VR)– based rehabilitation the VR-based rehabilitation that they lost order to identify a means of overcoming
in the early stage of recovery has track of time while watching real-time such difficulty and enhancing enjoyment
received attention as a way of filling this feedback and perceived that “a good during VR-based rehabilitation. Finally,
gap between reality and ideals, owing to intervention” would help them perse- we examined the effects of VR-based
its capacity to provide high-intensity, vere with rehabilitation for long periods rehabilitation on UE function and ADL
repetitive, and task-oriented training.7,8 by providing a flow experience.16 From performance in order to identify physi-
Furthermore, the outstanding character- these studies, we can assume that main- cal benefits. The results of this study pro-
istic of VR training—multisensory feed- taining a state of flow may be important vide information on implementation
back9,10—allows patients to recognize in inducing patients adherence to strategies necessary to set an appropriate
their own level of performance and to VR-based rehabilitation. level of difficulty, which enables patients
adjust their posture, which increases its with acute stroke to achieve a high
utility as a self-rehabilitation tool after The means of maintaining a flow state is level of enjoyment during VR-based
hospital discharge. suggested by Flow Theory.17 Flow The- rehabilitation.
ory defines flow as the feeling of com-
The effects of VR-based rehabilitation in plete and energized engagement in an Method
patients with acute stroke have been ver- activity, with a high level of enjoyment Participants
ified in several previous studies. Some and fulfillment. The theory hypothesizes A total of 11 inpatients with stroke were
randomized controlled trials11–14 exam- a Flow Zone, meaning that a person recruited from a rehabilitation center at
ined the effects of VR-based rehabilita- would be at a high level of enjoyment Korea University Anam Hospital in Seoul,
tion on upper extremity (UE) function when the level of difficulty of a task is South Korea, and assessed for their eligi-
using self-developed11 or commercially appropriate, with a balance between the bility. Three participants were excluded,
available12–14 devices. Virtual reality– difficulties of the task and the abilities of resulting in 8 enrolled participants. Par-
based rehabilitation showed improve- the person. Hence, according to Flow ticipants were chosen to reflect diversity
ments similar to those of conventional Theory, the level of enjoyment could be in age, sex, and physical and cognitive
physical therapy and occupational ther- an indicator of whether the level of dif- impairments. In particular, because we
apy,11,12,14 but greater than with recre- ficulty is appropriate. intended to achieve a broad representa-
ational therapy, such as playing cards.13 tion of physical and cognitive impair-
This theoretical framework was applied ments, we did not put a specific limit on
Even with these positive effects, caution in a study that examined the levels of physical or cognitive function when
should be taken when applying VR-based difficulty and enjoyment for 6 kinds of recruiting participants. Potential partici-
rehabilitation in patients in the early VR-based exercises in healthy young pants were first screened by a physiatrist
stage of recovery. Most patients in this adults.18 In this previous study, partici- for the following inclusion criteria: (1)
period have some extent of motor weak- pants rated the level of enjoyment high age ⱖ18 years, (2) onset of unilateral
ness and cognitive impairment and, when they perceived the difficulty to be stroke within the past 2 months, and (3)
therefore, are prone to experience phys- in the lower middle level, indicating that manual muscle test score of the affected
ical and cognitive challenges during the content had appropriate difficulties, shoulder joint ⬎1 (trace). Patients were
VR-based rehabilitation, which could balanced between the difficulties of the excluded if they had the following: (1)
result in depression or abandonment of task and the abilities of the person.18 severe listening or visual impairment, (2)
training.15 Thus, to facilitate continuous In patients with acute stroke, recent transient ischemic attack, (3) epilepsy,
participation in VR-based rehabilitation, studies qualitatively explored patient (4) headaches, or (5) dizziness. Written
patient-perceived difficulty during train- experiences during VR-based rehabilita- informed consent was obtained from all
ing needs to be explored, and strategies tion.15,16 However, little is known about participants.
the level of difficulty and enjoyment dur-

October 2016 Volume 96 Number 10 Physical Therapy f 1555


Patient-Friendly Strategies for Virtual Reality–Based Rehabilitation and Acute Stroke

In the workout mode of training,


participants could identify their own
3-dimensional avatar on the right side of
the screen, while simultaneously seeing
a virtual instructor, who appeared as a
human-like figure on the left side of the
screen; this way, participants could imi-
tate the movements of the virtual instruc-
tor. The following 4 patterns of UE move-
ment were guided by the virtual
instructor: (1) diagonal 1 flexion, (2)
diagonal 1 extension, (3) diagonal 2 flex-
ion, and (4) diagonal 2 extension. Diag-
onal 1 flexion and extension patterns
were combined in one session, as the
diagonal 1 extension pattern is the
reverse of the diagonal 1 flexion pattern.
Diagonal 2 flexion and extension pat-
terns also were combined for the same
reason. Two combined movement pat-
terns were designed to be repeated 8
times in sequence as a blocked practice
for a total of 5 minutes (Fig. 1A and B).
Figure 1.
Virtual reality (VR)– based rehabilitation software tailored to patients with hemiplegic stroke.
(A) and (B) show a participant performing the workout mode of training and the VR content
During the workout mode of training,
reflected back to the participant. (C) and (D) show a participant performing the game mode the software provided visual and audi-
of training and the VR content including a hammer (participant’s hand movement) and a tory feedback for the participants,
mole (target). The software was developed based on a proprioceptive neuromuscular facil- including knowledge of results (KR) and
itation stretching technique. knowledge of performance (KP). For
example, if the participant’s movement
was correct, the software provided ver-
bal praise, such as “excellent” or “good
Study Design Software. We designed a software job,” with light flashing around the par-
An exploratory mixed-method design program tailored to patients with hemi- ticipant’s avatar (KR). However, if the
using quantitative and qualitative analy- plegic stroke in the Applied Neuro- movement was not correct, the software
ses was adopted in this study.19,20 The Dynamic Laboratory of Korea University encouraged the participant with a com-
mixed-method design enabled collection through consultation with physical ther- ment, such as “raise your arm more” or
of information on participants’ experi- apists who routinely worked with this “try harder” (KP).
ences during VR-based rehabilitation and population at the hospital, and the ani-
the factors affecting those experiences mations were created by a specialized In the game mode of training, a mole
from various perspectives. company (Unimation Korea Inc, Seoul, appeared in a cave as a target, while a
South Korea). The therapeutic method hammer reflected the participant’s hand
VR-Based Rehabilitation System embedded in the software was based on movement. The moles were designed to
Hardware. The VR system used in this a proprioceptive neuromuscular facilita- appear one at a time in the following
study utilized a motion capture sensor tion stretching technique comprising patterns in random order: (1) right up
(Kinect, Microsoft Inc, Redmond, Wash- diagonal and spiral patterns of move- and then left down, (2) right down and
ington), portable notebook computer, ment using multiple joints. This tech- then left up, (3) left up and then right
and 127-cm (50-in) display monitor. The nique is widely used clinically to increase down, and (4) left down and then right
motion capture sensor, which was range of motion and relax the muscles,21 up. In the process of hitting the moles
placed on top of the display monitor, and its effects on improving UE function with the hammer, participants per-
tracked and captured 20 body segments in patients with stroke have been veri- formed diagonal 1 and 2 flexion and
of the participants and translated the col- fied.22,23 We designed 2 different training extension patterns of movement. The
lected data into on-screen action. modes to compare participants’ prefer- game took 5 minutes to complete
Thereby, participants were able to iden- ences and experiences: (1) a workout (Fig. 1C and D).
tify their own 3-dimensional avatar mode and (2) a game mode. Figure 1
reflected in a virtual environment on the shows participants conducting each During the game mode of training, if the
2-dimensional monitor screen without mode of training. hammer hit the mole with the correct
any haptic devices or controllers. angle and velocity, the mole disap-

1556 f Physical Therapy Volume 96 Number 10 October 2016


Patient-Friendly Strategies for Virtual Reality–Based Rehabilitation and Acute Stroke

peared, grimacing and saying “Ouch!” Procedure score (24 –30⫽normal, 18 –23⫽mild,
(KR). If the hammer did not hit the mole, Each participant attended 5 to 8 sessions and 0 –17⫽severe).27
it just disappeared from the cave within of the VR-based training program, which
5 seconds. The software guided the were held 3 days per week for approxi- A visual analog scale (VAS)—a 10-cm hor-
direction from where the mole would mately 20 to 30 minutes per day at the izontal line with anchor words placed at
appear with an arrow so that participants rehabilitation center of the hospital. both ends—was used to assess the levels
could easily perform diagonal 1 and 2 Other routine treatments, such as con- of difficulty (very easy–very challenging)
flexion and extension patterns of move- ventional physical therapy or occupa- and enjoyment (very boring–very enjoy-
ment (KP). Number of points accumu- tional therapy, were continued as usual able) of the participants after completion
lated by successfully hitting the mole during the intervention period. of each training mode at every session.28
was presented as scores in real time on Self-rated mood status assessment using a
the screen (KR). The software provided One researcher (J.K.), a certified physical VAS has been shown to have high reli-
participants’ total scores on the screen therapist, supervised the participants ability and validity in the health and clin-
after completion of both workout and throughout the intervention period and ical domains.28
game modes of training (KR). recorded the following data at every ses-
sion: (1) side of UE used for training Semistructured interviews were used to
Training Protocol (unaffected, affected, or bilateral), (2) further explore participants’ experi-
The training protocol consisted of work- level of assistance required for training ences and the factors affecting patient-
out and game modes of training, each of (none, minimal, or maximal), and (3) perceived difficulty and enjoyment after
which was conducted twice consecu- experience of pain. In particular, we completion of the entire intervention
tively, resulting in 4 sections for about 20 defined the level of assistance required period. Interviews were semistructured
minutes. However, participants were for training as follows: none (the to help participants answer the ques-
allowed to conduct each mode of train- patient did not require any verbal or tions easily and to facilitate participant
ing just once depending on their condi- physical assistance), minimal (the storytelling with open-ended questions
tion by judgment of the supervising patient required verbal cues or hand around topics, including: (1) Which
researcher. The order of the training commands without physical assistance), training mode, if any, was more challeng-
modes was planned using an online ran- or maximal (the patient required physi- ing or enjoyable? and (2) What aspects of
dom number generator (http://www. cal assistance). the training mode made you feel chal-
random.org) at every session to avoid lenged or enjoyment? Verbal and physi-
order bias. After finishing one mode of To explore participants’ experiences cal reactions during the interviews also
training, participants were given an during VR-based rehabilitation, levels of were recorded by the researcher (J.K.) in
approximately 5-minute break. Partici- difficulty, enjoyment, and training inten- the field notes,29 under the guidance of
pants were asked to conduct the training sity were assessed quantitatively at every the first author. Some caregivers were
in the sitting position. session. To further explore participants’ asked to accompany participants during
experiences and the factors affecting the interviews to provide clarification
The first day of training served as a those experiences, the first author (M.L.) and background about participant com-
“familiarization phase,” during which conducted a single semistructured inter- ments. Each interview took approxi-
participants were provided a guide to view immediately after completion of mately 30 minutes to complete. All inter-
practice the 2 different training modes the entire intervention. Physical out- views were audio-recorded with consent
using the unaffected UE; the familiariza- comes were assessed at baseline and of the participants and caregivers.
tion phase could be prolonged based on after completion of the entire interven-
a participant’s demands. After that, the tion by 2 intervention-blinded physical For training intensity, ratings of per-
“treatment phase” began, during which therapists with more than 3 years of clin- ceived exertion (RPEs) were used to
participants were asked to conduct the ical experience. measure patient-perceived training inten-
training using the affected UE or bilateral sity after completion of training at every
UEs depending on the judgment of the Data Collection session using the following ratings: 0 (no
physiatrist at the rehabilitation center. At baseline, demographic (age and sex) exertion at all), 1 (very light), 2 (light), 3
However, even if the physiatrist recom- and clinical (stroke etiology, site of (moderate), 4 (somewhat hard), 5 (hard),
mended use of the affected UE, the par- stroke, stroke onset, and severity of phys- 6 –7 (very hard), and 8 –10 (very, very
ticipants were allowed to use bilateral ical and cognitive impairments) data hard).30 A portable heart rate monitor
UEs if they asked to do so. Use of bilateral were obtained. Severity of stroke was (RS800CX, Polar Electro Inc, Lake Suc-
UEs was permitted for participants to based on the National Institutes of cess, New York) and heart rate sensor
avoid pain and due to the verified effects Health Stroke Scale (NIHSS) score (H2, Polar Electro Inc) were used to mea-
of bilateral arm training on improving UE (0⫽normal, 1– 4⫽minor, 5–15⫽moder- sure training intensity objectively in both
function after stroke.24,25 ate, 16 –20⫽moderately severe, and modes of training at every session.
⬎20⫽severe).26 Severity of cognitive Before starting each mode of training,
impairment was based on the Mini- each participant’s resting heart rate
Mental State Examination (MMSE)

October 2016 Volume 96 Number 10 Physical Therapy f 1557


Patient-Friendly Strategies for Virtual Reality–Based Rehabilitation and Acute Stroke

(RHR) was measured for 5 minutes in a ment with more than 10 years of clinical enjoyment using verbal or physical reac-
sitting position. experience and a sociologist with more tions, so we allowed them to continue
than 20 years of qualitative research the training. Participants’ age (32– 84
For physical outcome, the UE portion of experience) to verify whether partici- years), sex (male, n⫽4; female, n⫽4),
the Fugl-Meyer Assessment (UE-FMA)31 pants’ experiences and factors affecting and severity of stroke (NIHSS score⫽3–
was used to evaluate motor recovery in those experiences were accurately and 13) and cognitive impairment (MMSE
reflexes, range of motion, synergy, sen- fully represented. score⫽10 –29) varied. Participants’ char-
sation, and hand movement. The UE-FMA acteristics are shown in Table 1.
uses a scale of 0 to 66, and the score can For training intensity, the median RPE for
be divided into the following 4 parts: (1) each participant was obtained by arrang- Participants’ Experiences
shoulder, elbow, and forearm (36 ing the values from all sessions from low- Figure 2 shows the experiences of 4 par-
points); (2) wrist (10 points); (3) hand est to highest and choosing the middle ticipants during both modes of VR-based
(14 points); and (4) coordination (6 value. The mean heart rate (MHR) of rehabilitation and indicates the side of
points). The Manual Function Test each participant was obtained by averag- UE used for training. The experiences of
(MFT)32 was used to evaluate UE motor ing all values. For physical outcome mea- the other 4 participants, who had a dif-
dexterity. The MFT uses a scale of 0 to sures (ie, UE-FMA, MFT, BBT, and MBI), ficulty in measuring the levels of diffi-
32, and the score can be divided into the within-group changes from baseline to culty and enjoyment, were explored
following 2 parts: (1) shoulder, elbow, follow-up were analyzed using the Wil- through semistructured interviews,
and forearm (16 points) and (2) hand (16 coxon signed rank test because of the including their physical and verbal reac-
points). The Box and Block Test (BBT)33 small sample size (N⫽8). Significance tions. As training changed from the famil-
was used to evaluate gross manual dex- levels were set at P⬍.05 for all analyses. iarization phase to the treatment phase,
terity. The Modified Barthel Index Effect size (ie, Cohen d)37 was calculated the level of difficulty of some partici-
(MBI)34 was used to measure basic ADL by dividing the mean change scores by pants (n⫽3) tended to increase, whereas
performance as follows: 0 –24 (totally the standard deviation of the baseline that of enjoyment decreased at the same
dependent), 25– 49 (severely depen- scores. Effect size values above 0.80 time.
dent), 50 –74 (moderately dependent), were regarded as large, values of 0.50 to
75–90 (mildly dependent), 91–99 (mini- 0.80 were regarded as moderate, and val- In the treatment phase, participant 1 per-
mally dependent), and 100 (totally ues of 0.20 to 0.50 were regarded as formed training using bilateral UEs due
independent). small. Quantitative measurements were to motor weakness, without any assis-
analyzed using IBM SPSS version 21.0 tance. In the workout mode of training,
Data Analysis (IBM Corp, Armonk, New York). the level of enjoyment was consistently
The levels of difficulty and enjoyment, high, whereas that of difficulty was con-
which were measured using the VAS at Role of the Funding Source sistently low. In comparison, in the game
each session, are presented for each This research was supported by R&D mode of training, the level of difficulty
mode of training in Figure 2. For qualita- grant (No. 2013006) on rehabilitation by was slightly increased until the third ses-
tive analysis, all audio-recorded inter- Korea National Rehabilitation Center sion and then decreased.
views and verbal and physical reactions Research Institute, Ministry of Health &
recorded in the field notes were tran- Welfare. Participant 2 used the affected UE until
scribed verbatim into text by 2 the fourth session, following the physia-
intervention-blinded researchers to avoid Results trist’s recommendation, but changed to
selective coding of information.35 A sum- Eleven inpatients were recruited, but 3 bilateral UEs due to pain. She could con-
mative content analysis was used to eval- were excluded for the following reasons: duct the workout mode of training with-
uate the transcribed interviews and ver- unwillingness to participate (n⫽1) and out any assistance, whereas she needed
bal and physical reactions, which ineligibility according to the inclusion minimal assistance in the game mode of
involved counting and comparing key criteria (n⫽2). Thus, data analyses were training because she confused the direc-
words in the text with the object of based on 8 participants who attended 5 tion in which the hammer and moles
understanding the contextual meaning to 8 sessions of the VR-based training were located without verbal cues. In the
of the words.36 Each of the researchers program before being discharged from workout mode of training, the level of
independently coded the transcribed the hospital. However, in the course of enjoyment was consistently high follow-
interviews by sentence and determined the intervention period, we excluded 4 ing the third session, whereas that of
the main factors affecting patient- participants from VAS and RPE assess- difficulty gradually decreased. On the
perceived difficulty and enjoyment based ment because we judged that they could contrary, in the game mode of training,
on key words that were mentioned in the not reasonably determine the level of dif- the level of enjoyment gradually
codes. Finally, emerged factors and ficulty, enjoyment, and training intensity decreased, but that of difficulty tended to
sorted codes were assessed for their by indicating a level on the horizontal increase.
credibility through discussion with all line of the VAS. However, they had no
research members and 2 specialists (a significant difficulty performing the train- Participant 3 performed training using
professor of the physical therapy depart- ing and expressing their difficulty and bilateral UEs throughout the treatment

1558 f Physical Therapy Volume 96 Number 10 October 2016


Patient-Friendly Strategies for Virtual Reality–Based Rehabilitation and Acute Stroke

Participant 4 used bilateral UEs until the


fourth session on his demand, although
the physiatrist recommended use of the
affected UE, and then changed to the
affected UE from the fifth session. He
performed both modes of training with-
out any assistance by the time of the final
session, but in earlier sessions, he
showed a poor sense of velocity during
the game mode of training and required
minimal assistance. In the workout mode
of training, the level of enjoyment grad-
ually increased, whereas that of difficulty
abruptly increased in the middle of the
sessions when the training side changed
from bilateral UEs to the affected UE and
then decreased again. In the game mode
of training, the level of enjoyment was
rather low until the sixth session and
then sharply increased in the latter ses-
sions, whereas that of difficulty was
rather high until the middle of the ses-
sions and then slightly decreased in the
latter sessions.

Participant 5 performed training using


the affected UE without any assistance in
the workout mode of training, but she
could not catch the moles without min-
imal assistance in the game mode of
training. She was always ready to per-
form the training before the software
began, taking the training pose for her-
self. Her caregiver commented that she
did not like to go to other therapies, such
as using the cycle ergometer, and did not
cooperate with the therapists during
such training, as she did during VR-based
training.

Participants 6 through 8 performed train-


ing using bilateral UEs. Participant 6 per-
formed the workout mode of training
without assistance, showing a positive
attitude, including counting for himself
during training, whereas he required
minimal assistance during the game
mode of training. Participant 7 required
Figure 2. minimal assistance to keep participating
Patient-perceived levels of difficulty and enjoyment in each mode of training. “u” on the in the workout mode of training,
x-axis indicates the unaffected upper extremity was used; “b” indicates bilateral upper
whereas he could not continue the game
extremities were used. Otherwise, the affected upper extremity was used. P⫽participant.
mode of training without a physical ther-
apist’s active physical assistance until the
final session. Participant 8 initially could
phase due to motor weakness and com- moderate throughout the intervention not understand the rules of either modes
pensational movements and performed period, whereas that of enjoyment of training and did not actively cooperate
both modes of training without any assis- increased at the second session, slightly with the physical therapist’s assistance.
tance. In both modes of training, the dropped, and then was consistently However, from the third session, she
level of difficulty was consistently moderate.

October 2016 Volume 96 Number 10 Physical Therapy f 1559


Patient-Friendly Strategies for Virtual Reality–Based Rehabilitation and Acute Stroke

Table 1.
Participants’ Demographic and Clinical Characteristicsa

Participant Age More Affected Days Since Stroke NIHSS MMT MMSE
No. (y) Sex Side Onset Type Stroke Side Score Score Score

1 76 Female Right 20 Ischemic Left MCA 3 2⫹ 28

2 72 Female Left 37 Ischemic Right MCA, PCA 4 3⫺ 29

3 52 Male Left 36 Ischemic Right MCA 6 4 23

4 32 Male Right 16 Ischemic Left MCA 10 3⫺ 25

5 84 Female Right 33 Ischemic Cerebellar, PICA 3 4 14

6 36 Male Left 30 Hemorrhage Right MCA 13 2⫺ 17

7 77 Male Right 24 Ischemic Left MCA 7 2⫹ 10

8 77 Female Right 20 Hemorrhage Left MCA 7 4 14


a
MCA⫽middle cerebral artery, MMSE⫽Mini-Mental State Examination, MMT⫽manual muscle test, NIHSS⫽National Institutes of Health Stroke Scale,
PCA⫽posterior cerebral artery, PICA⫽posterior inferior cerebellar artery.

became to perform the workout mode of while conducting the game mode of not seem very difficult to me, but when
training with minimal assistance. training. Participant 5 also consistently I actually tried it, the scores were disap-
mentioned, “The workout mode of train- pointing. This made me depressed, and I
In summary, the majority of participants ing is more enjoyable, and the game came to prefer the workout mode. In the
(n⫽6) could perform the workout mode mode is too difficult for me.” Participant later sessions, as the range of motion of
of training independently but required 7 stated that he preferred the workout my shoulder increased, the scores also
minimal (n⫽3) to maximal (n⫽3) assis- mode of training because it is easy to increased, so I again enjoyed the game.”
tance in the game mode of training. Two conduct.
participants (participants 1 and 3) Novelty and Immediate Feedback
showed a relatively similar pattern of Experience of Pain Immediate feedback in the process of
changes in patient-perceived difficulty Experience of pain during training was training also emerged as a factor related
and enjoyment during both modes of found to be a major factor affecting to enjoyment. For example, participant 1
training, whereas 2 other participants patient-perceived difficulty and enjoy- preferred the game mode to the workout
(participants 2 and 4) showed different ment. For example, participant 2 said mode, as well as conventional therapy,
patterns of changes during each mode of that she became reluctant to do the train- because the game mode of training felt
training. ing due to pain in the back of her neck like just play for her. She stated, “Actu-
and flank while performing the training ally, the game was more difficult (than
Factors Affecting Participants’ using her affected UE at the second the workout mode) because I could not
Experiences through fourth sessions; she was absent predict where the moles would appear.
Five factors were identified as affecting from the training at the fifth session. Par- Nevertheless, I felt that the game was
patient-perceived difficulty and enjoy- ticipant 4 said, “I was afraid of coming to more enjoyable because if I caught the
ment: (1) ease of following the direc- conduct the training until the third ses- moles, they disappeared with a sound
tions, (2) experience of pain, (3) scores sion because it made me feel pain in my such as ‘Ouch!’ and it made me feel like
achieved, (4) novelty and immediate flank during the weekend.” Participant 8 being a child again.” She also said, “This
feedback, and (5) self-perceived said, “I initially felt enjoyment for the training (game mode) felt like just play
effectiveness. novelty of the training applying the VR for me, so it is more fun than the cycle
system; however, I could not feel enjoy- ergometer. . . . It (cycle ergometer) is so
Ease of Following the Directions ment anymore since the middle of the boring and hard for me.” Participant 3
Four participants suggested that ease of intervention period since I felt pain gave the researcher a “thumbs-up,” say-
imitating or following the training whenever I raised the UE like this (above ing, “I have never encountered these
affected their enjoyment and difficulty. 90° of the affected side of shoulder).” interesting games before; actually, I have
Participants 2 and 4 said that the work- never played games before.”
out training mode was more interesting Scores Achieved
because it enabled them to “imitate the Some participants (n⫽3) were very sen- Self-perceived Effectiveness
movements of the virtual trainer.” In par- sitive about the scores achieved and Two participants said that the workout
ticular, participant 2 implied the neces- became depressed and marked a low mode of training seemed to be more
sity of the virtual trainer by saying, level of enjoyment when they got disap- effective, so they preferred it, although
“Where is my trainer (whose movements pointing scores. Participant 4 men- the game mode was “not bad.” Partici-
I should follow)?” at every session, even tioned, “When I first saw the game, it did pant 6 said, “I could not extend my

1560 f Physical Therapy Volume 96 Number 10 October 2016


Patient-Friendly Strategies for Virtual Reality–Based Rehabilitation and Acute Stroke

Table 2. and 10% of the heart rate reserve of each


Mean Heart Rates Depending on the Mode of Traininga participant.
MHR in Workout MHR in Game
Mode of Training Mode of Training 10% of HRR Physical Outcomes
Participant No. RHR (bpm) (bpm) (bpm) (bpm)b Participants showed increases in UE
1 90⫾5 (83–98) 96⫾7 (85–104) 97⫾5 (87–103) 95 function and ADL performance (Tab. 3).
According to the UE-FMA, there were
2 96⫾3 (92–100) 100⫾4 (93–104) 99⫾3 (93–102) 101
significant increases in total score
3 77⫾15 (60–100) 81⫾14 (63–102) 86⫾16 (63–107) 84 (P⫽.018, d⫽.50) and shoulder, arm, and
4 92⫾3 (88–96) 95⫾5 (88–103) 97⫾7 (88–104) 102 elbow score (P⫽.018, d⫽.57), but not in
5 91⫾3 (86–95) 96⫾10 (89–112) 95⫾6 (90–105) 96
wrist score (P⫽.109, d⫽.24), hand score
(P⫽.102, d⫽.54), or coordination score
6 102⫾4 (97–108) 104⫾3 (101–108) 106⫾3 (103–110) 109
(P⫽.180, d⫽.31). According to the MFT,
7 90⫾6 (82–99) 98⫾6 (93–109) 99⫾8 (91–112) 94 there were significant increases in all UE
8 86⫾5 (81–92) 96⫾4 (91–99) 91 parts, including total score (P⫽.018,
a
d⫽.69), shoulder, elbow, and forearm
All indicators are presented as mean⫾standard deviation (range of values). MHR⫽median heart rate,
RHR⫽resting heart rate, HRR⫽heart rate reserve from the resting heart rate. score (P⫽.017, d⫽.59), and hand score
b
10% HRR according to the Karvonen formula. (P⫽.026, d⫽.68). Participants also dem-
onstrated significant increases in BBT
score (P⫽.043, d⫽.52) and MBI score
elbow at all when I first started the pant varied from 81 to 104 bpm in the (P⫽.012, d⫽.96).
(workout mode of) training; however, workout mode of training and from 86 to
yesterday, I found that my elbow fully 106 bpm in the game mode of training, Discussion
extended when I did that movement although we could not take the MHR of In this study, we explored patient-
lying down on the bed.” participant 8 in the game mode of train- perceived difficulty and enjoyment dur-
ing due to her complaint of fatigue. ing VR-based rehabilitation in the theo-
Training Intensity These values were approximately 10% of retical framework based on Flow Theory
The median RPE (interquartile range) of the heart rate reserve from the RHR of and the factors affecting those experi-
each participant was as follows: partici- each participant, according to the Kar- ences. As a result, we found that the level
pant 1, 3.00 (2.00 –3.00); participant 2, vonen formula,38 indicating that the of enjoyment could be an indicator of the
3.00 (2.00 – 4.00); participant 3, 1.50 VR-based rehabilitation had a very low appropriateness of the level of difficulty
(1.00 –2.25); and participant 4, 3.00 intensity. Table 2 shows the RHR, MHR, for participants with acute stroke, as
(1.25–3.00). The MHR of each partici- Flow Theory hypothesizes, because

Table 3.
Upper Extremity Function and Activities of Daily Living Performance Before and After the Interventiona

Change in
Measure Before Intervention After Intervention Score Pb Effect Sizec
FMA

Total (0–66) 16.00 (11.50–56.75) 43.00 (24.00–58.25) 27.00 .02d 0.50

Shoulder/elbow/forearm (0–36) 11.50 (6.75–29.75) 24.00 (15.50–32.25) 12.50 .02d 0.57

Wrist (0–10) 0.00 (0.00–8.00) 3.50 (0.00–8.25) 3.50 .11 0.24

Hand (0–14) 3.00 (0.25–13.50) 11.50 (2.50–13.50) 8.50 .10 0.54

Coordination (0–6) 3.00 (0.00–4.00) 4.00 (0.75–4.00) 1.00 .18 0.31

MFT

Total (0–32) 5.50 (4.00–20.25) 19.50 (8.75–24.00) 14.00 .02d 0.69

Shoulder/elbow/forearm (0–16) 5.50 (4.00–13.75) 11.50 (8.00–15.50) 6.00 .02d 0.59


d
Hand (0–16) 0.00 (0.00–6.50) 5.50 (0.75–11.75) 5.50 .03 0.68

BBT (no. of blocks) 0.00 (0.00–13.25) 4.50 (0.00–25.25) 4.50 .04d 0.52
d
MBI 29.00 (9.75–57.50) 56.50 (38.50–86.75) 27.50 .01 0.96
a
All indicators are presented as median (interquartile range). BBT⫽Box and Block Test, FMA⫽Fugl-Meyer Assessment, MBI⫽Modified Barthel Index,
MFT⫽Manual Function Test.
b
According to the Wilcoxon signed rank test.
c
Calculated using the Cohen d effect size equation.
d
P⬍.05.

October 2016 Volume 96 Number 10 Physical Therapy f 1561


Patient-Friendly Strategies for Virtual Reality–Based Rehabilitation and Acute Stroke

there was a reciprocal relationship ficulty, they tended to perceive a low sized ease of following the directions,
between levels of difficulty and enjoy- level of enjoyment at the same time, and some participants mentioned that the
ment, meaning that enjoyment occurred vice versa, indicating that the level of workout mode was easier than the game
when the difficulties of the task and the enjoyment could be indicator of appro- mode because they could “imitate the
abilities of the participant were priateness of the level of difficulty, as movements of the virtual trainer” rather
balanced. Flow Theory hypothesizes. A similar than judging the situation for them-
trend also was identified by a previous selves, even though the training intensity
Participants showed very different levels study of VR-based exercise in healthy was similar between the 2 modes, requir-
of difficulty and enjoyment during young adults, suggesting that partici- ing only approximately 10% of the heart
VR-based rehabilitation. That is, we pants rated the level of enjoyment high rate reserve. This situation occurred not
found that some participants seemed to when they perceived the difficulty to be only in participants with severe cogni-
easily adjust to VR-based rehabilitation, in the lower middle level.18 This result tive impairment, but also in those with
showing steady high or middle levels of implies that the participants tended to mild-to-moderate impairment. This result
enjoyment; some participants showed experience a high level of enjoyment implies that the range of patients who
many changes as time went on; and when they perceived that the task was are able to participate in VR-based reha-
other participants could not understand sufficiently easy to perform. For exam- bilitation in the early stage would differ
the rules of the training until completion ple, for participant 4, the level of enjoy- depending on the training mode and the
of the intervention period. According to ment sharply dropped immediately after method for implementing it. However,
Flow Theory, this finding can be inter- he changed the training side from the most previous studies about VR-based
preted to mean that enjoyment occurred unaffected UE to the bilateral UEs in the rehabilitation in patients with acute
when the difficulties of the task and the game mode of training; however, this stroke have adopted the game mode of
abilities of the participant were bal- recovered again to the initial level follow- training.11–14 Applying the workout
anced, and for some participants, ing the sixth session. This very fluid sit- mode of training in the very early stage
depending on their phase of recovery uation might be due to the young age of would be better to accommodate a wide
and other conditions, this balance was the participant (32 years) and can be range of patients with various levels of
not achieved in a certain part of or to the explained by his comments in the semi- stroke severity because the human-like
end of the intervention period.17 There- structured interview: “When I first saw virtual trainer is easy to follow.
fore, it might be necessary to use VR the game, it did not seem very difficult to
content with varied levels of difficulty to me, but when I actually tried it, the Second, experience of pain directly
accommodate a wide range of patients scores were disappointing. This made affected participants’ difficulty and
with various levels of stroke severity and me depressed, and I came to prefer the enjoyment. That is, experiences of pain
in different phases of recovery. workout mode. In the later sessions, as made some participants view the
the range of motion of my shoulder VR-based rehabilitation as something to
There were complex determinant factors increased, the scores also increased, so I avoid or fear. One participant was even
for such different levels of difficulty and again enjoyed the game.” This finding absent from training due to her pain
enjoyment, including differences in might verify the hypothesis of the Flow experience. Muscle soreness during
physical, cognitive, and proprioceptive Zone and might be a step forward from VR-based rehabilitation also emerged as a
abilities of the participants and their indi- previous studies15,16 that qualitatively main theme related to the feasibility of
vidual experiences. Cognitive ability explored patient experiences. In other training in patients with chronic stroke
alone did not appear to be the determin- words, we identified a reciprocal rela- in a previous study.39 Patients with acute
ing factor for such differences. For exam- tionship between levels of difficulty and stroke more commonly have motor
ple, some participants with normal cog- enjoyment in the patients with stroke weakness compared with patients with
nitive ability could not understand the while they performed VR-based rehabili- chronic stroke due to their drastically
rules of the game mode of training until tation, as it was assumed by Flow Zone, decreased activity since the recent onset
completion of the intervention period, and this finding might provide health of stroke. Therefore, it is certainly nec-
whereas other participants with mild professionals with clinically useful infor- essary to find a way for patients to avoid
cognitive impairment could enjoy the mation in determining the appropriate pain so they do not lose interest in the
game with only minimal assistance. level of difficulty of VR-based rehabilita- VR-based rehabilitation.
Therefore, it would not be appropriate tion, by reference to the level of
for health professionals or researchers to enjoyment. Third, feedback during training, includ-
assume the inapplicability of VR-based ing both KR and KP, affected patient-
rehabilitation beforehand only due to the Five main factors were identified as perceived difficulty and enjoyment, but
low cognitive ability. affecting patient-perceived difficulty and in opposite ways. That is, scores
enjoyment: ease of following the direc- achieved (KR) made some participants
We also found that there tended to be a tions, experience of pain, scores mark a low level of enjoyment, whereas
reciprocal relationship between levels of achieved, novelty and immediate feed- novelty and multisensory feedback in the
difficulty and enjoyment. That is, when back, and self-perceived effectiveness. process of training (KP) had a positive
participants perceived a high level of dif- First, for those participants who empha- effect on participants’ enjoyment. With

1562 f Physical Therapy Volume 96 Number 10 October 2016


Patient-Friendly Strategies for Virtual Reality–Based Rehabilitation and Acute Stroke

regard to the effect of KR, we can age and severity of stroke; therefore, the Professor Yoon provided project manage-
assume that participants tended to have a results cannot be generalized. Another ment. Dr Eun provided fund procurement.
lack of confidence due to their impair- limitation is that the physical improve- Professor Pyun provided participants, facili-
ments, which might make them sensitive ments could have been due to spontane- ties/equipment, and institutional liaisons. Dr
to their scores. Thus, it would be better ous recovery because participants were Chung provided consultation (including
review of manuscript before submission).
not to discourage patients by showing or within 2 months poststroke and conven-
The authors thank the physical therapists of
providing low scores. Not presenting the tional therapy was continued as usual Korea University Anam Hospital for their con-
numerical scores to the patient would be during the intervention period. Thus, fur- tribution to the design of software for this
one alternative. However, more funda- ther study using a control group is study. They also thank all participants of this
mentally, it would be better to progress needed to establish the causal effect of study.
the level of difficulty of the training con- VR on improved performance. Despite
Approval for this study was obtained from
sidering the process of the patient’s these limitations, to our knowledge, this
the Institutional Review Board of Korea Uni-
recovery in order to balance the diffi- is one of the first studies analyzing the versity Anam Hospital (ED13049).
culty of the task and the individual’s difficulties and enjoyments of patients
capability level,17 meaning that software with acute stroke in the theoretical This research was supported by R&D grant
should be developed to support the framework based on Flow Theory while (No. 2013006) on rehabilitation by Korea
patient being in the Flow Zone during they participated in VR-based rehabilita- National Rehabilitation Center Research
Institute, Ministry of Health & Welfare.
VR-based rehabilitation. Meanwhile, par- tion. Through this study, we suggested
ticipants who understood the rules of implementation strategies, which should Clinical trial registration: ISRCTN04144761
the game mode of training commonly be applied to VR-based rehabilitation for
DOI: 10.2522/ptj.20150271
reported immediate feedback in the pro- patients with acute stroke, including
cess of training as a main factor affecting those with severe physical and cognitive
their enjoyment. Therefore, we suggest impairments. References
applying the game mode of training with 1 Cooke EV, Mares K, Clark A, et al. The
multisensory feedback only in patients In conclusion, the level of enjoyment effects of increased dose of exercise-based
therapies to enhance motor recovery after
who understand the rules of the game in could be an indicator of the appropriate- stroke: a systematic review and meta-anal-
order to maintain their enjoyment. ness of the level of difficulty because ysis. BMC Med. 2010;8:60.
enjoyment occurred when the difficul- 2 French B, Thomas L, Leathley M, et al.
Fourth, participants’ self-perceived effec- ties of the task and the abilities of the Does repetitive task training improve func-
tional activity after stroke? A Cochrane sys-
tiveness affected their positive attitude participant were balanced. Therefore, tematic review and meta-analysis. J Reha-
toward training. The effect of the soft- we suggest an implementation strategy bil Med. 2010;42:9 –14.
ware program applied in the current for VR-based rehabilitation that supports 3 Kwakkel G, Van Peppen R, Wagenaar RC,
study on physical outcomes might be the patient by achieving balance et al. Effects of augmented exercise ther-
verified indirectly by comparison with between the difficulties of the task and apy time after stroke: a meta-analysis.
Stroke. 2004;35:2529 –2536.
the results of previous studies. For exam- the abilities of the patient, by reference
4 Bernhardt J, Dewey H, Thrift A, Donnan G.
ple, the increase of UE function mea- to the level of enjoyment. For overcom- Inactive and alone: physical activity within
sured by UE-FMA in the current study ing patient-perceived difficulty and the first 14 days of acute stroke unit care.
(169%) was higher than that of previous enhancing enjoyment, ease of following Stroke. 2004;35:1005–1009.
studies (64%–70%)11,12 that applied the directions would be best considered 5 Jutai JW, Teasell RW. The necessity and
VR-based rehabilitation in addition to in the very early stage, whereas multisen- limitations of evidence-based practice in
stroke rehabilitation. Top Stroke Rehabil.
conventional therapy; however, the sory feedback is more necessary in the 2003;10:71–78.
baseline UE function in the current study later phase for those patients who under- 6 Teasell R, Meyer MJ, McClure A, et al.
was significantly lower than in the pre- stand the rules of the game. Health pro- Stroke rehabilitation: an international per-
vious studies. Furthermore, the improve- fessionals should find a way for patients spective. Top Stroke Rehabil. 2009;16:
44 –56.
ment was great enough so that partici- to avoid pain during training. Knowledge
pants could perceive it themselves, of results should be presented to patients 7 Kleim JA, Jones TA. Principles of
experience-dependent neural plasticity:
resulting in a positive attitude toward in a positive manner. A training program implications for rehabilitation after brain
training. Therefore, it would be neces- sufficiently effective in improving physi- damage. J Speech Lang Hear Res. 2008;
sary to apply VR content to a training cal outcomes needs to be embedded in 51:S225–S239.
program adequately effective at improv- the VR contents in order to induce 8 Barreca S, Wolf SL, Fasoli S, Bohannon R.
Treatment interventions for the paretic
ing physical outcomes in order to induce patients to continue training with a pos- upper limb of stroke survivors: a critical
patients to continue training with a pos- itive attitude. review. Neurorehabil Neural Repair.
itive attitude. 2003;17:220 –226.
9 Sherry JL. Flow and media enjoyment.
Ms Lee, Dr Chung, and Mr Kim provided Commun Theory. 2004;14:328 –347.
Limitations, Strengths, and concept/idea/research design. Mr Lee pro-
Future Directions 10 Kim J, Son J, Ko N, Yoon B. Unsupervised
vided writing. Professor Pyun and Mr Kim virtual reality-based exercise program
This study explored the experiences of a provided data collection. Ms Lee, Dr Eun, improves hip muscle strength and balance
small sample of participants with varied and Professor Yoon provided data analysis. control in older adults: a pilot study. Arch
Phys Med Rehabil. 2013;94:937–943.

October 2016 Volume 96 Number 10 Physical Therapy f 1563


Patient-Friendly Strategies for Virtual Reality–Based Rehabilitation and Acute Stroke

11 da Silva Cameirão M, Bermúdez I Badia S, 20 Mortenson WB, Oliffe JL. Mixed methods 30 Borg G, Linderholm H. Perceived exertion
Duarte E, Verschure PF. Virtual reality research in occupational therapy: a survey and pulse rate during graded exercise in
based rehabilitation speeds up functional and critique. OTJR. 2009;29:14 –23. various age groups. Acta Med Scand.
recovery of the upper extremities after 1967;181:194 –206.
stroke: a randomized controlled pilot 21 Voss DE, Ionta MK, Myers BJ. Propriocep-
study in the acute phase of stroke using tive Neuromuscular Facilitation: Pat- 31 Gladstone DJ, Danells CJ, Black SE. The
the Rehabilitation Gaming System. Restor terns and Techniques. 3rd ed. New York, Fugl-Meyer assessment of motor recovery
Neurol Neurosci. 2011;29:287–298. NY: Lippincott Williams & Wilkins; 1985. after stroke: a critical review of its mea-
surement properties. Neurorehabil Neu-
12 Yin CW, Sien NY, Ying LA, et al. Virtual 22 Dickstein R, Hocherman S, Pillar T, Sha- ral Repair. 2002;16:232–240.
reality for upper extremity rehabilitation ham R. Stroke rehabilitation: three exer-
in early stroke: a pilot randomized con- cise therapy approaches. Phys Ther. 1986; 32 Miyamoto S, Hondo T, Suzukamo Y, et al.
trolled trial. Clin Rehabil. 2014;28:1107– 66:1233–1238. Reliability and validity of the Manual Func-
tion Test in patients with stroke. Am J
1114. 23 Ribeiro TS, de Sousa e Silva EM, Sousa Silva Phys Med Rehabil. 2009;88:247–255.
13 Saposnik G, Teasell R, Mamdani M, et al. WH, et al. Effects of a training program
Effectiveness of virtual reality using Wii based on the proprioceptive neuromuscu- 33 Desrosiers J, Bravo G, Hébert R, et al. Val-
gaming technology in stroke rehabilita- lar facilitation method on post-stroke idation of the Box and Block Test as a
tion: a pilot randomized clinical trial and motor recovery: a preliminary study. J measure of dexterity of elderly people:
proof of principle. Stroke. 2010;41:1477– Bodyw Mov Ther 2014;18:526 –532. reliability, validity, and norms studies.
Arch Phys Med Rehabil. 1994;75:751–
1484. 24 Waller SM, Whitall J. Bilateral arm training: 755.
14 Kwon JS, Park MJ, Yoon IJ, Park SH. why and who benefits? NeuroRehabilita-
Effects of virtual reality on upper extrem- tion. 2008;23:29 – 41. 34 Wade DT, Collin C. The Barthel ADL
Index: a standard measure of physical dis-
ity function and activities of daily living 25 Stoykov ME, Lewis GN, Corcos DM. Com-
performance in acute stroke: a double- ability? Int Disabil Stud. 1988;10:64 – 67.
parison of bilateral and unilateral training
blind randomized clinical trial. NeuroRe- for upper extremity hemiparesis in stroke. 35 Patton MQ. Qualitative Evaluation and
habilitation. 2012;31:379 –385. Neurorehabil Neural Repair. 2009;23: Research Methods. 2nd ed. Thousand
15 Celinder D, Peoples H. Stroke patients’ 945–953. Oaks, CA: Sage; 1990.
experiences with Wii Sports® during inpa- 26 Dewey HM, Donnan GA, Freeman EJ, et al. 36 Hsieh HF, Shannon SE. Three approaches
tient rehabilitation. Scand J Occup Ther. Interrater reliability of the National Insti- to qualitative content analysis. Qual
2012;19:457– 463. tutes of Health Stroke Scale: rating by neu- Health Res. 2005;15:1277–1288.
rologists and nurses in a community-based
16 Worthen-Chaudhari L, Whalen CN, Swen- 37 Cohen J. Statistical Power Analysis for
stroke incidence study. Cerebrovasc Dis.
dal C, et al. A feasibility study using inter- the Behavioral Sciences. 2nd ed. Hillsdale,
1999;9:323–327.
active graphic art feedback to augment NJ: Lawrence Earlbaum Associates; 1988.
acute neurorehabilitation therapy. Neu- 27 Folstein MF, Folstein SE, McHugh PR. 38 Pescatello L, Arena R, Riebe D, Thompson
roRehabilitation. 2013;33:481– 490. “Mini-mental state”: a practical method for P. ACSM’s Guidelines for Exercise Testing
grading the cognitive state of patients for
17 Csikszentmihalyi M. The Psychology of and Prescription. 9th ed. Philadelphia,
the clinician. J Psych Res. 1975;12:189 –
Optimal Experience. New York, NY: PA: Wolters Kluwer; 2014.
198.
Harper & Row; 1990. 39 Merlo AR, Goodman A, McClenaghan BA,
28 Ahearn EP. The use of visual analog scales
18 Thin AG, Hansen L, McEachen D. Flow Fritz SL. Participants’ perspectives on the
in mood disorders: a critical review.
experience and mood states while playing feasibility of a novel, intensive, task-
J Psych Res. 1997;31:569 –579.
body movement-controlled video games. specific intervention for individuals with
Games Culture. 2011;6:414 – 428. 29 Marshall C, Rossman GB. Designing Qual- chronic stroke: a qualitative analysis. Phys
itative Research. 4th ed. Thousand Oaks, Ther. 2013;93:147–157.
19 Greene JC, Caracelli VJ, Graham WF. CA: Sage; 2006.
Toward a conceptual framework for
mixed-method evaluation designs. Educ
Eval Policy Anal. 1989;11:255–274.

1564 f Physical Therapy Volume 96 Number 10 October 2016


Copyright of Physical Therapy is the property of American Physical Therapy Association and
its content may not be copied or emailed to multiple sites or posted to a listserv without the
copyright holder's express written permission. However, users may print, download, or email
articles for individual use.

You might also like