Professional Documents
Culture Documents
com
Review
ABSTRACT
Background Mobile technologies are steadily replacing
traditional assessment approaches for the recognition
and assessment of a sports concussion. Their ease of
access, while facilitating the early identication of a
concussion, also raises issues regarding the content of
the applications (apps) and their suitability for different
user groups.
Aim To locate and review apps that assist in the
recognition and assessment of a sports concussion and
to assess their content with respect to that of
internationally accepted best-practice instruments.
Methods A search of international app stores and of
the web using key terms such as concussion, sports
concussion and variants was conducted. For those apps
meeting the inclusion criteria, data were extracted on the
platform, intended users and price. The content of each
app was benchmarked to the Sport Concussion
Assessment Tool 2 (SCAT2) and Pocket SCAT2 using a
custom scoring scheme to generate a percentage
compliance statistic.
Results 18 of the 155 apps identied met the
inclusion criteria. Almost all (16/18) were available on
an iOS platform and only ve required a payment to
purchase. The apps were marketed for a wide range of
intended users from medical professionals to the general
public. The content of the apps varied from 0% to
100% compliance with the selected standard, and
symptom evaluation components demonstrated the
highest level of compliance.
Conclusions The surge in availability of apps in an
unregulated market raises concerns as to the
appropriateness of their content for different groups of
end users. The consolidation of best-practice concussion
instruments now provides a framework to inform the
development of future apps.
INTRODUCTION
1 of 8
Review
free iPhone and iPad app with their assessment tools provided
in an electronic format, while others have developed a range of
independent apps (eg, Hockey Canada Concussion
Awareness20). Collectively, the use of these apps represent an
important opportunity in translating21 this essential information
not only to sports medicine professionals but also to coaches,
teachers, team managers and the parents of young athletes. The
empowering of these individuals through the availability of
concussion-related apps is a valuable step in improving the early
diagnosis of a possible concussion, particularly at the community sports level where trained professionals may not be present.
Paramount is the necessity for these apps to deliver current bestpractice information and assessment procedures, contained in
foundation documents such as the recent CIS consensus statements.18 22
With the increased availability of concussion-related apps in
the consumer marketplace, it is timely to review these, in order
to provide the consumer and health professional with comparative information to inform their choice of app and whether the
content of the app is in keeping with current international bestpractice concussion management strategies. Thus, the purpose
of this review was to identify and appraise smartphone/tablet
apps related to the recognition and assessment of a sports
concussion.
METHODS
The key elements of a traditional systematic review methodology were used to locate all available apps relating to the
assessment of a sports concussion at the time of the search (10
April 2013). In addition, a systematic approach was taken to the
extraction of data and appraisal of content to reduce reviewer
bias. Although the apps were all evaluated with respect to their
content, no personal data were collected as part of this review.
Thus, ethical approval from a recognised ethics committee was
not sought.
app stores. The search strategy and selection process are presented in gure 1. The apps identied by the individual searches
were pooled and duplicates removed. Where the same app was
available on different platforms (iOS or Android), only one
version of the app was retained for analysis, with the iOS platform being preferred. Those apps meeting the inclusion criteria
were downloaded onto an Apple iPad2 (iOS 6) if they were
available through the Apple iTunes store, and onto a Samsung
Galaxy SII smartphone device (Android V.2.3.3) if available
through Google play for further exploration and analysis.
Downloading to the Apple iPad allowed the maximum exibility
in reviewing the apps.
Data extraction
A wide range of data about the apps was extracted from the
smartphone app stores description and by reviewing the apps in
operation. This information included: the purpose of the app,
the date of release or update, pricing, scientic underpinnings,
platform availability, intended user and whether it contained
social networking site/email sharing capabilities.
The data were extracted in a systematic manner by the principal investigator (HL) and veried by at least one other member
of the research team prior to entry into an Excel spreadsheet
(Microsoft 2010, Redmond, Washington, USA).
The content of each app was reviewed and appraised with reference to the content of either the Sport Concussion Assessment
Tool 2 (SCAT2)23 or the Pocket SCAT224 depending on the
intended user. Those apps which indicated in their description
or within the product that they were for use by medical personnel were reviewed with reference to SCAT2, which was considered as the gold standard in concussion recognition,
assessment and management at the time the apps were developed. This tool has been systematically developed by an international group of experts (the CIS group) and widely
disseminated internationally, cited in the research literature25
and adopted by major sporting organisations (eg, FIFA, IOC,
IIHF, AFL and the IRB) and modifed and adopted by others
(NFL, NHL and MLB). It was developed for use with athletes
10 years of age and over and for use by medical personnel or
persons working under the supervision of such. As no formal
appraisal tool was available, we developed a checklist and decision rule using an approach similar to that in a previous study
evaluating the content of concussion websites26 and to that used
in the content appraisal of other health-related apps.8 9 This
checklist considered whether the seven key elements (eg,
Symptoms, Physical Signs, Balance Assessment, etc) contained in the SCAT2 were included in the app. The checklist
also contained items relating to accuracy of measurement (eg, if
the symptoms were assessed on a seven-point Likert scale) and
whether the SCAT2 Score (overall score) was generated, to
provide a total of 10 items of interest. The content of each app
was initially examined with reference to the checklist by the
principal investigator (HL) and where there was doubt in the
scoring, this was veried by at least one member of the research
team who discussed the scoring before arriving at a joint decision. The presence of items was scored using a three-point categorical scoring system where 2=present and complete,
1=present but incomplete and 0=not included/not specied.
The score of the 10 items was summed (maximum score=20)
and presented as a SCAT2 compliance score (%) as an indication
of how closely the contents followed that of SCAT2.
2 of 8
Inclusion criteria
Apps were considered for inclusion in the study if they were
produced in English; designed and/or marketed to be used in
the recognition and assessment of concussion; available to the
sports medicine professional and/or the general public and
were a self-contained product. Apps were excluded if their
primary function was unrelated to concussion assessment (eg,
games and brain training apps), if they were essentially
scoring apps for particular concussion tests, or if they were
designed purely for the dissemination of information on
concussion.
Search strategy
Review
Figure 1 Search ow diagram.
3 of 8
RESULTS
Review
Table 1 App descriptors
App titleversion
Price
(US$)
Platform
(s)
Cognit1.0.0
04-07-2009
Free
iOS
Concussion1.2.0
10-05-2011
iOS
18-03-2013
US
$0.99*
US$4.99
03-01-2012
US$0.99
20-07-2012
Free
15-03-2013
Free
Concussion22.0.0
EmergencyConsult1.0.0
01-12-2012
19-03-2013
Free
Free
First Responder1.0.0
12-03-2013
Free
07-07-2012
Free
11-09-2012
US
$44.99
17-11-2011
Free
iOS
03-08-2011
Free
iOS
ReturnToPlay1.0.0
19-03-2013
Free
Return2Play1.3.0
09-04-2013
Free
SACTool Beta1.0.0
SCAT21.0.0
SCAT2Sport Concussion
Assessment Tool1.3.0
19-08-2011
28-04-2011
16-02-2012
Free
US$3.99
Free
iOS
Android
iOS
Android
Android
iOS
iOS
iOS
Android
iOS
Android
Android
iOS
Android
iOS
iOS
Android
iOS
Android
iOS
Android
iOS
Sharing
capabilities
Reference to a
standard or
guidelines
Link to a
central
database
NS
NS
Parents and
coaches
NS
NA
NS
NS
General
NA
NS
NS
Coaches
Medical
Email
Email
NS
NS
NS
NS
Coaches, parents
and teachers
General
NS
NS
NS
King-Devick
online system
Email
Email
NS
NA
NS
Intended user
Medical, AT,
coaches and
parents
HCPs, coaches and
parents
Coaches and
parents
Medical and
healthcare team
Patients and HCPs
NS
NS
NS
AT
Medical and HCPs
Medical and HCPs
NA
Email
Email
Y
Y
Y
NS
NS
Y
4 of 8
DISCUSSION
Review
Table 2 Apps benchmarked to SCAT2 content
Symptoms
App titleversion
Concussion1.2.0
Concussion Assessment &
Response: Sport Version
(CARE)1.1.0
Emergency Consult1.0.0
King-Devick Concussion
Screening Test2.3.0
ReturnToPlay1.0.0
SCAT21.0.0
SCAT2Sport Concussion
Assessment Tool1.3.0
Physical signs
Sideline
22
Seven-point
items Likert scale
Worse
with
physical/
mental
activity
LOC/
unresponsiveness
Balance/
Maddocks
unsteadiness
GCS score
2
2
2
2
2
2
2
1
2
0
1
0
2
0
0
0
2
0
1
2
2
2
2
2
0
2
2
0
2
2
Cognitive Balance
Coordination
SAC
Modified
BESS
FTN
SCAT2 SCAT2
total compliance
score score (%)
2
0
2
2
2
2
2
1
2
0
100
60
0
0
2
0
2
0
0
0
2
0
0
0
55
0
0
2
2
0
2
2
2
2
2
0
2
2
2
2
2
0
2
2
35
100
100
2, present and complete; 1, present but incomplete; 0, not included/not specified; apps, applications; BESS, Balance Error Scoring System; FTN, finger to nose; GCS, Glasgow Coma
Scale; LOC, loss of consciousness; SAC, Standardized Assessment of Concussion; SCAT2, Sport Concussion Assessment Tool 2.
Table 3
As some apps were clearly designed to be used by nonmedical personnel such as coaches or parents, these were referenced to the Pocket SCAT2 (the tool designed for this purpose).
As might be expected, many of the apps met these minimum
expected criteria for the recognition of a concussion and hence
demonstrated a high level of compliance to the core information
contained in the Pocket SCAT2; however, there was considerable variation in the content of these apps. In many instances,
this subset of apps is more important as it is directed at users
who may not be expected to have a sophisticated knowledge of
concussion or medical training, and may rely on the app to
provide prompts and an early indication of whether a player
might be concussed and thus in need of a medical consultation.
The public availability of apps allows anybody to download and
App titleversion
Symptoms
22 items
Memory function
Maddocks score
Balance testing
Modified BESS
Pocket SCAT2
compliance score (%)
Cognit1.0.0
Concussion1.2.0
Concussion Assessment & Response: Sport Version (CARE)1.1.0
Concussion Recognition & Response: Coach & Parent Version1.2.2
Concussion Signs and Symptoms2.3.0
Concussion Quick Check1.0.1
Concussion22.0.0
EmergencyConsult1.0.0
First Responder1.0.0
ImPACT Concussion Awareness Tool (ImCAT)1.1.0
King-Devick Concussion Screening Test2.3.0
PLAY IT SAFE Concussion Assessment (USA only)1.3.0
Pocket SCAT21.0.0
ReturnToPlay1.0.0
Return2Play1.3.0
SACTool Beta1.0.0
SCAT21.0.0
SCAT2Sport Concussion Assessment Tool1.3.0
1
2
2
1
1
1
1
1
1
1
0
1
2
1
1
0
2
2
0
2
0
0
0
1
2
2
2
0
0
0
2
0
0
0
2
2
0
1
2
0
0
0
1
0
0
0
0
1
1
0
0
0
2
2
17
83
67
17
17
33
67
50
50
17
0
33
83
17
17
0
100
100
2, present and complete; 1, present but incomplete; 0, not included/not specified; apps, applications; BESS, Balance Error Scoring System; SCAT2, Sport Concussion Assessment Tool 2.
5 of 8
Review
use an app. While this may be seen as widening the opportunity
to identify and manage a greater number of concussions, it also
provides the potential for less qualied individuals to use such
apps inappropriately. Fundamental to this point of availability is
whether there is an actual need for an app. While it can be
argued that they may represent nothing more than a hi-tech
way of presenting and managing information, they offer a range
of functions beyond this. The use of mobile technologies is now
becoming an integral component of everyday lifestyle, and thus
the availability of concussion apps promotes the global access to
information and facilitates documentation. Although not essential to the recognition and management of a concussion, mobile
apps provide a new dimension in concussion management.
Currently, not all health-related apps are formally regulated by
the FDA or any other agency, although this is likely to change in
the coming years.39 The purpose of such regulation is to provide
the consumer with the condence that the product can be used
safely by the intended user. In the interim, the onus is on the
developers of apps to provide the consumer with a welldocumented product, and a clear indication as to the intended
user group of the app. Of the apps identied in this study, it was
sometimes difcult to ascertain who the app was designed to be
used by; thus, these apps were evaluated both against SCAT223
and the Pocket SCAT2.24 Developers of future apps need to
ensure that their product is based on a current gold standard information (such as the SCAT3,30 Pocket CRT32 and the Child
SCAT331 assessment tools18); in addition, they need to be clearly
targeted at the appropriate user group (eg, medical doctors; athletic trainers, physiotherapists or the general public).
A feature of many of the apps identied in this study was the
ability to transmit information, either via email or to a host
database for research purposes. The ability to transmit this
information via email to a doctor or a medical clinic is a useful
feature; however, its usefulness is predicated on the recipient of
this information having the necessary background to understand
and interpret the information transmitted. A number of apps
provide the option of sending the individuals data to a database
for subsequent research purposes. This is of value to the database hosts and to the wider research community, although the
Table 4
Characteristics
Questions to ask
Intended use
Authority evidence
Commercialisation
Users comments
Producers support
Additional information and
resources
Data management
Research
Price
App, application.
6 of 8
Review
What are the new ndings?
This study provides a comparative review of mobile and
tablet applications (apps) available to be downloaded and
used in the recognition and assessment of a sports-related
concussion.
There was considerable diversity in the content of the apps
indicating the need for consideration as to the most
appropriate app for the designated purpose.
This review provides the developers of new apps, or those
upgrading current apps, with key information to include in
their products.
A checklist is provided to assist consumers in making a
choice as to the most appropriate app for their needs.
REFERENCES
1
CONCLUSION
The study provides an initial systematic review of the apps
developed to recognise and assess a sports-related concussion.
The ease of availability of these apps raises issues of how they
might be used and by whom. It is important that the sports
medicine practitioners and the public are assured that the
content of the apps represents the best-practice information and
is presented in an appropriate manner for the target audience.
As mobile technologies become an everyday part of our lives,
apps such as those reviewed here have a major role to play in
safeguarding the health of sports persons via the early recognition of a possible concussion.
Contributors OHA, HL, PM and SJS designed the study and prepared the
preliminary draft. HL, APB, AGS and SJS conducted the search and data extraction.
HL, SJS, PM and WHM developed the analysis strategy and criteria. All authors
contributed to the analysis, drafting and approved the nal version of the
manuscript.
Competing interests PM is a coinvestigator, collaborator or consultant on grants
relating to mild TBI funded by several governmental organisations. He is Co-Chair of
the Australian Centre for Research into Sports Injury and its Prevention (ACRISP),
which is one of the International Research Centres for Prevention of Injury and
Protection of Athlete Health supported by the International Olympic Committee
(IOC). He has a clinical and consulting practice in general and sports neurology. He
receives book royalties from McGraw-Hill and was employed in an editorial capacity
by the British Medical Journal Publishing Group from 2001 to 2008. He has been
reimbursed by the government, professional scientic bodies and sporting bodies for
presenting research relating to mild TBI and sports-related concussion at meetings,
scientic conferences and symposiums. He received consultancy fees in 2010 from
Axon Sports (USA) for the development of educational material (which was not
renewed) and has received support since 2001 from CogState Inc for research costs
Lee H, et al. Br J Sports Med 2015;49:499505. doi:10.1136/bjsports-2013-092930
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
7 of 8
Review
25
26
27
28
29
30
31
32
Alla S, Sullivan SJ, McCrory P, et al. Spreading the word on sports concussion:
citation analysis of summary and agreement, position and consensus statements on
sports concussion. Br J Sports Med 2011;45:1325.
Ahmed OH, Sullivan SJ, Schneiders AG, et al. Concussion information online:
evaluation of information quality, content and readability of concussion-related
websites. Br J Sports Med 2012;46:67583.
King-Devick Test LLC. King-Devick Concussion Screening Test-iTunes App Store
2013. [cited 14 May 2013]. https://itunes.apple.com/us/app/king-devickconcussion-screening/id424793542?mt=8
Aubry M, Cantu R, Dvorak J, et al. Summary and agreement statement of the rst
International Conference on Concussion in Sport, Vienna 2001. Br J Sports Med
2002;36:610.
McCrory P, Johnston K, Meeuwisse W, et al. Summary and agreement statement of
the 2nd International Conference on Concussion in Sport, Prague 2004. Br J Sports
Med 2005;39:196204.
SCAT3. Br J Sports Med 2013;47:25962.
ChildSCAT3. Br J Sports Med 2013;47:2636.
Pocket CRT. Br J Sports Med 2013;47:267.
8 of 8
33
34
35
36
37
38
39
Patricios J, Collins R, Braneld A, et al. The sports concussion note: should SCAT
become SCOAT? Br J Sports Med 2012;46:198201.
Ahmed OH, Pulman A. Concussion information on the move: the role of
mobile technology in concussion management. J Community Inform 2013;9.
TheConcussionToolbox.com. The Concussion Toolbox App 2013 [cited 19 May
2013]. http://theconcussiontoolbox.com/concussion-toolbox-app/
NFLevolution.com. X2 Biosystems app to help NFL doctors with concussions
next season 2013 [cited 19 May 2013]. http://www.nevolution.com/article/
X2-Biosystems-app-to-help-NFL-doctors-with-concussions-next-season?
ref=6015
Google play. SACTool BetaAndroid Apps on Google play 2013 [cited 14 May
2013]. https://play.google.com/store/apps/details?id=net.nutecht.sactool&hl=en
McCrea M, Kelly JP, Randolph C, et al. Standardized Assessment of Concussion
(SAC): on-site mental status evaluation of the athlete. J Head Trauma Rehabil
1998;13:2735.
U.S. Food and Drug Administration. Health Information Technologies: Administration
Perspectives on Innovation and Regulation 2013 [cited 14 May 2013]. http://www.
fda.gov/NewsEvents/Testimony/ucm344395.htm
doi: 10.1136/bjsports-2013-092930
Updated information and services can be found at:
http://bjsm.bmj.com/content/49/8/499
These include:
References
Email alerting
service
Topic
Collections
This article cites 23 articles, 17 of which you can access for free at:
http://bjsm.bmj.com/content/49/8/499#BIBL
Receive free email alerts when new articles cite this article. Sign up in the
box at the top right corner of the online article.
Notes