BRIEF REPORTS
A Best-Worst Scaling Experiment to Prioritize Caregiver
Concerns About ADHD Medication for Children
Melissa Ross, M.A., John F. P. Bridges, Ph.D., Xinyi Ng, B.S.Pharm., Lauren D. Wagner, M.S., Ph.D., Emily Frosch, M.D.,
Gloria Reeves, M.D., Susan dosReis, Ph.D.
Objective: The objective of this feasibility study was to de-
velop and pilot an instrument to elicit caregivers’ priorities
when initiating attention-deficit hyperactivity disorder (ADHD)
medication for their child.
Methods: A best-worst scaling experiment was used to rank
competing priorities when initiating ADHD medicine. Forty-six
participants were recruited for a two-phase study involving survey
development (phase 1, N521) and the survey pilot (phase 2,
N525). Best-worst scores and 95% confidence intervals in-
dicating the relative importance of 16 concerns were determined,
and t tests were used to determine the scores’ significance.
Attention-deficit hyperactivity disorder (ADHD) now affects
11% of U.S. children ages 17 or younger (1,2), and 3.5 million
are prescribed a stimulant medication (2). Children often
need medication—yet among caregivers, the acceptability of
medication is low, and there is much uncertainty about using
medication for their child (3–7). Even when medication is
initiated, many caregivers discontinue use within two years
(3,4,8).
Several studies have focused on caregivers’ perceptions
of treatment for ADHD, mostly among low-income families
from racial-ethnic minority groups. Caregivers initially do
not use medication, reluctantly turn to medication only after
exhausting all other options, and do not always view ADHD
medication as appropriate for children (3–5). However, prior
research has not elicited how caregivers’ priorities may in-
fluence decisions to initiate medication for their child (7,9).
Therefore, this feasibility study aimed to develop and pilot a
best-worst scaling instrument to assess caregivers’ priorities
when initiating ADHD medicine for their child. The Uni-
versity of Maryland Institutional Review Board approved
the study and granted a waiver of informed consent.
METHODS
Mixed methods were used to develop and test a best-worst
scaling instrument to elicit caregivers’ priority concerns when
208 ps.psychiatryonline.org
Results: The significance of best-worst scores for most
concerns indicated that the choices were purposeful.
Concerns about helping the child become a success-
ful adult, having a doctor who addresses caregivers’
concerns, and improving school behavior were ranked
highest.
Conclusions: The best-worst scaling method can elicit pri-
orities for children’s mental health treatment. Future work
using this method will guide family-centered care.
Psychiatric Services 2015; 66:208–211; doi: 10.1176/appi.ps.201300525
deciding whether to use ADHD medication for their child.
Best-worst scaling was preferred to a conjoint discrete-choice
experiment, often used in health care research (10,11), for
several reasons. Grounded in random utility theory, best-worst
scaling evokes tradeoffs by asking individuals to select one best
and one worst attribute among competing alternatives within
a profile. By comparison, conjoint experiments force selec-
tions among two or more different profiles. With best-worst
scaling, individuals select attributes that are of greatest value
to them relative to other shown attributes; as a result, in-
formation about what matters most to individuals is gained
(12–14). This provides more enriched information on hetero-
geneity of specific priority concerns than can be obtained from
selecting one profile containing multiple priorities (12–14). In
addition, best-worst scaling makes it possible to estimate and
compare the average utility of a profile’s attributes, whereas in
a conjoint discrete choice experiment, the reference group is
the whole scenario (14).
Two separate convenience samples were recruited from
two support organizations in metropolitan Baltimore for
caregivers of children with mental health needs. First, a sam-
ple of 21 caregivers participated in focus groups as part of the
development of the best-worst scaling instrument. A second
sample of 25 caregivers of children ages four to 14 with an
ADHD diagnosis participated in a pilot study of the best-worst
scaling instrument. The demographic characteristics of the
Psychiatric Services 66:2, February 2015

two samples were very similar. A majority (.75%) was African
American, and most (.85%) were the children’s biological
mothers.
Attribute statements for the best-worst scaling instrument
were identified by using data from a previous qualitative study
of caregivers’ experiences. The study examined the experiences
of caregivers as they came to terms with the ADHD diagnosis
and medication treatment (3,4). This prior work generated
a model of caregivers’ priorities in initiating medication that
was grounded in their views of treatment in term of appro-
priateness (for example, whether the child was too young),
anticipated effects (whether the medication would harm the
child), and symbolic representation (whether using medicine
meant being a bad parent) (3). This model was cross-referenced
with the published literature (5,6) to develop a list of attribute
statements.
In October 2012, a family support group leader from one of
the family organizations recruited caregivers for the first
sample. Caregivers were asked to participate in focus groups to
assess attribute statement relevance. Fifteen caregivers par-
ticipating in the first of two focus groups were presented with
26 attribute statements reflecting the potential priorities of
caregivers when considering whether to initiate ADHD med-
ication for their child. They were asked to classify the state-
ments into four categories (short-term concern, long-term
impact, societal views, and supportive network) or, if needed,
to suggest a new category. On the basis of this feedback, at-
tribute statements were revised and presented to a second
focus group of six members of the same support organization
for verification and relevance. No further amendments were
suggested.
Sixteen attribute statements for the best-worst scaling in-
strument were retained. The statements were divided evenly
by category, with each category containing two positively and
two negatively phrased statements. Two child psychiatrists
reviewed the clinical and practical relevance of the attribute
statements.
A balanced, incomplete block design was used to construct
the choice task profiles so that each attribute statement was
seen the same number of times and any two attribute state-
ments appeared together the same number of times. This
design ensured equal probability of selection for each attri-
bute statement. The survey had 16 choice task profiles, each
displaying six of the 16 attribute statements. [An example of
a best-worse choice task profile is available online as a data
supplement to this article.]
In each choice task profile, participants were asked to
think back to when they first learned of their child’s ADHD
diagnosis and some of the situations that influenced their
decision to initiate ADHD medication. They were instruc-
ted to select one attribute statement from among the six
choices that reflected the most important concern (best
choice) and then select one attribute statement that re-
flected the least important concern (worst choice) that
influenced their decision to initiate ADHD medication for
their child.
Psychiatric Services 66:2, February 2015
ROSS ET AL.
The family support group leader from a different orga-
nization helped to recruit caregivers for the pilot from five
support groups for families in the Baltimore metropolitan
area. During the pilot, conducted from November 2012 to
January 2013, 25 caregivers used paper and pencil to com-
plete the best-worst scaling instrument. All of the partic-
ipants had children between the ages of 4 and 14 who had
been diagnosed as having ADHD, and all used medication for
their child. Most of the children also were currently using
psychotherapy or had an individualized education plan.
The principal investigator and a graduate research as-
sistant attended the support group meetings, explained the
purpose of the survey, and provided instructions for com-
pleting the choice tasks. The pilot survey was completed, on
average, in 15 minutes. At the conclusion of the meeting,
participants were asked to provide feedback regarding the
clarity and relevance of the choice task profiles. No further
modifications were recommended.
Survey responses for each choice task profile were coded
into two binary variables. The statements chosen as best and
worst each received a score of 1, and the statements that were
not chosen as best or worst received a score of 0. Best-worst
scores for each attribute statement were calculated as the sum
of the best selections minus the sum of the worst selections
across all respondents divided by 150 (the number of times
each attribute statement was displayed [N56] multiplied by
25 participants) (15). A t test assessed if scores differed sig-
nificantly from 0 (a5.05), which would imply that selections
were not made at random but reflected stated priorities.
RESULTS
A positive best-worst score indicated that the attribute state-
ment was selected as most important more frequently than it
was selected as least important, and negative best-worst scores
indicated the opposite. The number of times each attribute
statement was chosen as best and worse is shown in Table 1,
along with each attribute’s mean best-worst score and 95%
confidence intervals. All attribute statements except “ADHD
medicine is not needed to control my child’s home behavior”
were significant (p,.05).
Ranking of mean best-worst scores from largest to smallest
was used to determine relative attribute importance. Overall,
using medication to help their child become a successful adult
(.41) was ranked highest, and concern that others would think
badly of the child if he or she used ADHD medicine (–.53)
was ranked lowest. [A list of attribute statements ranked by
highest to lowest best-worse score is available online as a data
supplementto this article.] Table 1 lists the attribute state-
ments in each category by order of importance. Control of
school behavior (.39) was the highest-ranked short-term
concern (p,.001). The role of medicine in helping the child
be a successful adult was a key long-term concern (p,.001).
The only positive score in the supportive-network category
was having a doctor who addressed the caregivers’ con-
cerns about ADHD medicine (p,.001). Scores for the other
ps.psychiatryonline.org 209
A BEST-WORST SCALING EXPERIMENT TO PRIORITIZE CAREGIVER CONCERNS ABOUT ADHD MEDICATION FOR CHILDREN

TABLE 1. Best-worst score for 16 attribute statements related to medication of children with ADHD, Children’s mental health ser-
ranked by importance within each category of concerns vices research has been lim-
Category and attribute statements Best-worse score (M)a 95% CI Bestb Worstc p ited by a lack of rigorous
Short-term concerns methods for eliciting caregiver
ADHD medicine is needed to control .39 .35 to .43 58 0 ,.001 priorities. Eliciting caregivers’
my child’s school behavior priorities early in the clinical
ADHD medicine side effects .23 .19 to .26 37 3 .038 encounter can guide family-
outweigh its benefits
ADHD medicine will help my child get .07 .03 to .10 21 11 ,.001
centered treatment planning.
better grades There were several limita-
ADHD medicine is not needed to –.05 –.02 to –.09 9 17 .057 tions. The sample was limited
control my child’s home behavior in diversity, size, and geo-
Long-term impact graphic locale and may not
ADHD medicine will help my child be .41 .36 to .45 63 2 ,.001 generalize to all caregivers of
a successful adult
children with ADHD. Although
ADHD medicine has risks that will .28 .24 to .32 43 1 ,.001
affect my child’s future health recruitment from different ad-
ADHD medicine will help my child .15 .12 to .19 28 5 ,.001 vocacy organizations can result
finish high school in potentially different sam-
ADHD medicine will limit my child’s –.09 –.06 to –.13 8 22 .005 ples, convenience sampling was
career options
used to recruit caregivers from
Supportive network homogeneous sources. The
The doctor addresses my concerns .29 .25 to .33 46 2 ,.001
about ADHD medicine
perspectives reflected the pri-
The school has pressured me to use –.30 –.29 to –.37 5 50 ,.001 orities of one parent—the
ADHD medicine for my child mother. Although we sought
My family does not see why my child –.33 –.29 to –.37 2 52 ,.001 continuous caregiver feed-
needs ADHD medicine back, this list of relevant at-
My friends agree with using ADHD –.43 –.38 to –.47 4 68 ,.001
medicine for my child
tribute statements may not be
exhaustive. However, attribute
Societal views
ADHD medicine will help my child .25 .21 to .30 45 7 ,.001 development was an iterative
get along with others feedback process in which
ADHD medicine will hurt my child’s –.05 –.02 to –.07 4 11 .034 statements were confirmed
self-esteem separately by several individ-
Giving my child ADHD medicine does –.30 –.25 to –.35 18 63 ,.001
uals. Finally, stated priorities
not mean I am a bad parent
Others will think badly of my child if –.53 –.48 to –.57 5 84 ,.001 were not correlated with treat-
he or she uses ADHD medicine ment adherence, but that was
a
The results are from a pilot survey of 25 family members of children with ADHD. The best-worse score represents
not the goal of this feasibility
the sum of times the attribute was chosen as the best (most important) concern minus the sum of times the attribute study.
was chosen as the worst (least important) concern, divided by 150 (the number of times the attribute was displayed The purpose of this feasi-
[N56] multiplied by the number of participants). Scores can be ranked from highest to lowest to reflect the order of
importance.
bility study was to test a best-
b
Number of times the attribute statement was chosen as the most important concern worst scaling instrument prior
c
Number of times the attribute statement was chosen as the least important concern to use in a larger comprehen-
sive survey. The instrument is
attributes in the category indicated that family, friends, and currently being used in a study that is designed to capture
school personnel were less important influences compared clinical diagnoses and receipt of mental health care services in
with doctors. With the exception of the child’s peer rela- order to assess the association between priorities and treatment
tions, all other attributes in the societal views category adherence.
were negative.
CONCLUSIONS
DISCUSSION
Caregivers’ priorities are nuanced and have an impact on
This study demonstrates the feasibility of best-worst scaling decisions about their child’s or adolescent’s mental health
for eliciting caregivers’ priorities in initiating medication for care. The methods described here may help to better de-
their child’s ADHD. Significant best-worst scores indicated fine, recognize, and understand caregivers’ priorities so
that choices were not random selections. Caregivers com- that clinicians may engage caregivers in shared decision
pleted the instrument with relative ease. making about treatment for their child. This could accel-
The caregiver-centered instrument holds great promise erate caregiver-centered outcomes research in children’s
for advancing family-centered research and clinical practice. mental health.

210 ps.psychiatryonline.org Psychiatric Services 66:2, February 2015


AUTHOR AND ARTICLE INFORMATION
Ms. Ross, Ms. Ng, Dr. Wagner, and Dr. dosReis are with the Department
of Pharmaceutical Health Services Research, University of Maryland
School of Pharmacy, Baltimore. Dr. Bridges is with the Department of
Health Policy and Management, Johns Hopkins Bloomberg School of
Public Health, Baltimore. Dr. Frosch is with the Division of Child and
Adolescent Psychiatry, Johns Hopkins University School of Medicine,
Baltimore. Dr. Reeves is with the Department of Psychiatry, University of
Maryland School of Medicine, Baltimore. Send correspondence to Dr.
dosReis (e-mail: sdosreis@rx.umaryland.edu).
This work was funded by a grant from the National Institute of Mental
Health (R34 MH093502) to Dr. dosReis. The authors are grateful to Osler
Andres, Pharm.D., and Katie Brant, Pharm.D., for their assistance in
piloting the survey and to Bev Butler, Jane Walker, and Angela Vaughn-
Lee for community outreach with the families who participated in the
pilot.
The authors report no financial relationships with commercial interests.
REFERENCES
1. Merikangas KR, He JP, Brody D, et al: Prevalence and treatment of
mental disorders among US children in the 2001–2004 NHANES.
Pediatrics 125:75–81, 2010
2. Visser SN, Danielson ML, Bitsko RH, et al: Trends in the parent-
report of health care provider–diagnosed and medicated attention-
deficit/hyperactivity disorder: United States, 2003–2011. Journal of
the American Academy of Child and Adolescent Psychiatry 53:
34–46 e2, 2014
3. dosReis S, Mychailyszyn MP, Evans-Lacko SE, et al: The meaning
of ADHD medication and parents’ initiation and continuity of
treatment for their child. Journal of Child and Adolescent Psy-
chopharmacology 19:377–383, 2009
4. dosReis S, Mychailyszyn MP, Myers M, et al: Coming to terms with
ADHD: how urban African-American families come to seek care
for their children. Psychiatric Services 58:636–641, 2007
5. Leslie LK, Plemmons D, Monn AR, et al: Investigating ADHD
treatment trajectories: listening to families’ stories about medication
Psychiatric Services 66:2, February 2015
ROSS ET AL.
use. Journal of Developmental and Behavioral Pediatrics 28:179–188,
2007
6. Bussing R, Zima BT, Gary FA, et al: Barriers to detection, help-
seeking, and service use for children with ADHD symptoms. Journal
of Behavioral Health Services and Research 30:176–189, 2003
7. dosReis S, Butz A, Lipkin PH, et al: Attitudes about stimulant
medication for attention-deficit/hyperactivity disorder among Af-
rican American families in an inner city community. Journal of
Behavioral Health Services and Research 33:423–430, 2006
8. Charach A, Ickowicz A, Schachar R: Stimulant treatment over five
years: adherence, effectiveness, and adverse effects. Journal of the
American Academy of Child and Adolescent Psychiatry 43:559–567,
2004
9. Fiks AG, Mayne S, Debartolo E, et al: Parental preferences and
goals regarding ADHD treatment. Pediatrics 132:692–702, 2013
10. Cunningham CE, Deal K, Rimas H, et al: Providing information to
parents of children with mental health problems: a discrete choice
conjoint analysis of professional preferences. Journal of Abnormal
Child Psychology 37:1089–1102, 2009
11. Bridges J, Kinter E, Kidane L, et al: Things are looking up since
we started listening to patients: recent trends in the application of
conjoint analysis in health, 1982–2007. The Patient 1:273–282,
2008
12. Flynn TN, Louviere JJ, Peters TJ, et al: Estimating preferences for
a dermatology consultation using best-worst scaling: comparison of
various methods of analysis. BMC Medical Research Methodology
8:76, 2008
13. Louviere JJ, Flynn TN: Using best-worst scaling choice experi-
ments to measure public perceptions and preferences for health-
care reform in Australia. The Patient 3:275–283, 2010
14. Flynn TN, Louviere JJ, Peters TJ, et al: Best-worst scaling: what it
can do for health care research and how to do it. Journal of Health
Economics 26:171–189, 2007
15. Gallego G, Bridges JF, Flynn T, et al: Using best-worst scaling in
horizon scanning for hepatocellular carcinoma technologies. In-
ternational Journal of Technology Assessment in Health Care 28:
339–346, 2012
ps.psychiatryonline.org 211