AHA/ASA Scientific Statement

Risk Adjustment of Ischemic Stroke Outcomes for

Comparing Hospital Performance
A Statement for Healthcare Professionals From the American Heart
Association/American Stroke Association
Irene L. Katzan, MD, MS, FAHA, Chair; John Spertus, MD, FAHA, Co-Chair;
Janet Prvu Bettger, ScD, FAHA*; Dawn M. Bravata, MD*;
Mathew J. Reeves, PhD, DVM, FAHA*; Eric E. Smith, MD, MPH, FAHA*;
Cheryl Bushnell, MD, MHS, FAHA; Randall T. Higashida, MD, FAHA;
Judith A. Hinchey, MD, FAHA; Robert G. Holloway, MD, MPH; George Howard, DrPH;
Rosemarie B. King, PhD, RN, FAHA; Harlan M. Krumholz, MD, FAHA;
Barbara J. Lutz, PhD, RN, CRRN, FAHA; Robert W. Yeh, MD, MSc, FAHA; on behalf of the
American Heart Association Stroke Council, Council on Quality of Care and Outcomes Research,
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017

Council on Cardiovascular and Stroke Nursing, Council on Cardiovascular Radiology and Intervention,
Council on Cardiovascular Surgery and Anesthesia, and Council on Clinical Cardiology

Background and PurposeStroke is the fourth-leading cause of death and a leading cause of long-term major disability
in the United States. Measuring outcomes after stroke has important policy implications. The primary goals of this
consensus statement are to (1) review statistical considerations when evaluating models that define hospital performance
in providing stroke care; (2) discuss the benefits, limitations, and potential unintended consequences of using various
outcome measures when evaluating the quality of ischemic stroke care at the hospital level; (3) summarize the evidence
on the role of specific clinical and administrative variables, including patient preferences, in risk-adjusted models of
ischemic stroke outcomes; (4) provide recommendations on the minimum list of variables that should be included in risk
adjustment of ischemic stroke outcomes for comparisons of quality at the hospital level; and (5) provide recommendations
for further research.
Methods and ResultsThis statement gives an overview of statistical considerations for the evaluation of hospital-level
outcomes after stroke and provides a systematic review of the literature for the following outcome measures for ischemic
stroke at 30 days: functional outcomes, mortality, and readmissions. Data on outcomes after stroke have primarily involved
studies conducted at an individual patient level rather than a hospital level. On the basis of the available information,
the following factors should be included in all hospital-level risk-adjustment models: age, sex, stroke severity, comorbid
conditions, and vascular risk factors. Because stroke severity is the most important prognostic factor for individual patients
and appears to be a significant predictor of hospital-level performance for 30-day mortality, inclusion of a stroke severity
measure in risk-adjustment models for 30-day outcome measures is recommended. Risk-adjustment models that do not
include stroke severity or other recommended variables must provide comparable classification of hospital performance

*Writing Group Section Leader.

The American Heart Association makes every effort to avoid any actual or potential conflicts of interest that may arise as a result of an outside relationship
or a personal, professional, or business interest of a member of the writing panel. Specifically, all members of the writing group are required to complete
and submit a Disclosure Questionnaire showing all such relationships that might be perceived as real or potential conflicts of interest.
This statement was approved by the American Heart Association Science Advisory and Coordinating Committee on December 12, 2013. A copy of the
document is available at http://my.americanheart.org/statements by selecting either the By Topic link or the By Publication Date link. To purchase
additional reprints, call 843-216-2533 or e-mail kelle.ramsay@wolterskluwer.com.
The American Heart Association requests that this document be cited as follows: Katzan IL, Spertus J, Bettger JP, Bravata DM, Reeves MJ, Smith EE,
Bushnell C, Higashida RT, Hinchey JA, Holloway RG, Howard G, King RB, Krumholz HM, Lutz BJ, Yeh RW; on behalf of the American Heart Association
Stroke Council, Council on Quality of Care and Outcomes Research, Council on Cardiovascular and Stroke Nursing, Council on Cardiovascular Radiology
and Intervention, Council on Cardiovascular Surgery and Anesthesia, and Council on Clinical Cardiology. Risk adjustment of ischemic stroke outcomes
for comparing hospital performance: a statement for healthcare professionals from the American Heart Association/American Stroke Association. Stroke.
2014;45:.
Expert peer review of AHA Scientific Statements is conducted by the AHA Office of Science Operations. For more on AHA statements and guidelines
development, visit http://my.americanheart.org/statements and select the Policies and Development link.
Permissions: Multiple copies, modification, alteration, enhancement, and/or distribution of this document are not permitted without the express
permission of the American Heart Association. Instructions for obtaining permission are located at http://www.heart.org/HEARTORG/General/Copyright-
Permission-Guidelines_UCM_300404_Article.jsp. A link to the Copyright Permissions Request Form appears on the right side of the page.
2014 American Heart Association, Inc.
Stroke is available at http://stroke.ahajournals.org DOI: 10.1161/01.str.0000441948.35804.77

1
 2StrokeMarch 2014

as models that include these variables. Stroke severity and other variables that are included in risk-adjustment models
should be standardized across sites, so that their reliability and accuracy are equivalent. There is a pressing need for
research in multiple areas to better identify methods and metrics to evaluate outcomes of stroke care.
ConclusionsThere are a number of important methodological challenges in undertaking risk-adjusted outcome
comparisons to assess the quality of stroke care in different hospitals. It is important for stakeholders to recognize these
challenges and for there to be a concerted approach to improving the methods for quality assessment and improvement.
(Stroke. 2014;45:00-00.)
Key Words: AHA Scientific Statements health policy outcomes stroke

E valuating patient outcomes is a central part of optimiz-

ing the quality of patient care. In an effort to improve
healthcare quality, regulatory bodies have begun to measure
outcomes after hospitalization for several diseases, including
pneumonia, myocardial infarction, and congestive heart fail-
ure. Similarly, stroke is an emerging, high-priority target for
such efforts. Stroke is the fourth-leading cause of death and a
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017
Discuss the benefits, limitations, and potential unin-
tended consequences of using various outcome mea-
sures, including 30-day mortality, 30-day all-cause
readmission, and 30-day functional status, when evalu-
ating the quality of stroke care at the hospital level
Summarize the evidence on the role of specific clinical
and administrative variables, including patient prefer-

cause of long-term major disability in the United States.1 It is
also one of the top 10 costliest conditions for the Centers for
Medicare & Medicaid Services (CMS).2 As with most disease
states, accurately measuring and comparing hospitals out-
comes is complex. Some outcome determinants are outside
a providers control, including patient factors such as age, the
severity of illness, comorbid conditions, and preferences for
particular treatments.3 Domains that are modifiable and poten-
tially improvable include the content and organization of the
stroke care delivery system. These include factors such as the
relationship between providers within a particular community
(eg, emergency medical services, hospitals, nursing support,
rehabilitation services, and home environments), hospice
availability, and the types and skills of providers. Hospitals
also have control over the use of acute (eg, cerebral reperfu-
sion) and chronic (eg, rehabilitation services) treatments. The
challenge in using outcomes as a measure of healthcare qual-
ity is to account for the unmodifiable factors so that variations
in outcomes between providers are attributable to the pro-
cesses of stroke care and more accurately reflect the quality
of care provided.
A clear link between quality of hospital care and outcomes
after stroke has not been well demonstrated in the literature.
The concept that differences in case-mixadjusted outcomes
reflect quality has also been challenged.4,5 However, given the
intense interest in systematically measuring stroke outcomes
and in comparing outcomes across different facilities so that
the quality of stroke care can be evaluated and optimized,
there is a need to define the relevant stroke outcomes to mea-
sure, as well as the factors that should be accounted for to
make comparisons across different facilities valid. The pres-
ent scientific statement strives to illuminate the benefits and
limitations of various stroke outcome measures and to provide
guidance on the factors that should be used in risk adjustment
so that these outcome measures will be most useful to clinical
scientists, clinicians, policy makers, and quality experts when
evaluating the quality of stroke care.
Specific goals of the statement are as follows:
Review statistical considerations when evaluating models
that define hospital performance in providing stroke care
ences, in risk-adjusted models of stroke outcomes
Provide recommendations on the minimum list of vari-
ables that should be included in risk adjustment for stroke
outcomes when quality is compared at the hospital level
Provide recommendations for further research
The present statement provides a systematic review of the lit-
erature that focuses on outcome measures for ischemic stroke.
It does not include measures for hemorrhagic stroke or tran-
sient ischemic attack because of differences in management
and outcomes for these conditions. The writing group chose
the 30-day time period as a reasonable compromise for a stan-
dardized framework for measuring outcomes from the hospi-
tals perspective, while recognizing that some outcomes, such
as functional status, continue to improve over time.
Methodology and Evidence Overview
The Stroke Council of the American Heart Association (AHA)/
American Stroke Association commissioned the assembled
authors, who represent the fields of vascular neurology, biosta-
tistics, epidemiology, health service research, nursing, rehabil-
itative medicine, and neuroradiology. The writing committee
was organized into sections, which were led by section leaders.
Literature review was performed and initial drafts were written
within each section. Data compilation and recommendation
was discussed and revised by the full committee over a series
of teleconferences that took place over 9 months.
Search criteria included articles published in English in
1980 through 2011 that were indexed in MEDLINE (PubMed).
Because of potential differences in healthcare practices and
outcomes across countries, this review included only stud-
ies from high-income countries, using a list provided by the
World Bank.6 Studies that included placebo groups from
clinical trials were reviewed because they may provide sys-
tematically collected and relevant data on outcomes of inter-
est. We were cognizant of the potential limitation of reduced
generalizability of data from such highly selected patients.
Search and article reviews were limited to studies of ischemic
stroke patients. Although the focus of the present statement is
on 30-day outcomes, we also included studies that evaluated
mortality at the time of discharge to avoid missing variables

in these studies found to be strongly associated with mortality.
In addition, we expanded our review to include analyses of
functional outcomes at 90 days, because studies have shown
that function and disability at 30 days are highly predictive of
function at 90 days.7,8
In the review of functional outcome measures, studies
that used a composite end point for poor outcomes that
combined death with severe disability or dependence were
excluded. Combining poor function and death prevents the
ability to discern the variables that predict poor function sepa-
rately from those that predict mortality and inappropriately
equates limitations in activities of daily living with death.9
From a quality perspective, it is likely that acute interventions
that have no effect on mortality may affect function, and these
relationships may not be detected with the use of a composite
end point of disability and death.10
Results
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017
Statistical Considerations
Differences between hospitals in the mortality, readmis-
sion, and functional status of stroke patients can be attribut-
able to hospital effects that occur because of differences
in the quality of hospital care, or to case-mix effects that
occur because of differences in the characteristics of patients
treated, or to random chance. When seeking to quantify hos-
pital quality, the most common application of risk-adjustment
methods is to use multivariable models to mitigate the con-
tribution of measured patient characteristics present at the
time of admission, such as differences in age, stroke severity,
comorbidities, and risk factors. This assumes, although it is
frequently unstated, that the remaining variation in outcomes
at the hospital level represents the direct effect of differences
in the quality of care delivered by the hospitals, after random
error is taken into account.11 However, studies have not always
shown associations between hospital risk-adjusted outcomes
and poorer-quality care.4,5,12 Possible explanations for the
inconsistent relationships mostly relate to fundamental limi-
tations of observational study designs,12 but shortcomings in
the underlying quality metrics to reflect changes in disease
outcomes should also be considered. We focus, in this sec-
tion, on the use of risk-adjustment methods to mitigate patient
case-mix effects so as to be able to compare outcomes across
different hospitals for the purpose of profiling, rather than on
generating individual patient-level prognostic models.13 The
latter approach to risk prediction at the individual level14 is
important to both the patient and physician but is best viewed
as a different application of these methods.
Methodological Standards for Risk-Adjustment Models
Used for Hospital Profiling
Krumholz and colleagues11 have previously summarized stan-
dards for the development of risk-adjustment models when
used for public reporting of healthcare providers outcomes
(Table1). Issues that are especially relevant to stroke include
the following:
1. Sufficiently high-quality and timely data: Data can
be identified from administrative databases, medical
records, or specific efforts to collect quality of care data
Katzan etal Ischemic Stroke Outcomes 3
Table 1. Preferred Attributes of Models Used for Publicly
Reported Outcomes
1. Clear and explicit definition of an appropriate patient sample
2. Clinical coherence of model variables
3. Sufficiently high-quality and timely data
4. Designation of an appropriate reference time before which covariates are
derived and after which outcomes are measured
5. Use of an appropriate outcome and a standardized period of outcome
assessment
6. Application of an analytical approach that takes into account the multilevel
organization of data
7. Disclosure of the methods used to compare outcomes, including disclosure
of performance of risk-adjustment methodology in derivation and validation
samples
Reproduced with permission from Krumholz etal.11 2006 American Heart
Association, Inc.
prospectively, such as Get With the Guidelines (GWTG)
Stroke.15 Administrative data are collected primarily for
billing purposes and may not capture important clinical
data known to be associated with outcomes.16 A signifi-
cant challenge for risk adjustment of stroke outcomes
with either administrative or clinical data is that mea-
sures of stroke severity, such as the National Institutes of
Health Stroke Scale (NIHSS), are frequently missing.17
Moreover, these missing data are typically not missing at
random, which has the potential to introduce important
selection biases.
2. Consistency in coding across sites: A further complica-
tion of using administrative data is that variation across
hospitals in the coding of comorbidities and risk factors
can have an important impact on risk-adjustment mod-
els and the accuracy of hospital profiling. Consistency
of coding can refer to the variation between hospitals in
how they define the presence or absence of a specific
comorbidity or risk factor. An example of this phenom-
enon is the coding of shock in heart failure or myocardial
infarction patients. Shock is an important predictor of
outcomes in these patients, but there are differences in
the coding of shock across centers that may represent
differences in the definition of shock rather than varia-
tion in its underlying prevalence.
Consistency of coding can also refer to variation in
the number of comorbidities and risk factors recorded
by a hospital for a given patient. Some hospitals rou-
tinely code for a larger number of comorbidities and risk
factors than other hospitals, and this deeper level of
coding can have important implications for the accuracy
of the risk-adjustment process. Variation in the depth of
coding has been shown to produce paradoxical increases
in the very differences for which the risk-adjustment
models are attempting to adjust.18,19
3. Designation of an appropriate reference time before
which covariates are derived and after which outcomes
are measured: The goal of risk adjustment is to adjust
for the condition of the patient at the time patient care
began (referred to as the reference time). This requires
identifying and distinguishing between historical
comorbid conditions that may have resolved at the time
of admission (which do not need to be adjusted for in
 4StrokeMarch 2014
the analysis), comorbid conditions present at the time
of admission (which do need to be adjusted for), and
complications or other events that developed during or
after the admission (for which no adjustment should
be made).20 The inclusion of factors that occurred after
the reference time may adjust away the differences
in quality of care, thereby defeating the central purpose
of the model itself. A limiting factor in risk-adjustment
models that use administrative data or clinical registries
is that they typically do not include information regard-
ing the timing of complications or comorbid conditions
in relation to the stroke event.
4. Use of an appropriate outcome and a standardized period
of outcome: Any outcome chosen as a measure of quality
of care should meet 2 fundamental standards. First, the
outcome should reflect an entity of interest to patients.
Second, evidence should exist to support the relation-
ship between the outcome and quality of care. The time
period over which the outcome is assessed also needs to
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017
be standardized. In clinical stroke trials, 90 days tends
to be the standard time period used for the evaluation
of functional outcomes, whereas the AHA/American
College of Cardiology task force on performance mea-
sures has recommended that stroke outcomes be mea-
sured at 30 days,21 which is also the time frame that
CMS has adopted for a majority of its publicly reported
outcome measures.22 We support the use of the shorter
30-day time period because it reduces the potential for
loss to follow-up and mitigates the effects of differ-
ences in care that occur after hospitalization. Outcomes
measured at hospital discharge may have some utility
because they are easier to collect, but they are limited
by differences in length of stay and the effects of patient
transfers between hospitals. In addition, outcomes mea-
sured at discharge are insufficient to assess improvement
in functional recovery, which typically occurs over sev-
eral weeks.23,24 It has been recommended that discharge
measures should only be used when validated against a
standardized period of assessment.11 We recognize that
different opinions on timing of outcomes measurement
exist, and we suggest more research be conducted to
understand the impact of using in-hospital measures
as proxies of the more desirable, but often impractical,
longer-term outcomes.
Model Development and Application of Risk-Adjustment
Models for Hospital Profiling
Selection of variables for inclusion in risk-adjustment mod-
els should follow the previous published guidelines by Harrel
etal13 and Iezzoni.25 The goal is to develop a model that cap-
tures the patient information that is significantly related to the
outcome of interest. For risk-adjustment modeling to accu-
rately identify hospital-level effects, the model must include
strong and valid patient-level predictors that accurately
stratify risk at the hospital level. It is therefore important to
recognize that if a strong patient-level predictor variable is
distributed similarly across different hospital populations,
then controlling for it in a model will have little effect on the
risk-adjusted hospital-level outcomes; only factors that are
both related to the outcome and are differentially distributed
between hospitals can act as confounders in the assessment of
hospital differences. For example, a study of US Department
of Veterans Affairs (VA) hospitals found that although the
NIHSS was a strong predictor of mortality among individual
ischemic stroke patients, it had little impact on the calculated
risk-adjusted standardized mortality rates, probably because
variation in both NIHSS and 30-day mortality was limited
across the VA hospitals in the study.26
The traditional statistical approach used for hospital profil-
ing involves the application of risk-adjustment models to gen-
erate hospital-specific risk-standardized ratios that are used to
rank order providers and to identify outlier hospitals. These
ratios are generated by comparing the observed event rate at
a specific hospital with the expected event rate, which is cal-
culated by applying the regression coefficients from the risk-
adjusted model to the hospitals specific patient population.
An observed/expected ratio >1 indicates higher than expected
events for a particular hospital, whereas a ratio <1 indicates
fewer than expected events. Often the observed/expected ratio
is multiplied by the average event rate in the underlying popu-
lation, which generates a risk-standardized outcome rate for
each hospital. This approach, referred to as indirect standard-
ization, has been heavily criticized for failing to account for
variation in case numbers across hospitals or for intrahospital
clustering effects.27,28
Newer methods based on hierarchical or multilevel mod-
els that use random effect terms to describe hospital-specific
effects address these deficiencies and are now the standard
approach used for hospital profiling by CMS.27,29,30 The first
level of a hierarchical model includes the patient character-
istics, whereas the second level includes hospital-specific
random intercepts that allow for different baseline event rates
between the hospitals. The hospital-specific intercepts also
account for the clustering of patients within each hospital.
Somewhat confusingly, random effects models are described
as generating predicted versus expected ratios, rather than the
observed versus expected ratios described previously. Using
mortality as an example, the expected number of deaths at a
hospital from a random effects model is obtained by apply-
ing the model regression coefficients (generated from all the
patients in the sample) to the patient population observed in
the particular hospital; adding the average of the hospital-
specific intercepts; transforming; and then summing over all
patients in the hospital to get the estimated number of deaths.
The predicted number of deaths is calculated similarly except
that the hospital-specific random intercept (which represents
the baseline mortality rate of the particular hospital) is sub-
stituted for the average hospital-specific intercept.29 The
predicted over expected ratio is often multiplied by the unad-
justed mortality rate for the total population to yield a risk-
standardized mortality rate.31 Other potential standardization
approaches include generalized estimating equations, which
can account for clustering without invoking random effects.
Assessment of Model Fit (Calibration) and Discrimination
There are 2 existing, albeit imperfect, methods for describing
the accuracy of a risk-adjustment model at the patient-level:
calibration and discrimination.
Calibration describes the degree to which a models pre-
dicted probabilities match the observed data.32 In a perfectly

calibrated model, a predicted risk estimate of x% means that
exactly x% of patients are observed to have the outcome.33
Large discrepancies between the number of observed and pre-
dicted events indicates that the model is not well calibrated
and does not fit the data well. There are 3 common explana-
tions for a poor fitting model:
1. Residual confounding, which can result when a critical
predictor variable is either missing or poorly measured
2. Failure to include important interactions between predic-
tor variables
3. Misspecification of the scale of a predictor variable,
for example, where a continuous variable such as age
is assumed to have a simple linear relationship rather
than a more complicated scale, such as a polynomial or
exponential
Discrimination describes the degree to which a model sepa-
rates (or discriminates between) those patients who do and do
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017
not go on to develop the outcome of interest.32,33 Discrimination
is measured by the C statistic. The closer the C statistic is to
1.0, the better the model can predict outcomes at an individual
patient level. An important limitation of the C statistic is that
it only measures the predictive power of a model at the patient
level and does not directly express the ability of the model
to accurately profile hospitals with respect to the hospital-
specific risk-standardized ratios.34 Both calibration and dis-
crimination statistics are important measures to be evaluated
when judging the performance of risk-adjustment models.
Further Methodological Issues in the Development of
Risk-Adjusted Models for Stroke
Limited Sample Size
Even with complete adjustment for all important prognos-
tic factors, differences between hospitals may be caused by
chance alone. Small hospitals, by definition, provide fewer
data, and so the precision of model estimates is lower; esti-
mates are made with more uncertainty, which is reflected by
wider confidence intervals.35 The frequency of the outcome
being modeled also impacts relative precision. Outcomes such
as 30-day case fatality and 30-day readmission after stroke
are not that common; unadjusted 30-day event rates for both
mortality and readmission are typically around 12% to 15%
depending on the population examined.17,36,37 Small case vol-
umes in combination with low frequency of outcome events
have a significant negative impact on the ability to identify
hospitals as outliers (as is discussed in Hospital Profiling:
Identification of Outlier Hospitals). Statistical methods such
as hierarchical or multilevel regression models38 and empirical
Bayes methods3941 are increasingly being used to address the
problem of risk adjustment across small hospitals or for health
events that are infrequent. Consideration of small case volume
is important in risk-adjustment models for stroke, because
some studies have shown a relationship between stroke vol-
umes and outcomes.42,43
Quantification of Outcomes as a Graded Response
Functional outcome measures, such as the modified Rankin
Scale (mRS), are recorded on an ordinal scale. Several statisti-
cal approaches are available for the analysis of such data, such
as a binary logistic model that uses a fixed dichotomous cut
Katzan etal Ischemic Stroke Outcomes 5
point (eg, <2 versus 2), a sliding dichotomy (or responder
analysis), or an ordinal analysis.44 Each method has its own
strengths and limitations, and no one method is always the
preferred choice.45 The reader is referred to 2 recent summa-
ries for further information on this issue.44,45
Hospital Profiling: Identification of Outlier Hospitals
Hospital-specific risk standardized ratios are commonly used
to rank order hospitals from highest to lowest performers.
Many quality improvement organizations and payers such as
CMS also use hospital-specific risk-standardized estimates to
identify outlier hospitals. Outlier hospitals may be identified
simply according to relative rank (eg, top 10%, bottom 10%)
or by identifying hospitals that are statistically significantly
different from a given benchmark or standard (commonly the
overall average mortality or readmission rate in the population
being evaluated). It is important to appreciate that unlike an
assessment of the performance of risk-adjustment models at
the patient level, in which the final outcome of each patient
(eg, alive or dead, readmitted or not) is known, the true rank
order of hospitals that describes their relative performance is
not known. Because the designation of outlier status can have
important consequences for individual hospitals, the accuracy
of such designations has been of keen interest to researchers in
the field.28,46 Findings from this research are quite sobering. In
2 seminal simulation studies conducted in the 1990s,35,47 it was
found that even in the face of perfect case-mix adjustment and
very large variations in quality between hospitals, mortality
rates did a very poor job of identifying low-quality hospitals,
identifying only a small minority of poorly performing hospi-
tals and generating far more false-positive than true-positive
results. These studies also illustrated that the ability to iden-
tify outlier hospitals was strongly influenced by hospital case
volumes and the underlying mortality rates.35,47 Other reports
have demonstrated that the accuracy of hospital profiling can
be influenced by the statistical methods used (for example,
fixed versus random effects models,40,4850 Bayesian versus fre-
quentist methods,51,52 the choice of modeling strategy used to
identify outlier or extreme providers,47 and the type and qual-
ity of data available5355).
Thirty-Day Mortality
Strengths and Weaknesses of 30-Day Mortality as a
Measure of Hospital-Level Quality of Care in
Ischemic Stroke
There are several reasons to consider the use of mortality as a
measure of quality in ischemic stroke care. First, death is obvi-
ously an important outcome. Second, death is easily measured
and reliably assessed through administrative data, at little
expense. Third, death after stroke is not uncommon, occurring
in 5% to 7% of cases in the acute care hospital,56 in 13% to 15%
at 30 days,17,57 and in 25% to 30% at 1 year.17,57 Fourth, there
is variation in mortality rates between hospitals, which could
be related to the quality of care. The variation in hospital isch-
emic stroke mortality occurs despite adjustment for age, sex,
and comorbid conditions. For example, variations in hospital
mortality have been associated with hospital size, the timing of
admission,42,5861 and aggressiveness in end-of-life care,62,63 all
of which may be associated with the quality of care provided.
 6StrokeMarch 2014
In a study of Medicare ischemic stroke patients admitted to
625 hospitals participating in a quality improvement program,
hospital 30-day risk-adjusted standardized mortality varied by
>75% on a relative basis between the bottom tenth percen-
tile (9.8%) and the top tenth percentile (17.8%).17 Even after
adjustment for the NIHSS, similar variability was observed.
Fifth, better organized stroke care is associated with lower
mortality. For example, organized inpatient ischemic stroke
care lowered the odds of mortality by 14% compared with
routine care according to a systematic review of controlled
trials.64 An observational study showed an absolute mortality
reduction of 2.5% in state-certified stroke centers compared
with noncertified centers.65 Beyond such structural measures,
processes of care are also associated with improved mortality.
Aspirin use in the first 48 hours has an impact on longer-term
(eg, 6 months) mortality, albeit small,66,67 and there is some
evidence that the use of dysphagia screens, prophylaxis for
deep venous thrombosis, and treatment of hypoxia decreases
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017
the risk of death or discharge to hospice.68
There are also important limitations or challenges to the
use of mortality as a measure of the quality of care in isch-
emic stroke. First, the outcome of patients with stroke, includ-
ing mortality, is associated with the initial severity of the
stroke,56,66 age, and comorbidities, none of which are modifi-
able, although they can be adjusted for in risk models. Second,
the degree to which ischemic stroke mortality is modifiable by
changes in processes is unclear. The only medical or surgical
interventions proven by randomized controlled trials to lower
the risk of death in ischemic stroke are early provision of aspi-
rin, which has a minor effect at 6 months,67,68 and hemicrani-
ectomy for malignant cerebral edema, which has a large effect
but is applicable to only a small fraction of patients.69 The big-
gest advance in acute stroke treatment in the past 20 years is the
use of thrombolytic therapy, yet thrombolysis with tissue-type
plasminogen activator does not reduce mortality (although
it does prevent disability).10,70 Therefore, although organized
inpatient care appears to lower mortality, the mechanism by
which it does so is unclear. Third, 30-day mortality reflects not
only the care provided by the acute admitting hospital but also
the care provided immediately after acute discharge, includ-
ing rehabilitative care, although hospitals use of such services
and their transitions from inpatient to outpatient care can be
an important opportunity for quality improvement. Finally,
short-term stroke mortality may be more reflective of patient/
family preferences than the provision of evidence-based care.
An earlier death may be preferable to some patients and their
families over a delayed death or prolonged state of severe dis-
ability, depending on the patients life stage, functional and
cognitive status, treatment preferences, and desired place of
death.71,72 As a result, life-sustaining treatments are sometimes
withheld in accordance with patients preferences, which
leads to early death. Therefore, hospitals that practice opti-
mal shared decision making could have higher mortality rates,
caused by higher rates of appropriate use of palliative care,
than centers that are less sensitive to patients preferences.
Because the actual patient and family preferences and values
(and the process of eliciting them) may be difficult to cap-
ture, this can limit the interpretability of mortality compari-
sons, and it underscores the importance of a broader range of
processes and outcome measures, including functional status,
for use in quantifying the quality of stroke care.
Review of Evidence on Models That Predict Mortality After
Ischemic Stroke
The systematic review identified 1168 articles related to pre-
diction of mortality after ischemic stroke, of which 10 were
deemed to have the most relevant information on individual
predictors of risk of death (Table2).56,7381 Key features of the
most relevant models are presented in Table2. The reported
discrimination, as measured by the C statistic, varied from
0.72 to 0.86, with most models having C statistics 0.80,
which suggests that ischemic stroke mortality can be pre-
dicted at the patient level with a degree of accuracy that is
often considered clinically meaningful.82
Most of the published models have design limitations that
make them inadequate for the purpose of hospital-level isch-
emic stroke mortality risk adjustment, however. Most were
designed to predict an individuals risk of death after ischemic
stroke and did not report the impact of risk adjustment on
hospital-level mortality rates. Some models predicted hospital
discharge mortality56,74,75 rather than mortality at a fixed time
point, such as 30 days. Some used postadmission information
that could reflect the quality of care provided73,79,80 or included
patients in the subacute phase of stroke.73 Some models were
derived or validated in selected populations, such as patients
participating in clinical trials,76 or have not been validated in
new independent populations,74,75,79,80 which potentially limits
their generalizability. Only some models excluded transfer-in
patients,56,74,79 although the inclusion of transferred patients
could result in double counting of the same stroke.
Despite the heterogeneity in patient populations and meth-
ods of the previously published models (Table2), some consis-
tent strong predictors of death after ischemic stroke are evident
at the patient level. Measures of stroke severity, either by a
standardized stroke scale score such as the NIHSS or by clini-
cal examination features, were consistently retained in models
for which such information was available. Likewise, age was
consistently included. Medical comorbidities were included in
many models, although the exact comorbidities in each model
varied. In studies that compared the predictive value of vari-
ous types of predictors, stroke severity and age explained the
greatest variation in predicting mortality at the patient level56,77
and substantially increased the accuracy of the models.56,78 In
a study of in-hospital mortality,56 the effect of the NIHSS on
reclassifying risk was assessed by use of the integrated dis-
crimination index, which quantifies the difference between 2
models with and without the candidate predictor of interest,
in the predicted risk of death in case subjects (ie, those who
died) versus control subjects (those who survived). A higher
integrated discrimination index indicates improved discrimina-
tion of one model over another. The integrated discrimination
index for the addition of the NIHSS to the model for predic-
tion of in-hospital mortality was 9.4%,56 which indicates a sub-
stantial improvement in classification of risk.83 In a study of
Medicare beneficiaries, there was a graded, nearly linear rela-
tionship between higher NIHSS and higher 30-day mortality.84
The discrimination of risk of 30-day mortality by NIHSS alone
(C statistic=0.82) was better than that of a model that included
 Katzan etal Ischemic Stroke Outcomes 7

Table 2. Ischemic Stroke Mortality Prediction Models

Setting Sample Size
First (Derivation (Derivation), Mortality C Statistic
Author Year Sample) n Outcome Significant Prognostic Factors Validation (Validation)
Counsell 2002 UK population 530 30 d Age, living alone, independent before Yes (in 3 stroke 0.85
et al73 based, stroke, Glasgow Coma Scale verbal registries in
including score, ability to lift arms and ability to Scotland, Italy,
outpatients walk and Australia with
1868 patients)
Hinchey 1998 31 Cleveland 223 Discharge Altered mental status, limb paralysis, No Not reported
et al74 (OH) hospitals neurological examination findings,
laboratory findings, CT or MR findings,
vital signs, time of symptom onset,
intubation, ECG findings, DNR order
within first 3 d
Iezzoni 1996 94 US 9407 Discharge Compared different sets of predictors: Yes (internal only) Medical record: 0.86;
et al75 hospitals (1) Medical recordcollected age, physiology score: 0.84;
demographics, diagnoses, procedures, discharge summaries:
examination findings, vital signs, and 0.720.75
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017

laboratory values; (2) physiology score

based on vital signs; and (3) 3 vendor-
specific disease severity measures
based on discharge abstracts
Lewis 2008 Randomized 537 30 d External validation of study by Counsell Yes 0.73
et al76 controlled trial etal73
Saposnik 2011 12 Canadian 8223 30 d Age, Canadian Neurological Scale, sex, Yes (population- 0.79
et al77 hospitals stroke subtype according to OCSP based sample of
classification, atrial fibrillation, CHF, 3270 hospitalized
previous myocardial infarction, smoker, ischemic stroke
cancer, renal dialysis, preadmission patients from
dependency, serum glucose Ontario, Canada)
Smith 2010 1036 US 164993 Discharge Age, NIHSS, mode of arrival to hospital, Yes (109995 Including NIHSS (39.7%
et al56 hospitals sex, atrial fibrillation, previous stroke separate patients of patients): 0.85; not
or TIA, coronary artery disease, history at the same 1036 including NIHSS: 0.72
of carotid stenosis, hypertension, hospitals)
diabetes mellitus, smoking, history of
dyslipidemia, arrival during weekday
working hours
Tabak 2007 71 US acute 89129 30-d hospital Age, ICD-9 codes, vital signs, laboratory Yes (195 other Age+laboratory findings:
et al78 care hospitals risk-adjusted values, altered mental status US hospitals), 0.75; age+laboratory
mortality but results were findings+ICD-9 variables:
not reported 0.76; age+laboratory
separately except findings+ICD-9
that they were variables+altered mental
similar status: 0.84
Wang 2003 Single 253 1y Unconsciousness, dysphagia, urinary Yes (217 patients Not reported
et al79 Australian incontinence, both sides affected, from same
hospital hyperthermia, ischemic heart disease, hospital)
peripheral vascular disease, diabetes
mellitus
Weimar 2002 23 German 1754 100 d Age, NIHSS at admission, fever within 72 h No Not reported
et al80 stroke centers
Weimar 2004 7 German 1079 100 d Age, NIHSS Yes (13 German 0.83 (Derivation cohort)
et al81 stroke centers hospitals separate
from the derivation
cohort)
CHF indicates congestive heart failure; CT, computed tomography; DNR, do not resuscitate; ICD-9, International Classification of Diseases, 9th Revision; MR, magnetic
resonance; NIHSS, National Institutes of Health Stroke Scale; OCSP, Oxford Community Stroke Project; TIA, transient ischemic attack; and UK, United Kingdom.

age and all other medical variables except NIHSS (C statis- is less certain, with fewer data than for stroke severity, age,
tic=0.71).84 The role of laboratory values or physiological find- and medical comorbidities.74,75,77,78 A recent systematic review
ings such as blood pressure in predicting ischemic stroke death found that most existing studies of the relationship between
 8StrokeMarch 2014
blood markers and ischemic stroke were of relatively low qual-
ity, without a clear consensus on highly predictive variables.85
Critical issues in the consideration of risk models for mor-
tality after acute ischemic stroke are whether or not clinical
assessment of stroke severity is necessary and, if so, whether
it is feasible. The importance of these issues has been height-
ened by CMSs recent decision to publicly report stroke mor-
tality using administrative data for risk adjustment, which
does not include a measure of stroke severity. A mortality risk
model was developed by the Yale-New Haven Health System/
Center for Outcomes Research and Evaluation that included
42 variables from the Medicare claims files from the index
hospitalization and from principal and secondary diagnosis
codes from hospitalizations, institutional outpatient visits, and
physician encounters in the 12 months before the index hos-
pitalization. It also included a variable to account for transfers
from an outside emergency department.86 In cases of hos-
pital transfers after admission, mortality is attributed to the
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017
first hospital. The discrimination of the CMS model has been
compared with a medical recordbased model that included a
stroke severity scale developed by the National Stroke Project
that consisted of retrospective chart abstraction of the pres-
ence or absence of (1) vision changes, (2) speech deficit, (3)
motor deficit, and (4) sensory deficit.87 The C statistic was
0.71 for the CMS model and 0.80 for the comparison medi-
cal recordbased model. The CMS and medical recordbased
model predictions showed moderately good correlation (intra-
class correlation coefficient 0.75).88 Analysis of differences in
hospital rankings between the 2 models was not performed.
The National Quality Forum considered a performance
measure for ischemic stroke mortality using the CMS/Yale
risk-adjustment model; however the performance measure
was withdrawn before a decision on endorsement could be
provided. A separate measure for stroke in-hospital mortal-
ity from the Agency for Healthcare Research and Quality has
been approved by the National Quality Forum since 2008.89
Although adjustment for stroke severity is clearly the most
important individual prognostic factor for stroke, a debated
question is whether adjustment for stroke severity would
change the order of ranking of hospital mortality from low-
est to highest. If the distribution of severity scores at each
hospital is similar, then hospital risk-adjusted mortality rates
may not change much, although individual patient-level pre-
dictions would be improved. A study of VA hospitals found
that adjustment for NIHSS caused little shift in hospital risk-
adjusted mortality26; however, there was little variability in
hospital-specific risk-standardized mortality rates across VA
hospitals, which may have limited the ability to detect dif-
ferences with inclusion of the NIHSS in the models. Indeed,
extrapolation of VA study results to the US population would
suggest that the risk-standardized mortality rate for 30-day
mortality is not able to differentiate hospital performance
well; however, differences between the VA stroke population
and the overall stroke population or Medicare-served popula-
tion may limit the generalizability of the findings. The VA
population was predominantly male, and approximately half
had an NIHSS score 2, which reflects mild stroke. It also
had a very low 30-day mortality rate (5.4%)26 compared with
Medicare stroke patients (14.2%)17 or a population-based
study in Canada (11.6%),77 which probably reflects prefer-
ential triage of acute stroke patients to non-VA hospitals. A
recent study of hospitals participating in the AHA/American
Stroke Associations GWTG-Stroke quality initiative also
evaluated the impact of NIHSS on hospital performance
rankings.90 It included 782 hospitals and 127950 Medicare
beneficiaries with ischemic stroke who had a documented
NIHSS. In this analysis, hospital transfers after admission
were excluded, whereas the CMS analysis included hospi-
tal transfers but adjusted for them. There was a significant
reclassification of hospitals when NIHSS was used in the
model. Among 782 hospitals ranked into 3 groups (top 20%,
middle 60%, or bottom 20%) of performance by the claims
model without NIHSS scores, 26.3% (n=206) were ranked
differently by the model with NIHSS scores. When classified
on the basis of expected performance as defined by the CMS
Hospital Compare program (based on having a 95% credible
interval for the hospital-specific random effect that did not
include the null value of zero, representing the overall hospi-
tal average), the NIHSS model reclassified 45 of 782 hospi-
tals (5.75%), including 15 of 26 hospitals that were classified
as worse than expected in the model without NIHSS.
Although this report provides important information on the
impact of including the NIHSS in risk-adjustment models
for 30-day mortality, some unanswered questions remain.
A limitation of the GWTG study is that 55% of patients
were excluded because the NIHSS was not documented in
the clinical record, and it is unknown whether inclusion of
these patients would have changed the findings. Although the
GWTG model-building strategy was designed to be similar to
that used to derive models used by CMS for public reporting
of ischemic stroke, the models do differ in some respects:
There were more variables retained in the GWTG model than
the CMS model; the GWTG model was derived and validated
in nontransferred patients, whereas the CMS model included
transfer patients and adjusted for transfer as another variable;
and the GWTG study population was based on clinical diag-
noses confirmed by chart review, whereas the CMS model
was derived and validated on the basis of cases identified by
administrative billing codes. Therefore, although the GWTG
report90 provides useful information on the added value of the
NIHSS for risk adjustment, it cannot be considered a direct
validation of the model used by CMS for public reporting.
If stroke severity information were readily available at little
expense, then clearly it would be desirable to adjust for it;
however, standardized stroke scales are not included in admin-
istrative data and are not routinely recorded in clinical practice
at all hospitals. Even among those participating in the national
GWTG-Stroke quality improvement program, the NIHSS
was recorded in only 40% of ischemic stroke patients.56 The
feasibility of collecting the NIHSS or a simpler indicator of
stroke severity at all hospitals in the United States has not
been tested, and there has been no policy incentive to col-
lect it to date. A performance measure to document admis-
sion stroke severity by use of a designated stroke scale would
provide such motivation. In addition, we know little about the
consistency of the quality of information from NIHSS scores
across sites. In US practice, the NIHSS has become the pre-
ferred stroke scale and is recommended over other scales by

AHA/American Stroke Association guidelines.91 However,
other validated stroke scales exist.77 A simpler stroke scale,
such as the 3-item scale developed by Singer and colleagues,92
if validated, may be easier to implement nationally. The dis-
criminative power for mortality risk prediction and feasibility
in practice of these other scales compared with the NIHSS has
not been tested comprehensively.
Given the current lack of widely available stroke severity
information, an important question is whether a model without
a severity measure can be used as a reasonable surrogate for
a preferable model that includes a severity measure. Adequate
surrogacy can be operationalized by determining whether the
output of the model without a stroke severity measure pro-
vides comparable risk discrimination and classification of
hospital performance as a model that includes severity. There
was controversy within the writing committee, however, about
whether this has been demonstrated adequately to date.26,90
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017
Conclusions
Conclusions on the minimum requirements for risk-adjustment
data when mortality is used as an evaluation of stroke care
quality at the hospital level are as follows:
1. Given our review of data on 30-day mortality models
that primarily involved studies of predictors of 30-day
mortality at the patient level, the following is suggested
as a minimum list of variables that should be considered
for inclusion: age, sex, stroke severity, comorbid condi-
tions, and vascular risk factors.
2. Stroke severity is the most important prognostic factor
for individual patients and appears to be a significant
predictor in hospital-level performance. Inclusion in the
prediction model is therefore recommended, particu-
larly if it can be captured among all patients and reliably
recorded by all hospitals.
3. Implementation without a measure of stroke severity will
increase the occurrence of hospital misclassification.
4. Risk-adjustment models of stroke mortality that do not
include stroke severity or other recommended variables
must provide comparable classification of hospital per-
formance as a model that includes stroke severity or
other missing variables.
5. Stroke severity and other variables that are used in
hospital-level risk-adjustment models of mortality after
ischemic stroke should be standardized across sites so
that their reliability and accuracy are equivalent.
The following represent needs for further research with
mortality used as an indicator of hospital quality, in order
of priority:
1. Research is urgently needed on the impact of hospital-
level risk adjustment of stroke severity, comorbid con-
ditions, and patient demographics on ischemic stroke
mortality rates and their influence on hospital ranking by
mortality, as well as the implications of misclassification
for patients and hospitals, especially referral centers.
2. Research is needed to evaluate the impact of missing
stroke severity variables on overall hospital performance.
3. More research is required to assess methods for more
reproducibly and feasibly reporting stroke severity,
Katzan etal Ischemic Stroke Outcomes 9
or a validated close surrogate, in all ischemic stroke
patients, suitable for incorporation into risk-adjustment
models. For example, research is needed on alternative
approaches to identifying stroke severity, such as using
automated algorithms to search standardized clinical
data stored in electronic health records.
4. More research is needed to identify hospital structures/
organizations and processes of care that are associated
with lower mortality. These processes could then be the
targets for focused quality improvement interventions to
reduce mortality.
5. Future work should be directed toward understanding
how hospital and physician practices related to end-of-
life treatment decisions impact 30-day stroke mortality.
Ideally, methods for the capture of patient (not physi-
cian/hospital) preferences should be developed so that
this information can be taken into consideration when
hospital risk-adjusted mortality rates are calculated.
6. More research is needed on how differences in stroke
systems of care, such as the capacity of providers and
the beds available within a community (home, nursing,
hospice), can impact 30-day stroke mortality.
7. More research is needed on the descriptive epidemiology
of how stroke patients are actually dying in real world
contemporary settings, with efforts to develop a typol-
ogy aligned with measuring and improving the quality
of care and identifying avoidable deaths.
Thirty-Day All-Cause Readmissions
Strengths and Weaknesses of Using 30-Day Readmission as
an Outcome for Evaluation of Hospital Quality of Care
There are 4 key strengths to the use of 30-day all-cause read-
mission as a measure of hospital quality of care. First, it is
patient-centered in that patients who experience this outcome
incur the disruption, risk, and indirect (and sometimes direct)
costs of the hospitalization and the clinical events that led
to it. Second, as an outcome, readmission may serve as the
end result of many distinct processes of care (some known
and some unknown) that collectively combine to enhance
or undermine a successful transition from hospital to post-
hospital care. Third, it is presumed, but unproven, that many
readmissions could be prevented if care were improved.
Hospital care involves the risks associated with multiple
handoffs between providers and, commonly, discontinuity
with outpatient providers. Research has shown that readmis-
sion rates for some diseases can be influenced by the qual-
ity of inpatient and outpatient care and that improvement in
care coordination can reduce readmission rates.9397 A number
of studies, many of them in patients with heart failure, have
demonstrated that improvements in care at the time of patient
discharge can reduce 30-day readmission rates.9397 Finally,
readmissions are costly, and a reduction in these events would
not only enhance the patient experience but could also reduce
healthcare spending. Estimates suggest that potentially pre-
ventable readmissions may cost Medicare $12 billion annu-
ally.98 Interventions to reduce readmissions have been shown
to decrease costs. For example, in a randomized study at
an urban, academic, safety-net hospital, the intervention
group that received discharge coordination, education, and
 10StrokeMarch 2014
f ollow-up services had lower hospital utilization rates within
30 days of the index discharge than the control group that
received usual discharge care, resulting in 33.9% lower costs
for the intervention group.94
Balancing these 4 factors that support the use of read-
mission rates to evaluate hospital performance are 3 key
limitations. First, not all readmissions are preventable,
and so the goal will never be zero, but the premise is that
the current rate includes many readmissions that could
be prevented by improving care. Because it is difficult to
determine the preventability of a specific readmission, it
is typical to use a measure of all-cause readmission rather
than a more specific metric of preventable readmission.
Measurement of all-cause readmission should exclude
planned or elective readmissions, which do not reflect on the
quality of care. The focus on all-cause readmissions, rather
than condition-specific readmissions, is valuable because
it broadens the focus from efforts to improve a narrow set
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017
of approaches (such as processes that will prevent recurrent
stroke) to more encompassing initiatives (such as medica-
tion reconciliation, patient education, follow-up care, and
communication between inpatient and outpatient providers)
aimed at improving the overall care within the hospital and
transitions from the hospital setting. The second limitation
is that readmissions are attributed to the hospital, but the
hospital may have limited control of care after discharge.
The presumption is that the hospital can exert influence on
its environment and that there is shared responsibility for
postdischarge care, an advantage of restricting the assess-
ments of readmission to only 30 days. Third, a readmission
measure could provide an undesirable incentive to deny a
patient a needed admission by reducing access for patients,
which ultimately could result in a worse outcome.
Time Frame and Exclusions
A readmission is defined as a readmission within 30 days of
the day of discharge from the index ischemic stroke hospital-
ization to any hospital. Therefore, if a patient is discharged
from hospital A and readmitted within 30 days of discharge
to hospital B, then this readmission should be assigned to
hospital A.
Patients with the following characteristics should be
excluded from the calculation of the readmission measure:
those who died during the index hospitalization, those who
left the index hospitalization against medical advice, patients
discharged to hospice, and those who were transferred to
another acute care facility (the final discharging facility is
the one that would be eligible for the calculation of a read-
mission measure for patients who are cared for by more than
one hospital). Moreover, within a given data set, if a patient
has 2 hospitalizations for ischemic stroke within a 30-day
period, the first hospitalization should be considered the
index event, and the second hospitalization should be con-
sidered the readmission.
The approach to exclusions of specific readmissions
reflects what has been proposed by CMS. Readmissions
that are attributable to an expected and appropriate conse-
quence of the index stroke admission should not be included
as a readmission for the purpose of evaluating hospital care
quality. For example, planned readmissions for carotid end-
arterectomy or carotid stenting should not be included in
the readmission outcome measure, unless the procedure
was performed after a patient presented with a recurrent
stroke or other acute condition. In contrast, patients with
stroke are often readmitted for pneumonia or other pulmo-
nary reasons, which would be included as a readmission.37,99
Admissions to an inpatient rehabilitation facility after dis-
charge from an index stroke hospitalization are not included
as a readmission for the purpose of this quality measure;
however, hospitalizations that occur from the inpatient reha-
bilitation facility back to an acute hospital would be counted
as a readmission.
Another potential approach focuses on potentially prevent-
able readmissions,100 which include only readmissions that fall
into one of several categories believed to be potentially pre-
ventable, such as complications plausibly related to care dur-
ing admission or continuation of reason for initial admission.
All patient refined diagnosis-related groups are used to clas-
sify reasons for hospitalizations and the relationship between
admission and readmission.101 The use of this method to filter
readmissions for inclusion in public reporting calculations
requires systematic evaluation.
Review of Evidence on Models That Predict Readmissions
After Ischemic Stroke
Lichtman etal102 conducted a systematic review of the litera-
ture about risk-adjustment models for the prediction of stroke
readmissions (Table3). Although no studies were identified
that reported a hospital-level risk-adjustment system, 16 pub-
lications were identified that reported risk-adjustment models
at the patient level.98,103117 This systematic review describes
specific patient characteristics that have been associated with
stroke readmissions at the patient level, including age, lon-
ger index hospital length of stay, worse physical functioning
after stroke, and increased number of hospitalizations before
stroke.102 It would not be appropriate to include length of stay
of the index hospitalization in risk-adjustment models built
for the purpose of evaluating hospital care quality given that
longer lengths of stay may be associated with poorer quality
stroke care.118
Many published patient-level readmission risk-adjustment
models include a measure of stroke severity.102 Few pub-
lished data are available to inform the discussion regarding
the importance of including a measure of stroke severity in
risk-adjustment models used to evaluate outcomes at the
hospital level (as opposed to the patient level). In an analy-
sis reported by Fonarow etal17 of 33102 patients cared for
at 1 of 404 hospitals that participate in the AHA/American
Stroke Associations GWTG-Stroke program, 30-day read-
mission rates were similar before and after adjustment for
stroke severity with the NIHSS. The unadjusted 30-day hospi-
tal readmission rates were as follows: 10th percentile, 5.9%;
90th percentile, 22.9%. The rates adjusted for NIHSS and
other patient characteristics were 5.6% and 18.9%, respec-
tively. Therefore, considerable variation in readmission rates
across hospitals was observed both with and without adjust-
ment for NIHSS. However, information about how measures
of model performance differed for models that included all
 Katzan etal Ischemic Stroke Outcomes 11

Table 3. Characteristics of Publications Examining Predictors of Readmission After Stroke Hospitalization

Unadjusted
Data Source No. of Hospitals/ Follow-up Readmission Analytical
Study (Study Period) Study Location No. of Patients Stroke Type Study Outcome Period Rate Model
Bohannon and Hospital administrative United States 1/326 Ischemic Same-hospital 1y 32.5% Logistic
Lee, 200398 data, stroke center (Connecticut) all-cause regression
specific database readmission
(20002001)
Bohannon and Hospital administrative United States 1/228 Ischemic Same-hospital 1y 37.3% Logistic
Lee, 2004103 data, medical chart (Connecticut) all-cause regression
abstraction, patient readmission
interviews
(20002001)
Camberg VA hospital United States Multiple (No. not Ischemic or All-cause 30 d, 6 16.6% (30 d), Proportional
et al, 1997104* administrative data, specified)/2261 hemorrhagic readmission mo, 1 y 43.8% (6 mo), hazards
Medicare data 58.8% (1 y) regression
(19881990)
Chuang Patient interviews Taiwan (Taipei) 7/489 Ischemic or All-cause 30 d 24.3% Logistic
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017

et al, 2005105 (19992000) hemorrhagic readmission regression

Heller et al, Hospital patient Australia (New 22/1075 Ischemic or Unplanned 1y 13% Logistic
2000106 administrative system, South Wales) hemorrhagic stroke-related regression
hospital separation readmission
database
(19951997)
Jia et al, VA hospital United States Multiple (No. not Ischemic or All-cause 1y 62.2% (all- Logistic
2007107 administrative data, (Florida) specified)/1818 hemorrhagic readmission, cause), 31.1% regression
Medicare data, stroke (stroke)
Medicaid data readmission
(20002001)
Johansen et Canadian Health Canada Multiple (No. not Ischemic or All-cause 1y 37.1% Logistic
al, 2006108 Person-Oriented specified)/2448 hemorrhagic readmission regression
Information Database
(19992001)
Kane et al, Patient interviews, United States 52/487 Stroke (type All-cause 1y 30% Instrumental
1998109 medical record unspecified) readmission variables
abstraction estimation
(19881989)
Kennedy, State database of United States Multiple (No. not Ischemic or No. of stroke 1y 12% Truncated
2005110 hospital administrative (California) specified)/38468 hemorrhagic events negative
data (2000) binomial
regression
Lichtman Medicare data (2002) United States 5070/366551 Ischemic All-cause 30 d 14.5% Cox
et al, 2009111 readmission, (all-cause), proportional
stroke-related 7.8% (stroke- hazards
readmission related) regression
Lindenauer et National hospital United States 45/9295 Ischemic All-cause 14 d Data not Generalized
al, 2007112 database readmission provided estimating
(20022005) equations
Roe et al, Medical record Australia 1/264 Stroke (type Same-hospital 30 d 6.5% Log-linear
1996113 abstraction unspecified) unplanned analysis
(19901992) all-cause
readmission
Smith et al, HMO administrative United States 422/9003 Ischemic All-cause 30 d, 1 y 14.0% (30 Cox
2005114 data and Medicare/ (primarily readmission, (for 30 d d all-cause), proportional
Medicaid data eastern half, 93 stroke survivors) 40.6% (1 y all- hazards
(19982000) metropolitan readmission cause), 9.4% regression
counties) (30 d stroke)
Smith et al, HMO administrative United Multiple (No. not Ischemic All-cause 30 d 12.9% (all- Cox
2006115 data, Medicare data, States (11 specified)/44099 readmission, cause), 7.4% proportional
and Medicaid data metropolitan stroke (stroke) hazards
(19982000) regions) readmission regression
(Continued)
 12StrokeMarch 2014

Table 3. Continued
Unadjusted
Data Source (Study No. of Hospitals/ Follow-up Readmission Analytical
Study Period) Study Location No. of Patients Stroke Type Study Outcome Period Rate Model

Sun and Toh, National Healthcare Singapore 3/6464 Ischemic All-cause 1y 37.4% (all- Cox
2009116 Group administrative readmission, cause), 10.5% proportional
database stroke (stroke) hazards
(20052007) readmission regression
Tseng and National Health Taiwan Multiple (No. not Ischemic, All-cause 1y 50% Logistic
Lin, 2009117 Insurance Research specified)/468 hemorrhagic, readmission regression
Database TIA, ill-
(20002001) defined, or
late effects
HMO indicates health maintenance organization; TIA, transient ischemic attack; and VA, Veterans Administration.
*Authors only report risk-adjusted models at 30 d.
Stroke-related readmission represents a composite outcome that includes stroke. Heller et al, 2000106: readmission for stroke or other cardiovascular disease;
Lichtman et al, 2009111: readmission for stroke or other related complications.
Rates calculated from data provided in the article.
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017

All-cause readmission defined as readmission for any cause other than diabetes mellitus and diabetes-related complications.
Reproduced with permission from Lichtman etal.102 2010 American Heart Association, Inc.

risk-adjustment variables with versus without the NIHSS
were not provided.
Minimum Standards for Model Discrimination and
Calibration for Models of 30-Day Readmission for
Evaluation of Hospital Quality of Care
Few studies have explicitly reported the performance char-
acteristics of models to predict short-term readmission after
stroke at the patient level. Model discrimination for 30-day
readmission reported by Fonarow etal17 was modest (C statis-
tic=0.59). By comparison, model discrimination for 30-day
mortality was significantly better (C statistic=0.70). Model
calibration was not reported. Current models used in the
public assessment of hospital quality for 30-day readmis-
sion after discharge for conditions such as myocardial infarc-
tion and heart failure have similar levels of discrimination
(C statistic=0.63 for readmission after myocardial infarction
discharge, and C statistic=0.60 for readmission after heart
failure discharge).119,120
Although the literature is scant, it is likely that future mod-
els predicting 30-day readmission used for the assessment of
hospital quality will also have marginal discriminatory abil-
ity. This relatively poor model performance reflects the reality
that readmission risk is driven by multiple factors, many of
which are unknown, not measured, or occur only after dis-
charge. Some of these factors are likely related to intrinsic
patient characteristics that affect readmission, whereas others
may include hospital processes that reflect quality of care.
Conclusions
Conclusions regarding the minimum requirements for risk-
adjustment data when measuring stroke readmissions for the
evaluation of stroke care quality at the hospital level are as
follows:
1. Data on the predictors of 30-day readmission for evalu-
ation of quality at a hospital level are limited. On the
basis of the review of available data, which involved pri-
marily studies of models designed for patient-level pre-
diction, the inclusion of the following minimum list of
variables is probably indicated: age, sex, stroke severity,
comorbid conditions, and risk factors. Assessment for
significance in models that evaluate performance at the
hospital level should be performed.
2. As discussed previously, stroke severity appears to be
a significant predictor in hospital-level performance of
mortality after stroke, although no data are available
for hospital-level performance of 30-day readmissions
after ischemic stroke. Using the same rationale as for the
outcome mortality, inclusion in the prediction model is
recommended.
3. Risk-adjustment models of readmission after stroke that
do not include stroke severity or other recommended
variables must provide comparable classification of hos-
pital performance as a model that includes stroke sever-
ity or other missing variables.
4. Stroke severity and other variables that are used in
hospital-level risk-adjustment models of readmissions
after ischemic stroke should be standardized across sites
so that their reliability and accuracy are equivalent.
Suggestions for future research and applications in models
of 30-day readmissions are as follows:
1. We conclude that to improve the utility of risk-adjusted
models, future researchers should seek to identify fac-
tors that (1) are not typically measured in administrative
data sources, (2) predict hospital readmission, (3) are not
related to hospital quality, and (4) are unevenly distrib-
uted among hospitals. These factors may be important
opportunities to further improve risk adjustment for
30-day readmissions.
2. Future work should be directed toward understanding how
patient preferences are related to poststroke readmission
(eg, preferences favoring discharge to home with visiting
nurse support versus discharge to an inpatient rehabilita-
tion facility versus discharge to a skilled nursing facility
versus preference for hospice care), whether data about
patient preferences should be included in models, and if
so, how these data can be included in administrative data.

Thirty-Day Functional Status
Strengths and Weaknesses of Functional Status as
a Measure of Hospital-Level Quality of Care in
Ischemic Stroke
There is both support for and challenges with adopting
functional status as a measure of the quality of acute care
for patients with ischemic stroke. Global disability-adjusted
life years lost because of stroke are projected to grow from
38 million in 1990 to 61 million in 2020.22 In the United
States, stroke is a leading cause of major long-term disability
among adults. In fact, patients are much more likely to be
left with disability than to die of their stroke. An estimated 2
million stroke survivors in the United States currently cope
with permanent stroke-related disabilities,1 and between 25%
and 74% require some assistance or are fully dependent on
caregivers for activities of daily living.121 Functional status
after stroke is a central issue to patients and their families
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017
and has a powerful impact on their lives. Assessment of func-
tional outcomes, therefore, has very high face validity and
is of high relevance to multiple groups, including almost all
patients and employers, health insurers, the local community,
and healthcare providers. Importantly, strong evidence exists
for the impact of care on functional outcomes.10,64,69,122 Most
therapies for acute stroke are aimed at minimizing impair-
ment after stroke and improving functional outcomes, and
only through the measurement of functional outcomes can
institutions that do a better job of providing these therapies be
recognized for their accomplishments. In addition, functional
measures have much greater sensitivity for the detection of
differences between patient groups or changes within patient
groups than patients vital status.
The greatest challenge with the measurement of functional
status is the ability to assess it reliably and systematically
across stroke patients in different settings. First, in clinical
practice, functional status is currently not uniformly mea-
sured in the United States across different healthcare settings.
There is no standard functional status assessment for patients
in acute care. The Functional Independence Measure (FIM)
is used in inpatient rehabilitation facilities and acute hospital
rehabilitation units, the Outcome and Assessment Information
Set (OASIS) is used in home health care, and the Minimum
Data Set (MDS) is used in skilled nursing facilities. Each
uses different scales and assesses different bodily functions,
activities, and areas of involvement.123 These setting-specific
measures and other instruments designed to measure a more
basic level of function are also limited by either floor or
ceiling effects.124,125 To improve the sensitivity to detect dif-
ferences between populations provided by these scales and
to provide a systematic method for obtaining this informa-
tion, the CMS is working to implement an instrument that
can be used across different healthcare settings.126 A second
limitation is the potential costs associated with the adoption
of functional status as an indicator of acute healthcare qual-
ity, particularly for assessment of postdischarge functioning.
Because patients with ischemic stroke have variable pat-
terns of care after the acute hospitalization,127 the introduc-
tion of a standard measure at a specific time after the event
creates opportunities to measure and improve care, but it
Katzan etal Ischemic Stroke Outcomes 13
also introduces challenges, including locating the patient and
obtaining a valid proxy response if it is not possible to obtain
information directly from the patient. A final limitation is that
30-day functional status may provide an incomplete insight
into a patients ultimate recovery, because patient function-
ing can improve after 30 days. Moreover, this recovery is not
solely under the locus of control of the acute hospital, because
the care provided after acute discharge, including rehabilita-
tive care, can improve outcomes. Approximately 50% to 70%
of stroke patients in the United States are discharged from
acute care to receive some form of postacute rehabilitative or
nursing care,128,129 so the assessment of functional status after
hospital discharge is likely to include the influence of factors
not associated with the acute inpatient stay. In other countries
where hospital lengths of stay are longer, rehabilitation may
be included in the initial hospitalization.130 These challenges
are great but not insurmountable relative to the importance of
measuring an outcome meaningful to patients, families, health
systems, and society.
Definitions of Functional Status After Ischemic Stroke
For the purposes of this report, we adopted the definitions of
the World Health Organizations International Classification
of Functioning, Disability and Health.11,24 Functioning is
an umbrella term for physiological functions of body sys-
tems (eg, neuromusculoskeletal, sensory, mental functions),
activities (execution of a task or action), and participation
(involvement in a life situation). Disability is an umbrella
term for problems, loss or significant deviation in body func-
tions (eg, visual, cognitive, speech impairments), activity
limitations, and participation restrictions (problems experi-
enced in involvement).13 In adopting a more global definition
of function that includes both functioning and disability, we
were able to examine a broader selection of objective and sub-
jective measures for their applicability as indicators of quality
of acute stroke care.
Review of Evidence on Models That Evaluate Predictors of
Functional Outcomes After Ischemic Stroke
Of the 706 articles reviewed that related to functional out-
comes after ischemic stroke, 19 studies that included 23
analytic models were identified as most relevant to explain-
ing functional outcomes after ischemic stroke (Table4).131149
Among the 19 studies that examined functional status at 30
or 90 days, none were designed to evaluate the quality of
care at the hospital level or predict outcomes at the provider
level. Each study examined functional status outcomes at the
patient level, with 13 studies specifying an independent vari-
able. Of those, 7 examined the relationship of specific patient
characteristics (including history of diabetes mellitus, serum
glucose levels on admission, pretreatment systolic blood pres-
sure, pretreatment Alberta Stroke Program Early CT score
[ASPECTS], pretreatment antiplatelet therapy, white matter
hyperintensities, gene variants, brachial artery flow-mediated
dilation, and posterior circulation infarction), and 6 examined
hospital interventions (thrombolytic therapy, tissue-type plas-
minogen activatorinduced recanalization, intravenous tissue-
type plasminogen activator, and acute statin therapy).
Studies were conducted in 18 countries with sample sizes
ranging from 120 to 4390 (Table3). The NIHSS was the most
 14StrokeMarch 2014

Table 4. Characteristics of Studies That Examined Functional Status

Stroke
No. of Patients/ Severity Outcome Favorable (Poor) Timing Data Source for Outcome
Study Study Location No. Analyzed Measure (Instrument) Outcome (Actual) Assessment
Denti et al, 2010131 Italy 1555/1549 GCS, OCSP, Rankin Scale mRS 02 (35) 30 d In-person in clinic or telephone
SSS, loss of interview with patient or
trunk control, caregiver
indwelling
catheter
Di Carlo et al, England, France, 2472/1983 NR Rankin Scale mRS 01 (25) 90 d In-person interview with patient
2006132 Germany, or proxy
Hungary, Italy,
Portugal, Spain
Finocchi et al, Italy 351/308 GCS, limb Oxford ODS 02 (35) 30 d NR
1996133 paresis Disability Scale
classification
German Stroke Germany 1470/1357 NIHSS Barthel Index BI 95100 (90) 90 d Telephone interview
Study Collaboration, (100)
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017

2004134
Kaarisalo et al, Finland 4390/3616 NR Categorical Totally or partially 30 d Medical record
2005135 description independent (28)
(completely
dependent)
Lin et al, 2011136 China 2683/2166 NIHSS Rankin Scale mRS 02 (35) 90 d Telephone interview or letter
questionnaire
Liou et al, 2010137 Taiwan 187/187 NIHSS Barthel Index, BI 95100 (90), 30 d In-person clinic visit or telephone
Rankin Scale mRS 01 (25) for severely disabled patients
Maguire et al, Australia 640/520 NIHSS Barthel Index, BI 95100 (90), 90 d Telephone follow-up primarily
2011138 Rankin Scale, mRS 02 (35),
Glasgow GOS 12 (35)
Outcome Scale
McCarron et al, Scotland 189/152 NIHSS, OCSP Barthel Index, BI >55 (55), mRS 90 d Observed prospectively
2000139 Rankin Scale 02 (35)
Ni Chroinin et al, Ireland 448/441 NIHSS Rankin Scale mRS 02 (35) 90 d In-person or telephone interview
2011140 by trained staff, medical record
as required
Ntaios et al, Switzerland 1446/NR NIHSS Rankin Scale mRS 02 (35) 90 d In-person in clinic by Rankin-
2010141 certified personnel
Rundek et al, Canada, United 1604/1264 NIHSS Barthel Index BI 95100 (90) 90 d NR
2004142 States
Santos-Garca Spain 120/120 NIHSS Rankin Scale mRS 02 (35) 90 d NR
et al, 2009143
Sharma et al, Singapore 130/114 NIHSS Rankin Scale mRS 01 (25) 90 d NR
2010144
Tao et al, 2010145 China 1962/1797 NIHSS Rankin Scale mRS 02 (35) 90 d NR
Tsivgoulis Canada, Spain, 351/292 NIHSS Rankin Scale mRS 02 (35) 90 d Medical records
et al, 2007146 United States
Tsivgoulis Canada, Spain, 192/160 NIHSS Rankin Scale mRS 02 (35) 90 d Medical records
et al, 2008147 United States
Uyttenboogaart The Netherlands 301/301 NIHSS Rankin Scale mRS 02 (35) 90 d Medical records
et al, 2008148
Weimar et al, Germany 4264/2458 NIHSS Barthel Index, BI 95100 (90), 90 d Telephone interview by
2002149 Rankin Scale mRS 01 (25) (100) trained interviewers or mail
questionnaire if not reached
BI indicates Barthel Index; GCS, Glasgow Coma Scale; GOS, Glasgow Outcome Scale; mRS, modified Rankin Scale; NIHSS, National Institutes of Health Stroke Scale;
NR, not reported; OCSP, Oxford Community Stroke Project; ODS, Oxford Disability Scale; and SSS, Scandinavian Stroke Scale.

common measure of stroke severity (78.9% of studies) at as mRS because not all studies indicated that the modified
presentation. Functional status at 30 and 90 days was mea- version was used, even when it was used), Oxford Disability
sured with the Barthel Index (BI), Rankin Scale (abbreviated Scale (known as Oxford or Oxfordshire Handicap Scale), and

an investigator-derived 3-level categorical measure. The BI is
a cumulative score in multiples of 5, with a range of 0 for
completely dependent to 100 for independence in 10 basic
activities of daily living with varying weights.101 The mRS is
used by clinicians to quantify 6 levels of functional recovery
(from a score of 0 [independent] to 5 for patients who require
constant care). The Oxfordshire Handicap Scale is an adapta-
tion of the mRS used to quantify handicap in 6 levels (score
of 05) as it relates to an individuals lifestyle and ability to
live independently.150
There are no existing recommendations for how to best ana-
lyze the entire range of data collected with any one of these
scales. A review of stroke outcome measures from 2000 found
most ordinal scales were dichotomized, and there was little
agreement in what was considered a favorable or positive
outcome.9 Among the 4 measures of functional status in this
review, a favorable outcome was analyzed 6 different ways at 2
time points: 5 different ways at 30 days (BI 95100, mRS 01,
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017
mRS 02, Oxfordshire Handicap Scale 02, and totally or
partially independent) and 4 different ways at 90 days (mRS
01, mRS 02, BI 60100, and BI 95100). The mRS was
used as an outcome assessment in 15 of the 23 analytic mod-
els reviewed. Five studies did not report the source of infor-
mation for functional status as an outcome; of those that did,
4 reported the mRS, BI, and investigator-derived scale came
from medical records; 6 gathered information with patients
in person; and 7 obtained the information by telephone. Only
3 studies specified whether the patient was the only source
of information or whether a proxy provided information to
evaluate function as the outcome.
The construction of the risk models varied substantially
across studies. All 19 studies categorized the outcome into 2
levels and used logistic regression to analyze predictors of func-
tion after stroke. Four studies presented 5 analytical models for
functional status at 30 days (Table5), and 15 studies presented
18 analytical models of functional status at 90 days (Tables6
and 7). There were 82 unique variables analyzed, and only 8
were examined in >50% of the models (age, sex, history of
atrial fibrillation, history of diabetes mellitus, history of hyper-
tension, current smoker, prestroke level of disability, and stroke
severity). At 30 days, at least half of the models examined age,
sex, history of diabetes mellitus, and stroke severity by NIHSS
or the Scandinavian Stroke Scale. Findings were similar for
studies that examined function at 90 days, with >50% of the
models including age; sex; history of diabetes mellitus, atrial
fibrillation, and hypertension; a prestroke mRS; and the NIHSS.
Every study examined medical history or presence of comorbid
conditions, but there was no standard set of conditions. Three
studies (16%) included a process measure (onset to treatment
time), and none of the models included hospital characteristics
or examined differences in outcomes acrosshospitals.
We were unable to identify a study that evaluated 30- or
90-day functional status outcomes for ischemic stroke patients
and the relationship with acute stroke care at the provider or
hospital level.
Choice of Functional Status Measure
The BI and mRS were the most commonly used scales in the
literature. In contrast to the BI, which is widely used in acute
Katzan etal Ischemic Stroke Outcomes 15
trials, the mRS is more responsive to change, has not demon-
strated ceiling effects,149 and provides for assessment of the
outcome measure of participation. Variable interrater reliabil-
ity is a potential weakness of the mRS that can be improved
through the use of structured interviewing and scoring of rat-
ers that is blinded to the treatment provided.151,152
Our synthesis of the literature supports findings from previ-
ous reviews that the most common approach is to dichotomize
the mRS for analysis. However, there is not a consistent cut
point for such dichotomization, where some use 1 and other
use 2 as indicating a favorable outcome. The interpretation
of the former is complete independence and the latter is no
significant disability. At this time, the lack of a universally
accepted level for dichotomization is problematic for recom-
mending an outcome for public reporting on hospitals quality
of stroke care.
A study of poststroke recovery by Duncan and colleagues153
highlights the challenges in arbitrarily setting a time point for
assessment and a cutoff for recovery using the mRS. They
showed that assessment of function using the mRS within the
first 14 days of the stroke identified 1.74% of the sample as
recovered when analyzed as mRS=0 to 1, and 12.2% were
recovered when an mRS of 0 to 2 was used. At 30 days, 5.5%
and 21.6%, respectively, were recovered, and by 6 months,
24.4% and 54.8% had attained mRS=0 to 1 and mRS=0 to 2
levels of function.
A greater concern is that by dichotomizing scales for the
assessment of hospital quality, it is possible that important dif-
ferences in care may be obscured. For example, if 2 hospitals
were to treat an identical patient and the patient in the first
hospital achieved an mRS of 5 (requires constant care) and
the patient in the second hospital achieved an mRS score of
3 (needs assistance with instrumental but not basic activities
of daily living), this difference would be obscured by either
dichotomization scheme. Recently, several experts have
examined approaches that use the range of data, such as ordi-
nal or sliding dichotomous, depending on the study design and
sample size.44,45 These methods, however, are not yet widely
used and may be sensitive to the distribution of specific data
sets, and the results may be more difficult to interpret.45,154
Conclusions
Conclusions regarding the instrument and measurement of
functional status at 30 days for the evaluation of stroke care
quality at the hospital level are as follows:
1. With full acknowledgment of the limitations of making
a firm recommendation, the use of the mRS as a global
measure of functional status, administered by a trained
mRS rater, may be considered to assess function at
30days.
2. Ideally, dichotomization of outcomes should be avoided;
however, until additional research can help determine the
best approach, the assessment of hospital quality with an
mRS score of 0 to 2 at 30 days as a conservative estimate
of good outcome may be considered, with the knowl-
edge that many patients continue to improve over time.
Conclusions regarding the minimum requirements for risk-
adjustment data when measuring functional status at 30 days
 16StrokeMarch 2014

Table 5. Significance of Variables Associated With Functional Status at 30 Days

Global Disability
(Independent vs
Mild, Severe): ODS 02:
BI 95100: mRS 01: mRS 02: Kaarisalo etal, Finocchi etal,
Candidate Variable Liou etal, 2010137 Liou etal, 2010137 Denti etal, 2010131 2005135 1996133
Sociodemographics
Age S S S S S
Sex NS NS NS S
Comorbidities/risk factors
Prior TIA S
Prior MI NS
CHD NS
Atrial fibrillation S
Diabetes mellitus NS NS S
Hypertension S S
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017

Hyperlipidemia NS NS
Smoking NS NS
Prestroke status
Premorbid mRS S
Stroke severity
GCS NS NS
NIHSS or SSS S S S
Loss of trunk control S
Severity of hemiparesis S
1 Medical complication
 S
Indwelling catheter S
Stroke origin, type
OCSP subtypes NS
Imaging characteristics
Imaging procedure for brain S
vessels
Infarct size S
Infarct volume NS NS
White matter hyperintensities S S
Other
Glycemia NS
Arterial BP NS
EEG alterations S
ECG alterations NS
BI indicates Barthel Index; BP, blood pressure; CHD, coronary heart disease; EEG, electroencephalogram; ellipses (. . .), data
not available; GCS, Glasgow Coma Scale; MI, myocardial infarction; mRS, modified Rankin Scale; NIHSS, National Institutes of
Health Stroke Scale; NS, not significant; OCSP, Oxford Community Stroke Project; ODS, Oxford Disability Scale; S, significant; SSS,
Scandinavian Stroke Scale; and TIA, transient ischemic attack.

for the evaluation of stroke care quality at the hospital level
are as follows:
1. The following minimum list of variables should be
included in risk-adjustment models: age, sex, stroke
severity, comorbid conditions and risk factors (including
prior stroke or transient ischemic attack, prior myocar-
dial infarction, coronary artery disease, atrial fibrilla-
tion, diabetes mellitus, hypertension, hyperlipidemia,
smoking, and alcohol use), prestroke physical function
(most commonly measured with the mRS), stroke sever-
ity, and stroke type. Assessment for significance in mod-
els that evaluate performance at the hospital level should
be performed.
2. Premorbid functioning and stroke severity are not col-
lected in a uniform fashion in patients with stroke. They
appear, however, to be significant predictors of functional
outcomes at 30 days in patient-level performance of
mortality after stroke, although no data are available for
hospital-level performance of functional outcomes after
 Katzan etal Ischemic Stroke Outcomes 17

Table 6. Significance of Variables Associated With Functional Status as Measured by the Rankin Scale at 90 Days
mRS 01 mRS 02
Di Carlo Sharma Weimar Lin Maguire McCarron Ni Chroinin Ntaios Santos-Garca Tao Tsivgoulis Tsivgoulis Uyttenboogaart
et al, et al, et al, et al, et al, et al, et al, et al, et al, et al, et al, et al, et al,
Candidate Variable 2006132 2010144 2002149 2011136 2011138 2000139 2011140 2010141 2009143 2010145 2007146 2008147 2008148
Sociodemographics
Age NR NS S S S S S S NS S S S
Sex NR NS NS NS NS NS NS S NS NS
Ethnicity NS NS
Living alone S
Comorbidities/risk
factors
Prior stroke S NS NS
Prior TIA NS NS
Prior MI NS S NS
CHD or CAD NS NS S NS
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017

Valvular HD NS
Other CVD NS
Atrial fibrillation S NS NS S S NS NS NS NS
Diabetes mellitus S NS S NS NS S NS
Hypertension NS NS NS NS NS NS NS
Hyperlipidemia NS NS NS
Renal dysfunction S
Smoking NS NS NS NS NS NS
Alcohol use NS NS NS
BMI NS
Family history NS
Prestroke status
Premorbid mRS S S S S S S
Preadmission
treatment
Statin therapy S
Antiplatelet therapy NS S
Antihypertensive NS
Stroke severity
GCS S
NIHSS S S S S S S S S NS S
Stroke origin, type
OCSP, Trial of Org S NS NS NS NS NS
subtypes, PCI/ACI
Cardioembolic S
stroke
Symptomatic ICH S
Occlusion site NS
Significant arterial
pathology
TIBI score NS NS
Complete S S
recanalization
Stroke imaging
characteristics
ASPECTS score or S S
infarct volume
(Continued)
 18StrokeMarch 2014

Table 6. Continued
mRS 01 mRS 02
Di Carlo Sharma Weimar Lin Maguire McCarron Ni Chroinin Ntaios Tao et Tsivgoulis Tsivgoulis Uyttenboogaart
et al, et al, et al, et al, et al, et al, et al, et al, Santos-Garca al, et al, et al, et al,
Candidate Variable 2006132 2010144 2002149 2011136 2011138 2000139 2011140 2010141 et al, 2009143 2010145 2007146 2008147 2008148

Early ischemic S S
changes in
brainCT
Leukoaraiosis S
Silent lesions NS
In-hospital evaluation
Hemoglobin NS
RDW NS
WBC NS
Serum creatinine NS
Serum glucose S S NS NS NS
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017

usCRP (mg/L) NS
Heart rate NS
BP
DBP NS NS
SBP NS NS NS NS S
Temperature NS
LVEF <30% NS
Process, treatment
Admission delay NS
Onset to treatment S S NS
time
IV tPA dose S
Statin therapy NS
Aspirin therapy NS
Other
Increased IMT NS
Endothelial S
dysfunction
Brachial arterial NS
FMD
COX-2 rs5275 S
COX-2 rs20417 NS
GP IIIa rs5918 NS
ACI indicates anterior circulation infarction; ASPECTS, Alberta Stroke Program Early CT score; BMI, body mass index; BP, blood pressure; CAD, coronary artery
disease; CHD, chronic heart disease; COX-2, cyclooxygenase 2; CT, computed tomography; CVD, cardiovascular disease; DBP, diastolic blood pressure; ellipses (. . .),
data not available; FMD, flow-mediated dilation; GCS, Glasgow Coma Scale; GP IIIa, glycoprotein IIIa; HD, heart disease; ICH, intracerebral hemorrhage; IMT, intima-
media thickness; IV, intravenous; LVEF, left ventricular ejection fraction; MI, myocardial infarction; mRS, modified Rankin Scale; NIHSS, National Institutes of Health
Stroke Scale; NR, not reported; NS, not significant; OCSP, Oxford Community Stroke Project; PCI, posterior circulation infarction; RDW, red cell distribution width;
S, significant; SBP, systolic blood pressure; SSS, Scandinavian Stroke Scale; TIA, transient ischemic attack; TIBI, Thrombolysis in Brain Ischemia; tPA, tissue-type
plasminogen activator; Trial of Org, Trial of Org 10172 in Acute Stroke; usCRP, ultrasensitive C-reactive protein; and WBC, white blood cell count.

stroke. Using the same rationale as for the outcome mor-
tality, inclusion in the prediction model is recommended.
3. Risk-adjustment models of functional status after isch-
emic stroke that do not include stroke severity or other
recommended variables must provide comparable classi-
fication of hospital performance as a model that includes
stroke severity or other missing variables.
4. Stroke severity and other variables that are used in
hospital-level risk-adjustment models of functional status
after ischemic stroke should be standardized across sites
so that their reliability and accuracy are equivalent.
The following are suggestions for future research related to
measuring functional status after stroke at the hospital level,
in order of priority:
1. Functional status is not uniformly captured, clinically or
administratively, after stroke. Implementation research
 Katzan etal Ischemic Stroke Outcomes 19

Table 7. Significance of Variables Associated With Functional Status as Measured by the Barthel Index
and Glasgow Outcome Scale at 90 Days
BI 60100 BI 95100 GOS 12
McCarron et al, German SSC Maguire et al, Rundek et al, Weimar et al, Maguire et al,
Candidate Variable 2000139 2004134 2011138 2004142 2002149 2011138
Sociodemographics
Age S S S S S S
Sex NS S S NS
Ethnicity NS NS
Living alone NS
Comorbidities/risk factors
Prior stroke S S
Prior MI NS NS NS
CHD NS NS
Atrial fibrillation NS NS NS S
Diabetes mellitus S NS NS S S
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017

Hypertension NS NS NS NS
Hyperlipidemia NS NS
Renal dysfunction S S
Other CVD NS
BMI NS NS
Smoking NS NS
Alcohol use NS NS
Prestroke status
Premorbid mRS S S S NS
Stroke severity
NIHSS total S S S S
Left arm weakness, NIHSS S
Right arm weakness, NIHSS S
Lenticulostriate infarction NS
Neurological complications NS
Stroke origin/type
OCSP or stroke type NS NS
In-hospital evaluation
Fever (temperature >38C) NS
Resource use
No. of days in ICU S
No. of days in hospital S
No. of days in rehab S
No. of days in NH S
Other
COX-2 rs20417 NS S
GP IIIa rs5918 S NS
COX-2 rs5275 NS NS
Country NS
BI indicates Barthel Index; BMI, body mass index; CHD, chronic heart disease; COX-2, cyclooxygenase 2; CVD, cardiovascular disease;
ellipses (. . .), data not available; GOS, Glasgow Outcome Scale; GP IIIa, glycoprotein IIIa; ICU, intensive care unit; MI, myocardial
infarction; mRS, modified Rankin Scale; NH, nursing home; NIHSS, National Institutes of Health Stroke Scale; NS, not significant; OCSP,
Oxford Community Stroke Project; S, significant; and SSC, Stroke Study Collaboration.

is urgently needed to examine the feasibility of assessing process for establishing a standardized measure of func-
functional status at a predetermined time (30 days) for tional status (including training of assessors) that can be
every stroke patient and to determine the resources and administered reliably.
 20StrokeMarch 2014
2. Research is needed to explore the variation in functional
status across hospitals and the contribution of specific
hospital characteristics after adjustment for patient char-
acteristics, including stroke severity at presentation.
3. Rehabilitation services after stroke have been clearly
shown to improve functional status.155 The receipt of
rehabilitation may depend on patient health insurance
status, as well as functional status at time of hospital
discharge; patients with severe poststroke impairment
or minimal impairment are less likely to receive reha-
bilitation services. Research is needed to determine how
best to adjust for receipt of these services after discharge
when hospital-based performance is evaluated.
4. Research is needed to guide the field on how to best ana-
lyze the mRS as an outcome when hospital quality is
examined.
Summary
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017
Each of the outcome measures evaluated in the present sci-
entific statement has strengths and weaknesses. Impairment
in functional status is the most common sequela of stroke
and is of high relevance to patients, providers, and society.
The feasibility of systematic assessment of functional sta-
tus after stroke, however, is unclear. Information on mortal-
ity can be obtained from administrative data sources and
has been used for public reporting for several other dis-
ease processes. Short-term stroke mortality, however, may
reflect patient and family preferences, which currently are
not quantifiable nor included in any models. High-quality
patient-centered care that allows patients to avoid burden-
some life-sustaining therapies may result in a hospital with
a higher mortality rate than a hospital that ignores patient
preferences and institutes aggressive care for all patients.
Readmission is a measure that focuses on healthcare use
and likely has less impact in the long term for patients than
functional status or mortality.
Each of these 3 measures has the potential to provide infor-
mation on different aspects of quality of care at the hospital
level. Functional status measures may reflect stroke interven-
tions and stroke-related processes; stroke mortality reflects
a delicate balance between evidence-based and preference-
based care; and the readmission measure may best reflect
discharge coordination processes. An examination of a com-
bination of these measures may provide the best overall evalu-
ation of hospital care, with measurement of functional status
having the most direct application for measuring the quality of
stroke-specific care.
The public reporting of stroke quality information is
presumed to motivate quality improvement by targeting
information to patients, referring physicians, and payers to
select higher-quality hospitals and to incentivize hospitals
and physicians to compete on quality by identifying areas
for performance improvement.156 It is critical to note that
the link between quality of hospital care and outcomes has
not been demonstrated robustly. In addition, the use of case-
mixadjusted outcomes may not reliably reflect differences
in quality of care.4,5 Unintended consequences, in addition to
inaccurate labeling of hospitals as good or poor performers,
potentially include incentives for hospitals and physicians to
avoid sicker patients, discount patient preferences to achieve
outcome goals, and divert resources away from equally or
more important quality improvement projects. In addition,
there could be a negative effect on provider morale when
profiling systems have significant misclassification errors.
Finally, of relevance to stroke mortality, another potential
negative or unintended consequence is the possibility of
steering patients and families away from a more palliative
approach in the setting of clear evidence to avoid prolong-
ing the dying process.157 Balancing the potential for benefit
against these risks is no easy task and will require close scru-
tiny and ongoing monitoring.
Although we focused our review on 3 main outcome mea-
sures (mortality, readmission, and functional status), there
are multiple additional measures that may prove to be useful
when evaluating quality of hospital care, such as complica-
tions and length of stay. These measures are easily obtainable
but are potentially subject to gaming by the manipulation of
hospital coding or the altering of discharge practices.
An important goal of the present scientific statement was
to make recommendations on a minimal set of variables to
be used in risk adjustment of models that evaluate care at the
hospital level. There is a striking lack of data on prediction
models for stroke outcomes at the hospital level. It is therefore
difficult to make definitive statements regarding the impor-
tance of specific variables in risk-adjustment models for the
evaluation of hospital performance. The paucity of available
evidence to identify a minimal data set underscores the impor-
tance and urgency of additional research to support quality
assessment. Variables that are common in all models include
age and comorbid conditions, as well as general vascular risk
factors. Severity of stroke, defined as either severity of impair-
ment or functional status, has been thought to be universally
important. It has strong face and content validity and has been
shown to be important for individual patient prediction. For
this and all other variables, it is theoretically possible that
despite its importance in individual patient prediction mod-
els, it may not provide significant information in models that
evaluate hospital-level performance if the distribution is simi-
lar across hospitals. However, hospitals that have comprehen-
sive stroke programs may receive patients with more severe
strokes through direct transport, which would preclude them
from exclusion in public reporting programs. With the devel-
opment of systems of care for stroke in the United States158,159
and certification of comprehensive stroke centers,160 the direct
transport by local emergency medical services in addition
to the subsequent transfer of patients with severe strokes to
selected stroke centers is likely to increase. Importantly, some
data are now available that demonstrate a significant impact
of adjustment for stroke severity on hospital performance for
30-day mortality,90 which supports clinical impressions of its
importance in risk-adjustment models in the evaluation of per-
formance at the hospital level.
The healthcare system is progressively moving toward
outcomes-based care, which will be used to drive reimburse-
ment and compel healthcare systems to focus on and optimize
the measurable outcomes of care. Advances in technology
will allow clinical information to be extracted more easily
from medical record systems and may aid in the assessment

of outcomes after hospital discharge. Along with the oppor-
tunity to advance the care of patients with stroke, we must
vigilantly seek to minimize the unintended consequences of
measuring hospital-based performance by use of inadequate
risk adjustment.
Future Research
There is a pressing need for research in multiple areas to better
identify methods and metrics to evaluate outcomes of stroke
care. In addition to the focused research needs described
within each outcome measure section, the following areas are
highlighted in order of priority:
1. There is a profound lack of data demonstrating the
impact of quality of hospital care on stroke outcomes.
It is critically important to define modifiable aspects of
stroke care that can improve outcomes.
2. Along similar lines, there is a pressing need to deter-
mine whether current case-mix adjustment methods
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017
can adequately discriminate quality of care provided by
hospitals.
3. Defining the factors, including stroke severity, thatare
most important to include in risk adjustment of hospital-
level models of mortality, readmissions, and functional
outcomes 30 days after a stroke is a critical research
priority.
4. Research is urgently needed to evaluate the ability of
models without the recommended list of variables (par-
ticularly stroke severity) to produce comparable dis-
crimination in hospital performance as models that are
otherwise similar but include these variables.
5. Future work should be directed toward understanding
patient and family preferences toward different treatment
approaches in the setting of severe stroke disability, limited
prognosis, or both, as well as how these preferences should
be quantified and incorporated into risk-adjustment meth-
ods. The development of methods to incorporate patient
preferences and provider adherence to patient preferences
into risk-modeling schemes is a priority.
6. The roles of depression, family functioning, and care-
giver support as intervening variables between acute
Katzan etal Ischemic Stroke Outcomes 21
care and poststroke functional status, mortality, and
readmissions need to be explored.
7. Future research is warranted to understand what variables
after time zero should be included in risk-adjustment
models for outcomes after stroke
8. Research is needed to identify the predictive ability of
risk-adjustment models based on automated data collec-
tion without manual chart review. These models could
include information derived from administrative billing
codes or electronic health records.
9. Research is needed on the distribution of patients with
varying stroke severity across hospitals and on the refer-
ral patterns for patients with severe stroke.
10. Research is needed to identify the most appropriate way
to deal with patients transferred in or out of hospitals.
Because of the current lack of data, these high-risk
patients are typically excluded from determinations of
hospital risk-adjusted outcomes. These patients will rep-
resent an increasingly important subgroup as systems of
care for stroke develop in the United States and transfers
increase.
11. The combination of multiple end points, such as depen-
dence or disability and death, is common in randomized
controlled trials as a method to gain statistical power161;
however, the inclusion of such a combination as the pri-
mary outcome measure reduces the interpretability of
the outcome. Research is needed to evaluate the use of
composite end points as outcome measures.
12. The difficulties in coalescing the data from literature
reviewed for the present scientific statement highlight
the need for greater clarity in reporting of prediction
models. Clinical, health services, and outcomes research
need to more clearly report how data were collected, the
variable structure, and how all variables performed in a
model.
13. There is a need for further study of the consistency in
coding across sites of key prognostic variables used in
risk-adjustment models.
14. More research is needed to understand the impact of
using in-hospital measures as proxies of the more desir-
able but often impractical longer-term outcomes.
 22StrokeMarch 2014

Disclosures
Writing Group Disclosures
Other Speakers
Writing Group Research Bureau/ Expert Ownership Consultant/
Member Employment Research Grant Support Honoraria Witness Interest Advisory Board Other
Irene L. Katzan Cleveland Clinic None None None None None None None
John Spertus Saint Lukes AHA; Genentech; None None None None Genentech* None
Hospital of Kansas NHLBI/NIH
City; University of
MissouriKansas
City
Janet Prvu Duke University AHRQ; PCORI None None None None None None
Bettger
Dawn M. Bravata Department of None None None None None None None
Veterans Affairs
Cheryl Bushnell Wake Forest NIH/NINDS World None None None None None
University Health Federation of
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017

Sciences Neurology*
Randall T. UCSF None None None None None None None
Higashida
Judith A. Hinchey Caritas Steward None None None None None None None
St. Elizabeths
Medical Center
Robert G. University of None None None None None None None
Holloway Rochester Medical
Center
George Howard University of Abbott* None None None None Abbott*; Bayer None
Alabama at (Executive
Birmingham Committee);
Brainsgate*;
Cerevast*;
MediciNova*; DSMB
for Mitsubishi*;
PhotoThera*
Rosemarie B. Northwestern NIH None None None None None None
King University
Harlan M. Yale University Medtronic None None None None United Healthcare Centers for Medicare
Krumholz & Medicaid Services
(CMS, contract to
develop performance
measures)
Barbara J. Lutz University of HRSA; NIH/NINR; None None None General American Case None
North Carolina- Rehabilitation Electric Management
Wilmington Nursing Foundation (stock Association (unpaid
grant (mentor)*; VA ownership) consultant)*
HSR&D grant*
Matthew J. Michigan State None None None None None None None
Reeves University
Eric E. Smith University of None None None None None Genentech* AHA GWTG Executive
Calgary and Steering
Committees (unpaid
volunteer)*
Robert W. Yeh Massachusetts None None None None None None None
General Hospital
This table represents the relationships of writing group members that may be perceived as actual or reasonably perceived conflicts of interest as reported on
the Disclosure Questionnaire, which all members of the writing group are required to complete and submit. A relationship is considered to be significant if (1) the
person receives $10000 or more during any 12-month period, or 5% or more of the persons gross income; or (2) the person owns 5% or more of the voting stock
or share of the entity, or owns $10000 or more of the fair market value of the entity. A relationship is considered to be modest if it is less than significant under
the preceding definition.
*Modest.
Significant.

Reviewer Disclosures
Other
Research Research
Reviewer Employment Grant Support
Mohammed University of Birmingham, UK None None
Mohammed
Patrick Romano University of California Davis None None
Gustavo Saposnik St Michaels Hospital, None None
University of Toronto
This table represents the relationships of reviewers that may be perceived as actual or reasonably perceived conflicts of interest as reported on the Disclosure
Questionnaire, which all reviewers are required to complete and submit. A relationship is considered to be significant if (1) the person receives $10000 or more during
any 12-month period, or 5% or more of the persons gross income; or (2) the person owns 5% or more of the voting stock or share of the entity, or owns $10000 or more
of the fair market value of the entity. A relationship is considered to be modest if it is less than significant under the preceding definition.
References
1. Go AS, Mozaffarian D, Roger VL, Benjamin EJ, Berry JD, Blaha MJ,
Dai S, Ford ES, Fox CS, Franco S, Fullerton HJ, Gillespie C, Hailpern
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017
SM, Heit JA, Howard VJ, Huffman MD, Judd SE, Kissela BM, Kittner
SJ, Lackland DT, Lichtman JH, Lisabeth LD, Mackey RH, Magid DJ,
Marcus GM, Marelli A, Matchar DB, McGuire DK, Mohler ER 3rd, Moy
CS, Mussolino ME, Neumar RW, Nichol G, Pandey DK, Paynter NP,
Reeves MJ, Sorlie PD, Stein J, Towfighi A, Turan TN, Virani SS, Wong
ND, Woo D, Turner MB; on behalf of the American Heart Association
Statistics Committee and Stroke Statistics Subcommittee. Heart disease
and stroke statistics2014 update: a report from the American Heart
Association. Circulation. 2014;129:e28e292.
2. Medicare Expenditure Panel Survey 2009. Household component sum-
mary tables. Agency for Healthcare Research and Quality Web site.
http://meps.ahrq.gov/mepsweb/data_stats/tables_compendia_hh_inter-
active.jsp. Accessed April 12, 2012.
3. Clancy CM, Eisenberg JM. Outcomes research: measuring the end
results of health care. Science. 1998;282:245246.
4. Lilford R, Mohammed MA, Spiegelhalter D, Thomson R. Use and mis-
use of process and outcome data in managing performance of acute med-
ical care: avoiding institutional stigma. Lancet. 2004;363:11471154.
5. Pitches DW, Mohammed MA, Lilford RJ. What is the empirical evidence
that hospitals with higher-risk adjusted mortality rates provide poorer
quality care? A systematic review of the literature. BMCHealth Serv
Res. 2007;7:91.
6. Country and lending groups. The World Bank Web site. http://data.
worldbank.org/about/country-classifications/country-and-lending-
groups#High_income. Accessed April 12, 2012.
7. Ovbiagele B, Lyden PD, Saver JL. Disability status at 1 month is a
reliable proxy for final ischemic stroke outcome. Neurology. 2010;
75:688692.
8. Duncan PW, Goldstein LB, Matchar D, Divine GW, Feussner J.
Measurement of motor recovery after stroke: outcome assessment and
sample size requirements. Stroke. 1992;23:10841089.
9. Duncan PW, Jorgensen HS, Wade DT. Outcome measures in acute stroke
trials: a systematic review and some recommendations to improve prac-
tice. Stroke. 2000;31:14291438.
10. The National Institute of Neurological Disorders and Stroke rt-PA Stroke
Study Group. Tissue plasminogen activator for acute ischemic stroke.
NEngl J Med. 1995;333:15811587.
11. Krumholz HM, Brindis RG, Brush JE, Cohen DJ, Epstein AJ, Furie K,
Howard G, Peterson ED, Rathore SS, Smith SC Jr, Spertus JA, Wang
Y, Normand SL. Standards for statistical models used for public report-
ing of health outcomes: an American Heart Association scientific state-
ment from the Quality of Care and Outcomes Research Interdisciplinary
Writing Group. Circulation. 2006;113:456462.
12. Parker C, Schwamm LH, Fonarow GC, Smith EE, Reeves MJ.
Stroke quality metrics: systematic reviews of the relationships to
patient-centered outcomes and impact of public reporting. Stroke.
2012;43:155162.
13. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models:
issues in developing models, evaluating assumptions and adequacy, and
measuring and reducing errors. Stat Med. 1996;15:361387.
14. Lloyd-Jones DM. Cardiovascular risk prediction: basic concepts, current
status, and future directions. Circulation. 2010;121:17681777.
Katzan etal Ischemic Stroke Outcomes 23
Speakers
Bureau/ Expert Ownership Consultant/
Honoraria Witness Interest Advisory Board Other
None None None None None
None None None None None
None None None None None
15. American Stroke Association. Get with the GuidelinesStroke.
American Heart Association Web site. http://www.heart.org/
HEARTORG/HealthcareProfessional/GetWithTheGuidelinesHFStroke/
GetWithTheGuidelinesStrokeHomePage/Get-With-The-Guidelines-
Stroke-Home-Page_UCM_306098_SubHomePage.jsp. Accessed July
7,2011.
16. Jollis JG, Ancukiewicz M, DeLong ER, Pryor DB, Muhlbaier LH, Mark
DB. Discordance of databases designed for claims payment versus clini-
cal information systems. Implications for outcomes research. AnnIntern
Med. 1993;119:844850.
17. Fonarow GC, Smith EE, Reeves MJ, Pan W, Olson D, Hernandez
AF, Peterson ED, Schwamm LH; Get With The Guidelines Steering
Committee and Hospitals. Hospital-level variation in mortality and
rehospitalization for Medicare beneficiaries with acute ischemic stroke.
Stroke. 2011;42:159166.
18. Mohammed MA, Deeks JJ, Girling A, Rudge G, Carmalt M, Stevens
AJ, Lilford RJ. Evidence of methodological bias in hospital standardised
mortality ratios: retrospective database study of English hospitals [pub-
lished correction appears in BMJ. 2009 Apr 1. doi:10.1136/bmj.b1348].
BMJ. 2009;338:b780.
19. Nicholl J. Case-mix adjustment in non-randomised observational
evaluations: the constant risk fallacy. JEpidemiol Community Health.
2007;61:10101013.
20. Lawthers AG, McCarthy EP, Davis RB, Peterson LE, Palmer RH,
Iezzoni LI. Identification of in-hospital complications from claims data:
is it valid? MedCare. 2000;38:785795.
21. Measuring and improving quality of care: a report from the American
Heart Association/American College of Cardiology First Scientific
Forum on Assessment of Healthcare Quality in Cardiovascular Disease
and Stroke. Circulation. 2000;101:14831493.
22. Hospital compare. Medicare.gov Web site. http://medicare.gov/hospi-
talcompare/. Accessed March 15, 2013.
23. Jrgensen HS, Nakayama H, Raaschou HO, Vive-Larsen J, Stier M,
Olsen TS. Outcome and time course of recovery in stroke, part II: time
course of recovery: the Copenhagen Stroke Study. Arch Phys Med
Rehabil. 1995;76:406412.
24. Demain S, Wiles R, Roberts L, McPherson K. Recovery plateau follow-
ing stroke: fact or fiction? Disabil Rehabil. 2006;28:815821.
25. Iezzoni LI, ed. Risk Adjustment for Measuring Health Care Outcomes.
3rd ed. Chicago, IL: Health Administration Press; 2003.
26. Keyhani S, Cheng E, Arling G, Li X, Myers L, Ofner S, Williams
LS, Phipps M, Ordin D, Bravata DM. Does the inclusion of stroke
severity in a 30-day mortality model change standardized mortality
rates at Veterans Affairs hospitals? Circ Cardiovasc Qual Outcomes.
2012;5:508513.
27. Normand S-LT, Shahian DM. Statistical and clinical aspects of hospital
outcomes profiling. Stat Sci. 2007;22:206226.
28. Normand S-LT, Glickman ME, Gatsonis CA. Statistical methods for pro-
filing providers of medical care: issues and applications. JAm Stat Assoc.
1997;92:803814.
29. Krumholz HM, Wang Y, Mattera JA, Han LF, Ingber MJ, Roman S,
Normand SL. An administrative claims model suitable for profiling hos-
pital performance based on 30-day mortality rates among patients with
an acute myocardial infarction. Circulation. 2006;113:16831692.
30. The COPSS-CMS White Paper Committee; Ash AS, Fienberg SE, Louis
TA, Normand S-LT, Stukel TA, Utts J. Statistical Issues in Assessing
 24StrokeMarch 2014
Hospital Performance: Commissioned by the Committee of Presidents of
Statistical Societies. https://www.cms.gov/Medicare/Quality-Initiatives-
Patient-Assessment-Instruments/HospitalQualityInits/Downloads/
Statistical-Issues-in-Assessing-Hospital-Performance.pdf. Accessed
August 9, 2012.
31. Shahian DM, Normand SL. Comparison of risk-adjusted hospital out-
comes. Circulation. 2008;117:19551963.
32. Cook NR. Use and misuse of the receiver operating characteristic curve
in risk prediction. Circulation. 2007;115:928935.
33. Diamond GA. What price perfection? Calibration and discrimination of
clinical prediction models. JClin Epidemiol. 1992;45:8589.
34. Austin PC, Reeves MJ. The relationship between the C-statistic of a risk-
adjustment model and the accuracy of hospital report cards: a Monte
Carlo study. MedCare. 2013;51:275284.
35. Thomas JW, Hofer TP. Accuracy of risk-adjusted mortality rate as a mea-
sure of hospital quality of care. MedCare. 1999;37:8392.
36. Carandang R, Seshadri S, Beiser A, Kelly-Hayes M, Kase CS, Kannel
WB, Wolf PA. Trends in incidence, lifetime risk, severity, and 30-day
mortality of stroke over the past 50 years. JAMA. 2006;296:29392946.
37. Bravata DM, Ho SY, Meehan TP, Brass LM, Concato J. Readmission and
death after hospitalization for acute ischemic stroke: 5-year follow-up in
the Medicare population. Stroke. 2007;38:18991904.
38. Snijders TAB, Bosker RJ. Multilevel Analysis: An Introduction to
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017
Basic and Advanced Multilevel Modeling. Thousand Oaks, CA: Sage
Publications; 1999.
39. Arling G, Reeves M, Ross J, Williams LS, Keyhani S, Chumbler N,
Phipps MS, Roumie C, Myers LJ, Salanitro AH, Ordin DL, Myers
J, Bravata DM. Estimating and reporting on the quality of inpatient
stroke care by Veterans Health Administration Medical Centers. Circ
Cardiovasc Qual Outcomes. 2012;5:4451.
40. Glance LG, Dick A, Osler TM, Li Y, Mukamel DB. Impact of changing the
statistical methodology on hospital and surgeon ranking: the case of the
New York State cardiac surgery report card. MedCare. 2006;44:311319.
41. Hannan EL, Wu C, DeLong ER, Raudenbush SW. Predicting risk-
adjusted mortality for CABG surgery: logistic versus hierarchical logis-
tic models. MedCare. 2005;43:726735.
42. Saposnik G, Baibergenova A, ODonnell M, Hill MD, Kapral MK,
Hachinski V; Stroke Outcome Research Canada (SORCan) Working
Group. Hospital volume and stroke outcome: does it matter? Neurology.
2007;69:11421151.
43. Svendsen ML, Ehlers LH, Ingeman A, Johnsen SP. Higher stroke unit
volume associated with improved quality of early stroke care and
reduced length of stay. Stroke. 2012;43:30413045.
44. Bath PM, Lees KR, Schellinger PD, Altman H, Bland M, Hogg C,
Howard G, Saver JL; European Stroke Organisation Outcomes Working
Group. Statistical analysis of the primary outcome in acute stroke tri-
als [published correction appears in Stroke. 2012;43:e100]. Stroke.
2012;43:11711178.
45. Howard G, Waller JL, Voeks JH, Howard VJ, Jauch EC, Lees KR,
Nichols FT, Rahlfs VW, Hess DC. A simple, assumption-free, and clini-
cally interpretable approach for analysis of modified Rankin outcomes.
Stroke. 2012;43:664669.
46. Goldstein H, Spiegelhalter DJ. League tables and their limitations: statis-
tical issues in comparisons of institutional performance. JR Statist Soc
A. 1996;159:385443.
47. Hofer TP, Hayward RA. Identifying poor-quality hospitals: can hos-
pital mortality rates detect quality problems for medical diagnoses?
MedCare. 1996;34:737753.
48. DeLong ER, Peterson ED, DeLong DM, Muhlbaier LH, Hackett S, Mark
DB. Comparing risk-adjustment methods for provider profiling. Stat
Med. 1997;16:26452664.
49. Austin PC, Alter DA, Tu JV. The use of fixed- and random-effects mod-
els for classifying hospitals as mortality outliers: a Monte Carlo assess-
ment. MedDecis Making. 2003;23:526539.
50. Jones HE, Spiegelhalter DJ. The identification of unusual health-care
providers from a hierarchical model. AmStat. 2011;65:154163.
51. Austin PC, Anderson GM. Optimal statistical decisions for hospital
report cards. MedDecis Making. 2005;25:1119.
52. Austin PC, Brunner LJ. Optimal Bayesian probability levels for hospital
report cards. Health Serv Outcomes Res Methods,. 2008;8:8097.
53. Austin PC, Tu JV, Alter DA, Naylor CD. The impact of under coding of
cardiac severity and comorbid diseases on the accuracy of hospital report
cards. MedCare. 2005;43:801809.
54. Fry DE, Pine MB, Jordan HS, Hoaglin DC, Jones B, Meimban
R. The hazards of using administrative data to measure surgical
quality [published correction appears in Am Surg. 2007;73:199]. AmSurg.
2006;72:10311037.
55. Ghali WA, Rothwell DM, Quan H, Brant R, Tu JV. A Canadian compari-
son of data sources for coronary artery bypass surgery outcome report
cards. AmHeart J. 2000;140:402408.
56. Smith EE, Shobha N, Dai D, Olson DM, Reeves MJ, Saver JL, Hernandez
AF, Peterson ED, Fonarow GC, Schwamm LH. Risk score for in-hospital
ischemic stroke mortality derived and validated within the Get With the
Guidelines-Stroke Program. Circulation. 2010;122:14961504.
57. Saposnik G, Hill MD, ODonnell M, Fang J, Hachinski V, Kapral MK;
Registry of the Canadian Stroke Network for the Stroke Outcome Research
Canada (SORCan) Working Group. Variables associated with 7-day, 30-day,
and 1-year fatality after ischemic stroke. Stroke. 2008;39:23182324.
58. Ogbu UC, Slobbe LC, Arah OA, de Bruin A, Stronks K, Westert
GP. Hospital stroke volume and case-fatality revisited. MedCare.
2010;48:149156.
59. Ogbu UC, Westert GP, Slobbe LC, Stronks K, Arah OA. A multifac-
eted look at time of admission and its impact on case-fatality among
a cohort of ischaemic stroke patients. JNeurol Neurosurg Psychiatry.
2011;82:813.
60. Fang J, Saposnik G, Silver FL, Kapral MK; Investigators of the Registry
of the Canadian Stroke Network. Association between weekend hospital
presentation and stroke fatality. Neurology. 2010;75:15891596.
61. Reeves MJ, Smith E, Fonarow G, Hernandez A, Pan W, Schwamm LH;
GWTG-Stroke Steering Committee and Investigators. Off-hour admis-
sion and in-hospital stroke case fatality in the Get With The Guidelines-
Stroke program. Stroke. 2009;40:569576.
62. Xian Y, Holloway RG, Noyes K, Shah MN, Friedman B. Racial differ-
ences in mortality among patients with acute ischemic stroke: an obser-
vational study. AnnIntern Med. 2011;154:152159.
63. Barnato AE, Chang CC, Farrell MH, Lave JR, Roberts MS, Angus DC. Is
survival better at hospitals with higher end-of-life treatment intensity?
MedCare. 2010;48:125132.
64. Stroke Unit Trialists Collaboration. Organised inpatient (stroke unit)
care for stroke. Cochrane Database Syst Rev. 2007;9:CD000197.
65. Xian Y, Holloway RG, Chan PS, Noyes K, Shah MN, Ting HH,
Chappel AR, Peterson ED, Friedman B. Association between stroke
center hospitalization for acute ischemic stroke and mortality. JAMA.
2011;305:373380.
66. CAST (Chinese Acute Stroke Trial) Collaborative Group. CAST: ran-
domised placebo-controlled trial of early aspirin use in 20,000 patients
with acute ischaemic stroke. Lancet. 1997;349:16411649.
67. Sandercock PA, Counsell C, Gubitz GJ, Tseng MC. Antiplatelet
therapy for acute ischaemic stroke. Cochrane Database Syst Rev.
2008;(3):CD000029.
68. Bravata DM, Wells CK, Lo AC, Nadeau SE, Melillo J, Chodkowski D,
Struve F, Williams LS, Peixoto AJ, Gorman M, Goel P, Acompora G,
McClain V, Ranjbar N, Tabereaux PB, Boice JL, Jacewicz M, Concato
J. Processes of care associated with acute stroke outcomes. Arch Intern
Med. 2010;170:804810.
69. Vahedi K, Hofmeijer J, Juettler E, Vicaut E, George B, Algra A, Amelink
GJ, Schmiedeck P, Schwab S, Rothwell PM, Bousser MG, van der Worp
HB, Hacke W; DECIMAL, DESTINY, and HAMLET Investigators.
Early decompressive surgery in malignant infarction of the middle
cerebral artery: a pooled analysis of three randomised controlled trials.
Lancet Neurol. 2007;6:215222.
70. Hacke W, Kaste M, Bluhmki E, Brozman M, Davalos A, Guidetti D,
Larrue V, Lees KR, Medeghri Z, Machnig T, Schneider D, von Kummer
R, Wahlgren N, Toni D; ECASS Investigators. Thrombolysis with
alteplase 3 to 4.5 hours after acute ischemic stroke. NEngl J Med.
2008;359:13171329.
71. Tengs TO, Yu M, Luistro E. Health-related quality of life after stroke a
comprehensive review. Stroke. 2001;32:964972.
72. Patrick DL, Pearlman RA, Starks HE, Cain KC, Cole WG, Uhlmann RF.
Validation of preferences for life-sustaining treatment: implications for
advance care planning. AnnIntern Med. 1997;127:509517.
73. Counsell C, Dennis M, McDowall M, Warlow C. Predicting outcome
after acute and subacute stroke: development and validation of new prog-
nostic models. Stroke. 2002;33:10411047.
74. Hinchey JA, Furlan AJ, Frank JI, Kay R, Misch D, Hill C. Is in-hospital
stroke mortality an accurate measure of quality of care? Neurology.
1998;50:619625.
75. Iezzoni LI, Shwartz M, Ash AS, Mackiernan YD. Predicting in-hospital
mortality for stroke patients: results differ across severity-measurement
methods. MedDecis Making. 1996;16:348356.

76. Lewis SC, Sandercock PA, Dennis MS; SCOPE (Stroke Complications
and Outcomes Prediction Engine) Collaborations; IST. Predicting out-
come in hyper-acute stroke: validation of a prognostic model in the
Third International Stroke Trial (IST3). JNeurol Neurosurg Psychiatry.
2008;79:397400.
77. Saposnik G, Kapral MK, Liu Y, Hall R, ODonnell M, Raptis S, Tu JV,
Mamdani M, Austin PC; Investigators of the Registry of the Canadian
Stroke Network for the Stroke Outcomes Research Canada (SORCan)
Working Group. IScore: a risk score to predict death early after hospital-
ization for an acute ischemic stroke. Circulation. 2011;123:739749.
78. Tabak YP, Johannes RS, Silber JH. Using automated clinical data for risk
adjustment: development and validation of six disease-specific mortality
predictive models for pay-for-performance. MedCare. 2007;45:789805.
79. Wang Y, Lim LL, Heller RF, Fisher J, Levi CR. A prediction model
of 1-year mortality for acute ischemic stroke patients. Arch Phys Med
Rehabil. 2003;84:10061011.
80. Weimar C, Ziegler A, Knig IR, Diener HC. Predicting func-
tional outcome and survival after acute ischemic stroke. JNeurol.
2002;249:888895.
81. Weimar C, Knig IR, Kraywinkel K, Ziegler A, Diener HC; German
Stroke Study Collaboration. Age and National Institutes of Health Stroke
Scale Score within 6 hours after onset are accurate predictors of outcome
after cerebral ischemia: development and external validation of prognos-
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017
tic models. Stroke. 2004;35:158162.
82. Ohman EM, Granger CB, Harrington RA, Lee KL. Risk stratification
and therapeutic decision making in acute coronary syndromes. JAMA.
2000;284:876878.
83. Ioannidis JP, Tzoulaki I. What makes a good predictor? The evidence
applied to coronary artery calcium score. JAMA. 2010;303:16461647.
84. Fonarow GC, Saver JL, Smith EE, Broderick JP, Kleindorfer DO, Sacco
RL, Pan W, Olson DM, Hernandez AF, Peterson ED, Schwamm LH.
Relationship of National Institutes of Health Stroke Scale to 30-day mor-
tality in Medicare beneficiaries with acute ischemic stroke. JAm Heart
Assoc. 2012;1:4250.
85. Whiteley W, Chong WL, Sengupta A, Sandercock P. Blood mark-
ers for the prognosis of ischemic stroke: a systematic review. Stroke.
2009;40:e380e389.
86. Medicare hospital quality initiative measure methodology. Centers for
Medicare & Medicaid Services Web site. http://www.cms.gov/Medicare/
Quality-Initiatives-Patient-Assessment-Instruments/HospitalQualityInits/
Measure-Methodology.html. Accessed May 14, 2013.
87. El Husseini N, Shea KJ, Goldstein LB. Concerns for the reliabil-
ity and validity of the National Stroke Project Stroke Severity Scale.
Cerebrovasc Dis. 2011;32:426430.
88. Measure methodology: stroke readmission and mortality mea-
sures. QualityNet Web site. http://www.qualitynet.org/dcs/Content
Server?c=Page&pagename=QnetPublic%2FPage%2FQnetTier3&cid=
1228773364875. Accessed September 16, 2013.
89. Acute stroke mortality rate (IQI 17), NQF measure 0467. National
Quality Forum Web site. http://www.qualityforum.org/Qps/QpsTool.asp
x#qpsPageState=%7B%22TabType%22%3A1,%22TabContentType%2
2%3A2,%22SearchCriteriaForStandard%22%3A%7B%22TaxonomyI
Ds%22%3A%5B%5D,%22SelectedTypeAheadFilterOption%22%3A%
7B%22ID%22%3A14043,%22FilterOptionLabel%22%3A%22stroke%
22,%22TypeOfTypeAheadFilterOption%22%3A1,%22TaxonomyId%2
2%3A0%7D,%22Keyword%22%3A%22stroke%22,%22PageSize%22
%3A%2225%22,%22OrderType%22%3A12,%22OrderBy%22%3A%
22DESC%22,%22PageNo%22%3A1,%22IsExactMatch%22%3Afalse,
%22QueryStringType%22%3A%22%22,%22ProjectActivityId%22%3
A%220%22%7D,%22SearchCriteriaForForPortfolio%22%3A%7B%2
2Tags%22%3A%5B%5D,%22FilterTypes%22%3A0,%22PageStartInd
ex%22%3A1,%22PageEndIndex%22%3A25,%22PageNumber%22%3
Anull,%22PageSize%22%3A%2225%22,%22SortBy%22%3A%22Titl
e%22,%22SortOrder%22%3A%22ASC%22%7D,%22ItemsToCompar
e%22%3A%5B%5D,%22StandardID%22%3A340,%22EntityTypeID%
22%3A1%7D. Accessed March 17, 2013.
90. Fonarow GC, Pan W, Saver JL, Smith EE, Reeves MJ, Broderick JP,
Kleindorfer DO, Sacco RL, Olson DM, Hernandez AF, Peterson ED,
Schwamm LH. Comparison of 30-day mortality models for profiling
hospital performance in acute ischemic stroke with vs without adjust-
ment for stroke severity. JAMA. 2012;308:257264.
91. Jauch EC, Saver JL, Adams HP Jr, Bruno A, Connors JJ, Demaerschalk
BM, Khatri P, McMullan PW Jr, Qureshi AI, Rosenfield K, Scott PA,
Summers DR, Wang DZ, Wintermark M, Yonas H; on behalf of the
American Heart Association Stroke Council; Council on Cardiovascular
Katzan etal Ischemic Stroke Outcomes 25
Nursing; Council on Peripheral Vascular Disease; Council on Clinical
Cardiology. Guidelines for the early management of patients with
acute ischemic stroke: a guideline for healthcare professionals from
the American Heart Association/American Stroke Association. Stroke.
2013;44:870947.
92. Singer OC, Dvorak F, du Mesnil de Rochemont R, Lanfermann H, Sitzer
M, Neumann-Haefelin T. A simple 3-item stroke scale: comparison with
the National Institutes of Health Stroke Scale and prediction of middle
cerebral artery occlusion. Stroke. 2005;36:773776.
93. Anderson C, Deepak BV, Amoateng-Adjepong Y, Zarich S. Benefits of
comprehensive inpatient education and discharge planning combined
with outpatient support in elderly patients with congestive heart failure.
Congest Heart Fail. 2005;11:315321.
94. Jack BW, Chetty VK, Anthony D, Greenwald JL, Sanchez GM, Johnson
AE, Forsythe SR, ODonnell JK, Paasche-Orlow MK, Manasseh C,
Martin S, Culpepper L. A reengineered hospital discharge program
to decrease rehospitalization: a randomized trial. AnnIntern Med.
2009;150:178187.
95. Naylor MD, Brooten D, Campbell R, Jacobsen BS, Mezey MD, Pauly
MV, Schwartz JS. Comprehensive discharge planning and home
follow-up of hospitalized elders: a randomized clinical trial. JAMA.
1999;281:613620.
96. Stauffer BD, Fullerton C, Fleming N, Ogola G, Herrin J, Stafford PM,
Ballard DJ. Effectiveness and cost of a transitional care program for heart
failure: a prospective study with concurrent controls. Arch Intern Med.
2011;171:12381243.
97. Voss R, Gardner R, Baier R, Butterfield K, Lehrman S, Gravenstein S.
The care transitions intervention: translating from efficacy to effective-
ness. Arch Intern Med. 2011;171:12321237.
98. Bohannon RW, Lee N. Hospital readmissions and deaths during the first
year after hospitalization for stroke. Conn Med. 2003;67:535539.
99. McGruder HF, Croft JB, Zheng ZJ. Characteristics of an ill-
defined diagnosis for stroke: opportunities for improvement. Stroke.
2006;37:781789.
100. Goldfield NI, McCullough EC, Hughes JS, Tang AM, Eastman B,
Rawlins LK, Averill RF. Identifying potentially preventable readmis-
sions. Health Care Financ Rev. 2008;30:7591.
101. Averill RF, Goldfield NI, Muldoon J, Steinbeck BA, Grant TM. A closer
look at all-patient refined DRGs. JAHIMA. 2002;73:4650.
102. Lichtman JH, Leifheit-Limson EC, Jones SB, Watanabe E, Bernheim
SM, Phipps MS, Bhat KR, Savage SV, Goldstein LB. Predictors of
hospital readmission after stroke: a systematic review. Stroke. 2010;
41:25252533.
103. Bohannon RW, Lee N. Association of physical functioning with
same-hospital readmission after stroke. AmJ Phys Med Rehabil.
2004;83:434438.
104. Camberg LC, Smith NE, Beaudet M, Daley J, Cagan M, Thibault
G. Discharge destination and repeat hospitalizations. MedCare.
1997;35:756767.
105. Chuang KY, Wu SC, Ma AH, Chen YH, Wu CL. Identifying factors asso-
ciated with hospital readmissions among stroke patients in Taipei. JNurs
Res. 2005;13:117128.
106. Heller RF, Fisher JD, DEste CA, Lim LL, Dobson AJ, Porter R. Death
and readmission in the year after hospital admission with cardiovas-
cular disease: the Hunter Area Heart and Stroke Register. MedJ Aust.
2000;172:261265.
107. Jia H, Zheng Y, Reker DM, Cowper DC, Wu SS, Vogel WB, Young GC,
Duncan PW. Multiple system utilization and mortality for veterans with
stroke. Stroke. 2007;38:355360.
108. Johansen HL, Wielgosz AT, Nguyen K, Fry RN. Incidence, comorbidity,
case fatality and readmission of hospitalized stroke patients in Canada.
CanJ Cardiol. 2006;22:6571.
109. Kane RL, Chen Q, Finch M, Blewett L, Burns R, Moskowitz M.
Functional outcomes of posthospital care for stroke and hip fracture pa-
tients under Medicare. JAm Geriatr Soc. 1998;46:15251533.
110. Kennedy BS. Does race predict stroke readmission? An analysis using
the truncated negative binomial model. JNatl Med Assoc. 2005;97:
699713.
111. Lichtman JH, Allen NB, Wang Y, Watanabe E, Jones SB, Goldstein
LB. Stroke patient outcomes in US hospitals before the start of the
Joint Commission Primary Stroke Center certification program. Stroke.
2009;40:35743579.
112. Lindenauer PK, Rothberg MB, Pekow PS, Kenwood C, Benjamin EM,
Auerbach AD. Outcomes of care by hospitalists, general internists, and
family physicians. NEngl J Med. 2007;357:25892600.
 26StrokeMarch 2014
113. Roe CJ, Kulinskaya E, Brisbane M, Brown R, Barter C. A methodol-
ogy for measuring clinical outcomes in an acute care teaching hospital.
JQual Clin Pract. 1996;16:203214.
114. Smith MA, Frytak JR, Liou JI, Finch MD. Rehospitalization and sur-
vival for stroke patients in managed care and traditional Medicare plans.
MedCare. 2005;43:902910.
115. Smith MA, Liou JI, Frytak JR, Finch MD. 30-Day survival and re-
hospitalization for stroke patients according to physician specialty.
Cerebrovasc Dis. 2006;22:2126.
116. Sun Y, Toh MP. Impact of diabetes mellitus (DM) on the health-care uti-
lization and clinical outcomes of patients with stroke in Singapore. Value
Health. 2009;12(suppl 3):S101S105.
1 17. Tseng MC, Lin HJ. Readmission after hospitalization for
stroke in Taiwan: results from a national sample. JNeurol Sci.
2009;284:5255.
118. Ingeman A, Andersen G, Hundborg HH, Svendsen ML, Johnsen SP.
In-hospital medical complications, length of stay, and mortality among
stroke unit patients. Stroke. 2011;42:32143218.
119. Keenan PS, Normand SL, Lin Z, Drye EE, Bhat KR, Ross JS, Schuur
JD, Stauffer BD, Bernheim SM, Epstein AJ, Wang Y, Herrin J, Chen J,
Federer JJ, Mattera JA, Krumholz HM. An administrative claims mea-
sure suitable for profiling hospital performance on the basis of 30-day
all-cause readmission rates among patients with heart failure. Circ
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017
Cardiovasc Qual Outcomes. 2008;1:2937.
120. Krumholz HM, Lin Z, Drye EE, Desai MM, Han LF, Rapp MT, Mattera
JA, Normand SL. An administrative claims measure suitable for profil-
ing hospital performance based on 30-day all-cause readmission rates
among patients with acute myocardial infarction. Circ Cardiovasc Qual
Outcomes. 2011;4:243252.
121. Kalra L, Langhorne P. Facilitating recovery: evidence for organized
stroke care. JRehabil Med. 2007;39:97102.
122. Dennis MS, Lewis SC, Warlow C; FOOD Trial Collaboration.
Routine oral nutritional supplementation for stroke patients in hos-
pital (FOOD): a multicentre randomised controlled trial. Lancet.
2005;365:755763.
123. Prvu Bettger JA, Coster WJ, Latham NK, Keysor JJ. Analyzing change in
recovery patterns in the year after acute hospitalization. Arch Phys Med
Rehabil. 2008;89:12671275.
124. Dromerick AW, Edwards DF, Diringer MN. Sensitivity to changes in dis-
ability after stroke: a comparison of four scales useful in clinical trials.
JRehabil Res Dev. 2003;40:18.
125. Kwon S, Hartzema AG, Duncan PW, Min-Lai S. Disability mea-
sures in stroke: relationship among the Barthel Index, the Functional
Independence Measure, and the Modified Rankin Scale. Stroke.
2004;35:918923.
126. Post Acute Care Payment Reform Demonstration. Centers for
Medicare and Medicaid Services Web site. http://www.cms.gov/
Medicare/Demonstration-Projects/DemoProjectsEvalRpts/Medicare-
Demonstrations-Items/CMS1201325.html. Accessed May 9, 2012.
127. Gage B, Morley M, Constantine R, Spain P, Allpress J, Garrity M,
Ingber M. Examining Relationships in an Integrated Hospital System:
Final Report. Washington, DC: Department of Health and Human
Services; 2008.
128. Buntin MB, Escarce JJ, Hoverman C, Paddock SM, Totten M, Wynn
BO. Effects of Payment Changes on Trends in Access to Post-Acute Care.
Santa Monica, CA: RAND Corporation; 2005.
129. Fonarow GC, Reeves MJ, Smith EE, Saver JL, Zhao X, Olson DW,
Hernandez AF, Peterson ED, Schwamm LH; GWTG-Stroke Steering
Committee and Investigators. Characteristics, performance measures,
and in-hospital outcomes of the first one million stroke and transient
ischemic attack admissions in Get With The Guidelines-Stroke. Circ
Cardiovasc Qual Outcomes. 2010;3:291302.
130. Westert GP, Lagoe RJ, Keskimki I, Leyland A, Murphy M. An interna-
tional study of hospital readmissions and related utilization in Europe
and the USA. Health Policy. 2002;61:269278.
131. Denti L, Scoditti U, Tonelli C, Saccavini M, Caminiti C, Valcavi R,
Benatti M, Ceda GP. The poor outcome of ischemic stroke in very
old people: a cohort study of its determinants. JAm Geriatr Soc.
2010;58:1217.
132. Di Carlo A, Lamassa M, Baldereschi M, Pracucci G, Consoli D, Wolfe
CD, Giroud M, Rudd A, Burger I, Ghetti A, Inzitari D; European
BIOMED Study of Stroke Care Group. Risk factors and outcome of sub-
types of ischemic stroke: data from a multicenter multinational hospital-
based registry: the European Community Stroke Project. JNeurol Sci.
2006;244:143150.
133. Finocchi C, Gandolfo C, Gasparetto B, Del Sette M, Croce R, Loeb C.
Value of early variables as predictors of short-term outcome in patients
with acute focal cerebral ischemia. Ital J Neurol Sci. 1996;17:341346.
134. German Stroke Study Collaboration. Predicting outcome after acute
ischemic stroke: an external validation of prognostic models. Neurology.
2004;62:581585.
135. Kaarisalo MM, Rih I, Sivenius J, Immonen-Rih P, Lehtonen A,
Sarti C, Mhnen M, Torppa J, Tuomilehto J, Salomaa V. Diabetes
worsens the outcome of acute ischemic stroke. Diabetes Res Clin Pract.
2005;69:293298.
136. Lin S, Wu B, Hao ZL, Kong FY, Tao WD, Wang DR, He S, Liu M.
Characteristics, treatment and outcome of ischemic stroke with atrial
fibrillation in a Chinese hospital-based stroke study. Cerebrovasc Dis.
2011;31:419426.
137. Liou LM, Chen CF, Guo YC, Cheng HL, Lee HL, Hsu JS, Lin RT, Lin
HF. Cerebral white matter hyperintensities predict functional stroke out-
come. Cerebrovasc Dis. 2010;29:2227.
138. Maguire J, Thakkinstian A, Levi C, Lincz L, Bisset L, Sturm J, Scott
R, Whyte S, Attia J. Impact of COX-2 rs5275 and rs20417 and GPIIIa
rs5918 polymorphisms on 90-day ischemic stroke functional outcome: a
novel finding. JStroke Cerebrovasc Dis. 2011;20:134144.
139. McCarron MO, Muir KW, Nicoll JA, Stewart J, Currie Y, Brown
K, Bone I. Prospective study of apolipoprotein E genotype and
functional outcome following ischemic stroke. Arch Neurol.
2000;57:14801484.
140. N Chrinn D, Callaly EL, Duggan J, Merwick , Hannon N, Sheehan
, Marnane M, Horgan G, Williams EB, Harris D, Kyne L, McCormack
PM, Moroney J, Grant T, Williams D, Daly L, Kelly PJ. Association be-
tween acute statin therapy, survival, and improved functional outcome
after ischemic stroke: the North Dublin Population Stroke Study. Stroke.
2011;42:10211029.
141. Ntaios G, Egli M, Faouzi M, Michel P. J-shaped association between
serum glucose and functional outcome in acute ischemic stroke. Stroke.
2010;41:23662370.
142. Rundek T, Nielsen K, Phillips S, Johnston KC, Hux M, Watson D; GAIN
Americas Investigators. Health care resource use after acute stroke in the
Glycine Antagonist in Neuroprotection (GAIN) Americas trial. Stroke.
2004;35:13681374.
143. Santos-Garca D, Blanco M, Serena J, Arias S, Milln M, Rodrguez-
Yez M, Leira R, Dvalos A, Castillo J. Brachial arterial flow mediated
dilation in acute ischemic stroke. EurJ Neurol. 2009;16:684690.
144. Sharma VK, Tsivgoulis G, Tan JH, Wong LY, Ong BK, Chan BP,
Teoh HL. Feasibility and safety of intravenous thrombolysis in mul-
tiethnic Asian stroke patients in Singapore. JStroke Cerebrovasc Dis.
2010;19:424430.
145. Tao WD, Kong FY, Hao ZL, Lin S, Wang DR, Wu B, Liu M. One-
year case fatality and disability after posterior circulation infarc-
tion in a Chinese hospital-based stroke study. Cerebrovasc Dis.
2010;29:376381.
146. Tsivgoulis G, Saqqur M, Sharma VK, Lao AY, Hill MD, Alexandrov AV;
CLOTBUST Investigators. Association of pretreatment blood pressure
with tissue plasminogen activator-induced arterial recanalization in acute
ischemic stroke. Stroke. 2007;38:961966.
147. Tsivgoulis G, Saqqur M, Sharma VK, Lao AY, Hoover SL, Alexandrov
AV; CLOTBUST Investigators. Association of pretreatment ASPECTS
scores with tPA-induced arterial recanalization in acute middle cerebral
artery occlusion. JNeuroimaging. 2008;18:5661.
148. Uyttenboogaart M, Koch MW, Koopman K, Vroomen PC, De Keyser
J, Luijckx GJ. Safety of antiplatelet therapy prior to intravenous throm-
bolysis in acute ischemic stroke. Arch Neurol. 2008;65:607611.
149. Weimar C, Kurth T, Kraywinkel K, Wagner M, Busse O, Haberl RL,
Diener HC; German Stroke Data Bank Collaborators. Assessment
of functioning and disability after ischemic stroke. Stroke.
2002;33:20532059.
150. Bamford JM, Sandercock PA, Warlow CP, Slattery J. Interobserver agree-
ment for the assessment of handicap in stroke patients. Stroke. 1989;
20:828.
151. Lees KR, Bath PM, Schellinger PD, Kerr DM, Fulton R, Hacke
W, Matchar D, Sehra R, Toni D; European Stroke Organization
Outcomes Working Group. Contemporary outcome measures in
acute stroke research: choice of primary outcome measure. Stroke.
2012;43:11631170.
152. Wilson JT, Hareendran A, Hendry A, Potter J, Bone I, Muir KW.
Reliability of the modified Rankin Scale across multiple raters: benefits
of a structured interview. Stroke. 2005;36:777781.

153. Duncan PW, Lai SM, Keighley J. Defining post-stroke recovery: impli-
cations for design and interpretation of drug trials. Neuropharmacology.
2000;39:835841.
154. Bath PM, Gray LJ, Collier T, Pocock S, Carpenter J; Optimising Analysis
of Stroke Trials (OAST) Collaboration. Can we improve the statistical
analysis of stroke trials? Statistical reanalysis of functional outcomes in
stroke trials. Stroke. 2007;38:19111915.
155. Management of Stroke Rehabilitation Working Group. VA/DoD Clinical
Practice Guideline for the Management of Stroke Rehabilitation.
Washington, DC: US Department of Veterans Affairs; October 2010.
156. Werner RM, Asch DA. The unintended consequences of publicly report-
ing quality information. JAMA. 2005;293:12391244.
157. Holloway RG, Quill TE. Mortality as a measure of quality: implications
for palliative and end-of-life care. JAMA. 2007;298:802804.
158. Schwamm LH, Pancioli A, Acker JE 3rd, Goldstein LB, Zorowitz
RD, Shephard TJ, Moyer P, Gorman M, Johnston SC, Duncan PW,
Gorelick P, Frank J, Stranne SK, Smith R, Federspiel W, Horton KB,
Magnis E, Adams RJ; American Stroke Associations Task Force on the
Development of Stroke Systems. Recommendations for the establish-
ment of stroke systems of care: recommendations from the American
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017
Katzan etal Ischemic Stroke Outcomes 27
Stroke Associations Task Force on the Development of Stroke Systems.
Stroke. 2005;36:690703.
159. Acker JE 3rd, Pancioli AM, Crocco TJ, Eckstein MK, Jauch EC,
Larrabee H, Meltzer NM, Mergendahl WC, Munn JW, Prentiss SM,
Sand C, Saver JL, Eigel B, Gilpin BR, Schoeberl M, Solis P, Bailey JR,
Horton KB, Stranne SK. Implementation strategies for emergency medi-
cal services within stroke systems of care: a policy statement from the
American Heart Association/American Stroke Association Expert Panel
on Emergency Medical Services Systems and the Stroke Council. Stroke.
2007;38:30973115.
160. Alberts MJ, Latchaw RE, Selman WR, Shephard T, Hadley MN,
Brass LM, Koroshetz W, Marler JR, Booss J, Zorowitz RD, Croft
JB, Magnis E, Mulligan D, Jagoda A, OConnor R, Cawley CM,
Connors JJ, Rose-DeRenzy JA, Emr M, Warren M, Walker MD; Brain
Attack Coalition. Recommendations for comprehensive stroke cen-
ters: a consensus statement from the Brain Attack Coalition. Stroke.
2005;36:15971616.
161. Freemantle N, Calvert M, Wood J, Eastaugh J, Griffin C. Composite
outcomes in randomized trials: greater precision but with greater uncer-
tainty? JAMA. 2003;289:25542559.
 Risk Adjustment of Ischemic Stroke Outcomes for Comparing Hospital Performance: A
Statement for Healthcare Professionals From the American Heart Association/American
Stroke Association
Irene L. Katzan, John Spertus, Janet Prvu Bettger, Dawn M. Bravata, Mathew J. Reeves, Eric E.
Smith, Cheryl Bushnell, Randall T. Higashida, Judith A. Hinchey, Robert G. Holloway, George
Downloaded from http://stroke.ahajournals.org/ by guest on September 6, 2017

Howard, Rosemarie B. King, Harlan M. Krumholz, Barbara J. Lutz and Robert W. Yeh

Stroke. published online January 23, 2014;

Stroke is published by the American Heart Association, 7272 Greenville Avenue, Dallas, TX 75231
Copyright 2014 American Heart Association, Inc. All rights reserved.
Print ISSN: 0039-2499. Online ISSN: 1524-4628

The online version of this article, along with updated information and services, is located on the
World Wide Web at:
http://stroke.ahajournals.org/content/early/2014/01/23/01.str.0000441948.35804.77

Permissions: Requests for permissions to reproduce figures, tables, or portions of articles originally published
in Stroke can be obtained via RightsLink, a service of the Copyright Clearance Center, not the Editorial Office.
Once the online version of the published article for which permission is being requested is located, click
Request Permissions in the middle column of the Web page under Services. Further information about this
process is available in the Permissions and Rights Question and Answer document.

Reprints: Information about reprints can be found online at:

http://www.lww.com/reprints

Subscriptions: Information about subscribing to Stroke is online at:

http://stroke.ahajournals.org//subscriptions/