Research

JAMA Internal Medicine | Original Investigation

Unrecognized Cardiovascular Emergencies

Among Medicare Patients
Daniel A. Waxman, MD, PhD; Hemal K. Kanzaria, MD, MSHPM; David L. Schriger, MD, MPH

Supplemental content
IMPORTANCE The Institute of Medicine described diagnostic error as the next frontier in
patient safety and highlighted a critical need for better measurement tools.

OBJECTIVES To estimate the proportions of emergency department (ED) visits attributable to

symptoms of imminent ruptured abdominal aortic aneurysm (AAA), acute myocardial
infarction (AMI), stroke, aortic dissection, and subarachnoid hemorrhage (SAH) that end in
discharge without diagnosis; to evaluate longitudinal trends; and to identify patient
characteristics independently associated with missed diagnostic opportunities.

DESIGN, SETTING, AND PARTICIPANTS This was a retrospective cohort study of all Medicare
claims for 2006 to 2014. The setting was hospital EDs in the United States. Participants
included all fee-for-service Medicare patients admitted to the hospital during 2007 to 2014
for the conditions of interest. Hospice enrollees and patients with recent skilled nursing
facility stays were excluded.

MAIN OUTCOMES AND MEASURES The proportion of potential diagnostic opportunities

missed in the ED was estimated using the difference between observed and expected ED
discharges within 45 days of the index hospital admissions as the numerator, basing expected
discharges on ED use by the same patients in earlier months. The denominator was estimated
as the number of recognized emergencies (index hospital admissions) plus unrecognized
emergencies (excess discharges).

RESULTS There were 1 561 940 patients, including 17 963 hospitalized for ruptured AAA,
304 980 for AMI, 1 181 648 for stroke, 19 675 for aortic dissection, and 37 674 for SAH. The
mean (SD) age was 77.9 (10.3) years; 8.9% were younger than 65 years, and 54.1% were
female. The proportions of diagnostic opportunities missed in the ED were as follows:
ruptured AAA (3.4%; 95% CI, 2.9%-4.0%), AMI (2.3%; 95% CI, 2.1%-2.4%), stroke (4.1%;
95% CI, 4.0%-4.2%), aortic dissection (4.5%; 95% CI, 3.9%-5.1%), and SAH (3.5%; 95% CI,
3.1%-3.9%). Longitudinal trends were either nonsignificant (AMI and aortic dissection) or
increasing (ruptured AAA, stroke, and SAH). Patient characteristics associated with
unrecognized emergencies included age younger than 65 years, dual eligibility for Medicare
and Medicaid coverage, female sex, and each of the following chronic conditions: end-stage
renal disease, dementia, depression, diabetes, cerebrovascular disease, hypertension,
coronary artery disease, and chronic obstructive pulmonary disease.

CONCLUSIONS AND RELEVANCE Among Medicare patients, opportunities to diagnose

ruptured AAA, AMI, stroke, aortic dissection, and SAH are missed in less than 1 in 20 ED
presentations. Further improvement may prove difficult.

Author Affiliations: Department of

Emergency Medicine, UCLA
(University of California, Los Angeles)
(Waxman, Schriger); RAND,
Santa Monica, California (Waxman);
Department of Emergency Medicine,
University of California, San Francisco
(Kanzaria).
Corresponding Author: Daniel A.
Waxman, MD, PhD, RAND,
JAMA Intern Med. doi:10.1001/jamainternmed.2017.8628 1776 Main St, Santa Monica, CA
Published online February 26, 2018. 90407 (dwaxman@rand.org).

(Reprinted) E1
© 2018 American Medical Association. All rights reserved.

Downloaded From: by a National University of Singapore User on 03/02/2018

 Research Original Investigation
A
ccurate and timely diagnosis is a cornerstone of high-
quality care. A report by the Institute of Medicine (IOM),
Improving Diagnosis in Health Care, calls diagnostic
error an underappreciated source of patient harm and “a blind
spot” in the patient safety movement.”1 The report cites the
inherent difficulty in measuring the incidence of diagnostic
error as an impediment to improvement and states that de-
veloping measurement tools should be an urgent research
priority.2 As motivational examples, the report presents sev-
eral vignettes of patients who are seen at the emergency de-
partment (ED) with symptoms of acute, life-threatening car-
diovascular emergencies.1
The IOM committee suggests that to quantify diagnostic er-
ror “one would need an estimate of the number of opportuni-
ties to make a diagnosis each year (denominator) and the num-
ber of times the diagnosis (health problem) is not made in an
accurate and timely manner or is not communicated to the
patient.”1(p7) Although the word error has the linguistic impli-
cation of fault, the committee distinguished between failure of
diagnostic process and the outcome of that process, choosing
an outcome-based definition of error as patient centric. They
note that one does not necessarily imply the other. For ex-
ample, a radiograph misread by a radiologist but correctly over-
read by the ordering physician would be a near miss but not a
diagnostic error. Conversely, if a patient is seen with an aortic
dissection and is discharged, then a diagnostic error occurred,
no matter how nonspecific the clinical presentation.
Defined this way, the frequency of diagnostic error can only
be measured retrospectively. If the goal is to identify specific
instances of diagnostic error, the presence of the disease dur-
ing the encounters in question would need to be adjudicated
after medical record review. However, to estimate the propor-
tion of diagnostic opportunities missed, we suggest that a sta-
tistical approach may offer a more accurate, reproducible, and
feasible alternative.
In this study, we consider the ED diagnosis of the follow-
ing 5 life-threatening emergencies: ruptured abdominal aor-
tic aneurysm (AAA), acute myocardial infarction (AMI), stroke,
aortic dissection, and subarachnoid hemorrhage (SAH). Adapt-
ing the IOM’s formulation, we measure the proportion of pa-
tients discharged home among those seen at the ED with symp-
toms retrospectively attributable to the underlying acute
cardiovascular pathology.
Methods
Study Population and Data Sources
We defined the study population as all fee-for-service (FFS)
Medicare patients newly diagnosed as having 1 of the 5 con-
ditions of interest during a hospitalization that began in the
ED during calendar years 2007 to 2014. The index conditions
were defined according to the coded principal discharge di-
agnosis (eTable 1 in the Supplement). To ensure that the pa-
thology was acute, that all preceding ED visits were cap-
tured, and that patients were not knowingly discharged home
despite the acute condition, index cases were excluded for the
following reasons: (1) lack of continuous FFS Medicare enroll-
E2 JAMA Internal Medicine Published online February 26, 2018 (Reprinted)
© 2018 American Medical Association. All rights reserved.
Downloaded From: by a National University of Singapore User on 03/02/2018
Unrecognized Cardiovascular Emergencies Among Medicare Patients
Key Points
Question Of emergency department visits attributable to
imminent ruptured abdominal aortic aneurysm, acute myocardial
infarction, stroke, aortic dissection, and subarachnoid hemorrhage,
what proportions end in discharge home without diagnosis?
Findings In this cohort study of Medicare claims, the proportion
of missed opportunities to diagnose these conditions in the
emergency department ranged from 2.3% (ruptured abdominal
aortic aneurysm) to 4.5% (aortic dissection). We found no evidence
for improvement across the 2007 to 2014 study time frame.
Meaning Among Medicare patients, opportunities to diagnose
these conditions in the emergency department are missed
infrequently, but further improvement may prove difficult.
ment in the 12 months preceding the index hospital admis-
sion, (2) previous enrollment in hospice, (3) a claim for a skilled
nursing facility stay within the preceding 30 days, (4) an ED
visit within the preceding 365 days where the index condi-
tion was listed as a diagnosis, and (5) the absence of
ED-specific charges (Medicare Provider Analysis Review File
[MedPAR]) or a matching outpatient ED claim (outpatient file)
indicating admission occurred via an ED.3 The data sources
were Medicare standard analytic files (100%) for the calendar
years 2006 to 2014 (allowing for a 1-year look-back period).
The study was approved by RAND’s institutional review
board. Informed consent was not applicable.
Conceptual Overview
We performed separate analyses for each of the 5 conditions.
Starting with the index hospital admission, we look back in time
to identify all ED visits in the preceding year. We assume that
the acute pathology giving rise to the index condition (eg, rup-
ture of a coronary plaque preceding AMI) begins in the hours,
days, or weeks before the index hospital admission. At some
point after the acute pathology appears, patients are seen at
the ED for symptoms that are in some way related (albeit some-
times nonspecific) and either are admitted to the hospital (or
transferred) and diagnosed or are discharged with an unrec-
ognized emergency. Those who are discharged return to the
ED one or more times and are eventually admitted.
We treat the presence of the underlying acute pathology at
ED discharge as unobservable in the data. Patients may coinci-
dentally be seen at the ED for problems unrelated to the car-
diovascular emergency at any time before the index hospital ad-
mission. We specify that the underlying pathology can start no
more than 45 days before the index hospital admission (based
on preliminary inspection of the data) and demonstrate in a sen-
sitivity analysis that results are robust to choices of shorter or
longer durations. We estimated the number of unrecognized
emergencies in each population as the number of “excess ED
discharges” within those 45 days. Excess discharges, defined
as the difference between the number observed and the num-
ber expected to occur, are based on the mean number of daily
ED discharges for the same patients earlier in the year. Logistic
regression was used to adjust for a slightly increasing rate with
each passing day owing to age and a survivorship bias (eg, an
jamainternalmedicine.com
 Unrecognized Cardiovascular Emergencies Among Medicare Patients
aggressive cancer is less likely to appear earlier in the year be-
cause we impose the constraint that all patients must survive
to the index hospital admission).
The primary study outcome (for each condition) is formu-
lated as the false-negative rate—the proportion of ED visits
where the diagnosis was not made (ie, the patient was dis-
charged)—among all visits where the acute pathology was
present. The numerator (unrecognized emergencies) is esti-
mated as the number of excess ED discharges. Because each
index patient has only one ED visit leading to correct diagno-
sis, we estimated the denominator as the sum of index
hospital admissions (recognized emergencies) plus excess
discharges (unrecognized emergencies).
Methodological Detail
We aggregated data across all available years. To evaluate lon-
gitudinal trends, we also performed yearly estimates per in-
dex hospital admission date. To calculate the expected num-
ber of discharges in the 45 days preceding the index hospital
admission, we expanded the data to 365 observations for each
patient. We fit a logistic regression model of whether an indi-
vidual had an ED discharge on any given day as a function of
the number of days to the index hospital admission, using ob-
servations that occurred 365 to 45 days before the index visit
to fit the model. We calculated expected discharges as the ag-
gregate of predicted probabilities for the 45 days previously
excluded. Because each patient’s observations were corre-
lated in time, only point estimates were used; we obtained 95%
CIs for the overall result via the bootstrap method.
To avoid counting official or unofficial transfers miscoded
as discharges, ED discharges were excluded if they occurred on
the same day as the index hospital admission (any) or if they oc-
curred 1 day prior and the discharge diagnosis was for the index
condition itself. Observation stays were treated as discharges if
patients were discharged home at the end of observation and
were treated as admissions if the patient was subsequently ad-
mitted to the hospital. As a sensitivity analysis, we show how re-
sults are affected by excluding ED visits that ended in observa-
tion and then discharge. We also show how results are affected
by casewise exclusion of patients hospitalized for any other rea-
son within 30 days of the index admission, to mitigate possible
confounding (see the Discussion section).
As an exploratory analysis meant to illustrate expanded ap-
plications of our approach, we measured multivariable associa-
tions between patient characteristics or socioeconomic factors
and unrecognized emergencies. To do so, we fit patient-level lo-
gistic models, with ED discharge during the 45-day preadmission
window (one or more) as the dependent variable, as a function
of age, sex, Medicare and Medicaid dual eligibility, race/ethnicity,
and a history of any of the following conditions (chosen a priori
on clinical grounds, defined by Chronic Conditions Data Ware-
house indicators provided by the Centers for Medicare & Med-
icaid Services4): end-stage renal disease, dementia, depression,
diabetes, hypertension, cerebrovascular disease, or ischemic
heart disease. We also included the number of ED discharges ear-
lier in the year (ie, the 320 days preceding the 45-day window)
as a covariate, to account for possible correlation between patient
characteristics and baseline ED use.
jamainternalmedicine.com
© 2018 American Medical Association. All rights reserved.
Downloaded From: by a National University of Singapore User on 03/02/2018
Original Investigation Research
Similarly, we identified ED discharge diagnoses occur-
ring with greater than expected frequency during the 45 days
before index hospital admissions compared with the preced-
ing 320 days. We report incidence rate ratios (IRRs) and exact
95% CIs for the 20 diagnoses that occur in greatest excess and
the 20 diagnoses with the highest IRRs.
Statistical Analysis
Analyses were performed with SAS Enterprise Guide (version 7.1;
SAS Institute Inc), hosted by the Centers for Medicare & Medic-
aid Services Virtual Research Data Center, and in Stata (version
13.1; StataCorp LP). We created 95% CIs for our main estimates
by clustered bootstrap at the level of the patient, sampling indi-
viduals with replacement over 1000 replications, fitting the
logistic model, and performing the post-fit calculations for each
replication, using the 2.5th and 97.5th percentiles of these results
for the 95% CI. To evaluate longitudinal trends, we fit a linear re-
gression (with robust SEs) of the logarithm of each year’s estimate
as a function of calendar year, and we report the mean annual
percentage change with the exponentiated coefficient.
Results
We identified 1 873 207 Medicare FFS beneficiaries with 1 of
the 5 conditions of interest. Of these, 152 484 were excluded
because index hospital admission via the ED could not be es-
tablished, 146 621 because they had previously been enrolled
in hospice (n = 19 852) and/or had a skilled nursing facility claim
within the prior 30 days (n = 131 220), and 12 162 because they
had a previous ED discharge for the index diagnosis. For the 5
conditions in aggregate, the mean (SD) age was 77.9 (10.6) years.
The proportion younger than 65 years ranged from 3.1% for rup-
tured AAA to 14.5% for SAH (8.9% overall) (Table 1). Mortality
during the index hospital admission ranged from 5.2% for
stroke to 48.2% for ruptured AAA.
Among the 1 561 940 individuals remaining after exclu-
sions, there were 902 159 ED discharges to home 1 to 365 days
before the index hospital admission. Histograms showing the
frequency of ED discharges as a function of the number of
elapsed days before the index hospital admission are shown
in Figure 1 and in the eFigure in the Supplement.
Table 2 lists the main study results. Estimates for the pro-
portion of ED diagnostic opportunities that were missed are
as follows: ruptured AAA (3.4%; 95% CI, 2.9%-4.0%), AMI
(2.3%; 95% CI, 2.1% -2.4%), stroke (4.1%; 95% CI, 4.0%-
4.2%), aortic dissection (4.5%; 95% CI, 3.9%-5.1%), and SAH
(3.5%; 95% CI, 3.1%-3.9%).
Figure 2 shows longitudinal trends. The mean annual
changes in the proportion of diagnostic opportunities missed
(ie, the relative change) were as follows: ruptured AAA (8.7%
per year increase; 95% CI, 1.5%-16.4%), AMI (2.3% per year de-
crease; 95% CI, 6.0% decrease to 1.6% increase), stroke (1.7%
per year increase; 95% CI, 1.0%-2.4%), aortic dissection (1.1%
per year decrease; 95% CI, 4.8% decrease to 2.7% increase),
and SAH (6.5% per year increase; 95% CI, 0.0%-13.5%).
Results of sensitivity analyses are listed in eTables 2, 3,
and 4 in the Supplement. Exclusion of patients who
(Reprinted) JAMA Internal Medicine Published online February 26, 2018 E3
 Research Original Investigation Unrecognized Cardiovascular Emergencies Among Medicare Patients

Table 1. Characteristics of Patients, by Index Conditiona

Ruptured Acute
Abdominal Myocardial Aortic Subarachnoid
Aortic Aneurysm Infarction Stroke Dissection Hemorrhage
Variable (n = 17 963) (n = 304 980) (n = 1 181 648) (n = 19 675) (n = 37 674)
Age, mean (SD), y 79.0 (8.4) 75.1 (10.7) 78.7 (10.2) 76.5 (10.6) 74.6 (11.6)
Age <65 y, No. (%) 556 (3.1) 38 481 (12.6) 92 875 (7.9) 1947 (9.9) 5463 (14.5)
Female sex, No. (%) 5763 (32.1) 133 162 (43.7) 671 266 (56.8) 10 280 (52.2) 24 677 (65.5)
Medicare and Medicaid 2578 (14.4) 59 993 (19.7) 278 897 (23.6) 3755 (19.1) 9357 (24.8)
dual eligibility, No. (%)
Race/ethnicity, No. (%)
White, not Hispanic 16 293 (90.7) 263 225 (86.3) 953 830 (80.7) 15 709 (79.8) 29 155 (77.4)
Black 861 (4.8) 19 218 (6.3) 138 527 (11.7) 2532 (12.9) 4149 (11.0)
Asian/Pacific Islander 239 (1.3) 4700 (1.5) 21 123 (1.8) 508 (2.6) 1162 (3.1)
Hispanic 397 (2.2) 14 210 (4.7) 55 958 (4.7) 708 (3.6) 2630 (7.0)
Other/unknown 173 (1.0) 3627 (1.2) 12 210 (1.0) 218 (1.1) 578 (1.5)
Chronic conditions, No. (%)
End-stage renal disease 523 (2.9) 7412 (2.4) 28 518 (2.4) 790 (4.0) 1037 (2.8)
Dementia 2415 (13.4) 37 163 (12.2) 270 389 (22.9) 2738 (13.9) 6434 (17.1)
Depression 14 005 (78.0) 221 298 (72.6) 790 427 (66.9) 13 494 (68.6) 24 864 (66.0)
Diabetes 4893 (27.2) 115 717 (37.9) 526 417 (44.5) 5594 (28.4) 13 073 (34.7)
Stroke/transient 3487 (19.4) 50 694 (16.6) 384 752 (32.6) 4077 (20.7) 9636 (25.6)
ischemic attack
Hypertension 14 729 (82.0) 245 875 (80.6) 1 059 969 (89.7) 17 537 (89.1) 30 881 (82.0)
Coronary artery disease 10 791 (60.1) 178 439 (58.5) 753 747 (63.8) 12 268 (62.4) 20 005 (53.1)
Chronic obstructive 7922 (44.1) 87 987 (28.9) 393 489 (33.3) 7496 (38.1) 11 641 (30.9)
pulmonary disease
Hospital mortality, No. (%)
Died during index 8666 (48.2) 33 612 (11.0) 61 398 (5.2) 4617 (23.5) 10 862 (28.8)
hospital admission
a
Each column represents a distinct study population. For each population, the number and proportion of patients with each characteristic are shown. In-hospital
mortality rates are given in the last row.

Figure 1. Emergency Department (ED) Discharges by Time Before Index Visit for Acute Myocardial Infarction

1600

Observed ED discharges
Expected ED discharges

1200
ED Discharges per Day

800

400

0
365 182 45 1
Time Before Index Hospital Admission, d

The timing of ED discharges among the 304 980 patients ultimately diagnosed
as having acute myocardial infarction. The height of each vertical bar represents
the observed number of daily ED discharges, and the regression line represents
the number expected. The logistic regression was fitted using the unshaded
portion of the graph (ie, days 365-46). The number of unrecognized diagnostic
opportunities is represented as the net difference between the number of ED
discharges observed (vertical bars) and the number expected (regression line)
during the 45 days preceding the index hospital admission (shaded area).
eFigure 1 in the Supplement provides a similar illustration of the other
4 conditions.

E4 JAMA Internal Medicine Published online February 26, 2018 (Reprinted) jamainternalmedicine.com

© 2018 American Medical Association. All rights reserved.

Downloaded From: by a National University of Singapore User on 03/02/2018

 Unrecognized Cardiovascular Emergencies Among Medicare Patients Original Investigation Research

Table 2. Results Summary (Aggregate Data for Index Hospital Admissions, 2007 to 2014)a
Potential Diagnostic
Index Observed Expected Opportunities Missed,
Variable Patients ED Discharges ED Discharges % (95% CI)
Ruptured abdominal aortic aneurysm 17 963 1603 970 3.4 (2.9-4.0)
Acute myocardial infarction 304 980 25 622 18 483 2.3 (2.1-2.4)
Stroke 1 181 648 142 743 92 348 4.1 (4.0-4.2)
Aortic dissection 19 675 2482 1565 4.5 (3.9-5.1)
Subarachnoid hemorrhage 37 674 4245 2871 3.5 (3.1-3.9)
a
Summarized are the principal study results, pooling index hospital admissions sum of excess discharges plus index hospital admissions, representing the
across 8 years. The total number of patients admitted for each disease is ratio of missed diagnostic opportunities to the sum of all diagnostic
shown, followed by the number of emergency department (ED) discharges opportunities (recognized and unrecognized emergencies). The 95% CIs were
observed to occur within 45 days of an index hospital admission, and the point calculated by clustered bootstrap as described in the Methodological Detail
estimate for the number expected for that same interval. The final column is subsection of the Methods section.
calculated as the ratio of excess ED discharges (observed and expected) to the

Figure 2. Unrecognized Emergencies by Type and Year

A Ruptured abdominal aortic aneurysm B Acute myocardial infarction (≈38 123 per year) C Stroke (≈147 706 per year)
(≈2245 per year)
7.5 7.5 7.5
Opportunities Missed, %

Opportunities Missed, %

Opportunities Missed, %
Potential Diagnostic

Potential Diagnostic

Potential Diagnostic
5.0 5.0 5.0

2.5 2.5 2.5

0 0 0
2007 2008 2009 2010 2011 2012 2013 2014 2007 2008 2009 2010 2011 2012 2013 2014 2007 2008 2009 2010 2011 2012 2013 2014
Year Year Year

D Aortic dissection (≈2459 per year) E Subarachnoid hemorrhage (≈4709 per year)
7.5 7.5
Opportunities Missed, %

Opportunities Missed, %
Potential Diagnostic

Potential Diagnostic

5.0 5.0

2.5 2.5

0 0
2007 2008 2009 2010 2011 2012 2013 2014 2007 2008 2009 2010 2011 2012 2013 2014
Year Year

Shown are longitudinal trends. Yearly point estimates and 95% CIs are calculated in the same manner as for the results presented in Table 2 but using the subset of
patients whose index hospital admission began during the calendar year.

h a d b e e n a d m itte d to t h e h o s p it a l w it h i n 3 0 d ay s Associations With Patient Characteristics

before their index event (and eliminating ED discharges In multivariable analyses that adjusted for demographics,
preceded by hospital admissions) reduced estimates chronic conditions, and the number of visits at baseline, ex-
slightly (eTable 2 in the Supplement). Exclusion of patients cess ED discharges within 45 days of index hospital admis-
admitted under observation status and then discharged sion were positively correlated with age younger than 65 years,
home (90 816 [10.1% of all ED discharges to home]) also female sex, Medicare and Medicaid dual eligibility, and a his-
reduced estimates slightly (eTable 3 in the Supplement). tory of end-stage renal disease, dementia, depression, stroke
Using 30 or 60 days rather than 45 days as the duration of or transient ischemic attack (TIA), hypertension, coronary ar-
the pre–index event window for the observed vs expected tery disease, and chronic obstructive pulmonary disease
calculation had negligible associations (eTable 4 in the (Table 3). The associations with race/ethnicity and diabetes var-
Supplement). ied across conditions.

jamainternalmedicine.com (Reprinted) JAMA Internal Medicine Published online February 26, 2018 E5

© 2018 American Medical Association. All rights reserved.

Downloaded From: by a National University of Singapore User on 03/02/2018

 Research Original Investigation Unrecognized Cardiovascular Emergencies Among Medicare Patients

Table 3. Multivariable Model Predicting Emergency Department (ED) Discharge Within 45 Days of Index Hospital Admissiona
Ruptured Acute
Abdominal Myocardial Aortic Subarachnoid
Variable Aortic Aneurysm Infarction Stroke Dissection Hemorrhage
Age, yb
<65 1.27 (0.94-1.73) 1.40 (1.33-1.48) 1.38 (1.35-1.42) 1.33 (1.09-1.62) 1.69 (1.48-1.92)
70-74 0.73 (0.59-0.91) 0.94 (0.90-1.00) 0.93 (0.90-0.95) 0.98 (0.81-1.17) 0.97 (0.85-1.11)
75-79 0.75 (0.61-0.93) 0.96 (0.91-1.02) 0.89 (0.87-0.92) 0.89 (0.74-1.06) 0.97 (0.85-1.11)
80-84 0.67 (0.54-0.83) 1.03 (0.98-1.09) 0.90 (0.88-0.92) 0.98 (0.81-1.17) 0.96 (0.84-1.10)
>84 0.83 (0.68-1.02) 1.05 (0.99-1.11) 0.92 (0.90-0.94) 0.85 (0.70-1.02) 0.94 (0.82-1.08)
Sexc
Female 1.25 (1.11-1.41) 1.14 (1.11-1.17) 1.12 (1.11-1.14) 1.19 (1.08-1.32) 1.00 (0.92-1.08)
Race/ethnicityd
Asian/Pacific 0.78 (0.45-1.35) 0.67 (0.59-0.77) 0.68 (0.65-0.72) 0.65 (0.46-0.93) 0.52 (0.40-0.69)
Black 1.35 (1.08-1.70) 1.18 (1.12-1.24) 1.09 (1.07-1.11) 0.93 (0.80-1.07) 1.00 (0.90-1.12)
Hispanic 1.16 (0.82-1.66) 0.91 (0.85-0.97) 0.90 (0.88-0.93) 1.03 (0.81-1.31) 0.91 (0.79-1.05)
Other/unknown 1.05 (0.60-1.85) 1.01 (0.88-1.14) 1.07 (1.01-1.14) 1.19 (0.78-1.82) 0.87 (0.65-1.17)
Medicare and Medicaid
dual eligibilitye
Dually eligible 1.23 (1.05-1.42) 1.40 (1.35-1.45) 1.24 (1.22-1.25) 1.33 (1.17-1.50) 1.29 (1.18-1.41)
Chronic conditionsf
End-stage renal disease 1.59 (1.22-2.07) 1.60 (1.49-1.71) 1.53 (1.48-1.58) 1.71 (1.40-2.08) 1.77 (1.51-2.09)
Dementia 1.22 (1.04-1.43) 1.22 (1.17-1.27) 1.17 (1.15-1.19) 1.10 (0.96-1.26) 1.21 (1.10-1.34)
Depression 1.27 (1.12-1.45) 1.32 (1.28-1.37) 1.23 (1.21-1.25) 1.31 (1.18-1.45) 1.27 (1.18-1.38)
Diabetes 0.96 (0.85-1.09) 1.03 (1.00-1.07) 1.05 (1.04-1.07) 0.99 (0.89-1.11) 1.10 (1.01-1.18)
Stroke/transient 0.94 (0.82-1.09) 1.14 (1.10-1.18) 1.25 (1.23-1.26) 1.14 (1.02-1.28) 1.28 (1.18-1.38)
ischemic attack
Hypertension 1.61 (1.30-1.98) 1.40 (1.33-1.47) 1.36 (1.33-1.40) 1.39 (1.14-1.70) 1.16 (1.04-1.30)
Coronary artery disease 1.30 (1.13-1.49) 1.43 (1.38-1.48) 1.14 (1.12-1.16) 1.19 (1.07-1.33) 1.14 (1.05-1.24)
Chronic obstructive 1.39 (1.23-1.56) 1.30 (1.26-1.34) 1.23 (1.21-1.24) 1.27 (1.15-1.40) 1.11 (1.03-1.20)
pulmonary disease
Baseline ED useg
No. of ED discharges 1.40 (1.30-1.50) 1.36 (1.34-1.39) 1.26 (1.25-1.27) 1.24 (1.17-1.31) 1.33 (1.27-1.38)
365-46 d before index
event (per discharge)
a b
Each column shows results (adjusted odds ratios) for a disease-specific Reference group is 65 to 69 years.
multivariable logistic regression model. The dependent variable, whether an c
Dichotomous vs male.
individual had one or more ED discharges within the 45 d preceding the index d
Reference group is white, non-Hispanic.
hospital admission, is modeled as a function of age, sex, Medicare and
e
Medicaid dual eligibility, race/ethnicity, the chronic conditions listed, and the Dichotomous vs not Medicaid eligible.
number of ED discharges in the 365 to 46 d before the index hospital f
Dichotomous condition vs no condition for each.
admission (which adjusts for an individual’s propensity to visit the ED for g
Continuous per ED visit.
reasons unrelated to the acute cardiovascular emergency). Variables were
chosen a priori, based on previously published studies, clinical plausibility, and
availability.

Discharge Diagnoses
Emergency department discharge diagnoses in greatest
excess during the 45 days before admission were as follows:
abdominal pain (unspecified site) for ruptured AAA (66
excess discharges; IRR for days 1-45 vs 46-365, 6.7; 95% CI,
4.5-9.4), chest pain (unspecified) for AMI (1194 excess dis-
charges; IRR, 2.8; 95% CI, 2.6-2.9), unspecified transient
cerebral ischemia for stroke (4532 excess discharges; IRR,
5.2; 95% CI, 5.0-5.4), chest pain (unspecified) for aortic dis-
section (149 excess discharges; IRR, 3.9; 95% CI, 3.3-4.7),
and headache for SAH (290 excess discharges; IRR, 5.7; 95%
CI, 4.9-6.6). eTable 5 in the Supplement lists the 20 diagno-
ses in greatest excess and the 20 diagnoses with the highest
IRR, for each condition.
Discussion
This study introduces what is to our knowledge a novel ap-
proach to measuring missed diagnostic opportunities in the ED.
Among Medicare patients with the 5 diseases studied, we esti-
mate that between 2.3% (AMI) and 4.5% (aortic dissection) of
ED visits for symptoms relating to an imminent emergency end
in discharge home without diagnosis. In a multivariable analy-
sis, we demonstrate that patients who are younger than 65 years,
female, or poor (ie, dually eligible for Medicare and Medicaid)
or those who have chronic medical conditions were at in-
creased risk. We see no evidence of improvement during the
time frame of this study from 2007 to 2014.

E6 JAMA Internal Medicine Published online February 26, 2018 (Reprinted) jamainternalmedicine.com

© 2018 American Medical Association. All rights reserved.

Downloaded From: by a National University of Singapore User on 03/02/2018

 Unrecognized Cardiovascular Emergencies Among Medicare Patients
Direct comparisons with previous reports are compli-
cated by differing definitions of outcomes and metrics.5,6
For example, some studies 6,7 define missed AMI as the
actual presence of an AMI at the time of ED discharge.
Herein, we define each condition as the presence of the
underlying acute pathology, where a ruptured plaque, an
intimal tear, a TIA, or a sentinel bleed would be counted. As
recommended in the IOM report cited at the beginning of
this article, we use the proportion of diagnostic opportuni-
ties missed (ie, the false-negative rate) as our metric of inter-
est. Compared with commonly used alternatives, our formu-
lation has the advantage of not being conditional on
patients’ propensity to visit the ED, allowing comparison
between populations or health systems.
We relax several assumptions common to previous work
and demonstrate that they are often violated. Whereas it has
been typical to consider ED discharges only within 1 or 2 weeks
of an index hospital admission,8-12 we took a data-driven ap-
proach and found that excess visits can be detected earlier.
Older literature supports this finding13-15; however, in an era
when patients are quicker to seek emergency care, the possi-
bility that acute cardiovascular pathology can remain undi-
agnosed for weeks may not be fully appreciated. A longer look-
back period amplifies the importance of accounting for
coincidental ED visits, which few previous studies do di-
rectly. We found that approximately two-thirds of the ED vis-
its within the 45 days preceding an index hospital admission
can be attributed to the population’s baseline rate of ED use.
Failure to account for baseline use would substantially over-
estimate the frequency of events.
The ED discharge diagnosis has often been used to retro-
spectively adjudicate whether an unrecognized emergency was
present at the time of an ED discharge.8,9,12,16,17 In our analy-
sis, we found that the most common diagnoses associated with
excess visits are what one might expect (eg, unspecified ab-
dominal pain for ruptured AAA), but we also found that seem-
ingly unrelated diagnoses are common. Published accounts of
high-profile diagnostic errors support this contention. For
example, the playwright Jonathan Larson was discharged from
2 EDs in the week before his death from an undiagnosed
aortic dissection, with diagnoses of “food poisoning” and
“viral syndrome.”18 While it is clear from subsequent investi-
gation that his ED visits were prompted by symptoms of
the dissection, discharge diagnosis alone could not have
distinguished between misdiagnosis and coincidence.
Limitations
Our method relies on a key assumption that nothing other than
the acute cardiovascular pathology should increase ED visits
just before the index hospital admission. One can imagine con-
founding by other factors that prompt ED visits and also cause
cardiovascular emergency to occur. For example, vascular sur-
gery might prompt ED visits for wound complications and may
increase the risk for postoperative myocardial infarction or
stroke. Confounding conditions would only influence results
if they arise acutely. For example, psychological stress might
precipitate both ED visits and cardiovascular emergencies, but
would be accounted for in the baseline rate of ED use if chronic.
jamainternalmedicine.com
© 2018 American Medical Association. All rights reserved.
Downloaded From: by a National University of Singapore User on 03/02/2018
Original Investigation Research
Because the most plausible confounders (particularly high-
risk surgery) would usually involve a hospitalization, we tested
the association of excluding patients whose index hospital ad-
mission was preceded by any other admission within 30 days,
and only a small association with results was found. To the
extent that confounding remains, it would bias estimates up-
ward (ie, the true proportion of emergencies unrecognized
would be lower than what we report).
In the other direction of bias (toward underestimating the
diagnostic error rate), we do not account for patients who die
before they are admitted to a hospital. This is a limitation of
most previous work on the topic. Future study might take the
approach of treating outpatient deaths as index events, al-
though the inaccuracy of death certificate diagnoses (particu-
larly for out-of-hospital deaths) presents a challenge. 19
Another limitation is that we do not account for diagnoses
that are not appreciated in the ED but made after admission
(or after death).
Our task was simplified by choosing to focus on cardio-
vascular emergencies generally understood to mandate hos-
pital admission from the ED (unless patients’ primary goal is
palliation). However, we found TIA to be among ED discharge
diagnoses occurring in excess in the weeks before stroke hos-
pital admissions, suggesting that for TIA the risk reduction
achieved with hospitalization might not always be perceived
to be worth the costs, financial or otherwise. Stroke esti-
mates must be interpreted with that caveat.
Conclusions
Among Medicare patients, opportunities to diagnose rup-
tured AAA, AMI, stroke, aortic dissection, and SAH are missed
in less than 1 in 20 ED presentations. Further improvement may
prove difficult.
The frequency of unrecognized emergencies can only be
estimated retrospectively; therefore, the call for better mea-
surement tools must be answered with studies such as this
one. Our study was not designed to consider trade-offs
between diagnostic sensitivity and specificity and thus can-
not address clinical decision making. However, the absence
of improvement over the 8-year time frame of the study
raises the question of whether our ability to diagnose these
acute emergencies has reached a plateau. Some researchers
have suggested a natural asymptote, where the costs or
risks of seeking additional diagnostic certainty become
prohibitive. 20,21 Studies of organization-level variation
aimed at identifying high-performing or low-performing
outliers might help us understand whether we can do better.
Regardless, measurement is important because technologi-
cal or practice innovation might improve or degrade diag-
nostic sensitivity in the future. Cardiovascular emergencies
provide a straightforward example of how surveillance for
excess ED visits can be used to monitor diagnostic error,
much as it has been used in other public health contexts.22
More widespread application of probabilistic approaches—to
other diagnoses and to other care venues—will require
further methodological innovation but seems inevitable.
(Reprinted) JAMA Internal Medicine Published online February 26, 2018 E7
 Research Original Investigation
ARTICLE INFORMATION
Accepted for Publication: December 18, 2017.
Published Online: February 26, 2018.
doi:10.1001/jamainternmed.2017.8628
Author Contributions: Dr Waxman had full access
to all of the data in the study and takes
responsibility for the integrity of the data and the
accuracy of the data analysis.
Study concept and design: All authors.
Acquisition, analysis, or interpretation of data:
All authors.
Drafting of the manuscript: All authors.
Critical revision of the manuscript for important
intellectual content: All authors.
Statistical analysis: Waxman, Schriger.
Obtained funding: Waxman.
Administrative, technical, or material support:
Kanzaria.
Study supervision: Waxman.
Conflict of Interest Disclosures: Dr Schriger
received salary support through an unrestricted
grant from the Korein Foundation. No other
conflicts are reported.
Funding/Support: RAND provided programming
support for this project. Data access for Dr Waxman
was supported through an interagency agreement
between the Office of the Assistant Secretary for
Planning and Evaluation (ASPE) (US Department of
Health & Human Services) and the Centers for
Medicare & Medicaid Services for the research
and analysis.
Role of the Funder/Sponsor: The funding sources
had no role in the design and conduct of the study;
collection, management, analysis, and
interpretation of the data; preparation, review, or
approval of the manuscript; and decision to submit
the manuscript for publication.
Additional Contributions: Asa Wilks, MPA,
provided programming assistance, and Carolyn
Rutter, PhD, helped develop the methods at an
early phase in the project. Both are affiliated with
RAND. No compensation was received.
REFERENCES
1. Institute of Medicine, National Academies of
Sciences, Engineering, and Medicine. Improving
E8 JAMA Internal Medicine Published online February 26, 2018 (Reprinted)
© 2018 American Medical Association. All rights reserved.
Downloaded From: by a National University of Singapore User on 03/02/2018
Unrecognized Cardiovascular Emergencies Among Medicare Patients
Diagnosis in Health Care. Washington, DC: National
Academies Press; 2015.
2. McGlynn EA, McDonald KM, Cassel CK.
Measurement is essential for improving diagnosis
and reducing diagnostic error: a report from the
Institute of Medicine. JAMA. 2015;314(23):2501-2502.
3. Research Data Assistance Center. How to
identify hospital claims for emergency room visits
in the Medicare claims data. https://www.resdac
.org/resconnect/articles/144. Published July 30,
2015. Accessed January 23, 2018.
4. Centers for Medicare & Medicaid Services.
Chronic Conditions Data Warehouse.
https://www.ccwdata.org/web/guest/condition
-categories. Accessed July 20, 2017.
5. Pope JH, Aufderheide TP, Ruthazer R, et al.
Missed diagnoses of acute cardiac ischemia in the
emergency department. N Engl J Med. 2000;342
(16):1163-1170.
6. McCarthy BD, Beshansky JR, D’Agostino RB,
Selker HP. Missed diagnoses of acute myocardial
infarction in the emergency department: results
from a multicenter study. Ann Emerg Med. 1993;22
(3):579-582.
7. Lee TH, Rouan GW, Weisberg MC, et al. Clinical
characteristics and natural history of patients with
acute myocardial infarction sent home from the
emergency room. Am J Cardiol. 1987;60(4):219-224.
8. Wilson M, Welch J, Schuur J, O’Laughlin K, Cutler
D. Hospital and emergency department factors
associated with variations in missed diagnosis and
costs for patients age 65 years and older with acute
myocardial infarction who present to emergency
departments. Acad Emerg Med. 2014;21(10):1101-
1108.
9. Schull MJ, Vermeulen MJ, Stukel TA. The risk of
missed diagnosis of acute myocardial infarction
associated with emergency department volume.
Ann Emerg Med. 2006;48(6):647-655.
10. Chua M, Ibrahim I, Neo X, Sorokin V, Shen L, Ooi
SB. Acute aortic dissection in the ED: risk factors
and predictors for missed diagnosis. Am J Emerg Med.
2012;30(8):1622-1626.
11. Vermeulen MJ, Schull MJ. Missed diagnosis of
subarachnoid hemorrhage in the emergency
department. Stroke. 2007;38(4):1216-1221.
12. Moy E, Barrett M, Coffey R, Hines AL,
Newman-Toker DE. Missed diagnoses of acute
myocardial infarction in the emergency
department: variation by patient and facility
characteristics. Diagnosis (Berl). 2014;2(1):29-40.
Accessed January 19, 2018. doi:10.1515/dx-2014-0053
13. Solomon HA, Edwards AL, Killip T. Prodromatan
acute myocardial infarction. Circulation. 1969;40
(4):463-471.
14. Kassell NF, Kongable GL, Torner JC, Adams HP
Jr, Mazuz H. Delay in referral of patients with
ruptured aneurysms to neurosurgical attention.
Stroke. 1985;16(4):587-590.
15. Adams HP Jr, Jergenson DD, Kassell NF, Sahs
AL. Pitfalls in the recognition of subarachnoid
hemorrhage. JAMA. 1980;244(8):794-796.
16. Newman-Toker DE, Moy E, Valente E, Coffey R,
Hines AL. Missed diagnosis of stroke in the
emergency department: a cross-sectional analysis
of a large population-based sample. Diagnosis (Berl)
. 2014;1(2):155-166.
17. Kowalski RG, Claassen J, Kreiter KT, et al. Initial
misdiagnosis and outcome after subarachnoid
hemorrhage. JAMA. 2004;291(7):866-869.
18. Nicholson J. State faults hosps for “Rent”
tragedy. New York Daily News. December 13, 1996.
http://www.nydailynews.com/archives/news/state
-faults-hosps-rent-tragedy-article-1.743185.
Accessed January 23, 2018.
19. Obermeyer Z, Cohn B, Wilson M, Jena AB,
Cutler DM. Early death after discharge from
emergency departments: analysis of national US
insurance claims data. BMJ. 2017;356:j239.
20. Graber M, Gordon R, Franklin N. Reducing
diagnostic errors in medicine: what’s the goal? Acad
Med. 2002;77(10):981-992.
21. Newman-Toker DE, McDonald KM, Meltzer DO.
How much diagnostic safety can we afford, and
how should we decide? A health economics
perspective. BMJ Qual Saf. 2013;22(suppl 2):ii11-ii20.
22. Heffernan R, Mostashari F, Das D, Karpati A,
Kulldorff M, Weiss D. Syndromic surveillance in
public health practice, New York City. Emerg Infect
Dis. 2004;10(5):858-864.
jamainternalmedicine.com