Clinical Nutrition xxx (2015) 1e5

Contents lists available at ScienceDirect

Clinical Nutrition
journal homepage: http://www.elsevier.com/locate/clnu

Original article

Identifying critically-ill patients who will benet most from

nutritional therapy: Further validation of the modied NUTRIC
nutritional risk assessment tool
Adam Rahman a, b, Rana M. Hasan a, Ravi Agarwala c, Claudio Martin a, d, e,
Andrew G. Day f, Daren K. Heyland f, g, h, *
a

Department of Medicine, University of Western Ontario, London, Ontario, Canada

Gastroenterology, St. Joseph's Healthcare Centre/London Health Sciences Centre, Canada
Department of Anesthesia, Section of Critical Care Medicine, Wake Forest School of Medicine, Winston-Salem, NC, USA
d
Critical Care/Trauma Centre, London Health Sciences Centre, Victoria Campus, Canada
e
Lawson Health Research Institute, Canada
f
Clinical Evaluation Research Unit, Kingston General Hospital, Kingston, Ontario, Canada
g
Department of Community Health and Epidemiology, Queen's University, Kingston, Ontario, Canada
h
Department of Medicine, Queen's University, Kingston, Ontario, Canada
b
c

a r t i c l e i n f o

s u m m a r y

Article history:
Received 5 June 2014
Accepted 21 January 2015

Introduction: Better tools are needed to assist in the identication of critically ill patients most likely to
benet from articial nutrition therapy. Recently, the Nutrition Risk in Critically ill (NUTRIC) score has
been developed for such purpose. The objective of this study was to externally validate a modied
version of the NUTRIC score in a second database.
Methods: We conducted a post hoc analysis of a database of a randomized control trial of intensive care
unit (ICU) patients with multi-organ failure. Data for all variables of the NUTRIC score with the exception
of IL-6 levels were collected. These included age, APACHE II score, SOFA score, number of co-morbidities,
days from hospital admission to ICU admission. The NUTRIC score was calculated using the exact same
thresholds and point system as developed previously except the IL-6 item was omitted. A logistic model
including the NUTRIC score, the nutritional adequacy and their interaction was estimated to assess if the
NUTRIC score modied the association between nutritional adequacy and 28-day mortality. We also
examined the association of elevated NUTRIC scores and 6-month month mortality and the interaction
between NUTRIC score and nutritional adequacy.
Results: A total of 1199 patients were analyzed. The mean total calories prescribed was 1817 cal (SD 312)
with total mean protein prescribed of 98.3 g (SD 23.6). The number of patients who received PN was 9.5%.
The overall 28-day mortality rate in this validation sample was 29% and the mean NUTRIC score was 5.5 (SD
1.6). Based on the logistic model, the odds of mortality at 28 days was multiplied by 1.4 (95% CI, 1.3e1.5) for
every point increase on the NUTRIC score. The mean (SD) nutritional adequacy was 50.2 (29.5) with an
interquartile range from 24.8 to 74.1. The test for interaction conrmed that the association between
nutritional adequacy and 28-day mortality is signicantly modied by the NUTRIC score (test for interaction p 0.029). In particular, there is a strong positive association between nutritional adequacy and 28
day survival in patients with a high NUTRIC score but this association diminishes with decreasing NUTRIC
score. Higher NUTRIC scores are also signicantly associated with higher 6-month mortality (p < 0.0001)
and again the positive association between nutritional adequacy and 6 month survival was signicantly
stronger (and perhaps only present) in patients with higher NUTRIC score (test for interaction p 0.038).
Conclusion: The NUTRIC scoring system is externally validated and may be useful in identifying critically
ill patients most likely to benet from optimal amounts of macronutrients when considering mortality as
an outcome.
2015 Elsevier Ltd and European Society for Clinical Nutrition and Metabolism. All rights reserved.

Keywords:
Nutrition
Nutritional risk
Critically ill patients
Intensive care unit

* Corresponding author. Queen's University, Kingston General Hospital, Angada 4 Room 5-416, 76 Stuart Street, Kingston, Ontario K7L 2V7, Canada. Fax: 1 613 548 2428.
E-mail address: dkh2@queensu.ca (D.K. Heyland).
http://dx.doi.org/10.1016/j.clnu.2015.01.015
0261-5614/ 2015 Elsevier Ltd and European Society for Clinical Nutrition and Metabolism. All rights reserved.

Please cite this article in press as: Rahman A, et al., Identifying critically-ill patients who will benet most from nutritional therapy: Further
validation of the modied NUTRIC nutritional risk assessment tool, Clinical Nutrition (2015), http://dx.doi.org/10.1016/j.clnu.2015.01.015

A. Rahman et al. / Clinical Nutrition xxx (2015) 1e5

1. Introduction
Heyland et al. previously proposed a novel scoring tool, the Nutrition Risk in Critically ill (NUTRIC) score, which is the rst nutritional
risk assessment tool developed and validated specically for intensive
care unit (ICU) patients [1]. Many other risk scores and assessment
tools exist to quantify nutrition risk [2e7] but none have been specically designed for ICU patients [7]. Indeed, they generally consider
all critically ill patients to be at high nutritional risk [2,8]. However, the
recognition that not all ICU patients will respond the same to nutritional interventions was the critical concept behind the NUTRIC score
[1,8,9]. The conceptual model incorporated candidate predictor
markers of acute starvation, chronic starvation, acute inammation
and chronic inammation [1,9]. All candidate predictors incorporated
into our nal model predictors were signicantly associated with 28day mortality [1]. Measure of under-nutrition, such as history or
reduced oral intake or recent weight loss, did not factor into the nal
model because of signicant amounts of missing data. The nal
composite score accurately identied those patients who had higher
mortality rates or survivors with longer lengths of stay. In addition,
there was an interaction between mortality, nutritional intake and
NUTRIC score suggesting that those with higher NUTRIC scores (6 or
more) beneted the most from increasing nutritional intake. However,
the inferences about the validity of the NUTRIC score are limited
because they are derived and validated in the same database.
Many methods of nutritional screening in hospitalized patients
are cumbersome and time-consuming and hence are not routinely
done [10]. The NUTRIC score is easy to calculate as it contains
variables that are mostly easy to obtain in the critical care setting,
with the exception of IL-6 levels which is not commonly measured.
In practice, many units are using the NUTRIC score without the IL-6
level and the question remains as to the validity of the validity of
the NUTRIC score without IL-6 level (modied NUTRIC score). The
second stage in development of a clinical ICU prediction model is
external validation [11]. The aim of this study is to externally validate [11] this modied NUTRIC score in a second, population of
critically ill patients. We hypothesize that the modied NUTRIC
score will retain its validity in this new database by omitting the IL6 levels, and we can increase the clinical utility of the tool.

Table 1
NUTRIC scoring system in original and second validation database.
Variables in
NUTRIC score

Age

APACHE II

SOFA

# Co-morbidities
Days from hospital
to ICU admit
IL6*

NUTRIC scoring
system
Range

Points

<50
50e<75
75
<15
15e<20
20e28
28
<6
6e<10
10
0e1
2
0e<1
1
0e<400
400

0
1
2
0
1
2
3
0
1
2
0
1
0
1
0
1

Score range [IQR]

Score mean SD
NUTRIC score discriminative performance
AUC
Gen R-Squared
Gen Max-rescaled
R-Squared

Original
development
sample
(n 598)

Second
validation
sample
(n 1, 199)

130 (21.7)
345 (57.7)
123 (20.6)
111 (18.6)
135 (22.6)
226 (37.8)
126 (21.1)
220 (36.8)
247 (41.3)
131 (21.9)
160 (26.8)
438 (73.2)
375 (62.7)
223 (37.3)
489 (81.8)
109 (18.2)
0e10 [3e6].
4.7 2.2

199 (16.6)
710 (59.2)
290 (24.2)
48 (4.0)
157 (13.1)
508 (42.4)
486 (40.5)
157 (13.1)
624 (52.0)
418 (34.9)
392 (32.7)
807 (67.3)
757 (63.1)
442 (36.9)

0e9 [4e7].
5.5 1.6

0.783
0.169
0.256

0.648
0.055
0.573

This study was a post hoc analysis of an existing database

derived from a randomized control trial conducted in 40 tertiary
ICU's in Europe and North America, after ethics approval was obtained. The purpose of the trial was to evaluate the effectiveness of
glutamine and antioxidant supplementation in critically ill patients
[12]. All patients were attempted to be fed according to the Canadian Critical Care Nutrition practice guidelines, independent of
study supplements [12]. The trial randomized 1223 mechanically
ventilated patients with multi-organ failure, with expected length
of stay >5days with a primary outcome of 28-day mortality.
The NUTRIC score was calculated using the same thresholds and
point system as developed previously (Table 1) except the IL-6 item
was omitted (IL-6 levels were not collected in original study). Thus,
the NUTRIC score ranges from 0 to 9 rather than 0 to 10, as originally dened. The absence of IL-6 may makes assessment of the
performance of the NUTRIC score more conservative, although it is
not expected that the absence of this one item will have a strong
impact on the score [1].

during the rst 28 ICU days while the patient remained ventilated
[12]. Only days prior to the date of death, ICU discharge or liberation
from mechanical intubation were counted (evaluable) toward
nutrition adequacy.
Logistic regression was used to assess the strength of the association between the NUTRIC score and 28-day mortality. Measures
of discrimination using the original data were compared to those
obtained from the original development database. Calibration (i.e.
goodness of t) of the score was assessed by graphing observed
mortality rates at each score against the mortality predicted by a
logistic model with NUTRIC score as a sole continuous predictor.
The statistical signicance of lack of t was tested by the HosmereLemeshow goodness of t test [13].
A logistic model including the NUTRIC score, the nutritional
adequacy and their product (interaction) was performed to assess if
the NUTRIC score modied the association between nutritional
adequacy and 28-day mortality. This model was stratied by
evaluable days since the length of stay could confound the relationship between nutritional adequacy and outcome due to the
ramping up of nutrition support over the rst several ICU days. For
ease of interpretation, gures were generated demonstrating the
association between nutrition adequacy and 28-day mortality
separately in patients with NUTRIC scores grouped as 0e5 and 6e9.
However, the test for interaction used NUTRIC score and nutrition
adequacy as continuous variables. Finally, the logistic model was
run separately in patients who did and did not have enteral feeding
interrupted to assess if increasing NUTRIC score is associated with
feeding intolerance.
Given capture of longer-term mortality rates in this database,
we used a similar modeling approach with Cox proportional hazards model [14] to estimate the overall association between NUTRIC
score and 6-month survival and if the NUTRIC score signicantly
modied the association between nutritional adequacy and 6month survival.

2.1. Statistical analysis

3. Results

Nutrition adequacy was dened as the total proportion of the

caloric prescription received (either enterally or parenterally)

Five patients withdrew consent prior to treatment and were not

evaluable for 28-day mortality, and the amount of calories received

2. Methods

Please cite this article in press as: Rahman A, et al., Identifying critically-ill patients who will benet most from nutritional therapy: Further
validation of the modied NUTRIC nutritional risk assessment tool, Clinical Nutrition (2015), http://dx.doi.org/10.1016/j.clnu.2015.01.015

A. Rahman et al. / Clinical Nutrition xxx (2015) 1e5

was not known for an additional 19 patients. Thus, the current

analysis included 1199 patients.
The overall 28-day mortality rate in this validation sample is
29.0% compared to 23.1% in the NUTRIC development sample. The
distribution of the items included in the NUTRIC risk score are
presented for both this validation sample and the previous development sample in Table 1. Fig. 1 compares the distribution of the
NUTRIC score between the current validation database and the
original developmental database. The NUTRIC score in the current
validation database has a higher mean and lower standard deviation compared to the original developmental sample [mean (SD)
5.5 (1.6) versus 4.7 (SD 2.2)].
The ability of the NUTRIC score to predict 28-day mortality is
summarized in Fig. 2. The overall discriminate ability of the NUTRIC
score for predicting 28-day mortality is lower in this validation
sample than in the original development sample (0.648 vs. 0.783).
In particular, the mortality rate for patients with a maximum
NUTRIC score of 9 was only 53% (8/15) in the current validation
sample compared to 71% (12/17) in the developmental sample. No
patients with a NUTRIC score of 0 or 1 died in either sample (0/45
for original sample and 0/14 for validation sample). Fig. 2, shows
the observed (circles) and logistic model predicted (line) mortality
rate by NUTRIC score. Although mortality generally increased with
NUTRIC score, the calibration of the model was statistically not
ideal (HosmereLemeshow goodness of t test p 0.013). However,
although statistically signicant, the lack of calibration did not
appear to be clinically important (Fig. 2). Using these validation
data, the logistic model estimated odds of mortality were multiplied by 1.4 (95% CI, 1.3e1.5) for every point increase on the NUTRIC
score.
The mean (SD) nutritional adequacy was 50.2 (29.5) with an
interquartile range from 24.8 to 74.1. Nutritional adequacy was not
correlated with NUTRIC score. Fig. 3, demonstrates that increased
nutritional adequacy appears to be associated with increased survival in patients with higher NUTRIC scores. The test for interaction
conrmed the association between nutritional adequacy and
mortality is signicantly modied by the NUTRIC score (test for
interaction p 0.013 and p 0.029 before and after controlling for
evaluable days).
Fig. 4 demonstrates that the association between 28-day mortality, NUTRIC score and nutritional adequacy was similar among
patients who had enteral feeding interrupted due to intolerance
(panel A) and those who did not (panel B). The 277 patients who
had enteral feeding interrupted due to intolerance received 5.5%

Fig. 1. Histogram of NUTRIC score.

Fig. 2. Predicted versus observed 28-day mortality.

less (95% CI 2.0%e9.1%, p 0.0023) of their caloric prescription than

patients who did not have any interruptions.
Higher NUTRIC scores are signicantly associated with higher 6month mortality (p < 0.0001) (Fig. 5). Patients who received <25%
of their caloric prescription had a higher mortality rate
(p < 0.0001). Using Cox proportional hazards model, there is a
signicant interaction between the NUTRIC score and the proportion of prescription received (test for interaction p 0.038). In
particular, for patients with a NUTRIC score of 6e9 each 25% increase in percent of caloric prescription received multiplied the
hazard rate (HR) of death by 0.82 (95% CI, 0.73e0.91), but there was
no signicant association identied between nutritional adequacy
and 6-month mortality in patients with lower NUTRIC scores.
When examining the proportional hazards assumption we noted
that the strong association between low caloric intake and higher
mortality appears to be limited to the rst couple weeks with no
clear difference in hazard rates after that.
4. Discussion
We set out to provide a second validation of the NUTRIC score in a
second database and this time, without IL-6 levels. We report that a
logistic model with NUTRIC score, excluding IL-6, as the sole

Fig. 3. Predicted probability of mortality versus nutrition received by nutrition risk

score.

Please cite this article in press as: Rahman A, et al., Identifying critically-ill patients who will benet most from nutritional therapy: Further
validation of the modied NUTRIC nutritional risk assessment tool, Clinical Nutrition (2015), http://dx.doi.org/10.1016/j.clnu.2015.01.015

A. Rahman et al. / Clinical Nutrition xxx (2015) 1e5

Fig. 5. NUTRIC scores versus 6-month mortality.

Fig. 4. Predicted probability of mortality vs nutrition received by nutrition risk score.

Panel A: Among 277 patients who had at least one interruption of EN due to intolerance. Panel B: Among 922 patients who never discontinued EN due to intolerance.

continuous independent variable predicted mortality with odds of

mortality multiplied by 1.4 (95% CI, 1.3e1.5) for every point increase
on the NUTRIC score. We demonstrate that increased nutritional
adequacy is associated with increased survival in patients with higher
NUTRIC scores (6) but not in patients with lower NUTRIC scores
(5). In practical terms the NUTRIC score without IL-6 levels may be
used to help determine which might benet from enteral supplementation, parenteral nutrition or strategies to enhance EN delivery.
One interpretation of the original NUTRIC validation study was
that sicker patients tolerate less nutrition rather than NUTRIC
identifying patients who benet more from nutrition. The observation that nutritional intake is stable across all ranges of NUTRIC
and that the relationship between NUTRIC, intake and mortality is
consistent in patient with and without feeding intolerance argues
against this notion.
The NUTRIC score may have utility in the design and interpretation of clinical trials of nutrition in the ICU setting. Based on our
observations, studies that include heterogeneous ICU patients, are
more likely to be negative than those that focus on high risk patients [1,15]. This may in part explain discordant results of recent
protein energy provision studies in ICU patients [15e17]. Future
clinical trials need to use the NUTRIC score or similar validated
measure in this population to describe nutritional risk.
We further analyzed effects of the modied NUTRIC score and
interaction with nutritional adequacy with 6-month mortality, not
performed in the original validation. Elevated NUTRIC score were

associated with increased 6-month mortality and more importantly there was signicant interaction with nutritional adequacy
with higher NUTRIC score.
One of the limitations of our study was our inability to obtain IL6 levels. Even though our intent was to examine the model without
IL-6, we cannot demonstrate whether the difference in discrimination was due to exclusion of this variable. The calibration of this
model (slightly lower area under curve) is likely secondary to a
more homogenous population of sicker patients, compared to the
original development data. The lack of calibration across the range
of scores was statistically signicant, and although this visually
appears to be an issue with low scores, we cannot explicitly make
that conclusion. We recognize that using mortality is also a limitation as nutritional therapy may have other benets for patients.
The major limitation of the NUTRIC score is uptake by clinicians
who may argue that there is little need for another risk score with
other risk score such APACHE II or SOFA score. These risks scores are
integrated within our NUTRIC scoring system but we recognize
calculation can be cumbersome limiting utility, and it has no value
for non-ICU patients. Others correctly point out that the NUTRIC
score does not contain traditional nutrition variables. Unfortunately, in an ICU setting, these variables depend on history from
family members, which was inconsistent in our initial validation
sample. With these limitations, however, we observe that
increasing nutritional adequacy in patients with elevated NUTRIC
scores experience lower 28 day mortality. That is, increased
nutritional intake will decrease mortality in high-risk patients,
identied by the NUTRIC score, which elevates the NUTRIC score
beyond a mere statistical measure. Indeed, the NUTRIC score may a
valuable in identifying high-risk patients who can benet from
increased nutritional provision in future studies.
5. Conclusion
We have demonstrated independent validation of the NUTRIC
score without IL-6 levels to help discriminate which ICU patients
will benet more (or less) from early adapted protein-energy provision. This scoring tool represents the rst nutritional risk
assessment tool developed and validated specically for ICU patients. The NUTRIC score is a practical, easy-to-use tool based on
variables that are easy to obtain in the critical care setting. We
assert that not all ICU patients are the same, and there are some
that benet more or less from articial protein-energy provision in
the critical care setting. We recognize that using mortality is also a
limitation as nutritional therapy may other benets for patients

Please cite this article in press as: Rahman A, et al., Identifying critically-ill patients who will benet most from nutritional therapy: Further
validation of the modied NUTRIC nutritional risk assessment tool, Clinical Nutrition (2015), http://dx.doi.org/10.1016/j.clnu.2015.01.015

A. Rahman et al. / Clinical Nutrition xxx (2015) 1e5

and that future clinical trials are required to answer discordant

results regarding energy provision in the ICU.
Conict of interest
None declared.
References
[1] Heyland D, et al. Identifying critically ill patients who benet the most from
nutrition therapy: the development and initial validation of a novel risk
assessment tool. Crit Care 2011 Nov 15;15(6):R268 [Epub].
[2] Detsky A, et al. What is subjective global assessment of nutritional status?
J Parenter Enteral Nutr 1987 JaneFeb;11(1):8e13.
[3] Group MA. A consistent and reliable tool for malnutrition screening. Nurs
Times 2003;99(46):26e7.
[4] Kruizenga H, et al. Development and validation of a hospital screening tool for
malnutrition: the short nutritional assessment questionnaire (SNAQ). Clin
Nutr 2005;24(1):75e82.
[5] Ferguson M, et al. Development of a valid and reliable malnutrition screening
tool for adult acute hospital patients. Nutrition 1999;15(6):458e64.
[6] Lim S, et al. Development and validation of 3-minute nutrition screening (3MinNS) tool for acute hospital patients in Singapore. Asia Pac J Clin Nutr
2009;18(3):395e403.

[7] Anthony P. Nutrition screening tools for hospitalized patients. Nutr Clin Pract
2008;23(4):373e82.
[8] Kondrup J, et al. ESPEN working group: nutritional risk screening (NRS 2002):
a new method based on an analysis of controlled clinical trials. Clin Nutr
2003;22(3):321e36.
[9] Jensen G, et al. International Consensus Guideline Committee: adult starvation
and disease-related malnutrition: a proposal for etiology-based diagnosis in
the clinical practice setting from the International Consensus Guideline
Committee Clinical Nutrition. J Parenter Enteral Nutr 2010;34(2):156e9.
[10] Murray M, et al. Nutritional assessment of intensive-care unit patients. Mayo
Clin Proc 1988;63(11):1106e15.
re J. How to derive and validate clinical prediction models for use in
[11] Labare
intensive care medicine. Intensive Care Med 2014;40:513e27.
[12] Heyland D, et al. A randomized trial of glutamine and antioxidants in critically
ill patients. N Engl J Med 2013;368:1489e97.
[13] Hosmer D, Lemeshow S. Applied logistic regression. New York: Wiley; 2000,
ISBN 0-471-61553-6.
[14] Cox D, Oakes D. Analysis of survival data. New York: Chapman & Hall; 1884,
ISBN 041224490X.
[15] Heyland D, Cahill N, Day A. Optimal amount of calories for critically ill patients: depends on how you slice the cake! Crit Care Med 2011;12:2619e26.
[16] Rahman A, Corbett C. Nutrition in the acute phase of critical illness. N Engl J
Med 2014 Jun 19;370(25):2450.
[17] Rahman A, Martin C, Heyland D. Nutrition therapy for the critically ill surgical
patient with aortic aneurysmal rupture: dening and improving current
practice 2013 Aug 23;2013 [Epub ahead of print].

Please cite this article in press as: Rahman A, et al., Identifying critically-ill patients who will benet most from nutritional therapy: Further
validation of the modied NUTRIC nutritional risk assessment tool, Clinical Nutrition (2015), http://dx.doi.org/10.1016/j.clnu.2015.01.015