Limitations in validating emergency department triage scales

Abstract
Objective
To examine whether current validation methods of emergency department triage scales actually
assess the instrument's validity.
Methods
Optimal methods of emergency department triage scale validation are examined in developed
countries and their application to developing countries is considered.
Results and conclusion
Numerous limitations are embedded in the process of validating triage scales. Methods of triage
scale validation in developed countries may not be appropriate and repeatable in developing
countries. Even in developed countries there are problems in conceptualising validation methods.
A new consensus building validation approach has been constructed and recommended for a
developing country setting. The Delphi method, a consensual validation process, is advanced as a
more appropriate alternative for validating triage scales in developing countries.
Emergency department (ED) triage is the process of sorting and filtering patients based on
medical priority. It aims to determine a patient's acuity level in order to facilitate timely and
effective care before their condition worsens. A patient's acuity level is defined as the urgency
for effective care. In the ED triage setting effective care is defined as the provision of an
intervention or treatment that reduces the patient's urgency for care or prevents clinical
deterioration.
1
If patients receive timely and effective care, triage has achieved its purpose (as
seen at point A in fig 11).

Figure 1 Triage scales are most valuable at (A), where triage facilitates optimal time to highly
effective care, and least valuable at (B), where triage facilitates delayed time and ineffective
care.
This illustration of triage is a highly simplified approach to a complex set of interrelationships. It
is acknowledged that additional variables may influence optimal time to care and effectiveness
of care significantly (such as variability in triage nurse decisions).
Go to:
RELIABILITY
The evaluation of a triage tool involves assessing reliability and validity.
2
Reliability refers to the
degree to which repeated assessments of the same patient with a triage instrument will deliver
the same acuity level. Interrater reliability determines whether there is significant variability
between different triage officers rating the same patient, and intrarater reliability assesses the
variability within a single triage officer rerating the same patient. Reliability makes no reference
to a criterion, and so only illustrates consistency with triage repetition. It shows nothing about its
validity (whether it is a reflection of the truth). A measure can therefore be highly reliable
without being valid.
3

Reliability can be estimated by evaluating different types of agreement. Percentage agreement,
the coefficient and the weighted coefficient are three common ways of measuring agreement
between raters,
4
but these measures can generate quite different values. Measuring only the
percentage agreement is not recommended because it does not take into account agreement
expected on chance alone.
5
The coefficient considers both percentage agreement between
raters and percentage agreement expected by chance; unfortunately, it does not take into account
the magnitude of disagreement, which may become significant in ordinal data. As a result, the
weighted coefficient has become the instrument of choice as it assigns different weights of
agreement according to the magnitude of disagreement, and enables more explicit comparisons
between different studies.
4
While the majority of research in triage has focused on interrater and
intrarater reliability, which has its uses, it is of greater importance to determine whether a triage
tool is in fact valid. We will therefore be focusing on the validity of a triage tool rather than its
reliability.
Go to:
VALIDITY
Validity refers to the degree with which the measured acuity level reflects the patient's true acuity
at the time of triage. The term valid implies that there is some sort of external reference or gold
standard which by definition has absolute accuracy.
3
Studies that aim to see how closely an
instrument approximates the truth, test criterion validity. Unfortunately it is not possible to
measure the truth for patient acuity,
6
as there are myriad events that can occur from the time that
a patient presents to the ED to the time of discharge (including the length of time to initiation of
care, the quality of that care, and nonmedical factors influencing disposalfor example, social
factors). As a result, surrogate outcome markers have been used as criteria to assess validity.
This has led to other ways of assessing validity for ED triage tools. The two most commonly
found in the literature are tests of predictive or consensual validity. These have been approached
in a unifying manner by Streiner and Norman, who reconceptualise a variety of notions of
validity commonly used in the literature as construct validity.
7

There is a hierarchy of validity testing in which criterion is the best (table 11).). Streiner and
Norman have shown that unlike the traditional classification of validity, predictive, consensual
and other types of validity are all seen as variants of construct validity.
7
Typically in developed
countries, criterion validity methods are used.

Table 1 Traditional validity testing versus Streiner and Norman's framework
We will use Streiner and Norman's conceptual framework to answer the following questions:
Do current methods of triage tool validation actually assess the validity and what are the
limitations underlying these methods?
How can these limitations be overcome with special reference to developing countries?
Go to:
CURRENT METHODS OF TRIAGE TOOL
ASSESSMENT AND THEIR LIMITATIONS
A number of different triage systems are used in developed countries. To date, four reliable
ordinal ED triage scales have been researched and published: the Australasian Triage Scale
(ATS),
8
the Canadian Triage Acuity Scale (CTAS),
9
the Emergency Triage Scale (aka
Manchester Triage Scale)
10
and the Emergency Severity Index (ESI).
11
While there has been
some focus on the reliability of triage tools, not much is published on their validity. Predictive
validity (a type of construct validity) is the most frequently used method of assessing tools.
12
It
considers the degree to which the triage acuity level is able to predict true acuity. Particular
outcomes, or events with timeordering, are selected as surrogate markers (such as mortality
rates, hospital admission rates, resource utilisation, and length of stay in hospital). There are
methodological problems with the use of this type of validity as it does not always answer the
core question: Is the triage instrument able to measure what is supposed to be measured? In
patients it does not measure acuity at the time of assessment (and is inherently confounded by the
effectiveness of the health care intervention).
Examples of predictive validity abound in the triage literature, as surrogate outcome markers are
practical to measure and are claimed to be closely associated with true acuity.
3
This has
compelled clinicians and researchers to utilise triage instruments as prediction tools. However,
our ability to identify and measure the relationship between patient acuity level and outcome
depends not only on the measurement of the surrogate outcome marker and the patient's acuity
level, but also very importantly on confounding variables such as variability in triage nurse
decisions, and delayed and ineffective treatment. These may affect the surrogate outcome
marker.
Go to:
HOW CAN THESE LIMITATIONS BE OVERCOME?
A detailed literature review revealed that very little has been published on triage in developing
countries. The World Health Organization reports that triage research is not a priority in low to
middleincome countries.
13
They have accordingly developed the Emergency Triage Assessment
and Treatment (ETAT)
14
for application to developing countries. While this subjective system
has been successfully implemented in Malawi, countries like India, Brazil and South Africa have
sought a more objective triage instrument based on physiology. They have either adopted the
triage instrument from a developed country or modified it to their own local context and needs
(Patriacia Neto, Quinta D'or Hospital, Rio de Janeiro, May 2007, personal communication).
South Africa has adapted the Modified Early Warning Score (MEWS) as the South African
Triage Scale after validating it on the local national population.
15
Some areas of Brazil have
adopted the CTAS, others the ESI.
During any validity testing an important distinction needs to be made between internal validity
(which refers to inferences about the source population), and external validity (whether
inferences may be generalised to people outside the source population).
16
A triage tool designed
for a developed country may be valid in that context, leading to favourable results that are
meaningful and have implications for action. If, however, the same triage tools were applied in a
developing country, results may vary due to different resources and skills. Similarly results may
vary when applying surrogate markers from developed countries to undertake validity testing in
developing countries. This variability may increase the random error in both triage acuity level
and outcome category; it would therefore be more appropriate to apply a locally developed tool
that is meaningful in the local context (has internal validity), but that may not be applicable in a
developed country (lack of external validity).
Whichever tool is used, an assessment of its usefulness in these settings is required. When
selecting surrogate outcome markers (such as mortality rates, hospital admission rates, resource
utilisation, and length of stay in hospital), it is assumed that there is systematic record keeping,
and that the care given is effective. While this may often be the case in developed countries, it is
typically not the case in developing countries. Poor record keeping and ineffective care may have
significant effects on surrogate outcome markers and patients' final dispositions. Markers such as
these are imperfect measures of patient acuity in the developing world. It is thus important to
identify and measure all confounding variables that may be affecting the surrogate outcome
marker: given the poor record keeping and lack of efficiency, this is unlikely to be feasible in
developing countries.
Delphi methodology
The Delphi method was developed in the 1950s by the RAND Corporation in California, USA.
17

The technique has diversified and is being applied to more mainstream social sciences, in
business and, in the last two decades, within the healthcare arena.
18
It is a consensus building
technique designed to gain insight into a particular field to enable decision making in areas
where published information is inadequate or nonexistent.
19
The approach of the Delphi
technique is to establish a panel of appropriate experts that have agreed to complete an iterative
process on a particular issue, with the key objective being to reach consensus.
20
Panellist
anonymity is maintained throughout the process and controlled feedback is provided from each
iterative round, resulting in a statistical aggregation of the group response.
18

The Delphi method is another form of construct validity that may be useful when assessing triage
scales in developing countries. It allows the development of a surrogate gold standard
determined by specialist panel consensus. The triage tool's validity may then be tested against
this construct of true underlying acuity that is consensually arrived at. There appear to be only
very few examples in the world literature that elaborate on the use of this form of construct
validity.
Wallis et al
21
used consensus from Delphi methodology to establish triage acuity levels against
which to test prehospital mass casualty triage tools: such methodology may be used in ED triage
tool assessment.
There are several reasons why the Delphi methodology is best suited to assessing ED triage tools
in developing countries. The Delphi technique eliminates potential bias due to individual group
dynamics and is financially feasible.
Limitations of the Delphi technique are mostly a result of poorly conducted studies rather than
fundamental problems. One of the weaknesses cited is that the response rates can be low and
often decrease as the rounds progress. However, nonresponse is typically very low in practice,
since most researchers have personally obtained assurance of participation. Similarly attrition
tends to be low and the researcher can easily ascertain the cause by talking with the dropouts.
22

Selection of the Delphi panel depends on the research question. Problems may arise with a lack
of representativeness in that only experts with an interest and involvement will become
participants. Another potential weakness of the Delphi as a consensus method is that it overlooks
important minority issues because it tries to obtain consensus.
23
However, despite these
limitations we believe that the Delphi process is the most appropriate form with which to test the
validity of triage tools in the developing world.
Go to:
CONCLUSION
In developing countries a form of construct validity derived from a consensual process appears to
be the most appropriate form of validation of triage tools. This is due to lack of criteria for true
acuity, confounding variables that relate to differential health care resources by level of
development, and lack of external validity of other triage scales.
24
We propose the Delphi
method when testing the South African Triage Scale. This is an example of construct validity
testing in the developing world.
Go to:
Abbreviations
ATS - Australasian Triage Scale
CTAS - Canadian Triage Acuity Scale
ED - emergency department
ESI - Emergency Severity Index
ETAT - Emergency Triage Assessment and Treatment
MEWS - Modified Early Warning Score
Go to:
Footnotes
Funding: None
Competing interests: None
Contributions: LW had the original idea; MT wrote the first draft; both authors contributed to the
final article.
Go to:
References
1. Bindman A B. Triage in accident and emergency departments. BMJ 1995. 311404. [PMC free
article] [PubMed]
2. Rutschmann O T, Kossovsky M, Geissbhler A. et al Interactive triage simulator revealed
important variability in both process and outcome of emergency triage. J Clin Epid 2006. 59615
621.621. [PubMed]
3. Lewis R J.Reliability and validity: meaning and measurement. Presented at 1999 Annual
Meeting of the Society for Academic Emergency Medicine (SAEM) in Boston, Massachusetts.
http://www.ambpeds.org/ReliabilityandValidity.pdf.
4. Jakobsson U, Westergren A. Statistical methods for assessing agreement for ordinal data.
Scand J Caring Sci 2005. 19427431.431. [PubMed]
5. Fernandes C B, Groth S J, Johnson L A. et alA uniform triage scale in emergency medicine.
American College of Emergency Physicians, 1999.
http://www.acep.org/NR/rdonlyres/EE2B10F751BE42A894B7779726144017/0/triagescaleip
.pdf.
6. Fernandes C M, Tanabe P, Gilboy N. et al Fivelevel triage: a report from the ACEP/ENA
fivelevel triage task force. J Emerg Nurs 2005. 313950.50. [PubMed]
7. Streiner D L, Norman G R. Precision and accuracy: Two terms that are neither. J Clin
Epidemiol 2006. 59327330.330. [PubMed]
8. Australasian College of Emergency Medicine. Guidelines for the Implementation of the
Australasian Triage Scale in Emergency Departments. Australasian College of Emergency
Medicine 1998.
9. Canadian Association of Emergency Physicians Implementation guidelines for the Canadian
Emergency Department Triage and Acuity Scale (CTAS). Canadian Association of Emergency
Physicians 1998.
10. Manchester Triage Group Emergency triage. London: BMJ Publishing Group, 1997.
11. Gilboy N, Tanabe P, Travers D A. et al The Emergency Severity Index implementation
handbook: a fivelevel triage system. Des Plaines, Illinois: Emergency Nurses Association, 2003.
12. Fan J, Al Darrab A, Eva K. et al Triage scales in the emergency department: a systematic
review. Ann Emerg Med 2005. 46S41.
13. Razzak, Junaid A, Kellermann, Arthur L. Emergency medical care in developing countries: is
it worthwhile? Bull World Health Organ 2002. 80900905.905. [PMC free article] [PubMed]
14. Robertson M A, Molyneux E M. Triage in the developing world can it be done? Arch Dis
Child 2001. 85208213.213. [PMC free article] [PubMed]
15. Gottschalk S B, Wood D, de Vries S. et al The cape triage score: a new triage system South
Africa. Proposal from the cape triage group. Emerg Med J 2006. 23149153.153. [PMC free
article] [PubMed]
16. Rothman K J, Greenland S. Modern epidemiology. Philadelphia: LippincottRaven
Publishers, 1998. 115147.147.
17. Dalkey N, Helmer O. An experimental application of the Delphi method to the use of
experts. Management Science 1963. 9458.
18. Jones J, Hunter D. Consensus methods for medical and health services research. BMJ 1995.
311376380.380. [PMC free article] [PubMed]
19. Pill J. The Delphi method: Substance, context, a critique and an annotated bibliography.
SocioEcon Plan Sci 1971. 55771.71.
20. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J
Adv Nurs 2000. 3210081015.1015. [PubMed]
21. Wallis L A, Carley S, Hodgetts C T. A procedure based alternative to the injury severity
score for major incident triage of children: results of a Delphi consensus process. Emerg Med J
2006. 23291295.295. [PMC free article] [PubMed]
22. Okoli C, Pawlowski S. The Delphi method as a research tool: an example, design
considerations and applications. Information and Management 2004. 421529.29.
23. Van Teijlingen E, Pitchforth E, Bishop C. et al Delphi method and nominal group techniques
in family planning and reproductive health research. J Fam Plann Reprod Health Care 2006.
32249252. [PubMed]
24. Fernandes C B, Groth S J, Johnson L A. et alA uniform triage scale in emergency medicine.
American College of Emergency Physicians, 1999;
http://www.acep.org/NR/rdonlyres/EE2B10F751BE42A894B7779726144017/0/triagescaleip
.pdf.





Emergency department triage revisited.
FitzGerald G, Jelinek GA, Scott D, Gerdtz MF.
Source
School of Public Health, Queensland University of Technology, Brisbane, Queensland,
Australia. gj.fitzgerald@qut.edu.au
Abstract
Triage is a process that is critical to the effective management of modern emergency
departments. Triage systems aim, not only to ensure clinical justice for the patient, but also to
provide an effective tool for departmental organisation, monitoring and evaluation. Over the last
20 years, triage systems have been standardised in a number of countries and efforts made to
ensure consistency of application. However, the ongoing crowding of emergency departments
resulting from access block and increased demand has led to calls for a review of systems of
triage. In addition, international variance in triage systems limits the capacity for benchmarking.
The aim of this paper is to provide a critical review of the literature pertaining to emergency
department triage in order to inform the direction for future research. While education,
guidelines and algorithms have been shown to reduce triage variation, there remains significant
inconsistency in triage assessment arising from the diversity of factors determining the urgency
of any individual patient. It is timely to accept this diversity, what is agreed, and what may be
agreeable. It is time to develop and test an International Triage Scale (ITS) which is supported by
an international collaborative approach towards a triage research agenda. This agenda would
seek to further develop application and moderating tools and to utilise the scales for international
benchmarking and research programmes.









Modern Triage in the Emergency Department
Background
Because the volume of patient admissions to an emergency department (ED) cannot be precisely
planned, the available resources may become overwhelmed at times (crowding), with resulting
risks for patient safety. The aim of this study is to identify modern triage instruments and assess
their validity and reliability.
Methods
Review of selected literature retrieved by a search on the terms emergency department and
triage.
Results
Emergency departments around the world use different triage systems to assess the severity of
incoming patients conditions and assign treatment priorities. Our study identified four such
instruments: the Australasian Triage Scale (ATS), the Canadian Triage and Acuity Scale
(CTAS), the Manchester Triage System (MTS), and the Emergency Severity Index (ESI). Triage
instruments with 5 levels are superior to those with 3 levels in both validity and reliability
(p<0.01). Good to very good reliability has been shown for the best-studied instruments, CTAS
and ESI (-statistics: 0.7 to 0.95), while ATS and MTS have been found to be only moderately
reliable (-statistics: 0.3 to 0.6). MTS and ESI are both available in German; of these two, only
the ESI has been validated in German-speaking countries.
Conclusion
Five-level triage systems are valid and reliable methods for assessment of the severity of
incoming patients conditions by nursing staff in the emergency department. They should be
used in German emergency departments to assign treatment priorities in a structured and
dependable fashion.
The emergency department is the crucial interface between the emergency medical services and
the hospital. As reflected in the year-on-year increases in patient numbers, however, emergency
departments are increasingly being selected as the route of primary access to the healthcare
system (Figure 1) (1). Deficits in preclinical patient guidance have been put forward as a possible
explanation for this trend (2).

Figure 1
Emergency departments of Nuremberg Hospital (Nuremberg Hospital South: dark bars;
Nuremberg Hospital North: light bars):
The volume of admissions to a given emergency department cannot be predicted with any great
accuracy, only a certain proportion of the patients have life-endangering or medically urgent
conditions (Figure 1) (3), and not all those admitted can be treated immediately or
simultaneously. Thus, patients with life-threatening injuries or illnesses need to be reliably
identified within minutes of arrival (4). Structured triage systems for emergency department
admissions are already in use in the German-speaking countries (3, 4) and the relevant medical
societies are calling for their introduction in nations with established hospital emergency services
(4, 5).
In the emergency department triage refers to the methods used to assess patients severity of
injury or illness within a short time after their arrival, assign priorities, and transfer each patient
to the appropriate place for treatment (5). In our view the term triage should be adopted in
German-speaking countries in preference to the various German words that have been used, e.g.,
Sichtung and Ersteinschtzung, as the latter are not clearly defined concepts. In some
European countries, among them Germany and Switzerland, triage is performed by specially
trained nursing staff (3 8). The aim of this study is to provide a systematic overview of
established instruments for triage in the emergency department and evaluate their validity and
reliability.
Go to:
Methods
Studies were retrieved from the Medline database by a search on the following terms (search
date : 5 January 2009): triage AND emergency department (n = 1587); five-level triage (n = 25);
Canadian Triage and Acuity Scale (n = 40); National Triage Scale (n = 17); Australasian Triage
Scale (n = 30); Manchester Triage System (n = 15); Emergency Severity Index (n = 26).
Relevant information was taken from review articles and original studies. Only articles from
peer-reviewed journals were considered (Table, eTable), and only studies with published validity
or reliability were then analyzed.

Table
Characteristics of the most important five-level triage instruments used in emergency
departments internationally

eTable
Published validity and reliability of the five-level triage scales listed in Table 1
Validity of a triage instrument
A method is described as valid if its results agree with the true value. With regard to triage
systems, the assigned priority level should correspond with the actual degree of urgency. In the
absence of a gold standard for the genuine degree of urgency, surrogate markers such as rate of
hospital admission, rate of admission to intensive care, mortality rate, and utilization of resources
are used to assess validity (5, 9).
Reliability of a triage instrument
The reliability or replicability of the results should be as high as possible; otherwise the method
is not sufficiently dependable (5). Ideally, different investigators should come to the same
conclusions regarding treatment priority. Reliability is described using the kappa statistic, where
= 0 indicates a random result and = 1 shows total agreement between two or more
measurements. Agreement is classified as follows: poor (<0.2), adequate (0.2<<0.4),
satisfactory (0.4<<0.6), good (0.6<<0.8), and very good (0.8<<1) (5). Some studies have
used weighted kappa statistics. Because usually only one of these two parameters is reported, the
results cannot be directly compared.
Go to:
Results
Overview of triage instruments
Registration of vital signs alone is not suitable for identification of critically ill patients in the
emergency department (10). Therefore various systems are used internationally to determine
initial treatment priorities. These range from unstructured classification according to ones own
experience (best guess [11]) over instruments such as a three-level traffic light system (red:
emergency; amber: urgent; green: non-urgent [12]) to four- and five-level scales (13 16). Some
of these instruments are just used at one particular institution, so their dependability is
insufficiently documented.
Five-level instruments are significantly correlated with resource utilization, rates of admission
for inpatient treatment, duration of emergency treatment, and frequency of transfer to intensive
care or mortality (14). Comparison of methods revealed that three-level triage systems have
insufficient reliability. The interobserver reliability between triage personnel and experts is low
( = 0.19 to 0.38 [5]), while that of the five-level system is significantly higher ( = 0.68; p<0.01
[17]). We will therefore review the literature on established five-level triage systems.
Go to:
The principal five-level triage instruments
Australasian Triage Scale
The Australasian Triage Scale (ATS) has been employed in all Australian emergency
departments since 1994 (7). Each level of priority has a defined time limit within which
evaluation by a doctor should begin (Table). The process data from individual hospitals and for
different regions are published on the Internet. Findings on the validity or reliability of this
instrument are available, although no prospective assessment of reliability has yet been carried
out (Box).
Go to:
Box
Validity and reliability of five-level triage instruments
Manchester Triage Scale (MTS)
Four analyses in adult patients (n = 50 to 167):
Analysis conducted by nursing staff
Validity of instrument only descriptively assessed in two studies: 67% of patients
with high priority (MTS levels 1 and 2; endpoint: transfer to intensive care unit)
were correctly identified. Of patients with cardiac chest pain, 86.8% were
correctly identified by nursing staff
The MTS shows moderate (to good) reliability ( = 0.31 to 0.62)
Two analyses in children (<16 years, n = 1065 to 13 554):
No statistics on reliability
In 40% to 54% of the children there was over-triage; in 12% to 15%, under-triage
Authors suggest modification of the instrument for children; validity in children
rated as satisfactory
Australasian Triage Scale (ATS)
Six analyses in adult emergency patients (n = 20 to 3650):
One analysis to evaluate validity of instrument showed correlation with inpatient
admission rate and agreement with mortality data published in Australia
Five studies in adult emergency patients yielded adequate to satisfactory
reliability ( = 0.25 to 0.56)
One study assessed dependability in evaluation of psychiatric patients (video
recording); the rate of agreement in triage assessment was only about 60%. The
authors conclude that the ATS is inadequate for correct evaluation of psychiatric
patients
Canadian Triage and Acuity Scale (CTAS)
Eight analyses in adult emergency patients (n = 50 to 32 261):
Significant correlation with hospital mortality and resource utilization (p<0.01)
Interobserver reliability reported as good to excellent ( = 0.68 to 0.89)
The instrument has become established in European countries
Four analyses in children:
Study size 54 to 1618 children
Good validity of the instrument, significant correlation between triage level and
resource utilization
Good reliability of the instrument in initial evaluation of young emergency
patients ( = 0.51 to 0.72)
Emergency Severity Index (ESI)
Twelve analyses in adult emergency patients (n = 202 to 3172):
The ESI triage system correlates significantly (p < 0.01) with hospital mortality
and resource utilization
Interobserver reliability reported as good to excellent ( = 0.46 to 0.91)
The instrument has become established in European countries
One analysis in children (<16 years, n = 150):
Good validity and very good interobserver reliability of the instrument ( = 0.82)
Further details of the studies can be found in the eTable
Canadian Triage and Acuity Scale
The Canadian Triage and Acuity Scale (CTAS) is based on the ATS and was developed in the
1990s by emergency physicians in New Brunswick, Canada (15). Since 1997 the parameters of
the CTAS have been compulsorily documented by the Canadian Institute of Health Information.
As in the ATS, the times from arrival to evaluation by a doctor are recorded.
In the CTAS an extensive list of presenting clinical complaints and symptoms is used to
determine the triage level. These include anamnestic parameters associated with high risk, e.g.,
intoxication, together with clinical signs, vital parameters, and symptoms such as shortness of
breath or abdominal pain. Triage must be repeated after a defined waiting time or when there is a
change in the patients symptoms. The validity and reliability of the CTAS are outstanding
(Box), and a modified instrument has been developed for evaluation of pediatric emergencies
(18). The descriptors and modifiers of the CTAS are encapsulated in a software application
(www.caep.ca). In rural areas of Canada the triage is sometimes carried out exclusively by
specially trained nurses, who then decide whether patients need to be transferred elsewhere for
further medical care.
Manchester Triage System
The Manchester Triage System (MTS) is used in emergency departments in Great Britain and, in
a modified translation, in German emergency departments (3, 13). The MTS follows a specific
approach: the patients principal presenting complaints are allocated to one of 52 flowchart
diagrams, e.g., head injury or abdominal pain. Key discriminators are defined for each of these
diagrams, such as danger to life, pain, or state of consciousness. When a new patient presents to
the emergency department the triage nurse assigns their reported complaints to a defined
algorithm and then determines the treatment priority with the aid of fixed rules that take account
of vital signs. The few studies that have been performed point to satisfactory reliability of this
tool (Box) (19).
Emergency Severity Index
The Emergency Severity Index (ESI) is a five-level triage algorithm that was developed in the
USA in the late 1990s (20). Treatment priority is decided on the basis of disease severity and the
expected resource needs (Figure 2). The triage algorithm consists of four decision points where
the trained triage nurse asks specific questions. First, patients with life-threatening conditions
(ESI levels 1 and 2) are identified. Unstable patients are typically assigned to ESI triage level 1,
e.g., in the presence of hemodynamic or respiratory instability. Patients with (potentially) life-
threatening symptoms, e.g., thoracic pain in acute coronary syndrome or loss of consciousness,
and also those with severe pain, psychiatric disorders, or states of intoxication, are assigned to
triage level 2. The remaining levels (3 to 5) are defined by the expected resource needs and vital
signs (Figure 2). Resources in this sense are services such as X-ray and administration of
intravenous medication that go beyond physical examination and are necessary to reach a
decision on how to proceed (6). Clinical studies show that this instrument also has good validity
and reliability in specific groups of patients such as children and the elderly (Box). A validated
translation of this tool into German has been published by a team from the emergency
department of the University Hospital Basel, Switzerland (8).

Figure 2
Triage algorithm of the Emergency Severity Index, version 4. An accompanying brief
description of this algorithm (8) supports the members of staff concerned during the triage
process. The triage system is explained in detail in the original English handbook ...
Other five-level triage instruments
Regional solutions such as the Gruppo Formazione Triage system in Italy (21), the Taiwan
Triage Scale (22), the Cape Triage Scale (23), and the Geneva Emergency Triage Scale (16)
have been used, and yet other tools have been employed in individual hospitals. Not all of these
systems have been described in journals with external peer review. Since some of these methods
draw on the above-mentioned triage instruments and some have not been comprehensively
evaluated, they will not be discussed further here.
Go to:
Discussion
Five-level triage instruments are the gold standard in emergency medicine worldwide. The best
studied and most widely distributed five-level systems are the ATS, CTAS, MTS, and ESI, all of
which possess satisfactory to very good validity and reliability. The MTS and ESI have been
translated into German. While the German-language version of the MTS has not yet been
validated, data on the validity and reliability of the ESI in German have been analyzed and the
results published (Grossmann FF et al.: Transporting clinical tools to new settings: cultural
adaptation and validation of the Emergency Severity Index in German. Ann Emerg Med 2010; in
press).
The literature shows that triage of emergency patients by trained nurses using a five-level system
has been successfully implemented in English-speaking countries (5). Our own observations
demonstrate that such systems can also be used safely by nursing staff in Germany and
Switzerland and improve on the quality provided by the subjective assessment widely employed
to date. This necessitates rethinking the organization of processes; more emphasis must be placed
on interdisciplinary differential diagnosis, risk stratification, and the treatment of acute diseases,
and this must be reflected in training. An example is provided by the key symptom of acute loss
of consciousness, which can probably be managed more effectively in interdisciplinary fashion,
as practiced in English-speaking countries. The introduction of a modern triage system in
Australia increased patient safety, improved both the organization of the work of the emergency
department staff and their job satisfaction, and reduced the patients waiting time as well as the
total time they spent in the emergency department ([24] and unpublished personal observations).
Moreover, the proportion of patients who leave the emergency department without seeing a
doctor because of the long waiting time is lowered significantly, by 50%, thus increasing patient
safety (24). It remains to be established to what extent these results can be replicated in
Germany.
The role of the five-level triage instruments in the assessment of patients with psychiatric
diseases and children has not been analyzed systematically, and in our opinion these tools should
not be introduced in the near future, given the specialized management of sychiatric and
pediatric emergencies prevalent in the German-speaking countries. In the ESI, patients with
acute psychiatric illnesses are assigned to ESI triage level 2, the same level as patients suffering
severe pain, on grounds of their high level of distress. In practice, this means that psychiatric
patients presenting to interdisciplinary emergency departments must always be treated
immediately, whatever the overall workload in the department.
Some triage instruments (CTAS, ATS, MTS) set time limits by when a certain proportion of
patients, depending on treatment priority, must have been evaluated by a doctor (Table). The ESI
takes a different approach for patients with low priority (ESI level 3 to 5): rather than fixed time
limits, the goal is evaluation of these patients as soon as possible depending on current workload.
Patients assigned to ESI level 1 must be treated immediately. Patients classified as level 2
receive nursing care straightaway, including continuous monitoring, and evaluation by a doctor
must follow within 10 min at the most (6). The time to first contact with a doctor in the
emergency department is one of the performance indicators in all triage systems and in some of
them is used for benchmarking. In the CTAS the patients priority level is re-evaluated after a
defined time in order to register any deterioration in status as early as possible (Table). The ESI
and ATS suggest that re-triagewhich we regard as necessarybe carried out only as and when
required. To increase patient safety, in our own emergency departments we enforce re-triage at
defined intervals even when using the ESI. The intervals are oriented on the stipulations of the
CTAS.
Evaluation of the triage instruments
The five-level triage scales are superior to three-level systems with regard to validity and
reliability (12). Some European societies therefore demand the use of a five-level triage system
in emergency departments in which treatment capacity is sometimes exceeded (4). Taking the
above-mentioned limitations of methodology into account, the highest numbers of publications
refer to the five-level instruments CTAS and ESI, both of which have been the subject of
multicenter studies and analyses carried out in Europe. The ATS is also well documented, but
published data on the MTS are sparse.
Five-level triage instruments in German
Among the triage instruments described herein, the MTS has been documented in detail in
German. In our view, however, problems arise from the fact that the German translation does not
make it quite clear why the original algorithm has been modified. For example, there are only 50
instead of 52 flowchart diagrams, the defined reaction times have been changed, and various
other modifications, e.g., in evaluation of pain, have been introduced. Moreover, concepts such
as hot adult and young pain, though defined in the book (13), are unaccustomed in German
and therefore may constitute sources of error. The MTS is widely used in Germany, but there are
no published data on the validity and reliability of the German version.
In the meantime a study has been conducted on the validity and reliability of the German
translation of the ESI (8). Initial analyses confirm high validity and reliability of this instrument
in German (Grossmann FF et al.: Transporting clinical tools to new settings: cultural adaptation
and validation of the Emergency Severity Index in German. Ann Emerg Med 2010; in press).
Because the training of nurses varies internationally, one might assume that triage instruments
developed in English-speaking countries require modification and adaptation to circumstances in
Germany. However, our experience with the implementation of the ESI in the emergency
departments of Nuremberg Hospital, Germany and the University Hospital Basel, Switzerland
shows that this triage instrument can be adopted without modification and safely applied by
nursing staff trained in Germany. The systematic data acquired in the course of triage yield
information on the severity of illness of patients presenting to German emergency departments
(Figure 1); this is important not only for epidemiological purposes but also for health policy and
economic analyses and prognoses.
Implementation and quality management
The implementation of a structured triage system in an emergency department is associated with
a transitional phase and requires careful planning involving all parties concerned, including the
nursing and medical staff. Besides the development of a training program, the consequences for
patient flow, the hospital information system, and the workflow of the interprofessional team
must be considered. Realistically, a modern triage instrument should be able to be implemented
within 9 to 12 months (6). Our own experience shows that any instrument that is implemented
has to be regularly evaluated and quality improvement measures developed jointly by the
members of the team. Case discussions are suggested (6). Another possibility is practice on an
interactive triage simulator, as done at the University Hospital Geneva, Switzerland (16).
Go to:
Synopsis
The introduction of structured triage by specially trained nursing staff in the emergency
department helps to accurately identify patients whose lives are endangered, especially at times
of insufficient treatment capacity. Five-level triage systems are therefore recommended by
national and international societies for emergency medicine (4, 5). If it has been decided to
implement a triage system, an instrument should be selected for which validity and reliability has
been demonstratedideally in the language of the country concerned. Apart from correct
identification of patients who require urgent medical care, such instruments enable estimation
and planning of resources (6).

Go to:
Key Messages
Because the volume of patient admissions cannot be precisely planned, an emergency
department may become overcrowded, so that severely ill and less ill patients are
competing for the available resources.
The goal of modern triage in the emergency department is to ascertain the severity of
illness of emergency patients in a structured way, establish treatment priorities, and
assign the patients to the appropriate place for treatment.
Five-level triage instruments are the gold standard in emergency medicine. The
Australasian Triage Scale, the Canadian Triage and Acuity Scale, the Manchester Triage
System, and the Emergency Severity Index are all validated instruments and are the most
widely used systems.
The Manchester Triage System and the Emergency Severity Index have been translated
into German and are already being used by trained nurses in some emergency
departments in the German-speaking countries.
Validated five-level triage systems should be introduced to emergency departments in the
German-speaking countries in order to ensure high patient safety, especially when there
is heavy pressure on resources.
Go to:
Acknowledgments
Translated from the original German by David Roseveare.
Go to:
Footnotes
Conflict of interest statement
The authors declare that no conflict of interest exists according to the guidelines of the
International Committee of Medical Journal Editors.
Go to:
References
1. Pitts SR, Niska RW, Xu J, Burt CW. National Hospital Ambulatory Medical Care Survey:
2006 emergency department summary. Natl Health Stat Report. 2008:138. [PubMed]
2. Steffen W, Tempka A, Klute G. Falsche Patientenanreize in der Ersten Hilfe der
Krankenhuser. Dtsch Arztebl. 2007;104:10881091.
3. Schellein O, Ludwig-Pistor F, Bremerich DH. Manchester triage system: Process optimization
in the interdisciplinary emergency department. Anaesthesist. 2008;58:163170. [PubMed]
4. Rutschmann OT, Siber RS, Hugli OW. Empfehlung der Schweizerischen Gesellschaft fr
Notfall- und Rettungsmedizin (SGNOR) zur Triage in Schweizer Notfallstationen. Schweiz
rztezeitung. 2009;90:12.
5. Fernandes CM, Tanabe P, Gilboy N, et al. Five-level triage: a report from the ACEP/ENA
Five-level Triage Task Force. J Emerg Nurs. 2005;31:3950. [PubMed]
6. Gilboy N, Tanabe P, Travers D, Rosenau AM, Eitel DR. MD: Agency for Healthcare
Research and Quality. 05-0046-2 ed. Rockville: AHRQ Publication No; 2005. Emergency
Severity Index, Version 4. Implementation Handbook.
7. Australasian College for Emergency Medicine. Guidelines on the implementation of the
Australasian Triage Scale in emergency departments.
www.acem.org.au/infocentre.aspx?docId=59#POLICIES. Accessed on 5 January 2009.
8. Grossmann FF, Delport K, Keller DI. Emergency Severity Index: Deutsche bersetzung eines
validen Triageinstruments. Notfall Rettungsmed. 2009;12:290292.
9. Twomey M, Wallis LA, Myers JE. Limitations in validating emergency department triage
scales. Emerg Med J. 2007;24:477479. [PMC free article] [PubMed]
10. Subbe CP, Slater A, Menon D, Gemmell L. Validation of physiological scoring systems in
the accident and emergency department. Emerg Med J. 2006;23:841845. [PMC free article]
[PubMed]
11. Wuerz R, Fernandes CM, Alarcon J. Inconsistency of emergency department triage.
Emergency Department Operations Research Working Group. Ann Emerg Med. 1998;32:431
435. [PubMed]
12. Wuerz RC, Milne LW, Eitel DR, Travers D, Gilboy N. Reliability and validity of a new five-
level triage instrument. Acad Emerg Med. 2000;7:236242. [PubMed]
13. Mackway-Jones K, Marsden J, Windle J. Das Manchester Triage System. 2nd edition. Bern:
Huber; 2006. Ersteinschtzung in der Notaufnahme.
14. Tanabe P, Gimbel R, Yarnold PR, Kyriacou DN, Adams JG. Reliability and validity of
scores on The Emergency Severity Index version 3. Acad Emerg Med. 2004;11:5965.
[PubMed]
15. Bullard MJ, Unger B, Spence J, Grafstein E. Revisions to the Canadian Emergency
Department Triage and Acuity Scale (CTAS) adult guidelines. CJEM. 2008;10:136151.
[PubMed]
16. Rutschmann OT, Kossovsky M, Geissbuhler A, et al. Interactive triage simulator revealed
important variability in both process and outcome of emergency triage. J Clin Epidemiol.
2006;59:615621. [PubMed]
17. Travers DA, Waller AE, Bowling JM, Flowers D, Tintinalli J. Five-level triage system more
effective than three-level in tertiary emergency department. J Emerg Nurs. 2002;28:395400.
[PubMed]
18. Warren DW, Jarvis A, LeBlanc L, Gravel J. Revisions to the Canadian Triage and Acuity
Scale paediatric guidelines (PaedCTAS) CJEM. 2008;10:224243. [PubMed]
19. van Veen M, Steyerberg EW, Ruige M, et al. Manchester triage system in paediatric
emergency care: prospective observational study. BMJ. 2008;337 [PMC free article] [PubMed]
20. Shelton R. The emergency severity index 5-level triage system. Dimens Crit Care Nurs.
2009;28:912. [PubMed]
21. Gruppo Formazione Triage. Triage infermieristico. 2nd edition. McGraw-Hill Companies;
2005.
22. Chi CH, Huang CM. Comparison of the Emergency Severity Index (ESI) and the Taiwan
Triage System in predicting resource utilization. J Formos Med Assoc. 2006;105:617625.
[PubMed]
23. Bruijns SR, Wallis LA, Burch VC. A prospective evaluation of the Cape triage score in the
emergency department of an urban public hospital in South Africa. Emerg Med J. 2008;25:398
402. [PubMed]
24. Richardson JR, Braitberg G, Yeoh MJ. Multidisciplinary assessment at triage: a new way
forward. Emerg Med Australas. 2004;16:4146. [PubMed]
25. Cooke MW, Jinks S. Does the Manchester triage system detect the critically ill? J Accid
Emerg Med. 1999;16:179181. [PMC free article] [PubMed]
e1. Cooke MW, Jinks S. Does the Manchester triage system detect the critically ill? J Accid
Emerg Med. 1999;16:179181. [PMC free article] [PubMed]
e2. Goodacre SW, Gillett M, Harris RD, Houlihan KP. Consistency of retrospective triage
decisions as a standardised instrument for audit. J Accid Emerg Med. 1999;16:322324. [PMC
free article] [PubMed]
e3. Speake D, Teece S, Mackway-Jones K. Detecting high-risk patients with chest pain. Emerg
Nurse. 2003;11:1921. [PubMed]
e4. Roukema J, Steyerberg EW, van Meurs A, Ruige M, van der Lei J, Moll HA. Validity of the
Manchester triage system in paediatric emergency care. Emerg Med J. 2006;23:906910. [PMC
free article] [PubMed]
e5. van Veen M, Steyerberg EW, Ruige M, et al. Manchester triage system in paediatric
emergency care: prospective observational study. BMJ. 2008;337 [PMC free article] [PubMed]
e6. van der Wulp I, van Baar ME, Schrijvers AJ. Reliability and validity of the Manchester triage
system in a general emergency department patient population in the Netherlands: results of a
simulation study. Emerg Med J. 2008;25:431434. [PubMed]
e7. Jelinek G, Little M. Inter-rater reliability of the national triage scale: over 11500 simulated
occasions of triage. Emerg Med. 1996;8:226230.
e8. Dilley S, Standen P. Victorian nurses demonstrate concordance in the application of the
national triage scale. Emerg Med. 1998;10:1218.
e9. Van Gerven R, Delooz H, Sermeus W. Systematic triage in the emergency department using
the Australian national triage scale: a pilot project. Eur J Emerg Med. 2001;8:37. [PubMed]
e10. Doherty SR, Hore CT, Curran SW. Inpatient mortality as related to triage category in three
New South Wales regional base hospitals. Emerg Med (Fremantle) 2003;15:334340. [PubMed]
e11. Considine J, LeVasseur SA, Villanueva E. The Australasian triage scale: examining
emergency department nurses performance using computer and paper scenarios. Ann Emerg
Med. 2004;44:516523. [PubMed]
e12. Creaton A, Liew D, Knott J, Wright M. Interrater reliability of the Australasian triage scale
for mental health patients. Emerg Med Australas. 2008;20:468474. [PubMed]
e13. Beveridge R, Ducharme J, Janes L, Beaulieu S, Walter S. Reliability of the Canadian
emergency department triage and acuity scale: interrater agreement. Ann Emerg Med.
1999;34:155159. [PubMed]
e14. Manos D, Petrie DA, Beveridge RC, Walter S, Ducharme J. Inter-observer agreement using
the Canadian emergency department triage and acuity scale. CJEM. 2002;4:1622. [PubMed]
e15. Worster A, Gilboy N, Fernandes CM, et al. Assessment of inter-observer reliability of two
five-level triage and acuity scales: a randomized controlled trial. CJEM. 2004;6:240245.
[PubMed]
e16. Grafstein E, Innes G, Westman J, Christenson J, Thorne A. Inter-rater reliability of a
computerized presenting-complaint-linked triage system in an urban emergency department.
CJEM. 2003;5:323329. [PubMed]
e17. Jimenez JG, Murray MJ, Beveridge R, et al. Implementation of the Canadian emergency
department triage and acuity scale (CTAS) in the principality of Andorra: Can triage parameters
serve as emergency department quality indicators? CJEM. 2003;5:315322. [PubMed]
e18. Goransson K, Ehrenberg A, Marklund B, Ehnfors M. Accuracy and concordance of nurses
in emergency department triage. Scand J Caring Sci. 2005;19:432438. [PubMed]
e19. Dong SL, Bullard MJ, Meurer DP, et al. Predictive validity of a computerized emergency
triage tool. Acad Emerg Med. 2007;14:1621. [PubMed]
e20. Dong SL, Bullard MJ, Meurer DP, Blitz S, Holroyd BR, Rowe BH. The effect of training on
nurse agreement using an electronic triage system. CJEM. 2007;9:260266. [PubMed]
e21. Bergeron S, Gouin S, Bailey B, Amre DK, Patel H. Agreement among pediatric health care
professionals with the pediatric Canadian triage and acuity scale guidelines. Pediatr Emerg Care.
2004;20:514518. [PubMed]
e22. Gravel J, Gouin S, Bailey B, Roy M, Bergeron S, Amre D. Reliability of a computerized
version of the pediatric Canadian triage and acuity scale. Acad Emerg Med. 2007;14:864869.
[PubMed]
e23. Ma W, Gafni A, Goldman RD. Correlation of the Canadian pediatric emergency triage and
acuity scale to ED resource utilization. Am J Emerg Med. 2008;26:893897. [PubMed]
e24. Gravel J, Gouin S, Manzano S, Arsenault M, Amre D. Interrater agreement between nurses
for the pediatric Canadian triage and acuity scale in a tertiary care center. Acad Emerg Med.
2008;15:12621267. [PubMed]
e25. Wuerz RC, Milne LW, Eitel DR, Travers D, Gilboy N. Reliability and validity of a new
five-level triage instrument. Acad Emerg Med. 2000;7:236242. [PubMed]
e26. Wuerz RC, Travers D, Gilboy N, Eitel DR, Rosenau A, Yazhari R. Implementation and
refinement of the emergency severity index. Acad Emerg Med. 2001;8:170176. [PubMed]
e27. Wuerz R. Emergency severity index triage category is associated with six-month survival.
ESI triage study group. Acad Emerg Med. 2001;8:6164. [PubMed]
e28. Travers DA, Waller AE, Bowling JM, Flowers D, Tintinalli J. Five-level triage system more
effective than three-level in tertiary emergency department. J Emerg Nurs. 2002;28:395400.
[PubMed]
e29. Eitel DR, Travers DA, Rosenau AM, Gilboy N, Wuerz RC. The emergency severity index
triage algorithm version 2 is reliable and valid. Acad Emerg Med. 2003;10:10701080.
[PubMed]
e30. Tanabe P, Gimbel R, Yarnold PR, Kyriacou DN, Adams JG. Reliability and validity of
scores on The emergency severity index version 3. Acad Emerg Med. 2004;11:5965. [PubMed]
e31. Tanabe P, Gimbel R, Yarnold PR, Adams JG. The emergency severity index (version 3) 5-
level triage system scores predict ED resource consumption. J Emerg Nurs. 2004;30:2229.
[PubMed]
e32. Baumann MR, Strout TD. Evaluation of the emergency severity index (version 3) triage
algorithm in pediatric patients. Acad Emerg Med. 2005;12:219224. [PubMed]
e33. Chi CH, Huang CM. Comparison of the emergency severity index (ESI) and the Taiwan
triage system in predicting resource utilization. J Formos Med Assoc. 2006;105:617625.
[PubMed]
e34. Worster A, Fernandes CM, Eva K, Upadhye S. Predictive validity comparison of two five-
level triage acuity scales. Eur J Emerg Med. 2007;14:188192. [PubMed]
e35. Baumann MR, Strout TD. Triage of geriatric patients in the emergency department: validity
and survival with the emergency severity index. Ann Emerg Med. 2007;49:234240. [PubMed]
e36. Elshove-Bolk J, Mencl F, van Rijswijck BT, Simons MP, van Vugt AB. Validation of the
emergency severity index (ESI) in self-referred patients in a European emergency department.
Emerg Med J. 2007;24:170174. [PMC free article] [PubMed]



Emergency medical care in developing countries: is it worthwhile?
Razzak JA, Kellermann AL.
Source
Department of Surgery, Yale University School of Medicine, New Haven, CT 06512, USA.
junaid.Razzak@yale.edu
Abstract
Prevention is a core value of any health system. Nonetheless, many health problems will
continue to occur despite preventive services. A significant burden of diseases in developing
countries is caused by time-sensitive illnesses and injuries, such as severe infections, hypoxia
caused by respiratory infections, dehydration caused by diarrhoea, intentional and unintentional
injuries, postpartum bleeding, and acute myocardial infarction. The provision of timely treatment
during life-threatening emergencies is not a priority for many health systems in developing
countries. This paper reviews evidence indicating the need to develop and/or strengthen
emergency medical care systems in these countries. An argument is made for the role of
emergency medical care in improving the health of populations and meeting expectations for
access to emergency care. We consider emergency medical care in the community, during
transportation, and at first-contact and regional referral facilities. Obstacles to developing
effective emergency medical care include a lack of structural models, inappropriate training foci,
concerns about cost, and sustainability in the face of a high demand for services. A basic but
effective level of emergency medical care responds to perceived and actual community needs
and improves the health of populations.


Emergency Department Triage Scales and Their Components: A
Systematic Review of the Scientific Evidence
Abstract
Emergency department (ED) triage is used to identify patients' level of urgency and treat them
based on their triage level. The global advancement of triage scales in the past two decades has
generated considerable research on the validity and reliability of these scales. This systematic
review aims to investigate the scientific evidence for published ED triage scales. The following
questions are addressed:
1. Does assessment of individual vital signs or chief complaints affect mortality during the
hospital stay or within 30 days after arrival at the ED?
2. What is the level of agreement between clinicians' triage decisions compared to each other or
to a gold standard for each scale (reliability)?
3. How valid is each triage scale in predicting hospitalization and hospital mortality?
A systematic search of the international literature published from 1966 through March 31, 2009
explored the British Nursing Index, Business Source Premier, CINAHL, Cochrane Library,
EMBASE, and PubMed. Inclusion was limited to controlled studies of adult patients (15 years)
visiting EDs for somatic reasons. Outcome variables were death in ED or hospital and need for
hospitalization (validity). Methodological quality and clinical relevance of each study were rated
as high, medium, or low. The results from the studies that met the inclusion criteria and quality
standards were synthesized applying the internationally developed GRADE system. Each
conclusion was then assessed as having strong, moderately strong, limited, or insufficient
scientific evidence. If studies were not available, this was also noted.
We found ED triage scales to be supported, at best, by limited and often insufficient evidence.
The ability of the individual vital signs included in the different scales to predict outcome is
seldom, if at all, studied in the ED setting. The scientific evidence to assess interrater agreement
(reliability) was limited for one triage scale and insufficient or lacking for all other scales. Two
of the scales yielded limited scientific evidence, and one scale yielded insufficient evidence, on
which to assess the risk of early death or hospitalization in patients assigned to the two lowest
triage levels on a 5-level scale (validity).
Introduction
Triage is a central task in an emergency department (ED). In this context, triage is viewed as the
rating of patients' clinical urgency [1]. Rating is necessary to identify the order in which patients
should be given care in an ED when demand is high. Triage is not needed if there is no queue for
care. Triage scales aim to optimize the waiting time of patients according to the severity of their
medical condition, in order to treat as fast as necessary the most intense symptom(s) and to
reduce the negative impact on the prognosis of a prolonged delay before treatment. ED triage is a
relatively modern phenomenon, introduced in the 1950s in the United States [2]. Triage is a
complex decision-making process, and several triage scales have been designed as decision-
support systems [3] to guide the triage nurse to a correct decision. Triage decisions may be based
on both the patients' vital signs (respiratory rate, oxygen saturation in blood, heart rate, blood
pressure, level of consciousness, and body temperature) and their chief complaints.
Internationally, no consensus has been reached on the functions that should be measured. Apart
from emergency care, triage may be used in other clinical activities, e.g. deciding on a certain
investigation [4] or treatment [5].
Since the early 1990s, several countries have developed and introduced ED triage [6-10].
Development of triage scales in some countries has been influenced largely by the seminal work
of FitzGerald [11], resulting in most of the triage scales developed in the 1990s and 2000s being
designed as 5-level scales. Of these, the Australian Triage Scale (ATS), Canadian Emergency
Department Triage and Acuity Scale (CTAS), Manchester Triage Scale (MTS), and Emergency
Severity Index (ESI) have had the greatest influence on modern ED triage [12-15]. Other scales
have not disseminated as widely around the globe, e.g. the Soterion Rapid Triage Scale (SRTS)
from the United States and the 4-level Taiwan Triage System (TTS) [6,7,9,16,17]. Some
countries, e.g. Australia, have a national mandatory triage scale while many European countries
lack such standards [7,9].
Patients may have a life-threatening condition, but show normal vital signs. Hence, in triaging
the patient it is important to consider information given by patients or accompanying persons
regarding the patient's chief complaints or medical history, which can provide essential
information about serious diseases. The chief complaints describe the incident or symptoms that
caused the patient to seek care.
In 2005, a joint task force of the American College of Emergency Physicians and the Emergency
Nurses Association published a review of the literature on ED triage scales. Based on expert
consensus and available evidence, the task force supported adoption of a reliable 5-level triage
scale, stating that either the CTAS or the ESI are good choices for ED triage [18]. In 2002, a
national survey conducted in Sweden identified the use of 37 different triage scales across the
country. Further, some 30 EDs did not use any type of triage scale [19].
This systematic review aims to investigate the scientific evidence underlying published ED triage
scales.
Objectives
The following questions are addressed:
1. In triage of adults at EDs, does assessment of individual vital signs or chief complaints affect
mortality during the hospital stay or within 30 days after arrival at the ED?
2. In adult ED patients, what is the level of agreement between clinicians' triage decisions
compared to each other or to a gold standard for each scale (i.e. the reliability of triage scales)?
3. In adult ED patients, how valid is each triage scale in predicting hospitalization and hospital
mortality?
Methods
A systematic search of the international literature published from 1966 through March 31, 2009
explored the British Nursing Index, Business Source Premier, CINAHL, Cochrane Library,
EMBASE, and PubMed. Inclusion was limited to studies of adult patients (15 years) visiting
EDs for somatic reasons. Another criterion for inclusion was that the study design must contain a
control, i.e. randomized controlled trials (RCT), observational studies with a control group based
on previously collected data, and before-after studies. Descriptive studies without a control group
and retrospective studies were excluded.
Inclusion criteria for vital signs and chief complaints used in triage scales
Studies analyzing individual vital signs or chief complaints
Outcome variable defined as death within 30 days after ED arrival or during the hospital stay
Inclusion criteria for reliability and validity of triage scales
Studies based on real patients triaged at EDs (validity)
Studies based on real patients triaged at EDs or fictitious patient scenarios (reliability)
Studies reporting reliability at separate triage levels (reliability)
Studies reporting mortality and hospitalization per triage level (validity)
Outcome variables defined as death in the ED or hospital, and need for hospitalization
(validity)
Exclusion criteria for studies on reliability of triage scales
Studies on interrater reproducibility are excluded in cases where any rater in the study had
access to retrospective data only.
Six experts from different professions and clinical specialties reviewed the studies,
independently in groups of 2 or 3, for quality by using methods validated for internal validity,
precision, and applicability (external validity) [20]. The methodological quality and clinical
relevance of each study was graded as high, medium, or low. Results from the studies that met
the inclusion criteria and quality standards were synthesized by applying the internationally
developed GRADE system [21].
In accordance with GRADE, the following factors were considered in appraising the overall
strength of the evidence: study quality, concordance/consistency, transferability/relevance,
precision of data, risk of publication bias, effect size, and dose-response. In synthesizing the
data, studies having low quality and relevance were included when studies of medium quality and
relevance were not available. Based on the overall quality and relevance of the studies reviewed,
each conclusion was rated as having strong, moderately strong, limited, or insufficient scientific
evidence. If studies were not available, this was noted [21].
Results
Figures 1 and 2 illustrate the results of the primary search.
Figure 1. Results of literature search and selection process.
Figure 2. Results of literature search and selection process regarding
reliability (10 articles), and validity (10 articles) of triage scales. One article studied both
reliability and validity and was rated differently due to the studied endpoint, low quality
regarding reliability and medium quality regarding validity.
Vital signs and chief complaints
Most of the studies that investigated associations between different vital signs or chief
complaints and mortality after ED arrival were observational cohort studies based on selected,
diagnosis-specific, patient groups. All of the studies were found to have medium quality and
relevance. Only a few studies included all patients (albeit limited to "medical" patients") that
arrived at the ED, regardless of diagnosis. Hence, studies of patients classified as surgical
disciplines were generally lacking. Several studies described compiled scales or indexes for
appraising the severity level of the patient's conditions, but provided no information on the
importance of specific vital signs or chief complaints. Hence, little or no evidence can be found
on the association between specific vital signs or reasons for the ED visit and mortality in the
group of general patients presenting in EDs.
Respiratory rate
Only a single study, which described the predictive importance of respiratory rate, fulfilled the
inclusion criteria [22]. The study aimed to assess whether the Rapid Acute Physiology Score
(RAPS) could be used to predict mortality in nonsurgical patients on ED arrival. It also aimed to
study whether an advanced version of RAPS, i.e. the Rapid Emergency Medicine Score (REMS),
could yield better predictive information [22].
RAPS was developed for prehospital care and involves assessing respiratory rate, pulse, blood
pressure, and the Glasgow Coma Scale (GCS). REMS is based on RAPS, but also assesses
oxygen saturation, body temperature, and age. In total, 11 751 patients were studied
prospectively after arrival at the ED of a university hospital in Sweden. Respiratory rate was
found to be a significant predictor of mortality during the hospital stay. A decrease of one step on
the RAPS scale was found to nearly double the risk of mortality within 30 days (Table 1).
Table 1. Does assessment of certain vital signs and chief complaints in emergency department
triage of adults have an impact on 30-day or in-hospital mortality?
Oxygen saturation in blood
Two studies used RAPS and REMS to predict acute mortality after ED arrival and specifically
studied the predictive importance of saturation [22,23]. Oxygen saturation was found to be one
of the three variables, along with age and level of consciousness, that best predicted mortality
during hospitalization.
Pulse
One study investigated the importance of assessing pulse in the ED as a means to predict
mortality during the hospital stay.
The study, which was conducted in Sweden [22], showed a significant association between the
pulse on arrival to the ED and mortality during the hospital stay in a group of 11 751 patients
receiving care for nonsurgical disorders. With a decrease of one step on the RAPS scale, 67% of
the patients showed an increased risk of mortality within 30 days.
Level of consciousness
The Swedish study (described above) also investigated the association between acute mortality
and the level of consciousness on arrival at the ED [22]. Another study used the same methods
mentioned above, i.e. RAPS and REMS [23], to analyze 5583 patients that had called the
emergency phone number and were classified as urgent. The study showed that level of
consciousness was one of three variables (age and saturation being the other two) that best
predicted mortality during the hospital stay. Another study analyzed 986 stroke patients on ED
arrival. Impaired level of consciousness appeared to be the best predictor of mortality during the
hospital stay [24].
Blood pressure and body temperature
The importance of blood pressure or body temperature in assessing the risk of acute mortality
after ED arrival could not be supported by the included studies due to the lack of scientific
evidence.
Chief complaints
Studies describing the association between different chief complaints and acute mortality were
found to be lacking.
Age
Three of the studies described above showed that the higher the patient's age, the greater the risk
of death within 30 days of hospital care following ED arrival [22-24]. The results showed an
increase in mortality of 5% per year. Furthermore, one study showed that older patients (above
75 years of age) with symptoms of coronary heart disease had a greater risk of death within 30
days after arrival at the ED compared to younger patients with the same symptoms [25] (Table
1).
Based on the studies described above, Table 2 summarizes assessments and comments regarding
the level of scientific evidence.
Table 2. Appraisal of scientific evidence according to GRADE - Association between vital
signs/chief complaints and acute mortality after arrival at the emergency department.
Interrater agreement of triage scales (reliability)
All 11 articles that were found to answer the question concerning reliability of triage scales and
met the defined inclusion criteria were observational studies. They addressed reliability of the
ATS [26], CTAS (including eTriage) [19,27-30], MTS [31], SRTS [6], and two locally produced
scales without names [8,32] (Table 3). Based on the quality review, 9 articles [6,8,19,26-31]
were found to be of low and 1 [32] of medium quality. One article was excluded due to deficient
quality resulting from high internal dropout [16]. Deficient external validity was the major
reason for the low- and medium-quality ratings of the studies. Selection of patients and triage
nurses were both found to be irrelevant or insufficiently described. Hence, 10 articles remained
as a basis for the conclusions.
Table 3. Reliability of triage scales
The scientific evidence was found to be insufficient to assess the reliability of ATS, CTAS,
MTS, SRTS and the Swiss scale (Table 4). However, limited scientific evidence was found in
assessing the reproducibility of the Brillman scale (North America) as having moderate interrater
agreement.
Table 4. Appraisal of scientific evidence (according to GRADE) - Reliability of triage scales.
Validity of triage scales regarding acute mortality and hospital admission rates
Mortality
None of the studies reported on hospital admission rates adjusted for age and gender or mortality
(Table 5). Since previous studies have shown that age is one of the major predictors of hospital
mortality [33,34] the scientific evidence was found to be insufficient to asses the validity of the
triage scales ATS, CTAS, and Medical Emergency Triage and Treatment System (METTS)
(Table 6). However, safety as measured by hospital mortality in patients graded as low risk
(triage levels 4-5/green-blue) by the triage systems may be regarded as one aspect of validity.
When assessing the above-mentioned triage scales' level of validity as regards mortality at the
lowest triage levels only (levels 4-5/green-blue), the quality and relevance of the studies were
found to be moderate. Hence, scientific evidence is limited.
Table 5. Studies on how the assessment of the urgency of need to see a physician according to
different triage systems could predict hospital mortality.
Table 6. Appraisal of scientific evidence (according to GRADE) - Validity of 5-level triage
scales measured by acute mortality.
Hospital admission rates in patients triaged as non-acute
Nine studies reported on admission rates for the ESI, ATS, and SRTS triage scales (Table 7).
The studies showed a range between 0.0% and 17.0% at level 5, the lowest triage level [6,16,35-
41]. A range was also observed in the age panorama (mean ages between 30 and 47 years) and in
hospital admission rates at triage level 4 (3%-33%): 18% to 33% for ATS, 6% to 10% for ESI,
and 3% for SRTS.
Table 7. Studies on how the assessment of the urgency of need to see a physician according to
different triage systems could predict hospitalization.
Seven of these studies were found to be of moderate and two of low quality and relevance, and
the scientific evidence for validity of admission rates for patients in the lowest triage levels
(levels 4-5/green-blue) was found to be limited (Table 8).
Table 8. Appraisal of scientific evidence (according to GRADE) - Safety of 5-level triage scales
as measured by hospitalisation rates in patients at triage level 5.
Discussion
Our systematic review shows that when adjudicated by standard criteria for study quality and
scientific evidence, the triage scales used in EDs are supported, at best, by limited evidence.
Often, the evidence is weaker, not above insufficient by the GRADE criteria. The ability of the
individual vital signs included in the different scales to predict outcome has seldom, or never,
been studied in the ED setting. The scientific evidence for assessing interrater agreement
(reproducibility) was limited for one triage scale (Brillman) whereas it was insufficient or
lacking for all other scales. Two of the scales (CTAS and ATS) offered limited scientific
evidence, and the scientific evidence for one scale (METTS) was insufficient to assess the risk of
early death or hospitalization in patients assigned to the two lowest triage levels in 5-level scales;
the studies showed the risk of death to be low, but a need for inpatient care was not excluded
(about 5% hospital admission rate on average). Studies on validity of the triage scales across all
levels, i.e. their ability to distinguish the urgency in patients assigned the five different levels,
were generally of low quality. Consequently, evidence was insufficient to assess the validity of
the scales.
As none of the studies reported on mortality rates adjusted for differences in age and gender
between the triage levels, we could not evaluate the validity of the triage scales across all triage
levels as regards the risk of early death. To estimate the safety of the scales, we studied early
death among patients assigned to the lowest triage levels (green and blue/4-5). Two triage scales
(ATS and CTAS) offered limited scientific evidence for assessing safety. In both scales, the
patients assigned to the two lowest triage levels had a very low risk of dying within 24 hours
after triage. Hence, in this respect, the scales are safe to use. Scientific evidence for METTS, the
newly developed Swedish triage scale, was found to be insufficient to assess safety. Since the
study recorded the risk of dying during the in-hospital stay, mortality was higher than in the
studies on ATS and CTAS.
In using the need of hospitalization as a measure of safety, the situation was found to be more
complex. Again, none of the studies reported on hospital admission rates adjusted for age and
gender, so we could not evaluate the validity of the triage scales across all triage levels.
However, on average, about 5% (in some studies up to 17%) of patients in the lowest (4-5/green-
blue) triage levels in ATS, ESI, and SRTS were reported to be admitted as inpatients. The
variations were wide not only between different triage scales, but also between studies using the
same scales. This indicates differences between the studies in (a) patient populations in the ED,
(b) access to hospital beds, (c) hospital admission policies and traditions, and/or (d) inaccurate
triage decisions (i.e. patients were rated as less urgent than their actual urgency).
No definitive conclusions could be drawn regarding which of the scales was the safest as
measured by the need of hospitalization. Hence, we suggest that none of the scales be used in
referral of patients in the lowest triage levels (4-5/green-blue), e.g. to primary care, without
further medical examination in the ED.
New diagnostic tests typically need to meet rigid criteria before they can be accepted for
widespread use. These criteria include documentation on precision. For non-laboratory tests,
interrater agreement (reliability) is a key precision issue. Our review shows that most triage
scales present insufficient scientific evidence for assessing interrater agreement. The study
designs used to estimate interrater agreement have often been suboptimal. Most of the studies are
based on fictitious cases rather than on authentic patients in real-life settings. The value of the
studies as regards interrater agreement is also compromised by the fact that the mean age of
patients assessed has either been low (as low as 30 years) or unreported. The generalizability to
real-life ED patients must therefore be questioned.
All 5-level triage scales present insufficient evidence on interrater variability. The few studies
that have been published (most of low quality) have reported widely divergent interrater
agreement, with kappa values ranging from 0.2 (slight agreement) to 0.9 (almost perfect). Only a
single study [32] presented limited scientific evidence. This was a 4-grade scale reporting a
kappa value of 0.45, a value usually considered to be in the moderate agreement range [42]. It is
evident that inter-observer agreement in triage scales must be documented in greater detail, and,
if low, actions must be taken to reduce variability.
The literature shows variations in the vital signs and chief complaints applied in triage scales. It
is unclear whether the selected vital signs are the best at distinguishing different risk groups.
Further, evidence supporting the selected thresholds for continuous variables is deficient. The
inclusion criteria for this systematic literature review place considerable emphasis on relevance.
Triage scales are intended to be used in EDs irrespective of specific symptoms or disease. Hence,
only studies of unselected patient populations in ED settings were included, greatly limiting the
number of studies on the ability of individual vital signs to predict outcome. Our literature search
revealed that many more studies had been performed in intensive care units, or soon after
hospital admission.
Regarding specific vital signs, limited scientific evidence supports the use of oxygen saturation
and consciousness level as predictors of mortality early after triage. However, scientific evidence
was found to be insufficient as regards respiration and pulse, blood pressure, and body
temperature. Hence, it remains unclear whether the selected vital signs are the best ones to use in
distinguishing different risk groups. Moderate scientific evidence indicated age as a predictor of
mortality early after triage, yet most triage scales do not take age into account.
MTS and eCTAS include the chief complaint leading to the ED visit, but we did not find any
studies that analyzed which of the chief complaints are important predictors of mortality early
after triage. It appears likely that in the construction of triage scales, much of the information
was deduced from studies performed in settings other than EDs.
Strengths and limitations
The strength of this review of the scientific literature on triage in the ED lies in its systematic
approach. Our search for relevant literature has been meticulous; the quality of the included
studies has been evaluated in a uniform manner; and the level of evidence has been summarized
using the GRADE methodology developed under the auspices of the World Health Organization
[21].
Our review is limited to ED triage in adult patients in somatic care. However, EDs are only part
of a continuum of services for acutely ill and injured patients. Studies are also needed in other
aspects along the continuum of care, e.g. prehospital, psychiatric, and pediatric triage. Other
limitations are ascribed to the volume and quality of the scientific literature available. Since all
studies were observational, none of the evidence came from randomized controlled trials, the
"gold standard" for evaluating new methods. As none of the studies met the standards for high
quality, we included studies of low and moderate quality in our review in accordance with the
creed in evidence based medicine to use the best available scientific evidence. Low study quality
affected the GRADE rating and was a reason why scientific evidence was rated as insufficient or
limited for so many aspects of so many scales.
Conclusions
This systematic literature review reveals shortcomings in the scientific evidence on which
presently available triage scales are based. Stronger scientific evidence is needed to determine
which of the vital signs and chief complaints have the greatest prognostic value in triage.
Interrater agreement (reliability), validity, and safety of triage scales need to be investigated
further, and head-to-head comparisons are needed to determine whether any of the scales have
advantages over others.
Limitations
This review was confined to ED triage scales for adult ED patients with non-psychiatric illnesses
or injuries. In the absence of an internationally agreed outcome measure for ED triage scale
validity, the proxy variables hospital admission and mortality were used in the current study.
These proxy variables have limitations with regards to ED triage scale validity as the variables
may be affected by events occurring after the triage assessment. Further, comparison between
ED triage scales need to be done with caution as there may be contextual differences influencing
the result.
Competing interests
The authors declare that they have no competing interests.
Authors' contributions
All authors contributed to study concept and design, and acquisition, analysis, and interpretation
of the data. Finally all authors read and approved the submitted manuscript.
References
1. Gerber P, McNair R: From Triage exxence and process. In Traige nursing secrets.
Edited by Gerber Zimmerman P, Herr R. Missouri: Mosby Inc; 2006.
2. Gilboy N, Travers D, Wuerz R: Re-evaluating triage in the new millennium: A
comprehensive look at the need for standardization and quality.
J Emerg Nurs 1999, 25:468-73. PubMed Abstract | Publisher Full Text
3. Bullard MJ, Unger B, Spence J, Grafstein E: Revisions to the Canadian Emergency
Department Triage and Acuity Scale (CTAS) adult guidelines.
CJEM 2008, 10:136-51. PubMed Abstract | Publisher Full Text
4. Jolliffe VM, Harris DW, Morris R, Wallacet P, Whittaker SJ: Can we use video images
to triage pigmented lesions?
Br J Dermatol 2001, 145:904-10. PubMed Abstract | Publisher Full Text
5. Alcazar JL, Royo P, Jurado M, Minguez JA, Garcia-Manero M, Laparte C, et al.: Triage
for surgical management of ovarian tumors in asymptomatic women: assessment of
an ultrasound-based scoring system.
Ultrasound Obstet Gynecol 2008, 32:220-5. PubMed Abstract | Publisher Full Text
6. Maningas PA, Hime DA, Parker DE, McMurry TA: The Soterion Rapid Triage
System: evaluation of inter-rater reliability and validity.
J Emerg Med 2006, 30:461-9. PubMed Abstract | Publisher Full Text
7. Parenti N, Ferrara L, Bacchi ML, Sangiorgi D, Lenzi T: Reliability and validity of two
four-level emergency triage systems.
Eur J Emerg Med 2009, 16:115-20. PubMed Abstract | Publisher Full Text
8. Rutschmann OT, Kossovsky M, Geissbuhler A, Perneger TV, Vermeulen B, Simon J, et
al.: Interactive triage simulator revealed important variability in both process and
outcome of emergency triage.
J Clin Epidemiol 2006, 59:615-21. PubMed Abstract | Publisher Full Text
9. Taboulet P, Moreira V, Haas L, Porcher R, Braganca A, Fontaine JP, et al.: Triage with
the French Emergency Nurses Classification in Hospital scale: reliability and
validity.
Eur J Emerg Med 2009, 16:61-7. PubMed Abstract | Publisher Full Text
10. Widgren B, Jourak M: Medical Emergency Triage and Treatment System (METTS):
A new protocol in primary triage and secondary priority decision in emergency
medicine.
J Emerg Med
In press

11. FitzGerald G: Emergency department triage. University of Brisbane: Brisbane; 1989.
12. Australasian College for Emergency Medicine: Guidelines on the implementation of
the Australasian triage scale in emergency departments. [http:/ / www.acem.org.au/
media/ policies_and_guidelines/ G24_Implementation__ATS.pdf] webcite
13. Gilboy N, Tanabe P, Travers D, Eitel D, Wuerz R: The Emergency Severity Index. In
Emergency Nurses Association. Des Plaines; 2003.
14. Manchester Triage Group: Emergency Triage. Second edition. Blackwell Publishing Inc:
Harayana; 2006.
15. Murray M, Bullard M, Grafstein E: Revisions to the Canadian Emergency
Department Triage and Acuity Scale implementation guidelines.
CJEM 2004, 6:421-7. PubMed Abstract | Publisher Full Text
16. Chi CH, Huang CM: Comparison of the Emergency Severity Index (ESI) and the
Taiwan Triage System in predicting resource utilization.
J Formos Med Assoc 2006, 105:617-25. PubMed Abstract | Publisher Full Text
17. Gottschalk SB, Wood D, DeVries S, Wallis LA, Bruijns S: The Cape Triage Score: a
new triage system South Africa. Proposal from the Cape Triage Group.
Emerg Med J 2006, 23:149-53. PubMed Abstract | Publisher Full Text |
PubMed Central Full Text
18. Fernandes CM, Tanabe P, Gilboy N, Johnson LA, McNair RS, Rosenau AM, et al.: Five-
level triage: a report from the ACEP/ENA Five-level Triage Task Force.
J Emerg Nurs 2005, 31:39-50.
quiz 118
PubMed Abstract | Publisher Full Text
19. Goransson K, Ehrenberg A, Marklund B, Ehnfors M: Accuracy and concordance of
nurses in emergency department triage.
Scand J Caring Sci 2005, 19:432-8. PubMed Abstract | Publisher Full Text
20. [http://www.sbu.se/upload/Publikationer/Content0/1/Bilagor_triage.pdf] webcite
21. Grading quality of evidence and strength of recommendations
[http://www.bmj.com/cgi/content/full/328/7454/1490] webcite
BMJ
22. Olsson T, Terent A, Lind L: Rapid Emergency Medicine score: a new prognostic tool
for in-hospital mortality in nonsurgical emergency department patients.
J Intern Med 2004, 255:579-87. PubMed Abstract | Publisher Full Text
23. Goodacre S, Turner J, Nicholl J: Prediction of mortality among emergency medical
admissions.
Emerg Med J 2006, 23:372-5. PubMed Abstract | Publisher Full Text |
PubMed Central Full Text
24. Arboix A, Garcia-Eroles L, Massons J, Oliveres M: Predictive factors of in-hospital
mortality in 986 consecutive patients with first-ever stroke.
Cerebrovasc Dis 1996, 6:161-5. Publisher Full Text
25. Han JH, Lindsell CJ, Hornung RW, Lewis T, Storrow AB, Hoekstra JW, et al.: The elder
patient with suspected acute coronary syndromes in the emergency department.
Acad Emerg Med 2007, 14:732-9. PubMed Abstract | Publisher Full Text
26. Considine J, Ung L, Thomas S: Triage nurses' decisions using the National Triage
Scale for Australian emergency departments.
Accid Emerg Nurs 2000, 8:201-9. PubMed Abstract | Publisher Full Text
27. Beveridge R, Ducharme J, Janes L, Beaulieu S, Walter S: Reliability of the Canadian
emergency department triage and acuity scale: interrater agreement.
Ann Emerg Med 1999, 34:155-9. PubMed Abstract | Publisher Full Text
28. Dong SL, Bullard MJ, Meurer DP, Blitz S, Ohinmaa A, Holroyd BR, et al.: Reliability of
computerized emergency triage.
Acad Emerg Med 2006, 13:269-75. PubMed Abstract | Publisher Full Text
29. Dong SL, Bullard MJ, Meurer DP, Colman I, Blitz S, Holroyd BR, et al.: Emergency
triage: comparing a novel computer triage program with standard triage.
Acad Emerg Med 2005, 12:502-7. PubMed Abstract | Publisher Full Text
30. Manos D, Petrie DA, Beveridge RC, Walter S, Ducharme J: Inter-observer agreement
using the Canadian Emergency Department Triage and Acuity Scale.
CJEM 2002, 4:16-22. PubMed Abstract | Publisher Full Text
31. van der Wulp I, van Baar ME, Schrijvers AJ: Reliability and validity of the
Manchester Triage System in a general emergency department patient population in
the Netherlands: results of a simulation study.
Emerg Med J 2008, 25:431-4. PubMed Abstract | Publisher Full Text
32. Brillman JC, Doezema D, Tandberg D, Sklar DP, Davis KD, Simms S, et al.: Triage:
limitations in predicting need for emergent care and hospital admission. Ann Emerg
Med.
1996, 27:493-500.
33. Safwenberg U, Terent A, Lind L: The Emergency Department presenting complaint
as predictor of in-hospital fatality.
Eur J Emerg Med 2007, 14:324-31. PubMed Abstract | Publisher Full Text
34. Safwenberg U, Terent A, Lind L: Differences in long-term mortality for different
emergency department presenting complaints.
Acad Emerg Med 2008, 15:9-16. PubMed Abstract | Publisher Full Text
35. Dent A, Rofe G, Sansom g: Which triage category patients die in hospital after being
admitted through emergency departments? A study in one teaching hospital.
Emerg Med 1999, 11:68-71. Publisher Full Text
36. Doherty SR, Hore CT, Curran SW: Inpatient mortality as related to triage category in
three New South Wales regional base hospitals.
Emerg Med (Fremantle) 2003, 15:334-40. Publisher Full Text
37. Eitel DR, Travers DA, Rosenau AM, Gilboy N, Wuerz RC: The emergency severity
index triage algorithm version 2 is reliable and valid.
Acad Emerg Med 2003, 10:1070-80. PubMed Abstract | Publisher Full Text
38. Tanabe P, Gimbel R, Yarnold PR, Kyriacou DN, Adams JG: Reliability and validity of
scores on The Emergency Severity Index version 3.
Acad Emerg Med 2004, 11:59-65. PubMed Abstract | Publisher Full Text
39. Van Gerven R, Delooz H, Sermeus W: Systematic triage in the emergency department
using the Australian National Triage Scale: a pilot project.
Eur J Emerg Med 2001, 8:3-7. PubMed Abstract | Publisher Full Text
40. Wuerz RC, Milne LW, Eitel DR, Travers D, Gilboy N: Reliability and validity of a new
five-level triage instrument.
Acad Emerg Med 2000, 7:236-42. PubMed Abstract | Publisher Full Text
41. Wuerz RC, Travers D, Gilboy N, Eitel DR, Rosenau A, Yazhari R: Implementation and
refinement of the emergency severity index.
Acad Emerg Med 2001, 8:170-6. PubMed Abstract | Publisher Full Text
42. Landis JR, Koch GG: An application of hierarchical kappa-type statistics in the
assessment of majority agreement among multiple observers.
Biometrics 1977, 33:363-74. PubMed Abstract | Publisher Full Text
43. Dong SL, Bullard MJ, Meurer DP, Blitz S, Akhmetshin E, Ohinmaa A, et al.: Predictive
validity of a computerized emergency triage tool.
Acad Emerg Med 2007, 14:16-21. PubMed Abstract | Publisher Full Text