ORIGINAL ARTICLE

Reliability of upper airway linear, area, and

volumetric measurements in cone-beam
computed tomography
Claudia Trindade Mattos,a Christiane Vasconcellos Cruz,b Thais Cristina Sobreira da Matta,c
Leonardo de Abreu Pereira,d Priscilla de Almeida Solon-de-Mello,e Anto^ nio Carlos de Oliveira Ruellas,f
and Eduardo Franzotti Sant’Annaf
Niteroi and Rio de Janeiro, Brazil

Introduction: Our objective was to assess the intraexaminer and interexaminer reliabilities of upper airway
linear, area, and volumetric measurements in cone-beam computed tomography. Methods: Cone-beam
computed tomography scans of 12 subjects were randomly selected from a pool of 132 orthodontic patients.
An undergraduate student, an orthodontist, and a dental radiologist independently made linear, area, and volu-
metric measurements. Linear anteroposterior and transversal measurements, cross-sectional area, sagittal
area, minimum axial area, and volume measurements were made. The intraclass correlation coefficient (ICC)
was used to assess intraexaminer and interexaminer reliabilities, and measurement errors were assessed.
Agreement was further assessed with the Bland-Altman method and 95% limits of agreement. Results: Overall,
the ICC values indicated good reliability for the measurements assessed. The ICC values were greater than 0.9
(excellent) for 93% of intraexaminer and 73% of interexaminer assessments. Transversal width measurements
and cross-sectional area at the level of the vallecula, however, had only moderate reliability (minimum ICC,
0.63), large 95% limits of agreement, and the greatest mean measurement errors (as high as 16% and 13%
of the mean measurements, respectively). Linear anteroposterior measurements; cross-sectional areas at the
levels of the palatal plane, soft palate, and tongue; and sagittal area and volume were reliable
measurements, with a minimum ICC of 0.93 and more restricted limits of agreement. Conclusions: Based
on these results, airway assessments by examiners with different backgrounds might have reliable anteropos-
terior linear measurements; cross-sectional areas at the levels of the palatal plane, soft palate, and tongue; and
sagittal area and volume. The unreliable measurements were linear width, cross-sectional area at the level of the
vallecula, and minimum axial area. (Am J Orthod Dentofacial Orthop 2014;145:188-97)

A
n increasing interest in cone-beam computed
a
tomography (CBCT) and the benefits it can bring
Professor, Department of Dental Clinics, Universidade Federal Fluminense,
Niteroi, Brazil. to diagnosis and treatment planning in the oral
b
PhD student, Department of Pediatric Dentistry and Orthodontics, Universidade and maxillofacial specialties can be seen in the literature
Federal do Rio de Janeiro, Rio de Janeiro, Brazil.
c
in the last decade.1-7 Particularly, CBCT airway
Postgraduate student, Department of Pediatric Dentistry and Orthodontics, Uni-
versidade Federal do Rio de Janeiro, Rio de Janeiro, Brazil. assessment has been studied by many authors, because
d
Undergraduate student, School of Dentistry, Universidade Federal do Rio de Ja- this 3-dimensional (3D) tool has many advantages over
neiro, Rio de Janeiro, Brazil. the way we analyzed the airway in the past: ie, mainly
e
Postgraduate student, Department of Pediatric Dentistry and Orthodontics, Uni-
versidade Federal do Rio de Janeiro, Rio de Janeiro, Brazil. through 2-dimensional cephalometric radiographs.8-14
f
Associate professor, Department of Orthodontics, Universidade Federal do Rio Two recent systematic reviews on airway analysis
de Janeiro, Rio de Janeiro, Brazil. with CBCT images, one focusing on clinical indications,
All authors have completed and submitted the ICMJE Form for Disclosure of
Potential Conflicts of Interest, and none were reported. technical parameters, and accuracy and reliability of up-
This work was supported by scholarships to individual authors and grants to Uni- per airway analysis,15 and the other on upper airway seg-
versidade Federal do Rio de Janeiro from CAPES, CNPq, and FAPERJ. mentation,16 show that some aspects are not yet
Address correspondence to: Claudia Trindade Mattos, Faculdade de Odontologia,
Disciplina de Ortodontia, Rua Mario Santos Braga, 30, 2
andar, sala 214, Centro, completely elucidated because of few adequate studies.
Niteroi, RJ, Brazil, CEP: 24020-140; e-mail, claudiatrindademattos@gmail.com. Some aspects regarding upper airway analysis that lack
Submitted, January 2013; revised and accepted, October 2013. evidence and still need to be addressed are the validity
0889-5406/$36.00
Copyright Ó 2014 by the American Association of Orthodontists. and reliability of CBCT-generated 3D models16; the
http://dx.doi.org/10.1016/j.ajodo.2013.10.013 influence of respiratory phase, tongue position, and
188
Mattos et al 189

Fig 1. A and B, Linear measurements at the levels of the palatal plane, soft palate, tongue, and
vallecula: anteroposterior length and transversal width, respectively; C-F, cross-sectional axial slice
areas at the levels of the palatal plane, soft palate, tongue, and vallecula, respectively.

mandibular morphology on the CBCT images obtained;
longitudinal and cross-sectional analysis; and determi-
nation of anatomic limits.15
Accuracy and reliability of airway measurements in
CBCT images have been tested. Lenza et al17 compared
the linear, area, and volumetric measurements by 2 exam-
iners and found no significant differences. However, they
did not use statistical tests to assess the reliability of the
measurements, since that was not the focus of the study.
Luiz and Szklo18 suggested that the results of more than 1
statistical strategy should be considered when assessing
the reliability of quantitative measurements.
The best approach in airway assessment is a thorough
analysis, including linear measurements, area, and vol-
ume, since volume information alone might not neces-
sarily represent or identify the locations of the relevant
constrictions.17
Nevertheless, airway measurements are not easy to
make and demand much attention and concentration.
Whether the examiner's background influences the mea-
surements when he or she has been properly calibrated
remains unanswered.
The purpose of this study was to assess intraexaminer
and interexaminer reliabilities of upper airway linear,
area, and volumetric measurements in CBCT images
among examiners with different backgrounds.
MATERIAL AND METHODS
The sample size estimates recommended by Walter
et al19 for reliability studies were followed, considering
r0 5 0.5 (minimum acceptable level of reliability),
r1 5 0.9 (expected level of reliability), a 5 0.05, b 5
0.2 (which implies a power test of 80%), n 5 2

American Journal of Orthodontics and Dentofacial Orthopedics February 2014  Vol 145  Issue 2
190 Mattos et al

Fig 2. A, Limits determined to assess airway volume; B, calculations of sagittal area and minimum
axial area (white line); C and D, morphology and calculation of airway volumes: right lateral and frontal
views, respectively.

(intraexaminer), and n 5 3 (interexaminer). We deter-
mined that a sample of CBCT images from 9 subjects
would be sufficient.
CBCT images of 12 patients were randomly selected
from an available pool of 132 images from patients in
the orthodontics clinics of the postgraduate course in
dentistry of the Universidade Federal do Rio de Janeiro
in Brazil. The exclusion criteria were cleft, systemic con-
dition, craniofacial syndrome, detectable airway pathol-
ogy, or previous orthognathic or craniofacial surgery.
The research was approved by the research ethics
committee of the Institute of Collective Health Studies
of the Universidade Federal do Rio de Janeiro, and
informed consent was signed by all subjects. The
methods and principles involved in the research com-
plied with the Helsinki Declaration.
The CBCT scans were obtained by 1 operator using
the same I-CAT tomograph (Imaging Sciences Interna-
tional, Hatfield, Pa). The subjects were positioned with
the Frankfort horizontal plane parallel to the floor and
instructed to maintain maximum intercuspation with
the tongue touching the palate and to avoid swallowing
during the scanning period. The imaging protocol used
were 120 kV, 5 mA, 13 3 17 cm field of view, 0.25-
mm voxel size, and a scanning time of 20 seconds.
Images were saved in DICOM files, and these files were
imported into Dolphin Imaging software (version 11.5;
Dolphin Imaging and Management Systems, Chats-
worth, Calif) to obtain the primary reconstructed images
(sagittal, coronal, and axial) and the 3D reconstructions.
Head orientation was the same for each CBCT image
performed by the same experienced operator. The

February 2014  Vol 145  Issue 2 American Journal of Orthodontics and Dentofacial Orthopedics
Mattos et al 191

Table I. Intraexaminer and interexaminer reliabilities estimated by ICC for each measurement
Intraexaminer ICC

Undergraduate student Orthodontist Radiologist Interexaminer ICC

Anteroposterior length (mm) at the level of
Palatal plane 0.981 0.979 0.955 0.983
Soft palate 0.989 0.957 0.987 0.979
Tongue 0.991 0.995 0.996 0.986
Vallecula 0.999 0.995 0.989 0.996
Width (mm) at the level of
Palatal plane 0.998 0.986 0.996 0.969
Soft palate 0.988 0.943 0.999 0.883
Tongue 0.942 0.637 0.976 0.853
Vallecula 0.987 0.855 0.989 0.664
Cross-sectional area (axial slice) (mm2) at the level of
Palatal plane 0.993 0.993 0.993 0.988
Soft palate 0.975 0.984 0.996 0.974
Tongue 0.935 0.974 0.987 0.960
Vallecula 0.993 0.984 0.989 0.696
Sagittal area (mm2) 0.983 0.979 0.985 0.977
Minimum axial area (mm2) 0.999 0.869 0.999 0.932
Volume (mm3) 0.995 0.987 0.994 0.992

Table II. Descriptive statistics and intraexaminer and interexaminer measurement errors assessed by the absolute
mean difference for each measurement
Measurement Intraexaminer Interexaminer

Mean Mean
Mean SD Minimum Maximum difference SD Minimum Maximum difference SD Minimum Maximum
Anteroposterior length (mm) at the level of
Palatal plane 20.70 2.91 17.06 26.61 0.48 0.45 0.00 1.88 0.49 0.39 0.03 1.39
Soft palate 10.35 1.88 8.38 13.76 0.25 0.26 0.00 1.15 0.30 0.19 0.00 0.75
Tongue 11.24 3.53 8.31 20.71 0.32 0.24 0.00 0.99 0.30 0.33 0.00 1.62
Vallecula 15.91 4.06 10.08 24.12 0.32 0.29 0.00 1.21 0.52 0.30 0.00 1.12
Width (mm) at the level of
Palatal plane 26.83 7.49 15.23 37.40 0.48 0.60 0.00 3.26 1.16 1.23 0.06 4.28
Soft palate 21.90 8.08 10.03 35.38 1.13 1.89 0.00 9.43 2.52 2.75 0.11 8.35
Tongue 26.21 6.94 16.62 37.68 2.61 3.55 0.00 15.98 3.52 3.30 0.01 10.98
Vallecula 20.43 7.49 9.81 35.56 1.41 2.25 0.11 10.96 3.29 3.26 0.11 9.25
Cross-sectional area (axial slice) (mm2) at the level of
Palatal plane 485.67 154.65 263.35 825.44 14.99 11.58 0.05 38.78 19.94 14.20 0.18 52.86
Soft palate 224.45 83.85 107.62 405.44 10.25 11.74 0.18 48.66 18.75 17.96 1.23 78.33
Tongue 253.48 75.40 136.90 423.20 13.43 13.54 0.75 56.41 18.33 14.02 0.07 54.27
Vallecula 232.62 79.24 126.17 359.46 8.74 7.96 0.65 30.30 31.15 30.00 0.03 112.14
Sagittal area 701.01 147.80 498.50 886.40 18.28 13.96 0.30 60.90 31.31 30.29 0.50 142.00
(mm2)
Minimum axial 198.41 61.52 103.10 291.20 8.33 14.92 0.00 53.40 12.79 23.28 0.00 74.20
area (mm2)
Volume (mm3) 17184.49 6238.42 7667.40 25512.40 464.68 423.78 11.30 1999.70 860.17 914.53 22.60 3492.20

horizontal reference plane was defined bilaterally by po-
rion, and the head on the right side view was tilted clock-
wise or counterclockwise until the palatal plane—defined
by the anterior nasal spine and the posterior nasal
spine—became parallel to the horizontal reference plane.
This condition was checked later on the midsagittal
plane slice. The transporionic plane was oriented
vertically, defined bilaterally by porion, and perpendic-
ular to the horizontal reference plane. The midsagittal
plane was oriented vertically, defined by nasion, and
perpendicular to the previously defined planes.
A fourth-year dental student (L.A.P.), an orthodon-
tist (T.C.S.M.), and a dental radiologist (C.V.C.) were
oriented, trained, and calibrated as examiners for

American Journal of Orthodontics and Dentofacial Orthopedics February 2014  Vol 145  Issue 2
192 Mattos et al

Table III. Bland-Altman means, standard deviations, 95% limits of agreement (LoA) for intraexaminer differences,
and paired t test P values to detect systematic errors
Undergraduate student Orthodontist

Mean (SD) 95% LoA t test P value Mean (SD) 95% LoA
Anteroposterior length (mm) at the level of
Palatal plane 0.19 (0.57) 0.93; 1.33 0.257 0.05 (0.57) 1.06; 1.18
Soft palate 0.06 (0.25) 0.56; 0.43 0.430 0.04 (0.51) 1.05; 0.96
Tongue 0.16 (0.46) 0.74; 1.06 0.249 0.20 (0.38) 0.53; 0.95
Vallecula 0.09 (0.21) 0.52; 0.32 0.148 0.13 (0.41) 0.67; 0.94
Width (mm) at the level of
Palatal plane 0.05 (0.39) 0.83; 0.71 0.632 0.29 (1.13) 2.50; 1.92
Soft palate 0.98 (1.40) 1.77; 3.74 0.043 1.36 (3.12) 7.48; 4.75
Tongue 0.44 (2.84) 6.02; 5.14 0.616 2.45 (6.55) 15.30; 10.40
Vallecula 0.37 (1.09) 1.77; 2.52 0.261 0.34 (4.39) 8.94; 8.26
Cross-sectional area (axial slice) (mm2) at the level of
Palatal plane 4.77 (19.14) 42.29; 32.74 0.406 5.61 (19.01) 42.87; 31.64
Soft palate 2.87 (20.32) 36.96; 42.70 0.634 0.16 (17.59) 34.32; 34.65
Tongue 2.42 (25.62) 47.78; 52.64 0.749 1.77 (17.25) 35.60; 32.05
Vallecula 3.65 (9.49) 22.25; 14.94 0.209 1.66 (13.06) 23.94; 27.27
Sagittal area (mm2) 7.62 (26.51) 59.58; 44.33 0.362 2.44 (29.54) 60.35; 55.45
Minimum axial area (mm2) 0.25 (0.71) 1.14; 1.65 0.264 7.10 (29.66) 51.03; 65.25
Volume (mm3) 196.82 (648.61) 1468.12; 1074.46 0.338 183.95 (949.42) 2044.84; 1676.92

airway analysis using images not included in this study.
The calibration protocol included an explanation of the
3D measurement tools in the Dolphin Imaging software
and a demonstration of the measurements to be made
for this research (see Video, available at www.ajodo.
org). The examiners were given 4 images for training
at will. Each examiner was considered calibrated
when an intraclass correlation coefficient (ICC) test be-
tween 2 time measurements and between his or her
measurements and those from the trainer was above
0.9 for all variables. After calibration, each examiner
independently made the linear, area, and volumetric
measurements twice in the CBCT images selected for
this study, with a 2-week interval between these pe-
riods. The images were randomly analyzed to allow a
blinded assessment, and the examiners did not have
access to their previous measurements at the second
analysis.
Eight linear measurements, 6 area measurements,
and 1 volumetric measurement were made for each im-
age. Initially, each examiner measured the airway's an-
teroposterior length in the midsagittal plane slice at 4
levels: palatal plane, soft palate, tongue, and vallecula.
At the soft palate and tongue levels, the smaller ante-
roposterior length was determined by visual inspection
and measured. In this same slice view, the coronal
plane was moved to the airway region until it reached
a position 1 mm posterior to the most posterior point
in the soft palate, tongue, or anterior pharyngeal wall
so that the coronal slice would fall into the airway
region in every level that it would have to be measured.
The airway's transversal width was then measured in
the coronal slice at the same levels of the measure-
ments in the sagittal slice. The axial plane was then
moved to each level of the previous measurements in
the sagittal slice, and the airway area was then calcu-
lated by the software after its limits were determined
by the examiner (Fig 1). Finally, the airway/sinus tool
from the software was accessed. Airway limits were
defined with this tool according to the following: up-
per limit, palatal plane extended until it reached the
posterior pharyngeal wall; lower limit, a plane parallel
to the palatal plane passing through the vallecula; pos-
terior limit, posterior pharyngeal wall; and anterior
limit, soft palate, tongue, and anterior pharyngeal
wall. The airway was scanned 3-dimensionally to
ensure that every aspect was included. The software
calculated the sagittal area, the minimum axial area,
and the volume of the airway within the established
limits (Fig 2).
The data were checked carefully, and extreme outliers
were removed. Intraexaminer reliability was calculated
by ICC for the measurements obtained by each examiner
at both times. ICC values were also used to assess inter-
examiner concordance by comparing their second mea-
surements. ICC values were estimated using a 2-way
mixed-effects model. Reliability was ranked according
to the ICC value and considered excellent when it was
above 0.9, good when it was between 0.75 and 0.9,
moderate when it was between 0.5 and 0.75, and poor

February 2014  Vol 145  Issue 2 American Journal of Orthodontics and Dentofacial Orthopedics
Mattos et al 193

Table III. Continued

Radiologist

t test P value Mean (SD) 95% LoA t test P value

0.735 0.11 (0.82) 1.72; 1.49 0.635

0.780 0.11 (0.29) 0.69; 0.47 0.222
0.100 0.10 (0.31) 0.72; 0.51 0.274
0.273 0.12 (0.59) 1.29; 1.05 0.488

0.414 0.08 (0.65) 1.20; 1.36 0.678

0.158 0.08 (0.41) 0.73; 0.89 0.527
0.222 0.20 (1.45) 2.65; 3.06 0.653
0.793 0.18 (1.14) 2.06; 2.43 0.597

0.328 7.21 (17.85) 42.20; 27.76 0.189

0.976 1.71 (7.36) 16.13; 12.71 0.438
0.740 6.73 (12.43) 31.11; 17.64 0.087
0.681 4.67 (12.37) 28.93; 19.57 0.217
0.779 12.25 (25.55) 62.34; 37.83 0.143
0.445 0.99 (2.24) 5.39; 3.41 0.174
0.516 401.47 (672.43) 1719.44; 916.49 0.076

when it was below 0.5. Intraexaminer and interexaminer
measurement errors (average of the absolute mean dif-
ferences) were determined for all variables assessed.
Measurement agreement was further assessed with the
Bland-Altman20 method through means, standard devi-
ations, and 95% limits of agreement, which were avail-
able as data and plots. Ninety comparisons were made in
this method, including 45 intraexaminer comparisons
(the 2 measurements for each of the 15 variables
measured by the 3 examiners) and 45 interexaminer
comparisons (15 variables measured by each pair of ex-
aminers: student and orthodontist, student and radiolo-
gist; and orthodontist and radiologist). Each comparison
generated a plot, and additional plots were constructed
by mixing data from more than 1 comparison by assign-
ing different colors for the comparisons. Paired t tests
were used to detect systematic intraexaminer and inter-
examiner errors in each comparison. P values below 0.01
were considered statistically significant.
RESULTS
Intraexaminer and interexaminer reliabilities estimated
by ICC values for each measurement are shown in Table
I. These results indicate good reliability for both intraexa-
miner and interexaminer assessments. The ICC values
were above 0.9 for 42 (93.3%) of the 45 intraexaminer as-
sessments, and none had poor reliability (ICC \0.5). The
ICC values were above 0.9 for 11 (73.3%) of the 15 inter-
examiner assessments, and none had poor reliability. The
less reliable measurements in this research, with moderate
reliability (0.5 \ICC \0.75), were the transversal widths at
the level of the tongue (intraexaminer for 1 examiner) and
at the level of the vallecula (interexaminer), and the cross-
sectional area at the level of the vallecula (interexaminer).
Table II shows measurement errors (average of the
absolute mean differences) in perspective, considering
their relationship to measurement means, and mini-
mum and maximum values (made by the researcher
[C.T.M.] who trained the examiners). The sample's
descriptive statistics show the variability of airway mea-
surements among the images of the subjects in this
study. As to the measurement errors, the greatest
mean differences (.10% of the mean measurement
value) in linear measurements were the transversal
widths at the level of the soft palate (interexaminer,
11.5%), at the level of the tongue (interexaminer,
13.4%), and at the level of the vallecula (interexaminer,
16.1%). The greatest mean differences in area mea-
surements were the cross-sectional area at the level
of the vallecula (interexaminer, 13.3%). The volumetric
mean differences were below 10% of the mean mea-
surement value. The maximum measurement error
was exceedingly high in relation to the maximum mea-
surement of some variables: intraexaminer, widths at
the levels of the soft palate (26.6%), tongue (42.4%),
and vallecula (30.8%); and interexaminer, widths at
the levels of the tongue (29.1%) and vallecula (26%);
cross-sectional area at the level of the vallecula
(31.1%); and minimum axial area (25.4%).

American Journal of Orthodontics and Dentofacial Orthopedics February 2014  Vol 145  Issue 2
194
Tables III and IV present Bland-Altman20 method
statistics, including mean differences, standard devia-
tions, 95% limits of agreement, and P value from paired
t tests. These results show wider limits (proportionally to
the measurements) for transversal width, cross-sectional
area, and minimum axial area slices.
Bland-Altman intraexaminer and interexaminer plots
for the minimum axial area are displayed in Figure 3. The
intraexaminer plot shows narrow limits for 2 examiners,
representing extremely good reliability, and wide, unac-
ceptable limits only for the orthodontist. The interexaminer
plot shows that in the comparisons of the measurements
from the orthodontist with the other 2 examiners, the dif-
ferences of 4 CBCT images were proportionally great, but
the differences between the measurements from the stu-
dent and the radiologist fell into a narrow interval.
Systematic errors were observed between examiners
in anteroposterior length, transversal width, and cross-
sectional area measurements at the level of the vallecula,
and in anteroposterior length at the level of the palatal
plane (Table IV).
DISCUSSION
Our findings indicate good intraexaminer and inter-
examiner reliabilities in upper airway anteroposterior
linear measurements, the sagittal area, and volumetric
measurements.
Airway volume is highly variable and can be influenced
by breathing stage and head posture,9 and might vary
among patients with different anteroposterior jaw rela-
tionships.21,22 Tso et al23 stated that both inspiration and
expiration can contribute to changes in airway volume. A
limitation of this study was that the respiratory phase could
not be controlled because the scanning time for the CBCT
image was 20 seconds. However, since our objective was
not to compare airway dimensions of different patients
or the dimensions of the same patient at different times,
this limitation probably had no effect on the results.
Head posture and tongue position were standardized.
Dolphin Imaging has been described as a software
that enables accurate airway volume measurements24,25
with few errors (1%).25 Among its advantages, Weiss-
heimer et al25 emphasized its friendly interface, quick
upper airway segmentation, good segmentation sensi-
tivity, possibility of checking segmentation in 2-
dimensional slices, and minimum cross-sectional area
analysis. However, the disadvantages include its cost,
lack of tools to correct or adjust airway segmentation
in 2-dimensional slices, and incompatibility of its
threshold interval units with other imaging softwares.25
Most studies published assessing the reliability of
airway measurements did not use an adequate statistical
February 2014  Vol 145  Issue 2 American Journal of Orthodontics and Dentofacial Orthopedics
Mattos et al
tool or, when they did, presented only ICC values.16,26
However, even high ICC values (.0.9) can hide
clinically important measurement errors.18 For that
reason, reliability was assessed in this study by ICC, mea-
surement errors, and the Bland-Altman20 method.
Our results show excellent intraexaminer and interexa-
miner (ICC .0.95) reliabilities in the anteroposterior
lengths at all levels. Measurement errors in this aspect
were also low: all mean differences were below 0.53
mm. These were probably the less-challenging measure-
ments in this study, since they are similar to measurements
in cephalometric analyses.27-29 Additionally, the sagittal
slice for this measurement was already established and
did not have to be chosen by the examiners; this
eliminated 1 step in the method and, most likely, 1
possibility of incorporating errors. Even though the
length at the levels of the soft palate and tongue needed
keener judgment so that the most constricted region
was really measured, the reliability was good. The
Bland-Altman20 results agree with this assertion.
Transversal width, on the contrary, showed moderate
intraexaminer (ICC .0.63) and interexaminer
(ICC .0.66) reliabilities, especially at the levels of the
tongue and vallecula, and the greatest measurement er-
rors proportionally (mean difference as high as 3.52 mm
and maximum difference as high as 15.98 mm), reaching
up to 42.2% of the maximum measurement. The 95%
limits of agreement of the Bland-Altman20 plots corrob-
orate these findings. This result might be due to the dif-
ficulty in selecting a specific coronal slice in the upper
airway region to measure transversal width, since few
anatomic references can be used in this sense. El
et al30 made transversal measurements in the coronal
plane similarly to those in this study, but they did not
specify which parameters they used to select this slice.
A good alternative to obtain the airway transversal width
at different levels might be to measure it directly in the
cross-sectional axial slices, as other authors did.11,31,32
Nonetheless, the reliability of these measurements
would then have to be tested.
Cross-sectional areas had excellent intraexaminer
and interexaminer (ICC .0.93) reliabilities, except at
the level of the vallecula, where the interexaminer coef-
ficient was only moderate (ICC 5 0.664). Measurement
errors were also proportionally greater at the vallecula
level (interexaminer mean difference of 31.15 mm2
and maximum difference of 112.14 mm2), reaching up
to 31.1% of the maximum measurement. These findings
might be due to the specific anatomy at this region,
where the interference of the epiglottis in the airway
can cause more measurement errors. The systematic er-
rors detected mostly at the level of the vallecula might
add to the conclusion that measurement in this region
Mattos et al 195

Table IV. Bland-Altman means, standard deviations, 95% limits of agreement (LoA) for interexaminer differences,
and paired t test P values to detect systematic errors
Student and orthodontist Student and radiologist Orthodontist and radiologist
t test P t test P t test P
Mean (SD) 95% LoA value Mean (SD) 95% LoA value Mean (SD) 95% LoA value
Anteroposterior length (mm) at the level of
Palatal plane 0.49 (0.55) 0.59; 1.57 0.011 0.03 (0.53) 1.02; 1.09 0.814 0.45 (0.45) 1.34; 0.43 0.005*
Soft palate 0.04 (0.35) 0.65; 0.74 0.664 0.10 (0.28) 0.46; 0.67 0.239 0.08 (0.43) 0.76; 0.94 0.522
Tongue 0.00 (0.59) 1.17; 1.16 0.166 0.04 (0.42) 0.78; 0.86 0.760 0.13 (0.33) 0.78; 0.51 0.183
Vallecula 0.66 (0.21) 0.24; 1.07 0.000y 0.41 (0.40) 0.37; 1.19 0.004* 0.25 (0.49) 1.21; 0.71 0.106
Width (mm) at the level of
Palatal plane 0.83 (2.14) 3.36; 5.03 0.025 0.34 (0.73) 1.10; 1.78 0.155 0.68 (1.25) 3.15; 1.77 0.100
Soft palate 1.26 (3.86) 8.82; 6.30 0.303 0.81 (3.78) 8.24; 6.61 0.943 1.31 (3.40) 5.37; 7.99 0.231
Tongue 1.26 (5.96) 12.95; 10.42 0.819 0.53 (4.88) 10.11; 9.03 0.181 1.72 (3.11) 7.82; 4.37 0.095
Vallecula 4.38 (3.91) 3.28; 12.05 0.010 0.26 (2.01) 4.22; 3.68 0.668 3.68 (3.77) 11.08; 3.71 0.009*
Cross-sectional area (axial slice) (mm2) at the level of
Palatal plane 1.23 (28.52) 57.14; 54.67 0.884 12.17 (25.95) 38.70; 63.05 0.212 9.78 (12.54) 14.80; 34.36 0.027
Soft palate 4.29 (14.97) 25.06; 33.64 0.364 13.94 (19.88) 25.02; 52.91 0.033 16.12 (31.67) 45.96; 78.22 0.213
Tongue 2.80 (25.61) 53.01; 47.39 0.711 9.99 (24.16) 37.36; 57.35 0.361 4.71 (18.09) 30.74; 40.17 0.408
Vallecula 26.59 (51.84) 75.02; 128.20 0.244 22.52 (16.13) 9.10; 54.15 0.001* 1.82 (42.36) 81.21; 84.87 0.889
Sagittal area 1.31 (33.60) 67.18; 64.55 0.894 1.55 (36.04) 69.09; 72.20 0.889 1.44 (22.06) 41.79; 44.68 0.832
(mm2)
Minimum axial 16.03 (24.78) 32.55; 64.62 0.057 0.07 (0.73) 1.51; 1.36 0.750 20.95 (28.92) 77.64; 35.74 0.056
area (mm2)
Volume (mm3) 194 (895.06) 1560.05; 0.468 94.74 (878.97) 1628.04; 0.728 130.05 (447.53) 1007.21; 0.358
1948.62 1817.53 747.10

*P \0.01; yP \0.001.

Fig 3. Bland-Altman20 plots for minimum axial area measurements. Dotted lines represent 95% limits
of agreement. A, Intraexaminer comparisons: black circles and lines, student; blue circles and lines,
orthodontist; red circles and lines, radiologist. B, Interexaminer comparisons: green circles and lines,
student and orthodontist; purple circles and lines, student and radiologist; orange circles and lines,
orthodontist and radiologist.

is more complex. Cross-sectional areas, including the
minimum axial area, are important parameters in airway
assessment because they have a direct implication in the
airflow, since they are perpendicular to it.11
Sagittal areas had ICC values all above 0.97, which in-
dicates that the limits determined for volume assessment
and minimum axial area were similar among examiners
and for each examiner. Measurement errors and
Bland-Altman20 results also indicate good intraexaminer
and interexaminer agreement.
The minimum axial area had reasonably good reli-
ability according to the ICC values (.0.86 and 0.93 for in-
traexaminer and interexaminer coefficients, respectively).
However, it also had wide 95% limits of agreement for

American Journal of Orthodontics and Dentofacial Orthopedics February 2014  Vol 145  Issue 2
196
some comparisons, reaching limits of agreement as low as
77.64 mm2 and as high as 65.25 mm2. Souza et al33
found 95% limits of agreement for the interexaminer
comparison of the minimum axial area of 27.49 to
29.69 and an interexaminer ICC value of 0.98; they
considered the reliability appropriate. The assessment of
the Bland-Altman20 plot for the intraexaminer and inter-
examiner comparison of this measurement contributes
to the understanding of the results (Fig 3). One examiner
showed great differences between the measurements at
the 2 times, and the comparisons of her measurements
with those of the other 2 examiners had great differences
also. The definition of the minimum axial area in the Dol-
phin Imaging software can be tricky, and discussion of this
tool is important because it might lead to errors. We have
observed that after defining the limits for measuring the
airway (Fig 2, A) in the software's airway/sinus tool, it is
possible to move the boundaries in which the minimum
axial area will be determined. That is easily visualized by
the red dotted lines in Figure 2, B. If the red lines are
moved to coincide with the upper and lower limits previ-
ously determined (green lines), the minimum axial area
frequently appears coincident with the lower limit, reach-
ing in reality some areas outside the limits and underesti-
mating the minimum axial area. When the red lines are
moved to a position immediately below the upper limit
and immediately above the lower limit—within the limits
and not coincident with them—the actual minimum axial
area is selected. The examiners in this study were oriented
to position the red lines within the limits, as mentioned
before. The failure to follow this instruction might have
contributed to some discrepancies in the results. There-
fore, this tool should be used with extreme attention.
Volume measurements showed the greatest coeffi-
cients (.0.98) and low measurement errors proportionally,
indicating excellent intraexaminer and interexaminer reli-
abilities. Souza et al33 found similar ICC values and higher
95% limits of agreement for volumetric measurements
(3390.36-2777.53, interexaminer) and considered the
reliability excellent.
Ghoneima and Kula10 reported good accuracy and
reliability for airway volume analysis when comparing
CBCT measurements and manual measurements of an
airway model. El and Palomo9 showed high reliability
and poor accuracy in airway volume calculations when
3 commercially available DICOM viewers were compared.
Oliveira et al1 and Cevidanes et al2 introduced the pro-
tocol we used, which included an undergraduate dental
student, an orthodontist, and a dental radiologist to assess
intraexaminer and interexaminer reliabilities with ICC
values in CBCT measurements. Good reliability among 3
examiners with different backgrounds can be attributed
to their careful calibration.1,2 These authors reported
February 2014  Vol 145  Issue 2 American Journal of Orthodontics and Dentofacial Orthopedics
Mattos et al
ICC values above 0.9 for 86%,1 90%,2 and 72%2 of the in-
traexaminer assessments, and 66%1 and 74%2 of the in-
terexaminer assessments. These percentages were similar
to those in this study. However, although they reported
good reliability, their results cannot be compared with
our findings because they did not analyze the upper
airway. Those authors focused on 3D landmark identifica-
tion1 and head orientation in CBCT-generated cephalo-
grams.2 Additionally, inferences concerning the 3 classes
of examiners are not the determinant in our study, since
just 1 person was used to represent each category.
Measurement errors above 10% of the average mea-
surement might indicate an increasing lack of agreement
as this percentage rises. Our results discourage the use of
linear transversal measurements in coronal-slice and
cross-sectional area measurements at the level of the
vallecula. The minimum axial area, which is an important
parameter in diagnosis and treatment follow-up, is also
subject to measurement errors and must be used with
special attention. Our results also indicate that airway
measurements should be performed carefully and
checked to prevent measurement errors.
Divergences in diagnostic measurements can affect
treatment planning and follow-up and thus can be clin-
ically significant. The comparison of the same subject's
airway measurements at 2 or more times, or the compar-
ison of different subjects' airway measurements, should
be reliable when they are assessed by the same calibrated
examiner. Nonetheless, airway measurements made by
different examiners might be less reliable, especially if
these examiners were not calibrated in the same way.
One limitation of this study was the sample size, which
was calculated according to the ICC. When we calculated
the precision of the limits of agreement, however, only the
anteroposterior linear measurements and the transversal
width at the level of the soft palate had adequate preci-
sion.20 As in the example of Bland and Altman,20 the
considerable discrepancies in some measurements in our
study lead to the conclusion that the degree of agreement
is not acceptable, but measurements considered reliable
(cross-sectional areas at the levels of the palatal plane,
soft palate, and tongue; sagittal area and volume) should
be retested in studies with larger sample sizes.
Future research should investigate intraexaminer and
interexaminer reliabilities for airway transversal widths
measured in cross-sectional axial slices. Further studies
assessing the reliability of airway measurements by using
more than 1 statistical approach are also encouraged.
CONCLUSIONS
Based on the results of this study, airway assessments
by examiners with different backgrounds might be reliable
for anteroposterior linear measurements; cross-sectional
Mattos et al
areas at the levels of the palatal plane, soft palate, and
tongue; and sagittal area (limits for volume assessment)
and volume. Airway assessments might be unreliable for
linear width measurements, cross-sectional area at the
level of the vallecula, and minimum axial area.
SUPPLEMENTARY DATA
Supplementary data associated with this article can
be found, in the online version, at http://dx.doi.org/
10.1016/j.ajodo.2013.10.013.
REFERENCES
1. Oliveira AEF, Cevidanes LHS, Philips C, Motta A, Burke B, Tyndall D.
Observer reliability of three-dimensional cephalometric landmark
identification on cone-beam computerized tomography. Oral
Surg Oral Med Oral Pathol Oral Radiol Endod 2009;107:256-65.
2. Cevidanes L, Oliveira AEF, Motta A, Phillips C, Burke B, Tyndall D.
Head orientation in CBCT-generated cephalograms. Angle Orthod
2009;79:971-7.
3. Lagravere MO, Low C, Flores-Mir C, Chung R, Carey JP, Heo G,
et al. Intraexaminer and interexaminer reliabilities of landmark
identification on digitized lateral cephalograms and formatted
3-dimensional cone-beam computerized tomography images.
Am J Orthod Dentofacial Orthop 2010;137:598-604.
4. De Vos W, Casselman J, Swennen GRJ. Cone-beam computerized
tomography (CBCT) imaging of the oral and maxillofacial region: a
systematic review of the literature. Int J Oral Maxillofac Surg 2009;
38:609-25.
5. Berco M, Rigall PH Jr, Miner RM, DeLuca S, Anderson NK, Will LA.
Accuracy and reliability of linear cephalometric measurements
from cone-beam computed tomography scans of a dry human
skull. Am J Orthod Dentofacial Orthop 2009;136:17.e1-9.
6. Cevidanes LHS, Styner MA, Proffit WR. Image analysis and super-
imposition of 3-dimensional cone-beam computed tomography
models. Am J Orthod Dentofacial Orthop 2006;129:611-8.
7. Lagravere MO, Gordon JM, Guedes IH, Flores-Mir C, Carey JP,
Heo G, et al. Reliability of traditional cephalometric landmarks as
seen in three-dimensional analysis in maxillary expansion treat-
ments. Angle Orthod 2009;79:1047-56.
8. Aboudara CA, Hatcher D, Nielsen IL, Miller A. A three-dimensional
evaluation of the upper airway in adolescents. Orthod Craniofac
Res 2003;6(Suppl 1):173-5.
9. El H, Palomo JM. Measuring the airway in 3 dimensions: a reli-
ability and accuracy study. Am J Orthod Dentofacial Orthop
2010;137(Suppl):S50.e1-9.
10. Ghoneima A, Kula K. Accuracy and reliability of cone-beam
computed tomography for airway volume analysis. Eur J Orthod
2013;35:256-61.
11. Abramson ZR, Susarla S, Tagoni JR, Kaban L. Three-dimensional
computed tomographic analysis of airway anatomy. J Oral Maxil-
lofac Surg 2010;68:363-71.
12. Sears CR, Miller AJ, Chang MK, Huang JC, Lee JS. Comparison of
pharyngeal airway changes on plain radiography and cone-beam
computed tomography after orthognathic surgery. J Oral Maxillo-
fac Surg 2011;69:e385-94.
13. Vizzotto MB, Liedke GS, Delamare EL, Sileira HD, Dutra V,
Silveira HE. A comparative study of lateral cephalograms and
cone-beam computed tomographic images in upper airway assess-
ment. Eur J Orthod 2012;34:390-3.
American Journal of Orthodontics and Dentofacial Orthopedics
197
14. Aboudara C, Nielsen I, Huang JC, Maki K, Miller AJ, Hatcher D.
Comparison of airway space with conventional lateral headfilms
and 3-dimensional reconstruction from cone-beam computed
tomography. Am J Orthod Dentofacial Orthop 2009;135:468-79.
15. Guijarro-Martınez R, Swennen GRJ. Cone-beam computerized
tomography imaging and analysis of the upper airway: a systematic
review of the literature. Int J Oral Maxillofac Surg 2011;40:1227-37.
16. Alsufyani N, Flores-Mir C, Major P. Three-dimensional segmenta-
tion of the upper airway using cone beam CT: a systematic review.
Dentomaxillofac Radiol 2012;41:276-84.
17. Lenza MG, Lenza MMO, Dalstra M, Melsen B, Cattaneo PM. An
analysis of different approaches to the assessment of upper airway
morphology: a CBCT study. Orthod Craniofac Res 2010;13:96-105.
18. Luiz RR, Szklo M. More than one statistical strategy to assess
agreement of quantitative measurements may usefully be
reported. J Clin Epidemiol 2005;58:215-6.
19. Walter SD, Eliasziw M, Donner A. Sample size and optimal designs
for reliability studies. Stat Med 1998;17:101-10.
20. Bland JM, Altman DG. Statistical methods for assessing agreement be-
tween two methods of clinical measurement. Lancet 1986;1:307-10.
21. Grauer D, Cevidanes LS, Styner MA, Ackerman JL, Proffit WR.
Pharyngeal airway volume and shape from cone-beam computed
tomography: relationship to facial morphology. Am J Orthod
Dentofacial Orthop 2009;136:805-14.
22. Claudino LV, Mattos CT, Ruellas ACO, Sant'Anna EF. Pharyngeal
airway characterization in adolescents related to facial skeletal
pattern: a preliminary study. Am J Orthod Dentofacial Orthop
2013;143:799-809.
23. Tso HH, Lee JS, Huang JC, Maki K, Hatcher D, Miller AJ. Evaluation of
the human airway using cone-beam computerized tomography. Oral
Surg Oral Med Oral Pathol Oral Radiol Endod 2009;108:768-76.
24. Alves M Jr, Baratieri C, Mattos CT, Brunetto D, Fontes Rda C,
Santos JR, et al. Is the airway volume being correctly analyzed?
Am J Orthod Dentofacial Orthop 2012;141:657-61.
25. Weissheimer A, Menezes LM, Sameshima GT, Enciso R, Pham J,
Grauer D. Imaging software accuracy for 3-dimensional analysis of
the upper airway. Am J Orthod Dentofacial Orthop 2012;142:801-13.
26. Donatelli RE, Lee S. How to report reliability in orthodontic research:
part 1. Am J Orthod Dentofacial Orthop 2013;144:156-61.
27. Kaygisiz E, Tuncer BB, Y€ uksel S, Tuncer C, Yildiz C. Effects of
maxillary protraction and fixed appliance therapy on the pharyn-
geal airway. Angle Orthod 2009;79:660-7.
28. Poon KH, Chay SH, Chiong KFW. Airway and craniofacial changes
with mandibular advancement device in Chinese with obstructive
sleep apnoea. Ann Acad Med Singap 2008;37:637-44.
29. Sheng C, Lin L, Su Y, Tsai H. Developmental changes in pharyngeal
airway depth and hyoid bone position from childhood to young
adulthood. Angle Orthod 2009;79:484-90.
30. El AS, El H, Palomo JM, Baur DA. A 3-dimensional airway analysis
of an obstructive sleep apnea surgical correction with cone beam
computed tomography. J Oral Maxillofac Surg 2011;69:2424-36.
31. Abramson Z, Susarla SM, Lawler M, Bouchard C, Troulis M,
Kaban LB. Three-dimensional computed tomographic airway anal-
ysis of patients with obstructive sleep apnea treated by maxillo-
mandibular advancement. J Oral Maxillofac Surg 2011;69:677-86.
32. Degerliyurt K, Ueki K, Hashiba Y, Marukawa K, Nakagawa K,
Yamamoto E. A comparative CT evaluation of pharyngeal airway
changes in Class III patients receiving bimaxillary surgery or
mandibular setback surgery. Oral Surg Oral Med Oral Pathol Oral
Radiol Endod 2008;105:495-502.
33. Souza KR, Oltramari-Navarro PV, Navarro Rde L, Conti AC,
Almeida MR. Reliability of a method to conduct upper airway analysis
in cone-beam computed tomography. Braz Oral Res 2013;27:48-54.
February 2014  Vol 145  Issue 2