Diagnostic Performance of Magnetic

Resonance Imaging and Computed

Tomography for Advanced Retinoblastoma
A Systematic Review and Meta-analysis
Marcus C. de Jong, MD, MSc,1 Pim de Graaf, MD, PhD,1 Daniel P. Noij, BSc,1 Sophia Gricke, MD,2
Philippe Maeder, MD,3 Paolo Galluzzi, MD,4 Herv J. Brisse, MD, PhD,5 Annette C. Moll, MD, PhD,6
Jonas A. Castelijns, MD, PhD,1 on behalf of the European Retinoblastoma Imaging Collaboration (ERIC)*
Purpose: To determine and compare the diagnostic performance of magnetic resonance imaging (MRI) and
computed tomography (CT) for the diagnosis of tumor extent in advanced retinoblastoma, using histopathologic
analysis as the reference standard.
Design: Systematic review and meta-analysis.
Participants: Patients with advanced retinoblastoma who underwent MRI, CT, or both for the detection of
tumor extent from published diagnostic accuracy studies.
Methods: Medline and Embase were searched for literature published through April 2013 assessing the
diagnostic performance of MRI, CT, or both in detecting intraorbital and extraorbital tumor extension of retinoblastoma. Diagnostic accuracy data were extracted from included studies. Summary estimates were based on a
random effects model. Intrastudy and interstudy heterogeneity were analyzed.
Main Outcome Measures: Sensitivity and specicity of MRI and CT in detecting tumor extent.
Results: Data of the following tumor-extent parameters were extracted: anterior eye segment involvement
and ciliary body, optic nerve, choroidal, and (extra)scleral invasion. Articles on MRI reported results of 591 eyes
from 14 studies, and articles on CT yielded 257 eyes from 4 studies. The summary estimates with their 95%
condence intervals (CIs) of the diagnostic accuracy of conventional MRI at detecting postlaminar optic nerve,
choroidal, and scleral invasion showed sensitivities of 59% (95% CI, 37%e78%), 74% (95% CI, 52%e88%), and
88% (95% CI, 20%e100%), respectively, and specicities of 94% (95% CI, 84%e98%), 72% (95% CI, 31%e
94%), and 99% (95% CI, 86%e100%), respectively. Magnetic resonance imaging with a high (versus a low)
image quality showed higher diagnostic accuracies for detection of prelaminar optic nerve and choroidal invasion,
but these differences were not statistically signicant. Studies reporting the diagnostic accuracy of CT did not
provide enough data to perform any meta-analyses.
Conclusions: Magnetic resonance imaging is an important diagnostic tool for the detection of local tumor extent
in advanced retinoblastoma, although its diagnostic accuracy shows room for improvement, especially with regard to
sensitivity. With only a fewdmostly olddstudies, there is very little evidence on the diagnostic accuracy of CT, and
generally these studies show low diagnostic accuracy. Future studies assessing the role of MRI in clinical decision
making in terms of prognostic value for advanced retinoblastoma are needed. Ophthalmology 2014;121:11091118 2014 by the American Academy of Ophthalmology.
*Supplemental material is available at www.aaojournal.org.

Retinoblastoma is the most frequent malignant ocular tumor

in children, typically presenting in the rst years of life. It
represents approximately 3% of all pediatric malignancies,
with an incidence of 1:17 000.1 Retinoblastoma can be
diagnosed accurately by funduscopy and ultrasound, which
typically demonstrates an intraocular vascularized and
calcied mass. Cross-sectional imaging is used primarily
for local tumor staging (related to metastatic risk) and
depiction of associated intracranial primitive neuroectodermal
 2014 by the American Academy of Ophthalmology
Published by Elsevier Inc.

tumors (mostly pineoblastoma).2,3 Magnetic resonance imaging (MRI) also can aid retinoblastoma diagnosis in case of
an unclear ocular medium. Treatment strategies are focused
on survival, preserving vision, and nally on avoiding
enucleation. Although the gold standard for diagnosis and
local staging relies on pathologic analysis, the treatment
strategy frequently is based on clinical and radiologic ndings only, notably in eyes treated conservatively. In patients
treated by primary enucleation, extraocular extension rst

ISSN 0161-6420/14/$ - see front matter

http://dx.doi.org/10.1016/j.ophtha.2013.11.021

1109

Ophthalmology Volume 121, Number 5, May 2014

must be ruled out by imaging to avoid leaving behind tumor
tissue after resection. Therefore, accurate local assessment
based on imaging is critical at diagnosis. As recommended
currently, we perform MRI in all newly diagnosed retinoblastoma patients, but in the past, MRI often was performed
as a second diagnostic step; we are not sure about the policies
in other institutions.4,5
The most important role of MRI is to aid in the decision
either to treat an eye conservatively or to enucleate the eye.
The prediction of high-risk features of retinoblastoma based
on clinical and radiologic features is pivotal in selecting the
best treatment option.6 Important risk factors for local
recurrence and metastasis are massive choroidal invasion,
scleral invasion, optic nerve invasion (ONI) posterior to
the lamina cribrosa (especially if the surgical resection
margin of the optic nerve is invaded), and involvement of
the anterior eye segment (AES).6e12
Several studies of the diagnostic accuracy of MRI and
only a few of computed tomography (CT) for various
tumor-extent parameters have been published. Computed
tomography has been considered important in detecting
calcications, but it recently was demonstrated that the
combination of ultrasound and MRI has the same sensitivity
and specicity compared with CT in detecting intratumoral
calcications in retinoblastoma.13 Moreover, whenever
possible CT should be avoided in young children because
it poses a signicant radiation risk,14e17 and even more so
in patients with hereditary retinoblastoma.4,13 However,
because the incidence of retinoblastoma is low, study populations generally are small. There is also considerable
heterogeneity (e.g., in terms of image quality) among the
studies. To overcome these problems, a critical systematic
review of the different studies is required.
The purpose of this study was to provide a complete
overview of available evidence and to determine and
compare the diagnostic performance of MRI and CT for the
detection of tumor extent in advanced retinoblastoma patients, with histopathologic analysis as the reference
standard.

Methods
We performed this study according to the Preferred Reporting
Items for Systematic Reviews and Meta-Analyses (PRISMA)
statement.18,19

Search Strategy
We searched Medline (PubMed) and Embase for English, Dutch,
German, and Spanish literature published through April 2013
evaluating tumor extent of retinoblastoma by MRI or CT. We also
included studies found in alternative ways (e.g., through checking
references in included studies). When necessary, we contacted
authors for additional data. The search included keywords corresponding to the index test MRI and CT, the target condition
(retinoblastoma), and diagnostic performance (Appendix A,
available at www.aaojournal.org).

Study Selection
Article titles and abstracts were reviewed independently for eligibility by 2 authors (M.C.J. and D.P.N.), and discrepancies were

1110

resolved by consensus. Studies were included if they met all of the

following criteria: (1) the study population consisted of retinoblastoma patients; (2) the study assessed diagnostic performance of
MRI, CT, or both as a diagnostic test for tumor invasion into the
ocular wall, ONI, or AES involvement; (3) histopathologic analysis
was used as the reference standard test; and (4) if at least one pair
of the absolute numbers of true-positive results and false-negative
results, or true-negative results and false-positive results were
available or could be derived adequately. To include true-positive
results, false-positive results, true-negative results, and falsenegative results in a meta-analysis, all 4 should be available.
Studies were excluded if they met one of the following criteria: (1)
the article was a review or meta-analysis, and (2) (potentially)
overlapping study populations were reported for the same outcome.
Diagnostic accuracy of tumor extension into the optic nerve can
be assessed in different ways. To avoid unclear denitions, in this
study we considered 3 categories: (1) if the optic nerve disc has
been invaded by tumor tissue (henceforth referred to as prelaminar
ONI), (2) invasion exactly into the lamina cribrosa (henceforth
referred to as intralaminar ONI), and (3) invasion posterior to the
lamina cribrosa (henceforth referred to as postlaminar ONI).

Data Extraction
Two authors (M.C.J. and D.P.N.) independently extracted study
data. Discrepancies were resolved by consensus. When studies
reported multiple sets of sensitivities and specicities from multiple readers separately, the set with the highest diagnostic odds ratio
(DOR) was used for the gures and meta-analysis. We did this to
prevent data from the same patient population from being used
twice. This could have resulted in an overestimation of the summary estimates; therefore, we also reported the overall summary
estimates including sensitivity and specicity from the reader with
the lowest DOR.

Risk of Bias Assessment

Ideally, studies included only eyes that were primarily enucleated
(i.e., retinoblastoma that was not previously treated) because retinoblastoma treatment can inuence the appearance of tumor-extent
parameters on MRI scans. However, some authors included both
secondary and primary enucleations or did not mention this at all in
their article. We performed sensitivity analyses by assessing the
effect of analyzing the diagnostic performance of studies that
explicitly state that they included only eyes that were primary
enucleations.
In an empirical study of bias in diagnostic tests, Lijmer et al20
demonstrated that study design can be very important to the results
of diagnostic tests; they showed that a case-control design caused
an overestimation of the DOR by 3 times, the use of different
reference tests overestimated the DOR twice, and not blinding the
test interpretation overestimated the DOR by 30%. However,
verication bias, a nonconsecutive or random patient selection, and
a retrospective study design did not seem to affect the DOR.
We used the Quality Assessment of Studies of Diagnostic Accuracy Included in Systematic Reviews (QUADAS-2) checklist to
assess the study quality in terms of the risk of bias and the applicability of included studies.21,22 Two authors (M.C.J. and D.P.N.)
independently assessed the study quality of the included articles.
Two authors (P.d.G. and H.J.B.) with 11 and 16 years experience, respectively, in ocular MRI independently assessed the
image quality (low, intermediate, or high) of MRI and CT scans
provided in the articles, resulting in semiquantitative quality scores.
The guidelines for imaging retinoblastoma by De Graaf et al4
served as a checklist for the quality assessment. They
recommended imaging of the eyes with a slice thickness of at

De Jong et al

Diagnostic Performance of Retinoblastoma Imaging

most 2 mm and a pixel size that is no larger than 0.5
0.5 mm. To

achieve this, it is advisable to use either a multichannel head coil
(3.0-T system) or surface coils (1.5-T system). If information on
slice thickness, pixel size, or both was not mentioned in the published articles, the included images served as a qualitative measure
for image quality. Proper sedation also is important to ensure a
high image quality in the usually very young retinoblastoma patient group. Discrepancies between image quality scores were
resolved by consensus.

Data Synthesis and Statistical Analysis

If tumor-extent parameters were analyzed in enough studies to
allow statistical analysis, we analyzed data using a bivariate
random effects regression model and provided summary estimates.23 This model assumes a binomial distribution of the withinstudy variability (i.e., the variability between sensitivity and
specicity within a study). The model assumes correlated normally
distributed random effects between studies. The inverse relation
between sensitivity and specicitydwhen the positivity criterion is
varieddcorresponds to the degree of correlation between the logit
sensitivity and logit specicity. We performed metaregression to
explore the effect of image quality on the diagnostic accuracy of
tumor-extent parameters if sample sizes allowed this.
We summarized the data of each study and overall estimates in
forest plots with a 95% condence interval (CI) of sensitivity and

specicity for each tumor-extent parameter. We also plotted these

numbers in receiver operator characteristic (ROC) spaces showing
the summary estimates with a 95% condence region. Potential
outliers have been identied in these ROC plots on the basis of the
position of individual studies relative to the other studies. We
performed sensitivity analyses by excluding these potential outliers, but these results should be interpreted with care.
We calculated the DOR along with sensitivity and specicity as
an overall measure of diagnostic performance. The advantage of
DOR is its independence from disease prevalence and the
approximately normal distribution of the natural logarithm of
DOR.24 As a general rule, diagnostic tests with a DOR of more
than 25 are considered moderately accurate, and tests with a
DOR of more than 100 are considered highly accurate.25,26
For the meta-analysis, we used the statistical software package
SAS version 9.3 (Proc NLMIXED; SAS Inc, Cary, NC). We
created the forest plots with Photoshop CS6 (Adobe, San Jose,
CA). We used Cochranes Review Manager version 5.2 (Copenhagen, Denmark) to create the ROC plots.

Results
Medline and Embase searches yielded 426 unique studies. We
excluded 376 articles based on title and abstract (Fig 1). We
excluded 32 studies based on the full text; see Figure 1 for

Figure 1. Flowchart showing systematic literature search. CT computed tomography; FN false-negative results; FP false-positive results; MRI
magnetic resonance imaging; TN true-negative results; TP true-positive results.

1111

Ophthalmology Volume 121, Number 5, May 2014

reasons for exclusion. The study characteristics show considerable
differences between the included studies (Table 1, available at
www.aaojournal.org). Eighteen studies met the inclusion criteria
for qualitative synthesis, and 13 studies were included in the
meta-analysis. The study population described by Schueler
et al27 overlapped with the more recent study by Lemke et al28;
therefore, we extracted data only from the study by Schueler
et al for tumor-extent parameters not included in the study by
Lemke et al (i.e., scleral invasion). All analyses were performed on
a per-eye basis and not a per-patient basis because some studies
included 2 eyes from 1 patient (Table 1, available at www.
aaojournal.org). Articles on MRI reported results of 591 eyes
from 14 studies (excluding Schueler et al), and articles on CT
yielded results of 257 eyes from 4 studies. The articles reported
ages at diagnosis of retinoblastoma that ranged from a mean of
11 months to a mean of 32 months. To avoid very large tables,
we split the tumor-extent parameters into 3 groups: ONI, ocular
wall invasion, and AES involvement and ciliary body invasion
(Tables 2e4, available at www.aaojournal.org). Data on the
diagnostic accuracy of MRI at detecting ONI, choroidal invasion,
and scleral invasion proved to be sufcient for meta-analysis.
The sensitivities and specicities of tumor-extent parameters are
presented in Tables 2 through 4 (available at www.aaojournal.org).
Forest plots show the sensitivities and specicities of each tumorextent parameter, with their 95% CIs as horizontal lines sorted
by sensitivity (Fig 2, available at www.aaojournal.org).

Summary Estimates
Compared with histopathologic analysis, the meta-analysis of MRI
for the different tumor-extent parameters gave us a sensitivity of
73% (95% CI, 45%e90%), a specicity of 96% (95% CI, 64%e
100%), and a DOR of 59.7 (95% CI, 4.36e818) for prelaminar
ONI from 8 studies (Table 5; Fig 2, available at www.
aaojournal.org). For intralaminar ONI, MRI showed a sensitivity
of 38% (95% CI, 0%e100%), a specicity of 89% (95% CI,
47%e99%), and a DOR of 4.84 (95% CI, 0.00e3.14
104) from
4 studies. For postlaminar ONI, MRI demonstrated a sensitivity
of 59% (95% CI, 37%e78%), a specicity of 94% (95% CI,
84%e98%), and a DOR of 20.9 (95% CI, 4.75e92.2) from 10
studies. For choroidal invasion, MRI showed a sensitivity of
74% (95% CI, 52%e88%), a specicity of 72% (95% CI, 31%e
94%), and a DOR of 7.38 (95% CI, 1.21e45.0) from 6 studies.
For scleral invasion, MRI demonstrated a sensitivity of 88%
(95% CI, 20%e100%), a specicity of 99% (95% CI, 86%e
100%), and a DOR of 503 (95% CI, 24.9e1.02
104) from 6
studies. The ROC plots show the summary estimates of
specicity on the x-axis and sensitivity on the y-axis for these 5
tumor-extent parameters with their respective 95% condence
areas (Fig 3).

Sensitivity Analysis and Metaregression

Based on the ROC plots, we identied 2 studies as potential outliers: the study by Wilson et al29 for postlaminar ONI and the study
by Khurana et al30 for choroidal invasion (Fig 3). After exclusion
of these potential outliers, the diagnostic accuracy of MRI for
postlaminar ONI showed a sensitivity of 61% (95% CI, 32%e
84%), a specicity of 96% (95% CI, 80%e99%), and a DOR of
34.6 (95% CI, 5.80e206). For choroidal invasion, MRI showed
a sensitivity of 77% (95% CI, 41%e94%), a specicity of 78%
(95% CI, 46%e94%), and a DOR of 11.8 (95% CI, 2.14e65.4).
We performed a sensitivity analysis of including data from
certain primary enucleations only. Four of 8 studies for prelaminar
ONI, 3 of 4 studies for intralaminar ONI, 7 of 10 studies for
postlaminar ONI, 4 of 6 studies for choroidal invasion, and 2 of 6

1112

studies for scleral invasion explicitly reported data based on primary enucleations only. Results based on this subset gave lower
DORs for most tumor-extent paramaters; only the DOR of
choroidal invasion did not show much of a difference, and for
scleral invasion, we could not calculate summary estimates
(Table 5).
We also assessed the effect of including data from the readers of
the MRI scans with the lowest DORdfrom studies that report
multiple readersdon the summary estimates. Three of 8 studies
assessing prelaminar ONI, 1 of 4 studies assessing intralaminar
ONI, and 2 of 10 studies assessing postlaminar ONI reported 2 sets
of sensitivity and specicity from 2 readers. When data from the
worst reader (i.e., with the lowest DOR) were included in the
analysis, the results did not change much (Table 5).
For prelaminar ONI and choroidal invasion, the results from
metaregression analysis showed higher DORs for studies with a
higher image quality. The results for postlaminar ONI showed a
higher DOR only after removal of a potential outlier. However,
none of these differences were statistically signicant (Table 6;
Fig 2, available at www.aaojournal.org).

Risk of Bias Assessment

See Figures 4 and 5 for the QUADAS summary scores regarding
risk of bias and concern of applicability of the included studies.
The entire list of QUADAS scores for each study is available in
the electronic supplement (Appendix B, available at www.
aaojournal.org).
None of the included studies had a case-control design. All
studies had the same reference standard because histopathologic
analysis was one of the inclusion criteria of this systematic review.
The MRI interpretation was blinded in 10 studies, but this was
unclear in the other 8 studies. Histopathologic interpretation was
blinded in 8 studies, whereas this was unclear in 10 studies. We
scored 7 of 18 studies as having an appropriate interval between
the MRI and enucleation, and we scored the other 11 studies as
unclear because they either did not report an interval at all or
because they reported relatively wide ranges (Table 1 and
Appendix B, available at www.aaojournal.org).
Only the most severe retinoblastoma cases were included
because enucleation is necessary for histopathologic assessment,
but we do not know how this affects the applicability of the results
for patients with less severe retinoblastoma. Three studies raised
applicability concerns for the index test: Lee et al31 report using 4
different MRI scanners, Brisse et al32 reported using various
different MRI and CT scanners, and John-Mikolajewski et al33
reported using contrast enhancement in only 3 of 11 patients.
Ainbinder et al34 did not describe the reference test in their
study, leading to an unclear concern of the applicability (Fig 5).
Only 9 studies explicitly reported that patients were treated with
primary enucleation (i.e., without other retinoblastoma treatment
before enucleation), one study mentioned the use of preoperative
chemotherapy in some patients but did not stratify the results, 2
studies were unclear about this issue, and 6 studies did not report
on this at all (Table 1, available at www.aaojournal.org).

Discussion
We were able to perform a meta-analysis of the diagnostic
accuracy of MRI for detection of optic nerve, choroidal, and
scleral invasion in retinoblastoma. Study data of extrascleral
invasion, AES involvement, and ciliary body invasion
proved insufcient for meta-analysis. Studies reporting the

De Jong et al

Diagnostic Performance of Retinoblastoma Imaging

Figure 3. Receiver operator characteristic spaces with 95% condence areas. The sizes of the rectangles correlate with the size of the study.

diagnostic accuracy of CT did not provide enough data to

perform any meta-analyses.

Optic Nerve Invasion

Postlaminar ONI is considered to be one of the most
important risk factors for metastasis and has been studied
most extensively of all tumor-extent parameters included in
this review. The summary estimates of the diagnostic accuracy of MRI are far from perfect, especially in terms of
sensitivity (Table 5; Fig 2, available at www.
aaojournal.org). Increasing image quality seems to have a
positive effect on the diagnostic accuracy of MRI,
especially with regard to sensitivity (Table 6).
Prelaminar ONI is not considered to be a risk factor, but
we believe the higher diagnostic accuracy of MRI techniques with a higher image quality does show the potential
of high-resolution MRI (Table 6; Fig 2, available at
www.aaojournal.org).
Four studies also examined tumor extension exactly into
the lamina cribrosa. Because intralaminar tumor invasion
can develop into postlaminar tumor invasion in the future or
because a false-negative result actually could be histopathologically a postlaminar invasion, this is a category that

deserves attention. MRI shows a reasonable specicity in

detecting intralaminar optic nerve invasion, but there is a
wide range of reported sensitivities (Table 5; Fig 2, available
at www.aaojournal.org).
High-resolution imaging techniques are critical for
borderline postlaminar invasion. The relatively high number
of false-negative results could be caused by missing minimal postlaminar invasions.32 We have extracted the degree
of tumor involvement per study (unfortunately, this was
scarcely reported) for true-positive and false-negative results in millimeters posterior to the lamina cribrosa to
illustrate this problem (Table 2, available at www.
aaojournal.org). False-positive results, however, can be
caused by posterior bulging of the lamina cribrosa secondary to an increased intraocular pressure.30e32 Other causes
of false-positive results may be inammation or endothelial
proliferation, which can mimic (residual) tumor invasion of
the optic nerve.35,36
The sensitivities (0%e43%) of postlaminar optic nerve
invasion on CT are quite low, even in patients with extensive involvement (Table 2, available at www.
aaojournal.org).32 Jacquemin et al37 showed that
nonvisualization of central retinal vessels resulted in a

1113

Ophthalmology Volume 121, Number 5, May 2014

Table 5. Summary Estimates of Diagnostic Performance of Magnetic Resonance Imaging for Retinoblastoma Tumor Extent
No. of True and False Test
Results in a Sample of 1000
Patientsz
Tumor Extent
Prelaminar
Overall
Only primary enucleationsx
Worst readerk
Intralaminar
Overall
Only primary enucleationsx
Worst readerk
Postlaminar
Overall
Only primary enucleationsx
Worst readerk
Choroidal
Overall
Only primary enucleationsx
Scleral
Overall
Only primary enucleationsx

Sensitivity* Specicity* Diagnostic Odds Ratio* No. of Studies Disease Prevalencey

TP

FP

TN

FN

73 (45e90) 96 (64e100) 59.7 (4.36e818)

80 (31e79) 81 (3e100) 17.7 (0.34e914)
70 (43e88) 95 (62e100) 48.7 (3.91e606)

8
4
8

47 (157/334)
70 (119/169)
47 (157/334)

342
564
329

23
55
24

507
241
506

128
140
141

38 (0e100) 89 (47e99) 4.84 (0.00e3.14
104)

16 (0e100) 84 (11e100) 1.02 (0.00e6.29
104)
39 (1e100) 89 (44e99) 4.97 (0.00e3.40
104)

4
3
4

17 (36/217)
19 (35/181)
17 (36/217)

62
31
64

92
128
94

742
679
740

104
162
102

59 (37e78) 94 (84e98)
56 (27e81) 93 (77e98)
58 (31e81) 94 (85e97)

20.9 (4.75e92.2)
16.0 (2.26e113)
19.9 (4.64e85.4)

10
7
10

17 (69/418)
18 (59/331)
17 (69/418)

97
100
95

53
61
53

781
761
782

68
78
70

74 (52e88) 72 (31e94)
69 (26e93) 78 (6e100)

7.38 (1.21e45.0)
7.73 (0.07e854)

6
4

42 (105/253)
44 (81/184)

309
302

165
123

420
437

107
139

6
2

4 (12/281)
5 (6/133)

38
IC

14
IC

944
IC

5
IC

88 (20e100) 99 (86e100) 503 (24.9e1.02
104)

IC
IC
IC

FN false negative; FP false positive; IC incalculable; TN true negative; TP true positive.

*Condence intervals are in parentheses.
y
The disease prevalence is dened by TP results and FN results divided by the total sample size ([TPFN]/[TPFPTNFN] in parentheses).
z
Based on the sensitivity, specicity, and disease prevalence of the tumor-extent parameter; sometimes the total sample size is 999 or 1001 due to rounding.
x
Studies that reported explicitly that their data was based on primary enucleations only (Tables 1 and 2).
k
Data from the worst reader in terms of diagnostic odds ratio in case studies reporting multiple sets of sensitivity and specicity from different readers
(Table 2). All other analyses include data from the best reader.

sensitivity of 100% and a specicity of 73%. However,

Brisse et al32 showed no signicant correlation between
nonvisualization of central retinal vessels and optic nerve
involvement (P 0.65).

Choroidal and (Extra)Scleral Invasion

As with postlaminar optic nerve invasion, high-resolution
imaging is mandatory to assess subtle invasions into the
choroid. Summary estimates of MRI in detecting choroidal
invasion also indicate room for improvement; studies with
a higher image quality show a higher diagnostic accuracy
(Tables 5 and 6; Fig 2, available at www.aaojournal.org).
Massive choroidal invasion is considered to be a risk
factor for metastasis. However, none of the included
studies reported diagnostic accuracy for massive
choroidal invasion separately. Only Chawla et al38
mentioned that 19 of 45 patients with choroidal invasion
had massive choroidal invasion, dened as at least 3 mm
of tumor ingrowth or reaching scleral tissue by the
International Retinoblastoma Staging Work Group.39 As
expected, (extra)scleral invasion is easier to detect
or exclude on MRI, showing a higher diagnostic
accuracy (Table 5; Table 3 and Fig 2, available at
www.aaojournal.org).
Evidence for the diagnostic accuracy of CT is scarce and
incomplete. Olivecrona et al40 reported that choroidal
invasion is not visible on CT images, but they did not
mention the total number of positive choroidal invasions

1114

on histopathologic analysis. In the same study, they

reported a sensitivity of 71% and a specicity of 98% for
scleral invasion, but 8 of 10 true-positive scleral invasions
extended profoundly into the orbit (Table 3 and Fig 2,
available at www.aaojournal.org).

Anterior Eye Segment Involvement and Ciliary Body

Invasion
Contrast enhancement of the AES on MRI scans gives
sensitivities of 68% to 93% and specicities of 43% to 82%
for the diagnosis of iris neoangiogenesis in 3 studies
(Table 4 and Fig 2, available at www.aaojournal.org). De
Graaf et al3 showed a sensitivity of 100% and a specicity
of 88% for MRI compared with AES abnormalities visible
with ophthalmoscopy. De Graaf et al41 also found that
the degree of AES enhancement on MRI matched
angiogenesis in the iris immunohistochemically (P
0.09). Anterior eye segment enhancement also has been
found to correlate with optic nerve invasion.32,41,42
Diagnosis of ciliary body invasion with MRI compared
with histopathologic examination showed sensitivities of
71% to 100% and specicities of 65% to 100% in a limited
number of studies (Table 4 and Fig 2, available at
www.aaojournal.org).

Limitations
Because the incidence of retinoblastoma is low, published
studies usually have small patient populations, resulting in

De Jong et al

Diagnostic Performance of Retinoblastoma Imaging

Figure 4. Graph showing the risk of bias and applicability concerns: review of authors judgements about each domain, presented as percentages across
included studies.

summary estimates with wide condence intervals. For ideal

diagnostic test comparison, retinoblastoma patients with
comparable severity of disease undergo the gold standard test
(histopathologic analysis) regardless of the index test (MRI
of CT). When the index test inuences the choice of whether
the gold standard test will be performed, this is called verication bias. However, a study design where this does not

Figure 5. Chart summarizing the risk of bias and applicability concerns:

review of authors judgements about each domain for each included study.
e high concern; ? unclear concern; low concern.

happen is unattainable in practice because of ethical considerations. In this case, enucleation depends on previous
clinical tests, but also on the index test. Verication bias can
lead to overestimation of sensitivity and underestimation of
specicity but generally does not inuence the DOR.20,43
Verication bias is likely to be present in all included
studies to varying degrees. Studies report varying prevalences of tumor-extent parameters (Tables 2e4, available at
www.aaojournal.org). Prevalence depends on more than
verication bias alone, but a higher than normal prevalence
may be attributed in part to verication bias. Inherently, the
results presented in this meta-analysis are based on patients
with severe retinoblastoma because only these patients undergo enucleation, allowing for a histopathologic evaluation.
To what extent these results are also generalizable to the
entire retinoblastoma patient population is difcult to say.
Studies have not always reported explicitly that the interpretation of the index test or histopathologic results was performed in a blinded manner. Not blinding the interpretation
can result in a slight overestimation of the DOR.20 Most
studies were unclear regarding whether their patients were
enrolled consecutively or randomly (yes, 4 studies; unclear,
13 studies; no, 1 study), and most studies either reported a
retrospective study design or did not mention this in their
article (retrospective, 9 studies; not specied, 7 studies;
prospective, 2 studies); however, Lijmer et al20 showed that
these study design aspects do not impact the DOR of a
diagnostic test. Some studies included both affected eyes of
1 patient, which might have caused an overestimation of
diagnostic accuracy; this issue was not part of the QUADAS
checklist. See Table 1 (available at www.aaojournal.org) for
studies that included more eyes than patients.
In meta-analyses, there is a risk that publication bias affects
the results. Publication bias usually occurs when small negative
studies (i.e., studies with poor diagnostic accuracy and a low
number of patients) are deemed uninteresting and are not published. Missing these studies could lead to an overestimation of
the summary estimates. We have chosen not to look at publication bias formally because of the lack of a good test.44
Heterogeneity across studies is a common limitation of
meta-analyses. The studies we included differ in terms of
imaging technique and quality, prevalence of tumor-extent
parameters, inclusion criteria (some also included secondary enucleations), and time between index and reference tests.
Sensitivity analysis by excluding studies that did not explicitly report only including primary enucleations showed
slightly lower diagnostic accuracy for most tumor-extent

1115

Ophthalmology Volume 121, Number 5, May 2014

Table 6. Metaregression of the Effect of Image Quality on Diagnostic Performance of Magnetic Resonance Imaging for Retinoblastoma
Tumor Extent
Tumor Extent
Prelaminar
Low qualityz
High qualityx
P value
Postlaminar
Low qualityz
High qualityx
P value
High qualityx (without outlierk)
P value
Choroidal
Low qualityz
High qualityx
P value
High qualityx (without outlier{)

Sensitivity*

Specicity*

Diagnostic Odds Ratio*

No. of Studies

Disease Prevalencey

57 (27e83)
83 (58e95)
0.12

97 (55e100)
93 (38e100)
0.59

47.0 (1.51e1.47
103)
67.2 (2.74e1.65
103)
0.85

4
4

43 (58/135)
50 (99/199)

59 (32e82)
58 (27e84)
0.94
76 (45e93)
0.26

94 (78e98)
94 (77e98)
0.98
96 (76e100)
0.63

21.1 (3.01e148)
20.5 (2.41e175)
0.98
84.8 (7.72e932)
0.30

5
5

21 (44/210)
12 (25/208)

11 (16/148)

71 (7e99)
77 (52e92)
0.80
IC

66 (16e95)
76 (21e97)
0.72
IC

4.84 (0.10e246)
10.9 (0.93e128)
0.62
IC

3
3

42 (64/151)
40 (41/102)

36 (24/66)

FN false negative; FP false positive; IC incalculable; TN true negative; TP true positive.

P values are for low quality versus high quality. Sensitivity, specicity, and disease prevalence are percentages.
*Condence intervals are in parentheses.
y
The disease prevalence is dened by true-positive results and false-negative results divided by the total sample size ([TPFN]/[TPFPTNFN] in
parentheses).
z
Includes studies with a low image quality score.
x
Includes studies with either an intermediate or a high image quality score (Table 1).
k
Without data from Wilson et al.29
{
Without data from Khurana et al.30

parameters, but the analysis was performed on a smaller

subset of studies, resulting in even wider condence intervals
(Table 5). We were able to analyze the effect of image quality
to some extent. We used a random effects model, which
adjusts the summary estimates and condence intervals for
between-study variations to a certain extent.
Because the consequences of an incorrect positive or
negative test results are considerable, it is important to
achieve a higher diagnostic accuracy. Unfortunately, not all
included studies reported the pixel size and section thickness
of their images; therefore, we had to use a semiquantitative
quality score to distinguish between low- and high-quality
studies. Most studies reported the use of sedation or anesthesia, but some studies did not (Table 1, available at www.
aaojournal.org). Even in patients under general anesthesia,
motion artifacts can pose a problem and can decrease the
image quality.45 Although the evidence is not very strong,
our results (for choroidal and prelaminar tumor invasion
and for postlaminar invasion after exclusion of 1 potential
outlier) did show a positive impact of image quality on
diagnostic accuracy. These results were not statistically
signicant. Nevertheless, we believe that high-quality images have the potential to increase diagnostic accuracy
considerably, and therefore, we think it is important that
MRI protocols are at least on par with the most recently
published guidelines by De Graaf et al,4 especially in
patients with (advanced) retinoblastoma for whom
conservative management is considered.
This systematic review gives an extensive overview of
the available evidence of the diagnostic accuracy of
tumor extent in advanced retinoblastoma. With only a

1116

fewdmostly olddstudies, there is very little evidence on

the diagnostic accuracy of CT, and generally these studies
show low diagnostic accuracy. For MRI, there is room
for improvement, especially for the test sensitivity.
Improvement of the diagnostic accuracy of MRI for retinoblastoma tumor-extent staging is pivotal for personalizing
retinoblastoma treatment, for example, for the justication
of conservative eye-sparing treatment strategies and for the
determination of correct surgical margins in case of unavoidable enucleation. Diagnostic accuracy studies of retinoblastoma tumor extent inherently suffer from a great deal
of verication bias, overestimating sensitivity and underestimating specicity. Clinical decisions, however, are not
based on only the test outcome of 1 tumor-extent parameter,
but rather on a combination of radiologic and clinical factors. How these results reect the value of MRI in the entire
clinical decision-making process and to what extent more
accurate imaging improves retinoblastoma care remains
difcult to predict. Larger prospective studies addressing the
role of cross-sectional imaging in the entire clinical
decision-making processdincluding its effect on patient
and eye survivaldare needed in the future. These results in
turn could be used for the development of clear decisionmaking criteria.

References
1. Moll AC, Kuik DJ, Bouter LM, et al. Incidence and survival of
retinoblastoma in The Netherlands: a register based study
1862e1995. Br J Ophthalmol 1997;81:55962.

De Jong et al

Diagnostic Performance of Retinoblastoma Imaging

2. Aerts I, Lumbroso-Le Rouic L, Gauthier-Villars M, et al.

Retinoblastoma. Orphanet J Rare Dis [serial online] 2006;1:31.
Available at: http://www.ojrd.com/content/1/1/31. Accessed
November 4, 2013.
3. De Graaf P, Barkhof F, Moll AC, et al. Retinoblastoma: MR
imaging parameters in detection of tumor extent. Radiology
2005;235:197207.
4. De Graaf P, Gricke S, Rodjan F, et al; European Retinoblastoma Imaging Collaboration (ERIC). Guidelines for imaging retinoblastoma: imaging principles and MRI
standardization. Pediatr Radiol 2012;42:214.
5. Rodjan F, de Graaf P, Brisse HJ, et al. Trilateral retinoblastoma: neuroimaging characteristics and value of routine brain
screening on admission. J Neurooncol 2012;109:53544.
6. Kaliki S, Shields CL, Rojanaporn D, et al. High-risk retinoblastoma based on international classication of retinoblastoma:
analysis of 519 enucleated eyes. Ophthalmology 2013;120:
9971003.
7. Chantada GL, Casco F, Fandio AC, et al. Outcome of patients
with retinoblastoma and postlaminar optic nerve invasion.
Ophthalmology 2007;114:20839.
8. Chantada GL, Dunkel IJ, Antoneli CB, et al. Risk factors for
extraocular relapse following enucleation after failure of chemoreduction in retinoblastoma. Pediatr Blood Cancer 2007;49:
25660.
9. Uusitalo MS, Van Quill KR, Scott IU, et al. Evaluation of
chemoprophylaxis in patients with unilateral retinoblastoma
with high-risk features on histopathologic examination. Arch
Ophthalmol 2001;119:418.
10. Honavar SG, Singh AD, Shields CL, et al. Postenucleation
adjuvant therapy in high-risk retinoblastoma. Arch Ophthalmol 2002;120:92331.
11. Bosaleh A, Sampor C, Solernou V, et al. Outcome of children
with retinoblastoma and isolated choroidal invasion. Arch
Ophthalmol 2012;130:7249.
12. Khelfaoui F, Validire P, Auperin A, et al. Histopathologic risk
factors in retinoblastoma: a retrospective study of 172 patients
treated in a single institution. Cancer 1996;77:120613.
13. Galluzzi P, Hadjistilianou T, Cerase A, et al. Is CT still useful
in the study protocol of retinoblastoma? AJNR Am J Neuroradiol 2009;30:17605.
14. Brenner D, Elliston C, Hall E, Berdon W. Estimated risks of
radiation-induced fatal cancer from pediatric CT. AJR Am J
Roentgenol 2001;176:28996.
15. Marees T, Moll AC, Imhof SM, et al. Risk of second malignancies in survivors of retinoblastoma: more than 40 years of
follow-up. J Natl Cancer Inst 2008;100:17719.
16. Wong FL, Boice JD, Abramson DH, et al. Cancer incidence
after retinoblastoma. Radiation dose and sarcoma risk. JAMA
1997;278:12627.
17. Mathews JD, Forsythe AV, Brady Z, et al. Cancer risk in
680000 people exposed to computed tomography scans in
childhood or adolescence: data linkage study of 11 million
Australians [report online]. BMJ 2013;346:f2360. Available at: http://www.bmj.com/content/346/bmj.f2360?view
long&pmid23694687. Accessed November 4, 2013.
18. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of
studies that evaluate healthcare interventions: explanation and
elaboration [report online]. BMJ 2009;339:b2700. Available
at: http://www.bmj.com/content/339/bmj.b2700. Accessed
November 4, 2013.
19. Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group.
Preferred reporting items for systematic reviews and meta-analyses:
the PRISMA statement [report online]. BMJ 2009;339:b2535.

20.
21.

22.

23.
24.
25.
26.

27.
28.
29.

30.

31.

32.

33.

34.

35.
36.

Available at: http://www.bmj.com/content/339/bmj.b2535?view

long&pmid19622551. Accessed November 4, 2013.
Lijmer JG, Mol BW, Heisterkamp S, et al. Empirical evidence
of design-related bias in studies of diagnostic tests. JAMA
1999;282:10616.
Whiting P, Rutjes AW, Reitsma JB, et al. The development of
QUADAS: a tool for the quality assessment of studies of
diagnostic accuracy included in systematic reviews. BMC
Med Res Methodol [serial online] 2003;3:25. Available at:
http://www.biomedcentral.com/1471-2288/3/25. Accessed
November 4, 2013.
Whiting PF, Rutjes AW, Westwood ME, et al. QUADAS-2
Group. QUADAS-2: a revised tool for the quality assessment
of diagnostic accuracy studies. Ann Intern Med 2011;155:
52936.
Arends LR, Hamza TH, van Houwelingen JC, et al. Bivariate
random effects meta-analysis of ROC curves. Med Decis
Making 2008;28:62138.
Glas AS, Lijmer JG, Prins MH, et al. The diagnostic odds
ratio: a single indicator of test performance. J Clin Epidemiol
2003;56:112935.
Deeks JJ. Systematic reviews in health care: systematic reviews of evaluations of diagnostic and screening tests. BMJ
2001;323:15762.
Tang BM, Eslick GD, Craig JC, McLean AS. Accuracy of
procalcitonin for sepsis diagnosis in critically ill patients:
systematic review and meta-analysis. Lancet Infect Dis
2007;7:2107.
Schueler AO, Hosten N, Bechrakis NE, et al. High resolution
magnetic resonance imaging of retinoblastoma. Br J Ophthalmol 2003;87:3305.
Lemke AJ, Kazi I, Mergner U, et al. RetinoblastomadMR
appearance using a surface coil in comparison with histopathological results. Eur Radiol 2007;17:4960.
Wilson MW, Rodriguez-Galindo C, Billups C, et al. Lack of
correlation between the histologic and magnetic resonance
imaging results of optic nerve involvement in eyes primarily
enucleated for retinoblastoma. Ophthalmology 2009;116:
155863.
Khurana A, Eisenhut CA, Wan W, et al. Comparison of
the diagnostic value of MR imaging and ophthalmoscopy
for the staging of retinoblastoma. Eur Radiol 2013;23:
127180.
Lee BJ, Kim JH, Kim DH, et al. The validity of routine
brain MRI in detecting post-laminar optic nerve involvement in retinoblastoma. Br J Ophthalmol 2012;96:
123741.
Brisse HJ, Guesmi M, Aerts I, et al. Relevance of CT and MRI
in retinoblastoma for the diagnosis of postlaminar invasion
with normal-size optic nerve: a retrospective study of 150
patients with histological comparison. Pediatr Radiol 2007;37:
64956.
John-Mikolajewski V, Messmer E, Sauerwein W,
Freundlieb O. Orbital computed tomography. Does it help in
diagnosing the inltration of choroid, sclera and/or optic nerve
in retinoblastoma? Ophthalmic Paediatr Genet 1987;8:1014.
Ainbinder DJ, Haik BG, Frei DF, et al. Gadolinium
enhancement: improved MRI detection of retinoblastoma
extension into the optic nerve. Neuroradiology 1996;38:
77881.
De Graaf P, Moll AC, Imhof SM, et al. Retinoblastoma and
optic nerve enhancement on MRI: not always extraocular
tumour extension [letter]. Br J Ophthalmol 2006;90:8001.
Ohguro H, Ohguchi S. A case of retinoblastoma suspected by
optic nerve invasion by MR imaging. Tumor Res 1999;34:837.

1117

Ophthalmology Volume 121, Number 5, May 2014

37. Jacquemin C, Karcioglu ZA. Detection of optic nerve
involvement in retinoblastoma with enhanced computed tomography. Eye (Lond) 1998;12:17983.
38. Chawla B, Sharma S, Sen S, et al. Correlation between clinical
features, magnetic resonance imaging, and histopathologic
ndings in retinoblastoma: a prospective study. Ophthalmology 2012;119:8506.
39. Sastre X, Chantada GL, Doz F, et al. International Retinoblastoma Staging Working Group. Proceedings of the
consensus meetings from the International Retinoblastoma
Staging Working Group on the pathology guidelines for the
examination of enucleated eyes and evaluation of prognostic
risk factors in retinoblastoma. Arch Pathol Lab Med 2009;133:
1199202.
40. Olivecrona H, Agerberg PA, Huaman A. European Radiology
Head and neck CT diagnosis of retinoblastoma with histopathologic correlations. Eur Radiol 1994;4:30713.
41. De Graaf P, van der Valk P, Moll AC, et al. Contrastenhancement of the anterior eye segment in patients with
retinoblastoma: correlation between clinical, MR imaging, and
histopathologic ndings. AJNR Am J Neuroradiol 2010;31:
23745.

42. Galluzzi P, Cerase A, Hadjistilianou T, et al. Retinoblastoma:

abnormal gadolinium enhancement of anterior segment of eyes
at MR imaging with clinical and histopathologic correlation.
Radiology 2003;228:68390.
43. Knottnerus JA. The effects of disease verication and referral
on the relationship between symptoms and diseases. Med
Decis Making 1987;7:13948.
44. Lau J, Ioannidis JPA, Terrin N, et al. The case of the
misleading funnel plot. BMJ 2006;333:597600.
45. Gizewski ER, Wanke I, Jurklies C, et al. T1 Gd-enhanced
compared with CISS sequences in retinoblastoma: superiority of T1 sequences in evaluation of tumour extension.
Neuroradiology 2005;47:5661.
46. Barkhof F, Smeets M, van der Valk P, et al. MR imaging in
retinoblastoma. Eur Radiol 1997;7:72631.
47. Sirin S, Schlamann M, Metz KA, et al. Diagnostic image
quality of gadolinium-enhanced T1-weighted MRI with and
without fat saturation in children with retinoblastoma. Pediatr
Radiol 2013;43:71624.
48. Song KD, Eo H, Kim JH, et al. Can preoperative MR imaging
predict optic nerve invasion of retinoblastoma? Eur J Radiol
2012;81:40415.

Footnotes and Financial Disclosures

Originally received: May 17, 2013.
Final revision: November 5, 2013.
Accepted: November 6, 2013.
Available online: March 6, 2014.

*A list of members of the European Retinoblastoma Imaging Collaboration

is available at www.aaojournal.org.
Manuscript no. 2013-785.

Department of Radiology, VU University Medical Center, Amsterdam,

The Netherlands.

Institute of Diagnostic and Interventional Radiology and Neuroradiology,

University Hospital Essen, Essen, Germany.

Department of Radiology, Centre Hospitalier Universitaire Vaudois and

University of Lausanne, Lausanne, Switzerland.

Unit of Diagnostic and Therapeutic Neuroradiology, Department of

Neurosciences, Azienda Ospedaliera Universitaria Senese, Siena, Italy.

Department of Radiology, Institut Curie, Paris, France.

Department of Ophthalmology, VU University Medical Center, Amsterdam, The Netherlands.

1118

Financial Disclosure(s):
The author(s) have no proprietary or commercial interest in any materials
discussed in this article.
Supported by the ODAS Foundation, Delft, The Netherlands (M.C.J.). The
European Retinoblastoma Imaging Collaboration (ERIC) is nancially
supported by a grant from the ODAS Foundation. The authors work was
independent of the funding organizations. The funding organizations had no
involvement in the design or conduct of this study, data management and
analysis, or manuscript preparation and review or authorization for
submission.
Correspondence:
Marcus C. De Jong, MD, MSc, Department of Radiology, VU University
Medical Center, P.O. Box 7057, 1007 MB Amsterdam, The Netherlands.
E-mail: mc.dejong@vumc.nl.