Professional Documents
Culture Documents
O
n 3 March, two studies appeared online women — to ask whether these seemingly contrast, detected 63%.) Mor’s panel already
that offered 19 pages of gloomy reading breakthrough biomarkers were better at iden- had a tortured history. A primary research
for anyone interested in cancer. They tifying women with early ovarian cancer than paper behind it had been criticized by other
focused on biological molecules, or the one flawed biomarker that had been in use scientists for allegedly using inappropriate
biomarkers, the presence of which in the blood for almost 30 years, CA-125. None of them statistical calculations and for optimistically
might be used to detect the earliest glimmers of was1,2. “CA-125 remains the ‘best of a bad concluding that the test would help women
ovarian cancer — a disease not normally dis- lot’,” read an accompanying perspective arti- before rigorous follow-up studies proved that it
covered until it has destroyed the ovaries and cle3. “The new candidates have fallen short of could. Yet for four months in 2008, the test was
rotted other parts of the body. The researchers, expectations.” sold to patients by Laboratory Corporation
coordinated by the Early Detection Research Tied in last place for its poor performance of America (LabCorp) in Burlington, North
Network (EDRN) of the US National Cancer among the biomarker panels was one iden- Carolina, the company that licensed the panel
Institute (NCI), had assembled 35 protein bio- tified by Gil Mor, a cancer biologist at Yale from Yale. LabCorp had marketed the test
markers, including 5 panels of proteins, that University in New Haven, Connecticut. Mor’s under the name OvaSure until the US Food
had looked the most promising in early stud- six-protein panel detected ovarian cancer in and Drug Administration (FDA) intervened
ies. They had carried out rigorous testing — only 34% of the women who were diagnosed and the company pulled it from the market.
screening blood samples from more than 1,000 with the disease within a year. (CA-125, by The panel offered “invaluable object lessons”
4 2 8 | NAT U R E | VO L 4 7 1 | 2 4 M A RC H 2 0 1 1
© 2011 Macmillan Publishers Limited. All rights reserved
FEATURE NEWS
Biomarkers might help to detect ovarian tumours a high-profile decision at Duke University in experts agree that potential biomarkers for early
2 4 M A RC H 2 0 1 1 | VO L 4 7 1 | NAT U R E | 4 2 9
© 2011 Macmillan Publishers Limited. All rights reserved
NEWS FEATURE
Mor recalls. “She was 29. She was a beautiful (PNAS), with Ward as a contributing author5. Feng and Gary Longton, another statistician
girl. An impressive girl. A medical student. And Before publication, Mor helped the Yale at the FHCRC, developed their own classifica-
I never understood what happened to her.” Office of Cooperative Research to prepare tion algorithms, and found that Mor’s test had a
Mor decided to leave medicine, which had a patent application. “A lot of companies sensitivity of 95% and specificity of 99%. They
been unable to save her, for research, which expressed interest in licensing the panel,” also calculated the positive predictive value
one day might. He earned a PhD studying says John Puziss, director of technology (PPV) of the test — the proportion of patients
ovarian cancer at the Weizmann Institute of licensing at Yale. LabCorp licensed the test who the test would diagnose with the disease
Science in Rehovot, Israel, before moving to in 2006, as did Millipore, a biomanufac- and do in fact have it. A high PPV means that
Yale in 1997. He went on to start a programme turing company based in Billerica, Mas- few people will be misdiagnosed, which is cru-
called Discovery to Cure, aiming to speed can- sachusetts. (Mor says that the royalties he cial when screening healthy people.
cer research to the clinic. The group began to and his co-inventors received “were not a Feng and Longton calculated the PPV at
build a bank of blood significant amount”.) 6.5%, too low for the test to be of much use for
and tissue samples, The test’s promising results had also caught screening. But separately, Mor was working with
including some from a “AS WE’RE the attention of researchers in the EDRN, a different figure, of 99.3%. The huge disparity
Yale clinic for women MOVING UP who were just putting together their valida- between the two values stemmed from the way
with a high risk of ovar-
ian cancer owing to a TO MULTIPLE tion study. Up to that point, most biomarkers
for detecting early ovarian cancer had only
that they calculated the figure and factored in
the prevalence of ovarian cancer — an impor-
family history of the MARKERS, ALL been shown to distinguish patients with diag- tant variable in calculating the PPV. Following
disease.“There was a lot
of excitement around
OUR BAD HABITS nosed cancer from healthy controls, but they
are intended to detect the disease in women
convention, Feng and Longton calculated the
PPV using the accepted prevalence in post-
that time for finding ARE COMING whose cancer is just budding, before symp- menopausal women, 1 in 2,500 (0.04%). But
proteins specific to can-
cer,” says Mor.
BACK TO BITE US toms develop. What the field needed was a
‘prospective’ study, run on blood samples from
Mor’s figure was calculated solely from the study
population, in which the prevalence was 46%.
In 2003, David Ward, IN A BIG WAY.” apparently healthy women, to see whether the “We calculated the PPV based on the popula-
then a geneticist at Yale, biomarkers could pinpoint those who would tion in the study, because we always intended
contacted Mor. Ward had co-founded Molec- later be diagnosed with ovarian cancer. Such the test for the high-risk population,” says Mor.
ular Staging, a company in New Haven that samples, from large numbers of women who “If you want to bring the test to the clinic, it
had developed a ‘high-throughput’ technique are tracked over months or years, are extremely has to be calculated based on the population
for quantifying multiple proteins in the blood difficult to come by. you’re going to study,” he says, noting that other
using arrays of antibodies4. He asked whether research studies work out the PPV for the study
he could use Mor’s samples to search for mark- PROBLEM DETECTION population in this way.
ers of early ovarian cancer. The EDRN found what was needed in the It’s a common mistake, believes McShane,
Mor had never been involved with bio- Prostate, Lung, Colorectal, and Ovarian who — like other statisticians — disagrees with
marker research — “I do biology of cancer, (PLCO) Cancer Screening Trial, sponsored Mor’s logic. “I see that a lot, but it is nowhere
not biomarker development,” he says — but and run by the NCI. Between 1992 and 2001, near the correct thing to do,” she says. Even
he signed up, intrigued by the clinical poten- the trial had been collecting blood at regu- in high-risk populations — women who are
tial of the technology. Ward had scoured the lar intervals from 155,000 women and men, screened every year because of their family his-
literature for proteins that had been associ- and screening them for cancer. By June 2006, tory or because they have tested positive for
ated with ovarian-cancer growth and malig- 118 of the women had developed ovarian mutations in tumour-suppressor genes BRCA1
nancy, and had come up with 169 candidates. or closely related cancers, and the EDRN or BRCA2 — the prevalence is around 0.5%,
Using the protein-quantification technique, researchers were now in a position to use them far below the 46% in
Ward’s company screened blood samples in to evaluate the most promising biomarkers Mor’s study population.
Mor’s tissue bank that came from two groups: for early detection. Ziding Feng, a biostatisti- “IT’S IMPORTANT Similar battles over the
women with newly diagnosed ovarian can- cian at the Fred Hutchinson Cancer Research FOR THE WHOLE correct use of statis-
cer who had been enrolled in Yale’s high-risk
clinic, and women who had come to the hos-
Center (FHCRC) in Seattle, Washington, and
coordinator of the EDRN, visited Mor to dis-
FIELD TO STEP tics litter the cancer-
biomarker field, said
pital for routine gynaecological exams. Using cuss whether his panel of four proteins could BACK AND LOOK researchers at the IOM
additional cancer-patient samples, they whit-
tled the list down to four proteins: leptin, pro-
be included in the study.
Mor was already in the process of refin-
AT WHAT IS meeting last year. “It’s
the type of thing where
lactin, osteopontin and insulin-like growth ing the panel: he had more patient samples, WRONG.” non-statisticians think
factor II. and wanted to add more markers, including statisticians are being
Mor worked to develop an algorithm that CA-125 and the protein macrophage migra- uptight about something that’s not going to
could automatically classify women as hav- tion inhibitory factor, to make the test more matter anyway,” says McShane.
ing cancer or not, depending on levels of these sensitive to cancer. LabCorp had been run- Mor prepared a paper reporting the latest
four proteins. When the team ran a new set of ning his new samples on assay kits manufac- work. But when Feng and Longton saw the
blood samples through the algorithm, they got tured by Millipore. (Ward, meanwhile, had page proofs, they noticed that the PPV value
astounding results. The test showed a sensitiv- moved to the Nevada Cancer Institute in Las was reported as 99.3%. They asked Mor to
ity of 95% (meaning it correctly detected 95% Vegas, and was not involved in data collection change it to the 6.5% that they had calcu-
of the ovarian-cancer cases) and a specific- or analysis.) lated, and to correct a few other typographical
ity of 95% (it erroneously classified only 5% When Mor showed Feng how he was ana- errors in the tables. “He agreed, so we signed
of healthy people as having cancer). “I was lysing his recent data, Feng was troubled. Mor off,” recalls Feng. But there was a miscommu-
delighted,” says Mor. On equivalent samples, asked him to go through the new results him- nication: Mor thought that Feng had agreed
CA-125 tests typically have a sensitivity of self, and Feng agreed to collaborate. “I do not to the use of the high PPV, and that everyone
70–80% and a specificity of around 95%. In do statistics,” says Mor. “That is not my field.” approved of the final manuscript.
May 2005, the findings were published in the The researchers also added the six-protein The paper was published online in Clini-
Proceedings of the National Academy of Sciences panel to the EDRN’s validation study. cal Cancer Research6 in February 2008, and to
4 3 0 | NAT U R E | VO L 4 7 1 | 2 4 M A RC H 2 0 1 1
© 2011 Macmillan Publishers Limited. All rights reserved
FEATURE NEWS
imagine how upset I was when I saw it in the might be a good way to do further validation. Canary group wrote a letter to Clinical Cancer
paper,” says Feng. “It’s very difficult to do that on large numbers Research, describing some of their complaints.
Feng called Mor. “I told him, those are of patients,” he says. “It’s extremely expensive. Meanwhile, Feng agreed to co-author a second
errors, we told you those are not correct.” Feng The only way to do the study is if LabCorp letter, criticizing the paper even though he was
also contacted the journal, the editor of which started distributing the test and enrolling a co-author.
asked Mor to submit a correction to fix the patients.” Mor notes that many tests, such as The fuss was already reaching the FDA,
PPV and the other typos. Mor agreed, adding mammography, have been offered to patients which on 7 August 2008 sent a letter to Lab-
the lower PPV as a footnote to the table and in as an aid to diagnosis even while data on the Corp saying that the test “has not received
a written correction. test are being collected.
“Was it the right time? adequate clinical validation, and may harm the
A few weeks later, Feng received an e-mail I don’t know,” he says. public health”. A second letter, sent by the FDA
with unwelcome news from a colleague: Lab- on 29 September 2008, alleged that LabCorp
Corp was preparing to market the panel, and CRITICAL BACKLASH did not have the necessary marketing clear-
was “hopeful that this test will be available to On 23 June 2008, LabCorp announced the ance or approval for the test from the FDA.
women by the end of the year”. availability of the OvaSure test, for between LabCorp replied to the FDA on 20 October,
“I was shocked,” says Feng. “I had no idea US$220 and $240. The press release said that disagreeing with the agency’s assertions, but
this was coming.” He thought that the mark- it was being offered to women with a high risk agreed to pull OvaSure from the market. It
ers should be validated further before they of the disease, and quoted Mor as saying he did so on 24 October 2008, just one day after
went to the clinic. In March 2008, Feng and was “pleased that this test is available to help Clinical Cancer Research published the critical
Mor saw each other at a meeting in Washing- physicians detect and treat ovarian cancer in letters from the Canary Foundation and Feng,
ton DC. “I told him, face to face, you cannot its earliest stages”. as well as a third from the Centers for Disease
do this,” says Feng. “You have to wait until Excited chatter about the test spread through Control and Prevention (CDC) in Atlanta,
after the PLCO validation. What you have patient forums and support groups, but it Georgia8–10. (Millipore continues to market
done is early discovery. If validation does not was soon countered by cautionary tales. Jean the biomarker panel for use in research, not
support your earlier claim, you’re making a McKibben, an ovarian-cancer survivor, by patients.)
significant error.” Mor does not recall this rushed to take OvaSure on the first day it was Mor was surprised by all three letters. In
encounter, but says that Feng’s “role was to available, and her results showed a 0.00 chance his published response11, he disputed some
analyse the data, not to make judgements of of cancer. A week later, scans showed that her of the criticisms and wrote that any concerns
a company decision”. cancer was back. She was crushed. “I wanted about commercialization should be taken
Now, Mor says that if he were preparing the this to work so badly,” she wrote on a discus- up with LabCorp. Stephen Anderson, vice-
paper again, he would include both the low sion board. president of investor relations at LabCorp,
and high values for the PPV. And he vacillates One week after LabCorp’s announce- says that OvaSure was not marketed as a test
about whether LabCorp’s decision to offer the ment, the Society of Gynecologic Oncolo- for detecting cancer recurrence, which was
test to women before it had undergone more gists in Chicago, Illinois, released a statement how some patients used it. He says that Lab-
validation studies was the right thing to do. expressing concern about OvaSure, saying Corp “continues to believe OvaSure offers a
2 4 M A RC H 2 0 1 1 | VO L 4 7 1 | NAT U R E | 4 3 1
© 2011 Macmillan Publishers Limited. All rights reserved
NEWS FEATURE
4 3 2 | NAT U R E | VO L 4 7 1 | 2 4 M A RC H 2 0 1 1
© 2011 Macmillan Publishers Limited. All rights reserved