1

How to Publish Counterexamples in 1 2 3 Easy Steps

Theodore P. Hill
School of Mathematics, Georgia Tech
hill@math.gatech.edu

Last updated September 29, 2009

Inspired by Rick Trebino’s How to Publish a Scientific Comment in 1 2 3 Easy Steps, and
encouraged by my friend and colleague Ron Fox’s treatise on Science and Integrity, I was
motivated to record one of my own recent experiences in mathematics.

In some sense, there is a significant difference between mathematics and other sciences - we
mathematicians are fond of modestly referring to our field as “the Queen of the Sciences” - and
that difference is largely embodied in the difference between a theory and a theorem. Theories
come and go – geocentric theories of the solar system were eventually replaced by
heliocentric theories, and one Big Bang theory by another. Theorems, on the other hand, are
more or less eternal. The Pythagorean Theorem, the Central Limit Theorem, and the
Fundamental Theorem of Algebra, re-verified by successive generations of scholars, are
accepted as true by essentially all scientists. There are no competing theories once the
hypotheses are agreed upon. Thus there are essentially no debates about whether or not
theorems are true, and published “Comments” about the truth of established theorems are
almost nonexistent.

This perceived permanence of theorems has a major drawback – the blind acceptance of a
published false theorem that has not been retracted is often the basis for future false
theorems. But we mathematicians also make at least our share of mistakes, some of which
find their way into print, and when that happens, correction of the record is imperative. In
mathematics, however, the correction does not occur in the form of a Comment or Debate or
Opinion, but rather in the form of a Counterexample. A Counterexample, in short, is a formal
proof, via an example, that a certain mathematical claim or theorem is false.

But even in the Queen of the Sciences, publishing Counterexamples is not always easy, for
exactly the same reasons that Trebino found was the case for publishing Comments in
physics. I am recording these events to further support Trebino’s findings, and in the hope that
young scientists may learn from our mistakes, and not be discouraged when they dare speak
out.

As Ron Fox pointed out to me, the present article is complementary to Trebino’s, in that
Trebino was trying to correct erroneous statements published about his own research, and I
was trying to correct research errors published by other scientists. Also, in contrast to
Trebino’s article, this chronology contains exact dates, quotes, and journal names, and is a
single sequence of concrete events, not a composite. Hence, the reader may find the humor
here slightly more subtle (and dark), since much of it must be gleaned from the actual words of
the participants. With a tip of the hat to Rick Trebino, here goes:
 2

Suppose that you discover serious mathematical errors in a widely-circulated journal published
by, say, the American Mathematical Society. The official AMS Code of Ethics
http://www.ams.org/secretary/ethics.html proudly proclaims, in part,

"The public reputation for honesty and integrity of the mathematical community and of
the Society is its collective treasure and its publication record is its legacy...

mathematicians have certain responsibilities, which include...To correct in a timely way

or to withdraw work that is erroneous”.

So, after writing out and checking your Counterexamples, you assume that correction of those
published errors in the Queen of the Sciences will consist of Three Easy Steps:

1. You politely notify the authors of the errors, providing concrete counterexamples, and the
authors then confirm (and perhaps contest some of) the errors.

2. You and the authors contact the Editor, who verifies the errors with an Associate Editor and
an expert referee.

3. The Editor publishes your counterexample proofs in a timely manner in the same journal
where the false proofs appeared.

Easy as  , no? Dream on!

The Players:

Au1. First author, a professor of political science at a leading U.S. university

Au2. Second author, an associate professor of mathematics at a smaller U.S. university
Au3. Third author, an assistant professor of economics at a European university
Ed1. Editor of the Notices of the American Mathematical Society, a professor of mathematics
at a large U.S. university
Ed2. Associate Editor of the Notices, a professor of mathematics at a leading U.S. university,
and member of the National Academy of Sciences

The Events:

December 2006
Under the editorial direction of Ed1, the Notices of the American Mathematical Society,
perhaps the most widely-read AMS journal, publishes an article by Au1-2-3. The article is
widely promoted by an official AMS News Release, podcast in Scientific American (Online),
and The Discovery Channel, among others.

6/1/07
The Editor of SIAM Review, a major publication of the Society for Industrial and Applied
Mathematics, asks me to review a different paper on the same general topic by Au1-2-3, who
cite their own December Notices article.
 3

7/11/07
I review the SIAM submission, find serious mathematical errors, and send my report on the
SIAM Review submission, including detailed proofs of the errors in the form of concrete
counterexamples to the main results. The SIAM Editor agrees with my findings, sends the
authors Au1-2-3 a copy of my report containing the counterexamples, and rejects the article.

[ASIDE] Not to worry – persistence is essential in publishing. Four months later (see
11/26/07), Au1-2-3 publish the same article, essentially without change, even after the authors
know there are errors in it (see 8/27/07).)

Curious as to how the errors arose in the first place, I study the original December 2006
Notices article cited by Au1-2-3, and find many of the same errors.

7/26/2007
After double-checking my findings with two experts in the field, I email Au2:
“Dear [Au2], It appears that there may be some serious mathematical errors in your
December 2006 in the Notices (since you are the mathematician on the team, I am
writing you directly, rather than [Au1]). I would like to talk to you on the phone, and ask
a few questions… Thanks in advance. Regards Ted Hill”

Later same day

I do not receive a direct reply from Au2, but instead from Au1, who emails me:
“You're quite right that there was at least one error, which we attempted to correct in the
attached Corringendum [sic]. Of course, we would like to hear of any other errors that
you might have found.”

Still later same day

I email Au1:
“Thanks for the quick reply, and the draft of your correction. Attached are some notes I
drafted - maybe I am missing something (but then so are two colleagues who also
checked it for me). I will try to study your proposed correction soon”

7/30/07
I email Au1-2-3 detailed proofs of errors in their Corrigendum, and tell them I sent the Editors
of the Notices a draft of my counterexamples “to help them coordinate the correction process
(assuming that they agree corrections need to be made).” I also send my latest
counterexamples to a third expert in the field, who also confirms my findings.

7/31/07
Au1 emails me:
“My coauthors and I have discussed [your counterexamples], and most, we think, are
based on misconceptions or misinterpretations of our results.”
 4

8/2/07
I email the Authors that:
“I sent you a detailed mathematical account of the errors in your article. I have no
reason to believe there is anything erroneous in what I sent you, and indeed had it
checked by three other mathematical experts. …you state that you think that
misconceptions and misinterpretations are the basis of my criticisms. Please give me
detailed arguments to support your claim, just as I have already done for you.”

Later same day

Authors reply:
“Your request for feedback would be a reasonable one if you had
not submitted your comments to the Notices…We will respond to the editor of the
Notices if asked to do so.”

8/3/07
I email Ed1 that:
“I have been in contact with Professors [Au1-2-3] about my corrections to their
December 2006 article in the Notices. This communication has been unproductive …
Therefore, please consider my 7/27/07 [Counterexamples] submission for publication in
the Notices.”

8/20/07
I email Authors that:
“I have looked over your Corrigendum (attached), and found what appear to be several
significant logical errors”, which I then describe in detail.

8/27/07
Au2 emails me:
“I would like to write a new corrigendum… [since] I do think that some of your comments
have merit and should be addressed in print”.

He tells me he also agrees that their SIAM Review submission contained similar errors, and
mentions that Au1 alleged a previous “plagiarism” issue(!) involving me, but Au1 did not give
him details.

[ASIDE] Science and mathematics is sometimes much like football and war – the best defense
is often perceived to be a good offense. Attack, attack, attack.

8/30/07
Au2 emails me once more:
“I wanted to thank you again for reading our work closely. We appreciate the time that
you have spent on this! (I'll speak for [Au1 and Au3] on this one.) And, we will certainly
thank you in print if given the chance.”

[ASIDE] But given numerous opportunities to thank me in print later (see 11/26/07 and
October 2008) they never did. Still waiting.
 5

8/31/07
I email Ed1 and also Ed2 (who has worked in the field of the Au1-2-3 papers) with concrete
counterexamples to the theorems and claims in the original Notices article, as well as
counterexamples to statements and proofs in the Au1-2-3 Corrigendum.

9/2/07
Ed1 emails me suggesting I should publish the Counterexamples elsewhere, and on the math
arXiv, and then the Notices can publish a short correction note.

[ASIDE] I send the Editor of the Notices my Counterexamples to theorems that appeared in
his journal under his watch, and his solution is for me is to publish the Counterexamples
elsewhere?!?!.

9/4/07
I respond that:
“In light of the media attention the [Au1-2-3] article is receiving (AMS News Release,
Scientific American, The Discovery Channel etc), it is therefore imperative that the
Notices (not some second-rate journal or on-line arXiv that many Notices readers like
me rarely reads) publish the Counterexamples, and publish them expeditiously, perhaps
also asking [Au1-2-3] to retract the paper.”

10/6/07
Au2 emails me:
“I had lunch with [Au1] a couple of weeks ago and he said that he was not interested in
writing a joint corrigendum (or whatever it would be called) which would include your
comments, examples, criticisms, etc. He believes that we should respond once your
final comments appear or are posted.”

[ASIDE] Apparently Au1 wants the truth decided by debate, and volunteers to make the
closing argument.

10/12/07
I email Ed1 and Ed2, copied to Au1-2-3:

“[Au1-2-3]'s refusal to correct their article or retract it in a timely manner (it may take
years for my Counterexamples to be published, if ever), is in direct violation of the AMS
Code of Ethics”.

I include the exact statement of the COE quoted in the beginning of this article, and
state that

“It has now been nearly three months since I brought these errors to the attention of the
authors and the Notices. Since the authors have gone on the record saying they will not
correct their paper until my Counterexamples are published, it is time for the Notices
editors to make a clean decision - accept or reject my submission. If the Notices wants
to reject my Counterexamples after three months delay, without refereeing it, and
knowing full well that publication of the Counterexamples elsewhere could take years,
 6

so be it. That will be part of the record. …It is the professional duty of the authors (and
editors), not the person who discovered the errors, to see that faulty statements and
proofs are corrected as soon as possible.”

10/18/07
To clarify the issues, I email Ed1-2 and Au1-2-3 specifying:
“a dozen errors in the original article, and attached detailed arguments and
counterexamples to support each of those claims.”

I also state:
“The option to have the Notices simply refer to un-refereed second-party URL’s for the
Counterexamples and/or Corrections is not acceptable -- websites’ contents may be
changed at any time, they do not carry the Notices’ stamp of approval that the original
article received, and they may not be available to all readers of the Notices, hardcopies
of which reach some very remote locations (and as [Ed2]’s note says, ALL those
Notices readers who read part or all of that paper deserve notification of any significant
errors in the paper).

“Nor is a Letter to the Editor the place to publish corrections, whether the publication be
Newsweek, New York Times or the Notices -- the journal itself must acknowledge the
errors, not some private individual or individuals.”

[ASIDE] These were wasted words on my part - Ed1’s Final Solution, a YEAR LATER (see
October 2008), is that he himself publishes a Note in the Letters to the Editor (i.e., a Letter to
the Editor from the Editor… also see AFTERMATH below).

Later same day

Ed2 emails Au1-2-3 and me:
“I propose that the four of your [sic] put your heads together, as scientists (not as
humans who may have previous history), with the sole purpose of enlightening your
audience. The issues should be mathematical, and usually mathematicians can reach
agreement on what is correct and what isn't.”

[ASIDE] It is the Queen of Science, after all – see Intro.

10/20/07
Au1 emails Ed2:
“We think Ted's critique is without merit [emphasis added].”

[Aside] If you can’t attack – deny, deny, deny. When Richard Pryor’s wife caught him in
flagrante delicto with another woman, he just denied it. “Who you gonna believe,” he asked his
wife, “me - or your lying eyes?”

Au1 continues:
“Becsause [sic] we have found a dialogue with Ted impossible, we think the only
solution is for Ted to finalize his critique. We will then offer a point-by-point response,
 7

and that will close matters. This critique and rebuttal could then be reviewed by you or
other referees for possible publication. We do not want to get into an endless debate,
which has already gone on too long. Finally, I would note that in a related area, game
theory, there have been published exchanges, such as between Borel and von
Neumann about which one discovered the minimax theorem. [Au2], [Au3], and I see no
alternative but to have such an exchange and let this close matters. I hope you agree.”

[ASIDE] Note that Au2 had already confirmed some of the errors (see 8/27/07); and that a
year later, after many more denials, Au1-2-3 finally admit many of the errors in print (see
October 2008).

10/21/07
Ed2 does not agree that a “Borel-von Neuman exchange” is the proper way to set the
mathematical record straight. He emails Ed1, Au1-2-3 and me saying:
“I'd like to get the science settled first, and then we can better decide how to get this info
to the audience”, and that he would like me to first identify one or two concrete issues to
start with. He asks Au1-2-3 for their Corrigendum, and also states, “For example, as I
understanding cake cutting, it seems to me that the second objection of Ted's, a
counterexample to Pareto optimality, is valid. Or am I missing something?”

I send my first counterexample issue.

10/22/07
Authors send Ed2 their Corrigendum to their original article, which Ed2 sends to me and asks
whether it addresses my concerns.

Later same day

I email Ed2 that the Corrigendum also has significant errors, which I spell out with concrete
counterexamples, and tell him that I had already sent those to Au1-2-3.

10/26/07
Authors send Ed2 a response to my first counterexample.
Ed2 emails me a copy of his reply to Authors – he doesn’t buy the Richard Pryor Defense:
“But this is not a response to the counterexample. Ted refers to prose on page 1314,
where you quite clearly say that cut-and-choose satisfies (2). As far as I can tell, it
doesn't, and Ted is correct in what he wrote. ..to say that Ted's counterexample is
"without merit" [see 10/20/07] just seems flatly wrong. Perhaps it can be explained, but
dismissal "without merit" doesn't work with me.”

Ed2 then asks me for a second concrete issue to focus on; I send it to him and Au1-2-3.

11/13/07
Authors email Ed2 their response to my second counterexample, stating:
“It is worth noting at the outset that we deliberately wrote an expository article, which the
Notices encourages, without all the formality of a mathematics research paper.”, and
conceding “Hill is correct that in the special case…” . They concluded “Our results are
consistent with the definitions we used, except for exceptional cases.” [emphasis
added].
 8

[ASIDE] This unique and creative defense is admirable – except for one tiny detail. Using their
logic, you could easily get your name in lights with exciting new results such as: Theorem: All
numbers are larger than three, which is 100% correct except for exceptional cases.

11/16/07
I email Ed2 a detailed refutation of Au1-2-3’s 11/13/07 “exceptional cases” response, including
my reaction to their “expository article” defense, stating:
“Even for expository articles, the reader expects that standard technical terms such as
“absolutely continuous” have the standard meaning, and the reader expects that if an
expository article contains items clearly labeled “Theorems” and “Proofs”, those should
be more or less correct and complete.”

11/26/07
Authors publish essentially the same paper they submitted to SIAM Review (see 7/11/07
above) in Dagstuhl Seminar Proceedings 07261, knowing that it contains errors I had found
four months earlier in my referee’s report (see 8/27/07).
They also publish a sequel to their original Notices article containing similar errors, in Dagstuhl
Seminar Proceedings 07261.

[ASIDE] In contrast to Au2’s promise to acknowledge my contribution (see 8/30/07 above),

neither of the articles contain any thanks to me, let alone correction of the mathematical errors.

12/15/07
To clarify where things stand, Ed2 asks me to provide him and Au1-2-3 with a summary of
what mathematical points I think we have agreed on, and what points we have not. I email
them a succinct list of ten concrete points, and say:
“If you believe any are incorrect, please let me know which ones, and what the
correction should be.”

Au1-2-3 do not respond to me, but email Ed2 requesting a decision on his part now that “the
math, as you wished, has been sorted out".

12/29/07
I email Ed2:
“After nearly six months, [Au1-2-3] are still dismissing 9 out of 10 of my concrete points,
including CX 1 that you carefully analyzed, CX 4 where their Nov 13 response admitted
“Hill is correct” (but claimed the case is “exceptional”), and CX 5 where the Nov 13
response said “...as illustrated by this counterexample [but] this case is exceptional”.
According to their latest response, however, they somehow feel that now “the math, as
you wished, has been sorted out"! …”

I ask Ed2 to have my Counterexample paper formally refereed.

2/14/08
Ed2 emails me that:
 9

“I'm afraid I've run out of steam for this endeavor. I think most of the evidence is in your
favor, and [Au1] has basically stonewalled. … It isn't the sort of thing the Notices (it is
not a journal of record, nor a research journal) will want to do much about in the way of
correction. Perhaps they should, but they won't want to spend more pages than [Au1-2-
3] did in order to explain what was wrong. …You know, correctly, in your heart that you
are right on most or all of the issues, and that I'm pretty much on your side as far as I've
gotten into things. That's not much, but…”

[ASIDE] Ahh, I feel better already, knowing correctly in my heart…

2/20/08
I email Ed1, informing him that Ed2 suggested that my earlier submission of Counterexamples
might be viewed as unacceptable because it is too long, and remind him of the AMS Code of
Ethics statement on integrity and correction of errors (see 10/12/07).

I attach a much shorter summary of the Counterexamples, and ask that it be considered for
publication in the Notices.

6/13/08
Under pressure, Ed1 finally has the revised shorter Counterexamples refereed, and sends
Au1-2-3 and me the referee’s report, telling us:
“The (anonymous, of course) referee was suggested by [Ed2] as someone all involved
would mutually respect, and I believe that upon reading the report appended below all
will agree an excellent job has been done.”

After spelling out detailed analyses of the points, the referee concludes:
“I agree that Hill's concerns are justified, and that the authors of [Au1-2-3] might want to
address Hill's concerns. I will leave it up to the Editor to decide exactly how that should
be done.”

7/11/08
Ed1 emails me that he has rejected for publication both my original and my streamlined
submissions of the Counterexamples, and says:
“I've decided that the way to inform the Notices readers about cake-cutting issues is via
an "Editor's Note" on the subject that will appear in the October issue”.

[ASIDE] Recall my explicit and uncontested objection to this solution in (10/18/07).

Ed1 appended a “courtesy copy” of his proposed Note, which acknowledged “serious
mathematical errors” in the original Au1-2-3 article, “which have been checked by an expert
referee, [and] verify the following mistakes…”

Ed1’s proposed Note also included final responses by Au1-2-3 (i.e., allowing them the last
word in the “exchange”, just as they requested (see 10/6/07 and 10/20/07). It also included the
Editor’s amazing policy statement
 10

“a final editorial comment: precision and exposition are not necessarily incompatible, although
when they do conflict the Notices tends to favor the second, or rather encourage authors to do
so”.

[ASIDE] This authoritarian decree surprised my mathematician colleagues and me, especially
since the Notices is an AMS journal, and its editors supposedly adhere to the AMS Code of
Ethics (see 10/12/07, and AFTERMATH below). But he is the Editor, so what do we have to
say about it? In fact, what do the Associate Editors have to say about it? I’ll wager ten-to-one
odds that Ed1 did not check with his Associate Editors when he made this statement, and
certainly not with the Associate Editor who is well known (e.g. Wikipedia) for saying

"A physics paper, like a newspaper article, is not meant for posterity; dotting of i's and
crossing of t's is meant to happen after the fact, and is not in any case the 'real' work of
a physicist. A mathematics paper, on the other hand, is supposed to be a work of art:
perfect, complete, and beautiful ... [Otherwise] mathematics can only go in one
direction: toward the profane."

7/14/08
I email Ed1 and Au2 that the Au1-2-3 response quoted in his proposed Editor’s Note contains
additional new mathematical errors, and I again provide explicit counterexamples.

Later same day

Ed1 emails me:
“I'm not going to make further changes before I put the piece into production.”

[ASIDE] In other words, the Editor is going to publish mathematical claims he has reason to
believe are false, simply because of the editorial inconvenience of checking and making the
corrections. Truth be damned, deadlines are deadlines.

October 2008
I had only made two main requests to Ed1: that the Notices publish corrections “expeditiously”
(see 9/4/07); and that the Notices publish the corrections in a proper forum, specifically, not in
Letters to the Editor (see 10/18/07).

But when the Notices finally publishes a “correction”, more than a year has passed since I
notified Ed1 of the errors, and the “correction” is published by the Editor as an Editor’s Note
buried in the bottom of its section Letters to the Editor:

[ASIDE] Editors can be creative too… But we can’t complain - the American Mathematical
Society has democratically elected officers, and as H. L. Mencken said, “ Democracy is the
theory that the common people know what they want and deserve to get it good and hard.”
 11

The Editor’s October 2008 Letter to Himself states, in part,

“Theodore P. Hill has found “serious mathematical errors” in the article … by [Au1-2-3] which
appeared in the December 2006 Notices and supplied analysis and detailed counterexamples
… which have been checked by an expert referee, verify the following mistakes…”

[ASIDE] The Editor’s Note also contains a summary of many of the errors, and Au1-2-3’s
acknowledgement of those errors – the same errors sent them on 7/26/07, denied by Au1-2-3
as “misconceptions” on 7/31/07, partially confirmed as having “merit” by Au2 on 8/27/07,
denied by Au1-2-3 as “without merit” on 10/20/07, partially confirmed by Ed2 on 10/26/07,
denied by Au1-2-3 as “exceptional cases” on 11/13/07, and confirmed by a referee on 6/13/08.
It also contains the new mathematical errors I had pointed out both to the Ed1 and to Au1-
2-3 (see 7/14/08), but not one of the essential counterexamples. And no retractions – in fact,
propagation of those errors is already in the mill…

“If two wrongs don’t make a right, try three.” (Author unknown)

May 2009

The Notices announces, without disclaimer, publication of a new book that contains essentially
all of the same errors as the original Au1-2-3 2006 article in the Notices.

Fall 2009

More than two years after first submitting them, I am still trying to publish the
Counterexamples.

AFTERMATH

At the end of his fascinating article, Trebino asks

“Okay, so the system is badly broken. How would I fix it?”

and he lists eleven suggestions. The scientific community does not exercise control over
individual scientists, nor should it, but the community does have a say in who edits its journals.
Regarding that fact, two of Trebino’s suggestions seem particularly relevant here:

Trebino #4:

No journal editor should be allowed to edit a Comment on a paper that he allowed to be

published. This is an obvious, unacceptable conflict of interest.

Trebino #11:

Finally, let’s face it: most journal editors are simply too arrogant and have lost sight of the
goal, which, it appears that I need to remind them of here, is to publish only truth.
Perhaps they could be required to take a course or two in humility.
 12

That would be a fine start. In the Real World, of course, that will not happen, so perhaps the
best the rest of us working scientists can do is to follow Trebino’s fine example, and share our
experiences and continue to question authority. In the words of my historian friend Professor
Hugh Hudson, “If we did not challenge accepted wisdom, if we didn’t challenge authority,” he
said, “we would still be arguing the world is flat.”