Back to American Economic Review
Although computation was not mentioned by the Editor, it is common for Editors and referees to come up with strange criticisms when dealing with computational papers. In this case, the curious definition of what papers are “in the literature” bears notice. My working assumption in being an editor or referee is that “the literature” means published work, not unpublished work. Without that rule, it becomes very difficult to resolve issues of priority and gives advantages to those who are able to publicize their work in major working paper series and in seminars attended by editors of the journals. In this case, an NBER working paper by a member of the AER Board of Editors was cited as being “in the literature”. The AER and AEA are not concerned by the issues raised by this practice.
I know that this is far from the worst case of a paper being rejected partly because of similarity to unpublished work by individuals closely associated with the editorial board of a journal. This is a case that I am free to discuss.
I have put some points in italics and brackets.
===============================
Rejection letter from EDITOR:
“To be sure, the referees agree that the paper is well executed and adds to the finance literature. However, they all argue that the basic implications for bond portfolios have appeared in the literature (e.g., [two published papers], and Lustig/Krueger)”
===============================
Email from me to EDITOR,
I was just forwarded your decision regarding Manuscript ID AER-2008-0766. I was disappointed with the fact that the referees did not read the paper carefully, but I have learned that there is no point in arguing over substance when it comes with dealing with journals like AER and Econometrica.
However, one of the comments you made in your letter really displayed bias that most journals would not admit to. You asserted that the Lustig/Krueger paper has “appeared in the literature.” This paper has, according to the websites of the authors, has not been published anywhere. In particular, Krueger refers to the July 24, 2008, version as being submitted for publication. Therefore, it appears that our paper was submitted to AER before that paper was accepted for publication anywhere, and that the AER decision also likely predates any publication acceptance for Krueger/Lustig.
Your use of the word “literature” apparently is not limited to “accepted for publication”. What is it about that paper that makes it “part of the literature?” Its appearance as an NBER working paper? If so, then I trust that AER will not publish anything that is not clearly superior or different from any of my NBER working papers. Or was it some other fact that lead you to say it was “part of the literature”?
I am, of course, being sarcastic. I know that AER would not say that NBER working papers count as prior publications for peer review purposes. The obvious truth here is that we submitted a paper to AER, but AER rejected it partly because Lustig and Krueger are friends of the AER clique and we aren’t.
This kind of behavior, which is corrupt in my opinion, is rampant in economics. I am glad to meet an editor that is so honest about it.
Ken
===============================
Dear Ken,
I went back and read my decision letter. It is pretty clear that I was giving you a summary of the feedback I received from the three referees. It is also clear that I mentioned three other papers that have appeared in print. I would never use as a sole critieria the fact that another paper is in working paper form and never have.
Also for the record:
1. I have never met Lustig so I would hardly describe him as a friend.
2. The only time I ever met Krueger was when he gave a seminar here three or four years ago, so I would also hardly describe him as a friend.
I’ve had my share of similar experiences, so I appreciate how you feel. But the decision, right or wrong, had nothing to do with any kind of favoritism,
Best Regards,
EDITOR
[It is curious that he has had similar experiences and “appreciates” how I “feel”. Also, my e-mail was not about my or anyone’s “feelings.” Editors should spend more time discussing editorial policies than making pointless comments about “feelings.”]
[It is also curious how the EDITOR insists that he is not a friend of either coauthor of the working paper. Of course, I never asserted that he was a friend of them, just that the “AER clique” was friends. The fact that Krueger is on the Board of Editors of AER by definition makes him a friend of the AER clique.]
[The assertion that he would “never use as a sole criteria the fact that another paper in in working paper form” reinforces my point. Surely he would use a published paper containing the same material as a sole reason for a rejection. He apparently agrees that there should be a distinction. However this assertion means nothing since editors and referees can always find multiple problems which they can use to embellish the main reason for a rejection.]
===============================
Email from me to EDITOR,
The mere mention of unpublished work in your letter indicates that you gave it weight, whereas unpublished work should be given zero weight. You have no control over what referees say, but you choose how to summarize their reports and you are free to ignore comments you view as irrelevant.
You may not be a friend of Lustig or Krueger, but what about others? If someone on the Board of Editors looked at this paper, he could have been a friend. Of course, you cannot give me any information regarding that. My reference to AER was clearly a reference to the journal, not just to you.
I might have ignored this if there was some content in the referee reports. Lacking that, I was not going to ignore this procedural matter.
Ken
===============================
Email from me to MANAGING EDITOR,
Today I received a rejection letter from Mark Gertler. One reason for the rejection was that the “basic implications … have appeared in the literature” and listed an unpublished NBER working paper as one of those papers that “have appeared in the literature.” When I contacted Gertler about this, all he said was that the unpublished paper was not the sole reason for the rejection. His response makes it clear that he considers it proper to consider unpublished work when evaluating a submission to AER.
I regard that as an outrageous practice. My understanding has been that journals are to limit their consideration of a submission to the published literature, and that has been the standard I have used. Without this rule, it becomes very difficult to sort out issues of priority and creates an unreasonable burden on authors as they try to describe how their paper contributes to “the literature.” Moreover, referring to an NBER working paper being “in the literature” creates an improper advantage for members of the NBER, a group whose membership is determined more by who they know than by the quality of their work.
We know that Mark Gertler finds nothing wrong with comparing submissions with unpublished work. That being the case, I will specifically ask that he not handle any future submission of mine, and will advise friends of his approach to editing. How many other Editors of AER use unpublished papers in their considerations of submissions? Is this a general policy of AER? If so, what is the justification?
Sincerely,
Ken Judd
===============================
Reply from AER Editor guy
[We now have the AER “policy” – it is up “to the discretion of the Coeditors, to be decided on a case-by-case basis.” Of course, we were never told what it was in this case that made it appropriate to consider unpublished work. Was it the fact that it was an NBER working paper? one author is an Ivy League professor? one author is on the Board of Editors of the AER? Some combination of these factors? Some other factors? This discretion leads to other questions. If Krueger/Lustig is submitted to AER, will my paper be considered “in the literature”? ]
===============================
[I cc’d the last e-mail from AER to some AEA officers. One replied, saying he “can’t get involved in individual papers.”Obviously I can’t get involved in individual papers.
[I am well aware that AEA officers will not want to get involved in issues regarding judgments about how original or how important a paper is. However, this issue is a procedural one. If he can’t get involved in this procedural matter for one paper, then why would he intervene if this happened ten times, or a hundred times? This is clearly an endorsement of the policy of unlimited discretion when it comes to judging which unpublished papers are “in the literature” and which ones are not.]