Below are accounts of two recent submissions of computational work to the Review of Economic Studies. I congratulate and thank them for the recent improvement they have made in dealing with computational work.
++++++++++++++++++++++++++++++++++++++++++++++
Don’t count on Editors knowing what is published in their journal
The Review of Economic Studies rejected a paper of mine that discussed the advantages of using continuous-time models instead of discrete-time models to analyze dynamic strategic interactions in oligopolistic markets. One of the few specific negative comments was that continuous-time models are not useful in econometric analysis because data comes at discrete times. The Review of Economic Studies is purportedly a leading journal on econometric estimation methods. Surely the Editor who rejected this paper (and who has published econometric work himself) is aware of the work on temporal aggregation going back 30 years. Did he think we were not aware of that work?
This is typical of how some journals often treat computational papers. They don’t know how to consider them seriously so they grab onto some claim that sounds right as a basis for rejection. This looks like such a case. It sounds reasonable to say that discrete-time data needs discrete-time models; after all, any analysis of apple data an apple market model. However, econometricians have worked to show that this is not true, working off the observation that integrating the infinitesimal generator of a continuous-time Markov Chain will produce the implied discrete-time Markov Chain.
My coauthor and I both contacted the Editor, asking him to explain how this comment could be taken seriously, and how the referee who made this comment could be taken seriously. The response to my e-mail was:
“You are entitled of course to think that I have made the wrong decision on your paper, but I would like to stress that neither I nor the Review are prejudiced against computational economics. I have used a lot of computations in my economics, I own your book and quite a few more on the topic, and so I definitely do not hold the subfield in contempt.”
This response is also typical in its misdirection. As you can see below, I never expressed any opinion on whether he made the wrong decision. I do not presume to know the objectives of the Review nor of the opportunity cost of pages. My comments were on specific details and aspects of the Editor’s comments, some of which puzzled me and some even appeared to contradict well-known results in econometrics. I made no request that he reconsider his decision; in fact, I sent my e-mail to him after the paper had been submitted elsewhere. If I presented this paper at a conference, I would expect a discussant to give some explanation for why he would say, for example, that discrete-time data requires discrete-time models. In this case, however, it was wrong to expect that of the Editor since he chose to make no explanation of his mathematical and statistical claims.
I found it amusing that he somehow thinks owning my book is a relevant response. I would gladly forego the royalties in exchange for a journal that would not waste my time by pretending to seriously consider my submissions.
I also was amused that he limited his claims to only owning my book and others.
Below is the full text of my e-mail to him.
============================================================================
Prof. [Editor]:
After reading through the correspondance on your consideration of “Avoiding the Curse of Dimensionality in Dynamic Stochastic Games” (RES 11002)., I have decided to comment on the attitude that REStud obviously has for computational methods and their use in economics.
In your rejection of our paper, you say
“At this stage of the editorial process I would normally look for evidence that the paper has significantly progressed towards a version that will be acceptable to the referees and to myself. Referee 1 notes clear improvements but is asking for a thorough rewrite; and Referee 2 has not been convinced by the changes in the paper.”
Let us recall what Referee 2 states. Most of his comments are speculations on what other, unspecified and unpublished, methods might do. The fact that you rely on undocumented claims from referees makes it clear that you have no interest in treating computation in a rigorous manner. Would you reject a paper on econometrics because someone speculated that “There are better ways to get a consistent estimator” but then not explain or document that claim? I doubt it, and the reason is because REStud takes econometrics seriously.
One of the few specific comments from Referee 2 was
“Moreover, I have never heard of IO problems with continuous time data (unlike finance), and it is rare to even see monthly or quarterly data.”
My coauthor sent you a message making the following points on this issue:
“Moreover, Referee #2 seems to confuse data availability with modeling when he/she claims that because data is yearly, “firms only make decisions every so often.” Clearly, how often firms’ decisions get recorded by some statistical agency has nothing to do with how often these firms make decisions. Indeed, Referee #2 acknowledges that “it is certainly true” that “firms make decisions often”. Taking the fact that these decisions get recorded once a year as evidence that a discrete-time model is more appropriate than a continuous-time model is nonsensical. After all, if decisions are made more or less continuously (and boards certainly meet more often than once a year and CEOs go to work on a daily basis), then the discrete-time model is plainly wrong. The correct approach is obviously to start with the model that best captures the underlying decision-making process and then “aggregate up” the predictions of the model to the frequency of the data.”
What is the relevance of this statement about data and the modeling of time? The only way to interpret this comment (even when considered in the full context in which it appears) is that discrete-time data cannot be useful in estimating continuous-time models. As an empirical economist you surely know that this claim is clearly contradicted by nearly 30 years of research on temporal aggregation. In fact, one of the more significant papers was published by Geweke in REStud. Since this comment appeared in a report which you clearly indicate played a role in your decision, it is natural to infer that you also agree with this assessment. The literature on temporal aggregation is well-known in the empirical literature, and your referee and you both tacitly claim to be experts on this matter by making such comments and relying on comments made by this referee. Therefore, the only conclusion I can draw is that you and your referee thought that [my coauthor] and I are just theorists who know nothing about dynamic estimation and figured that we would actually take this to be a serious comment.
Later in your letter, you say:
“And like Referee 2, I thought you relied too heavily on the 0.925 discount factor in PM, surely driven by considerations that do not weigh in the same way a decade later.”
This is a new issue, not mentioned in the first round of reports and letters. How is this relevant? Do you think that the relative advantages of our method go away with a 0.95 or 0.98 discount rate? On what basis do you believe that the past decade in hardware progress has made this issue less important? Our main point is that using a continuous-time model will improve the speed of any analysis by many orders of magnitude. How does an increase in the absolute speed of computers affect the relative speed of alternative modeling approaches? Also, as we clearly demonstrate in our paper, the contraction rate of the Gauss-Seidel iteration procedure is not simply related to the discount rate. This is a game, not dynamic programming, a fact clearly shown by our analysis of convergence rates.
You seem to assert that this new issue was influential in your decision, but by not asking for another revision you also makes clear your contempt for this research, and the development of numerical methods in general. It clear your attitude is “I think your use of 0.925 is not good, and I don’t care what the actual facts are.” This along with the fallacious and insulting reference estimating continuous-time models with discrete-time data makes clear your hostility to computational methods.
In your reply to my coauthor, you said “I will not comment on the quality of Referee 2’s report.” Of course, you do not need to. The fact that you rely on any of his criticisms speaks clearly about your view concerning Referee 2.
You also say “I also know that editors have their own idiosyncratic factors, and that reasonable people may agree to disagree.” How can reasonable people “disagree” on the comment about temporal aggregation? You raise the issue of the discount factor; is this an example of “disagreement”? Clearly not since you have no interest in seeing the facts. Disagreements are common, but the unwillingness to base discussions on clear and precise quantitative statements, and instead relying on speculation is not reasonable; it just shows clear contempt for the idea that these issues should be discussed in a serious manner.
Sincerely,
Dr. Kenneth L. Judd
Paul H. Bauer Senior Fellow
Hoover Institution
++++++++++++++++++++++++++++++++++++++++++++++
Go away!! Too much math!!
A computational paper on resolving uniqueness and existence issues in economic models was recently submitted to Review of Economic Studies. The Review summarily rejected it for being too technical.
I commend the Review for this improvement on their handling of computational work. I wish the previous editorial board had been so honest and clear about their dislike of technical material when my paper was originally submitted.