Economics is ignoring the potential of modern computational technologies and related mathematical methods. This website will give examples of the obstacles faced by those who want to bring these tools to economics, and address those who are creating those obstacles.
I have chosen to focus on the role that journals have played and make frequent reference to referee reports and editor letters I have received. I focus on editorial correspondence since I want to document my comments. This is a limited set of data, but the absence of other voices is no reason to dismiss these cases as unusual. It is obvious that most people have professional concerns that keep them from publicly challenging the leaders of the economics profession. Therefore, I have not asked others to share documentation regarding their experiences.
Some may not like the tone of my comments. To them I ask what would your tone be if
1. A journal lists four problems with your paper, tells you that it does not think that you have the ability to address these problems, and would not publish the paper as an article even if you successfully dealt with the criticisms.
2. The Co-Editors of a journal declared that there were no economists in your field that “met our requirements” for being an associate editor.
3. A journal rejected your paper because it was similar to other papers, including a two-year old unpublished working paper by one of their Associate Editors.
4. A journal Co-Editor discusses your student’s paper (which, by the way, both exposes the Co-Editor’s errors and demonstrates a superior alternative), asserts that his using state-of-the-art optimization software is like using a “magical black box,” and complains about the absence of a convergence theorem despite the paper’s clear demonstration that its method was quadratically convergent.
Many have told me that this is futile, that they have had the same or worse experiences, and that they have learned that there is no point challenging the editorial boards of major journals. Even if this is futile, it will give warning to others and prepare them for what to expect when they submit computationally intensive and novel work to those journals. In fact, I am doing those journals a valuable service since their actions show that they do not want to treat computational work seriously.
The real target of this website, however, is the general disdain for the idea that economists can and should make use of modern computational tools and methods. Students are given experience with sophisticated econometrics and statistics packages, but have little exposure to computational methods more generally. Few American economics departments make any effort to train their students in numerical methods, and in most cases this training is focused on a particular field of economics. There are individuals who could give students the kind of training they need and deserve, but the top departments (with one notable exception) have made no effort in acquiring their services.
The result is obvious. Suppose that an economist declared “I don’t trust TSP or any other econometrics package because they are black boxes ” or “I don’t trust RATS because it was written in Fortran and magically converted into computer commands by a ‘black box’ compiler”. I suspect that such a declaration would be met with howls of derisive laughter. However, there is no such reaction when essentially the same statement is made concerning the modern computational software that some are trying to bring to economics, software that is far more transparent and accessible than any compiler code.
This was exactly the case at a recent conference where an Econometrica Co-Editor discussed a paper applying standard math programming software to econometric estimation. This was not the usual discussant presentation but rather a conference where the discussant presented the paper. It was his job to fairly summarize the contribution and content of the paper, as well as make comments. He noted, correctly, that the authors used KNITRO, but then commented that KNITRO was a “magical black box”, clearly indicating that he was not comfortable with this. Why is he uncomfortable with KNITRO? Because he does not understand the mathematics and algorithmic details behind this “magical black box.” I agree that one should not use software if one does not understand the underlying math. We disagree in how one should respond to this lack of understanding. When I don’t understand some program, I study the related math, read about the convergence properties of the algorithm, and learn what kinds of problems for which the algorithm is an appropriate tool. What was the response of a Co-Editor of Econometrica (the leading journal in economics) to his lack of understanding of KNITRO? Did he say that he was going to learn about the IP and SQP algorithms implemented in KNITRO? NO! Did he say he was going to find out what kinds of problems KNITRO would solve? NO! Did he say that he was going to study the literature on IP and SQP methods to learn their convergence properties? NO!
His conclusion encapsulated the problems that computational work constantly faces. He ended by comparing this paper’s work with his own Nested Fixed-Point approach which uses Nelder-Mead. He admitted that this new method may be faster than NFP, but asserted that “there was no convergence theorem” in the paper. While this was true in some narrow literal sense, these conclusions were very misleading, ignoring the facts that the paper (i) pointed to the large mathematical literature showing that there are no good convergence properties for Nelder-Mead, (ii) provided new results documenting the slow, perhaps very slow, convergence of NFP, and (iii) stated clearly that the paper’s approach “is quadratically convergent, faster than the linear rate of the contraction mapping that is the NFP inner loop” and provided many references to the mathematics literature on convergence rates of modern constrained optimization methods, making it senseless to include any convergence theorem in the paper.
(Note: I like to document my characterizations of what other people say as much as possible. I have asked this Econometrica Co-Editor to send me his slides so that I could check that my notes are correct. He has refused, implying that he wants me to rely on my notes.)
This is not an isolated incident reflecting one individual’s bad day. This is typical for my conversations with Econometrica Co-Editors and Associate Editors, and correspondence concerning submissions of computationally intensive papers. The only rational conclusion I can draw is that they do not know basic numerical analysis, don’t want to learn the material, and don’t support expanding numerical expertise in economics. The problems are not unique to Econometrica; I have had the same experiences with Editors of the other leading journals. Some may take issue with calling Econometrica, AER, and REStud “leading” journals. The fact is that they are the leading journals, particularly since publishing in them is given such high weight in promotion and hiring decisions. The real issue is whether they are leading the profession in a good direction.
After his discussion, this Econometrica Co-Editor proudly pointed out to me that his department has a course on computational methods. I took a look at the course material. What do they teach at his department? “Use the simplest possible methods”, “Use one-dimensional algorithms as much as possible,” and “Avoid black boxes.”
I have two responses to the quest for simplicity and the fear of black boxes.
First, the physicist who told us to “make everything as simple as possible” gave us the field equations for general relativity.
Second, all boxes are black until you open your eyes and turn on the lights.
In this website, I report various conversations and correspondence I have had with journal editors and other leading economists, but I do not attach names to these accounts in order to minimize the notion that these are personal attacks. I do refer to groups of individuals, such as editorial boards of journals and officers of societies. I understand that unanimity is unlikely in group decision making, so I do not ascribe a group’s actions to each individual in that group. Furthermore, any individual is a member of these groups for a short time. However, these groups do seem to have patterns and reputations, and individual scholars have to deal with these organizations in a continuing manner, so it is appropriate to name the groups. If I have made any factual errors, please tell me so that I can correct them. Any correspondence regarding this website and its contents should be sent to kenneth.judd@gmail.com.
=============================
Subpages and topics:
Econometrica: Econometrica has the reputation of being the leading technical journal in economics. I have collected evidence concerning its expertise on numerical methods.
Economists vs Computation: A collection of comments and observations illustrating economists attitude towards computational methods
American Economic Review: Occasionally people do submit computational material to AER.
Review of Economic Studies: Examples of how REStud deals with computational work.
Games and Economic Behavior: An example of how GEB treats computational work.
Europe: Here’s an opportunity! Take it! European economists are less antagonistic to computational methods and encourage the development of computational expertise in economics.
Long Live Ludd! A fanciful account of an economics conference on computational methods preferred by most economists’.