Ian Ayres and John Donohue have a strongly worded letter to the editor that they have apparently sent to the Post-Dispatch. First is my letter to the editor:

St. Louis Post-Dispatch, July 21, 2003

Valid gun research

On March 9, and again on July 12, the Post-Dispatch has published editorials critically discussing my research regarding concealed-carry. These pieces have made it appear as though the academic debate is just between myself and a couple of critics. That is not the case.

Academics who have published refereed research in academic journals showing that right-to-carry laws reduce violent crime include: Carlisle Moody, David B. Mustard, John E. Whitley, David E. Olson, Florenz Plassmann, Nicolaus Tideman, Tomas B. Marvell, Mark A. Cohen, Stephen G. Bronars, and William A. Bartley. While some other studies claim the laws produce no change in violent crime rates, among all the national studies that have been done, there is not a single refereed academic publication concluding that these laws produce a significant increase in violent crime.

The Brookings Institution study that the Post-Dispatch cites was not published in a refereed journal. A version of that paper recently appeared in the Stanford Law Review, a student-edited, non-refereed journal. That research finds a temporary small increase in violent crime, followed by a drop in crime. Yet next to that article in the same publication appears another study by Plassmann and Whitley, who examine three additional years worth of data and find "annual reductions in murder rates between 1.5 and 2.3 percent for each additional year that a right-to-carry law is in effect. . . . the total benefit from reduced crimes usually ranges between about $2 billion and $3 billion per year."

John R. Lott Jr.

Resident Scholar, American Enterprise Institute

Washington, D.C.

LOAD-DATE: July 21, 2003

Here is their response that they e-mailed to Glenn Reynolds (August 20, 2003, posted at 12:40 PM):

On July 21, 2003, researcher John Lott wrote a letter to the editor in which he tried to shore up support for his now discredited theory that state adoption of laws allowing citizens to carry concealed handguns will lower crime. Although he refuses to acknowledge this fact, we showed in a recent article published in the Stanford Law Review that when the coding errors in his own data set are corrected, his own regressions show no such drop in crime. Lott tries to distract the readers by stating that our study illustrating his coding errors was only published in a law review, rather than in a peer-reviewed journal. But since Lott knows that merely correcting his errors did eliminate his finding, it is dishonest for him not to concede the fact. How can we assert that Lott knows that we correctly identified coding errors in his data? Because he had put his data set on his own web page before we found his errors, and he has now gone in and quietly corrected the errors that we identified. Lott should put an end to the charade and acknowledge that his own most recently published regressions, when corrected, offer no support for the more guns, less crime hypothesis. As the latest, peer-reviewed, article on this topic by researchers Tomislav Kovandzic and Thomas Marvell states: “we find no credible statistical evidence that increases in permit rate growth (and presumably more lawful gun carrying) leads to substantial reductions in violent crime, especially homicide. Similar to Ayres and Donohue (2003), we find that our best, albeit admittedly imperfect, statistical evidence indicates that increases in permit rate growth may actually lead to slight increases in crime.”

Professor Ian Ayres, Yale Law School Professor John Donohue, Stanford Law School

On substance there are several points:

1) The 1977 to 2000 county level data set has been up on my web site since February. To make the entire process easily accessible for others, the regression files are also available. Anyone who wants to critically examine the data set is thus completely free to do so and can compare it to the corrected output and to the paper file, both of which can be downloaded from the site. The data set was corrected, and the results labeled "corrected" figures and tables have been up since April. With well over 70,000 observations and over a hundred variables available in the data set, we are dealing with a few hundred data entries that contained mistakes. That amounts to thousandths of one percent of the data entries.

I have previously discussed these issues on this web site (6/7/03): "There are also papers by Plassmann and Whitley in the same issue of the Stanford Law Review as Ayres and Donohue's piece (an earlier version of the Plassmann and Whitley paper with me as a coauthor is available here and see also the data and corrected results are available at www.johnlott.org ) . . . ."

I am not quite sure what to make of Ayres and Donohue's claim: "How can we assert that Lott knows that we correctly identified coding errors in his data? Because he had put his data set on his own web page before we found his errors, and he has now gone in and quietly corrected the errors that we identified." This is not what transpired. During a call that I made to Ian Ayres back in January, he asked me for the data, thus prior to me putting the data on the web, and I immediately sent it to him, along with all the regressions so that it would be very easy for them to check the research. This should be easy to verify through the editors at the Stanford Law Review and I should have documentation on fedexing him the data. They informed us of their points in their reply and the corrections were made as noted above on my web site.

As to the claim that "correcting his errors did eliminate his finding." The data used in the Plassmann and Whitley paper can be downloaded at www.johnlott.org and one can readily see from the corrected tables and figures that while there are some changes in the results, this statement is false. Florenz Plassmann also has a detailed response here. For an additional discussion interested readers can also go here.

Unlike Ayres and Donohue, I have endeavored to make the data readily available in a timely manner and to explain how it was constructed.

Updated paragraph 8/22 (minor edit on 9/19/03) Additional statement with respect to point 1: The estimates do change somewhat, but the basic point is still clear. Whether one uses the types of statistical analysis that Ayres and Donohue use for all their county level regressions or whether one uses the type of methodology that Plassmann argues very strongly for in the Stanford Law Review piece with John Whitley, you still get a drop in crime. Clustering with the least squares esimates does affect some of the results, but Ayres and Donohue do not use clustering for any of their own results and do not discuss Plassmann's statements about using Poisson regressions and the "major" problems attempting to employ a linear specification on nonlinear data. Despite the Poisson regressions being the primary estimates in the Plassman and Whitley paper they are ignored and they continue to show significant drops in violent crime rates after the law is passed. It is also not clear that applying STATA's clustering command to all counties within a state provides an adequate solution to the problem of cross-correlation. First, clustering by state requires that the law has the same affect across all counties within the state and that is obviously not true with right-to-carry laws. The vast majority of states adopting right-to-carry laws had may issues laws to begin with and there was a large variation in the rate at which permits were issued in those counties (e.g., with more rural less densely populated counties issue permits at a much greater rate).  In addition, while it is possible that error terms are correlated across jurisdictions within a state, the more important correlations may be among neighboring jurisdictions across state lines. (Ayres and Donohue's reply where they first use clustering does not state when they are using clustering or not so I have shown what happens when clustering is not used.) Professors Eric Helland and Alex Tobarrok paper also provides an interesting approach that deals with this problem in an novel way and shows that the results are significant even after these correlations are accounted for.

2) Despite their continuing claims to the press, Ayres and Donohue's own papers do NOT provide any statistically significant evidence that violent crimes increase. I suggest that interested people carefully read Plassmann and Whitley's paper. But let me draw the readers to a couple of points:

a) The "Hybrid" Estimates imply an initial increase in crime

Plassmann and Whitley write (p. 1360, Section D):