Comparison of the NAS Panel Report on clustering and what David Mustard and I wrote in our original paper.


Last year charges flew over the use of clustering in the debate between myself and Ayres and Donohue. Well, whatever the other aspects of the debate and whatever other problems that I have with the National Academy of Sciences report on firearms, their report should make it fairly clear that I was correct or at the very least the extreme claims that were being floated last year were unfounded.

From my original paper with David Mustard, p. 18, fn. 46

One possible concern with these initial results arises from our use of an aggregate public policy variable (state right-to-carry laws) on county-level data. See Bruce C. Greenwald, A General Analysis of the Bias in the Estimated Standard Errors of Least Squares Coefficients, 22 J. Econometrics 323–38 (August 1983); and Brent R. Moulton, An Illustration of a Pitfall in Estimating the Effects of Aggregate Variables on Micro Units, 72 Rev. Econ. & Stat. 334 (1990). As Moulton writes: ‘‘If disturbances are correlated within the groupings that are used to merge aggregate with micro data, however, then even small levels of correlation can cause the standard errors from the ordinary least squares (OLS) to be seriously biased downward.’’ Yet, this should not really be a concern here because of our use of dummy variables for all the counties, which is equivalent to using state dummies as well as county dummies for all but one of the counties within each state. Using these dummy variables thus allows us to control for any disturbances that are correlated within any individual state.

From the National Academy of Sciences report on firearms, p. 138

However, investigators reporting cluster-adjusted standard errors do not formally explain the need for these adjustments. These adjustments, in fact, are not supported in the basic models specified in Equations 6.1 and 6.2. Instead, those who argue for presenting clustered standard errors often cite Moulton (1990) as the source of their belief that adjustments are needed. Moulton considered a model in which there is an additive source of variation (or additive effect) that is the same for all observations in the same cluster. He showed that ignoring this source of variation leads to standard errors that are too low. Investigators who make clustering corrections usually consider the counties in a state to constitute one of Moulton’s clusters and appear to believe that the absence of state-level additive effects in their models causes standard errors to be too low. The models estimated in this literature, including those of Lott and his critics, typically contain countylevel fixed effects (the constants gi in equations 6.1 and 6.2). Every county is always in the same state, so, any state-level additive effect simply adds a constant to the gi’s of the counties in that state. The constant may vary among states but is the same for all counties in the same state. The combined county- and state-level effects are indistinguishable from what would happen if there were no state-level effects but each gi for the counties in the same state were shifted by the same amount. Therefore, state-level effects are indistinguishable from county-level effects. Any state-level effects are automatically included in the g’s. There is no need for adjustments for state-level clustering.

_____________ Finally, note that the state level estimate also produce statistically significant estimates and there is no issue of clustering in that case. In addition, iif the data is threated as count data, the results are unaffected by the use of clustering (see Plassmann and Whitley).

Home (description of book, downloadable data sets, and discussions of previous controversies)

Academic papers:

Social Science Research Network

Book Reviews:

For a list of book reviews on The Bias Against Guns, click here.

List of my Op-eds

Posts by topic

Appalachian law school attack

Baghdad murder rate

Arming Pilots

Fraudulent website pretending to be run by me

The Merced Pitchfork Killings and Vin Suprynowicz's quote

Ayres and Donohue

Stanford Law Review

Mother Jones article


Craig Newmark

Eric Rasmusen

William Sjostrom

Dr. T's

Interview with National Review Online

Lyonette Louis-Jacques's page on Firearms Regulation Worldwide

The End of Myth: An Interview with Dr. John Lott

Cold Comfort, Economist John Lott discusses the benefits of guns--and the hazards of pointing them out.

An interview with John R. Lott, Jr. author of More Guns, Less Crime: Understanding Crime and Gun Control Laws

Some data not found at

Updated Media Analysis of Appalachian Law School Attack

Since the first news search was done additional news stories have been added to Nexis:

There are thus now 218 unique stories, and a total of 294 stories counting duplicates (the stories in yellow were duplicates): Excel file for general overview and specific stories. Explicit mentions of defensive gun use increase from 2 to 3 now.

Journal of Legal Studies paper on spoiled ballots during the 2000 Presidential Election

Data set from USA Today, STATA 7.0 data set

"Do" File for some of the basic regressions from the paper