Comparison of the NAS Panel Report on clustering and what David Mustard and I wrote in our original paper.

Last year charges flew over the use of clustering in the debate between myself and Ayres and Donohue. Well, whatever the other aspects of the debate and whatever other problems that I have with the National Academy of Sciences report on firearms, their report should make it fairly clear that I was correct or at the very least the extreme claims that were being floated last year were unfounded.

From my original paper with David Mustard, p. 18, fn. 46

One possible concern with these initial results arises from our use of an aggregate public policy variable (state right-to-carry laws) on county-level data. See Bruce C. Greenwald, A General Analysis of the Bias in the Estimated Standard Errors of Least Squares Coefficients, 22 J. Econometrics 323–38 (August 1983); and Brent R. Moulton, An Illustration of a Pitfall in Estimating the Effects of Aggregate Variables on Micro Units, 72 Rev. Econ. & Stat. 334 (1990). As Moulton writes: ‘‘If disturbances are correlated within the groupings that are used to merge aggregate with micro data, however, then even small levels of correlation can cause the standard errors from the ordinary least squares (OLS) to be seriously biased downward.’’ Yet, this should not really be a concern here because of our use of dummy variables for all the counties, which is equivalent to using state dummies as well as county dummies for all but one of the counties within each state. Using these dummy variables thus allows us to control for any disturbances that are correlated within any individual state.

From the National Academy of Sciences report on firearms, p. 138

However, investigators reporting cluster-adjusted standard errors do not formally explain the need for these adjustments. These adjustments, in fact, are not supported in the basic models specified in Equations 6.1 and 6.2. Instead, those who argue for presenting clustered standard errors often cite Moulton (1990) as the source of their belief that adjustments are needed. Moulton considered a model in which there is an additive source of variation (or additive effect) that is the same for all observations in the same cluster. He showed that ignoring this source of variation leads to standard errors that are too low. Investigators who make clustering corrections usually consider the counties in a state to constitute one of Moulton’s clusters and appear to believe that the absence of state-level additive effects in their models causes standard errors to be too low. The models estimated in this literature, including those of Lott and his critics, typically contain countylevel fixed effects (the constants gi in equations 6.1 and 6.2). Every county is always in the same state, so, any state-level additive effect simply adds a constant to the gi’s of the counties in that state. The constant may vary among states but is the same for all counties in the same state. The combined county- and state-level effects are indistinguishable from what would happen if there were no state-level effects but each gi for the counties in the same state were shifted by the same amount. Therefore, state-level effects are indistinguishable from county-level effects. Any state-level effects are automatically included in the g’s. There is no need for adjustments for state-level clustering.

_____________ Finally, note that the state level estimate also produce statistically significant estimates and there is no issue of clustering in that case. In addition, iif the data is threated as count data, the results are unaffected by the use of clustering (see Plassmann and Whitley).