John R. Lott - General Discussion on 1997 and 2002 surveys

Readers who would like to search for other details on these issues are advised to go to my other website and use the search function available at the top of the page.

With some recent attacks on me in a variety of places from the Washington Post to the Chicago Tribune to numerous other places, I thought that I should send out some responses for those who might be interested.

A) There have been many claims that I didn't conduct a survey in 1997 that was reported in one sentence on page 3 of my book, More Guns, Less Crime. In the attached file and also here, I present the complete statements from the people listed below.

1)Professor David Mustard confirms that I discussed doing the survey together with him many times during 1996; that he is "fairly confident" that he knew that I had completed the survey by myself in 1997; and that he personally spent a long time in August and September 1997 helping me replace some of the data lost in the July 1997 hard disk crash.

2) A statement by John Whitley, who at the time was a graduate student at the University of Chicago and is now at Adelaide University in Australia, saying that he believes that he met the students that conducted the survey. He also confirms my hard disk crash.

3) David Gross, a former assistant city prosecutor from Minneapolis, provides evidence that he was interviewed in my survey in 1997. There were only two other defensive gun surveys after 1995 and neither was done in 1997. Both those other surveys were done by Hemenway and were extremely different from the survey that I gave.

4) Geoff Huck, the editor at the University of Chicago Press who handled More Guns, Less Crime, remembers that I lost the computer file for my book in the computer crash and that part of the book was permanently lost, though it has been six years and he can't remember what part that was. While he no longer works for the Press and does not have his work e-mails, he does have one e-mail on his home computer from the end of July 1997 that helps verify the loss of material for my book.

5) Multiple academics also confirm my hard disk crash. Many of these academics were involved in co-authoring research with me and themselves suffered from this loss because it affected our joint research. They don't remember all the other data that was lost, but they can confirm that the hard disk crash was catastrophic and that I lost all the data that I had. All these statements are backed up by memos in the attached file. Eight academics wrote letters to the Post, but I could have gotten many others.

The reference to the original survey involves one number in one sentence in my book. Compared to the 98 percent number there was an earlier survey by Kleck that found 92 percent of defensive gun uses involved brandishing and warning shots and because the survey was asking people about events that occurred over a long period of time it is likely that it over emphasized more dramatic responses. (My number that is directly comparable to the 92 percent estimate is about 99 percent.) My point in the book was that defensive gun use rarely involves more "newsworthy" events where the attacker is killed and either survey would have made the general point. A general discussion of the different methodologies is provided here (

What is usually not mentioned is that the survey results that I used were biased against the claim that I was making. The relevant discussions in both of my books focus on media bias and the point was that the lack of coverage of defense gun uses is understandable if most uses simply involve brandishing where no one is harmed, no shots fired, no dead bodies on the ground, no crime actually committed. If others believe that the actual rate of brandishing is lower and I had used the results of Kleck, it becomes MORE difficult to explain the lack of news coverage of defensive gun uses. The two short discussions that I have on this issue in my two books thus choose results that are BIASED AGAINST the overall point that I am making, that the media is biased against guns.

One point should be made clear. While I lost all my data for my book, David Mustard as well as others to a lesser extent helped me replace all the crime data so that many dozens of academics around the country have been able to replicate every single regression in the book. With respect to the survey data that was lost, I obviously could not go to a primary source to replace that data, but as part of my new book that should be out in a couple of weeks (The Bias Against Guns), I replicated the survey and obtained fairly similar results. Science involves replication. That survey data has been available to any academics who have asked for it. The Washington Post's statement (2/11) that "a 1997 survey the researcher purportedly did to support claims in his provocative book, 'More Guns, Less Crime,'" is incorrect. We are talking about one number in one sentence.

Because of claims that no one would finance this research themselves, I have made my business and personal income tax forms from 1997 available to some other academics. Many statements have also been made about how long it would have taken me to do the survey. I designed the survey so that about 90 percent of those who took the survey only had to answer three very short questions (one question about whether they have been threatened by violence during the last year and then two short demographic questions). For those people, the survey would often take only 30 seconds. Only one percent of those surveyed were asked a maximum of seven question and even then the questioning lasted just a few minutes.

B) Did I Attribute the 98 Percent Brandishing Number to Others? No

Apparently, some credence is being given to the claim that I have attributed the 98 percent brandishing estimate to others instead of myself. Some are taking this as evidence that I never conducted the survey. Yet, the fact is I never attributed this number to anyone else other than myself. It is claimed that I attributed this number to Gary Kleck on one occasion as well as the Los Angeles Times, Gallup, and Peter Hart during a couple of other times. Attachments entitled "Did I Attribute the 98 Percent Brandishing Number to Others?" and "Cases where 98% number was used" provide a detailed discussion of these issues. However, two brief statements are useful here:

The Independence Institute web site posted a piece that contained the one reference to Gary Kleck¹s work.

------ Forwarded Message From: "Dave Kopel" <>

Date: Sun, 9 Mar 2003 13:07:49 -0700

To: <cut>

Subject: Re: FW: A quick question. John Lott

I've got no specific recollection of editing the piece, but the evidence seems to indicate that attributing the 98% figure to Kleck was an error by the Independence Institute, rather than an error on the author's part.

Dave Kopel

------ End of Forwarded Message

The second case of false attribution is that I claimed that the Los Angeles Times, Gallup and Peter Hart were the source of the 98 percent number. Where people get the claim is by combining two sentences in a Chicago Tribune op-ed that I had in 1998 (the same op-ed was also republished, for example, in the Washington Times).

The relevant passage from the op-ed reads:

"Other research shows that guns clearly deter criminals. Polls by the Los Angeles Times, Gallup and Peter Hart Research Associates show that there are at least 760,000, and possibly as many as 3.6 million, defensive uses of guns per year. In 98 percent of the cases, such polls show, people simply brandish the weapon to stop an attack."

If the reference in the second sentence had been to "these" polls and not "such" polls, I would think that the critics would have a much better argument. Instead, I view "such polls" as merely referring back to this type of polls and not those specific polls. Still there is admittedly an error in using the plural. The most plausible explanation is that I was describing what findings had been generated by the polls, in other words I was viewing them in general as a body of research.

C) As to other claims by Al Hunt on CNN that I have not given out data for the work that I did on the Florida election and for concealed handgun research are plain false. A list of just some of the critics to whom I have given those data is also provided in the attached collection of memos.

D) The Chicago Tribune story a couple of weeks ago adds to the list and references an academic who "wonders" whether I had really replicated the survey again in 2002. I have provided a list of student names who did the survey, provided the survey questions and methodology, shown the telephone records, and provided the data. If anyone would like to see any of this material, please e-mail In addition, James Knowles, a research assistant at AEI, sent the Tribune a letter, though it wasn't published:

From: "James Knowles" <>

Date: Mon Feb 17, 2003 7:12:38 PM US/Eastern

To: <> Cc: "James Knowles" <>

Subject: Re: *Another Misfire in the Academic Shootout on Guns* (Feb. 14)

To the Editor:

Ron Grossman closes Friday's news article "Another Misfire in the Academic Shootout on Guns" (Feb. 14) casting doubt as to whether John Lott Jr.'s 2002 survey on defensive gun use actually occurred. Grossman quotes Florida State criminologist Gary Kleck saying the Lott staff would have needed "500 evenings to do 2,000 interviews". Kleck's calculation assumes that any one staff member might conduct two surveys an evening. As supervising research assistant for the Lott survey, I believe that Messrs. Kleck and Grossman have misunderstood the nature of our work. Unlike Kleck's previous surveys, the Lott survey consisted of a maximum of seven questions asked of any respondent. With an average of five callers a night completing about 25 surveys each, we ended up with just over a thousand respondents in two weeks of calls. We have produced telephone records and disclosed the names and email addresses of surveyors to support the integrity of the survey. With that said, I believe the debate would be most intelligently furthered by moving past questions of whether these results were fabricated.


American Enterprise Institute

Washington, D.C., Feb. 17, 2003

(202) 862-7183

E) With respect to the charges in the Washington Post that David Mustard now believes that the evidence from the 1990s overturns our original work, David wrote the following to the Post though like all the other letters, it was not published:

Dear Letters Editor:

In his evaluation of the book "Evaluating Gun Policy", Saul Cornell stated, "John J. Donohue challenges economist John Lott's contention that states issuing permits for carrying concealed weapons have seen significant reductions in violent crime. In a commentary on this essay, David B. Mustard, Lott's original co-author, quibbles with Donohue but concedes that data from the 1990s show a rise in crime in those states that adopted concealed-carry laws."

I would like to correct this misrepresentation of my analysis, in which I write, "Unfortunately, many of [Donohue's] criticisms have already been addressed extensively in the literature" and that Donohue's own data show that "the passage of the law [that allows law-abiding citizens to carry concealed weapons] is associated with sharp decreases in murder, rape and robbery."


David B. Mustard

Terry College of Business

University of Georgia

Athens, GA 30602

Phone: (706) 542-3624

Other misleading charges about David's changing views on our research have also been made in other publications. If you would like a discussion of those, it can be provided. Unfortunately, absolutely none of the various supportive letters sent to the various newspapers have been published.

F) Response to the New York Times attack:

------ Forwarded Message

From: John Lott <>

Date: Tue, 04 Mar 2003 21:01:39 -0500

To: Letters New York Times <>

Subject: Pity the Poor Gun, 3/2/03, Section 7, Page 31 (earlier version of letter transposed numbers in date)

Dear Letters Editor:

Your comic strip would be funny if it didn't contain so many errors (Pity the Poor Gun, 3/2, Section 7, Page 31). 1) My hard disk crash is extremely well documented. 2) There is evidence of the survey's existence from both a couple of professors and from a survey participant. 3) I never attributed the 98% figure to the Los Angeles Times, Gallup, or Peter Hart. 4) The claims you raise are over one number in one sentence in my book More Guns, Less Crime. ALL the crime data in the book was replaced after the hard disk crash and has been given to dozens of academic researchers. Every single regression measuring the impact of guns on crime has been redone by others.

I should also mention that I replicated my lost 1997 survey in 2002 and got similar results--to be presented in my new book, The Bias Against Guns, when it is released in a couple weeks. Incidentally, only two other surveys have been done on this question since 1990 and one found that in 92% of all defensive gun uses either brandishing or firing a warning shot was sufficient to scare off the attacker.


John R. Lott, Jr.

Resident Scholar

American Enterprise Institute

1150 17th Street, NW

Washington, DC 20036

(202) 862-4884

------ End of Forwarded Message

This letter was never published.

Finally, a response to the claims about the use of the pseudonym in internet chat rooms is available at the very end of the attached "memos" statement.

Memos regarding 97&02

Did I attribute 98% to

Cases where 98% number was used

Home (description of book, downloadable data sets, and discussions of previous controversies)

Academic papers:

Social Science Research Network

Book Reviews:

For a list of book reviews on The Bias Against Guns, click here.

List of my Op-eds

Posts by topic

Appalachian law school attack

Baghdad murder rate

Arming Pilots

Fraudulent website pretending to be run by me

The Merced Pitchfork Killings and Vin Suprynowicz's quote

Ayres and Donohue

Stanford Law Review

Mother Jones article


Craig Newmark

Eric Rasmusen

William Sjostrom

Dr. T's

Interview with National Review Online

Lyonette Louis-Jacques's page on Firearms Regulation Worldwide

The End of Myth: An Interview with Dr. John Lott

Cold Comfort, Economist John Lott discusses the benefits of guns--and the hazards of pointing them out.

An interview with John R. Lott, Jr. author of More Guns, Less Crime: Understanding Crime and Gun Control Laws

Some data not found at

Updated Media Analysis of Appalachian Law School Attack

Since the first news search was done additional news stories have been added to Nexis:

There are thus now 218 unique stories, and a total of 294 stories counting duplicates (the stories in yellow were duplicates): Excel file for general overview and specific stories. Explicit mentions of defensive gun use increase from 2 to 3 now.

Journal of Legal Studies paper on spoiled ballots during the 2000 Presidential Election

Data set from USA Today, STATA 7.0 data set

"Do" File for some of the basic regressions from the paper