Supplemental information

Claim that survey data that I conducted in early 1997 was fraudulent. A computer hard disk crash lost the data for that survey, and people have claimed that the survey was a fraud.

Overview: A) The survey was redone in 2002 and the redone somewhat smaller survey produced similar results. That survey data was available at www.johnlott.org.
B) The survey results in the single paragraphs in the two books where I have referenced this survey data was biased against the claim that I was making. I argued that the simple defensive brandishing or warning shots are not news worthy. The higher the rate of defensive brandishing or warning shots, the easier it is to explain why the media is not biased when it doesn't cover most defensive gun uses. If I wanted to show that the media was more biased, I should have used the surveys with lower defensive brandishing rates. I have also explained why the length of the time people are asked to recall events over can explain the difference in the four surveys on brandishing that have been done over the last twenty years (two designed by me and two by Gary Kleck).
C) Two people who took the survey have said that they took it. John Hamilton, a retired private detective in Tennessee, was interviewed by Professor Jeff Parker at George Mason University. Parker also interviewed Hamilton's sister whom Hamilton told contemporaneously when he took the survey that he had taken it. The other person who has said that he took the survey was David Gross, a former prosecutor in Minnesota. James Lindgren found David Gross credible but thought that Gross might have taken a different 1996 survey, but Gross's statements as well as the survey data from the 1996 survey indicate that Gross took my 1997 survey (e.g., students conducting it, the many few questions involved, the questions asked). The data from the 1996 survey is available from me or from the ICPSR under Hemenway's name. Other people were able to confirm various other aspects, such as the timing of when the survey was done and that I talked to people at the time of the survey. I have also supplied my tax records from 1997 to Joe Olson a tax law professor and other professors that show large payments for research assistants. I have also looked at the 1999 survey data by Hemenway and it is clear based on the answer give for survey respondents in Tennessee and Minesota that neither Hamilton nor Gross could have taken that survey either.
D) As shown below, many people who were working with me on other projects knew of this hard disk crash when it occurred because data for projects that they were working on was also lost.

Bottom line: Science involves replication and I have always made my data available to others. In this case, I had redone this survey in 2002 before the controversy arose and made that data available to anyone who wants access to it. Despite the difference in years and the smaller sample, the survey results from the second survey were similar to those from the first survey. None of my critics have redone the survey using the questions that I provided.

Evidence on specific claims

1) Did I Attribute the 98 Percent Brandishing Number to Others? No

Apparently, some credence is being given to the claim that I have attributed the 98 percent brandishing estimate to others instead of myself. Some are taking this as evidence that I never conducted the survey. Yet, the fact is I never attributed this number to anyone else other than myself. It is claimed that I attributed this number to Gary Kleck on one occasion as well as the Los Angeles Times, Gallup, and Peter Hart during a couple of other times. Attachments entitled “Did I Attribute the 98 Percent Brandishing Number to Others?” and “Cases where 98% number was used” provide a detailed discussion of these issues. However, two brief statements are useful here:

The Independence Institute web site posted a piece that contained the one reference to Gary Kleck’s work.

------ Forwarded Message
From: "Dave Kopel"
Date: Sun, 9 Mar 2003 13:07:49 -0700
To:
Subject: Re: FW: A quick question. John Lott

I've got no specific recollection of editing the piece, but the evidence seems to indicate that attributing the 98% figure to Kleck was an error by the Independence Institute, rather than an error on the author's part.

Dave Kopel

See more recently:

----- Original Message -----
From: "Dave Kopel"
To: "Julian Sanchez"
Sent: Tuesday, April 01, 2003 9:37 PM
Subject: Re: Sorting out the Lott piece

Dear Julian,

. . .

Re-discovering Kleck's 98% figure from Social Problems (an article I thought about a lot in, say, 1994, but which I have rarely thought about recently) makes me believe that I added Kleck to the 98% sentence in the belief that the 98% figure came from him.

The fact that I added Kleck is, of course, also supported by the Rocky Mountain News version of the Lott article (a version I did not edit) which contains no reference to Kleck.

. .

Best wishes,
Dave Kopel

------ End of Forwarded Message

The second case of false attribution is that I claimed that the Los Angeles Times, Gallup and Peter Hart were the source of the 98 percent number. Where people get the claim is by combining two sentences in a Chicago Tribune op-ed that I had in 1998 (the same op-ed was also republished, for example, in the Washington Times).

The relevant passage from the op-ed reads:

“Other research shows that guns clearly deter criminals. Polls by the Los Angeles Times, Gallup and Peter Hart Research Associates show that there are at least 760,000, and possibly as many as 3.6 million, defensive uses of guns per year. In 98 percent of the cases, such polls show, people simply brandish the weapon to stop an attack.”

Webster’s Unabridged Dictionary defines the word “such” as “like or similar,” but not as the same. “Such” is merely referring back to this type of polls and not those specific polls. If the reference in the second sentence had been to “these” polls and not “such” polls, I would think that the critics would have an argument.

Statements by those who have made general statements about the 1997 survey

2) Statements by others documenting the loss of my computer hard disk in July 1997

From: Dan Kahan
Date: Thu Feb 13, 2003 12:49:32 AM US/Eastern
To: letters@washpost.com
Cc: John Lott
Subject: Feb. 11, "A Fabricated Fan and Many Doubts"

Dear Editor:
A column appearing in the Post yesterday (Feb. 11, "A Fabricated Fan and Many Doubts") implies that economist John Lott made up the claim that a computer malfunction destroyed data from his research on gun control. At the time Lott was engaged in this research, we were colleagues at the University of Chicago Law School. I clearly recall John relating the computer data-loss incident to me then -- many years before the current controversy about his work arose.
Just so you know, I'm not relating this information to you because I support Lott's position on guns (I don't). I'm relating it to you because I think journalists -- even the ones you employ to write political gossip columns like this one -- should live up to their professional obligation to check out the facts before they make claims harmful to an individual's reputation.
Yours,
Dan M. Kahan

*****************************************************************
Dan M. Kahan
Professor of Law
Yale Law School
PO Box 208215
New Haven, CT 06520 (regular mail)
127 Wall Street
New Haven, CT 06511 (courier)
(203) 432-8832
(815) 366-1458 (fax)

From: Floydzudd@aol.com
Date: Tue Feb 11, 2003 2:50:46 PM US/Eastern
To: letters@washpost.com
Subject: Article about John Lott in today's Post

Dear Editor:

The Washington Post unfairly casts doubt about whether John Lott suffered a hard disk crash on his computer in 1997 ( A Fabricated Fan and Many Doubts, February 11). I was co-authoring a paper with him at the time and I was affected by some data that were lost. We lost a very large data set that had been used to estimate the wage premium paid to workers exposed to long-term latent hazards in the workplace. The loss prevented us from performing additional research and significantly delayed publication.

Sincerely,

Richard L Manning, PhD
203 Putnam Road
New Canaan, CT 06840

From: Lawrence Kenny
Date: Tue Feb 11, 2003 2:21:58 PM US/Eastern
To: jlott@aei.org
Subject: Wash Post letter

John:
This is what I sent to the Washington Post.

John Lott and I worked together on a project examining the impact on government spending of women being granted voting privileges. Some of this research, utilizing older census data, was published in the Journal of Political Economy in December 1999. But the publication of other research utilizing recent survey data was set back when the basic data was lost in 1997 when John's hard disk crashed. Thus, assertions that John fabricated the story of his disk crashing are incorrect.

Lawrence W. Kenny
Professor of Economics
University of Florida

Date: Thu, 13 Feb 2003 23:55:13 -0800 (PST)
From: Jonathan Karpoff
To: letters@washpost.com
Subject: John Lott

Dear Editor,

A column the Post published this week implies that John Lott fabricated a story that a computer crash destroyed some data related to his gun research. I have collaborated with Lott on two research projects -- neither related to guns -- and remember him talking about the crash several years ago. The crash indirectly affected one of our projects, as Lott had to divert much time to re-create his lost databases. I recall him telling me how some of his philosophical opponents refused to help him by returning a copy of some of his data, despite the fact that the only reason they had the data in the first place was that Lott had given the data to them!

During our collaborations, John Lott has been an exemplar of integrity in academic research. It is not always easy to work with John, as we sometimes have disagreed over how best to conduct our tests and write up our results. But always, Lott has been honest, insightful, and willing to consider arguments and accept data that do not agree with his prior beliefs. He is an excellent social scientist.

It is time to put to bed any rumors that question Lott's credibility or seriousness as a researcher. Give him credit for taking unpopular positions, sticking to those positions in the face of vitriolic personal attacks, and sharing his data and exposing his research to scrutiny more openly than his opponents. You -- and I -- might not like like all of his conclusions. But that makes him all the more important to engage seriously in policy debates.

Sincerely,
Jonathan M. Karpoff

---------------------------------------------------------------------
Jonathan M. Karpoff
Norman J. Metcalfe Professor of Finance
Managing Editor, Journal of Financial and Quantitative Analysis
University of Washington School of Business
Box 353200
Seattle, WA 98195

Another letter by Karpoff:

---------- Forwarded message ----------
Date: Mon, 9 Jun 2003 15:18:49 -0700 (PDT)
From: Jonathan Karpoff
To: science_letters@aaas.org
Subject: John Lott editorial

Dear Editor,

I have worked with Lott on three research projects (none related to guns). The computer crash at which you snicker indirectly affected one of our projects, as Lott had to divert much time to re-create his lost databases. Although Lott got most of his data back, he really did lose some of it.

Lott re-created his lost data with a new survey. It is difficult to believe that, in a book filled with empirical tests that have been replicated by others, Lott would fabricate a single number, and that his made-up number would just happen to be statistically indistinguishable from the results of the new survey.

If you are concerned about research integrity, your aim is off. Throughout the controversy over gun research, Lott has shared his data with all comers. In contrast, some of Lott's critics refuse to share the data they use to attack Lott's findings. Which seems more suspicious to you?

You -- and I -- may not like or agree with all of Lott's conclusions. But no serious researcher can question Lott's integrity or skill as a social scientist. The whole point of Kennedy's editorial is about the importance of integrity in the conduct of research. I would hope that Science has higher verification standards than this piece shows.

Sincerely,
Jonathan M. Karpoff
---------------------------------------------------------------------
Jonathan M. Karpoff
Norman J. Metcalfe Professor of Finance
University of Washington School of Business
Box 353200
Seattle, WA 98195

phone: 206-685-4954
http://faculty.washington.edu/karpoff/
---------------------------------------------------------------------

From: "David B. Mustard"
Date: Wed Feb 12, 2003 12:04:50 PM US/Eastern
To: John Lott
Subject: Re: Sample letters that were sent in to the Washington Post

John,
This is a copy of my letter, which I sent to the Post yesterday morning shortly after we talked.

-----------
On February 11, 2003, in the article "A Fabricated Fan and Many Doubts", the Washington Post incorrectly questioned whether John Lott experienced a computer crash in the summer of 1997.

I can testify that Mr. Lott's computer crashed at that time and that he lost everything he had on his hard drive. He and I discussed this crash many times. John and I were co-authors on a project that ended prior to the crash. Because he lost all the data from our project, I replaced as much of the data as I could.

--
David B. Mustard
Terry College of Business
528 Brooks Hall
University of Georgia
Athens, GA 30602
Phone: (706) 542-3624
Fax: (706) 542-3376
http://www.terry.uga.edu/~dmustard/

From: John Whitley
Date: Tue Feb 11, 2003 5:50:16 PM US/Eastern
To: jlott@aei.org
Subject: [Fwd: Letter to the Editor]

Hello,

Here is the letter I sent to the Washington Post.

John Whitley

-------- Original Message --------
Subject: Letter to the Editor
Date: Wed, 12 Feb 2003 09:19:41 +1030
From: John Whitley
Organization: The University of Adelaide
To: Letters@washpost.com
CC: John Whitley

Dear Editor,

I believe you draw an inaccurate comparison between the issue of Dr. Lott's survey and the Dr. Bellesiles case in your article "A Fabricated Fan and Many Doubts" of February 11. I am not an expert on the Dr. Bellesiles case, but my understanding is that there was little or no contemporaneous corroborating evidence of the flood that he reported as destroying his records and that his results were not reproducible by other scholars in the field. Whatever the case is with Dr. Bellesiles, these are definitely not true in Dr. Lott's case.

There are numerous contemporaneous witnesses to Dr. Lott's hard-drive crash - I am one of them. I began working for Dr. Lott as a research assistant shortly after the hard-drive crash in 1997 and I distinctly remember him mentioning it to me at the time. More importantly, Dr. Lott's general results have been reproduced in many peer reviewed academic journals and he has a strong reputation among academics for distributing his data and publishing reproducible research.

Dr. Lott himself has now reproduced the survey and released the names of all people who worked on it and the phone records from the calls. The results are largely in line with his previous results and no one has questioned the integrity of the new survey.

Dr. Lott has a strong academic reputation for remaining faithful to the scientific protocol. Your comparison was unwarranted and inaccurate.

Sincerely,
John Whitley

Home Address:
Box 286
Ashton, SA 5005 Australia

Home Phone:
61 (0)8 8390 0504

--
John Whitley
School of Economics
Adelaide University, AUSTRALIA 5005
Ph : +61 8 8303 5500
Fax : +61 8 8223 1460
e-mail: john.whitley@adelaide.edu.au

3) Statements from David Mustard, John Whitley, and Geoff Huck

Statement in response to Jim Lindgren’s summary of my comments relating to John R. Lott’s Survey Work on Self-Defensive Uses of Guns by David B. Mustard, Thursday 16 October 2003

"In the summer and fall of 1996, I had numerous conversations with John Lott about how he was going to extend the original paper (JLS January 1997); a survey to learn more about defensive uses was one of those extensions."

"To summarize, I did not back off my claims of being “likely” or “fairly confident” that I heard about the survey in 1997."

Statement on John Lott’s Survey Work on Self-Defensive Uses of Guns by David B. Mustard Monday 10 February 2003

Background

John and I started working on our concealed carry paper in the fall of 1995. I was finishing my Ph.D. in Economics at the University of Chicago, and John was a faculty member. We worked on our paper intensively from about February 1996 to September 1996. We presented it at the American Law and Economics meetings in Chicago and at some workshops in the spring of 1996. We finished the JLS proofs in August or September. We then completed our work together and interacted much less frequently. In the summer of 1996 Lott talked about working on other projects related to guns. Once our paper was finished he started to work on a number of extensions. I did not work with John on any of these other projects because I had to finish my job market paper and send out my job applications (most due by 1 December 1996). In the rest of December 1996 I worked on my job market paper and practiced interviews. In January 1997 I went to the American Economic Association annual meetings and interviewed. I had campus visits through early February and signed my contract with the University of Georgia around the second or third week of February in 1997. From March to June I finished my dissertation and taught an undergraduate class at Chicago. In July I defended and traveled a lot, and my wife moved to Georgia. I moved to Georgia in early August.

Do I have direct first-hand evidence about John Lott’s survey?

I did not co-author the work on the survey with John, I did not work for John as a paid employee, and I do not know anyone who worked with John on the survey. I have not seen any survey instruments or primary data from the original survey.

I have copies of both the survey questions and the survey data from his 2002 survey that he did to replicate the original survey. I did not work with John on the follow-up survey, but was given the questions and data once the survey was completed. These data are qualitatively consistent with John’s statements about his first survey when we had talked about it.

Did John have a computer crash in the summer of 1997?

I believe beyond a reasonable doubt that John had a computer crash during the summer of 1997. On Friday 8 August 1997 my wife and I closed on our home in Athens, Georgia. While we were unloading our belongings in our new home a phone rang, which surprised us because we had not yet unpacked or plugged in our phones. We learned that the house had a phone in the garage, which we had not noticed. John Lott was on the phone. I believe that he told me at this time about his computer crash and having lost all his data. During the next few weeks I had additional conversations with John about his hard drive collapse and his lost data, and I agreed to provide him with all my data. My office at Georgia was not set up until late August (we were on quarters at that time and did not start until fairly late). I had a severe infection for a number of days in mid-late August. I returned to Chicago for about 5 days in late August (about August 27-1 September) for my graduation (on Friday 29 August) and to close out our affairs in Chicago. My grandfather died in early September and I returned to my home in Buffalo, NY for a number of days at that time. When I returned to Athens we had orientations and I had to prepare for classes. I likely provided John with the data from our concealed carry paper in mid-September to October. Did John specifically mention that he lost his survey data in his computer crash? John told me that he had lost all his data in the crash. He specifically told me that he lost all data related to our paper, which I later restored to the best of my ability. He also mentioned how he lost many things related to his book, which set him back in completing the book and forced him to eliminate some things he intended to include in the book.

Was their evidence of Lott’s intending to do a survey?

As we worked on the concealed carry paper, John talked about pursuing other projects to extend our work on concealed carry and guns. We talked extensively about self-defensive uses of guns and how we knew how frequently guns were used for self-defense and in what contexts they were used. John articulated a desire to learn more about self-defensive uses through a survey, just like he articulated an interest in many other extensions, like using city-level data (which we did not use in the 1997 paper), decomposing the effects more by time and place (which we started to do in the preliminary paper), decomposing the effect by different types of robbery classifications, and determining whether surrounding areas experienced increases in crime after a concealed carry law was passed. (This is a partial and not complete list of the extensions of the gun-related research). John generally followed through on these extensions. He further decomposed the effects by time and place and used city-level data in his book More Guns, Less Crime. He analyzed the effect on surrounding areas with Steve Bronars in “Criminal Deterrence, Geographic Spillovers, and the Right to Carry Concealed Handguns.” American Economic Review. Vol. 88, n2 (May): 475-479. He was unable to decompose the effect by robbery type because he was unable to find robbery data that allowed him to do so.

When do I first remember talking with John Lott about the survey?

I do not remember the first time John Lott and I talked about the survey. At the time there was nothing exceptional about the survey for me to associate with it and help me remember when I first learned about it.

I believe it likely that John informed me of the completed survey in 1997.

I think it highly probable that John told me he had completed the survey at the time of my talk at the Academics for the Second Amendment conference in Washington, DC in November 1998.

Comment on John Lott’s data distribution policies

John has always impressed me with his willingness to give out his data. To my knowledge John has always released his data to anyone who asks for it. In fact, we gave out our data about 4 months before the article even came out in print. We have now given our data to about 75 people from around the world; perhaps more. This behavior sets John apart from many scholars who will not release their data even after multiple requests to do so.

Another statement by David Mustard

Date: Tue Jun 3, 2003 4:04:39 PM US/Eastern

To Whom It May Concern,

. . . you allege that Mr. Lott did not have a severe computer problem in which he lost a substantial amount of data. I believe beyond a reasonable doubt that Mr. Lott had a computer crash during the summer of 1997. During the summer of 1997, my wife and I moved from Chicago to Athens, GA. On Friday, 8 August 1997, we closed on our new home. John called the evening we moved into our home. I believe he told me about his computer crash and that he lost all his data at this time.

During the next few weeks I had additional conversations with John about his computer crash and lost data. He specifically mentioned that he had lost all of the data related to our JLS paper and many things related to his book More Guns, Less Crime, which delayed the completion of the book and forced him to eliminate some things he intended to include in the book. I agreed to help John with all the data from our Journal of Legal Studies paper (Jan. 1997). I helped John with these data in the fall of 1997.

--
David B. Mustard
Terry College of Business
528 Brooks Hall
University of Georgia
Athens, GA 30602
Phone: (706) 542-3624
Fax: (706) 542-3376
http://www.terry.uga.edu/~dmustard/

evidence that part of original book being lost

From: Geoffrey Huck
To: jsanchez@cato.org
CC: John Lott
Date: Wednesday - January 15, 2003 9:57 AM
Subject: Re: John Lott Survey
Attachments Mime.822 (3600 bytes) [View] [Save As]

Dear Mr. Sanchez:
Since I no longer work for the University of Chicago Press, I don't have access to the Press's files. Nevertheless, I am pretty certain that there is nothing in those files that would help you. The Press generally did not keep any draft manuscripts, tables of contents, etc., after publication - there was simply not enough space to accommodate all that that would entail. I have purged all my Press files from my home computer, and I would doubt that there are any electronic files at the Press that would answer your questions. That leaves my archived home e-mail files (some of which I still have) and my memory (which isn't all that reliable). As to the latter, I have a vague recollection of a chapter or a section or sections of a chapter that had to be scrapped because of the computer crash, but I don't at this stage remember the subject of it (or them). At the time, we were talking about a variety of things John could do (e.g., including a chapter on mass public shootings). As to my e-mail archives, there are a couple of brief mentions in John's and my exchanges about the crash and loss of data, though I have found nothing explicitly about the defensive use of handguns in them (which doesn't mean anything in itself, since we were mostly talking on the phone and there must have been all kinds of things that were lost in the crash that we didn't discuss in the archived e-mails I still have). There is one message from July of 1997 that gives some sense of the magnitude of the problem we were dealing with, though. We fortunately did have an earlier draft of the manuscript in hard copy at the time of the crash and a disk with some - but not all - of the tables. My assistant Theresa let me know what was missing. I don't at this point know what these tables are - you might want to ask John about that.

Date: Fri, 25 Jul 1997 16:13:03 +0000
To: lott
From: Geoffrey Huck
Subject: More about tables

John: Theresa tells me that she has on the disk you gave her all the tables from 1 to 30 except 2, 5, 16, 21, 24, and 25, and 2 appendix tables. I asked her to e-mail you the contents of the disk. Hope this helps.
Geoff

There is also one e-mail message I have from February 1997 which contains an attachment that I'm not at this point able to decode. If that file turns out to be decodable and has anything of relevance to add to this question, I (or John) will let you know.
I hope you find this information useful.
Cordially,
Geoff

------ End of Forwarded Message

----- Forwarded message from John Whitley ----- Date: Mon, 10 Feb 2003 21:06:54 +1030 From: John Whitley Reply-To: John Whitley Subject: Survey by John Lott To: jlindgren@law.northwestern.edu

Hello,

John Lott has kept me informed on some of the highlights of your investigation into the survey he conducted in the mid-90's. In particular, he indicated you were having trouble finding corroborating evidence to his hard drive crash at the time and overall evidence that he conducted the survey. I can definitely vouch for the hard drive crash and I do have some recollections which may be from the survey, but I am not very sure of those.

I started graduate school in economics at the University of Chicago in 1996. I met John in my first year and began working for him as an RA the summer of 1997 when my core exams were finished (I don't know the exact dates off the top of my head, but I can look them up if you would like). That work was on city police departments for his paper on affirmative action in policing and we then went on to collaborate on a paper on safe-storage gun laws and have worked together several times since.

As I said above, I can definitely corroborate the hard drive crash. He mentioned the crash to me in at least one conversation we had in the months leading up to my starting as an RA. It came up because he was telling me he was telling me he was very far behind in his work because he was having to dig out from a hard drive crash. I think it came up in the context that he couldn't yet tell me what I was going to be working on when I started as his RA because he was so far behind, but I can't remember for sure. His specific comment was worded as a complaint about how "you think you have these things backed up and it never works" or something similar (it could have been that he wished he had backed it up or something, I don't really remember the specifics). Although I remember very little from those early conversations I had with John, that particular comment stuck out and I remembered it because, although I had had a computer as an undergraduate, I was really just then becoming computer literate as a serious user. I hadn't really thought about backing up hard drives and the like at the time and John's comment opened a range of issues I had never thought of. It has stuck in my head ever since because I have been more careful since and I remember it every time I change or use my backup systems.

Unfortunately, I can't directly corroborate the survey, but I do have one memory which may be related. I remember stopping by John's office one time I think during my first year and there were some undergraduates in the office. John was finishing up with them and my recollection is that he introduced me to them and then they left. I think he introduced their names (which I don't remember) and said that they had recently worked for him (although I don't remember if he said on what), they then left and I met with John to talk about working as an RA for him. I am pretty sure they were undergraduates because I seem to recall them being impressed when I said I was an econ graduate student (anything that inflates your ego during the first year of graduate school at the University of Chicago is a big deal at the time).

In that situation, what I really remember most is the scene and not the words. I don't know when exactly it was, but I can remember the room. It was in John's old office, when he was in the middle of the back wall of the Chicago Law School library (before he moved over near the stairwell to the smaller office). I can remember him sitting at his desk and the students (I am pretty sure there were two students, but not 100% positive) were standing between me at the door and John at his desk against the far wall. I think one was taller than me (I am 5'6") and had lighter hair while the other was shorter and had darker hair (the heights I am pretty sure of, the hair color I am less sure of). The taller one was closer to me and seemed to be more of the leader. For some reason, that image sticks in my head.

Unfortunately it is possible that I am mixing this scene up in my head with other events, but it is fairly clear in my head so I am at least reasonably confident in it. If my recollection is correct, it is entirely possible (very likely, in fact) that these were some of the students who had worked on the survey (I do have some vague recollection that they had coordinated something and that others may have been involved). Unfortunately that is all I can really remember on that one right now, sorry I can't be more specific.

I hope this helps. Please let me know if you need any more information or if I can be of any further assistance.

John Whitley

-- John Whitley School of Economics Adelaide University, AUSTRALIA 5005 Ph : +61 8 8303 5500 Fax : +61 8 8223 1460 e-mail: john.whitley@adelaide.edu.au

-----------------------------------------------------------

----- End forwarded message -----

4a) Jim Lindgren has noted the possibility that David Gross took a early 1996 survey by David Hemenway and not the survey by me in early 1997. During November 2005, Hemenway finally released the data from his 1996, despite years of requests. In any case, that data clearly shows that Gross could not have taken Hemenway's 1996 survey. In Hemenway's 1996 survey, there were eleven male Minnesotans who participated. Of those eleven only one said that they had a permit to carry a concealed handgun at that time. That person was 30 to 39, lived in a city, made $35,000 to $50,000, had one child under 18 living at home, and had one drink during the previous month. Of those eleven, only one person said that they believed that they would feel more safe if community carried guns. That person lived in a rural area, was 40 to 49, made $35,000 to $50,000, had one child under 18 living at home, and had not had a drink in the previous month. In 1996, David Gross had a permit to carry a concealed handgun, believed that right-to-carry laws would make the area safer, lived in a suburb (St. Louis Park), had two children living at home under age 18, owned a wine cellar and enjoys drinking beer by itself several times a month, was 47, and his income numbers do not match up. The number of guns that he owned at the time as well as many other responses do not even closely match up with what these respondents claimed were true. The answers to these questions simply do not match up with what David Gross would have told a surveyor.

I also checked the Hemenway's 1996 and 1999 surveys to see if it was possible that James Hamilton had taken them, and it is quite clear that he couldn't have taken them. 4b) Email from Joseph E. Olson
From: Joseph E. Olson
Subject: Jim Lindgren's "report"
Date: November 17, 2003 4:12:50 PM EST
To: Glenn Reynolds
Cc: , Eugene Volokh

        Jim Lindgren concludes that David Gross took a 1996 survey by a professional polling organization done as part of a study by Hemenway and Azrael at Harvard rather than a 1997 survey taken by students as part of a study John Lott of Chicago was conducting. 

          Mr. Gross disagrees.

           What follows is a compliation I've put together of a number of David Gross' e-mails to me discussing this situation.

            11/11/03
I don't know where James Lindgren gets off stating things as he does, for he is wrong about my improving memory. I expressly closed the issue out quickly.  Please, see my email of January 20, 2003 at 4:25 P.M. I spent a total of less than 24 hours involved in the discussion, start to finish. I don't believe that I said that the survey concerned the laws; I said that I was familiar with the carry laws. I think James Lindgren has that incorrectly in any notes he took.  That's his confusion; not mine.

  I sent my first message concerning my awareness of the issue at 4:26 P.M. on January 19, 2003. I talked with Mr. Lindgren once that evening, [shortly before] 10:46 P.M when I sent a message referring to his call.  Lindgren was impressed that I remembered the timing of the talk at the Minneapolis Athletic Club (some four years earlier) to within a couple of weeks, and that, when I said "a couple of years before" to John Lott, I meant "a couple of years" and not 1 3/4 years or 2 1/4 years. THAT, alone, excludes the 1996 Harvard study, because it WAS early 1997 when I answered the short survey, NOT 1996.

            01/20/03
 I believe that in my emails to you and in my conversation with James Lindgren, I have stimulated just about all of my memories and impressions that I'm going to have concerning the survey that I took, without having any substitutions and additions by any possible suggestion, express or implied. I do not wish to be seen as a partisan in this matter, for I am not. I also am unwilling to speculate. My recollections and impressions of the survey that I took are now closed to prevent any accusations that I am having "convenient" recollections. I simply refuse to get in the middle of this any more than I already have. I am a witness who has come forward; I don't have a dog in this fight.   *  *  *   
The bottom line is that listening to the audiotape [of Lott's 1999 speech], today, has confirmed the recollections that I had of the survey and the timing.  *  *  *
It goes beyond any statistical possibility that I was surveyed by anyone else, given the timing, the subject matter, its brevity, it's precise focus, etc.. The clincher, for me, was my recollection, last night, in my e-mail, that there was a question concerning warning shots as a distinct category of discharge as opposed to a shot directed/fired AT the criminal. From that e-mail, before I listened to the tape of his talk, today:  "I don't recall whether the survey was limited to permits to carry and usage, or whether the subject matter was the broader "defensive use" to include use on
private property. I think it was the broader question, because I know that I mentioned those two significant incidents in my life as defensive use. I seem to recall, at this point, some question about a "warning shot," concerning discharge of a firearm, but I'm not sure."   I had not listened to the tape, when I wrote that note, because I hadn't even begun to look for it yet. I had just concluded my conversation with James Lindgren and was responding to an earlier email reporting that I had been contacted by James Lindgren, already; and amplifying my comments on what I remembered; as well as obligating myself to try to find the audiotape, something I had recalled since I was first contacted, and which I had mentioned during my conversation with James Lindgren.
Yet, as clear as day, in the tape of the talk, as I listened this morning, was the distinction between warning shots as a discharge (a distinct category, or part of "brandishing") and shots directed/fired AT the criminal. Bingo! That was the sound of a synapse firing, ladies and gentlemen. :-)
 The 98% figure is not only set out in his talk, but further, briefly explored/defined. He does not, anywhere that I could find in the tape, say that he was the one who had developed the figure and its nuances. But it was clear to me, then, whoever did conduct the survey, that I had been a respondent in that survey; and I mentioned it to John Lott in passing [after his speech]. Now, I'm sure that it was John's survey. I'll bet the house on it. I have no reasonable doubts based on logic, evidence, and common sense.

          10/04/03
I am certainly willing to write a letter that says, "Dear Jim, if you had only suggested the possibility of Harvard/Cambridge, I could have easily dispelled your confusion, because . . .. ."

1)     Wrong year by a whole year!
2)    Wrong source: I very clearly recalled the "Chicago" connection, and as both my older brothers (one of whom used to live in Highland Park at 461 Lakeside Place and worked in the loop [and I currently have a friend who lives at 471 Lakeside Place; and, by the way, my brother's in-laws lived at 827 Bob-O-Link Road in Highland Park] WENT TO HARVARD (Cambridge, MA), I would have recalled any such connection, there, as significant, in that context and with that association. THAT didn't happen. Interestingly, this comparison (Cambridge vs. Chicago) has never surfaced before or been brought to my attention in any way, shape, or form. I could have easily and quickly dispelled any confusion in that regard, if only because of my demonstrated ability to remember associative detail as a mnemonic aide. If Lindgren had asked me about this, I would have given him an answer that clarified HIS confusion, based on the above. I have no confusion; not possible; no way; no how. The Chicago connection is something that I CLEARLY and IMMEDIATELY and INITIALLY recalled and discussed; and the lack of any Cambridge/Harvard connection is equally as firm and solid: negative.  There was no doubt about that at any time. There is no doubt, now. This is nuts.
    A.    And who said that a relatively locquatious (under the circumstances) respondent (me) would be cut off by the interviewer, because the last DGU was just over 24 months earlier, instead of listening, politely, and coding the information properly on the facts of the response.
3)     There were no 30 questions asked; no way. It was fewer than a dozen or so. Not many at all. SHORT; VERY SHORT. I was very clear on that, always.
4)     Warning shots issue. As developed as the "clincher" (in my mind) in earlier communications. End of discussion. I kept myself completely "pure" in my communications with you and in my discussions with Lindgren, I was uncoached in any fashion; unled by anyone; and undeterred by any criticism of my motives, because I had none, except the pursuit of the truth of the matter. Perhaps I was too honest, forthright, and forthcoming.
Again, there's no doubt in my mind, based on my reconstruction of the source, timing, and number and content of the questions, that it was John Lott's survey. I'm only sorry that I was forced to "work backwards" to the conclusion from significant intervening, associative markers so many years later. It's how I remember things and put them into context. Even Lindgren remarked on the accuracy and detail of my memory and its associations. The truth is like that.
I'm sure that the survey to which I responded was NOT the long, 1996, Harvard survey; and any inference from the circumstances that Lindgren draws in that regard utterly fails to exclude clearly the very short 1997 study from Chicago that had a warning shots question.  The circumstances of my recollection lead directly to Chicago and away from Cambridge (proper use of circumstantial evidence: he can not support even probable cause for inquiry (a search warrant) on what he has done.

  **********************************************
Professor Joseph Olson     Hamline University School of Law
tel.   (651) 523-2142          St. Paul, Minnesota  55104-1284
fax.  (651) 523-2236         
Statement by David Gross about taking the 1997 survey, with follow-up comments by me

From: David Gross [mailto:david.gross@tcq.net] Sent: Tuesday, February 11, 2003 3:00 AM To: John Lott Subject: RE: Question

Statement from David Gross:

The answer to your question is a factual one that emphasizes three things: 1) that I was the subject of a telephone survey in early 1997, the only one that I have been a respondent in; 2) that during your talk for the Center of the American Experiment in Minneapolis on January 27, 1999, you mentioned the "98% survey" of DGUs (not attributing it to yourself, at that time) in a manner that caused me to approach you and to comment that I had been a respondent in the "98% survey" that you had mentioned in your talk "a couple of years ago;" and 3) when I became aware of the controversy made over the issue of whether the "98% survey" had been, in fact conducted, I re-reminded you of my comment to you and wondering whether the survey I had responded to was the same one whose existence was, now, being questioned.

Both of those events preceded any controversy and are clear in my memory, the first event having been refreshed in 1999 and the second event being a "recording" of the first. I was not sure, at first, that the 1997 survey was in fact connected to the current controversy, but was willing to discuss my recollections and the circumstances with anyone who was willing to inquire.

James Lindgren contacted me the same evening of the day that I emailed you with my suspicions and probed my recollections in detail and in general. I was able to place the date of your talk in Minneapolis within a week of its occurrence (subsequently documented) and I gave the "details" of what I recalled to James Lindgren without the benefit of any "coaching" by you or any format or details of the challenged survey or not. I kept myself "pure" and untutored so that others could judge whether or not my recollections were sufficient to verify that it was your survey. I wanted to remain a neutral witness to the facts and not be able to be accused of manufacturing any memories. I think I accomplished that, for good or ill, along with the fact that I cannot claim a perfect, super-human memory, just a good one.

I'm still not absolutely sure of ALL of the details of the survey, but my clear recollection DOES include:

1. Its brevity: the fact that it was focused, short, and involved DGUs and something about permits to carry;

2. That it was in the evening, from a student in the Chicago area (for some reason, Northwestern University plays a part, an impression, in the memory that I have and that the student explained to me that Northwestern was in the Chicago area; but I knew that.);

3. After my conversation with James Lindgren and his questions, I recalled that it contained a questions as to whether the incident(s) had been reported and, I vaguely recalled, something about "warning shots." The whole conversation with James Lindgren had stimulated me to think hard about what it was that I recalled, to let it come back to me as best it could. James Lindgren did not "feed" me any information on the survey, if only because I refused to accept any implicit or explicit assumptions of fact that I could not recall. The conversation with James Lindgren had already stimulated the memory of the tape, whether or not I could find it or locate a substitute.

4. My clear recollection of the comment to you about having participated in the 1997 survey ("a couple of years ago"). Even I wanted to know, to recall what revelation it was that had startled me and had motivated me to make a comment to you about having been a respondent in that survey, even though I didn't know it was yours at the time.

5. It was the only phone survey on DGUs and such in which I have ever participated.

The rest is all about the PROCESS of association, recollection, corroboration, and verification of the circumstances.

During my conversation with James Lindgren, I recalled purchasing the tape of the talk from the Center (I mailed in a check and the tape was sent by mail) and obligated myself to attempt to find it. I know that I had not listened to it and expected to find it in its mailing envelope. Failing that, I could contact the Center to see if they had an archival copy, or the master. I was, also, absolutely clear that I had approached you, as you were leaving the talk in order to go the Minnesota Legislature and to testify on the issue. I was with Joe Olson and he can verify the fact that I did approach you, but he cannot verify the conversation "in passing." Frankly, you were preoccupied, and I felt a little foolish making the comment, "as if you cared." I did not attend the committee hearing. I know that I made the comment to you about my having been a participant in the survey you mentioned in your talk "a couple of years ago;" but during my conversation with James Lindgren, I was unable to "put my finger" on exactly what it was that caused me to reach that conclusion and motivate me to make the comment to you as you were being ushered to the committee hearing in St. Paul. It was some unique characteristic that you mentioned that made me sure that I had been a respondent in THAT survey you had referenced in the talk; and I wanted to know/remember what it was. The tape would either refresh my memory or it would fail to do so.

I found the tape and listened to it, making notes. When the talk got to the point where the survey you mentioned contained a distinction concerning "warning shots" as distinct from shots AT the attacker, my memory, the circumstances, and the motivation for my comment to you all "clicked." It was the warning shot issue that caused me, in 1999, to recall the uniqueness of the survey in 1997. My memory of the survey had been refreshed/jogged in 1999, and the incident including the comment to you, in my mind, serves as a "recording" of the clear recollection I had then. My memory was much clearer, then, in 1999, being only two years removed from the survey, instead of six years; but my memory of my emotional/intellectual reaction to the sophisticated handling of the warning shot issue (Aha!), my then certainty as to the uniqueness of the survey I had taken in 1997, and the physical action and effort to make the comment to you is, today, crystal clear, even though we are four years removed from that event. Most importantly, my vague recollection of the survey's question of the warning shots prior to listening to the tape, the talk's drawing a distinction between warning shots and shots AT the attacker, and I remembered what it was that caused me to react and to comment with certainty, then.

I can't help but to analyze "evidentiary" matters in the legal context, not only because that is my training and professional experience, but also because the legal context has such real, practical value as the embodiment of the human condition and the search for the "truth." These evidentiary rules are the distillation of human experience and have real value to real people in their ordinary affairs of daily life as well as the important ones. They help one to think clearly and to guide analysis of real situations. They work well in the context of real-life, human society.

I became sure that the survey to which I had responded was yours because my January 27, 1999 reaction and comment to you serve as a "recording" of my memory of the survey from 1997, at a minimum on the point of the warning shots aspect of the survey. I recalled enough "details" independently (short, Chicago, student, DGU, carry laws/permit, reported, and vaguely, warning shots) that the timing (which fact I supplied in 1999 and which was not mentioned in the talk) and the warning shots issue nail it down dramatically. In the absence of another survey going on with similar features, the rule of circumstantial evidence allows one to draw only one conclusion, which I unabashedly do, especially when combined with my direct memory: circumstantial evidence supports a finding of fact when it directly points to/allows the inference of the existence of a fact and, at the same time, reasonably excludes the existence of a contrary inference or conclusion.

Where is that other survey I could have been a respondent in? Even James Lindgren suggested to me that, in the absence of another survey with which I could have confused the two, it was clear that I had been a respondent in your survey.

If you look at my initial email to you, you'll see that my reaction and comment to you in 1999 were a main component of my willingness to come forward and possibly to shed some light on the matter. All the rest has been confirmation of the unique details, of most of which I was unaware until after I had exhausted my ability to remember/recall.

Now, some people have suggested that I have come forward in order to "cover" for you and that I am biased and willing to lie for you. I mean no offense; but John, I don't know you that well. I don't believe that I know anybody that well. Even though I follow the rules of evidence (and common sense) in my daily life in evaluating the credibility of people with whom I come into contact, it has become apparent to me that there is a dearth of common sense, analysis, or respect for others' integrity in the "politically correct" world out there. Even academia, where proven, valuable standards are supposed to be rigorously maintained by those trained and experienced in them, appears not to be immune from political pressures and the consequent lowering of standards. This is how thought control and fascism gain a foothold. That will be the community's loss comparable to the loss of the library at Alexandria.

The fact that I confirm the survey in 1997 has nothing to do with the validity of the conclusions you drew from it, the propriety of your use of it, under the circumstances, or its sufficiency. Those matters are pretty much beyond my expertise, although what I recall of it from the talk in Minneapolis seemed powerfully and intuitively reasonable and accurate, to me, based upon my understanding and experience, as well as from other research.

Note from John Lott: I know of no other self-defense surveys on guns that were done in 1997. Hemenway et. al. did a survey in the Spring of 1996 and Spring of 1999, but both of those surveys are not only in different years but also quite different than the survey described by Mr. Gross. All the other surveys have been done by professional polling services, not students. Hemenway et. al.’s polling firm was based in Albany, New York, while my survey was based out of Chicago. It is not clear how many questions were asked by the Hemenway et. al.’s survey since it is not stated how many questions were asked before the respondents discussed if they had used a gun defensively. After two questions identifying who has used a gun, up to an additional 30 open ended questions were asked. It is not possible for that survey to have been completed in the short period of time discussed by Gross. Also, after the 1994 National Survey of Private Ownership of Firearms National Survey of Private Ownership of Firearms, there were no other surveys that specifically asked respondents about “warning shots.”

.

.

5) Supplying my tax returns
I have also provided my tax returns for 1997 to several academics (e.g., one law professor who specializes in tax law, Professor Joe Olson at Hamline University Law School). While we did not retain the itemized list of deductions past the IRS three year requirement (and this controversy was six years later), the returns indicate that I had spent a large amount of money on RAs that year. John Whitley was the only other RA that I had hired that year and the amount that I paid him was just a tiny fraction of the total amount that I spent that year on RAs. John remembers me paying him out of my own pocket.

Other points

A) The point of using the survey in More Guns, Less Crime was to deal with the issue of media bias (it is exactly the same point raised in The Bias Against Guns). The point was that the lack of coverage of defense gun uses is understandable if most uses simply involve brandishing where no one is harmed, no shots fired, no dead bodies on the ground, no crime actually committed. If others believe that the actual rate of brandishing is lower and I had used the results of Kleck, it becomes MORE difficult to explain the lack of news coverage of defensive gun uses. The two short discussions that I have on this issue in my two books thus choose results that are BIASED AGAINST the overall point that I am making, that the media is biased against guns.

B) Compared to the 98 percent number there was an earlier survey by Kleck that found 92 percent of defensive gun uses involved brandishing and warning shots and because the survey was asking people about events that occurred over a long period of time it is likely that it over emphasized more dramatic responses. (My number that is directly comparable to the 92 percent estimate is about 99 percent.) My point in the book was that defensive gun use rarely involves more “newsworthy” events where the attacker is killed and either survey would have made the general point. I have a discussion that explains why his surveys get lower results here. C) The point of science is replication. I redid the survey in 2002 and got similar results.

John Lott's Website
Home

Johnlott.org (description of book, downloadable data sets, and discussions of previous controversies)

Academic papers:

Social Science Research Network

Book Reviews:

For a list of book reviews on The Bias Against Guns, click here.

---------------------------------
List of my Op-eds
---------------------------------

Posts by topic

Appalachian law school attack

Baghdad murder rate

Arming Pilots

Fraudulent website pretending to be run by me

Ayres and Donohue

Stanford Law Review

Mother Jones article

Vin Suprynowicz quote

Links

Craig Newmark

Eric Rasmusen

William Sjostrom

Dr. T's EconLinks.com

Interview with National Review Online

Lyonette Louis-Jacques's page on Firearms Regulation Worldwide

The End of Myth: An Interview with Dr. John Lott

Cold Comfort, Economist John Lott discusses the benefits of guns--and the hazards of pointing them out.

An interview with John R. Lott, Jr. author of More Guns, Less Crime: Understanding Crime and Gun Control Laws

Some data not found at www.johnlott.org:

Updated Media Analysis of Appalachian Law School Attack

Since the first news search was done additional news stories have been added to Nexis:

There are thus now 218 unique stories, and a total of 294 stories counting duplicates (the stories in yellow were duplicates): Excel file for general overview and specific stories. Explicit mentions of defensive gun use increase from 2 to 3 now.

Journal of Legal Studies paper on spoiled ballots during the 2000 Presidential Election

Data set from USA Today, STATA 7.0 data set

"Do" File for some of the basic regressions from the paper