New Op-ed: On Giving Terrorists the Best Advice

Here is a new piece that I have at Fox News:

Should a website post the best ideas for successful terrorist plots? Should we even discuss publicly how to stop terrorist attacks? This week, New York Times blogger Steven Levitt publicly posted terrorist plot suggestions. He claims that “by getting these ideas out in the open, it gives terror fighters a chance to consider and plan for these scenarios before they occur.”

Levitt clearly assumes that terrorists have already figured out the best ideas, but that our side has not. If anything, the reverse is probably true. There are vastly more Americans than terrorists possessing detailed information on American infrastructure, traffic flows, policing practices, etc. So terrorists could easily learn something. To make matters worse, there are also many home-grown mental basket cases who could get ideas on how to obtain worldwide attention. . . . .<.dd>

Steve Levitt's Correction Letter

Many comments have been posted on this letter based upon the news story written on it in the Chronicle of Higher Education, but few have seen Levitt's actual correction letter. Among those commenting on the original news stories please see Ted Frank, Ben Zycher, John Palmer, Craig Newmark, Robert Wallach, Clayton Cramer, Larry White, Steve Sailer, Xlrq, Jonathan Adler, Michael Munger, Steve Sailer again, Glenn Reynolds, Say Uncle, Jode Shoo, Singular Values, and
The Corner. Unfortunately, at this point, I am not allowed to really comment on this. One part of Levitt's letter that has not gotten any attention is the one that I think was his most important, his statement that:

"I also was aware at the time of the May 2005 emails to you that in connection with the preparation of conference issues for the JLE that the organizer of each conference issue needs to provide funding to the JLE to cover publication and mailing expenses. I did not mean to suggest that Dr. Lott did anything unlawful or improper in arranging for the payment of the publication expenses for the Conference Issue."

The Chronicle of Higher Education has had a couple of articles on all this:

See this from the Chronicle.

See also this:

According to the motion, new facts have come to Mr. Lott’s attention since last year that significantly alter the character of his complaint. For one thing, he says, new information has come out about what he calls Mr. Levitt’s malice toward him. The motion alleges that Mr. Levitt has publicly referred to Mr. Lott as “the anti-Christ” and that Mr. Levitt “offered publicly to pay colleagues if they would humiliate” Mr. Lott. (Mr. Levitt did not immediately reply to a request for comment today.)

One point that wasn't directly mentioned by anyone is that Levitt's response when asked to backup is claim that others hadn't replicated my research was that the research papers were not refereed. Not only is Levitt acknowledging that the papers backed up and replicated my research, but he is admitting that the papers that did so were refereed.

The date on this letter does not match when I received it.
Update: See also this follow up filing.

Judge Castillo issues decision on Lott v. Levitt

A copy of the judge's decision can be downloaded here. A Copy of Levitt's Correction Letter can be obtained here.

1) The Judge has found that one of the two counts of defamation involving Levitt can proceed.

p. 15: ". . . Levitt's email sounds as if he was "in possession of objectively verifiable facts." In his email Levitt states: 'It was not a peer refereed edition of the Journal. For $15,000 [Lott] was able to buy an issue and put in only work that supported him. My best friend was the editor and was outraged the press let Lott do this.' First, it would be unreasonable to interpret Levitt's unqualified statement that the journal edition was not 'peer refereed' as Levitt merely giving his opinion on the 'peers' chosen to review,or referee, the Special issue. Indeed, the editor of the Journal might be able to verify the truth or falsity of whether the Special Issue was reviewd by peers. Furthermore, while Levitt argues that one person's "'peer' in the academic realm may be another person's 'hack'," this distinction is not reasonable when discussing the review process at a top university's academic journal. Second, a reasonable reader would not interpret Levitt's assertion that "For $15,000 [Lott] was able to buy an issue and put in only work that supported him" as simply a statement of Levitt's opinion. Levitt's email appears to state objectively verifiable facts: that Lott paid $15,000 to control the content of the Special Issue. . . . Levitt's motion to dismiss Count II of Lott's Complaint is denied."

2) p. 7: "The applicable standard, however, is not that of the 'world of academic research and scholarship' that Lott describes. Rather, the critical question is how a 'reasonable reader' would interpret the phrase. The reasonable reader in this case is the general population . . . . In everyday language replicating results does not necessarily mean analyzing identical data in identical ways . . . ."

Response: I think that the market for the book was also aimed at academics. The book is apparently marketed to a large number of economics classes and is read by academics.

Levitt's correction letter


Many comments have been posted on this letter based upon the news story written on it in the Chronicle of Higher Education, but few have seen Levitt's actual correction letter. Among those commenting on the original news stories please see Ted Frank, Ben Zycher, John Palmer, Craig Newmark, Robert Wallach, Clayton Cramer, Larry White, Steve Sailer, Xlrq, Jonathan Adler, Michael Munger, Steve Sailer again, Glenn Reynolds, Say Uncle, Jode Shoo, Singular Values, and
The Corner. Unfortunately, at this point, I am not allowed to really comment on this. One part of Levitt's letter that has not gotten any attention is the one that I think was his most important, his statement that:

"I also was aware at the time of the May 2005 emails to you that in connection with the preparation of conference issues for the JLE that the organizer of each conference issue needs to provide funding to the JLE to cover publication and mailing expenses. I did not mean to suggest that Dr. Lott did anything unlawful or improper in arranging for the payment of the publication expenses for the Conference Issue."

The Chronicle of Higher Education has had a couple of articles on all this:

See this from the Chronicle.

See also this:

According to the motion, new facts have come to Mr. Lott’s attention since last year that significantly alter the character of his complaint. For one thing, he says, new information has come out about what he calls Mr. Levitt’s malice toward him. The motion alleges that Mr. Levitt has publicly referred to Mr. Lott as “the anti-Christ” and that Mr. Levitt “offered publicly to pay colleagues if they would humiliate” Mr. Lott. (Mr. Levitt did not immediately reply to a request for comment today.)

One point that wasn't directly mentioned by anyone is that Levitt's response when asked to backup is claim that others hadn't replicated my research was that the research papers were not refereed. Not only is Levitt acknowledging that the papers backed up and replicated my research, but he is admitting that the papers that did so were refereed.

The date on this letter does not match when I received it.
Update: See also this follow up filing

New Research that Abortion Increases Violent Crime Gets Attention

From today's Chicago Sun Times:

A high-profile economist is challenging the conclusion in the best-selling book Freakonomics by University of Chicago professor Steven D. Levitt that the legalization of abortion in the early 1970s led to a major drop in murder and other violent crimes a generation later.

John R. Lott Jr., a former U. of C. economist now teaching in New York, says the Supreme Court's 1973 Roe v. Wade decision actually caused violent crime to rise.

Lott and fellow researcher John Whitley plan to publish a paper in October in Economic Inquiry that questions Levitt's research on abortion and crime.

Lott and Levitt already were feuding over Lott's charge that Levitt had defamed him in Freakonomics.

In their new paper, Lott and Whitley say that legalization of abortion prompted a cultural change that increased the number of children born out of wedlock. Those children of unwed mothers caused murders to rise by more than 700 cases in 1998 alone, saddling the public with more than $3.3 billion in "victimization costs," the paper says.

On the other hand, Levitt's research found that Roe v. Wade resulted in a savings of $30 billion a year that crime would have cost the public.


More unwed mothers

His Freakonomics, co-authored by Stephen Dubner and published last year, says legalized abortion led to a large drop in murder and other violent crime in the late 1980s and early '90s, and continues to reduce crime.

The book suggests that if the aborted fetuses had instead been born, they would have become adults more likely to commit crimes because they were unwanted by their mothers.

To illustrate the point, the book says the five states that allowed abortion three years before Roe vs. Wade saw major declines in violent crime between 1988 and 1994 -- earlier than the other states.

But Lott says the Levitt study did not fully consider the increase of children born out of wedlock. His theory is that with the option of abortion, women became more likely to have premarital sex, but then had their babies and raised them as single parents.

Children born out of wedlock have had smaller investments in "human capital" by their parents and are more likely to get into trouble when they grow older, Lott says.

On average, his paper says, about 5 percent of whites were born out of wedlock from 1965 to 1969, rising two decades later to 16 percent. For blacks, the figure rose from about 35 percent to about 62 percent, the paper says.

Before legalized abortion, more than 70 percent of children born out of wedlock ended up in families with a father, but the fraction fell to 44 percent in 1984, according to the paper. . . . .

If you want to read the research, you can find it here. THe newspaper article says that I concede that abortion through its effect on "unwanted" births slightly reduces violent crime, but what I believe that I said is that it is possible. The net effect however is abortion increases violent crime.

Ariel Rubinstein Reviews Freakonomics

Ariel Rubinstein provides a critical review of Freakonomics. I guess that I had wish that Ariel had gotten more into the specific economics examples. I think that there were many mistakes in the book.

I thank him for sending it to me.

Levitt's argument for dismissal

Levitt's response to my lawyer's response to their motion to dismiss can be found here.

I haven't really had much of a chance to look at their response so far. A past post on this can be found here.

Lott v. Levitt Response FIled

Yesterday my lawyer filed a response to Levitt and HaperCollin's motion to dismiss the defamation case. For those interested, the response can be found here.

A past post on this can be found here.

LA Times Letter to Editor: Re "Shooting holes in a lawsuit," Opinion, May 31

Unfortunately, this letter was substantially shortened and somewhat altered the meaning due to space limits on letters. It is also unfortunate that the LA Times did not publish any letters from other academics who had written in.

Accusations on gun law research are unfounded
June 12, 2006

Re "Shooting holes in a lawsuit," Opinion, May 31

Jon Wiener claims that "nobody" tried to replicate my research, which demonstrated that right-to-carry laws reduce crime rates.

This is false. Not only has everyone who tried managed to replicate my findings, but many academics have gone beyond that and shown that right-to-carry laws reduce violent crime under a variety of approaches (see johnrlott.tripod.com/postsbyday/RTCResearch.html).

Wiener fails to note that since my research was published, not a single peer-reviewed publication has found that right-to-carry laws increase crime, and there is research that finds even larger drops than I found.

Wiener's Google search implies that many other scholars agree with his claims of research fraud. Yet Wiener fails to note that only about a sixth of the postings are actually scholars of any type.

Wiener claims that my defamation suit wants to "silence" certain claims of fraud. I and other academics have tried to engage Stephen D. Levitt in discussions on the accuracy of his claims, but he won't respond to us.

Instead, [Levitt] falsely charges that the research with which he disagrees was published only because the University of Chicago Press overrode the journal's editor, and that I bought the press' decision.

JOHN R. LOTT JR.

Here is one of the other letters that I know of from two academics:

In his op-ed piece, “Gun-research 'Freak'-out” (LA Times, May 31), Jon Wiener says, “Blocking the sale of a book based on a literal interpretation of a single word would be outrageous.” But this is not the issue. In Freakonomics, Steven Levitt and Stephen Dubner say, “When other scholars have tried to replicate [Lott’s] results, they found that right-to-carry laws simply don't bring down crime.” This statement is false under any reasonable interpretation. We replicated Lott’s results, as have many other scholars. If Levitt did not know this, then he was lax in his scholarship.
Levitt should know that among scholars “unable to replicate” is interpreted as “something is seriously wrong here.” Moreover, the phrase carries the suggestion of, at best, incompetence and, more likely, dishonesty.
Levitt should express his disagreement with Lott in a way that does not imply incompetence or dishonesty. It is thoroughly reasonable to require a publisher to revise a sentence for future sales if the existing sentence is false and defamatory.

Nicolaus Tideman
Blacksburg, VA

Florenz Plassmann
Ithaca, NY

See also:

Dear Mr. Goldberg:
At the end of his op-ed piece, “Gun-research 'Freak'-out” (LA Times, May 31), Jon Wiener says, “Blocking the sale of a book based on a literal interpretation of a single word would be outrageous.” But the relevant issue is not “a literal interpretation of a single word.” In Freakonomics, Steven Levitt and Stephen Dubner say, “When other scholars have tried to replicate his results, they found that right-to-carry laws simply don't bring down crime.” This statement is false under any reasonable interpretation of the words.
In our understanding, and in the understanding of colleagues we consulted, what is generally meant in economics by replicating someone’s work is gathering the same data and analyzing them in the way that the original researcher had analyzed them. We took an interest in the issue of guns and crime when Lott’s work was first published. We gathered the data that he had used, analyzed it in the way that he described and got essentially the same results. Thus we replicated Lott’s work.
In some circumstances, what is meant by replicating someone’s work is to gather similar data and analyze it in the same way. But that interpretation would not be relevant here, because (as often happen in economics) the data that Lott used were all of the data that were available.
Should “replicate” be stretched to mean “undertake similar analyses and reach similar results”? Well, we analyzed the guns and crime data in additional ways that we regarded as interesting and reached results that were broadly in agreement with those that Lott had reached.
Is there any reason why our work (“Does the Right to Carry Concealed Handguns deter Countable Crimes? Only a Count Analysis Can Say,” Journal of Law and Economics, October 2001, Volume 44, Number 2, Part 2, pp.771-798), should not count as a replication of Lott’s work? Wiener quotes in apparent endorsement Levitt’s assertion that, “for $15,000 [Lott] was able to buy an issue [of the Journal of Law and Economics] and put in only work that supported him.” This is not what happened. We presented our work at a conference that Lott organized. At Lott’s invitation, we submitted our work for a special issue of the Journal of Law and Economics that Lott arranged. A letter from an editor (Sam Peltzman, Professor of Economics at the University of Chicago) informed us that we would need to address the concerns of a referee who had reviewed our work before a decision would be made as to whether it would be published. A later letter for Peltzman informed us that our revision in response to the referee’s comments was accepted. Thus our work was not “put in” the Journal of Law and Economics by Lott but rather accepted for publication after scholarly review.
The basic issue between Lott and Levitt is not whether the results of either scholar can be replicated, but rather what statistical analyses are appropriate to make inferences about the effect on crime of allowing more citizens to carry guns. It is not unreasonable to require Levitt to find a way to express his analytical disagreement with Lott without implying that others who analyze the data in the way that Lott did do not get the same results. While lawyers are better suited to opine on what legal measures are appropriate to achieve this end, it seems to us not unreasonable, prima facie, to require a publisher to revise a sentence for future sales if the existing sentence is false and defamatory.

Sincerely,

Florenz Plassmann
Associate Professor of Economics, Binghamton University
Nicolaus Tideman
Professor of Economics, Virginia Tech

Will Donohue and Levitt predict that a massive new crime wave is coming?

Guttmacher Institute, associated with Planned Parenthood, claims that: "The rate of unintended births — unintended pregnancies carried to term — rose by 44 percent among poor women from 1994 to 2001, but declined by 8 percent for wealthier women." If true, I assume that those few who still believe that abortion massively reduces violent crime rates after the errors were discovered in the original research will be predicting that we will soon see a big increase in violent crime rates. If up to 80 percent of the changes in murder rates can be explained by abortion as has been argued, those who believe this relationship must be worried we could be in for big increases in murder. (Of course, the abortion is more likely to slightly increase crime.) People born in 1994 will become teenagers starting next year. Presumably, proponents of this theory will start warning people very soon.

News story on defamation suit: Lott v. Levitt

(Update) From the University of Chicago student newspaper the Maroon:

Freakonomics claim sparks defamation lawsuit
By Kim Velsey
April 21, 2006 in News
John Lott, Jr., a former visiting professor at the University filed a defamation lawsuit on April 10 against economic professor Steven Levitt, co-author of the New York Time bestseller Freakonomics

Lott said the book misrepresents his work on guns and crime, according to court documents. The lawsuit does not name journalist Stephen Dubner, though he co-wrote the book with Levitt.

Freakonomics, which melds Levitt’s economic essays with Dubner’s flowing prose, remains high on the bestseller list. The book’s success, however, may have prompted the legal action, as the lawsuit references the popularity of Freakonomics as a factor contributing to Lott’s damaged reputation.

The lawsuit states that the book “damages Lott’s reputation in the eyes of the academic community in which he works, and in the minds of hundreds of thousands of academics, college students, graduate students, and members of the general public who read Freakonomics.”

The contested material is on pages 133–134 of Freakonomics, in which Levitt writes that researchers have been unable to confirm Lott’s conclusion that right-to-carry gun laws actually reduce crime.

Freakonomics states, “Then there was the troubling allegation that Lott actually invented some of the survey data that supports his more-guns/less-crime theory. Regardless of whether or not the data was faked, Lott’s admittedly intriguing hypothesis doesn’t seem to be true. When other scholars have tried to replicate results, they found that right-to-carry laws simply don’t bring down crime.”

According to the lawsuit, Lott acknowledges that his findings have come under scrutiny in the academic community, but he maintains that he used “different data or methods to analyze the relationship between gun control laws and crime.”

The lawsuit states that scholars who have replicated Lott’s work have achieved the same results. “Every time that an economist or researcher have tried to replicate [Lott’s] results, he or she has confirmed Lott’s conclusion.”

Carl Moody, a professor of economics at the College of William and Mary, said he successfully replicated Lott’s findings and published the results in 2001. Moody said Levitt’s accusation is wrong.

The lawsuit, which also names Levitt’s publisher HarperCollins, states that the publisher acted with malice by failing to verify the statements. It seeks a court order to halt sales of Freakonomics until the statements are retracted or amended and also demands that Levitt and HarperCollins pay unspecified monetary damages.

HarperCollins would not comment on the lawsuit, but a company representative said, “HarperCollins Publishers firmly stands behind Freakonomics and its authors, Steven Levitt and Stephen Dubner.”

The ABC show 20/20 featured Freakonomics in an hour-long special on April 14. However, there was no mention of the lawsuit, and Levitt has yet to comment on it publicly. The book’s website, Freakonomics.com, which has Levitt’s and Dubner’s weblogs, includes a brief mention of the pending litigation.

“While we were away [in London promoting the paperback edition of Freakonomics], the economist John Lott filed a lawsuit claiming that Freakonomics has libeled him,” wrote Dubner on his blog.

Lott’s website made no mention of the lawsuit.

The lawsuit has opened up discussion on the veracity of Levitt’s claims and whether a lawsuit is an appropriate forum for an academic debate.

The litigation has also shed light on what can happen when an academic book attains blockbuster status.

“Most academic debate is so trivial no one would care,” Moody said. “If the book had appeared and no one had bought it, it wouldn’t be an issue. But Levitt is accusing this guy of falsifying his results in front of millions of people.”

Here is an earlier story from the Chicago Tribune:

A scholar known for his work on guns and crime filed a defamation lawsuit Monday against University of Chicago economist Steven Levitt, co-author of the best-seller "Freakonomics."

John Lott Jr. of Virginia, a former U. of C. visiting professor, alleges that Levitt defamed him in the book by claiming that other scholars had tried and failed to confirm Lott's conclusion that allowing people to carry concealed weapons reduces crime. Publishers Weekly ranked "Freakonomics" eighth this week for non-fiction hardcover books.

According to Levitt's book: "When other scholars have tried to replicate [Lott's] results, they found that right-to-carry laws simply don't bring down crime."

But according to Lott's lawsuit: "In fact, every time that an economist or other researcher has replicated Lott's research, he or she has confirmed Lott's conclusion."

By suggesting that Lott's results could not be replicated, Levitt is "alleging that Lott falsified his results," the lawsuit says.

Lott is seeking a court order to block further sales of "Freakonomics" until the offending statements are retracted and changed. He is also seeking unspecified money damages.

Lott acknowledged in the suit that some scholars have disagreed with his conclusions. But he said those researchers used "different data or methods to analyze the relationship between gun-control laws and crime" and made no attempt to "replicate" Lott's work.

The lawsuit alleges that Levitt and his publisher, HarperCollins Publishers Inc., made the statements with reckless disregard for whether they were true and that the book damaged Lott's reputation.

Neither Levitt nor HarperCollins officials could be reached Monday.

According to the lawsuit, Levitt also defamed Lott in an e-mail that Levitt sent to an economist in Texas last May. The e-mail described work that Lott published in an academic journal in 2001. It falsely stated that Lott's work had not been peer-reviewed and that Lott had blocked scholars with opposing views from appearing in the same issue of the journal, the lawsuit said.

Lott's books include "More Guns, Less Crime: Analyzing Crime and Gun Control Laws," published in 1998. Levitt won the John Bates Clark Medal for economists younger than 40 from the American Economic Association in 2003.

The case was assigned to U.S. District Judge Ruben Castillo.

From the Chronicle of Higher Education:

Last week Mr. Lott filed a defamation lawsuit against Steven D. Levitt, a professor of economics at the University of Chicago and a co-author of the best-selling Freakonomics: A Rogue Economist Explores the Hidden Side of Everything (William Morrow, 2005). (A copy of the complaint can be found on the blog Overlawyered.) Mr. Lott charges that in the book and in private e-mail messages, Mr. Levitt spread lies about the quality and integrity of Mr. Lott's work (The Chronicle, April 13). Much will hinge on exactly what Mr. Levitt meant by the words "replicate" and "peer refereed."

Interesting conference on the abortion and crime link

Today we just had an interesting conference on the abortion and crime link. There is a video of the event. We invited all the Americans who had finished work on this topic by the middle of February, so while it might appear that almost everyone who had finished a paper thought that the link between abortion and crime was nonexistent or possibly even positive. There was one paper that claimed to find evidence in favor of Donohue and Levitt, but that when the issues raised by Chris Foote were dealt with the significance of the effect went away. Video is available of the two sessions at the top right of the page.

The Economist Magazine on Steve Levitt and Abortion

Of course, lots of people have always thought Mr Levitt was in the wrong. Even if abortion cuts crime, it is still immoral, they fulminate. But this is largely beside the point: Mr Levitt's research does not take a position on abortion's social virtues, but aims merely to uncover its societal effects. Besides, for someone of Mr Levitt's iconoclasm and ingenuity, technical ineptitude is a much graver charge than moral turpitude. To be politically incorrect is one thing; to be simply incorrect quite another.

But what is most amazing is that despite all the statistical significance being eliminated from their panel data set being eliminated when the results are done Donohue and Levitt's way with the the data they want to use, John Donohue now says what they wanted to do:

It may be asking too much of the numbers to convince everybody. “The debate over abortion and crime will not be resolved within the parameters of our paper,” says Mr Donohue. He thinks the arrest figures are “muddy” and the state population data “sloppy”. Combining the two generates so much noise, it is hard for the statistical tests to hear anything. Ted Joyce, a professor at Baruch College (part of the City University of New York), who has had his own methodological disagreements with Messrs Donohue and Levitt, also thinks the debate is stretching the data too far. He points out that if you add controls for 50 states and 12 years—as Messrs Foote and Goetz do, and as Messrs Donohue and Levitt meant to do—you are, in effect, holding another 600 things constant. This robs the data of most of their variety, and of much of their ability to explain anything.

There was no warning that these were the wrong tests at all in either of their previous papers or in Levitt's book. Levitt now refers to the “collage of evidence," but the panel data was the only test that really amounted to anything worthwhile. The rest of the data was merely cross sectional or time series.

Incorrect claims about the data in More Guns, Less Crime

I have received an email comparing my work to that of Steve Levitt's regarding claimed coding errors in More Guns, Less Crime. Of course, this discussion is not very accurate.

1) There are no coding errors in the data used in “More Guns, Less Crime.” The book used crime data from 1977 to 1996, and as far as I know, no academics have claimed that there were any coding errors in that data. An interesting and useful Stanford Law Review article by Plassmann and Whitley added an additional four years to the data and the problem arose in this additional data. Overall, less than 200 cells out of 7.5 million cells were accidentally left blank. More important, the results that Plassmann and Whitley noted were the results which they thought were the correct ones were not affected at all by this minor change in the data.

2) I had put the data set on my website as a favor to Plassmann because it was very large and he did not have the ability to put it up for people to download. The data was corrected as soon as the missing cells were discovered and it was available for people to download. A note was added to the site to alert people to the correction. While I had helped them out on their paper, I let Plassmann and Whitley make their own statements about their results. My website reported the regressions in the same way that I had done them in all my previous estimates.

3) In statements here and here, I have discussed what was involved in Levitt's errors. The regressions where they left out the fixed effects were essentially the only ones that were at all useful in testing their hypothesis because they were the only ones that didn't require large degrees of aggregation to get to their "effective abortion rate" that were related to total murder rates in a state. The work that I did with Whitley directly related the age of the murder to the number of abortions when that murderer was born.

Thanks to Bob Thomas for his email.

Levitt's Abortion "Disappeared" When "Programming" Error Fixed

Levitt's Abortion results disappeared when programming Error Fixed

"The Boston Fed's Mr. Foote says he spotted a missing formula in the programming oversight of Mr. Levitt's original research. He argues that the programming oversight made it difficult to pick up other factors that might have influenced crime rates during the 1980s and 1990s . . . . He also argues that Levitt should have counted arrests on a per-capita basis. Instead, he coundted overall arrests. After he adjusted for both factors, Mr. Foote says, the abortion effect disappeared.

Christopher L. Foote and Christopher F. Goetz's paper can be found here. Personally, I think calling this a "programming oversight" is being much too nice (See my post here to see an example of what you would have to miss to not notice whether you have used fixed effects). More importantly everyone who works with panel data knows that you use fixed effects.

My own work concentrated on murder rates, but I also included fixed effects. Donohue and Levitt never provided us with all their data or their regressions and would never answer any questions that we had so I just assumed that they had included fixed effects from the beginning. It would have been nice if they had provided us with this same information years ago.

My paper with John Whitley is available here.
A letter that I had in the WSJ is available here.
James Q. Wilson's review of this debate is available here.

UPDATE:

See also Steve Sailer's past work on this topic.
See also Steve Sailer's compliation of what others are saying here.

UPDATE II (I have been asked about inaccurate claims regarding my own research in More Guns, Less Crime):

See this for a discussion about claimed errors in More Guns, Less Crime

Revisiting John Donohue's cancelling of our gun debate last fall

1) It was very disappointing that John Donohue backed out of our debate on guns last December with just two days to go. I had looked forward to the debate, and obviously had my plane tickets well in advance. It was a date that Donohue had picked (I had said that I could do all but one day over the end of November through December and I would have gone beyond December if necessary), and the first email that I had confirming the event was received in September. I ended up speaking at Chicago on another topic so that a later debate could be attempted to be set up again with Donohue.

2) Unfortunately, the second planned debate was also cancelled. The debate was rescheduled for April 13th this year, and I made sure to reconfirm it because of the previous cancelation. The student at Chicago who set up the debate said that even though he had confirmed the debate with me multiple times and even though we had taken a date that I was told that Donohue wanted, the claim is that the debate somehow hadn't been completely confirmed with Donohue. (You would think that if the previous cancelation was an accident, Donohue would have been careful to double check this time just in case there had been any possible misunderstandings, but no one is claiming that he checked anything.) Attempts were made to get Donohue's coauthor Steve Levitt to substitute for Donohue and debate either guns or abortion claims were unsuccessful. From an email that the organizer sent me after the second debate was cancelled: "You have been the most patient and flexible speaker I could imagine . . . ."

3) In conversations with Joe Cascio (the speaker coordinator for the UC Fed Soc Chapter), I had said several times if we were going to schedule a third try at a debate, we should do it on the abortion and crime claims, especially given the amount of attention that this poorly done empirical claim had gotten. Despite Donohue claiming that the debate would actually take place this time, he was unwilling to debate this hot topic. Steve Levitt was unable to debate (he was debating the topic but only with people who didn't seem to know much about the issue) and Dubner, Levitt's coauthor, was unwilling to debate the abortion claims. The University of Chicago Federalist Society was again willing to have me give another talk that wasn't in a debate format, but they insisted that I remove the statements regarding the canceled debates from my website. (They simply viewed my posts about the April 13th event being critical of the Federalist Society (I disagree), and the earlier post that they wanted removed because they didn't want to be involved in a debate about a debate.) I told them that I would correct anything that they told me was wrong, but I wasn't going to remove the postings. If people set up debates and back out at the last moment, they should be held accountable.


On the substance of Donohue's work on guns, I would direct people to this paper by Plassmann and Whitley. There are two straightforward points that they make. 1) That the graphs at the beginning of the Ayres and Donohue work are very misleading because they don't make it clear that the sample of states is changing. The crime rates clear fall for a decade and a half and the sudden increase isn't real for the remaining states. It just looks like an increase because the couple of rural states that remain did not experience the decline in murder rates that more urban states experienced after right-to-carry laws were passed. 2) The second claim has to do with them fitting an intercept and line to the data and then limiting their reported impact on crime to only five years. Often there is no problem with this, but as Plassmann and Whitley clearly show it doesn't fit in this case. The Ayres and Donohue approach implies that crime initially increase (despite the fact that the year by year data discussed in point (1) doesn't, and it is just an artifact of their estimates over predicting in the early years because they are fitting this straight line with an intercept shift to crime that is falling at an increasing rate. But to compound the problem, they only discuss what the estimates mean for the first five years. Even with this approach if they had picked the sixth year it would have reversed their claims.

I also think that it is pretty clear why they find it difficult to debate either guns or abortion. Steve Landsburg was very nice to suggest in April that Levitt and I have a discussion at Rochester this fall, but I haven't heard anything back. I immediately wrote back saying that I was interested. Landsburg wrote in April that:

"Our department would like to arrange, sometime in the fall, to have you and Levitt visit simultaneously and give back-to-back workshops on guns, abortion or both. We want to hear your criticisms of each others' work and your responses to those criticisms. We *don't* want this to turn into a debate; we want it to be an academic seminar with all the usual rules of logic and evidence. The presenter is presumed to have the floor; there is an expectation of periodic interruptions from the audience; but there's also someone in charge to cut off questioning if it starts to take over the seminar. We would most assuredly *not* advertise this event to the general public, because we don't want an audience of advocates for any point of view. Ideally, we'd have you both here for two days, with you presenting on one day and Levitt presenting on the other. We are open to alternative suggestions. We'd pay you both an honorarium of some size to be determined, though surely far less than you deserve. We'll do what we can to make it attractive, though."

I am posting this because of an email that I received asking questions about Donohue's cancelation last fall.

Abortion and Crime "debate" at the University of Chicago Scheduled for May 25th

The University of Chicago Federalist Society has tried for a third time to set up a debate between myself and John Donohue. Since the last two debates on the issue of guns at the University of Chicago were canceled at the last moment with Donohue withdrawing from one debate with just 2 days to go, I thought that we might have more luck scheduling a debate on another topic that is getting a lot of attention these days: abortion and crime. Donohue and Steve Levitt were the coauthors on a paper that got a lot of attention on this issue and Levitt as also recently coauthored a book with Steve Dubner that again goes over the issue. All three were asked to pick a time to debate the issue, but even though Donohue is free on the 25th and despite all the attention currently being given to the abortion research, none of them were willing to debate their work on abortion with me. (I think that I know why.) I will still be presenting on the 25th with the hope that Donohue will change his mind and defend his research.

Further discussions about John Donohue, including his backing out of one of the two previously scheduled debates on guns, are here.