« Grand Bargains | Main | A Different Kind of Hack Gap »

Lancet Redux

Daniel Davies provides a stirring defense of the methodology used in the Lancet survey of postwar Iraq mortality. Count me as convinced. We can say with a high (i.e., 95 percent) degree of confidence that as a result of the war somewhere between 8,000 and 196,000 more deaths have occurred in Iraq than would have occurred had the pre-war status quo continued. I would, however, like to re-emphasize what Daniel admits to be a legitimate critique of the Lancet article, namely that contrary to what the Lancet's editors said, this is not a study of civilian casualties, it's a study of excess deaths. As those of us who've been following the news know perfectly well, an awful lot of the dead people in Iraq are not civilians, but either Interim Government security forces or else insurgents. And then on top of that, there are a lot of civilians.

November 12, 2004 | Permalink

TrackBack

TrackBack URL for this entry:
https://www.typepad.com/services/trackback/6a00d8345160fd69e200d8346b0ef369e2

Listed below are links to weblogs that reference Lancet Redux:

Comments

actually I think that it should be rephrased as: 'the likelihood that if nothing really has changed, the range of deaths would be in the range indicated is less than 5%'. It's the classic Bayesian reversal problem

Posted by: Suresh | Nov 12, 2004 2:31:03 PM

"I think that the simple fact that we can say with 97.5% confidence that the war has made things worse rather than better is just as powerful "

Nah. Proponents of the war would say that many or most of the dead are Baathist thugs, or that "freedom" or some other positive consequence of the war is worth x number of civilian dead. The size of x is very important to a reasonable justification of the war. At least it is to me. Doesn't a "normalized distribution" mean that there is a 50/50 chance of 98000 excess deaths?

On the other hand, millions of Vietnamese died, and I rarely hear it mentioned as a factor in that war by those who defend it.

Posted by: bob mcmanus | Nov 12, 2004 2:34:01 PM


"between 8,000 and 196,000 more deaths"

This is about as useful as saying that Kerry (or Bush) will end up with between 4% and 98% of the vote.

The other point is, more than what? Since we don't have finely detailed numbers available for the folks Saddam was feeding to lions, running through wood chippers, providing a third nostril in the back of the head, et al, its really a guess if there are more or fewer deaths.

Posted by: Abdul Abulbul Amir | Nov 12, 2004 2:41:30 PM

Well Abdul, that is what the study looks at. But hey, spreading false stories about paper shredders is almost as good as actual research.

Posted by: Rob | Nov 12, 2004 2:47:49 PM

Abdul Abulbul Amir writes:

"between 8,000 and 196,000 more deaths"

... more than what?

Before the war. That's precisely what the study was looking at. They collected information on the number of people who died in a period before the war, and then an equivalent period after the war. There is no more reason to think they undercounted pre-war deaths (including from the imaginary wood chippers) than there is to think they undercounted post-war deaths.

Posted by: A Tiny | Nov 12, 2004 2:53:29 PM

more than what? Since we don't have finely detailed numbers available for the folks Saddam was feeding to lions, running through wood chippers, providing a third nostril in the back of the head, et al, its really a guess if there are more or fewer deaths.

If you have not even read ( or understood) the study, don't comment on it.

We know exactly "more than what," as that was the point of the study. They tried to determine the differential in death rates, pre- and post-Saddam. Pre Saddam, most deaths were due to heart attack. Post Saddam, most deaths are due to violence.

So more people are dying now, IN TOTAL, in Iraq, than were dying before, under Saddam, to the standard degree of certainty used in a lot of statistics.

And if you think a wide confidence interval makes it useless, you don't understand. You could also go risk getting your ass shot by doubling the number of data points, if you like. As it stands, this study is the only science we have that addresses the issue of the Iraqi mortality rate, and if we increased it.

Posted by: Timothy Klein | Nov 12, 2004 3:16:07 PM

This is about as useful as saying that Kerry (or Bush) will end up with between 4% and 98% of the vote.

Of course, you could have read the Crooked Timber thread, which talks about the actual distribution, before outing yourself as another idiotic hack. But that would have been far too difficult.

But perhaps you can go to Iraq and come up with some better numbers? You've got the (fake) name for it...

Posted by: ahem | Nov 12, 2004 3:41:00 PM


Since the bodies are still being uncovered from mass graves, there seems to be no very reliable "before" number. Are we to believe that the mass graves are largely filled with heart attack victims?

One question about the study, if a new mass grave is discovered with say, 10,000 bodies does that of necessity change the before number?

BTW, just because a guesstimate is done by a scientist in a scientific manner does not mean that it is any more accurate than the quality of the data.

Posted by: Abdul Abulbul Amir | Nov 12, 2004 3:42:38 PM

From the spirited defense. No survey from 1999. An _untested_ assumption that that mortality rates are similar to other countries after 1999. GIGO. The number of excess deaths may be much higher than 100,000 or it may be negative.

"No detailed on-the-ground survey has been carried out since 1999, and there is certainly no systematic data-gathering apparatus in Iraq which could give any more solid number. The authors of the study believe that the infant mortality rates in neighbouring countries are a better comparator than pre-oil for food Iraq, and since one of them is Richard Garfield, who was acknowledged as the pre-eminent expert on sanctions-related child deaths in the 1990s, there is no reason to gainsay them."


Posted by: Abdul Abulbul Amir | Nov 12, 2004 3:58:29 PM

Abdul, perhaps you'd also like to give us your take on Mary Darby Robinson's Vancenza, or the Dangers of Credulity. Or something else you haven't read.

Posted by: Sven | Nov 12, 2004 4:50:24 PM

I rate the Davies screed a Bond Martini - shakey, not stirring.

The main thing wrong with the Lancet study is the goofy methodology. Picking sample locations using random GPS coordinates, then checking for possession of death certificates by selected residents followed by extrapolation through statistical methods seems needlessly baroque. If death certificates are acceptable proxies for actual deaths, why not just ask the local government issuers of same how many of the things they've issued lately? Were the researchers to propose conducting a study of violent deaths in, say, Los Angeles using their cumbersome and indirect approach, I hope someone would pipe up, "Umm, couldn't you just call the County Hall of Records?"

In most of Iraq, there is little or no combat going on. Even incidents of terrorism against civilians are infrequent except in Baghdad and a few cities in Al Anbar province. Baghdad has a functioning city administration as do most other places in Iraq. The places that do not are, as the study admits, difficult to sample properly while crazed Jihdis are running around blowing things up and shooting people - even when the U.S. military is not. Either the Lancet's way or mine, we are unlikely to have usable data from Falloujah and the handful of lesser pestholes with which it keeps formation until they are suitably pacified.

I fail to see how ignoring an obvious, and likely far more accurate, source of data on violent civilian deaths than the Rube Goldberg machine constructed by the Lancet researchers makes it likelier that anyone with sense would credit either the accuracy of the alleged results or, indeed, the alleged motivational purity of the researchers. Indeed, Mr. Davies reiterates his assertion of the high-minded character of the Lancet researchers so often that I am reminded of the old saying, "When a man says, 'My word is my bond' you might want to check the whereabouts of your wallet."

I see this "study" as just another in a seemingly endless series of efforts by the Left to clothe a number some partisan hack pulled out of his ass in pseudo-scienctific drag. It can take its place in the Anal Archive alongside the 50,000 Americans who die each year from second-hand smoke, the 110,000 girls who die each year from eating disorders and the "record" number of female significant others battered each year by the churlish louts driven to frenzies of misogyny watching the Super Bowl.

Posted by: Dick Eagleson | Nov 12, 2004 4:56:43 PM

>As those of us who've been following the news know perfectly well, an awful lot of the dead people in Iraq are not civilians, but either Interim Government security forces or else insurgents

On the other hand, query whether they would be dead were it not for the American invasion. One might seriously ask why they should not be included among the "excess dead."

Posted by: raj | Nov 12, 2004 5:54:07 PM

On a side note, whatever people think of anything here, one should refrain from saying "but this is science!" as a defense.

_The Bell Curve_ was the result of science. As was the abortion/breast cancer sham. As was the okay for Thalidimide. As was the purported link between coffee and lung cancer. And on and on.

The question is whether the science is done well. And in the social sciences, this is always a particularly thorny question (this is not a 'bash' of social science. They are simply dealing with more complex and/or difficult to control variables than the physical and life sciences). The whole point of publishing scientific articles is to foster discussion of what their merits are. They are not a last word, they are in some sense a first word.

It goes without saying that mindless hack criticisms of the study are without merit, but nobody should act as if the mere act of questioning the merits of the methodology is in itself somehow biased or unwarranted.

This is exactly why politically charged research is so difficult to do well. Everyone can calmly discuss whether goldfish use X or Y method of mate selection. Once a finding supports/refutes a politically hot point of view, that calm discussion often gets lost in heated rhetoric.

Posted by: skippy | Nov 12, 2004 5:54:54 PM

>I see this "study" as just another in a seemingly endless series of efforts by the Left...

Shorter Dick Eagleson: I don't like the results, so I'll call them "Left"

Posted by: raj | Nov 12, 2004 5:55:46 PM

Abdul Abulbul Amir writes:

One question about the study, if a new mass grave is discovered with say, 10,000 bodies does that of necessity change the before number?

Here's a simple answer to a simple question: no.

Obviously it wouldn't change the before number, any more than counting the graves in Fallujah from US attacks on the city would change the after number.

Do you understand why this is? I assume you do, and are simply asking your question for reasons known only to yourself.

Posted by: A Tiny | Nov 12, 2004 6:06:15 PM

Abdul Abulbul Amir: You have absolutely no idea what you are talking about. None. Nada. Zilch. Zip.

Quit embarrassing yourself.

Mass graves? Are you daft? People that die, guess what? They knew other people. This study relies upon absolutely no public knowledge of mass graves, or Iraqi records of death, or anything of the like.

This survey went to 900 some odd house scattered about Iraq, they sat down with the families, and they asked them who, if anyone, in their house had died in the last few years, and how.

That is not a guesstimate -- that is a legitmate method of trying to measure a difficult quantity. If you think statistics is just "guessing," either go buy a text book on the subject and educate yourself, or stop the inane, pseudo-informed blather.

There are reasons why this study could be wrong, and the numbers should be accepted with caution, but you are just spouting crap about something you know nothing about.

Posted by: Timothy Klein | Nov 12, 2004 6:30:04 PM


"This survey went to 900 some odd house scattered about Iraq, they sat down with the families, and they asked them who, if anyone, in their house had died in the last few years, and how.

That is not a guesstimate -- that is a legitmate method of trying to measure a difficult quantity. If you think statistics is just "guessing," either go buy a text book on the subject and educate yourself, or stop the inane, pseudo-informed blather."


So they did not go to any houses where the whole family was dead and gone and thus no one to interview. If this methodology were used in Germany in 1946 you would no doubt come up far short of 6 million as whole families disappeared.

Further many of those in the mass graves would be known to be dead only to the killers. If uncle Abdul went missing, his fate may never be known.

Posted by: abdul abulbul amir | Nov 12, 2004 7:04:36 PM

Have any of the commenters here actually read the Crooked Timber articles, let alone the actual Lancet study?

Posted by: Hamilton Lovecraft | Nov 12, 2004 8:44:52 PM

Okay, a few things.
The Bell Curve is a legitimate study, and should not be included with the abortion-breast cancer link (which contained a grain of truth but was overall false - abortion does not cause breast cancer, but an aborted pregnancy does not have the anti-breast cancer benefit of a pregnancy carried to term) and the okay of thalidomide.
"Either the Lancet's way or mine, we are unlikely to have usable data from Falloujah and the handful of lesser pestholes with which it keeps formation until they are suitably pacified."
Which is why results from Fallujah were left out of the survey that produced the 100,000 number.
"If death certificates are acceptable proxies for actual deaths, why not just ask the local government issuers of same how many of the things they've issued lately?"
I'm assuming that a lot of that data has been lost or is difficul to get to?

Posted by: Glaivester | Nov 12, 2004 10:26:54 PM

Wonderful new slogan for the US: "Sure, we kill Iraqis faster than Saddam did, but at least they died free (in the sense that freedom's just another word for nothing left to lose)."

Posted by: bad Jim | Nov 13, 2004 12:06:19 AM

"Legitimate" and "The Bell Curve" shouldn't even be used in the same sentence...

Posted by: Scott Lemieux | Nov 13, 2004 12:45:44 AM

Have any of the commenters here actually read the Crooked Timber articles, let alone the actual Lancet study?

I've read the entire paper that the authors of the Lancet study wrote (it's short), thanks. I don't need any one to explain the statistics for me -- there is nothing wrong with them. Their methodology seems like standard fare medical case-work type stuff. They use a modelling technique that is somewhat complicated, but fully justified by the circumstances.

Posted by: Timothy Klein | Nov 13, 2004 1:40:38 AM

Abdul etc has come up with one legitimate point.

The study did look at households that had surviving members. It will underrepresent households where everybody died and all their relatives died.

So we have excess deaths among surviving households. We don't have a handle on households that were completely exterminated by Saddam in his last year or so, versus households that were completely exterminated by airstrikes etc since the invasion began.

Hearsay has it that Saddam wasn't doing much of that toward the end. And we were doing a lot of airstrikes. It would be possible to do a study that estimated such things. You could ask people how long they've had the house, and when it was a short time, what happened to the ones who had it before. THey're likely to know if the previous family got exterminated by Saddam. And it would be possible to ask them about nearby demolished houses.

My guess is that the results would tend to be nil. As people have pointed out, there's a claim that the country is mostly peaceful outside Baghdad and places where there are americans, and to the extent that's true there won't be a lot of airstrikes. Similarly, Saddam's exterminations tended to be at particular places. He didn't just kill 1% of the population scattered evenly. So that will probably be mostly missed too.

I'm not volunteering to do the study.

Posted by: J Thomas | Nov 13, 2004 1:47:32 AM

J Thomas | November 13, 2004 01:47 AM

Just to point out, you refer to something that Donald Rumsfeld would probably refer to as an "unknown unknown."

Posted by: raj | Nov 13, 2004 8:53:11 AM

No, this would be a known unknown. And the study could be extended to estimate it, turning it into a partly-known known.

Rumsfeld has inspired me to a new set of four types.

There are the things we want to know the truth about, and so we go out and look. These are active truths.

And there are the things we don't want to know about but we'll believe when they force themselves to our attention. These are passive truths.

Then there are the ones we are eager to falsify. These are active untruths.

And finally there are the things we will believe completely independent of any evidence. Passive untruths.

Not enough zing, but then I only just did it. Anybody who wants to pep it up is welcome to.

Posted by: J Thomas | Nov 13, 2004 9:18:11 AM

The comments to this entry are closed.