The "prosecutor's fallacy" in the Netherlands

A New York Times blog by Mark Buchanan on "The Prosecutor's Fallacy" provides a report on a conviction involving flawed statistical evidence. (The author also described the case in January in Nature. A more detailed analysis is at Richard Gill's website.)

Dutch nurse Lucia Isabella Quirina de Berk is serving a life sentence for seven murders and three attempted murders. Mr. Buchanan writes that:

Following a tip-off from hospital administrators, investigators looked into a series of “suspicious” deaths or near deaths in hospital wards where de Berk had worked from 1999 to 2001, and they found that Lucia had been physically present when many of them took place. A statistical expert calculated that the odds were only 1 in 342 million that it could have been mere coincidence.

After noting that the data collection was badly flawed and that "a more accurate number is something like 1 in 50," he adds that:

More seriously still – and here’s where the human mind really begins to struggle – the court, and pretty much everyone else involved in the case, appears to have committed a serious but subtle error of logic known as the prosecutor’s fallacy.

The big number reported to the court was an estimate (possibly greatly inflated) of the chance that so many suspicious events could have occured with Lucia present if she was in fact innocent. Mathematically speaking, however, this just isn’t at all the same as the chance that Lucia is innocent, given the evidence, which is what the court really wants to know.

Now, there are a great many instances of this fallacy of naively transposing P(E|H), the probability of the evidence E given the hypothesis H, into P(H|E), the probability of H given E. But surely this is not a more serious error than being off by a factor of some 10,000,000 in the estimate of P(E|H)! A p-value of 1/342,000,000 will be associated with a likelihood function that swamps any plausible prior probability distribution. See M.H. DeGroot, Doing What Comes Naturally: Interpreting a Tail Area as a Posterior Probability or a Likelihood Ratio, 68 J. Am. Stat. Ass'n 966 (1973).

Of course, Mr. Buchanan is right about one thing: the fallacy is subtle. He makes it when he describes the p-value as follows: "the odds were only 1 in 342 million that it could have been mere coincidence." The kind of statistics that appear to have been used in the case do not permit any quantitative statement about the probability that coincidence it the explanation for the nurse's presence. At best, they allow one to say that if that explanation is correct, it is exceedingly improbable that she would be present as often as (or even less often than) she was. Thus, if one wants avoid the transposition fallacy, one must say something like this: assuming the nurse's presence was mere coincidence, the probability of her being present at least as often as she was is only 1 in 342 million.

DHK