May 30, 2008
The Transposition Fallacy in Brown v. Farwell
Earlier this month, Ninth Circuit held in Brown v. Farwell, No. 07-15592 (9th Cir. May 5, 2008) that a prisoner was denied due process of law because of a mistake involving DNA evidence. Troy Brown had been tried and convicted in Carlin, Nevada, for sexual assault. A federal district court granted Brown's petition for habeas corpus relief because the evidence against him was insufficient to prove guilt beyond a reasonable doubt. The court of appeals affirmed. As the majority opinion by Judge Wardlaw summarizes the case, the state's "DNA expert Renee Romero of the Washoe County Sheriff's Office Crime Lab ... provided critical testimony that was later proved to be inaccurate and misleading. ... [A]bsent this faulty DNA testimony, there was not sufficient evidence to sustain Troy's conviction."
The court identified two faults in Romero's testimony. One involves the chance that one of Brown's four brothers would have the same DNA profile. The other is the transposition fallacy that has been recognized in court opinions at least since the California Supreme Court's famous opinion in People v. Collins, 438 P. 2d 33 (Cal. 1968). Twenty years after Collins, the Brown court wrote:
Here, Romero initially testified that Troy's DNA matched the DNA found in Jane's underwear, and that 1 in 3,000,000 people randomly selected from the population would also match the DNA found in Jane's underwear (random match probability). After the prosecutor pressed her to put this another way, Romero testified that there was a 99.99967 percent chance that the DNA found in Jane's underwear was from Troy's blood (source probability). This testimony was misleading, as it improperly conflated random match probability with source probability. In fact, the former testimony (1 in 3,000,000) is the probability of a match between an innocent person selected randomly from the population; this is not the same as the probability that Troy's DNA was the same as the DNA found in Jane's underwear, which would prove his guilt. Statistically, the probability of guilt given a DNA match is based on a complicated formula known as Bayes's Theorem, ... and the 1 in 3,000,000 probability described by Romero is but one of the factors in this formula. Significantly, another factor is the strength of the non-DNA evidence. Here, Romero improperly conflated random match and source probability, an error that is especially profound given the weakness of the remaining evidence against Troy. In sum, Romero's testimony that Troy was 99.99967 percent likely to be guilty was based on her scientifically flawed DNA analysis, which means that Troy was most probably convicted based on the jury's consideration of false, but highly persuasive, evidence.
This analysis is less than convincing because it misportrays Bayes' theorem -- which is not particularly complicated here -- and it fails to consider what the theorem means. The formula can be written compactly with a few symbols. Let ST stand for the proposition that Troy Brown is the source and MT stand for the fact that his DNA matches that of semen from the victim's clothing. Let S0 be the hypothesis that some individual unrelated to Troy is the source. The theorem simply states that
Odds of ST given MT =
(Odds of ST without considering MT) x
(Probability of MT given ST / Probability of MT given S0)
The formula could be extended to deal with the hypothesis that one of Troy's brothers is the source, but that is not a feature of the transposition fallacy that worried the court. Transposition consists of equating (a) the probability that DNA would match if it had come from an unrelated individual with the (b) probability that the DNA came from an unrelated individual given that it matched. Conceptually, these are quite different, from the probabilities often are approximately equal.
Bayes' Theorem is essentially a recipe for going from (a) to (b). In Brown, the formula indicates that the statement that "there was a 99.99967 percent chance that the DNA found in Jane's underwear was from Troy's blood" may not be so far off — if the only alternative worth considering is S0. Suppose, for the sake of illustration, that the other evidence was so weak that, before considering the DNA match, it was 1000 times more likely that an unrelated individual was the source. Let us also assume that the DNA samples from the semen and the defendant have the profiles reported by the laboratory. The formula now shows that the odds that Troy is the source are (1/1000) x [1/(1/3,000,000)], or 3000 to 1. The corresponding probability is 99.96667%. This is smaller than the 99.99967% reported in the case, but is the discrepancy a violation of due process?
One might argue that the error in the second decimal place or beyond rises to this level because the witness's description of the “chance that the DNA ... was from Troy” invites a more serious error. It encourages the jury to think that this chance is 99.9+% even though the figure ignores the possibility that one of Troy's four brother's was the rapist as well the other evidence in the case. This seems to be the basis of the majority's concern that the "complicated formula" requires other "factors" to be considered.
Once the argument is framed this way, however, it no longer is an argument about the sufficiency of the evidence. It is an argument about prejudice in the manner in which sufficient evidence is presented, and it has to confront the fact that the jury was given a separate number for the chance that Troy and any one of brothers would match. Romero testified that the chance of a match between two full siblings at any locus was 1/4. The DNA match in the case involved five independent loci, so the chance of a brother's having the same DNA profile then would be 1/4 to the fifth power, or 1/1024. Somehow Romero came up with value of 1/6500 instead. (If anything, the true figure would be somewhat larger than 1/1024 because 1/4 is only the change of identical alleles at each locus by descent. It does not consider the possibility that both parents might share some alleles; however, for rare alleles, this won't make much difference.)
All this provides ample material for cross-examination to bring out the fact that the criminalist's testimony could not be trusted because: (1) the 99.9+% figure ignores the possibility that close relatives such as brothers would match; (2) the chance that a single brother picked at random would match was not 1/6500; and (3) the chance that one or more of Troy's brothers would match would be larger still. The last probability is approximately 1/512 for the two brothers living in Carlin and 1/256 if we toss in another two brothers living across the state line in Utah. (In the habeas proceedings, Troy supplied a report from a geneticist, Larry Mueller, that gave the figure of 1/66 for the chance that at least one of the other four brothers would match.)
Judge O'Scannlain grasped what all these numbers really proved. His dissenting opinion maintained that “[t]hus, it was extremely unlikely that a random person committed the crime, and of the brothers, it was extremely unlikely that the specimen DNA would match not only Troy—as it did—but another brother. These probabilities put together still constitute overwhelming DNA evidence against Troy which the jury was entitled to consider.” (Well, if Mueller's perplexing figure of 1/66 were correct, the DNA match, standing alone, might not seem so overwhelming. The whole inquiry into probabilities concerning siblings could have been avoided had a recommendation of the 1996 National Research Council report on DNA evidence been followed. The NAS committee recommended testing close relatives whenever it is suggested that they might have committed the crime. I would bet that none of the four brothers matched, but I should not have to guess.)
Nevertheless, even Judge O'Scannlain's more perceptive discussion of the hypotheses about unrelated people and brothers does not necessarily dispose of the case. It means, as I indicated earlier, that the real issue is not the sufficiency of the evidence. He shows that a rational jury that understood the evidence as he did could have been persuaded beyond a reasonable doubt. The problem is that the evidence, as it was actually presented, was garbled by the failure of the prosecution, the criminalist, and the defense attorney to deal with a few simple probabilities intelligently. The jury might well have been confused about what the DNA actually proved and given it undue weight. As such, the system might have failed, as it often does, because of the inadequacies of its participants.
At last, we arrive at the fundamental issue in Brown. Does this kind of a system failure amount to a deprivation of due process, given that the defendant had the opportunity to correct the prosecution's overreaching. The majority thought that it could pretermit this issue. It wrote that “[b]ecause we affirm the district court's grant of Troy Brown's habeas petition on due process grounds, we need not reach his arguments regarding ineffective assistance of counsel.” But because the sufficiency theory of the majority is doubtful, the case leaves unresolved the basic questions -- Did the prosecutor's (apparently) negligent presentation of scientific evidence in this case deprive the defendant of due process? Did the defense attorney's (apparent) failure to challenge the prosecution's presentation deprive Troy Brown of a constitutional right?