August 11, 2008
Hot Tubbing: Old Wine in New Bottles for Expert Witnesses
The New York Times has discovered that expert witnesses retained by parties often are partisan. This certainly is fit to print, but is it news? Not to anyone who has been reading law reviews and opinions written during the past century or two. (For a good recent analysis, see Bernstein (2008).)
Still, the Times revealed that the Australians have discovered a way to improve expert testimony. They call it "hot tubbing.":
In that procedure, also called concurrent evidence, experts are still chosen by the parties, but they testify together at trial — discussing the case, asking each other questions, responding to inquiries from the judge and the lawyers, finding common ground and sharpening the open issues.
Interestingly, "Australian judges have embraced hot tubbing." According to UCLA law professor Jennifer Mnookin, "[t]he future ... may belong to Australia. 'Hot tubbing,' she said, 'is much more interesting than neutral experts.'”
If so, the movement will resemble the breakthrough of the Beatles "from Hamburg." (In 1961, when the band returned to Liverpool from Germany and made an appearance at The Cavern Club, some in the audience thought they were watching a German band.) Fifteen years ago, when I gave a traveling series of seminars to federal judges under the auspices of the Federal Judicial Center, I suggested that the judges experiment with this format. At least one judge was intrigued, saying that it sounded like the McLaughlin Show, but might be worth a try.
The idea certainly did not originate with me. I got it from a 1989 report of a blue-ribbon panel on statistics in the courtroom. The panel observed that
F.R.E. [Federal Rule of Evidence] 611 allows the trial judge to exercise power over the presentation of evidence to make it more effective and efficient. Many judges have used that authority in innovative ways to modify the traditional sequencing of evidence. For statistical matters, there are a variety of approaches tnhat might be attempted. When the reports of witnesses go together, the judge might allow their presentations to be combined and the witnesses to be questioned as a panel discussion ... .
Panel on Statistical Assessments as Evidence in the Courts (1989, 174). But "hot tubbing" is a lot catchier than "panel discussion," and the right packaging sells a product.
David E. Bernstein, Expert Witnesses, Adversarial Bias, and the (Partial) Failure of the Daubert Revolution, 93 Iowa L. Rev. 451–489 (2008)
Adam Liptak, American Exception: In U.S., Partisan Expert Witnesses Frustrate Many, N.Y. Times, Aug. 11, 2008, available at http://www.nytimes.com/2008/08/12/us/12experts.html?_r=1&8au&emc=au&oref=slogin
Panel on Statistical Assessments as Evidence in the Courts, The Evolving Role of Statistical Assessments as Evidence in the Courts (Stephen E. Fienberg ed. 1989)
The Birthday Problem in Las Vegas
The other week, an editorial in the Las Vegas Review-Journal misconstrued the now infamous 2001 findings of partial matches in the Arizona DNA database. The study was discussed on our blog on July 20, and I won't repeat the explanation of the birthday problem. You might think that people in Las Vegas would know more about winning combinations, but the editor presented the Arizona findings as proof that hundreds of thousands of Americans would be falsely incriminated by DNA profiling in criminal cases. His conclusion: "the odds of a 'coincidental match' with an innocent party -- the realistic odds, based on searches such as [the one in the Arizona database], not something out of astronomy book -- should be carefully explained."
The last remark got me thinking. Could the results of Arizona database study give an estimate of a random-match probability?
Mathematician Charles Brenner did some simplified calculations that can be adapted to this end. A standard DNA profile consists of 13 pairs of numbers. The numbers have to do with the lengths of various fragments of DNA at particular points (loci) on certain chromosomes. If a suspect and a crime-scene sample have the same fragment lengths at all 13 loci, then the match is strong evidence that the suspect (or an identical twin) is the source of the DNA. A partial match excludes the suspect as the source, and there are many ways for two 13-locus profiles to match in part but not in full. For example, Brenner pointed out that there are 715 ways to select 9 loci from the full 13. In the Arizona study, the analyst looked at all distinct pairings of the 65,493 people in the 2001 Arizona database. (This is where the birthday problem, with its combinatorial explosion, comes in.) Brenner reported that 65,493 x 65,492/2, or approximately 2,140,000,000 pairs, were compared. Since each pair of genotypes were checked for all 715 ways to get a 9-locus partial match, some 715 x 2.14 x 109 = 1.5×1012 nine-locus comparisons were made. Only 122 yielded matches. The empirically determined proportion is therefore about 8 x 10-11, or 1 in 12 trillion.
Let's compare this number with a theoretical estimate of the random-match probability -- one that assumes statistical independence of DNA alleles and loci. Brenner presents 1/13.66 as the probability of a random match at a single locus. Assuming independence, the probability of an exact match at 9 out of 9 such loci would be (1/13.66)9, or 4.5 x 10-11.* This agrees rather well with the empirical value of 8 x 10-11 in Arizona.
If this numerical exercise is any indication, the approach favored by the Las Vegas editor will not change things. The "realistic" probabilities that can be quoted in court on the basis of the number of 9-locus matches still will be astronomically small.
* The probability of a partial match, that is, of a match at 9 loci and a mismatch at the remaining 4 loci would be 715 x (1/13.66)9 x (1 - 1/13.66)4 = 1/(3.1×107). Of course, nobody would introduce this number in a real case because such a partial match excludes the suspect as the source of the crime-scene DNA. Partial matches like the ones in the Arizona database are not used to convict anyone. Rather, they are of interest because they raise a question as to whether there is an an excess of partial matches compared to the numbers that would be expected if the usual random-match probabilities are accurate. If there is a surprising excess -- something that is not yet clear -- then perhaps the standard calculation of random-match probabilities needs to be altered.
Charles Brenner, Arizona DNA Database Matches, Jan. 8, 2007, http://dna-view.com/ArizonaMatch.htm.
Editorial, DNA Evidence: What Are the Real Chances of Mistakes?, Las Vegas Review-Journal, Jul. 29, 2008,available at http://www.lvrj.com/opinion/26025944.html
D.H. Kaye, Letter, The Math Behind DNA Matching, Las Vegas Review-Journal, Aug. 01, 2008, available at http://www.lvrj.com/opinion/26171924.html