Friday, September 5, 2008
Genetics Datasets Closed Due to Forensic DNA Discovery
Until last Friday, the National Institutes of Health (NIH) and other groups had posted large amounts of aggregate human DNA data for easy access to researchers around the world. On Aug. 25, however, NIH removed the aggregate files of individual Genome Wide Association Studies (GWAS). The files, which include the Database of Genotypes and Phenotypes (dbGaP), run by the National Center for Biotechnology Information, and the Cancer Genetic Markers of Susceptibility database, run by the National Cancer Institute, remain available for use by researchers who apply for access and who agree to protect confidentiality using the same approach they do for individual-level study data.) The Wellcome Trust Case Control Consortium and the Broad Institute of MIT and Harvard also withdrew aggregate data.
The reason? The data keepers fear that police or other curious organizations or individuals might deduce whose DNA is reflected in the aggregated data, and hence, who participated in a research study. These data consist of SNPs -- Single Nucleotide Polymorphisms. These are differences in the base-pair sequences from different people at particular points in their genomes. Many SNPs are neutral -- they do not have have any impact on gene expression. Nonetheless, they can be helpful in determining the locations of nearby disease-related mutations.
The event that prompted the data keepers to act was the discovery at the Translational Genomics Research Institute (TGen) of a new way to check whether an individual's DNA is a part of a complex mixture of DNA (possibly from hundreds of people). According to the TGen report, Resolving Individuals Contributing Trace Amounts of DNA to Highly Complex Mixtures Using High-Density SNP Genotyping Microarrays, a statistic applied to intensity data from SNP microarrays (chips that detect tens of thousands of SNPs simultaneously) reveals whether the signals from an individual's many SNPs are consistent with the possibility that the individual is not in the mixture. (Sorry for the wordiness, but the article uses hypothesis testing, and "not in the mixture" is the null hypothesis.)
How could this compromise the research databases? As best as I understand it, the scenario is that someone first would acquire a sample from somewhere. Your neighbor might check your garbage, isolate some of your DNA, get a SNP-chip readout, and check it against the public database to see if you were a research subject who donated DNA. Or, the police might have a crime-scene sample. Then they would use a SNP-chip to get a profile to compare to the record on the public database to see if the profile probably is part of the mixture data there. Finally, if they got a match, the police would approach the researchers to get the matching individual's name.
Kathy Hudson, a public policy analyst at Johns Hopkins University, stated in an email that “While a fairly remote concern, and there are some protections even against subpoena, NIH did the right thing in acting to protect research participants.” However, scientists such David Balding in the U.K. are complaining that the restrictions on the databases are an overreaction. Indeed, an author of the TGen study is quoted as stating that the new policy is "a bit premature." See http://www.nature.com/news/2008/080904/full/news.2008.1083.html.
It seems doubtful that anonymity of the research databases has been breached, or will be in the immediate future, by this convoluted procedure. Of course, the longer-term implications remain to be seen, and the technique has obvious applications in forensic science. If the technique works as advertised, police will be able to take a given suspect and determine whether his DNA is part of a mixture from a large number of individuals that was recovered at a crime scene. Analyzing complex mixtures for identity is difficult to do with standard (STR-based) technology.
-DHK
References
Homer N, Szelinger S, Redman M, Duggan D, Tembe W, et al., Resolving Individuals Contributing Trace Amounts of DNA to Highly Complex Mixtures Using High-Density SNP Genotyping Microarrays, PLoS Genetics (2008). 4(8):e1000167. doi:10.1371/journal.pgen.1000167
DNA databases shut after identities compromised, Nature 455:13. Sept. 3, 2008
Natasha Gilbert, Researchers criticize genetic data restrictions, Nature Sept. 4, 2008, <http://www.nature.com/news/2008/080904/full/news.2008.1083.html>
September 5, 2008 | Permalink | Comments (1) | TrackBack (0)
Monday, August 11, 2008
Hot Tubbing: Old Wine in New Bottles for Expert Witnesses
The New York Times has discovered that expert witnesses retained by parties often are partisan. This certainly is fit to print, but is it news? Not to anyone who has been reading law reviews and opinions written during the past century or two. (For a good recent analysis, see Bernstein (2008).)
Still, the Times revealed that the Australians have discovered a way to improve expert testimony. They call it "hot tubbing.":
In that procedure, also called concurrent evidence, experts are still chosen by the parties, but they testify together at trial — discussing the case, asking each other questions, responding to inquiries from the judge and the lawyers, finding common ground and sharpening the open issues.
Interestingly, "Australian judges have embraced hot tubbing." According to UCLA law professor Jennifer Mnookin, "[t]he future ... may belong to Australia. 'Hot tubbing,' she said, 'is much more interesting than neutral experts.'”
If so, the movement will resemble the breakthrough of the Beatles "from Hamburg." (In 1961, when the band returned to Liverpool from Germany and made an appearance at The Cavern Club, some in the audience thought they were watching a German band.) Fifteen years ago, when I gave a traveling series of seminars to federal judges under the auspices of the Federal Judicial Center, I suggested that the judges experiment with this format. At least one judge was intrigued, saying that it sounded like the McLaughlin Show, but might be worth a try.
The idea certainly did not originate with me. I got it from a 1989 report of a blue-ribbon panel on statistics in the courtroom. The panel observed that
F.R.E. [Federal Rule of Evidence] 611 allows the trial judge to exercise power over the presentation of evidence to make it more effective and efficient. Many judges have used that authority in innovative ways to modify the traditional sequencing of evidence. For statistical matters, there are a variety of approaches tnhat might be attempted. When the reports of witnesses go together, the judge might allow their presentations to be combined and the witnesses to be questioned as a panel discussion ... .
Panel on Statistical Assessments as Evidence in the Courts (1989, 174). But "hot tubbing" is a lot catchier than "panel discussion," and the right packaging sells a product.
--DHK
References
David E. Bernstein, Expert Witnesses, Adversarial Bias, and the (Partial) Failure of the Daubert Revolution, 93 Iowa L. Rev. 451–489 (2008)
Adam Liptak, American Exception: In U.S., Partisan Expert Witnesses Frustrate Many, N.Y. Times, Aug. 11, 2008, available at http://www.nytimes.com/2008/08/12/us/12experts.html?_r=1&8au&emc=au&oref=slogin
Panel on Statistical Assessments as Evidence in the Courts, The Evolving Role of Statistical Assessments as Evidence in the Courts (Stephen E. Fienberg ed. 1989)
August 11, 2008 | Permalink | Comments (1) | TrackBack (0)
The Birthday Problem in Las Vegas
The other week, an editorial in the Las Vegas Review-Journal misconstrued the now infamous 2001 findings of partial matches in the Arizona DNA database. The study was discussed on our blog on July 20, and I won't repeat the explanation of the birthday problem. You might think that people in Las Vegas would know more about winning combinations, but the editor presented the Arizona findings as proof that hundreds of thousands of Americans would be falsely incriminated by DNA profiling in criminal cases. His conclusion: "the odds of a 'coincidental match' with an innocent party -- the realistic odds, based on searches such as [the one in the Arizona database], not something out of astronomy book -- should be carefully explained."
The last remark got me thinking. Could the results of Arizona database study give an estimate of a random-match probability?
Mathematician Charles Brenner did some simplified calculations that can be adapted to this end. A standard DNA profile consists of 13 pairs of numbers. The numbers have to do with the lengths of various fragments of DNA at particular points (loci) on certain chromosomes. If a suspect and a crime-scene sample have the same fragment lengths at all 13 loci, then the match is strong evidence that the suspect (or an identical twin) is the source of the DNA. A partial match excludes the suspect as the source, and there are many ways for two 13-locus profiles to match in part but not in full. For example, Brenner pointed out that there are 715 ways to select 9 loci from the full 13. In the Arizona study, the analyst looked at all distinct pairings of the 65,493 people in the 2001 Arizona database. (This is where the birthday problem, with its combinatorial explosion, comes in.) Brenner reported that 65,493 x 65,492/2, or approximately 2,140,000,000 pairs, were compared. Since each pair of genotypes were checked for all 715 ways to get a 9-locus partial match, some 715 x 2.14 x 109 = 1.5×1012 nine-locus comparisons were made. Only 122 yielded matches. The empirically determined proportion is therefore about 8 x 10-11, or 1 in 12 trillion.
Let's compare this number with a theoretical estimate of the random-match probability -- one that assumes statistical independence of DNA alleles and loci. Brenner presents 1/13.66 as the probability of a random match at a single locus. Assuming independence, the probability of an exact match at 9 out of 9 such loci would be (1/13.66)9, or 4.5 x 10-11.* This agrees rather well with the empirical value of 8 x 10-11 in Arizona.
If this numerical exercise is any indication, the approach favored by the Las Vegas editor will not change things. The "realistic" probabilities that can be quoted in court on the basis of the number of 9-locus matches still will be astronomically small.
--DHK
Note
* The probability of a partial match, that is, of a match at 9 loci and a mismatch at the remaining 4 loci would be 715 x (1/13.66)9 x (1 - 1/13.66)4 = 1/(3.1×107). Of course, nobody would introduce this number in a real case because such a partial match excludes the suspect as the source of the crime-scene DNA. Partial matches like the ones in the Arizona database are not used to convict anyone. Rather, they are of interest because they raise a question as to whether there is an an excess of partial matches compared to the numbers that would be expected if the usual random-match probabilities are accurate. If there is a surprising excess -- something that is not yet clear -- then perhaps the standard calculation of random-match probabilities needs to be altered.
References:
Charles Brenner, Arizona DNA Database Matches, Jan. 8, 2007, http://dna-view.com/ArizonaMatch.htm.
Editorial, DNA Evidence: What Are the Real Chances of Mistakes?, Las Vegas Review-Journal, Jul. 29, 2008,available at http://www.lvrj.com/opinion/26025944.html
D.H. Kaye, Letter, The Math Behind DNA Matching, Las Vegas Review-Journal, Aug. 01, 2008, available at http://www.lvrj.com/opinion/26171924.html
August 11, 2008 | Permalink | Comments (0) | TrackBack (0)
Friday, August 8, 2008
Fingerprints' Chemical "Footprints"?
Today's New York Times reports a story that appears in this week's Science. According to the Times, "With a new analytical technique, a fingerprint can now reveal much more than the identity of a person. It can now also identify what the person has been touching: drugs, explosives or poisons, for example." See full story HERE. In short, scientists (Demian R. Ifa, et al.) have used mass spectrometry to identify fingerprints after subjects' fingers were applied with various solutions, including drugs and explosives residue. The researchers suggest that this technology might have several uses, including identifying what substances particular people might have handled recently and being able to distinguish overlapping fingerprints, by tracing the chemical "footprint" of the individual fingerprints.
Although there may yet be much value in this research, this single report hardly demonstrates its value for forensic purposes. The researchers essentially identified the true-positive rate for this technology, and, so far as either the Times or the original article in Science report, the researchers have provided no data on false positives, true negatives, or false negatives. Moreover, this study was a highly controlled laboratory study, so we don't know whether the technology might confront excessive "noise" when applied to the general population. Indeed, given the reported amount of drug residue on United States currency, mass spectrometry that is too sensitive is likely to produce large numbers of false positives.
Hence, while the research results reported here are interesting and noteworthy, without considerable more work in this area, they appear a long way from daily forensic use.
--DLF
August 8, 2008 | Permalink | Comments (1) | TrackBack (0)
Sunday, July 20, 2008
DNA Database Woes and the Birthday Problem
The Los Angeles Times has reported that "A discovery leads to questions about whether the odds of people sharing genetic profiles are sometimes higher than portrayed. Calling the finding meaningless, the FBI has sought to block such inquiry." Actually, the discovery is not new, but the story is still unfolding.
According to the article,
State crime lab analyst Kathryn Troyer was running tests on Arizona's DNA database when she stumbled across two felons with remarkably similar genetic profiles.
The men matched at nine of the 13 locations on chromosomes, or loci, commonly used to distinguish people.
The FBI estimated the odds of unrelated people sharing those genetic markers to be as remote as 1 in 113 billion. But the mug shots of the two felons suggested that they were not related: One was black, the other white.
In the years after her 2001 discovery, Troyer found dozens of similar matches -- each seeming to defy impossible odds.
The key word here is "seeming." This is not the first time partial or even complete matches have appeared in a search of all pairs of DNA profiles in a law-enforcement database. Eight years ago, the National Commission on the Future of DNA Evidence (2000, 25 n.13) reported that
Although brothers and twins are rare in databases, they can be common among those pairs that are found by profile matching. John Buckleton (2000 personal communication) found that, among ten 6-locus matches in a New Zealand database of 10,907 records, all but 2 were brothers (including twins). This shows that the possibility of sibs cannot be ignored in database searches. We should note, however, that these could usually be identified as brothers, either by further investigation or by testing additional loci.
So close relatives are one possible explanation for a seeming surplus of partial matches.
A second consideration is statistical. The random-match probability of 1 in 113 billion quoted in the Times applies to a single comparison between a particular profile and a randomly selected, unrelated individual. It is not the probability that a search through all pairs of profiles in a database composed entirely of records from unrelated people will show a match. Because there are so many pairs to compare, that probability is much greater.
Suppose that there are 500,000 profiles in the database. How many possible pairs can be formed? The answer: 500,000 x 500,000 = 2.5 x 10^11 = 250 billion. How many of these are from different individuals? Answer: Subtract the 500,000 pairs [(1,1), (2,2), ... , (500,000, 500,000)]. That hardly changes anything, since 500,000 is nothing compared to 250,000,000,000. How many are from distinct pairs of people? Answer: Half, since the pair (1,2) is the same as (2,1), etc. Conclusion: There are almost 125 billion pairs to search.
How many comparisons would be expected to match if, for every comparison, the chance of a match is 1 in 113 billion? Answer: About 1. Even without relatives, the observation of a partial match in such a database would not be so surprising.
Of course these numbers do not pertain to the Arizona database. I do not know how large it was, and the chance of a match in each comparison was not constant. But the example shows why the random-match probability grossly understates the chance of a partial-match in an all-pairs trawl in a large database.
In probability theory, this situation is known as a birthday problem. The chance that one randomly selected person has the same birthday as mine is about 1/365. The chance that at least two people in a room full of people have the same birthday (whatever it might be) is much, much larger.
We can expect further studies of the databases for consistency with the estimated random-match probabilities. The article reports on several that have taken place so far. My prediction is that when the dust settles, the results will be inconclusive. Judges will struggle a bit with the birthday problem, and it will be difficult or impossible to determine all the close relatives in the database. Scientists who accept the existing random-match probabilities as reasonable estimates won't change their minds. Well, maybe they'll give up a power of ten or so. Individuals who distrust the estimates will continue to distrust them.
--DHK
References
Felch, Jason, and Maura Dolan. 2008. "How Reliable Is DNA in Identifying Suspects?" Los Angeles Times: July 20, 2008. <http://www.latimes.com/news/local/la-me-dna20-2008jul20,0,5133446.story>
National Commission on the Future of DNA Evidence 2000. The Future of Forensic DNA Testing: Predictions of the Research and Development Working Group. Washington DC: National Institute of Justice
http://en.wikipedia.org/wiki/Birthday_paradox
July 20, 2008 | Permalink | Comments (13) | TrackBack (0)
Wednesday, June 25, 2008
The Psychology of Fuel Efficiency
A recent discussion started by John Lynch on the Society for Judgment and Decisionmaking listserv focuses on an interesting new article by Larrick and Soll in Science, entitled the "MPG Illusion." The paper reemphasizes the point that statistical metrics matter. It argues that the traditional miles per gallon metric leads people to make inaccurate judgments on the benefits of more efficient cars.
For example, Richard Larrick in his podcast makes an argument along the following lines. Say you have the ability to trade in a 10 MPG SUV for a 20 MPG crossover, or a 25 MPG car for a 50 MPG hybrid. Which switch is better for the environment? As it turns out, the former, even though one might be tempted to say that the former only improves efficiency by 10 MPG while the latter improves it by 25. Assume a 100 mile trip. The SUV will consume 10 gallons versus 5 gallons for the crossover for a net savings of 5 gallons. The car will consume 4 gallongs versus 2 gallons for the hybrid for a net savings of 2 gallons.
It seems that since we drive given distances (e.g. 100 mi), rather than specific amounts of fuel, the MPG is a misleading measure of efficiency. Small increases in efficiency down at the low end make much more of a difference than at the high end. Larrick & Soll argue that an inverse ratio, gallons per 10,000 miles, might be a more useful measure.
More information is available on Larrick's website, which has links to the Science article, podcast, and supplemental materials.
--EKC
June 25, 2008 | Permalink | Comments (1) | TrackBack (0)
Tuesday, June 24, 2008
The persuasive power of neuroscience
The March issue of the Journal of Cognitive Neuroscience contains an article stimulated by the frequent appearance of news stories announcing the latest brain signature -- for love, aggression, greed, lying, etc. A group of researchers at Yale decided to investigate whether people can distinguish solid claims about these associations from poorly substantiated ones. The researchers wrote explanations for well-documented psychological phenomena. Some versions presented scientifically accepted rationales and sound reasoning. Other explanations were circular. People with no training in psychology or neuroscience distinguished the good from bad -- until an utterly irrelevant mention of the physical brain was added. The bad explanations became far more believable when they included a mention of neuroscience, while the good accounts got only a slight boost. People with advanced training in cognitive science were immune to this "seductive allure of neuroscience."
Does this finding have some bearing on the law's demand for validation of scientific evidence? Does it support a distinction between "soft" psychological testimony and testimony about brain imaging results?
--DHK
References
Weisberg, D. S.; Keil, F. C.; Goodstein, J.; Rawson, E.; & Gray, J. (2008). The Seductive Allure of Neuroscience Explanations. Journal of Cognitive Neuroscience, 20(3), 470-477.
The description of the study is adapted from the May/June 2008 issue of the Yale Alumni Magazine, p. 38.
June 24, 2008 | Permalink | Comments (0) | TrackBack (0)
Sunday, June 22, 2008
Rounding Up the Usual Suspects III: People v. Nelson
On April 5, 2008, I mentioned People v. Nelson, 48 Cal.Rptr.3d 399 (Ct. App. 3 Dist. 2006), rev. granted, 147 P.3d 1011 (Cal. 2006), as a leading case on the admissibility of the various probabilities associated with cold hits in DNA databases. Last week, the California Supreme Court affirmed.
The case arose from the rape and murder of a nineteen-year-old college student in 1976. Dennis Nelson was a suspect, but the evidence was inconclusive, and the case grew cold. Later, Nelson was convicted of a different rape. His DNA profile was entered into the state convicted-offender databank. In 2001, investigators discovered that this profile matched those derived from stains from the 1976 rape. At that point, there were 184,000 profiles in the database. According to the state, the match would occur “at random among unrelated individuals in about one in 950 sextillion African-Americans, one in 130 septillion Caucasians, and one in 930 sextillion Hispanics.” As the court adds, “[t]here are 21 zeros in a sextillion and 24 zeros in a septillion.”
Nelson moved to dismiss the resulting charges on the ground that the delay between the 1976 crime and the charges filed in 2002 deprived him of his right to a speedy trial. The superior court denied the motion. At trial, Nelson conceded that he had intercourse with the victim but claimed that it was consensual -- somebody else must have murdered her and left her body in the mud. That did not work either. The jury convicted Nelson of first degree murder, and the Court of Appeal affirmed.
The California Supreme Court reviewed two claims. First, with respect to the speedy-trial issue, it held that the 26-year delay between the offense and the prosecution caused only slight prejudice and was justified.
Second, the court considered whether the vanishingly small random-match probabilities should have been admitted. The court correctly held that inasmuch as the procedure underlying this calculation was generally accepted and uncontested, the only real issue was the relevance of a random-match probability in a database-trawl case.
At this point, however, the opinion unravels. It contains but a single, short paragraph to show why the statistic is relevant:
In a non-cold-hit case, we said that “[i]t is relevant for the jury to know that most persons of at least major portions of the general population could not have left the evidence samples.” (People v. Wilson, supra, 38 Cal.4th at p. 1245.) We agree with other courts that have considered the question (the Court of Appeal in this case; People v. Johnson, supra, 139 Cal.App.4th 1135; and Jenkins, supra, 887 A.2d 1013) that this remains true even when the suspect is first located through a database search. The database match probability ascertains the probability of a match from a given database. “But the database is not on trial. Only the defendant is.” (Modern Scientific Evidence, supra, § 32:11, pp. 118-119.) Thus, the question of how probable it is that the defendant, not the database, is the source of the crime scene DNA remains relevant. (Id. at p. 119.) The rarity statistic addresses this question.
As the co-author of the text of the treatise being quoted, I fear that these words are inconsistent with the portion of the court's opinion (in note 3) suggesting that the database-match probability also is relevant. If the issue is simply “how probable it is that the defendant, not the database, is the source of the crime scene DNA,” then the database-match probability is irrelevant. Unlike the “rarity statistic,” it does not figure into the probability that the named defendant is the source. The formulas are given and explained in a forthcoming article, Rounding Up the Usual Suspects: A Logical and Legal Analysis of DNA Trawling Cases. The Nelson court's theory that a variety of statistics are admissible in a database-trawl case does not withstand analysis. Or, if it does, it will take more analysis than this court has provided to explain why. The opportunity for such clarification may well arise, as there will some cases in which defense counsel will be interested in introducing the database-match probability, which can be orders of magnitude larger than the random-match probability.
In this particular case, however, the demand for an adjustment to the random-match probability is much ado about nothing. So what if the probability is 10–19 rather than 10–24? Having rejected the defense argument about general-acceptance, the court could simply have observed that the choice of a statistic could not have affected the outcome of the case. The court realized this, but it endorsed the 10–24 figure anyway.
-- DHK
References
People v. Nelson, No. S147051 (Cal. June 16, 2008), slip opinion available at http://www.courtinfo.ca.gov/opinions/documents/S147051.pdf
Dolan, Maura and Jason Felch. 2008. "California Supreme Court Ruling Allows 'Rarity' Statistic in DNA Cases." Los Angeles Times: June 17 available at http://www.latimes.com/news/science/la-me-dna17-2008jun17,0,3313471.story
Kaye, David H. 2009. "Rounding Up the Usual Suspects: A Logical and Legal Analysis of DNA Trawling Cases". North Carolina Law Review: in press. Prepublication draft available at SSRN: http://ssrn.com/abstract=1134205
June 22, 2008 | Permalink | Comments (0) | TrackBack (0)
Sunday, June 8, 2008
The Transposition Fallacy in the Los Angeles Times
In an earlier posting, I noted a story in the Los Angeles Times about the perceived need to adjust the probability for a random match when an individual emerges as a suspect because of a trawl through a database of DNA profiles. The reporters suggested that there was a grave injustice because "the prosecutor told the jury that the chance of such a coincidence was 1 in 1.1 million," but "jurors were not told the statistic that leading scientists consider the most significant: the probability that the database search had hit upon an innocent person. In Puckett's case, it was 1 in 3." They added that "the case is emblematic of a national problem."
The Times received some flak for this reporting. Not only do many leading statisticians dispute the claim that an adjustment for the size of the database searched produces the most significant statistic, but, it was said, the description of "1 in 3" as "the probability that the database had hit upon an innocent person" was wrong. The critical readers complained that, at best, 1/3 was the chance of a match to someone in the database if neither Puckett nor anyone else in the database were the source of the DNA in the bedroom of the murdered woman. It is not the chance that Puckett is not the source given that his DNA matches.
To equate the two probabilities is to slip into the transposition fallacy that P(A given B) = P(B given A). Conditional probabilities do not work this way. For instance, the chance that a card randomly drawn from a deck of ordinary playing cards is a picture card given that it is red is not the chance that it is red given that it is a picture card. The former probability is P(picture if red) = 6/26. The latter is P(red if picture) = 6/12.
The reporters responded with the following defense:
In our story, we did not write that there was a 1 in 3 chance that Puckett was innocent, which would be a clear example of the prosecutor's fallacy. Rather, we wrote: "Jurors were not told, however, the statistic that leading scientists consider the most significant: the probability that the database search had hit upon an innocent person. In Puckett's case, it was 1 in 3." The difference is subtle, but real.
Interestingly, when asked whether there was any difference on a listserve of evidence professors, two professors described the statement as ambiguous, while four saw it as a clear instance of transposition.
My view is that the following two statements are true:
1. IF THE DATABASE WERE INNOCENT (meaning that it does not contain the source of the crime-scene DNA and everyone in it is unrelated), then (prior to the trawl) the probability that SOMEONE (regardless of his or her name) would match is roughly 1/3.
2. IF THE DATABASE WERE INNOCENT, then (prior to the trawl) the probability that a man named Puckett would match is 1/N = 1/1,100,000.
But neither (1) nor (2) is equivalent to
3. The probability that the database search hit upon an innocent person named Puckett was 1/3.
--DHK
June 8, 2008 | Permalink | Comments (0) | TrackBack (0)
Thursday, June 5, 2008
fMRI, Lie Detection, and Statistics
I'm blogging from the AALS Mid-Year Conference on Evidence in Cleveland, where I just moderated a discussion this morning on fMRI and Lie Detection featuring Steve Laken (Cephos Corp.) and Mike Pardo (Alabama). Although the studies on fMRI lie detection have their limitations, the results so far are quite impressive, with accuracy rates in the 90% range. One wonders how soon they will make their way into court, where admissibility questions loom large. Even if the technology is in fact sufficiently reliable for Daubert (and what I saw this morning suggests that this is true), the inherent conservatism of the legal system, coupled with the bias against analogs to polygraphs, will make admissibility a tough hurdle for the technology. (For more on the bias against mind-reading devices, see this Note written by my student Leo Kittay.)
One striking aspect of the various discussions on fMRI during and after the session was the focus that people had on mechanism. Many people are concerned that researchers have not yet pinpointed specific areas of the brain associated with lying, or have not determined specific pathways for deception. Often, they are similarly concerned that other brain activities may "light up" the same regions. I'm skeptical, however, that these concerns really matter. While it may be desirable and interesting to know the specific mechanisms associated with deception, we really don't need to make such discoveries to have a practically useful lie detection machine. All that matters is that some model exists (here, presumably using brain scans) that can with reasonable accuracy separate liars from non-liars. How the model does that is in many ways beside the point. As Laken pointed out during the discussion, medical researchers often have little or idea about the specific mechanism for a drug's success, yet such a limitation never prevents us from using its therapeutic benefits as proven through statistical/epidemiological studies.
--EKC
June 5, 2008 | Permalink | Comments (2) | TrackBack (0)






