Science & Law Blog

My brother's DNA: Near-miss DNA searching

California has adopted an aggressive policy toward near-miss DNA searching -- something discussed in this blog before. The state is going to compare DNA profiles recovered from crime-scenes to those in its offender database (1) to see if there are any "cold hits" to convicted offenders and arrestees, and (2) to see if there are any almost-matching profiles that are likely to have come from a very close relative.

The first procedure has been upheld in case after case challenging its constitutionality (in the context of convicted offenders). Why would the second procedure be constitutionally defective? According to a Los Angeles Times article of April 26 on the California policy, some lawyers think it is an unreasonable search that might run afoul of the Fourth Amendment. The paper also quotes "Tania Simoncelli, science advisor to the American Civil Liberties Union," as asserting that "The fact that my brother committed a crime doesn't mean I should have to give up my privacy!"

This crie de coeur surely is sincere, and it may not be meant as a constitutional argument, but it is interesting to ask whether it supplies a plausible principle for applying the Fourth Amendment. Consider the following case: You have an identical twin brother. He robs a bank, is locked away in prison, and his DNA profile is put in an offender database. This can happen even though his DNA was not evidence in the bank robbery case and had nothing to do with that crime.

While your brother is out of circulation, you break into a house. cutting your hand on the glass of a window that you shattered to gain entry. A tiny bloodstain with your DNA on it is analyzed. The profile is compared to those in the database. It matches the one that is file perfectly -- your brother's -- because identical twins have the same DNA sequences. But the police know that your brother was in prison when the house was burgled. They scratch their heads until they realize that he might have an identical twin with identical DNA.

So the police investigate you and find plenty of other evidence against you. Now you are facing trial. You move to exclude evidence that your DNA matches that in the bloodstain on the ground that this discovery is the result of an unreasonable search, arguing that "the fact that my brother committed a crime doesn't mean I should have to give up my privacy!" Not only that, you contend that the rest of the evidence must be dismissed because all of it is the fruit of this illegal search.

I do not see how anyone (who agrees that convicted-offender databases that include bank robbers are constitutional) can argue that this search infringes the Fourth Amendment. It is too bad that you and your brother share the same DNA profile, but the police have not forced you to surrender your DNA, and you have no right to stop them from checking your brother's DNA to see if he might be responsible. By checking him, they learn something about you. You might not like it, but let's face it, this probably is not the first time that your brother got you into trouble.

Counter-arguments, anyone?

Of course, the California policy is not limited to identical twins. Furthermore, it involves partial matches and less complete information. All that I have tried to show is that the slogan that "the fact that my brother committed a crime doesn't mean I should have to give up my privacy!" does not settle any constitutional question. It states the conclusion of what must be a rather complex argument about (1) the privacy of information that identifies a class of individuals and (2) the power of the state to investigate one individual on the basis of information it legitimately obtains from another individual.

--DHK

Rounding Up the Usual Suspects II

Not long ago, I mentioned the DNA-database-trawl issue that has led to several confused court opinions. The evidentiary issue is whether a complete search through a database of DNA profiles that produces one and only one match is less probative than a simple match to a known suspect. Some researchers in the U.K. tendentiously call the former use of the database “speculative searching.” (Kaye 2006, 18).

Now, an article in the May 3 Los Angeles Times claims to have uncovered a national scandal of sorts. The reporters describe a recent “cold hit” case that they say

is emblematic of a national problem, The Times has found. [¶] Prosecutors and crime labs across the country routinely use numbers that exaggerate the significance of DNA matches in "cold hit" cases, in which a suspect is identified through a database search. [¶] Jurors are often told that the odds of a coincidental match are hundreds of thousands of times more remote than they actually are, according to a review of scientific literature and interviews with leading authorities in the field.

The article maintains that

[I]n cold hit cases, the investigation starts with a DNA match found by searching thousands, or even millions, of genetic profiles in an offender database. Each individual comparison increases the chance of a match to an innocent person. [¶] Nevertheless, police labs and prosecutors almost always calculate the odds as if the suspect had been selected randomly from the general population in a single try. [¶] The problem will only grow as the nation's criminal DNA databases expand. They already contain 6 million profiles.

This description portrays one approach to the issue as if it is the consensus in the scientific literature. It is not. There is disagreement about the need to adjust a random-match probability. Furthermore, if one counts the number of peer-reviewed articles on the subject, the dominant view is that adjustment is not necessary.

I won't present a full blown analysis here, but I will offer a thought on the statement that “[e]ach individual comparison increases the chance of a match to an innocent person.” It is true that if one searches a database of a million innocent people, all of whom are unrelated to the source of the crime-scene DNA, there are more opportunities for a match to an innocent person than if one searches a database of half a million innocent people, or than if one searches a database of one only one innocent person (i.e., the suspect). So sooner or later, searches of innocent databases will produce a false positive. Indeed, they already have.

But is the probability that an innocent database will contain a matching type the right question to ask? The probative value of a match depends on how much it shifts the odds in favor of the prosecution's claim that the matcher is the source of the crime-scene DNA. The enhancement in the odds grows progressively larger as the size of the database increases. The reason is simple. More and more people are definitively excluded as possible sources of the crime-scene DNA. This raises the probability that someone else in the population — including the matcher — is the source. In the limiting case of a database that includes every person on earth, the evidence of a single match in the database becomes conclusive (ignoring scenarios involving fraud or laboratory error).

It can be shown (and has been) that, due to this “exclusion effect,” the single match in the database raises the odds even more (at least slightly) than does testing a single person at random and finding that he matches. (E.g., Donnelly and Friedman 1999; Kaye 2008). Therefore, if there is any prejudice in the existing practice of reporting the random-match probability in the “cold hit” case, it is not because a cold hit in a large database is less probative than a cold hit in a small one!

In sum, searching large databases gives more information than searching small ones, and searching small ones is better than limiting a search to a single individual. The DNA evidence has more, not less, probative value in a database-search case than in a single-suspect case.

References:

Donnelly, Peter, and Richard D. Friedman. 1999. “DNA Database Searches and the Legal Consumption of Scientific Evidence.” Michigan Law Review 97: 931–984.

Kaye, D.H. 2008. “Rounding Up the Usual Suspects: A Legal and Logical Analysis of DNA Trawling Cases.” (submitted for publication).

Kaye, Jane. 2006. "Police Collection and Access to DNA Samples." Genomics, Society and Policy. 2: 16–27.

--DHK

Ghostwriting in Medical Journals

The Journal of the American Medical Assocation (JAMA) recently published a study discussing the disturbing practice of ghostwriting in medical journal articles -- in this case, involving Vioxx.  The JAMA article is here.   A related New York Times article can be found here.

--EKC

Low copy number DNA vindicated?

In January, we noted the unusual opinion of the trial court in the Omagh bombing case. Now, a second review of the Forensic Science Service's procedure for amplifying small quantities of DNA has appeared. According to an article in today's Guardian, a report undertaken for the Crown Prosecution Services concludes that "[t]iny samples of DNA evidence are safe to use in criminal prosecutions." However, the report also contains "21 recommendations to standardise procedures, including ensuring that police evidence-gathering kits are "DNA-clean", to avoid contamination with someone else's genetic profile, a national agreement on how to interpret the results from low-template DNA, and clear guidance on how courts should interpret the evidence."

--DHK

Rounding Up the Usual Suspects

Countries around the world have established databases consisting of the DNA profiles of suspected or convicted offenders. In the United States, state and federal databases combined in the FBI's National DNA Index System hold over five million convicted offender DNA profiles as well as those of some people who are merely arrested or detained. These identification databases have helped solve cases that have baffled investigators for decades. In one case, a federal database search linked a 58-year-old man suspected of raping at least 25 women in three states to semen on underwear from a 1973 rape.

When the DNA profile from a crime-scene stain matches one of those on file, the person identified by this “cold hit” will become a target of the investigation. A fresh sample will be taken from the suspect to verify the DNA match, and other evidence normally will reinforce the investigatory lead. In rare cases, prosecutors will even proceed with no other evidence. In one such case, a San Francisco jury convicted John Davis, already behind bars for robbery and other crimes, of the murder of his neighbor, Barbara Martz, nearly 22 years earlier. The database match was all the jurors had to go on. This was enough for a conviction, at least where the probability that a randomly selected, unrelated individual would match the crime-scene DNA sample — the “random-match probability” — was said to be “quadrillions-to-one.”

Cases like Davis that emanate from cold hits have been called “trawl cases” because “the DNA match itself made the defendant a suspect, and the match was discovered only by searching through a database of previously obtained DNA samples.” Peter Donnelly & Richard D. Friedman, DNA Database Searches and the Legal Consumption of Scientific Evidence, 97 Mich. L. Rev. 931, 932 (1999). These database-trawl cases can be contrasted with traditional “confirmation cases” in which “other evidence has made the defendant a suspect and so warranted testing his DNA.” Id.

In terms of this dichotomy, we must ask whether the fact that the defendant was selected for prosecution by trawling requires some adjustment to the random-match probability. Two committees of the National Academy of Sciences (NAS) thought so. In their influential reports on “DNA Forensic Science,” they reasoned that a match coming from a trawl is much less impressive than a match in a confirmation case — just as finding a tasty apple on the very first bite is more impressive than pawing through the whole barrel of apples to locate a succulent one. To account for the extra bites at the apples, they described approaches that would inflate the normal random-match probability.

The response has been disputation and litigation. Two early commentators, Bill Thompson and Simon Ford, gave “a Bayesian analysis” to suggest that “this evidence has no probative value.” William C. Thompson & Simon Ford, DNA Typing: Acceptance and Weight of the New Genetic Identification Tests, 75 Va. L. Rev. 45, 100 (1989). Ten years later, Donnelly and Friedman reached precisely the opposite conclusion. They applied Bayes' rule in more detail -- and correctly -- to show that the trawl actually increases the probative value of the match.

Recently, three appellate opinions on the issue have emerged -- United States v. Jenkins, 887 A.2d 1013 (D.C. 2005), People v. Johnson, 43 Cal.Rptr.3d 587 (Ct. App. 2006), and People v. Nelson, 48 Cal.Rptr.3d 399 (Ct. App. 3 Dist. 2006), rev. granted, 147 P.3d 1011 (Cal. 2006). In these cases, defendants argued that until the scientific community can agree on a single statistic to characterize the import of a database trawl, even the fact of a match should not be admitted. Even though the dispute in the scientific community is limited to the question of whether there is any reason to bother with the NAS adjustments to the probability figure, the trial court in Jenkins felt compelled to exclude the DNA evidence in its entirety.

The appellate courts all rejected the defense challenges, but their opinions fail to address the dispute in the scientific and legal literature. The avoidance mechanisms they employ are singularly unimpressive. In Nelson, for example, the court of appeal claimed that California's general acceptance standard for scientific evidence does not apply because after the database trawl identifies the suspect, a fresh sample from the suspect is typed. If the fresh sample matches, only this match is introduced at trial. In the court's view, it is as if the database trawl never took place.

To a statistician, this is a jay-dropping claim. The challenge is not to the use of a convicted-offender DNA database as an investigatory tool. The objection is to the use of the random-match probability at trial to gauge the power of the later match when the defendant has not been selected for DNA testing “at random” — that is to say, on the basis of factors that are uncorrelated with his DNA profile. When the defendant is selected for a later test precisely because of his known DNA profile, the replication adds no new information about the hypothesis that the defendant is unrelated to the actual perpetrator and just happens to have the matching DNA profile. It adds no information because the datum — a matching profile in the new sample — is just as probable when this hypothesis is true as when it is false. Replication helps eliminate the risk of a laboratory error in determining or reporting the DNA profile, but it has no further value in probing the possibility of a coincidental match.

Because the rationales presented in the three cases to date are unconvincing, the emerging case law needs to be reoriented to confront directly the competing statistical arguments about the meaning of a database match. Recent statistical literature seems to favor the view that no adjustment to the random-match probability is necessary, but this may just reflect the fact that most statisticians writing about forensic science are Bayesians rather than frequentists. Although Donnelly and Friedman have presented the Bayesian perspective forcefully and simply, it appears that it will take more to convince the courts that they need to think more deeply -- and more clearly -- about the subject.

--DHK. These comments are adapted from a forthcoming book, The Double Helix and the Law of Evidence: Controversies over the Admissibility of Genetic Evidence of Identity (Harvard Univ. Press).

Low Copy Number DNA Dealt a Low Blow?

Twenty-nine people died and 200 were wounded when a 255-kilogram car bomb exploded in a busy shopping area. No, this was not Baghdad or Jerusalem. It was Omagh, Northern Ireland, in 1998. The bomb was the work of splinter group calling itself the Real IRA. It was Northern Ireland's worst single terrorist atrocity.

Ten years later, Sean Hoey, a 38-year-old electrician was on trial for the murders. It was Britain's biggest murder trial.  Justice Reg Weir, sitting under Northern Ireland's Diplock non-jury system for terrorism trials, announced his verdict last month -- not guilty! Hoey waved to applauding family members as relatives of the victims gasped in shock. Families of the dead said they were stunned that Mr Hoey had been acquitted but pledged to press ahead with a civil action for 14 million pounds compensation against five men who they claim were responsible for the attack.

Shades of O.J. Simpson? A lynchpin in the case was DNA evidence. The prosecution maintained that bombs used in various attacks, including this one, had distinctive similarities in their construction and that Hoey had been involved in constructing them. His DNA, it claimed, had been found in connection with four of them (not including the Omagh bomb). However, the judge had harsh words for the treatment of this vital DNA evidence, saying that the recording, packaging, storage and transmission of some of the items was "thoughtless" and "slapdash." He found that the police and forensic laboratory did not take "appropriate DNA protective precautions" and that the police had engaged in "mendacious attempts to retrospectively ... alter the evidence" to hide this fact.

For good measure, the court added a discussion of the validity of Low Copy Number (LCN) DNA typing. Introduced by Britain's Forensic Science Service (FSS) in 1999, LCN is a term for one of several related methods for increasing the sensitivity of ordinary DNA testing. All the procedures start by "amplifying" a sample of DNA, that is, by producing a huge number of copies of the original molecules for analysis.  LCN pushes the amplication step (known as PCR, because it is based on the Polynerase Chain Reaction for copying stands of DNA) to its limits. It permits the duplication of just a few original molecules.

The defense experts criticized LCN for want of adequate validation. They reached this conclusion because: (1) LCN results were admitted as evidence in only three countries; (2) the US (which only used it for investigative purposes) employed "a different and much more stringent operating system"; (3) it lacked "an international agreement on validation"; and (4) only two scientific papers on the technique were published, and those were written by its inventors. The judge endorsed these criticisms.

This part of the opinion seems odd for a country that does not subject scientific evidence to special scrutiny -- that has no precedent comparable to the Frye or Daubert cases in the United States. The opinion justifies its discussion of these matters by quoting from a House of Commons'committee report calling for a " 'gate-keeping' test for expert evidence [building] on the US Daubert test."

The Crown Prosecution Service responded by carrying out a "precautionary internal review of current cases involving the FSS use of LCN DNA analysis." On 14 January 2008, it reported that it could not find

anything to suggest that any current problems exist with LCN. Accordingly we conclude that LCN DNA analysis provided by the FSS should remain available as potentially admissible evidence. Of course, the strength and weight such evidence is given in any individual case remains a matter to be considered, presented, and tested in the light of all the other evidence

DHK

Sources:

Queen v. Hoey, [2007] NICC 49, WE17021

Crown Prosecution Service, Press Release, Review of the use of Low Copy Number DNA Analysis in Current Cases: CPS Statement, Jan. 14, 2008, http://www.cps.gov.uk/news/pressreleases/101_08.html

Duncan Campbell and Vikram Dodd, Police Suspend Use of Discredited DNA Test after Omagh Acquittal, The Guardian, December 22, 2007, http://www.guardian.co.uk/Northern_Ireland/Story/0,,2231403,00.html

Anne Cadwallader, Omagh Bombing Suspect Acquitted, Reuters UK, Dec 20, 2007, http://today.reuters.co.uk/news/articlenews.aspx?type=topNews&storyid=2007-12-20T201408Z_01_L2062777_RTRUKOC_0_UK-IRISH-OMAGH.xml

"Predictive Medical Information" in DNA Databases for Law Enforcement

One objection to amassing databases of DNA profiles for law enforcement purposes is that the profiles themselves contain "predictive medical information." E.g., Joh (2006). The average person might take this to mean that the profiles used for identification also reveal whether a person is at risk for particular diseases. Yet, no one knows how to make such predictions, and no physician would be interested in the allegedly "predictive medical information." Nevertheless, Professor Simon Cole (2007) of the University of California at Irvine suggests that such statements do not contradict the claim of forensic scientists that no one can use the set of numbers in the standard CODIS profiles to predict whether anyone represented in the database will develop any disease. In Cole's view, it is just a way of saying that someday, somehow, meaningful predictive value might be discovered.

So construed, the assertion of "predictive information" is irrefutable.  To give the claim some real meaning, however, one must show that the possibility that the identifying features will turn out to be predictive of disease is more than idle speculation. In this regard, Cole makes the following argument:

Presumably, Professor Kaye would respond that his extrapolation of the future is more defensible than others because it is “based on current knowledge and practice.” It may be more defensible, but that does not mean it is any more likely to be correct. Would the current capability of genetics have been predictable from the state of knowledge and practice in 1960? If not, there is no reason to assume that the capabilities of genetics in 2050—when the law enforcement DNA databases we are building today will likely still be in place and encompass a large portion of the population—must be wholly predictable from the current state of theory and knowledge.

Cole is referring a paper (Kaye 2007a) that explains why, in light of basic principles of statistics and genetics that date back to Bayes and Galton, the alleged predictive power of the profiles is likely to remain too slight to permit useful inferences about disease status or propensity. To reach this conclusion, the paper discusses of all the known ways in which the profiles in a database might be used to predict future diseases.

This is, of course, quite different from blithely assuming that the future will resemble the past. And, I think it is better than assuming that just because we know more about molecular biology and medical genetics than we did in 1960, we will be able to accomplish this particular feat by 2050. (Kaye 2007b). Perhaps we will -- such a development would not violate any known laws of physics. But do any readers have a more specific reason to suspect that the CODIS STRs profiles will turn out to powerful predictors of any medical conditions?

--DH Kaye

* On CODIS and STR profiles, see, for example, FBI brochure, NIJ webpage, NIST technical information

Additional Thoughts on Maryland v. Rose (Fingerprints)

Having just read the opinion in Maryland v. Rose, my initial reaction is an odd mixture of yawns and gasps.  On the unreliability of fingerprints, the opinion is in many ways completely unremarkable.  The arguments offered against fingerprints have now been around for quite some time.  What is remarkable is how long it has taken for courts to begin acknowledging the problems with fingerprints.

The broader aspects of the opinion, however, are arguably more fascinating.  First, the judge invokes the "death is different" concept almost like an incantation, and then says nothing more about it.  Is the judge suggesting that the holding be limited to death penalty cases only?  While one could develop a theory by which the constitution influences the interpretation of Rule 702 in certain contexts, that conclusion is not immediately obvious.   In addition, it seems that the criticisms of fingerprints are sufficiently serious that the problem is not just confined to death cases.

Second, I am astonished at how the court almost cavalierly sidesteps the issue of being in a Frye jurisdiction and subject to a "general acceptance" standard.  While I have previously argued that Frye and Daubert operate similarly in practice, never did I expect that an opinion from a Frye jurisdiction would feel so extraordinarily "Daubertesque." 

Finally, the opinion is a testament to how influential the Daubert criteria and mindset have become.  The court emphasizes the use of objective standards, testing, and error rates.  One gets the sense that expert intuition was summarily shown the door. 

--EKC

And more on Fingerprints....

Here's a follow-up story on Judge Souder's exclusion of fingerprints for not being based on "a reliable factual foundation."  See Here.   There are some insightful comments from Sandy Zabell and some not-terribly-insightful comments from Thomas P. Mauriello, an adjunct professor at the University of Maryland.  Indeed, a comparison of Zabell's (a statistician) and Mauriello's (a criminologist) comments nicely illustrates all that is wrong with forensic science as it is practiced today.  The former offers the critical comments of the scientist, and the latter the true-believer.  Evidence, not faith and anecdotal experience, should be the currency by which the forensic identification sciences are measured.
-- DLF

Exclusion of Fingerprint Evidence

A Maryland trial court excluded the State's proffer of partial latent fingerprint evidence last Friday.  The judge based her ruling on the State's failure to demonstrate that the technology produced valid results.  Although the court used the Frye test, the opinion reads like a primer on Daubert.  The Baltimore Sun has had a few articles on the case, and an editorial.  One article and the PDF of the judge's decision can be found at the following link: Here.   The site is fully searchable, so you should be able to find the other articles fairly easily, if you are interested.
--DLF