Wednesday, April 13, 2016
Should Defendants Get Access to TrueAllele's Source Code?
A few days ago, the Huffington Post published a piece on the TrueAllele Casework system. According to the piece,
Cybergenetics, developer of computer automated systems and technology research data analysis, claims its TrueAllele Casework system prevents wrongful convictions by accurately matching the DNA of the perpetrator to the DNA evidence. TrueAllele’s computerized DNA interpretation system excels in situations where human forensics fail—when evidence contains a mix of three or more DNA samples. However, Cybergenetics’ refusal to share the source code behind the software proves problematic in courts. This source code, or programming code, is the key to software function. If Cybergenetics releases the code, its competitors could replicate it. But without the programming code, defense attorneys are unable to challenge the accuracy of TrueAllele. Likewise, prosecutors can’t authenticate it.
For $60,000, crime labs can buy TrueAllele software. According to Cybergenetics’ TrueAllele Process Overview Video, an analyst first assays the DNA evidence following a typical procedure such as PCR, a DNA amplification process. This DNA evidence can range from bodily fluids to skin cells. After the evidence is scanned, the computer fitted with the TrueAllele software finds the length and quantity of every data peak. Through complex, undisclosed codes and algorithms, the computer separates DNA mixtures into genotypes, solves kinship and paternity, and calculates match statistics.
Apparently, "[t]his groundbreaking technology helped convict criminals in over 500 cases in the past five years, with the majority of those convictions occurring last year." But is it reliable?
That was the question raised by Michael Robinson, who has been charged with murder.
Prosecutors used TrueAllele to link Robinson to DNA evidence found on a bandana near the crime scene. TrueAllele found that the DNA was 5.6 billion times more likely to belong to Robinson than to another suspect. If Robinson is convicted, he faces the death penalty. Relying on the Sixth Amendment Confrontation Clause, Robinson’s defense attorneys claim[ed] that access to the programming code is necessary in order to cross-examine Mark Perlin, founder of Cybergenetics and TrueAllele’s creator.
In an opinion in February, however, a Pennsylvania court denied Robinson's attorneys such access. According to the opinion,
release of the source code would not be reasonable under Pa. R. Crim. Pro. 573 (A). Dr. Mark Perlin, founder of Cybergenetics, stated in his April 2015 Declaration that disclosure of the source code would cause irreparable harm to the company, as other companies would be able to copy the code and potentially put him out of business....An order requiring Cybergenetics to produce the source code would be unreasonable, as release would have the potential to cause great harm to Cybergenetics. Rather than comply, Dr. Perlin could decline to act as a Commonwealth expert, thereby seriously handicapping the Commonwealth's case.
I'm sympathetic to the potential damage that compelled release of the source code could cause to Perlin's business and the State's case. But what about Robinson, who could be given the death penalty if convicted? According to defense counsel,
“Part of confronting a witness, especially an expert witness, is to be able to know what that expert is doing to arrive at their conclusions. Without the source code, Mr. Robinson cannot exercise his Sixth Amendment right.”
He's not alone in these concerns.
Courts in at least seven states have admitted TrueAllele results as evidence, against objections by defense lawyers whose clients have been linked to crimes by the software.
Pennsylvania will not be the exception. Robinson later withdrew his appeal of the court's ruling.
After reviewing everything related to TrueAllele, I think it is probably reliable. But is "probably" good enough?
In principle, shouldn't it be possible to conduct an independent test of the technology without access to the source code?
For example, you don't need to know the machinery inside a polygraph to show that it is useless, you can just do the properly blinded tests.
Posted by: anon | Apr 14, 2016 5:49:07 AM
PBS NewsHour ran an interesting piece last week about Texas and its Junk Science Law: http://to.pbs.org/1Xpg4Hz "It offers convicts a direct path to an appeal when there is new scientific evidence or evidence that contradicts what was used to convict," the article states. Until there is evidence to establish without a doubt the reliability of this DNA technology, I'd be proactive and put it in the JS category now. There are already too many wrongfully convicted people, and there will be more still if due process continues to be ignored.
Posted by: streetwriter | Apr 14, 2016 6:32:41 AM
What Anon maybe suggesting is something like a blinded control test to assess the accuracy and sensitivity of this TrueAllele software compared to other software. A blinded experiment could be created using a panel of mixed DNA samples, which would in theory contain varying amounts of DNA from several individuals. From this you should be able to analyze the software’s accuracy in correctly separating sequences as well as its level of sensitivity in detecting samples present in only trace amounts.
Posted by: San | Apr 14, 2016 11:55:24 AM
It should be treated as expert testimony. If the "expert" can't explain itself then right there it's not valid expert testimony.
Posted by: Megan Pawlak | Apr 14, 2016 4:16:11 PM
I'm surprised that disclosure wasn't ordered to an expert bound by privacy strictures. My understanding that such disclosures are routine in patent cases where software methods are in dispute but disclosure of source code would be economically damaging.
On the wider point, as an engineer, I would be concerned by the use of this type of software unless formal methods had been used, as would be required in safety critical or aircraft systems. However, this only ensures that the software correctly implements the specified technique that has been designed, you would also need to be able to validate that technique reliably produces a correct match(es) which is a question of biochemistry.
Posted by: Alex | Apr 15, 2016 12:25:01 AM
It would be unfair to release the company's proprietary source code.
That said, this is a very good reason why it shouldn't be getting used in a court of law. Simple as that.
Posted by: Paul | Apr 15, 2016 5:31:45 PM
Computers can make mistakes just like a human being. Nothing is infallible.
Posted by: Jen Roy | Apr 17, 2016 7:52:49 PM
This sounds like the plot of some dystopian sci-fi thriller. To borrow phrases from Marc Goodman's book "Future Crimes," without the source code we've no idea how much "garbage in, gospel out" is going on, at a time when we are blindly following the credo "In screen we trust." He adds, prophetically: "Mo' screens, mo' problems." To not know how the software is arriving at its conclusions is like having an eyewitness testify anonymously, from behind a curtain. If I were a defendant, I'd be outraged in such a situation, and even more angry if the court accepted that whatever the witness says must be true. That reminds me of how the cellphone tower evidence was presented in the Syed case. And we know now that it was not very accurate, after all.
Posted by: streetwriter | Apr 13, 2016 2:20:34 PM