April 20, 2007
Predicting Dangerousness and Flipping Coins
On April 20, NPR's Morning Edition broadcast an interview with Phillip Merideth, "a forensic psychiatrist and the chief medical office for Brentwood Behavioral Healthcare in Mississippi [and] a lawyer who teaches about mental health and the law." According to Dr. Merideth, "[p]sychiatric literature in the past has shown that efforts to predict — and I'm using the word predict in quotes — is no better than flipping a coin," Merideth says.
One wonders what Dr. Merideth knows about coins and probability, since he also says "you can predict who is at risk of committing violence. But you can't actually say who will." So which is it? Predictions based on flipping a coin are totally worthless. They bear no relationship to the actual risk. Are the psychiatric predictions equally worthless -- or are they at least slightly informative? You cannot have it both ways. Either risks can be identified or they cannot be. If they can be, the question then becomes how much better mental health professionals do than coins.
The issue confused Justice Blackmun in Barefoot v. Estelle, 463 U.S. 880 (1983). Quoting Ennis & Litwack, Psychiatry and the Presumption of Expertise: Flipping Coins in the Courtroom, 62 Calif. L. Rev. 693, 737 (1974), his dissenting opinion insisted that "[i]t is inconceivable that a judgment could be considered an `expert' judgment when it is less accurate than the flip of a coin." Consistently less accurate? A few seconds of reflection should reveal that this is impossible. How can anything be less accurate than flipping a coin?
The studies cited in the American Psychiatric Association's amicus brief in Barefoot actually demonstrate that the best predictions of mental health experts at the time were substantially better than coin tossing (a likelhihood ratio of about 8 rather than 1). See, e.g., Chrisopher Slobogin, Dangerousness and Expertise, 133 U.Pa. L. Rev. 97 (1984).
Of course, I am not arguing that psychiatric predictions are extremely accurate. In general, they are not. My point is simply that Dr. Meredith's statements about coins and pyschiatry are plainly false, and that debates over mental health law should be informed by more accurate information. Such information was provided to a different NPR program Talk of the Nation, just two days ago. John Monahan, Professor of Law and Professor of Psychology and Psychiatric Medicine at the University of Virginia Law School, pointed out that although "[i]t's been know for a very long time that psychologists and psychiatrists are not very good at predicting violence, ... they are better than chance." They may not be "very much better than chance," but "in recent years the science has improved considerably." The bottom line is that "our crystal balls are very murky, and we have a long way to go."