Thursday, August 14, 2014
D. Alex Winkelman , David V. Yokum , Lisette C. Cole , Shelby C. Thompson and Christopher T. Robertson (University of Arizona - James E. Rogers College of Law , University of Arizona - James E. Rogers College of Law , affiliation not provided to SSRN , affiliation not provided to SSRN and University of Arizona - James E. Rogers College of Law) has posted Randomized Experimentation as an Unbiased and Transparent Method for Harmless Error Analysis (Arizona State Law Journal, forthcoming 2015) on SSRN. Here is the abstract:
Trials are often imperfect. When inadmissible evidence is erroneously introduced or the jury is erroneously instructed on the law, appellate judges must determine whether the error was prejudicial or merely harmless. In making that assessment, appellate judges can only resort to speculation about the counterfactual question of whether the error changed the outcome, compared to the decision of a properly informed and instructed jury. Behavioral science suggests the appellate judges’ decisions are likely colored by confirmation biases, status quo biases, and “mental contamination” by exposure to the error. Even when appellate judges perform these analyses accurately, their decisions appear conclusory and illegitimate. Scholars and judges have roundly criticized this doctrine, but no solution has emerged.
We develop and pilot an unbiased and transparent method for making harmless error determinations, using randomized experiments with simulated jurors.
We propose that this method could be used to inform real harmless error decisions in real cases. Our results showed a high degree of correspondence between the assessments of real judges and our experimental method, which could be taken as a validation of the method and reassurance that it would not cause a radical change in the rates at which new trials are granted Still, across the thousands of cases in which harmless error determinations are made each year, there are reasons to expect that this empirical method will be more accurate in sorting the harmful from the harmless errors, since the method avoids the biases that likely infect current decisions by appellate judges. The transparency of our method should lend greater legitimacy.
For such a simulation method to be useful in real cases, courts must resolve questions about how litigants should get these results to appellate judges and must specify how much prejudice is too much, while also being sensitive to the limitations of statistical power, especially when prosecutors try to prove a negative. Our study is most useful as proof of concept.