Sunday, June 29, 2014

External vs. Internal Validity in Empirical Legal Studies

Last week, we have hosted in Israel a conference on Behavioral Legal Studies dealing with Cognition, Motivation and Moral Judgment. The conference was organized by Hebrew U and Bar –Ilan U and involved leading international psychology and legal scholars.

An interesting discussion that emerged in the conference, due to the mixed audience of psychologists and legal scholars was related to the interaction between psychology, behavioral economics and empirical legal studies.

This tension was demonstrated through the presentation of my longtime collaborator Doron Teichman on Anchoring legal standards. (Feldman, Schurr & Teichman, 2014).

Anchoring in short is usually defined as ”a cognitive bias that describes the common human tendency to rely too heavily on the first piece of information offered when making decision” (Shrotriya & Pandey, 2013).   

The classical studies in psychology (e.g. Tversky & Kahneman, 1974) which examined anchoring usually relied on the effect of some random 4 digits number or the number that emerged from a wheel of fortune on people’s judgment. In contrast, the studies done by legal scholars, including us, are usually related to damages asked by lawyers or numbers that emerged from a different legal source.

The mostly justified criticism of this argument is that legal usage of anchoring is not a pure anchoring effect because the original stimuli is not completely orthogonal to the target, meaning the stimuli is not clean and might carry other types of rational influence. Clearly, the number argued for by a lawyer, even in an adversarial system, does have some meaning regarding the size of the claim, relative to the wheel of fortune example in the original psychological experiment.

That being said, we ask ourselves whether empirical legal studies need to compete with psychology on the same term. If we take the example of the wheel of fortune vs. the number coming up from a legal suit, clearly the first one carries much greater internal validity – any kind of effect in the first case could not be interpreted as nothing but a cognitive bias. Admittedly, this is not the case when we look at the influence in judgment following a legal claim.

However if we see empirical legal studies as an academic community which tends to understand how cognitive biases might affect the legal system, clearly, the likelihood that someone would use a wheel of fortune to try and affect a judgment is highly unlikely.

Hence, nn my view, empirical legal studies should recognize that trade-offs do exist and that measuring pure effects is not the only measure of good science.

Science | Permalink


Post a comment