Another Example of Using Big Data to Improve Odds of Winning in Court

Back in February, I wrote a post on The The Early Days of Legal Analytics.  It discussed some of the innovations at Lex Machina, a legal start-up that uses Big Data to value contested patents and develop a litigation strategy designed to maximize value / minimize risk.  I recently came across another company, Premonition, that claims to use artificial intelligence to select lawyers with the best odds of achieving a favorable result. See Premonition Infographic at bottom of this post.

I spend a lot of time on the road talking to law firm lawyers and legal innovators, including legal start-ups.  Many large firm lawyers tend to dismiss new innovations without stopping to listen to, much less gather, relevant facts.  Likewise, there is a lot of puffery among legal start-ups as they try to land their first few customers.  Thus, I tend to apply a windage factor to accurately interpret what I am being told.

The image below contains Premonition's simple, one-sentence pitch.  I don't know about the product, but the concept is pretty clear.

The core benefit Premonition appears to offer is a list of lawyers with winning track records in front of specific judges. We don't need artificial intelligence (AI) to make that calculation.  A win-rate is a simple descriptive statistic, even if it has been filtered for a variety of matching criteria.

That said, AI could come in handy in building the requisite data sets.  As explained on Premonition's website, courts don't construct their case management systems so they can be vacuumed out by data mining companies. Indeed, local court officials would likely to be hostile to such requests because any resulting statistical model is unlikely to make them look good.  Indeed, the purpose of the model is, at least in part, to identify and exploit imperfections in the judicial process.  

Because the courts have no incentive to make life easy for Big Data vendors, Premonition's Chief Innovation Officer, Toby Unwin, claims to have tackled the data assembly problem by building a technology that scrapes and buckets the necessary data from the jumbled chaos of web portals for state and local courts.  Such a task, in theory, can be performed by fairly standard machine learning, which qualifies as AI, at least in some circles.  

Assuming Premonition has built a machine that can calculate win-rate of lawyers, is that information valuable to clients trying to maximize the likelihood of a favorable result?  I don't know, but its plausible enough to test with data.

Some data skeptics will argue that win-rates, whether high or low, are just artifacts of any normally distributed outcome.  The reasoning runs, "Two, three, and four sigma events occur in the ordinary course of life, but regression to the mean is pulling them back to the center. Thus, they are poor predictors of the future."  This reasoning is why many people buy indexed funds rather than shares in actively managed mutual funds.  Cutting the other way, the hedge fund industry is premised on the belief that some money managers are a lot better than others. Five-year return rates are aggregated and published in the industry trade press.  Some of the returns may be due to random luck, but some could be attributed to superior skill.  It is absurdly unlikely, for example, that Warren Buffet's success in buying and selling stocks is just a 60-year lucky streak.

In the case of win-rates in court, I can think of at least two plausible non-random factors that could affect outcomes:

1. Judicial bias or favoritism.  Judges, either consciously or unconsciously, may react differently to the case depending upon the advocate.  One does not have to wade too far into the political science literature to find peer-reviewed empirical studies that reveal that judges are influenced by more than just facts and law. 
2. The gap between credentials and bona fide skill.  Law has historically been a credence good.  This means the market relies on elite credentials and firm reputation as a proxy for skill.  Yet, it is plausible that some lawyers may lack the pedigree to get hired by large, elite law firms, yet they go on to develop outstanding legal skills, perhaps because of superior drive, intellectual curiosity, or "early at-bats" as a prosecutor or public defender. If these folks exist, Big Data can likely find them.

I can't vouch for Premontion's technology beyond two statements: (1) it sounds plausible, and (2) it is a waste of time to debate its usefulness because it's an empirical question that the market will answer in the relatively near term.  

Below is one of Premonition's infographics.