Moneyball-ing Legal Services (and Law Schools and Law Libraries)

We shouldn't be surprised that the new film, Moneyball, would lead to a discussion of the applicability of Moneyball-ing law firm hiring and retention, and the value of legal services provided to clients. Heck, seven years before the movie and within a year after the publication of Moneyball: The Art of Winning an Unfair Game (W.W. Norton, 2003) by Michael Lewis , law profs Paul Caren (Cincinnati) and Rafael Gely (then Cincinnati, now Missouri) sparked a flury of discussion in the legal academy with their landmark article, What Law Schools Can Learn from Billy Beane and the Oakland Athletics [SSRN], 82 Texas Law Review 1483 (2004). Quoting from the abstract:

In Moneyball, Michael Lewis takes an inside look at how in recent years the Oakland A's have achieved one of the best records in baseball despite having one of the lowest player payrolls. Cass Sunstein and Richard Thaler have argued that the book has large and profound implications for other professions. This review essay by a tax law professor and a labor law professor explores the book's large and profound implications for law schools.

...

In many ways, legal education is teeming with more inefficiencies than Beane uncovered in baseball. We argue that changes in the economic conditions of higher education and the legal profession, combined with increasing demands for accountability and transparency, created the market demand for measuring organizational success which U.S. News & World Report met with its annual law school rankings. We explore the implications of Moneyball for legal education in three areas.

First, we argue that law school rankings are here to stay and that the academy should work to devise ways to more accurately measure law school success. We advocate the comprehensive collection of data that users and organizations can weigh differently in arriving at competing rankings systems.

Second, we applaud efforts begun in the past decade to quantify individual faculty contributions to law school success. We support measures that take into account both quantitative and qualitative measurements of faculty performance. We provide data that confirm the relationship of productivity and impact measures of scholarship and provide support for isolating background and performance characteristics in predicting future faculty scholarly work.

Third, we use Billy Beane as a prototype and identify the qualities that enabled him to revolutionize baseball. We shift the focus here to deans and present data measuring decanal scholarly productivity and impact. We contrast these figures with the corresponding faculty data and distinguish deans' scholarly performance both in the period prior to becoming dean and while serving as dean. We also offer some surprising predictions, based on the data, of the qualities that a future dean will need to assume the mantle of the Billy Beane of legal education.

I'm thinking Paul, a diehard Boston Red Sox fan, and Rafael, if a baseball fan, hopefully a Cubs fan during his long-term stay teaching at Chicago-Kent Law, should have been offered cameo appearances in the Moneyball movie. But I digress.

Moneyball-ing in the Private Sector. So now comes two New Normal articles from the ABAJ. In If Legal Services Value Stats Were Created, Standardized, Law Clients Could Play ‘Moneyball’, Patrick J. Lamb writes

In sports, statistical analysis is a means to an end: securing the best win-loss record and winning championships. In law, the challenge is whether statistics can be a useful means of determining value, which is, like beauty, frequently in the eye of the beholder. But the efforts to at least circle around some common understandings of value are nothing but positive developments, and defining the kinds of analyses and statistics that are pertinent to hiring and retention of lawyers will assist law firms and clients in focusing on the same indices of value.

Paul Lippe asks and answers in Can ‘Moneyball’ Principles Be Applied to the Valuation of Legal Services?

Can we apply Moneyball-style analysis to law? The answer is a qualified “yes,” informed by three considerations:

First, value of services is inherently more nuanced than value in goods, and law is toward the more nuanced end of the spectrum of services (say more nuanced than a baseball player, a real estate agent or a travel agent, but probably less than a psychotherapist).

Second, value discussions have to be specific—value in sell-side mergers and acquisitions is different from buy-side, and altogether different from counseling to avoid employment discrimination claims.

Third, discussing value is always going to be useful, even though it doesn’t lead to one absolute standard.

Unfortunately, what many Moneyball commentators fail to emphasis is that once the statistical analysis used by the Oakland A's became institutionalized in professional baseball, the initial competitive advantage the A's had was lost. The playing field was leveled. Statistical analysis has become just another tool.

Quantifying value in legal services and in hiring and retention of lawyers is problematic at best as noted by Lamb and Lippe. Will Moneyball-ing in this context also level out the playing field? Without absolute standards, can the metrics be agreed upon? Will the stats used be accurate or gamed?

Moneyball-ing in the Legal Academy. While Caron and Gely's call for more comprehensive law school data so users and organizations can create alternative methodologies to US News Law School Rankings should be applauded, that hasn't happened and that wasn't what caused the flurry of debate in the legal academy. What did was the the second and third issue presented in their article, namely Moneyball-ing the legal academy in terms of law prof and law dean contributions to law school success by narrow focusing on the relationship between productivity and scholarly impact as if the game of the legal academy is careerism of law profs and their dean.

Isn' the game about producing practice ready graduates by an institutional scorecard of the outcomes of a profession education? One ouside the legal academy can't start Moneyball-ing that because even the current self-reported and unadited data provided by the "team" is unreliable. One might say, the legal academy's credibility issues compounded by the ABA failure to police law schools as their accrediting body has lead many to conclude that law schools have been Moneyball-ing gamed data to move up in the standings of the major leagues knows as US News Law School Rankings. And just like in baseball, the more teams that do this, the more level the fudged playing field is.

Moneyball-ing in Law Libraries. Well, actually first in academic law libraries. At least our data isn't known to fudged when it comes to the size of a academic library collection but I am reminded of the objections raised about the prospects of devaluing the volume count stat at a NOLA seesion. In our New Normal of Shed West Era-Digital First-Digital Only, I think the issue, the value of that metric, has been settled. It's not important.

Law librarian contributions to the success of their employers is not measured by the size of their collections but by the services they provide and are responsible for: saving costs by way of their negotations with vendors, executing efficencies by way of implementing sensible information technologies and e-communications, providing expert services to their user populations. 

Our duty is to our institutions but there is little no chance that some new metrics can be created. Providing a metric on expert services is too subjective unless opinion polls are based on scientific principles of opinion research. Just slapping together a bunch of questions for SurveyMonkey doesn't cut it.

There are metric that can be used to evaluate library websites in terms of traffic, what library website pages are visited the most, etc., as long as one carefully understands the limitations of web server logs, plus the frequency of updating library site pages, adding new ones and contact a librarian in real time by way of web links. One could also add law library (not law librarian) blogs, Facebook pages, Twitter accounts. In addition to traffic, frequency of posting institutionally relevant messages, plus the life-span of these alternative forms of web communications could be measured, if anyone really cared to do so.

Then there is cost saving by way of law library-vendor negotiations. Thanks to NDAs we can't share that data in any way, even if specific institutional identities were replaced by alternative identifiers. Hell, I don't think we can even share database-by-usage stats by type and subtype of law library market sectors. At least we can't do that via AALL mediums because that would be "anti-competitive." [JH]