Wednesday, September 9, 2015

"In Praise of Law Reviews (And Jargon-Filled, Academic Writing)"

SunsteinThat is the title of a forthcoming article by Cass Sunstein in the Michigan Law Review.  Sunstein has unusual standing to make this case because, in addition to his academic perches at Chicago and Harvard Law, he was tapped by President Obama to lead the Office of Information and Regulatory Affairs

Sunstein has written a remarkably thoughtful and balanced essay that I would encourage any fairminded lawyer, law student, and law professor to read.  Sunstein begins by recounting how pulling the levers of power in government made several of his fellow academics despair over the prospect of returning to academic writing. When Sunstein probed further, a colleague sent along a passage from Theodore Roosevelt:  

"It is not the critic who counts; not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood ... ."

It's a powerful passage that can be used to diminish those who write academic articles. But Sunstein subsequently references another quote, this one from John Maynard Keynes:

[T]he ideas of economists and political philosophers, both when they are right and when they are wrong, are more powerful than is commonly understood. Indeed the world is ruled by little else. Practical men, who believe themselves to be quite exempt from any intellectual influence, are usually the slaves of some defunct economist.

Sunstein acknowledges that Keynes' message is self-serving and unnecessarily demeans the intellect of those who carry the burden of leadership. But he also sees a kernel of truth--difficult problems often get solved by applying the grand ideas and concepts of academics.  The impact of academics is rarely immediate, but it can be enduring and profoundly influential.

To illustrate his point, Sunstein identifies a list of seven recent books by academic authors (by Balkin, Vermeule, Mashaw, Kaplow & Shavell, Revesz & Livermore, Cross, and Adler). All of them can trace their origins to earlier law review articles, all of them are hunting "big game", and most if not all of them are unlikely to be of immediate practical to the busy practitioner or judge.  But Sunstein suggests that we should be taking a longer view. Some of these books were on his shelf when he served in the government. He used them to address real world problems. The rest are scaffolding to reach something higher.

Sunstein organizes the core of his essay around the criticisms of the late Yale law professor Fred Rodell, suggesting that the author of the famous Goodbye to Law Reviews got it only half right.  Sure, the style and length of law review articles limit their readership, but Sunstein observes some countervailing benefits:

When they are working well, law reviews strongly discourage arguments that are glib, sloppy, circular, or narrowly ideological. They also require both development of and sympathetic engagement with competing points of view, rather than easy or rapid dismissals. Counterarguments are strongly encouraged, even mandatory. There is a kind of internal morality to the genre, one that is (I think) connected with and helps account for some of its rigidity. The morality involves respect for the integrity of the process of argument, which entails respect for a wide range of arguers as well. 

As someone who has written numerous law review articles, this description strikes me as entirely accurate.  Most of my work these days is applied--designing and measuring law school courses, evaluating outcomes, and trying to re-start the labor market so it clears on something more that LSAT scores of entering students.  Within this applied realm, which borders on the arena, I am often viewed as someone who is highly creative.  Yet, I can safely say that nearly all of the credit goes to the mental discipline and knowledge obtain through academic writing. That process fundamentally transformed my intellect. Further, I felt that way at the end of my first law review comment, which took roughly 500 hours to research and write during my 2L year of law school.  So I wanted to do it again.  

A lot of smart people in law tend to focus on what is immediate and practical--i.e., what will help with the work on their desk.  I can see this mindset in nascent form in a subset of my students who become impatient with classroom forays into legal theory or the social sciences.  I don't think this group can be won over.  By disposition, they can't see the value in reading academic work, so paying for its production seems even more pointless.

Granted, this conclusion does not resolve the harder issue of whether the current system of legal education over-incentivizes the production of legal scholarship by mandating, through ABA and AALS requirements, that most teachers be academic scholars. What is the optimal number of lawyer-scholars who should be subsidized by student tuition as opposed to grants or endowment? It may be less than the current number. Further, how those spots get allocated is another challenging issue with no simple resolution.  

That said, Sunstein is clearly right--whether they realize it or not, every capable legal problem-solver is standing on the shoulders of prior academic work.  It is misguided to conclude that future generations won't need new and better ideas vetted through an academic process.

September 9, 2015 in New and Noteworthy, Scholarship on legal education | Permalink | Comments (1)

Tuesday, September 1, 2015

What's driving the demographic gap between BigLaw leaders and their CEO/GC clients?

Picture1Las Vegas, NV.  The illustration to the left was just published in The American Lawyer.  It accompanied a story on how law firm leaders are significantly older than leaders in the large corporations they serve. See MP McQueen, The Generation Gap:  BigLaw's Aging Leaders, Aug 24, 2015. 

At least for me, this is a jarring graphic because it conveys so much truth.  Today's Millennials are so underwhelmed with the BigLaw model.  They like the pay and the perks, as it enables them to live well in attractive large market cities.  They can also quickly pay off their law school debt.  But precious few of them are all in. Illustrator James Steinberg totally nailed it.

There are numerous reasons for the culture divide, but as shown in the chart below, the most obvious is a very large age gap between leaders and entry-level workers -- it tends to be a lot larger in BigLaw than almost anywhere else: 4% of AmLaw 100 leaders are Gen X compared to 33% of NASDAQ-traded companies. 

These data beg the question, why are large law firms so out-of-sync with the institutions they serve?

One reason is certainly the ownership structure.  Any Fortune 500 or NASDAQ-traded company that got this top-heavy in its senior management would be getting killed on its stock price.  Under the Rule 5.4 prohibition on nonlawyer investors, law firms are spared the anxiety of having analysts and short sellers constantly evaluating their business. Yet, the absence of a public market means that law firm owners and managers cannot fully monetize the enterprise value they create. So what's the effect?  Very little enterprise value gets created.  Instead, lawyer/owners  focus on maximizing this year's net distributable income.

It is important to not knock the BigLaw model too hard.  For about a century, it worked extremely well, as US law firms steadily grew with their clients.  Each unit of economic growth produced some larger unit increase in legal complexity, so demand for sophisticated legal services was a steady upward sloping line. By following a simple model -- hire more associates, promote some to partners, lease more office space, repeat -- equity partners in the AmLaw 100 became millionaires.  

Today, BigLaw is getting grayer because the 100-year old gold factory is breaking down. Law firms' portion of corporate legal spending is no longer growing, as in-house lawyers, NewLaw managed services shops (United Lex, Axiom, Counsel on Call), and technology are all curbing demand for traditional law firm services.   The best economic play for 55- or 60-year old equity partner is to ride out the existing model with the dwindling but still substantial number of Baby Boomer senior in-house lawyers who are themselves not too anxious to change.  

This is not the story equity partners tell themselves; it's the logic that underlies the inertial path.  It's where we end up when we are no longer deeply invested in the places we work.  It's become a job.  I am not judging here; I'm describing what I have observed through hundreds of conversations with large firm partners.

The result of this dynamic is that a large proportion of BigLaw--but certainly not all of it--is just tinkering at the margins of change. A law firm can become more cost-effective for clients, at least in the short to medium term, by reducing reliance on associates.  Associates are expensive and are, by definition, getting paid to learn.  For the last 15-20 years, firms have shifted their leverage model to counsel, staff attorneys and nonequity partners, where (a) there is little to no training, (b) the margins are higher, and (c) the clients can't complain about inefficient associates. This is the Diamond Model, which substantially cuts out the entry-level lawyer.  See The Diamond Law Firm: A New Model or the Pyramid Unraveling? (Dec. 2013); Sea Change in the Legal Market, NALP Bulletin, Aug. 2013.


Unlike the original Pyramid Model, invented by Paul Cravath circa 1910, the Diamond Model is not a carefully conceived business strategy. Rather, it's a way to maximize this year's and next year's net distributable income without making difficult strategic tradeoffs. Yet, in the longer term, which is no longer too far off, the Diamond Model is a disaster.  The few associates who make it into large firms are grateful for the high pay and the training.  But very few if any are impressed with the business model.  Among Millennial lawyers, in-house is the new brass ring.

Law firms are filled with brilliant people. Why are they going down this road?  Three interrelated reasons:

  1. Lack of Experience. Today's law firm partners have little or no experience with strategy--for a hundred years, intelligence and hard work worked just fine.  This is not a change in strategy--it is having a strategy.  Then executing.  That's hard.
  2. Incentive Structures. Virtually all incentives inside firms today favor revenue generation; as a result, few partners have the mental whitespace to understand, much less think through, the changes that are occurring within the broader industry.  To fix the bridge, you have to slow down the traffic.
  3. To Big to Fix.  The first strategy mistake for the current generation of AmLaw 100 leaders was to become bigger without becoming measurably better.  Big firms filled with laterals is a difficult environment to share risk. Maximizing this year's distributable income becomes one of the few things people can agree on.

That said, I am not counting BigLaw out.  I am writing this blog post from the International Legal Technology Association (ILTA) conference in Las Vegas.  From far away, it is all too easy to treat BigLaw as a monolith--it's not.  At ILTA, professionals from several of the most innovative law firms are willing to pop the hood and share what they doing.  See Ahead of the Curve: Three Big Innovators in BigLaw, Aug. 25, 2014Suffice it to say, some firms are several years into strategies that have the potential to take market share from peer firms.  Further, the innovation teams inside these firms are having the time of their professional lives because the work is so collaborative and creative--the antithesis of billable hour work.  What is also clear is that many competitors just can't muster the leadership nerve to make similar investments. 

In the years to come, some BigLaw firms are going to pull away from the rest, becoming a magnet for talent and then clients.  Younger lawyers are going to thrive there.  Another portion of BigLaw is going to gradually fade away. 

September 1, 2015 in Current events, Data on the profession, Law Firms, Structural change | Permalink | Comments (4)

Wednesday, August 12, 2015

Of Transfers and Law-School-Funded Positions

1.      Many Elite Law Schools with Large Numbers of Transfers also Have Large Numbers of Law-School-Funded Positions

Several weeks ago, I participated in two separate conversations.  One was about when law-school-funded positions should be categorized as full-time, long-term, bar-passage-required (FLB) positions and one was about transfer students.  This prompted me to compare those schools that are “big players” in law-school-funded positions with those schools that are big players in the “transfer” market.  Interestingly, as shown in the chart below, there is a significant amount of overlap.

For the Class of 2014, of the 15 law schools with the most graduates in FLB positions, ten had a significant (net) number of transfer students in the summer of 2012.  (The chart is sorted based on 2014 FLB positions (in bold).  To provide context, the chart also includes the 2011 net transfer data and 2013 law-school-funded FLB data for these 10 schools.)

Law School

2011 Net Transfers

2013 Law-School-Funded FLB

2012 Net Transfers

2014 Law-School-Funded FLB

























































Note that in both 2013 and in 2014, six of the ten schools had more transfers than law-school-funded positions, suggesting that had they taken fewer transfers they might not have needed to provide as many law-school-funded positions. Phrased differently, this data suggests that with the transfer students, these law schools have too many graduates compared to the number of jobs the market is able to provide for their graduates.

2.      Adjusting to the Employment Market or Continuing to Attract Transfers and Provide Law-School-Funded Positions?

One might expect that a natural response to this “mismatch” between the number of graduates and the number of meaningful employment opportunities provided by the market would be to have fewer graduates (and fewer law-school-funded positions).  Indeed, for many of the schools in the chart above, the simplest way to do this would involve not accepting any transfer students (or accepting very few transfer students).  The first-year enrollment at these schools appears to be fairly-well calibrated with the number of meaningful employment opportunities provided by the market.  Of course, this would mean a significant loss of revenue.

But what happened at these ten law schools in the summer of 2013 and the summer 2014 with respect to transfer students? As shown in the chart below, almost all have continued to take large numbers of transfer students.  With knowledge that a not insignificant percentage of their graduates need the support of law-school-funded positions because they can’t find market positions, these law schools continue to take large numbers of transfers.  Indeed, the total number of net transfers at these ten law schools is even higher in 2013 and 2014 than in 2011 and 2012.



2014   Net Transfers

2013   Net Transfers




































3.   Why are These Schools Continuing to be Big Players in the Transfer Market and in Providing Law-School-Funded Jobs and Why Aren’t Other Schools Doing This as Well?

Many elite law schools are participating heavily in the transfer market and in providing law-school-funded jobs because they can and because it makes financial sense to do so.

As a general matter, only relatively elite law schools are able to attract large number of transfer students willing to pay $50,000 per year in tuition.  (This assumes that most transfers are paying full tuition. There is very little information available about scholarships in the transfer market, but anecdotes suggest that scholarships are uncommon.)  By taking large numbers of transfers, these schools generate revenue that funds general operations AND enables the school to fund post-graduate employment opportunities for a significant number of graduates.  According to NALP, most graduates in law-school-funded positions receive salaries of roughly $15,000-$30,000 per year.  Even if they have as many law-school-funded positions as they do transfers, the schools still net $70,000 to $88,000 per transfer student over the second and third year of law school even after accounting for the salaries for law-school-funded positions. (To be fair, some modest percentage of law-school-funded positions at several of these law schools may be great opportunities that are highly competitive and pay a salary comparable to a market salary – in excess of $40,000 per year.  Some of these may be public interest opportunities that some students find particularly attractive.  But the proliferation of law school funded positions (having grown from just over 500 in 2012 to more than 800 in 2014), with most of the growth occurring at relatively elite law schools, suggests that many of these positions do not fit the profile described in the preceding two sentences.)

Other schools would love to do this, but most simply don’t have the ability to attract significant numbers of transfer students.  Moreover, in the present legal education environment with declining enrollment at most schools, many law schools are running a deficit, and simply can’t afford to invest money in law-school-funded positions for their graduates.

Notably, up until this year, this effort was aided by the reporting of law-school-funded jobs as if they were the same as jobs provided by the market.  A school with law-school-funded positions that otherwise met the definition of FLB positions could report a higher percentage of its graduates in such positions.  This minimized the extent to which less than robust employment results might erode the schools’ ability to attract students and has allowed these elite schools to continue to attract large numbers of relatively highly-credentialed entering students (and transfers) along with the revenue they bring to the school.  For the Class of 2015, however, these law-school-funded positions will be reported separately from FLB positions provided by the market.

4.      What Questions Might this Raise for Students?

Students considering transferring to one of these elite schools should ask two questions: 1) What percentage of law-school-funded positions went to transfer students? and 2) How do employment outcomes for transfer students compare with employment outcomes for students who began at the school as first years?  (Even with the increased attention on transparency with respect to employment outcomes, one data point not presently collected relates to employment outcomes for transfer students.)  This isn’t to suggest that all transfers end up in law-school-funded positions.  Some transfer students may outperform some of the students who began at a law school as first years, both in terms of academic performance and in terms of relationship skills.  These transfer students may outcompete original first-year students for market employment opportunities.  But students considering transferring might want to assess whether their employment prospects really will be better at the school to which they might transfer as compared with the opportunities available to them if they remained at the school from which they are considering transferring, particularly if they are near the top of the class at the school from which they are considering transferring.

Students who had matriculated as first-years at one of these elite law schools, might want to ask the law school administration how and why having a large number of transfers is a good thing for those who matriculated as first-years at the elite law school.  Having the additional revenue might enhance the educational experience in some way, but having significantly more students competing for jobs would seem to be an unnecessary challenge. 

5.      Conclusion

The data on transfers in 2013 and 2014 suggests that at many elite law schools, there will continue to be more graduates than jobs provided by the market.  As a result, these law schools are likely to continue to provide law-school-funded positions for some number of their graduates. Indeed, the prospect of law-school-funded positions as a fall-back option if a market position is not available might provide some solace for students, including transfer students, at these elite law schools. 

Nonetheless, there is a further ripple effect.  With dozens of graduates from these elite law schools in law-school-funded positions looking for market jobs, it makes it even more challenging for the next year’s graduates from these elite schools to find market jobs and almost assures that many graduates will still need the support of law-school-funded positions in the coming years.

(I am grateful to Bernie Burk and others for helpful comments on earlier drafts of this posting.)

August 12, 2015 in Data on legal education, Data on the profession | Permalink | Comments (0)

Thursday, August 6, 2015

How is the entry-level legal job market in Australia?

AlsaNot good.  There are more law graduates than jobs, yet law schools are making matters worse by admitting more students in order to generate subsidies for other parts of the university. That the basic charge of the Australian Law Students Association (ALSA), according to this story in the Lawyers Weekly, a publication that covers the business of law in Australia.

Legal education is Australia is very different than the U.S.,  yet the dynamics of the two entry-level markets seem to be converging.  Law has historically been an undergraduate degree in Australia (LLB), but in recent years the JD has been added as a new and more prestigious way into the profession. Here is the statement of an ALSA spokesperson based on recent survey results of the ALSA membership.

ALSA are of the position that there is still an oversupply of graduates because of the increasing sizes of law schools and the duplication in the number of law schools across the country. ...

Many who have undertaken the Juris Doctor particularly expressed concerns in their survey responses, highlighting that they undertook the postgraduate law degree to further their job prospects. Instead, they are facing the worrying reality that there are fewer jobs available for law graduates as well as the fact that they are completing their degrees with a sizeable student debt.

The article then goes on to describe growing law student anxiety over employment and student loan debt.  Wow, different system but a very similar result.  

One of the advantages of the Australian LLB degree is that it is often combined with another undergraduate degree, typically by adding one year of additional study.  As a result, many LLBs don't go on to qualify for practice, but the legal training probably augments their worldly knowledge and critical thinking skills.  But alas, the Australians are starting to dilute their extremely generous higher education subsidies -- we are just much further down that road. Further, the true undercurrent here is the growing insecurity facing virtually all knowledge workers, Australian or US.  Legal education is just the bleeding edge of this problem.

August 6, 2015 in Current events, Data on legal education, Data on the profession, New and Noteworthy | Permalink | Comments (0)

Tuesday, August 4, 2015

Metrics and Legal Ops Professionals

In a recent post, I urged readers to visit a legal department with a large legal operations staff.   The goal?  To see the future of modern corporate law practice.  Fortunately, Bloomberg Law recently videotaped a legal ops panel moderated by Amar Sarwal of the ACC.  It contains a conversation rarely if ever heard in law schools or bar associations.

The three legal departments profiled are AIG (insurance), Marsh & McLennan (diversified financial and professional services), and GlaxoSmithKline (pharma).  Note the enormous emphasis on metrics, data, and technology.  Note also how the services of law firms are being put through a procurement process. 

August 4, 2015 in Blog posts worth reading, Current events, Data on the profession, Law Firms, Legal Departments, New and Noteworthy, Video interviews | Permalink | Comments (0)

Saturday, August 1, 2015

Legal Analytics to Build Substantive Legal Strategy

RavellexmachinaLex Machina and Ravel Law are two start-ups in the legal analytics space (I call companies like this "toolmakers"). Per an invite that arrived in my email yesterday, these Lex Machina and Ravel are collaborating on an upcoming webinar that shows lawyers how to use legal analytics to build a substantive legal strategy.  And not just any strategy, but one more likely to win.

Is this smoke and mirrors or the real deal? That remains an open empirical question.  But it's worth noting the venture capitalists who backed these companies are betting on the proposition that humans + machines > humans or machine. (This is Dan Katz's formulation.) For lawyers interested in staying current on the legal market (most of us, right?), it's likely worth attending.   Sure, Lex Machina and Ravel will be hawking their own products, but hey, educating your prospective clients is how one builds a market for complex technical products.  This skill will eventually be taught in law schools -- at least the good ones.  It is worth noting that both Lex Machina and Ravel trace their origins back to Stanford Law.  

Here is the essential information on the webinar.

How can you employ Legal Analytics in your practice to get clients and win cases?

New legal technologies are transforming the practice of law by enabling lawyers to uncover trends in the behavior of judges, parties, law firms and attorneys.

Ravel Law's case law analytics and Lex Machina's IP docket analytics enable lawyers to make data-driven decisions about case strategy and tactics.

Click here to register for a live 45-minute webinar on Thursday, August 6th at 11:00 am PT.

Host: Ralph Baxter, Advisor, Writer, and Speaker, Former Chair, Orrick 

Speakers: Owen Byrd, Chief Evangelist & General Counsel, Lex Machina & Daniel Lewis, CEO & Co-Founder, Ravel Law

Related posts:

August 1, 2015 in Current events, Innovations in law, New and Noteworthy | Permalink | Comments (0)

Wednesday, July 29, 2015

"Solicitors 'in denial' about threat from accountants"

Legalservices (321x207)That's the headline from today's Law Society Gazette, the publication of record for solicitors in England and Wales.  The UK is fairly far along in liberalization of its legal markets, progressing from the Clementi Report in 2004 to the Legal Services Act 2007 to the licensing of Alternative Business Structures in 2012.  Now several hundred entities have obtained ABS status.  

The Gazette article reports that accountants are poised to be large players in the ABA space:

Accountants will soon be competing directly with solicitor firms ‘on every high street in the country’, according to a leading financial advisor to the legal sector.

Ian Muirhead, chairman of Solicitors Independent Financial Advice, said he expects 750 accountancy firms – three times more than first envisaged – to move into probate work after securing an alternative business structure licence.

The Institute of Chartered Accountants in England & Wales has accredited 113 entities as an ABS since last October, having been accepted as an approved regulator almost a year ago. A further 34 applications are being processed.

Speaking at a Westminster Legal Policy, Muirhead said too many solicitor firms are ‘in denial’ about the threat from the accountancy profession.

‘Success will go to those who can manage businesses and I query whether that’s going to be the solicitors or whether solicitors are going to be the back room boys,’ he said.

Muirhead argued that law firms’ response so far has been focused on consolidation, mergers and acquisitions – but this risks playing into rivals’ hands.

‘[The response is] safety in numbers, more of the same, not thinking outside the legal silo, and therefore missing the opportunity of which many new ABSs are availing themselves, of providing a more diversified and holistic client service,’ he added. ... 

Some U.S. lawyers believe that liberalization won't come to the U.S. because the legal industry is too balkanized by state bar authorities.  

I think this view, however, is likely naive. The market can change because regulators change the rules (the UK). Alternatively, the market can change because clients change their buying habits in favor of nontraditional legal service providers that are financed by sophisticated nonlawyer investors (the US).  See, e.g., Is Axiom the Bellwether for Disruption in the Legal Industry, LWB, Nov. 10, 2013.

In the US, it is probably true that regulators lack the stomach to initiate a regulatory action where the client ostensibly being protected is a Fortune 500 corporation.  If the action ends up in federal court, the bar officials risk looking like protectors of the guild and have a decent chance of losing.  The prohibition against nonlawyer investment (MR 5.4) is based on the assumption that the nonlawyer profit motive will compromise lawyer independence, thus harming the unwitting and unsophisticated legal consumer.  But that does not describe IBM's or JP Morgan's relationships with sophisticated LPO or analytics shop (or any general counsel charged with stretching his or her legal dollar). As a result, the venture capital money flows in.

When liberalization is viewed in this light, there are probably more similarities between the US and UK than we might want to acknowledge. 

July 29, 2015 in Current events, Data on the profession, Innovations in law, Law Firms, New and Noteworthy, Structural change | Permalink | Comments (2)

Wednesday, July 22, 2015

What is more important for lawyers: where you go to law school or what you learned? (Part II)

If you're trying to maximize the financial value of an undergraduate degree, it is better to bet on course of study than college prestige.  Indeed, prestige is largely irrelevant to those who major in engineering, computer science, or math.  In contrast, prestige does matter for art & humanities grads, albeit the financial returns are significantly lower than their tech counterparts.  

These are some of the takeaways from Part I of this blog post. Part I also presented data showing that law is a mix of both: financial returns have been high (cf. "red" tech majors) and prestige matters (cf. "blue" arts & humanities crowd).  

The goal of Part II is to address the question of whether the pattern of high earnings/prestige sensitivity will change in the future. I think the answer to this question is yes, albeit most readers would agree that if law will change is a less interesting and important question than how it will change.  Speed of change is also relevant because, as humans, we want to know if the change is going to affect us or just the next generation of lawyers.

Shifts in the Legal Market

There are a lot of changes occurring in the legal market, and those changes are altering historical patterns of how legal services are being sold and delivered to clients. In the past, I have thrown around the term structural change, yet not with any clear definition.  To advance the conversation, I need to correct that lack of precision. 

In economics, there is a literature on structural change as applied to national or regional economies (e.g. moving from a developing nation to an industrial nation; or moving from an industrial to a knowledge-based economy).  Investors also focus on structural change within a specific industry because, obviously, large changes can affect investor returns.  When I have used the term structural change on this blog, it has been much closer to investor conceptions.  Investopedia offers a useful definition even if it's somewhat colloquial: 

Definition of 'structural change': An economic condition that occurs when an industry or market changes how it functions or operates. A structural change will shift the parameters of an entity, which can be represented by significant changes in time series data.

Under this definition, the legal industry is certainly undergoing structural change.  The proportion of law graduates getting a job in private practice has been on the decline for 30 years; over the last 35 years, the average age of the licensed lawyer has climbed from 39 to 49 despite record numbers of new law school graduates; the proportion of associates to partners has plummeted since the late 1980s.  See Is the Legal Profession Showing its Age? LWB, October 12, 2014.  Since the early 2000s, long before the great recession, associate-level hiring has been cut in half. See Sea Change in the Legal Market, NALP Bulletin, August 2013.

Likewise, among consumers of legal services, there is a lot of evidence to suggest that lower and middle class citizens can't afford a lawyer to solve life's most basic legal problems, thus leading to a glut of pro se litigants in state courts and many more who simply go without things like contracts and wills.  This troubling trend line was obscured by a boom in corporate legal practice, albeit now even rich corporations have become more sensitive to legal costs -- the sheer volume and complexity of legal need is outstripping their budgets.  In response to the lag in lawyer productivity and innovation, there is a ton of investor-backed enterprises that are now elbowing their way into the legal industry.  See A Counterpoint to "the most robust legal market that ever existed in this country"LWB, March 17, 2014.  

The impact of all this change -- structural or otherwise -- is now being felt by law schools. Applicants are down to levels not seen since the 1970s, yet we have dozens more law schools. It has been said by many that law schools are losing money, albeit we have zero data to quantify the problem.  Based on my knowledge of my own law school and several others I am close to, I am comfortable saying that we have real changes afoot that affect how the legal education market "functions or operates."

There is a sense among many lawyers and legal academics that the legal world changed after 2008. None of the "structural" changes I cite above are pegged in any way to the events of that year.  

What did change in 2008, however, was the national conversation on the legal industry, partially due to the news coverage of the mass law firm layoffs, partially due to important books by Richard Susskind and later Brian Tamanaha and Steve Harper, and partially due to a robust blogosphere.  This change in conversation emboldened corporate legal departments to aggressively use their new found market power, with "worthless" young associates getting hit the hardest.  This new conversation in turn exposed some of the risks of attending law school, which affected law school demand.  But alas, this was all fallout from deeper shifts in the market that were building for decades. Let's not blame the messengers.

Dimensions of Change

I am confident that the future of law is going to be a lot different than its past. But I want to make sure I break these changes into more discrete, digestible parts because (a) multiple stakeholders are affected, and (b) the drivers of change are coming from multiple directions.

Dimension 1: basic supply and demand for legal education

To unpack my point regarding multiple dimensions, let's start with legal education. Some of the challenges facing law schools today are entirely within the four corners of our own house.  Yet, legal education also has challenges (and opportunities) that arise from our connection to the broader legal industry.  This can be illustrated by looking at the relationship between the cost of legal education (which law schools control, although we may blame US News or the ABA) and entry level salaries (which are driven largely by the vagaries of a client-driven market).  

The chart below looks at these factors.  My proxy for cost is average student debt (public and private law schools) supplied by the ABA.  My income variables are median entry level salaries from NALP for law firm jobs and all entry level jobs.  2002 is the first year where I have all the requisite data.  But here is my twist:  I plot debt against entry-level salary based on percentage change since 2002.  


If a business nearly doubles its price during the same period when customer income is flat, demand is going to fall.  Thus, the sluggish entry-level market presents a difficult problem for legal education.  Sure, we can point to the favorable statistics from the AJD or the premium that a JD has historically conferred on lifetime earnings, but law professors are not the people who are signing the loan papers.  The chart above documents a changing risk/reward tradeoff.  To use the frame of Part I, the red dots are sinking into the blue dot territory, or at least that is the way prospective students are likely to view things.

Fortunately, smaller law school classes are going to be a partial corrective to low entry-level salaries.  The biggest law school class on record entered in the fall of 2010 (52,488); in 2014, the entering class had shrunk by over 27% (37,942). When entry-level supply is reduced by 25+%, upward pressure on salaries will build.  Yet, the composition of the legal economy and the nature of legal work is clearly changing.  Further, the rate of absorption of law school graduates into the licensed bar has been slowing for decades.  See Is the Legal Profession Showing its Age? LWB, October 12, 2014. It would be foolhardy to believe that time and fiscal austerity alone are going to solve our business problems. Instead, we need to better understand our role as suppliers to a labor market.

Dimension 2:  The content of legal education

The content of legal education is not necessarily fixed or static.  We could change the content, thus affecting how the market responds.  

To provide a simple example, one of my students is starting work this fall at Kirkland & Ellis.  From a financial perspective, this is a good employment outcome.  He will be moving to Chicago with his girlfriend who just received her MS in Information Systems from IU's Kelley School of Business.  The MS from Kelley is a very "red" degree.  It can also be completed in one year (30 credit hours).  Well before she graduated, this recent grad had competing offers from PWC and Deloitte, both in the $80,000 range.   For many Indiana Law students, an ideal post-grad outcome would be $80K in Chicago at an employer who provides challenging work and high-quality training.  Yet, my student's girlfriend got this ideal outcome in 1/3 the time and likely 1/2 the cost of an Indiana Law grad.  

Perhaps we should consider cross-pollinating these disciplines. A huge portion of the legal profession's economic challenges is attributable to flat lawyer productivity -- customers are struggling to pay for solutions to their legal needs.  Information systems are a huge part law's productivity puzzle.  Below is a chart I use in many of my presentations on the legal industry.  The chart summarizes the emerging legal ecosystem by plotting the Heinz-Laumann two-hemisphere model against Richard Susskind's bespoke-to-commodity continuum. [Click-on to enlarge.]


The key takeaway from this diagram is that the largest area of growth is going to be in the multidisciplinary green zone -- the legally trained working shoulder-to-shoulder with those skilled in information systems, statistics, software development, and computational linguistics, to name but a few.  These are "red" disciplines.  Do law schools want to be part of this movement?  Let me ask this another way -- do law schools want to be relevant to the bulk of the legal market that needs to be rationalized in order to maintain its affordability? Harvard grads will have options on Wall Street for the foreseeable future.  But 98% of law schools operate in a different market.  Further, some HLS grads, or students who might qualify for admission to Harvard, might prefer the big upside rewards that are only available in the green zone.  In short, a new hierarchy is emerging in law that is still very much up for grabs.

If an academic wants to better understand the rapidly changing nature of legal work, I would urge them to visit a large legal department with a substantial legal operations ("legal ops") staff.  These are the professionals who have been empowered by general counsel to find ways to drive up quality and drive down cost using data, process, and technology.  These are the folks who are making build-versus-buy decisions, putting pressure on law firms to innovate in order to hang on to legal work, and experimenting with NewLaw legal vendors. 

I am finishing up a story on legal ops professionals for the ABA Journal.  (By the way, legal ops exist in law firms as well as legal departments and green zone legal vendors. The role is most developed, however, in legal departments.)  My editor flagged the issue that virtually all of the legal ops people in the story did not graduate from prestigious law schools (or any law school).

My only response is that legal operations people have specialized skills and knowledge (often "red" but sometimes involving EQ) that others lack; without these skills, they can't do the job.  Legal ops people live in a world of outputs and metrics.  For example, are legal expenses and settlement amounts trending down over time -- yes or no? If so, by how much?  How much internal staff time does it take to negotiate a revenue contract? How much of this process can be automated? What will it take to get our staff to accept the new system?

As these examples show, a legal ops person is typically going to be evaluated based on measurable outputs -- do they get results? Where someone went to law school is an input that is likely irrelevant to the question.  The only qualifier is whether the curriculum of that school provided valuable, specialized domain knowledge -- most likely non-legal red skills but also skills related to teams, communication, and collaboration. 

Dimension 3:  The value of pedigree to the customer 

Law has historically been what economists call a “credence good.”  This means that a layperson has a difficult time assessing quality.  As a result, proxies for quality, such as pedigree or prestige, have historically been very important when hiring a lawyer or law firm.  

One of the reasons that the field of legal operations is gaining momentum is because it is creating tools and systems that enable clients to look past credentials to obtain information on things they really care about, such as cost, outcome, and speed of delivery. There are now companies coming into existence that are gathering data on lawyers' win-loss rates. See Another Example of Using Big Data to Improve Odds of Winning in Court, LWB, April 12, 2015.  Sure, apples-to-apples comparisons are very difficult to make -- every case is unique in some respect. But the amount of money at stake is large enough that the data challenges will be surmounted.  When that day arrives, we won't opine on the value of pedigree to legal outcomes; we'll just calculate it. More significantly, clients focused on outcomes will change their buying patterns.  Early returns I have seen suggest that the value of pedigree to legal outcomes may be close to negligible.

Do any of us care where the engineers who designed our smart phones went to college? Not really. We just care how well the smart phone works. 

In this respect, the future of law is likely headed in the direction of Google (a pure red company).  In the early days, the founders of Google favored grads of Caltech, Stanford and Berkeley.  But over time, the company learned that prestige of graduate school was a poor predictor of job success. Because Google lives and dies by its outputs, the company changed its hiring model to attract the most qualified engineers.  See George Anders, The Rare Find: How Great Talent Stand Out 1-5 (2012) (telling the story of how data changed the attitudes of Google founders regarding elite credentials and altered the Google hiring model).

I have lived long enough to know that the changes I describe above are not necessarily going to be welcomed by many lawyers and law professors.  If a group benefits from a lifelong presumption of merit, it is natural that group will resist evidence that the presumption is not fully warranted. Indeed, much of the skepticism will be rooted in subconscious emotion.  If the presumption is dashed, those of us in the elite crowd will have to spend our days competing with others and proving ourselves, or even worse, watching our kids soldier through it.  We have little to gain and a lot to lose in the world we are heading into.  Yet, behind the Rawls veil of ignorance, how can we complain?

So with the red-blue crosscurrents, is law school still worth the investment?

That is a relevant and reasonable question that many young people are contemplating.  I will offer my opinion, but markets are bound to follow their own logic. 

This is a time of enormous uncertainty for young people. Education clearly opens doors, but tuition is going up much faster than earnings.  Further, competition among knowledge workers is becoming more global, which is a check on wages.  Of course, if you don't invest in education, what are your options?

I am generally on the side of Michael Simkovic and Frank McIntrye that the education provided by a law degree, on average, significantly increases lifetime earnings.  See The Economic Value of a Law Degree (April 2013).  How could it not?  The law is too interconnected to every facet of society to not, on average, enhance the law grad's critical thinking skills. Nearly 15 years of out of law school and I regularly use what I learned at Chicago Law to solve problems and communicate solutions, particularly in my applied research work with law firms and legal departments. While my Chicago Law credential has value independent of the skills and knowledge I obtained (the red AJD bar chart in Part I strongly suggests that), I can't deny the additional value of the actual skills and knowledge I obtained to solve real world business problems. It's been substantial.

In general, I also agree with Deborah Jones Merritt that there is significant evidence that the entry-level market for lawyers is weak and oversaturated.  See What Happened to the Class of 2010? Empirical Evidence of Structural Change in the Legal Profession (April 2015).   The class of 2010 is not faring as well as the class of 2000.  Indeed, the lead economist for Payscale, Katie Bardaro, recently noted that wages are stagnating in many fields, but especially in the legal profession. "More law schools are graduating people than there are jobs for them...There’s an over-saturated labor market right now. That works to drive down the pay rate.” See Susan Adams, The Law Schools Whose Grads Earn the Biggest Paychecks in 2014, Forbes, Mar. 14, 2014. 

In the face of these stiff headwinds, I think law schools have an opportunity to pack more value into three years of education. See Dimension 2 above.  To be more specific, if you are a protege of Dan Katz at Chicago-Kent, you will have a lot of career options. Ron Staudt, also at Chicago-Kent, has quietly built a pipeline into the  law and technology space.  Oliver Goodenough and his colleague at Vermont Law are making rapid progress with a tech law curriculum.  And at Georgetown Law, Tanina Rostain and Ed Walters (CEO of Fastcase) provide courses that are cutting edge.  

But absent these types of future-oriented instruction, what is the value of a JD degree as it is commonly taught today? That value is clearly positive; I would even call it high.  But whether the value is sufficient to cover the cost of attendance is likely to vary from law grad to law grad.  Lord knows, in a world of variable tuition based on merit scholarships and merit scholarships that go away after the 1L year, the swing in cost can be a $250K plus interest.

What is killing law school applications these days is the lack of near certainty among prospective students that the time and expense of law school will pay off.  The world looks different than it did in the fall of 1997 when the vast majority of the AJD respondents entered law school. Tuition and debt loads are higher and high paying entry-level jobs are harder to obtain.

So what is the solution?  For students, it's to bargain shop for law schools, which is bad news for law schools.  For law schools, it's to add more value to an already valuable degree.  Some of that value will come in the form of red technical skills that will make lawyers more productive.  In turn, this will prime demand for more legal products and services.

July 22, 2015 in Blog posts worth reading, Data on legal education, Data on the profession, Legal Departments, Structural change | Permalink | Comments (0)

Sunday, July 19, 2015

What is more important for lawyers: where you go to law school or what you learned? (Part I)

The Economist reports a very interesting analysis from Payscale.  The questions being asked are pretty simple: If you want to generate earnings that justify the time and cost of an undergraduate education, what should you study and where should you enroll?

Lots of people have strong opinions on this set of questions, but Payscale has the data to answer them empirically. It turns out that at the undergraduates level, course of study is much more important than the prestige of the college or university you attend.  The hard evidence is shown below.


For those working in law or thinking about attending law school, a natural question to ask is whether the legal industry is closer to the blue dot (art & humanities) or red dot pattern (engineering/CS/math).  A second, related question whether the future of law is more blue or more red.

This a two-part blog post.  Part I tries to answer the first question, starting with a careful analysis of the undergraduate chart, which provides a valuable frame of reference that can be discussed more dispassionately (at least among lawyers and law students) than an analysis that questions the value of law school prestige and hierarchy.  

Part II, which I will post on Wednesday, explores the second, future-oriented question.  I will tip my hand now and say that the future of law will be less blue (arts & humanity) and more red (math/CS/engineering).  Within the legal industry, there will be winners and losers; but from the perspective of broader society, this change is a very good thing. 

Undergraduate ROI

In the Payscale chart above, the y-axis (vertical) is 20-year annualized returns from college fees paid.  The x-axis is selectivity, running from under 10 percent to near open admissions.  

The Payscale chart is a very good example of how data visualization can be used to communicate both core facts and useful nuance.  Here, the lede is unmistakable:  the red dots (engineering/CS/math) are overwhelming higher on the ROI scale than the blue dots (arts & humanities).  Sure, there are exceptions to this rule, but they don't occur very often. (Observe how rarely a blue dot is above the red fit-line.) This suggests it would be very foolish to get a blue degree and expect a red paycheck unless you have very good information (or skills or talent) that others lack.

The chart conveys another important piece of information -- the red fit-line is flat.  This means that for engineering/CS/math majors, prestige has not been very relevant to their eventual earnings.  I'll add a nuance here that some empirically savvy readers are bound to point out:  It is possible (indeed likely) that fees are higher at more selective schools. So if MIT costs twice as much as a public polytech, and both yield 12% over 20 years, one might wish they had gone to MIT.   Still, the flat trendline is surprising.  As a general matter, lower ranked schools are not dramatically cheaper than higher ranked schools, and many public schools are highly selective.  The flat red trendline suggests that there are (or were, remember these are historical data) many bargains out there.  If one is trying to maximize financial returns, the goal is to find a school that will, in the future, be well above the red fit-line (and avoid those below).  

The flat red fit-line is also surprising because college selectivity is almost certainly highly correlated with ACT or SAT scores, which our society often views as measures of general intelligence. Yet, there we have it -- a flat trendline. Four years of education seem to be more relevant than a standardized test score taken during high school.  That is heartening at many levels.

A third interesting trend -- the blue fit-line is sloped downward.  This suggests that in the arts & humanities, selectivity/prestige does have a financial payoff.  I don't think this will surprise many readers, albeit the prestige payoff is not very large. To use a simple metaphor, if you attend a more selective college or university to get your arts or humanity degree, you are likely to have a better house in the arts & humanities neighborhood.  But on average, you won't be able to afford the same neighborhood as the engineers, computer scientists, and math majors.

What about Law?

Moving on to law, if we want to examine the relationship between earnings and law school attended, the best available evidence is probably the After the JD Study (AJD), which is large, representative sample of law graduates who took and passed the bar in 2000.  

Data from AJD Wave 3 suggests that the financial returns are relatively strong for all law school graduates -- ten years out and graduates of Tier 4 schools have median earnings of $100,000 per year. As shown in chart below, this is akin to shifting the blue dots up into the red territory.  


The downward sloping fit-line remains, but that doesn't seem to matter very much to happiness. Other AJD data shows that regardless of tier of graduating school, AJD respondents show relatively high and uniform satisfaction with (a) the decision to become a lawyer, and (b) the value of the law degree as an investment. By 2010, 48% of respondents had no debt; only 5.1% had more than $100K in educational debt remaining. 

This is all good news.  But is it reasonable to extrapolate forward and assume the past is a fairly accurate barometer of the present and the future? 

One way to address that question is to ascertain what has changed since 2000.  As noted earlier, the AJD sample was composed of law graduates who passed the bar in the year 2000. Figures published by NALP and the ABA show that the percentage of full-time bar passage required jobs has dropped significantly over the last 13+ years -- from 77.3% for the class of 2000 to 57% for the class of 2013. That is a huge delta.


One of the reasons why law school applicants have plummeted is that the career path from JD graduates has become murky.  And that is a good place to start Part II

July 19, 2015 in Blog posts worth reading, Cross industry comparisons, Data on legal education, Data on the profession, Structural change | Permalink | Comments (3)

Thursday, May 14, 2015

Further Thoughts on the July 2014 Bar Results -- A Response to Erica Moeser

Late last fall, Erica Moeser responded to a letter from Dean Kathryn Rand of the University of North Dakota (on behalf of a large number of law school deans), reiterating that the NCBE had double-checked its scoring of the MBE on the July 2014 bar examination and could find no errors in its calculations.  Erica Moeser also took to the pages of the December 2014 issue of The Bar Examiner to further validate her conclusion that the historic drop in the mean MBE scaled score is attributable solely to the fact that the class that sat for the July 2014 bar exam was “less able” than the class that sat for the July 2013 bar exam.  In January, Dean Stephen Ferruolo of the University of San Diego also wrote to Erica Moeser requesting the release of more information on which to assess the July 2014 bar examination results in comparison with previous years’ results.  In February, Erica Moeser responded to Dean Ferruolo’s request by declining to provide more detailed information and reiterating her belief that the July 2014 scores “represent the first phase of results reflecting the dramatic and continuing downturn in law school applications.”

In an earlier blog posting, I explained why Erica Moeser is partly right (that the Class of 2014 could be understood to be slightly less able than the Class of 2013), but also explained why the decline in “quality” of the Class of 2014 does not explain the historic drop in mean MBE scaled score.  The decline in “quality” between the Class of 2013 and the Class of 2014 was modest, not historic, and would suggest that the decline in the mean MBE scaled score also should have been modest, rather than historic.  Similar declines in “quality” in the 2000s resulted in only modest declines in the MBE, suggesting that more was going on with the July 2014 exam. 

Others have written about these issues as well.  In January, Vikram Amar had a thoughtful reflection on Moeser’s statements and in recent weeks Debby Merritt has written a series of posts -- here, here, and here -- indicating in some detail why she believes, as I do, that the ExamSoft debacle in July could have impacted the MBE scaled scores in jurisdictions that used ExamSoft as well as in other jurisdictions.

I write now to take issue with four statements from Erica Moeser – three from her President’s Page in the December 2014 issue of the Bar Examiner and one from her letter responding to Dean Kathryn Rand.  I remain unpersuaded that the historic decline in the mean MBE scaled score is solely attributable to a decline in quality of the class that sat for the July 2014 bar examination and remain baffled that the NCBE refuses to acknowledge the possibility that issues with test administration may have exacerbated the decline in the performance on the July 2014 MBE.

Item One – Differential Declines in MBE Scores

In her December article, Moeser stated: 

I then looked to two areas for further corrobo­ration. The first was internal to NCBE. Among the things I learned was that whereas the scores of those we know to be retaking the MBE dropped by 1.7 points, the score drop for those we believe to be first-time takers dropped by 2.7 points. (19% of July 2014 test takers were repeaters, and 65% were believed to be first-time takers. The remaining 16% could not be tracked because they tested in jurisdictions that col­lect inadequate data on the MBE answer sheets.) The decline for retakers was not atypical; however, the decline for first-time takers was without precedent dur­ing the previous 10 years. (Emphasis in original.)

Moeser starts by referencing data that is not publicly available to support her cause.  This is unfortunate, because it makes it really hard to understand and critique the data.  Nevertheless, there are some inferences we can take from what she does disclose and some questions we can ask.  Moeser asserts that the 19% of MBE “retakers” saw an MBE drop of 1.7 points compared with MBE “retakers” in July 2013, while the 65% believed to be first-time takers saw a drop of 2.7 points compared with first-time takers in July 2013.  It would have been helpful here if Erica Moeser would have released publicly the declines among MBE retakers in the previous 10 years and the declines among first-time takers in the previous 10 years so that patterns could be assessed, particularly in relation to the changes in class composition for each of those years.  Without that information available it is hard to do much more with Moeser’s assertion.  (I find it odd that she would reference this point without providing the underlying data.) 

Nonetheless, this assertion raises other questions.  First, the overall decline in the mean MBE scaled score was 2.8 points. Moeser notes that 19% of takers (MBE retakers) had an average drop of 1.7 points, while 65% of takers (first-time takers) had an average drop of 2.7 points.  Unless there is something I am missing here, that should mean the remaining 16% of test-takers had to have an average decline of 4.51 points!  (This 16% of test-takers represents those who Moeser notes could not be tracked as first-time takers or MBE retakers “because they tested in jurisdictions that collect inadequate data on the MBE answer sheets.”) (Here is the equation --- 2.8 = (.19*1.7)+(.65*2.7)+(.16*x).  Solve for X. This translates to 2.8 = .323+1.755+.16x.  This translates to .722 = .16x and then .722/.16 = X.  X then equals 4.51.)  It would have helped, again, if Moeser had indicated which jurisdictions had these even larger declines in mean MBE scaled scores, as we could then look at the composition of graduates taking the bar in those jurisdictions to see if there was an unusual decline in entering class statistics in 2011 at the law schools from which most bar takers in those states graduated.

Item Two – The MPRE

In the December article, Moeser also stated:

I also looked at what the results from the Multistate Professional Responsibility Examination (MPRE), separately administered three times each year, might tell me. The decline in MPRE performance supports what we saw in the July 2014 MBE numbers. In 2012, 66,499 candidates generated a mean score of 97.57 (on a 50–150 scale). In 2013, 62,674 candidates generated a mean score of 95.65. In 2014, a total of 60,546 candi­dates generated a mean score of 93.57. Because many MPRE test takers are still enrolled in law school when they test, these scores can be seen as presaging MBE performance in 2014 and 2015.

At first blush, this looks like a pretty compelling argument, but Moeser’s selectiveness in looking at the data is troubling, and her failure to discuss whether the MPRE and MBE are meaningfully comparable test-taking experiences also is troubling.  Essentially, Moeser is making the following assertion – because the mean MPRE scaled score declined by 1.92 points between 2012 and 2013, we should have expected a large decline in the mean MBE scaled score in July 2014 (and because the mean MPRE scaled score declined another 2.08 points between 2013 and 2014, we should expect another large decline in the mean MBE scaled score in July 2015).

But the “relationship” between changes in the mean MPRE scaled score and changes in the mean MBE scaled score over the last decade does not support this assertion. If one looks at a decade’s worth of data, rather than data just for the last couple of years, the picture looks significantly more complicated, and suggests the collective performance on the MPRE may not tell us much at all about likely collective performance on the MBE in the following year. 


Mean MPRE Score


MBE Year

July Mean MBE Scaled Score




































































The data Moeser cites from the last two years conveniently makes her point, but it consists of a very small sample size.  The data over the last decade looks much more random.  In three of the nine years, the change is not in the same direction (MPRE 2005, 2006, 2010, MBE 2006, 2007, 2011).  In the six years where the change is in the same direction, there are two years in which the MBE change is significantly larger than the MPRE change (MPRE 2007, 2009, MBE 2008, 2010) and there are two years in which the MBE change is significantly smaller than the MBE change (MPRE 2011, 2012, MBE 2012, 2013).  In only two of the nine years, do the changes in the MPRE and MBE roughly approximate each other (MPRE 2008, 2013, MBE 2009, 2014).   Nonetheless, this remains a very small sample and more analysis of data over a longer period might be helpful to better understand how/whether changes in mean MPRE scores inform meaningfully changes in mean MBE scores the following year.  At this point, I think the predictive value seems marginal given the wide range of changes on a year-over-year basis.

Item Three – Mean LSAT Scores

In the December article, Moeser further stated:

Specifically, I looked at what happened to the overall mean LSAT score as reported by the Law School Admission Council for the first-year matricu­lants between 2010 (the class of 2013) and 2011 (the class of 2014). The reported mean dropped a modest amount for those completing the first year (from 157.7 to 157.4). What is unknown is the extent to which the effect of a change to reporting LSAT scores (from the average of all scores to the highest score earned) has offset what would otherwise have been a greater drop. (LSAC Research Reports indicate that roughly 30% of LSAT takers are repeaters and that this num­ber has increased in recent years.

This assertion is misguided for purposes of this comparison, a point Vikram Amar made in his post.  If we were comparing the first-year matriculants in 2009 with the first-year matriculants in 2010, the question of the change in reporting from average LSAT score to highest LSAT score would have mattered.  But the 2010 matriculants were the first class for which the mean was reported based on highest LSAT score and the 2011 matriculants were the second class for which the mean was reported based on highest LSAT score.  Thus, there is no “unknown” here.  The reported mean LSAT dropped only a modest amount between the matriculants in 2010 and the matriculants in 2011.  Nonetheless, the mean MBE scaled score in July 2014 decreased by an historic 2.8 points from the mean MBE scaled score in July 2013. 

Item Four – Administration Issues

In her letter to Dean Kathryn Rand, Moeser stated:  "To the extent that the statement you attached referenced both administration and scoring of the July 2014, bar examination, note that NCBE does not administer the exam; jurisdictions do."

This response suggests not only that the NCBE is not responsible for administering the bar examinations in the many different jurisdictions, but implicitly suggests that issues with administration could not have contributed to the historic decline in the mean MBE scaled score. 

Were there issues with administration?  Yes.   Could they have contributed to the historic decline in the mean MBE scaled score?  Yes.

Debby Merritt’s recent posts discuss the administration issues and the potential consequences of the administration issues in some detail.  In over forty states that used ExamSoft to administer the bar examination, the MBE came on Wednesday, after the essay portion of the exam on Tuesday.  But because of an ExamSoft technical problem, tens of thousands of test-takers, who were initially informed by their respective state board of bar examiners that they would FAIL THE EXAM if their essay answers were not uploaded in a timely manner, spent most of Tuesday night dealing with the profound stress of not being able to upload their exam answers and not being able to contact anyone at the board of bar examiners (who were not answering phones) or at ExamSoft (due to the flood of calls and emails from anxious, frustrated, stressed out exam takers) to figure out what was going on and what they should do. 

Given that this “administration” issue caused untold stress and anxiety for thousands of test-takers, who spent Tuesday night completely anxious and stressed out trying repeatedly and unsuccessfully to upload their essay answers, should it be a surprise that they might have underperformed somewhat on the MBE on Wednesday?  (If you want a sense of the stress and anxiety, check the twitter feed for the evening of Tuesday, July 29, 2014)

The responses from the boards of bar examiners to this issue with administration of the bar examination were far from uniform.  Different jurisdictions granted extensions at different times of the night on Tuesday, July 29, or on Wednesday, July 30, with some granting short extensions and some granting longer extensions.  Thus, in states that gave notice of an extension out earlier on Tuesday, July 29, test-takers may have had less stress and anxiety, while in those states that didn’t give notice of an extension out until later (or for which the extension was relatively short), or where there may not have been any communication regarding extensions of the submission deadline, test takers likely experienced more stress and anxiety.  (It would be worth studying exactly when each jurisdiction gave notice of an extension and whether there is any correlation between timing of notice of the extension and the relative performance of bar takers in those states.)

The NCBE’s unwillingness to acknowledge any issues with administration of the bar examination is all the more surprising at a time when the NCBE is pushing for adoption of the Uniform Bar Examination.  On its webpage, the NCBE states: “[The UBE] is uniformly administered, graded, and scored by user jurisdictions and results in a portable score that can be transferred to other UBE jurisdictions.” (Emphasis added.)  This simply was not true in July 2014.  The Uniform Bar Examination was administered under different exam conditions across jurisdictions.  First, three of the states administering the Uniform Bar Examination in July 2014 did not use ExamSoft – Arizona, Nebraska and Wyoming -- and therefore, bar takers in those states had a vastly different “exam administration” experience than bar takers in ExamSoft jurisdictions.  Across ExamSoft jurisdictions, different approaches to extensions also meant different administration experiences. Given the significance of consistent administration for the purpose of equating performance on a standardized exam like the bar exam, that the NCBE allows such varied approaches to administering a supposedly “uniform” exam strikes me as very problematic.

Many questions remain unanswered, largely because adequate information has not been made available on which to assess the various factors that might have contributed to the historic decline in the mean MBE scaled score.  With the release of February bar results and the NCBE’s publication of the 2014 statistical report, some additional information is now available to put the results of July 2014 in context.  In my next blog posting regarding the July 2014 bar results, I will delve into some of those statistics to see what they tell us.

(Edited as of May 20 to correct the 2013 MPRE and 2014 MBE change and corresponding discussion.)

May 14, 2015 in Current events, Data on legal education, Data on the profession | Permalink | Comments (0)

Wednesday, May 13, 2015

Andy Perlman Appointed Dean of Suffolk Law

A friend and colleague!  Here's the press release.  

May 13, 2015 | Permalink | Comments (0)

Tuesday, May 12, 2015

Reply to Reply to Lipshaw on Private Law

10315tJohn Goldberg (left) and Henry Smith (right) at the new and interesting New Private Law blog flattered me with a serious and thoughtful response to my earlier post. (This is a version of a comment I posted over there.)

The gist of the response is that I was overstating things dramatically.  That is, there is a meaningful distinction between public and private law when it comes to contract law. 10822t

To be clear, I was not suggesting that inclusion of contract law in the “private” category was meaningless. Indeed, I said it was likely a helpful heuristic for the stuff that interests you all. And the distinction between public and private is almost certainly well understood conventionally at the extremes – e.g., law created wholly between two private parties (by way of contract) is private, and law created between two states (a treaty) is public. The pushback from Matt Bodie, Steve Bainbridge and others was with respect to the close cases.

I agree my headline was (over)dramatic. It had the intended effect! But I was making two moderately serious points that were not nearly as dramatic as I think John and Henry characterized.

First, the definition game IS a lawyer’s game. I suppose if there were a statute passed that said “All law professors concentrating on public law shall be paid salary two times that of one concentrating on private law, ceteris paribus,” I’d be in the middle of the fray arguing about what is in and out of the basket. But I’m not sure what the distinction actually cashes out. My lingering concern is that the definition game in academia pretty easily turns into the discipline game (i.e. what’s in a discipline and what is not), and that’s a subject on which I’ve written extensively – and with a normative spin: lawyers (particularly business lawyers) need to be careful about getting locked into disciplinary tunnel vision.

Second, though we can discuss the impact of writing down an agreement on one’s moral compunction to abide by its terms, I have argued (see the Canadian Journal of Law & Jurisprudence essay on objectivity and subjectivity in contract law) that indeed our voluntary interpersonal (I-You) commitments are essentially moral, and that we objectify them precisely for the purpose of making them publicly resolvable. It tends to be an article of faith that contracts actually inhibit opportunism when it comes down to cases (as opposed to the high theory of the New Institutional Economics), and hence constitute “law” between the parties. I disagree, but spelling all that out would take more than blog posts and comments.

Having said all that, I wish the New Private Law blog success and long life! It’s now on my RSS feed.

May 12, 2015 | Permalink | Comments (0)

Thursday, May 7, 2015

Revisiting Conditional Scholarships

Having been one of the people who brought attention to the issue of conditional scholarships a few years ago, I feel compelled to offer a few insights on a rekindled conversation about conditional scholarships involving Jeremy Telman and Michael Simkovic and Debby Merritt.

I am not sure what prompted Prof. Telman to write about conditional scholarships, but the first sentence of his initial post seems to be a few years late: 

One of the ways in which law schools are allegedly inadequately transparent is in the award of merit scholarships conditional on the students’ achievement of a certain grade point average. (Emphasis added)

A few years ago, one accurately could have said that law schools were inadequately transparent regarding the awarding and retention of conditional scholarships.  I did say that in an article Prof. Telman describes as “interesting.” 

Today, this is no longer accurate, because we have much greater transparency regarding conditional scholarships given the disclosures mandated pursuant to Standard 509.

Thus, I am not sure anyone is alleging that law schools are inadequately transparent regarding conditional scholarships and I am not sure why this is once again an item for discussion.  It has been well settled and law schools and prospective law students have adjusted to a new reality.  Indeed, in his follow up posting, Prof. Telman essentially acknowledges this point:

It seems we are all agreed that the disclosure problems related to conditional scholarships have largely been addressed through the ABA website that enables students to comparison shop among scholarship offers from various schools and know their chances of retaining their conditional scholarships.

That said, given that Prof. Telman got the conversation started, I have a response to one of his assertions and some observations to share.

The general context of his posting (and Prof. Simkovic’s related posts) is that college students have lived with conditional scholarships without apparent problems so conditional scholarships shouldn’t present a concern for law students.  In making his case, Prof. Telman relies on my 2011 article to support a proposition that the article actually disproves in some detail.  Specifically, Prof. Telman states:

Professor Organ was able to find information about how scholarships work at 160 law schools.  That means that the information was out there.  Since Professor Organ was able to gather information about 160 law schools, it should not be difficult for students to gather relevant information about the one law school that they are considering attending. 

He further states:  “Why are law students assumed to be incapable of looking into standard grade normalizations curves for the first year?”  Prof. Telman seems to be suggesting that there actually weren’t any disclosure problems because “the information was out there.”  The information was not out there. 

To be more precise, in putting together the article, with the efforts of research assistants as well as my own sleuthing, I was able to find sufficient information from the NAPLA-SAPLA Book of Lists, the ABA-LSAC Guide, and law school web pages from which to classify 160 law schools regarding whether the law school had a competitive scholarship program or some other type of scholarship program.  If Prof. Telman would have looked carefully at the article, however, he would have noted that “only four of these 160 schools had any information posted on their webpages indicating renewal rates on scholarships.”  (A point Derek Tokarz makes in the comments to Prof. Telman’s post.)

Prospective law students not only need relevant information about one law school, they need relevant and comparable information about the set of three or five or seven law schools they are considering seriously.  Prior to the Standard 509 mandated disclosure of conditional scholarship information, it was profoundly difficult if not impossible for students to gather relevant information from a few or several law schools.  The information simply was not “out there.”

Indeed, two of the primary points of my article were to highlight the information asymmetry between law schools and prospective law students relating to competitive scholarships and to recommend greater disclosure of the number of students receiving competitive scholarships and the number who had them renewed (or had them reduced or eliminated).   

Prof. Merritt discusses in some depth this information asymmetry, noting particularly that college students who have been successful in retaining their conditional scholarships as undergrads do not appreciate the reality of the mandatory curve they will encounter in law school, a point Stephen Lubet also makes cogently in a comment to Prof. Telman’s post. (Indeed, to his credit, Prof. Telman acknowledges that prospective law students also may suffer from optimism bias in assessing their likelihood of retaining their scholarship.)

Regarding the need for greater disclosure, regardless of how savvy and sophisticated we would like to believe prospective law students might have been or might be, the nuances of conditional scholarships and mandatory curves were not things that were clearly understood in the era prior to the mandatory Standard 509 disclosure.  I noted in my article that many students posting on Law School Numbers valued their scholarships based on a three-year total, regardless of whether they were conditional scholarships, suggesting these students failed to appreciate that the “value” should be discounted by the risk of non-renewal.  I also spoke with pre-law advisors around the country regarding conditional scholarship and consistently was told that this information was very helpful because pre-law students (and sometimes pre-law advisors) had not appreciated the realities of conditional scholarships.

While there are other things mentioned by Prof. Telman, Prof. Simkovic and Prof. Merritt to which I could respond, this post is already long enough and I am not interested in a prolonged exchange, particularly given that many of the points to which I would respond would require a much more detailed discussion and more nuance than blog postings sometimes facilitate.  My 2011 article describes my views on competitive scholarship programs and their impact on law school culture well enough.  Accordingly, let me end with one additional set of observations about what has happened with conditional scholarships in an era of increased transparency.

In my follow up article available on SSRN, I analyzed the frequency of conditional scholarships generally and the extent to which conditional scholarships were utilized by law schools in different rankings tiers for the 2011-2012 academic year (the first year following the ABA's mandated disclosure of conditional scholarship retention rates). 

For the entering class in the fall of 2011, I noted that there were 140 law schools with conditional scholarship programs, and 54 law schools with scholarship renewal based only on good academic standing, one-year scholarships, or only need-based scholarship assistance.  I also noted that conditional scholarship programs were much less common among top-50 law schools than among bottom-100 law schools. 

Based on the data reported in fall of 2014 compiled by the ABA for the entering class in the fall of 2013 (the 2013-2014 academic year), the percentage of all entering first-year students with conditional scholarships has increased slightly (from 26.1% in fall 2011 to 29% in fall 2013), while the percentage of all entering first-year students who had their scholarships reduced or eliminated has decreased slightly (from 9% as of summer of 2012 to 8.4% as of summer of 2014).  The average renewal rate across law schools increased from 68.5% to 73%.

More significantly, however, the number of law schools with conditional scholarship programs has declined, while the number with other types of scholarship programs has increased fairly significantly.  By 2013-2014, there were 78 law schools with scholarships renewed based on good academic standing, with one-year scholarships or with only need-based scholarship assistance, a significant growth in just two years in the number of law schools going away from conditional scholarship programs -- 24 more law schools, a 40% increase.  This would seem to indicate that at least some law schools have decided conditional scholarships aren’t as good for law schools or for law students. 

May 7, 2015 in Data on legal education, Scholarship on legal education | Permalink | Comments (1)

"Private Law" is an Oxymoron

John Goldberg and Henry Smith at Harvard have geared up a new blog dedicated to private law, in the process generating questions from Matt Bodie and Steve Bainbridge, among others, about the definition of private law.

UnknownI need to be careful here because I am close to invoking the W word (Austin Sarat told me once it was a Woody Allen like moment when I just happened to pull "Philosophical Investigations" out of my briefcase at a conference, so I'm going simply to post a picture.)

First, I'm a definition skeptic. Creating definitional baskets and stuffing things in them is a professional and parlor game for lawyers (e.g., is something a sport or a game?).  The question is whether the language in which definitions are cast sufficiently cuts reality at the joints to be helpful in any situation where we really don't have to decide if it's in or out.  The law, of course, is full of in-or-out situations.  Is it a security or isn't it?  Is she a resident of Massachusetts or isn't she? But is it helpful here other than a heuristic (and probably a useful one) for the stuff that interests the bloggers?  What counts as private law more likely bears significant family resemblances to public law than is something capable of crisp distinction.

Second, and more significantly, I think "private law" is an oxymoron when you actually parse it, as opposed to using it as a quick-and-dirty category. The guy I pictured above is famous for, among other things, observing that there are no private languages.  What that means is that for us to communicate at all, we are incorporating an objective world into our own subjective thoughts.  In the area of "private law" I think about the most, it's a lawyer's conceit to believe that the contract IS the deal, except in the simplest of circumstances.  As I've written in the past, the essence of turning an inter-personal relationship into a contract is to objectify it; once you've done that, and turned either to a community standard or a third party to interpret your agreement, it isn't private anymore.

May 7, 2015 | Permalink | Comments (0)

Thursday, April 30, 2015

A Short Screed About the Obligatory "I Can Raise Money" Portion of the Academic Leadership Candidate's Pitch

The screed is about the naive view of development you get from time to time during the search for leaders of educational institutions:  "He'd be great with alums."  "She can really work a room." "He's really connected to the movers and shakers." 

Clipart-money-royalty-free-vector-clip-art-of-a-happy-retro-man-jumping-up-and-down-over-a-large-sack-of-coins-with-a-money-symbol-black-and-white-by-bestvector-1934Sorry. Sophisticated development is not about the ability to be charming or to work a room or to have access to wealthy people, although all those are nice, ceteris paribus.  Perhaps you can get lucky and find the donor who's just itching to give a $25 million naming gift to the Southern Euphoria University College of Law, but that's not generally how it works.  It's a long game, and more than anything you can't raise money for a sow's ear without at the same time demonstrating that, indeed, you are turning it into a silk purse.  The two are inextricably linked.

Jack Welch said famously that GE's businesses were going to be number one or number two in their field, or GE would get out.  The rationale for that isn't some Neutron Jack inhumanity.  It's that he understood business is fund-raising.  It's either raising funds in the form of sales revenue because you have goods and services you are proud to sell, or it's raising funds in the form of capital from investors because you have businesses you are proud to say have a future.  Being number one or number two is a quick and dirty heuristic - and a quotable line - for being a silk purse and not a sow's ear.  (I'll bet GE kept some very good businesses that were number three!)

There's a similar dichotomy between sales and marketing.  Sales is knocking on the door; marketing is the sophisticated process of having something to sell and persuading people they want it.  Marketing is about a value proposition: put your money here rather than somewhere else because we will deliver to you more utils than the competition.  Compelling value propositions take work and require huge amounts of effort within the four walls of the institution before you put them out to customers and investors.  The same is true of schools.

Beware the naive view of development.  Beware the pitch that is just about willingness to knock on doors without linking it back to the hard work of building a distinct institution. Beware the person who thinks there's a meaningful distinction between being an "inside dean" and an "outside dean."

End of screed.

April 30, 2015 | Permalink | Comments (0)

Monday, April 13, 2015


          This blog posting is designed to do three things.  First, following up on recent discussions regarding trends in applicants by Al Brophy at The Faculty Lounge and Derek Muller at Excess of Democracy, I provide a detailed analysis to project the likely total applicant pool we can expect at the end of the cycle based on trends from March through the end of the cycle in 2013 and 2014.  Second, using the likely total pool of applicants, I estimate the number of admitted students and matriculants, but also question whether the estimates might be too high given the decline in quality of the applicant pool in this cycle.  Third, building on the second point, I suggest that law schools in the lower half of the top tier are likely to see unusual enrollment/profile pressure that may then have a ripple effect down through the rankings.


Reviewing the 2013 and 2014 Cycles to Inform the 2015 Cycle

2013   Current Volume Summary Date


%   of Cycle

Projected   Total Applicant Pool

Jan.   25, 2013




Mar.   8, 2013




May   17, 2013




End   of Cycle





2014   Current Volume Summary Date


%   of Cycle

Projected   Total Applicant Pool

Jan.   31, 2014




Mar.   7, 2014




April   25, 2014




End   of Cycle





2015   Current Volume Summary Date


%   of Cycle

Projected   Total Applicant Pool

Jan.   30, 2015




Mar.   6, 2015




April   3, 2015




End   of Cycle



54,000   (Estimate)

        In each of the last two years, a modest surge in late applicants meant the final count exceeded the March/April projections by a couple thousand.  That would suggest that the current projection (for just under 53,000) likely understates the end of cycle applicant pool, which I am now estimating conservatively at 54,000 (down about 3% from 2014).  (In 2014, the amount by which the final pool total exceeded the early March projection was nearly 2,500.  With an estimated pool of 54,000 applicants, I am estimating that the final pool in 2015 will exceed the early March projection by roughly 2,000.)  (That said, if the employment results for 2014 graduates, which will be released shortly, show modest improvement over 2013, I anticipate that even more people might come off the fence and perhaps apply late for the fall 2015 class.)


        The chart below shows the number of applicants, admitted students and matriculants over the last three years along with an estimate for fall 2015 based on the assumption above that we have a total of 54,000 applicants this cycle.  With 1,700 fewer applicants, I am assuming 1,000 fewer admitted students (a slight increase in the percentage admitted from 2014), and then assuming the number of matriculants will reflect the three-year average for the percentage of admitted students who matriculate – 87%.  This would yield a first-year entering class of 36,975, down about 2.5% from 2014.   

Estimates of Admitted Students and Matriculants for 2015 Based on Trends in 2012-2014



Admitted   Students

Percent   of Applicants


Percent  of Admitted



















2015   (est.)






Why These Estimates for Admitted Students and Matriculants Might be Too High

        a.      Significant Decline in Applicants with LSATs of 165+

        Because of changes in the nature of the applicant pool in 2015, however, the estimates of the number of admitted students and number of matriculants in the chart above may be too high.  In 2014, almost all of the decrease in applicants came among those with LSATs of <165.  The pool of applicants with LSATs of 165+ in 2014 was only slightly smaller than in 2013 (7,477 compared with 7,496). Indeed, as a percentage of the applicant pool, those with LSATs of 165+ increased from 12.6% in 2013 to 13.4% in 2014.  This resulted in a slight increase in the number of matriculants with LSATs of 165+ in 2014 compared to 2013 (6,189 compared with 6,154).

        In the current cycle, however, the number of applicants with LSATs of 165+ was only 6,320 as of March 6, 2015. In 2013, there were 7,228 on March 8, 2013 (of a final total of 7,496).  In 2014, there were 7,150 on March 7 (of a final total of 7,477).  Thus, the average increase in applicants with LSATs of 165+ between early March and the end of the cycle is only about 4%.  That would suggest that we could anticipate having roughly 6,585 applicants with LSATs of 165+ at the end of the cycle – down nearly 900 from 2014 – over 12%.

Estimate of Number of Total Applicants for 2015 with LSATs of 165+ Based on Trends in 2013 and 2014


Applicants at 165+


Applicants at 165+

# Increase to end of Cycle

% Increase to end of Cycle

March 8, 2013


End of Cycle 2013




March 7, 2014


End of Cycle 2014




March 6, 2015


End of Cycle 2015 (est.)




        On a longer term basis, if the estimates in the preceding paragraphs are accurate, the entering class in fall of 2015 will again extend the slide in the number and percentage of first-year students with LSATs of 165+ that has been underway since the class that entered in fall of 2010.

Five-Year Trend in Applicants and Matriculants with LSATs of 165+  and Estimates for 2015


Applicants with LSATs of 165+

Matriculants   with LSATs of 165+

Percent   of Applicants Matriculating





















2015 (est.)




        Given that on average over the last three years roughly 82.4% of admitted students with LSATs of 165+ actually matriculated, one could expect that the 6,585 applicants would translate into 5,420 matriculants with LSATs of 165+ for fall 2015, a decline of nearly 770 from 2014.  Notably, this would represent a 45.9% drop in applicants with LSATs of 165+ since 2010 and a 42.8% drop in matriculants with LSATs of 165+ since 2010.

        b. Modest Decrease Among Applicants with LSATs <150

        On the other end of the LSAT distribution, it is a completely different story. Although the number of applicants with LSATs <150 also has declined, the decline has been more modest than among those with LSATs of 165+.  Moreover, those with LSATs of <150 are much more likely to apply late in the cycle.  In the last two years there has been significant growth among applicants with LSATs of <150 between early March and the end of the cycle.   As a result, I would estimate that we would have 18,350 applicants with LSATs of <150 by the end of this cycle, a decline of only about 4.5%.

Estimate of Number of Total Applicants for 2015 with LSATs of <150 Based on Trends in 2013 and 2014


Applicants with LSATs of <150


Applicants with LSATs of <150

# Increase

% Increase

March 8, 2013


End of Cycle 2013




March 7, 2014


End of Cycle 2014




March 6, 2015


End of Cycle 2015 (est.)




        With applicants with LSATs <150 making up a larger percentage of the declining applicant pool, the number of matriculants with LSATs of <150 actually had grown each year up until 2014, when the slight increase in matriculants with LSATs of 165+ was mirrored by a slight decrease in matriculants with LSATs <150. 

Five-Year Trend in Applicants and Matriculants with LSATs of <150 and Estimates for 2015


Applicants   with LSATs of <150

Matriculants   with LSATs of <150

Percent   of Applicants Matriculating





















2015 (est.)




        Given that the percentage of applicants with LSATs <150 matriculating has increased each of the last five years, it seems reasonable to expect another increase – to 47.4% -- resulting in roughly 8,700 matriculants with LSATs of <150, particularly given the decrease in the number of applicants with LSATs of 165+.  Even so, it seems unlikely to make up for the drop of nearly 770 matriculants among those with LSATs of 165+.  Notably, while the pool of applicants with LSATs <150 has decreased by of 30.9% since 2010, the number of matriculants has increased by 24.2%.

        Thus, while the smaller decline in applicants that is expected this year might suggest a correspondingly smaller decline in matriculants, with the weaker profile of the applicant pool in 2015 compared to 2014, it is quite possible that the total number of admitted students will be lower than the chart above suggests and that the corresponding number of matriculants also will be lower than the chart above suggests.

        Phrased differently, if there really is going to be a decline of roughly 770 matriculants just in the group with LSATs of 165+, then the total decline in matriculants may well be greater than the 950 estimated in the chart above.  Between 2013 and 2014, a decline in applicants of 3,700, almost all with LSATs of 164 and below, resulted in a decline in matriculants of 1,750, all with LSATs of 164 and below.  If the decline in applicants is 1,700 this cycle, with over half the decline among those with LSATs of 165+, with a decline of perhaps several hundred with LSATs between 150-164, and with a modest decrease (or possibly a slight increase) among those with LSATs <150, we may well see that the decline in admitted students and in matriculants is slightly larger than estimated in the chart above.


        One interesting side note is that the significant decrease in the number of applicants with LSATs of 165+ is likely to put significant pressure on a number of top-50 law schools as they try to hold their enrollment and their LSAT profiles.  Simply put, there are not enough applicants with LSATs of 165+ to allow all the law schools in the top-50 or so to maintain their profiles and their enrollment. 

        If the estimates above are correct – that there will be roughly 5420 matriculants with LSATs of 165+– and if we assume that at least a few hundred of these matriculants are going to be going to law schools ranked 50 or below either due to geography or scholarships or both – and if we assume that the top 15 law schools are likely to leverage rankings prestige (and perhaps scholarships) to hold enrollment and profile -- then the decrease of roughly 770 matriculants with LSATs of 165+ is going to be felt mostly among the law schools ranked 16-50 or so. 

        In 2014, the top 15 law schools probably had roughly 3,800 first-year matriculants with LSATs of 165+.  The schools ranked 16-50 likely had another 1,900 or so.  The remaining 500 plus matriculants with LSATs of 165 and above likely were scattered among other law schools lower in the rankings. Let’s assume the top-15 law schools manage to keep roughly 3,700 of the 3,800 they had in 2014.  Let’s assume law schools ranked 50 and below keep roughly 500 or so.  That means the law schools ranked between 16 and 50 have to get by with 1,220 matriculants with LSATs of 165+ rather than 1,900 last year.  While many schools will be dealing with the challenges of maintaining enrollment (and revenue) while trying to hold profile, this likely will be a particularly challenging year for law schools ranked between 16 and 50 that are trying to navigate concerns about enrollment (and revenue) with concerns about profile.  To the extent that those schools look toward applicants with lower LSAT profiles to maintain enrollment, that will then have a ripple effect through the law schools lower in the rankings.

April 13, 2015 in Data on legal education, Scholarship on legal education | Permalink | Comments (0)

Sunday, April 12, 2015

Another Example of Using Big Data to Improve Odds of Winning in Court

Back in February, I wrote a post on The The Early Days of Legal Analytics.  It discussed some of the innovations at Lex Machina, a legal start-up that uses Big Data to value contested patents and develop a litigation strategy designed to maximize value / minimize risk.  I recently came across another company, Premonition, that claims to use artificial intelligence to select lawyers with the best odds of achieving a favorable result. See Premonition Infographic at bottom of this post.

I spend a lot of time on the road talking to law firm lawyers and legal innovators, including legal start-ups.  Many large firm lawyers tend to dismiss new innovations without stopping to listen to, much less gather, relevant facts.  Likewise, there is a lot of puffery among legal start-ups as they try to land their first few customers.  Thus, I tend to apply a windage factor to accurately interpret what I am being told.

The image below contains Premonition's simple, one-sentence pitch.  I don't know about the product, but the concept is pretty clear.


The core benefit Premonition appears to offer is a list of lawyers with winning track records in front of specific judges. We don't need artificial intelligence (AI) to make that calculation.  A win-rate is a simple descriptive statistic, even if it has been filtered for a variety of matching criteria.

That said, AI could come in handy in building the requisite data sets.  As explained on Premonition's website, courts don't construct their case management systems so they can be vacuumed out by data mining companies. Indeed, local court officials would likely to be hostile to such requests because any resulting statistical model is unlikely to make them look good.  Indeed, the purpose of the model is, at least in part, to identify and exploit imperfections in the judicial process.  

Because the courts have no incentive to make life easy for Big Data vendors, Premonition's Chief Innovation Officer, Toby Unwin, claims to have tackled the data assembly problem by building a technology that scrapes and buckets the necessary data from the jumbled chaos of web portals for state and local courts.  Such a task, in theory, can be performed by fairly standard machine learning, which qualifies as AI, at least in some circles.  

Assuming Premonition has built a machine that can calculate win-rate of lawyers, is that information valuable to clients trying to maximize the likelihood of a favorable result?  I don't know, but its plausible enough to test with data.

Some data skeptics will argue that win-rates, whether high or low, are just artifacts of any normally distributed outcome.  The reasoning runs, "Two, three, and four sigma events occur in the ordinary course of life, but regression to the mean is pulling them back to the center. Thus, they are poor predictors of the future."  This reasoning is why many people buy indexed funds rather than shares in actively managed mutual funds.  Cutting the other way, the hedge fund industry is premised on the belief that some money managers are a lot better than others. Five-year return rates are aggregated and published in the industry trade press.  Some of the returns may be due to random luck, but some could be attributed to superior skill.  It is absurdly unlikely, for example, that Warren Buffet's success in buying and selling stocks is just a 60-year lucky streak.

In the case of win-rates in court, I can think of at least two plausible non-random factors that could affect outcomes:

  1. Judicial bias or favoritism.  Judges, either consciously or unconsciously, may react differently to the case depending upon the advocate.  One does not have to wade too far into the political science literature to find peer-reviewed empirical studies that reveal that judges are influenced by more than just facts and law. 
  2. The gap between credentials and bona fide skill.  Law has historically been a credence good.  This means the market relies on elite credentials and firm reputation as a proxy for skill.  Yet, it is plausible that some lawyers may lack the pedigree to get hired by large, elite law firms, yet they go on to develop outstanding legal skills, perhaps because of superior drive, intellectual curiosity, or "early at-bats" as a prosecutor or public defender. If these folks exist, Big Data can likely find them.

I can't vouch for Premontion's technology beyond two statements: (1) it sounds plausible, and (2) it is a waste of time to debate its usefulness because it's an empirical question that the market will answer in the relatively near term.  

Below is one of Premonition's infographics.



April 12, 2015 in Cross industry comparisons, Current events, Innovations in law, New and Noteworthy | Permalink | Comments (2)

Wednesday, March 18, 2015

Igniting Law Teaching Conference

Registration is now open for LegalED’s Igniting Law Teaching 2015.  The conference is Friday, March 20, 2015 from 9:00 am – 5:00 pm EST at American University Washington College of Law.  It is also available for live viewing by webcast.

The conference will feature talks by 30 law school academics and practitioners from the US, Canada and England in a TEDx-styled conference to share ideas on teaching methodologies.  LegalED’s Teaching Pedagogy video collection includes many of the talks from last year’s conference, which have been viewed collectively more than 5000 times.  The panels for this year include:

  • Law Teaching for the 21st Century
  • Applying Learning Theory to Legal Education
  • The Art and Craft of Law Teaching
  • Using Technological Tools for Legal Education, and Pathways to Practice. 

Here is a link to the topics, speakers and schedule.

 The Igniting Law Teaching conference is unlike other gatherings of law professors.  Here, talks will be styled as TEDx Talks, with each speaker on stage alone, giving a well scripted and performed 8 minute talk about an aspect of law school pedagogy.  In the end, we will create a collection of short videos on law school-related pedagogy that will inspire innovation and experimentation by law professors around the country, and the world, to bring more active learning and practical skills training into the law school curriculum.  The videos will be available for viewing by the larger academic community on LegalED, a website developed by a community of law professors interested in using online technologies to facilitate more active, problem-based learning in the classroom, in addition to better assessment and feedback.

 Readers are encouraged to attend the March 20th conference, either live or virtually

[HT: the tireless and thoughtful Michele Pistone of Villanova Law]

March 18, 2015 in Current events, Fun and Learning in the classroom | Permalink | Comments (0)

Saturday, February 7, 2015

The Early Days of Legal Analytics

LexMachina-logo1There is an interesting story in Forbes on Lex Machina, a legal start-up that provides analytics for use in patent litigation.  See Dan Fisher, Stanford-Bred Startup Uses Moneyball Stats to Handicap Judges, Forbes, Feb. 2, 2015.  The company was created by faculty at Stanford Computer Science and Stanford Law.  As the company emerged from the University, the reigns were handed to Josh Becker, a Stanford JD-MBA.  To date, the company has raised $8 million in start-up funding.  According to the Forbes article, the company's clients include some of the nation's large technology companies plus one-third of the AmLaw 100.

What makes Lex Machina so interesting is that the company is not a NewLaw service provider that trying to take marketshare. Instead, Lex Machina is a toolmaker.  It is a true Big Data company that provides analytics to (a) value contested patents and (b) protect/maximize that value through a litigation strategy that is informed by data.  

The impact of Lex Machina is hard to decipher, primarily because if it does provide an edge, the customers are unlikely to be too vocal. Just like a hedge fund with an effective trading strategy, why advertise the ingredients of your secret sauce? Indeed, compared to other toolmakers (e.g., predictive coding, expert systems) Lex Machina's benefits are less about efficiency and more about affecting the outcomes of cases -- who wins and by how much.  If Lex Machina is truly delivering, it will eventually touch-off a Big Data legal analytics arms race akin to the quant revolution on Wall Street.  Dan Katz frequently makes this point, and I think he is right.  The Forbes article makes the point that Lex Machina is already moving into adjacent areas of IP law and general commercial litigation.  

The broader legal industry is unlikely to notice Lex Machina until it has a substantial liquidity event -- i.e., it's acquired or goes public, making if founders far richer than the BigLaw partners and in-house lawyers they currently serve.  

If we are looking for early signs of a tipping point for legal analytics, one marker may be the number of Stanford Law grads who are turning down entry-level opportunities in BigLaw to pursue legal start-ups.  In recent years, Stanford Law grads fresh out of law school have gone on to found other venture-backed legal start-ups like Ravel Law, Judicata, and Law Gives.  Back in 2013, The Stanford Lawyer (SLS alumni magazine) had an extensive write-up with several examples.  See Sharon, Driscoll, A Positive Disruption, June 4, 2013.  In 2014, Stanford's CSO offered a program titled, An Alternative to BigLaw -- Startups.

The legal world isn't going away; it's just changing.

February 7, 2015 in Cross industry comparisons, Current events, Data on the profession, Innovations in law, New and Noteworthy, Structural change | Permalink | Comments (0)

Saturday, January 17, 2015

Stark on Curriculum Reform (and Law School Pre-reqs)

UnknownTina Stark, a pioneer in transactional skills education, has an interesting and highly readable take on how to organize a modern law school curriculum, including the first year.  It's all insightful, but I think the most powerful suggestion is the creation of three foundational modules - litigation, transactions, and legislation/regulation - in which the offerings would combine theory, doctrine, and skills.  

The piece is a transcript of her keynote address, "What Cornell Veterinary School Taught Me About Legal Education," at the Fourth Biennial Conference of Emory Law School's Center for Transactional Law and Education.

What stirred Tina's reflections was her son's matriculation from the classroom to the clinical portion of his veterinary education.  I posted similar thoughts some time ago reflecting on my son Matthew's (now Dr. Matthew Lipshaw, resident in pediatrics, Yale-New Haven Children's Hospital) med school experience.  I'm still persuaded that there are two things medical education has over legal education (I can't speak to veterinary): (a) the reimbursement system in health care provides resources for skills training not likely to be available any time soon for legal education, and (b) with all due respect to law students (and I know whereof I speak because I was one), even with a semester or a year in a clinical offering, med students work a hell of a lot harder than law students.  We'd have no problem getting both doctrinal and skills training into our students if they were at it six or seven days a week, twelve or thirteen hours a day.

If you download Tina's piece, there's a lagniappe at the end - some thoughts on why there's no law school pre-req for English composition, yet there is in med and engineering school!


January 17, 2015 | Permalink | Comments (2)