Wednesday, September 14, 2016

Lawyers for People versus Lawyers for Business

The work of lawyers is increasingly the work of businesses rather than people.  This conclusion flows from recently released Economic Census data, which is the U.S. Government's "official five-year measure of American business and the economy."

For the two most recent years (2007 and 2012), the Economic Census data includes an analysis called Revenues/Receipts by Class of Customer for Selected Industries.  The chart below compares these two years for Offices of Lawyers (NAICS 541110).

From 2007 to 2012, the share of total law office receipts shifted by about 5% away from individuals toward businesses. Revenues for Offices of Lawyers grew during this period from $225 billion to $246 billion.  However, when we run the numbers, the total receipts for lawyers serving people declined from $65 billion to $59 billion.  That is a relatively large absolute decline in just five years.  It suggests an actual contraction in the amount of legal work for people. Yet during this same period, the nation grew from 288 million to 302 million people

These fairly stark results continue the trendlines of the Chicago Lawyers I and II studies.  Chicago Lawyer I showed that roughly half of lawyers in Chicago in 1975 were working for people and half were working for corporations. This was the basis for the Heinz-Laumann two-hemisphere theory.  When the study was replicated in 1995 (Chicago Lawyers II), the data showed twice as many organizational lawyers versus people lawyers, so hemi (as in half) no longer applied.  Further, among lawyers in solo and small firms -- the primary practice setting for people lawyers -- income had dropped significantly in inflation-adjusted dollars. In contrast, lawyers in large firms and in corporate legal departments experienced significant gains.

If during the 2007 to 2012 timeframe the proportion of people work dropped from 29.1% to 23.9%, what does that number look like today?  We will not know definitively until 2020 or 2021 when the Census Bureau releases the class of customer data from the 2017 Economic Census, yet a further decline certainly seems likely, particularly as services like LegalZoom and RocketLawyer continue to target the retail market.  Separate and apart from these new entrants, to what extent is the diminution in people lawyers driven by declining real incomes within the middle class?  

It is possible that the archetypical images of private practice lawyers are becoming more and more out-of-sync with what is happening in the actual market.  For those creating law school curricula or setting policy around access to justice, we are going to need new mental models of what it means to be a lawyer.

September 14, 2016 in Current events, Data on the profession, New and Noteworthy | Permalink | Comments (1)

Monday, September 5, 2016

Building a Portfolio of Court Cases the Way a Quantitative Hedge Fund Buys Stock

For the last couple of years, Dan Katz has been telling me and anyone else who would listen that law will eventually be a subfield of finance. Following Dan's reasoning, this will occur because legal risk can be modeled and quantified like financial risk, thus enabling parties to allocate time, money, expertise based on probabilities. If the modeling is accurate within a fairly predictable range, it facilitates an investment strategy where the business side of legal risk is equally predictable. Add leverage and/or other people's money, and basically you have a hedge fund with legal claims as its primary asset class.

Based on a story in Sunday's Boston Globe, Katz may be on to something.  The story reports the launching of Legalist, which funds lawsuits based on the size and probability of recovery. This concept is not new, as companies like Burford Capital have moved into this market and are growing rapidly. 

Legalist is potentially different, however, because it pools together smaller and medium-sized legal claims that are likely to paid out.  Case evaluations are made using data algorithms that draw upon "a database of 15 million court cases from all over the United States." So, in theory, the company can generate a strong return by building a diversified portfolio of claims that are likely too small for the high-end litigation financiers like Burford. 

ShangOne of the co-founder of Legalist is Eva Shang (photo right), who is pursuing the idea thanks to a $100,000 grant from the Thiel Foundation.  This is the Peter Thiel organization that encourages undergraduates to drop out of college in favor of pursuing a promising start-up idea.  After forgoing her senior year at Harvard, Shang also earned a spot in Y Combinator, the famed Silicon Valley accelerator that has a strong record of picking and nurturing successful start-ups. The other co-founder is Christian Haigh, who is a master's degree student in computer science at Harvard.

The core idea here is that with the right quantity and quality of data, computers can be extremely useful in valuing legal claims on several dimensions: size of payout, likelihood of payout, total time to reach a resolution, etc.  Lawyers provide the same service, but with a sample size limited to their own experience.

The most intriguing part of the Boston Globe story is that Shang and Haigh originally thought that law firms would pay for their service in order to improve their own case assessments. But "about a month in, we realized that attorneys weren't all that interested in legal analytics."  The dialogue with lawyers, however, enabled them to learn about the field of litigation finance.

Based on these insights, the Legalist decided to pivot.  In the June 2016 press release from the Thiel Foundation, the company was described as "a legal analytics and alert platform that helps lawyers keep track of new developments in case law so that they can represent their clients more effectively." Today, Shang's LinkedIn page describes Legalist as an "Algorithmic litigation financing for small businesses with meritorious lawsuits" -- a description with a Y Combinator polish.

I have no idea whether Shang or Legalist will be successful.  However, the story provides another striking example of the reluctance or inability of lawyers--I don't know which--to consider data as a tool to better serve their clients and, perhaps as a result, to earn a higher income.  Conversations with lawyers on this topic often stall on what the data cannot effectively model and thus the mistakes that might result. The mindset seems to be, "find an example where it may not work and kill the concept." I really do believe that there are a handful of behavioral economics biases that apply with special vengeance to lawyers. 

Katz is likely right to think of law as a powerful use case for finance.  The goal is not about getting something right this time, but instead getting it right more often than not with a high degree of certainty.  In short, it's probability with reliable estimates of risk.  And for clients, that is valuable.  

If we pull on this string long enough, eventually it will be possible to quantify how, all else equal, particular law firms and lawyers affect the odds of winning a case.  When that happens, there will be strong incentives to deconstruct the skill sets and backgrounds of the most bankable lawyers.  Law will become less a credence good and accordingly the utility of longstanding signals of quality that law schools and law firms are built around will get tested by data and repriced.

I have no idea if this future will actual emerge.  Certainly a case could be made that were are better ways to resolve disputes than protracted litigation where armies of lawyers are hired to advance only one side of an argument. Regardless, I am confident that the practice of law is definitely going to change.

September 5, 2016 in Current events, Innovations in law, New and Noteworthy, Structural change | Permalink | Comments (1)

Thursday, July 14, 2016

International Legal Hackers Summit in Brooklyn this Weekend

Legal-Hackers-Logo-2The 2nd Annual International Legal Hackers Summit takes place this weekend in Brooklyn, New York. The event includes an impressive array of keynotes, panels, workshops, demos, and cultural activities from leaders of the legal hacking, legal technology and civic innovation communities.  U.S. legal education has a surprisingly strong showing. Full details here.

LearOne of the organizers, Dan Lear (@rightbrainlaw), Director of Industry Relations at Avvo, is offering weekend passes for a small number of law students interested in attending.  If you drop my name, Dan will feed you and help you network, starting with the Welcome Party tomorrow night at 61 Local Cafe & Public House.  If you are a law student in NYC this weekend and want to attend, please send me an email ( and I will connect you with Dan. 

I guarantee a learning experience that will be inspiring and fun.

July 14, 2016 in Current events, Innovations in law, Innovations in legal education | Permalink | Comments (0)

Sunday, May 1, 2016

Mixed Signals from the Legal Employment Market – Preliminary Results for the Class of 2015


The Class of 2015 employment summary reports have been posted by all ABA-accredited law schools, resulting in reporting of results for some states or regions.  The ABA Section of Legal Education and Admissions to the Bar released the complete Employment Summary spreadsheet for all law schools on its website yesterday (May 2) and updated it today (May 3) and likely will be updating it again tomorrow (to eliminate the double-counting for Hamline, William-Mitchell and Mitchell|Hamline).

In this initial post I provide a brief summary of the Class of 2015’s employment outcomes compared with the Class of 2014’s employment outcomes based on data from these spreadsheets as of described above.

In a subsequent post (posted on May 2) I provide a summary of changes in the reported number of law-school-funded, bar-passage-required positions between the Class of 2014 and the Class of 2015 as a result of changes in the classification and reporting of such positions.

Changes in the Percentage of Graduates and Number of Graduates in Full-Time, Long-Term Bar-Passage-Required and JD Advantage Jobs

Across all law schools for which the ABA has released employment summary data for the Class of 2015, the percentage of graduates in full-time, long-term bar-passage-required positions and full-time, long-term JD advantage positions increased from 69% for the Class of 2014 to 70.1% for the Class of 2015. This would appear to be modestly good news. When you disaggregate the two categories, the full-time, long-term bar-passage required positions went from 58% to 59.2% while the full-time, long-term JD advantage positions went from 11% to 10.9%.

Because there was a significant decline in the number of graduates across these law schools between 2014 and 2015, however, this modest increase in the percentage of graduates in these positions masks an actual decline in the number of graduates in such positions. There were 39,984 graduates in the Class of 2015 compared with 43,832 graduates in the Class of 2014, a decline of 3,848 graduates, or 8.8%. There were 28,029 graduates in the Class of 2015 with full-time, long-term bar-passage-required or JD advantage positions, compared with 30,234 graduates in the Class of 2014 with such positions, a decline of 2,205, or 7.3%.

When these totals are disaggregated, full-time, long-term bar-passage-required positions declined from 25,417 for the Class of 2014 to 23,687 for the Class of 2015, a decline of 1,730, or 6.8%. For full-time, long term JD advantage positions, the total went from 4,817 to 4,342, a decline of 475, or 9.9%.

(Please note that numbers for both 2014 and 2015 exclude law-school-funded positions from both categories.  The ABA's 2015 Law Graduate Employment Data sheet compares Class of 2014 INCLUDING law-school-funded positions with CLASS of 2015 EXCLUDING law-school-funded positions, which leads to slightly different results showing a more exaggerated decline in the number of graduates in full-time, long-term bar-passage-required and JD advantage jobs that also results in a decline in the percentage of graduates in such positions.)

Comparison of Full-Time, Long-Term Bar-Passage-Required Positions and JD Advantage Positions for the Class of 2014 and Class of 2015















Class of 2014








Class of 2015
















Changes in the Number and Percentage of Graduates Whose Employment Status is Unknown or Who Were Classified as Unemployed Seeking or Unemployed Not Seeking

Looking at the other end of the employment outcomes continuum, however, both the number and percentage of graduates who had unknown employment outcomes, or who classified as unemployed seeking or unemployed not seeking, declined slightly between the Class of 2014 and the Class of 2015. For the Class of 2014, there were 5,778 graduates whose employment status was unknown or who were classified as unemployed seeking or unemployed not seeking. This represented 13.2% of the 43,832 graduates. For the Class of 2015, however, there were only 5,200 graduates whose employment status was unknown or who were classified as unemployed seeking or unemployed not seeking. This represented 13% of the 39,984 graduates.

Searching for Explanations

In the coming weeks and months, there likely will be a number of commentators offering suggestions for why the Class of 2015 might have seen a decline in the number of graduates obtaining full-time, long-term bar-passage-required or JD advantage positions.

Part of the decline likely is attributable to the decline in the number and percentage of graduates passing the July bar exam, as reported by the NCBE in its annual statistics publications for each of the last three years.


First-Time Bar Takers in July from ABA-Accredited Law Schools*

First-Time Bar Passers in July from ABA-Accredited Law Schools

July Pass Rate Among First-Time Takers from ABA-Accredited Law Schools













*Note that the NCBE’s classification of first-time takers is over-inclusive in that it reflects not just graduates from May who are taking the bar exam for the first time in July, but also graduates from a prior year who might be taking the bar exam for the first-time in a given jurisdiction even if they have previously taken the bar exam in another jurisdiction. Thus first-time bar passers includes some people who are not part of the graduating cohort in a given year.

In the two-year period, then, between 2013 and 2015, the number of first-time takers from ABA-accredited law schools taking the July bar exam who passed the exam and became eligible for jobs requiring bar passage declined by roughly 9,100 and by nearly 23.5%. Moreover, the percentage of all first-time bar takers taking the February exam rather than the July exam also increased slightly between 2013 and 2015 from 18.7% to 19.7%, which might mean slightly more May 2015 graduates might not have been positioned to accept a full-time, long-term bar-passage-required or JD advantage position as of March 15, 2016, because they may have been studying for and taking the February 2016 bar exam.

Part of the decline also likely is attributable to market conditions in some parts of the country. For example, a recent story about graduates of Texas law schools noted that the decline in oil prices and tort reform may have impacted hiring in the Texas legal market for graduates of the Class of 2015. Once the full set of employment outcomes is available, it will be easier to assess the extent to which certain states or certain regions might have seen better or worse results than other states or regions.

Part of the decline also may be a manifestation of the impact of technology on the legal services market, with the possibility that the legal services market will have slightly fewer entry level positions over the near term.

One Possible Counterpoint

If this decline in the number of full-time, long-term bar passage required positions is a manifestation of a weakening job market law graduates, then one would expect that salary data also would demonstrate weakness. Once NALP publishes its report on the employment results for the Class of 2015 later this summer, we will have a chance to assess the extent to which salary trends are consistent with a weakening legal services market or suggest that the market remains somewhat competitive. If this decline in graduates taking jobs that are full-time, long-term bar passage required or JD advantage jobs is counterbalanced by a continuation of the year-over-year modest increases in mean and median salaries in recent years for  law graduates, it might suggest that that there is less market weakness than this initial employment summary might indicate.

Concluding Thoughts

For those thinking that the recent news about the improving situation with respect to applicants to law school is the beginning of an upward trend that will gradually return law schools to first-year class sizes in the 45,000 to 46,000 range, this employment outcomes data provides a cautionary tale. The fact that the employment market for law school graduates appears to have stagnated and even declined to some extent over the last two years may mean that risk averse potential law school applicants who focus on post-graduate employment opportunities when assessing whether to invest in a legal education may remain skittish about applying, such that this year’s good news on the applicant front may be somewhat short-lived.

(I am very grateful for the research assistance of Janelle Chambers in gathering data for this blog posting prior to the release of the ABA Employment Summary spreadsheet and for very helpful comments on earlier drafts of this blog posting from Scott Norberg and Bernie Burk and for the helpful insights of Debby Merritt as we worked on reconciling data in the ABA spreadsheets.)

May 1, 2016 in Current events, Data on legal education, Scholarship on legal education | Permalink | Comments (1)

Tuesday, March 8, 2016

Comments on Proposed Revisions to Standard 501

Standard 501 requires law schools to have sound admissions policies and to refrain from admitting applicants who are not capable of being successful in law school and on the bar exam. For many years, Standard 501 has received little attention, while Standard 316 – the bar passage standard – has received more attention. Accreditors focused more on outcomes – bar passage results – than inputs – the academic abilities of students admitted to law school. With the decline in the number of applicants to law school and the corresponding erosion of entering class credentials at many law schools, however, Standard 501 has begun to receive more attention.

Specifically, due to concerns that some law schools might be admitting students whose entering credentials suggest that they are not likely to be successful in law school or be able to pass the bar exam, the Standards Review Committee (“SRC”) of the ABA Section of Legal Education and Admissions to the Bar (“Section”) has proposed revisions to Standard 501. The Council for the Section will consider these proposed revisions at its upcoming March meeting.  

I am in favor of most of the suggested revisions to Standard 501 (as discussed below). I am not in favor of the SRC’s Proposal 2 which proposes an “attrition” threshold above which schools would face heightened attention. In a subsequent post, I will discuss the need for a more robust Standard 308, which addresses academic standards, as a corollary to Standard 501.

“Proposal 2” Presents Two Problems

In its Proposal 2, the SRC suggests adding a new Interpretation 501-3 – “A law school having a non-transfer attrition rate above ___ percent bears the burden of demonstrating that it is in compliance with the Standard.” (The SRC anticipated that the Council would insert a number, perhaps 10%, in place of the blank.) Proposal 2 presents two problems; it is too broad in scope, and it is unlikely to be effective in practice.

The proposed interpretation focuses on “non-transfer” attrition, when it should be focused solely on “academic attrition.”

Non-transfer attrition consists of two components – “academic attrition” and “other attrition.”

Academic attrition includes students involuntarily dismissed under a law school’s academic policies or students who leave voluntarily but would have been dismissed had they remained in school. As noted in my recent blog posting analyzing attrition data, academic attrition varies widely across law schools from zero to over 20%, particularly among law schools with relatively low LSAT/GPA profiles.

Other attrition includes students who leave law school for reasons other than academic attrition. They may have decided law school is not for them, or have had a family emergency or their own physical health concern or mental health concern that leads them to withdraw from law school. Other attrition has much less variability; at most law schools other attrition is in the 2% to 4% range.

Law schools should be able to look at historical trends regarding student performance at their law school to predict whether applicants with certain entering class credentials are likely to experience “academic attrition.” Thus, a high “academic attrition” rate may suggest a law school is admitting too many students who are unlikely to be successful. By contrast, law schools rarely are going to be able to identify in advance those students who are likely to fall into the “other attrition” category.

Accordingly, if Proposal 2 is going to move forward, I would strongly advise that it focus solely on “academic attrition” rather than on all non-transfer attrition.

(It is conceivable that the SRC chose non-transfer attrition rather than academic attrition because it was concerned law schools would opt to classify a student’s attrition as other attrition rather than academic attrition to avoid whatever threshold might be set for academic attrition. While that risk could be addressed through careful assessment of attrition data, if there is a strong desire to use non-transfer attrition, I would suggest that the non-transfer attrition threshold be set at a slightly higher percentage to recognize that many law schools regularly experience other attrition of between 2% and 4%.)

The proposed interpretation is unlikely to accomplish its intended purpose

Regardless of whether the threshold focuses solely on academic attrition or on non-transfer attrition, however, this proposed interpretation fails to account for how law schools are likely to respond to the new interpretation. The SRC may believe the proposed interpretation will make law schools refrain from admitting as many at risk students. While that is possible, it is actually as likely or more likely that some law schools will not change their admissions practices, but simply will adjust how they implement their academic dismissal policies or grading policies to keep academic attrition or non-transfer attrition below whatever threshold is established.

For example, assume the proposed interpretation set 10% as the academic attrition “threshold” for shifting to law schools the burden of demonstrating compliance with the Standard. In the 2014-15 academic year, an academic attrition threshold of 10% would have “caught” 30 law schools. One easily can imagine that a significant number of those law schools likely would simply adjust their academic dismissal policies or their grading policies so that they maintain academic attrition below the 10% threshold (even at the risk of noncompliance with the bar passage standard a few years later). (Indeed, 11 of the 30 law schools in 2014-15 with academic attrition more than 10% have an academic attrition rate between 10% and 11%, such that getting under 10% would not have been very difficult for these law schools.)

The SRC may be assuming that law schools don’t have “control” of how academic attrition actually functions. Perhaps the SRC believes that academic dismissal policies are fairly consistent across all law schools such that a given threshold (10%) would have comparable meaning and effect across all law schools. As noted above, however, academic attrition varies widely among similarly situated law schools, particularly those with relatively low LSAT/GPA profiles. This suggests that academic dismissal and grading policies differ across law schools or, phrased differently, that the way in which grading policies interact with academic dismissal policies varies widely. In reality, law schools have sufficient “local control” over their grading and academic dismissal policies that it would not be that difficult for law schools to avoid being “caught” by whatever academic attrition or non-transfer attrition threshold would get established in Standard 501.

The Other Three Suggested Revisions to Standard 501 are Generally Good Ideas

First, in Section 501(a), the SRC wants to replace “maintain” with “adopt, publish, and adhere to such that the standard will read: “A law school shall adopt, publish, and adhere to sound admission policies and practices consistent with the Standards, its mission, and the objectives of its program of legal education.” This clarifies that law schools have to have policies, have to publish policies, and have to adhere to the policies, all good things.

Second, in Section 501(b), the SRC wants to shift from a negative framework to a positive framework. The existing standard has a negative framework -- “A law school shall not admit an applicant who does not appear capable of satisfactorily completing its program of legal education and being admitted to the bar.”   The SRC recommends shifting to a positive framework – “A law school shall admit only applicants who appear capable of satisfactorily completing its program of legal education and being admitted to the bar.” (emphasis added) While I don’t feel strongly about this, I think it is easier to conceptualize this in the positive framework – focused on who should be admitted -- rather than on who should not be admitted. That said, it might help to maintain a singular focus on each “applicant” rather than shifting to the plural “applicants.” “A law school shall admit an applicant only if the applicant appears capable of satisfactorily completing its program of legal education and being admitted to the bar.” This keeps the focus on each individual applicant rather than the pool of applicants a law school admits.

Third, in the first interpretation – Interpretation 501-1 -- the SRC recommends adding a sentence that states: “Compliance with Standard 316 is not alone sufficient to comply with the Standard.” Standard 316 is the bar passage standard. This suggested revision is designed to highlight that Standard 501 is an independent standard that is not just derivative of Standard 316. This also strikes me as a useful change.

If Standard 501 is simply derivative of Standard 316, then there is no way to assess compliance with Standard 501 in “real time.” Rather, one would only assess compliance with Standard 501 by waiting three or four years to see whether the graduates of the law school comply with the bar passage standard. Standard 501 would be somewhat superfluous.

By adding this sentence, the SRC is suggesting that compliance with Standard 501 should be assessed “presently” by looking at the LSAT/GPA profile of matriculants, the law school’s experience with attrition, and the success of the law school’s academic support program, along with the law school’s historical bar passage results. Historical attrition and historical results on the bar exam in relation to prior graduates’ entering LSAT/GPA profile and prior graduates’ law school academic performance should inform the determination of whether the law school is continuing to admit only those applicants reasonably capable of being successful in the program of legal education and in passing the bar exam. If students with certain LSAT/GPA profiles over a three-year or four-year period consistently have performed poorly in law school and experienced academic attrition, or performed poorly on the bar exam, then the law school has data that would make it challenging for the law school to demonstrate that applicants with those profiles are “capable of satisfactorily completing its program of legal education and being admitted to the bar.”

(I am very grateful for the helpful comments of Debby Merritt and Scott Norberg on earlier drafts of this blog posting.)

March 8, 2016 in Current events, Data on legal education | Permalink | Comments (0)

Sunday, February 21, 2016

Legal Artificial Intelligence: Replacing or Helping Lawyers?

Logo-clioThis Tuesday at 11 am PT /2 pm ET, Clio, a leading cloud-based practice management company, will be offering a provocative webinar on  Legal Artificial Intelligence.  The legal media has hyped this issue with the help of some lawyers with little firsthand knowledge of the topic.  This program, however, has true experts, including ROSS Intelligence co-founders Andrew Arruda and Jimoh Ovbiagele along with Clio's Lawyer in Residence, Joshua Lenon.  The program is also CLE-eligible in at least some jurisdictions. 

The following questions will be covered:

  • What is classified as artificial intelligence (AI)?
  • How legal professionals can use AI?
  • What legal AI tools are in development?
  • How lawyers can get started with AI?

You can register online at:

February 21, 2016 in Current events, Innovations in law | Permalink | Comments (0)

Monday, January 11, 2016

Building the Legal Startup

If I were in northern California tomorrow evening, I would be headed to Stanford Law to attend this interesting program:

CodexBuilding the Legal Start-up

January 12 @ 5:00 pm - 8:00 pm 
5:00 – 9:00pm
Room 290, Stanford Law School

The event is sponsored by CodeX – The Stanford Center for Legal Informatics and Evolve Law.  Some very accomplished start-up folks are participating, including Eddie Hartman (LegalZoom), Josh King (Avvo), Steven Silberbach (Clio), and Jeroen Plink (advisor to Kira Systems who also built Practical Law Company's US operations before the sale to Thomson Reuters).  

The program is free and open to the public.  For those lucky enough to be near by, you can register here.  Remarkably, it is also broadcast live over the web!  

There is no question that northern California has become the hotbed location for legal start-ups.  Stanford Law is ideally situated to both study and facilitate this evolution.

January 11, 2016 in Current events, Innovations in law, New and Noteworthy | Permalink | Comments (0)

Monday, December 21, 2015

"Income at KPMG legal up 53 per cent"

GlobalegalpostThat is the headline this morning from the Global Legal Post, a publication that compiles legal news from around the world. The context is the UK, where changes in legal regulations now permit nonlawyers to own and control legal enterprises through alternative business structures, or ABSs.  

Victoria Basham writes:

KPMG, one of the Big Four accountancy firms to offer legal services through an alternative business structure, has reported a huge rise in income from the division and pledged further investment.

Net sales in its fledgling legal services division climbed 53 per cent in the year to 30 September. The firm’s newly-published annual report comments: ‘Despite [this growth] being from a relatively low base, it’s clear that clients really like our idea of wrapping multi-disciplinary legal advice around our other propositions.

Continue to invest

The report added that the firm would continue to invest heavily in new hires to grow the business in both existing and new areas, including corporate crime and regulation.

The story reports that three of the Big Four now offer legal services under an ABS license.  Basham credits the The Law Society Gazette as her source.

Related posts:

December 21, 2015 in Cross industry comparisons, Current events, Structural change | Permalink | Comments (0)

Sunday, December 6, 2015

The Opaqueness of Bar Passage Data and the Need for Greater Transparency

There has been a great deal of discussion lately over at The Faculty Lounge regarding declines in law school admissions standards, declines in bar passage rates, and the general relationship between LSAT scores and bar passage. Much of this discussion is clouded by the lack of meaningful data regarding bar passage results.  In this blog posting I will delineate several questions that just cannot be answered meaningfully based on the presently available bar passage data.

The national first-time bar passage rate among graduates of ABA-accredited law schools fell significantly in 2014. According to the NCBE’s statistics, the average pass rate from 2007-2013 for July first-time test-takers from ABA-accredited law schools was 83.6%, but fell to 78% in 2014. (2015 data won’t be available until next Spring when it is released by the NCBE.)

While there might be some reasons to believe these results were somewhat aberrational given that the objective criteria of the entering class in 2011 was only modestly less robust than the objective criteria of the entering class in 2010, and given the ExamSoft debacle with the July 2014 bar exam, the results are concerning, given that the objective criteria of the entering classes in 2012, 2013 and 2014 showed continued erosion. As the last two years have seen declines in the median MBE scaled score among those taking the July bar exam, the changes in entering class credentials over time suggest further declines in median MBE scaled scores (and bar passage rates) may be on the horizon.

In 2010, there were roughly 1,800 matriculants nationwide with LSATs of 144 or less. In 2012, there were roughly 2,600 matriculants nationwide with LSATs of 144 or less. In 2014, there were roughly 3,200 matriculants nationwide with LSATs of 144 or less. Recognizing that law school grades will be a better predictor of bar passage than LSAT scores, I think it is safe to say that entering law students with LSATs in this range are more likely than entering law students with higher LSATs to struggle on the bar exam.  Because the number of those entering law school with LSAT scores of 144 or less has grown substantially (particularly as a percentage of the entering class, more than doubling from less than 4% in 2010 to more than 8% in 2014), many are concerned that bar passage rates will continue to decline in the coming years.

While there has been a great deal of discussion regarding declines in admission standards and corresponding declines in bar passage standards, this discussion is profoundly limited because the lack of meaningful bar passage data presently provided by state boards of law examiners and by the ABA and ABA-accredited law schools means that we do not have answers to several important questions that would inform this discussion.

  1. What number/percentage of graduates from each law school (and collectively across law schools) sits for the bar exam in July following graduation and in the following February? Phrased differently, what number/percentage of graduates do not take a bar exam in the year following graduation?

This is a profoundly important set of questions as we look at employment outcomes and the number/percentage of graduates employed in full-time, long-term bar passage required positions. Given that only those who pass the bar exam can be in full-time, long-term bar passage required positions, it would be helpful to know the number/percentage of graduates who “sought” eligibility for such positions by taking a bar exam and the number/percentage of graduates who did not seek such eligibility. It also would be helpful to understand whether there are significant variations across law schools in terms of the number of graduates who take a bar exam (or do not take a bar exam) and whether those who do not take a bar exam are distributed throughout the graduating class at a given law school or are concentrated among those at the bottom of the graduating class. At present, however, this information simply is not available.

  1. What is the first-time, bar passage rate for graduates from ABA-accredited law schools?

One might think this would be known as ABA-accredited law schools are required to report first-time bar passage results. But the way in which first-time bar passage results are reported makes the data relatively unhelpful. Law schools are not required to report first-time bar passage for all graduates or even for all graduates who took a bar exam. Rather, law schools are only required to report first-time bar passage results for at least 70% of the total number of graduates each year. This means we do not know anything about first-time bar passage results for up to 30% of graduates of a given law school. Across all law schools, reported results account for roughly 84% of graduates, leaving a not insignificant margin of error with respect to estimating bar passage rates.

People would have been flabbergasted if the ABA had required reporting of employment outcomes for only 70% of graduates. Now that the ABA is requiring reporting on employment outcomes for all graduates, there is no good reason why the ABA should not be requiring bar passage accounting for all graduates, requiring law schools to note those who didn't take a bar exam, those who took and passed a bar exam, those who took and failed a bar exam, and those for whom bar status is unknown.  (Up until recently, some boards of law examiners were not reporting results to law schools, but my understanding is that the number of state boards of law examiners not reporting results to law schools is now fairly small.)

Notably, for 2011, 2012, and 2013, the average bar passage rate for first-time takers from all ABA-accredited law schools based on data reported by the law schools was consistently higher than the data reported by NCBE for the corresponding years (2011 – 83.8% v. 82%, 2012 – 81.8% v. 79%, 2013 – 82.4% v. 81%. (Moreover, first-time takers are not measured equivalently by the ABA and by the NCBE. The ABA reporting requirement focuses on graduates who took any bar exam for the first-time. The NCBE defines as first-time takers any person taking a bar exam in a given jurisdiction for the first-time. Thus, the NCBE set of first-time takers is broader, as it includes some people taking a bar exam for the second time (having taken the bar exam in another jurisdiction previously).

  1. What is the “ultimate” bar passage rate for graduates from ABA-accredited law schools?

Even though a number of commenters have noted that “ultimate” bar passage is more important than first-time bar passage, there is no publicly available data indicating the ultimate bar passage rate on a law school by law school basis for the graduates of each ABA-accredited law school. What number/percentage of graduates of a given law school who take a bar exam pass after the second attempt? What number/percentage of graduates of a given law school who take a bar exam pass after the third attempt? What number/percentage of graduates of a given law school never pass a bar exam? This information just is not publicly available at present.

While Standard 316, the bar passage accreditation standard, allows schools to meet the standard by demonstrating that 75% or more of those graduates who sat for a bar exam in the five most recent calendar years passed a bar exam, this “ultimate” bar passage data is not publicly disseminated. Thus, while first-time bar passage data is limited and incomplete for the reasons noted above, “ultimate” bar passage data on a law school by law school basis is actually not available.

The modest amount of information available on “ultimate” bar passage rates is not very helpful.  The LSAC National Longitudinal Bar Passage Study contains some analysis of "ultimate" bar passage rates, but it focused on the entering class in the fall of 1991, which it described as being “among the most academically able ever to enter” law school based on entering class statistics (page 14), a description that could not be used with the classes that have entered in the last year or two or three. It also does not contain any information about "ultimate" bar passage for graduates of individual law schools.  In addition, Law School Transparency has recently received some information from at least one law school that has requested anonymity. Much better “ultimate” bar passage information is needed to better inform many of the discussions about the relationship between entering class credentials and bar passage.

  1. How can we compare bar passage results from one jurisdiction to another?

Most state boards of law examiners do not present data regarding bar passage that allows reasonable bases for analyzing the results in ways that provide meaningful insight and a meaningful basis for comparison. Fewer than one-third of states publicly provide information in which a delineation is made between first-time takers and repeat takers on a law school by law school basis and only a few of these provide information about MBE scores on a school by school basis. Accordingly, it is very difficult to make meaningful comparisons of year-over-year results in the months following the July bar exam, because data is rarely reported in a consistent manner. The NCBE does provide statistics annually (in the spring) which includes a delineation of bar passage rates by state based on first-time test takers from ABA-accredited schools, but the NCBE does not provide MBE scores on a state by state basis (although it seemingly should be able to do this).


There is a need for much greater transparency in bar passage data from boards of law examiners and from the ABA and ABA-accredited law schools. It well may be that some law schools would be a more meaningful investment for "at-risk" students, those whose entering credentials might suggest they are at risk of failing the bar exam, because those law schools have done a better job of helping "at risk" students learn the law so that they are capable of passing the bar exam at higher rates than graduates of other law schools with comparable numbers of at risk students. It may well be that some jurisdictions provide "at risk" students a greater likelihood of passing the bar exam.  At the moment, however, that information just isn’t available. Much of the disagreement among various commentators about the relationships between admission standards and bar passage rates could be resolved with greater transparency – with the availability of much better data regarding bar passage results.

December 6, 2015 in Current events, Data on legal education, Scholarship on legal education | Permalink | Comments (0)

Thursday, December 3, 2015

"PwC expands into legal market"

FinancialReviewThat's the headline from the Financial Review, a leading Australian business newspaper.  The plot is nearly identical to a September post regarding accounting firms in India. See India, Big 4 and Elite Law Firms in Direct Competition for Highly Lucrative Advisory Work, LWB, Sept 16, 2015.   The salient point is not that accounting firms are outmaneuvering the law firms -- they're not, as both stories report a robust flow of laterals in both directions.  Rather, it's that the accounting firms are in the game at all. 

The story reports:

"There are bigger issues - alternative legal providers, the changing demands of what our people want in terms of non-lineal career paths, the cost pressures on our clients and the demands they place on their lawyers," Baker & McKenzie national managing partner Chris Freeland said.

"That's what keeps me awake at night," he said.

Behind closed doors, however, [the law firms] are genuinely worried about the accounting firms cutting into compliance, due diligence, employment and taxation work, and mergers and acquisitions advisory particularly in infrastructure and inbound investment.

Large law firms identified the accountants as their main rivals in a recent Macquarie Group legal benchmarking survey.

Some law firms are quietly shifting work to boutique accounting firms because they refuse to be in bed with their emerging adversaries.

The Australian legal market liberalized several years, making it possible for nonlawyers to own and control legal enterprises.  In contrast, India has rules that are much closer to the U.S.  Yet, when it comes to the accounting firms, the official rules don't seem to matter much, as the competitive dynamics vis-a-vis big accounting firms in these two countries are very similar.  

A simple explanation is that bar authorities in any country are loath to pursue unauthorized practice of law actions when the clients are multinational corporations and the providers are large accounting firms.  That is too big a fight.  Further, the rules on unauthorized practice are in place to protect clients, not the guild.  Thus, it is not surprising that the accounting firms are getting bolder.  

 The chart below (from The Economist) put things into perspective:


See Attack of the Bean-Counters, Economist, Mar 21, 2015.

December 3, 2015 in Blog posts worth reading, Cross industry comparisons, Current events, Data on the profession, Law Firms, Structural change | Permalink | Comments (2)

Sunday, November 29, 2015

The UK's Apprenticeship Levy -- a helpful reset to the legal labor market?

LawsocietyThe Law Society Gazette reports a new apprenticeship levy that will be imposed in 2017 on UK employers with more than £3 million in payroll.  The Gazette notes that the levy, which totals .5% of payroll, will sweep in nearly 200 legal employers. 

The program has a potentially clever twist that could prove to be an effective economic stimulus for the UK economy. Employers get a credit for the cost of their current apprenticeship programs.  For the UK legal industry, this means that the bigger firms will be fully paid up just by running their current training contract programs. Yet, the article also notes that "the levy may force a number of firms to develop apprenticeship programmes so that they get their money back."

This is an idea that draws on both liberal and conservative principles.  It's liberal because it mandates, through a tax, a strong national policy that favors human capital creation. Yet, it's also conservative because it lets employers opt out of the tax by running their own apprenticeship programs.  The result is an increase in paid entry-level training for young people and, invariably, some infrastructure being developed around ongoing apprenticeship programs, likely from nonprofits and trade associations who serve or orbit around specific industries. 

In the United States, there are roughly 3,000 law firm employers with a payroll of $5 million or more.  They account for roughly half of the $91 billion annual payroll of all US law firms (NAICS 541110 Offices of Lawyers).  Many law firms are not hiring because client demand is sluggish and it's perceived as more cost effective to use senior personnel who are already trained.  As a result, the US legal profession is graying significantly.  See Is the Legal Profession Showing Its Age, LWB, Oct 12, 2014; What is Driving the Demographic Gap between BigLaw Leaders and their CEO/GC Clients?, LWB, Sept 1. 2015.  

Consider the benefits of a program like this operating in the US.  A .5% apprenticeship levy on $45 billion would mean that no less than $225 million per year would be invested in entry-level training contracts in the legal field, with a significant number of legal employers getting off the sidelines to create their own programs.  Astute bar associations would likely step in to provide logistical and administrative support.  Further, the US Department of Labor already has a detailed legal framework around apprenticeships.

With this kind of financial and administrative support, it is plausible to imagine the US legal profession moving to a true apprenticeship model where training contracts replace the 3L year of law school.  I acknowledge this all sounds very fanciful, but a relatively modest employer apprenticeship tax may be better national policy that asking young people to take on more education-related debt. 

November 29, 2015 in Current events, Innovations in law, New and Noteworthy, Structural change | Permalink | Comments (1)

Sunday, October 25, 2015

Is there a right way to respond to the "Law School Debt Crisis" Editorial?

Amidst all the other newsworthy topics, the New York Times editorial board made law school debt the lead editorial for today's Sunday edition. And the story line is not good.  

The editorial starts with the bleak statistics for Florida Coastal Law School -- low median LSAT scores and high debt loads, casting doubt on whether its graduates can pass the bar exam and repay their federally financed student loans.  The editorial highlights Florida Coastal' for-profit status but goes on to note that the rest of legal education is not much better. 

A majority of American law schools, which have nonprofit status, are increasingly engaging in such behavior, and in the process threatening the future of legal education.

Why? The most significant explanation is also the simplest — free money.

The editorial details changes in federal higher education finance that created the Direct PLUS Loan program, which, over-and-above Federal Stafford Loans, underwrites up to the full cost of attendance as determined by each law school.  The combination of poor job prospects and high debt have depressed applicant volume.  As the Times editorial notes, the systemic impact has been to lower admissions standards to sweep in students who will, as a group, struggle to pass the bar exam following graduation.  Virtually all of this is financed by DOE loan money.

I don't think the typical member of the legal academy understands the precarious financial condition of legal education.  The precariousness exists on two levels: (1) our financial fate is in the hands of the federal government rather than private markets; and (2) the Times editorial suggests that we have a serious appearance problem, which draws down the political capital needed to control our own destiny.  With the political winds so goes our budgets. 

I think it is important for the Association of American Law Schools (AALS) to take some decisive action in the very near future.  In this blog post, I explain where the money comes from to keep the law school doors open and why, as a consequence, we need to pay closer attention to the public image of legal education.  I then offer some unsolicited advice to the AALS leadership. 

(1) Who pays our bills?  

Over the last decade, the federal government has, as a practical matter, taken over the financing of higher ed, including legal education.  

Here is how it works.  Any law student who needs to borrow money to attend law school is strongly incentivized to borrow money from the Department of Education (DOE).  Although the DOE loans carry high interest rates -- 6.8% for Stafford Loans and 7.9% for Grad Plus -- they include built-in debt relief programs that functionally act as insurance policies for the risk that a graduate's income is insufficient to make timely loan repayments.  Law school financial aid offices are set up around this financial aid model and make it very easy for students to sign the loan documents, pay their tuition, and get disbursements for living expenses.

In the short to medium term, this is good for the federal government because the loans are viewed as income-producing assets in the budgets that get presented to and approved by Congress. But in the longer term this could backfire if a large portion of students fail to repay their full loans plus interest.  Federal government accounting rules don't require projections beyond ten years.  But already the government is beginning to see the size of the coming write-downs for the large number of graduates who are utilizing the Public Service Loan Forgiven program, which has a ten-year loan forgiveness horizon. And it is causing the feds to revise their budgets in ways that are politically painful.  With the loan forgiveness programs for private sector law grads operating on a 20- to 25-year repayment window, the magnitude of this problem will only grow.  

The enormous risk here for law schools is that Congress or the DOE will change this system of higher education finance.  For example, the Times editorial calls for capping the amount of federal loans that can be used to finance a law degree.  Currently, the limit on Stafford Loans for graduate education is $20,500, but Grad Plus loans have no limit at all.  If the DOE were to cap Grad Plus at $29,500 per year, leading to a total three-year federal outlay of $150,000 per law student, this would have an enormous adverse impact on the typical law school budget.

Law School Transparency reports that the average law school debt load for a 2014 law graduate is $118,570, but we know very little about the full distribution.  Because of the pervasiveness of the reverse Robin Hood policy, which uses tuition dollars of low credentialed students to finance scholarships for their high credentialed peers, there is likely a significant percentage of students at most law schools who graduate with more than $150,000 in law school debt.   Further, according to US News, there are twelve law schools -- including three in the T14 -- where the average law school debt load is more the $150,000.  Although there are no statistics on the percentage of law students graduating with greater than $200,000 in law school debt, law students tell me this amount is common. 

I have translated this meager public information into the chart below. The area in green is the volume of money that could disappear from law school budgets if the federal government imposed a hard limit on federally financed law school lending.


Why would this money be at grave risk?  Two reasons:

First, private lenders will be reluctant to cover the entire shortfall.  For decades, private lenders played an important roll in law school finance.  But these lenders got pushed out of the market by the changes in federal higher ed finances described above.  Unfortunately, in the intervening years, the ratio of earning-power-to-debt has gotten too far out of whack.  To come back into this market, private lenders would need to be confident that loans would be repaid.  That likelihood is going to vary by law school and by law student, raising the cost of lending.  This means that, to varying degrees, virtually all law schools would have to sweat over money.  Unlike Grad Plus, private lenders may balk at financing full sticker tuition for lower credentialed students trying to attend the highest ranked school that admitted them.

Second, private lenders will not offer the same loan forgiveness options, such as IBR and Public Service Loan Forgiveness, currently offered by the federal government.  With the curtailed scope of these functional insurance programs, some portion of prospective law students will likely be unwilling to sign loan documents in excess of the federal lending cap.  Even very elite schools will feel the pain here.

(2) An appearance problem in the world of politics

I would bet a lot of money that law faculty have been emailing the Times editorial to one another, criticizing its lack of nuance.  But here is our problem.  We are not in a court where a judge will listen to our elegant presentation of facts and law.  Nor are we in the world of private markets where we can expect people to reliably follow their own economic self-interest.  We are in the realm of politics where sides get drawn based on appearance and political expediency.  To make matters worse, the legal academy just got lambasted by the paper of record on the left.

It is hard to argue that a cap on federal funding of legal education would be bad policy for students, the legal profession, taxpayers, or broader society.  Such a change would:

  1. Reduce the number of law grads going into a saturated labor market;
  2. Reduce the number of low credentialed students admitted to law school who will one day struggle to pass the bar;
  3. Reduce the risk of nonpayment of students loans currently borne by US taxpayers;
  4. Put in place serious cost-containment on legal education.

For law schools, however, such a change would produce layoffs and pay reductions.  And that may be the fate of the luckier schools.   It is widely known that most law schools are running deficits.  Central universities are looking for ways to wait out the storm.  But the cliff-like quality of a federal cap on law school lending would call the question of how much support is too much.  

What's the solution?

Legal education has a cost problem, but so does the entire higher ed establishment. Here is my unsolicited advice.

The leadership of the AALS needs to take a very strong public position that the trend lines plaguing higher ed need to be reversed.  This is not risky because it is so painfully obvious.  The AALS should then, in conjunction with the ABA, send a very public delegation to the Dept of Education. The delegation should be given a very simple charge:  Help the DOE

  1. Outline the systemic problems that plague higher education 
  2. Articulate the importance of sound policy to the national interest
  3. Formulate a fair and sustainable solution. 

I have faith that my legal colleagues would do a masterful job solving the problems of higher education.  And in the process, we'll discover that we have become the architects of a new system of higher ed finance that will be fair and equitable system for all stakeholders, including those employed in legal education.  That's right: act decisively to ensure a fair and equitable deal.  The only drawback is that it won't be the status quo that we'd instinctively like to preserve. 

October 25, 2015 in Blog posts worth reading, Current events, Data on legal education | Permalink | Comments (25)

Tuesday, September 22, 2015

American Bar Foundation Research Professor

AjayFor those in the academy doing interdisciplinary work in the law & society area, being a Research Professor at the American Bar Foundation (ABF)  is very close to nirvana.  Moreover, my former Indiana colleague Ajay Mehrotra is the ABF's new Director.  Based on our 12 years of working together, I can attest that Ajay would be an outstanding mentor and boss, albeit this description also describes his predecessor, Bob Nelson.  The ABF is just a great place to do potentially high impact research. 

Below is the official announcement for an ABF Research Professor opening. These positions come open only rarely

American Bar Foundation Research Professor

Pending budgetary approval, the American Bar Foundation (ABF) invites applications to join its Residential Faculty as a Research Professor. Beginning in the 2016-17 academic year, the position is an ongoing one, subject to periodic performance reviews.

We seek earlier-stage candidates with a PhD and/or JD with the potential for exemplary scholarship in law and the social sciences. Research area, discipline, and methodology are open. The ABF is strongly committed to diversity in hiring.

The ABF is an independent, scholarly research institute committed to social science research on law, legal institutions, and legal processes. Its faculty consists of leading scholars in the fields of law, sociology, psychology, political science, economics, history, and anthropology.

Research professors work independently. They are responsible for identifying appropriate topics for research, seeking external funding when possible, conducting research, and authoring books and articles to be published in scholarly journals.

The ABF offers competitive salary and benefits along with research support. If jointly appointed with law or social science faculties of Chicago-area institutions, the ABF works closely with these institutions to coordinate on matters such as salary, benefits, and other work arrangements.

Review of applications will begin on November 15, 2015. We ask that applicants submit a letter of application, a curriculum vitae, a writing sample, a brief (no more than 2-page) description of current research and plans for future research, and a list of three references.

Application letters should be addressed to Ajay K. Mehrotra, Director, and sent in electronic form to Erin Watt, Executive Assistant, at with the subject line “Faculty Search.” Queries about the application process can be directed to Ms. Watt at (312) 988-6582.

The American Bar Foundation encourages diversity in its workforce and seeks to provide equality of opportunity for all applicants and employees. All persons are considered for positions on the basis of job-related requirements. All decisions regarding recruiting, hiring, promotion, assignment, training, termination, and other terms and conditions of employment will be made without unlawful discrimination on the basis of race, color, national origin, ancestry, sex, sexual orientation, gender identity or expression, religion, age, disability, veteran status, pregnancy, or marital status, in accordance with the ABF’s commitment to equal opportunity and all governing laws.

September 22, 2015 in Current events, New and Noteworthy | Permalink | Comments (0)

Wednesday, September 16, 2015

In India, Big 4 and Elite Law Firms in Direct Competition for Highly Lucrative Advisory Work

Lawyers may have a monopoly over the practice of law, but what exactly does the practice of law encompass?  In most common law jurisdictions, the term is not even defined.  And there's likely a self-interested reason why.  Ambiguity produces uncertainty, and uncertainty is a major source of business risk.

In the face of significant uncertainty, why would investors fund a business that encroaches on lawyers' most lucrative work when they'll have to hire a battalion of lawyers to defeat the entire universe of lawyers in front of a judge who used to be a practicing lawyer (and may be one again)? Many lawyers would get rich losing this case for you.

Well, ambiguity may not be enough to permanently fend off the invasion.  There is a controversy taking shape in India that may foreshadow the end of the lawyer guild.  Under India's Advocates Act, only lawyers can own a law firm. Likewise, only chartered accountants can audit companies.  But what about advisory services related to business?  That's a area of tremendous potential overlap between these two professions.

Times-of-india-logoGlobally, the Big 4 have been making inroads on lucrative cross-border deal work -- not enough to mortally threaten the major law firms, but enough to get their attention.  Yet, according to this story in the The Economics Times (the leading business paper in India), it's the elite law firms putting the hurt on the Big 4, poaching talent in some of the Big 4's most lucrative practice areas.  Here's how the Indian journalists tell it:

MUMBAI: Two months after the Delhi Bar Council sent a legal notice to top professional services firms EY, KPMG, PwC and Deloitte, the simmering hostility between the Big 4 and the legal fraternity is increasingly coming out in the open.

Earlier this month, Zia Mody's AZB & Partners [the Wachtell Lipton of India] scooped up six forensic experts from EY, a move that has been seen by industry players as part of growing competition between leading law firms and consultancies for business, encroaching into each other's traditional bastions. ...

[Zia Mody, the managing partner of AZB, commented,] "The compliance and investigation practice that we have formalised would help us in due diligence, Indian anti-corruption law investigations, asset-recovery cases, private equity-related post investment investigations and in some anti-trust investigation cases."

Top legal firms in the country are diversifying into forensic operations and undertaking commercial diligence and investigations for their clients.

This comes at a time when consultancies like EY and KPMG are bulking up their teams beyond the traditional forte of audit and tax practices to expand into advisory services. ... 

"What law firms are doing is part of their evolution into full service providers," said Lalit Bhasin, president of the Society of Indian Law Firms, which in July filed a complaint against the Big 4 with the Bar Association of India, saying that they were practicing law without authorisation.

Side note on Lalit Bhasin -- he is the most prominent spokesperson for keeping in place the longstanding prohibition against foreign law firms operating inside India.  He is obviously not too keen on Big 4 accounting firms getting into his business.  The Times quotes a senior partner of a Big 4 firm, speaking on a condition of anonymity,

"The genesis of the problem is: lawyers are a close-knit community and if you don't belong to that club, you don't get lucrative work.  ... Now, the consultancies are increasingly getting their foot in the door, and doing the work at much lower price for which the lawyers would charge a bomb."

The story notes that the law firms and accounting firm are now competing to their mutual detriment, albeit the clients are unlikely complaining:

Consultancies and law firms are competing with each other in several other areas as well, and this is hurting the bottom lines of both sides as undercutting of fees has become rampant. "They (consultants) do an investment banking deal for a mere Rs 5 lakh [~$8000 US], for which a good law firm would charge around Rs 60 lakh [$90,000 US]," said the managing partner of a New Delhi-based big law firm.

Forensic is one of the most profitable business verticals for the Big 4. The total pie for forensics in India is estimated to be around .`850 crore [~$160 million] and is growing at 15-20 per cent year on year, industry experts said. Competition from lawyers, therefore, hurts the big auditors who among themselves control about 80 per cent of the market in India. Smaller rivals such as Alvarez & Marsal, FTI and Kroll [two US-based companies] control the rest.

Suffice it to say, the Big 4 aren't afraid to fight the elite bar in virtually any jurisdiction, as they have deep pockets and a large number of lawyers on their payroll, all of whom, we can be quite sure, are not engaged in the practice of law, whatever that term means.  

This is the beginning of the end of an era, and that's a good thing.  We lawyers/legal professionals will reinvent ourselves by finding new ways to add value.  In fact, that is already happening.  In the long run, we'll feel richer for it. Below is an infographic from The Times story that summarizes the Indian lawyer-accountant standoff.


September 16, 2015 in Cross industry comparisons, Current events, New and Noteworthy, Structural change | Permalink | Comments (0)

Tuesday, September 1, 2015

What's driving the demographic gap between BigLaw leaders and their CEO/GC clients?

Picture1Las Vegas, NV.  The illustration to the left was just published in The American Lawyer.  It accompanied a story on how law firm leaders are significantly older than leaders in the large corporations they serve. See MP McQueen, The Generation Gap:  BigLaw's Aging Leaders, Aug 24, 2015. 

At least for me, this is a jarring graphic because it conveys so much truth.  Today's Millennials are so underwhelmed with the BigLaw model.  They like the pay and the perks, as it enables them to live well in attractive large market cities.  They can also quickly pay off their law school debt.  But precious few of them are all in. Illustrator James Steinberg totally nailed it.

There are numerous reasons for the culture divide, but as shown in the chart below, the most obvious is a very large age gap between leaders and entry-level workers -- it tends to be a lot larger in BigLaw than almost anywhere else: 4% of AmLaw 100 leaders are Gen X compared to 33% of NASDAQ-traded companies. 

These data beg the question, why are large law firms so out-of-sync with the institutions they serve?

One reason is certainly the ownership structure.  Any Fortune 500 or NASDAQ-traded company that got this top-heavy in its senior management would be getting killed on its stock price.  Under the Rule 5.4 prohibition on nonlawyer investors, law firms are spared the anxiety of having analysts and short sellers constantly evaluating their business. Yet, the absence of a public market means that law firm owners and managers cannot fully monetize the enterprise value they create. So what's the effect?  Very little enterprise value gets created.  Instead, lawyer/owners  focus on maximizing this year's net distributable income.

It is important to not knock the BigLaw model too hard.  For about a century, it worked extremely well, as US law firms steadily grew with their clients.  Each unit of economic growth produced some larger unit increase in legal complexity, so demand for sophisticated legal services was a steady upward sloping line. By following a simple model -- hire more associates, promote some to partners, lease more office space, repeat -- equity partners in the AmLaw 100 became millionaires.  

Today, BigLaw is getting grayer because the 100-year old gold factory is breaking down. Law firms' portion of corporate legal spending is no longer growing, as in-house lawyers, NewLaw managed services shops (United Lex, Axiom, Counsel on Call), and technology are all curbing demand for traditional law firm services.   The best economic play for 55- or 60-year old equity partner is to ride out the existing model with the dwindling but still substantial number of Baby Boomer senior in-house lawyers who are themselves not too anxious to change.  

This is not the story equity partners tell themselves; it's the logic that underlies the inertial path.  It's where we end up when we are no longer deeply invested in the places we work.  It's become a job.  I am not judging here; I'm describing what I have observed through hundreds of conversations with large firm partners.

The result of this dynamic is that a large proportion of BigLaw--but certainly not all of it--is just tinkering at the margins of change. A law firm can become more cost-effective for clients, at least in the short to medium term, by reducing reliance on associates.  Associates are expensive and are, by definition, getting paid to learn.  For the last 15-20 years, firms have shifted their leverage model to counsel, staff attorneys and nonequity partners, where (a) there is little to no training, (b) the margins are higher, and (c) the clients can't complain about inefficient associates. This is the Diamond Model, which substantially cuts out the entry-level lawyer.  See The Diamond Law Firm: A New Model or the Pyramid Unraveling? (Dec. 2013); Sea Change in the Legal Market, NALP Bulletin, Aug. 2013.


Unlike the original Pyramid Model, invented by Paul Cravath circa 1910, the Diamond Model is not a carefully conceived business strategy. Rather, it's a way to maximize this year's and next year's net distributable income without making difficult strategic tradeoffs. Yet, in the longer term, which is no longer too far off, the Diamond Model is a disaster.  The few associates who make it into large firms are grateful for the high pay and the training.  But very few if any are impressed with the business model.  Among Millennial lawyers, in-house is the new brass ring.

Law firms are filled with brilliant people. Why are they going down this road?  Three interrelated reasons:

  1. Lack of Experience. Today's law firm partners have little or no experience with strategy--for a hundred years, intelligence and hard work worked just fine.  This is not a change in strategy--it is having a strategy.  Then executing.  That's hard.
  2. Incentive Structures. Virtually all incentives inside firms today favor revenue generation; as a result, few partners have the mental whitespace to understand, much less think through, the changes that are occurring within the broader industry.  To fix the bridge, you have to slow down the traffic.
  3. To Big to Fix.  The first strategy mistake for the current generation of AmLaw 100 leaders was to become bigger without becoming measurably better.  Big firms filled with laterals is a difficult environment to share risk. Maximizing this year's distributable income becomes one of the few things people can agree on.

That said, I am not counting BigLaw out.  I am writing this blog post from the International Legal Technology Association (ILTA) conference in Las Vegas.  From far away, it is all too easy to treat BigLaw as a monolith--it's not.  At ILTA, professionals from several of the most innovative law firms are willing to pop the hood and share what they doing.  See Ahead of the Curve: Three Big Innovators in BigLaw, Aug. 25, 2014Suffice it to say, some firms are several years into strategies that have the potential to take market share from peer firms.  Further, the innovation teams inside these firms are having the time of their professional lives because the work is so collaborative and creative--the antithesis of billable hour work.  What is also clear is that many competitors just can't muster the leadership nerve to make similar investments. 

In the years to come, some BigLaw firms are going to pull away from the rest, becoming a magnet for talent and then clients.  Younger lawyers are going to thrive there.  Another portion of BigLaw is going to gradually fade away. 

September 1, 2015 in Current events, Data on the profession, Law Firms, Structural change | Permalink | Comments (4)

Thursday, August 6, 2015

How is the entry-level legal job market in Australia?

AlsaNot good.  There are more law graduates than jobs, yet law schools are making matters worse by admitting more students in order to generate subsidies for other parts of the university. That the basic charge of the Australian Law Students Association (ALSA), according to this story in the Lawyers Weekly, a publication that covers the business of law in Australia.

Legal education is Australia is very different than the U.S.,  yet the dynamics of the two entry-level markets seem to be converging.  Law has historically been an undergraduate degree in Australia (LLB), but in recent years the JD has been added as a new and more prestigious way into the profession. Here is the statement of an ALSA spokesperson based on recent survey results of the ALSA membership.

ALSA are of the position that there is still an oversupply of graduates because of the increasing sizes of law schools and the duplication in the number of law schools across the country. ...

Many who have undertaken the Juris Doctor particularly expressed concerns in their survey responses, highlighting that they undertook the postgraduate law degree to further their job prospects. Instead, they are facing the worrying reality that there are fewer jobs available for law graduates as well as the fact that they are completing their degrees with a sizeable student debt.

The article then goes on to describe growing law student anxiety over employment and student loan debt.  Wow, different system but a very similar result.  

One of the advantages of the Australian LLB degree is that it is often combined with another undergraduate degree, typically by adding one year of additional study.  As a result, many LLBs don't go on to qualify for practice, but the legal training probably augments their worldly knowledge and critical thinking skills.  But alas, the Australians are starting to dilute their extremely generous higher education subsidies -- we are just much further down that road. Further, the true undercurrent here is the growing insecurity facing virtually all knowledge workers, Australian or US.  Legal education is just the bleeding edge of this problem.

August 6, 2015 in Current events, Data on legal education, Data on the profession, New and Noteworthy | Permalink | Comments (0)

Tuesday, August 4, 2015

Metrics and Legal Ops Professionals

In a recent post, I urged readers to visit a legal department with a large legal operations staff.   The goal?  To see the future of modern corporate law practice.  Fortunately, Bloomberg Law recently videotaped a legal ops panel moderated by Amar Sarwal of the ACC.  It contains a conversation rarely if ever heard in law schools or bar associations.

The three legal departments profiled are AIG (insurance), Marsh & McLennan (diversified financial and professional services), and GlaxoSmithKline (pharma).  Note the enormous emphasis on metrics, data, and technology.  Note also how the services of law firms are being put through a procurement process. 

August 4, 2015 in Blog posts worth reading, Current events, Data on the profession, Law Firms, Legal Departments, New and Noteworthy, Video interviews | Permalink | Comments (0)

Saturday, August 1, 2015

Legal Analytics to Build Substantive Legal Strategy

RavellexmachinaLex Machina and Ravel Law are two start-ups in the legal analytics space (I call companies like this "toolmakers"). Per an invite that arrived in my email yesterday, these Lex Machina and Ravel are collaborating on an upcoming webinar that shows lawyers how to use legal analytics to build a substantive legal strategy.  And not just any strategy, but one more likely to win.

Is this smoke and mirrors or the real deal? That remains an open empirical question.  But it's worth noting the venture capitalists who backed these companies are betting on the proposition that humans + machines > humans or machine. (This is Dan Katz's formulation.) For lawyers interested in staying current on the legal market (most of us, right?), it's likely worth attending.   Sure, Lex Machina and Ravel will be hawking their own products, but hey, educating your prospective clients is how one builds a market for complex technical products.  This skill will eventually be taught in law schools -- at least the good ones.  It is worth noting that both Lex Machina and Ravel trace their origins back to Stanford Law.  

Here is the essential information on the webinar.

How can you employ Legal Analytics in your practice to get clients and win cases?

New legal technologies are transforming the practice of law by enabling lawyers to uncover trends in the behavior of judges, parties, law firms and attorneys.

Ravel Law's case law analytics and Lex Machina's IP docket analytics enable lawyers to make data-driven decisions about case strategy and tactics.

Click here to register for a live 45-minute webinar on Thursday, August 6th at 11:00 am PT.

Host: Ralph Baxter, Advisor, Writer, and Speaker, Former Chair, Orrick 

Speakers: Owen Byrd, Chief Evangelist & General Counsel, Lex Machina & Daniel Lewis, CEO & Co-Founder, Ravel Law

Related posts:

August 1, 2015 in Current events, Innovations in law, New and Noteworthy | Permalink | Comments (0)

Wednesday, July 29, 2015

"Solicitors 'in denial' about threat from accountants"

Legalservices (321x207)That's the headline from today's Law Society Gazette, the publication of record for solicitors in England and Wales.  The UK is fairly far along in liberalization of its legal markets, progressing from the Clementi Report in 2004 to the Legal Services Act 2007 to the licensing of Alternative Business Structures in 2012.  Now several hundred entities have obtained ABS status.  

The Gazette article reports that accountants are poised to be large players in the ABA space:

Accountants will soon be competing directly with solicitor firms ‘on every high street in the country’, according to a leading financial advisor to the legal sector.

Ian Muirhead, chairman of Solicitors Independent Financial Advice, said he expects 750 accountancy firms – three times more than first envisaged – to move into probate work after securing an alternative business structure licence.

The Institute of Chartered Accountants in England & Wales has accredited 113 entities as an ABS since last October, having been accepted as an approved regulator almost a year ago. A further 34 applications are being processed.

Speaking at a Westminster Legal Policy, Muirhead said too many solicitor firms are ‘in denial’ about the threat from the accountancy profession.

‘Success will go to those who can manage businesses and I query whether that’s going to be the solicitors or whether solicitors are going to be the back room boys,’ he said.

Muirhead argued that law firms’ response so far has been focused on consolidation, mergers and acquisitions – but this risks playing into rivals’ hands.

‘[The response is] safety in numbers, more of the same, not thinking outside the legal silo, and therefore missing the opportunity of which many new ABSs are availing themselves, of providing a more diversified and holistic client service,’ he added. ... 

Some U.S. lawyers believe that liberalization won't come to the U.S. because the legal industry is too balkanized by state bar authorities.  

I think this view, however, is likely naive. The market can change because regulators change the rules (the UK). Alternatively, the market can change because clients change their buying habits in favor of nontraditional legal service providers that are financed by sophisticated nonlawyer investors (the US).  See, e.g., Is Axiom the Bellwether for Disruption in the Legal Industry, LWB, Nov. 10, 2013.

In the US, it is probably true that regulators lack the stomach to initiate a regulatory action where the client ostensibly being protected is a Fortune 500 corporation.  If the action ends up in federal court, the bar officials risk looking like protectors of the guild and have a decent chance of losing.  The prohibition against nonlawyer investment (MR 5.4) is based on the assumption that the nonlawyer profit motive will compromise lawyer independence, thus harming the unwitting and unsophisticated legal consumer.  But that does not describe IBM's or JP Morgan's relationships with sophisticated LPO or analytics shop (or any general counsel charged with stretching his or her legal dollar). As a result, the venture capital money flows in.

When liberalization is viewed in this light, there are probably more similarities between the US and UK than we might want to acknowledge. 

July 29, 2015 in Current events, Data on the profession, Innovations in law, Law Firms, New and Noteworthy, Structural change | Permalink | Comments (2)

Thursday, May 14, 2015

Further Thoughts on the July 2014 Bar Results -- A Response to Erica Moeser

Late last fall, Erica Moeser responded to a letter from Dean Kathryn Rand of the University of North Dakota (on behalf of a large number of law school deans), reiterating that the NCBE had double-checked its scoring of the MBE on the July 2014 bar examination and could find no errors in its calculations.  Erica Moeser also took to the pages of the December 2014 issue of The Bar Examiner to further validate her conclusion that the historic drop in the mean MBE scaled score is attributable solely to the fact that the class that sat for the July 2014 bar exam was “less able” than the class that sat for the July 2013 bar exam.  In January, Dean Stephen Ferruolo of the University of San Diego also wrote to Erica Moeser requesting the release of more information on which to assess the July 2014 bar examination results in comparison with previous years’ results.  In February, Erica Moeser responded to Dean Ferruolo’s request by declining to provide more detailed information and reiterating her belief that the July 2014 scores “represent the first phase of results reflecting the dramatic and continuing downturn in law school applications.”

In an earlier blog posting, I explained why Erica Moeser is partly right (that the Class of 2014 could be understood to be slightly less able than the Class of 2013), but also explained why the decline in “quality” of the Class of 2014 does not explain the historic drop in mean MBE scaled score.  The decline in “quality” between the Class of 2013 and the Class of 2014 was modest, not historic, and would suggest that the decline in the mean MBE scaled score also should have been modest, rather than historic.  Similar declines in “quality” in the 2000s resulted in only modest declines in the MBE, suggesting that more was going on with the July 2014 exam. 

Others have written about these issues as well.  In January, Vikram Amar had a thoughtful reflection on Moeser’s statements and in recent weeks Debby Merritt has written a series of posts -- here, here, and here -- indicating in some detail why she believes, as I do, that the ExamSoft debacle in July could have impacted the MBE scaled scores in jurisdictions that used ExamSoft as well as in other jurisdictions.

I write now to take issue with four statements from Erica Moeser – three from her President’s Page in the December 2014 issue of the Bar Examiner and one from her letter responding to Dean Kathryn Rand.  I remain unpersuaded that the historic decline in the mean MBE scaled score is solely attributable to a decline in quality of the class that sat for the July 2014 bar examination and remain baffled that the NCBE refuses to acknowledge the possibility that issues with test administration may have exacerbated the decline in the performance on the July 2014 MBE.

Item One – Differential Declines in MBE Scores

In her December article, Moeser stated: 

I then looked to two areas for further corrobo­ration. The first was internal to NCBE. Among the things I learned was that whereas the scores of those we know to be retaking the MBE dropped by 1.7 points, the score drop for those we believe to be first-time takers dropped by 2.7 points. (19% of July 2014 test takers were repeaters, and 65% were believed to be first-time takers. The remaining 16% could not be tracked because they tested in jurisdictions that col­lect inadequate data on the MBE answer sheets.) The decline for retakers was not atypical; however, the decline for first-time takers was without precedent dur­ing the previous 10 years. (Emphasis in original.)

Moeser starts by referencing data that is not publicly available to support her cause.  This is unfortunate, because it makes it really hard to understand and critique the data.  Nevertheless, there are some inferences we can take from what she does disclose and some questions we can ask.  Moeser asserts that the 19% of MBE “retakers” saw an MBE drop of 1.7 points compared with MBE “retakers” in July 2013, while the 65% believed to be first-time takers saw a drop of 2.7 points compared with first-time takers in July 2013.  It would have been helpful here if Erica Moeser would have released publicly the declines among MBE retakers in the previous 10 years and the declines among first-time takers in the previous 10 years so that patterns could be assessed, particularly in relation to the changes in class composition for each of those years.  Without that information available it is hard to do much more with Moeser’s assertion.  (I find it odd that she would reference this point without providing the underlying data.) 

Nonetheless, this assertion raises other questions.  First, the overall decline in the mean MBE scaled score was 2.8 points. Moeser notes that 19% of takers (MBE retakers) had an average drop of 1.7 points, while 65% of takers (first-time takers) had an average drop of 2.7 points.  Unless there is something I am missing here, that should mean the remaining 16% of test-takers had to have an average decline of 4.51 points!  (This 16% of test-takers represents those who Moeser notes could not be tracked as first-time takers or MBE retakers “because they tested in jurisdictions that collect inadequate data on the MBE answer sheets.”) (Here is the equation --- 2.8 = (.19*1.7)+(.65*2.7)+(.16*x).  Solve for X. This translates to 2.8 = .323+1.755+.16x.  This translates to .722 = .16x and then .722/.16 = X.  X then equals 4.51.)  It would have helped, again, if Moeser had indicated which jurisdictions had these even larger declines in mean MBE scaled scores, as we could then look at the composition of graduates taking the bar in those jurisdictions to see if there was an unusual decline in entering class statistics in 2011 at the law schools from which most bar takers in those states graduated.

Item Two – The MPRE

In the December article, Moeser also stated:

I also looked at what the results from the Multistate Professional Responsibility Examination (MPRE), separately administered three times each year, might tell me. The decline in MPRE performance supports what we saw in the July 2014 MBE numbers. In 2012, 66,499 candidates generated a mean score of 97.57 (on a 50–150 scale). In 2013, 62,674 candidates generated a mean score of 95.65. In 2014, a total of 60,546 candi­dates generated a mean score of 93.57. Because many MPRE test takers are still enrolled in law school when they test, these scores can be seen as presaging MBE performance in 2014 and 2015.

At first blush, this looks like a pretty compelling argument, but Moeser’s selectiveness in looking at the data is troubling, and her failure to discuss whether the MPRE and MBE are meaningfully comparable test-taking experiences also is troubling.  Essentially, Moeser is making the following assertion – because the mean MPRE scaled score declined by 1.92 points between 2012 and 2013, we should have expected a large decline in the mean MBE scaled score in July 2014 (and because the mean MPRE scaled score declined another 2.08 points between 2013 and 2014, we should expect another large decline in the mean MBE scaled score in July 2015).

But the “relationship” between changes in the mean MPRE scaled score and changes in the mean MBE scaled score over the last decade does not support this assertion. If one looks at a decade’s worth of data, rather than data just for the last couple of years, the picture looks significantly more complicated, and suggests the collective performance on the MPRE may not tell us much at all about likely collective performance on the MBE in the following year. 


Mean MPRE Score


MBE Year

July Mean MBE Scaled Score




































































The data Moeser cites from the last two years conveniently makes her point, but it consists of a very small sample size.  The data over the last decade looks much more random.  In three of the nine years, the change is not in the same direction (MPRE 2005, 2006, 2010, MBE 2006, 2007, 2011).  In the six years where the change is in the same direction, there are two years in which the MBE change is significantly larger than the MPRE change (MPRE 2007, 2009, MBE 2008, 2010) and there are two years in which the MBE change is significantly smaller than the MBE change (MPRE 2011, 2012, MBE 2012, 2013).  In only two of the nine years, do the changes in the MPRE and MBE roughly approximate each other (MPRE 2008, 2013, MBE 2009, 2014).   Nonetheless, this remains a very small sample and more analysis of data over a longer period might be helpful to better understand how/whether changes in mean MPRE scores inform meaningfully changes in mean MBE scores the following year.  At this point, I think the predictive value seems marginal given the wide range of changes on a year-over-year basis.

Item Three – Mean LSAT Scores

In the December article, Moeser further stated:

Specifically, I looked at what happened to the overall mean LSAT score as reported by the Law School Admission Council for the first-year matricu­lants between 2010 (the class of 2013) and 2011 (the class of 2014). The reported mean dropped a modest amount for those completing the first year (from 157.7 to 157.4). What is unknown is the extent to which the effect of a change to reporting LSAT scores (from the average of all scores to the highest score earned) has offset what would otherwise have been a greater drop. (LSAC Research Reports indicate that roughly 30% of LSAT takers are repeaters and that this num­ber has increased in recent years.

This assertion is misguided for purposes of this comparison, a point Vikram Amar made in his post.  If we were comparing the first-year matriculants in 2009 with the first-year matriculants in 2010, the question of the change in reporting from average LSAT score to highest LSAT score would have mattered.  But the 2010 matriculants were the first class for which the mean was reported based on highest LSAT score and the 2011 matriculants were the second class for which the mean was reported based on highest LSAT score.  Thus, there is no “unknown” here.  The reported mean LSAT dropped only a modest amount between the matriculants in 2010 and the matriculants in 2011.  Nonetheless, the mean MBE scaled score in July 2014 decreased by an historic 2.8 points from the mean MBE scaled score in July 2013. 

Item Four – Administration Issues

In her letter to Dean Kathryn Rand, Moeser stated:  "To the extent that the statement you attached referenced both administration and scoring of the July 2014, bar examination, note that NCBE does not administer the exam; jurisdictions do."

This response suggests not only that the NCBE is not responsible for administering the bar examinations in the many different jurisdictions, but implicitly suggests that issues with administration could not have contributed to the historic decline in the mean MBE scaled score. 

Were there issues with administration?  Yes.   Could they have contributed to the historic decline in the mean MBE scaled score?  Yes.

Debby Merritt’s recent posts discuss the administration issues and the potential consequences of the administration issues in some detail.  In over forty states that used ExamSoft to administer the bar examination, the MBE came on Wednesday, after the essay portion of the exam on Tuesday.  But because of an ExamSoft technical problem, tens of thousands of test-takers, who were initially informed by their respective state board of bar examiners that they would FAIL THE EXAM if their essay answers were not uploaded in a timely manner, spent most of Tuesday night dealing with the profound stress of not being able to upload their exam answers and not being able to contact anyone at the board of bar examiners (who were not answering phones) or at ExamSoft (due to the flood of calls and emails from anxious, frustrated, stressed out exam takers) to figure out what was going on and what they should do. 

Given that this “administration” issue caused untold stress and anxiety for thousands of test-takers, who spent Tuesday night completely anxious and stressed out trying repeatedly and unsuccessfully to upload their essay answers, should it be a surprise that they might have underperformed somewhat on the MBE on Wednesday?  (If you want a sense of the stress and anxiety, check the twitter feed for the evening of Tuesday, July 29, 2014)

The responses from the boards of bar examiners to this issue with administration of the bar examination were far from uniform.  Different jurisdictions granted extensions at different times of the night on Tuesday, July 29, or on Wednesday, July 30, with some granting short extensions and some granting longer extensions.  Thus, in states that gave notice of an extension out earlier on Tuesday, July 29, test-takers may have had less stress and anxiety, while in those states that didn’t give notice of an extension out until later (or for which the extension was relatively short), or where there may not have been any communication regarding extensions of the submission deadline, test takers likely experienced more stress and anxiety.  (It would be worth studying exactly when each jurisdiction gave notice of an extension and whether there is any correlation between timing of notice of the extension and the relative performance of bar takers in those states.)

The NCBE’s unwillingness to acknowledge any issues with administration of the bar examination is all the more surprising at a time when the NCBE is pushing for adoption of the Uniform Bar Examination.  On its webpage, the NCBE states: “[The UBE] is uniformly administered, graded, and scored by user jurisdictions and results in a portable score that can be transferred to other UBE jurisdictions.” (Emphasis added.)  This simply was not true in July 2014.  The Uniform Bar Examination was administered under different exam conditions across jurisdictions.  First, three of the states administering the Uniform Bar Examination in July 2014 did not use ExamSoft – Arizona, Nebraska and Wyoming -- and therefore, bar takers in those states had a vastly different “exam administration” experience than bar takers in ExamSoft jurisdictions.  Across ExamSoft jurisdictions, different approaches to extensions also meant different administration experiences. Given the significance of consistent administration for the purpose of equating performance on a standardized exam like the bar exam, that the NCBE allows such varied approaches to administering a supposedly “uniform” exam strikes me as very problematic.

Many questions remain unanswered, largely because adequate information has not been made available on which to assess the various factors that might have contributed to the historic decline in the mean MBE scaled score.  With the release of February bar results and the NCBE’s publication of the 2014 statistical report, some additional information is now available to put the results of July 2014 in context.  In my next blog posting regarding the July 2014 bar results, I will delve into some of those statistics to see what they tell us.

(Edited as of May 20 to correct the 2013 MPRE and 2014 MBE change and corresponding discussion.)

May 14, 2015 in Current events, Data on legal education, Data on the profession | Permalink | Comments (0)