Sunday, November 22, 2015

What is the impact of longer hours on lawyer satisfaction?

Every lawyer, law professor, law student, and legal commentator has an opinion on this question. Today we can test our views against actual data. 

IndianaLawyerThis fall, Lawyer Metrics was given the opportunity to analyze survey data supplied to us from by The Indiana Lawyer, the paper of record for the Indiana legal profession.  The sample included 516 respondents drawn from the paper's readership.  My colleague at Lawyer Metrics, Evan Parker, sliced and diced the data in a way the gave us some useful insights into the hours/satisfaction question, at least for a broad swath of lawyers in one midwestern state.

Below is a graphic that shows the average level of satisfaction on various dimensions for Indiana lawyers working 60+ hours per week.  


After the jump are graphs showing averages for lawyers working fewer hours per week.  But before clicking on the hyperlink, answer this question: Are Indiana lawyers with more moderate schedules on average more satisfied or less satisfied than their 60+ hour counterparts? Also, be a good sport and write down your reason why.  

Continue reading

November 22, 2015 in Blog posts worth reading, Data on the profession, Fun and Learning in the classroom | Permalink | Comments (0)

Monday, November 9, 2015

Part IV: Alumni Surveys, The Varied Career Paths of Law School Graduates

This is Part IV of a blog series that focuses on alumni surveys based on data for Northeastern Law alumni who graduated between 1971 and 2012 (n = 833, 21% response rate).  Prior posts covered data related to the pre-law (Part II) and law school (Part III) experience.  This final installment summarizes data on the careers of Northeastern alumni. 

Varied Careers

One of the most significant post-law school findings from the Northeastern alumni survey is the sheer breadth of careers.  Sure, we all know in a general sense that lawyers have very diverse careers, yet I found the sheer magnitude of that diversity both striking and surprising.

Below is a graphic that summarizes the percentage of Northeastern alumni who have worked in a particular practice settings,by decade of graduation.

% Alumni/ae who have worked in Practice Setting, by Decade of Graduation


To interpret this graphic [click on to enlarge], it is important to understand the composition of the underlying data.  The survey question asks, “Describe your previous employment history starting with your most recent employer first.”  Some graduates have only one job to report -- the one they started after graduation; others have had many.  These jobs are then classified by practice setting and binned into the six categories shown in the above graphic.  Note that bars total well beyond 100%. Why?  Because alumni are changing not just jobs, but also practice settings—on average, at least once, but sometimes two, three, or even four times over the course of several decades.

The graphic above conveys several significant pieces of information:

General point.  Legal careers are extremely varied.  As it has tightened up, the entry level market has become an area of intense scrutiny, and rightly so because it affects early career lawyers and law school applicant volume.  In contrast, the chart above reflects the longer view. It suggests that very able, motivated people who attend law school go on to varied careers that no one could have predicted at the time of enrollment, including--most significantly--the entering student.  These generational cohorts are a versatile group that comprise a disproportionate number of leaders in industry, government, and the nonprofit world.  Law schools cannot take full credit for this; we admit people of enormous potential.  Yet many alumni tell me that their legal training and knowledge has given them an enormous leg up. One law grad who is now a successful business executive recently asked me, "Why is it JD-advantaged? Why not the advantage of the JD?" 

Northeastern.  It is somewhat surprising that for Northeastern alumni who graduated during the 1970s, 80s, and 90s, 48% have worked in government.  That is a big number.  Northeastern’s mission and faculty emphasize public service. This same emphasis appears to be reflected in the careers of its graduates.

Changing Legal Ecosystem.  As noted in Posts II and III, because the Northeastern alumni survey spans multiple decades, it is possible that responses will be influenced by changes in the underlying legal economy. Stated simply, career opportunities and competition may have changed substantially between 1971 and 2012.  Such a pattern appears to be present here.  Specifically, 30% or more of graduates of the 1990s and 2000s have worked in private industry compared to 24% or less for those graduating in the 1970s and 80s.  This would be consistent with the incomplete absorption theory discussed in Part III.  See also Henderson, “Is the Legal Profession Showing its Age,” LWB, Oct 12, 2015.

Practicing versus Non-Practicing Lawyers

Another significant finding that flows from the Northeastern alumni survey are the workplace experiences of practicing versus non-practicing lawyers. 

Approximately 25% of respondents were not practicing lawyers but working, with no significant difference by decade cohort. The chart below compares these two groups based on 19 dimensions of workplace satisfaction. The question is drawn directly from the AJD Wave III:  “How satisfied are you with the following aspects of your current position?” 

Dimensions of Workplace Satisfaction, Practicing vs. Non-Practicing Lawyer


Choices ranged from 1 (highly dissatisfied) to 7 (highly satisfied).  The chart above summarizes the differential between the two groups.  For example, on Intellectual Challenge, we subtracted the non-practicing attorney average from the practicing attorney average.  The result is +.35 difference for practicing attorneys, meaning that they are more likely to find intellectual challenge in their work.  Likewise, the same results holds for the substance of one's work.  

In contrast, on workplace diversity, non-practicing lawyers were significantly more satisfied – on average, roughly 2/3 of a response point.  In fact, non-practicing lawyers were more likely to rate their workplaces higher on several surprising factors, including social value of work, performance reviews, work/life balance, and pro bono opportunities.

Can we generalize from these findings?

The results presented in this blog series reflect the collective experience of one law school’s alumni base – Northeastern.  There is no way to know if these results can be fairly generalized to the larger law graduate population, though there is a reasonable basis to believe that at least some of them can (e.g., the changing ecology of the legal job economy).  Yet, why speculate when the cost of collecting and analyzing the data is going down and the value of such applied research is going up?

AbfLet me reiterate my suggestion from Part I that a consortium of law schools should begin this effort under the aegis of the American Bar Foundation (the prime architect of the AJD Project).  Northeastern has agreed to donate the survey and research tools we created as part of the Outcomes Assessment Project.   Such an initiative would enable researchers to draw stronger conclusions from these data, including potentially laudatory school-level effects that can help the rest of legal education. 

I have been researching legal education for many years.  I have spent enough time with alumni at Indiana Law, Northeastern Law, and several other law schools to gain a strong impression that law school graduates are having, on balance, important, satisfying and high-impact careers.  Further, there is strong evidence that the legal industry is undergoing a significant structural change – that is much of what the Legal Whiteboard catalogs.  This structural change topic is of great interest to prospective students, lawyers, and the mainstream press.  Yet, these two themes--the careers of alumni and structural change--are related. 

If legal education wants to influence the narrative on the value of the JD degree, it is far better to rely on data rather than rhetoric.  My sense is that data on our alumni will tell a rich, balanced story that will enable us to make better decisions for all stakeholders, including prospective law students. Further, if we don’t gather high quality facts, we can expect to get outflanked by a blogosphere and a mainstream press that are armed with little more than anecdotes.  To a large extent, that is already happening.  Now is the time to catch up.


This blog post series would not have been possible without the dedication and world-class expertise of my colleague, Evan Parker PhD, Director of Analytics at Lawyer Metrics.  Evan generated all the graphics for the Northeastern Alumni/ae Survey and was indispensable in the subsequent analysis. He is a highly talented applied statistician who specializes in data visualization.  Evan, thanks for you great work!

For other “Varied Career Path” findings, see the full Alumni/ae Survey Report at the OAP website


Part I:  What Can We Learn by Studying Law School Alumni? A Case Study of One Law School

Part II, Alumni Surveys, Before-Law School

Part III: Alumni Surveys, During Law School

November 9, 2015 in Blog posts worth reading, Data on legal education, Data on the profession, Structural change | Permalink | Comments (0)

Wednesday, November 4, 2015

Part III: Alumni Surveys, Responses on the Law School Experience

Part II of this blog series reported that the top motivations to attend law school have remained the same for over four decades, at least for Northeastern University School of Law (NUSL).  Alumni reported the same underlying desire: to build a satisfying, intellectually challenging career where they could help individuals and improve society. This may be an image forged by pop culture and the idealism of youth, but it is also likely sincere.  It is the better side of our human nature. 

Part II also showed two motivations to attend law school – the desire for “transferable skills” and “eventual financial security"-- that did appear to be shifting over time.  I suggested that these shifts are more likely about a changing ecosystem than a fundamental shift in the type of people applying to law school.  

A similar ecological theme can be observed in the "During Law School" data. For example, since its reopening in 1968, Northeastern Law has required every graduate to complete four 11-week cooperative placements, usually in four different practice settings (e.g., government agency, public defender, large firm, public interest organization). As noted in Part I, students can be paid during co-op because it is a university rather than an ABA requirement. Cf. Karen Sloan, “The ABA says No to Paid Student Externships,” Nat’l L J, June 10, 2014.

One series of questions in the alumni survey specifically focused on the co-op experience, including co-op quality, what was learned, and whether they were paid.  The chart below reveals a steady, four-decade decline in the number of paid co-ops.


In the early 1970s, essentially all four co-ops were paid.  By the mid-80s, the average was down to three. Since the 2000s, the average has been two or fewer paid co-ops.

To my mind, the above trendline is compelling evidence of a steady, systemic shift in the legal ecosystem. I have written about this pattern in the past, suggesting that the rate of absorption of law grads into the licensed bar has been going down since the 1980s.  See Henderson, “Is the Legal Profession Showing its Age,” LWB, Oct 12, 2014 (noting that between 1980 and 2005, the average age of licensed lawyers increased from 39 to 49).  

When I saw this downward trendline for the first time, I recalled my numerous interviews with NUSL alumni/ae from the 1970s. In describing their co-ops, they spoke of opportunities that were plentiful and varied. I often heard the refrain, “I paid for law school mostly with my income from co-op.”  Note that during the 1970s, graduating from college was much less prevalent than today.  Law firms were also growing, with 1970 becoming a major inflection point in the rise of the large law firm. See Galanter & Palay, Tournament of Lawyers (1991) (seminal text collecting and analyzing data on the growth of large firms).

The trendline on paid co-ops also made me rethink what I heard from NUSL co-op employers. The school has roughly 900 employers who regularly or semi-regularly participate in co-op.  I heard several regular employers express strong preferences for “third or fourth co-ops." Why?  Because third or fourth co-op students already had significant legal experience and needed less training to be valuable to the employer.  Training is costly. Even if the trainee is unpaid, the lawyer-as-teacher is expending their own valuable time.  If an employer is going to provide training, they need a way to recapture that investment. Unpaid labor for eleven weeks is one potential way; if the labor is already partially trained, that is even better.

Unfortunately, doing a great job for a co-op employer does not guarantee permanent employment or even a modest wage for temporary work.  The legal ecosystem does not reliably and consistently support those outcomes. Yet, 20, 30, or 40 years ago, the dynamics were far more favorable. 

Obviously, in the year 2015, law grads are having a difficult time finding permanent, long-term professional employment (bar passage-required, JD-advantaged, or non-legal professional jobs).  The shortage of high-quality entry level jobs has given rise to criticisms that legal education needs more practical training.  The implicit assumption is that such a change will cure the underemployment problem.  I am skeptical that is true. 

A more likely explanation for law grad underemployment is that the supply of trained lawyers is in excess of demand, partially due to demographics and partially due to the inability of most citizens to afford several hours of a lawyer's time.  This is a very difficult problem to fix. But misdiagnosing the problem does not help.

To the extent a legal employer is looking for a practice-ready law grad, Northeastern’s co-op model is as likely to deliver that outcome as anything else I have observed.  My in-depth review for how co-op affects professional development is written up in OAP Research Bulletin No. 3.  Ironically, what may be the best practice-ready model among ABA-accredited law schools is a 50-year old program that most critics may not know exists. But see Mike Stetz, “Best Schools for Practical Training,” Nat’l Jurist, March 2015 (ranking Northeastern No. 1).

The experiential education crowd will be heartened by another “During Law School” finding.  Among 833 alumni respondents, there were more than 3,200 co-ops identified by practice setting.  Alumni were asked to identify their most valuable co-op and provide a narrative as to why. 

Below is a chart that plots the difference between the baseline frequency of a particular co-op practice setting and how often that practice setting was picked as the most valuable.  The scale is in standard deviation units, with “par” meaning that the practice setting was most valuable in the same proportion as its frequency in the overall sample.


It is not hard to see the common theme.  Co-ops where students can observe lawyers in action – or better yet, get stand-up time in court – were rated as much more valuable.  The table below captures some of the underlying narrative comments.


For other “During Law School” findings, see the full Alumni/ae Survey Report at the OAP website


Part I:  What Can We Learn by Studying Law School Alumni? A Case Study of One Law School

Part II, Alumni Surveys, Before-Law School

Part IV: Alumni Surveys, The Varied Career Paths of Law School Graduates

November 4, 2015 in Blog posts worth reading, Data on legal education, Data on the profession | Permalink | Comments (0)

Monday, November 2, 2015

Part II: Alumni Surveys, Pre-Law Characteristics and Motivations

Building on the introduction (Part I) of this blog series, our alumni survey of Northeastern University School of Law yielded cross-sectional data that span graduates from 1971 to 2012.  Because of the large time span, some of the most interesting responses to questions tend to fall into two buckets:

  1. What is staying the same?  Here we are looking for response patterns that are relatively stable and constant across age cohorts.
  2. What is changing?  Likewise, we are also interested in responses that appear to be changing as a function of time of graduation.

In the portion of our analysis that looked at pre-law characteristics and motivations, our most striking findings tended to fall into bucket #1. 

For example, below is a graphic summarizing responses to the question, “How important were the following goals in your decision to attend law school?” Responses are organized by decade of graduation.  They are ordered by most important to least important for respondents who graduated in 2000 or later.

                              Goals for Attending Law School, by Decade of Graduation

One of the most striking features is that the top three responses are essentially identical for all four age cohorts.  For each group, the desire to have a satisfying career, help individuals, and improve society were all, on average, very important in the decision to attend law school. 

Although there are differences across age cohorts, there remains relatively clear clustering by decade of graduation. (Query: would this same pattern hold true at other law schools?  One of the advantages of pooling data across schools is the ability to isolate a self-selection effect that operates at the school level.)

Yet, some factors appear to be changing over time, such as the importance of transferable skills and eventual financial security.  With each decade cohort, respondents are rating these factors progressively more important to their decision to attend law schools. Likewise, “other goals” appear to be progressively less important. 

These patterns (and others survey results I will report in Parts III and IV) suggest gradual changes in the knowledge worker ecosystem that require students to be more deliberate and focused in their decision to attend law school.  For example, costs of all of higher education are going up at the same time that the financial payoffs of traditional graduate and professional education are becoming less certain.  This is an ecological effect that is bound to have an influence on students and student decision making.  Although legal education would be part of this shift, the shift itself would not be unique to law.

This interpretation is consistent with our focus group discussions with Northeastern alumni.  This group queried whether the term “transferable skills” was even part of the lexicon when they were graduating from college.  Likewise, the group commented that the decision to attend law school during the 1970s and 1980s was not difficult because tuition was relatively low and jobs, including paid co-op jobs, were relatively plentiful. Although the legal market may be tighter and more complex than in earlier decades, the Northeastern alumni commented that the tradeoffs were changing for all knowledge workers.  

For other “Before Law School” findings, see the full Alumni/ae Survey Report at the OAP website


Part I:  What Can We Learn by Studying Law School Alumni? A Case Study of One Law School

Part III: Alumni Surveys, During Law School

Part IV: Alumni Surveys, The Varied Career Paths of Law School Graduates

November 2, 2015 in Blog posts worth reading, Data on legal education, Data on the profession, Important research | Permalink | Comments (1)

Part I: What Can We Learn by Studying Law School Alumni? A Case Study of One Law School

BryantgarthSeveral years ago, as the legal academy was beginning to work its way through the implications of the landmark “After the JD” Project (AJD), one of the principal investigators, Bryant Garth, commented to a group of fellow law professors that “within a few years it will be educational malpractice for law schools to not study their own alumni.”

Garth had special standing to make this claim, as he had launched the AJD during his long tenure at the American Bar Foundation and then went on to serve as Dean of Southwestern Law School in Los Angeles. While at Southwestern, Garth taught a short 1L course about legal careers that combined AJD findings with live interviews with Southwestern alumni. Despite decades of research studying lawyers, Garth gushed at how much he personally learned from these interviews and how the narratives were often surprising and inspiring, particularly for Southwestern students filled with apprehension at what the future might hold.

I had occasion to remember Garth’s observations in early 2011 when Emily Spieler, then the Dean of Northeastern University School of Law (NUSL), suggested that I study her alumni.

Northeastern Law

Northeastern is an interesting case study because for nearly 50 years the school has required four 11-week cooperative placements (or “co-ops”) as a condition of graduation. To facilitate completion within three years, the 1L year at Northeastern is taught in semesters while the 2L and 3L years are taught over eight alternating quarters. Summer-winter co-op students take classes during the fall and spring quarters, while fall-spring co-op students attend classes in the summer and winter quarters. Because co-ops are not for academic credit – they fulfill Northeastern University rather than ABA-accreditation requirements – students can be paid for the full 11 weeks. (More on that in Part III of this series.)

Dean Spieler wanted a third party to study Northeastern because, in her experience as dean, her many encounters with Northeastern alumni suggested to her that the School’s unusual education model was accelerating the professional development of its students and enabling them to make better, more informed career choices.

Acceleration of profession development is a very difficult effect to measure, but it is certainly plausible. In fact, the entire experiential law movement is largely premised on this claim. So I signed onto a multi-year initiative that we called the Outcomes Assessment Project (OAP).

The premise of the OAP was very unusual. Through a generous but anonymous benefactor, the research tools and templates developed for the OAP would be made available to other law schools interested in studying graduates. The intent is for law schools to accumulate data using similar methods and instruments, driving up the value of the data (because it is comparable across schools) while driving down the cost of collection and analysis.

There are many phases to the OAP, including those focused on admissions, the student experience, and co-op employers. Here, however, I wanted to write about what we learned from a survey of Northeastern’s alumni.

Last fall, we sent a survey instrument to Northeastern alumni who graduated from the law school between 1971 and 2012 (~4,000 law grads for which NUSL had a current email address). The survey instrument was substantially based on the AJD Wave III survey instrument, which was sent to a representative sample of law graduates from all ABA-accredited law schools who took the bar in the year 2000.

In contrast to the AJD, which has produced remarkable knowledge about law school grads from the year 2000, the OAP Alumni/ae Survey included four decades of law graduates from a single law school. Although this is not a true longitudinal sample, which samples the same people over time, this methodology enables cross-sectional comparisons between different cohorts of graduates (e.g., by decade of graduate or pre/post AJD).

The response rate of the Northeastern alumni survey was 21% (833 total completed questionnaires), which is relatively high for a long online survey. Because the resulting sample substantially mirrored the baseline data we had for Northeastern alumni practice areas and years of graduation, we were confident that the resulting sample was both representative and reliable.

Applied Research

Similar to the AJD, the OAP Alumni/ae Survey produced enough data to keep researchers busy for several years. Hopefully, these data will eventually be archived and aggregated at the American Bar Foundation or a similar institution in order to facilitate a broader and deeper understanding of legal careers.

However, the OAP was largely set up to be applied research. What does this mean? Here, the goal is, at least in part, to obtain data that is operational in nature, thus enabling a law school to examine and test fundamental assumptions and generate insights related to its stated goals and mission. In a word, to improve.

Further, when skillfully boiled down using data visualization, the findings themselves tend to be of great interest to all law school stakeholders, including alumni, faculty, administrative staff, current students, and prospective students. Interest is particularly piqued during times of transition and uncertainty, such as now, when law schools and the practicing bar are looking to each other to provide potential answers and support.

To makes results as accessible as possible, we decided to present the preliminary Alumni Survey results in a simple three-part framework:

  • Before Law School: pre-law characteristics and motivations
  • During Law School: the law school experience
  • After Law School: job mobility and satisfaction

This week, I am going to give a sampling of findings from all three sections – findings that will likely be of interest to a non-Northeastern audience of law faculty, practicing lawyers, and students. If you are interested in reading the entire preliminary report, it can be found online at the Northeastern OAP website.


Part II, Before-Law School

Part III: Alumni Surveys, During Law School

Part IV: Alumni Surveys, The Varied Career Paths of Law School Graduates

November 2, 2015 in Blog posts worth reading, Data on legal education, Data on the profession, Important research | Permalink | Comments (0)

Thursday, September 17, 2015

2015 Median MBE Scaled Score Arguably Declines Less Than Expected

Natalie Kitroeff at Bloomberg published earlier today an article with the first release of the median MBE scaled score for the July 2015 Bar Exam -- 139.9 -- a decline of 1.6 points from the July 2014 score of 141.5. 

While this represents a continuation of the downward trend that started last year (when the median MBE fell a historic 2.8 points from 144.3 in July 2013), the result is nonetheless somewhat surprising. 

The historic decline in the median MBE scaled score between 2013 and 2014 corresponded to a modest decline in the LSAT score profile of the entering classes between 2010 and 2011. 

As I discussed in my December blog posting on changing compositions of the entering classes since 2010, however, the decline in LSAT score profile of the entering classes between 2011 and 2012 was much more pronounced than the decline between 2010 and 2011.  Thus, one might have expected that the decline in the median MBE scaled score for 2015 would have been even larger than the decline between 2013 and 2014. 

But instead, the decline was only 1.6 points, just slightly more than half of the 2.8 point decline of the previous year.

Why would a demonstrably greater decline in the LSAT profile of the entering class between 2011 and 2012 (compared with 2010-2011) yield a manifestly smaller decline in the median MBE scaled score between 2014 and 2015 (compared with 2013-2014)?

This likely will remain a mystery for a long time, but my guess is that the ExamSoft debacle resulted in an aberrationally large decline in the median MBE scaled score between 2013 and 2014, such that the corresponding decline between 2014 and 2015 seems disproportionately smaller than one would have been expected.

Over on Law School Cafe, Debby Merritt has a very good description of the different factors that likely have impacted bar passage performance in July 2015.

Derek Muller has collected bar passage results for the several states that have released at least some results so far and has posted them on his Excess of Democracy blog.  Focusing only on overall bar passage rates, two states are "up," (North Dakota (6%) and Iowa (5%)), six are down between 1-5% (Missouri (-1%), Washington (-1%), Montana (-2%), Kansas (-3%), North Carolina (-4%), West Virginia (5%)), and four are down double-digits (Mississippi (-27%), New Mexico (-12%), Oklahoma (-11%), and Wisconsin (-10%).  (Last year 21 states were down 6% or more on first-time bar passage and six of those were down 10% or more.)

September 17, 2015 in Blog posts worth reading, Data on legal education, Data on the profession | Permalink | Comments (0)

Tuesday, September 1, 2015

What's driving the demographic gap between BigLaw leaders and their CEO/GC clients?

Picture1Las Vegas, NV.  The illustration to the left was just published in The American Lawyer.  It accompanied a story on how law firm leaders are significantly older than leaders in the large corporations they serve. See MP McQueen, The Generation Gap:  BigLaw's Aging Leaders, Aug 24, 2015. 

At least for me, this is a jarring graphic because it conveys so much truth.  Today's Millennials are so underwhelmed with the BigLaw model.  They like the pay and the perks, as it enables them to live well in attractive large market cities.  They can also quickly pay off their law school debt.  But precious few of them are all in. Illustrator James Steinberg totally nailed it.

There are numerous reasons for the culture divide, but as shown in the chart below, the most obvious is a very large age gap between leaders and entry-level workers -- it tends to be a lot larger in BigLaw than almost anywhere else: 4% of AmLaw 100 leaders are Gen X compared to 33% of NASDAQ-traded companies. 

These data beg the question, why are large law firms so out-of-sync with the institutions they serve?

One reason is certainly the ownership structure.  Any Fortune 500 or NASDAQ-traded company that got this top-heavy in its senior management would be getting killed on its stock price.  Under the Rule 5.4 prohibition on nonlawyer investors, law firms are spared the anxiety of having analysts and short sellers constantly evaluating their business. Yet, the absence of a public market means that law firm owners and managers cannot fully monetize the enterprise value they create. So what's the effect?  Very little enterprise value gets created.  Instead, lawyer/owners  focus on maximizing this year's net distributable income.

It is important to not knock the BigLaw model too hard.  For about a century, it worked extremely well, as US law firms steadily grew with their clients.  Each unit of economic growth produced some larger unit increase in legal complexity, so demand for sophisticated legal services was a steady upward sloping line. By following a simple model -- hire more associates, promote some to partners, lease more office space, repeat -- equity partners in the AmLaw 100 became millionaires.  

Today, BigLaw is getting grayer because the 100-year old gold factory is breaking down. Law firms' portion of corporate legal spending is no longer growing, as in-house lawyers, NewLaw managed services shops (United Lex, Axiom, Counsel on Call), and technology are all curbing demand for traditional law firm services.   The best economic play for 55- or 60-year old equity partner is to ride out the existing model with the dwindling but still substantial number of Baby Boomer senior in-house lawyers who are themselves not too anxious to change.  

This is not the story equity partners tell themselves; it's the logic that underlies the inertial path.  It's where we end up when we are no longer deeply invested in the places we work.  It's become a job.  I am not judging here; I'm describing what I have observed through hundreds of conversations with large firm partners.

The result of this dynamic is that a large proportion of BigLaw--but certainly not all of it--is just tinkering at the margins of change. A law firm can become more cost-effective for clients, at least in the short to medium term, by reducing reliance on associates.  Associates are expensive and are, by definition, getting paid to learn.  For the last 15-20 years, firms have shifted their leverage model to counsel, staff attorneys and nonequity partners, where (a) there is little to no training, (b) the margins are higher, and (c) the clients can't complain about inefficient associates. This is the Diamond Model, which substantially cuts out the entry-level lawyer.  See The Diamond Law Firm: A New Model or the Pyramid Unraveling? (Dec. 2013); Sea Change in the Legal Market, NALP Bulletin, Aug. 2013.


Unlike the original Pyramid Model, invented by Paul Cravath circa 1910, the Diamond Model is not a carefully conceived business strategy. Rather, it's a way to maximize this year's and next year's net distributable income without making difficult strategic tradeoffs. Yet, in the longer term, which is no longer too far off, the Diamond Model is a disaster.  The few associates who make it into large firms are grateful for the high pay and the training.  But very few if any are impressed with the business model.  Among Millennial lawyers, in-house is the new brass ring.

Law firms are filled with brilliant people. Why are they going down this road?  Three interrelated reasons:

  1. Lack of Experience. Today's law firm partners have little or no experience with strategy--for a hundred years, intelligence and hard work worked just fine.  This is not a change in strategy--it is having a strategy.  Then executing.  That's hard.
  2. Incentive Structures. Virtually all incentives inside firms today favor revenue generation; as a result, few partners have the mental whitespace to understand, much less think through, the changes that are occurring within the broader industry.  To fix the bridge, you have to slow down the traffic.
  3. To Big to Fix.  The first strategy mistake for the current generation of AmLaw 100 leaders was to become bigger without becoming measurably better.  Big firms filled with laterals is a difficult environment to share risk. Maximizing this year's distributable income becomes one of the few things people can agree on.

That said, I am not counting BigLaw out.  I am writing this blog post from the International Legal Technology Association (ILTA) conference in Las Vegas.  From far away, it is all too easy to treat BigLaw as a monolith--it's not.  At ILTA, professionals from several of the most innovative law firms are willing to pop the hood and share what they doing.  See Ahead of the Curve: Three Big Innovators in BigLaw, Aug. 25, 2014Suffice it to say, some firms are several years into strategies that have the potential to take market share from peer firms.  Further, the innovation teams inside these firms are having the time of their professional lives because the work is so collaborative and creative--the antithesis of billable hour work.  What is also clear is that many competitors just can't muster the leadership nerve to make similar investments. 

In the years to come, some BigLaw firms are going to pull away from the rest, becoming a magnet for talent and then clients.  Younger lawyers are going to thrive there.  Another portion of BigLaw is going to gradually fade away. 

September 1, 2015 in Current events, Data on the profession, Law Firms, Structural change | Permalink | Comments (3)

Wednesday, August 12, 2015

Of Transfers and Law-School-Funded Positions

1.      Many Elite Law Schools with Large Numbers of Transfers also Have Large Numbers of Law-School-Funded Positions

Several weeks ago, I participated in two separate conversations.  One was about when law-school-funded positions should be categorized as full-time, long-term, bar-passage-required (FLB) positions and one was about transfer students.  This prompted me to compare those schools that are “big players” in law-school-funded positions with those schools that are big players in the “transfer” market.  Interestingly, as shown in the chart below, there is a significant amount of overlap.

For the Class of 2014, of the 15 law schools with the most graduates in FLB positions, ten had a significant (net) number of transfer students in the summer of 2012.  (The chart is sorted based on 2014 FLB positions (in bold).  To provide context, the chart also includes the 2011 net transfer data and 2013 law-school-funded FLB data for these 10 schools.)

Law School

2011 Net Transfers

2013 Law-School-Funded FLB

2012 Net Transfers

2014 Law-School-Funded FLB

























































Note that in both 2013 and in 2014, six of the ten schools had more transfers than law-school-funded positions, suggesting that had they taken fewer transfers they might not have needed to provide as many law-school-funded positions. Phrased differently, this data suggests that with the transfer students, these law schools have too many graduates compared to the number of jobs the market is able to provide for their graduates.

2.      Adjusting to the Employment Market or Continuing to Attract Transfers and Provide Law-School-Funded Positions?

One might expect that a natural response to this “mismatch” between the number of graduates and the number of meaningful employment opportunities provided by the market would be to have fewer graduates (and fewer law-school-funded positions).  Indeed, for many of the schools in the chart above, the simplest way to do this would involve not accepting any transfer students (or accepting very few transfer students).  The first-year enrollment at these schools appears to be fairly-well calibrated with the number of meaningful employment opportunities provided by the market.  Of course, this would mean a significant loss of revenue.

But what happened at these ten law schools in the summer of 2013 and the summer 2014 with respect to transfer students? As shown in the chart below, almost all have continued to take large numbers of transfer students.  With knowledge that a not insignificant percentage of their graduates need the support of law-school-funded positions because they can’t find market positions, these law schools continue to take large numbers of transfers.  Indeed, the total number of net transfers at these ten law schools is even higher in 2013 and 2014 than in 2011 and 2012.



2014   Net Transfers

2013   Net Transfers




































3.   Why are These Schools Continuing to be Big Players in the Transfer Market and in Providing Law-School-Funded Jobs and Why Aren’t Other Schools Doing This as Well?

Many elite law schools are participating heavily in the transfer market and in providing law-school-funded jobs because they can and because it makes financial sense to do so.

As a general matter, only relatively elite law schools are able to attract large number of transfer students willing to pay $50,000 per year in tuition.  (This assumes that most transfers are paying full tuition. There is very little information available about scholarships in the transfer market, but anecdotes suggest that scholarships are uncommon.)  By taking large numbers of transfers, these schools generate revenue that funds general operations AND enables the school to fund post-graduate employment opportunities for a significant number of graduates.  According to NALP, most graduates in law-school-funded positions receive salaries of roughly $15,000-$30,000 per year.  Even if they have as many law-school-funded positions as they do transfers, the schools still net $70,000 to $88,000 per transfer student over the second and third year of law school even after accounting for the salaries for law-school-funded positions. (To be fair, some modest percentage of law-school-funded positions at several of these law schools may be great opportunities that are highly competitive and pay a salary comparable to a market salary – in excess of $40,000 per year.  Some of these may be public interest opportunities that some students find particularly attractive.  But the proliferation of law school funded positions (having grown from just over 500 in 2012 to more than 800 in 2014), with most of the growth occurring at relatively elite law schools, suggests that many of these positions do not fit the profile described in the preceding two sentences.)

Other schools would love to do this, but most simply don’t have the ability to attract significant numbers of transfer students.  Moreover, in the present legal education environment with declining enrollment at most schools, many law schools are running a deficit, and simply can’t afford to invest money in law-school-funded positions for their graduates.

Notably, up until this year, this effort was aided by the reporting of law-school-funded jobs as if they were the same as jobs provided by the market.  A school with law-school-funded positions that otherwise met the definition of FLB positions could report a higher percentage of its graduates in such positions.  This minimized the extent to which less than robust employment results might erode the schools’ ability to attract students and has allowed these elite schools to continue to attract large numbers of relatively highly-credentialed entering students (and transfers) along with the revenue they bring to the school.  For the Class of 2015, however, these law-school-funded positions will be reported separately from FLB positions provided by the market.

4.      What Questions Might this Raise for Students?

Students considering transferring to one of these elite schools should ask two questions: 1) What percentage of law-school-funded positions went to transfer students? and 2) How do employment outcomes for transfer students compare with employment outcomes for students who began at the school as first years?  (Even with the increased attention on transparency with respect to employment outcomes, one data point not presently collected relates to employment outcomes for transfer students.)  This isn’t to suggest that all transfers end up in law-school-funded positions.  Some transfer students may outperform some of the students who began at a law school as first years, both in terms of academic performance and in terms of relationship skills.  These transfer students may outcompete original first-year students for market employment opportunities.  But students considering transferring might want to assess whether their employment prospects really will be better at the school to which they might transfer as compared with the opportunities available to them if they remained at the school from which they are considering transferring, particularly if they are near the top of the class at the school from which they are considering transferring.

Students who had matriculated as first-years at one of these elite law schools, might want to ask the law school administration how and why having a large number of transfers is a good thing for those who matriculated as first-years at the elite law school.  Having the additional revenue might enhance the educational experience in some way, but having significantly more students competing for jobs would seem to be an unnecessary challenge. 

5.      Conclusion

The data on transfers in 2013 and 2014 suggests that at many elite law schools, there will continue to be more graduates than jobs provided by the market.  As a result, these law schools are likely to continue to provide law-school-funded positions for some number of their graduates. Indeed, the prospect of law-school-funded positions as a fall-back option if a market position is not available might provide some solace for students, including transfer students, at these elite law schools. 

Nonetheless, there is a further ripple effect.  With dozens of graduates from these elite law schools in law-school-funded positions looking for market jobs, it makes it even more challenging for the next year’s graduates from these elite schools to find market jobs and almost assures that many graduates will still need the support of law-school-funded positions in the coming years.

(I am grateful to Bernie Burk and others for helpful comments on earlier drafts of this posting.)

August 12, 2015 in Data on legal education, Data on the profession | Permalink | Comments (0)

Thursday, August 6, 2015

How is the entry-level legal job market in Australia?

AlsaNot good.  There are more law graduates than jobs, yet law schools are making matters worse by admitting more students in order to generate subsidies for other parts of the university. That the basic charge of the Australian Law Students Association (ALSA), according to this story in the Lawyers Weekly, a publication that covers the business of law in Australia.

Legal education is Australia is very different than the U.S.,  yet the dynamics of the two entry-level markets seem to be converging.  Law has historically been an undergraduate degree in Australia (LLB), but in recent years the JD has been added as a new and more prestigious way into the profession. Here is the statement of an ALSA spokesperson based on recent survey results of the ALSA membership.

ALSA are of the position that there is still an oversupply of graduates because of the increasing sizes of law schools and the duplication in the number of law schools across the country. ...

Many who have undertaken the Juris Doctor particularly expressed concerns in their survey responses, highlighting that they undertook the postgraduate law degree to further their job prospects. Instead, they are facing the worrying reality that there are fewer jobs available for law graduates as well as the fact that they are completing their degrees with a sizeable student debt.

The article then goes on to describe growing law student anxiety over employment and student loan debt.  Wow, different system but a very similar result.  

One of the advantages of the Australian LLB degree is that it is often combined with another undergraduate degree, typically by adding one year of additional study.  As a result, many LLBs don't go on to qualify for practice, but the legal training probably augments their worldly knowledge and critical thinking skills.  But alas, the Australians are starting to dilute their extremely generous higher education subsidies -- we are just much further down that road. Further, the true undercurrent here is the growing insecurity facing virtually all knowledge workers, Australian or US.  Legal education is just the bleeding edge of this problem.

August 6, 2015 in Current events, Data on legal education, Data on the profession, New and Noteworthy | Permalink | Comments (0)

Tuesday, August 4, 2015

Metrics and Legal Ops Professionals

In a recent post, I urged readers to visit a legal department with a large legal operations staff.   The goal?  To see the future of modern corporate law practice.  Fortunately, Bloomberg Law recently videotaped a legal ops panel moderated by Amar Sarwal of the ACC.  It contains a conversation rarely if ever heard in law schools or bar associations.

The three legal departments profiled are AIG (insurance), Marsh & McLennan (diversified financial and professional services), and GlaxoSmithKline (pharma).  Note the enormous emphasis on metrics, data, and technology.  Note also how the services of law firms are being put through a procurement process. 

August 4, 2015 in Blog posts worth reading, Current events, Data on the profession, Law Firms, Legal Departments, New and Noteworthy, Video interviews | Permalink | Comments (0)

Wednesday, July 29, 2015

"Solicitors 'in denial' about threat from accountants"

Legalservices (321x207)That's the headline from today's Law Society Gazette, the publication of record for solicitors in England and Wales.  The UK is fairly far along in liberalization of its legal markets, progressing from the Clementi Report in 2004 to the Legal Services Act 2007 to the licensing of Alternative Business Structures in 2012.  Now several hundred entities have obtained ABS status.  

The Gazette article reports that accountants are poised to be large players in the ABA space:

Accountants will soon be competing directly with solicitor firms ‘on every high street in the country’, according to a leading financial advisor to the legal sector.

Ian Muirhead, chairman of Solicitors Independent Financial Advice, said he expects 750 accountancy firms – three times more than first envisaged – to move into probate work after securing an alternative business structure licence.

The Institute of Chartered Accountants in England & Wales has accredited 113 entities as an ABS since last October, having been accepted as an approved regulator almost a year ago. A further 34 applications are being processed.

Speaking at a Westminster Legal Policy, Muirhead said too many solicitor firms are ‘in denial’ about the threat from the accountancy profession.

‘Success will go to those who can manage businesses and I query whether that’s going to be the solicitors or whether solicitors are going to be the back room boys,’ he said.

Muirhead argued that law firms’ response so far has been focused on consolidation, mergers and acquisitions – but this risks playing into rivals’ hands.

‘[The response is] safety in numbers, more of the same, not thinking outside the legal silo, and therefore missing the opportunity of which many new ABSs are availing themselves, of providing a more diversified and holistic client service,’ he added. ... 

Some U.S. lawyers believe that liberalization won't come to the U.S. because the legal industry is too balkanized by state bar authorities.  

I think this view, however, is likely naive. The market can change because regulators change the rules (the UK). Alternatively, the market can change because clients change their buying habits in favor of nontraditional legal service providers that are financed by sophisticated nonlawyer investors (the US).  See, e.g., Is Axiom the Bellwether for Disruption in the Legal Industry, LWB, Nov. 10, 2013.

In the US, it is probably true that regulators lack the stomach to initiate a regulatory action where the client ostensibly being protected is a Fortune 500 corporation.  If the action ends up in federal court, the bar officials risk looking like protectors of the guild and have a decent chance of losing.  The prohibition against nonlawyer investment (MR 5.4) is based on the assumption that the nonlawyer profit motive will compromise lawyer independence, thus harming the unwitting and unsophisticated legal consumer.  But that does not describe IBM's or JP Morgan's relationships with sophisticated LPO or analytics shop (or any general counsel charged with stretching his or her legal dollar). As a result, the venture capital money flows in.

When liberalization is viewed in this light, there are probably more similarities between the US and UK than we might want to acknowledge. 

July 29, 2015 in Current events, Data on the profession, Innovations in law, Law Firms, New and Noteworthy, Structural change | Permalink | Comments (2)

Wednesday, July 22, 2015

What is more important for lawyers: where you go to law school or what you learned? (Part II)

If you're trying to maximize the financial value of an undergraduate degree, it is better to bet on course of study than college prestige.  Indeed, prestige is largely irrelevant to those who major in engineering, computer science, or math.  In contrast, prestige does matter for art & humanities grads, albeit the financial returns are significantly lower than their tech counterparts.  

These are some of the takeaways from Part I of this blog post. Part I also presented data showing that law is a mix of both: financial returns have been high (cf. "red" tech majors) and prestige matters (cf. "blue" arts & humanities crowd).  

The goal of Part II is to address the question of whether the pattern of high earnings/prestige sensitivity will change in the future. I think the answer to this question is yes, albeit most readers would agree that if law will change is a less interesting and important question than how it will change.  Speed of change is also relevant because, as humans, we want to know if the change is going to affect us or just the next generation of lawyers.

Shifts in the Legal Market

There are a lot of changes occurring in the legal market, and those changes are altering historical patterns of how legal services are being sold and delivered to clients. In the past, I have thrown around the term structural change, yet not with any clear definition.  To advance the conversation, I need to correct that lack of precision. 

In economics, there is a literature on structural change as applied to national or regional economies (e.g. moving from a developing nation to an industrial nation; or moving from an industrial to a knowledge-based economy).  Investors also focus on structural change within a specific industry because, obviously, large changes can affect investor returns.  When I have used the term structural change on this blog, it has been much closer to investor conceptions.  Investopedia offers a useful definition even if it's somewhat colloquial: 

Definition of 'structural change': An economic condition that occurs when an industry or market changes how it functions or operates. A structural change will shift the parameters of an entity, which can be represented by significant changes in time series data.

Under this definition, the legal industry is certainly undergoing structural change.  The proportion of law graduates getting a job in private practice has been on the decline for 30 years; over the last 35 years, the average age of the licensed lawyer has climbed from 39 to 49 despite record numbers of new law school graduates; the proportion of associates to partners has plummeted since the late 1980s.  See Is the Legal Profession Showing its Age? LWB, October 12, 2014.  Since the early 2000s, long before the great recession, associate-level hiring has been cut in half. See Sea Change in the Legal Market, NALP Bulletin, August 2013.

Likewise, among consumers of legal services, there is a lot of evidence to suggest that lower and middle class citizens can't afford a lawyer to solve life's most basic legal problems, thus leading to a glut of pro se litigants in state courts and many more who simply go without things like contracts and wills.  This troubling trend line was obscured by a boom in corporate legal practice, albeit now even rich corporations have become more sensitive to legal costs -- the sheer volume and complexity of legal need is outstripping their budgets.  In response to the lag in lawyer productivity and innovation, there is a ton of investor-backed enterprises that are now elbowing their way into the legal industry.  See A Counterpoint to "the most robust legal market that ever existed in this country"LWB, March 17, 2014.  

The impact of all this change -- structural or otherwise -- is now being felt by law schools. Applicants are down to levels not seen since the 1970s, yet we have dozens more law schools. It has been said by many that law schools are losing money, albeit we have zero data to quantify the problem.  Based on my knowledge of my own law school and several others I am close to, I am comfortable saying that we have real changes afoot that affect how the legal education market "functions or operates."

There is a sense among many lawyers and legal academics that the legal world changed after 2008. None of the "structural" changes I cite above are pegged in any way to the events of that year.  

What did change in 2008, however, was the national conversation on the legal industry, partially due to the news coverage of the mass law firm layoffs, partially due to important books by Richard Susskind and later Brian Tamanaha and Steve Harper, and partially due to a robust blogosphere.  This change in conversation emboldened corporate legal departments to aggressively use their new found market power, with "worthless" young associates getting hit the hardest.  This new conversation in turn exposed some of the risks of attending law school, which affected law school demand.  But alas, this was all fallout from deeper shifts in the market that were building for decades. Let's not blame the messengers.

Dimensions of Change

I am confident that the future of law is going to be a lot different than its past. But I want to make sure I break these changes into more discrete, digestible parts because (a) multiple stakeholders are affected, and (b) the drivers of change are coming from multiple directions.

Dimension 1: basic supply and demand for legal education

To unpack my point regarding multiple dimensions, let's start with legal education. Some of the challenges facing law schools today are entirely within the four corners of our own house.  Yet, legal education also has challenges (and opportunities) that arise from our connection to the broader legal industry.  This can be illustrated by looking at the relationship between the cost of legal education (which law schools control, although we may blame US News or the ABA) and entry level salaries (which are driven largely by the vagaries of a client-driven market).  

The chart below looks at these factors.  My proxy for cost is average student debt (public and private law schools) supplied by the ABA.  My income variables are median entry level salaries from NALP for law firm jobs and all entry level jobs.  2002 is the first year where I have all the requisite data.  But here is my twist:  I plot debt against entry-level salary based on percentage change since 2002.  


If a business nearly doubles its price during the same period when customer income is flat, demand is going to fall.  Thus, the sluggish entry-level market presents a difficult problem for legal education.  Sure, we can point to the favorable statistics from the AJD or the premium that a JD has historically conferred on lifetime earnings, but law professors are not the people who are signing the loan papers.  The chart above documents a changing risk/reward tradeoff.  To use the frame of Part I, the red dots are sinking into the blue dot territory, or at least that is the way prospective students are likely to view things.

Fortunately, smaller law school classes are going to be a partial corrective to low entry-level salaries.  The biggest law school class on record entered in the fall of 2010 (52,488); in 2014, the entering class had shrunk by over 27% (37,942). When entry-level supply is reduced by 25+%, upward pressure on salaries will build.  Yet, the composition of the legal economy and the nature of legal work is clearly changing.  Further, the rate of absorption of law school graduates into the licensed bar has been slowing for decades.  See Is the Legal Profession Showing its Age? LWB, October 12, 2014. It would be foolhardy to believe that time and fiscal austerity alone are going to solve our business problems. Instead, we need to better understand our role as suppliers to a labor market.

Dimension 2:  The content of legal education

The content of legal education is not necessarily fixed or static.  We could change the content, thus affecting how the market responds.  

To provide a simple example, one of my students is starting work this fall at Kirkland & Ellis.  From a financial perspective, this is a good employment outcome.  He will be moving to Chicago with his girlfriend who just received her MS in Information Systems from IU's Kelley School of Business.  The MS from Kelley is a very "red" degree.  It can also be completed in one year (30 credit hours).  Well before she graduated, this recent grad had competing offers from PWC and Deloitte, both in the $80,000 range.   For many Indiana Law students, an ideal post-grad outcome would be $80K in Chicago at an employer who provides challenging work and high-quality training.  Yet, my student's girlfriend got this ideal outcome in 1/3 the time and likely 1/2 the cost of an Indiana Law grad.  

Perhaps we should consider cross-pollinating these disciplines. A huge portion of the legal profession's economic challenges is attributable to flat lawyer productivity -- customers are struggling to pay for solutions to their legal needs.  Information systems are a huge part law's productivity puzzle.  Below is a chart I use in many of my presentations on the legal industry.  The chart summarizes the emerging legal ecosystem by plotting the Heinz-Laumann two-hemisphere model against Richard Susskind's bespoke-to-commodity continuum. [Click-on to enlarge.]


The key takeaway from this diagram is that the largest area of growth is going to be in the multidisciplinary green zone -- the legally trained working shoulder-to-shoulder with those skilled in information systems, statistics, software development, and computational linguistics, to name but a few.  These are "red" disciplines.  Do law schools want to be part of this movement?  Let me ask this another way -- do law schools want to be relevant to the bulk of the legal market that needs to be rationalized in order to maintain its affordability? Harvard grads will have options on Wall Street for the foreseeable future.  But 98% of law schools operate in a different market.  Further, some HLS grads, or students who might qualify for admission to Harvard, might prefer the big upside rewards that are only available in the green zone.  In short, a new hierarchy is emerging in law that is still very much up for grabs.

If an academic wants to better understand the rapidly changing nature of legal work, I would urge them to visit a large legal department with a substantial legal operations ("legal ops") staff.  These are the professionals who have been empowered by general counsel to find ways to drive up quality and drive down cost using data, process, and technology.  These are the folks who are making build-versus-buy decisions, putting pressure on law firms to innovate in order to hang on to legal work, and experimenting with NewLaw legal vendors. 

I am finishing up a story on legal ops professionals for the ABA Journal.  (By the way, legal ops exist in law firms as well as legal departments and green zone legal vendors. The role is most developed, however, in legal departments.)  My editor flagged the issue that virtually all of the legal ops people in the story did not graduate from prestigious law schools (or any law school).

My only response is that legal operations people have specialized skills and knowledge (often "red" but sometimes involving EQ) that others lack; without these skills, they can't do the job.  Legal ops people live in a world of outputs and metrics.  For example, are legal expenses and settlement amounts trending down over time -- yes or no? If so, by how much?  How much internal staff time does it take to negotiate a revenue contract? How much of this process can be automated? What will it take to get our staff to accept the new system?

As these examples show, a legal ops person is typically going to be evaluated based on measurable outputs -- do they get results? Where someone went to law school is an input that is likely irrelevant to the question.  The only qualifier is whether the curriculum of that school provided valuable, specialized domain knowledge -- most likely non-legal red skills but also skills related to teams, communication, and collaboration. 

Dimension 3:  The value of pedigree to the customer 

Law has historically been what economists call a “credence good.”  This means that a layperson has a difficult time assessing quality.  As a result, proxies for quality, such as pedigree or prestige, have historically been very important when hiring a lawyer or law firm.  

One of the reasons that the field of legal operations is gaining momentum is because it is creating tools and systems that enable clients to look past credentials to obtain information on things they really care about, such as cost, outcome, and speed of delivery. There are now companies coming into existence that are gathering data on lawyers' win-loss rates. See Another Example of Using Big Data to Improve Odds of Winning in Court, LWB, April 12, 2015.  Sure, apples-to-apples comparisons are very difficult to make -- every case is unique in some respect. But the amount of money at stake is large enough that the data challenges will be surmounted.  When that day arrives, we won't opine on the value of pedigree to legal outcomes; we'll just calculate it. More significantly, clients focused on outcomes will change their buying patterns.  Early returns I have seen suggest that the value of pedigree to legal outcomes may be close to negligible.

Do any of us care where the engineers who designed our smart phones went to college? Not really. We just care how well the smart phone works. 

In this respect, the future of law is likely headed in the direction of Google (a pure red company).  In the early days, the founders of Google favored grads of Caltech, Stanford and Berkeley.  But over time, the company learned that prestige of graduate school was a poor predictor of job success. Because Google lives and dies by its outputs, the company changed its hiring model to attract the most qualified engineers.  See George Anders, The Rare Find: How Great Talent Stand Out 1-5 (2012) (telling the story of how data changed the attitudes of Google founders regarding elite credentials and altered the Google hiring model).

I have lived long enough to know that the changes I describe above are not necessarily going to be welcomed by many lawyers and law professors.  If a group benefits from a lifelong presumption of merit, it is natural that group will resist evidence that the presumption is not fully warranted. Indeed, much of the skepticism will be rooted in subconscious emotion.  If the presumption is dashed, those of us in the elite crowd will have to spend our days competing with others and proving ourselves, or even worse, watching our kids soldier through it.  We have little to gain and a lot to lose in the world we are heading into.  Yet, behind the Rawls veil of ignorance, how can we complain?

So with the red-blue crosscurrents, is law school still worth the investment?

That is a relevant and reasonable question that many young people are contemplating.  I will offer my opinion, but markets are bound to follow their own logic. 

This is a time of enormous uncertainty for young people. Education clearly opens doors, but tuition is going up much faster than earnings.  Further, competition among knowledge workers is becoming more global, which is a check on wages.  Of course, if you don't invest in education, what are your options?

I am generally on the side of Michael Simkovic and Frank McIntrye that the education provided by a law degree, on average, significantly increases lifetime earnings.  See The Economic Value of a Law Degree (April 2013).  How could it not?  The law is too interconnected to every facet of society to not, on average, enhance the law grad's critical thinking skills. Nearly 15 years of out of law school and I regularly use what I learned at Chicago Law to solve problems and communicate solutions, particularly in my applied research work with law firms and legal departments. While my Chicago Law credential has value independent of the skills and knowledge I obtained (the red AJD bar chart in Part I strongly suggests that), I can't deny the additional value of the actual skills and knowledge I obtained to solve real world business problems. It's been substantial.

In general, I also agree with Deborah Jones Merritt that there is significant evidence that the entry-level market for lawyers is weak and oversaturated.  See What Happened to the Class of 2010? Empirical Evidence of Structural Change in the Legal Profession (April 2015).   The class of 2010 is not faring as well as the class of 2000.  Indeed, the lead economist for Payscale, Katie Bardaro, recently noted that wages are stagnating in many fields, but especially in the legal profession. "More law schools are graduating people than there are jobs for them...There’s an over-saturated labor market right now. That works to drive down the pay rate.” See Susan Adams, The Law Schools Whose Grads Earn the Biggest Paychecks in 2014, Forbes, Mar. 14, 2014. 

In the face of these stiff headwinds, I think law schools have an opportunity to pack more value into three years of education. See Dimension 2 above.  To be more specific, if you are a protege of Dan Katz at Chicago-Kent, you will have a lot of career options. Ron Staudt, also at Chicago-Kent, has quietly built a pipeline into the  law and technology space.  Oliver Goodenough and his colleague at Vermont Law are making rapid progress with a tech law curriculum.  And at Georgetown Law, Tanina Rostain and Ed Walters (CEO of Fastcase) provide courses that are cutting edge.  

But absent these types of future-oriented instruction, what is the value of a JD degree as it is commonly taught today? That value is clearly positive; I would even call it high.  But whether the value is sufficient to cover the cost of attendance is likely to vary from law grad to law grad.  Lord knows, in a world of variable tuition based on merit scholarships and merit scholarships that go away after the 1L year, the swing in cost can be a $250K plus interest.

What is killing law school applications these days is the lack of near certainty among prospective students that the time and expense of law school will pay off.  The world looks different than it did in the fall of 1997 when the vast majority of the AJD respondents entered law school. Tuition and debt loads are higher and high paying entry-level jobs are harder to obtain.

So what is the solution?  For students, it's to bargain shop for law schools, which is bad news for law schools.  For law schools, it's to add more value to an already valuable degree.  Some of that value will come in the form of red technical skills that will make lawyers more productive.  In turn, this will prime demand for more legal products and services.

July 22, 2015 in Blog posts worth reading, Data on legal education, Data on the profession, Legal Departments, Structural change | Permalink | Comments (0)

Sunday, July 19, 2015

What is more important for lawyers: where you go to law school or what you learned? (Part I)

The Economist reports a very interesting analysis from Payscale.  The questions being asked are pretty simple: If you want to generate earnings that justify the time and cost of an undergraduate education, what should you study and where should you enroll?

Lots of people have strong opinions on this set of questions, but Payscale has the data to answer them empirically. It turns out that at the undergraduates level, course of study is much more important than the prestige of the college or university you attend.  The hard evidence is shown below.


For those working in law or thinking about attending law school, a natural question to ask is whether the legal industry is closer to the blue dot (art & humanities) or red dot pattern (engineering/CS/math).  A second, related question whether the future of law is more blue or more red.

This a two-part blog post.  Part I tries to answer the first question, starting with a careful analysis of the undergraduate chart, which provides a valuable frame of reference that can be discussed more dispassionately (at least among lawyers and law students) than an analysis that questions the value of law school prestige and hierarchy.  

Part II, which I will post on Wednesday, explores the second, future-oriented question.  I will tip my hand now and say that the future of law will be less blue (arts & humanity) and more red (math/CS/engineering).  Within the legal industry, there will be winners and losers; but from the perspective of broader society, this change is a very good thing. 

Undergraduate ROI

In the Payscale chart above, the y-axis (vertical) is 20-year annualized returns from college fees paid.  The x-axis is selectivity, running from under 10 percent to near open admissions.  

The Payscale chart is a very good example of how data visualization can be used to communicate both core facts and useful nuance.  Here, the lede is unmistakable:  the red dots (engineering/CS/math) are overwhelming higher on the ROI scale than the blue dots (arts & humanities).  Sure, there are exceptions to this rule, but they don't occur very often. (Observe how rarely a blue dot is above the red fit-line.) This suggests it would be very foolish to get a blue degree and expect a red paycheck unless you have very good information (or skills or talent) that others lack.

The chart conveys another important piece of information -- the red fit-line is flat.  This means that for engineering/CS/math majors, prestige has not been very relevant to their eventual earnings.  I'll add a nuance here that some empirically savvy readers are bound to point out:  It is possible (indeed likely) that fees are higher at more selective schools. So if MIT costs twice as much as a public polytech, and both yield 12% over 20 years, one might wish they had gone to MIT.   Still, the flat trendline is surprising.  As a general matter, lower ranked schools are not dramatically cheaper than higher ranked schools, and many public schools are highly selective.  The flat red trendline suggests that there are (or were, remember these are historical data) many bargains out there.  If one is trying to maximize financial returns, the goal is to find a school that will, in the future, be well above the red fit-line (and avoid those below).  

The flat red fit-line is also surprising because college selectivity is almost certainly highly correlated with ACT or SAT scores, which our society often views as measures of general intelligence. Yet, there we have it -- a flat trendline. Four years of education seem to be more relevant than a standardized test score taken during high school.  That is heartening at many levels.

A third interesting trend -- the blue fit-line is sloped downward.  This suggests that in the arts & humanities, selectivity/prestige does have a financial payoff.  I don't think this will surprise many readers, albeit the prestige payoff is not very large. To use a simple metaphor, if you attend a more selective college or university to get your arts or humanity degree, you are likely to have a better house in the arts & humanities neighborhood.  But on average, you won't be able to afford the same neighborhood as the engineers, computer scientists, and math majors.

What about Law?

Moving on to law, if we want to examine the relationship between earnings and law school attended, the best available evidence is probably the After the JD Study (AJD), which is large, representative sample of law graduates who took and passed the bar in 2000.  

Data from AJD Wave 3 suggests that the financial returns are relatively strong for all law school graduates -- ten years out and graduates of Tier 4 schools have median earnings of $100,000 per year. As shown in chart below, this is akin to shifting the blue dots up into the red territory.  


The downward sloping fit-line remains, but that doesn't seem to matter very much to happiness. Other AJD data shows that regardless of tier of graduating school, AJD respondents show relatively high and uniform satisfaction with (a) the decision to become a lawyer, and (b) the value of the law degree as an investment. By 2010, 48% of respondents had no debt; only 5.1% had more than $100K in educational debt remaining. 

This is all good news.  But is it reasonable to extrapolate forward and assume the past is a fairly accurate barometer of the present and the future? 

One way to address that question is to ascertain what has changed since 2000.  As noted earlier, the AJD sample was composed of law graduates who passed the bar in the year 2000. Figures published by NALP and the ABA show that the percentage of full-time bar passage required jobs has dropped significantly over the last 13+ years -- from 77.3% for the class of 2000 to 57% for the class of 2013. That is a huge delta.


One of the reasons why law school applicants have plummeted is that the career path from JD graduates has become murky.  And that is a good place to start Part II

July 19, 2015 in Blog posts worth reading, Cross industry comparisons, Data on legal education, Data on the profession, Structural change | Permalink | Comments (3)

Thursday, May 14, 2015

Further Thoughts on the July 2014 Bar Results -- A Response to Erica Moeser

Late last fall, Erica Moeser responded to a letter from Dean Kathryn Rand of the University of North Dakota (on behalf of a large number of law school deans), reiterating that the NCBE had double-checked its scoring of the MBE on the July 2014 bar examination and could find no errors in its calculations.  Erica Moeser also took to the pages of the December 2014 issue of The Bar Examiner to further validate her conclusion that the historic drop in the mean MBE scaled score is attributable solely to the fact that the class that sat for the July 2014 bar exam was “less able” than the class that sat for the July 2013 bar exam.  In January, Dean Stephen Ferruolo of the University of San Diego also wrote to Erica Moeser requesting the release of more information on which to assess the July 2014 bar examination results in comparison with previous years’ results.  In February, Erica Moeser responded to Dean Ferruolo’s request by declining to provide more detailed information and reiterating her belief that the July 2014 scores “represent the first phase of results reflecting the dramatic and continuing downturn in law school applications.”

In an earlier blog posting, I explained why Erica Moeser is partly right (that the Class of 2014 could be understood to be slightly less able than the Class of 2013), but also explained why the decline in “quality” of the Class of 2014 does not explain the historic drop in mean MBE scaled score.  The decline in “quality” between the Class of 2013 and the Class of 2014 was modest, not historic, and would suggest that the decline in the mean MBE scaled score also should have been modest, rather than historic.  Similar declines in “quality” in the 2000s resulted in only modest declines in the MBE, suggesting that more was going on with the July 2014 exam. 

Others have written about these issues as well.  In January, Vikram Amar had a thoughtful reflection on Moeser’s statements and in recent weeks Debby Merritt has written a series of posts -- here, here, and here -- indicating in some detail why she believes, as I do, that the ExamSoft debacle in July could have impacted the MBE scaled scores in jurisdictions that used ExamSoft as well as in other jurisdictions.

I write now to take issue with four statements from Erica Moeser – three from her President’s Page in the December 2014 issue of the Bar Examiner and one from her letter responding to Dean Kathryn Rand.  I remain unpersuaded that the historic decline in the mean MBE scaled score is solely attributable to a decline in quality of the class that sat for the July 2014 bar examination and remain baffled that the NCBE refuses to acknowledge the possibility that issues with test administration may have exacerbated the decline in the performance on the July 2014 MBE.

Item One – Differential Declines in MBE Scores

In her December article, Moeser stated: 

I then looked to two areas for further corrobo­ration. The first was internal to NCBE. Among the things I learned was that whereas the scores of those we know to be retaking the MBE dropped by 1.7 points, the score drop for those we believe to be first-time takers dropped by 2.7 points. (19% of July 2014 test takers were repeaters, and 65% were believed to be first-time takers. The remaining 16% could not be tracked because they tested in jurisdictions that col­lect inadequate data on the MBE answer sheets.) The decline for retakers was not atypical; however, the decline for first-time takers was without precedent dur­ing the previous 10 years. (Emphasis in original.)

Moeser starts by referencing data that is not publicly available to support her cause.  This is unfortunate, because it makes it really hard to understand and critique the data.  Nevertheless, there are some inferences we can take from what she does disclose and some questions we can ask.  Moeser asserts that the 19% of MBE “retakers” saw an MBE drop of 1.7 points compared with MBE “retakers” in July 2013, while the 65% believed to be first-time takers saw a drop of 2.7 points compared with first-time takers in July 2013.  It would have been helpful here if Erica Moeser would have released publicly the declines among MBE retakers in the previous 10 years and the declines among first-time takers in the previous 10 years so that patterns could be assessed, particularly in relation to the changes in class composition for each of those years.  Without that information available it is hard to do much more with Moeser’s assertion.  (I find it odd that she would reference this point without providing the underlying data.) 

Nonetheless, this assertion raises other questions.  First, the overall decline in the mean MBE scaled score was 2.8 points. Moeser notes that 19% of takers (MBE retakers) had an average drop of 1.7 points, while 65% of takers (first-time takers) had an average drop of 2.7 points.  Unless there is something I am missing here, that should mean the remaining 16% of test-takers had to have an average decline of 4.51 points!  (This 16% of test-takers represents those who Moeser notes could not be tracked as first-time takers or MBE retakers “because they tested in jurisdictions that collect inadequate data on the MBE answer sheets.”) (Here is the equation --- 2.8 = (.19*1.7)+(.65*2.7)+(.16*x).  Solve for X. This translates to 2.8 = .323+1.755+.16x.  This translates to .722 = .16x and then .722/.16 = X.  X then equals 4.51.)  It would have helped, again, if Moeser had indicated which jurisdictions had these even larger declines in mean MBE scaled scores, as we could then look at the composition of graduates taking the bar in those jurisdictions to see if there was an unusual decline in entering class statistics in 2011 at the law schools from which most bar takers in those states graduated.

Item Two – The MPRE

In the December article, Moeser also stated:

I also looked at what the results from the Multistate Professional Responsibility Examination (MPRE), separately administered three times each year, might tell me. The decline in MPRE performance supports what we saw in the July 2014 MBE numbers. In 2012, 66,499 candidates generated a mean score of 97.57 (on a 50–150 scale). In 2013, 62,674 candidates generated a mean score of 95.65. In 2014, a total of 60,546 candi­dates generated a mean score of 93.57. Because many MPRE test takers are still enrolled in law school when they test, these scores can be seen as presaging MBE performance in 2014 and 2015.

At first blush, this looks like a pretty compelling argument, but Moeser’s selectiveness in looking at the data is troubling, and her failure to discuss whether the MPRE and MBE are meaningfully comparable test-taking experiences also is troubling.  Essentially, Moeser is making the following assertion – because the mean MPRE scaled score declined by 1.92 points between 2012 and 2013, we should have expected a large decline in the mean MBE scaled score in July 2014 (and because the mean MPRE scaled score declined another 2.08 points between 2013 and 2014, we should expect another large decline in the mean MBE scaled score in July 2015).

But the “relationship” between changes in the mean MPRE scaled score and changes in the mean MBE scaled score over the last decade does not support this assertion. If one looks at a decade’s worth of data, rather than data just for the last couple of years, the picture looks significantly more complicated, and suggests the collective performance on the MPRE may not tell us much at all about likely collective performance on the MBE in the following year. 


Mean MPRE Score


MBE Year

July Mean MBE Scaled Score




































































The data Moeser cites from the last two years conveniently makes her point, but it consists of a very small sample size.  The data over the last decade looks much more random.  In three of the nine years, the change is not in the same direction (MPRE 2005, 2006, 2010, MBE 2006, 2007, 2011).  In the six years where the change is in the same direction, there are two years in which the MBE change is significantly larger than the MPRE change (MPRE 2007, 2009, MBE 2008, 2010) and there are two years in which the MBE change is significantly smaller than the MBE change (MPRE 2011, 2012, MBE 2012, 2013).  In only two of the nine years, do the changes in the MPRE and MBE roughly approximate each other (MPRE 2008, 2013, MBE 2009, 2014).   Nonetheless, this remains a very small sample and more analysis of data over a longer period might be helpful to better understand how/whether changes in mean MPRE scores inform meaningfully changes in mean MBE scores the following year.  At this point, I think the predictive value seems marginal given the wide range of changes on a year-over-year basis.

Item Three – Mean LSAT Scores

In the December article, Moeser further stated:

Specifically, I looked at what happened to the overall mean LSAT score as reported by the Law School Admission Council for the first-year matricu­lants between 2010 (the class of 2013) and 2011 (the class of 2014). The reported mean dropped a modest amount for those completing the first year (from 157.7 to 157.4). What is unknown is the extent to which the effect of a change to reporting LSAT scores (from the average of all scores to the highest score earned) has offset what would otherwise have been a greater drop. (LSAC Research Reports indicate that roughly 30% of LSAT takers are repeaters and that this num­ber has increased in recent years.

This assertion is misguided for purposes of this comparison, a point Vikram Amar made in his post.  If we were comparing the first-year matriculants in 2009 with the first-year matriculants in 2010, the question of the change in reporting from average LSAT score to highest LSAT score would have mattered.  But the 2010 matriculants were the first class for which the mean was reported based on highest LSAT score and the 2011 matriculants were the second class for which the mean was reported based on highest LSAT score.  Thus, there is no “unknown” here.  The reported mean LSAT dropped only a modest amount between the matriculants in 2010 and the matriculants in 2011.  Nonetheless, the mean MBE scaled score in July 2014 decreased by an historic 2.8 points from the mean MBE scaled score in July 2013. 

Item Four – Administration Issues

In her letter to Dean Kathryn Rand, Moeser stated:  "To the extent that the statement you attached referenced both administration and scoring of the July 2014, bar examination, note that NCBE does not administer the exam; jurisdictions do."

This response suggests not only that the NCBE is not responsible for administering the bar examinations in the many different jurisdictions, but implicitly suggests that issues with administration could not have contributed to the historic decline in the mean MBE scaled score. 

Were there issues with administration?  Yes.   Could they have contributed to the historic decline in the mean MBE scaled score?  Yes.

Debby Merritt’s recent posts discuss the administration issues and the potential consequences of the administration issues in some detail.  In over forty states that used ExamSoft to administer the bar examination, the MBE came on Wednesday, after the essay portion of the exam on Tuesday.  But because of an ExamSoft technical problem, tens of thousands of test-takers, who were initially informed by their respective state board of bar examiners that they would FAIL THE EXAM if their essay answers were not uploaded in a timely manner, spent most of Tuesday night dealing with the profound stress of not being able to upload their exam answers and not being able to contact anyone at the board of bar examiners (who were not answering phones) or at ExamSoft (due to the flood of calls and emails from anxious, frustrated, stressed out exam takers) to figure out what was going on and what they should do. 

Given that this “administration” issue caused untold stress and anxiety for thousands of test-takers, who spent Tuesday night completely anxious and stressed out trying repeatedly and unsuccessfully to upload their essay answers, should it be a surprise that they might have underperformed somewhat on the MBE on Wednesday?  (If you want a sense of the stress and anxiety, check the twitter feed for the evening of Tuesday, July 29, 2014)

The responses from the boards of bar examiners to this issue with administration of the bar examination were far from uniform.  Different jurisdictions granted extensions at different times of the night on Tuesday, July 29, or on Wednesday, July 30, with some granting short extensions and some granting longer extensions.  Thus, in states that gave notice of an extension out earlier on Tuesday, July 29, test-takers may have had less stress and anxiety, while in those states that didn’t give notice of an extension out until later (or for which the extension was relatively short), or where there may not have been any communication regarding extensions of the submission deadline, test takers likely experienced more stress and anxiety.  (It would be worth studying exactly when each jurisdiction gave notice of an extension and whether there is any correlation between timing of notice of the extension and the relative performance of bar takers in those states.)

The NCBE’s unwillingness to acknowledge any issues with administration of the bar examination is all the more surprising at a time when the NCBE is pushing for adoption of the Uniform Bar Examination.  On its webpage, the NCBE states: “[The UBE] is uniformly administered, graded, and scored by user jurisdictions and results in a portable score that can be transferred to other UBE jurisdictions.” (Emphasis added.)  This simply was not true in July 2014.  The Uniform Bar Examination was administered under different exam conditions across jurisdictions.  First, three of the states administering the Uniform Bar Examination in July 2014 did not use ExamSoft – Arizona, Nebraska and Wyoming -- and therefore, bar takers in those states had a vastly different “exam administration” experience than bar takers in ExamSoft jurisdictions.  Across ExamSoft jurisdictions, different approaches to extensions also meant different administration experiences. Given the significance of consistent administration for the purpose of equating performance on a standardized exam like the bar exam, that the NCBE allows such varied approaches to administering a supposedly “uniform” exam strikes me as very problematic.

Many questions remain unanswered, largely because adequate information has not been made available on which to assess the various factors that might have contributed to the historic decline in the mean MBE scaled score.  With the release of February bar results and the NCBE’s publication of the 2014 statistical report, some additional information is now available to put the results of July 2014 in context.  In my next blog posting regarding the July 2014 bar results, I will delve into some of those statistics to see what they tell us.

(Edited as of May 20 to correct the 2013 MPRE and 2014 MBE change and corresponding discussion.)

May 14, 2015 in Current events, Data on legal education, Data on the profession | Permalink | Comments (0)

Saturday, February 7, 2015

The Early Days of Legal Analytics

LexMachina-logo1There is an interesting story in Forbes on Lex Machina, a legal start-up that provides analytics for use in patent litigation.  See Dan Fisher, Stanford-Bred Startup Uses Moneyball Stats to Handicap Judges, Forbes, Feb. 2, 2015.  The company was created by faculty at Stanford Computer Science and Stanford Law.  As the company emerged from the University, the reigns were handed to Josh Becker, a Stanford JD-MBA.  To date, the company has raised $8 million in start-up funding.  According to the Forbes article, the company's clients include some of the nation's large technology companies plus one-third of the AmLaw 100.

What makes Lex Machina so interesting is that the company is not a NewLaw service provider that trying to take marketshare. Instead, Lex Machina is a toolmaker.  It is a true Big Data company that provides analytics to (a) value contested patents and (b) protect/maximize that value through a litigation strategy that is informed by data.  

The impact of Lex Machina is hard to decipher, primarily because if it does provide an edge, the customers are unlikely to be too vocal. Just like a hedge fund with an effective trading strategy, why advertise the ingredients of your secret sauce? Indeed, compared to other toolmakers (e.g., predictive coding, expert systems) Lex Machina's benefits are less about efficiency and more about affecting the outcomes of cases -- who wins and by how much.  If Lex Machina is truly delivering, it will eventually touch-off a Big Data legal analytics arms race akin to the quant revolution on Wall Street.  Dan Katz frequently makes this point, and I think he is right.  The Forbes article makes the point that Lex Machina is already moving into adjacent areas of IP law and general commercial litigation.  

The broader legal industry is unlikely to notice Lex Machina until it has a substantial liquidity event -- i.e., it's acquired or goes public, making if founders far richer than the BigLaw partners and in-house lawyers they currently serve.  

If we are looking for early signs of a tipping point for legal analytics, one marker may be the number of Stanford Law grads who are turning down entry-level opportunities in BigLaw to pursue legal start-ups.  In recent years, Stanford Law grads fresh out of law school have gone on to found other venture-backed legal start-ups like Ravel Law, Judicata, and Law Gives.  Back in 2013, The Stanford Lawyer (SLS alumni magazine) had an extensive write-up with several examples.  See Sharon, Driscoll, A Positive Disruption, June 4, 2013.  In 2014, Stanford's CSO offered a program titled, An Alternative to BigLaw -- Startups.

The legal world isn't going away; it's just changing.

February 7, 2015 in Cross industry comparisons, Current events, Data on the profession, Innovations in law, New and Noteworthy, Structural change | Permalink | Comments (0)

Sunday, January 4, 2015

Size of the US Legal Market by Type of Client

Washington, DC.  The AALS Section on Professional Responsibility hosted a vigorous discussion today on the evolving ethical duty of competency, a topic partially inspired by the recent changes to Model Rule 1.1 cmt. 8 (requiring lawyers to stay abreast of the "benefits and risks associated with relevant technology").  As part of this panel, I showed a chart on the size of the US legal market, which was promptly tweeted by CALI 's Director of Community Development, Sarah Glassmeyer, a law librarian who is a total data subversive in a style and manner I fully support.

Well, despite a less-than-optimal photo angle, the chart was retweeted and favorited, so I figured I ought to just post the actual chart here. [Click on to enlarge] Legal Market

In a competitive market, the threshold question, asked by potential entrants and those who might finance them, is often the same: "what is the size of the available (or addressible) market?" Because lawyers and law schools are feeling unprecedented economic pressure, I thought it would be worthwhile to run this exercise for the U.S. legal industry and break it down by type of client.

The figures above are estimates of 2014 receipts going to organizations and individuals in the business of providing legal services.  My calculations are derived from US Census Bureau data. They exclude the cost of in-house and government lawyers.  More granular calculation details will be laid out in a forthcoming publication.

At today's AALS Professional Responsibility session, technology was framed as an ethical issue. And that is certainly right:  technology can deliver enormous cost and quality benefits to clients, so we have both a fiduciary and professional duty to be up-to-date.  Yet, there is a flip-side here that is crucially important -- to ignore or fall behind on technology is to run the risk of commercial ruin. This axiom applies to lawyers in private practice and to law schools that want employers to hire their graduates. 

Building upon that theme, I used the Market Size chart to make two points today, one based on the high-end corporate market (right side of chart) and the other directed toward the individual consumer market (left side of chart). 

Re the corporate side, the data show that a relatively small roster of large corporations are spending vast sums each year on legal services -- more than $10 million per year for a publicly held company.  Because large national and international corporations are awash in a sea of growing legal complexity, they are turning to technology, process, and data to keep legal costs in line with overall company revenues.  From the perspective of a large corporate client, the typical junior law firm associate has little to offer.  A more seasoned partner or counsel is a better value, but this is by virtue of experience rather than technology or process.  As a result, law firm hiring remains stagnant, and more legal work is being taken in-house or given to LPOs or New Law legal service providers like Axiom, Elevate, or Novus Law.  It may take a generation for the law school--law firm--legal department supply chain to come into a reasonable alignment.  Right now, it's broken.

Re the individual retail market, the $232 annual legal spend per citizen means that there is not enough money go around to pay for all the legal need.   If a middle-class professional couple with kids has a contested divorce, that could easily chew-up $50,000 to $100,000 in legal fees.  A DUI is likely to cost $1,500.  A worker's comp claim might be 30% of an award.  Probate work runs well into the thousands.  In reality, most citizens go without.  One of our co-panelists today, retired US Magistrate Judge John Facciola, made the claim that 83% of American never talk to a lawyer to help them with a legal problem.  "The middle class is largely gone from federal court."  To my mind, technology is the only vehicle for tapping into a large latent market for legal services.  LegalZoom, Rocket Lawyer, Modria, Shake, and many other legal technology companies all see the potential here. And so do the venture capital and private equity firms that are funding them. 

 Today's panel was one of the most lively I have ever attended at AALS, owing in part to my excellent co-panelists but also an audience that asked some great, tough questions.  Many thanks to Andy Perlman (Suffolk Law) for organizing a terrific session and Natasha Martin (Seattle) for her skillful moderation of the panel.

January 4, 2015 in Current events, Data on the profession, Legal Departments, New and Noteworthy, Structural change | Permalink | Comments (2)

Tuesday, November 11, 2014

What Might Have Contributed to an Historic Year-Over-Year Decline In the MBE Mean Scaled Score?

The National Conference of Bar Examiners (NCBE) has taken the position that the historic drop in the MBE Mean Scaled Score of 2.8 points between the July 2013 administration of the bar exam (144.3) and the July 2014 administration of the bar exam (141.5) is solely attributable to a decline in the quality of those taking a bar exam this July.  Specifically, in a letter to law school deans, the NCBE stated that:  “Beyond checking and rechecking our equating, we have looked at other indicators to challenge the results.  All point to the fact that the group that sat in July 2014 was less able than the group that sat in July 2013.”

Notably, the NCBE does not indicate what other “indicators” it looked at “to challenge the results.”  Rather, the NCBE boldly asserts that the only fact that explains an historic 2.8 point drop in the MBE Mean Scaled Score is “that the group that sat in July 2014 was less able than the group that sat in July 2013."

I am not persuaded.   

(Neither is Brooklyn Law School Dean Nicholas Allard, who has responded by calling the letter “offensive” and by asking for a “thorough investigation of the administration and scoring of the July 2014 exam.”  Nor is Derek Muller, who earlier today posted a blog suggesting that the LSAT profile of the class of 2014 did not portend the sharp drop in MBE scores.)

I can’t claim to know how the NCBE does its scaled scoring, so for purposes of this analysis, I will take the NCBE at its word that it has “double-checked” all of its calculations and found that there are no errors in its scoring.

If we accept the premise that there are no scoring issues, then the historic decline in the MBE Mean Scaled Score is attributable either to a “less able” group taking the MBE in July 2014 or to issues associated with the administration of the exam or to some combination of the two.

The NCBE essentially has ignored the possibility that issues associated with the administration of the exam might have contributed to the historic decline in the MBE Mean Scaled Score and gone “all in” on the “less able” group explanation for the historic decline in the MBE Mean Scaled Score.  The problem for the NCBE is that it will be hard-pressed to demonstrate that the group that sat in July 2014 was sufficiently “less able” to explain the historic decline in the MBE Mean Scaled Score.

If one looks at the LSAT distribution of the matriculants in 2011 (who became the graduating class of 2014) and compares it with the LSAT distribution of the matriculants in 2010 (who became the graduating class of 2013), the NCBE probably is correct in noting that the group that sat in July 2014 is slightly “less able” than the group that sat in July 2013.  But for the reasons set forth below, I think the NCBE is wrong to suggest that this alone accounts for the historic drop in the MBE Mean Scaled Score.

Rather, a comparison of the LSAT profile of the Class of 2014 with the LSAT profile of the Class of 2013 would suggest that one could have anticipated a modest drop in the MBE Mean Scaled Score of perhaps .5 to 1.0.  The modest decrease in the LSAT profile of the Class of 2014 when compared with the Class of 2013, by itself, does not explain the historic drop of 2.8 reported in the MBE Mean Scaled Score between July 2013 and July 2014.


The “group” that sat in July 2014 is comprised of two subgroups of takers – first-time takers and those who failed a bar exam and are retaking the bar exam.  I am not sure the NCBE has any basis to suggest that those who failed a bar exam and are “retaking” the bar exam in 2014 were a less capable bunch than a comparable group that was “retaking” the bar exam in 2013 (or in some other year).

What about “first-time takers”?  That group actually consists of two subgroups as well – those literally taking the exam for the first time and those who passed an exam in one jurisdiction and are taking the exam for the “first-time” in another jurisdiction.  Again, I am not sure the NCBE has any basis to suggest that those who passed a bar exam and are taking a bar exam in another jurisdiction in 2014 were a less capable bunch than a comparable group that was taking a second bar exam in 2013.

So who’s left?  Those who actually were taking a bar exam for the very first time in July 2014 – the graduates of the class of 2014.  If we accept the premise that the “retakers” in 2014 were not demonstrably different than the “retakers” in 2013, than the group that was “less capable” in 2014 has to be the graduates of 2014, who the NCBE asserts are “less capable” than the graduates of 2013.


The objective criteria of the class that entered law school in the fall of 2011 (class of 2014) is slightly less robust than the class that entered law school in the fall of 2010 (class of 2013).  The question, however, is whether the drop in quality between the class of 2013 and the class of 2014 is large enough that we could anticipate that it would yield an historic drop in the MBE Mean Scaled Score of 2.8 points? 

The answer to that is no.

The difference in profile between the class of 2014 and the class of 2013 does not reflect an “historic” drop in quality and would seem to explain only some of the drop in MBE Mean Scaled Score, not a 2.8 point drop in MBE Mean Scaled Score.

To understand this better, let’s look at how the trends in student quality have related to changes in the MBE Mean Scaled Score over the last decade. 

Defining “student quality” can be a challenge.  A year ago, I noted changes over time in three “groups” of matriculants – those with LSATs at or above 165, those with LSATs of 150-164, and those with LSATs below 150, noting that between 2010 and 2013, the number at or above 165 has declined significantly while the number below 150 has actually grown, resulting in a smaller percentage of the entering class with LSATs at or above 165 and a larger percentage of the entering class with LSATs below 150. 

While the relatively simplistic calculations described above would provide some basis for anticipating declines in bar passage rates by 2016, they would not explain what is going on this year without more refinement.

In his blog posting earlier today, Derek Muller attempts to look at the strength of each class by calculating "projected MBE" scores drawing on an article from Susan Case and then comparing those to the actual MBE scores, showing some close relationship over time (until this year). I come to a similar conclusion using a different set of calculations of the "strength" of the graduating classes over the last several years based on the LSAT distribution profile of the matriculating classes three years earlier.

To develop this more refined analysis of the strength of the graduating classes over the last nine years, I used the LSAC’s National Decisions Profiles to identify the distribution of matriculants in ten five-point LSAT ranges – descending from 175-180 down to 130-134.  To estimate the “strength” of the respective entering classes, I applied a prediction of bar passage rates by LSAT scores to each five point grouping and came up with a “weighted average” bar passage prediction for each class. 

(In his article, Unpacking the BarOf Cut Scores, Competence and Crucibles, Professor Gary Rosin of the South Texas College of Law developed a statistical model for predicting bar passage rates for different LSAT scores.  I used his bar passage prediction chart to assess the “relative strength” of each entering class from 2001 through 2013. 


Prediction of Success on the Bar Exam Based on Lowest LSAT in Range





















Please note that for the purposes of classifying the relative strength of each class of matriculants, the precise accuracy of the bar passage predictions is less important than the fact of differential anticipated performance across groupings which allows for comparisons of relative strength over time.)

One problem with this approach is that the LSAC (and law schools) changed how they reported the LSAT profile of matriculants beginning with the entering class in the fall of 2010.  Up until 2009, the LSAT profile data reflected the average LSAT score of those who took the LSAT more than once.  Beginning with matriculants in fall 2010, the LSAT profile data reflects the highest LSAT score of those who took the LSAT more than once.  This makes direct comparisons between fall 2009 (class of 2012) and years prior and fall 2010 (class of 2013) and years subsequent difficult without some type of “adjustment” of profile in 2010 and beyond.

Nonetheless, the year over year change in the 2013-2014 time frame can be compared with year over year changes in the 2005-2012 time frame.

Thus, having generated these “weighted average” bar passage projections for each entering class starting with the class that began legal education in the fall of 2002 (class of 2005), we can compare these with the MBE Mean Scaled Score for each July in which a class graduated, particularly looking at the relationship between the change in relative strength and the change in the corresponding MBE Mean Scaled Score.  Those two lines are plotted below for the period from 2005-2012.  (To approximate the MBE Mean Scaled Score for graphing purposes, the strength of each graduating class is calculated by multiplying the weighted average predicted bar passage percentage, which has ranged from .801 to .826, times 175.)

Comparison of Class Strength Based on Weighted Average Class Strength (Weighted Average Bar Passage Prediction x 175) with the MBE Mean Scaled Score for 2005-2012


What this graph highlights is that between 2005 and 2012, year to year changes in the MBE Mean Scaled Score largely “tracked” year to year changes in the “quality” of the graduating classes.  But perhaps most significantly, the degree of change year over year in “quality” generally is reflected in the “degree” of change year over year in MBE Mean Scaled Scores.  From 2008 to 2009, the drop in “quality” of 1.5 from 144.6 to 143.1 actually was reflected in a drop in MBE Mean Scaled Scores from 145.6 to 144.7, a drop of 0.9 points.  Similarly, from 2009 to 2010, the drop in “quality” of 1.1 from 143.1 to 142 actually was reflected in a drop in the MBE Mean Scaled Scores from 144.7 to 143.6, a drop of 1.1 points.  This two-year drop in quality of 2.6 points from 144.6 to 142 corresponded to a two-year drop in MBE Mean Scaled Scores of 2.0 points from 145.6 to 143.6.

How does this help us understand what has happened in 2014 relative to 2013?  The decrease in quality of the class of 2014 relative to the class of 2013 using the “Weighted Average Bar Passage Projection” methodology above reflects a change from 145.1 to 144.2 – a drop of 0.9 (less than the year over year changes in 2009 and 2010).  Accordingly, one might anticipate a decline in MBE Mean Scaled Scores, but probably a decline slightly smaller than the declines experienced in 2009 and 2010 – declines of .9 and 1.1 point, respectively. 

Does the decline in quality between the Class of 2013 and the Class of 2014 explain some of the decline in MBE Mean Scaled Scores?  Certainly.  This analysis suggests a decline comparable to or slightly less than the declines in 2009 and 2010 should have been expected.

But that is not what we have experienced.  We have experienced an historic decline of 2.8 points.  Yet, the NCBE tells us that in looking at other indicators “all point to the fact that the group that sat in July 2014 is less able than the group that sat in July 2013.” 


What the NCBE fails to discuss, or even mention, is that there is one other “indicator” that was a distinctive aspect of the bar exam experience for the group that sat in July 2014 that the group that sat in July 2013 did not experience – the ExamSoft Debacle

For many of those in one of the many jurisdictions that used ExamSoft in July 2014, the evening between the essay portion of the bar exam and the MBE portion of the bar exam was spent in needless anxiety and stress associated with not being able to upload the essay portion of the exam.  This stress and anxiety were compounded by messaging that suggested the failure to upload in a timely manner would mean failing the bar exam (which messaging was only corrected late in the evening in some jurisdictions). 

In these ExamSoft jurisdictions, I can only imagine that some number of those taking the MBE on the second day of the exam were doing so with much less sleep and much less focus than might have been the case if there had not been issues with uploading the essay portion of the exam the night before.  If this resulted in “underperformance” on the MBE of just 1%-2% (perhaps missing two to four additional questions out of 200), this might have been enough to trigger a larger than expected decline in the MBE Mean Scaled Score.


It will be hard to assess the full reality of the July 2014 bar exam experience in historical context until 2015 when the NCBE releases its annual statistical analysis with state by state analyses of first-time bar passage rates.  It is very difficult to make comparisons across jurisdictions regarding the July 2014 bar exam at the present time because there is no standardized format among states for reporting results – some states report overall bar passage rates, some disaggregate first-time bar passage rates and some states report school specific bar passage rates.  To make meaningful comparisons year-over-year focused on the experience of each year’s graduates, the focus should be on first-time bar passage (even though as noted above, that also is a little over inclusive).

Nonetheless, the experience of one state, Iowa, casts significant doubt on the NCBE “story.”

The historical first-time bar passage rates in Iowa from 2004 to 2013 ranged from a low of 86% in 2005 to a high of 93% in 2009 and again in 2013.  In the nine-year period between 2005 and 2013, the year to year “change” in first-time bar passage rates never exceeded 3% and was plus or minus one or two percent in eight of the nine years.  In 2014, however, the bar passage rate fell to a new low of 84%, a decline of 9% -- more than four times the largest previous year-over-year decline in bar passage rates since 2004-2005.













First Time Bar Passage Rate

























Change from Prior Year
























The NCBE says that all indicators point to the fact that the group that sat in 2014 was “less able” than the group that sat in 2013.  But here is the problem for the NCBE.

Iowa is one of the states that used ExamSoft in which test-takers experienced problems uploading the exam.  The two schools that comprise the largest share of bar exam takers in Iowa are Drake and Iowa.  In July 2013, those two schools had 181 first-time takers (out of 282 total takers) and 173 passed the Iowa bar exam (95.6% bar passage rate).  In 2014, those two schools had 158 first-time takers (out of 253 total) and 135 passed the Iowa bar exam (85.4% bar passage rate), a drop of 10.2% year over year. 

Unfortunately for the NCBE, there is no basis to claim that the Drake and Iowa graduates were “less able” in 2014 than in 2013 as there was no statistical difference in the LSAT profile of their entering classes in 2010 and in 2011 (the classes of 2013 and 2014, respectively).  In both years, Iowa had a profile of 164/161/158.  In both years, Drake had a profile of 158/156/153.  This would seem to make it harder to argue that those in Iowa who sat in July 2014 were “less able” than those who sat in 2013, yet their performance was significantly poorer, contributing to the largest decline in bar passage rate in Iowa in over a decade.  The only difference between 2013 and 2014 for graduates of Drake and Iowa taking the bar exam for the first time in Iowa is that the group that sat in July 2014 had to deal with the ExamSoft debacle while the group that sat in July 2013 did not.


This analysis does not “prove” that the ExamSoft debacle was partly responsible for the historic decline in the MBE Mean Scaled Score between 2013 and 2014.  What I hope it does do is raise a serious question about the NCBE’s assertion that the “whole story” of the historic decline in the MBE Mean Scaled Score is captured by the assertion that the class of 2014 is simply “less able” than the class of 2013.

When the NCBE issues its annual report on 2014 sometime next year, we will be able to do a longitudinal analysis on a jurisdiction by jurisdiction basis to see whether jurisdictions which used ExamSoft had higher rates of anomalous results regarding year-over-year changes in bar passage rates for first-time takers.  When the NCBE announces next fall the MBE Mean Scaled Score for July 2015, we will be able to assess whether the group that sits for the bar exam in July 2015 (which is even more demonstrably “less able” than the class of 2014 using the weighted average bar passage prediction outlined above), generates another historic decline or whether it “outperforms” its indicators by perhaps performing in a manner comparable to the class of 2014 (suggesting that something odd happened with the class of 2014).

It remains to be seen whether law school deans and others will have the patience to wait until 2015 to analyze all of the compiled data regarding bar passage in July 2014 across all jurisdictions.  In the meantime, there is likely to be a significant disagreement over bar pass data and how it should be interpreted.

November 11, 2014 in Data on legal education, Data on the profession, Scholarship on legal education, Scholarship on the legal profession | Permalink | Comments (4)

Sunday, October 12, 2014

Is the Legal Profession Showing Its Age?

The figure below suggests that a growing number of students are attending law school but not going on to become lawyers.  This conclusion requires some explanation, which I will supply below.  Alternative explanations are also welcome, as I’d like to find a plausible narrative that foreshadows a brighter future for the licensed bar. [PDF version of this essay]


I generated this figure based on data from various editions of The Lawyer Statistical Report, which is periodically compiled by the American Bar Foundation (ABF).  The ABF's gets the underlying data from Martindale-Hubbell, which is a comprehensive directory of the licensed bar.  As of 2005, the sample was roughly 1 million lawyers who work in law firms, solo practice, in-house legal departments, government, and the judiciary.

The big surprise here is that the proportion of young lawyers (under age 35) has been declining for several decades.  And not by a little, but by a lot.  During this period, the median age went from 39 in 1980, to 41 in 1991, to 45 in 2000, to 49 in 2005.  See ABA Market Research Department.

I would be tempted to attribute a demographic shift of this magnitude to a computational error.  But that is unlikely because the underlying data were calculated at four different points in time, yet the results come together to produce a single, steady trendline -- a trendline that shows a licensed bar that is steadily aging.  

Another possible factor to consider is whether there are any significant data collection or sampling issues that skew the data in a manner that dramatically undercounts younger lawyers. For example, Martindale-Hubbell is largely irrelevant to today's younger lawyers.  So, in solo and small firm practices, where they are making the business decisions, we might expect plummeting subscription rates.  But subscribing and requesting the publication of additional biographical information (in the hope of garnering referral business) is not the same thing as being listed. Martindale Hubbell attempts to track lawyers who did not subscribe to the directory, as the near-universe level of inclusion increases the directory's value.  

To illustrate this point, consider that in 2000, the Lawyer Statistical Report (which relies on Martindale-Hubbell data) counted 909,000 lawyers.  According to the ABA, the total number of lawyers licensed in the US (compiled from state bar roles) was 1,022,000, and that almost certainly includes some double counting of lawyers licensed in more than one state.  While I have no doubt that younger lawyers are becoming harder to hunt down because of cell phones and home-based offices, the gap of missing lawyers is just not big enough to fully account for the sharp drop-off in younger lawyers. 

I have shown this chart to various law firms, legal departments, law faculty and bar association audiences.  Through this process, I have developed two working theories that are not mutually exclusive:

  1. Increased exits from law practice based on gender integration
  2. Slowing absorption of law graduates into the licensed bar

Theory 1: Gender Integration

One explanation is gender integration.  In short, over the last 40 years, more women have entered the legal profession; and as an empirical matter, they are much more likely to exit the workforce in order to focus on childcare.  Thus, more gender integration over time would cause a proportional decline in the younger lawyer cohort.

So let's examine the data.  According to the figure below, which shows number of male and female 1Ls enrolling by year at ABA-accredited law schools [Click-on to enlarge], the high water mark for male 1L enrollment occurred over 40 years ago -- in 1971!   The high water mark for female enrollment in percentage terms was 2000 (49.4%). In absolute numbers, the high was the class entering in the fall of 2009 (24,305).  


Presumably, the higher the percentage of female graduates, the lower the percentage of lawyers under the age of 35.  In 1968, a 22 year old female 1L, if she graduated from law school and stayed in the legal profession, would be part of the younger lawyer cohort in 1980. Yet, her 1L cohort included only 1,179 females (7.4% of all 1Ls).  By 1993 (12 years before 2005), the number of female 1Ls had increased to 19,059 (nearly 44%). So exits based on childcare factors would likely be increasing.  

I can readily accept gender integration as a partial, but not a complete, explanation.  Why? Because female exits are likely to be siphoning off a substantial portion of the over 35 cohort, as this group is still having and raising children.   It seems implausible that female lawyers are leaving in droves before age 35 (reducing the younger lawyer cohort) yet returning in droves thereafter (swelling the over 35 cohort).  Further, according to the figure above, the absolute number of law school graduates is increasing during this entire period.  Sheer numbers are likely a partial counterbalance to the impact of gender-related exits.

Theory 2:  Slowing Absorption of Younger Lawyers

It is important to keep in mind the magnitude of the overall slide in younger lawyers -- from 36% in 1980 to 13% in 2005.  One would think the trendline would be moving in the exact opposite direction -- that larger graduating classes would be replacing the much smaller number of law school graduates from 40 years earlier who were retiring or passing away.  But such a youth movement does not appear to be happening, at least based on data through 2005.

I think the most likely explanation is that the rate of absorption of law school graduates into the licensed bar has been steadily declining over time.  This explanation, which would affect men and women equally, is directionally consistent with the percentage of entry-level jobs in private practice, which has been declining since in the late 1980s. See figure below.


The slower absorption theory is also directionally consistent with the shifting demographics of large law firms, which now have more partners than associates.  See figure below.


Despite the higher number of partners compared to associates, it is worth noting that large law firms are not becoming more generous in sharing the partnership pie.   

Rather, the real sea change is the decline in the number of traditional law firm associates, who have been slowly supplanted by staff attorneys, permanent of counsel lawyers, and nonequity partners. Indeed, over 40% are large law firm partners (defined at AmLaw 200 / NLJ 250) are nonequity. Three decades ago, this category of partner was relatively rare.  See Henderson, An Empirical Study of Single-Tier Versus Two-Tier Partnerships in the Am Law 200, 84 NC L Rev 1691 (2006).  The growth of nonequity partners reflects a new kind of law firm leverage that relies on senior lawyers. The annual ALM/Major Lindsay & Africa study of partner compensation reveals that equity partners make dramatically higher incomes than nonequity partners and that the size of the pay gap is widening over time. See Ross Todd, A Widening Partner Pay Gap, American Lawyer, Sept 29, 2014.  

The primary advantage of nonequity partners and other senior lawyers, like permanent counsel, is that training costs fall to near zero. Cf. Elizabeth Olson, Corporations Drive Drop in Law Firms’ Use of Starting Lawyers, Study Finds, New York Times, Oct. 10. 2014 (showing drop over time in use of first year associates because clients are refusing to pay for training costs).

To my mind, however, the most persuasive support for the lower absorption theory is the simple delta between the growth in the licensed bar--which has clearly hit a plateau--and the size of graduating classes from ABA-accredited law schools--which, until recently, had been steadily increasing. The figure below shows these macro-level trendlines.


If younger lawyers were replacing older lawyers and also growing to keep pace with the broader economy, the under 35 young lawyers cohort would be getting bigger or at least remain relatively constant in size.  But instead, as the first figure in this essay showed, the younger lawyer cohort has gotten smaller.  Arguably, the simplest explanation for these patterns is that it has gotten much harder over time to parlay a JD degree into paid employment as a licensed lawyer.  So, faced with a saturated legal market, law school graduates have been pursuing careers outside of law.  

What Does This Mean?

The analysis above suggests that the JD Advantage / JD Preferred employment market started to take shape several decades ago, long before these terms were put in place by the ABA and NALP.  Yet, we really don't know about these careers.  To construct a more useful, informative narrative, we'd have to systematically study the career paths of our alumni.  That task is long overdue.  

I started teaching at Indiana Law in 2003.  Since I first saw the declining trendline for the young lawyer cohort, I have been thinking about the roughly 1,600 students who have taken my Corporations, Securities Regulation, Business Planning, Project Management, Law Firms as a Business Organization, and Legal Professions classes.  

  • What percentage are working as licensed lawyers?  
  • For those doing something different, where are they working?  
  • Has their legal education opened doors for them? 
  • Did those doors lead to interesting and remunerative work?

The After the JD Study is based on law school graduates who passed in the bar in the year 2000.  The Wave III results provide some clues to how at least one cohort of younger lawyers fared during their first ten years in practice.  

  • Roughly a quarter of the class of 2000 is no longer practicing law (remember the base sample excluded those who never took or passed the bar).
  • The migration out of practice is generally in the direction of private sector business.  
  • Ten years out, the median pay for full-time work is more than $100,000 for both men and women.  No tears need to be shed here.
  • Roughly three-quarters report being satisfied with their decision to attend law school. 

These statistics are generally encouraging, but some caution is in order, as the entry-level legal economy was quite different in 2000.  

Because of the law school transparency movement, we lack commensurable data between 2000 and 2013.  That is an important piece of information right there, as changes in collection and reporting standards were caused by student protests, including several lawsuits surrounding allegedly misleading employment data. Yet, we can cobble together some potentially useful comparisons:

Even if NALP's full-time legal positions in 2000 is a more expansive category than the ABA's full-time bar-passage jobs in 2013, the gap is startling -- over 20 percent!  Further, we have additional evidence of a major shift in the job market, as law firm summer associate positions have declined in size by more than 50% since in the early 2000s. See Henderson, Sea Change in the Legal Market, NALP Bulletin (Aug 2013).   Between 2008 and 2013, there has also been a drop in median starting salaries, from $72,000 to $62,500. See NALP, Employment for the Class of 2013 – Selected Findings

Demand Drops, but Supply Marchs On

Cumulatively, the trendlines presented in this essay suggest that we are on the tail end of a multi-decade structural shift in the legal economy.  So what comes next?

Law schools were recently taken to task in an editorial by the Young Lawyers Board of Philadelphia Legal Intelligencer.  See If Unchanged, Legal Education will Remain a Business in Decline, Legal Intelligencer, Sept 25, 2014.  According to the young lawyers, "One reason graduates have difficulty obtaining employment is that most of them need to be trained in how to practice law, and clients are unwilling to pay for training new lawyers. Law schools need to step up and train students on how to practice law."

I am very sympathetic to the young lawyers, but I think they are missing something essential.  A law school that improves the quality of its skills training reduces the training costs to prospective employers.  That is a good thing, but it does not change the underlying demand for legal services. And it appears that that demand is eroding on several fronts:  (a) wealthy corporations are balking at the price of outside counsel and looking for credible substitutes, (b) ordinary citizens are struggling to afford a lawyer at all, and (c) a new segment of the legal economy is emerging that is financed by nonlawyers and heavily focused on data, process, and technology, which taps into skill sets not traditionally taught in law school. See Henderson, A Counterpoint to "The most robust legal market that ever existed in this country", Legal Whiteboard, Mar 17, 2014.  


My own conclusion is that neither the organized bar nor the legal academy has a firm grip on the changes that are occurring in the legal marketplace.  This uncertainty and confusion is understandable in light of the magnitude of the shift.

Nonetheless, these market shifts create special urgency for legal educators because we can't teach what we don't understand.  The thesis of the Young Lawyers Board is surely right -- if unchanged, legal education will remain a business in decline.  Much of legal education today is premised on a 20th century professional archetype--an archetype that is, based on the data, becoming less and less relevant with each passing day.  Thus, we are under-serving our students.  And frankly, they are figuring that out.  

Change is hard for people and organizations they work in.  And law professors and law schools are no different.  The retooling of legal education will likely be a slow, painful process that will take the better part of a full generation to complete.  I am trying to do my part.

Yet, the brunt of the demographic shift falls on the licensed bar, which is getting older and thus weaker with each passing year.  This is a problem that belongs to the ABA, the state bars, and the state supreme courts, not the legal academy.   [PDF version of this essay]

October 12, 2014 in Data on the profession, Structural change | Permalink | Comments (1)

Sunday, August 24, 2014

Ahead of the Curve: Three Big Innovators in BigLaw

Nashville, TN.  It is time to put down the broad brush used to paint BigLaw as inefficient and out of touch.  At least for me, that is the big takeaway from the 2014 International Legal Technology Association (ILTA) conference, which took place this past week at the Gaylord Opryland Hotel in Nashville and included nearly 2,000 lawyers, administrators, staff, and vendors from around the world.

My takeaway is based on what I saw during the presentation session for the ILTA Most Innovative Law Firm Award.  The three finalists all qualify as big:  Bryan Cave (985 lawyers), Seyfarth Shaw (779 lawyers), and Littler Mendelson (1002 lawyers). Presenters from each firm had 15 minutes to share their innovations followed by 5 minutes of Q&A.  Afterwards, ILTA members in attendance casted ballots for first, second, and third place.

Kudos to Bryan Cave, Seyfarth Shaw, and Littler Mendelson for publicly sharing their innovations, as it demonstrates a commitment to the broader legal profession.

In this post, I will describe the salient points of each innovation. I will err on the side of detail because, when it comes to innovation in the legal space, there is a short supply of “guts of the operations” commentary.  I will then offer some macro-level observations.  As it turns out, BigLaw has on balance a surprisingly good hand to play.  Many will thrive, but at the expense of taking market share from the rest.


Bryan Cave

Presenter: John Alber, Strategic Technology Partner

Bryan Cave has developed an ingenious and highly efficient way to educate its lawyers on the economics of its business.  Prior to the presentation, I was familiar with the firm’s investment in a rigorous cost accounting system to guide the firm’s strategy and operations.

Yet, to get the full benefit out of such a system, the understanding needs to filter down to the individual lawyer-timekeeper level so that each lawyer-timekeeper can use the superior data to allocate time and effort in ways that strengthen the enterprise.  Even in the year 2014, many successful and skilled BigLaw lawyers confuse revenues with profit. And the confusion is understandable because portable books of business, which tend to be measured in terms of revenue, drive the valuation of lateral partners.  See Henderson & Zorn, Of Partners and Peacocks, Am. Law., February 2014.

Based on what I saw at ILTA, such confusion appears to have been substantially eliminated at Bryan Cave. No_math_arithmophobia

The core Bryan Cave innovation is a simple dashboard that tracks a variety of statistics at the lawyer, practice group, and firm level.  What is most striking about the Bryan Cave initiative is the sensitivity shown to the large percentage of lawyers who are not comfortable processing numbers (“arithmophobia” was the term used in the presentation).  The Bryan Cave innovation team dealt with this constraint in two ways.

1. The Octagon.  The Octagon is a data visualization technique that communicates eight key metrics in an octagon-shaped graphic.  Wondering what the term "data visualization" means? It's finding graphical ways to communicate complex multivariable data in a format that requires the end user, such as a lawyer, to have very little technical training.  The Octagon is a textbook example. It uses colors and distance from the center of the graphic to convey essential information related to origination, client relationships, matter management, days to bill, days to collect, hours billed, leverage, and profit margins. (There may be other octagons containing other metrics--the one we were shown appeared to be geared toward partners.)

Each lawyer each month gets a new updated Octagon; and that graphic communicates, through its shape, the lawyer’s relative contributions to the firm.  Specifically, there are distinctive patterns well known within the firm that tend to signal rainmaker, service partner, project manager, technical specialist, or some blend thereof.  The features of the Octagon also communicate how well a lawyer is performing in his or her various roles relative to his or her peers.  So, on a monthly basis, self-image confronts hard numbers.

This type of transparency is bound to have a profound effect on behavior.  (During another ILTA session I heard, from another Bryan Cave presenter, that since the introduction of the Octagon a couple of years ago, the average days to collect has fallen from 60 to 44.)

2. The Rosetta. Some lawyers are bound to prefer a story rather than a picture.  For these lawyers, the firm has created a narrative, referred to as the Rosetta, that translates the numbers into a diagnostic story of strengths, weaknesses, and, most importantly, specific prescriptive advice on how to improve.

But there is an interesting catch—the stories are all written with a computer algorithm.  How is this possible?  It’s a technology pioneered by a company called Narrative Science.  Note that computers that are fed nothing but a traditional baseball scoring sheet now routinely write sports stories that summarize the game for the local sports page.  This narrative summary accompanying the Octagon removes any lingering ambiguity regarding what the diagram means.  Further, all report generation, including practice-group level Octagon and Rosetta reports, has been entirely automated.

I am told that the Octagon and Rosetta programs can handle, and properly incentivize, work that is done on either a billable or alternative fee arrangement basis. If this is true, Bryan Cave has an innovation designed for the legal market of the future.

Some readers may be turned off that the Bryan Cave innovation may seem, on the surface anyway, entirely focused on law firm financial performance.  I am not. To my mind, this type of technology is valuable for communicating the fundamentals of the business.  This reduces the myths and false narratives that routinely take hold in data-poor environments.  This innovation is also timely because it is getting harder to give clients superior value while also delivering a strong return to the firm's owners -- the best of whom could lateral to another firm tomorrow.

The challenge of every BigLaw firm is getting all of the firm's stakeholders to row in the same direction. The combination of the Dashboard, Octagon, and Rosetta is a breakthrough in lawyer communication and, by extension, change management.  Bryan Cave attorneys have the information they need to both build their practices while also advancing the broader goals of the enterprise.

Seyfarth Shaw Seyfarth

Presenters: Kathy Perrelli, Chair of Litigation Practice; Kim Craig, Global Director of Legal Project Management.

Seyfarth Shaw’s innovation is the creation of a true Research & Development Department staffed by lawyers, project managers, technologists, and software developers.  The charge of Seyfarth’s R&D Department is to build solutions in advance of perceived client needs.  As the presenters mentioned, “we are not doing this because our clients are asking for these solutions; we are doing this because our clients will ask.”

Continue reading

August 24, 2014 in Blog posts worth reading, Current events, Data on the profession, Innovations in law, Law Firms | Permalink | Comments (1)

Thursday, May 1, 2014

What Ails the Large Law Firm? Will the Real FutureFirm Please Stand Up

TimeinabottleFive years ago this April, I helped organize a novel experiment on how to reengineer the modern law firm.  The occasion was FutureFirm 1.0, a collaborative competition in which teams of law firm partners, associates, and in-house lawyers to create a strategic plan for the fictional firm of Marbury & Madison (M&M).  The goal was a new business model that would enable the firm "to survive and thrive over the next 20 years."  See M&M Fact Pattern.   

We planned FutureFirm 1.0 in the fall of 2008, but by April 2009, things looked pretty unstable.  Deal flow had ground to a halt, and corporations were reluctant to fund noncrucial litigation.  Law firms in turn were rescinding offers to thousands of law students.  Further, the specter of law firm failure hung in the air.  Suffice it to say, the timing was not right for sharing the results of FutureFirm.  As a result, my analysis of the event, "What Ails the Large Law Firm?  Will the Real FutureFirm Please Stand Up," was never published or circulated.  

With five year anniversary of FutureFirm 1.0, I decided to uncork my time-in-a-bottle essay and post it on SSRN and JDSupra.  

Having not read this essay for five years, I am surprised at how well the FutureFirm analysis holds up.  Yet, the biggest takeaway from my FutureFirm experience is not the specifics of the analysis, but acclimating myself to the permanence of new change dynamic, much of which I can see through the participants of FurtureFirm 1.0.   

  • Two law firm partners subsequently left to start their own boutiques, one of which is aggressively moving into managed services in South Africa.  
  • Another law firm partner became a judge in King County, Washington (Seattle).
  • Several summer associates joined BigLaw only to leave within three or four years to become sophisticated in-house lawyers who are themselves driving change.
  • Several people in all roles have switched over to the business side.  Indeed, new legal businesses are actively being planned.

In the spring of 2014, the new normal is here to stay, and it has no froth.  FutureFirm was probably a fringe activity back in 2009.  Now, an event like FutureFirm would be one of the key places to go for answers.  Indeed, I have very serious senior in-house lawyers at Fortune 100 companies who want to run this type of colloborative competition to help better design tomorrow's legal departments.  So stay tuned for that.

I hope you are sufficiently curious to do a bit of time travel and give "What Ails the Large Law Firm?" a read.  I would welcome your thoughts and feedback. 

May 1, 2014 in Data on the profession, Innovations in law, Law Firms, Legal Departments, Structural change | Permalink | Comments (0)