The Legal Whiteboard

These Data Will Fundamentally Reshape the Legal Education Industry

Just a few days ago, the ABA Section on Legal Education and Admission to the Bar posted on its website, in downloadable spreadsheet format, the employment outcomes for the Class of 2011 (here). In a few short years, these data are going to fundamental reshape our industry.  The changes will make the industry better and stronger, but the journey to this better place is going to painful and disorienting for all law schools—that’s right, even the elite national law schools will be affected.

This is worth explaining in very simple and concrete terms.  The Class of 2011 employment data consists of 134 variables on 200 ABA-accredited law schools-- 26,800 discrete data points, which is enough fill a phonebook.  For a long time, the policy of the ABA was to do just that – publish a phonebook of data in the form of the ABA-LSAC Official Guide to Law Schools.  Well, decisions on where to attend law school are not free.  They require time and effort.  When a prospective student has to wade through a phonebook to assemble relevant data to make important decisions, many (most) will forgo the exercise altogether. 

This has two very important effects:

1. The quality of enrollment decisions goes down because, from the student perspective, the costs are too high.  That’s error #1.  But it is forgivable—decisionmaking is a skill taught in a top-notch legal program, not a prerequisite for applying.
2. To simplify their decisions, students gravitate to U.S. News ranking, which is a compact 4-page table that contains easy-to-understand comparative data. 

Yes, the U.S. News has serious flaws; and every year, overreliance on them produces tragic consequences in the form of excessive student debt.   Now, with the ABA employment "phonebook" in spreadsheet format, those a with modicum statistical skills and an internet connection can analyze, simplify and publish relevant statistics that will better inform the decision to attend law school. 

Consider the chart below:

When I created this simple pie chart, I started with a simple premise: If I am applying to law school, my minimum hope is that nine months after graduation I will be able to obtain a full-time, permanent professional job.  The phonebook has three columns of data that speak to this hope:

1. Bar Passaged Required Jobs, FTLT (i.e., Full-time, Long-Term)
2. JD-Advantage Jobs, FTLT
3. Professional Jobs, FTLT. 

All the other myriad data columns, parsing things by part-time, short-term, non-professional, unemployed, unknown, etc., do not meet the minimum hope.  So they are lumped together as “Other Outcomes.”  Clearly, for 1/3 of the Class of 2001, their full-time, permanent professional ambitions have not yet materialized. 

A reasonable next question is how these figures vary by U.S. News rank. The answer is reflected in the chart below.

Some observations:

• Least surprising. The outcomes are better at “national law school”--almost 90% are FTLT professional jobs.  I used the T14 cutoff because the composition of this group has not changed in two decades of U.S. News rankings. Few people would disagree that these schools have strong pull among legal employers.

• Most surprising.  There is a whole lot of Purple--i.e., “Other Outcomes”--throughout Tiers 1, 2, 3, and 4.  For over 90% of law schools, this is a very challenging legal market.  

• Biggest reality check.  The JD-Advantaged and Professional jobs appear to be, on balance, less desirable than those requiring bar passage. They increase nearly three times in relative proportion as we move from T14 to Tier 1 to Tier 2.  Many are likely compromise jobs—not as good as practicing law, but better than non-professional alternatives.

With these relative benchmarks in place, a prospective law student can look for law schools that are outperforming their U.S. News rankings.  And there are quite few. 

For example, in Tier 4, St. Mary’s (Texas) has 78.3% Bar Passage Required placement; Mississippi College’s figure is 75.3%; and Campbell (NC) is 71.4%.  These schools aren’t feeding BigLaw, but their graduates appear to be full-time practicing lawyer nine months after graduation.  What accounts for their success? Most of their graduates are probably in cities and towns far away from corporate practice.  Nonetheless, these schools have a clear niche they are filling.

In contrast, there are 20 schools in Tiers 1 and 2 that have less than 50% Bar Passage Required jobs.  What do they have in common?  Many are in big cities in the Northeast and Mid-Atlantic or California--large urban markets that are attractive for young professionals.  It is likely that too many young lawyers are chasing after a finite set of legal jobs.  It is worth noting, however, that these same schools also have rates of placements in JD Advantage and Professional jobs that are higher than other law schools at statistically significant levels. 

So many young law graduates are voting with their feet.  Better to stay in the city as a non-lawyer professional than to move to the South to be country lawyer doing small firm practice.  Although I suspect a large proportion of these grads will fare quite well, it is important to keep in mind that JD-Advantaged and Professional jobs are not a panacea—they are also in short-supply.  At most schools in Tiers 1, 2, 3, and 4, between 30% and 42% of graduates are either unemployed and underemployed in jobs that are either nonprofessional, part-time, or short-term.   Indeed, 4.3% (1,874) of all jobs for the Class of 2011 were funded by the law schools themselves!

As I have said previously (here and here), the current legal job market reflects a structural change in the legal sector—these numbers aren’t going to turn around in a year or two.

So what is going to happen?  Notwithstanding the heady optimism of the “Kaplan kids”, the ABA employment data, thanks to the blogosphere, is going to reduce information costs, making it easier for prospective law students to determine whether law school is a good investment.  The needle is going to move, just not as fast as a Chicago School economist might predict.

Further, expect students to aggressively negotiate for scholarship money.   Whether schools become more generous in merit aid, admit fewer students, or both, all signs point to shrinking budgets for law schools.

The utter transparency of a changing and stagnant legal market has potentially more dire consequences for law schools.  The lifeblood of the entire legal education establishment, including elite law schools, is federal student loans.  Our students get the same generous terms as graduates of medical and dental schools, who are not struggling to make six figure incomes.  The graphs above suggest that a large proportion of our students will be on Income-Based Repayment (IBR), which is – functionally – insurance in the event a high income fails to materialize in the years following graduation.  The downside risk of that insurance – lack of repayment of expected principal and interest—is borne by U.S. taxpayers. 

Right now, it is possible to estimate the size and probability of this downside risk. All the Federal Government has to do is add-up the shortfall between the repayment of principal and interest in normal repayment versus the monies actually being collected.  What percentage of graduates are on IBR?  What portion of their current principal and interest are they able to pay?  These are simple numbers that some enterprising journalist will eventually request. Further, they are legitmate public policy questions that we, the legal academy, should face long before the journalists get there.

Lawyers and law schools are not a favored interest group on Capitol Hill.  We need to plan for the extremely high probability that the financing of law schools will be dramatically altered in the years to come.  The longer we wait, the more painful and disastrous the transition.  Every law school will need a damn good story to justify continued federal loans.  And right now, many of us lack that story – being in Tier 1 or T14 (where debt loads tend to be the highest) won't mean anything if the math falls short.

In summary, our ivory tower is crumbling.  With the ABA putting the employment data in downloadable format on its website, law schools will have to do something completely new and scary to us—we are going to have to compete to keep our jobs and stay in business.  The litmus test is going to be the ability of our graduates to obtain remunerative professional work in a highly competitive global economy.   This is very serious work.

[Posted by Bill Henderson]

"Graduates Shun Legal Profession"

That's the headline of a story in the Sunday edition of The Age, one of Australia's leading daily newspapers.  Here is the nut:

ALMOST two-thirds of Australia's law graduates are not working as lawyers four months after they have completed their degrees, according to a study.

The Graduate Careers Australia survey of 1313 recent graduates from all over the country found that 64 per cent were not practising law between 2010 and 2011.

There was ''no way'' law firms could accommodate all the graduates from Australia's 31 law schools, La Trobe University's director of undergraduate studies, Heather King, said. ''It's a well-acknowledged fact that 40-50 per cent will not end up in a traditional law practice.''

These statistics may seem even bleaker than those that describe the U.S. legal market.  Yet, for two key reasons, these Australian students are far better off. First, the Australias follow the LLB model, which has some substantial advantages.  According to my Australian colleagues, a law undergraduate degree is often combined with a major in another field or discipline, such as business, accounting, sociology, or literature.  So a student's commitment to law as career is often tentative and, in many cases, hedged by another career interest.  Second, higher education in Australia enjoys a large national subsidy.  So law graduates typically graduate with little or no debt.

Ironically, as the story reports, some Australian universities are moving toward the J.D. model, essentially concluding the law is best taught to more mature students as a graduate discipline.

I agree that students ought to have a cost-effective way to opt out of law.  I also agree that law is best taught as a graduate discipline to students with some substantial life experience.  I don't, however, see an easy way to cost-effectively achieve both. The fact that the solution is not easy will, conversely, make the solution quite valuable.

[posted by Bill Henderson]

Review of Failing Law Schools, by Brian Tamanaha (Part I)

[Update: I edited the review below to remove three paragraphs from my analysis.  It was a metaphor that was not key to my review of Brian's book yet could be fairly viewed as insulting to readers I both respect and hoped to persuade.  I am sorry about that. It was a substantial change, so I am acknowledging it here. wdh.]

Many legal academics are going to dismiss Brian Tamanaha's book, Failing Law Schools, without ever reading a page.  A larger number may simply ignore it.  That is ironic, because this is the response one would expect if Tamanaha's account of a corrupt, self-indulgence academic culture were true. 

I have lived inside this culture since I joined the academy in 2002. And I can attest that very few people inside the academy believe that we are living the high life on the backs of our students. But in the year 2012, that perception does not matter very much.  Rather, the perception that matters is the one from the outside looking in. 

Over the last eighteen months or so, The New York Times, The Wall Street Journal, The Washington Post, The Atlantic, the legal press and countless blogs (many written by unhappy students) have relentlessly hammered away at law schools.

The  lay public, including most practicing lawyers, are looking for a definitive account that can explain the legal education's maelstrom.  Tamanaha's account is a veritable Brandeis Brief on what went wrong, chocked full of facts and history and persuasive analysis. 

It begins with a deal between the ABA and AALS to join forces to persuade the state bars to restrict entry to ABA accredited law schools (the ABA's goal) and thereby to elevate the stature of the legal professoriate (the AALS's goal).  Once this deal was struck -- in the early 20th century -- pretty much every change accrued to the benefit of the law faculties: higher salaries, lower teaching loads, the advent of administrators to lighten the burden of governance, and more freedom to pursue scholarly interests.  When U.S. News & World Report ranking appear in the early 1990s, the law schools are forced to make choices.  And our collective behavior suggests that vanity and prestige are all-too-likely to trump important principles like student diversity or honesty in reporting data.

For us law professors, here is our conundrum. From the outside looking in, things look bad, even corrupt.  Yet we don't feel we have done anything wrong.  We are certain that we lack the intent to cheat or defraud.  But that, unfortunately, is error #1.  As we all know, establishing intent is always a matter of circumstantial evidence.  So let's review that evidence from the perspective of the neutral fact finder.

Life is objectively good for us:  We have high salaries, social prestige, lots of travel, job security, and near absolute freedom to organize our time outside the three to six hours a week we teach, 30 weeks a years.  Against this backdrop, there is consensus among legal employers that we are not very good at practical training including, in the eyes of many, basic legal writing.  Moreover, the overproduction of lawyers creates problems for the legal profession as a whole.  Similarly, our students  are saddled with enormous debt and nothing we are doing curricularly seems geared to solving their burgeoning unemployment  or underemployment problem.  The federal government finances this "system."  And through Income-Based Repayment programs, the U.S. taxpayers are backstopping our high costs.

Because law faculty seems to be getting the long end of the bargain here, our subjective feelings of honesty and rectitude are unlikely to be viewed by many students, practicing lawyers, or the broader public as credible.  In fact, they may be viewed as insincere or out of touch.  How did things get so badly out of kilter? 

But for Tamanaha, some pesky journalists, angry students, and the ticking time-bomb of law students debt, I am confident that we law professors could coast along on our present track for another several decades.  As an insider, I can honestly testify that we believe--sincerely beheve--that we care about our students, the quality of their education, their debt loads, and their future job prospects.  But  looking at the same set of facts, history will draw its own conclusions. And Tamanaha, akin to a lawyer building a case, offers up a very compelling narrative that the dispassionate observer is likely to find convincing. 

Other bloggers and news outlets have commented on Tamanaha's book, often drawing very different conclusions. Compare Brian Leiter's Law School Updates and Orin Kerr at Volokh Conspiracy (Tamanaha's argument has merit, particularly when he suggests that lower ranked law schools should consider changing their models), with Scott Greenfield at Simple Justice (here and here) (Tamanaha describes an insular, out-of-touch professoriate from the top down that distains the input of practicing lawyers) and the Chronicle of Higher Education (subscription req'd)  (describing Tamanaha's thesis, "Law schools are bloated with too many underworked, overpaid professors whose salaries are supported by tuition increases that are making law school a losing bet for many students").

What are the proper inferences to draw? 

In late 2011, I reviewed a copy of Tamanaha's book as part of the peer-review process for University of Chicago Press.  My primary advice to Brian, communicated directly to him as well as his editors, was "to condemn the sin, not the sinner."  Legal academics may seem culpable for privileging their interests ahead of students, I said, but these are the same folks who need to be relied upon to fix the problem. (The alternative is that nearly all of U.S. legal education will collapse under the weight of high costs and fewer entry level legal jobs; and on many days, I think the latter is just as likely as the former.)  

Frankly, I don't know if my "condemn the sin, not the sinner" recommendation was good advice.  In order to change, the legal academy may need more pressure brought to bear from outside forces. This may happen if the legal academy is painted as more selfish, insular, elitist and out of touch than we already look now.  Congress and the Department of Education hold the ultimate trump card, and Tamanaha's book provides the essential supporting evidence for radical action.  If and when this happens, law faculties will be forced to pick sides.  

History is now playing out right before our eyes.  I believe there is a good chance that Brian Tamanaha's book will be viewed--by history at least--as a great act of courage.  The implication, of course, is that the rest of us will look foolish.

Brian discusses the bleak employment prospects of law schools, but (through no fault of his own) understates the nature of the structural change that is occuring in the U.S. and global market for legal services.  In Part II, I will write about some logical next steps for law schools looking to get ahead of the coming tsunami. 

[posted by Bill Henderson]

Unsustainable Law Student Debt

At Balkanization, Brian Tamanaha (Washington University) continues to shine a bright light on law student loan statistics.  See The Quickly Exploding Law School Debt Disaster. 

Viewing recently released 2011 data, Brian cites 17 law schools where the average debt exceeds $135,000 per student.  The vast majority of students at these schools will be forced into Income Based Repayment (IBR), which is, functionally, a federally administered insurance program for indebted law school graduates who fail to make a high five-figure or low six-figure income. It caps debt payment at 15% of income above some basic poverty level threshold. (In future years, it will drop to 10%.)  The downside of IBR is that unpaid interest is quickly capitalized, so a graduate's total debt load explodes upward, making it very difficult to afford things like a car or a home using debt finance. Then, as Brian suggests, buyer's remorse is going to set in for a whole generation of law school graduates. 

As a political issue, this is not going away.  I agree 100% with Brian's final line:  "This financial insanity will not stop until significant changes are made to the federal student loan program."  When this happens, every law school in the U.S. will be be affected.  As I said last week, it is time we get our houses in order.

[posted by Bill Henderson]

A Reply to the Empiricists at NWU Law

My blog post from last week, "Too Good for BigLaw: The Statistician Edition" has resulted in a minor kerfuffle with some of the distinguished empiricists at Northwestern Law.  See Dan Rodriguez, Law School Sorting and the Partnership Track:  Northwestern Empiricists Weigh In, Word on the Streeterville [The Blog of NWU Law Dean].  NWU Law folks were not impressed with my analysis.  Dan Rodriguez was gracious enough to send me the link at the same time his post, quoting the views of his colleagues, went live.  He has also encouraged me to reply publicly. 

I am happy to do that.  Let me start with big picture issues.  Then, for those folks with the curiousity and stamina to wade through arcane details--and experience tells me this is a small group--I will directly address, point by point, the the issues raised by Kate Litvak and Max Schanzenbach.  But at the outset, I will say that I am not conceding any ground.

Big Picture

First off, it is important to understand why this topic generated so much traffic.  See Above the Law, ABA Daily Journal, TaxProf, Instapundit,  Adam Smith. 

It all started with a provocative blog post by Vivia Chen, the columnist for The Careerist.  Vivia reviewed hiring and promotion data from the NLJ 250 Law School Hiring Survey and noted that elite law school graduates were becoming partner in very low numbers when compared to the hiring pipeline. Vivia editorialized on the numbers in a way that played into readers' fragile egos and insecurities.  Of course, that is her job, which she does very well.

In a nutshell, here is why people care -- or more precisely, get anxious -- about this topic:  it is conventional wisdom that graduation from elite law schools produces better career outcomes.  When that expectation is countered by actual marketplace data, people are surprised.  See, e.g., Bruce MacEwen,  "The Best & The Brightest" at  Adam Smith Esq. (leading blog on law firm economics).  Surprise is the first reason this issue got so much play.  Emotion is the second.

Emotion matters because very few lawyers and law professors are dispassionate on this topic.  When it comes to conventional wisdom on law school pedigree, we all have horses in the race.  Because we are human beings, we lawyers and law professors don't wait for balanced market data to develop our own entrenched worldviews.  When the conventional wisdom favors us, we go with it -- albeit we aren't really conscious this is happening. So when data upset the apple cart and potentially make us look complacent, our passions get aroused. 

The folks at Above the Law have built a entire business model around such predictable lawyer foibles.  The more chum thrown in the water, the higher the ad revenues.  It's just that simple.

Vivia's primary point, stated through metaphor, is that regional schools (such as Chicago Loyola) seem to be making partner at higher rates than the elite schools (such as Chicago).  This is a reasonable inference because the ratio of associates hired to partners promoted appears to be consistently high for elite law schools and very low for a large number of regional law schools.  This very point was made independently by Bruce MacEwen, who is a very sophisticated guy who advises law firms on strategy.

That said, there was ample opportunity for readers to draw spurious inferences from Vivia's metaphor-driven blog post.  Thus, to avoid any school-specific claims (a 1-year crossectional sample is not suitable for such a purpose), I pooled the schools by U.S. News ranking, drawing a line between elite and non-elite at the T14 mark.  Why T14?  Because these schools have played a closed loop of musical chairs for 20 years in the U.S. News rankings.  These schools would be viewed by most employers as "national" law schools.  

Here is what the data showed:

• Pipeline in:  53.7% T14, 46.3% non-T14
• Partners Promoted: 29.4% T14. 70.6% non-T14.

That is, well, an enormous skew. In 2011, for every 5.43 elite grads hired, a senior associate from an elite school makes partner.  For non-elite schools, that corresponding statistic is 1.95.  Vivia found these numbers surprising and somewhat counter-intuitive.  So did Bruce MacEwen, Above the Law, ABA Journal, etc.

There are ways to break down these numbers to gain additional insights, but the key point here is one of magnitude.  Elite law graduates are supposed to be smarter and more capable -- no one expects these folks to be on short side of any race, tournament or desired outcome.  The magnitude of hiring/promotion gap is the surprising fact that needs to be explained.

I had observed roughly the same skew several years ago (pooled 2007 and 2008 data) and alluded to it in this article, "Why is the Job Market Changing," Nat'l Jurist (Nov. 2010).  I also follow other relevant studies, such as The After the JD, which have noted differences between elite and non-elite graduates.   So I had a head start in thinking through possible explanations. I thus offered five theories, all of which could work in concert, to explain the large skew in the data:

1. Selection effects
2. Differences in first jobs
3. Intergeneraional privilege
4. Influence of admission criteria on the associate pipeline
5. "A Better Plan B" for elite grads

So, to be very clear, I am not using the NLJ 250 data to support the above theories.  It is the reverse: I am offering the above theories as a likely explanation for the very large skew between elite and nonelite grads.  Framed as a open-ended research question, it might be written, "why are elite grads not becoming BigLaw partners in numbers commensurate with hiring patterns and general presumptions of their higher ability?"  That is a mystery and a puzzle.

Statistics Minutiae  [After the jump ...]

Each of my five theories could be its own independent research study.  But that's a side show.  Testing the theories will not change the skew.  Don't forget the big picture -- we have a  counterinituitive data point that needs to be explained. 

A better approach is to triangulate what we know from historical patterns, published studies, the data itself, theory and commonsense and apply Occam's Razor to find a simple set of explanations that explain all the data points.  My blog post never made any strong claims on relative importance or validity of my theories.  I was just trying to formulate a reasonably plausible set of explanations.  When something is highly contestable and important, then we can go to the trouble of formal hypothesis testing.  But we have five theories here.  It is unlikely all five are wrong.  If they were, we'd need a sixth killer theory, heretofore unknown, to explain the skew.  Again, Occam's Razor.

To my mind, this is the source of disagreement between myself, Kate and Max:  they are focused on poking holes in my theories, but I want a satisfactory explanation for this very large skew.  Max doesn't offer any explanation, and Kate's explanations quickly wilt when Occam's Razor is applied.

From this point forward, this post may be of interest to readers with a large appetite for the inner workings of inferential statistics.

Here is Kate's macro level critique:

In short, the averages he is discussing cannot tell us anything useful. The main problem is selection effects combined with the level of data aggregation: (1) no individual-level data on hired/promoted lawyers; (2) all top-250 law firms are grouped together; (3) all non-T14 law schools are grouped together.

Unfortunately, I don't know what "useful" means here.  My analysis unearths some data points that are surprising and counter-intuitive -- so says the blogosphere.  I think the averages, viewed in light of my five theories, are potentially useful if I were a law firm partner and wanted to understand unwanted attrition in my law firm-- a potential multimillion dollar problem. If I am law professor, I might also find it useful to adjust my worldview so that fits surprising and counter-intuitive data points.  After all, we are occasionally called upon to provide career advice. 

Regarding Kate's three points on data limitations, it is possible break out the data as Kate suggests.  And when broken down, it boosters rather than detracts from my initial analysis. Let me be clear, however, on the precise sample I am working with:  

• It is called the NLJ 250 Law School Hiring Survey, but the sample is not complete.  Here is what it includes:  155 firms supplied data.  127 are AmLaw 200 (AmLaw 200 is based on revenues), and 27 firms not in the Am Law 200, but in the NLJ 250 (NLJ 250 is based on lawyer counts).
• Of the 127 AmLaw firms in the sample (63.5% response rate), median profits per partner is $970,000 ($1.2 million average) versus $807,000 ($1.1 million average) for the 73 that did not supply data.  So the sample is not comprised of the less profitable firms; they are, if anything, more profitable. The difference in means, however, is not statistically significant.
• The sample firms are larger: 567 median, 819 average for sample; 401 median, 472 average for the missing firms.

Based on the above, the sample appears to be a reasonable representation of the BigLaw universe.  Further, we know the directionality of the missing data, which is useful for interpreting any results. 

[One more sampling point: Kate objects to grouping all non-T14 schools together.  I disagee.  There is no point in breaking down the law schools further by U.S. News tier -- Andy Morriss and I documented that during the halycon days of the mid-2000s, outside the Top 25 or so, large law firm jobs were limited to between 0 and 15% of NLJ 250 entry level jobs.  In contrast, T14 was typically well north of 50%, even with large numbers of judicial clerkships.  There is one relevant divide here: regional versus national.  A few regional schools, such GW, Fordham, Emory, UCLA, USC and Texas, have a strong foothold for students in the top 30% or 50%.  After that, the grade cutoffs turn all other law schools to near decimal dust in terms of market pull.  I have parsed the data in previous years--these in-between schools are, well, in-between the regional and nationals in terms of outcomes.  Breaking them out won't tell us anything.]

As I noted in the original post, and pointed out by Kate, we almost certainly have massive selection effects at work here.  We know from other research that T14 grads gravitate to the most prestigious firms where the washout rates are very high.  See Zaring & Henderson.  The value of the partnership (higher profits) in combination with better outplacement prospects may make this tradeoff rational.  Sure enough, there are large selection effects: 61% of the T14 graduates in the sample joined firms in 2011 in the Top 50 as measured by Profits per Partner (PPP).  This supports the First Jobs theory. 

That said, even in the high prestige firms, non-elite grades are getting more than their pipeline's share of the partnership promotions (or, stated differently, T14s are leaving in larger proportions): 

• Among the 25 most profitable firm in the Am Law 200, T14 accounted for 76.0% of associates hired, compared to 24.0% for non-elites  graduates.  But for partnership, the numbers fall to 60% versus 40%.
• Same pattern if we broaden to 50 most profitable firms:  71.4% T14 hired, 28.6% non-T14;  for partners, 55.6% versus 44.4%.
• The pattern continues for AmLaw 200 firms outside the Top 50 PPP bracket:  40.5% T14 hired, 59.5% non-elite; for partners, 24.0% T14, 76.0% non elite. 

So, using the underlying raw numbers, let's translate these figures into ratios of associates hired to partners promoted:

• Top 25 PPP:    T14 = 10.1, non-T14 =3.9
• Top 50 PPP:    T14 = 4.8, non-T14 = 1.9
• Non Top 50:    T14 = 3.9, non-T14 = 1.8
• Non-AmLaw:   T14 = 2.0, non-T14 =0.8

In every bracket in the NLJ 250/AmLaw200, the T14 get superior hiring access but much lower rates of partnership.  Again, this is Ted Seto's research in another form. 

We can get more granular.  Among the 79 firms in the 2011 sample that (a) hired T14 and non-T14 associates, and (b) promoted at least one T14 lawyer to partner, there were 51 firms (65%) where the non-14 were getting promoted at rates that exceeded their pipeline percentages. Further, compared to the remaining 28 firms, these 51 firms had higher average Revenues per Lawyer ($772,000 versus $711,000) and higher average Profits per Partner ($1.2M versus $1.0M).  Perhaps non-Top 14 partners are good for business!

Had Kate had access to these data, she may not have replied so boldly. After hiring, T14 does not fare well in any bracket.  Again, why is that?

I am not saying the above data supports my five theories; rather, I am saying some combination of my five theories likely explain a good portion of the large, counter-intuitive gaps between elite and non-elite law school graduates. Here is the story in five bullet points:

• Selection Effects. If you are at a T14 school, BigLaw jobs are relatively plentiful. Interview well and you get the big paycheck.  In contrast, the Non-T14 crowd are comprised of a larger proportion of people who had to work very hard to get their foot in the BigLaw door.  These types of behaviors generally don't end after one gets hired. They become habits.
• First Jobs.  The T14 are going to the most prestigious firms where the washout rates are high. We showed this above. But the T14s washout in greater proportion than their Non-T14 peers.  So, by itself, this can't explain the overall NLJ 250/AmLaw 200 gap.
• Intergenerational Privilege.  We know from the After the JD study that elite law schools grads come from more affluence families.  See Dinovitizer & Garth, Lawyer Satisfaction in the Process of Structuring Legal Careers.  This is also the story line of Thomas Mann's Buddenbrooks novel -- affluence can produce less disciplined behavior among our progeny.
• Ratings-Driven Admissions Criteria.  We now throw away personal statements, resumes and letters of references when making admission decisions.  20 or 30 years ago we consulted commonsense when making admissions decisions.  So the pipeline to elite law schools, and hence to the NLJ 250, has likely been affected in deleterious ways.
• "A Better Plan B."  Elite grads have more options, presumably in gov't and business.  This is obvious.  But it is too speculative a category to explain the whole elite/non-elite gap, which is very large.

Kate's critiques in order:

• Selection effects:  Kate says I can't rule out that that the non-T14 crowd is getting hired and promoted because they have, say, important client connections.  Why would I try to rule this out?  It is a de minimis effect at best.  It fails Occam's Razor.
• First Jobs.  Kate says "probably true."  We can stop right there.  We are looking for the most plausible explanations.  Nonetheless, the data above are also responsive to rest of Kate's criticism--the T14/non T14 pattern holds up in every part of the BigLaw hierarchy.
• Intergenerational Privilege.  Kate thinks I don't have the data to document T14's higher family wealth.  But The After the JD research does have these data.  Further, Garth and Dinovitzer make an empirical case that it affects satisfaction in the large firm environment.  In short, we have a literature on this topic -- I am merely pointing to it.
• Rankings effect.  Kate thinks I lack any concrete data to make this claim.  She is right.  But we have all watched law schools make insane trade-offs in admissions to goose their rankings.  I am making the leap that this affects the labor pool in unhealthy ways.  Supply chain analytics are important in other industries.  Law firms are waking up to this.
• "A Better Plan B."  We both agree that we need better data to make sweeping claims on this one.  We also agree that elites will have the edge.  It is just a matter of measuring the size and direction (gov't, business, public interest, academia) of the edge.

What I find puzzling about Kate's comments is that she never addresses the large gap between T14 hiring and promotions.  To my mind, that is the storyline -- the puzzle that needs to be unraveled.  If she doesn't like my theories, then she should formulate something simpler and more plausible.  But let's focus on explaining the gap.  Where is the sixth killer theory?

Max makes an entirely different point.  Max writes:

What to make of Bill’s piece?  I don’t think it tells us much. Bill is looking at the conditional probabilities. ...   If firms are willing to take fewer risks on Loyola students (e.g., we hire the number one at Loyola and that’s it), then it’s not surprising that the conditional probability is higher. In my view, this result likely suggests a bias in favor of T14 students (perhaps for rational reasons regarding search costs), and an even stronger reason to attend an elite school if one’s goal is to be a top 250 law firm partner.

Max is certainly right.  If a student's goal is to be a BigLaw partner, he/she you will have an easier time getting hired if he/she attends an elite school.  You are the same person after all, so your behavior, motivation and ability can be treated as a constant -- getting your foot in the door is key. 

But Max overlooks the perspective of a law firm, which might look at the data and say, "My god, we have been calculating these conditional probabilities all wrong.  Clients aren't paying for training any more.  We can't afford this revolving door of T14s, who don't seem as engaged as the less elite grads."  So I think the data say quite a lot.  The huge presumption favoring elite grads created the Bi-Modal distribution; this propensity was arguably the beginning of the end of the BigLaw model where firms just passed off higher salaries onto clients.  No more.

Conclusion

Just like Phil Corboy, who could not get a job on LaSalle Street because he was Catholic, or Sandra Day O'Connor, who could not break into the Arizona corporate bar because she was a woman, or Joe Flom who was shut out of Wall Street because he was a Jew, there is always a contemporaneous narrative that justifies the status quo.  But eventually the prejudice becomes too expensive to bear.  And we change. 

The corporate law firm sector is now competing over market share for the first time ever.  Whether elite grads are better or worse hiring bets is now becoming a matter of strategic importance.  Unlike in earlier years, firms have to think twice before paying a snobbery tax for hiring criteria disconnected from, or even negatively correlated with, performance.

People's self image are all bound up with this topic. It has very little to do with rationality, and much more to do with identity and emotion.  But the takeaway is very simple: Pedigree does not reflect a natural ordering; it is socially constructed, initially based on substance and merit, but eventually perpetuated by inertia and vested interest.  Especially during times of transition and stress, it too can crumble.  This forebodes the regeneration of substance, and that is something that all of us should welcome.

"Too Good for BigLaw": The Statistician Edition

My good friend and provocateur Vivia Chen has posted a stir-the-pot column on the recent NLJ 250 Law School Hiring Survey.  The title of the column, "Too Good for BigLaw," is classic Vivia, speaking to our fragile egos as people and lawyers.  Reviewing the data on associates hired and partners promoted by law school, Vivia notes a significant shortfall in the number of elite law schools who become BigLaw partners.  One theory, suggested by Vivia, is that elite law school grads must have better options.  Regardless, the hierarchical nature of the legal profession may not be so neatly ordered after all.

I am confident that Vivia's column will create a wellspring of indignation among several thousand people who want to believe that getting into a fancy law school makes them permanently special.  And if they aren't special, the ruler must be broken.  I am the original source of the numbers, so I feel an obligation--albeit not a very big one--to reduce the anxiety level.  So below I wrote out a more dispassionate explanation of the numbers.  This is the Statistician Edition of "Too Good for BigLaw."

Vivia's Interpretation. 

The first point of clarification is that "Too Good for BigLaw" is one interpretation of the data -- one that is plausible, but others are plausible and perhaps more likely.  The virtue of Vivia's spin is that is gets your attention so she can make a simple, accurate point:  elite law school admission does not translate into Big Law partnership.  But one line is pure metaphor:  "If you want to make a safe bet ... put your money on the hardworking stiffs from ... Chicago—Loyola [rather than the 'wunderkids' from U of Chicago]."  If you are fixated on the literal, let me assure you that lots of other factors tend to intervene on the journey from law school to partner.  Hang on.  Don't panic. 

Limitations of the Data. 

The NLJ 250 Law School Hiring Survey is what is called a "cross-sectional" sample.  Think of a cross-sectional sample as a photo snapshot.  And, as life teaches us, snapshots can be misleading.  For example, if I said I was handsome in my 20s, ten photographs (one per year randomly drawn) would be more persuasive than a single phone.  (Given my druthers, I would prefer you look at the photo from my sister's wedding, where I was wearing a tuxedo and had a nice summer tan.)  Because snapshots are subject to random variability, the inferences to be drawn have to be properly cabined and qualified. 

In the case 2012 NLJ 250 data, we lack a reasonable basis for making strong school-specific claims. So, to be crystal clear, we cannot draw the inference that Chicago-Loyola is a better partnership bet that University of Chicago.  To draw stronger, more reliable inferences, we would need to average across multiple years. That said, if you doubt the accuracy of Vivia's regional-versus-elite law school metaphor, see Ted Seto, Where Do Partners Come From? (2011).

But What We Can Say?

Although it is improper to make (literal) school-specific claims from the 2012 data, it is possible to make stronger, more reliable inferences by pooling these data on observable school-level attributes, such as elite versus non-elite status based on U.S. News ranking.  This is appropriate because the school-level variability is, for the most part, random (good and bad years cancel each other out); and what is non-random (e.g., a economic recession) tends to apply to all law schools. 

Consider the following statistics on 2011 hiring and promotions of graduates of Top 14 versus non-Top 14 law schools (why T14? because these schools have played musical chairs in the U.S. News since the dawn of the rankings):

• Associates hired: 1,769 (T14), 1,525 (non-T14), or 53.7% to 46.3%
• Promotions:  326 (T14), 781 (non-T14), or 29.4% to 70.6%

Using the Associate hired/Partner Promoted ratio statistics referred to by Vivia, the ratio of associates hired to partners promoted is 5.43 for T14 versus 1.95 for the non-T14.  The ratio for all schools is 2.98.  So, there is a very large skew working against the elite law school grads.  The takeaway from these numbers is very straightforward.  There is a very big pipeline between T14 and BigLaw, but at some point before partnership, T14 associates tend to get off the train in disproportionately high numbers.

(A few readers may cling to the idea that one year's worth of data is not enough to draw the above inferences.  Maybe 2011 was a Black Swan, but please don't place any bets on it. I analyzed this same data four years ago and got essentially the same results.)

So Why Aren't the T14 Grads Dominating the BigLaw Partnership Ranks?

Good question.  Based on admissions criteria, these folks tend to have significantly higher test scores. And God knows, they enjoy a huge presumption of ability during law school recruiting--law firm hiring partners are incredibly brand-conscious. If partnership were the NCAA tournament, the T14 crowd would consistently be the number 1 and 2 seeds.

I have been thinking about this topic for several years.  Here are a few plausible theories, all of which can work in concert with one another.  Some are statistical, others are sociological:

1. Selection effects.  There are enormous selections effects at work.  In effect, we are pitting the #1 to 20 persons at Chicago-Loyola against anyone at U of Chicago.  It is unlikely that factors such as personality and motivation are identical in these two populations.  Another enormous selection effect is intrinsic interest in corporate law -- does anyone really believe that the 75% of Stanford, Penn or Harvard grads who start their careers in BigLaw have a burning passion to do technical, often times repetitive legal work for the Fortune 500? 

2. First jobs.  Elite graduates, whenever given the choice, tend to start at the most elite firms possible.  And, no surprise, these shops are the most highly leveraged and have the highest wash-out rates.  See Zaring & Henderson, Young Associates in Trouble (2008). But here is the big surprise: the next stop on the train is not somewhere else in the NLJ 250. These folks are not moving down; they are moving out.

3. Inter-generational privilege.  The After the JD study has documented that elite law school graduates tend to hail from more affluent families.  They also evince less interest in corporate law.  See Dinovitzer & Garth, Not that Into You, Am. Law. (Sept 2009).  When mom and dad are both lawyers, and grandpa owned a factory, maybe it's time to focus on art and travel.   In effect, one's inheritance becomes one's safety net.

4. Influence of admissions criteria.  Over the last 20 years, admissions committees have focused more and more on LSAT and UGPA; conversely, personal statements, letters of reference, and career histories hold very little sway.  This has fundamentally altered the BigLaw pipeline with students who are (excessively?) academic and lack significant brushes with real world adversity--not ideal grooming for a high stress professional service job.  I think these "supply chain" dynamics are uniformly overlooked by employers--big mistake.  Michigan Law circa 1982 is not Michigan Law circa 2012.

5. "A Better Plan B."  I know a lot of people in the law world will cling to the notion that elite law school graduates are running government agencies, leaving the law for Wall Street, and generally living very charmed lives.   I am sure there is something to this theory.  But I doubt it is carrying the load on the BigLaw associate/partner attrition puzzle.  My own class at U of Chicago (Class of 2001) has a broad assortment of legal careers -- but nothing too markedly different than many of the alumni of Indiana Law, where I teach.  Ten years out, lawyers from decent law schools tend to be having interesting careers -- with "interesting" being the core commonality.

Perhaps it is time we focused on the skills and attributes of successful law graduates rather than the name of the law school on their diplomas.  Law professors as a group are more alike than different.  Does anyone really believe that classes at an elite law school are much different -- let alone better -- than the instruction received at 100 regional law schools taught by professors from elite law schools? 

I think law schools can have a huge impact on the lives of students, but that is a strategy that  remains largely untapped.  And a topic for a future post.

[Posted by Bill Henderson]

Legal Education's Ninety-Five Theses

Brent E. Newton, an adjunct professor at Georgetown University Law Center, has posted a legal education reform piece on SSRN, entitled The Ninety-Five Theses: Systemic Reforms in the American Legal Education and Licensure   [Hat-tip TaxProf].  Judging by his title, Newton is hoping to spur a Reformation of legal education, akin to what Martin Luther did for Christianity in the 16th century.  If that is his agenda, I will not stand in his way.  

According to his GULC web bio, Newton is Deputy Staff Director of the U.S. Sentencing Commission; prior to that, he had a distinguished career as a public defender.  Newton is not the only adjunct-practitioner who has forcefully challenged U.S. legal education.  In 2008, Jason Dolin (solo practitioner, adjunct at Capital), published Opportunity Lost: How Law School Disappoints Law Students, the Public, and the Legal Profession.   In 2010, Steve Bennett (partner at Jones Day, adjunct at Fordham) published a law review article entitled, When Will Law Schools Change?

Law professors rarely engage with these critiques; to acknowledge these critiques, some might argue, is to give them oxygen and legitimacy.  I think this approach is a huge mistake.  Any enterprise interested in long-term success cares about the perceptions held by its stakeholders -- and adjuncts are definitely in that group.  In times of crisis, we need friends, not enemies.  Further, Newton, Dolin and Bennett are serious people and very capable lawyers.  If you leaf through these articles, you'll see that they read like Brandeis Briefs against the legal education establishment.  The authors present thoughtful, fact-based, and (albeit occasionally) trenchant arguments on why we, speaking as a legal education insider, should change. 

Simple question:  Can any of us identify a single historical example in which the establishment reformed itself because a critic effectively marshaled facts and logic to reveal the errors of its ways?  Institutional change doesn't happen that way -- facts and logic are no match for a few thousand egos and pious rationalizations for why others should change, but not me.  

The common storyline for institutional change is failure, with the rise of other institutions that better address the social, political and economic needs of stakeholders and broader society.  A less common narrative is institutional adaption, thanks in part to (1) the self-interest and survival instincts, and (2) the serendipity of timely, brilliant leadership.  (Does the legal academy have a few hundred great leaders?)

That said, Newton, Dolin and Bennett may be on the right side of history.  Because of the overproduction of law school graduates and their high levels of debt, we are now at a point when survival for a large proportion of law schools can no longer be taken for granted.  "What cannot go on forever, won't."  Herbert Stein, economist.  

Prediction:  In the next few years, some law schools will change and thrive.  Others won't and they will fail.  There will be nasty recriminations and gnashing of teeth.  A few at the very top will throw dice and decide not to change.  They will survive, but the innovations taking root in the rest of the law school hierarchy will make them look like anachronisms.  It will be a slow decay.  In the meantime, some aspects of the Post-Langdellian paradigm will look a lot like the suggestions made by Newton, Dolin and Bennett.  In twenty years, maybe sooner, the revolution will be over. Finally, Newton et al. will get a must deserved footnote.

[Posted by Bill Henderson]