Saturday, March 17, 2012

A Reply to the Empiricists at NWU Law

My blog post from last week, "Too Good for BigLaw: The Statistician Edition" has resulted in a minor kerfuffle with some of the distinguished empiricists at Northwestern Law.  See Dan Rodriguez, Law School Sorting and the Partnership Track:  Northwestern Empiricists Weigh In, Word on the Streeterville [The Blog of NWU Law Dean].  NWU Law folks were not impressed with my analysis.  Dan Rodriguez was gracious enough to send me the link at the same time his post, quoting the views of his colleagues, went live.  He has also encouraged me to reply publicly. 

I am happy to do that.  Let me start with big picture issues.  Then, for those folks with the curiousity and stamina to wade through arcane details--and experience tells me this is a small group--I will directly address, point by point, the the issues raised by Kate Litvak and Max Schanzenbach.  But at the outset, I will say that I am not conceding any ground.

Big Picture

First off, it is important to understand why this topic generated so much traffic.  See Above the Law, ABA Daily Journal, TaxProf, InstapunditAdam Smith

It all started with a provocative blog post by Vivia Chen, the columnist for The Careerist.  Vivia reviewed hiring and promotion data from the NLJ 250 Law School Hiring Survey and noted that elite law school graduates were becoming partner in very low numbers when compared to the hiring pipeline. Vivia editorialized on the numbers in a way that played into readers' fragile egos and insecurities.  Of course, that is her job, which she does very well.

In a nutshell, here is why people care -- or more precisely, get anxious -- about this topic:  it is conventional wisdom that graduation from elite law schools produces better career outcomes.  When that expectation is countered by actual marketplace data, people are surprised.  See, e.g., Bruce MacEwen,  "The Best & The Brightest" at  Adam Smith Esq. (leading blog on law firm economics).  Surprise is the first reason this issue got so much play.  Emotion is the second.

Emotion matters because very few lawyers and law professors are dispassionate on this topic.  When it comes to conventional wisdom on law school pedigree, we all have horses in the race.  Because we are human beings, we lawyers and law professors don't wait for balanced market data to develop our own entrenched worldviews.  When the conventional wisdom favors us, we go with it -- albeit we aren't really conscious this is happening. So when data upset the apple cart and potentially make us look complacent, our passions get aroused. 

The folks at Above the Law have built a entire business model around such predictable lawyer foibles.  The more chum thrown in the water, the higher the ad revenues.  It's just that simple.

Vivia's primary point, stated through metaphor, is that regional schools (such as Chicago Loyola) seem to be making partner at higher rates than the elite schools (such as Chicago).  This is a reasonable inference because the ratio of associates hired to partners promoted appears to be consistently high for elite law schools and very low for a large number of regional law schools.  This very point was made independently by Bruce MacEwen, who is a very sophisticated guy who advises law firms on strategy.

That said, there was ample opportunity for readers to draw spurious inferences from Vivia's metaphor-driven blog post.  Thus, to avoid any school-specific claims (a 1-year crossectional sample is not suitable for such a purpose), I pooled the schools by U.S. News ranking, drawing a line between elite and non-elite at the T14 mark.  Why T14?  Because these schools have played a closed loop of musical chairs for 20 years in the U.S. News rankings.  These schools would be viewed by most employers as "national" law schools.  

Here is what the data showed:

  • Pipeline in:  53.7% T14, 46.3% non-T14
  • Partners Promoted: 29.4% T14. 70.6% non-T14.

That is, well, an enormous skew. In 2011, for every 5.43 elite grads hired, a senior associate from an elite school makes partner.  For non-elite schools, that corresponding statistic is 1.95.  Vivia found these numbers surprising and somewhat counter-intuitive.  So did Bruce MacEwen, Above the Law, ABA Journal, etc.

There are ways to break down these numbers to gain additional insights, but the key point here is one of magnitude.  Elite law graduates are supposed to be smarter and more capable -- no one expects these folks to be on short side of any race, tournament or desired outcome.  The magnitude of hiring/promotion gap is the surprising fact that needs to be explained.

I had observed roughly the same skew several years ago (pooled 2007 and 2008 data) and alluded to it in this article, "Why is the Job Market Changing," Nat'l Jurist (Nov. 2010).  I also follow other relevant studies, such as The After the JD, which have noted differences between elite and non-elite graduates.   So I had a head start in thinking through possible explanations. I thus offered five theories, all of which could work in concert, to explain the large skew in the data:

  1. Selection effects
  2. Differences in first jobs
  3. Intergeneraional privilege
  4. Influence of admission criteria on the associate pipeline
  5. "A Better Plan B" for elite grads

So, to be very clear, I am not using the NLJ 250 data to support the above theories.  It is the reverse: I am offering the above theories as a likely explanation for the very large skew between elite and nonelite grads.  Framed as a open-ended research question, it might be written, "why are elite grads not becoming BigLaw partners in numbers commensurate with hiring patterns and general presumptions of their higher ability?"  That is a mystery and a puzzle.

Statistics Minutiae  [After the jump ...]

Continue reading

March 17, 2012 in Blog posts worth reading, Data on legal education, Data on the profession, Law Firms, New and Noteworthy, Scholarship on legal education, Structural change | Permalink | Comments (3)

Friday, March 9, 2012

"Too Good for BigLaw": The Statistician Edition

ViviaMy good friend and provocateur Vivia Chen has posted a stir-the-pot column on the recent NLJ 250 Law School Hiring Survey.  The title of the column, "Too Good for BigLaw," is classic Vivia, speaking to our fragile egos as people and lawyers.  Reviewing the data on associates hired and partners promoted by law school, Vivia notes a significant shortfall in the number of elite law schools who become BigLaw partners.  One theory, suggested by Vivia, is that elite law school grads must have better options.  Regardless, the hierarchical nature of the legal profession may not be so neatly ordered after all.

I am confident that Vivia's column will create a wellspring of indignation among several thousand people who want to believe that getting into a fancy law school makes them permanently special.  And if they aren't special, the ruler must be broken.  I am the original source of the numbers, so I feel an obligation--albeit not a very big one--to reduce the anxiety level.  So below I wrote out a more dispassionate explanation of the numbers.  This is the Statistician Edition of "Too Good for BigLaw."

Vivia's Interpretation. 

The first point of clarification is that "Too Good for BigLaw" is one interpretation of the data -- one that is plausible, but others are plausible and perhaps more likely.  The virtue of Vivia's spin is that is gets your attention so she can make a simple, accurate point:  elite law school admission does not translate into Big Law partnership.  But one line is pure metaphor:  "If you want to make a safe bet ... put your money on the hardworking stiffs from ... Chicago—Loyola [rather than the 'wunderkids' from U of Chicago]."  If you are fixated on the literal, let me assure you that lots of other factors tend to intervene on the journey from law school to partner.  Hang on.  Don't panic. 

Limitations of the Data

The NLJ 250 Law School Hiring Survey is what is called a "cross-sectional" sample.  Think of a cross-sectional sample as a photo snapshot.  And, as life teaches us, snapshots can be misleading.  For example, if I said I was handsome in my 20s, ten photographs (one per year randomly drawn) would be more persuasive than a single phone.  (Given my druthers, I would prefer you look at the photo from my sister's wedding, where I was wearing a tuxedo and had a nice summer tan.)  Because snapshots are subject to random variability, the inferences to be drawn have to be properly cabined and qualified. 

In the case 2012 NLJ 250 data, we lack a reasonable basis for making strong school-specific claims. So, to be crystal clear, we cannot draw the inference that Chicago-Loyola is a better partnership bet that University of Chicago.  To draw stronger, more reliable inferences, we would need to average across multiple years. That said, if you doubt the accuracy of Vivia's regional-versus-elite law school metaphor, see Ted Seto, Where Do Partners Come From? (2011).

But What We Can Say?

Although it is improper to make (literal) school-specific claims from the 2012 data, it is possible to make stronger, more reliable inferences by pooling these data on observable school-level attributes, such as elite versus non-elite status based on U.S. News ranking.  This is appropriate because the school-level variability is, for the most part, random (good and bad years cancel each other out); and what is non-random (e.g., a economic recession) tends to apply to all law schools. 

Consider the following statistics on 2011 hiring and promotions of graduates of Top 14 versus non-Top 14 law schools (why T14? because these schools have played musical chairs in the U.S. News since the dawn of the rankings):

  • Associates hired: 1,769 (T14), 1,525 (non-T14), or 53.7% to 46.3%
  • Promotions:  326 (T14), 781 (non-T14), or 29.4% to 70.6%

Using the Associate hired/Partner Promoted ratio statistics referred to by Vivia, the ratio of associates hired to partners promoted is 5.43 for T14 versus 1.95 for the non-T14.  The ratio for all schools is 2.98.  So, there is a very large skew working against the elite law school grads.  The takeaway from these numbers is very straightforward.  There is a very big pipeline between T14 and BigLaw, but at some point before partnership, T14 associates tend to get off the train in disproportionately high numbers.

(A few readers may cling to the idea that one year's worth of data is not enough to draw the above inferences.  Maybe 2011 was a Black Swan, but please don't place any bets on it. I analyzed this same data four years ago and got essentially the same results.)

So Why Aren't the T14 Grads Dominating the BigLaw Partnership Ranks?

Good question.  Based on admissions criteria, these folks tend to have significantly higher test scores. And God knows, they enjoy a huge presumption of ability during law school recruiting--law firm hiring partners are incredibly brand-conscious. If partnership were the NCAA tournament, the T14 crowd would consistently be the number 1 and 2 seeds.

I have been thinking about this topic for several years.  Here are a few plausible theories, all of which can work in concert with one another.  Some are statistical, others are sociological:

1. Selection effects.  There are enormous selections effects at work.  In effect, we are pitting the #1 to 20 persons at Chicago-Loyola against anyone at U of Chicago.  It is unlikely that factors such as personality and motivation are identical in these two populations.  Another enormous selection effect is intrinsic interest in corporate law -- does anyone really believe that the 75% of Stanford, Penn or Harvard grads who start their careers in BigLaw have a burning passion to do technical, often times repetitive legal work for the Fortune 500? 

2. First jobs.  Elite graduates, whenever given the choice, tend to start at the most elite firms possible.  And, no surprise, these shops are the most highly leveraged and have the highest wash-out rates.  See Zaring & Henderson, Young Associates in Trouble (2008). But here is the big surprise: the next stop on the train is not somewhere else in the NLJ 250. These folks are not moving down; they are moving out.

3. Inter-generational privilege.  The After the JD study has documented that elite law school graduates tend to hail from more affluent families.  They also evince less interest in corporate law.  See Dinovitzer & Garth, Not that Into You, Am. Law. (Sept 2009).  When mom and dad are both lawyers, and grandpa owned a factory, maybe it's time to focus on art and travel.   In effect, one's inheritance becomes one's safety net.

4. Influence of admissions criteria.  Over the last 20 years, admissions committees have focused more and more on LSAT and UGPA; conversely, personal statements, letters of reference, and career histories hold very little sway.  This has fundamentally altered the BigLaw pipeline with students who are (excessively?) academic and lack significant brushes with real world adversity--not ideal grooming for a high stress professional service job.  I think these "supply chain" dynamics are uniformly overlooked by employers--big mistake.  Michigan Law circa 1982 is not Michigan Law circa 2012.

5. "A Better Plan B."  I know a lot of people in the law world will cling to the notion that elite law school graduates are running government agencies, leaving the law for Wall Street, and generally living very charmed lives.   I am sure there is something to this theory.  But I doubt it is carrying the load on the BigLaw associate/partner attrition puzzle.  My own class at U of Chicago (Class of 2001) has a broad assortment of legal careers -- but nothing too markedly different than many of the alumni of Indiana Law, where I teach.  Ten years out, lawyers from decent law schools tend to be having interesting careers -- with "interesting" being the core commonality.

Perhaps it is time we focused on the skills and attributes of successful law graduates rather than the name of the law school on their diplomas.  Law professors as a group are more alike than different.  Does anyone really believe that classes at an elite law school are much different -- let alone better -- than the instruction received at 100 regional law schools taught by professors from elite law schools? 

I think law schools can have a huge impact on the lives of students, but that is a strategy that  remains largely untapped.  And a topic for a future post.

[Posted by Bill Henderson]

March 9, 2012 in Blog posts worth reading, Data on legal education, Data on the profession, Law Firms | Permalink | Comments (18)

Wednesday, February 1, 2012

Legal Education's Ninety-Five Theses

Picture1Brent E. Newton, an adjunct professor at Georgetown University Law Center, has posted a legal education reform piece on SSRN, entitled The Ninety-Five Theses: Systemic Reforms in the American Legal Education and Licensure   [Hat-tip TaxProf].  Judging by his title, Newton is hoping to spur a Reformation of legal education, akin to what Martin Luther did for Christianity in the 16th century.  If that is his agenda, I will not stand in his way.  

According to his GULC web bio, Newton is Deputy Staff Director of the U.S. Sentencing Commission; prior to that, he had a distinguished career as a public defender.  Newton is not the only adjunct-practitioner who has forcefully challenged U.S. legal education.  In 2008, Jason Dolin (solo practitioner, adjunct at Capital), published Opportunity Lost: How Law School Disappoints Law Students, the Public, and the Legal Profession.   In 2010, Steve Bennett (partner at Jones Day, adjunct at Fordham) published a law review article entitled, When Will Law Schools Change?

Law professors rarely engage with these critiques; to acknowledge these critiques, some might argue, is to give them oxygen and legitimacy.  I think this approach is a huge mistake.  Any enterprise interested in long-term success cares about the perceptions held by its stakeholders -- and adjuncts are definitely in that group.  In times of crisis, we need friends, not enemies.  Further, Newton, Dolin and Bennett are serious people and very capable lawyers.  If you leaf through these articles, you'll see that they read like Brandeis Briefs against the legal education establishment.  The authors present thoughtful, fact-based, and (albeit occasionally) trenchant arguments on why we, speaking as a legal education insider, should change. 

Simple question:  Can any of us identify a single historical example in which the establishment reformed itself because a critic effectively marshaled facts and logic to reveal the errors of its ways?  Institutional change doesn't happen that way -- facts and logic are no match for a few thousand egos and pious rationalizations for why others should change, but not me.  

The common storyline for institutional change is failure, with the rise of other institutions that better address the social, political and economic needs of stakeholders and broader society.  A less common narrative is institutional adaption, thanks in part to (1) the self-interest and survival instincts, and (2) the serendipity of timely, brilliant leadership.  (Does the legal academy have a few hundred great leaders?)

That said, Newton, Dolin and Bennett may be on the right side of history.  Because of the overproduction of law school graduates and their high levels of debt, we are now at a point when survival for a large proportion of law schools can no longer be taken for granted.  "What cannot go on forever, won't."  Herbert Stein, economist.  

Prediction:  In the next few years, some law schools will change and thrive.  Others won't and they will fail.  There will be nasty recriminations and gnashing of teeth.  A few at the very top will throw dice and decide not to change.  They will survive, but the innovations taking root in the rest of the law school hierarchy will make them look like anachronisms.  It will be a slow decay.  In the meantime, some aspects of the Post-Langdellian paradigm will look a lot like the suggestions made by Newton, Dolin and Bennett.  In twenty years, maybe sooner, the revolution will be over. Finally, Newton et al. will get a must deserved footnote.

[Posted by Bill Henderson]

Picture2

February 1, 2012 in Current events, Data on legal education, Innovations in legal education, Scholarship on legal education, Structural change | Permalink | Comments (1)