Friday, March 9, 2012

"Too Good for BigLaw": The Statistician Edition

ViviaMy good friend and provocateur Vivia Chen has posted a stir-the-pot column on the recent NLJ 250 Law School Hiring Survey.  The title of the column, "Too Good for BigLaw," is classic Vivia, speaking to our fragile egos as people and lawyers.  Reviewing the data on associates hired and partners promoted by law school, Vivia notes a significant shortfall in the number of elite law schools who become BigLaw partners.  One theory, suggested by Vivia, is that elite law school grads must have better options.  Regardless, the hierarchical nature of the legal profession may not be so neatly ordered after all.

I am confident that Vivia's column will create a wellspring of indignation among several thousand people who want to believe that getting into a fancy law school makes them permanently special.  And if they aren't special, the ruler must be broken.  I am the original source of the numbers, so I feel an obligation--albeit not a very big one--to reduce the anxiety level.  So below I wrote out a more dispassionate explanation of the numbers.  This is the Statistician Edition of "Too Good for BigLaw."

Vivia's Interpretation. 

The first point of clarification is that "Too Good for BigLaw" is one interpretation of the data -- one that is plausible, but others are plausible and perhaps more likely.  The virtue of Vivia's spin is that is gets your attention so she can make a simple, accurate point:  elite law school admission does not translate into Big Law partnership.  But one line is pure metaphor:  "If you want to make a safe bet ... put your money on the hardworking stiffs from ... Chicago—Loyola [rather than the 'wunderkids' from U of Chicago]."  If you are fixated on the literal, let me assure you that lots of other factors tend to intervene on the journey from law school to partner.  Hang on.  Don't panic. 

Limitations of the Data

The NLJ 250 Law School Hiring Survey is what is called a "cross-sectional" sample.  Think of a cross-sectional sample as a photo snapshot.  And, as life teaches us, snapshots can be misleading.  For example, if I said I was handsome in my 20s, ten photographs (one per year randomly drawn) would be more persuasive than a single phone.  (Given my druthers, I would prefer you look at the photo from my sister's wedding, where I was wearing a tuxedo and had a nice summer tan.)  Because snapshots are subject to random variability, the inferences to be drawn have to be properly cabined and qualified. 

In the case 2012 NLJ 250 data, we lack a reasonable basis for making strong school-specific claims. So, to be crystal clear, we cannot draw the inference that Chicago-Loyola is a better partnership bet that University of Chicago.  To draw stronger, more reliable inferences, we would need to average across multiple years. That said, if you doubt the accuracy of Vivia's regional-versus-elite law school metaphor, see Ted Seto, Where Do Partners Come From? (2011).

But What We Can Say?

Although it is improper to make (literal) school-specific claims from the 2012 data, it is possible to make stronger, more reliable inferences by pooling these data on observable school-level attributes, such as elite versus non-elite status based on U.S. News ranking.  This is appropriate because the school-level variability is, for the most part, random (good and bad years cancel each other out); and what is non-random (e.g., a economic recession) tends to apply to all law schools. 

Consider the following statistics on 2011 hiring and promotions of graduates of Top 14 versus non-Top 14 law schools (why T14? because these schools have played musical chairs in the U.S. News since the dawn of the rankings):

  • Associates hired: 1,769 (T14), 1,525 (non-T14), or 53.7% to 46.3%
  • Promotions:  326 (T14), 781 (non-T14), or 29.4% to 70.6%

Using the Associate hired/Partner Promoted ratio statistics referred to by Vivia, the ratio of associates hired to partners promoted is 5.43 for T14 versus 1.95 for the non-T14.  The ratio for all schools is 2.98.  So, there is a very large skew working against the elite law school grads.  The takeaway from these numbers is very straightforward.  There is a very big pipeline between T14 and BigLaw, but at some point before partnership, T14 associates tend to get off the train in disproportionately high numbers.

(A few readers may cling to the idea that one year's worth of data is not enough to draw the above inferences.  Maybe 2011 was a Black Swan, but please don't place any bets on it. I analyzed this same data four years ago and got essentially the same results.)

So Why Aren't the T14 Grads Dominating the BigLaw Partnership Ranks?

Good question.  Based on admissions criteria, these folks tend to have significantly higher test scores. And God knows, they enjoy a huge presumption of ability during law school recruiting--law firm hiring partners are incredibly brand-conscious. If partnership were the NCAA tournament, the T14 crowd would consistently be the number 1 and 2 seeds.

I have been thinking about this topic for several years.  Here are a few plausible theories, all of which can work in concert with one another.  Some are statistical, others are sociological:

1. Selection effects.  There are enormous selections effects at work.  In effect, we are pitting the #1 to 20 persons at Chicago-Loyola against anyone at U of Chicago.  It is unlikely that factors such as personality and motivation are identical in these two populations.  Another enormous selection effect is intrinsic interest in corporate law -- does anyone really believe that the 75% of Stanford, Penn or Harvard grads who start their careers in BigLaw have a burning passion to do technical, often times repetitive legal work for the Fortune 500? 

2. First jobs.  Elite graduates, whenever given the choice, tend to start at the most elite firms possible.  And, no surprise, these shops are the most highly leveraged and have the highest wash-out rates.  See Zaring & Henderson, Young Associates in Trouble (2008). But here is the big surprise: the next stop on the train is not somewhere else in the NLJ 250. These folks are not moving down; they are moving out.

3. Inter-generational privilege.  The After the JD study has documented that elite law school graduates tend to hail from more affluent families.  They also evince less interest in corporate law.  See Dinovitzer & Garth, Not that Into You, Am. Law. (Sept 2009).  When mom and dad are both lawyers, and grandpa owned a factory, maybe it's time to focus on art and travel.   In effect, one's inheritance becomes one's safety net.

4. Influence of admissions criteria.  Over the last 20 years, admissions committees have focused more and more on LSAT and UGPA; conversely, personal statements, letters of reference, and career histories hold very little sway.  This has fundamentally altered the BigLaw pipeline with students who are (excessively?) academic and lack significant brushes with real world adversity--not ideal grooming for a high stress professional service job.  I think these "supply chain" dynamics are uniformly overlooked by employers--big mistake.  Michigan Law circa 1982 is not Michigan Law circa 2012.

5. "A Better Plan B."  I know a lot of people in the law world will cling to the notion that elite law school graduates are running government agencies, leaving the law for Wall Street, and generally living very charmed lives.   I am sure there is something to this theory.  But I doubt it is carrying the load on the BigLaw associate/partner attrition puzzle.  My own class at U of Chicago (Class of 2001) has a broad assortment of legal careers -- but nothing too markedly different than many of the alumni of Indiana Law, where I teach.  Ten years out, lawyers from decent law schools tend to be having interesting careers -- with "interesting" being the core commonality.

Perhaps it is time we focused on the skills and attributes of successful law graduates rather than the name of the law school on their diplomas.  Law professors as a group are more alike than different.  Does anyone really believe that classes at an elite law school are much different -- let alone better -- than the instruction received at 100 regional law schools taught by professors from elite law schools? 

I think law schools can have a huge impact on the lives of students, but that is a strategy that  remains largely untapped.  And a topic for a future post.

[Posted by Bill Henderson]

Blog posts worth reading, Data on legal education, Data on the profession, Law Firms | Permalink


IMHO #1 and #2 are by far the biggest contributing factors, with a subtle issue not mentioned by your post: the career services programs at T14 schools are utterly, totally incompetent at directing students towards non-Big Law jobs. Ask about public interest, or small firms, or plaintiff's work, and they'll look at you blankly and wish you the best.

The "self selection" thus should occur much earlier, like during law school, but it doesn't because the students don't really have the ability to get off of the Big Law train and do what they want. Instead, the ride along, make some money, and realize they have no interest in making a lifestyle out of their work, and so jump to greener (but often less lucrative) pastures.

Posted by: Max Kennerly | Mar 11, 2012 9:38:37 AM

It would be interesting to know how many of the T14 grads go from Big Law to in-house counsel.

Does the sex ratio give any hints? Are more of the T14 Big Law associates women?

How about field? Are fewer of them in tax or real estate, and are those easier fields to make partner in?

Posted by: Eric Rasmusen | Mar 11, 2012 11:04:44 AM

Here's another possibility. The real question is why employers would hire T14's if they know (as they must) that they wash out more often. An answer is that they don't know *which* T14's are going to wash out, but they do know, in advance, how to pick other gradutes who aren't. Suppose that the average Chicago grad is good enough to make partner, but the bottom one isn't, and Chicago cleverly grades them all the same. Employers hire all of them, knowing that half will be good enough. The average Indiana grad is not good enough to make partner, so he doesn't get hired. But Indiana designs its grading and honors so firms CAN tell who the top 10 Indiana students are in a given year and that they will all make partner. Those 10 get hired--- and no other Indiana grads--- and they all make partner.

There's a story with something of that flavor in my working paper,

``Coarse Grades,'' with Rick Harbaugh. Certifiers of quality often report only coarse grades to the public despite having measured quality more finely, e.g., "A" instead of "98". Why? We show that using coarse grades can actually result in more information reaching the public, because it encourages low-quality individuals or firms to become certified. In our model the certifier aims to minimize public uncertainty over quality subject to the feasibility constraint of voluntary certification at a fixed cost. Moving from the best exact grading scheme to the best coarse one (a) induces more participation and (b) reduces public uncertainty. or in her possibility. Maybe the

Posted by: Eric Rasmusen | Mar 11, 2012 11:09:49 AM

WHy do you assume everyone will be upset? Why the sarcasm?

Posted by: BH | Mar 11, 2012 3:00:31 PM

To me, this supports what students have been saying for years. Where you go to school disproportionately affects where you get hired. Firms would rather hire someone less likely to make partner than someone from a school with less prestige.

For prospective students choosing between schools, the message is clear. To be hired by biglaw out of a non-top14 school, you have to be three times as good as a top 14 grad. No wonder students pay full price at top schools instead of taking hefty scholarships at, say, Indiana. No wonder law school costs so much.

Posted by: Justin | Mar 11, 2012 4:19:43 PM

To BH@6:00:
Elite law school grads are not becoming partners at rates proportionate with their rate of hire. If the #1 and #2 seeds in the NCAA tournament had a less than fifty percent winning average in March Madness, we would be surprised and wonder who was creating the brackets and what criteria they used.

Think about Moneyball. Statistics were used to show that the scouts were mis-pricing talent, due to ingrained biases, myths, and prejudice, and the scouts got indignant. Since human are involved, I suspect the same pattern is likely to be true here.

Posted by: Bill Henderson | Mar 11, 2012 5:39:59 PM

If you are a prospective law student looking at your choices, there are two big problems with using this data as a guide.
First, it takes for granted the misleading "law firm as tournament" model. Academics, most of whom have spent scant time in law firms, treat partnership as the prize and the reason for entering big firms. Even when the model is made elastic, it misleads. Associates go to big law firms for two core reasons - to make real money now, and to build their human capital. At various inflection points in their career they may - or may not - have the option or the need to take that human capital to different employers. Many of the most desirable lateral moves require a jump well before partner, and many lateral employers are much more top school brand conscious than big law. The legal academy is an extreme example, with the sweet spot for leaving practice being two to four years in, and with a Yale/Harvard domination followed by a rest of the T14 plurality followed by slim pickings. I don't have data, but my friends who moved successfully from law firms to investment banks, private equity, hot boutiques or in house either tended to do it very young, at an early career level, or well after making partner, at a high lateral level, and not so much in between. I'm betting, again based on only anecdotal evidence, that the early career lateral options also favor elite degrees, mediated somewhat by having a big law pedigree (which itself is T14 biased). People with opportunities to leave law firms for more rewarding jobs will leave, and I'm betting that elite graduates have more of those options. The pool of partners, especially once you get past the very most lucrative and prestigious firms, comprises those who didn't have better options as much as those who succeeded. To use your basketball tournament analogy, you might not be looking at the NCAA tournament, but instead at the NIT tournament with some of the stronger seeds able to move laterally to the NCAA tourney or the Olympics midway through.
The second problem is that this lumps the AmLaw 250 into one pool, and therefore merges some very different kinds of places. Making partner at a regional firm doing routine work may be a good professional and personal choice, but it's very different from making partner in a top money center firm doing cutting edge work and paying rock star compensation. I wonder what the numbers look like with cuts at AmLaw 25, 50, and 100 levels.
That said, I think it's a good reminder that neither graduation from a top law school nor first job placement at an elite law firm guarantee in any way a satisfying career, and I think that's more true as training at top firms has both declined and diverged due to hyper-specialization from what lower tier firms might value. I've known some really smart people with really good grades from really good schools and with very impressive first jobs who have ended up in some really depressing places - all the way down to by the hour document review. It's not just a problem with the drive or qualifications of law graduates; it's also an issue of whether the elite firms still deliver what used to be part of the value proposition, which was a good career somewhere for their alumni, even if not at the firm. As David Maister has observed, professionals don't come out of school looking for a job, they come out looking for a career, and it's not clear to me that expectation gets met in the current market.

Posted by: Ray Campbell | Mar 11, 2012 7:31:50 PM

My short questions grew out of some of what Ray Campbell was suggesting. Why do people think partnership in a law firm is the prize of all prizes? The write up suggests that the failure of elite law grads to dominate the partnership ranks is some sort of failure, that their attendance at an elite school was not worth it given the fact that they do not automatically end up as a partner in a law firm.
It is also written with a tone of joy at the supposed discomfort that this information will supposedly cause. My experiences with my classmates is similar to Campbell's. Many who started out in firms stayed long enough to pay down loans and then went to other, generally, good positions in-house, in government, in corporations doing things other than legal work, becoming politicians, becoming law professors, or becoming stay at home mothers. I say "generally" because, as was indicated, going to an elite school is no guarantee that you will be successful for the rest of your life. That much is clear.

It would be good to know what graduates of elite schools are doing once they leave firms-- anecdotes are not reliable. Interestingly enough my professors at my elite law school portrayed going to a firm as a failure. "Do something else with your life" they said, "That stuff is for people who cannot do anything better."They were convinced that there were far more important things to do with our degrees. We had loans to pay back, so we listened politely and went off to work.

Posted by: BH | Mar 11, 2012 10:12:27 PM

BH: There is no joy being expressed here. What makes the topic emotionally volatile is the tremendous insecurity among lawyers re where they went to law schools -- "was that initial sorting accurate?" is subconsciously a work for many many lawyers. Some hope the answer is yes, others no. This anxiety exists because lawyer development and lawyer workplace fit have not been to date a topic of empirical study.

The bizarre disparities in the NLJ 250 data suggest such work is warrant. Hence the five specific theories set forth in the blog post.

Re whether partnership is a prize, it is not for many people. But the disparity between elite and non-elite graduates is very large. As I said, part of it may be A Better Plan B for elite grads -- but the disparity is too large for that theory to do all the work. If it were, we would not have to speculate on what happens after the associate years. Another is intensity of preference by first generation professionals, who are over-represented in elite schools. Perhaps have levels of motivation are more important that paper credentials. That theory comports with what we know about immigration -- opportunity is in the eyes of the beholder.

To my mind, these are all empirical questions. And the stakes are large enough that they are worth answering with data.

Posted by: Bill Henderson | Mar 12, 2012 6:18:57 AM

Better data would be how many Loyola grads were hired by biglaw in the 3-year period from 99-01 and how many made partner in the 3-year period from 09-11.

Among working partners, you'll find believers that the selection effect is biggest by far. But those from T14 tend to have drank the kool-aid, and would prefer to hire from their brand name.

Remember Liar's Poker? paraphrase - successful Masters of the Universe had economics degrees; ergo they should hire people with economics degrees. No analysis as to whether correlation = causation.

Posted by: JG | Mar 12, 2012 8:38:54 AM

Are there completed longitudinal studies on the long term career paths of law firm associates?

Posted by: BH | Mar 12, 2012 9:04:39 AM


I commented on vivia's post as well, referring to the likelihood that this was an artifact of nlj 250 jobs drying up in 2011, depressing entry level hiring for loyola this year against a partnership class selected back in 2003 or so. This post gives me some more confidence that that's not the case, but it's not impossible to think of scenarios that could account for this years ratios as well as the ratios for several years ago (over hiring into the partnership before the economic ramifications of the financial crisis became clear). In particular, I question your statement that what is not random tends to affect all schools, since, as has been well publicized, The recession has affected different tiers of schools differently. Columbia has been more harmed by the recession than Yale, Georgetown has done more poorly than Columbia, and so on down the line. Crappy schools have been disproportionately affected in terms of hiring recently, and any set of data that doesn't explicitly take that into account is questionable.

That said, the other potential explanations are intriguing, and the are certainly some effects that probably aren't getting accounted for here either such as clerkships that would more than likely tend to reinforce your claims, so... Well, I'd be interested to see how it all comes out.

Posted by: Stu | Mar 12, 2012 9:22:52 AM

No, I am afraid paragraph two of your post is very much an expression of schadenfreude.

Posted by: BH | Mar 12, 2012 10:20:47 AM

The data doesn't account for the fact that a significantly higher number of the grads of lower-tier law school that end up in Biglaw don't start there. Many don't have the credentials to get biglaw jobs out the gate. On the other hand, just about every grad of a top-10 law school that ends up in biglaw did so either right out the gate or immediately after a clerkship.

Posted by: DA | Mar 12, 2012 1:58:30 PM

Bill, I agree this deserves an empirical study. From my anecdotal observations, some of the T14 graduates I've seen leave big law move to happy places (in house, other good firms, government, etc.) but some have not. While I think the happy lateral moves are a big part of the explanation, I suspect you are right when you say that explanation doesn't carry all the load. The question I have, from watching good people struggle in the marketplace, is what big law trains you to do if you stay there past the restart button phase, and want to market your skills elsewhere. Some people question whether big law trains young lawyers (or young lawyers they see as not on the fast track) at all. Having been a partner in big law, I'm skeptical about that, because the nature of the work product requires training the young lawyers, and a lot of the training comes naturally as their work is revised (the non-brain dead will look at the marked up work and use it as a learning experience). Things may have changed since I left practice, but I doubt it. The live question for me is whether big law trains young lawyers with high level transferable job skills. As the practice of law has evolved, the work has gotten much more specialized. What you are trained in to be a useful mid and senior level association may not include skills that are valued elsewhere - particularly if you are moving from the money center firms to regional firms or smaller general practice firms. Certainly, you are unlikely to be first chairing trials or negotiating deals. Your skill interpreting a specific federal statute or untangling regulations related to exotic derivatives might be useful in the kind of practice your AmLaw 25 firm has, but not so useful to a firm serving 100 employee businesses or based in the third largest city in the state. At some fundamental level, this bears on the nature of the profession - if someone highly competent (but not economically necessary to the firm at a level that would make them partnership material) at one firm has very little value at other firms, are we really in the same profession? It also has big implications for the business model of firms that shed six out of seven associates. If big law cannot be counted on as the gateway to a career as a professional (put differently, if the consequence of not making partner is personal tragedy instead of a soft landing at another good job as a professional), what's the value proposition for big law firms competing for top talent? Notwithstanding where you find the talent, the kind of high level consultancy work law firms do depends on talent, and it's going to get harder to hire people if the consequence of being a regulatory specialist in a firm that doesn't need another partner in that specialty is career death.

Posted by: Ray Campbell | Mar 12, 2012 4:50:14 PM

I'm a graduate from a top 5 law school and also a partner at a top 25 firm for many years. I think at a high level of abstraction this is accurate,a lthough individual variations are wide. E.g., in my firm Chicago grads do very well but Stanford and Yale grads do horribly. I will add one factor I have observed to your list. Making partner is not just ability + stamina. Private practice is a competitive service business. I observe a tendency that the better the academic record, the less commitment to the mundane aspects of serving clients. As well, one does not get paid very much for telling a client "no" even when correct. Lower tier firms, and by extension lower tier graduates, seem to be more willing to say yes to a client to get business than a good legal analysis warrants.

Posted by: mt | Mar 13, 2012 7:36:50 AM

My perspective for the past 20 years has been that of sales coach to (mostly) AmLaw 200 partners. My long view is that this discussion suggests that firms struggle with predictive capability, whether regarding lawyering skills or business-generation skills. I recall a conversation many years ago with the head of the associates committee at a top-15 NYC firm, who said, "I tell them the truth, i.e., that, statistically, any associate's chances of making partner here are pretty slim, whether due to their own decision or the firm's. They should, therefore, consider this a place to get their ticket punched. If it turns out that they're a great partner fit and want to be here, so much the better, but it's not the way to bet." This leader consciously installed a "Plan B" mentality because he felt that, statistically speaking, that was reality, and it would be dishonest not to.

From this and other exposure, I've concluded that BigLaw considered the high associate attrition rate acceptable. Despite some early development cost, firms were accustomed to having associates largely billable from a pretty early stage of career, and being hugely profitable from their 3rd or 4th years onward. If somebody left after 7-8 years, it was inconvenient, but not a disaster. After all, the firm had already made $3-4 million on each from the spread between billings and compensation. (Now that clients are no longer willing to finance associates' early OJT, that attitude may shift.)

It seems that clients share firms' law-school-brand bias. Otherwise, why would they ever have tolerated paying BigLaw rates for rookie lawyers who have no useful skills or experience? Clearly, they were betting on the come, i.e., that this Ivy-educated lawyer would some day grow up to be valuable to them. That would seem to make T14 associates inherently more billable, earlier. If you know you're going to turn over 75% of associates within 6-7 years (or whatever the actual ratios are), you want them billable as soon as possible. In such case, the benefit of the doubt goes to the T14 newbie.

BigLaw hiring- and development practices seem mired in a world that no longer exists, i.e., one in which firms and client relationships remain stable for decades, where one's law school pedigree was as good a hiring criteria as any. Now, though, the "brushes with adversity" that Bill Henderson suggests elites lack, and the resiliency gained from such, may become very critical. The "BigLaw Lawyer" job description has changed a lot in the past four years, but nowhere near as radically as it's yet to change as the "new normal" settles in. It appears that law schools, BigLaw hiring philosophy, and associate-management thinking all lag badly.

Posted by: Mike O'Horo | Mar 13, 2012 10:13:18 AM

@Mike O'Horo. "That would seem to make T14 associates inherently more billable, earlier. If you know you're going to turn over 75% of associates within 6-7 years (or whatever the actual ratios are), you want them billable as soon as possible. In such case, the benefit of the doubt goes to the T14 newbie." BINGO.

Posted by: Publius Novus | Mar 19, 2012 7:46:10 AM

Post a comment