Sunday, April 8, 2007

Rankings, Leiter, and LSAT: "The Tyranny of Lists"

Posted by Alan Childress

Jeff Lipshaw has posted helpfully on whether law school rankings have blue-coastal bias (a torch carried on by ELS Blog here and good comments here), and he has charted the largely-clumped distribution of the reputational scores in USNWR both by judges/lawyers and by "peers," i.e., law school 723714_multiple_choice Players. 

Since then, on April 5, Brian Leiter posted this updated 2007 rankings of law schools by student numerical quality, which is based on LSAT.  Leiter ranks the top 40 schools according to 75th LSAT percentile, and then 25th LSAT percentile.  Tulane faculty got an email from a colleague alerting us to this and noting our school made the list (as #40) in terms of 25th percentile.  In reply, another colleague emailed a response for which I asked permission to post here, as long as we are on the subject of rankings and law school "quality."  The response reminded me of some of Nancy Rapoport's cautions, and said what I 070061289001_sclzzzzzzz_aa240_ wish I had said.

So welcome to our distinguished guest blogger Raymond T. Diamond, Tulane's John Koerner Professor of Law and the coauthor of a prize-winning legal history book on Brown v. Board's origins and legacy (right).  We appreciate his allowing us to reproduce his email:

Interesting survey, but I wonder about drawing normative content from it.  It's bad enough that so many schools, not excluding our own, make choices on the basis of the perceived need to improve or maintain their US News rankings, but this one is based almost to the point of exclusion on the the norm that a high LSAT class is better than a low one.

Surely there are other norms -- gender, racial, and geographical diversity for one, the desire to serve friends and alumni of the university, the desire to serve a host city, region, or state, and the choice to serve particular areas of legal inquiry and practice -- that influence and that should influence the admissions practices of law schools, including our own.

Having said all this, Leiter doesn't add anything not previously available to the body of information available to the public.  One can figure this out by trolling through US News or ABA data -- undoubtedly what Leiter did.  But I do worry about the tyranny of lists.

http://lawprofessors.typepad.com/legal_profession/2007/04/rankings_leiter.html

Blogging, Guest Bloggers from the Academy, Hot Topics | Permalink

TrackBack URL for this entry:

http://www.typepad.com/services/trackback/6a00d8341bfae553ef00d834f4b1c353ef

Listed below are links to weblogs that reference Rankings, Leiter, and LSAT: "The Tyranny of Lists" :

Comments

I wonder if the 'tyranny of lists' is better described as the 'tyranny of numbers,' and therefore related to a larger problem, namely, excessive formalism, perhaps caused by 'the triumph and prestige of the physical sciences, or ingrained ways of thinking in a highly monetary society, or both...' (Richard Miller). In other words, it may be symptomatic of an academic culture possessed by what Nicholas Rescher calls 'a penchant for quantities,' a 'fetish for measurement:' 'People incline to think that if something significant is to be said, then you can say it with numbers and thereby transmute it into a meaningful measurement. They endorse Lord Kelvin’s dictum that "When you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind." But when one looks at the issue more clearly and critically, one finds there is no convincing reason to think this is so on any universal and pervasive basis.' As Rescher reminds us, 'the things you cannot quantify in the context of an inquiry may well turn out to be the most important.'

Deirdre (at the time Donald) McCloskey has well documented the devastating effects the rhetoric of scientism and mathematical formalism has had on economics (acknowledging, of course, that 'it would be idiotic to object to the mere existence of mathematics in economics') in The Rhetoric of Economics (1985) and Knowledge and Persuasion in Economics (1994). A nice introduction and historical background to such concerns is found in Theodore M. Porter's Trust in Numbers: The Pursuit of Objectivity in Science and Public Life (1995).

Posted by: Patrick S. O'Donnell | Apr 8, 2007 11:54:40 AM

Bravo, Alan, for posting this--and here's my own comment about Brian Leiter's use of the LSATs to measure "quality": http://money-law.blogspot.com/2007/04/sorry-brian-you-cant-measure-student.html.

Posted by: Nancy Rapoport | Apr 8, 2007 3:32:10 PM

Good post, Nancy, though I could not get that link to work and suggest this: http://money-law.blogspot.com/2007/04/sorry-brian-you-cant-measure-student.html . You also mention that if he did not phrase it as measuring quality, you would not say a "peep." I have 2 reactions: (1) I still would object, and suspect you would too, because labeling it less 'importantly' would not slay the hypnotic and unjustified effect these numbers and ranking by them have had, and (2) your link to peeps is cute but does not rival the scientific research site http://www.peepresearch.org/ on peep social behavior, health effects of alcohol and smoking, and peep pressure.

I think you only said you would not peep in order to link peeps, which is wholly understandable, but I think you'd agree that it is high time we stop pretending that a 161 LSAT means anything different from a 159 LSAT. Yet a school with those disparate medians will fare very differently in almost every ranking. The science of those rankings is worse than http://www.peepresearch.org/ because at least the authors of peepresearch presumably know they are joking.

And there was a post on 16 dressed-up peep diaramas from the Washington Post. can't find the link now. I know a lot about peeps: horses love them.

Posted by: Childress | Apr 8, 2007 4:28:19 PM

Clarification about peeps and horses. They do love peeps, but you are supposed to let them get a little stale before feeding them to Trigger. Pierce the plastic packaging and let sit overnight before feeding them.

Posted by: Childress | Apr 8, 2007 8:03:23 PM

Clarification on rankings and Leiter: I am waiting for him to come out with a rating system that uses the objective measure of schools with the word Texas in their name not qualified by the words Southern, South, Tech, or A&M.

Posted by: Childress | Apr 8, 2007 8:06:25 PM

Post a comment