Monday, April 14, 2008

More on the MPRE Minimum Passing Score, How Do States Pick a Number?, and California Takers Just Dodged a '100' Bullet

Posted by Alan Childress

We had this guest-post last week, discussing the MPRE passing score and where certain scores fall on a distribution or curve.  After, guest-poster and statboy Dr. John McSweeny wrote these follow-up thoughts (including some specific pass rates for certain MPRE scores):

I would have loved to be the fly on the wall when the discussions took place at various state bars about what cut points to use, especially given that there appears to be no validity data.  That is, we don't really know what score, if any, predicts a career of ethical practice. 

The discussions must have boiled down to how many students they were willing to pass (or flunk).  A score of 85 passes roughly 75% of the takers, a score of 80 passes a little more than 85%, and a score of 75 passes about 90% of all takers. 

What about those states that use odd figures like 79 or 86?  Were these numbers the result of compromises made by a committee? Of course, there is no logical or mathematical reason for preferring a nice, round number like 85 instead of 86.  They just seem more aesthetic if they are divisible by the number of fingers on one of our hands!
I don't know the answer to the how-they-chose question, though I welcome comments and links to any source that explains the policies or politics behind these.  I note, however, that California long had a relatively easy passing score of 79, then its Committee of Bar Examiners published this 2004 proposal that would have made the passing score 100.  By John's numbers, assuming the national mean is close to the median (and I would hope it is), that would have meant that California would have switched from passing more than 86% to flunking over half!  It is pretty hard to justify such a major change of policy, and a passing score so much higher than everyone else's. 

And can we really say that those at the top of a bell curve are really distinguishable and worthy of minute parsing which would result in huge impact on careers for similar-scoring people at the national median?  That the average candidate is unworthy of becoming a lawyer?  I can live with the arbitrary line-drawing at the left tail where the candidates have already missed a lot of questions, but I am uncomfortable with labeling an above-average kid who got a 99 as unethical while his friend making a 100 lucks out.  (Especially if the difference is not in the raw numbers of questions missed, but rather in the scaling done after to correct for a historically 'easier' exam, by a hair.)  Anyway, it seems inherently wrong to me to focus any crucial cut point at the top of a bell curve!

California did wind up -- its says here in this official notice dated Dec. 2005 -- raising its passing score to 86, effective Jan. 2008.  Takers in the fall or earlier can pass with the old lowbar, 79.  That now makes California at the highest end of state minima, compared to almost all states.  (See this 2005 summary chart, state-by-state [though it still shows Cal as 79, and keep in mind that New York is now 86 85], or this chart from 2004 also by states).  Cal's new 86 bar is still quite a jump from that lax 79 legacy.  (Here is an Inter Alia blog post on the switch to 86, and also this one from John Steele in '05.)  I don't know how 86 was picked over 100, though I assume it was a choice -- a public statement -- to be the highest/hardest anywhere (tied with Utah and just above New York), if not that euphonic 100.  At least the above-average test-taker will not flunk, to piss off Lake Wobegon.

UPDATE:  See article on MPRE statistics from Dr. Susan Case, head of testing for NCBE, here.

http://lawprofessors.typepad.com/legal_profession/2008/04/more-on-the-mpr.html

Bar Discipline & Process | Permalink

TrackBack URL for this entry:

http://www.typepad.com/services/trackback/6a00d8341bfae553ef00e551cd9a8c8833

Listed below are links to weblogs that reference More on the MPRE Minimum Passing Score, How Do States Pick a Number?, and California Takers Just Dodged a '100' Bullet:

Comments

I think the reason for CA to set the minimum passing score on the MPRE at an unrounded, scaled 86.0 is that the unscaled score of 86 is equal to 7 questions missed out of the 50 (of 60) that count. The fact that the score is then scaled just did not matter as it was probably considered a neutral fact with scores being nudged up or down slightly each year from the raw score. It would be interesting to know the range of how many points the scale generally swings the score; it may not be more than the 2% which each question is worth in the raw score.

Posted by: David | Mar 7, 2010 10:29:33 AM

Well, I clearly advertised my ignorance on the scaling of the MPRE in the post I submitted earlier. I am, frankly, flabbergasted that a passing score is generally around the 50% mark, although, having taken the exam yesterday, I can say that a good number of the questions turn on a very subtle difference between 2 possible answers. I am embarrassed to say I only studied for about 5 hours the night before and wished I had spent double that when trying to remember in which circumstance the attorney was required to obtain the client's signed, informed consent compared to merely requiring that the attorney give full, written disclosure. As it was, on the number of questions where I was simply guessing between two answers, I despaired of passing when I thought I could only miss 7 questions of whichever 50 were to count. However, I am much more confident now that I know the probability suggests I will pass when I was able to eliminate 2 incorrect responses on perhaps 80% of the questions (figuring I knew the correct response to more than 20% of the questions).

Posted by: David | Mar 7, 2010 12:42:55 PM

Well, I took the MPRE this morning. For a scaled score of 86, about how many questions do I have to get right? I have googled this question and can't seem to find a decent answer. Any guidance?

Posted by: Nicholas | Aug 6, 2010 10:13:46 PM

Hey, Nicholas and David, Congratulations on this next step to your law license (and keep reading Mike's posts here at our blog to see how to keep it!). Nicholas, no one can answer that question exactly since it varies from administration to administration and by the scaling. That is true of most such nationwide exams with a mix of questions, old, new, and experimental. I am sure, as he is, that David's first estimate that 8 missed questions would fail you, is wrong, and believe that his second Comment is closer to the mark. Thus there is a wide margin for error even in the toughest (86) states, and certainly not just getting 7 out of 50 or 86% right, as he first wrote. I very much appreciated that he came back and fixed that since it may have caused panic among surfers! I agree with him, though cannot know for sure [the difference between knowledge and belief is one key to questions on witnesses' lying, right?], that even getting about 25 or 26 right should pass, out of 50, but that would not be true if the particular exam has too many easy questions and it gets scaled down. Yet even then it would not get close to the 8-wrong-fail precipice he first thought. I would think that 30 of 50 or 60% is safe in most states, and 33 is really safe.

Sounds too easy, but David is right that a very ethical person could easily not see the difference between two choices. And even if you picked the safest thing to do -- just go ahead and give written disclosure and get written consent -- that could be wrong because it is more than is required. So you do have to study for this.

All of this is why I originally wrote the the 100 proposed by California was too high. There is no reason why half the students should fail, especially an exam with intentionally hard and subtle questions. I like it the way it is in most states: you could not pass it without studying specifically for it, and knowing all the basics of the Model Rules, yet once you have done that you should pass, especially if you take sample questions before as part of your study.

Any suggestions on good or bad study aids currently?

Posted by: Alan Childress | Aug 7, 2010 8:56:16 AM

As far as the test went, unless you are a fast reader, 2 hours is really not enough time to answer the questions. The questions I recommend studying for involve fees, client conflicts, and government agency conflicts when Ls go into private practice dealing with same matter directly. Take your time and don't show off by finishing first, you'll never see these people again.

Posted by: chris m | Mar 17, 2012 4:40:35 PM

I just received my score back from the August exam. I received a scaled score of 97 after only studying 1-1/2 hours in my car on the morning of the exam from a Kaplan outline that I bought on eBay - only reading the marked highly tested sections. The score was by no means stellar but good enough to pass. I honestly thought I failed. I was guessing way to many times as a great many questions turned on very fine rule distinctions between two choices.

Posted by: Leroy S | Sep 18, 2013 5:04:26 PM

Post a comment