Friday, April 11, 2008

What is the Average MPRE Score and What Percentage Passes? ... And Where Does Your Score Fall?

[The MPRE is required in most jurisdictions before admission to the bar, and each state mandates its own passing score, usually between 79 and 86, out of a scaled 150 possible.  Jeff posted last year on some useful observations on the exam process, and also mused on the numerical realities of it -- with helpful comments by Bill Henderson of IU.  Following up on that, we have a guest-post from John McSweeny, Ph.D., a psych professor in the med school of the University of Toledo with expertise in stats.  By night, he is also a UT law student and a recent taker (and passer) of the MPRE.  Jeff reports that John's findings as reported below are in line with how Bill Henderson later explained it last year to Jeff on a cocktail napkin [how all great teaching is done].  I thought our readers and MPRE takers would want to see these numbers and John's observations. -- Alan Childress]

      Given my background in statistics and psychometric methods, I was curious about MPRE scores and what they meant, including where certain scores fell in the distribution of scores from all of the candidates who took the test.  Accordingly, I contacted the American College Testing program (ACT), which developed the test, but they wouldn't tell me anything beyond what was on the website, i.e., that the range was 50 to 150 and the mean [average] was 100.  Of course, this irritated me to no end and launched me on a quest to find out the information I wanted.

 
With a little bit of searching I found that the National Council of Bar Examiners [which apparently contracts the MPRE out to ACT] has a website where you can download a PDF file with the statistics for the MPRE including the means and standard deviations, on an annual basis. These are excerpts from The Bar Examiner. If you want to go to the site it is here.
 
Looking at the last two years which have been reported, we can see that in 2005 the actual mean was 98.72 and the standard deviation, or SD, was 19.72; for 2006 the mean was 97.97 and the SD was 19.72 (same as in 2005).  A weighted average mean from these two years = 98.33 and the SD (of course) = 19.72.
 
You can use this information to estimate a Z-score with a mean of zero and an SD of one and then go to a table to determine an approximate percentile equivalent.  The formula is Z = (X - 98.33)/19.72 where X is your scaled score.  Thus, if you received a score of 85 (the passing score in Ohio [and many other states]), Z = - 0.676.  This would be equivalent to an approximate percentile of 25, i.e., 75% of the persons taking the test did better than you did.  Thus, it appears that the magic number of 85 used in several states, including Ohio, is set at the 1st quartile.  Of course, I do not know this for a fact, but the evidence sure points in that direction.
 
Some caveats are in order:
 
1) The distributions appear to vary slightly from year to year.  Thus, the distribution for 2008 may be slightly different from that in 2005 and 2006.
 
2) The scores do not appear to be entirely normal (especially in 2006) and the percentile equivalents for Z scores are based on the standard normal distribution.  Thus, these statistics give you only an approximation of where your score lies relative to others. 
 
A few more comments by John appear below the fold.
[UPDATE:  Two follow-up posts here and here discuss the curve and numbers with more detail, link to state-by-state minimum scores (including news on NY and California), and note an article written by the head of number crunching for the NCBE about the exam. Plus more from John to follow in this post...]

Further observations [by John], cont.:

 
1) There appears to be a slight downward drift in the mean since the test was standardized in 1999.  I believe that the scores were set to have a mean of 100 and an SD of 20 in 1999.  If we use our Z-score formula for a scaled score of 85 we get Z = -0.75 which is approximately equivalent to the 23rd percentile.  The difference in practical terms from the figures I used previously is small, and people like scores that are whole numbers rather than scores with decimal places.

 
2) I checked the 1999 stats and interestingly the actual mean listed was 99.07. No SD was listed but the percentage of candidates exceeding different scores was listed; 85 indeed was at the 25th percentile.

 
3) It would be interesting to know if there has been any attempt at a criterion validity study, i.e., does one's MPRE score have any predictive value with one's tendency to be a recipient of professional discipline or malpractice lawsuits in one's career as a lawyer? Carrying out such a study would be a daunting (and expensive) task, however. We will probably never see it.

 
4 ) The reason that ACT is reluctant to release MPRE test score information may be due to their contract with the National Conference of Bar Examiners (NCBE) , which is probably the organization that owns the intellectual property, i.e., the MPRE, and the rights to release information about it. Thus,  ACT is likely under contract to provide testing services for NCBE. This is a different from the situation with the ACT College Admissions test, which is owned by ACT itself.  Again, just a guess, but it seems plausible.

http://lawprofessors.typepad.com/legal_profession/2008/04/what-is-the-ave.html

Bar Discipline & Process | Permalink

TrackBack URL for this entry:

http://www.typepad.com/services/trackback/6a00d8341bfae553ef00e551ca95018833

Listed below are links to weblogs that reference What is the Average MPRE Score and What Percentage Passes? ... And Where Does Your Score Fall?:

Comments

Your refer to referencing "a table" when calculating what percentage of people did better than you. Where can one find this table?

thanks,

arielle

Posted by: arielle Anderson | Dec 9, 2008 12:12:07 PM

Arielle,

The "table" I was referring to is in most basic statistics books. It concerns the area under different points in the standard normal distribution.

Posted by: John McSweeny | Jan 8, 2009 7:41:04 PM

I don't understand.. when i plug in my score of 126 into the formula, I get Z = 1.40..? how do I convert that into a percentage like you did with [.676 = 75%] ??

Posted by: Matt | Apr 11, 2010 8:25:32 AM

I took my ethics class in law school seriously. I took studying for the MPRE seriously. But, folks, out in the real world most attorney ethics rules are ignored, especially when it comes to billing. No one cares; and if you try to report it, not only will the bad guys get away with it, but you can kiss your career goodbye.

Posted by: Tom Smiley | Apr 12, 2010 9:15:32 AM

use this site to move from z score to percentile:

http://davidmlane.com/hyperstat/z_table.html

Posted by: mpre taker | Apr 15, 2010 8:33:51 AM

I passed the MPRE confortably on the first try, but I sympathize with those who have had difficulty. The big secret to the MPRE is not that it is easy, but rather that they don't expect you to get too many questions correct.

Part of the problem is that many of the ABA Rules of Professional Conduct are broad and vague. Many of them could be read to prohibit conduct that is both necessary and beneficial to the efficient and accurate delivery of legal services.

For example, the rules on confidentiality (1.6 and 1.9(c)) purport to prohibit "revealing" any information that somehow related to a current or prior representation, even if that information is generally known. There is no exemption for information that is generally known or readily ascertainable to attorneys or relevant subject matter experts. They only allow "using" but not "revealing" such information. By contrast, employment agreements and other non-disclosure provisions in business and technical fields typically include these broader exemptions, as such non-proprietary information is considered the employee's stock in the trade.

In theory, any attorney who specializes in a given type of case violates the literal language of this rule. An attorney who has defended multiple DUI cases would cite the same technical information regarding field sobriety tests and breathylizers from case to case. If a case results in a published decision, the attorney would need to cite it, thereby revealing it, in future cases dealing with similar fact patterns or legal issues, just as any other attorney would.

Similarly a patent attorney needs to be able to access generally known information about the subject matter, information he may have used in prior cases not adverse to the current one. Most patent applications and all issued patents from the attorney's prior work are published and become part of the "prior art" in the field. Any competent subject-matter expert is expected to be able to find and cite to these materials. If the confidentiality rules were consistently enforced as literally written, it would be hard for clients to find attorneys with knowledge and experience in the relevant subject matter.

Some attorneys at CLE events and some professors also tell "war stories" about publicly known aspects of prior cases to help teach new lawyers. These experiences can be valuable training, yet unfortunately are technically violations of these broad rules as currently written.

In some ongoing cases, there may be very good reasons not to reveal certain generally known information, but those situations could better be dealt with in other rules specifically designed for those purposes, instead of including non-confidential, public domain information in a confidentiality rule as they currently do.

This is just one example of how the current rules are overly broad, and thus make it difficult for practitioners to figure out what to do in practice, and for MPRE test takers to figure out which answer the test makers desired. Perhaps eventually the ABA will clean up some of the rules, so they are more narrowly tailored to prevent the specific harm that prompted them, while not prohibiting potentially legitimate, beneficial conduct. For starters, they could clean up these confidentiality rules as I discussed here to exempt information that is generally known or readily ascertainable by prudent attorneys and subject-matter experts.

Posted by: Should You Be A Lawyer Website | Sep 10, 2010 10:40:31 PM

Have these score results changed significantly since the advent of all of the bar exam preparatory courses that are still on the market? I am talking namely about barpassers, beatthebar, BarBri, Attack Sheets, and Kaplan. It seems that recently its become easier to pass at least the CBX because in great part to these mnemonic tools. It would be an interesting study to find out what, if any, correlation exists between general trends in test scores and these mnemonic tools.

Posted by: Elizabeth Waverly | Dec 28, 2010 3:29:59 PM

"Have these score results changed significantly since the advent of all of the bar exam preparatory courses that are still on the market? I am talking namely about barpassers, beatthebar, BarBri, Attack Sheets, and Kaplan. It seems that recently its become easier to pass at least the CBX because in great part to these mnemonic tools. It would be an interesting study to find out what, if any, correlation exists between general trends in test scores and these mnemonic tools."

Yes, good question Elizabeth. I would be curious to find out the answer of this question too!

Posted by: Michael Westerland | Dec 30, 2010 9:09:43 AM

Dear Psych PhD.
When one comes o this posting looking to find an "average" score for this exam, we, as "law students," are [typically] neither aware nor concerned with learning about the nature of statistical distribution, at least insofar as concerns most technical jargon (e.g., standard deviation, "z score," t-tails, analysis of variances, etc.).
Please, just a layperson's estimate... AVERAGE SCORE????
I promise not to hold your reputation to such a response.

Thanks a bunch

Posted by: R.Courte | Sep 7, 2011 8:36:58 PM

In laypwersons' terms: Average MPRE score = 100, 25th percentile = 85, 75th percentile = 115.
(Note: These are approximations but close enough for practical use).

A. John McSweeny, J.D., Ph.D. (Now a Lawyer too)

Posted by: John McSweeny | Jan 3, 2012 10:55:37 AM

O.K. Percentiles are great. But how many questions do you have to get right in order to avoid being in the bottom quartile? I understand that it varies with each administration of the test, but there has to be a predictable range. So does anybody have a range of correct answers one needs to have in order to safely get a scaled score of 85? Do we need 25 correct answers out of 50? 35? 45? 55?...

Posted by: Joe Dokes | Mar 17, 2012 3:19:10 PM

So all these words and not a real answer?

How many actually have to be answered correctly to pass?

Posted by: John | Aug 7, 2012 9:42:28 AM

I can dig all the nifty math, but Mr. McSweeny and I would still like to know approximately what raw score is needed to achieve a scaled score of 85.

Posted by: W. McCabe | Nov 2, 2012 1:43:51 PM

Agreed with the last three posts! The only question that any test-taker cares about is, how many questions out of 50 (or 60 including experimental) do we need to get right in order to pass? I love statistics but not when it answers the wrong questions!

Posted by: MPRE Taker | Oct 30, 2014 2:13:05 AM

I don't speak for John, but I am pretty sure that the answer is: it varies with each test and administration of it. It is a scaled score. It is relative to other takers' scores, even after adjustment for the history of a particular question. John's post answers the only question that can meaningfully be asked, unless the administrators ever posted a list of the average passing scaled score over the past few years (and even then, passing based on which state's requirement?). It's a little like asking what score you need to win a football game: I could tell you that if you score 50 you are very likely to win, or that if you score 3 you are very likely to lose, but I could never tell you what the minimum winning score is, in advance, without more information (and even then it'd be a statistical guess based on what has been enough in most cases). So I honestly do think John is answering the right question, and would be wrong if he told you in advance that the minimum time to win the women's New York Marathon is, say, 2:21.12. But I get that it'd be useful to know that "mark" and know that if you "shoot for" that time/score, you'd have an x percent chance of winning the race/passing the exam. The examiners have not released such a prediction or history. I do think knowing the percentages, mean, and probabilities is very useful, so I appreciate John's rundown.

Alan

Posted by: Alan Childress | Nov 5, 2014 7:51:57 AM

Post a comment