Will MBE Scores Be Artificially Low in July?

COVID-19 continues to cause problems throughout the US.  I don't want to minimize others' struggles, but the effect on the bar exam is still yet to be determined.  Some states' postponement until September should help social distancing to keep students safe, but will it decrease MBE scores for July takers?

The MBE is scaled based on anchor questions.  Anchor questions are questions that are used across multiple tests.  The NCBE won't state how many anchor questions are on the 200 question test, even when directly asked at their conference for law schools.  My best estimate is between 50 and 60 questions are anchor questions.  Anchor questions help produce scaled MBE scores.  Scaling attempts to ensure a score in one year is equivalent to the same score in other years.  A basic explanation of scaling is the NCBE compares the scores of current takers as a whole on the anchor questions to previous years.  They then conclude whether the group of takers is stronger or weaker than previous years.  The MBE is then graded as a whole considering how the takers as a whole did on the full exam compared to previous years.  The process determines the difficulty of the test and produces the scale. 

The scaling process makes assumptions about the candidate pool based on scores on the anchor questions.  With numerous bar exams postponed to September, the question becomes whether the different candidate pool will change the assumption about the quality of takers.

I think the clear answer is yes, scores will be effected, and the scores will be lower.  9 out of the top 11 and 21 of the top 31 US News Ranked Law Schools are located in states postponing the bar exam.  I will acknowledge that US News Rankings are a poor way to rate schools.  However, the reliance on the LSAT has relevance here.  The NCBE states the LSAT correlates to MBE scores (they gave the precise relationship at the aforementioned conference for law schools).  If students with the highest LSAT scores are not taking the exam until September, the July takers will appear to be weaker than previous years.  Even if those same postponing states include students with low scores (ie - California with non-ABA school takers), the pool itself will not have the outlier high MBE scores.  The risk is significant that scores will be effected merely by the applicant pool.  

The NCBE may correct for this phenomenon.  They could wait until after the September exam to score the test.  They could have information that those states would not affect the general average of the pool.  However, the NCBE is so opaque that we have no idea what they plan to do to correct for the potential problem.  They could have supported any number of alternatives to the bar exam and MBE this summer.  They chose to stand by the current bar exam in a pandemic when even the most basic understanding of scaling raises serious concerns about scores on the July exam.  I would call on examinees to demand answers, but I witnessed numerous student groups around the country be completely ignored by state bars.  Now is the time for Supreme Courts to begin understanding the mechanics of the bar and demand more from the NCBE.  As of now, I truly believe some July takers will be unsuccessful solely based on NCBE practices and not competency to practice law.

(Steven Foster)