This Year's Explanation?

Another year, another set of results to explain away.  The NCBE released the national MBE mean last week, and the change over the last 2 years is massive.  The NCBE ignored 2020 results, and just compared to 2019.  The "changed test" and "different sample size" is an easy explanation.  The explanation also ignores a nearly 6 point drop from last year's July MBE score.  Excuses abound, but if the test is reliable and easy to scale (or they could have used the same test as 2019) the 6 point drop is inexcusable.  2020 graduates dealt with the immediate impact of a pandemic and social unrest.  My rudimentary understand of the LSAC reports indicates 2020's graduates had worse LSAT scores than 2021 and 2019.  Why the anomaly in scores?

I will humor the NCBE for my next query.  If 2020 is an outlier and we ignore those statistics, the 2019 comparison also doesn't make sense.  2021 results were .7 points lower than 2019.  I may be wrong, but I believe 2021 graduates had much larger numbers of the high scoring LSAT takers in the pool.  They should have been closer to 2020 scores than 2019.  If the LSAT correlates to MBE scores (which both the LSAC and NCBE claim it does), then why did the 2021 national MBE mean drop?  "Less able" test takers is no longer an acceptable answer.

I have a number of theories, and the real answer probably includes numerous factors.  Most of my theories revolve around the fundamental thought that the MBE tests more than the ability to practice law. 

2021 graduates endured longer COVID-19 interruptions.  They may have been tired of zoom and online education, and bar prep is primarily online.  The online fatigue may have led to less work.  

Students may have worked at law firms more last summer.  This thought comes from anecdotal conversations, but some jobs decreased in 2020.  When those jobs came back, students did what they could to keep jobs during uncertain times.  That may have included working at firms more and on bar prep less.  They could also make this choice because working is more fun than bar prep, and they didn't get to work as much in the previous year.  Students may have also been in harder financial times from not working the previous year.  

I could give many reasons, but overall, I believe students did less bar prep work last summer.  However, should the MBE really be a test of commitment over 10 weeks?  Why should students be required to devote that amount of time to a test regarding the jurisdiction of nowhere?  RAP, the rule of sevens, common law burglary, and many other rules provide an obstacle to practice.  The test seems to assess someone's ability to financially and emotionally devote extreme time to a task for 10 weeks.  Is that really what we should assess to become an attorney?  The NCBE's creation of a testing task force implicitly confirms the MBE is a poor instrument, but they continue to administer it.  Shouldn't we stop using a poor instrument even if the alternative isn't ready?  Many questions, but my guess is all we will hear is *crickets*.

(Steven Foster)