Dear Practicing Attorneys: Please Stop Giving Our Bar Students Inaccurate Advice.

I still fondly remember the judge for whom I interned as a 3L.  Knowing that bar prep was coming up and sensing my anxiety, he asked me about my plan.  I told him that the bar prep company each day would provide lectures, outlines to read, some more outlines to read, and then finish things off with some outlines to read.  When I told him that the program started just after Memorial Day and ended the day before the exam, he was astonished.  His advice was to save myself all that money, take three weeks off from work, and study from July 4^th until the exam.  He said that would be plenty.

Of all the advice my judge gave me, this was the one bit I did not take.  His guidance was well-intentioned, and I appreciated his attempts to calm me down.  But as the Type-A person that I am, I could not rest without feverishly checking off each scheduled study item.  His was advice I could not take.    

Twenty-something years later, students still receive that advice.  They insist:  “The partner at my firm said that she took just two weeks off for the exam and did just fine.”  The partner professed:  “You’re a smart kid.  You don’t need to do all that work.  Just watch the videos, read the outlines, and you’ll pass.”  Happy to internalize this message so as to mentally corroborate the partner’s flattering assessment, students’ confirmation biases drive them to adopt suboptimal learning behaviors. 

And then they fail the bar exam.

The practicing lawyers who give this advice sometimes believe that the bar exam world is a static place devoid of change.  However, recent substantial reforms severely limit the applicability of their experiences.  Below the fold, I describe those changes and how they require more careful advising. 

First, the NCBE has improved the exam¹ specifically to make it harder for the “smart kids”² to pass just because they are smart.  Remember those questions, let’s call them “Roman Numeral Style,” where you get a fact pattern, then a call of the question, and then a series of roman numeral-enumerated statements?  Then, the answer choices say something like “I and II but not III” or “II and III but not I.”  And, do you remember how the Roman Numeral Style alone allowed you to eliminate certain answers just based on their enumeration and not a substantial understanding of the law? 

Well, the NCBE got rid of those.  In fact, the NCBE’s methods have become much more sophisticated.  They now rely heavily on psychometricians – literally, people who have doctoral degrees in the science of testing.  Those psychometricians carefully improved the format and substance of the MBE questions to make them more “valid, reliable, and fair.”  Those are terms of art, not vague colloquial verbiage, and psychometricians can test questions for validity, reliability, and fairness.  As a result, the questions are “tighter” now in that they actually test knowledge of the law and analytical skills rather than just whether someone can logically game the questions.  Therefore, the “smart kids” cannot (usually) pass without genuine study.

Second, the advice on study time is similarly flawed.  Many attorneys do not know that the MBE added a seventh subject of Civil Procedure and that doing so negatively impacted applicants’ “cognitive load.”  In addition, many jurisdictions changed cut scores and/ or added subjects to the local part of the exam.  On top of THAT, many jurisdictions have diversified the way they test, especially the dozens of jurisdictions adopting the UBE.  When I took Massachusetts in 2000, for instance, we took the MBE like everyone else and then encountered ten local law essays on the second test day.  However, now that Massachusetts has adopted the UBE, it includes multiple-choice questions, very different essays, and a performance test.  As a result, students simply have more to study and practice than they did when I took Florida in 1997 and Massachusetts in 2000.  As such, you should be advising applicants to complete more work, not less.

Third, we know a lot more now about the science of learning than even just a few years ago.  Back when my judge took the bar exam, the cognitive load was not as substantial, the testing itself was less precise, cut scores were more favorable, and more students could get away with suboptimal study methods.  Applicants simply had more margin for error than today.  Therefore, telling a 2022 student just to watch videos and read outlines repeatedly (methods cognitive science tells us are less effective) sets them up for falling below that slim margin of error.  Do you know what Adaptibar is?  Do you know the date on which students should switch from massed to mixed practice?  Do you know what score students should achieve on their bar prep company’s midterm?  Do you know how the UBE changed your jurisdiction’s exam?  Have you attended multiple conferences at the NCBE about how they test?  Do you know anything about spaced repetition, metacognition, and forced recall practice?  If you do not know these things, you should not be tinkering with students’ microscopic margin of error.

I am not saying that you do not play a critical role in applicants’ bar passage.  You certainly do.  Please, definitely tell students to work hard.  Let them know that you believe they are capable of passing.  Tell them about your experience walking into the bar exam and hearing someone say something ludicrous like “the Supreme Court just ruled all torts unconstitutional.”  These messages absolutely help.  But my concern is with blanket statements (based on a single experience from 20+ years ago with an exam that no longer exists) that will undermine our instructions based on current information and extensive full-time attention to the field.  Just as trying some cases in 1997-2003 does not make me an expert on litigating in 2022, sitting for a single bar exam in 1997 does not make one an expert on taking a very different bar exam today.

In sum, your well-intentioned advice should not contradict the experts.  We have studied the correlation between completion percentage and bar passage.  We have collaborated with hundreds of other academic and bar support faculty around the country.  We have had training from those who actually create and grade the exam.  Your encouragement given to your new associate/ Assistant District Attorney/ Assistant Public Defender is invaluable.  But please realize that inaccurate advice will hurt your would-be lawyer and leave you with an employee without a bar card.     

----------

[1]  I do not mean to express an unqualified endorsement of the bar exam status quo.  Serious issues exist in how we currently test bar applicants, the most important of which is how the exam impacts Students of Color.    

[2]  A more accurate term would actually be “privileged kids.”

 

Louis Schulze

FIU Law