Sunday, July 14, 2019
I still remember the kindly judge for whom I interned as a 3L. Knowing that bar prep was coming up and sensing my anxiety, he called me into chambers. “Louie,* have a seat.”
(* Remember, we’re talking about Boston. Anyone named “Louis” is called “Louie,” whether they like it or not. My co-clerks in the Superior Court were “Sully,” “Fitzy,” and “Other Sully.”)
Anyway, “Louie,” he said, “you’re a smart* kid. If you do half the work in that bar prep program, you’ll pass just fine.”
(* I’ll note that this is properly pronounced “smaahht.” See supra at Boston.)
He continued: “My firm* gave me two weeks off to study for the bar, and I did just fine. So stop worrying about spending three months studying.”
(* If I remember correctly, the firm was called “Oldguy, Oldguy & Deadguy, LLP.” Somehow, they made the group of Dan Aykroyd’s business school chums in “Trading Places” look like the picture of diversity.)
My judge’s advice was well-intentioned, and I appreciated his attempt to calm me down. But, the Type-A, neurotic kind of guy I was (errata: am), mostly ignored this advice and studied with the kind of ferocity only those with a festering inferiority complex can muster.*
(* I can thank my significant other at the time for the bar exam-related inferiority complex. An Ivy League law student, she’d repeatedly say, “It’s not like you went to Harvard.” Luckily, our relationship did not last much longer. Ironically, neither did her legal career.)
Many of our students are not so lucky, though. They hear this same tone of advice and happily digest it as a welcome counterthesis to the admonitions of that overly-intense ASP/ bar exam professor. “The partner at my firm said that Schulze is crazy.”*
(* A fair point. No objections so far.)
“You don’t need to do 1,500 Adaptibar questions or whatever. Just watch the videos, read the outlines, and you’ll pass.” The student then spends a relaxing summer watching some videos, hanging out with friends, and going to the beach. (Meanwhile, I’m in my office slowly rocking back and forth in the fetal position after seeing the student's stats and completion percentage data.)
Then, the student fails the bar exam.
The practicing lawyers who give this advice might think that the bar exam world remains a static place where nothing changes. But, the substantial changes to the bar exam over the last five to ten years severely limit the applicability of their experiences. Here are those changes and why practicing attorneys need to be careful with their advice.
First, the National Conference of Bar Examiners has made changes specifically to make it harder for the “smart kids” to pass just because they are smaahht. Remember those questions, let’s call them “Roman Numeral Style,” where you get a fact pattern, then a call of the question, and then a series of roman numeral-enumerated statements? Then, the answer choices say something like “I and II but not III” or “II and III but not I.” And, do you remember how the Roman Numeral Style alone allowed you to eliminate certain answers just based on their enumeration and not a substantial understanding of the law?
Well, the NCBE got rid of those. In fact, the NCBE’s methods have become much more sophisticated. They now rely heavily on the advice of psychometricians – literally, people who have doctoral degrees in the science of testing. Those psychometricians carefully improved the format and substance of the MBE questions to make them more “valid, reliable, and fair.” Those are terms of art, not general terms, and psychometricians can test questions for validity, reliability, and fairness. As a result, the questions are “tighter” now in that they actually test knowledge of the law and analytical skill rather than just whether someone can logically game the questions. Therefore, the smart kids cannot (usually) pass without genuine study.
Second, the advice regarding study time is similarly flawed. Many practicing attorneys do not know that the MBE added a seventh subject of Civil Procedure and that doing so impacted each applicant’s “cognitive load” in a negative way. In addition, many jurisdictions have raised cut scores and/ or added subjects to the state portion of the exam. And, on top of THAT, many jurisdictions have diversified the way they test on the bar exam, especially the dozens of jurisdictions adopting the UBE. When I took Massachusetts in 2000, for instance, we took the MBE like everyone else and then suffered through enjoyed ten local law essays on the Massachusetts day. Now that Massachusetts has adopted the UBE, however, it includes multiple choice questions, very different essays, and a performance test. As a result, students simply have more to study and practice than they did when I took Florida in 1997 and Massachusetts in 2000. As such, they do indeed need to put in more time than perhaps was the case in the past.
Third, we know a lot more now about the science of learning than legal educators knew even just a few years ago. Back when my judge took the bar exam -- when the cognitive load was not as substantial, the testing itself was less precise, and cut scores were more favorable -- more students could get away with weak study methods. They simply had more margin for error than students currently taking the exam. Therefore, telling a 2019 student just to watch videos and read outlines repeatedly, which cognitive science tells us is less effective, sets them up for falling below that margin of error and producing a failing grade. Do you know how Adaptibar uses interleaved learning? Do you know anything about spaced repetition, metacognition, and forced recall practice? Do you know the date on which students should switch from massed to mixed practice? Do you know what score students should achieve on their bar prep company’s midterm? Do you know how the UBE changed your jurisdiction’s exam? Have you attended multiple conferences at the NCBE about how they test? If you do not know these things, you should not be tinkering with a student's microscopic margin of error for passing the bar.
Now, I am not saying that you do not play a critical role in students’ bar passage. You absolutely do. Please, absolutely tell students to work hard. Let them know that they are capable of passing if they do the work. Tell them about your experience walking into the bar exam and hearing some moron say that the Supreme Court just ruled all torts unconstitutional. These messages absolutely help. But, my concern here is with blanket statements (based on a single experience from 20 years ago with an exam that no longer exists) that will undermine my instruction based on current information and our full-time attention to this field over the last decade. Just as trying some cases from 1997-2002 does not make me an expert on managing a substantial litigation caseload in 2019, sitting for one bar exam in 1999 does not make one an expert on taking a very different bar exam in 2019.
In sum, and bringing my Boston references full-circle: The Boston Red Sox lost the 2003 American League Championship because Manager Grady Little relied on strategies from the 1980s. Just a year later, Terry Francona won Boston’s first World Series in 86 years because he relied on experts using sophisticated data analyses and 21st-century methods. If you shower students with bar preparation advice from Grady Little’s 1980s, you are making it more likely that you will be paying a lot of salary to associates/ ADAs/ staff attorneys who cannot practice law because they failed the bar exam.
Please support our messaging, and encourage your new would-be lawyers to work hard and study smart.*