Law School Academic Support Blog

Editor: Goldie Pritchard
Michigan State University

Sunday, June 9, 2024

More things change, the more they stay the same

The NCBE released results of the NextGen pilot testing last week.  You can read their report here.

Amazing ASPers throughout the country already started the discussion within our Listserv and on conference calls.  I don't want to reiterate (and/or plagiarize) their wonderful insights.  If some of my arguments sound familiar to someone else's, I sincerely apologize.  I will also provide my shameless plug for joining the bar advocacy committee.  Our complaints are being heard by bar associations, so you are making a difference.

My initial thought turned into the title to this post.  NextGen was supposed to be revolutionary and assess what lawyers do on a daily basis.  The test purported to require less memorization and focus more on practical skills.  The vision was grand, but the result seems to be a lackluster assessment in a slightly different format.

First, and probably most important, the report does not provide transparent data for performance differences among groups of participants.  The report vaguely states that the test narrowed the performance gap with some of the new questions.  The report does not fully say how much narrowing or which groups saw improved scores (it gave a few small examples).  The report also doesn't indicate the baseline for determining "examinee competency".  If the NCBE still uses prior standardized tests (LSAT mainly) to create the baseline for what performance difference is acceptable, then even small narrowing makes the new test seem better.  Most of us argue LSAT/standardized test differences are not based on competency.  Without knowing their baseline of what gaps are expected, I don't trust their argument the new test doesn't continue patterns of discrimination.  Slightly less discrimination isn't persuasive.  I want to be completely wrong here, but the impact is too great to not demand more transparency. 

Second, they aren't following their goal.  Attorneys don't memorize mountains of rules, so proponents of NextGen said it would move away from memorization.  I will concede that a slight decrease happened (no Secured Transactions, yay!).  However, NextGen continually adds back more material to memorize.  Family Law is back on the exam.  The starred vs. non-starred debate rages where we all know students will need to deeply know/memorize material even though the outline says to have a general understanding.  General understanding is too vague to create a good plan, and any reasonable person would over-prepare for the bar exam.  Wills, Trusts, and Estates keeps inching closer to fully back on the exam.  The report says that providing the Federal Rules of Evidence didn't help students/they didn't use them.  Of course students who already studied or engaged in some form of studying (either in bar prep or a for-credit bar class) didn't use the rules.  They already committed them to memory, or possibly, the stakes weren't high enough for them to pull up the rules because missing the questions had no impact on individually passing the exam.  

My last statement is about technology.  I believe the report makes clear the NCBE cares more about efficiency than true assessment.  The report discussed technological efficiency and ability to administer on computer.  The report (and all pilot tests to my knowledge) ignore the cognitive requirements of a 100% online exam (with multiple page PTs).  Why didn't the NCBE give a large number of participants the test on paper and the rest on computer?  My guess, because the answer would be clear.  Hard copies are better for active reading, notes, etc.  Students would most likely perform better with paper exams.

I sincerely hope I am wrong about my assessment of NextGen.  I hope it is a completely different and fair assessment.  My fear is that we are getting a different format of a test that will continue to determine who can reach their dream of practicing law based on screen reading, memorization, fast toggling, standardized testing, and a slew of other irrelevant skills.

(Steven Foster)

https://lawprofessors.typepad.com/academic_support/2024/06/more-things-change-the-more-they-stay-the-same.html

Bar Exam Issues, Bar Exam Preparation | Permalink

Comments

Post a comment