Law School Academic Support Blog

Academic and Bar Support Scholarship Spotlight

Jennifer A. Gundlach and Jessica R. Santangelo, An Empirical Study of the Relationship Between Metacognitive Skills, Performance in a Bar Prep Course and Bar Passage, 1 J. Law Teaching & Learning 100 (2024).

This article builds on our prior research about metacognition and its importance for law students’ learning. We hypothesized that given our past findings about the relationship between metacognition and academic performance in law school, it was possible that metacognition might also play an important role in success on the bar exam.

Our current study documents law students’ metacognitive skills during a final semester bar prep course and examines the relationship between those students’ metacognitive skills and bar passage. We found that students are capable of gaining metacognitive knowledge and regulation skills during law school and even as late as the last semester of law school. We also found evidence that instruction and prompts to practice metacognitive regulation during the first year of law school had a long-term impact on students’ continued use of those skills. This evidence is important because we also found, as we have in prior studies, that students’ success in a final semester 3L bar preparation course, as well as their cumulative law school GPA, are associated with their level of metacognitive knowledge and regulation skills. While we did not find evidence of a direct relationship between metacognitive skills and bar passage, there was a relationship between bar passage and both course performance and cumulative GPA. Accordingly, we contend that metacognitive skills are an indirect support of bar passage given that they contribute to success in law school, which in turn supports success on the bar exam. We conclude that, based on the relationship between metacognitive skills, academic success in law school, and bar passage, law schools have an ethical obligation to support law faculty in explicitly and intentionally incorporating metacognitive skills instruction into the law curriculum.

[Posted by:  Louis Schulze, FIU Law]

The Importance of Being AASE

I love Oscar Wilde’s The Importance of Being Earnest. I mean who doesn’t love a fast talking piece that is entirely a play on words from beginning to end? It was an oasis in an otherwise uninspired year of high school English reading for me.[1] As one would guess, it is convoluted and hard to wrap your brain around. It all seems a bit unknowable-almost the same way the breadth and depth of academic support work can get lost in translation. We all know how exhausting it is to tell people what we do over and over.

I also think most people I have met in law school academia find it “interesting” that there is a national organization of academic support educators.[2] But as we move towards more status equity, having this resource will prove imperative. We need to find a way to get to a critical mass of tenured and tenure track positions (like Legal Writing faculty have done) in order to gain better equity footing across the board. This is where having a national organization is key. As a national organization, we can collect and share voluminous data about how ASP faculty (and non-faculty!) are treated, paid, supported, unsupported, and fit into the law school community. In short, we can, if we work together, make long overdue progress. Here are some things we can do to move the needle forward:

1. Amplify each other. Tell another AASE member’s Deans or other significant administrators about our colleagues. Did you like a presentation, article, book, helpful tip from them? Great. Send an email. I know a lot of people do this, but ASP folks, let it rain!

2. Use the AASE website as a place where we:
   1. Share resources;
   2. Post jobs;
   3. Post scholarship;
   4. Post teaching resources;
   5. And (copying entirely from the Legal Writing folks) accumulate resources to further this advocacy.[3]

3. We will be doing the AASE Survey again this year. Fill. It. Out. Data is everything in this fight. If we have better participation, we can use the data for salary and status benchmarking in a much more credible way. And then we need to disseminate the data widely-so that even the fancy schmancy consultants who do salary benchmarking can find it without much effort.[4]

4. Consider advocating for a place at the rankings table. I’ve written about this before and I stand by it: if we are an asset to our school then they are going to want to put a ring on it.[5]

5. Finally, consider advocating for a place at all the tables that have not had a seat for us before. We know what we do and how well we do it, but we need to shout about that from the rooftops. We need to showcase our brand. It is not something that comes easily for us, but we really need to be a bit more braggy about our accomplishments-because they are abundant, and people ought to know.

One great quote from The Importance of Being Earnest is, “[e]ven before I met you I was far from indifferent to you.” Let’s make this the new narrative about Academic Support.

(Liz Stillman)

 

[1] That kind of word choked, fast moving dialogue was something I loved about Gilmore Girls as well. 

[2] Although there are many specialized groups in law school academia that seem even more narrowly focused.

[3] Comme ça (as the French would say): https://www.lwionline.org/resources/status-related-advocacy

[4] Yes, I’m still a little salty about not having any salary benchmark when it was reviewed last year at my school despite having the AASE survey results in the freaking building.

[5] Apologies to Beyoncé.

An Experiment in Using ChatGPT-4 to Draft ASP Practice Problems

These days, many (most?) ASPs use course-based support measures.  In the "contextualized model," the doctrinal instructor teaches the doctrinal course, while the ASP instructor teaches a skills/ ASP course with problems from the doctrinal course.  In the "embedded model," the ASP instructor teaches the doctrinal course but embeds ASP training into that course.  In both types, practice problems, model answers, and feedback are key elements of training students on the skills they must build.

A perennial challenge is sourcing those problems.  While some doctrinal instructors may write or edit such materials for the contextualized ASP course, time constraints often make this impossible.  And drafting problems in the embedded model is no easier.  As such, those of us in this field find ourselves writing practice problems and spending much time doing so.  

I then read a post on the Faculty Lounge by Rick Bales of Ohio Northern called "Using AI to Help Flip the Law School Classroom."  Building on Bridget Crawford's recent helpful series of posts, Rick described specific steps in using ChatGPT-4 to draft effective practice materials. 

Based on his recommendations, I decided to experiment.  We are moving one of our ASP courses to the embedded model I described above.  The new course in the program will be Criminal Procedure (Investigations), which I have not taught since the dawn of time (or at least back when Mapp v. Ohio was still a thing).  Suffice it to say that I have a lot of drafting to do.

But using ChatGPT-4 to accomplish this seems promising.  I entered a command (... is that what the kids call it these days?) similar to Rick's but using a Crim Pro fact pattern I have been dying to use.  (Yes, it is Chief Quimby conducting a search of Moe's Tavern for illegal substances in the "Flaming Moe" cocktail and later interrogating Moe in a Rhode Island v. Innis sort of way.)  I specified that I wanted ChatGPT to create an essay question with a model answer, five MBE-like multiple-choice questions, and a thorough explanation of the correct answers and analysis involved in the MCQs.

In thirty seconds, ChatGPT created materials that would take me hours to write.  The questions were strong, and the essay model answer and MCQ explanations were good but imperfect.  I will edit a few substantive points and change the materials to gender-neutral language, but the answers' use of CREAC/ IRAC was quite strong, the rule sections were good, and the analysis was solid.  

I am fairly optimistic about using ChatGPT-4 to create practice problems.  I plan to be cautious, especially because ChatGPT-4 does not have the most recent cases, but those details are tweakable in the days before releasing the assignment.

 

Happy drafting ... sort of.

 

Louis Schulze 

FIU Law

More things change, the more they stay the same

The NCBE released results of the NextGen pilot testing last week.  You can read their report here.

Amazing ASPers throughout the country already started the discussion within our Listserv and on conference calls.  I don't want to reiterate (and/or plagiarize) their wonderful insights.  If some of my arguments sound familiar to someone else's, I sincerely apologize.  I will also provide my shameless plug for joining the bar advocacy committee.  Our complaints are being heard by bar associations, so you are making a difference.

My initial thought turned into the title to this post.  NextGen was supposed to be revolutionary and assess what lawyers do on a daily basis.  The test purported to require less memorization and focus more on practical skills.  The vision was grand, but the result seems to be a lackluster assessment in a slightly different format.

First, and probably most important, the report does not provide transparent data for performance differences among groups of participants.  The report vaguely states that the test narrowed the performance gap with some of the new questions.  The report does not fully say how much narrowing or which groups saw improved scores (it gave a few small examples).  The report also doesn't indicate the baseline for determining "examinee competency".  If the NCBE still uses prior standardized tests (LSAT mainly) to create the baseline for what performance difference is acceptable, then even small narrowing makes the new test seem better.  Most of us argue LSAT/standardized test differences are not based on competency.  Without knowing their baseline of what gaps are expected, I don't trust their argument the new test doesn't continue patterns of discrimination.  Slightly less discrimination isn't persuasive.  I want to be completely wrong here, but the impact is too great to not demand more transparency. 

Second, they aren't following their goal.  Attorneys don't memorize mountains of rules, so proponents of NextGen said it would move away from memorization.  I will concede that a slight decrease happened (no Secured Transactions, yay!).  However, NextGen continually adds back more material to memorize.  Family Law is back on the exam.  The starred vs. non-starred debate rages where we all know students will need to deeply know/memorize material even though the outline says to have a general understanding.  General understanding is too vague to create a good plan, and any reasonable person would over-prepare for the bar exam.  Wills, Trusts, and Estates keeps inching closer to fully back on the exam.  The report says that providing the Federal Rules of Evidence didn't help students/they didn't use them.  Of course students who already studied or engaged in some form of studying (either in bar prep or a for-credit bar class) didn't use the rules.  They already committed them to memory, or possibly, the stakes weren't high enough for them to pull up the rules because missing the questions had no impact on individually passing the exam.  

My last statement is about technology.  I believe the report makes clear the NCBE cares more about efficiency than true assessment.  The report discussed technological efficiency and ability to administer on computer.  The report (and all pilot tests to my knowledge) ignore the cognitive requirements of a 100% online exam (with multiple page PTs).  Why didn't the NCBE give a large number of participants the test on paper and the rest on computer?  My guess, because the answer would be clear.  Hard copies are better for active reading, notes, etc.  Students would most likely perform better with paper exams.

I sincerely hope I am wrong about my assessment of NextGen.  I hope it is a completely different and fair assessment.  My fear is that we are getting a different format of a test that will continue to determine who can reach their dream of practicing law based on screen reading, memorization, fast toggling, standardized testing, and a slew of other irrelevant skills.

(Steven Foster)

Congrats AASE Award Winners!!

Congrats to these amazing ASPers who won AASE awards this year, including our very own Liz Stillman.

AASE IMPACT AWARD- ELIZABETH STILLMAN

Photo here:

https://www.suffolk.edu/academics/faculty/e/s/estillman

AASE Impact Award- this award is given to someone who dedicates their time (we know that as ASP/Bar Studies professionals, we wear a lot of hats, and it takes time, focus, dedication, and even sacrifice to support AASE) and energy to AASE and our amazing profession. We’ve always been an important part of every law school, and now is our time to shine as skills becomes embedded in every part of legal education. This award is not limited to members who have been here for a long time either. It’s for anyone and everyone making a real difference in ASP/Bar Studies.

This year’s AASE Impact Award goes to Elizabeth Stillman. Liz Stillman is an associate professor in Suffolk University Law School’s Academic Support Program. She also teaches in the Political Science and Legal Studies Department for the Suffolk University College of Arts and Sciences as well as in the accelerated and LL.M. programs at the Law School. She received her BA from Tufts University, and her JD from Northeastern University. Liz was previously an Assistant Corporation Counsel for the City of New York where she prosecuted juvenile delinquency cases in Queens and Manhattan Family Courts. She is a Suffolk University Ambassador for Inclusion and certified in English as a Method of Instruction (EMI). Liz is also a past president of the New England Consortium of Academic Support Professionals.

Liz receives this award in recognition of her decades of dedicated support of students and of the ASP community. She is not afraid to speak out in support of equity for ASP faculty and staff; she is generous with her time on national committees; and she is always there for anyone who needs support, guidance, or encouragement as they navigate the ASP field. Liz has a particular gift for working with neuro-diverse students and those who require accommodations. She embodies the humor, compassion, warmth, intelligence, dedication, and passion that make the academic support field a great place to work. Liz is also a frequent blogger on the Law School Academic Support Blog. Thank you for your continued service, energy, and steadfast commitment to this incredibly important pillar of legal education, Liz! You are always an inspiration!

 

AASE OUTSTANDING SCHOLAR AWARD – Nachman Gutowski

Photo here:

https://law.unlv.edu/faculty/nachman-gutowski

• AASE Outstanding Scholar Award- this award is for an outstanding scholar whose work is promoting our field. We talk a lot about equity and increasing our status in the legal academy, and scholarship is one of the ways we can accomplish this goal. Scholarship is the currency recognized across the academy, and it is something AASE is working hard to promote and encourage. We want to celebrate our community’s scholarly successes!

This year’s AASE Outstanding Scholar Award goes to Nachman Gutowski. Nachman Gutowski is an Assistant Professor-in-Residence and Director of Academic Success Program for the University of Nevada, Las Vegas, William S. Boyd School of Law. He received his JD from the University of Miami School of Law, and his MM from the Frost School of Music at the University of Miami.  Prior to joining the William S. Boyd School of Law, he served as Director of Accreditation and Associate Professor of Academic Success and Bar Preparation at St. Thomas University, Benjamin L. Crump College of Law. He previously spent 8 years in the legal education-focused, corporate, national bar review industry supporting law schools from Puerto Rico through the southeast United States.

Nachman’s scholarly efforts, spanning critical analyses of discriminatory practices in legal licensure to the ethical considerations of artificial intelligence in legal education, have been driven by a commitment to challenge existing paradigms and advocate for systemic changes that enhance inclusivity and adaptability within legal education. His writing shows his dedication to the principles and mission that AASE champions, and focus on the pressing challenges and opportunities in legal education today. Congratulations, Nachman!

A selection of his scholarly works includes:

• NextGen Licensure & Accreditation, 22 U.N.H. L. Rev. 2 (Forthcoming 2024): This article explores the evolving landscape of legal licensure and accreditation, emphasizing the need for adaptability in standards that reflect contemporary legal practices.
• Navigating the AI Revolution: Challenges and Opportunities in Legal Practice and Education, Virginia Lawyers Weekly & Michigan Lawyers Weekly (2024): Addresses the seismic shifts AI technologies are creating in legal practice and education, offering insights into effective adaptation strategies.
• STOP THE COUNT; The Historically Discriminatory Nature of the Bar Exam Requires Adjustments in How Bar Passage Rates are Reported, If at All, 21 SEATTLE J. SOC. JUST. 589 (2023): This article has been downloaded from the Seattle University repository nearly 900 times in the less than 1 year since its publication. It covers many compelling arguments for reevaluating the reporting schemes nationally for the bar exam results, particularly focused on acknowledging and responding as a way to mitigate historical inequalities.
• How Are Bar Exam Results Reported? Research Summary; Raising the Bar (2023) and A National Guide (2023): Two pieces offering comprehensive insights into the reporting of bar exam results, highlighting variability and its implications for equity. The long-labored process of identifying, labeling, and directing where, how, and the impact of how each jurisdiction reports bar results is organized in a simple and easy-to-use format, made available for free to everyone.
• AI in Legal Education: Drafting Policies for Balancing Innovation and Integrity (2023): Provides guidelines for integrating AI into legal education in ways that promote ethical use and innovation.

 

AASE OUTSTANDING DEBUT in ASP/BAR STUDIES AWARD – MARIA FLORENCIA CORNU LAPORT

 

Photo here:

https://www.stu.edu/law/faculty-staff/faculty/mariacornulaport/

• AASE Outstanding Debut in ASP/Bar Studies Award- this award is for our newest members and colleagues in the academic success world. Those who are just getting their feet wet and doing amazing things with their students, faculty, and law school. We want to celebrate them and welcome them into AASE and our inclusive and supportive community.

This year’s AASE Outstanding Debut in ASP/Bar Studies Award goes to Maria Florencia Cornu Laport. M. Florencia Cornu Laport is the Director and Associate Professor of Academic Success and Bar Preparation at the St. Thomas University College of Law in Miami Gardens, FL. She received her JD first from the University of the Republic, Uruguay, and a U.S. JD at the St. Thomas University School of Law where she also received her LLM. Prior to joining the ASP world, Florencia worked as an attorney in Uruguay, also serving as a law clerk to the late Justice Hipolito Rodriguez Caorsi at the Supreme Court of Justice.

At St. Thomas University, she found the ideal environment to develop her biggest passions: law education as a mechanism of personal development for the students, an opportunity to provide inspiring and transformative experiences for the students; and the legal profession conceived as a tool to protect the vulnerable, to serve the underserved, and to work for social justice.

Florencia has stormed onto the ASP scene and was elected to be a part of the Executive Committee of the Academic Support Section of the American Association of Law Schools (AALS), where she will be joining professors and administrators from across the country aligned in the interest of improving academic support and bar preparation offerings for future lawyers. She has been incredibly active in AASE and her energy and enthusiasm for the field is contagious. We look forward to all of her future contributions!

Academic and Bar Support Scholarship Spotlight

Nachman N. Gutowski, NextGen Licensure & Accreditation, 22 U.N.H. L. Rev. 311 (2024).

From the abstract:

The Bar Exam is changing. The National Conference of Bar Examiners is pushing full steam ahead with a replacement for the current elements that make up the Uniform Bar Exam (UBE). This new exam, called the NextGen Bar Exam (NextGen), is scheduled to launch in Summer 2026. Current American Bar Association (ABA) accreditation standards do not consider the coming changes. A full picture of what the adjustments will look like is hazy and very much in the trial stages still. These shifts impact current law students, the legal education practices of law schools, and accreditation standards. There is a near-universal agreement that changes are overdue to the current legal licensure format. Simultaneously, alternatives to the NextGen, and even to the “need” for any summative licensure exam, are being actively explored.

Performance on the Bar Exam is used as a measurement tool by the American Bar Association for law schools to maintain accreditation. Standard 316, commonly referred to as Ultimate Bar Passage, has undergone several changes over its short life; yet, even in its current iteration, it fails to meaningfully consider what is just around the corner. There is no question that the Bar Exam continues to have racially discriminatory, disparate outcomes and impacts. Making matters worse, the use of aggregate limited durational performance data on post-graduation individual licensure exams as a meaningful metric by which accreditation is affected is inconsistent with accepted practices in similarly situated professions. Rectifying some baseline injustices can start with acknowledging how changes starting in 2026 are unaccounted for in the current standard. Adjusting or removing current prelicensure requirements and standards, either in ABA accreditation requirements for law schools or in educational prerequisites on examinees placed before the exam itself, would go a long way to align stated accreditation goals with licensure outcomes.

[Posted by Louis Schulze, FIU Law]