Ryznar: "Exams in the Time of ChatGPT"

Margaret Ryznar recently posted to SSRN her article Exams in the Time of ChatGPT, Washington and Lee Law Review Online (forthcoming).  Here is the abstract:

This article offers various methods to administer assessments while maintaining their integrity—after asking artificial intelligence writing tool ChatGPT for its views on the matter.  The sophisticated response of the chatbot, which students can use in their written work, only raises the stakes of figuring out how to administer exams fairly. 

 

From the Introduction:

I was ironically in the middle of grading a law school exam when my phone rang.  I ignored it, squinting at the exam in front of me, when my phone rang again.  This time, I lost my focus and reached for my phone.  

It was a text from my former college roommate: “Not sure if you’ve been following any of the buzz about ChatGPT, but this is pretty astonishing.”  She linked to an article claiming that something called the GPT-4 could pass the bar exam.

I’m not sure that’s possible, I thought before texting back and returning to grading.   

Just to be sure, the next day, I logged into ChatGPT.  I typed a question about the dead hand doctrine in Trusts & Estates and waited.  I had read hundreds of answers to some iteration of this question during the last decade of teaching at various different law schools.  Upon reading the answer from ChatGPT, I immediately knew that ChatGPT would be a force with which to reckon in legal education and practice.  And, ChatGPT would get even smarter with time—the more ChatGPT interacts with users and processes new information, the more intelligent it becomes. 

Since its introduction in late 2022, ChatGPT has created some concern in higher education, including in terms of assessing students.  Thus far, Chat GPT passed the bar exam that is used to license lawyers and the United States Medical Licensing Exam (USMLE) that is used to license doctors.  ChatGPT would have earned approximately a C+ at the University of Minnesota Law School and a B/B- in an MBA program at the Wharton School of the University of Pennsylvania, both highly regarded graduate schools.  As one English teacher said, “Let me be candid (with apologies to all of my current and former students): What GPT can produce right now is better than the large majority of writing seen by your average teacher or professor.” 

Read the rest of the article here.