Guest Post: Can ChatGPT Prepare Me for Oral Argument?

This is a guest post by Jayne Woods, Associate Teaching Professor of Law at University of Missouri School of Law. All faults in fixing bullets points are Professor Dysart's.

As Spring approaches, many legal writing faculty are gearing up to teach oral advocacy to the 1L students.  While there is much discussion and debate about ChatGPT’s role in legal writing, I began to wonder if ChatGPT has a part to play in oral advocacy as well.

We preach to our students that preparation is the key to success in moot court and oral advocacy generally, and we give them strategies like outlining, summarizing relevant case law, and practicing responses to anticipated questions.  So I decided to let ChatGPT flex its muscles as a preparation assistant.

First, I asked it to create an outline based on the general legal framework of my students’ appellate brief problem, which involves both a facial and an as-applied First Amendment challenge to a municipality’s advertising guideline for its public transit system that bars religious content.  The outline looked very much like the examples we give our students:

I. Introduction

• • Introduce the case and state the issue on appeal
  • Introduce the appellant and their argument
  • Provide a roadmap of the oral argument

II. Standard of Review

• • Explain the standard of review for the district court's decision to deny the preliminary injunction
  • Argue that the district court applied the wrong standard of review and that the appellate court should apply de novo review

III. Facial Challenge

• • Argue that the city's advertising guideline prohibiting religious content is unconstitutional on its face
  • Explain that the guideline is content-based and subject to strict scrutiny
  • Argue that the guideline fails strict scrutiny because it is not narrowly tailored to serve a compelling government interest

IV. As-Applied Challenge

• • Argue that the city's application of the guideline to the church's religious advertisement is unconstitutional
  • Explain that the application of the guideline amounts to viewpoint discrimination
  • Argue that the city has not provided a compelling government interest for the discrimination
  • Provide examples of similar advertisements that were accepted by the city, but were not religious

V. Precedent

• • Discuss relevant precedent that supports the church's argument
  • Distinguish precedent that the city may rely on to support its guideline

VI. Conclusion

• • Summarize the argument and emphasize the importance of protecting the First Amendment rights of the church

This seems pretty good, though it does need some tweaking, as our problem does not involve strict scrutiny.  But this strikes me as a great starting point for our students who will be creating their first-ever outline for an appellate oral argument.  In a similar vein, it could help a new attorney making their first argument before an appellate court.

ChatGPT is also adept at creating concise, brief summaries of case law, and I asked it to create summaries of five of the most apposite cases involved, including the court’s reasoning for each.  Here are two examples:

Rosenberger v. Rector and Visitors of the University of Virginia (1995): The court held that the University of Virginia's refusal to fund a Christian student publication was unconstitutional viewpoint discrimination. The court reasoned that the university's policy of denying funding to religious organizations while funding secular organizations violated the First Amendment's Free Speech Clause by discriminating against religious viewpoints.

Archdiocese of Washington v. Washington Metropolitan Area Transit Authority (2017): The court held that the Washington Metropolitan Area Transit Authority's policy of rejecting an advertisement from the Archdiocese of Washington because of its religious content was unconstitutional. The court reasoned that the policy was viewpoint discrimination, as the transit authority allowed advertisements from secular organizations while excluding advertisements with religious messages.

These summaries are wonderfully concise snippets that could help an advocate quickly and easily recall a case for discussion purposes.  That said, the summary for the Archdiocese case is inaccurate; in fact, the Archdiocese lost that case because it failed to establish viewpoint discrimination. I repeatedly asked ChatGPT if it was sure about the accuracy of that summary and, though, it repeatedly admitted error, it continued insisting that the Archdiocese prevailed.  The inaccuracy was likely a result of how ChatGPT works.  It is based, in part, on the statistical structure of language, meaning its responses involve the most likely words in the most likely sequence in response to the prompt.  See https://www.assemblyai.com/blog/how-chatgpt-actually-works/.  When considering that the Archdiocese case is an outlier, ChatGPT’s response makes more sense; in most cases like Archdiocese, the party in the Archdiocese’s position prevails because it is able to establish viewpoint discrimination.  Even though I knew ChatGPT’s answer was incorrect, it was so certain that it caused me to both question my own recollection of the case and look it up again to verify.  But verifying the accuracy of the cases relied on is also good practice for an advocate.

I next asked it to generate some questions that a court might ask during an appellate argument.  This was where it got really cool.  It generated twenty-eight distinct questions and likely would have generated more had the system not become overloaded at the time.  Here are some examples:

1. How does the Supreme Court's decision in cases such as Board of Regents v. Southworth and Rosenberger v. Rector and Visitors of the University of Virginia affect the outcome of this case?
2. Are there any specific regulations or guidelines that the bus company must follow when deciding whether to accept or reject an ad?
3. Has the bus company previously accepted similar ads from other religious or non-religious groups, and if so, how does it justify the differential treatment in this case?
4. Did the bus company have a reasonable basis for concluding that the religious ad would be likely to cause disruption or harm, and if so, what evidence supports this conclusion?
5. If the plaintiff prevails, how might this decision impact other cases involving government-owned or government-operated advertising platforms, such as airports or public transit systems?

After having it pose questions, I tried to simulate an actual oral argument.  Specifically, I asked if we could engage in questions and answers where ChatGPT posed as a court and I posed as the attorney answering questions during an oral argument.  It enthusiastically agreed and then proceeded to simply create a transcript of an oral argument involving both questions from a court and answers from an attorney.  It turns out that, while ChatGPT is very adept at answering questions, it’s not terribly inquisitive.  I did manage to get a very brief back and forth after instructing it to ask me a very specific question, which concluded with ChatGPT thanking me for my argument:

Prompt:  ask me a question about my position that a city's advertising guideline barring religious content amounts to viewpoint discrimination

ChatGPT:  Certainly, counsel. How does your argument that the city's advertising guideline barring religious content amounts to viewpoint discrimination relate to the Supreme Court's holding in Rosenberger v. Rector and Visitors of the University of Virginia?

My response:  In Rosenberger, the Supreme Court indicated that, if the government opens a topic for speech, it may not then prohibit speech on that topic solely because of the viewpoint of the speaker. Here, the city allowed an advertisement promoting a children's Easter egg hunt but precluded my client's ad promoting an Easter worship service. Obviously, the city opened up the topic of Easter; thus, its refusal to accept my client's ad was solely because of my client's nature as a house of worship.

ChatGPT:  Thank you for your response, counsel. Can you further explain how your argument aligns with the Supreme Court's holding in Good News Club v. Milford Central School and Lamb's Chapel v. Center Moriches Union Free School District?

My response:  Yes, your honor. In both Good News Club and Lamb's Chapel, this Court encountered similar scenarios, where the government opened up public buildings for certain purposes but then barred religious entities from using the buildings because those entities were seeking to promote Christian viewpoints on the same topics the buildings were opened up to secular groups.

ChatGPT:  Thank you for your response, counsel. Based on your argument and the relevant Supreme Court precedents, it appears that you are contending that the city's advertising guideline barring religious content amounts to viewpoint discrimination, which is unconstitutional under the Free Speech Clause of the First Amendment. Thank you for your argument, counsel.

              This was great, albeit brief, practice.  I tried getting a longer discussion using some of the questions it had previously generated, but each time, ChatGPT asked a single question, I responded, and it simply agreed with me.  While this was great for boosting my confidence, it was not great at preparing me for hard questions.  So I went back and reviewed the transcript it created of a mock oral argument.  This was great!  It provided not only probing questions but also insightful answers:

Court: Counsel for the appellant, can you explain why you believe the city's guideline prohibiting ads with religious content violates the First Amendment?

Appellant: Yes, your honor. The city's guideline banning religious content in advertisements is not content-neutral because it discriminates against religious speech. Under the Supreme Court's holdings in Rosenberger, Good News, and Lamb's Chapel, any policy that discriminates against religious viewpoints is unconstitutional under the Free Speech Clause of the First Amendment.

Court: Counsel, the city argues that the advertising guideline is neutral and generally applicable, and does not target religious speech or practices. Can you respond to that argument?

Appellant: Your honor, the city's advertising guideline is not neutral and generally applicable. The guideline specifically prohibits religious content in advertisements, while allowing ads with secular content. This is precisely the kind of content-based discrimination that the Supreme Court has held violates the First Amendment.

Court: Counsel, what evidence do you have to show that the city's guideline is not neutral and generally applicable?

Appellant: Your honor, the city's guideline itself is evidence that it is not neutral and generally applicable. The guideline singles out religious content for exclusion, while allowing ads with other types of content. Additionally, the city's decision to reject our client's advertisement for a religious service shows that the guideline is not applied neutrally. If the guideline were truly neutral and generally applicable, our client's advertisement would not have been rejected simply because it contained religious content.

Court: Counsel, the city argues that the guideline is necessary to avoid entanglement between government and religion. How do you respond to that argument?

Appellant: Your honor, the city's argument that the guideline is necessary to avoid entanglement between government and religion is not persuasive. The Supreme Court has repeatedly held that the government may not discriminate against religious speech or practices simply to avoid entanglement with religion. In fact, such discrimination itself constitutes impermissible government entanglement with religion. Therefore, the city's guideline is unconstitutional under the Free Speech Clause of the First Amendment.

I envision a student or advocate using this as a self-evaluation.  The student or advocate could hide the potential answers, provide their own responses, and then check their answers against the samples.  My overall takeaway:  as with most things related to ChatGPT, it provides a great starting point but still needs human eyes to fact-check and finesse.  But I think ChatGPT has great potential to assist advocates in preparing for oral argument.