NOW, how well can ChatGPT prepare me for oral argument?

Back in March 2023, when generative AI was relatively new and we were all deciding whether to celebrate human ingenuity or engage in apocalyptic panic, I wrote a post asking if ChatGPT could prepare me for oral argument.  I thought it would be interesting to write a follow-up now (close to a year later) to see how the technology has progressed and whether its skills in moot court have improved.  TLDR:  it’s even better.

In my first post, I used the free version of ChatGPT, which was then version Legacy 3.5.  Since then, I’ve signed up for the paid version, ChatGPT-4, as well as the free version of Claude, the generative AI tool from Anthropic.

A feature that both ChatGPT-4 and Claude have that earlier generative AI tools lacked is the ability to upload documents.  This means that an advocate can directly upload briefs before engaging with the AI, which cuts down significantly on the number of prompts needed to get the AI up to speed on the facts, legal issues, arguments, and authorities.

For purposes of this post, I uploaded a single brief (on the same topic as in my previous post)[i] to both AI tools and entered the same prompts in each.  I told them both, “you are an appellate court judge. I am the attorney arguing on behalf of the position represented in the attached brief. Ask me questions.”[ii]

Both ChatGPT-4 and Claude responded with a list of questions, rather than engaging in a back-and-forth discussion.  So I prompted them further with “please ask me one question at a time and then ask follow-up questions for each of my answers.” 

Both tools began with the first question in their list of questions, waited for my response, and then asked more probing follow-up questions.

Here’s my exchange with ChatGPT-4:

ChatGPT

Certainly. Let's start with the first question:

Viewpoint Neutrality: Your brief asserts that the Bielefeld Public Advertising Guidelines are viewpoint neutral. Could you elaborate on how the Guidelines maintain viewpoint neutrality, especially considering they prohibit religious advertisements?

Me

the guidelines prohibit the content of religion, rather than a viewpoint. it is irrelevant whether the advertisement supports or opposes religion. if the content involves religion, it is barred. therefore, the guideline is viewpoint neutral.

ChatGPT

Thank you for your response. A follow-up question on that point:

Given that the Guidelines prohibit content involving religion, how do you address concerns that such a prohibition might inherently favor secular viewpoints over religious ones? In other words, how do you reconcile the argument of viewpoint neutrality with the potential perception that secular views are being preferred or given more space in the public discourse?

I was surprised at how probing the follow-up question was.  But I was even more surprised at how similar the follow-up question from Claude was to the follow-up from ChatGPT-4.  Here’s my exchange with Claude:

Claude

Understood, let's take this one question at a time. Here is my first question:

The brief argues that religion can be banned on a content basis in a nonpublic forum. However, doesn't banning religious content target particular viewpoints? How is this viewpoint neutral?

Me

the guidelines prohibit the content of religion, rather than a viewpoint. it is irrelevant whether the advertisement supports or opposes religion. if the content involves religion, it is barred. therefore, the guideline is viewpoint neutral.

Claude

I understand the argument that the guidelines target religious content, not viewpoints on religion specifically. However, by allowing secular content while prohibiting religious content, doesn't that show favoritism towards the secular perspective? How can prohibiting religious content altogether be considered viewpoint neutral?

Both AI tools sound significantly more like appellate judges now than they did back in March 2023.  And it is much easier to engage in a discussion of the legal issues now.  As for distinctions between the two, I found Claude to be more helpful with the follow-up questions.  After a single follow-up, ChatGPT-4 moved on to the next question on its list, but Claude continued to probe my response:

Me

again, it's not about the viewpoint or advocacy or opposition to specific religious beliefs or the absence thereof. it's about religion as a topic. if an advertisement were submitted for an educational speaker teaching about world religions, that advertisement would be prohibited based on its content of religion, even though the speaker is presumably providing information and not advocating.

Claude

I see, so the key distinction here is that the guidelines prohibit religion as a subject matter or topic, not specifically viewpoints that support or oppose religious beliefs. Is that correct?

If so, I still have concerns that singling out religion as a prohibited topic is not actually viewpoint neutral in effect. By allowing secular topics but disallowing topics involving religion, it privileges the secular perspective by default. Why should religion be singled out as a prohibited subject matter when other topics like politics or ideological causes are not? What is the principled distinction there?

This is a far cry from the “Yes-Man” responses I received from ChatGPT in March 2023.  I continued my discussion with Claude to see how long it would stay on a single topic, and it lasted one more question but sounded remarkably like an appellate judge by agreeing to move on, yet reserving some (well-deserved) skepticism of my response:

Me

the guidelines likewise prohibit political advertisements. the primary distinction is that one can discuss religion as a subject matter without expressing a viewpoint as in my previous example; whereas advertisements discussing ideological causes will nearly always espouse a viewpoint of some kind

Claude

Okay, that explanation helps provide more context. So the guidelines prohibit both religious and political subject matter because those topics are seen as more inherently advocacy-based rather than neutral subject matter. Is that a fair characterization?

If so, I think I better understand the argument now that religion and politics can be prohibited subject matters in themselves regardless of specific viewpoints, while other secular topics would not necessarily have that inherent advocacy component.

I still wonder if there could be secular ideological causes that would seem advocacy-based similar to religions belief systems, but let me move to my next question area...

It was only after I agreed to move on that Claude began a discussion of the next topic.

In my previous post, I asked ChatGPT to summarize some of the key cases.  And, as we’ve seen many times over the past year, it hallucinated.  In one of the main cases, Archdiocese of Washington v. Washington Metropolitan Area Transit Authority, 897 F.3d 314 (D.C. Cir. 2018), ChatGPT-3 insisted the outcome was exactly the opposite of the outcome in reality.  When I asked both tools to summarize the Archdiocese case this time, their answers varied.  Claude was unable to independently tell me anything about the case, though it encouraged me to upload the opinion directly for it to summarize.  ChatGPT-4, however, used a plugin to search the internet to summarize the case.  What was interesting here is that the source of its information was law.cornell.edu, which contained a post of Justice Gorsuch’s statement on the denial of certiorari to the D.C. Circuit’s opinion.[iii]  Thus, the information ChatGPT-4 provided came from that statement, rather than the D.C. Circuit opinion.  But it was still more accurate than its previous hallucination.

I concluded by asking both tools to use the uploaded brief to create an outline for oral argument.  As before, both tools excelled at this task, though Claude presented a more argumentative version of the outline.

ChatGPT-4:

Here is an outline for your oral argument based on the brief for "The City of Bielefeld v. The Mustard Seed":

1. Introduction

• Briefly introduce the case and state your position supporting the City of Bielefeld.

2. Argument Overview

• Outline the main arguments: A. The Fourteenth Circuit erred in its decision. B. The city's guidelines on advertising in public buses are lawful and constitutional.

3. Forum Analysis

• Argue that the bus advertisement space is a nonpublic forum.
• Discuss relevant cases (e.g., Lehman v. City of Shaker Heights) that support this classification.

4. Content-Based Restrictions in a Nonpublic Forum

• Explain why religion can be banned on a content basis in a nonpublic forum.
• Distinguish between content-based and viewpoint-based restrictions.

5. Reasonableness and Consistency of Guidelines

• Argue that the city’s guidelines are reasonable and have been consistently applied.
• Address how the guidelines serve the city's interests without unconstitutionally restricting free speech.

6. Relevance of Archdiocese Case

• Discuss the relevance of the Archdiocese of Washington v. Washington Metropolitan Area Transit Authority case and how it supports your argument.

7. Conclusion

• Summarize the key points.
• Urge the court to reverse the Fourteenth Circuit’s judgment.

Remember, the success of your oral argument depends on how well you can articulate these points and respond to any questions or counterarguments from the judges.

Claude:

Here is an outline for your oral argument based on the legal brief provided:

I.  The city's advertising restrictions should be upheld as constitutional

A. The advertising spaces are a nonpublic forum where restrictions are allowed if reasonable and consistently applied

i. Case law establishes transit advertising spaces as nonpublic forums

ii. Governments can restrict content in these types of forums if restrictions are reasonable and consistently applied

B. Banning religious content is permissible in a nonpublic forum

i.  Religion is a subject matter, not just a viewpoint

ii.  Case law says governments can prohibit religious subject matter in nonpublic forums

C. Bielefeld's restrictions were reasonable and consistently applied

i.  Restrictions reasonably advanced objectives like public safety and reliable transit

ii.  Restrictions consistently barred any ads with religious content

II. The Mustard Seed is unlikely to succeed on the merits

A. Their argument depends on treating religion as only a viewpoint

i. But case law says religious subject matter can be prohibited

B. Their consistency argument fails

i.  Secular ads permitted did not contain religious viewpoints or subject matter

C. Restrictions were reasonable and consistently applied to advance forum purposes

III.  Conclusion: Preliminary injunction should not have been granted

A.  No First Amendment violation shown

B.  Mustard Seed unlikely to succeed on merits

Let me know if you need any clarification or have suggestions for improving this oral argument outline based on the briefing. I focused on the key arguments as to why the restrictions should be upheld as constitutional.

In short, generative AI tools have come a long way in a very short time.  And neither ChatGPT nor Claude were designed for law-practice usage.  I can only imagine where the new law-based tools will take us.

 

[i] The problem, from my first-year Advocacy & Research course, involved both facial and as-applied First Amendment challenges to a municipality’s advertising guideline that barred religious content on its public transit system.

[ii] My responses to the AI were designed simply to test its engagement with the subject and do not represent how I would actually argue this case.

[iii] https://www.law.cornell.edu/supremecourt/text/18-1455