Lemann on the Duty to Warn and Automation

Alex Lemann has posted to SSRN The Duty to Warn in the Age of Automation.  The abstract provides:

Autonomous vehicles are expected to drive far more safely than humans do, and yet they create novel risks of their own. This Article explores how the risks of autonomous vehicles should be communicated to those who buy and use them.

In low-level automation systems, where drivers are required to monitor their cars’ driving, instructions provide the information drivers need to understand their role and perform it safely. The fatal crashes involving low-level automation that have already occurred show that instructions alone are not sufficient, and that cars must be designed to account for humans’ tendency to lose focus when engaged in “passive vigilance.”

High-level automation, in which the only driver is an algorithm and there is nothing for the human passenger to do, presents a thornier problem. What form disclosure of the risks of highly autonomous vehicles should take depends on what we expect disclosure to accomplish.

One model of disclosure is utilitarian. Here the goal is to nudge people in the direction of better choices, and disclosures are tailored to encourage optimal behavior. Implicitly adopting this framework, scholars who have addressed this issue argue that the risks of autonomous driving should be presented numerically: disclose to consumers a price, like a risk-rated insurance premium, as an indication of how well the autonomous vehicle they are about to buy performs on the roadways.

Another model of disclosure, however, is deontological. Built on ideas like consent and autonomy, this view of disclosure aims to provide people with salient notice of the risks they might choose to encounter, so that they are not subjected to risk involuntarily. This account has deep roots in our legal culture and, I argue, counsels against disclosing the risks of autonomous vehicles solely in the form of a price. Instead or perhaps in addition, customers should be given a qualitative sense of the hazards they face in driverless cars.