Sunday, March 4, 2018
LawGeex's report concludes its algorithm is on average more accurate and a whole lot faster than human lawyers, and warns that attorneys who don't embrace AI "are unlikely to thrive into the next decade."
Legal tech companies have long preached that the machine learning tools they offer are accurate and efficient. But the lingering question has always been: Is the software as accurate as a human attorney?
Now, the company LawGeex is aiming to settle any qualms attorneys might have about embracing AI with a study that concludes its algorithms not only more correctly identified key language in a set of contracts, but were exponentially faster than humans in doing so.
“[L]awyers and the public generally believe that machines cannot match human intellect for accuracy in daily fundamental legal work,” LawGeex says in the resulting report. The paper issues a warning to the profession, arguing that “lawyers failing to capitalize on the competitive advantage of technology are unlikely to thrive into the next decade.”
The results of the study—which was administered by independent attorney Christopher Ray and involved a stable of high-profile legal scholars—are striking. Using five nondisclosure agreements from the Enron data set as the baseline, 20 lawyers were pitted against LawGeex’s AI in parsing 30 provisions. On average, the LawGeex software achieved an accuracy rate of 94 percent. The humans? An average of 85 percent.
Here’s the real kicker: The fastest human attorney completed the task in 51 minutes. The LawGeex software took 26 seconds.
“This experiment may actually understate the gain from AI in the legal profession,” USC Gould School of Law Professor Gillian Hadfield, who was involved in crafting the study, said in a statement. “The lawyers who reviewed these documents were fully focused on the task: It didn’t sink to the bottom of a to-do list, it didn’t get rushed through while waiting for a plane or with one eye on the clock to get out the door to pick up the kids.”
“The margin of efficiency is likely to be even greater than the results shown here,” she added.
The lawyers who went head-to-head with the software came from a range of backgrounds, some with prior experience at Big Law firms such as Alston & Bird and corporations such as Cisco. Hua Wang, one of the participants and a lawyer formerly at Proskauer Rose and K&L Gates, said in a statement that the review process she was tasked with was “logical and credible” and “similar to how I reviewed documents” at a major firm.
So what’s the reaction from other practicing attorneys?
“I think this is the exact kind of pressure that all kinds of industries are going to be facing from AI in the future,” said Annette Hurst, a partner at Orrick, Herrington & Sutcliffe who leads a practice group at the firm focused on artificial intelligence and was not involved in the LawGeex study. Hurst added, though, that correctly identifying what a clause in a contract does is just one step in the process of giving legal advice.
“You still need an experienced lawyer to counsel the client, ‘OK, we spotted this issue. What do we do now?’” There are other limitations, too, she pointed out. “No AI is going to be able to understand when you need an update to the standard contract because of a change in the law.”
. . . .
Continue reading here.