Skip to content
Tech News
← Back to articles

ChatGPT-Using Lawyer Cited Its Fake Witnesses and Police Testimony in Court

read original more articles
Why This Matters

This case highlights the risks of relying on AI tools like ChatGPT without proper verification, especially in critical legal contexts. It underscores the importance for professionals to understand AI limitations to prevent misinformation and potential legal consequences. The incident serves as a cautionary tale for both the legal industry and AI users about the need for human oversight.

Key Takeaways

Reuters reports:

A defense lawyer appealing his client's murder conviction submitted a brief containing made-up police testimony and witnesses fabricated by OpenAI's ChatGPT, New Mexico's highest court said. The New Mexico Supreme Court on Wednesday fined the attorney, Stephen Aarons [for $5,000], and held him in contempt for failing to verify the accuracy of the court filing, which Aarons said he prepared with help from the AI program... and said they will refer him to an attorney disciplinary board for investigation.

Aarons in a statement to Reuters said he had used ChatGPT to summarize the trial proceedings when he agreed to take up the defendant's appeal last year, and did not understand the degree to which AI could "hallucinate" facts. "I am remorseful but hopeful that the disciplinary board takes into account it was an honest mistake," he said.... Dozens of lawyers have been sanctioned for filing briefs where AI made up case citations or misquoted the law. Aarons' filing appears to have gone further, containing fabricated witness testimony in a criminal appeal...

Aarons told the court at an August 21 hearing that he fed a computer-generated transcript and other case materials to ChatGPT, presuming it would generate "a bulletproof summary." The justices sounded incredulous that Aarons was not fully aware of how AI can make mistakes. "Counsel, do you watch the news? Do you listen to the radio? Do you read anything about what's going on in the world?" Justice C. Shannon Bacon said at the hearing. "Because the problem with lawyers relying on AI hallucinations is an above-the-fold story every single day."

"OpenAI did not immediately respond to a request for comment."

Read more of this story at Slashdot.