Lawyer Fined $5,000 Over ChatGPT-Hallucinated Testimony

The New Mexico Supreme Court has fined a lawyer $5,000 for submitting false testimony created by generative AI in an appeal of a murder conviction. It also found him in contempt of court, making clear his responsibility for failing to verify facts and legal authorities. As reported by Emma Roth of The Verge, citing Reuters reporting, the penalty was conveyed. The document at issue was an AI-generated brief that included testimony from non-existent witnesses. It is a case in which misinformation from generative AI entered judicial proceedings in criminal appeals, an area involving human life and liberty.

The lawyer disciplined was Stephen Aarons. He appealed his client’s murder conviction. In the filing submitted at that time, he included testimony by fictitious witnesses and police testimony. A court document issued Wednesday found that he had failed to verify factual claims and legal authorities in the AI-generated brief. It was treated seriously as conduct undermining the authenticity of evidence, the foundation of criminal justice.

Fabricated Testimony and How It Was Discovered

The brief at issue contained false testimony by entirely fabricated witnesses. The court document explicitly pointed this out.

The brief contained false testimony by entirely fabricated witnesses

It was not only the very existence of the witnesses at issue. It also included false descriptions of the shooter’s clothing and appearance. In an appeal, consistency with the trial record is strictly checked. Testimony not in the record comes to light during cross-checking. In criminal cases, witness examination records and police records are core materials. The fictitious statements were discovered because of contradictions with those records.

As reported by Emma Roth of The Verge, citing the court documents obtained by Reuters, these facts were presented. Summaries and drafting by generative AI may look well-formed at first glance. However, they can lack correspondence with sources and the record. In this case, that gap surfaced in a criminal appeal. The fact that misinformation was submitted in proceedings directly tied to sentencing and guilt or innocence was viewed as grave.

Tough Questioning in Court and the Lawyer’s

Explanation

At a hearing held in August, Aarons admitted that he had used ChatGPT. He explained to the effect that he used it believing he could obtain a bulletproof summary of the court record. He was pressed on using it in practice without understanding the dangers of generative AI. Judge C. Shannon Bacon pressed him with questions while referring to daily news coverage.

Counsel, do you watch the news. Do you listen to the radio. Do you read what’s going on in the world. The problem of lawyers relying on AI hallucinations is widely reported day after day

These remarks were conveyed by Reuters as part of the hearing. The judge’s comments showed the understanding that AI misinformation is not a rare exception. It was a warning about the repeated misuse of generative AI in legal practice. Aarons acknowledged at the hearing that his awareness of the risks was lax. In a statement to Reuters, he said to the effect that it had been an honest mistake.

I am remorseful, but I ask that it be considered in disciplinary proceedings that it was an honest mistake

In criminal defense, protecting the client’s rights is a lawyer’s duty. At the same time, a lawyer also owes a duty of candor to the court. Submitting false materials was deemed a violation of both duties. The fine and the contempt finding reflected that assessment.

A String of AI Fabrication Problems in

Legal Filings

This is not the first AI fabrication problem in legal documents. Last year, filings submitted by two law firms were called into question. A judge harshly condemned them as containing numerous false, inaccurate, and misleading legal citations and quotations. A lawyer for MyPillow’s Mike Lindell was also fined over a similar problem. The reason was including AI-generated erroneous citations and fictitious case citations in a brief.

As reported by Emma Roth of The Verge, these past cases were also noted as related articles. As more lawyers use generative AI in practice, the number of discovered cases is increasing. Because the citation format is well-formed, they look at first glance like proper citations. However, docket numbers and case names may not exist. The wording of statutes may also be subtly altered. They are typically discovered when courts or opposing parties check the original sources.

In the United States, federal district courts and state courts are moving to establish disclosure rules for AI use. When generative AI is used, they require disclosure of the scope of use and verification methods. Not all courts have adopted the same standards. Even so, the principle that the filing lawyer bears responsibility for verification is common. This penalty reaffirmed that principle.

The Structure of Generative AI Use Without

Verification

Generative AI generates expressions most likely to come next from large volumes of language data. It is not a system that searches for and returns factual records. Therefore, even fluent and persuasive-looking text may have no basis. This phenomenon is called hallucination. Even without malice, non-existent names or case numbers can be generated. In fields with standardized formats such as legal documents, discovery is especially delayed.

The lawyer in this case left the accuracy of the summary to AI. He skipped cross-checking against the trial record and verifying primary sources. He apparently did not recognize the need for verification because the output looked well-formed. Pressure to improve work efficiency may also have been a factor. Deadline management in appeals is strict, and the volume of records is enormous. The motive to reduce the burden of summarization is understandable. However, in a criminal appeal, a single piece of testimony can determine the outcome. His use was assessed as lacking a balance between efficiency and accuracy.

As shown by HalluSquatting Attack Exploiting AI Hallucinations Emerges, fictitious names and information produced by hallucinations are also exploited for attacks. The common point is treating non-existent resources as if they existed. The danger of incorporating the technology into practice without understanding its characteristics extends beyond legal work. Similar challenges are seen in fields requiring record accuracy, such as medicine and finance. Fluency of output and factual accuracy are separate measures. This distinction needs to be built into operations.

Measures and Operations Required in Legal Practice

In legal practice, the challenge is not banning generative AI but incorporating verification steps. A design that separates drafting assistance from fact-finding is seen as effective. AI should handle issue organization and draft creation. Testimony and case citations must always be checked against the trial record and case reporters. A system for review by two or more people is also important. Operations are required in which the filing supervisor leaves a record of having checked the original sources.

On the technical side, citation-grounding features and source-linking can be used. There is also a method using retrieval-augmented generation to reference only record documents. Even so, it does not guarantee final accuracy. All names and dates in outputs must be doubted and verified. In-house education and training are also seen as essential. AI mechanisms and limitations, and verification procedures, need to be standardized. Disclosure of the scope of AI use as accountability to clients is also seen as spreading.

In criminal cases, the impact of misinformation is especially large. If the basis for a claim of innocence is fictitious, it harms the client’s interests. It wastes court time and undermines trust in the judiciary. The $5,000 fine in this case carries symbolic meaning beyond the amount. The contempt finding signaled a violation of professional ethics. It is seen as having the effect of reminding lawyers in other jurisdictions of their verification responsibility.

Editorial Opinion

In the short term, development of generative AI usage guidelines at law firms is seen as accelerating. Operations requiring source verification when drafting documents may spread. Corporate legal departments that include verification steps as engagement conditions are also seen as increasing. In the long term, change is seen as extending to both legal education and technology design. AI verification training at law schools may become mandatory. On the generative AI side, citation-grounding features are seen as becoming standardized. The question remaining from the editorial team is where responsibility lies. Whether it is user negligence or provider design deficiency remains unresolved. Discussion is needed on what should be permitted in areas where misinformation involves human life.

References

• “Lawyer fined $5K over AI-hallucinated witnesses in a murder case”, by Emma Roth — The Verge, 2026-09-11T20:44:02.000Z (ARR)
• Source URL: https://www.theverge.com/ai-artificial-intelligence/994207/chatgpt-new-mexico-lawyer-fined-murder-appeal