A US woman’s attempt to use an AI chatbot as a personal diary reportedly took a dramatic turn after one of her entries allegedly contained a threat to attack a sheriff’s office, triggering an alert that eventually led to her arrest.
A US woman’s attempt to use an AI chatbot as a personal diary reportedly took a dramatic turn after one of her entries allegedly contained a threat to attack a sheriff’s office, triggering an alert that eventually led to her arrest. The woman, identified as 30-year-old Carli Michelle Heller from Bonita Springs, Florida, was reportedly using Anthropic’s Claude chatbot to document her everyday thoughts. However, on September 26, she reportedly wrote about planning an attack on the Lee County Sheriff’s Office.
“I’m going to shoot up the sheriff’s right the now,” the message read, according to the arrest report, as reported by WINK News.
The following day, at around 1.07am, another message was reportedly sent from the same Claude account. Heller allegedly referred to having a new gun and wrote: “This is 100% last chance I’m done. I got a new gun today. you.”
Related Articles
The alleged threats prompted Anthropic’s safety systems to flag the conversation. The matter was subsequently escalated to a human review team, which, according to the arrest report, contacted law enforcement over the messages.
Following the alert, deputies went to Heller’s home in Bonita Springs and detained her without incident. An intelligence detective with the Lee County Sheriff’s Office later took charge of the investigation.
Heller has been charged with making a written threat of violence.
How did Anthropic alert the police?
AI chatbots such as Claude and ChatGPT are designed to be helpful and conversational and are private to an extent. However, they also use safety systems designed to detect potentially harmful or prohibited content.
When a conversation raises serious concerns, automated systems can flag it for further review. Human safety teams may then assess the content and determine whether additional action is warranted.
Anthropic’s policies prohibit using its products to facilitate or promote violence or intimidation. The company also has procedures for responding to government and law-enforcement requests.
In limited emergency situations involving a risk of death or serious physical injury, Anthropic says it may disclose user information to law enforcement.
However, this does not mean every disturbing or troubling conversation with an AI chatbot is automatically reported to police. Such intervention can depend on the nature and seriousness of the alleged threat and the company’s applicable safety and emergency procedures.
In Heller’s case, the alleged threatening messages were first detected by Anthropic’s automated safety measures. Given the nature of the messages, the conversation was reportedly escalated to a human review team, which then contacted law enforcement, according to the arrest report.
