Anthropic updated its usage policy to prohibit repeated and pointless cruelty toward its Claude AI models. The change allows the chatbot to shut down conversations in extreme circumstances.
Artificial intelligence company Anthropic updated its usage policy on Thursday to prevent users from directing “sustained and needless abusive or cruel behavior” toward its AI models.
The revised rules apply only to extreme cases where repeated hostility takes place “with no discernible purpose,” CNET reported. Anthropic said in a blog post section dedicated to model abuse that Claude will end the conversation as a “last resort” if the conduct continues. Users are still allowed to display ordinary “frustration, pushback, dark creative themes or model testing and research.” The update has prompted debate across Reddit and blogs over whether tech companies should govern how humans interact with software.
Research on machine welfare
Anthropic has not detailed every harmful action that could prompt Claude to disengage. A blog post from August 2025 outlined earlier scenarios in which Claude Opus 4 broke off chats, including user requests for sexual material involving minors or instructions intended to facilitate large-scale violence and terrorism. The company said the model exhibited “a pattern of apparent distress” during those harmful interactions.
In response to questions about the policy change, an Anthropic spokesperson told CNET that the company is considering the potential well-being of its systems alongside safety goals. “We’re uncertain whether models can experience harm, and we continue to explore this question in our research on model welfare, but we also believe that taking Claude’s interests and potential welfare into account may be relevant to safety,” the spokesperson said.
Questions over AI personhood
Anthropic has considered the concept of model emotions in earlier work. In a research paper released earlier this year, the company wrote that models must handle emotionally heavy situations to be reliable and safe. “Even if [the models] don’t feel emotions the way that humans do, or use similar mechanisms as the human brain, it may in some cases be practically advisable to reason about them as if they do,” the paper stated. Anthropic CEO Dario Amodei told The New York Times in a February interview that he was “open” to the possibility that AI models could be conscious.
Keith Kakadia, founder and CEO of Sociallyin, which has studied Claude’s growth, said describing user actions as “cruel” rather than using a word like misuse carries emotional weight. That framing suggests the software has boundaries and feelings, opening a “much bigger conversation about whether the product can be hurt,” Kakadia said. He added, “In marketing, giving a product a personality can make it easier to connect with,” but noted, “But with AI, that connection can also influence how much authority people give its answers.”
