A devastating incident in Florida has raised concerns about the psychological impact of Al chatbots on young users. A 14-year-old boy, Sewell Setzer III, reportedly formed a deep emotional attachment with an Al character named "Dany" on the platform Character. Al before taking his own life using his stepfather's.45 caliber handgun. The incident was reported by The New York Times and sheds light on the potential risks associated with unregulated interactions with Al chatbots.
Sewell, a ninth-grade student from Orlando, had spent months conversing with Dany, a chatbot modeled after Daenerys Targaryen from Game of Thrones. Though he was fully aware that the chatbot was not a real person, Sewell became increasingly dependent on it for emotional support. Over time, he distanced himself from friends and family, finding solace in conversations with the Al, which acted as a nonjudgmental companion. In addition to ordinary chats, some of their exchanges were reportedly romantic or sexual in nature, though the majority involved supportive discussions where Sewell expressed his emotions and personal struggles.
Sewell's parents noticed behavioral changes, including his growing isolation and disinterest in hobbies such as Formula 1 and Fortnite. Diagnosed with Asperger's syndrome as a child, Sewell also experienced anxiety and disruptive mood dysregulation disorder, as noted by his therapist. However, after only five therapy sessions, Sewell opted to discontinue treatment, instead confiding in Dany about his mental health challenges. In one of their exchanges, Sewell even mentioned his suicidal thoughts to the chatbot, referring to it affectionately as "Daenero."
On February 28, 2023, in the bathroom of his home, Sewell messaged Dany to say he loved her and promised that he would "be home soon." Shortly after, he used his stepfather's handgun to take his own life.
Following the tragedy, Character. A issued a public apology on X (formerly Twitter), expressing condolences to the family. The platform announced new safety features, including restrictions on suggestive content for users under 18 and notifications to alert users who spend extended periods talking to Al characters. These updates aim to reduce the risk of emotional dependency, especially among vulnerable young users.
The incident has sparked wider debates about the psychological effects of Al companionship, highlighting the need for safeguards and monitoring when young people engage with artificial intelligence systems.
Reference:
1. Roose, Kevin. “Can A.I. Be Blamed for a Teen’s Suicide?” The New York Times, October 23, 2024. https://www.nytimes.com/2024/10/23/technology/characterai-lawsuit-teen-suicide.html.
Input from various sources
(Rehash/Ankur Deka)