California Wants Chatbots to State They’re Not a Person

A new bill introduced in California could force AI firms to continually remind young users that their chatbot friend is indeed not a real person.

The Big Picture: AI companies, including Character AI, ElevenLabs, and Replika, have faced major backlash from users and their families after they had to deal with deleted lovers, manipulation toward violence, and, in the most tragic case, teen suicide. California, which is typically ahead of the tech-regulation curve, hopes that forcing chatbots to state their artificiality can save lives.

Between the Lines: California wants to put it in writing that humans and chatbots aren’t the same thing.

• SB 243,  introduced by California State Senator Steve Padilla, would force AI companies to make their chatbots “periodically remind kids that a chatbot is AI and not human,” per The Verge.
• It would also “limit” the companies from using “addictive engagement patterns” that keep users hooked to their screens talking with the chatbots.
• The companies would also have to send an annual report to the California Department of Health Care Services, explicitly outlining how often the platforms detected suicidal ideation among minors using their platforms.
• Additionally, the companies would have to tell users that their chatbots aren’t available for some kids — think of a rating system for engaging with AI.

The Future: Padilla notes that he’s trying to protect kids from the “addictive, isolating, and influential aspects” of AI — sounds familiar to similar arguments made about limiting social media access, which both the federal and state governments were woefully behind in doing. Considering the impact that platform algorithms had on young users — which are, well, child’s play in comparison to AI — lawmakers may be vigilant in ensuring they aren’t caught so off-guard by the impacts of unregulated chatbots.