California Governor Gavin Newsom signed a number of bills into law on Monday, the bulk of which are designed to create safeguards in the AI industry and address potential harms to children and young users.
One of the more prominent pieces of legislation, Senate Bill 243, will force AI companies to create meaningful guardrails that stop chatbots from encouraging self-harm in young users. Companies will need to develop protocols that stop the bots from producing content related to “suicidal ideation, suicide, or self-harm,” says the bill’s author, Democratic Senator Steve Padilla. The chatbot operators would also be required to provide “a notification that refers users to crisis service providers and require annual reporting on the connection between chatbot use and suicidal ideation to help get a more complete picture of how chatbots can impact users’ mental health.”
AI chatbots have increasingly been involved in mental health incidents, including ones involving suicides and murder. A lawsuit filed against OpenAI by the family of a teenager who recently killed himself implicates the company’s chatbot, ChatGPT, in the young man’s suicide.
The bill includes a private right of action that gives Californians the “right to pursue legal actions against noncompliant and negligent developers,” said Padilla. In other words, if AI companies don’t comply with the new regulation, families will be well within their rights to sue those companies.
“These companies have the ability to lead the world in innovation, but it is our responsibility to ensure it doesn’t come at the expense of our children’s health,” said Padilla.
Also signed into law on Monday was a bill, AB 56, that would append warnings to social media platforms similar to those featured on cigarettes. According to Newsom’s website, social media companies will now have to come with warning labels that “warn young users about the harms associated with extended use of social media platforms.”
Another bill, the Digital Age Assurance Act, forces platforms to institute age-verification mechanisms that protect young users from certain kinds of content. The legislation forces users to enter their age and birthday when setting up a new device, The Verge reports. In that sense, Newsom follows in the footsteps of a number of conservative states, which have instituted age-verification regulations in recent years.
At the same time, while the governor brought a number of new AI regulations into existence, he vetoed multiple laws that would have instituted harsh new punishments on tech platforms.
One of the vetoed bills, Assembly Bill 1064, or the Leading Ethical AI Development for Kids Act, would have essentially banned companies from providing young users with “companion chatbots” unless they could demonstrate that their products were not going to harm the children. The legislation would have ensured “that our children are not the subject of companion chatbots and AI therapists,” said the bill’s author, Assemblymember Rebecca Bauer-Kahan. SFGate reports that Newsom vetoed the bill amidst “a massive lobbying push from tech companies.”
The other vetoed bill, Senate Bill 771, would have instituted large fines on social media platforms that failed to clean up violent and discriminatory content on their platforms. Websites would have faced fines of up to a $1 million if content on their site violated California’s civil rights laws in such a way that a user was harmed.
Newsom seemed to support the legislation’s overall goal, but said he would rather rely on existing laws to deal with it. “I support the author’s goal of ensuring that our nation-leading civil rights laws apply equally both online and offline,” Newsom said, in a statement explaining his veto. “I am concerned, however, that this bill is premature. Our first step should be to determine if, and to what extent, existing civil rights laws are sufficient to address violations perpetrated through algorithms.”
Gizmodo reached out to the governor’s office for comment.
The vetos may have been disappointing to tech critics. However, under Newsom, California has proven itself to be a state leader in regulatory approaches to technology. The state’s California Consumer Privacy Act was one of the first comprehensive privacy laws in the nation, and a model for other states.
Last week, Newsom also signed into law a number of new privacy regulations that will give Californians more control over their data, including a bill that will force web browsers to include an “opt-out” function that allows users to automatically exclude themselves from data collection covered by the state’s CCPA. The new AI regulations add to that legacy, showing that, whether the attempts are perfect or not, California is—at the very least—trying to reign in Big Tech.