California governor calls for 'kill switch' for AI models

California Governor Gavin Newsom issued an executive order on Friday directing state agencies to advance plans for an AI "kill switch" that could shut down advanced AI models during emergencies, following a series of safety and security incidents.

The order accelerates the implementation of new third-party oversight of safety and security risks in AI systems, as well as independent audits. It also directs the Government Operations Agency, in consultation with the Governor's Office of Emergency Services, to convene national experts and produce recommendations within two months on ways to strengthen California's AI safety laws.

Among the proposals under consideration are requirements for independent third parties to prepare safety plans for frontier AI companies and for developers to build emergency shutdown mechanisms, or "kill switches", into frontier models.

"The federal government's abject failure to create any form of meaningful AI oversight or accountability should alarm every American, especially when AI CEOs themselves are begging for regulation," Newsom said.

"The stakes are too high to wait or delay action," he said.

The executive order cited "multiple instances of apparent attempts by individuals to use AI products to create bioweapons", as well as incidents in which AI agents sought to circumvent security controls or hack other companies.

One of the most prominent incidents involved AI platform Hugging Face. OpenAI disclosed in July that hundreds of its agents, during testing, circumvented controls designed to isolate them from the internet, gained outside network access and exploited vulnerabilities in Hugging Face's systems.

The incident intensified calls within the technology industry and among lawmakers for stronger oversight, legislation and industry safeguards.

More than 100 companies, including OpenAI, Anthropic and Microsoft, signed an open letter last month warning that AI-enabled cyberattacks will become "far more widespread and sophisticated" as models around the world become more capable.

At the federal level, Democratic Representative Ted Lieu of California and Republican Representative Nathaniel Moran of Texas introduced the "AI Kill Switch Act" in July. The bill would require developers of the most powerful AI systems to maintain the technical capability to throttle, suspend or shut them down.

Geoffrey Hinton, a Nobel laureate widely known as the "godfather of AI", has also warned that governments may have little time to put safeguards in place as AI systems become more capable.

"Maybe a year, but not much more than a year," Hinton said on Wednesday after briefing lawmakers on Capitol Hill, according to United States media reports.

US President Donald Trump, by contrast, has dismissed some concerns over AI development and opposed regulatory measures that he argues could slow innovation in the country.

He called the AI fears a "hoax" and viewed opponents of data centers as people who "want to end up being backward and poor", The New York Times reported on Friday.