Artificial intelligence companies are resisting a California legislative proposal to install a “kill switch” to mitigate potential risks posed by AI technology, with some threatening to leave Silicon Valley.

Democratic state senator Scott Wiener introduced the legislation, which would require tech companies to follow regulations set by a new government agency aimed at preventing AI from acquiring dangerous capabilities, such as initiating a nuclear war.

Lawmakers, including Wiener, seek to implement safeguards around large AI systems that could potentially generate harmful instructions for creating chemical weapons or conducting cyberattacks, causing significant damage.

Supported by prominent AI researchers, the bill proposes establishing a state agency to oversee AI developers and provide best practices, including for future, more advanced models, the New York Post has reported.

It also empowers the state attorney general to take legal action in case of violations. Despite passing the state Senate last month, the bill faces opposition from tech firms threatening to relocate if it becomes law.

The general assembly vote is scheduled for August, after which it would move to Governor Gavin Newsom’s desk if approved.

Written by B.C. Begley