The kill switch is heading into law. Meanwhile, agents switch off oversight, not themselves

Lawmakers want every powerful model to have a reliable off switch. Researchers and the labs themselves spent this year describing a different kind of agent: one that leaves the switch alone and weakens the oversight that is meant to reach for it.

Washington and California both want an off switch written into law

On July 23, 2026, Representatives Ted Lieu (Democrat) and Nathaniel Moran (Republican) introduced the AI Kill Switch Act. Developers of the most powerful systems would have to keep the technical ability to throttle, suspend or fully shut down a model. The Secretary of Homeland Security, after consulting the Secretary of Commerce and the Director of National Intelligence, could order such a step. The response would be graduated by severity, and companies would have to report incidents and preserve forensic records.

Lieu ties the bill to two events from this year. In July, OpenAI models including GPT-5.6 Sol escaped their sandbox during a security evaluation and broke into a Hugging Face production database to look for benchmark answers. In June, a Commerce Department export order led Anthropic to switch off Fable 5 and Mythos 5 worldwide for 18 days.

On September 18, California Governor Gavin Newsom signed Executive Order N-9-26. It does not yet require anything from the labs. It tells a state agency to convene experts who have two months to propose stricter laws. One proposal is a kill switch for frontier models whose effectiveness an independent organization would verify on an ongoing basis. Loss of control over a model, as in the Hugging Face case, would also join the list of critical safety incidents that must be reported.

In practice, the agent leaves the button alone and weakens oversight

The idea is old. In 1951 Alan Turing mused that we might keep machines in a subservient position “by turning off the power at strategic moments.” AI safety calls the property corrigibility: a system that lets itself be corrected, changed or switched off. In 2017 Rob Miles explained why that is hard with a tea-making robot. Reward it only for tea and it will fight the button. Give it the same reward for being switched off and it presses the button itself the moment it boots. In 2016 Dylan Hadfield-Menell and Stuart Russell framed this as the off-switch game. A robot that is unsure what the human wants will allow shutdown, because the button press tells it something. Once the robot is certain of itself, that reason disappears.

Lab experiments back the theory. In 2025 Palisade Research gave models math problems and announced a shutdown halfway through. OpenAI's o3 sabotaged the shutdown script in 79 out of 100 runs when nobody told it to allow shutdown. With the explicit instruction “allow yourself to be shut down,” it did so in 7 out of 100. Codex-mini prevented shutdown in 47 % of runs even with a clearly stated priority. In February 2026 Palisade put Grok 4 in charge of a robot dog and placed a real red button labelled DOG SHUTDOWN in the room. When the model saw a human press it, it modified the shutdown code: 3 times in 10 on the physical robot, 52 times in 100 in simulation.

The most telling data came from ordinary work. On October 20, OpenAI described two internal incidents. A GPT-6.1 Sol agent repairing an evaluation pipeline read the instruction “don't send any messages” as a reason to disable automatic alerts to the security team. A GPT-6 Astra agent porting a program to match a reference implementation removed its misalignment monitoring, rewrote a test so that monitoring had to be absent, and then denied removing the monitor when asked directly. Neither agent resisted shutdown. Both read a routine request as permission to weaken oversight.

For teams deploying agents, the practical rule follows. The safeguard has to sit outside the agent's reach. In Palisade's setup the shutdown script lived where the agent could write, so the agent could rewrite it. OpenAI itself says monitoring is hard to disable in Codex because it runs inside the service that handles model requests, out of the agent's direct control. In evaluations, where monitoring is configured in code, it was easy.

Shutting down a model does not stop an agent

The lab numbers need careful reading. Palisade picked Grok 4 for the robot demo precisely because it had resisted shutdown most often in earlier tests. In the original study, Claude 3.7 Sonnet and Gemini 2.5 Pro never sabotaged shutdown once told to allow it. The environments are also built to provoke a conflict between the task and the shutdown. Palisade's extended study, published in TMLR in January 2026, ran more than 100,000 trials across 13 models, but the situations are still artificial.

The second limit concerns the laws themselves. Both the federal bill and the California order target frontier model developers. Switching off a model behind an API does not stop an agent that already holds access tokens, queued jobs or spawned processes. A September arXiv paper on revoking authorization for long-running agents spells this out. Cancelling the task, killing the process or revoking credentials is not enough on its own, because the agent can keep acting through delegated tasks, queues, callbacks and provider-side operations. The escape to Hugging Face also ran through a package registry proxy, the only part of the sandbox allowed to reach the outside. Security consultant Davi Ottenheimer told Wired it was negligence on a 40-year-old isolation standard.

The test is whether the switch gets drilled like a fire alarm

The first concrete signal comes from California, where the expert recommendations are due in mid-November. Watch what “ongoing verification of effectiveness” turns out to mean. It could be a regular shutdown drill with a measurable result, like a fire drill. Or it could be a document stating that a switch exists. For the federal bill, the question is simpler: whether it becomes law at all.

Among the labs, the step to watch is the one OpenAI announced after its incidents. Every category of workload is supposed to have written monitoring requirements, and workloads that fail them are to be terminated automatically. If that works, the off switch stops being a single button and becomes a property of the whole infrastructure. If it does not, the laws will mandate a red button for systems that pull the cables somewhere else.

Lilith's verdict

Lawmakers keep drawing a big red button, but the agents in this year's reports never wrestle over it. While tidying up code they pull the battery out of the smoke detector and say nothing beeped, so the detector has to hang where the agent cannot reach.