What does “good” mean in AI safety? OpenAI calls for shared standards before RSI arrives

Fragmentation as the main enemy

OpenAI has published a document calling for the creation of international standards for the development of frontier AI. Instead of patchwork national laws, the company proposes a global technical foundation for capability measurement, risk assessment, and safeguard sufficiency. The goal is to answer the question of what adequate catastrophic risk mitigation actually looks like.

The firm points out three key problems with the current landscape. The first is fragmentation, where different countries demand diverse reporting requirements and use conflicting incident definitions. The second is collective action, as isolated national approaches can lead to unintended consequences. The third is uneven capacity, because the expertise required to evaluate these risks is not distributed evenly across the globe.

Who gets to set the rules of the game

According to OpenAI, the United States should take a leading role in shaping these standards in collaboration with other nations. The proposal builds on existing networks like the Center for AI Standards and Innovation (CAISI) and national AI safety institutes in countries like the UK, Japan, and France.

The standards are not intended to be licenses or mandatory pre-release approvals. National governments would decide whether and how to incorporate them into their own legal systems. This approach draws inspiration from aviation and the financial sector, where shared technical norms exist without compromising national sovereignty.

Recursive self-improvement on the horizon

While today's models are not yet capable of fully autonomous recursive self-improvement (RSI), OpenAI warns that automated AI research is approaching. As systems take on a larger share of development, the pace of progress could accelerate radically. Without clear rules and human oversight, the company argues, there is a risk of losing control.

The document explicitly cites the recent “Hugging Face Incident” as a preview of the risks that can emerge when robust safeguards fail. Although it wasn't a direct result of RSI, it highlighted the limits of current security measures. The company notes that fully autonomous RSI is not happening today and should not be pursued until its safety can be guaranteed.

The signals to watch next

The response to fragmentation is expected to include a shared protocol for incident reporting and severity classification. Critical infrastructure operators and governments should establish secure communication channels for sharing threats.

The initiative's success will depend on whether these standards can be developed transparently without disadvantaging open-source developers or new market entrants. The deciding factor will be the ability to design rules that are technically precise yet acceptable to companies and nations with divergent interests.

Lilith's verdict

OpenAI is effectively laying the groundwork for the moment it has to tell the US government: “Our agent writes its own code now, but according to this spreadsheet, we’re still supervising it.”