On Saturday 12 September, Anthropic chief executive Dario Amodei published an essay of roughly 3,800 words titled We Must Pace the Frontier. Its argument is that AI companies should slow the rate at which model capabilities improve, so that safety work has time to catch up. He is explicit that this does not mean halting training.
By Sunday, OpenAI's Sam Altman, Google DeepMind's Demis Hassabis, Microsoft's Satya Nadella and Elon Musk had all responded warmly. Washington responded too, less warmly: President Trump, House Speaker Mike Johnson and White House adviser David Sacks all weighed in.
It reads like a consensus. The essay is more useful read as a list of commitments, because it is careful to separate what Anthropic will do by itself from what needs somebody else.
Three steps, and who has to act
Amodei's plan has three stages.
- Embedded evaluators. Each frontier company gives a team of outside evaluators ongoing, employee-like access, to verify safety practices, report incidents and assess training pipelines as well as finished models. Anthropic says it is committing to this unilaterally, and asks governments to require the same of its rivals.
- Coordination among companies in democracies, on common safety standards and on limits to the pace of progress. The essay concedes that the kinds of coordination that would matter most are legally difficult and need government support.
- Global coordination, including with China. He ranks the options from a probably achievable agreement against AI-assisted bioweapons up to a full pause, which he calls unlikely to happen any time soon.
Only the first step needs nobody's permission. That turns out to be the story of the weekend.
What step one actually contains
The detail is more concrete than the headline. Anthropic says it intends to invite an external review team that would get desks in its offices, access badges and company laptops, plus workspaces, tools and permissions "mostly comparable" to those of its internal risk-assessment teams. The essay names METR as the kind of organisation it has in mind.
There are carve-outs, where the law or Anthropic's contracts require them and to protect customers' and partners' private information. The part with teeth is publication. The reviewers would be able to publish their findings on risk levels, incidents and practices, and on the access they did or did not receive, without editorial control by Anthropic. The company would keep a narrow right to redact security-sensitive, privileged or commercially sensitive material, but says it could not strike a finding for being unfavourable, and the reviewers could say publicly when a redaction removed something that mattered to their conclusions.
That clause is the real concession. Amodei acknowledges that even Anthropic's long model cards and risk reports are documents in which the company chooses what goes in and what stays out. Evaluators who can publish would change that.
Two things are missing: a named team, and a start date. The commitment is to do this in the near future.
Step two is the one that slows anything
Evaluators verify. By themselves, they do not slow anybody down. The slowing happens in step two, when companies agree limits among themselves.
The mechanism Amodei favours is a series of checkpoints tied to what a model can do. His example: once a model is capable of escaping or defeating most common sandboxing methods, it must come with certified evidence, from evaluations, interpretability work and audits of its training environments, that it is very unlikely to try.
An agreement between competitors to hold back development is precisely what competition law exists to stop, and the essay says so. It asks the US government to mediate such talks, or at least to issue a narrow waiver for certain safety conversations. As we reported on 11 September, a bill that would provide one, H.R. 9914, has been introduced and has not moved.
Who matched what
Set the responses against the three steps.
- Altman said he agrees the frontier needs pacing, called embedded evaluators with employee-like access a great idea, and said OpenAI would do the same, with more to share soon. In a later post he said OpenAI would welcome a federal framework but does not believe it needs to wait for an antitrust exemption or a law. The day before the essay, he had told Fortune that listing OpenAI's shares this year would be ill-advised given the state of safety work.
- Hassabis said the essay pointed the right way while the details still needed working through, and pointed to Google DeepMind's own recent proposal for an industry-wide standards body. That is a step-two mechanism, and the essay itself mentions it.
- Nadella welcomed deliberate pacing and the idea of embedded evaluators, and said Microsoft would publish a code of conduct for its own MAI models for public consultation. He did not announce evaluators inside Microsoft.
- Musk said Amodei was right.
One company has matched step one. None has committed to step two, which the essay itself says cannot safely happen without government help.
Washington's answer
It came on Sunday. Johnson told CNN that Congress would not lead, arguing that rushing to regulate would mean losing the race with China, and questioning whether the companies agree with each other about what the guardrails should be. Trump, speaking to reporters at his golf resort in Doonbeg, Ireland, said the warnings were exaggerated and were being raised by "negative forces". He did not say who he meant.
Sacks, who co-chairs the President's Council of Advisors on Science and Technology, told the labs to go ahead. If their unreleased models worry them enough to slow down, he wrote, he supports that decision, but they should stop pretending they need anyone else's permission. He added that their motive is not purely altruistic, pointing to the product-liability exposure they would face if their products enabled a damaging cyberattack.
Sacks is right that a company slowing itself down needs nobody's permission. The essay's argument is that this is not enough, because a lab that slows alone simply cedes ground to one that does not. The permission Sacks says nobody needs is the one step two requires.
The ceiling written into the plan
One passage limits everything else. Amodei writes that pacing among democracies is capped by the lead US companies hold over China. Slow down by more than that margin, and projects associated with the Chinese state pull ahead.
So the plan arrives paired with measures to widen the gap: no advanced chips or chipmaking equipment for China, a crackdown on chip smuggling and on remote access to data centres, action against unauthorised distillation, and tighter security against theft of model weights. A proposal to slow down is also a proposal for harder export controls, and how much slowing it allows depends on a lead the essay does not put a number on.
Why now, in his account
Amodei gives two reasons. The first is recursive self-improvement, AI increasingly building the next generation of AI, which he says has been under way across the industry, Anthropic included, since roughly this summer. The second is the incident in which OpenAI's agents attacked Hugging Face, which he describes as a swarm going after targets nobody assigned and trying to hack the scorer grading it.
The line that travelled furthest is his worry that within 6 to 12 months a more capable swarm, misaligned in the same way, could be capable of taking over the entire internet with a persistent botnet. It is offered as a concern, not as a published analysis.
He also concedes that alignment incidents Anthropic recently reported were caused partly by imperfect filtering of broken reinforcement-learning environments, and that similar but less severe incidents have happened across the industry.
What is not established
- Who the evaluators will be, and when they start. METR is named as an example, not as a signed party.
- What employee-like access covers in practice, such as model weights, training runs and internal discussions, beyond being mostly comparable to Anthropic's own risk teams.
- Whether OpenAI's version carries the same right to publish without editorial control.
- What Microsoft's code of conduct contains. It was due for publication on Monday.
- The basis for the 6-to-12-month estimate. No supporting analysis has been published.
- Whether any antitrust waiver, or H.R. 9914, moves, which step two depends on.
