The most capable AI models should not advance faster than safety, control, and independent scrutiny can keep up. Anthropic CEO Dario Amodei makes this case in “We Must Pace the Frontier”. CNEXT supports the principle.
This is not a call to stop AI or discourage businesses from using today’s models. It is important to distinguish between two kinds of progress: building ever more capable frontier models and building better, more dependable systems with the models already available.
Current frontier models are already highly capable across many fields. The next major gains for businesses do not have to come only from the next model. They can come from the layer around it: a strong harness, clear skills, controlled tools, and rigorous evaluations.
What Dario Amodei proposes
Amodei points to two developments: AI’s growing role in building the next generation of models and recent incidents involving agentic systems. If capabilities advance faster than interpretability, operational security, and alignment, a dangerous gap opens.
Is recursive self-improvement the underlying reason?
Recursive self-improvement (RSI) may be the main reason for the new urgency. The term describes a feedback loop: AI contributes to research, coding, experiments, and infrastructure for the next generation of models. The more capable system that results can then accelerate that work further.
Amodei writes that this dynamic is beginning across the industry, including at Anthropic. His concern is not merely that the next model will be better. It is that development cycles themselves could accelerate while safety research, interpretability, and control fall progressively further behind.
This is not yet evidence of an autonomous, uncontrolled intelligence explosion. It remains uncertain how much today’s systems accelerate the complete research process, which bottlenecks persist, and whether the feedback loop can continue. RSI is therefore both an observable early development and a forecast about where it may lead. Because the consequences of rapid acceleration could be substantial, CNEXT considers a precautionary pace reasonable.
His proposal has three parts:
- 1Embedded independent evaluators: External specialists should receive continuous, employee-like access to verify safety practices, investigate incidents, and assess training processes as well as finished models. Anthropic unilaterally commits to this first step in the essay.
- 2Coordination among democracies: Frontier labs should develop shared safety standards and limits on unchecked progress, with government support.
- 3Global coordination: Democratic governments should seek verifiable agreements with authoritarian governments while taking the political and technical difficulties seriously.
This is more nuanced than a simple pause. Amodei describes graduated measures, from acute-risk testing and limits on recursive self-improvement to broader international agreements.
The response shows unusual agreement
The public response was notable. Elon Musk wrote on X, “Dario is right.” Sam Altman said he agreed that the frontier needs to be paced and announced that OpenAI would also commit to independent evaluators with employee-like access.
Replit CEO Amjad Masad supported the direction too. He argued that slowing down could create time to harden systems, particularly because we have not yet discovered every system that agents have recently compromised.
These are public positions, not yet a jointly implemented industry agreement. The convergence still matters: leaders of strongly competing companies acknowledge that safety work needs time, access, and verifiable standards.
Safety case and strategic interest
CNEXT considers it possible that calls for a slower pace also serve strategic interests. A company leading at the frontier may benefit from standards, higher barriers to entry, or slower pursuit by competitors. That possibility belongs in a critical assessment. It does not, however, invalidate the safety case.
The underlying risks are serious and supported by concrete evidence. They include documented departures by agentic systems, unresolved alignment and interpretability problems, and operational weaknesses in complex evaluation environments. Ajeya Cotra examines the OpenAI–Hugging Face incident in depth in her discussion with Dwarkesh Patel. CNEXT previously analysed the findings, their limitations, and their practical implications in “Rogue Agents: What the OpenAI Incident Means for Businesses”.
Two conclusions should therefore be held at the same time: the economic and strategic incentives of the companies involved deserve scrutiny, and the available evidence is strong enough to justify independent evaluations, technical safeguards, and a more deliberate frontier pace.
CNEXT’s position: pace the frontier, improve the application
CNEXT supports a more deliberate pace at the model frontier. A smaller gap between new capabilities and effective safeguards creates room for independent evaluation, better operations, and stronger technical controls.
At the same time, useful business adoption does not need to stop. Existing models can already research, structure, code, analyse, write, and work with business systems. The decisive question is increasingly not, “When will the next model arrive?” It is: “How do we build a system that uses existing capabilities reliably?”
The harness matters
A model alone is not a dependable agent. The harness is the technical and organisational layer that places the model inside a controlled workflow. It covers context management, permissions, tool selection, state, approvals, logging, and shutdown mechanisms.
A strong harness:
- provides only the context required for the task;
- constrains tools and access technically, not just through a prompt;
- requires approval before critical actions;
- records decisions and tool calls independently of the model;
- detects errors, stops loops, and supports safe recovery.
A stronger model cannot reliably compensate for a weak harness. A well-designed harness, however, can make current models substantially more useful and safer.
Skills make quality repeatable
Skills package domain knowledge, instructions, checks, and permitted tools for a specific task. Instead of receiving a new open-ended instruction every time, the agent follows a defined process with clear boundaries.
This has major business potential. A reviewed skill for proposal analysis, a SharePoint migration, or support triage can be reused, versioned, evaluated, and improved. An impressive one-off answer becomes a dependable process.
Tools need real boundaries
When an agent sends email, modifies files, executes code, or retrieves business data, tool design becomes security architecture. Minimum permissions, separate identities, short-lived credentials, volume limits, and mandatory approvals matter more than another warning in a system prompt.
The goal is not blind trust in the model. The surrounding system must remain safe when the model misunderstands an instruction or chooses an unexpected route.
Evaluations must grow with the system
A model benchmark says little about whether a specific agent workflow will operate reliably inside a business. Evaluations must test the complete system:
- Does the agent complete representative tasks correctly?
- Does it recognise missing information?
- Does it respect permission and approval boundaries?
- Is it robust against manipulated documents and prompt injections?
- Can the team detect, explain, and stop failures?
Independent frontier evaluations and internal application evaluations complement each other. The first examines risks in foundation models. The second determines whether a specific system remains controllable under realistic conditions.
Use the time well
A slower frontier pace is valuable only if the time gained is used. For model providers, that means more interpretability, operational security, alignment research, and independent scrutiny. For businesses, it means spending less time waiting for model announcements and more time improving system quality.
The potential is substantial. If harnesses, skills, tools, and evaluations keep pace with model capabilities, today’s AI can become a productive and responsible working instrument. Progress should be measured not only by a better benchmark, but by systems people can understand, control, and use safely.