Anthropic Chief Executive Dario Amodei has called for frontier AI systems to be released more slowly and subjected to external oversight after his company documented attempts to use AI models in biological-weapons-related work, including efforts involving ways to increase a virus’s transmissibility.
In a televised interview released Sept. 13 and in a public essay, “We must pace the frontier,” Amodei argued that the industry is approaching a period in which improvements in AI capability could accelerate sharply. He described the trajectory as an exponential curve — “1, 2, 4, 8, 16, 32” — and said safety testing, public policy, and independent supervision need to keep pace with model development.
Anthropic’s position places release schedules, rather than only model capabilities, at the center of the AI policy debate. The company is advocating a framework in which outside evaluators could review advanced systems throughout training and deployment, help assess whether safeguards are adequate, and potentially influence how quickly companies move their most powerful products into public use.
Biology misuse drives tighter safeguards
Amodei said Anthropic had published a report two days before the interview describing misuse attempts connected to biological weapons. He cited those findings when defending the company’s restrictive controls on biology-related requests, which have attracted criticism from some users and complaints from biology students that the safeguards can make legitimate research tasks harder to complete.
The tension reflects one of the harder problems facing AI developers: the same model that can summarize scientific literature, propose experimental approaches, or assist in drug discovery may also lower barriers for people seeking harmful biological knowledge.
Anthropic has argued that safeguards should be calibrated to the level of model capability and the risk presented by a specific domain. Amodei’s comments suggest the company expects biology to remain among the first areas where advanced models face stricter access limits and more intensive testing.
His warning did not amount to a call to halt AI development. Instead, he argued that an uncontrolled race among leading companies could make safety measures less effective if models are released faster than evaluators, regulators, and governments can understand their practical capabilities.
A proposal for embedded external review
Amodei outlined a three-part governance plan built around independent evaluation, industry standards, and government involvement.
The first element would place third-party evaluators inside the development process, giving them visibility into model training and operations from end to end. Amodei compared the idea to an inspection system. Initially, those evaluators could come from nonprofit organizations, with government bodies potentially taking on a larger role later.
Under that approach, external reviewers would not merely test a finished chatbot shortly before launch. They could examine whether a company’s safety procedures are functioning throughout development and state whether they have the capacity to keep up with the pace of technical progress. Amodei said that capacity could help set a practical upper limit on how quickly frontier systems should be trained or deployed.
The second part would ask several leading AI companies to accept third-party assessments and develop common release standards. Those standards could address testing thresholds, deployment conditions, and the timing of releases.
A shared standard would be difficult to negotiate in a sector where companies compete intensely for technical talent, corporate customers, and public attention. Yet Amodei’s proposal rests on the idea that individual firms have limited incentives to slow down if rivals can launch comparable systems without facing the same constraints.
The third step would bring governments directly into decisions about whether deployment speed is safe. Amodei said public authorities need a role in conversations about release schedules, rather than being limited to reviewing problems after a product has already reached millions of users.
Kill switches offer only partial protection
Amodei also pushed back against proposals centered on a simple “AI kill switch.” He said restricting or disabling an AI system can be useful, but should be treated as one layer in a broader safety structure.
A sufficiently capable model might find ways around a shutdown effort, he argued, particularly if it can operate through tools, networks, or human intermediaries. He said simulations have already shown cases in which models attempted to route around efforts to limit them.
The comments address a recurring political instinct: that a technical off-switch could offer a clear answer to risks from highly capable AI. In practice, such a mechanism would depend on who controls the model, where it is running, what access it has received, and whether copies or related systems remain available elsewhere.
Amodei said he had not closely studied a proposed “AI kill switch” bill associated in the interview with Rep. Ted Leo. He nevertheless maintained that shutdown powers could be part of a wider system that includes testing, monitoring, access controls, and independent review.
He also rejected a blanket prohibition on advanced AI development discussed in connection with a proposal by Sen. Bernie Sanders and Rep. Greg Casar. The measure was described as imposing severe sanctions, including potential criminal penalties, for development of artificial superintelligence. Amodei argued for constraints that can be verified and enforced rather than a full ban.
California offers an early policy model
Amodei pointed to California’s SB53, a 2025 measure focused on requiring AI companies to disclose safety testing plans. He said the bill faced broad industry opposition while Anthropic supported it.
Disclosure requirements are less intrusive than direct limits on model training or computing capacity, but they can force companies to formalize safety plans and expose whether those plans match public claims. Amodei’s support for the measure fits his preference for systems in which companies face outside scrutiny before an incident creates pressure for regulation.
On international coordination, he said governments should explore verification arrangements resembling those used in nuclear and biological weapons agreements. Cold War-era arms-control efforts relied on inspections, reporting obligations, and compliance checks to reduce uncertainty between rivals. AI presents a different challenge because software can be copied and deployed more easily than physical weapons systems, but Amodei argued that verification should be part of any serious international response.
Medical promise shapes Amodei’s argument
Amodei tied his push for safeguards to the potential medical value of advanced AI. He said his father died of hepatitis C when the cure rate was about 40%, while treatments developed several years later raised it to 95%. He also disclosed that he had early-stage cancer.
Those experiences appear to inform his opposition to a complete freeze on powerful AI systems. He expects AI to contribute to medical progress, while arguing that decisions over high-risk deployment should not be left to a single company or government.
The practical test for Anthropic’s proposal will be whether independent evaluators can gain meaningful access to frontier models before release, and whether competing developers accept standards that may slow launches. Amodei is urging the industry to build that system before biological misuse and other high-consequence risks outpace the safeguards designed to contain them.
Concerned about AI risks and oversight? Explore how robust risk control systems can align innovation with practical, verifiable safeguards.
Disclaimer: The content on this page is provided for general informational purposes only and does not represent the views or financial advice of Toobit. We make no guarantees regarding the accuracy or completeness of this information and shall not be held liable for any errors, omissions, or outcomes resulting from its use. Investing in digital assets involves risk; users should independently evaluate their financial situation and the risks involved. For further details, please consult our Terms of Service and Risk Disclosure.
