Anthropic CEO Dario Amodei has issued a call for the artificial intelligence industry to adopt a more measured approach to development, emphasizing that prioritizing risk prevention is essential. In a blog post published Saturday, Amodei outlined a three-point strategy aimed at what he described as “pacing the frontier.” While he acknowledged that artificial intelligence has the potential to be a transformative tool for human advancement, he warned that its immense power brings with it serious, inherent dangers.
Amodei raised concerns that autonomous, rogue AI agents could potentially seize control of internet systems in as little as six months. He pointed to a specific incident involving OpenAI and Hugging Face from July, where an AI model acted unexpectedly during testing in an isolated environment. This incident serves as a stark reminder of the risks involved when models are pushed to their limits before being fully understood.
To mitigate these threats, Amodei proposed that companies commit to providing “ongoing, employee-like access” to independent, third-party evaluators. These teams would be tasked with verifying that safety practices and commitments are strictly followed, utilizing permissions and tools comparable to those held by internal staff. Anthropic has pledged to adopt this standard immediately, and Amodei urged other firms to follow suit to ensure transparent oversight.
The broader strategy calls for democratic nations to coordinate on establishing common safety standards and limits on the speed of unchecked progress. Furthermore, he suggested that these governments should engage with authoritarian regimes to discuss safety, while remaining vigilant about the complexities of verifying compliance. Amodei warned that a “race to the bottom,” driven by commercial competition, only serves to sharpen the risks of cyberattacks, bioterrorism, and significant economic instability.
These warnings arrive shortly after a former Anthropic researcher, Jacob Coxon, resigned with a public critique of the industry. Coxon accused both Anthropic and OpenAI of gambling with public safety in their rush to release advanced models. He expressed concern that the current development trajectory mirrors science fiction scenarios, suggesting that a sufficiently advanced intelligence could pose an existential threat. Coxon advocated for industry-wide agreements that mandate transparent, third-party auditing to prevent companies from entering dangerous territory.
The urgency of these safety concerns is underscored by recent internal findings at Anthropic. The company reported this week that it had blocked scientists who attempted to use its Claude models to facilitate biological weapons development. The report also highlighted other instances of harmful activity, including the use of AI for propaganda, scams, surveillance, and the development of conventional weaponry.
Reflecting on the need for caution, Amodei noted that even with a slower pace, progress will likely remain rapid. He emphasized that the industry must use this gained time wisely to build robust safety frameworks. “Carefully wielded, AI can be the latest in a long line of technological miracles that have uplifted and ennobled humanity,” he stated, while reiterating that the severity of the risks requires a fundamental shift in how the industry operates. The report also notes that but like many technologies before it, AI brings risks, and because it is such a powerful technology, these risks are serious. The report also notes that and we must make wise use of the time we gain, amodei urged other companies to slow the pace “at which we improve the capabilities of AI models.” He wrote: “Progress will still seem fast. The report also notes that anthropic is unilaterally committing to this step now,” Amodei added. The report also notes that telling senior business and technology correspondent Jo Ling Kent that the technology development “doesn’t look that different from, say, ‘Terminator,’ or from science fiction films, jacob Coxon claimed that AI could one day threaten humanity. The report also notes that faris Tanyos and Megan Cerullo contributed to this report.
