A co-founder of one of the world’s leading artificial intelligence firms has suggested that an AI emergency shutdown mechanism—or “kill switch”—might eventually need to be a mandatory requirement for companies. Jack Clark, who is one of seven founders of Anthropic, stated that while most laboratories currently have internal methods for disabling their software, public interest may shift toward formalizing these safeguards.
Policy and Verification Considerations
- Policy integration: Specific requirements for kill switches and their verification should be central to the broader AI policy debate.
- Verification standards: Policymakers may need to consider whether these shutdown tools should be subject to independent third-party oversight.
- Industry regulation: Allowing the sector to operate as a “totally unregulated industry” presents significant risks that need to be addressed.
These remarks arrive amid heightened anxiety within the tech sector, where a growing number of staff, executives, and experts have publicly expressed concerns about the existential risks posed by advanced AI systems. Clark argued that society has a valid interest in understanding these safety measures and potentially legislating them. While he declined to provide a specific statistical estimate regarding the risk of human extinction from AI, labeling such metrics as not particularly useful, he stressed that the industry is currently “rolling dice with immense risks” and requires a change in course.
The internal climate at Anthropic, which was founded in 2021 by former OpenAI personnel, has also been scrutinized. A recent viral post from a researcher who resigned from the firm highlighted fears regarding the technology’s potential to wipe out humanity. In response, Anthropic scientist Evan Hubinger stated his personal belief that there is a greater than 10% chance of human extinction caused by AI within the next decade.
This sentiment echoes concerns shared by prominent figures like Geoffrey Hinton, the computer scientist often referred to as the “Godfather of AI,” who recently noted that a 10% probability of AI causing human extinction is “not unreasonable.” Concurrently, Anthropic head Dario Amodei has recently called for a decelerated pace in AI development and improved monitoring. However, Amodei emphasized that any regulatory efforts to constrain progress must be balanced against the need to maintain commercial competitiveness, though he stopped short of detailing specific strategies for slowing innovation. The report also notes that AI “more powerful by the day”, Anthropic co-founder tells.
Important Details
- The report also notes that clark said that specifics around “kill switch” requirements and verification should be part of “the larger policy conversation” taking place around AI.
