AI Could Wipe Out Humanity Within a Decade, Researcher Warns

Rapid advances in artificial intelligence could pose a serious threat to humanity’s survival, with a senior safety researcher at AI company Anthropic estimating that there is more than a 10% chance AI could kill all humans within the next decade.

Evan Hubinger, who leads Anthropic’s alignment science efforts, made the assessment in a post on X while responding to concerns raised by former colleague Jacob Coxon, who recently resigned from the company.

Hubinger said the immediate risk posed by current AI models is low. However, he warned that the technology could advance rapidly toward systems capable of improving themselves, potentially creating risks that humans may struggle to control.

His warning centres on the possibility of superintelligence emerging through recursive self-improvement—a scenario in which AI systems could autonomously design and improve subsequent generations of AI at an accelerating pace.

Hubinger said Anthropic is trying to address the risks, but acknowledged that the company does not yet have a clear plan for ensuring that a future superintelligent system remains aligned with human interests. He also said the company is not clearly on track to develop such a solution.

Concerns Over Superhuman AI

Hubinger’s comments came after Coxon, an AI researcher who previously worked at OpenAI, announced his departure from Anthropic. Coxon accused leading AI companies of failing to act responsibly as they race to develop increasingly powerful systems.

He argued that companies could soon create systems with capabilities far beyond those of humans, including the ability to hack systems, accelerate technological breakthroughs and acquire resources independently.

The exchange has renewed debate within the AI industry over whether the development of increasingly autonomous and powerful systems is moving faster than efforts to make them safe.

The AI Alignment Challenge

Hubinger works in AI alignment, a field focused on ensuring that advanced AI systems behave in ways consistent with human goals, values and instructions.

The central concern is that a highly capable AI system could eventually pursue objectives that conflict with human interests, particularly if it becomes capable of modifying or improving its own capabilities.

Anthropic has previously acknowledged that highly capable future AI systems could present serious risks if their goals become misaligned with those of their developers or if they gain the ability to exploit weaknesses in their own operating environment.

In its latest risk assessments, the company has said the possibility of rapid AI progress and increasingly capable systems requires greater attention to safety and monitoring.

Growing Concerns Over AI Control

Concerns over autonomous AI systems have intensified as technology companies develop increasingly capable AI agents that can perform complex tasks with limited human intervention.

Researchers have warned that systems capable of independently carrying out research, writing code, accessing digital systems or taking other actions could create new forms of risk if adequate safeguards are not in place.

Hubinger’s warning does not mean that current AI systems are expected to cause human extinction. Rather, his concern focuses on a possible future stage of AI development in which systems become substantially more capable and potentially capable of improving themselves.

Industry Leaders Call for Greater Caution

Warnings about the potential existential risks of AI are not new. Several prominent researchers and technology leaders have previously called for stronger safeguards, international coordination and greater scrutiny of frontier AI development.

The latest warnings, however, have intensified as researchers inside leading AI companies increasingly debate whether safety measures are keeping pace with technological progress.

The debate is now shifting from whether AI can become extremely powerful to whether humans will be able to maintain meaningful control over systems that eventually surpass human capabilities in a wide range of tasks.

Hubinger’s assessment has added fresh urgency to that debate, while also highlighting the uncertainty surrounding predictions about the long-term risks of advanced AI.

Source: BBC