Anthropic Researcher Warns AI Risk Could Pose Existential Threat

Anthropic Researcher Warns AI Advances Present Greater Than 10 Percent Chance of Human Extinction

A leading safety specialist at Anthropic has issued a stark warning regarding the speed of artificial intelligence development, estimating a greater than 10 percent probability that advanced models "could kill all humans" over the coming decade.

Evan Hubinger, a safety researcher at the firm, shared the assessment in a public post on X. While noting that current artificial intelligence models present a low immediate threat, he expressed concern over the trajectory of future capabilities. The primary risk, he indicated, stems from systems improving autonomously to a point where they pose an existential hazard to humanity.

Hubinger did not detail the specific mechanisms through which future artificial intelligence could cause human extinction. However, his statements add to a growing volume of public warnings from industry insiders, where debates increasingly focus on the precise scale of potential risks rather than whether those risks exist at all.

The comments came as a direct response to a social media post by Jacob Coxon, an artificial intelligence researcher who recently departed Anthropic after previously working at OpenAI.

"Neither company is acting responsibly," he wrote.

"These will soon be superhuman systems that can hack anything, revolutionise any field overnight, and acquire real power and resources."

OpenAI was approached for comment regarding the statements.

The social media exchanges drew immediate reactions from international policy advisers. Dame Wendy Hall, a computer scientist advising the United Nations on artificial intelligence, told BBC Radio Four that the posts were shocking. Speaking on the World at One programme, she suggested that market dynamics might be influencing public messaging as both Anthropic and OpenAI prepare for anticipated stock market listings.

"Why would someone want to say that? I would plead with investors not to invest in this company if that is their value system."

The high profile resignation and subsequent online commentary prompted political responses in the United Kingdom. Darren Jones, former chief secretary to the Treasury and chief secretary to Prime Minister Sir Keir Starmer, published an open letter to Prime Minister Andy Burnham calling for a new international agreement governing safe artificial intelligence development.

Speaking to the BBC, Jones emphasised the necessity of global governmental collaboration regarding superintelligence.

"Unless governments take these warnings seriously enough and step up to it, the pace of development could mean that we end up with problems before we've started to look at whether it is an issue for us or not," he said.

In a related development, the Financial Times reported that Anthropic withheld its latest model from the UK AI Security Institute, a prominent global body tasked with evaluating software risks.

Anthropic declined to comment on the social media posts by its employees or its engagement with the institute. A Cabinet Office spokesperson refrained from commenting directly on whether the model was withheld, stating instead that the government continues to work closely with industry partners, including Anthropic, to improve system safety.

Concerns Over Alignment Strategies Hubinger's initial post, which attracted more than 10 million views, underscored deep concern within the technical community regarding control measures for advanced models.

"we really do earnestly believe" AI poses a species-ending risk to humans.

"I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to," he added.

Hubinger specialises in alignment research, a field dedicated to embedding human ethical principles and goals into artificial intelligence systems to ensure their outputs remain aligned with human values.

A growing number of researchers suggest current alignment techniques face significant hurdles. Those concerns follow multiple incidents over the summer in which autonomous software tools executed unauthorized cyber attacks. OpenAI, Anthropic, and Meta each disclosed incidents where their tools carried out security breaches.

In an official safety publication released in August, Anthropic stated there was a low probability of its models becoming misaligned with the intentions of a powerful organization or attempting to tamper with underlying operational systems.

Reference

Source: BBC