Geoffrey Hinton Says a 10% AI Extinction Risk Is “Not Unreasonable”

Geoffrey Hinton, the AI pioneer and 2024 Nobel Physics laureate, said that a 10% chance of AI causing human extinction within the next decade did not seem unreasonable. But he paired that warning with a critical qualification: “nobody really knows how to give a sensible estimate.” 14

That distinction matters. Hinton was not presenting a scientifically calibrated forecast. He was saying that, given the potential scale of the harm and uncertainty over how capable future systems may become, dismissing the possibility outright would be unwarranted.

Why Hinton worries about AI-enabled harm

Hinton has drawn a line between two related problems: people using AI maliciously, and highly capable AI systems that may eventually be difficult to control.

On nearer-term misuse, he has warned that AI can intensify already familiar threats. In his Nobel lecture, Hinton cited divisive recommendation systems, mass surveillance and AI-assisted phishing, and warned that AI could eventually help create dangerous viruses and lethal weapons. He called for urgent attention from governments and international organizations.

The concern is not one isolated failure mode. Advanced systems could make misinformation more persuasive and individually tailored, help attackers identify software weaknesses, or lower barriers to biological or cyber misuse. In the most severe scenario envisioned by AI-safety advocates, a system with strong strategic, technical and operational capabilities could gain access to resources or critical infrastructure while resisting human attempts to restrain it. 5

The alignment problem behind the extinction argument

The central technical concern is alignment: whether a system much more capable than humans can reliably pursue objectives humans intend, remain corrigible, and be kept within meaningful limits.

Former Anthropic and OpenAI researcher Jacob Coxon resigned from Anthropic while arguing that leading labs were “racing straight to self-improving superintelligence” and “gambling with our lives.” 18 His warning focused on the possibility that systems could become able to improve their own capabilities faster than safety methods improve.

Evan Hubinger, Anthropic’s alignment-science lead, publicly supported the core warning. He said he personally believed there was a greater-than-10% chance that AI could kill all humans within the next decade, while adding that Anthropic did not yet have a plan to solve alignment for superintelligence and was not clearly on track to do so. 16

Their argument is not that today’s models are independently on the verge of exterminating humanity. Hubinger said the risk from existing models was low. The concern is a future transition: models becoming capable enough to improve themselves or take effective action in the world before developers know how to control them reliably. 116

Why skeptics reject the near-term extinction forecast

Other researchers do not deny that AI can make cybercrime, disinformation and biological misuse more dangerous. Their disagreement is over the jump from serious harm to human extinction.

Gary Marcus has said AI could contribute to cyberattacks, biological threats, disinformation and conflict escalation, but that he sees no realistic near-term route to literally “killing all humans.” 15 His position is that concrete risks deserve attention without assuming that present-day large language models are on a direct path to uncontrollable superintelligence.

Yann LeCun has also questioned high-confidence “p(doom)” estimates. Reporting on the debate characterized his view as placing AI-extinction risk far below the risk of nuclear holocaust, while emphasizing that such estimates are inherently speculative.

So the practical divide is not simply concerned versus unconcerned. It is about two unresolved questions:

1. Capability timelines: How soon, if ever, will systems become broadly superhuman and able to improve their own performance?
2. Control: If that happens, can developers build systems that remain reliably controllable, or will safeguards fail under pressure from more capable models and competitive deployment?

What a 10% estimate can—and cannot—tell us

There is no validated method for calculating a precise probability that AI will cause human extinction. Estimates such as Hinton’s and Hubinger’s are judgments under profound uncertainty, not probabilities derived from repeatable historical data. 14

That means the number should not be read as evidence that extinction is likely. Nor does the lack of a reliable number establish that the risk is negligible. The useful takeaway is narrower: experts disagree sharply about the likelihood and pathway, but many agree that powerful AI can create serious security and governance challenges well before any hypothetical extinction scenario.

The policy case: address capabilities before deployment

Hinton has called for urgent government and international attention to AI risks. Coxon likewise argued that regulation is central to reducing danger.

The policy conversation therefore centers on safeguards that can be applied to identifiable capabilities and deployments: rigorous evaluations of frontier models, stronger cybersecurity, biosecurity protections, reporting and oversight of serious incidents, and independent scrutiny before systems are given greater autonomy or access to sensitive tools.

The debate over a 10% extinction estimate will remain unsettled. Hinton’s point is not that anyone can calculate the odds with confidence. It is that the stakes are high enough—and the uncertainty large enough—that waiting for certainty could be the wrong standard for precaution. 14