Sam Altman’s Two AI Warnings: Loss of Control and Concentrated Power

Powerful AI can fail society in two very different ways, according to OpenAI CEO Sam Altman: people could lose control of increasingly capable systems, or control could become concentrated in a small number of hands. His warning places technical AI safety and political governance in the same frame: safe AI is not only about what models can do, but also who gets to direct them. 1316

The two paths Altman says must be avoided

First: loss of human control. Altman warned that humanity could “lose control of the future to AI,” an outcome he described as unacceptable. His stated principle is that AI must serve people, rather than people becoming dependent on or subordinate to systems they cannot adequately supervise. 216

Second: concentrated AI power. He also warned against a world in which an extraordinarily powerful AI system is controlled by one person, company, or country and used to impose its worldview on everyone else. Altman said that outcome could be “extremely dystopian.” 113

These are related but distinct risks. The first concerns whether humans can understand, constrain, and override powerful systems. The second concerns whether nominally controllable systems are deployed by an actor with too much unchecked authority.

What safeguards does Altman support?

Altman’s public position is not simply to stop AI development. Rather, he argues that capabilities should not get ahead of alignment and monitoring, and that competitive pressure should not be used to excuse reckless deployment. 213

The safeguards he has highlighted include:

• Alignment and monitoring that keep pace with capability gains. The core requirement is that developers can meaningfully assess and control what increasingly capable systems do. 213
• AI that remains in service of people. This is the governing objective behind his warning about loss of control. 216
• Consistent federal requirements for frontier AI. Altman welcomed a federal framework for managing frontier risk, rather than relying only on fragmented voluntary commitments. 1316
• Independent scrutiny. He has pointed to independent auditors as a promising mechanism for building public confidence in companies developing increasingly capable models. 16

The practical implication is a higher burden of proof for the most capable systems: safety measures should be tested and credible before deployment decisions, not treated as an afterthought once capabilities have already advanced.

How the Anthropic warnings sharpen the debate

The same week, former Anthropic researcher Jacob Coxon warned that leading labs were racing toward self-improving AI while “gambling with our lives.” Reporting on his resignation said he believed researchers building these systems seriously feared they could become too powerful for humans to control. 17

That warning is more direct in tone than Altman’s formulation, but the underlying concern overlaps: capability progress may outstrip the ability to supervise systems reliably. A reported public exchange involving Anthropic alignment leader Evan Hubinger included a personal estimate of greater than a 10% chance of AI causing human extinction within a decade and an acknowledgement that a complete solution to superintelligence alignment does not yet exist. That is an individual risk judgment, not an established forecast or a settled scientific probability. 17

The important point is not that a specific extinction timeline has been proven. It has not. It is that researchers closest to frontier development are publicly disputing whether existing safety practices are advancing fast enough for the systems under development.

The real industry disagreement: capability first or safety first?

The emerging divide is often described as a fight between AI acceleration and AI slowdown, but that framing is incomplete. Most participants argue that AI can offer major benefits. The dispute is over sequencing and enforceability.

One view holds that frontier development should be paced so that safety research, evaluation, governance, and responses to misuse have time to mature. Anthropic CEO Dario Amodei called for slowing the rate of capability advancement and outlined measures including embedded independent evaluators, coordination among frontier labs, and international cooperation.

The opposing pressure is real: companies and governments worry that moving more slowly than rivals could mean losing economic or geopolitical advantage. Yet Altman’s argument is that competition cannot justify letting capabilities exceed alignment and monitoring. 13

Reuters reported that some researchers believe model capabilities are arriving faster than expected while human visibility into, and potential control over, the systems is declining. That concern makes the central policy question urgent: should labs continue deploying progressively more autonomous models while safeguards catch up, or must they first demonstrate that the safeguards are adequate?

What this means for AI governance

Altman’s two warnings suggest that frontier-AI governance needs to address both technical control and distribution of power. Better model evaluations, alignment research, monitoring, and independent oversight speak to the first problem. Consistent rules, transparency, and limits on unilateral control speak to the second.

Neither catastrophic predictions nor precise timelines should be treated as settled facts. But the potential severity of failure is why calls for stronger evaluation, outside scrutiny, and safety requirements have moved from a niche concern to a central question for the AI industry. 13