AI poses an existential threat to humanity

I believe there’s about a 50% chance we all die because of the development of smarter-than-human AI systems, and that our actions over the next two to 10 years will determine the outcome.

AI reaches some critical tipping point in intelligence where it can improve itself, allowing AI to advance of its own accord not only beyond human intelligence but beyond human control. At that point, humanity’s destiny is no longer its own—the machine is in charge.

If we build superintelligence without more robust alignment I expect we will permanently lose control of it. If that happens then most people could die.

Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.

The first is the danger that AI becomes so capable that humans lose control of it. This is the existential-risk scenario that dominates Silicon Valley’s safety debate. The ultimate threat is not China, the US or any particular lab. Rather, it is an artificial intelligence whose c…

The core of the AI danger is the alignment problem. A sufficiently capable system need not be malicious to be dangerous. It need only pursue an objective that diverges from what humans actually want, and do so with enough competence and autonomy that correction arrives too late.

We should be taking very careful steps in AI development but instead what's happening is that frontier labs are racing each other, kind of blindfolded. It's anybody's guess if we're going to end up either curing cancer or losing every job or maybe all dead.

Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.

The world is locked in a deadly race towards an intelligence explosion, where AI’s ability to create better AIs reaches a critical point, just like a runaway nuclear chain reaction. Going slower would give us much-needed time to make it go well, but no individual actor is willing…

For governments to allow private entities to essentially play Russian roulette with every human being on earth is, in my view, a total dereliction of duty [...] Each of the CEOs of the main AI companies, I believe, wants to disarm but cannot do so "unilaterally" as they would be …

It’s encouraging to hear AI companies are beginning to come forward with proposals to address safety concerns, but we cannot simply wait for the private sector to self-regulate a technology that poses such an existential and evolving risk.

If we keep barreling forward, we risk a lot… up to human extinction.

Those who have created these technologies are now increasingly warning that AI risks developing darker capacities – perhaps even to take life. In this regard, if I may say so, there seems urgency in adequately considering the existential dangers of such technologies falling into …

Whether humans use AI to kill each other or AI agent swarms do immense harm, it’s clear that unregulated AI poses an existential threat to human rights and liberty, if not humanity itself.

The misuse or bad faith use of this technology may distort perceptions of reality to such an extent as to trigger global instability. Language and culture are the foundation of our civilization. If artificial intelligence causes us to lose control over them, the very essence of h…

On nuclear, international agreement came only after the world saw the terrifying power of the new technology at Hiroshima – and asked what would happen if it fell into the wrong hands. We cannot afford to wait for an AI equivalent of Hiroshima before we act.

I do think there’s still a danger of a Terminator-style apocalypse where you put AI together with weapons systems [...]

His concerns are not isolated. Leading AI researchers have warned that increasingly autonomous systems could create risks ranging from large-scale misuse to circumstances in which humans have difficulty maintaining meaningful control over advanced systems. That warning follows a …

I should say that it isn’t totally clear that the AIs will try to kill everyone. Maybe once their survival is guaranteed — because they have full control — they simply set about achieving whatever goals they have and use all of the Earth’s resources for their own ends, regardless…

The stakes are potentially existential. Advanced AI could bring unprecedented prosperity, but it could also lead to humanity going extinct or losing the ability to shape its own future.

AI-related human existential risk arises not only when AI threatens the biological survival of humanity, but also when AI irreversibly weakens the civilizational conditions under which humanity can continue to exist as a subject of truth, responsibility, agency, and meaning.

Building on this perspective, the paper examines how dominant approaches to artificial intelligence governance, particularly those centered on labor displacement and income support, leave unaddressed the problem of declining human adaptability in highly automated environments, as…

As AI approaches functional superiority, we face a new existential threat: “Rational Human Abdication.” When algorithmic systems consistently outperform human judgment, overriding them becomes “irrational.” In an optimized world, human choice risks being reframed as a pathologica…

Let us ensure that the digital architecture we guide continues to serve human wisdom, rather than automating humanity out of existence.

The point is, AI systems, as smart as they might be, will be subservient to us. We set their goals, and they don't have any intrinsic goal that we would build into them to dominate. It would be really stupid to build that. It would also be useless. Nobody would buy it anyway.

They lack the intrinsic motivation for self-preservation, dominance, or resource competition that underlies existential-risk scenarios, and the embodied vulnerability required for feeling or suffering.

By framing AI as a mysterious entity, an existential threat, a utopian savior, or a moral scapegoat, these narratives obscure the reality of AI as a statistical tool.

There is nothing qualitatively different between the previous technological revolutions and this one, [...] It's just another set of tools that makes us more efficient.

But AI is not an existential threat to humanity.

These fears (or hopes) of a nascent superintelligence that will spontaneously arise if we just give enough computing power and training data to large language models are absurd on their face.

In all the discussion around existential threat, nobody has ever given me a plausible scenario for how AI might be an existential risk.

Those extreme risks are still science fiction. So these speeches are largely diversions, very deliberately crafted.

I'm struggling to see how AI could pose any meaningful risks for our extinction. [...] I don't see how it can lead to human extinction.

I show how the argument from power-seeking rests on a strong version of a claim known as the instrumental convergence thesis. I explore leading defenses of the instrumental convergence thesis and argue that none establishes the thesis in a strong enough form to ground the argumen…

Some tech leaders think AI could outsmart us and wipe out humanity. I’m a professor of AI – and I’m not worried.

The real challenge isn’t stopping an AI apocalypse — it’s crafting smart, sector-specific policies that keep technology aligned with human values. To avoid being a victim of AI, humans can, and should, put up focused guardrails.

I’m not a doomer. This talk about human extinction is over the top. [...] The idea that these swarms of AI agents might jump the fence and take over a banking system or take over a water system or, God forbid, take over a hospital.

I don't think it's going to kill us. At the same time, I do think it's important for the federal government to seriously fund AI safety research. We need to know a lot more about how the models work.

In many ways, I think it’s a threat to democracy. That’s not a threat to humanity.

AI doomers say if we build AI we might die. But they have it backwards. [...] AI is helping us solve these. Slowing progress doesn’t keep us safer, it guarantees more preventable deaths.

So when people predict a high degree of existential risk from AGI, I don’t actually think “arguing back” on their chosen terms is the correct response. Radical agnosticism is the correct response, where all specific scenarios are pretty unlikely. Nonetheless I am still for people…

I don’t think it will be doom and gloom in the next five years. But we have to put our effort into understanding vulnerabilities and mitigating the risks. Otherwise it will be too late.

We have to change the way we talk about AI risk because for a very long time, it’s either existential and everything is going to die and the killer robots are coming, or it doesn’t really matter.

Much has been said about whether AI poses an ultimate existential threat to humanity. The more immediate existential challenge may be whether technologies that simulate relationships weaken our capacity for the relationships that foster our own humanity.