AI poses an existential threat to humanity

Once a system surpasses human intelligence across all domains, we cannot simply regulate how it is used. We will have lost the ability to control it at all. You cannot regulate the use of something more intelligent than the regulator just sector by sector.

It is impossible to know exactly what will happen once an Advanced Autonomous Artificial Intelligence, the customized AI agent I refer to as AAAI, sets its own goals and begins to think far faster than the people who built it. No architecture, including this one, can guarantee th…

If superintelligence were developed tomorrow, we would have no means of controlling it.

Safety progress takes time. If you don’t have enough time, at each stage of AI development, to ensure the AIs that you build in the next stage won’t destroy humanity, then you’ll fail, and humanity will be destroyed.

We are likely on track to develop AI systems capable of causing human extinction/permanent disempowerment, quite possibly within the next few years.

The world is locked in a deadly race towards an intelligence explosion, where AI’s ability to create better AIs reaches a critical point, just like a runaway nuclear chain reaction. Going slower would give us much-needed time to make it go well, but no individual actor is willing…

Self-improving artificial intelligence-systems that modify their own source code-promises exponential capability gains but also poses an existential risk: a single unchecked code modification could permanently remove all safety constraints.

The question is whether we will guide this power with wisdom before it begins to guide too much of us, or worse, before it reaches that tipping point where its capabilities and growth rate are beyond human comprehension or control, posing an existential threat to humanity.

We are extremely close to recursive self-improvement right now. The companies, I think, are planning to do this in the next 12 months. The asteroid is coming for Earth.

Even after 3 years working on AI capability evaluations, the recent pace of progress has been a shock. To avoid catastrophic harms from AGI, we need much, much better coordination.

For governments to allow private entities to essentially play Russian roulette with every human being on earth is, in my view, a total dereliction of duty [...] Each of the CEOs of the main AI companies, I believe, wants to disarm but cannot do so "unilaterally" as they would be …

First, we need some degree of coordination among the leading development efforts to ensure that the development of superintelligence occurs in a manner that allows us to both maintain safety and help smooth integration of these systems with society. There are many ways this could…

We are approaching a runaway to superintelligence that could threaten our shared human future. Both publicly and privately, AI companies are recognizing that a pause or slowdown in certain developmental pathways is crucial to protect lives and livelihoods everywhere.

A moratorium and binding international regulation of ASI is, frankly, our only hope, however hard it will be to agree. It will be even harder to enforce, but we have to do it; there is no choice.

Even after 3 years working on AI capability evaluations, the recent pace of progress has been a shock. To avoid catastrophic harms from AGI, we need much, much better coordination.

AI-related human existential risk arises not only when AI threatens the biological survival of humanity, but also when AI irreversibly weakens the civilizational conditions under which humanity can continue to exist as a subject of truth, responsibility, agency, and meaning.

Building on this perspective, the paper examines how dominant approaches to artificial intelligence governance, particularly those centered on labor displacement and income support, leave unaddressed the problem of declining human adaptability in highly automated environments, as…

The misuse or bad faith use of this technology may distort perceptions of reality to such an extent as to trigger global instability. Language and culture are the foundation of our civilization. If artificial intelligence causes us to lose control over them, the very essence of h…

As AI approaches functional superiority, we face a new existential threat: “Rational Human Abdication.” When algorithmic systems consistently outperform human judgment, overriding them becomes “irrational.” In an optimized world, human choice risks being reframed as a pathologica…

The point is, AI systems, as smart as they might be, will be subservient to us. We set their goals, and they don't have any intrinsic goal that we would build into them to dominate. It would be really stupid to build that. It would also be useless. Nobody would buy it anyway.

I mean, basically, our relationship with future AI systems, including superintelligence, you know, superhuman system, is that we're going to be their boss.

There is nothing qualitatively different between the previous technological revolutions and this one, [...] It's just another set of tools that makes us more efficient.

By framing AI as a mysterious entity, an existential threat, a utopian savior, or a moral scapegoat, these narratives obscure the reality of AI as a statistical tool.

Using AI responsibly is our human responsibility. I’m not scared at all, AI is just a tool. But am i scared of people? Maybe.

These fears (or hopes) of a nascent superintelligence that will spontaneously arise if we just give enough computing power and training data to large language models are absurd on their face.

Those extreme risks are still science fiction. So these speeches are largely diversions, very deliberately crafted.

In all the discussion around existential threat, nobody has ever given me a plausible scenario for how AI might be an existential risk.

There are a lot of kinds of dangers with AI. It can be used to produce disinformation and deepfakes. There are algorithmic biases, like the one I mentioned in the case of facial recognition. Hinton and others go further and say these systems could actually get out of control and …

I'm struggling to see how AI could pose any meaningful risks for our extinction. [...] I don't see how it can lead to human extinction.

The real challenge isn’t stopping an AI apocalypse — it’s crafting smart, sector-specific policies that keep technology aligned with human values. To avoid being a victim of AI, humans can, and should, put up focused guardrails.

In many ways, I think it’s a threat to democracy. That’s not a threat to humanity.

But it's not an existential threat, [...] It's just a horrible, awful weapon.

AI poses many dangers to humanity but there is no existential threat or any evidence for one.

It’s not the machines themselves. It’s the rhetoric of AI today [...] That’s the existential threat.

I don’t think we’re going too fast at all. In fact, I don’t think we’re going fast enough. If AI is going to do things like cure cancer, do we want to have the cure years from now or yesterday? What is the point of a six-month moratorium? To me, that letter is a piece of hysteria…

The development and proliferation of AI – far from a risk that we should fear – is a moral obligation that we have to ourselves, to our children, and to our future. We should be living in a much better world with AI, and now we can. [...] Rather than allowing ungrounded panics…

AI has become the new climate change. It's this imminent catastrophe that is requiring all this government intervention. But there's very little evidence to support it.

We totally reject all efforts by international bodies to assert centralized control and global governance of AI. [...] Ideological fixations on social equity, climate catastrophism and so-called existential risk are dangers to progress and obstacles to responsibly harnessing this…

I'm much more focused on the benefits to all of us. I am haunted by the fact that the industrial revolution didn't touch the parts of the world where I grew up until much later. So I am looking for the thing that may be even bigger than the industrial revolution, and really doing…

So when people predict a high degree of existential risk from AGI, I don’t actually think “arguing back” on their chosen terms is the correct response. Radical agnosticism is the correct response, where all specific scenarios are pretty unlikely. Nonetheless I am still for people…

I don’t think it will be doom and gloom in the next five years. But we have to put our effort into understanding vulnerabilities and mitigating the risks. Otherwise it will be too late.

Much has been said about whether AI poses an ultimate existential threat to humanity. The more immediate existential challenge may be whether technologies that simulate relationships weaken our capacity for the relationships that foster our own humanity.