AI poses an existential threat to humanity

It is impossible to know exactly what will happen once an Advanced Autonomous Artificial Intelligence, the customized AI agent I refer to as AAAI, sets its own goals and begins to think far faster than the people who built it. No architecture, including this one, can guarantee th…

Once a system surpasses human intelligence across all domains, we cannot simply regulate how it is used. We will have lost the ability to control it at all. You cannot regulate the use of something more intelligent than the regulator just sector by sector.

If superintelligence were developed tomorrow, we would have no means of controlling it.

If a system more intelligent than any human were to act in ways at odds with human values—which is very likely—the consequences could be catastrophic.

And in the worst case, we indeed lose control altogether.

The world is locked in a deadly race towards an intelligence explosion, where AI’s ability to create better AIs reaches a critical point, just like a runaway nuclear chain reaction. Going slower would give us much-needed time to make it go well, but no individual actor is willing…

Self-improving artificial intelligence-systems that modify their own source code-promises exponential capability gains but also poses an existential risk: a single unchecked code modification could permanently remove all safety constraints.

The question is whether we will guide this power with wisdom before it begins to guide too much of us, or worse, before it reaches that tipping point where its capabilities and growth rate are beyond human comprehension or control, posing an existential threat to humanity.

We are extremely close to recursive self-improvement right now. The companies, I think, are planning to do this in the next 12 months. The asteroid is coming for Earth.

Safety progress takes time. If you don’t have enough time, at each stage of AI development, to ensure the AIs that you build in the next stage won’t destroy humanity, then you’ll fail, and humanity will be destroyed.

AI-related human existential risk arises not only when AI threatens the biological survival of humanity, but also when AI irreversibly weakens the civilizational conditions under which humanity can continue to exist as a subject of truth, responsibility, agency, and meaning.

Besides loss of control, the other catastrophic possibility is humans using AI to construct an eventually worldwide dictatorship. A small group of humans could concentrate all the power that AI will have, especially if we achieve AGI or superintelligence. And it would be much har…

The misuse or bad faith use of this technology may distort perceptions of reality to such an extent as to trigger global instability. Language and culture are the foundation of our civilization. If artificial intelligence causes us to lose control over them, the very essence of h…

Building on this perspective, the paper examines how dominant approaches to artificial intelligence governance, particularly those centered on labor displacement and income support, leave unaddressed the problem of declining human adaptability in highly automated environments, as…

We are likely on track to develop AI systems capable of causing human extinction/permanent disempowerment, quite possibly within the next few years.

Given the growing warnings from leading scientists, industry figures and Nobel laureates that advanced AI systems could pose existential risks on a par with nuclear or biological threats, does the Minister agree that the current reliance on voluntary commitments through the AI Se…

A moratorium and binding international regulation of ASI is, frankly, our only hope, however hard it will be to agree. It will be even harder to enforce, but we have to do it; there is no choice.

An off switch refers to building the necessary technical, legal, and institutional infrastructure to make it possible for humanity to shut down unsafe AI systems and AI projects. It will also make it possible to verify that there is no dangerous AI development globally, if humani…

This is not a ban or even a moratorium in the usual sense. It’s simply a proposal to require adequate safety measures for a technology that, according to its developers, has a significant chance to cause human extinction. Is that too much to ask?

Barring an international agreement, humanity will quite likely not have the ability to build safe superintelligence by the time the first superintelligence is built. Therefore, pursuing superintelligence at this stage is quite likely to cause the permanent disempowerment or extin…

There is nothing qualitatively different between the previous technological revolutions and this one, [...] It's just another set of tools that makes us more efficient.

The point is, AI systems, as smart as they might be, will be subservient to us. We set their goals, and they don't have any intrinsic goal that we would build into them to dominate. It would be really stupid to build that. It would also be useless. Nobody would buy it anyway.

I mean, basically, our relationship with future AI systems, including superintelligence, you know, superhuman system, is that we're going to be their boss.

The real challenge isn’t stopping an AI apocalypse — it’s crafting smart, sector-specific policies that keep technology aligned with human values. To avoid being a victim of AI, humans can, and should, put up focused guardrails.

By framing AI as a mysterious entity, an existential threat, a utopian savior, or a moral scapegoat, these narratives obscure the reality of AI as a statistical tool.

These fears (or hopes) of a nascent superintelligence that will spontaneously arise if we just give enough computing power and training data to large language models are absurd on their face.

Those extreme risks are still science fiction. So these speeches are largely diversions, very deliberately crafted.

In all the discussion around existential threat, nobody has ever given me a plausible scenario for how AI might be an existential risk.

There are a lot of kinds of dangers with AI. It can be used to produce disinformation and deepfakes. There are algorithmic biases, like the one I mentioned in the case of facial recognition. Hinton and others go further and say these systems could actually get out of control and …

One is that artificial general intelligence is still very far away. We’ve made a lot of progress in AI, but there’s far, far more still to go. It’s a very long journey. We’ve come a thousand miles, but there’s a million more to go. So, a lot of the talk that we hear as if AI is j…

In many ways, I think it’s a threat to democracy. That’s not a threat to humanity.

But it's not an existential threat, [...] It's just a horrible, awful weapon.

I don’t think we’re going too fast at all. In fact, I don’t think we’re going fast enough. If AI is going to do things like cure cancer, do we want to have the cure years from now or yesterday? What is the point of a six-month moratorium? To me, that letter is a piece of hysteria…

The development and proliferation of AI – far from a risk that we should fear – is a moral obligation that we have to ourselves, to our children, and to our future. We should be living in a much better world with AI, and now we can. [...] Rather than allowing ungrounded panics…

I'm much more focused on the benefits to all of us. I am haunted by the fact that the industrial revolution didn't touch the parts of the world where I grew up until much later. So I am looking for the thing that may be even bigger than the industrial revolution, and really doing…

So when people predict a high degree of existential risk from AGI, I don’t actually think “arguing back” on their chosen terms is the correct response. Radical agnosticism is the correct response, where all specific scenarios are pretty unlikely. Nonetheless I am still for people…

Much has been said about whether AI poses an ultimate existential threat to humanity. The more immediate existential challenge may be whether technologies that simulate relationships weaken our capacity for the relationships that foster our own humanity.