AI poses an existential threat to humanity

The Charitable Purpose of SAIF is to deliver project strategy which includes fundamental research with “Tool AI” systems in strategic cooperation with the mission of the rapidly developing Safe AI Community to achieve policy development and advocacy,...

And in the worst case, we indeed lose control altogether.

The misuse or bad faith use of this technology may distort perceptions of reality to such an extent as to trigger global instability. Language and culture are the foundation of our civilization. If artificial intelligence causes us to lose control ov...

Now, I don't quote that just to be revered as the prophet that I obviously am, but because my highest concern about artificial intelligence, both then and now, is that despite my consistent efforts, we haven't addressed the existential threat that it...

If superintelligence were developed tomorrow, we would have no means of controlling it.

The question is whether we will guide this power with wisdom before it begins to guide too much of us, or worse, before it reaches that tipping point where its capabilities and growth rate are beyond human comprehension or control, posing an existent...

If a system more intelligent than any human were to act in ways at odds with human values—which is very likely—the consequences could be catastrophic.

Given the growing warnings from leading scientists, industry figures and Nobel laureates that advanced AI systems could pose existential risks on a par with nuclear or biological threats, does the Minister agree that the current reliance on voluntary...

And by the way, there's the existential threat that as AI becomes smarter than humans, it can [...] become independent and lead to catastrophic impacts.

I distinguish a robust, systematic and possibly existential hazard to human wellbeing produced by AI advancement and proliferation.

As this glorious Earth is being celebrated while simultaneously being destroyed on the back end of it, I thought it would be appropriate not to worry about Iran, not to worry about climate change, but to worry about a third existential threat, which ...

We at the Futures Project think that there's a 70% chance of all humans dead or something similarly bad.

These loss of control incidents are concerning because they are the precursors to agents that could permanently evade human control and act adversarially in ways that we cannot detect or stop. This is why hundreds of leading scientists, business lead...

For governments to allow private entities to essentially play Russian roulette with every human being on earth is, in my view, a total dereliction of duty. [...] Each of the CEOs of the main AI companies, I believe, wants to disarm, Russell said, but...

This is our final invention. After that, humanity will not invent anything, and this is our last invention. So in such times, we need to think that what we are making should not go into the wrong hands.

I have gotten a lot more worried about existential, catastrophic AI risk — the kind of thing that would kill us instead of just rendering us comfortably impotent — than I was three years ago.

The real extinction-level AI safety challenge, the reason we’re nowhere close to surviving superintelligence, is something else — something AI companies decided they won’t mention anymore, because it exposes their AI safety efforts as a shockingly in...

The scale of the danger really could cripple civilization or cause extinction, and the possibility of this alone is reason enough to pursue pausing frontier AI development.

Once a system surpasses human intelligence across all domains, we cannot simply regulate how it is used. We will have lost the ability to control it at all. You cannot regulate the use of something more intelligent than the regulator just sector by s...

I urge the Minister to formally acknowledge extinction risk from superintelligent AI as a national security priority and to lead on international efforts to prohibit superintelligence development.

A moratorium and binding international regulation of ASI is, frankly, our only hope, however hard it will be to agree. It will be even harder to enforce, but we have to do it; there is no choice.

I entirely agree with his premise that there is real danger ahead of us if we do not take care and we do not understand what we are dealing with. [...] It is not often that we face choices so stark and so difficult, and where on the one hand there is...

Meanwhile, the UK’s AI Security Institute has warned: “In a worst-case scenario, this … could lead to catastrophic, irreversible loss of control over advanced AI systems”. The problem is that the companies developing superintelligence do not know the...

Even AI companies appear to agree with the scale of the risk. For example, the CEOs of OpenAI, Anthropic and Google DeepMind signed a statement that others have referred to today about the extinction risk posed by AI.

However, there are dangers. We would be mad to ignore them because many of the same people who built this technology—people who have won Nobel Prizes and Turing Awards—are warning that AI poses an extinction risk to humanity.

In the face of these threats, I urge the Government to take the following steps: first, to publicly recognise that superintelligence poses an extinction threat to humanity; secondly, for the UK to prevent the development of superintelligence on its s...

We are approaching a runaway to superintelligence that could threaten our shared human future. Both publicly and privately, AI companies are recognizing that a pause or slowdown in certain developmental pathways is crucial to protect lives and liveli...

An off switch refers to building the necessary technical, legal, and institutional infrastructure to make it possible for humanity to shut down unsafe AI systems and AI projects. It will also make it possible to verify that there is no dangerous AI d...

AI might literally lead to human extinction, in the same way that humans have caused many other species to go extinct.

We are likely on track to develop AI systems capable of causing human extinction/permanent disempowerment, quite possibly within the next few years.

We should be careful to construct in a way where we still continue to exist, and we can live in harmony with it. [...] Very few resources are being put into it. Maybe one percent.

Many people working in AI feel like they're living within a lie. Privately, they know that the current reckless pursuit of superintelligent AI poses an extinction threat to our species, but publicly they keep quiet so as to not rock the boat and risk...

Right now…the winner of any AI race between the U.S. and China is the AI. We need to be working to make sure that is not the case. I think it is very important that the U.S. AI sector remains ahead of the Chinese AI sector, but if that's at the expen...

If we build it using anything remotely like modern methods, on anything remotely like the current understanding or lack of understanding that we have about AI, then yeah, building it anytime soon would be a death sentence.

We are extremely close to recursive self-improvement right now. The companies, I think, are planning to do this in the next 12 months. The asteroid is coming for Earth.

Today, I finally feel the existential threat that AI is posing. When AI becomes overly good and disrupts everything, what will be left for humans to do? And it's when, not if.

Let's use this technology to benefit humankind, and let's regulate it to make sure they don't destroy humankind. I don't think that's a contradiction. I think that's common sense, and I think that's where most Americans are. [...] It's one thing if w...

I do think there’s still a danger of a Terminator-style apocalypse where you put AI together with weapons systems [...]

It poses an existential threat because it offers answers faster than humans can ask the questions that help them contemplate their existence.

Barring an international agreement, humanity will quite likely not have the ability to build safe superintelligence by the time the first superintelligence is built. Therefore, pursuing superintelligence at this stage is quite likely to cause the per...

If we create general superintelligences, I don’t see a good outcome long-term for humanity. So there is X-risk, existential risk, everyone’s dead.

If we create general superintelligences, I don’t see a good outcome long-term for humanity. So there is X-risk, existential risk, everyone’s dead.

The view that “mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war” is now mainstream, with that statement being endorsed by a who’s who of AI experts and though...

It was not powerful enough to destroy humanity. AI is. If religious wars resume [...] with AI and nuclear bombs, I don't think humanity would survive.

I guess one thing I could just say first is it’s not necessarily a 1-in-10 chance that we go extinct due to advanced AI; it was a 1-in-10 chance of existential catastrophe due to AI. I think it might have been Paul Christiano where I saw this laid ou...

the most likely result of building a superhumanly smart AI, under anything remotely like the current circumstances, is that literally everyone on Earth will die.

[...] it also has the potential to cause profound harm. [...] they could endanger the very existence of humanity. We must manage all these dangers [...].

I don’t get folk who say there is no existential risk from AGI. [...] There’s so many ways to wipe out humanity [...]

My concern with AI is, actually, existential, and existential risk is defined as many, many, many, many people harmed or killed.

My view is that autonomous weapons are the most dangerous existential threat that AI poses to humanity.

I fear that AI may replace humans altogether. If people design computer viruses, someone will design AI that replicates itself.

It could kill us all. [...] We don't want to have a Terminator outcome.

the greatest existential risks over the coming decades or century arise from certain, anticipated technological breakthroughs that we might make in particular, machine super intelligence,

I’m going to describe how the gains we make in artificial intelligence could ultimately destroy us.

If greater-than-human artificial general intelligence is invented without due caution, [...] the human species will be extinct in very short order.

Development of superhuman machine intelligence (SMI) [1] is probably the greatest threat to the continued existence of humanity. There are other threats that I think are more certain to happen (for example, an engineered virus with a long incubation ...

Once you start to make machines that are rivalling and surpassing humans with intelligence, it's going to be very difficult for us to survive.

How soon could such an intelligent robot be built? The coming advances in computing power seem to make it possible by 2030. And once an intelligent robot exists, it is only a small step to a robot species—to an intelligent robot that can make evolved...

AI could end humanity. We're working to prevent that.

Besides loss of control, the other catastrophic possibility is humans using AI to construct an eventually worldwide dictatorship. A small group of humans could concentrate all the power that AI will have, especially if we achieve AGI or superintellig...

If AI doesn't kill us, and there's no permanent dictatorship, and there's no permanent underclass, then we get a post-scarcity society, plus superintelligent AIs that we can set to working on other problems like disease and social decay.

In 2024 I would have said it's about 40-50% likely that LLMs scaled up to ASI would end up killing us all; now I would say that it's only about 5-8% likely even with no additional progress on alignment, and more like 1-2% likely simpliciter.

Once AI becomes powerful enough to be truly dangerous, it's maximizing the risk of an irreversible anti-human outcome that even you will deeply regret.

When AI researchers warn of extinction and tech leaders build doomsday bunkers, prudence demands we listen. Superintelligence without proper safeguards could be the ultimate expression of human hubris—power without moral restraint.

Within thirty years, we will have the technological means to create superhuman intelligence. Shortly after, the human era will be ended.

I distil two types of human extinction threat: System AI risk where AI is incorporated into powerful systems that are prone to catastrophic failure; and Agentic AI threat involving superintelligent AI evading human control and value-alignment.

[Palantir had effectively signalled a willingness] to add to nuclear Armageddon the AI-driven threat to humanity's existence.

Sadly, I now feel that we’re living the movie “Don’t look up” for another existential threat: unaligned superintelligence. [...] We need more time. [...] Just look up!

Most of the AI safety community has been far too coy about extinction risk. We're not. It's not that complicated: AI smarter than humanity poses an unacceptable risk of human extinction, and we need an [international ban].

Throughout my time here, I've repeatedly seen how hard it is to truly let our values govern our actions. I've seen this within myself, within the organization, where we constantly face pressures to set aside what matters most, and throughout broader ...

Suppose that frontier AI development is centralized to a single project under tight international controls, with all other development banned internationally. By far the likeliest outcome of this is that we all die. A centralized group of internatio...

Suppose that frontier AI development is centralized to a single project under tight international controls, with all other development banned internationally. By far the likeliest outcome of this is that we all die. A centralized group of internation...

It’s very frightening. [...] an AI system that doesn’t take over the world, that doesn’t destroy humanity, then [...] follow basic human laws.

First, we need some degree of coordination among the leading development efforts to ensure that the development of superintelligence occurs in a manner that allows us to both maintain safety and help smooth integration of these systems with society. ...

So when people predict a high degree of existential risk from AGI, I don’t actually think “arguing back” on their chosen terms is the correct response. Radical agnosticism is the correct response, where all specific scenarios are pretty unlikely. Non...

These fears (or hopes) of a nascent superintelligence that will spontaneously arise if we just give enough computing power and training data to large language models are absurd on their face.

There is nothing qualitatively different between the previous technological revolutions and this one, [...] It's just another set of tools that makes us more efficient.

The real challenge isn’t stopping an AI apocalypse — it’s crafting smart, sector-specific policies that keep technology aligned with human values. To avoid being a victim of AI, humans can, and should, put up focused guardrails.

Those extreme risks are still science fiction. So these speeches are largely diversions, very deliberately crafted.

AI has become the new climate change. It's this imminent catastrophe that is requiring all this government intervention. But there's very little evidence to support it.

We totally reject all efforts by international bodies to assert centralized control and global governance of AI. [...] Ideological fixations on social equity, climate catastrophism and so-called existential risk are dangers to progress and obstacles ...

Some tech leaders think AI could outsmart us and wipe out humanity. I’m a professor of AI – and I’m not worried.

It’s not the machines themselves. It’s the rhetoric of AI today [...] That’s the existential threat.

In all the discussion around existential threat, nobody has ever given me a plausible scenario for how AI might be an existential risk.

There are a lot of kinds of dangers with AI. It can be used to produce disinformation and deepfakes. There are algorithmic biases, like the one I mentioned in the case of facial recognition. Hinton and others go further and say these systems could ac...

I don’t think we’re going too fast at all. In fact, I don’t think we’re going fast enough. If AI is going to do things like cure cancer, do we want to have the cure years from now or yesterday? What is the point of a six-month moratorium? To me, that...

The development and proliferation of AI – far from a risk that we should fear – is a moral obligation that we have to ourselves, to our children, and to our future. We should be living in a much better world with AI, and now we can. [...] Rather t...

In many ways, I think it’s a threat to democracy. That’s not a threat to humanity.

AI isn't going to destroy mankind. [...] Machines are incredible to power things but it is only in fantasy that they are malignant.

I have to admit, I don’t get it, [...] I’m struggling to see how AI could pose any meaningful risk for our extinction.

But it's not an existential threat, [...] It's just a horrible, awful weapon.

AI poses many dangers to humanity but there is no existential threat or any evidence for one.

Over 1,300 technologists and leaders signed our open letter calling AI a force for good rather than an existential threat to humanity.

I'm much more focused on the benefits to all of us. I am haunted by the fact that the industrial revolution didn't touch the parts of the world where I grew up until much later. So I am looking for the thing that may be even bigger than the industria...

Despite this, it currently seems to be unlikely to become an existential threat to humanity.

I'm on record as being skeptical of that scenario and dubious about the value of putting a lot of effort into worrying about it now.

Using AI responsibly is our human responsibility. I’m not scared at all, AI is just a tool. But am i scared of people? Maybe.

AI is not a magic wand, but it is not a terminator. It’s not a harbinger of utopia or dystopia.

Using AI responsibly is our human responsibility. I’m not scared at all, AI is just a tool. But am i scared of people? Maybe.

With AI especially, I am really optimistic [...] naysayers [...] doomsday scenarios — I just, I don't understand it. I think it's really negative [...] it is pretty irresponsible.

Well, I’ll tell you anyway: personally, I’m not worried. I have been trying to build something capable of replacing me since the 1980s, because I consider that effort exciting and fascinating, and because I believe that it’s the natural next step for...

he does a good job of arguing against the hysteria that Artificial Intelligence is an existential threat to humanity.

The risks of self-improving intelligent machines are grossly exaggerated and ought not serve as a distraction from the existential risks we already face, [...]

AI is not going to exterminate us, it’s going to empower us, [...] AI has a huge potential benefit for humanity.

Does this mean that machines will replace us? I don't feel that it makes much sense to think in terms of "us" and "them." I much prefer the attitude of Hans Moravec of Carnegie-Mellon University, who suggests that we think of those future intelligent...

Scaring people by saying non-sensical things which are not going to happen like AI is an existential threat to humanity, is ridiculous.

The future utopia is ours to invent; the dystopia is ours to choose. [Pessimists paint a future with widespread job loss, social manipulation, erosion of human agency and existential threats from AI — but] these fears are largely unfounded, myopic, a...

I did not sign that letter for four reasons. First, it is useless, because it arrives late. [...] The second reason is that its foundations look more like what I consider science fiction: it relies on a generic description of artificial intelligence ...