Expert opinions on Existential Risk

If anyone, anywhere builds a superhuman artificial intelligence using present methods, the most likely outcome is catastrophe.

The question that the noble Lord posed was: do we therefore go for a moratorium? That would be highly desirable, but I do not think that it will be possible in the short term.

My own view, after talking with colleagues in the AI Security Institute and the Alan Turing Institute, is that a moratorium would be unenforceable.

We do this pretraining on human data, and then we get something that… understands human values fairly innately now.

If it takes over that's what will happen. [...] We don't know how to do that. We don't even know if it's possible.

What this proof shows is that there is no finite set of guardrails that is universally robust against adversarial prompts.

Capability testing must be the primary input for open-release decisions because open-weight mitigation remains unreliable, and labs and governments must identify models that are too dangerous for unrestricted release.

However, the solution here is absolutely not to restrict open model development in the name of security—cyber, national, or otherwise.

We at the Futures Project think that there's a 70% chance of all humans dead or something similarly bad.

So when people predict a high degree of existential risk from AGI, I don’t actually think “arguing back” on their chosen terms is the correct response. Radical agnosticism is the correct response, where all specific scenarios are pretty unlikely. Non...

Those extreme risks are still science fiction. So these speeches are largely diversions, very deliberately crafted.

There appear to be no obvious technical barriers to creating AI systems whose computational and architectural features could give rise to consciousness.

Because of how human nature has been expressed throughout our history and continues to express itself today, developing conscious machines with what for humans might be undetectable and incomprehensible minds is a monumentally dangerous idea.

There's no good reason to think AI is, or could ever be, conscious.

AI has gone from "dumber than a child" to "expert level" in a few years in many domains. The gap between "expert level" and "above top geniuses" is smaller, so we expect it to take less time. This has been a pattern in fields like chess and Go, where...

Throwing more words and GPUs into the word-guessing program won't make it sentient. [...] one of our mares gives birth to a locomotive.

I would like to see a CERN for AI [...] where all the top scientists come together and see whether they can make a super intelligence. And, if they can, they contain it into a system where it can't just go out and persuade people to let it run the wo...

Well, there are positive signs, for instance, like Henry Kissinger was recently suggested in foreign affairs that the US cooperate with China on this issue now, but before it's too late. So I think some people are recognizing the the importance of tr...

I call on the governments of the world to take immediate action to end this danger by permanently banning the development of artificial general intelligence and machine superintelligence.