Expert opinions on Existential Risk

Suppose I say to a chatbot, I saw the Grand Canyon flying to Chicago. And the chatbot says, that can't be right. The Grand Canyon is much too big to fly to Chicago. And I say, no, no, no, it was me flying to Chicago. While I was flying to Chicago, I ...

Computational functionalism claims that executing certain computations is sufficient for consciousness, regardless of the physical mechanisms implementing those computations. This view neglects a compelling alternative: that subcomputational biologic...

A conscious being isn’t a word-guessing app that knows more words and has more computing power to guess with.

AI has gone from "dumber than a child" to "expert level" in a few years in many domains. The gap between "expert level" and "above top geniuses" is smaller, so we expect it to take less time. This has been a pattern in fields like chess and Go, where...

Throwing more words and GPUs into the word-guessing program won't make it sentient. [...] one of our mares gives birth to a locomotive.

And by the way, there's the existential threat that as AI becomes smarter than humans, it can [...] become independent and lead to catastrophic impacts.

So when people predict a high degree of existential risk from AGI, I don’t actually think “arguing back” on their chosen terms is the correct response. Radical agnosticism is the correct response, where all specific scenarios are pretty unlikely. Non...

These fears (or hopes) of a nascent superintelligence that will spontaneously arise if we just give enough computing power and training data to large language models are absurd on their face.

[...] And the whole world might have an interest in saying, hold on, wait -- let's not make artificial super intelligence until we can be sure that it can be controlled or done safely. [...] We can have all that while at the same time working togethe...

I come from the position of saying that moratoriums will not work. But we can work in co-operation with other nations that have already started regulating, such as South Korea and Australia, as well as work with our AI Security Institute in the Unite...

Alignment is not solved but it increasingly looks solvable [...] Since then a lot has happened: pretraining continued improving and RL became a much bigger deal.

If it takes over that's what will happen. [...] We don't know how to do that. We don't even know if it's possible.

AI alignment is not something that works in theory but is difficult to put into practice. It's something that doesn't work in theory.

I would like to see a CERN for AI [...] where all the top scientists come together and see whether they can make a super intelligence. And, if they can, they contain it into a system where it can't just go out and persuade people to let it run the wo...

Well, there are positive signs, for instance, like Henry Kissinger was recently suggested in foreign affairs that the US cooperate with China on this issue now, but before it's too late. So I think some people are recognizing the the importance of tr...

I call on the governments of the world to take immediate action to end this danger by permanently banning the development of artificial general intelligence and machine superintelligence.

When you open source a big model and give people the weights [...] You get people using that open source model and training it, refining it to do something bad like cyberattacks or making biological weapons because it's much easier to refine the mode...

there has to be full open source here. [...] if you're worried about AI-generated pathogens, [...] Let's do a Manhattan Project for biological defense.