Expert opinions on AI Risk

There's much more to do, including giving CAISI the authority to create the standards we need to know whether a frontier model is safe before it ships, not after something goes wrong.

However, it is important to differentiate regulating applications (which we need) vs. regulating the technology (which is ill-advised).

Suppose I say to a chatbot, I saw the Grand Canyon flying to Chicago. And the chatbot says, that can't be right. The Grand Canyon is much too big to fly to Chicago. And I say, no, no, no, it was me flying to Chicago. While I was flying to Chicago, I ...

Computational functionalism claims that executing certain computations is sufficient for consciousness, regardless of the physical mechanisms implementing those computations. This view neglects a compelling alternative: that subcomputational biologic...

A conscious being isn’t a word-guessing app that knows more words and has more computing power to guess with.

AI has gone from "dumber than a child" to "expert level" in a few years in many domains. The gap between "expert level" and "above top geniuses" is smaller, so we expect it to take less time. This has been a pattern in fields like chess and Go, where...

Throwing more words and GPUs into the word-guessing program won't make it sentient. [...] one of our mares gives birth to a locomotive.

Alignment is not solved but it increasingly looks solvable [...] Since then a lot has happened: pretraining continued improving and RL became a much bigger deal.

If it takes over that's what will happen. [...] We don't know how to do that. We don't even know if it's possible.

AI alignment is not something that works in theory but is difficult to put into practice. It's something that doesn't work in theory.

Nefarious actors can use open models maliciously, [...] whereas developers of closed models have more means to identify and disrupt this malicious use.

These studies, on their own, are insufficient evidence to demonstrate increased marginal societal risk from open foundation models.

Transparency is the greatest safeguard. [...] When codes are open and shared, millions of young minds can make them safer and better.

I would like to see a CERN for AI [...] where all the top scientists come together and see whether they can make a super intelligence. And, if they can, they contain it into a system where it can't just go out and persuade people to let it run the wo...

Well, there are positive signs, for instance, like Henry Kissinger was recently suggested in foreign affairs that the US cooperate with China on this issue now, but before it's too late. So I think some people are recognizing the the importance of tr...

I call on the governments of the world to take immediate action to end this danger by permanently banning the development of artificial general intelligence and machine superintelligence.

“If we pursue [our current approach], then we will eventually lose control over the machines. But, we can take a different route that actually leads to AI systems that are beneficial to humans,” said Russell. “We could, in fact, have a better civiliz...

We have a very natural instinct to say: we want as much interpretability as possible. One of the sci-fi worry cases is that agents start speaking in languages to each other that we don't understand. [...] I bet there are ways to define boundaries tha...

[...] principles requiring that AI be explicable are misguided. We should be deciding which decisions require explanations. Automation is still an option;

When you open source a big model and give people the weights [...] You get people using that open source model and training it, refining it to do something bad like cyberattacks or making biological weapons because it's much easier to refine the mode...

there has to be full open source here. [...] if you're worried about AI-generated pathogens, [...] Let's do a Manhattan Project for biological defense.