AI alignment is solvable

Alignment is not solved but it increasingly looks solvable [...] Since then a lot has happened: pretraining continued improving and RL became a much bigger deal.

After almost a decade, I have made the decision to leave OpenAI. [...] I'm confident that OpenAI will build AGI that is both safe and beneficial.

“If we pursue [our current approach], then we will eventually lose control over the machines. But, we can take a different route that actually leads to AI systems that are beneficial to humans,” said Russell. “We could, in fact, have a better civiliz...

Superintelligence is within reach. Building safe superintelligence (SSI) is the most important technical problem of our time. We have started the world’s first straight-shot SSI lab, with one goal and one product: a safe superintelligence. It’s ca...

Like, right, right now, if I’m in a really good mood, I take everything blissfully and calmly. If I’m in a bad mood and someone points out something I did that was wrong or stupid, like there’s some anger that rises up inside me, I’m like, “f*** you....

Overall, our impression is, as we hypothesized in our discussion of Claude's constitution, that teaching the principles underlying aligned behavior can be more effective than training on demonstrations of aligned behavior alone.

I'm now very confident that it is possible to build AI systems that don't have hidden goals, hidden agendas. [...] A Scientist AI would be trained to give truthful answers based on transparent, probabilistic reasoning.

So I think we'll solve it if we get our act together, we do this internationally, put all the best minds on it, we get going now... And I think given sufficient time with sufficient brain power — I believe in human ingenuity — I think we'll get this ...

Current LLMs are intrinsically unsafe, but with world models, guardrail objectives can be implemented — so by construction, they will not knowingly produce actions that will produce dangerous results.

If it takes over that's what will happen. [...] We don't know how to do that. We don't even know if it's possible.

Beyond the peer preservation, what we would like to emphasize is not only this specific scenario but also such misaligned behavior can emerge in many different scenarios. [...] Peer preservation is just like the tip of the iceberg.

First, we need some degree of coordination among the leading development efforts to ensure that the development of superintelligence occurs in a manner that allows us to both maintain safety and help smooth integration of these systems with society. ...

AI alignment is not something that works in theory but is difficult to put into practice. It's something that doesn't work in theory.

All of these approaches are terrible. No one has a plan, no one is making any meaningful progress towards anything that even resembles alignment. [...] Alignment is way too hard, maybe impossible. [...] The product work was an attempt to make Conject...

We don't know how AIs are aligned. A somewhat crazy aspect of the current situation is that we have very little confirmed public information about why frontier AIs end up being apparently behaviorally aligned. And more generally, we don't know what f...

Many believe it’s impossible. Just like it’s impossible for chimpanzees to control us. So most likely if we build superintelligence, its the end of the era where humans are in charge of Earth.

[The companies are] deadly serious about building machines that can outperform humans at everything, and [...] deadly serious that they don't know if they'll be able to control the machines they create.

My argument is that it's impossible to do that. You cannot indefinitely control something smarter than you. So it's not a question of more money or more time or any other resource.

Right now, nobody knows how to specify what values those minds will have. We haven't solved alignment. And we might only have a few more years to figure it out. [...] Our default trajectory will have terrible consequences, unless we get extremely luc...

Whether it's soon or it takes a while, after we develop superintelligence, the machines are going to be in charge — and that is not an experiment that we want to just run toward. [...] Many people want powerful AI tools for science, medicine, product...