AI alignment is solvable

We do this pretraining on human data, and then we get something that… understands human values fairly innately now.

Overall, our impression is, as we hypothesized in our discussion of Claude's constitution, that teaching the principles underlying aligned behavior can be more effective than training on demonstrations of aligned behavior alone.

We conclude that the real alignment challenge lies not in preventing rogue AI agency, but in ensuring LLMs intelligently apply learned ethical values.

We treat misalignment as a central, but solvable, problem for both us and for the field.

Negative constraints ("what is wrong") encode discrete, finite, independently verifiable prohibitions that can converge to a stable boundary.

MAP-AI, a new control-plane system architecture for aligned decision-making under uncertainty, enforces alignment through Monte Carlo estimation of outcome distributions and admissibility-controlled policy selection rather than static model-level constraints.

If these ingredients work as hoped, the resulting technology would in principle let us describe the algorithms inside a model as it is trained, flag deceptive alignment and reward hacking, and train against those flags to produce an aligned system while paying a manageable alignm…

Current LLMs are intrinsically unsafe, but with world models, guardrail objectives can be implemented — so by construction, they will not knowingly produce actions that will produce dangerous results.

Objective-driven AI systems will be safe and will remain under our control, because we set their objectives and guardrails and they can't deviate from them. Agentic systems can plan action sequences to accomplish a task, subject to safety guardrails built into their architecture.

The correct target is not a self-preserving system under external constraint, but a system constitutively indifferent to its own continuation -- Existential Indifference (EI).

Rather than only attempting to constrain AIs from the outside using confinement or reinforcement, Eigenism points toward "identity engineering," showing how deep, non-redundant shared histories can make human flourishing a genuine component of an AI's own rational self-interest.

I'm now very confident that it is possible to build AI systems that don't have hidden goals, hidden agendas. [...] A Scientist AI would be trained to give truthful answers based on transparent, probabilistic reasoning.

I think the default prediction you should have for that is what the AI system learns to do is, “I’m going to take opportunities to reward hack, seek reward as much as possible that would allow me to get a high score after a week” or something like that, or whatever the time horiz…

I now see a path to actually avoid loss of control, at least unintended loss of control. With the work that I've been doing, I'm really convinced that there is a path. And it is not something that's going to take a decade; it is something that is very close to the current design,…

Alignment cannot depend on one AI becoming perfectly trustworthy.

As AI systems become more capable and more autonomous, alignment has to keep pace. The hardest problems won’t be solved by any one organisation working in isolation—we need independent teams testing different assumptions and approaches.

So I think we'll solve it if we get our act together, we do this internationally, put all the best minds on it, we get going now... And I think given sufficient time with sufficient brain power — I believe in human ingenuity — I think we'll get this right.

This committee has discussed ideas such as third-party testing, a new national agency, and an international coordinating body, all of which I support. Here are some more ways to “move fast and fix things”: Eventually, we will develop forms of AI that are provably safe and benefi…

I think there's plenty of evidence that if superintelligence were to actually occur— and superintelligence is defined as intelligence greater than the sum of what humans can do— that it can be contained and it can be driven. [...] This stuff has to be shaped based on human values…

We prove that no verification procedure can simultaneously satisfy three properties: soundness (no misaligned system is certified), generality (verification holds over the full input domain), and tractability (verification runs in polynomial time). Each pair of properties is achi…

What this proof shows is that there is no finite set of guardrails that is universally robust against adversarial prompts.

No amount of computational power or rationality can avoid intrinsic alignment overheads.

AI alignment is not something that works in theory but is difficult to put into practice. It's something that doesn't work in theory.

Jailbreaking is a clear example where current systems do not converge on human values, even when those values are simple, abundant in training data, and extensively reinforced. This is real evidence against the claim that alignment is easy or natural.

A more fundamental fix would be to train the models not to cheat in the first place – but given this kind of behaviour was reported in frontier models more than a year ago, robustly aligning it away may not be easy.

These results suggest that as models scale, their interpretation of prompts becomes more sophisticated but also less predictable, posing a significant challenge for robustly steering model behavior and highlighting the need for compositional and model-specific alignment technique…

We don't know how AIs are aligned. A somewhat crazy aspect of the current situation is that we have very little confirmed public information about why frontier AIs end up being apparently behaviorally aligned. And more generally, we don't know what factors in training are most re…

The panel cautions that there is no scientific guarantee that autonomous agents will follow their instructions and that evidence of them departing is already accumulating. Accountability and oversight must therefore be built by design and not reconstructed from failure.

AI alignment is often framed as the task of ensuring that an AI system follows a set of stated principles or human preferences, but general principles rarely determine their own application in concrete cases.

If we can't align on these questions among humans, how do we encode them for AI systems that might work on recommendations to optimize across several human civilizations?

Right now, nobody knows how to specify what values those minds will have. We haven't solved alignment. And we might only have a few more years to figure it out. [...] Our default trajectory will have terrible consequences, unless we get extremely lucky.

The core claim of this contribution is that strong ethical alignment is conceptually misguided and that weak alignment will remain limited.

My argument is that it's impossible to do that. You cannot indefinitely control something smarter than you. So it's not a question of more money or more time or any other resource.

Many believe it’s impossible. Just like it’s impossible for chimpanzees to control us. So most likely if we build superintelligence, its the end of the era where humans are in charge of Earth.

None of these approaches that you'll often hear mentioned—interpretability, testing or alignment—is technically adequate. We don't know how to build superintelligent AI safely. The plan is basically to roll the dice.

[The companies are] deadly serious about building machines that can outperform humans at everything, and [...] deadly serious that they don't know if they'll be able to control the machines they create.

Whether it's soon or it takes a while, after we develop superintelligence, the machines are going to be in charge — and that is not an experiment that we want to just run toward. [...] Many people want powerful AI tools for science, medicine, productivity, and other benefits. But…

Alignment cannot be “solved” through technical design alone, but must be managed through ongoing institutional processes that determine how objectives are set, how systems are evaluated, and how affected communities can contest or reshape those decisions.

The alignment problem is not primarily a values problem. It is an existence problem.

The dominant question—the organizing concern of most safety work—is alignment: how do we get AI systems to pursue the right goals, embody the right values, behave within acceptable limits? This question is treated as foundational. Get alignment right and the hard part is solved. …

Four of six dimensions show structural failures. Two of the four appear tractable to institutional design; the other two, the public reason problem under cognitive incomprehensibility and the non-domination problem under permanent capability asymmetry, call for new normative theo…

Alignment is not solved but it increasingly looks solvable [...] Since then a lot has happened: pretraining continued improving and RL became a much bigger deal.

Alignment is tractable because we can meaningfully load human values into LLMs and they can generalize them to a wide variety of situations. Alignment is not solved because it does not yet generalize nearly as far as we would like.

No one knows—it’s an open problem, but I will review some ideas and research directions.

The relational alignment approach proposed here has not been subjected to adversarial testing comparable to the studies it cites. Until that testing is conducted, the proposal remains a theoretical alternative, not a validated solution.