AI alignment is solvable

I believe it’s an engineering problem ... and we all need to hope that’s an engineering problem. If it’s not an engineering problem, it’s not solvable.

Alignment is not as hard to solve as many claim, but it is in the end an algorithmic problem which a lot of ML community mostly stopped working on. Formulation of alignment stated by three (or four) laws of robotics can take us very far, so we roughly know the objective. The tric…

Current and near-future AI systems are tools, alignable through engineering.

Current LLMs are intrinsically unsafe, but with world models, guardrail objectives can be implemented — so by construction, they will not knowingly produce actions that will produce dangerous results.

I do think that what we have is adequate in the sense that with system prompting alone it seems possible to get over the hump of being able to trust recursive self improvement. And so automated alignment research, delegating the discovery of these techniques seems within reach th…

I believe that we can and must solve technical alignment, and that interpretability is the way there.

Alignment is supposed to be hard too, and it may well be. But the usual arguments assume we can’t see inside models. We assume we get one training setup, and it has to be right in advance. If we can read and understand the system, maybe we can make simple, careful changes instead…

If these ingredients work as hoped, the resulting technology would in principle let us describe the algorithms inside a model as it is trained, flag deceptive alignment and reward hacking, and train against those flags to produce an aligned system while paying a manageable alignm…

Understanding scheming is what makes alignment solvable.

Yea this is the most important result of the year, at the level of significance of “Attention Is All You Need”. It brings us from “is alignment even solvable” to “we may solve alignment in the next year.”

In the sense of, can we really teach the models what collective human values look like? I think the answer is clearly yes, and we have a big effort going on there.

We do this pretraining on human data, and then we get something that… understands human values fairly innately now.

Simply: Alignment is solvable because we can meaningfully load human values into LLMs and they can generalize them to a wide variety of situations.

Overall, our impression is, as we hypothesized in our discussion of Claude's constitution, that teaching the principles underlying aligned behavior can be more effective than training on demonstrations of aligned behavior alone.

Rather than only attempting to constrain AIs from the outside using confinement or reinforcement, Eigenism points toward "identity engineering," showing how deep, non-redundant shared histories can make human flourishing a genuine component of an AI's own rational self-interest.

Alignment by default (in any of the above senses) does not hold, but alignment is tractable enough that some combination of targeted technical research, non-default but plausible policy interventions, and elevated (though not unrealistic) safety concerns and altruism among AI lab…

As AI systems become more capable and more autonomous, alignment has to keep pace. The hardest problems won’t be solved by any one organisation working in isolation—we need independent teams testing different assumptions and approaches.

So I think we'll solve it if we get our act together, we do this internationally, put all the best minds on it, we get going now... And I think given sufficient time with sufficient brain power — I believe in human ingenuity — I think we'll get this right.

Yes, I certainly believe it's possible to do it if we go slow enough. [...] In this case, you need to go slow enough that you don't make any mistakes.

The alignment problem is solvable in principle. Whether it is solved in practice depends on whether the necessary resources, talent, and institutional commitment are mobilized before the systems that need to be aligned become too capable to correct.

It can and will be mitigated if the right research is done and proper precautions are taken and we are not racing at absurd speeds — better models will help solve alignment.

As I see it, solving alignment for LLMs isn’t just impossible, it’s incoherent: we can’t even articulate what it would look like to succeed.

The conventional view is that these concerns are about technology, and that if we take enough time then we can perfect or align the technology to produce a single benevolent superintelligence. I think this view of alignment is fundamentally flawed. Humanity is not a monoculture. …

First, alignment is not well defined, as it presumes a sufficiently stable and identifiable set of human intentions or values. Second, where a concrete notion of alignment can be specified, it is not solvable.

AI alignment is often framed as the task of ensuring that an AI system follows a set of stated principles or human preferences, but general principles rarely determine their own application in concrete cases.

If we can't align on these questions among humans, how do we encode them for AI systems that might work on recommendations to optimize across several human civilizations?

Solving alignment, at the limit the field cares about, means reliably controlling something far more intelligent than every human involved in building it. Not a calculator. Not a protein folder. A general mind that outthinks us across the board. We even named the target: superint…

My argument is that it's impossible to do that. You cannot indefinitely control something smarter than you. So it's not a question of more money or more time or any other resource.

Many believe it’s impossible. Just like it’s impossible for chimpanzees to control us. So most likely if we build superintelligence, its the end of the era where humans are in charge of Earth.

AI alignment is impossible. Not difficult — impossible.

Whether it's soon or it takes a while, after we develop superintelligence, the machines are going to be in charge — and that is not an experiment that we want to just run toward. [...] Many people want powerful AI tools for science, medicine, productivity, and other benefits. But…

We prove that no verification procedure can simultaneously satisfy three properties: soundness (no misaligned system is certified), generality (verification holds over the full input domain), and tractability (verification runs in polynomial time). Each pair of properties is achi…

What this proof shows is that there is no finite set of guardrails that is universally robust against adversarial prompts.

These results suggest that as models scale, their interpretation of prompts becomes more sophisticated but also less predictable, posing a significant challenge for robustly steering model behavior and highlighting the need for compositional and model-specific alignment technique…

A more fundamental fix would be to train the models not to cheat in the first place – but given this kind of behaviour was reported in frontier models more than a year ago, robustly aligning it away may not be easy.

Jailbreaking is a clear example where current systems do not converge on human values, even when those values are simple, abundant in training data, and extensively reinforced. This is real evidence against the claim that alignment is easy or natural.

None of these approaches that you'll often hear mentioned—interpretability, testing or alignment—is technically adequate. We don't know how to build superintelligent AI safely. The plan is basically to roll the dice.

Alignment cannot be “solved” through technical design alone, but must be managed through ongoing institutional processes that determine how objectives are set, how systems are evaluated, and how affected communities can contest or reshape those decisions.

Four of six dimensions show structural failures. Two of the four appear tractable to institutional design; the other two, the public reason problem under cognitive incomprehensibility and the non-domination problem under permanent capability asymmetry, call for new normative theo…

But alignment is no solution: it is an unsolved scientific and technical problem whose solutions—to the extent that we have them—cannot simply be imposed on every AI company operating on Earth.

If the alignment problem is hard (as I think it might well be), I expect that some restraint of this form will in fact be required in order for humanity to survive and remain empowered.

The alignment problem is not primarily a values problem. It is an existence problem.

The dominant question—the organizing concern of most safety work—is alignment: how do we get AI systems to pursue the right goals, embody the right values, behave within acceptable limits? This question is treated as foundational. Get alignment right and the hard part is solved. …

I used to always think that, you know, alignment will just fall out. I no longer believe that. So I think this alignment thing may be a harder, different problem.

How well we will solve alignment is basically a coin toss.

I think that could work. I don’t think we have a strong argument that the pragmatic mixture of approaches will get all the way there, but it just seems very dicey, and our understanding of the dynamics involved is very weak.

No one knows—it’s an open problem, but I will review some ideas and research directions.

Many AI researchers grant these concerns and recognize that the hard version of the alignment problem is unsolved; however, they generally believe that the better models of the future will help solve it. I fear we may already be near the point where models systematically bias the…

Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.