We should adopt AI 2040 Plan A

We recommend an international deal to avoid a dangerous race to superintelligence.

Plan A is the single most thorough and thoughtful plan that I’m aware of for what the world should do as we approach Superintelligence.

Plan A seems like a good plan for handling powerful AI, or at least the best plan anyone’s written up.

I think it’s the best summary of the spectrum of possible plans for preventing catastrophe from AI.

It is by no means perfect, but I’d love to see critics lay out a stronger competing vision; seems pretty hard.

AI 2040: Plan A is a compelling look at one path forward. To me, the key insight is the importance of layered defenses.

If you are looking for the path that gives a reasonable policy layout to slowdown the development of AI models. Then Plan A is it.

AI 2040: Plan A is a compelling look at one path forward. To me, the key insight is the importance of layered defenses.

I'm very impressed by the proposed Total Research Transparency. I actually found it appealing even beyond the many reasons mentioned in the plan.

Build Plan-A assets at Plan-D speed. The race is the default, so move like it is live: agents, swarms, shipping. But in every branch where things go well, the assets that survive are the ones opacity cannot fake: verified track record, provenance, distribution, judgment.

I disagree with the report massively. The biggest of it being that the AI race isn’t just a technology race. It’s a game theory problem.

The game theory doesn't work out this way. In two player models, the US and China can always each get a geopolitical power advantage from developing an AI more powerful than socially optimal, and so have an incentive to defect.

The document’s fundamental mistake is to confuse a shared interest with an incentive-compatible bargain.

The tech development seems ~plausible but the politics feel quite naive.

I struggle with what to say about the new AI 2040: Plan A website. It all seems so implausible to me that I’m not sure where to start.

But total research transparency is a burdensome requirement, with a huge surface area to be monitored. It dovetails with other features of AI 2040 – in particular, the premise of an international deal to govern data centers and AI training. But total research transparency is not …

Total research transparency feels a bit too galaxy-brained for me.

Given the level of political will and international coordination in the story, why can't they just dismantle the compute supply chain?

This is a recipe for authoritarianism.

We should not make big sacrifices of property rights and cede lots of power to new institutions unless there’s an extremely clear vision of the risks and solutions at hand.

I doubt their Plan A would work as written.

Personally, I’d want a far slower handover.

We should try to make AI as intelligent as possible but keep it away from controlling the effectors that can change the world. A person needs to sit between the AI and any effectors.

A “Plan A” that assumes all relevant events will be a strict consequence of AI development, deployment and user decisions will almost certainly be less useful than one that doesn’t.

The plan may be politically unrealistic. It may depend too heavily on compute remaining the main bottleneck. It may underestimate covert projects and overestimate international trust. But it is still a serious contribution because it makes the alternative to racing concrete.

I do think we should do that work, and give it, and its details, serious consideration.

I think it is a truly tremendous contribution to the state of AI policy and strategy. We really gotta figure out how, as humanity, we can realistically navigate the development of ASI. This isn't the best plan I've seen for that, but it is the best plan that I've seen that has be…

But my problem is that I feel great uncertainty and have no idea which of the two worlds (or some other third thing) we’re living in?