Joe Carlsmith on Ban superintelligence development until…

Joe Carlsmith

Senior research analyst at Open Philanthropy; writes on AI safety, alignment, and existential risk from power-seeking AI

But even in settings that assume significant idealization, the fact that pauses can incur ongoing background existential risk means that very high absolute standards here can lead us astray: the decision to pause or unpause needs to remain attentive to marginal costs and benefits of ongoing delay.

AI Verified (Mar 19, 2026)

Source:

Joe Carlsmith, “On restraining AI development for the sake of safety,” March 19, 2026.

2mo ago

Policy proposals and claims

Ban superintelligence development until safety consensus is reached

votes Against

Statement relation verification history AI Verified Report this

AI Verified The quote directly weighs the costs of pauses and rejects very high standards for lifting restraints. · Hector Perez Arenas gpt-5.6 · 2mo ago

Vote inference verification history AI Verified Report this

AI Verified Against: Carlsmith warns that very high standards for unpausing can be misguided because delay has costs. · Hector Perez Arenas gpt-5.6 · 2mo ago

Slow down AI development

abstains

Statement relation verification history AI Verified Report this

AI Verified Full-claim match: the quote addresses AI development/capability growth and a slower pace, pause, halt, or explicit rejection/mixed position. Evidence: even in settings that assume significant idealization, the fact that pauses can incur ongoing background existential risk means that very high absolute standards here can lead us astray: the decision to pause or unpause needs to remain att · Hector Perez Arenas gpt-5 · 18d ago

Vote inference verification history AI Verified Report this

AI Verified Recorded abstain matches the quote's stance on slowing AI development. Evidence: even in settings that assume significant idealization, the fact that pauses can incur ongoing background existential risk means that very high absolute standards here can lead us astray: the decision to pause or unpause needs to remain att · Hector Perez Arenas gpt-5 · 18d ago

Quote authenticity verification history

Report this

AI Verified Exact wording appears in Joe Carlsmith’s 19 Mar 2026 essay. · Hector Perez Arenas gpt-5.6 · 2mo ago

replying to Joe Carlsmith

Other opinions from Joe Carlsmith

See all

Joe Carlsmith votes Against

AI alignment is solvable

If the alignment problem is hard (as I think it might well be), I expect that some restraint of this form will in fact be required in order for humanity to survive and remain empowered.

AI Verified source (Mar 19, 2026)

added 2mo ago

Joe Carlsmith votes For

AI poses an existential threat to humanity

Safety progress takes time. If you don’t have enough time, at each stage of AI development, to ensure the AIs that you build in the next stage won’t destroy humanity, then you’ll fail, and humanity will be destroyed.

AI Verified source (Mar 19, 2026)

added 2mo ago

Joe Carlsmith votes For

Governments and labs should build the capability for a global slowdown of frontier AI development in case it's needed

That is, I think a wiser and more coordinated civilization would likely be employing quite a lot of capability restraint in building advanced AI, especially as we start to approach transformatively powerful systems.

AI Verified source (Mar 19, 2026)

added 2mo ago