Joe Carlsmith

If the alignment problem is hard (as I think it might well be), I expect that some restraint of this form will in fact be required in order for humanity to survive and remain empowered.

If our ability to make increasingly capable AI systems safe can’t benefit from such labor in a comparable way, then at least for relatively hard problem profiles, and absent large amounts of capability restraint, I think we’re likely headed for disas...