Comment by Ajeya Cotra

I think that what we want to do as a society, if we gain confidence that we’re at the starting point of this intelligence explosion, is to redirect as much of that AI labour as we can from further AI R&D to things that could help protect us from future generations of AIs — both in terms of AI takeover risk and also in terms of a wide range of other problems that might be created for society by increasingly powerful AI. [...] I do think that it’s important to not blow through that window, to monitor as we’re approaching it and to monitor how long we have. But I think I’m fundamentally fairly optimistic about trying to use early transformative AI systems, like early systems that automate a lot of things, to automate the process of controlling and aligning and managing risks from the next generation of systems, who then automate the process of managing those risks from the generation after, and so on.