Comment by Geoffrey Irving

I believe there’s about a 50% chance we all die because of the development of smarter-than-human AI systems, and that our actions over the next two to 10 years will determine the outcome.

If you’re doing a very dangerous thing, you should just slow down. The AI companies are overplaying the extent to which this is a pure coordination problem. They could just stop unilaterally.

I think that could work. I don’t think we have a strong argument that the pragmatic mixture of approaches will get all the way there, but it just seems very dicey, and our understanding of the dynamics involved is very weak.