Comment by Geoffrey Irving

I think that could work. I don’t think we have a strong argument that the pragmatic mixture of approaches will get all the way there, but it just seems very dicey, and our understanding of the dynamics involved is very weak.

Now. My take is that if we were to carefully analyse this question of exactly when we should slow down, it would be a while ago in the past, because we’re just too close to this crazy future. Trying to be super precise about exactly when in the futur...

Yeah. I think the main uncertainty here is: are the models going to be good not just at verifiable tasks with clean rewards, but also fuzzier things — intuition, fuzzy planning, this kind of thing? I think people are overweighting the probability tha...