Comment by Geoffrey Irving

Yeah. I think the main uncertainty here is: are the models going to be good not just at verifiable tasks with clean rewards, but also fuzzier things — intuition, fuzzy planning, this kind of thing? I think people are overweighting the probability that they’re only good at the verifiable part. And if that is wrong, then we have seen so much progress over the last while that while the softer things lag, I don’t think they lag by years, say — they lag by a smaller amount of time, and that can carry us quite far in the next few years. And we’ve seen such rapid progress in the last couple of years that that could continue to go very quickly and keep speeding up. I think it could be slower, and I’m hoping it’s slower — that would be very nice — but that’s sort of the worry.