Comment by Stella Biderman

The problem is, I don’t think this will do anything. Dario’s embedded evaluators are more like bank auditors (his analogy) or International Atomic Energy Agency inspectors (mine). They don’t tell people what to do; they watch and report on violations of rules. But in order for auditors like that to accomplish anything, they need to be backed by significant amounts of government power, and it’s hard to imagine that meaningfully happening. [...] Let’s go back in time and imagine that I’m an embedded evaluator at OpenAI starting the summer of last year. I’m given full access to everything and allowed to write any reports I want.