Comment by Marius Hobbhahn

Require independent embedded evaluations to close the internal deployment gap. Qualified independent third parties should test models throughout development and internal use, with employee-equivalent access.

The current regime is definitely not optimal. While third-party evaluators try hard to be unbiased, a lab can decide to stop working with them any day for any reason whatsoever. We really need more stringent regulations to improve the incentives for ...

I think it’s a good plan and I hope we see much more work on this, especially from the people who are deadset on building AGI.