Comment by Jonas Freund

Third-party evaluators mostly test frontier AI models through an API. That misses the risks that come from how companies build and use AI internally. [...] Independent evaluators get employee-like access to a company’s internal systems, staff, and documentation. [...] What do we recommend? [...] Give evaluators the same access as employees doing similar risk assessments. Run them continuously rather than as one-off visits. [...] Bottom line: Embedded assessments are urgently needed. More external evaluators should offer the service. And more companies should host them.