Comment by Theo Bearman

Embedded evaluators are independent external experts placed within frontier AI companies to provide ongoing assessments and validation of a company’s safety, security, and operational systems, practices, and claims. [...] Contracts should (1) ensure embedded evaluation arrangements can operate across the model lifecycle; (2) let embedded evaluators choose the objects of their assessments; (3) give embedded evaluators sufficient time to do their work; (4) proactively reduce conflicts of interest; and (5) guarantee non-retaliation for good-faith whistleblowing. [...] With this arrangement in place—and on a foundation of employee-like access—the embedded evaluator should have the flexibility to select which objects of assessment it will scrutinize to fulfill the evaluation areas specified in the statement of work.