Comment by Evelyn Aswad

Anthropic’s October 15 announcement states evaluators should have “[e]nough access to meaningfully evaluate risks. Beyond access to the latest models, evaluators also need visibility into how those models were trained, into how our safeguards function, and more.” [...] Meaningful transparency is also needed about the contractual arrangements between the company under evaluation and the entity evaluating it for the public to fully understand, among other things, the extent of the evaluators’ access to information instead of being given an illustrative listing of exempted information. [...] It will be difficult for the public to accept the legitimacy of such audits without greater clarity on the material aspects of access to information in the potentially myriad bilateral legal arrangements that are coming down the pike.