Require open source AI models to include safety benchmarks in release notes

Policy and safeguards must evolve just as swiftly [...] requiring safety assurances for powerful open releases before the window for meaningful intervention closes.

MCML is perfect for documenting open source models. Publish Model Cards and Dataset Cards alongside your model releases [...] Evaluation Card (benchmarking and testing)

[...] model cards support responsible AI development and the adoption of robust, industry-wide standards for broad transparency and evaluation practices

[W]e authored a paper addressing the rapidly evolving ecosystem of publicly available AI models [...] we proposed standardization of model cards and extensions to include safety, security, and data governance and pedigree information.

Class III (Open Model): [...] includes [...] evaluation results, a model card, and a data card.

platforms that host libraries of open-source AI models to ensure that models are uploaded with appropriate system and safety information

[T]he developer of an open source model could take a secure by design approach [...] train the model in a publicly verifiable way.

[T]he relevant authorities should encourage the use of open-source platforms that are open to inspection and robustly tested for safety.

Consider the potential risks of open-sourcing models. [...] publishing a transparency report (such as a model card) [...] risk assessment evaluation results.

Unsecured “open-source” AI systems pose a massive series of threats [...]. They deserve no exemptions and should be regulated just like other high-risk AI systems.

With the open source, there were all the assets for you to play with, [...] you could go about providing that testing.

deal with both open and closed-source models. [...] If independent testing were done and results on an intelligible scale were published

Open source provides an opportunity for many people to examine those technologies, to test them in a variety of settings and to provide fixes.

For pre-deployment risk assessments for open source, [...] models, [...] are released with a model card. Model cards [...] will generally come with a set of evaluations.

Disclose details such as testing methodologies, evaluation criteria, results, limitations, and gaps for any internal and external evaluations conducted prior to release.

This highlighted the importance of disclosing key details regarding dataset demographic make-up, performance metrics for different demographic groups, and specific mitigation efforts [...].

Select models where [...] performance on evaluation and red-teaming efforts are disclosed, over models that don’t – model cards can help with providing this information;

Congress should require standardized documentation [...] require model validation report [...] require all AI models, open source, or otherwise, that produce content

AI models consist of multiple layers: [...] Documentation, including evaluation results and model and data cards

Regulatory obligations imposed on open source AI providers [...] should also not lose sight of the AI Act’s objective to prevent harm.

providers of open-source GPAISR models will be subject to additional safety and security obligations prescribed in Article 55(1a-d). [...] conduct evaluations using established safety benchmarks.

For models with risk above the lowest level, most evaluation results and methods should be public, including any performed mitigations.

[...] platforms that host libraries of open-source AI models to ensure that models are uploaded with appropriate system and safety information

The organization publishes model cards for the AI models it produces, which (minimally) include the following elements: [...] (4) evaluation details (e.g., methodology, metrics, outcomes).

I do not believe this is the best approach to protecting the public from real threats posed by the technology.

Providers [...] shall [...] the results of its evaluation. The obligations [...] shall not apply to providers of AI models [...] a free and open-source licence [...].

There should be zero liability at the model layer. [...] Open source is the heart of software innovation and this bill slows it down.

The obligation set out in this Article shall not apply to providers of general-purpose AI models that are released under a free and open-source licence [...] unless the general-purpose AI models present systemic risks.

Meta states that it until such consensus is arrived, it will be premature to require specific measurements and benchmarks, [...].

I don't see any risk associated with open sourcing models. I only see benefits.

However, it is important to differentiate regulating applications (which we need) vs. regulating the technology (which is ill-advised).

The one-size-fits-all approach of the EU AI Act [...] creates significant and impractical barriers for providers of open-source foundation models.

The question that people are debating is whether it makes sense to regulate research and development of AI. And I don't think it does.

Regulating the open-source release of GPAI models is broadly unnecessary

Likely Not Required:Model card; Evaluation code; Validation datasets; Benchmarking dataset; All other data documentation