ai-ethics open-source policy research-policy ai-governance regulations ai-safety open-science ai-regulation transparency

Share this with friends, journalists and politicians

Require open source AI models to include safety benchmarks in release notes

Key insights

Many supportive quotes treat model cards or similar transparency reports as the practical vehicle for publishing safety benchmarks and evaluation details.
Several proponents want disclosure paired with independent testing, verification, or third-party access, not only developer-produced summaries.
A major fault line is scope: some quotes say open and closed models alike need safety reporting, while one opposing institutional quote explicitly exempts free and open-source licensed models from an evaluation-results obligation.
Another fault line is timing: supporters call for pre-release evaluation and publication, whereas opponents say requiring specific benchmarks is premature until consensus develops.
Some supportive quotes favor a risk-based or standards-based trigger for public reporting, suggesting debate even among supporters over whether every open-source model should face the same requirement.

Arguments for

Transparency and accountability

Proponents say policy should demand transparency, allow independent safety evaluations, and publish results so developers are publicly accountable. They argue that when testing information is shared only at companies' discretion, governments and the public lack clear, up-to-date information about capabilities and risks.

We urgently need policies and institutions that will demand transparency, allow for independent safety evaluations, and enforce accountability. The well-being of the next generation is at stake.

Vivek Murthy · May 5, 2026 · full quote
As these tools become part of everyday life, it's important that they're designed to be safe, trustworthy, and appropriate for different stages of development. That's why independent evaluation and public accountability matter.

Wojciech Zaremba · May 5, 2026 · full quote
Results of testing for AI models’ capabilities and risks. A lack of clear, up-to-date information about what the world’s best AI systems are capable of puts the U.S. government at a huge disadvantage in understanding how the AI frontier is progressing and what kinds of responses …

Helen Toner · Apr 22, 2026 · full quote
It will establish safety standards, build open-source evaluations that AI developers can run against their models, independently test AI products, and publish the results to provide transparency and accountability.

Common Sense Media · May 5, 2026 · full quote
Now we need to open that with more open benchmarks, verification, independent testing, making sure that models reflect linguistic diversity, open science, and technical standards.

Rebecca Finlay · Jul 7, 2026 · full quote

Standardized model documentation

Supporters argue that open-source releases should be accompanied by standardized, machine-readable model documentation with safety, security, data-governance, and quantitative evaluation details. They present shared formats as a practical way to make disclosures comparable and usable.

The way to get there is a shared machine-readable format. I call it a Safety Framework Card.

Kanupriya Yakhmi · Jul 24, 2026 · full quote
The model card is restructured from prose README into a machine-readable structured document organized around three principles: (1) heritage transparency — full disclosure of training data, base model, and alignment lineage; (2) alignment provenance — what safety data, at which s…

Hoki Kim · May 7, 2026 · full quote
In November 2024, we authored a paper addressing the rapidly evolving ecosystem of publicly available AI models and their potential implications for security and safety. In this paper we proposed standardization of model cards and extensions to include safety, security, and data …

Huzaifa Sidhpurwala · Sep 3, 2025 · full quote
MCML is perfect for documenting open source models. Publish Model Cards and Dataset Cards alongside your model releases [...] Evaluation Card (benchmarking and testing)

MCML Standard · 2026 · full quote
AI models consist of multiple layers: [...] Documentation, including evaluation results and model and data cards

OECD.AI · full quote

Pre-release benchmark testing

Another line of support is mandatory pre-release evaluation against defined risk categories or uplift benchmarks, with summaries or results published before release. These quotes emphasize rigorous measurement, harmonized safety benchmarks, and evaluation methods that remain useful even if a model can tell it is being tested.

Developers should test their models against each of the four Enumerated Risk categories and publish a summary of the results.

Anthropic · Jun 2026 · full quote
A more tractable near-term proposal is mandatory pre-release evaluation against a defined set of uplift benchmarks, particularly for chemical and biological capability, with results published before weights ship.

Marisol Vega · Feb 19, 2026 · full quote
Researchers also consistently advocate for rigorous measurement and evaluation protocols to assess a model’s safety prior to publication. International bodies like the United Nations can play a critical role in coordinating global standards, ensuring that the drive for open innov…

Independent International Scientific Panel on Artificial Intelligence · Jul 2026 · full quote
Priorities include: evaluations that still work even when a model can determine if it is being tested and may therefore understate its harmful capabilities (“sandbagging”) or conceal misalignment (“alignment faking”), backed by secure access for independent third parties; assessi…

The 2026 Singapore Consensus on Global AI Safety Research Priorities · Jul 7, 2026 · full quote
Eclipse PanEval provides a unified, vendor-neutral framework to evaluate AI models for capability, safety, and cybersecurity in line with EU regulations.

Qigang Zhu · Mar 24, 2026 · full quote

Open models need safeguards

Some supporters argue that open-source or open-weight models should not be exempt from safety reporting, because policymakers must deal with both open and closed models and unsecured open systems can pose serious threats. In this view, publishing intelligible testing results and transparency reports can help prevent harm and encourage constructive competition.

Unsecured “open-source” AI systems pose a massive series of threats [...]. They deserve no exemptions and should be regulated just like other high-risk AI systems.

David Evan Harris · Dec 4, 2023 · full quote
policymakers and regulators have no choice but to deal with both open and closed-source models. [...] If independent testing were done and results on an intelligible scale were published, I think that would encourage constructive competition.

Professor John McDermid · Nov 8, 2023 · full quote
Consider the potential risks of open-sourcing models. [...] publishing a transparency report (such as a model card) [...] risk assessment evaluation results.

Department for Science, Innovation and Technology. · Oct 27, 2023 · full quote
Regulatory obligations imposed on open source AI providers [...] should also not lose sight of the AI Act’s objective to prevent harm.

Mozilla Foundation · full quote
providers of open-source GPAISR models will be subject to additional safety and security obligations prescribed in Article 55(1a-d). [...] conduct evaluations using established safety benchmarks.

Yacine Jernite · full quote

Arguments against

Technology-layer overreach

Opponents argue that open source is central to software innovation and that regulating AI research, development, or the model layer is ill-advised. From this perspective, rules and liability should focus on applications rather than the underlying model.

Open source is the heart of software innovation and this bill slows it down. [...] There should be zero liability at the model layer.

Anjney Midha · Jun 19, 2024 · full quote
However, it is important to differentiate regulating applications (which we need) vs. regulating the technology (which is ill-advised).

Andrew Ng · Dec 11, 2023 · full quote
The question that people are debating is whether it makes sense to regulate research and development of AI. And I don't think it does.

Yann LeCun · 2023 · full quote

Premature standards

Another objection is that specific benchmark requirements are premature before consensus exists on what should be measured. One institutional quote reflects this approach by exempting providers under a free and open-source licence from an obligation to provide evaluation results.

Until consensus develops, it is premature to require specific measurements and benchmarks.

Meta · Mar 26, 2024 · full quote
Providers [...] shall [...] the results of its evaluation. The obligations [...] shall not apply to providers of AI models [...] a free and open-source licence [...].

European Parliament and Council of the European Union · Jun 13, 2024 · full quote

Open release as benefit

A further skeptical view is that open-sourcing models is not itself associated with risk and is instead seen as broadly beneficial. This view undercuts the case for mandatory benchmark disclosures tied to release.

So I don't see any risk associated with open sourcing models. I only see benefits.

Arthur Mensch · Aug 4, 2024 · full quote
Regulating the open-source release of GPAI models is broadly unnecessary

Alex Engler · 2022 · full quote

Middle ground

Risk-tiered reporting

Some quotes support public evaluation disclosure mainly for models above the lowest risk level or once standard evaluations exist. That points to a graded approach in which reporting expands with risk and with the maturity of the benchmark itself.

For models with risk above the lowest level, most evaluation results and methods should be public, including any performed mitigations.

Jack Titus · Mar 5, 2024 · full quote
Develop and publish monitorability evaluations. Standardized evaluations and metrics can improve assessments for how monitorable an AI model’s reasoning process is. These tests would measure the clarity, coherence, and faithfulness of a model’s Chain of Thought, providing a concr…

Frontier Model Forum · Jan 27, 2026 · full quote

Across the quotes, support is strongest for pairing open-model releases with documented safety evaluations, especially through standardized model cards and with opportunities for independent review. Opposition centers on concerns that such mandates would regulate the technology layer, slow open-source innovation, or arrive before measurement standards are settled. The remaining disagreement is mainly about scope, timing, and whether open-source releases should be treated like other high-risk AI systems.

Cast Your Vote:

Results (48 votes):

Total (48 votes)

For 38 (79%) Abstain 0 (0%) Against 10 (21%)

New Top

For (38)

Kanupriya Yakhmi
PhD student at the University of the Cumberlands and author on AI safety documentation and governance.

I am this person! Add a sourced quote Report comment

votes For and says:

The way to get there is a shared machine-readable format. I call it a Safety Framework Card.
AI Verified source (Jul 24, 2026)

added 2mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Thinking Machines
AI model developer whose Inkling model card reports pre-release safety evaluations and benchmarks.

I am this person! Add a sourced quote Report comment

votes For and says:

Inkling reports reasoning, agentic, factuality, chat, vision, audio, and safety benchmark results at effort=0.99.
AI Verified source (Jul 14, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Rebecca Finlay
CEO of Partnership on AI.

I am this person! Add a sourced quote Report comment

votes For and says:

Now we need to open that with more open benchmarks, verification, independent testing, making sure that models reflect linguistic diversity, open science, and technical standards.
AI Verified source (Jul 7, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Independent International Scientific Panel on Artificial Intelligence
United Nations independent scientific panel that issued its July 2026 preliminary report on artificial intelligence.

I am this person! Add a sourced quote Report comment

votes For and says:

Researchers also consistently advocate for rigorous measurement and evaluation protocols to assess a model’s safety prior to publication. International bodies like the United Nations can play a critical role in coordinating global standards, ensuring...
more AI Verified source (Jul 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Google DeepMind
AI research company.

I am this person! Add a sourced quote Report comment

votes For and says:

DiffusionGemma underwent careful scrutiny, input data pre-processing, and post-training evaluations as reported in this card to help mitigate the risk of these biases.
AI Verified source (Jun 10, 2026)

added 1mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Anthropic
AI safety research company

I am this person! Add a sourced quote Report comment

votes For and says:

Developers should test their models against each of the four Enumerated Risk categories and publish a summary of the results.
AI Verified source (Jun 2026)

added 2mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Hoki Kim
Professor at Chung-Ang University researching AI governance and open-weight foundation-model policy.

I am this person! Add a sourced quote Report comment

votes For and says:

The model card is restructured from prose README into a machine-readable structured document organized around three principles: (1) heritage transparency — full disclosure of training data, base model, and alignment lineage; (2) alignment provenance ...
more AI Verified source (May 7, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Wojciech Zaremba
Head of AI Resilience at the OpenAI Foundation; OpenAI co-founder.

I am this person! Add a sourced quote Report comment

votes For and says:

As these tools become part of everyday life, it's important that they're designed to be safe, trustworthy, and appropriate for different stages of development. That's why independent evaluation and public accountability matter.
AI Verified source (May 5, 2026)

added 2mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Vivek Murthy
Former U.S. Surgeon General (19th and 21st); physician and public health advocate

I am this person! Add a sourced quote Report comment

votes For and says:

We urgently need policies and institutions that will demand transparency, allow for independent safety evaluations, and enforce accountability. The well-being of the next generation is at stake.
AI Verified source (May 5, 2026)

added 2mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Ellen Pack
Co-CEO of Common Sense Media.

I am this person! Add a sourced quote Report comment

votes For and says:

The Youth AI Safety Institute's role is to set a high bar: rigorous standards, independent testing, and transparent results that raise the bar for everyone.
AI Verified source (May 5, 2026)

added 2mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Common Sense Media
Kids media and tech nonprofit

I am this person! Add a sourced quote Report comment

votes For and says:

It will establish safety standards, build open-source evaluations that AI developers can run against their models, independently test AI products, and publish the results to provide transparency and accountability.
AI Verified source (May 5, 2026)

added 2mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Helen Toner
Interim Executive Director at Georgetown University's Center for Security and Emerging Technology (CSET); former OpenAI board member

I am this person! Add a sourced quote Report comment

votes For and says:

Results of testing for AI models’ capabilities and risks. A lack of clear, up-to-date information about what the world’s best AI systems are capable of puts the U.S. government at a huge disadvantage in understanding how the AI frontier is progressin...
more AI Verified source (Apr 22, 2026)

added 2mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Marisol Vega
Writer on AI governance and dual-use risks from open-weight model releases.

I am this person! Add a sourced quote Report comment

votes For and says:

A more tractable near-term proposal is mandatory pre-release evaluation against a defined set of uplift benchmarks, particularly for chemical and biological capability, with results published before weights ship.
AI Verified source (Feb 19, 2026)

added 2mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Frontier Model Forum
Frontier AI industry forum

I am this person! Add a sourced quote Report comment

votes For and says:

Develop and publish monitorability evaluations. Standardized evaluations and metrics can improve assessments for how monitorable an AI model’s reasoning process is. These tests would measure the clarity, coherence, and faithfulness of a model’s Chain...
more AI Verified source (Jan 27, 2026)

added 2mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
OpenAI
AI research organization

I am this person! Add a sourced quote Report comment

votes For and says:

In the US, we support state efforts to align around common frameworks such as California SB 53, the New York RAISE Act, and Illinois SB 315, which emphasize transparency, public reporting around catastrophic-risk evaluations and safety incidents, whi...
more AI Verified source (2026)

added 2mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Centre for Future Generations
European think tank on emerging technologies

I am this person! Add a sourced quote Report comment

votes For and says:

Policy and safeguards must evolve just as swiftly [...] requiring safety assurances for powerful open releases before the window for meaningful intervention closes.
AI Verified source (2026)

added 7mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Huzaifa Sidhpurwala
Red Hat security engineer

I am this person! Add a sourced quote Report comment

votes For and says:

In November 2024, we authored a paper addressing the rapidly evolving ecosystem of publicly available AI models and their potential implications for security and safety. In this paper we proposed standardization of model cards and extensions to inclu...
more AI Verified source (Sep 3, 2025)

added 8mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Vincent Caldeira
CTO of Red Hat in APAC

I am this person! Add a sourced quote Report comment

votes For and says:

Class III (Open Model): The entry point for openness. This involves the partial release of components which typically includes the model architecture, final model weights, and basic documentation, like a technical report, evaluation results, a model ...
more AI Verified source (Jul 15, 2025)

added 8mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Julia Smakman
Legal researcher, Ada Lovelace Institute

I am this person! Add a sourced quote Report comment

votes For and says:

platforms that host libraries of open-source AI models to ensure that models are uploaded with appropriate system and safety information
AI Verified source (2025) 1 of 2

added 8mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Cheryl Seah
Director, Drew & Napier LLC

I am this person! Add a sourced quote Report comment

votes For and says:

Select models where [...] performance on evaluation and red-teaming efforts are disclosed, over models that don’t – model cards can help with providing this information;
AI Verified source (Nov 2024)

added 8mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Jack Titus
AI policy writer at FAS

I am this person! Add a sourced quote Report comment

votes For and says:

For models with risk above the lowest level, most evaluation results and methods should be public, including any performed mitigations.
AI Verified source (Mar 5, 2024)

added 8mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
David Evan Harris
UC Berkeley public scholar, tech policy advisor

I am this person! Add a sourced quote Report comment

votes For and says:

Unsecured “open-source” AI systems pose a massive series of threats [...]. They deserve no exemptions and should be regulated just like other high-risk AI systems.
AI Verified source (Dec 4, 2023)

added 8mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Professor John McDermid
Professor of safety-critical systems

I am this person! Add a sourced quote Report comment

votes For and says:

policymakers and regulators have no choice but to deal with both open and closed-source models. [...] If independent testing were done and results on an intelligible scale were published, I think that would encourage constructive competition.
AI Verified source (Nov 8, 2023)

added 8mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Irene Solaiman
Hugging Face chief policy officer, researcher

I am this person! Add a sourced quote Report comment

votes For and says:

For pre-deployment risk assessments for open source, [...] models, [...] are released with a model card. Model cards [...] will generally come with a set of evaluations.
AI Verified source (Nov 8, 2023)

added 8mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Department for Science, Innovation and Technology.
UK government department

I am this person! Add a sourced quote Report comment

votes For and says:

Consider the potential risks of open-sourcing models. [...] publishing a transparency report (such as a model card) [...] risk assessment evaluation results.
AI Verified source (Oct 27, 2023)

added 8mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate

Loading more opinions...

Abstain (0)

Loading more opinions...

Against (8)

Mark Zuckerberg
Meta CEO; Facebook co-founder

I am this person! Add a sourced quote Report comment

votes Against and says:

Countries should not restrict open-weight AI models.
AI Verified source (Sep 2, 2026)

added 1mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Arthur Mensch
Mistral AI co-founder and CEO

I am this person! Add a sourced quote Report comment

votes Against and says:

So I don't see any risk associated with open sourcing models. I only see benefits.
AI Verified source (Aug 4, 2024)

added 8mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Anjney Midha
a16z general partner

I am this person! Add a sourced quote Report comment

votes Against and says:

Open source is the heart of software innovation and this bill slows it down. [...] There should be zero liability at the model layer.
AI Verified source (Jun 19, 2024)

added 8mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Meta
Social media and AI company

I am this person! Add a sourced quote Report comment

votes Against and says:

Until consensus develops, it is premature to require specific measurements and benchmarks.
AI Verified source (Mar 26, 2024)

added 8mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Andrew Ng
Baidu; Stanford CS faculty; founded Coursera and Google Brain

I am this person! Add a sourced quote Report comment

votes Against and says:

However, it is important to differentiate regulating applications (which we need) vs. regulating the technology (which is ill-advised).
AI Verified source (Dec 11, 2023)

added 7mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate

Loading more opinions...

More h/ai-safety votes

AI labs should give independent evaluators employee-like access

Democracies should share frontier AI models with allies

Grant whistleblower protections to AI safety researchers

If leading AI labs adopted a windfall clause (pre-committing to broadly share profits above a high threshold), it would meaningfully reduce competitive pressure to cut safety corners

Establish an international AI incident reporting system

Back