Open-source AI is more dangerous than closed-source AI

Stripping guardrails from an open-source AI model is trivial for anyone who wants both.

Because open-weight safeguards can often be removed, public release of such models may make it impossible to prevent a wide range of actors from accessing dangerous capabilities.

In this paper, we show that open-weight safeguards are susceptible to simpler strategies that, despite being well known, have not been systematically evaluated against these safeguards. Specifically, we evaluate two low-cost attacks--abliteration and prefilling--that do not rely …

Compounding the problem, labs in China often release dual-use capable models as open-weight. Once a model is open-weight, safeguards that do exist can be removed, making the model available to any state or non-state actor to use for malicious purposes, including the cyber and CBR…

Whilst safeguards applied by responsible model developers can limit misuse of AI, these protections can often be bypassed (and in open‑weight models can be removed or are absent from the start). Publicly-available AI model weights can be modified, and safeguards removed entirely.

You can’t patch a released, open-weight model. This is the property that breaks the assumptions of the prior model of responsible disclosure. Open-source software can be patched in place; a deployer updates and the hazard closes. An open-weight model cannot. A new version is a ne…

High-capability open source models are difficult to control. Gated or retracted access is not possible once a model has been released into the open domain, leaving the possibility of malicious application.

Open-weight AI models have a sovereignty advantage in that sensitive data can remain local and the AI producer cannot revoke access. The flip side is that fine-tuning can also degrade or remove safeguards against misuse. Producers of open-weight AI models are unable to monitor mo…

When a frontier lab releases model weights publicly, the decision is irreversible in a way that no other software release is: the weights can be copied, fine-tuned, stripped of safety mitigations, and redistributed before any downstream harm is even detected.

Not deploying high-risk systems with open weights: Avoiding open-weight release of a system can be a strong defence against many downstream risks that they pose.

Don't be surprised if these capabilities are soon in the hands of ransomware crews, intelligence agencies and a "lone operator renting compute by the hour."

Because they can also be run more cheaply than proprietary models, the point at which open-weight models become dangerous and practical to use may arrive before they fully catch up with frontier models.

We then simulate the computational resources needed for an actor to launch a series of digital societal harm campaigns - such as disinformation botnets, sexual extortion schemes, voice-cloning fraud, and others - using low-compute open-source models and find nearly all studied ca…

An attacker can run it locally without safety guardrails, fine-tune it against their specific targets, and operate with zero visibility to any provider or defender.

Everybody can download and operate their own state-of-the-art model and use it for great things and terrible things.

Frontier labs can put classifiers in front of these capabilities, but open-weight models a few months behind cannot, and the copies already downloaded never will.

The proliferation in open-weight large language models has created a significant gap in AI governance. While proprietary closed-source models have incorporated extensive safety training to prevent harmful responses, open-weight systems can be readily downloaded and run locally wi…

Nefarious actors can use open models maliciously, [...] whereas developers of closed models have more means to identify and disrupt this malicious use.

Open-weight models – models of which the core parameters are made available for others to run and adapt – are an important medium-term issue. Evidence suggests they may lag the most advanced closed models by only four to eight months, and that their safeguards are often easy to r…

When models are open-weight, though, they are fundamentally more challenging to safeguard than closed models.

As policymakers and regulators work through questions of AI safety, the distinction that matters is not open versus closed, but transparent and secure versus opaque and unexamined. Open models and open tooling should be recognized as defensive assets that enable transparency, ind…

Open-source AI allows the public to audit and verify algorithms, [...] It is also more secure because any attacks or exploits can be identified and remediated.

We acknowledge it is worth monitoring the security implications of open source models that may reach frontier capabilities. But for the most part, the transparency that is inherent to open source makes them safer and more secure, because more engineers and researchers can tune ou…

But closing the ecosystem is not the answer. Openness is what lets a community find vulnerabilities, build safeguards, and improve models over time, rather than concentrating that work behind a few closed systems.

Open source does not eliminate risk. It changes how risk is managed. It allows more researchers, developers, and defenders to examine systems, test assumptions, surface weaknesses, and harden code under real-world conditions. [...] If frontier models are increasingly capable of f…

The alliance used its launch to urge regulators to treat open models as ‘defensive assets, not liabilities,’ warning that blanket restrictions would weaken cyber defenses and concentrate power in a handful of closed AI providers.

When a frontier model is attacking you and moving laterally inside your infrastructure, defenders need wide access to near-frontier tools within hours or even minutes, rather than being pointed toward a closed-door platform.

Open models, a valuable defense tool, are today being tuned, evaluated, and run in production to actively improve the security of enterprises.

Open development produces stronger defense.

Shared access to open-source AI systems is important for protection in a world where AI can be used to hack into electronic systems, like voting systems and benefits infrastructure.

One of the ways to achieve both individual empowerment and checks and balances is to make sure you put the technology in everybody's hands. I’m personally more worried about centralization than I am about any of the specific risks others are talking about.

I see the danger of this concentration of power through proprietary AI systems as a much bigger danger than everything else. What works against this is people who think that for reasons of security, we should keep AI systems under lock and key because it’s too dangerous to put it…

With the right choices, open weight AI can expand opportunity, strengthen competition, extend American technological leadership, mitigate risk, and ensure that the benefits of this extraordinary technology are shared broadly across our economy.

Bria is proud to sign on to “Open Weights and American AI Leadership,” a coalition statement on why open, inspectable models matter for a healthy, competitive AI ecosystem.

Open-source AI fosters innovation, improves security, and lowers costs—critical benefits for businesses, researchers, and everyday users around the world.

Open models, like any powerful technology, can be misused — including through attempts to weaken safeguards or repurpose capabilities for cyber attacks — but those risks are not unique to open systems, and they must be managed wherever advanced AI is deployed.

From a business perspective, the difference between open and closed is a little bit overblown. From a security perspective, the difference between open and closed models is, for some intents and purposes, overblown. The most important thing is how powerful a model is. If a model …

Study of 34 AI models from 10 global providers finds open-source AI models are not less safe, reasoning models are hardest to exploit and smaller models are the most vulnerable.

The researchers found that available evidence does not show that open models are riskier than closed models or information already available [...] like online searching.

China’s open models are no more dangerous than any other open source software a company may use.

The right way to do this is if a model, before it is released in open weights or in other ways, a model should be tested. Can it be used by people? Are we sure we’re not going to lose control of it? Does it have the capabilities or intentions that could be dangerous for people? I…

From a business perspective, the difference between open and closed is a little bit overblown. From a security perspective, the difference between open and closed models is, for some intents and purposes, overblown. The most important thing is how powerful a model is. If a model …

Not deploying high-risk systems with open weights: Avoiding open-weight release of a system can be a strong defence against many downstream risks that they pose.

Restricting access to a dual-use AI model is precautionary only if it delays harmful actors more than defenders. That condition varies across actors: a state agency or organized criminal group may obtain a substitute through theft, distillation, intermediated access, independent …

The governance of open-weight artificial intelligence (AI) models has been framed as a binary choice: openness as risk, restriction as safety. This paper challenges that framing, arguing that access restrictions, without governed alternatives, may displace risks rather than reduc…

It is unquestionably both. We can use the frontier models but also open weight models to prove a solution which gives defenders powerful capabilities and on their terms.