Expert opinions on AI Safety

OpenAI has long believed there should ultimately be an international organization that helps coordinate leading AI efforts to reduce catastrophic risk. One goal of such an organization should be to make it possible for the world to take coordinated a...

One of the most important ways to reduce the risks surrounding frontier AI could be to develop a trusted, effective system of verification...

The risk of AI using recursive learning to improve itself is only theoretical, and has never been shown to have occurred in the real world. [...] A freeze would certainly help Anthropic to maintain its leading position in B2B AI systems and perhaps e...

While we would applaud any serious effort to regulate potentially dangerous AI technologies, any attempt to tie the hands of states in their efforts to keep working people safe is not acceptable. We oppose preemption of state authority to regulate th...

Rather than allow protections to exist in only a handful of states or force innovators to navigate dozens of different legal regimes, our framework would establish one national standard.

Even if a company had a product that poses clear risks, they would still be free to release it, so long as they told the government the risks. [...] There is a lot of good stuff around guidance, but I hope that we ultimately see more that is binding,...

According to the proposed rule, a covered US person will be required to provide quarterly reports to BIS if the US person “engages in, or plans, within six months, to engage in ‘applicable activities.’” “Applicable activities” include: “Conducting a...

Establishing a pre-release review or licensing regime for AI companies would grant the government, particularly the executive branch, significant control over AI technologies that could hinder innovation or control expression. [...] Such an approach ...

AI has gone from "dumber than a child" to "expert level" in a few years in many domains. The gap between "expert level" and "above top geniuses" is smaller, so we expect it to take less time. This has been a pattern in fields like chess and Go, where...

Plot the same improvements on a linear y-axis and you get steady incremental gains, impressive but not the hockey stick that triggered the panic. [...] We don't have evidence that Mythos is actually an important step towards broad superintelligence.

If AI doesn't kill us, and there's no permanent dictatorship, and there's no permanent underclass, then we get a post-scarcity society, plus superintelligent AIs that we can set to working on other problems like disease and social decay.

So when people predict a high degree of existential risk from AGI, I don’t actually think “arguing back” on their chosen terms is the correct response. Radical agnosticism is the correct response, where all specific scenarios are pretty unlikely. Non...

AI has become the new climate change. It's this imminent catastrophe that is requiring all this government intervention. But there's very little evidence to support it.

SB 315 takes the safety practices leading labs already follow voluntarily—publishing a safety framework, transparent reporting, protecting whistleblowers—and helps establish a baseline that every leading AI developer is expected to meet. [...] Illino...

Connecticut already has strong whistleblower laws that protect employees from retaliation. [...] creating a separate, AI-specific whistleblower regime could lead to confusion, duplication, and unintended consequences for employers operating in highly...

Currently I just see elected legislatures quite captured by the tech industry and incapable or unwilling to regulate.

Yet if the federal government feels it has no say or no control, it will lunge and take over the whole thing. We thus want sustainable methods of perpetual interference that a) are actually somewhat useful from a safety perspective, and b) give gover...

New York is poised to be the first government in the United States to do what Americans have been screaming for: require basic guardrails for AI safety. Developers have promised to keep us safe, and this bipartisan bill simply ensures that they keep ...

A [central] problem with regulating frontier AI models is that a new feature emerges in these systems that is not tested, testable. [...] As long as you have this new emergent power, you have deep reasoning, deep capabilities, and they will make mist...

There is AI that is not responding to instructions, to commands, and there are documented cases of that. [...] If we don't consider ways to manage what we know is occurring already, and there is use of autonomous weapons already, if we are not lookin...

I come from the position of saying that moratoriums will not work. But we can work in co-operation with other nations that have already started regulating, such as South Korea and Australia, as well as work with our AI Security Institute in the Unite...

We cannot ignore the danger of a race to the bottom between competing tech companies or between states, rogue or otherwise. A moratorium and binding international regulation of ASI is, frankly, our only hope, however hard it will be to agree. It will...

[This is] a significant overreach of the U.N.'s mandate and competence. We will not cede authority over AI to international bodies that may be influenced by authoritarian regimes seeking to impose their vision of controlled surveillance societies.

If a country has powerful servers and chips made in Taiwan, but the model, the pipeline, and the governance model are still controlled with the values of Silicon Valley or of Beijing, then you have not democratized.

It is not permissible to entrust lethal or otherwise irreversible decisions to artificial systems. No algorithm can make war morally acceptable. (Magnifica humanitas, §198)

Personal view: I admire @AnthropicAI sticking by their principles but disagree with the principle itself. Putin won't fight fair so we should have autonomous AI weapons for sure.

There are some digital products (say CSAM and AI letting terrorists make bioweapons) that I oppose regardless of whether they are open-source or not, but I'm overall supportive of open source, and you can easily verify that my MIT research group defa...

Finally, to avoid that open-source models are accessed and retrofitted for malicious purposes a potential solution is to create self-destruct codes

AI is just an information processing and retrieval system akin to a search engine, which can be used in many ways. [...] The fact that criminals use them is a corollary of what AI models are: namely, tools.

The first thing that we've decided as Character.AI is that we will remove the ability for under-18 users to engage in any open-ended chats with AI on our platform.

Among the most concerning aspects of the bill is its lack of a parental consent option that would allow a child to use these products. [...] Different families may have different views on when a child should or shouldn't access any technology, and th...

Current LLMs are intrinsically unsafe, but with world models, guardrail objectives can be implemented — so by construction, they will not knowingly produce actions that will produce dangerous results.

Beyond the peer preservation, what we would like to emphasize is not only this specific scenario but also such misaligned behavior can emerge in many different scenarios. [...] Peer preservation is just like the tip of the iceberg.

AI alignment is not something that works in theory but is difficult to put into practice. It's something that doesn't work in theory.