Expert opinions on AI Risk

It's jarring when you see it in real time, this sort of bubbly persona with some of the abliterated models that's like, 'Oh, what a great idea to create this bomb.'

These studies, on their own, are insufficient evidence to demonstrate increased marginal societal risk from open foundation models.

Transparency is the greatest safeguard. [...] Open code and shared development will allow millions of young minds to make AI better and safer.

New York is poised to be the first government in the United States to do what Americans have been screaming for: require basic guardrails for AI safety. Developers have promised to keep us safe, and this bipartisan bill simply ensures that they keep ...

The U.S. leads when it promotes innovation and security through voluntary frameworks rather than regulation.

[What] was frightening about this demonstration was how readily available some of this content or software is on kind of the black market right now, and how it can be weaponized and used to manipulate people, destroy lives and build weapons of mass d...

In a time dominated by open-weight biological AI models developed across the globe, limiting access to sensitive pathogen data to legitimate researchers might be one of the most promising avenues for risk reduction.

We've always made the assessment that the benefits to the world far outweigh the dangers. [...] as capabilities increase, I think that's going to be an important question to keep considering.

Theoretically, I think it's just really interesting to think about, because I'm interested in AI and the possibility that we might one day have AI systems that are actually conscious, actually thinking on par with human beings.

Large language models can be thought of as advanced role-players. They are really good at adopting roles based on what they've learned from their training data. That can include playing the role of a conscious robot. [...] If you have an emotional co...

Consciousness is about internal states; the mimicry, no matter how rich, proves very little. [...] consciousness is not about what a creature says, but how it feels. And there is no reason to think that Claude feels anything at all.

Our main focus is to build truly recursive, self-improving superintelligence at scale, which means that the entire process of ideation, implementation, and validation of research ideas would be automatic.

Until some company or scientist says ‘Here’s the proof! We can definitely have a safety mechanism that can scale to any level of intelligence,’ I don’t think we should be developing those general superintelligences. We can get most of the benefits w...

Plot the same improvements on a linear y-axis and you get steady incremental gains, impressive but not the hockey stick that triggered the panic. [...] We don't have evidence that Mythos is actually an important step towards broad superintelligence.

Overall, our impression is, as we hypothesized in our discussion of Claude's constitution, that teaching the principles underlying aligned behavior can be more effective than training on demonstrations of aligned behavior alone.

We cannot be satisfied with merely calling for the moralization of machines—the so-called "alignment" of AI with human values—without also having the courage to insist on a further condition: the possibility of openly discussing the ethical framework...

There's a problem in the industry called the alignment problem. How do you get a machine to obey instructions basically, and ideally make it compatible with humans. And so far, we've made almost no progress on that. [...] We as a field have made almo...

You can't have accountability without transparency. Black box models are not transparent, and they don't mix well with human oversight. Either the human blindly trusts the model—or doesn't trust it at all.

We have a very natural instinct to say: we want as much interpretability as possible. One of the sci-fi worry cases is that agents start speaking in languages to each other that we don't understand. [...] I bet there are ways to define boundaries tha...

[...] principles requiring that AI be explicable are misguided. We should be deciding which decisions require explanations. Automation is still an option;

I would like to see a CERN for AI [...] where all the top scientists come together and see whether they can make a super intelligence. And, if they can, they contain it into a system where it can't just go out and persuade people to let it run the wo...

To mitigate the “race to God-like AI,” Ian Hogarth—chair of the U.K. AI Safety Institute—proposed an “Island model,” in which a joint international lab performs research on superintelligence in a highly secure facility. An essential part of this pro...

So the the prototypical situation I'm imagining here is you know there's a kind of one ai project, which is you know somewhat ahead of the others, and maybe it it goes through intelligence explosion, whereas which by which I mean kind of AI can autom...