Expert opinions on Existential Risk

Oh, it certainly could, yes. I think anybody who said that there's no way it'll lead to the extinction of humans just isn't facing reality.

AI has become the new climate change. It's this imminent catastrophe that is requiring all this government intervention. But there's very little evidence to support it.

[What] was frightening about this demonstration was how readily available some of this content or software is on kind of the black market right now, and how it can be weaponized and used to manipulate people, destroy lives and build weapons of mass d...

In a time dominated by open-weight biological AI models developed across the globe, limiting access to sensitive pathogen data to legitimate researchers might be one of the most promising avenues for risk reduction.

We've always made the assessment that the benefits to the world far outweigh the dangers. [...] as capabilities increase, I think that's going to be an important question to keep considering.

There is AI that is not responding to instructions, to commands, and there are documented cases of that. [...] If we don't consider ways to manage what we know is occurring already, and there is use of autonomous weapons already, if we are not lookin...

My own view, after talking with colleagues in the AI Security Institute and the Alan Turing Institute, is that a moratorium would be unenforceable. Instead, I support the proposal made this week by the noble Baroness, Lady Harding, to set up a commis...

Theoretically, I think it's just really interesting to think about, because I'm interested in AI and the possibility that we might one day have AI systems that are actually conscious, actually thinking on par with human beings.

Large language models can be thought of as advanced role-players. They are really good at adopting roles based on what they've learned from their training data. That can include playing the role of a conscious robot. [...] If you have an emotional co...

Consciousness is about internal states; the mimicry, no matter how rich, proves very little. [...] consciousness is not about what a creature says, but how it feels. And there is no reason to think that Claude feels anything at all.

Our main focus is to build truly recursive, self-improving superintelligence at scale, which means that the entire process of ideation, implementation, and validation of research ideas would be automatic.

Until some company or scientist says ‘Here’s the proof! We can definitely have a safety mechanism that can scale to any level of intelligence,’ I don’t think we should be developing those general superintelligences. We can get most of the benefits w...

Plot the same improvements on a linear y-axis and you get steady incremental gains, impressive but not the hockey stick that triggered the panic. [...] We don't have evidence that Mythos is actually an important step towards broad superintelligence.

Overall, our impression is, as we hypothesized in our discussion of Claude's constitution, that teaching the principles underlying aligned behavior can be more effective than training on demonstrations of aligned behavior alone.

We cannot be satisfied with merely calling for the moralization of machines—the so-called "alignment" of AI with human values—without also having the courage to insist on a further condition: the possibility of openly discussing the ethical framework...

There's a problem in the industry called the alignment problem. How do you get a machine to obey instructions basically, and ideally make it compatible with humans. And so far, we've made almost no progress on that. [...] We as a field have made almo...

I would like to see a CERN for AI [...] where all the top scientists come together and see whether they can make a super intelligence. And, if they can, they contain it into a system where it can't just go out and persuade people to let it run the wo...

To mitigate the “race to God-like AI,” Ian Hogarth—chair of the U.K. AI Safety Institute—proposed an “Island model,” in which a joint international lab performs research on superintelligence in a highly secure facility. An essential part of this pro...

So the the prototypical situation I'm imagining here is you know there's a kind of one ai project, which is you know somewhat ahead of the others, and maybe it it goes through intelligence explosion, whereas which by which I mean kind of AI can autom...