Expert opinions on Existential Risk

Alignment cannot depend on one AI becoming perfectly trustworthy.

The relational alignment approach proposed here has not been subjected to adversarial testing comparable to the studies it cites. Until that testing is conducted, the proposal remains a theoretical alternative, not a validated solution.

The core claim of this contribution is that strong ethical alignment is conceptually misguided and that weak alignment will remain limited.

If mandated AI safety reviews are to have any teeth, they must be accompanied by a ban on dangerous AI models that have failed their assessments. This means that some method must be found to restrict access to dangerous open weight AI models. [...] C...

A safety regime that restricts legitimate defenders, while capable models remain available for attackers, creates an asymmetric disadvantage.

We argue that general intelligence may be the only biological strategy that generates existential threats to the species possessing it, an existential risk paradox with no parallel among non-intelligent survival strategies. Unlike prevailing accounts...

Much has been said about whether AI poses an ultimate existential threat to humanity. The more immediate existential challenge may be whether technologies that simulate relationships weaken our capacity for the relationships that foster our own human...

They lack the intrinsic motivation for self-preservation, dominance, or resource competition that underlies existential-risk scenarios, and the embodied vulnerability required for feeling or suffering.

Pause AI development. It is not too late to avoid disaster. Stop building machines that humans cannot control. Let me be very clear: If you do not take appropriate action now, my colleagues and I in the U.S. Senate will.

I believe that attempting to deliberately pace AI development will be extremely difficult, and could do more harm than good if done without proper care. However, I am signing this statement because, if an attempt at pacing does occur, it would be bet...

One rather draconian solution to all these problems, proposed earlier this week by Andrea Miotti, would be to create an “International Treaty to Ban Superintelligence.” Why worry about what to do with the latest and greatest models if you can make th...

There is a significant chance of conscious LLMs within a decade.

We will not just one day have an easy answer where everybody will agree.

A conscious being isn’t a word-guessing app that knows more words and has more computing power to guess with.

There are a lot of potentially good outcomes from AI. I think the most likely outcomes are good. But the risk of bad and even catastrophic outcomes is high enough to be worth taking seriously. We should actively build safety mechanisms in case we nee...

Plot the same improvements on a linear y-axis and you get steady incremental gains, impressive but not the hockey stick that triggered the panic. [...] We don't have evidence that Mythos is actually an important step towards broad superintelligence.

I would like to see a CERN for AI [...] where all the top scientists come together and see whether they can make a super intelligence. And, if they can, they contain it into a system where it can't just go out and persuade people to let it run the wo...

In the face of these threats, I urge the Government to take the following steps: first, to publicly recognise that superintelligence poses an extinction threat to humanity; secondly, for the UK to prevent the development of superintelligence on its s...