Expert opinions on Existential Risk

Yea this is the most important result of the year, at the level of significance of “Attention Is All You Need”. It brings us from “is alignment even solvable” to “we may solve alignment in the next year.”

I used to always think that, you know, alignment will just fall out. I no longer believe that. So I think this alignment thing may be a harder, different problem.

Fundamentally, we just do not know how to do alignment. If somebody came with a thousand-page document that explained exactly how to do everything, we couldn’t get a model to learn exactly that.

My belief is that today’s models are not conscious, or if they are, it’s a very strange form of it. [...] Maybe the substrate matters, but maybe it doesn’t, and maybe energy flowing through matter can do it. And then in that world, sure, you can imag...

“I would be surprised if current AI models develop consciousness,” he said, “but we can’t entirely rule it out.”

AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by human...

Turns out, we were being unnecessarily pessimistic. Even though the planet is on fire, extreme temperatures will not be the ones to knock us out of the evolutionary game. (So much for our soggy paper straws.) Instead, we’ve decided to build something...

We have to change the way we talk about AI risk because for a very long time, it’s either existential and everything is going to die and the killer robots are coming, or it doesn’t really matter.

We do need to have some safeguards in place to make sure there are no unintended consequences, but, no, I do not think AI is going to imperil human existence.

Physicists faced a version of this problem in the 1950s. No single lab could afford the machine they needed, so they pooled resources and built one together, with shared data and shared credit. This piece sketches what the same move could look like f...

It is irresponsible for society to allow them to move forward and make these products even more advanced. That’s why I am introducing legislation to immediately pause the development of increasingly powerful AI and ban the creation of systems that hu...

For a long time many of us thought of RSI as the way AGI turns into superintelligence. [...] I was advocating what I called a semi-hard takeoff: a period where the self-improvement was real but not that fast, followed by a point -- once the AGI fully...

I do not think this will happen. I don’t think that automating AI R&D will rapidly produce domain-general superintelligence. I don’t think that a data centre of geniuses, largely separated from the rest of the economy, will figure out how to cure Alz...

This suggests pacing the advances: not training or deploying AIs without a strong safety case that convinces independent experts.

I believe that attempting to deliberately pace AI development will be extremely difficult, and could do more harm than good if done without proper care. However, I am signing this statement because, if an attempt at pacing does occur, it would be bet...

We will only encourage super intelligence. We’re going to encourage it, not rein it in.

The term “open-weight AI model” means an AI model for which the model weights are made publicly available without restriction on access. [...] “meaningful uplift” means specific operational assistance that would materially advance an adversary’s CBRN...

They tried to pass laws to stifle [free open-source AI software]. ... After that narrative lost credibility, and our government no longer believes that, thank God, the same people said, “Oh, AI could create bioweapons.” After the RAND and other studi...