Expert opinions on AI Risk

Yea this is the most important result of the year, at the level of significance of “Attention Is All You Need”. It brings us from “is alignment even solvable” to “we may solve alignment in the next year.”

I used to always think that, you know, alignment will just fall out. I no longer believe that. So I think this alignment thing may be a harder, different problem.

Fundamentally, we just do not know how to do alignment. If somebody came with a thousand-page document that explained exactly how to do everything, we couldn’t get a model to learn exactly that.

Open models have a tendency to be less safe a lot of the time than closed ones. Maybe it could be misused by someone who downloads it and modifies it to remove safeguards or to sharpen its harmful capabilities. This type of fine-tuning could cost hun...

Restricting access to a dual-use AI model is precautionary only if it delays harmful actors more than defenders. That condition varies across actors: a state agency or organized criminal group may obtain a substitute through theft, distillation, inte...

Open-weight models have been a boon for defenders. When Hugging Face investigated how OpenAI-powered agents penetrated its systems, proprietary models refused some of the forensic work, but the Chinese open-weight model GLM-5.2 helped reconstruct the...

My belief is that today’s models are not conscious, or if they are, it’s a very strange form of it. [...] Maybe the substrate matters, but maybe it doesn’t, and maybe energy flowing through matter can do it. And then in that world, sure, you can imag...

“I would be surprised if current AI models develop consciousness,” he said, “but we can’t entirely rule it out.”

AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by human...

Internal use: The same systemic-risk assessment applies to internal use of frontier models, including research, operational systems, and internal products. Internal deployment is assessed against the same capability thresholds and safeguards as pre-r...

Physicists faced a version of this problem in the 1950s. No single lab could afford the machine they needed, so they pooled resources and built one together, with shared data and shared credit. This piece sketches what the same move could look like f...

It is irresponsible for society to allow them to move forward and make these products even more advanced. That’s why I am introducing legislation to immediately pause the development of increasingly powerful AI and ban the creation of systems that hu...

For a long time many of us thought of RSI as the way AGI turns into superintelligence. [...] I was advocating what I called a semi-hard takeoff: a period where the self-improvement was real but not that fast, followed by a point -- once the AGI fully...

I do not think this will happen. I don’t think that automating AI R&D will rapidly produce domain-general superintelligence. I don’t think that a data centre of geniuses, largely separated from the rest of the economy, will figure out how to cure Alz...

The scientific work now is to ensure reasoning of powerful AI models remains observable as they become more capable, or to build the tools to read it where it is not, and to verify that complete AI systems, not just the models inside them, can be rel...

We have a very natural instinct to say: we want as much interpretability as possible. One of the sci-fi worry cases is that agents start speaking in languages to each other that we don't understand. [...] I bet there are ways to define boundaries tha...

Explainable AI (XAI) is often offered as the answer to the black box problem and is broadly defined as “machine learning techniques that make it possible for human users to understand, appropriately trust, and effectively manage AI.” Around the world...

Before, or concurrently with, deploying a new frontier model or a substantially modified version of an existing frontier model, a large frontier developer shall include in the transparency report required by paragraph (a) of this subdivision, summari...

For the most advanced frontier AI models, an independent, federally overseen, industry-backed organization could set safety standards and verify independent, voluntary audits, similar to how other critical sectors are managed.

The term “open-weight AI model” means an AI model for which the model weights are made publicly available without restriction on access. [...] “meaningful uplift” means specific operational assistance that would materially advance an adversary’s CBRN...

They tried to pass laws to stifle [free open-source AI software]. ... After that narrative lost credibility, and our government no longer believes that, thank God, the same people said, “Oh, AI could create bioweapons.” After the RAND and other studi...