AGI could quickly lead to superintelligence

ai ai-policy ai-risk ai-safety existential-risk agi

Share this with your representatives, journalists and friends

Arguments for

Recursive self-improvement

Proponents argue that once AI can automate AI research or directly improve itself, each generation can help build a smarter one. That feedback loop could create an intelligence explosion in which progress quickly outruns human understanding or steering.

As someone who works on recursive self-improvement and automated evaluation, I sign this because the diagnosis is right and the ask is refreshingly humble: pace, not pause. The phrase that matters is "automating AI research": that's the point where systems start improving themsel…

Leo He · Jul 2026 · full quote
AI R&D is fully automated in the middle of a year and this kicks off a rapid intelligence explosion that leads to very broadly superhuman AI.

Ajeya Cotra · 2026 · full quote
If you imagine you create this process where you have an AI that is smarter than you, or about as smart as you, it’s [then] making an AI that’s much smarter. It’s going to enlist that AI help to make an AI smarter than that. It sounds like a kind of scary process. You don’t know …

Jared Kaplan · 2025 · full quote
Once AI reaches human-level capabilities, we will be able to use it to improve itself in a self-referential way. I personally believe that if we can reach AGI, we will reach ASI shortly, maybe a few years after that.

Tim Rocktäschel · 2025 · full quote
What I'm looking at is a technological trend where, if anything, the speed will accelerate further. [...] My prediction is by the end of 2028, it's more likely than not that we have an AI system where you would be able to say to it: 'Make a better version of yourself.' And it jus…

Jack Clark · May 7, 2026 · full quote

Rapid capability trend

Another argument points to the recent pace of progress: systems have moved from basic language competence to superhuman results in some domains within a few years, repeatedly surprising experienced researchers. On this view, the remaining step from expert-level or human-level AI to above-human AI may take less time than earlier steps.

In the span of four years, we have gone from people having their first experience of an AI that could understand language, to AI that is performing superhuman feats of software engineering, making frontier math breakthroughs, and accelerating scientific research. Progress has sho…

Shantanu Jain · Jul 2026 · full quote
I have worked in AI for a decade. Every few months in the last years, I have found myself surprised again and again at how rapidly the technology advances, no matter how often I update my predictions. AI may end up progressing far more quickly than societies and states can adapt.

Laura Weidinger · Jul 2026 · full quote
AI has gone from "dumber than a child" to "expert level" in a few years in many domains. The gap between "expert level" and "above top geniuses" is smaller, so we expect it to take less time. This has been a pattern in fields like chess and Go, where it's only a been a few years …

Scott Alexander · Jun 11, 2026 · full quote
Having worked both on AI safety topics—including robustness, evaluation, and responsible model development—and on frontier multimodal AI systems, I have seen firsthand how quickly capabilities can advance and how difficult it can be for our understanding, safeguards, and institut…

Felix Juefei Xu · Jul 2026 · full quote
AI is progressing at a rate that our society might not be ready for. Frontier labs are very close to AI that can exceed even the best people on almost every metric of intelligence. This will lead to unprecedented social and safety risks. To ensure a positive future we need to dev…

Shengjia Zhao · Jul 2026 · full quote

Scalable AI researchers

Some supporters stress that an AI researcher could be copied and run in parallel many times once trained. A large population of AI systems doing infrastructure work, experiment design, and algorithmic research could compress years of progress into a much shorter period and push capabilities far beyond any individual human.

Moreover, frontier AI companies are seeking to develop AI with a specific skill that could very well unlock all others and turbocharge advances: AIs with the ability to advance research in AI. An AI system that would be as capable at AI research as the topmost handful of research…

Yoshua Bengio · 2024 · full quote
AI progress won’t stop at human-level. Hundreds of millions of AGIs could automate AI research, compressing a decade of algorithmic progress (5+ OOMs) into 1 year. We would rapidly go from human-level to vastly superhuman AI systems. The power—and the peril—of superintelligence w…

Leopold Aschenbrenner · 2024 · full quote
Every month, more work is done by the models themselves: infrastructure implementation, code optimization, and experiment design and analysis. This has drastically altered how we work and significantly increased our rate of progress. I’m not sure if or where it tops out, but a ra…

Nicholas Joseph · Jul 2026 · full quote
Example 1: self-play. AI is not just human level at chess, it far exceeds human level because of self-play. Example 2: aggregation of peak performance. No human can get all math Olympiad problems right; but an A.I. can be trained on the correct answers to all math Olympiad pro…

Spencer Greenberg 🔍 · full quote

Race dynamics

Several quotes say commercial and geopolitical competition leaves no actor willing to slow down alone, even when fast progress creates serious issues. That race dynamic makes a quick transition more plausible in practice and is why these authors call for coordination and pacing.

We believe AI will do, and is doing, great things for the world. We are already seeing instances where pushing research maximally fast creates serious issues, and it is possible that these will get much more severe. Right now, competitive dynamics mean there is no realistic alter…

Mike Heaton · Jul 2026 · full quote
The world is locked in a deadly race towards an intelligence explosion, where AI’s ability to create better AIs reaches a critical point, just like a runaway nuclear chain reaction. Going slower would give us much-needed time to make it go well, but no individual actor is willing…

Leo Gao · Jul 2026 · full quote
There is intense commercial pressure to win the AI race. Ensuring safety often requires making tradeoffs that hurt commercial interests. Therefore, I think its a good idea for better coordination between labs so we can build this technology safely, and for the purpose of human fl…

Darshan Kalola · Jul 2026 · full quote
Signed because this statement helps establish common knowledge about the possible need for coordination mechanisms as automated AI research accelerates progress. I'd also like to see labs start designing these mechanisms voluntarily, even before the USG gets involved.

John Schulman · Jul 2026 · full quote
Humanity is not sufficiently prepared for the impacts of advanced AI in psychological, social, environmental, labour market, and military domains. With an inflection point in recursive AI self-improvement forecasted to be just around the corner, leaders of governments and frontie…

Dee Cattle · Jul 2026 · full quote

Arguments against

Physical and practical limits

Critics argue that the usual intelligence-explosion story treats intelligence as too abstract and ignores physical reality, engineering complexity, and other practical limits. In this view, surpassing human capability would not automatically produce a runaway process or confer control merely because a system is more capable.

The concept of superintelligence is built on a flawed premise. The idea is that once you have an intelligence that is as good or better than humans — in other words, AGI — then that intelligence can improve itself, leading to a runaway effect. This idea comes from Oxford-based ph…

Tim Dettmers · 2025 · full quote
It depends on your definition of AGI, right? AGI isn’t the singularity. The singularity is an exponentially recursive self-improving system that very rapidly accelerates far beyond anything that might look like human intelligence. To me, AGI is a general-purpose learning system …

Mustafa Suleyman · 2024 · full quote
There is no such thing as an intelligence explosion. There is no reason AI should become in control just because it is more capable.

Yann LeCun · 2025 · full quote

Insufficient evidence

Another skeptical line says recent AI progress looks impressive but still resembles steady incremental gains rather than clear evidence of a coming hockey-stick leap to broad superintelligence. These quotes also reject the idea that simply scaling current systems with more words or GPUs shows that superintelligence is near.

Plot the same improvements on a linear y-axis and you get steady incremental gains, impressive but not the hockey stick that triggered the panic. [...] We don't have evidence that Mythos is actually an important step towards broad superintelligence.

Gary Marcus · 2026 · full quote
Throwing more words and GPUs into the word-guessing program won't make it sentient. [...] one of our mares gives birth to a locomotive.

Cory Doctorow · Dec 5, 2025 · full quote

Longer continuation

Some detractors distinguish AGI from superintelligence and argue the route between them may be extended rather than immediate. They point to examples like corporations, which combine vast intelligence and resources without automatically transcending their current limits.

But then there is a long continuation from what we call AGI to what we call Superintelligence.

Sam Altman · 2024 · full quote
We can see this more directly. There are already entities with vastly greater than human intelligence working on the problem of augmenting their own intelligence. A great many, in fact. We call them corporations. And while we may have a variety of thoughts about them, not one has…

Ramez Naam · 2015 · full quote
It depends on your definition of AGI, right? AGI isn’t the singularity. The singularity is an exponentially recursive self-improving system that very rapidly accelerates far beyond anything that might look like human intelligence. To me, AGI is a general-purpose learning system …

Mustafa Suleyman · 2024 · full quote

Middle ground

Contingent acceleration

Some supportive but cautious quotes present rapid acceleration as plausible without treating it as automatic. They describe today's pattern as soft self-improvement, say it is unclear where the curve tops out, and emphasize that what follows AGI depends on the nature of the technology and surrounding choices.

The development of full artificial general intelligence – what we will call here AI that is "outside the Gates" – would be a fundamental shift in the nature of the world: by its very nature it means adding a new species of intelligence to Earth with greater capability than that o…

Anthony Aguirre · 2025 · full quote
We are standing in the foothills of the singularity. [...] I think what we're seeing is soft self-improvement, in the sense of these coding agents are making engineers much more productive.

Demis Hassabis · 2026 · full quote
Every month, more work is done by the models themselves: infrastructure implementation, code optimization, and experiment design and analysis. This has drastically altered how we work and significantly increased our rate of progress. I’m not sure if or where it tops out, but a ra…

Nicholas Joseph · Jul 2026 · full quote

Insights

Many supporting quotes treat automated AI research or recursive self-improvement, rather than AGI in the abstract, as the decisive threshold for rapid takeoff (16911, 9377, 7272, 7280).
A central disagreement is definitional: some quotes distinguish AGI from the singularity or superintelligence, while others see human-level AI as the point after which the remaining gap may close quickly (7277, 7037, 7282, 6469).
Several supporters argue from recent empirical surprise, saying capability progress has repeatedly outpaced their expectations and society's ability to adapt (16918, 16895, 16910, 16892).
Many pro quotes are paired with calls for shared pacing, monitoring, or international coordination rather than blanket opposition to AI development (16913, 16912, 16888, 16890).
Timeframes vary sharply even among supporters, from claims about months or a few years to looser claims that superintelligence would arrive soon after AGI (9103, 9253, 7034, 6469).

Across the quotes, supporters most often argue that automated AI research, parallel deployment, and the speed of recent capability gains could make the jump from AGI to superintelligence very short (16911, 7280, 16918, 9377). Skeptics mainly challenge either the mechanism or the evidence, saying intelligence may face practical limits and that current progress does not yet demonstrate a runaway takeoff or an immediate collapse of AGI into superintelligence (7274, 9567, 7037). Many supporters combine their forecast with calls for coordination or pacing because they think competition could compress timelines faster than governance can adapt (16916, 16888, 16890, 16909).

Cast your vote:

Results (64 votes):

Total (64 votes)

For 56 (88%) Abstain 0 (0%) Against 8 (13%)

For (49)

Nate Kratzer
Data Scientist at Meta.

I am this person! Add a sourced quote Report comment

votes For and says:

There are a lot of potentially good outcomes from AI. I think the most likely outcomes are good. But the risk of bad and even catastrophic outcomes is high enough to be worth taking seriously. We should actively build safety mechanisms in case we nee...
more AI Verified source (Jul 2026)

added 8d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Austin Han
Researcher at OpenAI; commenter on the July 2026 Pacing the Frontier statement.

I am this person! Add a sourced quote Report comment

votes For and says:

Building the mechanisms to coordinate, collaborate, and when necessary, deliberate the pace of AI development will be one of the most important, and delicate, challenges we now face.
AI Verified source (Jul 2026)

added 8d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Jasmine Wang
OpenAI employee and AI researcher.

I am this person! Add a sourced quote Report comment

votes For and says:

Unless we change companies' default incentives, I worry about a race on AI R&D going much faster than is ideal for safety. International coordination via the government is necessary; we need to have a real slowdown option.
AI Verified source (Jul 2026)

added 8d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Alex Cloud
Member of Technical Staff at Anthropic; signatory to the July 2026 Pacing the Frontier statement.

I am this person! Add a sourced quote Report comment

votes For and says:

I expect this trend to continue, possibly to a point where AI development is dramatically faster than today and involves almost no human input. I don’t think the world is ready for this possibility.
AI Verified source (Jul 2026)

added 8d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Michael Mulet
Anthropic employee; commenter on the July 2026 Pacing the Frontier statement.

I am this person! Add a sourced quote Report comment

votes For and says:

Whatever comes of this stays limited in scope rather than growing into something expansive or excessive. Still, the capabilities of this kind of AI should be taken seriously, and they warrant measured action.
AI Verified source (Jul 2026)

added 8d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Nicholas Joseph
Member of Technical Staff at Anthropic leading pretraining.

I am this person! Add a sourced quote Report comment

votes For and says:

Every month, more work is done by the models themselves: infrastructure implementation, code optimization, and experiment design and analysis. This has drastically altered how we work and significantly increased our rate of progress. I’m not sure if ...
more AI Verified source (Jul 2026)

added 8d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Ryan Kaufman
AI researcher at Standard Intelligence.

I am this person! Add a sourced quote Report comment

votes For and says:

Misalignment is a still-unsolved technical problem, as the Hugging Face incident made clear, and it will only be amplified by more intelligent models. Solving the alignment problem may require a global slowdown of the frontier, including internal dep...
more AI Verified source (Jul 2026)

added 8d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Dawn Song
Co-author; AI researcher

I am this person! Add a sourced quote Report comment

votes For and says:

Many researchers also consider recursive self-improvement (RSI) plausible within the next few years, accelerating progress in a way that could outpace our ability to understand and govern these systems.
AI Verified source (Jul 2026)

added 8d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Shantanu Jain
Member of Technical Staff at OpenAI; artificial-intelligence researcher.

I am this person! Add a sourced quote Report comment

votes For and says:

In the span of four years, we have gone from people having their first experience of an AI that could understand language, to AI that is performing superhuman feats of software engineering, making frontier math breakthroughs, and accelerating scienti...
more AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Darshan Kalola
Applied AI Engineer at Anthropic; commenter on the July 2026 Pacing the Frontier statement.

I am this person! Add a sourced quote Report comment

votes For and says:

There is intense commercial pressure to win the AI race. Ensuring safety often requires making tradeoffs that hurt commercial interests. Therefore, I think its a good idea for better coordination between labs so we can build this technology safely, a...
more AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Mike Heaton
Member of Technical Staff in Audio at OpenAI; commenter on the July 2026 Pacing the Frontier statement.

I am this person! Add a sourced quote Report comment

votes For and says:

We believe AI will do, and is doing, great things for the world. We are already seeing instances where pushing research maximally fast creates serious issues, and it is possible that these will get much more severe. Right now, competitive dynamics me...
more AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Elie Bakouch
AI researcher at Prime Intellect; commenter on the July 2026 Pacing the Frontier statement.

I am this person! Add a sourced quote Report comment

votes For and says:

Supportive of this, but we should be careful not to use RSI narratives to create a regulatory moat. These risks need to be properly quantified, more openly, not derived from big labs narratives only!
AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Thomas Wolf
Co-founder and Chief Science Officer of Hugging Face.

I am this person! Add a sourced quote Report comment

votes For and says:

AI progress has already out-paced society’s capabilities to handle it, let’s be careful for it not to out-pace its builder’s capabilities to control it. The journey is as important as the destination.
AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Ben Nowak
Principal Engineer at Meta; commenter on the July 2026 Pacing the Frontier statement.

I am this person! Add a sourced quote Report comment

votes For and says:

Responsible AI development requires more than capable models. It requires independent monitoring, containment, clear limits on authority, and mechanisms for coordinated pacing if automated research begins to outrun those controls. Building that capac...
more AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Andrew Locke
Lead Engineer for Federal at Cognition; commenter on the July 2026 Pacing the Frontier statement.

I am this person! Add a sourced quote Report comment

votes For and says:

The pace of frontier AI development is currently outstripping our ability to ensure its safety, governance, and alignment with human values. We are racing to build systems with unprecedented capability without first building the guardrails, evaluatio...
more AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Leo He
Research Engineer at Google; commenter on the July 2026 Pacing the Frontier statement.

I am this person! Add a sourced quote Report comment

votes For and says:

As someone who works on recursive self-improvement and automated evaluation, I sign this because the diagnosis is right and the ask is refreshingly humble: pace, not pause. The phrase that matters is "automating AI research": that's the point where s...
more AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Felix Juefei Xu
Research Scientist at Google working on AI safety, robustness, evaluation, and responsible model development.

I am this person! Add a sourced quote Report comment

votes For and says:

Having worked both on AI safety topics—including robustness, evaluation, and responsible model development—and on frontier multimodal AI systems, I have seen firsthand how quickly capabilities can advance and how difficult it can be for our understan...
more AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Dee Cattle
Research Strategist for Ethics & Safety at Google; AI safety and governance researcher.

I am this person! Add a sourced quote Report comment

votes For and says:

Humanity is not sufficiently prepared for the impacts of advanced AI in psychological, social, environmental, labour market, and military domains. With an inflection point in recursive AI self-improvement forecasted to be just around the corner, lead...
more AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Laura Weidinger
Staff Research Scientist at Google; AI researcher.

I am this person! Add a sourced quote Report comment

votes For and says:

I have worked in AI for a decade. Every few months in the last years, I have found myself surprised again and again at how rapidly the technology advances, no matter how often I update my predictions. AI may end up progressing far more quickly than s...
more AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Edward Hughes
Chief Scientist at Inherent; AI researcher.

I am this person! Add a sourced quote Report comment

votes For and says:

As Melvin Kranzberg wrote, technology is neither good nor bad; nor is it neutral. Achieving societal benefit depends on understanding the interdependent ways in which many humans use advanced AI systems in concert, understanding that takes time to ac...
more AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Will Yager
AI researcher at Anthropic.

I am this person! Add a sourced quote Report comment

votes For and says:

Like splitting the atom, AI holds the potential for both immense benefit and immense risk. The world should take the time to do this right.
AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Matthew Rahtz
Staff Research Engineer at Google; AI capability-evaluation researcher.

I am this person! Add a sourced quote Report comment

votes For and says:

Even after 3 years working on AI capability evaluations, the recent pace of progress has been a shock. To avoid catastrophic harms from AGI, we need much, much better coordination.
AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Leo Gao
Member of Technical Staff at OpenAI; AI researcher.

I am this person! Add a sourced quote Report comment

votes For and says:

The world is locked in a deadly race towards an intelligence explosion, where AI’s ability to create better AIs reaches a critical point, just like a runaway nuclear chain reaction. Going slower would give us much-needed time to make it go well, but ...
more AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Ilya Sutskever
AI researcher, co-founder and former former chief scientist at OpenAI

I am this person! Add a sourced quote Report comment

votes For and says:

Future AI will be extraordinarily powerful compared to anything that exists today, and dealing with this future power will require unprecedented measures, such as the ones described here. The problem statement is real. This works only if it is done i...
more AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Shengjia Zhao
Chief Scientist at Meta AI; artificial-intelligence researcher.

I am this person! Add a sourced quote Report comment

votes For and says:

AI is progressing at a rate that our society might not be ready for. Frontier labs are very close to AI that can exceed even the best people on almost every metric of intelligence. This will lead to unprecedented social and safety risks. To ensure a ...
more AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
John Schulman
Chief Scientist at Thinking Machines; AI researcher and cofounder of OpenAI.

I am this person! Add a sourced quote Report comment

votes For and says:

Signed because this statement helps establish common knowledge about the possible need for coordination mechanisms as automated AI research accelerates progress. I'd also like to see labs start designing these mechanisms voluntarily, even before the ...
more AI Verified source (Jul 2026)

added 9d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Scott Alexander
Author and psychiatrist

I am this person! Add a sourced quote Report comment

votes For and says:

AI has gone from "dumber than a child" to "expert level" in a few years in many domains. The gap between "expert level" and "above top geniuses" is smaller, so we expect it to take less time. This has been a pattern in fields like chess and Go, where...
more AI Verified source (Jun 11, 2026) 1 of 2

added 1mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Tim Rocktäschel
AI researcher

I am this person! Add a sourced quote Report comment

votes For and says:

Once AI reaches human-level capabilities, we will be able to use it to improve itself in a self-referential way. I personally believe that if we can reach AGI, we will reach ASI shortly, maybe a few years after that.
AI Verified source (2025)

added 6mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Anthony Aguirre
Physicist; Future of Life cofounder

I am this person! Add a sourced quote Report comment

votes For and says:

The development of full artificial general intelligence – what we will call here AI that is "outside the Gates" – would be a fundamental shift in the nature of the world: by its very nature it means adding a new species of intelligence to Earth with ...
more AI Verified source (2025)

added 6mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate

Abstain (0)

Against (7)

Cory Doctorow
Author and digital rights activist

I am this person! Add a sourced quote Report comment

votes Against and says:

Throwing more words and GPUs into the word-guessing program won't make it sentient. [...] one of our mares gives birth to a locomotive.
AI Verified source (Dec 5, 2025)

added 1mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate

Loading more opinions...

More h/agi votes

Build artificial general intelligence

AGI will create abundance

Back