More than 20 researchers and executives, including AI pioneers Geoffrey Hinton and Yoshua Bengio, Anthropic co-founder Jack Clark and OpenAI chief scientist Jakub Pachocki, have warned governments to prepare for a runaway “intelligence explosion,” the Guardian reported Monday. Their paper, “What if automating AI R&D triggers an intelligence explosion?”, published via the Cambridge Programme on AI Science & Policy, argues that AI systems already help improve their own technology and that automating AI research and development is the most likely path to a dramatic speedup.
The authors define an intelligence explosion as a “dramatic AI-driven acceleration of AI progress, compressing advances that would otherwise take years into months or less.” Once systems reach expert-level AI R&D, the paper says, a single developer could run a workforce equivalent to “millions” of top human researchers. “An intelligence explosion could be the most consequential technological development in human history,” the paper states, “compressing years of progress into months or less, threatening human control over AI systems, and severely eroding checks on power within and between states, companies, and branches of government.” They add: “Once an intelligence explosion begins, the window for action may close.”
Policy priorities in the report include requiring transparent progress reports on AI-related R&D and embedding independent auditors in companies; finding ways to constrain breakneck development; and preparing societies to adapt. The Guardian notes Anthropic says AI now produces 80% of its own code, while OpenAI uses autonomous agents in areas such as training new models. Both companies have agreed to independent evaluators assessing their models.
The authors say productivity gains from AI R&D automation have not yet hit the threshold needed to trigger an explosion, but gains from newer systems are likely approaching it, and some projects that take humans months could be fully automated by 2028. Impacts remain uncertain: benefits could include medical and technological leaps, while risks include biological and cyber threats moving faster than counters, loss of human control as people exit the R&D loop, and states converting a modest lead into a decisive one. Preparing for those scenarios, they write, should be an urgent priority at the highest levels of government.