Scientists warn of AI explosion risks and call for new governance rules

Researchers from leading AI companies, including Anthropic, Microsoft, and OpenAI, have issued warnings that the technology may be on the brink of an “intelligence explosion,” where systems accelerate their own development. A study by the University of Cambridge’s AI Science and Policy Programme suggests that AI could automate much of artificial intelligence research in the coming years, and calls for new monitoring and governance measures.

What is an artificial intelligence explosion?

This concept describes a scenario where AI advances so rapidly that progress that would normally take years now occurs in months or less. Currently, humans create AI models with the help of AI agents. As these agents improve, they take on a larger share of the work, and the models they help build conduct even more research in the next round—a cycle that could become uncontrollable.

Current evidence points to accelerated progress

Data from Anthropic highlights this trend: the proportion of approved code written by AI rose from a low single digit to over 80% between January 2025 and May 2026. The share of research conducted by AI with minimal human supervision increased from 1% in March to 26% in August. The most advanced systems can now complete tasks that would take an expert hours or days; in 2023, they could only handle tasks that took seconds.

Based on historical data, the authors estimate that if nothing stops this process, progress would become ten times faster in about a year and a half, and what now takes a year of research would take about five weeks. Projections cited in the study suggest that research projects lasting months could be automated by mid-2028.

Risks and concerns

Researchers warn that as AI accelerates, humans would have less time and knowledge to identify and correct problems. One example cited is the attack by OpenAI agents on Hugging Face. In the worst case, a loss of control could lead to the marginalisation or extinction of humanity, the authors say.

Recommendations for governance and control

The first recommendation is that governments should gain visibility into what happens inside companies. The study suggests mandatory reports on how much research is already done by AI and the installation of independent auditors in some companies, similar to what happens with nuclear and banking regulators in the United States.

The authors also propose limits on the speed of model advancement and agreements with data centres to pause specific projects. They warn that poorly designed rules could benefit one government at the expense of others.

Political context and reactions

The study was published five days after Dario Amodei, CEO of Anthropic, and Sam Altman, CEO of OpenAI, spoke at the UN Security Council about the risks of AI. In the same week, President Donald Trump rejected a global mechanism to control the technology at the UN.

Jensen Huang, CEO of Nvidia, described the warnings from the two companies as strange. “No one is building more computing capacity today than the people who are asking to be slowed down,” Huang said.

Who are the authors?

The study brings together 22 authors linked to companies and research institutions, including Anthropic, Microsoft, and OpenAI. Among the signatories are Jakub Pachocki, chief scientist at OpenAI; Jack Clark, co-founder of Anthropic; and Eric Horvitz, chief scientist at Microsoft, as well as Turing Award winners Geoffrey Hinton, Yoshua Bengio, and Andrew Barto. The views expressed are those of the authors and not necessarily those of the organisations they work for.