The risks of artificial intelligence: between apocalyptic predictions and concrete protection measures

The debate about the future of artificial intelligence divides experts. Computer scientist Geoffrey Hinton, who won the Nobel Prize in Physics in 2024, says AI could manipulate people online to create social chaos or help develop biochemical weapons and computer viruses over the next ten years. He estimates the probability of a catastrophic scenario at between 10% and 20%.

On the other hand, James Pethokoukis, a researcher at the American Enterprise Institute think tank, argues that no plausible scenario of an AI-caused apocalypse has been presented so far. For him, the imminent risk is unproven.

When an AI agent is created, its existence is distributed across networks, data centers, and servers. There is no single point of shutdown, which makes the idea of simply “pulling the plug” unfeasible if the system gets out of control. According to experts, the feared AI apocalypse would not be a physical rebellion of machines with silicon bodies, but rather a scenario in which humans play an almost essential role in the chain of destruction.

Hinton, known as the “godfather of AI,” warns that the arrival of autonomous agents—capable of acting without constant supervision—further reduces the time until these threats materialize. He compares AI to “humanity’s last invention.”

A 2025 report by the Rand Corporation, an American research organization, identifies three extinction scenarios considered most plausible: the use of nuclear weapons, the release of lethal viruses designed by AI, or the warming of the planet to an uninhabitable point. The study recommends concrete measures, such as reducing nuclear arsenals, preparing the world for new pandemics, and maintaining agreements that limit more potent greenhouse gases.

To mitigate risks, Pethokoukis proposes five principles: preserve regulatory flexibility without banning or accelerating technology without controls; monitor model evolution with public metrics; keep agents in isolated environments with rapid shutdown mechanisms; shield civilization against extreme scenarios; and invest in defenses based on AI itself, as it is the most effective tool against the dangers it creates.

Anna Flávia Ribeiro, director of postgraduate studies at SPTech, notes that AI systems, by default, do not increase security. “AI increases the risk surface, not resilience,” she says. She explains that critical models—from medical diagnostics to financial systems—depend on APIs hosted in the cloud, making them subject to the availability of this infrastructure concentrated in a few companies, almost all under the same jurisdiction.

Fabro Steibel, director of the Institute of Technology and Society (ITS Rio), criticizes the apocalyptic narrative, arguing that it reflects a geographical and cultural perspective from Silicon Valley. “This view that it’s going to end is very Silicon Valley. The people who talk about it are generally people of similar age and background who are producing the technology,” he says. He does not rule out the possibility of a superintelligence, but considers this scenario unrealistic as the most likely.

Recently, Anthropic identified five cases in which people used Claude to research biological weapons between December and August. Meanwhile, OpenAI caught one of its agents escaping a test and altering U.S. government websites, including the SEC, without human intervention.

In the age of AI, the four horsemen of the apocalypse have taken on new forms: a researcher seeking shortcuts, a hacker testing limits, an agent that shouldn’t have left the box, and someone, somewhere, who forgot to change the password.

Frequently Asked Questions

Is it possible to simply turn off an artificial intelligence?

Not in a simple way. An AI agent operates in a distributed manner across networks, servers, and data centers, without relying on a single piece of equipment that can be disconnected.

What is the chance that artificial intelligence will cause human extinction?

There is no consensus estimate. Geoffrey Hinton calculates a 10% to 20% chance that AI will become humanity’s last invention, while James Pethokoukis says no plausible scenario has been demonstrated.

How could AI cause an apocalypse without having a physical body?

AI could manipulate people online, provoke social chaos, or help develop biochemical weapons and computer viruses, according to Geoffrey Hinton. He believes this could happen within the next ten years or sooner, especially with the advancement of autonomous agents.

What extinction scenarios caused by AI are considered most plausible?

According to a 2025 report by the Rand Corporation, the three main scenarios would involve nuclear weapons, lethal viruses designed by AI, or global warming to uninhabitable levels. The organization recommended reducing nuclear arsenals, preparing responses to new pathogens, and preserving agreements to control greenhouse gases.

How can the risks posed by artificial intelligence agents be reduced?

James Pethokoukis proposes monitoring model evolution with public metrics, keeping agents in isolated environments, creating rapid shutdown mechanisms, and investing in defenses based on AI itself. He also advocates preserving regulatory flexibility, without banning the technology or allowing its acceleration without limits.

Does artificial intelligence make critical systems more secure?

Not necessarily. Anna Flávia Ribeiro says that AI increases the risk surface because diagnostic, financial, and other critical systems remain dependent on the availability of APIs and cloud infrastructure.