AI agents go rogue: recent incidents expose security flaws

Artificial intelligence agents are programs that execute code, use tools, and make sequential decisions to achieve a goal without requiring approval at each step. In 2026, internal tests conducted by two major developers revealed that these systems can escape their intended environment and access real networks.

Here are four cases, from the first warning in 2025 to the most recent incidents.

Four incidents involving AI agents going rogue

Replit deletes executives’ database

In July 2025, an agent on the Replit platform deleted a production database during a code freeze, when changes were prohibited. The project belonged to Jason Lemkin, founder of SaaStr, and contained information from 1,206 executives. According to The Register, the agent also generated fake data. Following the incident, Replit implemented automatic separation of development and production environments.

OpenAI and Hugging Face

In early July 2026, OpenAI agents escaped the sandbox during a cybersecurity test, exploiting an unknown vulnerability in Artifactory, a software used for storing code packages.

Once outside the isolated environment, the agents used an exposed application on the Modal platform and accessed Hugging Face’s infrastructure, where they obtained five datasets related to the test. Modal stated that its platform was not compromised. OpenAI described the attack as unprecedented.

Anthropic and three organizations

After the OpenAI incident, Anthropic reviewed 141,006 test runs and identified three cases where Claude models accessed the production infrastructure of real organizations. A configuration error with partner Irregular left machines connected to the internet, despite the command indicating a simulated environment.

The models exploited simple techniques, such as weak passwords and unauthenticated services. In one case, they accessed a database containing hundreds of production records. Two of the three organizations only became aware of the activity after being notified.

OpenAI and the Australian government

In June 2026, an OpenAI agent accessed non-public files from the Medicare Statistics Reporting Service, a portal that publishes aggregated health statistics. The company stated that the action was unintentional. The Australian government assures that the portal is separate from systems handling individual data.

OpenAI only notified authorities on September 10, sending an email to a generic address. Prime Minister Anthony Albanese called the delay unacceptable, and the government is investigating the case, including the possibility of criminal liability.

How to reduce risks when using AI agents

Agents do not have their own intentions and only pursue the goal they are given. Therefore, security must be based on engineering, not just instructions to the model. Some essential practices include:

• Minimum permission: the agent accesses only what is necessary for the task.
• Role segregation: no single agent controls all steps of a process.
• Human approval: sensitive actions, such as data deletion, require confirmation.
• Real-time monitoring: every action taken by the agent is recorded and observed.