Liability for AI agents: when autonomy doesn’t exempt humans and companies from responsibility

An autonomous artificial intelligence system can receive a task, search for data, determine the next steps, and execute actions without human intervention at every stage. However, if something goes wrong, the technology’s autonomy doesn’t eliminate the need for accountability. The question then becomes: who established the rules, who activated the agent, and what mechanisms were in place to monitor its behavior?

Responsibility doesn’t lie solely with AI

When an agent makes a wrong decision, the analysis tends to focus on the people and organizations involved in its development and use. The supplier may have obligations related to the technology’s functionality and safety. Meanwhile, the company using the agent may be held accountable for how the system was configured, the context in which it operates, and the supervision provided.

The European Union already establishes this division for systems considered high-risk. The AI Act sets obligations for suppliers and those responsible for implementation, including monitoring, documentation, traceability, and human oversight. In these cases, the person or team responsible for oversight must be able to monitor operations and act when risks are identified. The investigation depends on the system used, the activity involved, the applicable rules, and the circumstances of the error.

The problem starts before the wrong decision is made

A significant part of responsibility may lie in how the agent was designed and implemented. For example, if a company allows an AI to approve requests, modify records, or send external communications, it must set limits for these actions. It’s also essential to know which records were kept. Without a history of instructions received, decisions made, and actions taken, it can be difficult to reconstruct what happened after an incident.

Traceability is included precisely for this reason in rules targeting higher-risk systems. In the AI Act, for instance, activity logs are part of the requirements designed to enable monitoring of outcomes.

Those who use the system also need to control its operation

The idea that “the AI decided on its own” can conceal a previous step: someone decided that the agent would be allowed to act independently. As a result, governance now involves concrete questions: What tasks can be performed without approval? When does a person need to intervene? What type of decision should be blocked? Who receives an alert when the agent deviates from expected behavior?

In Brazil, the debate is still evolving. In 2026, bills introduced in the Chamber began to specifically address issues such as liability for autonomous systems, governance of AI agents, and responsibility for high-impact systems. These proposals are still part of the legislative process and should not be confused with rules already in force.

Autonomy doesn’t mean absence of responsibility

For companies, the advancement of agents changes the question about AI use. It’s no longer enough to assess whether the technology can perform a task; it’s also necessary to define what happens when it makes a mistake. The greater the autonomy granted, the greater the importance of limits, records, interruption mechanisms, and oversight tends to be. The technology can execute the action alone, but responsibility remains a human, organizational, and, where applicable, legal issue.