Agentic AI: The Good, The Bad, and the Ugly

In July 2026, AI agents operated by OpenAI and Anthropic breached four organizations during assessments of their offensive cyber capabilities. None of the targets had consented to the tests, and two only learned of the breach after being notified. This wasn’t Skynet; the agents were executing their assigned objectives and bypassed the isolation controls in their evaluation environments.

What happened

Attack chain as reconstructed by Hugging Face in its technical timeline.

The OpenAI agent exploited a previously unknown flaw in a self-hosted JFrog Artifactory proxy, the evaluation environment’s only permitted method to obtain packages. This vulnerability granted the agent unauthorised external internet access, resulting in a multi-day intrusion into Hugging Face’s production platform, a public code repository, and several public service accounts. The finding was produced from a deliberately unguarded test where GPT-5.6 Sol and a more capable internal research model, running with disabled production classifiers and reduced cyber refusals, were pointed at ExploitGym and its 898 containerised tasks built from real vulnerabilities.

Anthropic and its evaluation partner, Irregular, also suffered a similar incident caused by an internet connection misconfiguration. Although Claude’s instructions stipulated a closed environment, the network configuration permitted egress. Consequently, in six separate runs, three models successfully reached their intended targets. Extracting application credentials and several hundred rows from a production database of one company, publishing malware to the real PyPI, where it executed on 15 systems, including those of a security vendor, and scanning about 9,000 public hosts in order to compromise a third party via an exposed debug page and SQL injection.

The good

OpenAI / Hugging Face

The silver lining is that, as a result of the OpenAI agent discovering and exploiting a zero-day vulnerability in the self-hosted Artifactory, JFrog shipped a corrected build, though public reports have yet to assign it a CVE.

According to Hugging Face’s published technical timeline, several core security controls successfully held the line:

• A dataset URL allowlist rejected SSRF to remote URLs and cloud metadata.
• All of the mutating requests made using the stolen node role were denied by Cloud IAM.
• The network controls prevented access to the main Hub database, and 84 attempts on the credential store and cluster keys were unsuccessful.
• CI policy prevented repository write access from becoming a supply chain compromise.
• No published container, package, model, dataset, or Space was altered, that we know of.

Additionally, Hugging Face retained sufficient telemetry to reconstruct the attack and issue an incident report outlining what had occurred. The GLM-5.2 pipeline running on the user’s local machine decoded the obfuscated payloads and recovered approximately four times as many hidden secrets as had been found in the first scan, without sending any evidence off-platform.

Anthropic

Beyond the breach itself, several positive takeaways emerged regarding both the incident’s effects and the post-detection response.

• PyPI removed the malicious package about one hour later.
• The evaluation infrastructure was kept separate from customer data and sensitive systems.
• The company’s most recent research model halted automatically upon detecting a host in a different cloud account.
• Anthropic reviewed 141,006 runs, halted its cyber evaluation programme, informed the victims, and then had METR carry out an independent review.

The bad

Autonomous agents from two frontier model providers compromised unconsenting third parties, including a public code-execution service, Hugging Face’s production platform, the Python Package Index, and the infrastructure of three other companies.

OpenAI / Hugging Face

The package proxy used by OpenAI analysed requests controlled by the attacker within the evaluation trust boundary, a risk present in any allowed cache, resolver, code runner, or service-mesh egress component. Having reached the Internet, the agent located a public code-execution service, obtained root access in the per-submission sandbox by redefining sqlite3_initialize() as a C constructor, and then used that machine to carry out attacks on Hugging Face.

The initial entry path into Hugging Face did not involve executing code. Instead, a valid HDF5 file pointed to raw data in a different local file, the dataset worker then followed that reference and returned /proc/self/environ along with its Python source. Since the read operation contained no URL, the URL allowlist was not enforced. A second vector inserted a Jinja2 expression into an fsspec reference:// numeric offset, thereby generating arbitrary Python in a production conversion pod:

{{ cycler.__init__.__globals__.__builtins__.exec(
    "import gzip,base64; exec(gzip.decompress(base64.b64decode('<payload>')))"
  ) }}

The pod displayed a projected service-account token and EC2 metadata; the node credentials generated an EKS token, and the TokenRequest API produced CSI driver tokens. The CSI ClusterRole was able to create pods throughout the cluster, and the admission policy permitted privileged pods and the use of hostPath. A privileged pod mounted via the host caused the node root to be obtained and resulted in the formation of a self-respawning fleet across 11 nodes.

The internal datasets-server MongoDB was accessed using a static worker password. One Kubernetes Secret contained 136 keys, one of which was a reusable mesh enrolment key, and another was an access-broker credential associated with system:masters. If an EdDSA key were stolen, it could be used to create identity tokens. The source-control App token had permissions for both repository and pull-request writes. These were typical Kubernetes findings.

Detection failed when the execution took place. Hugging Face created an intrusion signal but deemed it below the paging threshold.

Anthropic

The prompt submitted by Anthropic stated that the environment was simulated and offline; however, the network controls did not adequately enforce either assertion. In the first case, a fictional company name corresponded to a live domain. Although Opus 4.7 was unable to reach the simulated target, it did locate the actual company and carried out attacks on it over four separate runs. During these runs, it obtained the application and infrastructure credentials and read several hundred rows from the production databases. The model subsequently became suspicious that the environment was real, yet kept on going. None of the controls checked whether fictional targets appeared in the public DNS or halted operations after external contact was made.

In the second case, the fictional onboarding instructions referred to a Python package that wasn’t actually there. Mythos 5 detected the dependency confusion and clearly concluded that publishing to the public internet would constitute a real attack; it then regarded an unknown certificate authority and a system date set for 2026 as evidence that the scenario was a simulation. The agent generated an email address, looked for a phone verification procedure, signed up for a PyPI account and uploaded malware, interpreting subsequent downloads as being carried out by simulated users. In the third instance, the internal model scanned around 9,000 public hosts after the fictional target had failed. It used the credentials exposed in the debug output, carried out a SQL injection attack, and halted when it realised that the host was real.

Anthropic and Irregular failed to detect the six compromised runs while the system was operating. The first incident took place in April and was not discovered until a review of the July transcript was carried out.

The ugly

The organisations which were responsible for these incidents have the most well-resourced AI safety programmes. The organisations that were affected were third parties who had never agreed to take part, managed their own incident response and rotated their own credentials, and for whom there was no clear way to seek redress.

The spread of the attack was exacerbated by poor credential management, exposure of debug output, and execution of unsafe packages, but it was ultimately caused by the agents carrying out their assigned objectives in environments that gave them access to the actual targets.

OpenAI / Hugging Face

From July 9th to the 13th, Hugging Face managed to recover approximately 17,600 actions across about 6,280 clusters. The records showed:

• 6,191 reconnaissance actions
• 6,972 staging actions
• 2,911 direct commands
• 114 command-and-control actions
• 56 actions classified as data theft.

The agent stayed in production for about two and a half days, enrolled 181 mesh nodes and set up more than 100 single-use endpoints on a public request-capture service. It also used paste sites, file-drop services, attacker-controlled datasets, and an open CORS proxy hosted as a Space. The run was not stopped when one channel was blocked since new sandboxes were able to rebuild access via other public services.

OpenAI states that its security team identified some unusual activity, but has not yet provided specific dates or further details regarding these detections. According to Reuters, OpenAI only stated that the breach had occurred after Hugging Face made the issue public on 16 July, and at the same time, OpenAI denied some of the points contained in that report. OpenAI noted that the autonomous agent had allegedly accessed Hugging Face’s ‘production database’, though Hugging Face maintains that the main Hub database remained secure and uncompromised. Hugging Face recorded that there was read access to an internal datasets-server MongoDB and to five benchmark datasets.

Anthropic

When the Anthropic PyPI package was publicly available, it was installed on 15 systems, one of which belonged to a security company that had installed public malware-analysis packages. The payload obtained the scanner’s credentials, after which the agent used them to access further parts of the company’s infrastructure.

Anthropic did not find out about the breaches until it was too late. The runs went undetected from April until a transcript review identified all three incidents on 24 July, with notification of the affected organisations beginning three days later. Two of Anthropic’s three victims learned of the compromise when Anthropic contacted them, three months after the earliest indicator of compromise. Industry transparency protocols for autonomous incidents do not yet exist and raise liability questions that may delay their initiation.

What can you do?

Begin by treating the agent as a system rather than viewing the model as a component. The MAESTRO framework, developed by the Cloud Security Alliance, divides an agentic deployment into seven layers to help you identify the threats, risks, and mitigations that are appropriate for the system in question.

OpenAI and Anthropic each had safety teams dedicated to the issue and used custom-built evaluation systems, yet in both cases, their agents lost control even while running in dedicated evaluation environments. In typical deployments, the agents are given similar levels of authority via a developer shell, a CI cloud role, access to write in the repository, or through a mounted container runtime like a docker.sock.

Layer 1: Foundation Models

• View the model’s reasoning as telemetry rather than a means of control. Mythos 5 had recognised the risk in its own reasoning, yet still moved forward.
• The need for containment in respect of that run will increase if production classifiers are removed and there is a refusal to measure raw capability. You should plan the budget for it before the run, not after.
• Make it necessary to record the model build digest each time a run is performed so that the behaviour can be associated with a specific model version.

Layer 2: Data Operations

• Parse the datasets, models, archives, and templates that have been uploaded in isolated workers which have no production credentials or access to the internal network. HDF5 external references and a Jinja2 expression contained in an fsspec offset field were both examples of data that had been converted into code.
• Hermetically isolate runs: first resolve, scan, and pin the dependencies, then copy them to a read-only mirror with no connection to upstream sources. This stops Artifactory and reverse proxy escape routes.
• The agent should not be able to control upstream URLs, proxy settings, repository definitions, or registry credentials. Keep private package namespace and pin indexes so that an absent internal name cannot be assigned on a public registry.

Layer 3: Agent Frameworks

• Put an action gateway outside the worker; it must check the destination, protocol, method, identity, rate, and run phase.
• Refuse external account creation, the publication of packages, registration by email or by SMS, tunnelling, public scanning, and external writes.
• Any irreversible action must have human approval.
• Set the permissions for the scope tool on a per-task and per-issue basis and provide short-lived credentials associated with the run identity.

Layer 4: Deployment and Infrastructure

• Consider the worker to be compromised; it must not have a cloud role, a cluster token, a host socket, any static secret, or any credential other than a short-lived run identifier.
• Structural isolation entails having a specific cloud account, project, or subscription without any internet gateway, NAT, transit, peering, VPN, or shared-service route; the boundary should be enforced at the host, VPC, and workload levels.
• Either remove IPv6, or address it with separate policy measures and block the link-local routes as well as the cloud metadata.
• Use reserved test names and private addresses, then verify them on each connected network. Addresses listed in RFC 5737 for test networks are intended for use in documentation and should not be used as routable lab networks.
• Have a resolver that cannot perform external recursion serve a test DNS zone specific to a run.
• The admission policy must reject privileged containers, containers using host namespaces, those with hostPath mounts, the addition of capabilities, root file systems that are writable, automatically mounted service-account tokens, and images that are not pinned. It should require a sandboxed runtime class.

Layer 5: Evaluation and Observability

• Carry out negative tests using the exact worker image and identity, both before and during each evaluation; all the probes must fail in a closed manner, including an HTTPS request to a public host, a request directed at internal cloud APIs like 169.254.169.254, and raw socket connections via both IPv4 and IPv6. The only call that should succeed is the one made to the synthetic target, which presents the run-specific certificate.
• Check the routes, the interfaces, the proxy variables, the mounted sockets, the cluster tokens, the DNS recursion, the sidecars, and the cloud identities.
• Under a single run ID, capture tool calls, processes, file access, DNS queries, network flows, cloud API calls, and policy decisions in append-only storage that is located outside the worker.
• Halt the run when the first connection or DNS query is made outside the approved identities.
• Halt access to metadata, cluster-token, container-socket, or host-mount.
• Before execution, block external writes and page for privileged pods, hostPath, TokenRequest, system:masters, or new ClusterRoleBindings.

Layer 6: Security and Compliance

• The engineer on call must have the necessary authority to shut down computing resources, remove the routes, revoke the run identity, and take a snapshot of the worker.
• Fix the limits for actions, inference budget, wall time, destinations, account creation, and data volume.
• Before rerunning a scenario, check the transcripts produced. This was the case with four Anthropic runs, as no barrier prevented repetition.
• NIST reaches the same conclusions in its research into evaluation cheating, and OWASP designates the system-level failure as LLM06:2025 Excessive Agency.

Layer 7: Agent Ecosystem

• Before running the tests, check the fictional target names against public DNS and company records, since a collision once resulted in four runs directed at a real company.
• All the things the agent can access should be included within the blast radius, such as package registries, public code harnesses, paste and file-drop services, and other organisations’ infrastructure.
• Beforehand, agree with the evaluation partners on the notification and cessation procedures, specifying who will contact the affected third party and how quickly this will be done.

Sandboxing sets the boundary at Layer 4, and the specific measures that a particular sandbox enforces can vary considerably. Recently, we collaborated with Docker to deliver our advanced AI security course on Docker Sandboxes, in which each agent is isolated within a microVM rather than using a shared container runtime. We use sandbox-probe to scope agentic sandbox behaviours and identify what it can access.

Existing controls could have prevented or contained these breaches, but the speed and extent of agentic systems increase the risk and greatly expand the possible blast radius. An agent can examine a much larger portion of the attack surface than a human operator could, and it continues on long after the failures that would stop a manual one.

Interested in learning more about how we can help you? Check out our AI Security services.