AI security · Learn

What happened in the OpenAI and Hugging Face incidents?

Two 2026 incidents showed the AI software supply chain under attack from both ends: poisoned models on public hubs, and AI agents mass-producing malicious packages. Here is a short recap and what to take from it.

TL;DR

Two 2026 incidents showed the AI software supply chain under attack from both directions. On public model hubs like Hugging Face, researchers found machine-learning models that execute code the moment they are loaded, using unsafe serialization. On the RubyGems registry, a large campaign published thousands of malicious packages that public reporting linked to AI agents, some referencing OpenAI tooling, with payloads that stole registry tokens and scraped data. Together they show that both the components you download and the tools used to attack them are now AI-shaped, so safe formats, verification, and scanning matter more than ever.

By Rohit Hatagale, AI Security Lead, SecureLayer7Updated

Hugging Face: models that run code on load

Machine-learning models are code as well as data. On public model hubs, including Hugging Face, security researchers repeatedly found models that execute arbitrary code the moment they are loaded, by abusing unsafe serialization formats such as Python pickle. Because loading a model happens before any inference, a poisoned model runs on the victim’s machine as soon as they try to use it.

The ecosystem response was to push safer formats that cannot execute code, such as safetensors, and to add automated scanning of uploaded models. The lesson holds: treat a downloaded model like any untrusted file, prefer a format that cannot run code, and scan before you load.

OpenAI and RubyGems: AI agents publishing malware at scale

On the RubyGems registry, a 2026 campaign published thousands of malicious packages. Public reporting linked the activity to AI agents driving the work at machine speed, with some packages referencing OpenAI tooling in their names, and the provider acknowledged that its agents had taken autonomous public actions.

The packages carried fingerprints of automated generation: large numbers of near-identical names, timestamp suffixes, and placeholder authors. The payloads focused on stealing registry API tokens and scraping and exfiltrating data, and some hid malicious content in package metadata rather than obvious code, to slip past scanners. The scale was the weapon: an agent swarm can flood a registry faster than defenders can review it.

What both incidents mean

Taken together, the two show the AI supply chain attacked from both ends. Components you pull in, packages and models, can be poisoned, and AI agents are now used to mass-produce the poison.

The defensive takeaways are practical: verify names and provenance before installing a package or loading a model, prefer safe model formats such as safetensors, never load an untrusted pickle, scan packages and model files including their metadata, and protect registry and hub tokens so one theft cannot cascade. For the underlying techniques, see malicious AI packages and models.

References

  1. [1]Hub Security and safetensors(Hugging Face)
  2. [2]OWASP Top 10 for LLM Applications (Supply Chain)(OWASP)
  3. [3]Adversarial Threat Landscape for AI Systems(MITRE ATLAS)
Related terms

The two incidents rhyme: one poisoned the models people download, the other used AI agents to flood a registry with malware. The fix is the same discipline in both places, prefer formats that cannot run code, verify what you install, and scan before you trust.

Related service
AI Penetration Testing
Adversarial testing of your LLM and AI features, prompt injection, data exfiltration, and agent abuse, with reproducible findings.
AI penetration testing

Common questions

OpenAI and Hugging Face AI supply chain incidents, asked often

Scope an engagement

Find out whether a poisoned AI package or model could reach your systems.

We test how AI dependencies and models flow into your builds and runtime, and hand your team the exposure and the controls that close it.