Model poisoning (part of OWASP LLM04:2025) is tampering with a model's parameters so it carries hidden malicious behaviour. Unlike training-data poisoning, which corrupts the data, model poisoning targets the weights directly, through a backdoored pre-trained checkpoint, a malicious fine-tune or adapter, or a tampered model file. The model passes normal evaluation but produces attacker-chosen output when a trigger appears. Defend by sourcing models from trusted origins, verifying integrity, and evaluating for backdoors.
What model poisoning is
A model's behaviour lives in its weights. Model poisoning is any tampering with those weights to embed behaviour the operator did not intend, most often a backdoor: the model works normally on ordinary inputs but produces attacker-chosen output when a specific trigger (a phrase, token, or pattern) appears. Because the change is in the parameters, it survives normal testing and ships with the model.
How the attack works
Common vectors: a backdoored pre-trained model published to a hub and downloaded by victims, a malicious fine-tune or adapter that inserts a trigger while appearing to improve the model, and a tampered checkpoint file, including unsafe serialization formats that execute code on load. A poisoned model can be made to emit a target output, leak data, or lower its guardrails only when triggered, so it passes benchmarks and review. Shown for defensive testing.
How to defend against it
Treat models like any other supply-chain dependency. Source them from trusted, verifiable origins, check signatures and hashes, and prefer safe serialization (for example safetensors) so loading a file cannot execute code. Evaluate models for backdoor behaviour, not just accuracy, and re-evaluate after any third-party fine-tune or adapter. Pin versions, keep provenance, and isolate model loading.
References
A backdoored model passes every benchmark until the trigger fires. Talk to a security expert about vetting the models in your stack.