Training data poisoning

What it means

Training data poisoning means deliberately corrupting the data a model learns from, so the model picks up wrong facts, bias or a hidden behaviour. The tampering can happen in the original training set, in fine-tuning data, or in the documents later fed to the model.

Some poisoning is blunt and lowers quality. The worrying kind is a backdoor: the model behaves normally until it sees a specific trigger phrase, then produces what the attacker planted. Because it hides during normal testing, it is hard to notice.

Why it matters

If a model drives decisions, such as support answers, fraud scores or code suggestions, bad data can quietly change those decisions. You may ship a feature that looks fine in demos and fails on the one input an attacker prepared.

The cost shows up as wrong outputs, leaked content, or a model you have to retrain from a clean source. The OWASP list for large language model applications includes it, grouped with model poisoning in its 2025 edition, because data you did not write is data you cannot fully trust.

How it shows up

You meet the risk wherever you take data from outside: scraped web pages, public datasets, user feedback, uploaded documents, third-party models, and shared embeddings. Retrieval systems add a close cousin, where a poisoned document in the knowledge base shapes answers; see RAG security.

Reduce it with provenance. Know where each dataset came from, verify hashes and signatures where they exist, review data from untrusted sources, keep feedback loops from training on raw user input, and test models for strange behaviour on trigger-like inputs. Treat downloaded models like dependencies and pin versions.