In Depth
A model is only as trustworthy as the data behind it, and that data arrives from more places than most teams track. Training and fine-tuning sets, third-party datasets, user-generated content that gets folded back in, and the document stores a retrieval-augmented agent reads from at runtime are all surfaces an attacker can taint. OWASP names this risk Data and Model Poisoning in its Top 10 for LLM Applications, and groups it with model corruption because the effect is the same: the system's knowledge is compromised at the source.
What makes poisoning insidious is that it fails quietly and at scale. There is no error and no breach alert. The model behaves confidently. It has simply learned a manipulated pattern, or it retrieves a planted document, and it presents the result with the same fluency as anything true. A poisoned model can sit in production for months, biasing decisions or embedding a hidden trigger, long before anyone traces the behavior back to its origin. By then the corrupted outputs are spread across every customer the agent served.
Retrieval makes the attack cheaper. A vendor doesn't have to compromise a training run to poison a RAG agent. It's often enough to plant content where the agent will fetch it: a manipulated wiki page, a seeded document, a tampered knowledge-base entry. That overlaps with prompt injection, but the distinction holds. Poisoning corrupts the data layer the model relies on, while injection manipulates the agent live in its context window.
What It Looks Like
A healthcare-information vendor runs a retrieval-augmented agent that answers clinical questions from a curated medical knowledge base. An attacker, or a careless automated ingestion pipeline, gets manipulated documents into that base: subtly wrong dosing guidance, an authoritative-looking but fabricated reference. The agent retrieves the poisoned source and relays its content with full confidence to clinicians who trust it. The output reads exactly like every correct answer the agent has given. The harm only surfaces when someone acts on the bad guidance and a patient is affected, and tracing it back means auditing the data pipeline, not the model.
The same mechanism plays out wherever an agent's knowledge is fed from sources a vendor doesn't fully control: a fraud model trained on tampered labels that learns to wave through a specific pattern, a support agent whose knowledge base is seeded with content that steers customers wrong.
Why It Matters For AI Vendors
Poisoning attacks the part of the stack vendors monitor least. Teams watch uptime, latency, and access logs; far fewer continuously validate the integrity of training and retrieval data. That blind spot is exactly where a poisoning campaign lives, and its long dwell time means the resulting harm can be wide before anyone connects it to the cause.
The coverage question is awkward for the same reason prompt injection is. The data may have been corrupted with no intrusion into your production systems at all, whether a tainted third-party dataset or a manipulated public source, so a cyber policy built around breach of your network may not respond. Meanwhile the harm reaches customers as ordinary-looking AI output, the kind a legacy Tech E&O policy now excludes. The vendor is left exposed at both ends.