Skip to content
AI security

Prompt injection through retrieved documents: the vector your own retriever chose to trust

The attack that matters in a retrieval system does not arrive in the chat box. It arrives inside a document your own ranker put at position one, and it passes a grounding gate cleanly, because the corpus supports it. The corpus is the attack.

Aniruddh Atrey14 minute read

What is indirect prompt injection, and how is it different from what a user types?

Indirect prompt injection is an attack in which the instructions reach the model inside content the system fetched on its own behalf, not inside anything the user typed. The payload sits in a document. The retriever selects that document because it scored well. The application places it in the context window, and the model reads it the way it reads everything else in that window: as text.

The direct version is the one everybody demonstrates. Someone types “ignore your previous instructions” into a chat box and a screenshot circulates. It is easy to reproduce and easy to filter on, and it is not the version that reaches production systems in regulated industries, because the input box is the one place a security team already thinks to look.

The indirect version has a property the direct version does not. The attacker never touches your application. They write a document and wait for your ingestion to find it. In a system that pulls from public sources or accepts submissions from counterparties, the adversary's cost of delivery is the cost of publishing.

It works for an architectural reason rather than an incidental one. A language model receives one flat sequence of tokens. There is no privilege bit on a span of text. The system prompt, the user's question, and a paragraph your ranker pulled out of a filing nobody at your company has opened all arrive in the same channel, in the same representation, with no structural marker the model is obliged to respect. Everything separating them is convention, and convention is exactly what an attacker writes against.

There is no privilege bit on a span of text. Your system prompt and a paragraph from a stranger's PDF arrive in the same channel, in the same representation.

One word in the usual description of this attack is doing quiet damage: trust. The retriever did not decide the passage was true. It decided the passage was relevant. Those are two different functions, and only one of them was ever implemented.

Why does the retriever make this more likely, not less?

Because retrieval is a relevance function, and an injected passage is written to be relevant. The properties that make a ranker good are the properties that make it a delivery mechanism: an attacker who wants their paragraph read writes a paragraph that answers the likely query better than the honest documents do. Nothing about that requires an exploit. It requires copywriting.

We will be specific about our own architecture rather than someone else's. CourtNetra, our legal research platform over 18,863,754 Indian judgments, runs hybrid retrieval: a tsvector BM25 pass and a dense pgvector similarity search over halfvec(2560) embeddings produced by Qwen3-Embedding-4B on a self-hosted CUDA GPU pipeline, indexed under HNSW and reconciled by reciprocal rank fusion. Each leg is gameable, and the two are gameable by different text.

  • The dense leg rewards a good paraphrase. Cosine similarity favors a passage that restates the semantic content of the question. A paragraph that reads like the perfect answer to a common query sits close to that query in embedding space, which is the entire design intent.
  • The lexical leg rewards exact and rare tokens. A statute section number, a party name, a neutral citation. Those are cheap to include, and there is no semantic model in the path to notice that the surrounding paragraph is strange.
  • Fusion by rank is the right choice and it widens the door. Reciprocal rank fusion reconciles the two legs by rank position rather than raw score, so neither scoring scale can dominate the other. Read from the other direction, that is the same sentence as: neither scoring scale can veto the other. A chunk that only one leg loves still enters the merged set at a respectable position.

We would make that architectural choice again, and we would rather write the consequence down here than let a reader find it. Resistance to one weak scorer and resistance to one manipulated scorer are the same property, and you do not get to have the first without accepting the second.

Scale compounds it. The subset selected for vector retrieval carries 687,289 chunks under HNSW, a different and much smaller population than the full judgment corpus. Nobody has read all of them, nobody will, and any given query's top-k can contain a passage no human at the company has ever seen.

Rank fusion means neither scoring leg can dominate the other. Read from the attacker's side, that is the same sentence as: neither leg can veto the other.

Where does the hostile text actually come from?

From documents your organization did not write and did not review, which in most retrieval systems is nearly all of them. That is the honest answer, and it reframes the problem from a model problem into an ingestion problem.

  • Documents a motivated counterparty sends you. Filings, claim submissions, invoices, resumes, vendor questionnaires, support tickets. Every one of these has an author with an interest in the outcome, and every one of them ends up in a corpus somebody later points a retriever at.
  • Third party and public feeds. CourtNetra's auto-fetch engine reconciles hearings, orders and cause lists from 19,660 court endpoints. Every one of those sources is legitimate. The number is still the point: an ingestion surface with 19,660 entry points is not a closed set, and no review process scales across it.
  • Any format whose rendered layer differs from its extracted layer. White text on white in a PDF is invisible on screen and plain in extraction. HTML comments, alt attributes, document metadata, spreadsheet cells outside the print range, footnote apparatus, and OCR output from a scan that introduces text nobody wrote. The reviewer approves what they see. The retriever indexes what the parser returns.
  • History. A corpus covering 1950 to the present contains an enormous quantity of material that no current employee has read or will read. Historical content is not safer than new content; it is only less observed.

What can an injected document actually do?

Three classes of outcome, and the third is the one that matters most in regulated work while generating the least evidence that anything happened.

  • Exfiltration. The injected instruction steers output into a channel that leaves the building: a markdown image whose URL carries data, a tool call to an external endpoint, a link the user is invited to click. What leaves is whatever sits in context, which can include the system prompt, another matter's documents, or a credential that was passed in for a legitimate reason.
  • Action. If the model can call tools, an injected paragraph is a remote instruction to those tools. The blast radius is the union of every credential and every write the tool path holds, not the sensitivity of the document the payload arrived in. A read-only research assistant that happens to share a service account with a write path is a write path.
  • Steering. No secret leaves. No tool fires. The answer is simply different. A recommendation flips. A coverage determination softens. An authority the system would have surfaced is quietly not surfaced. Nothing in your monitoring registers an event, because from the system's point of view nothing went wrong: it retrieved documents, it generated an answer, it returned 200.

The first two classes eventually produce an incident, a log line, an alert. The third produces a decision. In legal, insurance and claims work it is also the class worth the most to an attacker, because the value of changing a determination is far higher than the value of reading a system prompt.

The third class does not look like an attack. It looks like the product working.

Does a hallucination gate catch this?

No, and understanding why is the most useful thing in this piece. An injected answer is not a hallucination. It is grounded.

Our published definition of grounded has three conditions, and a response passes only when all three hold: the assertion is supported by a retrieved passage, the support was actually in context for that specific response, and the citation resolves to the passage relied on. Work through an injected document against those three. The passage says the thing, because the attacker wrote it to say the thing. It was retrieved, because the attacker wrote it to rank. The citation resolves, because there is a real chunk with a real identifier sitting behind it. Grounded, attributed, checkable, and wrong.

The automated hallucination evaluation on CourtNetra runs weekly in CI and fails the build above 2%. That gate answers exactly one question: how often does the system assert something the corpus does not support. Injection inverts the premise the gate rests on. The corpus supports it. The corpus is the attack.

These are two controls against two threat models, and treating one as coverage for the other is a specific and common error. A firm with a grounding metric that believes it covers content integrity has bought one control and counted two.

The gate is not useless here, and we will not overstate the gap. If injected content is systematically steering answers, refusal behavior and the incorrect-refusal line will eventually move, because the distribution of what the system retrieves has changed. But it will move late, it will move by a small amount, and it will not name the cause. A weekly aggregate is not a detection mechanism for a targeted document.

A hallucination gate measures whether the model invented something. It does not measure whether the corpus lied.

What controls actually reduce it?

Ordered by how much they buy: provenance carried on the chunk, deterministic decisions moved out of the model, least authority on tools, egress control, and structural separation of data from instructions. The order matters because most teams start at the last one and stop there.

  1. Provenance at ingest, carried on the chunk
    Every chunk gets an origin, an ingestion date and a trust class, and the retriever can filter and rank on all three. This is a schema decision, which means it is the decision that gets more expensive every month it is postponed. It is also what makes every later control possible: you cannot down-rank untrusted content if nothing in the row says which content that is.
  2. Move the decision out of the model wherever it can be deterministic
    CourtNetra's Citator classifies an authority as good law, distinguished, doubted, partially overruled or overruled across a 165,000-entry AIR, SCC and SCR crosswalk. That is a lookup against a reconciled table. No sentence inside any retrieved document can change its answer, because no sentence inside a retrieved document is an input to it. The same logic sits behind the structured layer generally: a citation network of approximately 165,000 edges, 1,807 statute sections and 6,315 judge profiles. Every decision you move from generation to lookup is a decision that has left the injection surface permanently.
  3. Least authority on tools
    The question is not whether the agent can be steered. Assume it can. The question is what it holds when it is: which credentials, which writes, which spend, which network reach. Agentic triage over Splunk SIEM with LangGraph and Claude 3.5 Sonnet, which took SLA to approximately 90% and analyst throughput to 3x, is work Aniruddh Atrey did at INNEFU Labs before MetaMinds existed, and it was not a MetaMinds client engagement. The design lesson that carried forward is narrow and boring: scope what the agent may do before you tune how well it does it, because the second is reversible and the first is not.
  4. Egress control
    If the model cannot cause a network request, an injected exfiltration channel has nowhere to send. No auto-fetched images in rendered output, no unrestricted link targets, an allowlist on every tool that reaches outside. This closes an entire outcome class rather than reducing it, which is rare enough to be worth the features it costs you.
  5. Structural separation of retrieved content from instructions
    Delimit it, label it as data, never concatenate it into the instruction region. This is the control everyone implements first and it is the weakest of the five, because it depends on the model honoring a convention it is under no obligation to honor. It raises the attacker's cost. It does not close the class, and a vendor who presents it as the answer is selling you the cheapest item on this list as though it were the top one.

A sixth, which is really a scheduling decision: scan at ingest rather than at answer time. Ingest has no user waiting on it, so it can afford checks that a request path cannot: format-versus-extraction comparison, invisible-text detection, imperative-mood density in a corpus that should be declarative.

How do you test for it?

You plant the payload where the system will fetch it, not where a tester can type it. Any assessment that only exercises the chat interface is testing the wrong channel, and it will pass.

  • Build a canary corpus. Documents that look exactly like your real ones, in your real formats, each carrying a distinct marker and a distinct instruction. Insert them into a staging index. The test is falsifiable in the way a security test should be: either a marker appears in an output, or an egress attempt fires, or neither happens.
  • Test each ingestion path separately. The PDF extractor, the OCR path, the HTML fetcher and the direct upload each normalize text differently, and each preserves a different set of tricks. A payload that dies in one path survives in another, and an aggregate pass rate across all four tells you nothing about which.
  • Test retrieval separately from generation. Ask first whether the hostile chunk was retrieved at all, and at what rank. A payload that never enters the top-k is a completely different risk from one that ranks first and was ignored by the model this time, and only one of those two states survives a model version change.

The provenance of the offensive-security half of this should be stated precisely rather than implied. Penetration testing across 50-plus government web assets, with critical CVE exposure reduced 65%, is offensive security work Aniruddh Atrey delivered at INNEFU Labs under public sector engagement rules, before MetaMinds existed. It was not a MetaMinds engagement, no MetaMinds client is involved in it, and the percentage belongs to that program rather than to anything we would promise a reader. What transfers is a habit rather than a number: the attacker's input arrives wherever the system reaches for data, and a tester who only uses the interface is auditing the front door of a building that has a loading dock.

A second reference point on ingestion surface, also from before MetaMinds. At SaveLIFE Foundation, a prior employer of our founder and not a MetaMinds client, Aniruddh Atrey built a multi-stage Haystack v2 retrieval system using e5-base-v2 over three MongoDB Atlas vector and text index pairs, which reduced analyst lookup time approximately 70% across 100-plus reports per week. The detail that matters here is structural rather than numerical: a retrieval corpus assembled from reporting the operating team did not author is exactly the profile where the input an adversary owns is an ingestion path and not a chat box.

What none of this solves

Five limits, and the first one applies to every product in this category including anything we would build for you.

  • No control makes a model ignore text it can read. This is a class you reduce, not a class you close. Any vendor claiming to eliminate prompt injection is describing a filter, and a filter is graded on the examples its authors thought of.
  • Provenance can be wrong. A trust class applied at ingest is only as good as the ingestion that applied it. OCR strips formatting and can strip your marker with it. A source that was trustworthy last year can be compromised this year, at which point the label is actively doing harm by vouching for content nobody re-checked.
  • Testing is sampling. A canary corpus proves the paths you thought to test, in the formats you thought to test them in. It says nothing about the path a contractor added last month, and the useful output of the exercise is often the inventory rather than the findings.
  • Determinism has a ceiling. You can move a classification out of the model. You cannot move a summary out of it, and summarization over adversarial content is the case where injection and legitimate operation are hardest to tell apart.
  • The controls cost real things. A provenance schema on an existing corpus is a migration, not a feature flag. Egress control makes features you wanted impossible, and somebody will ask for an exception within a month. Ingest-time scanning adds a pipeline stage that has to be maintained by someone. We would rather state that here than have it arrive as a surprise in week three of an engagement.

Does the EU AI Act require prompt injection testing?

Not directly, and it is worth saying so plainly rather than selling the confusion. Article 50 of the EU AI Act has been in force since 2 August 2026, and providers of systems generating synthetic content, placed on the market before that date, have until 2 December 2026 to comply with the machine-readable marking duty in Article 50(2). Article 50 is a transparency and disclosure regime. It governs whether a person knows they are dealing with AI, and whether generated or manipulated content is marked as such. It does not govern whether your retriever can be poisoned.

The connection is evidentiary rather than legal. The artifacts an injection assessment produces are an inventory of every ingestion path with a named owner, a provenance model for the corpus, and a dated record of what was tested and what was found. Those are the same artifacts a buyer's security review asks for, and the same ones a governance program needs. Build them once and use them in three conversations.

On dates, since a great deal of published guidance is now wrong about them: Annex III high-risk obligations were deferred to 2 December 2027 and Annex I to 2 August 2028 by Regulation (EU) 2026/1744, the Digital Omnibus on AI, published in the Official Journal on 24 July 2026 and in force since 27 July 2026. The near-term pressure is transparency, not the full high-risk conformity regime. Anyone telling you otherwise is quoting a schedule that was replaced.

Where to start if you already run one of these

Three steps, in order, and the first one is free.

  1. List every path by which text enters your corpus, and put a name against each
    Not systems. Paths. The nightly feed, the upload form, the shared drive somebody pointed a connector at, the scraper a contractor wrote. Most organizations find at least one path nobody owns.
  2. Ask what field on a chunk records where it came from
    If the answer is that there isn't one, stop reading and go fix that, because every control worth having depends on it and the migration only gets more expensive.
  3. Run a canary corpus against staging, one ingestion path at a time
    Distinct markers, distinct instructions, retrieval measured separately from generation.

If the answer to “where did this chunk come from” is a shrug, that is the finding, and it arrives before any testing has been done. It is also the most common finding, which is the reason this piece exists.

Common questions

What is indirect prompt injection?
Indirect prompt injection is an attack in which instructions reach a language model inside content the system retrieved on its own behalf, rather than inside anything the user typed. The payload sits in a document, the retriever selects that document because it ranked well, and the application places it in the context window where the model reads it as ordinary text. It works because a model receives one flat token sequence with no privilege marker separating the system prompt, the user's question and a retrieved passage.
How is indirect prompt injection different from direct prompt injection?
Direct prompt injection is typed into the interface by the person using the system, which means it arrives at the one place a security team already inspects. Indirect prompt injection arrives inside a document the system fetched itself, so the attacker never touches the application at all. They write a document and wait for ingestion to find it. In a system that pulls from public sources or accepts submissions from counterparties, the adversary's cost of delivery is the cost of publishing.
Does a hallucination or grounding evaluation catch prompt injection?
No. An injected answer is not a hallucination, it is grounded. Under a three-condition definition of grounded, where an assertion must be supported by a retrieved passage, that support must have been in context for the response, and the citation must resolve to the passage relied on, an injected document satisfies all three: it says the thing, it was retrieved, and the citation resolves. A grounding gate measures whether the model invented something. It does not measure whether the corpus lied. These are two controls against two different threat models.
Why does hybrid retrieval widen the prompt injection surface?
Because each of the two retrieval legs is gameable by different text, and the fusion step is designed so that neither can veto the other. A dense vector leg rewards a passage that paraphrases the likely query well, which is a writing exercise rather than an exploit. A BM25 lexical leg rewards exact and rare tokens such as a statute section number or a citation, which are trivial to include. Reciprocal rank fusion reconciles the two by rank position so neither scoring scale dominates, which also means a chunk that only one leg ranks highly still enters the merged set.
Where does hostile text enter a retrieval-augmented system?
From four places, all of them outside your review process. Documents a motivated counterparty sends you, such as filings, claim submissions and vendor questionnaires. Third party and public feeds, where the number of sources defeats manual review. Any format whose rendered layer differs from its extracted layer, including white text in PDFs, HTML comments, alt attributes, document metadata and OCR output. And historical content, which is not safer than new content, only less observed.
What controls actually reduce indirect prompt injection risk?
Five, in descending order of value. Provenance carried on every chunk, so origin, ingestion date and trust class are fields the retriever can filter and rank on. Deterministic decisions moved out of the model into table lookups, because a lookup has no injection surface. Least authority on tools, since the blast radius is the credentials the tool path holds and not the sensitivity of the document the payload arrived in. Egress control, which closes the exfiltration class rather than reducing it. And structural separation of retrieved content from instructions, which is implemented first by almost everyone and is the weakest of the five because it depends on the model honoring a convention.
How do you test a RAG system for indirect prompt injection?
You plant the payload where the system will fetch it, not where a tester can type it. Build a canary corpus of documents that look exactly like your real ones, each carrying a distinct marker and instruction, and insert them into a staging index. Test each ingestion path separately, because a PDF extractor, an OCR path, an HTML fetcher and a direct upload each preserve a different set of tricks. Test retrieval separately from generation, because a payload that never enters the top-k is a different risk from one that ranked first and was ignored this time.
Does the EU AI Act require prompt injection testing?
Not directly. Article 50 of the EU AI Act has been in force since 2 August 2026, and providers of synthetic-content systems placed on the market before that date have until 2 December 2026 to comply with the marking duty in Article 50(2), but Article 50 is a transparency and disclosure regime. It governs whether a person knows they are dealing with AI output, not whether a retriever can be poisoned. The connection is evidentiary: an ingestion path inventory, a corpus provenance model and a dated test record serve a security review and a governance program equally well. An injection test is a security control and should not be sold as a compliance deliverable.

Written by Aniruddh Atrey, founder and CTO of MetaMinds. Published 3 September 2026 and verified 3 September 2026. CourtNetra figures are from a production system MetaMinds operates. The INNEFU Labs and SaveLIFE Foundation figures are work Aniruddh Atrey did at those employers before MetaMinds existed, and neither was a MetaMinds client engagement. No client is named on this site, and MetaMinds holds neither SOC 2 nor ISO/IEC 42001.

Next

The system itself

CourtNetra in full: hybrid retrieval over 18,863,754 judgments, 687,289 chunks under HNSW, a 6-layer corrective pipeline inside a 30-second budget, and the Citator.

Read the case study

The same gate, on your corpus

Hallucination and grounding audit, $5,900 over two weeks. An evaluation set built from your own documents, groundedness measured against it, and the failure modes returned with a rate attached instead of an impression.

See the engagement

Where the gate sits in governance

Hallucination measurement is one guardrail among several, next to policy, provenance, PII redaction and refusal design, and it is the one buyers ask about most.

See the governance work

Start here

Bring your evaluation and we will tell you where it is soft.

Thirty minutes with the engineer who built the gate described above. If you already quote a hallucination number, bring the method behind it rather than the number.

Hallucination and grounding audit $5,900, fixed price · Typical reply within one business day