How do I make retrieval-augmented generation respect document permissions?

To make RAG respect document permissions, you need authorization at retrieval time, not just at generation time. The safest pattern is:

Core principle

A model should only see chunks from documents the current user is allowed to access.

That means permissions must be enforced in the retriever / vector search layer and ideally again in the post-retrieval filtering layer.

---

Recommended architecture

1) Attach access-control metadata to every chunk

When you ingest documents, store permission info alongside each chunk, such as:

• tenant_id
• document_id
• owner_id
• allowed_user_ids
• allowed_group_ids
• role
• classification
• ACL version
• expiry / embargo

Example metadata:

{
  "chunk_id": "c123",
  "document_id": "doc456",
  "tenant_id": "t1",
  "allowed_groups": ["finance", "legal"],
  "allowed_users": ["u789"],
  "classification": "confidential"
}

---

2) Filter before or during vector search

At query time, determine the user’s effective permissions, then constrain retrieval so only authorized chunks can be returned.

Ways to do this:

A. Metadata filtering in the vector DB

If your vector store supports metadata filters, use them:

• match tenant_id
• require allowed_groups intersects user groups
• require allowed_users contains user
• require classification <= clearance

This is the simplest and most common pattern.

B. Pre-filter candidate documents in a separate ACL system

For more complex permissions:

• query your document index / ACL database first
• get the set of authorized document IDs
• use those IDs as a filter in vector search

C. Hybrid retrieval with ACL-aware candidate generation

If you use keyword + vector retrieval:

• both retrievers should apply the same authorization filter
• merge only authorized results

---

3) Re-rank and validate again

Even with filtered retrieval, add a second check:

• after retrieval, verify every result is authorized
• if a result fails, drop it

This protects against:

• bugs in the vector DB filter
• stale ACLs
• incorrect metadata
• query-time permission changes

---

4) Pass only authorized context to the LLM

Never send unauthorized chunks to the model “just in case.”
If the model never sees them, it can’t leak them.

---

Common implementation patterns

Pattern 1: ACL in metadata

Good for:

• simple per-user / per-group access
• multi-tenant systems
• most enterprise RAG

Flow:

1. User logs in
2. App loads user groups/roles
3. Vector search includes ACL filters
4. Results are verified
5. LLM gets only allowed chunks

---

Pattern 2: Separate authorization service

Good for:

• complex policies
• inherited permissions
• document-level exceptions
• row-level security

Flow:

1. User query arrives
2. Auth service computes authorized doc IDs
3. Retriever searches only within that set
4. Post-filter for safety

This is more scalable when ACL logic is sophisticated.

---

Pattern 3: Security-trimmed index

Good for:

• very strict environments
• small to medium corpora
• high-assurance setups

You maintain separate indexes/views per security domain, e.g.:

• public index
• finance index
• HR index

This reduces the chance of leakage but increases operational overhead.

---

Important gotchas

Don’t rely on prompts

Telling the model “only answer from allowed docs” is not sufficient.
Prompt instructions do not enforce security.

Don’t retrieve first, filter later in the model

If unauthorized text reaches the model context window, it can leak.

Watch out for embeddings leakage

Even if you don’t show raw text, metadata or similar-chunk retrieval can still reveal sensitive info if permissions aren’t enforced.

Keep permissions and indexes in sync

If ACLs change, cached retrieval results or stale indexes can create exposures.

Handle prompt injection in documents

A malicious document can try to override instructions. Use:

• strong system prompts
• content isolation
• allowlisted tools
• document sanitization / quoting

This is separate from permissions, but important in RAG security.

---

Practical checklist

• Store ACL metadata with each chunk
• Compute user entitlements at query time
• Apply authorization filters in retrieval
• Verify retrieved chunks again after search
• Send only authorized chunks to the LLM
• Audit retrieval results and access decisions
• Re-index or update metadata when ACLs change
• Test for leakage with unauthorized-user queries

---

Minimal secure flow

1. User authenticates.
2. System resolves their roles/groups/tenant.
3. Retriever searches only chunks the user can access.
4. Retrieved chunks are checked again against policy.
5. LLM answers using only those chunks.
6. Logs record which docs were accessed.

---

If you want, I can show:

• a reference architecture
• a SQL + vector DB example
• or code for Pinecone / Weaviate / Elasticsearch / pgvector access control filtering.