Troubleshooting permission issues in retrieval-aware AI usually means checking three layers:

1. Source access — can the user/service read the underlying data?
2. Retrieval access — is the retriever allowed to query/index that data?
3. Generation access — are you filtering what gets sent to the model and returned to the user?

Here’s a practical checklist.

1) Reproduce the issue with a known user

Identify:

• the exact user/service account
• the document/query that should be accessible
• the one that should be blocked

Then test with the same identity the app uses:

• end user OAuth token
• service principal / app token
• delegated vs. app-only permissions

A common bug is the app works for admins but not for normal users because it’s accidentally using elevated credentials.

2) Verify source-system permissions

Make sure the identity has read access in the source system:

• SharePoint / Google Drive / Confluence / Box / S3 / SQL database
• row-level or document-level permissions
• folder inheritance and explicit denies

Check:

• Can the user access the file directly?
• Can the service account access it?
• Are there nested groups or inherited ACLs?

If retrieval uses an index, confirm the crawler/indexer had access to the document at ingestion time.

3) Check ingestion and indexing

Permission bugs often start here.

Look for:

• documents indexed without ACL metadata
• stale ACLs after a document moved or changed owners
• deleted users/groups still referenced in ACLs
• sync jobs failing to refresh permissions
• partial ingestion that indexed content but not access labels

Useful question:
Does the index store per-document ACLs or security filters?

If not, the retriever may return unauthorized content even if the source is protected.

4) Inspect retrieval-time filtering

At query time, ensure the system filters candidates by permissions before ranking or before returning snippets.

Check whether:

• ACL checks happen before retrieval ranking
• permission filters are applied to both documents and chunks
• metadata filters are correctly translated into query constraints
• hybrid search merges authorized results only

A common failure:

• the system retrieves top-k chunks first
• then applies authorization after
• unauthorized content still influences rankings or appears in citations/snippets

5) Validate chunk-level vs document-level permissions

If a document is split into chunks:

• do all chunks inherit the document’s ACL?
• can some chunks have different access rules?
• are citations pointing to a chunk the user cannot open?

If permissions are document-level but retrieval is chunk-level, make sure every chunk carries the parent security metadata.

6) Examine identity propagation

In multi-service systems, the user identity may be lost between components.

Check:

• frontend → API gateway → retrieval service → vector DB → source connector
• JWT claims preserved?
• user/group info passed correctly?
• token exchange working?
• session impersonation enabled?

If the retriever only sees a generic app identity, it may overexpose content or under-return results.

7) Review model-side leakage

Even if retrieval is correct, the model can leak data from:

• conversation history
• cached prompts
• prior retrieved context
• shared memory across tenants

Verify:

• per-user session isolation
• no cross-tenant caching of retrieved passages
• no reuse of context from another request
• logs don’t store sensitive retrieved text unnecessarily

8) Check logs and audit trails

Look at:

• authorization decision logs
• document access logs
• retrieval query logs
• ACL evaluation logs
• denied vs allowed counts

Good logs to have:

• user identity
• query ID
• document/chunk ID
• reason for allow/deny
• source of ACL data
• timestamp of ACL sync

If you see “allow” with no reason or missing identity, that’s a red flag.

9) Test edge cases

Common permission edge cases:

• user is in multiple groups with conflicting rules
• document owner changed
• group membership recently updated
• deleted or disabled accounts
• external guests / federated identities
• public vs private content mixed in one source
• cached embeddings for revoked docs

Also test:

• exact same query as an admin vs normal user
• newly granted access not showing up
• revoked access still appearing

10) Confirm output filtering

Before returning an answer, ensure the app:

• strips unauthorized citations/snippets
• does not quote private text from blocked sources
• doesn’t reveal document titles if those are sensitive
• falls back gracefully when no authorized documents are found

If the answer says “I found 3 documents” but shows a blocked title, the output filter is incomplete.

---

Fast diagnostic sequence

If you want a quick path:

1. Try direct source access with the affected user.
2. Inspect indexed ACL metadata for the target doc/chunk.
3. Check retrieval logs to see whether the item was retrieved despite lacking access.
4. Verify permission filtering happens before ranking/response formatting.
5. Confirm identity propagation through every service hop.

---

Common root causes

• stale ACL sync
• missing ACL metadata on chunks
• app using service credentials instead of user credentials
• permission checks only at the UI layer
• no filtering after vector search
• group membership not expanded correctly
• cache not invalidated after revocation

---

Good preventive practices

• enforce authorization at retrieval time, not just UI time
• store and query ACLs alongside embeddings
• use deny-by-default
• audit every retrieval decision
• regularly re-sync permissions
• test with least-privilege accounts
• isolate tenant/session caches

If you want, I can also give you:

• a step-by-step debugging runbook
• a sample permission-check architecture for RAG
• or a checklist for specific platforms like SharePoint, Confluence, or Google Drive.