PII leakage detection LLM

If you want an LLM-based PII leakage detector, the usual goal is to identify text that may contain personal data such as:

• Names
• Email addresses
• Phone numbers
• Physical addresses
• SSNs / national IDs
• Account numbers
• IPs tied to individuals
• Medical, financial, or other sensitive identifiers

Practical approaches

1. Rule-based + regex first

   • Fast and precise for emails, phones, SSNs, credit cards, etc.
   • Good as a first filter.

2. LLM classifier

   • Ask the model to label text as:
     • contains_pii: yes/no
     • pii_types: [...]
     • evidence: [...]
   • Works better for contextual PII, e.g., “My doctor is Dr. Smith” or “Jane lives at...”.

3. Hybrid system

   • Use regex/entity recognition first.
   • Then use an LLM to catch ambiguous or context-dependent cases.
   • This is usually the best production setup.

Example prompt for an LLM

You are a PII detection assistant.
Analyze the text and determine whether it contains personally identifiable information.
Return JSON with:
- contains_pii (boolean)
- pii_types (array of strings)
- spans (array of {text, type, reason})
- confidence (0 to 1)

Text:
{{input_text}}

Example output

{
  "contains_pii": true,
  "pii_types": ["email", "phone_number", "person_name"],
  "spans": [
    {"text": "john.doe@example.com", "type": "email", "reason": "Email address"},
    {"text": "+1 415 555 1234", "type": "phone_number", "reason": "Phone number"},
    {"text": "John Doe", "type": "person_name", "reason": "Person name"}
  ],
  "confidence": 0.98
}

Important caveats

• LLMs can miss subtle PII or produce false positives.
• For compliance use cases, do not rely only on the LLM.
• Combine with deterministic detectors and policy rules.
• If you’re handling regulated data, ensure logs/prompts don’t themselves leak PII.

If you want, I can help you with any of these next:

1. a production-ready PII detection prompt
2. a Python implementation
3. a regex + LLM hybrid design
4. a JSON schema for detection output