Posted inAI

Stop Leaking Data: Building a PII Redaction Gateway for LLMs

AI tutorial - IT technology blog
AI tutorial - IT technology blog

The Invisible Privacy Leak

Modern apps live and breathe third-party LLMs like GPT-4 or Claude. Every time an employee pastes a customer email or a medical query into a prompt, your company’s risk profile spikes. Imagine a user asking: “Can you summarize my medical report for John Doe, living at 123 Main St?” That sensitive data is no longer under your control. It now sits on a third-party server, creating a massive regulatory headache for GDPR, HIPAA, or SOC2 compliance.

This isn’t just about trusting AI providers. Data retention policies, model training on user inputs, and potential breaches all create liability. Enterprise clients won’t sign contracts if PII leaves your infrastructure. You need a way to strip the sensitive bits before they hit the open internet.

Why Regex Fails at PII Detection

Human language is messy. You could write a thousand Regular Expressions (Regex) to catch emails and phone numbers, but you’ll still miss names, addresses, or medical conditions. Regex is a blunt instrument. It works for fixed patterns like credit cards but fails when a user writes “John is the name I go by” instead of “My name is John.”

The solution is an AI Privacy Gateway. This middleman sits between your app and the external API. It uses Natural Language Processing (NLP) to find sensitive entities and swaps them with anonymous placeholders. Once the AI responds, the gateway restores the original data for the user.

I have deployed this architecture in production environments. It provides a level of security that simple string filtering cannot match. It’s the difference between a hard-coded filter and a system that actually understands context.

Quick Start: Redacting PII in 5 Minutes

Microsoft Presidio is the go-to open-source library for this task. It combines Spacy’s NLP capabilities with logic-based recognizers to identify sensitive data with high precision.

1. Install Dependencies

pip install presidio-analyzer presidio-anonymizer spacy
python -m spacy download en_core_web_lg

2. Basic Redaction Script

The following script identifies names and phone numbers, then replaces them with generic tags. It’s a clean way to ensure your LLM never sees the actual identity of your users.

from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine
from presidio_anonymizer.entities import OperatorConfig

# Initialize engines
analyzer = AnalyzerEngine()
anonymizer = AnonymizerEngine()

text_to_scrub = "My name is John Doe and my phone number is 212-555-1234."

# Analyze the text for PII
results = analyzer.analyze(text=text_to_scrub, entities=["PERSON", "PHONE_NUMBER"], language='en')

# Anonymize the detected entities
anonymized_result = anonymizer.anonymize(
    text=text_to_scrub,
    analyzer_results=results
)

print(f"Original: {text_to_scrub}")
print(f"Anonymized: {anonymized_result.text}")

Run this, and “John Doe” becomes <PERSON>. The phone number becomes <PHONE_NUMBER>. Your external LLM only sees the sanitized version.

How a Production Gateway Handles Re-identification

A basic script isn’t enough for a real-world app. If the AI responds with “Hello <PERSON>, how can I help you?”, your user will be confused. You need to reverse the process before the text hits the UI.

The Mapping Workflow

To restore data, maintain a temporary mapping of placeholders to original values. If your app is distributed, use a Redis cache with a short TTL (Time-To-Live). The workflow follows these steps:

  1. Interception: Capture the user prompt.
  2. Detection: Presidio finds “John Doe”.
  3. Substitution: Replace “John Doe” with a unique key like {{USER_0}}.
  4. API Call: Send the scrubbed prompt to OpenAI.
  5. Response: OpenAI returns a message using the {{USER_0}} key.
  6. Re-identification: Swap {{USER_0}} back to “John Doe” for the final display.

Implementation Example

def privacy_gateway_request(user_input):
    results = analyzer.analyze(text=user_input, language='en')
    
    operators = {
        "PERSON": OperatorConfig("replace", {"new_value": "{{USER_NAME}}"}),
        "EMAIL_ADDRESS": OperatorConfig("replace", {"new_value": "{{USER_EMAIL}}"})
    }
    
    scrubbed = anonymizer.anonymize(text=user_input, analyzer_results=results, operators=operators)
    
    # Store {{USER_NAME}} -> original name in a mapping table here
    return scrubbed.text

# Example usage
prompt = "Send an email to [email protected] about the project."
clean_prompt = privacy_gateway_request(prompt)

The 99.9% Reliability Layer: LLM Verification

Presidio is fast, but it can miss subtle PII. To bridge that gap, I often add a second pass using a small, local LLM like Llama 3 (8B) or Phi-3. These run on your own hardware, so no data leaks out.

Local models don’t need to be creative. They just need to be good editors. Use a prompt like: “Identify all names and IDs in this text. Output only a JSON list.” By combining Presidio’s speed with a local LLM’s reasoning, you catch typos that would otherwise bypass a standard NLP model.

Lessons from the Field

Deploying these gateways reveals a few practical realities. Keep these points in mind for your implementation:

  • Performance Overhead: Analyzing a 500-word prompt usually adds 150ms to 300ms of latency. This is often unnoticeable compared to the 2-5 seconds an LLM takes to stream a response.
  • Context Preservation: Sometimes the AI needs specific data to be useful. If you’re building a medical bot, redacting the symptoms makes the AI fail. Fine-tune your entity list to keep essential context.
  • Custom Patterns: Use Presidio’s PatternRecognizer for internal company formats. If your employee IDs look like “EMP-9988”, a custom regex ensures they never leave the building.
  • Safe Logging: Never log the raw user prompt. Only log the scrubbed version. If your logs contain PII, you’ve just moved the security risk from the AI provider to your own database.

Building a gateway is a one-time investment that slashes your risk profile. It gives you the freedom to use the world’s best models while keeping your data exactly where it belongs: under your control.

Share:
Tags: