Input filtering and PII

Normalise before you match

A filter sees bytes, not intentions. Before a pattern runs, decode the text: Unicode NFKC folds compatibility forms, a strip removes zero-width and bidirectional control characters, HTML entities become their characters, and whitespace collapses. A cap on length comes first, because a megabyte of input is a denial of service before it is an injection.

The reason is adversarial, not cosmetic. An attacker who can choose the encoding chooses whether your regex matches: the word ignore with a zero-width space between two letters is the same word to a reader and a different string to a matcher. Normalise once, keep the original for display, and match on the normalised copy.

Length caps are also a cost control. The model charges by the token, the classifier charges by the character, and an input rejected before either runs costs almost nothing. Put the cap at the edge, not inside the prompt, where it is already too late.

Find PII, then decide

PII is not one thing. There are structured identifiers a regex can find — email, phone, card, IBAN, national id — quasi-identifiers that need context — a name, an address, a date of birth — and secrets that should never be in a prompt at all — API keys, session tokens, private keys. Match the structured ones exactly, including the checksum, and use context for the rest.

Then decide what to do, per category, before the request leaves. Blocking is the safest and the most annoying; redaction is irreversible and simple; tokenization keeps a mapping so the answer can be rehydrated. The one rule that is not a trade-off is the log: record the decision and the category, never the raw value.

The checksum is not a luxury. A sixteen-digit order number and a card number have the same shape, and only the checksum separates them. A detector without it produces the false positives that get the detector removed, which is worse than no detector.

PII detection with Presidio

from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine

analyzer = AnalyzerEngine()
anonymizer = AnonymizerEngine()

text = "Email [email protected] or call +44 20 7946 0958 about card 4111 1111 1111 1111."
results = analyzer.analyze(text=text, language="en")
clean = anonymizer.anonymize(text=text, analyzer_results=results)
print(clean.text)

What filtering cannot do

Filtering reduces accidental leakage and raises the cost of an attack. It does not stop a paraphrase, a translation, an image, a base64 blob or a patient attacker with many turns. A block-list of phrases is a sample of the space, and the space is open. Treat a filter as one layer, never as the boundary.

False positives are a real product cost. A name detector that redacts May breaks every date; a card regex that ignores the checksum redacts order numbers. Measure the false-positive rate on real traffic, because a guardrail that makes the product unusable is a guardrail that gets switched off.

Filtering also has to run on the way out. A model can repeat a secret that was in its context, and an output filter is the last chance to catch it before the user or the log sees it. Input and output are two doors, and both need a lock.

The data you keep

Prompts and completions are data. They sit in your logs, your provider’s logs, your evaluation set and sometimes your analytics. Set retention deliberately, redact before the request leaves, hash identifiers you only need to join on, and allow-list the fields that may enter a prompt at all.

PII inside an embedding is still PII. Deleting a record has to reach the vector index and the cache, or the deletion is a promise the system does not keep. The same is true of a fine-tune: once the data is in the weights, deletion is a retrain.

Ask the provider what it retains and for how long, and put that answer in the design document. A zero-retention agreement is a feature of the vendor, not of your code, and it can change with a plan or a policy update. The design should survive that change.