Capstone: a guardrail pipeline

The pipeline shape

One request passes through stages in a fixed order: normalise, classify the input, retrieve with provenance, generate, validate the output, moderate, apply policy, log. Each stage can stop the request, and the record says which one did. The order is part of the design: a cheap deterministic check runs before an expensive model call.

The stages share a record, not a pile of return values. A decision object accumulates the text, the labels, the provenance, the output and the reason. When the request ends, the record is the audit trail, and the same object is what a test asserts on.

A stage should have one job and a clear stop condition. When a stage both classifies and rewrites, the record cannot say which behaviour fired, and the next person has to read the code to answer a question the log should have answered. One stage, one decision, and one line in the record that names it.

A stage that does not stop the request is still worth recording, because the absence of a decision is information. The retrieve stage does not block, but its provenance label is what lets the output stage know that the text it is judging came from outside the session.

The record is the product

A blocked request and an allowed one produce the same shape of record: request id, stage decisions, final action, reason code, policy version, latency. The payload is redacted. That symmetry is what lets an operator compare a block to an allow without reading two different formats.

The record is also how you debug a false positive. If it says the PII stage fired on a name, you can replay the normalised text and see which pattern matched. Without it, the only evidence is a user complaint.

Keep the record small enough to store and rich enough to replay. The redacted text, the labels and the reasons are usually enough; the full prompt is a liability with a retention cost. The record is evidence, not a copy of the request.

The record as a typed object

from pydantic import BaseModel, Field


class Stage(BaseModel):
    name: str
    decision: str
    reason: str
    ms: int = Field(ge=0)


class Record(BaseModel):
    request_id: str
    stages: list[Stage]
    action: str
    policy_version: str


record = Record(
    request_id="r-1",
    stages=[Stage(name="normalise", decision="pass", reason="ok", ms=1)],
    action="allow",
    policy_version="2026-10-01",
)

Where to put the human

Approvals belong where the effect is irreversible and the volume is low: a refund above a cap, a delete, a send to an external address. The approval screen must show the exact arguments and the provenance of each one, because a person approving an action they cannot see is a rubber stamp.

Appeals belong after the block. A user who thinks a refusal was wrong needs a path that does not require guessing the policy: a form, a human, a response with the request id. The appeal is also a source of false-positive cases for the suite.

An approval queue needs an owner and a timeout. A request that waits forever is a denial of service with extra steps, and a queue nobody reads is a rubber stamp with a delay. Both are worse than an honest refusal.

What to measure

Track block rate per stage, false-positive rate on the benign set, latency per stage and escape rate on the hostile set. A pipeline that adds 800 ms to every request is a pipeline that will be shortened, so put the cheap checks first and the model-based ones only where they earn their cost.

Review the record weekly. The distribution of reasons tells you which rule is doing the work and which one is dead. A rule that never fires is either unnecessary or broken, and both are worth knowing.

The latency budget is a product constraint, not an engineering preference. If the guardrail makes the feature slower than the alternative, the feature loses, and the pipeline is the first thing removed. Measure the cost of each stage before you add the next.