An error taxonomy

An exception is not a failure mode

A provider 429, a timeout, a schema violation, an empty retrieval, a refusal, a policy block and a user typo are all failures, and they need different responses. One boolean error attribute collapses them into a single number that nobody can act on. The first job of an error taxonomy is to give each class a name and an owner, so the alert reaches the person who can change it. The taxonomy should be small enough that an on-call engineer can name the class from memory.

Classify at the boundary where you still know the context: the HTTP status, the exception type, the validator that rejected the output, the retriever that returned nothing. Once the request is a row in a warehouse, the cause is a guess. Record the class, the provider code and a short message; keep the stack trace in the log, not in the metric label, where it would explode the series count. The HTTP status is often the only reliable signal; provider messages are prose, and prose changes.

Retryable is a property of the class

Rate limits and server errors are usually transient; a 400, a schema error and a refusal are not. Retrying a malformed request spends money to get the same answer. Retrying a refusal without changing the prompt is the same mistake with more latency. The distinction belongs in the taxonomy, not in a comment beside the retry call. A timeout is ambiguous: the call may have succeeded on the server, so a retry can duplicate an effect.

Encode retryability next to the class, not in the message. A retry policy that parses strings is a policy that breaks when a provider rewords an error. Cap the attempts, add jitter, and count the retry as a separate span so the cost and latency are visible. A retry that is invisible is a cost that nobody budgets. The retry span should carry the attempt number and the delay, or the tail latency has no explanation.

Retry the class, not the string

from tenacity import retry, retry_if_exception_type, stop_after_attempt, wait_exponential_jitter

@retry(
    retry=retry_if_exception_type(RateLimitError),
    stop=stop_after_attempt(3),
    wait=wait_exponential_jitter(initial=1, max=10),
)
def call_model(prompt):
    return client.complete(prompt)

Record the outcome the user saw

The internal class and the user-visible outcome are different. A 429 that succeeded on the second attempt is not a user-visible error; a fallback answer is a degraded success; a refusal shown as an apology is a product outcome, not an exception. Track both, or the error rate will disagree with support tickets. The gap between them is usually where the interesting bugs live. A user who is told the answer is degraded can decide whether to trust it.

Define success per feature before you build the dashboard. For a support bot it might be an accepted answer with a citation; for a code assistant it might be a patch that applies. The definition is a product decision, and it belongs in the same document as the error classes. A dashboard built on a vague definition will be argued with rather than used. The definition should be narrow enough to compute and broad enough to match what the user expects.

Error budgets and SLOs

An error budget is the amount of failure a service is allowed in a window. It turns reliability into a number the team can spend: if the budget is exhausted, feature work pauses until the cause is fixed. The budget is only meaningful if success is defined and measured on every request, not estimated from a sampled trace set. A budget that nobody has ever spent is probably not being measured.

Retries convert failures into latency, which is still a cost. A service that retries three times and succeeds has a 100 percent success rate and a p95 that includes the retries. Measure both, and alert on the user-visible outcome rather than the provider status. A dashboard that shows only the final status will hide the incident until the retries stop working. Alert on the rate of requests that needed a retry, not only on the ones that ultimately failed.