Attributes and prompt capture

Attributes are the columns

A span is a row; attributes are its columns. Every filter, group-by and alert is a query over them, so the names and types have to be decided like a schema, not invented per call. Model, prompt version, tenant, feature, token counts, finish reason and cache hit are the columns most teams need first. The names are cheap to change before the first dashboard and expensive after. A schema that changes every sprint is not a schema; it is a set of nicknames.

Use the provider’s semantic conventions where they exist, especially for model and token fields. A dashboard built on gen_ai.request.model can be reused across services; one built on model_name in one place and model in another cannot. Stable keys are worth more than short ones. A convention is a contract with every future query. The cost of a rename is paid by every query written against the old key, usually at the worst time.

Cardinality is a budget

Every distinct value in a metric label creates a series, and every series costs memory and query time. A user id on a histogram is a denial-of-service attack you wrote yourself. Keep metrics on bounded dimensions such as model, feature and status; put request ids and prompt hashes on spans, where you look at one row at a time. The bill arrives as a slow dashboard, not as an error. A label with a million values does not just cost money; it makes every aggregation slower for everyone.

The same attribute can be safe in one backend and ruinous in another. Decide per destination: what the trace store indexes, what the metric store aggregates, and what never leaves the process. A useful rule is that anything unbounded is an identifier, and identifiers belong in traces. Write the rule down where the instrumentation is defined, or the next feature will break it. The trace store can afford identifiers because it finds one row; the metric store is built to add many.

Recording a generation with its usage

from langfuse import get_client, observe

@observe(as_type="generation")
def answer(question, context):
    reply = model.complete(question, context)
    get_client().update_current_generation(
        model="gpt-4.1-mini",
        input=question,
        output=reply.text,
        usage_details={"input_tokens": reply.usage.input, "output_tokens": reply.usage.output},
    )
    return reply.text

Capture the prompt in layers

Store the template id and version, the variable values, and optionally the rendered prompt. The first two are small, structured and safe to keep; the rendered text is what the model actually saw and is the only way to debug a template bug or a tokenizer surprise. Most teams need the first two on every request and the third on a sample. The template version is the unit of review: a diff there is a change you can discuss before production.

Keep the layers separate. A hash of the rendered prompt detects change without storing the text; the template version explains the intent; the variables explain the instance. When a customer asks what was sent, the layers let you answer without keeping every prompt forever. The hash also lets you deduplicate identical prompts without reading them. A hash is also a safe join key between a trace and a prompt registry that never stores the text.

Tool calls are prompts too

A tool result becomes model input on the next turn, so it deserves the same care as a prompt. Record the tool name, the argument keys, the result size and whether it was truncated. Do not record secrets in arguments; a key name is usually enough. Arguments are also where a prompt injection arrives, so their shape matters as much as their values. A tool that returns a PDF and a tool that returns three lines should not share one size budget.

Truncation is a common cause of confident wrong answers: the model sees the first part of a document and never the sentence that changes the meaning. A boolean attribute and the original size make that visible in one query. The same applies to retrieved chunks that were dropped by a reranker. Size caps are part of the record, not an afterthought. The flag is cheap; reconstructing why an answer missed the key sentence is not.