Tokens, cost, latency, prompt and model version — in every trace span, following the OpenTelemetry gen_ai.* semantic conventions rather than home-grown names: otherwise dashboards and backends don’t understand your attributes. Those conventions now live in their own GenAI repository and are still at Development status, so pin the version you emit. Quality drift is caught by continuous evals in production, not by user complaints.
The conventions define span attributes (gen_ai.request.model, gen_ai.usage.input_tokens, …) and two client metrics, gen_ai.client.token.usage and gen_ai.client.operation.duration, so token spend and latency can be aggregated without reading spans. The reason to follow them rather than invent names is the same as for any telemetry contract (29): a backend can only build a cost dashboard over attributes it recognizes.
span.SetAttributes(
// standard gen_ai.* attributes — off-the-shelf dashboards understand them
attribute.String("gen_ai.provider.name", "anthropic"),
attribute.String("gen_ai.request.model", "claude-haiku-4-5-20251001"),
attribute.Int("gen_ai.usage.input_tokens", usage.In),
attribute.Int("gen_ai.usage.output_tokens", usage.Out),
// custom ones — only where no standard attribute exists
attribute.String("app.prompt_sha", promptSHA),
attribute.Float64("app.cost_usd", cost),
attribute.Bool("app.cache_hit", hit),
)
// Content (prompts, responses, tool arguments, retrieval queries)
// is NOT written to the span: the gen_ai conventions explicitly warn it holds PII.Content is a separate stream with different rules. Capture is off by default and enabled selectively; PII redaction happens before the exporter, not in the backend; length is truncated; sampling is enabled; metadata and content have different retention and ACLs; tenant separation is mandatory; secrets in span attributes are not allowed. A regular trace carries the prompt version (prompt_sha), not the prompt itself.
What it buys in an incident. Daily spend doubles overnight. With prompt_sha and token counts on every span, the question “what changed” is one group-by: a new prompt version that pulls three times as many retrieved chunks into context, or a cache hit rate that fell after a key format changed. Without them it is a week of guessing from the provider’s invoice. Cost is an SLI like latency (78), and it moves with the same deploys.
Limits. Telemetry says how much and how slow, not how good: a span cannot tell a correct answer from a fluent wrong one, which is why quality comes from evals on sampled production traffic (51). Sample traces, but keep every error and every refusal — rare failures are what sampling drops first.