Why AI Observability in a Media Pipeline Is Never a One-Time Setup

Your media pipeline ran ten thousand times last night. Background removal, captioning, thumbnail generation, each one fanning out to a handful of inference endpoints per request. This morning you cannot say what it cost, which model version it actually hit, or which of those steps quietly blew its latency budget on the requests that timed out. The work happened. The evidence did not.

That blind spot is what AI observability fixes. The job is concrete: capture a structured trace for every model call, then use those traces to answer three questions. What did this call cost, how long did it take, and did the output meet quality expectations. The part worth planning for up front is that this is not a thing you set up once and forget. Change one model or one agent and the whole flow can behave differently, so the instrumentation that catches it is a standing capability, not a one-time install.

Full disclosure: Layermetry is one of the off-the-shelf layers that ships this tracing baked in, so we will lay out both paths, building it yourself and adopting a layer, and let you decide. First, the actual mechanics, because you need to understand what you are choosing to own.

TL;DR

• Emit one OpenTelemetry GenAI semantic conventions span per model call, with gen_ai.operation.name, gen_ai.request.model, gen_ai.usage.input_tokens, and gen_ai.usage.output_tokens at minimum.
• The GenAI conventions are in Development (experimental) status as of mid-2026. Pin your instrumentation to a specific semconv version and set OTEL_SEMCONV_STABILITY_OPT_IN to avoid silent dashboard breakage on upgrade.
• Separate SLO tiers for synchronous and async media jobs. Interactive thumbnail APIs and batch video transforms have latency budgets that differ by an order of magnitude.
• Prompt and image content capture is off by default. Treat it as a privacy subsystem, not a flag you flip in development.
• Cost translation (token counts to dollars) and media-specific quality scoring both require extension attributes beyond the standard gen_ai.* set.
• This is infrastructure you either own forever or adopt as a layer. The close lays out the honest cost of each.

One OTel span per model call is the right unit of observability

The unit of observability is one OTel span per inference call. When a media request arrives, the pipeline span (pipeline.request) acts as the parent, and each downstream model call becomes a child span.

The attributes that live on every leaf span

The gen_ai attribute registry defines what lives on each leaf span. The spec marks gen_ai.operation.name and gen_ai.provider.name as required, gen_ai.request.model as conditionally required when it is available, and gen_ai.response.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, and gen_ai.response.finish_reasons as recommended. For multimodal inference, gen_ai.output.type is conditionally required, and you set it to its "image" value when a call returns an image. For agentic calls, the canonical hierarchy is invoke_agent at the top, chat spans per LLM call, and execute_tool spans per tool invocation.

All of these attributes are in Development (experimental) stability. They now live in the dedicated open-telemetry/semantic-conventions-genai repository after splitting from the main semconv repo. Attributes in the old registry location are marked deprecated and redirect there.

ONE TRACE, MANY SPANS

A single media request fans out into one gen_ai span per model call; attributes live on the leaf.

One well-formed span gives you cost, latency, and provenance in one place

Emit one well-formed span per call and you get cost, latency, and provenance together across every model the pipeline touches. See the concept reference to understand how these attributes structure your observability stack. Two histogram metrics pair with these spans: gen_ai.client.operation.duration (seconds) for latency and gen_ai.client.token.usage (filterable by gen_ai.token.type) for token volume.

Synchronous and async media jobs need separate SLO tiers

Synchronous and asynchronous media jobs sit in fundamentally different SLO tiers, and confusing them leads to either over-provisioned infrastructure or alert fatigue.

An interactive API and an overnight batch differ by an order of magnitude

An interactive thumbnail API, where a user waits on the result, needs a TTFT (time to first token) P99 (99th-percentile) budget in the sub-second range. Vendors commonly cite 300ms as the target for interactive use cases. A background batch job transcoding thousands of assets overnight can use a P99 budget an order of magnitude looser. See our SLO guidance for how to apply these tiers to your own pipeline.

Measure goodput, not raw throughput

The metric that captures real service quality is goodput: requests per second that actually meet the SLO, not raw throughput. Track it alongside gen_ai.client.operation.duration histograms.

For per-token decomposition, the time-per-output-token formula has two parts. Take the end-to-end latency, subtract TTFT to isolate the decode phase, then divide that by the total output tokens minus one. Use it to distinguish prefill-bound from decode-bound slowdowns.

Page on burn rate, not on every breach

Set the on-call trigger at a 14.4x burn rate. If a service is consuming its weekly SLO error budget in under 12 hours, that is the threshold for a page. Anything slower than that is a burn-rate conversation in the next business-hours review.

There is also inter-token latency (ITL), which matters most for streaming responses. Vendor targets for interactive chat land around 50ms P99. For batch transforms with no streaming, ITL is irrelevant.

Prompt and content capture requires a privacy subsystem, not a flag

Prompt and completion content is not captured by default in OTel instrumentation, and that default is correct for production. Opt in explicitly using a capture_message_content flag in your instrumentation config, and do so only after a PII (personally identifiable information) scrubbing layer is in place.

Sanitize in one wrapper, not at every call site

The cleanest architecture implements a single instrumentation wrapper that sanitizes user-supplied text and image metadata before any content leaves the request path. Do not scatter PII logic across individual call sites. When you do capture content, store it as span events rather than long-lived indexed span attributes. Indexed attributes flow into your backend's search index and may be retained indefinitely. Span events are a lighter-weight record, less likely to surface in broad queries. Our tracing infrastructure guide covers these instrumentation patterns in depth.

Sample at three layers

Apply sampling at three layers. Run 100% tracing in development and staging. In production, head-based sampling at 10 to 30 percent keeps tracing infrastructure costs manageable. Reserve 10 to 20 percent of production traffic for LLM-as-judge eval scoring. Then use the OTel Collector tail sampling processor to retain full-content traces selectively. The latency, status_code, numeric_attribute, and ottl_condition policies let you keep only error traces and slow outliers for forensic review. Key configuration parameters are decision_wait (default 30 seconds) and num_traces (default 50,000 in-memory).

The gen_ai.* spec is text-shaped, so media pipelines add a media.* extension namespace

The OTel GenAI semconv is text-and-token shaped. gen_ai.output.type="image" exists and covers the basic signal that a call produced an image output, but the spec defines nothing for image dimensions, codec, transform operation type, output format, or visual quality score. That is not a gap the spec is likely to fill quickly. The convention is designed as a cross-provider lingua franca for inference calls, not a domain-specific media taxonomy.

Name your media attributes the way OTel names its own

The OTel project's own guidance on how to name your span attributes uses the pattern {domain}.{component}.{property} and reserves the otel.* prefix exclusively for the project itself. Following that pattern, we recommend a media.* custom namespace alongside your gen_ai.* attributes. Examples: media.image.width, media.image.height, media.transform.type, media.output.format. These are authorial recommendations, not published OTel conventions. Declare them as your own extension and document the namespace in your team's internal semconv registry.

Cost in dollars is a custom attribute too

Cost attribution requires a custom attribute. Translate gen_ai.usage.input_tokens and gen_ai.usage.output_tokens to dollars by multiplying by provider-specific pricing via a pricing_json configuration, a mechanism that open-source instrumentation libraries and managed observability clouds both support. Then attach the result as a custom attribute such as gen_ai.usage.cost_usd. That attribute is not in the OTel spec, so flag it as such in your schema documentation so future tooling authors don't mistake it for a standardized field.

Quality scoring lives outside the spec entirely

OTel is the data plane for your inference calls. Quality scoring, toxicity detection, and hallucination checking are not in scope for the spec. An external evaluation layer is required, and that is where the 2026 tool landscape fills the gap. See the evaluation layer architecture for more detail on orchestrating LLM-as-judge scoring across your pipeline.

The tracing tools form complementary layers, not competing choices

Think of the stack as complementary layers rather than competing products. OTel GenAI semconv is the lingua franca, so every span you emit in this format can flow into any backend.

Auto-instrumentation, evaluation, and metrics are different jobs

Open-source auto-instrumentation libraries implement the semconv across many providers and frameworks so you do not hand-write every span. On top of that sit the evaluation and UI layers. Several of these projects are backed by larger vendors through 2026 acquisitions yet remain permissively open-source (Apache 2.0 and MIT among them), with their managed clouds continuing as standalone services. The practical takeaway is that the open-source core stays available regardless of who owns the company.

Mature open-source eval-and-tracing platforms in this category cover tracing, cost tracking, and LLM-as-judge scoring in a single self-hosted stack, with adoption running into the tens of thousands of stars and usage across a large share of the largest enterprises. Others are OTel-native and ship built-in judges for faithfulness, toxicity, and hallucination.

Watch for two convention families before you build dashboards

One operational note matters more than the brand on the box. Some instrumentation layers use an earlier semantic-convention family that predates OTel GenAI and differs from the OTel GenAI span taxonomy. Pick one convention family and standardize across the pipeline before you start writing dashboards, so your panels agree about what a span means.

Attach eval scores to the same spans

For quality evaluation, attach LLM-as-judge scores to spans using gen_ai.evaluation.score.value (a double attribute) and gen_ai.evaluation.name (a string attribute identifying the metric). Evaluation techniques to consider are G-Eval (custom natural-language criteria returning a 0 to 1 score), DAG-based metrics (deterministic branching logic), and Arena G-Eval (pairwise comparison). Run judges on 10 to 20 percent of production traffic for statistically meaningful signal, and cross-validate judge scores against human annotations before treating them as authoritative in automated alerts. Standard open-source metrics, time-series, and trace backends plus the OTel Collector handle the infrastructure-level metrics for the inference servers themselves.

So do you own this, or adopt a layer that ships it?

Here is the whole tension in one line. Everything above is real, ongoing infrastructure. The span schema, the SLO tiers, the privacy subsystem, the cost-translation logic, the eval layer. None of it is a one-time setup, because the conventions are still experimental and your pipeline keeps changing underneath it. So the honest question is not how to instrument a model call. It is whether instrumenting and maintaining this for the life of your product is where your team's effort should go.

The honest side-by-side

This is the table to argue over in a roadmap meeting. It is not built to make one column win. It is built so you can see where your effort goes on each path.

Figures and conventions current as of June 2026. The GenAI semconv is experimental, so verify at source before committing a schema.

Where Layermetry fits as one of the adopt-path options

This is where the disclosure from the top pays off. Layermetry routes media work like background removal, captioning, and thumbnail generation through multiple inference endpoints, and it emits the OTel GenAI spans for those calls as part of the layer rather than as a roadmap promise. The cost, latency, and provenance for each model call are there because that is what the layer is, not something your team stands up and then maintains as the spec moves. For a team that needs observable model calls in a media pipeline this sprint, without owning the span schema and the eval plumbing forever, that is the adopt path made concrete. It is one option in the decision, not the only one.

Both answers are honest

Build it yourself if observability is your edge, if compliance forbids third-party runtime code, or if you want total control over every attribute and judge. On that path you carry zero third-party dependency, and you pay for it with the standing cost of owning an experimental, moving target. Adopt a layer if model-call observability is plumbing rather than the product, and you would rather spend your roadmap on what differentiates you. On that path you ship this sprint and hand the upkeep to a vendor, and you accept a bounded surface and a live dependency in return. The right answer is whichever column above describes your team.

FAQ

What OpenTelemetry attributes should I add to every model call in a media pipeline?

At minimum, emit gen_ai.operation.name, gen_ai.provider.name, gen_ai.request.model, gen_ai.response.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, and gen_ai.response.finish_reasons on every inference span. For media work, set gen_ai.output.type to its "image" value where it applies, then layer on custom media.* attributes (image dimensions, transform type, output format) following the OTel {domain}.{component}.{property} naming rule. Treat the media.* names as your own recommendation, not a published standard.

Are the OpenTelemetry GenAI semantic conventions stable enough to build on in 2026?

They are in Development (experimental) status as of mid-2026, with no published stabilization timeline, and they now live in the dedicated open-telemetry/semantic-conventions-genai repository. You can build on them, and a defensive setup keeps you steady. Set the OTEL_SEMCONV_STABILITY_OPT_IN environment variable and pin your instrumentation to a specific semconv version so a convention change does not silently break your dashboards on the next upgrade.

How do I capture prompt and output content safely without leaking PII?

Keep content capture off by default, which is how OTel instrumentation ships. When you do need content, sanitize PII inside the instrumentation wrapper rather than at each call site, and store raw content as span events instead of long-lived indexed span attributes. Sample aggressively (10 to 30 percent head-based in production) and use the Collector tail sampling processor to retain full content only for error or slow traces.

Should I build model-call observability myself or adopt a layer that ships it?

Both are honest. Building it yourself gives you total control and no third-party dependency, and you pay for it with the standing cost of owning the span schema, the cost-translation logic, the privacy subsystem, and the eval layer as the conventions and your pipeline keep changing. Adopting a layer that ships tracing baked in gets you instrumented model calls this sprint and hands the upkeep to a vendor, and you accept a bounded surface and a live dependency in return. The deciding factor is whether observability is your edge or just plumbing you would otherwise build and maintain forever.

Instrumenting model calls is infrastructure work, not a dashboard exercise. The spans and metrics described here form a base layer that cost accounting, SLO enforcement, and quality evaluation all build on, and that base layer is something you own and maintain, or adopt and let someone else maintain. Get the span schema right first and the rest of the observability stack has somewhere to attach. To see how these tracing patterns fit into a complete media pipeline, the Layermetry architecture docs walk through the full integration.