Overview
Stream production conversations straight from your agent using the standard OpenTelemetry SDK, with no UserTrace SDK and no transcript uploads. Point an OTLP exporter at/v1/traces, add your API key header, and follow the
OTel GenAI semantic conventions.
Conversations are evaluated automatically as they close, and the results show up in
Production Results.
If you already run OpenTelemetry, you need two things: a second exporter pointed at
UserTrace, and
gen_ai.conversation.id on your spans. Everything else is convention you
may already emit.Before you start
Choose which evals run on ingested conversations. In the dashboard, open the Agent Setup page and scroll to the bottom, to the Live traces section. Until you do, conversations ingest and close cleanly but nothing is scored. Two settings there:Sending spans
application/json and application/x-protobuf, both of which stock OTel
exporters send unmodified, and Content-Encoding: gzip. Returns 200 with an empty OTLP
ExportTraceServiceResponse, which is what exporters expect to parse. Any other content
type is 415.
In OTLP/JSON,
traceId and spanId are hex strings, a deliberate deviation from
the usual protobuf-to-JSON mapping, which would base64 them. Stock exporters get this
right; only hand-built payloads need care.What we read
Span ids and timestamps come free from OTLP; you never set them yourself.
Spans without
gen_ai.operation.name are ignored. You can point an existing
TracerProvider at UserTrace and your HTTP, database and framework spans pass through
harmlessly.Your own metadata
Any attribute outside thegen_ai.* namespace, such as journey.id, tenant,
region or service.name, is captured as a searchable tag on the conversation. There’s no
UserTrace-specific convention here; they’re ordinary OTel attributes, on the span or on
the resource.
Retries are safe
Rows are keyed on the span id, so re-sending a batch inserts nothing. This matters because OTLP exporters retry on their own. At-least-once delivery is the norm, not an edge case.Closing a conversation
GET /api/prod-evaluations/sessions/?run=Live traces once they finish, typically within
seconds. Conversations are grouped into a daily run, so a day’s traffic is one row in
the dashboard rather than thousands.
A conversation left idle for 24 hours is closed automatically, through the same path
so a crashed client doesn’t strand its data. Calling
/end yourself just makes
results arrive sooner.