Skip to main content
Elementum can send traces of its AI agents to your OpenTelemetry backend or to an AI evaluation platform, such as W&B Weave. Each agent turn arrives as a small tree of spans: the agent invocation, every call to the language model, and every tool the agent ran. The spans follow the OpenTelemetry GenAI semantic conventions, so the gen_ai.* attributes carry the model, token usage, the messages sent to the model, and its reply. This page describes the spans and what they contain. For every attribute, see the GenAI attribute reference. To use the traces for evaluation, see Evaluating AI with traces.
LLM call and tool spans carry the full text of prompts, replies, tool arguments, and tool results. Both GenAI and General endpoints receive them. Read Prompt and response content before you connect an endpoint.

Which endpoint receives what

A GenAI endpoint receives only the spans that describe AI interactions. A General endpoint receives everything a GenAI endpoint does, plus automation spans and other AI activity spans. Use a GenAI endpoint when the receiver is an evaluation platform that should see only model and agent activity. Both endpoint types are set up the same way. See Add a GenAI endpoint.

Span types

Span names include the agent name, model, or tool name when Elementum knows it. Otherwise the name falls back to a fixed name: To group spans, filter on gen_ai.operation.name or openinference.span.kind, not on the span name. Span names change with the model and the tool. All of these spans have kind INTERNAL, resource service.name = elementum, and instrumentation scope elementum, like automation spans.

Span tree

One agent turn in which the model called one tool:
  • Every LLM call and tool run of the turn is a direct child of invoke_agent. They are siblings, in start-time order.
  • A turn with no tool calls has one chat span. A turn with several tool rounds alternates chat and execute_tool.
  • When the model asks for several tools in one reply, one execute_tool span covers all of them. Its gen_ai.tool.* attributes describe only one of those tools. The full list is in the tool_calls of the previous chat span’s gen_ai.output.messages.

Parents on each endpoint type

Elementum sends each endpoint a tree made only of the spans that endpoint receives. A span’s parent is its nearest ancestor that the endpoint also receives. Agent turns of one conversation usually arrive as separate traces. They share gen_ai.conversation.id. Use that attribute to see a whole conversation. When one agent calls another over A2A, the called agent’s invoke_agent is a separate root on a GenAI endpoint. It is not nested under the calling agent. Use gen_ai.conversation.id and timing to connect them.

Voice calls

Voice agents produce spans too. Two kinds of call exist, depending on how the voice agent is set up.

Calls through Elementum’s voice provider

Realtime voice calls

Each realtime call starts a new trace:
Realtime spans carry timings, turn outcomes, and model details, but no transcript text. Voice spans also carry voice.* attributes. See Voice attributes.

Span status and errors

On a failed span, elementum.error.type is the class name of the error and elementum.error.message is the error’s message text. This differs from automation spans, which carry only a fixed label. An AI error message can quote details from the model provider’s response. A failed tool run has status ERROR and a span event, tool.execution.failed, with the tool name, the kind of failure, and its duration.

Span events

Unlike automation spans, AI spans can carry span events. They mark progress steps of the turn, for example message.llm.call.completed. Their names and attributes are not a stable contract and can change. Events that Elementum marks as sensitive are never sent. Exception events and stack traces are never sent.

Prompt and response content

GenAI spans exist so you can evaluate what your agents said. They carry content. How content is handled:
  • Content is always included. There is no setting to send these spans without it. To keep content out of a backend, drop or scrub the content attributes in an OpenTelemetry Collector in front of that backend, or do not forward to it.
  • General and GenAI endpoints receive the same content. A General endpoint that receives AI spans stores prompts and replies too.
  • Elementum does not redact personal data from prompts, replies, or tool results. Tool results can contain record data that the tool read.
  • Voice PINs are masked. In voice transcripts and summaries, PINs a caller gives are replaced by a mask. Each transcript message is cut to 1,000 characters, and tool arguments are cut to 2,000 characters.
  • Very long values can be cut off. Elementum limits the size of one attribute value. A very long conversation history can arrive truncated.

What is never sent

  • Elementum internal platform spans: database calls, queues, and internal service calls.
  • Organization and environment ids. Spans are routed by them, but they are not attributes.
  • Inputs and outputs Elementum records for its own use on other AI activity spans.
  • Exception events and stack traces.
  • Attributes that are not in the GenAI attribute reference. Elementum sends only attribute keys under gen_ai., elementum., voice., llm., openinference., input., and output..

Personal data

Treat any endpoint that receives AI spans as a store of conversation data. Check its access controls, region, and retention before you connect it, and apply the same rules you apply to conversation history. Next: GenAI attribute reference.