No spans arrive
Work through these in order.- Test the endpoint. In
Organization Settings > Platform > Trace Forwarding, select Test Connection on the endpoint. If it fails, fix the URL or credentials first. For HTTP, the URL must include the traces path, for examplehttps://otlp.example.com:4318/v1/traces. - Find the test span. Search your backend for a span named
elementum.connection-test. If the test says Message delivered but you cannot find the span, your backend accepted it and stored it somewhere else. Check the dataset, index, tenant, or project the credentials point to. - Check the endpoint type. Automation spans go only to General endpoints, never to GenAI endpoints. A GenAI endpoint receives spans only when an AI agent runs.
- Check that the endpoint is enabled. A disabled endpoint receives nothing.
- Check the environment. Open
Organization Settings > Platform > Environments, select the environment the automation runs in, and confirm the endpoint is listed under Trace Forwarding. Endpoints are attached per environment. - Wait a minute. Configuration changes can take about a minute to apply. Spans usually arrive within seconds after that.
- Run something that produces spans. Trigger an automation and look for
service.name = elementum. Spans are sent as each step ends, so a run that is still in progress may show only some spans. - Check your endpoint’s response time. Elementum waits up to 5 seconds for each batch. If your collector or backend is slower than that, batches are dropped. Put an OpenTelemetry Collector in front of a slow backend if you need more time to accept each batch.
Some spans are missing from a run
The Elementum spans are not attached to my span
You senttraceparent on a call to Elementum, but the Elementum spans are a separate trace or a separate root.
- Use a webhook trigger. Only webhook triggers continue a caller’s trace. Other APIs and trigger types start their own trace.
- Check the header. It must be a valid W3C
traceparent:00-<32 lower-case hex>-<16 lower-case hex>-<2 hex>, with a trace id and span id that are not all zeros. Invalid headers are ignored, and the webhook is not rejected. - Check that the parent span exists. The trigger’s parent is the span id in your header. If your application did not export that span, your backend shows the trigger with a missing parent.
- Check your sampling. If your application sampled its own span out, your backend has only Elementum’s spans. Elementum always sends its spans, whatever the sampled flag says.
The workflow span is not under the trigger span
For runs that did not continue your trace, the workflow span can arrive without a parent, as a second top-level span in the trigger’s trace. This is current behavior. Its span link points to the trigger, and every action span carries the run id, so the run is still complete. See Parenting of the workflow span.The partner system’s spans are not under the Elementum call
- Turn on Include Trace Header on that Send API Request step and publish the workflow.
- Confirm the partner reads
traceparent. Check the request it received. - A redirect to a different host, scheme, or port drops the header.
- If you also added trace headers on the step, the option replaces them with Elementum’s
traceparent. - The partner must send its spans to a backend where you can see them.
Duplicate spans
Organization or environment ids are missing
Elementum does not put organization or environment ids on spans. Spans are routed by organization and environment, so everything an endpoint receives comes from the environments attached to it. To label spans, add resource attributes on the endpoint, for exampledeployment.environment = production. Use one endpoint per environment if you need different labels.
AI agent spans
Sampling
- Elementum sends every automation span it records. There is no sampling on Elementum’s side for those spans, and they are always marked as sampled.
- The sampled flag in a
traceparentyou send does not change what Elementum sends. - Sampling in your collector or backend applies to Elementum spans like any others. With tail-based sampling, keep traces that contain
service.name = elementumspans with errors.
Volume
Expect one trigger span per trigger firing, one workflow span per run, one span per action attempt, and one client span per Send API Request call. A busy automation on a record-update trigger can produce many spans. Use your collector to filter, for example by dropping spans withelementum.automation.system_task = true.
Privacy and security
What data leaves Elementum
The lists below are for automation spans. AI agent spans are different. They carry the full text of prompts, replies, tool arguments, and tool results, the error message text, and span events, and they go to General and GenAI endpoints alike. See Prompt and response content and Personal data on AI spans. For automation spans, only the attributes listed in the attribute reference are sent. In summary:- Ids: run, automation, workflow, step, owning Object, user, and request correlation ids.
- Fixed values: trigger types, statuses, and action types.
- Names entered in Elementum: the workflow name, step names, and the owning Object’s name.
- For HTTP calls: method, host, port, status code, and the URL with query values removed.
- Timings of each span.
What is never sent on automation spans
- Record field values, comments, attachments, or email content.
- Request and response bodies, and header values, of Send API Request calls.
- Action inputs and outputs, including script code and results.
- Error messages and stack traces. A failure is described only by a fixed label or an error class name.
- Credentials. User names and passwords are removed from
url.full, and OAuth token requests are not traced. - Elementum’s internal spans and internal trace headers.
Points to review
url.fullkeeps the URL path. If a Send API Request puts a secret or personal data in the path, it appears on the span. Put such values in query parameters or headers, which are not recorded.- Step names and workflow names are sent as typed. Avoid personal data in names.
elementum.user_ididentifies a user in your organization by id.- Any endpoint that receives AI agent spans, General or GenAI, stores conversation content. Check its access controls, region, and retention before you attach it to an environment where agents run.
Transport and credentials
- Use an
https://endpoint so spans are encrypted in transit. - Endpoint credentials are stored by Elementum and shown only as set after saving. Rotate them by entering a new value. Use Custom Headers authentication for API keys, not Additional Headers, whose values are visible on the settings page.
- Only Organization Administrators can view or change trace forwarding.
Turning it off
Disable the endpoint, or detach it from the environment. Elementum stops sending within about a minute. Spans already delivered stay in your backend under your retention settings.FAQ
Does tracing slow down or affect automations?
Does tracing slow down or affect automations?
No. Spans are sent in the background. A failure to record or send spans never changes the result of a run.
How quickly do spans arrive?
How quickly do spans arrive?
Usually within seconds after each step ends.
Can I get spans for runs that happened before I set up forwarding?
Can I get spans for runs that happened before I set up forwarding?
No. Only runs after the endpoint is enabled and attached are sent.
Why is elementum.source missing on one span?
Why is elementum.source missing on one span?
That span is the trigger of a run that continued your trace. The continued trigger span arrives without
elementum.source. Filter on service.name = elementum to select every Elementum span.Why does service.name say elementum for every step?
Why does service.name say elementum for every step?
All automation spans use one service name, whichever part of Elementum ran the step. Use the span name and
elementum.automation.executor to tell steps apart. You can replace service.name with a resource attribute on the endpoint.What should I send to Elementum support?
What should I send to Elementum support?
The run id (
elementum.automation.execution_id), the trace id, and, for a specific step, its elementum.toe_id.