Skip to main content
The Send API Request action can pass the automation’s trace on to the system it calls. If that system is traced and sends its spans to the same backend, its spans appear under the Elementum call in one trace.
The HTTP client span and the Include Trace Header option are being released. If you do not see the option on the Send API Request step, or GET and POST client spans under send_api_request, the release has not reached your environment yet.

Turn it on

  1. Open the automation and select the Send API Request step.
  2. Turn on Include Trace Header.
  3. Publish the workflow.
The option is off by default. It is set per step, so you choose which calls carry trace context. The step describes the option this way: “Adds a W3C traceparent header so the receiving system can continue this automation’s trace. Trace headers you add above (traceparent, tracestate, b3) are replaced.”

What Elementum sends

With Include Trace Header on, each request carries one header:
  • The trace id is the run’s trace id. If the run continued your trace, it is your trace id.
  • The span id names the Elementum HTTP client span for this call. The receiving system’s server span becomes a child of that client span.
  • No tracestate and no baggage are sent.
The receiving system must be instrumented to read traceparent. Most OpenTelemetry HTTP server instrumentations do this by default.

Trace headers you set on the step

You can add trace headers yourself in the step’s headers list, for example to pass a fixed trace context to a partner. What is sent depends on the option: When the option is on and the step already lists a trace header, the builder shows this warning: “Trace Headers Will Be Replaced. With Include Trace Header on, the trace headers listed above are not sent. Elementum adds its own traceparent instead.” Header names treated as trace headers, matched without regard to case:
  • traceparent, tracestate, baggage
  • b3, x-b3-traceid, x-b3-spanid, x-b3-parentspanid, x-b3-sampled, x-b3-flags
  • uber-trace-id, x-amzn-trace-id
  • Any header starting with uberctx-, ot-tracer-, or x-datadog-

When the header is not sent

Even with the option on, Elementum sends no traceparent in these cases:

Elementum internal trace headers are never sent

Elementum traces its own infrastructure separately. Those internal trace headers are removed from every Send API Request call, whether Include Trace Header is on or off. The systems you call never receive Elementum’s internal trace ids or baggage.

The HTTP client span

Every Send API Request call that runs inside a traced action produces an HTTP client span, whether or not Include Trace Header is on. The span is named after the HTTP method, for example GET or POST, and has kind CLIENT. Its parent is the send_api_request action span. The span is marked as an error for a 4xx or 5xx response, a timeout, a connection failure, or a failure while reading the response body. The span ends when the response has been read, so its duration includes the body download. Full definitions are in the attribute reference.

URL redaction

url.full never contains query values or credentials: Query parameter names and the URL path are kept. If an API puts a secret or personal data in the path, that value appears in url.full. Put those values in query parameters or headers.

OAuth resends

When a call using OAuth is rejected with 401 or 403, Elementum refreshes the token and sends the call once more. You then see two client spans under the same action span: the first with status code 401 or 403 and error status, the second with elementum.automation.http.resend_count = 1.

Check that it works

  1. Turn on Include Trace Header on a step that calls an instrumented service, and publish.
  2. Run the automation.
  3. In your backend, open the run’s trace. Under send_api_request you should see the client span, and under it the receiving service’s server span.
  4. If the receiving service’s span is a separate trace, check that it reads traceparent, and that the call did not redirect to another host.
Calling another Elementum automation’s webhook with the option on continues the trace into that automation. See Calls between Elementum automations.