Skip to main content

OPENTELEMETRY

Swamp uses OpenTelemetry (OTel) for distributed tracing. Tracing is opt-in and controlled entirely through standard OTel environment variables. When no provider is registered, all tracer and span operations are no-ops with zero overhead.

Configuration

Variable Purpose Default
OTEL_EXPORTER_OTLP_ENDPOINT Collector URL (tracing off when unset) (unset = off)
OTEL_TRACES_EXPORTER Exporter: otlp, console, or none otlp
OTEL_SERVICE_NAME Service name in traces swamp
OTEL_EXPORTER_OTLP_HEADERS Auth headers (comma-separated key=val) (none)

Tracing is enabled when OTEL_EXPORTER_OTLP_ENDPOINT is set or OTEL_TRACES_EXPORTER is console. When neither condition is met, no SDK packages are loaded and no async hooks are installed.

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 swamp workflow run my-workflow

Setting OTEL_TRACES_EXPORTER=console prints spans to stderr without requiring a collector endpoint.


Span Naming Convention

All spans use the prefix swamp. followed by the domain and operation:

swamp.<domain>.<operation>

Span Hierarchy

A workflow run produces a trace with the following structure:

swamp.cli
  ├─ swamp.lock.acquire
  ├─ swamp.datastore.sync
  └─ swamp.workflow.run.command
       └─ swamp.workflow.run
            ├─ swamp.workflow.evaluate
            ├─ swamp.workflow.job
            │    ├─ swamp.workflow.step
            │    │    └─ swamp.model.method
            │    │         └─ swamp.driver.execute
            │    └─ swamp.workflow.step
            │         └─ swamp.model.method
            │              └─ swamp.driver.execute
            └─ swamp.workflow.job
                 └─ swamp.workflow.step
                      └─ swamp.model.method
                           └─ swamp.driver.execute

CLI Root Span

Every CLI invocation creates a swamp.cli root span encompassing the entire command lifecycle.

Attribute Type Description
swamp.command string Top-level command name
swamp.subcommand string Subcommand name
swamp.version string Swamp version
swamp.args string Sanitized positional arguments
swamp.option_keys string Command-specific option names
swamp.global_options string Global flags (--json, --verbose, etc)

Libswamp Generator Spans

Every libswamp generator is wrapped with withGeneratorSpan, producing a span for the full duration of the operation. Generator spans automatically detect kind: "error" events and mark the span status as ERROR.

Domain Span Names
auth swamp.auth.login
audit swamp.audit.timeline
data swamp.data.get, .list, .search, .versions, .rename, .gc
datastore swamp.datastore.setup, .status, .sync.command, .lock.status, .lock.release
extension swamp.extension.pull, .push, .search, .update, .yank, .rm, .fmt, .list
issue swamp.issue.create
model swamp.model.create, .delete, .edit, .get, .search, .validate
model swamp.model.method.run, .method.describe, .method.history.logs
model swamp.model.output.search, .output.get, .output.logs, .output.data
report swamp.report.describe, .get, .search
repo swamp.repo.create, .upgrade
source swamp.source.fetch, .clean
telemetry swamp.telemetry.stats
type swamp.type.describe, .search
update swamp.update.check
vault swamp.vault.create, .describe, .edit, .get, .list_keys, .put, .search, .type_search
workflow swamp.workflow.create, .delete, .edit, .get, .search, .validate, .schema
workflow swamp.workflow.run.command, .history.get, .history.logs, .history.search, .run_search

Domain-Layer Spans

The workflow execution engine and method execution service create fine-grained spans for the execution hierarchy.

Span Name Attributes
swamp.workflow.run workflow.name, workflow.id, workflow.run_id
swamp.workflow.evaluate workflow.name, workflow.expressions_evaluated
swamp.workflow.job job.name, job.status
swamp.workflow.step step.name, job.name, step.task.type
swamp.model.method model.name, model.type, method.name
swamp.driver.execute driver.type, model.type

Infrastructure Spans

Span Name Attributes
swamp.lock.acquire lock.key, lock.label
swamp.datastore.sync sync.direction (pull/push), sync.file_count
swamp.repo.init (none)

Context Propagation

In-Process

The OTel SDK uses AsyncLocalStorageContextManager to propagate trace context through async call chains. tracer.startActiveSpan() and tracer.startSpan() establish parent-child relationships automatically within the active context.

  • Extensions running in-process (raw driver) inherit the active span context.
  • yield* delegation in async generators preserves context across generator boundaries.

Cross-Process

For out-of-process execution, W3C Trace Context headers are propagated via environment variables.

Step Component Action
1 Host process injectTraceContext() extracts traceparent/tracestate
2 ExecutionRequest traceHeaders field carries the headers
3 Docker driver Passes headers as TRACEPARENT/TRACESTATE env vars
4 Container Extension reads env vars and connects to parent trace

SDK Lifecycle

initTracing()
  ├─ No endpoint set, exporter not console → return (no-op)
  └─ Endpoint set or console exporter
       ├─ Create AsyncLocalStorageContextManager
       ├─ Create BasicTracerProvider with Resource
       ├─ Add BatchSpanProcessor with exporter
       └─ Register as global provider

runCli(args)          ← all spans created during execution

shutdownTracing()     ← flush pending spans, disable context manager

All OTel SDK packages (sdk-trace-base, otlp-transformer, context-async-hooks, resources, semantic-conventions) are loaded via dynamic import() only when tracing is enabled. @opentelemetry/api is statically imported — it returns no-op implementations when no provider is registered.

The OTLP exporter uses Deno's native fetch API instead of the Node.js http/https modules. @opentelemetry/otlp-transformer handles JSON serialization of spans.


Utilities

withSpan(name, attributes, fn)

Wraps an async function with a span. Creates the span, sets it as active, executes the function, records errors, and ends the span in all code paths.

await withSpan("swamp.workflow.run", { "workflow.name": name }, async () => {
  // span is active for the duration
});

withGeneratorSpan(name, attributes, generator)

Wraps an async generator with a span. The span starts when iteration begins and ends when the generator completes, throws, or is abandoned. Events with kind: "error" are detected and recorded as span errors.

yield * withGeneratorSpan("swamp.data.get", { "data.name": name }, getData());

getTracer()

Returns the swamp tracer from the global provider. Returns a no-op tracer when tracing is not initialized.

injectTraceContext()

Extracts W3C Trace Context headers (traceparent, tracestate) from the current active context into a Record<string, string>. Returns an empty object when tracing is disabled.

extractTraceContext(headers)

Creates a context from W3C Trace Context headers for connecting incoming spans to an existing trace.


Behavior When Disabled

When tracing is disabled:

Component Behavior
getTracer() Returns a no-op tracer
tracer.startSpan() Returns a no-op span (all-zeros traceId)
withSpan() Calls the wrapped function directly
withGeneratorSpan() Iterates the wrapped generator directly
injectTraceContext() Returns an empty object
SDK packages Not loaded
Async hooks Not installed