OPENTELEMETRY
Swamp uses OpenTelemetry (OTel) for distributed tracing. Tracing is opt-in and controlled entirely through standard OTel environment variables. When no provider is registered, all tracer and span operations are no-ops with zero overhead.
Configuration
| Variable | Purpose | Default |
|---|---|---|
OTEL_EXPORTER_OTLP_ENDPOINT |
Collector URL (tracing off when unset) | (unset = off) |
OTEL_TRACES_EXPORTER |
Exporter: otlp, console, or none |
otlp |
OTEL_SERVICE_NAME |
Service name in traces | swamp |
OTEL_EXPORTER_OTLP_HEADERS |
Auth headers (comma-separated key=val) |
(none) |
Tracing is enabled when OTEL_EXPORTER_OTLP_ENDPOINT is set or
OTEL_TRACES_EXPORTER is console. When neither condition is met, no SDK
packages are loaded and no async hooks are installed.
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 swamp workflow run my-workflowSetting OTEL_TRACES_EXPORTER=console prints spans to stderr without requiring
a collector endpoint.
Span Naming Convention
All spans use the prefix swamp. followed by the domain and operation:
swamp.<domain>.<operation>Span Hierarchy
A workflow run produces a trace with the following structure:
swamp.cli
├─ swamp.lock.acquire
├─ swamp.datastore.sync
└─ swamp.workflow.run.command
└─ swamp.workflow.run
├─ swamp.workflow.evaluate
├─ swamp.workflow.job
│ ├─ swamp.workflow.step
│ │ └─ swamp.model.method
│ │ └─ swamp.driver.execute
│ └─ swamp.workflow.step
│ └─ swamp.model.method
│ └─ swamp.driver.execute
└─ swamp.workflow.job
└─ swamp.workflow.step
└─ swamp.model.method
└─ swamp.driver.executeCLI Root Span
Every CLI invocation creates a swamp.cli root span encompassing the entire
command lifecycle.
| Attribute | Type | Description |
|---|---|---|
swamp.command |
string |
Top-level command name |
swamp.subcommand |
string |
Subcommand name |
swamp.version |
string |
Swamp version |
swamp.args |
string |
Sanitized positional arguments |
swamp.option_keys |
string |
Command-specific option names |
swamp.global_options |
string |
Global flags (--json, --verbose, etc) |
Libswamp Generator Spans
Every libswamp generator is wrapped with withGeneratorSpan, producing a span
for the full duration of the operation. Generator spans automatically detect
kind: "error" events and mark the span status as ERROR.
| Domain | Span Names |
|---|---|
auth |
swamp.auth.login |
audit |
swamp.audit.timeline |
data |
swamp.data.get, .list, .search, .versions, .rename, .gc |
datastore |
swamp.datastore.setup, .status, .sync.command, .lock.status, .lock.release |
extension |
swamp.extension.pull, .push, .search, .update, .yank, .rm, .fmt, .list |
issue |
swamp.issue.create |
model |
swamp.model.create, .delete, .edit, .get, .search, .validate |
model |
swamp.model.method.run, .method.describe, .method.history.logs |
model |
swamp.model.output.search, .output.get, .output.logs, .output.data |
report |
swamp.report.describe, .get, .search |
repo |
swamp.repo.create, .upgrade |
source |
swamp.source.fetch, .clean |
telemetry |
swamp.telemetry.stats |
type |
swamp.type.describe, .search |
update |
swamp.update.check |
vault |
swamp.vault.create, .describe, .edit, .get, .list_keys, .put, .search, .type_search |
workflow |
swamp.workflow.create, .delete, .edit, .get, .search, .validate, .schema |
workflow |
swamp.workflow.run.command, .history.get, .history.logs, .history.search, .run_search |
Domain-Layer Spans
The workflow execution engine and method execution service create fine-grained spans for the execution hierarchy.
| Span Name | Attributes |
|---|---|
swamp.workflow.run |
workflow.name, workflow.id, workflow.run_id |
swamp.workflow.evaluate |
workflow.name, workflow.expressions_evaluated |
swamp.workflow.job |
job.name, job.status |
swamp.workflow.step |
step.name, job.name, step.task.type |
swamp.model.method |
model.name, model.type, method.name |
swamp.driver.execute |
driver.type, model.type |
Infrastructure Spans
| Span Name | Attributes |
|---|---|
swamp.lock.acquire |
lock.key, lock.label |
swamp.datastore.sync |
sync.direction (pull/push), sync.file_count |
swamp.repo.init |
(none) |
Context Propagation
In-Process
The OTel SDK uses AsyncLocalStorageContextManager to propagate trace context
through async call chains. tracer.startActiveSpan() and tracer.startSpan()
establish parent-child relationships automatically within the active context.
- Extensions running in-process (raw driver) inherit the active span context.
yield*delegation in async generators preserves context across generator boundaries.
Cross-Process
For out-of-process execution, W3C Trace Context headers are propagated via environment variables.
| Step | Component | Action |
|---|---|---|
| 1 | Host process | injectTraceContext() extracts traceparent/tracestate |
| 2 | ExecutionRequest |
traceHeaders field carries the headers |
| 3 | Docker driver | Passes headers as TRACEPARENT/TRACESTATE env vars |
| 4 | Container | Extension reads env vars and connects to parent trace |
SDK Lifecycle
initTracing()
├─ No endpoint set, exporter not console → return (no-op)
└─ Endpoint set or console exporter
├─ Create AsyncLocalStorageContextManager
├─ Create BasicTracerProvider with Resource
├─ Add BatchSpanProcessor with exporter
└─ Register as global provider
runCli(args) ← all spans created during execution
shutdownTracing() ← flush pending spans, disable context managerAll OTel SDK packages (sdk-trace-base, otlp-transformer,
context-async-hooks, resources, semantic-conventions) are loaded via
dynamic import() only when tracing is enabled. @opentelemetry/api is
statically imported — it returns no-op implementations when no provider is
registered.
The OTLP exporter uses Deno's native fetch API instead of the Node.js
http/https modules. @opentelemetry/otlp-transformer handles JSON
serialization of spans.
Utilities
withSpan(name, attributes, fn)
Wraps an async function with a span. Creates the span, sets it as active, executes the function, records errors, and ends the span in all code paths.
await withSpan("swamp.workflow.run", { "workflow.name": name }, async () => {
// span is active for the duration
});withGeneratorSpan(name, attributes, generator)
Wraps an async generator with a span. The span starts when iteration begins and
ends when the generator completes, throws, or is abandoned. Events with
kind: "error" are detected and recorded as span errors.
yield * withGeneratorSpan("swamp.data.get", { "data.name": name }, getData());getTracer()
Returns the swamp tracer from the global provider. Returns a no-op tracer when tracing is not initialized.
injectTraceContext()
Extracts W3C Trace Context headers (traceparent, tracestate) from the
current active context into a Record<string, string>. Returns an empty object
when tracing is disabled.
extractTraceContext(headers)
Creates a context from W3C Trace Context headers for connecting incoming spans to an existing trace.
Behavior When Disabled
When tracing is disabled:
| Component | Behavior |
|---|---|
getTracer() |
Returns a no-op tracer |
tracer.startSpan() |
Returns a no-op span (all-zeros traceId) |
withSpan() |
Calls the wrapped function directly |
withGeneratorSpan() |
Iterates the wrapped generator directly |
injectTraceContext() |
Returns an empty object |
| SDK packages | Not loaded |
| Async hooks | Not installed |