#365 Report execute throws are advisory: workflow marked succeeded, exit 0, AND report output is discarded

Summary

When a workflow-scope report's execute(context) throws, swamp:

1. Logs a WRN line of shape Running report: "<name>" → ✗ "<message>",
2. Marks the workflow run as succeeded,
3. Returns exit code 0 from swamp workflow run, and
4. Silently discards the report's {markdown, json} output — neither the markdown nor the JSON data artifact is persisted, so nothing downstream (swamp data get, follow-up steps, CI scripts) can recover what the report would have written.

This combination makes throwing from a report effectively useless as a failure mechanism: the workflow looks like it passed, the exit code says success, and the report's diagnostic detail is lost. The WRN line is the only artefact, and it's only visible in the live log of the run that triggered it.

Where (the docs gap)

The swamp-report skill's report-types.md defines a report as:

interface ReportDefinition {
  description: string;
  scope: ReportScope;
  labels?: string[];
  execute(context: ReportContext): Promise<ReportResult>;
}

interface ReportResult {
  markdown: string;
  json: Record<string, unknown>;
}

There is no mention of throw semantics — what happens if execute rejects/throws, whether thrown errors propagate to workflowStatus, whether partial output is preserved, whether the exit code reflects report failures. The runtime behavior is therefore implementation- defined, and the implementation is surprising on all four counts above.

WorkflowReportContext carries workflowStatus: "succeeded" | "failed" but it's the pre-report job status; reports cannot set it. There's no documented hook (return value, context method, env signal) by which a report can declare "the workflow should fail because of what I found."

Reproduction

1. Write a workflow-scope report whose execute does its work, then throws when some condition is met:

   export const report = {
     name: "@example/my-audit",
     scope: "workflow" as const,
     execute: async (context: any) => {
       // ... compute findings ...
       const trippers = findings.filter(/* something */);
       if (trippers.length > 0) {
         throw new Error(`gate tripped: ${trippers.length} finding(s)`);
       }
       return { markdown: "...", json: { ok: true } };
     },
   };

2. Require the report in a workflow YAML (reports.require:).

3. Run a workflow whose data produces at least one tripper.

4. Observe.

Expected

At least one of the following should hold:

• The thrown error propagates to workflowStatus/exit-code (so CI sees a failure), or
• The report has a documented way to signal "fail the workflow" without losing its output (e.g. a gateTripped/status field in ReportResult, or context.failWorkflow(reason)), or
• The throw is documented as advisory-only and the docs point users at the alternative.

In all cases, the report's {markdown, json} output (or whatever partial output exists before the throw) should be persisted — losing diagnostic detail because the report wanted to fail is the most surprising behavior of the four.

Actual

Observed against swamp 20260516.045246.0-sha.e6eda98d:

• Live log (sanitized):

  HH:MM:SS.fff WRN workflow·run·<workflow> Running report: "@example/my-audit" → ✗ "gate tripped: 1 finding(s)"
  HH:MM:SS.fff INF workflow·run·<workflow> Workflow "succeeded"

• Exit code from swamp workflow run: 0

• swamp data get --workflow <workflow> report-example-my-audit-json → not found (the data artifact was never persisted because execute threw before returning)

• report-swamp-workflow-summary-json's status field: "succeeded"

So every signal a downstream consumer might check says "success."

Impact

This affects any pattern where a report wants to gate the workflow on its own analysis:

• Audit findings gate: report computes that one or more findings fail at the configured severity threshold and wants the workflow run to fail.
• Drift detector: report compares declared-vs-live state and wants to surface as a workflow failure.
• Compliance check: report verifies a control and wants CI to gate on it.

None of these can be implemented inside the report today. They have to move outside the workflow into a post-workflow run step, and have to re-read the report's persisted data — which means the report must return (not throw), include the gate decision in its JSON output, and let a shell wrapper (or CI step) read the JSON and exit non-zero.

The "report has to never throw" rule is also not discoverable: a developer's instinct is "throw on failure," they observe the WRN line during testing, see "Workflow succeeded," assume swamp is treating the throw as a soft signal, and miss the data-loss aspect entirely.

Workaround

Return normally with the gate decision in JSON, and gate at the shell level:

// in execute:
return {
  markdown: renderMarkdown(...),
  json: {
    gateTripped: trippers.length > 0,
    failOn: threshold,
    trippers: trippers.map(/* ... */),
    // ... other fields ...
  },
};

# post-workflow
swamp workflow run <workflow> && bin/audit-gate.sh <workflow>
# where audit-gate.sh reads report-<name>-json and exits non-zero on gateTripped

This works and we use it, but it pushes the gate semantics into shell script and out of the report extension. It also can't help anyone whose report runs inside a context where they don't control the shell wrapper (e.g. a swamp Lab-orchestrated job).

Suggested resolutions (any one is sufficient)

1. Propagate throws to workflow status. Make a thrown report execute fail the workflow run (status: failed, non-zero exit). Document it. Smallest behavioral change. Doesn't address data loss on throw, but the error log already carries the message.

2. Add a structured fail field to ReportResult. E.g. {markdown, json, fail?: { reason: string }}. If fail is present, the workflow run fails with that reason but the report's output is still persisted. Cleanest contract — output is preserved, intent is explicit, no error/exception coupling.

3. Document the current behavior and persist partial output. If the intended behavior is "report failures are advisory," at minimum persist any output the report produced before throwing (or persist an auto-generated stub noting the throw + its message) and document the advisory-only contract clearly in report-types.md so users stop expecting the throw to gate the workflow.

Option 2 is the cleanest long-term; option 1 is the smallest patch.

Environment

• swamp 20260516.045246.0-sha.e6eda98d
• Workflow-scope report (scope: "workflow")
• macOS 25.4.0 / Deno bundled with swamp