Skip to main content
← Back to list
01Issue
FeatureShippedSwamp CLI
Assigneesstack72

Relationships

#517 workflow validate: fail on references to unknown model instances (typo'd modelIdOrName)

Opened by stack72 · 6/1/2026· Shipped 6/2/2026

Problem

Follow-up from swamp-club#506 (PR #1485). That fix makes swamp workflow validate fail when a step's model type cannot be resolved. But a step that references a model instance by a name that does not exist — e.g. a typo like modelIdOrName: htpp-poll — still resolves to model_not_found and is silently skipped as a pass.

So validate still cannot catch a misspelled model instance reference, which is part of the original "silently PASSES" class of bugs.

Why it was left out of #506

model_not_found is deliberately skipped today because a model instance may legitimately be created at run time by an upstream step (direct type execution) or out-of-band, and swamp's own integration tests (integration/workflow_test.ts) rely on validating a workflow's structure before its models exist. Flipping model_not_found to a hard failure broke those tests, confirming it changes an intended contract.

During #506 I prototyped a conservative guard: fail model_not_found only when no direct-execution step in the workflow tree produces that instance name (recursing nested workflows; bail to "skip" if any producer name is a dynamic CEL expression). It worked and caught typos, but it still failed the "validate structure before models exist" integration tests — so it's a genuine product decision, not a pure bug fix.

Proposed solution (needs a decision)

Decide the contract for unknown model instance references in workflow validate:

  1. Producer-aware failure — fail model_not_found only when no step in the workflow could produce the instance (the prototype above). Update the structural integration tests to provision a real model so "valid workflow" is genuinely valid. Catches typos; changes the "validate before models exist" contract.
  2. Opt-in strictness — add a --strict flag (or config) that turns model_not_found into a failure, leaving the default lenient.
  3. Warning severity — introduce a non-pass/non-fail "warning" outcome in workflow validation (renderer is currently binary pass/fail), surfaced in output + --json, optionally exit non-zero.

Notes

  • Renderer/WorkflowValidationResult is currently binary pass/fail; option 3 is a larger change to output and JSON shape.
  • The prototype implementation and tests are in the #506 branch history if useful as a starting point.
02Bog Flow
OPENTRIAGEDIN PROGRESSSHIPPED+ 1 MOREASSIGNED+ 7 MOREREVIEW+ 3 MOREPR_MERGED+ 1 MORENOTIFICATION_SKIPPED

Shipped

6/2/2026, 5:01:29 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack726/1/2026, 10:36:31 PM

Sign in to post a ripple.