Skip to main content

WORKFLOWS

A workflow is a YAML file that orchestrates the execution of model methods and other workflows. Workflows live in the workflows/ directory within a swamp repository.

File Structure

workflows/
  workflow-e182b5c7-41e7-4c9a-a9f8-455b7bb5ce12.yaml
  workflow-d14de37a-b931-4363-97e9-fe9ec9637b60.yaml

The filename is workflow-{uuid}.yaml.

Top-Level Fields

id: d14de37a-b931-4363-97e9-fe9ec9637b60
name: deploy-pipeline
description: A multi-stage deployment pipeline
trigger:
  schedule: "0 3 * * *"
tags:
  team: platform
inputs:
  type: object
  properties:
    region:
      type: string
      default: us-east-1
  required:
    - region
jobs:
  - name: build
    steps:
      - name: compile
        task:
          type: model_method
          modelIdOrName: builder
          methodName: run
version: 1
driver: docker
driverConfig:
  image: "node:20"
reports:
  require:
    - summary
  skip:
    - debug-report

id

Unique identifier for the workflow.

Property Value
Type string (UUID v4)
Required Yes
Default Auto-generated on create
Format xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

name

Human-readable name for the workflow.

Property Value
Type string
Required Yes
Default None

Constraints:

  • Minimum 1 character.
  • Must be unique within the repository.
  • Must not contain .., \, or null bytes (path traversal protection).
  • / is only allowed in scoped names matching @[a-z0-9_-]+/[a-z0-9_-]+(\/[a-z0-9_-]+)* (e.g., @team/deploy-pipeline).

description

Human-readable description of the workflow's purpose.

Property Value
Type string
Required No
Default None

trigger

Scheduling configuration for automatic execution.

Property Value
Type object
Required No
Default None

trigger.schedule

Cron expression for recurring execution.

Property Value
Type string
Required No
Format Standard cron (5 fields): minute hour dom month dow

Validated at parse time using the Croner library. Invalid expressions cause a schema validation error.

trigger:
  schedule: "0 3 * * *" # Daily at 3:00 AM
trigger:
  schedule: "*/15 * * * *" # Every 15 minutes
trigger:
  schedule: "0 3,12 * * *" # 3:00 AM and 12:00 PM daily

tags

Key-value labels attached to the workflow.

Property Value
Type Record<string, string>
Required No
Default {} 
tags:
  team: platform
  env: production

inputs

JSON Schema definition for structured inputs provided at runtime. Uses the same InputsSchema format as model definition inputs.

Property Value
Type InputsSchema
Required No
Default None 
inputs:
  type: object
  properties:
    environments:
      type: array
      description: Target environments
    region:
      type: string
      default: us-east-1
  required:
    - environments

Input values are available in CEL expressions as inputs.* within step task inputs and forEach expressions.

jobs

Ordered list of jobs to execute.

Property Value
Type Job[]
Required Yes
Minimum 1

See Job below.

version

Workflow version number.

Property Value
Type integer
Required No
Default 1

Must be a positive integer.

driver

Default execution driver for all jobs and steps.

Property Value
Type string
Required No
Default None

Steps and jobs can override this value. See the Model Definitions reference for available driver types.

driverConfig

Configuration for the default execution driver.

Property Value
Type Record<string, unknown>
Required No
Default None

Available keys depend on the driver. See the Model Definitions reference for driver configuration fields.

reports

Post-execution report selection.

Property Value
Type ReportSelection
Required No
Default None

Uses the same format as model definition reports.

reports:
  require:
    - summary
    - name: deployment-report
      methods:
        - apply
  skip:
    - debug-report

Job

A job groups related steps within a workflow. Jobs with no dependencies on each other execute in parallel.

jobs:
  - name: build
    description: Compile and test
    steps:
      - name: compile
        task:
          type: model_method
          modelIdOrName: builder
          methodName: run
    dependsOn: []
    weight: 0
    driver: raw
    driverConfig: {}

name

Job name, unique within the workflow.

Property Value
Type string
Required Yes
Minimum 1 char

description

Human-readable description of the job.

Property Value
Type string
Required No
Default None

steps

Ordered list of steps to execute within the job.

Property Value
Type Step[]
Required Yes
Minimum 1

See Step below.

dependsOn

Dependencies on other jobs with trigger conditions.

Property Value
Type JobDependency[]
Required No
Default []

Each element specifies which job to depend on and under what condition:

dependsOn:
  - job: build
    condition:
      type: succeeded

See Trigger Conditions for the full condition schema.

weight

Numeric weight for deterministic ordering of jobs at the same dependency level.

Property Value
Type number
Required No
Default 0

Lower weights execute first. Jobs with the same weight and dependency level are ordered alphabetically by name.

driver

Execution driver override for all steps in this job. Takes precedence over the workflow-level driver.

Property Value
Type string
Required No
Default None

driverConfig

Driver configuration override for this job. Takes precedence over the workflow-level driverConfig.

Property Value
Type Record<string, unknown>
Required No
Default None

Step

A step is a single unit of work within a job. Steps with no dependencies on each other execute in parallel.

steps:
  - name: deploy-env
    description: Deploy to environment
    task:
      type: model_method
      modelIdOrName: deployer
      methodName: apply
    forEach:
      item: env
      in: "${{ inputs.environments }}"
    dependsOn:
      - step: compile
        condition:
          type: succeeded
    weight: 0
    dataOutputOverrides:
      - specName: result
        lifetime: 30d
        garbageCollection: 5
        tags:
          stage: deploy
        vary:
          - environment
    allowFailure: false
    driver: docker
    driverConfig:
      image: "node:20"

name

Step name, unique within the job.

Property Value
Type string
Required Yes
Minimum 1 char

When forEach is used and the step name contains ${{ }} expressions, the name is evaluated with the iteration context to produce unique names per iteration.

description

Human-readable description of the step.

Property Value
Type string
Required No
Default None

task

The work to execute. See Step Task below.

Property Value
Type StepTask
Required Yes

forEach

Iteration configuration. When set, the step expands into one execution per element.

Property Value
Type ForEach
Required No
Default None

See ForEach below.

dependsOn

Dependencies on other steps within the same job.

Property Value
Type StepDependency[]
Required No
Default [] 
dependsOn:
  - step: compile
    condition:
      type: succeeded

See Trigger Conditions for the full condition schema.

weight

Numeric weight for deterministic ordering of steps at the same dependency level.

Property Value
Type number
Required No
Default 0

Lower weights execute first. Steps with the same weight and dependency level are ordered alphabetically by name.

dataOutputOverrides

Overrides for data output specifications produced by the step's task.

Property Value
Type DataOutputOverride[]
Required No
Default None

See DataOutputOverride below.

allowFailure

When true, a step failure does not fail the job. Subsequent steps with succeeded conditions against this step will not trigger, but completed and always conditions will.

Property Value
Type boolean
Required No
Default false

driver

Execution driver override for this step. Takes precedence over job-level and workflow-level driver settings.

Property Value
Type string
Required No
Default None

driverConfig

Driver configuration override for this step.

Property Value
Type Record<string, unknown>
Required No
Default None

Step Task

A step task defines the work a step executes. Two task types are available.

Model Method (model_method)

Invokes a method on a model definition.

task:
  type: model_method
  modelIdOrName: deployer
  methodName: apply
  inputs:
    target: "${{ inputs.region }}"
Field Type Required Description
type "model_method" Yes Task type discriminator
modelIdOrName string Yes Model definition name or UUID
methodName string Yes Method to invoke on the model
inputs Record<string, unknown> No Input values passed to the method

inputs values support CEL expressions using the ${{ }} syntax.

Workflow (workflow)

Invokes another workflow as a nested execution.

task:
  type: workflow
  workflowIdOrName: notification-workflow
  inputs:
    channel: "#deployments"
Field Type Required Description
type "workflow" Yes Task type discriminator
workflowIdOrName string Yes Workflow name or UUID
inputs Record<string, unknown> No Input values passed to the workflow

Nested workflows have a maximum depth of 10. Cyclic references (workflow A invoking workflow B which invokes workflow A) are detected and rejected.

ForEach

Expands a step into multiple executions, one per element in the evaluated collection.

forEach:
  item: env
  in: "${{ inputs.environments }}"
Field Type Required Description
item string Yes Variable name bound to each element
in string Yes CEL expression evaluating to a list or map

When iterating over a list, self.{item} is set to each element.

When iterating over a map, self.{item} is set to an object with key and value fields for each entry.

Expanded steps inherit the original step's dependencies. If the step name contains ${{ }} expressions, they are evaluated with the iteration context to produce unique step names. Otherwise, the iteration value or index is appended as a suffix (e.g., deploy-env-staging, deploy-env-production).

Trigger Conditions

Trigger conditions control when a job or step executes relative to its dependencies. They are specified in the condition field of dependsOn entries.

Leaf Conditions

Type Evaluates to true when
always Always (unconditional)
succeeded The dependency succeeded
failed The dependency failed
completed The dependency completed (succeeded or failed)
skipped The dependency was skipped 
dependsOn:
  - job: build
    condition:
      type: succeeded

Composite Conditions

Leaf conditions can be composed with boolean logic.

and — all child conditions must be true. Requires at least 2 conditions.

condition:
  type: and
  conditions:
    - type: succeeded
    - type: not
      condition:
        type: skipped

or — any child condition must be true. Requires at least 2 conditions.

condition:
  type: or
  conditions:
    - type: succeeded
    - type: failed

not — inverts a single condition.

condition:
  type: not
  condition:
    type: failed

Composite conditions nest recursively.

DataOutputOverride

Overrides data output specifications for data produced by a step's task.

Field Type Required Description
specName string Yes Output spec name to override
lifetime Lifetime No Override data retention
garbageCollection GarbageCollectionPolicy No Override version retention
tags Record<string, string> No Additional tags to merge with output tags
vary string[] No Input key names to vary by (composite data names)

Lifetime

How long data is retained.

Value Description
Duration string 1h, 5m, 10d, 2w, 1mo, 10y
ephemeral Deleted when the process ends
infinite Never automatically deleted
job Lives until the job completes
workflow Lives until the workflow completes

Duration format: {number}{unit} where unit is h (hours), m (minutes), d (days), w (weeks), mo (months), or y (years). Zero durations (e.g., 0h) are normalized to workflow.

GarbageCollectionPolicy

How many versions to retain.

Value Description
integer Keep N most recent versions
Duration string Keep versions created within the duration 
dataOutputOverrides:
  - specName: result
    lifetime: 30d
    garbageCollection: 5
    tags:
      stage: deploy
    vary:
      - environment

Driver Resolution Order

When a step executes, the driver is resolved from the most specific level:

  1. Step driver / driverConfig
  2. Job driver / driverConfig
  3. Workflow driver / driverConfig
  4. Model definition driver / driverConfig
  5. Default raw

The first non-empty value at each level wins.

CEL Expressions

String values in step task inputs and forEach.in support CEL expressions using the ${{ }} wrapper.

task:
  type: model_method
  modelIdOrName: deployer
  methodName: apply
  inputs:
    region: "${{ inputs.region }}"
    artifact: "${{ model.builder.resource.result.result.attributes.path }}"
    secret: "${{ vault.get('infra', 'deploy-key') }}"

Context Variables

Variable Description
self The current model definition: self.name, self.tags.*, self.globalArguments.*
model All models: model.<name>.definition.*, model.<name>.resource.<spec>.<instance>.attributes.*
inputs Workflow or model runtime inputs: inputs.<property>
env Process environment variables: env.<VAR_NAME>
vault Secrets: vault.get('<vault>', '<key>')
data Versioned data: data.latest(...), data.version(...), data.findBySpec(...), data.query(...)
file File contents: file.contents('<model>', '<spec>')

When forEach is active, the iteration variable is available as self.{item}. For example, with forEach: { item: "env", in: [...] }, each iteration sets self.env to the current element.

After a step completes, its model's execution data and output data are available to subsequent steps through the model context. For example, model.builder.execution.status and model.builder.resource.result.result.attributes.*.

See the CEL Expressions reference for the full expression language.

Execution Order

Jobs and steps are sorted topologically using Kahn's algorithm with weighted tie-breaking.

  1. Dependency level — nodes are grouped into levels based on the dependency graph. Level 0 has no dependencies, level 1 depends only on level 0, etc.
  2. Weight — within the same level, lower weights execute first.
  3. Name — within the same level and weight, names are sorted alphabetically for determinism.

Nodes at the same level with no mutual dependencies execute in parallel.

Cyclic dependencies are detected and produce a validation error with the cycle path (e.g., job-a -> job-b -> job-a).

Validation

swamp workflow validate checks a workflow against its schema:

$ swamp workflow validate multi-stage
Validating: multi-stage
  ✓ Schema validation
  ✓ Unique job names
  ✓ Unique step names in job 'build'
  ✓ Unique step names in job 'deploy'
  ✓ Unique step names in job 'notify'
  ✓ Valid job dependency references
  ✓ Valid step dependency references in job 'build'
  ✓ Valid step dependency references in job 'deploy'
  ✓ Valid step dependency references in job 'notify'
  ✓ No cyclic job dependencies
  ✓ No cyclic step dependencies in job 'build'
  ✓ No cyclic step dependencies in job 'deploy'
  ✓ No cyclic step dependencies in job 'notify'
Summary: 13/13 validations passed
Result: PASSED

Omit the name to validate all workflows in the repository:

$ swamp workflow validate
Validating all workflows...

multi-stage
  ✓ Schema validation
  ...
Summary: 1/1 workflows passed
Overall: PASSED

Validation rules:

  • id must be a valid UUID v4.
  • name must be at least 1 character, unique within the repository, with no path traversal sequences.
  • version must be a positive integer.
  • tags values must be strings.
  • trigger.schedule must be a valid cron expression.
  • jobs must contain at least one job.
  • Job names must be unique within the workflow.
  • Step names must be unique within each job.
  • Each job must contain at least one step.
  • Job dependency references must name existing jobs.
  • Step dependency references must name existing steps within the same job.
  • No cyclic dependencies among jobs.
  • No cyclic dependencies among steps within each job.

Complete Example

id: d14de37a-b931-4363-97e9-fe9ec9637b60
name: multi-stage
description: A multi-stage deployment pipeline
trigger:
  schedule: "0 3 * * *"
tags:
  team: platform
  env: production
inputs:
  type: object
  properties:
    environments:
      type: array
      description: Target environments
    region:
      type: string
      default: us-east-1
  required:
    - environments
jobs:
  - name: build
    description: Compile and test
    weight: 0
    steps:
      - name: compile
        description: Build the artifacts
        task:
          type: model_method
          modelIdOrName: builder
          methodName: run
          inputs:
            target: "${{ inputs.region }}"
        weight: 0
        allowFailure: false
      - name: test
        description: Run tests
        task:
          type: model_method
          modelIdOrName: test-runner
          methodName: execute
        dependsOn:
          - step: compile
            condition:
              type: succeeded
        weight: 1
        allowFailure: false
  - name: deploy
    description: Deploy to each environment
    dependsOn:
      - job: build
        condition:
          type: succeeded
    weight: 10
    steps:
      - name: deploy-env
        description: Deploy to environment
        task:
          type: model_method
          modelIdOrName: deployer
          methodName: apply
        forEach:
          item: env
          in: "${{ inputs.environments }}"
        dataOutputOverrides:
          - specName: result
            lifetime: 30d
            garbageCollection: 5
            tags:
              stage: deploy
            vary:
              - environment
  - name: notify
    description: Send notifications
    dependsOn:
      - job: deploy
        condition:
          type: completed
    weight: 20
    steps:
      - name: send-notification
        task:
          type: workflow
          workflowIdOrName: notification-workflow
          inputs:
            channel: "#deployments"
        allowFailure: true
version: 1
driver: docker
driverConfig:
  image: "node:20"
  memory: "1g"
reports:
  require:
    - name: deployment-report
      methods:
        - apply
    - summary
  skip:
    - debug-report
← PreviousOpenTelemetryNext →Vaults