CatalogStore constructor runs createSchema before migrateIfNeeded; v1→v2 upgrade fails on existing repos
Opened by keeb · 4/8/2026· Shipped 4/9/2026
Description
After PR #1145 merged (promoting provenance fields to first-class DataRecord columns), any swamp repo with a pre-existing v1 catalog database fails to open with ERR_SQLITE_ERROR: no such column: workflow_run_id. The schema-version migration logic exists but is unreachable because of the order of calls in the constructor.
Steps to reproduce
- Have a swamp repo that was created/used with a swamp version prior to PR #1145 (so
.swamp/data/_catalog.dbexists with the v1 schema) - Upgrade swamp via
swamp repo upgradeto a version containing PR #1145 - Run any command that opens the repo, e.g.
swamp workflow validate <name>
Expected: the catalog migration drops the v1 table and rebuilds it with the v2 schema (which is exactly what migrateIfNeeded() is designed to do)
Actual: the command crashes with:
FTL error Error: no such column: workflow_run_id in
CREATE INDEX IF NOT EXISTS idx_catalog_workflow_run_id ON catalog(workflow_run_id);
CREATE INDEX IF NOT EXISTS idx_catalog_step_name ON catalog(step_name);
...
at CatalogStore.createSchema (file://.../catalog_store.ts:84:13)
at new CatalogStore (file://.../catalog_store.ts:79:10)Root cause
src/infrastructure/persistence/catalog_store.ts lines 74-81:
constructor(dbPath: string) {
ensureDirSync(dirname(dbPath));
this.db = new DatabaseSync(dbPath);
this.db.exec("PRAGMA busy_timeout=5000");
this.db.exec("PRAGMA journal_mode=WAL");
this.createSchema(); // <-- runs first; throws on existing v1 table
this.migrateIfNeeded(); // <-- unreachable
}createSchema() (lines 83-122) issues:
CREATE TABLE IF NOT EXISTS catalog (... workflow_run_id ..., step_name ..., ...);
CREATE INDEX IF NOT EXISTS idx_catalog_workflow_run_id ON catalog(workflow_run_id);
CREATE INDEX IF NOT EXISTS idx_catalog_step_name ON catalog(step_name);When the catalog table already exists (v1), CREATE TABLE IF NOT EXISTS is a no-op, so the table keeps its old shape. Then CREATE INDEX runs against the v1 table and fails because the columns it references don't exist.
migrateIfNeeded() would handle exactly this case (it drops the old table and lets createSchema() recreate it under v2), but the constructor never reaches it because createSchema() already threw.
Fix
Reorder the constructor so the schema version is checked before any v2-specific DDL runs. The migration needs catalog_meta to exist in order to read schema_version, so the sequence is:
- Create
catalog_meta(v2-independent, present in both schemas) - Read schema version, drop the old
catalogtable if it's stale - Create
catalog(v2)
A minimal patch:
constructor(dbPath: string) {
ensureDirSync(dirname(dbPath));
this.db = new DatabaseSync(dbPath);
this.db.exec("PRAGMA busy_timeout=5000");
this.db.exec("PRAGMA journal_mode=WAL");
// Ensure catalog_meta exists so migration can read schema_version
this.db.exec(`
CREATE TABLE IF NOT EXISTS catalog_meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL
);
`);
this.migrateIfNeeded(); // drop stale v1 catalog before creating v2
this.createSchema();
}createSchema() already uses CREATE TABLE IF NOT EXISTS for catalog_meta, so the duplicate is harmless. Or createSchema() could be split into createMetaTable() + createCatalogTable() for clarity.
Workaround
Manually delete the catalog db (it's a self-healing local cache, no data loss):
rm .swamp/data/_catalog.db .swamp/data/_catalog.db-wal .swamp/data/_catalog.db-shmThe next swamp command rebuilds it via backfill against the on-disk data.
Environment
- swamp version:
20260206.200442.0-sha.(post PR #1145) - OS: Linux 6.18.13-arch1-1
- Affects: any repo with pre-#1145 catalog data (i.e. essentially every existing repo upgraded after the merge)
Why this matters
PR #1145 is the migration enabling the data-query refactor (#1123). Every repo upgrading to it currently bricks until the user manually deletes the catalog cache. The fix is a 2-line constructor reorder.
Shipped
No activity in this phase yet.
Sign in to post a ripple.