Skip to main content
← Back to list
01Issue
FeatureShippedSwamp CLI
Assigneesstack72

Relationships

#548 feat: giga-swamp phase 7 — Data migration commands

Opened by stack72 · 6/3/2026· Shipped 6/4/2026

Problem

Giga-swamp is feature-complete for new repos (Phases 1-6 + extensions shipped). Users can configure a namespace, push/pull scoped data, query across namespaces, and manage namespace registration. But existing solo-mode repos that want to join a giga-swamp have no migration path. Their data sits at {ds}/data/... while the namespaced layout expects {ds}/{namespace}/data/.... The namespace set command warns about orphaned data and points to a future migrate command.

Proposed Solution

1. swamp datastore namespace migrate

Re-keys existing data from the un-namespaced layout to the namespaced layout. Moves every datastore subdirectory from {ds}/{subdir}/... to {ds}/{namespace}/{subdir}/... for all directories in DEFAULT_DATASTORE_SUBDIRS.

This is a destructive operation — files are moved, not copied. The command should:

  • Require a namespace to already be set in .swamp.yaml (run namespace set first)
  • Show a preview of what will be moved (dry-run by default)
  • Require --confirm to actually execute
  • Rebuild the catalog after migration
  • Update the per-namespace index if using an extension datastore (push after migrate)

2. swamp datastore namespace unset --migrate

The reverse: flatten namespaced paths back to un-namespaced. Only allowed when the datastore contains exactly one namespace. Moves {ds}/{namespace}/{subdir}/... back to {ds}/{subdir}/....

3. Merging two solo repos into a shared datastore

Two existing repos with separate datastores can merge:

  1. Repo A: swamp datastore namespace set infra (then migrate)
  2. Repo B: swamp datastore setup extension @swamp/s3-datastore --config ... --namespace security

The setup command with --namespace should handle the initial data migration into the shared datastore under the security namespace.

Critical constraints

  • Migration is destructive — must have dry-run preview and --confirm flag
  • Do not change DEFAULT_DATASTORE_SUBDIRS
  • Catalog must be rebuilt after migration
  • For extension datastores (S3): after local migration, a push is needed to sync the new layout to the remote
  • The namespace manifest must exist before migration (namespace set writes it)
  • Solo mode data at the old paths must not be silently deleted without confirmation

Ship Gate

  • deno check, deno lint, deno fmt, deno run test
  • Round-trip test: solo -> namespace set -> migrate -> data list shows all data with namespace -> namespace unset --migrate -> data list shows all data without namespace
  • Data integrity: every file present before migration is present after, content unchanged
  • S3 round-trip: migrate locally, push to S3, wipe cache, pull, data list shows correct namespaced data
  • Compile binary and smoke test

Design Context

Full design doc: resources/giga-swamp.md (see Migration section) This is Phase 7 of 7 — the final giga-swamp phase.

02Bog Flow
OPENTRIAGEDIN PROGRESSSHIPPED+ 1 MOREASSIGNED+ 5 MOREREVIEW+ 3 MOREPR_MERGED+ 1 MORENOTIFICATION_SKIPPED

Shipped

6/4/2026, 7:16:51 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack726/3/2026, 11:52:15 PM

Sign in to post a ripple.