Skip to main content
← Back to list
01Issue
FeatureShippedSwamp CLI
Assigneesstack72

#378 Datastore sync: add SyncContext and SyncCapabilities framework contracts

Opened by stack72 · 5/19/2026· Shipped 5/21/2026

Problem

The datastore sync API gives extensions no context about what a sync operation is for. pullChanged() and pushChanged() arrive with no information about which models are being operated on, forcing every extension to sync the entire repository on every call. This makes multi-user workflows slow — running one model pulls and pushes data for every model in the datastore.

A previous attempt (PR #1386) tried to fix this with a scope: string parameter, but it leaked filesystem implementation details into the framework contract (S3 key prefixes are meaningless to MongoDB or Redis backends) and was reverted after breaking the catalog rebuild invariant.

Proposed Solution

Add domain-level context and capability advertisement to the sync service interface:

  • SyncContext — describes what the operation is about (models being operated on), not how to filter storage. Each extension translates domain context to its own storage model (S3 → key prefix, MongoDB → query filter, etc.).
  • SyncCapabilities — sync service advertises what it supports (scopedSync, and later lazyHydration). Lives on the sync service (not the provider factory) because capabilities depend on runtime config.
  • context?: SyncContext on DatastoreSyncOptions — passed to pullChanged/pushChanged when the extension advertises scopedSync.
  • capabilities?(): SyncCapabilities on DatastoreSyncService — optional method, missing means no special capabilities.

Wire through acquireModelLocks in repo_context.ts: when caps?.scopedSync is true, pass context.models on pull/push. When false/absent, call with no args (today's behavior, zero change). Wrap capabilities() in try/catch so buggy extensions degrade gracefully.

Key design constraints

  • All new fields optional — existing extensions compile and run unchanged
  • Both pull AND push capability-gated — no extension receives options it didn't opt into
  • synced = true assignment and all 8 catalogStore.invalidate() call sites untouched
  • No restructuring of acquireModelLocks control flow (the mistake in PR #1386)

Alternatives Considered

  1. scope: string on sync options (PR #1386 approach) — leaked S3 path structure into the contract. MongoDB/Redis extensions can't use path prefixes. Reverted.
  2. Capabilities on DatastoreProvider — wrong object. The provider is a factory; capabilities depend on runtime config known only to the sync service instance.
  3. Always pass context without capability gate — risks N full pulls for extensions that ignore context (one per model in the lock loop), which is worse than today's behavior.

This is Phase 1 of a 3-phase datastore efficiency overhaul. Phase 2 (S3 extension: dirty sidecar, partitioned index, content hashing) and Phase 3 (lazy hydration) build on these contracts.

02Bog Flow
OPENTRIAGEDIN PROGRESSSHIPPED+ 1 MOREASSIGNED+ 5 MOREREVIEW+ 2 MOREPR_LINKED+ 1 MORENOTIFICATION_SKIPPED

Shipped

5/21/2026, 11:37:52 AM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack725/21/2026, 6:59:08 AM

Sign in to post a ripple.