Skip to content
Source
Sign inSign up
TanStack/db

Persisted Electric collection ends up permanently empty: diverging schemaVersions wipe rows, schema reset leaves the resume point behind #1589

Send to my agent
Closedeikowagenknecht(GitHub profile, opens in a new tab) opened this issue 1 comment

Packages: @tanstack/browser-db-sqlite-persistence 0.2.0, @tanstack/db-sqlite-persistence-core 0.2.0, @tanstack/electric-db-collection 0.3.6, @tanstack/db 0.6.8

We hit a case where a persisted Electric collection comes up ready with 0 rows on every page load, with nothing logged, and never recovers. It took a while to track down because it looked random. There seem to be two separate bugs that line up to cause it.

Diverging schemaVersions wipe rows through a shared coordinator adapter slot

createBrowserWASQLitePersistence caches one adapter per (schemaMismatchPolicy, schemaVersion) and calls coordinator.setAdapter(adapter) on every resolvePersistenceForCollection (see browser-persistence.js). BrowserCollectionCoordinator only has a single adapter slot, so whichever collection resolved last owns it.

If you have two persisted collections with different schemaVersions, leader-side RPCs (pullSince, applyLocalMutations, ensureRemoteSubset) for collection A can end up running through collection B's adapter. SQLiteCorePersistenceAdapter.ensureCollectionReady then compares A's registry schema_version against the other adapter's version, sees a mismatch, and under sync-present-reset deletes all of A's rows and rewrites the registry version. With both collections live the wipes flip-flop depending on RPC timing, which is why it looked random for us (the difference between F5 and a hard reload was pure timing).

A schema-mismatch reset doesn't clear collection_metadata, so Electric's resume point survives the wipe

The reset deletes rows, tombstones and applied_tx, but leaves collection_metadata intact. @tanstack/electric-db-collection keeps its resume point there (electric:resume, offset + handle, keyed by shape identity = url + params). So on the next load the collection restores 0 rows, reads the stale resume point, resumes the stream "up to date" past all the data, and marks itself ready. Nothing triggers a refetch, and the session even rewrites the resume record, so it stays empty across reloads.

Reproduction

const persistence = createBrowserWASQLitePersistence({ database, coordinator });

const a = createCollection(persistedCollectionOptions({
  ...electricCollectionOptions({ id: "a", shapeOptions: { url: shapeA }, getKey: r => r.id }),
  persistence, schemaVersion: 2,
}));
const b = createCollection(persistedCollectionOptions({
  ...electricCollectionOptions({ id: "b", shapeOptions: { url: shapeB }, getKey: r => r.id }),
  persistence, schemaVersion: 1, // different version
}));
  1. Sync both collections (put a few thousand rows in one to make it obvious) and confirm rows are persisted.
  2. Reload a few times. Depending on which adapter holds the coordinator slot when an RPC fires, one collection's table gets wiped while its electric:resume survives.
  3. After that it loads ready with 0 rows on every reload, nothing logged. scanRows returns 0 rows, but loadCollectionMetadata still shows a kind: "resume" record.

Workaround

Using one shared schemaVersion for every collection on the same persistence avoids the cross-wiring. We also bake that version into the shape URL, so bumping it changes Electric's shape identity and discards the stale resume point too.

  • #1419 (closed) used the same diverging-schemaVersion setup (todos schemaVersion: 2, projects schemaVersion: 1) on a shared db/coordinator. It was closed by recommending a single shared persistence instance to fix a transaction-conflict error, but that doesn't help here: the coordinator still has one adapter slot, so diverging schemaVersions on a shared persistence still cross-wire and wipe.
  • #1478 / #1493 are about rows being wiped while the resume point survives, but via the progressive atomic-swap truncate. The second bug here is a different trigger (the schema-mismatch reset not clearing collection_metadata) ending in the same empty state.
  • #1443 is Electric + coordinator never reaching ready. This is the opposite: it reaches ready, just with 0 rows.
  1. Confirming the second bug ("a schema-mismatch reset leaves the resume point behind") on React Native, with no browser coordinator involved — so it reproduces independently of the shared-adapter-slot issue:

    • @tanstack/react-native-db-sqlite-persistence 0.2.6 / @tanstack/db-sqlite-persistence-core 0.2.6
    • @tanstack/electric-db-collection 0.3.12 / @tanstack/db 0.6.14
    • op-sqlite driver, single schemaVersion across all collections

    Sequence:

    1. Existing SQLite file at schemaVersion: 2; app ships schemaVersion: 3.
    2. SQLiteCorePersistenceAdapter.handleSchemaMismatch (sync-present-reset) deletes the collection's rows, tombstones, applied_tx, and persisted indexes — but there is no DELETE FROM collection_metadata in that transaction, so the electric:resume entry survives.
    3. Next boot: electric.js reads the surviving cursor (canUsePersistedResume has no "do local rows actually exist" cross-check), resumes from the stored offset+handle, Electric replies up-to-date with no data, and the collection marks ready with 0 rows — persisted and in-memory. It never re-snapshots, so the state is permanent.

    Why it looks random (matching the OP's experience): any collection whose upstream rows changed after the stored offset partially self-heals via replayed change messages. In our app only the two tables that hadn't changed in days stayed empty while everything else quietly rebuilt itself — which made it look like a data-specific sync failure rather than a reset bug.

    Verified server-side with curl: replaying the persisted offset+handle returns 200 + up-to-date + no rows, while offset=-1 for the same shape returns the full snapshot.

    Workaround we're shipping (heals existing poisoned installs too): wrap adapter.loadCollectionMetadata — if it returns an electric:resume entry with kind: "resume" while the collection's row table is empty, drop that entry, forcing the Electric layer to take a fresh snapshot. Cost is one redundant empty-snapshot fetch for genuinely empty collections.

    Suggested fix: handleSchemaMismatch should delete the collection's collection_metadata rows (or at minimum the electric:resume key) inside the same reset transaction. Belt-and-braces: electric-db-collection could also decline canUsePersistedResume when the persisted row count is zero, which would make the whole class of "rows gone, cursor kept" states self-healing.

Sign in to join this conversation.