fix(electric-db-collection): preserve persisted rows on progressive resume #1493
main fromrefs/pull/1493/head·merged by KyleAMathews(GitHub profile, opens in a new tab) Summary#
Fixes TanStack/db#1478.
Progressive Electric collections wrapped with SQLite persistence can hydrate rows from the persisted store, then wipe them when the resumed Electric stream sends an up-to-date control message without replaying snapshot rows.
This changes progressive startup so a valid persisted Electric resume point is treated as already past the initial atomic-swap phase. That preserves hydrated persisted rows while still allowing normal progressive initial sync to use the existing buffer/truncate/apply behavior.
Root cause#
Progressive mode uses the first up-to-date message to finish initial sync by truncating the collection and replaying buffered stream rows. With persisted resume metadata, the stream starts from a saved handle/offset, so it may legitimately send up-to-date with no old rows. The collection had already hydrated those rows from persistence, but the initial atomic swap ran anyway and truncated them.
Changes#
- Initialize progressive
hasReceivedUpToDatewhen using a compatible persisted Electric resume point. - Add a regression test that hydrates persisted rows, resumes with saved Electric metadata, receives
up-to-datewithout snapshot rows, and verifies rows remain in memory and persistence. - Add a patch changeset for
@tanstack/electric-db-collection.
Validation#
pnpm --filter @tanstack/electric-db-collection test -- --coverage.enabled false
pnpm --filter @tanstack/electric-db-collection build
pnpm exec eslint packages/electric-db-collection/src/electric.ts packages/electric-db-collection/tests/electric.test.ts --max-warnings=0
I also verified this patch in a real Next.js app using browser OPFS SQLite persistence: after resetting local SQLite, loading a persisted progressive Electric collection, and refreshing/warm-loading twice, the collection stayed populated instead of dropping to zero rows.
Summary by CodeRabbit#
Bug Fixes
- Fixed progressive resume behavior to prevent saved collection data from being truncated, ensuring persisted rows remain intact when resuming from previously stored Electric shape metadata.
Tests
- Added an integration test covering persisted progressive resume, verifying hydrated persisted rows stay visible through the initial progressive phase and update correctly after resumed change and up-to-date control messages.
Chores
- Published a patch release entry for the collection package to deliver the fix.
- 1
KyleAMathews(GitHub profile, opens in a new tab) approved these changes
thanks!
More templates
@tanstack/angular-db
npm i https://pkg.pr.new/@tanstack/angular-db@1493@tanstack/browser-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/browser-db-sqlite-persistence@1493@tanstack/capacitor-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/capacitor-db-sqlite-persistence@1493@tanstack/cloudflare-durable-objects-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/cloudflare-durable-objects-db-sqlite-persistence@1493@tanstack/db
npm i https://pkg.pr.new/@tanstack/db@1493@tanstack/db-ivm
npm i https://pkg.pr.new/@tanstack/db-ivm@1493@tanstack/db-sqlite-persistence-core
npm i https://pkg.pr.new/@tanstack/db-sqlite-persistence-core@1493@tanstack/electric-db-collection
npm i https://pkg.pr.new/@tanstack/electric-db-collection@1493@tanstack/electron-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/electron-db-sqlite-persistence@1493@tanstack/expo-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/expo-db-sqlite-persistence@1493@tanstack/node-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/node-db-sqlite-persistence@1493@tanstack/offline-transactions
npm i https://pkg.pr.new/@tanstack/offline-transactions@1493@tanstack/powersync-db-collection
npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1493@tanstack/query-db-collection
npm i https://pkg.pr.new/@tanstack/query-db-collection@1493@tanstack/react-db
npm i https://pkg.pr.new/@tanstack/react-db@1493@tanstack/react-native-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/react-native-db-sqlite-persistence@1493@tanstack/rxdb-db-collection
npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1493@tanstack/solid-db
npm i https://pkg.pr.new/@tanstack/solid-db@1493@tanstack/svelte-db
npm i https://pkg.pr.new/@tanstack/svelte-db@1493@tanstack/tauri-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/tauri-db-sqlite-persistence@1493@tanstack/trailbase-db-collection
npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1493@tanstack/vue-db
npm i https://pkg.pr.new/@tanstack/vue-db@1493commit:
d7ca8ebKyleAMathews(GitHub profile, opens in a new tab) closed this
Sign in to join this conversation.
No actionable comments were generated in the recent review. 🎉
ℹ️ Recent review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID:
6e626445-8d4d-421e-8971-4d3e35e67643📥 Commits
Reviewing files that changed from the base of the PR and between eec58b0b1389e89e1542f2d2a18f8b95ffcc40b0 and d7ca8ebc225d6cb501543ecb45b9538f2272a96c.
📒 Files selected for processing (1)
packages/electric-db-collection/tests/electric.test.ts🚧 Files skipped from review as they are similar to previous changes (1)
📝 Walkthrough
Walkthrough#
Progressive resume state now reuses valid persisted resumes without initial atomic-swap buffering. The test adapter supports seeded persisted rows and committed mutations, and an integration test verifies hydrated rows survive resume and subsequent updates.
Changes#
Progressive Persisted Resume
packages/electric-db-collection/src/electric.ts,.changeset/progressive-persisted-resume.mdhasReceivedUpToDatefrom progressive mode and persisted-resume validity, with a patch changeset documenting the fix.packages/electric-db-collection/tests/electric.test.tsup-to-date.Estimated code review effort: 2 (Simple) | ~12 minutes
Sequence Diagram(s)#
Possibly related issues
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
#1478by preserving hydrated rows while resuming from persisted Electric metadata.✨ Finishing Touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
❤️ Share