Skip to content
Source
Sign inSign up
TanStack/db

fix(sqlite-persistence): unblock first coordinated writes #1899

MergedKyleAMathews(GitHub profile, opens in a new tab)merged 3 commits intomain fromrefs/pull/1899/headmerged by KyleAMathews(GitHub profile, opens in a new tab)

🎯 Changes

A coordinator-backed SQLite Collection can now complete its first source commit when hydration and leader election overlap. This addresses a startup cycle consistent with #1443. A late follower can also find an existing leader before its first write, without waiting for the next periodic heartbeat.

The issue reports Electric data arriving while a Browser-coordinated Collection never becomes ready. During startup, hydration can await a source commit while that commit awaits a leader route. Previously, the election read could queue behind that hydration scope. A separate Collection's write could also hold the database writer lock while waiting for scheduler access.

The coordinator waits for an initial route before it sends a source commit or local mutation. If a follower has no route, it broadcasts a request that contains no mutation. An existing leader answers with its current heartbeat. If no leader exists, local election can continue. The write waits until it has a route or reaches the existing 10-second route deadline.

Election reads the stream position outside the shared scheduler. Regular writes enter the scheduler before they request the database writer lock. Hydration-scoped work continues to use its supplied unscheduled adapter.

A queued write checks leader identity and term again while it holds the writer lock. It returns NOT_LEADER without persistence if ownership changed. This PR does not retry a write after that response. The route wait and subsequent RPC retain separate timeout budgets.

The Browser tests cover a late follower whose leader has stopped periodic heartbeats. They check the route request, heartbeat reply, one mutation RPC, and one durable apply. Other controlled tests reproduce the hydration and election cycles. These tests do not run a real Electric stream over an OPFS worker. This PR does not add preemption or claim a measured latency improvement.

Verification

  • Browser persistence suite: 362 passed.
  • SQLite core suite: 566 passed, 2 todo.
  • Electron persistence suite: 155 passed.
  • SQLite core build passed.
  • ESLint reported no errors and 10 existing warnings.

✅ Checklist

  • I have tested this code locally with pnpm test.

🚀 Release Impact

  • This change affects published code, and I have generated a changeset.
  • This change is docs/CI/dev-only (no release).

Related to #1443.

Summary by CodeRabbit

  • Bug Fixes
    • Coordinated writes now wait for an initial leadership route when needed and are rejected if leadership changes before they can be persisted.
    • Persistence work is scheduled to let leader election finish during hydration, including when writes affect different collections.
    • Followers that join after a heartbeat can request the current leadership route without waiting for the next periodic heartbeat.
    1. Review in Change Stack →

      Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

      No actionable comments were generated in the recent review. 🎉

      ℹ️ Recent review info
      ⚙️ Run configuration

      Configuration used: defaults

      Review profile: CHILL

      Plan: Advanced

      Run ID: 0b63a4cc-0e59-4389-a557-a3515f612a25

      📥 Commits

      Reviewing files that changed from the base of the PR and between c4ac5377894aa734fe1fea464db08536c9701a0a and bb625c2791a9f7c948001f27e76dd4db404856cd.

      📒 Files selected for processing (3)
      • .changeset/fix-first-coordinated-write-route.md
      • packages/browser-db-sqlite-persistence/tests/browser-coordinator.test.ts
      • packages/db-sqlite-persistence-core/src/broadcast-coordinator.ts
      🚧 Files skipped from review as they are similar to previous changes (1)
      • .changeset/fix-first-coordinated-write-route.md

      Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 4 remain after this review.


      📝 Walkthrough

      Walkthrough

      The SQLite coordinator waits for an initial leadership route before dispatching coordinated writes. It schedules leader-side index creation and durable writes before taking the writer lock, and rejects writes when leadership changes before persistence. The SQLite adapter exposes regular-scope execution and reads stream positions outside its regular-operation scheduler.

      Changes

      SQLite coordinated writes

      Layer / File(s)Summary
      Schedule persistence work before locking
      packages/db-sqlite-persistence-core/src/persisted.ts, packages/db-sqlite-persistence-core/src/sqlite-core-adapter.ts, packages/db-sqlite-persistence-core/src/broadcast-coordinator.ts, packages/browser-db-sqlite-persistence/tests/browser-coordinator.test.ts
      Adapters can run work in a regular-operation scope. Leader-side index creation and durable writes use that scope before acquiring the database-wide writer lock. Stream-position reads bypass the regular-operation scheduler. Tests cover writes during scheduled hydration, including writes across collections.
      Wait for the initial leadership route
      packages/db-sqlite-persistence-core/src/broadcast-coordinator.ts, packages/browser-db-sqlite-persistence/tests/browser-coordinator.test.ts
      Mutation requests wait when the collection has no known leader. Leaders answer route requests with a heartbeat. Tests cover requests before the first route, late followers, and a timeout after repeated stream-position read errors.
      Recheck leadership before persistence
      packages/db-sqlite-persistence-core/src/broadcast-coordinator.ts, packages/browser-db-sqlite-persistence/tests/browser-coordinator.test.ts, .changeset/fix-first-coordinated-write-route.md
      Durable writes verify coordinator and leadership state under the writer lock. Invalid state returns NOT_LEADER. Tests cover queued writes after leadership transfers and verify that rejected writes are not applied or published.

      Priority: ⬇️ Low

      Estimated code review effort: 3 (Moderate) | ~25 minutes

      Change: Bug fix

      Merge Risk: ⚪ Minimal · up to bb625

      The first coordinated write can proceed once a leader route is known, and the investigated routing and leadership-transfer concerns do not block merging.

      Security Architecture Review

      Security architecture risk: 🔵 Low · up to bb625

      The new ordering addresses a startup write cycle, and writes queued across a leadership change are checked again before persistence. No security bypass was established. Cross-tab behavior during an in-flight write remains less certain than the tested queued-write case.

      Retained concerns No architecture-level concerns identified.

      Security review details

      Security Blast Radius

      • inferred — A participant able to send messages on the database's broadcast channel could misdirect or delay a follower's route through a forged heartbeat, affecting coordinated writes for that database and collection. The inspected write gate does not let that route alone persist data.

      Trust Boundaries and Controls

      • observed — The route is selected from channel-derived state, while the receiving coordinator gates RPC handling on local leadership and revalidates leader identity and term at the durable-write boundary. The unbound heartbeat sender and leader fields were also present in the inspected preceding revision, so they are not established as a new PR regression.

      Resilience and Maintainability Implications

      • observed — The ownership check occurs before awaiting adapter persistence. Leadership release does not wait for that operation, so it can complete afterward; this post-check window also exists in the inspected preceding revision. SQLite records the transaction atomically, while consumers reject older-term committed events and use sequence-gap recovery, leaving the full cross-instance outcome dependent on operation ordering.

      Hardening Proposals

      • proposed — Exercise leadership release during an actual adapter commit, followed by election in another instance, to establish the persisted-position and externally visible recovery behavior of the new scheduling order.
      🚥 Pre-merge checks | ✅ 5
      ✅ Passed checks (5 passed)
      Check nameStatusExplanation
      Docstring Coverage✅ PassedNo functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 4…
      Linked Issues check✅ PassedCheck skipped because no linked issues were found for this pull request.
      Out of Scope Changes check✅ PassedCheck skipped because no linked issues were found for this pull request.
      Title check✅ PassedThe title clearly and concisely identifies the SQLite persistence fix and its purpose: unblocking first coordinated writes.
      Description check✅ PassedThe description explains the motivation, implementation, test coverage, verification results, related issue, and release impact. The test checklist remains unchecked even though the verification secti…
      ✨ Finishing Touches
      📝 Generate docstrings
      • Commit to this branch
      • Create a new PR
      🧪 Generate unit tests (beta)
      • Commit to this branch
      • Create a new PR

      Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

      ❤️ Share
      Comment `@coderabbitai help` to get the list of available commands.
    2. More templates

      @tanstack/angular-db

      npm i https://pkg.pr.new/@tanstack/angular-db@1899

      @tanstack/browser-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/browser-db-sqlite-persistence@1899

      @tanstack/capacitor-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/capacitor-db-sqlite-persistence@1899

      @tanstack/cloudflare-durable-objects-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/cloudflare-durable-objects-db-sqlite-persistence@1899

      @tanstack/db

      npm i https://pkg.pr.new/@tanstack/db@1899

      @tanstack/db-ivm

      npm i https://pkg.pr.new/@tanstack/db-ivm@1899

      @tanstack/db-sqlite-persistence-core

      npm i https://pkg.pr.new/@tanstack/db-sqlite-persistence-core@1899

      @tanstack/electric-db-collection

      npm i https://pkg.pr.new/@tanstack/electric-db-collection@1899

      @tanstack/electron-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/electron-db-sqlite-persistence@1899

      @tanstack/expo-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/expo-db-sqlite-persistence@1899

      @tanstack/node-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/node-db-sqlite-persistence@1899

      @tanstack/offline-transactions

      npm i https://pkg.pr.new/@tanstack/offline-transactions@1899

      @tanstack/powersync-db-collection

      npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1899

      @tanstack/query-db-collection

      npm i https://pkg.pr.new/@tanstack/query-db-collection@1899

      @tanstack/react-db

      npm i https://pkg.pr.new/@tanstack/react-db@1899

      @tanstack/react-native-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/react-native-db-sqlite-persistence@1899

      @tanstack/react-router-with-db

      npm i https://pkg.pr.new/@tanstack/react-router-with-db@1899

      @tanstack/rxdb-db-collection

      npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1899

      @tanstack/solid-db

      npm i https://pkg.pr.new/@tanstack/solid-db@1899

      @tanstack/svelte-db

      npm i https://pkg.pr.new/@tanstack/svelte-db@1899

      @tanstack/tauri-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/tauri-db-sqlite-persistence@1899

      @tanstack/trailbase-db-collection

      npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1899

      @tanstack/vue-db

      npm i https://pkg.pr.new/@tanstack/vue-db@1899

      commit: bb625c2

    3. Size Change: 0 B

      Total Size: 171 kB

      ℹ️ View Unchanged
      FilenameSize
      packages/db/dist/esm/client.js3.66 kB
      packages/db/dist/esm/collection-options.js236 B
      packages/db/dist/esm/collection/change-events.js1.44 kB
      packages/db/dist/esm/collection/changes.js2.4 kB
      packages/db/dist/esm/collection/cleanup-queue.js794 B
      packages/db/dist/esm/collection/events.js481 B
      packages/db/dist/esm/collection/index.js4.4 kB
      packages/db/dist/esm/collection/indexes.js1.99 kB
      packages/db/dist/esm/collection/lifecycle.js2.44 kB
      packages/db/dist/esm/collection/mutations.js2.61 kB
      packages/db/dist/esm/collection/state.js6.94 kB
      packages/db/dist/esm/collection/subscription.js8.8 kB
      packages/db/dist/esm/collection/sync.js5.03 kB
      packages/db/dist/esm/collection/transaction-metadata.js144 B
      packages/db/dist/esm/deferred.js207 B
      packages/db/dist/esm/errors.js5.34 kB
      packages/db/dist/esm/event-emitter.js964 B
      packages/db/dist/esm/index.js3.82 kB
      packages/db/dist/esm/indexes/auto-index.js829 B
      packages/db/dist/esm/indexes/base-index.js1.14 kB
      packages/db/dist/esm/indexes/basic-index.js2.07 kB
      packages/db/dist/esm/indexes/btree-index.js2.26 kB
      packages/db/dist/esm/indexes/index-registry.js820 B
      packages/db/dist/esm/indexes/reverse-index.js376 B
      packages/db/dist/esm/live-query-adapter.js318 B
      packages/db/dist/esm/live-query-observer.js3.69 kB
      packages/db/dist/esm/live-query-options.js702 B
      packages/db/dist/esm/live-query-window-controller.js4.36 kB
      packages/db/dist/esm/local-only.js989 B
      packages/db/dist/esm/local-storage.js2.17 kB
      packages/db/dist/esm/optimistic-action.js359 B
      packages/db/dist/esm/paced-mutations.js496 B
      packages/db/dist/esm/proxy.js3.32 kB
      packages/db/dist/esm/query/builder/clone-query.js748 B
      packages/db/dist/esm/query/builder/functions.js1.47 kB
      packages/db/dist/esm/query/builder/index.js6.72 kB
      packages/db/dist/esm/query/builder/query-ir.js116 B
      packages/db/dist/esm/query/builder/ref-proxy.js1.28 kB
      packages/db/dist/esm/query/compiler/evaluators.js2.04 kB
      packages/db/dist/esm/query/compiler/expressions.js603 B
      packages/db/dist/esm/query/compiler/group-by.js4.14 kB
      packages/db/dist/esm/query/compiler/index.js9.11 kB
      packages/db/dist/esm/query/compiler/joins.js2.99 kB
      packages/db/dist/esm/query/compiler/lazy-targets.js1.12 kB
      packages/db/dist/esm/query/compiler/order-by.js1.91 kB
      packages/db/dist/esm/query/compiler/parent-routes.js319 B
      packages/db/dist/esm/query/compiler/query-equivalence.js455 B
      packages/db/dist/esm/query/compiler/route-metadata.js1.24 kB
      packages/db/dist/esm/query/compiler/select.js1.59 kB
      packages/db/dist/esm/query/effect.js5.13 kB
      packages/db/dist/esm/query/equality-value-identity.js591 B
      packages/db/dist/esm/query/expression-helpers.js1.43 kB
      packages/db/dist/esm/query/ir-stable-identity.js4.18 kB
      packages/db/dist/esm/query/ir.js1.74 kB
      packages/db/dist/esm/query/live-query-collection.js391 B
      packages/db/dist/esm/query/live/bucket-facade-adapter.js2.73 kB
      packages/db/dist/esm/query/live/collection-config-builder.js7.01 kB
      packages/db/dist/esm/query/live/collection-registry.js264 B
      packages/db/dist/esm/query/live/collection-subscriber.js2.29 kB
      packages/db/dist/esm/query/live/internal.js145 B
      packages/db/dist/esm/query/live/materialized-pipeline.js2.32 kB
      packages/db/dist/esm/query/live/ordered-source-loader.js4.32 kB
      packages/db/dist/esm/query/live/subset-demand-controller.js1.26 kB
      packages/db/dist/esm/query/live/utils.js1.14 kB
      packages/db/dist/esm/query/optimizer.js3.11 kB
      packages/db/dist/esm/query/query-once.js359 B
      packages/db/dist/esm/query/runtime-reference-identity.js572 B
      packages/db/dist/esm/query/subset-dedupe.js497 B
      packages/db/dist/esm/scheduler.js1.34 kB
      packages/db/dist/esm/SortedMap.js1.3 kB
      packages/db/dist/esm/strategies/debounceStrategy.js247 B
      packages/db/dist/esm/strategies/queueStrategy.js428 B
      packages/db/dist/esm/strategies/throttleStrategy.js246 B
      packages/db/dist/esm/sync-persistence.js530 B
      packages/db/dist/esm/transactions.js3.71 kB
      packages/db/dist/esm/utils.js1.21 kB
      packages/db/dist/esm/utils/array-utils.js270 B
      packages/db/dist/esm/utils/browser-polyfills.js304 B
      packages/db/dist/esm/utils/btree.js4.51 kB
      packages/db/dist/esm/utils/callbacks.js174 B
      packages/db/dist/esm/utils/comparison.js1.49 kB
      packages/db/dist/esm/utils/cursor.js677 B
      packages/db/dist/esm/utils/error.js167 B
      packages/db/dist/esm/utils/get-or-create.js155 B
      packages/db/dist/esm/utils/index-optimization.js2.42 kB
      packages/db/dist/esm/utils/type-guards.js230 B
      packages/db/dist/esm/utils/uuid.js449 B
      packages/db/dist/esm/virtual-props.js360 B
      compressed-size-action::db-package-size
    4. Size Change: 0 B

      Total Size: 7.34 kB

      ℹ️ View Unchanged
      FilenameSize
      packages/react-db/dist/esm/DbProvider.js317 B
      packages/react-db/dist/esm/HydrationBoundary.js263 B
      packages/react-db/dist/esm/index.js330 B
      packages/react-db/dist/esm/live-query-internals.js282 B
      packages/react-db/dist/esm/useLiveInfiniteQuery.js1.9 kB
      packages/react-db/dist/esm/useLiveQuery.js2.68 kB
      packages/react-db/dist/esm/useLiveQueryEffect.js355 B
      packages/react-db/dist/esm/useLiveSuspenseQuery.js812 B
      packages/react-db/dist/esm/usePacedMutations.js401 B
      compressed-size-action::react-db-package-size

Sign in to join this conversation.