Skip to content
Source
Sign inSign up
TanStack/db

perf(offline-transactions): bound bulk replay lookup work #1958

MergedKyleAMathews(GitHub profile, opens in a new tab)merged 2 commits intomain fromrefs/pull/1958/headmerged by KyleAMathews(GitHub profile, opens in a new tab)

Large offline outboxes can spend quadratic work on two lookup steps during replay. This change bounds those lookup steps by the snapshot size.

Before and after

StepBeforeAfter
Bulk retry updateEach updated transaction searched the pending array.One ID map replaces pending records in one pass.
Filtered replayEach stored transaction searched the retained array.One ID set identifies excluded records in one pass.

The scheduler still sorts updated records by creation time. Duplicate updates keep the last value, and updates for unknown IDs do not add work. Filtered replay keeps the same retained IDs.

The work tests count ID reads and known array scan work. The original code fails both bounds: 4,160 total ID reads (4,096 updated IDs plus 64 pending IDs) for 64 bulk updates, and 3,775 retained-ID reads for 100 filtered records. The repaired paths use 128 total ID reads and 50 retained-ID reads, respectively. Two other quadratic designs also fail. The tests do not set an elapsed-time threshold.

This change covers bulk update lookup and filtered replay membership. It does not change scheduler admission, storage reads, or provider execution. It follows the still-relevant offline work claims from #1542.

Verification

  • The full @tanstack/offline-transactions test suite passed: 202 tests in 17 files.
  • Package typecheck, changed-file lint, and changed-file formatting checks passed.
  • The original code and two alternative quadratic implementations failed the new work assertions at their intended checkpoints.

The changeset requests a patch release for @tanstack/offline-transactions.

Summary by CodeRabbit

  • Performance

    • Bulk retry updates and filtered transaction replay require less processing as the number of pending transactions grows. Transactions retain their existing order, and replay continues to include only the transactions selected for retry.
    • Repeated updates to the same pending transaction use the latest update; updates for unknown transactions are ignored.
  • Tests

    • Added coverage for bulk update ordering, repeated updates, unknown transaction IDs, and filtered replay behavior.
    1. Review in Change Stack →

      Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

      No actionable comments were generated in the recent review. 🎉

      ℹ️ Recent review info
      ⚙️ Run configuration

      Configuration used: defaults

      Review profile: CHILL

      Plan: Advanced

      Run ID: cc6a18bb-6071-4a40-9bb2-f0ba1546d6fc

      📥 Commits

      Reviewing files that changed from the base of the PR and between d28a2b9cc48d3f9445e04b480b29169477482160 and c8c0a4a73ae60e1a3afe9c5a817ce436bf5b88d8.

      📒 Files selected for processing (2)
      • packages/offline-transactions/src/executor/KeyScheduler.ts
      • packages/offline-transactions/tests/filtered-replay-work.test.ts

      Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 3 remain after this review.


      📝 Walkthrough

      Walkthrough

      The patch changes pending-transaction updates to use an ID map and filtered replay exclusion to use a Set. Tests cover bounded work, ordering, repeated updates, unknown IDs, and retained outbox transactions. A patch Changeset entry is added.

      Changes

      Offline replay

      Layer / File(s)Summary
      Bulk scheduler updates
      packages/offline-transactions/src/executor/KeyScheduler.ts, packages/offline-transactions/tests/KeyScheduler.test.ts
      updateTransactions maps updates by transaction ID and replaces matching pending transactions. Tests cover bounded ID reads, ordering, repeated updates, and unknown IDs.
      Filtered replay selection
      packages/offline-transactions/src/executor/TransactionExecutor.ts, packages/offline-transactions/tests/filtered-replay-work.test.ts, .changeset/streamline-offline-replay-work.md
      loadPendingTransactions uses a Set to identify excluded transaction IDs. A test checks bounded work and retained outbox IDs. The Changeset describes the patch release.

      Priority: ⬇️ Low

      Estimated code review effort: 2 (Simple) | ~10 minutes

      Change: Refactor

      Merge Risk: ⚪ Minimal · up to c8c0a

      Bulk retry updates and filtered replay reduce lookup work while preserving transaction selection and ordering. No material merge-blocking risk is evident.

      🚥 Pre-merge checks | ✅ 5
      ✅ Passed checks (5 passed)
      Check nameStatusExplanation
      Title check✅ PassedThe title clearly and concisely identifies the main change: bounding bulk replay lookup work in offline transactions.
      Description check✅ PassedThe description provides detailed change motivation, before-and-after behavior, verification results, scope, and release impact. It does not use the template headings or include the checklist items, b…
      Docstring Coverage✅ PassedNo functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 4…
      Linked Issues check✅ PassedCheck skipped because no linked issues were found for this pull request.
      Out of Scope Changes check✅ PassedCheck skipped because no linked issues were found for this pull request.
      ✨ Finishing Touches
      📝 Generate docstrings
      • Commit to this branch
      • Create a new PR
      🧪 Generate unit tests (beta)
      • Commit to this branch
      • Create a new PR
      • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

      Autopilot is currently an internal CodeRabbit preview.


      Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

      ❤️ Share
      Comment `@coderabbitai help` to get the list of available commands.
    2. More templates

      @tanstack/angular-db

      npm i https://pkg.pr.new/@tanstack/angular-db@1958

      @tanstack/browser-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/browser-db-sqlite-persistence@1958

      @tanstack/capacitor-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/capacitor-db-sqlite-persistence@1958

      @tanstack/cloudflare-durable-objects-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/cloudflare-durable-objects-db-sqlite-persistence@1958

      @tanstack/db

      npm i https://pkg.pr.new/@tanstack/db@1958

      @tanstack/db-ivm

      npm i https://pkg.pr.new/@tanstack/db-ivm@1958

      @tanstack/db-sqlite-persistence-core

      npm i https://pkg.pr.new/@tanstack/db-sqlite-persistence-core@1958

      @tanstack/electric-db-collection

      npm i https://pkg.pr.new/@tanstack/electric-db-collection@1958

      @tanstack/electron-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/electron-db-sqlite-persistence@1958

      @tanstack/expo-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/expo-db-sqlite-persistence@1958

      @tanstack/node-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/node-db-sqlite-persistence@1958

      @tanstack/offline-transactions

      npm i https://pkg.pr.new/@tanstack/offline-transactions@1958

      @tanstack/powersync-db-collection

      npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1958

      @tanstack/query-db-collection

      npm i https://pkg.pr.new/@tanstack/query-db-collection@1958

      @tanstack/react-db

      npm i https://pkg.pr.new/@tanstack/react-db@1958

      @tanstack/react-native-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/react-native-db-sqlite-persistence@1958

      @tanstack/react-router-with-db

      npm i https://pkg.pr.new/@tanstack/react-router-with-db@1958

      @tanstack/rxdb-db-collection

      npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1958

      @tanstack/solid-db

      npm i https://pkg.pr.new/@tanstack/solid-db@1958

      @tanstack/svelte-db

      npm i https://pkg.pr.new/@tanstack/svelte-db@1958

      @tanstack/tauri-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/tauri-db-sqlite-persistence@1958

      @tanstack/trailbase-db-collection

      npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1958

      @tanstack/vue-db

      npm i https://pkg.pr.new/@tanstack/vue-db@1958

      commit: c8c0a4a

    3. Size Change: 0 B

      Total Size: 175 kB

      ℹ️ View Unchanged
      FilenameSize
      packages/db/dist/esm/client.js3.7 kB
      packages/db/dist/esm/collection-options.js236 B
      packages/db/dist/esm/collection/change-events.js1.45 kB
      packages/db/dist/esm/collection/changes.js2.41 kB
      packages/db/dist/esm/collection/cleanup-queue.js822 B
      packages/db/dist/esm/collection/events.js481 B
      packages/db/dist/esm/collection/index.js4.46 kB
      packages/db/dist/esm/collection/indexes.js2.06 kB
      packages/db/dist/esm/collection/lifecycle.js2.69 kB
      packages/db/dist/esm/collection/mutations.js2.61 kB
      packages/db/dist/esm/collection/state.js8.38 kB
      packages/db/dist/esm/collection/subscription.js8.81 kB
      packages/db/dist/esm/collection/sync.js5.09 kB
      packages/db/dist/esm/collection/transaction-metadata.js144 B
      packages/db/dist/esm/deferred.js207 B
      packages/db/dist/esm/errors.js5.49 kB
      packages/db/dist/esm/event-emitter.js964 B
      packages/db/dist/esm/index.js3.94 kB
      packages/db/dist/esm/indexes/auto-index.js841 B
      packages/db/dist/esm/indexes/base-index.js1.26 kB
      packages/db/dist/esm/indexes/basic-index.js2.05 kB
      packages/db/dist/esm/indexes/btree-index.js2.33 kB
      packages/db/dist/esm/indexes/index-registry.js820 B
      packages/db/dist/esm/indexes/reverse-index.js376 B
      packages/db/dist/esm/live-query-adapter.js318 B
      packages/db/dist/esm/live-query-observer.js4.71 kB
      packages/db/dist/esm/live-query-options.js731 B
      packages/db/dist/esm/live-query-window-controller.js4.4 kB
      packages/db/dist/esm/local-only.js989 B
      packages/db/dist/esm/local-storage.js2.17 kB
      packages/db/dist/esm/optimistic-action.js359 B
      packages/db/dist/esm/paced-mutations.js702 B
      packages/db/dist/esm/persisted-readiness.js195 B
      packages/db/dist/esm/proxy.js3.32 kB
      packages/db/dist/esm/query/builder/clone-query.js766 B
      packages/db/dist/esm/query/builder/functions.js1.45 kB
      packages/db/dist/esm/query/builder/index.js6.82 kB
      packages/db/dist/esm/query/builder/query-ir.js116 B
      packages/db/dist/esm/query/builder/ref-proxy-identity.js198 B
      packages/db/dist/esm/query/builder/ref-proxy.js1.35 kB
      packages/db/dist/esm/query/builder/wrapper-identity.js221 B
      packages/db/dist/esm/query/compiler/evaluators.js2.06 kB
      packages/db/dist/esm/query/compiler/expressions.js603 B
      packages/db/dist/esm/query/compiler/group-by.js4.16 kB
      packages/db/dist/esm/query/compiler/index.js9.1 kB
      packages/db/dist/esm/query/compiler/joins.js2.99 kB
      packages/db/dist/esm/query/compiler/lazy-targets.js1.13 kB
      packages/db/dist/esm/query/compiler/order-by.js1.99 kB
      packages/db/dist/esm/query/compiler/parent-routes.js319 B
      packages/db/dist/esm/query/compiler/query-equivalence.js455 B
      packages/db/dist/esm/query/compiler/route-metadata.js1.24 kB
      packages/db/dist/esm/query/compiler/select.js1.59 kB
      packages/db/dist/esm/query/effect.js4.98 kB
      packages/db/dist/esm/query/equality-value-identity.js591 B
      packages/db/dist/esm/query/expression-helpers.js1.45 kB
      packages/db/dist/esm/query/ir-stable-identity.js4.22 kB
      packages/db/dist/esm/query/ir.js1.7 kB
      packages/db/dist/esm/query/live-query-collection.js391 B
      packages/db/dist/esm/query/live/bucket-facade-adapter.js2.73 kB
      packages/db/dist/esm/query/live/collection-config-builder.js6.58 kB
      packages/db/dist/esm/query/live/collection-registry.js264 B
      packages/db/dist/esm/query/live/collection-subscriber.js2.16 kB
      packages/db/dist/esm/query/live/graph-scheduler.js303 B
      packages/db/dist/esm/query/live/internal.js145 B
      packages/db/dist/esm/query/live/materialized-pipeline.js2.32 kB
      packages/db/dist/esm/query/live/ordered-source-loader.js4.47 kB
      packages/db/dist/esm/query/live/subset-demand-controller.js1.67 kB
      packages/db/dist/esm/query/live/utils.js1.2 kB
      packages/db/dist/esm/query/optimizer.js2.93 kB
      packages/db/dist/esm/query/query-once.js359 B
      packages/db/dist/esm/query/runtime-reference-identity.js630 B
      packages/db/dist/esm/query/subset-dedupe.js497 B
      packages/db/dist/esm/scheduler.js1.14 kB
      packages/db/dist/esm/SortedMap.js1.63 kB
      packages/db/dist/esm/strategies/debounceStrategy.js331 B
      packages/db/dist/esm/strategies/queueStrategy.js488 B
      packages/db/dist/esm/strategies/throttleStrategy.js386 B
      packages/db/dist/esm/sync-persistence.js530 B
      packages/db/dist/esm/transactions.js3.73 kB
      packages/db/dist/esm/utils.js1.22 kB
      packages/db/dist/esm/utils/array-utils.js270 B
      packages/db/dist/esm/utils/browser-polyfills.js304 B
      packages/db/dist/esm/utils/btree.js3.01 kB
      packages/db/dist/esm/utils/callbacks.js174 B
      packages/db/dist/esm/utils/comparison.js1.59 kB
      packages/db/dist/esm/utils/cursor.js677 B
      packages/db/dist/esm/utils/error.js167 B
      packages/db/dist/esm/utils/get-or-create.js155 B
      packages/db/dist/esm/utils/index-optimization.js2.42 kB
      packages/db/dist/esm/utils/type-guards.js230 B
      packages/db/dist/esm/utils/uuid.js449 B
      packages/db/dist/esm/virtual-props.js360 B
      compressed-size-action::db-package-size
    4. Size Change: 0 B

      Total Size: 8.06 kB

      ℹ️ View Unchanged
      FilenameSize
      packages/react-db/dist/esm/DbProvider.js317 B
      packages/react-db/dist/esm/HydrationBoundary.js263 B
      packages/react-db/dist/esm/index.js330 B
      packages/react-db/dist/esm/live-query-internals.js282 B
      packages/react-db/dist/esm/useLiveInfiniteQuery.js1.93 kB
      packages/react-db/dist/esm/useLiveQuery.js3.3 kB
      packages/react-db/dist/esm/useLiveQueryEffect.js355 B
      packages/react-db/dist/esm/useLiveSuspenseQuery.js881 B
      packages/react-db/dist/esm/usePacedMutations.js401 B
      compressed-size-action::react-db-package-size

Sign in to join this conversation.