perf(offline-transactions): bound bulk replay lookup work #1958
main fromrefs/pull/1958/head·merged by KyleAMathews(GitHub profile, opens in a new tab) Large offline outboxes can spend quadratic work on two lookup steps during replay. This change bounds those lookup steps by the snapshot size.
Before and after#
| Step | Before | After |
|---|---|---|
| Bulk retry update | Each updated transaction searched the pending array. | One ID map replaces pending records in one pass. |
| Filtered replay | Each stored transaction searched the retained array. | One ID set identifies excluded records in one pass. |
The scheduler still sorts updated records by creation time. Duplicate updates keep the last value, and updates for unknown IDs do not add work. Filtered replay keeps the same retained IDs.
The work tests count ID reads and known array scan work. The original code fails both bounds: 4,160 total ID reads (4,096 updated IDs plus 64 pending IDs) for 64 bulk updates, and 3,775 retained-ID reads for 100 filtered records. The repaired paths use 128 total ID reads and 50 retained-ID reads, respectively. Two other quadratic designs also fail. The tests do not set an elapsed-time threshold.
This change covers bulk update lookup and filtered replay membership. It does not change scheduler admission, storage reads, or provider execution. It follows the still-relevant offline work claims from #1542.
Verification#
- The full
@tanstack/offline-transactionstest suite passed: 202 tests in 17 files. - Package typecheck, changed-file lint, and changed-file formatting checks passed.
- The original code and two alternative quadratic implementations failed the new work assertions at their intended checkpoints.
The changeset requests a patch release for @tanstack/offline-transactions.
Summary by CodeRabbit#
Performance
- Bulk retry updates and filtered transaction replay require less processing as the number of pending transactions grows. Transactions retain their existing order, and replay continues to include only the transactions selected for retry.
- Repeated updates to the same pending transaction use the latest update; updates for unknown transactions are ignored.
Tests
- Added coverage for bulk update ordering, repeated updates, unknown transaction IDs, and filtered replay behavior.
- 1
More templates
- todos
- offline-first-electron
- @tanstack/db-example-react-next-ssr-e2e
- tanstack-start-example-basic
- @tanstack/db-example-paced-mutations-demo
- tanstack-start-db-projects-example
- @tanstack/db-example-react-start-ssr-e2e
- @tanstack/db-example-react-todo
- offline-transactions-react-native
- shopping-list-react-native
- @tanstack/db-example-solid-todo
@tanstack/angular-db
npm i https://pkg.pr.new/@tanstack/angular-db@1958@tanstack/browser-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/browser-db-sqlite-persistence@1958@tanstack/capacitor-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/capacitor-db-sqlite-persistence@1958@tanstack/cloudflare-durable-objects-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/cloudflare-durable-objects-db-sqlite-persistence@1958@tanstack/db
npm i https://pkg.pr.new/@tanstack/db@1958@tanstack/db-ivm
npm i https://pkg.pr.new/@tanstack/db-ivm@1958@tanstack/db-sqlite-persistence-core
npm i https://pkg.pr.new/@tanstack/db-sqlite-persistence-core@1958@tanstack/electric-db-collection
npm i https://pkg.pr.new/@tanstack/electric-db-collection@1958@tanstack/electron-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/electron-db-sqlite-persistence@1958@tanstack/expo-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/expo-db-sqlite-persistence@1958@tanstack/node-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/node-db-sqlite-persistence@1958@tanstack/offline-transactions
npm i https://pkg.pr.new/@tanstack/offline-transactions@1958@tanstack/powersync-db-collection
npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1958@tanstack/query-db-collection
npm i https://pkg.pr.new/@tanstack/query-db-collection@1958@tanstack/react-db
npm i https://pkg.pr.new/@tanstack/react-db@1958@tanstack/react-native-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/react-native-db-sqlite-persistence@1958@tanstack/react-router-with-db
npm i https://pkg.pr.new/@tanstack/react-router-with-db@1958@tanstack/rxdb-db-collection
npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1958@tanstack/solid-db
npm i https://pkg.pr.new/@tanstack/solid-db@1958@tanstack/svelte-db
npm i https://pkg.pr.new/@tanstack/svelte-db@1958@tanstack/tauri-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/tauri-db-sqlite-persistence@1958@tanstack/trailbase-db-collection
npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1958@tanstack/vue-db
npm i https://pkg.pr.new/@tanstack/vue-db@1958commit:
c8c0a4aSize Change: 0 B
Total Size: 175 kB
compressed-size-action::db-package-sizeℹ️ View Unchanged
Filename Size packages/db/dist/esm/client.js3.7 kB packages/db/dist/esm/collection-options.js236 B packages/db/dist/esm/collection/change-events.js1.45 kB packages/db/dist/esm/collection/changes.js2.41 kB packages/db/dist/esm/collection/cleanup-queue.js822 B packages/db/dist/esm/collection/events.js481 B packages/db/dist/esm/collection/index.js4.46 kB packages/db/dist/esm/collection/indexes.js2.06 kB packages/db/dist/esm/collection/lifecycle.js2.69 kB packages/db/dist/esm/collection/mutations.js2.61 kB packages/db/dist/esm/collection/state.js8.38 kB packages/db/dist/esm/collection/subscription.js8.81 kB packages/db/dist/esm/collection/sync.js5.09 kB packages/db/dist/esm/collection/transaction-metadata.js144 B packages/db/dist/esm/deferred.js207 B packages/db/dist/esm/errors.js5.49 kB packages/db/dist/esm/event-emitter.js964 B packages/db/dist/esm/index.js3.94 kB packages/db/dist/esm/indexes/auto-index.js841 B packages/db/dist/esm/indexes/base-index.js1.26 kB packages/db/dist/esm/indexes/basic-index.js2.05 kB packages/db/dist/esm/indexes/btree-index.js2.33 kB packages/db/dist/esm/indexes/index-registry.js820 B packages/db/dist/esm/indexes/reverse-index.js376 B packages/db/dist/esm/live-query-adapter.js318 B packages/db/dist/esm/live-query-observer.js4.71 kB packages/db/dist/esm/live-query-options.js731 B packages/db/dist/esm/live-query-window-controller.js4.4 kB packages/db/dist/esm/local-only.js989 B packages/db/dist/esm/local-storage.js2.17 kB packages/db/dist/esm/optimistic-action.js359 B packages/db/dist/esm/paced-mutations.js702 B packages/db/dist/esm/persisted-readiness.js195 B packages/db/dist/esm/proxy.js3.32 kB packages/db/dist/esm/query/builder/clone-query.js766 B packages/db/dist/esm/query/builder/functions.js1.45 kB packages/db/dist/esm/query/builder/index.js6.82 kB packages/db/dist/esm/query/builder/query-ir.js116 B packages/db/dist/esm/query/builder/ref-proxy-identity.js198 B packages/db/dist/esm/query/builder/ref-proxy.js1.35 kB packages/db/dist/esm/query/builder/wrapper-identity.js221 B packages/db/dist/esm/query/compiler/evaluators.js2.06 kB packages/db/dist/esm/query/compiler/expressions.js603 B packages/db/dist/esm/query/compiler/group-by.js4.16 kB packages/db/dist/esm/query/compiler/index.js9.1 kB packages/db/dist/esm/query/compiler/joins.js2.99 kB packages/db/dist/esm/query/compiler/lazy-targets.js1.13 kB packages/db/dist/esm/query/compiler/order-by.js1.99 kB packages/db/dist/esm/query/compiler/parent-routes.js319 B packages/db/dist/esm/query/compiler/query-equivalence.js455 B packages/db/dist/esm/query/compiler/route-metadata.js1.24 kB packages/db/dist/esm/query/compiler/select.js1.59 kB packages/db/dist/esm/query/effect.js4.98 kB packages/db/dist/esm/query/equality-value-identity.js591 B packages/db/dist/esm/query/expression-helpers.js1.45 kB packages/db/dist/esm/query/ir-stable-identity.js4.22 kB packages/db/dist/esm/query/ir.js1.7 kB packages/db/dist/esm/query/live-query-collection.js391 B packages/db/dist/esm/query/live/bucket-facade-adapter.js2.73 kB packages/db/dist/esm/query/live/collection-config-builder.js6.58 kB packages/db/dist/esm/query/live/collection-registry.js264 B packages/db/dist/esm/query/live/collection-subscriber.js2.16 kB packages/db/dist/esm/query/live/graph-scheduler.js303 B packages/db/dist/esm/query/live/internal.js145 B packages/db/dist/esm/query/live/materialized-pipeline.js2.32 kB packages/db/dist/esm/query/live/ordered-source-loader.js4.47 kB packages/db/dist/esm/query/live/subset-demand-controller.js1.67 kB packages/db/dist/esm/query/live/utils.js1.2 kB packages/db/dist/esm/query/optimizer.js2.93 kB packages/db/dist/esm/query/query-once.js359 B packages/db/dist/esm/query/runtime-reference-identity.js630 B packages/db/dist/esm/query/subset-dedupe.js497 B packages/db/dist/esm/scheduler.js1.14 kB packages/db/dist/esm/SortedMap.js1.63 kB packages/db/dist/esm/strategies/debounceStrategy.js331 B packages/db/dist/esm/strategies/queueStrategy.js488 B packages/db/dist/esm/strategies/throttleStrategy.js386 B packages/db/dist/esm/sync-persistence.js530 B packages/db/dist/esm/transactions.js3.73 kB packages/db/dist/esm/utils.js1.22 kB packages/db/dist/esm/utils/array-utils.js270 B packages/db/dist/esm/utils/browser-polyfills.js304 B packages/db/dist/esm/utils/btree.js3.01 kB packages/db/dist/esm/utils/callbacks.js174 B packages/db/dist/esm/utils/comparison.js1.59 kB packages/db/dist/esm/utils/cursor.js677 B packages/db/dist/esm/utils/error.js167 B packages/db/dist/esm/utils/get-or-create.js155 B packages/db/dist/esm/utils/index-optimization.js2.42 kB packages/db/dist/esm/utils/type-guards.js230 B packages/db/dist/esm/utils/uuid.js449 B packages/db/dist/esm/virtual-props.js360 B Size Change: 0 B
Total Size: 8.06 kB
compressed-size-action::react-db-package-sizeℹ️ View Unchanged
Filename Size packages/react-db/dist/esm/DbProvider.js317 B packages/react-db/dist/esm/HydrationBoundary.js263 B packages/react-db/dist/esm/index.js330 B packages/react-db/dist/esm/live-query-internals.js282 B packages/react-db/dist/esm/useLiveInfiniteQuery.js1.93 kB packages/react-db/dist/esm/useLiveQuery.js3.3 kB packages/react-db/dist/esm/useLiveQueryEffect.js355 B packages/react-db/dist/esm/useLiveSuspenseQuery.js881 B packages/react-db/dist/esm/usePacedMutations.js401 B KyleAMathews(GitHub profile, opens in a new tab) closed this
Sign in to join this conversation.
Navigate logical layers of code changes, visualize relationships, and explore their blast radius.
No actionable comments were generated in the recent review. 🎉
ℹ️ Recent review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Advanced
Run ID:
cc6a18bb-6071-4a40-9bb2-f0ba1546d6fc📥 Commits
Reviewing files that changed from the base of the PR and between d28a2b9cc48d3f9445e04b480b29169477482160 and c8c0a4a73ae60e1a3afe9c5a817ce436bf5b88d8.
📒 Files selected for processing (2)
packages/offline-transactions/src/executor/KeyScheduler.tspackages/offline-transactions/tests/filtered-replay-work.test.tsIncluded review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 3 remain after this review.
📝 Walkthrough
Walkthrough#
The patch changes pending-transaction updates to use an ID map and filtered replay exclusion to use a Set. Tests cover bounded work, ordering, repeated updates, unknown IDs, and retained outbox transactions. A patch Changeset entry is added.
Changes#
Offline replay
packages/offline-transactions/src/executor/KeyScheduler.ts,packages/offline-transactions/tests/KeyScheduler.test.tsupdateTransactionsmaps updates by transaction ID and replaces matching pending transactions. Tests cover bounded ID reads, ordering, repeated updates, and unknown IDs.packages/offline-transactions/src/executor/TransactionExecutor.ts,packages/offline-transactions/tests/filtered-replay-work.test.ts,.changeset/streamline-offline-replay-work.mdloadPendingTransactionsuses a Set to identify excluded transaction IDs. A test checks bounded work and retained outbox IDs. The Changeset describes the patch release.Priority: ⬇️ Low
Estimated code review effort: 2 (Simple) | ~10 minutes
Change: Refactor
Merge Risk: ⚪ Minimal · up to
c8c0aBulk retry updates and filtered replay reduce lookup work while preserving transaction selection and ordering. No material merge-blocking risk is evident.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
✨ Finishing Touches
📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
❤️ Share