fix(db): drop stale direct optimistic rows after txid sync #1547
main fromrefs/pull/1547/head·merged by samwillis(GitHub profile, opens in a new tab) Summary#
Fix stale optimistic rows persisting when sync confirms a different server-generated key (e.g., Postgres GENERATED ALWAYS AS IDENTITY). After a direct collection.insert() with a temporary client ID, the server-assigned row would appear alongside the stale optimistic row instead of replacing it.
Root Cause#
CollectionStateManager maintained separate pendingOptimisticDirectUpserts / pendingOptimisticDirectDeletes sets that exempted direct transactions from stale-row cleanup. When sync confirmed a row under a different key than the client's temp key, the stale optimistic row could never be cleaned up — no sync transaction would ever reference the old client key, but the direct-transaction exemption kept it alive indefinitely.
Approach#
Remove the direct-transaction exemption entirely. All completed optimistic mutations now go through the same stale-row check: kept only if pendingSyncKeys.has(key). This is safe because direct transactions stay in persisting state (keeping the optimistic row visible via the active-transaction loop) until the sync backend delivers the matching txid. By the time the transaction completes, the sync data is already available.
Also removed the now-dead DIRECT_TRANSACTION_METADATA_KEY constant and its usage in mutations.ts.
Key Invariants#
- A direct transaction's
mutationFn(e.g.,awaitTxId()) does not resolve until sync delivers the matching txid - While a transaction is in
persistingstate, its optimistic rows are kept alive by the active-transaction loop, not the pending-optimistic sets - Once
completed, the pending-optimistic stale check is the sole gatekeeper
Non-goals#
- Adding observability/logging to
state.ts(systemic gap, separate effort) - Covering optimistic delete with server-generated keys (deletes always target known keys)
Verification#
pnpm vitest run --pool-options.threads.maxThreads=2 packages/db/tests packages/electric-db-collection/tests
All 2364 tests pass. 2 pre-existing failures in collection-subscribe-changes.test.ts (also fail on main).
Files Changed#
packages/db/src/collection/state.ts— RemovependingOptimisticDirect*sets, simplify stale-row check topendingSyncKeys.has(key)onlypackages/db/src/collection/mutations.ts— RemoveDIRECT_TRANSACTION_METADATA_KEYimport and metadata taggingpackages/db/src/collection/transaction-metadata.ts— Deleted (orphaned)packages/db/tests/collection.test.ts— Two new tests: optimistic insert replaced by server key, live query reflects the key swappackages/electric-db-collection/tests/electric.test.ts— One new test: txid sync confirms different server-generated key
Summary by CodeRabbit#
Bug Fixes
- Fixed stale optimistic rows persisting when sync confirms a different server-generated key during insert operations.
Tests
- Added comprehensive tests for optimistic insert sync behavior, verifying proper handling when server-generated keys replace client-provided keys.
Chores
- Updated CI to use Node.js 22.13.
Documentation
- Added Git and PR hygiene guidelines.
- 1
Size Change: +57 B (+0.05%)
Total Size: 122 kB
📦 View Changed
Filename Size Change packages/db/dist/esm/collection/mutations.js2.46 kB -12 B (-0.49%) packages/db/dist/esm/collection/state.js5.33 kB +69 B (+1.31%) compressed-size-action::db-package-sizeℹ️ View Unchanged
Filename Size packages/db/dist/esm/collection/change-events.js1.39 kB packages/db/dist/esm/collection/changes.js1.38 kB packages/db/dist/esm/collection/cleanup-queue.js810 B packages/db/dist/esm/collection/events.js434 B packages/db/dist/esm/collection/index.js3.61 kB packages/db/dist/esm/collection/indexes.js1.99 kB packages/db/dist/esm/collection/lifecycle.js1.69 kB packages/db/dist/esm/collection/subscription.js3.74 kB packages/db/dist/esm/collection/sync.js2.88 kB packages/db/dist/esm/collection/transaction-metadata.js144 B packages/db/dist/esm/deferred.js207 B packages/db/dist/esm/errors.js5.01 kB packages/db/dist/esm/event-emitter.js748 B packages/db/dist/esm/index.js3.01 kB packages/db/dist/esm/indexes/auto-index.js829 B packages/db/dist/esm/indexes/base-index.js729 B packages/db/dist/esm/indexes/basic-index.js2.05 kB packages/db/dist/esm/indexes/btree-index.js2.17 kB packages/db/dist/esm/indexes/index-registry.js820 B packages/db/dist/esm/indexes/reverse-index.js538 B packages/db/dist/esm/local-only.js890 B packages/db/dist/esm/local-storage.js2.1 kB packages/db/dist/esm/optimistic-action.js359 B packages/db/dist/esm/paced-mutations.js496 B packages/db/dist/esm/proxy.js3.75 kB packages/db/dist/esm/query/builder/functions.js1.37 kB packages/db/dist/esm/query/builder/index.js5.79 kB packages/db/dist/esm/query/builder/ref-proxy.js1.23 kB packages/db/dist/esm/query/compiler/evaluators.js1.83 kB packages/db/dist/esm/query/compiler/expressions.js430 B packages/db/dist/esm/query/compiler/group-by.js3.56 kB packages/db/dist/esm/query/compiler/index.js6.67 kB packages/db/dist/esm/query/compiler/joins.js2.5 kB packages/db/dist/esm/query/compiler/lazy-targets.js918 B packages/db/dist/esm/query/compiler/order-by.js1.74 kB packages/db/dist/esm/query/compiler/select.js1.4 kB packages/db/dist/esm/query/effect.js4.77 kB packages/db/dist/esm/query/expression-helpers.js1.43 kB packages/db/dist/esm/query/ir.js1.25 kB packages/db/dist/esm/query/live-query-collection.js360 B packages/db/dist/esm/query/live/collection-config-builder.js8.34 kB packages/db/dist/esm/query/live/collection-registry.js264 B packages/db/dist/esm/query/live/collection-subscriber.js1.93 kB packages/db/dist/esm/query/live/internal.js145 B packages/db/dist/esm/query/live/utils.js1.81 kB packages/db/dist/esm/query/optimizer.js2.92 kB packages/db/dist/esm/query/predicate-utils.js2.97 kB packages/db/dist/esm/query/query-once.js359 B packages/db/dist/esm/query/subset-dedupe.js960 B packages/db/dist/esm/scheduler.js1.3 kB packages/db/dist/esm/SortedMap.js1.3 kB packages/db/dist/esm/strategies/debounceStrategy.js247 B packages/db/dist/esm/strategies/queueStrategy.js428 B packages/db/dist/esm/strategies/throttleStrategy.js246 B packages/db/dist/esm/transactions.js3.02 kB packages/db/dist/esm/utils.js927 B packages/db/dist/esm/utils/array-utils.js273 B packages/db/dist/esm/utils/browser-polyfills.js304 B packages/db/dist/esm/utils/btree.js5.61 kB packages/db/dist/esm/utils/comparison.js1.05 kB packages/db/dist/esm/utils/cursor.js457 B packages/db/dist/esm/utils/index-optimization.js1.54 kB packages/db/dist/esm/utils/type-guards.js157 B packages/db/dist/esm/virtual-props.js360 B Caution
Review failed#
Failed to post review comments
Note
Reviews paused#
It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the
reviews.auto_review.auto_pause_after_reviewed_commitssetting.Use the following commands to manage reviews:
@coderabbitai resumeto resume automatic reviews.@coderabbitai reviewto trigger a single review.
Use the checkboxes below for quick actions:
- ▶️ Resume reviews
- 🔍 Trigger review
📝 Walkthrough
Walkthrough#
This PR fixes stale optimistic inserts by adding post-commit pruning logic for direct pending optimistic keys, refactors metadata formatting in transaction creation, and introduces comprehensive test coverage. It also updates CI to Node.js 22.13, adds a release changeset, and documents Git/PR hygiene practices.
Changes#
Stale Optimistic Key Cleanup and Metadata Refactor
Layer / File(s) Summary Post-commit pruning of stale optimistic keys
packages/db/src/collection/state.tscommitPendingTransactionsnow prunes completed direct optimistic upserts and deletes not present inchangedKeys, removing stale keys from pending maps, restoring visibility state frompreviousOptimisticUpserts, and clearing direct trackers.Direct transaction metadata formatting
packages/db/src/collection/mutations.tsInsert, update, and delete transaction metadata objects are refactored from multi-line to single-line literals using DIRECT_TRANSACTION_METADATA_KEY.Test coverage for optimistic replacement
packages/db/tests/collection.test.ts,packages/db/tests/utils.ts,packages/electric-db-collection/tests/electric.test.tsLive-query utilities imported; two collection tests verify optimistic insert removal and live-query updates when server returns a different key; mock config adds optional startSyncfield; Electric integration test validates txid-to-server-key replacement with metadata propagation.Release and Infrastructure Updates
Layer / File(s) Summary Patch release changeset
.changeset/fix-stale-optimistic-server-key.mdDocuments the fix for stale optimistic rows not being cleaned up when sync confirms a different server-generated key. CI Node.js version and Git hygiene
.github/workflows/e2e-tests.yml,AGENTS.mdE2E workflow updated to Node.js 22.13; AGENTS.md adds Git and PR Hygiene section covering branch history rewriting restrictions and best practices. Estimated code review effort#
🎯 3 (Moderate) | ⏱️ ~22 minutes
Possibly related issues#
- TanStack/db#1442: Implements the post-commit pruning of pendingOptimisticDirectUpserts/pendingOptimisticDirectDeletes that directly addresses the stale optimistic row retention bug.
Poem#
🐰 A carrot was planted, then swept by the wind, The server said "No, use this key instead!" With pruning so swift, the old leaf is gone, Fresh metadata glows where optimism shone. 🌱
🚥 Pre-merge checks | ✅ 5
✏️ Tip: You can configure your own custom pre-merge checks in the settings.✅ Passed checks (5 passed)
Check name Status Explanation Title check ✅ Passed The PR title clearly summarizes the main change: fixing stale optimistic rows that persist after sync confirms a server-generated key. Description check ✅ Passed The PR description includes a comprehensive summary of changes, root cause analysis, approach, key invariants, verification steps, and files changed; it exceeds the template requirements by providing detailed technical context. Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request. Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request. ✨ Finishing Touches
📝 Generate docstrings
- Create stacked PR
- Commit on current branch
🧪 Generate unit tests (beta)
- Create PR with unit tests
- Commit unit tests in branch `fix/electric-txid-server-generated-key`
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment `@coderabbitai help` to get the list of available commands and usage tips.Size Change: 0 B
Total Size: 4.24 kB
compressed-size-action::react-db-package-sizeℹ️ View Unchanged
Filename Size packages/react-db/dist/esm/index.js249 B packages/react-db/dist/esm/useLiveInfiniteQuery.js1.32 kB packages/react-db/dist/esm/useLiveQuery.js1.34 kB packages/react-db/dist/esm/useLiveQueryEffect.js355 B packages/react-db/dist/esm/useLiveSuspenseQuery.js567 B packages/react-db/dist/esm/usePacedMutations.js401 B 🧹 Nitpick comments (1)
packages/db/src/collection/state.ts (1)
523-535: ⚡ Quick winRemove dead code:
pendingSyncKeysis collected but never used.The
pendingSyncKeysSet is populated at lines 524-529 but never read - the filtering that previously used it was removed in this refactor. The comment at line 523 ("only when a sync is pending") is also stale since the conditional seeding no longer occurs.♻️ Proposed fix to remove dead code and stale comment
- // Seed optimistic state with pending optimistic mutations only when a sync is pending - const pendingSyncKeys = new Set<TKey>() - for (const transaction of this.pendingSyncedTransactions) { - for (const operation of transaction.operations) { - pendingSyncKeys.add(operation.key as TKey) - } - } + // Seed optimistic state with pending optimistic mutations. + // Stale keys are pruned later in commitPendingTransactions when sync confirms. for (const [key, value] of this.pendingOptimisticUpserts) {As per coding guidelines: "Unused code (dead code, commented-out code, debug artifacts)" should be removed, and "Update or remove comments when refactoring code to keep documentation in sync with implementation".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/db/src/collection/state.ts` around lines 523 - 535, Remove the dead code and stale comment: delete the unused pendingSyncKeys Set declaration and its population loop (references to pendingSyncKeys and the for-loop iterating this.pendingSyncedTransactions), and update the comment that mentions "only when a sync is pending" to reflect the current behavior; leave the existing seeding of this.optimisticUpserts from pendingOptimisticUpserts and this.optimisticDeletes from pendingOptimisticDeletes intact (they are the active logic).🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. Nitpick comments: In `@packages/db/src/collection/state.ts`: - Around line 523-535: Remove the dead code and stale comment: delete the unused pendingSyncKeys Set declaration and its population loop (references to pendingSyncKeys and the for-loop iterating this.pendingSyncedTransactions), and update the comment that mentions "only when a sync is pending" to reflect the current behavior; leave the existing seeding of this.optimisticUpserts from pendingOptimisticUpserts and this.optimisticDeletes from pendingOptimisticDeletes intact (they are the active logic).
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID:
cbdc375e-a1cf-4b61-a611-ddb7f54781e2📥 Commits
Reviewing files that changed from the base of the PR and between 2735c3e5aa358900a4fa73f144be7f23a99cc38b and 8613e6526f01a9eaae1661f2a63a226affe8aa4c.
📒 Files selected for processing (2)
.changeset/fix-stale-optimistic-server-key.mdpackages/db/src/collection/state.ts
✅ Files skipped from review due to trivial changes (1)
- .changeset/fix-stale-optimistic-server-key.md
samwillis(GitHub profile, opens in a new tab) approved these changes
LGTM, and makes perfect sense!
samwillis(GitHub profile, opens in a new tab) closed this
Sign in to join this conversation.
More templates
@tanstack/angular-db
@tanstack/browser-db-sqlite-persistence
@tanstack/capacitor-db-sqlite-persistence
@tanstack/cloudflare-durable-objects-db-sqlite-persistence
@tanstack/db
@tanstack/db-ivm
@tanstack/db-sqlite-persistence-core
@tanstack/electric-db-collection
@tanstack/electron-db-sqlite-persistence
@tanstack/expo-db-sqlite-persistence
@tanstack/node-db-sqlite-persistence
@tanstack/offline-transactions
@tanstack/powersync-db-collection
@tanstack/query-db-collection
@tanstack/react-db
@tanstack/react-native-db-sqlite-persistence
@tanstack/rxdb-db-collection
@tanstack/solid-db
@tanstack/svelte-db
@tanstack/tauri-db-sqlite-persistence
@tanstack/trailbase-db-collection
@tanstack/vue-db
commit:
68992d5