fix(db): preserve exact D2 rows during sync reconciliation #1794
main fromrefs/pull/1794/head·opened 🎯 Changes#
Fixes #1783.
D2 applies an update by removing the old row and adding the new one. The row being removed must exactly match the row previously added. The state diverged like this:
- An optimistic insert added one snapshot of the row to D2.
- Sync confirmation emitted another insert with the same key. This was correctly skipped so D2 would not count the row twice.
- D2 therefore kept the optimistic snapshot, while the collection later described the row using its sync-confirmed snapshot.
- A later update tried to remove the collection’s snapshot, but D2 held the optimistic snapshot.
- The snapshots represented the same key but were not identical, so D2 threw
Query contributors with the same row key are not congruent.
Any differing property can cause this; it is not specific to $synced or $origin. A focused test demonstrates the same mismatch with a normal status property. Changing the hashing would only hide the mismatch by making different rows appear equal. The correct fix is to remove the exact row that was previously added.
I bisected the regression to #1740. That PR added the congruence check which exposed the incorrect removal. This check is correct; the source bookkeeping was not.
The fix remembers the exact row sent to D2 for each key and uses it for later updates and deletes. The same bookkeeping also prevents duplicate inserts.
The regression test covers the original optimistic insert and synchronous sync confirmation, then verifies that the collection and live query each contain exactly one updated row.
✅ Checklist#
- I have tested this code locally with
pnpm test.
🚀 Release Impact#
- This change affects published code, and I have generated a changeset.
- This change is docs/CI/dev-only (no release).
Summary by CodeRabbit#
Bug Fixes
- Fixed live queries throwing when optimistically inserted rows are updated after synchronous sync confirmation.
- Improved handling of row updates, deletions, and duplicate inserts to keep query and collection results consistent.
- Ensured updates correctly replace previously received row values without stale or duplicate data.
Tests
- Added coverage for optimistic insert updates and accurate row reconciliation.
- 1
We hit this in production on 0.8.7 through a path that does not involve an optimistic insert or
writeInsert: plaincollection.updatecalls on an existing key, confirmed by ordinary sync commits. The trigger is a new update on a key starting while a sync commit for that key is already deferred behind an earlier in-flight mutation. Claude reduced it to the script below against bare@tanstack/db@0.8.7(main is at the 0.8.7 commit as of 2026-09-06). Applying this PR's diff to 0.8.7 makes it pass, so posting here in case the extra case is useful for the regression suite.import { createCollection, createLiveQueryCollection, eq } from '@tanstack/db' let hooks const releases = new Map() const col = createCollection({ id: 'rows', getKey: (r) => r.id, startSync: true, sync: { rowUpdateMode: 'full', sync: (p) => { hooks = p p.begin() p.write({ type: 'insert', value: { id: 'c1', n: 0 } }) p.commit() p.markReady() }, }, // Persist resolves when the test releases it, so confirmations can be ordered explicitly. onUpdate: ({ transaction }) => new Promise((resolve) => releases.set(transaction.mutations[0].modified.n, resolve)), }) const lq = createLiveQueryCollection({ startSync: true, query: (q) => q.from({ c: col }).where(({ c }) => eq(c.id, 'c1')), }) lq.subscribeChanges(() => {}) col.subscribeChanges((changes) => { for (const ch of changes) console.log(' event', ch.type, ch.value.n, ch.value.$synced, 'prev', ch.previousValue?.n, ch.previousValue?.$synced) }) const mutate = (n) => col.update('c1', (d) => { d.n = n }).isPersisted.promise.catch(() => {}) const confirm = (n) => { hooks.begin() hooks.write({ type: 'update', value: { id: 'c1', n } }) hooks.commit() releases.get(n)() } const tick = () => new Promise((r) => setTimeout(r, 0)) // m = collection.update, c = server confirmation for that update. // c1 arrives while m2 is in flight, so its commit is deferred; m3 starts before it applies. for (const step of ['m1', 'm2', 'c1', 'm3', 'c2', 'c3', 'm4']) { const n = Number(step[1]) if (step[0] === 'm') mutate(n) else confirm(n) await tick() console.log(step, 'collection.$synced =', col.get('c1').$synced, '| liveQuery.$synced =', lq.toArray[0]?.$synced, 'n =', lq.toArray[0]?.n) }Output on 0.8.7 (Node 24, last steps only):
event update 3 false prev 2 false m3 collection.$synced = false | liveQuery.$synced = false n = 3 c2 collection.$synced = false | liveQuery.$synced = false n = 3 c3 collection.$synced = true | liveQuery.$synced = false n = 3 <- row flipped to synced, no change event event update 4 false prev 3 true <- previousValue is a row the live query never received m4 Error: Query contributors with the same row key are not congruentAfter that, every further write on the key throws.
Two sequences that do not throw, for contrast:
m1 m2 c1 c2 m3(the deferred commits apply before the next update starts, and the$syncedflip is published as an event) andm1 m2 m3 c1 c2 c3 m4(all three in flight, then all confirmations).So the collection-side defect in this path is that the last deferred commit updates the visible row without publishing the change, and the subscriber then retracts the collection's
previousValueinstead of the row it actually contributed. This PR's exact-row bookkeeping inCollectionSubscriberhandles the second half, which is enough to keep the live query consistent. For completeness, #1795 alone does not change the outcome here. The missing publication on the state side may still be worth a cell in the publication oracle's overlap matrix mentioned on #1783.This was investigated and reduced with Claude Code, and the script and output above are from actual runs.
Closing as superseded by merged #1797, which incorporates and tests exact D2 before-images during source reconciliation. The consolidated PR also includes direct Query DB optimistic write-back regression traces. This branch no longer needs a separate merge.
KyleAMathews(GitHub profile, opens in a new tab) closed this
Sign in to join this conversation.
No actionable comments were generated in the recent review. 🎉
ℹ️ Recent review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Team
Run ID:
0769a2a9-3cfa-48e8-9002-cb8e12f189d0📥 Commits
Reviewing files that changed from the base of the PR and between 68366ecaeef6c12a13402b558bd4a68d7519442f and 8abd32e080818e4f143dc192ca72429b1e9e702b.
📒 Files selected for processing (5)
.changeset/sixty-nights-refuse.mdpackages/db/src/query/effect.tspackages/db/src/query/live/collection-subscriber.tspackages/db/src/query/live/utils.tspackages/db/tests/collection-subscriber-duplicate-inserts.test.tsIncluded review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review.
📝 Walkthrough
Walkthrough#
Live query and effect pipelines now track the exact rows contributed to D2. Updates and deletes retract those stored rows. Tests cover row reconciliation and updates to synchronously confirmed optimistic inserts. A patch changeset records the fix.
Changes#
Live query reconciliation
packages/db/src/query/live/utils.tsprepareChangesForD2tracks sent rows in a map. It removes duplicate inserts and setspreviousValueto the exact previously sent row for updates and deletes.packages/db/src/query/effect.ts,packages/db/src/query/live/collection-subscriber.tspackages/db/tests/collection-subscriber-duplicate-inserts.test.ts,.changeset/sixty-nights-refuse.mdEstimated code review effort: 3 (Moderate) | ~20 minutes
Merge Risk: 🔵 Low · up to
8abd3The PR correctly preserves exact rows during live-query reconciliation, preventing duplicate and update failures in normal operation. A bounded recovery risk remains if graph processing fails after bookkeeping is updated, which warrants owner awareness or follow-up but does not block merge under normal checks.
Suggested reviewers:
kevin-dp,kyleamathewsSequence Diagram(s)#
🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)#
✅ Passed checks (4 passed)
#1783by tracking exact rows for D2 reconciliation, preventing congruence errors after optimistic inserts and sync confirmation. The regression test covers the required optim…Full details: Linked Issues check
Explanation
The changes address issue
#1783by tracking exact rows for D2 reconciliation, preventing congruence errors after optimistic inserts and sync confirmation. The regression test covers the required optimistic insert, synchronous confirmation, and later update flow.Full details: Docstring Coverage
Explanation
Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 4 files. (1 skipped: 1 unsupported.)
✨ Finishing Touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
❤️ Share