feat(db): ordered snapshot / layout-revision contract (RFC #1623 phase 4) #1669
main fromrefs/pull/1669/head·merged by KyleAMathews(GitHub profile, opens in a new tab) Phase 4 of the live-query platform RFC (#1623): the ordered snapshot / layout contract. Stacked on #1642 (observer migration) — review/merge that first; this PR's base is refactor/live-query-observer so the diff is Phase 4 only.
Problem#
An orderBy live query that reorders its rows without changing any projected row value (an "order-only move") is swallowed by the collection's value-diff: .values()/.entries() re-sort internally, but no change event fires, so useLiveQuery keeps rendering the stale order. This is the last universal expected-fail in the cross-adapter conformance suite (tracked as #1601).
Approach#
The RFC is explicit that this should be "an explicit layout-revision requirement, not a hidden forced-update path". So instead of forging a row update:
- The live-query flush captures the retracted side of each change and, after
commit(), detects an order-only move (projected value deep-equal,orderByIndexmoved) and publishes a first-class empty layout-change notification via a newCollectionChangesManager.emitLayoutChangeEvent(). It reuses the existing empty-batch delivery already used for the ready signal — subscribers re-read, nothing is faked. - The shared observer snapshot gains
layoutRevision, which increments on any visible membership, ordering, or order-only-move change. This is the canonical contract the RFC wants for future fine-grained materializers; adapters currently pick up the reorder through their existing wholesale re-read.
Result#
order-only-move is removed from UNIVERSAL_EXPECTED_FAIL and now passes on all five adapters (React, Vue, Svelte, Solid, Angular) — 26/26 conformance each. Full @tanstack/db suite green (2464 tests), all five adapter suites green.
Coverage#
packages/db/tests/live-query-order-only-move.test.ts— core mechanism: republish +layoutRevisionbump on an order-only move; no bump when order is unchanged (no spurious notification); bump on membership change.- Conformance
order-only-movescenario now a real pass across all adapters.
Relationship to #1601#
@v-anton's #1601 fixes the same bug via a forced row update. This PR takes the RFC-sanctioned layout-revision approach instead (a distinct, first-class notification + layoutRevision), so it supersedes rather than duplicates that path. Happy to coordinate on which lands — flagging for maintainer decision.
🤖 Generated with Claude Code
Summary by CodeRabbit#
- New Features
- Added a framework-agnostic live-query observer with granular or whole-snapshot updates.
- Live-query snapshots now expose data, status, ordering, and revision information.
- Bug Fixes
- Ordered live queries now republish when items move without changing their values.
- Improved handling of nested includes, filtering, truncation, refetching, and stale keys.
- Documentation
- Updated release notes to clarify live-query synchronization behavior.
- 1
More templates
@tanstack/angular-db
npm i https://pkg.pr.new/@tanstack/angular-db@1669@tanstack/browser-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/browser-db-sqlite-persistence@1669@tanstack/capacitor-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/capacitor-db-sqlite-persistence@1669@tanstack/cloudflare-durable-objects-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/cloudflare-durable-objects-db-sqlite-persistence@1669@tanstack/db
npm i https://pkg.pr.new/@tanstack/db@1669@tanstack/db-ivm
npm i https://pkg.pr.new/@tanstack/db-ivm@1669@tanstack/db-sqlite-persistence-core
npm i https://pkg.pr.new/@tanstack/db-sqlite-persistence-core@1669@tanstack/electric-db-collection
npm i https://pkg.pr.new/@tanstack/electric-db-collection@1669@tanstack/electron-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/electron-db-sqlite-persistence@1669@tanstack/expo-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/expo-db-sqlite-persistence@1669@tanstack/node-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/node-db-sqlite-persistence@1669@tanstack/offline-transactions
npm i https://pkg.pr.new/@tanstack/offline-transactions@1669@tanstack/powersync-db-collection
npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1669@tanstack/query-db-collection
npm i https://pkg.pr.new/@tanstack/query-db-collection@1669@tanstack/react-db
npm i https://pkg.pr.new/@tanstack/react-db@1669@tanstack/react-native-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/react-native-db-sqlite-persistence@1669@tanstack/rxdb-db-collection
npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1669@tanstack/solid-db
npm i https://pkg.pr.new/@tanstack/solid-db@1669@tanstack/svelte-db
npm i https://pkg.pr.new/@tanstack/svelte-db@1669@tanstack/tauri-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/tauri-db-sqlite-persistence@1669@tanstack/trailbase-db-collection
npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1669@tanstack/vue-db
npm i https://pkg.pr.new/@tanstack/vue-db@1669commit:
c27d572Size Change: +692 B (+0.54%)
Total Size: 128 kB
📦 View Changed
Filename Size Change packages/db/dist/esm/collection/change-events.js1.44 kB +6 B (+0.42%) packages/db/dist/esm/collection/changes.js1.43 kB +24 B (+1.71%) packages/db/dist/esm/collection/index.js3.83 kB +100 B (+2.68%) packages/db/dist/esm/collection/state.js5.51 kB +33 B (+0.6%) packages/db/dist/esm/collection/subscription.js3.82 kB +74 B (+1.98%) packages/db/dist/esm/collection/sync.js2.94 kB +56 B (+1.94%) packages/db/dist/esm/live-query-observer.js2.29 kB +262 B (+12.93%) ⚠️ packages/db/dist/esm/query/live/collection-config-builder.js9.24 kB +137 B (+1.5%) compressed-size-action::db-package-sizeℹ️ View Unchanged
Filename Size packages/db/dist/esm/collection/cleanup-queue.js810 B packages/db/dist/esm/collection/events.js434 B packages/db/dist/esm/collection/indexes.js1.99 kB packages/db/dist/esm/collection/lifecycle.js1.7 kB packages/db/dist/esm/collection/mutations.js2.47 kB packages/db/dist/esm/collection/transaction-metadata.js144 B packages/db/dist/esm/deferred.js207 B packages/db/dist/esm/errors.js5.13 kB packages/db/dist/esm/event-emitter.js748 B packages/db/dist/esm/index.js3.21 kB packages/db/dist/esm/indexes/auto-index.js829 B packages/db/dist/esm/indexes/base-index.js784 B packages/db/dist/esm/indexes/basic-index.js2.17 kB packages/db/dist/esm/indexes/btree-index.js2.29 kB packages/db/dist/esm/indexes/index-registry.js820 B packages/db/dist/esm/indexes/reverse-index.js557 B packages/db/dist/esm/live-query-adapter.js318 B packages/db/dist/esm/local-only.js916 B packages/db/dist/esm/local-storage.js2.12 kB packages/db/dist/esm/optimistic-action.js359 B packages/db/dist/esm/paced-mutations.js496 B packages/db/dist/esm/proxy.js3.75 kB packages/db/dist/esm/query/builder/functions.js1.47 kB packages/db/dist/esm/query/builder/index.js5.84 kB packages/db/dist/esm/query/builder/ref-proxy.js1.24 kB packages/db/dist/esm/query/compiler/evaluators.js1.89 kB packages/db/dist/esm/query/compiler/expressions.js430 B packages/db/dist/esm/query/compiler/group-by.js3.56 kB packages/db/dist/esm/query/compiler/index.js6.67 kB packages/db/dist/esm/query/compiler/joins.js2.5 kB packages/db/dist/esm/query/compiler/lazy-targets.js923 B packages/db/dist/esm/query/compiler/order-by.js1.74 kB packages/db/dist/esm/query/compiler/select.js1.53 kB packages/db/dist/esm/query/effect.js4.77 kB packages/db/dist/esm/query/expression-helpers.js1.43 kB packages/db/dist/esm/query/ir.js1.25 kB packages/db/dist/esm/query/live-query-collection.js360 B packages/db/dist/esm/query/live/collection-registry.js264 B packages/db/dist/esm/query/live/collection-subscriber.js1.93 kB packages/db/dist/esm/query/live/internal.js145 B packages/db/dist/esm/query/live/utils.js1.81 kB packages/db/dist/esm/query/optimizer.js2.92 kB packages/db/dist/esm/query/predicate-utils.js2.97 kB packages/db/dist/esm/query/query-once.js359 B packages/db/dist/esm/query/subset-dedupe.js960 B packages/db/dist/esm/scheduler.js1.3 kB packages/db/dist/esm/SortedMap.js1.3 kB packages/db/dist/esm/strategies/debounceStrategy.js247 B packages/db/dist/esm/strategies/queueStrategy.js428 B packages/db/dist/esm/strategies/throttleStrategy.js246 B packages/db/dist/esm/transactions.js3.04 kB packages/db/dist/esm/utils.js927 B packages/db/dist/esm/utils/array-utils.js273 B packages/db/dist/esm/utils/browser-polyfills.js304 B packages/db/dist/esm/utils/btree.js5.61 kB packages/db/dist/esm/utils/comparison.js1.15 kB packages/db/dist/esm/utils/cursor.js457 B packages/db/dist/esm/utils/index-optimization.js2.39 kB packages/db/dist/esm/utils/type-guards.js157 B packages/db/dist/esm/utils/uuid.js449 B packages/db/dist/esm/virtual-props.js360 B Size Change: 0 B
Total Size: 3.81 kB
compressed-size-action::react-db-package-sizeℹ️ View Unchanged
Filename Size packages/react-db/dist/esm/index.js249 B packages/react-db/dist/esm/useLiveInfiniteQuery.js1.32 kB packages/react-db/dist/esm/useLiveQuery.js920 B packages/react-db/dist/esm/useLiveQueryEffect.js355 B packages/react-db/dist/esm/useLiveSuspenseQuery.js567 B packages/react-db/dist/esm/usePacedMutations.js401 B KyleAMathews(GitHub profile, opens in a new tab) requested changes
Code review#
Found 2 issues, both reproduced with failing regression tests against
d7628551f219cab27ddd542bffda40c8ee7f5aad:- A graph flush containing both an ordinary projected-value update and an order-only move publishes twice.
commit()synchronously emits the ordinary row batch, thenemitLayoutChangeEvent()emits a second empty batch even though listeners already observe the final values and order. My regression expected one observer callback and received two. Please coalesce layout dirtiness into the commit publication (or otherwise suppress the separate layout event when that commit already publishes), and cover this with an exact-one-callback mixed-batch test.
- The layout fix does not cover ordered child collections produced by
includes. On retract-then-insert, the insertion side replacesexisting.valuebut leaves the retractedorderByIndex; later the child collection commits without any layout-only signal. I reproduced this with a child projection that omits its sort field: after movingc1behindc2, the child collection remained ordered[c1, c2]instead of[c2, c1]. Please preserve the insertion-side order metadata, publish child layout-only changes through the same atomic mechanism, and add an ordered-includes regression.
Generated with Claude Code
- A graph flush containing both an ordinary projected-value update and an order-only move publishes twice.
Reproductions for the requested changes#
I verified both findings against head
d7628551f219cab27ddd542bffda40c8ee7f5aadwith focused tests. Here are the essential regressions to add.1. Mixed value update + order-only move must publish exactly once#
Using the existing ordered query whose projection omits
age:it(`publishes a mixed value update and order-only move exactly once`, async () => { const source = makeSource() const lq = await makeOrderedByAge(source) const observer = createLiveQueryObserver< { id: string; name: string }, string >(lq as any) let notifications = 0 observer.subscribe(() => notifications++) notifications = 0 // exclude subscribeChanges' initial-state publication source.utils.begin() source.utils.write({ type: `update`, value: { id: `1`, name: `Alicia`, age: 30 }, }) source.utils.write({ type: `update`, value: { id: `2`, name: `Bob`, age: 99 }, }) source.utils.commit() const after = observer.getSnapshot() expect((after.data as Array<Person>).map(({ id, name }) => [id, name])).toEqual([ [`1`, `Alicia`], [`3`, `Carol`], [`2`, `Bob`], ]) expect(notifications).toBe(1) })Result on the PR head:
AssertionError: expected 2 to be 1The first callback comes from
commit()'s ordinary row update. The collection already has its final value and ordering at that point. The post-commitemitLayoutChangeEvent()then produces a redundant second callback. Layout dirtiness needs to participate in the same publication as row changes; a layout-only flush should publish only when no ordinary publication represents that flush.2. Ordered included child must consume the new order metadata and publish its move#
it(`publishes an ordered included child move exactly once`, async () => { const parents = createCollection( mockSyncCollectionOptions<{ id: string }>({ id: `parents`, getKey: ({ id }) => id, initialData: [{ id: `p1` }], }), ) const children = createCollection( mockSyncCollectionOptions<{ id: string parentId: string name: string position: number }>({ id: `children`, getKey: ({ id }) => id, initialData: [ { id: `c1`, parentId: `p1`, name: `One`, position: 1 }, { id: `c2`, parentId: `p1`, name: `Two`, position: 2 }, ], }), ) const lq = createLiveQueryCollection((q) => q.from({ parent: parents }).select(({ parent }) => ({ id: parent.id, children: q .from({ child: children }) .where(({ child }) => eq(child.parentId, parent.id)) .orderBy(({ child }) => child.position) .select(({ child }) => ({ id: child.id, name: child.name })), })), ) await lq.preload() const childCollection = lq.get(`p1`)!.children let notifications = 0 const subscription = childCollection.subscribeChanges( () => notifications++, { includeInitialState: false }, ) children.utils.begin() children.utils.write({ type: `update`, value: { id: `c1`, parentId: `p1`, name: `One`, position: 3 }, }) children.utils.commit() expect([...childCollection.values()].map(({ id }) => id)).toEqual([ `c2`, `c1`, ]) expect(notifications).toBe(1) subscription.unsubscribe() })Result on the PR head:
AssertionError: expected [ 'c1', 'c2' ] to deeply equal [ 'c2', 'c1' ]There are two gaps in this path:
- On retract-then-insert,
existing.valueis replaced by the insertion-side value, butexisting.orderByIndexis not replaced by the insertion-side index. - The child collection calls
commit()but has no equivalent layout-only publication when the projected child value is unchanged.
Relevant code:
- https://github.com/TanStack/db/blob/d7628551f219cab27ddd542bffda40c8ee7f5aad/packages/db/src/query/live/collection-config-builder.ts#L886-L918
- https://github.com/TanStack/db/blob/d7628551f219cab27ddd542bffda40c8ee7f5aad/packages/db/src/query/live/collection-config-builder.ts#L1977-L2005
I also tested the stable-rank case (
age: 20 → 21, still first). After excluding the expected initial subscription publication, it already produces zero callback delta, preserves snapshot identity, and keepslayoutRevisionstable. No change is needed for that case.- On retract-then-insert,
Thanks for the thorough review, Kyle — both findings were spot on. Fixed in c9ec751d, with the two regressions you specified added first (43f1a7d6) so the failure is on record.
1. Mixed value update + order-only move now publishes exactly once. Replaced
hasOrderOnlyMovewithneedsLayoutOnlyPublication, which fires the layout event only when the commit published nothing else. Any real insert/delete/value-changed update already notifies subscribers (who re-read the re-sorted collection), so the separate layout event is skipped in that case. Covered by the exact-one-callback mixed-batch test.2. Ordered
includeschildren now consume the insertion-side order metadata and publish their move. Two gaps, both closed:- The child accumulate replaced
existing.valueon the insert side but left the retractedorderByIndex. It now updatesorderByIndexon insert and capturespreviousValue/previousOrderByIndexon retract (both the single-level output and the nested-includes buffer accumulate). - The child flush committed without a layout-only publication when the projected child value was unchanged. It now runs the same
needsLayoutOnlyPublicationcheck after the childcommit()and publishes through a sharedemitLayoutChangehelper.
Covered by the ordered-includes regression.
Depth. Since
flushIncludesStateis recursive and the child layout-emit runs at every level, I added a two-level nested-includes regression (org → teams → members, moving a grandchild) to guard that this holds beyond one level — verified it goes red if the child-flush publication is removed. An independent pass also confirmed the exactly-once contract and re-sort at 1/2/3 levels, plus the insert/delete-combined-with-move and multiple-moves-in-one-commit cases.Stable-rank case (sort field changes but position doesn't, value unchanged) needs no change, as you noted — it produces zero layout events; there's a test asserting that too.
Full
@tanstack/dbsuite and all five adapter conformance suites are green.- The child accumulate replaced
KyleAMathews(GitHub profile, opens in a new tab) requested changes
[P1] Tie the layout revision to the sync transaction that changes the layout#
This is reproducible on the current head,
df9403a.markLayoutChange()currently increments the collection-widelayoutRevisionand setspendingLayoutChangebefore the live-query result collection callscommit(). A normal sync commit is not always applied at that point:commitPendingTransactions()parks committed sync transactions while a user transaction on the same collection ispersisting.That lets the layout clock describe state that is not visible yet.
Deterministic reproduction#
- Create a live query ordered by
age, but project only{ id, name }. - Read a detached observer snapshot. Its initial order is
[2, 1, 3]. - Start a non-optimistic mutation on the result collection and keep its
onUpdatehandler unresolved. The user transaction is nowpersisting. - Update source row
2so its age moves it to the end. The projected row value remains deep-equal. - The query pipeline calls
markLayoutChange(), advancinglayoutRevision, then commits its result-collection sync transaction. - The sync transaction is parked behind the persisting mutation, so the visible order is still
[2, 1, 3]. - A detached
getSnapshot()sees the new layout revision and caches the old entries against it. - Resolve the mutation. The parked sync transaction applies and the collection order becomes
[1, 3, 2]. - Because the projected values are deep-equal, no row event or state-revision increment occurs. The layout revision also does not advance again.
- The next detached snapshot sees neither revision change and remains stuck on
[2, 1, 3].
I added that regression locally. Before the fix it fails at the final assertion:
expected [ '2', '1', '3' ] to deeply equal [ '1', '3', '2' ]There is a related attached-observer hazard: while the sync transaction is parked, an unrelated forced publication can consume the collection-wide
pendingLayoutChangeflag. The eventual order-only commit then has neither row events nor a layout marker, so it cannot send the corrective publication.Requested fix#
Please store layout dirtiness on the exact pending sync transaction that produced it, rather than on the collection:
interface PendingSyncedTransaction<...> { // existing fields layoutChanged: boolean }_markLayoutChange()should mark the active, uncommitted sync transaction. WhencommitPendingTransactions()actually applies committed transactions, it should OR theirlayoutChangedflags and advance/publish the collection layout revision after applying state and immediately before emitting the final batch.This preserves the required properties:
- The revision advances only when the matching reordered state becomes visible.
- The layout signal and row-event batch publish atomically.
- Multiple transactions applied in one flush coalesce into one layout publication.
- Unrelated optimistic recomputations cannot steal the marker.
- Cleanup naturally discards the marker with the pending transaction.
I verified this design locally: the new regression turns green, all 10 order-only layout tests pass, and the broader collection transaction suites pass (149 tests total, with no type errors). ESLint, Prettier, and
git diff --checkalso pass.The mixed membership/order batching and nested ordered-
includesfixes on this head look correct. This transaction-boundary race is the remaining blocker.- Create a live query ordered by
Actionable comments posted: 1
🧹 Nitpick comments (8)
packages/db/src/collection/sync.ts (1)
74-88: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winReuse
getActivePendingSyncTransaction()here.Lines 76-85 repeat the lookup and both guards already implemented by the private helper
getActivePendingSyncTransaction()at lines 292-306 of this file. The behavior is identical, including both error types. Reuse the helper so future changes to the guard logic apply to every caller.As per coding guidelines: "Extract common logic into utility functions when identical or near-identical code blocks appear in multiple places".
♻️ Proposed refactor
/** Mark the active sync transaction as changing collection layout. */ public markLayoutChange(): void { - const pendingTransaction = - this.state.pendingSyncedTransactions[ - this.state.pendingSyncedTransactions.length - 1 - ] - if (!pendingTransaction) { - throw new NoPendingSyncTransactionWriteError() - } - if (pendingTransaction.committed) { - throw new SyncTransactionAlreadyCommittedWriteError() - } - - pendingTransaction.layoutChanged = true + this.getActivePendingSyncTransaction().layoutChanged = true }Note:
getActivePendingSyncTransactionis declared belowmarkLayoutChange. Method hoisting inside a class makes this call order safe.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/db/src/collection/sync.ts` around lines 74 - 88, Update markLayoutChange() to obtain the transaction through the existing private getActivePendingSyncTransaction() helper instead of duplicating the pending transaction lookup and committed-state guards, then set layoutChanged on the returned transaction while preserving the helper’s existing error behavior.Source: Coding guidelines
packages/db/src/query/live/collection-config-builder.ts (1)
905-912: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winOne multiplicity-accumulation rule is implemented three times in this file. Each site applies the same four rules to a
Changes<T>entry: adddeletes, recordpreviousValue/previousOrderByIndexon retract, addinsertsand replacevalueon insert, and overwriteorderByIndexonly when it is defined. This PR added the same twoprevious*lines to all three sites, so a future change to the order-only-move contract must land in three places; a missed site produces a silent layout-notification gap. Extract oneapplyMultiplicity<T>(existing, value, orderByIndex, multiplicity)helper and call it from every site.packages/db/src/query/live/collection-config-builder.ts#L905-L912: replace the inline branches insetupIncludesOutputwithapplyMultiplicity(existing, childResult, _orderByIndex, multiplicity).packages/db/src/query/live/collection-config-builder.ts#L1337-L1344: replace the inline branches insetupNestedPipelineswith the same call.packages/db/src/query/live/collection-config-builder.ts#L2340-L2343: replace the retract and insert branches inaccumulateChangeswithapplyMultiplicity(changes, value, orderByIndex, multiplicity).
As per coding guidelines: "Extract common logic into utility functions when identical or near-identical code blocks appear in multiple places".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/db/src/query/live/collection-config-builder.ts` around lines 905 - 912, The multiplicity accumulation logic is duplicated across three sites; extract a shared generic applyMultiplicity<T> helper that handles deletes, previousValue/previousOrderByIndex on retracts, inserts, value replacement, and defined orderByIndex updates. In packages/db/src/query/live/collection-config-builder.ts at lines 905-912 and 1337-1344, replace the inline branches with applyMultiplicity(existing, childResult, _orderByIndex, multiplicity); at lines 2340-2343, replace the retract/insert branches with applyMultiplicity(changes, value, orderByIndex, multiplicity), preserving all existing behavior.Source: Coding guidelines
packages/db/tests/live-query-observer.test.ts (1)
593-617: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd empty-source no-op coverage.
The order-only move and stable-rank assertions already exist. Add an empty-source test that asserts
datais[]andlayoutRevisionremains unchanged after a no-op commit.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/db/tests/live-query-observer.test.ts` around lines 593 - 617, Add a test near the existing layoutRevision observer cases that creates an empty source, subscribes a live query observer, records its initial snapshot, performs a begin/commit cycle without writes, and asserts the snapshot data remains [] while layoutRevision is unchanged; dispose the observer afterward.Source: Coding guidelines
packages/db/tests/conformance/suite.ts (1)
42-42: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueUpdate the stale section comment for
order-only-move.UNIVERSAL_EXPECTED_FAILis now empty, but line 686 still labels the following section---- tail: universal expected-fail ---------------------------. Theorder-only-movescenario at line 688 is now an ordinary passing scenario. Update or remove that heading so readers do not treat the scenario as expected-fail.As per coding guidelines: "Keep comments that explain non-obvious behavior... remove outdated comments when refactoring".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/db/tests/conformance/suite.ts` at line 42, Update or remove the stale “universal expected-fail” section heading near the order-only-move scenario, since UNIVERSAL_EXPECTED_FAIL is empty and order-only-move is a normal passing scenario. Ensure the surrounding comment accurately describes the scenario’s current status.Source: Coding guidelines
packages/angular-db/src/index.ts (1)
261-266: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAngular reads the collection directly; other adapters read the observer snapshot.
syncDataFromCollectionreadscurrentCollection.entries(),values(), andstatuson every notify. Vue, Svelte, and Solid readobserver.getSnapshot().statusthroughsyncFromObserver(seepackages/vue-db/src/useLiveQuery.ts:371-377). The observer maintainsvisibleStatusand a captured entries view, so the two sources can report different statuses for the same publication. Consider readingobserver.getSnapshot()forstatus(and optionallystate/data) so all adapters share one status contract.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/angular-db/src/index.ts` around lines 261 - 266, The Angular sync path currently reads collection state directly, unlike other adapters that use the observer snapshot. Update syncDataFromCollection and its call sites in the subscription flow to consume observer.getSnapshot() for status, and reuse snapshot state/data where applicable, preserving the post-subscribe synchronization behavior.packages/db/src/live-query-observer.ts (1)
495-500: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winA throwing listener stops delivery for the whole batch.
subRecord.listener(publication.changes)runs unguarded. If one listener throws, the loop exits, the remaining targets never receive this publication, and publications still inpublicationQueuestay queued until the nextemit(). Framework listeners run user code (Vue/Svelte/Solid state updates), so a throw is reachable. Consider isolating each listener call and rethrowing after the queue drains.♻️ Suggested isolation
if (deliver) { + let deliveryError: unknown + let hasError = false for (const subRecord of publication.targets) { if (this.disposed) return - subRecord.listener(publication.changes) + try { + subRecord.listener(publication.changes) + } catch (error) { + if (!hasError) { + hasError = true + deliveryError = error + } + } } + if (hasError) throw deliveryError }Note: rethrowing after the inner loop still aborts the outer
while. If you want the queue to always drain, collect the error and rethrow after the loop ends.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/db/src/live-query-observer.ts` around lines 495 - 500, Update the publication delivery logic around the listener invocation in the observer’s emit flow to isolate each subRecord.listener call, allowing all targets and queued publications to be processed even when a listener throws. Capture the first thrown error, continue draining the inner and outer publication queues, then rethrow after queue processing completes while preserving the disposed checks and existing delivery behavior.packages/solid-db/tests/useLiveQuery.test.tsx (1)
586-586: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winReplace the fixed setup delays with
waitFor.Lines 586, 641, 646, and 665 gate assertions on fixed 10ms, 20ms, and 50ms sleeps. Under a loaded CI runner these windows can expire before the collection settles, which makes the tests flaky. The first new test at Line 109 already uses
waitForcorrectly.Keep the synchronous assertions that are the subject of the tests (Lines 592-594 after
setMinAge, Lines 113-114 aftersetCurrent). Change only the setup waits.♻️ Proposed change for the narrowing test setup
- await new Promise((resolve) => setTimeout(resolve, 50)) - expect(rendered.result.state.size).toBe(3) // all three ages > 10 + // all three ages > 10 + await waitFor(() => expect(rendered.result.state.size).toBe(3))♻️ Proposed change for the superseded-continuation test setup
- await new Promise((resolve) => setTimeout(resolve, 10)) - expect(rendered.result.isLoading).toBe(true) + await waitFor(() => expect(rendered.result.isLoading).toBe(true)) // Switch collections while the slow fetch is still awaiting readiness. setUseSlow(false) - await new Promise((resolve) => setTimeout(resolve, 10)) - expect(rendered.result.state.has(`1`)).toBe(true) + await waitFor(() => expect(rendered.result.state.has(`1`)).toBe(true))For the final wait at Line 665, await the observable effect instead of a fixed delay:
markReadyA!() - await new Promise((resolve) => setTimeout(resolve, 20)) - - expect(rendered.result.state.has(`stale`)).toBe(false) + await waitFor(() => expect(rendered.result.status).toBe(`ready`)) + expect(rendered.result.state.has(`stale`)).toBe(false)Also applies to: 641-646, 665-665
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/solid-db/tests/useLiveQuery.test.tsx` at line 586, Replace the fixed setup sleeps at the identified waits in useLiveQuery tests with waitFor-based conditions that observe the collection state or effect completion. Preserve the synchronous assertions after setMinAge and setCurrent unchanged, and for the final wait await the observable effect rather than using a timeout; modify only the setup synchronization.packages/solid-db/src/useLiveQuery.ts (1)
412-439: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winThree adapters re-implement the same observer-delta materialization. Each granular adapter applies
insert/update/deleteto a keyed map, then rebuilds that map fromobserver.getSnapshot().statewhenchangesisundefined. Only the map type and the batching wrapper differ. A future change to the status-only rebuild rule must then be made three times, and a missed site silently diverges the keyed view from the ordered data.Extract one helper into
@tanstack/db, for exampleapplyObserverChanges(map, changes, snapshotState), and call it from each adapter inside that adapter's own batching primitive.packages/solid-db/src/useLiveQuery.ts#L412-L439: call the shared helper inside the existingbatch(...), keeping theReactiveMapas the target.packages/svelte-db/src/useLiveQuery.svelte.ts#L424-L450: call the shared helper inside the existinguntrack(...), keeping theSvelteMapas the target.packages/vue-db/src/useLiveQuery.ts#L402-L426: call the shared helper directly, keeping thereactive(new Map())as the target.
As per coding guidelines: "Extract common logic into utility functions when identical or near-identical code blocks appear in multiple places".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/solid-db/src/useLiveQuery.ts` around lines 412 - 439, Extract the shared observer-delta materialization into an `@tanstack/db` helper such as applyObserverChanges(map, changes, snapshotState), including insert/update/delete handling and rebuilding from snapshotState when changes is undefined. In packages/solid-db/src/useLiveQuery.ts#L412-L439, call it inside batch(...) with the ReactiveMap; in packages/svelte-db/src/useLiveQuery.svelte.ts#L424-L450, call it inside untrack(...) with the SvelteMap; and in packages/vue-db/src/useLiveQuery.ts#L402-L426, call it directly with the reactive Map. Keep each adapter’s existing batching behavior and subsequent synchronization/status updates unchanged.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. Inline comments: In @.changeset/live-query-observer.md: - Around line 12-16: Update the changeset description around createLiveQueryObserver and the lifecycle summary to accurately reflect that useLiveQuery starts sync via startSyncImmediate before observer creation for directly supplied and callback-returned collections. Either document this direct-collection exception explicitly, or remove that pre-activation in useLiveQuery so the observer owns activation on its first committed subscription consistently. --- Nitpick comments: In `@packages/angular-db/src/index.ts`: - Around line 261-266: The Angular sync path currently reads collection state directly, unlike other adapters that use the observer snapshot. Update syncDataFromCollection and its call sites in the subscription flow to consume observer.getSnapshot() for status, and reuse snapshot state/data where applicable, preserving the post-subscribe synchronization behavior. In `@packages/db/src/collection/sync.ts`: - Around line 74-88: Update markLayoutChange() to obtain the transaction through the existing private getActivePendingSyncTransaction() helper instead of duplicating the pending transaction lookup and committed-state guards, then set layoutChanged on the returned transaction while preserving the helper’s existing error behavior. In `@packages/db/src/live-query-observer.ts`: - Around line 495-500: Update the publication delivery logic around the listener invocation in the observer’s emit flow to isolate each subRecord.listener call, allowing all targets and queued publications to be processed even when a listener throws. Capture the first thrown error, continue draining the inner and outer publication queues, then rethrow after queue processing completes while preserving the disposed checks and existing delivery behavior. In `@packages/db/src/query/live/collection-config-builder.ts`: - Around line 905-912: The multiplicity accumulation logic is duplicated across three sites; extract a shared generic applyMultiplicity<T> helper that handles deletes, previousValue/previousOrderByIndex on retracts, inserts, value replacement, and defined orderByIndex updates. In packages/db/src/query/live/collection-config-builder.ts at lines 905-912 and 1337-1344, replace the inline branches with applyMultiplicity(existing, childResult, _orderByIndex, multiplicity); at lines 2340-2343, replace the retract/insert branches with applyMultiplicity(changes, value, orderByIndex, multiplicity), preserving all existing behavior. In `@packages/db/tests/conformance/suite.ts`: - Line 42: Update or remove the stale “universal expected-fail” section heading near the order-only-move scenario, since UNIVERSAL_EXPECTED_FAIL is empty and order-only-move is a normal passing scenario. Ensure the surrounding comment accurately describes the scenario’s current status. In `@packages/db/tests/live-query-observer.test.ts`: - Around line 593-617: Add a test near the existing layoutRevision observer cases that creates an empty source, subscribes a live query observer, records its initial snapshot, performs a begin/commit cycle without writes, and asserts the snapshot data remains [] while layoutRevision is unchanged; dispose the observer afterward. In `@packages/solid-db/src/useLiveQuery.ts`: - Around line 412-439: Extract the shared observer-delta materialization into an `@tanstack/db` helper such as applyObserverChanges(map, changes, snapshotState), including insert/update/delete handling and rebuilding from snapshotState when changes is undefined. In packages/solid-db/src/useLiveQuery.ts#L412-L439, call it inside batch(...) with the ReactiveMap; in packages/svelte-db/src/useLiveQuery.svelte.ts#L424-L450, call it inside untrack(...) with the SvelteMap; and in packages/vue-db/src/useLiveQuery.ts#L402-L426, call it directly with the reactive Map. Keep each adapter’s existing batching behavior and subsequent synchronization/status updates unchanged. In `@packages/solid-db/tests/useLiveQuery.test.tsx`: - Line 586: Replace the fixed setup sleeps at the identified waits in useLiveQuery tests with waitFor-based conditions that observe the collection state or effect completion. Preserve the synchronous assertions after setMinAge and setCurrent unchanged, and for the final wait await the observable effect rather than using a timeout; modify only the setup synchronization.🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID:
7e53d52d-57eb-461b-a2b5-05314344178b📥 Commits
Reviewing files that changed from the base of the PR and between ad88d0751db9723dfb9f164ebfcef88d52b6efa3 and 619c3a37f1843df0094008351c1542db164a114a.
📒 Files selected for processing (34)
.changeset/live-query-observer.md.changeset/live-query-order-only-move.mdpackages/angular-db/src/index.tspackages/angular-db/tests/conformance.test.tspackages/angular-db/tests/inject-live-query.test.tspackages/db/src/collection/change-events.tspackages/db/src/collection/changes.tspackages/db/src/collection/index.tspackages/db/src/collection/lifecycle.tspackages/db/src/collection/state.tspackages/db/src/collection/subscription.tspackages/db/src/collection/sync.tspackages/db/src/errors.tspackages/db/src/index.tspackages/db/src/live-query-observer.tspackages/db/src/query/live/collection-config-builder.tspackages/db/src/query/live/types.tspackages/db/tests/conformance/contract.tspackages/db/tests/conformance/suite.tspackages/db/tests/live-query-observer.test.tspackages/db/tests/live-query-order-only-move.test.tspackages/react-db/src/useLiveQuery.tspackages/react-db/tests/conformance.test.tsxpackages/react-db/tests/useLiveQuery.eager-onstorechange.test.tsxpackages/react-db/tests/useLiveQuery.strictmode.test.tsxpackages/solid-db/src/useLiveQuery.tspackages/solid-db/tests/conformance.test.tsxpackages/solid-db/tests/useLiveQuery.test.tsxpackages/svelte-db/src/useLiveQuery.svelte.tspackages/svelte-db/tests/conformance.svelte.test.tspackages/svelte-db/tests/useLiveQuery.svelte.test.tspackages/vue-db/src/useLiveQuery.tspackages/vue-db/tests/conformance.test.tspackages/vue-db/tests/useLiveQuery.test.ts
@@ -0,0 +1,16 @@1+---2+'@tanstack/db': minor3+'@tanstack/react-db': patch4+'@tanstack/vue-db': patch5+'@tanstack/svelte-db': patch6+'@tanstack/solid-db': patch7+'@tanstack/angular-db': patch8+---9+10+Add an internal shared live-query observer and migrate all five framework adapters to it11+12+Introduces `createLiveQueryObserver` in `@tanstack/db`: given a resolved live-query collection (or `null` for a disabled query) it owns the lifecycle every adapter used to re-implement — sync activation on first subscribe, change and status subscriptions, a snapshot with stable identity per state revision for wholesale consumers, and delivery of the raw `ChangeMessage[]` for granular consumers. React, Vue, Svelte, Solid, and Angular's live-query hooks now materialize from the observer instead of their own hand-rolled subscription/status/snapshot machinery, keeping each adapter's native reactivity and each adapter's data-loading policy (wholesale adapters subscribe without initial state; granular adapters seed from it).13+14+The observer is an **internal, unstable contract** for TanStack DB's official adapters — it is exported so the adapter packages can consume it, but it is not a public extension point yet and its API may change in any release.15+16+The migration also fixes several live-query lifecycle defects: status-only transitions (`error`, `cleaned-up`) now reach mounted consumers; snapshot identity is stable across unsubscribe/resubscribe and stays fresh while detached; dispatch is FIFO and non-reentrant with subscriptions identified by record rather than callback; disposing during the synchronous initial replay no longer leaks the collection subscription; subscribing after dispose throws instead of registering a dead listener; Solid guards its async resource continuations against superseded collections; and constructing an observer no longer activates sync (activation belongs to the first committed subscription).📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Qualify the sync-activation ownership claim.
Line 12 says
createLiveQueryObserverowns sync activation on first subscribe. However,packages/react-db/src/useLiveQuery.ts:317-448still callsstartSyncImmediate()before creating the observer for directly supplied and callback-returned collections. This also conflicts with Line 16, which says activation belongs to the first committed subscription. Document the direct-collection exception, or delegate this activation to the observer consistently.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.changeset/live-query-observer.md around lines 12 - 16, Update the changeset description around createLiveQueryObserver and the lifecycle summary to accurately reflect that useLiveQuery starts sync via startSyncImmediate before observer creation for directly supplied and callback-returned collections. Either document this direct-collection exception explicitly, or remove that pre-activation in useLiveQuery so the observer owns activation on its first committed subscription consistently.✅ Addressed in commits a9e13d5 to c27d572
KyleAMathews(GitHub profile, opens in a new tab) approved these changes
Verified the transaction-scoped layout revision fix on 619c3a37. The parked detached-observer regression now passes, the layout revision remains unchanged until the sync transaction is applied, and the broader collection/order-only suites pass (149 tests, no type errors). The implementation also removes the collection-wide pending marker, so unrelated publications cannot consume it. Ready from my review.
🧹 Nitpick comments (1)
packages/db/tests/query/includes-oracle.property.test.ts (1)
129-154: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winThe generated histories cannot produce an order-only move.
normalizePutpins each row position at the firstputfor that key:positions.get(action.id) ?? action.position. Every laterputfor the same key reuses the stored position. Therefore the fuzz property at Line 984 can changevalueandgroup, but neverposition. Order-only moves are only covered by the hardcodedconfirmedChildReorderSeedand the#1444seed, which both bypassensureActionsTargetRows.This PR targets order-only moves, so the property test provides no random coverage of them. Consider letting a repeat
putchange the position while keeping the correlation keys stable.♻️ Proposed change to allow position moves in generated histories
const normalizePut = (): HistoryAction => { keys.add(action.id) - const position = positions.get(action.id) ?? action.position - positions.set(action.id, position) - return { ...action, type: `put`, position } + positions.set(action.id, action.position) + return { ...action, type: `put`, position: action.position } }Note:
positionsthen only tracks the latest position for non-putactions, which already readcurrent?.positioninapplyAction.Also applies to: 1201-1207
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/db/tests/query/includes-oracle.property.test.ts` around lines 129 - 154, Update normalizePut in the history generator so repeated put actions for an existing key can use a newly generated action.position, while preserving the key’s correlation identity and initial-position behavior as appropriate. Adjust positionsByLevel tracking so it stores the latest position needed by subsequent non-put actions, allowing generated histories to include order-only moves and keeping the corresponding logic around the other normalizePut occurrence consistent.🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. Nitpick comments: In `@packages/db/tests/query/includes-oracle.property.test.ts`: - Around line 129-154: Update normalizePut in the history generator so repeated put actions for an existing key can use a newly generated action.position, while preserving the key’s correlation identity and initial-position behavior as appropriate. Adjust positionsByLevel tracking so it stores the latest position needed by subsequent non-put actions, allowing generated histories to include order-only moves and keeping the corresponding logic around the other normalizePut occurrence consistent.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID:
9f2d4cf4-06ad-4ddb-9471-82cc04abae01📥 Commits
Reviewing files that changed from the base of the PR and between 619c3a37f1843df0094008351c1542db164a114a and c27d57213d7661964b5556f8acf1df17394bf610.
📒 Files selected for processing (3)
.changeset/live-query-observer.mdpackages/db/src/collection/sync.tspackages/db/tests/query/includes-oracle.property.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- .changeset/live-query-observer.md
- packages/db/src/collection/sync.ts
KyleAMathews(GitHub profile, opens in a new tab) approved these changes
Re-reviewed current head c27d5721 after the main merge. The follow-up cleanly promotes the two includes-oracle cases to positive regressions, reuses the existing active-sync-transaction guard, and corrects the observer changeset wording. The equivalent merged tree passes the full @tanstack/db suite (2,534 tests, 5 skipped, no type errors). Ready.
@KyleAMathews Thanks for re-reviewing. #1669 is green and mergeable now—please merge when ready: https://github.com/TanStack/db/pull/1669
#1675 is clean and stacked behind it.
KyleAMathews(GitHub profile, opens in a new tab) closed this
Sign in to join this conversation.
📝 Walkthrough
Walkthrough#
Live-query collections now track state and layout revisions, publish order-only changes, and expose layout-aware notifications. A framework-agnostic observer provides stable snapshots, subscription modes, lazy activation, queued delivery, preload, and disposal. Tests cover observer lifecycle, nested reordering, and stale-key removal.
Changes#
Live-query layout event propagation
packages/db/src/collection/*,packages/db/src/query/live/*Shared live-query observer
packages/db/src/live-query-observer.tspackages/db/tests/live-query-observer.test.tsOrder-move regression and release validation
packages/db/tests/live-query-order-only-move.test.ts,packages/db/tests/query/includes-oracle.property.test.tspackages/db/tests/conformance/suite.ts,.changeset/*Estimated code review effort: 4 (Complex) | ~60 minutes
Possibly related issues
Possibly related PRs
Suggested reviewers:
kyleamathews🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)#
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
❤️ Share