fix(solid-db): clone rows in syncDataFromCollection so reconcile sees in-place mutations #1604
main fromrefs/pull/1604/head·opened Fixes part of #1571.
What#
useLiveQuery's rendered data array missed updates whenever the underlying collection mutated a row in place. The upstream pipeline does exactly that — flushIncludesState in packages/db/src/query/live/collection-config-builder.ts writes new toArray include values onto the parent row via setIncludedValue(parentResult, …) without changing its identity. When syncDataFromCollection then called
reconcile(Array.from(currentCollection.values()))(prev).filter(Boolean)
Solid's reconcile short-circuited on next === target at the parent-row level (and again on the include array reference if reached). The reactive state ReactiveMap still updated correctly — subscribeChanges hands state.set a fresh row reference per change event — but the data array fell out of sync.
The reporter @Tarmean documented this for toArray includes specifically, with flushIncludesState cloning rows for the events but mutating the rows the collection itself returns from .values().
How#
syncDataFromCollection now shallow-clones each row (and any array-valued field on it) before handing the snapshot to reconcile. The wrapper is small and lives next to the sync helper so the rationale is local:
const cloneRowForReconcile = <T,>(v: T): T => {
if (v === null || typeof v !== 'object' || Array.isArray(v)) return v
const source = v as Record<string, unknown>
const out: Record<string, unknown> = {}
for (const key in source) {
const val = source[key]
out[key] = Array.isArray(val) ? [...val] : val
}
return out as T
}
Field-level diffing then actually runs and toArray include arrays propagate. Non-object values (primitives) pass through; arrays at the top level pass through (reconcile already iterates them); object values get a one-level copy with arrays inside them also copied so the include-array reference changes.
The state ReactiveMap path is untouched — subscribeChanges was already getting fresh row references per change event, so only the Array.from(currentCollection.values()) sync needed help.
Tests#
Adds toArray include reactivity (#1571) describe block in packages/solid-db/tests/useLiveQuery.test.tsx:
- Set up two collections (
projects,issues) and auseLiveQuerythat projects{ id, name, issueTitles: toArray(...) }. - Insert a new child issue tied to project Alpha.
- Assert the parent row's
issueTitlesarray grows from['Bug in Alpha']to['Bug in Alpha', 'Feature for Alpha']in the rendereddataarray (not just in thestatemap). - Asserts Beta's include is untouched.
Without the clone, the rendered data array stays at the original include content; the test would fail on expect(alphaAfter.issueTitles.length).toBe(2). All 39 tests in packages/solid-db/tests/useLiveQuery.test.tsx pass. Lint clean (per AGENTS.md).
Scope#
This addresses the clone-in-reconcile half of #1571. @Tarmean's report flagged a second, related case:
If an include is initially empty, the proxy is not attached to the parent. It remains null. This is why the initial child field can be null, breaking reactivity for collection includes as well.
That one lives at packages/db/src/query/live/collection-config-builder.ts:1851-1867 and is a different code path (proxy attachment, not consumer-side reconcile). Happy to file it as a separate issue and take a look if maintainers agree it's the same bug class.
Notes for maintainers#
- The clone-on-sync trades a small allocation per row per sync for correctness. For typical UIs (hundreds of rows) this is far cheaper than the missed-update bug. If the perf trade-off is unwelcome, the alternative is to fix this upstream in
flushIncludesStateso include writes produce new parent row identities — happy to switch directions on guidance. - I considered using
state.values()instead ofcurrentCollection.values()(sincestatealready holds fresh references persubscribeChangesevent), butstate's insertion order doesn't match the collection's sort order on inserts, so thedataarray would drift. Cloning preserved the sort order fromcurrentCollection.values().
Changeset#
@tanstack/solid-db: patch
Summary by CodeRabbit#
Bug Fixes#
- Fixed live queries not updating correctly when included related data changed (such as after inserting a new child entry). Included fields now refresh reliably so the displayed arrays reflect the most recent matching records.
- Added regression coverage to ensure
toArray-based includes update as expected, including cases where an initially empty include expands after the first matching child insert.
- 1
🧹 Nitpick comments (1)
packages/solid-db/tests/useLiveQuery.test.tsx (1)
2480-2546: ⚡ Quick winAdd an empty-include regression variant for
#1571.This test validates the non-empty path, but it should also cover a parent that starts with
issueTitles = []and then receives its first child insert. That edge case is the most likely to regress for include reactivity.💡 Suggested test extension
+ it(`should update when a toArray include starts empty and gets first child`, async () => { + const projects = createCollection( + mockSyncCollectionOptions<IncludeProject>({ + id: `toarray-include-projects-empty`, + getKey: (p) => p.id, + initialData: [{ id: 3, name: `Gamma` }], + }), + ) + const issues = createCollection( + mockSyncCollectionOptions<IncludeIssue>({ + id: `toarray-include-issues-empty`, + getKey: (i) => i.id, + initialData: [], + }), + ) + + const rendered = renderHook(() => + useLiveQuery((q) => + q.from({ p: projects }).select(({ p }) => ({ + id: p.id, + issueTitles: toArray( + q + .from({ i: issues }) + .where(({ i }) => eq(i.projectId, p.id)) + .select(({ i }) => i.title), + ), + })), + ), + ) + + await waitFor(() => { + const gamma = rendered.result().find((r: any) => r.id === 3) + expect(gamma.issueTitles).toEqual([]) + }) + + issues.utils.begin() + issues.utils.write({ + type: `insert`, + value: { id: 30, projectId: 3, title: `First Gamma issue` }, + }) + issues.utils.commit() + + await waitFor(() => { + const gamma = rendered.result().find((r: any) => r.id === 3) + expect(gamma.issueTitles).toEqual([`First Gamma issue`]) + }) + })As per coding guidelines, “Test corner cases including: empty arrays/sets, single-element collections, undefined vs null values, resolved promises, async race conditions, and limit/offset edge cases.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/solid-db/tests/useLiveQuery.test.tsx` around lines 2480 - 2546, Add a regression test variant to cover the edge case where a parent row has an empty issueTitles array and then receives its first child insert, which is the most likely scenario to regress for include reactivity. Create a new test case or extend the existing one to verify that when a child issue is inserted for a project that currently has no issues (empty issueTitles array), the parent row's toArray include field properly reflects the new child item. This ensures the clone-on-sync fix handles the transition from an empty array to a populated array correctly.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. Nitpick comments: In `@packages/solid-db/tests/useLiveQuery.test.tsx`: - Around line 2480-2546: Add a regression test variant to cover the edge case where a parent row has an empty issueTitles array and then receives its first child insert, which is the most likely scenario to regress for include reactivity. Create a new test case or extend the existing one to verify that when a child issue is inserted for a project that currently has no issues (empty issueTitles array), the parent row's toArray include field properly reflects the new child item. This ensures the clone-on-sync fix handles the transition from an empty array to a populated array correctly.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID:
7c172331-41b7-431f-951a-373b5b3b3bf4📥 Commits
Reviewing files that changed from the base of the PR and between 9a5e8e5445435c59ede20d3080d09b95355a67dd and c6b2860561c21c5e3edab6ea194e65dcfd8c2794.
📒 Files selected for processing (3)
.changeset/solid-db-reconcile-clone-rows.mdpackages/solid-db/src/useLiveQuery.tspackages/solid-db/tests/useLiveQuery.test.tsx
Tip
For best results, initiate chat on the files or code changes.
@tsushanth: Great — both regression paths are now locked in. Theempty-then-first-childvariant is particularly important since an empty array is the most common initial state for a freshtoArrayinclude. Having both cases in the test suite makes it much harder for the shallow-clone logic to regress silently. Looks good!🐇 ✨
If you found this review helpful, would you consider giving us a shout-out on X?
Thank you for using CodeRabbit!
I think this change maybe should wait until this PR was merged? https://github.com/TanStack/db/pull/1598
I'm not super deep in solidjs or tanstack-db, but my understanding of reconcile is as follows:
- It matches (pointer- or id-) equal common prefixes and suffixes
- Then, if we have an id key, match the rest by the id key. It defaults to 'id' and checks if the first new row has a non-null property with that name
- Otherwise, if we pass merge=true to reconcile (we don't), match the rest by index
- Otherwise, don't match
Problem: this change would intentionally remove all pointer equality. So if the row doesn't happen to have an 'id' field, we lose all incrementalization and solid would fully tear down+recreate For blocks.
Behavior
reconcile is for reconciling an incoming snapshot against existing store state. With the default key, array items are matched by "id" where possible and otherwise fall back to positional or reference matching. Non-wrappable values are replaced directly. When merge is false, reconciliation prefers replacing non-matching branches. When merge is true, it pushes updates deeper into the structure.
https://docs.solidjs.com/reference/store-utilities/reconcile
Either way, thank you for your work on this!
Thanks for the careful read @Tarmean — you're right that the clone gives up row-level pointer equality, and with the current default reconcile key (
'id') that's a real downside for tables whose rows don't have anidfield.The reason I went with the clone is that
reconcile'snext === targetshort-circuit fires at the row level too: if the include flush mutatesparent.issueTitlesin place without changing the parent reference, the field never reaches subscribers no matter what reconcile does deeper. So fixing this in solid-db alone seems to force some identity break per row.#1598's
key: '$key'does address my concern about the clone (matching survives the new reference), so if it lands first this PR becomes mostly compatible. If you'd rather not have a window where both PRs disagree on key strategy, I'm happy to either (a) park this one until #1598 merges and rebase on top, or (b) instead chase the root cause influshIncludesStateso the collection produces a new row reference whenever an include changes — letting reconcile bail at the parent untouched. Let me know which fits the roadmap better.Thanks @Tarmean — your reading of reconcile's matching is right, and it's worth being precise about whether #1604 stands on its own, gates on #1598, or should be rethought.
The question this turns on: does #1598 (
key: '$key') alone fix the toArray-include problem in #1571?I don't think it does, and that's the case for keeping both. The reproduction in #1571 is:
- The live collection holds row
R = {..., issueTitles: [...]}. flushIncludesState(inpackages/db/src/query/live/collection-config-builder.ts) callssetIncludedValue(storedParent, resultPath, childValue)— it mutates the stored parent row's include field in place (storedParent !== parentResultshort-circuit at line 1622 only skips redundant work; the parent is mutated by the precedingsetIncludedValueonparentResult, and the stored row is mutated here).syncDataFromCollectionreadscollection.values()and gets back the sameRreference it returned last time.reconcile([R])(prev)sees prefix-matchnext[0] === prev[0](same object), and the recursivereconcile(R, R)short-circuits on reference equality before thekeyoption is ever consulted.
#1598 fixes the matching step (which row is "old R" and which is "new R" when references diverge), which matters for the cases #1524 surfaces. But the in-place-mutation case never reaches matching: reconcile bails at the recursive same-reference check first. So
key: '$key'and shallow-clone-before-reconcile are independent failure modes.That's why #1604 only forces a fresh shallow copy of each row and any array-valued field. After that, reconcile's normal logic — pointer equality on still-deep-equal scalars, key match on the rest — runs as advertised.
You're right that breaking row-level reference equality has a real cost. Two readings:
- After #1598 lands: matching falls through to
$key(always present on tanstack-db rows), so we still find the right rows even with cloned references. The extra work is field-level diff inside matched rows; for{name, id, scalar, scalar}-shaped rows that's bounded. - Without #1598: no-id user data falls back to positional + reference matching, and shallow-clone breaks reference matching. Stable-order lists still match positionally; reorders would tear down. That's the regression you're describing.
A few paths:
- Wait for #1598, rebase #1604 on top of it. With
$keymatching in place the cost is bounded and predictable, and the two fixes compose. - Fix at the source: stop mutating stored parent rows in place in
flushIncludesState. Heavier change, and you'd know better than me whether downstream consumers rely on stable refs from the collection map. - Tighten the shallow-clone to only clone fields that
flushIncludesStatemutates (the include values), not the whole row. That preserves row-level pointer equality for the prefix/suffix match path, at the cost of some coupling between solid-db andcollection-config-builder's internals.
My lean is (1), with (3) as a follow-up if field-level diff turns out to be hot. Happy to convert this PR to draft until #1598 merges, then rebase and confirm the test still demonstrates the fix.
One other note: the second sub-issue in #1571 (empty initial include → null proxy, never reactive) isn't addressed by either PR. That looks like it needs the
useLiveQueryproxy plumbing to widennull → arrayto be reactive, which is independent of both fixes. Happy to take that on as a separate PR once we've sorted this one.- The live collection holds row
Closing — this has been open without activity long enough that it's more useful to withdraw and let a fresh PR reflect current
main/masterthan to keep pinging for review on stale state. Happy to reopen or resubmit if there's still interest.tsushanth(GitHub profile, opens in a new tab) closed this
Sign in to join this conversation.
No actionable comments were generated in the recent review. 🎉
ℹ️ Recent review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID:
7121fb4b-6058-4c8b-a8d9-97154ac58e2d📥 Commits
Reviewing files that changed from the base of the PR and between c6b2860561c21c5e3edab6ea194e65dcfd8c2794 and 95d2d407022879587307185c6bea6cb60015238a.
📒 Files selected for processing (1)
packages/solid-db/tests/useLiveQuery.test.tsx🚧 Files skipped from review as they are similar to previous changes (1)
📝 Walkthrough
Walkthrough#
useLiveQueryin@tanstack/solid-dbis patched to shallow-clone each collection row (including array-valued fields) before passing data to Solid'sreconcile. This prevents reconcile from short-circuiting on stable in-place-mutated parent references, sotoArrayinclude field updates now propagate to the UI. Regression tests validate the fix and a patch changeset is recorded.Changes#
toArray include reactivity fix
cloneRowForReconcilehelper andsyncDataFromCollectionupdatepackages/solid-db/src/useLiveQuery.tscloneRowForReconcileto shallow-clone rows and their array-valued fields;syncDataFromCollectionnow passes cloned rows toreconcileinstead of raw collection values, ensuring field-level diffing runs when the upstream collection mutates rows in place.toArrayinclude reactivity (#1571)packages/solid-db/tests/useLiveQuery.test.tsxtoArrayand adds a newdescribeblock with parent/child collections and auseLiveQueryusingtoArraysubqueries; two tests verify that inserting a child updates only the affected parent's included array and that inserting the first child widens an initially empty included array..changeset/solid-db-reconcile-clone-rows.md@tanstack/solid-dbdescribing the row-cloning fix tosyncDataFromCollection.Estimated code review effort#
🎯 2 (Simple) | ⏱️ ~12 minutes
Possibly related issues#
toArrayinclude changes propagate correctly through Solid's reactivity system.Suggested reviewers#
Poem#
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
✨ Finishing Touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
❤️ Share