fix(db): deduplicate loadedSubsets and join key requests #1554
main fromrefs/pull/1554/head·opened 🎯 Changes#
Supersedes #1205.
Two related fixes for unbounded loadedSubsets growth:
subscription.ts:
requestSnapshotandloadNextItemsnow checkisPredicateSubsetbefore pushing toloadedSubsets, skipping entries already covered by an existing superset predicate.joins.ts: A
loadedJoinKeysSet tracks keys already loaded across pipeline batches. Subsequent batches filter out known keys before callingrequestSnapshot, preventing repeated requests for the same join keys.
Without these fixes, join lazy-loading accumulates duplicate LoadSubsetOptions entries, causing SQL queries and query keys to grow unboundedly — eventually hitting "too many SQL variables" errors.
✅ Checklist#
- I have tested this code locally with
pnpm test:pr. - I have followed the steps in the Contributing guide.
🚀 Release Impact#
- This change affects published code, and I have generated a changeset.
- This change is docs/CI/dev-only (no release).
Summary by CodeRabbit#
Bug Fixes
- Prevented redundant subset and join-key load requests during subscriptions and lazy joins, stopping unnecessary growth of tracked loads and avoiding repeated snapshot operations. Loading status still reflects concurrent on-demand loads until all complete.
Tests
- Added and updated tests validating deduplication, preload behavior, and concurrent-load handling in live query join scenarios.
- 1
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/db/src/collection/subscription.ts (1)
380-391: ⚠️ Potential issue | 🟠 Major | ⚡ Quick winDedup bookkeeping happens after issuing
loadSubset, which can break unload symmetry.loadSubset(loadOptions)is executed before dedup checks, so duplicate/subset requests can still be issued but not recorded inloadedSubsets. On unsubscribe, those untracked loads won’t get matching unload calls.Suggested fix
- const syncResult = this.collection._sync.loadSubset(loadOptions) - - // Pass the raw loadSubset result to the caller for external tracking - opts?.onLoadSubsetResult?.(syncResult) - - // Track this loadSubset call so we can unload it later const isAlreadyCovered = this.loadedSubsets.some((existing) => isPredicateSubset(loadOptions, existing), ) - if (!isAlreadyCovered) { + if (!isAlreadyCovered) { + const syncResult = this.collection._sync.loadSubset(loadOptions) + opts?.onLoadSubsetResult?.(syncResult) this.loadedSubsets.push(loadOptions) + if (trackLoadSubsetPromise) { + this.trackLoadSubsetPromise(syncResult) + } + } else { + // Optional: report no-op load to caller if needed by caller contract + opts?.onLoadSubsetResult?.(true) } - - const trackLoadSubsetPromise = opts?.trackLoadSubsetPromise ?? true - if (trackLoadSubsetPromise) { - this.trackLoadSubsetPromise(syncResult) - }Apply equivalent ordering in
requestLimitedSnapshotas well.Also applies to: 623-634
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/db/src/collection/subscription.ts` around lines 380 - 391, Reorder the dedup bookkeeping so you check isAlreadyCovered via isPredicateSubset and update loadedSubsets before calling this.collection._sync.loadSubset — only call loadSubset when not already covered — and still invoke opts?.onLoadSubsetResult with the actual load result when you do call it (or a noop/early-return when skipped); apply the same reordering/logic in requestLimitedSnapshot as well so unload symmetry is preserved for both code paths.🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. Inline comments: In `@packages/db/src/query/compiler/joins.ts`: - Around line 321-325: The code is adding joinKeys into loadedJoinKeys before confirming the async loadSubset succeeded, which can prevent retries when loadSubset later rejects; in the requestSnapshot / join handling logic (referencing requestSnapshot, loadSubset, loadedJoinKeys, and joinKeys), move the loadedJoinKeys.add(key) updates so they only run after the async loadSubset/requestSnapshot completes successfully (i.e., inside the success path or then block), and ensure failed loads do not mark keys as loaded so subsequent batches can retry. In `@packages/db/tests/query/loaded-subsets-dedup.test.ts`: - Around line 164-176: The test currently may pass vacuously if no 'in' filters are produced; update the loop that iterates newCalls and inspects extractSimpleComparisons(call.where) to assert that at least one filter with operator === 'in' was seen. For example, introduce a counter or collect matching filters (referencing newCalls, extractSimpleComparisons, filters and filter.operator) and after the loop add an assertion that the count/collection length is > 0 before relying on contains/not.toContain checks. - Around line 94-112: The test never triggers the second snapshot request; after calling await liveQuery.preload() capture initialCallCount as you do, then invoke and await the second snapshot action (e.g., await liveQuery.requestSnapshot() or the appropriate method that re-requests subsets) to actually exercise dedup logic, then assert loadSubsetCalls.length remains equal to initialCallCount; reference liveQuery.preload(), liveQuery.requestSnapshot(), and loadSubsetCalls in the change. - Line 125: Replace the unsafe cast by correctly typing parentWrite: change its type from the ad-hoc (msg: { type: string; value: Parent }) => void to ChangeMessageOrDeleteKeyMessage<Parent, number> (import ChangeMessageOrDeleteKeyMessage from ../../src/types.js) and then assign parentWrite = write (no "as any"); ensure the generic keyof/number matches the test's Parent and TKey usage so the assignment type-checks. --- Outside diff comments: In `@packages/db/src/collection/subscription.ts`: - Around line 380-391: Reorder the dedup bookkeeping so you check isAlreadyCovered via isPredicateSubset and update loadedSubsets before calling this.collection._sync.loadSubset — only call loadSubset when not already covered — and still invoke opts?.onLoadSubsetResult with the actual load result when you do call it (or a noop/early-return when skipped); apply the same reordering/logic in requestLimitedSnapshot as well so unload symmetry is preserved for both code paths.🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID:
5754dc72-3946-45bc-b96e-1abdc624f6a8📥 Commits
Reviewing files that changed from the base of the PR and between be656be5c5fa65cc1c2bf76bc0b60be3d9d2a577 and f8828b345290e98b84b08abb7fed4176fb3f32a8.
📒 Files selected for processing (4)
.changeset/fix-loaded-subsets-dedup.mdpackages/db/src/collection/subscription.tspackages/db/src/query/compiler/joins.tspackages/db/tests/query/loaded-subsets-dedup.test.ts
packages/db/src/query/compiler/joins.ts lines 321 to 325@@ -314,4 +318,8 @@314318 optimizedOnly: true,315319 })316320317- if (!loaded) {321+ if (loaded) {322+ for (const key of joinKeys) {323+ loadedJoinKeys.add(key)324+ }325+ } else {⚠️ Potential issue | 🟠 Major | ⚡ Quick win
loadedJoinKeysis updated before async load success, which can suppress needed retries.requestSnapshotcan returntrueeven if the underlying asyncloadSubsetlater rejects. Adding keys immediately means failed keys won’t be retried in later batches.Suggested fix
- const loaded = lazySourceSubscription.requestSnapshot({ + let loadResult: Promise<void> | true = true + const loaded = lazySourceSubscription.requestSnapshot({ where: inArray(lazyJoinRef, joinKeys), optimizedOnly: true, + onLoadSubsetResult: (result) => { + loadResult = result + }, }) if (loaded) { - for (const key of joinKeys) { - loadedJoinKeys.add(key) - } + if (loadResult instanceof Promise) { + loadResult + .then(() => { + for (const key of joinKeys) loadedJoinKeys.add(key) + }) + .catch(() => { + // keep keys retriable + }) + } else { + for (const key of joinKeys) loadedJoinKeys.add(key) + } } else {🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/db/src/query/compiler/joins.ts` around lines 321 - 325, The code is adding joinKeys into loadedJoinKeys before confirming the async loadSubset succeeded, which can prevent retries when loadSubset later rejects; in the requestSnapshot / join handling logic (referencing requestSnapshot, loadSubset, loadedJoinKeys, and joinKeys), move the loadedJoinKeys.add(key) updates so they only run after the async loadSubset/requestSnapshot completes successfully (i.e., inside the success path or then block), and ensure failed loads do not mark keys as loaded so subsequent batches can retry.@@ -0,0 +92,21 @@92+ )93+94+ await liveQuery.preload()95+96+ const initialCallCount = loadSubsetCalls.length97+ expect(initialCallCount).toBeGreaterThan(0)98+99+ const firstCall = loadSubsetCalls[0]!100+ expect(firstCall.where).toBeDefined()101+102+ const filters = extractSimpleComparisons(firstCall.where)103+ expect(filters).toEqual([104+ {105+ field: [`parentId`],106+ operator: `in`,107+ value: expect.arrayContaining([1, 2, 3]),108+ },109+ ])110+111+ expect(loadSubsetCalls.length).toBe(initialCallCount)112+ })⚠️ Potential issue | 🟠 Major | ⚡ Quick win
The first regression test doesn’t actually trigger a second snapshot request.
The test name says repeated
requestSnapshot, but there is no second action afterpreload(). As written,expect(loadSubsetCalls.length).toBe(initialCallCount)is trivially true.Suggested fix
- const initialCallCount = loadSubsetCalls.length + const initialCallCount = loadSubsetCalls.length expect(initialCallCount).toBeGreaterThan(0) @@ - expect(loadSubsetCalls.length).toBe(initialCallCount) + // Trigger another batch with only already-seen join keys + // (e.g. re-emit parent rows or perform a no-op update that reflows the active side) + await liveQuery.preload() + await flushPromises() + + expect(loadSubsetCalls.length).toBe(initialCallCount)As per coding guidelines, "Always add unit tests that reproduce a bug before fixing it to ensure the bug is fixed and prevent regression".
📝 Committable suggestion
‼️ IMPORTANT Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
await liveQuery.preload() const initialCallCount = loadSubsetCalls.length expect(initialCallCount).toBeGreaterThan(0) const firstCall = loadSubsetCalls[0]! expect(firstCall.where).toBeDefined() const filters = extractSimpleComparisons(firstCall.where) expect(filters).toEqual([ { field: [`parentId`], operator: `in`, value: expect.arrayContaining([1, 2, 3]), }, ]) // Trigger another batch with only already-seen join keys // (e.g. re-emit parent rows or perform a no-op update that reflows the active side) await liveQuery.preload() await flushPromises() expect(loadSubsetCalls.length).toBe(initialCallCount) })🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/db/tests/query/loaded-subsets-dedup.test.ts` around lines 94 - 112, The test never triggers the second snapshot request; after calling await liveQuery.preload() capture initialCallCount as you do, then invoke and await the second snapshot action (e.g., await liveQuery.requestSnapshot() or the appropriate method that re-requests subsets) to actually exercise dedup logic, then assert loadSubsetCalls.length remains equal to initialCallCount; reference liveQuery.preload(), liveQuery.requestSnapshot(), and loadSubsetCalls in the change.✅ Addressed in commit 96350c1
@@ -0,0 +105,21 @@105+ field: [`parentId`],106+ operator: `in`,107+ value: expect.arrayContaining([1, 2, 3]),108+ },109+ ])110+111+ expect(loadSubsetCalls.length).toBe(initialCallCount)112+ })113+114+ it(`should deduplicate join key requests across pipeline batches`, async () => {115+ let parentBegin: () => void116+ let parentWrite: (msg: { type: string; value: Parent }) => void117+ let parentCommit: () => void118+119+ const parents = createCollection<Parent>({120+ id: `dedup-parents-sync`,121+ getKey: (p) => p.id,122+ sync: {123+ sync: ({ begin, write, commit, markReady }) => {124+ parentBegin = begin125+ parentWrite = write as any⚠️ Potential issue | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash # Verify whether additional `as any` casts exist in this test file and inspect surrounding context. rg -n -C2 '\bas any\b' packages/db/tests/query/loaded-subsets-dedup.test.tsRepository: TanStack/db
Length of output: 232
🏁 Script executed:
#!/bin/bash set -euo pipefail FILE="packages/db/tests/query/loaded-subsets-dedup.test.ts" echo "== All occurrences of 'as any' (context) ==" rg -n -C2 '\bas any\b' "$FILE" || true echo echo "== All occurrences of '\bany\b' (context) ==" rg -n -C1 '\bany\b' "$FILE" || true echo echo "== Context around line 125 ==" nl -ba "$FILE" | sed -n '105,145p'Repository: TanStack/db
Length of output: 526
🏁 Script executed:
#!/bin/bash set -euo pipefail FILE="packages/db/tests/query/loaded-subsets-dedup.test.ts" echo "== Context around parentWrite = write as any (lines ~110-140) ==" sed -n '110,140p' "$FILE" echo echo "== Check for any other 'as any' occurrences ==" rg -n '\bas any\b' "$FILE" || true echo echo "== Check for any 'any' type usages (broad heuristic) ==" rg -n --type-add 'ts:*.ts' --type ts '\bany\b' "$FILE" || trueRepository: TanStack/db
Length of output: 1166
🏁 Script executed:
#!/bin/bash set -euo pipefail rg -n --hidden --no-ignore-vcs "createCollection" packages/db/src packages/db/lib packages/db || trueRepository: TanStack/db
Length of output: 50441
🏁 Script executed:
#!/bin/bash set -euo pipefail # Find the sync callback type that provides { begin, write, commit, markReady } rg -n --hidden --no-ignore-vcs "markReady" packages/db/src packages/db/lib packages/db || trueRepository: TanStack/db
Length of output: 42108
🏁 Script executed:
#!/bin/bash set -euo pipefail # Find the type signature for 'sync:' option and its 'write' parameter rg -n --hidden --no-ignore-vcs "sync:\s*\{" packages/db/src packages/db/lib packages/db/tests || trueRepository: TanStack/db
Length of output: 22260
🏁 Script executed:
#!/bin/bash set -euo pipefail FILE="packages/db/src/types.ts" rg -n "interface Sync|type Sync|SyncConfig|sync:|write\s*:" "$FILE"Repository: TanStack/db
Length of output: 384
🏁 Script executed:
#!/bin/bash set -euo pipefail FILE="packages/db/src/types.ts" # Show likely relevant ranges around earlier markReady definition and sync types sed -n '300,420p' "$FILE" sed -n '420,560p' "$FILE"Repository: TanStack/db
Length of output: 8597
🏁 Script executed:
#!/bin/bash set -euo pipefail FILE="packages/db/src/types.ts" rg -n "interface Sync|type Sync|SyncConfig|sync:|write\s*:" "$FILE"Repository: TanStack/db
Length of output: 384
🏁 Script executed:
#!/bin/bash set -euo pipefail FILE="packages/db/src/types.ts" sed -n '300,420p' "$FILE" sed -n '420,560p' "$FILE"Repository: TanStack/db
Length of output: 8597
🏁 Script executed:
#!/bin/bash set -euo pipefail FILE="packages/db/tests/query/loaded-subsets-dedup.test.ts" echo "== Usages of parentWrite in this file ==" rg -n "parentWrite" "$FILE" echo echo "== Context around parentWrite usages ==" rg -n -C3 "parentWrite" "$FILE"Repository: TanStack/db
Length of output: 1068
🏁 Script executed:
#!/bin/bash set -euo pipefail FILE="packages/db/tests/query/loaded-subsets-dedup.test.ts" sed -n '1,120p' "$FILE"Repository: TanStack/db
Length of output: 3449
🏁 Script executed:
#!/bin/bash set -euo pipefail FILE="packages/db/src/types.ts" rg -n "export type OperationType|type OperationType|OperationType\s*=" "$FILE" sed -n '200,320p' "$FILE"Repository: TanStack/db
Length of output: 4266
🏁 Script executed:
#!/bin/bash set -euo pipefail FILE="packages/db/src/collection/index.ts" sed -n '1,220p' "$FILE"Repository: TanStack/db
Length of output: 7386
Remove
write as anyby typingparentWritetoChangeMessageOrDeleteKeyMessage(loaded-subsets-dedup.test.ts:125)writeis already typed bySyncConfigas(message: ChangeMessageOrDeleteKeyMessage<Parent, TKey>) => void, butparentWriteis annotated as(msg: { type: string; value: Parent }) => void, forcing theas anycast. UpdateparentWritetoChangeMessageOrDeleteKeyMessage<Parent, number>(import the type from../../src/types.js) and assignparentWrite = writewithoutany.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/db/tests/query/loaded-subsets-dedup.test.ts` at line 125, Replace the unsafe cast by correctly typing parentWrite: change its type from the ad-hoc (msg: { type: string; value: Parent }) => void to ChangeMessageOrDeleteKeyMessage<Parent, number> (import ChangeMessageOrDeleteKeyMessage from ../../src/types.js) and then assign parentWrite = write (no "as any"); ensure the generic keyof/number matches the test's Parent and TKey usage so the assignment type-checks.✅ Addressed in commit 4c1262e
@@ -0,0 +156,21 @@156+157+ parentBegin!()158+ parentWrite!({ type: `insert`, value: { id: 4, name: `Parent D` } })159+ parentCommit!()160+ await flushPromises()161+162+ const newCalls = loadSubsetCalls.slice(callCountAfterPreload)163+164+ for (const call of newCalls) {165+ if (!call.where) continue166+ const filters = extractSimpleComparisons(call.where)167+ for (const filter of filters) {168+ if (filter.operator === `in`) {169+ const values = filter.value as Array<number>170+ expect(values).toContain(4)171+ expect(values).not.toContain(1)172+ expect(values).not.toContain(2)173+ expect(values).not.toContain(3)174+ }175+ }176+ }⚠️ Potential issue | 🟡 Minor | ⚡ Quick win
This assertion block can pass vacuously if no
infilter is produced.Add an explicit assertion that at least one
inpredicate was inspected; otherwise regressions can slip through with zero relevant filters.Suggested fix
+ let assertedInPredicate = false for (const call of newCalls) { if (!call.where) continue const filters = extractSimpleComparisons(call.where) for (const filter of filters) { if (filter.operator === `in`) { + assertedInPredicate = true const values = filter.value as Array<number> expect(values).toContain(4) expect(values).not.toContain(1) expect(values).not.toContain(2) expect(values).not.toContain(3) } } } + expect(assertedInPredicate).toBe(true)📝 Committable suggestion
‼️ IMPORTANT Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
let assertedInPredicate = false for (const call of newCalls) { if (!call.where) continue const filters = extractSimpleComparisons(call.where) for (const filter of filters) { if (filter.operator === `in`) { assertedInPredicate = true const values = filter.value as Array<number> expect(values).toContain(4) expect(values).not.toContain(1) expect(values).not.toContain(2) expect(values).not.toContain(3) } } } expect(assertedInPredicate).toBe(true)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/db/tests/query/loaded-subsets-dedup.test.ts` around lines 164 - 176, The test currently may pass vacuously if no 'in' filters are produced; update the loop that iterates newCalls and inspects extractSimpleComparisons(call.where) to assert that at least one filter with operator === 'in' was seen. For example, introduce a counter or collect matching filters (referencing newCalls, extractSimpleComparisons, filters and filter.operator) and after the loop add an assertion that the count/collection length is > 0 before relying on contains/not.toContain checks.✅ Addressed in commit 4c1262e
♻️ Duplicate comments (1)
packages/db/tests/query/loaded-subsets-dedup.test.ts (1)
100-118: ⚠️ Potential issue | 🟠 Major | ⚡ Quick winFirst dedup regression test is still not exercising the second request path.
On Line 100 you call
preload()once, then on Line 117 assert call count didn’t grow, but no second snapshot/preload action is performed. This can still pass vacuously and won’t catch regressions in repeated-request dedupe.As per coding guidelines, "Always add unit tests that reproduce a bug before fixing it to ensure the bug is fixed and prevent regression".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/db/tests/query/loaded-subsets-dedup.test.ts` around lines 100 - 118, The test only calls liveQuery.preload() once then asserts loadSubsetCalls length unchanged, so it never exercises the repeated-request dedupe path; update the test (around liveQuery.preload(), loadSubsetCalls and extractSimpleComparisons usage) to perform the second preload/snapshot call (e.g., call liveQuery.preload() or the snapshot-triggering helper a second time), capture loadSubsetCalls.length before and after the second call, and assert the call count does not increase and the second request (if present) has the same filters as the first by reusing firstCall = loadSubsetCalls[0] and comparing extractSimpleComparisons on any subsequent call (or asserting no new call was pushed).🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. Duplicate comments: In `@packages/db/tests/query/loaded-subsets-dedup.test.ts`: - Around line 100-118: The test only calls liveQuery.preload() once then asserts loadSubsetCalls length unchanged, so it never exercises the repeated-request dedupe path; update the test (around liveQuery.preload(), loadSubsetCalls and extractSimpleComparisons usage) to perform the second preload/snapshot call (e.g., call liveQuery.preload() or the snapshot-triggering helper a second time), capture loadSubsetCalls.length before and after the second call, and assert the call count does not increase and the second request (if present) has the same filters as the first by reusing firstCall = loadSubsetCalls[0] and comparing extractSimpleComparisons on any subsequent call (or asserting no new call was pushed).
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID:
3471791a-036d-4347-979d-54625d24bc5f📥 Commits
Reviewing files that changed from the base of the PR and between f8828b345290e98b84b08abb7fed4176fb3f32a8 and 4c1262e03717224d79c45beceaefc6c8b71efca9.
📒 Files selected for processing (3)
packages/db/src/collection/subscription.tspackages/db/tests/collection-subscription.test.tspackages/db/tests/query/loaded-subsets-dedup.test.ts
Closing as superseded per RFC #1657 Phase 0 cleanup. The adapter-level DeduplicatedLoadSubset layer now structurally deduplicates resolved and in-flight subset coverage and prevents the reported backend predicate/query growth. The production point fixes here would duplicate that responsibility. The current tests do not directly demonstrate bounded growth, so any test-only follow-up should stress overlapping requests and assert bounded backend calls rather than transplanting these tests unchanged. Thank you for the investigation and contribution.
KyleAMathews(GitHub profile, opens in a new tab) closed this
Sign in to join this conversation.
No actionable comments were generated in the recent review. 🎉
ℹ️ Recent review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID:
6f60ab4e-6121-4bfc-a272-10399e7cd820📥 Commits
Reviewing files that changed from the base of the PR and between 4c1262e03717224d79c45beceaefc6c8b71efca9 and 96350c105498577b9678bd4eaa7d31e4d9d64d69.
📒 Files selected for processing (1)
packages/db/tests/query/loaded-subsets-dedup.test.ts📝 Walkthrough
Walkthrough#
This PR deduplicates subset loads: subscription snapshot requests now skip when an existing loaded subset covers the predicate, and join lazy-loading tracks loaded join keys to avoid redundant indexed snapshot requests. Tests and a changeset were added.
Changes#
Prevent loadedSubsets unbounded growth through deduplication
packages/db/src/collection/subscription.tsisPredicateSubsetand apply it inrequestSnapshotandrequestLimitedSnapshotso_sync.loadSubset,onLoadSubsetResult,loadedSubsets.push, and promise tracking happen only when the new predicate isn't already covered.packages/db/src/query/compiler/joins.tsloadedJoinKeysduring tap interception: deduplicate/filter nulls, exclude already-loaded keys, request indexed snapshots for remaining keys, and record successfully-loaded keys; fallback to full-collection load on index-unavailable.packages/db/tests/query/loaded-subsets-dedup.test.ts,packages/db/tests/collection-subscription.test.tsloadSubsetcalls to assertin-filter decomposition and that repeated join-key/subset requests do not cause unboundedloadedSubsetsgrowth; update a concurrent-load test to use distinct where expressions..changeset/fix-loaded-subsets-dedup.md@tanstack/dbdescribing the loadedSubsets deduplication fix.Sequence Diagram(s):
🎯 3 (Moderate) | ⏱️ ~25 minutes
🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)#
✅ Passed checks (4 passed)
✨ Finishing Touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
❤️ Share