Skip to content
Source
Sign inSign up
TanStack/db

fix: reconcile duplicate live query child inserts #1600

MergedKyleAMathews(GitHub profile, opens in a new tab)merged 6 commits intomain fromrefs/pull/1600/headmerged by kevin-dp(GitHub profile, opens in a new tab)

Fix live-query includes reconciliation so .update() flows that re-emit an existing joined child row update the internal child collection instead of attempting a duplicate insert.

User-visible impact: updating a queryCollectionOptions source used in a live query with joined/included child rows no longer crashes with either the secondary LIVE_QUERY_INTERNAL TypeError or the underlying duplicate child insert error.

Root Cause

The user action is .update(), but that update triggers live-query graph reconciliation. During reconciliation, flushIncludesState writes changes into internal child collections such as:

__child-collection:detailed-surplus-agreement-line-items-product-30587

From that internal collection's perspective, a derived/joined row can be emitted as an insert even though the user operation was an update. When the child collection already contained that key, sync correctly rejected the duplicate insert:

Cannot insert document with key "302935" from sync because it already exists in the collection "__child-collection:..."

There was also a secondary issue in the duplicate-key error path: generic/internal collection configs do not necessarily have live-query internals in config.utils, but sync.ts assumed they did while constructing DuplicateKeySyncError context.

Approach

The fix has two parts:

  1. In flushIncludesState, compute whether the internal child collection already has the child key before writing an insert-only child change. If the child already exists, write an internal update instead of an insert.

    const key = entry.syncMethods.collection.getKeyFromItem(change.value)
    const childAlreadyExists = entry.syncMethods.collection.has(key)
    
    if (change.inserts > 0 && change.deletes === 0) {
      entry.syncMethods.write({
        value: change.value,
        type: childAlreadyExists ? `update` : `insert`,
      })
    }
  2. In the duplicate-key sync error path, guard optional config.utils before reading LIVE_QUERY_INTERNAL, preserving enhanced context when present and defaulting the flags when absent.

Key Invariants

  • Existing internal child rows must not receive duplicate sync inserts during live-query includes reconciliation.
  • New internal child rows should still be inserted normally.
  • Existing live-query-specific duplicate-key diagnostics should be preserved when LIVE_QUERY_INTERNAL is present.
  • Collection configs without utils must not throw a secondary TypeError while constructing duplicate-key errors.

Non-goals

  • This does not change public .update() semantics.
  • This does not change query collection refetch behavior.
  • This does not redesign includes materialization or child collection identity.

Trade-offs

The targeted fix keeps reconciliation local to the internal child collection write decision. A broader alternative would be changing upstream D2/includes change classification, but the child collection is the place with direct knowledge of whether a given child key is already materialized.

Verification

cd packages/db
pnpm vitest run tests/collection.test.ts -t "config has no utils" --coverage.enabled=false
pnpm vitest run tests/query/includes.test.ts --coverage.enabled=false --pool-options.threads.maxThreads=2
cd packages/query-db-collection
pnpm vitest run tests/query.test.ts -t "duplicate insert into includes child" --coverage.enabled=false

CI is green for PR #1600, including:

  • Test
  • Run E2E Tests
  • Build Example Site
  • Preview

Files changed

  • packages/db/src/collection/sync.ts
    • Guard access to optional config.utils before reading LIVE_QUERY_INTERNAL.
  • packages/db/src/query/live/collection-config-builder.ts
    • Convert insert-only internal child changes to updates when the child key already exists.
  • packages/db/tests/collection.test.ts
    • Add regression coverage for duplicate-key sync errors when collection config has no utils.
  • packages/db/tests/query/includes.test.ts
    • Add coverage for joined includes child reconciliation.
  • packages/query-db-collection/tests/query.test.ts
    • Add coverage for query collection update/refetch feeding a live-query includes child collection.
  • .changeset/fix-sync-duplicate-key-utils.md
    • Add patch changeset for @tanstack/db.

Fixes #1599

Summary by CodeRabbit

Bug Fixes

  • Improved duplicate-key sync failure handling in live queries, preventing unexpected TypeErrors and ensuring the correct duplicate-key error is raised.
  • Refined live query includes propagation for joined child collections to avoid duplicate child-row insertions during update flows.
  • Improved child write behavior to correctly choose between inserts and updates based on whether the child already exists.

Tests

  • Added regression tests covering duplicate-key sync behavior and joined include update scenarios.
  • Extended existing test assertions and imports to validate the new error and propagation behavior.
    1. Review Change Stack

      No actionable comments were generated in the recent review. 🎉

      ℹ️ Recent review info
      ⚙️ Run configuration

      Configuration used: defaults

      Review profile: CHILL

      Plan: Pro

      Run ID: af881539-6d7b-474f-b473-ac221f464346

      📥 Commits

      Reviewing files that changed from the base of the PR and between 7cba2013eb4bba92872aa95530c0e14ec5246b72 and 8305b9d353f958aed4450c3de965a7b4714b32ff.

      📒 Files selected for processing (2)
      • packages/db/tests/query/includes.test.ts
      • packages/query-db-collection/tests/query.test.ts
      🚧 Files skipped from review as they are similar to previous changes (2)
      • packages/db/tests/query/includes.test.ts
      • packages/query-db-collection/tests/query.test.ts

      📝 Walkthrough

      Walkthrough

      CollectionSyncManager.write() now uses optional chaining when accessing this.config.utils and LIVE_QUERY_INTERNAL, preventing a TypeError for collections created without live-query utils. The child include logic in flushIncludesState is refined to decide between insert and update based on whether the child key already exists in the collection. Regression tests and a patch changeset are added.

      Changes

      Sync duplicate-key fix and child include insert/update improvements

      Layer / File(s)Summary
      Safe utils access in sync write path
      packages/db/src/collection/sync.ts, .changeset/fix-sync-duplicate-key-utils.md
      sync.ts derives internal via optional chaining on this.config.utils before accessing LIVE_QUERY_INTERNAL, so DuplicateKeySyncError is constructed safely when utils is undefined. The changeset documents the patch.
      Regression test for duplicate insert without utils
      packages/db/tests/collection.test.ts
      Test suite imports DuplicateKeySyncError and SyncConfig types, and a new test asserts that when a collection is created without utils and a duplicate insert occurs, the error type is DuplicateKeySyncError rather than TypeError.
      Child include insert versus update logic
      packages/db/src/query/live/collection-config-builder.ts, packages/db/tests/query/includes.test.ts
      flushIncludesState now computes the child key and checks whether it already exists in the collection, using that flag to conditionally emit insert or update for pure-insert changes. A regression test verifies that deleting and reinserting a parent row with a joined include does not cause duplicate child inserts.
      QueryCollection integration test
      packages/query-db-collection/tests/query.test.ts
      Test utilities import mockSyncCollectionOptions, and a new test constructs a live query joining a sync-backed line-items collection to a query-backed products collection, updates a product, and verifies the child data reflects the update without throwing or creating duplicate inserts.

      Estimated code review effort

      🎯 3 (Moderate) | ⏱️ ~25 minutes

      • TanStack/db#1599: Addresses the TypeError when accessing utils[LIVE_QUERY_INTERNAL] on an undefined utils object inside CollectionSyncManager.write() during sync reconciliation for collections created with queryCollectionOptions. This PR implements the fix by guarding the access with optional chaining so undefined utils are handled safely.

      Suggested reviewers

      • samwillis

      Poem

      🐇 Hoppin' through the sync code with care, A TypeError lurked like a snare. With ?. in place, We guard the null space— Child includes now know when to insert or dare! 🥕

      🚥 Pre-merge checks | ✅ 5
      ✅ Passed checks (5 passed)
      Check nameStatusExplanation
      Title check✅ PassedThe title 'fix: reconcile duplicate live query child inserts' accurately describes the primary change in the PR.
      Description check✅ PassedThe PR description comprehensively covers all required sections with detailed change motivation, root cause analysis, approach, verification steps, and affected files.
      Linked Issues check✅ PassedThe PR directly addresses both issues from #1599: guards optional config.utils access before reading LIVE_QUERY_INTERNAL, and converts insert-only child changes to updates when the child key already exists.
      Out of Scope Changes check✅ PassedAll changes are directly scoped to fixing the linked issue #1599: sync.ts guards utils access, collection-config-builder.ts prevents duplicate child inserts, and test files provide regression coverage.
      Docstring Coverage✅ PassedDocstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
      ✏️ Tip: You can configure your own custom pre-merge checks in the settings.
      ✨ Finishing Touches
      📝 Generate docstrings
      • Create stacked PR
      • Commit on current branch
      🧪 Generate unit tests (beta)
      • Create PR with unit tests
      • Commit unit tests in branch `issue-1599-red-green`

      Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

      ❤️ Share
      Comment `@coderabbitai help` to get the list of available commands and usage tips.
    2. More templates

      @tanstack/angular-db

      npm i https://pkg.pr.new/@tanstack/angular-db@1600

      @tanstack/browser-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/browser-db-sqlite-persistence@1600

      @tanstack/capacitor-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/capacitor-db-sqlite-persistence@1600

      @tanstack/cloudflare-durable-objects-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/cloudflare-durable-objects-db-sqlite-persistence@1600

      @tanstack/db

      npm i https://pkg.pr.new/@tanstack/db@1600

      @tanstack/db-ivm

      npm i https://pkg.pr.new/@tanstack/db-ivm@1600

      @tanstack/db-sqlite-persistence-core

      npm i https://pkg.pr.new/@tanstack/db-sqlite-persistence-core@1600

      @tanstack/electric-db-collection

      npm i https://pkg.pr.new/@tanstack/electric-db-collection@1600

      @tanstack/electron-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/electron-db-sqlite-persistence@1600

      @tanstack/expo-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/expo-db-sqlite-persistence@1600

      @tanstack/node-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/node-db-sqlite-persistence@1600

      @tanstack/offline-transactions

      npm i https://pkg.pr.new/@tanstack/offline-transactions@1600

      @tanstack/powersync-db-collection

      npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1600

      @tanstack/query-db-collection

      npm i https://pkg.pr.new/@tanstack/query-db-collection@1600

      @tanstack/react-db

      npm i https://pkg.pr.new/@tanstack/react-db@1600

      @tanstack/react-native-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/react-native-db-sqlite-persistence@1600

      @tanstack/rxdb-db-collection

      npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1600

      @tanstack/solid-db

      npm i https://pkg.pr.new/@tanstack/solid-db@1600

      @tanstack/svelte-db

      npm i https://pkg.pr.new/@tanstack/svelte-db@1600

      @tanstack/tauri-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/tauri-db-sqlite-persistence@1600

      @tanstack/trailbase-db-collection

      npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1600

      @tanstack/vue-db

      npm i https://pkg.pr.new/@tanstack/vue-db@1600

      commit: 8305b9d

    3. Size Change: +25 B (+0.02%)

      Total Size: 123 kB

      📦 View Changed
      FilenameSizeChange
      packages/db/dist/esm/collection/sync.js2.88 kB+2 B (+0.07%)
      packages/db/dist/esm/query/live/collection-config-builder.js8.38 kB+23 B (+0.28%)
      ℹ️ View Unchanged
      FilenameSize
      packages/db/dist/esm/collection/change-events.js1.39 kB
      packages/db/dist/esm/collection/changes.js1.38 kB
      packages/db/dist/esm/collection/cleanup-queue.js810 B
      packages/db/dist/esm/collection/events.js434 B
      packages/db/dist/esm/collection/index.js3.62 kB
      packages/db/dist/esm/collection/indexes.js1.99 kB
      packages/db/dist/esm/collection/lifecycle.js1.69 kB
      packages/db/dist/esm/collection/mutations.js2.47 kB
      packages/db/dist/esm/collection/state.js5.33 kB
      packages/db/dist/esm/collection/subscription.js3.74 kB
      packages/db/dist/esm/collection/transaction-metadata.js144 B
      packages/db/dist/esm/deferred.js207 B
      packages/db/dist/esm/errors.js5.01 kB
      packages/db/dist/esm/event-emitter.js748 B
      packages/db/dist/esm/index.js3.09 kB
      packages/db/dist/esm/indexes/auto-index.js829 B
      packages/db/dist/esm/indexes/base-index.js729 B
      packages/db/dist/esm/indexes/basic-index.js2.05 kB
      packages/db/dist/esm/indexes/btree-index.js2.17 kB
      packages/db/dist/esm/indexes/index-registry.js820 B
      packages/db/dist/esm/indexes/reverse-index.js538 B
      packages/db/dist/esm/local-only.js916 B
      packages/db/dist/esm/local-storage.js2.12 kB
      packages/db/dist/esm/optimistic-action.js359 B
      packages/db/dist/esm/paced-mutations.js496 B
      packages/db/dist/esm/proxy.js3.75 kB
      packages/db/dist/esm/query/builder/functions.js1.47 kB
      packages/db/dist/esm/query/builder/index.js5.84 kB
      packages/db/dist/esm/query/builder/ref-proxy.js1.24 kB
      packages/db/dist/esm/query/compiler/evaluators.js1.83 kB
      packages/db/dist/esm/query/compiler/expressions.js430 B
      packages/db/dist/esm/query/compiler/group-by.js3.56 kB
      packages/db/dist/esm/query/compiler/index.js6.67 kB
      packages/db/dist/esm/query/compiler/joins.js2.5 kB
      packages/db/dist/esm/query/compiler/lazy-targets.js918 B
      packages/db/dist/esm/query/compiler/order-by.js1.74 kB
      packages/db/dist/esm/query/compiler/select.js1.4 kB
      packages/db/dist/esm/query/effect.js4.77 kB
      packages/db/dist/esm/query/expression-helpers.js1.43 kB
      packages/db/dist/esm/query/ir.js1.25 kB
      packages/db/dist/esm/query/live-query-collection.js360 B
      packages/db/dist/esm/query/live/collection-registry.js264 B
      packages/db/dist/esm/query/live/collection-subscriber.js1.93 kB
      packages/db/dist/esm/query/live/internal.js145 B
      packages/db/dist/esm/query/live/utils.js1.81 kB
      packages/db/dist/esm/query/optimizer.js2.92 kB
      packages/db/dist/esm/query/predicate-utils.js2.97 kB
      packages/db/dist/esm/query/query-once.js359 B
      packages/db/dist/esm/query/subset-dedupe.js960 B
      packages/db/dist/esm/scheduler.js1.3 kB
      packages/db/dist/esm/SortedMap.js1.3 kB
      packages/db/dist/esm/strategies/debounceStrategy.js247 B
      packages/db/dist/esm/strategies/queueStrategy.js428 B
      packages/db/dist/esm/strategies/throttleStrategy.js246 B
      packages/db/dist/esm/transactions.js3.03 kB
      packages/db/dist/esm/utils.js927 B
      packages/db/dist/esm/utils/array-utils.js273 B
      packages/db/dist/esm/utils/browser-polyfills.js304 B
      packages/db/dist/esm/utils/btree.js5.61 kB
      packages/db/dist/esm/utils/comparison.js1.05 kB
      packages/db/dist/esm/utils/cursor.js457 B
      packages/db/dist/esm/utils/index-optimization.js1.54 kB
      packages/db/dist/esm/utils/type-guards.js157 B
      packages/db/dist/esm/utils/uuid.js449 B
      packages/db/dist/esm/virtual-props.js360 B
      compressed-size-action::db-package-size
    4. Size Change: 0 B

      Total Size: 4.24 kB

      ℹ️ View Unchanged
      FilenameSize
      packages/react-db/dist/esm/index.js249 B
      packages/react-db/dist/esm/useLiveInfiniteQuery.js1.32 kB
      packages/react-db/dist/esm/useLiveQuery.js1.34 kB
      packages/react-db/dist/esm/useLiveQueryEffect.js355 B
      packages/react-db/dist/esm/useLiveSuspenseQuery.js567 B
      packages/react-db/dist/esm/usePacedMutations.js401 B
      compressed-size-action::react-db-package-size
    5. Actionable comments posted: 2

      🤖 Prompt for all review comments with AI agents
      Verify each finding against current code. Fix only still-valid issues, skip the
      rest with a brief reason, keep changes minimal, and validate.
      
      Inline comments:
      In `@packages/db/tests/query/includes.test.ts`:
      - Around line 617-621: Replace the fixed `setTimeout(resolve, 10)` sleep in this
      test with a condition-based wait that directly polls for the expected child
      items snapshot. Instead of sleeping for a fixed duration, create a polling loop
      that repeatedly checks if the childItems from the product matches the expected
      array, and only resolve the promise when the condition is satisfied. This
      eliminates the timing sensitivity and prevents test flakiness when async
      propagation is slower.
      
      In `@packages/query-db-collection/tests/query.test.ts`:
      - Around line 292-298: The regression assertion in the vi.waitFor() callback is
      validating products.get() directly, which doesn't test if the includes-child
      reconciliation is working properly and can miss duplicate-insert bugs in the
      join logic. Change the assertion to validate the included product collection
      from the live query row instead, so the test actually guards against regression
      in the child include reconciliation. Modify the expect statement inside
      vi.waitFor() to access the product data through the included child collection
      relationship rather than through direct products.get() access.
      🪄 Autofix (Beta)

      Fix all unresolved CodeRabbit comments on this PR:

      • Push a commit to this branch (recommended)
      • Create a new PR with the fixes

      ℹ️ Review info
      ⚙️ Run configuration

      Configuration used: defaults

      Review profile: CHILL

      Plan: Pro

      Run ID: 2c8e03e7-4488-41b6-bd03-dd6d99f54b83

      📥 Commits

      Reviewing files that changed from the base of the PR and between dbb87aaae613d3096b8e9e2f76ba7ec477e46f89 and c1832723c0ea39520ed2ad12d5f495e6a4df847b.

      📒 Files selected for processing (4)
      • packages/db/src/query/live/collection-config-builder.ts
      • packages/db/tests/collection.test.ts
      • packages/db/tests/query/includes.test.ts
      • packages/query-db-collection/tests/query.test.ts
      🚧 Files skipped from review as they are similar to previous changes (1)
      • packages/db/tests/collection.test.ts
      OutdatedResolvedby coderabbitai[bot](GitHub profile, opens in a new tab)
      Resolvedby coderabbitai[bot](GitHub profile, opens in a new tab)
    6. Installed : https://pkg.pr.new/@tanstack/react-db@c183272

      This seems to done the trick i no longer see the error in the console and updates are working again thanks for the quick response!!

Sign in to join this conversation.