Skip to content
Source
Sign inSign up
TanStack/db

fix(db-ivm): use row keys for stable ORDER BY tie-breaking #957

Mergedsamwillis(GitHub profile, opens in a new tab)merged 2 commits intomain fromrefs/pull/957/headmerged by samwillis(GitHub profile, opens in a new tab)

Summary

  • Replace hash-based object ID tie-breaking with direct key comparison for deterministic ordering when ORDER BY values are equal
  • Simplify internal data structures by removing the TaggedValue abstraction
  • Remove globalObjectIdGenerator dependency from the topK operators

Problem

When multiple rows have equal ORDER BY values, the previous implementation used globalObjectIdGenerator.getId(key) to generate a numeric tag for tie-breaking. This had several issues:

  1. Hash collisions: The simple hash function could produce the same value for different keys
  2. Unstable for objects: Object references get auto-incrementing IDs, which aren't stable across page reloads
  3. Unnecessary complexity: The TaggedValue<K, V> = [K, V, Tag] 3-tuple added overhead

Solution

Use the row key directly for tie-breaking:

// Before (hash-based, potentially unstable):
const tieBreakerA = getTag(a)  // globalObjectIdGenerator.getId(key)
const tieBreakerB = getTag(b)
return tieBreakerA - tieBreakerB

// After (stable, deterministic):
if (aKey === bKey) return 0
if (aKey < bKey) return -1
return 1

This works because:

  • Row keys are always string | number (from collection's getKey)
  • Each key appears at most once in the topK (enforced by multiplicity tracking)
  • Direct comparison is deterministic and stable across sessions

Changes

  • topKWithFractionalIndex.ts:
    • Replace TaggedValue<K, V> with [K, T] tuple
    • Use direct key comparison for tie-breaking
    • Remove globalObjectIdGenerator import
    • Remove tagValue, getKey, getVal, getTag helpers
  • topKWithFractionalIndexBTree.ts: Update type signatures to use [K, T]

Test plan

  • All existing db-ivm tests pass (267 tests)
  • All existing db package tests pass (1718 tests)
  • ORDER BY tests specifically verified
    1. 🦋 Changeset detected

      Latest commit: 97bd6c931c8d8b0ef3da785d0eacf56e0b7ac681

      The changes in this PR will be included in the next version bump.

      This PR includes changesets to release 14 packages
      NameType
      @tanstack/db-ivmPatch
      @tanstack/dbPatch
      @tanstack/angular-dbPatch
      @tanstack/db-collection-e2ePatch
      @tanstack/electric-db-collectionPatch
      @tanstack/offline-transactionsPatch
      @tanstack/powersync-db-collectionPatch
      @tanstack/query-db-collectionPatch
      @tanstack/react-dbPatch
      @tanstack/rxdb-db-collectionPatch
      @tanstack/solid-dbPatch
      @tanstack/svelte-dbPatch
      @tanstack/trailbase-db-collectionPatch
      @tanstack/vue-dbPatch

      Not sure what this means? Click here to learn what changesets are.

      Click here if you're a maintainer who wants to add another changeset to this PR

    2. More templates

      @tanstack/angular-db

      npm i https://pkg.pr.new/@tanstack/angular-db@957

      @tanstack/db

      npm i https://pkg.pr.new/@tanstack/db@957

      @tanstack/db-ivm

      npm i https://pkg.pr.new/@tanstack/db-ivm@957

      @tanstack/electric-db-collection

      npm i https://pkg.pr.new/@tanstack/electric-db-collection@957

      @tanstack/offline-transactions

      npm i https://pkg.pr.new/@tanstack/offline-transactions@957

      @tanstack/powersync-db-collection

      npm i https://pkg.pr.new/@tanstack/powersync-db-collection@957

      @tanstack/query-db-collection

      npm i https://pkg.pr.new/@tanstack/query-db-collection@957

      @tanstack/react-db

      npm i https://pkg.pr.new/@tanstack/react-db@957

      @tanstack/rxdb-db-collection

      npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@957

      @tanstack/solid-db

      npm i https://pkg.pr.new/@tanstack/solid-db@957

      @tanstack/svelte-db

      npm i https://pkg.pr.new/@tanstack/svelte-db@957

      @tanstack/trailbase-db-collection

      npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@957

      @tanstack/vue-db

      npm i https://pkg.pr.new/@tanstack/vue-db@957

      commit: 97bd6c9

    3. Size Change: 0 B

      Total Size: 87.2 kB

      ℹ️ View Unchanged
      FilenameSize
      ./packages/db/dist/esm/collection/change-events.js1.39 kB
      ./packages/db/dist/esm/collection/changes.js977 B
      ./packages/db/dist/esm/collection/events.js388 B
      ./packages/db/dist/esm/collection/index.js3.24 kB
      ./packages/db/dist/esm/collection/indexes.js1.1 kB
      ./packages/db/dist/esm/collection/lifecycle.js1.67 kB
      ./packages/db/dist/esm/collection/mutations.js2.31 kB
      ./packages/db/dist/esm/collection/state.js3.43 kB
      ./packages/db/dist/esm/collection/subscription.js2.55 kB
      ./packages/db/dist/esm/collection/sync.js2.37 kB
      ./packages/db/dist/esm/deferred.js207 B
      ./packages/db/dist/esm/errors.js4.19 kB
      ./packages/db/dist/esm/event-emitter.js748 B
      ./packages/db/dist/esm/index.js2.66 kB
      ./packages/db/dist/esm/indexes/auto-index.js742 B
      ./packages/db/dist/esm/indexes/base-index.js766 B
      ./packages/db/dist/esm/indexes/btree-index.js1.87 kB
      ./packages/db/dist/esm/indexes/lazy-index.js1.1 kB
      ./packages/db/dist/esm/indexes/reverse-index.js513 B
      ./packages/db/dist/esm/local-only.js837 B
      ./packages/db/dist/esm/local-storage.js2.1 kB
      ./packages/db/dist/esm/optimistic-action.js359 B
      ./packages/db/dist/esm/paced-mutations.js496 B
      ./packages/db/dist/esm/proxy.js3.75 kB
      ./packages/db/dist/esm/query/builder/functions.js733 B
      ./packages/db/dist/esm/query/builder/index.js3.96 kB
      ./packages/db/dist/esm/query/builder/ref-proxy.js917 B
      ./packages/db/dist/esm/query/compiler/evaluators.js1.35 kB
      ./packages/db/dist/esm/query/compiler/expressions.js430 B
      ./packages/db/dist/esm/query/compiler/group-by.js1.8 kB
      ./packages/db/dist/esm/query/compiler/index.js1.96 kB
      ./packages/db/dist/esm/query/compiler/joins.js2 kB
      ./packages/db/dist/esm/query/compiler/order-by.js1.25 kB
      ./packages/db/dist/esm/query/compiler/select.js1.07 kB
      ./packages/db/dist/esm/query/expression-helpers.js1.43 kB
      ./packages/db/dist/esm/query/ir.js673 B
      ./packages/db/dist/esm/query/live-query-collection.js360 B
      ./packages/db/dist/esm/query/live/collection-config-builder.js5.33 kB
      ./packages/db/dist/esm/query/live/collection-registry.js264 B
      ./packages/db/dist/esm/query/live/collection-subscriber.js1.74 kB
      ./packages/db/dist/esm/query/live/internal.js130 B
      ./packages/db/dist/esm/query/optimizer.js2.56 kB
      ./packages/db/dist/esm/query/predicate-utils.js2.91 kB
      ./packages/db/dist/esm/query/subset-dedupe.js921 B
      ./packages/db/dist/esm/scheduler.js1.3 kB
      ./packages/db/dist/esm/SortedMap.js1.18 kB
      ./packages/db/dist/esm/strategies/debounceStrategy.js247 B
      ./packages/db/dist/esm/strategies/queueStrategy.js428 B
      ./packages/db/dist/esm/strategies/throttleStrategy.js246 B
      ./packages/db/dist/esm/transactions.js2.9 kB
      ./packages/db/dist/esm/utils.js881 B
      ./packages/db/dist/esm/utils/browser-polyfills.js304 B
      ./packages/db/dist/esm/utils/btree.js5.61 kB
      ./packages/db/dist/esm/utils/comparison.js852 B
      ./packages/db/dist/esm/utils/index-optimization.js1.51 kB
      ./packages/db/dist/esm/utils/type-guards.js157 B
      compressed-size-action::db-package-size
    4. Size Change: 0 B

      Total Size: 3.35 kB

      ℹ️ View Unchanged
      FilenameSize
      ./packages/react-db/dist/esm/index.js225 B
      ./packages/react-db/dist/esm/useLiveInfiniteQuery.js1.17 kB
      ./packages/react-db/dist/esm/useLiveQuery.js1.12 kB
      ./packages/react-db/dist/esm/useLiveSuspenseQuery.js431 B
      ./packages/react-db/dist/esm/usePacedMutations.js401 B
      compressed-size-action::react-db-package-size
    5. kevin-dp(GitHub profile, opens in a new tab) approved these changes

      Way better to tie break on the key rather than a hash. Make sure to fix the problem with the current tie-breaker though such that it correctly compares keys of different types (strings vs numbers).

      OutdatedResolvedby samwillis(GitHub profile, opens in a new tab)
      OutdatedResolvedby samwillis(GitHub profile, opens in a new tab)
    6. kevin-dp(GitHub profile, opens in a new tab)replied on packages/db-ivm/src/operators/topKWithFractionalIndex.ts line 269

      There is a correct implementation in comparison.ts in the db package in your follow up PR:

      /**
       * Compares two keys (string | number) for deterministic ordering.
       * Strings come before numbers, then sorted within type.
       */
      export function compareKeys<TKey extends string | number>(
        a: TKey,
        b: TKey
      ): number {
        // Same type: compare directly
        if (typeof a === typeof b) {
          if (a < b) return -1
          if (a > b) return 1
          return 0
        }
        // Different types: strings come before numbers
        return typeof a === `string` ? -1 : 1
      }
    7. samwillis(GitHub profile, opens in a new tab)replied on packages/db-ivm/src/operators/topKWithFractionalIndex.ts line 269

      I've used the one from the followup PR, and placed it in utils.ts with it exported from the package. Will change the followup to use the same implementation so we only have one copy.

    8. 🎉 This PR has been released!

      Thank you for your contribution!

Sign in to join this conversation.