Skip to content
Source
Sign inSign up
TanStack/db

fix(db): suggest indexes when manual indexing needs one #1942

MergedKyleAMathews(GitHub profile, opens in a new tab)merged 6 commits intomain fromrefs/pull/1942/headmerged by KyleAMathews(GitHub profile, opens in a new tab)

🎯 Changes

A large Collection with manual indexing now suggests an index when a filtered query uses a field without one. Eager indexing no longer prints that advice before it creates the index.

For example, assume source holds 1,101 rows and this query filters url:

q.from({ row: source }).where(({ row }) => eq(row.url, 'url-7'))
autoIndexBeforeAfter
'off' or omittedNo collection-size suggestion and no index.One suggestion and no automatic index.
'eager'One suggestion, then one automatic index.No suggestion, and the automatic index remains.

The query path had two guards. One skipped manual WHERE fields before analysis. The other restricted size advice to eager mode. Both guards now keep automatic index creation in eager mode and run size advice only for manual mode in development. A matching manual index, a Collection at the threshold, or disabled advice keeps the suggestion silent.

Verification

The seven-case oracle failed four cases on main at ae2eb3fb: explicit manual mode, default manual mode, an unrelated index, and eager mode. All seven cases pass on this branch. The 17 existing auto-index tests and 12 compiler tests pass, with no type errors. The production file removes four more lines than it adds.

The compiler tests use synthetic Collections. Their fixtures now declare empty index maps and zero sizes, which the manual advice path reads.

cd packages/db
../../node_modules/.bin/vitest run tests/query/compiler/basic.test.ts tests/query/compiler/subquery-caching.test.ts tests/index-suggestion-oracle.test.ts tests/collection-auto-index.test.ts --coverage.enabled=false --pool-options.threads.maxThreads=2

The oracle runs in the package's test:oracles campaign. The coverage map records its limits. A patch changeset covers @tanstack/db.

✅ Checklist

  • I have tested this code locally with pnpm test.

🚀 Release Impact

  • This change affects published code, and I have generated a changeset.
  • This change is docs/CI/dev-only (no release).

Fixes #1700

Summary by CodeRabbit

  • New Features
    • In development mode, receive index suggestions for queries on collections with more than 1,000 rows when the queried field has no matching index. Suggestions do not create an index, and are not shown when eager indexing is enabled.
  • Documentation
    • Added guidance on the scope and coverage of index suggestions.
    1. More templates

      @tanstack/angular-db

      npm i https://pkg.pr.new/@tanstack/angular-db@1942

      @tanstack/browser-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/browser-db-sqlite-persistence@1942

      @tanstack/capacitor-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/capacitor-db-sqlite-persistence@1942

      @tanstack/cloudflare-durable-objects-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/cloudflare-durable-objects-db-sqlite-persistence@1942

      @tanstack/db

      npm i https://pkg.pr.new/@tanstack/db@1942

      @tanstack/db-ivm

      npm i https://pkg.pr.new/@tanstack/db-ivm@1942

      @tanstack/db-sqlite-persistence-core

      npm i https://pkg.pr.new/@tanstack/db-sqlite-persistence-core@1942

      @tanstack/electric-db-collection

      npm i https://pkg.pr.new/@tanstack/electric-db-collection@1942

      @tanstack/electron-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/electron-db-sqlite-persistence@1942

      @tanstack/expo-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/expo-db-sqlite-persistence@1942

      @tanstack/node-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/node-db-sqlite-persistence@1942

      @tanstack/offline-transactions

      npm i https://pkg.pr.new/@tanstack/offline-transactions@1942

      @tanstack/powersync-db-collection

      npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1942

      @tanstack/query-db-collection

      npm i https://pkg.pr.new/@tanstack/query-db-collection@1942

      @tanstack/react-db

      npm i https://pkg.pr.new/@tanstack/react-db@1942

      @tanstack/react-native-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/react-native-db-sqlite-persistence@1942

      @tanstack/react-router-with-db

      npm i https://pkg.pr.new/@tanstack/react-router-with-db@1942

      @tanstack/rxdb-db-collection

      npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1942

      @tanstack/solid-db

      npm i https://pkg.pr.new/@tanstack/solid-db@1942

      @tanstack/svelte-db

      npm i https://pkg.pr.new/@tanstack/svelte-db@1942

      @tanstack/tauri-db-sqlite-persistence

      npm i https://pkg.pr.new/@tanstack/tauri-db-sqlite-persistence@1942

      @tanstack/trailbase-db-collection

      npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1942

      @tanstack/vue-db

      npm i https://pkg.pr.new/@tanstack/vue-db@1942

      commit: ca82b02

    2. Review in Change Stack →

      Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

      No actionable comments were generated in the recent review. 🎉

      ℹ️ Recent review info
      ⚙️ Run configuration

      Configuration used: defaults

      Review profile: CHILL

      Plan: Advanced

      Run ID: a1f94eff-c376-4cb0-bcae-a8089cc5cfee

      📥 Commits

      Reviewing files that changed from the base of the PR and between ae2eb3fbf3a314a7f043f0d963fd336d8c1f1ec8 and ca82b02d3366d7918cc49d925dd81b4f65c50f5a.

      📒 Files selected for processing (8)
      • .changeset/fix-manual-index-suggestions.md
      • docs/contributing/oracle-coverage.md
      • docs/contributing/oracle-reviews/issue-1700-index-suggestions.md
      • packages/db/package.json
      • packages/db/src/indexes/auto-index.ts
      • packages/db/tests/index-suggestion-oracle.test.ts
      • packages/db/tests/query/compiler/basic.test.ts
      • packages/db/tests/query/compiler/subquery-caching.test.ts

      Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 0 remain after this review.


      📝 Walkthrough

      Walkthrough

      Development-mode index checks now provide collection-size suggestions for unindexed fields when eager indexing is disabled. Eager indexing creates missing indexes without the collection-size check. A new oracle test and supporting documentation cover the conditions and results.

      Changes

      Collection-size index suggestions

      Layer / File(s)Summary
      Development-mode suggestion behavior
      packages/db/src/indexes/auto-index.ts, .changeset/fix-manual-index-suggestions.md
      When development mode is enabled and eager indexing is disabled, field checks can report collection-size advice without creating an index. The changeset records when the advice is shown or omitted.
      Oracle coverage and review record
      packages/db/tests/index-suggestion-oracle.test.ts, packages/db/package.json, packages/db/tests/query/compiler/*, docs/contributing/oracle-coverage.md, docs/contributing/oracle-reviews/issue-1700-index-suggestions.md
      A seven-scenario oracle checks advice conditions, query results, suggestion metadata, index counts, and cleanup. The test command and compiler fixtures are updated, and the documentation records oracle coverage and limits.

      Priority: ⬇️ Low

      Estimated code review effort: 3 (Moderate) | ~20 minutes

      Change: Bug fix · Severity of issue fixed: Low

      Merge Risk: ⚪ Minimal · up to ca82b

      Large manually indexed collections receive index advice without automatic index creation, while eager indexing avoids redundant advice. No merge-blocking issue remains; merge after normal checks.

      🚥 Pre-merge checks | ✅ 4 | ❌ 1

      ❌ Failed checks (1 warning)

      Check nameStatusExplanationResolution
      Docstring Coverage⚠️ WarningDocstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 4 files. (4 skipped: 4…Write docstrings for the functions missing them to satisfy the coverage threshold.
      ✅ Passed checks (4 passed)
      Check nameStatusExplanation
      Title check✅ PassedThe title clearly and concisely describes the main change: suggesting indexes for manual indexing when needed.
      Description check✅ PassedThe description covers the change, motivation, behavior across indexing modes, verification results, release impact, changeset, and linked issue. The template checklist remains unchecked for `pnpm tes…
      Linked Issues check✅ PassedIssue #1700 requires collection-size advice for large Collections with manual indexing and an unindexed filtered field. It also requires no redundant advice for eager indexing. ensureIndexForField n…
      Out of Scope Changes check✅ PassedThe changes stay within issue #1700. The new oracle test, test-script entry, compiler mock updates, oracle documentation, review record, and changeset support or document the index-suggestion behavior…
      Full details: Docstring Coverage

      Explanation

      Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 4 files. (4 skipped: 4 unsupported.)

      • Fix all pre-merge checks with AI
      ✨ Finishing Touches 💡 1
      📝 Generate docstrings 💡
      • Commit to this branch
      • Create a new PR
      🧪 Generate unit tests (beta)
      • Commit to this branch
      • Create a new PR

      Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

      ❤️ Share
      Comment `@coderabbitai help` to get the list of available commands.
    3. Size Change: +17 B (+0.01%)

      Total Size: 174 kB

      📦 View Changed
      FilenameSizeChange
      packages/db/dist/esm/indexes/auto-index.js841 B+17 B (+2.06%)
      ℹ️ View Unchanged
      FilenameSize
      packages/db/dist/esm/client.js3.7 kB
      packages/db/dist/esm/collection-options.js236 B
      packages/db/dist/esm/collection/change-events.js1.45 kB
      packages/db/dist/esm/collection/changes.js2.41 kB
      packages/db/dist/esm/collection/cleanup-queue.js794 B
      packages/db/dist/esm/collection/events.js481 B
      packages/db/dist/esm/collection/index.js4.46 kB
      packages/db/dist/esm/collection/indexes.js2.03 kB
      packages/db/dist/esm/collection/lifecycle.js2.69 kB
      packages/db/dist/esm/collection/mutations.js2.61 kB
      packages/db/dist/esm/collection/state.js8.32 kB
      packages/db/dist/esm/collection/subscription.js8.81 kB
      packages/db/dist/esm/collection/sync.js5.09 kB
      packages/db/dist/esm/collection/transaction-metadata.js144 B
      packages/db/dist/esm/deferred.js207 B
      packages/db/dist/esm/errors.js5.49 kB
      packages/db/dist/esm/event-emitter.js964 B
      packages/db/dist/esm/index.js3.88 kB
      packages/db/dist/esm/indexes/base-index.js1.21 kB
      packages/db/dist/esm/indexes/basic-index.js2.1 kB
      packages/db/dist/esm/indexes/btree-index.js2.29 kB
      packages/db/dist/esm/indexes/index-registry.js820 B
      packages/db/dist/esm/indexes/reverse-index.js376 B
      packages/db/dist/esm/live-query-adapter.js318 B
      packages/db/dist/esm/live-query-observer.js3.69 kB
      packages/db/dist/esm/live-query-options.js731 B
      packages/db/dist/esm/live-query-window-controller.js4.37 kB
      packages/db/dist/esm/local-only.js989 B
      packages/db/dist/esm/local-storage.js2.17 kB
      packages/db/dist/esm/optimistic-action.js359 B
      packages/db/dist/esm/paced-mutations.js702 B
      packages/db/dist/esm/proxy.js3.32 kB
      packages/db/dist/esm/query/builder/clone-query.js766 B
      packages/db/dist/esm/query/builder/functions.js1.45 kB
      packages/db/dist/esm/query/builder/index.js6.82 kB
      packages/db/dist/esm/query/builder/query-ir.js116 B
      packages/db/dist/esm/query/builder/ref-proxy-identity.js198 B
      packages/db/dist/esm/query/builder/ref-proxy.js1.35 kB
      packages/db/dist/esm/query/builder/wrapper-identity.js221 B
      packages/db/dist/esm/query/compiler/evaluators.js2.06 kB
      packages/db/dist/esm/query/compiler/expressions.js603 B
      packages/db/dist/esm/query/compiler/group-by.js4.16 kB
      packages/db/dist/esm/query/compiler/index.js9.1 kB
      packages/db/dist/esm/query/compiler/joins.js2.99 kB
      packages/db/dist/esm/query/compiler/lazy-targets.js1.13 kB
      packages/db/dist/esm/query/compiler/order-by.js1.99 kB
      packages/db/dist/esm/query/compiler/parent-routes.js319 B
      packages/db/dist/esm/query/compiler/query-equivalence.js455 B
      packages/db/dist/esm/query/compiler/route-metadata.js1.24 kB
      packages/db/dist/esm/query/compiler/select.js1.59 kB
      packages/db/dist/esm/query/effect.js4.98 kB
      packages/db/dist/esm/query/equality-value-identity.js591 B
      packages/db/dist/esm/query/expression-helpers.js1.45 kB
      packages/db/dist/esm/query/ir-stable-identity.js4.22 kB
      packages/db/dist/esm/query/ir.js1.7 kB
      packages/db/dist/esm/query/live-query-collection.js391 B
      packages/db/dist/esm/query/live/bucket-facade-adapter.js2.73 kB
      packages/db/dist/esm/query/live/collection-config-builder.js6.52 kB
      packages/db/dist/esm/query/live/collection-registry.js264 B
      packages/db/dist/esm/query/live/collection-subscriber.js2.16 kB
      packages/db/dist/esm/query/live/graph-scheduler.js303 B
      packages/db/dist/esm/query/live/internal.js145 B
      packages/db/dist/esm/query/live/materialized-pipeline.js2.32 kB
      packages/db/dist/esm/query/live/ordered-source-loader.js4.47 kB
      packages/db/dist/esm/query/live/subset-demand-controller.js1.67 kB
      packages/db/dist/esm/query/live/utils.js1.2 kB
      packages/db/dist/esm/query/optimizer.js2.93 kB
      packages/db/dist/esm/query/query-once.js359 B
      packages/db/dist/esm/query/runtime-reference-identity.js630 B
      packages/db/dist/esm/query/subset-dedupe.js497 B
      packages/db/dist/esm/scheduler.js1.14 kB
      packages/db/dist/esm/SortedMap.js1.3 kB
      packages/db/dist/esm/strategies/debounceStrategy.js331 B
      packages/db/dist/esm/strategies/queueStrategy.js488 B
      packages/db/dist/esm/strategies/throttleStrategy.js386 B
      packages/db/dist/esm/sync-persistence.js530 B
      packages/db/dist/esm/transactions.js3.73 kB
      packages/db/dist/esm/utils.js1.21 kB
      packages/db/dist/esm/utils/array-utils.js270 B
      packages/db/dist/esm/utils/browser-polyfills.js304 B
      packages/db/dist/esm/utils/btree.js4.51 kB
      packages/db/dist/esm/utils/callbacks.js174 B
      packages/db/dist/esm/utils/comparison.js1.5 kB
      packages/db/dist/esm/utils/cursor.js677 B
      packages/db/dist/esm/utils/error.js167 B
      packages/db/dist/esm/utils/get-or-create.js155 B
      packages/db/dist/esm/utils/index-optimization.js2.42 kB
      packages/db/dist/esm/utils/type-guards.js230 B
      packages/db/dist/esm/utils/uuid.js449 B
      packages/db/dist/esm/virtual-props.js360 B
      compressed-size-action::db-package-size
    4. Size Change: 0 B

      Total Size: 7.97 kB

      ℹ️ View Unchanged
      FilenameSize
      packages/react-db/dist/esm/DbProvider.js317 B
      packages/react-db/dist/esm/HydrationBoundary.js263 B
      packages/react-db/dist/esm/index.js330 B
      packages/react-db/dist/esm/live-query-internals.js282 B
      packages/react-db/dist/esm/useLiveInfiniteQuery.js1.9 kB
      packages/react-db/dist/esm/useLiveQuery.js3.3 kB
      packages/react-db/dist/esm/useLiveQueryEffect.js355 B
      packages/react-db/dist/esm/useLiveSuspenseQuery.js820 B
      packages/react-db/dist/esm/usePacedMutations.js401 B
      compressed-size-action::react-db-package-size

Sign in to join this conversation.