Skip to content
Source
Sign inSign up
TanStack/db

Live query collections can't take autoIndex / defaultIndexType, so an ordered query over one gets no index #2292

Send to my agent
ClosedJaime02(GitHub profile, opens in a new tab) opened this issue 1 comment

Problem

A live query collection used as the source of another query (as in "Caching Intermediate Results") has no indexes. LiveQueryCollectionConfig doesn't accept autoIndex or defaultIndexType, and CollectionConfigBuilder.getConfig() doesn't forward them. So an orderBy().limit() over it warns on every compile:

[TanStack DB] [live-query-1] orderBy with limit requires an index on "total" for efficient lazy loading. Consider collection.createIndex((row) => row.total) or autoIndex: 'eager' with a defaultIndexType.

The only fix is calling createIndex by hand on the collection useLiveQuery returns:

  • during render;
  • guarded against duplicates;
  • with each orderBy's compare options copied by hand (for example nulls: 'last').

Any sort not foreseen still warns.

Proposal

Accept autoIndex and defaultIndexType in LiveQueryCollectionConfig (and the useLiveQuery config) and forward them to the collection, so a downstream orderBy builds its matching index the way it does on base collections. Inheriting them from the source collections, or a DbClient default, would also work.

const base = createLiveQueryCollection({ query: expensive, autoIndex: 'eager', defaultIndexType: BasicIndex })

Benchmark

Setup: 5,000 orders, 20,000 items. The expensive part (a filter plus a grouped sum joined to every order) is cached as its own live query collection, and the page query over it is rebuilt for each of 10 page changes, pages of 20. @tanstack/db 0.15.1, Bun, 3 runs.

Source collectionFirst pageEach next pageIndex warnings
No index (bun bench.ts chained)300–410 ms42–59 ms20 (2 per page)
Manual createIndex (bun bench.ts chainedIndex)307–388 ms8.8–9.9 ms0
bench.ts
// bun bench.ts rebuild | setWindow | chained | chainedIndex
import { BasicIndex, createCollection, createLiveQueryCollection, eq, localOnlyCollectionOptions, sum } from '@tanstack/db'

const indexed = { autoIndex: 'eager', defaultIndexType: BasicIndex } as const
const orders = createCollection(localOnlyCollectionOptions({ ...indexed, getKey: (o: any) => o.id,
  initialData: Array.from({ length: 5000 }, (_, i) => ({ id: i, status: i % 3 ? 'open' : 'closed' })) }))
const items = createCollection(localOnlyCollectionOptions({ ...indexed, getKey: (i: any) => i.id,
  initialData: Array.from({ length: 20000 }, (_, i) => ({ id: i, orderId: i % 5000, amount: (i * 7919) % 1000 })) }))

// Expensive part: filter + grouped aggregate joined to every order
const filtered = (q: any) => q.from({ order: orders })
  .join({ t: q.from({ item: items }).groupBy(({ item }: any) => item.orderId)
    .select(({ item }: any) => ({ orderId: item.orderId, total: sum(item.amount) })) },
    ({ order, t }: any) => eq(order.id, t.orderId))
  .where(({ order }: any) => eq(order.status, 'open'))
  .select(({ order, t }: any) => ({ id: order.id, total: t.total }))
const page = (q: any, from: any, offset: number) => q.from({ row: from })
  .orderBy(({ row }: any) => row.total, 'desc').orderBy(({ row }: any) => row.id).limit(20).offset(offset)
const live = async (query: any) => {
  const c = createLiveQueryCollection({ startSync: true, query })
  await c.toArrayWhenReady()
  return c as any
}

const mode = process.argv[2]
let warnings = 0
console.warn = () => warnings++
const times: Array<number> = []
const timed = async (fn: () => Promise<unknown>) => { const t = performance.now(); await fn(); times.push(performance.now() - t) }

if (mode === 'rebuild') {
  // What useLiveQuery does when the page is part of the queryKey
  for (let p = 0; p < 10; p++) await timed(async () => (await live((q: any) => page(q, filtered(q), p * 20))).cleanup())
} else if (mode === 'setWindow') {
  let c: any
  await timed(async () => { c = await live((q: any) => page(q, filtered(q), 0)) })
  for (let p = 1; p < 10; p++) await timed(async () => { const r = c.utils.setWindow({ offset: p * 20, limit: 20 }); if (r !== true) await r })
} else {
  // Base cached as its own live query collection, the page query over it is rebuilt per page
  let base: any
  await timed(async () => {
    base = await live(filtered)
    if (mode === 'chainedIndex') base.createIndex((row: any) => row.total, { indexType: BasicIndex })
  })
  for (let p = 0; p < 10; p++) await timed(async () => (await live((q: any) => page(q, base, p * 20))).cleanup())
}
const next = times.slice(-9)
console.log(mode, `first ${times[0]!.toFixed(1)} ms`, `next pages avg ${(next.reduce((a, b) => a + b) / next.length).toFixed(1)} ms`, `index warnings ${warnings}`)

Related: #2190 / #2196 (same warning when compare options differ; here no index can be configured at all), #2179 (adds an option to the live query config; these would sit beside it)

  1. 🎉 The fix for this issue has been released!

    Fixing PRs: https://github.com/TanStack/db/pull/2314

    Backfilled notification for published release candidates.

Sign in to join this conversation.