Skip to content
Source
Sign inSign up
TanStack/db

feat: add expression helpers for parsing LoadSubsetOptions in queryFn #786

MergedKyleAMathews(GitHub profile, opens in a new tab)merged 14 commits intoquery-driven-sync fromrefs/pull/786/headmerged by samwillis(GitHub profile, opens in a new tab)

Summary

Adds reusable expression parsing utilities to help developers translate TanStack DB predicates (where, orderBy, limit) into their API's format when implementing query collections with predicate push-down.

Changes

New Expression Helpers (packages/query-db-collection/src/expression-helpers.ts)

  • parseWhereExpression: Parse where clauses with custom handlers for each operator
  • parseOrderByExpression: Parse order by into simple array format
  • extractSimpleComparisons: Extract simple AND-ed filters for basic use cases
  • parseLoadSubsetOptions: Convenience function to parse all options at once
  • walkExpression, extractFieldPath, extractValue: Lower-level helpers for custom parsing

Documentation

Added comprehensive section to docs/collections/query-collection.md:

  • How LoadSubsetOptions are passed via ctx.meta.loadSubsetOptions
  • Expression helper usage with REST API examples
  • GraphQL example
  • API reference for all helper functions
  • Query key builder pattern
  • Tips and best practices

Tests

35 test cases with 93.65% coverage:

  • All helper functions tested
  • Various operator combinations (eq, lt, gt, and, or, etc.)
  • Edge cases (null/undefined, nested expressions)
  • Integration test with complex real-world query

Why This Matters

Without these helpers, developers had to manually parse expression AST trees:

// Before - complex and error-prone
if (where && where.type === 'func' && where.name === 'eq') {
  const field = where.args[0].path[0]
  const value = where.args[1].value
  // ... more traversal logic
}

With helpers, it's straightforward:

// After - simple and maintainable
const filters = parseWhereExpression(where, {
  handlers: {
    eq: (field, value) => ({ [field.join('.')]: value }),
    lt: (field, value) => ({ [`${field.join('.')}_lt`]: value }),
    and: (...conditions) => Object.assign({}, ...conditions)
  }
})

Example Usage

queryFn: async (ctx) => {
  const { where, orderBy, limit } = ctx.meta.loadSubsetOptions

  // Parse into your API format
  const parsed = parseLoadSubsetOptions({ where, orderBy, limit })

  // Build API request
  const params = new URLSearchParams()
  parsed.filters.forEach(({ field, operator, value }) => {
    if (operator === 'eq') {
      params.set(field.join('.'), String(value))
    }
  })

  return fetch(`/api/products?${params}`).then(r => r.json())
}

🤖 Generated with Claude Code

    1. 🦋 Changeset detected

      Latest commit: d0377c67e2e519218aaaad9418d43585ce26d7a0

      The changes in this PR will be included in the next version bump.

      This PR includes changesets to release 11 packages
      NameType
      @tanstack/dbPatch
      @tanstack/query-db-collectionPatch
      @tanstack/angular-dbPatch
      @tanstack/electric-db-collectionPatch
      @tanstack/powersync-db-collectionPatch
      @tanstack/react-dbPatch
      @tanstack/rxdb-db-collectionPatch
      @tanstack/solid-dbPatch
      @tanstack/svelte-dbPatch
      @tanstack/trailbase-db-collectionPatch
      @tanstack/vue-dbPatch

      Not sure what this means? Click here to learn what changesets are.

      Click here if you're a maintainer who wants to add another changeset to this PR

    2. More templates

      @tanstack/angular-db

      npm i https://pkg.pr.new/@tanstack/angular-db@786

      @tanstack/db

      npm i https://pkg.pr.new/@tanstack/db@786

      @tanstack/db-ivm

      npm i https://pkg.pr.new/@tanstack/db-ivm@786

      @tanstack/electric-db-collection

      npm i https://pkg.pr.new/@tanstack/electric-db-collection@786

      @tanstack/offline-transactions

      npm i https://pkg.pr.new/@tanstack/offline-transactions@786

      @tanstack/powersync-db-collection

      npm i https://pkg.pr.new/@tanstack/powersync-db-collection@786

      @tanstack/query-db-collection

      npm i https://pkg.pr.new/@tanstack/query-db-collection@786

      @tanstack/react-db

      npm i https://pkg.pr.new/@tanstack/react-db@786

      @tanstack/rxdb-db-collection

      npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@786

      @tanstack/solid-db

      npm i https://pkg.pr.new/@tanstack/solid-db@786

      @tanstack/svelte-db

      npm i https://pkg.pr.new/@tanstack/svelte-db@786

      @tanstack/trailbase-db-collection

      npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@786

      @tanstack/vue-db

      npm i https://pkg.pr.new/@tanstack/vue-db@786

      commit: d0377c6

    3. Size Change: +1.4 kB (+1.67%)

      Total Size: 85.5 kB

      FilenameSizeChange
      ./packages/db/dist/esm/index.js2.63 kB+139 B (+5.58%)🔍
      ./packages/db/dist/esm/query/builder/functions.js733 B+127 B (+20.96%)🚨
      ./packages/db/dist/esm/query/expression-helpers.js1.14 kB+1.14 kB (new file)🆕
      ℹ️ View Unchanged
      FilenameSize
      ./packages/db/dist/esm/collection/change-events.js1.38 kB
      ./packages/db/dist/esm/collection/changes.js977 B
      ./packages/db/dist/esm/collection/events.js388 B
      ./packages/db/dist/esm/collection/index.js3.24 kB
      ./packages/db/dist/esm/collection/indexes.js1.1 kB
      ./packages/db/dist/esm/collection/lifecycle.js1.67 kB
      ./packages/db/dist/esm/collection/mutations.js2.26 kB
      ./packages/db/dist/esm/collection/state.js3.43 kB
      ./packages/db/dist/esm/collection/subscription.js2.42 kB
      ./packages/db/dist/esm/collection/sync.js2.12 kB
      ./packages/db/dist/esm/deferred.js207 B
      ./packages/db/dist/esm/errors.js4.11 kB
      ./packages/db/dist/esm/event-emitter.js748 B
      ./packages/db/dist/esm/indexes/auto-index.js742 B
      ./packages/db/dist/esm/indexes/base-index.js766 B
      ./packages/db/dist/esm/indexes/btree-index.js1.87 kB
      ./packages/db/dist/esm/indexes/lazy-index.js1.1 kB
      ./packages/db/dist/esm/indexes/reverse-index.js513 B
      ./packages/db/dist/esm/local-only.js837 B
      ./packages/db/dist/esm/local-storage.js2.04 kB
      ./packages/db/dist/esm/optimistic-action.js359 B
      ./packages/db/dist/esm/paced-mutations.js496 B
      ./packages/db/dist/esm/proxy.js3.22 kB
      ./packages/db/dist/esm/query/builder/index.js3.84 kB
      ./packages/db/dist/esm/query/builder/ref-proxy.js917 B
      ./packages/db/dist/esm/query/compiler/evaluators.js1.34 kB
      ./packages/db/dist/esm/query/compiler/expressions.js674 B
      ./packages/db/dist/esm/query/compiler/group-by.js1.8 kB
      ./packages/db/dist/esm/query/compiler/index.js1.96 kB
      ./packages/db/dist/esm/query/compiler/joins.js2 kB
      ./packages/db/dist/esm/query/compiler/order-by.js1.25 kB
      ./packages/db/dist/esm/query/compiler/select.js1.07 kB
      ./packages/db/dist/esm/query/ir.js673 B
      ./packages/db/dist/esm/query/live-query-collection.js360 B
      ./packages/db/dist/esm/query/live/collection-config-builder.js5.26 kB
      ./packages/db/dist/esm/query/live/collection-registry.js264 B
      ./packages/db/dist/esm/query/live/collection-subscriber.js1.77 kB
      ./packages/db/dist/esm/query/live/internal.js130 B
      ./packages/db/dist/esm/query/optimizer.js2.6 kB
      ./packages/db/dist/esm/query/predicate-utils.js2.88 kB
      ./packages/db/dist/esm/query/subset-dedupe.js921 B
      ./packages/db/dist/esm/scheduler.js1.21 kB
      ./packages/db/dist/esm/SortedMap.js1.18 kB
      ./packages/db/dist/esm/strategies/debounceStrategy.js237 B
      ./packages/db/dist/esm/strategies/queueStrategy.js418 B
      ./packages/db/dist/esm/strategies/throttleStrategy.js236 B
      ./packages/db/dist/esm/transactions.js2.9 kB
      ./packages/db/dist/esm/utils.js881 B
      ./packages/db/dist/esm/utils/browser-polyfills.js304 B
      ./packages/db/dist/esm/utils/btree.js5.61 kB
      ./packages/db/dist/esm/utils/comparison.js660 B
      ./packages/db/dist/esm/utils/index-optimization.js1.51 kB
      ./packages/db/dist/esm/utils/type-guards.js157 B
      compressed-size-action::db-package-size
    4. Size Change: 0 B

      Total Size: 3.34 kB

      ℹ️ View Unchanged
      FilenameSize
      ./packages/react-db/dist/esm/index.js225 B
      ./packages/react-db/dist/esm/useLiveInfiniteQuery.js1.17 kB
      ./packages/react-db/dist/esm/useLiveQuery.js1.11 kB
      ./packages/react-db/dist/esm/useLiveSuspenseQuery.js431 B
      ./packages/react-db/dist/esm/usePacedMutations.js401 B
      compressed-size-action::react-db-package-size
    5. packages/db/src/query/expression-helpers.ts lines 67 to 71
      @@ -0,0 +49,21 @@
      49+/**
      50+ * Options for customizing how WHERE expressions are parsed
      51+ */
      52+export interface ParseWhereOptions<T = any> {
      53+ /**
      54+ * Handler functions for different operators.
      55+ * Each handler receives the parsed field path(s) and value(s) and returns your custom format.
      56+ *
      57+ * Common operators:
      58+ * - eq: equality (=)
      59+ * - gt: greater than (>)
      60+ * - gte: greater than or equal (>=)
      61+ * - lt: less than (<)
      62+ * - lte: less than or equal (<=)
      63+ * - and: logical AND
      64+ * - or: logical OR
      65+ * - in: IN clause
      66+ */
      67+ handlers: {
      68+ [operator: string]: (...args: Array<any>) => T
      69+ }

      can we improve the type here such that it knows the operators that the IR supports?

      packages/query-db-collection/src/expression-helpers.ts line 38Outdated
      @@ -0,0 +18,21 @@
      18+ * lt: (field, value) => ({ [`${field}_lt`]: value }),
      19+ * and: (filters) => Object.assign({}, ...filters)
      20+ * })
      21+ *
      22+ * // Extract sort information
      23+ * const sort = parseOrderByExpression(orderBy)
      24+ *
      25+ * return api.getProducts({ ...filters, sort, limit })
      26+ * }
      27+ * ```
      28+ */
      29+
      30+import type { IR } from "@tanstack/db"
      31+
      32+type BasicExpression<T = any> = IR.BasicExpression<T>
      33+type OrderBy = IR.OrderBy
      34+
      35+/**
      36+ * Represents a simple field path extracted from an expression
      37+ */
      38+export type FieldPath = Array<string>

      We may want to support accessing elements in a array:

      export type FieldPath = Array<string | number>
      packages/db/src/query/expression-helpers.ts line 323
      @@ -0,0 +280,21 @@
      280+
      281+/**
      282+ * Extracts all simple comparisons from a WHERE expression.
      283+ * This is useful for simple APIs that only support basic filters.
      284+ *
      285+ * Note: This only works for simple AND-ed conditions. Complex OR/nested conditions
      286+ * will require using parseWhereExpression with custom handlers.
      287+ *
      288+ * @param expr - The WHERE expression to parse
      289+ * @returns Array of simple comparisons
      290+ *
      291+ * @example
      292+ * ```typescript
      293+ * const comparisons = extractSimpleComparisons(where)
      294+ * // Returns: [
      295+ * // { field: ['category'], operator: 'eq', value: 'electronics' },
      296+ * // { field: ['price'], operator: 'lt', value: 100 }
      297+ * // ]
      298+ * ```
      299+ */
      300+export function extractSimpleComparisons(

      We may want this to throw if if finds something it can't extract such as an or

      packages/query-db-collection/src/expression-helpers.ts lines 273 to 277Outdated
      @@ -0,0 +257,21 @@
      257+export function parseOrderByExpression(
      258+ orderBy: OrderBy | undefined | null
      259+): Array<ParsedOrderBy> {
      260+ if (!orderBy || orderBy.length === 0) {
      261+ return []
      262+ }
      263+
      264+ return orderBy.map((clause: IR.OrderByClause) => {
      265+ const field = extractFieldPath(clause.expression)
      266+
      267+ if (!field) {
      268+ throw new Error(
      269+ `ORDER BY expression must be a field reference, got: ${clause.expression.type}`
      270+ )
      271+ }
      272+
      273+ return {
      274+ field,
      275+ direction: clause.compareOptions.direction,
      276+ nulls: clause.compareOptions.nulls,
      277+ }

      we need to return the local / string comparison options too

    6. Mostly there except for the above comments. I will rebase #773 on this and use it.

    7. @KyleAMathews there are a few tests failing too, looks like just types

    8. samwillis(GitHub profile, opens in a new tab) approved these changes

      Looks great. I'll merge it into #763

    9. Cool! BTW, I've been thinking we'll probably want to move something like these utilities into a shared package as other collections will surely want to do something similar. The babel of the IR haha.

Sign in to join this conversation.