feat(query-db): support eager initial data #1683
main fromrefs/pull/1683/head·merged by KyleAMathews(GitHub profile, opens in a new tab) Adds typed, collection-local initialData and initialDataUpdatedAt support to eager Query Collections. Initial Query responses now materialize immediately as collection rows while retaining TanStack Query's cache identity, freshness, and wrapped-response semantics.
Root Cause#
Query Collections already materialized data preloaded into a QueryClient, but they could not declare initial data per collection. Applications sharing one client across many collections therefore had to choose between overly broad client defaults and imperative cache seeding. Forwarding defaults unchanged was also unsafe for on-demand subsets and placeholder data: neither has collection-wide row-membership semantics.
Approach#
- Add typed top-level
initialDataandinitialDataUpdatedAtoptions using the original Query response type. - Forward those options only for eager observers, then reuse the existing
select, materialization, and row-ownership pipeline. - Reject explicit collection-level initial data in on-demand mode and suppress client-wide initial-data defaults for derived subset observers.
- Suppress
placeholderDatadefaults because placeholder results are observer-local UI state, not Query cache data suitable for normalized rows. - Document the authority, lifecycle, wrapped-response, write, ownership, and persistence semantics.
Key Invariants#
- The Query cache remains authoritative; existing cached or hydrated data wins over a later initializer.
- Exact Query keys identify shared documents, even when collections share a
QueryClient. selectmaterializes rows without replacing the original wrapped response in the Query cache.- Stale initial rows remain available during refetch and reconcile through normal ownership rules.
- Placeholder data and collection-wide initializers never become rows for arbitrary on-demand subsets.
- Persistence needs no new format or temporary-row metadata.
Non-goals#
- Materializing
placeholderDataas DB rows. - Defining a subset-aware initializer for on-demand collections.
- Giving each collection a private cache document when it uses the same exact Query key.
- Changing direct-write or persistence formats.
Trade-offs#
On-demand mode rejects explicit initial-data options rather than guessing subset membership. Callers that know an exact derived key can still seed or hydrate that Query cache entry. Eager mode suppresses placeholder defaults to keep observer presentation state out of the collection's shared normalized state.
Verification#
pnpm vitest run packages/query-db-collection/tests/query.test.ts --maxWorkers=1
pnpm --filter @tanstack/query-db-collection build
pnpm --filter @tanstack/query-db-collection lint
git diff --check
The focused Query Collection suite covers fresh and stale initial data, function initializers, wrapped response projection, shared clients and exact keys, existing-cache precedence, on-demand rejection/default suppression, and placeholder-default suppression. Lint passes with existing warnings.
Files changed#
packages/query-db-collection/src/query.ts— adds the public options and observer initialization policy.packages/query-db-collection/src/errors.ts— adds the on-demand configuration error.packages/query-db-collection/tests/query.test.ts— adds initialization and isolation coverage.docs/collections/query-collection.md— documents usage and user-visible semantics.docs/collections/query-initial-placeholder-data-design.md— records authority, ownership, transitions, writes, and persistence decisions..changeset/young-cats-initialize.md— records the minor package feature.
Summary by CodeRabbit#
New Features
- Added eager collection support for TanStack Query’s
initialDataandinitialDataUpdatedAt. - Initial data is immediately available as collection rows while preserving wrapped response data.
- Fresh initial data can prevent an unnecessary fetch; stale data is refreshed and reconciled with server results.
- Added safeguards so placeholder data is not materialized as collection rows.
- Added clear errors when initial data is used with on-demand collections.
- Added eager collection support for TanStack Query’s
Documentation
- Documented initial data behavior, cache precedence, wrapped responses, and placeholder data limitations.
- 1
More templates
@tanstack/angular-db
npm i https://pkg.pr.new/@tanstack/angular-db@1683@tanstack/browser-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/browser-db-sqlite-persistence@1683@tanstack/capacitor-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/capacitor-db-sqlite-persistence@1683@tanstack/cloudflare-durable-objects-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/cloudflare-durable-objects-db-sqlite-persistence@1683@tanstack/db
npm i https://pkg.pr.new/@tanstack/db@1683@tanstack/db-ivm
npm i https://pkg.pr.new/@tanstack/db-ivm@1683@tanstack/db-sqlite-persistence-core
npm i https://pkg.pr.new/@tanstack/db-sqlite-persistence-core@1683@tanstack/electric-db-collection
npm i https://pkg.pr.new/@tanstack/electric-db-collection@1683@tanstack/electron-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/electron-db-sqlite-persistence@1683@tanstack/expo-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/expo-db-sqlite-persistence@1683@tanstack/node-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/node-db-sqlite-persistence@1683@tanstack/offline-transactions
npm i https://pkg.pr.new/@tanstack/offline-transactions@1683@tanstack/powersync-db-collection
npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1683@tanstack/query-db-collection
npm i https://pkg.pr.new/@tanstack/query-db-collection@1683@tanstack/react-db
npm i https://pkg.pr.new/@tanstack/react-db@1683@tanstack/react-native-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/react-native-db-sqlite-persistence@1683@tanstack/rxdb-db-collection
npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1683@tanstack/solid-db
npm i https://pkg.pr.new/@tanstack/solid-db@1683@tanstack/svelte-db
npm i https://pkg.pr.new/@tanstack/svelte-db@1683@tanstack/tauri-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/tauri-db-sqlite-persistence@1683@tanstack/trailbase-db-collection
npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1683@tanstack/vue-db
npm i https://pkg.pr.new/@tanstack/vue-db@1683commit:
b187e6bSize Change: 0 B
Total Size: 125 kB
compressed-size-action::db-package-sizeℹ️ View Unchanged
Filename Size packages/db/dist/esm/collection/change-events.js1.43 kB packages/db/dist/esm/collection/changes.js1.38 kB packages/db/dist/esm/collection/cleanup-queue.js810 B packages/db/dist/esm/collection/events.js434 B packages/db/dist/esm/collection/index.js3.62 kB packages/db/dist/esm/collection/indexes.js1.99 kB packages/db/dist/esm/collection/lifecycle.js1.69 kB packages/db/dist/esm/collection/mutations.js2.47 kB packages/db/dist/esm/collection/state.js5.48 kB packages/db/dist/esm/collection/subscription.js3.74 kB packages/db/dist/esm/collection/sync.js2.88 kB packages/db/dist/esm/collection/transaction-metadata.js144 B packages/db/dist/esm/deferred.js207 B packages/db/dist/esm/errors.js5.1 kB packages/db/dist/esm/event-emitter.js748 B packages/db/dist/esm/index.js3.16 kB packages/db/dist/esm/indexes/auto-index.js829 B packages/db/dist/esm/indexes/base-index.js767 B packages/db/dist/esm/indexes/basic-index.js2.06 kB packages/db/dist/esm/indexes/btree-index.js2.19 kB packages/db/dist/esm/indexes/index-registry.js820 B packages/db/dist/esm/indexes/reverse-index.js557 B packages/db/dist/esm/live-query-adapter.js318 B packages/db/dist/esm/local-only.js916 B packages/db/dist/esm/local-storage.js2.12 kB packages/db/dist/esm/optimistic-action.js359 B packages/db/dist/esm/paced-mutations.js496 B packages/db/dist/esm/proxy.js3.75 kB packages/db/dist/esm/query/builder/functions.js1.47 kB packages/db/dist/esm/query/builder/index.js5.84 kB packages/db/dist/esm/query/builder/ref-proxy.js1.24 kB packages/db/dist/esm/query/compiler/evaluators.js1.89 kB packages/db/dist/esm/query/compiler/expressions.js430 B packages/db/dist/esm/query/compiler/group-by.js3.56 kB packages/db/dist/esm/query/compiler/index.js6.67 kB packages/db/dist/esm/query/compiler/joins.js2.5 kB packages/db/dist/esm/query/compiler/lazy-targets.js923 B packages/db/dist/esm/query/compiler/order-by.js1.74 kB packages/db/dist/esm/query/compiler/select.js1.53 kB packages/db/dist/esm/query/effect.js4.77 kB packages/db/dist/esm/query/expression-helpers.js1.43 kB packages/db/dist/esm/query/ir.js1.25 kB packages/db/dist/esm/query/live-query-collection.js360 B packages/db/dist/esm/query/live/collection-config-builder.js9.1 kB packages/db/dist/esm/query/live/collection-registry.js264 B packages/db/dist/esm/query/live/collection-subscriber.js1.93 kB packages/db/dist/esm/query/live/internal.js145 B packages/db/dist/esm/query/live/utils.js1.81 kB packages/db/dist/esm/query/optimizer.js2.92 kB packages/db/dist/esm/query/predicate-utils.js2.97 kB packages/db/dist/esm/query/query-once.js359 B packages/db/dist/esm/query/subset-dedupe.js960 B packages/db/dist/esm/scheduler.js1.3 kB packages/db/dist/esm/SortedMap.js1.3 kB packages/db/dist/esm/strategies/debounceStrategy.js247 B packages/db/dist/esm/strategies/queueStrategy.js428 B packages/db/dist/esm/strategies/throttleStrategy.js246 B packages/db/dist/esm/transactions.js3.04 kB packages/db/dist/esm/utils.js927 B packages/db/dist/esm/utils/array-utils.js273 B packages/db/dist/esm/utils/browser-polyfills.js304 B packages/db/dist/esm/utils/btree.js5.61 kB packages/db/dist/esm/utils/comparison.js1.11 kB packages/db/dist/esm/utils/cursor.js457 B packages/db/dist/esm/utils/index-optimization.js2.39 kB packages/db/dist/esm/utils/type-guards.js157 B packages/db/dist/esm/utils/uuid.js449 B packages/db/dist/esm/virtual-props.js360 B Size Change: 0 B
Total Size: 4.22 kB
compressed-size-action::react-db-package-sizeℹ️ View Unchanged
Filename Size packages/react-db/dist/esm/index.js249 B packages/react-db/dist/esm/useLiveInfiniteQuery.js1.32 kB packages/react-db/dist/esm/useLiveQuery.js1.33 kB packages/react-db/dist/esm/useLiveQueryEffect.js355 B packages/react-db/dist/esm/useLiveSuspenseQuery.js567 B packages/react-db/dist/esm/usePacedMutations.js401 B Size Change: 0 B
Total Size: 125 kB
compressed-size-action::db-package-sizeℹ️ View Unchanged
Filename Size packages/db/dist/esm/collection/change-events.js1.43 kB packages/db/dist/esm/collection/changes.js1.38 kB packages/db/dist/esm/collection/cleanup-queue.js810 B packages/db/dist/esm/collection/events.js434 B packages/db/dist/esm/collection/index.js3.62 kB packages/db/dist/esm/collection/indexes.js1.99 kB packages/db/dist/esm/collection/lifecycle.js1.69 kB packages/db/dist/esm/collection/mutations.js2.47 kB packages/db/dist/esm/collection/state.js5.48 kB packages/db/dist/esm/collection/subscription.js3.74 kB packages/db/dist/esm/collection/sync.js2.88 kB packages/db/dist/esm/collection/transaction-metadata.js144 B packages/db/dist/esm/deferred.js207 B packages/db/dist/esm/errors.js5.1 kB packages/db/dist/esm/event-emitter.js748 B packages/db/dist/esm/index.js3.16 kB packages/db/dist/esm/indexes/auto-index.js829 B packages/db/dist/esm/indexes/base-index.js767 B packages/db/dist/esm/indexes/basic-index.js2.06 kB packages/db/dist/esm/indexes/btree-index.js2.19 kB packages/db/dist/esm/indexes/index-registry.js820 B packages/db/dist/esm/indexes/reverse-index.js557 B packages/db/dist/esm/live-query-adapter.js318 B packages/db/dist/esm/local-only.js916 B packages/db/dist/esm/local-storage.js2.12 kB packages/db/dist/esm/optimistic-action.js359 B packages/db/dist/esm/paced-mutations.js496 B packages/db/dist/esm/proxy.js3.75 kB packages/db/dist/esm/query/builder/functions.js1.47 kB packages/db/dist/esm/query/builder/index.js5.84 kB packages/db/dist/esm/query/builder/ref-proxy.js1.24 kB packages/db/dist/esm/query/compiler/evaluators.js1.89 kB packages/db/dist/esm/query/compiler/expressions.js430 B packages/db/dist/esm/query/compiler/group-by.js3.56 kB packages/db/dist/esm/query/compiler/index.js6.67 kB packages/db/dist/esm/query/compiler/joins.js2.5 kB packages/db/dist/esm/query/compiler/lazy-targets.js923 B packages/db/dist/esm/query/compiler/order-by.js1.74 kB packages/db/dist/esm/query/compiler/select.js1.53 kB packages/db/dist/esm/query/effect.js4.77 kB packages/db/dist/esm/query/expression-helpers.js1.43 kB packages/db/dist/esm/query/ir.js1.25 kB packages/db/dist/esm/query/live-query-collection.js360 B packages/db/dist/esm/query/live/collection-config-builder.js9.1 kB packages/db/dist/esm/query/live/collection-registry.js264 B packages/db/dist/esm/query/live/collection-subscriber.js1.93 kB packages/db/dist/esm/query/live/internal.js145 B packages/db/dist/esm/query/live/utils.js1.81 kB packages/db/dist/esm/query/optimizer.js2.92 kB packages/db/dist/esm/query/predicate-utils.js2.97 kB packages/db/dist/esm/query/query-once.js359 B packages/db/dist/esm/query/subset-dedupe.js960 B packages/db/dist/esm/scheduler.js1.3 kB packages/db/dist/esm/SortedMap.js1.3 kB packages/db/dist/esm/strategies/debounceStrategy.js247 B packages/db/dist/esm/strategies/queueStrategy.js428 B packages/db/dist/esm/strategies/throttleStrategy.js246 B packages/db/dist/esm/transactions.js3.04 kB packages/db/dist/esm/utils.js927 B packages/db/dist/esm/utils/array-utils.js273 B packages/db/dist/esm/utils/browser-polyfills.js304 B packages/db/dist/esm/utils/btree.js5.61 kB packages/db/dist/esm/utils/comparison.js1.11 kB packages/db/dist/esm/utils/cursor.js457 B packages/db/dist/esm/utils/index-optimization.js2.39 kB packages/db/dist/esm/utils/type-guards.js157 B packages/db/dist/esm/utils/uuid.js449 B packages/db/dist/esm/virtual-props.js360 B Size Change: 0 B
Total Size: 4.22 kB
compressed-size-action::react-db-package-sizeℹ️ View Unchanged
Filename Size packages/react-db/dist/esm/index.js249 B packages/react-db/dist/esm/useLiveInfiniteQuery.js1.32 kB packages/react-db/dist/esm/useLiveQuery.js1.33 kB packages/react-db/dist/esm/useLiveQueryEffect.js355 B packages/react-db/dist/esm/useLiveSuspenseQuery.js567 B packages/react-db/dist/esm/usePacedMutations.js401 B @KyleAMathews i had Fable review the PR and chatted through it with Fable. This is what we landed on:
Verdict: approve the architecture; a few things worth addressing before merge — one real risk (persistence interplay), one changeset omission, and some test gaps relative to the PR's own design doc. CI is fully green (unit, E2E, example build, lint/autofix).
Issue 1 — persistence interplay can clobber newer persisted rows (main risk)#
On restart with SQLite persistence, the Query cache is empty but persisted rows exist under
until-revalidatedretention (retainedQueriesPendingRevalidation). WithinitialDataconfigured:- Observer creation seeds the cache;
subscribeToQueryimmediately fires a success result. handleQueryResultsees the pending-revalidation flag and callsreconcileSuccessfulResult(query.ts:1534), which reconciles the persisted baseline against the static seed — deleting persisted rows the seed omits and clearing the retention flag (query.ts:1494).- Crucially, if
initialDataUpdatedAtis omitted, query-core stampsdataUpdatedAt = Date.now(), so the seed always looks fresh and no refetch follows for the duration ofstaleTime.
So a retention mechanism meaning "keep these rows until the server revalidates them" is now satisfiable by static config data with no network involved, silently regressing the user's last-seen state. Mechanically it's "same as a stale server response," but there's no server in the loop. Suggestions:
- At minimum: a persistence +
initialDatatest (the design doc's own step 5 promises exactly this) and a docs warning to always setinitialDataUpdatedAtwhen persistence is enabled. - Worth considering: when a retained query's success result was never fetched (
query.state.dataUpdateCount === 0/isFetchedAfterMount === false), trigger a revalidating refetch instead of treating the seed as revalidation.
Issue 2 — changeset omits two silent behavior changes#
The changeset only advertises the new feature, but the PR also changes existing behavior for collections that never opt in:
- Client-default
placeholderDatano longer materializes (for all query collections). - Client-default
initialDatano longer seeds on-demand subset observers.
Both are deliberate correctness fixes, but they remove rows an app may currently be (mis)relying on. They should be called out in the changeset so the release notes warn anyone affected.
Issue 3 — test coverage vs. the PR's own claims#
Covered well: fresh/stale seeds, function-initializer-evaluated-once, wrapped envelope preserved in cache, shared-client scoping, existing-cache precedence, on-demand rejection and default suppression, placeholder suppression.
Missing relative to the design doc's behavior table and its own implementation sequence (steps 3–5):
- Refetch error retains initial rows — explicitly promised in the docs ("an error retains the initial rows") but untested; cheap to add.
- Direct writes (
writeInsert/writeUpdate) against a wrapped initial envelope. - Cleanup/late-notification and the persistence scenarios above.
Given that, "Closes #346" is a stretch by the design doc's own sequencing ("close or narrow #346 only after these contracts ship" — step 6). Suggest "Advances #346" unless the error-retention and persistence tests land in this PR.
Minor#
docs/collections/query-initial-placeholder-data-design.mdis internal design material in the published docs folder; it's not registered indocs/config.jsonso it won't render in the site nav. Consider moving it out ofdocs/(e.g., anrfcs/location) or linking it from RFC #1643 instead.
- Observer creation seeds the cache;
KyleAMathews(GitHub profile, opens in a new tab) closed this
Sign in to join this conversation.
No actionable comments were generated in the recent review. 🎉
ℹ️ Recent review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID:
711bbc83-cf04-4fad-b8da-eac96f612650📥 Commits
Reviewing files that changed from the base of the PR and between 5b3fd1ddf267a0e50c634d87d05fd90d78e57867 and b187e6be360d0e1ab204e3545d4f072a4a80a525.
📒 Files selected for processing (4)
.changeset/young-cats-initialize.mddocs/collections/query-initial-placeholder-data-design.mdpackages/query-db-collection/src/query.tspackages/query-db-collection/tests/query.test.ts🚧 Files skipped from review as they are similar to previous changes (2)
📝 Walkthrough
Walkthrough#
Query Collections now support eager
initialDataandinitialDataUpdatedAt, including wrapped-response projection throughselect. On-demand collections reject collection-level initializers, while placeholder data is prevented from becoming normalized collection rows.Changes#
Eager initial data support
docs/collections/query-initial-placeholder-data-design.md,docs/collections/query-collection.mdpackages/query-db-collection/src/errors.ts,packages/query-db-collection/src/query.tspackages/query-db-collection/tests/query.test.ts,.changeset/young-cats-initialize.mdEstimated code review effort: 3 (Moderate) | ~25 minutes
Sequence Diagram(s)#
Possibly related PRs
Suggested reviewers:
kevin-dp🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)#
✅ Passed checks (4 passed)
#346.✨ Finishing Touches
📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning
There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.
🔧 ESLint
ESLint install timed out. The project may have too many dependencies for the sandbox.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
❤️ Share