eagerly run live queries while collections are loading #658
main fromrefs/pull/658/head·merged by samwillis(GitHub profile, opens in a new tab) Summary#
Refactors the live query execution model to support eager materialization. Live queries now process and display data as it arrives from source collections during sync, rather than waiting for all source collections to reach a "ready" state before executing.
Key Changes#
- Eager Execution: Live queries now run immediately when triggered via
.preload()or.startSync(), processing data as it arrives from source collections - Separated Status & Data Management: Status updates now use
on('status:change')event listeners, whilesubscribeChangesis used solely for data updates - Error Propagation: Live queries automatically transition to error state when source collections error or are manually cleaned up
- Optimistic Transactions: Local optimistic mutations are immediately visible in live queries even while source collections are still in "loading" state, with proper rollback behavior when mutations aren't persisted via sync
- Lifecycle Simplification: Removed
initialCommitstatus from collection lifecycle across entire codebase - Auto-indexing: Enabled auto-indexing during
loadingstate to support eager query execution - Public API: Made
_lifecycleproperty public on Collection interface for internal library use - Framework Updates: Updated all framework adapters (React, Vue, Solid, Svelte, Angular) to remove
initialCommitreferences and validate eager execution behavior
Testing#
Core Package (@tanstack/db):
- Added comprehensive test suite for eager execution in
query-while-syncing.test.ts(24 tests covering basic queries, joins, multiple sources, error handling, and both.startSync()and.preload()patterns) - Added test validating optimistic transaction visibility and rollback behavior during loading state
- Added tests validating multiple sync transactions before
markReady()in collection tests - Added tests for change event emission during multiple sync transactions
- Refactored tests to use modern vitest
describe.eachpattern for parameterized testing with bothautoIndex: 'off'andautoIndex: 'eager' - All tests use fake timers for faster, more deterministic execution
Framework Packages:
- Added eager execution tests to all framework packages validating that live queries show data while
isLoading == true:- React (
@tanstack/react-db): 4 new tests (33 total) - Vue (
@tanstack/vue-db): 4 new tests (23 total) - Solid (
@tanstack/solid-db): 4 new tests (20 total) - Svelte (
@tanstack/svelte-db): 4 new tests (22 total) - Angular (
@tanstack/angular-db): 3 new tests (18 total)
- React (
- Each framework package now validates:
- State visibility during sync while
isLoading == true - Filtered query results during sync
- Join query results during sync (where applicable)
- Status updates when source is marked ready with no data
- State visibility during sync while
- Fixed all
initialCommitreferences in framework adapter implementations
Test Coverage:
- Total: 1,531 tests pass across all packages (up from 1,281)
- Added 19 framework-specific tests validating eager execution
- Added 24 core package tests for eager execution scenarios
User-Facing Impact#
Users will see live query results update in real-time as data loads, providing immediate feedback during sync operations. This includes both synced data and local optimistic updates, creating a more responsive user experience even before the underlying data is fully loaded. They can use the collection status to show loading state while the data streams in. This behavior is consistent across all framework adapters (React, Vue, Solid, Svelte, Angular).
- 1
More templates
@tanstack/angular-db
npm i https://pkg.pr.new/@tanstack/angular-db@658@tanstack/db
npm i https://pkg.pr.new/@tanstack/db@658@tanstack/db-ivm
npm i https://pkg.pr.new/@tanstack/db-ivm@658@tanstack/electric-db-collection
npm i https://pkg.pr.new/@tanstack/electric-db-collection@658@tanstack/query-db-collection
npm i https://pkg.pr.new/@tanstack/query-db-collection@658@tanstack/react-db
npm i https://pkg.pr.new/@tanstack/react-db@658@tanstack/rxdb-db-collection
npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@658@tanstack/solid-db
npm i https://pkg.pr.new/@tanstack/solid-db@658@tanstack/svelte-db
npm i https://pkg.pr.new/@tanstack/svelte-db@658@tanstack/trailbase-db-collection
npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@658@tanstack/vue-db
npm i https://pkg.pr.new/@tanstack/vue-db@658commit:
2816a73Size Change: +222 B (+0.29%)
Total Size: 76.3 kB
Filename Size Change ./packages/db/dist/esm/collection/lifecycle.js1.8 kB -22 B (-1.21%) ./packages/db/dist/esm/collection/state.js3.78 kB -43 B (-1.12%) ./packages/db/dist/esm/collection/sync.js1.63 kB -21 B (-1.27%) ./packages/db/dist/esm/indexes/auto-index.js794 B -34 B (-4.11%) ./packages/db/dist/esm/query/live/collection-config-builder.js3.03 kB +342 B (+12.74%) ⚠️ compressed-size-action::db-package-sizeℹ️ View Unchanged
Filename Size ./packages/db/dist/esm/collection/change-events.js963 B ./packages/db/dist/esm/collection/changes.js1.01 kB ./packages/db/dist/esm/collection/events.js660 B ./packages/db/dist/esm/collection/index.js3.31 kB ./packages/db/dist/esm/collection/indexes.js1.16 kB ./packages/db/dist/esm/collection/mutations.js2.52 kB ./packages/db/dist/esm/collection/subscription.js1.83 kB ./packages/db/dist/esm/deferred.js230 B ./packages/db/dist/esm/errors.js3.1 kB ./packages/db/dist/esm/index.js1.58 kB ./packages/db/dist/esm/indexes/base-index.js835 B ./packages/db/dist/esm/indexes/btree-index.js2 kB ./packages/db/dist/esm/indexes/lazy-index.js1.21 kB ./packages/db/dist/esm/indexes/reverse-index.js577 B ./packages/db/dist/esm/local-only.js827 B ./packages/db/dist/esm/local-storage.js2.02 kB ./packages/db/dist/esm/optimistic-action.js294 B ./packages/db/dist/esm/proxy.js3.86 kB ./packages/db/dist/esm/query/builder/functions.js615 B ./packages/db/dist/esm/query/builder/index.js4.04 kB ./packages/db/dist/esm/query/builder/ref-proxy.js938 B ./packages/db/dist/esm/query/compiler/evaluators.js1.55 kB ./packages/db/dist/esm/query/compiler/expressions.js760 B ./packages/db/dist/esm/query/compiler/group-by.js2.04 kB ./packages/db/dist/esm/query/compiler/index.js2.04 kB ./packages/db/dist/esm/query/compiler/joins.js2.52 kB ./packages/db/dist/esm/query/compiler/order-by.js1.21 kB ./packages/db/dist/esm/query/compiler/select.js1.28 kB ./packages/db/dist/esm/query/ir.js785 B ./packages/db/dist/esm/query/live-query-collection.js340 B ./packages/db/dist/esm/query/live/collection-subscriber.js1.92 kB ./packages/db/dist/esm/query/optimizer.js3.08 kB ./packages/db/dist/esm/SortedMap.js1.24 kB ./packages/db/dist/esm/transactions.js3 kB ./packages/db/dist/esm/utils.js1.01 kB ./packages/db/dist/esm/utils/browser-polyfills.js365 B ./packages/db/dist/esm/utils/btree.js6.01 kB ./packages/db/dist/esm/utils/comparison.js754 B ./packages/db/dist/esm/utils/index-optimization.js1.73 kB Size Change: 0 B
Total Size: 1.47 kB
compressed-size-action::react-db-package-sizeℹ️ View Unchanged
Filename Size ./packages/react-db/dist/esm/index.js152 B ./packages/react-db/dist/esm/useLiveQuery.js1.32 kB KyleAMathews(GitHub profile, opens in a new tab) approved these changes
Great stuff! I don't think this would show up as improvement in our current benchmarks but for anything that does chunked loading like electric, this should be quite a bit faster.
samwillis(GitHub profile, opens in a new tab) closed this
❤️ works like a charm - closing #641
Sign in to join this conversation.
🦋 Changeset detected#
Latest commit: 2816a73dba13792c45eb2ba0c98edf5c7e1ca43d
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 12 packages
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR