Skip to content
Source
Sign inSign up
TanStack/db

feat(db): publish completed query snapshots #2010

Openalec-watts(GitHub profile, opens in a new tab)wants to merge 6 commits intomain fromrefs/pull/2010/headopened

Queries over persisted on-demand sources can have locally available rows while their required subsets are still loading. This proposal lets applications persist a completed live-query result, restore it, and replace it atomically once the current sources and reachable demands complete—including an empty result.

This reworks the draft following Kyle's feedback. The earlier publishUnconfirmedOrderedResults option is removed.

const result = createCollection(persistedCollectionOptions({
  ...liveQueryCollectionOptions({
    id: 'versioned-query-and-user-scope',
    snapshotPublication: 'complete',
    getKey: row => row.id,
    query: q => q.from({ row: source }).select(({ row }) => row),
  }),
  persistence,
  schemaVersion: 1,
  syncMode: 'eager', // Restore the output; inputs can remain on-demand.
  initialRender: { strategy: 'network-first', networkTimeoutMs: 0 },
}))

Behavior

  • The opt-in policy retains the previous completed result while current sources or reachable demands are unsettled. The first completed publication truncates restored output and installs the replacement in one sync transaction.
  • Later changes that open new demands also retain the completed result until their demands settle. Already-covered updates publish normally. This prevents a later partial join from overwriting the saved completed view.
  • Completion metadata is saved with the rows. A new empty database or a mismatched saved window is not an eligible completed snapshot.
  • A persisted complete query supplies its own restore boundary. A direct peer reading one of its sources still has its independent restore obligation. Network errors remain distinguishable from derived-query errors; a zero-delay fallback cannot conceal an already-failed query.
  • Collection readiness and preload() still wait for current provider coverage. Default progressive publication is unchanged.

Scope and tradeoffs

This composes the existing persistence wrapper, metadata, query graph, demand tracking and Collection transaction boundary. It adds no second persistence backend, query cache registry or recovery state machine.

Callers must provide a stable getKey, an id/schema version covering the query definition, parameters, selected shape and user scope, and a durable row comparator for ordered persisted output. Graph-local order indices and Collection-valued include facades are not portable cache data. The guide contains an ordered example.

It saves additional result rows and still fetches from the provider. Fresh data, including edits that need new demand, can wait for the slowest dependency. Changed server data can cause one final visible replacement. This is not a cross-provider database transaction or a universal no-flashing guarantee. Cold queries gain no cached-result shortcut.

Verification

  • Pinned, frozen dependency installation for DB, persistence-core, React and query-collection dependency graphs.
  • Builds: DB IVM, DB, persistence-core and React.
  • 186 persisted-readiness/publication cases, including the 128-cell initial grammar, derived/shared-source receiving paths, optimistic rollback, source/query failure, metadata/window eligibility, atomic persistence payloads, and later demand convergence.
  • 930 DB checks across live-query Collections, ordered loader/lifecycle, pagination, options, observer histories/snapshots, query identity and the newly merged GC oracles. The existing fixed and random campaigns ran.
  • 60 React hook tests. This is a regression run, not a claim of new native/browser/framework scheduling coverage.
  • DB and persistence-core TypeScript; changed-file ESLint, oracle lint and diff checks.
  • Deliberately removing the complete-publication hold, using an initial-only hold, collapsing shared-source restore contexts, or omitting policy from identity makes the corresponding check fail. Cold-cache network failure and the zero-deadline query-error race also have recorded RED runs.
  • pnpm size:compare --base upstream/main --packages db,react-db,db-sqlite-persistence-core --files: typical app +423 bytes gzip (+0.5%); DB export bundle +442 bytes gzip (+0.4%); React and persistence-core package bundles unchanged.

The new promise is tested through controlled persistence/Collection/observer boundaries. Native SQLite/OPFS restart, crash durability, arbitrary provider histories and cross-framework scheduling remain outside these receipts. The versioned oracle review record documents the exact evidence and limits. This remains a draft for design review.

Why each file changes

FileReason
packages/db/src/query/live/types.tsDefine the opt-in complete-publication policy.
packages/db/src/query/live/collection-config-builder.tsReuse source/demand readiness for publication, atomically replace restored rows, and persist window-specific completion metadata.
packages/db/src/live-query-observer.tsLet a persisted completed view cover its graph's restore boundary without covering a direct peer or changing leaf-source error classification.
packages/db/src/persisted-readiness.tsReject failed-network fallback without an eligible snapshot and prevent a zero-delay fallback from masking a query error.
packages/db/src/live-query-options.tsInclude the publication policy in derived config identity.
packages/db/mangle-cache.jsonRegister the new private metadata helper with the existing production minifier.
packages/db-sqlite-persistence-core/tests/persisted-readiness-oracle.test.tsExtend the primary model/driver and its finite histories at the persistence and observation boundaries.
packages/db/tests/query/ir-stable-identity-oracle.test.tsCheck policy equality versus reusable runtime config identity.
packages/db/src/query/live/ARCHITECTURE.mdState the new optional publication contract and its boundaries.
docs/contributing/glossary.mdAlign complete-publication and persisted-readiness vocabulary.
docs/contributing/oracle-coverage.mdRecord receiving paths and untested boundaries with the existing owner.
docs/contributing/oracle-reviews/completed-query-snapshots.mdPreserve RED/GREEN, mutant classification and self-review evidence.
docs/guides/live-queries.mdExplain composition, durable ordering, identity/invalidation and tradeoffs.
.changeset/completed-query-snapshots.mdRequest a DB minor release for the new option.

Cleanup removed the earlier provisional option, its tests/docs and changeset. Production growth is confined to the new publication/restore contract. The implementation reuses the incumbent graph, metadata and transaction mechanisms; preserving its new certificate/window and shared-source distinctions prevents eliminating the added branches outright.

    1. 🦋 Changeset detected

      Latest commit: 4f90f1d47f241b268a519abc583c743591700ec2

      The changes in this PR will be included in the next version bump.

      This PR includes changesets to release 30 packages
      NameType
      @tanstack/dbMinor
      @tanstack/angular-dbPatch
      @tanstack/capacitor-db-sqlite-persistence-e2e-appPatch
      @tanstack/db-sqlite-persistence-corePatch
      @tanstack/electric-db-collectionPatch
      @tanstack/expo-db-sqlite-persistence-e2e-appPatch
      @tanstack/offline-transactionsPatch
      @tanstack/powersync-db-collectionPatch
      @tanstack/preact-dbPatch
      @tanstack/query-db-collectionPatch
      @tanstack/react-dbPatch
      @tanstack/rxdb-db-collectionPatch
      @tanstack/solid-dbPatch
      @tanstack/svelte-dbPatch
      @tanstack/tauri-db-sqlite-persistence-e2e-appPatch
      @tanstack/trailbase-db-collectionPatch
      @tanstack/vue-dbPatch
      todosPatch
      offline-transactions-react-nativePatch
      shopping-list-react-nativePatch
      @tanstack/db-example-react-next-ssr-e2ePatch
      @tanstack/db-example-paced-mutations-demoPatch
      @tanstack/browser-db-sqlite-persistencePatch
      @tanstack/capacitor-db-sqlite-persistencePatch
      @tanstack/cloudflare-durable-objects-db-sqlite-persistencePatch
      @tanstack/electron-db-sqlite-persistencePatch
      @tanstack/expo-db-sqlite-persistencePatch
      @tanstack/node-db-sqlite-persistencePatch
      @tanstack/react-native-db-sqlite-persistencePatch
      @tanstack/tauri-db-sqlite-persistencePatch

      Not sure what this means? Click here to learn what changesets are.

      Click here if you're a maintainer who wants to add another changeset to this PR

    2. Important

      Draft PR not reviewed

      Draft PRs are not automatically reviewed by default.

      • Trigger a manual review

      To automatically review draft PRs, update your CodeRabbit configuration:

      reviews:
        auto_review:
          drafts: true
      • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

      Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

      ❤️ Share
      Comment `@coderabbitai help` to get the list of available commands.
    3. I'm curious — isn't this rather blink-y? Render immediately the partial set and then the UI jumps a bit later when the full set comes in? Or do you mostly have everything and it's just one or two items arriving later? Just curious in general to hear more about why you have this feature.

Sign in to join this conversation.