fix(db): page ordered direct left-join filters #1909
Merged
KyleAMathews(GitHub profile, opens in a new tab)merged 16 commits into
main fromrefs/pull/1909/head·merged by KyleAMathews(GitHub profile, opens in a new tab) Showing 18 changed files with 88 additions and 1 deletion (loading 15).
x
@@ -0,0 +1,5 @@
+---
+'@tanstack/db': patch
+---
+
+Page ordered windows filtered by a direct LEFT join without loading the entire root collection. Wait for joined rows before continuing and publish complete Collection and Effect windows after child changes.
@@ -106,7 +106,7 @@
| Optimistic state | [history model](https://github.com/TanStack/db/blob/main/packages/db/tests/optimistic-history-oracle.ts), [generated histories](https://github.com/TanStack/db/blob/main/packages/db/tests/optimistic-transaction-oracle.property.test.ts), [truncate capture ownership](https://github.com/TanStack/db/blob/main/packages/db/tests/collection-truncate-ownership-oracle.property.test.ts), [outcomes](https://github.com/TanStack/db/blob/main/packages/db/tests/optimistic-history-outcomes.test.ts), [publication](https://github.com/TanStack/db/blob/main/packages/db/tests/optimistic-history-publication.test.ts) | Independent whole-row snapshots, rollback dependencies, captured truncate ownership, metadata and prior-value events. Never rebase a pending snapshot merely to simplify the model. |
| Drafts and native values | [proxy](https://github.com/TanStack/db/blob/main/packages/db/tests/proxy.test.ts), [detachment](https://github.com/TanStack/db/blob/main/packages/db/tests/proxy-detachment-contract.test.ts), [iteration](https://github.com/TanStack/db/blob/main/packages/db/tests/proxy-iteration-contract.test.ts) | Native-operation controls, exact patches and actual stored rows; alias/cycle/adversarial-key histories. General native-mutator and symbol-write support is not established by a plain-object oracle. |
| Query DB and observer | [ownership](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/ownership-lifecycle.oracle.test.ts), [load lifecycle](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/load-subset-lifecycle-oracle.test.ts), [observer histories](https://github.com/TanStack/db/blob/main/packages/db/tests/live-query-observer-history.property.test.ts) | Real QueryClient boundary, applied-settlement barriers for existing and cached demand, mutation-handler Collection identity, terminal fetch-record lifecycle, and a per-listener eligibility ledger, not a duplicate dispatch queue. The bounded handler grammar crosses insert, update, delete, parameter and mutation-alias access, refetch, and clearError. The mutation overlap grammar crosses both start orders. Check reentry, peer survival, FIFO and disposal independently of final rows. |
-| Ordered acquisition | [pagination](https://github.com/TanStack/db/blob/main/packages/db/tests/query/pagination-oracle.property.test.ts), [ordered work](https://github.com/TanStack/db/blob/main/packages/db/tests/query/ordered-work-oracle.property.test.ts), [ordered lifecycle](https://github.com/TanStack/db/blob/main/packages/db/tests/query/ordered-lifecycle-oracle.property.test.ts), [issue #1880 ordered-repair review](oracle-reviews/issue-1880-ordered-repair.md), [issue #1882 custom local collation review](oracle-reviews/issue-1882-custom-local-collation.md) | Complete finite provider results, inherited locale and bounded/unbounded custom collation with exact request options, query-level inheritance across source aliases, real lexical/numeric disagreement, custom comparator ties, scan/index paths, pending windows, ties/nulls, lease ownership, Effect callback gates, and documented repair timing. Custom collation proves local full-source acquisition only; it does not establish provider cursor capability or backend collation fidelity. Sibling-source input during a synchronous ordered continuation returns to graph work before another ordered acquisition. Request completion is not proof of unrequested source extent. |
+| Ordered acquisition | [pagination](https://github.com/TanStack/db/blob/main/packages/db/tests/query/pagination-oracle.property.test.ts), [ordered work](https://github.com/TanStack/db/blob/main/packages/db/tests/query/ordered-work-oracle.property.test.ts), [ordered lifecycle](https://github.com/TanStack/db/blob/main/packages/db/tests/query/ordered-lifecycle-oracle.property.test.ts), [ordered loader state](https://github.com/TanStack/db/blob/main/packages/db/tests/query/ordered-source-loader-state.test.ts), [graph scheduler](https://github.com/TanStack/db/blob/main/packages/db/tests/query/scheduler.test.ts), [issue #1880 ordered-repair review](oracle-reviews/issue-1880-ordered-repair.md), [issue #1882 custom local collation review](oracle-reviews/issue-1882-custom-local-collation.md), [issue #1898 relation-filter review](oracle-reviews/issue-1898-relation-filter.md), [PR #1909 external review](oracle-reviews/pr-1909-external-review.md) | Complete finite provider results, inherited locale and bounded/unbounded custom collation with exact request options, query-level inheritance across source aliases, real lexical/numeric disagreement, custom comparator ties, scan/index paths, pending windows, ties/nulls, lease ownership, Effect callback gates, and documented repair timing. Direct LEFT-joined filters cover local finite prefixes, pending child demand, child removal, and anti-join publication with custom full-source and unindexed fallback loading; unrelated include demand cannot stall the root continuation, and an empty joined replay wakes held Effect callbacks; remote relation hints remain outside this owner. Graph scheduler checks coalescing, dependency order, synchronous loader input, and repeated-alias failure ownership. The ordered-work oracle checks that a synchronous root refill reaches D2 before joined publication; a mutant that omitted the post-loader graph step failed at the second child demand. Custom collation proves local full-source acquisition only; it does not establish provider cursor capability or backend collation fidelity. Sibling-source input during a synchronous ordered continuation returns to graph work before another ordered acquisition. Request completion is not proof of unrequested source extent. A window move during existing joined demand waits for its root continuation, including an independently held root request; current demand failure rejects the move, while retirement and an obsolete rejection do not. A bounded ordered repair resumes its refill after joined demand settles. The ordered-source-loader state test checks that an explicit failed-request retry blocked by joined demand retains its generation until it can issue a request. Multiple simultaneously pending joined plans during a window move and a public-query failed-retry overlap still need dedicated witnesses. |
| Join equality and cold acquisition | `packages/db/tests/query/cold-join-reconciliation-oracle.test.ts` | Independent recomputation for cold acquisition plus direct join/predicate equivalence across established equality domains. Binary/string and nullish classes, replacement histories, raw on-demand values, and both scan/auto-index paths are explicit; compound join syntax is not claimed. |
| Opaque backend pagination | [window oracle](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/cursor-pagination.oracle.test.ts), [cache histories](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/cursor-pagination.cache-oracle.test.ts), [cache publication](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/cursor-pagination.publication-oracle.test.ts), [browser acquisition boundaries](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/cursor-pagination.boundary-oracle.test.ts), [QueryCollection integration](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/cursor-pagination.integration.test.ts) | Full filter/sort/slice reference, opaque token transport, actual Query cache expiry/invalidation/GC, forced refresh during growth, protocol failure publication/recovery, bounded slice work, nested cancellation/replacement, reader abort, browser retry defaults, manual-write cache isolation, and production window publications. Stable backend sequences; not snapshot guarantees for changing endpoints. Peek-ahead remains enabled. |
| Electric and TrailBase | [Electric histories](https://github.com/TanStack/db/blob/main/packages/electric-db-collection/tests/electric-oracle.property.test.ts), [recovery histories](https://github.com/TanStack/db/blob/main/packages/electric-db-collection/tests/electric-recovery-oracle.test.ts), [held resume snapshots](https://github.com/TanStack/db/blob/main/packages/electric-db-collection/tests/electric-resume-snapshot-races.test.ts), [PostgreSQL semantics](https://github.com/TanStack/db/blob/main/packages/electric-db-collection/e2e/sql-predicate-semantics.e2e.test.ts), [TrailBase contract](https://github.com/TanStack/db/blob/main/packages/trailbase-db-collection/tests/ORACLE.md) | Installed SDK delivery/framing, independent predicates, exact subscription arguments, restart/reset lineage, held certification and durability races, source-order publication before durability, and late errors. The queued-presence property runs identical fixed/random generators plus isolated seed-and-path replay across insert, update, delete, and truncate callbacks. The recovery fixtures use a mocked ShapeStream; they do not establish live Electric-service framing or native persistence-host behavior. |
@@ -119,6 +119,14 @@
| Boundary refinements | [cleanup/restart](https://github.com/TanStack/db/blob/main/packages/db/tests/collection-cleanup-restart-oracle.test.ts), [issue #1891 async-cleanup review](oracle-reviews/issue-1891-async-cleanup.md), [issue #1891 cleanup-start follow-up](oracle-reviews/issue-1891-cleanup-start-followup.md), [metadata publication](https://github.com/TanStack/db/blob/main/packages/db/tests/collection-metadata-publication-oracle.property.test.ts), [state retention](https://github.com/TanStack/db/blob/main/packages/db/tests/collection-state-retention-oracle.property.test.ts), [acquisition cells](https://github.com/TanStack/db/blob/main/packages/db/tests/collection-subscription-lifecycle-oracle.test.ts), [D2 source reconciliation](https://github.com/TanStack/db/blob/main/packages/db/tests/d2-source-reconciliation-oracle.property.test.ts), [top-K support windows](https://github.com/TanStack/db/blob/main/packages/db-ivm/tests/operators/topk-support-window-oracle.test.ts), [nested Query work](https://github.com/TanStack/db/blob/main/packages/query-db-collection/tests/includes-work-counter-oracle.test.ts) | Explicit lifecycle products, independent source maps and weighted relations, exact publication cuts, support/multiplicity, and value-plus-work observations. These refine the larger subsystem models; they do not replace them. |
| Small structures and test mechanics | [SortedMap](https://github.com/TanStack/db/blob/main/packages/db/tests/SortedMap.test.ts), [cleanup queue](https://github.com/TanStack/db/blob/main/packages/db/tests/cleanup-queue.property.test.ts), [guarded replay](https://github.com/TanStack/db/blob/main/packages/db/tests/oracle-replay.test.ts) | Map/full-sort and appointment-list models with executed target/seed/path checks. Callback-reentrant scheduling is outside the initial cleanup-queue domain. |
+The ordered-work owner also checks that an Effect with `skipInitial` waits for
+joined demand before ending initial callback suppression. A deterministic work