fix(db): keep overlapping ordered requests loading and release what they supersede #2194
Merged
KyleAMathews(GitHub profile, opens in a new tab)merged 44 commits into
main fromrefs/pull/2194/head·merged by KyleAMathews(GitHub profile, opens in a new tab) Showing 52 changed files with 13 additions and 3 deletions (loading 50).
x
@@ -0,0 +1,9 @@
+---
+'@tanstack/db': patch
+---
+
+Keep ordered live queries accurate and loading when source work overlaps. A window with `orderBy` and `limit` no longer stops loading, keeps old rows, or holds acquisitions in these cases: a failed or superseded request, a truncate replay, a source change during a repair, or an on-demand joined source. Recovery after a failure uses one full-source read, and a window move whose load succeeded now resolves.
+
+When an adapter's `unloadSubset` throws, TanStack DB now reports the error as an uncaught error and continues as if the release succeeded. The error is no longer a subset failure. It does not set `lastError` or `lastSubsetError`, fail a truncate replay, reject `setWindow()`, or dispose an Effect. `unsubscribe()`, `releaseSnapshot()`, and Effect `dispose()` no longer throw or reject with it.
+
+The newest `setWindow()` call now wins. A pending source recovery that defers two calls no longer lets the older one set the window. A superseded call that succeeds now resolves only after the newest call settles, so awaiting it shows the window that won.
4 additions and 3 deletions+4-3docs/contributing/oracle-coverage.mdModified3 lines can hide codeView file
@@ -63,7 +63,7 @@
| Query DB SSR dehydration | Fixed QueryClient/Seroval boundary | Successful on-demand entries with a compound predicate, a request signal, and an ordered cursor with a custom comparator retain their row and user metadata through Seroval's reconstructed stream payload and Query Core hydration; request options are absent from the stream, while query functions retain the original values. The TanStack Start router and browser Collection resume path still need a host witness. |
-| Ordered acquisition | Complete | Exact demand identity, applied settlement, independent pagination recomputation, request work, lifecycle products, replay authority, source-generation readiness, and transaction-refinement abort boundaries are literate. An initial full-source ordered load shares the synchronous readiness cut ([historical review](oracle-reviews/initial-full-source-sync-readiness.md)). The ordered-lifecycle oracle covers an inexpressible-boundary fallback before first publication when every acquisition returns literal `true`. It also checks a later null-boundary fallback after the first joined window publishes, across literal, Promise, and held-receipt responses, including a held receipt after child demand settles ([review](oracle-reviews/issue-2152-initial-ordered-fallback.md)). |
+| Ordered acquisition | Complete | Exact demand identity, applied settlement, independent pagination recomputation, request work, lifecycle products, replay authority, source-generation readiness, and transaction-refinement abort boundaries are literate. An initial full-source ordered load shares the synchronous readiness cut ([historical review](oracle-reviews/initial-full-source-sync-readiness.md)). The ordered-lifecycle oracle covers an inexpressible-boundary fallback before first publication when every acquisition returns literal `true`. It also checks a later null-boundary fallback after the first joined window publishes, across literal, Promise, and held-receipt responses, including a held receipt after child demand settles ([review](oracle-reviews/issue-2152-initial-ordered-fallback.md)). The ordered loader state oracle also has a generated request-authority lane (`ordered-loader.request-authority`, fixed and random campaigns). It drives the real loader over a fake subscription through settlements in any order, failures, deferred invalidation and report reactions, ordinary passes and window moves, and began with four laws from a TLA+ model (#2191): a successful repair keeps its acquisitions until a covering read succeeds; quiet loading holds nothing older than its last authoritative read; a superseded request's failure is recovered by a full-source read without an explicit retry; and once quiet, an invalidation and a pass issue a read unless a full-source read covers the source. Scenarios also throw from the first release callback, answer a repair synchronously from installed rows, and let a page's own delivery invalidate ordering; named replays keep the traces they found, each of which failed on main (a throwing release that unwound a resumed repair, a synchronous repair waiting behind a superseded request, and a self-invalidating page whose tie was never retired). On main it found a superseded tie that failed after its repair finished and a newer invalidation's reaction was deferred, which left refills blocked; a named replay keeps that trace. A stale prefix that retires the repair's prefix fails the held law, and a stale request held after the repair finished fails the leak law. The lane also crosses unexpressible boundaries, joined work that defers loading, truncate replay that succeeds or stays pending, optimistic departures, source deletes, the bounded backoff retry, window moves that shrink or widen, plans that read only the full source, and reports inside the loader's own first page. A replay owns the requests issued before its truncate, and when the subscription still holds a full-source lease the settled replay covers the source. These dimensions found three more defects, each kept as a named replay: a repair prefix that settled after a replay started its replacement took the replacement's chain tag and was never retired; a replay counted a full-source read as complete after an explicit retry had released it; and a failure from before a truncate dropped the next repair's staged step. Each of the seven defect replays fails on the loader at the branch's merge base with main. A later law, one request at a time, found that a replaced repair chain's prefix still counted as current and issued a tie beside the replacement's prefix; a chain is now superseded by any newer authority start. The failed-lease law also counts a covering replay, which a named replay reaches. Repeated random campaigns then found a repair request that settled after full-source coverage and stayed held; full coverage now retires it. The held law counts a later prefix that a successful replay reacquired while it was in flight. Four histories from TCP's known error classes followed: a request the provider never answers (a lost segment), a release callback that truncates synchronously (a reentrant reset), a window offset other than zero, and an at-most-once release law with no release while a replay owns the demand unless an explicit retry replaces it or a held full-source lease covers it. They found that the backoff retry's release pass kept releasing failed demands after a release started a truncate; every release pass now stops on a reentrant reset. The oracle now observes epoch moves as calls to `invalidateSourceOrdering()`, so it tells a replacement chain from a chain step exactly. The loader's work slot and debt now move only through a transition table that throws `OrderedLoaderInvariantError` on an illegal move or an incoherent work and debt pair; the oracle records any such error as a broken law, so its generated histories fuzz the table. A covering-lease law (a full-source lease whose read succeeded or whose demand a replay reacquired is never released) found that a full-source transport failing after a covering replay turned complete coverage into an own failure; the table then found that a repair request failing after a replay finished its chain did the same. Mutants that drop each new guard fail. A release callback that runs an ordinary pass synchronously, as a graph run does when the release delivers deletes outside a transaction, found that the pass issued a page while another request was settling, and that request then issued its tie; every release pass now blocks refinement until it unwinds. A dispose command and a release callback that disposes the loader check that a disposed loader issues and releases nothing; mutants that drop the loader's own `active` checks in the release pass and in failure handling survive because dispose resets the cursor, so those checks were removed. TLA+ run 3 (`LoaderTable.tla`) models the table with truncate and replay; its safe rules pass with 9,368,122 distinct states, and each of the four guards from this round, removed alone, produces a counterexample. Reverting any one fix fails its replay and a campaign. The generated histories also stack a second truncate on a pending replay and truncate after a failed replay, each with a reach witness. A lease missing from its map during a release pass is now an invariant violation; no history reaches it. A second high review found that a covering replay did not retire a superseded failed lease from before its truncate, because the failed-lease law exempted every replay-owned request. The law now judges a superseded failure that was live (it failed in the incarnation it was issued in) once a covering replay settles after it, and the replay retires those leases. The subscriber now schedules the replay's graph run in a finally. The replay-refinement oracle is the receiving witness: its generated ordered section (`replay.ordered-window-faults`) drives a real on-demand Collection and an ordered live query through rank moves, held and failed loads, truncates, and unloads that throw without a truncate, while a replay starts, and while it settles. Its settled-window law found that a release throwing after a successful ordered load left the window on old rows, because the builder counted the rejected participant as a failed load (also on main); the loader now marks release errors and the builder publishes. It also fails without the subscriber's finally. The section now also crosses tied ranks (its law compares the window's rank sequence and each shown row's current rank), a peer live query on the same source, offset 1, descending order, and explicit window moves; a move that rejects after a failed source recovery leaves the window, and one rejected only by a release failure still moves. A reported-faults law accepts injected faults as rejected loads, thrown releases, rejected moves, or errors the subscription rethrows from a microtask, and rejects any other escaped error. These found that a window move whose load succeeded was recorded as failed when a release threw, so the window stayed private until another move. A throwing release is an adapter bug, so the loader now reports it uncaught from a microtask and changes nothing else; the reported-faults law rejects a move that a release failure rejects, and the request-authority oracle requires each throwing release to be reported uncaught exactly once (a mutant that swallows the error fails). It also crosses a joined marker source: a LEFT join that keeps marked or unmarked rows, and an inner join. The marker source is eager or on-demand, with marker loads that are held, fail, and truncate. This found that an on-demand joined source whose loads finished without delivering a row left the window on old rows, because nothing ran the graph that the pending joined work held; the subscriber now runs it when a joined source becomes ready. A move whose load a truncate aborts may reject with an `AbortError` and leaves the window; a named replay reaches that branch. Recovery truncates the marker source too, because a failed marker replay keeps the last complete publication until that source replays. A hostile-mutant pass over the loader's guards found guards that no ordered oracle kills. A differential instrument then ran 3,000 generated request-authority histories through the loader and each mutant and compared every request's shape, outcome, and release. Three guards the code makes redundant are deleted: the backoff timer guard, the explicit-move check in the retry callback, and the `requiresFullSource` check on a report. An ignored-report law now kills two of the mutants that changed public requests: a report issues no authoritative request before any request settled (a request in flight still owes an exact result for its own options) or while the window is inside the loader's own first prefix. Named replays reach both, and the random campaigns judge them in quiet histories. One mutant still changes requests without a law: dropping the optimistic-refill flag turns a refill into a refetching prefix. Six more are equivalent in the fake but stay, because the fake cannot reach their premise or they encode a documented rule: the own-first-page check behind a held sync transaction (the fake never holds one), a truncate moving complete coverage back to held, clearing the pending request when a full-source read starts (the fake never answers a full-source read synchronously), staging a joined `more` continuation, the staged-continuation guard before an unindexed prefix, and the drain re-entry guard (a long synchronous chain would otherwise recurse). An include dimension adds a `toArray` child that reads the marker source for each shown row, and an include-children law requires each shown row's markers to equal the model wherever the window law holds; a mutant that never loads newly demanded child keys fails it in an include-only scenario. A newest-move law requires a superseded window move to resolve only after the newest move settles (a superseded move that fails still rejects with its own error), and the model lets the newest move that resolved set the window; it found that a superseded move resolved as soon as its own load settled, so awaiting it showed neither window. The terminal-error law also allows `error` after a load fails while the query is still loading, the documented initial-load failure; a lazy join source is exempt. Limits: includes are one level deep. It also found that a release failure in lazy joined demand during a window move rejected the move, because the subscription reported it as a load error. A window law now lets the newest move win; it found that two moves deferred by a pending source recovery could resume in the opposite order, so the older request replaced the newer window. `setWindow()` now ignores a deferred call that a newer call superseded. A terminal-error law allows a final `error` only after a lazy join demand load, on either side, failed outside a replay; named replays reach that branch and the aborted-move branch. Core now reports a throwing `unloadSubset` uncaught and fails nothing with it; the subscription lifecycle, replay, Effect disposal, includes-temporal, and subset-error oracles check the uncaught report, once, with the adapter's error. The request-authority oracle now also generates requests that throw while they start, the last history only TLA+ covered, with an error-identity law: a failed request's participant reports that request's own error. It found that a superseded failure whose recovery read threw at its start reported the start error instead (review finding 6). A cost law then checks, after loading is quiet and a bounded repair is available, that one invalidation reads no full source and issues at most one chain per ordering epoch move (in the fake, a prefix, a tie, and a refill); a mutant that always repairs with a full-source read fails it. The replay-refinement section's status law requires a quiet live query and its peer to stay ready when no load failed. Limits: whether a release runs the graph at once or defers it to a transaction belongs to the scheduler, and the oracle generates both forms without checking the scheduler's choice; the fake's rows do not move except for the unexpressible boundary; publication holds stay with other tests; the laws do not judge requests issued before a truncate; a hung authoritative request suspends the leak law for its unfinished chain; a recovery start that throws during a superseded failure is rare in the campaigns, so a named replay owns it. |
| Indexed predicate filtering | Complete for bounded dotted-path collision | An independent row filter checks direct predicates and public callback predicates for a dotted scalar field beside a nested path, with and without an index. Selected-field ordering checks the same distinction through `$selected`. |
| WHERE predicate publication | Complete for bounded predicate and sync-transaction grammar | The contract, Kleene reference evaluator, snapshot and change-history grammar, three subscriber consumers plus direct snapshot, and per-commit key-set refinement check are literate. A fixed same-key update checks live-row and direct-subscriber payloads; a focused descriptor boundary checks stored-row prefilter safety. Generated cleanup and restart histories for filtered subscribers remain open. |
@@ -253,7 +253,8 @@