A second live query over a refetched on-demand source keeps a short joined window #2155
Send to my agentProblem#
Two live-query Collections can read the same on-demand source. After a refetch, the second Collection can show a short ordered window. It stays short until a different change starts a load.
The ordered-work oracle in #2132 found this with a generated history. The failure also occurs with the production code on main, so #2132 does not cause it.
Minimal trace#
- On-demand source with a refetching provider. The provider sends no change until a refetch, and each refetch keeps every served ordered range whole.
- Query: rows
LEFT JOINmarkers on the row's join key (link),WHEREthe marker exists,ORDER BY rank ASCwith an index onrank. - Collection A has
limit 2. Collection B (the second live-query Collection) haslimit 3. - Rows 9, 0 and 1 have rank 0 and a marker. Row 2 has rank 1 and no marker. 200 filler rows have rank 100 and up, with markers.
Steps:
- The backend inserts row 10, with rank 1 and no marker. The provider does not send it yet.
- Row 0 changes its join key, so it no longer matches a marker. The provider does not send this yet.
- The provider refetches.
Expected window of Collection B: ranks [0, 0, 100] (rows 9, 1 and the first filler row). Actual window: [0, 0]. Collection A shows the expected window.
Ingredients#
The failure needs all of these:
- A refetching provider. A forwarding provider passes.
- A second live-query Collection on the same source. One Collection with
limit 3passes. - A join on the row's join key. The same history with marker changes passes.
- A tie at the window boundary with a row that the provider never sent. Without the insert of row 10, the history passes.
A source filter instead of a joined filter also passes.
Hypothesis#
After the refetch, row 0 leaves both windows in one commit. Collection B must load past the rank-1 tie group, where every row fails the join. Two causes are possible:
- Collection B's ordered loader does not request again, because its last page has the same count and boundary as before.
- The rows that Collection A already loaded answer the request of Collection B.
Then nothing proves the window, and no request follows.
Evidence#
- Witness trace: oracle seed 3103989100 of the ordered-work oracle in #2132. #2132 pins it as an
it.failsreplay, and excludes the refetching-provider and second-Collection combination from the random grammar until this fix.
This issue was written by Isaac.
Update from #2154 (commit
e34948bf1).The indexed trace now passes. #2154 fixed a loader stall. A Query that Query DB answers synchronously from its cache writes rows that start a graph turn inside the request. That turn's request for more data was ignored, and the staged continuation then waited for a graph turn that never came. With the fix, the pinned replay in
packages/db/tests/query/ordered-work-oracle.property.test.tsexpects the right peer window. It is renamed "shows a peer's full joined window after a refetch removes a tied row". The fix's comments do not trace why this closed the peer trace. Check that before you rely on it.An unindexed trace still fails. With the peer and refetching-provider pairing restored in the ordered-work grammar, random seed
-1471487158fails:{"path":"on-demand","provider":"refetch-signal","peer":true,"filter":"joined","via":"marker","indexed":false,"limit":1,…} peer rows after step 2: expected [ 100 ] to deeply equal [ 100, 101 ]It fails the same way without the #2154 loader fix, so it is a separate defect. The grammar still excludes the pairing, and the coverage map names this case.
To close this issue:
- Replay that seed with
TANSTACK_DB_ORACLE_SEED=-1471487158. - Fix it, then restore the pairing in the grammar.
- Add the oracle extension from #2186: combine held answers with the
refetch-signalprovider. The same mixed-provider harness serves both.
This comment was written by Isaac.
- Replay that seed with
Sign in to join this conversation.
Handoff: reproduction, request trace, and a located dead end#
This adds what the next agent needs. Line numbers refer to PR #2154 at
abc08f1c8(branchfix-query-db-refetch-ordered-gap). The relevant loader code is unchanged frommain.Reproduce#
The pinned replay in
packages/db/tests/query/ordered-work-oracle.property.test.ts:The replay passes today because it expects the symptom: it requires
checkEagerWindowto reject with exactlypeer rows after step 2: expected [ +0, +0 ] to deeply equal [ +0, +0, 100 ]. Once this is fixed, replaceawait expect(checkEagerWindow(...)).rejects.toThrow(...)with a plainawait checkEagerWindow(...).The case (as pinned):
The random grammar excludes this pairing:
withoutPeerRefetch(around line 4680) dropspeer: truewithprovider: 'refetch'and with the newprovider: 'refetch-signal'. Remove that filter after the fix, and run the campaigns with more runs (TANSTACK_DB_ORACLE_RUNS_MULTIPLIER=10); replay any counterexample withTANSTACK_DB_ORACLE_SEED,TANSTACK_DB_ORACLE_PATHandTANSTACK_DB_ORACLE_PROPERTY(seepackages/db/tests/oracle-config.ts).Provider request trace#
I logged each answered request (temporary logging, since removed).
sub0/sub1are Collection A (limit 2) and its Effect;sub2is the peer Collection B (limit 3):Every row at rank 1 fails the join, so B still needs one row. It should continue with
cursor: rank > 1, which reaches the rank-100 fillers. It never sends that request.Where the loader stops#
Instrumenting
OrderedSourceLoader(packages/db/src/query/live/ordered-source-loader.ts) for B's loader shows this sequence for the final tie request:What happens:
loadBoundary, ~l.925) settles synchronously (canSettleSynchronously, ~l.648).getGraphInputRevision() !== requestGraphInputRevision, so the sync branch (~l.836–845) skipsdrainNoInputContinuations.complete(undefined, true)stages themorecontinuation (continueOrStage,l.670) for the next graph pass to consume inl.175).loadMore(loadMoreIfNeeded→loadMorereturns at thethis.requestingguard (l.157).loadMore. Rows 2 and 10 fail the join and change no window row, so no publication or size change follows. The stagedmorecontinuation is never consumed. Nothing proves B's window, and no request follows.During the initial load, the same staging path ran with
inputRevSame=true.drainNoInputContinuationsthen consumed the continuation immediately, so the initial load completed.This refines the issue's first hypothesis. The loader does decide to continue, but the continuation it stages has no consumer. I did not confirm why a single Collection with
limit 3passes. A likely cause is that its rows reach the graph through a different request, so the tie settles with the input revision unchanged. Check this first: the fix should cover every staged continuation whose graph pass ran under therequestingguard, not just this order of events.Fix directions (unverified)#
loadMoreoncerequestingclears, when the in-request passes were swallowed by the guard.Each must keep the synchronous-cut laws in the ordered-work and pagination oracles green (
keeps rows, request traces, batches, errors, and liveness equal).Related, possibly same class: query-db refill after a marker removal#
packages/query-db-collection/tests/refetch-ordered-prefix-oracle.test.tshas anit.failswitness, "a current provider refills a joined window after a marker removal (known loader defect)". It expects window[5, 1]but gets[5]. An earlier trace showed that on the final fliploadMorecomputesneeded 1with boundary 203 and issuesloadPage(1, 203). No further request follows. That exact request was already served during an earlier refetch, so query-db answers it from the cached Query result without fetching. The load settles with no new rows, and the refill stops there.Both are refills that stop after a request settles without proving the window. Check whether the fix for this issue also turns that
it.failsgreen; if it does, change it toit. That case also involves query-db's cached answers, so it may need its own fix.Not fixed by
invalidateLoadedSubsets()#PR #2154 adds a sync-adapter report,
invalidateLoadedSubsets(), which triggers each ordered loader's bounded prefix repair. It does not fix this issue: the failing provider in this replay keeps served ranges whole and never reports. The ordered-work oracle's newrefetch-signalprovider still excludes the peer pairing for the same reason.This comment was written by Isaac.