Skip to content
Source
Sign inSign up
TanStack/db

Close the remaining #2136-class gaps in the ordered refetch oracles #2186

Send to my agent
OpenKyleAMathews(GitHub profile, opens in a new tab) opened this issue 2 comments

Summary

Follow-up oracle work for the #2136 bug class: a loader that treats rows as loaded after the provider's state has moved. #2154 closed the reported trace and strengthened the query-db refetch oracle. This issue records the histories, paths and observations that are still outside the oracles, so a later agent can close more of the class.

What #2154 established

packages/query-db-collection/tests/refetch-ordered-prefix-oracle.test.ts:

  • Window law and coverage law, after preload and after each refetch, invalidation and hold release.
  • Generated campaigns: marker flips, lagging backend changes, a peer observe query, invalidation, and optimistic update and delete holds across a refetch.
  • A premise-directed held campaign for the loader's first-page skip.
  • At four times the run count, six random seeds pass.

Each new dimension found real bugs: four in the loader, two in the gate. So the finding rate has not yet reached zero, and a green campaign is weak evidence beyond these dimensions.

Remaining work

  • Held answers combined with a reporting refetch (ordered-work oracle). Combine the holding provider with the refetch-signal provider, and keep the ledger law (the Collection holds exactly the provider's sent rows). This would have found the stale-prefix retirement bug fixed in #2154 as a generated failure. It is a natural part of #2155.
  • #2155, unindexed peer pairing. Seed -1471487158 fails with peer rows after step 2: expected [ 100 ] to deeply equal [ 100, 101 ].
  • Window check after marker flips in the query-db campaigns. During #2154, both campaigns at 1× passed with a window check after every flip (provider current, no hold). Add that law.
  • Order values. The query-db model types ranks as numbers. Add null, undefined and NaN order values. A gate bug with undefined order values was found by review, not by the oracle.
  • Providers. The campaigns use an uncapped cursor provider only. They need offset-only (#2182) and page-capped (#2183) providers.
  • Refetch kinds. Interval, focus and reconnect refetches share the invalidation path, but no history drives them.
  • Windows during a hold. No window is checked while an optimistic transaction persists (#2169, #2148).
  • Optimistic inserts are not generated.
  • Preload during a hold. The harness defers awaiting a preload that starts during a hold (#2184).
  • Multi-tab. Follower tabs never get the report (#2174). That needs a witness in the real-tab coordinator lane.
  • A joined user refetch is not reported (#2185).
  • Process. When an oracle's limits section hands a case to another owner, the coverage map should name that owner's witness. #2154's review found an overlap case that the named owner did not cover.

Pointers

  • Coverage map row: "Query DB ordered refetch" in docs/contributing/oracle-coverage.md.
  • Loader rules: packages/db/src/query/live/ARCHITECTURE.md, "Ordered requests, continuation, and recovery".

This issue was written by Isaac.

  1. More work for this class, from a second high review of #2154 (head 4a8e168c6). The review fixed one regression and three smaller defects. These findings were not fixed and need follow-up.

    • Reports split across a slow refetch. Query DB batches reports only within one setTimeout(0). An invalidateQueries refetch can settle its pages at different times. Then each changed page can report on its own. Each report restarts the bounded repair in every paging ordered live query on the Collection. The oracle's "a step reports at most once" law holds only because its fetches settle in one turn.
      • Witness idea: give the query-db refetch oracle's provider per-request latency.
      • Fix idea: report once no ordered page of the Collection is still fetching. A fetch that fails must not lose the report.
    • Reports have no scope. Every ordered loader on the Collection runs revalidateLoadedCoverage(), whatever its predicate or order. Each listener also schedules a graph run (collection-subscriber.ts, effect.ts). The first-page skip limits the cost for top-N windows.
      • Fix idea: send the page's demand key with the report. Schedule a graph run only when the loader starts a repair.
    • Deferred deletes during retirement. retiringSupersededAcquisitions ignores the deletes that a repair's own retirement causes, but only while the release callbacks run. An adapter that removes released rows later, such as persisted retention with gcTime, still triggers an ordering invalidation. Its next first page then cannot retire the repair's prefix, which is the leak that #2154 fixed for synchronous removal.
      • Witness idea: the persisted-collection oracle, with a retention delay.
    • Persistence and the first-page skip. The skip refuses while core holds an accepted sync transaction back. Under the SQLite persistence wrapper, the report could fire before the wrapper queues its rows in core. An earlier reading of the code concluded that the wrapper accepts only after it queues them, so this is probably covered, but no test drives it.
      • Witness idea: a persisted query-db refetch whose first page's end moves earlier during a durable write.

    This comment was written by Isaac.

  2. The ordered loader state refactor and its model-predicted bug are tracked in #2191. That issue carries the TLA+ model, its results and the plan: an oracle lane, the F2 refactor, then a model of the publication barrier after #2177.

    This comment was written by Isaac.

Sign in to join this conversation.