Skip to content
Source
Sign inSign up
TanStack/db

Ordered window starts two full-source reads after a signalling refetch with no unexpressible page end #2268

Send to my agent
OpenKyleAMathews(GitHub profile, opens in a new tab) opened this issue 1 comment

Summary

After one signalling refetch, a joined ordered window with an unindexed source starts two full-source reads. The ordered-work oracle allows none for this history: no page in it ends on a null or NaN rank, so the documented full-source fallback does not apply.

This is a different counterexample from #2221. That one needs a NaN rank, an anti-joined filter, and a descending window, and it exceeds a bound of one. This one has a single row with rank 0, a joined filter, an ascending window, and exceeds a bound of zero.

The failure is on main. A randomized CI campaign found it while testing an unrelated pull request (#2103, which does not change packages/db).

Law

packages/db/tests/query/ordered-work-oracle.property.test.ts, in the eager indexed ordered windows lane: after a signalling refetch, if no page end became unexpressible, the loader starts no new full-source request.

if (signalsRefetch && unexpressiblePageEnds === unexpressibleBefore) {
  expect(
    fullSourceRequests,
    `full-source requests after a signalling refetch at step ${step}`,
  ).toBe(fullSourceBefore)
}

Reproduce

On main at 7e46f81ad, from packages/db:

TANSTACK_DB_ORACLE_SEED=2146201763 npx vitest run --coverage.enabled=false \
  tests/query/ordered-work-oracle.property.test.ts \
  -t "eager indexed ordered windows keeps rows and per-change work bounded for a random or replayed seed"

Result, every run:

AssertionError: full-source requests after a signalling refetch at step 1: expected 2 to be +0

Shrunk counterexample:

{"path":"on-demand","provider":"refetch-signal","latency":"immediate","cancellation":"cooperative","peer":false,"filter":"joined","via":"marker","indexed":false,"direction":"asc","nulls":"first","terms":1,"limit":1,"offset":0,"initial":[],"history":[{"type":"upsert","id":0,"rank":0,"eligible":false}]}

The cause is not diagnosed. The seed was first seen in CI job Test (db-1) for #2103 at cb9db1da1, which reproduced the same counterexample.

  1. A second seed reaches the same law, probably through the same cause. CI found it on #2103 (Test (db-1) at 9d0ee7234), where it timed out at 5 s because shrinking took about 9 s.

    On main at 7e46f81ad, from packages/db:

    TANSTACK_DB_ORACLE_SEED=1534047018 npx vitest run --coverage.enabled=false --testTimeout=60000 \
      tests/query/ordered-work-oracle.property.test.ts \
      -t "eager indexed ordered windows keeps rows and per-change work bounded for a random or replayed seed"
    AssertionError: full-source requests after a signalling refetch at step 2: expected 2 to be 1

    Shrunk counterexample: the same joined, ascending, nulls: first, unindexed window with an ineligible upsert, here with a null rank and a window move:

    {"path":"on-demand","provider":"refetch-signal","latency":"immediate","cancellation":"cooperative","peer":false,"filter":"joined","via":"marker","indexed":false,"direction":"asc","nulls":"first","terms":1,"limit":2,"offset":0,"initial":[],"history":[{"type":"upsert","id":0,"rank":null,"eligible":false},{"type":"move","offset":0,"limit":1}]}

    Under the default 5 s test timeout, this seed reports a timeout instead of the assertion.

Sign in to join this conversation.