Skip to content
Source
Sign inSign up
TanStack/db

Ordered-replay oracle: terminal error after a failed settle following truncate markers #2224

Send to my agent
ClosedKyleAMathews(GitHub profile, opens in a new tab) opened this issue 2 comments

Summary

The random campaign of packages/db/tests/query/load-subset-replay-refinement-oracle.test.ts › "Ordered windows across truncate replays › keeps the ordered window under load and release faults" found a counterexample on main at 30175be48 (#2194). The failure is deterministic when the seed is replayed.

Replay

cd packages/db
TANSTACK_DB_ORACLE_SEED=1817832416 \
TANSTACK_DB_ORACLE_PATH=115:1:2:2:2 \
TANSTACK_DB_ORACLE_PROPERTY=replay.ordered-window-faults \
pnpm vitest run tests/query/load-subset-replay-refinement-oracle.test.ts -t "release faults"

Counterexample (shrunk)

[
  [{ "type": "hold" }, { "type": "truncateMarkers" }, { "type": "settle", "ok": false }],
  { "direction": "asc", "offset": 0, "limit": 1, "ties": false, "peer": false,
    "joined": "none", "markerMode": "on-demand", "includes": true }
]
AssertionError: convergence: terminal error: expected false to be true
  at checkOrderedReplay (tests/query/load-subset-replay-refinement-oracle.test.ts:1875)

The shown window is in an error state, but the oracle's model has no failed load that explains it (fatalLoadFailed() is false). Either production shows a terminal error that the history does not justify, or the model is missing a case where a failed settle after truncate markers is fatal.

Where it was seen

CI for #2219 (job Test (db-1), run 39726325795). #2219 does not change @tanstack/db; CI tests that PR merged with main. The replay above fails with main's packages/db.

This issue was written by Isaac.

  1. Fixed by #2229 (merged as 1640de834).

    On main at 1640de834, the shrunk trace from this issue passes: hold, truncateMarkers, then a failed settle, on an ascending limit: 1 window with joined: "none", markerMode: "on-demand", and includes: true. A second random-campaign seed that found the same class also passes now: TANSTACK_DB_ORACLE_SEED=1476128014 TANSTACK_DB_ORACLE_PATH=191:3:9:4:4:4:4 TANSTACK_DB_ORACLE_PROPERTY=replay.ordered-window-faults.

    This turned out to be a gap in the model, not a production defect. The failed load is new include demand issued during the marker replay, so the terminal error is fatal. #2229's model now classifies it that way, through its new-demand-failed-during-marker-replay state.

    A follow-up PR keeps this exact trace as a named replay in load-subset-replay-refinement-oracle.test.ts, with terminal: true.

    #2239 reports the same trace and is closed as a duplicate.

  2. 🎉 The fix for this issue has been released!

    Fixing PRs: https://github.com/TanStack/db/pull/2231, https://github.com/TanStack/db/pull/2266

    Backfilled notification for published release candidates.

Sign in to join this conversation.