Problem#
The generated histories refine the model property in packages/db/tests/source-health-oracle.test.ts sometimes fails its reach check in CI. No law fails. The error is:
Error: source-health.history: generated histories reached these states fewer than 1 times, so their laws were not enforced there: markReady on a ready run (0), reload's insert of a kept row throws (0)
Evidence#
- PR #2231, run 40735418214, job
Test (db-1), seed 727029764. All 8251 tests passed. Only this reach assertion failed.
- Branch
investigate/progressive-network-first, run 40719621050, has the same error.
- Locally, three full runs of the file with
TANSTACK_DB_ORACLE_SEED=727029764 pass (504/504). The miss does not come from the seed alone. It appears to depend on timing under CI load (2 workers, sharded).
Cause (not verified)#
The property runs 60 histories. A history reaches these two states only if a random reach prefix (from 6 prefixes, fc.option(..., { freq: 6 })) or random steps produce them, and the async order agrees. With 60 runs, a run that reaches neither state is possible.
Possible fixes#
- Add a fixed history for each of the two states that always runs through the same driver and comparison, so reach does not depend on chance.
- Or make the generator choose a reach prefix in a fixed order for the first runs, so each declared state gets at least one history.
- Or record the states as probabilistic in the coverage map, with a maintainer decision.
The oracle came from #2168, and the reach requirement came from #2278. PR #2231 does not change this file.
This issue was written by Isaac.