Summary#
In packages/db, coverage is on by default (vite.config.ts: coverage: { enabled: true, provider: 'istanbul', include: ['src/**/*'] }), so every local pnpm test run pays for instrumentation. I measured the cost on the full suite:
| Run (285 files, 4 workers) | Elapsed | Total CPU |
|---|
| Istanbul coverage (default) | 144 s | 507 s |
--coverage.enabled=false | 96 s | 367 s |
| Difference | +48 s (+50%) | +140 s (+38%) |
Repro#
From packages/db:
/usr/bin/time -p pnpm vitest --run --maxWorkers=4
/usr/bin/time -p pnpm vitest --run --maxWorkers=4 --coverage.enabled=false
Environment: Apple M3 Max (16 cores, 128 GB), Node v26.12.0, pnpm 11.1.0.
Goal#
Reduce test overhead while keeping the same coverage and checks that CI enforces today. CI collects DB coverage per shard and merges it with vitest --mergeReports in .github/workflows/pr.yml, and that should keep working.
Possible directions (for discussion)#
- Turn coverage off by default for local runs and enable it explicitly in
scripts/ci-tests.mjs / CI, so CI coverage doesn't change.
- Compare
provider: 'v8' with Istanbul for overhead and report parity before considering a switch.
I'm happy to do the follow-up PR, with before/after numbers, once there's agreement on a direction.
Follow-up: here is where the rest of the cost goes, beyond coverage.
Method. I ran
packages/dbwith--maxWorkers=4, changing one condition per run. Times are wall time and total CPU (user+sys). I measured on commit26d38a4cdwith Node 26.12.0, on an M3 Max. A repeated run of the same condition matched within 0.02 s. On Node 24.18.0 withorigin/main+ #2275 (300 files), the coverage cost has the same shape: 122 s / 421 s without coverage, 165 s / 522 s with it.tests/oracle-replay.test.tstypecheck: { enabled: true }tscenvironment: 'jsdom'for all filestests/indexed-db/retirement-oracle.test.tsNotes:
oracle-replay.test.tsruns 38 tests. Each one callsspawnSyncto startnode --import tsx tests/oracle-replay.ts, which spawnsvitest run. One replay costs about 1.7 s, and Vitest's own work is 0.55 s of that. The rest is process startup. The file runs serially in one worker for 77 s, which sets a floor on the suite's wall time.Next I'm looking at reducing the replay test's process startup without weakening what it checks.