1# Thread titles
2
3Generates thread titles for real thread shapes with the server's Codex title generator, then sets each new title against the old one for a blind review.
4
5## Run
6
8cd evals
9pnpm thread-titles --model <codex model> --out runs/titles
10pnpm thread-titles --model <codex model> --out runs/titles-new --baseline runs/titles/results.json
12
13It uses your local Codex login, like the server does.
14`--case <id>` limits the run to one case. Repeat the flag for multiple cases.
15`--initial` titles from the first message only, as a new thread does.
16A run writes `results.json` (titles and latency), `review.json` (titles A and B in random order, with a rubric), and `answer-key.json`. Score `review.json` before you open `answer-key.json`.
17
18## Add a case
19
20Add an entry to `cases.ts` with an `id`, a `source`, the thread `messages`, the `previousTitle`, an optional `projectTitle`, and a `rubric` that says what a good title names.