Skip to content
Source
Sign inSign up

fix(clients): answer history no longer shows provider context from review comments #35897

Mergedjuliusmarmingemerged 3 commits intomain fromrefs/pull/35897/headmerged by juliusmarminge View on GitHub

juliusmarmingecommented

Review item 1 was correct: the earlier helper removed every <t3_context version="1">…</t3_context> span, so XML a user typed into an answer was lost, and an answer that was only that XML displayed as empty. The helper now proves an envelope came from provider projection before removing it. projectPendingUserInputAnswer projects each chip separately with projectComposerContextForProvider, which writes [Kind: label; ref=<id>] + \n\n<t3_context version="1">\n<context kind="…" id="<id>"… … </t3_context>. composerContextProviderTextForDisplay walks the provider markers, and for each it checks that the envelope prefix starts right after the marker and that the entry's id attribute equals the marker's ref (escaped as escapeAttribute escapes it). Only then does it replace marker plus envelope with [Kind: label], cut at the first </t3_context>; payload text is escaped so that tag cannot appear inside an entry. Everything else, including typed XML and markers with no envelope, is returned unchanged, as on main. The reviewer's two examples now display in full, so hasQuestionAnswer stays true and the preview shows the answer. The existing repro test case gained an assertion that typed envelope XML stays unchanged. A throwaway test (not committed) ran real projectPendingUserInputAnswer output with two review-comment chips, one comment containing </t3_context>, through getQuestionAnswerText and got 'See [Review comment: notes.md L2] and [Review comment: notes.md L5] please'. Worst case on adversarial input: the marker regex /[([A-Z][^[]:;\n]): ([^[];\n]); ref=([^[]\s]+)]/g has character classes that cannot cross '[' or ']', so its scan is close to linear. Each marker then gets one fixed-prefix startsWith, one indexOf for the closing quote and one for the closing tag. Consumed spans advance the cursor and matches before it are skipped, so total work stays about linear in the answer's length.

Fixes pingdotgg/t3code#35843

Reproduction on main

Commit 95cb068c10 (origin/main). Ran T3CODE_DEV_SEED=0 vp run dev with a fresh database and added a scratch git project with one uncommitted line in notes.md. Started a thread on the Replay (dev) provider's question scenario. That scenario's question is choice-only, so as a temporary, uncommitted test fixture I pointed it at a copy of its transcript with the enum removed, making the question free-text. Answer submission, storage and display ran as they are on main. In the web client (headless Chromium via Playwright), I typed 'See my note: ' into the answer, added a file comment 'Rename this to something clearer' on notes.md line 2 (the chip 'notes.md L2' appeared in the answer), and pressed Submit. The answered-question work-log row read 'See my note: [Review comment: notes.md L2; ref=review-comment_file-comment-…] <context kind="review-comment" …> file: notes.md range: L2 (1-1) section: File comment comment: Rename this to something clearer md: line changed '. Expanding the row showed the same block as multi-line raw text under the question. A plain answer is unaffected because it contains no envelope. The failing unit test on factory/repro-35843 shows the same thing at the shared helpers.

Evidence

Real web client, Chromium, 1280×800, light theme, fresh synthetic databases. Start a free-text Replay question, type “See my note: ”, add a file comment “Rename this to something clearer” on notes.md L2, submit the answer with its chip, then expand answer history.

Proven: the scenario fails on base and passes on head. Base displays the internal context XML, comment text, and code; head retains the user's words and readable comment mention without the envelope.

Before — 95cb068c10After — bb43f3bb61
ResultFailed: context envelope found in answer historyPassed: no envelope; readable answer retained
Submitted chipBefore: answer with review-comment chipAfter: answer with review-comment chip
SummaryBefore: answered-question summaryAfter: answered-question summary
Expanded historyBefore: raw provider context exposedAfter: readable answer without provider context
RecordingBefore interactionAfter interaction

Scenario source · Temporary startup fixture patch. Apply the patch before running the scenario; it removes the replay question's enum identically on both sides. Neither artifact was committed.

Not shown:

  • Desktop and mobile were not run; this proof covers the real web client using the shared answer-history helper.
  • Literal user-authored XML preservation and other context kinds were not exercised; this scenario targets the reported review-comment answer.

Tests

vp test run packages/client-runtime/src/work-log/userInput.test.ts packages/shared/src/composerContextReferences.test.ts: 2 files, 40 tests passed. A throwaway test (deleted, not committed) ran real projectPendingUserInputAnswer output with two review-comment chips through getQuestionAnswerText: passed, output shown in summary. vp run check:changed: format, lint, knip, typecheck (35 packages) and tests (3 files) all pass.

Not in this change

  • Answers are still stored in provider form on the server. Storing the composer form plus records, as normal chat messages do, would let clients render real chips, but that is a contract and server change.
  • An answer chip whose record is missing still shows its raw markdown t3-context link, which is unchanged from main.
  • The user's own text is not escaped when an answer is projected for the provider, so a user who types an exact marker followed by a matching envelope is indistinguishable from projected output. That behaviour was already on main and is out of scope here.

Who did the work

  • Written by Claude Opus 5.5, in Claude Code
  • Reviewed by GPT-6.1 Sol, in Codex, and by Claude Opus 5.5, each on its own
  • Proof captured by GPT-6.1 Sol and checked by Claude Opus 5.5

Run by the t3code factory in T3 Code.

    1. github-actions[bot](GitHub profile, opens in a new tab) added labelsvouch:trusted

    2. tanstack-source added labelsbug

    3. Approvability

      Verdict: Approved at bb43f3b

      Macroscope's review found this PR approvable — Answer histories and previews now show readable context labels instead of provider-context envelopes. The change is limited to display formatting and leaves stored answers and provider inputs unchanged.

      You can add or adjust custom eligibility rules. [Learn more](https://docs.macroscope.com/approvability#example-custom-eligibility-rules).
    4. Thread transfer impact

      ✅ Thread transfer remains within every enforced ceiling.

      ℹ️ The exact PR base did not have a successful artifact. Baseline uses the latest successful main measurement shown below.

      ProviderMetricMain baselineThis PRImpactPR ceiling
      CodexTotal thread wire5.0 KiB5.0 KiB0 B (0.0%)6.8 KiB✅
      CodexThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
      CodexLive turn WebSocket wire1.2 KiB1.2 KiB0 B (0.0%)2.0 KiB✅
      CodexLive turn WebSocket decoded21.5 KiB21.5 KiB0 B (0.0%)24.4 KiB✅
      CodexLive turn messages110 (0.0%)8✅
      ClaudeTotal thread wire5.0 KiB5.0 KiB0 B (0.0%)6.8 KiB✅
      ClaudeThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
      ClaudeLive turn WebSocket wire1.2 KiB1.2 KiB0 B (0.0%)2.0 KiB✅
      ClaudeLive turn WebSocket decoded21.9 KiB21.9 KiB0 B (0.0%)24.4 KiB✅
      ClaudeLive turn messages110 (0.0%)8✅

      Other scenarios

      All 64 scenario measurements match main.

      All 64 scenario measurements
      ProviderScenarioMetricMain baselineThis PRImpactPR ceiling
      Codexshort turn resumeWebSocket decoded10.5 KiB10.5 KiB0 B (0.0%)12.7 KiB✅
      Codexshort turn resumeWebSocket messages220 (0.0%)2✅
      Codexlong turn running, cold openHTTP decoded281.7 KiB281.7 KiB0 B (0.0%)302.7 KiB✅
      Codexlong turn running, cold openrows75750 (0.0%)75✅
      Codexlong turn running, history pageHTTP decoded92.4 KiB92.4 KiB0 B (0.0%)107.4 KiB✅
      Codexlong turn running, history pagerows75750 (0.0%)75✅
      Codexlong turn finished, cold openHTTP decoded381.5 KiB381.5 KiB0 B (0.0%)410.2 KiB✅
      Codexlong turn finished, cold openrows1511510 (0.0%)151✅
      Codexlong turn finished, socket cold openWebSocket decoded381.6 KiB381.6 KiB0 B (0.0%)410.2 KiB✅
      Codexlong turn finished, socket cold openWebSocket messages220 (0.0%)2✅
      Codexlong turn finished, socket cold openrows1511510 (0.0%)151✅
      Codexlong turn finished, work pageHTTP decoded186.2 KiB186.2 KiB0 B (0.0%)200.2 KiB✅
      Codexlong turn finished, work pagerows1501500 (0.0%)150✅
      Codexidle thread resumeWebSocket decoded65 B65 B0 B (0.0%)200 B✅
      Codexidle thread resumeWebSocket messages110 (0.0%)1✅
      Codexfleet shell snapshotHTTP decoded42.4 KiB42.4 KiB0 B (0.0%)53.7 KiB✅
      Codexfleet shell snapshotrows27270 (0.0%)27✅
      Codexfleet shell subscribeWebSocket decoded65 B65 B0 B (0.0%)200 B✅
      Codexfleet shell subscribeWebSocket messages110 (0.0%)1✅
      Codexfleet shell resume checkHTTP decoded18 B18 B0 B (0.0%)100 B✅
      Codexfleet shell resume checkrows0000✅
      Codexstale shell reloadHTTP decoded42.4 KiB42.4 KiB0 B (0.0%)53.7 KiB✅
      Codexstale shell reloadrows27270 (0.0%)27✅
      Codexstale shell resumeWebSocket decoded140 B140 B0 B (0.0%)400 B✅
      Codexstale shell resumeWebSocket messages220 (0.0%)2✅
      Codexstale todo list cold openHTTP decoded72.4 KiB72.4 KiB0 B (0.0%)83.0 KiB✅
      Codexstale todo list cold openrows75750 (0.0%)75✅
      Codexstale todo list cold opennodes220 (0.0%)4✅
      Codexstale todo list cold opensubagents220 (0.0%)4✅
      Codexstale todo list cold openplans110 (0.0%)2✅
      Codexstale todo list cold openproviderTurns110 (0.0%)2✅
      Codexstale todo list cold opencheckpoints110 (0.0%)2✅
      Claudeshort turn resumeWebSocket decoded10.6 KiB10.6 KiB0 B (0.0%)12.7 KiB✅
      Claudeshort turn resumeWebSocket messages220 (0.0%)2✅
      Claudelong turn running, cold openHTTP decoded283.3 KiB283.3 KiB0 B (0.0%)302.7 KiB✅
      Claudelong turn running, cold openrows75750 (0.0%)75✅
      Claudelong turn running, history pageHTTP decoded93.4 KiB93.4 KiB0 B (0.0%)107.4 KiB✅
      Claudelong turn running, history pagerows75750 (0.0%)75✅
      Claudelong turn finished, cold openHTTP decoded384.2 KiB384.2 KiB0 B (0.0%)410.2 KiB✅
      Claudelong turn finished, cold openrows1511510 (0.0%)151✅
      Claudelong turn finished, socket cold openWebSocket decoded384.3 KiB384.3 KiB0 B (0.0%)410.2 KiB✅
      Claudelong turn finished, socket cold openWebSocket messages220 (0.0%)2✅
      Claudelong turn finished, socket cold openrows1511510 (0.0%)151✅
      Claudelong turn finished, work pageHTTP decoded188.1 KiB188.1 KiB0 B (0.0%)200.2 KiB✅
      Claudelong turn finished, work pagerows1501500 (0.0%)150✅
      Claudeidle thread resumeWebSocket decoded66 B66 B0 B (0.0%)200 B✅
      Claudeidle thread resumeWebSocket messages110 (0.0%)1✅
      Claudefleet shell snapshotHTTP decoded43.0 KiB43.0 KiB0 B (0.0%)53.7 KiB✅
      Claudefleet shell snapshotrows27270 (0.0%)27✅
      Claudefleet shell subscribeWebSocket decoded66 B66 B0 B (0.0%)200 B✅
      Claudefleet shell subscribeWebSocket messages110 (0.0%)1✅
      Claudefleet shell resume checkHTTP decoded18 B18 B0 B (0.0%)100 B✅
      Claudefleet shell resume checkrows0000✅
      Claudestale shell reloadHTTP decoded43.0 KiB43.0 KiB0 B (0.0%)53.7 KiB✅
      Claudestale shell reloadrows27270 (0.0%)27✅
      Claudestale shell resumeWebSocket decoded140 B140 B0 B (0.0%)400 B✅
      Claudestale shell resumeWebSocket messages220 (0.0%)2✅
      Claudestale todo list cold openHTTP decoded73.4 KiB73.4 KiB0 B (0.0%)83.0 KiB✅
      Claudestale todo list cold openrows75750 (0.0%)75✅
      Claudestale todo list cold opennodes220 (0.0%)4✅
      Claudestale todo list cold opensubagents220 (0.0%)4✅
      Claudestale todo list cold openplans110 (0.0%)2✅
      Claudestale todo list cold openproviderTurns110 (0.0%)2✅
      Claudestale todo list cold opencheckpoints110 (0.0%)2✅

      Baseline: ed2a319 · PR result: bb43f3b · Source CI: success

      Scenario and decoded snapshot size

      10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

      • Codex decoded thread snapshot: 108.5 KiB
      • Claude decoded thread snapshot: 108.8 KiB

      Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

    5. Explain Change →

      No actionable comments were generated in the recent review.

      Walkthrough

      String answers in the work log now use a shared formatter to remove matching provider context envelopes and render review-comment markers without reference IDs. Tests cover history text, previews, and unmatched XML.

      Changes: Answer display

      Layer / File(s)Summary
      Format provider context for display
      packages/shared/src/composerContextReferences.ts
      A shared function removes a closed context envelope when it follows a marker with a matching reference ID. It renders the marker without the reference ID and preserves unmatched text.
      Use display text in work-log answers
      packages/client-runtime/src/work-log/userInput.ts, packages/client-runtime/src/work-log/userInput.test.ts
      String answers use the shared formatter. Tests check history and preview output, preservation of the chip label, and unchanged standalone XML.

      Estimated code review effort: 2 (Simple) | ~10 minutes

      Change: Bug fix · Severity of issue fixed: Low

      Suggested reviewers: maria-rcks, t3dotgg

      Merge Risk: Minimal · up to bb43f

      This change hides generated context blocks in answer history while keeping the review-comment label. Stored answers are not changed, so merge risk is minimal.

      Pre-merge checks: Passed 4

      Check nameStatusExplanation
      Title checkPassedThe title names the fix: answer history no longer shows provider context from review comments. It matches the main change in userInput.ts and composerContextReferences.ts, which hide the generated t3_…
      Linked Issues checkPassedIssue #35843 requires hiding the generated provider block from answer history and row previews for review-comment answers, keeping the answer readable, and preserving plain-text answers. …
      Out of Scope Changes checkPassedThe whole-PR diff against the merge base 95cb068c10 touches the shared helper composerContextReferences.ts (new function only), the client-runtime userInput.ts (one-line routing change in getQuestionA…
      Description checkPassedThe description follows the template sections (Problem, Change, Scope, Verification) and is detailed. It states the problem and reproduction on main, links issue #35843, and documents the change: a pr…
      Full details: Title check

      Explanation

      The title names the fix: answer history no longer shows provider context from review comments. It matches the main change in userInput.ts and composerContextReferences.ts, which hide the generated t3_context envelope and keep the review-comment label. The title follows the conventional commit form the template asks for.

      Full details: Linked Issues check

      Explanation

      Issue #35843 requires hiding the generated &lt;t3_context&gt; provider block from answer history and row previews for review-comment answers, keeping the answer readable, and preserving plain-text answers. At head, composerContextProviderTextForDisplay (packages/shared/src/composerContextReferences.ts) replaces a provider marker plus its envelope with '[Kind: label]' only when the envelope starts right after the marker and its id attribute equals the marker ref. Everything else is returned unchanged. getQuestionAnswerText (packages/client-runtime/src/work-log/userInput.ts) now routes string answers through this helper, so the shared answer-display path covers history and previews. Plain strings with no envelope pass through unchanged. The reported test runs cover the helper and the question-answer display. Not independently verified: that every web, desktop, and mobile consumer calls getQuestionAnswerText, and the pending evidence section in the PR description. These do not change the coding verdict.

      Full details: Out of Scope Changes check

      Explanation

      The whole-PR diff against the merge base 95cb068c10 touches the shared helper composerContextReferences.ts (new function only), the client-runtime userInput.ts (one-line routing change in getQuestionAnswerText), and userInput.test.ts (regression and preservation tests). Each change supports the #35843 objective. No unrelated files or behavior changes were identified in the reviewed evidence. Server-side storage is explicitly unchanged, which matches the issue scope.

      Full details: Description check

      Explanation

      The description follows the template sections (Problem, Change, Scope, Verification) and is detailed. It states the problem and reproduction on main, links issue #35843, and documents the change: a provider-envelope check tied to the marker ref, with typed XML and envelope-less markers left unchanged. Verification includes focused test runs (40 tests passed across 2 files), a throwaway test with two review-comment chips, and a check:changed run. Before/after screenshots and a recording are embedded, and the author lists model and harness. Remaining gaps: the Scope section does not say why this is a bug fix rather than a feature, and the Verification section lists desktop and mobile as not run. The description states these gaps openly. Both are minor, so the description is mostly complete.

      Finishing Touches

      Generate unit tests (beta)

      • Commit to this branch
      • Create a new PR
      • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
      Recent review info

      Run configuration

      Commits

      Reviewing files that changed from the base of the PR and between 8b23382191ba7c7db69978d6a310a4244c566741 and bb43f3bb6160d39394b553e94b09b93656097d5c.

      Files selected for processing: 3

      • packages/client-runtime/src/work-log/userInput.test.ts
      • packages/client-runtime/src/work-log/userInput.ts
      • packages/shared/src/composerContextReferences.ts

      Limit details: You’ve used all 10 included reviews currently available. Your 17 included PR review attempts over the past 7 days set your current allowance at 10 reviews per hour.

    6. juliusmarminge merged thisafff0d3

    7. juliusmarminge closed this

Sign in to join this conversation.