1# Composer context references
2
3 For maintainers. Using T3 Code? See [docs/user](../user/).
4
5Inline context references let a user message point at a typed payload from an exact position in
6its prose: an image, a file, a terminal excerpt, a picked page element, a preview annotation, a
7review comment, a file mention, or a skill. This document covers the wire contract and the pure
8codecs. Editor, rendering, and clipboard behaviour land in later PRs and get their own sections
9here as they arrive.
10
11## Two linked concepts
12
13A **context record** is the payload. It lives in `message.context.records`, keyed by a
14 `contextId`. Records never contain bytes: image and file records bind to an existing
15 `ChatAttachment` by id.
16A **context reference** is one occurrence in the document. It is a Markdown link in
17 `message.text` that carries only the kind and the `contextId`. Two references can point at one
18 record. A reference's label is display text and never identity.
19
20[`composerContext.ts`][contract] defines version 1 of the record union. Every record has
21`version`, `contextId`, `kind`, and `label`, plus kind-specific fields with bounded lengths. The
22union is open: a kind this build does not know decodes to `UnknownContextRecord` with its
23`payload` preserved, and known kinds are excluded from that member so a malformed image record
24fails its own schema rather than sliding through unchecked. `OrchestrationMessageContext` wraps
25the records with `ForwardCompatibleArray`, so one undecodable record is dropped instead of failing
26the whole message. The field is optional on the V2 conversation message and the commands that
27start or queue a turn; the orchestrator carries it through untouched.
28
29## Identity namespaces
30
31`ComposerContextId`: durable payload identity. Branded. Values match `[a-z0-9_-]+` and do not
32 require a `ctx_` prefix.
33`ComposerContextReferenceId` (`ref_…`): one document occurrence. Branded. Lives in editor state,
34 not on the wire and is regenerated when canonical Markdown is reparsed.
35`ChatAttachmentId`: the existing server-owned attachment resource. A record's `attachmentId` is
36 a binding, not the chip's identity, so upload normalization can rename the resource without
37 rewriting references.
38
39Clients mint context and reference ids. Shared code does not, because the Effect lint plugin
40rejects direct `crypto.randomUUID()` there.
41
42## Canonical reference syntax
43
44[`composerContextReferences.ts`][shared] owns the grammar:
45
47[label](t3-context://v1/<kind>/<contextId>)
48
50
51The parser accepts exactly the `t3-context:` scheme, the `v1` host, one kind segment matching
52`[a-z][a-z0-9-]{0,39}`, and one id segment matching `[a-z0-9_-]{1,128}` case-insensitively. Query
53strings, fragments, credentials, and extra segments are rejected. Labels are sanitized to survive
54a Markdown link (no brackets or line breaks, at most 200 characters, never empty). Links that fail
55to parse are ordinary text. `collectComposerInlineTokens` already rejects URI schemes for file
56links, so a context link is never mistaken for a mention.
57
58## Provider projection
59
60`projectComposerContextForProvider({ text, records })` builds what the provider reads:
61
62Every reference becomes an in-place marker: `[Image: shot.png; ref=ctx_1]`.
63A trailing `<t3_context version="1">` envelope holds one `<context kind id>` entry per unique
64 referenced id, in first-reference order. Records that are never referenced are not emitted.
65 A referenced id with no record becomes `<context … unavailable="true"/>`. Mention and skill
66 records produce a marker but no entry. Unknown kinds emit their payload as JSON.
67Captured text is data: any `<` that would open or close `t3_context` or `context` is escaped,
68 so a terminal line or PR comment cannot forge a record.
69Assistant citation links are expanded too (`expandAssistantCitationsForProvider`).
70
71Attachment bytes travel on the existing attachment channel; the envelope only carries metadata.
72Text without references or citations is returned unchanged.
73
74## Legacy messages
75
76Messages sent before this feature carry trailing `<terminal_context>`, `<element_context>`, and
77`<preview_annotation>` blocks, `<review_comment>` blocks, and U+FFFC terminal placeholders.
78[`composerContextLegacy.ts`][legacy] upgrades them in memory:
79
80Review blocks become references in place. Blocks that trailed the original text are appended
81 last, matching the old send order.
82Trailing blocks peel off the end in reverse send order (preview, element, terminal).
83Placeholders bind to terminal entries in order; entries without a placeholder are appended.
84Ids are deterministic (`legacy_<kind>_<n>`) so re-running the upgrade is idempotent.
85
86Event history is never rewritten. Existing web parsers in `apps/web/src/lib/` stay until the
87transcript renderer moves to records.
88
89[contract]: ../../packages/contracts/src/composerContext.ts
90[shared]: ../../packages/shared/src/composerContextReferences.ts
91[legacy]: ../../packages/shared/src/composerContextLegacy.ts
92
93## Editor model (web and desktop)
94
95`ComposerContextReferenceExtension` in
96[`ComposerPromptEditorTiptap.tsx`](../../apps/web/src/components/ComposerPromptEditorTiptap.tsx) is
97the inline Tiptap node for every context kind. It stores `kind`, `contextId`, `label`, and a
98per-occurrence `referenceId`, and its text content is the canonical link. The prompt string carries
99payload identity and position, so rebuilding the editor restores equivalent chips; it does not
100preserve the editor-local occurrence ids. Old drafts that used the U+FFFC ordinal placeholder
101migrate on hydration: placeholders bind to the terminal contexts in array order, then any context
102the prompt does not mention is prepended as a link.
103
104Records stay in the draft store's typed arrays for now. The editor builds a `Map` keyed by
105`contextId` from them (`composerContextRecordsFromDraft`) and provides it through
106`ComposerContextRecordsContext`. `ComposerContextReferenceChip` looks the record up and renders
107the kind's chip; an unknown kind or a missing record renders the unresolved chip instead of
108vanishing. Removing a chip removes only that occurrence; the composer's change handler compares
109the referenced ids against the draft array and drops records no chip points at.
110
111Version 1 deliberately keeps kind presentation explicit in each client instead of exposing a
112runtime handler registry. The contract and codecs are shared; web/desktop render rich chips and
113mobile renders the readable label. Add a registry only when a third-party or runtime-defined kind
114must provide behaviour that cannot ship with the client. Likewise, a durable occurrence id belongs
115in the canonical reference syntax only if a future feature needs to address one occurrence across
116serialization boundaries.
117
118Terminal context now follows the same send path as every other context record: the persisted
119message keeps its canonical link and structured record, while the provider projection replaces
120the link with a readable marker and includes the excerpt once in the context envelope. The legacy
121trailing `<terminal_context>` form is parsed only when reading messages sent by older clients.
122
123## Sending and reading messages
124
125The composer sends `message.text` as canonical prose with reference links and
126`message.context.records` built from the draft (`buildMessageContext` in
127`apps/web/src/lib/composerContextRecords.ts`). Expired terminal excerpts are dropped from both.
128The server projects provider text at turn start (`projectComposerContextForProvider`), so the persisted
129message stays readable and the provider receives markers plus one envelope.
130
131Review comments and preview annotations enter the draft through store mutators. A mounted composer
132registers a context insertion handler so panel-originated references land at its current or
133last-known caret; when no composer is mounted, the store appends them. Terminal excerpts and
134attachments use the same caret-first behavior. Removing a chip in the editor removes the record;
135removing a preview screenshot thumbnail removes its annotation and chip.
136
137The transcript resolves a message with `resolveUserMessageContext`: structured context is used as
138is, older messages are upgraded in memory. `ChatMarkdown` renders `t3-context://` links through
139`renderContextReference`, which the timeline maps to chips through the web context-presentation
140registry. The registry declares compact, details, and expanded capabilities for every known kind,
141rejects duplicate surface handlers, and provides the unresolved fallback. Terminal excerpts,
142elements, review comments, and preview annotations open structured details popovers; images and
143videos use the shared media modal. Mobile renders context links as their labels.
144
145Pull-request summaries currently travel as review-comment records with optional typed
146`pullRequest` metadata. The metadata is a snapshot of the number, title, URL, branches, state, and
147draft flag at attachment time. Web and desktop render the compact `#number` label and derive its
148status tone from that snapshot. Hover shows the snapshot details; activation resolves the URL
149against the current environment and opens the pull request in the thread's right panel. Records
150written before the metadata was added retain their legacy details and neutral pull-request tone.
151
152## Attachments
153
154Image and file records use the draft attachment's local id as `contextId` and carry an
155`attachmentId` binding. The composer sends the upload's pending id (or the local id on the
156data-URL path, via the optional `id` on `UploadChatImageAttachment`); the server's `Normalizer`
157rewrites every image and file record to the persisted id it assigns, so the stored message binds
158records to real resources. Optimistic rows bind to local ids and are replaced by the server copy.
159
160In the composer, attaching a file or image inserts a chip at the caret (appended when the editor
161cannot take input). Files exist only as chips: a file whose last chip is deleted is removed and
162its upload released, and drafts that predate references get a chip appended on hydration. Images
163keep the thumbnail shelf as their inventory; deleting a chip leaves the image, and removing a
164thumbnail that is still referenced asks for confirmation before removing both. Old drafts do not
165gain image chips.
166
167In the transcript an image chip opens the gallery preview, a video chip opens the media preview,
168and another file chip opens or downloads the file. The gallery still shows every image; file rows
169remain only for files no chip references.
170
171## Clipboard
172
173Every copy path writes the canonical Markdown as `text/plain` and, when the selection holds
174chips, a structured fragment under `web application/x-t3-context-fragment+json`
175(`ComposerContextClipboardFragment`: version, source environment/thread/message, records; no
176bytes, no URLs). Composer copy and cut add it through a Lexical command listener; transcript
177selection copy adds it from an `onCopyCapture` on the user message body while chips re-emit their
178links through `data-markdown-copy`; the whole-message button writes both through `ClipboardItem`
179and falls back to plain text.
180
181On paste the composer decodes the fragment before the plain text. Records the draft does not
182already hold are imported: terminal excerpts and review comments as they are, preview
183annotations rebuilt from their record, and images or files re-fetched through the source
184environment's asset URL and attached under a fresh local id, with the pasted link rewritten to
185that id. A pasted binary reads as an unresolved chip until its bytes arrive; a fragment from
186another environment leaves binaries unresolved. Rendered chips and sent messages are never
187mutated by a paste.
188
189Pasting across threads, projects, or environments uses the same path: the client mints an asset
190URL from the source environment, downloads the bytes, and attaches them here. There is no
191server-side clone; if the source is unreachable or the attachment is gone, a toast says so and
192the chip stays unresolved.
193
194## Ids, persistence, and the stash
195
196Producers keep their own id grammars; `toComposerContextId` folds anything outside
197`[a-z0-9_-]` into a slug plus a hash, deterministically, at the reference and record boundary.
198A preview annotation's context id is derived from `annotation-<id>` so it stays distinct from its
199screenshot image, whose attachment id is the annotation id; the record links the two through
200`screenshotContextId`.
201
202`projection_thread_messages.context_json` persists records, so a restart or projection reload
203keeps chips resolvable. Prompt stash entries carry `records` for terminal excerpts, review
204comments, and preview annotations; stashing moves them out of the draft and restoring imports
205them back through the same importer the paste path uses.
206
207Context produced by other panels reaches the caret through `setContextInsertionHandler`: a
208mounted composer registers an inserter for its draft and the store falls back to appending.