Skip to content
Source
Sign inSign up
TanStack/ai
Open History to see the last commit

ChatMiddleware.md

511 lines12.1 KB

1---
2id: ChatMiddleware
3title: ChatMiddleware
4---
5​
6# Interface: ChatMiddleware\<TContext\>
7​
8Defined in: [packages/ai/src/activities/chat/middleware/types.ts:421](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L421)
9​
10Chat middleware interface.
11​
12All hooks are optional. Middleware is composed in array order:
13- `onConfig`: config piped through middlewares in order (first transform influences later)
14- `onChunk`: each output chunk is fed into the next middleware in order
15​
16## Examples
17​
18```ts
19const loggingMiddleware: ChatMiddleware = {
20 name: 'logging',
21 onStart(ctx) { console.log('Chat started', ctx.requestId) },
22 onChunk(ctx, chunk) { console.log('Chunk:', chunk.type) },
23 onFinish(ctx, info) { console.log('Done:', info.duration, 'ms') },
25```
26​
27```ts
28const redactionMiddleware: ChatMiddleware = {
29 name: 'redaction',
30 onChunk(ctx, chunk) {
31 if (chunk.type === 'TEXT_MESSAGE_CONTENT') {
32 return { ...chunk, delta: redact(chunk.delta) }
33 }
34 },
36```
37​
38## Extended by
39​
40- [`DefinedChatMiddleware`](DefinedChatMiddleware.md)
41​
42## Type Parameters
43​
44### TContext
45​
46`TContext` = `unknown`
47​
48## Properties
49​
50### name?
51​
52```ts
53optional name: string;
54```
55​
56Defined in: [packages/ai/src/activities/chat/middleware/types.ts:423](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L423)
57​
58Optional name for debugging and identification
59​
60***
61​
62### onAbort()?
63​
64```ts
65optional onAbort: (ctx, info) => void | Promise<void>;
66```
67​
68Defined in: [packages/ai/src/activities/chat/middleware/types.ts:573](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L573)
69​
70Called when the chat run is aborted.
71Exactly one of onFinish/onAbort/onError will be called per run.
72​
73#### Parameters
74​
75##### ctx
76​
77[`ChatMiddlewareContext`](ChatMiddlewareContext.md)\<`TContext`\>
78​
79##### info
80​
81[`AbortInfo`](AbortInfo.md)
82​
83#### Returns
84​
85`void` \| `Promise`\<`void`\>
86​
87***
88​
89### onAfterToolCall()?
90​
91```ts
92optional onAfterToolCall: (ctx, info) => void | Promise<void>;
93```
94​
95Defined in: [packages/ai/src/activities/chat/middleware/types.ts:537](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L537)
96​
97Called after a tool execution completes (success or failure).
98​
99#### Parameters
100​
101##### ctx
102​
103[`ChatMiddlewareContext`](ChatMiddlewareContext.md)\<`TContext`\>
104​
105##### info
106​
107[`AfterToolCallInfo`](AfterToolCallInfo.md)
108​
109#### Returns
110​
111`void` \| `Promise`\<`void`\>
112​
113***
114​
115### onBeforeToolCall()?
116​
117```ts
118optional onBeforeToolCall: (ctx, hookCtx) =>
119 | BeforeToolCallDecision
120| Promise<BeforeToolCallDecision>;
121```
122​
123Defined in: [packages/ai/src/activities/chat/middleware/types.ts:529](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L529)
124​
125Called before a tool is executed.
126Can observe, transform args, skip execution, or abort the run.
127​
128#### Parameters
129​
130##### ctx
131​
132[`ChatMiddlewareContext`](ChatMiddlewareContext.md)\<`TContext`\>
133​
134##### hookCtx
135​
136[`ToolCallHookContext`](ToolCallHookContext.md)
137​
138#### Returns
139​
140 \| [`BeforeToolCallDecision`](../type-aliases/BeforeToolCallDecision.md)
141 \| `Promise`\<[`BeforeToolCallDecision`](../type-aliases/BeforeToolCallDecision.md)\>
142​
143***
144​
145### onChunk()?
146​
147```ts
148optional onChunk: (ctx, chunk) =>
149 | void
150 | AGUIEvent
151 | AGUIEvent[]
152 | Promise<void | AGUIEvent | AGUIEvent[] | null>
153 | null;
154```
155​
156Defined in: [packages/ai/src/activities/chat/middleware/types.ts:515](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L515)
157​
158Called for every chunk yielded by chat().
159Can observe, transform, expand, or drop chunks.
160​
161#### Parameters
162​
163##### ctx
164​
165[`ChatMiddlewareContext`](ChatMiddlewareContext.md)\<`TContext`\>
166​
167##### chunk
168​
169[`AGUIEvent`](../type-aliases/AGUIEvent.md)
170​
171#### Returns
172​
173 \| `void`
174 \| [`AGUIEvent`](../type-aliases/AGUIEvent.md)
175 \| [`AGUIEvent`](../type-aliases/AGUIEvent.md)[]
176 \| `Promise`\<void \| AGUIEvent \| AGUIEvent\[\] \| null\>
177 \| `null`
178​
179void (pass through), chunk (replace), chunk[] (expand), null (drop)
180​
181***
182​
183### onConfig()?
184​
185```ts
186optional onConfig: (ctx, config) =>
187 | void
188 | Partial<ChatMiddlewareConfig>
189 | Promise<
190 | void
191 | Partial<ChatMiddlewareConfig>
192 | null>
193 | null;
194```
195​
196Defined in: [packages/ai/src/activities/chat/middleware/types.ts:462](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L462)
197​
198Called to observe or transform the chat configuration.
199Called at init and at the beginning of each agent iteration.
200​
201Return a partial config to merge with the current config, or void to pass through.
202Only the fields you return are overwritten — everything else is preserved.
203​
204#### Parameters
205​
206##### ctx
207​
208[`ChatMiddlewareContext`](ChatMiddlewareContext.md)\<`TContext`\>
209​
210##### config
211​
212[`ChatMiddlewareConfig`](ChatMiddlewareConfig.md)
213​
214#### Returns
215​
216 \| `void`
217 \| `Partial`\<[`ChatMiddlewareConfig`](ChatMiddlewareConfig.md)\>
218 \| `Promise`\<
219 \| `void`
220 \| `Partial`\<[`ChatMiddlewareConfig`](ChatMiddlewareConfig.md)\>
221 \| `null`\>
222 \| `null`
223​
224***
225​
226### onError()?
227​
228```ts
229optional onError: (ctx, info) => void | Promise<void>;
230```
231​
232Defined in: [packages/ai/src/activities/chat/middleware/types.ts:582](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L582)
233​
234Called when the chat run encounters an unhandled error.
235Exactly one of onFinish/onAbort/onError will be called per run.
236​
237#### Parameters
238​
239##### ctx
240​
241[`ChatMiddlewareContext`](ChatMiddlewareContext.md)\<`TContext`\>
242​
243##### info
244​
245[`ErrorInfo`](ErrorInfo.md)
246​
247#### Returns
248​
249`void` \| `Promise`\<`void`\>
250​
251***
252​
253### onFinish()?
254​
255```ts
256optional onFinish: (ctx, info) => void | Promise<void>;
257```
258​
259Defined in: [packages/ai/src/activities/chat/middleware/types.ts:564](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L564)
260​
261Called when the chat run completes normally.
262Exactly one of onFinish/onAbort/onError will be called per run.
263​
264#### Parameters
265​
266##### ctx
267​
268[`ChatMiddlewareContext`](ChatMiddlewareContext.md)\<`TContext`\>
269​
270##### info
271​
272[`FinishInfo`](FinishInfo.md)
273​
274#### Returns
275​
276`void` \| `Promise`\<`void`\>
277​
278***
279​
280### onIteration()?
281​
282```ts
283optional onIteration: (ctx, info) => void | Promise<void>;
284```
285​
286Defined in: [packages/ai/src/activities/chat/middleware/types.ts:504](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L504)
287​
288Called at the start of each agent loop iteration, after a new assistant message ID
289is created. Use this to observe iteration boundaries.
290​
291#### Parameters
292​
293##### ctx
294​
295[`ChatMiddlewareContext`](ChatMiddlewareContext.md)\<`TContext`\>
296​
297##### info
298​
299[`IterationInfo`](IterationInfo.md)
300​
301#### Returns
302​
303`void` \| `Promise`\<`void`\>
304​
305***
306​
307### onStart()?
308​
309```ts
310optional onStart: (ctx) => void | Promise<void>;
311```
312​
313Defined in: [packages/ai/src/activities/chat/middleware/types.ts:498](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L498)
314​
315Called when the chat run starts (after initial onConfig).
316​
317#### Parameters
318​
319##### ctx
320​
321[`ChatMiddlewareContext`](ChatMiddlewareContext.md)\<`TContext`\>
322​
323#### Returns
324​
325`void` \| `Promise`\<`void`\>
326​
327***
328​
329### onStructuredOutputConfig()?
330​
331```ts
332optional onStructuredOutputConfig: (ctx, config) =>
333 | void
334 | Partial<StructuredOutputMiddlewareConfig>
335 | Promise<
336 | void
337 | Partial<StructuredOutputMiddlewareConfig>
338 | null>
339 | null;
340```
341​
342Defined in: [packages/ai/src/activities/chat/middleware/types.ts:486](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L486)
343​
344Called at the start of the final structured-output call (when the chat
345was invoked with outputSchema). Pipes through middleware in order, like
346onConfig, but with access to the JSON Schema being sent to the provider.
347​
348Return a partial to shallow-merge into the current config, or void to
349pass through.
350​
351Fires BEFORE onConfig at the structured-output boundary. onConfig also
352re-fires at the same boundary with ctx.phase === 'structuredOutput',
353receiving the post-onStructuredOutputConfig view of the config (minus
354outputSchema). Use onConfig for general-purpose transforms that apply
355to every adapter call; use this hook when you need to transform the
356outputSchema or apply structured-output-specific behavior.
357​
358#### Parameters
359​
360##### ctx
361​
362[`ChatMiddlewareContext`](ChatMiddlewareContext.md)\<`TContext`\>
363​
364##### config
365​
366[`StructuredOutputMiddlewareConfig`](StructuredOutputMiddlewareConfig.md)
367​
368#### Returns
369​
370 \| `void`
371 \| `Partial`\<[`StructuredOutputMiddlewareConfig`](StructuredOutputMiddlewareConfig.md)\>
372 \| `Promise`\<
373 \| `void`
374 \| `Partial`\<[`StructuredOutputMiddlewareConfig`](StructuredOutputMiddlewareConfig.md)\>
375 \| `null`\>
376 \| `null`
377​
378***
379​
380### onToolPhaseComplete()?
381​
382```ts
383optional onToolPhaseComplete: (ctx, info) => void | Promise<void>;
384```
385​
386Defined in: [packages/ai/src/activities/chat/middleware/types.ts:546](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L546)
387​
388Called after all tool calls in an iteration have been processed.
389Provides aggregate data about tool execution results, approvals, and client tools.
390​
391#### Parameters
392​
393##### ctx
394​
395[`ChatMiddlewareContext`](ChatMiddlewareContext.md)\<`TContext`\>
396​
397##### info
398​
399[`ToolPhaseCompleteInfo`](ToolPhaseCompleteInfo.md)
400​
401#### Returns
402​
403`void` \| `Promise`\<`void`\>
404​
405***
406​
407### onUsage()?
408​
409```ts
410optional onUsage: (ctx, usage) => void | Promise<void>;
411```
412​
413Defined in: [packages/ai/src/activities/chat/middleware/types.ts:555](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L555)
414​
415Called when usage data is available from a RUN_FINISHED chunk.
416Called once per model iteration that reports usage.
417​
418#### Parameters
419​
420##### ctx
421​
422[`ChatMiddlewareContext`](ChatMiddlewareContext.md)\<`TContext`\>
423​
424##### usage
425​
426[`UsageInfo`](UsageInfo.md)
427​
428#### Returns
429​
430`void` \| `Promise`\<`void`\>
431​
432***
433​
434### optionalRequires?
435​
436```ts
437optional optionalRequires: readonly CapabilityHandle[];
438```
439​
440Defined in: [packages/ai/src/activities/chat/middleware/types.ts:445](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L445)
441​
442Capabilities this middleware uses if present but does not require.
443Non-gating: never causes a validation error. Read with
444`getX(ctx, { optional: true })`.
445​
446***
447​
448### provides?
449​
450```ts
451optional provides: readonly CapabilityHandle[];
452```
453​
454Defined in: [packages/ai/src/activities/chat/middleware/types.ts:438](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L438)
455​
456Capabilities this middleware provides. Each declared capability MUST be
457provided (via its `provide` accessor) inside `setup`, or `chat()` throws
458after the setup phase.
459​
460***
461​
462### requires?
463​
464```ts
465optional requires: readonly CapabilityHandle[];
466```
467​
468Defined in: [packages/ai/src/activities/chat/middleware/types.ts:431](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L431)
469​
470Capabilities this middleware requires. `chat()` validates that some
471middleware (or the adapter) provides each one; unsatisfied requirements are
472a compile-time error (array coverage / builder) and a runtime error before
473the adapter runs.
474​
475***
476​
477### sandbox?
478​
479```ts
480optional sandbox: ChatSandboxHooks<TContext>;
481```
482​
483Defined in: [packages/ai/src/activities/chat/middleware/types.ts:591](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L591)
484​
485Sandbox file-event hooks. Fire when a sandbox provided by `withSandbox` is
486active during the run and a file is created/changed/deleted. Server-side.
487​
488***
489​
490### setup()?
491​
492```ts
493optional setup: (ctx) => void | Promise<void>;
494```
495​
496Defined in: [packages/ai/src/activities/chat/middleware/types.ts:453](https://github.com/TanStack/ai/blob/main/packages/ai/src/activities/chat/middleware/types.ts#L453)
497​
498Provisioning hook. Runs FIRST — before `onConfig` (init) — across all
499middleware in array order. Use it to call `provide` accessors so later
500middleware (`onConfig` onward) can consume the capabilities. Receives the
501stable context; does NOT receive the mutable config.
502​
503#### Parameters
504​
505##### ctx
506​
507[`ChatMiddlewareContext`](ChatMiddlewareContext.md)\<`TContext`\>
508​
509#### Returns
510​
511`void` \| `Promise`\<`void`\>