Skip to content
Source
Sign inSign up
TanStack/ai
Open History to see the last commit

ChatMiddleware.md

420 lines9.8 KB

1---
2id: ChatMiddleware
3title: ChatMiddleware
4---
5​
6# Interface: ChatMiddleware
7​
8Defined in: [packages/typescript/ai/src/activities/chat/middleware/types.ts:350](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/middleware/types.ts#L350)
9​
10Chat middleware interface.
11​
12All hooks are optional. Middleware is composed in array order:
13- `onConfig`: config piped through middlewares in order (first transform influences later)
14- `onChunk`: each output chunk is fed into the next middleware in order
15​
16## Examples
17​
18```ts
19const loggingMiddleware: ChatMiddleware = {
20 name: 'logging',
21 onStart(ctx) { console.log('Chat started', ctx.requestId) },
22 onChunk(ctx, chunk) { console.log('Chunk:', chunk.type) },
23 onFinish(ctx, info) { console.log('Done:', info.duration, 'ms') },
25```
26​
27```ts
28const redactionMiddleware: ChatMiddleware = {
29 name: 'redaction',
30 onChunk(ctx, chunk) {
31 if (chunk.type === 'TEXT_MESSAGE_CONTENT') {
32 return { ...chunk, delta: redact(chunk.delta) }
33 }
34 },
36```
37​
38## Properties
39​
40### name?
41​
42```ts
43optional name: string;
44```
45​
46Defined in: [packages/typescript/ai/src/activities/chat/middleware/types.ts:352](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/middleware/types.ts#L352)
47​
48Optional name for debugging and identification
49​
50***
51​
52### onAbort()?
53​
54```ts
55optional onAbort: (ctx, info) => void | Promise<void>;
56```
57​
58Defined in: [packages/typescript/ai/src/activities/chat/middleware/types.ts:472](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/middleware/types.ts#L472)
59​
60Called when the chat run is aborted.
61Exactly one of onFinish/onAbort/onError will be called per run.
62​
63#### Parameters
64​
65##### ctx
66​
67[`ChatMiddlewareContext`](ChatMiddlewareContext.md)
68​
69##### info
70​
71[`AbortInfo`](AbortInfo.md)
72​
73#### Returns
74​
75`void` \| `Promise`\<`void`\>
76​
77***
78​
79### onAfterToolCall()?
80​
81```ts
82optional onAfterToolCall: (ctx, info) => void | Promise<void>;
83```
84​
85Defined in: [packages/typescript/ai/src/activities/chat/middleware/types.ts:436](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/middleware/types.ts#L436)
86​
87Called after a tool execution completes (success or failure).
88​
89#### Parameters
90​
91##### ctx
92​
93[`ChatMiddlewareContext`](ChatMiddlewareContext.md)
94​
95##### info
96​
97[`AfterToolCallInfo`](AfterToolCallInfo.md)
98​
99#### Returns
100​
101`void` \| `Promise`\<`void`\>
102​
103***
104​
105### onBeforeToolCall()?
106​
107```ts
108optional onBeforeToolCall: (ctx, hookCtx) =>
109 | BeforeToolCallDecision
110| Promise<BeforeToolCallDecision>;
111```
112​
113Defined in: [packages/typescript/ai/src/activities/chat/middleware/types.ts:428](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/middleware/types.ts#L428)
114​
115Called before a tool is executed.
116Can observe, transform args, skip execution, or abort the run.
117​
118#### Parameters
119​
120##### ctx
121​
122[`ChatMiddlewareContext`](ChatMiddlewareContext.md)
123​
124##### hookCtx
125​
126[`ToolCallHookContext`](ToolCallHookContext.md)
127​
128#### Returns
129​
130 \| [`BeforeToolCallDecision`](../type-aliases/BeforeToolCallDecision.md)
131 \| `Promise`\<[`BeforeToolCallDecision`](../type-aliases/BeforeToolCallDecision.md)\>
132​
133***
134​
135### onChunk()?
136​
137```ts
138optional onChunk: (ctx, chunk) =>
139 | void
140 | AGUIEvent
141 | AGUIEvent[]
142 | Promise<void | AGUIEvent | AGUIEvent[] | null>
143 | null;
144```
145​
146Defined in: [packages/typescript/ai/src/activities/chat/middleware/types.ts:414](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/middleware/types.ts#L414)
147​
148Called for every chunk yielded by chat().
149Can observe, transform, expand, or drop chunks.
150​
151#### Parameters
152​
153##### ctx
154​
155[`ChatMiddlewareContext`](ChatMiddlewareContext.md)
156​
157##### chunk
158​
159[`AGUIEvent`](../type-aliases/AGUIEvent.md)
160​
161#### Returns
162​
163 \| `void`
164 \| [`AGUIEvent`](../type-aliases/AGUIEvent.md)
165 \| [`AGUIEvent`](../type-aliases/AGUIEvent.md)[]
166 \| `Promise`\<void \| AGUIEvent \| AGUIEvent\[\] \| null\>
167 \| `null`
168​
169void (pass through), chunk (replace), chunk[] (expand), null (drop)
170​
171***
172​
173### onConfig()?
174​
175```ts
176optional onConfig: (ctx, config) =>
177 | void
178 | Partial<ChatMiddlewareConfig>
179 | Promise<
180 | void
181 | Partial<ChatMiddlewareConfig>
182 | null>
183 | null;
184```
185​
186Defined in: [packages/typescript/ai/src/activities/chat/middleware/types.ts:361](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/middleware/types.ts#L361)
187​
188Called to observe or transform the chat configuration.
189Called at init and at the beginning of each agent iteration.
190​
191Return a partial config to merge with the current config, or void to pass through.
192Only the fields you return are overwritten — everything else is preserved.
193​
194#### Parameters
195​
196##### ctx
197​
198[`ChatMiddlewareContext`](ChatMiddlewareContext.md)
199​
200##### config
201​
202[`ChatMiddlewareConfig`](ChatMiddlewareConfig.md)
203​
204#### Returns
205​
206 \| `void`
207 \| `Partial`\<[`ChatMiddlewareConfig`](ChatMiddlewareConfig.md)\>
208 \| `Promise`\<
209 \| `void`
210 \| `Partial`\<[`ChatMiddlewareConfig`](ChatMiddlewareConfig.md)\>
211 \| `null`\>
212 \| `null`
213​
214***
215​
216### onError()?
217​
218```ts
219optional onError: (ctx, info) => void | Promise<void>;
220```
221​
222Defined in: [packages/typescript/ai/src/activities/chat/middleware/types.ts:481](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/middleware/types.ts#L481)
223​
224Called when the chat run encounters an unhandled error.
225Exactly one of onFinish/onAbort/onError will be called per run.
226​
227#### Parameters
228​
229##### ctx
230​
231[`ChatMiddlewareContext`](ChatMiddlewareContext.md)
232​
233##### info
234​
235[`ErrorInfo`](ErrorInfo.md)
236​
237#### Returns
238​
239`void` \| `Promise`\<`void`\>
240​
241***
242​
243### onFinish()?
244​
245```ts
246optional onFinish: (ctx, info) => void | Promise<void>;
247```
248​
249Defined in: [packages/typescript/ai/src/activities/chat/middleware/types.ts:463](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/middleware/types.ts#L463)
250​
251Called when the chat run completes normally.
252Exactly one of onFinish/onAbort/onError will be called per run.
253​
254#### Parameters
255​
256##### ctx
257​
258[`ChatMiddlewareContext`](ChatMiddlewareContext.md)
259​
260##### info
261​
262[`FinishInfo`](FinishInfo.md)
263​
264#### Returns
265​
266`void` \| `Promise`\<`void`\>
267​
268***
269​
270### onIteration()?
271​
272```ts
273optional onIteration: (ctx, info) => void | Promise<void>;
274```
275​
276Defined in: [packages/typescript/ai/src/activities/chat/middleware/types.ts:403](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/middleware/types.ts#L403)
277​
278Called at the start of each agent loop iteration, after a new assistant message ID
279is created. Use this to observe iteration boundaries.
280​
281#### Parameters
282​
283##### ctx
284​
285[`ChatMiddlewareContext`](ChatMiddlewareContext.md)
286​
287##### info
288​
289[`IterationInfo`](IterationInfo.md)
290​
291#### Returns
292​
293`void` \| `Promise`\<`void`\>
294​
295***
296​
297### onStart()?
298​
299```ts
300optional onStart: (ctx) => void | Promise<void>;
301```
302​
303Defined in: [packages/typescript/ai/src/activities/chat/middleware/types.ts:397](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/middleware/types.ts#L397)
304​
305Called when the chat run starts (after initial onConfig).
306​
307#### Parameters
308​
309##### ctx
310​
311[`ChatMiddlewareContext`](ChatMiddlewareContext.md)
312​
313#### Returns
314​
315`void` \| `Promise`\<`void`\>
316​
317***
318​
319### onStructuredOutputConfig()?
320​
321```ts
322optional onStructuredOutputConfig: (ctx, config) =>
323 | void
324 | Partial<StructuredOutputMiddlewareConfig>
325 | Promise<
326 | void
327 | Partial<StructuredOutputMiddlewareConfig>
328 | null>
329 | null;
330```
331​
332Defined in: [packages/typescript/ai/src/activities/chat/middleware/types.ts:385](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/middleware/types.ts#L385)
333​
334Called at the start of the final structured-output call (when the chat
335was invoked with outputSchema). Pipes through middleware in order, like
336onConfig, but with access to the JSON Schema being sent to the provider.
337​
338Return a partial to shallow-merge into the current config, or void to
339pass through.
340​
341Fires BEFORE onConfig at the structured-output boundary. onConfig also
342re-fires at the same boundary with ctx.phase === 'structuredOutput',
343receiving the post-onStructuredOutputConfig view of the config (minus
344outputSchema). Use onConfig for general-purpose transforms that apply
345to every adapter call; use this hook when you need to transform the
346outputSchema or apply structured-output-specific behavior.
347​
348#### Parameters
349​
350##### ctx
351​
352[`ChatMiddlewareContext`](ChatMiddlewareContext.md)
353​
354##### config
355​
356[`StructuredOutputMiddlewareConfig`](StructuredOutputMiddlewareConfig.md)
357​
358#### Returns
359​
360 \| `void`
361 \| `Partial`\<[`StructuredOutputMiddlewareConfig`](StructuredOutputMiddlewareConfig.md)\>
362 \| `Promise`\<
363 \| `void`
364 \| `Partial`\<[`StructuredOutputMiddlewareConfig`](StructuredOutputMiddlewareConfig.md)\>
365 \| `null`\>
366 \| `null`
367​
368***
369​
370### onToolPhaseComplete()?
371​
372```ts
373optional onToolPhaseComplete: (ctx, info) => void | Promise<void>;
374```
375​
376Defined in: [packages/typescript/ai/src/activities/chat/middleware/types.ts:445](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/middleware/types.ts#L445)
377​
378Called after all tool calls in an iteration have been processed.
379Provides aggregate data about tool execution results, approvals, and client tools.
380​
381#### Parameters
382​
383##### ctx
384​
385[`ChatMiddlewareContext`](ChatMiddlewareContext.md)
386​
387##### info
388​
389[`ToolPhaseCompleteInfo`](ToolPhaseCompleteInfo.md)
390​
391#### Returns
392​
393`void` \| `Promise`\<`void`\>
394​
395***
396​
397### onUsage()?
398​
399```ts
400optional onUsage: (ctx, usage) => void | Promise<void>;
401```
402​
403Defined in: [packages/typescript/ai/src/activities/chat/middleware/types.ts:454](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/activities/chat/middleware/types.ts#L454)
404​
405Called when usage data is available from a RUN_FINISHED chunk.
406Called once per model iteration that reports usage.
407​
408#### Parameters
409​
410##### ctx
411​
412[`ChatMiddlewareContext`](ChatMiddlewareContext.md)
413​
414##### usage
415​
416[`UsageInfo`](UsageInfo.md)
417​
418#### Returns
419​
420`void` \| `Promise`\<`void`\>