Skip to content
Source
Sign inSign up
TanStack/ai
Open History to see the last commit

runtime-context.md

317 lines9.9 KB

1---
2title: Runtime Context
3id: runtime-context
4order: 2
5description: "Pass typed runtime dependencies to TanStack AI tools and middleware without serializing them to the model or AG-UI protocol context."
6keywords:
7 - tanstack ai
8 - runtime context
9 - typed context
10 - tools context
11 - middleware context
12 - ag-ui context
13---
14​
15Runtime context is application state you pass to tool implementations and middleware. Use it for request-scoped or client-local dependencies such as authenticated users, database clients, tenancy, feature flags, audit loggers, or browser services.
16​
17Runtime context is not prompt context and is not the AG-UI `RunAgentInput.context` field. It is never sent to the model automatically.
18​
19## How Type Safety Works
20​
21Runtime context is checked from the point of view of the code that consumes it. Tools and middleware declare the context shape they need, and `chat()`, `ChatClient`, and framework hooks check that the `context` value you pass satisfies those requirements.
22​
23The source of truth is:
24​
25- `toolDefinition(...).server<TContext>(...)` for server tools.
26- `toolDefinition(...).client<TContext>(...)` for client tools.
27- `ChatMiddleware<TContext>` for middleware.
28​
29This means the context value is the implementation detail you provide at runtime, while tools and middleware are the contract. TanStack AI infers the required context from every typed tool and middleware in the call, merges those requirements, and checks your `context` option against the result.
30​
31```typescript
32import { chat, toServerSentEventsResponse, toolDefinition, type ChatMiddleware } from "@tanstack/ai";
33import { openaiText } from "@tanstack/ai-openai";
34​
35type UserContext = {
36 userId: string;
37};
38​
39type TenantContext = {
40 tenantId: string;
41};
42​
43const currentUserTool = toolDefinition({
44 name: "current_user",
45 description: "Read the current user",
46}).server<UserContext>((_input, ctx) => {
47 return { userId: ctx.context.userId };
48});
49​
50const tenantMiddleware: ChatMiddleware<TenantContext> = {
51 name: "tenant",
52 onStart(ctx) {
53 console.log(ctx.context.tenantId);
54 },
55};
56​
57export async function POST(request: Request) {
58 const { messages } = await request.json();
59 const stream = chat({
60 adapter: openaiText("gpt-5.5"),
61 messages,
62 tools: [currentUserTool],
63 middleware: [tenantMiddleware],
64 context: {
65 userId: "user_123",
66 tenantId: "tenant_456",
67 },
68 });
69 return toServerSentEventsResponse(stream);
71```
72​
73In this example, the tool requires `UserContext` and the middleware requires `TenantContext`, so the `context` value must satisfy both. If you remove `tenantId`, TypeScript reports an error because `tenantMiddleware` declared that it needs it.
74​
75This is intentional. The `context` object alone should not decide what tools and middleware are allowed to read. The consumers define their requirements, and the call site proves that it supplied them. Untyped tools and middleware still work; they receive `unknown` context and do not force a `context` option.
76​
77This inference also works when reusable tools or middleware are declared outside the `chat()` call and passed in as arrays. A consumer can opt into optional runtime context by declaring `TContext | undefined`; then the `context` option can be omitted when all typed consumers accept `undefined`. If a context value is provided, it still has to satisfy every typed consumer.
78​
79The same rule applies on the client:
80​
81```typescript
82import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";
83import { toolDefinition } from "@tanstack/ai";
84​
85type ClientRuntimeContext = {
86 currentTabId: string;
87};
88​
89const inspectClientContext = toolDefinition({
90 name: "inspect_client_context",
91 description: "Inspect local browser context",
92}).client<ClientRuntimeContext & { mode: "debug" }>((_input, ctx) => {
93 return {
94 tabId: ctx.context.currentTabId,
95 mode: ctx.context.mode,
96 };
97});
98​
99useChat({
100 connection: fetchServerSentEvents("/api/chat"),
101 tools: [inspectClientContext],
102 context: {
103 currentTabId: "settings",
104 mode: "debug",
105 },
106});
107```
108​
109Because the client tool declares `ClientRuntimeContext & { mode: "debug" }`, `useChat()` requires a `context` value with both `currentTabId` and the literal `mode: "debug"`.
110​
111## Server Runtime Context
112​
113Define the context type once, use it in server tools and middleware, then pass the matching `context` value to `chat()`.
114​
115```typescript
116import {
117 chat,
118 toServerSentEventsResponse,
119 toolDefinition,
120 type ChatMiddleware,
121} from "@tanstack/ai";
122import { openaiText } from "@tanstack/ai-openai";
123import { z } from "zod";
124import { requireUser, db } from "./auth";
125​
126type AppContext = {
127 userId: string;
128 tenantId: string;
129 db: {
130 notes: {
131 findMany(args: { userId: string; tenantId: string }): Promise<Array<{ title: string }>>;
132 };
133 };
134};
135​
136const listNotes = toolDefinition({
137 name: "list_notes",
138 description: "List notes for the current user",
139 inputSchema: z.object({}),
140 outputSchema: z.array(z.object({ title: z.string() })),
141}).server<AppContext>(async (_input, ctx) => {
142 return ctx.context.db.notes.findMany({
143 userId: ctx.context.userId,
144 tenantId: ctx.context.tenantId,
145 });
146});
147​
148const auditMiddleware: ChatMiddleware<AppContext> = {
149 name: "audit",
150 onStart(ctx) {
151 console.log("chat started", {
152 requestId: ctx.requestId,
153 userId: ctx.context.userId,
154 tenantId: ctx.context.tenantId,
155 });
156 },
157};
158​
159export async function POST(request: Request) {
160 const { messages } = await request.json();
161 const user = await requireUser(request);
162​
163 const stream = chat({
164 adapter: openaiText("gpt-5.5"),
165 messages,
166 tools: [listNotes],
167 middleware: [auditMiddleware],
168 context: {
169 userId: user.id,
170 tenantId: user.tenantId,
171 db,
172 },
173 });
174​
175 return toServerSentEventsResponse(stream);
177```
178​
179When any tool or middleware in a `chat()` call declares a concrete context type, TypeScript checks the `context` value against that type. Existing untyped tools and middleware continue to work; their `ctx.context` type remains `unknown`.
180​
181## Client Runtime Context
182​
183Client runtime context is local to `ChatClient` and framework hooks. It is passed to client tool implementations and is not serialized to the server.
184​
185```typescript
186import { createChatClientOptions } from "@tanstack/ai-client";
187import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";
188import { toolDefinition } from "@tanstack/ai";
189​
190type ClientContext = {
191 currentTabId: string;
192 toast(message: string): void;
193};
194​
195const notifyUser = toolDefinition({
196 name: "notify_user",
197 description: "Show a notification in the current browser tab",
198}).client<ClientContext>((_input, ctx) => {
199 ctx.context.toast(`Updated tab ${ctx.context.currentTabId}`);
200 return { ok: true };
201});
202​
203const chatOptions = createChatClientOptions({
204 connection: fetchServerSentEvents("/api/chat"),
205 tools: [notifyUser],
206 context: {
207 currentTabId: "settings",
208 toast: (message) => window.alert(message),
209 },
210});
211​
212const chat = useChat(chatOptions);
213```
214​
215Use client context for local dependencies only. Do not put values there expecting the server to receive them.
216​
217## Client-to-Server Handoff
218​
219To send serializable client data to the server, use `forwardedProps`, validate it in your route, and explicitly map it into the server runtime context.
220​
221```typescript
222import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";
223import { toolDefinition } from "@tanstack/ai";
224​
225type ClientContext = {
226 currentTabId: string;
227 toast(message: string): void;
228};
229​
230const notifyUser = toolDefinition({
231 name: "notify_user",
232 description: "Show a notification in the current browser tab",
233}).client<ClientContext>((_input, ctx) => {
234 ctx.context.toast(`Updated tab ${ctx.context.currentTabId}`);
235 return { ok: true };
236});
237​
238// Client
239useChat({
240 connection: fetchServerSentEvents("/api/chat"),
241 tools: [notifyUser],
242 forwardedProps: {
243 tenantId: "tenant_456",
244 },
245 context: {
246 currentTabId: "settings",
247 toast: (message) => window.alert(message),
248 },
249});
250```
251​
252```typescript
253// Server
254import {
255 chat,
256 chatParamsFromRequest,
257 toServerSentEventsResponse,
258} from "@tanstack/ai";
259import { openaiText } from "@tanstack/ai-openai";
260import { requireUser } from "./auth";
261import { serverTools } from "./tools";
262​
263type AppContext = {
264 userId: string;
265 tenantId: string;
266};
267​
268export async function POST(request: Request) {
269 const params = await chatParamsFromRequest(request);
270 const user = await requireUser(request);
271​
272 const tenantId =
273 typeof params.forwardedProps.tenantId === "string"
274 ? params.forwardedProps.tenantId
275 : user.defaultTenantId;
276​
277 const stream = chat({
278 adapter: openaiText("gpt-5.5"),
279 messages: params.messages,
280 tools: serverTools,
281 context: {
282 userId: user.id,
283 tenantId,
284 } satisfies AppContext,
285 });
286​
287 return toServerSentEventsResponse(stream);
289```
290​
291Treat `forwardedProps` as client-controlled input. Validate and allowlist every field before using it to build server runtime context.
292​
293## AG-UI Context
294​
295AG-UI also defines `RunAgentInput.context`, usually as protocol-level context entries for interoperable agents. TanStack AI surfaces that field through `chatParamsFromRequest`, but it is separate from `chat({ context })`.
296​
297TanStack AI does not automatically copy AG-UI `params.aguiContext` into runtime context. If you want to use AG-UI context values, validate and map them yourself. `params.context` is a deprecated alias of `params.aguiContext` kept for backward compatibility.
298​
299```typescript
300import { chat, chatParamsFromRequest, toServerSentEventsResponse } from "@tanstack/ai";
301import { openaiText } from "@tanstack/ai-openai";
302import { buildRuntimeContextFrom } from "./context";
303import { serverTools } from "./tools";
304​
305export async function POST(request: Request) {
306 const params = await chatParamsFromRequest(request);
307​
308 const stream = chat({
309 adapter: openaiText("gpt-5.5"),
310 messages: params.messages,
311 tools: serverTools,
312 context: buildRuntimeContextFrom(params.aguiContext),
313 });
314​
315 return toServerSentEventsResponse(stream);
317```