Open History to see the last commit
1---
2title: Code Mode
3id: code-mode
4order: 1
5description: "Let LLMs write and execute TypeScript programs that orchestrate tools in a secure sandbox with TanStack AI Code Mode — fewer loops, richer logic."
6keywords:
7 - tanstack ai
8 - code mode
9 - sandbox
10 - typescript execution
11 - tool orchestration
12 - execute_typescript
13 - ai agents
14---
15
16Code Mode lets an LLM write and execute TypeScript programs inside a secure sandbox. Instead of making one tool call at a time, the model writes a short script that orchestrates multiple tools with loops, conditionals, `Promise.all`, and data transformations — then returns a single result.
17
18You already have a chat app that uses [tools](../tools/tools). By the end of this guide, you'll have Code Mode set up so the LLM can compose those tools in TypeScript and execute them in a single sandbox call.
19
20## Why Code Mode?
21
22### Reduced context window usage
23
24In a traditional agentic loop, every tool call adds a round-trip of messages: the model's tool-call request, the tool result, then the model's next reasoning step. A task that touches five tools can easily consume thousands of tokens in back-and-forth.
25
26With Code Mode the model emits one `execute_typescript` call containing a complete program. The five tool invocations happen inside the sandbox, and only the final result comes back — one request, one response.
27
28### The LLM decides how to interpret tool output
29
30When tools are called individually, the model must decide what to do with each result in a new turn. With Code Mode, the model writes the logic up front: filter, aggregate, compare, branch. It can `Promise.all` ten API calls, pick the best result, and return a summary — all in a single execution.
31
32### Type-safe tool execution
33
34Tools you pass to Code Mode are converted to typed function stubs that appear in the system prompt. The model sees exact input/output types, so it generates correct calls without guessing parameter names or shapes. TypeScript annotations in the generated code are stripped automatically before execution.
35
36### Secure sandboxing
37
38Generated code runs in an isolated environment (V8 isolate, QuickJS WASM, or Cloudflare Worker) with no access to the host file system, network, or process. The sandbox has configurable timeouts and memory limits.
39
40## Getting Started
41
42### 1. Install packages
43
44```bash
45pnpm add @tanstack/ai @tanstack/ai-code-mode zod
46```
47
48Pick an isolate driver:
49
50```bash
51# Node.js — fastest, uses V8 isolates (requires native compilation)
52pnpm add @tanstack/ai-isolate-node
53
54# QuickJS WASM — no native deps, works in browsers and edge runtimes
55pnpm add @tanstack/ai-isolate-quickjs
56
57# Cloudflare Workers — run on the edge
58pnpm add @tanstack/ai-isolate-cloudflare
59```
60
61### 2. Define tools
62
63Define your tools with `toolDefinition()` and provide a server-side implementation with `.server()`. These become the `external_*` functions available inside the sandbox.
64
65```typescript
66import { toolDefinition } from "@tanstack/ai";
67import { z } from "zod";
68
69const fetchWeather = toolDefinition({
70 name: "fetchWeather",
71 description: "Get current weather for a city",
72 inputSchema: z.object({ location: z.string() }),
73 outputSchema: z.object({
74 temperature: z.number(),
75 condition: z.string(),
76 }),
77}).server(async ({ location }) => {
78 const res = await fetch(`https://api.weather.example/v1?city=${location}`);
79 return res.json();
80});
81```
82
83### 3. Create the Code Mode tool and system prompt
84
85```typescript
86import { createCodeMode } from "@tanstack/ai-code-mode";
87import { createNodeIsolateDriver } from "@tanstack/ai-isolate-node";
88
89const { tool, systemPrompt } = createCodeMode({
90 driver: createNodeIsolateDriver(),
91 tools: [fetchWeather],
92 timeout: 30_000,
93});
94```
95
96### 4. Use with `chat()`
97
98```typescript
99import { chat } from "@tanstack/ai";
100import { openaiText } from "@tanstack/ai-openai";
101
102const result = await chat({
103 adapter: openaiText(),
104 model: "gpt-4o",
105 systemPrompts: [
106 "You are a helpful weather assistant.",
107 systemPrompt,
108 ],
109 tools: [tool],
110 messages: [
111 {
112 role: "user",
113 content: "Compare the weather in Tokyo, Paris, and New York City",
114 },
115 ],
116});
117```
118
119The model will generate something like:
120
121```typescript
122const cities = ["Tokyo", "Paris", "New York City"];
123const results = await Promise.all(
124 cities.map((city) => external_fetchWeather({ location: city }))
125);
126
127const warmest = results.reduce((prev, curr) =>
128 curr.temperature > prev.temperature ? curr : prev
129);
130
131return {
132 comparison: results.map((r, i) => ({
133 city: cities[i],
134 temperature: r.temperature,
135 condition: r.condition,
136 })),
137 warmest: cities[results.indexOf(warmest)],
138};
139```
140
141All three API calls happen in parallel inside the sandbox. The model receives one structured result instead of three separate tool-call round-trips.
142
143## API Reference
144
145### `createCodeMode(config)`
146
147Creates both the `execute_typescript` tool and its matching system prompt from a single config object. This is the recommended entry point.
148
149```typescript
150const { tool, systemPrompt } = createCodeMode({
151 driver, // IsolateDriver — required
152 tools, // Array<ServerTool | ToolDefinition> — required, at least one
153 timeout, // number — execution timeout in ms (default: 30000)
154 memoryLimit, // number — memory limit in MB (default: 128, Node + QuickJS drivers)
155 getSkillBindings, // () => Promise<Record<string, ToolBinding>> — optional dynamic bindings
156});
157```
158
159**Config properties:**
160
161| Property | Type | Description |
162|----------|------|-------------|
163| `driver` | `IsolateDriver` | The sandbox runtime to execute code in |
164| `tools` | `Array<ServerTool \| ToolDefinition>` | Tools exposed as `external_*` functions. Must have `.server()` implementations |
165| `timeout` | `number` | Execution timeout in milliseconds (default: 30000) |
166| `memoryLimit` | `number` | Memory limit in MB (default: 128). Supported by Node and QuickJS drivers |
167| `getSkillBindings` | `() => Promise<Record<string, ToolBinding>>` | Optional function returning additional bindings at execution time |
168
169The tool returns a `CodeModeToolResult`:
170
171```typescript
172interface CodeModeToolResult {
173 success: boolean;
174 result?: unknown; // Return value from the executed code
175 logs?: Array<string>; // Captured console output
176 error?: {
177 message: string;
178 name?: string;
179 line?: number;
180 };
181}
182```
183
184### `createCodeModeTool(config)` / `createCodeModeSystemPrompt(config)`
185
186Lower-level functions if you need only the tool or only the prompt. `createCodeMode` calls both internally.
187
188```typescript
189import { createCodeModeTool, createCodeModeSystemPrompt } from "@tanstack/ai-code-mode";
190
191const tool = createCodeModeTool(config);
192const prompt = createCodeModeSystemPrompt(config);
193```
194
195### `IsolateDriver`
196
197The interface that sandbox runtimes implement. You do not implement this yourself — pick one of the provided drivers:
198
199```typescript
200interface IsolateDriver {
201 createContext(config: IsolateConfig): Promise<IsolateContext>;
202}
203```
204
205**Available drivers:**
206
207| Package | Factory function | Environment |
208|---------|-----------------|-------------|
209| `@tanstack/ai-isolate-node` | `createNodeIsolateDriver()` | Node.js |
210| `@tanstack/ai-isolate-quickjs` | `createQuickJSIsolateDriver()` | Node.js, browser, edge |
211| `@tanstack/ai-isolate-cloudflare` | `createCloudflareIsolateDriver()` | Cloudflare Workers |
212
213For full configuration options for each driver, see [Isolate Drivers](./code-mode-isolates.md).
214
215### Advanced
216
217These utilities are used internally and are exported for custom pipelines:
218
219- **`stripTypeScript(code)`** — Strips TypeScript syntax using esbuild, converting to plain JavaScript.
220- **`toolsToBindings(tools, prefix?)`** — Converts TanStack AI tools into `Record<string, ToolBinding>` for sandbox injection.
221- **`generateTypeStubs(bindings, options?)`** — Generates TypeScript type declarations from tool bindings for system prompts.
222
223## Choosing a Driver
224
225For a full comparison of drivers with all configuration options, see [Isolate Drivers](./code-mode-isolates.md).
226
227In brief: use the **Node driver** for server-side Node.js (fastest, V8 JIT), **QuickJS** for browsers or portable edge deployments (no native deps), and the **Cloudflare driver** when you deploy to Cloudflare Workers.
228
229## Custom Events
230
231Code Mode emits custom events during execution that you can observe through the TanStack AI event system. These are useful for building UIs that show execution progress, debugging, or logging.
232
233| Event | When | Payload |
234|-------|------|---------|
235| `code_mode:execution_started` | Code execution begins | `{ timestamp, codeLength }` |
236| `code_mode:console` | Each `console.log/error/warn/info` call | `{ level, message, timestamp }` |
237| `code_mode:external_call` | Before an `external_*` function runs | `{ function, args, timestamp }` |
238| `code_mode:external_result` | After a successful `external_*` call | `{ function, result, duration }` |
239| `code_mode:external_error` | When an `external_*` call fails | `{ function, error, duration }` |
240
241To display these events in your React app, see [Showing Code Mode in the UI](./client-integration).
242
243## Model Compatibility
244
245Code Mode asks the model to write valid TypeScript that calls your tools through the sandbox bridge. Not every model handles this equally — many small or older models mishandle the `external_*` calling conventions even when the system prompt is explicit. We track a single multi-step benchmark (joining three tables, filtering customers who bought from every product category, aggregating spend per category) against a gold reference. The full harness lives at `packages/typescript/ai-code-mode/models-eval/`.
246
247| Rank | Model | Stars | Acc | Comp | TS | CME | Latency | Tokens |
248|------|-------|:-----:|:---:|:----:|:--:|:---:|--------:|-------:|
249| 1 | `grok:grok-4-1-fast-non-reasoning` | ★★★ | 10 | 9 | 6 | 10 | 7.0s | — |
250| 2 | `ollama:gpt-oss:20b` | ★★★ | 10 | 8 | 6 | 5 | 45.1s | 23.6k |
251| 3 | `anthropic:claude-haiku-4-5` | ★★★ | 10 | 10 | 7 | 10 | 9.4s | 8.5k |
252| 4 | `gemini:gemini-2.5-flash` | ★★★ | 10 | 7 | 5 | 9 | 7.3s | 6.9k |
253| 5 | `ollama:nemotron-cascade-2` | ★★★ | 10 | 9 | 5 | 5 | 60.4s | 11.7k |
254| 6 | `openai:gpt-4o-mini` | ★★☆ | 10 | 8 | 8 | 10 | 19.2s | 8.7k |
255| 7 | `ollama:gemma4:31b` | ★★☆ | 10 | 8 | 4 | 5 | 264.2s | 6.4k |
256
257**Columns**
258
259- **Stars** — overall weighted rating (1-3) combining accuracy, comprehensiveness, code quality, code-mode efficiency, speed, token efficiency, and stability.
260- **Acc / Comp / TS / CME** — Anthropic-judged subscores out of 10: accuracy vs gold, comprehensiveness, TypeScript quality, code-mode efficiency (fewer wasted attempts is better).
261- **Latency** — wall-clock time for the full agentic loop.
262- **Tokens** — total prompt + completion tokens. Grok's adapter does not report usage.
263
264**Takeaways**
265
266- **Strongest cloud picks:** Grok 4.1 Fast, Claude Haiku 4.5, and Gemini 2.5 Flash all finish under 10s and handle the multi-step task cleanly. Claude Haiku 4.5 has the highest comprehensiveness score (10/10).
267- **Strongest local pick:** `ollama:gpt-oss:20b` is the best local performer at 45s with zero compilation failures. `ollama:nemotron-cascade-2` is a close second.
268- **Avoid:** the smaller `gemma4` (9.6 GB) and the other local models commented out at the top of `eval-config.ts` (`granite4:3b`, `ministral-3`, `mistral:7b`, `qwen3:8b`, etc.) — they either ignore the `external_queryTable` shape, hallucinate results, or refuse to invoke `execute_typescript`.
269- **Caveat:** this is a single‑prompt benchmark. Local model results can vary noticeably between runs; use these as a rough capability filter rather than a definitive ranking.
270
271Reproduce locally:
272
273```bash
274cd packages/typescript/ai-code-mode/models-eval
275pnpm install
276pnpm eval # full suite (needs cloud API keys + Anthropic for judging)
277pnpm eval -- --ollama-only # local models only
278pnpm eval -- --no-judge # skip Anthropic-based judging
279```
280
281## Tips
282
283- **Start simple.** Give the model 2-3 tools and a clear task. Code Mode works best when the model has a focused set of capabilities.
284- **Prefer `Promise.all` tasks.** Code Mode shines when the model can parallelize work that would otherwise be sequential tool calls.
285- **Use `console.log` for debugging.** Logs are captured and returned in the result, making it easy to see what happened inside the sandbox.
286- **Keep tools focused.** Each tool should do one thing well. The model will compose them in code.
287- **Check the system prompt.** Call `createCodeModeSystemPrompt(config)` and inspect the output to see exactly what the model will see, including generated type stubs.
288
289## Next Steps
290
291- [Showing Code Mode in the UI](./client-integration) — Display execution progress in your React app
292- [Code Mode with Skills](./code-mode-with-skills) — Add persistent, reusable skill libraries
293- [Isolate Drivers](./code-mode-isolates) — Compare Node, QuickJS, and Cloudflare sandbox runtimes