Skip to content
Source
Sign inSign up
TanStack/ai
Open History to see the last commit

code-mode.md

303 lines13.5 KB

1---
2title: Code Mode
3id: code-mode
4order: 1
5description: "Let LLMs write and execute TypeScript programs that orchestrate tools in a secure sandbox with TanStack AI Code Mode — fewer loops, richer logic."
6keywords:
7 - tanstack ai
8 - code mode
9 - sandbox
10 - typescript execution
11 - tool orchestration
12 - execute_typescript
13 - ai agents
14---
15​
16Code Mode lets an LLM write and execute TypeScript programs inside a secure sandbox. Instead of making one tool call at a time, the model writes a short script that orchestrates multiple tools with loops, conditionals, `Promise.all`, and data transformations — then returns a single result.
17​
18You already have a chat app that uses [tools](../tools/tools). By the end of this guide, you'll have Code Mode set up so the LLM can compose those tools in TypeScript and execute them in a single sandbox call.
19​
20## Why Code Mode?
21​
22### Reduced context window usage
23​
24In a traditional agentic loop, every tool call adds a round-trip of messages: the model's tool-call request, the tool result, then the model's next reasoning step. A task that touches five tools can easily consume thousands of tokens in back-and-forth.
25​
26With Code Mode the model emits one `execute_typescript` call containing a complete program. The five tool invocations happen inside the sandbox, and only the final result comes back — one request, one response.
27​
28### The LLM decides how to interpret tool output
29​
30When tools are called individually, the model must decide what to do with each result in a new turn. With Code Mode, the model writes the logic up front: filter, aggregate, compare, branch. It can `Promise.all` ten API calls, pick the best result, and return a summary — all in a single execution.
31​
32### Type-safe tool execution
33​
34Tools you pass to Code Mode are converted to typed function stubs that appear in the system prompt. The model sees exact input/output types, so it generates correct calls without guessing parameter names or shapes. TypeScript annotations in the generated code are stripped automatically before execution.
35​
36### Secure sandboxing
37​
38Generated code runs in an isolated environment (V8 isolate, QuickJS WASM, native QuickJS on Bun, Cloudflare Worker, or Daytona sandbox) with no access to the host file system, network, or process. The sandbox has configurable timeouts and memory limits.
39​
40## Getting Started
41​
42### 1. Install packages
43​
44```bash
45pnpm add @tanstack/ai @tanstack/ai-code-mode zod
46```
47​
48Pick an isolate driver:
49​
50```bash
51# Node.js — fastest, uses V8 isolates (requires native compilation)
52pnpm add @tanstack/ai-isolate-node
53​
54# QuickJS WASM — no native deps, works in browsers and edge runtimes
55pnpm add @tanstack/ai-isolate-quickjs
56​
57# QuickJS Bun — native QuickJS via bun:ffi, fastest option on Bun
58bun add @tanstack/ai-isolate-quickjs-bun
59​
60# Cloudflare Workers — run on the edge
61pnpm add @tanstack/ai-isolate-cloudflare
62​
63# Daytona sandboxes — run in a remote Daytona sandbox
64pnpm add @tanstack/ai-isolate-daytona @daytona/sdk
65```
66​
67### 2. Define tools
68​
69Define your tools with `toolDefinition()` and provide a server-side implementation with `.server()`. These become the `external_*` functions available inside the sandbox.
70​
71```typescript group=code-mode
72import { toolDefinition } from "@tanstack/ai";
73import { z } from "zod";
74​
75const fetchWeather = toolDefinition({
76 name: "fetchWeather",
77 description: "Get current weather for a city",
78 inputSchema: z.object({ location: z.string() }),
79 outputSchema: z.object({
80 temperature: z.number(),
81 condition: z.string(),
82 }),
83}).server(async ({ location }) => {
84 const res = await fetch(`https://api.weather.example/v1?city=${location}`);
85 return res.json();
86});
87```
88​
89### 3. Create the Code Mode tool and system prompt
90​
91```typescript group=code-mode
92import { createCodeMode } from "@tanstack/ai-code-mode";
93import { createNodeIsolateDriver } from "@tanstack/ai-isolate-node";
94​
95const { tool, systemPrompt } = createCodeMode({
96 driver: createNodeIsolateDriver(),
97 tools: [fetchWeather],
98 timeout: 30_000,
99});
100```
101​
102### 4. Use with `chat()`
103​
104```typescript group=code-mode
105import { chat } from "@tanstack/ai";
106import { openaiText } from "@tanstack/ai-openai";
107​
108const result = await chat({
109 adapter: openaiText("gpt-5.5"),
110 systemPrompts: [
111 "You are a helpful weather assistant.",
112 systemPrompt,
113 ],
114 tools: [tool],
115 messages: [
116 {
117 role: "user",
118 content: "Compare the weather in Tokyo, Paris, and New York City",
119 },
120 ],
121});
122```
123​
124The model will generate something like:
125​
126```typescript ignore
127const cities = ["Tokyo", "Paris", "New York City"];
128const results = await Promise.all(
129 cities.map((city) => external_fetchWeather({ location: city }))
130);
131​
132const warmest = results.reduce((prev, curr) =>
133 curr.temperature > prev.temperature ? curr : prev
134);
135​
136return {
137 comparison: results.map((r, i) => ({
138 city: cities[i],
139 temperature: r.temperature,
140 condition: r.condition,
141 })),
142 warmest: cities[results.indexOf(warmest)],
143};
144```
145​
146All three API calls happen in parallel inside the sandbox. The model receives one structured result instead of three separate tool-call round-trips.
147​
148## API Reference
149​
150### `createCodeMode(config)`
151​
152Creates both the `execute_typescript` tool and its matching system prompt from a single config object. This is the recommended entry point.
153​
154```typescript ignore
155const { tool, systemPrompt } = createCodeMode({
156 driver, // IsolateDriver — required
157 tools, // Array<ServerTool | ToolDefinition> — required, at least one
158 timeout, // number — execution timeout in ms (default: 30000)
159 memoryLimit, // number — memory limit in MB (default: 128, Node + QuickJS drivers)
160 getSkillBindings, // () => Promise<Record<string, ToolBinding>> — optional dynamic bindings
161});
162```
163​
164**Config properties:**
165​
166| Property | Type | Description |
167|----------|------|-------------|
168| `driver` | `IsolateDriver` | The sandbox runtime to execute code in |
169| `tools` | `Array<ServerTool \| ToolDefinition>` | Tools exposed as `external_*` functions. Must have `.server()` implementations |
170| `timeout` | `number` | Execution timeout in milliseconds (default: 30000) |
171| `memoryLimit` | `number` | Memory limit in MB (default: 128). Supported by Node and QuickJS drivers |
172| `getSkillBindings` | `() => Promise<Record<string, ToolBinding>>` | Optional function returning additional bindings at execution time |
173​
174The tool returns a `CodeModeToolResult`:
175​
176```typescript
177interface CodeModeToolResult {
178 success: boolean;
179 result?: unknown; // Return value from the executed code
180 logs?: Array<string>; // Captured console output
181 error?: {
182 message: string;
183 name?: string;
184 line?: number;
185 };
187```
188​
189### `createCodeModeTool(config)` / `createCodeModeSystemPrompt(config)`
190​
191Lower-level functions if you need only the tool or only the prompt. `createCodeMode` calls both internally.
192​
193```typescript
194import { createCodeModeTool, createCodeModeSystemPrompt } from "@tanstack/ai-code-mode";
195import { config } from "./config";
196​
197const tool = createCodeModeTool(config);
198const prompt = createCodeModeSystemPrompt(config);
199```
200​
201### `IsolateDriver`
202​
203The interface that sandbox runtimes implement. You do not implement this yourself — pick one of the provided drivers:
204​
205```typescript
206import type { IsolateConfig, IsolateContext } from "@tanstack/ai-code-mode";
207​
208interface IsolateDriver {
209 createContext(config: IsolateConfig): Promise<IsolateContext>;
211```
212​
213**Available drivers:**
214​
215| Package | Factory function | Environment |
216|---------|-----------------|-------------|
217| `@tanstack/ai-isolate-node` | `createNodeIsolateDriver()` | Node.js |
218| `@tanstack/ai-isolate-quickjs` | `createQuickJSIsolateDriver()` | Node.js, browser, edge |
219| `@tanstack/ai-isolate-quickjs-bun` | `createQuickJSBunIsolateDriver()` | Bun |
220| `@tanstack/ai-isolate-cloudflare` | `createCloudflareIsolateDriver()` | Cloudflare Workers |
221| `@tanstack/ai-isolate-daytona` | `createDaytonaIsolateDriver()` | Daytona sandboxes |
222​
223For full configuration options for each driver, see [Isolate Drivers](./code-mode-isolates.md).
224​
225### Advanced
226​
227These utilities are used internally and are exported for custom pipelines:
228​
229- **`stripTypeScript(code)`** — Strips TypeScript syntax using sucrase (edge-safe, no native binary), converting to plain JavaScript.
230- **`toolsToBindings(tools, prefix?)`** — Converts TanStack AI tools into `Record<string, ToolBinding>` for sandbox injection.
231- **`generateTypeStubs(bindings, options?)`** — Generates TypeScript type declarations from tool bindings for system prompts.
232​
233## Choosing a Driver
234​
235For a full comparison of drivers with all configuration options, see [Isolate Drivers](./code-mode-isolates.md).
236​
237In brief: use the **Node driver** for server-side Node.js (fastest, V8 JIT), **QuickJS** for browsers or portable edge deployments (no native deps), **QuickJS Bun** for Bun servers (native QuickJS via `bun:ffi`), the **Cloudflare driver** when you deploy to Cloudflare Workers, and the **Daytona driver** when you want execution inside a full remote Linux sandbox.
238​
239## Custom Events
240​
241Code Mode emits custom events during execution that you can observe through the TanStack AI event system. These are useful for building UIs that show execution progress, debugging, or logging.
242​
243| Event | When | Payload |
244|-------|------|---------|
245| `code_mode:execution_started` | Code execution begins | `{ timestamp, codeLength }` |
246| `code_mode:console` | Each `console.log/error/warn/info` call | `{ level, message, timestamp }` |
247| `code_mode:external_call` | Before an `external_*` function runs | `{ function, args, timestamp }` |
248| `code_mode:external_result` | After a successful `external_*` call | `{ function, result, duration }` |
249| `code_mode:external_error` | When an `external_*` call fails | `{ function, error, duration }` |
250​
251To display these events in your React app, see [Showing Code Mode in the UI](./client-integration).
252​
253## Model Compatibility
254​
255Code Mode asks the model to write valid TypeScript that calls your tools through the sandbox bridge. Not every model handles this equally — many small or older models mishandle the `external_*` calling conventions even when the system prompt is explicit. We track a single multi-step benchmark (joining three tables, filtering customers who bought from every product category, aggregating spend per category) against a gold reference. The full harness lives at `packages/ai-code-mode/models-eval/`.
256​
257| Rank | Model | Stars | Acc | Comp | TS | CME | Latency | Tokens |
258|------|-------|:-----:|:---:|:----:|:--:|:---:|--------:|-------:|
259| 1 | `grok:grok-4-1-fast-non-reasoning` | ★★★ | 10 | 9 | 6 | 10 | 7.0s | — |
260| 2 | `ollama:gpt-oss:20b` | ★★★ | 10 | 8 | 6 | 5 | 45.1s | 23.6k |
261| 3 | `anthropic:claude-haiku-4-5` | ★★★ | 10 | 10 | 7 | 10 | 9.4s | 8.5k |
262| 4 | `gemini:gemini-2.5-flash` | ★★★ | 10 | 7 | 5 | 9 | 7.3s | 6.9k |
263| 5 | `ollama:nemotron-cascade-2` | ★★★ | 10 | 9 | 5 | 5 | 60.4s | 11.7k |
264| 6 | `openai:gpt-4o-mini` | ★★☆ | 10 | 8 | 8 | 10 | 19.2s | 8.7k |
265| 7 | `ollama:gemma4:31b` | ★★☆ | 10 | 8 | 4 | 5 | 264.2s | 6.4k |
266​
267**Columns**
268​
269- **Stars** — overall weighted rating (1-3) combining accuracy, comprehensiveness, code quality, code-mode efficiency, speed, token efficiency, and stability.
270- **Acc / Comp / TS / CME** — Anthropic-judged subscores out of 10: accuracy vs gold, comprehensiveness, TypeScript quality, code-mode efficiency (fewer wasted attempts is better).
271- **Latency** — wall-clock time for the full agentic loop.
272- **Tokens** — total prompt + completion tokens. Grok's adapter does not report usage.
273​
274**Takeaways**
275​
276- **Strongest cloud picks:** Grok 4.1 Fast, Claude Haiku 4.5, and Gemini 2.5 Flash all finish under 10s and handle the multi-step task cleanly. Claude Haiku 4.5 has the highest comprehensiveness score (10/10).
277- **Strongest local pick:** `ollama:gpt-oss:20b` is the best local performer at 45s with zero compilation failures. `ollama:nemotron-cascade-2` is a close second.
278- **Avoid:** the smaller `gemma4` (9.6 GB) and the other local models commented out at the top of `eval-config.ts` (`granite4:3b`, `ministral-3`, `mistral:7b`, `qwen3:8b`, etc.) — they either ignore the `external_queryTable` shape, hallucinate results, or refuse to invoke `execute_typescript`.
279- **Caveat:** this is a single‑prompt benchmark. Local model results can vary noticeably between runs; use these as a rough capability filter rather than a definitive ranking.
280​
281Reproduce locally:
282​
283```bash
284cd packages/ai-code-mode/models-eval
285pnpm install
286pnpm eval # full suite (needs cloud API keys + Anthropic for judging)
287pnpm eval -- --ollama-only # local models only
288pnpm eval -- --no-judge # skip Anthropic-based judging
289```
290​
291## Tips
292​
293- **Start simple.** Give the model 2-3 tools and a clear task. Code Mode works best when the model has a focused set of capabilities.
294- **Prefer `Promise.all` tasks.** Code Mode shines when the model can parallelize work that would otherwise be sequential tool calls.
295- **Use `console.log` for debugging.** Logs are captured and returned in the result, making it easy to see what happened inside the sandbox.
296- **Keep tools focused.** Each tool should do one thing well. The model will compose them in code.
297- **Check the system prompt.** Call `createCodeModeSystemPrompt(config)` and inspect the output to see exactly what the model will see, including generated type stubs.
298​
299## Next Steps
300​
301- [Showing Code Mode in the UI](./client-integration) — Display execution progress in your React app
302- [Code Mode with Skills](./code-mode-with-skills) — Add persistent, reusable skill libraries
303- [Isolate Drivers](./code-mode-isolates) — Compare Node, QuickJS, QuickJS Bun, Cloudflare, and Daytona sandbox runtimes