Skip to content
Source
Sign inSign up
TanStack/ai
Open History to see the last commit

code-mode-isolates.md

219 lines8.4 KB

1---
2title: Code Mode Isolate Drivers
3id: code-mode-isolates
4order: 4
5description: "Compare Code Mode sandbox drivers — Node isolated-vm, QuickJS WASM, and Cloudflare Workers — and choose the right runtime for your deployment."
6keywords:
7 - tanstack ai
8 - code mode
9 - isolate driver
10 - isolated-vm
11 - quickjs
12 - cloudflare workers
13 - sandbox
14 - secure execution
15---
16​
17Isolate drivers provide the secure sandbox runtimes that [Code Mode](./code-mode.md) uses to execute generated TypeScript. All drivers implement the same `IsolateDriver` interface, so you can swap them without changing any other code.
18​
19## Choosing a Driver
20​
21| | Node (`isolated-vm`) | QuickJS (WASM) | Cloudflare Workers |
22|---|---|---|---|
23| **Best for** | Server-side Node.js apps | Browsers, edge, portability | Edge deployments on Cloudflare |
24| **Performance** | Fast (V8 JIT) | Slower (interpreted) | Fast (V8 on Cloudflare edge) |
25| **Native deps** | Yes (C++ addon) | None | None |
26| **Browser support** | No | Yes | N/A |
27| **Memory limit** | Configurable | Configurable | N/A |
28| **Stack size limit** | N/A | Configurable | N/A |
29| **Setup** | `pnpm add` | `pnpm add` | Deploy a Worker first |
30​
31---
32​
33## Node.js Driver (`@tanstack/ai-isolate-node`)
34​
35Uses V8 isolates via the [`isolated-vm`](https://github.com/laverdet/isolated-vm) native addon. This is the fastest option for server-side Node.js applications because generated code runs in the same V8 engine as the host, under JIT compilation, with no serialization overhead beyond tool call boundaries.
36​
37### Installation
38​
39```bash
40pnpm add @tanstack/ai-isolate-node
41```
42​
43`isolated-vm` is a native C++ addon and must be compiled for your platform. It requires Node.js 18 or later.
44​
45### Usage
46​
47```typescript
48import { createNodeIsolateDriver } from '@tanstack/ai-isolate-node'
49​
50const driver = createNodeIsolateDriver({
51 memoryLimit: 128, // MB
52 timeout: 30_000, // ms
53})
54```
55​
56### Options
57​
58| Option | Type | Default | Description |
59|--------|------|---------|-------------|
60| `memoryLimit` | `number` | `128` | Maximum heap size for the V8 isolate, in megabytes. Execution is terminated if this limit is exceeded. |
61| `timeout` | `number` | `30000` | Maximum wall-clock time per execution, in milliseconds. |
62​
63### How it works
64​
65Each `execute_typescript` call creates a fresh V8 isolate. Your tools are bridged into the isolate as async reference functions — when generated code calls `external_myTool(...)`, the call crosses the isolate boundary back into the host Node.js process, executes your tool implementation, and returns the result. Console output (`log`, `error`, `warn`, `info`) is captured and returned in the execution result. The isolate is destroyed after each call.
66​
67---
68​
69## QuickJS Driver (`@tanstack/ai-isolate-quickjs`)
70​
71Uses [QuickJS](https://bellard.org/quickjs/) compiled to WebAssembly via Emscripten. Because the sandbox is a WASM module, it has no native dependencies and runs anywhere JavaScript runs: Node.js, browsers, Deno, Bun, and Cloudflare Workers (without deploying a separate Worker).
72​
73### Installation
74​
75```bash
76pnpm add @tanstack/ai-isolate-quickjs
77```
78​
79### Usage
80​
81```typescript
82import { createQuickJSIsolateDriver } from '@tanstack/ai-isolate-quickjs'
83​
84const driver = createQuickJSIsolateDriver({
85 memoryLimit: 128, // MB
86 timeout: 30_000, // ms
87 maxStackSize: 524288, // bytes (512 KiB)
88})
89```
90​
91### Options
92​
93| Option | Type | Default | Description |
94|--------|------|---------|-------------|
95| `memoryLimit` | `number` | `128` | Maximum heap memory for the QuickJS VM, in megabytes. |
96| `timeout` | `number` | `30000` | Maximum wall-clock time per execution, in milliseconds. |
97| `maxStackSize` | `number` | `524288` | Maximum call stack size in bytes (default: 512 KiB). Increase for deeply recursive code; decrease to catch runaway recursion sooner. |
98​
99### How it works
100​
101QuickJS WASM uses an asyncified execution model — the WASM module can pause while awaiting host async functions (your tools). Executions are serialized through a global queue to prevent concurrent WASM calls, which the asyncify model does not support. Fatal errors (memory exhaustion, stack overflow) are detected, the VM is disposed, and a structured error is returned. Console output is captured and returned with the result.
102​
103> **Performance note:** QuickJS interprets JavaScript rather than JIT-compiling it, so compute-heavy scripts run slower than with the Node driver. For typical LLM-generated scripts that are mostly waiting on `external_*` tool calls, this difference is not significant.
104​
105---
106​
107## Cloudflare Workers Driver (`@tanstack/ai-isolate-cloudflare`)
108​
109Runs generated code inside a [Cloudflare Worker](https://workers.cloudflare.com/) at the edge. Your application server sends code and tool schemas to the Worker via HTTP; the Worker executes the code and calls back when it needs a tool result. This keeps your tool implementations on your server while sandboxed execution happens on Cloudflare's global network.
110​
111### Installation
112​
113```bash
114pnpm add @tanstack/ai-isolate-cloudflare
115```
116​
117### Usage
118​
119```typescript
120import { createCloudflareIsolateDriver } from '@tanstack/ai-isolate-cloudflare'
121​
122const driver = createCloudflareIsolateDriver({
123 workerUrl: 'https://my-code-mode-worker.my-account.workers.dev',
124 authorization: process.env.CODE_MODE_WORKER_SECRET,
125 timeout: 30_000,
126 maxToolRounds: 10,
127})
128```
129​
130### Options
131​
132| Option | Type | Default | Description |
133|--------|------|---------|-------------|
134| `workerUrl` | `string` | — | **Required.** Full URL of the deployed Cloudflare Worker. |
135| `authorization` | `string` | — | Optional value sent as the `Authorization` header on every request. Use this to prevent unauthorized access to your Worker. |
136| `timeout` | `number` | `30000` | Maximum wall-clock time for the entire execution (including all tool round-trips), in milliseconds. |
137| `maxToolRounds` | `number` | `10` | Maximum number of tool-call/result cycles per execution. Prevents infinite loops when generated code calls tools in a loop. |
138​
139### Deploying the Worker
140​
141The package exports a ready-made Worker handler at `@tanstack/ai-isolate-cloudflare/worker`. Create a `wrangler.toml` and a worker entry file:
142​
143```toml
144# wrangler.toml
145name = "code-mode-worker"
146main = "src/worker.ts"
147compatibility_date = "2024-01-01"
148​
149[unsafe]
150bindings = [{ name = "eval", type = "eval" }]
151```
152​
153```typescript
154// src/worker.ts
155export { default } from '@tanstack/ai-isolate-cloudflare/worker'
156```
157​
158Deploy:
159​
160```bash
161wrangler deploy
162```
163​
164### How it works
165​
166The driver implements a request/response loop for tool execution:
167​
168```
169Driver (your server) Worker (Cloudflare edge)
170───────────────────── ─────────────────────────
171Send: code + tool schemas ──────▶ Execute code
172 ◀────── Return: needs tool X with args Y
173Execute tool X locally
174Send: tool result ──────▶ Resume execution
175 ◀────── Return: final result / needs tool Z
176...repeat until done...
177```
178​
179Each round-trip adds network latency, so the `maxToolRounds` limit both prevents runaway scripts and caps the maximum number of cross-continent hops. Console output from all rounds is aggregated and returned in the final result.
180​
181> **Security:** The Worker requires `UNSAFE_EVAL` (local dev) or the `eval` unsafe binding (production) to execute arbitrary code. Restrict access using the `authorization` option or Cloudflare Access policies.
182​
183---
184​
185## The `IsolateDriver` Interface
186​
187All three drivers satisfy this interface, exported from `@tanstack/ai-code-mode`:
188​
189```typescript
190interface IsolateDriver {
191 createContext(config: IsolateConfig): Promise<IsolateContext>
193​
194interface IsolateConfig {
195 bindings: Record<string, ToolBinding>
196 timeout?: number
197 memoryLimit?: number
199​
200interface IsolateContext {
201 execute(code: string): Promise<ExecutionResult>
202 dispose(): Promise<void>
204​
205interface ExecutionResult<T = unknown> {
206 success: boolean
207 value?: T
208 logs: Array<string>
209 error?: NormalizedError
211```
212​
213You can implement this interface to build a custom driver — for example, a Docker-based sandbox or a Deno subprocess.
214​
215## Next Steps
216​
217- [Code Mode](./code-mode) — Core setup, API reference, and getting started guide
218- [Showing Code Mode in the UI](./client-integration) — Display execution progress in your React app
219- [Code Mode with Skills](./code-mode-with-skills) — Add persistent, reusable skill libraries