TanStack AI version#
@tanstack/ai-sandbox-local-process@1.0.0, @tanstack/ai-sandbox@1.3.0 (also main at 1888233)
Framework/Library version#
Node.js v26.13.0 on Linux x86_64 (server-side sandbox, no UI framework)
Describe the bug and the steps to reproduce it#
LocalProcessHandle.exec() (packages/ai-sandbox-local-process/src/handle.ts, lines 887-890) collects command output per data event with stdout += d.toString('utf8') and stderr += d.toString('utf8'). Whenever a multi-byte UTF-8 sequence straddles two pipe reads, both halves decode into U+FFFD replacement characters.
The same file already documents this issue and uses a streaming TextDecoder (decodeStream, lines 139-161) for spawnProcess(), while exec() still decodes each Buffer chunk independently.
Steps to reproduce:
npm i @tanstack/ai-sandbox-local-process@1.0.0 @tanstack/ai-sandbox@1.3.0
node repro.mjs
import { localProcessSandbox } from "@tanstack/ai-sandbox-local-process";
const handle = await localProcessSandbox().create({});
const text = "日本語のテキスト\n".repeat(4000);
await handle.fs.write("notes.txt", text);
const { stdout } = await handle.process.exec("cat notes.txt");
console.log("same as file:", stdout === text);
console.log("U+FFFD count:", [...stdout].filter((c) => c === "\ufffd").length);
await handle.destroy();
Actual output:
same as file: false
U+FFFD count: 2
Expected output:
same as file: true
U+FFFD count: 0
Collecting Buffer chunks and decoding once on close with Buffer.concat(...).toString('utf8') (or using a streaming TextDecoder for stdout and stderr) avoids splitting multi-byte sequences.
Your Minimal, Reproducible Example - (Sandbox Highly Recommended)#
https://gist.github.com/trackoor/8efa2e08a9f5e2d05c59708f32db154e
Screenshots or Videos (Optional)#
No response
Do you intend to try to help solve this bug with your own PR?#
No, because I do not have time to dig into it
Terms & Code of Conduct#
Credit: https://ucsbplse.github.io/
I've fixed this in #2162.
exec()now callssetEncoding('utf8')on stdout and stderr, so Node's StringDecoder keeps a split multi-byte character until the rest arrives, the same wayspawn()already does. A test covering split€on both streams is included.