fix(ai-sandbox-local-process): decode exec() output across chunks #2162
main fromrefs/pull/2162/head·opened View on GitHub🎯 Changes#
Fixes #2153.
exec() decoded each stdout/stderr chunk on its own with d.toString('utf8'). When a pipe splits a multi-byte UTF-8 character across two reads, both halves become U+FFFD.
The fix calls setEncoding('utf8') on both streams. Node then decodes with a StringDecoder, which keeps a partial sequence until the rest arrives. spawn() already handles this with decodeStream. A truncated sequence at the end of the stream still comes out as U+FFFD, the same as spawn().
Test: a new case in local-process.test.ts writes the first byte of € to stdout and stderr, then the rest 50 ms later, and checks that exec() returns € for both. It mirrors the existing spawn() test.
Docs: not changed. This is a bug fix with no API change.
✅ Checklist#
- I have followed the steps in the Contributing guide.
- I have tested code changes locally with
pnpm run test:pr, or these tests do not apply to this pull request. - I fully understand the code in this pull request, including any code generated with AI assistance.
- Docs: I updated
docs/for this change, or this change is not user-facing. - Changeset: I added a changeset (
pnpm changeset), or this PR does not change a published package.
🚀 Release Impact#
- This change affects published code, and I have generated a changeset.
- This change is docs/CI/dev-only (no release).
Summary by CodeRabbit#
- Bug Fixes
- Fixed
exec()output decoding so UTF-8 characters split across reads are returned correctly in both standard output and standard error.
- Fixed
- 1
mohamadgarmabi(GitHub profile, opens in a new tab) linked this to fix #2153
tanstack-source added labelsbug
No actionable comments were generated in the recent review.
Walkthrough
-
exec()now decodes stdout and stderr as UTF-8 across stream chunks. Unit and end-to-end tests check that split UTF-8 bytes produce the expected€character in both outputs.Changes: Local process exec UTF-8 decoding
Layer / File(s) Summary Exec stream decoding and unit validation
packages/ai-sandbox-local-process/src/handle.ts,packages/ai-sandbox-local-process/tests/local-process.test.ts,.changeset/local-process-exec-utf8.mdexec()uses stream-level UTF-8 decoding for stdout and stderr. The unit test checks that split bytes produce€in both outputs. The changeset records the patch.End-to-end output validation
testing/e2e/src/routes/api.sandbox-local-process-exec.ts,testing/e2e/src/routeTree.gen.ts,testing/e2e/tests/sandbox-local-process-exec.spec.tsThe route runs a script that writes split UTF-8 bytes to stdout and stderr, then returns the execution result. The generated route tree registers the endpoint. The Playwright test checks the output, exit code, and absence of replacement characters. Estimated code review effort: 2 (Simple) | ~10 minutes
Change: Bug fix · Severity of issue fixed: Medium
Suggested reviewers:
alemtuzlak
Merge Risk:
Minimal · up toec661-
This fixes corruption of multi-byte characters in exec() output when they are split across reads. The change is small, matches the linked issue, and is covered by unit and end-to-end tests. I found no material merge risk.
Pre-merge checks:
4-
Check name Status Explanation Title check The title names the affected package and the behavior fixed: exec() output is now decoded across chunks. It matches the main change in the changeset, handle.ts, and the new tests. The title says 'deco… Description check The description follows the template. It has the Changes section with the cause, the fix, the test, and a note that docs were not changed. All checklist items are present and marked. It links issue #2…Linked Issues check [ #2153] requiresLocalProcessHandle.exec()to decode stdout and stderr without corrupting multi-byte UTF-8 characters split across pipe reads. The whole-PR summary shows `packages/ai-sandbox-local-p…Out of Scope Changes check The whole-PR diff against merge base 1888233contains these areas. (1)handle.ts:exec()stdout and stderr collection only, which implements#2153. (2)local-process.test.ts: one new unit test…Full details: Title check
-
Explanation
The title names the affected package and the behavior fixed: exec() output is now decoded across chunks. It matches the main change in the changeset, handle.ts, and the new tests. The title says 'decode exec() output across chunks' and does not name the UTF-8 symptom, but it still identifies the primary change.
Full details: Description check
-
Explanation
The description follows the template. It has the Changes section with the cause, the fix, the test, and a note that docs were not changed. All checklist items are present and marked. It links issue
#2153and explains the StringDecoder approach. The release impact section is filled in. It is mostly complete. Minor gap: the Changes section does not state the changeset file explicitly, though the checklist marks it done.
Full details: Linked Issues check
-
Explanation
[
#2153] requiresLocalProcessHandle.exec()to decode stdout and stderr without corrupting multi-byte UTF-8 characters split across pipe reads. The whole-PR summary showspackages/ai-sandbox-local-process/src/handle.tschanged inexec()(+4/-2). The change callssetEncoding('utf8')on stdout and stderr before collectingdata. This replaces the per-chunkd.toString('utf8')calls. Node uses aStringDecoderfor these streams, which holds a partial sequence until the remaining bytes arrive. Automated coverage exists inpackages/ai-sandbox-local-process/tests/local-process.test.ts, which writes the first byte of€to stdout and stderr, writes the rest after 50 ms, and asserts that both outputs equal€. An end-to-end test intesting/e2e/tests/sandbox-local-process-exec.spec.tsalso checks split-byte output through/api/sandbox-local-process-exec. Both tests cover the reported defect. The issue has no non-coding requirements. The truncated-sequence case still yields U+FFFD, which the issue does not require to change. The reviewed summary is the evidence for the handle.ts change; no source hunk was re-read in this review.
Full details: Out of Scope Changes check
-
Explanation
The whole-PR diff against merge base
1888233contains these areas. (1)handle.ts:exec()stdout and stderr collection only, which implements#2153. (2)local-process.test.ts: one new unit test for split-character decoding. (3).changeset/local-process-exec-utf8.md: a patch changeset, which the PR description requires. (4)testing/e2e/src/routes/api.sandbox-local-process-exec.ts,testing/e2e/tests/sandbox-local-process-exec.spec.ts, and the generatedtesting/e2e/src/routeTree.gen.tsentries: an end-to-end route and spec that exercise the same split-byte scenario throughexec(). These support the objective as tests. TherouteTree.gen.tschanges are generated registration for the new route. No public declarations change, and thespawnProcess()path is not modified. The new code is connected to#2153and no unrelated change is shown.
-
Finishing Touches
-
Generate unit tests (beta)
-
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Recent review info-
Run configuration-
- Source mode: Auto
- Repository: TanStack/ai
- Branch: main
- File: .coderabbit.yaml
- Commit: 1888233
- Layers: Repository YAML
- Review profile: CHILL
- Plan: Advanced
- Run ID:
ea1332c7-4296-430e-bed4-22ab557d9064
Commits-
Reviewing files that changed from the base of the PR and between f843c1c7eb2c7dcd43733651b16646d3332559cb and ec6611537b2a281a657b07fcadd0917665ef4ca7.
Files selected for processing: 3-
testing/e2e/src/routeTree.gen.tstesting/e2e/src/routes/api.sandbox-local-process-exec.tstesting/e2e/tests/sandbox-local-process-exec.spec.ts
Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.
-
-
github-actions[bot](GitHub profile, opens in a new tab) assignedtombeckenham
Thanks for the PR, @mohamadgarmabi! 🙌 @tombeckenham will take a look.
Automated pre-review checks
- ✅ CI passing
- ✅ No merge conflicts
- ✅ Changeset present
- ⚠️ No E2E test changes detected — behavior changes need coverage under
testing/e2e/(see CONTRIBUTING)
github-actions[bot](GitHub profile, opens in a new tab) added labelswaiting-on: maintainer
Added E2E coverage in
testing/e2e/tests/sandbox-local-process-exec.spec.ts(exec() output with 2/3/4-byte UTF-8 characters split across chunks, on both stdout and stderr). Could a maintainer approve the workflow runs when you have a moment? Thanks!This pull request is useful. It fixes
exec()UTF-8 decoding in the local-process sandbox. It links #2153. No open pull request does the same work.TanStack Source bot, value agent
github-actions[bot](GitHub profile, opens in a new tab) added labelswaiting-on: author
github-actions[bot](GitHub profile, opens in a new tab) removed labelswaiting-on: maintainer
AlemTuzlak approved these changes
AlemTuzlak turned on merge when ready (merge)
View your CI Pipeline Execution ↗ for commit ec6611537b2a281a657b07fcadd0917665ef4ca7
Command Status Duration Result nx run-many --targets=build --exclude=examples/...✅ Succeeded 20s View ↗ 💡 With Ultracache enabled, Nx Cloud learns each task's file inputs in CI and uses them on later runs. Turn on Ultracache in workspace settings ↗ · Read docs ↗
☁️ Nx Cloud last updated this comment at
2026-11-19 12:26:04UTC@tanstack/ai
npm i https://pkg.pr.new/@tanstack/ai@2162@tanstack/ai-acp
npm i https://pkg.pr.new/@tanstack/ai-acp@2162@tanstack/ai-angular
npm i https://pkg.pr.new/@tanstack/ai-angular@2162@tanstack/ai-anthropic
npm i https://pkg.pr.new/@tanstack/ai-anthropic@2162@tanstack/ai-bedrock
npm i https://pkg.pr.new/@tanstack/ai-bedrock@2162@tanstack/ai-byteplus
npm i https://pkg.pr.new/@tanstack/ai-byteplus@2162@tanstack/ai-claude-code
npm i https://pkg.pr.new/@tanstack/ai-claude-code@2162@tanstack/ai-client
npm i https://pkg.pr.new/@tanstack/ai-client@2162@tanstack/ai-cloudflare
npm i https://pkg.pr.new/@tanstack/ai-cloudflare@2162@tanstack/ai-code-mode
npm i https://pkg.pr.new/@tanstack/ai-code-mode@2162@tanstack/ai-code-mode-snippets
npm i https://pkg.pr.new/@tanstack/ai-code-mode-snippets@2162@tanstack/ai-codex
npm i https://pkg.pr.new/@tanstack/ai-codex@2162@tanstack/ai-cohere
npm i https://pkg.pr.new/@tanstack/ai-cohere@2162@tanstack/ai-compaction
npm i https://pkg.pr.new/@tanstack/ai-compaction@2162@tanstack/ai-computer-local
npm i https://pkg.pr.new/@tanstack/ai-computer-local@2162@tanstack/ai-computer-playwright
npm i https://pkg.pr.new/@tanstack/ai-computer-playwright@2162@tanstack/ai-computer-sandbox
npm i https://pkg.pr.new/@tanstack/ai-computer-sandbox@2162@tanstack/ai-computer-use
npm i https://pkg.pr.new/@tanstack/ai-computer-use@2162@tanstack/ai-devtools-core
npm i https://pkg.pr.new/@tanstack/ai-devtools-core@2162@tanstack/ai-durable-stream
npm i https://pkg.pr.new/@tanstack/ai-durable-stream@2162@tanstack/ai-effect
npm i https://pkg.pr.new/@tanstack/ai-effect@2162@tanstack/ai-elevenlabs
npm i https://pkg.pr.new/@tanstack/ai-elevenlabs@2162@tanstack/ai-event-client
npm i https://pkg.pr.new/@tanstack/ai-event-client@2162@tanstack/ai-fal
npm i https://pkg.pr.new/@tanstack/ai-fal@2162@tanstack/ai-gemini
npm i https://pkg.pr.new/@tanstack/ai-gemini@2162@tanstack/ai-grok
npm i https://pkg.pr.new/@tanstack/ai-grok@2162@tanstack/ai-grok-build
npm i https://pkg.pr.new/@tanstack/ai-grok-build@2162@tanstack/ai-groq
npm i https://pkg.pr.new/@tanstack/ai-groq@2162@tanstack/ai-harness
npm i https://pkg.pr.new/@tanstack/ai-harness@2162@tanstack/ai-harness-cli
npm i https://pkg.pr.new/@tanstack/ai-harness-cli@2162@tanstack/ai-isolate-cloudflare
npm i https://pkg.pr.new/@tanstack/ai-isolate-cloudflare@2162@tanstack/ai-isolate-daytona
npm i https://pkg.pr.new/@tanstack/ai-isolate-daytona@2162@tanstack/ai-isolate-e2b
npm i https://pkg.pr.new/@tanstack/ai-isolate-e2b@2162@tanstack/ai-isolate-node
npm i https://pkg.pr.new/@tanstack/ai-isolate-node@2162@tanstack/ai-isolate-quickjs
npm i https://pkg.pr.new/@tanstack/ai-isolate-quickjs@2162@tanstack/ai-isolate-quickjs-bun
npm i https://pkg.pr.new/@tanstack/ai-isolate-quickjs-bun@2162@tanstack/ai-llmgateway
npm i https://pkg.pr.new/@tanstack/ai-llmgateway@2162@tanstack/ai-lovable
npm i https://pkg.pr.new/@tanstack/ai-lovable@2162@tanstack/ai-mcp
npm i https://pkg.pr.new/@tanstack/ai-mcp@2162@tanstack/ai-memory
npm i https://pkg.pr.new/@tanstack/ai-memory@2162@tanstack/ai-mistral
npm i https://pkg.pr.new/@tanstack/ai-mistral@2162@tanstack/ai-muse
npm i https://pkg.pr.new/@tanstack/ai-muse@2162@tanstack/ai-neon
npm i https://pkg.pr.new/@tanstack/ai-neon@2162@tanstack/ai-octane
npm i https://pkg.pr.new/@tanstack/ai-octane@2162@tanstack/ai-ollama
npm i https://pkg.pr.new/@tanstack/ai-ollama@2162@tanstack/ai-ollaya
npm i https://pkg.pr.new/@tanstack/ai-ollaya@2162@tanstack/ai-openai
npm i https://pkg.pr.new/@tanstack/ai-openai@2162@tanstack/ai-opencode
npm i https://pkg.pr.new/@tanstack/ai-opencode@2162@tanstack/ai-openrouter
npm i https://pkg.pr.new/@tanstack/ai-openrouter@2162@tanstack/ai-perplexity
npm i https://pkg.pr.new/@tanstack/ai-perplexity@2162@tanstack/ai-persistence
npm i https://pkg.pr.new/@tanstack/ai-persistence@2162@tanstack/ai-preact
npm i https://pkg.pr.new/@tanstack/ai-preact@2162@tanstack/ai-react
npm i https://pkg.pr.new/@tanstack/ai-react@2162@tanstack/ai-reactor
npm i https://pkg.pr.new/@tanstack/ai-reactor@2162@tanstack/ai-remix
npm i https://pkg.pr.new/@tanstack/ai-remix@2162@tanstack/ai-sandbox
npm i https://pkg.pr.new/@tanstack/ai-sandbox@2162@tanstack/ai-sandbox-blaxel
npm i https://pkg.pr.new/@tanstack/ai-sandbox-blaxel@2162@tanstack/ai-sandbox-boxd
npm i https://pkg.pr.new/@tanstack/ai-sandbox-boxd@2162@tanstack/ai-sandbox-cloudflare
npm i https://pkg.pr.new/@tanstack/ai-sandbox-cloudflare@2162@tanstack/ai-sandbox-daytona
npm i https://pkg.pr.new/@tanstack/ai-sandbox-daytona@2162@tanstack/ai-sandbox-docker
npm i https://pkg.pr.new/@tanstack/ai-sandbox-docker@2162@tanstack/ai-sandbox-e2b
npm i https://pkg.pr.new/@tanstack/ai-sandbox-e2b@2162@tanstack/ai-sandbox-local-process
npm i https://pkg.pr.new/@tanstack/ai-sandbox-local-process@2162@tanstack/ai-sandbox-railway
npm i https://pkg.pr.new/@tanstack/ai-sandbox-railway@2162@tanstack/ai-sandbox-sprites
npm i https://pkg.pr.new/@tanstack/ai-sandbox-sprites@2162@tanstack/ai-sandbox-upstash-box
npm i https://pkg.pr.new/@tanstack/ai-sandbox-upstash-box@2162@tanstack/ai-sandbox-vercel
npm i https://pkg.pr.new/@tanstack/ai-sandbox-vercel@2162@tanstack/ai-skills
npm i https://pkg.pr.new/@tanstack/ai-skills@2162@tanstack/ai-solid
npm i https://pkg.pr.new/@tanstack/ai-solid@2162@tanstack/ai-svelte
npm i https://pkg.pr.new/@tanstack/ai-svelte@2162@tanstack/ai-typesafe
npm i https://pkg.pr.new/@tanstack/ai-typesafe@2162@tanstack/ai-utils
npm i https://pkg.pr.new/@tanstack/ai-utils@2162@tanstack/ai-vercel-gateway
npm i https://pkg.pr.new/@tanstack/ai-vercel-gateway@2162@tanstack/ai-vertex
npm i https://pkg.pr.new/@tanstack/ai-vertex@2162@tanstack/ai-vue
npm i https://pkg.pr.new/@tanstack/ai-vue@2162@tanstack/ai-worldlabs
npm i https://pkg.pr.new/@tanstack/ai-worldlabs@2162@tanstack/openai-base
npm i https://pkg.pr.new/@tanstack/openai-base@2162@tanstack/preact-ai-devtools
npm i https://pkg.pr.new/@tanstack/preact-ai-devtools@2162@tanstack/react-ai-devtools
npm i https://pkg.pr.new/@tanstack/react-ai-devtools@2162@tanstack/solid-ai-devtools
npm i https://pkg.pr.new/@tanstack/solid-ai-devtools@2162@tanstack/svelte-ai-devtools
npm i https://pkg.pr.new/@tanstack/svelte-ai-devtools@2162commit:
ec66115
Sign in to join this conversation.
🦋 Changeset detected#
Latest commit: ec6611537b2a281a657b07fcadd0917665ef4ca7
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR