Skip to content
Source
Sign inSign up
TanStack/ai

fix(ai-sandbox-local-process): decode exec() output across chunks #2162

Openmohamadgarmabi(GitHub profile, opens in a new tab)wants to merge 4 commits intomain fromrefs/pull/2162/headopened View on GitHub

🎯 Changes

Fixes #2153.

exec() decoded each stdout/stderr chunk on its own with d.toString('utf8'). When a pipe splits a multi-byte UTF-8 character across two reads, both halves become U+FFFD.

The fix calls setEncoding('utf8') on both streams. Node then decodes with a StringDecoder, which keeps a partial sequence until the rest arrives. spawn() already handles this with decodeStream. A truncated sequence at the end of the stream still comes out as U+FFFD, the same as spawn().

Test: a new case in local-process.test.ts writes the first byte of € to stdout and stderr, then the rest 50 ms later, and checks that exec() returns € for both. It mirrors the existing spawn() test.

Docs: not changed. This is a bug fix with no API change.

✅ Checklist

  • I have followed the steps in the Contributing guide.
  • I have tested code changes locally with pnpm run test:pr, or these tests do not apply to this pull request.
  • I fully understand the code in this pull request, including any code generated with AI assistance.
  • Docs: I updated docs/ for this change, or this change is not user-facing.
  • Changeset: I added a changeset (pnpm changeset), or this PR does not change a published package.

🚀 Release Impact

  • This change affects published code, and I have generated a changeset.
  • This change is docs/CI/dev-only (no release).

Summary by CodeRabbit

  • Bug Fixes
    • Fixed exec() output decoding so UTF-8 characters split across reads are returned correctly in both standard output and standard error.
    1. 🦋 Changeset detected

      Latest commit: ec6611537b2a281a657b07fcadd0917665ef4ca7

      The changes in this PR will be included in the next version bump.

      This PR includes changesets to release 1 package
      NameType
      @tanstack/ai-sandbox-local-processPatch

      Not sure what this means? Click here to learn what changesets are.

      Click here if you're a maintainer who wants to add another changeset to this PR

    2. tanstack-source added labelsbug

    3. Explain Change →

      No actionable comments were generated in the recent review.

      Walkthrough

      exec() now decodes stdout and stderr as UTF-8 across stream chunks. Unit and end-to-end tests check that split UTF-8 bytes produce the expected € character in both outputs.

      Changes: Local process exec UTF-8 decoding

      Layer / File(s)Summary
      Exec stream decoding and unit validation
      packages/ai-sandbox-local-process/src/handle.ts, packages/ai-sandbox-local-process/tests/local-process.test.ts, .changeset/local-process-exec-utf8.md
      exec() uses stream-level UTF-8 decoding for stdout and stderr. The unit test checks that split bytes produce € in both outputs. The changeset records the patch.
      End-to-end output validation
      testing/e2e/src/routes/api.sandbox-local-process-exec.ts, testing/e2e/src/routeTree.gen.ts, testing/e2e/tests/sandbox-local-process-exec.spec.ts
      The route runs a script that writes split UTF-8 bytes to stdout and stderr, then returns the execution result. The generated route tree registers the endpoint. The Playwright test checks the output, exit code, and absence of replacement characters.

      Estimated code review effort: 2 (Simple) | ~10 minutes

      Change: Bug fix · Severity of issue fixed: Medium

      Suggested reviewers: alemtuzlak

      Merge Risk: Minimal · up to ec661

      This fixes corruption of multi-byte characters in exec() output when they are split across reads. The change is small, matches the linked issue, and is covered by unit and end-to-end tests. I found no material merge risk.

      Pre-merge checks: Passed 4

      Check nameStatusExplanation
      Title checkPassedThe title names the affected package and the behavior fixed: exec() output is now decoded across chunks. It matches the main change in the changeset, handle.ts, and the new tests. The title says 'deco…
      Description checkPassedThe description follows the template. It has the Changes section with the cause, the fix, the test, and a note that docs were not changed. All checklist items are present and marked. It links issue #2…
      Linked Issues checkPassed[#2153] requires LocalProcessHandle.exec() to decode stdout and stderr without corrupting multi-byte UTF-8 characters split across pipe reads. The whole-PR summary shows `packages/ai-sandbox-local-p…
      Out of Scope Changes checkPassedThe whole-PR diff against merge base 1888233 contains these areas. (1) handle.ts: exec() stdout and stderr collection only, which implements #2153. (2) local-process.test.ts: one new unit test…
      Full details: Title check

      Explanation

      The title names the affected package and the behavior fixed: exec() output is now decoded across chunks. It matches the main change in the changeset, handle.ts, and the new tests. The title says 'decode exec() output across chunks' and does not name the UTF-8 symptom, but it still identifies the primary change.

      Full details: Description check

      Explanation

      The description follows the template. It has the Changes section with the cause, the fix, the test, and a note that docs were not changed. All checklist items are present and marked. It links issue #2153 and explains the StringDecoder approach. The release impact section is filled in. It is mostly complete. Minor gap: the Changes section does not state the changeset file explicitly, though the checklist marks it done.

      Full details: Linked Issues check

      Explanation

      [#2153] requires LocalProcessHandle.exec() to decode stdout and stderr without corrupting multi-byte UTF-8 characters split across pipe reads. The whole-PR summary shows packages/ai-sandbox-local-process/src/handle.ts changed in exec() (+4/-2). The change calls setEncoding('utf8') on stdout and stderr before collecting data. This replaces the per-chunk d.toString('utf8') calls. Node uses a StringDecoder for these streams, which holds a partial sequence until the remaining bytes arrive. Automated coverage exists in packages/ai-sandbox-local-process/tests/local-process.test.ts, which writes the first byte of € to stdout and stderr, writes the rest after 50 ms, and asserts that both outputs equal €. An end-to-end test in testing/e2e/tests/sandbox-local-process-exec.spec.ts also checks split-byte output through /api/sandbox-local-process-exec. Both tests cover the reported defect. The issue has no non-coding requirements. The truncated-sequence case still yields U+FFFD, which the issue does not require to change. The reviewed summary is the evidence for the handle.ts change; no source hunk was re-read in this review.

      Full details: Out of Scope Changes check

      Explanation

      The whole-PR diff against merge base 1888233 contains these areas. (1) handle.ts: exec() stdout and stderr collection only, which implements #2153. (2) local-process.test.ts: one new unit test for split-character decoding. (3) .changeset/local-process-exec-utf8.md: a patch changeset, which the PR description requires. (4) testing/e2e/src/routes/api.sandbox-local-process-exec.ts, testing/e2e/tests/sandbox-local-process-exec.spec.ts, and the generated testing/e2e/src/routeTree.gen.ts entries: an end-to-end route and spec that exercise the same split-byte scenario through exec(). These support the objective as tests. The routeTree.gen.ts changes are generated registration for the new route. No public declarations change, and the spawnProcess() path is not modified. The new code is connected to #2153 and no unrelated change is shown.

      Finishing Touches

      Generate unit tests (beta)

      • Create a new PR
      • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

      Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

      Share

      Recent review info

      Run configuration

      • Source mode: Auto
      • Repository: TanStack/ai
      • Branch: main
      • File: .coderabbit.yaml
      • Commit: 1888233
      • Layers: Repository YAML
      • Review profile: CHILL
      • Plan: Advanced
      • Run ID: ea1332c7-4296-430e-bed4-22ab557d9064
      Commits

      Reviewing files that changed from the base of the PR and between f843c1c7eb2c7dcd43733651b16646d3332559cb and ec6611537b2a281a657b07fcadd0917665ef4ca7.

      Files selected for processing: 3

      • testing/e2e/src/routeTree.gen.ts
      • testing/e2e/src/routes/api.sandbox-local-process-exec.ts
      • testing/e2e/tests/sandbox-local-process-exec.spec.ts

      Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

    4. Thanks for the PR, @mohamadgarmabi! 🙌 @tombeckenham will take a look.

      Automated pre-review checks

      • ✅ CI passing
      • ✅ No merge conflicts
      • ✅ Changeset present
      • ⚠️ No E2E test changes detected — behavior changes need coverage under testing/e2e/ (see CONTRIBUTING)
      Automated triage — a human review follows.
    5. github-actions[bot](GitHub profile, opens in a new tab) added labelswaiting-on: maintainer

    6. Added E2E coverage in testing/e2e/tests/sandbox-local-process-exec.spec.ts (exec() output with 2/3/4-byte UTF-8 characters split across chunks, on both stdout and stderr). Could a maintainer approve the workflow runs when you have a moment? Thanks!

    7. tanstack-sourceBotcommented

      This pull request is useful. It fixes exec() UTF-8 decoding in the local-process sandbox. It links #2153. No open pull request does the same work.

      TanStack Source bot, value agent

    8. github-actions[bot](GitHub profile, opens in a new tab) added labelswaiting-on: author

    9. github-actions[bot](GitHub profile, opens in a new tab) removed labelswaiting-on: maintainer

    10. AlemTuzlak approved these changes

    11. AlemTuzlak turned on merge when ready (merge)

    12. View your CI Pipeline Execution ↗ for commit ec6611537b2a281a657b07fcadd0917665ef4ca7

      CommandStatusDurationResult
      nx run-many --targets=build --exclude=examples/...✅ Succeeded20sView ↗

      💡 With Ultracache enabled, Nx Cloud learns each task's file inputs in CI and uses them on later runs. Turn on Ultracache in workspace settings ↗ · Read docs ↗


      ☁️ Nx Cloud last updated this comment at 2026-11-19 12:26:04 UTC

    13. Open in StackBlitz

      @tanstack/ai

      npm i https://pkg.pr.new/@tanstack/ai@2162

      @tanstack/ai-acp

      npm i https://pkg.pr.new/@tanstack/ai-acp@2162

      @tanstack/ai-angular

      npm i https://pkg.pr.new/@tanstack/ai-angular@2162

      @tanstack/ai-anthropic

      npm i https://pkg.pr.new/@tanstack/ai-anthropic@2162

      @tanstack/ai-bedrock

      npm i https://pkg.pr.new/@tanstack/ai-bedrock@2162

      @tanstack/ai-byteplus

      npm i https://pkg.pr.new/@tanstack/ai-byteplus@2162

      @tanstack/ai-claude-code

      npm i https://pkg.pr.new/@tanstack/ai-claude-code@2162

      @tanstack/ai-client

      npm i https://pkg.pr.new/@tanstack/ai-client@2162

      @tanstack/ai-cloudflare

      npm i https://pkg.pr.new/@tanstack/ai-cloudflare@2162

      @tanstack/ai-code-mode

      npm i https://pkg.pr.new/@tanstack/ai-code-mode@2162

      @tanstack/ai-code-mode-snippets

      npm i https://pkg.pr.new/@tanstack/ai-code-mode-snippets@2162

      @tanstack/ai-codex

      npm i https://pkg.pr.new/@tanstack/ai-codex@2162

      @tanstack/ai-cohere

      npm i https://pkg.pr.new/@tanstack/ai-cohere@2162

      @tanstack/ai-compaction

      npm i https://pkg.pr.new/@tanstack/ai-compaction@2162

      @tanstack/ai-computer-local

      npm i https://pkg.pr.new/@tanstack/ai-computer-local@2162

      @tanstack/ai-computer-playwright

      npm i https://pkg.pr.new/@tanstack/ai-computer-playwright@2162

      @tanstack/ai-computer-sandbox

      npm i https://pkg.pr.new/@tanstack/ai-computer-sandbox@2162

      @tanstack/ai-computer-use

      npm i https://pkg.pr.new/@tanstack/ai-computer-use@2162

      @tanstack/ai-devtools-core

      npm i https://pkg.pr.new/@tanstack/ai-devtools-core@2162

      @tanstack/ai-durable-stream

      npm i https://pkg.pr.new/@tanstack/ai-durable-stream@2162

      @tanstack/ai-effect

      npm i https://pkg.pr.new/@tanstack/ai-effect@2162

      @tanstack/ai-elevenlabs

      npm i https://pkg.pr.new/@tanstack/ai-elevenlabs@2162

      @tanstack/ai-event-client

      npm i https://pkg.pr.new/@tanstack/ai-event-client@2162

      @tanstack/ai-fal

      npm i https://pkg.pr.new/@tanstack/ai-fal@2162

      @tanstack/ai-gemini

      npm i https://pkg.pr.new/@tanstack/ai-gemini@2162

      @tanstack/ai-grok

      npm i https://pkg.pr.new/@tanstack/ai-grok@2162

      @tanstack/ai-grok-build

      npm i https://pkg.pr.new/@tanstack/ai-grok-build@2162

      @tanstack/ai-groq

      npm i https://pkg.pr.new/@tanstack/ai-groq@2162

      @tanstack/ai-harness

      npm i https://pkg.pr.new/@tanstack/ai-harness@2162

      @tanstack/ai-harness-cli

      npm i https://pkg.pr.new/@tanstack/ai-harness-cli@2162

      @tanstack/ai-isolate-cloudflare

      npm i https://pkg.pr.new/@tanstack/ai-isolate-cloudflare@2162

      @tanstack/ai-isolate-daytona

      npm i https://pkg.pr.new/@tanstack/ai-isolate-daytona@2162

      @tanstack/ai-isolate-e2b

      npm i https://pkg.pr.new/@tanstack/ai-isolate-e2b@2162

      @tanstack/ai-isolate-node

      npm i https://pkg.pr.new/@tanstack/ai-isolate-node@2162

      @tanstack/ai-isolate-quickjs

      npm i https://pkg.pr.new/@tanstack/ai-isolate-quickjs@2162

      @tanstack/ai-isolate-quickjs-bun

      npm i https://pkg.pr.new/@tanstack/ai-isolate-quickjs-bun@2162

      @tanstack/ai-llmgateway

      npm i https://pkg.pr.new/@tanstack/ai-llmgateway@2162

      @tanstack/ai-lovable

      npm i https://pkg.pr.new/@tanstack/ai-lovable@2162

      @tanstack/ai-mcp

      npm i https://pkg.pr.new/@tanstack/ai-mcp@2162

      @tanstack/ai-memory

      npm i https://pkg.pr.new/@tanstack/ai-memory@2162

      @tanstack/ai-mistral

      npm i https://pkg.pr.new/@tanstack/ai-mistral@2162

      @tanstack/ai-muse

      npm i https://pkg.pr.new/@tanstack/ai-muse@2162

      @tanstack/ai-neon

      npm i https://pkg.pr.new/@tanstack/ai-neon@2162

      @tanstack/ai-octane

      npm i https://pkg.pr.new/@tanstack/ai-octane@2162

      @tanstack/ai-ollama

      npm i https://pkg.pr.new/@tanstack/ai-ollama@2162

      @tanstack/ai-ollaya

      npm i https://pkg.pr.new/@tanstack/ai-ollaya@2162

      @tanstack/ai-openai

      npm i https://pkg.pr.new/@tanstack/ai-openai@2162

      @tanstack/ai-opencode

      npm i https://pkg.pr.new/@tanstack/ai-opencode@2162

      @tanstack/ai-openrouter

      npm i https://pkg.pr.new/@tanstack/ai-openrouter@2162

      @tanstack/ai-perplexity

      npm i https://pkg.pr.new/@tanstack/ai-perplexity@2162

      @tanstack/ai-persistence

      npm i https://pkg.pr.new/@tanstack/ai-persistence@2162

      @tanstack/ai-preact

      npm i https://pkg.pr.new/@tanstack/ai-preact@2162

      @tanstack/ai-react

      npm i https://pkg.pr.new/@tanstack/ai-react@2162

      @tanstack/ai-reactor

      npm i https://pkg.pr.new/@tanstack/ai-reactor@2162

      @tanstack/ai-remix

      npm i https://pkg.pr.new/@tanstack/ai-remix@2162

      @tanstack/ai-sandbox

      npm i https://pkg.pr.new/@tanstack/ai-sandbox@2162

      @tanstack/ai-sandbox-blaxel

      npm i https://pkg.pr.new/@tanstack/ai-sandbox-blaxel@2162

      @tanstack/ai-sandbox-boxd

      npm i https://pkg.pr.new/@tanstack/ai-sandbox-boxd@2162

      @tanstack/ai-sandbox-cloudflare

      npm i https://pkg.pr.new/@tanstack/ai-sandbox-cloudflare@2162

      @tanstack/ai-sandbox-daytona

      npm i https://pkg.pr.new/@tanstack/ai-sandbox-daytona@2162

      @tanstack/ai-sandbox-docker

      npm i https://pkg.pr.new/@tanstack/ai-sandbox-docker@2162

      @tanstack/ai-sandbox-e2b

      npm i https://pkg.pr.new/@tanstack/ai-sandbox-e2b@2162

      @tanstack/ai-sandbox-local-process

      npm i https://pkg.pr.new/@tanstack/ai-sandbox-local-process@2162

      @tanstack/ai-sandbox-railway

      npm i https://pkg.pr.new/@tanstack/ai-sandbox-railway@2162

      @tanstack/ai-sandbox-sprites

      npm i https://pkg.pr.new/@tanstack/ai-sandbox-sprites@2162

      @tanstack/ai-sandbox-upstash-box

      npm i https://pkg.pr.new/@tanstack/ai-sandbox-upstash-box@2162

      @tanstack/ai-sandbox-vercel

      npm i https://pkg.pr.new/@tanstack/ai-sandbox-vercel@2162

      @tanstack/ai-skills

      npm i https://pkg.pr.new/@tanstack/ai-skills@2162

      @tanstack/ai-solid

      npm i https://pkg.pr.new/@tanstack/ai-solid@2162

      @tanstack/ai-svelte

      npm i https://pkg.pr.new/@tanstack/ai-svelte@2162

      @tanstack/ai-typesafe

      npm i https://pkg.pr.new/@tanstack/ai-typesafe@2162

      @tanstack/ai-utils

      npm i https://pkg.pr.new/@tanstack/ai-utils@2162

      @tanstack/ai-vercel-gateway

      npm i https://pkg.pr.new/@tanstack/ai-vercel-gateway@2162

      @tanstack/ai-vertex

      npm i https://pkg.pr.new/@tanstack/ai-vertex@2162

      @tanstack/ai-vue

      npm i https://pkg.pr.new/@tanstack/ai-vue@2162

      @tanstack/ai-worldlabs

      npm i https://pkg.pr.new/@tanstack/ai-worldlabs@2162

      @tanstack/openai-base

      npm i https://pkg.pr.new/@tanstack/openai-base@2162

      @tanstack/preact-ai-devtools

      npm i https://pkg.pr.new/@tanstack/preact-ai-devtools@2162

      @tanstack/react-ai-devtools

      npm i https://pkg.pr.new/@tanstack/react-ai-devtools@2162

      @tanstack/solid-ai-devtools

      npm i https://pkg.pr.new/@tanstack/solid-ai-devtools@2162

      @tanstack/svelte-ai-devtools

      npm i https://pkg.pr.new/@tanstack/svelte-ai-devtools@2162

      commit: ec66115

Sign in to join this conversation.