[Bug]: Storage cleanup removes an active thread's worktree, with its ignored files, hours after creation #33881
Send to my agentBefore submitting#
- I searched existing issues and did not find a duplicate.
- I included enough detail to reproduce or investigate the problem.
Area#
apps/web
Steps to reproduce#
Summary#
The hourly storage cleanup ran git worktree remove on the worktree of a thread that was 10 hours old, not archived and not settled. The thread was idle between user messages. A long-running detached process was still writing into the worktree. Everything git ignores was lost with it: about 4.6 GB of run output, node_modules and a generated build project. On the thread's next message T3 logged "recreating missing worktree" and gave it an empty fresh checkout, with no notice to the user or the agent.
Version and settings#
- T3 server 0.1.8 (
t3 serve), Linux, Claude agent provider. settings.json:"storageCleanup": { "worktreeAfterDays": 15, "worktreeUnchanged": true }worktreeKeepWhennot set (defaultuncommitted-changes).
Timeline (local time)#
| Time | Event |
|---|---|
| ~05:30 | Thread created in a new worktree from origin/master. The agent never commits in it; its PR work is in a second worktree. |
| 05:30 to 15:17 | The agent builds there and runs long jobs that write about 4.6 GB into the git-ignored logs/ folder. |
| ~15:17 | The agent ends its turn. A detached job it started keeps running and writing into logs/. |
| 15:19:44 | Storage cleanup sweep starts . |
| 15:20:19 | INFO: storage cleanup removed worktree { threadId: '<thread-id>' }. The detached job dies mid-write. |
| 15:24:39 | The user sends a message. WARN: provider turn start recreating missing worktree { threadId: '<thread-id>', worktreePath: …, branch: '<branch>' }. The agent finds an empty checkout. |
Why it was picked#
From the bundled server code (the StorageCleanup worktree pass):
eligible = integrated && settings.worktreeUnchanged;
if (eligible) removalReason = remote === null ? "no commits beyond the project branch" : "no commits beyond the default branch";
worktreeUnchanged: truemakes a worktree whose HEAD is an ancestor of the default branch eligible right away.worktreeAfterDays: 15doesn't hold it back, so a worktree created this morning can go at the next sweep.- "Changes" means uncommitted tracked or untracked files only. Ignored files (
logs/,node_modules, build output) don't count, so gigabytes of results look like an empty worktree. - The liveness checks look at the thread's idle state, open terminals and live provider sessions. They don't look at processes whose working directory is inside the worktree. A detached build or test job started by the agent doesn't protect it.
- Nothing tells the thread. The next turn silently recreates the worktree, so the agent and the user only find out when files are missing.
Expected#
At least one of these:
worktreeAfterDaysapplies to every cleanup reason, including "no commits beyond the default branch", so a worktree under 15 days old is never removed for being unchanged.- A thread that isn't archived or settled keeps its worktree, whatever its branch state.
- A worktree with any process running inside it (cwd or open files) is skipped, like an open terminal.
- If a worktree that holds ignored files is about to be removed, it is skipped, or at least the ignored size is shown and the user is asked first.
- When a removed worktree is recreated, the thread gets a visible notice saying when and why it was removed.
Impact#
Lost: the outputs of four multi-hour device test runs (per-run results, database copies, grading files). One run was killed mid-way and its partial results are gone. The phone time can't be recovered: each run takes 2.5 to 3 hours on a shared device.
Steps to reproduce#
- Set
storageCleanup.worktreeUnchanged: trueand leaveworktreeAfterDaysat any value. - Start a thread in a new worktree on a branch with no commits beyond the default branch.
- In the worktree, write large output into a git-ignored folder, and start a detached process (
setsid sleep 3600 &) with that folder as its cwd. - Let the thread go idle and wait for the next storage cleanup sweep (hourly).
- The worktree is removed, and the process loses its cwd. The next message to the thread logs "recreating missing worktree".
Expected behavior#
.
Actual behavior#
.
Impact#
Minor bug or occasional failure
Version or commit#
No response
Environment#
No response
Logs or stack traces#
Screenshots, recordings, or supporting files#
No response
Workaround#
No response
smonev(GitHub profile, opens in a new tab) added labelsbug
smonev(GitHub profile, opens in a new tab) added labelsneeds-triage
smonev(GitHub profile, opens in a new tab) closed this
Sign in to join this conversation.
it was a t3 setting - "Delete unchanged worktrees" and it was ON.