CI / Speed and debugging
Debug a job in its container
A job fails in CI but passes on your machine, and the log does not tell you enough. Run the job again with debug, and look inside its container: the files, the environment and the versions of the tools.
4 min read
This works like tmate on GitHub Actions or "Rerun job with SSH" on CircleCI. After its last step, the container stays for 15 minutes, also when the job failed.
You need:
- Write access to the repository. In a protected run, you need admin access.
Tip
To have your own coding agent read the failed log on your computer, click Send to my agent on the run page. See Send to your agent.
1. Run the failed jobs again with debug
Use one of these:
- On the run page, click Re-run failed jobs with debug.
- In a terminal, run
src ci rerun <run> --failed --debug.src run rerun <run> --failed --debugdoes the same, asgh run rerundoes. - From an agent, call the MCP tool
rerun_ci_runwithfailedOnly: trueanddebug: true.
The jobs run again. When a job finishes its last step, its container stays for 15 minutes. The job stays "running" until then.
2. Find the key of the job
The key is the name of the job with its matrix cell, like test or test (shard: 1):
src ci view <run>
3. Run commands in the container
Run one command. The words after -- are the command:
src ci exec <run> "test (shard: 1)" -- node --version
src ci exec <run> "test (shard: 1)" -- cat test-results/report.txt
The command runs in the checkout of the job, /workspace/repo. Give another folder with --cwd /workspace. The exit code of src ci exec is the exit code of the command.
Two more ways to run commands:
- A shell: run
src ci shell <run> "test (shard: 1)", and type one command on each line.cdchanges the folder for the next lines. Typeexitto leave. - The web: open the run, select the job, and type the command in the Debug card. The output shows under the field.
Note
The shell is not a full terminal. Each line runs as its own command, so there are no prompts, no editors and no Ctrl+C.
4. End the debug window
When you are done, end the 15 minutes early, so the job ends and its container goes:
src ci exec <run> "test (shard: 1)" --done
On the web, click End the debug window. Cancelling the run ends it too.
Done
You looked inside the container of the failed job, with the environment of the job. The job then ended, and its container went.
You can also run commands in the container of a job while its steps run, without debug. The container goes when the job ends.
What works
Each command:
- Runs for at most 60 seconds. A longer command is stopped, and the answer says so.
- Gives you the exit code and the last 64 KiB of the standard output and of the standard error. When the output is longer, the answer says that it was cut.
- Gets the environment of the job:
CI, theTSS_*variables, the CI variables of the job, and a placeholder for each secret. A secret's value never enters the container, and a command can send it only to the hosts of the secret, as a step can. - Goes to the audit trail of the run, with the person who ran it. Its output does not.
Who can run commands:
- Only people with write access can run commands, also with a scoped token that has the
jobs:runaction. Agent sessions and CI jobs cannot. For a run of a private pull request, you need to see the pull request too. - In a protected run, only admins can run commands, and only admins run it again with debug. A protected run is a run at the trust level
protected, like a push to a branch whose branch rule requires a pull request. Trust levels gives the level of each event. Its jobs hold the protected secrets, write the protected cache and can push to protected branches. A command in its container can use all three. - A job of a hidden workflow runs no commands of people, also for admins, and a hidden run does not run again with debug. Its files must stay hidden.
The timing of the job shows the 15 minutes as its own part, Debug window. See Find out where your CI time goes.
The same feature is in REST and MCP: POST /api/v1/repos/<owner>/<repo>/ci/runs/<run>/jobs/<job>/exec with { "command": "ls -la", "cwd": "/workspace/repo" }, or { "done": true }, and the MCP tool exec_in_ci_job. See the CI reference.