Skip to content
Source
Sign inSign up

Install dependencies and share caches

Work with agents / Local workspaces

Install dependencies and share caches

An install in each workspace takes minutes, and it fills the disk with copies of the same packages. src ws makes each dependency folder (a layer) once for each set of inputs, and gives the next workspace a clone.

5 min read

Note

The built-in recipes work with no config and need no trust. The steps below are for a project that needs another install command or another layer.

1. Change a recipe or add a layer

Add .source/workspace.yaml to the repository:

layers:
  # Change one field of a built-in recipe.
  npm:
    install: npm ci --no-audit
  # Turn a built-in recipe off.
  bun: false
  # Add a layer of your own.
  tools:
    paths: [tools/bin]
    keyFiles: [tools/versions.txt]
    install: node scripts/fetch-tools.mjs
    env:
      TOOLS_HOME: tools/bin

A new layer needs paths, keyFiles, and install or native. See The fields of a layer.

2. Trust the config

src ws trust

Until you trust it, a built-in recipe that the config changes uses its built-in command. A layer that only the config defines is marked skipped, not trusted. See Trust the commands of the config.

3. Create a workspace

src ws create try-config

The output says which strategy ran for each layer, and how long it took:

Created try-config at /code/.src-ws/app/ws-1 (baseline 1a2b3c4d5e6f, from HEAD)
Layers of try-config:
  pnpm: installed in 19.3s

A second create with the same lockfile says cloned from the cache, or says that the layers were ready in its folder from the pool. When you are done, run src ws close try-config.

Done

Each new workspace gets its own copy of each layer, with the strategy that the output names.

How a workspace gets its layers

src ws create copies the files first. Then, for each layer, it tries these strategies in order:

  1. Tool-native sharing. For example, pnpm 10.12.1 or later links one shared store into each workspace.
  2. A clone of a cached layer with the same key.
  3. The install command of the layer.

The key of a layer is a hash of the inputs that decide its content:

  • The manifests and the lockfile, for example package.json and pnpm-lock.yaml.
  • The name and version of the package manager, and its config files, for example .npmrc and pnpm-workspace.yaml.
  • The runtime version files: .nvmrc, .node-version and .python-version.
  • The OS and the CPU.

When an install runs for a new key, src saves a clone of the result in ../.src-ws/<repository folder>/.layers/<key>. The next workspace with the same key clones it.

  • Each workspace gets its own writable copy of each layer. A package that an agent adds in one workspace does not show in another.
  • Links in a layer, for example node_modules/@scope/pkg in a monorepo, point at the packages of the same workspace. If a cached layer has a link into another workspace, src does not clone it. It runs the install.

Built-in recipes

src picks a recipe when it finds its lockfile at the root of the repository:

LockfileLayer foldersStrategies
pnpm-lock.yamlnode_modules of each packagepnpm 10.12.1 or later: pnpm install --frozen-lockfile with the global virtual store. Older pnpm: the same install, which links the packages from the store of pnpm.
yarn.locknode_modules of each packageA clone, then yarn install --immutable (--frozen-lockfile for yarn 1).
bun.lock, bun.lockbnode_modules of each packagebun install --frozen-lockfile only.
package-lock.jsonnode_modules of each packageA clone, then npm ci.
uv.lock.venvuv sync --frozen, with UV_LINK_MODE=clone.
requirements.txt.venvpython3 -m venv .venv (python on Windows), then pip install -r requirements.txt.
  • One package manager owns node_modules. If the repository has more than one lockfile, the first row in the table wins.
  • pnpm and bun link or clone their packages from their own store, so src never clones their node_modules.
  • With the pnpm global virtual store, a package must declare the types that its declarations import. Add missing type dependencies through packageExtensions in the repository's pnpm-workspace.yaml, then update its lockfile. TanStack Source pins the missing React type edges there. src does not add these extensions to other repositories.
  • A .venv holds absolute paths, so src never clones it. uv and pip each keep a shared download cache, so a second install downloads nothing.
  • With pnpm older than 10.12.1, create says that a newer pnpm makes workspaces faster.

The fields of a layer

FieldWhat it does
pathsThe folders of the layer, as globs: node_modules, packages/*/node_modules, **/node_modules. The last part is a folder name.
keyFilesGlobs of tracked files whose content goes into the key.
installThe install command. The shell runs it in the workspace folder.
nativeA command for tool-native sharing, tried first.
relockA command that makes the lockfile again from the manifests. accept and sync run it when both sides changed a lockfile that detect names.
pathBoundtrue for a layer with absolute paths. src never clones it.
clonefalse to never clone a cached copy.
envVariables for the install command and for src ws exec.
detectThe layer applies only when one of these files is at the root.

Caches

Build tools keep caches. Some are safe for many workspaces at the same time, and some are not.

Shared caches are in ../.src-ws/<repository folder>/.caches/<name>. src ws exec points each tool at its folder:

ToolVariable
TurborepoTURBO_CACHE_DIR
NxNX_CACHE_DIRECTORY
sccacheSCCACHE_DIR

Each workspace keeps its own copy of these caches:

  • node_modules/.vite
  • .next/cache
  • .wrangler
  • Rust target/, when sccache is not on PATH. When it is, src ws exec sets RUSTC_WRAPPER=sccache, and the builds share compiled crates through sccache.

create clones these caches from the base folder when the base folder has them. src ws prepare copies them from your checkout into the base folder when copy-on-write works and your checkout is on the same disk.

Add or change a cache in the caches section:

caches:
  # A shared cache: src ws exec sets the variable to its folder.
  bazel:
    env: BAZEL_DISK_CACHE
  # A cache of each workspace, cloned from the base folder.
  astro:
    paths: ["**/.astro"]
  # Turn a built-in cache off.
  turbo: false

src ws gc trims the shared caches of the repository to 10 GiB. It deletes the oldest files first.

  • It never trims the pnpm store of a disk image (see pnpm on the image), because a store with missing files breaks the next install. Run pnpm store prune for it.
  • It also deletes the cached layers that no open workspace and no kept base folder uses.
  • It deletes the ready workspaces above the size of the pool.