Skip to content
Source
Sign inSign up

remote.md

192 lines11.6 KB

1# Remote architecture
2​
3Each connection joins a client to one environment over HTTP and WebSocket. The
4environment owns providers, execution, files, and durable state. Direct access,
5Tailscale, SSH, and T3 Connect change how the client reaches that server; they do
6not introduce another execution model. See
7[remote access](../user/remote-access.md) for setup.
8​
9## Identity is independent of the route
10​
11An environment keeps its ID across server restarts and endpoint changes. Saved
12connections are local to a client profile; the server's identity and state are
13not. A repository identity can correlate clones across environments, but never
14routes work between them. A project and its threads belong to one environment;
15an agent reaches another environment's threads only through a peer link.
16The canonical key follows the `upstream` remote when one exists, so pull request
17features target the repository a fork tracks. A fork also reports its own
18`origin`, and clients group and label by that, so a fork never collapses into a
19checkout of its upstream.
20​
21[Environment ID initialization](../../apps/server/src/environment/ServerEnvironment.ts)
22must publish a complete ID atomically. Repair of an empty ID file retains a
23recovery file so concurrent or delayed initializers choose the same winner.
24Removing that recovery state as ordinary temporary-file cleanup can change the
25identity underneath an already-running server.
26​
27Advertised endpoints are reachability hints. Only the connecting device can
28prove that a route works. In particular, a host's loopback address refers to a
29different machine when another device opens it. Endpoint selection must not
30silently fall back to loopback when a shareable endpoint is unavailable.
31​
32A saved environment holds an ordered list of routes, and the
33[driver](../../packages/client-runtime/src/connection/driver.ts) connects over
34the first that works. Each direct route is first checked with the public
35descriptor, so a saved LAN address that a different machine answers on another
36network receives no credential. That check is not proof of a working route:
37when every route stays silent, each is still tried. A route that fails to
38connect, including a blocked one such as a signed-out T3 Connect, moves on to
39the next; only an incompatible server stops the walk, because it is the same
40server on every route. While connected over a later route the
41[supervisor](../../packages/client-runtime/src/connection/supervisor.ts)
42preflights the earlier ones and replaces the session when one would connect.
43Preflight includes authorization so a route that answers but rejects this
44client never costs a working session; a route that still fails afterwards is
45held back for a cooldown so a flaky network cannot bounce the connection.
46​
47A connected server reports the LAN and tailnet addresses it is bound to, and the
48client saves them as learned routes. A learned route reuses the credential of
49the route it was learned over: the T3 Connect access token, which is not bound
50to an origin because each DPoP proof names the URL it signs, or the paired
51bearer token. A paired token only learns a plain HTTP address off the tailnet
52when it was itself paired over plain HTTP, so a token paired over HTTPS never
53starts travelling unencrypted. Learned routes the server stops reporting are
54dropped, which is how a changed LAN address replaces the old one; routes the
55user saved are never touched. The reported addresses are hints like any advertised endpoint, so a
56learned route still has to answer as this environment before it is used.
57​
58A background service with no configured host is what makes this work for
59headless machines. It listens on every IPv4 interface and closes each
60connection unless both addresses of the direct peer are loopback, private, or
61tailnet IPv4. That keeps a VPS's public address closed, including a cloud VM
62whose public address is NATed to a private interface. It is not a firewall:
63T3 Connect and Tailscale Serve reach the server over loopback by design, and a
64proxy that rewrites the source address looks private. One listener per address would
65miss a DHCP change, and the wildcard keeps the reported addresses current.
66​
67Pull request routing trust covers the whole route list. Adding or changing a route
68revokes it; reordering does not, because the same addresses remain trusted.
69​
70## Peer links
71​
72A [peer link](../../apps/server/src/peer/PeerLinks.ts) lets one environment's
73agents work in another. The linking environment signs in to the other's `/mcp`
74as an outside MCP client, with a pairing code from it, and keeps that session's
75token in its own secret store. The other environment needs nothing new: it
76sees one more OAuth client, lists it in Sharing & access, and revokes it there.
77Unlinking on the linking side only forgets the token.
78​
79- **The Linked environments beta (`linkedEnvironmentsEnabled`) gates both
80 sides.** Off on the linking side, `PeerLinks.link`, link requests and
81 `PeerForwarding.call`/`links` refuse; `readThread`, `waitForThread` and
82 `interruptThread` do not, so a remote task already running still completes
83 and can still be cancelled. Off on the receiving side, link registrations are
84 refused and link sessions get a 403, which the linking side reports as
85 retryable so the task's follower waits rather than failing it.
86- **A tool runs in a linked environment when its parameters take
87 `OrchestratorMcpEnvironmentTarget`** as `environmentId`.
88 [`McpToolAccess.toLayer`](../../apps/server/src/mcp/McpToolAccess.ts) sends
89 such a call there once the tool's access declaration has checked the caller,
90 so a read-only or ended caller is refused before anything is sent. The
91 thread or project the call names is checked where it lives. A tool that acts
92 as the calling thread itself cannot take a linked environment.
93- **Every forwarded call carries the calling agent's modes** in
94 `T3-Mode-Limit`, which the receiving `/mcp` only ever narrows the client's
95 approved access with. A plan-mode agent therefore cannot start full-access
96 work through a full-access link.
97- **A descriptor's environment id filters addresses; it does not prove
98 identity.** The token goes only to an address that reports the linked id, so
99 a LAN address that now reports another machine's never receives it. Anything can
100 report any id, though, so only addresses from the user's client count as a
101 machine's, and never a loopback one, which names whichever machine opens
102 it. An address an agent gives is recorded alone, and
103 [`linkRequestTarget`](../../packages/client-runtime/src/linkRequest.ts) names
104 a machine for it only when the client already reaches exactly one machine
105 there. An address alone never moves an unexpired link to another address.
106- **All callers on one side share the link's single session on the other**,
107 so idempotency keys are hashed with the caller's namespace before they
108 leave. Without that, two agents reusing a key would collide on the far side.
109- **A forwarded wait is split into calls of at most 50 s**, because the
110 receiving `/mcp` sends nothing while a wait is open and T3 Connect's edge
111 drops idle requests after about 100 s.
112​
113**Work a link paired below full access starts stays within that link.** The
114receiving environment knows a session is a peer link because the linking side
115registers its OAuth client with the `t3code-peer-link` software id, and the
116marker is signed into the session. Threads that session launches carry it, with
117the access it was paired with, as an immutable `linkOrigin`, and so do the
118subagents, forks and `create_threads` they derive. The rules live in
119[`mcp/linkOrigin.ts`](../../apps/server/src/mcp/linkOrigin.ts) and run in the
120shared access declarations, not tool by tool:
121​
122- linked work may change only threads with the same origin, not the user's own
123 threads and not another link's;
124- it cannot change projects, settings or scheduled tasks, nor use the receiving
125 environment's own links;
126- its project setup scripts are skipped
127 ([`ProjectSetupScriptRunner`](../../apps/server/src/project/ProjectSetupScriptRunner.ts)
128 checks every path that runs them).
129​
130Reads are not fenced, and a client's own `thread.create` can never carry
131`linkOrigin`.
132​
133**A delegated task can run in a linked environment.** Its child is then an
134ordinary thread there, so the parent here records the task without a child
135thread of its own (`remoteChild` instead of `childThreadId`). A follower in
136[`RemoteDelegation`](../../apps/server/src/peer/RemoteDelegation.ts) waits on
137that thread and completes the task with `delegated_task.remote.complete`. That
138internal command reuses the parent half of a local finalize, so the parent
139wakes the same way. Open remote tasks are followed again at startup, which is
140also what keeps restart recovery from treating them as abandoned provider work.
141​
142The link is routing, not isolation: an agent the link starts runs as the
143receiving environment's user. **A full-access link is therefore not fenced**:
144its agents can already do anything that user can, so the fence would only stop
145T3's own tools. Its work still carries `linkOrigin`, so clients show where it
146came from. A stamp without the access, from before it was recorded, is fenced.
147​
148## Hosted web is a client
149​
150The hosted web app stores its connection catalog in the browser and connects
151directly to each environment. It does not proxy traffic or hold server-side
152pairing state. Hosting the UI over HTTPS therefore cannot make a plain HTTP LAN
153backend accessible from that browser context.
154​
155A [hosted pairing URL](../../apps/web/src/hostedPairing.ts) identifies the backend
156in its query and carries the pairing secret in its fragment. Fragments stay out
157of requests to the hosted origin. The browser exchanges the secret with the
158environment and strips it from its history. Moving the token into a query
159parameter would disclose it to the wrong origin.
160​
161## Access and process ownership are different
162​
163Tailscale supplies an endpoint for ordinary pairing, so it needs no separate
164environment type. Authentication remains the environment's responsibility for
165every route. See [environment authentication](./environment-auth.md) and the
166[T3 Connect trust boundary](./t3-connect.md).
167​
168SSH can launch a server as well as forward a port. Desktop main owns that
169lifecycle because it can spawn SSH and handle authentication prompts. The
170renderer uses the forwarded endpoint through the shared connection runtime.
171[SSH cleanup](../../packages/ssh/src/tunnel.ts) stops a remote server only if the
172launcher owns it; a server it discovered already running must survive a client
173disconnect. Reconnection restores the forward before opening the application
174transport.
175​
176Remote servers can outlive several client releases. Clients must use advertised
177capabilities and handle their absence, rather than assume their own version
178describes the server. Process replacement belongs to the launcher's
179[update protocol](./server-updates.md); the connection runtime handles the
180resulting disconnect.
181​
182### Desktop without a local environment
183​
184Desktop normally launches its own primary server, but the desktop setting `localEnvironmentEnabled`
185(`apps/desktop/src/settings/DesktopAppSettings.ts`) turns that off. Changing it relaunches the app;
186no local state is deleted. On the next start the main process skips port selection, server exposure,
187and the primary and WSL backends, and opens the window right away. The renderer sees this through
188`desktopBridge.getLocalEnvironmentEnabled()`: `readPrimaryEnvironmentTarget` returns null, so primary
189auth and platform-managed discovery are skipped and only saved environments (pairing, relay, SSH)
190connect. This is possible because the desktop renderer is not served by the backend: the `t3code://`
191scheme serves the bundled client from disk (Vite in development) and API traffic always goes to the
192environment's own URL.