Open History to see the last commit
1# Release Checklist
2
3 For maintainers. Using T3 Code? See [docs/user](../user/).
4
5This document covers the unified release workflow for stable and nightly desktop releases.
6
7Beta is the product label, not a release channel or version suffix. Release channels
8remain stable and nightly. To ship 0.1.0 Beta, use `channel=stable` and
9`version=0.1.0`. Removing the Beta label later is a branding change independent of
10the version.
11
12## What the workflow does
13
14Workflow: `.github/workflows/release.yml`
15Triggers:
16manual `workflow_dispatch` with `channel=stable`, the normal way to ship stable. Stable
17 and nightly dispatches must select `main`; preview may select any branch. The channel defaults
18 to preview so an omitted selection cannot publish a stable release.
19push tag matching `v*.*.*` for a stable release of an explicit commit
20scheduled nightly check every 30 minutes
21manual `workflow_dispatch` with `channel=nightly`
22manual `workflow_dispatch` with `channel=preview`, the maintainers' test train. It exercises the release flow up to a draft release (build, sign, notarize, smoke) for a commit that end users must never receive, which is how an unmerged branch or a risky change gets a real release run before it lands. It builds the triggering commit with nightly's versioning under the `preview` prerelease identifier (`0.0.41-preview.<date>.<run>`) and leaves its GitHub prerelease as a draft. It publishes nothing to npm. It stays a draft because a draft has no git tag, and published preview tags hide nightlies from the updater (see [Update feed](#update-feed)). Do not publish a preview draft by hand. Preview is not on the schedule, its desktop builds carry no update feed, and no updater manifest (`latest*.yml`, `nightly*.yml`, blockmaps) is attached, so a stable or nightly install cannot be offered one. The only ways onto a new preview are by hand: the draft release, which maintainers can open, and the run's Actions artifacts. A draft's files are not public, so `T3CODE_CHANNEL=preview` and `t3 update --channel preview` see only previews published before this rule, and a fresh `npx t3@preview` fails while npm's `preview` dist-tag names a draft. Rebase a branch onto `main` before dispatching a preview from it. The run uses the branch's own `release.yml`, and an older copy publishes the preview with a tag. The release itself is named as a maintainer test build and its body is a warning rather than generated notes: a changelog of unmerged branch history is not a changelog, and nightly and stable notes are unaffected because each series resolves its previous tag within its own channel. The hosted web app, AUR, and Discord announcements are skipped. Keep it; it costs nothing when idle.
23A manual stable release builds the commit of the latest published nightly, not `main` HEAD.
24 Nightly is the release candidate: verify the nightly, then promote it. Merges to `main` keep
25 landing while you verify and never leak into the stable build.
26The version defaults to the one the nightly previewed (`0.0.39-nightly.*` ships as `0.0.39`).
27 Pass the `version` input to override it, for example for a minor bump.
28The stable tag is created on the nightly's commit when the GitHub Release is published.
29Pushing a `vX.Y.Z` tag by hand still works and builds exactly the tagged commit. Use it when
30 the commit to ship is not the latest nightly, such as a cherry-picked fix on a release branch.
31Runs lint, typecheck, and tests alongside artifact builds. Publishing waits for every check.
32Reads the shared production T3 Connect relay URL and Clerk client configuration before packaging clients.
33Builds the platform-independent JS (server bundle, web client, Electron main) once in the `build_bundle` job and hands it to every platform job as the `js-bundle` artifact; the platform jobs only package it, so no runner rebuilds it.
34Builds six desktop artifacts in parallel for both channels, each as its own job (`desktop_<platform>_<arch>`, one call of `release-desktop.yml`), gated only on the bundle. macOS and Linux targets build on hardware of their own architecture. Both Windows targets build and sign (with jsign) on a Linux runner, which packages faster than Windows runners, and a Windows job of the target's architecture (`windows-2025` or `windows-11-arm`, the `verify_runner`) runs the packaged natives and the CLI archive. The Windows jobs embed the same-arch Linux CLI archive as the WSL runtime and wait for that artifact partway through, not for the whole Linux job:
35macOS `arm64` DMG
36macOS `x64` DMG
37Linux `x64` and `arm64` AppImage and `.deb`, from one electron-builder run. The `.deb` updates in the app through electron-updater, which installs it with `dpkg`.
38Windows `x64` and `arm64` NSIS installer
39Opens the packaged Linux x64 AppImage (under Xvfb) and macOS arm64 DMG against a seeded database and opens a thread (`scripts/smoke-desktop-app.ts`). A build whose backend or bundle cannot start fails here, so nothing is published. The database in `scripts/fixtures/desktop-launch` is an old snapshot on purpose: the packaged server must migrate it forward, as it does for upgrading users. Do not regenerate it on schema changes.
40Publishes one GitHub Release with all produced files.
41Stable tags with a suffix after `X.Y.Z` (for example `1.2.3-alpha.1`) are published as GitHub prereleases.
42Only plain stable `X.Y.Z` releases are marked as the repository's latest release.
43Nightly runs are always GitHub prereleases and never marked latest.
44Automatically generated release notes are pinned to the previous tag in the same channel, so stable compares to the previous stable tag and nightly compares to the previous nightly tag.
45The release job's **Write release notes** step (`scripts/generate-release-notes.ts`) builds the body: a computed header line, a short summary from `gpt-6.1-terra`, and GitHub's generated list folded under `<details>`. It needs the `OPENAI_API_KEY` secret. The desktop update popover shows up to seven highlights and the number of other fixes and performance improvements. Mobile-only commits count toward release totals but cannot be highlights. The offered release's age comes from the update feed's `releaseDate`. The nightly Discord post shows the part above `<details>`. If summary generation fails, the release uses GitHub's generated notes.
46Includes Electron auto-update metadata (for example `latest*.yml`, `nightly*.yml`, and `*.blockmap`) in release assets.
47Builds a self-contained CLI archive per platform (`t3-<version>-<platform>-<arch>.tar.gz`, `.zip` on Windows) in the same job as that target's desktop artifact and attaches them to the GitHub Release with a `SHA256SUMS` file, on every channel, for five targets: macOS arm64, Linux x64 and arm64, Windows x64 and arm64. Every archive is smoke-tested on hardware of its own architecture, and every one but the Windows archives is also built and signed there. There is no macOS x64 archive: Node single-executables are unsupported on x64 macOS (the SEA docs list macOS as arm64 only) and the binary segfaults on start; the x64 desktop app is Electron and unaffected.
48The archive holds the server as a Node single-executable (`scripts/build-cli-archive.ts`), so unpacking it needs neither Node, npm, nor a compiler. It is the only form in which T3 Code manages a runtime: the desktop's SSH environments, the boot service, `t3 update`, and the install scripts all download and verify this archive against `SHA256SUMS`. The `t3` npm package exists for people who run `npx t3` or `npm install -g t3` themselves, and it downloads this same archive too; nothing in the product installs from npm. The `curl | sh` installers are `scripts/install.sh` and `scripts/install.ps1`; the marketing site copies them into its `public/` at build time (`apps/marketing/scripts/stage-install-scripts.mjs`) and serves them at `t3.codes/install.sh` and `/install.ps1`.
49The root `package.json` pins `engines.node` to the exact Node version used by development, CI and standalone executables. Keep it exact: cross-target builds download that version, and native executable builds reject a different host Node version. TEMPORARY: Linux arm64 executables build only on a Linux arm64 host and are injected with postject, because Node's `--build-sea` output cannot load on 16 or 64 KiB page kernels. Revert when [nodejs/node#67048](https://github.com/nodejs/node/issues/67048) is fixed. Runtime changes need a new T3 release version because installed runtimes are cached by that version.
50macOS archives are signed with the Developer ID certificate and notarized when the Apple secrets are present (ad hoc otherwise, which still runs from `curl`/`tar` installs). Windows executables use the same Azure Trusted Signing setup as the installer. Every native addon in the macOS archive is signed too, since the hardened runtime refuses unsigned libraries.
51Each archive is extracted and executed (`scripts/smoke-cli-archive.ts`) on its build runner, or on the `verify_runner` for Windows.
52After the GitHub Release is published, publishes the `t3` npm package with OIDC trusted publishing from the same workflow file. It is only a launcher (`scripts/npm-launcher/t3.mjs`): on first run it downloads its exact version's CLI archive from the GitHub Release, verifies it against `SHA256SUMS`, and unpacks it into `<T3 home>/runtime/versions/<version>`, the layout the install scripts and `t3 update` share. The executables are not on npm. They used to be, as `@t3code/t3-<platform>-<arch>` optional dependencies, but npm's publish-time scanning held those 80-250 MB packages back for up to an hour after the launcher was live, and every install in that window got a launcher with no executable. Those packages stay on npm at their old versions and get no new ones. Because the launcher reads from the release, `publish_cli` runs after `release`, never before.
53stable releases publish npm dist-tag `latest`
54nightly releases publish npm dist-tag `nightly`
55preview releases publish nothing to npm
56one-time setup: package `t3` needs a trusted publisher registered for this workflow file (see below).
57Builds the hosted web app on Vercel while the desktop jobs run, and makes it live only after a release is published:
58stable releases are aliased to the `latest` hosted app channel
59nightly releases are aliased to the `nightly` hosted app channel
60Signing is optional and auto-detected per platform from secrets.
61
62## Pull request macOS previews
63
64Labeling a PR `preview:mac` publishes a signed, notarized Apple Silicon DMG with T3 Connect enabled
65to the rolling `desktop-preview` prerelease, and works for fork PRs. The label is a one-shot request
66for the commit it is applied to: the trusted workflow removes it once the build is in hand, and later
67pushes do not build until a maintainer applies it again. Every signed preview is therefore a
68per-commit maintainer decision, which matters because the result carries the Developer ID signature.
69Vouching a contributor lets their labeled commits be signed; it is not a standing grant. The build is
70split so the Developer ID certificate never shares a job with PR code:
71
72`.github/workflows/desktop-macos-preview.yml` runs on `pull_request` with no secrets and builds
73 only the JS bundle from the PR (the same `js-bundle` artifact `release.yml` produces).
74`.github/workflows/desktop-macos-preview-publish.yml` runs on `workflow_run` from `main`. It
75 refuses unless the PR is open, still labeled, its head is the built commit, and the author is a
76 bot, a collaborator, or listed in `.github/VOUCHED.td` (read from the default branch, so a PR cannot vouch
77 for itself). It then packages and signs the bundle through `release-desktop.yml` checked out at
78 `main`, so packaging, native helpers, and the Electron/desktop dependencies come from `main`, not
79 the PR. Only the version and the public T3 Connect identifiers in `.env.example` are read from the
80 PR commit, as data, so the signed app's passkey entitlement matches the bundle. A PR that changes
81 packaging must use the `channel=preview` release train above instead.
82
83Before handing the bundle to the signing runner, the trusted workflow validates its ZIP entries
84and accepts only regular files under `server/dist` and `desktop/dist-electron`, plus the directory
85entries that lead to those roots. The artifact cannot
86overwrite packaging code or installed dependencies. The bundle is copied into the app, never executed,
87on the signing runner. The
88`pull_request_target` cleanup job in the publish workflow removes the download when the PR closes, or
89when the label is removed by hand before a build consumed it, and never checks out PR code.
90
91## Required release credentials
92
93Stable releases require these GitHub Actions secrets in addition to the platform and deployment
94credentials documented below:
95
96`RELEASE_APP_ID`
97`RELEASE_APP_PRIVATE_KEY`
98
99The finalize job uses them to commit and push aligned package versions to `main` as the Release App.
100GitHub Release publication uses the repository-scoped workflow token so it has a rate-limit quota
101independent from the shared Release App installation.
102
103## T3 Connect relay deployment
104
105The relay is a shared control plane versioned separately from client releases. Stable and nightly
106client builds must point at the same relay so users see the same linked environments when switching
107release channels.
108
109`.github/workflows/deploy-relay.yml` deploys Alchemy stage `prod` on every push to `main`. Stable
110and nightly releases read the relay URL and Clerk client configuration from the existing
111`production` GitHub Actions environment before building desktop, CLI, or hosted web artifacts. That
112environment admits only `main` and `v*` tags, so preview releases, which may run from any branch,
113take the same public identifiers from `.env.example`; keep it in step with the `production` values.
114Every channel reads the relay client tracing config from Alchemy state with the repository-level
115`CLOUDFLARE_API_TOKEN`, so moving that token into `production` would break preview releases.
116
117Required repository variables shared by relay deployments:
118
119`CLOUDFLARE_ACCOUNT_ID`
120`PLANETSCALE_ORGANIZATION`
121`AXIOM_ORG_ID`
122
123Required repository secrets shared by relay deployments:
124
125`CLOUDFLARE_API_TOKEN`
126`PLANETSCALE_API_TOKEN_ID`
127`PLANETSCALE_API_TOKEN`
128`AXIOM_TOKEN`
129
130Required `production` environment variables:
131
132`RELAY_API_ZONE_NAME`
133`RELAY_TUNNEL_ZONE_NAME`
134`CLERK_PUBLISHABLE_KEY`
135`CLERK_JWT_AUDIENCE`
136`CLERK_JWT_TEMPLATE`
137`CLERK_CLI_OAUTH_CLIENT_ID`
138`APNS_ENVIRONMENT`
139`APNS_TEAM_ID`
140`APNS_KEY_ID`
141`APNS_BUNDLE_ID`
142
143Optional `production` environment variables:
144
145`RELAY_DOMAIN` when overriding the derived `relay.<RELAY_API_ZONE_NAME>` domain
146`RELAY_TUNNEL_CLEANUP_MODE` with `off`, `dry-run`, or `enabled`. Missing and blank values use
147 `off`.
148`RELAY_LEGACY_TUNNEL_CLEANUP_MODE` with the same values, for tunnels whose host never registered
149 recovery. Missing and blank values use `off`.
150
151Required `production` environment secrets:
152
153`CLERK_SECRET_KEY`
154`APNS_PRIVATE_KEY`
155
156Optional `production` environment secrets:
157
158`T3CHAT_LINK_SECRET` for the T3 Chat account link. It must match
159 `T3CODE_LINK_SECRET` on T3 Chat.
160
161After changing a variable or secret, run the **Deploy T3 Connect relay** workflow manually from
162`main` with **force** unchecked. Alchemy compares the values the Worker reads and redeploys it when
163one changed. Check **force** only to redeploy resources with no detected change.
164
165The account-scoped repository credentials are consumed by Alchemy while provisioning relay stages; they
166are not bound into the relay Worker. The production deployment uses an Axiom personal access token,
167so `AXIOM_ORG_ID` must accompany `AXIOM_TOKEN`. The `prod` stage owns the retained PlanetScale
168database. Local personal stages provision isolated branches from it and are never deployed by CI.
169Production adopts the configured relay API and tunnel DNS zones as retained Cloudflare resources.
170Personal stages reference the production-owned zones.
171
172Developers deploy personal stages locally:
173
175vp run --filter t3code-relay deploy -- --stage "$USER" --env-file .env.local
177
178A same-repository PR labeled `relay:preview` deploys stage `pr-<number>` through
179`.github/workflows/relay-preview.yml`, after approval in the `relay-preview` environment. Removing the
180label or closing the PR runs `.github/workflows/relay-preview-cleanup.yml`, which destroys the stage
181and sweeps its managed tunnels and DNS records. A nightly run destroys stages of closed labeled PRs
182whose teardown failed. Fork PRs get no preview.
183
184### Managed tunnel cleanup rollout
185
186Keep `RELAY_TUNNEL_CLEANUP_MODE=off` for the first production deploy. That deploy applies the
187nullable allocation migration and adds the recovery endpoints. Web and mobile clients need no
188coordinated release. CLI and desktop server builds must reach users before cleanup is enabled,
189because those builds register recovery and replace a deleted tunnel after wake.
190
191Deploy the relay and migration with cleanup `off`.
192Release the server build and confirm current hosts register recovery. Older hosts stay marked
193 legacy and are only candidates under the legacy switch below.
194Set `dry-run`, run a relay deploy, and read the sweep counters (`scanned`, `wouldDelete`,
195 `skippedLegacy`, `skippedOrphan`, `failed`, `truncated`) across several sweeps. Each sweep records
196 them, and the active `mode`, as `relay.managed_endpoint_reaper.*` attributes on its
197 `relay.managed_endpoint_reaper.sweep` span in Axiom.
198Run the disposable-host canary below.
199Set `enabled` only after the canary recovers without a server restart.
200
201The job runs every five minutes with a five-minute grace period for tunnels that lost their
202connector, so a candidate is usually removed five to ten minutes after it goes down. Tunnels that
203never connected wait an hour. One sweep attempts at most 100 deletions, so a backlog takes longer.
204Changing `RELAY_TUNNEL_CLEANUP_MODE`, including turning cleanup off during an incident, needs a relay
205deploy without force. Confirm the new `mode` on the next sweep span.
206
207To roll back, set cleanup to `off` and run a relay deploy before downgrading any host. Keep the
208recovery endpoints deployed while current server builds are in use. The nullable columns can stay.
209
210### Legacy tunnel cleanup
211
212A legacy tunnel belongs to a host that never registered recovery, usually one that went offline
213before the recovery build shipped. `RELAY_LEGACY_TUNNEL_CLEANUP_MODE` deletes these once Cloudflare
214reports them down, or never connected, for more than 7 days. It is independent of
215`RELAY_TUNNEL_CLEANUP_MODE`, and every other check still applies.
216
217A deleted legacy tunnel keeps its allocation, so its hostname is kept. When the host comes back:
218
219On a build with recovery, the connector is rejected and the host requests a replacement tunnel at
220 the same hostname.
221On an older build with a CLI link, startup provisions a new tunnel.
222On an older build linked from web or mobile, the host stays offline until T3 Code on that computer
223 is updated.
224
225Ship the web and mobile builds that show the offline reason before enabling legacy cleanup, so a
226user whose host is affected sees what to do. The relay adds the `tunnel_released_at` allocation
227column in its first deploy with this change; the legacy switch stays `off` until you set it.
228
229Run `vp run --filter t3code-relay tunnels:census` with a read-only Cloudflare token. It counts
230 tunnels in every relay stage. The reaper only sees its own stage's tunnels, so clean up the rest
231 by hand.
232Set the legacy mode to `dry-run`, deploy, and read `wouldDeleteLegacy`, `legacyOver30Days`,
233 `totalDown`, and `totalInactive` on the sweep spans for a day. `wouldDeleteLegacy` counts only the
234 tunnels a sweep inspected, at most 500 per status. `totalDown` and `totalInactive` are Cloudflare's
235 counts of this stage's tunnels down for over five minutes and never connected for over an hour.
236 They include ones the reaper skips, so they are an upper bound on the backlog. The share of `wouldDeleteLegacy` in each sweep's `scanned` estimates how
237 much of that total is eligible.
238Run the legacy steps of the disposable-host canary below.
239Before enabling, confirm the web and mobile builds that show the "update T3 Code on that computer"
240 message are live. Without them, a user whose older host lost its tunnel only sees it as offline.
241Set the legacy mode to `enabled`. One sweep deletes at most 100 tunnels, four at a time, and
242 stops starting new deletions after 90 seconds. A backlog of 20,000 takes about 17 hours if each
243 sweep finishes its 100. Watch `deletedLegacy`, `attempted`, `failed`, and `truncated`. An
244 `attempted` well under 100 with `truncated` set means the sweep stopped early: either the time
245 budget ran out or Cloudflare rate-limited a deletion. The counters don't say which; the relay
246 logs a warning with the Cloudflare error for each failed deletion.
247
248In Axiom, filter the relay traces dataset on `name == "relay.managed_endpoint_reaper.sweep"` and
249chart the `attributes.custom.relay.managed_endpoint_reaper.*` fields over time.
250
251Set the legacy mode back to `off` and deploy if any of these happen:
252
253`failed` stays above a few per sweep. Read the warning log for the Cloudflare error.
254Users report an environment that is offline with the update message after they have updated T3
255 Code on that computer and restarted it.
256Relay request errors rise while sweeps run. Deletions share the Postgres connection pool with
257 request handlers.
258
259Turning the legacy mode off stops new legacy deletions; `RELAY_TUNNEL_CLEANUP_MODE` keeps deleting
260tunnels of hosts with recovery while it is `enabled`. Deleted tunnels stay deleted; their hosts
261recover as described above.
262
263### Disposable-host canary
264
265This test has not been run against a real Cloudflare account. Run it against a disposable relay
266stage, test Cloudflare account, disposable host, and disposable T3 home. Keep production cleanup at
267`off` or `dry-run` until it passes. Do not stop a daily-use T3 server.
268
269Deploy the disposable stage with cleanup `dry-run`. Link a first disposable environment through
270 web or mobile settings and confirm its tunnel is healthy and recovery is registered.
271Stop that host and restart the same T3 home on a different local port. Confirm the public
272 hostname reaches the new port and sends nothing to the old one.
273Link a second disposable environment with a server build that predates recovery registration.
274 Capture its managed `cloudflared` child PID, confirm it belongs to that host, and pause only that
275 child with `kill -STOP <legacy-pid>`. Wait until Cloudflare reports it down for over five minutes.
276Capture the first environment's `cloudflared` child PID from its server logs, confirm ownership,
277 and pause it with `kill -STOP <first-pid>`. Wait until Cloudflare reports it down for over five
278 minutes.
279Confirm dry-run counts the first tunnel in `wouldDelete` and the second in `skippedLegacy`.
280Set cleanup `enabled` on the disposable stage and deploy. Confirm in the test Cloudflare account
281 that the first tunnel is deleted and the legacy tunnel still exists.
282Resume the first child with `kill -CONT <first-pid>`. Confirm the running server detects the
283 repeated rejection, requests recovery, and becomes reachable at the same hostname without a
284 restart.
285Resume the legacy child with `kill -CONT <legacy-pid>` and confirm its tunnel reconnects.
286Repeat with a physical sleep and wake cycle on a disposable laptop before broad rollout.
287
288Legacy cleanup, on the same disposable stage:
289
290Set `RELAY_LEGACY_TUNNEL_GRACE_MINUTES=10` and the legacy mode to `dry-run`, then deploy. The
291 override shortens the 7-day grace period and is ignored on `prod`. Pause the legacy child again
292 and wait until Cloudflare reports it down for over ten minutes.
293Confirm the sweep counts it in `wouldDeleteLegacy`, then set the legacy mode to `enabled` and
294 deploy. Confirm the legacy tunnel is deleted and its allocation row remains.
295With the legacy host still on its old build, resume the child. A CLI-linked host provisions a
296 new tunnel on its next restart; a web- or mobile-linked host stays offline.
297Update that host to the current build and start it. Confirm it requests recovery and is
298 reachable at the same hostname.
299Remove `RELAY_LEGACY_TUNNEL_GRACE_MINUTES` from the disposable stage.
300
301## Marketing site deployment
302
303On nightly releases, the release workflow builds the same commit as a staged
304production deployment of the marketing site's Vercel project while the desktop
305jobs run, and promotes it with `vercel promote` after the release is published.
306Stable releases do not deploy the marketing site because they can promote an
307older nightly commit.
308
309The job looks up the `t3code-marketing` project using the existing `VERCEL_TOKEN`
310and `VERCEL_ORG_ID` secrets. It also respects the optional `VERCEL_TEAM_SLUG`
311variable. The Vercel project's root directory must be `apps/marketing`.
312Git deployments remain disabled in `apps/marketing/vercel.ts`.
313
314## Mobile TestFlight and store releases
315
316Merges to main publish compatible OTAs and build iOS prereleases with the `testflight`
317profile at version `3.0.0`. Build numbers increment automatically. The committed app
318version remains the App Store release version; Android keeps using that version.
319
320To release iOS, dispatch **Mobile EAS Production** with `mode=build`,
321`profile=production`, `platform=ios`, and the intended version, such as `2.0.1`.
322The workflow commits and pushes the version before building and uploading it.
323Select that build in App Store Connect for App Store review.
324
325To force a TestFlight rebuild, select `profile=testflight`, `platform=ios`, and
326leave the version blank. Submit the first `3.0.0` build for external beta review
327in App Store Connect. Later builds may still require review; uploads alone do
328not guarantee external approval. TestFlight builds expire after 90 days.
329
330Both release and TestFlight binaries receive compatible production-channel OTAs,
331separated by their major-version fingerprints. Manual `mode=update` uses the
332selected profile's version. Before releasing `3.0.0` to the App Store, advance
333`build.testflight.env.T3CODE_MOBILE_VERSION` in `apps/mobile/eas.json` to the next
334unreleased major and get its first beta build approved.
335
336## Android APK mirror
337
338`.github/workflows/mobile-android-apk.yml` checks the Google Play Alpha closed-testing
339track hourly. Each new beta gets one `android-beta-<build-number>` prerelease
340containing its APK. Unchanged checks publish nothing, and interrupted draft
341uploads resume on the next check.
342
343Normal Nightly releases include the newest processed internal-testing APK available
344when the release publishes. Internal builds do not create separate Android releases.
345If Play credentials or a processed APK are unavailable, or the download takes more
346than five minutes, the Nightly can still publish without an APK. Android submissions
347continue to go to internal testing; promote a build to Alpha to publish it as a beta.
348
349The APK is the universal APK Play generates and signs from the
350uploaded bundle, so no signing key lives in GitHub. The job skips itself until
351`GOOGLE_PLAY_SERVICE_ACCOUNT_JSON` holds a Play service account key with the "Release to
352production, exclude devices, and use Play App Signing" permission, which downloading generated
353APKs requires. The job reads tracks without opening a Play edit, so it can share the service
354account EAS Submit uses.
355
356## Hosted web app release deployment
357
358The hosted app is intentionally not deployed by Vercel's Git integration. The
359web project disables automatic Git deployments in `apps/web/vercel.ts` via
360`git.deploymentEnabled: false`. `.github/workflows/release.yml` builds the web
361app with Vercel CLI as a staged production deployment (`--skip-domain`) while
362the desktop jobs run, and aliases the channel domains to it after the GitHub
363Release succeeds.
364
365Required GitHub Actions secrets:
366
367`VERCEL_TOKEN`
368`VERCEL_ORG_ID`
369`VERCEL_PROJECT_ID`
370
371Optional GitHub Actions variables:
372
373`VERCEL_TEAM_SLUG`: overrides the Vercel CLI scope when the team slug is preferred over the `VERCEL_ORG_ID` secret.
374`T3CODE_WEB_ROUTER_URL`: defaults to `https://app.t3.codes`.
375`T3CODE_WEB_LATEST_DOMAIN`: defaults to `latest.app.t3.codes`.
376`T3CODE_WEB_NIGHTLY_DOMAIN`: defaults to `nightly.app.t3.codes`.
377
378Required Vercel domains:
379
380`app.t3.codes`: the router domain users open, updated by stable releases.
381`latest.app.t3.codes`: channel alias updated by stable releases.
382`nightly.app.t3.codes`: channel alias updated by nightly releases.
383
384The router domain uses `apps/web/vercel.ts` routes. Users opt into a channel by
385visiting `/__t3code/channel?channel=latest` or
386`/__t3code/channel?channel=nightly`; the router stores the
387`t3code_web_channel` cookie and rewrites future requests on `app.t3.codes` to
388the matching channel alias.
389
390The release deploy job rewrites release package versions before upload so the
391hosted app's About panel renders the release version. Stable deploys alias the
392same deployment to both the `latest` channel and the router domain so the router
393rules stay current. Nightly deploys only alias the `nightly` channel. The job
394also passes `VITE_HOSTED_APP_CHANNEL=latest|nightly`, which renders the hosted
395update track selector in the About panel. Changing the selector navigates
396through `/__t3code/channel` on the router domain so the user's channel cookie is
397updated before redirecting to the hosted app root.
398
399One-time Vercel dashboard setup:
400
401Confirm the web project root directory remains `apps/web`.
402Add the three domains above to the web project.
403Disable automatic Git deployments in the dashboard if desired; the committed
404 `vercel.ts` setting is the source-of-truth, but disconnecting Git in the
405 dashboard is also safe.
406Run one stable release deployment, or manually alias the current stable
407 deployment, so `app.t3.codes` points at a deployment containing the router
408 rules in `apps/web/vercel.ts`. Future stable releases keep this alias current.
409
410## Nightly builds
411
412Workflow: `.github/workflows/release.yml`
413Triggers:
414scheduled check every 30 minutes
415manual `workflow_dispatch` with `channel=nightly`
416Automatic nightlies require new commits and at least six hours since the last nightly was published, including manual nightlies.
417To pause automatic nightlies, run `gh variable set PAUSE_NIGHTLY -b true -R pingdotgg/t3code`. Scheduled runs then skip. Stable, preview, and manual nightly runs are unaffected. Resume with `gh variable delete PAUSE_NIGHTLY -R pingdotgg/t3code`.
418Manual nightlies bypass the time and change checks. Nightly runs remain serialized. Scheduled runs wait for an active nightly to finish, then check the publication gap before building.
419Runs the same desktop quality gates and artifact matrix as the tagged release flow.
420Publishes a GitHub prerelease only:
421current tag format: `vX.Y.Z-nightly.YYYYMMDD.<run_number>`
422`nightly-v...` is accepted only as a legacy previous-nightly tag
423release name includes the short commit SHA
424`make_latest` is always `false`
425Uses the next stable patch version as the nightly base. For example, `0.0.17` produces nightlies on `0.0.18-nightly.*`.
426Publishes Electron auto-update metadata to the dedicated `nightly` updater channel, so desktop users can opt into that track independently from stable.
427Publishes the `t3` npm launcher to the `nightly` npm dist-tag using the same nightly version.
428Does not commit version bumps back to `main`.
429
430## Server self-update release invariant
431
432Connected servers update to the client's exact version, not to an npm dist-tag. Servers that update
433themselves download that version's CLI archive from the GitHub Release; servers run through npm are
434handed a `t3@<version>` command to run. Every released client version must therefore have its
435release archives, and then its npm launcher, available before users can receive that client.
436
437The workflow enforces this ordering:
438
439`release` publishes the GitHub Release with the CLI archives and `SHA256SUMS`. Stable and
440 nightly upload to a draft and publish only after all assets are attached, so a new version in the
441 desktop update feed already has its archives. **Update everything** relies on this to update
442 background-service hosts before npm has the version. Rerunning `release` after publication
443 replaces assets in a public release, so service updates can fail until the upload finishes.
444`publish_cli` publishes the `t3@<version>` launcher, which downloads from that release, on
445 stable and nightly. A preview stays a draft and publishes no launcher.
446`deploy_web` depends on both before moving the hosted channel to the new client.
447 `build_web` builds that client earlier with `vercel deploy --prod --skip-domain`, which
448 leaves the custom domains alone but moves the project's own `*.vercel.app` production
449 hostname. That hostname is behind Vercel SSO, so users only get the client through the
450 custom domains.
451
452Desktop auto-update reads the GitHub Release directly, so a desktop client can see the new version
453a few minutes before `t3@<version>` is on npm. Only the copied manual command for npm-run servers
454depends on npm, and it works once `publish_cli` finishes.
455
456Preserve these dependencies when changing the release graph. Publishing a client first would leave
457the **Update server** action targeting a package version that does not exist yet.
458
459For a release smoke test, confirm `npm view t3@<version> version` returns the expected version, then
460connect the new client to a server on the previous version and verify that the update action
461reconnects to the matching server. When the release adds database migrations, verify that the
462remote update applies them and reconnects. A failed trial must restore the database snapshot and
463restart the previous server. If the installed launcher does not support the target protocol,
464verify that the update stops before restart and run `npx t3@<version> service update` once on the
465server machine. Also test the manual or desktop-managed guidance when those environments are
466available.
467
468## Update feed
469
470A nightly install finds the newest nightly in up to three places.
471
472`releases.atom` lists the 10 newest git tags in the repository, of any kind, and a day's
473 `preview` tags can sort above its `nightly` ones. Every install reads it first and takes its first
474 nightly entry. Nightlies before `0.1.10-nightly.20261112.4603`, and stable `0.1.9` and older
475 switched to the nightly channel, read nothing else. Ten other tags in a row hide every nightly
476 from them.
477`releases/download/update-feed/nightly.json` names the newest nightly. Installs that know the
478 pointer read it after the feed and take the newer of the two, so their updates do not depend on
479 tag order. The `update-feed` release holds that file. Never delete it.
480The GitHub REST API, without auth, at 60 requests per hour per IP. Installs from `4603` on scan
481 it when the feed has no nightly. Installs that know the pointer do so only when the pointer also
482 offers nothing new.
483
484The release workflow's `Check the update feed` job runs on every scheduled tick, unless
485`PAUSE_NIGHTLY` is set, and after every nightly. It waits in the same queue as nightlies and
486previews. It rewrites a missing or outdated pointer. Its last step fails when the first nightly
487in the feed is not the newest one. A feed it could not read is only a warning: that says nothing
488about the tags, and the next tick reads it again.
489
490When that step fails, installs that read only the feed cannot reach the newest nightly. If the
491feed still names an older nightly first, installs from `4603` up to the first build with the
492pointer miss it too, because their API scan runs only when the feed has no nightly at all. The job
493log lists the tags in the feed. The usual cause is a preview dispatched from a branch that was not
494rebased onto `main`. Delete the tags nothing needs with `git push origin :refs/tags/<tag>`.
495Converting a release to a draft keeps its tag, so it does not help. To keep this from happening,
496anything users should not be offered stays a draft or uses one rolling tag, like `desktop-preview`.
497
498To pull a bad nightly, delete its release and tag, then run
499`gh release delete-asset update-feed nightly.json`. Installs fall back to the feed until the next
500unpaused tick or manual nightly rewrites the pointer. Without that command, installs that know the
501pointer report an update error until then. If npm's `nightly` dist-tag still names the pulled
502build, also move it to the last good one with `npm dist-tag add t3@<version> nightly`. The channel
503switch in Settings reads that tag and refuses an older desktop build.
504
505## Desktop auto-update notes
506
507Updater runtime: `apps/desktop/src/updates/DesktopUpdates.ts`.
508`electron-updater` adapter: `apps/desktop/src/electron/ElectronUpdater.ts`.
509`apps/desktop/src/main.ts` only wires the updater layers into the desktop runtime.
510Update UX:
511Background checks run on startup delay + interval.
512No automatic download or install.
513The desktop UI shows a rocket update button when an update is available; click once to download, click again after download to restart/install.
514Provider: GitHub Releases (`provider: github`) configured at build time.
515Repository slug source:
516`T3CODE_DESKTOP_UPDATE_REPOSITORY` (format `owner/repo`), if set.
517otherwise `GITHUB_REPOSITORY` from GitHub Actions.
518Required release assets for updater:
519platform installers (`.exe`, `.dmg`, `.AppImage`, `.deb`, plus macOS `.zip` for Squirrel.Mac update payloads)
520channel metadata: `latest*.yml` for stable releases, `nightly*.yml` for nightly releases
521`*.blockmap` files (used for differential downloads)
522macOS metadata note:
523`electron-updater` reads `latest-mac.yml` on stable and `nightly-mac.yml` on nightly, for both Intel and Apple Silicon.
524The workflow merges the per-arch mac manifests into one channel-specific mac manifest before publishing the GitHub Release.
525
526### Windows payload topology and update validation
527
528Windows packages the bundled server and only its runtime-external/native
529dependency closure in `resources/server.asar`. Native modules and helper
530executables declared as unpacked by that archive must be present at the matching
531paths below `resources/server.asar.unpacked`. The Windows-native backend reads
532the archive in place through Electron. Packaged Windows builds also ship
533`resources/wsl-runtime.tar.gz` plus its SHA-256 sidecar: the Linux CLI archive
534(`t3-<version>-linux-<arch>.tar.gz`, the same arch as the Windows host) built
535by the Linux desktop job and handed to the Windows desktop build as
536`--wsl-runtime`, copied in verbatim so WSL runs the exact bytes a Linux user
537downloads. WSL verifies and extracts that archive
538into `~/.t3/wsl-runtime/sha256-<archive-digest>` inside the selected distro,
539then reuses it for later launches of the same update.
540
541Windows keeps JavaScript and package metadata inside `app.asar` and unpacks only
542native libraries and helper executables. Avoid enabling whole-package smart
543unpacking: each loose file adds work to NSIS installation and counts against
544the payload limit.
545
546The artifact builder rejects a Windows package when any of these invariants
547break:
548
549`resources/server.asar` is absent or does not contain the server entry.
550Any file marked unpacked in the ASAR header is absent from
551 `resources/server.asar.unpacked`.
552On same-architecture Windows builds, the packaged primary cannot load the fff
553 native library from inside `server.asar` through its `.unpacked` sibling.
554The isolated, extracted sidecar cannot load the server entry with plain Node.
555A Windows build given `--wsl-runtime` omits the WSL archive or SHA-256
556 sidecar, or the sidecar digest does not match the emitted archive.
557The emitted WSL archive is not a Linux CLI release archive: it must unpack to
558 a single `t3-<version>-linux-<arch>` directory holding `t3`, `client/`, and
559 `node_modules/` with the Linux node-pty binary, and must not carry a loose
560 server bundle (`bin.mjs`).
561The external Windows resource monitor is absent.
562The unpacked Windows application contains more than 80 files.
563
564Cross-architecture Windows builds retain every structural and extracted-sidecar
565check, but skip executing the target Electron binary. A same-architecture build
566for each release target must exercise the primary native-load probe.
567
568NSIS differential packaging remains enabled. A sidecar layout transition can
569produce a larger one-time download; subsequent small releases retain their
570blockmaps, with a 60 MB maximum for a representative sidecar-to-sidecar update.
571
572## 0) npm OIDC trusted publishing setup (CLI)
573
574`build_bundle` builds the version-matched `t3.tgz` with `node scripts/build-npm-launcher.ts` and
575uploads it as the `npm-launcher` artifact. After the GitHub Release is published, `publish_cli`
576checks out only what it needs to set up the project's Node.js, downloads that exact upload by the
577artifact ID `build_bundle` outputs (never by name), checks the tarball version, and runs
578`npm publish --provenance` on it. That is the only package published per release.
579
580Checklist:
581
582Confirm the npm org owns package `t3`.
583For `t3`, configure a Trusted Publisher in the npm package settings:
584Provider: GitHub Actions
585Repository: this repo
586Workflow file: `.github/workflows/release.yml`
587Environment (if used): match your npm trusted publishing config
588Ensure npm account and org policies allow trusted publishing for the package.
589Create release tag `vX.Y.Z` and push; the workflow publishes the GitHub Release and then the
590 launcher with npm dist-tag `latest`.
591Nightly runs publish with npm dist-tag `nightly`; preview runs do not publish.
592
593## 1) Release validation and unsigned builds
594
595There is no dry-run tag path. Pushing any accepted non-nightly tag, including
596`v0.0.0-test.1`, classifies the run as the stable channel. It publishes `t3` with npm dist-tag
597`latest`, creates a real GitHub Release, aliases the hosted app to `latest.app.t3.codes` and
598`app.t3.codes`, and can commit a version bump to `main` in the finalize job. Do not push a test tag
599to validate the workflow.
600
601The workflow has no non-publishing `workflow_dispatch` mode. Use normal CI or local quality gates to
602validate checks and builds without shipping. To exercise the complete release graph at lower stable
603risk, manually dispatch `channel=nightly`; this still publishes a real nightly npm package, GitHub
604prerelease, desktop updater release, hosted nightly alias, and marketing site, but it does not update stable app aliases or
605commit a version bump to `main`. Only run it when a real nightly release is acceptable.
606
607Manual `channel=stable` is also a real stable-channel release. Omitting signing secrets only makes
608platform artifacts unsigned; it does not prevent publication.
609
610## 2) Apple signing + notarization setup (macOS)
611
612Required secrets used by the workflow:
613
614`CSC_LINK`
615`CSC_KEY_PASSWORD`
616`APPLE_API_KEY`
617`APPLE_API_KEY_ID`
618`APPLE_API_ISSUER`
619`MACOS_PROVISIONING_PROFILE` (base64-encoded provisioning profile with Associated Domains)
620
621Required repository variables:
622
623`APPLE_TEAM_ID`
624
625Optional repository variables:
626
627`CLERK_PASSKEY_RP_DOMAINS`: comma-separated RP-domain override. By default, the build derives the
628 domain from the production Clerk publishable key.
629
630Checklist:
631
632Apple Developer account access:
633Team has rights to create Developer ID certificates.
634Create an explicit App ID for `com.t3tools.t3code` and enable Associated Domains.
635Create a `Developer ID Application` certificate and a compatible provisioning profile for that
636 App ID with Associated Domains enabled.
637Export the certificate + private key as `.p12` from Keychain.
638Base64-encode the `.p12` and store as `CSC_LINK`.
639Base64-encode the provisioning profile and store it as `MACOS_PROVISIONING_PROFILE`.
640Store the `.p12` export password as `CSC_KEY_PASSWORD`, and set `APPLE_TEAM_ID` to the
641 10-character Apple Developer Team ID.
642In App Store Connect, create an API key (Team key).
643Add API key values:
644`APPLE_API_KEY`: contents of the downloaded `.p8`
645`APPLE_API_KEY_ID`: Key ID
646`APPLE_API_ISSUER`: Issuer ID
647Complete the Clerk Native API and AASA setup in [T3 Connect setup](./connect-setup.md#desktop-passkeys).
648Re-run a tag release and confirm macOS artifacts are signed/notarized and contain the expected
649 `com.apple.developer.associated-domains` entitlement.
650
651Notes:
652
653`APPLE_API_KEY` is stored as raw key text in secrets.
654The workflow writes it to a temporary `AuthKey_<id>.p8` file at runtime.
655The workflow decodes `MACOS_PROVISIONING_PROFILE`, validates it with `security cms`, and passes it
656 to the desktop packager.
657In-app browser passkeys depend on the same profile. The packager adds each entitlement only when
658 the profile grants it, because macOS will not launch an app that claims more than its profile
659 allows. The build log reports which ones it enabled.
660`keychain-access-groups` (`<TEAM_ID>.com.t3tools.t3code.webauthn`) enables Touch ID passkeys.
661 Profiles that grant the team's keychain groups (`<TEAM_ID>.*`) cover it.
662`com.apple.developer.web-browser.public-key-credential` lets the system passkey sheet (iCloud
663 Keychain, password managers, phones, security keys) serve any site. Apple grants it as a
664 managed capability: the Account Holder requests it through the
665 [macOS Browsers Passkeys form](https://developer.apple.com/contact/request/macos-browsers-passkeys/).
666 After approval, regenerate the profile and update `MACOS_PROVISIONING_PROFILE`.
667
668## 3) Azure Trusted Signing setup (Windows)
669
670Required secrets used by the workflow:
671
672`AZURE_TENANT_ID`
673`AZURE_CLIENT_ID`
674`AZURE_CLIENT_SECRET`
675`AZURE_TRUSTED_SIGNING_ENDPOINT`
676`AZURE_TRUSTED_SIGNING_ACCOUNT_NAME`
677`AZURE_TRUSTED_SIGNING_CERTIFICATE_PROFILE_NAME`
678`AZURE_TRUSTED_SIGNING_PUBLISHER_NAME`
679
680Checklist:
681
682Create Azure Trusted Signing account and certificate profile.
683Record ATS values:
684Endpoint
685Account name
686Certificate profile name
687Publisher name
688Create/choose an Entra app registration (service principal).
689Grant service principal permissions required by Trusted Signing.
690Create a client secret for the service principal.
691Add Azure secrets listed above in GitHub Actions secrets.
692Re-run a tag release and confirm Windows installer is signed.
693
694## 4) Ongoing release checklist
695
696Pick the latest nightly and verify it: run the smoke test above against its artifacts and
697 check the nightly channel for regressions.
698Dispatch the Release workflow with `channel=stable`. Leave `version` empty unless the version
699 should differ from the one the nightly previewed.
700Confirm the `Resolve release commit` notice names the nightly tag and commit you verified. If a
701 newer nightly published in between, the run builds that one instead.
702Verify workflow steps:
703preflight passes
704release quality checks pass
705`build_bundle` and all platform builds pass
706`publish_cli` publishes the exact release version after the release job
707release job uploads expected files
708Smoke test downloaded artifacts.
709
710## 5) Troubleshooting
711
712macOS build unsigned when expected signed:
713Check all Apple secrets plus `APPLE_TEAM_ID` are populated and non-empty.
714Confirm the provisioning profile belongs to `APPLE_TEAM_ID.com.t3tools.t3code` and includes
715 Associated Domains.
716Windows build unsigned when expected signed:
717Check all Azure ATS and auth secrets are populated and non-empty.
718Build fails with signing error:
719Retry with secrets removed to confirm unsigned path still works.
720Re-check certificate/profile names and tenant/client credentials.