Open History to see the last commit
1# alchemy
2
3Alchemy Effect is an Infrastructure-as-Effects (IaE) framework that extends Infrastructure-as-Code (IaC) by combining business logic and infrastructure config into a single, type-safe program expressed as Effects.
4
5It includes a core IaC engine built with Effect. Effect provides the foundation for type-safe, composable, and testable infrastructure programs. It brings errors into the type-system and provides declarative/composable retry logic that ensure proper and reliable handling of failures.
6
7# Concepts
8
9**Cloud Provider** - a cloud provider that offers a set of Services, e.g. AWS, Azure, GCP, Cloudflare, Stripe, Planetscale, Neon, etc.
10**Service** - a collection of Resources, Functions, and Bindings offered by a Cloud Provider.
11**Resource** - a named entity that is configuted with "Input Properties" and produces "Output Attributes". May or may not have Binding Contract.
12**Input Properties** - the properties passed as input to configure a Resource. Otherwise known as the "desired state" of the Resource.
13**Output Attributes** - the attributes produced by a Resource. Otherwise known as the "current state" of the Resource.
14**Stable Properties** - properties that are not affected by an Update, e.g. the ID or ARN of a Resource.
15**Function** (aka. **Runtime**) - a special kind of Resource that includes a runtime implementation expressed as a Function producing an `Effect<A, Err, Req>`. The `Req` type captures runtime dependencies, from which Infrastructure Dependencies are inferred.
16**Resource Provider** (see [Provider](./packages/alchemy/src/Provider.ts))
17
18A Resource Provider implements the following Lifecycle Operations:
19
20**Diff** - compares new props with old props and determines if the Resource needs to be updated or replaced. For updates, it can also specify a list of Stable Properties that will not be changed by the update.
21**Read** - reads the current state of a Resource and returns the current Output Attributes. May return `Unowned(attrs)` to signal an existing-but-foreign resource that the engine should refuse to take over unless `--adopt` is set.
22**Pre-Create** - an optional operation that creates a stub of a Resource before reconcile runs. Used to resolve circular dependencies — e.g. Function A and B depend on each other, so we create a stub of Function A first and then `reconcile` later wires up the real dependency.
23**Observe** - an optional, read-only companion to Pre-Create. A cycle pre-creates on unresolved props, before Read can check ownership, so the engine first asks Observe what exists at the identity Pre-Create would claim. It returns the same shape as Read (`undefined`, attributes, or `Unowned(attrs)`), or `Unresolved` when that identity depends on props that haven't resolved yet; the engine refuses an `Unowned` result unless `--adopt` is set, and checks an `Unresolved` identity once it resolves, before Reconcile. Implement it whenever Pre-Create can write to a resource that already exists.
24**Reconcile** - converges a Resource's actual cloud state to the desired state described by the new Input Properties. Called for both first-time provisioning and subsequent updates. The provider receives `output` (current Attributes) and `olds` (previous Props) which may both be `undefined` on a greenfield create, both defined on an update, or `output !== undefined && olds === undefined` on an adoption. See the **Reconciler doctrine** section below for the required shape.
25**Delete** - deletes an existing Resource. It must be designed as idempotent because it is always possible for state persistence to fail after the delete operation is called. If the resource doesn't exist during deletion, it should not be considered an error.
26**Capability** - a runtime requirement of a Function (e.g. require read access to an `R2Bucket`, or `SQS.SendMessage` on a `SQS.Queue`). A Capability is modeled as one or more `Binding.Service`s. Where the underlying API distinguishes access levels, split it into `Read` / `Write` / `ReadWrite` services (see the **Read/Write/ReadWrite binding convention** below); otherwise a single service suffices. Each service typically ships two interchangeable implementations: a native **binding** (`*Binding`) and a token-scoped **HTTP** client (`*Http`).
27**Binding.Service** - an Effect Service that exposes a `.bind(resource)` method returning a typed runtime client. Its outer (init) Effect resolves the host Function/Worker and its environment, then registers the deploy-time binding — environment variables, IAM policy statements (AWS), or a native Cloudflare binding — by calling ``host.bind`${resource}`(data)``, guarded by `!globalThis.__ALCHEMY_RUNTIME__` so it is a no-op once running inside the deployed Function/Worker. Provided as a Layer on the **Function/Worker** Effect so it gets bundled into the Lambda/Worker. See [Binding](./packages/alchemy/src/Binding.ts).
28**Binding** - data attached to a target Function/Worker via ``host.bind`${resource}`(data)`` from inside a `Binding.Service`. The binding data is collected on the Stack during plan/deploy. Bindings enable circular references between Resources — e.g. a capability binds `{ policyStatements: [...] }` (AWS) or `{ bindings: [...] }` (Cloudflare) onto the host. The Resource Provider then receives the resolved binding data in its `reconcile` lifecycle operation via the `bindings` parameter.
29**Binding Contract** - the shape of data a Resource accepts from Bindings. For example, a Lambda Function accepts `{ env?: Record<string, any>, policyStatements?: PolicyStatement[] }` because it needs environment variables and IAM policies. A Cloudflare Worker accepts `{ bindings: Worker.Binding[] }` for its native binding system. The Binding Contract is declared as the fourth type parameter on the `Resource` interface. See [Lambda Function](./packages/aws/src/Lambda/Function.ts) and [Cloudflare Worker](./packages/cloudflare/src/Workers/Worker.ts).
30**Dependency** - Resources depend on other Resources through two mechanisms:
31Output Properties of one Resource passed as Input Properties to another Resource (non-circular, directed acyclic graph)
32Bindings that attach data (IAM policies, env vars, Cloudflare bindings) from one Resource to another, enabling circular references between Resources.
33**Output** - a reference to (or derived from) a Resource's "Output Attributes". E.g. Bucket.bucketArn
34**Stack** - a collection of Resources, Functions, and Bindings that are deployed together.
35**Stack Name** - the name of a Stack, e.g. `my-stack`
36**Stage** - the stage of a Stack, e.g. `dev`, `prod`, `dev-sam`
37**Stack Instance** - a deployed instance of a Stack+Stage
38**Resource Type** - the type of a Resource, e.g. `Bucket`, `Instance`
39**Physical Name** - a unique name for a Resource, e.g. `my-bucket-1234567890`. It is usually best to generate them using the built-in createPhysicalName utility function which generates
40**Logical ID** - the logical ID identifying a resource within a Stack, e.g. `my-bucket`. It is stable across creates, updates, deletes and replaces.
41**Instance ID** - a unique identifier for an instance of a Resource. It is stable across creates, updates and deletes. It changes when a resource is replaced. It is truncated and used as the suffix of the Physical Name.
42**Event Source** - a special kind of Binding between a Function and a Resource that produces events that invoke the Function, e.g. `SQS.QueueEventSource`. Implemented as a `Binding.Service` whose init Effect both registers the runtime event listener on the host and, at deploy time, yields the event-source mapping resource (or calls the cloud provider API to create/update it).
43**Replacement** - the process of replacing a Resource with a new one. A new one is created, downstream dependencies are updated with the new reference, and then the old one is deleted. Or, the old one is deleted first and then the new one is created.
44**Dependency Violation** - an error that some APIs call when an operation cannot be performed because a dependency is not met. E.g. you cannot delete an EIP until the NAT Gateway it is attached to is deleted. Lifecycle operations typically retry Dependency Violations.
45**Eventual Consistency** - create/update/delete operations can be eventually consistent leading to a variety of failure modes. For example, a Resource may be created but not yet available for use, or a Resource may be deleted but still appear in the console. Errors caused by eventual consistency should be retried, and lifecycle operations/tests should be carefully designed to wait for consistency before proceeding.
46**Retryable Error** - an error that can be retried. E.g. a Dependency Violation, Eventual Consistency Error, Transient Failure, etc.
47**Non-Retryable Error** - an error that cannot be retried. E.g. a Validation Error, Authorization Error, etc.
48**Retry Policy** - a policy for retrying errors. E.g. a fixed delay, exponential backoff, max retries, while some condition is true, or until some condition is true/false, etc.
49
50# File System Conventions
51
52First-class sibling repositories we maintain live in `submodules/`:
53
54`submodules/distilled` — generated Effect SDKs (workspace packages). Initialized by `git submodule update --init`. In a linked Alchemy worktree, the `post-checkout` hook (`scripts/bootstrap-distilled.mjs`) adds it as a worktree of the primary checkout's distilled and clones unchanged files from there, so on APFS, btrfs or XFS it shares their disk blocks (about 1.1 GB saved per worktree). `ALCHEMY_DISTILLED_CLONE=0` writes a full copy instead.
55`submodules/floci` — our fork of the local AWS emulator. Skipped by default; fetch with `git submodule update --init --checkout -- submodules/floci`.
56
57Each Service's Resources follow the same pattern. Resource contract and provider are co-located in the same file. Each Capability lives in its own file(s) named after the capability and access level (`Binding.Service` contract + the `*Binding` / `*Http` implementations).
58
60# source files
61packages/{cloud}/src/{Service}/index.ts # re-exports resources, capability contracts, and impl layers
62packages/{cloud}/src/{Service}/{Resource}.ts # resource contract + resource provider
63packages/{cloud}/src/{Service}/{Capability}.ts # Binding.Service contract + runtime client interface
64# test files
65packages/integration-tests/test/{Cloud}/{Service}/{Resource}.test.ts
66# docs (auto-generated from source-code JSDoc - DO NOT manually edit)
67website/src/content/docs/providers/{Cloud}/{Resource}.md # API reference, generated by `pnpm docs:gen`
69
70A capability that exposes distinct access levels is split into a contract per level plus interchangeable native-binding and HTTP implementations, with shared scaffolding kept in *un-exported* helper files (see the **Read/Write/ReadWrite binding convention** below). For example, the Cloudflare R2 bucket capability:
71
73packages/cloudflare/src/R2/BucketTypes.ts # shared types + error (exported)
74packages/cloudflare/src/R2/BucketRead.ts # BucketRead Binding.Service + ReadBucketClient (exported)
75packages/cloudflare/src/R2/BucketWrite.ts # BucketWrite Binding.Service + WriteBucketClient (exported)
76packages/cloudflare/src/R2/BucketReadWrite.ts # BucketReadWrite Binding.Service + ReadWriteBucketClient (exported)
77packages/cloudflare/src/R2/BucketBinding.ts # shared worker-binding scaffolding (NOT exported from index)
78packages/cloudflare/src/R2/BucketReadBinding.ts # ReadBucketBinding layer + makeRead (exported)
79packages/cloudflare/src/R2/BucketWriteBinding.ts # WriteBucketBinding layer + makeWrite (exported)
80packages/cloudflare/src/R2/BucketReadWriteBinding.ts # ReadWriteBucketBinding layer (exported)
81packages/cloudflare/src/R2/BucketHttp.ts # shared HTTP/token scaffolding (NOT exported from index)
82packages/cloudflare/src/R2/BucketReadHttp.ts # ReadBucketHttp layer (exported)
83packages/cloudflare/src/R2/BucketWriteHttp.ts # WriteBucketHttp layer (exported)
84packages/cloudflare/src/R2/BucketReadWriteHttp.ts # ReadWriteBucketHttp layer (exported)
86
87Examples of actual paths:
88
90packages/aws/src/S3/Bucket.ts # S3 Bucket resource + provider
91packages/aws/src/S3/GetObject.ts # S3 GetObject capability (Binding.Service)
92packages/aws/src/S3/PutObject.ts # S3 PutObject capability (Binding.Service)
93packages/aws/src/SQS/Queue.ts # SQS Queue resource + provider
94packages/aws/src/SQS/SendMessage.ts # SQS SendMessage capability
95packages/aws/src/Kinesis/Stream.ts # Kinesis Stream resource + provider
96packages/aws/src/Kinesis/PutRecord.ts # Kinesis PutRecord capability
97packages/aws/src/Lambda/Function.ts # Lambda Function resource + provider
98packages/aws/src/DynamoDB/Table.ts # DynamoDB Table resource + provider
99packages/aws/src/DynamoDB/GetItem.ts # DynamoDB GetItem capability
100packages/aws/src/EC2/Vpc.ts # VPC resource + provider
101packages/aws/src/EC2/Subnet.ts # Subnet resource + provider
103
104## Package boundaries
105
106`packages/alchemy` is the provider-free core (engine, CLI, state, bundling, runtime hosts). Every provider is its own package under `packages/` (`@alchemy.run/aws`, `@alchemy.run/cloudflare`, `@alchemy.run/gcp`, …), as are shared building blocks (`@alchemy.run/docker`, `@alchemy.run/kubernetes`, `@alchemy.run/redis`, `@alchemy.run/sql`, `@alchemy.run/domain`). Core never imports a provider package; a provider reaches core only through `alchemy/*` specifiers and declares its CLI, auth and state stores through its `Contributions` module.
107
108A provider package's `exports` are generated from `src/`: its root, one entry per directory `index.ts` (`@alchemy.run/aws/S3`), and `./*` for every file (`@alchemy.run/aws/S3/Bucket`). Run `pnpm exports:sync` after adding or removing a directory; CI runs `pnpm exports:check`. Name a module nobody outside the package should import `internal.ts`, or put it under an `internal/` directory: published packages block both, and package source must not import another package's `internal/`.
109
110A package that uses another declares it in `package.json`: a `dependency` when its own providers always need it (AWS and GCP depend on `@alchemy.run/kubernetes` for their cluster adapters), an optional `peerDependency` (plus a `devDependency`) for an opt-in integration (`@alchemy.run/drizzle` on `@alchemy.run/cloudflare`). A package's own `declare global` block hides alchemy's globals under TypeScript 7, so build-time globals live in core's `Phase.ts`, and each package's `src/globals.d.ts` imports them.
111
112Provider suites, and core suites that deploy to a provider, live in `packages/integration-tests` and run with `pnpm test:integration`; `packages/alchemy/test` stays provider-free.
113
114# The Resource Factory Process
115
116Alchemy resource coverage is produced as a **software factory**: fleets of agents implement and live-test IaC resources in waves, and every API mismatch the tests surface is fed back as a patch to distilled. The factory was used to take Cloudflare from 36 to 239 cataloged resources (250 test files / 600+ test cases, 1000+ patched operations) and is the template for every future provider.
117
118## The flywheel
119
122 ^ │
123 │ v
124 update statuses <── regenerate service <── patch distilled
126
127**Catalog** — fan out research agents over the provider's distilled service modules (one batch per thematic group). Each agent reads the generated SDK (`submodules/distilled/packages/{cloud}/src/services/{service}.ts`), cross-references the vendor API docs, and writes a self-contained design spec to `processes/{Cloud}/catalog/{service}.md`: resources, namespaces, props/attrs with replacement rules, lifecycle-to-operation mapping, scope (account/zone), testability, priority. The coordinator aggregates a machine-readable `summary.json` + human `INDEX.md` that tracks `implemented | partial | missing` per resource — this is the factory's order book.
128**Implement + test** in waves (below). Tests run against the real cloud (`pnpm test --profile testing`); zone-scoped tests use the standing test zone (`alchemy-test-2.us` via `findZoneByName`).
129**Patch the SDK, never the consumer** — every `UnknownCloudflareError`, out-of-union status error, or wrong request/response schema found by a test becomes an RFC 6902 JSON Patch against the service's Smithy model, under `submodules/distilled/packages/{cloud}/patches/{service}/{op}.json` (see the Typed Error Doctrine section). Regenerate only that service. The typed union improves for every future consumer of the SDK — that is the flywheel's output.
130**Update the catalog** statuses after each wave and pick the next batch from the order book. Repeat until everything left is documented as out of scope (deprecated APIs, billing/data-only endpoints, closed-beta, needs-external-systems).
131
132## Orchestration rules (the coordinator)
133
134**One workflow at a time**, ~12 concurrent agents (cap ≈ CPU cores − 2). Two parallel workflows double throughput but also double crash blast-radius — only do it when the machine and budget clearly allow.
135**One agent per distilled service.** Service ownership is the unit of isolation: only the owner may touch `patches/{service}/` and regenerate `src/services/{service}.ts`, so generator runs never race. An agent may own several resources of its service; a very large same-service backlog (e.g. zero-trust) runs as a **sequential chain** of agents inside the workflow, in parallel with all other services.
136**Shared-file discipline.** `Providers.ts` and the provider barrel `index.ts` are edited by every agent: single minimal insertions only, re-read and retry on edit conflict, never rewrite wholesale.
137**`Layer.mergeAll` ceiling.** Keep `Providers.ts`'s provider layers in *nested* `Layer.mergeAll` groups (~90 entries each). A flat ~200-argument call exceeds tsc's variadic inference and **silently drops the tail layers** from the inferred union, producing baffling `Provider<X> is not assignable to StackServices` cascades across every test file.
138**Crash resilience.** Word every task as *assess-first*: "partial work may exist from an interrupted run — list your dirs, read existing files, check registration, FINISH rather than rewrite." Completed agent results are banked in the workflow journal even if the workflow dies; the coordinator recovers them from `journal.jsonl` and re-dispatches only the lost tasks.
139**The coordinator (not agents) does**: the authoritative type-check, distilled lib rebuilds, combined verification runs, catalog/index updates, cross-cutting fixes (shared-file restructures, Effect-version API migrations), and deterministic mass transforms (mechanical codemods are scripted centrally, not fanned out).
140**Monitor for stalls**: an agent transcript that hasn't been written for ~5 minutes with no child test process is stalled — kill and re-dispatch; don't wait.
141
142## Resource budget: one type-checker for the whole factory
143
144`tsrs -b` over the workspace is expensive; dozens of agents running it concurrently thrashes the machine (and concurrent `tsbuildinfo` writes race). Instead:
145
146**Agents are banned** from running `tsrs`/`tsc` or `pnpm build` (root or distilled) in any form. The coordinator owns type-checking and runs a one-shot `pnpm exec tsrs -b` at wave boundaries.
147**The test runner resolves distilled from `src/*.ts` directly, NOT the built `lib/`** (`alchemy-test` runs in plain bun, which resolves the `bun` export condition natively). So a regenerated service is **immediately test-visible** the moment `bun scripts/generate.ts --resource {service}` (+ oxfmt) finishes — there is nothing to rebuild and **nothing to wait for**. Do NOT sleep and do NOT gate a test re-run on a build after regenerating. This applies to response-schema patches as well as error-tag-only patches.
148
149## Speed doctrine: never wait on a hang
150
151Run tests with `pnpm test` (works from the repo root or `packages/alchemy`; suite paths are relative to `packages/alchemy`, e.g. `test/Cloudflare/...`). The runner enforces per-test timeouts and prints the currently-running tests whenever nothing finishes for 10s — use that list to identify a hang, read the run's log under `.alchemy/log/test/`, find the cause (unbounded retry, infinite pagination, the engine deadlock below), and fix the root cause. Never just re-run hoping.
152Per-test timeout ≤ 90–120s (`{ timeout: ... }` on the test, or `--timeout` for the whole run). A suite needing more than ~3–5 minutes total is a bug.
153Every `Effect.retry`/`Effect.repeat` is bounded: `times ≤ 8–10`, total backoff under ~45–60s. Never poll for asynchronous provisioning slower than ~90s — skipIf-gate instead.
154**Known engine bug**: a deploy that *replaces* a resource while simultaneously *removing* its old dependency deadlocks. Keep both dependencies deployed across replacement steps in tests (see `test/Cloudflare/R2/BucketEventNotification.test.ts`).
155**Three-iteration budget**: if a suite is not green after ~3 fix iterations and the blocker is platform behavior (entitlement, slow async provisioning, beta API), implement fully, skipIf-gate with the typed tag and exact error, verify a skip-clean run, and report honestly. Do not burn an hour on one resource.
156
157## Entitlement gating pattern
158
159Most enterprise/plan-gated resources are still fully implementable; only the live lifecycle is gated:
160
161**Probe once** against the real API to capture the exact rejection (code + message). If it surfaces as an untyped catch-all, patch distilled first so the entitlement error is a typed tag (e.g. `MagicTransitNotOnboarded` code 1012, `SaasQuotaNotAllocated` code 1404, `AdvancedCertificateManagerRequired` code 1450).
162Keep an **ungated probe test** that asserts the typed tag is returned — this proves both the patch and the gating are correct, forever, at near-zero cost.
163skipIf-gate the full lifecycle behind an env var (`CLOUDFLARE_TEST_MAGIC_TRANSIT=1`, `CLOUDFLARE_TEST_DLP=1`, …) so an entitled account can run it unchanged.
164Record the testability verdict (`yes | limited | no`) and the exact error in the catalog notes.
165
166Deprecated APIs (superseded by Rulesets etc.), billing/subscription objects, pure data-APIs, and closed-beta endpoints are *documented as out of scope* in `INDEX.md` rather than implemented.
167
168## What every agent task prompt must include
169
170A wave task prompt is a contract. Include, every time:
171
172The **distilled service the agent owns** and the resources to build (with namespace + directory).
173**Assess-first** instruction (finish partial work, don't rewrite).
174The reading list: this file's Reconciler doctrine + Typed Error Doctrine, the catalog spec, the distilled service module + existing patches, current exemplar resources/tests (account-level CRUD, zone singleton capture-and-restore, observe-before-delete), and the registration files.
175The **type-check/build ban** (above) — agents never run `tsc`/`pnpm build`; the coordinator owns type-checking.
176The **speed doctrine** (above) verbatim — agents rediscover unbounded waits otherwise.
177The **Typed Error Doctrine** hard rule with the patch-regenerate command for *their* service only.
178Registration discipline for the shared files, including the nested-mergeAll note.
179Test requirements: `test.provider`, start **and** end with `stack.destroy()`, deterministic names (engine default or constant), out-of-band verification via distilled, typed wait-until-gone, replacement coverage where applicable.
180Known footguns: `diff` receives `Input<Props>` — narrow with `isResolved(news)` before property access; never `Input<T>` in declared Props; Effect 4 APIs (`Effect.result` + `Result.isSuccess/isFailure`, not `Effect.either`/`effect/Either`); JSON Patches address the Smithy model, so shape IDs are `com.cloudflare.{service}#Name` and member names are wire names (snake_case) — the camelCase TS surface is derived at codegen; fixtures (CSRs, PEMs, JWKS) are generated once and checked in as constants, never at test time.
181A **structured result schema**: `{ service, resources, testsPassed, testCommand, files, patches (with reasons), skippedTests (with exact errors), notes }` — the coordinator aggregates these into the catalog.
182
183# Documentation Generation
184
185**Source of truth:** The source code is the single source of truth for all API documentation. JSDoc comments in `packages/*/src/**/*.ts` are extracted and used to generate the public API reference markdown.
186
187:::warning
188**Never edit the generated markdown files** under `website/src/content/docs/providers/{Cloud}/`. They are overwritten on every regeneration.
189
190To "update the docs", edit the JSDoc on the source `.ts` file (resource-level JSDoc on the exported `const`, plus field-level JSDoc on each prop/attribute) and re-run the generator. There is no separate doc file to update.
191:::
192
193**How to generate docs:**
194
196pnpm docs:gen # -> website/src/content/docs/providers/{Cloud}/{Resource}.md
198
199This is the only doc generator that produces user-facing output. ([scripts/generate-api-reference.ts](./scripts/generate-api-reference.ts)) does the following:
200
201Discovers documented files across its configured source roots — `packages/{cloud}/src/{Service}/` plus flat single-provider packages like `packages/better-auth/src/` (mapped onto a synthetic provider directory, e.g. `BetterAuth/`)
202Parses TypeScript with the TypeScript 7 API shipped by `tsrs` (`tsrs/unstable/*`)
203Extracts the page-level summary plus Markdown section/example blocks from JSDoc on the export tagged `@resource`, `@binding`, `@layer`, or `@primitive` (a building block exported as its module's `make`, e.g. `Domain.Event`; its page is named after the file)
204Writes one markdown file per page at `website/src/content/docs/providers/{Provider}/{Name}.md`
205
206**Layer pages** (`@layer`) document exported Layer factories — the pluggable implementations of a Context service (e.g. better-auth's database layers). Alongside the shared tags, a Layer declares what it satisfies and needs:
207
209/**
210 * Neon database layer for Better Auth over Neon's serverless driver.
211 *
212 * ### Connecting from a Worker or Lambda
213 * **Example:** Worker or Lambda with Neon-backed Better Auth
214 * ```typescript
215 * // ...
216 * ```
217 *
218 * @layer
219 * @provides BetterAuth.Database
220 * @peer @neondatabase/serverless
221 * @product Neon
222 */
223export const Neon = (url: ConnectionSource, options?: NeonOptions): Layer.Layer<Database> => ...
225
226`@provides <Service.Tag>` — the Context service tag(s) the Layer satisfies (repeatable)
227`@peer <package>` — optional peer dependencies the Layer needs at runtime (repeatable)
228
229Both render as a metadata line under the page's `Source:` blockquote. Every Layer implementation should carry these annotations — a Layer without them is an undocumented integration surface.
230
231Keep all API-generator metadata tags (`@resource`, `@binding`, `@layer`,
232`@primitive`, `@product`, `@category`, `@title`, `@provides`, and `@peer`) together at the very end of
233the JSDoc block. Do not put prose, sections, or examples after them: TypeScript
234treats subsequent content as part of the block tag, which breaks editor JSDoc
235rendering. The generator also supports `@group` as an alias for `@category` and
236`@label` as an alias for `@product`. `@title <Heading>` overrides a section's
237heading, which is otherwise the export name, for when the name alone is ambiguous
238on its page (`SQL.D1` documents as `SQL` on the D1 page).
239
240After editing JSDoc on a resource, run `pnpm docs:gen` to refresh the website
241docs. Run `pnpm docs:check-jsdoc` to validate the source layout, or
242`pnpm docs:fix-jsdoc` to normalize it automatically.
243
244**Writing good documentation:** When adding or updating a resource, ensure all Props and Attrs have JSDoc comments:
245
247export interface BucketProps {
248 /**
249 * Name of the bucket. If omitted, a unique name will be generated.
250 * Must be lowercase and between 3-63 characters.
251 */
252 bucketName?: string;
253
254 /**
255 * Whether to delete all objects when the bucket is destroyed.
256 * @default false
257 */
258 forceDestroy?: boolean;
259}
261
262The `@default` tag is used to document default values and will appear in the generated documentation.
263
264### Examples and Sections (IMPORTANT)
265
266**Examples are critical for documentation.** Every resource should have examples demonstrating common use cases. Use Markdown headings and labels on the main Resource export to organize examples into a navigable table of contents.
267
268**Format:**
269
270`### <Section Title>` - Creates a heading in the Examples section and adds an entry to the Quick Reference table of contents
271`**Example:** <Example Title>` - Creates a labeled code example (must follow a section heading)
272A level-three heading that is ordinary page prose rather than an example section must end with `<!-- api-prose -->`; the generator removes the marker and preserves it as `###` in the output
273Code blocks inside examples use standard markdown fenced code blocks (` `)
274Put the API-generator metadata tags after all sections and examples
275
276**Example:**
277
279/**
280 * An S3 bucket for storing objects.
281 *
282 * ### Creating a Bucket
283 * **Example:** Basic Bucket
284 * ```typescript
285 * const bucket = yield* Bucket("my-bucket", {});
286 * ```
287 *
288 * **Example:** Bucket with Force Destroy
289 * ```typescript
290 * const bucket = yield* Bucket("my-bucket", {
291 * forceDestroy: true,
292 * });
293 * ```
294 *
295 * ### Reading Objects
296 * **Example:** Get Object from Bucket
297 * ```typescript
298 * const response = yield* getObject(bucket, { key: "my-key" });
299 * const body = yield* Effect.tryPromise(() => response.Body?.transformToString());
300 * ```
301 *
302 * ### Writing Objects
303 * **Example:** Put Object to Bucket
304 * ```typescript
305 * yield* putObject(bucket, {
306 * key: "hello.txt",
307 * body: "Hello, World!",
308 * contentType: "text/plain",
309 * });
310 * ```
311 *
312 * @resource
313 */
314export const Bucket = Resource<...>("AWS.S3.Bucket");
316
317This generates:
318
319A "Quick Reference" section with links to each `###` section
320An "Examples" section with organized code examples under each section heading
321
322**Best practices for examples:**
323
324Start with the simplest use case and progress to more complex ones
325Include examples for all major capabilities (GetObject, PutObject, etc.)
326Show real-world patterns like error handling, combining with other resources
327Use descriptive titles that explain what the example demonstrates
328
329# Workflow (per-service inner loop)
330
331This is the inner loop that each factory agent (see **The Resource Factory Process** above) executes for the service it owns — it applies equally when a single engineer brings up one service by hand.
332
333Development of Alchemy-Effect Resources is heavily pattern based. Each Service has many Resources that each have 0 or more Capabilities and Event Sources. When working on a new Service, the following steps should be followed.
334
335Research the AWS Service and identify its Resources, Identifier Types, Structs, Capabilities, and Event Sources. Refer to the corresponding Terraform Provider, Pulumi Provider, and CloudFormation docs for that service (use the provided tools specifically for searching these docs for services and resources).
336
337Example (abbreviated):
338
339Service: S3
340
341Resources:
342
343Bucket
344BucketPolicy
345etc.
346
347Bucket Capabilities:
348
349GetObject
350PutObject
351DeleteObject
352
353Identifier Types:
354
355Bucket Name
356Bucket ARN
357
358Structs:
359
360CorsRule
361LifecycleConfiguration
362
363Document each of the Resource interfaces
364
365Include the following information:
366
367ResourceName, e.g. Bucket, Instance, Queue
368Input Properties (for each property: Name, Type, Description, Default Value, Required, Constraints, Replaces: true/false)
369Output Attributes (for each attribute: Name, Type, Description)
370
371Document each of the Capabilities and Bindings
372
373Include the following information:
374
375Capability Name, e.g. `GetObject`, `PutObject` (it maps 1:1 with an AWS API)
376Constraints (e.g. `Key`)
377IAM Policies (how the capability maps to an IAM Policy, e.g. Effect: Allow, Action: s3:GetObject, Resource: `arn:aws:s3:::${bucketName}/${Key}`)
378Environment Variables (what environment variables should be added to a Lambda Function so that it can access the capability, e.g. `BUCKET_NAME`, `BUCKET_ARN`, `QUEUE_URL`, `QUEUE_ARN`, etc.)
379
380Research and design each of the Lifecycle Operations
381
382**Diff** - identify which properties are always stable across any update, which properties change conditionally depending on new and old values, which properties trigger a replacement. This is usually just a distinct list, but can sometimes require if-this-then-that logic. Document it explicitly and exhaustively. Cross-reference with AWS CloudFormation, Terraform Provider and Pulumi Provider docs.
383
384:::warning
385You should almost never use `no-op` in the Diff. No-op should be explicitly designed as a way to say "i know this property changed, but i don't want it to trigger an update". This is an edge-case and not the norm. Usually you want diff to return `undefined` or `void` to let the engine apply the default update logic. Diff is usually just use as an optimization or to identify replacement instead of update.
386:::
387
388**Read** - determine which API calls are required to read the Output Attributes of a Resource from the Cloud Provider state (otherwise known as refresh or synchronize resource state). This is usually a single Get{Resource} API call, but can be a complex set of calls depending on the Service. Read can also be called without the current Output Attributes because of past state persistence failures. These cases are handled by computing the deterministic Physical Name and looking it up or by searching for Resources using tags (if the Cloud Provider supports it). Read may return `Unowned(attrs)` when the resource exists but lacks our ownership tags, signalling the engine to gate adoption behind `--adopt` or `adopt(true)`.
389**Pre-Create** - determine if the Resource needs a pre-create operation. This is usually only the case for the special Function/Runtime Resources like AWS Lambda Functions. If it is required, then document which API call(s) should be called and what the empty (unit) input properties are. E.g. a Lambda Function takes a simple script that exports a no-op handler function.
390**Reconcile** - determine the API calls needed to converge the cloud's actual state to the desired state described by the new Input Properties. Reconcile must be a single flow that works whether the resource is missing (greenfield create), pre-existing under our ownership (update), or freshly adopted (`output` defined but `olds` absent). See the **Reconciler doctrine** section below.
391**Delete** - determine which APIs should be called and in what order to delete an existing Resource. Delete should be idempotent so that if the resource has already been deleted, it is not considered an error. It is common for deletions to fail because of Dependency Violations or Eventual Consistency Errors. These are not always called Dependency Violations in the API docs, so attention should be paid to investigating each API's possible error codes and how they should be handled by the Delete operation. Should we retry for a period of time, indefinitely, or fail immediately?
392
393# Reconciler doctrine
394
395The provider's `reconcile` function replaces the legacy `create` + `update` pair. It runs every time the engine wants to make the cloud match the desired state — whether that's the first time the resource is being provisioned, a routine update, or a takeover after `read` returned an existing cloud resource.
396
397It receives `output: Attributes | undefined` and `olds: Props | undefined`:
398
399| `output` | `olds` | Meaning |
400| ------------ | ------------ | ------------------------------------------------- |
401| `undefined` | `undefined` | Greenfield — no prior physical resource |
402| defined | defined | Routine update — engine-owned resource |
403| defined | `undefined` | Adoption — engine adopted via `read` |
404
405A reconciler MUST work correctly for all three combinations. It MUST NOT branch the body on `output === undefined` and run different code paths for "create" vs "update". That pattern is just rename-and-branch and re-introduces every assumption the old `create`/`update` split made. Instead, write one flow:
406
4092. Ensure — if the resource is missing, call createX. Catch AlreadyExists/ConflictException
410 as a race and continue. Wait for active state if applicable.
4113. Sync — for each mutable aspect (settings, sub-resources, tags, policy):
412 - read OBSERVED cloud state (not olds)
413 - compute desired state from news + bindings
414 - diff observed against desired
415 - apply only the delta API call (skip the API entirely on no-op)
4164. Return — re-read final state if needed; return the fresh Attributes shape
418
419Key invariants:
420
421**Observation > assumption.** Cloud state is authoritative. `olds` is at most a hint to skip a no-op API call; it is never the source of truth for what's actually deployed.
422**Each sync step is independently idempotent.** Crash mid-reconcile, re-run, you converge.
423**`output` is treated as a cache** for stable identifiers (physical name, ARN, immutable id). It is NOT a guarantee that the resource still exists. If it doesn't, observation falls through to "missing" and ensure recreates.
424**`AlreadyExists`/`NotFoundException`/`ResourceInUseException`-style errors are caught**, not propagated — they're races or eventual-consistency, not failures.
425**Tags use observed cloud tags as the diff baseline**, not `olds.tags` or `output.tags`. Adoption may bring you a resource with foreign tags that need to be reconciled.
426
427:::warning
428**Do not write `if (output === undefined) { /* create body */ } else { /* update body */ }`.** That is rename-and-branch, not reconciliation. The reconciler's body is one observe-ensure-sync flow that produces correct cloud state regardless of starting point.
429:::
430
431The canonical reference reconcilers cover the common shapes:
432
433[S3 Bucket](./packages/aws/src/S3/Bucket.ts) — uses `ensureBucketExists` + `syncBucketTags` + `syncBucketPolicy` helpers; each helper is itself a tiny reconciler.
434[SQS Queue](./packages/aws/src/SQS/Queue.ts) — observe via `getQueueUrl`, ensure via `createQueue` (tolerates `QueueNameExists` race), sync attributes by diffing `getQueueAttributes` against desired, sync tags.
435[Kinesis Stream](./packages/aws/src/Kinesis/Stream.ts) — many mutable aspects (mode, shards, retention, encryption, metrics), each its own observed-vs-desired sync block.
436[DynamoDB Table](./packages/aws/src/DynamoDB/Table.ts) — multi-API observation (table + tags + PITR + TTL), per-aspect diffing, GSI delta application.
437[EC2 Vpc](./packages/aws/src/EC2/Vpc.ts) — auto-assigned id, observe via `describeVpcs([output.vpcId])` with NotFound fallback to create, sync DNS attrs by reading `describeVpcAttribute`, sync tags from observed `vpc.Tags`.
438[Lambda Function](./packages/aws/src/Lambda/Function.ts) — uses `createOrUpdateFunction` / `createOrUpdateFunctionUrl` / `attachBindings` helpers, each idempotent.
439[Cloudflare Worker](./packages/cloudflare/src/Workers/Worker.ts) — non-AWS API; the underlying `putWorker` is a true upsert, so reconcile observes existing settings and delegates.
440
441Existence-only resources (Lambda Permission, EC2 Route, EC2 RouteTableAssociation, IAM AccessKey, etc.) have nothing mutable beyond their identity. Their reconciler is just observe → if missing, create. There is no sync step.
442
443# SQL database migrations
444
445Every SQL database resource applies schema migrations at deploy time through a `migrations` prop. A database resource without it is a gap to fill. Users get the same DX on every cloud:
446
448const db = yield* GCP.SQL.Database("App", {
449 instance: instance.instanceName,
450 migrations: "./migrations", // a directory, { dir, table }, or a Drizzle.Schema
451 migrationsUser: user.userName,
452 migrationsPassword: password.text,
453});
455
456The shared engine lives in `@alchemy.run/sql/Migrations`. It reads the directory (flat `.sql` files, drizzle-kit, or Prisma layouts) and keeps history in Alchemy's `__alchemy_migrations` table. It converts a foreign drizzle-kit/Prisma/wrangler table once, and fails with `RewrittenMigrationHistoryError` when an applied file is edited or removed. The provider never reimplements any of this. It supplies one thing, a `SqlExecutor` for its database:
457
459// Props
460migrations?: MigrationsInput;
461// Attributes
462migrationsDir: string | undefined;
463migrationsTable: string | undefined;
464migrationsHashes: Record<string, string>;
465
466// diff: pending files or a table move update in place, never replace
467if (yield* diffMigrations({ news, output })) return { action: "update" } as const;
468
469// reconcile: last, once the database is ready and every other aspect is synced
470const migrationsInput = migrationsInputOf(news);
471const run = migrationsInput
472 ? yield* runMigrations({
473 input: migrationsInput,
474 stamped: stampedOf(output),
475 withExecutor: (apply) => withExecutor(apply), // acquire a SqlExecutor, apply, release
476 })
477 : undefined;
478return { ...attrs, ...migrationsAttrs({ input: migrationsInput, run, output }) };
480
481Rules:
482
483**Where the prop goes.** Put it on the resource that owns one database's schema: the database (`GCP.SQL.Database`, Cloudflare D1, Turso), the branch (Neon, PlanetScale), or the instance or cluster when it serves exactly one database (`AWS.RDS.DBInstance`, `AWS.DSQL.Cluster`, the primary `GCP.AlloyDB.Instance`). Migrations need a running endpoint, so they go on the resource that is reconciled after the endpoint is ready, not on a parent that is created before it.
484**Transport, in order of preference.** First, an HTTPS data API, which needs no network route from the deploying machine (D1 HTTP, the RDS Data API, Cloud SQL and AlloyDB `executeSql`). Second, a direct driver connection over a public endpoint or the cloud's connector. Third, a runner inside the private network (`AWS.RDS.MigrationRunner`). When there is more than one, expose `migrationsTransport`, and default to the data API whenever the target has it enabled.
485**Credentials.** Use the platform's own credentials when it has them: RDS managed secrets, Neon and PlanetScale minted roles, D1 API tokens. Otherwise take `migrationsUser` and `migrationsPassword` (`string | Redacted<string>`). Never ask the user to stage a secret just for the transport; when an API only reads passwords from a secret store, the provider stages and removes one itself.
486**Executors — reuse, don't rewrite.** `@alchemy.run/sql/Migrations` has the transport pieces; a provider writes only the call that is specific to its cloud:
487`withPgExecutor(config, use)` / `withMySqlExecutor(config, use)` — open a `pg`/`mysql2` connection (connection string, TLS, or a pre-connected `stream`), run, always close.
488`makeStatementExecutor({ dialect, execute, batch })` — any API that runs a SQL string and returns rows (Cloud SQL and AlloyDB `executeSql`). The provider supplies `execute`, failing on SQL errors inside a 200; `batch: "transaction"` wraps each migration in `BEGIN`/`COMMIT`, `"atomic"` relies on the API committing a call as a unit. `rowsFromCells` maps GCP-style `columns` + `rows[].values[]`.
489`splitSqlScript` / `scanSqlScript` — split a script into statements for APIs that take one at a time (PostgreSQL, MySQL, GoogleSQL quoting and comment rules; `scanSqlScript` adds an offset-aligned literal mask for capability checks).
490`migrationError(context)`, `retryLogin` / `probeExecutor` (a just-created user's login delay, retried before any batch), `migrationAttributesOf` / `MigrationAttributes`.
491
492 Targets that can't commit DDL atomically set `transactionalDdl: false` and override `applyMigration`: DSQL checkpoints each statement, and Spanner runs each DDL statement as an operation named after the migration and statement, so a crashed or fixed deploy resumes instead of rerunning applied statements.
493**Errors.** Every transport failure surfaces as `MigrationError` with the database's own message. A SQL error inside a 200 response, as the data APIs return it, is a failure, not a success.
494**Tests.** Write one live test per transport. Deploy one migration and verify the table and `__alchemy_migrations` out-of-band. Add a file and redeploy; only the new file runs. Redeploy unchanged and assert that `stack.plan` reports no update. Destroy and verify the database is gone. Write migration files into a scoped temp directory (`fs.makeTempDirectoryScoped`) and pass it as `migrations`.
495**Docs.** Add a `### Migrations` section to the resource JSDoc, with one example per transport and credential source.
496
497References: [AWS.RDS.DBInstance](./packages/aws/src/RDS/DBInstance.ts) + [Migrations.ts](./packages/aws/src/RDS/Migrations.ts) (three transports), [Turso Database](./packages/turso/src/Database.ts) (one HTTP executor), [GCP.SQL.Database](./packages/gcp/src/SQL/Database.ts) (data API with a staged secret, plus the Cloud SQL connector), [GCP.Spanner.Database](./packages/gcp/src/Spanner/Database.ts) (non-transactional DDL, a GoogleSQL dialect in the shared engine).
498
499Research and design the test cases for each resource. Test cases can be single or multi-step. Single-step test cases are just testing a single create success or failure mode. Multi-step cases are testing a sequence of operations, starting with create and then updating or replacing the resource multiple times. Test cases should be designed to be exhaustive and cover all possible success and failure modes, starting from simple happy paths to long, complicated aggregate (including other resources) smoke tests.
500Implement the Resource contract and Provider in `packages/{cloud}/src/{Service}/{Resource}.ts`.
501
502The Resource contract (Props, Attributes, Binding Contract) and the Resource Provider (lifecycle operations) are co-located in the same file.
503
504Read through the established examples to understand the pattern:
505
506[S3 Bucket](./packages/aws/src/S3/Bucket.ts)
507[SQS Queue](./packages/aws/src/SQS/Queue.ts)
508[DynamoDB Table](./packages/aws/src/DynamoDB/Table.ts)
509[Kinesis Stream](./packages/aws/src/Kinesis/Stream.ts)
510[Lambda Function](./packages/aws/src/Lambda/Function.ts)
511[VPC](./packages/aws/src/EC2/Vpc.ts)
512[Subnet](./packages/aws/src/EC2/Subnet.ts)
513
514The Resource interface takes four type parameters: `Resource<Type, Props, Attributes, BindingContract>`.
515
517export interface Stream extends Resource<
518 "AWS.Kinesis.Stream",
519 StreamProps,
520 {
521 streamName: string;
522 streamArn: string;
523 streamStatus: StreamStatus;
524 }
525> {}
526
527export const Stream = Resource<Stream>("AWS.Kinesis.Stream");
529
530For Resources that accept Bindings (like Lambda Function), include a fourth type parameter for the Binding Contract:
531
533export interface Function extends Resource<
534 "AWS.Lambda.Function",
535 FunctionProps,
536 {
537 functionArn: string;
538 functionName: string;
539 functionUrl: string | undefined;
540 roleName: string;
541 roleArn: string;
542 },
543 {
544 env?: Record<string, any>;
545 policyStatements?: PolicyStatement[];
546 }
547> {}
549
550:::warning
551**Never use `Input<T>` in declared Props interfaces.** Declare plain types (`zoneId: string`, `ips?: string[]`, nested structs with plain fields). The `Resource` machinery applies `Input` automatically — and `Input<T>` is deep (it recursively distributes over arrays and object fields), so even nested references like `memberships: [{ identifier: zone.zoneId }]` accept `Output<string>` without any explicit annotation. Writing `Input<string>` in a Props interface produces a redundant double-wrap.
552
554// ❌ wrong
555export interface LoadBalancerProps {
556 zoneId: Input<string>;
557}
558
559// ✅ right — the engine wraps automatically and deeply
560export interface LoadBalancerProps {
561 zoneId: string;
562}
564
565`Input<T>` in a *function signature* is still legitimate when the function genuinely receives unresolved values at runtime (e.g. helpers that resolve tag maps, or `DurableObjectNamespace.from(scriptName: Input<string>)`).
566:::
567
568Implement each Capability as a `Binding.Service` in `packages/{cloud}/src/{Service}/{Capability}.ts`.
569
570A single `Binding.Service` does both halves of a capability in one Effect:
571
572**Init (outer) Effect** — resolves the host Function/Worker and its environment, then registers the deploy-time binding by calling ``host.bind`${resource}`(data)`` (environment variables + IAM policy statements for AWS, native bindings for Cloudflare), guarded by `!globalThis.__ALCHEMY_RUNTIME__` so it becomes a no-op once running inside the deployed Function/Worker.
573**Runtime (inner) callable** — the typed client returned to the caller; its methods require `Alchemy.RuntimeContext` (see below).
574
575There is no separate deploy-time policy object and nothing to register in `providers()` — the implementation layer is provided directly on the Function/Worker Effect.
576
577Read through the established capabilities to understand the pattern:
578
579[S3 GetObject](./packages/aws/src/S3/GetObject.ts), [S3 PutObject](./packages/aws/src/S3/PutObject.ts)
580[SQS SendMessage](./packages/aws/src/SQS/SendMessage.ts), [DynamoDB GetItem](./packages/aws/src/DynamoDB/GetItem.ts)
581[Kinesis PutRecord](./packages/aws/src/Kinesis/PutRecord.ts), [Lambda InvokeFunction](./packages/aws/src/Lambda/InvokeFunction.ts)
582Access-split Cloudflare capabilities: [R2 Bucket](./packages/cloudflare/src/R2/), [KV Namespace](./packages/cloudflare/src/KV/), [Queue](./packages/cloudflare/src/Queues/)
583
584For Event Sources, see:
585
586[SQS QueueEventSource](./packages/aws/src/SQS/QueueEventSource.ts)
587[S3 BucketEventSource](./packages/aws/src/S3/BucketEventSource.ts)
588
589Each capability exports its contract plus one or more implementation layers:
590
592// 1. The Binding.Service class (the contract) + a bind alias for ergonomic use
593export class PutRecord extends Binding.Service<...>()("AWS.Kinesis.PutRecord") {}
594export const putRecord = PutRecord.bind;
595
596// 2. The implementation layer — resolves the host + environment, registers the
597// binding inline (guarded by __ALCHEMY_RUNTIME__), and returns the runtime client.
598export const PutRecordLive = Layer.effect(
599 PutRecord,
600 Effect.gen(function* () {
601 const host = yield* Worker; // or the AWS Function host
602 const env = yield* WorkerEnvironment; // or Lambda.FunctionEnvironment
603 return Effect.fn(function* (stream: Stream) {
604 if (!globalThis.__ALCHEMY_RUNTIME__) {
605 // AWS: { policyStatements: [...] } Cloudflare: { bindings: [...] }
606 yield* host.bind`${stream}`({ policyStatements: [...] });
607 }
608 return /* typed runtime client closing over `env` */;
609 });
610 }),
611);
613
614Provide the implementation layer on the **Function/Worker** Effect (`Effect.provide(PutRecordLive)`).
615
616### Read/Write/ReadWrite binding convention
617
618When a capability's API distinguishes access levels (R2 `head`/`get`/`list` vs `put`/`delete`; KV `get`/`getWithMetadata`/`list` vs `put`/`delete`), split it into three `Binding.Service`s so consumers can request least privilege, each with two interchangeable implementations:
619
620**`{Cap}Read.ts` / `{Cap}Write.ts` / `{Cap}ReadWrite.ts`** — the `Binding.Service` class + runtime client interface + a `bind` alias (e.g. `ReadBucket = BucketRead.bind`). `ReadWrite`'s client interface `extends` both the `Read` and `Write` client interfaces.
621**`{Cap}Binding.ts`** — *shared* worker-binding scaffolding: a `make{Cap}Binding({ makeClient })` that resolves `WorkerEnvironment` + host `Worker`, registers the native binding via ``host.bind`${resource}`(...)`` (guarded by `__ALCHEMY_RUNTIME__`), plus a `make{Cap}Helpers` returning the low-level `raw`/`use`/`tryPromise` primitives. **Do NOT export this file from `index.ts`.**
622**`{Cap}ReadBinding.ts` / `{Cap}WriteBinding.ts` / `{Cap}ReadWriteBinding.ts`** — `Layer.effect` implementations over the native binding (`ReadBucketBinding`, …) plus the `makeRead`/`makeWrite` client builders. `ReadWrite` composes the read + write builders.
623**`{Cap}Http.ts`** — *shared* HTTP/token scaffolding: a `makeHttp{Cap}Binding({ permissionGroups, makeClient })` that mints a scoped `AccountApiToken` with the right permission groups, binds the token's `value`/`accountId` into the Worker, and resolves the per-operation scope. **Do NOT export this file from `index.ts`.**
624**`{Cap}ReadHttp.ts` / `{Cap}WriteHttp.ts` / `{Cap}ReadWriteHttp.ts`** — `Layer.effect` implementations over the cloud's HTTP API (`ReadBucketHttp`, …). Operations the HTTP API can't support `Effect.die` with a typed error.
625
626Rules:
627
628**Keep shared scaffolding internal.** Re-export only the contracts, the per-level layers, and the client builders from `index.ts`. Exporting `{Cap}Binding.ts`/`{Cap}Http.ts` leaks generic helper names into the flat `Cloudflare`/`AWS` namespace and collides across services.
629**Use service-unique helper names.** Avoid generic `makeHelpers`/`makeWrite`; prefix with the capability (`makeQueueHelpers`, `makeWriteQueueClient`) so re-exported builders never clash.
630**Namespace the public surface.** Export the service both flatly and as a namespace (`export * as KV from "./KV/index.ts"`) so callers write `Cloudflare.KV.ReadWriteNamespace(ns)`. The bind-alias callables drop the redundant service prefix (`ReadWriteNamespace`, not `ReadWriteKVNamespace`); the underlying classes/interfaces keep it (`KVNamespaceReadWrite`, `ReadWriteKVNamespaceClient`).
631**No resource-level `bind`.** The Resource is plain `Resource<T>("Cloud.Type")` with no `bind:` field; callers bind via the namespaced capability, not `resource.bind`.
632**Single-mode capabilities stay single.** A producer-only capability (e.g. Cloudflare Queue's send-only producer, which has no runtime read) ships just the `Write` service — don't invent a `Read` the runtime can't satisfy. The HTTP impl can still cover both directions where the API does.
633
634Reference: [Cloudflare R2 Bucket](./packages/cloudflare/src/R2/), [KV Namespace](./packages/cloudflare/src/KV/), [Queue](./packages/cloudflare/src/Queues/).
635
636### Runtime-only methods: color with `Alchemy.RuntimeContext`
637
638The runtime callable returned by a `Binding.Service` (the inner Effect inside `.bind(resource)`'s return) **must** declare `Alchemy.RuntimeContext` as a requirement. This is how Alchemy models "this code can only run inside a deployed Function/Worker" at the type level — analogous to a colored function.
639
641import type { RuntimeContext } from "../../RuntimeContext.ts";
642
643export class GetItem extends Binding.Service<
644 GetItem,
645 <T extends Table>(
646 table: T,
647 ) => Effect.Effect<
648 (
649 request: GetItemRequest,
650 ) => Effect.Effect<
651 DynamoDB.GetItemOutput,
652 DynamoDB.GetItemError,
653 RuntimeContext // ← runtime-only
654 >
655 >
656>()("AWS.DynamoDB.GetItem") {}
658
659Rules:
660
661**Outer Effect** (the `bind(resource)` setup) runs at the Function's init phase. It does NOT require `RuntimeContext`.
662**Inner Effect** (the actual SDK invocation) only makes sense inside a running Function. It MUST require `RuntimeContext`.
663Resolve cloud-environment services (`WorkerEnvironment`, AWS SDK clients, etc.) once during Layer construction and close over them. Do NOT leak `WorkerEnvironment` / `Lambda.FunctionEnvironment` onto the runtime callable — that couples downstream service code to a specific cloud and breaks Layer encapsulation. The Function/Worker runtime satisfies `RuntimeContext` automatically.
664The implementation can return `Effect.Effect<A, E>` without explicitly providing `RuntimeContext` (it's contravariant in `R`); just declare it on the interface.
665
666Why this matters: consumers can build cloud-agnostic services on top of bindings using `Layer.effect(Tag, ...)` without polluting their service interface with `WorkerEnvironment`. See [Layers concept](./website/src/content/docs/infrastructure-as-effects/layers.mdx).
667
668After implementing, re-export the contract and implementation layers from the service's `index.ts` (but keep the shared `{Cap}Binding.ts`/`{Cap}Http.ts` scaffolding un-exported).
669
670### Isolate scope vs request scope (runtime bridges)
671
672**Layer construction is isolate-scoped; the effects built services expose are request-scoped.**
673
674**At layer build / Worker init (instance scope)** a layer MAY resolve services and env/config, register listeners and `bind` declarations, assemble `Effect.fn` clients, and perform one-shot I/O that produces a plain cached value (e.g. fetch a secret and cache it for a client). It MUST NOT acquire **disposable** resources — connections, pools, streams, anything with a finalizer (`Layer.scoped` / `Effect.acquireRelease` / init-level `Effect.addFinalizer`) — or retain I/O-backed *objects* or promises across events (workerd pins them to the creating request's IoContext). The runtime bridges (Worker event, Durable Object call, Workflow run, Lambda invoke) build the layer stack **once per instance** on the first event. Instance finalizers run at instance shutdown at best: **never on workerd** (no teardown hook), and in a **best-effort 500 ms SIGTERM window on Lambda** (the generated entry registers an internal extension to obtain it and closes the instance scope on SIGTERM; not delivered on hard failures). Server processes (Containers, ECS Tasks) close their root scope on graceful exit.
675**At request scope**, anything needing I/O or cleanup is an effect requiring `Scope.Scope`, acquired lazily per call. Every bridge provides a fresh `Scope` per event; `Effect.addFinalizer` in a handler attaches to it and runs after the response (registered with `ctx.waitUntil` on workerd; settled inline on Lambda). Per-request memoization keys on the scope object (`yield* Effect.scope`) — see [Drizzle/Postgres.ts](./packages/drizzle/src/Postgres.ts) for the canonical WeakMap pattern. One pool/socket per event is the law on workerd (sockets are IoContext-pinned); Hyperdrive is the cross-request pooler.
676
677### Effect workflow engines
678
679Users write durable workflows once with `effect/workflow` and core's `alchemy/Workflows`, and run them on each cloud's native durable runtime. A provider whose cloud has a durable execution engine (Cloudflare Workflows and Durable Objects, Lambda durable functions, Azure Durable Functions on the Durable Task Scheduler, …) ships an **engine**: a plain `Layer` that implements Effect's `WorkflowEngine` on that runtime. The workflow code doesn't change between clouds; only the engine Layer does.
680
681The DX every engine must keep:
682
684// workflows.ts — cloud-agnostic
685export const Greet = Workflow.make("Greet", {
686 payload: { id: Schema.String, name: Schema.String },
687 success: Schema.String,
688 idempotencyKey: ({ id }) => id,
689});
690
691export const GreetLive = Workflows.layer(
692 Greet,
693 Effect.gen(function* () {
694 const receipts = yield* Cloudflare.R2.WriteBucket(Receipts); // bindings resolve once, at Layer build
695 return Effect.fn(function* ({ name }) {
696 return yield* Activity.make({ name: "greet", success: Schema.String, execute: Effect.succeed(`hello ${name}`) });
697 });
698 }),
699);
700
701// a runtime — start and read runs through the engine the runtime provides
702const greet = yield* Workflows.RunWorkflow(Greet);
703const run = yield* greet.start({ id: "g-1", name: "sam" });
704yield* run.status; // NotStarted | Running | Completed | Failed | Interrupted | Died
705yield* run.signal(Approval, "agent-7");
706yield* run.interrupt;
708
709`Workflows.layer` marks a workflow `Implemented`, and `RunWorkflow` requires it, so running a workflow nothing implements is a type error. Keep that property in every engine.
710
711There are two engine shapes; pick the one the cloud's runtime forces:
712
713**In-process engine** — the durable construct is hosted by the same runtime that starts runs. The engine is a Layer the user provides next to the workflow Layers: `GreetLive.pipe(Layer.provideMerge(Cloudflare.Workflows.NativeEngine))`. References: [NativeEngine.ts](./packages/cloudflare/src/Workflows/NativeEngine.ts) (one Cloudflare Workflow class hosts every workflow; steps are `step.do`, sleeps `step.sleep`, signals Workflow events) and [DurableObjectEngine.ts](./packages/cloudflare/src/Workflows/DurableObjectEngine.ts) (one Durable Object per run, journal in SQLite, wake-ups are alarms).
714**Host + remote engine** — runs execute in a dedicated durable runtime resource. The provider ships a `<Runtime>WorkflowFunction`-style resource that lists the workflows it hosts (`.make(props, layer)` is a type error when the Layer leaves one out), and a `<Runtime>Engine(...hosts)` Layer other runtimes provide to start and read runs there; it also grants the caller access to each host. Reference: [AWS.Lambda.WorkflowFunction](./packages/aws/src/Lambda/WorkflowFunction.ts) + `AWS.Lambda.DurableFunctionEngine`.
715
716Build an engine from the shared pieces, never a private copy:
717
718`alchemy/WorkflowEncoding` — payload/result/exit codecs and `CurrentActivity`; results are cached by schema identity.
719`Workflows.Implemented` — the marker a remote engine provides for the workflows its hosts list.
720The cloud's own primitives map onto Effect's: activities → durable steps (replayed, not re-run, once finished), `DurableClock.sleep` → the runtime's durable wait (no compute billed while waiting), `DurableDeferred` → the runtime's external-event/callback mechanism, interrupt → fail the pending wait so compensations run, child workflows → runs on the same engine.
721Respect platform limits in the engine (instance-id length and charset, max wait, propagation after deploy) and retry only the typed transient errors.
722
723Every engine passes one shared conformance suite, [`test/Workflows/conformance/`](./packages/integration-tests/test/Workflows/conformance/), driven over HTTP against a deployed fixture. Don't copy it. The suite owns the workflows (`workflows.ts`), the routes (`app.ts`), and the cases (`conformance.ts`), drawn from Effect's own engine tests and from bugs found here: results, idempotent starts, durable sleeps that wake on time (between and inside activities), signals (early, repeated, late), compensations (on failures and interrupts), defects (captured, caught, retried, `SuspendOnFailure` with resume), `Activity.retry`, `Activity.raceAll`, finalizers and `ensuring` across suspensions, parallel activities, interrupts (while waiting, sleeping, on the way to a wait, after finishing), and child workflows (waiting and failing). An engine supplies a fixture that serves `app` over `WorkflowsLive` with its engine and a `Journal` layer over its cloud's storage, then calls `conformance({ test, url, napsWithinMillis, recoveryWithinMillis, timeout, unsupported })` from its suite. See [WorkflowEngines.test.ts](./packages/integration-tests/test/Cloudflare/Workers/WorkflowEngines.test.ts) (a Durable Object journal) and [WorkflowFunction.test.ts](./packages/integration-tests/test/AWS/Lambda/WorkflowFunction.test.ts) (a DynamoDB journal). A case an engine fails is an engine bug, not a reason to skip it. What the platform genuinely can't do goes in `unsupported`, and the engine dies with a clear defect the case checks for, never a hang: Cloudflare Workflows and Lambda durable functions can't look a step's result up before running it, so neither supports `raceAll`. An engine with a local provider runs the cases in dev too.
724
725Each engine gets a docs page at `website/src/content/docs/{cloud}/compute/effect-workflows.mdx` with the same headings as the AWS and Cloudflare pages (define a workflow, implement its steps, run it on the engine, start and read a run, sleep durably, wait for a signal, undo a step, use bindings in a step), and an `examples/{cloud}-effect-workflows` example.
726
727:::tip
728If you need to know what AWS region or account ID the resource is being created/updated in, you can use this inside any of the lifecycle operations.
729
731const region = yield * Region;
732const account = yield * Account;
734
735:::
736
737:::warning
738You should favor getting the region/account INSIDE the lifecycle operations instead of inside the Layer effect like this because then it's scoped to the resource isntead of the resource provider:
739
741reconcile: Effect.fn(function* ({ id, news, output, session }) {
742 const { accountId, region } = yield* AWSEnvironment.current;
743});
745
746:::
747
748:::warning
749Do not use `Effect.orDie` in the lifecycle operations since this will crash the whole IaC engine.
750:::
751
752:::warning
753**Never use `async`/`await`, raw `Promise`, `node:fs/promises`, `node:fs`, `node:os`, or `pathe` directly in resource code.** Always use the Effect platform services so that effects remain composable, traceable, retryable, and testable:
754
755| Don't | Do |
756| ---------------------------------------------------- | ----------------------------------------------------------- |
757| `import fs from "node:fs/promises"` | `const fs = yield* FileSystem.FileSystem` |
758| `await fs.readFile(p, "utf8")` | `yield* fs.readFileString(p)` |
759| `await fs.mkdtemp(...)` | `yield* fs.makeTempDirectory({ prefix: ... })` |
760| `import path from "pathe"` / `node:path` | `const path = yield* Path.Path` |
761| `await fetch(...)` | `yield* HttpClient.HttpClient` + `HttpClientRequest` |
762| `Effect.promise(() => listSqlFiles(dir))` | Make `listSqlFiles` itself return `Effect` and `yield*` it |
763| `new Promise((res) => setTimeout(res, ms))` | `yield* Effect.sleep(Duration.millis(ms))` |
764| `process.env.DOCKER_CONFIG` | `yield* Config.String("DOCKER_CONFIG")` (`Config.withDefault` / `Config.option` when optional) |
765
766Sync, CPU-only Node APIs (e.g. `crypto.createHash().update().digest()`, `process.cwd()`, `Buffer`, `TextEncoder`) must still be wrapped in `Effect.sync(() => ...)` (or `Effect.try` if they can throw) so the call participates in the Effect runtime — tracing, interruption, and error channels. Don't call them as bare expressions inside `Effect.gen`.
767
769const hash = yield* Effect.sync(() =>
770 crypto.createHash("sha256").update(input).digest("hex"),
771);
772const cwd = yield* Effect.sync(() => process.cwd());
774
775Environment variables are configuration: read them through `effect/Config`, never `process.env`, so a provided `ConfigProvider` (`.env` files, profiles, tests) is honored. `Effect.sync` is only for process facts `Config` can't express, like `process.platform` or `process.cwd()`. Module-scope test gates (`test.skipIf(!!process.env.FAST)`) run outside Effect and are exempt.
776
777This applies to **lifecycle operations, helpers, AND tests**. Tests must use `FileSystem.FileSystem`/`Path.Path` for any file/path access (see [Database.test.ts](./packages/integration-tests/test/Cloudflare/D1/Database.test.ts) for the pattern).
778:::
779
780:::tip
781If a Resource supports tags, you should always include the internal Alchemy tags to brand the resource with the app, stage and logical ID so that we can "know" that we created it and are responsible for it.
782
784reconcile: Effect.fn(function* ({ id, news, output, session }) {
785 const internalTags = yield* createInternalTags(id);
786 const userTags = news.tags ?? {};
787 const allTags = { ...internalTags, ...userTags };
788});
790
791:::
792
793:::warning
794Do not roll your own tag diffing logic, always use `diffTags` from [Tags.ts](./packages/alchemy/src/Tags.ts), and diff against **observed cloud tags** (not `olds.tags` or `output.tags`). Adoption can hand you a resource whose tags don't match what we last persisted.
795
797reconcile: Effect.fn(function* ({ id, news, output, session }) {
798 const internalTags = yield* createInternalTags(id);
799 const newTags = { ...news.tags, ...internalTags };
800 // Read tags fresh from the cloud so adoption (where tags may not match
801 // what we last persisted) converges correctly.
802 const oldTags = yield* fetchObservedTags(/* … */);
803 // Option 1. use `upsert` if the API expects you to create/update tags in one call
804 const { removed, upsert } = diffTags(oldTags, newTags);
805 // Option 2. use `added` and `updated` if the API expects you to create/update tags in separate calls
806 const { removed, added, updated } = diffTags(oldTags, newTags);
807 // Option 3. use `upsert` only if the API doesn't expect you to remove tags (only PUT/UPDATe)
808 const { upsert } = diffTags(oldTags, newTags);
810
811:::
812
813Implement the test cases in `packages/integration-tests/test/{Cloud}/{Service}/{Resource}.test.ts`.
814
815### Resource and runtime testing conventions
816
817Choose the deployment-based pattern that fits the behavior under test:
818
819**Resource lifecycle and reconciliation:** use `test.provider` and `stack.deploy` for create, update, replacement, and deletion scenarios.
820**Bindings and runtime behavior:** prefer a deployed Worker/Function fixture that exposes the behaviors under test. Share its deployment with `beforeAll`/`afterAll` and exercise it over HTTP or the relevant event source. Follow [Test Fixtures for Effect-Native Workers / Functions](#test-fixtures-for-effect-native-workers--functions).
821**Lifecycle changes that affect runtime behavior:** a `test.provider` case can deploy a fixture and exercise it between updates.
822
823Keep regression cases in the relevant existing resource or binding suite. A Worker lifecycle feature belongs in `test/Cloudflare/Workers/Worker.test.ts`; binding fixtures belong beside their owning suite in `fixtures/`.
824
825Assume Docker is installed. Never `skipIf`-gate a test on Docker availability; a missing daemon fails the test.
826Start and end `test.provider` cases with `stack.destroy()`. Fixture suites deploy in `beforeAll` and destroy in `afterAll`. Use deterministic resource identities and verify deletion through the cloud API.
827Exercise create, update, redeploy, and removal through the stack. **Do not mock HTTP or SDK responses, invoke provider lifecycle methods directly, or call extracted reconciliation helpers as a substitute for a deployment.**
828Verify the resulting cloud state through distilled and exercise the deployed runtime when relevant. Use real SDK calls on test-owned resources to establish out-of-band state or verify API update semantics. Keep provider behavior assertions driven by `stack.deploy`.
829For a partial-update regression, configure non-default neighboring settings, exercise the update, and verify that unrelated settings remain unchanged. Ensure the relevant update path actually runs.
830Keep production logic in its natural resource/provider location. Do not extract one-off reconciliation functions or modules solely to make them directly testable.
831
832Pure utility and engine unit tests remain appropriate for their own behavior; they do not replace live resource lifecycle coverage.
833
834Read through the established test cases before continuing so that you understand the pattern and structure of the test cases.
835
836[S3 Bucket Test Cases](./packages/integration-tests/test/AWS/S3/Bucket.test.ts)
837[SQS Queue Test Cases](./packages/integration-tests/test/AWS/SQS/Queue.test.ts)
838[Lambda Function Test Cases](./packages/integration-tests/test/AWS/Lambda/Function.test.ts)
839[Kinesis Stream Test Cases](./packages/integration-tests/test/AWS/Kinesis/Stream.test.ts)
840[DynamoDB Table Test Cases](./packages/integration-tests/test/AWS/DynamoDB/Table.test.ts)
841[VPC Test Cases](./packages/integration-tests/test/AWS/EC2/Vpc.test.ts)
842[Subnet Test Cases](./packages/integration-tests/test/AWS/EC2/Subnet.test.ts)
843
844:::warning
845Never use `Date.now()`, UUIDs or other random values when constructing the physical name of a resource. Every run must be deterministic and leave no orphaned resources. You should either:
846
847Not provide a name, and rely on the resource provider to generate one from the app, stage and logical ID. This is the default for every test.
848Only when the test is about explicitly provided names, construct a deterministic one unique to each test case and to the user running it: derive it from `stack.stage` (`test_$USER` by default). It must be the same on each subsequent run of the test case.
849Only for a resource the API can never delete (RealtimeKit apps, Intel indicator feeds), use one shared constant name that every run adopts, so the suite owns a single standing instance instead of leaking one per user.
850 :::
851
852Consider implementing an aggregate Smoke test that brings together multiple resources that are often used together.
853
854See the [VPC Smoke Test](./packages/integration-tests/test/AWS/EC2/Vpc.smoke.test.ts) for an example.
855
856Add the resource-level JSDoc (`###` section headings + `**Example:**` labels, with generator metadata tags last) and field-level JSDoc on each prop/attribute on the source `.ts` file. Run `pnpm docs:check-jsdoc`, then `pnpm docs:gen` to refresh `website/src/content/docs/providers/{Cloud}/{Resource}.md`. Do NOT manually edit the generated markdown.
857
858# Typed Error Doctrine (distilled)
859
860## How distilled is built (Smithy + JSON Patch)
861
862Distilled is a Smithy-based SDK factory. Every provider package (`submodules/distilled/packages/{cloud}`) runs the same pipeline:
863
864**Convert** — the provider's spec source is converted into Smithy 2.0 JSON models, one per service, in `.generated-specs/{service}.json`. Cloudflare mines them from the downloaded API docs (`scripts/spec-to-smithy.ts` over `specs/api/resources/**`); AWS consumes the official `api-models-aws` Smithy models submodule directly. Hand-authored models for APIs the spec source doesn't cover live in `manual-specs/`.
865**Patch** — an **RFC 6902 JSON Patch chain** (files shaped `{ "description": ..., "patches": [ops] }`) is applied to the provider's intermediary spec before codegen. For Cloudflare, patches in `patches/{service}/*.json` target the **Smithy model**, applied in filename order with `*.manual.json` files last; `_metadata.json` carries service-level `/metadata/keyDictionary` and `/metadata/opAliases`. OpenAPI-sourced providers (Neon, PlanetScale, Stripe, …) patch the **OpenAPI document** upstream of the smithy conversion instead. A patch whose target path is stale (no longer in the model) warns and is skipped; a malformed patch **fails the generator run**.
866**Generate** — the shared smithy→SDK compiler in `@distilled.cloud/core/codegen` compiles each patched model into an Effect SDK module at `src/services/{service}.ts` plus the barrel.
867
868Consequences:
869
870**Never edit `src/services/*.ts`** — regeneration overwrites it. Anything wrong in the generated SDK (missing error, wrong request/response schema, misnamed operation or member) is fixed with a JSON Patch (`add`/`remove`/`replace`/`move` on the model) in `patches/{service}/`.
871Patch paths address the **Smithy model**: shape IDs are `com.cloudflare.{service}#Name`, and member names are **wire names** (snake_case). The camelCase TS surface is derived at codegen; `move` ops rename shapes and members.
872
873## The doctrine
874
875Every error a distilled operation can produce in practice MUST be a tagged error in that operation's **type-level** error union. The catch-all classes (`UnknownCloudflareError`, `CloudflareHttpError`, and the status-derived classes like `NotFound`/`BadRequest` that distilled leaves out of the typed union) exist only to *surface* gaps — they are never something alchemy code handles.
876
877**When you hit an unmatched error** (an `UnknownCloudflareError`, or you find yourself wanting to check `CloudflareHttpError.status` or an out-of-union `NotFound`), the fix is ALWAYS a distilled patch, never a catch in alchemy:
878
879Note the error's code / status / message from the failure output.
880Add or extend `submodules/distilled/packages/cloudflare/patches/{service}/{operation}.json` with a JSON Patch that (a) adds an error structure carrying the `smithy.api#error` trait and `com.cloudflare.protocols#errorMatchers` matchers, and (b) attaches it to the operation's `errors` list. Use a **meaningful, resource-specific tag** (e.g. `WidgetNotFound`, not a bare `NotFound`):
881
883 {
884 "description": "Type the not-found error on getWidget",
885 "patches": [
886 {
887 "op": "add",
888 "path": "/shapes/com.cloudflare.widgets#WidgetNotFound",
889 "value": {
890 "type": "structure",
891 "members": {
892 "code": { "target": "smithy.api#Integer" },
893 "message": { "target": "smithy.api#String" }
894 },
895 "traits": {
896 "smithy.api#error": "client",
897 "com.cloudflare.protocols#errorMatchers": [{ "code": 1234 }]
898 }
899 }
900 },
901 {
902 "op": "add",
903 "path": "/shapes/com.cloudflare.widgets#GetWidget/errors",
904 "value": [{ "target": "com.cloudflare.widgets#WidgetNotFound" }]
905 }
906 ]
907 }
909
910 If the operation already has an `errors` array (from an earlier patch), append with `"path": ".../errors/-"` instead of adding the whole array. Matchers may combine `code`, `status`, and `message` (a string, or `{ "includes": "..." }` / `{ "matches": "..." }`) — e.g. `[{ "status": 400, "message": { "includes": "snippet not found" } }]` when Cloudflare misuses 400 for a missing resource. Prefer matching the Cloudflare error `code` when one exists; fall back to `status` + `message` otherwise. The most specific matcher wins; ties break by declaration order.
911
912Regenerate ONLY that service: `cd submodules/distilled/packages/cloudflare && bun scripts/generate.ts --resource {service}` (then format: `pnpm exec oxfmt src/services/{service}.ts`). A warned-stale or failed patch is a bug in your patch — fix it; never leave a red generate.
913Handle the now-typed tag in alchemy code and re-run the tests.
914
915**AWS is the one exception to the JSON Patch format**: it layers typed-error metadata over the official Smithy models with a per-service schema file `submodules/distilled/packages/aws/patches/{service}.json` (error categories, aliases, synthetic errors with message matchers — see `submodules/distilled/packages/aws/scripts/spec-schema.ts`), regenerated with `cd submodules/distilled/packages/aws && bun scripts/generate.ts --sdk {service}`. The doctrine is identical; only the patch dialect differs.
916
917**Forbidden patterns** — these defeat the type system and must never appear in alchemy code or tests:
918
920// ❌ unknown-typed structural predicates
921const isNotFoundError = (e: unknown): boolean =>
922 Predicate.hasProperty(e, "_tag") && (e as { _tag: unknown })._tag === "NotFound";
923
924// ❌ widening casts to duck-typed tags
925Effect.retry({ while: (e) => (e as { _tag?: string })._tag === "Forbidden" })
926
927// ❌ catching the catch-all HTTP error by status
928Effect.catchIf((e) => e._tag === "CloudflareHttpError" && e.status === 404, ...)
930
931**Required patterns** — fully inferred, no casts, no `unknown`:
932
934// ✅ catch a typed tag
935.pipe(Effect.catchTag("WidgetNotFound", () => Effect.void))
936
937// ✅ retry while a typed tag is observed (e is the op's inferred error union)
938Effect.retry({ while: (e) => e._tag === "WidgetNotFound", schedule, times })
939
940// ✅ multiple tags
941Effect.catchTag(["WidgetNotFound", "Gone"], () => Effect.succeed(undefined))
943
944If `Effect.catchTag("SomeTag", ...)` fails to typecheck, that is the signal that distilled's union is missing the error — patch distilled (step 2 above); do not loosen the alchemy-side types.
945
946# Provider Modes & Local Providers (doctrine)
947
948Some resource types have TWO implementations: a **live** provider (converges real cloud state, `alchemy deploy`) and a **local** provider (emulates the resource on the developer's machine, `alchemy dev`). Provider mode is a first-class engine concept — see [ProviderMode.ts](./packages/alchemy/src/ProviderMode.ts), [Local/ProviderLayer.ts](./packages/alchemy/src/Local/ProviderLayer.ts), [Local/LocalProvider.ts](./packages/alchemy/src/Local/LocalProvider.ts), and the user-facing guide at [website/src/content/docs/infrastructure-as-code/local-provider.mdx](./website/src/content/docs/infrastructure-as-code/local-provider.mdx).
949
950Engine semantics (never re-implement these per provider):
951
952**Register both variants with `ProviderLayer.dual(cls, { live: () => ..., local: () => ... })`** — never select one at layer build. The run-default variant builds eagerly; the other is lazy and memoized, so mode-specific dependency layers (workerd, Docker) MUST compose *inside* the `local` thunk (e.g. `LocalWorkerProvider().pipe(Layer.provide(localRuntimeServices()))`), never globally — a plain deploy must not construct local machinery unless a local-mode row needs deleting. Dep layers shared across several local providers must be **module-memoized layer references** so the build MemoMap dedupes them to one instance.
953**Every state commit stamps `providerMode`.** A persisted mode that differs from the resolved mode plans a REPLACEMENT (the provider diff is not consulted), and every delete — replacement old generations, GC chain draining, orphans, `destroy`, `sync`/`tail`/`logs` — resolves the provider variant of the row's *stamped* mode via `findProviderByType(type, mode)`. Legacy rows without a stamp are assumed **live** (pre-mode engines and mode-agnostic providers only ever acted on the real cloud), so a dev run replaces them like any stamped live row — assuming the run's mode instead would silently adopt a deployed live resource as a local instance and leak it untracked. The exception is an unstamped row whose attrs carry the local identity marker (`dev:`-prefixed ids, see `stampedMode` in `ProviderMode.ts`): it was reconciled by a pre-stamping `alchemy dev` run and is treated as **local**, so a plain destroy/deploy never hands its `dev:` identity to the live cloud API.
954**`Alchemy.remote()`** opts a resource or scope OUT of local emulation during dev (captured at registration like `adopt()`; a no-op during deploy). There is deliberately no `local()` — dev mode IS the local default; tests simulate a dev run by overriding `AlchemyContext.dev` (`inDev` in [test.resources.ts](./packages/alchemy/test/test.resources.ts)) or `Test.make({ dev: true })`. Conflicting decorations on one FQN die with `ConflictingProviderModeError`. Single-implementation providers are **mode-agnostic**: they satisfy any requested mode, never stamp, never replace on a switch — dev runs over constructs mixing emulatable and live-only resources (DNS records) just work.
955
956## `LocalProvider.make` — long-running local providers
957
958A local provider whose physical resource is a **running process** (dev server, workerd instance) MUST be built with `LocalProvider.make(cls, providersUrl, spec)` — do not hand-roll FiberMap/instance-registry/hash machinery in the provider:
959
960**`resolveConfig(ctx)`** — the restart surface. Plain, canonically-hashable data only (no closures or runtime objects: derive plain *descriptors* here and materialize `BindingHook`s etc. inside `start` — see the descriptor/hook split in [LocalWorkerProvider.ts](./packages/cloudflare/src/Workers/LocalWorkerProvider.ts)). Must be cheap and side-effect-free — it runs inside `diff` on every plan. Its canonical hash decides noop-vs-restart AND the same value is handed to `start`, so "what changed?" and "what starts?" can never drift. Deliberately EXCLUDE runtime wiring observed at start time (e.g. queue consumers read from `LocalRuntimeState`) — sibling reconciles drive those via restart hooks, not config.
961**`start(ctx)`** — boot ONE instance in the ambient `Scope` and return Attributes at *readiness*; the process keeps running until the runner closes the scope on restart/delete. For processes that can die on their own, fork `ctx.invalidate` after the exit so the next plan reports `update`.
962**`stop(ctx)`** — delete-only cleanup for state that intentionally survives restarts (URL proxies kept for address stability, restart hooks). NOT called on restarts.
963Generated for you: instance registry, per-id lock (restarts never interleave), config-hash noop/restart, instanceId-guarded delete (a create-first replacement's old-generation delete cannot kill its successor), `list`. The provider is wrapped in `RpcProvider`, so during `alchemy dev` the registry lives in the sidecar and survives user-code hot reloads.
964**Never use `Hash.structure` to compare configs** — it XOR-folds sibling fields, so a value mirrored into two subtrees (env ↔ bindings) cancels out. The helper's canonical hash (sorted-key JSON + sha256, Redacted/bytes/cycle-aware) is the law.
965
966Registry-style local providers (the "instance" is an in-memory row, e.g. Cloudflare's local Queue mutating `LocalRuntimeState`) do NOT use `LocalProvider.make` — write a plain provider and register it via `dual`.
967
968Reference implementations: [Command/Dev.ts](./packages/alchemy/src/Command/Dev.ts) (minimal: spawn + URL readiness), [Cloudflare Workers LocalWorkerProvider.ts](./packages/cloudflare/src/Workers/LocalWorkerProvider.ts) (full-size: bundler watch loop, cross-restart proxy in `stop`), [Cloudflare Queues Queue.ts](./packages/cloudflare/src/Queues/Queue.ts) (registry-style, no runner). Engine coverage lives in [test/provider-mode.test.ts](./packages/alchemy/test/provider-mode.test.ts) and the "provider modes" describes in [plan.test.ts](./packages/alchemy/test/plan.test.ts) / [apply.test.ts](./packages/alchemy/test/apply.test.ts).
969
970## Local tests (`{Resource}.local.test.ts`)
971
972Every resource with a local (dev) provider gets a **`{Resource}.local.test.ts`** co-located with its live test, using `Test.make({ providers, dev: true })`. Dev-mode tests run local providers behind a **file-scoped RPC sidecar by default** — the same `RpcProviderProxy` topology the real `alchemy dev` command uses, where `providerServicesEffect` layers (e.g. `localRuntimeServices()`) are EMPTY in the test process and RPC-backed providers run their lifecycle in the sidecar. This is what catches a plain local provider that needs those services in the main process (the class of bug behind #1007, which in-process tests masked). `Test.make({ ..., sidecar: false })` opts a file back into the in-process topology — reserve it for debugging provider code or pinning the in-process path a plain `alchemy deploy` takes when deleting `providerMode: "local"` rows.
973
974The suite must cover, minimum:
975
976**The local roundtrip** — deploy the resource + a worker binding it (file-based fixture `main`, never inline `script` — unsupported in dev) and drive the binding over HTTP against the local simulator. Assert the resource's identity carries the `dev:` marker (proof no cloud call ran; for the Worker itself the marker is a `http://localhost:<port>` `url`).
977**The `Alchemy.remote()` opt-out** — the same shape with the resource piped through `Alchemy.remote()`: assert a real (non-`dev:`) identity, round-trip through the remote-proxied binding, verify out-of-band via distilled that the write landed in the real cloud resource, and after `stack.destroy()` verify the cloud resource is gone (pins the stamped-mode delete path).
978
979Reference: [KV Namespace.local.test.ts](./packages/integration-tests/test/Cloudflare/KV/Namespace.local.test.ts) (includes the mixed local + live stack), [R2 Bucket.local.test.ts](./packages/integration-tests/test/Cloudflare/R2/Bucket.local.test.ts), [D1 Database.local.test.ts](./packages/integration-tests/test/Cloudflare/D1/Database.local.test.ts) (includes local migrations and the #1007 regression), [Queues Queue.local.test.ts](./packages/integration-tests/test/Cloudflare/Queues/Queue.local.test.ts) (produce→consume through RPC-backed providers). The harness contract itself is pinned by [TestSidecar.test.ts](./packages/integration-tests/test/Local/TestSidecar.test.ts).
980
981# Test Fixtures for Effect-Native Workers / Functions
982
983To test runtime behavior of an Effect-native Worker, Workflow, Lambda, etc., write a **fixture** that defines the Worker/Function with the bindings under test and exposes one HTTP route per behavior, then write a **test** that deploys the fixture once via `beforeAll` and drives it over HTTP.
984
985## File system layout
986
987Put fixtures in a `fixtures/` directory next to the test file. Each test suite owns its own fixtures — never reach across suites:
988
990packages/integration-tests/test/{Cloud}/{Service}/{Resource}.test.ts
991packages/integration-tests/test/{Cloud}/{Service}/fixtures/{worker|workflow|handler}.ts
993
994## Fixture shape
995
996Resolve the bindings, expose one route per behavior, default-export the class so the test can deploy it directly:
997
999// fixtures/worker.ts
1000import * as Cloudflare from "@/Cloudflare/index.ts";
1001import * as Effect from "effect/Effect";
1002import { HttpServerRequest } from "effect/http/HttpServerRequest";
1003import * as HttpServerResponse from "effect/http/HttpServerResponse";
1004import { Gateway } from "./gateway.ts";
1005
1006export default class TestWorker extends Cloudflare.Worker<TestWorker>()(
1007 "TestWorker",
1008 {
1009 main: import.meta.url,
1010 },
1011 Effect.gen(function* () {
1012 const aiGateway = yield* Cloudflare.AI.QueryGateway(Gateway);
1013
1014 return {
1015 fetch: Effect.gen(function* () {
1016 const request = yield* HttpServerRequest;
1017 if (request.url.startsWith("/url")) {
1018 const url = yield* aiGateway.getUrl().pipe(Effect.orDie);
1019 return yield* HttpServerResponse.json({ url });
1020 }
1021 return HttpServerResponse.text("ok");
1022 }),
1023 };
1024 }).pipe(Effect.provide(Cloudflare.AI.QueryGatewayBinding)),
1025) {}
1027
1028## Test shape
1029
1030Compose a `Stack` that deploys the fixture, share one deploy across the file with `beforeAll`/`afterAll`, drive it via `HttpClient`, and retry the first request through edge propagation:
1031
1033// Service.test.ts
1034import * as Alchemy from "@/index.ts";
1035import * as Cloudflare from "@/Cloudflare";
1036import * as Test from "@alchemy.run/test/Alchemy";
1037import { expect } from "@alchemy.run/test";
1038import * as Effect from "effect/Effect";
1039import * as Schedule from "effect/Schedule";
1040import * as HttpClient from "effect/http/HttpClient";
1041import TestWorker from "./fixtures/worker.ts";
1042
1043const { test, beforeAll, afterAll, deploy, destroy } = Test.make({
1044 providers: Cloudflare.providers(),
1045});
1046
1047const Stack = Alchemy.Stack(
1048 "ServiceTestStack",
1049 { providers: Cloudflare.providers(), state: Cloudflare.state() },
1050 Effect.gen(function* () {
1051 const worker = yield* TestWorker;
1052 return { url: worker.url.as<string>() };
1053 }),
1054);
1055
1056const stack = beforeAll(deploy(Stack));
1057afterAll.skipIf(!!process.env.NO_DESTROY)(destroy(Stack));
1058
1059test(
1060 "deployed worker exercises the binding",
1061 Effect.gen(function* () {
1062 const { url } = yield* stack;
1063 const client = yield* HttpClient.HttpClient;
1064
1065 const res = yield* client.get(`${url}/url`).pipe(
1066 Effect.retry({ schedule: Schedule.exponential("500 millis"), times: 10 }),
1067 );
1068 expect(res.status).toBe(200);
1069 const body = (yield* res.json) as { url: string };
1070 expect(body.url).toContain("gateway.ai.cloudflare.com");
1071 }),
1072 { timeout: 180_000 },
1073);
1075
1076Notes:
1077
1078`Test.make({ providers: Cloudflare.providers() })` gives you `test`, `beforeAll`, `afterAll`, `deploy`, `destroy`.
1079`beforeAll(deploy(Stack))` returns a handle (`stack` above) that every `test` body can `yield*` to get the stack outputs.
1080`afterAll.skipIf(!!process.env.NO_DESTROY)(destroy(Stack))` is the standard cleanup — set `NO_DESTROY=1` locally to keep the deployment around between runs while iterating.
1081Always retry the first request (`Schedule.exponential("500 millis")`) — fresh workers.dev URLs and Lambda function URLs take a few seconds to start serving 200s.
1082For POST: use `client.post(url)` for empty bodies, or `HttpClient.execute(HttpClientRequest.post(url).pipe(HttpClientRequest.bodyJsonUnsafe(body)))` for typed bodies.
1083**Never use `while (Date.now() < deadline)` loops to poll** for an async side effect (a workflow status, a cron fire, a queue drain, eventual-consistency read, etc.). Use `Effect.repeat` with a `Schedule` and an `until` predicate so the polling participates in the Effect runtime — tracing, interruption, and error propagation work correctly, and the intent is declarative. Cap iterations with `times: N` (or a bounded schedule) so the test fails fast instead of running until the test timeout:
1084
1086 // good — declarative, bounded, interruption-safe
1087 const value = yield* fetchValue.pipe(
1088 Effect.repeat({
1089 schedule: Schedule.spaced("5 seconds"),
1090 until: (v) => v.ready,
1091 times: 36,
1092 }),
1093 );
1094
1095 // bad — opaque loop, ignores interruption, leaks into the test timeout
1096 let value: Value | undefined;
1097 const deadline = Date.now() + 180_000;
1098 while (Date.now() < deadline) {
1099 value = yield* fetchValue;
1100 if (value.ready) break;
1101 yield* Effect.sleep("5 seconds");
1102 }
1104
1105 See [CronEventSource.test.ts](./packages/integration-tests/test/Cloudflare/Workers/CronEventSource.test.ts) for a real-world example (polling a DO via the worker's `/times` route until the cron handler fires).
1106
1107## Reference implementations
1108
1109Cloudflare.AI.Gateway — [worker fixture](./packages/integration-tests/test/Cloudflare/AI/fixtures/TestWorker.ts) + [test](./packages/integration-tests/test/Cloudflare/AI/Gateway.test.ts) (the deploy+fetch case lives at the bottom of the file)
1110Cloudflare D1Connection — [worker fixture](./packages/integration-tests/test/Cloudflare/D1/fixtures/effect-worker.ts) + [test](./packages/integration-tests/test/Cloudflare/D1/D1Binding.test.ts)
1111Cloudflare Workflow — [workflow fixture](./packages/integration-tests/test/Cloudflare/Workers/fixtures/workflow/test-workflow.ts) + [worker fixture](./packages/integration-tests/test/Cloudflare/Workers/fixtures/workflow/workflow-worker.ts) + [test](./packages/integration-tests/test/Cloudflare/Workers/Workflow.test.ts)
1112Cloudflare Cron Trigger — [worker + DO fixture](./packages/integration-tests/test/Cloudflare/Workers/fixtures/cron/cron-worker.ts) + [test](./packages/integration-tests/test/Cloudflare/Workers/CronEventSource.test.ts) (cron handler writes to a DO; test polls a fetch route with `Effect.repeat` until the scheduled handler fires)
1113Cloudflare Images — [effect fixture](./packages/integration-tests/test/Cloudflare/Images/fixtures/effect-worker.ts) + [async fixture](./packages/integration-tests/test/Cloudflare/Images/fixtures/async-worker.ts) + [test](./packages/integration-tests/test/Cloudflare/Images/Images.test.ts)
1114AWS Lambda (DynamoDB bindings) — [Lambda fixture](./packages/integration-tests/test/AWS/DynamoDB/handler.ts) + [test](./packages/integration-tests/test/AWS/DynamoDB/Bindings.test.ts) (one `describe("<BindingName>")` per binding, all driving the same deployed Lambda)
1115
1116# Spec-Driven Service Bring-Up
1117
1118Use @processes/AWS.md as the source of truth for bringing a single AWS service from zero to full coverage.
1119
1120That process covers:
1121
1122deriving resources, bindings, event sources, and helpers from distilled
1123the audit-driven implementation loop
1124deterministic checks for registration and binding test coverage
1125Lambda fixture testing conventions
1126learned conventions like no auto-marshalling and one `describe("<BindingName>")` block per binding
1127
1128Keep `AGENTS.md` high-level and update @processes/AWS.md when the process evolves.
1129
1130When a canonical resource needs mutable event-source configuration and there is any chance of circularity, prefer a resource binding contract over a plain input prop. DynamoDB Streams is the reference case: `Table` owns the actual stream state, while `streams(table)` injects that state via bindings and the runtime-specific layer handles the subscription mechanics. See @processes/AWS.md for the DynamoDB Streams case study.
1131
1132# Build and Type Checking
1133
1134Always run type checking before committing changes:
1135
1137pnpm exec tsrs -b
1139
1140[`tsrs`](https://www.npmjs.com/package/tsrs) is a Rust port of the TypeScript 7 compiler, taking the same flags as `tsc` and running several times faster. `typescript` stays installed for compatibility: `pnpm exec tsc -b` is the reference check if you suspect a tsrs conformance bug.
1141
1142This runs the TypeScript compiler in build mode, which checks all projects in the workspace (including the distilled packages, which are project references). This is critical because CI will fail if there are type errors.
1143
1145pnpm typecheck # rebuild stale dist packages, then tsrs -b
1146pnpm ensure:built # rebuild cloudflare-runtime / frontend-frameworks if stale
1147pnpm build # in a package: builds it and its workspace dependencies
1149
1150## Running tests
1151
1152Tests run on **alchemy-test** (`@alchemy.run/test`, in `packages/alchemy-test`), our own single-process, Effect-native test runner. The CLI is vitest/bun-test compatible: positional paths (files or directories) and `-t` work the same way.
1153
1154There are two entry points, one per test package, both working identically from the repo root and from the package directory:
1155
1156`pnpm test <options>` — core suites in `packages/alchemy/test` (paths relative to `packages/alchemy`).
1157`pnpm test:integration <options>` — provider suites and core suites that deploy to a provider, in `packages/integration-tests/test` (paths relative to `packages/integration-tests`).
1158
1159Compose the flags you need — there are no per-variant package scripts. NOTE: `pnpm test`, not `bun test` — the latter invokes bun's own built-in test runner.
1160
1162# a suite, against the real cloud
1163pnpm test:integration test/Cloudflare/{Service}/{Resource}.test.ts --profile testing
1164
1165# a directory, filtered by test name
1166pnpm test:integration test/Cloudflare/Workers -t "cron" --profile testing
1167
1168# positional args that aren't real paths are file-name substring filters
1169pnpm test:integration Bucket --profile testing # every *Bucket* test file
1170
1171# skip the slow tests (replaces the old FAST=1 env prefix)
1172pnpm test --fast --profile testing
1173
1174# interactive TUI (humans only — never in an agent shell)
1175pnpm test --tui
1177
1178(`examples/` still use plain `bun test`.)
1179
1180### Cleaning leaked test resources
1181
1182If an interrupted live-cloud test leaves resources behind, do **not** add
1183test-specific API cleanup helpers, adoption workarounds, or alternate names to
1184make the test pass. Clean the testing account with the account-wide teardown
1185command, then rerun the failing test:
1186
1188pnpm nuke
1189pnpm clear:state --profile testing
1191
1192Lifecycle tests should continue to validate normal stack ownership and cleanup;
1193they must not silently adopt or directly delete leaked resources from earlier
1194runs.
1195
1196Additional flags beyond vitest:
1197
1198| Flag | Default | Purpose |
1199| ----------------- | ------- | ------------------------------------------------------------- |
1200| `-t <regex>` | — | Test-name pattern (regex, like bun/vitest) tested against the full nested title (`file > describes > name`), so any fragment matches regardless of nesting. An invalid regex degrades to a literal substring instead of erroring. Remember to escape regex metacharacters when filtering literally: `-t "create \(default\)"` |
1201| `--profile <name>` | — | Sets `ALCHEMY_PROFILE` before any test module is imported. Use `--profile testing` for live-cloud suites instead of an `ALCHEMY_PROFILE=testing` env prefix |
1202| `--stage <name>` | — | Sets `ALCHEMY_TEST_STAGE` before any test module is imported, replacing `Test.make`'s default stage for the run. The default is `test_$USER` in a main checkout and `test_$USER_<worktree>` in a linked git worktree, so two worktrees running the same suite never share deployments |
1203| `--fast` | off | Sets `FAST=1` before imports — suites `skipIf(process.env.FAST)` their slow tests (long-provisioning resources, smoke tests). Replaces the `FAST=1` env prefix |
1204| `--timeout <ms>` | 120000 | Default per-test timeout |
1205| `--retry <n>` | 2 | Re-runs of a failing test body (use `--retry 0` when debugging) |
1206| `--concurrency <n\|unbounded>` | 32 | Files running concurrently (one bun process, no forks). Bounded by default — unbounded saturates the event loop on large suites and produces spurious 0ms `beforeAll` timeouts |
1207| `--sequential` | off | Run tests within each file sequentially |
1208| `--tui` | off | Opt-in interactive TUI (default is line-per-test output) |
1209
1210Output behavior (plain mode, the default):
1211
1212The collection phase (importing every test file, ~45s for the full suite) reports `collecting N/TOTAL test files (elapsed) <current file>` — repainted in place on a TTY, printed every 5s otherwise. A run that appears stuck before any test starts is almost always just collecting; if it really is stuck, the line names the file whose import hangs.
1213One line per test as it finishes, prefixed with a `[done/total]` progress counter so the remaining count is visible; a **failing test prints its error and captured output inline** immediately.
1214Passing tests' output is swallowed on the console — but **every** test's output (passes included) is streamed to a per-run log at **`.alchemy/log/test/{timestamp}-pid{pid}.log`** (relative to the cwd), so concurrent runs in different terminals never trample each other. The absolute path (with line/KB counts) is printed at the end of every run (`Full log: …`) — read that file when you need the complete record, e.g. a passing test's deploy output or a hang's partial log. The file is appended live, so it's readable mid-run; logs older than a week are pruned automatically (by stat mtime).
1215If nothing finishes for 10s, the runner prints the list of currently-running tests with elapsed times — the first place to look when a run seems hung.
1216Exit code is non-zero if any test failed.
1217
1218Runner semantics to know:
1219
1220Everything runs in ONE bun process: files run concurrently (respecting `describe.sequential`), imports of all test files happen up-front (they must be lazy and pure — registration only, no top-level side effects beyond `Test.make`/`describe`/`test`).
1221Tests that mutate process-global state (e.g. `process.env.PATH`, `process.chdir`, `globalThis.__ALCHEMY_RUNTIME__`) must pass `{ exclusive: true }` in the test options to take the whole-process write lock. Tests that only contend for a shared cloud resource (an account singleton, a quota) pass a label instead — `{ exclusive: "aws:ses-account" }` serializes them with other holders of that label while the rest of the run continues. The same option works on `beforeAll`/`afterAll` hooks.
1222The harness (`@alchemy.run/test` package) provides `describe`, `it`/`test` (incl. `it.effect`/`it.live`), hooks, `layer`, `expect`, and `assert`.
1223The runner runs in plain bun, so distilled resolves from `src/*.ts` via the `bun` export condition — a regenerated service is test-visible immediately, no `lib/` rebuild required.
1224
1225## The convergence loop: nuke → test → census → fix-fleet
1226
1227Ironing out the AWS suite is an iterative loop, driven by a coordinator, that terminates only when the suite is green AND the account is clean for **two consecutive rounds**. "Green" alone is not the bar — a passing test that leaves cloud resources behind is a provider bug by definition.
1228
1229Each iteration:
1230
1231**Clean slate** — first run `aws sso login` (the alchemy `testing` profile and the raw `aws` CLI ride the same SSO session; an expired token mid-round breaks the pipeline with auth errors — only escalate to a human if the login doesn't complete automatically). Then `pnpm nuke --yes` (deletes every alchemy-tagged cloud resource; `scripts/nuke.sh` already spares state buckets, SSO roles, and AWS-managed singletons) then `pnpm -C stacks alchemy state delete Nuke --config ./src/nuke.ts --profile testing --recursive`. Never overlap nuke with a running suite. Plain `pnpm clear:state` lacks the profile and dies on expired Cloudflare OAuth; nuke without `--yes` hangs on an interactive confirm in non-interactive shells.
1232**Full suite, bounded** — `pnpm test:integration test/AWS --profile testing`. The runner defaults to `--concurrency 32`; NEVER override it to `unbounded` on a full-suite run: all ~775 files' `beforeAll` deploys start at once, the event loop saturates, and hundreds of fake 0ms `beforeAll TimeoutError` failures drown the real signal (heap is ~8.5 GB regardless of N — the constraint is CPU, not memory). Target ≤10 min wall-clock, hard cap 128; measured 32 → ~21 min clean. The saturation tell is `beforeAll` failures at 0ms; real failures fail slow. If the cap can't reach 10 min, the residual is individual slow files — skipIf-gate them per the speed doctrine.
1233**Leak census** — `pnpm nuke --dry-run` after the suite; diff against the pre-suite baseline. Worklist = **failed services ∪ leaking services** (a service can pass green and still leak). Leave the leaked resources LIVE as forensic evidence for the fix agents; carry-over holdouts that survive repeated nuke passes (stuck deletes) go on the worklist too — their delete path is the bug.
1234**Fix-fleet workflow** — one agent per service on the worklist (account-singleton services — CloudTrail, Config, SecurityHub, GuardDuty, ControlTower, IdentityCenter — run as a sequential chain; everything else fans out). Each agent gets its exact failures, its leak inventory, and this root-cause priority: **provider bug > distilled patch > test fix** — never paper over a provider leak in the test. Each agent runs ONLY its own suite (`pnpm test:integration test/AWS/{Service} --profile testing`), audits its tests for non-deterministic names (rely on PhysicalName auto-naming; random data in message payloads/idempotency tokens is fine), verifies zero orphans from its service via out-of-band distilled list/describe calls, and reports a structured result. Agents never run tsc/build and never run the account-wide nuke.
1235**Gate** — the coordinator runs the one-shot `pnpm exec tsrs -b`, fixes cross-cutting fallout, commits the iteration (+ distilled submodule bump when patches were made), and loops back to 0. Terminate on two consecutive iterations of green-suite + census reduced to documented-undeletable residue (e.g. BackupSearch terminal records, Contributor-Insights rules with no delete API, keys in scheduled deletion).
1236
1237## Multi-agent sessions: the coordinator owns type-checking
1238
1239When many agents work concurrently (see **The Resource Factory Process**), do NOT let each agent run `pnpm exec tsrs -b` — concurrent runs thrash the machine and race on `tsbuildinfo`. Agents never invoke the compiler; the coordinator runs a one-shot `pnpm exec tsrs -b` at wave boundaries as the authoritative type check.
1240
1241## Build Commands
1242
1243| Command | Description |
1244| ----------------- | -------------------------------------------------------------------------------------------- |
1245| `pnpm exec tsrs -b` | Type check all projects (always run before committing) |
1246| `pnpm build` | Clean, type check, and build the alchemy package |
1247| `pnpm build:clean` | Full clean rebuild: cleans all artifacts, reinstalls dependencies, builds, and downloads env |
1248
1249Use `pnpm build:clean` when you encounter stale build artifacts or dependency issues. It runs:
1250
1251`pnpm clean .` - Removes all untracked files except .env
1252`pnpm install` - Reinstalls dependencies
1253`pnpm build` - Builds the project
1254`pnpm download:env` - Downloads environment files
1255
1256# Tutorial Documentation Standard
1257
1258Tutorials under `website/src/content/docs/tutorial/` are **step-by-step and granular**: every code snippet introduces exactly **one** new thing, followed by a short prose explanation of just that thing. Each step gets its own `##` heading.
1259
1260**Anti-pattern** — one snippet that adds multiple distinct changes, followed by a numbered list or bullet list explaining each:
1261
1263## Bind the DO to the Worker
1264
1266+import Counter from "./counter.ts";
1267+import { HttpServerRequest } from "...";
1268
1269 Effect.gen(function* () {
1270+ const counters = yield* Counter;
1271 return {
1272 fetch: Effect.gen(function* () {
1273+ const request = yield* HttpServerRequest;
1274+ if (request.url.startsWith("/counter/") && ...) {
1275+ const next = yield* counters.getByName(name).increment();
1276+ return HttpServerResponse.text(String(next));
1277+ }
1278 return HttpServerResponse.text("Hello!");
1279 }),
1280 };
1281 })
1283
1284Two things just happened:
1285`yield* Counter` registers the DO ...
1286`counters.getByName(name)` returns a typed stub ...
1288
1289**Correct** — split into one heading per step, each with one snippet and one explanation:
1290
1292## Bind the DO to the Worker
1293
1295+import Counter from "./counter.ts";
1296
1297 Effect.gen(function* () {
1298+ const counters = yield* Counter;
1299 ...
1300 })
1302
1303`yield* Counter` registers the DO with the Worker (binding + class-migration metadata) and hands you the namespace.
1304
1305## Call the DO from `fetch`
1306
1308+import { HttpServerRequest } from "...";
1309
1310 fetch: Effect.gen(function* () {
1311+ const request = yield* HttpServerRequest;
1312+ if (request.url.startsWith("/counter/") && ...) {
1313+ const next = yield* counters.getByName(name).increment();
1314+ return HttpServerResponse.text(String(next));
1315+ }
1316 return HttpServerResponse.text("Hello!");
1317 })
1319
1320`counters.getByName(name)` returns a typed stub — `increment()` and `get()` round-trip through Cloudflare's RPC machinery.
1322
1323Rules of thumb:
1324
1325If you find yourself writing "Two/three things just happened", "A few things are happening here", or a numbered/bulleted list explaining separate parts of a single snippet — **split the snippet**.
1326One concept ⇒ one heading ⇒ one diff snippet ⇒ one explanation paragraph (no bullets).
1327Bullet/numbered lists are fine when they describe a recap, prerequisites, or genuinely list-shaped content (e.g. "the Worker now handles two routes: PUT and GET" at the end). They are **not** fine as a substitute for splitting a compound snippet.
1328A single API call that internally does several things (e.g. `Cloudflare.upgrade()`) doesn't need splitting — describe its behavior in prose.
1329Use `diff lang="typescript"` blocks so each step shows what's added on top of the previous step.
1330
1331# Package Skills
1332
1333A package can ship an agent skill under `packages/{package}/skills/{name}/` (e.g. [alchemy-domain](./packages/domain/skills/alchemy-domain/SKILL.md)): a `SKILL.md` plus `references/*.md`. The skill is an **entrypoint**, not a second copy of the docs. It gives an agent the workflow, the rules, and a short template per topic. The website holds the depth.
1334
1335**`SKILL.md` links the docs hub and `https://alchemy.run/llms.txt`**, which lists every page, so an agent can find what the skill leaves out.
1336**Every `##`/`###` section ends with a `More:` line** linking the docs page, or the exact `#section`, that covers it in depth:
1337
1339 More: [Saga: Undo a step when a later one fails](https://alchemy.run/domain/saga#undo-a-step-when-a-later-one-fails).
1341
1342**Links must resolve.** Use absolute `https://alchemy.run/...` URLs (the skill is read outside the website), not `website/...` paths. After `pnpm --filter website build`, check that each linked page exists under `website/dist/` and that each `#anchor` is an `id` on it.
1343**Keep the skill in step with the docs.** Renaming a docs page or heading means updating the skill links to it. Adding a docs page a skill covers means adding a `More:` link from its section.
1344**Keep sections short.** If a section grows into an explanation, move the explanation to the docs page and link it.
1345
1346# Pull Request Conventions
1347
1348When you automatically open a PR, it MUST follow this structure:
1349
1350**Title**: Use conventional commit format (e.g. `fix(website): mobile theme metas`, `feat(aws/s3): add bucket lifecycle rules`).
1351**Description heading levels**: NEVER use `#` or `##` in the PR description. The smallest heading allowed is `###`. The PR description must NOT begin with its own title heading — GitHub already renders the PR title above it.
1352**Content**: Aim for the minimal content needed to convey the idea.
1353Use simple sentences. If there are multiple discrete changes, use bullet points.
1354**Prefer code snippets over prose.** A short ` ```ts ` or ` ```diff ` block showing the new/changed shape is worth more than a paragraph explaining it. Reach for code first; only add prose to fill in the "why" the snippet can't show on its own.
1355Be direct and succinct. Cut adjectives, justifications, and anything that reads like marketing copy. If a sentence is restating what the diff already shows, delete it.
1356**Never include a "Test plan", "Testing", or checklist of TODOs.** PR descriptions document the change, not the verification process. If something needs manual verification, follow the draft-PR rule below.
1357Skip examples for trivial fixes, internal refactors, or doc-only changes.
1358
1359Example PR description (good — code snippet does the talking):
1360
1363
1364```ts
1365// every Layer.effect(State, …) site now wraps construction:
1366makeLocalState().pipe(recordStateStoreInit("local"))
1367```
1368
1369Dashboard groups projects by kind from these spans (Axiom can't APL-query metric datasets).
1371**Outstanding work / testing / review needed**: If there are outstanding steps, manual testing required, or review items, DO NOT leave a comment on the PR and DO NOT include them in the PR description. Instead:
1372Mark the PR as **draft**.
1373Tell the user (in the chat that initiated the PR creation) what is outstanding.
1374
1375:::warning
1376**Markdown content must reach GitHub verbatim** — un-escaped backticks, fenced code blocks, etc. The reliable shape is to write the description to a file and pass `--body-file <path>` to `gh pr create` / `gh pr edit`:
1377
1379# write the body to a temp file (use Write tool, not echo/cat heredoc)
1380gh pr edit 179 --body-file /tmp/pr-body.md
1382
1383Do **not** inline the body via `--body "$(cat <<'EOF' ... EOF)"`. Even with a single-quoted heredoc some shells / `gh` versions still mangle backticks and backslashes; the resulting PR body ends up with literal `\`` sequences instead of inline code spans. `--body-file` sidesteps shell quoting entirely.
1384
1385If you need to update an already-created PR's body, prefer `gh pr edit --body-file ...`. If that silently no-ops (older `gh` versions), fall back to `gh api -X PATCH repos/<owner>/<repo>/pulls/<n> -F body=@/tmp/pr-body.md`.
1386:::
1387
1388The summary goes at the very top of the description as plain prose — NO heading above it, no `### Summary`, nothing. The PR title already serves as the title; do not repeat or re-title it. Only add `###` subheadings further down if the description genuinely has multiple sections worth separating.
1389
1390Example PR description (good):
1391
1394
1395- Read theme from `localStorage` on mount before first paint
1396- Add `<meta name="theme-color">` per theme so mobile chrome matches
1398
1399Example PR description (BAD — do not do this):
1400
1403### Summary ← no, summary needs no heading
1404Persist the user's theme...
1406
1407# Blog / Release Notes Conventions
1408
1409Release blog posts live in `website/src/content/docs/blog/` named
1410`YYYY-MM-DD-beta-NN.md` (date = the release date).
1411
1412**Frontmatter `title` format:** `<version> - <short title>`, e.g.
1413`2.0.0-beta.45 - Config & RPC Workers`. The title renders in the
1414blog TOC sidebar, so the descriptive suffix must be short — a few
1415words that fit neatly on one line. Lead with the version so the
1416list stays sorted and scannable.
1417
1418Writing style (match the existing beta posts, e.g.
1419[beta.41](./website/src/content/docs/blog/2026-05-20-beta-41.md),
1420[beta.44](./website/src/content/docs/blog/2026-05-22-beta-44.md)):
1421
1422**Lean, concise, zero-fluff.** Illustrate each new
1423 feature/fix and link to the relevant docs/tutorials/guides.
1424Read each PR in the release's changelog to understand what
1425 actually changed before writing.
1426Lead with the headline features (one `##` heading each, with a
1427 short code snippet or `diff`), then fold the long tail into an
1428 `## Also in this release` bullet list.
1429Put breaking changes in a `:::caution` callout at the top.
1430Cite PRs inline as `([#NNN](…/pull/NNN))` and credit external
1431 contributors by name.
1432End with a `## Where to go next` list of doc links plus the
1433 CHANGELOG and compare links.