tombeckenham/hyperframes-tanstack-cloudflare
| Name | Last commit message | Last commit date |
|---|---|---|
README.md
Hyperframes Web#
Chat with an AI agent that makes videos for you — live, in your browser, on your own Cloudflare account.
Describe a video ("a 10-second launch teaser for my app", "explain how jet engines work") and a coding agent authors a HeyGen HyperFrames HTML video composition inside a Cloudflare Sandbox container, orchestrated by TanStack AI. You watch it take shape in a live player, open the full editing studio in another window, and render the result to MP4 — all served by one Cloudflare Worker.
Powered by#
- HeyGen HyperFrames — the video
framework doing the actual work: compositions are plain HTML + GSAP
timelines, previewed in a browser studio, linted and rendered
deterministically to MP4 by the
hyperframesCLI. If an agent can write HTML, it can make video. - TanStack AI — the agent plumbing:
useChaton the client, typed tool definitions on the server, and the@tanstack/ai-sandbox-cloudflaretopology this app forks — a Durable Object coordinator driving a coding agent that lives inside a sandbox, with host tools bridged over MCP. - Cloudflare Sandboxes —
full Linux containers addressable from a Worker: the coding-agent CLI
(
grokorclaude), Node, Chromium, and ffmpeg run there, per-thread, with exec/file APIs and quick tunnels back out. Plus TanStack Start for the app shell and shadcn/Base UI for the chat components.
What you'll see#
- Chat — every message is a video brief. One-click demo briefs (product teaser, kinetic quote, logo sting…) skip the interview and start authoring immediately.
- Player tab — the thread's live composition, before any render, with transport controls and frame-accurate scrubbing. It refreshes itself every time the agent finishes a turn, and opens on a preset 5-second intro baked into the sandbox image so there's something to watch from the first page load.
- Renders tab — the MP4s and bundled compositions the agent has published, straight out of R2.
- Hyperframes Studio (header button) — the sandbox's full
hyperframes previewediting studio, opened in its own window through a Cloudflare quick tunnel.
What you need#
- A Cloudflare Workers Paid plan — Cloudflare Containers require it.
- An agent API key — set
XAI_API_KEYorANTHROPIC_API_KEY:- xAI (
XAI_API_KEY) → Grok Build (wins if both set) - Anthropic (
ANTHROPIC_API_KEY) → Claude Code The sandbox image ships both CLIs; selection is automatic (seesrc/lib/harness.ts).
- xAI (
- For local development only: Bun and Docker Desktop (the Cloudflare Vite plugin builds and runs the sandbox container locally; OrbStack cannot run Cloudflare containers).
Deploy it#
One click#
- Hit Deploy to Cloudflare above. Cloudflare clones this repo into your GitHub/GitLab account, provisions the R2 bucket, sets up CI (Workers Builds), and deploys. You can rename the Worker, repo, and bucket on the setup page.
- Add an agent key — the one thing the button can't do for you.
In the Cloudflare dashboard: your Worker → Settings → Variables and
Secrets → add
XAI_API_KEY(Grok Build) orANTHROPIC_API_KEY(Claude Code). From a checkout:bunx wrangler secret put XAI_API_KEY(orANTHROPIC_API_KEY). Without one, deploys succeed but every agent run fails at startup with a clear error. If both are set, Grok Build wins. - Give the container a few minutes. The sandbox image finishes
provisioning → readyafter the Worker deploy. When the studio loads, the Player should show the intro animation — that's your proof the container is up.
From then on, every push to main on your new repo builds and deploys
automatically.
From a checkout#
git clone <your fork> && cd hyperframes-tanstack-cloudflare
bun install
bunx wrangler r2 bucket create hyperframes-studio-renders # match wrangler.jsonc
bunx wrangler secret put XAI_API_KEY # Grok Build (wins if both set)
# or: bunx wrangler secret put ANTHROPIC_API_KEY # Claude Code
bun run deploy # build + wrangler deploy
Want CI without the button? Connect the repo to
Workers Builds
(Worker → Settings → Builds) — the Worker name in the dashboard must
match name in wrangler.jsonc.
Renaming things#
Everything configurable lives in wrangler.jsonc: the Worker name, the R2
bucket_name (must match the bucket you create), and the container
instance_type (default standard-4; renders are CPU- and memory-hungry).
Two pipeline facts worth knowing#
- Branch builds do not build the container. They run
wrangler versions upload, which skips the docker image entirely — a green branch build proves nothing about the Dockerfile. Image problems only surface on themainbuild. - Container rollout lags the deploy by a few minutes, and existing chat threads stay pinned to containers from the old image until they recycle. Test image changes with a fresh thread.
Run it locally#
bun install
cp .dev.vars.example .dev.vars # set XAI_API_KEY or ANTHROPIC_API_KEY
bun run dev # http://localhost:3001 — Docker Desktop must be running
No tunnel or public hostname is needed locally: the container reaches the
Worker at host.docker.internal:3001, and previews come through quick
tunnels.
Commands#
bun run dev # vite dev on :3001 — Worker + DOs + container in workerd
bun run build # also regenerates src/routeTree.gen.ts (commit it)
bun run deploy # bun run build && wrangler deploy
bun run typecheck # tsc --noEmit
bun run test # bun test
bun run lint # oxlint
bun run format # oxfmt
bun run cf-typegen # wrangler types (after editing wrangler.jsonc)
Port map#
| Port | Owner |
|---|---|
| 3000 | Reserved by the sandbox control plane — never bind it |
| 3001 | The Worker dev server (bun run dev) |
| 3002 | hyperframes preview studio inside the sandbox |
| 8787 | wrangler dev (if used directly) |
How it works#
Browser ──POST /api/run──▶ Start server route ──DO RPC──▶ RunCoordinator DO ──▶ Sandbox container
▲ (SSE bridge) (drives chat()) (grok/claude CLI +
└──────── SSE ─────────────────┘ hyperframes CLI)
- The browser chats through
POST /api/run, which bridgesuseChat's SSE expectation to the coordinator's POST-then-WebSocket run protocol — over theRUN_COORDINATORbinding, never a self-fetch. - The
RunCoordinatorDurable Object owns each run: it drives the agent loop underctx.waitUntil, appends every chunk to a durable,seq-indexed log, and serves resumable WebSocket tails. No Worker invocation ever holds a multi-minute agent loop. Harness selection is automatic:XAI_API_KEY→ Grok Build (wins if both set), elseANTHROPIC_API_KEY→ Claude Code (src/lib/harness.ts). - The sandbox container carries both coding-agent CLIs (
grokandclaude) and the whole HyperFrames toolchain (Node 22, Chromium, ffmpeg), plus a ready-made project at/workspace/studiothe agent authors in — its running preview hot-reloads straight into what the user is watching. Host tools — preview tunnels, R2 publishing, the authoring recipe — are served to the in-sandbox agent over MCP at/_bridge/:runId, gated by a per-run bearer token. - The preview sandbox boots on page load (
POST /api/preview"ensure"), its tunnel is health-polled without waking a sleeping container ("probe"), and a stopped sandbox surfaces as an explicit Resume button rather than a dead pane. - The Player plays the live composition through
GET /api/player, a same-origin proxy of the preview server's bundled-composition route — same-origin because the player drives compositions viacontentWindow, which a cross-origin tunnel URL forbids. - Published compositions and MP4 renders live in R2, served by
/p/*and/r/*.
When something breaks#
- "the tanstack MCP server hasn't come up" — almost always a 404 on the
container's callback to
/_bridge, meaning the bridge origin is wrong. Locally: keepserver.host: trueinvite.config.ts. Deployed: the origin derives from the request host;PUBLIC_HOSTNAMEis an override only. - Player says "the preview has stopped" — the sandbox slept after inactivity. Hit Resume; published compositions and renders are safe, anything unpublished lived on the container's ephemeral disk.
- Quick-tunnel hostnames may not resolve through your local DNS — fresh
*.trycloudflare.comnames can be invisible to ISP/VPN resolvers while the browser (secure DNS) sees them fine. That is why tunnel verification runs inside the container and the health probe falls back to DNS-over-HTTPS before declaring a tunnel dead. - The hyperframes CLI ships a detached auto-updater that spawns a
background
npm install -gand exits without waiting — lethal inside a docker build and wrong in a version-pinned image. The Dockerfile disables it (HYPERFRAMES_NO_AUTO_INSTALL=1) before the CLI's first invocation. Five consecutive deploys failed before this was found. - Previews break under Vite if switched to
exposePort+ hostname proxying — the dev middleware hijacks/@vite/client. Quick tunnels avoid the whole class; leave them be. - Tunnels require
SANDBOX_TRANSPORT: "rpc"(set inwrangler.jsonc) — on the defaulthttptransportsandbox.tunnelsthrows.
Security model (short version)#
- A visitor's
threadIdisSHA-256(sessionId ‖ threadKey)— derived server-side from an HttpOnly cookie, never taken from the request body. The package's unauthenticated/runstrigger is deliberately not routed. - Containers are pinned per-thread and carry the selected harness API key, which is why the two lines above are load-bearing.
- Published, shareable LLM-authored HTML is served from
/p/*withContent-Security-Policy: sandbox allow-scripts— an opaque origin, no cookies, no same-origin fetch. - The live Player composition (
/api/player) is served same-origin without that sandbox — a deliberate, documented trade: the player needscontentWindowaccess, and the route only ever serves a composition to the session cookie that owns the thread which authored it. - The R2 bucket mixes public artifacts with single-use upload tickets; every public route allowlists exactly one key prefix.