Skip to content
Source
Sign inSign up
TanStack/db

browser-db-sqlite-persistence: option to load the wa-sqlite WASM by URL instead of inlined base64 #2101

Send to my agent
Closedlloydclosson(GitHub profile, opens in a new tab) opened this issue 4 comments

What

@tanstack/browser-db-sqlite-persistence (seen on 0.2.30) ships its OPFS worker prebuilt as dist/assets/opfs-worker-*.js, with wa-sqlite's WebAssembly inlined as a data:application/wasm;base64,… URL. That makes the worker about 1.7 MB raw (about 573 KB brotli). Most of that is the base64 WASM, which compresses worse than the binary and cannot be compiled by WebAssembly.instantiateStreaming.

src/opfs-worker.ts calls SQLiteESMFactory() with no options, and the package offers no option to pass a locateFile, a wasmBinary or a worker of one's own.

Ask

An option for where the WASM comes from. Any of these would do:

  • a wasmUrl (or locateFile) option on openBrowserWASQLiteOPFSDatabase, passed to the worker and on to SQLiteESMFactory({ locateFile });
  • a createWorker option, so an app can build the worker from source with its own bundler;
  • or ship the worker without inlining the WASM, since @journeyapps/wa-sqlite already resolves new URL("wa-sqlite.wasm", import.meta.url), which Vite, Rollup and webpack all emit as a separate asset.

What we do meanwhile

We patch dist/esm/opfs-worker.js so the consumer's Vite build makes the worker from the package's own source:

function WorkerWrapper(options) {
  return new Worker(new URL("../../src/opfs-worker.ts", import.meta.url), {
    type: "module",
    name: options?.name
  });
}

With Vite 8 that gives an 88 KB worker plus a separate 1,129,661 B wa-sqlite-*.wasm: 468 KB brotli in total instead of 573 KB, compiled by streaming. It works in dev too, because Vite's dep optimizer rewrites the worker URL. We would like to drop the patch once the package has an option for it.

  1. Could you test out https://github.com/TanStack/db/pull/2103 to see if it just works for y'all? I'd rather not add a option unless we really need to.

  2. @lloydclosson lemme know when you have a chance to test it — we're doing the first DB 1.0 RC soon so I'd like to make this change before then if possible

  3. I ran into this too, and wanted to post another use case for this.

    Use case: strict CSP. Because the worker embeds the WASM as a data: URI, our app has to allow connect-src 'self' data: just so Emscripten can fetch it. Without data:, the open fails in the worker with:

    failed to asynchronously prepare wasm: NetworkError: Failed to execute 'send' on 'XMLHttpRequest': Failed to load 'data:application/wasm;base64,…'

    Test of #2103 (head 0bca94e). I built the package from the PR branch and dropped it into our app (React 19, Vite production build, Chromium) in place of 0.2.32. The app runs under an enforced CSP of connect-src 'self' (no data:) and script-src 'self' 'wasm-unsafe-eval'.

    • Without the PR, five of our offline-durability e2e specs fail, all on the error above.
    • With the PR, all 16 specs pass. They cover reload-and-replay of offline edits and reading the OPFS mirror while the API is down.
    • Vite picked up the separate assets automatically with no build changes needed on our side. Now it emits a 209 kB worker (down from 1.7 MB) and a hashed wa-sqlite-*.wasm alongside it.

    We'd drop data: from our CSP once this ships. Thanks!

  4. 🎉 The fix for this issue has been released!

    Fixing PRs: https://github.com/TanStack/db/pull/2103

    Backfilled notification for published release candidates.

Sign in to join this conversation.