feat(PowerSync): add attachments support #1616
main fromrefs/pull/1616/head·opened 🎯 Changes#
Derived from Steven's efforts in https://github.com/powersync-ja/powersync-js/pull/983, and addresses https://github.com/TanStack/db/discussions/1563.
Problem#
PowerSync ships an attachment helper for syncing files (photos, documents) between local and remote storage. It's separate from regular synced tables: a local-only attachments table tracks each file's lifecycle (QUEUED_UPLOAD, SYNCED, QUEUED_DELETE), and an AttachmentQueue drives uploads/downloads in the background.
TanStackDB, on the other hand, gives you an optimistic, reactive, joinable view over synced data. For users who want to use the attachment helper alongside the PowerSync+TanstackDB integration there are blockers. Saving a file (in the local-only attachments table) and associating it with a record (e.g. setting user.photo_id) are two independent writes which could make data races and fatal errors a problem for data consistency.
The original POC (powersync-js#983) proved this integration was viable. This PR productionises a a subset of it as reusable functionality.
Solution#
A TanStackDBAttachmentQueue that extends the SDK's AttachmentQueue (for saving and deleting a file) and backs it with a TanStack DB collection.
The package owns the collection-backed saveFile/delete implementation and leaves the wiring to the application (covered in documentation).
✅ Checklist#
- I have tested this code locally with
pnpm test.
🚀 Release Impact#
- This change affects published code, and I have generated a changeset.
Summary by CodeRabbit#
- New Features
- Added attachment support for PowerSync DB collections through a TanStack DB-backed attachment queue.
- Supports atomic saves and deletes, custom attachment IDs, related-record updates, and local attachment URLs.
- Added rollback and cleanup handling when attachment operations fail.
- Documentation
- Added setup and usage guidance for attachment collections, queues, watching attachments, and local URLs.
- Tests
- Added coverage for attachment lifecycle, synchronization, cleanup, custom IDs, concurrent saves, and transaction rollback.
- 1
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. Inline comments: In @.changeset/curly-planets-lead.md: - Around line 1-3: The changeset for `@tanstack/powersync-db-collection` is labeled patch, but this release introduces a new public export and raises the `@powersync/common` peer minimum, so update the changeset in curly-planets-lead.md to at least minor (or the appropriate breaking level per your policy). Keep the package name the same and change only the release type so the generated release notes reflect the expanded surface area and peer dependency bump. In `@docs/collections/powersync-collection.md`: - Around line 1195-1207: The watcher example in the referenced attachment-ID flow is fire-and-forgetting onUpdate, which can let overlapping updates resolve out of order. Update the example around livePhotoIds.stateWhenReady, livePhotoIds.subscribeChanges, and the onUpdate callback to serialize each async invocation by awaiting the previous one (for example through a chained promise or equivalent queue) so the initial state and later change notifications are processed in order. Also mirror the same awaited pattern in the test helper that currently reproduces this race. In `@packages/powersync-db-collection/src/attachments.ts`: - Around line 79-80: The `addAttachment` flow in `Attachments` writes the blob with `localStorage.saveFile()` before the transaction, but failures in `insert`, `updateHook`, or `commit()` leave the file orphaned on disk. Update the attachment write path to track the just-saved local URI and delete it on every exception path, including transaction rollback and any thrown hook/commit error, while keeping successful saves intact. Also add a regression test around `updateHook` that throws to verify the DB changes roll back and the local file is cleaned up.🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID:
9f1f73be-698d-4312-886c-aeb36c8b4f13📥 Commits
Reviewing files that changed from the base of the PR and between 45617c45b761ee32d6cacc1a4ee9e653a27680dd and cd8b19117900e68212ecde4b5577fd104695a343.
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (6)
.changeset/curly-planets-lead.mddocs/collections/powersync-collection.mdpackages/powersync-db-collection/package.jsonpackages/powersync-db-collection/src/attachments.tspackages/powersync-db-collection/src/index.tspackages/powersync-db-collection/tests/attachments.test.ts
@@ -0,0 +1,3 @@1+---2+'@tanstack/powersync-db-collection': patch3+---📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
patchis too small for this release.This PR adds a new public export and raises the package's
@powersync/commonpeer minimum from^1.41.0to^1.57.0. Shipping that as a patch hides a new surface area and can break consumers that have not upgraded their PowerSync peer yet. Please bump this changeset to at leastminor(or whatever level your peer-dependency policy treats as breaking).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.changeset/curly-planets-lead.md around lines 1 - 3, The changeset for `@tanstack/powersync-db-collection` is labeled patch, but this release introduces a new public export and raises the `@powersync/common` peer minimum, so update the changeset in curly-planets-lead.md to at least minor (or the appropriate breaking level per your policy). Keep the package name the same and change only the release type so the generated release notes reflect the expanded surface area and peer dependency bump.✅ Addressed in commits dc7f13e to 2718e1b
@@ -1101,0 +1187,21 @@1187+ )1188+1189+ const mapper = (item) =>1190+ ({1191+ id: item.photo_id,1192+ fileExtension: "jpg",1193+ }) satisfies WatchedAttachmentItem1194+1195+ // 1. Report the initial set of referenced attachment IDs.1196+ const initialState = await livePhotoIds.stateWhenReady()1197+ onUpdate(Array.from(initialState.values()).map(mapper))1198+1199+ // 2. Re-emit the whole set on every change (the queue expects the holistic state).1200+ livePhotoIds.subscribeChanges(() => {1201+ onUpdate(livePhotoIds.map(mapper))1202+ })1203+1204+ // 3. Clean up when sync stops.1205+ abortSignal.addEventListener("abort", () => livePhotoIds.cleanup(), {1206+ once: true,1207+ })🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Serialize the async
onUpdatecalls in this example.The watcher example fire-and-forgets
onUpdatefor both the initial state and subsequent changes. If two updates arrive close together, those promises can resolve out of order and stale attachment sets can win. Please show an awaited/serialized pattern here; the test helper currently mirrors the same race.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/collections/powersync-collection.md` around lines 1195 - 1207, The watcher example in the referenced attachment-ID flow is fire-and-forgetting onUpdate, which can let overlapping updates resolve out of order. Update the example around livePhotoIds.stateWhenReady, livePhotoIds.subscribeChanges, and the onUpdate callback to serialize each async invocation by awaiting the previous one (for example through a chained promise or equivalent queue) so the initial state and later change notifications are processed in order. Also mirror the same awaited pattern in the test helper that currently reproduces this race.@@ -0,0 +60,21 @@60+ this.collection = params.attachmentsCollection61+ }62+63+ /**64+ * Saves a file to local storage and queues it for upload to remote storage.65+ *66+ * Exposes an `updateHook` option which is called inside a TanStackDB transaction,67+ * relational associations with the provided attachment ID should be made in this hook.68+ */69+ async save({70+ data,71+ fileExtension,72+ mediaType,73+ metaData,74+ id,75+ updateHook,76+ }: SaveOptions): Promise<AttachmentQueueRow> {77+ const resolvedId = id ?? (await this.generateAttachmentId())78+ const filename = `${resolvedId}.${fileExtension}`79+ const localUri = this.localStorage.getLocalUri(filename)80+ const size = await this.localStorage.saveFile(localUri, data)🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Roll back the saved file when the transaction fails.
saveFile()runs before the collection transaction starts. Ifinsert,updateHook, orcommit()throws, the DB work is rolled back but the blob stays on disk, so this API is no longer atomic and leaks orphaned local files. Please clean up the just-written file on every failure path and add a regression test that throws fromupdateHook.Also applies to: 99-116
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/powersync-db-collection/src/attachments.ts` around lines 79 - 80, The `addAttachment` flow in `Attachments` writes the blob with `localStorage.saveFile()` before the transaction, but failures in `insert`, `updateHook`, or `commit()` leave the file orphaned on disk. Update the attachment write path to track the just-saved local URI and delete it on every exception path, including transaction rollback and any thrown hook/commit error, while keeping successful saves intact. Also add a regression test around `updateHook` that throws to verify the DB changes roll back and the local file is cleaned up.✅ Addressed in commits dc7f13e to 2718e1b
@samwillis - could you review this one please?
I ran the exact package suite locally and all 92 tests pass, but I still need a few correctness fixes before merging this:
saveFile()runs before the collection transaction, so a failed insert, hook, or commit leaves an orphaned local file. Please clean it up on failure and add a regression test.- The docs show async
updateHookcallbacks, but the hook is called inside synchronoustransaction.mutate()and its promise is ignored. Any mutation after anawaitescapes the transaction. The contract and examples need to be explicitly synchronous, or the implementation needs an actually awaited atomic design. watchAttachmentscalls need to be serialized so an older async update can't win after a newer one.- This adds public API and raises the PowerSync peer floor from 1.41 to 1.57, so the changeset should be
minor, notpatch.
The earlier review threads are marked resolved, but the current diff still has these behaviors.
Thanks for the feedback!
saveFile()runs before the collection transaction, so a failed insert, hook, or commit leaves an orphaned local file. Please clean it up on failure and add a regression test.
Done.
- The docs show async
updateHookcallbacks, but the hook is called inside synchronoustransaction.mutate()and its promise is ignored. Any mutation after anawaitescapes the transaction. The contract and examples need to be explicitly synchronous, or the implementation needs an actually awaited atomic design.
Fixed outdated docs, this is no longer async.
watchAttachmentscalls need to be serialized so an older async update can't win after a newer one.
I don't believe this comment is correct/needed. They should already be serialized (at least by our SDK).
- This adds public API and raises the PowerSync peer floor from 1.41 to 1.57, so the changeset should be
minor, notpatch.
Done.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
packages/powersync-db-collection/src/attachments.ts (2)
36-41: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winDocument (and consider enforcing) that
updateHookmust be synchronous.Transaction.mutate's callback wrapping only covers the synchronous portion of the callback:registerTransaction/unregisterTransactionwrapcallback()directly, withunregisterTransactionin afinallythat runs as soon ascallback()returns. An asyncupdateHookthat does work after anawaitruns outside the transaction context, so its mutations would not be committed atomically with the attachment write — contradicting the doc's claim that "any mutations made to other collections are committed atomically with it." This is the exact concern flagged in the PR review (updateHook callbacks must be synchronous or implement an awaited atomic design) and is still not reflected in the docs or type signature.Update the doc to state the requirement explicitly, and apply the same clarification to
DeleteOptions.updateHook.📝 Proposed doc fix
/** - * Called within the same TanStackDB transaction as the attachment write, - * so any mutations made to other collections are committed atomically with it. + * Called synchronously within the same TanStackDB transaction as the attachment write, + * so any mutations made to other collections are committed atomically with it. + * An async callback escapes the transaction boundary: mutations made after an + * `await` inside this hook are not part of this transaction. */ updateHook?: (attachment: AttachmentQueueRow) => void🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/powersync-db-collection/src/attachments.ts` around lines 36 - 41, Update the updateHook documentation in the attachment write options to explicitly require a synchronous callback and warn that asynchronous work after an await is outside the transaction. Apply the same clarification to DeleteOptions.updateHook, while preserving the existing callback type and atomicity description.
73-96: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winGuard against reusing an existing attachment id before writing the file.
The docstring for
id(Line 31) already documents this: reusing an existing id overwrites that attachment's local file, the insert is then rejected, and cleanup deletes the overwritten file — leaving the pre-existing record without a file.save()writes the file at Line 84 before any check againstthis.collection, so this data-loss path is real, not just theoretical, and is easy to prevent.Reject the id up front instead of documenting the footgun.
🛡️ Proposed fix
const resolvedId = id ?? (await this.generateAttachmentId()) + if (id !== undefined && this.collection.get(resolvedId)) { + throw new Error(`Attachment with id ${resolvedId} already exists`) + } const filename = `${resolvedId}.${fileExtension}`🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/powersync-db-collection/src/attachments.ts` around lines 73 - 96, Update AttachmentQueue.save to validate an explicitly provided id against this.collection before calling localStorage.saveFile. Reject existing attachment ids with an error and preserve normal id generation and saving for new attachments.🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. Outside diff comments: In `@packages/powersync-db-collection/src/attachments.ts`: - Around line 36-41: Update the updateHook documentation in the attachment write options to explicitly require a synchronous callback and warn that asynchronous work after an await is outside the transaction. Apply the same clarification to DeleteOptions.updateHook, while preserving the existing callback type and atomicity description. - Around line 73-96: Update AttachmentQueue.save to validate an explicitly provided id against this.collection before calling localStorage.saveFile. Reject existing attachment ids with an error and preserve normal id generation and saving for new attachments.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID:
47ee24a9-dbe5-456e-9288-09f20abaf25b📥 Commits
Reviewing files that changed from the base of the PR and between 2250d3592a1bd844c39052c5d308aaaab1b89b0c and 2718e1bf99dadc824db89d22ce10a145de9c6fad.
📒 Files selected for processing (4)
.changeset/curly-planets-lead.mddocs/collections/powersync-collection.mdpackages/powersync-db-collection/src/attachments.tspackages/powersync-db-collection/tests/attachments.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- .changeset/curly-planets-lead.md
- docs/collections/powersync-collection.md
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. Inline comments: In `@packages/powersync-db-collection/src/attachments.ts`: - Around line 34-43: Update the SaveOptions and DeleteOptions updateHook types to reject Promise or thenable return values, and add runtime checks around both save and delete hook invocation to detect a returned thenable and fail before it can escape the transaction. Add a regression test covering an async updateHook and verify that mutations after await are not committed outside the attachment transaction.🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID:
1c16d74c-cec4-4138-94d6-b0f7dde63c58📥 Commits
Reviewing files that changed from the base of the PR and between 2718e1bf99dadc824db89d22ce10a145de9c6fad and 5d3b5000fd831d953b9088a9811e492301a98ddb.
📒 Files selected for processing (2)
packages/powersync-db-collection/src/attachments.tspackages/powersync-db-collection/tests/attachments.test.ts
@@ -0,0 +23,21 @@23+export interface SaveOptions {24+ data: AttachmentData25+ fileExtension: string26+ mediaType?: string27+ metaData?: string28+ /**29+ * Optional custom ID. If not provided, a UUID will be generated.30+ *31+ * Rejected if an attachment with this ID is already in the queue.32+ */33+ id?: string34+ /**35+ * Called synchronously within the same TanStackDB transaction as the attachment write,36+ * so any mutations made to other collections are committed atomically with it.37+ *38+ * Must not be async. `Transaction.mutate` unregisters the ambient transaction as soon as39+ * this callback returns, so any mutation made after an `await` inside the hook escapes the40+ * transaction and is not committed atomically with the attachment. Do asynchronous work41+ * before calling `save` or `delete`.42+ */43+ updateHook?: (attachment: AttachmentQueueRow) => void🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🌐 Web query:
Does TypeScript allow an async function to be assigned to a callback type declared as(value: T) => void?💡 Result:
Yes, TypeScript allows an async function to be assigned to a callback type declared as (value: T) => void [1][2]. This is due to a specific feature of the TypeScript type system: when a function type specifies a return type of void, it is interpreted as "the caller will ignore the return value," rather than "the function must return nothing" [1][3]. Consequently, any function that returns a value (including a Promise returned by an async function) can be assigned to a callback type that returns void, as the return value will simply be ignored by the caller [1][2]. However, there are important nuances to be aware of: 1. Safety and Potential Issues: While TypeScript permits this assignment, it does not track or handle the Promise returned by the async function [2]. If the async function throws an error, the caller (which is expecting a void-returning callback) will not be able to catch or handle the Promise rejection, which can lead to unhandled promise rejections [2]. 2. Linting Rules: Many developers use linting tools, such as
@typescript-eslint/no-misused-promises, to explicitly forbid this pattern [4]. These tools are designed to catch this specific scenario to prevent the silent failure of async operations passed as callbacks [2][4]. 3. Literal Function Definitions: Note that while you can assign an async function to a variable typed as => void, TypeScript enforces stricter rules if you explicitly annotate a function definition itself as returning void [1]. In that case, the function cannot return a value, including a Promise [1][2]. Summary: While the TypeScript compiler allows this assignment by design [1], it is often considered a "gotcha" because it can hide asynchronous errors [2]. It is recommended to use caution and, in many professional codebases, utilize linting rules to prevent it [2][4].Citations:
- 1: https://www.typescriptlang.org/docs/handbook/2/functions.html
- 2: https://www.learningtypescript.com/articles/void-returning-function-assignability
- 3: https://cosmiclearn.com/typescript/callbacks.php
- 4: https://github.com/microsoft/TypeScript/issues/38752
🏁 Script executed:
#!/bin/bash set -euo pipefail # Locate and inspect the relevant file and hook usage without executing repository code. fd -a '^attachments\.ts$' . | sed 's#^\./##' file="$(fd '^attachments\.ts$' . | head -n 1)" echo "FILE=$file" wc -l "$file" ast-grep outline "$file" --view expanded || true echo "### hook-related lines" nl -ba "$file" | sed -n '1,260p'Repository: TanStack/db
Length of output: 1413
🏁 Script executed:
#!/bin/bash set -euo pipefail file="./packages/powersync-db-collection/src/attachments.ts" echo "Save/delete hook-related lines" awk 'NR>=23 && NR<=60 { printf "%4d %s\n", NR, $0 } NR>=79 && NR<=190 { printf "%4d %s\n", NR, $0 }' "$file" echo echo "TypeScript assignments to void return callbacks in repository (text search)" rg -n "updateHook|return => void|void\>|" packages -g '*.ts' -g '*.tsx' | head -n 160Repository: TanStack/db
Length of output: 18683
🏁 Script executed:
#!/bin/bash set -euo pipefail node <<'JS' const ts = require('typescript'); const code = ` interface Options { fn: (s: string) => void; } async function asyncCallback(s: string): Promise<void> { const v = await Promise.resolve(1); console.log(v); } const options: Options = { fn: asyncCallback }; options.fn('x'); options.fn(new Promise<string>(_ => {})); `.replace(/\n/g, '\r\n'); const options = { target: ts.ScriptTarget.ESNext, strict: true, noImplicitAny: true, noUnusedLocals: true, }; const compilerOptions = ts.convertCompilerOptionsForCodegen({}, options); const host = ts.createCompilerHost(compilerOptions); const program = ts.createProgram(['test.ts'], compilerOptions, { ...host, getSourceFile: (fileName, languageVersion, onError, shouldCreateNewSourceFile, implementationCache) => fileName === 'test.ts' ? ts.createSourceFile(fileName, code, languageVersion, true, ts.ScriptKind.TS) : host.getSourceFile(fileName, languageVersion, onError, shouldCreateNewSourceFile, implementationCache), fileExists: fileName => fileName === 'test.ts' || host.fileExists(fileName), }); const diagnostics = ts.getPreEmitDiagnostics(program); console.log(diagnostics.length === 0 ? 'NO_Diagnostics' : diagnostics.map(d => `${d.code}: ${ts.flattenDiagnosticMessageText(d.messageText, ' ')}` ).join('\n')); const programOptions = program.getCompilerOptions(); console.log('noImplicitAny:', programOptions.noImplicitAny === true); console.log('strict:', programOptions.strict === true); JSRepository: TanStack/db
Length of output: 836
🌐 Web query:
TypeScript async functions assigned to void callbacks noImplicitAny💡 Result:
In TypeScript, assigning an async function to a callback expecting a return type of void can lead to unexpected behavior and type errors, particularly regarding noImplicitAny and assignability [1][2][3]. The primary reasons for these issues are: 1. Return Type Inconsistency: TypeScript allows functions returning non-void types to be assigned to functions returning void in many cases because void-returning functions ignore their return values [4][5]. However, this "void-swallowing" behavior does not consistently apply to async functions, which always return a Promise [1][6]. Consequently, an async function might not be considered assignable to a type expecting => void, even though a synchronous function returning a value would be [1][6]. 2. Implicit Any Errors: When using function overloads or generic callbacks, TypeScript may struggle to infer parameter types when an async function is passed [3]. Because the compiler cannot match the async function against the specific overload (often due to the Promise vs. void return mismatch), it may fail to resolve the parameter types, resulting in implicit any (TS7006) errors [2][3]. 3. Recommendations: - Always use Promise instead of void for the return type of async functions to explicitly indicate that they return a promise that resolves to nothing [4][7]. - If you control the callback signature, update it to accept Promise | void or use a generic type T to allow both sync and async implementations [5][8]. - If you encounter implicit any errors with overloaded functions, explicitly annotate the parameters of the async callback to help TypeScript resolve the overload [3]. By explicitly typing async callbacks with Promise, you avoid relying on the inconsistent assignability rules that govern void-returning functions [1][4].
Citations:
- 1: https://github.com/microsoft/TypeScript/issues/33420
- 2: https://github.com/microsoft/TypeScript/issues/48088
- 3: https://stackoverflow.com/questions/56861255/overloaded-typescript-function-parameter-does-not-work-with-async-keyword
- 4: https://stackoverflow.com/questions/59664670/typescript-async-function-return-type-void-vs-promisevoid
- 5: https://stackoverflow.com/questions/73389071/typescript-function-type-assignment
- 6: https://github.com/microsoft/TypeScript/issues/55149
- 7: https://stackoverflow.com/questions/64574816/how-to-type-async-function-in-typescript-when-its-a-void-function
- 8: https://github.com/microsoft/TypeScript/issues/43921
Enforce the synchronous
updateHookcontract.Both
SaveOptions.updateHookandDeleteOptions.updateHookdeclare(attachment: AttachmentQueueRow) => void, but callers ignore the returned Promise from async hooks, so mutations afterawaitcan run outside the transaction. Update the API to reject async return values in both options types, add a runtime guard for returned thenables, and add a regression test that passes an async hook and verifies no mutation escapes the transaction.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/powersync-db-collection/src/attachments.ts` around lines 34 - 43, Update the SaveOptions and DeleteOptions updateHook types to reject Promise or thenable return values, and add runtime checks around both save and delete hook invocation to detect a returned thenable and fail before it can escape the transaction. Add a regression test covering an async updateHook and verify that mutations after await are not committed outside the attachment transaction.Source: Coding guidelines
I think this is overkill?
@Chriztiaan, agreed. This is overkill for the protection it provides.TypeScript cannot reliably reject an
asynccallback assigned to avoidcallback type. A runtime thenable check also runs too late to prevent work after anawaitfrom escaping the transaction.The documented synchronous
updateHookcontract is sufficient here. I will withdraw this finding.🐇 <!-- -->
You are interacting with an AI system.Chriztiaan(GitHub profile, opens in a new tab)replied on
packages/powersync-db-collection/src/attachments.tslines 43 to 52I think this is overkill?
coderabbitai[bot](GitHub profile, opens in a new tab)replied on
packages/powersync-db-collection/src/attachments.tslines 43 to 52@Chriztiaan, agreed. This is overkill for the protection it provides.TypeScript cannot reliably reject an
asynccallback assigned to avoidcallback type. A runtime thenable check also runs too late to prevent work after anawaitfrom escaping the transaction.The documented synchronous
updateHookcontract is sufficient here. I will withdraw this finding.🐇 <!-- -->
You are interacting with an AI system.More templates
@tanstack/angular-db
npm i https://pkg.pr.new/@tanstack/angular-db@1616@tanstack/browser-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/browser-db-sqlite-persistence@1616@tanstack/capacitor-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/capacitor-db-sqlite-persistence@1616@tanstack/cloudflare-durable-objects-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/cloudflare-durable-objects-db-sqlite-persistence@1616@tanstack/db
npm i https://pkg.pr.new/@tanstack/db@1616@tanstack/db-ivm
npm i https://pkg.pr.new/@tanstack/db-ivm@1616@tanstack/db-sqlite-persistence-core
npm i https://pkg.pr.new/@tanstack/db-sqlite-persistence-core@1616@tanstack/electric-db-collection
npm i https://pkg.pr.new/@tanstack/electric-db-collection@1616@tanstack/electron-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/electron-db-sqlite-persistence@1616@tanstack/expo-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/expo-db-sqlite-persistence@1616@tanstack/node-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/node-db-sqlite-persistence@1616@tanstack/offline-transactions
npm i https://pkg.pr.new/@tanstack/offline-transactions@1616@tanstack/powersync-db-collection
npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1616@tanstack/query-db-collection
npm i https://pkg.pr.new/@tanstack/query-db-collection@1616@tanstack/react-db
npm i https://pkg.pr.new/@tanstack/react-db@1616@tanstack/react-native-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/react-native-db-sqlite-persistence@1616@tanstack/rxdb-db-collection
npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1616@tanstack/solid-db
npm i https://pkg.pr.new/@tanstack/solid-db@1616@tanstack/svelte-db
npm i https://pkg.pr.new/@tanstack/svelte-db@1616@tanstack/tauri-db-sqlite-persistence
npm i https://pkg.pr.new/@tanstack/tauri-db-sqlite-persistence@1616@tanstack/trailbase-db-collection
npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1616@tanstack/vue-db
npm i https://pkg.pr.new/@tanstack/vue-db@1616commit:
5d3b500tannerlinsley reviewed
@@ -0,0 +76,21 @@76+ * Saves a file to local storage and queues it for upload to remote storage.77+ *78+ * Exposes an `updateHook` option which is called inside a TanStackDB transaction,79+ * relational associations with the provided attachment ID should be made in this hook.80+ */81+ async save({82+ data,83+ fileExtension,84+ mediaType,85+ metaData,86+ id,87+ updateHook,88+ }: SaveOptions): Promise<AttachmentQueueRow> {89+ const resolvedId = id ?? (await this.generateAttachmentId())90+91+ /**92+ * Checked before the file is written. Writing first would overwrite the existing93+ * attachment's local file, and the cleanup below would then delete it — leaving the94+ * pre-existing record pointing at a file that no longer exists.95+ */96+ if (this.collection.get(resolvedId)) {Concurrent saves with the same custom
idcan both pass this check because it runs beforewithAttachmentContext. Both then write the same URI; when the second insert fails, its catch deletes that URI, removing the first successful attachment file. Please serialize the duplicate check and file write with the attachment context, and add a concurrent-save regression test so a losing save cannot overwrite or delete the winner file.Addressed, thank you for the feedback.
Chriztiaan(GitHub profile, opens in a new tab)replied on
packages/powersync-db-collection/src/attachments.tsline 96Addressed, thank you for the feedback.
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
Diff Package Supply Chain
SecurityVulnerability Quality Maintenance License npm/@powersync/node@0.18.1 ⏵ 0.19.2 npm/@powersync/common@1.49.0 ⏵ 1.57.0 +9
Hey @tannerlinsley, friendly poke here :) Are there other reviewers I can bother to rubberstamp this?
@Chriztiaan @tannerlinsley I independently reproduced both reported bugs at
fc55bee66c05376e594aa7e377ab0a96f7087405, using the locked@powersync/common@1.57.0, real SQLite, the real collection adapter, and actual SDKsyncStorage()completion writes.I recommend holding merge for these two correctness fixes. The original suite passes: 95 runtime tests plus one type test. Eight added boundary cases yield four failures and four passing controls; the native-SDK failure is a causal control for the same upload race, not a fourth distinct bug. The added tests typecheck and lint clean.
1. Reusing an ID before a fresh collection is ready destroys the existing local file#
Save an attachment, construct another collection against the same table without waiting for readiness, then save different bytes under the same ID/extension. The collection lookup misses the existing SQL row. The new save rejects, the original SQL row survives, but its original file is gone. The otherwise identical ready-collection case preserves the file and bytes.
The guard must establish identity/ownership before a destructive file write. Waiting for collection readiness addresses this particular witness but is not an authoritative duplicate check for stale/partial caches. Failure cleanup must only remove resources owned by the failed operation.
An additional boundary probe uses two queue instances sharing the same DB/storage and ID: one save succeeds, the other rejects, and the loser removes the winner's file. This needs either shared-resource ownership/reservation or an explicit, enforced single-owner contract; the existing per-instance mutex does not cover it.
2. Delete during upload is overwritten by SDK completion#
Hold the remote upload; call the TanStack queue's delete with an updateHook that detaches the related row; verify SQLite contains QUEUED_DELETE and a null foreign key; release the upload. Two further real sync passes make zero remote delete calls, leaving the uploaded remote file behind. Delete-before-upload and delete-after-upload controls pass.
The native SDK AttachmentQueue alone also reproduces this: saveFile → hold upload → deleteFile → release upload changes SQL state from QUEUED_DELETE (2) to SYNCED (3). The SDK captures a row before I/O and later saves the stale completed row unconditionally. Serializing the individual writes does not preserve a delete committed between them.
This belongs at the SDK completion boundary, rather than a second attachment state machine in TanStack. Completion must reconcile with current committed intent and preserve outstanding remote deletion. Watcher/archive behavior also needs to be exercised before choosing the exact merge/update rule.
Next step / coordination#
We're starting a bounded lifecycle oracle on top of this PR: independent desired intent and local/remote bytes; real SDK completion SQL; controlled transport timing; cold/warm caches; delete before/during/after upload; rollback and ownership assertions; fixed witnesses plus replayable generated histories. No production changes have been made by this review.
Could the SDK maintainers coordinate the completion fix and confirm which SDK line this integration should target alongside #1688? That PR upgrades to v2 and currently conflicts with main. A source check of SDK main
fb05b28bacdd291e4e768b0a9d96b7c877314887still shows the unconditional completion save; the executable repro here is specifically against 1.57.0.For completeness, the earlier concern about serializing successive watched snapshots does not reproduce: a controlled test confirms the SDK context already serializes those callbacks. The synchronous-hook documentation and minor changeset are also present.
Executable test-only patch against fc55bee66 (intentionally RED)
Run from
packages/powersync-db-collectionafter building this checkout's core packages:pnpm exec vitest run tests/attachments.test.ts --coverage.enabled=false --typecheck.enabled=falsediff --git a/packages/powersync-db-collection/tests/attachments.test.ts b/packages/powersync-db-collection/tests/attachments.test.ts index 193f8fa0c..1cb5833e9 100644 --- a/packages/powersync-db-collection/tests/attachments.test.ts +++ b/packages/powersync-db-collection/tests/attachments.test.ts @@ -1,7 +1,9 @@ import { randomUUID } from 'node:crypto' import { tmpdir } from 'node:os' import { join } from 'node:path' +import pDefer from 'p-defer' import { + AttachmentQueue, AttachmentState, AttachmentTable, Schema, @@ -212,6 +214,260 @@ describePowerSync(`PowerSync AttachmentQueue (TanStackDB)`, () => { } describe(`save`, () => { + it(`serializes successive watched snapshots through the SDK context`, async () => { + const fixture = await setup() + let update!: Parameters<WatchAttachments>[0] + const queue = fixture.createQueue({ + archivedCacheLimit: 100, + watchAttachments: (callback) => { + update = callback + }, + }) + const record = await queue.save({ + data: createMockJpegBuffer(), + fileExtension: `jpg`, + }) + await queue.syncStorage() + await queue.startSync() + const entered = pDefer<void>() + const release = pDefer<void>() + const held = queue.withAttachmentContext(async () => { + entered.resolve() + await release.promise + }) + await entered.promise + const completed: Array<number> = [] + const first = update([]).then(() => { + completed.push(1) + }) + const second = update([{ id: record.id, fileExtension: `jpg` }]).then( + () => { + completed.push(2) + }, + ) + try { + await Promise.resolve() + expect(completed).toEqual([]) + release.resolve() + await Promise.all([held, first, second]) + expect(completed).toEqual([1, 2]) + expect( + await fixture.db.get(`SELECT state FROM attachments WHERE id = ?`, [ + record.id, + ]), + ).toEqual({ state: AttachmentState.SYNCED }) + } finally { + release.resolve() + await Promise.all([held, first, second]) + } + }) + + it(`preserves the winner file across two queue instances sharing storage`, async () => { + const fixture = await setup() + const first = fixture.createQueue() + const second = fixture.createQueue() + const bothWritten = pDefer<void>() + const saveFile = fixture.localStorage.saveFile.bind(fixture.localStorage) + let writes = 0 + vi.spyOn(fixture.localStorage, `saveFile`).mockImplementation( + async (...args) => { + const size = await saveFile(...args) + if (++writes === 2) bothWritten.resolve() + await bothWritten.promise + return size + }, + ) + const results = await Promise.allSettled([ + first.save({ + id: `shared-id`, + data: createMockJpegBuffer(), + fileExtension: `jpg`, + }), + second.save({ + id: `shared-id`, + data: new Uint8Array([7, 8, 9]).buffer, + fileExtension: `jpg`, + }), + ]) + expect( + results.filter((result) => result.status === `fulfilled`), + ).toHaveLength(1) + expect( + await fixture.localStorage.fileExists( + fixture.localStorage.getLocalUri(`shared-id.jpg`), + ), + ).toBe(true) + }) + + it.each([`ready`, `cold`] as const)( + `preserves an existing file when reused through a %s collection`, + async (phase) => { + const fixture = await setup() + const original = await fixture.createQueue().save({ + id: `reused-after-reopen`, + data: createMockJpegBuffer(), + fileExtension: `jpg`, + }) + const collection = + phase === `ready` + ? fixture.attachmentsCollection + : createCollection( + powerSyncCollectionOptions({ + database: fixture.db, + table: APP_SCHEMA.props.attachments, + }), + ) + onTestFinished(() => collection.cleanup()) + const queue = new TanStackDBAttachmentQueue({ + db: fixture.db, + attachmentsCollection: collection, + localStorage: fixture.localStorage, + remoteStorage: fixture.remoteStorage, + watchAttachments: () => {}, + }) + onTestFinished(() => queue.stopSync()) + if (phase === `cold`) + expect(collection.get(original.id)).toBeUndefined() + await expect( + queue.save({ + id: original.id, + data: new Uint8Array([7, 8, 9]).buffer, + fileExtension: `jpg`, + }), + ).rejects.toThrow() + expect( + await fixture.db.getOptional( + `SELECT id FROM attachments WHERE id = ?`, + [original.id], + ), + ).toEqual({ id: original.id }) + expect(await fixture.localStorage.fileExists(original.local_uri!)).toBe( + true, + ) + expect( + new Uint8Array( + await fixture.localStorage.readFile(original.local_uri!), + ), + ).toEqual(new Uint8Array(createMockJpegBuffer())) + }, + ) + + it.each([`before`, `during`, `after`] as const)( + `preserves delete intent %s an SDK upload`, + async (timing) => { + const fixture = await setup() + const queue = fixture.createQueue() + const uploaded = pDefer<void>() + const release = pDefer<void>() + const remoteFiles = new Set<string>() + fixture.uploadFile.mockImplementation(async (_, attachment) => { + uploaded.resolve() + await release.promise + remoteFiles.add(attachment.id) + }) + fixture.deleteFile.mockImplementation((attachment) => { + remoteFiles.delete(attachment.id) + return Promise.resolve() + }) + const userId = randomUUID() + const record = await queue.save({ + data: createMockJpegBuffer(), + fileExtension: `jpg`, + updateHook: (attachment) => { + fixture.usersCollection.insert({ + id: userId, + name: `owner`, + email: null, + photo_id: attachment.id, + }) + }, + }) + let sync: Promise<void> | undefined + try { + if (timing !== `before`) { + sync = queue.syncStorage() + await uploaded.promise + if (timing === `after`) { + release.resolve() + await sync + } + } + await queue.delete({ + id: record.id, + updateHook: () => { + fixture.usersCollection.update(userId, (row) => { + row.photo_id = null + }) + }, + }) + expect( + await fixture.db.get(`SELECT state FROM attachments WHERE id = ?`, [ + record.id, + ]), + ).toEqual({ state: AttachmentState.QUEUED_DELETE }) + expect( + await fixture.db.get(`SELECT photo_id FROM users WHERE id = ?`, [ + userId, + ]), + ).toEqual({ photo_id: null }) + release.resolve() + await sync + // Complete two real SDK passes, not a timeout waiting for a mock state. + await queue.syncStorage() + await queue.syncStorage() + expect(fixture.deleteFile).toHaveBeenCalledTimes(1) + expect(remoteFiles.has(record.id)).toBe(false) + expect(await fixture.localStorage.fileExists(record.local_uri!)).toBe( + false, + ) + } finally { + release.resolve() + await sync + } + }, + ) + + it(`keeps a native SDK delete queued when upload completion arrives`, async () => { + const fixture = await setup() + const queue = new AttachmentQueue({ + db: fixture.db, + localStorage: fixture.localStorage, + remoteStorage: fixture.remoteStorage, + watchAttachments: () => {}, + }) + const uploaded = pDefer<void>() + const release = pDefer<void>() + fixture.uploadFile.mockImplementation(async () => { + uploaded.resolve() + await release.promise + }) + const record = await queue.saveFile({ + data: createMockJpegBuffer(), + fileExtension: `jpg`, + }) + const sync = queue.syncStorage() + try { + await uploaded.promise + await queue.deleteFile({ id: record.id }) + expect( + await fixture.db.get(`SELECT state FROM attachments WHERE id = ?`, [ + record.id, + ]), + ).toEqual({ state: AttachmentState.QUEUED_DELETE }) + release.resolve() + await sync + expect( + await fixture.db.get(`SELECT state FROM attachments WHERE id = ?`, [ + record.id, + ]), + ).toEqual({ state: AttachmentState.QUEUED_DELETE }) + } finally { + release.resolve() + await sync + await queue.stopSync() + } + }) + it(`writes the local file and inserts a QUEUED_UPLOAD row into the collection`, async () => { const { createQueue, attachmentsCollection, localStorage } = await setup() const queue = createQueue()Companion fixes and oracle: https://github.com/powersync-ja/tanstack-db/pull/10 (targets your feat/powersync-attachments branch). Tanner’s startup fix informed a targeted-ID live-query lease that also works with on-demand collections. It adds duplicate/file-ownership guards and preserves SDK filename-based storage relocation. Local gate: 121 tests pass, types/lint/build pass. The upload/delete completion race remains an upstream blocker: the companion includes five separately runnable failing native-SDK/integration checks for both success and retry completion, with no SDK workaround or expected-failure waiver. See the companion body and ATTACHMENT-ORACLE.md for commands and exact scope.
Sign in to join this conversation.
Note
Reviews paused#
It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the
reviews.auto_review.auto_pause_after_reviewed_commitssetting.Use the following commands to manage reviews:
@coderabbitai resumeto resume automatic reviews.@coderabbitai reviewto trigger a single review.Use the checkboxes below for quick actions:
📝 Walkthrough
Walkthrough#
Adds a TanStack DB-backed attachment queue with public exports, setup documentation, and tests. Attachments can be saved and deleted inside collection transactions, with examples for watching, syncing, linked-row updates, and cached URIs.
Changes#
TanStackDB attachment queue
packages/powersync-db-collection/src/attachments.ts,packages/powersync-db-collection/src/index.ts,packages/powersync-db-collection/package.json,.changeset/curly-planets-lead.mddocs/collections/powersync-collection.md,packages/powersync-db-collection/tests/attachments.test.tswatchAttachments, with test setup for attachment state tracking.packages/powersync-db-collection/src/attachments.ts,docs/collections/powersync-collection.md,packages/powersync-db-collection/tests/attachments.test.tspackages/powersync-db-collection/src/attachments.ts,docs/collections/powersync-collection.md,packages/powersync-db-collection/tests/attachments.test.tsEstimated code review effort: 4 (Complex) | ~60 minutes
Sequence Diagram(s)#
🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)#
✅ Passed checks (4 passed)
✨ Finishing Touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
❤️ Share