packages/db-sqlite-persistence-core
| Name | Last commit message | Last commit date |
|---|---|---|
| .. | ||
README.md
@tanstack/db-sqlite-persistence-core#
Shared SQLite persistence primitives for TanStack DB. Runtime-specific wrappers (Node, Electron, React Native, Cloudflare Durable Objects) build on top of this package.
What this package provides#
- Generic persisted collection wrapper utilities
- Shared persistence/coordinator protocol types
- SQLite core persistence adapter (
createSQLiteCorePersistenceAdapter) - Validation and storage key helpers
- Shared error types
This package intentionally does not include a concrete SQLite engine
binding. Provide a runtime SQLiteDriver implementation from a wrapper package.
Exported API (complete)#
Persisted wrapper and protocol APIs#
PersistedMutationEnvelopeProtocolEnvelope<TPayload>LeaderHeartbeatTxCommittedEnsureRemoteSubsetRequestEnsureRemoteSubsetResponseReleaseRemoteSubsetRequestReleaseRemoteSubsetResponseRemoteSubsetOwnerDuplicateRemoteSubsetOwnerErrorTransportedLoadSubsetOptionsRemoteSubsetWirePrimitiveRemoteSubsetWireTypedArrayRemoteSubsetWireRecordRemoteSubsetWireValueRemoteSubsetWireExpressionRemoteSubsetWireCompareOptionsRemoteSubsetWireOrderByClauseRemoteSubsetWireCursorRemoteSubsetWireValueErrortoTransportedLoadSubsetOptions(...)ApplyLocalMutationsRequestApplyLocalMutationsResponseApplyCommittedTxRequestApplyCommittedTxResponsePullSinceRequestPullSinceResponseCollectionResetPersistedIndexSpecPersistedRowMetadataMutation<TKey>PersistedCollectionMetadataMutationReplayableTxDelta<T, TKey>PersistedScannedRow<T, TKey>PersistedRowScanOptionsPersistedKeySetEvidencePersistedTx<T, TKey>PersistenceAdapterSQLiteDriverPersistedCollectionCoordinatorPersistedCollectionPersistencePersistedCollectionModePersistedCollectionLeadershipStatePersistedCollectionUtilsPersistedSyncWrappedOptions<T, TKey, TSchema, TUtils>PersistedLocalOnlyOptions<T, TKey, TSchema, TUtils>SingleProcessCoordinatorvalidatePersistedCollectionCoordinator(...)persistedCollectionOptions(...)encodePersistedStorageKey(...)decodePersistedStorageKey(...)createPersistedTableName(...)
PersistedCollectionPersistence can optionally implement:
resolvePersistenceForMode(mode)(legacy mode-aware resolution)resolvePersistenceForCollection({ collectionId, mode, schemaVersion })(collection-aware resolution)
persistedCollectionOptions(...) now supports schemaVersion per collection
and resolves persistence using:
- collection id
- inferred mode (
sync-presentorsync-absent) - optional
schemaVersion
This lets runtime wrappers expose one shared persistence instance per database while still handling per-collection schema versions correctly.
Coordinator contract#
Every PersistedCollectionCoordinator must implement
requestApplyCommittedTx(collectionId, tx). A sync transaction with durable
effects is routed in full to the persistence adapter owned by that collection.
The PersistedTx includes any truncate, row mutations, row metadata mutations,
collection metadata mutations, and stream position. Multiprocess coordinators
must preserve the complete transaction when they forward it to the elected
writer. An effect-free source commit does not allocate a stream position,
publish an invalidation, or call persistence; the next durable transaction
retains the next coordinator-owned sequence.
This is a required invariant, not a capability to detect at commit time. The
TypeScript interface requires the method, and persistedCollectionOptions(...)
validates custom coordinators when it configures a collection. An untyped
integration that omits the method throws
InvalidPersistedCollectionCoordinatorError before the sync source starts or
publishes rows. Implementations must not fall back to a row-only route such as
requestApplyLocalMutations(...), because that would discard transaction
semantics.
After a mutating RPC transport failure, replay is allowed only while the
requester can prove the same non-null leader id and term still own the route.
If either value was unknown for the first attempt, or either value changes,
the request rejects with IndeterminateCommitError and requires application
reconciliation. Coordinators never retry an indeterminate mutation against an
unknown or replacement leader.
SingleProcessCoordinator also implements the complete route. It is always
leader and has no cross-process transport, but it applies the transaction to
the resolved adapter registered for the specific collection. The Browser and
Electron coordinators provide the corresponding elected-owner route in their
runtime packages.
An adapter/storage rejection is surfaced as
PersistedCollectionDurabilityError, never as CONFLICT. The error preserves
the original local cause and its code and path fields. A transported
PERSISTENCE_ERROR response carries the source code/path that can cross the
coordinator boundary. Source writes remain published before their applied
receipt settles; if persistence then fails, the receipt rejects with the named
error and the collection enters its existing error lifecycle state. The
collection does not continue, retry, or fall back to a partial writer.
Remote subset wire contract#
requestEnsureRemoteSubset(collectionId, options) accepts the full live
LoadSubsetOptions input. Before choosing a local owner or remote transport,
every built-in coordinator projects it once to the exported
TransportedLoadSubsetOptions contract. A registered remote-subset owner
receives that transported type, never live-only fields such as signal or
subscription.
Expression values may contain structured-clone-safe primitives, plain records,
arrays (including sparse arrays), Date, zero-position RegExp, fixed
ArrayBuffer, fixed DataView, the views listed by the exported
RemoteSubsetWireTypedArray union (Int8Array, Uint8Array,
Uint8ClampedArray, Int16Array, Uint16Array, Int32Array, Uint32Array,
Float32Array, Float64Array, BigInt64Array, and BigUint64Array), Map,
and Set. Cycles and aliases are preserved. Other view types; shared,
resizable, growable, or detached buffers; accessors; non-enumerable or symbol
keys; native expandos; custom prototypes; promises; weak collections;
functions; and symbols are outside the contract. A rejected value throws the exported
RemoteSubsetWireValueError before owner work or transport, with its exact
path in the request. The wire contract does not provide codecs, coercion, or
lossy normalization for unsupported values.
Remote subset ownership is lease-based. A coordinator gives each accepted
request object a stable acquisition identity scoped to its collection and
requester. Retrying the same acquisition coalesces with the original; a second
request object with equal values is an independent lease. Releasing a lease
calls the registered owner's required unloadSubset(...) method with the exact
transported options object that was loaded. Releasing it again is a no-op.
EnsureRemoteSubsetRequest.acquisitionId is the wire identity established by
acquisition; ReleaseRemoteSubsetRequest.acquisitionId must carry that same
identity so the elected owner releases the matching lease.
The owner also provides a required onError(error) callback. A load or unload
throw by the actual elected owner, or a runtime thenable rejection from either
operation, is reported through that callback with the original value before
the awaited request rejects. A follower transport or remote-owner admission
failure rejects the acquisition and remains eligible for the existing demand
retry; it is not a failure of that follower's local owner lifecycle. Retirement
and disposal observe and report actual owner failures while continuing cleanup
of sibling leases; they do not retry or silently recover the failed lifecycle.
Only one remote subset owner may be registered per collection in a coordinator.
A second live registration throws DuplicateRemoteSubsetOwnerError instead of
replacing the first, and startup reports the named failure before readiness
rather than leaving a rejected registration promise unobserved. Released
acquisition identities remain as dedupe tombstones for the coordinator's
existing RPC dedupe horizon, then expire during later subset traffic. When a
leader loses ownership or is disposed, it unloads every transferred lease. A
requester that still owns a lease replays that same acquisition when the next
leader is observed. There is no adapter fallback or direct-writer escape hatch
for missing ownership.
The named-error and exact-path guarantee covers values the boundary can
identify with standard JavaScript reflection. A fully transparent Proxy is
outside the public wire type and cannot be portably distinguished from its
target. Untyped callers must not rely on Proxy identity, traps, or a
Proxy-specific diagnostic; no non-standard detection is attempted.
Source replay ordering#
The persisted wrapper buffers authoritative source sync transactions and
replays them into the Collection in FIFO order, one transaction at a time.
Local optimistic mutations remain visible immediately; this queue governs
source publication and its durable suffix. A source may mark a transaction
immediate when it must update the authoritative base beneath an active
optimistic layer. The wrapper still gives that transaction one FIFO turn and
writes its durable suffix before admitting the next source turn.
The wrapper deliberately rejects schedules that would require a second
ordering mechanism. An immediate source transaction cannot enter while an
earlier normal source publication is waiting, and a source transaction cannot
cross a hydration cycle. These cases throw
InvalidPersistedCollectionConfigError instead of reordering data or waiting
in a dependency cycle.
Atomic resume snapshots#
Persistence adapters implement
loadResumeSnapshot(collectionId, options) to let a sync source certify a
persisted resume baseline. One call must read rows, collection metadata, stream
position, reset epoch, and key-set evidence from the same atomic database
snapshot. includeRows: false requests the same certification data without
materializing rows; requiredIndexSignatures carries the indexes needed by a
row-bearing snapshot.
PersistedKeySetEvidence.status has three states:
consistent: the persisted rows match the adapter's durable expected-key ledger.incompatible: row loss, substitution, or a reset-generation change makes the saved resume baseline unsafe.unknown: the adapter has no authoritative pre-migration key set and does not claim completeness.
The method is required because the versioned metadata.persistence capability
always carries hydration, durable row scanning, certification, evidence, and
generation ownership as one complete bundle. Sync wrappers must forward the
capability object unchanged rather than copying individual methods. Adapter
methods are invoked with their receiver and may rely on instance state through
this.
SyncMetadataApi.persistence is always present. Core sync sources receive
null, which explicitly means that no persistence bridge is active. The
persisted wrapper replaces that sentinel with the complete versioned
capability. A wrapper that omits the property is invalid and fails before a
consumer can resume or query against uncertified durable state.
SQLite core adapter APIs#
SQLiteCoreAdapterOptionsSQLitePullSinceResult<TKey>SQLiteCorePersistenceAdapterDEFAULT_APPLIED_TX_PRUNE_MAX_ROWSDEFAULT_APPLIED_TX_PRUNE_MAX_AGE_SECONDScreateSQLiteCorePersistenceAdapter(...)
Error APIs#
PersistedCollectionCoreErrorInvalidPersistedCollectionConfigErrorInvalidSyncConfigErrorInvalidPersistedCollectionCoordinatorErrorInvalidPersistenceAdapterErrorPersistedCollectionDurabilityErrorOptionsPersistedCollectionDurabilityErrortoPersistedCollectionDurabilityError(...)IndeterminateCommitRequestTypeIndeterminateCommitErrorOptionsIndeterminateCommitErrorInvalidPersistedStorageKeyErrorInvalidPersistedStorageKeyEncodingErrorPersistenceUnavailableError
Adapter and coordinator commit failures reject the applied receipt and become
the collection's terminal sync error as PersistedCollectionDurabilityError.
That error preserves its original cause, code, and path. Hydration and
source errors remain distinct and are not wrapped as durability errors.
Typical usage (via runtime wrappers)#
In most applications, use a runtime package directly:
@tanstack/node-db-sqlite-persistence@tanstack/browser-db-sqlite-persistence@tanstack/electron-db-sqlite-persistence@tanstack/react-native-db-sqlite-persistence@tanstack/cloudflare-durable-objects-db-sqlite-persistence
Those packages provide concrete drivers and runtime wiring.