Skip to content
Source
Sign inSign up
TanStack/ai
Open History to see the last commit

overview.md

132 lines5.2 KB

1---
2title: Overview
3id: memory-overview
4order: 1
5description: "Give a TanStack AI chat() call memory across turns and sessions. memoryMiddleware recalls relevant memory into the prompt before the model runs, then saves each finished turn through a pluggable adapter."
6keywords:
7 - tanstack ai
8 - memory
9 - long-term memory
10 - retrieval
11 - persistence
12 - middleware
13 - rag
14 - personalization
15---
16​
17Your assistant forgets everything the moment a session ends. A user tells it their name
18this week; next week it asks again. `memoryMiddleware` fixes that. It gives a `chat()`
19run memory that survives across turns and across sessions.
20​
21It works in two moves. Before the model runs, it **recalls** relevant memory from a
22pluggable adapter and adds it to the system prompt. After the run finishes, it **saves**
23the turn. The save is deferred, so it never blocks streaming.
24​
25Reach for it when you need recall across turns or sessions. To keep the last few messages
26of the same request, just pass them in `messages`. Memory is overkill for that.
27​
28Everything lives in `@tanstack/ai-memory`: the middleware, the adapter contract, and the
29built-in and vendor adapters.
30​
31> Want a copy-paste setup? See the [Quickstart](./quickstart). Building an adapter for a
32> backend that isn't shipped? See the [Custom Adapter](./custom-adapter) guide.
33​
34## When to reach for it
35​
36| Need | Use this |
37|------|----------|
38| "Remember what the user told me last week" | Memory middleware with a persistent adapter |
39| "Each user has their own context" | Memory middleware with a scoped adapter |
40| "Use a hosted memory service (mem0, Honcho, Hindsight)" | The matching vendor adapter |
41| Keep the last few turns in the same request | Pass them in `messages`, skip memory |
42​
43## The contract: recall and save
44​
45A memory adapter has one identifier and two verbs. Extraction, ranking, rendering, and
46storage are all the adapter's job. The middleware never looks inside a record.
47​
48| Member | Purpose |
49|--------|---------|
50| `id` | Stable identifier used in logs and devtools. |
51| `recall(scope, query)` | Return what's relevant to `query` within `scope`: a rendered `systemPrompt`, optional `fragments`, and optional LLM `tools` plus `toolGuidance`. |
52| `save(scope, turn)` | Persist a finished `{ user, assistant }` turn. Extraction happens here. Returns one `SaveReceipt` per write. |
53| `inspect(scope)?` | Optional. A full snapshot for a devtools panel. |
54| `listFacts(scope)?` | Optional. A flat fact list for a devtools panel. |
55​
56```ts
57// The MemoryAdapter contract, from `@tanstack/ai-memory`:
58import type { MemoryAdapter } from '@tanstack/ai-memory'
59```
60​
61Built-in adapters, each a tree-shakeable subpath:
62​
63```ts
64import { inMemory } from '@tanstack/ai-memory/in-memory'
65import { redis } from '@tanstack/ai-memory/redis'
66```
67​
68Vendor adapters:
69​
70```ts
71import { hindsight } from '@tanstack/ai-memory/hindsight'
72import { mem0 } from '@tanstack/ai-memory/mem0'
73import { honcho } from '@tanstack/ai-memory/honcho'
74```
75​
76See [Adapters](./adapters) for every adapter and its options.
77​
78## How a turn flows
79​
801. **Recall** runs before the model, during the run's `init` phase.
81 `adapter.recall(scope, userText)` returns memory, and the middleware adds the
82 `systemPrompt`, `toolGuidance`, and any `tools` to the run.
832. **Save** runs after the stream finishes, deferred through `ctx.defer` so it never
84 blocks the response. The middleware hands the `{ user, assistant }` turn to
85 `adapter.save(scope, turn)`.
86​
87To add telemetry, watch memory in devtools, persist without recalling, or handle
88failures, see [Operating memory](./operating).
89​
90## Scope and security
91​
92`MemoryScope` is the isolation boundary. It is an alias of the shared `Scope` identity
93type from `@tanstack/ai` — the same vocabulary used by persistence — so memory and chat
94center on one conversation key:
95​
96```ts
97// MemoryScope is an alias of Scope from `@tanstack/ai`, exported by `@tanstack/ai-memory`:
98type MemoryScope = {
99 threadId: string // required — same as ChatMiddlewareContext.threadId
100 userId?: string
101 tenantId?: string
102 namespace?: string // reserved — no adapter keys on it yet
104```
105​
106Always resolve scope on the server from trusted session/auth state. A client-originated
107`threadId` is fine only after you validate it belongs to the session user; never accept
108`userId`/`tenantId` from the request body alone. The function form of `scope` runs per
109request and only sees what your server attached to the chat context:
110​
111```ts
112import { memoryMiddleware } from '@tanstack/ai-memory'
113import type { MemoryAdapter } from '@tanstack/ai-memory'
114​
115declare const adapter: MemoryAdapter
116declare function getSession(ctx: unknown): { threadId: string; userId: string }
117​
118memoryMiddleware({
119 adapter,
120 scope: (ctx) => {
121 const session = getSession(ctx) // your server-validated session
122 return { threadId: session.threadId, userId: session.userId }
123 },
124})
125```
126​
127## Next steps
128​
129- [Quickstart](./quickstart): wire `memoryMiddleware` into a real `chat()` call
130- [Adapters](./adapters): every adapter's options, with an example of each
131- [Custom Adapter](./custom-adapter): implement `recall`/`save` for a backend that isn't shipped
132- [Operating memory](./operating): options, telemetry, devtools events, and failure behavior