Skip to content
Source
Sign inSign up
TanStack/ai
Open History to see the last commit

sse-protocol.md

362 lines9.3 KB

1---
2title: Server-Sent Events (SSE) Protocol
3id: sse-protocol
4description: "TanStack AI's Server-Sent Events protocol spec — the recommended streaming transport for chat and media generations, with auto-reconnection."
5keywords:
6 - tanstack ai
7 - sse
8 - server-sent events
9 - streaming protocol
10 - protocol spec
11 - eventsource
12---
13​
14Server-Sent Events (SSE) is a standard HTTP-based protocol for server-to-client streaming. It provides:
15​
16- ✅ **Automatic reconnection** - Browser handles connection drops
17- ✅ **Event-driven** - Native browser EventSource API
18- ✅ **Simple protocol** - Text-based, easy to debug
19- ✅ **Wide support** - Works in all modern browsers
20- ✅ **Efficient** - Single long-lived HTTP connection
21​
22This document describes how TanStack AI transmits StreamChunks over Server-Sent Events (SSE), the recommended protocol for most use cases.
23​
24## Protocol Specification
25​
26### HTTP Request
27​
28**Method:** `POST`
29​
30**Headers:**
31```http
32Content-Type: application/json
33```
34​
35**Body:**
36```json
38 "messages": [
39 {
40 "role": "user",
41 "content": "Hello, how are you?"
42 }
43 ],
44 "data": {
45 // Optional additional data
46 }
48```
49​
50### HTTP Response
51​
52**Status:** `200 OK`
53​
54**Headers:**
55```http
56Content-Type: text/event-stream
57Cache-Control: no-cache
58Connection: keep-alive
59```
60​
61**Body:** Stream of SSE events
62​
63---
64​
65## SSE Format
66​
67Each StreamChunk is transmitted as an SSE event with the following format:
68​
69```
70data: {JSON_ENCODED_CHUNK}\n\n
71```
72​
73### Key Points
74​
751. **Each event starts with `data: `**
762. **Followed by the JSON-encoded chunk**
773. **Ends with double newline `\n\n`**
784. **No event names or IDs** (not required for our use case)
79​
80### Examples
81​
82#### Content Chunk
83​
84```
85data: {"type":"content","id":"chatcmpl-abc123","model":"gpt-5.2","timestamp":1701234567890,"delta":"Hello","content":"Hello","role":"assistant"}\n\n
86```
87​
88#### Tool Call Chunk
89​
90```
91data: {"type":"tool_call","id":"chatcmpl-abc123","model":"gpt-5.2","timestamp":1701234567891,"toolCall":{"id":"call_xyz","type":"function","function":{"name":"get_weather","arguments":"{\"location\":\"SF\"}"}},"index":0}\n\n
92```
93​
94#### Done Chunk
95​
96```
97data: {"type":"done","id":"chatcmpl-abc123","model":"gpt-5.2","timestamp":1701234567892,"finishReason":"stop","usage":{"promptTokens":10,"completionTokens":5,"totalTokens":15}}\n\n
98```
99​
100---
101​
102## Stream Lifecycle
103​
104### 1. Client Initiates Connection
105​
106```typescript
107// Client code
108const response = await fetch('/api/chat', {
109 method: 'POST',
110 headers: {
111 'Content-Type': 'application/json',
112 },
113 body: JSON.stringify({ messages }),
114});
115```
116​
117### 2. Server Sends Response Header
118​
119```http
120HTTP/1.1 200 OK
121Content-Type: text/event-stream
122Cache-Control: no-cache
123Connection: keep-alive
124```
125​
126### 3. Server Streams Chunks
127​
128The server sends multiple `data:` events as chunks are generated:
129​
130```
131data: {"type":"content","id":"msg_1","model":"gpt-5.2","timestamp":1701234567890,"delta":"The","content":"The"}\n\n
132data: {"type":"content","id":"msg_1","model":"gpt-5.2","timestamp":1701234567891,"delta":" weather","content":"The weather"}\n\n
133data: {"type":"content","id":"msg_1","model":"gpt-5.2","timestamp":1701234567892,"delta":" is","content":"The weather is"}\n\n
134data: {"type":"content","id":"msg_1","model":"gpt-5.2","timestamp":1701234567893,"delta":" sunny","content":"The weather is sunny"}\n\n
135data: {"type":"done","id":"msg_1","model":"gpt-5.2","timestamp":1701234567894,"finishReason":"stop"}\n\n
136```
137​
138### 4. Stream Completion
139​
140After the final chunk, the server sends a completion marker:
141​
142```
143data: [DONE]\n\n
144```
145​
146Then closes the connection.
147​
148---
149​
150## Error Handling
151​
152### Server-Side Errors
153​
154If an error occurs during generation, send an error chunk:
155​
156```
157data: {"type":"error","id":"msg_1","model":"gpt-5.2","timestamp":1701234567895,"error":{"message":"Rate limit exceeded","code":"rate_limit_exceeded"}}\n\n
158```
159​
160Then close the connection.
161​
162### Connection Errors
163​
164SSE provides automatic reconnection:
165- Browser automatically reconnects on connection drop
166- Server can send `retry:` field to control reconnection delay
167- Client can handle `error` events from EventSource
168​
169---
170​
171## Implementation
172​
173### Server-Side (Node.js/TypeScript)
174​
175TanStack AI provides `toServerSentEventsStream()` and `toServerSentEventsResponse()` utilities:
176​
177```typescript
178import { chat, toServerSentEventsResponse } from '@tanstack/ai';
179import { openaiText } from '@tanstack/ai-openai';
180​
181export async function POST(request: Request) {
182 const { messages } = await request.json();
183​
184 const stream = chat({
185 adapter: openaiText('gpt-5.2'),
186 messages,
187 });
188​
189 // Automatically converts StreamChunks to SSE format
190 return toServerSentEventsResponse(stream);
192```
193​
194**What `toServerSentEventsResponse()` does:**
1951. Creates a `ReadableStream` from the async iterable
1962. Wraps each chunk as `data: {JSON}\n\n`
1973. Sends `data: [DONE]\n\n` at the end
1984. Sets proper SSE headers
1995. Handles errors and cleanup
200​
201### Client-Side (Browser/Node.js)
202​
203TanStack AI provides `fetchServerSentEvents()` connection adapter:
204​
205```typescript
206import { useChat, fetchServerSentEvents } from '@tanstack/ai-react';
207​
208const { messages, sendMessage } = useChat({
209 connection: fetchServerSentEvents('/api/chat'),
210});
211```
212​
213**What `fetchServerSentEvents()` does:**
2141. Makes POST request with messages
2152. Reads response body as stream
2163. Parses SSE format (`data:` prefix)
2174. Deserializes JSON chunks
2185. Yields StreamChunk objects
2196. Stops on `[DONE]` marker
220​
221### Manual Implementation (Advanced)
222​
223If you need custom handling:
224​
225#### Server
226```typescript
227export async function POST(request: Request) {
228 const { messages } = await request.json();
229 const encoder = new TextEncoder();
231 const stream = new ReadableStream({
232 async start(controller) {
233 try {
234 for await (const chunk of chat({ adapter: openaiText('gpt-5.2'), messages })) {
235 const sseData = `data: ${JSON.stringify(chunk)}\n\n`;
236 controller.enqueue(encoder.encode(sseData));
237 }
238 controller.enqueue(encoder.encode('data: [DONE]\n\n'));
239 controller.close();
240 } catch (error) {
241 const errorChunk = {
242 type: 'error',
243 error: { message: error.message }
244 };
245 controller.enqueue(encoder.encode(`data: ${JSON.stringify(errorChunk)}\n\n`));
246 controller.close();
247 }
248 }
249 });
250​
251 return new Response(stream, {
252 headers: {
253 'Content-Type': 'text/event-stream',
254 'Cache-Control': 'no-cache',
255 'Connection': 'keep-alive',
256 },
257 });
259```
260​
261#### Client
262```typescript
263const response = await fetch('/api/chat', {
264 method: 'POST',
265 headers: { 'Content-Type': 'application/json' },
266 body: JSON.stringify({ messages }),
267});
268​
269const reader = response.body!.getReader();
270const decoder = new TextDecoder();
271let buffer = '';
272​
273while (true) {
274 const { done, value } = await reader.read();
275 if (done) break;
277 buffer += decoder.decode(value, { stream: true });
278 const lines = buffer.split('\n');
279 buffer = lines.pop() || '';
281 for (const line of lines) {
282 if (line.startsWith('data: ')) {
283 const data = line.slice(6);
284 if (data === '[DONE]') continue;
286 const chunk = JSON.parse(data);
287 // Handle chunk...
288 }
289 }
291```
292​
293---
294​
295## Debugging
296​
297### Inspecting SSE Traffic
298​
299**Browser DevTools:**
3001. Open Network tab
3012. Look for requests with `text/event-stream` type
3023. View response as it streams in
303​
304**cURL:**
305```bash
306curl -N -X POST http://localhost:3000/api/chat \
307 -H "Content-Type: application/json" \
308 -d '{"messages":[{"role":"user","content":"Hello"}]}'
309```
310​
311The `-N` flag disables buffering to see real-time output.
312​
313**Example Output:**
314```
315data: {"type":"content","id":"msg_1","model":"gpt-5.2","timestamp":1701234567890,"delta":"Hello","content":"Hello"}
316​
317data: {"type":"content","id":"msg_1","model":"gpt-5.2","timestamp":1701234567891,"delta":" there","content":"Hello there"}
318​
319data: {"type":"done","id":"msg_1","model":"gpt-5.2","timestamp":1701234567892,"finishReason":"stop"}
320​
321data: [DONE]
322```
323​
324---
325​
326## Advantages of SSE
327​
3281. **Built-in Reconnection** - Browser handles connection drops automatically
3292. **Simpler than WebSocket** - No handshake, just HTTP
3303. **Server-to-Client Only** - Matches chat streaming use case perfectly
3314. **Wide Browser Support** - Works everywhere (except IE11)
3325. **Proxy-Friendly** - Works through most HTTP proxies
3336. **Easy to Debug** - Plain text format, visible in DevTools
334​
335---
336​
337## Limitations
338​
3391. **One-Way Communication** - Server to client only (fine for streaming responses)
3402. **HTTP/1.1 Connection Limits** - Browsers limit concurrent connections per domain (6-8)
3413. **No Binary Data** - Text-only (not an issue for JSON chunks)
3424. **HTTP/2 Streams** - Can be more efficient but SSE works fine
343​
344---
345​
346## Best Practices
347​
3481. **Always set proper headers** - `Content-Type`, `Cache-Control`, `Connection`
3492. **Send `[DONE]` marker** - Helps client know when to close
3503. **Handle errors gracefully** - Send error chunk before closing
3514. **Use compression** - Enable gzip/brotli at the reverse proxy level
3525. **Set timeouts** - Prevent hanging connections
3536. **Monitor connection count** - Watch for connection leaks
354​
355---
356​
357## See Also
358​
359- [Chunk Definitions](./chunk-definitions) - StreamChunk type reference
360- [HTTP Stream Protocol](./http-stream-protocol) - Alternative protocol
361- [Connection Adapters Guide](../chat/connection-adapters) - Client implementation
362- [MDN: Server-Sent Events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events)