Skip to content
Source
Sign inSign up
TanStack/ai
Open History to see the last commit

streaming.md

188 lines4.9 KB

1---
2title: Streaming
3id: streaming-responses
4order: 2
5description: "Stream AI responses in real time with TanStack AI — async iterable chunks, chunk strategies, and partial JSON for responsive chat UIs."
6keywords:
7 - tanstack ai
8 - streaming
9 - streaming responses
10 - real-time ai
11 - async iterable
12 - chunks
13 - partial json
14---
15​
16TanStack AI supports streaming responses for real-time chat experiences. Streaming allows you to display responses as they're generated, rather than waiting for the complete response.
17​
18## How Streaming Works
19​
20When you use `chat()`, it returns an async iterable stream of chunks:
21​
22```typescript
23import { chat } from "@tanstack/ai";
24import { openaiText } from "@tanstack/ai-openai";
25​
26const stream = chat({
27 adapter: openaiText("gpt-5.2"),
28 messages,
29});
30​
31// Stream contains chunks as they arrive
32for await (const chunk of stream) {
33 console.log(chunk); // Process each chunk
35```
36​
37## Server-Side Streaming
38​
39Convert the stream to an HTTP response using `toServerSentEventsResponse`:
40​
41```typescript
42import { chat, toServerSentEventsResponse } from "@tanstack/ai";
43import { openaiText } from "@tanstack/ai-openai";
44​
45export async function POST(request: Request) {
46 const { messages } = await request.json();
47​
48 const stream = chat({
49 adapter: openaiText("gpt-5.2"),
50 messages,
51 });
52​
53 // Convert to HTTP response with proper headers
54 return toServerSentEventsResponse(stream);
56```
57​
58## Client-Side Streaming
59​
60The `useChat` hook automatically handles streaming:
61​
62```typescript
63import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";
64​
65const { messages, sendMessage, isLoading } = useChat({
66 connection: fetchServerSentEvents("/api/chat"),
67});
68​
69// Messages update in real-time as chunks arrive
70messages.forEach((message) => {
71 // Message content updates incrementally
72});
73```
74​
75## Stream Events (AG-UI Protocol)
76​
77TanStack AI implements the [AG-UI Protocol](https://docs.ag-ui.com/introduction) for streaming. Stream events contain different types of data:
78​
79### AG-UI Events
80​
81- **RUN_STARTED** - Emitted when a run begins
82- **TEXT_MESSAGE_START/CONTENT/END** - Text content streaming lifecycle
83- **TOOL_CALL_START/ARGS/END** - Tool invocation lifecycle
84- **STEP_STARTED/STEP_FINISHED** - Thinking/reasoning steps
85- **RUN_FINISHED** - Run completion with finish reason and usage
86- **RUN_ERROR** - Error occurred during the run
87​
88> **Tip:** Some models expose their internal reasoning as thinking content that streams before the response. See [Thinking & Reasoning](./thinking-content).
89​
90### Thinking Chunks
91​
92Thinking/reasoning is represented by AG-UI events `STEP_STARTED` and `STEP_FINISHED`. They stream separately from the final response text:
93​
94```typescript
95for await (const chunk of stream) {
96 if (chunk.type === "STEP_FINISHED") {
97 console.log("Thinking:", chunk.content); // Accumulated thinking content
98 console.log("Delta:", chunk.delta); // Incremental thinking token
99 }
101```
102​
103Thinking content is automatically converted to `ThinkingPart` in `UIMessage` objects. It is UI-only and excluded from messages sent back to the model.
104​
105## Connection Adapters
106​
107TanStack AI provides connection adapters for different streaming protocols:
108​
109### Server-Sent Events (SSE)
110​
111```typescript
112import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";
113​
114const { messages } = useChat({
115 connection: fetchServerSentEvents("/api/chat"),
116});
117```
118​
119### HTTP Stream
120​
121```typescript
122import { useChat, fetchHttpStream } from "@tanstack/ai-react";
123​
124const { messages } = useChat({
125 connection: fetchHttpStream("/api/chat"),
126});
127```
128​
129### Custom Stream
130​
131```typescript
132import { stream } from "@tanstack/ai-react";
133​
134const { messages } = useChat({
135 connection: stream(async (messages, data, signal) => {
136 // Custom streaming implementation
137 const response = await fetch("/api/chat", {
138 method: "POST",
139 body: JSON.stringify({ messages, ...data }),
140 signal,
141 });
142 // Return async iterable
143 return processStream(response);
144 }),
145});
146```
147​
148## Monitoring Stream Progress
149​
150You can monitor stream progress with callbacks:
151​
152```typescript
153const { messages } = useChat({
154 connection: fetchServerSentEvents("/api/chat"),
155 onChunk: (chunk) => {
156 console.log("Received chunk:", chunk);
157 },
158 onFinish: (message) => {
159 console.log("Stream finished:", message);
160 },
161});
162```
163​
164## Cancelling Streams
165​
166Cancel ongoing streams:
167​
168```typescript
169const { stop } = useChat({
170 connection: fetchServerSentEvents("/api/chat"),
171});
172​
173// Cancel the current stream
174stop();
175```
176​
177## Best Practices
178​
1791. **Handle loading states** - Use `isLoading` to show loading indicators
1802. **Handle errors** - Check `error` state for stream failures
1813. **Cancel on unmount** - Clean up streams when components unmount
1824. **Optimize rendering** - Batch updates if needed for performance
1835. **Show progress** - Display partial content as it streams
184​
185## Next Steps
186​
187- [Connection Adapters](./connection-adapters) - Learn about different connection types
188- [API Reference](../api/ai) - Explore streaming APIs