Open History to see the last commit
1# @tanstack/ai
2
3A powerful, open-source AI SDK with a unified interface across multiple providers. No vendor lock-in, no proprietary formats, just clean TypeScript and honest open source.
4
5## Features
6
7- **Multi-Provider Support** - OpenAI, Anthropic, Ollama, Google Gemini
8- **Unified API** - Same interface across all providers
9- **Structured Streaming** - JSON chunks with token deltas, tool calls, and usage stats
10- **Tool/Function Calling** - First-class support for AI function calling (OpenAI & Anthropic)
11- **React Hooks** - Simple `useChat` hook with v5 API (you control input state)
12- **TypeScript First** - Full type safety throughout
13- **Zero Lock-in** - Switch providers at runtime without code changes
14
15## Installation
16
17```bash
18# Core library
19npm install @tanstack/ai
20
21# Provider adapters (install what you need)
22npm install @tanstack/ai-openai
23npm install @tanstack/ai-anthropic
24npm install @tanstack/ai-ollama
25npm install @tanstack/ai-gemini
26
27# React hooks (for frontend chat UIs)
28npm install @tanstack/ai-react
29```
30
31## Architecture
32
33### Core Concepts
34
35**1. Provider-Agnostic Design**
36
37All providers implement the same `AIAdapter` interface:
38
39```typescript
40interface AIAdapter {
41 // Chat
42 chatCompletion(options: ChatCompletionOptions): Promise<ChatCompletionResult>;
43 chatStream(options: ChatCompletionOptions): AsyncIterable<StreamChunk>;
44
45 // Text generation
46 generateText(options: TextGenerationOptions): Promise<TextGenerationResult>;
47
48 // Summarization
49 summarize(options: SummarizationOptions): Promise<SummarizationResult>;
50
51 // Embeddings
52 createEmbeddings(options: EmbeddingOptions): Promise<EmbeddingResult>;
53}
54```
55
56**2. Structured Streaming**
57
58Streams return typed JSON chunks instead of raw strings:
59
60```typescript
61type StreamChunk =
62 | ContentStreamChunk // Text tokens with delta + accumulated content
63 | ToolCallStreamChunk // Function call information
64 | DoneStreamChunk // Completion signal with token usage
65 | ErrorStreamChunk; // Error information
66```
67
68**3. Adapter Pattern**
69
70The `AI` class wraps any adapter and provides a consistent API:
71
72```typescript
73const ai = new AI(adapter); // Initialize with adapter
74await ai.chat(options); // Standard chat
75ai.streamChat(options); // Structured streaming
76ai.setAdapter(newAdapter); // Switch providers
77```
78
79## API Reference
80
81### Core Library
82
83#### AI Class
84
85**Constructor**
86
87```typescript
88constructor(adapter: AIAdapter)
89```
90
91Create an AI instance with a specific provider adapter.
92
93#### Methods
94
95##### `chat(options): Promise<ChatCompletionResult>`
96
97Non-streaming chat completion.
98
99```typescript
100const result = await ai.chat({
101 model: "gpt-3.5-turbo",
102 messages: [
103 { role: "system", content: "You are helpful" },
104 { role: "user", content: "Hello!" },
105 ],
106 temperature: 0.7,
107 maxTokens: 1000,
108});
109
110console.log(result.content);
111console.log(result.usage.totalTokens);
112```
113
114##### `streamChat(options): AsyncIterable<StreamChunk>`
115
116Structured streaming with JSON chunks. **Automatically executes tools** if they have `execute` functions.
117
118```typescript
119for await (const chunk of ai.streamChat({
120 model: "gpt-3.5-turbo",
121 messages: [{ role: "user", content: "Hello" }],
122 tools: [...], // Optional: Tools with execute functions
123 toolChoice: "auto", // Optional: "auto" | "none" | { type, function }
124 maxIterations: 5, // Optional: Max tool calling loops (default: 5)
125})) {
126 switch (chunk.type) {
127 case "content":
128 console.log("Delta:", chunk.delta);
129 console.log("Full:", chunk.content);
130 break;
131 case "tool_call":
132 console.log("Tool:", chunk.toolCall.function.name);
133 console.log("Args:", chunk.toolCall.function.arguments);
134 // Tool is executed automatically if execute function is defined!
135 break;
136 case "done":
137 console.log("Finish:", chunk.finishReason);
138 console.log("Usage:", chunk.usage);
139 break;
140 case "error":
141 console.error("Error:", chunk.error.message);
142 break;
143 }
144}
145```
146
147**Automatic Tool Execution:** If tools have `execute` functions, `streamChat` will:
148
1491. Detect tool calls from the AI
1502. Execute the tools automatically
1513. Add results to the conversation
1524. Continue streaming the final response
1535. Repeat up to `maxIterations` times
154
155##### `generateText(options): Promise<TextGenerationResult>`
156
157Generate text from a prompt.
158
159```typescript
160const result = await ai.generateText({
161 model: "gpt-3.5-turbo-instruct",
162 prompt: "Once upon a time",
163 maxTokens: 100,
164});
165
166console.log(result.text);
167```
168
169##### `summarize(options): Promise<SummarizationResult>`
170
171Summarize text.
172
173```typescript
174const result = await ai.summarize({
175 model: "gpt-3.5-turbo",
176 text: "Long text to summarize...",
177 style: "bullet-points", // "bullet-points" | "paragraph" | "concise"
178 maxLength: 200,
179});
180
181console.log(result.summary);
182```
183
184##### `embed(options): Promise<EmbeddingResult>`
185
186Generate embeddings.
187
188```typescript
189const result = await ai.embed({
190 model: "text-embedding-ada-002",
191 input: ["Text 1", "Text 2"],
192});
193
194console.log(result.embeddings); // number[][]
195```
196
197##### `setAdapter(adapter): void`
198
199Switch providers at runtime.
200
201```typescript
202ai.setAdapter(new AnthropicAdapter({ apiKey: "..." }));
203```
204
205#### Helper Functions
206
207##### `toStreamResponse(stream): Response`
208
209Convert a `StreamChunk` async iterable to an HTTP Response with proper SSE headers.
210
211```typescript
212import { toStreamResponse } from "@tanstack/ai";
213
214const stream = ai.streamChat({ model, messages });
215return toStreamResponse(stream);
216```
217
218Returns a `Response` with:
219
220- `Content-Type: text/event-stream`
221- `Cache-Control: no-cache`
222- `Connection: keep-alive`
223- Automatic JSON encoding of chunks
224- Error handling
225- `[DONE]` completion marker
226
227##### `toReadableStream(stream): ReadableStream`
228
229Convert a `StreamChunk` async iterable to a `ReadableStream` for more control.
230
231```typescript
232import { toReadableStream } from "@tanstack/ai";
233
234const stream = ai.streamChat({ model, messages });
235const readableStream = toReadableStream(stream);
236
237// Use in custom Response
238return new Response(readableStream, {
239 headers: { "Custom-Header": "value" },
240});
241```
242
243### Types
244
245#### Message
246
247```typescript
248interface Message {
249 role: "system" | "user" | "assistant" | "tool";
250 content: string | null;
251 name?: string;
252 toolCalls?: ToolCall[]; // For assistant messages with tool calls
253 toolCallId?: string; // For tool response messages
254}
255```
256
257#### Tool
258
259```typescript
260interface Tool {
261 type: "function";
262 function: {
263 name: string;
264 description: string;
265 parameters: Record<string, any>; // JSON Schema
266 };
267 execute?: (args: any) => Promise<string> | string; // Optional: auto-execute
268}
269```
270
271**Note:** If `execute` is provided, `streamChat` will automatically:
272
2731. Detect when the AI calls this tool
2742. Execute the function with the parsed arguments
2753. Add the result back to the conversation
2764. Continue streaming the final response
277
278#### StreamChunk Variants
279
280**ContentStreamChunk**
281
282```typescript
283{
284 type: "content"
285 id: string
286 model: string
287 timestamp: number
288 delta: string // New token(s)
289 content: string // Full accumulated content
290 role?: "assistant"
291}
292```
293
294**ToolCallStreamChunk**
295
296```typescript
297{
298 type: "tool_call"
299 id: string
300 model: string
301 timestamp: number
302 toolCall: {
303 id: string
304 type: "function"
305 function: {
306 name: string
307 arguments: string // Incremental JSON arguments
308 }
309 }
310 index: number
311}
312```
313
314**DoneStreamChunk**
315
316```typescript
317{
318 type: "done"
319 id: string
320 model: string
321 timestamp: number
322 finishReason: "stop" | "length" | "content_filter" | "tool_calls" | null
323 usage?: {
324 promptTokens: number
325 completionTokens: number
326 totalTokens: number
327 }
328}
329```
330
331**ErrorStreamChunk**
332
333```typescript
334{
335 type: "error"
336 id: string
337 model: string
338 timestamp: number
339 error: {
340 message: string
341 code?: string
342 }
343}
344```
345
346### React Hooks
347
348#### useChat
349
350The `useChat` hook provides complete chat functionality for React applications.
351
352```typescript
353import { useChat } from "@tanstack/ai-react";
354
355const {
356 messages, // Current message list
357 sendMessage, // Send a message (you manage input state)
358 isLoading, // Is generating response
359 error, // Current error
360 append, // Add message programmatically
361 reload, // Reload last response
362 stop, // Stop generation
363 clear, // Clear all messages
364} = useChat({
365 api: "/api/chat",
366 onChunk: (chunk) => console.log(chunk),
367});
368
369// Simple usage
370await sendMessage("Hello!");
371```
372
373**Key Features:**
374
375- Maintains message state automatically
376- Sends messages to your API endpoint
377- Parses streaming `StreamChunk` responses
378- Updates messages in real-time
379- You control input state (v5 API style)
380
381See `packages/ai-react/README.md` for full documentation and backend examples.
382
383## Advanced Usage
384
385### Automatic Tool Calling
386
387Define tools with `execute` functions and `streamChat` handles everything automatically:
388
389```typescript
390import type { Tool } from "@tanstack/ai";
391
392const tools: Tool[] = [
393 {
394 type: "function",
395 function: {
396 name: "get_weather",
397 description: "Get current weather",
398 parameters: {
399 type: "object",
400 properties: {
401 location: { type: "string", description: "City name" },
402 },
403 required: ["location"],
404 },
405 },
406 execute: async ({ location }: { location: string }) => {
407 const weather = await fetchWeather(location);
408 return JSON.stringify(weather);
409 },
410 },
411 {
412 type: "function",
413 function: {
414 name: "calculate",
415 description: "Perform calculations",
416 parameters: {
417 type: "object",
418 properties: {
419 expression: { type: "string" },
420 },
421 required: ["expression"],
422 },
423 },
424 execute: async ({ expression }: { expression: string }) => {
425 const result = eval(expression);
426 return JSON.stringify({ result });
427 },
428 },
429];
430
431// Tools are executed automatically!
432for await (const chunk of ai.streamChat({
433 model: "gpt-3.5-turbo",
434 messages: [
435 { role: "user", content: "What's the weather in Paris and what's 2+2?" },
436 ],
437 tools,
438 toolChoice: "auto",
439 maxIterations: 5, // Limit tool calling loops (default: 5)
440})) {
441 if (chunk.type === "content") {
442 process.stdout.write(chunk.delta);
443 }
444 // Tools are called and executed automatically - no manual handling needed!
445}
446```
447
448**Key Features:**
449
450- ✅ Define `execute` function with each tool
451- ✅ `streamChat` automatically detects tool calls
452- ✅ Executes tools automatically
453- ✅ Adds results back to conversation
454- ✅ Continues until final response (up to `maxIterations`)
455- ✅ You just handle the stream - no manual tool logic!
456
457### Backend Helper - toStreamResponse
458
459Convert streaming responses to HTTP responses with one line:
460
461```typescript
462import { AI, toStreamResponse } from "@tanstack/ai";
463import { OpenAIAdapter } from "@tanstack/ai-openai";
464
465const ai = new AI(new OpenAIAdapter({ apiKey: "..." }));
466
467// Express/Node.js
468app.post("/api/chat", async (req, res) => {
469 const { messages } = req.body;
470
471 const stream = ai.streamChat({
472 model: "gpt-3.5-turbo",
473 messages,
474 tools, // Optional
475 });
476
477 // One line to convert to HTTP response!
478 return toStreamResponse(stream);
479});
480
481// Next.js App Router
482export async function POST(req: Request) {
483 const { messages } = await req.json();
484
485 const stream = ai.streamChat({
486 model: "gpt-3.5-turbo",
487 messages,
488 });
489
490 return toStreamResponse(stream);
491}
492
493// TanStack Start
494export const Route = createFileRoute("/api/chat")({
495 server: {
496 handlers: {
497 POST: async ({ request }) => {
498 const { messages } = await request.json();
499
500 return toStreamResponse(
501 ai.streamChat({ model: "gpt-3.5-turbo", messages })
502 );
503 },
504 },
505 },
506});
507```
508
509The helper handles:
510
511- ✅ Server-Sent Events formatting
512- ✅ Proper headers (Content-Type, Cache-Control, Connection)
513- ✅ JSON encoding of chunks
514- ✅ Error handling and error chunks
515- ✅ Completion marker `[DONE]`
516
517### Provider-Specific Configuration
518
519Each adapter accepts its own configuration:
520
521```typescript
522// OpenAI
523new OpenAIAdapter({
524 apiKey: string,
525 organization: string,
526 baseURL: string,
527 timeout: number,
528 maxRetries: number,
529});
530
531// Anthropic
532new AnthropicAdapter({
533 apiKey: string,
534 baseUrl: string,
535 timeout: number,
536 maxRetries: number,
537});
538
539// Ollama (local)
540new OllamaAdapter({
541 host: string, // Default: "http://localhost:11434"
542});
543
544// Gemini
545new GeminiAdapter({
546 apiKey: string,
547});
548```
549
550## Examples
551
552### Basic Chat
553
554```typescript
555import { AI } from "@tanstack/ai";
556import { OpenAIAdapter } from "@tanstack/ai-openai";
557
558const ai = new AI(new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }));
559
560const response = await ai.chat({
561 model: "gpt-3.5-turbo",
562 messages: [{ role: "user", content: "Explain quantum computing" }],
563});
564
565console.log(response.content);
566```
567
568### Streaming
569
570```typescript
571for await (const chunk of ai.streamChat({
572 model: "gpt-3.5-turbo",
573 messages: [{ role: "user", content: "Tell me a story" }],
574})) {
575 if (chunk.type === "content") {
576 process.stdout.write(chunk.delta);
577 }
578 if (chunk.type === "done") {
579 console.log(`\nTokens: ${chunk.usage?.totalTokens}`);
580 }
581}
582```
583
584### Switching Providers
585
586```typescript
587// Start with OpenAI
588const ai = new AI(new OpenAIAdapter({ apiKey: "..." }));
589
590// Switch to Anthropic - same code works!
591ai.setAdapter(new AnthropicAdapter({ apiKey: "..." }));
592const response = await ai.chat({ model: "claude-3-sonnet-20240229", messages });
593
594// Switch to local Ollama
595ai.setAdapter(new OllamaAdapter());
596const response2 = await ai.chat({ model: "llama2", messages });
597```
598
599### Text Generation
600
601```typescript
602const result = await ai.generateText({
603 model: "gpt-3.5-turbo-instruct",
604 prompt: "Write a haiku about TypeScript",
605 maxTokens: 50,
606});
607
608console.log(result.text);
609```
610
611### Summarization
612
613```typescript
614const longText = `...`; // Your long text
615
616const summary = await ai.summarize({
617 model: "gpt-3.5-turbo",
618 text: longText,
619 style: "bullet-points",
620 maxLength: 300,
621});
622
623console.log(summary.summary);
624```
625
626### Embeddings
627
628```typescript
629const result = await ai.embed({
630 model: "text-embedding-ada-002",
631 input: "Semantic search query",
632});
633
634const vector = result.embeddings[0]; // number[]
635console.log(`Dimensions: ${vector.length}`);
636```
637
638### React Hook - useChat
639
640Build chat interfaces with the `useChat` hook:
641
642```typescript
643import { useChat } from "@tanstack/ai-react";
644
645function ChatComponent() {
646 const { messages, sendMessage, isLoading, error } = useChat({
647 api: "/api/chat",
648 });
649
650 const [input, setInput] = useState("");
651
652 return (
653 <div>
654 {/* Message list */}
655 {messages.map((message) => (
656 <div key={message.id}>
657 <strong>{message.role}:</strong> {message.content}
658 </div>
659 ))}
660
661 {/* Input - you control the state */}
662 <input
663 value={input}
664 onChange={(e) => setInput(e.target.value)}
665 onKeyDown={(e) => {
666 if (e.key === "Enter") {
667 sendMessage(input);
668 setInput("");
669 }
670 }}
671 placeholder="Type a message..."
672 disabled={isLoading}
673 />
674 <button
675 onClick={() => {
676 sendMessage(input);
677 setInput("");
678 }}
679 disabled={isLoading || !input.trim()}
680 >
681 {isLoading ? "Sending..." : "Send"}
682 </button>
683
684 {error && <div>Error: {error.message}</div>}
685 </div>
686 );
687}
688```
689
690The hook handles:
691
692- Message state management
693- Sending messages to your API endpoint
694- Streaming response parsing
695- Loading and error states
696- Automatic message updates as chunks arrive
697
698**You control the input** - the hook only manages message state and API communication.
699
700## Examples
701
702### Standalone Scripts
703
704```bash
705# Install and build
706pnpm install && pnpm build
707
708# Run examples (set API keys first)
709cd examples
710
711# See basic usage of all providers
712pnpm quick-start
713
714# See streaming in action
715pnpm streaming-demo
716
717# See tool calling with both OpenAI & Anthropic (same code!)
718pnpm tool-calling
719```
720
721**Note:** Set `OPENAI_API_KEY` and/or `ANTHROPIC_API_KEY` environment variables to see the examples work with real API calls.
722
723### Full TanStack Start Application
724
725A complete chat application with tool calling, built with TanStack Start and our React hooks:
726
727```bash
728cd examples/ts-chat
729
730# Set your API key
731echo "ANTHROPIC_API_KEY=sk-ant-..." > .env
732
733# Install and run
734pnpm install
735pnpm dev
736```
737
738Open http://localhost:3000/demo/tanchat to see:
739
740- Real-time streaming chat with Claude
741- Tool calling for guitar recommendations
742- Beautiful UI with markdown formatting
743- All powered by `@tanstack/ai-react`
744
745### Interactive CLI
746
747We provide a full-featured CLI for testing and demos:
748
749```bash
750# Interactive chat
751pnpm cli chat --provider openai
752
753# Tool calling demo (OpenAI & Anthropic)
754pnpm cli tools --provider openai
755pnpm cli tools --provider anthropic
756
757# See JSON stream chunks
758pnpm cli chat --provider openai --debug
759
760# Other commands
761pnpm cli generate --provider anthropic --prompt "..."
762pnpm cli summarize --provider gemini --text "..." --style concise
763pnpm cli embed --provider openai --text "..."
764```
765
766See `examples/cli/README.md` for full CLI documentation.
767
768## Package Structure
769
770```
771@tanstack/ai/
772├── packages/
773│ ├── ai/ # Core library
774│ │ ├── types.ts # Type definitions
775│ │ ├── ai.ts # Main AI class
776│ │ ├── base-adapter.ts # Adapter base class
777│ │ └── stream-utils.ts # Streaming utilities
778│ ├── ai-openai/ # OpenAI adapter
779│ ├── ai-anthropic/ # Anthropic adapter
780│ ├── ai-ollama/ # Ollama adapter
781│ ├── ai-gemini/ # Google Gemini adapter
782│ └── ai-react/ # React hooks
783│ ├── use-chat.ts # Chat hook
784│ └── types.ts # React-specific types
785└── examples/
786 ├── cli/ # Interactive CLI demo
787 ├── ts-chat/ # TanStack Start full app
788 ├── quick-start.js # Basic examples
789 ├── streaming-demo.js # Streaming demo
790 └── tool-calling-example.js # Tool calling demo
791```
792
793## Provider Support Matrix
794
795| Feature | OpenAI | Anthropic | Ollama | Gemini |
796| --------------- | ------ | --------- | ------ | ------ |
797| Chat | ✅ | ✅ | ✅ | ✅ |
798| Streaming | ✅ | ✅ | ✅ | ✅ |
799| Tool Calling | ✅ | ✅ | ⏳ | ⏳ |
800| Text Generation | ✅ | ✅ | ✅ | ✅ |
801| Summarization | ✅ | ✅ | ✅ | ✅ |
802| Embeddings | ✅ | ❌ | ✅ | ✅ |
803
804✅ = Fully supported | ⏳ = Planned | ❌ = Not supported by provider
805
806## Development
807
808```bash
809# Install dependencies
810pnpm install
811
812# Build all packages
813pnpm build
814
815# Run in dev mode
816pnpm dev
817
818# Type checking
819pnpm typecheck
820
821# Clean build artifacts
822pnpm clean
823```
824
825## Contributing
826
827We welcome contributions! This is a community-driven project providing a truly open alternative to proprietary AI SDKs.
828
829## License
830
831MIT - Use freely, modify, share. No strings attached.
832
833## Philosophy
834
835Unlike certain companies that use open source as marketing only to lock you into paid services, @tanstack/ai is committed to remaining truly open source. No enshittification, no bait-and-switch, just honest software that respects developers.
836
837---
838
839Built with ❤️ by the open-source community.