@tanstack/ai#
A powerful, open-source AI SDK with a unified interface across multiple providers. No vendor lock-in, no proprietary formats, just clean TypeScript and honest open source.
Features#
- Multi-Provider Support - OpenAI, Anthropic, Ollama, Google Gemini
- Unified API - Same interface across all providers
- Structured Streaming - JSON chunks with token deltas, tool calls, and usage stats
- Tool/Function Calling - First-class support for AI function calling (OpenAI & Anthropic)
- React Hooks - Simple
useChathook with v5 API (you control input state) - TypeScript First - Full type safety throughout
- Zero Lock-in - Switch providers at runtime without code changes
Installation#
# Core library
npm install @tanstack/ai
# Provider adapters (install what you need)
npm install @tanstack/ai-openai
npm install @tanstack/ai-anthropic
npm install @tanstack/ai-ollama
npm install @tanstack/ai-gemini
# React hooks (for frontend chat UIs)
npm install @tanstack/ai-react
Architecture#
Core Concepts#
1. Provider-Agnostic Design
All providers implement the same AIAdapter interface:
interface AIAdapter {
// Chat
chatCompletion(options: ChatCompletionOptions): Promise<ChatCompletionResult>;
chatStream(options: ChatCompletionOptions): AsyncIterable<StreamChunk>;
// Text generation
generateText(options: TextGenerationOptions): Promise<TextGenerationResult>;
// Summarization
summarize(options: SummarizationOptions): Promise<SummarizationResult>;
// Embeddings
createEmbeddings(options: EmbeddingOptions): Promise<EmbeddingResult>;
}
2. Structured Streaming
Streams return typed JSON chunks instead of raw strings:
type StreamChunk =
| ContentStreamChunk // Text tokens with delta + accumulated content
| ToolCallStreamChunk // Function call information
| DoneStreamChunk // Completion signal with token usage
| ErrorStreamChunk; // Error information
3. Adapter Pattern
The AI class wraps any adapter and provides a consistent API:
const ai = new AI(adapter); // Initialize with adapter
await ai.chat(options); // Standard chat
ai.streamChat(options); // Structured streaming
ai.setAdapter(newAdapter); // Switch providers
API Reference#
Core Library#
AI Class#
Constructor
constructor(adapter: AIAdapter)
Create an AI instance with a specific provider adapter.
Methods#
chat(options): Promise<ChatCompletionResult>#
Non-streaming chat completion.
const result = await ai.chat({
model: "gpt-3.5-turbo",
messages: [
{ role: "system", content: "You are helpful" },
{ role: "user", content: "Hello!" },
],
temperature: 0.7,
maxTokens: 1000,
});
console.log(result.content);
console.log(result.usage.totalTokens);
streamChat(options): AsyncIterable<StreamChunk>#
Structured streaming with JSON chunks. Automatically executes tools if they have execute functions.
for await (const chunk of ai.streamChat({
model: "gpt-3.5-turbo",
messages: [{ role: "user", content: "Hello" }],
tools: [...], // Optional: Tools with execute functions
toolChoice: "auto", // Optional: "auto" | "none" | { type, function }
maxIterations: 5, // Optional: Max tool calling loops (default: 5)
})) {
switch (chunk.type) {
case "content":
console.log("Delta:", chunk.delta);
console.log("Full:", chunk.content);
break;
case "tool_call":
console.log("Tool:", chunk.toolCall.function.name);
console.log("Args:", chunk.toolCall.function.arguments);
// Tool is executed automatically if execute function is defined!
break;
case "done":
console.log("Finish:", chunk.finishReason);
console.log("Usage:", chunk.usage);
break;
case "error":
console.error("Error:", chunk.error.message);
break;
}
}
Automatic Tool Execution: If tools have execute functions, streamChat will:
- Detect tool calls from the AI
- Execute the tools automatically
- Add results to the conversation
- Continue streaming the final response
- Repeat up to
maxIterationstimes
generateText(options): Promise<TextGenerationResult>#
Generate text from a prompt.
const result = await ai.generateText({
model: "gpt-3.5-turbo-instruct",
prompt: "Once upon a time",
maxTokens: 100,
});
console.log(result.text);
summarize(options): Promise<SummarizationResult>#
Summarize text.
const result = await ai.summarize({
model: "gpt-3.5-turbo",
text: "Long text to summarize...",
style: "bullet-points", // "bullet-points" | "paragraph" | "concise"
maxLength: 200,
});
console.log(result.summary);
embed(options): Promise<EmbeddingResult>#
Generate embeddings.
const result = await ai.embed({
model: "text-embedding-ada-002",
input: ["Text 1", "Text 2"],
});
console.log(result.embeddings); // number[][]
setAdapter(adapter): void#
Switch providers at runtime.
ai.setAdapter(new AnthropicAdapter({ apiKey: "..." }));
Helper Functions#
toStreamResponse(stream): Response#
Convert a StreamChunk async iterable to an HTTP Response with proper SSE headers.
import { toStreamResponse } from "@tanstack/ai";
const stream = ai.streamChat({ model, messages });
return toStreamResponse(stream);
Returns a Response with:
Content-Type: text/event-streamCache-Control: no-cacheConnection: keep-alive- Automatic JSON encoding of chunks
- Error handling
[DONE]completion marker
toReadableStream(stream): ReadableStream#
Convert a StreamChunk async iterable to a ReadableStream for more control.
import { toReadableStream } from "@tanstack/ai";
const stream = ai.streamChat({ model, messages });
const readableStream = toReadableStream(stream);
// Use in custom Response
return new Response(readableStream, {
headers: { "Custom-Header": "value" },
});
Types#
Message#
interface Message {
role: "system" | "user" | "assistant" | "tool";
content: string | null;
name?: string;
toolCalls?: ToolCall[]; // For assistant messages with tool calls
toolCallId?: string; // For tool response messages
}
Tool#
interface Tool {
type: "function";
function: {
name: string;
description: string;
parameters: Record<string, any>; // JSON Schema
};
execute?: (args: any) => Promise<string> | string; // Optional: auto-execute
}
Note: If execute is provided, streamChat will automatically:
- Detect when the AI calls this tool
- Execute the function with the parsed arguments
- Add the result back to the conversation
- Continue streaming the final response
StreamChunk Variants#
ContentStreamChunk
{
type: "content"
id: string
model: string
timestamp: number
delta: string // New token(s)
content: string // Full accumulated content
role?: "assistant"
}
ToolCallStreamChunk
{
type: "tool_call"
id: string
model: string
timestamp: number
toolCall: {
id: string
type: "function"
function: {
name: string
arguments: string // Incremental JSON arguments
}
}
index: number
}
DoneStreamChunk
{
type: "done"
id: string
model: string
timestamp: number
finishReason: "stop" | "length" | "content_filter" | "tool_calls" | null
usage?: {
promptTokens: number
completionTokens: number
totalTokens: number
}
}
ErrorStreamChunk
{
type: "error"
id: string
model: string
timestamp: number
error: {
message: string
code?: string
}
}
React Hooks#
useChat#
The useChat hook provides complete chat functionality for React applications.
import { useChat } from "@tanstack/ai-react";
const {
messages, // Current message list
sendMessage, // Send a message (you manage input state)
isLoading, // Is generating response
error, // Current error
append, // Add message programmatically
reload, // Reload last response
stop, // Stop generation
clear, // Clear all messages
} = useChat({
api: "/api/chat",
onChunk: (chunk) => console.log(chunk),
});
// Simple usage
await sendMessage("Hello!");
Key Features:
- Maintains message state automatically
- Sends messages to your API endpoint
- Parses streaming
StreamChunkresponses - Updates messages in real-time
- You control input state (v5 API style)
See packages/ai-react/README.md for full documentation and backend examples.
Advanced Usage#
Automatic Tool Calling#
Define tools with execute functions and streamChat handles everything automatically:
import type { Tool } from "@tanstack/ai";
const tools: Tool[] = [
{
type: "function",
function: {
name: "get_weather",
description: "Get current weather",
parameters: {
type: "object",
properties: {
location: { type: "string", description: "City name" },
},
required: ["location"],
},
},
execute: async ({ location }: { location: string }) => {
const weather = await fetchWeather(location);
return JSON.stringify(weather);
},
},
{
type: "function",
function: {
name: "calculate",
description: "Perform calculations",
parameters: {
type: "object",
properties: {
expression: { type: "string" },
},
required: ["expression"],
},
},
execute: async ({ expression }: { expression: string }) => {
const result = eval(expression);
return JSON.stringify({ result });
},
},
];
// Tools are executed automatically!
for await (const chunk of ai.streamChat({
model: "gpt-3.5-turbo",
messages: [
{ role: "user", content: "What's the weather in Paris and what's 2+2?" },
],
tools,
toolChoice: "auto",
maxIterations: 5, // Limit tool calling loops (default: 5)
})) {
if (chunk.type === "content") {
process.stdout.write(chunk.delta);
}
// Tools are called and executed automatically - no manual handling needed!
}
Key Features:
- ✅ Define
executefunction with each tool - ✅
streamChatautomatically detects tool calls - ✅ Executes tools automatically
- ✅ Adds results back to conversation
- ✅ Continues until final response (up to
maxIterations) - ✅ You just handle the stream - no manual tool logic!
Backend Helper - toStreamResponse#
Convert streaming responses to HTTP responses with one line:
import { AI, toStreamResponse } from "@tanstack/ai";
import { OpenAIAdapter } from "@tanstack/ai-openai";
const ai = new AI(new OpenAIAdapter({ apiKey: "..." }));
// Express/Node.js
app.post("/api/chat", async (req, res) => {
const { messages } = req.body;
const stream = ai.streamChat({
model: "gpt-3.5-turbo",
messages,
tools, // Optional
});
// One line to convert to HTTP response!
return toStreamResponse(stream);
});
// Next.js App Router
export async function POST(req: Request) {
const { messages } = await req.json();
const stream = ai.streamChat({
model: "gpt-3.5-turbo",
messages,
});
return toStreamResponse(stream);
}
// TanStack Start
export const Route = createFileRoute("/api/chat")({
server: {
handlers: {
POST: async ({ request }) => {
const { messages } = await request.json();
return toStreamResponse(
ai.streamChat({ model: "gpt-3.5-turbo", messages })
);
},
},
},
});
The helper handles:
- ✅ Server-Sent Events formatting
- ✅ Proper headers (Content-Type, Cache-Control, Connection)
- ✅ JSON encoding of chunks
- ✅ Error handling and error chunks
- ✅ Completion marker
[DONE]
Provider-Specific Configuration#
Each adapter accepts its own configuration:
// OpenAI
new OpenAIAdapter({
apiKey: string,
organization: string,
baseURL: string,
timeout: number,
maxRetries: number,
});
// Anthropic
new AnthropicAdapter({
apiKey: string,
baseUrl: string,
timeout: number,
maxRetries: number,
});
// Ollama (local)
new OllamaAdapter({
host: string, // Default: "http://localhost:11434"
});
// Gemini
new GeminiAdapter({
apiKey: string,
});
Examples#
Basic Chat#
import { AI } from "@tanstack/ai";
import { OpenAIAdapter } from "@tanstack/ai-openai";
const ai = new AI(new OpenAIAdapter({ apiKey: process.env.OPENAI_API_KEY }));
const response = await ai.chat({
model: "gpt-3.5-turbo",
messages: [{ role: "user", content: "Explain quantum computing" }],
});
console.log(response.content);
Streaming#
for await (const chunk of ai.streamChat({
model: "gpt-3.5-turbo",
messages: [{ role: "user", content: "Tell me a story" }],
})) {
if (chunk.type === "content") {
process.stdout.write(chunk.delta);
}
if (chunk.type === "done") {
console.log(`\nTokens: ${chunk.usage?.totalTokens}`);
}
}
Switching Providers#
// Start with OpenAI
const ai = new AI(new OpenAIAdapter({ apiKey: "..." }));
// Switch to Anthropic - same code works!
ai.setAdapter(new AnthropicAdapter({ apiKey: "..." }));
const response = await ai.chat({ model: "claude-3-sonnet-20240229", messages });
// Switch to local Ollama
ai.setAdapter(new OllamaAdapter());
const response2 = await ai.chat({ model: "llama2", messages });
Text Generation#
const result = await ai.generateText({
model: "gpt-3.5-turbo-instruct",
prompt: "Write a haiku about TypeScript",
maxTokens: 50,
});
console.log(result.text);
Summarization#
const longText = `...`; // Your long text
const summary = await ai.summarize({
model: "gpt-3.5-turbo",
text: longText,
style: "bullet-points",
maxLength: 300,
});
console.log(summary.summary);
Embeddings#
const result = await ai.embed({
model: "text-embedding-ada-002",
input: "Semantic search query",
});
const vector = result.embeddings[0]; // number[]
console.log(`Dimensions: ${vector.length}`);
React Hook - useChat#
Build chat interfaces with the useChat hook:
import { useChat } from "@tanstack/ai-react";
function ChatComponent() {
const { messages, sendMessage, isLoading, error } = useChat({
api: "/api/chat",
});
const [input, setInput] = useState("");
return (
<div>
{/* Message list */}
{messages.map((message) => (
<div key={message.id}>
<strong>{message.role}:</strong> {message.content}
</div>
))}
{/* Input - you control the state */}
<input
value={input}
onChange={(e) => setInput(e.target.value)}
onKeyDown={(e) => {
if (e.key === "Enter") {
sendMessage(input);
setInput("");
}
}}
placeholder="Type a message..."
disabled={isLoading}
/>
<button
onClick={() => {
sendMessage(input);
setInput("");
}}
disabled={isLoading || !input.trim()}
>
{isLoading ? "Sending..." : "Send"}
</button>
{error && <div>Error: {error.message}</div>}
</div>
);
}
The hook handles:
- Message state management
- Sending messages to your API endpoint
- Streaming response parsing
- Loading and error states
- Automatic message updates as chunks arrive
You control the input - the hook only manages message state and API communication.
Examples#
Standalone Scripts#
# Install and build
pnpm install && pnpm build
# Run examples (set API keys first)
cd examples
# See basic usage of all providers
pnpm quick-start
# See streaming in action
pnpm streaming-demo
# See tool calling with both OpenAI & Anthropic (same code!)
pnpm tool-calling
Note: Set OPENAI_API_KEY and/or ANTHROPIC_API_KEY environment variables to see the examples work with real API calls.
Full TanStack Start Application#
A complete chat application with tool calling, built with TanStack Start and our React hooks:
cd examples/ts-chat
# Set your API key
echo "ANTHROPIC_API_KEY=sk-ant-..." > .env
# Install and run
pnpm install
pnpm dev
Open http://localhost:3000/demo/tanchat to see:
- Real-time streaming chat with Claude
- Tool calling for guitar recommendations
- Beautiful UI with markdown formatting
- All powered by
@tanstack/ai-react
Interactive CLI#
We provide a full-featured CLI for testing and demos:
# Interactive chat
pnpm cli chat --provider openai
# Tool calling demo (OpenAI & Anthropic)
pnpm cli tools --provider openai
pnpm cli tools --provider anthropic
# See JSON stream chunks
pnpm cli chat --provider openai --debug
# Other commands
pnpm cli generate --provider anthropic --prompt "..."
pnpm cli summarize --provider gemini --text "..." --style concise
pnpm cli embed --provider openai --text "..."
See examples/cli/README.md for full CLI documentation.
Package Structure#
@tanstack/ai/
├── packages/
│ ├── ai/ # Core library
│ │ ├── types.ts # Type definitions
│ │ ├── ai.ts # Main AI class
│ │ ├── base-adapter.ts # Adapter base class
│ │ └── stream-utils.ts # Streaming utilities
│ ├── ai-openai/ # OpenAI adapter
│ ├── ai-anthropic/ # Anthropic adapter
│ ├── ai-ollama/ # Ollama adapter
│ ├── ai-gemini/ # Google Gemini adapter
│ └── ai-react/ # React hooks
│ ├── use-chat.ts # Chat hook
│ └── types.ts # React-specific types
└── examples/
├── cli/ # Interactive CLI demo
├── ts-chat/ # TanStack Start full app
├── quick-start.js # Basic examples
├── streaming-demo.js # Streaming demo
└── tool-calling-example.js # Tool calling demo
Provider Support Matrix#
| Feature | OpenAI | Anthropic | Ollama | Gemini |
|---|---|---|---|---|
| Chat | ✅ | ✅ | ✅ | ✅ |
| Streaming | ✅ | ✅ | ✅ | ✅ |
| Tool Calling | ✅ | ✅ | ⏳ | ⏳ |
| Text Generation | ✅ | ✅ | ✅ | ✅ |
| Summarization | ✅ | ✅ | ✅ | ✅ |
| Embeddings | ✅ | ❌ | ✅ | ✅ |
✅ = Fully supported | ⏳ = Planned | ❌ = Not supported by provider
Development#
# Install dependencies
pnpm install
# Build all packages
pnpm build
# Run in dev mode
pnpm dev
# Type checking
pnpm typecheck
# Clean build artifacts
pnpm clean
Contributing#
We welcome contributions! This is a community-driven project providing a truly open alternative to proprietary AI SDKs.
License#
MIT - Use freely, modify, share. No strings attached.
Philosophy#
Unlike certain companies that use open source as marketing only to lock you into paid services, @tanstack/ai is committed to remaining truly open source. No enshittification, no bait-and-switch, just honest software that respects developers.
Built with ❤️ by the open-source community.