Skip to content
Source
Sign inSign up
TanStack/ai
Open History to see the last commit

README.md

331 lines8.2 KB

1# TanStack AI Python FastAPI Example
2​
3This is a Python FastAPI server example that demonstrates agentic workflows with automatic tool execution using TanStack AI's Python SDK.
4​
5## Features
6​
7- 🤖 **Agentic workflows** with automatic tool execution
8- 🔧 **Built-in tools**: Weather lookup and timezone information
9- 📡 **SSE streaming** support for real-time responses
10- 🔄 **Agent loop strategies** with configurable iteration limits
11- ✅ Compatible with `@tanstack/ai-client`'s `fetchServerSentEvents` adapter
12- 📝 Type-safe request/response models using Pydantic
13​
14## Setup
15​
16### Prerequisites
17​
18- Python 3.8 or higher
19- pip (Python package installer)
20​
21### Step-by-Step Setup
22​
231. **Navigate to the project directory:**
24​
25```bash
26cd examples/python-fastapi
27```
28​
292. **Create a virtual environment (recommended):**
30​
31A virtual environment keeps dependencies isolated from your system Python installation.
32​
33```bash
34python3 -m venv venv
35```
36​
373. **Activate the virtual environment:**
38 - **On macOS/Linux:**
39​
40 ```bash
41 source venv/bin/activate
42 ```
43​
44 - **On Windows:**
45 ```bash
46 venv\Scripts\activate
47 ```
48​
49 You should see `(venv)` in your terminal prompt when activated.
50​
514. **Install dependencies:**
52​
53```bash
54pip install -r requirements.txt
55```
56​
57This will install all required packages (FastAPI, Anthropic SDK, Pydantic, etc.).
58​
595. **Set up environment variables:**
60​
61Copy `env.example` to `.env` and add your Anthropic API key:
62​
63```bash
64cp env.example .env
65# Edit .env and add your ANTHROPIC_API_KEY
66```
67​
686. **Run the server:**
69​
70```bash
71python anthropic-server.py
72```
73​
74Or using uvicorn directly:
75​
76```bash
77uvicorn anthropic-server:app --reload --port 8000
78```
79​
80The server will start on `http://localhost:8000`
81​
82### Deactivating the Virtual Environment
83​
84When you're done, you can deactivate the virtual environment:
85​
86```bash
87deactivate
88```
89​
90**Note:** The `venv/` directory is already included in `.gitignore`, so it won't be committed to version control.
91​
92## API Endpoints
93​
94### POST `/chat`
95​
96Streams chat responses in SSE format with automatic tool execution.
97​
98**Request Body:**
99​
100```json
102 "messages": [
103 {
104 "role": "user",
105 "content": "What's the weather in San Francisco?"
106 }
107 ],
108 "data": {
109 "model": "claude-3-5-sonnet-20241022"
110 }
112```
113​
114**Response:** Server-Sent Events stream with `StreamChunk` format:
115​
116```
117data: {"type":"content","id":"...","model":"claude-3-5-sonnet-20241022","timestamp":1234567890,"delta":"Let","content":"Let","role":"assistant"}
118​
119data: {"type":"tool_call","id":"...","model":"claude-3-5-sonnet-20241022","timestamp":1234567890,"toolCall":{"id":"call_123","type":"function","function":{"name":"get_weather","arguments":"{\"location\":\"San Francisco\"}"}},"index":0}
120​
121data: {"type":"tool_result","id":"...","model":"claude-3-5-sonnet-20241022","timestamp":1234567891,"toolCallId":"call_123","content":"{\"temperature\":62,\"conditions\":\"Foggy\",\"location\":\"San Francisco\"}"}
122​
123data: {"type":"content","id":"...","model":"claude-3-5-sonnet-20241022","timestamp":1234567892,"delta":"The","content":"The","role":"assistant"}
124​
125data: {"type":"done","id":"...","model":"claude-3-5-sonnet-20241022","timestamp":1234567893,"finishReason":"stop","usage":{"promptTokens":150,"completionTokens":75,"totalTokens":225}}
126​
127data: [DONE]
128```
129​
130### GET `/health`
131​
132Health check endpoint.
133​
134## Available Tools
135​
136The server includes two built-in tools that are automatically executed:
137​
138### 1. `get_weather`
139​
140Get weather information for a city (returns static demo data).
141​
142**Parameters:**
143​
144- `location` (required): City name (e.g., "San Francisco", "New York", "London")
145- `unit` (optional): Temperature unit - "celsius" or "fahrenheit" (default: fahrenheit)
146​
147**Supported Cities:**
148​
149- San Francisco, New York, London, Tokyo, Paris, Sydney
150​
151**Example:**
152​
153```json
155 "messages": [
156 {
157 "role": "user",
158 "content": "What's the weather in Tokyo in celsius?"
159 }
160 ]
162```
163​
164### 2. `get_time`
165​
166Get current time in a specific timezone (returns static demo data).
167​
168**Parameters:**
169​
170- `timezone` (required): Timezone code (e.g., "PST", "EST", "UTC")
171​
172**Supported Timezones:**
173​
174- UTC, PST, EST, GMT, JST, AEST
175​
176**Example:**
177​
178```json
180 "messages": [
181 {
182 "role": "user",
183 "content": "What time is it in Tokyo?"
184 }
185 ]
187```
188​
189## Usage with TanStack AI Client
190​
191This server is compatible with the TypeScript TanStack AI client:
192​
193```typescript
194import { ChatClient, fetchServerSentEvents } from '@tanstack/ai-client'
195​
196const client = new ChatClient({
197 connection: fetchServerSentEvents('http://localhost:8000/chat'),
198})
199​
200await client.sendMessage('Hello!')
201```
202​
203## StreamChunk Format
204​
205The `tanstack-ai` package emits the following `StreamChunk` types:
206​
207- **`content`**: Text content updates with delta and accumulated content
208- **`tool_call`**: Tool/function call events with arguments
209- **`tool_result`**: Results from tool execution
210- **`done`**: Stream completion with finish reason and usage stats
211- **`error`**: Error events
212- **`tool-input-available`**: Tool inputs ready for client-side execution
213- **`approval-requested`**: Tool requiring user approval
214​
215See `packages/python/tanstack-ai/src/tanstack_ai/types.py` for full type definitions.
216​
217## Agentic Flow
218​
219The server uses TanStack AI's agentic flow features:
220​
2211. **Tool Registration**: Tools are defined with JSON Schema and execute functions
2222. **Automatic Execution**: When Claude calls a tool, it's automatically executed
2233. **Result Injection**: Tool results are added to the conversation
2244. **Loop Control**: Agent can iterate up to 5 times (configurable)
2255. **Streaming**: All events (content, tool calls, results) are streamed to the client
226​
227```python
228# Example tool definition
229weather_tool = tool(
230 name="get_weather",
231 description="Get the current weather for a location",
232 input_schema={
233 "type": "object",
234 "properties": {
235 "location": {"type": "string"},
236 },
237 "required": ["location"],
238 },
239 execute=get_weather_impl,
241​
242# Use in chat with automatic execution
243async for chunk in chat(
244 adapter=adapter,
245 model="claude-3-5-sonnet-20241022",
246 messages=messages,
247 tools=[weather_tool],
248 agent_loop_strategy=max_iterations(5),
249):
250 yield format_sse_chunk(chunk)
251```
252​
253## Supported Models
254​
255- ✅ **Anthropic** (Claude models) - fully implemented
256 - claude-3-5-sonnet-20241022 (recommended for tool calling)
257 - claude-3-5-haiku-20241022
258 - claude-3-opus-20240229
259 - And more...
260​
261## Project Structure
262​
263```
264python-fastapi/
265├── anthropic-server.py # FastAPI server example
266├── requirements.txt # Python dependencies (includes tanstack-ai package)
267├── env.example # Environment variables template
268└── README.md # This file
269```
270​
271## Architecture
272​
273The server uses the `tanstack-ai` Python SDK located at `packages/python/tanstack-ai/`:
274​
275```
276FastAPI Server (anthropic-server.py)
277 ↓
278TanStack AI SDK (tanstack-ai package)
279 ↓
280AnthropicAdapter
281 ↓
282Claude API
283```
284​
285**Key Components:**
286​
287- **`anthropic-server.py`**: FastAPI endpoints and HTTP streaming
288- **`AnthropicAdapter`**: Converts Anthropic events to StreamChunks
289- **`ChatEngine`**: Orchestrates the agentic loop
290- **`ToolCallManager`**: Manages tool execution
291- **`chat()` function**: Main entry point for agentic chat
292​
293The package is installed as an editable dependency, making development easy.
294​
295## Testing the Server
296​
297### With curl:
298​
299```bash
300curl -X POST http://localhost:8000/chat \
301 -H "Content-Type: application/json" \
302 -d '{
303 "messages": [
304 {
305 "role": "user",
306 "content": "What'\''s the weather in San Francisco and what time is it in Tokyo?"
307 }
308 ]
309 }'
310```
311​
312### With the TanStack AI Client:
313​
314```typescript
315import { ChatClient, fetchServerSentEvents } from '@tanstack/ai-client'
316​
317const client = new ChatClient({
318 connection: fetchServerSentEvents('http://localhost:8000/chat'),
319})
320​
321await client.sendMessage("What's the weather in New York?")
322```
323​
324## Notes
325​
326- The server uses CORS middleware allowing all origins (configure for production)
327- Default model is `claude-3-5-sonnet-20241022` (better for tool calling)
328- Tools are automatically executed on the server side
329- Agent loop allows up to 5 iterations (configurable via `max_iterations`)
330- Error handling converts exceptions to error StreamChunks
331- All tool executions are logged for debugging