Adapter Fallback System#
The AI SDK includes a powerful automatic fallback system that allows you to specify multiple adapters in priority order. If one adapter fails due to rate limits, service outages, or errors, the SDK automatically tries the next adapter in the list.
Features#
- ✅ Type-safe adapter ordering - TypeScript validates adapter names
- ✅ Global fallback configuration - Set default order in constructor
- ✅ Per-request override - Customize adapter order for specific calls
- ✅ Automatic retry - Seamlessly switches to next adapter on failure
- ✅ Detailed error reporting - See which adapters failed and why
- ✅ Works with all methods - chat, stream, generate, summarize, embed
Basic Usage#
Global Fallback Order#
Configure a default fallback order in the constructor:
const ai = new AI({
adapters: {
primary: new OpenAIAdapter({ apiKey: "..." }),
backup: new AnthropicAdapter({ apiKey: "..." }),
local: new OllamaAdapter(),
},
fallbackOrder: ["primary", "backup", "local"], // Try in this order
});
// Use global fallback order
const result = await ai.chat({
adapters: [], // Empty = use global fallbackOrder
model: "gpt-4",
messages: [{ role: "user", content: "Hello!" }],
});
Per-Request Fallback Order#
Override the global order for specific requests:
// Try local first, then cloud (for privacy or cost savings)
const result = await ai.chat({
adapters: ["local", "primary", "backup"], // Custom order for this request
model: "llama3",
messages: [{ role: "user", content: "Hello!" }],
});
Single Adapter Mode#
Use a specific adapter without fallback:
// No fallback - if this fails, error is thrown
const result = await ai.chat({
adapter: "primary", // Single adapter
model: "gpt-4",
messages: [{ role: "user", content: "Hello!" }],
});
Configuration#
Constructor Options#
interface AIConfig<T extends AdapterMap> {
adapters: T; // Required: Map of adapter names to adapter instances
fallbackOrder?: ReadonlyArray<keyof T & string>; // Optional: Default fallback order
}
Method Options#
All methods support two modes:
Single Adapter Mode:
{
adapter: "adapter-name", // Type-safe: must be a valid adapter name
model: "...", // Type-safe: must be valid for the adapter
// ... other options
}
Fallback Mode:
{
adapters: ["first", "second", "third"], // Type-safe array of adapter names
model: "...", // Must work with at least one adapter
// ... other options
}
Use Cases#
1. Rate Limit Protection#
const ai = new AI({
adapters: {
openai: new OpenAIAdapter({ apiKey: "..." }),
anthropic: new AnthropicAdapter({ apiKey: "..." }),
},
fallbackOrder: ["openai", "anthropic"],
});
// If OpenAI hits rate limit, automatically uses Anthropic
const result = await ai.chat({
adapters: [],
model: "gpt-4", // Will use OpenAI if available
messages: [{ role: "user", content: "Hello!" }],
});
2. Cost Optimization#
const ai = new AI({
adapters: {
local: new OllamaAdapter(), // Free local model
cloud: new OpenAIAdapter({ apiKey: "..." }), // Paid cloud model
},
fallbackOrder: ["local", "cloud"], // Try cheap option first
});
// Tries local Ollama first, falls back to OpenAI if needed
const result = await ai.chat({
adapters: [],
model: "llama3",
messages: [{ role: "user", content: "Simple task" }],
});
3. Service Availability#
const ai = new AI({
adapters: {
primary: new OpenAIAdapter({ apiKey: "..." }),
backup1: new AnthropicAdapter({ apiKey: "..." }),
backup2: new GeminiAdapter({ apiKey: "..." }),
},
fallbackOrder: ["primary", "backup1", "backup2"],
});
// Ensures your app stays online even if one service is down
const result = await ai.chat({
adapters: [],
model: "gpt-4",
messages: [{ role: "user", content: "Hello!" }],
});
4. Geographic Failover#
const ai = new AI({
adapters: {
usEast: new OpenAIAdapter({ apiKey: "...", baseURL: "https://us-east.api..." }),
usWest: new OpenAIAdapter({ apiKey: "...", baseURL: "https://us-west.api..." }),
eu: new OpenAIAdapter({ apiKey: "...", baseURL: "https://eu.api..." }),
},
fallbackOrder: ["usEast", "usWest", "eu"],
});
5. Different Strategies per Operation#
// For chat: prioritize quality (cloud first)
await ai.chat({
adapters: ["openai", "anthropic", "local"],
model: "gpt-4",
messages: [userMessage],
});
// For embeddings: prioritize speed (local first)
await ai.embed({
adapters: ["local", "openai"], // Local is faster
model: "nomic-embed-text",
input: "Text to embed",
});
Error Handling#
Automatic Retry#
The SDK automatically tries each adapter in order:
const ai = new AI({
adapters: {
first: new OpenAIAdapter({ apiKey: "..." }),
second: new AnthropicAdapter({ apiKey: "..." }),
third: new OllamaAdapter(),
},
fallbackOrder: ["first", "second", "third"],
});
try {
const result = await ai.chat({
adapters: [],
model: "gpt-4",
messages: [{ role: "user", content: "Hello!" }],
});
// Success with one of the adapters
} catch (error) {
// All three adapters failed
}
Detailed Error Information#
When all adapters fail, you get detailed information about each failure:
try {
await ai.chat({
adapters: ["first", "second"],
model: "invalid-model",
messages: [],
});
} catch (error) {
console.error(error.message);
// Output:
// All adapters failed for chat:
// - first: Rate limit exceeded (429)
// - second: Service temporarily unavailable (503)
}
Console Warnings#
During fallback, warnings are logged for debugging:
// Console output:
// [AI] Adapter "primary" failed for chat: Rate limit exceeded
// [AI] Adapter "backup" failed for chat: Service unavailable
// (Then either succeeds with next adapter or throws final error)
Type Safety#
Adapter Names are Type-Checked#
const ai = new AI({
adapters: {
openai: new OpenAIAdapter({ apiKey: "..." }),
anthropic: new AnthropicAdapter({ apiKey: "..." }),
},
fallbackOrder: ["openai", "anthropic"], // ✅ Valid
// fallbackOrder: ["gpt", "claude"], // ❌ TypeScript Error
});
// ✅ Valid adapter names
await ai.chat({
adapters: ["openai", "anthropic"],
model: "gpt-4",
messages: [],
});
// ❌ TypeScript Error - invalid adapter names
// await ai.chat({
// adapters: ["openai-gpt4", "claude"],
// model: "gpt-4",
// messages: [],
// });
Model Compatibility#
When using fallback mode, the model must be compatible with at least one adapter:
// ✅ gpt-4 is valid for OpenAI
await ai.chat({
adapters: ["openai", "anthropic"],
model: "gpt-4", // Works with OpenAI
messages: [],
});
// ⚠️ Note: In fallback mode, TypeScript allows any model from any adapter
// Runtime will fail if no adapter supports the model
Best Practices#
1. Configure Global Fallback Order#
Set a sensible default in the constructor:
const ai = new AI({
adapters: { /* ... */ },
fallbackOrder: ["primary", "secondary", "tertiary"], // Good default
});
2. Use Compatible Models#
When using fallback, ensure the model works with at least one adapter in your list:
// ❌ Bad - gpt-4 doesn't work with Anthropic or Ollama
await ai.chat({
adapters: ["anthropic", "ollama"],
model: "gpt-4", // Will fail on both adapters
messages: [],
});
// ✅ Good - choose compatible models per adapter
const modelMap = {
openai: "gpt-4",
anthropic: "claude-3-5-sonnet-20241022",
ollama: "llama3",
};
3. Order by Priority#
Put your preferred adapter first:
// Quality first
fallbackOrder: ["openai", "anthropic", "local"]
// Cost first
fallbackOrder: ["local", "openai", "anthropic"]
// Speed first
fallbackOrder: ["local", "openai", "anthropic"]
4. Handle Errors Gracefully#
try {
const result = await ai.chat({
adapters: ["primary", "backup"],
model: "gpt-4",
messages: [userMessage],
});
return result;
} catch (error) {
// All adapters failed - provide user-friendly error
return { error: "Service temporarily unavailable. Please try again later." };
}
5. Log Adapter Usage#
Track which adapters are being used:
const result = await ai.chat({
adapters: ["primary", "backup"],
model: "gpt-4",
messages: [userMessage],
});
// Log successful adapter for monitoring
console.log("Request completed successfully");
Limitations#
Model Compatibility#
In fallback mode, TypeScript can't enforce that the model works with all adapters:
// TypeScript allows this, but it will fail at runtime
await ai.chat({
adapters: ["anthropic"], // Claude models only
model: "gpt-4", // OpenAI model - will fail!
messages: [],
});
Solution: Use single-adapter mode for strict type safety, or carefully choose compatible models.
Streaming Edge Cases#
When streaming fails midway, the SDK can't retry with another adapter (stream already started):
// If stream fails after yielding chunks, error is thrown
for await (const chunk of ai.streamChat({
adapters: ["primary", "backup"],
model: "gpt-4",
messages: [],
})) {
// If this fails after first chunk, backup won't be tried
}
Solution: Fallback only applies before streaming starts.
Examples#
See /examples/adapter-fallback-demo.ts for comprehensive examples including:
- Global vs per-request fallback
- Rate limit handling
- Cost optimization
- Service availability
- Error handling
- Type safety demonstrations
API Reference#
AIConfig#
interface AIConfig<T extends AdapterMap> {
adapters: T;
fallbackOrder?: ReadonlyArray<keyof T & string>;
}
Method Options#
All methods accept either:
// Single adapter mode
{
adapter: keyof T & string;
model: ExtractModels<T[adapter]>;
// ... method-specific options
}
// Fallback mode
{
adapters: ReadonlyArray<keyof T & string>;
model: UnionOfModels<T>;
// ... method-specific options
}
Migration Guide#
From Single Adapter#
Before:
const ai = new AI({
adapters: { openai: new OpenAIAdapter({ apiKey: "..." }) }
});
await ai.chat({
adapter: "openai",
model: "gpt-4",
messages: [],
});
After (with fallback):
const ai = new AI({
adapters: {
openai: new OpenAIAdapter({ apiKey: "..." }),
anthropic: new AnthropicAdapter({ apiKey: "..." }),
},
fallbackOrder: ["openai", "anthropic"],
});
await ai.chat({
adapters: [], // Use fallback
model: "gpt-4",
messages: [],
});
See Also#
- Type Safety Guide - Model validation per adapter
- Examples - Working code examples
- API Documentation - Full API reference