Skip to content
Source
Sign inSign up
TanStack/ai
Open History to see the last commit

skill_spec.md

193 lines19.5 KB

TanStack AI — Skill Spec

TanStack AI is a type-safe, provider-agnostic AI SDK for building AI-powered applications. It provides chat completion, streaming, isomorphic tool calling, media generation, and code execution across React, Solid, Vue, Svelte, and Preact, with adapters for OpenAI, Anthropic, Gemini, Ollama, Grok, Groq, and OpenRouter.

Domains

DomainDescriptionSkills
Building chat experiencesEnd-to-end chat — server endpoints, streaming, client hooks, message renderingchat-experience, structured-outputs
Implementing tool callingIsomorphic tools, server/client execution, approval flows, lazy discoverytool-calling
Generating media contentImage, video, TTS, transcription via activity-specific adaptersmedia-generation
Executing LLM-generated codeCode Mode sandbox setup, isolate drivers, skills systemcode-mode
Configuring adapters and modelsProvider selection, type safety, model options, runtime switchingadapter-configuration
Transport and protocolAG-UI protocol, SSE/HTTP stream, custom backend connectionscustom-backend-integration, ag-ui-protocol
Extending behaviorMiddleware hooks for analytics, caching, observabilitymiddleware

Skill Inventory

SkillTypeDomainWhat it coversFailure modes
chat-experiencecorechat-experienceschat(), useChat, streaming, SSE/HTTP responses, message formats, multimodal, thinking12
tool-callingcoretool-systemtoolDefinition(), .server()/.client(), approval, lazy discovery, rendering6
media-generationcoremedia-generationgenerateImage/Video/Speech/Transcription, summarize, generation hooks4
code-modecorecode-executioncreateCodeModeTool, isolate drivers, skills, client events4
structured-outputscorechat-experiencesoutputSchema on chat(), schema conversion3
adapter-configurationcoreadapter-managementProvider adapters, modelOptions, type safety, extending, reasoning5
custom-backend-integrationcompositiontransport-protocolCustom ConnectionAdapter, SSE/HTTP stream connections3
ag-ui-protocolcoretransport-protocolAG-UI events, StreamChunk types, SSE/NDJSON formats3
middlewarecoreextensibilityLifecycle hooks, tool caching, analytics, event firing3

Failure Mode Inventory

Chat Experience (12 failure modes)

#MistakePrioritySourceCross-skill?
1Using monolithic openai() instead of openaiText()CRITICALmigration guide—
2Using Vercel AI SDK patterns (streamText, generateText)CRITICALmaintainer interview—
3Using Vercel createOpenAI() provider patternCRITICALmaintainer interview—
4Using toResponseStream instead of toServerSentEventsResponseHIGHmigration guide—
5Passing model as separate parameter to chat()HIGHmigration guide—
6Nesting temperature/maxTokens in options objectHIGHmigration guide—
7Using providerOptions instead of modelOptionsHIGHmigration guide—
8Implementing custom SSE stream instead of toServerSentEventsResponseHIGHmaintainer interview—
9Implementing custom onEnd instead of middlewareHIGHmaintainer interviewmiddleware
10Importing from @tanstack/ai-client instead of framework packageHIGHmaintainer interview—
11Not handling RUN_ERROR events in streamingMEDIUMdocs—

Tool Calling (6 failure modes)

#MistakePrioritySourceCross-skill?
1Missing @standard-schema/spec causes tool types to be unknownCRITICALissue #235adapter-configuration
2Not passing tool definitions to both server and clientHIGHdocs + interview—
3Multiple client tools stall in same roundHIGHissue #302—
4Server tool output missing from UIMessage partsHIGHissue #176—
5Anthropic null tool input stalling loopsHIGHissue #265adapter-configuration
6Tool results always stringified, blocking multimodalMEDIUMissue #363—

Media Generation (4 failure modes)

#MistakePrioritySourceCross-skill?
1Using removed embedding() functionHIGHmigration guide—
2Forgetting toServerSentEventsResponse with TanStack StartHIGHmaintainer interview—
3Not downloading OpenAI image URLs before expiryMEDIUMdocs—
4Using stream:true for unsupported activitiesMEDIUMdocs—

Code Mode (4 failure modes)

#MistakePrioritySourceCross-skill?
1Passing API keys/secrets to sandboxCRITICALdocs—
2Not setting timeout for executionHIGHsource code—
3Node isolated-vm platform incompatibilityHIGHsource code—
4Expecting identical behavior across driversMEDIUMdocs—

Structured Outputs (3 failure modes)

#MistakePrioritySourceCross-skill?
1Trying to implement provider-specific structured output strategiesHIGHmaintainer interview—
2Using convertSchemaToJsonSchema with ArkTypeMEDIUMissue #276—
3Missing required array in OpenAI schemaMEDIUMsource code—

Adapter Configuration (5 failure modes)

#MistakePrioritySourceCross-skill?
1Missing @standard-schema/spec makes all types anyCRITICALissues #235, #191tool-calling, structured-outputs
2Confusing legacy monolithic with tree-shakeable adapterHIGHissue #407—
3Ollama silently drops systemPromptsHIGHissue #388—
4Anthropic prompt caching fails on system promptsHIGHissue #379—
5Wrong API key environment variable nameMEDIUMsource code—

Custom Backend Integration (3 failure modes)

#MistakePrioritySourceCross-skill?
1Providing both connect and subscribe+sendHIGHsource assertion—
2SSE browser connection limitsMEDIUMdocs—
3HTTP stream without reconnectionMEDIUMdocs—

AG-UI Protocol (3 failure modes)

#MistakePrioritySourceCross-skill?
1Message format doesn't match AG-UI specHIGHissue #311—
2Proxy buffering breaks SSEMEDIUMdocs—
3Assuming fixed event sequenceMEDIUMdocs—

Middleware (3 failure modes)

#MistakePrioritySourceCross-skill?
1Structured output runs outside middlewareHIGHissue #390—
2Trying to modify StreamChunks (read-only)MEDIUMdocs—
3Middleware exceptions breaking streamMEDIUMdocs—

Tensions

TensionSkillsAgent implication
Type safety vs. quick prototypingadapter-configuration <-> chat-experienceAgents skip @standard-schema/spec and use dynamic model strings, silently losing all type safety
Tree-shaking vs. API discoverabilityadapter-configuration <-> chat-experience, media-generationAgents import from package root or use wildcards, defeating bundle optimization
Server/client tool state asymmetrytool-calling <-> chat-experienceAgents build tool UI that checks part.output, working for client tools but failing for server tools
AG-UI protocol vs. internal formatag-ui-protocol <-> custom-backend-integrationAgents use TanStack AI message format, breaking interop with standard AG-UI clients

Cross-References

FromToReason
chat-experiencetool-callingMost chats include tools; tools need definitions on both server and client
chat-experienceadapter-configurationChat requires adapter selection; adapter choice affects available features
chat-experiencemiddlewareAnalytics, logging, and caching set up alongside chat
tool-callingcode-modeCode Mode is advanced alternative to tools for complex multi-step operations
structured-outputsadapter-configurationJust use outputSchema — but adapter choice determines internal strategy transparently
media-generationadapter-configurationEach media activity requires a specific activity adapter
custom-backend-integrationag-ui-protocolCustom backends must implement SSE or HTTP stream format
code-modechat-experienceCode Mode always used on top of chat; requires handling custom events

Subsystems & Reference Candidates

SkillSubsystemsReference candidates
media-generationImage (OpenAI, Gemini), Video (OpenAI), TTS (OpenAI, Gemini, ElevenLabs), Transcription (OpenAI)—
code-modeNode.js isolated-vm, QuickJS WASM, Cloudflare Workers—
adapter-configurationOpenAI, Anthropic, Gemini, Ollama, Grok, Groq, OpenRouterModel metadata (50+ models), Provider-specific modelOptions (10+ per adapter)
chat-experience——
tool-calling——
structured-outputs——
custom-backend-integration——
ag-ui-protocol——
middleware——

Remaining Gaps

SkillQuestionStatus
chat-experienceRecommended pattern for message persistence across page reloads?open
tool-callingExpected behavior when server tool throws — terminate loop or return error to LLM?open
adapter-configurationIs @standard-schema/spec intended as hard dep or peer dep? Silent degradation seems unintentional.open
ag-ui-protocolHow much AG-UI spec compliance is intended? Message format divergence — design choice or known gap?open
middlewareIs structured output running outside middleware (#390) a known limitation or bug?open
structured-outputsShould skill teach provider strategies? Resolved: always use outputSchema, adapter handles it.resolved
chat-experienceImport from ai-client or framework package? Resolved: always framework package, only ai-client for vanilla JS.resolved

Key Rules from Maintainer Interview

These rules must be embedded in every relevant skill:

  1. Always import from framework package (e.g., @tanstack/ai-react), never from @tanstack/ai-client — unless vanilla JS.
  2. Always use outputSchema on chat() for structured outputs — never implement provider-specific strategies.
  3. Always ask the user which adapter and model they want when implementing features — suggest the latest model.
  4. Always prompt the user about Code Mode when they're building chat — it's an option they should know about.
  5. Tools must be passed to both server (chat()) and client (useChat/clientTools) — this is the #1 implicit knowledge gap.
  • Core skills: chat-experience, tool-calling, media-generation, code-mode, structured-outputs, adapter-configuration, ag-ui-protocol, middleware
  • Framework skills: None needed separately — framework-specific guidance folded into chat-experience and media-generation
  • Lifecycle skills: None identified (no migration between major versions yet — library is pre-1.0)
  • Composition skills: custom-backend-integration
  • Reference files: adapter-configuration (model metadata, provider options)

Composition Opportunities

LibraryIntegration pointsComposition skill needed?
TanStack StartServer functions with generation hooks (must use toServerSentEventsResponse)No — documented within skills, but toServerSentEventsResponse is a critical integration point
React Router v7API routes for chat endpointsNo — standard framework routing
Next.jsAPI routes / App Router for chat endpointsNo — standard framework routing
ZodTool schemas, structured output schemasNo — Zod is a core dependency, not a composition