Skip to content
Source
Sign inSign up
TanStack/schemas

State serverTools silence only where the provider says no, and read Gemini Google Search grounding #721

Mergedtombeckenhammerged 4 commits intomain fromfix/707-server-tools-statedmerged by tombeckenham View on GitHub

tombeckenhamcommented

A null serverTools now carries factSources.serverTools: { derivation: "source-silent", sourceUrl } only when the provider states the model takes no built-in tool: a per-model "no", or a per-model list that is complete. A page that just leaves a model out states nothing, so that row stays null with no entry. Of the 66 chat rows in #707, 27 get a silent entry, 1 gets an array and 38 stay null with no entry.

Why null plus source-silent, not []

  • ModelInfo.serverTools is documented as "null when unknown", and the gap report already treats an empty list as unknown (outputModalitiesEmpty). An empty list would be a new meaning.
  • TanStack AI already reads source-silent as "this null is final" (#273), so the reader needs no new rule.

The OpenAPI spec already allowed one source-silent object for the whole field (FactSourceMap). Only two descriptions changed, so MODEL_ROW_SHAPE keeps its hash. The client was regenerated (comments only) and has a changeset.

How it works

  • serverToolsSilent(url) in fact-sources.ts is the one helper. An adapter calls it only after the per-model source loaded, parsed and said no. A source that fails still goes through tryDocs, docsFailures and absent: unavailable, so it never turns into silence.
  • ModelFactSources.serverTools now has the type Record<string, FactSource> | SourceSilentEvidence. SourceSilentEvidence moved to types.ts.
  • Rows with no per-model tool page can get exact-model ledger entries, using the #273 mechanism. serverTools is a new evidence-only ledger key (EVIDENCE_KEYS in completeness.ts). The gap report does not score it, so no score changes. A test checks this.

Per provider (66 rows)

  • openai (19): 7 silent, 12 no entry. Each model page has an Endpoints table. A page is silent only when it marks Responses, every Realtime route and Live all "Not supported", and lists no hosted tool. That covers gpt-audio, gpt-audio-2025-08-28, gpt-audio-1.5, gpt-audio-mini and gpt-audio-mini-2025-12-15, which are Chat Completions only. gpt-5-search-api and its snapshot have no model page. Their ledger entries say they always search and take no tool type (https://developers.openai.com/api/docs/guides/tools-web-search.md). The 12 realtime and live rows (gpt-realtime*, gpt-live-1*, gpt-realtime-translate, gpt-realtime-whisper) support a Realtime or Live route, and their pages list no tools, so they get no entry. Nine live pages mark Responses supported but have no "Supported tools" section (gpt-4, gpt-4-turbo, gpt-image-1 and others). I did not make that a parse error, because it would fail those pages on every poll. They state nothing, so they get no entry.
  • mistral (16): 16 silent. Each row's model page states its Features, and none of them is a conversations tool. The row is silent at that page, for example https://docs.mistral.ai/models/codestral-25-08.
  • gemini (12): 3 silent, 1 array, 8 no entry.
    • Silence comes only from a bound page whose | Capabilities | row marks no tool supported.
    • The antigravity (3), deep-research (2) and 2.5 computer-use pages have no Capabilities row, and the Gemma 4 rows (2) have no model page. These 8 get no entry.
    • gemini-flash-latest copies its changelog target's tools, as it already copies modalities and price. When the target's page failed, the alias is unavailable too.
    • Ledger entries remain for aqa, gemini-flash-lite-latest and gemini-pro-latest (its target gemini-3-pro-preview shut down on 2026-03-09).
  • byteplus (11): 1 silent, 10 no entry. The Tool use table (https://docs.byteplus.com/en/docs/ModelArk/1330310) marks MCP per model. Only seed-2-0-code-preview-260328 is marked Not Supported. The other 10 are not in the table.
  • groq (5): 5 no entry. https://console.groq.com/docs/tool-use/built-in-tools.md is not exhaustive. https://console.groq.com/docs/browser-search also names openai/gpt-oss-safeguard-20b, so a model the first page leaves out states nothing.
  • grok (3): 3 no entry. The files page lists agentic models with "e.g.", so a model it leaves out states nothing. I did not parse the multi-agent page (https://docs.x.ai/developers/model-capabilities/text/multi-agent.md). Its "Built-in Tools Support" says "e.g." and names doc tool names (code_execution, collections_search), not the request type ids (code_interpreter, file_search).

Gemini Google Search label

Google renamed the capability label from "Search grounding" to "Grounding with Google Search", for example on https://aistudio.google.com/docs/models/gemini-3.5-flash.md. The parser only knew the old label, so googleSearch was dropped on every current page. The parser now accepts both labels, tested with a trimmed real fixture (fixtures/gemini-3.5-flash.md.txt). DOCS_CACHE_VERSION goes to v45 after main's v44. The Gemini page sections and OpenAI pages are cached with new fields, so the bump also clears those entries.

Checks

  • The tests use stubbed fetches with real or trimmed real pages.
    • Gemini: the Genie 4 fixture for a Capabilities row with no tool, an antigravity-style page with no entry, and an alias whose target failed.
    • OpenAI: audio is silent; realtime, live and a gpt-4 Responses page with no tools section get no entry.
    • BytePlus: Not Supported is silent; a model missing from the table gets no entry.
    • Groq and Grok: a model the page leaves out gets no entry.
    • Mistral, ledger evidence, and the score is unchanged.
  • tsc (root and examples), oxlint --type-aware, oxfmt --check, and OpenAPI and client regeneration (no drift) all pass.
  • The workers project passes 308 tests. The full vitest run passes 2567 tests. The one failure is scripts/npm-pack.test.ts (publint). It fails in this workspace because publint is not installed and @modelschemas/rate-card@0.2.4 is not on npm, which is unrelated to this change.

Fixes #707

    1. tombeckenham pushed18336b5

    2. tombeckenham merged this13ddb28

    3. tombeckenham closed this

Sign in to join this conversation.