Skip to content
Source
Sign inSign up
TanStack/schemas

Add LLM Gateway provider (#402) #405

Mergedtombeckenhammerged 1 commit intomain fromrefs/pull/405/headmerged by tombeckenham View on GitHub

tombeckenhamcommented

Closes #402.

Adds LLM Gateway as the llmgateway provider. Both of its sources are public and need no key.

What it does

  • Schemas come from https://api.llmgateway.io/openapi.json. A local sync gives 8 endpoints: chat, embeddings, images, audio speech/transcriptions, videos and moderations, with no warnings. /v1/messages, /v1/rerank, /v1/search and the other non-OpenAI paths are not classified.
  • Models come from https://api.llmgateway.io/v1/models. A local poll writes 921 rows: 697 priced, 341 linked to an upstream row.

Pricing

A bare id like gpt-oss-120b is routed by a live weighted score (docs.llmgateway.io/features/routing), so what it bills is not fixed.

  • Bare rows (gpt-oss-120b): a card only when every route quotes the same price.
  • Pinned rows (groq/gpt-oss-120b): LLM Gateway's documented provider/model syntax. There is one row per route, with that route's price and max_output.
  • Not used: the listing's top-level pricing. It is undocumented and is usually the cheapest route, not what you are billed.
  • Fails closed: a route with a nonzero price key whose unit isn't clear (image, audio, per_second, ocr_page, about 60 routes) gets no card rather than a guessed one.

sameAs

  • Where links come from: the route's own providerId + externalId, never family (which has values like "Qwen3.8 27B").
  • Which rows link: pinned rows always; bare rows only when they have a single route.
  • New namespace rows: aws-bedrock → amazon-bedrock, together-ai → together, vertex-anthropic → google-vertex-anthropic.
  • Left unmapped: bytedance and alibaba, which are ambiguous between the international and CN providers.

Shared change

ModelInfo.upstream is new. It lets a listing row state its upstream directly, and poll-models prefers it over ProviderConfig.upstreamModelIdentity(rawId). The id-only hook can't see the route.

Gap report (local)

  • llmgateway scores 0.77. Vercel is at 0.89 and OpenRouter at 0.90.
  • contextWindow is the main gap. It is left null on pinned rows of multi-route models, because the listing only gives the model's largest context and a pinned route can be a smaller deployment.
  • replayReasoningContent stays unknown. The spec accepts the field but never says when it's needed, so it isn't verified silence and there's no docs/source-silent/llmgateway.md.
  • One raw id collides. Qwen3.8-27B and qwen3.8-27b share a row id, and the existing first-wins rule keeps the first.

Checks

  • Automated: bun run test (2134 passed), lint and typecheck all pass.
  • Local end to end: the admin spec sync and the 15-minute cron poll both ran on the local dev server.
    1. Deploying with  Cloudflare Workers  Cloudflare Workers

      The latest updates on your project. Learn more about integrating Git with Workers.

      StatusNameLatest CommitPreview URLUpdated (UTC)
      ✅ Deployment successful!
      View logs
      modelschemas94dd3d0aCommit Preview URLOct 22 2026, 05:48 AM
    2. tombeckenham merged thisf53fd04

    3. tombeckenham closed this

Sign in to join this conversation.