Add LLM Gateway provider (#402) #405
Merged
tombeckenhammerged 1 commit into
main fromrefs/pull/405/head·merged by tombeckenham View on GitHubCloses #402.
Adds LLM Gateway as the llmgateway provider. Both of its sources are public and need no key.
What it does#
- Schemas come from
https://api.llmgateway.io/openapi.json. A local sync gives 8 endpoints: chat, embeddings, images, audio speech/transcriptions, videos and moderations, with no warnings./v1/messages,/v1/rerank,/v1/searchand the other non-OpenAI paths are not classified. - Models come from
https://api.llmgateway.io/v1/models. A local poll writes 921 rows: 697 priced, 341 linked to an upstream row.
Pricing#
A bare id like gpt-oss-120b is routed by a live weighted score (docs.llmgateway.io/features/routing), so what it bills is not fixed.
- Bare rows (
gpt-oss-120b): a card only when every route quotes the same price. - Pinned rows (
groq/gpt-oss-120b): LLM Gateway's documentedprovider/modelsyntax. There is one row per route, with that route's price andmax_output. - Not used: the listing's top-level
pricing. It is undocumented and is usually the cheapest route, not what you are billed. - Fails closed: a route with a nonzero price key whose unit isn't clear (
image, audio,per_second,ocr_page, about 60 routes) gets no card rather than a guessed one.
sameAs#
- Where links come from: the route's own
providerId+externalId, neverfamily(which has values like"Qwen3.8 27B"). - Which rows link: pinned rows always; bare rows only when they have a single route.
- New namespace rows:
aws-bedrock→ amazon-bedrock,together-ai→ together,vertex-anthropic→ google-vertex-anthropic. - Left unmapped:
bytedanceandalibaba, which are ambiguous between the international and CN providers.
Shared change#
ModelInfo.upstream is new. It lets a listing row state its upstream directly, and poll-models prefers it over ProviderConfig.upstreamModelIdentity(rawId). The id-only hook can't see the route.
Gap report (local)#
- llmgateway scores 0.77. Vercel is at 0.89 and OpenRouter at 0.90.
contextWindowis the main gap. It is left null on pinned rows of multi-route models, because the listing only gives the model's largest context and a pinned route can be a smaller deployment.replayReasoningContentstays unknown. The spec accepts the field but never says when it's needed, so it isn't verified silence and there's nodocs/source-silent/llmgateway.md.- One raw id collides.
Qwen3.8-27Bandqwen3.8-27bshare a row id, and the existing first-wins rule keeps the first.
Checks#
- Automated:
bun run test(2134 passed), lint and typecheck all pass. - Local end to end: the admin spec sync and the 15-minute cron poll both ran on the local dev server.
- 1
tombeckenham merged thisf53fd04
tombeckenham closed this
Sign in to join this conversation.
Deploying with
Cloudflare Workers#
The latest updates on your project. Learn more about integrating Git with Workers.
View logs