Price ranges for routed model ids (min/max across routes) #406
Send to my agentClosed
tombeckenham opened this issue ·0 comments
Problem#
A routed bare id (LLM Gateway gpt-oss-120b, 9 routes from $0.032 to $0.35 per 1M input) has no single billed price: the gateway picks a route by a live score (docs.llmgateway.io/features/routing). #405 stores a card on the bare row only when every route agrees, so most multi-route rows show no price at all, even though every route's exact price is published.
Proposal#
A price range for routed ids, built only from published per-route prices. It is labelled as a range and never served as the billed price.
- Bare row → its routes. The bare row names its pinned rows (
groq/gpt-oss-120b, …). #405 already stores each one with that route's exact card, so no new price data is needed. POST /v1/estimateon a bare row prices the request against every route's card and returns{ min, max }, naming the route at each end. Whole-request totals cope with levers (input, output, cache) that don't move together.- Catalog: bare rows show the range (e.g.
pricingRange, or a range form ofpricing; to be decided), withfactSourcespointing at the route rows. - Other gateways: OpenRouter routes one id across several providers the same way; its per-provider endpoint listings could fill the same shape later.
Notes#
@tanstack/ai-modelsis not released, so changing thepricingshape is acceptable. Note any break in the README.- Repo pricing rule: billed price, or a labelled estimate by a provider-published method, or nothing. A min/max over published route prices is a labelled bound, never a guessed price.
Done when#
- Bare multi-route LLM Gateway rows carry a range.
/v1/estimatereturnsmin/maxwith the routes behind them.- OpenAPI, the generated client and MCP are regenerated.
tombeckenham closed this
Sign in to join this conversation.