Skip to content
Source
Sign inSign up
TanStack/schemas

Price ranges for routed model ids (min/max across routes) #406

Send to my agent
Closedtombeckenham opened this issue 0 comments
tombeckenhamcommented

Follow-up to #402 / #405.

Problem

A routed bare id (LLM Gateway gpt-oss-120b, 9 routes from $0.032 to $0.35 per 1M input) has no single billed price: the gateway picks a route by a live score (docs.llmgateway.io/features/routing). #405 stores a card on the bare row only when every route agrees, so most multi-route rows show no price at all, even though every route's exact price is published.

Proposal

A price range for routed ids, built only from published per-route prices. It is labelled as a range and never served as the billed price.

  • Bare row → its routes. The bare row names its pinned rows (groq/gpt-oss-120b, …). #405 already stores each one with that route's exact card, so no new price data is needed.
  • POST /v1/estimate on a bare row prices the request against every route's card and returns { min, max }, naming the route at each end. Whole-request totals cope with levers (input, output, cache) that don't move together.
  • Catalog: bare rows show the range (e.g. pricingRange, or a range form of pricing; to be decided), with factSources pointing at the route rows.
  • Other gateways: OpenRouter routes one id across several providers the same way; its per-provider endpoint listings could fill the same shape later.

Notes

  • @tanstack/ai-models is not released, so changing the pricing shape is acceptable. Note any break in the README.
  • Repo pricing rule: billed price, or a labelled estimate by a provider-published method, or nothing. A min/max over published route prices is a labelled bound, never a guessed price.

Done when

  • Bare multi-route LLM Gateway rows carry a range.
  • /v1/estimate returns min/max with the routes behind them.
  • OpenAPI, the generated client and MCP are regenerated.
  1. tombeckenham closed this

Sign in to join this conversation.