Populate activity (and model-to-route schema binding) on grain=provider catalogs #50
Send to my agentOpenStory’s /models catalog is built on modelschemas: list rows with activity in {image,video,audio}, then fetch a per-endpoint input schema at /v1/schemas/{provider}/{activity}/{endpointId}?kind=input.
That works for fal (spec.grain: model). It fails for native xAI Grok (spec.grain: provider) even though the docs advertise discovery by activity and a schema for any provider generation endpoint.
What we hit#
GET /v1/models?provider=grok returns 12 models. Every row looks like:
{
"rawId": "grok-imagine-image-2.0",
"activity": null,
"displayName": null,
"capabilities": null
}
So this is empty:
GET https://modelschemas.com/v1/models?provider=grok&activity=image
# { "count": 0, "models": [] }
Schemas exist, but they are keyed by HTTP route, not model id:
GET /v1/schemas/grok → image: [v1/images/generations, v1/images/edits], video: [v1/videos/generations, …]
GET /v1/schemas/grok/image → shared request body; `model` is a string field (example: "grok-imagine-image")
There is no:
GET /v1/schemas/grok/image/grok-imagine-image-2.0?kind=input
Fal Grok wrappers do have both (xai/grok-imagine-image/v2.0/text-to-image, activity: "image", per-endpoint schema). Native Grok (grok-imagine-image-2.0, grok-imagine-video-1.5, …) does not.
This is not Grok-only#
Same grain: provider pattern on other catalogs:
| Provider | Model-list activity | Schema grain |
|---|---|---|
| fal | set per endpoint | per model endpoint |
| grok | null on every row | per HTTP route (v1/images/generations) |
| openai | null on every row (incl. gpt-image-2) | per HTTP route (images/generations) |
| openrouter | null (has displayName / pricing) | per HTTP route (chat/completions; some video paths are per-model) |
| gemini | set (image / video / chat / …) | still per HTTP route (:generateContent, :predictLongRunning) |
| elevenlabs | set (audio) | per HTTP route (v1/text-to-speech/{voice_id}) |
| bfl | set (image / video) | per HTTP route |
Grok and OpenAI are the worst case: you cannot even filter ?activity=image to find Imagine / gpt-image.
Ask#
- Set
activity(anddisplayName) on grain=provider model rows — at least grok + openai. Gemini already does this; grok Imagine models are obviouslyimage/video. - Bind each model to its generation route schema, so a client can go model id → input JSON Schema without hardcoding
v1/images/generations. Options we could consume:GET /v1/schemas/grok/image/grok-imagine-image-2.0?kind=input(alias onto the shared route schema, withmodelconst/enum narrowed), or- a field on the model row (
schemaEndpointId,generationPath, …) pointing at the route schema that already exists.
(1) alone would let us list native Grok on a media catalog. (2) is what we need to auto-build a Run form the way we do for fal.
Consumer#
openstory-so/openstory /models (and Images/Videos, which already call native xAI when a team key is present).
tombeckenham added labelsenhancement
tombeckenham closed this
Sign in to join this conversation.