Skip to content
Source
Sign inSign up
TanStack/ai

@tanstack/openai-base: strict structured output skips $defs/definitions, so recursive zod schemas are rejected #2076

Send to my agent
Openmadebyjulz(GitHub profile, opens in a new tab) opened this issue 0 comments

TanStack AI version

@tanstack/openai-base@1.0.2 (same code on main; we hit it on 1.0.0 with @tanstack/ai-openai@1.0.1)

Framework/Library version

Node 24, zod 4.6.7

Describe the bug and the steps to reproduce it

coerceStrictSchema (behind makeStructuredOutputCompatible / makeStructuredOutputCompatibleWithMap) applies the strict Structured Outputs rules to properties, items and anyOf. It never goes into $defs or definitions. Every object schema in a definition pool reaches the API unchanged: optional properties are left out of required and are not widened to null.

zod 4 puts every recursive schema into a definition pool and points to it with $ref. A recursive outputSchema is common in practice, for example a filter tree where and/or hold more filters. With such a schema, strict response_format is built from a half-converted schema. OpenAI and Azure OpenAI reject it, because strict mode requires required to list every key in properties.

This differs from #1213. That issue covers tools, which fall back to strict: false when they meet $ref/$defs. Structured output has no such fallback. The request is sent strict and fails.

Repro (no API key needed; it shows the schema that goes on the wire):

import { makeStructuredOutputCompatible } from "@tanstack/openai-base";
import { z } from "zod";

const node = z.object({
  label: z.string(),
  note: z.string().optional(),
  get children() { return z.array(node); },
});
const json = z.toJSONSchema(z.object({ root: node }), { target: "draft-7" });
console.log(JSON.stringify(makeStructuredOutputCompatible(json, json.required), null, 2));

Output (trimmed):

{
  "type": "object",
  "properties": { "root": { "$ref": "#/definitions/__schema0" } },
  "required": ["root"],
  "additionalProperties": false,
  "definitions": {
    "__schema0": {
      "type": "object",
      "properties": {
        "label": { "type": "string" },
        "note": { "type": "string" },
        "children": { "type": "array", "items": { "$ref": "#/definitions/__schema0" } }
      },
      "required": ["label", "children"],
      "additionalProperties": false
    }
  }
}

The top level is converted. Inside __schema0, note is still optional and not nullable.

Expected: each schema in $defs/definitions goes through the same strict conversion. In this example that means "note": { "type": ["string", "null"] } and "required": ["label", "note", "children"].

Fix we run as a pnpm patch (on the built dist/esm/utils/schema-converter.js, at the end of coerceStrictSchema, just before the return):

for (const defsKey of ["$defs", "definitions"]) {
  if (!isSchemaObject(result[defsKey])) continue;
  result[defsKey] = Object.fromEntries(Object.entries(result[defsKey]).map(([name, def]) => {
    if (!isSchemaObject(def)) return [name, def];
    const nested = coerceStrictSchema(def, def.required || []);
    hasUntrackableAnyOfWidening ||= nested.nullWideningMap !== undefined || nested.hasUntrackableAnyOfWidening;
    return [name, nested.schema];
  }));
}

Open point: the null-widening map is keyed by position in the tree, so it cannot follow a $ref. With the patch, a field the conversion widened to null inside a definition comes back as null, not as an omitted key. A schema that uses .optional() there then fails validation on null. We work around this with .nullish() inside recursive schemas. A full fix would carry a map per definition and resolve it at each $ref.

The snippet above runs as-is with @tanstack/openai-base and zod installed.

Terms & Code of Conduct

  • I agree to follow this project's Code of Conduct
  • I understand that if my bug cannot be reliable reproduced in a debuggable environment, it will probably not be fixed and this issue may even be closed.
  1. tanstack-source added labelsbug

  2. github-actions[bot](GitHub profile, opens in a new tab) added labelswaiting-on: maintainer

Sign in to join this conversation.