Read Mistral reasoning and web search from the HTML guides #694
Mergedtombeckenhammerged 1 commit into
main from693-mistral-reasoning-and-web-search-docs-pages-404-after-docs-mmerged by tombeckenham View on GitHubtombeckenham linked this to fix #693
- src/server/providers/mistral-pricing.ts line 1272@@ -1251,9 +1257,16 @@1257+ const page = mistralDocsMarkdown(markdown)1258+ if (!/allows any of our models to access the web/.test(page)) {12511259 throw new Error('mistral websearch guide: no any-model statement')12521260 }1253- if (!/work with the \[Conversations API\]/.test(markdown)) {1261+ if (!/work with the \[Conversations API\]/.test(page)) {12541262 throw new Error('mistral websearch guide: no Conversations API scope')12551263 }1256- const versions = markdown.split(/^There are two versions:$/m)[1] ?? ''1257- const ids = [...versions.matchAll(/^- `([a-z][a-z0-9_]*)`:/gm)].map(1258- (match) => match[1] ?? '',1259- )1264+ const versions = page.split(/^There are two versions:$/m)[1] ?? ''1265+ const ids: Array<string> = []1266+ for (const line of versions.split('\n')) {1267+ const match = /^- `([a-z][a-z0-9_]*)`:/.exec(line)1268+ if (match?.[1]) {1269+ ids.push(match[1])1270+ continue1271+ }1272+ if (ids.length > 0 && line.trim() !== '') breaksrc/server/providers/reasoning-config.ts lines 51 to 52@@ -25,0 +32,21 @@32+ 'https://docs.mistral.ai/inference/conversations/reasoning'33+34+/**35+ * Mistral's docs site returns a full HTML document where `llms.txt` still36+ * names a `.md` twin. The reasoning and web-search parsers read the37+ * markdown shape, so a rendered page becomes that shape: code becomes38+ * backticks, links become `[text]`, list items become bullets. A markdown39+ * body is returned unchanged.40+ */41+export function mistralDocsMarkdown(body: string): string {42+ if (!/<!doctype html|<html[\s>]/i.test(body)) return body43+ let text = body44+ .replace(/<script\b[^>]*>[\s\S]*?<\/script>/gi, '')45+ .replace(/<style\b[^>]*>[\s\S]*?<\/style>/gi, '')46+ const main = text.match(/<main\b[^>]*>[\s\S]*?<\/main>/i)47+ text = main?.[0] ?? text48+ // Code before links, so a linked code span is still a backtick.49+ // Links need brackets: the web-search parser matches `[Conversations API]`.50+ text = text51+ .replace(/<code\b[^>]*>/gi, '`')52+ .replace(/<\/code>/gi, '`')src/server/providers/reasoning-config.ts line 342@@ -272,3 +327,16 @@272327 efforts,273328 })274329 }330+ // Own effort names on the bullet itself. Applied after the shared331+ // high/none list so that list does not replace them. A list with no332+ // `none` leaves mandatory null until prose says the model always reasons.333+ const namedEfforts = new Set<string>()334+ const named =335+ /`([a-z0-9][a-z0-9.-]*)`\)?:\s*Supports\s+([^.\n]{0,200}?)\sreasoning effort/g336+ for (const match of prose.matchAll(named)) {337+ const id = match[1]338+ const efforts = [...(match[2] ?? '').matchAll(/`([a-z]+)`/g)].flatMap(339+ (found) => (found[1] ? [found[1]] : []),340+ )341+ if (!id || efforts.length === 0) continue342+ out.set(id, { mode: 'effort', mandatory: null, efforts })src/server/providers/reasoning-config.ts lines 379 to 381@@ -275,3 +361,21 @@361+ }362+ for (const id of namedEfforts) {363+ const reasoning = out.get(id)364+ if (!reasoning || reasoning.efforts?.includes('none')) continue365+ if (!statesAlwaysReasons(prose, id)) continue366+ out.set(id, { ...reasoning, mandatory: true })367+ }275368 return out276369 }277370371+/** The id's section says reasoning cannot be turned off. */372+function statesAlwaysReasons(prose: string, id: string): boolean {373+ const needle = '`' + id + '`'374+ let from = 0375+ while (from < prose.length) {376+ const at = prose.indexOf(needle, from)377+ if (at < 0) return false378+ const rest = prose.slice(at)379+ const heading = rest.search(/\n#{2,3} /)380+ const block = heading < 0 ? rest.slice(0, 1600) : rest.slice(0, heading)381+ if (/always reasons/i.test(block)) return true
tombeckenham merged this0f713db
tombeckenham merged this0f713db
tombeckenham closed this
Sign in to join this conversation.
Walkthrough
Mistral reasoning and web-search documentation retrieval now uses rendered HTML pages. The parsers normalize HTML before reading reasoning controls and web-search tool IDs. Tests and source documentation use the updated URLs and cover the guide content.
Changes: Mistral documentation parsing
src/server/providers/reasoning-config.ts,src/server/providers/reasoning-config.test.ts,src/server/providers/adapters/mistral.test.ts,docs/native-reasoning-sources.mdnone. Tests and documentation cover the guide’s model-specific controls.src/server/providers/mistral-pricing.ts,src/server/providers/mistral-pricing.test.ts,src/server/providers/adapters/mistral.test.tsEstimated code review effort: 3 (Moderate) | ~20 minutes
Change: Bug fix · Severity of issue fixed: Medium
Merge Risk:
Low · up to
0ab31The change moves Mistral doc parsing to the HTML guides, which fixes the 404s. A few edge cases in HTML conversion could produce wrong or incomplete reasoning and tool metadata for some models. These are narrow and worth fixing before or soon after merge.
Pre-merge checks:
4
#693asks for working docs URLs for both constants, a re-check that the parsers still find the reasoning table, the web-search any-model sentence, the Conversations scope, and tool IDs, and fail…#693:- URL constant updates in reasoning-config.ts and mistral-pricing.ts (per summary), and matching test fixture updates in mistral.test.ts and mistra…
Full details: Title check
Explanation
The title "Read Mistral reasoning and web search from the HTML guides" matches the main change. The PR points the reasoning and web-search constants at the rendered HTML guides, because the .md twins return 404. The title is specific and names both affected features and the source format. No title-based evidence conflicts with the changeset.
Full details: Linked Issues check
Explanation
Issue
#693asks for working docs URLs for both constants, a re-check that the parsers still find the reasoning table, the web-search any-model sentence, the Conversations scope, and tool IDs, and fail-loud behavior with no fallback.Evidence at head:
MISTRAL_REASONING_URL(src/server/providers/reasoning-config.ts) now points tohttps://docs.mistral.ai/inference/conversations/reasoning, the HTML page that serves the guide. The PR description states that the.mdtwin 404s.MISTRAL_WEBSEARCH_URL(mistral-pricing.ts) is changed to the HTML web-search page per the raw summary. I did not inspect the mistral-pricing.ts diff directly; this rests on the summary.mistralDocsMarkdownconverts rendered HTML into the markdown shape the parsers read. Non-HTML input is returned unchanged. No fallback URL is added.parseMistralReasoningruns on the converted text. Per the summary,parseMistralWebSearchToolsstill throws when the any-model sentence, Conversations scope, or all tool IDs are missing. This is consistent with the fail-loud requirement.mistral-large-4/mistral-large-4-0andzai-glm-5-3-flashis updated to the guide wording. This is a parser change needed to keep the reasoning table correct on the new page. The diff shows it: named-level bullets, the shared-ladder pairing, and mandatory marking when the section states the model always reasons.No concrete unmet coding requirement is found in the evidence reviewed. Uncertainty remains about the live page content, since it is not re-fetched during this review.
Full details: Out of Scope Changes check
Explanation
All changes connect to the Mistral docs fix in
#693:mistralDocsMarkdownHTML-to-markdown conversion. It is needed to parse the HTML pages the constants now point to.mistral-large-4mandatory case is explicitly required by the guide text.The paired-ID copy and the
statesAlwaysReasonssection heuristic go beyond the minimal URL fix. They serve the changed guide structure described in the PR, so they are not unrelated. No unrelated change is demonstrated.Finishing Touches
Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Share