← All cheatsheets

AI CLI

LLM APIs & Structured Outputs — calls, tools, and JSON you can trust

How to shape a chat/messages API call and force reliable, schema-valid JSON out of a modern LLM (Claude Opus 4.x / Sonnet): system + messages anatomy, streaming, tool/function calling to pin the output shape, token and cost awareness, retries with idempotency, and recovering from refusals, truncation, and rate limits.

Docs-checkedllmapistructured-outputtool-usejson-schema

Quick reference — reach for these first

Messages callPOST /v1/messages { model, max_tokens, messages }The core request: pick MODEL, cap MAX_TOKENS (required), send a messages[] array of {role:'user'|'assistant', content}. Headers: x-api-key, anthropic-version: 2023-06-01.
System promptbody.system = "ROLE + RULES + OUTPUT CONTRACT"System is a TOP-LEVEL string, not a messages entry. Put the persona, constraints, and 'return only JSON matching the schema' contract here so it steers every turn.
Structured-output requesttool_choice = { type: 'tool', name: 'TOOL_NAME' }Force the model to call one tool whose input_schema IS your output schema. The arguments come back as a JSON object already shaped — no prose to strip.
Tool schematools:[{ name, description, input_schema:{JSON_SCHEMA} }]Declare the function the model may call. input_schema is a JSON Schema with 'required' and 'type' — this is the contract the model fills in.
Validate + retryvalidate(out, schema) || reprompt(error)Never trust unparsed output. Validate against the schema; on failure send ONE corrective turn echoing the validator error, then fail closed after a bounded retry.

One structured-output request: force a tool call, validate, loop or finish

Structured-output request flow with tool use, validation, and retryA client sends system plus messages plus a forced tool schema to the Messages API; the response stop_reason branches to a tool loop, schema validation, or a one-shot corrective reprompt.Client buildssystem + messagestools + tool_choiceMessages APImodel, max_tokenstemperature 0stop_reason?Responsecontent[] blocksstop_reasontool_use: run tool, append tool_result → loopValidate JSONagainst schemaend_turnvalid → useinvalid → one corrective reprompt into messagesmax_tokens → truncation (raise it) · 429 → back off · refusal → surface, do not retry

Text equivalent: Flow: client builds a request with system prompt, messages, tools, and tool_choice forcing one tool. The Messages API returns a response with a stop_reason. If stop_reason is tool_use, the client runs the tool and appends a tool_result, then loops back to the API. If end_turn, the client validates the JSON against the schema: valid output is used; invalid output triggers one corrective reprompt back into messages. Side notes: max_tokens too low causes truncation (raise it); a 429 means back off and retry; a refusal stop_reason is surfaced, not retried.

Common tasks

Task · Command · What to check

TaskCommandWhat to check
Export your API key to the environment, never hard-code itexport ANTHROPIC_API_KEY="YOUR_KEY"echo ${ANTHROPIC_API_KEY:+set} prints 'set' and the key is absent from source/git
Send a minimal messages callcurl https://api.anthropic.com/v1/messages -H "x-api-key: $ANTHROPIC_API_KEY" -H "anthropic-version: 2023-06-01" -H "content-type: application/json" -d '{"model":"CLAUDE_MODEL","max_tokens":256,"messages":[{"role":"user","content":"ping"}]}'HTTP 200 and JSON body has content[0].text and stop_reason:"end_turn"
Move persona and output rules into the system fieldjq '.system="You are a STRICT extractor. Reply ONLY by calling the tool."'Request body has a top-level "system" string; messages[] contains no system role
Pin determinism for structured workjq '.temperature=0'Two identical requests return the same tool input JSON (modulo rare ties)
Define the output as a tool input_schematools:[{name:"emit_contact",description:"Return the extracted contact",input_schema:{type:"object",required:["name","email"],properties:{...}}}]Schema parses as valid JSON Schema (type:object, required[], properties)
Force the model to call that one tooljq '.tool_choice={type:"tool",name:"emit_contact"}'Response stop_reason is "tool_use" and content has a tool_use block
Read the structured result from the tool_use blockjq '.content[] | select(.type=="tool_use") | .input'You get a JSON object, not a string of prose to post-process
Validate the object against your schema before using itpython -c "import jsonschema,sys,json; jsonschema.validate(json.load(open('out.json')), json.load(open('schema.json')))"Exit code 0; a mutated field raises ValidationError
Reprompt ONCE with the validator error on invalid outputmessages.append({role:"user",content:"Your JSON failed: ERROR_TEXT. Re-emit valid JSON."})Second attempt validates; bounded to <=1–2 retries then fail closed
Raise max_tokens when output is cut offjq '.max_tokens=2048'stop_reason flips from "max_tokens" to "end_turn"/"tool_use" and JSON is complete
Stream tokens for latency-sensitive UIsjq '.stream=true' # consume SSE: message_start, content_block_delta, message_stopSSE events arrive incrementally; reassembled body equals the non-streamed result
Make a retried POST safe (don't double-charge)-H "Idempotency-Key: $(uuidgen)" # only if your provider supports itThe Anthropic Messages API has NO idempotency header — dedupe client-side (skip the resend once you've got a response); Message Batches ARE idempotent. Providers like OpenAI accept an Idempotency-Key.
Back off on 429 using the Retry-After headersleep $(curl -sD- ... | awk '/retry-after/{print $2}'); # else exponential: 1s,2s,4s + jitterTransient 429s recover within the retry budget instead of hard-failing
Count tokens to estimate cost before sending big inputsPOST /v1/messages/count_tokens with the same model+messages+toolsReturned input_tokens matches your budget; large jobs are chunked

Extract a contact from free text into schema-valid JSON, then evaluate it

structured-output

1
Problem: a customer email mentions a person; you need a typed {name,email,company,priority} object you can insert into a DB — no prose, no guessing.
2
Define the output AS a tool input_schema (this is the contract the model fills in)
tools=[{"name":"emit_contact","description":"Return the extracted contact.","input_schema":{"type":"object","additionalProperties":false,"required":["name","email","priority"],"properties":{"name":{"type":"string"},"email":{"type":"string","format":"email"},"company":{"type":["string","null"]},"priority":{"type":"string","enum":["low","normal","high"]}}}}]
3
Send the copy-ready request: system sets the contract, temperature 0 pins it, tool_choice forces the one tool
curl https://api.anthropic.com/v1/messages -H "x-api-key: $ANTHROPIC_API_KEY" -H "anthropic-version: 2023-06-01" -H "content-type: application/json" -d '{"model":"CLAUDE_MODEL","max_tokens":512,"temperature":0,"system":"You are a strict extractor. Reply ONLY by calling emit_contact. If a field is unknown, use null; never invent an email.","tool_choice":{"type":"tool","name":"emit_contact"},"tools":[/* schema from step 2 */],"messages":[{"role":"user","content":"SAMPLE_INPUT"}]}'
4
SAMPLE INPUT (what goes in the user message content)
5
Read the structured result from the tool_use block (already a JSON object, not text)
jq '.content[] | select(.type=="tool_use") | .input' resp.json
6
EXPECTED JSON (the tool_use .input)
7
EVALUATE — schema validation plus field/grounding checks, not just 'it parsed'
python -c "import json,jsonschema; o=json.load(open('out.json')); s=json.load(open('schema.json')); jsonschema.validate(o,s); assert o['email'] in open('input.txt').read(), 'email not grounded in input'; assert o['priority'] in ('low','normal','high'); print('PASS')"
8
On failure (ValidationError or ungrounded email) send ONE corrective turn, then fail closed
messages += [{"role":"user","content":"Your output failed validation: VALIDATOR_ERROR. Re-call emit_contact with valid JSON."}]  # retry once, else surface error
# idempotency is provider-specific: Anthropic single Messages have none (dedupe client-side); Message Batches are idempotent

Troubleshooting — symptom → first check → next check

SymptomFirst checkEstablishesNext check
Model wraps the JSON in prose or ```json fencesAre you parsing content[0].text instead of a tool_use block?Free-text completion was used, so the shape is unenforcedSwitch to tool use with tool_choice:{type:'tool',name:...} and read .input; the arguments are already a JSON object
JSON.parse / json.load throws on the outputRun the raw string through a JSON validator and read the exact error offsetOutput is malformed or truncated, not just mis-shapedValidate against the schema and send ONE corrective turn quoting the error; if it recurs, check stop_reason for max_tokens
Output ends mid-object, stop_reason:"max_tokens"Inspect response.stop_reason and the trailing charactersThe generation hit the token cap before finishingRaise max_tokens, shrink the schema/prompt, or ask for fewer items per call
HTTP 429 Too Many RequestsRead the Retry-After response header and your rate-limit tierYou exceeded requests- or tokens-per-minute, not a bug in the callHonor Retry-After, else exponential backoff with jitter; dedupe client-side (or an Idempotency-Key where the provider supports one) so retries don't duplicate work
Same prompt yields different JSON each runIs temperature set, and is the shape enforced by a tool schema?Sampling variance is leaking into structureSet temperature:0 AND force tool use so the schema constrains the shape even when wording varies
stop_reason:"refusal" or a safety decline instead of dataRe-read the input for disallowed content and the stop_reason fieldThe model declined; no valid tool call was producedDo not retry blindly — surface the refusal to the caller, adjust the task/input, and never fabricate the missing JSON
tool_choice set but model still answers in textConfirm tool_choice.name matches a name in tools[] exactlyThe forced-tool directive isn't binding to a declared toolFix the name mismatch; ensure input_schema is valid JSON Schema so the tool is accepted
Costs climb faster than expectedCompare count_tokens output to billed input/output tokensLarge or repeated context (not a pricing error) is the driverTrim/system-cache stable context, batch items, and cap max_tokens to the smallest that fits the schema

References

Examples target the Anthropic Messages API (anthropic-version 2023-06-01) as the concrete stand-in for a modern chat/messages API; CLAUDE_MODEL stands for a current model id such as a Claude Opus 4.x or Sonnet model. Field names (system, max_tokens, tools[].input_schema, tool_choice, stop_reason) follow that API; other providers use equivalent fields (e.g. response_format / function calling) with the same force-a-shape-then-validate pattern. Idempotency-Key and count_tokens availability depend on the provider.