AI CLI
LLM APIs & Structured Outputs — calls, tools, and JSON you can trust
How to shape a chat/messages API call and force reliable, schema-valid JSON out of a modern LLM (Claude Opus 4.x / Sonnet): system + messages anatomy, streaming, tool/function calling to pin the output shape, token and cost awareness, retries with idempotency, and recovering from refusals, truncation, and rate limits.
Quick reference — reach for these first
POST /v1/messages { model, max_tokens, messages }The core request: pick MODEL, cap MAX_TOKENS (required), send a messages[] array of {role:'user'|'assistant', content}. Headers: x-api-key, anthropic-version: 2023-06-01.body.system = "ROLE + RULES + OUTPUT CONTRACT"System is a TOP-LEVEL string, not a messages entry. Put the persona, constraints, and 'return only JSON matching the schema' contract here so it steers every turn.tool_choice = { type: 'tool', name: 'TOOL_NAME' }Force the model to call one tool whose input_schema IS your output schema. The arguments come back as a JSON object already shaped — no prose to strip.tools:[{ name, description, input_schema:{JSON_SCHEMA} }]Declare the function the model may call. input_schema is a JSON Schema with 'required' and 'type' — this is the contract the model fills in.validate(out, schema) || reprompt(error)Never trust unparsed output. Validate against the schema; on failure send ONE corrective turn echoing the validator error, then fail closed after a bounded retry.One structured-output request: force a tool call, validate, loop or finish
Text equivalent: Flow: client builds a request with system prompt, messages, tools, and tool_choice forcing one tool. The Messages API returns a response with a stop_reason. If stop_reason is tool_use, the client runs the tool and appends a tool_result, then loops back to the API. If end_turn, the client validates the JSON against the schema: valid output is used; invalid output triggers one corrective reprompt back into messages. Side notes: max_tokens too low causes truncation (raise it); a 429 means back off and retry; a refusal stop_reason is surfaced, not retried.
Common tasks
Task · Command · What to check
| Task | Command | What to check |
|---|---|---|
| Export your API key to the environment, never hard-code it | export ANTHROPIC_API_KEY="YOUR_KEY" | echo ${ANTHROPIC_API_KEY:+set} prints 'set' and the key is absent from source/git |
| Send a minimal messages call | curl https://api.anthropic.com/v1/messages -H "x-api-key: $ANTHROPIC_API_KEY" -H "anthropic-version: 2023-06-01" -H "content-type: application/json" -d '{"model":"CLAUDE_MODEL","max_tokens":256,"messages":[{"role":"user","content":"ping"}]}' | HTTP 200 and JSON body has content[0].text and stop_reason:"end_turn" |
| Move persona and output rules into the system field | jq '.system="You are a STRICT extractor. Reply ONLY by calling the tool."' | Request body has a top-level "system" string; messages[] contains no system role |
| Pin determinism for structured work | jq '.temperature=0' | Two identical requests return the same tool input JSON (modulo rare ties) |
| Define the output as a tool input_schema | tools:[{name:"emit_contact",description:"Return the extracted contact",input_schema:{type:"object",required:["name","email"],properties:{...}}}] | Schema parses as valid JSON Schema (type:object, required[], properties) |
| Force the model to call that one tool | jq '.tool_choice={type:"tool",name:"emit_contact"}' | Response stop_reason is "tool_use" and content has a tool_use block |
| Read the structured result from the tool_use block | jq '.content[] | select(.type=="tool_use") | .input' | You get a JSON object, not a string of prose to post-process |
| Validate the object against your schema before using it | python -c "import jsonschema,sys,json; jsonschema.validate(json.load(open('out.json')), json.load(open('schema.json')))" | Exit code 0; a mutated field raises ValidationError |
| Reprompt ONCE with the validator error on invalid output | messages.append({role:"user",content:"Your JSON failed: ERROR_TEXT. Re-emit valid JSON."}) | Second attempt validates; bounded to <=1–2 retries then fail closed |
| Raise max_tokens when output is cut off | jq '.max_tokens=2048' | stop_reason flips from "max_tokens" to "end_turn"/"tool_use" and JSON is complete |
| Stream tokens for latency-sensitive UIs | jq '.stream=true' # consume SSE: message_start, content_block_delta, message_stop | SSE events arrive incrementally; reassembled body equals the non-streamed result |
| Make a retried POST safe (don't double-charge) | -H "Idempotency-Key: $(uuidgen)" # only if your provider supports it | The Anthropic Messages API has NO idempotency header — dedupe client-side (skip the resend once you've got a response); Message Batches ARE idempotent. Providers like OpenAI accept an Idempotency-Key. |
| Back off on 429 using the Retry-After header | sleep $(curl -sD- ... | awk '/retry-after/{print $2}'); # else exponential: 1s,2s,4s + jitter | Transient 429s recover within the retry budget instead of hard-failing |
| Count tokens to estimate cost before sending big inputs | POST /v1/messages/count_tokens with the same model+messages+tools | Returned input_tokens matches your budget; large jobs are chunked |
Extract a contact from free text into schema-valid JSON, then evaluate it
structured-output
tools=[{"name":"emit_contact","description":"Return the extracted contact.","input_schema":{"type":"object","additionalProperties":false,"required":["name","email","priority"],"properties":{"name":{"type":"string"},"email":{"type":"string","format":"email"},"company":{"type":["string","null"]},"priority":{"type":"string","enum":["low","normal","high"]}}}}]curl https://api.anthropic.com/v1/messages -H "x-api-key: $ANTHROPIC_API_KEY" -H "anthropic-version: 2023-06-01" -H "content-type: application/json" -d '{"model":"CLAUDE_MODEL","max_tokens":512,"temperature":0,"system":"You are a strict extractor. Reply ONLY by calling emit_contact. If a field is unknown, use null; never invent an email.","tool_choice":{"type":"tool","name":"emit_contact"},"tools":[/* schema from step 2 */],"messages":[{"role":"user","content":"SAMPLE_INPUT"}]}'jq '.content[] | select(.type=="tool_use") | .input' resp.json
python -c "import json,jsonschema; o=json.load(open('out.json')); s=json.load(open('schema.json')); jsonschema.validate(o,s); assert o['email'] in open('input.txt').read(), 'email not grounded in input'; assert o['priority'] in ('low','normal','high'); print('PASS')"messages += [{"role":"user","content":"Your output failed validation: VALIDATOR_ERROR. Re-call emit_contact with valid JSON."}] # retry once, else surface error
# idempotency is provider-specific: Anthropic single Messages have none (dedupe client-side); Message Batches are idempotentTroubleshooting — symptom → first check → next check
| Symptom | First check | Establishes | Next check |
|---|---|---|---|
| Model wraps the JSON in prose or ```json fences | Are you parsing content[0].text instead of a tool_use block? | Free-text completion was used, so the shape is unenforced | Switch to tool use with tool_choice:{type:'tool',name:...} and read .input; the arguments are already a JSON object |
| JSON.parse / json.load throws on the output | Run the raw string through a JSON validator and read the exact error offset | Output is malformed or truncated, not just mis-shaped | Validate against the schema and send ONE corrective turn quoting the error; if it recurs, check stop_reason for max_tokens |
| Output ends mid-object, stop_reason:"max_tokens" | Inspect response.stop_reason and the trailing characters | The generation hit the token cap before finishing | Raise max_tokens, shrink the schema/prompt, or ask for fewer items per call |
| HTTP 429 Too Many Requests | Read the Retry-After response header and your rate-limit tier | You exceeded requests- or tokens-per-minute, not a bug in the call | Honor Retry-After, else exponential backoff with jitter; dedupe client-side (or an Idempotency-Key where the provider supports one) so retries don't duplicate work |
| Same prompt yields different JSON each run | Is temperature set, and is the shape enforced by a tool schema? | Sampling variance is leaking into structure | Set temperature:0 AND force tool use so the schema constrains the shape even when wording varies |
| stop_reason:"refusal" or a safety decline instead of data | Re-read the input for disallowed content and the stop_reason field | The model declined; no valid tool call was produced | Do not retry blindly — surface the refusal to the caller, adjust the task/input, and never fabricate the missing JSON |
| tool_choice set but model still answers in text | Confirm tool_choice.name matches a name in tools[] exactly | The forced-tool directive isn't binding to a declared tool | Fix the name mismatch; ensure input_schema is valid JSON Schema so the tool is accepted |
| Costs climb faster than expected | Compare count_tokens output to billed input/output tokens | Large or repeated context (not a pricing error) is the driver | Trim/system-cache stable context, batch items, and cap max_tokens to the smallest that fits the schema |
References
- Anthropic — Messages API referencedocs.anthropic.com
- Anthropic — Tool use (function calling)docs.anthropic.com
- Anthropic — Streaming messagesdocs.anthropic.com
- Anthropic — Handling stop reasonsdocs.anthropic.com
- Anthropic — Rate limits & error handlingdocs.anthropic.com
Examples target the Anthropic Messages API (anthropic-version 2023-06-01) as the concrete stand-in for a modern chat/messages API; CLAUDE_MODEL stands for a current model id such as a Claude Opus 4.x or Sonnet model. Field names (system, max_tokens, tools[].input_schema, tool_choice, stop_reason) follow that API; other providers use equivalent fields (e.g. response_format / function calling) with the same force-a-shape-then-validate pattern. Idempotency-Key and count_tokens availability depend on the provider.