How do I get structured JSON from an LLM API?

To get structured JSON from an LLM API, the best approach is to use a JSON-constrained output mode if the API supports it. If it doesn’t, you can still make it reliable with prompting, validation, and retries.

Best options

1) Use native JSON / schema mode

Many modern LLM APIs support one of these:

• JSON mode: model must output valid JSON
• Structured outputs / JSON schema: model must match a schema you define
• Function/tool calling: model returns arguments in a structured format

This is the most reliable way.

Example idea

{
  "name": "Alice",
  "age": 30,
  "skills": ["Python", "SQL"]
}

If the API supports schemas, define required fields and types so the model can’t drift.

---

2) Prompt for JSON explicitly

If structured mode isn’t available, instruct the model very clearly:

Prompt example

Return only valid JSON. No markdown, no code fences, no explanation.
The JSON must match this shape:
{ "name": string, "age": number, "skills": string[] }

This helps, but it’s less reliable than native schema enforcement.

---

3) Validate and retry

Always parse the response with a JSON parser on your side.

If parsing fails:

1. Reject the response
2. Ask the model to исправить / regenerate
3. Optionally provide the validation error

This is important even when using JSON mode, because edge cases happen.

---

4) Use a schema validator

After parsing, validate the object against a schema:

• JSON Schema
• Pydantic
• Zod
• Marshmallow
• Ajv

That lets you enforce:

• required fields
• types
• enums
• nested objects
• arrays
• string formats

---

Recommended pattern

1. Ask the model for structured output
2. Parse JSON
3. Validate against schema
4. Retry on failure
5. Never trust raw model text

---

Example workflow

If you want extraction from text:

Input

“Alice is 30 and knows Python and SQL.”

Desired JSON

{
  "name": "Alice",
  "age": 30,
  "skills": ["Python", "SQL"]
}

Use:

• a strict schema
• low temperature
• JSON-only output
• validation

---

Practical tips

• Set temperature low for consistency
• Keep schemas simple
• Avoid asking for extra prose
• Prefer tool/function calling or schema mode when available
• Strip code fences if you’re using prompt-only methods, but don’t rely on that alone

---

Rule of thumb

If your API offers:

• structured outputs → use that
• function calling → use that
• only plain text → prompt + parse + validate + retry

If you want, I can show you how to do this in OpenAI, Anthropic, or generic Python/Node code.