What should I use for reliable JSON output from an LLM?

If you want reliable JSON from an LLM, the best approach is usually:

1) Use structured output / function calling if available

This is the most reliable option.

• OpenAI: use Structured Outputs or function calling
• Other model providers may have similar features

Why it’s best:

• The model is constrained to a schema
• You get valid JSON much more consistently
• It’s easier to validate and retry

2) Validate with a schema

Even with structured output, always validate the result.

Good tools:

• JSON Schema validation
• Pydantic if you’re in Python
• Zod if you’re in TypeScript
• Ajv for JSON Schema in JS/TS

3) Retry on failure

If parsing or validation fails:

• send the error back to the model
• ask it to correct only the JSON
• keep retries limited

4) Keep the schema simple

Models are more reliable when:

• fields are clearly named
• types are explicit
• nested structures are not too deep
• enums are used instead of free-form text when possible

5) Don’t rely on prompt-only JSON

Plain prompting like “respond in JSON only” is much less reliable than schema-constrained output.

---

Recommended stack

• LLM structured output / function calling
• JSON Schema
• Validator + retry loop

If you want, I can show:

1. a Python example with Pydantic,
2. a TypeScript example with Zod, or
3. the best prompting pattern for fallback JSON mode.