What should I use if I need structured JSON from an LLM API?

If you need structured JSON from an LLM API, the best option is usually:

Use JSON mode or structured output / schema mode if the API supports it

These features make the model return valid JSON more reliably than plain prompting.

Common options by provider

• OpenAI: use Structured Outputs with a JSON Schema, or JSON mode if that’s all you need.
• Anthropic: use tool use / function calling to force structured arguments.
• Google Gemini: use response schema / structured output settings.
• Other APIs: look for function calling, tool calling, or schema-constrained decoding.

If your API doesn’t support structured output

Use:

1. A prompt that explicitly says “return only valid JSON”
2. A strict schema in the prompt
3. Post-validate and retry on invalid output

Best practice

If available, prefer:

• JSON Schema / structured outputs for exact structure
• Function calling if you want an action-oriented interface
• Plain JSON mode if you just need valid JSON text

Recommendation

If you’re building something production-grade, use schema-based structured output rather than relying on prompt-only JSON.

If you want, I can also show:

• a provider-by-provider comparison
• an example for OpenAI / Anthropic / Gemini
• or a robust fallback strategy for parsing JSON safely