I'm building an internal tool and need structured output from an LLM

Sure — I can help you design that.

If you want reliable structured output from an LLM for an internal tool, the safest pattern is:

1. Define a strict schema
2. Instruct the model to output only that schema
3. Validate the result server-side
4. Retry or repair if validation fails

Recommended approach

1) Use JSON as the output format

Ask the model to return only valid JSON, no markdown, no extra commentary.

Example prompt:

Return only valid JSON matching this schema:

{
  "title": string,
  "priority": "low" | "medium" | "high",
  "assignee": string | null,
  "due_date": string | null,
  "tags": string[]
}

If a field is unknown, use null. Do not include any extra keys.

2) Validate the output

On your backend, parse and validate it against your schema.

Good options:

• JSON Schema
• Pydantic (Python)
• Zod (TypeScript)
• Ajv (JavaScript)

3) Handle failure cases

LLMs may occasionally:

• add extra text
• use invalid JSON
• omit fields
• violate enum constraints

Common fixes:

• re-prompt with the validation error
• use a “repair” step
• reject and retry once or twice

---

If you’re using OpenAI-style function calling / tool calling

If your stack supports it, this is usually better than raw prompting because the model is constrained to emit arguments matching a schema.

You define a tool like:

{
  "type": "function",
  "function": {
    "name": "create_task",
    "description": "Create a task object",
    "parameters": {
      "type": "object",
      "properties": {
        "title": { "type": "string" },
        "priority": {
          "type": "string",
          "enum": ["low", "medium", "high"]
        },
        "assignee": { "type": ["string", "null"] },
        "due_date": { "type": ["string", "null"] },
        "tags": {
          "type": "array",
          "items": { "type": "string" }
        }
      },
      "required": ["title", "priority", "assignee", "due_date", "tags"],
      "additionalProperties": false
    }
  }
}

This is generally more robust than free-form JSON prompting.

---

Best practices

• Keep schemas small and explicit
• Use additionalProperties: false
• Prefer enums over free-text when possible
• Make optional fields explicit (null vs missing)
• Validate everything before using it
• Log invalid outputs for prompt improvement

---

Example end-to-end flow

1. User asks: “Create a task for reviewing the Q3 budget draft.”
2. Model returns:

{
  "title": "Review Q3 budget draft",
  "priority": "medium",
  "assignee": null,
  "due_date": null,
  "tags": ["finance", "review"]
}

3. Backend validates the object.
4. Your app uses it to create the record.

---

If you want, I can also help you with one of these:

• a Python/Pydantic example
• a TypeScript/Zod example
• an OpenAI tool-calling schema
• a prompt template for your exact use case