Structured Outputs and Validation

Structured output narrows an integration boundary. A response can be valid JSON yet violate the expected schema, cite nonexistent evidence, contain unsafe text, or make an unsupported claim. Parse and validate at the boundary, reject unknown fields when appropriate, and keep semantic checks separate.

Prerequisites

Know JSON objects, Python types, and the prompt contract from the previous lesson.

Learning goals

• distinguish parsing, structural validation, and semantic validation.
• reject missing, mistyped, empty, and unexpected fields.
• design explicit fallback behavior for invalid output.

Worked example

This small validator implements only the stated teaching schema. Production code should use a maintained JSON Schema implementation and pin the dialect.

import json

def validate(payload):
    errors = []
    if not isinstance(payload.get("answer"), str):
        errors.append("answer must be a string")
    citations = payload.get("citations")
    if not isinstance(citations, list) or not citations or not all(isinstance(item, str) for item in citations):
        errors.append("citations must be a non-empty list of strings")
    extra = sorted(set(payload) - {"answer", "citations"})
    if extra:
        errors.append(f"unexpected fields: {', '.join(extra)}")
    return not errors, errors

good = json.loads('{"answer":"Receipts are retained for seven years.","citations":["kb-retention"]}')
bad = json.loads('{"answer":7,"citations":[],"debug":"hidden"}')
good_valid, _ = validate(good)
bad_valid, bad_errors = validate(bad)

print(f"valid={good_valid} answer={good['answer']} citations={good['citations']}")
print(f"invalid={bad_valid} errors={bad_errors}")

Output:

valid=True answer=Receipts are retained for seven years. citations=['kb-retention']
invalid=False errors=['answer must be a string', 'citations must be a non-empty list of strings', 'unexpected fields: debug']

A retry may include validation errors, but cap retry counts and preserve the original request ID. If validation continues to fail, use a typed refusal or fallback rather than passing malformed data downstream.

Common mistakes

• Treating syntactically valid JSON as schema-valid or factually correct.
• Silently coercing types such as 7 into strings.
• Accepting unknown fields that downstream code later interprets.
• Letting model-produced URLs, SQL, filenames, or tool arguments bypass destination-specific validation.

Practice

Add an uncertainty enum with values supported, insufficient_evidence, and conflicting_evidence. Test missing and unknown values.

Next steps

Embeddings and Semantic Search introduces the representation used by many retrieval systems.

Primary sources

Checked 2026-10-16: JSON Schema Draft 2020-12, JSON Schema validation vocabulary, and Python json.