Structured AI Output: Valid JSON Can Still Contain the Wrong Answer

A parser-friendly response solves one important problem

An application asks a model to extract a date and receives a paragraph explaining the document instead. Structured-output features address this kind of integration friction by constraining the response to a specified shape. That can simplify downstream code and reduce format-related failures. It does not establish that the date is correct. A perfectly valid object can contain an invented value, the wrong unit, or a field taken from the wrong section. The application needs to preserve the distinction between structural validity and semantic truth.

Google and Anthropic document schema-based output capabilities with supported features and limitations. Their current guidance should be checked for the selected model and API. These features provide a stronger format contract than merely requesting JSON in prose, but the application still has to validate meaning and handle failure states. This article examines an original hypothetical document-extraction workflow. It does not rank providers or claim that a schema can guarantee the accuracy of information inferred from an ambiguous source.

The schema should express the real information model

Suppose a document contains a shipment date, an invoice date, and a payment deadline. A field named date is too vague even if it is constrained to a string. The schema should name the intended concept and define its representation. A clear field description can help the model, while application validation should enforce the expected format. Avoid asking one field to represent several meanings and hoping the surrounding prompt resolves the ambiguity consistently. The data model should make the intended distinction visible.

Units deserve the same care. A numeric amount without a currency, a distance without a unit, or a timestamp without a time-zone interpretation can be structurally complete and operationally unusable. Represent the information the next system actually needs. Where the source uses an unusual format, preserve the original text alongside a normalized value when appropriate. This gives reviewers a way to inspect the transformation rather than trusting a normalized field whose origin has been discarded.

Missing information needs a legitimate value

A required field can pressure a model to produce something even when the source does not contain the answer. Design an explicit missing-value representation that the supported schema can express. Distinguish absent, unreadable, ambiguous, and not applicable when those differences matter to the workflow. Do not use a plausible placeholder that downstream code may interpret as real data. The application should know whether a value was observed, normalized, inferred under a permitted rule, or left unresolved.

For the shipment example, a missing dispatch date should not become the invoice date merely because both look like dates. If the business permits an inference, state the rule and preserve that the result was inferred. Otherwise, route the field for review. A useful extraction system is allowed to return incomplete information when the document is incomplete. The goal is faithful structured evidence, not a filled form at any cost. This principle often reduces downstream correction work more than another round of format prompting.

Validation should include relationships among fields

Individual fields can pass type checks while contradicting each other. A line-item total may not match the quantity and unit price. An end date may precede a start date. A selected identifier may not exist in the authoritative catalog. Add domain checks where the application has reliable rules or reference data. These checks should produce clear review signals rather than silently rewriting values in a way the source does not support. The system needs to preserve evidence when a contradiction is discovered.

Keep normalization separate from correction. Converting a clearly stated date into a standard representation is different from deciding that the document must have meant another date. The first can often be automated through a defined transformation. The second may require a person or an external authoritative source. If the application merges both into one invisible cleanup step, reviewers can no longer tell whether the model extracted the value or the system invented a repair. That ambiguity makes errors harder to investigate.

Source evidence makes review faster

When a field matters, retain a reference to the page, section, or source span that supports it, where the document system permits that. A reviewer can then compare the extracted value with the evidence without searching the entire file. The evidence should correspond to the specific claim, not merely link to a large document. For image-based material, a supported location reference can be helpful, but the application should not claim precise grounding if the selected model or pipeline does not provide it reliably.

Build an evaluation set with representative documents and reviewed field values. Include poor scans, repeated headings, unusual layouts, and documents with genuinely missing information. Measure field-level correctness and the rate of unsupported filled values. A high document-level success score can conceal a systematic mistake in one important field. Break down results enough to reveal that pattern. The review burden also matters: a system that flags every field as uncertain may be safe in one sense but provide little practical assistance.

Failure handling should preserve the distinction between a rejected request, an interrupted response, and a completed object that fails semantic checks. Retrying may help some technical failures, but repeated calls do not create missing source information. Define when the workflow stops and asks for review. Avoid a loop that keeps requesting a complete object until the model eventually produces a plausible one. That can select for confidence rather than correctness.

Let structure support evidence rather than replace it

A practical implementation can treat the model’s response as a candidate record: parse it through the supported schema, apply domain checks, attach evidence, and promote it to an accepted record only when the relevant conditions are met. The conditions can differ by field and consequence. A low-risk label may be accepted automatically, while a payment destination or critical identifier may require stronger verification. This makes the automation proportionate to the work it performs.

Structured output is a valuable engineering tool because it makes the interface between a model and software more predictable. Its strongest use is in a pipeline that still asks whether the content is supported. Define the information model, permit honest missing values, validate relationships, and preserve provenance. Valid JSON is a useful beginning. A dependable application reaches the further standard of a record that is meaningful, traceable, and appropriate for the action taken from it.