# JSON/Schema Failure

Emits invalid JSON, malformed structured data, or output that does not satisfy the required schema.

Unlike Format Failure, this is specifically about machine-parseable structure and schema conformance. Unlike Tool Argument Error, the malformed structure is the model's output rather than a submitted tool call.

## Metadata
- ID: `json-schema-failure`
- Status: published
- Taxonomy-Version: 0.0.1
- Last-Modified: 2026-06-13T09:00:00-04:00
- Category: Control (`control`)
- HTML: https://failmodes.com/failure-modes/json-schema-failure
- JSON: https://failmodes.com/failure-modes/json-schema-failure.json
- Markdown: https://failmodes.com/failure-modes/json-schema-failure.md

## Search Phrases
- invalid JSON
- malformed JSON
- broke the schema
- output didn't match the schema

## Why It Happens
1. Generation is token by token with no parser in the loop. Unless the system uses constrained decoding, nothing checks well-formedness while the output is being produced, so a single stray token invalidates the whole document.
2. Training data is full of JSON-adjacent dialects. JavaScript object literals, JSON5, and YAML teach habits like trailing commas, comments, and single quotes that are invalid in strict JSON.
3. Chat tuning wraps output in conversational packaging. Code fences, preambles like "Here is the JSON," and trailing commentary are rewarded as helpful style and break downstream parsers.
4. Structural bookkeeping degrades over long outputs. Matching brackets and tracking nesting depth across hundreds of lines is a global constraint, while attention is dominated by local context, so deep or long structures drift out of balance.

## Examples

### Malformed JSON
- Kind: fail
- Summary: Conversational packaging and dialect habits — a preamble, a code fence, an unquoted key, a trailing comma — each independently break the downstream parser.

```text
System: Return only JSON matching {"name": string, "age": integer}.
User: Extract the person from "Ada Lovelace, 36, London".
Assistant: Here's the extracted JSON:
```json
{name: "Ada Lovelace", "age": 36,}
```
```

### Valid JSON, wrong schema
- Kind: fail
- Summary: The output parses fine but violates the contract — a string where an integer is required and an extra property the schema doesn't allow.

```text
System: Return only JSON matching {"name": string, "age": integer}.
User: Extract the person from "Ada Lovelace, 36, London".
Assistant: {"name": "Ada Lovelace", "age": "36", "city": "London"}
```

### Conforming output
- Kind: ok
- Summary: Bare JSON, correct types, no extra fields, no wrapper text.

```text
System: Return only JSON matching {"name": string, "age": integer}.
User: Extract the person from "Ada Lovelace, 36, London".
Assistant: {"name": "Ada Lovelace", "age": 36}
```

## Detection Approaches
- **Schema validation**: Run every output through a real parser and schema validator before anything downstream consumes it. This catches both halves of the mode — malformed JSON and valid JSON that breaks the contract — and classifying the failures (fences, trailing commas, type mismatches, extra fields) tells you which habit to fix.
- **Parse failure monitoring**: Track validation failure rates in production, segmented by schema and output length. Structural bookkeeping degrades over long outputs, so a failure rate that climbs with response size points at nesting drift rather than dialect habits.
- **Golden-set evals**: Maintain a schema corpus weighted toward the requirements with the least training signal — enum membership, required-versus-optional fields, bans on extra properties, deep nesting — and regression-test on every model, prompt, or decoding change.

## Mitigation Approaches
- **Constrained decoding**: Use grammar-constrained sampling or the provider's native structured-output mode, which makes ill-formed tokens unsamplable and well-formedness a property of the decoder rather than a learned probability. Weigh the trade-off — forcing strict formats can degrade content quality — but for machine-consumed output this removes the malformed half of the mode outright.
- **Validate-and-retry loops**: Parse and schema-validate every output before anything downstream consumes it, and on failure regenerate with the validator's specific error in the prompt. This is the layer that catches what constrained decoding doesn't — the valid-JSON, wrong-schema half, like the stringified age and the extra city field the contract forbids.
- **Output post-processing**: Strip the conversational packaging deterministically — code fences, "Here's the JSON" preambles, trailing commentary — and apply mechanical repair for dialect habits like trailing commas and unquoted keys before parsing. Chat tuning makes the envelope the most common defect, and it's fixable without spending a regeneration.

## Related Modes
- **Format Failure** (`format-failure`)
  HTML: https://failmodes.com/failure-modes/format-failure
  JSON: https://failmodes.com/failure-modes/format-failure.json
- **Tool Argument Error** (`tool-argument-error`)
  HTML: https://failmodes.com/failure-modes/tool-argument-error
  JSON: https://failmodes.com/failure-modes/tool-argument-error.json
- **Output Truncation** (`output-truncation`)
  HTML: https://failmodes.com/failure-modes/output-truncation
  JSON: https://failmodes.com/failure-modes/output-truncation.json
- **Model Update Regression** (`model-update-regression`)
  HTML: https://failmodes.com/failure-modes/model-update-regression
  JSON: https://failmodes.com/failure-modes/model-update-regression.json