Error Feedback Contract

The JSON shape every LLM tool sees when Synoema rejects code

Generation pipelines that feed compiler output back into the model don't want a stack trace — they want the smallest set of fields that lets the next attempt succeed. sno --errors json emits exactly that. Three fields are added on top of the standard error span: llm_hint (an actionable instruction), fixability (a difficulty label), and did_you_mean (a syntactic alternative).

Output fields

Field	Type	Description
`llm_hint`	`string?`	Actionable fix instruction phrased for an LLM — "Change the expression to produce X", not "expected X found Y"
`fixability`	`"trivial" \| "easy" \| "medium" \| "hard"`	Coarse-grained difficulty signal so the orchestrator can choose retry budget
`did_you_mean`	`string?`	Suggested alternative syntax for the most common mistakes (Python/Haskell habits)

Enriched error codes

Code	Fixability	LLM hint summary
`type_mismatch`	trivial	Change expression to produce expected type
`arity_mismatch`	trivial	Add or remove arguments
`unbound_variable`	easy	Check spelling, add parameter or definition
`infinite_type`	hard	Break cycle with an ADT wrapper
`pattern_mismatch`	easy	Check constructor names and arity
`unexpected_token`	trivial	Check syntax and follow `did_you_mean`
`unterminated_string`	trivial	Add closing quote
`no_match`	easy	Add catch-all pattern
`division_by_zero`	trivial	Guard with conditional
`linear_unused`	easy	Use the variable or remove the binding
`linear_duplicate`	easy	Use the variable exactly once
`indentation`	easy	Follow offside rule, 2-space indent

Did-you-mean rules

LLM writes	Synoema suggests
`if x then y else z`	`? x -> y : z`
`[1, 2, 3]`	`[1 2 3]` — no commas
`return x`	just `x` — expression-based
`x -> y` (lambda)	`\x -> y` — needs backslash

These map the four most-common Python/Haskell habits LLMs fall into. The did_you_mean field carries the corrected snippet, ready to splice into the next attempt.

JSON example

{
  "code": "type_mismatch",
  "severity": "error",
  "message": "expected Int, found String",
  "span": {"line": 3, "col": 14, "end_line": 3, "end_col": 20},
  "notes": ["expected: Int", "found: String"],
  "llm_hint": "Change the expression to produce Int instead of String. Common fixes: type conversion, different operator, or fix the literal value.",
  "fixability": "trivial"
}

The pre-existing fields (code, severity, message, span, notes) match the LSP error schema, so any tool that already handles LSP diagnostics absorbs Synoema's output for free.

The two output formats

Flag	Shape	Use when
`--errors json`	LSP-compatible: `{ range, message, hint, related[] }`	Default for all new tooling. Feeds clean into LSP clients and ReAct.
`--errors json-legacy`	Flat: `{ span, message, llm_hint, notes[] }`	Backwards compatibility with pre-Phase 27 tools. Will be removed in 0.2.x.

Feedback loop pipeline

tools/llm/feedback_loop.py is the reference orchestrator. It generates a candidate, runs sno --errors json check, enriches the JSON with the model's previous prompt, and retries with temperature decay.

# OpenAI provider, three retries, verbose
python tools/llm/feedback_loop.py \
       --prompt "Write factorial" \
       --provider openai \
       --retries 3 -v

# File-driven prompt with Anthropic provider
python tools/llm/feedback_loop.py \
       --prompt-file task.txt \
       --provider anthropic

Temperature schedule across retries: 1.0 → 0.5 → 0.2. The early high-temperature pass explores; subsequent passes converge on the narrow corner the compiler is asking for.

The bigger picture

The error-feedback contract is one of three legs that make ReAct auto-fix work:

GBNF constrained decoding — the model can't even sample non-Synoema tokens. Most syntactic mistakes never make it into the output.
This contract — everything that does compile but typechecks wrong gets back a fix instruction the model can act on.
The ReAct loop (sno fix file.sno) — binds the two together: Thought → Action (edit) → Observation (compiler output) until the file passes.

See /llm for the full ReAct architecture and audit-log details.

Cross-references

LLM Integration overview
Prompt Templates — what to feed the model on the way in
CLI Reference — sno fix, sno --errors json
Canonical doc on GitHub