#JSONMend

#Features

#0.5.0: "may I execute this" in one call

• SafetyPolicy: five named profiles — permissive, interactive, execution, agent, strict — over the safety-relevant Options. Options::with_policy replaces only the safety knobs and keeps the protocol ones (delta stream, prose handling), and SafetyPolicy::of_options reads a set of options back to the profile it is (or None).
• analyze returns the whole plan:

  let plan = @jsonmend.analyze(text,
    options=@jsonmend.Options::for_agent(),
    schema=@jsonmend.Schema::parse("{!query:string,!limit:integer}"))
  plan.decision          // Safe | CosmeticRepair | LossyRepair | Ambiguous | Invalid
  plan.is_executable()   // the only question a gate has to ask
  plan.repairs           // rule, action, position, confidence class, information loss
  plan.prompt()          // ready to feed back to the model when it is not executable
  plan.to_json()         // machine channel: decision/exactness/value/retry/repairs

  analyze never raises — a refusal is part of the return value — and costs 1.00–1.03× a plain repair call.
• Schema + semantic_readiness: declare {!query:string,!limit:integer} and get two fractions for the same stream — syntax ready and schema ready — plus the members that are missing or of the wrong kind.
• Self-healing loop example: code/examples/agent_loop feeds plan.prompt() back to a scripted "model" and retries; four episodes cover E011 ambiguity, E010 truncation, E070 skeleton mismatch, and giving up when a retry makes no progress.
• Live demo: static/demo/live.html replays four real scenarios frame by frame (truncated → retry → executed; ambiguous → retry → executed; single reading → executed; long tail → ready at 19%).

#Everything else

• One-shot repair with repair: repaired text, a list of repair records, and the completeness of the input.
• One-shot parsing with parse: the repaired input as a Json tree plus the same records.
• Streaming partial parsing with PartialParser: feed (or feed_bytes) any number of chunks, take snapshot() at any time, then finish().
• Repair records for every change: stable code (R001…R022), position (UTF-16 offset plus 1-based line/column), severity, and a fixed English message.
• Exactness labels: every result carries Exact / Cosmetic / Lossy, derived from the severities of the records. Options::max_loss turns the first repair that exceeds the budget into a fatal E010, so "repair, then execute the arguments" cannot happen silently.
• Ambiguity gate: a repair that admits a second, equally self-consistent reading (["a" "b"], ["a"+"b"], [fast], {"a":1} {"b":2}) is recorded with an Ambiguity class and refused with E011 under Options::reject_ambiguous / Options::for_execution(), before the exactness gate. 10/10 adversarial cases are flagged and refused, while both reference libraries answer all 10 with a silent value (docs/EVAL_AMBIGUITY.md).
• Reject as feedback: MendError::retry_hint (and --retry-hint) turns a refusal into a prompt fragment — error code, repair code, position, the second reading, a bounded context window with a caret — that can be fed straight back to the model that produced the input.
• Field readiness (FR-29): field_readiness reports, for every top-level member, the input fraction at which its value is already final, and FieldTracker does the same from the delta stream, so a caller can stop the generation as soon as the fields it needs are settled. The decile curve replaces the "95% coverage" figure that an object made structurally uninformative.
• Incremental deltas: with Options::collect_deltas, drain_deltas() returns Set / AppendText / Close operations with RFC 6901 JSON Pointer paths, so a consumer can patch its own view instead of re-reading snapshots.
• LLM provider adapters in the jsonmend/llm sub-package (experimental): Anthropic fine-grained tool streaming and OpenAI-compatible tool-call deltas, an SSE framer, and a text stream that reads the JSON out of delta.text / delta.content. The end of a call is reported as Complete, MaxTokens, Interrupted or ProviderError with the provider's own stop_reason verbatim; ping and unknown event types are counted and ignored. jsonmend llm on the command line.
• Strict mode (Mode::Strict) turns the first repair into a fatal E010, and per-rule switches disable individual rules the same way.
• Resource limits: nesting depth, total input length, and single-token length (E020/E021/E022).
• Policies for duplicate keys, non-finite numbers, multiple root values, Markdown fences, leading prose, ambiguous quotes, and output indentation.
• Multi-backend: the library is pure MoonBit with no regex and no recursion; bench runs on native and js.
• CLI and browser demo: cmd/jsonmend for the command line, examples/web for the browser.

#Why you need it

#Install

moon add zhanshishan/jsonmend

import {
  "zhanshishan/jsonmend" @jsonmend,
}

#Quick start

try {
  let result = @jsonmend.repair("{'a': 1,}")   // RepairResult
  println(result.text)                         // {"a":1}
  println(result.repairs.length())             // number of repair records
} catch {
  e => println(e.to_string())
}

#Streaming example

try {
  let p = @jsonmend.PartialParser::new()
  p.feed("{\"name\":\"Mo")
  let s1 = p.snapshot()
  match s1.value {
    Some(v) => println(v.stringify())          // {"name":"Mo"}
    None => println("<none>")
  }
  println(s1.completeness is @jsonmend.Partial) // true
  p.feed("n\"}")
  let done = p.finish()
  println(done.text)                            // {"name":"Mon"}
  println(done.completeness is @jsonmend.Complete) // true
} catch {
  e => println(e.to_string())
}

try {
  let p = @jsonmend.PartialParser::new(options={
    ..@jsonmend.Options::default(),
    incomplete_number: @jsonmend.TrimToValid,
    pending_key: @jsonmend.NullKey,
  })
  p.feed_bytes(b"[1, 1.")
  let s = p.snapshot()
  // s.value is [1,1]: the unfinished number is trimmed to its longest valid prefix.
  p.feed_bytes(b"5]")
  let done = p.finish()
  println(done.text)  // [1,1.5]
} catch {
  e => println(e.to_string())
}

#End-to-end example: refuse, feed back, then execute

== first attempt
model   : {"city":["Paris" "London"]}
gate    : E011 adjacent_strings - the gate cannot pick a reading
refused : the gate refused to pick a reading
prompt  :
  JSONMend refused the arguments: E011 R006 InsertComma at line 1 column 18 (offset 17).
  Reason: two adjacent strings may be two elements or one concatenated string
  Ambiguity: adjacent_strings
  Re-emit the arguments as valid JSON with exactly one reading.

== re-emitted
model   : {"city":["Paris","London"]}
execute : {"city":["Paris","London"]}

#Options

#Repair rules overview

#Repair records and error codes

#Snapshot semantics

• Stable prefix. Values that are already closed — complete strings, numbers, literals, and containers — never change in later snapshots. Only the last unfinished element can change.
• Monotonicity. The snapshot sequence grows monotonically. Choosing TrimToValid deliberately breaks monotonicity for unfinished numbers (a snapshot showing 12 can later become 123); this is documented behaviour, not a bug.
• Unfinished number. With the default incomplete_number = Drop, an unfinished number is not shown at all. With TrimToValid, the longest valid prefix is shown, or nothing if there is no valid prefix.
• Unfinished key. A key whose closing quote has not arrived never appears. A dangling key (key plus colon, no value) is omitted with the default pending_key = OmitKey, or shown as "key": null with NullKey.
• Unfinished string. The received prefix appears; an incomplete escape sequence is held back until it is complete.
• Literal prefix. A partial literal such as tr is not shown in a snapshot; it is completed at finish.
• Open containers. An open array or object appears with the content confirmed so far, and completeness is Partial.
• open_path locates the value currently being built as Array[PathSegment], where a segment is Key(String) or Index(Int32). Synthetic frames, such as the array wrapper added by multi_root = AsArray, are skipped.

#Command line and demo

moon build cmd/jsonmend --target native --release
echo "{'a': 1,}" | ./_build/native/release/build/cmd/jsonmend/jsonmend.exe

jsonmend [选项] [文件]

读取文件或标准输入，把修复后的 JSON 写到标准输出。

  --strict            严格模式：需要修复就报错（退出码 5）
  --check             只检查，不输出；需要修复时退出码 1
  --report            把修复记录写到标准错误
  --indent N          输出缩进空格数，默认紧凑输出
  --multi-root MODE   first、array 或 error，默认 first
  --skip-prose        跳过 JSON 之前的说明文字
  --quote-lookahead   启用引号歧义前瞻
  --max-repairs N     最多记录多少条修复记录，默认 10000
  --max-loss LEVEL    精确度上限：exact、cosmetic 或 lossy，默认 lossy
  --for-execution     执行档：等价于 --max-loss cosmetic --reject-ambiguous
  --reject-ambiguous  拒绝有第二种读法的修复，报 E011（退出码 5）
  --retry-hint        拒绝时在标准错误上打印可直接回灌模型的提示
  --chunk N           分块喂入：每次喂 N 个码元（≤1048576），用来验证流式路径（默认一次喂完）
  --explain           在标准错误上逐条解释修复，并打印精确度小结
  --batch             逐行读取 {"id","input"} JSON，逐行输出结果 JSON
                      与 --chunk N 同用可把每条输入按 N 字节分块喂入

jsonmend readiness [选项] [文件]
  打印字段就绪曲线：每个顶级字段在输入消耗到多少时已定稿。
  --step N            每 N 个码元观察一次，默认按长度自适应

jsonmend llm [选项] [文件]
  把模型的事件流喂给 provider 适配器，每个工具调用输出一行结果 JSON。
  --provider NAME     anthropic（默认）或 openai
  --sse               输入是 SSE 文本（默认按每行一个事件 JSON 读取）
  --text              读取文本增量（delta.content / text_delta）而不是工具调用参数（配 --skip-prose 可跳过前言）
  --follow            每次增量都输出一行，而不是只在结束时输出
  -h, --help          显示帮助
  --version           显示版本

文件为 `-` 或缺省时读取标准输入。

退出码：0 成功（可能有修复）；1 --check 发现需要修复；2 用法错误；
        3 输入无效；4 资源限制；5 严格模式违规；70 内部错误。

#Testing

cd code
moon test                      # default backend (wasm-gc)
moon test --target native      # host backend
moon test --target js          # JavaScript backend
bash tests/cli_smoke.sh        # 62 command line assertions
bash tests/examples.sh         # example inputs vs code/examples/cases/*.expected
node ../code/tests/demo_smoke.mjs   # the committed browser bundle still answers
python3 tests/diff/run_diff.py --cli ./_build/native/release/build/cmd/jsonmend/jsonmend.exe
python3 tests/eval/run_eval.py --cli ./_build/native/release/build/cmd/jsonmend/jsonmend.exe
python3 tests/eval/run_eval_paths.py --cli ./_build/native/release/build/cmd/jsonmend/jsonmend.exe
python3 tests/eval/check_amb_guard.py    # the adversarial suite's guard still fires
python3 tests/eval/run_ambiguity.py --cli ./_build/native/release/build/cmd/jsonmend/jsonmend.exe
python3 tests/eval/refcompare.py --cli ./_build/native/release/build/cmd/jsonmend/jsonmend.exe
python3 tests/gen_llm_fixtures.py --check
python3 tests/memory_baseline.py --cli ./_build/native/release/build/cmd/jsonmend/jsonmend.exe
python3 tests/eval/run_independent.py    # vendored nst/JSONTestSuite, independent of our own corpus
python3 tests/eval/run_readiness.py --cli ./_build/native/release/build/cmd/jsonmend/jsonmend.exe --check
moon run examples/agent_loop --target native --release

#Differences from jsonrepair

#Performance

moon run bench --target native --release
moon run bench --target js --release

#Known limitations

• The FR-16 event interface is a thin wrapper. feed_with / feed_bytes_with iterate drain_deltas, and the callback runs inside feed, so a callback must not feed the same parser again. It is not a separate push interface (the reasoning is in docs/ONEPAGE.md).
• Only ... in a value position is dropped. R020 handles [1,...,3] and {"a":1,...}; an ellipsis anywhere else is not treated as a placeholder.
• A truncated literal prefix is completed regardless of the following character. For example, a bare tru followed by an unexpected character is completed to true and the unexpected character is handled separately.
• incomplete_number and pending_key only affect snapshots. They do not change finish: the final result trims the number (R011) and fills a dangling key with null (R008) under their own rules.
• Lone surrogates differ by backend. On the wasm backend, \uD800-style lone surrogates survive in the output; on the other backends the InvalidUtf8 (E040) path substitutes U+FFFD.
• The jsonmend/llm and delta sub-packages are experimental. Their APIs may change in a minor release; the core repair / parse / PartialParser surface is the stable one.
• The CLI is native-only. cmd/jsonmend/moon.pkg declares supported_targets = "native".
• Recovery guarantees are heuristic, not a conformance claim. JSONMend is not a JSON standard, and it does not guarantee a particular interpretation for every malformed input; the differential report is the evidence for the behaviour it does document.
• Json numbers follow the core representation. Very large integers may lose precision in the tree, although the repaired text keeps the number's original digits.
• quote_lookahead is off by default. When enabled, a sequence of strings that is missing commas is biased toward being read as string content.
• Repair records are capped by max_repairs (default 10000). Further repairs are applied but not recorded, and the result sets repairs_truncated; the CLI prints a note to standard error.
• Incremental deltas are correct but not cheap. Each change allocates one Delta plus a path array, so a 1 MB document produces ~195 000 records and takes 1.40–1.46 × the baseline. In the agent scenario (4 KB, 3-code-unit chunks) the whole stream is 0.115 ms, so the overhead is not on the critical path and the compact buffer is no longer the next thing to build.
• --skip-prose only skips prose before a container root. A scalar root preceded by prose (Sure!\n42) still raises E002.
• Memory is 13.7 × the input (177.9 MB for a 12.95 MB document). On the same input Node's JSON.parse is 10.6 × and Python's json.loads is 14.3 ×, so a 4 × budget is not realistic for any tree-building parser, Node included. Reaching it requires a "render text without building a tree" mode (docs/MEMORY.md).
• The per-feed p99 is only an upper bound. The runtime clock is millisecond-resolution and exposes no allocation counter, so the scenario benchmark reports an amortised µs/feed plus the count of feeds that reached the clock resolution (docs/BENCHMARK.md).
• The readiness curve covers top-level members only. A nested member ({"arguments":{…}}) is ready only when its container closes; the curve costs ~1.8 ms/KiB, and early stopping saves ~0 on flat records by construction.
• Text mode does not skip prose by default. --skip-prose starts a value only at { or [, so it would skip a scalar root (0, 42, false); prose followed by a scalar root still raises E002.
• The JS backend still allocates one Position per code unit. The n/2n/4n ratio therefore reaches 2.3–2.8 against a 2.3 target. Absolute throughput is well above the 5 MB/s floor, so this is recorded as a decision rather than fixed.
• Parser instances are not thread-safe. Use one instance per input stream.

#Roadmap

• : zhanshishan/jsonmend 0.3.0, 0.4.0 and 0.5.0 are published (moon publish exited 0 for 0.5.0).
• : static/demo/live.html.
• : Schema + semantic_readiness.
• Constraints in the declaration: the skeleton language names types only, not minimum / enum / maxLength.
• Make semantic_readiness incremental: today it replays the document (32.7× repair); the delta-driven FieldTracker path could compute the same numbers as members settle.
• Make the delta stream cheap: encode operations into a reusable buffer and share path prefixes, targeting ≤ 1.11 × the baseline instead of 1.40–1.46 × (lower priority now that the scenario benchmark shows it is off the critical path).
• : done as the feed_with thin wrapper.
• Record real provider streams to replace the documented event shapes (needs credentials; tests/record_stream.py is ready).
• Extend the readiness curve to nested members and give it a zero-sampling, delta-driven implementation.
• Add a text-only mode (want_value = false). The cross-implementation baseline shows the 4 × budget is unrealistic for any tree builder (Node itself is 10.6 ×), so this needs the metric changed first.
• Add a fourth panel to the browser demo that plays a recorded LLM event stream, so the label and the growing arguments are visible without a terminal.
• Split the engine's current position from its recorded positions to remove the per-code-unit allocation on the JS backend.
• Widen the differential corpus further and add memory-budget tests for the streaming path.
• Keep docs/RULES.md, docs/ERRORS.md, docs/DIVERGENCES.md, docs/BENCHMARK.md, and docs/TRACEABILITY.md in sync with the code.

#Contributing

• CONTRIBUTING.md (part of the release checklist) covers the development environment, how to run the tests, commit conventions, and how to add a repair rule — a rule change must update docs/RULES.md, docs/TRACEABILITY.md, and the golden tests together.
• Issue templates cover defects (input, options, expected versus actual output, and backend) and new-rule proposals.
• SECURITY.md (part of the release checklist) describes how to report a problem: this library processes untrusted input.

#License and acknowledgements

• jsonrepair (TypeScript) and json-repair (Python) are used as behavioural oracles in the differential report. They are used only for comparison; no code from either project is vendored.
• The MoonBit core library (moonbitlang/core/json, moonbitlang/core/string) provides the Json type and string handling.