#spell.mbt

For reviewers. This repository is the submission — acceptance reads the git snapshot, so everything needed is in-tree:

Conformance (official Hunspell corpus)	.good 841/848 = 99.2% · .wrong 611/613 = 99.7%
Differential test vs hunspell 1.7.3	235,976 real words: 199 disagreements = 0.084%
Tests	182 (wasm / wasm-gc / js) · 186 (native)
Direct lookup vs hunspell	parity (0.34–0.38 vs 0.36–0.38 µs/word)
wasm artifact	142.2 KiB, with no C++ runtime

ACCEPTANCE.md is the requirement-by-requirement self-check against all five acceptance yardsticks — charter stage three (9 items), charter chapter seven (10 items), the website's 6 standards, the organisers' 4 review dimensions, and this project's own proposal. It also names which proposal commitments were exceeded and which single commitment turned out to be untrue and was fixed. Every figure above comes from a script in this repository: bash check-all.sh re-runs all of them.

This file is README.mbt.md — MoonBit type-checks .mbt.md files, so every block below marked mbt check is compiled by moon check (an unmarked block is not). README.md is a symlink to this file so that GitHub renders it.

Status: 0.9.6. The .aff/.dic parsers, the affix engine, the spell() judgement engine, the suggestion engine, the public API and the check / suggest CLI subcommands are implemented and tested on all four backends. See Not implemented yet for what suggestion generation still leaves out (ngram/MAXNGRAMSUGS candidates in particular).

#Conformance

• Total = every non-empty .sug line (173 across the 37 .sug files).
• Passing = the line's first (best) suggestion occurs somewhere in the suggestion list this library returned for some wrong word, matched to a distinct wrong word in input order (a maximum monotone matching, so each expected line is used at most once and ordering is respected). Ranking within our list is therefore not required to pass this row. 141/173 = 81.5%.
• For reference, the stricter "our first suggestion equals the expected best suggestion" is 124/173 = 71.7%, and reproducing a whole .sug file exactly — Hunspell's own test criterion, in order and with no extra suggestion — holds for 7/37 suites (18.9%).

Measurement note: the harness emits exactly one verdict per non-empty input line and counts those verdicts, so a corpus line that itself contains a space is now scored correctly. Earlier runs paired the two streams positionally with paste -d' ', which silently misaligned every line containing a space — morph.good has 16 of them (drink eat, …) — and scored them as failures whatever the engine said. conformance/run.sh counts verdicts and also prints the historical positional totals (825/848, 611/613) next to the counted ones so the two stay comparable; the 16-verdict difference between 825 and 841 is the size of that measurement artefact, not a behaviour change. Tab-terminated lines such as utf8_bom.good's are unaffected either way, because awk splits on tabs as well as spaces.

#Differential test against the real hunspell

#Performance

bash bench/run.sh

#Loading a real dictionary vs checking words

#Head-to-head with Hunspell (native vs native)

Run-to-run range, so no single ratio is over-read. Repeating bash bench/run.sh on this machine moves the direct-hit row between 0.89× and 1.06× (ours 0.34–0.38 µs/word, hunspell 0.36–0.38) and the all-miss row between 6.30× and 6.44× (ours 9.38–9.60 µs/word, hunspell 1.49). That spread is why this section claims parity on hits and "≈6× slower" on misses instead of quoting one run's ratio as if it were a property of the code. The table above is one such run.

#Scaling on synthetic dictionaries

#Release artifact size

#What is not measured

#Why

#Ecosystem relevance

• Pure MoonBit, no FFI. All six library packages (src/aff, src/dic, src/affix, src/spell, src/suggest, src/api) are MoonBit only. The library is therefore usable from wasm, wasm-gc, js and native alike — the test suite passes on each of the four backends.
• A real dictionary format, not a toy word list. It reads the Hunspell .aff/.dic files that LibreOffice, Firefox and macOS already ship, so a MoonBit program can consume existing language data instead of a bespoke format.
• A wasm-first text component. preferred_target = "wasm"; the CLI is ~103 KiB of wasm with no C++ runtime, which is the shape a browser or edge-worker text feature wants.
• An API a MoonBit developer can actually read. moon doc / the mooncakes.io page carries a /// comment for every public item, and the module root (import { "Careylq/spell" }) is a nine-item facade — load, check, suggest and four metadata accessors.
• Dogfooding. examples/doccheck uses this library, through its own public API, to spell-check the prose of a MoonBit repository: it extracts candidate words from Markdown and from /// / // comments and reports what the library rejects, with file and line. It is a working demonstration of the library as a component, not a snippet.

#What the dogfooding run found

bash examples/doccheck/run.sh          # scans this repository

#Install

moon add Careylq/spell

import {
  "Careylq/spell",
}

import {
  "Careylq/spell",
} for "test"

#Quick start

///|
test {
  let dictionary = @spell.load("SET UTF-8\nSFX S Y 1\nSFX S 0 s .", "1\ncat/S")
  assert_true(@spell.check(dictionary, "cat"))
  assert_true(@spell.check(dictionary, "cats"))
  assert_false(@spell.check(dictionary, "dog"))
  assert_eq(@spell.rule_count(dictionary), 1)
}

///|
test {
  let dictionary = @spell.load("SET UTF-8", "1\ncat") catch {
    error => abort("could not load the dictionary: \{error.to_string()}")
  }
  assert_true(@spell.check(dictionary, "cat"))
}

///|
test {
  let dictionary = @spell.load(
    "SET UTF-8\nTRY abcdefghijklmnopqrstuvwxyz\nREP 1\nREP ph f", "3\nform\nhello\nphantom",
  )
  assert_true(@spell.check(dictionary, "form"))
  assert_eq(@spell.suggest(dictionary, "phorm", 5), ["form"])
  // a correct word needs no correction
  assert_eq(@spell.suggest(dictionary, "hello", 5), [])
}

#Command line

moon run cmd/main -- check --aff en_US.aff --dic en_US.dic --words -

• --words - reads one word per line from stdin; --words <file> reads a file.
• Prints one line per input word — 1 (correct) or 0 (incorrect) — in input order. Empty input lines produce no output at all, so the output lines up one-to-one with the non-empty input lines.
• The dictionary is parsed and indexed once, then every word is checked against it.
• If the .aff or .dic cannot be read or parsed the command prints a message on stderr and exits non-zero — it never silently prints 0s.

moon run cmd/main -- suggest --aff en_US.aff --dic en_US.dic --words -

• One line per non-empty input word, holding its suggestions joined by , (an empty line when there is none), in input order.
• The input word itself is never among its suggestions, and every suggestion is a word check accepts.

#Runnable example

moon run examples/basic

bash examples/doccheck/run.sh

#Scope

#Implemented

• .aff parser — SET, LANG, FLAG, AF, AM, PFX, SFX, REP, TRY, KEY, IGNORE, WORDCHARS, CHECKSHARPS, COMPLEXPREFIXES, BREAK, ICONV/OCONV, every COMPOUND* and CHECKCOMPOUND* directive, and the special-flag directives (NOSUGGEST, KEEPCASE, FORBIDDENWORD, NEEDAFFIX, PSEUDOROOT, CIRCUMFIX, ONLYINCOMPOUND). An affix rule's condition may be omitted (it then matches every stem). Unmodelled directives are collected in an unrecognized list rather than dropped. Errors carry 1-based line numbers.
• .dic parser — entry count (text after the count on the same line is ignored, as Hunspell does), word/FLAGS, optional morphological fields, \/ escaping (a leading /, as in /usr/share/myspell/, belongs to the word), and flag decoding for all four FLAG modes (single-char, long, num, UTF-8).
• Affix rule engine — matches_condition (Hunspell's simplified pattern language: ., [abc], [^abc], literals; anchored to the start for prefixes and the end for suffixes) and apply_rule (condition then strip then add). Unicode-correct: matching iterates code points, so non-BMP characters work.
• spell() judgement — the correctness core:
  • direct dictionary hits, with homonyms treated as alternatives (foo, foo/X and foo/Y make foo correct, while a FORBIDDENWORD homonym rejects it);
  • affix-derived forms found by reverse lookup (reconstruct the candidate stem from the word, check the stem's flag and the rule condition, then re-apply the rule as a guard) rather than by generating every form;
  • one prefix and one suffix on the same stem when both blocks declare cross_product, including the case where the prefix is licensed by the suffix's continuation flags (undrinkable from drink/RQ and SFX R 0 able/PS .);
  • affix continuation flags / twofold suffix stripping — SFX A 0 s/123 . gives the derived form the flags 1, 2 and 3, so a second suffix can be chained onto it and a prefix can sit in front of the pair (unfoosbar);
  • AF alias entries expand to the flag vectors they name, in the .dic and in continuation-flag lists alike;
  • NEEDAFFIX and ONLYINCOMPOUND are honoured on dictionary entries and on affix continuation classes alike (foos is not a word when SFX A 0 s/XB . gives it NEEDAFFIX; foosbar is);
  • capitalisation rules: all-lowercase, Capitalised, ALL CAPS and mixed case (mixed case must match exactly), plus the upper-cased first letter of a mixed-case word (ULinda finds uLinda) and the LANG tr/az/crh dotted/dotless I casing (İZMİR finds İzmir, Işık finds ışık);
  • CHECKSHARPS: an uppercased word may spell a dictionary ß as SS (PROZESSIONSSTRASSE finds Prozessionsstraße), a capital sharp s (ẞ, U+1E9E) folds to ß, and a KEEPCASE word containing ß may be capitalised or uppercased with SS but not with a sharp s (MÜßIG stays wrong);
  • special flags FORBIDDENWORD (a root homonym survives a forbidden one), KEEPCASE (including on compound parts), NEEDAFFIX (with PSEUDOROOT as its deprecated spelling), ONLYINCOMPOUND and CIRCUMFIX (an affix with the flag needs an affix of the opposite kind with it);
  • IGNORE characters are dropped from dictionary entries, from affix strip/add text and from the checked word before any comparison (Arabic harakat, right-to-left marks);
  • a trailing . is accepted for abbreviations when WORDCHARS declares ., and a number (ASCII or Arabic-Indic digits) with one separator (., ,, -) between digits is accepted when WORDCHARS declares that separator (1.12345, 4,2, 42-42);
  • BREAK splitting, recursively: the declared break points, or Hunspell's defaults -, ^- and -$ (foo-bar-foo-bar), with BREAK 0 switching breaking off;
  • a line of whitespace-separated words is correct when every word on it is correct, and a .dic word pair (compound word) blocks the space-free compound;
  • compounds of two or more parts, recursive over the whole word: COMPOUNDFLAG/COMPOUNDBEGIN/COMPOUNDMIDDLE/COMPOUNDEND/COMPOUNDLAST choose the flag each position needs, COMPOUNDMIN (default 3) the minimum part length, and a part may itself be affixed — its compound flag comes from the stem or from the affix's continuation class, an affix "inside" the word needs COMPOUNDPERMITFLAG, COMPOUNDFORBIDFLAG removes the derived form (or the plain first/middle word) from compounding, and a suffix whose continuation class carries ONLYINCOMPOUND is a Fuge-element that must be followed by another part;
  • COMPLEXPREFIXES trades the second suffix level for a second prefix level, so tekmetouro is reachable when the inner prefix hands the outer one its flag;
  • CHECKCOMPOUNDPATTERN boundaries (endchars[/flag] beginchars[/flag], the special 0 unmodified-stem pattern, and the optional replacement that allows the simplified form), CHECKCOMPOUNDDUP, CHECKCOMPOUNDTRIPLE with SIMPLIFIEDTRIPLE, CHECKCOMPOUNDCASE, CHECKCOMPOUNDREP, COMPOUNDWORDMAX and COMPOUNDSYLLABLE (the Hungarian "more words when few syllables" exception), and FORCEUCASE (a compound whose last part carries it must be capitalised);
  • COMPOUNDRULE patterns over the parts' compound flags, with * (zero or more), ? (zero or one) and parenthesized multi-character flags, matched by recursive decomposition of the word (so 1*-style rules such as n*mp accept 100th).
• Encodings at the CLI — the SET directive is sniffed from the .aff bytes and used to decode the .aff, the .dic and the word input: UTF-8, ISO8859-1, ISO8859-15 (the eight code points that differ from Latin-1) and other single-byte encodings as Latin-1. A UTF-8 input file next to a legacy-encoded dictionary is detected and decoded as UTF-8, and a leading UTF-8 BOM is stripped.
• suggest() suggestion engine (src/suggest):
  • a correct word (check accepts it) needs no correction, so the result is empty;
  • REP replacements from the .aff, applied at every occurrence, on the word and its lower-case form, with ^/$ anchors and _ as a space (phorm → form, alot → a lot);
  • the ph: "inner REP table" — every .dic field ph:value becomes a replacement rule value → the entry's word, in dictionary order, in the capitalised and ALL-CAPS spellings too. The manual's three forms are handled: plain (which ph:wich), the trailing * that strips the last character of both sides (pretty ph:prity* behaves as prit → prett) and the explicit -> form (happy ph:hepi->happi). A word pair is a real word with a space (a lot ph:alot suggests a lot, and its flags come from its last token);
  • MAP related-character groups — each group (MAP uü, MAP ß(ss)) lets any member be replaced by any other member, at several positions of the same word, so Fruhstuck → Frühstück (two u → ü) and gross → groß (ss → ß) are reachable. A parenthesized run is one multi-character alternative, and the recursion is budgeted so a pathological table cannot stall the engine;
  • edit distance 1 — deletions, adjacent transpositions, replacements and insertions, with replacements/insertions restricted to the TRY characters (an ASCII alphabet when TRY is absent);
  • case variations — lower-case, Capitalised, ALL CAPS, the possessive stem (Unicef's → UNICEF's) and the CHECKSHARPS SS spelling (MÜßIG → MÜSSIG);
  • splitting into two words, and with a hyphen when TRY/WORDCHARS declares -;
  • a bounded edit distance 2: double deletion, long swap, single-character move and two independent adjacent transpositions — the kinds the manual names, O(n²) rather than O(n²·|TRY|²), so it stays cheap;
  • the PHONE phonetic pass — the Aspell-derived table-driven transcription is implemented (character classes, - retention, < re-scan, digit priorities, ^/$ anchors, _ as the empty output) and every dictionary word is indexed by its key once at load time; a .dic ph: field is that entry's key (xxxxxxxxxx ph:Brasilia is pronounced like Brasilia). Candidates are dressed in the input's capitalisation (kt → cat, Kt → Cat, KT → CAT). A dictionary with no PHONE line skips the pass and pays nothing;
  • ranking and filtering — every candidate must be accepted by check and must not be a NOSUGGEST entry; results are deduplicated, the input word is never returned, REP/ph: hits rank first, then keyboard-adjacent (via KEY) typos, then early TRY characters, then shorter edit distance, with the phonetic candidates last, and an alphabetical tiebreak.
  • no n-gram tier — MAXNGRAMSUGS, MAXDIFF and ONLYMAXDIFF are parsed into the AST, but no n-gram candidate is generated, because the hunspell(5) manual does not define the similarity score (see Not implemented yet). A future tier would be a fallback below the passes above, not a replacement for them.
• Public façade — src/api plus a re-export from the module root: load, check, suggest, encoding, flag_type_name, rule_count, entry_count.
• Runnable examples — examples/basic (the API end to end) and examples/doccheck (the library spell-checking a MoonBit repository's own prose).

#Not implemented yet

• ALL-CAPS input against a mixed-case dictionary form — Hunspell matches OPENOFFICE.ORG to OpenOffice.org, UNICEF'S to UNICEF's and IPOD to iPod. The library only folds an ALL-CAPS word to its lowercase and capitalised spellings, so those (allcaps, allcaps_utf, allcaps2 — 6 verdicts over 4 distinct spellings) are still lost, and one allcaps2 forbidden word (iPodos) is consequently accepted. The hunspell(5) manual documents the flags (KEEPCASE, CHECKSHARPS) but not the casing algorithm itself.
• The Hungarian "moving rule" — LANG hu activates a hard-wired rule that lets a COMPOUNDFORBIDFLAG stem rebuild itself inside a hyphenated compound (forróvíz-tartály). COMPOUNDWORDMAX/COMPOUNDSYLLABLE are implemented, so this one word is all that hu still misses; the manual only names the rule, it does not define the rebuild.
• limit-multiple-compounding's three-part typo check — that corpus expects a three-or-more-part compound to be rejected when it is one edit away from a dictionary word (foobarbaz vs goobarbaz). No directive requests it and the manual does not describe it, so it is not guessed at.
• COMPOUNDROOT / SYLLABLENUM — parsed into the AST but not applied; no corpus suite exercises them.
• The n-gram candidate generator — Hunspell's last suggestion channel is absent. The .aff directives that configure it are parsed into the AST (AffFile::max_ngram_sugs, AffFile::max_diff, AffFile::only_max_diff), but no candidate is generated from them. The reason is that the hunspell(5) manual — checked in every published form available: the English text in 1.7.0, 1.7.2 and 1.7.3, the corpus's own man/hunspell.5, and its Hungarian translation (man/hu/hunspell.5, which alone names a default of 5 for MAXNGRAMSUGS; the English text names none) — describes the pass only as a "similarity search through the dictionary words based on common 1-, 2-, 3-, and 4-character sequences", and gives MAXDIFF's default (5) and range (0-10) plus ONLYMAXDIFF's "remove all bad n-gram suggestions". It never defines the score itself: the per-order n-gram weights, the normalisation, the cutoff MAXDIFF selects, the default MAXNGRAMSUGS, or what counts as a "bad" suggestion. That algorithm exists only in Hunspell's LGPL-2.1 suggestmgr.cxx, which this Apache-2.0 project must not copy, so the scorer is not guessed at. The gap is also not uniform across the corpus: MAXNGRAMSUGS 0 switches the pass off in 12 of the 37 .sug suites, 1463589, 1463589_utf and base_utf set it to 1, and the rest leave it at the default — so a guessed scorer risks regressing at least the twelve that switch it off, rather than only adding suggestions.
• Other suggestion refinements still out of scope — FORCEUCASE is parsed and used for judgement but does not drive a suggestion. Edit distance 2 is only the bounded O(n²) subset listed above (no two arbitrary replacements, no replacement-plus-insertion). The PHONE pass treats a ^^ pattern as a plain start anchor followed by re-scanning from the end of the match instead of a fully separate sub-word (no corpus table uses ^^). One COMPOUNDRULE-style case also remains: a CHECKCOMPOUNDPATTERN replacement is not tried when the endchars leave a doubled letter across the join (a contrived kroom+om b case), matching Hunspell only for the corpus shapes. This is why .sug is at 81.5% rather than higher.
• Unicode upper-casing beyond the special cases — upper-casing is ASCII-only plus the explicit İ/ı and ß rules; a fully general Unicode case table is not shipped. FULLSTRIP needs no special code (a rule may already strip the whole stem) and PSEUDOROOT is accepted as the deprecated spelling of NEEDAFFIX.
• FFI bindings to the Hunspell C++ library. The parsing, affix, judgement and suggestion logic is written from scratch in MoonBit and links against no speller — see Native code for the one exception, a small stdin shim in the CLI.
• Any UI / editor plugin.

#Native code

#Design

src/aff/      .aff lexer + parser -> typed AST
src/dic/      .dic parser + dictionary storage
src/affix/    affix rule engine (condition matching, strip/add)
src/spell/    spell() judgement: indexed dictionary, reverse lookup, case rules
src/suggest/  suggest(): REP, edit distance 1/2, case, splitting, ranking
src/api/      public facade: load, check, suggest, metadata
cmd/main/     CLI (the `check` and `suggest` subcommands)
examples/     runnable examples (`basic`, `doccheck`)
conformance/  conformance harnesses (`run.sh` for .good/.wrong, `suggest.sh` for .sug)

#Development

moon check      # type / lint check
moon build      # build
moon test       # run tests
moon fmt        # format
moon info       # regenerate .mbti interfaces

bash check-all.sh          # reports land in .final-check/

#Community articles

• 《AI 写了 7300 行 MoonBit：编译器能验的，和验不了的》 — how this library was built with an AI assistant, and the part that matters more: the three verification layers, and why a 2.31× performance regression and a 1.9-point conformance understatement both survived a green moon check, a green test suite and an unchanged conformance number. The fix in both cases was re-measuring, not adding features.
  • 知乎 — https://zhuanlan.zhihu.com/p/2087606024068976886
  • 掘金 — https://juejin.cn/post/7689644399769796649
  (Chinese. Every measured number in it was taken from this repository, but one has moved since publication: the article and the cover say 178 / 182 tests, and the repository now has 182 / 186. Two of the three increments come from mbt check blocks added to the Quick start above and to README.zh.mbt.md: a block containing a test is executed by moon test, so documenting a behaviour now enforces it. The third is the Chinese README's own examples. Where the article and this repository ever disagree, this file and the scripts that produce it win — which is the article's own point.)

#License