🥮 mooncakes.io
    Sign in

    casekit

    Collect and compare MoonBit behavior across compiler backends

    testing
    differential-testing
    cross-backend
    Download zip
    Version
    0.1.0
    License
    MIT
    Repository
    Last updated
    18 days ago
    Downloads
    4

    Dependencies

    #MoonBit CaseKit

    Record the behavior of a MoonBit program and compare it across native, JavaScript, Wasm and Wasm-GC. CaseKit reports the case ID, nested path, expected value and actual value when runs disagree. Use it when porting a library, changing a compiler version, or checking serialization and numeric behavior across backends.

    The collection, validation and comparison library is written in MoonBit. A native MoonBit CLI compares saved transcripts; a small Python script runs your observation package on multiple backends and connects the results to CI.

    #Try it

    Prerequisites: MoonBit, a C compiler for the native backend, Node.js for JS, and Python 3.10+. MoonBit supplies moonrun for Wasm. The initial release was tested with moonc v0.10.14 and moon 0.1.20260920 on macOS; the workflow also checks Linux with the current stable toolchain.

    git clone https://github.com/kaigeliang/moonbit-casekit.git
cd moonbit-casekit
moon update
python3 tools/check_targets.py

    The command runs the Unicode/JSON example on all four backends. The first target (native by default) is the reference. Each execution gets its own directory under output/casekit/, containing transcripts and comparison reports.

    # Other runnable use cases
python3 tools/check_targets.py --package src/examples/numeric
python3 tools/check_targets.py --package src/examples/collections

# Print an intentional mismatch, including nested array and type differences
moon run --target native src/examples/canary

# Select backends, optimization mode and an explicit floating-point tolerance
python3 tools/check_targets.py --targets native js --release --abs-tol 0.000001

    The canary intentionally constructs different values. It demonstrates diagnostics; it is not evidence of a compiler defect.

    #Record observations in your library

    Version 0.1.0 is available on Mooncakes. Add it to your MoonBit project with moon add kaigeliang/casekit@0.1.0.

    You can also use both modules in a local MoonBit workspace. For sibling directories, create moon.work in their parent:

    members = ["./my-library", "./moonbit-casekit"]

    Add "kaigeliang/casekit@0.1.0" to your module's moon.mod import declaration. In an observation executable's moon.pkg, use:

    import {
      "kaigeliang/casekit" @casekit,
    }
    pkgtype(kind: "executable")

    Then record JSON values with stable case IDs:

    fn main raise {
      let suite = @casekit.Suite::new("my-library")
      suite.observe("sorted", [1, 2, 3].to_json())
      suite.observe("message", "你好🌙".to_json())
      suite.observe_number("decimal-sum", 0.1 + 0.2)
      println(suite.encode())
    }

    Call the runner from this checkout, pointing to your module and executable:

    python3 tools/check_targets.py --project ../my-library --package src/observations

    The executable must print exactly one transcript to stdout. Send diagnostic logs elsewhere. observe takes a deep snapshot, so later mutation of a JSON array or object cannot change an already recorded case. Empty suites and duplicate IDs are errors. See the examples and public API.

    #Compare saved runs

    moon build --target native
_build/native/debug/build/cli/cli.exe expected.json actual.json
_build/native/debug/build/cli/cli.exe --json expected.json actual.json
_build/native/debug/build/cli/cli.exe --abs-tol 0.000001 expected.json actual.json

    Both CLI and runner use these exit codes:

    CodeMeaning
    0All recorded cases match
    1Valid runs differ
    2Invalid input, invalid settings, build/runtime failure or timeout

    The library also compares decoded transcripts directly:

    let report = @casekit.compare(@casekit.decode(expected), @casekit.decode(actual))
    println(report.render())
    println(report.encode()) // JSON report for another tool

    #Comparison rules

    • Cases are matched by ID. Case order and object key order do not matter; array order does. Missing cases, missing fields, null and different JSON types remain distinct.
    • Nested differences use JSON Pointer paths, such as /items/0/name. An empty path means the case root. Human output shows the first 20 differences by default; the JSON report retains all differences.
    • Numbers compare exactly by default. Explicit tolerances allow abs(a-b) <= abs_tol or a relative error bounded by rel_tol. Tolerances apply to every numeric observation in that comparison.
    • Use observe_number for floating-point values: it rejects NaN and infinity before JSON conversion. Calling .to_json() yourself may already convert a non-finite value to null, which CaseKit cannot recover.
    • Encode exact 64-bit integers as strings. The comparator preserves differences between retained high-precision JSON number spellings, but is not an arbitrary precision arithmetic library. With tolerance enabled, numbers use Double precision. Positive and negative zero compare equal; record a string or bit pattern when their distinction matters.
    • Strings compare without Unicode normalization. Observation values have a nesting limit of 64; decoded transcripts have a limit of 2,097,152 UTF-16 code units.

    CaseKit checks the observations you choose. Matching runs do not prove program correctness, and the reference backend is not automatically correct. Keep inputs deterministic, seed random generators and avoid recording timestamps or unordered iteration results unless those are the behavior under test.

    moon test --target ... runs assertions on each backend; CaseKit adds a reusable record-and-compare layer when you want to detect disagreement without writing an expected value for every case. It does not generate test inputs or shrink failures.

    See the v1 protocol for transcript and report details.

    #Run checks

    moon fmt --check
moon test --target native
moon test --target js
moon test --target wasm
moon test --target wasm-gc
moon build --target native
python3 -m unittest discover -s tests -v

    The core imports only MoonBit core packages. The native CLI uses moonbitlang/async; Python orchestration uses only the standard library.

    #License

    MIT.

    CaseError

    source
    pub(all) suberror CaseError {
      Invalid(String)
    }

    Errors in a suite's identity or recorded values.

    Difference

    source
    pub(all) struct Difference {
      case_id : String
      path : String
      kind : DifferenceKind
      expected : Json?
      actual : Json?
    } derive(ToJson)

    A difference at a JSON Pointer path within a case. The empty path is its root.

    Difference::to_json

    source
    fn Difference::to_json(Difference) -> Json

    DifferenceKind

    source
    pub(all) enum DifferenceKind {
      MissingCase
      ExtraCase
      ValueMismatch
      TypeMismatch
      MissingField
      ExtraField
    } derive(Eq, ToJson)

    Category of an observable difference.

    DifferenceKind::equal

    source

    DifferenceKind::not_equal

    source
    fn DifferenceKind::not_equal(x : DifferenceKind, y : DifferenceKind) -> Bool

    DifferenceKind::to_json

    source

    Report

    source
    pub(all) struct Report {
      suite : String
      total_cases : Int
      matched_cases : Int
      abs_tol : Double
      rel_tol : Double
      differences : Array[Difference]
    } derive(ToJson)

    Comparison result. An error reading inputs is raised, never represented as a match.

    Report::encode

    source
    fn Report::encode(self : Report) -> String raise CaseError

    Machine-readable report with stable field ordering.

    Report::is_match

    source
    fn Report::is_match(self : Report) -> Bool

    Report::render

    source
    fn Report::render(self : Report, max_differences? : Int) -> String raise CaseError

    Human-readable summary. Values and IDs are JSON escaped, keeping each diagnostic on one line.

    Report::to_json

    source
    fn Report::to_json(Report) -> Json

    Suite

    source
    pub struct Suite {
      name : String
      // private fields
    }

    A named collection of observations, keyed by a stable case identifier.

    Suite::encode

    source
    fn Suite::encode(self : Suite) -> String raise CaseError

    Encode a versioned observation document. No diagnostic text is included.

    Suite::new

    source
    fn Suite::new(name : String) -> Suite raise CaseError

    Create a suite. Use the same name on each backend.

    Suite::observe

    source
    fn Suite::observe(self : Suite, id : String, value : Json) -> Unit raise CaseError

    Record a value. Case IDs must be unique within a suite.

    Suite::observe_number

    source
    fn Suite::observe_number(self : Suite, id : String, value : Double) -> Unit raise CaseError

    Record a floating-point value without silently converting NaN or infinity to null.

    Suite::size

    source
    fn Suite::size(self : Suite) -> Int

    Number of recorded cases.

    canonical

    source
    fn canonical(value : Json) -> String raise CaseError

    Deterministic JSON serialization: object keys are sorted, array order is preserved.

    compare

    source
    fn compare(expected : Suite, actual : Suite, abs_tol? : Double, rel_tol? : Double) -> Report raise CaseError

    Compare case IDs, ignoring observation order. Missing and extra cases are differences.

    decode

    source
    fn decode(input : String) -> Suite raise CaseError

    Read a CaseKit v1 transcript. Empty, truncated, unknown-version and duplicate-case documents are errors.

    Source Files

    Powered by MoonBit

    Site sourceReport issuePackagesBuild queueSkillsStatistics

    © 2026 mooncakes.io