moonbit-target-parity

    Reusable cross-target behavior contract comparison for MoonBit programs.

    moonbit
    cross-target
    testing
    wasm
    conformance
    Download zip
    Version
    0.3.0
    License
    Apache-2.0
    Last updated
    11 hours ago
    Downloads
    5

    #MoonBit Target Parity

    **给 MoonBit 多后端项目使用的行为契约对照库。**同一个测试场景在 wasm、wasm-gc、js、native 等目标运行后,把退出码、标准输出和可选标准错误交给纯 MoonBit API;工具会同时比较后端之间的一致性与提交的预期结果,因此所有后端一起发生相同回归也会被发现。

    差异可以输出为 JSON、Markdown 或自包含 HTML,HTML 可离线打开并适合作为 CI artifact。

    #为什么做这个

    MoonBit 项目可以面向多个编译后端。moon test --target all 能在多个目标上运行项目自己的断言;MoonBit Target Parity 补充的是同一组输入和场景下,对程序可观察结果进行跨目标对照,适合序列化器、协议解析器、数值库、命令行工具和其他需要保持输出一致的库。

    它不推断未运行的场景。某个目标没有结果、目标重复、出现未声明目标或输入不完整时,状态为 inconclusive,不会把缺证据当成通过。

    #真实跨目标示例

    安装 MoonBit 和 Node.js 后,在仓库根目录运行:

    node scripts/check_backends.mjs

    脚本会分别运行同一份 MoonBit 探针:

    moon run --target wasm cmd/probe moon run --target wasm-gc cmd/probe moon run --target js cmd/probe moon run --target native cmd/probe

    然后将这些真实捕获结果交给 MoonBit 比较器,并逐目标检查配置中提交的 expectations。完整四目标矩阵要求本机具备相应 MoonBit 运行时和 native C 编译器。只想在支持的目标上做本地验证时,可显式指定,例如:

    $env:MOON_PARITY_TARGETS = 'wasm,wasm-gc,js' node scripts/check_backends.mjs

    显式省略某个目标只验证所列矩阵,不构成对未列目标的结论。若某目标在运行时无法启动,该目标按缺失证据报告。

    运行器可接受自定义配置文件,声明 MoonBit package 路径、目标、场景名称、参数、stdin、超时及比较策略:

    node scripts/check_backends.mjs examples/portable-json.json

    package_path 必须是仓库内相对 package 路径;参数以数组传入,不经过 shell。适配器用临时文件把捕获结果交给 MoonBit CLI,并在退出时清除该临时目录。

    如需自包含 HTML 报告,可设置格式环境变量并重定向输出:

    $env:MOON_PARITY_FORMAT = 'html' node scripts/check_backends.mjs examples/portable-json.json > _build/target-parity.html

    仓库内的 examples/synthetic-divergence.json 是人工构造的负向报告例子,可运行:

    node scripts/compare_file.mjs examples/synthetic-divergence.json

    它会生成两个字段差异并以退出码 1 结束;该 fixture 不是实际后端输出或真实缺陷案例。

    #MoonBit API

    调用方负责在每个后端运行相同的场景,并把捕获结果传给库:

    let run : @parity.ScenarioRun = {
    schema_version: 1,
    name: "portable-json",
    expected_targets: ["wasm", "js", "native"],
    observations: [
    { target: "wasm", exit_code: 0, stdout: "{\"ok\":true}\n", stderr: "" },
    { target: "js", exit_code: 0, stdout: "{\"ok\":true}\n", stderr: "" },
    { target: "native", exit_code: 0, stdout: "{\"ok\":true}\n", stderr: "" },
    ],
    policy: @parity.ComparePolicy::strict(),
    }
    let report = @parity.compare_scenario(run)
    println(@parity.report_to_markdown(report))

    只有跨后端互相比对,无法发现所有后端一起产生的同一错误结果。需要把已审核结果作为基线时,为每个目标提交 expectation,再调用 contract API:

    let contract : @parity.ContractScenario = {
    scenario: run,
    expectations: [
    { target: "wasm", exit_code: 0, stdout: "{\"ok\":true}\n", stderr: "" },
    { target: "js", exit_code: 0, stdout: "{\"ok\":true}\n", stderr: "" },
    { target: "native", exit_code: 0, stdout: "{\"ok\":true}\n", stderr: "" },
    ],
    }
    let report = @parity.compare_contract_scenario(contract)
    println(@parity.report_to_markdown(report))

    基线非空时必须为每个声明目标提供一条 expectation;少目标返回 inconclusive,结果不符返回 divergent,差异会标记为 expected-result。留空 expectations 才是 parity-only 模式。

    安装已发布版本:

    moon add zhangbowen2006/moonbit-target-parity@0.3.0

    调用方的 moon.pkg 中导入:

    import {
    "zhangbowen2006/moonbit-target-parity" @parity,
    }

    从仓库源码开发和复验:

    moon test --deny-warn node scripts/test_cli.mjs node scripts/check_backends.mjs

    #比较规则

    • 总是比较退出码和 stdout。
    • 默认把 CRLF/CR 统一为 LF;是否删除一个末尾换行由 trim_one_final_newline 明确控制。
    • compare_stderr 默认开启;跨后端运行器自身日志可能混在 stderr 时,可对该场景显式关闭。退出码和 stdout 仍然比较。
    • compare_stdout_as_json 开启后,合法 JSON stdout 按 JSON 结构比较:对象成员顺序不重要,数组顺序仍重要;差异报告包含转义过的 JSON Pointer 路径。JSON 无法解析时回退到文本精确比较;该选项不会替代 JSON 格式有效性测试。
    • strip_ansi_sgr 和 trim_trailing_whitespace_per_line 默认关闭;确认颜色控制码或行尾空格只是噪声时才开启。
    • stdout/stderr 文本差异的 Markdown 报告附带 unified diff;LCS 表超过 1,000,000 行对时输出资源上限摘要。
    • 第一个实际出现的 expected_targets 作为参考目标,目标顺序由调用者声明并写入报告。
    • 已观察到跨目标差异或预期结果回归时返回 divergent。没有差异但缺少、重复或多出目标,或者声明不足两个目标时返回 inconclusive。
    • 一个场景可报告字段 exit_code、stdout、stderr 差异;套件 API 可聚合多个场景。

    CLI/CI 状态码:0 通过、1 已确认差异、2 输入错误或证据不完整。

    #边界

    库只比较调用方提供的文本结果,不启动隔离沙箱、不生成测试输入、不判定程序业务是否正确,也不能证明没列出的输入行为一致。二进制 stdout 暂不支持;调用方可先提供稳定编码或摘要。比较 JSON 时,对象字段顺序可忽略,但浮点容差、字段忽略规则和通用字符串遮罩暂不支持。

    #验证与 CI

    GitHub Actions 执行严格检查、构建、测试、格式和公共接口快照;目标矩阵在 CI 上真实运行 wasm、wasm-gc、js、native。本地完整命令:

    moon check --target all --deny-warn moon build --target all moon test --target all --deny-warn moon fmt --check moon info node scripts/test_cli.mjs node scripts/check_backends.mjs moon package --list

    #项目资料

    ComparePolicy

    pub(all) struct ComparePolicy {
    normalize_line_endings : Bool
    trim_one_final_newline : Bool
    compare_stderr : Bool
    compare_stdout_as_json : Bool
    strip_ansi_sgr : Bool
    trim_trailing_whitespace_per_line : Bool
    } derive(ToJson,
    FromJson
    )

    Controls only explicitly documented normalization. Exit codes are always compared; stdout is always compared; stderr can be opted out of.

    ComparePolicy::strict

    ComparePolicy::to_json

    ContractScenario

    pub(all) struct ContractScenario {
    scenario : ScenarioRun
    expectations : Array[Observation]
    } derive(ToJson,
    FromJson
    )

    An optional accepted result contract for each declared target. When a contract is provided, parity and expected-result checks both have to pass.

    ContractScenario::to_json

    FieldDifference

    pub(all) struct FieldDifference {
    target : String
    kind : String
    field : String
    path : String?
    reference : String
    observed : String
    } derive(ToJson)

    FieldDifference::to_json

    Observation

    pub(all) struct Observation {
    target : String
    exit_code : Int
    stdout : String
    stderr : String
    } derive(ToJson,
    FromJson
    )

    One target's observable result for a named, identical test scenario.

    Observation::to_json

    fn Observation::to_json(Observation) -> Json

    ParityReport

    pub(all) struct ParityReport {
    scenario : String
    status : Status
    diagnostics : Array[String]
    reference_target : String?
    compared_targets : Array[String]
    missing_targets : Array[String]
    unexpected_targets : Array[String]
    duplicate_targets : Array[String]
    differences : Array[FieldDifference]
    }

    ScenarioRun

    pub(all) struct ScenarioRun {
    schema_version : Int
    name : String
    expected_targets : Array[String]
    observations : Array[Observation]
    policy : ComparePolicy
    } derive(ToJson,
    FromJson
    )

    Declares the targets expected for one scenario and the captured outcomes. Target names are labels; callers may use official MoonBit targets or explicitly named host/runtime variants.

    ScenarioRun::to_json

    fn ScenarioRun::to_json(ScenarioRun) -> Json

    Status

    pub enum Status {
    Pass
    Divergent
    Inconclusive
    }

    SuiteReport

    pub(all) struct SuiteReport {
    status : Status
    scenario_count : Int
    passed : Int
    divergent : Int
    inconclusive : Int
    reports : Array[ParityReport]
    }

    compare_contract_scenario

    fn compare_contract_scenario(contract : ContractScenario) -> ParityReport

    Compare target parity and any committed expected-result contract. An empty expectations array means parity-only mode.

    compare_contract_suite

    fn compare_contract_suite(contracts : Array[ContractScenario]) -> SuiteReport

    Aggregate scenarios that include per-target expected results.

    compare_scenario

    fn compare_scenario(run : ScenarioRun) -> ParityReport

    Compare captured outcomes for one scenario. Missing, duplicate, or undeclared target observations make the report inconclusive unless a concrete divergence has already been observed.

    compare_suite

    fn compare_suite(runs : Array[ScenarioRun]) -> SuiteReport

    Compare a batch of independent scenarios and aggregate their status.

    report_exit_code

    fn report_exit_code(report : ParityReport) -> Int

    Process status contract for CI: 0 pass, 1 known divergence, 2 incomplete data.

    report_to_json

    fn report_to_json(report : ParityReport) -> String

    report_to_markdown

    fn report_to_markdown(report : ParityReport) -> String

    suite_report_exit_code

    fn suite_report_exit_code(report : SuiteReport) -> Int

    suite_report_to_html

    fn suite_report_to_html(report : SuiteReport) -> String

    Render a self-contained, dependency-free HTML summary for browser review.

    suite_report_to_json

    fn suite_report_to_json(report : SuiteReport) -> String

    unified_text_diff

    fn unified_text_diff(expected : String, observed : String) -> String

    Render a deterministic full-context unified diff for textual output. Inputs above one million line-pairs use a bounded summary instead of an unbounded quadratic table.