🥮 mooncakes.io

    moon-missingness

    MoonBit library for missing data mechanism diagnosis and imputation

    missing data
    missingness
    imputation
    knn
    mice
    csv
    mcar
    mar
    Download zip
    Version
    0.3.1
    License
    Apache-2.0
    Repository
    Last updated
    2 months ago
    Downloads
    48

    No dependencies

    #MoonMissingness

    #Positioning

    MoonMissingness is a MoonBit library for diagnosing and repairing missing data in small tabular workflows. It provides reusable operations for data-quality analysis, deterministic imputation, CSV preparation, and holdout validation across MoonBit's WebAssembly and native targets.

    The library deliberately stays below the DataFrame and full machine-learning framework layers. Its input model is Array[Array[Double?]], where None represents a missing cell.

    #Installation

    Add the published module to a MoonBit project:

    moon add liuminyuan/moon-missingness@0.3.1

    Then import the packages that your program uses in moon.pkg:

    ///|
    import {
      "liuminyuan/moon-missingness/diagnose",
      "liuminyuan/moon-missingness/impute",
    }

    #Core capabilities

    • Diagnose rectangular shape, missing rates, observed counts, row/column summaries, joint patterns, pairwise relationships, and conditional signals.
    • Screen missingness mechanisms with documented MCAR/MAR engineering heuristics, and generate deterministic synthetic missingness for tests.
    • Impute with constant, mean, median, nearest-neighbour, regression, forward and backward sequence filling, rolling, hot-deck, iterative MICE-style, ensemble, and uncertainty-aware strategies.
    • Parse and normalize CSV data, including quoted fields, escaped quotes, missing-value tokens, numeric conversion, schema profiling, and row filters.
    • Evaluate repairs with deterministic holdouts, MAE/RMSE, completeness reports, strategy selection, calibration, and reproducible benchmark checksums.

    #Quick start

    ///|
    let dataset : Array[Array[Double?]] = [
      [Some(1.0), None, Some(3.0)],
      [Some(2.0), Some(4.0), None],
      [None, None, Some(9.0)],
    ]

    ///|
    let rates = @diagnose.missing_rate(dataset)

    ///|
    let report = @diagnose.missing_rate_report(dataset)

    ///|
    let filled = @impute.impute_median(dataset)

    ///|
    let with_indicators = @impute.append_missing_indicators(dataset)

    #CLI

    The repository includes two runnable programs:

    moon run cmd/main
moon run --target native cmd/bench

    cmd/main demonstrates the basic diagnosis and median-imputation flow. cmd/bench runs a deterministic native workload and prints machine-readable summary fields, checksums, and holdout errors.

    #Architecture

    • diagnose: validation, summaries, pattern analysis, matrices, simulation, sampling, and missingness screening.
    • impute: simple, KNN, sequence, regression, iterative, robust, ensemble, uncertainty, calibration, pipeline, and evaluation helpers.
    • csv: dependency-free CSV parsing, profiling, normalization, and transforms.
    • frame: nullable numeric-table operations, joins, windows, statistics, reshaping, feature engineering, matrix utilities, and neighbor processing.
    • quality: contracts, profiling, drift, repair, monitoring, policy selection, threshold scoring, synthetic fixtures, and deterministic evaluation.
    • cmd/main: minimal usage example.
    • cmd/bench: reproducible native benchmark.

    Public interface files are generated with moon info. The implementation has no runtime dependency on Python, C, or a hosted service.

    #Benchmark

    The reference workload is a deterministic synthetic 200 x 12 table with seed 17, requested missing rate 0.2, 20 checksum rounds, KNN k=5, and three MICE-style iterations. The measured run generated 477 missing cells, 24 patterns, and a quality score of 0.80125.

    The same run produced these checksums:

    mean_impute_checksum=64851810.91227517
knn_impute_checksum=64779603.6
mice_impute_checksum=64893757.64677953
holdout_cells=1061
holdout_mae=Some(649.1812912346851)
holdout_rmse=Some(750.081167953114)

    The full reproduction command and timing record are in benchmarks/README.md. MCAR/MAR values are screening heuristics, not claims to implement a formal Little chi-square test.

    #Tests

    Run the local test suite with:

    moon check --deny-warn
moon build --target all
moon test --deny-warn
moon check --target all
moon test --target all

    The tests cover empty and ragged inputs, all-missing columns, invalid parameters, quoted CSV fields, deterministic masks, imputation strategies, holdout metrics, and boundary conditions across supported targets.

    #Continuous integration

    GitHub Actions installs the current stable MoonBit toolchain and checks Ubuntu, macOS, and Windows. The workflow runs warning-free checks, all-target builds, all-target tests, formatting, generated interface verification, and a native benchmark smoke test. See .github/workflows/test.yml.

    #License

    MoonMissingness is available under the Apache License 2.0. See LICENSE for the full text and SOURCES.md for source, generated-file, and fixture notes.

    Powered by MoonBit

    Site sourceReport issueModulesSkillsCLIBuild queueStatistics

    © 2026 mooncakes.io