moon-missingness

MoonBit library for missing data mechanism diagnosis and imputation

missing data
missingness
imputation
knn
mice
csv
mcar
mar
moon add liuminyuan/moon-missingness@0.2.2
Download zip
Version
0.2.2
License
Apache-2.0
Last updated
3 hours ago
Downloads
4
README

#MoonMissingness

#Positioning

MoonMissingness is a MoonBit library for diagnosing and repairing missing data in small tabular workflows. It provides reusable operations for data-quality analysis, deterministic imputation, CSV preparation, and holdout validation across MoonBit's WebAssembly and native targets.

The library deliberately stays below the DataFrame and full machine-learning framework layers. Its input model is Array[Array[Double?]], where None represents a missing cell.

#Core capabilities

  • Diagnose rectangular shape, missing rates, observed counts, row/column summaries, joint patterns, pairwise relationships, and conditional signals.
  • Screen missingness mechanisms with documented MCAR/MAR engineering heuristics, and generate deterministic synthetic missingness for tests.
  • Impute with constant, mean, median, nearest-neighbour, regression, forward and backward sequence filling, rolling, hot-deck, iterative MICE-style, ensemble, and uncertainty-aware strategies.
  • Parse and normalize CSV data, including quoted fields, escaped quotes, missing-value tokens, numeric conversion, schema profiling, and row filters.
  • Evaluate repairs with deterministic holdouts, MAE/RMSE, completeness reports, strategy selection, calibration, and reproducible benchmark checksums.

#Quick start

///|
let dataset : Array[Array[Double?]] = [
[Some(1.0), None, Some(3.0)],
[Some(2.0), Some(4.0), None],
[None, None, Some(9.0)],
]

///|
let rates = @diagnose.missing_rate(dataset)

///|
let report = @diagnose.missing_rate_report(dataset)

///|
let filled = @impute.impute_median(dataset)

///|
let with_indicators = @impute.append_missing_indicators(dataset)

#CLI

The repository includes two runnable programs:

moon run cmd/main moon run --target native cmd/bench

cmd/main demonstrates the basic diagnosis and median-imputation flow. cmd/bench runs a deterministic native workload and prints machine-readable summary fields, checksums, and holdout errors.

#Architecture

  • diagnose: validation, summaries, pattern analysis, matrices, simulation, sampling, and missingness screening.
  • impute: simple, KNN, sequence, regression, iterative, robust, ensemble, uncertainty, calibration, pipeline, and evaluation helpers.
  • csv: dependency-free CSV parsing, profiling, normalization, and transforms.
  • cmd/main: minimal usage example.
  • cmd/bench: reproducible native benchmark.

Public interface files are generated with moon info. The implementation has no runtime dependency on Python, C, or a hosted service.

#Benchmark

The reference workload is a deterministic synthetic 200 x 12 table with seed 17, requested missing rate 0.2, 20 checksum rounds, KNN k=5, and three MICE-style iterations. The measured run generated 477 missing cells, 24 patterns, and a quality score of 0.80125.

The same run produced these checksums:

mean_impute_checksum=64851810.91227517 knn_impute_checksum=64779603.6 mice_impute_checksum=64893757.64677953 holdout_cells=1061 holdout_mae=Some(649.1812912346851) holdout_rmse=Some(750.081167953114)

The full reproduction command and timing record are in benchmarks/README.md. MCAR/MAR values are screening heuristics, not claims to implement a formal Little chi-square test.

#Tests

Run the local test suite with:

moon check --deny-warn moon test --deny-warn moon check --target all moon test --target all

The tests cover empty and ragged inputs, all-missing columns, invalid parameters, quoted CSV fields, deterministic masks, imputation strategies, holdout metrics, and boundary conditions across supported targets.

#Continuous integration

GitHub Actions installs the current stable MoonBit toolchain and checks Ubuntu, macOS, and Windows. The workflow runs warning-free checks, all-target tests, formatting, generated interface verification, and a native benchmark smoke test. See .github/workflows/test.yml.

#License

MoonMissingness is available under the Apache License 2.0. See LICENSE for the full text and SOURCES.md for source, generated-file, and fixture notes.

Powered by MoonBit

Site sourceReport issuePackagesBuild queueSkillsStatistics

© 2026 mooncakes.io