moondriftkit

    Streaming histogram drift detection and stability reports for MoonBit.

    drift-detection
    histogram
    monitoring
    streaming
    data-quality
    Download zip
    Version
    0.2.0
    License
    Apache-2.0
    Last updated
    3 months ago
    Downloads
    29

    No dependencies

    #MoonDriftKit

    MoonDriftKit is a MoonBit foundation library for streaming data drift detection, histogram stability reports, and deterministic monitoring evidence.

    It is designed for model input monitoring, feature distribution checks, telemetry quality gates, experiment guardrails, and data pipeline regression tests.

    #Core Features

    • Equal-width integer bucket specifications.
    • Compact baseline and current histograms.
    • Basis-point distribution shares for deterministic cross-backend results.
    • Drift reports with score, maximum bucket shift, shifted bucket count, and stable/watch/drift levels.
    • Fixed-size streaming windows for recent values.
    • Configurable drift policies for conservative or strict alerting.
    • Stable JSON export for CI logs, dashboards, and reproducible review evidence.
    • CLI demo for quick smoke testing.

    #Quick Example

    ///|
    test {
    let spec = BucketSpec::new(0, 100, 5)
    let reference = Histogram::new(spec).add_many([42, 45, 48, 51, 54])
    let current = Histogram::new(spec).add_many([70, 72, 75, 80, 86])
    let report = compare_histograms(reference, current)

    assert_true(report.is_alert())
    assert_eq(report.level_name(), "drift")
    }

    #Streaming Window

    ///|
    test {
    let spec = BucketSpec::new(0, 100, 5)
    let window = DriftWindow::new(spec, 3).push(10).push(20).push(30).push(90)

    assert_eq(window.length(), 3)
    assert_eq(window.histogram().total, 3)
    }

    #Policy Tuning

    ///|
    test {
    let spec = BucketSpec::new(0, 100, 5)
    let reference = Histogram::new(spec).add_many([10, 20, 30, 40, 50])
    let current = Histogram::new(spec).add_many([10, 20, 30, 60, 90])
    let report = compare_histograms_with_policy(
    reference,
    current,
    DriftPolicy::strict(),
    )

    assert_true(report.level == Drift)
    }

    #Design Notes

    MoonDriftKit intentionally uses integer basis points rather than platform floating point math. This keeps reports stable across JavaScript, WebAssembly, Wasm-GC, and native builds, which is important for CI and contest review.

    The default score is a symmetric population shift score:

    score_bp = sum(abs(current_share_bp - reference_share_bp)) / 2

    It is not a replacement for every statistical test. It is a small, inspectable foundation for practical drift gates and data quality signals.

    #Commands

    moon check --target all moon test --target wasm moon test --target wasm-gc moon run cmd/main

    #Project Boundary

    MoonDriftKit does not provide plotting, dataframe storage, model training, or network ingestion. It focuses on the reusable drift-monitoring core that other MoonBit tools can embed.

    BucketDelta

    pub(all) struct BucketDelta {
    index : Int
    lower : Int
    upper : Int
    reference_bp : Int
    current_bp : Int
    delta_bp : Int
    } derive(Eq,
    Debug
    )

    Per-bucket comparison in basis points.

    BucketSpec

    pub(all) struct BucketSpec {
    min : Int
    max : Int
    buckets : Int
    } derive(Eq,
    Debug
    )

    Equal-width integer bucket specification.

    BucketSpec::bucket_count

    fn BucketSpec::bucket_count(self : BucketSpec) -> Int

    BucketSpec::index_of

    fn BucketSpec::index_of(self : BucketSpec, value : Int) -> Int

    BucketSpec::lower_bound

    fn BucketSpec::lower_bound(self : BucketSpec, index : Int) -> Int

    BucketSpec::new

    fn BucketSpec::new(min : Int, max : Int, buckets : Int) -> BucketSpec

    BucketSpec::upper_bound

    fn BucketSpec::upper_bound(self : BucketSpec, index : Int) -> Int

    DriftLevel

    pub(all) enum DriftLevel {
    Stable
    Watch
    Drift
    } derive(Eq,
    Debug
    )

    DriftPolicy

    pub(all) struct DriftPolicy {
    watch_score_bp : Int
    drift_score_bp : Int
    watch_max_delta_bp : Int
    drift_max_delta_bp : Int
    watch_shifted_buckets : Int
    drift_shifted_buckets : Int
    } derive(Eq,
    Debug
    )

    Tunable thresholds for classifying drift reports.

    DriftPolicy::classify

    fn DriftPolicy::classify(self : DriftPolicy, score_bp : Int, max_delta_bp : Int, shifted : Int) -> DriftLevel

    DriftPolicy::default

    fn DriftPolicy::default() -> DriftPolicy

    DriftPolicy::strict

    fn DriftPolicy::strict() -> DriftPolicy

    DriftReport

    pub(all) struct DriftReport {
    level : DriftLevel
    score_bp : Int
    max_delta_bp : Int
    shifted_buckets : Int
    reference_total : Int
    current_total : Int
    buckets : Array[BucketDelta]
    } derive(Eq,
    Debug
    )

    Drift report comparing a baseline histogram with a current histogram.

    DriftReport::is_alert

    fn DriftReport::is_alert(self : DriftReport) -> Bool

    DriftReport::level_name

    fn DriftReport::level_name(self : DriftReport) -> String

    DriftReport::to_json

    fn DriftReport::to_json(self : DriftReport) -> String

    DriftWindow

    pub(all) struct DriftWindow {
    spec : BucketSpec
    capacity : Int
    values : Array[Int]
    } derive(Eq,
    Debug
    )

    Fixed-size integer window for streaming monitoring examples.

    DriftWindow::histogram

    fn DriftWindow::histogram(self : DriftWindow) -> Histogram

    DriftWindow::length

    fn DriftWindow::length(self : DriftWindow) -> Int

    DriftWindow::new

    fn DriftWindow::new(spec : BucketSpec, capacity : Int) -> DriftWindow

    DriftWindow::push

    fn DriftWindow::push(self : DriftWindow, value : Int) -> DriftWindow

    Histogram

    pub(all) struct Histogram {
    spec : BucketSpec
    counts : Array[Int]
    total : Int
    } derive(Eq,
    Debug
    )

    Compact integer histogram used for baseline and current windows.

    Histogram::add

    fn Histogram::add(self : Histogram, value : Int) -> Histogram

    Histogram::add_many

    fn Histogram::add_many(self : Histogram, values : Array[Int]) -> Histogram

    Histogram::count_at

    fn Histogram::count_at(self : Histogram, index : Int) -> Int

    Histogram::new

    fn Histogram::new(spec : BucketSpec) -> Histogram

    Histogram::share_bp_at

    fn Histogram::share_bp_at(self : Histogram, index : Int) -> Int

    Histogram::to_json

    fn Histogram::to_json(self : Histogram) -> String

    compare_histograms

    fn compare_histograms(reference : Histogram, current : Histogram) -> DriftReport

    compare_histograms_with_policy

    fn compare_histograms_with_policy(reference : Histogram, current : Histogram, policy : DriftPolicy) -> DriftReport