moonbench

    A tiny MoonBit benchmark statistics toolkit for stopwatch-style measurements.

    benchmark
    stopwatch
    statistics
    moonbit
    osc2026
    Download zip
    Author
    Version
    0.3.0
    License
    MIT
    Last updated
    2 months ago
    Downloads
    28

    #moonbench

    moonbench is a small MoonBit benchmark reporting toolkit for stopwatch-style measurements, external timing samples, and CI performance budgets.

    It is designed for the MoonBit OSC2026 ecosystem package track: clear API boundary, runnable tests, a minimal CLI example, CI configuration, and an OSI-approved open-source license. It complements the official moon bench workflow instead of replacing it.

    #Why

    MoonBit projects often need lightweight timing summaries while developing parsers, algorithms, CLI tools, and examples. moonbench does not own the clock source. It accepts elapsed nanoseconds from your runtime, then provides stable aggregate calculations and compact report strings.

    For quick demos and lightweight tooling, benchmark can also run a function repeatedly and measure it with the standard environment clock. For production tools, benchmark_with_clock accepts an injected Clock adapter so callers can use a monotonic platform clock or deterministic test clock. For CI smoke checks, Budget, meets_budget, and budget_report turn timing summaries into pass/fail reports. For release review, compare_to_baseline, within_regression_budget, and regression_report compare current results with a saved baseline. Imported samples can be cleaned with SampleFilter, filter_samples, and trim_samples before reporting.

    See docs/moon-bench-comparison.md for the difference from official moon bench.

    #Install

    Add the package to an existing MoonBit project:

    moon add sa2360/moonbench

    Then import it from a package with an alias:

    { "import": [ { "path": "sa2360/moonbench", "alias": "bench" } ] }

    Use the package from MoonBit code:

    let set = @bench.sample_set_from([1_000_000, 1_200_000, 1_100_000])
    println(@bench.sample_set_json(set))

    #API

    • empty() -> Summary
    • singleton(elapsed_ns : Int) -> Summary
    • sample_set() -> SampleSet
    • sample_set_from(samples : Array[Int]) -> SampleSet
    • SampleFilter
    • filter_samples(set : SampleSet, filter : SampleFilter) -> SampleSet
    • trim_samples(set : SampleSet, drop_each_side : Int) -> SampleSet
    • drop_warmup(set : SampleSet, warmup_count : Int) -> SampleSet
    • clamp_samples(set : SampleSet, min_ns : Int, max_ns : Int) -> SampleSet
    • Budget
    • Comparison
    • Quality
    • Clock
    • system_clock() -> Clock
    • measure(run : () -> Unit) -> Int
    • measure_with_clock(clock : Clock, run : () -> Unit) -> Int
    • benchmark(iterations : Int, run : () -> Unit) -> Summary
    • benchmark_with_clock(iterations : Int, clock : Clock, run : () -> Unit) -> SampleSet
    • has_samples(summary : Summary) -> Bool
    • Summary::add_sample(summary : Summary, elapsed_ns : Int) -> Summary
    • SampleSet::add_sample(set : SampleSet, elapsed_ns : Int) -> SampleSet
    • merge(left : Summary, right : Summary) -> Summary
    • percentile_ns(set : SampleSet, pct : Int) -> Int
    • variance_ns2(set : SampleSet) -> Double
    • std_dev_ns(set : SampleSet) -> Double
    • distribution(set : SampleSet) -> Distribution
    • spread_ns(summary : Summary) -> Int
    • relative_spread_ppm(summary : Summary) -> Int
    • std_dev_ppm(set : SampleSet) -> Int
    • quality(set : SampleSet, tolerance_ppm : Int) -> Quality
    • samples_per_second(summary : Summary) -> Int
    • is_stable(summary : Summary, tolerance_ppm : Int) -> Bool
    • summary_csv_header() -> String
    • summary_csv(summary : Summary) -> String
    • summary_json(summary : Summary) -> String
    • distribution_json(distribution : Distribution) -> String
    • sample_set_json(set : SampleSet) -> String
    • summary_markdown(summary : Summary) -> String
    • sample_set_markdown(set : SampleSet) -> String
    • quality_report(set : SampleSet, tolerance_ppm : Int) -> String
    • format_ns(ns : Int) -> String
    • describe(summary : Summary) -> String
    • meets_budget(summary : Summary, budget : Budget) -> Bool
    • budget_report(summary : Summary, budget : Budget) -> String
    • compare_to_baseline(current : Summary, baseline : Summary) -> Comparison
    • within_regression_budget(current : Summary, baseline : Summary, tolerance_ppm : Int) -> Bool
    • regression_report(current : Summary, baseline : Summary, tolerance_ppm : Int) -> String

    #Example

    let summary = @bench.benchmark(5, () => {
    let mut total = 0
    for i in 0..<20_000_000 {
    total = total + i % 17
    }
    ignore(total)
    })

    println(@bench.describe(summary))
    println("spread=\{@bench.format_ns(@bench.spread_ns(summary))}")
    println("stable(5%)=\{@bench.is_stable(summary, 50_000)}")

    Example output, values depend on the machine:

    count=5, mean=13 ms, min=11 ms, max=23 ms, throughput=72/s spread=12 ms stable(5%)=false

    #Reports and Percentiles

    let set = @bench.sample_set_from([100, 140, 160, 200])
    let dist = @bench.distribution(set)

    println(@bench.summary_csv_header())
    println(@bench.summary_csv(set.summary))
    println(@bench.sample_set_json(set))
    println("p95=\{@bench.format_ns(dist.p95_ns)}")

    This produces machine-readable output for CI systems, release notes, dashboards, or GitHub Actions artifacts. The raw sample set can come from moonbench, from another runtime clock, or from benchmark logs collected by a separate tool.

    #Imported Samples, Cleanup, and Regression Checks

    let imported = @bench.sample_set_from([0, 12_000_000, 12_400_000, 80_000_000])
    let cleaned = @bench.filter_samples(
    imported,
    @bench.SampleFilter::{ min_ns: 1, max_ns: 50_000_000, drop_zero: true },
    )
    let baseline = @bench.sample_set_from([12_200_000, 12_200_000, 12_300_000])

    println(@bench.sample_set_markdown(cleaned))
    println(@bench.regression_report(cleaned.summary, baseline.summary, 80_000))

    This path is useful when another benchmark runner already produced timing data. moonbench can focus on cleanup, summaries, release-note tables, and regression decisions without taking over the benchmark execution step.

    #Data-Backed Example

    The repository includes concrete run data in data/benchmark-runs.csv and notes in docs/benchmark-data.md. These files record local moon run cmd/main output and an imported-sample scenario, including summary CSV fields, p50/p95/p99 percentiles, and budget status.

    #Platform Clock Adapter

    let mut tick = 0
    let fake_clock = @bench.Clock::{
    name: "deterministic",
    now_ns: () => {
    tick = tick + 10
    tick
    },
    }

    let set = @bench.benchmark_with_clock(3, fake_clock, () => ignore(1 + 1))
    println(@bench.sample_set_json(set))

    Applications can replace the fake clock with a monotonic native clock while keeping the same statistics and report layer.

    #Run

    moon check moon build moon fmt moon info moon test moon run cmd/main

    cmd/main uses system_clock(), which is backed by the portable env.now() clock. On platforms where that clock is millisecond-grained, very short samples may round to 0 ns. In that case the demo budget line can occasionally print fail even though the command exits successfully and the CSV/JSON reports are still valid. Use an injected monotonic Clock for stricter production budgets.

    #OSC2026 Checklist

    • Public repository: https://github.com/sa2360/moonbit.
    • MoonBit as main language: source is in .mbt files.
    • README: this file explains motivation, API, example, and commands.
    • Runnable example: cmd/main.
    • Tests: moonbench_test.mbt.
    • CI: .github/workflows/ci.yml runs moon check, moon build, formatting, moon info, moon test, and the example.
    • License: MIT.
    • Mooncakes release: run moon login, then moon publish.

    Budget

    pub(all) struct Budget {
    max_mean_ns : Int
    max_spread_ppm : Int
    } derive(Eq,
    Debug
    )

    Simple performance budget for CI or documentation smoke checks.

    Clock

    pub(all) struct Clock {
    name : String
    now_ns : () -> Int
    }

    Clock adapter used by benchmark helpers.

    Production tools can pass a platform-specific monotonic clock. Tests can pass a deterministic clock so examples do not depend on wall time.

    Comparison

    pub(all) struct Comparison {
    baseline_mean_ns : Int
    current_mean_ns : Int
    delta_ns : Int
    delta_ppm : Int
    faster : Bool
    slower : Bool
    } derive(Eq,
    Debug
    )

    Difference between a current summary and a baseline summary.

    delta_ppm is measured against the baseline mean. Positive values are slower, negative values are faster.

    Distribution

    pub(all) struct Distribution {
    p50_ns : Int
    p90_ns : Int
    p95_ns : Int
    p99_ns : Int
    variance_ns2 : Double
    std_dev_ns : Double
    } derive(
    Debug
    )

    Distribution statistics derived from all recorded samples.

    Quality

    pub(all) struct Quality {
    sample_count : Int
    spread_ppm : Int
    std_dev_ppm : Int
    stable : Bool
    } derive(Eq,
    Debug
    )

    Compact quality signal for a sample set.

    The fields are expressed in parts per million so CI thresholds can stay integer based. 100_000ppm means 10 percent.

    SampleFilter

    pub(all) struct SampleFilter {
    min_ns : Int
    max_ns : Int
    drop_zero : Bool
    } derive(Eq,
    Debug
    )

    Sample cleanup settings for noisy clocks and imported benchmark logs.

    max_ns <= 0 means "no upper bound". Set drop_zero when a coarse clock can occasionally round very short samples to 0 ns.

    SampleSet

    pub(all) struct SampleSet {
    samples : Array[Int]
    summary : Summary
    } derive(Eq,
    Debug
    )

    Raw benchmark samples plus the derived aggregate summary.

    Use SampleSet when a caller needs percentiles, variance, CSV/JSON reports, or later export of the original elapsed samples.

    SampleSet::add_sample

    fn SampleSet::add_sample(set : SampleSet, elapsed_ns : Int) -> SampleSet

    Add one elapsed sample and update aggregate statistics.

    Summary

    pub(all) struct Summary {
    count : Int
    total_ns : Int
    min_ns : Int
    max_ns : Int
    mean_ns : Int
    } derive(Eq,
    Debug
    )

    Aggregated timing statistics for stopwatch-style benchmark samples.

    The package keeps the data model intentionally small: callers can feed elapsed nanoseconds from any clock source and use the derived summary in CLI tools, test reports, CI logs, or documentation examples.

    Summary::add_sample

    fn Summary::add_sample(summary : Summary, elapsed_ns : Int) -> Summary

    Add one elapsed sample to an existing summary.

    benchmark

    fn benchmark(iterations : Int, run : () -> Unit) -> Summary

    Execute run repeatedly and aggregate the measured elapsed times.

    benchmark_with_clock

    fn benchmark_with_clock(iterations : Int, clock : Clock, run : () -> Unit) -> SampleSet

    Execute run repeatedly and keep every measured elapsed sample.

    budget_report

    fn budget_report(summary : Summary, budget : Budget) -> String

    Produce a compact pass/fail budget report for CI logs.

    clamp_samples

    fn clamp_samples(set : SampleSet, min_ns : Int, max_ns : Int) -> SampleSet

    Clamp samples into a min/max range while keeping the sample count.

    This is useful when a dashboard should keep every iteration but cap known sensor, clock, or runner artifacts before reporting percentiles.

    compare_to_baseline

    fn compare_to_baseline(current : Summary, baseline : Summary) -> Comparison

    Compare current benchmark results against a baseline summary.

    describe

    fn describe(summary : Summary) -> String

    Produce a one-line report suitable for CLI output and CI logs.

    distribution

    fn distribution(set : SampleSet) -> Distribution

    Compute common latency percentiles and variance for a sample set.

    distribution_json

    fn distribution_json(distribution : Distribution) -> String

    Machine-readable JSON report for common distribution statistics.

    drop_warmup

    fn drop_warmup(set : SampleSet, warmup_count : Int) -> SampleSet

    Drop the first N samples while preserving the original order of the rest.

    This is intended for benchmark warm-up samples, for example the first run after loading a CLI, parser, or runtime cache.

    empty

    fn empty() -> Summary

    Return an empty summary.

    filter_samples

    fn filter_samples(set : SampleSet, filter : SampleFilter) -> SampleSet

    Keep only samples that match the cleanup filter.

    This is useful when imported data contains warm-up artifacts, known outlier bounds, or 0 ns values caused by a coarse demo clock.

    format_ns

    fn format_ns(ns : Int) -> String

    Format a nanosecond duration into a compact human-readable unit.

    has_samples

    fn has_samples(summary : Summary) -> Bool

    Return true when the summary contains at least one sample.

    is_stable

    fn is_stable(summary : Summary, tolerance_ppm : Int) -> Bool

    Return true when the min/max spread is inside a parts-per-million tolerance.

    A tolerance of 100_000 means 10 percent.

    measure

    fn measure(run : () -> Unit) -> Int

    Measure one execution of run and return elapsed nanoseconds.

    This helper uses the standard environment clock, so it is portable and intended for lightweight tool output rather than laboratory-grade timing.

    measure_with_clock

    fn measure_with_clock(clock : Clock, run : () -> Unit) -> Int

    Measure one execution of run with a caller-provided clock.

    meets_budget

    fn meets_budget(summary : Summary, budget : Budget) -> Bool

    Return true when a summary is non-empty and fits the given performance budget.

    merge

    fn merge(left : Summary, right : Summary) -> Summary

    Merge two summaries. Empty summaries are identity values.

    percentile_ns

    fn percentile_ns(set : SampleSet, pct : Int) -> Int

    Return a percentile in nanoseconds using linear interpolation.

    pct is clamped into [0, 100]. Empty sample sets return 0.

    quality

    fn quality(set : SampleSet, tolerance_ppm : Int) -> Quality

    Summarize whether a sample set is stable enough for a tolerance threshold.

    quality_report

    fn quality_report(set : SampleSet, tolerance_ppm : Int) -> String

    Human-readable quality report for CI logs and benchmark notes.

    regression_report

    fn regression_report(current : Summary, baseline : Summary, tolerance_ppm : Int) -> String

    Produce a compact pass/fail regression report for release checks.

    relative_spread_ppm

    fn relative_spread_ppm(summary : Summary) -> Int

    Min/max spread relative to the mean, expressed in parts per million.

    sample_set

    fn sample_set() -> SampleSet

    Return an empty sample set.

    sample_set_from

    fn sample_set_from(samples : Array[Int]) -> SampleSet

    Build a sample set from elapsed nanosecond values.

    sample_set_json

    fn sample_set_json(set : SampleSet) -> String

    Machine-readable JSON report including summary and distribution fields.

    sample_set_markdown

    fn sample_set_markdown(set : SampleSet) -> String

    Markdown report including summary and percentile fields.

    samples_per_second

    fn samples_per_second(summary : Summary) -> Int

    Integer operations per second, using nanoseconds as the time base.

    singleton

    fn singleton(elapsed_ns : Int) -> Summary

    Start a summary with one elapsed sample in nanoseconds.

    spread_ns

    fn spread_ns(summary : Summary) -> Int

    Difference between the slowest and fastest samples in nanoseconds.

    std_dev_ns

    fn std_dev_ns(set : SampleSet) -> Double

    Sample standard deviation in nanoseconds.

    std_dev_ppm

    fn std_dev_ppm(set : SampleSet) -> Int

    Sample standard deviation relative to the mean, expressed in ppm.

    summary_csv

    fn summary_csv(summary : Summary) -> String

    Machine-readable CSV row for a summary.

    summary_csv_header

    fn summary_csv_header() -> String

    CSV header for summary rows.

    summary_json

    fn summary_json(summary : Summary) -> String

    Machine-readable JSON report for a summary.

    summary_markdown

    fn summary_markdown(summary : Summary) -> String

    Markdown table header and row for a summary.

    system_clock

    fn system_clock() -> Clock

    Return the system clock adapter used by the default benchmark helpers.

    env.now() currently exposes milliseconds, so the adapter converts the value to nanoseconds for a single public time unit.

    trim_samples

    fn trim_samples(set : SampleSet, drop_each_side : Int) -> SampleSet

    Drop the same number of sorted samples from the low and high ends.

    If the requested trim would remove every sample, an empty set is returned.

    variance_ns2

    fn variance_ns2(set : SampleSet) -> Double

    Sample variance in square nanoseconds.

    within_regression_budget

    fn within_regression_budget(current : Summary, baseline : Summary, tolerance_ppm : Int) -> Bool

    Return true when the current mean is not slower than the tolerated baseline.

    Source Files

    Powered by MoonBit

    Site sourceReport issuePackagesBuild queueSkillsStatistics

    © 2026 mooncakes.io