moonboost

    Gradient boosted decision trees in pure MoonBit: predict XGBoost and LightGBM models bit for bit, and train histogram trees, on native, wasm, wasm-gc and js.

    gbdt
    xgboost
    machine-learning
    gradient-boosting
    decision-tree
    lightgbm
    wasm
    Download zip
    Version
    0.1.0
    License
    Apache-2.0
    Last updated
    yesterday
    Downloads
    15

    #moonboost

    CI

    Gradient boosted decision trees (GBDT) in pure MoonBit.

    moonboost loads models trained by XGBoost or LightGBM and reproduces their predictions bit for bit, from one codebase that runs on the native, wasm, wasm-gc and js backends. No Python, no C++ runtime, no ONNX conversion step.

    Try it in your browser: a 200-tree XGBoost model scoring California housing blocks in the page, a bit-for-bit check against XGBoost on 2,000 rows, and a race against onnxruntime-web on the same model (source in docs/).

    #Why

    Most GBDT models are trained in Python but need to run somewhere else: a browser, an edge function, a mobile app, a game server. Today that usually means shipping a multi-megabyte runtime or re-implementing tree walking by hand. moonboost is a small, dependency-free predictor whose output you can trust because it is tested against XGBoost itself.

    #Status

    FeatureStatus
    XGBoost JSON models (gbtree)done
    Objectives: reg:*, binary:logistic, binary:logitraw, count:poisson, reg:gamma, reg:tweedie, multi:softprob, multi:softmaxdone
    Missing values (NaN, absent features)done
    Categorical splitsdone
    Leaf indices (pred_leaf)done
    Batch prediction from CSVdone
    CLIdone
    dart (tree weights) and gblinear boostersdone
    Multiclass models with one base score per class (XGBoost 3.x)done
    LightGBM text models (numerical and categorical splits, all missing-value modes, every built-in objective)done
    Verified tree-walking kernel (moon prove)done
    Blocked batch predictiondone
    ES module for browsers and Node (web/)done
    Browser demo (docs/, GitHub Pages)done
    XGBoost UBJSON models (.ubj)done
    Training: histogram trees for reg:squarederror, binary:logistic, multi:softprob, multi:softmax, written as XGBoost JSONdone
    Training in browsers: JS, and a WebAssembly GC module (webgc/)done

    Later: multithreaded training and batch prediction, once MoonBit ships official support for parallelism. Everything here runs on one thread today.

    #Training

    ///|
    fn train_example(x : DenseMatrix, y : Array[Float]) -> Unit raise {
    let params = {
    ..TrainParams::default(),
    objective: "binary:logistic",
    num_round: 100,
    }
    let json = train_xgboost_json(x, y, params~) // save it; XGBoost loads it too
    let model = Model::from_xgboost_json(json)
    ignore(model.predict_batch(x))
    }

    Features are cut into at most max_bin bins at observed values, trees grow depth-wise on gradient histograms (the smaller child is built, the larger is the parent minus it), and each split learns where missing values go. Gradient pairs are quantised per tree to integers packed into one 64-bit word, so a histogram slot is one integer add and the subtraction is exact (leaf values still use the exact gradient sums). Below depth 2 each subtree copies its rows' bins (one byte each when they fit) and gradients into a contiguous buffer, so deep, scattered nodes stay in cache. The result is an ordinary XGBoost JSON model: XGBoost loads it and predicts the same float32 bits as moonboost. Training is single-threaded.

    Against XGBoost 3.4.1 (tree_method="hist", one thread, same depth, eta and bins), 100 rounds on 15,000 synthetic rows with 8 features and 5% missing values (bench/train_data.py, bench/train_compare.py; ARM sandbox):

    Taskmoonboost nativeXGBoostTest metric, moonboost / XGBoostXGBoost on the moonboost model
    Regression251 ms315 msRMSE 0.4327 / 0.43045,000 / 5,000 bit-identical
    Binary229 ms267 mslog loss 0.1628 / 0.16325,000 / 5,000
    3 classes705 ms817 mslog loss 0.2740 / 0.274715,000 / 15,000

    On the GBM-perf airline data (1M training rows, 100k test rows, 100 rounds, depth 10, eta 0.1, one thread pinned to one core of an otherwise idle 4-vCPU x86 AMD EPYC VM; the 8 categorical columns are label-encoded, which differs from GBM-perf's own setup), two runs each:

    LibraryTraining timeTest AUC
    moonboost native3.9 s, 3.9 s0.7537
    moonboost wasm-gc (Node 26)5.6 s0.7537
    moonboost JS (Node 26)12.6 s0.7537
    XGBoost 3.4.1 hist7.5 s, 7.7 s0.7533
    LightGBM 4.7.0 (num_leaves=1024, max_depth=10)6.0 s, 6.1 s0.7545

    moonboost native takes about half the time of XGBoost and about 35% less than LightGBM here (bench/gbmperf_compare.py; all three in the same session). moonboost's times leave out reading the CSV.

    Training also runs in the browser, and every build makes the same model bit for bit. webgc/ is a 37 KB WebAssembly GC module with a loader, web/train_wasm.mjs (bench/train_wasm_bench.mjs times it); in Node it trains the airline model above faster than native XGBoost and LightGBM. The ES module's train does the same in plain JS, at two to three times the wasm-gc time: there the quantised sums are Doubles (Int64 would be a BigInt) and the buffers typed arrays.

    The only other GBDT trainer for browsers we found is ml-xgboost 1.1.2 (XGBoost compiled to WebAssembly with Emscripten, 2018). On the first 100k rows of the airline data, 30 rounds, depth 10, tree_method=hist, in Node 26: 1.01 s against 0.30 s for moonboost wasm-gc and 0.79 s for moonboost JS (bench/ml_xgboost_bench.cjs). Past about 150k rows ml-xgboost stops: its loader copies the data onto a 5 MB stack.

    #Correctness

    Every XGBoost test model is trained by XGBoost 3.4.1 (tools/gen_fixtures.py), and the expected outputs are stored as raw float32 bit patterns. moon test checks that:

    • raw margins are identical bit for bit to XGBoost's predict(output_margin=True), on every backend;
    • final predictions are within 2 ulp (they pass through exp, whose last bit may differ between libm implementations).

    Covered today: regression, binary classification with 10% missing values, multiclass softprob and softmax (including unbalanced classes, where XGBoost 3.x keeps one base score per class), Poisson, Gamma and Tweedie regression, binary:logitraw, a model with two categorical features, DART models (in the 3.x layout and the older dart booster layout), and gblinear regression, logistic and multiclass models. CI regenerates the fixtures from the pinned XGBoost release and fails if anything drifts.

    On the three benchmark models below, all 30,000 final predictions match XGBoost exactly on both the native and the JS build.

    The trick is to do what XGBoost does: keep every value in 32-bit floats, compare x < threshold in float32, start from the objective's base margin, and accumulate trees in order.

    DART and gblinear add a product (leaf * weight, x * w) to a running sum. XGBoost's x86-64 build rounds the product first, and so does moonboost on every backend; the reference fixtures come from x86-64. Builds of XGBoost for aarch64 fuse that multiply-add into one instruction, so for these two boosters an aarch64 XGBoost can differ from x86-64 XGBoost (and from moonboost) in the last bit.

    LightGBM models are checked the same way against LightGBM 4.7.0 (tools/gen_lgb_fixtures.py, float64 bit patterns): raw scores and leaf indices bit for bit, and final predictions bit for bit too. The cases cover regression (also with reg_sqrt), binary (also with a sigmoid parameter), multiclass, multiclassova, Poisson, Gamma, Tweedie, both cross-entropy objectives, zeros as missing values, use_missing=false, categorical features with missing and negative codes, and random forest mode. LightGBM compares doubles (x <= threshold), turns NaN into 0 unless the split handles NaN, reads values within 1e-35 of zero as zero, and adds trees in double; moonboost does the same. Its sigmoid, softmax and exponential outputs call the C library's exp, which is not the same function on every platform; moonboost calls the C exp on native builds and, on the web targets, a port of glibc's (the x86-64 FMA build), checked bit for bit against glibc on 2,000 inputs including the overflow and subnormal ranges.

    #Verified kernel

    Tree walking lives in its own package, kernel/, written so that moon prove (Why3 + Z3) can check it. CI runs the proofs on every push. What is proved:

    • Forest::new returns a forest only if every split node points strictly forward to in-range children, every root is in range, and every categorical split refers to a valid slice of the category table;
    • for such a forest, Forest::leaf always terminates, never indexes out of bounds (nodes, row, category table) and returns a leaf.

    So a malformed or malicious model file can be rejected, but cannot make prediction loop forever or read out of bounds. To keep floating point out of the proof, the kernel compares integer keys: float_key maps every non-NaN float32 to an Int with the same order (tested over random bit patterns, zeros, subnormals and infinities). Which way a split goes is not part of the proof; that is covered by the differential tests against XGBoost.

    #Performance

    Batch prediction, 10,000 rows of 32 features (5% missing), single thread, on an aarch64 Linux VM (Neoverse-V2). µs per row, best of several runs, lower is better. XGBoost is timed through its C API (XGBoosterPredictFromDense, nthread=1), so no Python is in its loop. All numbers in a table come from one session on the same VM; between sessions they move by up to about 25%.

    Modelmoonboost nativemoonboost JS (Node 22)moonboost wasm / wasm-gc (moonrun)XGBoost 3.4.1 C API
    100 trees, depth 40.380.910.70 / 0.771.00
    500 trees, depth 83.35.84.5 / 5.612.4
    1000 trees, depth 108.515.711.0 / 13.634.2

    One row per call (rows predicted one at a time, µs per row), which is what an online caller pays. For XGBoost this is one XGBoosterPredictFromDense call per row, so its per-call overhead is included.

    Modelmoonboost nativemoonboost JS (Node 22)moonboost wasm / wasm-gc (moonrun)XGBoost 3.4.1 C API
    100 trees, depth 40.73.12.2 / 1.712.2
    500 trees, depth 85.212.412.5 / 9.191
    1000 trees, depth 1017.56945 / 27316

    Loading the 1000-tree model (a 19 MB JSON file) takes 325 ms native and 360 ms in Node, where the JS build parses with the engine's own JSON.parse; XGBoost's Booster.load_model takes about 580 ms on the same file.

    All 30,000 predictions are bit-for-bit identical to XGBoost's. Reproduce with python3 bench/gen_models.py, then moonboost bench, nodebench/node_bench.mjs, and bench/xgb_capi_bench.c (build line at its top).

    #Against onnxruntime-web

    The usual way to run an XGBoost model in a browser is to convert it to ONNX (onnxmltools) and load it with onnxruntime-web. bench/web_bench.mjs compares that with moonboost's JS build in Node 22, same machine, same 10,000 rows, one thread each (ort.env.wasm.numThreads = 1, WebAssembly backend), each engine in a fresh process. Means of two runs:

    Modelmoonboost (JS)onnxruntime-web 1.31.1 (wasm)
    100 trees, depth 4cold start52 ms312 ms
    batch, µs/row0.923.4
    one row per call, µs3.639
    probabilities equal to XGBoost's10,000 / 10,0005,776 / 10,000
    500 trees, depth 8cold start272 ms573 ms
    batch, µs/row5.828.6
    one row per call, µs12.6117
    probabilities equal to XGBoost's10,000 / 10,0005,037 / 10,000
    1000 trees, depth 10cold start431 ms1,191 ms
    batch, µs/row15.365.2
    one row per call, µs62276
    probabilities equal to XGBoost's10,000 / 10,0005,053 / 10,000

    Cold start is import, model load and the first prediction. Where onnxruntime-web differs from XGBoost, it is by at most 1.2e-7, which is enough to reorder near-ties in a ranking. One-row-per-call times move by up to about 40% between runs on this VM; the other columns by a few percent. What ships to the browser: moonboost is one 181 KB module (56 KB gzipped, LightGBM support included); onnxruntime-web's WebAssembly build is ort.wasm.min.mjs (54 KB) plus ort-wasm-simd-threaded.mjs (25 KB) plus a 14.8 MB .wasm file (3.8 MB gzipped). Model files are smaller in ONNX (97 KB against 223 KB of JSON for the small model).

    The same comparison in a browser, bench/browser_bench.mjs: headless Chromium 141 through Playwright, the files served from localhost, a fresh browser context for every run, one thread each. Cold start here also includes fetching the model. Medians of three runs:

    Modelmoonboost (JS)onnxruntime-web 1.31.1 (wasm)
    100 trees, depth 4cold start59 ms466 ms
    batch, µs/row0.993.1
    one row per call, µs4.734
    probabilities equal to XGBoost's10,000 / 10,0005,776 / 10,000
    500 trees, depth 8cold start331 ms792 ms
    batch, µs/row6.127.6
    one row per call, µs14.4121
    probabilities equal to XGBoost's10,000 / 10,0005,037 / 10,000
    1000 trees, depth 10cold start526 ms1,536 ms
    batch, µs/row16.061.6
    one row per call, µs75336
    probabilities equal to XGBoost's10,000 / 10,0005,053 / 10,000

    XGBoost has no browser build; it is compared natively in the tables above.

    How the batch path works (lockstep.mbt):

    • Ranks instead of floats. The distinct split thresholds of each feature are sorted once. A row value is replaced by its rank among them, found by a branch-free search, and a split on the k-th threshold (from 0) goes right when rank > k. Exact, because x < t_k holds exactly when at most k thresholds are <= x.
    • Rank tables (JS and wasm). Each feature's thresholds are bucketed by value, b = floor((x - lo) * scale). That map is monotone in x, so every threshold in an earlier bucket is <= x and every one in a later bucket is > x; a rank is one table lookup plus at most a few compares inside the bucket. Features whose thresholds crowd into one bucket keep the search. Ranks are computed one feature at a time across a block of rows.
    • Missing values cost nothing. Each feature has two rank slots, one holding 0 and one holding a value above every threshold when the input is missing. A split reads whichever slot matches its default direction.
    • Self-looping leaves. Children sit side by side, so the next node is left + go_right, and a leaf is its own left child with a threshold no rank reaches. Every row takes the same number of steps through a tree, so 64 rows walk it in lockstep with no branches. On JS and wasm, four rows walk two trees to their leaves at once in locals: eight independent chains of loads, and no node index written back between levels.
    • One row, many trees. A single-row call walks sixteen trees to their leaves side by side (four on plain wasm, where more spill), so their cache misses overlap, instead of using the block machinery.
    • Pointer-free trees on JS. The JS build pads every tree to the forest's depth and stores it as an implicit heap (children of node h at 2h and 2h + 1; a shallow leaf is extended by nodes that always go left), with split words in an Int32Array and leaves in a Float32Array. A step then reads no child index, which V8 rewards; native and wasm keep the compact layout, which is faster there.
    • Tree chunks, margins in registers (JS). For single-output models the heap forest is cut into chunks of about 32 KB. Four rows walk four trees of a chunk at once and carry on through the whole chunk with their running sums in locals, so the margin never goes back to memory and the chunk stays in L1 while the 256 rows of a block pass through it.
    • Values instead of ranks for short forests. When the forest takes few steps per feature (at most 32 tree levels per feature, like the 100-tree model), ranking every feature costs more than the walk. Such forests compare the float32 row value with the split value directly. A missing value is stored as -inf in the slot read by default-left splits and +inf in the default-right slot, and padded heap nodes split on NaN, so they always go left. The comparison becomes a 0/1 value with +(a >= b), which V8 compiles without a branch. Native and wasm build the same padded heap with each split packed into one 64-bit word, so a step is a single load: native packs the split value's bits above the slot and compares floats; wasm packs the value's float_key and turns the integer difference's sign into the step (((k - t) >> 63) + 1), since a float compare there costs a branch.

    Leaves are still added in tree order in float32, which is what keeps the results identical. Models with categorical splits use the verified kernel. The JS bundle minified with esbuild is 181 KB, 56 KB gzipped, with no other runtime.

    #Against compiled engines

    TL2cgen (Treelite's C code generator) and lleaves (LLVM-compiled LightGBM) turn a model into machine code ahead of time. Same VM, one thread, 10,000 rows, µs per row in batch; each engine's predictions compared bit for bit with the library that trained the model.

    XGBoost models (the ones above). TL2cgen 1.1.0 C code built with gcc -O3-march=native in 64 translation units, best of quantize on and off; bench/tl2cgen_bench.sh:

    Modelmoonboost nativeTL2cgenXGBoost 3.4.1 C API
    100 trees, depth 40.601.681.03
    500 trees, depth 83.523.512.5
    1000 trees, depth 108.210434.8
    equal to XGBoost10,000 / 10,0006,355 to 7,121 / 10,000

    The generated C for the two larger models takes about 1 and 3 to 4 minutes to compile on this 2-core VM.

    LightGBM models of the same shape (binary, 32 features, 5% missing, num_leaves = 2^depth; bench/lgb_models.py). lleaves 1.3.0 is timed through its Python predict on the whole batch (n_jobs=1); bench/lleaves_bench.py:

    Modelmoonboost nativelleaves
    100 trees, depth 40.380.96
    500 trees, depth 83.712.9
    1000 trees, depth 1013.5not measured (compiling took over 20 minutes)
    equal to LightGBM10,000 / 10,0005,262 and 5,284 / 10,000

    moonboost needs no compile step: a LightGBM model loads into the same branch-free walk as XGBoost models, with doubles where LightGBM uses them. Each feature value is turned once per row into the number every split on it compares (a missing value or zero that a split sends one way by rule becomes ±infinity), trees up to 10 levels are laid out as implicit heaps, and float inputs walk a packed copy where each double threshold is replaced by the largest float not above it, which decides x <= t exactly for any float x. Larger forests compare integer ranks of the values instead, each node packed into one 64-bit word (child, slot and threshold rank), eight rows walking each tree side by side over blocks of 256 rows.

    #On x86-64

    The same benchmarks on a 4-vCPU AMD EPYC VM (Debian 13, glibc 2.41, gcc 14.2), every engine pinned to one core in one session, best of three. This machine also runs Intel oneDAL (scikit-learn-intelex 2026.2, daal4py.mb.convert_model), which has no ARM build; it is timed through its Python predict_proba on the whole batch (bench/onedal_bench.py), as LightGBM and lleaves are through their predict. µs per row in batch:

    XGBoost modelmoonboost nativeoneDALXGBoost 3.4.1 C APITL2cgen
    100 trees, depth 40.300.520.861.60
    500 trees, depth 83.35.18.828.7
    1000 trees, depth 107.613.225.7not measured
    equal to XGBoost10,000 / 10,0005,156 to 5,852 / 10,0006,355 to 7,121 / 10,000

    LightGBM modelmoonboost nativeoneDALLightGBM 4.7.0lleaves
    100 trees, depth 40.340.533.10.72
    500 trees, depth 83.55.139.314.0
    1000 trees, depth 1010.614.5127.5not measured
    equal to LightGBM10,000 / 10,0003,911 to 4,972 / 10,0005,262 and 5,284 / 10,000

    oneDAL's differences from the original libraries are at most 2.2e-7. Every moonboost prediction on this machine matched XGBoost and LightGBM bit for bit, as on ARM. lleaves took over nine minutes to compile the 500-tree model.

    Loading, in ms (read the file, parse, build the prediction form, first load in a fresh process; oneDAL's figure is its conversion from an already loaded booster):

    Model filemoonboost nativeXGBoost / LightGBMoneDAL conversion
    XGBoost, 500 trees (9.1 MB JSON)127159604
    XGBoost, 1000 trees (19 MB JSON)2693701,927
    LightGBM, 500 trees (text)38442,048
    LightGBM, 1000 trees (38 MB text)1211497,865

    #Usage

    Add the package:

    moon add Juwan-Hwang/moonboost

    Load a model saved with booster.save_model("model.json") and predict:

    ///|
    fn predict_iris(json_text : String) -> Unit raise {
    let model = @moonboost.Model::from_xgboost_json(json_text)
    let row : Array[Float] = [5.1, 3.5, 1.4, 0.2]
    let probs = model.predict(row) // class probabilities
    let margin = model.predict_margin(row) // raw scores
    let leaves = model.predict_leaf(row) // leaf index per tree
    println("\{probs} \{margin} \{leaves}")
    }

    Missing values are NaN. Categorical features are passed as their integer category codes, as XGBoost sees them.

    Batch prediction:

    ///|
    fn predict_csv(
    model : @moonboost.Model,
    csv_text : String,
    ) -> Array[Float] raise {
    let data = @moonboost.DenseMatrix::from_csv(csv_text, header=true)
    model.predict_batch(data) // row-major, model.output_width() values per row
    }

    LightGBM text models (booster.save_model("model.txt")) load into a LightGBMModel. LightGBM works in doubles throughout, so rows are Doubles and so are the results:

    ///|
    fn predict_lightgbm(text : String) -> Unit raise {
    let model = @moonboost.LightGBMModel::from_text(text)
    let row : Array[Double] = [5.1, 3.5, 1.4, 0.2]
    let probs = model.predict(row) // what LightGBM's predict() returns
    let raw = model.predict_raw(row) // predict(raw_score=True)
    let leaves = model.predict_leaf(row) // predict(pred_leaf=True)
    println("\{probs} \{raw} \{leaves}")
    // many float rows at once, row-major, num_tree_per_iteration values per row
    let data = @moonboost.DenseMatrix::new([5.1, 3.5, 1.4, 0.2], cols=4)
    println(model.predict_batch(data))
    }

    #JavaScript

    moon build --target js --release

    import { load_model, predict, predict_batch } from "./_build/js/release/build/web/web.js"; const model = load_model(jsonText); predict(model, [5.1, 3.5, 1.4, 0.2]); // rows may be a Float32Array (used as is) or any array of numbers predict_batch(model, new Float32Array(rowMajorValues), 4); // 4 columns

    LightGBM models use load_lightgbm(text), predict_lightgbm(model, row) and predict_lightgbm_raw(model, row). An XGBoost .ubj file loads with load_model_ubjson(bytes), where bytes is a Uint8Array. train(rows, cols, labels, objective, num_class, num_round, max_depth, eta) trains in the page and returns XGBoost JSON text (-1 or "" keeps a default). For two to three times the speed, train with the WebAssembly GC module (build it with moon build --target wasm-gc --release; it is _build/wasm-gc/release/build/webgc/webgc.wasm):

    import { loadTrainer } from "./train_wasm.mjs"; const train = await loadTrainer("webgc.wasm"); const json = train(rows, cols, labels, "binary:logistic", -1, 100, 6, 0.3);

    See examples/node/iris.mjs.

    #CLI

    moon run cmd/main --target js -- info examples/iris/model.json moon run cmd/main --target js -- predict examples/iris/model.json examples/iris/data.csv --header moon run cmd/main --target native -- train rows.csv labels.csv -o model.json --objective binary:logistic --rounds 100

    0.99185115,0.005436964,0.0027118726 0.992132,0.0047212564,0.003146739 ...

    --margin prints raw scores and --leaf prints the leaf reached in each tree. LightGBM text models work the same way; the CLI tells the formats apart by content. XGBoost UBJSON models are read when the file name ends in .ubj.

    #Development

    moon test --target all # unit + differential tests python3 tools/gen_fixtures.py && moon fmt # regenerate XGBoost fixtures python3 tools/gen_lgb_fixtures.py && moon fmt # regenerate LightGBM fixtures python3 tools/gen_demo.py && sh tools/build_demo.sh # rebuild the browser demo

    #License and attribution

    Apache-2.0. moonboost reads the XGBoost JSON and LightGBM text model formats and follows their prediction semantics; no XGBoost or LightGBM source code is included. XGBoost is Apache-2.0, © the XGBoost contributors. LightGBM is MIT, © Microsoft Corporation and the LightGBM developers. lgb_exp.mbt ports the double-precision exp of Arm's optimized-routines (as used by glibc), MIT OR Apache-2.0 WITH LLVM-exception, © Arm Limited; its table and coefficients are copied from glibc 2.41's x86-64 libm.

    ModelError

    pub(all) suberror ModelError {
    InvalidModel(String)
    Unsupported(String)
    } derive(Eq,
    Debug
    )

    Errors raised while loading or validating a model.

    ModelError::equal

    fn ModelError::equal(ModelError, ModelError) -> Bool

    ModelError::not_equal

    fn ModelError::not_equal(x : ModelError, y : ModelError) -> Bool

    Compiled

    type Compiled

    Every tree of a model packed into one verified @kernel.Forest, plus the leaf weights indexed by global node id.

    DenseMatrix

    pub struct DenseMatrix {
    data : FixedArray[Float]
    rows : Int
    cols : Int
    }

    Row-major dense feature matrix. Missing values are NaN.

    DenseMatrix::from_csv

    fn DenseMatrix::from_csv(text : StringView, header? : Bool, sep? : StringView) -> DenseMatrix raise ModelError

    Parse numeric CSV. Empty cells, NA, NaN and ? become missing values. When header is true the first non-empty line is skipped.

    DenseMatrix::from_fixed

    fn DenseMatrix::from_fixed(data : FixedArray[Float], cols~ : Int) -> DenseMatrix raise ModelError

    Like DenseMatrix::new, but takes ownership of data without copying it.

    DenseMatrix::new

    fn DenseMatrix::new(data : Array[Float], cols~ : Int) -> DenseMatrix raise ModelError

    DenseMatrix::row

    fn DenseMatrix::row(self : DenseMatrix, i : Int) -> ArrayView[Float]

    View of row i.

    LightGBMModel

    pub struct LightGBMModel {
    trees : Array[LightGBMTree]
    num_class : Int
    num_tree_per_iteration : Int
    num_feature : Int
    objective : String
    feature_names : Array[String]
    average_output : Bool
    // private fields
    }

    A model saved by LightGBM (Booster.save_model("model.txt"), the text format). Predictions follow LightGBM's CPU predictor: every value is a double, trees are added in order, and the result is bit-for-bit what LightGBM's predict returns.

    LightGBMModel::from_text

    fn LightGBMModel::from_text(text : StringView) -> LightGBMModel raise ModelError

    Load a LightGBM text model (Booster.save_model or model_to_string). Numerical and categorical splits, all three missing-value modes, and every built-in objective's output transform are supported; linear trees are not.

    LightGBMModel::num_trees

    fn LightGBMModel::num_trees(self : LightGBMModel) -> Int

    Number of trees in the model.

    LightGBMModel::predict

    fn LightGBMModel::predict(self : LightGBMModel, row : ArrayView[Double]) -> Array[Double]

    Final predictions (predict()): the raw scores, averaged over the rounds for a random forest, then the objective's output transform.

    LightGBMModel::predict_batch

    fn LightGBMModel::predict_batch(self : LightGBMModel, data : DenseMatrix) -> Array[Double]

    Predictions for every row of data, num_tree_per_iteration values per row (row-major). Same results as predict on each row.

    LightGBMModel::predict_leaf

    fn LightGBMModel::predict_leaf(self : LightGBMModel, row : ArrayView[Double]) -> Array[Int]

    Leaf index reached in every tree (predict(pred_leaf=True)).

    LightGBMModel::predict_raw

    fn LightGBMModel::predict_raw(self : LightGBMModel, row : ArrayView[Double]) -> Array[Double]

    Raw scores, one per class (predict(raw_score=True)): the sum of the trees. Like LightGBM, a random forest's raw score is not averaged.

    LightGBMTree

    pub struct LightGBMTree {
    split_feature : Array[Int]
    threshold : Array[Double]
    decision_type : Array[Int]
    left : Array[Int]
    right : Array[Int]
    leaf_value : Array[Double]
    cat_boundaries : Array[Int]
    cat_threshold : Array[UInt]
    }

    One tree of a LightGBM model, kept in LightGBM's own form: double thresholds and leaf values, decision_type flags, and children where a negative index ~i is leaf i.

    Model

    pub struct Model {
    trees : Array[Tree]
    tree_group : Array[Int]
    num_group : Int
    num_feature : Int
    base_margin : Array[Float]
    linear : Array[Float]
    objective : Objective
    feature_names : Array[String]
    // private fields
    }

    A boosted tree ensemble.

    tree_group[i] is the output group (class) that tree i contributes to. Single-output models have one group; multi:* models have one per class. base_margin[g] is the starting score of group g, already in margin space. A linear model (gblinear) has no trees and keeps its weights in linear: the weight of feature f for group g at f * num_group + g, then one bias per group.

    Model::from_xgboost_json

    fn Model::from_xgboost_json(text : StringView) -> Model raise ModelError

    Load a model saved by XGBoost's Booster.save_model("model.json").

    Supports the gbtree booster with numerical and categorical splits, DART models (tree weights, in both the 3.x layout and the older dart booster object) and the gblinear booster. Categorical features are passed as their integer codes.

    Model::from_xgboost_ubjson

    fn Model::from_xgboost_ubjson(data : BytesView) -> Model raise ModelError

    Loads a model saved by XGBoost as UBJSON (booster.save_model("m.ubj")).

    Model::linear

    fn Model::linear(weights~ : Array[Float], num_group~ : Int, num_feature~ : Int, base_score~ : Float, objective~ : Objective, feature_names? : Array[String]) -> Model raise ModelError

    A linear model, like XGBoost's gblinear booster. weights holds the weight of feature f for group g at f * num_group + g, followed by one bias per group. Every group starts from base_score (XGBoost uses the first entry of its base score for all groups of a linear model).

    Model::new

    fn Model::new(trees~ : Array[Tree], tree_group~ : Array[Int], num_group~ : Int, num_feature~ : Int, base_score~ : Float, objective~ : Objective, feature_names? : Array[String], base_scores? : Array[Float]) -> Model raise ModelError

    Model::num_trees

    fn Model::num_trees(self : Model) -> Int

    Number of trees in the ensemble.

    Model::output_width

    fn Model::output_width(self : Model) -> Int

    Number of values predict returns per row.

    Model::predict

    fn Model::predict(self : Model, row : ArrayView[Float]) -> Array[Float]

    Final predictions for one row: probabilities for classifiers, values for regressors, or the class index for multi:softmax.

    Model::predict_batch

    fn Model::predict_batch(self : Model, m : DenseMatrix) -> Array[Float]

    Predict every row of m. The result is row-major with self.output_width() values per row. Gives exactly the same values as calling predict on each row.

    Model::predict_leaf

    fn Model::predict_leaf(self : Model, row : ArrayView[Float]) -> Array[Int]

    Leaf index reached in every tree, like XGBoost's pred_leaf=True.

    Model::predict_margin

    fn Model::predict_margin(self : Model, row : ArrayView[Float]) -> Array[Float]

    Raw scores (one per output group) before the objective's link function. Accumulates in tree order in 32-bit floats, as XGBoost's CPU predictor does.

    Model::predict_one

    fn Model::predict_one(self : Model, row : ArrayView[Float]) -> Float

    Single-output convenience: the first (and only) prediction for row.

    Model::train

    fn Model::train(data : DenseMatrix, label : ArrayView[Float], params? : TrainParams) -> Model raise ModelError

    Trains a model (see train_xgboost_json).

    Objective

    pub(all) enum Objective {
    Identity(String)
    Logistic(String)
    Exp(String)
    Softprob(Int)
    Softmax(Int)
    } derive(Eq,
    Debug
    )

    How raw tree sums (margins) are turned into predictions.

    Objective::base_margin

    fn Objective::base_margin(self : Objective, base_score : Float) -> Float

    Convert a user-facing base_score into margin space, the way XGBoost's ProbToMargin does.

    Objective::equal

    fn Objective::equal(Objective, Objective) -> Bool

    Objective::from_name

    fn Objective::from_name(name : String, num_class? : Int) -> Objective raise ModelError

    Map an XGBoost objective name to its link function.

    Objective::name

    fn Objective::name(self : Objective) -> String

    XGBoost's canonical objective name.

    Objective::not_equal

    fn Objective::not_equal(x : Objective, y : Objective) -> Bool

    Objective::transform

    fn Objective::transform(self : Objective, margins : Array[Float]) -> Unit

    Apply the link function to the margins of one row, in place. For Softmax the result has a single element: the predicted class.

    TrainParams

    pub(all) struct TrainParams {
    objective : String
    num_class : Int
    num_round : Int
    max_depth : Int
    eta : Double
    lambda : Double
    gamma : Double
    min_child_weight : Double
    max_bin : Int
    }

    Training settings, named as in XGBoost.

    TrainParams::default

    fn TrainParams::default() -> TrainParams

    XGBoost's defaults: squared error, 10 rounds, depth 6, eta 0.3, lambda 1, 256 bins.

    Tree

    pub struct Tree {
    left : Array[Int]
    right : Array[Int]
    feature : Array[Int]
    value : Array[Float]
    default_left : Array[Bool]
    categories : Array[Array[Int]?]
    // private fields
    }

    A single regression tree stored in the flat array layout used by XGBoost.

    Node i is a leaf when left[i] == -1. For a leaf, value[i] is the leaf weight. For a split node, value[i] is the split threshold and the sample goes left when x[feature[i]] < value[i]. Missing values (NaN) follow default_left[i].

    A node may instead be a categorical split: categories[i] holds the sorted category codes that go right; every other non-missing code goes left.

    All arithmetic is done in 32-bit floats, matching XGBoost's bst_float, so that comparisons against thresholds behave bit-for-bit the same.

    Tree::leaf_index

    fn Tree::leaf_index(self : Tree, row : ArrayView[Float]) -> Int

    Index of the leaf reached by row. Features beyond the end of row are treated as missing.

    Tree::max_feature

    fn Tree::max_feature(self : Tree) -> Int

    Largest feature index used by any split, or -1 for a single-leaf tree.

    Tree::new

    fn Tree::new(left~ : Array[Int], right~ : Array[Int], feature~ : Array[Int], value~ : Array[Float], default_left~ : Array[Bool], categories? : Array[Array[Int]?]) -> Tree raise ModelError

    Tree::num_nodes

    fn Tree::num_nodes(self : Tree) -> Int

    Number of nodes in the tree.

    Tree::predict

    fn Tree::predict(self : Tree, row : ArrayView[Float]) -> Float

    Leaf weight reached by row.

    category_key

    fn category_key(x : Float) -> Int

    Key of a categorical feature value, following XGBoost: the value is truncated to an integer code, and negative or too-large codes become -1, which never matches a category and so always goes left.

    float_key

    fn float_key(x : Float) -> Int

    Integer key of a non-NaN float32 whose order matches float order: x < y if and only if float_key(x) < float_key(y). Both zeros map to 0. Keys of finite and infinite floats are all greater than @kernel.MISSING.

    train_xgboost_json

    fn train_xgboost_json(data : DenseMatrix, label : ArrayView[Float], params? : TrainParams) -> String raise ModelError

    Trains a model and returns it as an XGBoost JSON document, which both moonboost and XGBoost load. label has one entry per row: the target, a 0/1 label, or a class index.