Gradient boosted decision trees in pure MoonBit: predict XGBoost and LightGBM models bit for bit, and train histogram trees, on native, wasm, wasm-gc and js.
| Feature | Status |
|---|---|
| XGBoost JSON models (gbtree) | done |
| Objectives: reg:*, binary:logistic, binary:logitraw, count:poisson, reg:gamma, reg:tweedie, multi:softprob, multi:softmax | done |
| Missing values (NaN, absent features) | done |
| Categorical splits | done |
| Leaf indices (pred_leaf) | done |
| Batch prediction from CSV | done |
| CLI | done |
| dart (tree weights) and gblinear boosters | done |
| Multiclass models with one base score per class (XGBoost 3.x) | done |
| LightGBM text models (numerical and categorical splits, all missing-value modes, every built-in objective) | done |
| Verified tree-walking kernel (moon prove) | done |
| Blocked batch prediction | done |
| ES module for browsers and Node (web/) | done |
| Browser demo (docs/, GitHub Pages) | done |
| XGBoost UBJSON models (.ubj) | done |
| Training: histogram trees for reg:squarederror, binary:logistic, multi:softprob, multi:softmax, written as XGBoost JSON | done |
| Training in browsers: JS, and a WebAssembly GC module (webgc/) | done |
///|
fn train_example(x : DenseMatrix, y : Array[Float]) -> Unit raise {
let params = {
..TrainParams::default(),
objective: "binary:logistic",
num_round: 100,
}
let json = train_xgboost_json(x, y, params~) // save it; XGBoost loads it too
let model = Model::from_xgboost_json(json)
ignore(model.predict_batch(x))
}| Task | moonboost native | XGBoost | Test metric, moonboost / XGBoost | XGBoost on the moonboost model |
|---|---|---|---|---|
| Regression | 251 ms | 315 ms | RMSE 0.4327 / 0.4304 | 5,000 / 5,000 bit-identical |
| Binary | 229 ms | 267 ms | log loss 0.1628 / 0.1632 | 5,000 / 5,000 |
| 3 classes | 705 ms | 817 ms | log loss 0.2740 / 0.2747 | 15,000 / 15,000 |
| Library | Training time | Test AUC |
|---|---|---|
| moonboost native | 3.9 s, 3.9 s | 0.7537 |
| moonboost wasm-gc (Node 26) | 5.6 s | 0.7537 |
| moonboost JS (Node 26) | 12.6 s | 0.7537 |
| XGBoost 3.4.1 hist | 7.5 s, 7.7 s | 0.7533 |
| LightGBM 4.7.0 (num_leaves=1024, max_depth=10) | 6.0 s, 6.1 s | 0.7545 |
| Model | moonboost native | moonboost JS (Node 22) | moonboost wasm / wasm-gc (moonrun) | XGBoost 3.4.1 C API |
|---|---|---|---|---|
| 100 trees, depth 4 | 0.38 | 0.91 | 0.70 / 0.77 | 1.00 |
| 500 trees, depth 8 | 3.3 | 5.8 | 4.5 / 5.6 | 12.4 |
| 1000 trees, depth 10 | 8.5 | 15.7 | 11.0 / 13.6 | 34.2 |
| Model | moonboost native | moonboost JS (Node 22) | moonboost wasm / wasm-gc (moonrun) | XGBoost 3.4.1 C API |
|---|---|---|---|---|
| 100 trees, depth 4 | 0.7 | 3.1 | 2.2 / 1.7 | 12.2 |
| 500 trees, depth 8 | 5.2 | 12.4 | 12.5 / 9.1 | 91 |
| 1000 trees, depth 10 | 17.5 | 69 | 45 / 27 | 316 |
| Model | moonboost (JS) | onnxruntime-web 1.31.1 (wasm) | |
|---|---|---|---|
| 100 trees, depth 4 | cold start | 52 ms | 312 ms |
| batch, µs/row | 0.92 | 3.4 | |
| one row per call, µs | 3.6 | 39 | |
| probabilities equal to XGBoost's | 10,000 / 10,000 | 5,776 / 10,000 | |
| 500 trees, depth 8 | cold start | 272 ms | 573 ms |
| batch, µs/row | 5.8 | 28.6 | |
| one row per call, µs | 12.6 | 117 | |
| probabilities equal to XGBoost's | 10,000 / 10,000 | 5,037 / 10,000 | |
| 1000 trees, depth 10 | cold start | 431 ms | 1,191 ms |
| batch, µs/row | 15.3 | 65.2 | |
| one row per call, µs | 62 | 276 | |
| probabilities equal to XGBoost's | 10,000 / 10,000 | 5,053 / 10,000 |
| Model | moonboost (JS) | onnxruntime-web 1.31.1 (wasm) | |
|---|---|---|---|
| 100 trees, depth 4 | cold start | 59 ms | 466 ms |
| batch, µs/row | 0.99 | 3.1 | |
| one row per call, µs | 4.7 | 34 | |
| probabilities equal to XGBoost's | 10,000 / 10,000 | 5,776 / 10,000 | |
| 500 trees, depth 8 | cold start | 331 ms | 792 ms |
| batch, µs/row | 6.1 | 27.6 | |
| one row per call, µs | 14.4 | 121 | |
| probabilities equal to XGBoost's | 10,000 / 10,000 | 5,037 / 10,000 | |
| 1000 trees, depth 10 | cold start | 526 ms | 1,536 ms |
| batch, µs/row | 16.0 | 61.6 | |
| one row per call, µs | 75 | 336 | |
| probabilities equal to XGBoost's | 10,000 / 10,000 | 5,053 / 10,000 |
| Model | moonboost native | TL2cgen | XGBoost 3.4.1 C API |
|---|---|---|---|
| 100 trees, depth 4 | 0.60 | 1.68 | 1.03 |
| 500 trees, depth 8 | 3.5 | 23.5 | 12.5 |
| 1000 trees, depth 10 | 8.2 | 104 | 34.8 |
| equal to XGBoost | 10,000 / 10,000 | 6,355 to 7,121 / 10,000 |
| Model | moonboost native | lleaves |
|---|---|---|
| 100 trees, depth 4 | 0.38 | 0.96 |
| 500 trees, depth 8 | 3.7 | 12.9 |
| 1000 trees, depth 10 | 13.5 | not measured (compiling took over 20 minutes) |
| equal to LightGBM | 10,000 / 10,000 | 5,262 and 5,284 / 10,000 |
| XGBoost model | moonboost native | oneDAL | XGBoost 3.4.1 C API | TL2cgen |
|---|---|---|---|---|
| 100 trees, depth 4 | 0.30 | 0.52 | 0.86 | 1.60 |
| 500 trees, depth 8 | 3.3 | 5.1 | 8.8 | 28.7 |
| 1000 trees, depth 10 | 7.6 | 13.2 | 25.7 | not measured |
| equal to XGBoost | 10,000 / 10,000 | 5,156 to 5,852 / 10,000 | 6,355 to 7,121 / 10,000 |
| LightGBM model | moonboost native | oneDAL | LightGBM 4.7.0 | lleaves |
|---|---|---|---|---|
| 100 trees, depth 4 | 0.34 | 0.53 | 3.1 | 0.72 |
| 500 trees, depth 8 | 3.5 | 5.1 | 39.3 | 14.0 |
| 1000 trees, depth 10 | 10.6 | 14.5 | 127.5 | not measured |
| equal to LightGBM | 10,000 / 10,000 | 3,911 to 4,972 / 10,000 | 5,262 and 5,284 / 10,000 |
| Model file | moonboost native | XGBoost / LightGBM | oneDAL conversion |
|---|---|---|---|
| XGBoost, 500 trees (9.1 MB JSON) | 127 | 159 | 604 |
| XGBoost, 1000 trees (19 MB JSON) | 269 | 370 | 1,927 |
| LightGBM, 500 trees (text) | 38 | 44 | 2,048 |
| LightGBM, 1000 trees (38 MB text) | 121 | 149 | 7,865 |
moon add Juwan-Hwang/moonboost///|
fn predict_iris(json_text : String) -> Unit raise {
let model = @moonboost.Model::from_xgboost_json(json_text)
let row : Array[Float] = [5.1, 3.5, 1.4, 0.2]
let probs = model.predict(row) // class probabilities
let margin = model.predict_margin(row) // raw scores
let leaves = model.predict_leaf(row) // leaf index per tree
println("\{probs} \{margin} \{leaves}")
}///|
fn predict_csv(
model : @moonboost.Model,
csv_text : String,
) -> Array[Float] raise {
let data = @moonboost.DenseMatrix::from_csv(csv_text, header=true)
model.predict_batch(data) // row-major, model.output_width() values per row
}///|
fn predict_lightgbm(text : String) -> Unit raise {
let model = @moonboost.LightGBMModel::from_text(text)
let row : Array[Double] = [5.1, 3.5, 1.4, 0.2]
let probs = model.predict(row) // what LightGBM's predict() returns
let raw = model.predict_raw(row) // predict(raw_score=True)
let leaves = model.predict_leaf(row) // predict(pred_leaf=True)
println("\{probs} \{raw} \{leaves}")
// many float rows at once, row-major, num_tree_per_iteration values per row
let data = @moonboost.DenseMatrix::new([5.1, 3.5, 1.4, 0.2], cols=4)
println(model.predict_batch(data))
}moon build --target js --releaseimport { load_model, predict, predict_batch } from "./_build/js/release/build/web/web.js";
const model = load_model(jsonText);
predict(model, [5.1, 3.5, 1.4, 0.2]);
// rows may be a Float32Array (used as is) or any array of numbers
predict_batch(model, new Float32Array(rowMajorValues), 4); // 4 columnsimport { loadTrainer } from "./train_wasm.mjs";
const train = await loadTrainer("webgc.wasm");
const json = train(rows, cols, labels, "binary:logistic", -1, 100, 6, 0.3);moon run cmd/main --target js -- info examples/iris/model.json
moon run cmd/main --target js -- predict examples/iris/model.json examples/iris/data.csv --header
moon run cmd/main --target native -- train rows.csv labels.csv -o model.json --objective binary:logistic --rounds 1000.99185115,0.005436964,0.0027118726
0.992132,0.0047212564,0.003146739
...moon test --target all # unit + differential tests
python3 tools/gen_fixtures.py && moon fmt # regenerate XGBoost fixtures
python3 tools/gen_lgb_fixtures.py && moon fmt # regenerate LightGBM fixtures
python3 tools/gen_demo.py && sh tools/build_demo.sh # rebuild the browser demoimpl Show for ModelErrortype Compiledpub struct DenseMatrix {
data : FixedArray[Float]
rows : Int
cols : Int
}fn DenseMatrix::from_csv(text : StringView, header? : Bool, sep? : StringView) -> DenseMatrix raise ModelErrorpub struct LightGBMModel {
trees : Array[LightGBMTree]
num_class : Int
num_tree_per_iteration : Int
num_feature : Int
objective : String
feature_names : Array[String]
average_output : Bool
// private fields
}fn Model::train(data : DenseMatrix, label : ArrayView[Float], params? : TrainParams) -> Model raise ModelErrorpub(all) struct TrainParams {
objective : String
num_class : Int
num_round : Int
max_depth : Int
eta : Double
lambda : Double
gamma : Double
min_child_weight : Double
max_bin : Int
}fn category_key(x : Float) -> Intfn float_key(x : Float) -> Intfn train_xgboost_json(data : DenseMatrix, label : ArrayView[Float], params? : TrainParams) -> String raise ModelErrorInstall
Download zipGradient boosted decision trees in pure MoonBit: predict XGBoost and LightGBM models bit for bit, and train histogram trees, on native, wasm, wasm-gc and js.