Sign in

    mooncoda

    Validated compositional geometry, balances and multivariate analysis in pure MoonBit

    compositional-data
    aitchison
    ilr
    pca
    Download zip
    Author
    Version
    0.1.0
    License
    MIT
    Last updated
    8 hours ago
    Downloads
    3

    #MoonCoDA

    纯 MoonBit 的组成数据几何与多变量分析库,模块 usersD609/mooncoda@0.1.0,MIT 许可。 面向每行表示同一整体中各部分的正数权重、计数或比例,不把封闭比例当作相互独立的普通变量。

    #用于什么

    • 材料与混合物分析:对齐组分名称,计算可解释 SBP balances、中心、变化矩阵,并做 ILR PCA 与重建。
    • 表格/计数数据管道:明确处理零值,保留替换质量审计,再生成 CLR/ALR/ILR 特征供其他工具复用。
    • 资源份额监测:对 CPU/IO/idle 等组成拟合基线,做 Aitchison 聚类或对新样本计算正则化 ILR 高斯评分。

    这些是可复用的数学工作流,不是已部署客户案例;仓库不含真实个人数据。三条端到端例子见 examples/workflows/main.mbt。

    #0.1.0 已实现

    部分能力
    数值与几何scale-safe closure、CLR/ALR/ILR 及逆变换、扰动/逆扰动、power、内积、距离、测地插值
    基与数据Helmert 基、完整正交 SBP、自定义对比基验证、单 balance、命名且防御复制的数据表
    预处理乘法零替换及替换量报告、原始单位伪计数、子组成、特征重排、穷尽分组 amalgamation
    统计加权中心、中心化、加权离散度、CLR/ALR/ILR 协方差、变化矩阵、总方差
    模型Jacobi 对称特征分解、ILR PCA/解释率/新样本投影/重建、确定性 k-means/预测、Cholesky 正则化高斯评分
    查询与集成距离矩阵、稳定近邻、平方距离 medoid、有界 JSON 协议、命令行

    非目标: ALDEx2、ANCOM、Dirichlet Monte Carlo 差异丰度推断、显著性检验、p 值、医学建议、 一般 LP/QP、随机 k-means++、模型持久化、稀疏/缺失值插补、分布式或流式大数据。 高斯输出是 ILR 坐标空间的密度,未加入 simplex Lebesgue 测度 Jacobian,不能解释为概率或告警阈值。

    #环境和目标

    验证基线:MoonBit compiler v0.10.14+7d59c7ec9,moon 0.1.20260920。 CI 固定该工具链。库无第三方运行依赖,仅使用 moonbitlang/core。 库支持 wasm-gc、wasm、js、native;JS 命令行需要 Node.js,native 构建需要系统 C 编译器。 本机 Windows 已执行前三个后端的 check/test/build/示例,以及 native check;native 编译/运行由 Ubuntu CI 验证。 不声称任意新工具链均兼容,也不提供浏览器 UI 或网络服务。

    #安装与快速开始

    在已有 MoonBit 模块中执行精确版本安装:

    moon add usersD609/mooncoda@0.1.0

    在消费方包的 moon.pkg 中加入:

    import {
    "usersD609/mooncoda" @coda,
    }
    options("is-main": true)

    以下可作为 main.mbt(moon run .);返回的组成总和为 1,保留原始比例:

    ///|
    fn main {
    try {
    let x = [60.0, 30.0, 10.0]
    let b = @coda.helmert_basis(3)
    let z = @coda.ilr(x, b)
    let restored = @coda.ilr_inverse(z, b)
    assert_true(@coda.distance(x, restored) < 1.0e-9)
    println("ILR roundtrip passed")
    } catch {
    _ => panic()
    }
    }

    #命名数据、降维与模型

    在上述 try 块中可直接使用:

    let data = @coda.dataset(
    ["sand", "silt", "clay"],
    ["lot1", "lot2", "lot3", "lot4"],
    [[60.0, 30.0, 10.0], [55.0, 30.0, 15.0],
    [10.0, 30.0, 60.0], [15.0, 35.0, 50.0]],
    )
    let fit = @coda.pca(data, 2)
    let scores = fit.transform(data)
    let mixtures = fit.reconstruct(scores)
    let groups = @coda.kmeans(data, 2)
    let detector = @coda.gaussian(data, 0.01)
    let distances = detector.squared_mahalanobis(data)
    assert_true(mixtures.length() == 4 && distances.length() == 4)
    assert_true(groups.converged())

    对独立新样本调用 fit.transform(new_data)、groups.predict(new_data) 或 detector.squared_mahalanobis(new_data)。特征必须同名同序;需要重排时显式 new_data.select_features(data.feature_names())。训练和评分必须使用一致的零替换策略。

    #零值是显式策略

    let report = @coda.multiplicative_replace([0.0, 2.0, 4.0], delta=0.001)
    let transformed = @coda.clr(report.composition())
    assert_true(report.zero_count() == 1 && transformed.length() == 3)

    closure 接受零;所有 log-ratio 和 dataset 拒绝零。乘法替换先 closure,每个零填入 delta, 其余部分共同缩放,保留非零比例;replaced_mass() 是在封闭组成中分配给原零的总质量。 伪计数 add_pseudocount 则在原始单位加数,不具备尺度不变性。本库不判断零是结构零还是采样零。

    #JSON 命令行与嵌入协议

    从仓库运行(POSIX shell 示例;Windows 推荐用 Python subprocess 参数数组避免引号被 shell 改写):

    moon run cmd/mooncoda --target js -- '{"op":"pca","features":["a","b","c"],"samples":["s1","s2","s3"],"rows":[[1,2,4],[2,3,1],[4,1,2]],"components":2}'

    传入一个 JSON 参数;输出一行 JSON。也可在库内调用 process_json(text),不需要文件或网络访问。

    共同字段:op 字符串、features 唯一非空列名数组、samples 唯一非空行名数组、rows 正数矩阵。 未知字段会报错,不会悄悄忽略。操作如下:

    op额外字段结果字段(均含 ok/operation/features/samples)
    analyze无;要求至少两行center, clr, ilr, variation, total_variance
    pcacomponents 整数 1..D-1;至少两行scores, reconstructed, eigenvalues, explained_ratio, loadings_ilr, center
    clusterclusters 整数;可选 max_iterations 默认 100assignments(0 起)、centers、counts、inertia、iterations、converged
    scoreridge 正数squared_mahalanobis, negative_log_density_ilr, rank(0 起降序索引), ridge, center

    score 协议对请求内样本拟合并评分;独立训练/预测使用库模型 API。协议没有隐藏缓存或跨请求模型。 失败响应:{"ok":false,"code":"invalid|shape|budget|numerical","message":"..."}; 参数个数错误为 usage。CLI 的失败状态通过 JSON ok=false 表达,不通过非零退出码表达,调用方必须检查 ok。 错误消息不回显输入值。JSON 重复键遵循 core parser 的后值覆盖行为,不保证重复键拒绝。

    #数学约定与限制

    • 组成维数 D=2..64;数据集 N=1..1024。数值必须有限且绝对值不超过 1e300。
    • 正数经过 closure 后若下溢为 0,或逆变换 softmax 下溢,抛出 Numerical,不返回被静默截断的组成。
    • Basis.rows() 为 (D-1)×D,行正交且零和;公差 1e-10。ILR 符号遵循该基,Helmert 第 k 行前 k 项为正。
    • clr_inverse 接受未中心化实向量,执行稳定 softmax;ALR 的 reference 是 0 起的分母位置。
    • 协方差默认 ddof=1(样本),仅支持 0/1;高斯明确使用 ddof=0,再加绝对 ILR ridge。
    • PCA 对 ILR 样本协方差进行分解;特征值降序,轴最大绝对分量定正;重复特征值不保证跨基轴唯一。 解释率以截断前总方差为分母;常量数据返回全零解释率。Jacobi 最多 100 sweeps,并检查残差。
    • k-means 从第一行开始 farthest-first 初始化,平局按输入顺序,空簇保留原中心,重复样本可能产生空簇。 converged 仅表示迭代中标签不再变化;预算耗尽仍返回与最终中心最近的标签,不能视为收敛或全局最优。 k≤min(N,64),迭代 1..500,N×(D-1)×k×iterations≤20,000,000。
    • 完整距离矩阵与 medoid 限 N≤512。medoid 最小化平方 Aitchison 距离和,报告为 sum_squared_distance()。
    • ridge 在 (0,1e12];过小 ridge 可能被舍入吞没,此时正定分解会报错。负对数密度可能为负。
    • JSON 最多 1,000,000 UTF-16 code units、16 层嵌套;命令行还受操作系统 argv 长度限制。 这是有界批处理库,不是对抗性多租户沙箱;在服务端仍应限制 CPU/请求频率与内存。

    #检查、测试与构建

    moon info moon fmt moon fmt --check moon check --deny-warn --target wasm-gc moon test --deny-warn --target wasm-gc moon build --deny-warn --target wasm-gc moon run examples/workflows --target wasm-gc # 对 wasm / js / native 重复上述目标命令 moon build --target js --deny-warn python tools/verify_oracle.py python tools/audit_source.py

    25 个测试声明涵盖合法/非法输入、零策略、尺度不变性、ILR 等距、矩阵残差、PCA 重建、 空簇/迭代上限、模型防御复制和协议预算;每个目标独立执行。 独立 Python 标准库预言机以固定 seed=6092026 生成 48 组矩阵,通过 Node CLI 发出 168 次请求, 核对 CLR、中心、ILR、变化矩阵、总方差、解析二维 PCA 特征值、Gauss-Jordan 高斯二次型及聚类目标; 没有把测试通过率等同为任意输入正确性,也没有声明性能优越性。

    #架构、来源与交付资料

    原创实现,不是 scikit-bio 的逐行移植;参考其 BSD-3-Clause composition API 的数学语义, 未复制源码、数据或测试样本。与 MoonBit BioSeqs 的 CLR/ALDEx2 存在概念邻接,但本库不实现差异丰度推断。 详见 NOTICE。代码及文档为 MIT;无医疗、科学推断或官方比赛验收保证。

    CodaError

    pub(all) suberror CodaError {
    Invalid(String)
    Shape(String)
    Budget(String)
    Numerical(String)
    } derive(
    Debug
    )

    Basis

    pub struct Basis {
    contrasts : Array[Array[Double]]
    parts : Int
    }

    Rows are orthonormal, zero-sum contrasts; columns follow feature order.

    Basis::dimension

    fn Basis::dimension(self : Basis) -> Int

    Basis::rows

    fn Basis::rows(self : Basis) -> Array[Array[Double]]

    Cholesky

    pub struct Cholesky {
    lower : Array[Array[Double]]
    logdet : Double
    }

    Validated positive-definite matrix factorization, with triangular solves.

    Cholesky::log_determinant

    fn Cholesky::log_determinant(self : Cholesky) -> Double

    Cholesky::lower_triangle

    fn Cholesky::lower_triangle(self : Cholesky) -> Array[Array[Double]]

    Cholesky::quadratic

    fn Cholesky::quadratic(self : Cholesky, rhs : Array[Double]) -> Double raise CodaError

    Cholesky::solve

    fn Cholesky::solve(self : Cholesky, rhs : Array[Double]) -> Array[Double] raise CodaError

    Clusters

    pub struct Clusters {
    features : Array[String]
    basis : Basis
    centers : Array[Array[Double]]
    assignments : Array[Int]
    counts : Array[Int]
    inertia : Double
    iterations : Int
    converged : Bool
    }

    Clusters::assignments

    fn Clusters::assignments(self : Clusters) -> Array[Int]

    Clusters::centers

    fn Clusters::centers(self : Clusters) -> Array[Array[Double]] raise CodaError

    Clusters::converged

    fn Clusters::converged(self : Clusters) -> Bool

    Clusters::counts

    fn Clusters::counts(self : Clusters) -> Array[Int]

    Clusters::inertia

    fn Clusters::inertia(self : Clusters) -> Double

    Clusters::iterations

    fn Clusters::iterations(self : Clusters) -> Int

    Clusters::predict

    fn Clusters::predict(self : Clusters, data : Dataset) -> Array[Int] raise CodaError

    Dataset

    pub struct Dataset {
    features : Array[String]
    samples : Array[String]
    cells : Array[Array[Double]]
    }

    Observations are rows, features are columns. Public accessors return copies.

    Dataset::clr_rows

    fn Dataset::clr_rows(self : Dataset) -> Array[Array[Double]] raise CodaError

    Dataset::feature_names

    fn Dataset::feature_names(self : Dataset) -> Array[String]

    Dataset::ilr_rows

    fn Dataset::ilr_rows(self : Dataset, b : Basis) -> Array[Array[Double]] raise CodaError

    Dataset::part_count

    fn Dataset::part_count(self : Dataset) -> Int

    Dataset::row_count

    fn Dataset::row_count(self : Dataset) -> Int

    Dataset::rows

    fn Dataset::rows(self : Dataset) -> Array[Array[Double]]

    Dataset::sample_names

    fn Dataset::sample_names(self : Dataset) -> Array[String]

    Dataset::select_features

    fn Dataset::select_features(self : Dataset, order : Array[String]) -> Dataset raise CodaError

    Explicit name alignment prevents silently comparing different column orders.

    Dataset::select_samples

    fn Dataset::select_samples(self : Dataset, indices : Array[Int]) -> Dataset raise CodaError

    Gaussian

    pub struct Gaussian {
    features : Array[String]
    basis : Basis
    location : Array[Double]
    covariance : Array[Array[Double]]
    factor : Cholesky
    ridge : Double
    }

    Gaussian density is with respect to ILR coordinates, NOT simplex volume.

    Gaussian::center

    fn Gaussian::center(self : Gaussian) -> Array[Double] raise CodaError

    Gaussian::coordinate_covariance

    fn Gaussian::coordinate_covariance(self : Gaussian) -> Array[Array[Double]]

    Gaussian::log_determinant

    fn Gaussian::log_determinant(self : Gaussian) -> Double

    Gaussian::negative_log_density

    fn Gaussian::negative_log_density(self : Gaussian, data : Dataset) -> Array[Double] raise CodaError

    Negative log density can be negative; it is not a calibrated anomaly probability.

    Gaussian::rank

    fn Gaussian::rank(self : Gaussian, data : Dataset) -> Array[Int] raise CodaError

    Rank by descending squared Mahalanobis distance; ties keep input order.

    Gaussian::ridge

    fn Gaussian::ridge(self : Gaussian) -> Double

    Gaussian::squared_mahalanobis

    fn Gaussian::squared_mahalanobis(self : Gaussian, data : Dataset) -> Array[Double] raise CodaError

    Medoid

    pub struct Medoid {
    sample : String
    index : Int
    objective : Double
    }

    Medoid::sample_index

    fn Medoid::sample_index(self : Medoid) -> Int

    Medoid::sample_name

    fn Medoid::sample_name(self : Medoid) -> String

    Medoid::sum_squared_distance

    fn Medoid::sum_squared_distance(self : Medoid) -> Double

    Sum of squared distances, deliberately not called a distance.

    Neighbor

    pub struct Neighbor {
    sample : String
    index : Int
    separation : Double
    }

    Neighbor::distance

    fn Neighbor::distance(self : Neighbor) -> Double

    Neighbor::sample_index

    fn Neighbor::sample_index(self : Neighbor) -> Int

    Neighbor::sample_name

    fn Neighbor::sample_name(self : Neighbor) -> String

    Pca

    pub struct Pca {
    features : Array[String]
    basis : Basis
    location : Array[Double]
    axes : Array[Array[Double]]
    variances : Array[Double]
    total : Double
    }

    Pca::center

    fn Pca::center(self : Pca) -> Array[Double] raise CodaError

    Pca::eigenvalues

    fn Pca::eigenvalues(self : Pca) -> Array[Double]

    Pca::explained_ratio

    fn Pca::explained_ratio(self : Pca) -> Array[Double]

    Fractions refer to total pre-truncation variance; constant data returns zeros.

    Pca::loadings

    fn Pca::loadings(self : Pca) -> Array[Array[Double]]

    Pca::reconstruct

    fn Pca::reconstruct(self : Pca, scores : Array[Array[Double]]) -> Array[Array[Double]] raise CodaError

    Reconstruction uses the fitted center and retained axes, not raw proportions.

    Pca::transform

    fn Pca::transform(self : Pca, data : Dataset) -> Array[Array[Double]] raise CodaError

    Replacement

    pub struct Replacement {
    values : Array[Double]
    count : Int
    mass : Double
    }

    Replacement::composition

    fn Replacement::composition(self : Replacement) -> Array[Double]

    Replacement::replaced_mass

    fn Replacement::replaced_mass(self : Replacement) -> Double

    Replacement::zero_count

    fn Replacement::zero_count(self : Replacement) -> Int

    Spectrum

    pub struct Spectrum {
    values : Array[Double]
    vectors : Array[Array[Double]]
    residual : Double
    }

    Descending eigenvalues with corresponding unit eigenvectors stored as rows.

    Spectrum::eigenvalues

    fn Spectrum::eigenvalues(self : Spectrum) -> Array[Double]

    Spectrum::eigenvectors

    fn Spectrum::eigenvectors(self : Spectrum) -> Array[Array[Double]]

    Spectrum::max_residual

    fn Spectrum::max_residual(self : Spectrum) -> Double

    Largest absolute entry of Av-lambdav, in original matrix units.

    add_pseudocount

    fn add_pseudocount(x : Array[Double], amount : Double) -> Array[Double] raise CodaError

    Add an explicit pseudocount in the original count unit, then close.

    alr

    fn alr(x : Array[Double], reference : Int) -> Array[Double] raise CodaError

    alr_covariance

    fn alr_covariance(data : Dataset, reference : Int, ddof? : Int) -> Array[Array[Double]] raise CodaError

    Correlation of log-ratios sharing a named denominator, not raw proportions.

    alr_inverse

    fn alr_inverse(z : Array[Double], reference : Int) -> Array[Double] raise CodaError

    amalgamate

    fn amalgamate(x : Array[Double], groups : Array[Array[Int]]) -> Array[Double] raise CodaError

    Groups must form a disjoint, exhaustive partition. This is NOT an isometry.

    balance

    fn balance(x : Array[Double], numerator : Array[Int], denominator : Array[Int]) -> Double raise CodaError

    One interpretable balance, usable without completing a whole SBP.

    basis

    fn basis(rows : Array[Array[Double]]) -> Basis raise CodaError

    center

    fn center(data : Dataset) -> Array[Double] raise CodaError

    center_distance

    fn center_distance(left : Dataset, right : Dataset) -> Double raise CodaError

    Between-center distance is descriptive, not a hypothesis test.

    centralize

    fn centralize(data : Dataset) -> Dataset raise CodaError

    Remove the geometric center by inverse perturbation; labels are retained.

    cholesky

    fn cholesky(matrix : Array[Array[Double]]) -> Cholesky raise CodaError

    closure

    fn closure(x : Array[Double]) -> Array[Double] raise CodaError

    Normalize nonnegative weights without summing large raw numbers. Zeros are allowed here; log-ratio routines require strictly positive parts.

    clr

    fn clr(x : Array[Double]) -> Array[Double] raise CodaError

    Center logarithms directly, avoiding closure underflow for wide ratios.

    clr_covariance

    fn clr_covariance(data : Dataset, ddof? : Int) -> Array[Array[Double]] raise CodaError

    clr_inverse

    fn clr_inverse(z : Array[Double]) -> Array[Double] raise CodaError

    Stable softmax. Reject an unrepresentable positive composition.

    dataset

    fn dataset(features : Array[String], samples : Array[String], rows : Array[Array[Double]]) -> Dataset raise CodaError

    distance

    fn distance(x : Array[Double], y : Array[Double]) -> Double raise CodaError

    gaussian

    fn gaussian(data : Dataset, ridge : Double, basis? : Basis) -> Gaussian raise CodaError

    Maximum-likelihood covariance (ddof=0) plus absolute ILR ridge > 0.

    geometric_mean

    fn geometric_mean(x : Array[Double]) -> Double raise CodaError

    Geometric mean may overflow/underflow only when the result is unrepresentable.

    helmert_basis

    fn helmert_basis(parts : Int) -> Basis raise CodaError

    ilr

    fn ilr(x : Array[Double], b : Basis) -> Array[Double] raise CodaError

    ilr_covariance

    fn ilr_covariance(data : Dataset, b : Basis, ddof? : Int) -> Array[Array[Double]] raise CodaError

    ilr_inverse

    fn ilr_inverse(z : Array[Double], b : Basis) -> Array[Double] raise CodaError

    inner

    fn inner(x : Array[Double], y : Array[Double]) -> Double raise CodaError

    interpolate

    fn interpolate(x : Array[Double], y : Array[Double], t : Double) -> Array[Double] raise CodaError

    Constant-speed Aitchison geodesic, including bounded extrapolation.

    kmeans

    fn kmeans(data : Dataset, count : Int, max_iterations? : Int) -> Clusters raise CodaError

    Deterministic farthest-first initialization; empty clusters retain their centers. Convergence means unchanged labels, not merely an exhausted iteration budget.

    medoid

    fn medoid(data : Dataset) -> Medoid raise CodaError

    Observed medoid minimizes total squared Aitchison separation.

    multiplicative_replace

    fn multiplicative_replace(x : Array[Double], delta? : Double) -> Replacement raise CodaError

    Replace each zero by delta after closure, scaling nonzero parts by 1-z*delta. Delta is a modeling choice, not an estimate of a structural zero.

    nearest

    fn nearest(data : Dataset, query : Array[Double], count : Int) -> Array[Neighbor] raise CodaError

    Deterministic stable ranking: tied distances retain sample order.

    pairwise_distances

    fn pairwise_distances(data : Dataset) -> Array[Array[Double]] raise CodaError

    Full symmetric distance matrix, bounded to avoid accidental quadratic memory.

    pca

    fn pca(data : Dataset, components : Int, basis? : Basis) -> Pca raise CodaError

    Sample-covariance PCA in isometric coordinates; zero eigenvalues are retained.

    permute

    fn permute(x : Array[Double], order : Array[Int]) -> Array[Double] raise CodaError

    Restore feature order after a permutation (not after feature removal).

    perturb

    fn perturb(x : Array[Double], y : Array[Double]) -> Array[Double] raise CodaError

    Aitchison addition, implemented in log space.

    perturb_inverse

    fn perturb_inverse(x : Array[Double], y : Array[Double]) -> Array[Double] raise CodaError

    positive

    fn positive(x : Array[Double]) -> Unit raise CodaError

    power

    fn power(x : Array[Double], exponent : Double) -> Array[Double] raise CodaError

    process_json

    fn process_json(text : String) -> String

    Pure bounded JSON interface. Errors never echo user-provided input or labels. JSON duplicate keys follow the core parser's last-value semantics.

    sbp_basis

    fn sbp_basis(partition : Array[Array[Int]]) -> Basis raise CodaError

    Convert a complete sequential binary partition to orthonormal balances. A row uses +1 and -1 for its two groups and 0 for parts outside that split.

    subcomposition

    fn subcomposition(x : Array[Double], indices : Array[Int]) -> Array[Double] raise CodaError

    symmetric_eigen

    fn symmetric_eigen(matrix : Array[Array[Double]]) -> Spectrum raise CodaError

    Symmetric cyclic Jacobi, scaled before iteration; no silent nonconvergence.

    total_variance

    fn total_variance(data : Dataset, ddof? : Int) -> Double raise CodaError

    variation_matrix

    fn variation_matrix(data : Dataset, ddof? : Int) -> Array[Array[Double]] raise CodaError

    Entry (i,j) is sample variance of log(x_i/x_j).

    weighted_center

    fn weighted_center(data : Dataset, weights : Array[Double]) -> Array[Double] raise CodaError

    weighted_dispersion

    fn weighted_dispersion(data : Dataset, weights : Array[Double]) -> Double raise CodaError