🥮 mooncakes.io

    moonbit-mirostat

    Model-independent Mirostat adaptive text sampling for MoonBit

    llm
    sampling
    mirostat
    decoding
    Download zip
    Version
    0.1.0
    License
    Apache-2.0
    Repository
    Last updated
    21 hours ago
    Downloads
    2

    #moonbit-mirostat

    A model-independent MoonBit implementation of Mirostat adaptive sampling. A caller passes one logit row and one uniform random draw per generated token. The library returns the chosen token, its probability and surprise under the original model distribution, the retained candidate count, and the updated feedback state.

    The package implements both algorithms in the Mirostat paper: V1 estimates a Zipf exponent and chooses an adaptive top-k; V2 retains tokens whose model surprise is at most the current mu. The feedback rule is mu := mu - eta * (observed_surprise - tau). tau, mu, and surprise use bits.

    #When to use it

    • A MoonBit text-generation loop can control the observed surprise of generated tokens without fixing a top-k or top-p threshold for every step.
    • A WebAssembly application can run sampling locally after a model inference backend supplies logits, without routing token selection through JavaScript.
    • A decoding experiment can replay identical logits and uniform draws through V1 and V2, then compare their retained vocabulary and surprise traces.

    This package does not run a language model, tokenize text, fetch weights, or own an RNG. The caller chooses how to obtain logits and randomness. It does not claim exact behavioral equivalence to a particular model runtime: runtimes may apply penalties, grammar masks, temperature, or other transformations before Mirostat, and those transformations change the input distribution.

    #Build and run

    moon check --deny-warn
moon test --deny-warn
moon run cmd/main
moon run cmd/experiment
moon run cmd/generate
moon run cmd/batch
moon run cmd/weights
moon run cmd/process
moon run cmd/sweep
moon run cmd/json

    cmd/main prints five reproducible decisions for each variant. cmd/experiment runs both variants for 1000 draws from the same static Zipf distribution and seed, then prints CSV summary metrics. This is a deterministic algorithm fixture, not a language-model quality benchmark. cmd/generate demonstrates a context-dependent callback and an end token using a tiny deterministic model fixture. cmd/batch, cmd/weights, and cmd/process demonstrate atomic batching, weight inputs, and explicit logit preprocessing. cmd/sweep evaluates a set of surprise targets on the same controlled fixture. cmd/json prints the JSON wire formats for a WebAssembly host.

    A typical integration creates one sampler per generation session:

    let config = @mirostat.Config::new(3.0, eta=0.1).unwrap()
    let sampler = @mirostat.Sampler::new(config, version=@mirostat.Version::v2())
    let step = sampler.sample([4.0, 2.0, 1.0], 0.42).unwrap()
    println(step.token())

    The uniform draw must be finite and in [0, 1). A model row must contain at least one finite logit. Negative infinity masks a token; NaN and positive infinity are errors. Token indices are zero based and preserve the incoming logit order. Equal probabilities rank by smaller token index, so a fixed input and draw produce a repeatable result. reset() restores initial mu and the step count. replay() accepts aligned arrays of rows and draws, and summarize() reports mean surprise, retained count, and absolute error against the target.

    If the model backend produces non-negative weights, use sample_weights(); the package normalizes them. preview() and preview_weights() show the candidate count, retained model mass, and expected surprise without drawing or changing state. ParkMiller supplies a reproducible portable RNG for examples; sample_with_rng() consumes it automatically. Production hosts can continue to pass uniform draws from their own RNG.

    checkpoint() and restore() support a generation branch that may need to roll back; fork() creates a separate session with the same feedback state. replay_atomic() restores state if a later row in a batch fails, while replay() retains earlier successful steps. Neither function owns the caller's random source.

    generate() drives a context-dependent model callback with generated token history and an optional end token. For a fixed distribution used repeatedly in an experiment, PreparedRow caches its softmax and full ranking, and sample_prepared() reuses them. TraceAccumulator computes online summary metrics without retaining the whole trace.

    For a model server that decodes several requests in one inference batch, sample_batch() accepts aligned sampler, logit-row, and uniform arrays. sample_batch_atomic() restores every sampler if any row fails. Each array entry should refer to a distinct generation session.

    #Host integration

    For a host that makes the categorical draw itself, call candidates() to get the token ids, original probabilities, and normalized candidate probabilities. After selecting one of those token ids, call observe_token() to validate it against the current Mirostat candidate set and update feedback. This is useful when a model runtime already owns its random generator or implements a fused sampling kernel.

    The jsonio package provides a compact JSON protocol over the same public library. init_json creates the initial portable state, session_json takes one next-token row together with prior state_mu and state_steps, batch_json processes multiple independent sessions atomically, and preview_json inspects a candidate set without consuming a draw. weight_json is the one-shot counterpart for a backend that returns non-negative weights. These helpers do not perform model inference or file/network I/O.

    #Validation scope

    The test suite covers stable softmax, masked logits, normalization, candidate selection, V1 and V2 feedback, state rollback, batch atomicity, JSON decoding, fixed-seed replay, synthetic Zipf drift, and WebAssembly-compatible execution. The synthetic experiments verify that the feedback loop approaches feasible target surprise levels for their fixtures. They do not establish text quality, model quality, or performance for a particular vocabulary size.

    #Numerical and state behavior

    Softmax subtracts the largest logit before exponentiation. The Zipf slope estimator uses adjacent ranked probabilities up to m; a flat or underflowed distribution uses a small positive limiting slope. V1 clamps estimated k to the vocabulary. V2 always retains the most probable token, even if its surprise exceeds mu. The selected token's surprise is measured against the original softmax, before truncation and renormalization. An invalid row or draw returns an error without changing sampler state. In a replay, earlier valid rows remain committed if a later row fails.

    V1 uses a bounded heap to inspect the highest m probabilities and then select its adaptive top-k candidates. V2 scans the vocabulary and sorts only the tokens below its surprise cutoff. Both methods still inspect every logit at each step. Large-vocabulary low-latency decoders should benchmark the package with their model and target settings before adoption.

    For controlled comparisons, sample_top_k, sample_top_p, and sample_temperature use the same stable probability and draw routines. Additional comparison samplers cover min-p, epsilon, and locally typical sampling. A caller may apply mask_logits, bias_logits, or penalize_repetition before Mirostat; these transformations are explicit so the reported surprise is based on the transformed model row. entropy, prefix_mass, expected_prefix_surprise, and perplexity provide small diagnostics for comparing distributions with an observed trace.

    #Origin and license

    This is an original MoonBit implementation of the algorithms and equations in Sourya Basu et al., Mirostat: A Neural Text Decoding Algorithm That Directly Controls Perplexity, ICLR 2021. The authors' Python research code was used as a reference for terminology and algorithm scope; this repository does not copy its source files. The upstream code is MIT licensed. This MoonBit package is licensed under Apache-2.0; see LICENSE.

    CandidateSet

    source
    pub struct CandidateSet {
      tokens : Array[Int]
      conditional : Array[Double]
      original : Array[Double]
      mu : Double
    } derive(
    Debug    )

    The current adaptive candidate distribution for an external categorical sampler. Arrays align by index: tokens[i] has conditional[i] probability after truncation and original[i] probability before truncation.

    CandidateSet::conditional

    source
    fn CandidateSet::conditional(self : CandidateSet) -> Array[Double]

    CandidateSet::conditional_probability

    source
    fn CandidateSet::conditional_probability(self : CandidateSet, token : Int) -> Double?

    Conditional probability after Mirostat truncation, or None if this token is not in the current candidate set.

    CandidateSet::mu

    source
    fn CandidateSet::mu(self : CandidateSet) -> Double

    CandidateSet::original

    source
    fn CandidateSet::original(self : CandidateSet) -> Array[Double]

    CandidateSet::original_probability

    source
    fn CandidateSet::original_probability(self : CandidateSet, token : Int) -> Double?

    Probability under the original model distribution, or None if this token is not in the current candidate set.

    CandidateSet::position

    source
    fn CandidateSet::position(self : CandidateSet, token : Int) -> Int?

    Return the candidate-list position for a vocabulary token. This position indexes the arrays returned by conditional() and original().

    CandidateSet::to_repr

    source

    CandidateSet::tokens

    source
    fn CandidateSet::tokens(self : CandidateSet) -> Array[Int]

    Checkpoint

    source
    pub struct Checkpoint {
      config : Config
      version : Version
      mu : Double
      steps : Int
    } derive(Eq, 
    Debug    )

    A snapshot of one sampler's feedback state and configuration. A checkpoint can be restored only to a sampler with the same configuration and variant.

    Checkpoint::equal

    source
    fn Checkpoint::equal(Checkpoint, Checkpoint) -> Bool

    Checkpoint::mu

    source
    fn Checkpoint::mu(self : Checkpoint) -> Double

    Checkpoint::not_equal

    source
    fn Checkpoint::not_equal(x : Checkpoint, y : Checkpoint) -> Bool

    Checkpoint::steps

    source
    fn Checkpoint::steps(self : Checkpoint) -> Int

    Checkpoint::to_repr

    source

    Config

    source
    pub struct Config {
      tau : Double
      eta : Double
      initial_mu : Double
      m : Int
    } derive(Eq, 
    Debug    )

    Mirostat feedback configuration. Tau and mu are measured in bits.

    Config::equal

    source
    fn Config::equal(Config, Config) -> Bool

    Config::eta

    source
    fn Config::eta(self : Config) -> Double

    Config::initial_mu

    source
    fn Config::initial_mu(self : Config) -> Double

    Config::m

    source
    fn Config::m(self : Config) -> Int

    Config::new

    source
    fn Config::new(tau : Double, eta? : Double, initial_mu? : Double, m? : Int) -> Result[Config, SamplingError]

    Config::not_equal

    source
    fn Config::not_equal(x : Config, y : Config) -> Bool

    Config::tau

    source
    fn Config::tau(self : Config) -> Double

    Config::to_repr

    source

    Generation

    source
    pub struct Generation {
      tokens : Array[Int]
      steps : Array[Step]
      reason : StopReason
    } derive(Eq, 
    Debug    )

    Generation::equal

    source
    fn Generation::equal(Generation, Generation) -> Bool

    Generation::not_equal

    source
    fn Generation::not_equal(x : Generation, y : Generation) -> Bool

    Generation::reason

    source
    fn Generation::reason(self : Generation) -> StopReason

    Generation::steps

    source
    fn Generation::steps(self : Generation) -> Array[Step]

    Generation::to_repr

    source

    Generation::tokens

    source
    fn Generation::tokens(self : Generation) -> Array[Int]

    ParkMiller

    source
    pub struct ParkMiller {
      state : Int
    } derive(
    Debug    )

    A small reproducible RNG for examples, replay fixtures, and portable sampling. Applications may supply uniforms from their own RNG instead.

    ParkMiller::new

    source
    fn ParkMiller::new(seed : Int) -> Result[ParkMiller, SamplingError]

    ParkMiller::next_int

    source
    fn ParkMiller::next_int(self : ParkMiller) -> Int

    Schrage's method computes (16807 * state) mod (2^31 - 1) without an overflowing 32-bit multiplication.

    ParkMiller::next_uniform

    source
    fn ParkMiller::next_uniform(self : ParkMiller) -> Double

    The result lies strictly inside (0, 1), which satisfies the sampler's required [0, 1) interval.

    ParkMiller::state

    source
    fn ParkMiller::state(self : ParkMiller) -> Int

    ParkMiller::to_repr

    source

    PreparedRow

    source
    pub struct PreparedRow {
      probabilities : Array[Double]
      order : Array[Int]
    } derive(
    Debug    )

    Immutable-by-convention model row prepared for repeated draws. The probabilities and complete rank order are computed only once.

    PreparedRow::from_logits

    source
    fn PreparedRow::from_logits(logits : Array[Double]) -> Result[PreparedRow, SamplingError]

    PreparedRow::from_weights

    source
    fn PreparedRow::from_weights(weights : Array[Double]) -> Result[PreparedRow, SamplingError]

    PreparedRow::probability

    source
    fn PreparedRow::probability(self : PreparedRow, token : Int) -> Result[Double, SamplingError]

    PreparedRow::to_repr

    source

    PreparedRow::vocabulary

    source
    fn PreparedRow::vocabulary(self : PreparedRow) -> Int

    Preview

    source
    pub struct Preview {
      vocabulary : Int
      kept_tokens : Int
      retained_mass : Double
      expected_surprise : Double
      mu : Double
    } derive(Eq, 
    Debug    )

    Candidate-set diagnostics computed without drawing a token or updating mu.

    Preview::equal

    source
    fn Preview::equal(Preview, Preview) -> Bool

    Preview::expected_surprise

    source
    fn Preview::expected_surprise(self : Preview) -> Double

    Preview::kept_tokens

    source
    fn Preview::kept_tokens(self : Preview) -> Int

    Preview::mu

    source
    fn Preview::mu(self : Preview) -> Double

    Preview::not_equal

    source
    fn Preview::not_equal(x : Preview, y : Preview) -> Bool

    Preview::retained_mass

    source
    fn Preview::retained_mass(self : Preview) -> Double

    Preview::to_repr

    source

    Preview::vocabulary

    source
    fn Preview::vocabulary(self : Preview) -> Int

    Sampler

    source
    pub struct Sampler {
      config : Config
      version : Version
      mu : Double
      steps : Int
    } derive(
    Debug    )

    A session owns only feedback state; the model owns logits and the caller owns RNG.

    Sampler::candidates

    source
    fn Sampler::candidates(self : Sampler, logits : Array[Double]) -> Result[CandidateSet, SamplingError]

    Sampler::candidates_weights

    source
    fn Sampler::candidates_weights(self : Sampler, weights : Array[Double]) -> Result[CandidateSet, SamplingError]

    Sampler::checkpoint

    source
    fn Sampler::checkpoint(self : Sampler) -> Checkpoint

    Sampler::fork

    source
    fn Sampler::fork(self : Sampler) -> Sampler

    Copy a sampler's feedback state into a separate mutable session. This supports branching generation without sharing subsequent feedback.

    Sampler::from_state

    source
    fn Sampler::from_state(config : Config, version : Version, state : SamplerState) -> Sampler

    Sampler::mu

    source
    fn Sampler::mu(self : Sampler) -> Double

    Sampler::new

    source
    fn Sampler::new(config : Config, version? : Version) -> Sampler

    Sampler::observe_prepared_token

    source
    fn Sampler::observe_prepared_token(self : Sampler, row : PreparedRow, token : Int) -> Result[Step, SamplingError]

    Validate an externally chosen token against a precomputed model row.

    Sampler::observe_token

    source
    fn Sampler::observe_token(self : Sampler, logits : Array[Double], token : Int) -> Result[Step, SamplingError]

    Advance Mirostat feedback after an external sampler chooses a token from this library's candidate set. The token is checked against the same adaptive truncation that sample would use. This lets a host own the RNG and categorical draw while MoonBit maintains the feedback state.

    Sampler::observe_weighted_token

    source
    fn Sampler::observe_weighted_token(self : Sampler, weights : Array[Double], token : Int) -> Result[Step, SamplingError]

    Sampler::preview

    source
    fn Sampler::preview(self : Sampler, logits : Array[Double]) -> Result[Preview, SamplingError]

    Inspect the next candidate set on a logits row without consuming a draw.

    Sampler::preview_prepared

    source
    fn Sampler::preview_prepared(self : Sampler, row : PreparedRow) -> Result[Preview, SamplingError]

    Preview a prepared row without redoing softmax or ranking.

    Sampler::preview_weights

    source
    fn Sampler::preview_weights(self : Sampler, weights : Array[Double]) -> Result[Preview, SamplingError]

    Sampler::reset

    source
    fn Sampler::reset(self : Sampler) -> Unit

    Sampler::restore

    source
    fn Sampler::restore(self : Sampler, checkpoint : Checkpoint) -> Result[Unit, SamplingError]

    Sampler::sample

    source
    fn Sampler::sample(self : Sampler, logits : Array[Double], uniform : Double) -> Result[Step, SamplingError]

    One complete Mirostat step. State changes only after every calculation succeeds, so malformed logits cannot corrupt an ongoing generation.

    Sampler::sample_prepared

    source
    fn Sampler::sample_prepared(self : Sampler, row : PreparedRow, uniform : Double) -> Result[Step, SamplingError]

    Draw using a prepared model row. Useful in simulations and repeated sampling from unchanged logits; real autoregressive models usually return a different row each step and can use sample directly.

    Sampler::sample_weights

    source
    fn Sampler::sample_weights(self : Sampler, weights : Array[Double], uniform : Double) -> Result[Step, SamplingError]

    Sample from an already computed model distribution, supplied as weights. The weights are normalized internally and need not sum to one.

    Sampler::sample_with_rng

    source
    fn Sampler::sample_with_rng(self : Sampler, logits : Array[Double], rng : ParkMiller) -> Result[Step, SamplingError]

    Convenience step using this package's reproducible generator. It consumes one draw even if the supplied logits are invalid; callers requiring exact rollback should validate or manage the RNG separately.

    Sampler::state

    source
    fn Sampler::state(self : Sampler) -> SamplerState

    Sampler::steps

    source
    fn Sampler::steps(self : Sampler) -> Int

    Sampler::to_repr

    source

    Sampler::version

    source
    fn Sampler::version(self : Sampler) -> Version

    SamplerState

    source
    pub struct SamplerState {
      mu : Double
      steps : Int
    } derive(Eq, 
    Debug    )

    Portable feedback state for a host that stores Mirostat session data between calls. The configuration and version must be stored separately.

    SamplerState::equal

    source

    SamplerState::mu

    source
    fn SamplerState::mu(self : SamplerState) -> Double

    SamplerState::new

    source
    fn SamplerState::new(mu : Double, steps : Int) -> Result[SamplerState, SamplingError]

    SamplerState::not_equal

    source
    fn SamplerState::not_equal(x : SamplerState, y : SamplerState) -> Bool

    SamplerState::steps

    source
    fn SamplerState::steps(self : SamplerState) -> Int

    SamplerState::to_repr

    source

    SamplingError

    source
    pub enum SamplingError {
      EmptyLogits
      AllMasked
      NonFiniteLogit(Int)
      InvalidUniform
      InvalidProbability(Int)
      InvalidParameter(String)
      NumericalFailure
    } derive(Eq, 
    Debug    )

    Errors are values so callers can decide whether to skip a malformed model row.

    SamplingError::equal

    source

    SamplingError::invalid_parameter

    source
    fn SamplingError::invalid_parameter(message : String) -> SamplingError

    SamplingError::message

    source
    fn SamplingError::message(self : SamplingError) -> String

    SamplingError::not_equal

    source
    fn SamplingError::not_equal(x : SamplingError, y : SamplingError) -> Bool

    SamplingError::to_repr

    source

    Step

    source
    pub struct Step {
      token : Int
      original_probability : Double
      observed_surprise : Double
      kept_tokens : Int
      mu_before : Double
      mu_after : Double
    } derive(Eq, 
    Debug    )

    Step::equal

    source
    fn Step::equal(Step, Step) -> Bool

    Step::kept_tokens

    source
    fn Step::kept_tokens(self : Step) -> Int

    Step::mu_after

    source
    fn Step::mu_after(self : Step) -> Double

    Step::mu_before

    source
    fn Step::mu_before(self : Step) -> Double

    Step::not_equal

    source
    fn Step::not_equal(x : Step, y : Step) -> Bool

    Step::observed_surprise

    source
    fn Step::observed_surprise(self : Step) -> Double

    Step::original_probability

    source
    fn Step::original_probability(self : Step) -> Double

    Step::to_repr

    source

    Step::token

    source
    fn Step::token(self : Step) -> Int

    StopReason

    source
    pub enum StopReason {
      Limit
      EndToken
      Observer
    } derive(Eq, 
    Debug    )

    StopReason::end_token

    source
    fn StopReason::end_token() -> StopReason

    StopReason::equal

    source
    fn StopReason::equal(StopReason, StopReason) -> Bool

    StopReason::limit

    source
    fn StopReason::limit() -> StopReason

    StopReason::not_equal

    source
    fn StopReason::not_equal(x : StopReason, y : StopReason) -> Bool

    StopReason::observer

    source
    fn StopReason::observer() -> StopReason

    StopReason::to_repr

    source

    StreamGeneration

    source
    pub struct StreamGeneration {
      tokens : Array[Int]
      reason : StopReason
    } derive(Eq, 
    Debug    )

    StreamGeneration::count

    source
    fn StreamGeneration::count(self : StreamGeneration) -> Int

    StreamGeneration::equal

    source

    StreamGeneration::not_equal

    source
    fn StreamGeneration::not_equal(x : StreamGeneration, y : StreamGeneration) -> Bool

    StreamGeneration::reason

    source

    StreamGeneration::to_repr

    source

    StreamGeneration::tokens

    source
    fn StreamGeneration::tokens(self : StreamGeneration) -> Array[Int]

    SurpriseWindow

    source
    pub struct SurpriseWindow {
      capacity : Int
      values : Array[Double]
      next : Int
      count : Int
      total : Double
    } derive(
    Debug    )

    Fixed-size online window for monitoring recent surprise without retaining an entire generation trace. Values are measured in bits.

    SurpriseWindow::capacity

    source
    fn SurpriseWindow::capacity(self : SurpriseWindow) -> Int

    SurpriseWindow::clear

    source
    fn SurpriseWindow::clear(self : SurpriseWindow) -> Unit

    SurpriseWindow::count

    source
    fn SurpriseWindow::count(self : SurpriseWindow) -> Int

    SurpriseWindow::mean

    source
    fn SurpriseWindow::mean(self : SurpriseWindow) -> Result[Double, SamplingError]

    SurpriseWindow::new

    source
    fn SurpriseWindow::new(capacity : Int) -> Result[SurpriseWindow, SamplingError]

    SurpriseWindow::push

    source
    fn SurpriseWindow::push(self : SurpriseWindow, surprise : Double) -> Result[Unit, SamplingError]

    SurpriseWindow::push_step

    source
    fn SurpriseWindow::push_step(self : SurpriseWindow, step : Step) -> Result[Unit, SamplingError]

    SurpriseWindow::to_repr

    source

    TraceAccumulator

    source
    pub struct TraceAccumulator {
      target : Double
      count : Int
      total_surprise : Double
      total_kept : Double
      absolute_error : Double
      final_mu : Double
    } derive(
    Debug    )

    Online trace statistics for applications that do not want to retain every step. Push only successful Mirostat decisions into this accumulator.

    TraceAccumulator::count

    source
    fn TraceAccumulator::count(self : TraceAccumulator) -> Int

    TraceAccumulator::new

    source
    fn TraceAccumulator::new(target : Double) -> Result[TraceAccumulator, SamplingError]

    TraceAccumulator::push

    source
    fn TraceAccumulator::push(self : TraceAccumulator, step : Step) -> Result[Unit, SamplingError]

    TraceAccumulator::summary

    source

    TraceAccumulator::to_repr

    source

    Newer MoonBit toolchains no longer implicitly promote these methods.

    TraceSpread

    source
    pub struct TraceSpread {
      minimum : Double
      maximum : Double
      variance : Double
      median : Double
      p90 : Double
      minimum_kept : Int
      maximum_kept : Int
    } derive(Eq, 
    Debug    )

    Dispersion diagnostics for token-level observed surprise. Variance is the population variance in squared bits. Percentiles use nearest lower rank.

    TraceSpread::equal

    source
    fn TraceSpread::equal(TraceSpread, TraceSpread) -> Bool

    TraceSpread::maximum

    source
    fn TraceSpread::maximum(self : TraceSpread) -> Double

    TraceSpread::maximum_kept

    source
    fn TraceSpread::maximum_kept(self : TraceSpread) -> Int

    TraceSpread::median

    source
    fn TraceSpread::median(self : TraceSpread) -> Double

    TraceSpread::minimum

    source
    fn TraceSpread::minimum(self : TraceSpread) -> Double

    TraceSpread::minimum_kept

    source
    fn TraceSpread::minimum_kept(self : TraceSpread) -> Int

    TraceSpread::not_equal

    source
    fn TraceSpread::not_equal(x : TraceSpread, y : TraceSpread) -> Bool

    TraceSpread::p90

    source
    fn TraceSpread::p90(self : TraceSpread) -> Double

    TraceSpread::to_repr

    source

    TraceSpread::variance

    source
    fn TraceSpread::variance(self : TraceSpread) -> Double

    TraceSummary

    source
    pub struct TraceSummary {
      count : Int
      mean_surprise : Double
      mean_kept : Double
      mean_absolute_error : Double
      final_mu : Double
    } derive(Eq, 
    Debug    )

    Aggregate diagnostics for a completed sequence of Mirostat steps.

    TraceSummary::count

    source
    fn TraceSummary::count(self : TraceSummary) -> Int

    TraceSummary::equal

    source

    TraceSummary::final_mu

    source
    fn TraceSummary::final_mu(self : TraceSummary) -> Double

    TraceSummary::mean_absolute_error

    source
    fn TraceSummary::mean_absolute_error(self : TraceSummary) -> Double

    TraceSummary::mean_kept

    source
    fn TraceSummary::mean_kept(self : TraceSummary) -> Double

    TraceSummary::mean_surprise

    source
    fn TraceSummary::mean_surprise(self : TraceSummary) -> Double

    TraceSummary::not_equal

    source
    fn TraceSummary::not_equal(x : TraceSummary, y : TraceSummary) -> Bool

    TraceSummary::to_repr

    source

    Version

    source
    pub enum Version {
      V1
      V2
    } derive(Eq, 
    Debug    )

    Version::equal

    source
    fn Version::equal(Version, Version) -> Bool

    Version::not_equal

    source
    fn Version::not_equal(x : Version, y : Version) -> Bool

    Version::to_repr

    source

    Version::v1

    source
    fn Version::v1() -> Version

    Version::v2

    source
    fn Version::v2() -> Version

    allow_token_ids

    source
    fn allow_token_ids(logits : Array[Double], allowed_tokens : Array[Int]) -> Result[Array[Double], SamplingError]

    Keep only a sparse set of allowed token ids. A valid nonempty support must remain after applying the allowlist and any masks already in the logits.

    bias_logits

    source
    fn bias_logits(logits : Array[Double], biases : Array[Double]) -> Result[Array[Double], SamplingError]

    Add a per-token logit bias. Negative infinity in the base row remains masked, and every bias must be finite. A bias of zero leaves the row alone.

    block_token_ids

    source
    fn block_token_ids(logits : Array[Double], blocked : Array[Int]) -> Result[Array[Double], SamplingError]

    Mask a sparse list of forbidden token ids without building a full Boolean mask in the caller. Repeated ids have the same effect as one occurrence.

    draw_candidates

    source
    fn draw_candidates(probabilities : Array[Double], candidates : Array[Int], uniform : Double) -> Result[Int, SamplingError]

    Draw from an arbitrary list of distinct candidate token indices. This accepts a partial vocabulary and retains the supplied candidate order.

    draw_prefix

    source
    fn draw_prefix(probabilities : Array[Double], order : Array[Int], keep : Int, uniform : Double) -> Result[Int, SamplingError]

    Draw from a ranked prefix using one externally supplied uniform number. The caller controls randomness, making a run exactly replayable.

    entropy

    source
    fn entropy(probabilities : Array[Double]) -> Result[Double, SamplingError]

    Shannon entropy of a model row, in bits. Zero-mass tail entries contribute nothing, as in the continuous limit of -p log p.

    epsilon_prefix_size

    source
    fn epsilon_prefix_size(probabilities : Array[Double], order : Array[Int], epsilon : Double) -> Result[Int, SamplingError]

    Keep the ranked prefix whose tokens each have probability at least the absolute epsilon threshold. At least the best token survives.

    estimated_k

    source
    fn estimated_k(mu : Double, exponent : Double, vocabulary : Int) -> Result[Int, SamplingError]

    The paper's adaptive top-k estimate. Near s=1 the limiting expression uses ln(vocabulary size), avoiding a 0/0 numerical failure.

    expected_prefix_surprise

    source
    fn expected_prefix_surprise(probabilities : Array[Double], order : Array[Int], keep : Int) -> Result[Double, SamplingError]

    Expected observed surprise when drawing from a renormalized prefix. The surprise itself is measured against the full original model distribution.

    generate

    source
    fn generate(sampler : Sampler, rng : ParkMiller, model : (Array[Int]) -> Result[Array[Double], SamplingError], limit : Int, end_token : Int?) -> Result[Generation, SamplingError]

    Drive an autoregressive model callback until an end token or step limit. The callback receives a copy of the generated prefix and must return the next logit row. The RNG is consumed only after a model row is available. Any error preserves successful prior sampler steps and RNG draws.

    generate_from_prompt

    source
    fn generate_from_prompt(sampler : Sampler, rng : ParkMiller, model : (Array[Int]) -> Result[Array[Double], SamplingError], prompt : Array[Int], limit : Int, end_token : Int?) -> Result[Generation, SamplingError]

    Generate after a pre-existing prompt. The callback sees prompt followed by generated tokens; the returned token list contains only new tokens.

    generate_stream

    source
    fn generate_stream(sampler : Sampler, rng : ParkMiller, model : (Array[Int]) -> Result[Array[Double], SamplingError], prompt : Array[Int], limit : Int, end_token : Int?, observer : (Step) -> Bool) -> Result[StreamGeneration, SamplingError]

    Stream each successful decision to an observer and avoid retaining Step metadata. Return false from the observer to stop after that token. End token detection takes priority when both conditions apply to one step.

    mask_logits

    source
    fn mask_logits(logits : Array[Double], allowed : Array[Bool]) -> Result[Array[Double], SamplingError]

    Apply an allow-mask. This is useful when a grammar or host application restricts the next token vocabulary. At least one allowed finite logit is required. The input array is not mutated.

    min_p_prefix_size

    source
    fn min_p_prefix_size(probabilities : Array[Double], order : Array[Int], min_p : Double) -> Result[Int, SamplingError]

    Keep every token whose original model probability is at least a fraction of the best token's probability. The best token is always kept.

    normalize_weights

    source
    fn normalize_weights(weights : Array[Double]) -> Result[Array[Double], SamplingError]

    Normalize externally supplied non-negative weights. This is useful when a model backend already supplies probabilities rather than logits.

    penalize_counts

    source
    fn penalize_counts(logits : Array[Double], history : Array[Int], presence : Double, frequency : Double) -> Result[Array[Double], SamplingError]

    Subtract an additive presence cost once per seen token and an additive frequency cost for every occurrence. Costs may be negative to reward repetition. This is a preprocessing option outside Mirostat feedback.

    penalize_repetition

    source
    fn penalize_repetition(logits : Array[Double], history : Array[Int], penalty : Double) -> Result[Array[Double], SamplingError]

    Apply the usual sign-aware repetition penalty once for each token that appears in history. This is a caller-selected preprocessing step, not part of the Mirostat algorithm itself.

    perplexity

    source
    fn perplexity(average_surprise : Double) -> Result[Double, SamplingError]

    Perplexity corresponding to average surprise measured in bits.

    prefix_entropy

    source
    fn prefix_entropy(probabilities : Array[Double], order : Array[Int], keep : Int) -> Result[Double, SamplingError]

    Entropy of the renormalized candidate prefix, in bits. It differs from expected model surprise because the latter uses original probabilities.

    prefix_kl

    source
    fn prefix_kl(probabilities : Array[Double], order : Array[Int], keep : Int) -> Result[Double, SamplingError]

    KL divergence in bits from the renormalized prefix distribution to the original distribution. It depends only on retained original probability mass and is zero when the entire support is kept.

    prefix_mass

    source
    fn prefix_mass(probabilities : Array[Double], order : Array[Int], keep : Int) -> Result[Double, SamplingError]

    Original model mass covered by a ranked candidate prefix.

    prefix_surprise_variance

    source
    fn prefix_surprise_variance(probabilities : Array[Double], order : Array[Int], keep : Int) -> Result[Double, SamplingError]

    Variance of token surprise under the renormalized candidate prefix. This exposes how widely token-level surprises may fluctuate around their mean.

    probabilities

    source
    fn probabilities(logits : Array[Double]) -> Result[Array[Double], SamplingError]

    Stable softmax. Subtracting the maximum avoids overflow for large logits.

    rank

    source
    fn rank(probabilities : Array[Double]) -> Result[Array[Int], SamplingError]

    Sort token ids by descending probability; ties favor the lower id.

    replay

    source
    fn replay(sampler : Sampler, rows : Array[Array[Double]], draws : Array[Double]) -> Result[Array[Step], SamplingError]

    Replay precomputed model rows and RNG draws, returning every decision. A failed row leaves the sampler at the last successful step.

    replay_atomic

    source
    fn replay_atomic(sampler : Sampler, rows : Array[Array[Double]], draws : Array[Double]) -> Result[Array[Step], SamplingError]

    Replay a batch as one transaction. If any row fails, feedback state is restored to its value before the batch.

    rolling_surprise

    source
    fn rolling_surprise(steps : Array[Step], window : Int) -> Result[Array[Double], SamplingError]

    Average surprise over the most recent window tokens at every position. The initial entries use all observations available so far.

    running_surprise

    source
    fn running_surprise(steps : Array[Step]) -> Array[Double]

    Cumulative average surprise after each generated token. This is the observed cross-entropy estimate used to assess long-run target control.

    sample_batch

    source
    fn sample_batch(samplers : Array[Sampler], logits : Array[Array[Double]], uniforms : Array[Double]) -> Result[Array[Step], SamplingError]

    Sample one next token for each independent generation session. Input arrays must align by index. On an error, earlier sessions have already advanced.

    sample_batch_atomic

    source
    fn sample_batch_atomic(samplers : Array[Sampler], logits : Array[Array[Double]], uniforms : Array[Double]) -> Result[Array[Step], SamplingError]

    Sample all sessions as a transaction. If any row fails, every sampler is restored to its previous feedback state. External RNG state is unchanged because uniforms are supplied as values.

    sample_epsilon

    source
    fn sample_epsilon(logits : Array[Double], epsilon : Double, uniform : Double) -> Result[Int, SamplingError]

    sample_min_p

    source
    fn sample_min_p(logits : Array[Double], min_p : Double, uniform : Double) -> Result[Int, SamplingError]

    Sample from a min-p filtered model row.

    sample_shared_prepared

    source
    fn sample_shared_prepared(samplers : Array[Sampler], row : PreparedRow, uniforms : Array[Double]) -> Result[Array[Step], SamplingError]

    Draw one token for each independent session from a shared prepared model row. This is useful for Monte Carlo comparisons using one fixed logits row while each session maintains its own Mirostat feedback state.

    sample_temperature

    source
    fn sample_temperature(logits : Array[Double], temperature : Double, uniform : Double) -> Result[Int, SamplingError]

    Independent temperature sampling without truncation.

    sample_top_k

    source
    fn sample_top_k(logits : Array[Double], k : Int, uniform : Double) -> Result[Int, SamplingError]

    Stateless top-k sampling is useful as a controlled comparison for Mirostat.

    sample_top_p

    source
    fn sample_top_p(logits : Array[Double], threshold : Double, uniform : Double) -> Result[Int, SamplingError]

    Stateless nucleus sampling on the original model distribution.

    sample_typical

    source
    fn sample_typical(logits : Array[Double], threshold : Double, uniform : Double) -> Result[Int, SamplingError]

    summarize

    source
    fn summarize(steps : Array[Step], target : Double) -> Result[TraceSummary, SamplingError]

    Summarize a trace against the requested surprise target.

    surprise

    source
    fn surprise(probability : Double) -> Result[Double, SamplingError]

    Information content in bits under the untruncated model distribution.

    top_indices

    source
    fn top_indices(probabilities : Array[Double], k : Int) -> Result[Array[Int], SamplingError]

    Return the k highest-probability token indices in descending order. A size-k heap avoids sorting the full vocabulary when k is small.

    top_p_prefix_size

    source
    fn top_p_prefix_size(probabilities : Array[Double], order : Array[Int], threshold : Double) -> Result[Int, SamplingError]

    Return the smallest top-probability prefix with mass at least threshold.

    trace_spread

    source
    fn trace_spread(trace : Array[Step]) -> Result[TraceSpread, SamplingError]

    typical_order

    source
    fn typical_order(probabilities : Array[Double]) -> Result[Array[Int], SamplingError]

    Sort tokens by the distance between their surprise and the model entropy. Zero-probability tokens are placed last, with token id breaking ties.

    typical_prefix_size

    source
    fn typical_prefix_size(probabilities : Array[Double], order : Array[Int], threshold : Double) -> Result[Int, SamplingError]

    Choose the smallest locally typical prefix with cumulative mass at least threshold. Unlike top-p, the candidate order need not follow probability.

    v2_prefix_size

    source
    fn v2_prefix_size(probabilities : Array[Double], order : Array[Int], mu : Double) -> Result[Int, SamplingError]

    V2 uses the current surprise target as a cutoff on original token probabilities. It always keeps at least the most likely token.

    with_temperature

    source
    fn with_temperature(logits : Array[Double], temperature : Double) -> Result[Array[Double], SamplingError]

    Scale logits by positive temperature before another sampling operation.

    zipf_exponent

    source
    fn zipf_exponent(probabilities : Array[Double], order : Array[Int], m : Int) -> Result[Double, SamplingError]

    Estimate the Zipf exponent from adjacent probability ratios in the top m. This follows the paper's least-squares estimator and uses log base two.

    Source Files

    Powered by MoonBit

    Site sourceReport issuePackagesBuild queueSkillsStatistics

    © 2026 mooncakes.io