🥮 mooncakes.io

    moonbit-treespec

    MoonBit tree speculative decoding algorithms and reproducible offline experiments

    llm
    decoding
    speculative-decoding
    inference
    simulation
    Download zip
    Author
    Version
    0.1.0
    License
    Apache-2.0
    Repository
    Last updated
    21 hours ago
    Downloads
    1

    #MoonBit TreeSpec

    The canonical package documentation is README.md. See docs/algorithm.md for the corrected sampling contract, proof sketch, test evidence, and explicit backend limits.

    AdaptiveBenchmark

    source
    pub struct AdaptiveBenchmark {
      model_id : String
      prompt_tokens : Int
      output_tokens : Int
      baseline_target_requests : Int
      adaptive_target_requests : Int
      adaptive_draft_requests : Int
      adaptive_rounds : Int
      adaptive_candidate_nodes : Int
      adaptive_accepted_nodes : Int
      planned_depths : Array[Int]
    }

    AdaptiveBenchmark::planned_depths

    source
    fn AdaptiveBenchmark::planned_depths(self : AdaptiveBenchmark) -> Array[Int]

    AdaptiveBenchmark::render

    source
    fn AdaptiveBenchmark::render(self : AdaptiveBenchmark) -> String

    Stable text output suitable for a checked-in benchmark record. The request ratio is algorithmic work, not an assertion about latency or throughput.

    AdaptiveBenchmark::target_request_ratio

    source
    fn AdaptiveBenchmark::target_request_ratio(self : AdaptiveBenchmark) -> Double

    AdaptiveBenchmark::target_request_reduction

    source
    fn AdaptiveBenchmark::target_request_reduction(self : AdaptiveBenchmark) -> Int

    AdaptiveDecodingRun

    source
    pub struct AdaptiveDecodingRun {
      run : DecodingRun
      planned_depths : Array[Int]
    }

    A batch-decoding run together with the proposal depth selected before each round. The trace makes adaptive behavior inspectable without exposing a mutable policy to callers after decoding has finished.

    AdaptiveDecodingRun::output_tokens

    source
    fn AdaptiveDecodingRun::output_tokens(self : AdaptiveDecodingRun) -> Int

    AdaptiveDecodingRun::planned_depths

    source
    fn AdaptiveDecodingRun::planned_depths(self : AdaptiveDecodingRun) -> Array[Int]

    AdaptiveDecodingRun::target_requests

    source
    fn AdaptiveDecodingRun::target_requests(self : AdaptiveDecodingRun) -> Int

    AdaptivePolicy

    source
    pub struct AdaptivePolicy {
      depth : Int
      min_depth : Int
      max_depth : Int
      target_acceptance : Double
    }

    AdaptivePolicy::depth

    source
    fn AdaptivePolicy::depth(self : AdaptivePolicy) -> Int

    AdaptivePolicy::new

    source
    fn AdaptivePolicy::new(min_depth : Int, max_depth : Int, target_acceptance : Double) -> AdaptivePolicy

    AdaptivePolicy::observe

    source
    fn AdaptivePolicy::observe(self : AdaptivePolicy, accepted : Int, proposed : Int) -> Unit

    AdaptiveTreeError

    source
    pub enum AdaptiveTreeError {
      InvalidWidth
      InvalidDepth
      InvalidBudget
      InvalidEntropyThreshold
      EmptyLogitSchedule
      ProbabilityFailure
    } derive(Eq, 
    Debug    )

    an explicit global node budget.

    AdaptiveTreePolicy

    source
    pub struct AdaptiveTreePolicy {
      min_width : Int
      max_width : Int
      min_depth : Int
      max_depth : Int
      max_nodes : Int
      entropy_threshold : Double
      target_acceptance : Double
      depth : Int
    }

    different properties: expected draft quality and local token ambiguity.

    AdaptiveTreePolicy::build

    source
    fn AdaptiveTreePolicy::build(self : AdaptiveTreePolicy, prefix : Array[Int], depth_logits : Array[Array[Double]]) -> Result[DraftTree, AdaptiveTreeError]

    rows per parent, then use plan_tree_batch to execute them.

    AdaptiveTreePolicy::build_from_batch_model

    source
    fn AdaptiveTreePolicy::build_from_batch_model(self : AdaptiveTreePolicy, prefix : Array[Int], model : (Array[Array[Int]]) -> Result[Array[Array[Double]], String]) -> Result[ModelDraftTree, ModelTreeError]

    Entropy-adaptive counterpart of TreePolicy::build_from_batch_model. Branching width is computed separately for each parent row in a batch.

    AdaptiveTreePolicy::build_from_batch_model_with_depth_limit

    source
    fn AdaptiveTreePolicy::build_from_batch_model_with_depth_limit(self : AdaptiveTreePolicy, prefix : Array[Int], depth_limit : Int, model : (Array[Array[Int]]) -> Result[Array[Array[Double]], String]) -> Result[ModelDraftTree, ModelTreeError]

    Build an adaptive draft tree while capping the current depth. Decoders use this at an output boundary so they never score suffix tokens that cannot be emitted in the requested budget.

    AdaptiveTreePolicy::build_from_model

    source
    fn AdaptiveTreePolicy::build_from_model(self : AdaptiveTreePolicy, prefix : Array[Int], model : (Array[Int]) -> Result[Array[Double], String]) -> Result[ModelDraftTree, ModelTreeError]

    AdaptiveTreePolicy::depth

    source
    fn AdaptiveTreePolicy::depth(self : AdaptiveTreePolicy) -> Int

    AdaptiveTreePolicy::describe

    source
    fn AdaptiveTreePolicy::describe(self : AdaptiveTreePolicy) -> String

    Explain the active policy state in a stable format for benchmark metadata.

    AdaptiveTreePolicy::new

    source
    fn AdaptiveTreePolicy::new(min_width : Int, max_width : Int, min_depth : Int, max_depth : Int, max_nodes : Int, entropy_threshold : Double, target_acceptance : Double) -> Result[AdaptiveTreePolicy, AdaptiveTreeError]

    AdaptiveTreePolicy::observe

    source
    fn AdaptiveTreePolicy::observe(self : AdaptiveTreePolicy, accepted : Int, proposed : Int) -> Unit

    does not change width: width remains tied to next-step uncertainty.

    AdaptiveTreePolicy::width_for_logits

    source
    fn AdaptiveTreePolicy::width_for_logits(self : AdaptiveTreePolicy, logits : Array[Double]) -> Result[Int, AdaptiveTreeError]

    unbounded number of policy states.

    BaselineError

    source
    pub enum BaselineError {
      EmptySchedule
      ProbabilityFailure(Int)
    } derive(Eq, 
    Debug    )

    apples-to-apples: one target distribution produces exactly one token.

    BenchmarkError

    source
    pub enum BenchmarkError {
      InvalidModelId
      InvalidOutputBudget
      BaselineFailure(TreeExperimentError)
      AdaptiveFailure(TreeExperimentError)
      OutputLengthMismatch
    } derive(Eq, 
    Debug    )

    around the same callbacks when it owns the transport and hardware.

    BudgetError

    source
    pub enum BudgetError {
      InvalidLimit
      TargetCallLimit
      OutputTokenLimit
      DraftTokenLimit
    } derive(Eq, 
    Debug    )

    request into an unbounded amount of draft, target, or KV-cache work.

    CliError

    source
    pub enum CliError {
      UnknownCommand(String)
      DemoFailure
      WorkloadFailure
      TreePlanFailure
      ExperimentFailure
    } derive(Eq, 
    Debug    )

    lets downstream tools reuse the same demonstrations.

    ConstraintError

    source
    pub enum ConstraintError {
      InvalidVocabulary(Int)
      InvalidToken(Int)
      EmptyAllowedSet
      EmptyStopSequence
      DistributionMismatch
      SamplingFailure
    } derive(Eq, 
    Debug    )

    without coupling TreeSpec to one tokenizer.

    DecisionKind

    source
    pub enum DecisionKind {
      Accepted
      Rejected
    } derive(Eq, 
    Debug    )

    trace can be inspected or serialized by a host application.

    DecisionKind::label

    source
    fn DecisionKind::label(self : DecisionKind) -> String

    DecisionTrace

    source
    pub struct DecisionTrace {
      round : Int
      node_id : Int?
      token : Int
      kind : DecisionKind
      target_probability : Double
      draft_probability : Double
      threshold : Double
    }

    DecisionTrace::summary

    source
    fn DecisionTrace::summary(self : DecisionTrace) -> String

    DecodeBudget

    source
    pub struct DecodeBudget {
      max_target_batches : Int
      max_output_tokens : Int
      max_draft_tokens : Int
    }

    for a dimension it does not need to constrain.

    DecodeBudget::allow_emission

    source
    fn DecodeBudget::allow_emission(self : DecodeBudget, metrics : DecodeMetrics, emitted_count : Int) -> Result[Unit, BudgetError]

    caller that supplies a target backend capable of accepting an entire path.

    DecodeBudget::allow_round

    source
    fn DecodeBudget::allow_round(self : DecodeBudget, metrics : DecodeMetrics, proposal_depth : Int) -> Result[Unit, BudgetError]

    is checked against the proposal depth as a safe upper bound.

    DecodeBudget::demo

    source

    A modest default for CLI demonstrations, not a hidden global policy.

    DecodeBudget::describe_remaining

    source
    fn DecodeBudget::describe_remaining(self : DecodeBudget, metrics : DecodeMetrics) -> String

    Explain the remaining headroom in plain text for a CLI or a request log.

    DecodeBudget::max_draft_tokens

    source
    fn DecodeBudget::max_draft_tokens(self : DecodeBudget) -> Int

    DecodeBudget::max_output_tokens

    source
    fn DecodeBudget::max_output_tokens(self : DecodeBudget) -> Int

    DecodeBudget::max_target_batches

    source
    fn DecodeBudget::max_target_batches(self : DecodeBudget) -> Int

    DecodeBudget::new

    source
    fn DecodeBudget::new(max_target_batches : Int, max_output_tokens : Int, max_draft_tokens : Int) -> Result[DecodeBudget, BudgetError]

    DecodeMetrics

    source
    pub struct DecodeMetrics {
      target_batches : Int
      draft_tokens : Int
      emitted_tokens : Int
      accepted_tokens : Int
      rejected_rounds : Int
    }

    Logical scheduling counters used by offline simulations. target_batches is not an observed number of model API calls; TreeExperiment separately records actual target_queries from its provider callbacks.

    DecodeMetrics::acceptance_rate

    source
    fn DecodeMetrics::acceptance_rate(self : DecodeMetrics) -> Double

    DecodeMetrics::copy

    source

    Copy counters to avoid mutating historical reports through shared state.

    DecodeMetrics::empty

    source

    DecodeMetrics::record

    source
    fn DecodeMetrics::record(self : DecodeMetrics, result : VerifyResult, proposal : DraftProposal) -> Unit

    DecodeMetrics::tokens_per_target_batch

    source
    fn DecodeMetrics::tokens_per_target_batch(self : DecodeMetrics) -> Double

    DecodeSession

    source
    pub struct DecodeSession {
      generated : Array[Int]
      metrics : DecodeMetrics
      budget : DecodeBudget
      cache : KvSlots
      stop_reason : StopReason
      pending : (Int, Int)?
    }

    lightweight slot model to a real KV cache implementation.

    DecodeSession::apply_round

    source
    fn DecodeSession::apply_round(self : DecodeSession, transaction : Int, result : ReplayRoundResult) -> Result[SessionSnapshot, SessionError]

    committed here because a target backend owns that new token's cache entry.

    DecodeSession::cancel_round

    source
    fn DecodeSession::cancel_round(self : DecodeSession, transaction : Int) -> Result[Unit, SessionError]

    Cancel a reservation after a model provider fails, without ending the request.

    DecodeSession::finish

    source
    fn DecodeSession::finish(self : DecodeSession) -> Unit

    application observes its EOS token.

    DecodeSession::generated

    source
    fn DecodeSession::generated(self : DecodeSession) -> Array[Int]

    DecodeSession::is_running

    source
    fn DecodeSession::is_running(self : DecodeSession) -> Bool

    DecodeSession::metrics

    source

    DecodeSession::new

    source
    fn DecodeSession::new(prefix : Array[Int], budget : DecodeBudget, cache_capacity : Int) -> DecodeSession

    DecodeSession::prepare_round

    source
    fn DecodeSession::prepare_round(self : DecodeSession, depth : Int) -> Result[(Int, Array[Int]), SessionError]

    round. Call apply_round with the result only if this function succeeds.

    DecodeSession::release_cache

    source
    fn DecodeSession::release_cache(self : DecodeSession, transaction : Int) -> Result[Unit, SessionError]

    serving code a distinct lifecycle hook when it evicts a completed request.

    DecodeSession::render

    source
    fn DecodeSession::render(self : DecodeSession) -> String

    DecodeSession::stop_reason

    source
    fn DecodeSession::stop_reason(self : DecodeSession) -> StopReason

    DecodeTrace

    source
    pub struct DecodeTrace {
      entries : Array[DecisionTrace]
    }

    DecodeTrace::accepted_count

    source
    fn DecodeTrace::accepted_count(self : DecodeTrace) -> Int

    DecodeTrace::empty

    source
    fn DecodeTrace::empty() -> DecodeTrace

    DecodeTrace::push

    source
    fn DecodeTrace::push(self : DecodeTrace, entry : DecisionTrace) -> Unit

    DecodeTrace::rejected_count

    source
    fn DecodeTrace::rejected_count(self : DecodeTrace) -> Int

    DecodeTrace::render

    source
    fn DecodeTrace::render(self : DecodeTrace) -> String

    DecodingRun

    source
    pub struct DecodingRun {
      generated : Array[Int]
      target_queries : Int
      draft_queries : Int
      logical_target_batches : Int
      candidate_nodes : Int
      accepted_nodes : Int
    }

    Generated contains only new tokens, excluding the prompt.

    DeterministicRng

    source
    pub struct DeterministicRng {
      state : Int
    }

    method so intermediate multiplication stays within signed 32-bit range.

    DeterministicRng::new

    source
    fn DeterministicRng::new(seed : Int) -> Result[DeterministicRng, RngError]

    DeterministicRng::next

    source
    fn DeterministicRng::next(self : DeterministicRng) -> Int

    Advance once and return an integer in [1, 2147483646].

    DeterministicRng::next_below

    source
    fn DeterministicRng::next_below(self : DeterministicRng, bound : Int) -> Result[Int, RngError]

    important property is exact replay from the same seed.

    DeterministicRng::next_unit

    source
    fn DeterministicRng::next_unit(self : DeterministicRng) -> Double

    Return a portable unit-interval value in [0, 1).

    DeterministicRng::state

    source
    fn DeterministicRng::state(self : DeterministicRng) -> Int

    DeterministicRng::uniforms

    source
    fn DeterministicRng::uniforms(self : DeterministicRng, count : Int) -> Result[Array[Double], RngError]

    Fill caller-owned fixture arrays with deterministic random thresholds.

    DiagnosticCheck

    source
    pub struct DiagnosticCheck {
      name : String
      status : DiagnosticStatus
      detail : String
    }

    One named, independently meaningful preflight check.

    DiagnosticCheck::passed

    source
    fn DiagnosticCheck::passed(self : DiagnosticCheck) -> Bool

    DiagnosticCheck::render

    source
    fn DiagnosticCheck::render(self : DiagnosticCheck) -> String

    DiagnosticReport

    source
    pub struct DiagnosticReport {
      checks : Array[DiagnosticCheck]
    }

    actionable issues at once instead of fixing one exception at a time.

    DiagnosticReport::add

    source
    fn DiagnosticReport::add(self : DiagnosticReport, name : String, passed : Bool, detail : String) -> Unit

    DiagnosticReport::empty

    source

    DiagnosticReport::failure_count

    source
    fn DiagnosticReport::failure_count(self : DiagnosticReport) -> Int

    DiagnosticReport::passed

    source
    fn DiagnosticReport::passed(self : DiagnosticReport) -> Bool

    DiagnosticReport::render

    source
    fn DiagnosticReport::render(self : DiagnosticReport) -> String

    DiagnosticStatus

    source
    pub enum DiagnosticStatus {
      Passed
      Failed
    } derive(Eq, 
    Debug    )

    suitable for CI logs or a command-line health check.

    DiagnosticStatus::label

    source
    fn DiagnosticStatus::label(self : DiagnosticStatus) -> String

    DraftError

    source
    pub enum DraftError {
      EmptyProposal
      TokenOutOfRange(Int)
      InvalidDistribution(Int)
      LengthMismatch
    } derive(Eq, 
    Debug    )

    and real inference backends.

    DraftProposal

    source
    pub struct DraftProposal {
      prefix : Array[Int]
      tokens : Array[DraftToken]
    }

    DraftProposal::full_path

    source
    fn DraftProposal::full_path(self : DraftProposal) -> Array[Int]

    DraftProposal::length

    source
    fn DraftProposal::length(self : DraftProposal) -> Int

    DraftProposal::proposed_tokens

    source
    fn DraftProposal::proposed_tokens(self : DraftProposal) -> Array[Int]

    DraftToken

    source
    pub struct DraftToken {
      token : Int
      distribution : Array[Double]
    }

    DraftTree

    source
    pub struct DraftTree {
      prefix : Array[Int]
      nodes : Array[TreeNode]
    }

    DraftTree::children

    source
    fn DraftTree::children(self : DraftTree, parent : Int?) -> Array[TreeNode]

    DraftTree::max_depth

    source
    fn DraftTree::max_depth(self : DraftTree) -> Int

    DraftTree::node

    source
    fn DraftTree::node(self : DraftTree, id : Int) -> TreeNode?

    DraftTree::node_count

    source
    fn DraftTree::node_count(self : DraftTree) -> Int

    DraftTree::path_to

    source
    fn DraftTree::path_to(self : DraftTree, id : Int) -> Array[Int]

    ExperimentError

    source
    pub enum ExperimentError {
      EmptyExperiment
      IncompatibleRecord(Int)
    } derive(Eq, 
    Debug    )

    reproducible algorithmic work measures.

    ExperimentPair

    source
    pub struct ExperimentPair {
      case_name : String
      baseline : ExperimentRecord
      speculative : ExperimentRecord
    }

    Imported records carry caller-declared provenance. For internally matched model execution prefer compare_tree_decoding, which runs both sides itself.

    ExperimentPair::accepted_per_draft_token

    source
    fn ExperimentPair::accepted_per_draft_token(self : ExperimentPair) -> Double

    ExperimentPair::logical_batch_reduction

    source
    fn ExperimentPair::logical_batch_reduction(self : ExperimentPair) -> Double

    ExperimentPair::new

    source
    fn ExperimentPair::new(case_name : String, baseline : ExperimentRecord, speculative : ExperimentRecord) -> Result[ExperimentPair, ExperimentError]

    ExperimentPair::render

    source
    fn ExperimentPair::render(self : ExperimentPair) -> String

    ExperimentRecord

    source
    pub struct ExperimentRecord {
      name : String
      metrics : DecodeMetrics
      generated_tokens : Int
      prompt : Array[Int]
      target_model_id : String
    }

    One named run under a fixed prompt and model fixture.

    ExperimentRecord::from_result

    source
    fn ExperimentRecord::from_result(name : String, target_model_id : String, prompt : Array[Int], result : SimulationResult) -> Result[ExperimentRecord, ExperimentError]

    ExperimentRecord::target_batches

    source
    fn ExperimentRecord::target_batches(self : ExperimentRecord) -> Int

    ExperimentRecord::tokens_per_call

    source
    fn ExperimentRecord::tokens_per_call(self : ExperimentRecord) -> Double

    ExperimentSummary

    source
    pub struct ExperimentSummary {
      cases : Int
      reduction : SampleSummary
      acceptance : SampleSummary
      speculative_tokens_per_call : SampleSummary
    }

    Aggregate view of paired target-call savings and speculative acceptance.

    ExperimentSummary::render_markdown

    source
    fn ExperimentSummary::render_markdown(self : ExperimentSummary) -> String

    KvSlots

    source
    pub struct KvSlots {
      slots : Array[SlotState]
      next_transaction : Int
    }

    KvSlots::capacity

    source
    fn KvSlots::capacity(self : KvSlots) -> Int

    KvSlots::commit

    source
    fn KvSlots::commit(self : KvSlots, transaction : Int, keep : Int) -> Result[Unit, SlotError]

    KvSlots::free_count

    source
    fn KvSlots::free_count(self : KvSlots) -> Int

    KvSlots::new

    source
    fn KvSlots::new(capacity : Int) -> KvSlots

    KvSlots::release_committed

    source
    fn KvSlots::release_committed(self : KvSlots, transaction : Int) -> Result[Unit, SlotError]

    merely reserved or unknown transaction is a lifecycle error.

    KvSlots::reserve

    source
    fn KvSlots::reserve(self : KvSlots, count : Int) -> Result[(Int, Array[Int]), SlotError]

    KvSlots::rollback

    source
    fn KvSlots::rollback(self : KvSlots, transaction : Int) -> Result[Unit, SlotError]

    KvSlots::state_at

    source
    fn KvSlots::state_at(self : KvSlots, index : Int) -> Result[SlotState, SlotError]

    Return the lifecycle state of one cache slot.

    ModelDraftTree

    source
    pub struct ModelDraftTree {
      tree : DraftTree
      provider_calls : Int
    }

    ModelTreeError

    source
    pub enum ModelTreeError {
      InvalidConfiguration
      ProviderFailure(String)
      BatchResultLengthMismatch
      InvalidLogits
      InvalidTree
      VocabularyMismatch
    } derive(Eq, 
    Debug    )

    Context-sensitive model boundary. A provider returns next-token logits for the supplied prefix. This reference backend makes sequential calls; it does not claim to implement GPU tree attention or a batched forward pass.

    NodeScore

    source
    pub struct NodeScore {
      node_id : Int
      distribution : Array[Double]
    }

    cannot accidentally misalign a sibling branch.

    NodeScore::new

    source
    fn NodeScore::new(node_id : Int, distribution : Array[Double]) -> NodeScore

    OnlineMoments

    source
    pub struct OnlineMoments {
      count : Int
      mean : Double
      squared_deviation_sum : Double
      minimum : Double
      maximum : Double
    }

    A one-pass accumulator using Welford's numerically stable variance update.

    OnlineMoments::add

    source
    fn OnlineMoments::add(self : OnlineMoments, value : Double) -> Unit

    Add one observed value without retaining raw experiment records.

    OnlineMoments::count

    source
    fn OnlineMoments::count(self : OnlineMoments) -> Int

    OnlineMoments::empty

    source

    OnlineMoments::maximum

    source
    fn OnlineMoments::maximum(self : OnlineMoments) -> Result[Double, StatisticsError]

    OnlineMoments::mean

    source
    fn OnlineMoments::mean(self : OnlineMoments) -> Result[Double, StatisticsError]

    OnlineMoments::minimum

    source
    fn OnlineMoments::minimum(self : OnlineMoments) -> Result[Double, StatisticsError]

    OnlineMoments::population_variance

    source
    fn OnlineMoments::population_variance(self : OnlineMoments) -> Result[Double, StatisticsError]

    Use the unbiased form below when the suite is a sample from production.

    OnlineMoments::sample_variance

    source
    fn OnlineMoments::sample_variance(self : OnlineMoments) -> Result[Double, StatisticsError]

    PolicyError

    source
    pub enum PolicyError {
      EmptyDepths
      InvalidWidth
      InvalidBudget
      ProbabilityFailure
    } derive(Eq, 
    Debug    )

    logits while the policy owns node budgets, widths, and stable tie-breaking.

    ProbabilityError

    source
    pub enum ProbabilityError {
      EmptyLogits
      InvalidTemperature
      InvalidProbability
      InvalidSample
    } derive(Eq, 
    Debug    )

    Numerically stable probability primitives for decoding algorithms.

    ReplayDecodeError

    source
    pub enum ReplayDecodeError {
      InvalidDepth(Int)
      DraftModelFailure(Int)
      TargetModelFailure(Int)
      ProbabilityFailure(Int)
      VerificationFailure
    } derive(Eq, 
    Debug    )

    that a real draft/target inference integration must make.

    ReplayError

    source
    pub enum ReplayError {
      EmptyReplay
      EmptyVocabulary(Int)
      VocabularyMismatch(Int)
      ContextMismatch(Int)
      Exhausted(Int)
    } derive(Eq, 
    Debug    )

    protocol.

    ReplayModel

    source
    pub struct ReplayModel {
      steps : Array[ReplayStep]
      vocabulary_size : Int
      cursor : Int
    }

    Stateful cursor over deterministic model responses.

    ReplayModel::describe_next

    source
    fn ReplayModel::describe_next(self : ReplayModel) -> String

    full logits are intentionally omitted because they can be very large.

    ReplayModel::new

    source
    fn ReplayModel::new(steps : Array[ReplayStep]) -> Result[ReplayModel, ReplayError]

    silently compare incompatible draft and target distributions later.

    ReplayModel::next_logits

    source
    fn ReplayModel::next_logits(self : ReplayModel, context : Array[Int]) -> Result[Array[Double], ReplayError]

    same token prefix recorded by the fixture.

    ReplayModel::position

    source
    fn ReplayModel::position(self : ReplayModel) -> Int

    ReplayModel::remaining

    source
    fn ReplayModel::remaining(self : ReplayModel) -> Int

    ReplayModel::reset

    source
    fn ReplayModel::reset(self : ReplayModel) -> Unit

    fixture safely reusable across multiple strategy comparisons.

    ReplayModel::vocabulary_size

    source
    fn ReplayModel::vocabulary_size(self : ReplayModel) -> Int

    ReplayRoundResult

    source
    pub struct ReplayRoundResult {
      proposal : DraftProposal
      verification : VerifyResult
    }

    black box.

    ReplayRoundResult::proposal

    source

    ReplayRoundResult::verification

    source

    ReplayStep

    source
    pub struct ReplayStep {
      expected_context : Array[Int]
      logits : Array[Double]
    }

    One response emitted by a replayable logits provider.

    ReplayStep::context

    source
    fn ReplayStep::context(self : ReplayStep) -> Array[Int]

    ReplayStep::logits

    source
    fn ReplayStep::logits(self : ReplayStep) -> Array[Double]

    ReplayStep::new

    source
    fn ReplayStep::new(expected_context : Array[Int], logits : Array[Double]) -> ReplayStep

    RngError

    source
    pub enum RngError {
      InvalidSeed(Int)
      InvalidBound(Int)
    } derive(Eq, 
    Debug    )

    portable generator whose state can be recorded in a benchmark report.

    SampleSummary

    source
    pub struct SampleSummary {
      count : Int
      minimum : Double
      maximum : Double
      mean : Double
      population_variance : Double
      median : Double
      p90 : Double
    }

    A compact immutable summary convenient for serializing experiment results.

    SampleSummary::render

    source
    fn SampleSummary::render(self : SampleSummary, label : String) -> String

    SamplingConfig

    source
    pub struct SamplingConfig {
      temperature : Double
      top_k : Int
      top_p : Double
      min_p : Double
    }

    top_p = 1.0 and min_p = 0.0 mean their respective filters are off.

    SamplingConfig::default

    source

    Default is pure temperature sampling with no vocabulary truncation.

    SamplingConfig::new

    source
    fn SamplingConfig::new(temperature? : Double, top_k? : Int, top_p? : Double, min_p? : Double) -> Result[SamplingConfig, SamplingError]

    SamplingConfig::temperature

    source
    fn SamplingConfig::temperature(self : SamplingConfig) -> Double

    SamplingConfig::top_k

    source
    fn SamplingConfig::top_k(self : SamplingConfig) -> Int

    SamplingError

    source
    pub enum SamplingError {
      InvalidTopK(Int)
      InvalidTopP(Double)
      InvalidMinP(Double)
      InvalidDistribution
      ProbabilityFailure
      InvalidTemperature
    } derive(Eq, 
    Debug    )

    explicit, reproducible, and independently testable.

    ScoredTree

    source
    pub struct ScoredTree {
      distributions : Array[Array[Double]]
      provider_calls : Int
    }

    SessionError

    source
    pub enum SessionError {
      BudgetFailure
      CacheFailure
      AlreadyFinished
      InvalidRound
      RoundPending
    } derive(Eq, 
    Debug    )

    a ReplayRoundResult or adapt their own inference backend to that shape.

    SessionSnapshot

    source
    pub struct SessionSnapshot {
      generated : Array[Int]
      metrics : DecodeMetrics
      stop_reason : StopReason
    }

    session internals to logging or UI code.

    SessionSnapshot::generated

    source
    fn SessionSnapshot::generated(self : SessionSnapshot) -> Array[Int]

    SessionSnapshot::stop_reason

    source
    fn SessionSnapshot::stop_reason(self : SessionSnapshot) -> StopReason

    SimulationError

    source
    pub enum SimulationError {
      EmptySchedule
      VocabularyMismatch
      ProbabilityError
      VerificationError
    } derive(Eq, 
    Debug    )

    CI benchmarks, and reproducible documentation examples.

    SimulationResult

    source
    pub struct SimulationResult {
      generated : Array[Int]
      metrics : DecodeMetrics
      rounds : Int
    }

    SimulationRound

    source
    pub struct SimulationRound {
      draft_logits : Array[Array[Double]]
      target_logits : Array[Array[Double]]
      draft_uniforms : Array[Double]
      accept_uniforms : Array[Double]
      fallback_uniforms : Array[Double]
    }

    SlotError

    source
    pub enum SlotError {
      InvalidSlot(Int)
      NotFree(Int)
      InvalidTransaction(Int)
    } derive(Eq, 
    Debug    )

    SlotState

    source
    pub enum SlotState {
      Free
      Reserved(Int)
      Committed(Int)
    } derive(Eq, 
    Debug    )

    values: it records the slot lifecycle an inference adapter must follow.

    StatisticsError

    source
    pub enum StatisticsError {
      EmptySample
      InvalidQuantile(Double)
    } derive(Eq, 
    Debug    )

    short request runs rather than millions of observations.

    StopReason

    source
    pub enum StopReason {
      Running
      Requested
      TargetCallBudget
      OutputTokenBudget
      DraftTokenBudget
      CacheCapacity
    } derive(Eq, 
    Debug    )

    finish requested by its caller from an enforced resource budget stop.

    StopReason::label

    source
    fn StopReason::label(self : StopReason) -> String

    StopSequence

    source
    pub struct StopSequence {
      tokens : Array[Int]
    }

    can represent EOS, a delimiter, or a multi-token marker with the same API.

    StopSequence::length

    source
    fn StopSequence::length(self : StopSequence) -> Int

    StopSequence::matches_suffix

    source
    fn StopSequence::matches_suffix(self : StopSequence, output : Array[Int]) -> Bool

    True only if the current output ends in this exact sequence.

    StopSequence::new

    source
    fn StopSequence::new(tokens : Array[Int]) -> Result[StopSequence, ConstraintError]

    TokenMask

    source
    pub struct TokenMask {
      allowed : Array[Bool]
    }

    A vocabulary-sized boolean mask. true means the token may be emitted.

    TokenMask::allow_all

    source
    fn TokenMask::allow_all(vocabulary_size : Int) -> Result[TokenMask, ConstraintError]

    TokenMask::allows

    source
    fn TokenMask::allows(self : TokenMask, token : Int) -> Bool

    TokenMask::apply

    source
    fn TokenMask::apply(self : TokenMask, distribution : Array[Double]) -> Result[Array[Double], ConstraintError]

    Mask a probability distribution and renormalize the retained mass.

    TokenMask::except

    source
    fn TokenMask::except(vocabulary_size : Int, token_ids : Array[Int]) -> Result[TokenMask, ConstraintError]

    Start with every token allowed and selectively prohibit listed ids.

    TokenMask::intersect

    source
    fn TokenMask::intersect(self : TokenMask, other : TokenMask) -> Result[TokenMask, ConstraintError]

    both active for one decoding step.

    TokenMask::only

    source
    fn TokenMask::only(vocabulary_size : Int, token_ids : Array[Int]) -> Result[TokenMask, ConstraintError]

    Start with every token forbidden and selectively permit listed ids.

    TokenMask::vocabulary_size

    source
    fn TokenMask::vocabulary_size(self : TokenMask) -> Int

    TreeBatchError

    source
    pub enum TreeBatchError {
      InvalidTree
      MissingNode(Int)
      DuplicateScore(Int)
      MissingScore(Int)
      InvalidScore(Int)
    } derive(Eq, 
    Debug    )

    tensor runtimes and model-specific cache formats.

    TreeBatchPlan

    source
    pub struct TreeBatchPlan {
      layers : Array[Array[TreeQuery]]
      query_count : Int
    }

    fully flattened batch and cache-aware level-by-level execution.

    TreeBatchPlan::flatten

    source
    fn TreeBatchPlan::flatten(self : TreeBatchPlan) -> Array[TreeQuery]

    TreeBatchPlan::layer

    source
    fn TreeBatchPlan::layer(self : TreeBatchPlan, depth : Int) -> Array[TreeQuery]

    TreeBatchPlan::layer_count

    source
    fn TreeBatchPlan::layer_count(self : TreeBatchPlan) -> Int

    TreeBatchPlan::query_count

    source
    fn TreeBatchPlan::query_count(self : TreeBatchPlan) -> Int

    TreeBatchPlan::render

    source
    fn TreeBatchPlan::render(self : TreeBatchPlan) -> String

    Render a stable, model-free execution plan useful in CI fixture diffs.

    TreeError

    source
    pub enum TreeError {
      EmptyTree
      InvalidRoot
      InvalidParent(Int, Int)
      DuplicateNodeId(Int)
      InvalidNodeDistribution(Int)
      InvalidNodeToken(Int)
      BudgetExceeded(Int, Int)
    } derive(Eq, 
    Debug    )

    traversal and simple validation.

    TreeEvaluateError

    source
    pub enum TreeEvaluateError {
      TreeInvalid
      TargetLengthMismatch
      RandomLengthMismatch
      InvalidTarget(Int)
      InvalidRandom(Int)
      InconsistentSiblings(Int)
    } derive(Eq, 
    Debug    )

    Exact residual verification for a deterministic candidate tree. Each sibling is a point-mass proposal: accept with its current residual probability; after rejection remove that token and renormalize. Raw draft probabilities guide candidate construction, not this acceptance law.

    TreeEvaluation

    source
    pub struct TreeEvaluation {
      accepted_nodes : Array[Int]
      rejected_nodes : Array[Int]
      skipped_nodes : Array[Int]
      committed_path : Array[Int]
      emitted : Array[Int]
    }

    TreeEvaluation::evaluated_count

    source
    fn TreeEvaluation::evaluated_count(self : TreeEvaluation) -> Int

    TreeExperiment

    source
    pub struct TreeExperiment {
      prompt : Array[Int]
      config : TreeExperimentConfig
      baseline : DecodingRun
      speculative : DecodingRun
    }

    TreeExperiment::render

    source
    fn TreeExperiment::render(self : TreeExperiment) -> String

    TreeExperimentConfig

    source
    pub struct TreeExperimentConfig {
      width : Int
      depth : Int
      node_budget : Int
      output_tokens : Int
      seed : Int
    }

    TreeExperimentConfig::new

    source
    fn TreeExperimentConfig::new(width : Int, depth : Int, node_budget : Int, output_tokens : Int, seed : Int) -> Result[TreeExperimentConfig, TreeExperimentError]

    TreeExperimentError

    source
    pub enum TreeExperimentError {
      InvalidConfiguration
      ModelFailure(ModelTreeError)
      TargetFailure(String)
      InvalidProbabilities
      VerificationFailure(TreeEvaluateError)
      InvalidRandomSeed
    } derive(Eq, 
    Debug    )

    Both sides of a comparison use the same pure target provider, prompt, and output budget. Callbacks are evaluated sequentially; measured query counts are separate from hypothetical parallel target batches. No latency claim can be inferred from a reduction in batches alone.

    TreeNode

    source
    pub struct TreeNode {
      id : Int
      parent : Int?
      token : Int
      distribution : Array[Double]
      depth : Int
    }

    TreePolicy

    source
    pub struct TreePolicy {
      width : Int
      max_nodes : Int
    }

    TreePolicy::build

    source
    fn TreePolicy::build(self : TreePolicy, prefix : Array[Int], depth_logits : Array[Array[Double]]) -> Result[DraftTree, PolicyError]

    TreePolicy::build_from_batch_model

    source
    fn TreePolicy::build_from_batch_model(self : TreePolicy, prefix : Array[Int], depth : Int, model : (Array[Array[Int]]) -> Result[Array[Array[Double]], String]) -> Result[ModelDraftTree, ModelTreeError]

    Build a draft tree with one batch-provider call per tree level. Candidate prefixes for the next level are known only after this level is scored, so a single all-level draft request would be invalid.

    TreePolicy::build_from_model

    source
    fn TreePolicy::build_from_model(self : TreePolicy, prefix : Array[Int], depth : Int, model : (Array[Int]) -> Result[Array[Double], String]) -> Result[ModelDraftTree, ModelTreeError]

    Query each parent prefix separately, including distinct siblings' paths. Width is capped by the vocabulary and the global node budget.

    TreePolicy::new

    source
    fn TreePolicy::new(width : Int, max_nodes : Int) -> Result[TreePolicy, PolicyError]

    TreeQuery

    source
    pub struct TreeQuery {
      node_id : Int
      parent_id : Int?
      depth : Int
      context : Array[Int]
    }

    Context excludes the candidate token identified by node_id. Returned logits predict that token, not its successor.

    TreeQuery::context

    source
    fn TreeQuery::context(self : TreeQuery) -> Array[Int]

    TreeQuery::node_id

    source
    fn TreeQuery::node_id(self : TreeQuery) -> Int

    VerifyError

    source
    pub enum VerifyError {
      TargetLengthMismatch
      RandomLengthMismatch
      InvalidTargetDistribution(Int)
      ProbabilityFailure(Int)
      InvalidProposal
    } derive(Eq, 
    Debug    )

    distribution when draft and target distributions are valid.

    VerifyResult

    source
    pub struct VerifyResult {
      emitted : Array[Int]
      accepted_count : Int
      rejected_at : Int?
    }

    VerifyResult::accepted_all

    source
    fn VerifyResult::accepted_all(self : VerifyResult, proposal : DraftProposal) -> Bool

    WorkloadConfig

    source
    pub struct WorkloadConfig {
      rounds : Int
      proposal_depth : Int
      vocabulary_size : Int
      agreement : Double
    }

    residual sampling and rejection handling.

    WorkloadConfig::demo

    source

    A small default workload appropriate for the command-line example.

    WorkloadConfig::new

    source
    fn WorkloadConfig::new(rounds : Int, proposal_depth : Int, vocabulary_size : Int, agreement : Double) -> Result[WorkloadConfig, WorkloadError]

    WorkloadConfig::proposal_depth

    source
    fn WorkloadConfig::proposal_depth(self : WorkloadConfig) -> Int

    WorkloadConfig::rounds

    source
    fn WorkloadConfig::rounds(self : WorkloadConfig) -> Int

    WorkloadConfig::vocabulary_size

    source
    fn WorkloadConfig::vocabulary_size(self : WorkloadConfig) -> Int

    WorkloadError

    source
    pub enum WorkloadError {
      InvalidRounds(Int)
      InvalidDepth(Int)
      InvalidVocabulary(Int)
      InvalidAgreement(Double)
      RandomFailure
    } derive(Eq, 
    Debug    )

    vocabulary shape, and scheduling paths in CI without downloading weights.

    accept_probability

    source
    fn accept_probability(target : Double, draft : Double) -> Result[Double, ProbabilityError]

    align_node_scores

    source
    fn align_node_scores(tree : DraftTree, scores : Array[NodeScore]) -> Result[Array[Array[Double]], TreeBatchError]

    omitted, and malformed scores at the boundary with model code.

    append_replay_round

    source
    fn append_replay_round(generated : Array[Int], metrics : DecodeMetrics, result : ReplayRoundResult) -> Unit

    This helper makes it hard for benchmark callers to forget metric updates.

    argmax

    source
    fn argmax(values : Array[Double]) -> Result[Int, SimulationError]

    benchmark_adaptive_batch_decoding

    source
    fn benchmark_adaptive_batch_decoding(model_id : String, prompt : Array[Int], draft : (Array[Array[Int]]) -> Result[Array[Array[Double]], String], target : (Array[Array[Int]]) -> Result[Array[Array[Double]], String], policy : AdaptiveTreePolicy, output_tokens : Int, seed : Int) -> Result[AdaptiveBenchmark, BenchmarkError]

    Run a matched baseline and adaptive-tree experiment through the same pure target batch provider. The baseline submits exactly one context per request; adaptive scoring may submit many contexts per request. Providers must return one logits row for each input context, in the same order.

    cli_help

    source
    fn cli_help() -> String

    compare_tree_decoding

    source
    fn compare_tree_decoding(prefix : Array[Int], draft : (Array[Int]) -> Result[Array[Double], String], target : (Array[Int]) -> Result[Array[Double], String], config : TreeExperimentConfig) -> Result[TreeExperiment, TreeExperimentError]

    A single entry point fixes the target, prompt, and output length on both sides. Different random-number consumption means sampled text need not be identical; distribution-preservation tests are separate from work counts.

    context_for_node

    source
    fn context_for_node(tree : DraftTree, node_id : Int) -> Result[Array[Int], TreeBatchError]

    well as the tree path.

    decode_adaptive_tree_from_batch_models

    source
    fn decode_adaptive_tree_from_batch_models(prefix : Array[Int], draft : (Array[Array[Int]]) -> Result[Array[Array[Double]], String], target : (Array[Array[Int]]) -> Result[Array[Array[Double]], String], policy : AdaptiveTreePolicy, output_tokens : Int, seed : Int) -> Result[AdaptiveDecodingRun, TreeExperimentError]

    Decode through batch callbacks using the entropy-adaptive tree policy. Width is selected independently for each draft parent row. After a round, the next depth moves toward the observed accepted-path fraction; rejected siblings are deliberately not counted as accepted tokens.

    decode_autoregressive

    source
    fn decode_autoregressive(prefix : Array[Int], target_logits : Array[Array[Double]], uniforms : Array[Double]) -> Result[SimulationResult, BaselineError]

    decode_baseline_from_model

    source
    fn decode_baseline_from_model(prefix : Array[Int], target : (Array[Int]) -> Result[Array[Double], String], config : TreeExperimentConfig) -> Result[DecodingRun, TreeExperimentError]

    Run an ordinary autoregressive baseline against the same model interface.

    decode_tree_from_batch_models

    source
    fn decode_tree_from_batch_models(prefix : Array[Int], draft : (Array[Array[Int]]) -> Result[Array[Array[Double]], String], target : (Array[Array[Int]]) -> Result[Array[Array[Double]], String], config : TreeExperimentConfig) -> Result[DecodingRun, TreeExperimentError]

    End-to-end tree decoding through batch providers. Draft expansion submits one frontier batch per depth; target scoring submits one batch per round. target_queries and draft_queries count provider invocations, not token rows, so callers can compare transport work with the sequential adapter.

    decode_tree_from_model

    source
    fn decode_tree_from_model(prefix : Array[Int], draft : (Array[Int]) -> Result[Array[Double], String], target : (Array[Int]) -> Result[Array[Double], String], config : TreeExperimentConfig) -> Result[DecodingRun, TreeExperimentError]

    The model callback must be a deterministic function of its input tokens. Randomness for token sampling is owned by the caller, not by the model.

    demo_schedule

    source
    fn demo_schedule() -> Array[SimulationRound]

    describe_workload

    source
    fn describe_workload(config : WorkloadConfig, seed : Int) -> String

    potentially large logits. It is suitable for a README or CI log.

    diagnose_batch_provider

    source
    fn diagnose_batch_provider(contexts : Array[Array[Int]], expected_vocabulary : Int, model : (Array[Array[Int]]) -> Result[Array[Array[Double]], String]) -> DiagnosticReport

    Call a batch provider twice on fixed token contexts and report whether it satisfies TreeSpec's integration contract. This preflight performs no decoding; use it before a real benchmark to catch row reordering, a wrong vocabulary, invalid logits or a stateful provider.

    diagnose_proposal

    source
    fn diagnose_proposal(proposal : DraftProposal) -> DiagnosticReport

    Check a single-path proposal without attempting target-model execution.

    diagnose_tree

    source
    fn diagnose_tree(tree : DraftTree) -> DiagnosticReport

    a target-query context by the batch planner.

    diagnose_workload

    source
    fn diagnose_workload(schedule : Array[SimulationRound]) -> DiagnosticReport

    acceptance outcome.

    distribution_for_sampling

    source
    fn distribution_for_sampling(logits : Array[Double], config : SamplingConfig) -> Result[Array[Double], SamplingError]

    configuration portable across a draft worker and a target worker.

    evaluate_tree

    source
    fn evaluate_tree(tree : DraftTree, target_distributions : Array[Array[Double]], uniforms : Array[Double], fallback_uniforms : Array[Double]) -> Result[TreeEvaluation, TreeEvaluateError]

    Targets in node order score the prefix BEFORE each node. Siblings must share one target row. Acceptance draws are indexed by node; fallback draws by depth minus one. Draws must be independent of candidate construction and one another. At a leaf this round ends; the next round produces more tokens.

    filter_min_p

    source
    fn filter_min_p(distribution : Array[Double], threshold : Double) -> Result[Array[Double], SamplingError]

    every retained probability is at least threshold * max_probability.

    filter_top_k

    source
    fn filter_top_k(distribution : Array[Double], limit : Int) -> Result[Array[Double], SamplingError]

    Keep the limit most likely tokens. limit = 0 keeps every token.

    filter_top_p

    source
    fn filter_top_p(distribution : Array[Double], threshold : Double) -> Result[Array[Double], SamplingError]

    cumulative original probability reaches threshold.

    generate_workload

    source
    fn generate_workload(config : WorkloadConfig, seed : Int) -> Result[Array[SimulationRound], WorkloadError]

    verifier, so recording the seed is sufficient for regression replay.

    log_sum_exp

    source
    fn log_sum_exp(values : Array[Double]) -> Result[Double, ProbabilityError]

    make_proposal

    source
    fn make_proposal(prefix : Array[Int], tokens : Array[Int], distributions : Array[Array[Double]]) -> Result[DraftProposal, DraftError]

    make_tree

    source
    fn make_tree(prefix : Array[Int], nodes : Array[TreeNode]) -> Result[DraftTree, TreeError]

    normalize_weights

    source
    fn normalize_weights(weights : Array[Double]) -> Result[Array[Double], SamplingError]

    impossible token.

    normalized_entropy

    source
    fn normalized_entropy(logits : Array[Double]) -> Result[Double, AdaptiveTreeError]

    by log(1).

    percentile

    source
    fn percentile(values : Array[Double], quantile : Double) -> Result[Double, StatisticsError]

    For example, 0.5 is the median, and the endpoints are min and max.

    plan_tree_batch

    source
    fn plan_tree_batch(tree : DraftTree) -> Result[TreeBatchPlan, TreeBatchError]

    ancestors and malformed depths from leaking into a model backend.

    probability_at

    source
    fn probability_at(distribution : Array[Double], index : Int) -> Double

    proposal_from_replay

    source
    fn proposal_from_replay(draft : ReplayModel, prefix : Array[Int], depth : Int, draft_uniforms : Array[Double]) -> Result[DraftProposal, ReplayDecodeError]

    Sample every draft token from the distribution used for verification. The supplied draws must be independent of acceptance and fallback draws.

    prune_tree

    source
    fn prune_tree(tree : DraftTree, max_depth : Int, max_nodes : Int) -> DraftTree

    ranked_indices

    source
    fn ranked_indices(distribution : Array[Double]) -> Result[Array[Int], SamplingError]

    their lower token id, which is useful when test fixtures need stability.

    render_metrics

    source
    fn render_metrics(metrics : DecodeMetrics) -> String

    that need a text artifact without pulling in JSON or terminal libraries.

    render_simulation

    source
    fn render_simulation(result : SimulationResult) -> String

    render_tree_evaluation

    source
    fn render_tree_evaluation(value : TreeEvaluation) -> String

    residual_distribution

    source
    fn residual_distribution(target : Array[Double], draft : Array[Double]) -> Result[Array[Double], ProbabilityError]

    run_benchmark_demo

    source
    fn run_benchmark_demo() -> Result[String, CliError]

    the same benchmark API users call with a real batch inference callback.

    run_cli_command

    source
    fn run_cli_command(command : String) -> Result[String, CliError]

    return a typed error so embedding applications can show their own help.

    run_demo

    source
    fn run_demo() -> Result[String, SimulationError]

    run_provider_preflight_demo

    source
    fn run_provider_preflight_demo() -> Result[String, CliError]

    satisfy before it is used for decoding or benchmark collection.

    run_replay_round

    source
    fn run_replay_round(draft : ReplayModel, target : ReplayModel, prefix : Array[Int], depth : Int, accept_uniforms : Array[Double], fallback_uniforms : Array[Double], draft_uniforms : Array[Double]) -> Result[ReplayRoundResult, ReplayDecodeError]

    Execute one replay-backed speculative round.

    run_round

    source
    fn run_round(prefix : Array[Int], round : SimulationRound) -> Result[VerifyResult, SimulationError]

    run_tree_batch_experiment_demo

    source
    fn run_tree_batch_experiment_demo() -> Result[String, TreeExperimentError]

    Runnable batch-backend companion to run_tree_experiment_demo. It uses the same deterministic logits but exposes the provider-level request reduction that an embedding inference runtime can turn into parallel work.

    run_tree_experiment_demo

    source
    fn run_tree_experiment_demo() -> Result[String, TreeExperimentError]

    Small context-sensitive fixtures; no trained weights or external data.

    run_tree_plan_demo

    source
    fn run_tree_plan_demo() -> Result[String, CliError]

    which keeps terminal output short and deterministic.

    run_workload_demo

    source
    fn run_workload_demo() -> Result[String, CliError]

    sample_categorical

    source
    fn sample_categorical(distribution : Array[Double], unit_interval : Double) -> Result[Int, ProbabilityError]

    sample_constrained_logits

    source
    fn sample_constrained_logits(logits : Array[Double], config : SamplingConfig, mask : TokenMask, unit_interval : Double) -> Result[Int, ConstraintError]

    vocabulary mask. Randomness is supplied by the caller for replayability.

    sample_logits

    source
    fn sample_logits(logits : Array[Double], config : SamplingConfig, unit_interval : Double) -> Result[Int, SamplingError]

    be replayed bit-for-bit.

    score_tree_from_batch_model

    source
    fn score_tree_from_batch_model(tree : DraftTree, model : (Array[Array[Int]]) -> Result[Array[Array[Double]], String]) -> Result[ScoredTree, ModelTreeError]

    Score every reachable parent context in one real batch-provider invocation. The callback receives full token prefixes, so it can use a remote endpoint, a tensor runtime, or a cache-aware native backend without changing the verifier. Returned rows must retain input order.

    score_tree_from_model

    source
    fn score_tree_from_model(tree : DraftTree, model : (Array[Int]) -> Result[Array[Double], String]) -> Result[ScoredTree, ModelTreeError]

    Evaluate each unique parent context once, then share its distribution among siblings. provider_calls counts real callback invocations.

    score_tree_layers_from_batch_model

    source
    fn score_tree_layers_from_batch_model(tree : DraftTree, model : (Array[Array[Int]]) -> Result[Array[Array[Double]], String]) -> Result[ScoredTree, ModelTreeError]

    Score one tree depth at a time. This is useful to backends that retain KV cache state across levels or impose a maximum batch size. Unlike the sequential adapter, siblings still share one batch row and provider_calls is the number of actual batch invocations.

    simulate

    source
    fn simulate(prefix : Array[Int], schedule : Array[SimulationRound]) -> Result[SimulationResult, SimulationError]

    softmax

    source
    fn softmax(logits : Array[Double], temperature? : Double) -> Result[Array[Double], ProbabilityError]

    sorted_copy

    source
    fn sorted_copy(values : Array[Double]) -> Array[Double]

    trace in a benchmark report.

    summarize_experiment

    source
    fn summarize_experiment(pairs : Array[ExperimentPair]) -> Result[ExperimentSummary, ExperimentError]

    summarize_sample

    source
    fn summarize_sample(values : Array[Double]) -> Result[SampleSummary, StatisticsError]

    top_tokens

    source
    fn top_tokens(logits : Array[Double], width : Int) -> Result[Array[(Int, Array[Double])], PolicyError]

    trace_single_path

    source
    fn trace_single_path(round : Int, proposal : DraftProposal, target_distributions : Array[Array[Double]], accept_uniforms : Array[Double]) -> Result[DecodeTrace, VerifyError]

    validate_proposal

    source
    fn validate_proposal(proposal : DraftProposal) -> Result[Unit, DraftError]

    validate_tree

    source
    fn validate_tree(tree : DraftTree, node_budget? : Int) -> Result[Unit, TreeError]

    verify_from_replay

    source
    fn verify_from_replay(target : ReplayModel, proposal : DraftProposal, accept_uniforms : Array[Double], fallback_uniforms : Array[Double]) -> Result[VerifyResult, ReplayDecodeError]

    one forward pass.

    verify_proposal

    source
    fn verify_proposal(proposal : DraftProposal, target_distributions : Array[Array[Double]], accept_uniforms : Array[Double], fallback_uniforms : Array[Double]) -> Result[VerifyResult, VerifyError]

    Source Files

    Powered by MoonBit

    Site sourceReport issuePackagesBuild queueSkillsStatistics

    © 2026 mooncakes.io