soehi/moonlm/cli - mooncakes.io

soehi/moonlm/cli does not have a README file

AgentCliOptions

type AgentCliOptions

Options for one agent run.

GenOptions

source

pub(all) struct GenOptions {
  max_tokens : Int
  sampler : @soehi/moonlm/sampler.Sampler
Sampler
  seed : UInt64
  use_chat_template : Bool
  dump_tokens : Bool
} derive(@moonbitlang/core/debug.Debug
Debug)

Options shared by generate, chat and bench.

MlFile

source

#external
pub type MlFile

Clojure to libc's FILE *. #external keeps the C pointer opaque so the MoonBit side never dereferences it.

apply_chat_template

source

fn apply_chat_template(user : String, system : String) -> String

Wrap a user message in SmolLM2's ChatML template.

The real template is stored in the GGUF as Jinja (tokenizer.chat_template) and this build does not interpret Jinja; the ChatML form below is what that template renders for a single user turn. --raw bypasses it for raw completion prompts and for comparing against a non-instruct reference.

dump_logits

source

fn dump_logits(path : String, ids : Array[Int], top : Int) -> Int

Dump the top-k logits after evaluating a prompt given as token ids.

Exists for the cross-implementation check: comparing sampled tokens cannot tell a real bug from a near-tie broken by floating-point drift, so this reports the logit values at each position. A reference implementation's logprob for the same position is the other half of the comparison.

The prompt is taken as ids rather than text on purpose — it removes the tokenizer from the experiment, so a mismatch can only come from the forward pass.

exit_with

source

fn exit_with(code : Int) -> Unit

Terminate the process with code.

Declared here rather than used directly because exit is a native-only libc symbol; routing it through this file keeps the non-native entry point free of platform assumptions.

print_raw

source

fn print_raw(s : String) -> Unit

Write s to stdout without a trailing newline, then flush.

Core only offers println, which is line-buffered; streaming a token at a time needs the bytes to reach the terminal immediately, so this goes through libc and flushes explicitly.

profile_decode

source

fn profile_decode(path : String, iters : Int) -> Int

Time each stage of a decode step over iters iterations on a real model.

Reported per-stage so a bottleneck is attributable: the LM head and the FFN are the two stages that dominate a 135M model, and they need different fixes.

read_file

source

fn read_file(path : String) -> Result[Bytes, String]

Read an entire file into memory.

Returns Err with a message rather than panicking when the file is missing or unreadable, so a mistyped path produces a diagnostic and a non-zero exit status instead of a traceback.

Peak allocation is one copy of the file: the size is checked first (which costs no memory), and the bytes are then allocated once on the C side.

read_line

source

fn read_line() -> String?

Read one line from stdin, without the trailing newline.

Returns None at end of input, which is how interactive chat knows to stop. A fixed-size buffer is used because lines in a chat session are short; a longer line is returned truncated rather than dropped.