🥮 mooncakes.io
    Sign in

    gguf

    Read-only GGUF container parsing and ggml block dequantization, in pure MoonBit

    gguf
    quantization
    binary
    Download zip
    Author
    Version
    0.1.0
    License
    Apache-2.0
    Repository
    Last updated
    10 hours ago
    Downloads
    3

    #soehi/gguf

    Read-only GGUF container parsing and ggml block dequantization, in pure MoonBit. No FFI, no third-party dependencies; builds and tests on native and wasm-gc.

    This is the container + numeric layer of the MoonLM project, split out so it can be reused on its own: anything that needs to open a GGUF and get floats out of it can depend on this module without also taking the inference engine, the tokenizer, or the agent runtime.

    #Packages

    ImportWhat it holds
    soehi/ggufThe GGUF container: header, metadata key/values, tensor directory.
    soehi/gguf/tensorThe ggml type table (all 43 ids) and block dequantization kernels.

    #Scope

    Read-only parsing. Given a Bytes holding a GGUF file, you get the version, the alignment, every metadata key/value, and the tensor directory (name, dims, type, offset, element count, byte size). No file I/O is performed here — the buffer comes from the caller, which is what lets this package work unchanged on a browser, where there is no filesystem.

    Dequantization to f32. Six block types are decoded: f32, f16, q4_0, q5_0, q8_0, q4_K, q6_K. Every kernel is a line-by-line transcription of the matching dequantize_row_* in llama.cpp's ggml/src/ggml-quants.c, with byte offsets taken from the block_* structs in ggml/src/ggml-common.h. The expected values in the golden tests were produced by an independent numpy reference and asserted to round-trip, so agreement is evidence rather than tautology.

    Bounded failure. A malformed container returns a named error with an offset, never a trap. A short or misaligned payload is rejected rather than truncated. A type this build cannot decode is reported by name, with its block size still computed correctly so a directory listing stays usable.

    #Non-goals

    These are deliberately absent, and adding any of them would be a feature request rather than a bug fix:

    • Writing GGUF. This module does not produce files. Serializing a container needs a policy for alignment and metadata ordering that the reader cannot infer.
    • Quantization. Only dequantization. Producing q4_0/q5_0 blocks means choosing a quantization objective, which is a different problem with a different set of tradeoffs.
    • Inference. No matvec, no attention, no sampling. The consumer supplies the arithmetic; this module stops at "give me the numbers for this block".
    • Tokenizer, model config, or engine glue. Those live in the MoonLM module and are what depend on this one, not the other way round.

    #Verification

    • Golden vectors per block type, at 1e-6 relative tolerance, on both targets.
    • Cross-checked against llama.cpp's own Python gguf reader on real models (header, all metadata, the complete tensor directory, and decoded values).
    • Malformed-input coverage: truncation at many offsets, byte-level mutation, and one hand-aimed case per parser guard.

    See docs/gguf-notes.md for the byte layouts, the type table, and the implementation notes.

    #Maintenance

    Maintained by soehi (the MoonLM author) until this module has an independent issue tracker.

    • Semver. Breaking changes bump the major version. Within a major version, new dequantizable types and new metadata helpers are additive and land in a minor; bug fixes and error-message improvements land in a patch.
    • Issues. Please file them in the moonlm repository until this module gets its own tracker. Prefix the title with gguf: so it can be found.
    • License. Apache-2.0, same as MoonLM.

    GgufError

    source
    pub(all) suberror GgufError {
      Truncated(offset~ : Int64, needed~ : Int64, remaining~ : Int64)
      BadMagic(found~ : UInt)
      UnsupportedVersion(version~ : UInt, supported_max~ : UInt)
      UnknownValueType(offset~ : Int64, type_id~ : UInt)
      BadAlignment(value~ : UInt)
      AlignmentWrongType(type_id~ : UInt)
      ArrayNestingTooDeep(offset~ : Int64, depth~ : Int)
      IllegalArrayElementType(offset~ : Int64, type_id~ : UInt)
      CountOverflow(offset~ : Int64, count~ : UInt64, elem_size~ : Int64)
      CountTooLarge(offset~ : Int64, count~ : UInt64)
      TensorCountImplausible(count~ : UInt64, remaining~ : Int64)
      TooManyDims(offset~ : Int64, n_dims~ : UInt, max_dims~ : Int)
      DataSectionOutOfRange(data_start~ : Int64, buffer_len~ : Int64)
      TensorOutOfRange(name~ : String, offset~ : Int64, nbytes~ : Int64, data_len~ : Int64)
      TensorNotSupported(name~ : String, type_id~ : UInt, type_name~ : String)
      DequantFailed(name~ : String, reason~ : String)
    } derive(Eq, 
    Debug    )

    Error type for GGUF container parsing.
    impl Show for GgufError

    GgufError::message

    source
    fn GgufError::message(self : GgufError) -> String

    Human-readable rendering of a GgufError, always naming the offset where the parse gave up.

    ByteCursor

    source
    pub(all) struct ByteCursor {
      buf : Bytes
      pos : Int64
    } derive(
    Debug    )

    A read position within buf.

    pos is an absolute byte offset and is always kept within [0, buf.length()]. Constructed with ByteCursor::new and moved only through the read_* methods, which advance it on success.

    ByteCursor::is_at_end

    source
    fn ByteCursor::is_at_end(self : ByteCursor) -> Bool

    Whether the cursor has reached the end of the buffer.

    ByteCursor::len

    source
    fn ByteCursor::len(self : ByteCursor) -> Int64

    Total buffer length, independent of the current position.

    ByteCursor::new

    source
    fn ByteCursor::new(buf : Bytes) -> ByteCursor

    Create a cursor positioned at the start of buf.

    ByteCursor::read_bytes

    source
    fn ByteCursor::read_bytes(self : ByteCursor, n : Int64) -> Result[Bytes, GgufError]

    Read n bytes and advance past them.

    ByteCursor::read_f32

    source
    fn ByteCursor::read_f32(self : ByteCursor) -> Result[Float, GgufError]

    Read a little-endian f32 as raw IEEE 754 bits.

    ByteCursor::read_f64

    source
    fn ByteCursor::read_f64(self : ByteCursor) -> Result[Double, GgufError]

    Read a little-endian f64.

    ByteCursor::read_i16

    source
    fn ByteCursor::read_i16(self : ByteCursor) -> Result[Int, GgufError]

    Read a little-endian signed i16, sign-extended.

    ByteCursor::read_i8

    source
    fn ByteCursor::read_i8(self : ByteCursor) -> Result[Int, GgufError]

    Read a little-endian signed i8, sign-extended.

    ByteCursor::read_string

    source
    fn ByteCursor::read_string(self : ByteCursor) -> Result[(String, Bytes), GgufError]

    Read a GGUF string: a u64 byte length followed by that many UTF-8 bytes.

    Invalid UTF-8 is not fatal — GGUF files in the wild occasionally carry non-UTF-8 chat templates — so the bytes are preserved and the caller sees (text, raw) where text is a lossy decode.

    ByteCursor::read_u16

    source
    fn ByteCursor::read_u16(self : ByteCursor) -> Result[Int, GgufError]

    Read a little-endian u16 as an Int in 0..65535.

    ByteCursor::read_u32

    source
    fn ByteCursor::read_u32(self : ByteCursor) -> Result[UInt, GgufError]

    Read a little-endian u32 as an unsigned bit pattern.

    ByteCursor::read_u64

    source
    fn ByteCursor::read_u64(self : ByteCursor) -> Result[UInt64, GgufError]

    Read a little-endian u64.

    ByteCursor::read_u8

    source
    fn ByteCursor::read_u8(self : ByteCursor) -> Result[Int, GgufError]

    Read an unsigned byte (0..255).

    ByteCursor::remaining

    source
    fn ByteCursor::remaining(self : ByteCursor) -> Int64

    Bytes not yet consumed.

    ByteCursor::seek

    source
    fn ByteCursor::seek(self : ByteCursor, pos : Int64) -> Result[Unit, GgufError]

    Position the cursor at an absolute offset, which must lie within the buffer.

    ByteCursor::skip

    source
    fn ByteCursor::skip(self : ByteCursor, n : Int64) -> Result[Unit, GgufError]

    Advance by n bytes after checking they exist.

    GgufFile

    source
    pub(all) struct GgufFile {
      version : UInt
      alignment : UInt
      tensors : Array[TensorInfo]
      metadata : Array[(String, MetaValue)]
      data_start : Int64
      buf : Bytes
    }

    A parsed GGUF container. Holds a reference to the caller's buffer so tensor payloads can be sliced on demand rather than copied at parse time.

    GgufFile::buffer

    source
    fn GgufFile::buffer(self : GgufFile) -> Bytes

    The whole buffer this file was parsed from.

    Exposed so a caller can hold one shared reference and address tensors by offset, instead of copying each tensor's payload out and holding a second copy of the weights.

    GgufFile::data_len

    source
    fn GgufFile::data_len(self : GgufFile) -> Int64

    Bytes available from data_start to the end of the buffer.

    This is the upper bound a tensor's offset + nbytes may reach; it can exceed the sum of all tensor sizes when the file has trailing padding or embedded alignment gaps.

    GgufFile::dequant_tensor

    source
    fn GgufFile::dequant_tensor(self : GgufFile, index : Int) -> Result[Array[Float], GgufError]

    Dequantize tensor index into a freshly allocated Array[Float].

    The buffer is sliced first so the tensor package sees a region it can bounds-check on its own, and so a payload that sits past the buffer is reported as a GGUF-level error rather than a generic buffer overrun.

    GgufFile::get

    source
    fn GgufFile::get(self : GgufFile, key : String) -> MetaValue?

    Look up a metadata value by key. Returns the first match, since GGUF keys are unique in practice and a duplicate would be corruption.

    GgufFile::get_double

    source
    fn GgufFile::get_double(self : GgufFile, key : String) -> Double?

    Look up a float-valued metadata entry.

    GgufFile::get_int

    source
    fn GgufFile::get_int(self : GgufFile, key : String) -> Int64?

    Look up an integer-valued metadata entry, widening any integer width.

    GgufFile::get_string

    source
    fn GgufFile::get_string(self : GgufFile, key : String) -> String?

    Look up a string-valued metadata entry.

    GgufFile::has_key

    source
    fn GgufFile::has_key(self : GgufFile, key : String) -> Bool

    Whether a key is present.

    GgufFile::index_of_tensor

    source
    fn GgufFile::index_of_tensor(self : GgufFile, name : String) -> Int?

    Find a tensor directory entry by name, returning its index.

    GgufFile::len

    source
    fn GgufFile::len(self : GgufFile) -> Int64

    Total buffer length, including the header, metadata and data sections.

    GgufFile::metadata_count

    source
    fn GgufFile::metadata_count(self : GgufFile) -> Int

    GgufFile::tensor

    source
    fn GgufFile::tensor(self : GgufFile, name : String) -> TensorInfo?

    Find a tensor directory entry by name.

    GgufFile::tensor_bytes

    source
    fn GgufFile::tensor_bytes(self : GgufFile, index : Int) -> Result[Bytes, GgufError]

    The raw payload bytes of tensor index, spanning nbytes from the data section.

    Returns an owned copy. Callers that keep the file buffer alive should prefer tensor_span, which avoids duplicating the payload.

    GgufFile::tensor_count

    source
    fn GgufFile::tensor_count(self : GgufFile) -> Int

    GgufFile::tensor_span

    source
    fn GgufFile::tensor_span(self : GgufFile, index : Int) -> Result[TensorSpan, GgufError]

    Locate tensor index's payload without copying it.

    This is what a caller that keeps the file buffer alive should use; the returned offsets address the same Bytes that buffer() exposes.

    MetaArray

    source
    pub(all) enum MetaArray {
      Materialized(Array[MetaValue])
      Summarized(MetaArraySummary)
    } derive(Eq, 
    Debug    )

    An array value, either kept whole or reduced to a bounded summary.
    impl Show for MetaArray

    MetaArraySummary

    source
    pub(all) struct MetaArraySummary {
      elem_type : UInt
      count : UInt64
      prefix : Array[MetaValue]
    } derive(Eq, 
    Debug    )

    A bounded view of a large array.

    MetaValue

    source
    pub(all) enum MetaValue {
      U8(Int)
      I8(Int)
      U16(Int)
      I16(Int)
      U32(UInt)
      I32(Int)
      F32(Float)
      Bool(Bool)
      Str(String)
      Array(MetaArray)
      U64(UInt64)
      I64(Int64)
      F64(Double)
    } derive(Eq, 
    Debug    )

    A GGUF metadata value.

    The u64/i64/f64 variants exist even though GGUF v2 never emits them, because v3 files can and the type ids are stable across versions.
    impl Show for MetaValue

    MetaValue::array_elem_type

    source
    fn MetaValue::array_elem_type(self : MetaValue) -> UInt?

    The element type id of an array value.

    MetaValue::array_len

    source
    fn MetaValue::array_len(self : MetaValue) -> Int?

    Extract an array length whether the array was materialized or summarized.

    MetaValue::as_double

    source
    fn MetaValue::as_double(self : MetaValue) -> Double?

    Extract a floating-point value as Double.

    MetaValue::as_int

    source
    fn MetaValue::as_int(self : MetaValue) -> Int64?

    Extract an integer regardless of which integer width carried it, so callers do not have to special-case u32 versus u64.

    MetaValue::as_string

    source
    fn MetaValue::as_string(self : MetaValue) -> String?

    Convenience accessors used by the CLI and integrations.

    MetaValue::to_text

    source
    fn MetaValue::to_text(self : MetaValue) -> String

    Render a metadata value for listings and diagnostics.

    Long strings are truncated: a chat template can run to kilobytes and would otherwise swamp the output, while the type still has to be visible.

    MetaValue::type_id

    source
    fn MetaValue::type_id(self : MetaValue) -> UInt

    The value_type id of a metadata value, as it would be written on disk.

    TensorInfo

    source
    pub(all) struct TensorInfo {
      name : String
      dims : Array[UInt64]
      type_id : UInt
      offset : UInt64
      n_elems : Int64
      nbytes : Int64?
    } derive(Eq, 
    Debug    )

    One entry of the tensor directory.

    n_elems is the product of dims. nbytes is the on-disk payload size, or None for type ids whose block size this build does not know (removed types and ids past the enum) — such a tensor is still listed so a directory dump can show it, but it cannot be range-checked or dequantized.

    TensorInfo::ggml_type

    source

    The ggml type of this tensor, resolved through the tensor package's table.

    TensorInfo::is_dequantizable

    source
    fn TensorInfo::is_dequantizable(self : TensorInfo) -> Bool

    Whether this build can dequantize the tensor's type.

    TensorInfo::n_dims

    source
    fn TensorInfo::n_dims(self : TensorInfo) -> Int

    Number of dimensions, for symmetry with the on-disk n_dims.

    TensorSpan

    source
    pub(all) struct TensorSpan {
      start : Int
      byte_len : Int
    } derive(Eq, 
    Debug    )

    Where a tensor's payload lives: an offset into the file buffer and its length in bytes.

    array_materialize_limit

    source
    let array_materialize_limit : Int

    Arrays longer than this are summarized instead of materialized. 1024 keeps the memory cost of a small file's metadata flat while still giving callers the full value for anything they are likely to inspect element-by-element.

    array_prefix_len

    source
    let array_prefix_len : Int

    Number of leading elements kept when an array is summarized.

    decode_utf8_lossy

    source
    fn decode_utf8_lossy(raw : Bytes) -> String

    Decode UTF-8, substituting U+FFFD for malformed sequences.

    @encoding/utf8.decode_lossy is used rather than the raising variant so a bad template or token string degrades to readable text instead of aborting the whole parse.

    gguf_default_alignment

    source
    let gguf_default_alignment : UInt

    Alignment used when general.alignment is absent (GGUF_DEFAULT_ALIGNMENT).

    gguf_magic

    source
    let gguf_magic : UInt

    The magic bytes GGUF, read as a little-endian u32.

    gguf_max_dims

    source
    let gguf_max_dims : Int

    Maximum tensor rank GGUF can carry (GGML_MAX_DIMS).

    gguf_version_max

    source
    let gguf_version_max : UInt

    Highest GGUF version this parser accepts.

    is_scalar_value_type

    source
    fn is_scalar_value_type(type_id : UInt) -> Bool

    Whether type_id names a scalar value type (i.e. everything except array).

    parse

    source
    fn parse(buf : Bytes) -> Result[GgufFile, GgufError]

    Parse a GGUF container from an in-memory buffer.

    Accepts GGUF v2 and v3; v1 and anything newer are refused. Every failure mode in the format — bad magic, truncation, unknown metadata or tensor type ids, counts that overflow, tensor payloads outside the buffer — produces a GgufError carrying the file offset, and no input can cause a panic or an out-of-bounds read.

    Array-valued metadata beyond the summarize threshold is folded into a bounded summary; use parse_materializing_metadata to keep the full arrays.

    parse_materializing_metadata

    source
    fn parse_materializing_metadata(buf : Bytes) -> Result[GgufFile, GgufError]

    Like parse, but materializes every array-valued metadata entry in full.

    The tokenizer needs the complete tokenizer.ggml.tokens and tokenizer.ggml.merges arrays (about 100k strings for SmolLM2), which the default summary would truncate. Everything else — validation, tensor directory, data section — behaves identically.

    quote_short

    source
    fn quote_short(s : String) -> String

    Quote a string for display, collapsing it to one line and truncating long values so a single metadata entry cannot dominate the output.

    read_meta_value

    source
    fn read_meta_value(c : ByteCursor, type_id : UInt, materialize? : Bool) -> Result[MetaValue, GgufError]

    Read one metadata value given its already-read value_type id.

    value_type_name

    source
    fn value_type_name(type_id : UInt) -> String

    The on-disk name of a value_type id, for error messages and listings.

    Source Files

    Powered by MoonBit

    Site sourceReport issuePackagesBuild queueSkillsStatistics

    © 2026 mooncakes.io