🥮 mooncakes.io

flate

Pure-MoonBit, runtime-agnostic DEFLATE (RFC 1951) engine with gzip/zlib wrappers — suspendable and io-free.

compression
deflate
inflate
gzip
zlib
rfc1951
moon add moonbit-community/flate@0.4.0
Download zip
Version
0.4.0
License
Apache-2.0
Repository
Last updated
last month
Downloads
639
README

#flate

Pure-MoonBit DEFLATE (RFC 1951) — a runtime-agnostic, suspendable, io-free compression engine, with thin gzip (RFC 1952) and zlib (RFC 1950) wrappers. layout

#Architecture

ENCODE: bytes → raw DEFLATE
═══════════════════════════

 deflate_all     Deflater::step             deflate_all_optimal(input)
  one-shot         streaming, suspendable;    one-shot, offline (zopfli-style)
  greedy           sync flush, preset dict
        │              │                           │
        └──────┬───────┘                           │
               ▼                                    ▼
 ┌─ parser ─────────────────┐    ┌─ parser ───────────────────┐
 │ lz77.mbt                 │    │ optimal_parse.mbt          │
 │ greedy/lazy hash chains  │    │ squeeze: iterated          │
 │ (levels 1-9; 0 = stored) │    │ cost-optimal shortest path │
 ├─ block driver ───────────┤    ├─ planner ──────────────────┤
 │ block_planner.mbt (strm) │    │ optimal_plan.mbt           │
 │ or deflate_all.mbt inline│    │ content-driven splitting   │
 │ both: 16 KB blocks       │    │ (FindMinimum + resplit)    │
 └────────────┬─────────────┘    └─────────────┬──────────────┘
              └────────────┬───────────────────┘
                           ▼
           (tokens, ll_freq, d_freq)    ← the seam: any parser/planner
                           │              pair feeds the same writer
                           ▼
           block_writer.mbt  ◄──  huffman_build.mbt
           one block: stored /      (package-merge length-limited
           fixed / dynamic,          canonical codes)
           whichever is smallest
                           │
                           ▼
           bitwriter.mbt (LSB-first)  →  raw DEFLATE bit stream


DECODE: raw DEFLATE → bytes
═══════════════════════════

           raw DEFLATE bit stream
                    │
           huffman_table.mbt — code lengths → zlib-style chunked tables
                    │
        ┌───────────┴─────────────┐
        ▼                         ▼
 Inflater::step             inflate_all
 (inflate.mbt)              (inflate_all.mbt)
 streaming, suspendable     one-shot, growable output —
 (snapshot/rollback retry); trusted input only
 32 KB window, 1-byte
 output progress,
 preset dictionary


CONTAINERS: thin framing over the engine
════════════════════════════════════════

 gzip/  Encoder/Decoder, gzip_compress/gunzip            (RFC 1952)
        10 B header (optional fields skipped in O(1)) + CRC-32 + ISIZE;
        decodes concatenated multi-member streams (§2.2)
 zlib/  Encoder/Decoder, zlib_compress/zlib_decompress   (RFC 1950)
        2 B header (+ FDICT dictionary id) + Adler-32
 checksum/  incremental CRC-32 / Adler-32 digests

 shared by both pipelines:
   tables.mbt  RFC 1951 symbol tables, fixed codes, window size
   status.mbt  Done / NeedMoreInput / NeedMoreOutput

#Effort tiers

All standard DEFLATE on the wire. Levels 0-9 (zlib's tuning table) on deflate_all / Deflater; deflate_all_optimal adds zopfli-style iterated optimal parsing (optimal_parse.mbt) plus content-driven block splitting (optimal_plan.mbt) — drop-in replacements for the greedy parser (lz77.mbt) and the threshold planner (block_planner.mbt) behind the same (tokens, frequencies) → emit_block seam.

The containers expose both tiers: gzip_compress / zlib_compress and their streaming Encoders take level 0-9; gzip_compress_optimal / zlib_compress_optimal apply the offline zopfli path (one-shot only — a streaming encoder cannot suspend a whole-input optimal parse), for compress-once, serve-forever artifacts.

#
InflateError

source
pub suberror InflateError {
  InflateError(String)
}

Error raised when a DEFLATE stream is malformed.

#
Deflater

source
pub struct Deflater {
  // private fields
}

Streaming DEFLATE compressor : a push-based, suspendable state machine. It emits the stream block-by-block as input arrives , and chooses fixed- or dynamic-Huffman per block.

#
Deflater::is_finished

source
fn Deflater::is_finished(self : Deflater) -> Bool

Whether the final compressed stream has been fully emitted.

#
Deflater::new

source
fn Deflater::new(level? : Int) -> Deflater

Create a fresh compressor. level 0-9 (clamped) trades speed for ratio with zlib-equivalent tuning; 0 emits stored blocks only, 6 is the default.

#
Deflater::reset

source
fn Deflater::reset(self : Deflater) -> Unit

Reset to the start of a fresh DEFLATE stream, keeping the compression level and reusing allocations across many streams (e.g. ZIP entries). A preset dictionary, if any, must be set again.

#
Deflater::set_dictionary

source
fn Deflater::set_dictionary(self : Deflater, dict : BytesView) -> Unit

Preload the match window with a preset dictionary (the last 32 KB of dict), so emitted back-references may reach into it (zlib deflateSetDictionary). Must be called on a fresh stream — right after new() or reset(), before any input. The decoder needs the same dictionary.

#
Deflater::step

source
fn Deflater::step(self : Deflater, input : BytesView, output : MutArrayView[Byte], end? : Bool, flush? : Bool) -> (Status, Int, Int)

Run one compression step. Buffers input, emits as many blocks as the accrued input allows, and drains compressed bytes into the mutable view output. Returns (status, consumed, produced); consumed is always the whole input view. Pass end=true (with any remaining input, or none) to finalize, and keep calling with end=true while it returns NeedMoreOutput to drain the rest. Pass flush=true to sync-flush (zlib Z_SYNC_FLUSH): everything buffered is compressed and the bit stream is padded to a byte boundary with an empty stored block, so all produced bytes are final and transmittable while the stream continues. An empty output only accumulates input (no bytes are produced).

#
Inflater

source
pub struct Inflater {
  // private fields
}

Streaming DEFLATE decompressor (RFC 1951): a push-based, suspendable state machine. All cross-call state lives here, so it pauses on NeedMoreInput/NeedMoreOutput and resumes on the next step. Supports stored, fixed- and dynamic-Huffman blocks. step makes progress with as little as one byte of output room (the output view): a match too large for the buffer is copied in pieces across calls.

#
Inflater::is_finished

source
fn Inflater::is_finished(self : Inflater) -> Bool

Whether the end of the DEFLATE stream has been reached.

#
Inflater::new

source
fn Inflater::new() -> Inflater

Create a fresh decompressor positioned at the start of a DEFLATE stream.

#
Inflater::reset

source
fn Inflater::reset(self : Inflater) -> Unit

Reset to the start of a fresh DEFLATE stream, reusing allocations (the 32 KB window and Huffman table storage) across many streams (e.g. ZIP entries). The stale window contents are unreachable afterwards: any distance reaching past the new stream's start is rejected (dist > wpos). A preset dictionary, if any, must be set again.

#
Inflater::set_dictionary

source
fn Inflater::set_dictionary(self : Inflater, dict : BytesView) -> Unit

Preload the history window with a preset dictionary (the last 32 KB of dict), so the stream's back-references may reach into it (zlib inflateSetDictionary). Must be called on a fresh stream — right after new() or reset(), before any input.

#
Inflater::step

source
fn Inflater::step(self : Inflater, input : BytesView, output : MutArrayView[Byte]) -> (Status, Int, Int) raise

Run one decompression step. Reads from input, writes into the mutable view output. Returns (status, consumed, produced), where produced is the number of bytes written into output. An empty output makes no progress (it returns NeedMoreOutput with 0 produced), so supply at least one byte; one byte always suffices to advance.

Input contract: a decoding unit (a block header, or a literal/match) is read atomically — a unit needs every one of its bytes visible in the same call, and a stored-block header alone spans ~5 bytes. When a unit's bytes are not all present yet, step consumes nothing (consumed == 0) and returns NeedMoreInput. So on NeedMoreInput the next call must re-present every still-unconsumed byte plus more — i.e. feed a non-shrinking, growing input view (drop only the reported consumed prefix). Feeding a fixed-size sliding window smaller than one unit deadlocks: the unit never fits and consumed stays 0 forever.

#
Status

source
pub(all) enum Status {
  Done
  NeedMoreInput
  NeedMoreOutput
} derive(Eq, 
Debug)

Outcome of one step of the streaming DEFLATE state machine.

The engine is runtime-agnostic: it moves bytes between an input view and an output buffer and suspends by returning one of these, so any driver — a synchronous loop or an async adapter — can resume it.

#
deflate_all

source
fn deflate_all(input : Bytes, level? : Int) -> Bytes

Compress bytes into a complete raw DEFLATE stream held entirely in memory. Because the whole input is in hand, no sliding window is needed: this tokenizes directly over input, advancing a single offset, with none of the streaming Deflater's per-block buffering.

#
deflate_all_optimal

source
fn deflate_all_optimal(input : Bytes, iterations? : Int) -> Bytes

Compress with zopfli-style effort: content-driven block splitting plus iterated optimal parsing, with a second splitting pass over the optimal tokens (blocksplittinglast). Tens to hundreds of times slower than deflate_all — for compress-once, serve-forever artifacts. iterations is the squeeze count per block (zopfli's default is 15).

#
inflate_all

source
fn inflate_all(input : Bytes) -> Bytes raise InflateError

Decompress a complete raw DEFLATE stream held entirely in memory

Source Files

Powered by MoonBit

Site sourceReport issuePackagesBuild queueSkillsStatistics

© 2026 mooncakes.io