#flate

    Pure-MoonBit DEFLATE (RFC 1951) — a runtime-agnostic, suspendable, io-free compression engine in moonbitlang/core/compress/flate. The sibling compress/gzip (RFC 1952) and compress/zlib (RFC 1950) packages provide compressed formats; archive/zip (APPNOTE.TXT) provides ZIP archives.

    gzip, zlib, and zip share CRC-32 and Adler-32 implementations in the internal moonbitlang/core/internal/checksum package. This package is an implementation detail and cannot be imported from outside moonbitlang/core.

    #Import

    Add the packages you use to moon.pkg:

    import { "moonbitlang/core/compress/flate", "moonbitlang/core/compress/gzip", "moonbitlang/core/compress/zlib", "moonbitlang/core/archive/zip", }

    #Quick start

    The one-shot API is useful when the complete payload is already in memory. Read-only codec inputs accept BytesView, so a slice of a larger buffer needs no ownership conversion at the call site:

    ///|
    test "README raw DEFLATE round-trip" {
    let source = b"runtime-agnostic streaming compression"
    let compressed = @flate.deflate_all(source)
    @test.assert_eq(@flate.inflate_prefix(compressed).data, source)
    }

    The direct-buffer APIs accept MutArrayView[Byte], including slices of arrays. Pass the same strategy to deflate_bound and compression; the bound panics for a negative input length or an unrepresentable result. Compression returns None if its encoded output does not fit, while decompression raises OutputLimitExceeded(limit~, observed~) if its decoded output does not fit. Reuse Compressor and Decompressor across streams to retain their workspace.

    ///|
    test "README caller-buffer DEFLATE" {
    let source = b"direct input and caller-owned output"
    let compressor = @flate.Compressor(strategy=Standard(level=6))
    let compressed_buffer = FixedArray::make(
    @flate.deflate_bound(source.length()),
    b'\x00',
    )
    guard compressor.compress_into(source, compressed_buffer.mut_view())
    is Some(compressed_len) else {
    @test.fail("deflate_bound was too small")
    }
    let compressed = Bytes::from_array(compressed_buffer[:compressed_len])
    let decompressor = @flate.Decompressor()
    let output = FixedArray::make(source.length(), b'\x00')
    let decoded = decompressor.decompress_into(compressed, output.mut_view())
    @test.assert_eq(decoded.consumed, compressed.length())
    @test.assert_eq(Bytes::from_array(output[:decoded.produced]), source)
    }

    Use Compressor(...).compress_into(input, output) or Decompressor(...).decompress_into(input, output) for a single call, and retain the object to reuse workspace across calls. Decompressor::decompress_into returns DecompressResult with consumed and produced. The default mode=Exact rejects trailing bytes; use mode=Prefix to leave them unread, with consumed locating their start. Use the streaming APIs when input or output must suspend.

    Compressor::compress(input) returns independent Bytes and reuses its workspace across calls, without requiring a caller-owned output buffer. Decompressor::decompress(input) is its counterpart: the output grows as in inflate_exact and inflate_prefix, returning Decompressed with owned data and the input consumed. Pass mode=Prefix to read a stream of unknown length from the front of a buffer and go on behind it:

    ///|
    test "README prefix decode" {
    // A record: a raw DEFLATE stream, then whatever follows it.
    let payload = b"the stream ends with its final block"
    let record = @flate.deflate_all(payload) + b"what follows"
    let decompressor = @flate.Decompressor()
    let result = decompressor.decompress(record, mode=Prefix)
    @test.assert_eq(result.data, payload)
    let rest = record[result.consumed:]
    @test.assert_eq(rest.to_owned(), b"what follows")
    }

    The streaming API is a pure push state machine. It owns no I/O object, so both synchronous and asynchronous callers can supply input and drain output:

    ///|
    test "README streaming IncrementalEncoder" {
    let source = b"small buffers exercise suspension and resume"
    let encoder = @flate.IncrementalEncoder()
    let chunk = FixedArray::make(3, b'\x00')
    let compressed = @buffer.Buffer()
    let mut first = true
    for ;; {
    let input = if first { source[:] } else { b"" }
    let result = encoder.step(
    input,
    chunk.mut_view(),
    action=if first { Finish } else { Continue },
    )
    let consumed = result.consumed
    let produced = result.produced
    @test.assert_eq(consumed, if first { source.length() } else { 0 })
    first = false
    for i in 0..<produced {
    compressed.write_byte(chunk[i])
    }
    if result.status is Done {
    break
    }
    }
    @test.assert_eq(@flate.inflate_prefix(compressed.to_bytes()).data, source)
    }

    #State-machine contract

    • step and finish return a #valtype StepResult containing status, consumed, and produced. The counts describe exactly the prefixes accepted and written by that call. Drop only input[:result.consumed]; large input views may be partially consumed when output is backpressured, keeping internal memory bounded. Later calls and resets do not change an earlier result. A failing call returns no result, and its output buffer may already contain partial output.
    • NeedMoreOutput preserves all pending work. Resume with another non-empty output view; one byte is sufficient.
    • IncrementalDecoder owns the bounded tail of an incomplete atomic unit. On NeedMoreInput, feed the next non-overlapping chunk; no growing replay window is required.
    • Encoders take one DeflateAction: Continue, SyncFlush, or Finish. Pass the requested action with the final input of that batch. If consumed < input.length(), re-present that suffix with the same action. Once the complete view is accepted, the request remains latched across output backpressure.
    • Decoder step(input, output) methods accept a mutable output view, including slices of fixed arrays or arrays. The engine chooses its copy strategy internally and releases caller buffers after each call. gzip and zlib checksum the produced range and drain pending body input before accepting more, so output can be produced while result.consumed is zero. Re-present the unaccepted input on the next call.
    • Raw IncrementalDecoder callers own input completion: NeedMoreInput after the input source is exhausted means the stream is truncated. On NeedMoreOutput, keep calling step with the unconsumed suffix and fresh output space, even if the suffix is empty; stop at Done.
    • gzip/zlib decoders use step(input, output) for ordinary input. Once the source reports EOF and every supplied byte has been accepted, call check_end() with no buffers. It returns Done, returns NeedMoreOutput when output space is required to continue, or raises an error for incomplete input. Continue draining with step(b"", output) after NeedMoreOutput; EOF remains latched and later input is not accepted. Do not call check_end while any source suffix remains unaccepted. A raw decoder must be discarded or reset after errors; container decoders stably rethrow their first error.
    • Once Done is returned, new input is not consumed; reset the raw engine or gzip/zlib encoder before reuse, or create a new decoder wrapper. A reset encoder produces the same bytes as a fresh one with the same options and driving schedule, while reusing its workspace.
    • Configure a raw preset dictionary through IncrementalEncoder(...)/IncrementalDecoder(...), or while starting a fresh stream through reset(dictionary=...); dictionary selection cannot be mutated after a stream begins.
    • gzip/zlib decoders may accept transport read-ahead beyond their trailer; after Done, recover that suffix with unused_input() before parsing the next protocol frame. For an exact one-member gzip boundary, construct the decoder with multistream=false.
    • Raw failures expose a stable InflateErrorKind (Truncated, Corrupt, TrailingData, OutputLimitExceeded, Cancelled, or InvalidArgument) alongside diagnostic text; gzip/zlib similarly expose GzipErrorKind/ZlibErrorKind.

    #Exact and bounded one-shot decoding

    inflate_exact returns owned Bytes and rejects trailing bytes with TrailingData. inflate_prefix stops at the first final block and returns Decompressed; use its consumed count to find the suffix. Neither function accepts a mode argument. The reusable Decompressor methods still default to mode=Exact and accept mode=Prefix for prefix decoding. The whole-buffer signatures are:

    inflate_exact(input, dictionary?, max_output?, cancelled?, two_pass?=false) -> Bytes inflate_prefix(input, dictionary?, max_output?, cancelled?, two_pass?=false) -> Decompressed Decompressor::decompress(self, input, mode?=Exact, max_output?, cancelled?) -> Decompressed Decompressor::decompress_into(self, input, output, mode?=Exact, cancelled?) -> DecompressResult

    Decompressed contains owned data and consumed; DecompressResult contains consumed and produced. All four calls raise InflateError on failure. Construct Decompressor(dictionary=...) to own the dictionary's final 32 KiB and reuse it independently for each stream. Dictionary history is not part of returned data, output limits, or produced counts. Raw DEFLATE has no dictionary checksum; a wrong dictionary is not guaranteed to be detected. inflate_exact, inflate_prefix, Decompressor::decompress, and gzip/zlib decompress accept max_output and cancelled. gzip counts output across all members; zlib accepts dictionary on both compress and decompress. The compressor uses the last 32 KiB as history and records the full dictionary checksum.

    ///|
    test "README exact and limited inflate" {
    let source = b"bounded convenience API"
    let compressed = @flate.deflate_all(source)
    @test.assert_eq(
    @flate.inflate_exact(compressed, max_output=source.length()),
    source,
    )
    }

    Omitting max_output permits output up to the capacity of Bytes; negative limits raise InvalidArgument. OutputLimitExceeded(limit~, observed~) carries UInt64 counts. observed includes the rejected chunk or match, not necessarily the complete decoded size or bytes written. For *_into, the limit is the output buffer's capacity. These calls are not transactional: an error, including TrailingData, may leave bytes written inside the supplied output view.

    A cancelled callback returning true raises Cancelled. It is polled regularly during decoding and, for gzip/zlib, checksum verification; gzip also polls while scanning headers and joining members. A failed Decompressor remains reusable and returned results remain independent of later calls.

    Both inflate_exact(two_pass=true) and inflate_prefix(two_pass=true) count and validate first, then decode into one exactly sized allocation. They trade a second decode pass for lower peak memory; the default grows the output in one pass. Prefix decoding still returns the first stream's consumed boundary and leaves trailing input unread in either path.

    #ZIP container

    The @zip package provides metadata-only range reads, in-memory archives, and incremental writing over the same compression engine.

    Use DirectoryReader to browse a ZIP without loading the complete file or decoding its payloads. It supports classic ZIP beyond 2 GiB and ZIP64, with UInt64 archive sizes, file offsets, and declared entry sizes. The caller owns file or HTTP range I/O; the reader requests only a bounded tail, necessary ZIP64 records, and directory chunks:

    ///|
    fn browse_zip(
    size : UInt64,
    read_range : (UInt64, Int32) -> Bytes raise,
    ) -> @zip.Directory raise {
    let reader = @zip.DirectoryReader(
    size,
    limits=DirectoryLimits(max_directory_bytes=16777216UL, max_entries=100000UL),
    )
    let mut input : Bytes = b""
    for ;; {
    match reader.advance(input) {
    NeedRead(range) => input = read_range(range.offset(), range.length())
    Done(directory) => return directory
    }
    }
    }

    Start with empty input, then supply the exact bytes from each requested range. Accumulate short transport reads before submitting them. All ranges must come from the same immutable archive. Requests are at most 65,557 bytes; directory chunks are at most 64 KiB, and records crossing chunks are retained in a bounded buffer. Memory grows with directory metadata, rather than compressed or decoded payload sizes. DirectoryLimits caps encoded directory bytes and entry count; the in-memory listing can hold at most 2^31 - 1 entries. No declared payload size causes a payload allocation or counts against these directory limits.

    Directory::entries() preserves directory order and duplicate names. DirectoryEntry exposes the UTF-8 name, compressed/uncompressed UInt64 sizes, raw compression method number, declared CRC-32, directory/encryption indicators, creator system, external attributes, and packed DOS modification date/time (without a timezone). Unknown compression methods and encrypted payloads remain listable when the directory itself is readable. Multi-disk archives and encrypted central directories are unsupported. Listing validates directory structure and referenced bounds; it does not read local headers, inflate files, or verify CRCs. An optional trailing directory digital-signature record is accepted after checking its signature marker and length; its cryptographic contents are not verified. InvalidRead identifies incorrect response lengths or input outside an active request. Errors remain latched; repeated empty calls after Done return the same listing. Callers can cancel by stopping between requests.

    Open a listed entry with Directory::open_entry(index) to create an EntryReader. Indices distinguish duplicate names. It extracts Store and Deflate entries from bounded ranges, including ZIP64 and entries with more than 2 GiB of decoded data. EntryReadLimits(max_output_bytes=...) sets a UInt64 output ceiling; oversized declarations are rejected before I/O, and actual decoded output is checked before reporting bytes to the caller.

    ///|
    fn extract_entry(
    directory : @zip.Directory,
    index : Int32,
    read_range : (UInt64, Int32) -> Bytes raise,
    emit : (BytesView) -> Unit raise,
    ) -> Unit raise {
    let reader = directory.open_entry(index)
    let output = FixedArray::make(65536, b'\x00')
    let mut input : Bytes = b""
    for ;; {
    let status = reader.step(input, output.mut_view())
    let produced = reader.last_produced()
    if produced > 0 {
    emit(Bytes::from_array(output[:produced]))
    }
    match status {
    NeedRead(range) => input = read_range(range.offset(), range.length())
    NeedMoreOutput => input = b""
    Done => return
    }
    }
    }

    Start with empty input. A NeedRead response requires exactly the requested bytes; after NeedMoreOutput, submit empty input with another output buffer. The reader retains unconsumed compressed bytes, so callers never replay an accepted range. Input and output arrays are released before each return. Read last_produced() after every successful step, including those returning NeedRead or Done. total_consumed() and total_produced() report UInt64 compressed-payload and decoded byte counts. Memory is bounded by one compressed chunk, local metadata, the codec workspace, and the caller's output buffer.

    Done verifies the local header against the directory, exact compressed and decoded sizes, CRC-32, and any classic or ZIP64 data descriptor. Output produced earlier remains unverified; discard partial output on failure. Like read_all, zero-length Deflate entries with zero compressed bytes are accepted; CRC-32 and any data descriptor are still checked. Unsupported methods and encrypted entries remain listable but cannot be opened. InvalidEntryIndex identifies an invalid index, SizeMismatch an inconsistent decoded size, and InvalidRead a range-response protocol error. A failed reader stably rethrows its error with last_produced() zero; repeated empty steps after completion return Done. Cancel by stopping between calls.

    Use Archive for in-memory editing and serialization. read(BytesView) still decodes and verifies every entry, and write(Archive) returns one complete Bytes; these convenience paths retain their Int32 buffer limits. Use DirectoryReader / EntryReader for reading large archives and chunked Writer input/output for creating them:

    ///|
    test "README zip round-trip" {
    let archive = @zip.Archive()
    archive.add("hello.txt", b"hello, zip")
    let bytes = @zip.write(archive)
    let parsed = @zip.read_all(bytes)
    @test.assert_eq(parsed.get("hello.txt"), Some(b"hello, zip"))
    }

    Supported features include STORED and DEFLATE entries, ZIP64, data descriptors, UTF-8 names, and archive comments. read_all accepts ReadLimits to bound archive size (including the complete retained source), entry count, and decompressed sizes, plus a cancelled callback. Exceeding a limit raises ZipError(LimitExceeded(kind~, limit~, observed=actual), message); limit and actual values are UInt64 on both memory and directory paths.

    read_all verifies each entry's CRC-32, raising ChecksumMismatch for a damaged payload. Cancellation remains Cancelled throughout parsing, decoding and checksum verification.

    Entry exposes dos_date, dos_time, creator_system, and external_attributes. These fields survive replace and are written back in both write modes. read_all retains a copy of the archive plus all decoded payloads in memory.

    write(archive, preserve=true) reuses unchanged source records; an unmodified archive round-trips byte-for-byte, including prefixes, gaps and directory order. After an edit, local records and directory entries keep their respective order; bytes outside records, such as prefixes and gaps, are omitted. max_package_bytes covers the full retained source, including these bytes. Supply max_output_bytes to bound the serialized size before allocating it. write returns the complete ZIP as Bytes.

    For incremental output, use Writer with a synchronous sink. add accepts a complete file; begin_entry / write / end_entry accept chunks. Supply size when known; unknown sizes use ZIP64.

    ///|
    test "README incremental ZIP writer" {
    // A real folder adapter passes a buffered file-write callback here.
    let output = @buffer.Buffer()
    let writer = @zip.Writer(chunk => output.write_bytes(chunk))
    writer.add("small.txt", b"small file")
    writer.begin_entry("chunked.txt", size=6UL)
    writer.write(b"abc")
    writer.write(b"def")
    writer.end_entry()
    writer.finish()
    let archive = @zip.read_all(output.to_bytes())
    @test.assert_eq(archive.get("chunked.txt"), Some(b"abcdef"))
    }

    Call finish to write the central directory. The sink must consume each chunk before returning. If it raises an error, the writer becomes unusable and the partial output must be discarded; the sink's error reaches the caller unchanged. The writer's own failures are ZipErrors: InvalidState for a call out of order (no open entry, an entry already open, a reentrant sink call, or use after finish or a sink failure) and SizeMismatch when an entry's input exceeds or falls short of its declared size. A size error leaves the entry open, so the caller can supply the missing bytes and call end_entry again.

    Large individual entries use begin_entry / repeated write / end_entry; their declared size and cumulative positions are UInt64. The writer selects ZIP64 and data descriptors internally. add(BytesView) remains convenient for a complete in-memory entry. Directory metadata is retained until finish.

    Writer retains directory metadata, current-entry metadata and codec buffers. add processes a complete file during the call; write feeds chunks through the codec or directly to the sink. Folder traversal and file I/O belong to the caller.

    #Architecture

    The raw engine shares Huffman construction, block writing and format validation across its encoding and decoding paths:

    PathImplementation
    Whole-buffer compressiondeflate_all.mbt, lz77.mbt
    Streaming compressiondeflate.mbt, block_planner.mbt, lz77.mbt
    Content-driven block splittingsplit_plan.mbt
    Iterated optimal parsing and splittingoptimal_parse.mbt, optimal_plan.mbt
    Block encoding and bit outputblock_writer.mbt, huffman_build.mbt, bitwriter.mbt
    Whole-buffer decompressioninflate_all.mbt
    Streaming decompressioninflate.mbt
    Shared decoding tables and validationhuffman_table.mbt, tables.mbt, decode_rules.mbt

    One-pass inflate_exact and inflate_prefix use the direct MemDecoder path for every combination of dictionary, output limit, and cancellation callback. With two_pass=true, both functions drive IncrementalDecoder to count and validate, then decode into one exactly sized allocation. Decompressor::decompress_into uses the direct decoder with caller-owned output, Decompressor::decompress with an output that grows. Differential tests compare one-shot and streaming decoding across randomized chunk/output schedules, truncations and bit flips.

    The gzip and zlib packages add framing, checksums and streaming wrappers. ZIP provides in-memory archives and a synchronous incremental writer. The internal checksum package supplies CRC-32 and Adler-32 independently of the codecs.

    #Optional fast stored blocks

    fast_store=true opts into a heuristic speed/ratio trade-off. It samples byte variety, frequency skew and local repetition; small inputs also get a periodicity check. Inputs below 1 KiB skip the classifier. A candidate judged unlikely to compress is emitted as DEFLATE stored blocks without match search or Huffman construction. The default is false: existing calls retain their normal compression behavior.

    ///|
    let codec = @flate.Compressor(strategy=Standard(level=6), fast_store=true)

    ///|
    let raw = @flate.deflate_all(input, fast_store=true)

    ///|
    let stream = @flate.IncrementalEncoder(level=6, fast_store=true)

    ///|
    let archive_bytes = @zip.write(archive, fast_store=true)

    The option is also available on all deflate_all strategies, Compressor, gzip/zlib compression functions and streaming encoders, all ZIP serialization functions, and zip.Writer. Whole-buffer APIs classify the entire input; streaming encoders decide per block and retain stored bytes as match history. ZIP entries declared Deflate remain method 8; this option selects stored blocks inside DEFLATE, rather than changing the entry to ZIP method 0. Preserved ZIP payloads are reused; record offsets may be patched.

    This is not a proof of incompressibility. Long repeated patterns or useful dictionary matches can be missed, potentially making output much larger. Enable it only when that compression-ratio trade-off is acceptable. Decoding remains lossless, and the normal output bounds and output limits still apply.

    #Effort tiers

    deflate_all, Compressor, and gzip/zlib compress share strategy, defaulting to Standard(level=6):

    • Standard(level=...): fixed-cadence blocks with zlib-derived level tuning.
    • Split(level=...): tokenize once, then choose content-driven block cuts.
    • Optimal(iterations=...): iterated optimal parsing and content-driven splitting.

    Levels are clamped to 0-9; iterations to at least 1. All strategies produce standard DEFLATE. Output size depends on the input and chosen strategy. IncrementalEncoder retains its level parameter and bounded 16 KiB input blocks. All raw whole-buffer compression entry points accept dictionary; Compressor owns its last 32 KiB and reuses it for each stream. The output bound is unchanged.

    gzip/zlib re-export Strategy, Status, StepResult, and DeflateAction. Their streaming encoders take level 0-9 and support reset.

    #API conventions

    Read-only inputs accept BytesView; existing Bytes values convert at the call site without copying. Compressor::compress and compress_into reuse workspace for repeated streams.

    ZIP writing uses write(archive, preserve=true, max_output_bytes=n) for byte-preserving, bounded output. The options are independent. Construct read bounds with ReadLimits(max_entries=..., max_package_bytes=...,max_entry_uncompressed_bytes=..., max_total_uncompressed_bytes=...). max_package_bytes also bounds the source retained for byte-preserving writes. Metadata-only reads use separate DirectoryLimits, without decoded-size limits.

    ZIP layout controls are internal: Archive::add no longer takes data_descriptor, and Entry no longer exposes data_descriptor(). The former max_preserved_source_bytes limit and PreservedSourceBytes error kind are replaced by the existing package-size limit. Limit errors now carry 64-bit ceilings and actual values.

    Internal format and tuning constants are private. Use deflate_bound to size compression output. Filename-based compression selection belongs to the caller; the CLI supplies its own policy.

    ZIP payload CRC and gzip FHCRC are verified. A damaged checksum raises ChecksumMismatch; malformed ZIP DEFLATE data raises InvalidDeflate, and cancellation during ZIP decompression remains Cancelled. deflate_bound rejects negative or unrepresentable capacities.

    Use != for inequality and @debug.Repr(value) for debugging.

    InflateError

    pub suberror InflateError {
    InflateError(InflateErrorKind, String)
    }

    Error raised while decoding a raw DEFLATE stream.

    Compressor

    type Compressor

    A reusable, whole-buffer raw DEFLATE compressor. Unlike the suspendable IncrementalEncoder, it reads the input directly and writes one complete stream into either a caller-provided buffer or reusable growable output storage.

    Compressor::Compressor

    fn Compressor::Compressor(strategy? : Strategy, dictionary? : BytesView, fast_store? : Bool) -> Compressor

    Create a reusable whole-buffer compressor with the selected strategy. The last 32 KiB of dictionary are owned and reused for every stream. fast_store may bypass parsing and use stored blocks for sampled inputs.

    Compressor::compress

    fn Compressor::compress(self : Compressor, input : BytesView) -> Bytes

    Compress a complete stream, reusing the parsing and output workspace. The returned bytes are independent of subsequent calls to this compressor.

    Compressor::compress_into

    fn Compressor::compress_into(self : Compressor, input : BytesView, output : MutArrayView[Byte]) -> Int?

    Compress input directly into output, returning its byte length. None means that output was too small; any prefix written before exhaustion is unspecified. Reuse one Compressor and one caller-owned output buffer to avoid per-stream workspace and result-buffer allocation.

    DecompressResult

    pub struct DecompressResult {
    consumed : Int
    produced : Int
    } derive(Eq,
    Debug
    )

    Byte counts from a successful whole-buffer decompression call.

    DecompressResult::equal

    Decompressed

    pub struct Decompressed {
    data : Bytes
    consumed : Int
    } derive(Eq,
    Debug
    )

    Owned output and the compressed prefix consumed by a whole-buffer decode.

    Decompressed::equal

    Decompressor

    type Decompressor

    A reusable whole-buffer decoder with independent history for each stream.

    Decompressor::Decompressor

    fn Decompressor::Decompressor(dictionary? : BytesView) -> Decompressor

    Own the final 32 KiB of the optional dictionary for reuse across streams.

    Decompressor::decompress

    fn Decompressor::decompress(self : Decompressor, input : BytesView, mode? : InputMode, max_output? : Int, cancelled? : () -> Bool) -> Decompressed raise InflateError

    Decode with reusable workspace; max_output limits output bytes, and Prefix permits trailing input.

    Decompressor::decompress_into

    fn Decompressor::decompress_into(self : Decompressor, input : BytesView, output : MutArrayView[Byte], mode? : InputMode, cancelled? : () -> Bool) -> DecompressResult raise InflateError

    Decode into the view; Exact rejects trailing input, and errors may leave output written.

    DeflateAction

    pub(all) enum DeflateAction {
    Continue
    SyncFlush
    Finish
    } derive(Eq,
    Debug
    )

    Requested action for one streaming compression step.

    The action applies after the complete input view has been consumed. If a step reports partial consumption, re-present the remaining suffix with the same action.

    DeflateAction::equal

    IncrementalDecoder

    type IncrementalDecoder

    Streaming DEFLATE decompressor (RFC 1951): a push-based, suspendable state machine. All cross-call state lives here, so it pauses on NeedMoreInput/NeedMoreOutput and resumes on the next step. Supports stored, fixed- and dynamic-Huffman blocks. step makes progress with as little as one byte of output room (the output view): a match too large for the buffer is copied in pieces across calls.

    IncrementalDecoder::IncrementalDecoder

    fn IncrementalDecoder::IncrementalDecoder(dictionary? : BytesView) -> IncrementalDecoder

    Create a fresh decompressor positioned at the start of a DEFLATE stream. If supplied, dictionary preloads the history window with its final 32 KiB.

    IncrementalDecoder::is_finished

    fn IncrementalDecoder::is_finished(self : IncrementalDecoder) -> Bool

    Whether the end of the DEFLATE stream has been reached.

    IncrementalDecoder::reset

    fn IncrementalDecoder::reset(self : IncrementalDecoder, dictionary? : BytesView) -> Unit

    Reset to the start of a fresh DEFLATE stream, reusing allocations (the 32 KB window and Huffman table storage) across many streams (e.g. ZIP entries). The stale window contents are unreachable afterwards: any distance reaching past the new stream's start is rejected (dist > wpos). Pass dictionary to preload the fresh stream's history window.

    IncrementalDecoder::step

    fn IncrementalDecoder::step(self : IncrementalDecoder, input : BytesView, output : MutArrayView[Byte]) -> StepResult raise InflateError

    Run one decompression step. Reads from input, writes into output, and returns StepResult with this call's status, consumed and produced byte counts. Drop exactly input[:result.consumed]. An empty output makes no progress (NeedMoreOutput, zero produced); one output byte always suffices to advance. A failing call returns no result; its output buffer may already contain partial output.

    Headers and Huffman symbols remain atomic internally, but an incomplete unit's small tail is retained by the engine. Therefore callers may feed ordinary non-overlapping chunks: on NeedMoreInput the complete supplied view has been accepted and need not be re-presented. The caller owns input completion: if the input source has ended and this returns NeedMoreInput, report a truncated stream. NeedMoreOutput still requires draining with fresh output space, even after all input has been accepted. Once any error is raised, later calls stably raise the same error until reset().

    IncrementalEncoder

    type IncrementalEncoder

    Streaming DEFLATE compressor : a push-based, suspendable state machine. It emits the stream block-by-block as input arrives, and chooses fixed- or dynamic-Huffman per block.

    IncrementalEncoder::IncrementalEncoder

    fn IncrementalEncoder::IncrementalEncoder(level? : Int, dictionary? : BytesView, fast_store? : Bool) -> IncrementalEncoder

    Create a fresh compressor. level 0-9 (clamped) trades speed for ratio with level-specific tuning; 0 emits stored blocks only, 6 is the default. dictionary, when supplied, preloads the match window with its last 32 KB; the decoder must use the same preset dictionary. Optional fast_store samples each block and may skip compression; it is off by default and may increase output size, including when a dictionary could help.

    IncrementalEncoder::is_finished

    fn IncrementalEncoder::is_finished(self : IncrementalEncoder) -> Bool

    Whether the final compressed stream has been fully emitted.

    IncrementalEncoder::reset

    fn IncrementalEncoder::reset(self : IncrementalEncoder, dictionary? : BytesView) -> Unit

    Reset to the start of a fresh DEFLATE stream, keeping the compression level and fast_store policy, reusing allocations across streams (e.g. ZIP entries). With no dictionary, the history is cleared. Supplying one immediately preloads its last 32 KB for the new stream; the decoder must use the same preset dictionary.

    IncrementalEncoder::step

    fn IncrementalEncoder::step(self : IncrementalEncoder, input : BytesView, output : MutArrayView[Byte], action? : DeflateAction) -> StepResult

    Run one compression step. Accepts as much of input as fits in one bounded block window, emits blocks, and drains compressed bytes into output. Returns StepResult with the status and this call's consumed/produced counts. Drop only input[:result.consumed] and re-present the suffix. With a small output buffer a large input may be consumed only partially, which propagates output backpressure instead of growing internal memory with the caller's input size. Once a final block has been emitted, calls only drain that block and consume no new input; after completion they idempotently return Done with both counts zero.

    Use action=Finish with the final input view. The action is accepted after that entire view has been consumed, then remains latched across NeedMoreOutput; if a call reports partial consumption, re-present the suffix with action=Finish. Later input is left unconsumed.

    action=SyncFlush similarly requests zlib-style Z_SYNC_FLUSH: everything buffered is compressed and the bit stream is padded to a byte boundary with an empty stored block, so all produced bytes are final and transmittable while the stream continues. A sync-flush is sticky across output backpressure once accepted. Continue only accepts input. An empty output may accept at most one bounded block window before returning NeedMoreOutput.

    InflateErrorKind

    pub(all) enum InflateErrorKind {
    Truncated
    Corrupt
    TrailingData
    OutputLimitExceeded(limit~ : UInt64, observed~ : UInt64)
    Cancelled
    InvalidArgument
    } derive(Eq,
    Debug
    )

    Stable classification for failures raised by the raw DEFLATE APIs.

    InflateErrorKind::equal

    InputMode

    pub(all) enum InputMode {
    Exact
    Prefix
    } derive(Eq,
    Debug
    )

    Whether whole-buffer decoding requires an exact input boundary.

    InputMode::equal

    fn InputMode::equal(InputMode, InputMode) -> Bool

    Status

    pub(all) enum Status {
    Done
    NeedMoreInput
    NeedMoreOutput
    } derive(Eq,
    Debug
    )

    Outcome of one step of the streaming DEFLATE state machine.

    The engine is runtime-agnostic: it moves bytes between an input view and an output buffer and suspends by returning one of these, so any driver — a synchronous loop or an async adapter — can resume it.

    Status::equal

    fn Status::equal(Status, Status) -> Bool

    StepResult

    pub(all) struct StepResult {
    status : Status
    consumed : Int
    produced : Int
    } derive(Eq,
    Debug
    )

    Result of one streaming codec call. The counts describe the input prefix accepted and the output prefix written by this call, including when it returns Done. Keep the result to inspect it independently of later calls.

    StepResult::equal

    fn StepResult::equal(StepResult, StepResult) -> Bool

    Strategy

    pub(all) enum Strategy {
    Standard(level~ : Int)
    Split(level~ : Int)
    Optimal(iterations~ : Int)
    } derive(Eq,
    Debug
    )

    Whole-buffer compression strategy. Levels are clamped to 0-9; level 0 emits stored blocks only.

    Strategy::equal

    fn Strategy::equal(Strategy, Strategy) -> Bool

    Strategy::optimal

    fn Strategy::optimal(iterations? : Int) -> Strategy

    Select optimal parsing and splitting with 15 iterations by default.

    Strategy::split

    fn Strategy::split(level? : Int) -> Strategy

    Select content-driven splitting with compression level 6 by default.

    deflate_all

    fn deflate_all(input : BytesView, strategy? : Strategy, dictionary? : BytesView, fast_store? : Bool) -> Bytes

    Compress bytes into a complete raw DEFLATE stream held entirely in memory. Defaults to Standard(level=6). Opt-in fast_store may bypass any strategy and emit stored blocks after a heuristic check, trading compression ratio for speed; it is disabled by default. dictionary seeds the history with its last 32 KiB; the decoder must use the same dictionary.

    deflate_bound

    fn deflate_bound(input_size : Int, strategy? : Strategy) -> Int

    An output capacity sufficient for the selected strategy, with or without fast_store. Panics for negative input sizes or an unrepresentable bound.

    inflate_exact

    fn inflate_exact(input : BytesView, dictionary? : BytesView, max_output? : Int, cancelled? : () -> Bool, two_pass? : Bool) -> Bytes raise InflateError

    Decode exactly one stream, rejecting trailing input; two_pass uses an exactly sized output allocation.

    inflate_prefix

    fn inflate_prefix(input : BytesView, dictionary? : BytesView, max_output? : Int, cancelled? : () -> Bool, two_pass? : Bool) -> Decompressed raise InflateError

    Decode a stream prefix and return its consumed boundary; two_pass uses an exactly sized output allocation.