zip

    Bounded ZIP archives with Store, Deflate and Zstandard writing and Deflate64 reading, in pure MoonBit.

    zip
    archive
    deflate
    zstandard
    Download zip
    Author
    Version
    0.1.0
    License
    Apache-2.0
    Last updated
    14 hours ago
    Downloads
    4

    #gaato/zip

    ZIP archives in pure MoonBit for Native, JS, Wasm and Wasm GC. Read and write Store (0), Deflate (8) and Zstandard (93), read Deflate64 (9), edit archives in memory, or use range readers and a writer over a caller-supplied sink. The API follows MoonBit core's archive/zip.

    #Install

    moon add gaato/zip

    Import "gaato/zip" in your moon.pkg.

    #Usage

    ///|
    test "create, read, edit and rewrite a ZIP" {
    let archive = @zip.Archive()
    archive.add("hello.txt", b"hello", compression=Zstandard)
    let bytes = @zip.write(archive)
    let decoded = @zip.read_all(
    bytes,
    limits=@zip.ReadLimits(
    max_package_bytes=1024 * 1024,
    max_entries=100,
    max_entry_uncompressed_bytes=1024 * 1024,
    max_total_uncompressed_bytes=4 * 1024 * 1024,
    ),
    )
    assert_eq(decoded.get("hello.txt"), Some(b"hello"[:]))
    assert_true(decoded.replace("hello.txt", b"updated"))
    assert_eq(
    @zip.read_all(@zip.write(decoded)).get("hello.txt"),
    Some(b"updated"[:]),
    )
    }

    DirectoryReader::advance returns NeedRead(range) or Done(directory). Supply exactly the requested range from the same immutable source. Listing retains metadata, including encrypted entries and unknown methods, without reading their payloads. Directory::open_entry rejects unsupported methods and encryption before any read. Drive EntryReader::step with the requested bytes and an output slice; take last_produced() after every successful step, including Done. Output remains unverified until Done checks sizes and CRC.

    Writer takes a synchronous byte sink. Use add for a complete file or begin_entry, write, end_entry for chunks, then finish. Sink failures propagate and poison the writer. level is the level the selected codec takes: it is passed unchanged to gaato/deflate or gaato/zstd. With gaato/deflate 0.4.0, Deflate levels 0 through 12 are available, including near-optimal levels 10–12. preserve=true on write reuses unchanged source records; a pristine archive is byte identical, including an SFX prefix. Editing drops bytes outside the records. Deflate64 is read-only: write (including preserve=true), Writer::add and Writer::begin_entry reject it with InvalidArgument. A rejected writer argument emits no bytes and leaves the writer ready for another entry.

    #Limits and errors

    Set ReadLimits for untrusted input: defaults allow up to the in-memory Int32 capacity. DirectoryLimits bounds directory bytes and entry count; EntryReadLimits bounds decoded output. Range offsets and lengths of entries are 64-bit, including ZIP64 archives larger than 4 GiB. max_output_bytes bounds the final buffer returned by write; it does not bound the model or temporary encoded payloads. Cancellation is polled by read_all; range readers are cancelled by stopping between steps.

    Both readers scan the last 65,535 + 22 bytes for EOCD records whose declared comment fits in the file; an overrunning comment is rejected. Tier 1 checks all candidates ending exactly at EOF. A nonempty directory must start with PK\x01\x02 at boundary - cd_size, with a nonnegative archive start boundary - cd_size - cd_offset; the boundary is the classic EOCD or resolved ZIP64 end record. An empty directory (zero entries and size) needs no prefix gap: cd_offset == boundary. Exactly one valid candidate is used; multiple valid candidates raise Corrupt (APPNOTE 4.3.1: "MUST have only one"). If all fail, the earliest candidate's error is raised, without trying tier 2. Tier 2, only if none end at EOF, tries fitting candidates from last to earliest and uses the first passing the same checks, tolerating trailing data.

    The shared engine tries a valid, agreeing ZIP64 locator/record before classic, including during the signature probes. If the selected ZIP64 directory fails full parsing, it retries classic with the SFX gap and the same structural probe; if both fail, it reports the classic error. It does not reopen the EOCD search after full parsing fails. Entry and byte limits apply only after selection; directory limits do not cap speculative probes or temporary memory. Bound transport work separately or cancel between range steps. A uniquely plausible forgery or two ZIP64/classic readings can still be ambiguous in intent; use a trusted source boundary or external authentication when that distinction matters.

    The ten-reader survey comparison records the tested versions and warnings. In this compact table, R means real entries, 0 an empty decoy, and E rejection; warning details and the 7-Zip header-error/listing distinction are in the full table. .NET and core are inferred from the supplied sources; the other rows were replayed.

    Readera normalb garbagec zerosd decoye decoy + garbagef SFXf2 adjustedg truncated
    gaato/zip, both APIsRRRRRRRE
    Python / unzip / libarchiveRRR00RRR
    Go / Rust zipRRR00RRE
    7-ZipRRRRRERE
    JavaRRR0RRRE
    .NETRRR00ERE
    moonbitlang/coreREEEEERE
    minizip-ng CLIRRREERRR

    Failures raise ZipError(kind, message). Branch on kind: container errors include invalid records, size or CRC mismatch, unsupported features, resource limits and cancellation. Codec(kind) preserves the compression codec's gaato/codec.ErrorKind, including corrupt, truncated and trailing data. Shared or overlapping local records are refused. A listing checks declared extents; opening checks the actual header and descriptor against the next record. Error messages are diagnostic text.

    This package performs no filesystem extraction. Path safety (../, absolute paths and symlinks) is the caller's responsibility. Encryption and split archives are unsupported. Wrapped classic entry counts above 65,535 without ZIP64 are unsupported. Names must be UTF-8. Zstandard ZIPs need readers with method 93 support.

    #License

    Apache-2.0. Derived from MoonBit core's archive/zip, copyright 2026 International Digital Economy Academy. See the repository's LICENSE and NOTICE.

    ZipError

    pub suberror ZipError {
    ZipError(ZipErrorKind, String)
    } derive(
    Debug
    )

    Structured ZIP failure: a stable category plus a human-readable diagnostic.

    Show prints <kind>: <message> on one line, such as MissingEndOfCentral: zip: end of central directory not found. @debug.Debug prints the constructor form.
    impl Show for ZipError

    ZipError::to_string

    fn ZipError::to_string(self : ZipError) -> String

    Format a ZIP error with its machine-readable category and diagnostic.

    Archive

    type Archive

    An in-memory ZIP archive: an ordered list of entries plus an archive comment. Reading a ZIP materializes this; writing serializes it back.

    Archive::Archive

    fn Archive::Archive() -> Archive

    Create an empty archive.

    Archive::add

    fn Archive::add(self : Archive, name : String, data : BytesView, compression? : Compression) -> Unit

    Append a new entry, retaining its payload as Bytes. The default method is Deflate; pass Store to keep the payload verbatim. Serialization chooses the local-header and descriptor layout; use Writer for chunked input.

    Archive::comment

    fn Archive::comment(self : Archive) -> BytesView

    The archive-level comment bytes (empty when none).

    Archive::copy

    #alias(clone, deprecated="Use `Archive::copy` instead")
    fn Archive::copy(self : Archive) -> Archive

    Returns an independently mutable snapshot of this archive. The copy shares entry payloads and source storage but owns a fresh entries array, so add/replace/remove on either side never affects the other.

    Archive::entries

    fn Archive::entries(self : Archive) -> ArrayView[Entry]

    The archive's entry list in order.

    Archive::get

    fn Archive::get(self : Archive, name : StringView) -> BytesView?

    A read-only view of an entry payload, or None when absent.

    Archive::remove

    fn Archive::remove(self : Archive, name : StringView) -> Bool

    Remove the entry named name. Returns false when no such entry exists.

    Archive::replace

    fn Archive::replace(self : Archive, name : StringView, data : BytesView) -> Bool

    Replace the first entry named name, retaining its method, position, DOS date/time, creator system and external attributes. CRC is recomputed and raw records are discarded for re-encoding. Returns false if the name is absent.

    Archive::set_comment

    fn Archive::set_comment(self : Archive, comment : BytesView) -> Unit

    Replace the archive-level comment. Long comments (over 65535 bytes) are rejected at write time. Changing the comment invalidates a retained end-record template, so a later byte-preserving write regenerates the trailer with the new comment.

    Compression

    pub(all) enum Compression {
    Store
    Deflate
    Deflate64
    Zstandard
    } derive(Eq,
    Debug
    )

    ZIP compression methods. Deflate64 is read-only; writing it raises InvalidArgument.

    Compression::equal

    fn Compression::equal(Compression, Compression) -> Bool

    Compare compression methods through dot syntax.

    Directory

    type Directory

    A directory listing in central-directory order, including duplicate names. Owns metadata only. A successful listing does not validate local headers, decompress payloads, or verify their CRC-32 values.

    Directory::comment

    fn Directory::comment(self : Directory) -> BytesView

    The archive comment, owned independently of supplied read buffers.

    Directory::entries

    Entries in central-directory order. This view cannot modify the listing.

    Directory::open_entry

    fn Directory::open_entry(self : Directory, index : Int, limits? : EntryReadLimits) -> EntryReader raise ZipError

    Open an entry by directory index, preserving the identity of duplicate names. No I/O occurs here. Unknown methods, unsupported flags, encryption, and declared output exceeding the limit are rejected before requesting data.

    DirectoryEntry

    type DirectoryEntry

    Metadata declared by a central-directory entry, without its payload. Sizes remain 64-bit even when the payload cannot fit in a MoonBit Bytes. Unknown compression methods and encrypted payloads can still be listed.

    DirectoryEntry::compressed_size

    fn DirectoryEntry::compressed_size(self : DirectoryEntry) -> UInt64

    The declared compressed payload size, excluding headers and descriptors.

    DirectoryEntry::crc32

    fn DirectoryEntry::crc32(self : DirectoryEntry) -> UInt

    The declared CRC-32. Directory reading does not verify payload integrity.

    DirectoryEntry::creator_system

    fn DirectoryEntry::creator_system(self : DirectoryEntry) -> Int

    The host-system identifier from the entry's version-made-by field.

    DirectoryEntry::dos_date

    fn DirectoryEntry::dos_date(self : DirectoryEntry) -> Int

    Original packed DOS modification date (year/month/day); no timezone is stored.

    DirectoryEntry::dos_time

    fn DirectoryEntry::dos_time(self : DirectoryEntry) -> Int

    Original packed DOS modification time (hour/minute/two-second units).

    DirectoryEntry::external_attributes

    fn DirectoryEntry::external_attributes(self : DirectoryEntry) -> UInt

    Original external file attributes, interpreted according to creator_system.

    DirectoryEntry::is_directory

    fn DirectoryEntry::is_directory(self : DirectoryEntry) -> Bool

    Whether the name or DOS/Unix file attributes identify a directory.

    DirectoryEntry::is_encrypted

    fn DirectoryEntry::is_encrypted(self : DirectoryEntry) -> Bool

    Whether the entry's general-purpose flag declares payload encryption. This does not imply support for decrypting it or for encrypted directories.

    DirectoryEntry::method_id

    fn DirectoryEntry::method_id(self : DirectoryEntry) -> Int

    The ZIP compression method number, including methods this library cannot decode.

    DirectoryEntry::name

    fn DirectoryEntry::name(self : DirectoryEntry) -> String

    The entry name, decoded using the same UTF-8 policy as read_all.

    DirectoryEntry::uncompressed_size

    fn DirectoryEntry::uncompressed_size(self : DirectoryEntry) -> UInt64

    The declared uncompressed size; listing never allocates this many bytes.

    DirectoryLimits

    type DirectoryLimits

    Limits on the finally chosen directory, never declared payload sizes. Structural ZIP64/classic selection happens before these checks, so these are not limits on speculative metadata I/O or temporary selection memory. Defaults permit any directory byte size and at most 2^31 - 1 entries.

    DirectoryLimits::DirectoryLimits

    fn DirectoryLimits::DirectoryLimits(max_directory_bytes? : UInt64, max_entries? : UInt64) -> DirectoryLimits

    Construct directory bounds. A zero ceiling permits only an empty resource.

    DirectoryReader

    type DirectoryReader

    Metadata-only ZIP reader driven by caller-supplied byte ranges. Supports classic ZIP and ZIP64 beyond 2 GiB without retaining the archive or inflating payloads. Retains owned metadata plus a bounded incomplete directory record. File reads, HTTP requests, source consistency, and cancellation between calls belong to the caller. No I/O callback is retained or invoked.

    DirectoryReader::DirectoryReader

    fn DirectoryReader::DirectoryReader(archive_size : UInt64, limits? : DirectoryLimits) -> DirectoryReader

    Start reading an archive of the supplied total byte size. No bytes are read until the caller drives advance and fulfills the returned ranges.

    DirectoryReader::advance

    fn DirectoryReader::advance(self : DirectoryReader, input : BytesView) -> DirectoryStep raise ZipError

    Initially supply empty input; thereafter supply exactly the bytes requested by the preceding NeedRead. Accumulate transport short reads before calling. All ranges must refer to the same immutable archive. Input is borrowed only for this call; retained metadata and incomplete records are copied. Requests are at most 65,557 bytes, and directory chunks at most 64 KiB. Done validates directory structure and referenced bounds, not local headers or payload integrity. Empty calls after completion return the same listing. Any error is latched and rethrown on subsequent calls.

    DirectoryStep

    pub(all) enum DirectoryStep {
    NeedRead(ReadRange)
    Done(Directory)
    }

    The next range to read, or a completed, independently owned metadata listing.

    Entry

    type Entry

    A single archive entry: its decoded payload plus the metadata the central directory carried for it. Entries produced by read_all retain their source records for byte-preserving rewrites; entries built with Archive::add do not.

    Entry::compression

    fn Entry::compression(self : Entry) -> Compression

    The compression method used for this entry.

    Entry::crc32

    fn Entry::crc32(self : Entry) -> UInt

    The entry's stored CRC-32: the central-directory value for entries read from an archive, or computed at add time for entries created in memory.

    Entry::creator_system

    fn Entry::creator_system(self : Entry) -> Int

    ZIP host-system identifier; zero (DOS) for newly added entries.

    Entry::data

    fn Entry::data(self : Entry) -> BytesView

    A read-only view of the decompressed payload.

    Entry::dos_date

    fn Entry::dos_date(self : Entry) -> Int

    Packed DOS modification date; zero for newly added entries.

    Entry::dos_time

    fn Entry::dos_time(self : Entry) -> Int

    Packed DOS modification time; zero for newly added entries.

    Entry::external_attributes

    fn Entry::external_attributes(self : Entry) -> UInt

    File attributes interpreted by creator_system; zero for newly added entries.

    Entry::name

    fn Entry::name(self : Entry) -> String

    The entry name decoded as UTF-8.

    EntryReadLimits

    type EntryReadLimits

    Bounds for streaming one entry. The ceiling applies to actual decoded bytes and also rejects an oversized directory declaration before reading payloads.

    EntryReadLimits::EntryReadLimits

    fn EntryReadLimits::EntryReadLimits(max_output_bytes? : UInt64) -> EntryReadLimits

    Construct a decoded-output ceiling. Omission permits any UInt64 byte count.

    EntryReader

    type EntryReader

    Range-driven extraction of one Store or Deflate entry. Created by Directory::open_entry so metadata and archive bounds cannot be forged. Retains at most one 64 KiB compressed response, bounded local metadata, and the inflater's workspace. Sizes and cumulative progress remain 64-bit.

    EntryReader::last_produced

    fn EntryReader::last_produced(self : EntryReader) -> Int

    Output written by the latest successful step, including NeedRead and Done.

    EntryReader::step

    fn EntryReader::step(self : EntryReader, input : BytesView, output : MutArrayView[Byte]) -> EntryStep raise ZipError

    Initially supply empty input. After NeedRead, supply exactly the requested bytes from the same immutable archive used for the directory listing. After NeedMoreOutput, supply empty input and another output array; accepted input is retained internally and must not be replayed. Read last_produced after every normal return and process output[:last_produced()] before reusing it. Input and output buffers are released before returning. An empty output array suspends payload decoding. Requests are at most 65,535 bytes for local metadata and 64 KiB for payloads. Cancel by stopping between calls. Done verifies the local header, exact compressed/decoded sizes, CRC-32 and any data descriptor. Earlier output is unverified until Done. On error the output is unspecified, last_produced is zero, and the error remains latched.

    EntryReader::total_consumed

    fn EntryReader::total_consumed(self : EntryReader) -> UInt64

    Cumulative compressed payload bytes consumed, excluding headers/descriptors.

    EntryReader::total_produced

    fn EntryReader::total_produced(self : EntryReader) -> UInt64

    Cumulative decoded bytes. Integrity is established only after Done.

    EntryStep

    pub(all) enum EntryStep {
    NeedRead(ReadRange)
    NeedMoreOutput
    Done
    }

    Supply the requested range, resume with empty input and another output buffer, or finish after all integrity checks have succeeded. Every successful step may produce output, including a step returning NeedRead or Done.

    LimitKind

    pub(all) enum LimitKind {
    PackageBytes
    Entries
    EntryUncompressedBytes
    TotalUncompressedBytes
    DirectoryBytes
    OutputBytes
    } derive(Eq,
    Debug
    )

    Which bounded resource was exceeded during a bounded read or write. The payload-related kinds guard decoded allocations; DirectoryBytes bounds metadata-only directory input; OutputBytes bounds serialization size.

    LimitKind::equal

    fn LimitKind::equal(LimitKind, LimitKind) -> Bool

    Compare resource kinds through dot syntax.

    ReadLimits

    pub struct ReadLimits {
    max_package_bytes : Int
    max_entries : Int
    max_entry_uncompressed_bytes : Int
    max_total_uncompressed_bytes : Int
    }

    Resource bounds applied to a bounded read. Every field is a ceiling in bytes (or, for max_entries, a count); ReadLimits() imposes no limits beyond the in-memory Int32 capacity. max_package_bytes also bounds the complete source retained for byte-preserving rewrites.

    ReadLimits::ReadLimits

    fn ReadLimits::ReadLimits(max_package_bytes? : Int, max_entries? : Int, max_entry_uncompressed_bytes? : Int, max_total_uncompressed_bytes? : Int) -> ReadLimits

    Construct resource bounds. Omitted ceilings default to the largest Int. Each supplied ceiling must be non-negative; read_all raises InvalidArgument for a negative one and rejects an exceeded ceiling before accepting the corresponding resource.

    ReadRange

    type ReadRange

    One bounded request into the same immutable archive supplied to the reader. The offset is absolute; the length describes a single in-memory buffer.

    ReadRange::length

    fn ReadRange::length(self : ReadRange) -> Int

    Exact byte count to supply on the next call, at most 65,557 bytes.

    ReadRange::offset

    fn ReadRange::offset(self : ReadRange) -> UInt64

    Absolute byte offset from the start of the archive.

    Writer

    type Writer

    An incremental ZIP writer over a synchronous byte sink. add handles an already loaded file; begin_entry/write/end_entry handle bounded chunks. Retains central-directory records, current-entry metadata and codec buffers. The sink must consume each view before returning. A sink exception poisons the writer and is propagated unchanged; no further output can be emitted.

    Writer::Writer

    fn Writer::Writer(sink : (BytesView) -> Unit raise, level? : Int) -> Writer

    Create a ZIP writer. The sink supplies file/socket I/O or collects chunks; this package itself performs no I/O. Calling finish writes the directory. level is passed unchanged to the selected codec; 6 is the default. Writer methods raise ZipError for their own failures and propagate sink errors unchanged.

    Writer::add

    fn Writer::add(self : Writer, name : String, data : BytesView, compression? : Compression) -> Unit raise

    Compress and emit one complete file immediately using the whole-buffer codec helper. Chunked encoder caches are used by begin_entry instead. Stored files are sent directly to the sink without an intermediate copy. Read-only Deflate64 is rejected with InvalidArgument before emitting bytes.

    Writer::begin_entry

    fn Writer::begin_entry(self : Writer, name : String, compression? : Compression, size? : UInt64) -> Unit raise

    Open a chunked entry. Supply the known uncompressed size to use classic headers when it fits safely; unknown sizes reserve ZIP64 fields up front. The declared size is checked before accepting input and before closing. Read-only Deflate64 is rejected with InvalidArgument before emitting bytes.

    Writer::end_entry

    fn Writer::end_entry(self : Writer) -> Unit raise

    Finish the current entry and emit its data descriptor. The next file may then be started. One encoder per used compression method and one shared 64 KiB output buffer are retained, reset and reused until finish or failure.

    Writer::finish

    fn Writer::finish(self : Writer, comment? : BytesView) -> Unit raise

    Write the central directory and close the ZIP. Further writes are rejected. The optional archive comment must fit the ZIP 16-bit comment length.

    Writer::is_failed

    fn Writer::is_failed(self : Writer) -> Bool

    Whether an emission failure has made this writer unusable.

    Writer::is_finished

    fn Writer::is_finished(self : Writer) -> Bool

    Whether the directory and archive trailer have been successfully emitted.

    Writer::position

    fn Writer::position(self : Writer) -> UInt64

    Number of successfully emitted bytes, including headers and descriptors.

    Writer::write

    fn Writer::write(self : Writer, bytes : BytesView) -> Unit raise

    Consume a chunk of the current entry, updating CRC while its bytes are hot. Produced compressed blocks are sent to the sink before this call returns.

    ZipErrorKind

    pub(all) enum ZipErrorKind {
    Truncated
    InvalidSignature
    MissingEndOfCentral
    Corrupt
    UnsupportedCompression(Int)
    InvalidUtf8
    UnsupportedFeature
    Codec(
    ErrorKind
    )
    InvalidRead(expected~ : Int, actual~ : Int)
    InvalidEntryIndex(Int)
    SizeMismatch
    ChecksumMismatch
    LimitExceeded(kind~ : LimitKind, limit~ : UInt64, observed~ : UInt64)
    Cancelled
    InvalidArgument
    InvalidState
    } derive(Eq,
    Debug
    )

    Machine-readable category for a ZIP container failure. The accompanying ZipError message is intended for diagnostics rather than branching.

    ZipErrorKind::equal

    Compare ZIP failure categories through dot syntax.

    read_all

    fn read_all(bytes : BytesView, limits? : ReadLimits, cancelled? : () -> Bool) -> Archive raise ZipError

    Read a ZIP archive from bytes, decoding every entry. limits bounds the package size, entry count, per-entry and total decompressed bytes, and the complete source retained for byte-preserving rewrites; cancelled is polled across parsing, decoding, and CRC-32 verification. Every decoded entry must match its stored CRC-32. Failures carry a stable ZipErrorKind plus a diagnostic.

    write

    fn write(archive : Archive, preserve? : Bool, max_output_bytes? : Int, level? : Int) -> Bytes raise ZipError

    Serialize archive into a ZIP byte stream. Entry DOS date/time, creator system and external attributes are retained regardless of preserve.

    By default every entry is encoded fresh. With preserve=true, unchanged entries reuse their source records with updated offsets. A pristine archive round-trips byte-for-byte, including prefixes, gaps and directory ordering. After any edit, records are rebuilt in their respective local and directory orders; bytes outside records (such as prefixes and gaps) are not retained in the output. Replaced and newly added entries are encoded normally. level is passed unchanged to the selected codec, as the level it takes. If max_output_bytes is supplied, the output is sized before allocation and an oversized archive raises LimitExceeded(OutputBytes, ...). A negative limit or any read-only Deflate64 entry raises InvalidArgument, including when preserve=true could otherwise reuse the source records.