Sign in

    zipc

    Typed, in-memory ZIP archive library for MoonBit (deflate via gmlewis/flate)

    zip
    archive
    compression
    deflate
    Download zip
    Version
    0.2.0
    License
    Apache-2.0
    Last updated
    4 months ago
    Downloads
    2K

    #zipc

    A typed, in-memory ZIP archive library for MoonBit, ported in spirit from the OCaml zipc library.

    Its focus is the archive model, not the codec: a value-oriented Archive / Member / File API that preserves per-entry Unix mode and modification time, produces deterministic (byte-stable) output, and verifies CRC-32 on every decode. The DEFLATE compression itself is delegated to the mature gmlewis/flate (a port of Go's compress/flate, with real LZ77 and fixed/dynamic Huffman blocks).

    Verified interoperable with the reference C zlib/gzip and with unzip / Python's zipfile in both directions (see the tests). The examples below use the same APIs the test suite exercises.

    #Scope

    In scope

    • A pure, in-memory ZIP archive value type: Archive / Member / File with add / find / mem / remove / each / to_array.
    • Per-entry Unix mode and mtime preservation.
    • Deterministic serialization (members written in path order).
    • Stored and Deflate methods, produced and extracted natively, with CRC-32 verified on decode.
    • Byte-native throughout (Bytes in, Bytes out) — no lossy String round-trips.

    Non-goals (use a dedicated library if you need these)

    • ZIP64 — archives are limited to 65535 members and ~2 GiB per entry (ZIP32). For larger archives, see hustcer/fzip.
    • Encryption, multi-part / split archives, and streaming (the whole archive is held in memory for encode/decode).
    • Tunable compression levels — the backend compresses at a single fast level. Choose per file between File::stored_of_bytes (no compression) and File::deflate_of_bytes (compressed).
    • Being a general codec library. Only raw DEFLATE (deflate/inflate, the format ZIP entries use) is exposed, as a thin Bytes -> Bytes convenience. For zlib, gzip, streaming or dictionaries, use gmlewis/flate, gmlewis/zlib, gmlewis/gzip directly.

    Entries that use methods other than Stored/Deflate round-trip as opaque Compression::Other payloads (they are preserved but not decoded).

    #Install

    moon add moonbit-community/zipc

    import { "moonbit-community/zipc", "moonbit-community/zipc/deflate", // only if you want the codec convenience }

    #Archives — moonbit-community/zipc

    Build an archive of members (files and directories), serialise it to ZIP bytes, and parse it back. Unix mode and modification time are preserved.

    ///|
    test {
    // build
    let archive = @zipc.Archive::empty()
    archive.add(
    @zipc.Member::make(
    "readme.txt",
    File(@zipc.File::stored_of_bytes(b"hello")),
    ),
    )
    |> ignore
    archive.add(
    @zipc.Member::make(
    "src/data.bin",
    File(
    @zipc.File::deflate_of_bytes(b"compress me, compress me, compress me"),
    ),
    mode=0o644,
    ),
    )
    |> ignore
    let bytes = archive.to_bytes()

    // parse and extract
    let parsed = @zipc.Archive::of_bytes(bytes)
    guard parsed.find("readme.txt").unwrap().kind() is File(f)
    assert_eq(f.to_bytes(), b"hello")
    }

    File::deflate_of_bytes automatically falls back to Stored when DEFLATE would not shrink the data. File::to_bytes verifies the entry's CRC-32 and size.

    #Codec convenience — moonbit-community/zipc/deflate

    A thin Bytes -> Bytes layer over the backend for raw DEFLATE (RFC 1951) — the format ZIP entries use. inflate raises DeflateError on malformed input. For zlib, gzip, streaming or dictionaries, use gmlewis/* directly.

    ///|
    test {
    let data = b"the quick brown fox the quick brown fox"
    assert_eq(@deflate.inflate(@deflate.deflate(data)), data)
    }

    #Compared to other MoonBit options

    • gmlewis/flate / gmlewis/gzip / gmlewis/zlib are codecs only — no ZIP archive layer. This library builds on gmlewis/flate rather than competing with it.
    • moonbitlang/async/gzip is a gzip stream transform for HTTP content-encoding; it is not a ZIP archive tool.
    • hustcer/fzip (an fflate port) is the broader toolkit: more formats, streaming, ZIP64 reading, more mature. If you just want zip_sync(files) -> bytes, prefer it.

    What this library offers that the others don't is the typed, deterministic, metadata-preserving archive model — the right choice when you need reproducible archives or faithful filesystem round-tripping (permissions and timestamps), rather than just packing a bag of bytes.

    #Dependencies

    gmlewis/flate and gmlewis/io (Apache-2.0) for the DEFLATE codec. Checksums use this module's own crc32 package.

    #License

    Apache-2.0. See LICENSE.

    ZipError

    pub suberror ZipError {
    ZipError(String)
    }

    impl Show for ZipError

    Archive

    pub struct Archive {
    members : Map[String, Member]
    }

    An in-memory ZIP archive: a set of members keyed by path.

    Archive::add

    fn Archive::add(self : Archive, mem : Member) -> Archive

    Add (or replace) a member, keyed by its (already normalised) path. Mutates and returns the archive for chaining.

    Archive::each

    fn Archive::each(self : Archive, f : (Member) -> Unit) -> Unit

    Apply f to each member, in path order.

    Archive::empty

    fn Archive::empty() -> Archive

    Archive::find

    fn Archive::find(self : Archive, path : String) -> Member?

    Look up the member at path.

    Archive::is_empty

    fn Archive::is_empty(self : Archive) -> Bool

    Archive::mem

    fn Archive::mem(self : Archive, path : String) -> Bool

    Whether a member with path exists.

    Archive::member_count

    fn Archive::member_count(self : Archive) -> Int

    Archive::of_bytes

    fn Archive::of_bytes(data : Bytes) -> Archive raise ZipError

    Parse ZIP bytes into an archive. Raises ZipError on malformed input.

    Archive::remove

    fn Archive::remove(self : Archive, path : String) -> Archive

    Remove the member at path, if present.

    Archive::to_array

    fn Archive::to_array(self : Archive) -> Array[Member]

    The members, sorted by path (so serialisation is deterministic).

    Archive::to_bytes

    fn Archive::to_bytes(self : Archive) -> Bytes raise ZipError

    Serialise the archive to ZIP bytes. Members are written in path order, so the output is deterministic. Parse it back with Archive::of_bytes.

    Example:

    test {
    let archive = @zipc.Archive::empty()
    archive.add(
    @zipc.Member::make("hi.txt", File(@zipc.File::stored_of_bytes(b"hi"))),
    )
    |> ignore
    let bytes = archive.to_bytes()
    let parsed = @zipc.Archive::of_bytes(bytes)
    guard parsed.find("hi.txt").unwrap().kind() is File(f)
    assert_eq(f.to_bytes(), b"hi")
    }

    Compression

    pub enum Compression {
    Stored
    Deflate
    Other(Int)
    } derive(Eq)

    impl Show for Compression

    File

    pub struct File {
    compression : Compression
    data : Bytes
    decompressed_size : Int
    decompressed_crc32 : Int
    }

    A file's data as it is stored in the archive.

    data is the on-disk payload: the raw bytes for Stored, or a raw DEFLATE stream for Deflate. decompressed_size and decompressed_crc32 describe the original content.

    File::can_extract

    fn File::can_extract(self : File) -> Bool

    Whether this file can be decompressed natively (Stored or Deflate).

    File::compressed_size

    fn File::compressed_size(self : File) -> Int

    Size of the stored (possibly compressed) payload, in bytes.

    File::compression

    fn File::compression(self : File) -> Compression

    File::decompressed_crc32

    fn File::decompressed_crc32(self : File) -> Int

    File::decompressed_size

    fn File::decompressed_size(self : File) -> Int

    File::deflate_of_bytes

    fn File::deflate_of_bytes(data : Bytes) -> File raise ZipError

    Create a Deflate-compressed file from data. If compression does not shrink the data (e.g. already-compressed or tiny inputs), the file is stored uncompressed instead.

    Example:

    test {
    let file = @zipc.File::deflate_of_bytes(b"aaaaaaaaaaaaaaaaaaaaaaaaaaaa")
    inspect(file.compression(), content="Deflate")
    assert_eq(file.to_bytes(), b"aaaaaaaaaaaaaaaaaaaaaaaaaaaa")
    }

    File::stored_of_bytes

    fn File::stored_of_bytes(data : Bytes) -> File raise ZipError

    Create a Stored (uncompressed) file from data.

    File::to_bytes

    fn File::to_bytes(self : File) -> Bytes raise

    Decompress the file back to its original bytes, verifying the CRC-32. Raises ZipError for unsupported methods or on a CRC / size mismatch.

    Member

    pub struct Member {
    path : String
    mode : Int
    mtime : Int
    kind : MemberKind
    }

    An archive member: a path plus its kind, Unix mode and modification time (Unix seconds).

    Member::is_dir

    fn Member::is_dir(self : Member) -> Bool

    Whether this member is a directory.

    Member::kind

    fn Member::kind(self : Member) -> MemberKind

    Member::make

    fn Member::make(path : String, kind : MemberKind, mode? : Int, mtime? : Int) -> Member raise ZipError

    Build a member. mode defaults to 0o755 for directories and 0o644 for files; mtime defaults to the DOS epoch. Raises ZipError if the path is too long.

    Member::mode

    fn Member::mode(self : Member) -> Int

    Member::mtime

    fn Member::mtime(self : Member) -> Int

    Member::path

    fn Member::path(self : Member) -> String

    MemberKind

    pub(all) enum MemberKind {
    Dir
    File(File)
    }

    bytes_has_magic

    fn bytes_has_magic(data : Bytes) -> Bool

    Whether data starts with a ZIP local-file-header or empty-archive EOCD signature.

    dos_datetime_components

    fn dos_datetime_components(dos_time : Int, dos_date : Int) -> (Int, Int, Int, Int, Int, Int)

    dos_datetime_to_unix

    fn dos_datetime_to_unix(dos_time : Int, dos_date : Int) -> Int

    dos_epoch

    let dos_epoch : Int

    Unix timestamp of the DOS epoch (1980-01-01 00:00:00 UTC), the earliest time representable in a ZIP entry.

    format_dos_datetime

    fn format_dos_datetime(dos_time : Int, dos_date : Int) -> String

    make_dos_datetime

    fn make_dos_datetime(year : Int, month : Int, day : Int, hour : Int, minute : Int, second : Int) -> (Int, Int)

    unix_to_dos_datetime

    fn unix_to_dos_datetime(unix_timestamp : Int) -> (Int, Int)