#fzip – High-performance compression library for MoonBit

• Deflate, GZIP, Zlib, and ZIP compression and decompression
• Synchronous APIs plus streaming APIs for DEFLATE/GZIP/Zlib
• ZIP64 metadata support for in-memory ZIP archives that still fit the sync API limits
• Validation and size limits for malformed input, zip bombs, path traversal, and corrupted data

#Benchmark

#Features

• Pure MoonBit implementation with no external dependencies (compiled to Wasm-GC)
• Support for DEFLATE, GZIP, Zlib, and ZIP formats
• Automatic format detection for decompression
• Streaming APIs for chunk-based DEFLATE/GZIP/Zlib compression and decompression
• In-memory ZIP read/write support, including ZIP64 metadata, data descriptors, listing, and CRC-32 validation
• Configurable input and output limits
• 360+ tests covering format correctness, edge cases, security scenarios, and malformed input

#Installation

moon add hustcer/fzip

{
  "deps": {
    "hustcer/fzip": "0.8.2"
  }
}

#Quick Start

// Compress (GZIP)
let data = @fzip.str_to_u8("Hello, MoonBit!")
let compressed = @fzip.gzip_sync(data)

// Decompress (auto-detect format)
let original = @fzip.decompress_sync(compressed)
let text = @fzip.str_from_u8(original)

println(text) // "Hello, MoonBit!"

#Current Status

#Working Features

• DEFLATE compression/decompression - Full deflate algorithm implementation
• GZIP format support - Deflate with GZIP headers, timestamps, metadata, and CRC-32 checksums
• Zlib format support - Deflate with Zlib headers and Adler-32 checksums
• ZIP archive support - Write, list, and extract ZIP archives
• ZIP64 metadata support - Read and write ZIP64 metadata for archives that fit the current in-memory sync API limits
• ZIP data descriptors - Read entries that use central-directory sizes and CRC-32
• Streaming compression - Stream-based handlers for chunk-based data processing
• Input validation - Size limits, path traversal detection, checksum validation, and malformed metadata rejection
• Auto-detection - Decompression that recognizes GZIP, Zlib, or raw DEFLATE
• Test coverage - Tests for encoding, decoding, edge cases, and security-related regressions

#API Compatibility

• Type-safe synchronous and stream-based representations (ondata callbacks).
• Error propagation through MoonBit's raise Error mechanisms.
• Zero-dependency string encoding/decoding implementations built-in.

#Detailed API

#ZIP

// Create ZIP archive
let files : Array[(String, FixedArray[Byte])] = [
  ("hello.txt", @fzip.str_to_u8("Hello!")),
  ("binary.dat", my_bytes),
]
let archive = @fzip.zip_sync(files)

// Extract ZIP archive
let extracted = @fzip.unzip_sync(archive)
for entry in extracted {
  let (filename, content) = entry
  println("\{filename}: \{content.length()} bytes")
}

// List files without extracting
let infos = @fzip.unzip_list(archive)
for info in infos {
  println("\{info.name}: original size \{info.original_size} (compressed: \{info.size})")
}

// Extract ZIP archive and verify each entry CRC-32
let checked = @fzip.unzip_sync(archive, opts={ verify_checksum: true })

#ZIP64 metadata support

• entry compressed or uncompressed size reaches 0xFFFFFFFF
• entry local-header offset reaches 0xFFFFFFFF
• entry count reaches 0xFFFF
• central directory size or offset reaches 0xFFFFFFFF

let archive = @fzip.zip_sync_checked(files) catch {
  FzipError(code~, message~) => {
    // Handle Zip64ValueTooLarge, ExtraFieldTooLong,
    // FilenameTooLong, InvalidZipData, or another writer error.
    return
  }
}

#Automatic Decompression

// compress_sync defaults to gzip_sync
let compressed = @fzip.compress_sync(data)

// decompress_sync automatically detects GZIP, Zlib, or DEFLATE
let original = @fzip.decompress_sync(compressed)

#Streaming

let stream = @fzip.DeflateStream::new()
stream.ondata = Some(FlateStreamHandler(fn(data, is_final) {
  // handle output chunk
}))
stream.push(chunk1)
stream.push(chunk2, final_=true)

#Security Features

• Size limits: Configurable max output (default 100MB) and input (default 1GB) sizes; ZIP extraction also caps total sync output and entry fan-out
• Checksum verification:
  • GZIP and Zlib verify their container checksums by default
  • ZIP entry CRC-32 verification is opt-in via unzip_sync(..., opts={ verify_checksum: true }); this preserves the historical fast ZIP extraction path and avoids an extra checksum pass when callers already trust the archive source
• Compression ratio check: ZIP files with compression ratios > 1000:1 are rejected
• Path traversal protection: ZIP entries with unsafe paths (../, absolute paths) are rejected
• ZIP metadata validation: ZIP filenames and extra fields are length-limited; encrypted and multi-disk ZIP archives are rejected
• Data descriptor support: ZIP entries with general-purpose bit 3 are bounded by central-directory sizes and verified with CRC-32
• Strict UTF-8 decoding: malformed UTF-8 is rejected unless Latin-1 decoding is explicitly requested

// With all security features (default)
let original = @fzip.gunzip_sync(compressed, opts={
  out: None,
  dictionary: None,
  max_output_size: 10485760,   // 10MB limit
  max_input_size: 104857600,   // 100MB limit
  verify_checksum: true,        // Verify CRC-32 (default)
})

// Skip checksum verification when integrity is checked elsewhere
let original = @fzip.gunzip_sync(compressed, opts={
  verify_checksum: false,       // Skip CRC-32 for ~4x faster decompression
  ..
})

#Development

moon check              # Type check
moon build              # Full build
moon test               # Run tests
moon test -v            # Verbose output
moon fmt                # Format code
moon bench              # Run benchmarks

#License