🥮 mooncakes.io
    Sign in

    moon-car

    Bounded CARv1 archive streaming, verification and offset indexing for MoonBit.

    car
    ipld
    archive
    cid
    Download zip
    Version
    0.1.0
    License
    Apache-2.0
    Repository
    Last updated
    17 hours ago
    Downloads
    7

    Dependencies

    #MoonCAR

    Pure MoonBit CARv1 archive reading, writing, streaming, integrity verification and exact CID offset indexing. Reusable data-exchange infrastructure for the MoonBit ecosystem. Apache-2.0 licensed.

    The archive layer is implemented here. MoonLoom provides CID, multihash, varints and hash providers; 2515050242/cbor provides CBOR. See the ecosystem audit.

    #Run from source

    Install the MoonBit stable toolchain. Tested with moonc v0.10.14+7d59c7ec9, moon 0.1.20260920.

    git clone https://github.com/CaptainK-65/moon-car.git
cd moon-car
moon update
moon check --target all --deny-warn
moon test --target all
moon build --target all
moon run examples/pack
moon run examples/stream
moon run examples/lookup
moon run cmd/mooncar --target native -- verify fixtures/carv1-basic.car

    The pure library and examples support wasm, wasm-gc, JS and native. The file CLI uses moonbitlang/async and runs on native; a C compiler is required.

    #Use as a library

    Add "CaptainK-65/moon-car@0.1.0" and "ggbond44439/moonloom@0.1.6" to module imports, then import their root packages as @car and @cid in your moon.pkg.

    let cid = @cid.cid_from_content(
      @cid.RAW_CODEC, @cid.Sha2_256, b"hello",
      @cid.sha2_provider(), @cid.Limits::default(),
    ).unwrap()
    let bytes = @car.encode(@car.Header::new([cid]), [@car.Block::new(cid, b"hello")])
    let archive = @car.Archive::decode(bytes)
    archive.verify(require_roots=true)
    let payload = archive.get(cid).data()

    Call these operations in a function that allows raise CarError. See exact signatures in the generated public interface.

    #Three complete use cases

    Use caseProcessingRunnable example
    DistributionHash objects, write archive, read and verify itmoon run examples/pack
    Transport snapshotFeed seven-byte chunks, consume and verify events, finish at EOFmoon run examples/stream
    Object lookupScan once, index full CID bytes, locate and verify a payloadmoon run examples/lookup

    Writer::start emits the header; write emits one section and its absolute offsets. The caller supplies the sink. Decoder::feed emits completed header/block events; always call finish at EOF. Parse errors poison the decoder. Events from a failing feed call are discarded. Provide bounded chunks and release consumed events.

    #File CLI

    moon run cmd/mooncar --target native -- pack bundle.car README.md
moon run cmd/mooncar --target native -- inspect bundle.car
moon run cmd/mooncar --target native -- verify bundle.car
moon run cmd/mooncar --target native -- get bundle.car CID extracted.bin
moon run cmd/mooncar --target native -- rewrite bundle.car rewritten.car

    pack creates raw blocks and makes every input a root. It does not preserve filenames or create UnixFS metadata. inspect checks structure; verify checks every occurrence and root presence; get verifies the selected payload. rewrite verifies first, preserves source order and duplicates, and requires a new output path. pack and get replace their output file.

    #Policies and budgets

    • Canonical DAG-CBOR header: exactly roots and version, ordered keys, definite lengths, minimal integers, tag 42 with identity-prefixed CID.
    • CARv1 only; CIDv0 and CIDv1; canonical unsigned-varint section framing.
    • Empty root arrays, header-only archives, empty payloads and duplicates are allowed.
    • Defaults: 1 MiB header, 16 MiB block section including CID, 1024 roots, 1,000,000 sections, 256 MiB archive, 1024-byte CID digests.
    • Declared budgets are checked before payload allocation. Limits::new accepts custom positive budgets; root/block counts may be zero. Offsets use MoonBit Int.
    • Parsing, hash integrity and root presence are separate checks. None proves transitive DAG closure. Unsupported hash algorithms fail verification.
    • Same ordered inputs yield the same output; universal canonical graph ordering is not claimed.

    Decoder retains one section plus prefix capacity; returned events also own payloads. Feeding an entire archive in one call may retain that call's payloads. Archive retains the original bytes plus CID/offset metadata and materializes requested blocks. The convenience encoder and CLI use bounded whole-archive memory.

    #Evidence and scope

    Independent IPLD fixture, exact offsets, byte-for-byte rewrite, all split positions, malformed headers, overflow, budget boundaries, duplicate corruption and binary payload/chunk matrices. See acceptance and design.

    Version 0.1 excludes CARv2, persistent storage/indexes, UnixFS and DAG closure traversal. The October competition proposal must be written by the participant.

    CarError

    source
    pub(all) suberror CarError {
      InvalidHeader(String)
      InvalidCid(String)
      InvalidVarint(Int)
      LimitExceeded(String, UInt64, UInt64)
      Truncated(Int)
      InvalidState(String)
      Integrity(Int, String)
      MissingRoot(String)
      NotFound
    } derive(
    Debug    )

    CarError::to_repr

    source

    Archive

    source
    pub struct Archive {
      bytes : Bytes
      header : Header
      entries : Array[Entry]
      by_cid : Map[Bytes, Array[Entry]]
      limits : Limits
    }

    Validated CAR bytes with an exact full-CID offset index. Duplicate sections remain visible in source order. No payloads are retained in the index.

    Archive::block_at

    source
    fn Archive::block_at(self : Archive, index : Int) -> Block raise CarError

    Read a section by source-order index, including individual duplicates.

    Archive::bytes

    source
    fn Archive::bytes(self : Archive) -> Bytes

    Archive::decode

    source
    fn Archive::decode(bytes : Bytes, limits? : Limits) -> Archive raise CarError

    Archive::entries

    source
    fn Archive::entries(self : Archive) -> Array[Entry]

    Archive::get

    source

    Return the first section for an exact CID. Hash integrity is checked separately.

    Archive::get_all

    source
    fn Archive::get_all(self : Archive, cid : 
    Cid    ) -> Array[Block] raise CarError

    Archive::header

    source
    fn Archive::header(self : Archive) -> Header

    Archive::length

    source
    fn Archive::length(self : Archive) -> Int

    Archive::locations

    source
    fn Archive::locations(self : Archive, cid : 
    Cid    ) -> Array[Location]

    Archive::require_roots

    source
    fn Archive::require_roots(self : Archive) -> Unit raise CarError

    Check root block presence only. This does not prove transitive DAG closure.

    Archive::verify

    source
    fn Archive::verify(self : Archive, provider? : 
    HashProvider    , require_roots? : Bool) -> Unit raise CarError

    Verifies every section, including all duplicate occurrences. A matching earlier duplicate must never hide a corrupt later duplicate. Root presence is optional.

    Block

    source
    pub struct Block {
      cid : 
    Cid
      data : Bytes
    }

    Block::cid

    source

    Block::data

    source
    fn Block::data(self : Block) -> Bytes

    Block::encode

    source
    fn Block::encode(self : Block, limits? : Limits) -> Bytes raise CarError

    Encode a single CAR block section; this does not assert hash validity.

    Block::new

    source
    fn Block::new(cid : 
    Cid    , data : Bytes) -> Block

    Block::verify

    source
    fn Block::verify(self : Block, provider? : 
    HashProvider    , limits? : Limits) -> Unit raise CarError

    Verify a block against its CID using a caller-selected hash provider.

    Decoder

    source
    pub struct Decoder {
      limits : Limits
      prefix : 
    Buffer
      body : 
    Buffer
      stage : Stage
      header_seen : Bool
      position : Int
      section_start : Int
      body_start : Int
      count : Int
      closed : Bool
      failed : Bool
    }

    Incremental parser retaining at most one bounded section plus a ten-byte prefix. feed returns completed events only; finish is mandatory to detect truncation. After a parse error the decoder is poisoned and cannot be resumed.

    Decoder::feed

    source
    fn Decoder::feed(self : Decoder, chunk : BytesView) -> Array[Event] raise CarError

    Decoder::finish

    source
    fn Decoder::finish(self : Decoder) -> Unit raise CarError

    Decoder::new

    source
    fn Decoder::new(limits? : Limits) -> Decoder

    Decoder::position

    source
    fn Decoder::position(self : Decoder) -> Int

    Entry

    source
    pub struct Entry {
      cid : 
    Cid
      location : Location
    }

    Entry::cid

    source

    Entry::location

    source
    fn Entry::location(self : Entry) -> Location

    Event

    source
    pub(all) enum Event {
      Header(Header)
      Block(Block, Location)
    }

    pub struct Header {
      roots : Array[
    Cid    ]
    }

    Header::encode

    source
    fn Header::encode(self : Header, limits? : Limits) -> Bytes raise CarError

    Encode a canonical CARv1 header including its unsigned-varint length prefix.

    Header::new

    source

    Header::roots

    source

    Limits

    source
    pub struct Limits {
      max_header : Int
      max_block : Int
      max_roots : Int
      max_blocks : Int
      max_archive : Int
    }

    Parser budgets apply before allocating a declared payload.

    Limits::default

    source
    fn Limits::default() -> Limits

    Limits::new

    source
    fn Limits::new(max_header? : Int, max_block? : Int, max_roots? : Int, max_blocks? : Int, max_archive? : Int) -> Limits raise CarError

    Location

    source
    pub(all) struct Location {
      section_offset : Int
      data_offset : Int
      data_length : Int
      section_length : Int
    } derive(
    Debug    )

    Absolute offsets into the CARv1 bytes, including the header and framing.

    Location::to_repr

    source

    Stage

    source
    type Stage

    Writer

    source
    pub struct Writer {
      limits : Limits
      header : Header
      position : Int
      count : Int
      started : Bool
    }

    Produces one framed section per call. The caller owns the output sink. Errors do not advance the writer; successfully emitted sections are immutable.

    Writer::new

    source
    fn Writer::new(header : Header, limits? : Limits) -> Writer

    Writer::position

    source
    fn Writer::position(self : Writer) -> Int

    Writer::start

    source
    fn Writer::start(self : Writer) -> Bytes raise CarError

    Writer::write

    source
    fn Writer::write(self : Writer, block : Block) -> (Bytes, Location) raise CarError

    encode

    source
    fn encode(header : Header, blocks : Array[Block], limits? : Limits) -> Bytes raise CarError

    Write blocks in caller order. Equal roots and block order yield equal bytes.

    Source Files

    Powered by MoonBit

    Site sourceReport issuePackagesBuild queueSkillsStatistics

    © 2026 mooncakes.io