moonvar

    Fixed-width and variable-length integer encodings for MoonBit.

    varint
    varint-encoding
    encoder
    decoder
    parser
    wire-format
    serialization
    binary-serialization
    variable-length-integer
    leb128
    uleb128
    sleb128
    signed-leb128
    dwarf
    webassembly
    wasm
    zigzag
    protobuf
    hpack
    qpack
    rfc-7541
    prefix-integer
    quic
    quic-varint
    rfc-9000
    http3
    lenenc
    length-encoded-integer
    mysql
    mysql-protocol
    grpc
    endian
    endianness
    big-endian
    little-endian
    byte-order
    network-byte-order
    fixed-width
    binary
    moonbit
    Download zip
    Version
    0.3.0
    License
    Apache-2.0
    Last updated
    12 hours ago
    Downloads
    8K
    moonvar


    // The fixed widths, either way round, reading nothing it was not given.
    let buf = Buffer()
    @fixed.write_u32(buf, 0x12345678)
    @fixed.write_u16(buf, 0x1234, order=Little)
    let wire = buf.to_bytes()
    assert_eq(wire, b"\x12\x34\x56\x78\x34\x12")
    assert_eq(@fixed.read_u32(wire[:]), Some(305419896))
    assert_eq(@fixed.read_u32(wire[:], at=10), None) // the view is shorter

    // A number the format gives no width to: the fewest octets that hold it.
    assert_eq(@fixed.encode(1337UL), b"\x05\x39")
    assert_eq(@fixed.decode(b"\x05\x39"[:]), Some(1337))

    // One number, in each protocol's own spelling.
    assert_eq(@quic.encode(1337UL), b"\x45\x39")
    assert_eq(@prefix.encode(1337, prefix=5), b"\x1f\x9a\x0a")
    assert_eq(@leb128.encode(1337UL), b"\xb9\x0a")
    assert_eq(@lenenc.encode(1337UL), b"\xfc\x39\x05")

    // Signed values have two spellings, and they are not the same octets: protobuf folds the
    // sign onto an unsigned value, DWARF and WebAssembly keep it in the last group.
    assert_eq(@leb128.encode(@leb128.zigzag(-2L)), b"\x03")
    assert_eq(@leb128.encode_signed(-2L), b"\x7e")

    #Packages

    EncodingWritten by
    fixedByte order for u8…u64, i8…i64, f32, f64, the 24-bit length protocols use, a width carried in a variable, and the fewest octets a value fitsEverything
    quicTwo bits name the width, sixty-two carry the value — RFC 9000 §16QUIC, HTTP/3
    prefixA value in the low N bits, continuing when it does not fit — RFC 7541 §5.1HPACK, QPACK
    leb128Seven bits an octet, least significant first; zigzag for protobuf's signed values, and the signed form DWARF and WebAssembly writeprotobuf, DWARF, WebAssembly
    lenencOne octet below 251, otherwise a marker and two, three or eight little-endian octetsMySQL, MariaDB

    Each package is separate and carries no dependency, so wanting protobuf's varint does not mean linking QUIC's.

    #The same number, five ways

    1337, as each package writes it:

    fixedquicprefix(5)leb128lenenc
    05 3945 391f 9a 0ab9 0afc 39 05

    Two octets, two, three, two, three — and each one is the length some wire already expects. quic spends two bits on the width; prefix spends five on being a value that shares an octet with flags; leb128 spends one bit per octet on saying "another follows"; lenenc spends a whole octet on a marker; fixed spends nothing and pays for it with width.

    #Reading is half of it

    • Nothing read past the end. Every fixed reader checks the width it needs and answers None when the view is short. The variable-length readers raise a Refused that names the fault: Truncated, or past the ceiling the encoding allows.
    • A width without writing it. Every package has size(): how many octets a value takes, without producing them. A length field has to be measured before the payload it describes is written.
    • Signed values stay short. leb128.zigzag folds a negative onto an unsigned value, so -1 costs one octet rather than ten. leb128.encode_signed is the other spelling: the sign in the last group, which is what DWARF and WebAssembly write.
    • A value with no width agreed. fixed.encode writes the fewest octets that hold a number and fixed.decode reads a run as one — the pair for a length the format does not put a width in front of.

    #Install

    moon add moonbitstack/moonvar

    Then import the package whose wire you speak:

    import {
    "moonbitstack/moonvar/quic",
    }

    #Examples

    src/examples/tourThe whole surface in one run: every encoding, both directions, and the octet count each costs
    src/examples/frameA length-prefixed frame — a varint length, then a payload — with the truncated case refused rather than half-read

    moon run src/examples/tour moon run src/examples/frame

    #Licence

    Apache-2.0. See LICENSE.