moonbase

    Encodings for MoonBit: base16, base32, base36, base58, base62 and base64, each with the same five faces — encode, encode_bytes, decode, decode_bytes and decode_lossy — and failures that name where they happened.

    base16
    base32
    base36
    base58
    base62
    base64
    Download zip
    Version
    0.4.0
    License
    Apache-2.0
    Last updated
    49 minutes ago
    Downloads
    9

    #moonbase

    check and test

    Encodings for MoonBit: base16, base32, base36, base58, base62 and base64.

    Six encodings, one shape. Every package has the same five functions in the same order, so reading one is learning all of them:

    @base64.encode(data[:]) // "bW9vbmJhc2U="
    @base64.encode(data[:], kind=Url, padding=false) // "bW9vbmJhc2U"
    @base64.decode("bW9vbmJhc2U=") // raises on bad input
    @base64.decode_lossy(pem_body) // skips newlines, never fails
    @base64.encode_bytes(data[:]) // ASCII bytes, no String on the way

    #Install

    { "deps": { "moonbitstack/moonbase": "0.4.0" } }

    Then import only what you use — an application that needs base64 does not link base58's big-integer core:

    { "import": ["moonbitstack/moonbase/base64"] }

    #The five faces

    FaceFor
    encode(input : BytesView, …) -> Stringthe ordinary case
    encode_bytes(input : BytesView, …) -> Byteswire formats that hold bytes — an HTTP header, a gRPC metadata value — without a round trip through String
    decode(input : StringView, …) -> Bytes raise Malformedinput you want checked
    decode_bytes(input : BytesView, …) -> Bytes raise Malformedthe same, for callers who never had a string
    decode_lossy(input : StringView, …) -> Bytesa PEM body full of newlines, a fingerprint full of colons, a token with spaces in it

    Variants are a named argument, not another function name:

    @base32.encode(data[:], kind=Hex) // RFC 4648 §7, sort-order preserving
    @base16.encode(data[:], kind=Upper) // 666F6F
    @base64.encode(data[:], padding=false) // no trailing '='

    base36, base58 and base62 add two more for whole numbers, where leading zeros have no meaning:

    @base62.encode_int(123456789) // "8M0kX"
    @base62.decode_int("8M0kX") // 123456789

    #Failures say where

    try @base64.decode("Zm*vYg==") catch { Bad(at~, char~) => … // at = 2, char = '*' Truncated(at~) => … // the input ends mid-group Padding(at~) => … // '=' where it cannot be Checksum(at~) => … // for the encodings that carry one }

    Want the old "just tell me it failed"? try? @base64.decode(s) gives you a Result back.

    #What each encoding is for

    PackageSpecificationNotes
    base16RFC 4648 §8byte-aligned, so leading zeros survive; encode_colons writes fingerprints
    base32§6, §7kind=Hex keeps byte order under sorting
    base36multibase k/Knumbers and identifiers
    base58Bitcoinno 0, O, I or l; leading zero bytes stay as leading 1
    base62digits, upper, lower; short identifiers
    base64§4, §5kind=Url for tokens and file names; decode_any reads either alphabet

    #Custom alphabets

    The big-integer core is public, so any positional system is two lines:

    let crockford = @moonbase.Alphabet::new("0123456789ABCDEFGHJKMNPQRSTVWXYZ")
    @moonbase.encode(data[:], crockford)

    Alphabet::new raises when the alphabet is wrong (repeated character, not ASCII, fewer than two digits) — it often comes from configuration. Alphabet::of is the same thing for literals in your own source, and aborts instead.

    #Examples and tests

    examples/tour prints every face, including what a failure looks like:

    moon run examples/tour --target wasm-gc

    54 tests on all four backends, pinned to the authoritative vectors: RFC 4648 §10 for base16, base32 and base64; Bitcoin Core's base58_encode_decode.json for base58; JavaScript's toString(36) for base36; hand-computed values for base62.

    moon check --target all --deny-warn moon test --target all

    #License

    Apache-2.0.

    BadAlphabet

    pub(all) suberror BadAlphabet {
    NotAscii(at~ : Int, char~ : Char)
    Repeated(at~ : Int, char~ : Char)
    TooFew(count~ : Int)
    } derive(Eq,
    Debug
    )

    Why an alphabet could not be built.

    Separate from [Malformed] on purpose: a malformed input is the caller's data, a malformed alphabet is the caller's code.

    BadAlphabet::equal

    fn BadAlphabet::equal(BadAlphabet, BadAlphabet) -> Bool

    BadAlphabet::not_equal

    fn BadAlphabet::not_equal(x : BadAlphabet, y : BadAlphabet) -> Bool

    Malformed

    pub(all) suberror Malformed {
    Bad(at~ : Int, char~ : Char)
    Truncated(at~ : Int)
    Padding(at~ : Int)
    Checksum(at~ : Int)
    } derive(Eq,
    Debug
    )

    Why a string could not be decoded, and where.

    Decoding fails for four reasons and each of them names the offending index, because "it did not decode" is not enough to fix a malformed document. The old spelling returned None and said nothing; callers who only want that much can still write try? decode(s).

    Malformed::equal

    fn Malformed::equal(Malformed, Malformed) -> Bool

    Malformed::not_equal

    fn Malformed::not_equal(x : Malformed, y : Malformed) -> Bool

    Alphabet

    pub struct Alphabet {
    digits : Array[Char]
    lookup : Array[Int]
    base : Int
    }

    An ordered set of digit characters defining a positional numeral system. The character at index i is the digit of value i, so the first character is always the zero digit. Build one with [Alphabet::new] and hand it to [encode] / [decode]; the base58 and base62 packages are thin wrappers that pin a specific alphabet.

    Alphabet::base

    fn Alphabet::base(self : Alphabet) -> Int

    The radix of this alphabet — the number of distinct digits.

    Alphabet::char_of

    fn Alphabet::char_of(self : Alphabet, v : Int) -> Char

    The character for digit value v, where 0 <= v < base.

    Alphabet::new

    fn Alphabet::new(chars : String) -> Alphabet raise BadAlphabet

    Build an [Alphabet] from its ordered digit characters, e.g. Alphabet::new("0123456789abcdef") for lowercase hexadecimal.

    Fails rather than aborts: an alphabet often comes from configuration, and a library has no business killing the program over a value it was handed.

    Alphabet::of

    fn Alphabet::of(chars : String) -> Alphabet

    An alphabet written as a literal in source, where a mistake is a programming error rather than bad input.

    This is the one place the library still aborts: new exists for alphabets that arrive at runtime, and this one for the six that ship with the library and are covered by its tests.

    Alphabet::value_of

    fn Alphabet::value_of(self : Alphabet, c : Char) -> Int?

    The digit value of c in this alphabet, or None if c is not one of its characters.

    ascii

    fn ascii(text : String) -> Bytes

    A string of ASCII, as the bytes it is written with.

    decode

    fn decode(input : StringView, alphabet : Alphabet) -> Bytes raise Malformed

    Decode a string over alphabet back to the original bytes.

    Fails at the first character outside the alphabet, naming where it was. Round-trips with [encode].

    decode_bytes

    fn decode_bytes(input : BytesView, alphabet : Alphabet) -> Bytes raise Malformed

    Decode from ASCII bytes, for callers who never had a string.

    decode_int

    fn decode_int(input : StringView, alphabet : Alphabet) -> UInt64 raise Malformed

    Read a whole number in this base.

    decode_lossy

    fn decode_lossy(input : StringView, alphabet : Alphabet) -> Bytes

    Decode, skipping anything outside the alphabet.

    Never fails, which is what a payload wrapped in newlines or punctuation needs. What it cannot do is tell you the input was wrong.

    encode

    fn encode(input : BytesView, alphabet : Alphabet) -> String

    Encode raw bytes to a string over alphabet.

    Leading zero bytes map to leading zero-digit characters; the rest is read as a big-endian integer and rewritten in the target base. Round-trips with [decode].

    encode_bytes

    fn encode_bytes(input : BytesView, alphabet : Alphabet) -> Bytes

    The same encoding, as the ASCII bytes it would be written with.

    encode_int

    fn encode_int(value : UInt64, alphabet : Alphabet) -> String

    Write a whole number in this base, without the leading-zero rule that the byte-oriented faces keep.

    lookup_of

    fn lookup_of(chars : String) -> Array[Int]

    The 128-entry table that turns a character into its digit value.

    lookup_of_both

    fn lookup_of_both(lower : String, upper : String) -> Array[Int]

    The same table, accepting either case, for the encodings whose decoders do.

    pack

    fn pack(input : BytesView, bits : Int, digits : Array[Char], pad : Bool) -> String

    The bit packer behind base16, base32 and base64.

    The three differ in one number — how many bits a character carries — and in their alphabets. Writing the packing once means a fix to the padding rules or the error positions lands in all three at once, and it is why each of those packages is a table and five one-line functions.

    pack_bytes

    fn pack_bytes(input : BytesView, bits : Int, digits : Array[Char], pad : Bool) -> Bytes

    The same packing, as the ASCII bytes it would be written with.

    Wire formats — an HTTP header, a gRPC metadata value — hold bytes, and going through a String only to turn it back into bytes costs a pass over the data for nothing.

    text_of

    fn text_of(input : BytesView) -> String

    ASCII bytes, as the string they spell.

    unpack

    fn unpack(input : StringView, bits : Int, lookup : Array[Int], lossy : Bool) -> Bytes raise Malformed

    Unpack characters back into bytes.

    lossy decides what an unknown character means: skipped, which is what a PEM body full of newlines needs, or an error naming where it was.

    unpack_bytes

    fn unpack_bytes(input : BytesView, bits : Int, lookup : Array[Int], lossy : Bool) -> Bytes raise Malformed

    Unpack from ASCII bytes, for callers who never had a string.