moonjson

    moonjson — the JSON family for MoonBit: RFC 8259 JSON, JSONC, JSON5 and JSON Lines over one data model, with JSON Pointer, and failures that name the line and column.

    json
    json5
    jsonc
    jsonlines
    ndjson
    json-pointer
    parser
    moonbit
    Download zip
    Version
    0.4.0
    License
    Apache-2.0
    Last updated
    yesterday
    Downloads
    303

    #moonjson

    The JSON family for MoonBit: one tree, several ways of writing it.

    let config = @json.loads("{\"host\":\"localhost\",\"port\":8080}")
    @json.dumps(config) // {"host":"localhost","port":8080}
    @json.dumps(config, indent=2) // over several lines

    // The same document, written the way a settings file is written.
    @jsonc.loads("{\"port\": 8080, // the one we bound\n}")

    // One place in a document, named (RFC 6901).
    @pointer.get(config, "/host") // "localhost"

    Run moon run examples/tour for the whole surface in one go.

    #The surface

    One set of verbs, the ones Python's json, PyYAML and tomllib share:

    text → treebytes → treetree → texttree → bytes
    one documentloadsloaddumpsdump
    a stream of themloads_allload_alldumps_alldump_all

    The s is on the form that takes or answers a string, as it is in Python. The plural set exists exactly where the format defines a streamlines is a stream and has only those; json, jsonc and json5 define one document and have only the singular set. moonyaml follows the same rule and has both, because --- makes both meaningful.

    Not from_str and to_string: to_string already means Show::to_string in MoonBit, and @json.to_string(x) beside x.to_string() is a sentence that reads two ways.

    // Read a document, change one field, write it back.
    let cfg = @jsonc.loads(text[:])
    let cfg = @pointer.set(cfg, "/spec/replicas", Json::number(5))
    @json.dumps(cfg, indent=2)

    #Packages

    PackageWhatSpecification
    jsonJSON, exactlyRFC 8259, ECMA-404
    jsoncJSON with comments and trailing commaswhat VS Code accepts
    json5JSON5: unquoted names, single quotes, hexadecimal, Infinityspec.json5.org
    linesJSON Lines / NDJSON: one document to a linejsonlines.org
    pointerJSON PointerRFC 6901
    moonjsonThe scanner the dialects share, the writer, and Flavor

    Each has the same four faces — parse, parse_bytes, write, write_bytes so learning one is learning all of them. Writing is always strict JSON, whatever the document was written in.

    #Configuration

    How to read is a value. Flavor carries both what syntax a dialect accepts and what it does at the edges — how deep the nesting may go, what a lone surrogate means, what a repeated member name means — and every reader takes one:

    @json.loads(text) // strict JSON
    @jsonc.loads(text) // the same, plus comments
    @json.loads(text, flavor=@moonjson.Flavor::new(depth=100))
    @json.loads(text, flavor={ ..@moonjson.strict, duplicates: Reject })

    Two layers, the later overriding the earlier: the dialect's preset < the flavor you pass. The two ways of building one — the constructor and the record update — are the same thing, and a test asserts it.

    There are no per-call mirrors of the individual settings here, and the reason is the arithmetic: four dialects times four entry points is sixteen signatures, and a setting mirrored into all of them costs sixteen lines every time one is added. A dialect is chosen per use site rather than per call, so the record is where it belongs. Writing is the other way round, so its three settings are arguments:

    @json.dumps(value) // compact
    @json.dumps(value, indent=2) // over several lines
    @json.dumps(value, ascii=true) // escape everything above U+007E
    @json.dumps(value, sort=true) // members in order of name

    #The defaults, and where they come from

    SettingDefaultWhy that one
    depth500Everyone bounds it and nobody agrees on the number — Jackson refuses past 1000, serde_json past 128. A hundred thousand open brackets is a denial of service, not a document
    surrogatesrefuseImplementations disagree and JSONTestSuite files these cases as implementation-defined, so the safer side is the default: a lone half cannot be written back out
    duplicatesLastJavaScript, Python, Go and serde all keep the last one
    bomskippedRFC 8259 §8.1 forbids writing one and says nothing about reading one; refusing would reject half of what Windows tooling produces
    asciioffA UTF-8 body is the normal case now. Python's ensure_ascii defaults the other way because it predates that
    sortoffThe order a document was written in is information; a diff is the reason to discard it, and a diff can ask

    #The tree

    The data model is core's Json, not one of our own. Anything that already speaks @json.Json speaks this, with no conversion.

    A number keeps the text it was written as, so a document read and written again comes back unchanged — including a twenty-digit integer and 1e400, which a double cannot hold and which a parser that goes through one silently rewrites. A number JSON has no syntax for, which JSON5 can produce, is written as null, as every JavaScript implementation writes it.

    Members keep their order.

    #Failures

    Reading raises Malformed, which says what went wrong and where — offset, line and column:

    try @json.loads(source) catch {
    e => println("line \{e.at().line}, column \{e.at().column}")
    }

    Nesting is refused past 500 levels rather than taken to the stack: a document of a hundred thousand open brackets is a denial of service, not a document.

    A lone half of a surrogate pair is refused. It is not a character, it cannot be written back out, and any value put in its place would be a guess.

    #What is checked

    The strict parser is measured against cases drawn from JSONTestSuite, with every verdict taken from a third-party parser rather than asserted from memory — 95 documents that must be accepted or refused, and are. JSON5 is measured against the document json5.org puts on its front page, and JSON Pointer against every example RFC 6901 §5 prints.

    #What is not here yet

    Hjson and the other text dialects; CBOR, MessagePack, BSON and the other binary encodings; PostgreSQL's and SQLite's jsonb; JSON Patch, JSONPath, JSON Schema and canonical serialisation. They are planned in that order; the tracking list lives with the project.

    YAML and TOML are their own repositories. HOCON and HCL belong with configuration, because their substitutions and merges are evaluation, not syntax.

    #Install

    moon add moonbitstack/moonjson

    #Licence

    Apache-2.0.

    Malformed

    pub(all) suberror Malformed {
    Unexpected(At, Char)
    Truncated(At)
    BadEscape(At)
    BadNumber(At)
    Trailing(At)
    TooDeep(At)
    BadUtf8(At)
    Repeated(At, String)
    } derive(Eq,
    Debug
    )

    Why a document could not be read, and where.

    Malformed::at

    fn Malformed::at(self : Malformed) -> At

    Where an error happened, as a person would say it.

    Malformed::equal

    fn Malformed::equal(Malformed, Malformed) -> Bool

    Malformed::not_equal

    fn Malformed::not_equal(x : Malformed, y : Malformed) -> Bool

    pub(all) struct At {
    at : Int
    line : Int
    column : Int
    } derive(Eq,
    Debug
    )

    Where something went wrong.

    The offset is in UTF-16 code units from the start, which is what a [StringView] indexes by; line and column are one-based, because that is how every editor counts and an error a person cannot find in their file is half an error.

    At::equal

    fn At::equal(At, At) -> Bool

    At::not_equal

    fn At::not_equal(x : At, y : At) -> Bool

    At::to_repr

    Duplicates

    pub(all) enum Duplicates {
    Last
    First
    Reject
    } derive(Eq,
    Debug
    )

    What a repeated member name means.

    Every mainstream implementation keeps the last one — JavaScript, Python, Go and serde all do — so that is the default. The other two are here because a document with two "id" members is usually a mistake, and a reader that wants to hear about it should not have to write its own parser.

    Duplicates::equal

    fn Duplicates::equal(Duplicates, Duplicates) -> Bool

    Duplicates::not_equal

    fn Duplicates::not_equal(x : Duplicates, y : Duplicates) -> Bool

    Flavor

    pub(all) struct Flavor {
    comments : Bool
    trailing_commas : Bool
    unquoted_keys : Bool
    single_quotes : Bool
    extra_numbers : Bool
    extra_escapes : Bool
    extra_space : Bool
    depth : Int
    surrogates : Bool
    duplicates : Duplicates
    } derive(Eq,
    Debug
    )

    How to read a document: what syntax to accept, and what to do at the edges.

    The dialects in this family differ only in their syntax — the tree they produce is the same — so the scanner is written once and each dialect is a set of switches. A dialect outside this repository can be described the same way without forking anything.

    The last three fields are not syntax but policy. They are here rather than as arguments because they are set once for a deployment and because this family has sixteen entry points: a policy added here costs one line, and a policy added to every signature costs sixteen.

    Flavor::equal

    fn Flavor::equal(Flavor, Flavor) -> Bool

    Flavor::new

    fn Flavor::new(comments? : Bool, trailing_commas? : Bool, unquoted_keys? : Bool, single_quotes? : Bool, extra_numbers? : Bool, extra_escapes? : Bool, extra_space? : Bool, depth? : Int, surrogates? : Bool, duplicates? : Duplicates) -> Flavor

    Build a dialect by naming the parts that differ from strict JSON.

    The same thing can be written { ..@moonjson.strict, depth: 100 }; this form exists because a named argument reads better when several parts differ.

    Flavor::not_equal

    fn Flavor::not_equal(x : Flavor, y : Flavor) -> Bool

    Flavor::to_repr

    commented

    let commented : Flavor

    JSON with comments and trailing commas — what editors and their configuration files have settled on, tsconfig.json being the best-known.

    dump

    fn dump(value : Json, indent? : Int, ascii? : Bool, sort? : Bool) -> Bytes

    The same, as UTF-8 bytes — the form a body goes onto a wire in.

    dumps

    fn dumps(value : Json, indent? : Int, ascii? : Bool, sort? : Bool) -> String

    Write a tree as strict JSON, whatever dialect it was read from.

    indent of zero, the default, writes the compact form with no space anywhere; anything larger writes one line per member with that many spaces a level, and a space after each colon.

    A number that came in as text goes back out as that text, so a document read and written again is unchanged. Only a number with no such text is formatted, and a number that is not finite — which JSON5 can produce and JSON cannot write — becomes null, as every JavaScript implementation does.

    load

    fn load(input : BytesView, flavor : Flavor, bom? : Bool) -> Json raise Malformed

    Read one document from bytes, which is how one arrives over a wire. A leading byte-order mark is skipped by default. RFC 8259 §8.1 forbids writing one and says nothing about refusing to read one, and refusing would mean rejecting half the files Windows tooling produces.

    loads

    fn loads(input : StringView, flavor : Flavor) -> Json raise Malformed

    Read one document in the given dialect.

    The whole input must be one value: anything after it, other than whitespace and — where the dialect allows them — comments, is [Trailing], not silently ignored. A parser that stops at the first value turns a truncated file into a plausible one.

    relaxed

    let relaxed : Flavor

    JSON5: everything ECMAScript 5 would have accepted.

    strict

    let strict : Flavor

    RFC 8259 and nothing more.