🥮 mooncakes.io

    jmespath

    A standards-oriented JMESPath parser and JSON query engine in MoonBit.

    jmespath
    json
    query
    parser
    Download zip
    Version
    0.1.17
    License
    Apache-2.0
    Repository
    Last updated
    5 days ago
    Downloads
    146

    No dependencies

    #MoonJMES

    #0.1.17 grouping and negation precedence

    Parentheses now end a projection's scope. Given {"a":[{"b":1},{"b":2}]}, (a[*])[0] returns {"b":1} rather than [], and (a[*]).b returns null rather than [1,2]. The ungrouped a[*].b still projects over each item. Unary ! now binds above dot, flatten and filter suffixes, while indexes and active projection tails retain their binding. Use !(a.b) to negate a full path; !a.b accesses b on the boolean produced by !a and returns null. See acceptance evidence for verification/publication status.

    #0.1.16 flatten syntax validation

    The flatten operator is the contiguous token []. Previously [ ] and brackets split by tabs or newlines silently flattened arrays, including inside skipped boolean branches. The lexer now recognizes [] as one token and rejects split forms during compilation. Whitespace around [] and inside nonempty index, slice and wildcard brackets remains supported. Regression tests cover nested references, filters and preservation of valid batch neighbors. See acceptance evidence for verification/publication status.

    #0.1.15 expression references

    Version 0.1.15 checks expression-reference placement at compile time. An & reference must begin a function argument. A misplaced reference now reports syntax even in a skipped boolean branch or an empty filter.

    #Verification target isolation

    Full verification explicitly tests the debug/release CLI files it builds, ignoring an inherited JMES_CLI_PATH override. The receipt records those paths. Standalone harnesses still allow this override for downloaded-package testing. This is a verification-only fix; the verified version at that checkpoint was 0.1.14.

    #Additional verification audit

    The repository now checks special object keys, duplicate fields and escaped text in both output formats and build modes. Verification reports use actual harness counts and discard an older success receipt when a new run starts. This audit changes test tooling only; the verified version at that checkpoint was 0.1.14. See acceptance evidence.

    #0.1.14 numeric representation validation

    Library callers can attach original numeric text through Json::number(..., repr=...). Previously that text was serialized verbatim even if it said null, contained multiple JSON values or disagreed with the stored number. Bounded serialization and query input validation now reject these values with invalid_value. Matching large-number spelling is preserved. This concerns programmatically constructed Json values, not a demonstrated CLI JSON input vulnerability. See verification and release evidence.

    #0.1.13 consistent filter syntax validation

    Malformed [ ?condition] filters were accepted at the root or inside function arguments but rejected after a field. Filter openers are now checked uniformly: [? must be adjacent, while [? condition ] and field [? condition ] remain valid. This is a syntax consistency fix. Literal strings containing [ ? are unchanged. See verification and publication evidence.

    #0.1.12 compound expression references

    Queries such as map(&a || b, @), map(&a > 1, @) and sort_by(@, &a || b) previously returned invalid_type despite valid input. The parser now captures the complete expression after &, including pipes, while preserving function argument boundaries and existing resource guards. See verification and publication evidence.

    #0.1.11 string allocation fix

    An 8,000,000-character length(reverse(@)) query previously aborted both build modes under the 192 MiB test heap and lost neighboring batch responses. Reversal now checks the work budget before output allocation and avoids temporary character arrays. String length counts Unicode scalars with a metered iterator. Oversized work returns an isolated limit_exceeded error. Large legitimate strings may need a higher trusted budget; this is not a universal memory guarantee. See release evidence for verification and publication status.

    简体中文 | English

    CI License

    MoonJMES is a native MoonBit implementation of JMESPath, the standard query language for JSON. It turns nested API responses, cloud inventories, audit records, and configuration files into smaller JSON results without embedding JavaScript or shipping data to a service.

    The engine passes 892/892 cases in the official JMESPath compliance suite at pinned revision 53abcc37901891cf4308fcd910eab287416c4609. The result is reproducible with node scripts/conformance.mjs; see docs/conformance.md.

    #Why this project

    MoonBit already has general JSON and jq-style tools, but the Mooncakes search at project start returned no JMESPath implementation. JMESPath is widely used in cloud and automation tooling, has a formal grammar, and has a language-neutral test suite. That gives this project a clear ecosystem boundary and an objective definition of compatibility.

    #Features

    • identifiers, quoted identifiers, current node, JSON and raw-string literals
    • object/list wildcards, flattening, projections, filters, indexes, and slices
    • pipes, comparisons, boolean expressions, multi-select lists and hashes
    • all standard scalar, collection, conversion, and expression-reference functions, including map, sort_by, min_by, and max_by
    • reusable compiled expressions and one-shot JSON/text APIs
    • batch execution with per-request structured errors
    • configurable expression, AST depth, evaluation-step, and result limits
    • a file/stdin CLI with deterministic JSON output
    • pure MoonBit engine; Node.js is used only by the CLI host adapter

    #Quick start

    Install the published library from Mooncakes:

    moon add JingLan0v0/jmespath

    The package documentation is available at mooncakes.io/docs/JingLan0v0/jmespath.

    moon add installs a library dependency, not a global CLI executable. This release supports the JS backend; CI pins MoonBit 0.10.14+7d59c7ec9 and Node.js 24.14.0 (see toolchain.json). To run the CLI, clone this repository and query the included file:

    git clone https://github.com/JingLan0v0/moonbit-jmespath.git
cd moonbit-jmespath
moon run cmd/main --target js -- "instances[?state == 'running'].{id: id, zone: zone}" examples/cloud-inventory.json

    Or pipe JSON through standard input:

    echo '{"items":[{"name":"A","price":9},{"name":"B","price":12}]}' \
  | moon run cmd/main --target js -- 'items[?price >= `10`].name' -

    Result:

    [
  "B"
]

    #Library API

    Import the package in moon.pkg:

    import {
      "JingLan0v0/jmespath" @jmespath,
      "moonbitlang/core/json",
    }

    let input = @json.parse("{\"people\":[{\"name\":\"Ada\",\"age\":36}]}")
    let query = @jmespath.compile("people[?age >= `18`].name")
    let result = query.search(input)

    Public entry points: compile, Compiled::search, search, search_json, and search_batch. Use stringify_bounded(result, pretty=true) to serialize library results with the same output safeguards as the CLI.

    The fragment above belongs inside a function that handles or propagates errors. For a complete executable with imports, custom limits, query reuse and error handling, run moon run examples/library --target js from this checkout. See the API and error guide.

    #Batch API and CLI

    Batch input is an array of independent requests:

    [
  {"expression":"name","data":{"name":"Ada"}},
  {"expression":"length()","data":null}
]

    moon run cmd/main --target js -- --batch examples/batch.json

    Each response has ok: true and result, or ok: false and a structured error. Invalid expressions and malformed request entries therefore do not discard other batch results. Invalid top-level JSON still exits with code 2. Batch mode exits 0 even when individual requests fail: inspect every ok field. The included file adds a third successful fallback query after the error.

    #Three runnable scenarios

    The examples directory covers cloud inventory selection, deployment-policy findings, and cost-report sorting. Run all three on Windows with ./examples/run.ps1; the script compares each result with committed JSON.

    #Resource limits and errors

    Defaults are 16,384 expression characters, AST depth 256, 1,000,000 evaluation steps, and 100,000 cumulatively materialized result elements. Custom limits must be valid, and AST depth cannot exceed 256. Batch input is capped at 10,000 requests; the CLI reads at most 16 MiB and decodes UTF-8 strictly, accepting an optional UTF-8 BOM.

    Version 0.1.3 also bounds active evaluation calls to 128, checks JSON input and output iteratively, and limits aggregate batch successes. The JSON guard limits depth to 256 (root at zero), expanded nodes to 1,000,000 and the conservative output estimate to 16 Mi UTF-16 units. See the exact resource contract.

    Errors use stable categories: syntax, invalid_json, invalid_type, invalid_arity, invalid_value, unknown_function, invalid_batch, and limit_exceeded. A library caller receives JmesPathError; the CLI writes a single diagnostic to stderr and exits with code 2. This describes single-query or whole-batch failures; per-item batch errors stay in the JSON response on stdout, as described above.

    #Verification

    moon fmt --check
moon check --target js
moon test --target js
moon info --target js
node scripts/conformance.mjs
node scripts/conformance.mjs --release
node scripts/verify.mjs

    The GitHub Actions matrix runs on Windows and Ubuntu with the pinned compiler. Architecture and trust boundaries are in docs/architecture.md.

    #September acceptance bug fixed in 0.1.2

    The rejected release could crash while parsing an expression made of 4,000 ! prefixes followed by @ (4,001 characters, below the default length limit). The parser checked AST depth only after recursion, so a three-request batch produced no output and exited with status 1 in both debug and release. The parser now enforces recursion depth while building the AST and checks the depth of each constructed node. The offending request returns a structured limit_exceeded error; the requests before and after it still return results. Single-query CLI mode exits with status 2 and a diagnostic instead of crashing. Regression tests cover the exact batch shape, single-query mode, nested grouping and the allowed depth boundary in both CLI builds. The original 0.1.1 release remains available for historical comparison.

    A Chinese project overview for contest information review is available in docs/project-introduction.zh-CN.md.

    #Project boundary

    #October hardening in 0.1.3

    The October audit reproduced a separate stack overflow in 0.1.2: 255 nested to_array(...) calls passed AST validation but crashed the debug evaluator. Execution now checks active call depth before descending. Short repeated [@,@] pipes could also expand exponentially during serialization while barely increasing the old result counter. An iterative output guard now counts each expanded occurrence; iterative JSON equality consumes the query work budget. join checks size before allocation, object producers charge keys before insertion, and batch successes share an output budget with room reserved for errors. Non-finite arithmetic returns invalid_value instead of invalid JSON.

    Verification covers 52 unit tests, three scenarios, and, in each CLI build mode, 33 adversarial checks plus 256 deterministic grammar mutations. Test subprocesses have a timeout and heap cap. CI enforces warning-free checks and exact official error categories in both modes, on Windows and Ubuntu, and saves downloadable JSON evidence. See acceptance evidence and competition status.

    MoonJMES implements the JMESPath specification. The 0.1.x line has no custom query extensions, keeping results portable across compliant JMESPath implementations. Version 0.1.10 is published on Mooncakes and verified from a fresh consumer in debug and release. Older versions remain available for historical comparison. See CHANGELOG.md.

    #Null semantics and usability fixes in 0.1.4

    An additional audit found behavior not covered by the pinned official vectors: multi-select lists/hashes must preserve a null input, while a subexpression must still evaluate its right-hand function on a null left result. For example, missing.not_null(@, 'fallback') now returns "fallback" for {}, and [foo] returns null for null input. Added regressions cover projections, map, error propagation, and non-null false-like values. Existing resource guards remain.

    Quick-start commands now use committed input files, a complete public API example runs in both modes, and the PowerShell scenario runner works outside the checkout. See CHANGELOG.md, usage and contributing. Release evidence is in acceptance.md.

    #Numeric conversion memory fix in 0.1.5

    to_number no longer parses arbitrary encoded arrays/objects merely to return null. An encoded 4.8 MB object array crashed 0.1.4 under a 192 MiB test heap in both modes; scalar prefiltering now preserves the batch's neighboring results. Numeric scanning also consumes the work budget. This is a conversion fix, not a universal memory guarantee: core transport decoding can still exhaust a constrained heap. See resource limits and the release evidence.

    #Decoder preflight in 0.1.6

    The 12 MB escaped-string case left open in 0.1.5 now fails before JSON decoder allocation. Constant-space lexical scanning caps raw string tokens at 8 Mi UTF-16 units, lexical tokens (including keys) at 1,000,000, and nesting at 1,024. This intentionally adds restrictions within the overall 16 MiB input cap. Single-query, batch transport and expression JSON decoding share this check. The public parse_json_bounded supports safe text ingestion by library callers. Whole-transport failures return limit_exceeded, exit 2, empty stdout; item isolation still applies once the batch is decoded. Regression tests exercise both modes under the same 192 MiB test heap. This is not a universal peak-memory guarantee. See resource limits and evidence.

    #CLI output and diagnostic fixes in 0.1.7

    Output failures (including a downstream pipe closing early) now exit 2 rather than throwing an unhandled Node exception. If stderr is available, a short I/O diagnostic is emitted; if both streams fail, only exit status can report it. Diagnostics escape embedded controls, line separators and bidi formatting so invalid options cannot create forged log lines or terminal commands. Normal JSON stdout is unchanged. Output may be partial if a destination fails during writing; consumers must check exit status. Five new subprocess cases pass in both modes, on Windows/Ubuntu and from the downloaded release. See usage and release evidence.

    #Unicode ordering corrected in 0.1.8

    String ordering now follows Unicode code points rather than UTF-16 code units, fixing incorrect results across BMP and supplementary-plane characters in sort, min, max and their _by variants. For example U+E000 must precede U+10000. The comparator uses iterators without allocating character arrays; there is no locale collation or normalization. Regression cases include empty strings, prefixes, ties, combining sequences and U+10FFFF. Numeric extremes and existing behavior also pass. See usage and evidence.

    #Intermediate overflow consistency in 0.1.9

    Non-finite scalar results are now rejected before parent expressions consume them. Previously type(sum(@)) on [1e308,1e308] succeeded while direct sum correctly failed. Arithmetic/conversion overflow can no longer hide behind a type check, comparison or collection wrapper. Lazy boolean branches remain lazy, batches isolate the error and compiled queries remain reusable. This enforces the finite-JSON policy; it does not improve floating-point precision or change the running-sum algorithm for avg. See resource limits and evidence.

    #Ordering work budgets in 0.1.10

    Ordering scans, comparisons and Unicode iterator advances now consume max_steps across sort/min/max and their _by variants. Stable bottom-up merge sorting also meters scratch copies and merge writes, and stops immediately on budget errors. Long common prefixes can no longer perform unmetered comparison work. Inputs stay unchanged, equal-key order is preserved, and later batch items still run. Large legitimate sorts may now need a higher explicitly chosen trusted budget; this is an intentional tightening, not a wall-clock guarantee. See resource contract and release evidence.

    #License

    Apache-2.0. See THIRD_PARTY_NOTICES.md for the verification-time relationship with the official test suite.

    JmesPathError

    source
    pub(all) suberror JmesPathError {
      JmesPathError(Diagnostic)
    }

    CompareOp

    source
    type CompareOp derive(Eq, 
    Debug    )

    CompareOp::equal

    source
    fn CompareOp::equal(CompareOp, CompareOp) -> Bool

    CompareOp::not_equal

    source
    fn CompareOp::not_equal(x : CompareOp, y : CompareOp) -> Bool

    CompareOp::to_repr

    source

    Compiled

    source
    pub struct Compiled {
      source : String
      expr : Expr
      limits : Limits
    }

    Compiled::search

    source

    Compiled::source

    source
    fn Compiled::source(self : Compiled) -> String

    Diagnostic

    source
    pub(all) struct Diagnostic {
      code : String
      offset : Int
      message : String
    } derive(Eq, 
    Debug    )

    Diagnostic::equal

    source
    fn Diagnostic::equal(Diagnostic, Diagnostic) -> Bool

    Diagnostic::not_equal

    source
    fn Diagnostic::not_equal(x : Diagnostic, y : Diagnostic) -> Bool

    Diagnostic::to_repr

    source

    Expr

    source

    Expr::equal

    source
    fn Expr::equal(Expr, Expr) -> Bool

    Expr::not_equal

    source
    fn Expr::not_equal(x : Expr, y : Expr) -> Bool

    Expr::to_repr

    source

    Limits

    source
    pub(all) struct Limits {
      max_expression_chars : Int
      max_ast_depth : Int
      max_steps : Int
      max_results : Int
    } derive(Eq, 
    Debug    )

    Limits::equal

    source
    fn Limits::equal(Limits, Limits) -> Bool

    Limits::not_equal

    source
    fn Limits::not_equal(x : Limits, y : Limits) -> Bool

    Limits::to_repr

    source

    Token

    source

    Token::equal

    source
    fn Token::equal(Token, Token) -> Bool

    Token::not_equal

    source
    fn Token::not_equal(x : Token, y : Token) -> Bool

    Token::to_repr

    source

    TokenKind

    source
    type TokenKind derive(Eq, 
    Debug    )

    TokenKind::equal

    source
    fn TokenKind::equal(TokenKind, TokenKind) -> Bool

    TokenKind::not_equal

    source
    fn TokenKind::not_equal(x : TokenKind, y : TokenKind) -> Bool

    TokenKind::to_repr

    source

    compile

    source
    fn compile(source : String, limits? : Limits) -> Compiled raise JmesPathError

    default_limits

    source
    fn default_limits() -> Limits

    parse_json_bounded

    source
    fn parse_json_bounded(text : String) -> 
    Json     raise JmesPathError

    Preflight text allocation budgets, then parse JSON. This does not impose per-query JSON shape limits; batch callers retain independent item checks.
    fn search(expression : String, data : 
    Json    , limits? : Limits) -> 
    Json     raise JmesPathError

    search_batch

    source

    Runs independent queries from a JSON array of {expression, data} objects. Errors are returned per item, so one invalid query or malformed request does not abort the batch. The top-level input must still be an array.

    search_json

    source
    fn search_json(expression : String, json_text : String, limits? : Limits) -> 
    Json     raise JmesPathError

    stringify_bounded

    source
    fn stringify_bounded(value : 
    Json    , pretty? : Bool) -> String raise JmesPathError

    Serialize after checking nesting, expanded node count, and a conservative output-size estimate. Pretty output uses two spaces and is also bounded.

    version

    source
    fn version() -> String

    Source Files