Sign in

    jq

    A jq clone implemented in MoonBit

    jq
    json
    moonbit
    Download zip
    Author
    Version
    0.1.1
    License
    Apache-2.0
    Last updated
    7 months ago
    Downloads
    2K

    #mizchi/jq

    A jq clone implemented in MoonBit.

    #Features

    • Lexer, recursive descent parser, callback-based evaluator
    • 87.3% compatibility with jq's official test suite (151/173 verified tests)
    • 50+ builtin functions
    • Target: js (Node.js)

    #Supported Syntax

    CategoryFeatures
    Basic. (identity), .foo, .[n], .[], .. (recursive descent)
    Operators\|, ,, + - * / %, == != < > <= >=, and or not
    Construction[expr], {k: v}, {foo} (shorthand), (expr)
    Controlif-then-elif-else-end, try-catch, .foo?, try expr
    Bindingas $x, reduce, def f: body;, def f(a;b): body;
    Builtinslength, keys, values, type, select, map, map_values, empty, add, sort, sort_by, group_by, unique, unique_by, reverse, flatten, min, max, min_by, max_by, first, last, limit, range, has, contains, inside, any, all, tostring, tonumber, tojson, fromjson, ascii_downcase, ascii_upcase, split, join, startswith, endswith, ltrimstr, rtrimstr, trim, ltrim, rtrim, trimstr, abs, fabs, floor, ceil, round, explode, implode, to_entries, from_entries, with_entries, walk, transpose, bsearch, indices, index, rindex, not, recurse, debug, getpath, IN
    Formats@base64, @base64d, @json, @text, @html, @uri
    Typesnumbers, strings, booleans, nulls, arrays, objects, iterables, scalars

    #Known Incompatibilities with jq

    #Parser Limitations

    • {x:-.} (unary minus as object value without parens) is not supported. Use {x:(-.)}.
    • try -.? does not propagate to catch correctly.
    • String interpolation \(expr) is not supported.

    #Numeric Precision

    • Numbers use IEEE 754 double precision. Integers larger than 2^53 overflow to Infinity instead of being represented exactly.
    • jq outputs 2.0 for some integer-valued float results (e.g., 1+1 with string input gives 2.0 in jq, 2 here).
    • Scientific notation in output (e.g., 1e-1, 9E+999999999) may differ.

    #Unicode / String Handling

    • Control characters (\b, \f) are output as literal characters instead of \u00XX escape sequences.
    • Unicode escapes like \u03bc may be output as the actual character (μ) instead of the escape form.
    • contains with NUL bytes (\u0000) may not match correctly due to JavaScript string handling.

    #Slice Indexing

    • Float slice indices like .[1.2:3.5] use truncation (.[1:3]) instead of jq's rounding behavior (.[1:4]).

    #Unsupported Features

    • Assignment operators: =, |=, +=, -=, *=, /=, %=, //=
    • Alternative operator: //
    • Destructuring bind: as [$a, $b], as {$a}
    • Alternative destructuring: ?//
    • String interpolation: \(expr)
    • Control flow: while, until, label, break, foreach
    • Path operations: path, paths, setpath, delpaths, del, pick
    • Format strings: @csv, @tsv, @sh, @urid
    • Module system: import, include, modulemeta
    • I/O: input, inputs, input_line_number
    • Regex: test (pattern), match, capture, scan, sub, gsub, splits
    • Other: env, builtins, $__loc__, $ENV, nth, skip, toboolean, utf8bytelength, halt, stderr, debug(msg), sqrt, sin, cos, atan, log, exp, pow, strftime, strptime, now, date, JOIN, REGEX

    #walk Behavior

    • walk(f) where f produces multiple outputs generates a combinatorial explosion instead of jq's first-output behavior.

    #Benchmark (js target, pre-compiled eval)

    BenchmarkOps/secvs NaiveDescription
    identity4,910,61010.8x.
    field_access3,805,34510.5x.a
    nested_field539,3582.9x.a.b.c.d
    sort325,0852.3xsort (10 elements)
    keys_length468,1234.3xkeys \| length
    object_construct397,7034.0x{x: .a, y: .b}
    parse_complex321,20025.1xif .a > 0 then ... end
    reduce297,40626.6xreduce .[] as $x (0; . + $x)
    group_by_map2,516,18795.9xgroup_by(.k) \| map(...)
    split_join250,2976.1xsplit(",") \| join("-")
    map111,3053.1xmap(. + 1) (10 elements)
    recursive_descent99,4854.0x[.. \| numbers]
    array_iterate83,6732.8x[.[] \| . * 2] (10 elements)
    select_filter48,0066.2x[.[] \| select(. > 5)] (10 elements)
    pipe_chain36,1604.7x.[] \| . * 2 \| . + 1 \| tostring \| length

    Measured on js target with @bench. "vs Naive" compares pre-compiled eval against naive parse+eval per call.

    #Optimizations Applied

    • compile/run_compiled separation: Parse the filter once, evaluate many times
    • Persistent scope chain: Linked-list env makes bind_var O(1) (no array copy)
    • eval_single fast path: Single-output filters (Identity, Field, Literal, Pipe chains, variable refs) bypass callback/closure overhead
    • AST optimization pass: Eliminates Pipe(Identity, x) → x, strips Paren wrappers at compile time
    • Pre-baked $ prefix: Variable names include $ at parse time, avoiding runtime string concatenation

    #Quick Commands

    just # check + test just fmt # format code just check # type check just test # run tests just run # run main

    #License

    Apache-2.0

    #mizchi/jq

    A jq clone implemented in MoonBit. Provides compile, run, and run_string for JSON query/transform.

    #Quick Start

    ///|
    test {
    inspect(
    @jq.run_string(".foo", "{\"foo\":42}"),
    content=(
    #|["42"]
    ),
    )
    }

    #API

    #run_string — One-shot string-based execution

    ///|
    test {
    inspect(
    @jq.run_string("[.[] | select(. > 3)]", "[1,2,3,4,5]"),
    content=(
    #|["[4,5]"]
    ),
    )
    }

    #run — Execute with Json value

    ///|
    test {
    let input = @json.parse("{\"a\":1,\"b\":2}")
    let results = @jq.run("{sum: (.a + .b)}", input)
    inspect(
    results[0].stringify(),
    content=(
    #|{"sum":3}
    ),
    )
    }

    #compile + run_compiled — Pre-compile for repeated use

    ///|
    test {
    let filter = @jq.compile("map(. * 2)")
    let r1 = @jq.run_compiled(filter, @json.parse("[1,2,3]"))
    let r2 = @jq.run_compiled(filter, @json.parse("[10,20]"))
    inspect(r1[0].stringify(), content="[2,4,6]")
    inspect(r2[0].stringify(), content="[20,40]")
    }

    #Error Handling

    ///|
    test {
    let result = @jq.run_string(".foo", "not json") catch {
    @jq.JqError::JqError(msg) => ["Error: " + msg]
    }
    inspect(result[0].has_prefix("Error:"), content="true")
    }

    #Examples

    #Field Access and Pipes

    ///|
    test {
    let input = "{\"user\":{\"name\":\"alice\",\"age\":30}}"
    inspect(
    @jq.run_string(".user.name", input),
    content=(
    #|["\"alice\""]
    ),
    )
    inspect(
    @jq.run_string(".user | keys", input),
    content=(
    #|["[\"age\",\"name\"]"]
    ),
    )
    }

    #Array Operations

    ///|
    test {
    let arr = "[3,1,4,1,5,9,2,6]"
    inspect(
    @jq.run_string("sort | unique", arr),
    content=(
    #|["[1,2,3,4,5,6,9]"]
    ),
    )
    inspect(
    @jq.run_string("map(. * 2) | add", arr),
    content=(
    #|["62"]
    ),
    )
    }

    #Reduce

    ///|
    test {
    inspect(
    @jq.run_string("reduce .[] as $x (0; . + $x)", "[1,2,3,4,5]"),
    content=(
    #|["15"]
    ),
    )
    }

    #Object Construction

    ///|
    test {
    let input = "[{\"name\":\"a\",\"val\":1},{\"name\":\"b\",\"val\":2}]"
    inspect(
    @jq.run_string("map({(.name): .val}) | add", input),
    content=(
    #|["{\"a\":1,\"b\":2}"]
    ),
    )
    }

    #Conditionals

    ///|
    test {
    inspect(
    @jq.run_string(
    "map(if . > 0 then \"pos\" elif . == 0 then \"zero\" else \"neg\" end)", "[-1,0,1]",
    ),
    content=(
    #|["[\"neg\",\"zero\",\"pos\"]"]
    ),
    )
    }

    #User-Defined Functions

    ///|
    test {
    inspect(
    @jq.run_string(
    "def double: . * 2; def inc: . + 1; map(double | inc)", "[1,2,3]",
    ),
    content=(
    #|["[3,5,7]"]
    ),
    )
    }

    #Format Strings

    ///|
    test {
    inspect(
    @jq.run_string("@base64", "\"hello\""),
    content=(
    #|["\"aGVsbG8=\""]
    ),
    )
    inspect(
    @jq.run_string("@base64d", "\"aGVsbG8=\""),
    content=(
    #|["\"hello\""]
    ),
    )
    }

    JqError

    pub(all) suberror JqError {
    JqError(String)
    }

    Error type for jq compilation and evaluation failures.
    impl Show for JqError

    ArithOp

    pub enum ArithOp {
    Add
    Sub
    Mul
    Div
    Mod
    }

    Arithmetic operation type used in filter expressions.
    impl Show for ArithOp

    CmpOp

    pub enum CmpOp {
    Eq
    Ne
    Lt
    Gt
    Le
    Ge
    }

    Comparison operation type used in filter expressions.
    impl Show for CmpOp

    Filter

    pub enum Filter {
    Identity
    RecurseOp
    Field(String)
    Index(Int)
    Slice(Int?, Int?)
    Iterate
    Pipe(Filter, Filter)
    Comma(Filter, Filter)
    Literal(Json)
    ArrayConstruct(Filter?)
    ObjectConstruct(Array[(ObjKey, Filter)])
    Paren(Filter)
    Try(Filter)
    TryCatch(Filter, Filter)
    IfThenElse(Filter, Filter, Filter)
    Binding(String, Filter, Filter)
    Reduce(Filter, String, Filter, Filter)
    FuncDef(String, Array[String], Filter, Filter)
    FuncCall(String, Array[Filter])
    DynIndex(Filter)
    PostfixDynIndex(Filter, Filter)
    DynSlice(Filter, Filter?, Filter?)
    Neg(Filter)
    Arith(ArithOp, Filter, Filter)
    Compare(CmpOp, Filter, Filter)
    LogicAnd(Filter, Filter)
    LogicOr(Filter, Filter)
    }

    A compiled jq filter AST node.
    impl Show for Filter

    ObjKey

    pub enum ObjKey {
    StrKey(String)
    ExprKey(Filter)
    }

    Object key type: either a static string or a dynamic expression.
    impl Show for ObjKey

    compile

    fn compile(filter_str : String) -> Filter raise JqError

    Compile a jq filter string into a reusable Filter AST.

    run

    fn run(filter_str : String, input : Json) -> Array[Json] raise JqError

    Compile and execute a jq filter string against JSON input.

    run_compiled

    fn run_compiled(filter : Filter, input : Json) -> Array[Json] raise JqError

    Execute a pre-compiled filter against JSON input.

    run_string

    fn run_string(filter_str : String, input_str : String) -> Array[String] raise JqError

    Compile and execute a jq filter, with JSON string input/output.