#MoonLoom

$ moonloom query --file examples/access.log --stats \
    "where status >= 500 | stats count() as failures by path | sort failures desc"

+--------------------+----------+
| path               | failures |
+--------------------+----------+
| /api/orders?page=2 | 2        |
| /api/orders?page=1 | 1        |
| /health            | 1        |
+--------------------+----------+
rows in 21, scanned 21, filtered 17, groups 3, out 3

#Requirements

• The MoonBit toolchain, with moonc 0.10.14 or newer. The interfaces in this repository were generated with moon 0.1.20260920 and moonc v0.10.14.
• The engine packages (moonloom, ascii, text, query, index, plan, exec, render) import only moonbitlang/core. The command line adds moonbitlang/x for file access, which is the official MoonBit utility package.

#Quick start

moon check --target all
moon build --target wasm
moon test --deny-warn --target wasm
moon run examples/demo                  # a self-contained tour, no files needed
moon run --target js cmd/main -- query --file examples/access.log \
  "where status >= 500 | stats count() as n by path | sort n desc"

#What the query language does

from <name> | where ... | stats ... | sort ... | limit ...

# sales: revenue per region, largest first
moon run --target js cmd/main -- query --file examples/sales.csv --output markdown \
  "stats sum(amount) as revenue by region | sort revenue desc"

# JSON Lines: nested objects become dotted field names
moon run --target js cmd/main -- query --file examples/events.jsonl \
  "where http.status >= 500 | stats count() as n, percentile(duration_ms, 95) as p95 by service"

# plain text: a pattern turns prose into columns
moon run --target js cmd/main -- query --file examples/service.log --format lines \
  "parse \"{time} {level} {service}: {message}\" | where level == \"error\""

#Reading formats

$ moonloom schema --file examples/events.jsonl
format jsonl  records 10  skipped 0
+-------------+------+---------+---------+----------+
| name        | kind | present | missing | distinct |
+-------------+------+---------+---------+----------+
| time        | text | 10      | 0       | 9        |
| level       | text | 10      | 0       | 3        |
| duration_ms | num  | 10      | 0       | 10       |
| http.status | int  | 10      | 0       | 6        |
...

#How it executes

$ moonloom explain --file examples/access.log \
    "where remote_host == \"203.0.113.51\" | fields time_text, method, path, status"

plan
  scan            <input>
                  rows=21
  index           remote_host=203.0.113.51  kept=3/21 (14.3%)
  1. filter       remote_host == "203.0.113.51"
  2. project      time_text, method, path, status
  columns         remote_host, time_text, method, path, status

1. Index pruning. Every column is indexed as a value-to-rows map unless it is unique per row, in which case indexing it would prune nothing. Equality predicates are collected from the first where, their posting lists are intersected, and only those rows are read. The filter still runs, because the index only removes rows it can prove cannot match.
2. Column pruning. The planner collects the fields the pipeline actually reads and reports them on the columns line. A stage after an aggregation can only name group keys and aggregate outputs, so it does not widen the set.

$ moonloom query --file examples/access.log --stats "where status >= 500 | stats count() as n by path"
...
rows in 21, scanned 21, filtered 17, groups 3, out 3

#Using it as a library

match @exec.query_text(source, "where level == \"error\" | stats count() as n by service") {
  Ok(result) => println(@render.render_table(result))
  Err(message) => println("query error: " + message)
}

let dataset = @text.parse_auto(source, format="logfmt")
let query = @query.Query::parse(text).unwrap()
let index = @index.TableIndex::build(dataset.table)
let plan = @plan.compile(query, dataset.table, Some(index))
println(plan.explain())
let result = @exec.run(query, dataset.table, Some(index))

#What it does not do

• It is not a streaming engine. A whole input is read into memory before a query runs, so the working set is the size of the data. --limit caps how many records are read, which helps, but there is no windowed or incremental execution yet.
• It is not a full regular expression engine. =~ uses the toolchain's matcher; a pattern that will not compile falls back to a substring test rather than failing the query.
• It does not write. There is no INSERT, UPDATE or DELETE, and no storage layer; MoonLoom reads text and answers questions about it.
• Joins are out of scope. One query looks at one input.
• The index answers conjunctions only. A predicate under or is evaluated by the filter instead of the index, which is correct but slower.
• Group ordering is by group value, not by first appearance, so results are reproducible regardless of input order. A query that needs another order says so with sort.
• Display width is approximate. The renderer implements the common East Asian Width ranges, not the full property, so a handful of emoji may still misalign a column.
• Floats in a table are rounded to six decimals for readability; the JSON and CSV exports keep the value unchanged.

#Reproducing the checks

moon version --all
moon check --deny-warn --target all
moon build --target wasm
moon test --deny-warn --target wasm
moon test --deny-warn --target wasm-gc
moon test --deny-warn --target js
moon fmt && git diff --exit-code
moon info && git diff --exit-code
moon run --target wasm examples/demo

#Related work

• sundaysebasidian-byte/moon-loglens streams JSON Lines with filters and grouped metrics. It reads one format, has no query parser, no plan, no index and no command line, so filters are expressed by calling its API rather than by writing a query.
• ihb2032/MoonFrame is a DataFrame with an expression engine over structured data. It expects tabular input; recovering a table from an access log, a syslog file or a pattern is left to the caller.
• sqhyyy/moonlogfmt-lens parses, redacts and diffs logfmt specifically.
• moonbit-community/sqlparser is an extensible SQL parser with no executor.

#Environment note

#License