moonast

    Code generation for MoonBit from annotated declarations.

    codegen
    code-generation
    code-generator
    source-generation
    proc-macro
    derive-macro
    annotation-processor
    ast
    macro
    macros
    derive
    attribute
    attributes
    annotation
    annotations
    metaprogramming
    quote
    syntax-tree
    parser
    moonbit
    Download zip
    Version
    0.2.7
    License
    Apache-2.0
    Last updated
    12 hours ago
    Downloads
    3K

    #$# moonast

    The annotation writes the file.

    #demo.describe above a struct  ·  a file of methods out


    #What it does

    A generator reads declarations you have already written, finds the annotations on them, and writes the code those annotations ask for. moonast is the three pieces that make that possible, one job each:

    LayerPackageWhat it does
    Readmoonast/genParses a file and gives every declaration as a plain view: Struct, Enum, Func. Types are kept as the source text that wrote them.
    Writemoonast/quoteA template is real MoonBit with $holes. It is parsed, each hole is filled by replacing the node it stands for, and the official printer lays the result out.
    Generatemoonast/gen, moonast/clirender puts a header above the code; run is the whole main of a generator binary.

    Between them, moonast/attr turns an annotation's payload into typed arguments.

    #A complete derive

    The generator is written once, as a module of its own. A project that uses it writes one declaration, one rule in moon.mod, and one line in moon.pkg.

    #1 · The declaration you annotate

    // models.mbt
    #demo.describe
    pub struct Point {
    x : Int
    y : Int
    }

    #2 · The generator, written once

    // src/main.mbt — give every annotated struct a `describe` method
    struct Describe {}

    impl @gen.Generator for Describe with fn name(_) {
    "describe"
    }

    impl @gen.Generator for Describe with fn generate(_, decls) {
    let out = []
    for decl in decls {
    guard decl is Struct(name~, fields~, attrs~, ..) &&
    @attr.find(attrs, "demo.describe") is Some(_) else {
    continue
    }
    let text = name + "(" + fields.map(f => f.name).join(", ") + ")"
    let source =
    #|fn $T::describe(self : $T) -> String {
    #| ignore(self)
    #| $text
    #|}
    out.push(
    @quote.items(source)
    .set("T", @quote.ident(name))
    .set("text", @quote.string(text))
    .render(),
    )
    }
    @quote.join(out)
    }

    async fn main {
    @cli.run(Describe::{}, @env.args()[1:])
    }

    // moon.pkg of that package import { "moonbitstack/moonast/quote", "moonbitstack/moonast/attr", "moonbitstack/moonast/gen", "moonbitstack/moonast/cli", "moonbitlang/async", "moonbitlang/core/env", }

    #3 · The rules that keep it up to date

    // moon.mod of the project that uses the generator rule(name: "describe", command: "moonx <module>@<version> -C \"$mod_dir\" \"$input\" -o \"$output\"")

    // moon.pkg of the package whose declarations it reads dev_build(rule: "describe", input: "models.mbt", output: "describe.g.mbt") formatter(ignore: [ "describe.g.mbt" ])

    #4 · The file it writes

    Running moon run . --target native -- models.mbt -o describe.g.mbt writes:

    // Generated by describe from models.mbt. Edit those files, not this one.

    ///|
    fn Point::describe(self : Point) -> String {
    ignore(self)
    "Point(x, y)"
    }

    From then on moon check, moon build and moon run regenerate it whenever models.mbt changes.

    #An annotation that takes arguments

    #demo.getters(skip=token) gives every field a getter except the ones named. The payload is read against a spec, so a missing or mistyped argument is refused with the annotation's line and column instead of producing code:

    impl @gen.Generator for Getters with fn generate(_, decls) {
    let out = []
    for decl in decls {
    guard decl is Struct(name~, fields~, attrs~, ..) else { continue }
    guard @attr.find(attrs, "demo.getters") is Some(annotation) else { continue }
    let skip = @attr.Spec::new("demo.getters")
    .many("skip", Ident)
    .bind(annotation)
    .strs("skip")
    for f in fields {
    guard !skip.contains(f.name) else { continue }
    let source =
    #|fn $T::$field(self : $T) -> $Ty {
    #| self.$field
    #|}
    out.push(
    @quote.items(source)
    .set("T", @quote.ident(name))
    .set("field", @quote.ident(f.name))
    .set("Ty", @quote.ty(f.ty).render())
    .render(),
    )
    }
    }
    @quote.join(out)
    }

    #demo.getters(skip=token)
    pub struct User {
    id : Int
    name : String
    token : String
    }

    // Generated by getters from models.mbt. Edit those files, not this one.

    ///|
    fn User::id(self : User) -> Int {
    self.id
    }

    ///|
    fn User::name(self : User) -> String {
    self.name
    }

    #The three packages on their own

    A package that builds code can use any of them without a generator binary. Each block below is what this page runs before publishing.

    A template is filled and rendered; render gives a Code, and its text is what the printer writes:

    let source =
    #|fn $T::$field(self : $T) -> $Ty {
    #| self.$field
    #|}
    let field = @quote
    .items(source)
    .set("T", @quote.ident("User"))
    .set("field", @quote.ident("id"))
    .set("Ty", @quote.ty("Int").render())
    .render()
    assert_eq(
    field.to_string(),
    "///|\nfn User::id(self : User) -> Int {\n self.id\n}\n",
    )

    An annotation's arguments are read against a spec, and an argument that does not fit is refused rather than handed over as text:

    let users =
    #|#demo.getters(skip=token)
    #|pub struct User {
    #| id : Int
    #| token : String
    #|}
    guard @gen.read("models.mbt", users) is [decl] else { abort("one declaration") }
    guard @attr.find(decl.attrs(), "demo.getters") is Some(annotation) else {
    abort("the struct is annotated")
    }
    let skipped = @attr
    .Spec::new("demo.getters")
    .many("skip", Ident)
    .bind(annotation)
    .strs("skip")
    assert_eq(skipped.join(", "), "token")

    A declaration's body is one of four shapes, and a generator that writes an implementation per shape asks for it once:

    let shapes =
    #|pub struct Rgb(Int, Int, Int)
    #|
    #|pub enum Shape {
    #| Empty
    #| Circle(Double)
    #|}
    let kinds = @gen.read("shapes.mbt", shapes).map(decl => {
    match decl.shape() {
    Some(Tuple(arity~)) => "\{decl.name()} takes \{arity}"
    Some(Cases(cases~)) => "\{decl.name()} has \{cases.length()} cases"
    _ => "nothing to write"
    }
    })
    assert_eq(kinds.join("; "), "Rgb takes 3; Shape has 2 cases")

    #Interface

    #moonast/gen

    ItemTakesGives
    read(file, source)a file name and its sourceits declarations, or Refused
    render(generator, inputs)a generator and its inputsthe file to write
    Generator::name()—the name in the header
    Generator::generate(decls)every declarationthe code to write
    Decl—Struct, Enum, Func or Other
    Decl::shape()—Empty, Tuple, Record or Cases: what a type's body is
    Decl::params()—the generic parameters, ["A", "B"]
    Decl::name(), Decl::is_public()—what the declaration introduces, for the tuple structs Other holds
    Decl::attrs(), Decl::raw()—the annotations on it, the syntax tree behind it
    Decl::raises()—what a function says after its return type: empty, raise or raise E
    Field—a struct field: its name, type as written, mutable, optional, annotations
    Shape::Cases holds Case—one variant: its name, the Shape it carries, its annotations
    Param, Pass—a parameter and how it is passed
    Param::declared(), Param::passed()—a parameter as a wrapper declares it and passes it on

    #moonast/quote

    ItemTakesGives
    items(source)MoonBit declarations with holesa Template
    expr(source)one expression with holesa Template
    ty(source)one type with holesa Template
    Template::set(hole, code)a hole's name, a Codethe template, filled
    Template::splice(hole, codes)a list hole's name, codesthe template, spliced
    Template::render()—a Code, or Refused
    ident(name)a name, or @pkg.namea Code, or Refused
    string(text)any texta Code
    bytes(text)any text, as UTF-8a b"…" Code whose bytes outside ASCII are \xNN
    int(value), bool(value)a literala Code
    join(codes)declaration codesone Code, or Refused
    Code::items()—the official syntax tree, when it is declarations
    Code::to_string()—the source text

    #moonast/attr

    ItemTakesGives
    name_of(attr)an annotationits full name, lib.name
    find(attrs, name)annotations, a namethe first one called that
    all(attrs, name)annotations, a nameevery one called that
    under(attrs, lib)annotations, a prefixevery one under lib.
    malformed(attrs, lib)annotations, a prefixevery one under lib. that does not parse
    Spec::new(name)an annotation's namea spec with no parameters
    Spec::positional(key, kind, optional?)by position or name, required unless optional=truethe spec
    Spec::required(key, kind)required, by namethe spec
    Spec::optional(key, kind)may be left outthe spec
    Spec::many(key, kind, positional?)may be given again, by name or as bare argumentsthe spec
    Spec::bind(attr)an annotationits arguments, or Refused
    Args::get, str, strs, bool, hasa parameter's keywhat was given for it

    Kind is Str, Bool, Ident, Path or Type; each Args reader answers in the kind its spec asked for. Path is a name after a qualifier, contract=registry.create, which is how a value in another namespace is written: the annotation grammar has no @pkg.name.

    #moonast/cli

    ItemTakesGives
    run(generator, args)a generator, [-C <dir>] <inputs...> -o <output> [--check]writes the file, or refuses when it is stale

    #Holes

    HoleTakesWritten asGives
    $Ta type or a constructorfn f(x : $T)fn f(x : Int)
    $namea value, an expression, or a name$value + 141 + 1
    @pkg.$namea name inside a packageapp : @web.$Tapp : @web.App
    $name*a list of them[$items*][1, 2, 3]
    $name in a stringa name or a string, as text"field $key""field first name"
    $name in a patterna name, a string, or a literal$case => 1Empty => 1
    $$a literal dollar sign"$$5""$5"

    A hole's case is part of the grammar: upper case where a type or constructor goes, lower case where a value or a binder goes. A list hole expands among statements, array elements, and the arguments of a function application.

    #Annotation arguments

    The parser reads strings, true/false, names and nested calls as annotation arguments. There are no arrays and no numbers, a generic type has to be written as a string, and a keyword cannot be a label. A list is written as a repeated key, tags="a", tags="b", which Spec::many accepts, or as bare arguments, #demo.derive(A, B), which Spec::many accepts with positional=true.

    When a payload is outside that grammar, the annotation has no name as far as moonast is concerned, so find does not see it. malformed does: a generator asks it first and reports what it finds.

    #Refusals

    Every fallible call gives a Refused with a message, and with a position where there is one to give:

    FromRefused when
    quotethe template is not MoonBit, a hole is left empty, a fill names no hole, or a fill does not fit where it stands
    attrthe payload cannot be read, a required parameter is missing, one is unknown, mistyped, or given twice
    genan input file is not MoonBit
    clithe command line is wrong, --check found the output out of date, or the generator refused, in which case Generator carries its sentence

    #Examples

    DirectoryWhat it is
    src/examples/tourThe whole surface in one run: read, bind, generate.
    src/examples/describeA minimal derive generator: #demo.describe gives a struct describe.
    src/examples/gettersA generator whose annotation takes arguments: #demo.getters(skip=token).
    src/examples/serde-deriveA data-model derive: #serde.derive(Serialize, Deserialize), with #serde.rename and #serde.skip.
    src/examples/proton-registrarA command registrar: #proton.command(contract=…) gathers the annotated functions into register_commands.
    src/examples/playThe same three packages compiled to JavaScript, which is what the site's playground runs.

    #Licence

    Apache-2.0.