marianoguerra/wax/syntax/trivia does not have a README file

    Associated

    pub(all) struct Associated {
    before : Array[Entry]
    within : Array[Entry]
    after : Array[Entry]
    } derive(Eq,
    Debug
    )

    Where a comment sits relative to the node that owns it.

    Context

    pub struct Context {
    entries : Array[Entry]
    at_start_of_line : Bool
    prev_token_end : Int
    locations : Array[
    Location
    ]
    }

    Collection state, threaded through lexing.

    The reference makes the grammar a Menhir functor over this context so every semantic action can register spans. MoonBit has no functors, so phase 2 will hold it in a package-level Ref instead — see parser/parser.mbty.

    Context::entries

    fn Context::entries(self : Context) -> ArrayView[Entry]

    The collected trivia, in source order.

    The reference accumulates onto the front of a list and reverses on use; pushing onto an array keeps source order directly.

    Context::locations

    Context::new

    fn Context::new() -> Context

    Context::record_pos

    fn Context::record_pos(self : Context, loc :
    Location
    ) -> Unit

    Record that an AST node spans loc, so trivia can be attached to it later.

    Context::report_item

    fn Context::report_item(self : Context, kind : TriviaKind, loc :
    Location
    , content : String) -> Unit

    Record a comment (or annotation) spanning loc.

    A line comment runs to the end of its line, so it leaves the lexer at the start of the next one; a block comment does not.

    Context::report_newline

    fn Context::report_newline(self : Context) -> Unit

    Record a newline.

    A newline while already at the start of a line means the line just ended was empty, which is the only way a blank line is detected.

    Context::report_token

    fn Context::report_token(self : Context, pos : Int) -> Unit

    Record that a real token ending at byte offset pos was consumed.

    Entry

    pub(all) struct Entry {
    anchor : Int
    trivia : Trivia
    position : TriviaPos
    } derive(Eq,
    Debug
    )

    Locations

    pub struct Locations {
    marked :
    Set
    [SpanKey]
    }

    The set of spans the printer actually looks up.

    Filled by a dry printing pass; see associate.

    Locations::contains

    Locations::mark

    Locations::new

    fn Locations::new() -> Locations

    SpanKey

    A span reduced to its byte offsets.

    Keying on offsets alone is deliberate: it is cheap to hash and compare, and the filename is the same for every span in one parse.

    Table

    pub struct Table {
    entries : Map[SpanKey, Associated]
    seen :
    Set
    [SpanKey]
    }

    A location-keyed trivia table.

    Table::empty

    fn Table::empty() -> Table

    Table::get

    The trivia attached to loc, or nothing.

    A span yields its trivia ONCE: several nodes can record the same range (a Get instruction and the identifier it wraps span the same name), the printer looks each up, and only the first may carry the comments.

    Trivia

    pub(all) enum Trivia {
    Item(content~ : String, kind~ : TriviaKind, loc~ :
    Location
    )
    BlankLine
    } derive(Eq,
    Debug
    )

    TriviaKind

    pub(all) enum TriviaKind {
    LineComment
    BlockComment
    Annotation
    } derive(Eq,
    Debug
    )

    TriviaPos

    pub(all) enum TriviaPos {
    LineStart
    Inline
    } derive(Eq,
    Debug
    )

    Whether a piece of trivia began a line or trailed a token on one.

    The printer needs the distinction: a comment on its own line stays on its own line, while one trailing foo; // why has to be held back and emitted after the separator rather than before it.

    associate

    fn associate(ctx : Context, collect : (Locations) -> Unit) -> (Table, Array[Entry])

    Associate collected trivia with the spans the printer will look up.

    collect is a dry printing pass that records those spans. Association runs over the spans that are BOTH a parse node and looked up by the printer -- the two sets clip each other in opposite directions and neither alone will do:

    • dropping a parse node the printer skips keeps its comments from being silently lost; they bubble up to an enclosing node that does print.
    • dropping a looked-up span that no parse node owns keeps a comment from landing somewhere that is not a source construct. The printer stamps some output with the span of a mere TOKEN, and such a span can outrank the real nodes and steal the file's leading comments.

    Returns the table and the leftovers no location owns -- trailing comments past the last node, or the whole file when there are no nodes at all. The caller prints those as tail trivia.

    drop_trailing_blank_lines

    fn drop_trailing_blank_lines(entries : Array[Entry]) -> Array[Entry]

    Drop trailing blank lines from a run of trivia.

    empty_assoc

    let empty_assoc : Associated