README

marianoguerra/wax/syntax/trivia does not have a README file

#
Associated

pub(all) struct Associated {
before : Array[Entry]
within : Array[Entry]
after : Array[Entry]
} derive(Eq,
Debug
)

Where a comment sits relative to the node that owns it.

#
Context

pub struct Context {
entries : Array[Entry]
at_start_of_line : Bool
prev_token_end : Int
locations : Array[
Location
]
}

Collection state, threaded through lexing.

The reference makes the grammar a Menhir functor over this context so every semantic action can register spans. MoonBit has no functors, so phase 2 will hold it in a package-level Ref instead — see parser/parser.mbty.

#
Context::entries

fn Context::entries(self : Context) -> ArrayView[Entry]

The collected trivia, in source order.

The reference accumulates onto the front of a list and reverses on use; pushing onto an array keeps source order directly.

#
Context::locations

#
Context::new

fn Context::new() -> Context

#
Context::record_pos

fn Context::record_pos(self : Context, loc :
Location
) -> Unit

Record that an AST node spans loc, so trivia can be attached to it later.

#
Context::report_item

fn Context::report_item(self : Context, kind : TriviaKind, loc :
Location
, content : String) -> Unit

Record a comment (or annotation) spanning loc.

A line comment runs to the end of its line, so it leaves the lexer at the start of the next one; a block comment does not.

#
Context::report_newline

fn Context::report_newline(self : Context) -> Unit

Record a newline.

A newline while already at the start of a line means the line just ended was empty, which is the only way a blank line is detected.

#
Context::report_token

fn Context::report_token(self : Context, pos : Int) -> Unit

Record that a real token ending at byte offset pos was consumed.

#
Entry

pub(all) struct Entry {
anchor : Int
trivia : Trivia
position : TriviaPos
} derive(Eq,
Debug
)

#
Locations

pub struct Locations {
marked :
Set
[SpanKey]
}

The set of spans the printer actually looks up.

Filled by a dry printing pass; see associate.

#
Locations::contains

#
Locations::mark

#
Locations::new

fn Locations::new() -> Locations

#
SpanKey

A span reduced to its byte offsets.

Keying on offsets alone is deliberate: it is cheap to hash and compare, and the filename is the same for every span in one parse.

#
Table

pub struct Table {
entries : Map[SpanKey, Associated]
seen :
Set
[SpanKey]
}

A location-keyed trivia table.

#
Table::empty

fn Table::empty() -> Table

#
Table::get

The trivia attached to loc, or nothing.

A span yields its trivia ONCE: several nodes can record the same range (a Get instruction and the identifier it wraps span the same name), the printer looks each up, and only the first may carry the comments.

#
Trivia

pub(all) enum Trivia {
Item(content~ : String, kind~ : TriviaKind, loc~ :
Location
)
BlankLine
} derive(Eq,
Debug
)

#
TriviaKind

pub(all) enum TriviaKind {
LineComment
BlockComment
Annotation
} derive(Eq,
Debug
)

#
TriviaPos

pub(all) enum TriviaPos {
LineStart
Inline
} derive(Eq,
Debug
)

Whether a piece of trivia began a line or trailed a token on one.

The printer needs the distinction: a comment on its own line stays on its own line, while one trailing foo; // why has to be held back and emitted after the separator rather than before it.

#
associate

fn associate(ctx : Context, collect : (Locations) -> Unit) -> (Table, Array[Entry])

Associate collected trivia with the spans the printer will look up.

collect is a dry printing pass that records those spans. Association runs over the spans that are BOTH a parse node and looked up by the printer -- the two sets clip each other in opposite directions and neither alone will do:

  • dropping a parse node the printer skips keeps its comments from being silently lost; they bubble up to an enclosing node that does print.
  • dropping a looked-up span that no parse node owns keeps a comment from landing somewhere that is not a source construct. The printer stamps some output with the span of a mere TOKEN, and such a span can outrank the real nodes and steal the file's leading comments.

Returns the table and the leftovers no location owns -- trailing comments past the last node, or the whole file when there are no nodes at all. The caller prints those as tail trivia.

#
drop_trailing_blank_lines

fn drop_trailing_blank_lines(entries : Array[Entry]) -> Array[Entry]

Drop trailing blank lines from a run of trivia.

#
empty_assoc

let empty_assoc : Associated