🥮 mooncakes.io
    Sign in

    tree_sitter_mbt

    MoonBit-native Tree-sitter runtime and generator (work in progress)

    tree-sitter
    parser
    incremental
    Download zip
    Version
    0.1.0
    License
    Apache-2.0
    Repository
    Last updated
    5 hours ago
    Downloads
    4

    #moonbit-community/tree_sitter_mbt

    MoonBit-native Tree-sitter runtime and generator work in progress.

    This repository targets upstream Tree-sitter 0.27.0 at commit 4455da71029b13fa113956d1470c75c85db8fdc5. The mapped upstream source roots are:

    • runtime: ../tree-sitter/lib/src
    • public C API reference: ../tree-sitter/lib/include/tree_sitter/api.h
    • generator: ../tree-sitter/crates/generate/src
    • CLI: ../tree-sitter/crates/cli/src

    The v1 native target is Tree-sitter language ABI 15. The MoonBit core packages stay platform-IO free; file IO is isolated in CLI-facing packages.

    #Package Layout

    • api: shared public value types such as points, ranges, edits, encodings, and callback-facing scanner errors.
    • root package: stable opaque runtime facade types such as Language, Parser, Tree, Node, and TreeCursor.
    • runtime: internal runtime implementation package.
    • runtime/tables: generated table construction, validation, metadata, and scanner descriptor boundary.
    • grammar: grammar.json parsing and normalized input grammar IR.
    • generator: MoonBit-native generator entry point returning generated sources and metadata without writing files.
    • query: query parser, cursor, captures, and predicate filtering helpers.
    • highlight: highlight configuration and event generation over query/runtime.
    • tags: tags configuration and tag extraction over query/runtime.
    • cli: in-memory command argument parsing and command runners.
    • cli/native: file-backed adapters around the CLI command runners.
    • cmd/tree-sitter-mbt: executable package; currently supports file-backed generate, parse, test, query, and tags. Dynamic language loading is still rejected; grammar.json and tree-sitter.json based loading is supported.
    • test_fixtures: embedded fixture constants for all-target tests.

    Generated parser packages should import the root package and runtime/tables. Packages with external tokens should also import api for callback-facing types. Import aliases are generator options, not hard-coded source paths.

    #Runtime Boundary

    The root package intentionally owns the stable Language, Parser, Tree, Node, and TreeCursor facade in v1. Parser/tree/node implementation code currently lives in the root package rather than being fully moved under runtime; this is a documented package-boundary divergence while the v1 table model and corpus behavior stabilize.

    #CLI Boundary

    The cli package remains an all-target, in-memory command layer. It may depend on the root package, generator, query, highlight, and tags, but it does not perform direct filesystem operations. The cli/native package owns file reads and writes via moonbitlang/async/fs and delegates to cli runners. Native generate reads an adjacent tree-sitter.json when present and uses metadata.version for generated language metadata; if that file is absent, the metadata defaults to 0.0.0.

    IncludedRangesError

    source
    pub(all) suberror IncludedRangesError {
      InvalidRange(index~ : Int, range~ : 
    Range    )
      Overlap(index~ : Int, previous~ : 
    Range    , current~ : 
    Range    )
    } derive(Eq, 
    Debug    )

    LanguageError

    source
    pub(all) suberror LanguageError {
      UnsupportedAbi(Int)
    } derive(Eq, 
    Debug    )

    ParseError

    source
    pub(all) suberror ParseError {
      NoLanguage
      LanguageMismatch
      Cancelled(ParseState)
      InvalidInputCallback(byte_offset~ : UInt, point~ : 
    Point    , reason~ : String)
      InvalidInputDecode(byte_offset~ : UInt, reason~ : String)
      ScannerError(
    ScannerError    )
      LexError(byte_offset~ : UInt, state_id~ : 
    StateId    )
      ParseTableError(state_id~ : 
    StateId    , symbol_id~ : 
    SymbolId    )
      ParseStackUnderflow
    } derive(Eq, 
    Debug    )

    Input

    source
    type Input

    Input::bytes

    source
    fn Input::bytes(self : Input) -> Bytes

    Input::encoding

    source

    Input::from_bytes

    source

    Input::from_callback

    source

    Input::from_custom_bytes

    source
    fn Input::from_custom_bytes(source : Bytes, decode : (Bytes, Int) -> InputDecodeResult) -> Input

    Input::from_custom_callback

    source
    fn Input::from_custom_callback(read : (UInt, 
    Point    ) -> InputChunk, decode : (Bytes, Int) -> InputDecodeResult) -> Input

    Input::from_string

    source
    fn Input::from_string(source : String, encoding? : 
    InputEncoding    ) -> Input

    Input::source

    source
    fn Input::source(self : Input) -> String

    InputChunk

    source
    pub(all) struct InputChunk {
      chunk : Bytes?
      bytes_read : Int
    } derive(Eq, 
    Debug    )

    InputChunk::eof

    source
    fn InputChunk::eof() -> InputChunk

    InputChunk::from_bytes

    source
    fn InputChunk::from_bytes(chunk : Bytes) -> InputChunk

    InputChunk::new

    source
    fn InputChunk::new(chunk~ : Bytes?, bytes_read~ : Int) -> InputChunk

    InputDecodeResult

    source
    pub(all) enum InputDecodeResult {
      CodePoint(Int, bytes~ : Int)
      DecodeError
    } derive(Eq, 
    Debug    )

    Language

    source
    type Language

    Language::abi_version

    source
    fn Language::abi_version(self : Language) -> Int

    Language::external_token_count

    source
    fn Language::external_token_count(self : Language) -> Int

    Language::field_count

    source
    fn Language::field_count(self : Language) -> Int

    Language::field_id_for_name

    source
    fn Language::field_id_for_name(self : Language, name : String) -> 
    FieldId    ?

    Language::field_name_for_id

    source
    fn Language::field_name_for_id(self : Language, id : 
    FieldId    ) -> String?

    Language::from_validated_tables

    source

    Language::lookahead_iterator

    source

    Language::metadata

    source

    Language::name

    source
    fn Language::name(self : Language) -> String

    Language::next_state

    source

    Language::state_count

    source
    fn Language::state_count(self : Language) -> Int

    Language::subtypes

    source

    Language::supertypes

    source

    Language::symbol_count

    source
    fn Language::symbol_count(self : Language) -> Int

    Language::symbol_for_name

    source
    fn Language::symbol_for_name(self : Language, name : String, is_named~ : Bool) -> 
    SymbolId    ?

    Language::symbol_name

    source

    Language::symbol_type

    source

    Language::token_count

    source
    fn Language::token_count(self : Language) -> Int

    LogType

    source
    pub(all) enum LogType {
      Lex
      Parse
    } derive(Eq, 
    Debug    )

    Logger

    source
    type Logger

    Logger::log

    source
    fn Logger::log(self : Logger, kind : LogType, message : String) -> Unit

    Logger::new

    source
    fn Logger::new(callback : (LogType, String) -> Unit) -> Logger

    LookaheadIterator

    source
    type LookaheadIterator

    LookaheadIterator::current_symbol

    source

    LookaheadIterator::current_symbol_name

    source
    fn LookaheadIterator::current_symbol_name(self : LookaheadIterator) -> String?

    LookaheadIterator::language

    source

    LookaheadIterator::next

    source
    fn LookaheadIterator::next(self : LookaheadIterator) -> Bool

    LookaheadIterator::reset

    source

    LookaheadIterator::reset_state

    source

    LookaheadIterator::state

    source

    Node

    source
    type Node

    Node::child

    source
    fn Node::child(self : Node, index : Int) -> Node

    Node::child_by_field_id

    source
    fn Node::child_by_field_id(self : Node, field_id : 
    FieldId    ) -> Node

    Node::child_by_field_name

    source
    fn Node::child_by_field_name(self : Node, name : String) -> Node

    Node::child_count

    source
    fn Node::child_count(self : Node) -> Int

    Node::child_with_descendant

    source
    fn Node::child_with_descendant(self : Node, descendant : Node) -> Node

    Node::descendant_count

    source
    fn Node::descendant_count(self : Node) -> Int

    Node::descendant_for_byte_range

    source
    fn Node::descendant_for_byte_range(self : Node, start_byte : UInt, end_byte : UInt) -> Node

    Node::descendant_for_point_range

    source
    fn Node::descendant_for_point_range(self : Node, start_point : 
    Point    , end_point : 
    Point    ) -> Node

    Node::edit

    source

    Node::end_byte

    source
    fn Node::end_byte(self : Node) -> UInt

    Node::end_point

    source

    Node::eq

    source
    fn Node::eq(self : Node, other : Node) -> Bool

    Node::field_name_for_child

    source
    fn Node::field_name_for_child(self : Node, index : Int) -> String?

    Node::field_name_for_named_child

    source
    fn Node::field_name_for_named_child(self : Node, index : Int) -> String?

    Node::first_child_for_byte

    source
    fn Node::first_child_for_byte(self : Node, byte : UInt) -> Node

    Node::first_child_for_point

    source
    fn Node::first_child_for_point(self : Node, point : 
    Point    ) -> Node

    Node::first_named_child_for_byte

    source
    fn Node::first_named_child_for_byte(self : Node, byte : UInt) -> Node

    Node::first_named_child_for_point

    source
    fn Node::first_named_child_for_point(self : Node, point : 
    Point    ) -> Node

    Node::grammar_symbol

    source

    Node::grammar_type

    source
    fn Node::grammar_type(self : Node) -> String

    Node::has_changes

    source
    fn Node::has_changes(self : Node) -> Bool

    Node::has_error

    source
    fn Node::has_error(self : Node) -> Bool

    Node::is_error

    source
    fn Node::is_error(self : Node) -> Bool

    Node::is_extra

    source
    fn Node::is_extra(self : Node) -> Bool

    Node::is_missing

    source
    fn Node::is_missing(self : Node) -> Bool

    Node::is_named

    source
    fn Node::is_named(self : Node) -> Bool

    Node::is_null

    source
    fn Node::is_null(self : Node) -> Bool

    Node::language

    source
    fn Node::language(self : Node) -> Language

    Node::named_child

    source
    fn Node::named_child(self : Node, index : Int) -> Node

    Node::named_child_count

    source
    fn Node::named_child_count(self : Node) -> Int

    Node::named_descendant_for_byte_range

    source
    fn Node::named_descendant_for_byte_range(self : Node, start_byte : UInt, end_byte : UInt) -> Node

    Node::named_descendant_for_point_range

    source
    fn Node::named_descendant_for_point_range(self : Node, start_point : 
    Point    , end_point : 
    Point    ) -> Node

    Node::next_named_sibling

    source
    fn Node::next_named_sibling(self : Node) -> Node

    Node::next_parse_state

    source

    Node::next_sibling

    source
    fn Node::next_sibling(self : Node) -> Node

    Node::parent

    source
    fn Node::parent(self : Node) -> Node

    Node::parse_state

    source

    Node::previous_named_sibling

    source
    fn Node::previous_named_sibling(self : Node) -> Node

    Node::previous_sibling

    source
    fn Node::previous_sibling(self : Node) -> Node

    Node::start_byte

    source
    fn Node::start_byte(self : Node) -> UInt

    Node::start_point

    source

    Node::symbol

    source

    Node::to_sexp

    source
    fn Node::to_sexp(self : Node) -> String

    Node::type_

    source
    fn Node::type_(self : Node) -> String

    ParseOptions

    source
    pub(all) struct ParseOptions {
      cancellation_state : ParseState?
      progress_callback : (ParseState) -> Bool?
    }

    ParseOptions::cancelled

    source
    fn ParseOptions::cancelled(state : ParseState) -> ParseOptions

    ParseOptions::default

    source
    fn ParseOptions::default() -> ParseOptions

    ParseOptions::progress

    source
    fn ParseOptions::progress(callback : (ParseState) -> Bool) -> ParseOptions

    ParseOptions::with_progress

    source
    fn ParseOptions::with_progress(self : ParseOptions, callback : (ParseState) -> Bool) -> ParseOptions

    ParseState

    source
    pub(all) struct ParseState {
      current_byte_offset : UInt
      has_error : Bool
    } derive(Eq, 
    Debug    )

    ParseState::initial

    source
    fn ParseState::initial() -> ParseState

    ParseState::new

    source
    fn ParseState::new(current_byte_offset~ : UInt, has_error~ : Bool) -> ParseState

    Parser

    source
    type Parser

    Parser::included_ranges

    source

    Parser::language

    source
    fn Parser::language(self : Parser) -> Language?

    Parser::logger

    source
    fn Parser::logger(self : Parser) -> Logger?

    Parser::new

    source
    fn Parser::new() -> Parser

    Parser::parse

    source
    fn Parser::parse(self : Parser, input : Input, old_tree? : Tree?, options? : ParseOptions) -> Tree raise ParseError

    Parser::parse_bytes

    source
    fn Parser::parse_bytes(self : Parser, input : Bytes, old_tree? : Tree?, options? : ParseOptions) -> Tree raise ParseError

    Parser::parse_string

    source
    fn Parser::parse_string(self : Parser, input : String, old_tree? : Tree?, options? : ParseOptions) -> Tree raise ParseError

    Parser::reset

    source
    fn Parser::reset(self : Parser) -> Unit

    Parser::set_included_ranges

    source
    fn Parser::set_included_ranges(self : Parser, ranges : Array[
    Range    ]) -> Unit raise IncludedRangesError

    Parser::set_language

    source
    fn Parser::set_language(self : Parser, language : Language) -> Unit raise LanguageError

    Parser::set_logger

    source
    fn Parser::set_logger(self : Parser, logger : Logger?) -> Unit

    Tree

    source
    type Tree

    Tree::changed_ranges

    source
    fn Tree::changed_ranges(self : Tree, other : Tree) -> Array[
    Range    ]

    Tree::copy

    source
    fn Tree::copy(self : Tree) -> Tree

    Tree::edit

    source

    Tree::included_ranges

    source

    Tree::language

    source
    fn Tree::language(self : Tree) -> Language

    Tree::root_node

    source
    fn Tree::root_node(self : Tree) -> Node

    Tree::root_node_with_offset

    source
    fn Tree::root_node_with_offset(self : Tree, byte_offset~ : UInt, point_offset~ : 
    Point    ) -> Node

    Tree::to_sexp

    source
    fn Tree::to_sexp(self : Tree) -> String

    Tree::walk

    source
    fn Tree::walk(self : Tree) -> TreeCursor

    TreeCursor

    source
    type TreeCursor

    TreeCursor::copy

    source
    fn TreeCursor::copy(self : TreeCursor) -> TreeCursor

    TreeCursor::current_depth

    source
    fn TreeCursor::current_depth(self : TreeCursor) -> Int

    TreeCursor::current_descendant_index

    source
    fn TreeCursor::current_descendant_index(self : TreeCursor) -> Int

    TreeCursor::current_field_id

    source

    TreeCursor::current_field_name

    source
    fn TreeCursor::current_field_name(self : TreeCursor) -> String?

    TreeCursor::current_node

    source
    fn TreeCursor::current_node(self : TreeCursor) -> Node

    TreeCursor::goto_descendant

    source
    fn TreeCursor::goto_descendant(self : TreeCursor, descendant_index : Int) -> TreeCursor?

    TreeCursor::goto_first_child

    source
    fn TreeCursor::goto_first_child(self : TreeCursor) -> TreeCursor?

    TreeCursor::goto_first_child_for_byte

    source
    fn TreeCursor::goto_first_child_for_byte(self : TreeCursor, byte : UInt) -> TreeCursor?

    TreeCursor::goto_first_child_for_byte_with_index

    source
    fn TreeCursor::goto_first_child_for_byte_with_index(self : TreeCursor, byte : UInt) -> (Int, TreeCursor)?

    TreeCursor::goto_first_child_for_point

    source
    fn TreeCursor::goto_first_child_for_point(self : TreeCursor, point : 
    Point    ) -> TreeCursor?

    TreeCursor::goto_first_child_for_point_with_index

    source
    fn TreeCursor::goto_first_child_for_point_with_index(self : TreeCursor, point : 
    Point    ) -> (Int, TreeCursor)?

    TreeCursor::goto_last_child

    source
    fn TreeCursor::goto_last_child(self : TreeCursor) -> TreeCursor?

    TreeCursor::goto_next_sibling

    source
    fn TreeCursor::goto_next_sibling(self : TreeCursor) -> TreeCursor?

    TreeCursor::goto_parent

    source
    fn TreeCursor::goto_parent(self : TreeCursor) -> TreeCursor?

    TreeCursor::goto_previous_sibling

    source
    fn TreeCursor::goto_previous_sibling(self : TreeCursor) -> TreeCursor?

    TreeCursor::reset

    source
    fn TreeCursor::reset(self : TreeCursor, node : Node) -> TreeCursor

    TreeCursor::reset_to

    source
    fn TreeCursor::reset_to(self : TreeCursor, other : TreeCursor) -> TreeCursor

    language_from_generated_tables

    source

    Source Files

    Powered by MoonBit

    Site sourceReport issuePackagesBuild queueSkillsStatistics

    © 2026 mooncakes.io