tmgrammar

TextMate grammar based syntax highlighter for MoonBit

textmate
grammar
syntax-highlighting
shiki
moon add mizchi/tmgrammar@0.1.0
Download zip
Author
Version
0.1.0
License
MIT
Last updated
7 months ago
Downloads
28
README

#mizchi/tmgrammar

TextMate grammar based syntax highlighter for MoonBit.

Built on oniguruma-to-es - converts Oniguruma regex patterns to JavaScript RegExp compatible patterns.

Supported targets: js, wasm, native

#Installation

moon add mizchi/tmgrammar

#Module Structure

tmgrammar/ ├── oniguruma/ # Oniguruma regex to JS RegExp converter ├── shiki/ # Shiki-compatible HTML highlighter ├── tui/ # Terminal ANSI highlighter └── grammars/ # Language grammar definitions (37 languages)

#Basic Usage

#Oniguruma Module

{ "import": [ "mizchi/tmgrammar/oniguruma" ] }

// Convert pattern with default options (ES2024 target)

///|
let result = @oniguruma.to_pattern("hello")

// Convert with flags

///|
let result = @oniguruma.to_pattern("pattern", flags="im")

// Convert with specific target

///|
let result = @oniguruma.to_pattern("pattern", target=@oniguruma.ES2018)

#Shiki Module

{ "import": [ "mizchi/tmgrammar/shiki" ] }

// Create highlighted lines

///|
let lines = [
@shiki.HighlightedLine::new([
@shiki.HighlightToken::new("const", scopes=["keyword"]),
@shiki.HighlightToken::new(" x", scopes=["variable"]),
]),
]

// Render to HTML with theme

///|
let html = @shiki.render_to_html(lines, @shiki.dark_plus_theme())

#TUI Module (Terminal ANSI)

{ "import": [ "mizchi/tmgrammar/tui" ] }

// Basic colorization
///|
let colored = @tui.colorize("Hello", fg=@tui.Red, style=@tui.Bold)

// RGB TrueColor support
///|
let rgb_text = @tui.colorize("Orange", fg=@tui.Rgb(255, 165, 0))

// Render tokens with theme
///|
let lines = [
@tui.TermLine::new([
@tui.TermToken::new("fn ", scopes=["keyword"]),
@tui.TermToken::new("main", scopes=["entity.name.function"]),
]),
]
let output = @tui.render_to_ansi(lines, @tui.dark_theme())

// With line numbers
///|
let options = @tui.TermRenderOptions::new(line_numbers=true)
let output = @tui.render_to_ansi(lines, @tui.monokai_theme(), options~)

Available themes: dark_theme(), light_theme(), monokai_theme()

#Grammar Plugin System

Each language grammar is available as a separate module, allowing you to load only the languages you need:

{ "import": [ { "path": "mizchi/tmgrammar/grammars/javascript", "alias": "js_grammar" }, { "path": "mizchi/tmgrammar/grammars/typescript", "alias": "ts_grammar" } ] }

// Get grammar definition

///|
let js = @js_grammar.grammar()

// Compile all patterns to JS RegExp

///|
let compiled = js.compile()

// Access raw patterns

///|
let patterns = @js_grammar.all_patterns()

///|
let count = @js_grammar.pattern_count()

#Available Grammar Modules (37 languages)

Systems & Native
  • grammars/c - C
  • grammars/cpp - C++
  • grammars/rust - Rust
  • grammars/go - Go
  • grammars/swift - Swift

JVM
  • grammars/java - Java
  • grammars/kotlin - Kotlin
  • grammars/scala - Scala

.NET
  • grammars/csharp - C#

Scripting
  • grammars/javascript - JavaScript/JSX
  • grammars/typescript - TypeScript/TSX
  • grammars/python - Python
  • grammars/ruby - Ruby
  • grammars/php - PHP
  • grammars/lua - Lua
  • grammars/bash - Bash/Shell

Functional
  • grammars/haskell - Haskell
  • grammars/elixir - Elixir

Mobile
  • grammars/dart - Dart/Flutter

Web Frontend
  • grammars/html - HTML
  • grammars/css - CSS
  • grammars/scss - SCSS
  • grammars/vue - Vue
  • grammars/svelte - Svelte
  • grammars/astro - Astro

Data & Config
  • grammars/json - JSON
  • grammars/yaml - YAML
  • grammars/toml - TOML
  • grammars/xml - XML
  • grammars/protobuf - Protocol Buffers
  • grammars/graphql - GraphQL
  • grammars/sql - SQL

DevOps & Tooling
  • grammars/docker - Dockerfile
  • grammars/make - Makefile

Other
  • grammars/markdown - Markdown
  • grammars/vim - Vim script
  • grammars/moonbit - MoonBit

#Supported Features

  • Literals and escape sequences
  • Character classes [a-z], [^0-9]
  • Shorthand classes \d, \w, \s
  • Quantifiers *, +, ?, {n,m}
  • Lazy quantifiers *?, +?
  • Groups (), (?:), (?<name>)
  • Lookahead (?=), (?!)
  • Lookbehind (?<=), (?<!)
  • Unicode properties \p{...}, \P{...}
  • Anchors ^, $, \b, \B
  • Alternation |

#TextMate Grammar Support

Tested against real-world TextMate grammar patterns from shikijs/textmate-grammars-themes:

LanguagePatternsPassedSuccess Rate
JavaScript346346100%
TypeScript333333100%
Swift32931997%
PHP32025680%
C++21618385%
Python216216100%
Ruby20520499%
C171171100%
Bash14414198%
Java140140100%
CSS13613599%
Go1259979%
Markdown123123100%
Lua112112100%
Scala112112100%
HTML11210997%
Svelte109109100%
SCSS104104100%
Rust8989100%
Vue7272100%
Vim6666100%
SQL6666100%
GraphQL6262100%
Kotlin5858100%
Make5151100%
YAML454498%
TOML4444100%
MoonBit4343100%
Astro3535100%
Haskell3535100%
Protobuf3333100%
XML3131100%
Elixir2626100%
Dart2626100%
C#2424100%
JSON1919100%
Docker77100%
Total4003386196%

Note: PHP/C++/Go have complex patterns using advanced Oniguruma features.

#See Also