regexp

A regular expression engine for MoonBit based on VM execution designed for predictable time complexity

regexp
moon add moonbitlang/regexp@0.3.5
Download zip
Version
0.3.5
License
Apache-2.0
Last updated
6 months ago
Downloads
23K
README

#🔍 regexp.mbt

⚠️ API STABILITY NOTICE This is a alpha release with a stabilizing API. While core functionality is complete and well-tested, API changes may occur in future versions as we refine the implementation.

Regular expression engine for MoonBit — inspired by Russ Cox's regex series.

#⚡ Quick Start

///|
test {
// Compile once, use everywhere
let regexp = @regexp.compile("a(bc|de)f")
guard regexp.match_("xxabcf") is Some(result)
inspect(
result.results(),
content=(
#|[Some("abcf"), Some("bc")]
),
)

// Write a simple split with regexp
fn split(regexp : @regexp.Regexp, target : StringView) -> Array[StringView] {
let result = []
loop target {
"" => ()
str => {
let res = regexp.execute(str)
result.push(res.before())
continue res.after()
}
}
result
}

let re = @regexp.compile("_+")
inspect(
split(re, "1_2__3__4__5_____6"),
content=(
#|["1", "2", "3", "4", "5", "6"]
),
)
}

#🎯 Core API

#Build & Execute

  • compile(pattern) → Creates an Engine
  • engine.execute(text) → Returns MatchResult

#Inspect Results

  • result.matched()Bool
  • result.get(index) → Capture group content
  • result.results() → Iterator over all matches

#Named Groups & Advanced

  • engine.group_by_name(name) → Find group index by name
  • engine.group_count() → Total capture groups
  • result.groups() → Get named group content

#🎪 Syntax Playground

FeatureExampleWhat it does
LiteralsabcMatch exact text
Wildcardsa.c. matches any character
Quantifiersa+, b*, c?One or more, zero or more, optional
Rangesa{2,5}Between 2-5 repetitions
Classes[a-z], [^0-9]Character sets, negated sets
Groups(abc), (?:xyz)Capturing, non-capturing
Named(?<word>abc)Named capture groups
Choicecat\|dogMatch either option
Anchors^start, end$Line boundaries
Escapes\\u{41}, \\u0041Unicode escapes, standard escapes
Unicode Props\\p{L}, \\p{Nd}Unicode general categories
Backrefs ⚠️(.)\\1Reference previous captures

#🌍 Unicode Property Support

Match characters by their Unicode general categories:

///|
test "unicode properties" {
// Matching gc=L
let regex = @regexp.compile("\\p{Letter}+")
inspect(
regex.execute("Hello 世界").results(),
content=(
#|[Some("Hello")]
),
)

// Matching gc=N
let regex = @regexp.compile("\\p{Number}+")
inspect(
regex.execute("123 and 456").results(),
content=(
#|[Some("123")]
),
)
}

Supported Propertes:

#🔄 Backreferences

⚠️ Performance Warning: Backreferences can cause exponential time complexity in worst cases!

///|
test "backreferences" {
// Palindrome detection (simple)
let palindrome = @regexp.compile("^(.)(.)\\2\\1")
inspect(
palindrome.execute("abba").results(),
content=(
#|[Some("abba"), Some("a"), Some("b")]
),
)

// HTML tag matching
let html_regex = @regexp.compile("<([a-zA-Z]+)[^>]*>(.*?)</\\1>")
let result = html_regex.execute("<div class='test'>content</div>")
inspect(
result.results(),
content=(
#|[Some("<div class='test'>content</div>"), Some("div"), Some("content")]
),
)
}

#💡 Real Examples

///|
test "character classes" {
// Email validation (simplified)
let email = @regexp.compile(
(
#|[\w-]+@[\w-]+\.\w+
),
)
let email_result = email.execute("user@example.com").results()
inspect(
email_result,
content=(
#|[Some("user@example.com")]
),
)
// Extract numbers
let numbers = @regexp.compile(
(
#|\d+\.\d{2}
),
)
let result = numbers.execute("Price: $42.99").results()
inspect(
result,
content=(
#|[Some("42.99")]
),
)

// Named captures for parsing
let parser = @regexp.compile(
(
#|(?<year>\d{4})-(?<month>\d{2})-(?<day>\d{2})
),
)
let date_result = parser.execute("2024-03-15")
inspect(
date_result.groups(),
content=(
#|{"year": "2024", "month": "03", "day": "15"}
),
)
}

#🚨 Error Handling

///|
test {
try {
let _ = @regexp.compile("a(b")
// Oops! Missing )
} catch {
RegexpError(err=MissingParenthesis, source_fragment=_) =>
println("Fix your regex! 🔧")
_ => ()
}
}

#⚡ Performance Characteristics

  • Predictable complexity — Designed to avoid catastrophic backtracking (except with backreferences)
  • VM-based — Structured interpreter design
  • Unicode support — Character set and property support

Built with reliability and correctness as primary goals.

#🔍 Implementation Notes

#Behavior Differences from Other Engines

This implementation has some behavior differences compared to other popular regex engines:

  1. Empty Character Class Handling:
    • In JavaScript: [][] is parsed as two character classes with no characters
    • In Golang: [][] is parsed as one character class containing ] and [
    • In MoonBit: we follow the JavaScript interpretation

  2. Empty Alternatives Behavior:
    • Expressions like (|a)* and (|a)+ have specific behavior that may differ from other implementations
    • See Golang issue #46123 for related discussion

  3. Backreferences:
    • Backreferences are supported but may impact the complexity guarantees of the engine

#
RegexpError

pub suberror RegexpError {
RegexpError(Err, StringView)
}

Parsing error exception type

Contains error information and related string view context
impl Show for RegexpError

#
Err

pub enum Err {
InternalError
InvalidCharClass
InvalidEscape
InvalidNamedCapture
InvalidRepeatOp
InvalidRepeatSize
MissingBracket
MissingParenthesis
MissingRepeatArgument
TrailingBackslash
UnexpectedParenthesis
}

Regular expression parsing error types

Defines various error conditions that may be encountered during parsing
impl Show for Err

#
MatchResult

type MatchResult

Regular expression match result.

Contains the complete result information of a regular expression match, including:
  • The original input text.
  • An array of capture group position information.
  • A map of named capture groups.

Structure of the capture group array:
  • Index 0, 1: Start and end positions of the full match.
  • Index 2, 3: Start and end positions of the first capture group.
  • Index 4, 5: Start and end positions of the second capture group.
  • And so on...

#
MatchResult::after

fn MatchResult::after(self : MatchResult) -> StringView

Gets the remaining text after the match.

If the match was not successful, this will return the empty string.

test {
let engine = compile("hello")
let result = engine.execute("hello world")
inspect(result.after(), content=" world")
let result = engine.execute("say yes")
inspect(result.after(), content="")
}

#
MatchResult::before

fn MatchResult::before(self : MatchResult) -> StringView

Gets the text before the match.

If the match was not successful, this will return the entire input text.

test {
let engine = compile("world")
let result = engine.execute("hello world")
inspect(result.before(), content="hello ")
let result = engine.execute("say hello")
inspect(result.before(), content="say hello")
}

#
MatchResult::get

fn MatchResult::get(self : MatchResult, index : Int) -> StringView?

Gets the content of a capture group by its index.

Returns the corresponding matched text based on the capture group's index. Index 0 represents the full match, while indices 1 and above represent individual capture groups.

Args:
  • self: The match result object.
  • index: The index of the capture group (0 for the full match).

Returns:
  • Some(view): The text view corresponding to the capture group.
  • None: If the index is invalid or the capture group did not match.

Example:
test {
let result = compile(".(bc)").execute("abc")
inspect(
result.get(0),
content=(
#|Some("abc")
),
)
let first_group = result.get(1)
inspect(
first_group,
content=(
#|Some("bc")
),
)
}

#
MatchResult::groups

fn MatchResult::groups(self : MatchResult) -> Map[String, StringView]

Returns a map containing all named capture groups and their matched contents.

Parameters:

  • self : The match result object containing named capture group information.

Returns a map where keys are the names of capture groups and values are their corresponding matched content. If a named group did not participate in the match or was not matched, its value will be None.

Example:

test {
let engine = compile("(?<word>\\w+)\\s+(?<num>\\d+)")
let result = engine.execute("hello 123")
inspect(
result.groups(),
content=(
#|{"word": "hello", "num": "123"}
),
)
}

#
MatchResult::matched

fn MatchResult::matched(self : MatchResult) -> Bool

Checks if the match was successful.

Determines whether the regular expression found a match in the input text. An empty capture group array indicates that no match was found.

Args:
  • self: The match result object.

Returns: true if a match was found, otherwise false.

Example:
test {
let result = compile("text").execute("text")
inspect(result.matched(), content="true")
}

#
MatchResult::results

fn MatchResult::results(self : MatchResult) -> Array[StringView?]

Gets an iterator over the content of all valid capture groups.

Returns an iterator that contains the content of all successfully matched capture groups, including the full match (index 0) and all other capture groups. Only valid matched content is returned, skipping any unmatched capture groups.

Args:
  • self: The match result object.

Returns: An iterator containing the content of all valid capture groups.

Example:
test {
let result = compile("(\\w)+").execute("abc")
inspect(
result.results(),
content=(
#|[Some("abc"), Some("c")]
),
)
}

#
Regexp

type Regexp

#
Regexp::execute

fn Regexp::execute(self : Regexp, input : StringView) -> MatchResult

Executes a regular expression match on the input text.

Uses the compiled regular expression engine to perform a match on the specified input text, returning a MatchResult object containing the match results and capture group information. It uses a leftmost-first matching strategy, returning the first match found.

Args:
  • self: The regular expression execution engine.
  • input: The input text to match against.

Returns: A MatchResult object containing the match status and capture group information.

Example:
test {
let engine = compile("a(bc|de)f")
let result = engine.execute("xxabcf")
if result.matched() {
inspect(
result.get(0),
content=(
#|Some("abcf")
),
)
}
}

#
Regexp::execute_with_remainder

#deprecated("Use `Regexp::execute` and `MatchResult::before`/`after` instead.")
fn Regexp::execute_with_remainder(self : Regexp, input : StringView) -> (MatchResult, StringView)

Executes a regular expression match on the input text.

Uses the compiled regular expression engine to perform a match on the specified input text, returning a MatchResult object containing the match results and capture group information. It uses a leftmost-first matching strategy, returning the first match found.

Args:
  • self: The regular expression execution engine.
  • input: The input text to match against.

Returns: A tuple of
  • MatchResult object containing the match status and capture group information.
  • StringView representing the remaining text after the match.

Example:
test {
let engine = compile("a(bc|de)f")
let result = engine.execute("abcfxx")
guard result.matched()
inspect(result.before(), content="")
inspect(result.after(), content="xx")
let result = engine.execute("axxf")
guard !result.matched()
inspect(result.before(), content="axxf")
inspect(result.after(), content="")
}

#
Regexp::group_by_name

fn Regexp::group_by_name(self : Regexp, name : String) -> Int?

Gets the index of a capture group by its name.

Finds the index of a capture group with the specified name in the results. Used to access named capture groups, such as those defined in the form (?<name>pattern).

Args:
  • self: The regular expression execution engine.
  • name: The name of the capture group.

Returns: The index of the capture group, or None if the group name does not exist.

Example:
test {
let engine = compile("(?<word>\\w+)")
inspect(engine.group_by_name("word"), content="Some(1)")
}

#
Regexp::group_count

fn Regexp::group_count(self : Regexp) -> Int

Gets the total number of capture groups in the regular expression.

Returns the number of capture groups defined in the regular expression pattern, including both named and anonymous groups. Group with index 0 always represents the entire match, and actual capture groups start from index 1.

Args:
  • self: The regular expression execution engine.

Returns: The total number of capture groups (including the full match at index 0).

Example:
test {
let engine = compile("(a)(b|c)")
inspect(engine.group_count(), content="3")
}

#
Regexp::group_names

fn Regexp::group_names(self : Regexp) -> Array[String]

Gets the names of all named capture groups in the regular expression.

Returns an iterator that contains the names of all named capture groups in the regular expression. Only explicitly named capture groups, such as those defined in the form (?<name>pattern), are returned.

Args:
  • self: The regular expression execution engine.

Returns: An iterator containing all group names.

Example:
test {
let engine = compile("(?<first>a)(?<second>b)")
inspect(
engine.group_names(),
content=(
#|["first", "second"]
),
)
}

#
Regexp::match_

fn Regexp::match_(self : Regexp, input : StringView) -> MatchResult?

Executes a regular expression match on the input text.

Uses the compiled regular expression engine to perform a match on the specified input text, returning a MatchResult object containing the match results and capture group information. It uses a leftmost-first matching strategy, returning the first match found.

Args:
  • self: The regular expression execution engine.
  • input: The input text to match against.

Returns: A tuple of
  • MatchResult object containing the match status and capture group information.

Example:
test {
let engine = compile("a(bc|de)f")
guard engine.match_("abcfxx") is Some(result)
inspect(result.after(), content="xx")
assert_true(engine.match_("axxf") is None)
}

#
compile

fn compile(regexp : StringView, flags? : StringView) -> Regexp raise RegexpError

Compiles a regular expression into an execution engine.

Parses and compiles a regular expression string into a reusable execution engine. The compiled engine can be executed multiple times, avoiding the overhead of repeated parsing and compilation.

Args:
  • regexp: A string view of the regular expression.

Returns: The compiled regular expression execution engine.

Throws: Error_ if the regular expression syntax is invalid.

Example:
test {
let engine = compile("a(bc|de)f")
inspect(
engine.execute("xxabcf").results(),
content=(
#|[Some("abcf"), Some("bc")]
),
)
}