regexp

    A regular expression engine for MoonBit based on VM execution designed for predictable time complexity

    regexp
    Download zip
    Version
    0.3.5
    License
    Apache-2.0
    Last updated
    7 months ago
    Downloads
    32K

    #🔍 regexp.mbt

    ⚠️ API STABILITY NOTICE This is a alpha release with a stabilizing API. While core functionality is complete and well-tested, API changes may occur in future versions as we refine the implementation.

    Regular expression engine for MoonBit — inspired by Russ Cox's regex series.

    #⚡ Quick Start

    ///|
    test {
    // Compile once, use everywhere
    let regexp = @regexp.compile("a(bc|de)f")
    guard regexp.match_("xxabcf") is Some(result)
    inspect(
    result.results(),
    content=(
    #|[Some("abcf"), Some("bc")]
    ),
    )

    // Write a simple split with regexp
    fn split(regexp : @regexp.Regexp, target : StringView) -> Array[StringView] {
    let result = []
    loop target {
    "" => ()
    str => {
    let res = regexp.execute(str)
    result.push(res.before())
    continue res.after()
    }
    }
    result
    }

    let re = @regexp.compile("_+")
    inspect(
    split(re, "1_2__3__4__5_____6"),
    content=(
    #|["1", "2", "3", "4", "5", "6"]
    ),
    )
    }

    #🎯 Core API

    #Build & Execute

    • compile(pattern) → Creates an Engine
    • engine.execute(text) → Returns MatchResult

    #Inspect Results

    • result.matched()Bool
    • result.get(index) → Capture group content
    • result.results() → Iterator over all matches

    #Named Groups & Advanced

    • engine.group_by_name(name) → Find group index by name
    • engine.group_count() → Total capture groups
    • result.groups() → Get named group content

    #🎪 Syntax Playground

    FeatureExampleWhat it does
    LiteralsabcMatch exact text
    Wildcardsa.c. matches any character
    Quantifiersa+, b*, c?One or more, zero or more, optional
    Rangesa{2,5}Between 2-5 repetitions
    Classes[a-z], [^0-9]Character sets, negated sets
    Groups(abc), (?:xyz)Capturing, non-capturing
    Named(?<word>abc)Named capture groups
    Choicecat\|dogMatch either option
    Anchors^start, end$Line boundaries
    Escapes\\u{41}, \\u0041Unicode escapes, standard escapes
    Unicode Props\\p{L}, \\p{Nd}Unicode general categories
    Backrefs ⚠️(.)\\1Reference previous captures

    #🌍 Unicode Property Support

    Match characters by their Unicode general categories:

    ///|
    test "unicode properties" {
    // Matching gc=L
    let regex = @regexp.compile("\\p{Letter}+")
    inspect(
    regex.execute("Hello 世界").results(),
    content=(
    #|[Some("Hello")]
    ),
    )

    // Matching gc=N
    let regex = @regexp.compile("\\p{Number}+")
    inspect(
    regex.execute("123 and 456").results(),
    content=(
    #|[Some("123")]
    ),
    )
    }

    Supported Propertes:

    #🔄 Backreferences

    ⚠️ Performance Warning: Backreferences can cause exponential time complexity in worst cases!

    ///|
    test "backreferences" {
    // Palindrome detection (simple)
    let palindrome = @regexp.compile("^(.)(.)\\2\\1")
    inspect(
    palindrome.execute("abba").results(),
    content=(
    #|[Some("abba"), Some("a"), Some("b")]
    ),
    )

    // HTML tag matching
    let html_regex = @regexp.compile("<([a-zA-Z]+)[^>]*>(.*?)</\\1>")
    let result = html_regex.execute("<div class='test'>content</div>")
    inspect(
    result.results(),
    content=(
    #|[Some("<div class='test'>content</div>"), Some("div"), Some("content")]
    ),
    )
    }

    #💡 Real Examples

    ///|
    test "character classes" {
    // Email validation (simplified)
    let email = @regexp.compile(
    (
    #|[\w-]+@[\w-]+\.\w+
    ),
    )
    let email_result = email.execute("user@example.com").results()
    inspect(
    email_result,
    content=(
    #|[Some("user@example.com")]
    ),
    )
    // Extract numbers
    let numbers = @regexp.compile(
    (
    #|\d+\.\d{2}
    ),
    )
    let result = numbers.execute("Price: $42.99").results()
    inspect(
    result,
    content=(
    #|[Some("42.99")]
    ),
    )

    // Named captures for parsing
    let parser = @regexp.compile(
    (
    #|(?<year>\d{4})-(?<month>\d{2})-(?<day>\d{2})
    ),
    )
    let date_result = parser.execute("2024-03-15")
    inspect(
    date_result.groups(),
    content=(
    #|{"year": "2024", "month": "03", "day": "15"}
    ),
    )
    }

    #🚨 Error Handling

    ///|
    test {
    try {
    let _ = @regexp.compile("a(b")
    // Oops! Missing )
    } catch {
    RegexpError(err=MissingParenthesis, source_fragment=_) =>
    println("Fix your regex! 🔧")
    _ => ()
    }
    }

    #⚡ Performance Characteristics

    • Predictable complexity — Designed to avoid catastrophic backtracking (except with backreferences)
    • VM-based — Structured interpreter design
    • Unicode support — Character set and property support

    Built with reliability and correctness as primary goals.

    #🔍 Implementation Notes

    #Behavior Differences from Other Engines

    This implementation has some behavior differences compared to other popular regex engines:

    1. Empty Character Class Handling:
      • In JavaScript: [][] is parsed as two character classes with no characters
      • In Golang: [][] is parsed as one character class containing ] and [
      • In MoonBit: we follow the JavaScript interpretation

    2. Empty Alternatives Behavior:
      • Expressions like (|a)* and (|a)+ have specific behavior that may differ from other implementations
      • See Golang issue #46123 for related discussion

    3. Backreferences:
      • Backreferences are supported but may impact the complexity guarantees of the engine

    RegexpError

    pub suberror RegexpError {
    RegexpError(Err, StringView)
    }

    Parsing error exception type

    Contains error information and related string view context
    impl Show for RegexpError

    Err

    pub enum Err {
    InternalError
    InvalidCharClass
    InvalidEscape
    InvalidNamedCapture
    InvalidRepeatOp
    InvalidRepeatSize
    MissingBracket
    MissingParenthesis
    MissingRepeatArgument
    TrailingBackslash
    UnexpectedParenthesis
    }

    Regular expression parsing error types

    Defines various error conditions that may be encountered during parsing
    impl Show for Err

    MatchResult

    type MatchResult

    Regular expression match result.

    Contains the complete result information of a regular expression match, including:
    • The original input text.
    • An array of capture group position information.
    • A map of named capture groups.

    Structure of the capture group array:
    • Index 0, 1: Start and end positions of the full match.
    • Index 2, 3: Start and end positions of the first capture group.
    • Index 4, 5: Start and end positions of the second capture group.
    • And so on...

    MatchResult::after

    fn MatchResult::after(self : MatchResult) -> StringView

    Gets the remaining text after the match.

    If the match was not successful, this will return the empty string.

    test {
    let engine = compile("hello")
    let result = engine.execute("hello world")
    inspect(result.after(), content=" world")
    let result = engine.execute("say yes")
    inspect(result.after(), content="")
    }

    MatchResult::before

    fn MatchResult::before(self : MatchResult) -> StringView

    Gets the text before the match.

    If the match was not successful, this will return the entire input text.

    test {
    let engine = compile("world")
    let result = engine.execute("hello world")
    inspect(result.before(), content="hello ")
    let result = engine.execute("say hello")
    inspect(result.before(), content="say hello")
    }

    MatchResult::get

    fn MatchResult::get(self : MatchResult, index : Int) -> StringView?

    Gets the content of a capture group by its index.

    Returns the corresponding matched text based on the capture group's index. Index 0 represents the full match, while indices 1 and above represent individual capture groups.

    Args:
    • self: The match result object.
    • index: The index of the capture group (0 for the full match).

    Returns:
    • Some(view): The text view corresponding to the capture group.
    • None: If the index is invalid or the capture group did not match.

    Example:
    test {
    let result = compile(".(bc)").execute("abc")
    inspect(
    result.get(0),
    content=(
    #|Some("abc")
    ),
    )
    let first_group = result.get(1)
    inspect(
    first_group,
    content=(
    #|Some("bc")
    ),
    )
    }

    MatchResult::groups

    fn MatchResult::groups(self : MatchResult) -> Map[String, StringView]

    Returns a map containing all named capture groups and their matched contents.

    Parameters:

    • self : The match result object containing named capture group information.

    Returns a map where keys are the names of capture groups and values are their corresponding matched content. If a named group did not participate in the match or was not matched, its value will be None.

    Example:

    test {
    let engine = compile("(?<word>\\w+)\\s+(?<num>\\d+)")
    let result = engine.execute("hello 123")
    inspect(
    result.groups(),
    content=(
    #|{"word": "hello", "num": "123"}
    ),
    )
    }

    MatchResult::matched

    fn MatchResult::matched(self : MatchResult) -> Bool

    Checks if the match was successful.

    Determines whether the regular expression found a match in the input text. An empty capture group array indicates that no match was found.

    Args:
    • self: The match result object.

    Returns: true if a match was found, otherwise false.

    Example:
    test {
    let result = compile("text").execute("text")
    inspect(result.matched(), content="true")
    }

    MatchResult::results

    fn MatchResult::results(self : MatchResult) -> Array[StringView?]

    Gets an iterator over the content of all valid capture groups.

    Returns an iterator that contains the content of all successfully matched capture groups, including the full match (index 0) and all other capture groups. Only valid matched content is returned, skipping any unmatched capture groups.

    Args:
    • self: The match result object.

    Returns: An iterator containing the content of all valid capture groups.

    Example:
    test {
    let result = compile("(\\w)+").execute("abc")
    inspect(
    result.results(),
    content=(
    #|[Some("abc"), Some("c")]
    ),
    )
    }

    Regexp

    type Regexp

    Regexp::execute

    fn Regexp::execute(self : Regexp, input : StringView) -> MatchResult

    Executes a regular expression match on the input text.

    Uses the compiled regular expression engine to perform a match on the specified input text, returning a MatchResult object containing the match results and capture group information. It uses a leftmost-first matching strategy, returning the first match found.

    Args:
    • self: The regular expression execution engine.
    • input: The input text to match against.

    Returns: A MatchResult object containing the match status and capture group information.

    Example:
    test {
    let engine = compile("a(bc|de)f")
    let result = engine.execute("xxabcf")
    if result.matched() {
    inspect(
    result.get(0),
    content=(
    #|Some("abcf")
    ),
    )
    }
    }

    Regexp::execute_with_remainder

    #deprecated("Use `Regexp::execute` and `MatchResult::before`/`after` instead.")
    fn Regexp::execute_with_remainder(self : Regexp, input : StringView) -> (MatchResult, StringView)

    Executes a regular expression match on the input text.

    Uses the compiled regular expression engine to perform a match on the specified input text, returning a MatchResult object containing the match results and capture group information. It uses a leftmost-first matching strategy, returning the first match found.

    Args:
    • self: The regular expression execution engine.
    • input: The input text to match against.

    Returns: A tuple of
    • MatchResult object containing the match status and capture group information.
    • StringView representing the remaining text after the match.

    Example:
    test {
    let engine = compile("a(bc|de)f")
    let result = engine.execute("abcfxx")
    guard result.matched()
    inspect(result.before(), content="")
    inspect(result.after(), content="xx")
    let result = engine.execute("axxf")
    guard !result.matched()
    inspect(result.before(), content="axxf")
    inspect(result.after(), content="")
    }

    Regexp::group_by_name

    fn Regexp::group_by_name(self : Regexp, name : String) -> Int?

    Gets the index of a capture group by its name.

    Finds the index of a capture group with the specified name in the results. Used to access named capture groups, such as those defined in the form (?<name>pattern).

    Args:
    • self: The regular expression execution engine.
    • name: The name of the capture group.

    Returns: The index of the capture group, or None if the group name does not exist.

    Example:
    test {
    let engine = compile("(?<word>\\w+)")
    inspect(engine.group_by_name("word"), content="Some(1)")
    }

    Regexp::group_count

    fn Regexp::group_count(self : Regexp) -> Int

    Gets the total number of capture groups in the regular expression.

    Returns the number of capture groups defined in the regular expression pattern, including both named and anonymous groups. Group with index 0 always represents the entire match, and actual capture groups start from index 1.

    Args:
    • self: The regular expression execution engine.

    Returns: The total number of capture groups (including the full match at index 0).

    Example:
    test {
    let engine = compile("(a)(b|c)")
    inspect(engine.group_count(), content="3")
    }

    Regexp::group_names

    fn Regexp::group_names(self : Regexp) -> Array[String]

    Gets the names of all named capture groups in the regular expression.

    Returns an iterator that contains the names of all named capture groups in the regular expression. Only explicitly named capture groups, such as those defined in the form (?<name>pattern), are returned.

    Args:
    • self: The regular expression execution engine.

    Returns: An iterator containing all group names.

    Example:
    test {
    let engine = compile("(?<first>a)(?<second>b)")
    inspect(
    engine.group_names(),
    content=(
    #|["first", "second"]
    ),
    )
    }

    Regexp::match_

    fn Regexp::match_(self : Regexp, input : StringView) -> MatchResult?

    Executes a regular expression match on the input text.

    Uses the compiled regular expression engine to perform a match on the specified input text, returning a MatchResult object containing the match results and capture group information. It uses a leftmost-first matching strategy, returning the first match found.

    Args:
    • self: The regular expression execution engine.
    • input: The input text to match against.

    Returns: A tuple of
    • MatchResult object containing the match status and capture group information.

    Example:
    test {
    let engine = compile("a(bc|de)f")
    guard engine.match_("abcfxx") is Some(result)
    inspect(result.after(), content="xx")
    assert_true(engine.match_("axxf") is None)
    }

    compile

    fn compile(regexp : StringView, flags? : StringView) -> Regexp raise RegexpError

    Compiles a regular expression into an execution engine.

    Parses and compiles a regular expression string into a reusable execution engine. The compiled engine can be executed multiple times, avoiding the overhead of repeated parsing and compilation.

    Args:
    • regexp: A string view of the regular expression.

    Returns: The compiled regular expression execution engine.

    Throws: Error_ if the regular expression syntax is invalid.

    Example:
    test {
    let engine = compile("a(bc|de)f")
    inspect(
    engine.execute("xxabcf").results(),
    content=(
    #|[Some("abcf"), Some("bc")]
    ),
    )
    }