🥮 mooncakes.io

    moonbitstack/moonregex/prop does not have a README file

    CharSet

    source
    pub struct CharSet {
      ranges : Array[(Int, Int)]
    } derive(Eq)

    A set of code points, kept as sorted disjoint inclusive ranges.

    Character classes, \p{...}, \w and the rest are all this one shape, so the set algebra a pattern needs — union for [a-z], difference for --, intersection for &&, complement for [^…] — lives here once.
    impl Debug for CharSet

    CharSet::any

    source
    fn CharSet::any() -> CharSet

    Every code point, surrogates included.

    CharSet::complement

    source
    fn CharSet::complement(self : CharSet) -> CharSet

    Complement over every code point: what [^…] means.

    CharSet::contains

    source
    fn CharSet::contains(self : CharSet, c : Int) -> Bool

    Whether a code point is in the set.

    CharSet::difference

    source
    fn CharSet::difference(self : CharSet, other : CharSet) -> CharSet

    Difference: the V1 set algebra's --.

    CharSet::empty

    source
    fn CharSet::empty() -> CharSet

    The empty set.

    CharSet::equal

    source
    fn CharSet::equal(CharSet, CharSet) -> Bool

    CharSet::fold

    source
    fn CharSet::fold(self : CharSet) -> CharSet

    Close the set under simple case folding: add the folded form of everything in it, and the unfolded form of everything that folds into it.

    This is what (?i) does to a class at parse time: fold the pattern, not the input, so the executor stays a plain membership test. Only single code point folds exist here, so ß does not grow an ss — matching it against SS is out of scope, exactly as it is in Python's re.

    A code point with no simple mapping is its own orbit, and U+0130 is one: CaseFolding.txt gives it a full mapping only, so (?i)İ matches İ and nothing else. re lowercases it to a plain i, which is a different text; the library does not, because İ, i and i followed by a combining dot are three texts and only the last two are the same one.

    CharSet::intersection

    source
    fn CharSet::intersection(self : CharSet, other : CharSet) -> CharSet

    Intersection: the V1 set algebra's &&.

    CharSet::is_empty

    source
    fn CharSet::is_empty(self : CharSet) -> Bool

    Whether the set holds nothing.

    CharSet::not_equal

    source
    fn CharSet::not_equal(x : CharSet, y : CharSet) -> Bool

    CharSet::of

    source
    fn CharSet::of(ranges : Array[(Int, Int)]) -> CharSet

    Build a set from ranges, sorting and merging them first.

    The input may be in any order and may overlap; the stored form is always sorted, disjoint and coalesced, which is what makes contains a binary search and the algebra linear.

    CharSet::ranges

    source
    fn CharSet::ranges(self : CharSet) -> Array[(Int, Int)]

    The stored ranges, sorted and disjoint.

    CharSet::single

    source
    fn CharSet::single(c : Int) -> CharSet

    A set holding exactly one code point.

    CharSet::symmetric_difference

    source
    fn CharSet::symmetric_difference(self : CharSet, other : CharSet) -> CharSet

    Symmetric difference: the V1 set algebra's ~~.

    CharSet::to_repr

    source

    CharSet::union

    source
    fn CharSet::union(self : CharSet, other : CharSet) -> CharSet

    Union: [a-z] or, in the V1 set algebra, ||.

    ascii_digit

    source
    let ascii_digit : CharSet

    ascii_space

    source
    let ascii_space : CharSet

    ascii_word

    source
    let ascii_word : CharSet

    The same three, restricted to ASCII, which is what the A flag and the ECMA-262 dialect without u mean by them.

    binary

    source
    fn binary(name : StringView) -> CharSet?

    A binary property: Alphabetic, White_Space, Emoji.

    block

    source
    fn block(name : StringView) -> CharSet?

    A block: Basic Latin, BasicLatin, CJK Unified Ideographs.

    A block is one range, so the usual answer needs no table of its own.

    category

    source
    fn category(name : StringView) -> CharSet?

    A general category: L, Lu, Uppercase_Letter, casedletter.

    char_of_name

    source
    fn char_of_name(name : StringView) -> Int?

    The code point a Unicode character name denotes — what \N{name} answers from. Names are read loosely, the way unicodedata.lookup reads them.

    digit

    source
    fn digit() -> CharSet

    \d: decimal digits, and only those.

    fold

    source
    fn fold(c : Char) -> Char

    The simple fold of a code point: the one character it becomes when case stops counting. A code point whose fold is longer than one character, ß to ss, is its own, since a class holds code points, not strings.

    fold_code

    source
    fn fold_code(code : Int) -> String

    The full fold of one code point, as text.

    Two of them come back as more than one character — ß and İ — and the second is the one re folds differently; fold_text says why.

    fold_into

    source
    fn fold_into(code : Int, out : Array[Int]) -> Unit

    Append the full fold of code to out.

    The cache is what keeps a folded comparison cheap enough to sit in the matcher: the alternative is building a string per character per attempt.

    fold_pairs

    source
    fn fold_pairs() -> Array[(Int, Int)]

    The fold pairs both ways, sorted by source — what a class is closed under.

    The code points that change under casefolding are the ones a class grows from, and each of them brings its whole fold class with it, so the pairs come out of the orbit table instead of a walk of the code space.

    fold_text

    source
    fn fold_text(text : StringView) -> String

    The full fold of a run of text: ß becomes ss, fi becomes fi, and a case-insensitive comparison of two strings is one comparison of these.

    This is what the f flag buys over the simple fold: a literal is no longer one code point against one code point, because a single character may stand for two and two for one.

    U+0130 is the one code point where the answer here is not what the references answer. CaseFolding.txt gives it a full mapping to i and a combining dot; Python's re and the regex package both fold it to a plain i instead. Those are different texts — İ is one character and i̇ is two, and a lone i is neither — so a pattern holding İ matches i̇ here and does not match i. The reading is deliberate: folding is a relation between texts, and the Unicode data is what says which texts are the same.

    property

    source
    fn property(name : StringView) -> CharSet?

    Any property, however it is spelled — the shape \p{...} takes.

    name=value is accepted too: Script=Latin, General_Category=Lu, Block=CJK Unified Ideographs. The left side picks the table, the right side the entry; without it, every table is searched in the order a Unicode name would be looked up: category, script, block, binary property.

    script

    source
    fn script(name : StringView) -> CharSet?

    A script: Latin, Latn, Han.

    The short code and the long name are one name to the UCD, so its own reader resolves the name and the table answers with that script's ranges.

    space

    source
    fn space() -> CharSet

    \s: the White_Space property.

    version

    source
    fn version() -> String

    Which Unicode the answers follow.

    word

    source
    fn word() -> CharSet

    \w under the Python and mrab spellings: letters, numbers and the underscore. Python's own words for it are "alphanumeric (as str.isalnum has it) plus _", which is exactly these categories.

    Source Files