🥮 mooncakes.io

    unicode

    Unicode in MoonBit

    unicode
    bidi
    idna
    nfc
    nfd
    nfkc
    nfkd
    punycode
    Download zip
    Version
    0.7.1
    License
    Apache-2.0
    Repository
    Last updated
    8 days ago
    Downloads
    26K

    #unicode

    moonbit-community/unicode is the compatibility umbrella for the Unicode library that is split into independently consumable modules. It preserves the original package paths for existing users by forwarding to the feature modules.

    The generated tables target Unicode 18.0.0.

    #Packages

    PackageContents
    moonbit-community/unicodeGeneral_Category, East_Asian_Width, Line_Break, Joining_Type, Joining_Group, Hangul_Syllable_Type, binary properties (White_Space, Alphabetic, Lowercase, Uppercase, XID_Start, XID_Continue, Default_Ignorable_Code_Point, Grapheme_Extend, Regional_Indicator, Prepended_Concatenation_Mark, Emoji_Presentation, Emoji_Modifier, Emoji_Modifier_Base), emoji variation sequences, and case mapping
    moonbit-community/unicode/normalizationNormalization Forms from UAX #15
    moonbit-community/unicode/casefoldFull case folding, canonical caseless matching, and NFKC_Casefold
    moonbit-community/unicode/punycodePunycode from RFC 3492
    moonbit-community/unicode/bidiBidirectional algorithm from UAX #9
    moonbit-community/unicode/idnaIDNA processing from UTS #46

    New code should depend on the feature module it needs directly:

    moon add moonbit-community/unicode-data
moon add moonbit-community/unicode-normalization
moon add moonbit-community/unicode-casefold
moon add moonbit-community/punycode
moon add moonbit-community/unicode-bidi
moon add moonbit-community/unicode-idna

    See the repository README for the full API reference, development commands, and release instructions.

    EastAsianWidth

    source

    Unicode East_Asian_Width values See UAX #11: https://www.unicode.org/reports/tr11/

    GeneralCategory

    source

    Unicode General_Category values (two-letter categories) See UAX #44: https://www.unicode.org/reports/tr44/#General_Category_Values

    GeneralCategoryGroup

    source

    Unicode General_Category major class (single-letter categories) See UAX #44: https://www.unicode.org/reports/tr44/#General_Category_Values

    HangulSyllableType

    source

    Unicode Hangul_Syllable_Type values See the Unicode Standard, section 3.12, Conjoining Jamo Behavior: https://www.unicode.org/versions/latest/core-spec/chapter-3/#G24646

    JoiningGroup

    source

    Unicode Joining_Group values See the Unicode Standard, section 9.2, Arabic: https://www.unicode.org/versions/latest/core-spec/chapter-9/#G7480

    JoiningType

    source

    Unicode Joining_Type values See the Unicode Standard, section 9.2, Arabic: https://www.unicode.org/versions/latest/core-spec/chapter-9/#G7480

    LineBreak

    source

    Unicode Line_Break values See UAX #14: https://www.unicode.org/reports/tr14/#Table1

    east_asian_width

    source

    Get the East_Asian_Width for a character.

    East_Asian_Width classifies characters by the width they have in East Asian typography. It is a property of a single character, not a column count: nonspacing marks and control characters have a value too. Code points the Unicode Character Database does not list are Neutral, except that unassigned code points in the CJK ideograph blocks and in planes 2 and 3 are Wide.

    general_category

    source

    Get the General_Category for a character

    hangul_syllable_type

    source

    Get the Hangul_Syllable_Type for a character.

    Hangul_Syllable_Type tells the conjoining jamo apart from the precomposed syllables: a leading consonant, a vowel, a trailing consonant, a syllable of a leading consonant and a vowel, or one that also has a trailing consonant. Every other character, including the Hangul compatibility jamo and unassigned code points, is NotApplicable.

    It is narrower than Grapheme_Cluster_Break, whose V value also covers a few characters outside Hangul.

    has_emoji_variation_sequence

    source
    fn has_emoji_variation_sequence(c : Char) -> Bool

    Check whether a character starts an emoji variation sequence.

    These are the characters emoji-variation-sequences.txt lists: each can be followed by U+FE0E VARIATION SELECTOR-15 to ask for a text presentation or by U+FE0F VARIATION SELECTOR-16 to ask for an emoji presentation. The set contains characters on both sides of Emoji_Presentation, such as U+2764 HEAVY BLACK HEART, which is text by default, and U+231A WATCH, which is emoji by default. Most emoji are not in it, since they have no text presentation to select.

    is_alphabetic

    source
    fn is_alphabetic(c : Char) -> Bool

    Check whether a character has the Unicode Alphabetic property.

    Alphabetic covers the letters (General_Category L), letter numbers (Nl), and Other_Alphabetic characters such as the vowel signs of Indic scripts. Decimal digits are not Alphabetic.

    is_default_ignorable_code_point

    source
    fn is_default_ignorable_code_point(c : Char) -> Bool

    Check whether a character has the Unicode Default_Ignorable_Code_Point property.

    Default_Ignorable_Code_Point covers characters that should not be displayed when a font has no glyph for them, such as U+00AD SOFT HYPHEN, the zero width joiners, the bidirectional controls, the variation selectors, and the tag characters. It also covers some unassigned code points reserved for future characters of this kind.

    is_emoji_modifier

    source
    fn is_emoji_modifier(c : Char) -> Bool

    Check whether a character has the Unicode Emoji_Modifier property.

    Emoji_Modifier covers the five skin tone modifiers U+1F3FB to U+1F3FF.

    is_emoji_modifier_base

    source
    fn is_emoji_modifier_base(c : Char) -> Bool

    Check whether a character has the Unicode Emoji_Modifier_Base property.

    Emoji_Modifier_Base covers the characters whose appearance a following Emoji_Modifier changes, such as U+1F44D THUMBS UP SIGN.

    is_emoji_presentation

    source
    fn is_emoji_presentation(c : Char) -> Bool

    Check whether a character has the Unicode Emoji_Presentation property.

    Emoji_Presentation covers the characters displayed as emoji by default. It is narrower than the Emoji property: U+2764 HEAVY BLACK HEART and the ASCII digits are emoji only when followed by U+FE0F VARIATION SELECTOR-16. See UTS #51: https://www.unicode.org/reports/tr51/

    is_grapheme_extend

    source
    fn is_grapheme_extend(c : Char) -> Bool

    Check whether a character has the Unicode Grapheme_Extend property.

    Grapheme_Extend covers the nonspacing and enclosing marks (General_Category Mn and Me) and Other_Grapheme_Extend characters such as U+200C ZERO WIDTH NON-JOINER and a few spacing marks. U+200D ZERO WIDTH JOINER is not Grapheme_Extend.

    is_lowercase

    source
    fn is_lowercase(c : Char) -> Bool

    Check whether a character has the Unicode Lowercase property.

    Lowercase is broader than General_Category Ll: it also covers Other_Lowercase characters such as U+00AA FEMININE ORDINAL INDICATOR, modifier letters like U+02B0, and the small Roman numerals. Titlecase letters (Lt) are neither Lowercase nor Uppercase.

    is_prepended_concatenation_mark

    source
    fn is_prepended_concatenation_mark(c : Char) -> Bool

    Check whether a character has the Unicode Prepended_Concatenation_Mark property.

    Prepended_Concatenation_Mark covers format characters that enclose or extend under the digits or letters following them, such as U+0600 ARABIC NUMBER SIGN and U+06DD ARABIC END OF AYAH. Unlike most format characters they are visible.

    is_punctuation

    source
    fn is_punctuation(c : Char) -> Bool

    Check whether a character is punctuation.

    Punctuation is the General_Category group P, covering the connector, dash, open, close, initial, final, and other punctuation categories (Pc, Pd, Ps, Pe, Pi, Pf, and Po). Symbols such as $ (Sc) and + (Sm) are not punctuation.

    The predicate uses its own compact table instead of the General_Category trie, so code that only tests punctuation does not link the whole category table.

    is_regional_indicator

    source
    fn is_regional_indicator(c : Char) -> Bool

    Check whether a character has the Unicode Regional_Indicator property.

    Regional_Indicator covers the 26 regional indicator symbols U+1F1E6 to U+1F1FF, which pair up to form emoji flags.

    is_uppercase

    source
    fn is_uppercase(c : Char) -> Bool

    Check whether a character has the Unicode Uppercase property.

    Uppercase is broader than General_Category Lu: it also covers Other_Uppercase characters such as the capital Roman numerals and the circled capital Latin letters. Titlecase letters (Lt) are neither Lowercase nor Uppercase.

    is_white_space

    source
    fn is_white_space(c : Char) -> Bool

    Check whether a character has the Unicode White_Space property.

    White_Space covers the ASCII controls TAB, LF, VT, FF, and CR, the space separators (General_Category Zs), U+0085 NEXT LINE, and the line and paragraph separators U+2028 and U+2029. It does not include U+200B ZERO WIDTH SPACE or U+FEFF ZERO WIDTH NO-BREAK SPACE.

    is_xid_continue

    source
    fn is_xid_continue(c : Char) -> Bool

    Check whether a character has the Unicode XID_Continue property.

    XID_Continue is the recommended continuation class for Unicode default identifiers and includes every XID_Start character.

    is_xid_start

    source
    fn is_xid_start(c : Char) -> Bool

    Check whether a character has the Unicode XID_Start property.

    XID_Start is the recommended start class for Unicode default identifiers.

    joining_group

    source

    Get the Joining_Group for a character.

    Joining_Group classifies the letters of cursive scripts such as Arabic, Syriac, and Manichaean by the shape they join with: letters that differ only in their dots, like U+0628 ARABIC LETTER BEH and U+062A ARABIC LETTER TEH, share a group. Characters that are not such letters, including unassigned code points, are NoJoiningGroup.

    The value Pe shares its name with a GeneralCategory value: write JoiningGroup::Pe or GeneralCategory::Pe where the type is not known from the context.

    joining_type

    source

    Get the Joining_Type for a character.

    Joining_Type describes how a character takes part in cursive joining in scripts such as Arabic, Syriac, and Mongolian. Transparent characters, mostly nonspacing marks and format characters, do not interrupt joining between the characters on either side of them. Characters that take no part in cursive joining, including unassigned code points, are NonJoining.

    line_break

    source

    Get the Line_Break for a character.

    Line_Break is the property the line breaking rules of UAX #14 are written in terms of. It is a property of a single character, and the value is the one the Unicode Character Database assigns: rule LB1 of the algorithm, which resolves AI, CJ, SA, SG, and XX to other classes, is not applied. Unassigned code points are XX, except in ranges reserved for characters of a known class, such as ID in the CJK ideograph blocks and in planes 2 and 3.

    to_lowercase

    source
    fn to_lowercase(c : Char) -> String

    Get the full lowercase mapping for a character Returns a String containing one or more lowercase characters For example, 'İ' (U+0130) maps to "i\u{0307}" (i with combining dot above)

    to_simple_lowercase

    source
    fn to_simple_lowercase(c : Char) -> Char

    Get the simple lowercase mapping for a character Returns the character itself if no mapping exists

    to_simple_titlecase

    source
    fn to_simple_titlecase(c : Char) -> Char

    Get the simple titlecase mapping for a character Returns the character itself if no mapping exists

    to_simple_uppercase

    source
    fn to_simple_uppercase(c : Char) -> Char

    Get the simple uppercase mapping for a character Returns the character itself if no mapping exists

    to_titlecase

    source
    fn to_titlecase(c : Char) -> String

    Get the full titlecase mapping for a character Returns a String containing one or more titlecase characters For example, 'ß' (U+00DF) maps to "Ss", and 'ffi' (U+FB03) maps to "Ffi"

    to_uppercase

    source
    fn to_uppercase(c : Char) -> String

    Get the full uppercase mapping for a character Returns a String containing one or more uppercase characters For example, 'ß' (U+00DF) maps to "SS", and 'ffi' (U+FB03) maps to "FFI"

    Source Files