Sign in

    unicode-data

    Unicode Character Database lookup and case mapping for MoonBit

    unicode
    ucd
    case-mapping
    general-category
    Download zip
    Version
    0.6.0
    License
    Apache-2.0
    Last updated
    5 days ago
    Downloads
    345

    #unicode-data

    Unicode Character Database lookup, general categories, East Asian width, binary properties (White_Space, Alphabetic, Lowercase, Uppercase, XID_Start, XID_Continue), and case mapping for MoonBit. This module uses Unicode 18.0.0 data.

    moon add moonbit-community/unicode-data

    See the repository root README for API documentation and examples.

    #Low-level data packages

    Import the package for the lookup you need. Each package owns its generated tables and exposes lookup functions; the tables themselves remain private.

    Package under moonbit-community/unicode-data/Lookups
    decompositionCanonical and compatibility decomposition mappings
    compositionComposition pairs and composition exclusions
    cccCanonical Combining Class
    general_categoryGeneral Category ordinal and is_mark
    east_asian_widthEast_Asian_Width ordinal
    case_mappingSimple and full uppercase, lowercase, and titlecase mappings
    xidXID_Start and XID_Continue
    alphabeticAlphabetic
    lowercaseLowercase
    uppercaseUppercase
    white_spaceWhite_Space

    moonbit-community/unicode-data/data remains a compatibility package that re-exports the old lookup functions with deprecation notices pointing to these packages. The high-level moonbit-community/unicode-data interface is unchanged: its general_category(c) returns GeneralCategory, with GeneralCategoryGroup available through .group(). These enums remain defined in the root package. The low-level general_category package retains lookup_general_category(c) -> Int, returning the existing 0–29 category ordinals.

    EastAsianWidth

    pub(all) enum EastAsianWidth {
    A
    F
    H
    N
    Na
    W
    } derive(Eq)

    Unicode East_Asian_Width values See UAX #11: https://www.unicode.org/reports/tr11/

    EastAsianWidth::equal

    EastAsianWidth::not_equal

    fn EastAsianWidth::not_equal(x : EastAsianWidth, y : EastAsianWidth) -> Bool

    EastAsianWidth::output

    fn EastAsianWidth::output(self : EastAsianWidth, logger : &Logger) -> Unit

    EastAsianWidth::to_string

    fn EastAsianWidth::to_string(self : EastAsianWidth) -> String

    GeneralCategory

    pub(all) enum GeneralCategory {
    Lu
    Ll
    Lt
    Lm
    Lo
    Mn
    Mc
    Me
    Nd
    Nl
    No
    Zs
    Zl
    Zp
    Pc
    Pd
    Ps
    Pe
    Pi
    Pf
    Po
    Sm
    Sc
    Sk
    So
    Cc
    Cf
    Cs
    Co
    Cn
    } derive(Eq)

    Unicode General_Category values (two-letter categories) See UAX #44: https://www.unicode.org/reports/tr44/#General_Category_Values

    GeneralCategory::equal

    GeneralCategory::group

    Get the major category group (single-letter category) for a GeneralCategory

    GeneralCategory::not_equal

    fn GeneralCategory::not_equal(x : GeneralCategory, y : GeneralCategory) -> Bool

    GeneralCategory::output

    fn GeneralCategory::output(self : GeneralCategory, logger : &Logger) -> Unit

    GeneralCategory::to_string

    fn GeneralCategory::to_string(self : GeneralCategory) -> String

    GeneralCategoryGroup

    pub(all) enum GeneralCategoryGroup {
    L
    M
    N
    Z
    P
    S
    C
    } derive(Eq,
    Debug
    )

    Unicode General_Category major class (single-letter categories) See UAX #44: https://www.unicode.org/reports/tr44/#General_Category_Values

    GeneralCategoryGroup::equal

    GeneralCategoryGroup::not_equal

    GeneralCategoryGroup::output

    fn GeneralCategoryGroup::output(self : GeneralCategoryGroup, logger : &Logger) -> Unit

    GeneralCategoryGroup::to_string

    fn GeneralCategoryGroup::to_string(self : GeneralCategoryGroup) -> String

    east_asian_width

    fn east_asian_width(c : Char) -> EastAsianWidth

    Get the East_Asian_Width for a character.

    East_Asian_Width classifies characters by the width they have in East Asian typography. It is a property of a single character, not a column count: nonspacing marks and control characters have a value too. Code points the Unicode Character Database does not list are Neutral, except that unassigned code points in the CJK ideograph blocks and in planes 2 and 3 are Wide.

    general_category

    fn general_category(c : Char) -> GeneralCategory

    Get the General_Category for a character

    is_alphabetic

    fn is_alphabetic(c : Char) -> Bool

    Check whether a character has the Unicode Alphabetic property.

    Alphabetic covers the letters (General_Category L), letter numbers (Nl), and Other_Alphabetic characters such as the vowel signs of Indic scripts. Decimal digits are not Alphabetic.

    is_lowercase

    fn is_lowercase(c : Char) -> Bool

    Check whether a character has the Unicode Lowercase property.

    Lowercase is broader than General_Category Ll: it also covers Other_Lowercase characters such as U+00AA FEMININE ORDINAL INDICATOR, modifier letters like U+02B0, and the small Roman numerals. Titlecase letters (Lt) are neither Lowercase nor Uppercase.

    is_uppercase

    fn is_uppercase(c : Char) -> Bool

    Check whether a character has the Unicode Uppercase property.

    Uppercase is broader than General_Category Lu: it also covers Other_Uppercase characters such as the capital Roman numerals and the circled capital Latin letters. Titlecase letters (Lt) are neither Lowercase nor Uppercase.

    is_white_space

    fn is_white_space(c : Char) -> Bool

    Check whether a character has the Unicode White_Space property.

    White_Space covers the ASCII controls TAB, LF, VT, FF, and CR, the space separators (General_Category Zs), U+0085 NEXT LINE, and the line and paragraph separators U+2028 and U+2029. It does not include U+200B ZERO WIDTH SPACE or U+FEFF ZERO WIDTH NO-BREAK SPACE.

    is_xid_continue

    fn is_xid_continue(c : Char) -> Bool

    Check whether a character has the Unicode XID_Continue property.

    XID_Continue is the recommended continuation class for Unicode default identifiers and includes every XID_Start character.

    is_xid_start

    fn is_xid_start(c : Char) -> Bool

    Check whether a character has the Unicode XID_Start property.

    XID_Start is the recommended start class for Unicode default identifiers.

    to_lowercase

    fn to_lowercase(c : Char) -> String

    Get the full lowercase mapping for a character Returns a String containing one or more lowercase characters For example, 'İ' (U+0130) maps to "i\u{0307}" (i with combining dot above)

    to_simple_lowercase

    fn to_simple_lowercase(c : Char) -> Char

    Get the simple lowercase mapping for a character Returns the character itself if no mapping exists

    to_simple_titlecase

    fn to_simple_titlecase(c : Char) -> Char

    Get the simple titlecase mapping for a character Returns the character itself if no mapping exists

    to_simple_uppercase

    fn to_simple_uppercase(c : Char) -> Char

    Get the simple uppercase mapping for a character Returns the character itself if no mapping exists

    to_titlecase

    fn to_titlecase(c : Char) -> String

    Get the full titlecase mapping for a character Returns a String containing one or more titlecase characters For example, 'ß' (U+00DF) maps to "Ss", and 'ffi' (U+FB03) maps to "Ffi"

    to_uppercase

    fn to_uppercase(c : Char) -> String

    Get the full uppercase mapping for a character Returns a String containing one or more uppercase characters For example, 'ß' (U+00DF) maps to "SS", and 'ffi' (U+FB03) maps to "FFI"