🥮 mooncakes.io

    moon-collate

    Unicode Collation Algorithm and DUCET sort-key toolkit for MoonBit

    unicode
    collation
    uca
    ducet
    sorting
    Download zip
    Version
    0.2.0
    License
    Apache-2.0
    Repository
    Last updated
    4 hours ago
    Downloads
    2

    Dependencies

    #MoonCollate

    MoonCollate — Unicode order, made visible

    MoonCollate is a pure MoonBit Unicode collation toolkit. It turns Unicode text into comparable collation elements and reusable sort keys using the Unicode Collation Algorithm (UCA) and the Default Unicode Collation Element Table (DUCET).

    Status: v0.2.0. Conformance claims are made only for test profiles recorded in docs/conformance.md.

    CI Collation Lab Unicode 17

    #Why this library exists

    Code-point order is not human text order. Accents, case, punctuation, contractions, expansions, and canonical equivalence all affect how users expect text to sort. MoonCollate fills the collation layer between MoonBit's Unicode property/normalization packages and higher-level internationalization, search, table, and database software.

    #Public surface

    • configurable primary through identical comparison strengths;
    • non-ignorable and shifted alternate handling;
    • deterministic binary sort keys;
    • DUCET contractions, expansions, and implicit weights;
    • canonical-equivalence-safe comparison through NFD normalization;
    • structured explanation traces for debugging and education;
    • stable collection helpers and a cached binary-search index;
    • a native CLI and a browser-based Collation Lab.

    ///|
    test {
      let collator = default_collator()
      inspect(collator.compare("resume", "résumé").to_string(), content="Less")
    }

    The committed Unicode 17 short conformance profiles currently cover 208,039 non-ignorable and 229,829 shifted adjacent pairs with zero failures. See the user guide, algorithm notes, and exact conformance statement.

    #Quick start

    Add the published package to another MoonBit module:

    moon add CaptainK-65/moon-collate@0.2.0

    ///|
    test {
      let collator = default_collator().with_numeric(true)
      assert_eq(collator.sort(["file10", "file2", "file1"]), [
        "file1", "file2", "file10",
      ])
    }

    moon run cmd/main -- compare resume résumé
moon run cmd/main -- sort --numeric file10 file2 file1
moon run cmd/main -- explain café

    #Scope boundary

    Version 0.2.0 implements the Unicode 17.0.0 default collation profile. It is not a database, a text shaping engine, or a full ICU/CLDR locale tailoring replacement. See docs/ecosystem-audit.md for the public ecosystem overlap audit and docs/limitations.md for explicit non-goals.

    #Development

    moon update
moon check --target all
moon test --target all
moon run --target native cmd/conformance
moon run --target native cmd/conformance -- --shifted
moon run --target native cmd/conformance -- --full
moon run --target native cmd/conformance -- --full --shifted
moon info
moon fmt

    The Unicode data generator is written in MoonBit script mode and lives under tools/. Generated tables are reproducible from the pinned Unicode inputs in third_party/unicode/ and are not counted as hand-written source.

    #License

    MoonCollate is licensed under Apache-2.0. Unicode data files remain subject to the Unicode License v3; see THIRD_PARTY_NOTICES.md.

    AlternateHandling

    source
    pub(all) enum AlternateHandling {
      NonIgnorable
      Shifted
    } derive(Eq, ToJson, 
    Debug    )

    Treatment of variable collation elements such as spaces and punctuation.

    CaseFirst

    source
    pub(all) enum CaseFirst {
      CaseOff
      UpperFirst
      LowerFirst
    } derive(Eq, ToJson, 
    Debug    )

    Optional case ordering applied at tertiary strength.

    CollationElement

    source
    pub(all) struct CollationElement {
      primary : Int
      secondary : Int
      tertiary : Int
      quaternary : Int
      variable : Bool
    } derive(Eq, ToJson, 
    Debug    )

    One collation element as defined by UTS #10.

    CollationElement::is_variable

    source
    fn CollationElement::is_variable(self : CollationElement) -> Bool

    CollationElement::new

    source
    fn CollationElement::new(primary~ : Int, secondary~ : Int, tertiary~ : Int, quaternary? : Int, variable? : Bool) -> CollationElement

    Create a collation element with explicit level weights.

    CollationElement::primary

    source
    fn CollationElement::primary(self : CollationElement) -> Int

    CollationElement::quaternary

    source
    fn CollationElement::quaternary(self : CollationElement) -> Int

    CollationElement::secondary

    source
    fn CollationElement::secondary(self : CollationElement) -> Int

    CollationElement::tertiary

    source
    fn CollationElement::tertiary(self : CollationElement) -> Int

    CollationGroup

    source
    pub(all) struct CollationGroup {
      representative : String
      values : Array[String]
    } derive(Eq, ToJson, 
    Debug    )

    A run of strings that compare equal at one collator's configured strength.

    CollationGroup::length

    source
    fn CollationGroup::length(self : CollationGroup) -> Int

    CollationGroup::representative

    source
    fn CollationGroup::representative(self : CollationGroup) -> String

    CollationGroup::values

    source
    fn CollationGroup::values(self : CollationGroup) -> Array[String]

    CollationIndex

    source
    pub(all) struct CollationIndex {
      collator : Collator
      indexed_values : Array[String]
      indexed_keys : Array[SortKey]
    }

    A sorted, immutable string index with cached collation keys.

    CollationIndex::contains

    source
    fn CollationIndex::contains(self : CollationIndex, target : String) -> Bool

    CollationIndex::count_equal

    source
    fn CollationIndex::count_equal(self : CollationIndex, target : String) -> Int

    Count values equal to target without allocating an equal-range array.

    CollationIndex::equal_range

    source
    fn CollationIndex::equal_range(self : CollationIndex, target : String) -> Array[String]

    Return all indexed values equal to target at configured strength.

    CollationIndex::is_empty

    source
    fn CollationIndex::is_empty(self : CollationIndex) -> Bool

    CollationIndex::length

    source
    fn CollationIndex::length(self : CollationIndex) -> Int

    CollationIndex::lower_bound

    source
    fn CollationIndex::lower_bound(self : CollationIndex, target : String) -> Int

    Return the insertion position before all values equal to target.

    CollationIndex::new

    source
    fn CollationIndex::new(collator : Collator, values : Array[String]) -> CollationIndex

    Build an immutable index for repeated lookup and range queries.

    CollationIndex::range

    source
    fn CollationIndex::range(self : CollationIndex, lower : String, upper : String) -> Array[String]

    Return values in the half-open collation interval [lower, upper).

    An empty array is returned when the bounds compare equal or are reversed.

    CollationIndex::upper_bound

    source
    fn CollationIndex::upper_bound(self : CollationIndex, target : String) -> Int

    Return the insertion position after all values equal to target.

    CollationIndex::values

    source
    fn CollationIndex::values(self : CollationIndex) -> Array[String]

    CollationTrace

    source
    pub(all) struct CollationTrace {
      original : String
      normalized : String
      steps : Array[TraceStep]
      key : SortKey
    } derive(Eq, ToJson, 
    Debug    )

    Full diagnostic trace for one input string.

    CollationTrace::normalized

    source
    fn CollationTrace::normalized(self : CollationTrace) -> String

    CollationTrace::original

    source
    fn CollationTrace::original(self : CollationTrace) -> String

    CollationTrace::sort_key

    source
    fn CollationTrace::sort_key(self : CollationTrace) -> SortKey

    CollationTrace::steps

    source

    Collator

    source
    pub(all) struct Collator {
      strength : Strength
      alternate : AlternateHandling
      case_first : CaseFirst
      backwards_secondary : Bool
      numeric : Bool
    }

    Immutable configuration and entry point for Unicode collation.

    Collator::alternate_handling

    source
    fn Collator::alternate_handling(self : Collator) -> AlternateHandling

    Collator::case_first

    source
    fn Collator::case_first(self : Collator) -> CaseFirst

    Collator::compare

    source
    fn Collator::compare(self : Collator, left : String, right : String) -> Ordering

    Compare two strings according to this collator's configuration.

    Collator::compare_detailed

    source
    fn Collator::compare_detailed(self : Collator, left : String, right : String) -> ComparisonReport

    Compare strings and retain enough structured evidence for diagnostics.

    Collator::deduplicate

    source
    fn Collator::deduplicate(self : Collator, values : Array[String]) -> Array[String]

    Remove collation-equal duplicates, retaining the first input occurrence.

    Collator::equivalent

    source
    fn Collator::equivalent(self : Collator, left : String, right : String) -> Bool

    Return true when two strings are equal at the configured strength.

    Collator::explain

    source
    fn Collator::explain(self : Collator, text : String) -> CollationTrace

    Explain normalization, DUCET mappings, and the resulting sort key.

    Collator::group_equal

    source
    fn Collator::group_equal(self : Collator, values : Array[String]) -> Array[CollationGroup]

    Group adjacent collation-equal values after a stable sort.

    Collator::is_backwards_secondary

    source
    fn Collator::is_backwards_secondary(self : Collator) -> Bool

    Collator::is_numeric

    source
    fn Collator::is_numeric(self : Collator) -> Bool

    Collator::is_sorted

    source
    fn Collator::is_sorted(self : Collator, values : Array[String]) -> Bool

    Return whether values are already ordered for this collator.

    Collator::less

    source
    fn Collator::less(self : Collator, left : String, right : String) -> Bool

    Return true when left sorts before right.

    Collator::maximum

    source
    fn Collator::maximum(self : Collator, values : Array[String]) -> String?

    Return the greatest value, or None for an empty input.

    Collator::minimum

    source
    fn Collator::minimum(self : Collator, values : Array[String]) -> String?

    Return the least value, or None for an empty input.

    Collator::new

    source
    fn Collator::new(strength? : Strength, alternate? : AlternateHandling, case_first? : CaseFirst, backwards_secondary? : Bool, numeric? : Bool) -> Collator

    Construct a collator. Defaults match the root DUCET profile.

    Collator::sort

    source
    fn Collator::sort(self : Collator, values : Array[String]) -> Array[String]

    Return a stable collation-ordered copy. The input array is not modified.

    Collator::sort_key

    source
    fn Collator::sort_key(self : Collator, text : String) -> SortKey

    Produce a stable multi-level UCA sort key for text.

    Collator::strength

    source
    fn Collator::strength(self : Collator) -> Strength

    Collator::with_alternate

    source
    fn Collator::with_alternate(self : Collator, alternate : AlternateHandling) -> Collator

    Create a copy with different variable-weight handling.

    Collator::with_backwards_secondary

    source
    fn Collator::with_backwards_secondary(self : Collator, enabled : Bool) -> Collator

    Create a copy with backwards-secondary comparison enabled or disabled.

    Collator::with_case_first

    source
    fn Collator::with_case_first(self : Collator, case_first : CaseFirst) -> Collator

    Create a copy with a different case-first policy.

    Collator::with_numeric

    source
    fn Collator::with_numeric(self : Collator, enabled : Bool) -> Collator

    Create a copy with numeric collation enabled or disabled.

    Collator::with_strength

    source
    fn Collator::with_strength(self : Collator, strength : Strength) -> Collator

    Create a copy with a different comparison strength.

    ComparisonDifference

    source
    pub(all) struct ComparisonDifference {
      level : DifferenceLevel
      index : Int
      left_weight : Int
      right_weight : Int
    } derive(Eq, ToJson, 
    Debug    )

    Location and weights of the first decisive sort-key difference.

    ComparisonDifference::index

    source

    ComparisonDifference::left_weight

    source
    fn ComparisonDifference::left_weight(self : ComparisonDifference) -> Int

    ComparisonDifference::level

    source

    ComparisonDifference::right_weight

    source
    fn ComparisonDifference::right_weight(self : ComparisonDifference) -> Int

    ComparisonReport

    source
    pub(all) struct ComparisonReport {
      ordering : Ordering
      difference : ComparisonDifference?
      left_key : SortKey
      right_key : SortKey
    } derive(Eq, ToJson, 
    Debug    )

    A comparison result with both keys and the first decisive difference.

    ComparisonReport::difference

    source

    ComparisonReport::left_key

    source
    fn ComparisonReport::left_key(self : ComparisonReport) -> SortKey

    ComparisonReport::ordering

    source

    ComparisonReport::right_key

    source
    fn ComparisonReport::right_key(self : ComparisonReport) -> SortKey

    DifferenceLevel

    source
    pub(all) enum DifferenceLevel {
      PrimaryLevel
      SecondaryLevel
      TertiaryLevel
      QuaternaryLevel
      IdenticalLevel
    } derive(Eq, ToJson, 
    Debug    )

    The first sort-key level that decides a comparison.

    DifferenceLevel::to_string

    source
    fn DifferenceLevel::to_string(self : DifferenceLevel) -> String

    Ordering

    source
    pub(all) enum Ordering {
      Less
      Equal
      Greater
    } derive(Eq, ToJson, 
    Debug    )

    Comparison result returned by a collator.

    Ordering::to_string

    source
    fn Ordering::to_string(self : Ordering) -> String

    Convert an ordering result to a stable textual representation.

    SortKey

    source
    pub(all) struct SortKey {
      primary : Array[Int]
      secondary : Array[Int]
      tertiary : Array[Int]
      quaternary : Array[Int]
      identical : Array[Int]
    } derive(Eq, ToJson, 
    Debug    )

    A stable, lexicographically comparable collation key.

    SortKey::flatten

    source
    fn SortKey::flatten(self : SortKey, strength : Strength) -> Array[Int]

    Flatten a structured key through strength, separating levels with zero. All emitted UCA weights are non-zero, so separators remain unambiguous.

    SortKey::identical

    source
    fn SortKey::identical(self : SortKey) -> Array[Int]

    SortKey::primary

    source
    fn SortKey::primary(self : SortKey) -> Array[Int]

    SortKey::quaternary

    source
    fn SortKey::quaternary(self : SortKey) -> Array[Int]

    SortKey::secondary

    source
    fn SortKey::secondary(self : SortKey) -> Array[Int]

    SortKey::tertiary

    source
    fn SortKey::tertiary(self : SortKey) -> Array[Int]

    SortKey::to_hex

    source
    fn SortKey::to_hex(self : SortKey, strength : Strength) -> String

    Format a flattened key as uppercase hexadecimal words.

    Strength

    source
    pub(all) enum Strength {
      Primary
      Secondary
      Tertiary
      Quaternary
      Identical
    } derive(Eq, ToJson, 
    Debug    )

    The strongest collation level considered by a comparison.

    TraceStep

    source
    pub(all) struct TraceStep {
      input : Array[Int]
      elements : Array[CollationElement]
      source : String
    } derive(Eq, ToJson, 
    Debug    )

    A single mapping decision recorded by the explanation engine.

    TraceStep::elements

    source
    fn TraceStep::elements(self : TraceStep) -> Array[CollationElement]

    TraceStep::input

    source
    fn TraceStep::input(self : TraceStep) -> Array[Int]

    TraceStep::source

    source
    fn TraceStep::source(self : TraceStep) -> String

    UCA_VERSION

    source
    let UCA_VERSION : String

    Unicode Collation Algorithm version implemented by this release.

    UNICODE_VERSION

    source
    let UNICODE_VERSION : String

    Unicode version used by the bundled DUCET data.

    VERSION

    source
    let VERSION : String

    MoonCollate library version.

    default_collator

    source
    fn default_collator() -> Collator

    Return the default Unicode collator.

    Source Files

    Powered by MoonBit

    Site sourceReport issuePackagesBuild queueSkillsStatistics

    © 2026 mooncakes.io