fst

    Compact immutable byte-string sets and maps, ported from Rust fst 0.4.7

    fst
    automaton
    dictionary
    search
    compression
    Download zip
    Version
    0.1.0
    License
    MIT
    Last updated
    4 hours ago
    Downloads
    4

    No dependencies

    #MoonFST

    starAndHonor/fst 将 Rust fst 0.4.7 的紧凑有序集合与映射移植到 MoonBit。键是任意字节串,值覆盖完整 UInt64 范围;相同前缀和等价后缀可以共享图结构,适合构建只读词典、自动补全索引、符号表和静态路由表。

    运行库使用纯 MoonBit,不链接 Rust/C 库,也不依赖 JavaScript FFI。Rust 参考程序、Python 脚本及真实负载工具的 C 文件仅用于开发验收;C 代码只负责文件 I/O 和 Linux RSS 读数,不参与 FST 算法。

    #安装与快速运行

    在消费项目根目录安装 starAndHonor/fst@0.1.0:

    moon add starAndHonor/fst@0.1.0

    也可以在 consumer/moon.mod 中声明依赖:

    name = "demo/consumer" import { "starAndHonor/fst@0.1.0", }

    在 consumer/moon.pkg 中添加测试导入:

    import { "starAndHonor/fst" @fst, "starAndHonor/fst/automaton" @automaton, } for "test"

    下面是完整的 consumer/quickstart_test.mbt:

    test {
    // 便捷构造器复制、排序输入;Set 自动合并重复键。
    let words = @fst.FstSet::from_strings(["月亮", "月光", "太阳", "月亮"])
    assert_eq(words.len(), 3UL)
    assert_eq(words.prefix_str("月").strings(), ["月亮", "月光"])

    // 编辑距离按 Unicode scalar value 计算。
    let fuzzy = @automaton.levenshtein("月亮", 1)
    assert_eq(words.search(fuzzy).strings(), ["月亮", "月光"])

    // Map 默认拒绝重复键;UInt64 不经过浮点数转换。
    let counts = @fst.FstMap::from_strings([
    ("apple", 10UL),
    ("apricot", 20UL),
    ("banana", 18446744073709551615UL),
    ])
    assert_eq(counts.get_str("banana"), Some(18446744073709551615UL))
    let restored = @fst.FstMap::from_bytes(counts.to_bytes())
    assert_eq(restored.get_str("apple"), Some(10UL))

    // 已有序的数据可以增量写入,避免先复制并排序所有键。
    let builder = @fst.Builder::map()
    builder.insert(b"a", 1UL)
    builder.insert(b"ab", 2UL)
    builder.insert(b"b", 3UL)
    let built = builder.into_map()
    assert_eq(built.get(b"ab"), Some(2UL))

    let index = built.as_fst().index()
    assert_eq(index.rank(b"ab"), (1UL, true))
    assert_eq(index.prefix_count(b"a"), 2UL)
    assert_eq(index.select(2UL).unwrap().key, b"b")

    // 对匹配结果直接计数、定位和分页,无须收集所有命中键。
    let matching = index.search_index(@automaton.prefix_bytes(b"a"))
    assert_eq(matching.len(), 2UL)
    assert_eq(matching.select(1UL).unwrap().key, b"ab")
    assert_eq(index.page_reverse(0UL, 1)[0].key, b"b")

    let statistics = index.prefix_value_statistics(b"a")
    assert_eq(statistics.sum, Some(3UL))
    assert_eq(index.top(1)[0].key, b"b")

    let cursor = built.as_fst().cursor()
    assert_true(cursor.advance_bytes(b"ab"))
    assert_eq(cursor.value(), Some(2UL))
    }

    在消费项目根目录运行 moon test . --target wasm-gc --deny-warn。

    #本地工作区

    发布前或需要修改源码时,可以使用本地工作区。将仓库 moonfst/ 与消费项目 consumer/ 放在同一目录,在它们的父目录创建 moon.work:

    members = [ "./moonfst", "./consumer", ]

    消费项目仍使用上面的依赖与导入配置;工作区按模块名解析本地成员,版本号不会触发 Mooncakes 下载。此配置遵循 MoonBit 的本地依赖说明。在父目录运行 moon -C consumer test . --target wasm-gc --deny-warn。

    #词典示例

    在源码仓库根目录运行:

    moon run examples/dictionary --target wasm-gc

    示例演示建表、中文前缀、Unicode 模糊查询、rank/page、最高词频和词频合计。也可以用 --target native 或 --target js。

    #数据与顺序

    • 键按无符号字节的字典序排列;较短的公共前缀先出现。空键、NUL 和非法 UTF-8 都是合法的字节键。
    • *_str、from_strings 使用 UTF-8 编码,不进行大小写折叠、规范化或语言相关排序。调用 strings() 遇到非法 UTF-8 会报错,且保留开头的 BOM 字符。
    • Builder::set 要求非递减键,相同键合并;Builder::map 要求严格递增键,相同键报错。有序要求是 UTF-8 字节序,不能依赖 MoonBit 的 UTF-16 字符串比较来预排序任意文本。
    • FstMap::from_entries 和 from_strings 可选择 Reject、KeepFirst、KeepLast、Sum、Minimum、Maximum 重复键策略。加法溢出明确报错。
    • 构建结果不可原地修改。批量变更可以合并有序流,再构建新 FST。

    registry_capacity 限制构建器保留的等价节点缓存;淘汰缓存可能降低后缀共享率,不改变查询结果。因此不承诺任意缓存大小都得到全局最小图。构建器的输出字节缓冲仍在内存中。

    #查询与集合操作

    FstSet、FstMap 都提供精确查询、前缀流、自动机搜索和有界范围流。KeyRange 的上下界可分别包含或排除;between 是左闭右开,closed 是两端包含。Stream 按字节序惰性遍历,支持 next、take、skip、count、close。

    merge / MergeStream 支持多路并集、交集、差集和对称差集,并保留每个值的来源编号。Map 合并再显式选择重复值的归约策略。多路对称差集使用奇数次出现语义。

    本项目增加以下可选工具:

    • FstIndex:缓存每个节点的后缀路径数,提供 rank、select、前缀计数、上下界、区间计数和分页;索引构建按图节点与边计算,不枚举全部键。缓存需要额外内存。
    • SearchIndex:缓存 FST 与自动机的乘积图,对匹配结果提供计数、rank/select、范围查询和分页。max_states 限制乘积状态数;超限明确报错。自定义自动机的状态及剪枝承诺必须在索引整个使用期内保持稳定。
    • ValueStatistics 与 top:FstIndex 可按前缀、键范围或序号区间统计条数、最小值、最大值与总和;总和超出 UInt64 时为 None,其他字段仍精确。top 使用子图最大值优先探索,按值降序返回结果,同值按字节序排列。
    • ReverseStream:按字节序降序惰性遍历,支持范围和自动机搜索;自动机仍从键的首字节向后消费。FstIndex 和 SearchIndex 的 page_reverse 支持从大键一侧分页。
    • Cursor:逐字节推进、回退、原子批量推进、定位和独立分叉。适合词法扫描或自定义遍历;推进失败保留原位置。

    EntrySource 让应用接入按字节序严格递增的外部数据源;Stream 与经过顺序验证的 ArraySource 均实现此接口。filter_source、map_source、window_source 和 fold_source 允许惰性过滤、修改值、截取与归约,Builder::extend_source 和 merge_sources 可直接消费数据。CheckedSource 检查外部源的单调性;检查、过滤、值映射与窗口适配器在读取或回调失败后关闭底层源,并保持关闭,重复 close 不重复传递。多路合并也在失败时关闭全部输入。提前停止消费时应显式 close,同一可消费源不能通过别名并发迭代。降序的 ReverseStream 不实现要求升序的 EntrySource。

    #自动机

    automaton 包包含精确字节匹配、前缀、字节子序列、并/交/补、starts_with、单字节范围、字节长度约束,以及可验证的显式 DFA 构建器。Matcher 实现公开的 Automaton trait,可直接传给 search,也支持用户自定义实现。

    Levenshtein 自动机将插入、删除和替换各计为一次编辑,按 Unicode scalar value 计算,表情等补充平面字符计为一个标量,组合附加符号单独计数。它不把转置视为一次编辑,也不自动合并规范等价文本。例如 é 与 e 加组合重音之间的距离为 2。

    Levenshtein 只接受严格有效的 UTF-8:截断、多余 continuation byte、过长编码、代理项及超出 U+10FFFF 的编码均不匹配。构造时预先建立有界 DFA,搜索时不维护增长缓存。max_states 和 max_work 超限返回 AutomatonError;不能通过超限后静默剪枝来产生不完整结果。复合自动机也检查状态编号乘积的上限。

    自定义自动机的 can_match / will_always_match 是剪枝承诺,必须保持保守正确。状态 ID 在一次遍历中应稳定;EOF 转移只在完整键结尾应用一次。

    #格式兼容与内存边界

    写入格式为 Rust fst v3,包括小端整数、节点编码、输出因子和 masked CRC32C 校验。固定参考版本是 fst = "=0.4.7",上游提交 fbd5992376f176c35af2644d74efad58f5656e5c。双向验收比较完整键值与查询结果;不同构建器可以产生等价但字节不同的压缩图。

    Fst::from_bytes 在开放查询前验证校验和及完整可达图,包括地址、边顺序、节点范围、接受键数量和路径输出溢出。读取器还识别旧版 v1/v2 元数据;跨语言参考验收主要针对 v3。

    当前实现接收并完整验证内存 Bytes。精确查询和遍历直接读取压缩节点,公开节点检查才生成独立数组;不提供 mmap 或磁盘流式构建。地址受当前 Int32 可表示范围限制,不能据此承诺 Rust 上游的超大索引规模。FstIndex 会额外保留解码节点、累计计数和值统计;SearchIndex 还需乘积图缓存。

    #验证

    以下开发命令从源码仓库根目录运行。Mooncakes 发布归档保留运行库、单元测试、示例及说明文档;基准、外部语料、Rust/Python 工具和跨语言集成入口仅在源码仓库提供。

    #合成数据基准

    与 Rust fst 0.4.7 的可复现 native release 基准:

    python3 scripts/benchmark.py --output benchmarks/results/my-run

    覆盖构建、命中/近似未命中查询、完整遍历、前缀查询、并集和编码大小;默认两种数据分布、三个规模,每项 21 个样本,并验证两边结果。测量方法见 benchmarks/README.md,2026-11-06 查询优化后的实测见基准报告,原始慢版本的基线报告保留用于对照。

    该次合成数据测量中,两种分布、三个规模的排序命中与近似未命中查询均快于 Rust,编码文件尺寸相同;10 万条时查询加速约 1.55~2.08 倍。全量遍历基本持平,并集仍有差距。具体实现、安全前提、逐字节互操作和防优化屏障控制实验见优化与验收说明。这些数字对应报告记录的源码与环境,不代表后续改动后的重新测量。

    #真实词典、查询顺序与 RSS

    在 Linux 上运行真实负载工具:

    python3 scripts/workload_benchmark.py \ --sizes 10000 100000 200000 --orders sorted shuffled \ --samples 7 --warmups 2 --rounds 3 --memory-rounds 3 \ --output benchmarks/results/my-real-run

    工具按固定提交与 SHA256 读取 MIT 授权的 Jieba 词典,保留词频,从 349,045 个唯一词条中抽样;A/B 各 N 条、重叠 N/2。每词一次查询,比较排序与固定种子打乱的顺序,以及 50% 命中混合请求。近似未命中是在完整 UTF-8 词条后追加 FF;这些请求不是实际查询日志,也没有按词频加权。Python 独立校验 A/B、并集、查询顺序及操作结果。

    2026-11-07 的优化后真实词典报告采用测量方法 revision 2:两端共享不可变输入键,查询只保留批次级防优化处理,每项 3 轮 × 7 次采样。shuffled 运行组中,1 万、10 万、20 万条并集耗时分别为 Rust 的 80.8%、78.9%、83.4%;sorted 运行组的 10 万条并集基本持平。20 万条 shuffled 命中/未命中/混合查询耗时分别为 Rust 的 73.2%、73.9%、53.2%。三档编码逐字节相同,分别为 123,328、1,035,223、1,949,554 B。这些是单机中位数,完整分位数与限制见报告。

    旧真实词典基线保留原始证据;它使用旧 Rust 输入深拷贝和逐查询屏障,不能与 revision 2 混合汇总。实现改动与受控前后比较见并集优化说明。语料来源和许可证见语料说明。

    内存测量为 baseline/build/load/union 各自启动新进程,读取 Linux /proc/self/status 的 VmRSS 与 VmHWM,保留操作前、操作后和内容验证后的快照。RSS 包含输入表示、分配器保留及运行时;高水位包含准备阶段,不能视为精确堆分配或操作净内存。load 使用预先导出的 FST:Rust 打开并校验 CRC,MoonBit 还验证完整可达图,两者工作量不同。全新进程不等于冷缓存,以上均为单机 native 热运行,不外推到磁盘吞吐或其他后端。

    #正确性复验

    当前实测工具链为 moon 0.1.20261104 与 moonc v0.10.20+65efd5b93。配置采用 moon.mod / moon.pkg,整型索引采用 Int32,公开 trait 方法使用显式 extend。JavaScript 测试还需要 Node.js,native 测试需要工具链要求的 C 编译器。

    moon fmt --check moon test --target native --deny-warn moon test --target js --deny-warn moon test --target wasm --deny-warn moon test --target wasm-gc --deny-warn

    跨语言验收另需 Python 3 和 Rust/Cargo:

    cargo test --locked --manifest-path oracle/Cargo.toml python3 scripts/generate_fixtures.py --check python3 scripts/verify_oracle.py python3 scripts/interop.py --target native --target js --target wasm-gc python3 scripts/random_interop.py --target native

    测试材料覆盖任意字节、完整 u64 边界、共享后缀、稠密分支、长路径、损坏输入和固定种子的随机差分。自动机另有独立标量 DP 对照和严格 UTF-8 解码对照。具体参考协议见 oracle/README.md,二进制样本见 fixtures/README.md,完整复验步骤见开发工具说明。

    python3 scripts/count_loc.py 将生产 MoonBit、测试、生成文件和其他语言工具分开统计。实现细节见 ARCHITECTURE.md;来源与许可见 THIRD_PARTY.md 和 LICENSE。

    EntrySource

    pub(open) trait EntrySource {
    fn next(Self) -> Entry? raise FstError
    fn close(Self) -> Unit
    }

    A consumable byte-lexicographic stream of unique keys. Sources own their iteration position and must not be consumed concurrently through aliases. Applications can implement this for file readers and database cursors.

    FstError

    pub(all) suberror FstError {
    InvalidArgument(String)
    OutOfOrder(Bytes, Bytes)
    DuplicateKey(Bytes)
    BuilderClosed
    InvalidFormat(String)
    InvalidUtf8
    Overflow
    ResourceLimit(String)
    } derive(
    Debug
    )

    Errors reported at the public library boundary.

    FstError::to_repr

    Explicit method exports for MoonBit 0.10.19. These preserve the intended public dot-call API without implicitly promoting every future trait method.

    ArraySource

    pub struct ArraySource {
    // private fields
    }

    A source backed by a copied array of immutable entries.

    ArraySource::close

    fn ArraySource::close(self : ArraySource) -> Unit

    ArraySource::from_sorted

    fn ArraySource::from_sorted(entries : Array[Entry]) -> ArraySource raise FstError

    Validate input order once without constructing an intermediate FST.

    ArraySource::from_sorted_keys

    fn ArraySource::from_sorted_keys(keys : Array[Bytes]) -> ArraySource raise FstError

    Set input permits and coalesces consecutive repeated keys.

    ArraySource::next

    fn ArraySource::next(self : ArraySource) -> Entry? raise FstError

    ArraySource::remaining

    fn ArraySource::remaining(self : ArraySource) -> Int

    Bound

    pub(all) enum Bound {
    Unbounded
    Included(Bytes)
    Excluded(Bytes)
    } derive(Eq,
    Debug
    )

    Inclusive or exclusive lexicographic range endpoint.

    Bound::equal

    fn Bound::equal(Bound, Bound) -> Bool

    Bound::not_equal

    fn Bound::not_equal(x : Bound, y : Bound) -> Bool

    Bound::to_repr

    Builder

    pub struct Builder {
    // private fields
    }

    Sorted incremental construction, with a bounded state-deduplication cache.

    Builder::add

    fn Builder::add(self : Builder, key : Bytes) -> Unit raise FstError

    Builder::add_str

    fn Builder::add_str(self : Builder, key : String) -> Unit raise FstError

    Builder::extend_entries

    fn Builder::extend_entries(self : Builder, entries : Array[Entry]) -> Unit raise FstError

    Errors leave previously inserted entries intact; later entries are unprocessed.

    Builder::extend_source

    fn Builder::extend_source(self : Builder, source : &EntrySource) -> Unit raise FstError

    Extend a builder from an application source without temporary arrays. Set builders coalesce sorted duplicates; map builders require unique keys.

    Builder::extend_stream

    fn Builder::extend_stream(self : Builder, stream : Stream) -> Unit raise FstError

    Extend a sorted builder directly without materializing the result set.

    Builder::finish

    fn Builder::finish(self : Builder) -> Bytes raise FstError

    Finish once. Afterwards the builder rejects inserts and further finishes.

    Builder::insert

    fn Builder::insert(self : Builder, key : Bytes, value : UInt64) -> Unit raise FstError

    Builder::insert_str

    fn Builder::insert_str(self : Builder, key : String, value : UInt64) -> Unit raise FstError

    Builder::into_fst

    fn Builder::into_fst(self : Builder) -> Fst raise FstError

    Builder::into_map

    fn Builder::into_map(self : Builder) -> FstMap raise FstError

    Builder::into_set

    fn Builder::into_set(self : Builder) -> FstSet raise FstError

    Builder::map

    fn Builder::map(registry_capacity? : Int, fst_type? : UInt64) -> Builder raise FstError

    Map keys must be strictly increasing. Values span the full UInt64 range.

    Builder::set

    fn Builder::set(registry_capacity? : Int, fst_type? : UInt64) -> Builder raise FstError

    Duplicate set keys are accepted and counted once. Keys must be sorted.

    Builder::statistics

    fn Builder::statistics(self : Builder) -> BuilderStatistics

    BuilderStatistics

    pub(all) struct BuilderStatistics {
    keys : UInt64
    compiled_nodes : Int
    reused_nodes : Int
    registry_entries : Int
    unfinished_nodes : Int
    bytes_written : Int
    } derive(Eq,
    Debug
    )

    Observable builder memory/compression counters.

    BuilderStatistics::equal

    BuilderStatistics::not_equal

    fn BuilderStatistics::not_equal(x : BuilderStatistics, y : BuilderStatistics) -> Bool

    CheckedSource

    pub struct CheckedSource {
    // private fields
    }

    A validating adapter. A source error or order violation permanently closes this adapter, preventing stale data from leaking after a caught error.

    CheckedSource::close

    fn CheckedSource::close(self : CheckedSource) -> Unit

    CheckedSource::new

    CheckedSource::next

    fn CheckedSource::next(self : CheckedSource) -> Entry? raise FstError

    Cursor

    pub struct Cursor {
    // private fields
    }

    A decoded byte cursor, useful for custom tokenization and automata. Advancing one cursor never mutates the immutable FST or another cursor.

    Cursor::address

    fn Cursor::address(self : Cursor) -> Int

    Cursor::advance

    fn Cursor::advance(self : Cursor, byte : Byte) -> Bool raise FstError

    Advance if this byte has an outgoing transition. A failed advance leaves the cursor exactly where it was, which permits alternative token attempts.

    Cursor::advance_bytes

    fn Cursor::advance_bytes(self : Cursor, suffix : Bytes) -> Bool raise FstError

    Advance the entire suffix atomically; on failure the starting position is restored. Bytes before the failed byte are not left consumed.

    Cursor::back

    fn Cursor::back(self : Cursor) -> Bool

    Rewind one byte. The root cannot be rewound.

    Cursor::completions

    fn Cursor::completions(self : Cursor) -> Stream raise FstError

    Iterate the accepted extensions of the current cursor prefix.

    Cursor::depth

    fn Cursor::depth(self : Cursor) -> Int

    Cursor::fork

    fn Cursor::fork(self : Cursor) -> Cursor

    Duplicate cursor position and path; future movement is independent.

    Cursor::key

    fn Cursor::key(self : Cursor) -> Bytes

    Cursor::output_prefix

    fn Cursor::output_prefix(self : Cursor) -> UInt64

    Output accumulated on transitions; it excludes a final-state output.

    Cursor::reset

    fn Cursor::reset(self : Cursor) -> Unit

    Cursor::seek

    fn Cursor::seek(self : Cursor, key : Bytes) -> Bool raise FstError

    Seek from the root. Failure restores the previous position.

    Cursor::transitions

    Return an owned snapshot so callers cannot modify the cursor's nodes.

    Cursor::truncate

    fn Cursor::truncate(self : Cursor, depth : Int) -> Unit raise FstError

    Keep a prefix of the current path; requesting a deeper or negative depth is an error rather than silently changing the requested position.

    Cursor::value

    fn Cursor::value(self : Cursor) -> UInt64?

    The full map value if the current prefix is an accepted key.

    DuplicatePolicy

    pub(all) enum DuplicatePolicy {
    Reject
    KeepFirst
    KeepLast
    Sum
    Minimum
    Maximum
    } derive(Eq,
    Debug
    )

    How an unsorted map constructor resolves duplicate keys.

    DuplicatePolicy::equal

    DuplicatePolicy::not_equal

    fn DuplicatePolicy::not_equal(x : DuplicatePolicy, y : DuplicatePolicy) -> Bool

    Entry

    pub(all) struct Entry {
    key : Bytes
    value : UInt64
    } derive(Eq,
    Debug
    )

    One owned key/value result. Keys are arbitrary bytes, including NUL.

    Entry::equal

    fn Entry::equal(Entry, Entry) -> Bool

    Entry::not_equal

    fn Entry::not_equal(x : Entry, y : Entry) -> Bool

    Entry::to_repr

    FilterSource

    pub struct FilterSource {
    // private fields
    }

    Filter entries lazily while preserving their key order.

    FilterSource::close

    fn FilterSource::close(self : FilterSource) -> Unit

    FilterSource::next

    fn FilterSource::next(self : FilterSource) -> Entry? raise FstError

    Fst

    pub struct Fst {
    // private fields
    }

    An immutable validated FST. Arrays exposed by inspection are fresh copies.

    Fst::byte_size

    fn Fst::byte_size(self : Fst) -> Int

    Fst::common_prefixes

    fn Fst::common_prefixes(self : Fst, input : Bytes) -> Array[Entry] raise FstError

    All stored keys that are prefixes of input, shortest first.

    Fst::contains

    fn Fst::contains(self : Fst, key : Bytes) -> Bool raise FstError

    Fst::contains_str

    fn Fst::contains_str(self : Fst, key : String) -> Bool raise FstError

    Fst::cursor

    fn Fst::cursor(self : Fst) -> Cursor raise FstError

    Fst::from_bytes

    fn Fst::from_bytes(data : Bytes) -> Fst raise FstError

    Validate the complete reachable graph and checksum before exposing it.

    Fst::fst_type

    fn Fst::fst_type(self : Fst) -> UInt64

    Fst::get

    fn Fst::get(self : Fst, key : Bytes) -> UInt64? raise FstError

    Fst::get_str

    fn Fst::get_str(self : Fst, key : String) -> UInt64? raise FstError

    Fst::index

    fn Fst::index(self : Fst) -> FstIndex raise FstError

    Build suffix counts without enumerating accepted strings or using recursion.

    Fst::is_empty

    fn Fst::is_empty(self : Fst) -> Bool

    Fst::len

    fn Fst::len(self : Fst) -> UInt64

    Fst::longest_prefix

    fn Fst::longest_prefix(self : Fst, input : Bytes) -> Entry? raise FstError

    Find the longest accepted prefix, including the empty key if present.

    Fst::node

    fn Fst::node(self : Fst, address : Int) ->
    Node
    raise FstError

    Inspect a copy of a decoded node. Invalid addresses return format errors.

    Fst::prefix

    fn Fst::prefix(self : Fst, prefix : Bytes) -> Stream raise FstError

    Fst::prefix_str

    fn Fst::prefix_str(self : Fst, prefix : String) -> Stream raise FstError

    Fst::reverse

    fn Fst::reverse(self : Fst, range? : KeyRange) -> ReverseStream raise FstError

    Traverse greatest keys first without collecting and reversing a forward traversal. Memory is proportional to the active path.

    Fst::root_address

    fn Fst::root_address(self : Fst) -> Int

    Fst::search

    fn Fst::search(self : Fst, matcher : &
    Automaton
    , range? : KeyRange) -> Stream raise FstError

    Automata prune subtrees before their compressed-node cursors are opened.

    Fst::search_reverse

    fn Fst::search_reverse(self : Fst, matcher : &
    Automaton
    , range? : KeyRange) -> ReverseStream raise FstError

    Automaton states still consume each key from its first byte to its last; reverse traversal changes only the order in which complete keys are emitted.

    Fst::statistics

    fn Fst::statistics(self : Fst) -> Statistics raise FstError

    Structural analysis is proportional to nodes and transitions, not keys.

    Fst::stream

    fn Fst::stream(self : Fst, range? : KeyRange) -> Stream raise FstError

    Fst::to_bytes

    fn Fst::to_bytes(self : Fst) -> Bytes

    Fst::to_dot

    fn Fst::to_dot(self : Fst) -> String raise FstError

    Build a DOT graph without expanding shared nodes more than once.

    Fst::version

    fn Fst::version(self : Fst) -> UInt64

    FstIndex

    pub struct FstIndex {
    // private fields
    }

    Optional in-memory index for rank/select. Construction costs O(V log V + E) time and O(V + E) space; the compact FST itself does not require this cache.

    FstIndex::as_fst

    fn FstIndex::as_fst(self : FstIndex) -> Fst

    FstIndex::cached_nodes

    fn FstIndex::cached_nodes(self : FstIndex) -> Int

    FstIndex::floor

    fn FstIndex::floor(self : FstIndex, query : Bytes) -> Entry?

    Greatest key <= query.

    FstIndex::len

    fn FstIndex::len(self : FstIndex) -> UInt64

    FstIndex::lower_bound

    fn FstIndex::lower_bound(self : FstIndex, query : Bytes) -> Entry?

    Smallest key >= query.

    FstIndex::page

    fn FstIndex::page(self : FstIndex, offset : UInt64, limit : Int, range? : KeyRange) -> Array[Entry] raise FstError

    Jump to a page without consuming its preceding keys. Offset is relative to the selected range; a zero limit and an out-of-range offset are empty.

    FstIndex::page_reverse

    fn FstIndex::page_reverse(self : FstIndex, offset : UInt64, limit : Int, range? : KeyRange) -> Array[Entry] raise FstError

    Offset is relative to the greatest key in the selected range. Use this for backward pagination without enumerating the preceding descending results.

    FstIndex::predecessor

    fn FstIndex::predecessor(self : FstIndex, query : Bytes) -> Entry?

    Greatest key < query.

    FstIndex::prefix_count

    fn FstIndex::prefix_count(self : FstIndex, prefix : Bytes) -> UInt64

    Number of keys starting with prefix. This takes O(prefix length * log fanout), even when the subtree represents millions of keys.

    FstIndex::prefix_count_str

    fn FstIndex::prefix_count_str(self : FstIndex, prefix : String) -> UInt64

    FstIndex::prefix_ranks

    fn FstIndex::prefix_ranks(self : FstIndex, prefix : Bytes) -> (UInt64, UInt64)

    Produce a rank interval for every key under a prefix.

    FstIndex::prefix_value_statistics

    fn FstIndex::prefix_value_statistics(self : FstIndex, prefix : Bytes) -> ValueStatistics

    All matching keys under a prefix, including the prefix itself if present.

    FstIndex::range_count

    fn FstIndex::range_count(self : FstIndex, range : KeyRange) -> UInt64

    FstIndex::range_ranks

    fn FstIndex::range_ranks(self : FstIndex, range : KeyRange) -> (UInt64, UInt64)

    Convert a byte range into a half-open ordinal interval. Reversed or contradictory bounds produce an empty interval at the lower endpoint.

    FstIndex::rank

    fn FstIndex::rank(self : FstIndex, key : Bytes) -> (UInt64, Bool)

    Returns (number of keys strictly less than key, whether key exists). Missing keys have a useful insertion rank as in a sorted array.

    FstIndex::rank_str

    fn FstIndex::rank_str(self : FstIndex, key : String) -> (UInt64, Bool)

    FstIndex::search_index

    fn FstIndex::search_index(self : FstIndex, matcher : &
    Automaton
    , max_states? : Int) -> SearchIndex raise FstError

    Build an automaton/FST product using iterative traversal. State identities and pruning promises must stay stable for the lifetime of the returned index.

    FstIndex::select

    fn FstIndex::select(self : FstIndex, ordinal : UInt64) -> Entry?

    Zero-based order statistic. Selecting len or larger returns None.

    FstIndex::top

    fn FstIndex::top(self : FstIndex, limit : Int, range? : KeyRange) -> Array[Entry] raise FstError

    Extract the top outputs within a byte range using cached subtree maxima. Results are ordered by descending value, with byte-lexical key tie breaks. The frontier is a best-first heap, so a high-scoring narrow subtree can be explored before a large subtree whose outputs are all lower.

    FstIndex::upper_bound

    fn FstIndex::upper_bound(self : FstIndex, query : Bytes) -> Entry?

    Smallest key > query.

    FstIndex::value_statistics

    fn FstIndex::value_statistics(self : FstIndex, range? : KeyRange) -> ValueStatistics

    Aggregate all outputs in a range using cached subtree summaries. Whole subtrees are consumed in one step; their keys are never enumerated.

    FstIndex::value_statistics_at

    fn FstIndex::value_statistics_at(self : FstIndex, lower : UInt64, upper : UInt64) -> ValueStatistics raise FstError

    Aggregate a half-open interval of ordinal positions. Both endpoints may equal len, but a reversed interval or endpoint beyond len is an error.

    FstMap

    pub struct FstMap {
    // private fields
    }

    Map view: byte-string keys and unsigned 64-bit values.

    FstMap::as_fst

    fn FstMap::as_fst(self : FstMap) -> Fst

    FstMap::as_set

    fn FstMap::as_set(self : FstMap) -> FstSet

    FstMap::contains

    fn FstMap::contains(self : FstMap, key : Bytes) -> Bool raise FstError

    FstMap::difference

    fn FstMap::difference(self : FstMap, other : FstMap) -> FstMap raise FstError

    FstMap::entries

    fn FstMap::entries(self : FstMap) -> Array[Entry] raise FstError

    FstMap::from_bytes

    fn FstMap::from_bytes(data : Bytes) -> FstMap raise FstError

    FstMap::from_entries

    fn FstMap::from_entries(entries : Array[Entry], duplicates? : DuplicatePolicy, registry_capacity? : Int) -> FstMap raise FstError

    Resolve duplicates in original input order, independently of sort stability.

    FstMap::from_sorted

    fn FstMap::from_sorted(entries : Array[Entry], registry_capacity? : Int) -> FstMap raise FstError

    Build from strictly increasing byte keys. Duplicate keys are an error.

    FstMap::from_stream

    fn FstMap::from_stream(stream : Stream, registry_capacity? : Int) -> FstMap raise FstError

    FstMap::from_strings

    fn FstMap::from_strings(entries : Array[(String, UInt64)], duplicates? : DuplicatePolicy, registry_capacity? : Int) -> FstMap raise FstError

    FstMap::get

    fn FstMap::get(self : FstMap, key : Bytes) -> UInt64? raise FstError

    FstMap::get_str

    fn FstMap::get_str(self : FstMap, key : String) -> UInt64? raise FstError

    FstMap::intersection

    fn FstMap::intersection(self : FstMap, other : FstMap, policy : DuplicatePolicy) -> FstMap raise FstError

    FstMap::is_empty

    fn FstMap::is_empty(self : FstMap) -> Bool

    FstMap::len

    fn FstMap::len(self : FstMap) -> UInt64

    FstMap::map_values

    fn FstMap::map_values(self : FstMap, f : (Bytes, UInt64) -> UInt64 raise?) -> FstMap raise

    Transform values while preserving key order and compact output factoring.

    FstMap::prefix

    fn FstMap::prefix(self : FstMap, prefix : Bytes) -> Stream raise FstError

    FstMap::prefix_str

    fn FstMap::prefix_str(self : FstMap, prefix : String) -> Stream raise FstError

    FstMap::rebuild

    fn FstMap::rebuild(self : FstMap, registry_capacity? : Int) -> FstMap raise FstError

    FstMap::reverse

    fn FstMap::reverse(self : FstMap, range? : KeyRange) -> ReverseStream raise FstError

    FstMap::same_entries

    fn FstMap::same_entries(self : FstMap, other : FstMap) -> Bool raise FstError

    Compare logical key/value pairs rather than implementation-dependent bytes.

    FstMap::search

    fn FstMap::search(self : FstMap, matcher : &
    Automaton
    , range? : KeyRange) -> Stream raise FstError

    FstMap::slice

    fn FstMap::slice(self : FstMap, range : KeyRange) -> FstMap raise FstError

    FstMap::stream

    fn FstMap::stream(self : FstMap, range? : KeyRange) -> Stream raise FstError

    FstMap::string_entries

    fn FstMap::string_entries(self : FstMap) -> Array[(String, UInt64)] raise FstError

    FstMap::symmetric_difference

    fn FstMap::symmetric_difference(self : FstMap, other : FstMap) -> FstMap raise FstError

    FstMap::to_bytes

    fn FstMap::to_bytes(self : FstMap) -> Bytes

    FstMap::union

    fn FstMap::union(self : FstMap, other : FstMap, policy : DuplicatePolicy) -> FstMap raise FstError

    FstSet

    pub struct FstSet {
    // private fields
    }

    Set view of an immutable FST.

    FstSet::as_fst

    fn FstSet::as_fst(self : FstSet) -> Fst

    FstSet::contains

    fn FstSet::contains(self : FstSet, key : Bytes) -> Bool raise FstError

    FstSet::contains_str

    fn FstSet::contains_str(self : FstSet, key : String) -> Bool raise FstError

    FstSet::difference

    fn FstSet::difference(self : FstSet, other : FstSet) -> FstSet raise FstError

    FstSet::from_bytes

    fn FstSet::from_bytes(data : Bytes) -> FstSet raise FstError

    Interpret any valid FST as a set by ignoring outputs, as Rust fst does.

    FstSet::from_keys

    fn FstSet::from_keys(keys : Array[Bytes], registry_capacity? : Int) -> FstSet raise FstError

    Sort a copy; the caller's array is never reordered.

    FstSet::from_sorted

    fn FstSet::from_sorted(keys : Array[Bytes], registry_capacity? : Int) -> FstSet raise FstError

    Construct from sorted byte keys. Equal consecutive keys are coalesced.

    FstSet::from_stream

    fn FstSet::from_stream(stream : Stream, registry_capacity? : Int) -> FstSet raise FstError

    Stream values and keys into a fresh zero-output set.

    FstSet::from_strings

    fn FstSet::from_strings(keys : Array[String], registry_capacity? : Int) -> FstSet raise FstError

    Strings are indexed by their UTF-8 bytes without normalization.

    FstSet::intersection

    fn FstSet::intersection(self : FstSet, other : FstSet) -> FstSet raise FstError

    FstSet::is_disjoint

    fn FstSet::is_disjoint(self : FstSet, other : FstSet) -> Bool raise FstError

    FstSet::is_empty

    fn FstSet::is_empty(self : FstSet) -> Bool

    FstSet::is_subset

    fn FstSet::is_subset(self : FstSet, other : FstSet) -> Bool raise FstError

    Short-circuit comparison that does not materialize either set.

    FstSet::is_superset

    fn FstSet::is_superset(self : FstSet, other : FstSet) -> Bool raise FstError

    FstSet::keys

    fn FstSet::keys(self : FstSet) -> Array[Bytes] raise FstError

    FstSet::len

    fn FstSet::len(self : FstSet) -> UInt64

    FstSet::prefix

    fn FstSet::prefix(self : FstSet, prefix : Bytes) -> Stream raise FstError

    FstSet::prefix_str

    fn FstSet::prefix_str(self : FstSet, prefix : String) -> Stream raise FstError

    FstSet::rebuild

    fn FstSet::rebuild(self : FstSet, registry_capacity? : Int) -> FstSet raise FstError

    Re-encode keys with the current writer and chosen registry budget.

    FstSet::reverse

    fn FstSet::reverse(self : FstSet, range? : KeyRange) -> ReverseStream raise FstError

    FstSet::same_keys

    fn FstSet::same_keys(self : FstSet, other : FstSet) -> Bool raise FstError

    Compare logical keys; byte encodings may differ due to registry budgets.

    FstSet::search

    fn FstSet::search(self : FstSet, matcher : &
    Automaton
    , range? : KeyRange) -> Stream raise FstError

    FstSet::slice

    fn FstSet::slice(self : FstSet, range : KeyRange) -> FstSet raise FstError

    Keep only a range without allocating an intermediate array of keys.

    FstSet::stream

    fn FstSet::stream(self : FstSet, range? : KeyRange) -> Stream raise FstError

    Results remain byte sorted and own their key bytes.

    FstSet::strings

    fn FstSet::strings(self : FstSet) -> Array[String] raise FstError

    Fails if any accepted key is not valid UTF-8.

    FstSet::symmetric_difference

    fn FstSet::symmetric_difference(self : FstSet, other : FstSet) -> FstSet raise FstError

    FstSet::to_bytes

    fn FstSet::to_bytes(self : FstSet) -> Bytes

    FstSet::union

    fn FstSet::union(self : FstSet, other : FstSet) -> FstSet raise FstError

    IndexedValue

    pub(all) struct IndexedValue {
    index : Int
    value : UInt64
    } derive(Eq,
    Debug
    )

    The value and zero-based source index of a key in a merge result.

    IndexedValue::equal

    IndexedValue::not_equal

    fn IndexedValue::not_equal(x : IndexedValue, y : IndexedValue) -> Bool

    KeyRange

    pub(all) struct KeyRange {
    lower : Bound
    upper : Bound
    } derive(Eq,
    Debug
    )

    A half-open or closed byte range; each endpoint has its own inclusion rule.

    KeyRange::all

    fn KeyRange::all() -> KeyRange

    KeyRange::between

    fn KeyRange::between(lower : Bytes, upper : Bytes) -> KeyRange

    KeyRange::closed

    fn KeyRange::closed(lower : Bytes, upper : Bytes) -> KeyRange

    KeyRange::contains

    fn KeyRange::contains(self : KeyRange, key : Bytes) -> Bool

    KeyRange::equal

    fn KeyRange::equal(KeyRange, KeyRange) -> Bool

    KeyRange::not_equal

    fn KeyRange::not_equal(x : KeyRange, y : KeyRange) -> Bool

    KeyRange::prefix

    fn KeyRange::prefix(prefix : Bytes) -> KeyRange

    KeyRange::to_repr

    MapSource

    pub struct MapSource {
    // private fields
    }

    Transform only values; keys and ordering are preserved.

    MapSource::close

    fn MapSource::close(self : MapSource) -> Unit

    MapSource::next

    fn MapSource::next(self : MapSource) -> Entry? raise FstError

    MergeStream

    pub struct MergeStream {
    // private fields
    }

    Streaming merge, using O(number of sources) heads plus each stream's active path. Inputs are consumed and must not be shared during iteration.

    MergeStream::close

    fn MergeStream::close(self : MergeStream) -> Unit

    MergeStream::collect

    fn MergeStream::collect(self : MergeStream) -> Array[MergedEntry] raise FstError

    MergeStream::count

    fn MergeStream::count(self : MergeStream) -> UInt64 raise FstError

    MergeStream::into_map

    fn MergeStream::into_map(self : MergeStream, policy : DuplicatePolicy, registry_capacity? : Int) -> FstMap raise FstError

    Choose how to combine shared keys explicitly; no silent UInt64 overflow.

    MergeStream::into_set

    fn MergeStream::into_set(self : MergeStream, registry_capacity? : Int) -> FstSet raise FstError

    MergeStream::new

    fn MergeStream::new(sources : Array[Stream], operation : Operation) -> MergeStream raise FstError

    Each supplied stream must be independent; ownership is transferred by convention. Zero inputs produce an empty result for every operation.

    MergeStream::next

    fn MergeStream::next(self : MergeStream) -> MergedEntry? raise FstError

    MergeStream::reduce_with

    fn MergeStream::reduce_with(self : MergeStream, f : (MergedEntry) -> UInt64 raise?) -> FstMap raise

    Custom reductions can use source indices for priority or weighted merging.

    MergeStream::take

    fn MergeStream::take(self : MergeStream, count : Int) -> Array[MergedEntry] raise FstError

    MergedEntry

    pub(all) struct MergedEntry {
    key : Bytes
    values : Array[IndexedValue]
    } derive(Eq,
    Debug
    )

    Values are ordered by ascending source index.

    MergedEntry::equal

    fn MergedEntry::equal(MergedEntry, MergedEntry) -> Bool

    MergedEntry::not_equal

    fn MergedEntry::not_equal(x : MergedEntry, y : MergedEntry) -> Bool

    MergedEntry::reduce

    fn MergedEntry::reduce(self : MergedEntry, policy : DuplicatePolicy) -> Entry raise FstError

    Collapse grouped outputs using a checked arithmetic policy.

    Operation

    pub(all) enum Operation {
    Union
    Intersection
    Difference
    SymmetricDifference
    } derive(Eq,
    Debug
    )

    Multiway operations on sorted streams. Difference subtracts every stream after the first. SymmetricDifference keeps keys with odd source count.

    Operation::equal

    fn Operation::equal(Operation, Operation) -> Bool

    Operation::not_equal

    fn Operation::not_equal(x : Operation, y : Operation) -> Bool

    ReverseStream

    pub struct ReverseStream {
    // private fields
    }

    A descending byte-lexicographic traversal. It intentionally does not implement EntrySource, whose contract requires ascending keys.

    ReverseStream::close

    fn ReverseStream::close(self : ReverseStream) -> Unit

    ReverseStream::collect

    fn ReverseStream::collect(self : ReverseStream) -> Array[Entry] raise FstError

    ReverseStream::count

    fn ReverseStream::count(self : ReverseStream) -> UInt64 raise FstError

    ReverseStream::keys

    fn ReverseStream::keys(self : ReverseStream) -> Array[Bytes] raise FstError

    ReverseStream::next

    fn ReverseStream::next(self : ReverseStream) -> Entry? raise FstError

    ReverseStream::skip

    fn ReverseStream::skip(self : ReverseStream, count : Int) -> Int raise FstError

    ReverseStream::strings

    fn ReverseStream::strings(self : ReverseStream) -> Array[String] raise FstError

    ReverseStream::take

    fn ReverseStream::take(self : ReverseStream, count : Int) -> Array[Entry] raise FstError

    SearchIndex

    pub struct SearchIndex {
    // private fields
    }

    An optional index over the product of an FST and a byte automaton. It counts accepted suffixes without materializing matching keys. Its memory bound is explicit because a product graph can exceed the size of the FST.

    SearchIndex::cached_states

    fn SearchIndex::cached_states(self : SearchIndex) -> Int

    Actual stored product count entries. Always-matching suffixes are shortcut entries and do not require copies of their entire descendant subgraph.

    SearchIndex::is_empty

    fn SearchIndex::is_empty(self : SearchIndex) -> Bool

    SearchIndex::len

    fn SearchIndex::len(self : SearchIndex) -> UInt64

    Count of matching keys, with no output array or key expansion.

    SearchIndex::page

    fn SearchIndex::page(self : SearchIndex, offset : UInt64, limit : Int, range? : KeyRange) -> Array[Entry] raise FstError

    SearchIndex::page_reverse

    fn SearchIndex::page_reverse(self : SearchIndex, offset : UInt64, limit : Int, range? : KeyRange) -> Array[Entry] raise FstError

    Page matching keys in descending order without changing the reusable index.

    SearchIndex::prefix_count

    fn SearchIndex::prefix_count(self : SearchIndex, prefix : Bytes) -> UInt64

    SearchIndex::range_count

    fn SearchIndex::range_count(self : SearchIndex, range : KeyRange) -> UInt64

    SearchIndex::range_ranks

    fn SearchIndex::range_ranks(self : SearchIndex, range : KeyRange) -> (UInt64, UInt64)

    SearchIndex::rank

    fn SearchIndex::rank(self : SearchIndex, query : Bytes) -> (UInt64, Bool)

    Number of matching keys strictly less than query, and whether query itself is both stored in the FST and accepted by the automaton.

    SearchIndex::select

    fn SearchIndex::select(self : SearchIndex, ordinal : UInt64) -> Entry?

    Select the nth matching key in byte-lexicographic order.

    SearchIndex::stream

    fn SearchIndex::stream(self : SearchIndex, range? : KeyRange) -> Stream raise FstError

    Return a lazy traversal using the same predicate, without consuming this reusable index. A page or traversal never changes cached state counts.

    Statistics

    pub(all) struct Statistics {
    keys : UInt64
    bytes : Int
    nodes : Int
    transitions : Int
    final_nodes : Int
    max_fanout : Int
    min_key_length : Int
    max_key_length : Int
    } derive(Eq,
    Debug
    )

    Structural statistics, computed without enumerating accepted strings.

    Statistics::equal

    fn Statistics::equal(Statistics, Statistics) -> Bool

    Statistics::not_equal

    fn Statistics::not_equal(x : Statistics, y : Statistics) -> Bool

    Stream

    pub struct Stream {
    // private fields
    }

    Lazy lexicographic traversal. Only the active path and compressed-node cursors are held; transition arrays are never materialized for traversal.

    Stream::close

    fn Stream::close(self : Stream) -> Unit

    Stop early and release the active traversal stack.

    Stream::collect

    fn Stream::collect(self : Stream) -> Array[Entry] raise FstError

    Stream::count

    fn Stream::count(self : Stream) -> UInt64 raise FstError

    Stream::keys

    fn Stream::keys(self : Stream) -> Array[Bytes] raise FstError

    Stream::next

    fn Stream::next(self : Stream) -> Entry? raise FstError

    Stream::skip

    fn Stream::skip(self : Stream, count : Int) -> Int raise FstError

    Stream::string_entries

    fn Stream::string_entries(self : Stream) -> Array[(String, UInt64)] raise FstError

    Stream::strings

    fn Stream::strings(self : Stream) -> Array[String] raise FstError

    UTF-8 decoding is strict and preserves a leading BOM as a character.

    Stream::take

    fn Stream::take(self : Stream, count : Int) -> Array[Entry] raise FstError

    Consume at most count results without exhausting the stream.

    ValueStatistics

    pub(all) struct ValueStatistics {
    count : UInt64
    minimum : UInt64?
    maximum : UInt64?
    sum : UInt64?
    } derive(Eq,
    Debug
    )

    Aggregate map outputs. An empty group has count 0, no extrema, and sum 0. Sum is None only when the exact mathematical total exceeds UInt64::MAX; individual outputs, extrema and counts remain exact.

    ValueStatistics::equal

    ValueStatistics::not_equal

    fn ValueStatistics::not_equal(x : ValueStatistics, y : ValueStatistics) -> Bool

    WindowSource

    pub struct WindowSource {
    // private fields
    }

    Skip and bound results without consuming anything at construction.

    WindowSource::close

    fn WindowSource::close(self : WindowSource) -> Unit

    WindowSource::next

    fn WindowSource::next(self : WindowSource) -> Entry? raise FstError

    collect_source

    fn collect_source(source : &EntrySource) -> Array[Entry] raise FstError

    filter_source

    fn filter_source(source : &EntrySource, predicate : (Entry) -> Bool raise FstError) -> FilterSource

    fold_source

    fn[A] fold_source(source : &EntrySource, initial : A, step : (A, Entry) -> A raise?) -> A raise

    Fold without allocating a collection of all results.

    map_source

    fn map_source(source : &EntrySource, transform : (Entry) -> UInt64 raise FstError) -> MapSource

    merge

    fn merge(inputs : Array[Fst], operation : Operation, range? : KeyRange) -> MergeStream raise FstError

    Create independent streams, even if the same FST appears more than once.

    merge_sources

    fn merge_sources(sources : Array[&EntrySource], operation : Operation) -> MergeStream raise FstError

    Merge external readers as well as FST streams. Inputs are validated as consumed, so invalid ordering cannot silently corrupt the result.

    prefix_successor

    fn prefix_successor(prefix : Bytes) -> Bytes?

    Return the exclusive upper bound of all byte strings with this prefix.

    window_source

    fn window_source(source : &EntrySource, offset : UInt64, limit : UInt64) -> WindowSource