MoonCDC

    Durable SQLite change capture and resumable incremental projections for MoonBit.

    Download zip
    Version
    0.3.9
    License
    Apache-2.0
    Last updated
    50 minutes ago
    Downloads
    87

    #MoonCDC

    CI

    MoonBit Change Data Capture:SQLite 持久化变更捕获与可恢复增量投影。

    MoonCDC 为已有 SQLite 表安装事务内的行级触发器,记录带类型的增、删、改事件,提供一致性快照、消费者游标、JSON/JSONL 传输及目标库恢复。项目包含可复用 MoonBit 库和原生命令行工具。SQLite 引擎通过社区绑定复用,不计入本项目代码量。

    0.2.0(2026-11-06)修复 REAL 保真问题。 0.1.0 的 SQLite 十进制格式化会使部分数值(包括 1e300)发生舍入偏移,导致数值变化,或使快照后的正常更新触发 before-image 冲突。新版本以整数有效位和二进制指数精确传输 REAL,恢复前仍严格比较变更前镜像。已有 0.1.0 捕获与目标须按升级步骤重新初始化;升级不能恢复旧日志中已经丢失的精度。

    当前版本为 0.3.9(2026-11-09)。 更新至下述固定 MoonBit 工具链和依赖,并重新通过 Windows 全量验证,包括 ±1e300 快照后的普通更新、5,134 个有限 REAL 样本的独立位级比较、并发同步、进程中断恢复和大 BLOB 的 JSONL 往返。同步状态机、公开业务 API、捕获及传输格式保持兼容;不修复旧日志中已经丢失的精度。复验范围与边界见本次记录,历史修复见 CHANGELOG。

    升级要求: 完整的 0.2.0/0.3.0/0.3.1 捕获须在维护窗口运行 upgrade-capture;符合当前结构要求的 0.3.2–0.3.7 捕获沿用现有触发器。更新时替换全部同步及日志清理进程,按使用手册检查源结构和目标一致性。

    #适用场景

    • 业务库 → 报表库:首次快照初始化,随后仅应用变更。同步中断后根据目标库持久化游标恢复,并补齐遗漏的源端确认。
    • 本地数据 → 下游索引或缓存:导出包含 before/after 镜像的 JSONL,下游持久化处理结果后再确认。删除及主键变化显式可见;外部副作用仍需下游自行保证幂等。
    • 按条件建立数据子集:保留完整主键,选择和重命名列,按类型筛选行。进入筛选范围产生插入,离开产生删除;全部被过滤的批次仍推进游标。
    • 同步任务运行检查:查看消费者积压、预演应用、比较同一水位的数据、诊断结构/触发器变化,并按所有消费者的最低游标清理日志。

    #已实现能力

    组件能力
    类型模型NULL、Int64、有限 Double、文本、BLOB;REAL 使用整数有效位与二进制指数编码,BLOB 使用十六进制编码
    捕获普通表持久化 AFTER 触发器;事务回滚不留下事件;复合主键及主键更新
    初始化同一事务读取快照、水位并登记消费者
    增量消费源标识隔离、日志连续性检查、批次预算、确认时比较旧游标
    SQLite 目标数据与游标同事务提交;重放不重复应用;before-image 冲突时整批回滚
    筛选投影类型严格的过滤器、列选择/重命名、目标持久化规则、防止恢复时偷换规则
    传输JSON、带 header/commit 的 JSONL;增量批次解码、截断检测、分批/合批、相邻事件压缩
    运维暂停/恢复、消费者分叉和检查点登记、保留策略预览、分批清理、诊断、审计、安全卸载

    #工具链与依赖

    0.3.9 的验证基线为 MoonBit v0.10.20+65efd5b93(编译器构建日期 2026-11-05),对应 moon / moonrun 0.1.20261104 (940aa18)(构建日期 2026-11-04)。历史版本的发布包不受本次升级影响。

    • moonbit-community/sqlite3@0.2.4(该版本内置 SQLite 3.49.1)
    • moonbitlang/async@0.22.7、moonbitlang/x@0.5.7
    • 原生构建需要 C 编译器:Windows 使用 MSVC,Linux 使用 GCC/Clang。
    • 测试脚本使用 Python 3.12+;JS 测试需要 Node.js 22+。

    安装指定版本时设置官方安装器的 MOONBIT_INSTALL_VERSION=0.10.20+65efd5b93,参见 MoonBit 安装说明。纯类型/传输/筛选模块支持 native、JS 和 wasm-gc;SQLite 适配器与 CLI 仅支持 native。

    #快速运行

    moon update moon build --target native moon run cmd/main --target native -- help python examples/demo.py

    示例创建临时源库和目标库,通过独立 Python SQLite 连接写入,再调用 CLI 同步并审计,不修改已有数据库。

    实际数据库示例(source.db 必须已存在,report.db 必须尚不存在):

    moon run cmd/main --target native -- plan source.db orders customers moon run cmd/main --target native -- install source.db orders customers moon run cmd/main --target native -- bootstrap source.db report.db reporting moon run cmd/main --target native -- sync source.db report.db reporting moon run cmd/main --target native -- audit source.db report.db moon run cmd/main --target native -- status source.db moon run cmd/main --target native -- retention source.db moon run cmd/main --target native -- prune-bounded source.db --limit 1000

    每个写入源库的连接必须启用 recursive_triggers=ON 和 trusted_schema=ON。 前者防止 REPLACE 隐式删除漏记,MoonCDC 自身连接自动设置;后者是 SQLite 对触发器内表值 PRAGMA 的执行要求,在验证基线中默认开启。外部应用需确认这两项连接设置。trusted_schema=OFF 会使写入以 unsafe use of virtual table 失败;需要保持该安全策略的应用不适用当前捕获方式。

    完整操作和恢复见 使用手册,协议见 传输格式,一致性边界见 设计契约。

    #作为库使用

    包导入 tlhuecdkoyg/MoonCDC 与 tlhuecdkoyg/MoonCDC/sqlite。SQLite API 为同步、可抛错 API:

    let source = @sqlite.Database::open("source.db")
    defer source.close()
    let target = @sqlite.Database::open("report.db")
    defer target.close()
    source.install_capture(["orders"]) |> ignore
    target.initialize_projection(source.snapshot("reporting"))
    @sqlite.synchronize(source, target, "reporting") |> ignore

    安装与快照仅在初始化时执行;恢复直接调用 synchronize。完整 API 参见各包 pkg.generated.mbti 与 SQLite 测试。

    Database::query 对 INSERT / UPDATE / DELETE / REPLACE / WITH 查询提供语句级错误回滚,包括 RETURNING 行数超限;已有事务使用内部保存点。execute 保留 SQLite 原生语句行为,OR FAIL 可能保留成功前缀。显式 OR ROLLBACK 仍可终止调用者事务,PRAGMA 设置按 SQLite 规则处理。详见库接口事务边界。

    #正确性边界

    • 这是行级变更流与最终一致投影。一个源事务可能跨多个目标批次,不保留任意源事务的边界,不承诺跨系统 exactly-once。
    • 安装改变源库结构并增加写入开销。首次快照持有写锁;大规模初始数据需评估锁等待和内存。
    • 不支持虚拟表、隐藏/生成列、无显式主键的表、非 BINARY 主键排序规则或自动 DDL 迁移。结构变化需重新初始化。
    • 不支持涉及捕获表的外键 ON UPDATE CASCADE / SET NULL / SET DEFAULT 动作环,包括经过未捕获表的路径。
    • 不支持外键删除动作及后续更新返回同一捕获表并修改主键的路径。两类检查均按结构保守判定,当前行数据不改变结果;普通无环级联、纯删除 CASCADE 及无上述路径的非主键自引用 SET NULL / SET DEFAULT 可用。
    • 目标表不复制源库索引、外键、其他约束和业务触发器,应由同步任务专用;外部修改可能导致后续应用失败。
    • 同一目标的筛选规则不可变,修改需要新快照和新目标。独立 project-* 传输变换由调用方负责规则一致性。
    • 捕获源表不能包含其他业务触发器;同一 main schema 中后加的持久触发器会使验证和后续写入失败。连接私有的 TEMP 业务触发器无法由持久 main 触发器完整检查,调用方必须保证写入连接不安装此类触发器。非 UTF-8 数据库须先按业务流程迁移;非法 UTF-8 TEXT 会使快照、消费或审计明确失败,任意字节应使用 BLOB。
    • SQLite 捕获及快照统一将 REAL 的正负零规范化为 +0.0,包括源库实际保留负零符号的 BLOB/无类型列。非零有限 REAL 保留 binary64 位模式,独立核心 wire 编解码仍保留负零。审计按该规范化语义比较,不承诺保存源库零的符号位。
    • 快照和批次在预算内保存在内存;逐事件读取限制累积量,但单条 SQLite 值仍需解码。wire budget 使用 UTF-16 代码单元,CLI 输入预算使用文件字节。
    • 不抵御恶意元数据修改。检测结构漂移和日志缺口不等于恢复已经丢失的事件。
    • 快照文件输出、目标初始化与源端登记不构成跨资源事务。失败可能留下消费者登记或空目标文件,按手册处理;不自动删除用户文件。

    #验证与代码统计

    python scripts/verify.py python scripts/real_regression.py python scripts/count_loc.py

    验证包含格式/编译检查、三个后端的适用测试、原生构建,以及 CLI 独立进程与 Python SQLite 集成测试。覆盖外部写入、Int64/NUL 文本/BLOB/实数、持久化恢复、终止未提交写入、重放、筛选、截断输入、保留策略和目标分歧。scripts/concurrency_regression.py 另行运行 200 次外部提交、两个共享消费者同步任务与并发日志清理,核查最终全部数据、游标和日志连续性。

    scripts/framing_regression.py 验证接近默认事件预算的大 BLOB 经 JSONL 导出、导入、确认和重放后逐字节一致。scripts/crash_recovery_regression.py 在真实读锁阻塞提交期间终止同步进程,分别检查目标提交前回滚,以及目标已提交、源端尚未确认时的重启恢复;两端最终数据与游标均须一致。这些测试不能替代断电及存储设备故障验证。

    独立 REAL 回归脚本在原生构建后可单独运行:python scripts/real_regression.py。脚本使用 Python SQLite 写入和读取数据库,以 struct.pack 比较源、目标的全部 64 位及存储类型,不复用 MoonCDC 的解码器或相等判断。其 5,134 个有限值覆盖全部有限指数区间、随机有效位、±1e300、最大有限数、次正规数和零,并验证 JSON/JSONL 快照、持续更新、删除和重放。核心库测试另行检查负零 wire 往返及非法编码拒绝。

    统计只计生产 .mbt 非空、非注释行,排除测试、示例、生成接口、依赖、构建产物、帮助文本和 traits.mbt。正常代码的括号与声明行计入,SQLite 引擎及 Python 测试不计入 4,000 行要求。

    #原创与许可证

    捕获协议、游标恢复、筛选投影、运维接口及 CLI 为本项目实现,非现有 CDC 工具的移植。复用依赖及许可证见 NOTICE。本项目采用 Apache-2.0。

    CdcError

    pub suberror CdcError {
    Invalid(String)
    Conflict(String)
    Stale(String)
    Corrupt(String)
    } derive(
    Debug
    )

    CdcError::to_repr

    #deprecated("`CdcError::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn CdcError::to_repr(CdcError) ->
    Repr

    AuditReport

    pub(all) struct AuditReport {
    cursor : Cursor
    tables : Array[TableAudit]
    } derive(Eq,
    Debug
    )

    AuditReport::consistent

    fn AuditReport::consistent(self : AuditReport) -> Bool

    AuditReport::equal

    #deprecated("`AuditReport::equal` is deprecated, use `Eq::equal` instead.")
    fn AuditReport::equal(AuditReport, AuditReport) -> Bool

    AuditReport::not_equal

    #deprecated("`AuditReport::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn AuditReport::not_equal(x : AuditReport, y : AuditReport) -> Bool

    AuditReport::to_repr

    #deprecated("`AuditReport::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn AuditReport::to_repr(AuditReport) ->
    Repr

    AuditReport::to_wire

    Batch

    pub(all) struct Batch {
    start : Cursor
    end : Cursor
    high_water : Int64
    events : Array[Event]
    } derive(Eq,
    Debug
    )

    Batch::compact_adjacent

    fn Batch::compact_adjacent(self : Batch, schemas : Array[Table]) -> Batch raise CdcError

    Reduce consecutive changes of the same row while preserving observable final state. Events for other rows and primary-key changes act as barriers: no cross-row reordering. This is for projections, not audit trails, and does not preserve intermediate images.

    Batch::equal

    #deprecated("`Batch::equal` is deprecated, use `Eq::equal` instead.")
    fn Batch::equal(Batch, Batch) -> Bool

    Batch::from_jsonl

    fn Batch::from_jsonl(text : String) -> Batch raise CdcError

    Batch::from_wire

    fn Batch::from_wire(value :
    Json
    ) -> Batch raise CdcError

    Batch::not_equal

    #deprecated("`Batch::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn Batch::not_equal(x : Batch, y : Batch) -> Bool

    Batch::split

    fn Batch::split(self : Batch, max_events? : Int, max_wire_units? : Int) -> Array[Batch] raise CdcError

    Rechunk a transport without losing empty cursor intervals introduced by filtering. The byte-independent budget measures serialized UTF-16 code units, not UTF-8 bytes.

    Batch::summarize

    fn Batch::summarize(self : Batch, schemas : Array[Table]) ->
    Json
    raise CdcError

    Human-readable activity summary; counts row events rather than source transactions.

    Batch::to_jsonl

    fn Batch::to_jsonl(self : Batch) -> String raise CdcError

    Framed JSONL: header, event records and a required commit trailer.

    Batch::to_repr

    #deprecated("`Batch::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn Batch::to_repr(Batch) ->
    Repr

    Batch::to_wire

    Batch::validate

    fn Batch::validate(self : Batch) -> Unit raise CdcError

    BatchDecoder

    pub struct BatchDecoder {
    max_events : Int
    max_line_units : Int
    max_total_units : Int
    total : Int
    pending : StringBuilder
    pending_units : Int
    start : Cursor?
    end : Cursor?
    high_water : Int64
    events : Array[Event]
    failed : Bool
    finished : Bool
    }

    BatchDecoder::feed

    fn BatchDecoder::feed(self : BatchDecoder, chunk : String) -> Unit raise CdcError

    BatchDecoder::finish

    fn BatchDecoder::finish(self : BatchDecoder) -> Batch raise CdcError

    BatchDecoder::new

    fn BatchDecoder::new(max_events? : Int, max_line_units? : Int, max_total_units? : Int) -> BatchDecoder raise CdcError

    Column

    pub(all) struct Column {
    name : String
    declared_type : String
    required : Bool
    key_position : Int
    } derive(Eq,
    Debug
    )

    Column::equal

    #deprecated("`Column::equal` is deprecated, use `Eq::equal` instead.")
    fn Column::equal(Column, Column) -> Bool

    Column::not_equal

    #deprecated("`Column::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn Column::not_equal(x : Column, y : Column) -> Bool

    Column::to_repr

    #deprecated("`Column::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn Column::to_repr(Column) ->
    Repr

    Column::to_wire

    ColumnSelection

    pub(all) struct ColumnSelection {
    source : String
    target : String
    } derive(Eq,
    Debug
    )

    ColumnSelection::equal

    #deprecated("`ColumnSelection::equal` is deprecated, use `Eq::equal` instead.")
    fn ColumnSelection::equal(ColumnSelection, ColumnSelection) -> Bool

    ColumnSelection::not_equal

    #deprecated("`ColumnSelection::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn ColumnSelection::not_equal(x : ColumnSelection, y : ColumnSelection) -> Bool

    ColumnSelection::to_repr

    #deprecated("`ColumnSelection::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn ColumnSelection::to_repr(ColumnSelection) ->
    Repr

    Cursor

    pub(all) struct Cursor {
    source : String
    sequence : Int64
    } derive(Eq,
    Debug
    )

    Cursor::equal

    #deprecated("`Cursor::equal` is deprecated, use `Eq::equal` instead.")
    fn Cursor::equal(Cursor, Cursor) -> Bool

    Cursor::from_wire

    fn Cursor::from_wire(value :
    Json
    ) -> Cursor raise CdcError

    Cursor::not_equal

    #deprecated("`Cursor::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn Cursor::not_equal(x : Cursor, y : Cursor) -> Bool

    Cursor::to_repr

    #deprecated("`Cursor::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn Cursor::to_repr(Cursor) ->
    Repr

    Cursor::to_wire

    Cursor::validate

    fn Cursor::validate(self : Cursor) -> Unit raise CdcError

    Event

    pub(all) struct Event {
    sequence : Int64
    table : String
    operation : Operation
    before : Row?
    after : Row?
    } derive(Eq,
    Debug
    )

    Event::equal

    #deprecated("`Event::equal` is deprecated, use `Eq::equal` instead.")
    fn Event::equal(Event, Event) -> Bool

    Event::from_wire

    fn Event::from_wire(value :
    Json
    ) -> Event raise CdcError

    Event::not_equal

    #deprecated("`Event::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn Event::not_equal(x : Event, y : Event) -> Bool

    Event::to_repr

    #deprecated("`Event::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn Event::to_repr(Event) ->
    Repr

    Event::to_wire

    Event::validate

    fn Event::validate(self : Event) -> Unit raise CdcError

    Operation

    pub(all) enum Operation {
    Insert
    Update
    Delete
    } derive(Eq,
    Debug
    )

    Operation::equal

    #deprecated("`Operation::equal` is deprecated, use `Eq::equal` instead.")
    fn Operation::equal(Operation, Operation) -> Bool

    Operation::name

    fn Operation::name(self : Operation) -> String

    Operation::not_equal

    #deprecated("`Operation::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn Operation::not_equal(x : Operation, y : Operation) -> Bool

    Operation::parse

    fn Operation::parse(name : String) -> Operation raise CdcError

    Operation::to_repr

    #deprecated("`Operation::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn Operation::to_repr(Operation) ->
    Repr

    Predicate

    pub(all) enum Predicate {
    Always
    Equal(String, Value)
    IsNull(String)
    IntegerBetween(String, Int64, Int64)
    TextPrefix(String, String)
    All(Array[Predicate])
    Any(Array[Predicate])
    Not(Predicate)
    } derive(Eq,
    Debug
    )

    Typed filters deliberately avoid SQL coercion and floating-point integer casts.

    Predicate::equal

    #deprecated("`Predicate::equal` is deprecated, use `Eq::equal` instead.")
    fn Predicate::equal(Predicate, Predicate) -> Bool

    Predicate::from_wire

    Predicate::matches

    fn Predicate::matches(self : Predicate, table : Table, row : Row) -> Bool raise CdcError

    Predicate::not_equal

    #deprecated("`Predicate::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn Predicate::not_equal(x : Predicate, y : Predicate) -> Bool

    Predicate::to_repr

    #deprecated("`Predicate::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn Predicate::to_repr(Predicate) ->
    Repr

    Predicate::to_wire

    Predicate::validate

    fn Predicate::validate(self : Predicate, table : Table) -> Unit raise CdcError

    ProjectionPlan

    pub(all) struct ProjectionPlan {
    rules : Array[TableRule]
    } derive(Eq,
    Debug
    )

    ProjectionPlan::equal

    #deprecated("`ProjectionPlan::equal` is deprecated, use `Eq::equal` instead.")
    fn ProjectionPlan::equal(ProjectionPlan, ProjectionPlan) -> Bool

    ProjectionPlan::from_wire

    ProjectionPlan::identity

    fn ProjectionPlan::identity(tables : Array[Table]) -> ProjectionPlan raise CdcError

    Create an editable identity plan from a captured schema manifest.

    ProjectionPlan::not_equal

    #deprecated("`ProjectionPlan::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn ProjectionPlan::not_equal(x : ProjectionPlan, y : ProjectionPlan) -> Bool

    ProjectionPlan::project_batch

    fn ProjectionPlan::project_batch(self : ProjectionPlan, batch : Batch) -> Batch raise CdcError

    Cursor intervals survive filtering, including batches containing no output rows.

    ProjectionPlan::project_snapshot

    fn ProjectionPlan::project_snapshot(self : ProjectionPlan, snapshot : Snapshot) -> Snapshot raise CdcError

    ProjectionPlan::to_repr

    #deprecated("`ProjectionPlan::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn ProjectionPlan::to_repr(ProjectionPlan) ->
    Repr

    ProjectionPlan::to_wire

    ProjectionPlan::validate

    fn ProjectionPlan::validate(self : ProjectionPlan) -> Unit raise CdcError

    Row

    pub(all) struct Row {
    values : Array[Value]
    } derive(Eq,
    Debug
    )

    Row::equal

    #deprecated("`Row::equal` is deprecated, use `Eq::equal` instead.")
    fn Row::equal(Row, Row) -> Bool

    Row::from_wire

    fn Row::from_wire(value :
    Json
    ) -> Row raise CdcError

    Row::key

    fn Row::key(self : Row, table : Table) -> String raise CdcError

    Row::not_equal

    #deprecated("`Row::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn Row::not_equal(x : Row, y : Row) -> Bool

    Row::to_repr

    #deprecated("`Row::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn Row::to_repr(Row) ->
    Repr

    Row::to_wire

    fn Row::to_wire(self : Row) ->
    Json
    raise CdcError

    Row::validate

    fn Row::validate(self : Row, table : Table) -> Unit raise CdcError

    RowDifference

    pub(all) struct RowDifference {
    table : String
    key : String
    kind : String
    columns : Array[String]
    } derive(Eq,
    Debug
    )

    RowDifference::equal

    #deprecated("`RowDifference::equal` is deprecated, use `Eq::equal` instead.")
    fn RowDifference::equal(RowDifference, RowDifference) -> Bool

    RowDifference::not_equal

    #deprecated("`RowDifference::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn RowDifference::not_equal(x : RowDifference, y : RowDifference) -> Bool

    RowDifference::to_repr

    #deprecated("`RowDifference::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn RowDifference::to_repr(RowDifference) ->
    Repr

    RowDifference::to_wire

    Snapshot

    pub(all) struct Snapshot {
    cursor : Cursor
    tables : Array[TableSnapshot]
    } derive(Eq,
    Debug
    )

    Snapshot::equal

    #deprecated("`Snapshot::equal` is deprecated, use `Eq::equal` instead.")
    fn Snapshot::equal(Snapshot, Snapshot) -> Bool

    Snapshot::from_jsonl

    fn Snapshot::from_jsonl(text : String, max_rows? : Int, max_units? : Int) -> Snapshot raise CdcError

    Snapshot::from_wire

    Snapshot::not_equal

    #deprecated("`Snapshot::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn Snapshot::not_equal(x : Snapshot, y : Snapshot) -> Bool

    Snapshot::row_count

    fn Snapshot::row_count(self : Snapshot) -> Int

    Snapshot::schemas

    fn Snapshot::schemas(self : Snapshot) -> Array[Table]

    Snapshot::to_jsonl

    fn Snapshot::to_jsonl(self : Snapshot) -> String raise CdcError

    Snapshot::to_repr

    #deprecated("`Snapshot::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn Snapshot::to_repr(Snapshot) ->
    Repr

    Snapshot::to_wire

    Snapshot::validate

    fn Snapshot::validate(self : Snapshot) -> Unit raise CdcError

    Statement

    pub(all) struct Statement {
    sql : String
    parameters : Array[Value]
    } derive(Eq,
    Debug
    )

    Parameterized SQL plus exact SQLite values; identifiers are separately quoted.

    Statement::equal

    #deprecated("`Statement::equal` is deprecated, use `Eq::equal` instead.")
    fn Statement::equal(Statement, Statement) -> Bool

    Statement::not_equal

    #deprecated("`Statement::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn Statement::not_equal(x : Statement, y : Statement) -> Bool

    Statement::to_repr

    #deprecated("`Statement::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn Statement::to_repr(Statement) ->
    Repr

    Table

    pub(all) struct Table {
    name : String
    columns : Array[Column]
    } derive(Eq,
    Debug
    )

    Table::capture_select_sql

    fn Table::capture_select_sql(self : Table) -> String raise CdcError

    Read source values as tagged SQL JSON, validating TEXT bytes before any connection binding converts them to UTF-16. Requires a UTF-8 database.

    Table::column_index

    fn Table::column_index(self : Table, name : String) -> Int?

    Table::create_projection_sql

    fn Table::create_projection_sql(self : Table, target : String) -> String raise CdcError

    Table::delete_statement

    fn Table::delete_statement(self : Table, row : Row, target? : String) -> Statement raise CdcError

    Table::equal

    #deprecated("`Table::equal` is deprecated, use `Eq::equal` instead.")
    fn Table::equal(Table, Table) -> Bool

    Table::from_wire

    fn Table::from_wire(value :
    Json
    ) -> Table raise CdcError

    Table::insert_statement

    fn Table::insert_statement(self : Table, row : Row, target? : String) -> Statement raise CdcError

    Table::key_indexes

    fn Table::key_indexes(self : Table) -> Array[Int]

    Table::key_predicate

    fn Table::key_predicate(self : Table, row : Row) -> Statement raise CdcError

    Table::lookup_statement

    fn Table::lookup_statement(self : Table, row : Row, target? : String) -> Statement raise CdcError

    Table::not_equal

    #deprecated("`Table::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn Table::not_equal(x : Table, y : Table) -> Bool

    Table::row_expression

    fn Table::row_expression(self : Table, prefix : String, legacy_v020? : Bool) -> String raise CdcError

    Table::select_sql

    fn Table::select_sql(self : Table) -> String raise CdcError

    Table::signature

    fn Table::signature(self : Table) -> String

    Table::to_repr

    #deprecated("`Table::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn Table::to_repr(Table) ->
    Repr

    Table::to_wire

    Table::trigger_sql

    fn Table::trigger_sql(self : Table, table_id : Int, operation : Operation, legacy_v020? : Bool, legacy_v031? : Bool) -> String raise CdcError

    Generate the current capture definition. Legacy flags only reconstruct known definitions for audited upgrades; do not use them to install new capture.

    Table::update_statement

    fn Table::update_statement(self : Table, before : Row, after : Row, target? : String) -> Statement raise CdcError

    Table::validate

    fn Table::validate(self : Table) -> Unit raise CdcError

    TableAudit

    pub(all) struct TableAudit {
    table : String
    expected_rows : Int
    actual_rows : Int
    missing : Int
    unexpected : Int
    changed : Int
    samples : Array[RowDifference]
    } derive(Eq,
    Debug
    )

    TableAudit::consistent

    fn TableAudit::consistent(self : TableAudit) -> Bool

    TableAudit::equal

    #deprecated("`TableAudit::equal` is deprecated, use `Eq::equal` instead.")
    fn TableAudit::equal(TableAudit, TableAudit) -> Bool

    TableAudit::not_equal

    #deprecated("`TableAudit::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn TableAudit::not_equal(x : TableAudit, y : TableAudit) -> Bool

    TableAudit::to_repr

    #deprecated("`TableAudit::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn TableAudit::to_repr(TableAudit) ->
    Repr

    TableAudit::to_wire

    TableRule

    pub(all) struct TableRule {
    source : Table
    target : String
    columns : Array[ColumnSelection]
    filter : Predicate
    } derive(Eq,
    Debug
    )

    TableRule::equal

    #deprecated("`TableRule::equal` is deprecated, use `Eq::equal` instead.")
    fn TableRule::equal(TableRule, TableRule) -> Bool

    TableRule::not_equal

    #deprecated("`TableRule::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn TableRule::not_equal(x : TableRule, y : TableRule) -> Bool

    TableRule::output_schema

    fn TableRule::output_schema(self : TableRule) -> Table raise CdcError

    TableRule::project_event

    fn TableRule::project_event(self : TableRule, event : Event) -> Event? raise CdcError

    TableRule::project_row

    fn TableRule::project_row(self : TableRule, row : Row) -> Row? raise CdcError

    TableRule::to_repr

    #deprecated("`TableRule::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn TableRule::to_repr(TableRule) ->
    Repr

    TableSnapshot

    pub(all) struct TableSnapshot {
    schema : Table
    rows : Array[Row]
    } derive(Eq,
    Debug
    )

    TableSnapshot::equal

    #deprecated("`TableSnapshot::equal` is deprecated, use `Eq::equal` instead.")
    fn TableSnapshot::equal(TableSnapshot, TableSnapshot) -> Bool

    TableSnapshot::not_equal

    #deprecated("`TableSnapshot::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn TableSnapshot::not_equal(x : TableSnapshot, y : TableSnapshot) -> Bool

    TableSnapshot::to_repr

    #deprecated("`TableSnapshot::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn TableSnapshot::to_repr(TableSnapshot) ->
    Repr

    Value

    pub(all) enum Value {
    Null
    Integer(Int64)
    Real(Double)
    Text(String)
    Blob(Bytes)
    } derive(Eq,
    Debug
    )

    SQLite storage classes. Integers are never converted through floating point.

    Value::equal

    #deprecated("`Value::equal` is deprecated, use `Eq::equal` instead.")
    fn Value::equal(Value, Value) -> Bool

    Value::from_wire

    fn Value::from_wire(value :
    Json
    ) -> Value raise CdcError

    Value::not_equal

    #deprecated("`Value::not_equal` is deprecated, use `Eq::not_equal` instead.")
    fn Value::not_equal(x : Value, y : Value) -> Bool

    Value::storage_class

    fn Value::storage_class(self : Value) -> String

    Value::to_repr

    #deprecated("`Value::to_repr` is deprecated, use `@moonbitlang/core/debug.Debug::to_repr` instead.")
    fn Value::to_repr(Value) ->
    Repr

    Value::to_wire

    Value::validate

    fn Value::validate(self : Value) -> Unit raise CdcError

    array_field

    fn array_field(value :
    Json
    , name : String) -> Array[
    Json
    ] raise CdcError

    boolean_field

    fn boolean_field(value :
    Json
    , name : String) -> Bool raise CdcError

    check_name

    fn check_name(name : String) -> Unit raise CdcError

    compare_schemas

    fn compare_schemas(expected : Array[Table], actual : Array[Table]) ->
    Json
    raise CdcError

    Explain schema incompatibilities without attempting an unsafe automatic migration.

    compare_snapshots

    fn compare_snapshots(expected : Snapshot, actual : Snapshot, sample_limit? : Int) -> AuditReport raise CdcError

    compare_table

    fn compare_table(expected : TableSnapshot, actual : TableSnapshot, sample_limit? : Int) -> TableAudit raise CdcError

    decode_hex

    fn decode_hex(text : String) -> Bytes raise CdcError

    encode_hex

    fn encode_hex(bytes : Bytes) -> String

    find_table

    fn find_table(tables : Array[Table], name : String) -> Table raise CdcError

    integer_field

    fn integer_field(value :
    Json
    , name : String) -> Int64 raise CdcError

    merge_batches

    fn merge_batches(batches : Array[Batch], max_events? : Int) -> Batch raise CdcError

    Join adjacent envelopes. Gaps, overlaps and unrelated source identities fail closed.

    parse_integer

    fn parse_integer(text : String) -> Int64 raise CdcError

    parse_json

    fn parse_json(text : String) ->
    Json
    raise CdcError

    quote_identifier

    fn quote_identifier(name : String) -> String raise CdcError

    quote_literal

    fn quote_literal(text : String) -> String raise CdcError

    schemas_from_wire

    fn schemas_from_wire(text : String) -> Array[Table] raise CdcError

    schemas_to_wire

    fn schemas_to_wire(tables : Array[Table]) -> String

    source_schema_sql

    fn source_schema_sql() -> Array[String]

    string_field

    fn string_field(value :
    Json
    , name : String) -> String raise CdcError

    target_schema_sql

    fn target_schema_sql() -> Array[String]

    trigger_name

    fn trigger_name(table_id : Int, operation : Operation) -> String raise CdcError

    validate_cursor

    fn validate_cursor(cursor : Cursor, source : Cursor, floor : Int64) -> Unit raise CdcError