🥮 mooncakes.io
    Sign in

    capnp

    Cap'n Proto serialization runtime and schema code generation for MoonBit.

    capnproto
    serialization
    codegen
    binary
    Download zip
    Author
    Version
    0.1.0
    License
    MIT
    Repository
    Last updated
    11 hours ago
    Downloads
    2

    Dependencies

    #Cap'n Proto for MoonBit

    Checks

    用同一份 .capnp 协议定义，在 MoonBit 与 C++、Python 等语言之间交换二进制数据。本项目提供 MoonBit 序列化运行库和 capnp 编译器插件，从正式的 schema 生成带类型的 MoonBit reader / builder。

    适合已有 Cap'n Proto 协议的服务、数据采集程序、离线处理工具和二进制文件消费者接入 MoonBit。协议定义、字段布局和已有数据格式可以继续使用，业务代码无需手写偏移或经 JSON 中转。这里实现的是序列化层；网络传输和 RPC 不在本版本范围内。

    #可运行的完整流程

    examples/telemetry 演示一个数据处理任务：

    1. MoonBit 用生成的 API 写入任务 ID、纳秒时间戳、读数和质量标记。
    2. Python 使用 pycapnp（独立的 C++ Cap'n Proto 实现）读取数据，剔除缺失值和 NaN，计算统计结果。
    3. MoonBit 读取 Python 写出的结果，核对数量、均值、极值和超过 JavaScript 安全整数范围的 64 位 ID。

    这是仓库提供的互操作示例，不代表已有外部用户部署。它说明该库如何用于实际的跨语言批处理；相同接口也可用于文件交换和消息队列的负载编解码。

    #安装与快速运行

    需要 MoonBit 0.10.14 或更新版本。CI 固定验证 0.10.14+7d59c7ec9。运行库支持 wasm、wasm-gc、js、native，无需 C++ Cap'n Proto 动态库。

    在自己的 MoonBit 项目中添加依赖：

    moon add Ridge-Lab/capnp@0.1.0

    包的 moon.pkg 中导入：

    import {
      "Ridge-Lab/capnp",
    }

    直接运行本仓库的例子：

    git clone https://github.com/Ridge-Lab/capnp-mbt
cd capnp-mbt
moon update
moon run --target js examples/roundtrip

    输出包含 Read job 9007199254740993 和 6 个样本。生成代码已随例子提交，因此运行例子无需先安装 capnp。Node.js 需支持 ESM；本项目在 Node.js 24 上验证。

    #从 schema 生成 MoonBit API

    先安装官方 Cap'n Proto 编译器及 native 后端所需 C 编译器。在本仓库根目录执行：

    moon build --target native --release cmd/capnpc-moonbit
capnp compile -o./_build/native/release/build/cmd/capnpc-moonbit/capnpc-moonbit.exe:examples/telemetry/generated examples/telemetry/schema/telemetry.capnp
moon fmt
moon check --target native --deny-warn

    插件接收官方 CodeGeneratorRequest，由官方编译器处理 schema 语法和字段布局。所有所需声明（含 import）生成到同一个 capnp_generated.mbt，同时在目标目录缺少 moon.pkg 时创建运行库导入。多份相关 schema 应在一次 capnp compile 调用中传入；每次生成会替换目标文件。

    也可使用请求文件和 Node.js，不必编译 native 插件：

    capnp compile -o- fixtures/conformance.capnp > request.bin
moon run --target js cmd/capnpc-moonbit -- --request request.bin --out generated
moon fmt

    二进制重定向使用 Bash 或 cmd.exe；旧版 PowerShell 的文本管道可能损坏二进制内容。Windows native 构建说明见开发文档。

    使用生成 API 的代码片段（完整可运行代码见 sample.mbt）：

    let message = @capnp.Builder::new()
    let batch = @telemetry.BatchBuilder::init_root(message)
    batch.set_job_id(9007199254740993UL)
    batch.set_source("room-temperature")
    let samples = batch.init_samples(1)
    samples.get(0).set_value(21.5)
    samples.get(0).set_quality(@telemetry.Valid)
    let bytes = message.finish()
    let received = @telemetry.BatchReader::from_message(@capnp.read_message(bytes))
    assert_eq(received.get_samples().get(0).get_value(), 21.5)

    生成包在使用方 moon.pkg 中以 @telemetry 导入。reader 返回字段视图，实际访问指针时验证目标范围；Text 转换会进行 UTF-8 验证。低层接口见 pkg.generated.mbti。

    #功能与边界

    能力当前范围
    数据类型Void、Bool、所有整数和浮点宽度、Text、Data、enum、struct、嵌套声明
    容器和字段primitive / pointer / inline-composite list、嵌套 list、group、union、标量及复合默认值
    帧和指针标准分段帧、近指针、单跳和双跳 far pointer 的读取；writer 生成单段帧
    压缩与流标准 packed 编解码；frame_length 识别部分输入和串联帧边界
    防护指针越界检查、共享遍历预算、嵌套深度限制、段数限制、解包和 writer 分配上限
    演进缺失字段默认值、额外字段跳过、未知 enum 值保留、兼容的 list / struct 扩展
    未实现RPC、capability、AnyPointer、泛型、常量声明、canonical encoding、builder 原位编辑收到的消息

    不支持的 schema 和命名冲突会返回诊断。运行库错误使用 CapnpError，生成器错误使用 CodegenError。

    默认 reader 遍历预算为 8,388,608 words，嵌套深度 64，最多 512 段；unpack 默认最多输出 64 MiB。reader 的预算会被重复访问消耗，按需缓存已取得的视图。read_message 要求恰好一帧，流式输入先用 frame_length 划分。builder 使用追加式 arena，替换指针不会回收旧对象，finish() 返回快照。详见兼容性和资源限制。

    #验证

    moon fmt --check
moon check --target all --deny-warn
moon test --target all --deny-warn
moon build --target all --release

    测试包含官方编译器生成的固定样例、packed 字节对照、多段与 double-far 指针、默认值、union/group、生成 API、截断与恶意输入。CI 在四个后端执行检查、构建和测试，并额外执行官方编译器插件和 MoonBit → Python/C++ → MoonBit 流程。测试数据来源与再生方法可独立复核。本项目尚未进行生产负载验证，不宣称优于其他实现的性能。

    #来源与许可证

    MIT。运行库按公开编码规范实现；生成器接入官方插件协议。不是 C++ 库的 FFI 包装，也未使用 MoonBit SWE-AGI 中的 Cap'n Proto 解码题解。独立贡献包括 writer、packing、类型代码生成及跨语言工作流；生态中已有相关题目并不等于已有相同 SDK。

    第三方依赖、测试数据及生成文件的来源见 docs/PROVENANCE.md。

    CapnpError

    source
    pub(all) suberror CapnpError {
      InvalidMessage(String)
      Bounds(String)
      Limit(String)
      TypeMismatch(String)
      Unsupported(String)
    } derive(
    Debug    )

    Builder

    source
    pub struct Builder {
      // private fields
    }

    A bounded arena producing one standard Cap'n Proto stream segment. Handles share the arena; scalar writes do not copy the message.

    Builder::allocated_words

    source
    fn Builder::allocated_words(self : Builder) -> Int

    Builder::finish

    source
    fn Builder::finish(self : Builder) -> Bytes

    Return a snapshot with the standard single-segment stream framing.

    Builder::init_root

    source
    fn Builder::init_root(self : Builder, data_words : Int, pointer_count : Int) -> StructBuilder raise CapnpError

    Allocate a new root. Reinitializing replaces the pointer; old arena space remains allocated and is still charged to the allocation limit.

    Builder::new

    source
    fn Builder::new(max_words? : Int) -> Builder raise CapnpError

    The default allocation limit is 64 MiB, including the root pointer.

    ListBuilder

    source
    pub struct ListBuilder {
      // private fields
    }

    ListBuilder::clear_pointer

    source
    fn ListBuilder::clear_pointer(self : ListBuilder, index : Int) -> Unit raise CapnpError

    ListBuilder::element_size

    source
    fn ListBuilder::element_size(self : ListBuilder) -> Int

    ListBuilder::init_list

    source
    fn ListBuilder::init_list(self : ListBuilder, index : Int, element_size : Int, length : Int) -> ListBuilder raise CapnpError

    ListBuilder::init_struct

    source
    fn ListBuilder::init_struct(self : ListBuilder, index : Int, data_words : Int, pointer_count : Int) -> StructBuilder raise CapnpError

    ListBuilder::init_struct_list

    source
    fn ListBuilder::init_struct_list(self : ListBuilder, index : Int, length : Int, data_words : Int, pointer_count : Int) -> ListBuilder raise CapnpError

    ListBuilder::length

    source
    fn ListBuilder::length(self : ListBuilder) -> Int

    ListBuilder::set_bool

    source
    fn ListBuilder::set_bool(self : ListBuilder, index : Int, value : Bool) -> Unit raise CapnpError

    ListBuilder::set_data

    source
    fn ListBuilder::set_data(self : ListBuilder, index : Int, value : Bytes) -> Unit raise CapnpError

    ListBuilder::set_float32

    source
    fn ListBuilder::set_float32(self : ListBuilder, index : Int, value : Float) -> Unit raise CapnpError

    ListBuilder::set_float64

    source
    fn ListBuilder::set_float64(self : ListBuilder, index : Int, value : Double) -> Unit raise CapnpError

    ListBuilder::set_int16

    source
    fn ListBuilder::set_int16(self : ListBuilder, index : Int, value : Int) -> Unit raise CapnpError

    ListBuilder::set_int32

    source
    fn ListBuilder::set_int32(self : ListBuilder, index : Int, value : Int) -> Unit raise CapnpError

    ListBuilder::set_int64

    source
    fn ListBuilder::set_int64(self : ListBuilder, index : Int, value : Int64) -> Unit raise CapnpError

    ListBuilder::set_int8

    source
    fn ListBuilder::set_int8(self : ListBuilder, index : Int, value : Int) -> Unit raise CapnpError

    ListBuilder::set_text

    source
    fn ListBuilder::set_text(self : ListBuilder, index : Int, value : String) -> Unit raise CapnpError

    ListBuilder::set_uint16

    source
    fn ListBuilder::set_uint16(self : ListBuilder, index : Int, value : UInt) -> Unit raise CapnpError

    ListBuilder::set_uint32

    source
    fn ListBuilder::set_uint32(self : ListBuilder, index : Int, value : UInt) -> Unit raise CapnpError

    ListBuilder::set_uint64

    source
    fn ListBuilder::set_uint64(self : ListBuilder, index : Int, value : UInt64) -> Unit raise CapnpError

    ListBuilder::set_uint8

    source
    fn ListBuilder::set_uint8(self : ListBuilder, index : Int, value : Byte) -> Unit raise CapnpError

    ListBuilder::set_void

    source
    fn ListBuilder::set_void(self : ListBuilder, index : Int) -> Unit raise CapnpError

    ListBuilder::struct_at

    source
    fn ListBuilder::struct_at(self : ListBuilder, index : Int) -> StructBuilder raise CapnpError

    Return the existing element of an inline composite list without allocation.

    ListReader

    source
    pub struct ListReader {
      // private fields
    }

    A lazy list view, including lists of inline composite structs.

    ListReader::element_size

    source
    fn ListReader::element_size(self : ListReader) -> Int

    Return the wire element-size code, 0 through 7.

    ListReader::get_bool

    source
    fn ListReader::get_bool(self : ListReader, index : Int) -> Bool raise CapnpError

    ListReader::get_data

    source
    fn ListReader::get_data(self : ListReader, index : Int) -> Bytes raise CapnpError

    ListReader::get_float32

    source
    fn ListReader::get_float32(self : ListReader, index : Int) -> Float raise CapnpError

    ListReader::get_float64

    source
    fn ListReader::get_float64(self : ListReader, index : Int) -> Double raise CapnpError

    ListReader::get_int16

    source
    fn ListReader::get_int16(self : ListReader, index : Int) -> Int raise CapnpError

    ListReader::get_int32

    source
    fn ListReader::get_int32(self : ListReader, index : Int) -> Int raise CapnpError

    ListReader::get_int64

    source
    fn ListReader::get_int64(self : ListReader, index : Int) -> Int64 raise CapnpError

    ListReader::get_int8

    source
    fn ListReader::get_int8(self : ListReader, index : Int) -> Int raise CapnpError

    ListReader::get_list

    source
    fn ListReader::get_list(self : ListReader, index : Int) -> ListReader raise CapnpError

    ListReader::get_pointer

    source
    fn ListReader::get_pointer(self : ListReader, index : Int) -> PointerReader raise CapnpError

    ListReader::get_struct

    source
    fn ListReader::get_struct(self : ListReader, index : Int) -> StructReader raise CapnpError

    Read an inline struct, or view a primitive/pointer element as an evolved struct. Bit lists cannot be upgraded to struct lists under the Cap'n Proto specification.

    ListReader::get_text

    source
    fn ListReader::get_text(self : ListReader, index : Int) -> String raise CapnpError

    ListReader::get_uint16

    source
    fn ListReader::get_uint16(self : ListReader, index : Int) -> UInt raise CapnpError

    ListReader::get_uint32

    source
    fn ListReader::get_uint32(self : ListReader, index : Int) -> UInt raise CapnpError

    ListReader::get_uint64

    source
    fn ListReader::get_uint64(self : ListReader, index : Int) -> UInt64 raise CapnpError

    ListReader::get_uint8

    source
    fn ListReader::get_uint8(self : ListReader, index : Int) -> Byte raise CapnpError

    ListReader::length

    source
    fn ListReader::length(self : ListReader) -> Int

    Message

    source
    pub struct Message {
      // private fields
    }

    An immutable, framed Cap'n Proto message. Readers share its traversal budget.

    Message::remaining_traversal_words

    source
    fn Message::remaining_traversal_words(self : Message) -> Int

    Message::root

    source
    fn Message::root(self : Message) -> StructReader raise CapnpError

    Message::root_pointer

    source
    fn Message::root_pointer(self : Message) -> PointerReader

    Message::segment_count

    source
    fn Message::segment_count(self : Message) -> Int

    PointerReader

    source
    pub struct PointerReader {
      // private fields
    }

    A lazy pointer. Its target is validated when it is dereferenced.

    PointerReader::as_data

    source
    fn PointerReader::as_data(self : PointerReader, default? : Bytes) -> Bytes raise CapnpError

    PointerReader::as_list

    source
    fn PointerReader::as_list(self : PointerReader) -> ListReader raise CapnpError

    PointerReader::as_struct

    source
    fn PointerReader::as_struct(self : PointerReader) -> StructReader raise CapnpError

    PointerReader::as_text

    source
    fn PointerReader::as_text(self : PointerReader, default? : String) -> String raise CapnpError

    PointerReader::is_null

    source
    fn PointerReader::is_null(self : PointerReader) -> Bool

    StructBuilder

    source
    pub struct StructBuilder {
      // private fields
    }

    StructBuilder::clear_data

    source
    fn StructBuilder::clear_data(self : StructBuilder, byte_offset : Int, byte_length : Int) -> Unit raise CapnpError

    StructBuilder::clear_pointer

    source
    fn StructBuilder::clear_pointer(self : StructBuilder, slot : Int) -> Unit raise CapnpError

    StructBuilder::init_list

    source
    fn StructBuilder::init_list(self : StructBuilder, slot : Int, element_size : Int, length : Int) -> ListBuilder raise CapnpError

    Element size is the standard wire code: 0 void, 1 bit, 2 byte, 3 uint16, 4 uint32, 5 uint64, 6 pointer. Struct lists use init_struct_list.

    StructBuilder::init_struct

    source
    fn StructBuilder::init_struct(self : StructBuilder, slot : Int, data_words : Int, pointer_count : Int) -> StructBuilder raise CapnpError

    StructBuilder::init_struct_list

    source
    fn StructBuilder::init_struct_list(self : StructBuilder, slot : Int, length : Int, data_words : Int, pointer_count : Int) -> ListBuilder raise CapnpError

    StructBuilder::set_bool

    source
    fn StructBuilder::set_bool(self : StructBuilder, bit_offset : Int, value : Bool, default? : Bool) -> Unit raise CapnpError

    Set a bit in the data section, XOR'd with its schema default.

    StructBuilder::set_data

    source
    fn StructBuilder::set_data(self : StructBuilder, slot : Int, value : Bytes) -> Unit raise CapnpError

    StructBuilder::set_float32

    source
    fn StructBuilder::set_float32(self : StructBuilder, byte_offset : Int, value : Float, default? : Float) -> Unit raise CapnpError

    StructBuilder::set_float64

    source
    fn StructBuilder::set_float64(self : StructBuilder, byte_offset : Int, value : Double, default? : Double) -> Unit raise CapnpError

    StructBuilder::set_int16

    source
    fn StructBuilder::set_int16(self : StructBuilder, byte_offset : Int, value : Int, default? : Int) -> Unit raise CapnpError

    StructBuilder::set_int32

    source
    fn StructBuilder::set_int32(self : StructBuilder, byte_offset : Int, value : Int, default? : Int) -> Unit raise CapnpError

    StructBuilder::set_int64

    source
    fn StructBuilder::set_int64(self : StructBuilder, byte_offset : Int, value : Int64, default? : Int64) -> Unit raise CapnpError

    StructBuilder::set_int8

    source
    fn StructBuilder::set_int8(self : StructBuilder, byte_offset : Int, value : Int, default? : Int) -> Unit raise CapnpError

    StructBuilder::set_text

    source
    fn StructBuilder::set_text(self : StructBuilder, slot : Int, value : String) -> Unit raise CapnpError

    StructBuilder::set_uint16

    source
    fn StructBuilder::set_uint16(self : StructBuilder, byte_offset : Int, value : UInt, default? : UInt) -> Unit raise CapnpError

    StructBuilder::set_uint32

    source
    fn StructBuilder::set_uint32(self : StructBuilder, byte_offset : Int, value : UInt, default? : UInt) -> Unit raise CapnpError

    StructBuilder::set_uint64

    source
    fn StructBuilder::set_uint64(self : StructBuilder, byte_offset : Int, value : UInt64, default? : UInt64) -> Unit raise CapnpError

    StructBuilder::set_uint8

    source
    fn StructBuilder::set_uint8(self : StructBuilder, byte_offset : Int, value : Byte, default? : Byte) -> Unit raise CapnpError

    StructReader

    source
    pub struct StructReader {
      // private fields
    }

    A view of a struct's data and pointer sections.

    StructReader::data_size_bytes

    source
    fn StructReader::data_size_bytes(self : StructReader) -> Int

    StructReader::get_bool

    source
    fn StructReader::get_bool(self : StructReader, bit_offset : Int, default? : Bool) -> Bool raise CapnpError

    StructReader::get_data

    source
    fn StructReader::get_data(self : StructReader, index : Int, default? : Bytes) -> Bytes raise CapnpError

    StructReader::get_float32

    source
    fn StructReader::get_float32(self : StructReader, byte_offset : Int, default? : Float) -> Float raise CapnpError

    StructReader::get_float64

    source
    fn StructReader::get_float64(self : StructReader, byte_offset : Int, default? : Double) -> Double raise CapnpError

    StructReader::get_int16

    source
    fn StructReader::get_int16(self : StructReader, byte_offset : Int, default? : Int) -> Int raise CapnpError

    StructReader::get_int32

    source
    fn StructReader::get_int32(self : StructReader, byte_offset : Int, default? : Int) -> Int raise CapnpError

    StructReader::get_int64

    source
    fn StructReader::get_int64(self : StructReader, byte_offset : Int, default? : Int64) -> Int64 raise CapnpError

    StructReader::get_int8

    source
    fn StructReader::get_int8(self : StructReader, byte_offset : Int, default? : Int) -> Int raise CapnpError

    StructReader::get_list

    source
    fn StructReader::get_list(self : StructReader, index : Int) -> ListReader raise CapnpError

    StructReader::get_pointer

    source
    fn StructReader::get_pointer(self : StructReader, index : Int) -> PointerReader raise CapnpError

    StructReader::get_struct

    source
    fn StructReader::get_struct(self : StructReader, index : Int) -> StructReader raise CapnpError

    StructReader::get_text

    source
    fn StructReader::get_text(self : StructReader, index : Int, default? : String) -> String raise CapnpError

    StructReader::get_uint16

    source
    fn StructReader::get_uint16(self : StructReader, byte_offset : Int, default? : UInt) -> UInt raise CapnpError

    StructReader::get_uint32

    source
    fn StructReader::get_uint32(self : StructReader, byte_offset : Int, default? : UInt) -> UInt raise CapnpError

    StructReader::get_uint64

    source
    fn StructReader::get_uint64(self : StructReader, byte_offset : Int, default? : UInt64) -> UInt64 raise CapnpError

    StructReader::get_uint8

    source
    fn StructReader::get_uint8(self : StructReader, byte_offset : Int, default? : Byte) -> Byte raise CapnpError

    StructReader::pointer_count

    source
    fn StructReader::pointer_count(self : StructReader) -> Int

    frame_length

    source
    fn frame_length(input : Bytes, offset? : Int, max_segments? : Int, max_message_bytes? : Int) -> Int? raise CapnpError

    Length of one standard (unpacked) Cap'n Proto frame at offset. Returns None when more input is needed. Limits are checked from the header before waiting for, or copying, the declared payload.

    pack

    source
    fn pack(input : Bytes) -> Bytes raise CapnpError

    Apply Cap'n Proto packing to a word-aligned byte sequence, including its standard stream header when packing a complete message.

    read_message

    source
    fn read_message(bytes : Bytes, traversal_limit_words? : Int, nesting_limit? : Int, max_segments? : Int) -> Message raise CapnpError

    Read exactly one message using standard (unpacked) stream framing. Trailing bytes are rejected; use frame_length to select a frame from a stream.

    unpack

    source
    fn unpack(input : Bytes, max_output_bytes? : Int) -> Bytes raise CapnpError

    Decode Cap'n Proto packing. Reject truncated runs and stop before an allocation would exceed max_output_bytes (64 MiB by default).

    Source Files

    Powered by MoonBit

    Site sourceReport issuePackagesBuild queueSkillsStatistics

    © 2026 mooncakes.io