Sign in

    moonmcap

    MoonBit-native MCAP message-log codec, indexed queries and file transformations

    mcap
    robotics
    binary
    logging
    index
    Download zip
    Version
    0.15.32
    License
    MIT
    Last updated
    6 hours ago
    Downloads
    147

    #MoonMCAP

    新增数据质量与故障复现工作流:可配置质量规则、持续事件定位、带上下文的片段导出、有界触发录制,以及显式锚点时钟校正。附可直接运行的配置,文件互操作通过独立 Python MCAP reader/writer 验证;时钟校正不声称真实设备精度。

    0.15.32 的 验证记录:49 个工作流、Wasm 185 项 / JS 207 项测试通过,0.10.14 与 0.10.17 双工具链验证通过。

    统一组合查询入口与复现说明见 异步查询管线。

    0.15.31 的 历史验证记录:48 个工作流、Wasm 173 项 / JS 195 项测试通过;0.10.14 与 0.10.17 已验证,本地当前工具链文档生成通过。

    新增能力:MFLD2 分页字段索引、按需解码与大数组、ROS 1 载荷、跨话题时间关联、持久录制会话。

    0.15.30 的 历史验证记录包含 47 个独立工作流和当时的测试、源码统计。

    可选工作线程在 Node.js 中并行解压和解码,保留排序、预算和背压;附耗时及进程内存实测。

    贯通恢复任务把 Chunk 副本、最终排序段和归并组进度保存在同一任务中,支持进程中断后复用。

    字段范围索引通过块级数值/字符串摘要保守排除不命中块,并保留最后的精确筛选与完整结果。

    异步分页索引可组合本地/远程 MIDX2、持久 HTTP 缓存和消息解码,完整审核后按页查询。

    HTTP 远程日志支持有界持久页缓存、并发预取、ETag 快照校验、校验失败重取与中断恢复,见HTTP 缓存。

    消息结构解码:内置 JSON、Protobuf FileDescriptorSet 和 ROS 2 ros2msg / XCDR1 解码,可接入同步、异步筛选、投影与精确聚合,支持自定义插件。见消息解码和 CDR 范围与样例。

    0.15.24 本地验证报告记录 41 组回归、11 个语料案例、4719 条公开 CDR 消息的独立解码对照,以及 200 条消息的并行/串行对照,区分真实录制、转换数据与合成样例。

    MoonBit 原生 MCAP 0 容器库,提供标准记录编解码、三种压缩模式、磁盘游标、异步录制、HTTP 范围查询、外部排序合并和诊断恢复。当前模块版本 0.15.32,见 更新记录 和 迁移说明。

    #安装与运行

    块级保存点为可独立校验的索引块保存持久副本,输入未完成时也能保留录入进度。

    外部聚合在分组缓冲超限时溢写临时磁盘,精确归并并逐组输出 NDJSON。

    分页磁盘索引使用外部排序和有限页缓存,提供百万条实际构建与查询报告。

    公开数据验证分别记录真实机械臂日志与复杂合成分布,提供来源、固定指纹和独立参考核对。

    大文件诊断按记录完整校验,提供首错范围、读取统计和实际恢复消息时间证据。

    统一异步查询组合本地/HTTP、侧车、JSON 条件、跨文件分页和 NDJSON 导出。

    时间回放提供倍速、暂停、单步、跳转,以及有背压的异步消息回调。

    日志差异比较区分消息、声明、附件和元数据变化,忽略压缩及索引布局差异。

    流式聚合支持 JSON 数值统计、字段分组和时间窗口,累计十进制值保持精度。

    JSON 内容查询支持精确数字条件、字段投影、跨文件分页及流式转换筛选。

    可恢复合并任务支持逐输入保存点、持久归并检查点、已完成排序段复用,以及任务查看和清理。

    文件 I/O提供直接字节桥接、可关闭文件句柄、有界预读及物理读取统计。

    需要 moonc >= 0.10.14,已验证 v0.10.14+7d59c7ec9 和 v0.10.17+d971670cd;文件操作示例需要 Node.js 22.3 及以上。工具链安装见 MoonBit 官方下载页。

    git clone https://github.com/LilaLola67/moonmcap.git cd moonmcap moon update moon check moon build moon test moon run cmd/main

    JS 目标:moon test --target js、moon run cmd/main --target js。内存示例录制 6 条消息和一个附件,再读取、统计、查询和裁剪,最后输出 MCAP_HEX= 数据用于互操作验证。

    可运行文件样例提供无压缩/LZ4/Zstd 的实际输入文件、裁剪/合并/恢复的预期输出及一键验证命令:python examples/run.py。

    安装 Mooncakes 发行版本:

    moon add LilaLola67/moonmcap@0.15.32

    已发布版本的独立安装验证:python tools/install_smoke.py --version 0.15.32。脚本在全新使用者目录中安装发行包,编译使用者自己的 worker 入口,运行查询、分页字段索引、二进制视图、ROS1、时间关联、录制恢复,以及质量、事件、上下文、触发录制和时钟映射共 11 类公开接口样例;产物保存在 _interop/installed/。

    也可从 GitHub 克隆复现全部样例;发布包校验 可生成经过文件清单检查的模块归档。LZ4/Zstd、xxHash 与字节缓冲使用发行包内的独立内部组件,保留上游 Apache-2.0 许可证及来源,详见 第三方说明;异步包使用固定版本的 moonbitlang/async 0.20.2。运行时不需要 Python,也不调用外部压缩进程。

    #API 示例

    安装依赖后,使用方在 moon.pkg 中导入 "LilaLola67/moonmcap" @mcap:

    let recorder = @mcap.new_builder()
    let channel = recorder.register_channel("/events", "json")
    recorder.append(channel, 100UL, b"{\"value\":1}")
    let bytes = recorder.finish(compression=@mcap.Zstd, messages_per_chunk=256)
    let log = @mcap.read_log(bytes)
    let query = { ..@mcap.all_query(), topics: ["/events"], start_time: 100UL, end_time: 200UL }
    let messages = @mcap.query_file(bytes, query).messages
    let sliced = @mcap.slice_log(log, query)
    let output = @mcap.write_indexed(sliced, compression=@mcap.Lz4)

    调用会抛出 McapError,应在声明该错误的函数或 try/catch 内使用。时间是 UInt64 纳秒,查询区间两端包含;升序按 (时间,原始物理位置) 排序,倒序反转整个次序,支持偏移和数量分页。Uncompressed、Lz4、Zstd 是三种写入模式。

    连续读取时,可以复用同一个索引与游标:

    let cursor = @mcap.build_query_index(log).cursor(query)
    let first_page = cursor.next_page(32)
    let second_page = cursor.next_page(32)
    cursor.seek(150UL) // 包含时间边界;不重新应用 offset
    let progress = cursor.progress()
    cursor.cancel()

    游标最多保留每个匹配通道的一个归并头部,不为每页重新构造完整候选数组。offset、limit 作用于整个遍历;seek 重置本次进度和 limit,取消后不能重开。索引仍持有完整内存快照。完整约定和示例见 消息游标,运行 moon run cmd/cursor 或 moon run cmd/cursor --target js。

    #大文件与异步工作流

    功能行为与边界
    磁盘游标逐页按需加载、处理时间重叠块、seek/取消;候选负载超预算显式失败
    流式变换临时文件支持大文件排序、采样、分页及对齐;NDJSON 逐条导出
    跨文件数据集冲突 ID 重映射、全局分页与跳转、来源编号;不生成合并文件
    侧车索引为无索引文件建立持久定位表,SHA-256 过期检查、显式快速身份模式、跨文件接入
    异步读写与轮转实际异步宿主 I/O、等待输出接纳、任务取消;每个完成段独立可读
    HTTP 查询Range + 强 ETag + If-Match,超时、有限重试、活动请求中止;拒绝整文件回退
    外部排序合并乱序输入写临时排序段,多轮归并、稳定重复次序、声明重映射、临时空间预算及清理
    独立块恢复显式跳过坏块、校验后续索引块并输出范围清单;始终标为部分提取

    异步文件和 HTTP 宿主支持 JS/Node;同步核心、磁盘游标协议和排序算法支持 Wasm。原路径的解压/排序仍在当前线程运行,可选工作线程后端分担解压和负载解码;异步 I/O 本身不意味着多线程压缩。负载预算分别计账,不是总进程内存上限,见 内存实测。

    #容器与压缩

    覆盖 MCAP 0 的全部 15 类标准记录,包括 Statistics、消息/块索引、附件与元数据索引和 SummaryOffset。读取时检查记录顺序、声明依赖、摘要副本、统计内容、物理偏移以及数据区、摘要、块和附件 CRC。可读取顺序、分块、有/无摘要、有/无摘要偏移、无消息索引的合法文件,并在重写时重新生成统计及索引。

    • LZ4:标准帧读写;读取独立/依赖块、块校验及内容校验。
    • Zstd:标准单帧读写;读取 raw/RLE/compressed 块、未知内容大小及可选校验;写端采用固定编码策略,不提供压缩等级调节。
    • 每个 Chunk 独立判断压缩方式,允许同一文件混用三种模式。
    • 默认记录大小上限 16 MiB、单块解压上限 64 MiB、字符串 1 MiB、记录数量 100,000,可通过 Limits 配置。新增的 max_decompressed_bytes 限制单块展开,嵌套记录也计入文件记录预算。

    兼容范围与可复现案例 说明了具体边界。容器读写保留原始消息字节;可选解码适配器支持 JSON / Protobuf 和明确范围的 ROS 2 XCDR1 结构查询,完整 ROS profile 语义需要业务方处理。

    #日志操作

    • 主题/时间裁剪,清理无引用声明并保留附件、元数据。
    • 按内容去重 schema/channel,合并时安全重映射 ID 并保留重复消息。
    • 平移消息和附件时间,检查 UInt64 溢出/下溢。
    • 时序统计:消息数、数据量、时间间隔、乱序与序号异常;与标准 Statistics 记录分别提供 API。
    • 查询游标:逐条/分批取消息、稳定分页、双向时间跳转、取消和进度;同一索引可共享给多个独立游标。
    • 录制器提供声明去重、自动序号、快照、添加失败原子性及结束状态。
    • 分片记录流解码与连续前缀恢复。压缩块完全解码并通过校验后才接纳消息,遇到损坏或截断就停止,不穿过坏字节重同步。

    #文件示例与验证

    下面的 JS 命令将文件转为 Zstd 索引文件:

    moon run cmd/interop --target js -- input.mcap output.mcap zstd rewrite

    还支持 slice(筛选 /imu 的 20–30 秒闭区间)、merge(在参数末尾追加第二个文件路径)和 recover,用于复现完整工作流。该命令的文件访问使用 Node.js;核心编解码、压缩及变换在 Wasm 和 JS 上均可运行。

    python -m pip install mcap==1.3.0 lz4==4.4.5 zstandard==0.25.0 python tools/interop.py python tools/conformance.py python tools/count_source.py

    • MoonBit 单元/回归测试:Wasm 185 项、JS 207 项;JS 增加 Node 适配器和真实异步暂停、取消、背压、轮转状态测试。
    • 真实文件验证:30 项;包含 3 输入压缩 × 2 摘要布局 × 3 输出压缩 = 18 组双向读写检查,以及顺序文件、无消息索引、无摘要偏移、混合压缩和查询/合并/恢复工作流。
    • 真实文件验证还覆盖按需读取、三种流式输出、8 项 CLI、4 项批量变换、8 项诊断修复,以及 5 项异步录制、15 项 HTTP 故障、5 项磁盘排序、9 项独立块恢复、108 项固定种子差分检查和 4 组进程 RSS 测量。执行命令见下方,结果位于 _interop/ 各子目录的 report.json。GitHub Actions 完整执行并上传报告。
    • 新增 45 项大文件诊断与恢复故障用例,百万条消息与实际超过 2 GiB 文件的本地验证见 规模报告。常规 CI 使用缩小规模,完整规模测试可手动启用。

    python tools/range_benchmark.py python tools/stream_workflows.py python tools/cli_workflows.py python tools/batch_workflows.py python tools/repair_workflows.py python tools/async_workflows.py python tools/http_workflows.py python tools/external_workflows.py python tools/block_workflows.py python tools/transform_stream_workflows.py python tools/dataset_workflows.py python tools/sidecar_workflows.py python tools/json_workflows.py python tools/checkpoint_workflows.py python tools/input_checkpoint_workflows.py python tools/io_workflows.py python tools/aggregate_workflows.py python tools/diff_workflows.py python tools/replay_workflows.py python tools/async_dataset_workflows.py python tools/source_diagnostic_workflows.py python tools/large_source_workflows.py python tools/fuzz_workflows.py python tools/memory_benchmark.py python tools/corpus_workflows.py python tools/disk_index_workflows.py python tools/disk_index_scale.py python tools/aggregate_external_workflows.py python tools/chunk_checkpoint_workflows.py python tools/protobuf_workflows.py python tools/cached_http_workflows.py python tools/protojson_workflows.py python tools/cdr_workflows.py python tools/async_paged_workflows.py python tools/field_index_workflows.py python tools/chunk_pipeline_workflows.py python tools/worker_workflows.py python tools/worker_benchmark.py python tools/package_audit.py

    #文件工具

    通用命令行 提供 inspect、query、slice、merge、external-merge、recover、recover-blocks、rewrite、transform、validate、reindex。主题、UInt64 纳秒时间、分页、压缩和预算均可配置;查询支持 JSON/CSV,输出文件拒绝覆盖已有路径。

    moon run cmd/mcap --target js -- query examples/files/zstd-summary.mcap --topic /imu --start 20000000000 --end 30000000000 moon run cmd/mcap --target js -- validate examples/files/zstd-summary.mcap moon run cmd/mcap --target js -- reindex broken-summary.mcap --output rebuilt.mcap --compression zstd

    批量变换 提供主题映射、按通道时间窗采样、时间对齐、附件和元数据筛选。结构化诊断与修复 区分数据区和索引/摘要问题,已知时给出偏移、记录类型与预期/实际值;只从完整有效的数据区重建索引,数据损坏可显式选择连续前缀恢复或索引块部分提取。

    Python mcap==1.3.0 仅为开发阶段的官方参考工具,其包版本不是文件格式版本。生成测试不计入有效源码;tools/count_source.py 只统计根目录手写核心的非空、非注释 .mbt 行,排除测试、生成接口、示例及 internal/zstd 引用代码。

    #边界

    不支持外部压缩字典、拼接帧、skippable frame 或自定义压缩算法。消息体可显式执行 JSON 条件和投影,Protobuf 和受支持的 ros2msg/XCDR1 可先按日志内定义解码;其他编码保留原始字节。原有文件 API 接收已加载的 Bytes;RangeReader 和 Node.js 文件适配器支持 按需读取,快速模式不校验未读取的数据。StreamWriter 支持同步输出、增量校验和与分块/摘要预算;旧 LogBuilder 仍提供完整快照和原子添加。

    scan_file 可取得未知原始记录;Log 是语义模型,重写会重建容器,不保留未知记录和扩展字段的原始字节。未提供 CRC 的内容无法仅凭解析证明未损坏。互操作结果限于已执行的矩阵与声明边界,不等于所有未来扩展格式都已验证。

    #原创与参考

    MCAP 容器实现独立编写,依据 官方规范,参考生态为 foxglove/mcap(MIT)。容器核心采用 MIT,引用的 Zstd 实现及压缩依赖遵循其 Apache-2.0 许可证,详见 third_party。公开项目检索记录 说明了检索范围与相邻项目差异。

    McapError

    pub suberror McapError {
    Invalid(String)
    } derive(
    Debug
    )

    AggregateOptions

    pub(all) struct AggregateOptions {
    value_path : String
    group_paths : Array[String]
    by_topic : Bool
    window : UInt64
    origin : UInt64
    max_groups : Int
    group_bytes : Int
    invalid : JsonInvalidPolicy
    }

    AggregateRow

    type AggregateRow

    AggregateRun

    type AggregateRun

    AggregateSpill

    type AggregateSpill

    AggregateState

    type AggregateState

    Attachment

    pub(all) struct Attachment {
    log_time : UInt64
    create_time : UInt64
    name : String
    media_type : String
    data : Bytes
    } derive(Eq,
    Debug
    )

    Attachment::equal

    fn Attachment::equal(Attachment, Attachment) -> Bool

    Attachment::not_equal

    fn Attachment::not_equal(x : Attachment, y : Attachment) -> Bool

    AttachmentIndex

    pub(all) struct AttachmentIndex {
    offset : UInt64
    length : UInt64
    log_time : UInt64
    create_time : UInt64
    data_size : UInt64
    name : String
    media_type : String
    }

    BinaryField

    pub(all) struct BinaryField {
    path : String
    offset : Int
    byte_length : Int
    elements : Int
    element_type : String
    stride : Int
    little_endian : Bool
    aligned : Bool
    } derive(Eq,
    Debug
    )

    BlockRecoveryEvidence

    pub(all) struct BlockRecoveryEvidence {
    start : UInt64
    end : UInt64
    recovered : Bool
    messages : Int
    actual_start : UInt64?
    actual_end : UInt64?
    advertised_start : UInt64
    advertised_end : UInt64
    reason : String?
    }

    BlockRecoveryReport

    pub(all) struct BlockRecoveryReport {
    recovered : Array[RecoveredBlock]
    skipped : Array[SkippedBlock]
    messages : Int
    output_bytes : UInt64
    source_complete : Bool
    }

    CachedChunk

    type CachedChunk

    CdrDecoder

    pub struct CdrDecoder {
    name : String
    definitions : Map[String, Array[CdrField]]
    options : CdrOptions
    field_count : Int
    ros1 : Bool
    }

    CdrDecoder::decode

    fn CdrDecoder::decode(self : CdrDecoder, bytes : Bytes) -> Json raise McapError

    CdrDecoder::view

    fn CdrDecoder::view(self : CdrDecoder, bytes : Bytes, selection? : DecodeSelection) -> PayloadView raise McapError

    CdrField

    type CdrField

    CdrOptions

    pub(all) struct CdrOptions {
    schema_bytes : Int
    message_bytes : Int
    string_bytes : Int
    items : Int
    fields : Int
    depth : Int
    int64_as_string : Bool
    }

    CdrProjection

    type CdrProjection

    CdrWire

    type CdrWire

    Channel

    pub(all) struct Channel {
    id : Int
    schema_id : Int
    topic : String
    message_encoding : String
    metadata : Array[(String, String)]
    } derive(Eq,
    Debug
    )

    Channel::equal

    fn Channel::equal(Channel, Channel) -> Bool

    Channel::not_equal

    fn Channel::not_equal(x : Channel, y : Channel) -> Bool

    Channel::to_repr

    Chunk

    pub(all) struct Chunk {
    start_time : UInt64
    end_time : UInt64
    records : Array[Record]
    }

    ChunkIndex

    pub(all) struct ChunkIndex {
    start_time : UInt64
    end_time : UInt64
    chunk_start : UInt64
    chunk_length : UInt64
    message_index_offsets : Array[(Int, UInt64)]
    message_index_length : UInt64
    compression : String
    compressed_size : UInt64
    uncompressed_size : UInt64
    }

    ChunkJobState

    type ChunkJobState

    ChunkPart

    type ChunkPart

    ChunkReadRequest

    pub(all) struct ChunkReadRequest {
    span : ReadSpan
    chunk_bytes : Int
    expanded_bytes : Int
    compression : String
    }

    ChunkSegment

    type ChunkSegment

    ClockAnchor

    pub(all) struct ClockAnchor {
    source : UInt64
    target : UInt64
    }

    ClockMap

    pub struct ClockMap {
    anchors : Array[ClockAnchor]
    extrapolate : Bool
    }

    ClockMap::json

    fn ClockMap::json(self : ClockMap) -> Json

    ClockMap::map

    fn ClockMap::map(self : ClockMap, time : UInt64) -> UInt64 raise McapError

    ClockMap::residuals

    fn ClockMap::residuals(self : ClockMap, probes : Array[ClockAnchor]) -> Json raise McapError

    Independent held-out probes; anchor interpolation residual alone is not evidence of calibration accuracy. Signed residual is predicted minus target.

    CommandOptions

    pub(all) struct CommandOptions {
    command : String
    inputs : Array[String]
    output : String
    query : Query
    compression : Compression
    format : String
    max_file_bytes : Int
    cache_bytes : Int
    chunk_messages : Int
    full : Bool
    transform : TransformOptions
    external : ExternalOptions
    temp_dir : String
    indexes : Array[String]
    fast_index : Bool
    index_bytes : Int
    content : JsonFilter?
    job_dir : String
    resume_job : Bool
    aggregate : AggregateOptions
    diff : DiffOptions
    }

    Compression

    pub(all) enum Compression {
    Uncompressed
    Lz4
    Zstd
    } derive(Eq,
    Debug
    )

    Compression::equal

    fn Compression::equal(Compression, Compression) -> Bool

    Compression::name

    fn Compression::name(self : Compression) -> String

    Compression::not_equal

    fn Compression::not_equal(x : Compression, y : Compression) -> Bool

    ContextClip

    pub(all) struct ContextClip {
    log : Log
    report : Json
    }

    ContextRule

    pub(all) struct ContextRule {
    topic : String
    depth : Int
    max_age : UInt64
    required : Bool
    }

    Cursor

    type Cursor

    CursorRun

    type CursorRun

    DataInspection

    type DataInspection

    Dataset

    pub struct Dataset {
    sources : Array[QuerySource]
    declarations : Log
    remaps : Array[Map[Int, Int]]
    limits : Limits
    }

    Dataset::channels

    fn Dataset::channels(self : Dataset) -> Array[Channel]

    Dataset::cursor

    fn Dataset::cursor(self : Dataset, query : Query, pending_bytes? : Int) -> DatasetCursor raise McapError

    Dataset::remap_message

    fn Dataset::remap_message(self : Dataset, source_index : Int, message : Message) -> Message raise McapError

    Dataset::schemas

    fn Dataset::schemas(self : Dataset) -> Array[Schema]

    DatasetCursor

    pub struct DatasetCursor {
    dataset : Dataset
    original : Query
    budget : Int
    streams : Array[MessageStream]
    heap : Array[DatasetMessage]
    query : Query
    state : FileCursorState
    initialized : Bool
    loaded : Int
    advance : Int?
    remaining : Int
    offset_left : Int
    emitted : Int
    skipped : Int
    peak : Int
    }

    DatasetCursor::cancel

    fn DatasetCursor::cancel(self : DatasetCursor) -> Unit

    DatasetCursor::next

    fn DatasetCursor::next(self : DatasetCursor) -> Message? raise McapError

    DatasetCursor::next_page

    fn DatasetCursor::next_page(self : DatasetCursor, size : Int) -> Array[DatasetMessage] raise McapError

    DatasetCursor::next_record

    fn DatasetCursor::next_record(self : DatasetCursor) -> DatasetMessage? raise McapError

    DatasetCursor::progress

    DatasetCursor::seek

    fn DatasetCursor::seek(self : DatasetCursor, time : UInt64) -> Unit raise McapError

    DatasetCursor::write_ndjson

    fn DatasetCursor::write_ndjson(self : DatasetCursor, sink : WriteSink) -> Int raise McapError

    DatasetMessage

    pub(all) struct DatasetMessage {
    source_index : Int
    message : Message
    } derive(Eq,
    Debug
    )

    DatasetProgress

    pub(all) struct DatasetProgress {
    state : FileCursorState
    sources : Int
    materialized_sources : Int
    loaded_sources : Int
    emitted_messages : Int
    skipped_messages : Int
    pending_bytes : Int
    peak_pending_bytes : Int
    } derive(Eq,
    Debug
    )

    DecodeSelection

    pub(all) struct DecodeSelection {
    paths : Array[String]
    inline_array_items : Int
    scan_items : Int
    } derive(Eq,
    Debug
    )

    Diagnostic

    pub(all) struct Diagnostic {
    code : DiagnosticCode
    offset : UInt64?
    opcode : Byte?
    message : String
    expected : UInt64?
    actual : UInt64?
    } derive(Eq,
    Debug
    )

    DiagnosticCode

    pub(all) enum DiagnosticCode {
    BadEnvelope
    BadRecord
    BadDeclaration
    BadCompression
    BadChecksum
    BadIndex
    BadSummary
    TruncatedInput
    LimitExceeded
    } derive(Eq,
    Debug
    )

    DiagnosticCode::name

    fn DiagnosticCode::name(self : DiagnosticCode) -> String

    DiffBag

    type DiffBag

    DiffOptions

    pub(all) struct DiffOptions {
    json_payloads : Bool
    bucket_bytes : Int
    auxiliary_bytes : Int
    max_details : Int
    detail_bytes : Int
    fallback_bytes : Int
    limits : Limits
    }

    DiffResult

    type DiffResult

    DiskIndexEntry

    type DiskIndexEntry

    DiskSidecarAudit

    pub struct DiskSidecarAudit {
    reader : DiskSidecarReader
    page : Int
    previous : DiskIndexEntry?
    failed : Bool
    }

    DiskSidecarAudit::feed

    fn DiskSidecarAudit::feed(self : DiskSidecarAudit, bytes : Bytes) -> Unit raise McapError

    DiskSidecarAudit::finish

    DiskSidecarAudit::poll

    DiskSidecarCursor

    pub struct DiskSidecarCursor {
    reader : DiskSidecarReader
    original : Query
    allowed : Map[Int, Bool]
    query : Query
    position : Int
    offset : Int
    remaining : Int
    state : FileCursorState
    }

    DiskSidecarCursor::cancel

    fn DiskSidecarCursor::cancel(self : DiskSidecarCursor) -> Unit

    DiskSidecarCursor::next

    DiskSidecarCursor::next_page

    fn DiskSidecarCursor::next_page(self : DiskSidecarCursor, size : Int) -> Array[Message] raise McapError

    DiskSidecarCursor::seek

    fn DiskSidecarCursor::seek(self : DiskSidecarCursor, time : UInt64) -> Unit raise McapError

    DiskSidecarPoll

    pub(all) enum DiskSidecarPoll {
    DiskDone
    DiskMessage(Message)
    DiskIndexRead(ReadSpan)
    DiskSourceRead(ReadSpan)
    }

    DiskSidecarPollCursor

    pub struct DiskSidecarPollCursor {
    reader : DiskSidecarReader
    original : Query
    allowed : Map[Int, Bool]
    query : Query
    low : Int
    high : Int
    searching : Bool
    position : Int
    offset : Int
    remaining : Int
    state : FileCursorState
    current : (Int, Array[DiskIndexEntry])?
    waiting_page : Int?
    waiting_entry : SidecarEntry?
    ready : Message?
    }

    DiskSidecarPollCursor::cancel

    fn DiskSidecarPollCursor::cancel(self : DiskSidecarPollCursor) -> Unit

    DiskSidecarPollCursor::feed_index

    fn DiskSidecarPollCursor::feed_index(self : DiskSidecarPollCursor, bytes : Bytes) -> Unit raise McapError

    DiskSidecarPollCursor::feed_source

    fn DiskSidecarPollCursor::feed_source(self : DiskSidecarPollCursor, bytes : Bytes, expanded? : Bytes) -> Unit raise McapError

    DiskSidecarPollCursor::pending_bytes

    fn DiskSidecarPollCursor::pending_bytes(self : DiskSidecarPollCursor) -> Int

    DiskSidecarPollCursor::poll

    DiskSidecarPollCursor::seek

    fn DiskSidecarPollCursor::seek(self : DiskSidecarPollCursor, time : UInt64) -> Unit raise McapError

    DiskSidecarPollCursor::skip_source

    fn DiskSidecarPollCursor::skip_source(self : DiskSidecarPollCursor) -> Unit raise McapError

    DiskSidecarReader

    pub struct DiskSidecarReader {
    index : ReadSource
    base : SidecarReader
    count : Int
    width : Int
    start : UInt64
    cache_limit : Int
    pages : Array[(Int, Array[DiskIndexEntry])]
    page_reads : Int
    cache_hits : Int
    peak_pages : Int
    }

    DiskSidecarReader::channels

    DiskSidecarReader::cursor

    DiskSidecarReader::declaration_log

    fn DiskSidecarReader::declaration_log(self : DiskSidecarReader) -> Log

    DiskSidecarReader::info

    DiskSidecarReader::poll_cursor

    fn DiskSidecarReader::poll_cursor(self : DiskSidecarReader, query : Query) -> DiskSidecarPollCursor raise McapError

    DiskSidecarReader::query_source

    fn DiskSidecarReader::query_source(self : DiskSidecarReader) -> QuerySource raise McapError

    DiskSidecarReader::validate_entries

    fn DiskSidecarReader::validate_entries(self : DiskSidecarReader) -> Unit raise McapError

    EventDetector

    pub struct EventDetector {
    rule : EventRule
    enter : JsonFilter
    leave : JsonFilter?
    previous : UInt64?
    candidate : UInt64?
    confirmed : UInt64?
    ended : UInt64?
    finished : Bool
    }

    EventDetector::feed

    fn EventDetector::feed(self : EventDetector, message : Message) -> Array[EventSignal] raise McapError

    Feed only this rule's topic, in ascending log-time order. Continuity is sampled evidence; a long gap closes an active event as incomplete.

    EventDetector::finish

    fn EventDetector::finish(self : EventDetector) -> EventSpan?

    EventRule

    pub(all) struct EventRule {
    topic : String
    enter : String
    leave : String?
    duration : UInt64
    max_gap : UInt64
    cooldown : UInt64
    pre : UInt64
    post : UInt64
    }

    EventSignal

    pub(all) enum EventSignal {
    EventStarted(UInt64, UInt64)
    EventClosed(EventSpan)
    }

    EventSpan

    pub(all) struct EventSpan {
    start : UInt64
    confirmed : UInt64
    end : UInt64
    closed : Bool
    } derive(Eq,
    Debug
    )

    EventWindow

    pub(all) struct EventWindow {
    start : UInt64
    end : UInt64
    events : Array[EventSpan]
    }

    ExactDecimal

    type ExactDecimal

    ExternalOptions

    pub(all) struct ExternalOptions {
    run_bytes : Int
    merge_bytes : Int
    fan_in : Int
    spool_bytes : UInt64
    run_limit : Int
    declaration_bytes : Int
    limits : Limits
    output : StreamOptions
    }

    ExternalReport

    pub(all) struct ExternalReport {
    input_files : Int
    messages : UInt64
    runs_created : Int
    merge_passes : Int
    peak_run_bytes : Int
    peak_merge_bytes : Int
    peak_spool_bytes : UInt64
    output_bytes : UInt64
    }

    FieldBounds

    type FieldBounds

    FieldIndex

    pub struct FieldIndex {
    digest : String
    size : UInt64
    decoder_identity : String
    paths : Array[String]
    zones : Map[UInt64, FieldZone]
    }

    FieldIndex::info

    fn FieldIndex::info(self : FieldIndex) -> Json

    FieldIndex::location_filter

    fn FieldIndex::location_filter(self : FieldIndex, filter : JsonFilter) -> ((UInt64, Int) -> Bool raise McapError) raise McapError

    Conservative physical-region predicate. Missing and differently sized regions remain candidates. Always apply the exact payload predicate after pruning.

    FieldIndex::to_bytes

    fn FieldIndex::to_bytes(self : FieldIndex, max_bytes? : Int) -> Bytes raise McapError

    FieldPage

    pub struct FieldPage {
    index : FieldIndex
    blooms : Map[UInt64, Array[Bytes]]
    first : UInt64
    end : UInt64
    }

    FieldPage::end_offset

    fn FieldPage::end_offset(self : FieldPage) -> UInt64

    FieldPage::first_offset

    fn FieldPage::first_offset(self : FieldPage) -> UInt64

    FieldPage::may_match

    fn FieldPage::may_match(self : FieldPage, offset : UInt64, span : Int, filter : JsonFilter) -> Bool raise McapError

    FieldPageLayout

    pub(all) struct FieldPageLayout {
    header_bytes : Int
    page_bytes : Int
    pages : Int
    } derive(Eq,
    Debug
    )

    FieldQueryPlan

    pub struct FieldQueryPlan {
    reader : RangeReader
    filter : JsonFilter
    decoder : PayloadDecoder
    all_skipped : Bool
    total : Int
    skipped : Int
    start_bytes : UInt64
    }

    FieldQueryPlan::info

    fn FieldQueryPlan::info(self : FieldQueryPlan) -> Json

    FieldQueryPlan::query_source

    fn FieldQueryPlan::query_source(self : FieldQueryPlan) -> QuerySource raise McapError

    FieldZone

    type FieldZone

    FileCursor

    pub struct FileCursor {
    reader : RangeReader
    original : Query
    candidates : Array[ChunkIndex]
    heap : Array[FileItem]
    allowed : Map[Int, Bool]
    pending_budget : Int
    span_budget : Int
    active : Query
    fallback : QueryCursor?
    state : FileCursorState
    loaded : Int
    visited : Int
    emitted : Int
    skipped : Int
    offset_left : Int
    remaining : Int
    pending_bytes : Int
    peak_pending : Int
    }

    FileCursor::cancel

    fn FileCursor::cancel(self : FileCursor) -> Unit

    FileCursor::current_chunk_location

    fn FileCursor::current_chunk_location(self : FileCursor) -> MessageLocation raise McapError

    FileCursor::feed

    fn FileCursor::feed(self : FileCursor, bytes : Bytes) -> Unit raise McapError

    FileCursor::feed_expanded

    fn FileCursor::feed_expanded(self : FileCursor, original : Bytes, expanded : Bytes) -> Unit raise McapError

    Host worker must be trusted; original properties, output CRC and indexes are checked.

    FileCursor::next

    fn FileCursor::next(self : FileCursor) -> Message? raise McapError

    FileCursor::next_page

    fn FileCursor::next_page(self : FileCursor, size : Int) -> Array[Message] raise McapError

    FileCursor::poll

    fn FileCursor::poll(self : FileCursor) -> FilePoll raise McapError

    Nonblocking protocol: perform FileRead externally, then feed exactly its bytes.

    FileCursor::progress

    fn FileCursor::progress(self : FileCursor) -> FileProgress

    FileCursor::seek

    fn FileCursor::seek(self : FileCursor, time : UInt64) -> Unit raise McapError

    FileCursor::skip_chunk

    fn FileCursor::skip_chunk(self : FileCursor) -> Unit raise McapError

    Skip an externally rejected region without consuming result offset/limit.

    FileCursor::upcoming_reads

    fn FileCursor::upcoming_reads(self : FileCursor, maximum : Int) -> Array[ChunkReadRequest] raise McapError

    FileCursor::write_ndjson

    fn FileCursor::write_ndjson(self : FileCursor, sink : WriteSink) -> Int raise McapError

    FileCursorState

    pub(all) enum FileCursorState {
    FileActive
    FileExhausted
    FileCancelled
    FileFailed
    } derive(Eq,
    Debug
    )

    FileDiagnosis

    pub(all) struct FileDiagnosis {
    valid : Bool
    data_valid : Bool
    data_crc_verified : Bool
    messages : Int
    chunks : Int
    data_end : UInt64?
    issues : Array[Diagnostic]
    } derive(Eq,
    Debug
    )

    FileItem

    type FileItem

    FilePoll

    pub(all) enum FilePoll {
    FileMessage(Message)
    FileRead(ReadSpan)
    FileDone
    }

    FileProgress

    pub(all) struct FileProgress {
    state : FileCursorState
    candidate_chunks : Int
    loaded_chunks : Int
    emitted_messages : Int
    skipped_messages : Int
    pending_messages : Int
    pending_bytes : Int
    peak_pending_bytes : Int
    used_fallback : Bool
    } derive(Eq,
    Debug
    )

    FileStatistics

    pub(all) struct FileStatistics {
    message_count : UInt64
    schema_count : Int
    channel_count : UInt
    attachment_count : UInt
    metadata_count : UInt
    chunk_count : UInt
    start_time : UInt64
    end_time : UInt64
    channel_message_counts : Array[(Int, UInt64)]
    } derive(Eq,
    Debug
    )

    MCAP Statistics record (opcode 0x0b), distinct from timing diagnostics.

    FileStatistics::equal

    FileStatistics::not_equal

    fn FileStatistics::not_equal(x : FileStatistics, y : FileStatistics) -> Bool

    pub(all) struct Footer {
    summary_start : UInt64
    summary_offset_start : UInt64
    summary_crc : UInt
    } derive(Eq,
    Debug
    )

    Footer::equal

    fn Footer::equal(Footer, Footer) -> Bool

    Footer::not_equal

    fn Footer::not_equal(x : Footer, y : Footer) -> Bool

    Footer::to_repr

    pub(all) struct Header {
    profile : String
    library : String
    } derive(Eq,
    Debug
    )

    Header::equal

    fn Header::equal(Header, Header) -> Bool

    Header::not_equal

    fn Header::not_equal(x : Header, y : Header) -> Bool

    Header::to_repr

    IndexRepair

    pub(all) struct IndexRepair {
    bytes : Bytes
    original : FileDiagnosis
    result : FileDiagnosis
    }

    IndexVerification

    pub(all) enum IndexVerification {
    VerifyContent((ReadSource) -> String raise McapError)
    TrustIdentity(String)
    }

    JoinMode

    pub(all) enum JoinMode {
    JoinAsOf
    JoinNearest
    } derive(Eq,
    Debug
    )

    JoinOptions

    pub(all) struct JoinOptions {
    mode : JoinMode
    tolerance : UInt64
    missing : MissingMatch
    buffer_bytes : Int
    buffer_messages : Int
    } derive(Eq,
    Debug
    )

    JoinPoll

    pub(all) enum JoinPoll {
    JoinReadLeft
    JoinReadRight
    JoinRow(TemporalMatch)
    JoinDone
    }

    JsonCondition

    type JsonCondition

    JsonDocumentParser

    type JsonDocumentParser

    JsonFilter

    pub struct JsonFilter {
    condition : JsonCondition
    projections : Array[(String, Array[String])]
    policy : JsonInvalidPolicy
    maximum : Int
    depth : Int
    }

    JsonFilter::matches

    fn JsonFilter::matches(self : JsonFilter, payload : Bytes) -> Bool raise McapError

    JsonFilter::project

    fn JsonFilter::project(self : JsonFilter, payload : Bytes) -> Json raise McapError

    JsonFilter::required_paths

    fn JsonFilter::required_paths(self : JsonFilter) -> Array[String]

    JsonFilter::selected_paths

    fn JsonFilter::selected_paths(self : JsonFilter) -> Array[String]

    JsonInvalidPolicy

    pub(all) enum JsonInvalidPolicy {
    JsonError
    JsonSkip
    } derive(Eq)

    Limits

    pub(all) struct Limits {
    max_record_bytes : Int
    max_string_bytes : Int
    max_records : Int
    max_decompressed_bytes : Int
    } derive(Eq,
    Debug
    )

    Limits::equal

    fn Limits::equal(Limits, Limits) -> Bool

    Limits::not_equal

    fn Limits::not_equal(x : Limits, y : Limits) -> Bool

    Limits::to_repr

    Log

    pub(all) struct Log {
    header : Header
    schemas : Array[Schema]
    channels : Array[Channel]
    messages : Array[Message]
    metadata : Array[Metadata]
    attachments : Array[Attachment]
    } derive(Eq,
    Debug
    )

    Log::equal

    fn Log::equal(Log, Log) -> Bool

    Log::not_equal

    fn Log::not_equal(x : Log, y : Log) -> Bool

    Log::to_repr

    LogBuilder

    pub struct LogBuilder {
    log : Log
    limits : Limits
    closed : Bool
    next_schema : Int
    next_channel : Int
    sequences : Map[Int, UInt]
    }

    Stateful recorder. Failed additions are atomic and finishing freezes the builder.

    LogBuilder::add_attachment

    fn LogBuilder::add_attachment(self : LogBuilder, value : Attachment) -> Unit raise McapError

    LogBuilder::add_metadata

    fn LogBuilder::add_metadata(self : LogBuilder, value : Metadata) -> Unit raise McapError

    LogBuilder::append

    fn LogBuilder::append(self : LogBuilder, channel_id : Int, log_time : UInt64, data : Bytes, publish_time? : UInt64) -> Unit raise McapError

    LogBuilder::finish

    fn LogBuilder::finish(self : LogBuilder, indexed? : Bool, messages_per_chunk? : Int, compression? : Compression) -> Bytes raise McapError

    LogBuilder::message_count

    fn LogBuilder::message_count(self : LogBuilder) -> Int

    LogBuilder::register_channel

    fn LogBuilder::register_channel(self : LogBuilder, topic : String, message_encoding : String, schema_id? : Int, metadata? : Array[(String, String)]) -> Int raise McapError

    LogBuilder::register_schema

    fn LogBuilder::register_schema(self : LogBuilder, name : String, encoding : String, data : Bytes) -> Int raise McapError

    LogBuilder::snapshot

    fn LogBuilder::snapshot(self : LogBuilder) -> Log

    LogStats

    pub(all) struct LogStats {
    schema_count : Int
    channel_count : Int
    message_count : Int
    payload_bytes : UInt64
    topics : Array[TopicStats]
    }

    MergeState

    type MergeState

    Message

    pub(all) struct Message {
    channel_id : Int
    sequence : UInt
    log_time : UInt64
    publish_time : UInt64
    data : Bytes
    } derive(Eq,
    Debug
    )

    Message::equal

    fn Message::equal(Message, Message) -> Bool

    Message::not_equal

    fn Message::not_equal(x : Message, y : Message) -> Bool

    Message::to_repr

    MessageIndex

    pub(all) struct MessageIndex {
    channel_id : Int
    entries : Array[(UInt64, UInt64)]
    }

    MessageLocation

    pub(all) struct MessageLocation {
    offset : UInt64
    span : Int
    inner : Int?
    } derive(Eq,
    Debug
    )

    MessageStream

    pub struct MessageStream {
    next_message : () -> Message? raise McapError
    stop : () -> Unit
    pending : () -> Int
    }

    MessageStream::cancel

    fn MessageStream::cancel(self : MessageStream) -> Unit

    MessageStream::next

    fn MessageStream::next(self : MessageStream) -> Message? raise McapError

    Metadata

    pub(all) struct Metadata {
    name : String
    entries : Array[(String, String)]
    } derive(Eq,
    Debug
    )

    Metadata::equal

    fn Metadata::equal(Metadata, Metadata) -> Bool

    Metadata::not_equal

    fn Metadata::not_equal(x : Metadata, y : Metadata) -> Bool

    Metadata::to_repr

    MetadataIndex

    pub(all) struct MetadataIndex {
    offset : UInt64
    length : UInt64
    name : String
    }

    MissingMatch

    pub(all) enum MissingMatch {
    JoinKeep
    JoinDrop
    } derive(Eq,
    Debug
    )

    PagedFieldHeader

    pub struct PagedFieldHeader {
    template : FieldIndex
    layout : FieldPageLayout
    }

    PagedFieldHeader::decode_page

    fn PagedFieldHeader::decode_page(self : PagedFieldHeader, bytes : Bytes) -> FieldPage raise McapError

    PagedFieldHeader::layout

    PayloadDecoder

    pub struct PayloadDecoder {
    decode_payload : (Schema?, Channel, Bytes) -> Json raise McapError
    }

    PayloadDecoder::decode_message

    fn PayloadDecoder::decode_message(self : PayloadDecoder, message : Message, declarations : Log, max_bytes? : Int) -> Message raise McapError

    PayloadDecoder::require_filter_fields

    fn PayloadDecoder::require_filter_fields(self : PayloadDecoder, filter : JsonFilter) -> PayloadDecoder raise McapError

    Fail explicitly when an exact condition would inspect an opaque array.

    PayloadView

    pub struct PayloadView {
    source : Bytes
    value : Json
    binary : Map[String, BinaryField]
    visited : Int
    }

    PayloadView::binary_fields

    fn PayloadView::binary_fields(self : PayloadView) -> Array[BinaryField]

    PayloadView::info

    fn PayloadView::info(self : PayloadView) -> Json

    PayloadView::json

    fn PayloadView::json(self : PayloadView) -> Json

    PayloadView::read_binary

    fn PayloadView::read_binary(self : PayloadView, path : String, offset : Int, length : Int, max_bytes? : Int) -> Bytes raise McapError

    Offsets are relative to a field's raw payload bytes, excluding array count.

    PayloadView::read_elements

    fn PayloadView::read_elements(self : PayloadView, path : String, start : Int, count : Int, max_elements? : Int) -> Json raise McapError

    PayloadView::write_binary

    fn PayloadView::write_binary(self : PayloadView, path : String, sink : WriteSink, block_bytes? : Int, cancelled? : () -> Bool) -> UInt64 raise McapError

    ProtoCache

    type ProtoCache

    ProtoEnum

    type ProtoEnum

    ProtoField

    type ProtoField

    ProtoItem

    type ProtoItem

    ProtoMessage

    type ProtoMessage

    ProtoValue

    type ProtoValue

    ProtoWire

    type ProtoWire

    ProtobufDecoder

    pub struct ProtobufDecoder {
    messages : Map[String, ProtoMessage]
    enums : Map[String, ProtoEnum]
    options : ProtobufOptions
    field_count : Array[Int]
    }

    ProtobufDecoder::decode

    fn ProtobufDecoder::decode(self : ProtobufDecoder, name : String, payload : Bytes) -> Json raise McapError

    ProtobufDecoder::decode_selected

    fn ProtobufDecoder::decode_selected(self : ProtobufDecoder, name : String, payload : Bytes, selection : DecodeSelection) -> Json raise McapError

    Skips materializing unrequested top-level fields. Selected nested messages retain normal merge/map/oneof semantics before the final nested projection.

    ProtobufOptions

    pub(all) struct ProtobufOptions {
    descriptor_bytes : Int
    message_bytes : Int
    depth : Int
    items : Int
    fields : Int
    int64_as_string : Bool
    protojson : Bool
    emit_defaults : Bool
    }

    QualityRule

    pub(all) struct QualityRule {
    topic : String
    minimum_messages : Int
    max_gap : UInt64?
    check_sequence : Bool
    condition : String?
    warning : Bool
    }

    QualityState

    type QualityState

    Query

    pub(all) struct Query {
    topics : Array[String]
    start_time : UInt64
    end_time : UInt64
    descending : Bool
    offset : Int
    limit : Int
    }

    QueryCursor

    pub struct QueryCursor {
    index : QueryIndex
    query : Query
    heap : Array[CursorRun]
    state : QueryCursorState
    matching : Int
    skipped : Int
    emitted : Int
    remaining : Int
    }

    A cursor shares the index's immutable snapshot, retaining one head per run. It does not retain a second array of all matching messages or reread files.

    QueryCursor::cancel

    fn QueryCursor::cancel(self : QueryCursor) -> Unit

    Cancellation is terminal and idempotent, releasing pending run references.

    QueryCursor::next

    fn QueryCursor::next(self : QueryCursor) -> Message?

    Return the next message, or None after exhaustion/cancellation.

    QueryCursor::next_page

    fn QueryCursor::next_page(self : QueryCursor, size : Int) -> Array[Message] raise McapError

    Page size must be positive. An invalid size leaves cursor position unchanged.

    QueryCursor::progress

    fn QueryCursor::progress(self : QueryCursor) -> QueryProgress

    QueryCursor::seek

    fn QueryCursor::seek(self : QueryCursor, time : UInt64) -> Unit raise McapError

    Restart inclusively at time within the original interval, in query direction. Offset is not reapplied; limit and progress restart. Cancelled cursors stay closed. Seeking outside the original interval fails without changing traversal state.

    QueryCursorState

    pub(all) enum QueryCursorState {
    Active
    Exhausted
    Cancelled
    } derive(Eq,
    Debug
    )

    QueryDiagnostics

    pub(all) struct QueryDiagnostics {
    total_chunks : Int
    candidate_chunks : Int
    matching_messages : Int
    used_file_indexes : Bool
    }

    QueryIndex

    pub struct QueryIndex {
    log : Log
    by_channel : Map[Int, Array[Int]]
    all_messages : Array[Int]
    }

    An in-memory time index, not an on-disk MCAP MessageIndex.

    QueryIndex::cursor

    fn QueryIndex::cursor(self : QueryIndex, query : Query) -> QueryCursor raise McapError

    Offset/limit apply once across all pages. Duplicate topic filters are harmless. The snapshot is shared with the index; returned message values and Bytes are immutable.

    QueryIndex::query

    fn QueryIndex::query(self : QueryIndex, query : Query) -> Array[Message] raise McapError

    QueryIndex::query_source

    fn QueryIndex::query_source(self : QueryIndex) -> QuerySource raise McapError

    QueryProgress

    pub(all) struct QueryProgress {
    state : QueryCursorState
    matching_messages : Int
    skipped_messages : Int
    emitted_messages : Int
    remaining_messages : Int
    buffered_heads : Int
    } derive(Eq,
    Debug
    )

    Counts refer to the current traversal, reset by seek. Remaining respects limit.

    QueryResult

    pub(all) struct QueryResult {
    messages : Array[Message]
    diagnostics : QueryDiagnostics
    }

    QuerySource

    pub struct QuerySource {
    declarations : Log
    open : (Query, Int) -> MessageStream raise McapError
    materialized : Bool
    }

    QuerySource::calibrate_time

    fn QuerySource::calibrate_time(self : QuerySource, mapping : ClockMap, same_publish_clock? : Bool) -> QuerySource raise McapError

    Query times and pagination are in corrected log-time. Scan the original calibrated domain and filter exactly, avoiding unsafe inverse rounding.

    QuerySource::declaration_log

    fn QuerySource::declaration_log(self : QuerySource) -> Log

    QuerySource::decode_payload

    fn QuerySource::decode_payload(self : QuerySource, decoder : PayloadDecoder, max_bytes? : Int) -> QuerySource raise McapError

    QuerySource::filter_json

    fn QuerySource::filter_json(self : QuerySource, filter : JsonFilter) -> QuerySource raise McapError

    QuerySource::stream

    fn QuerySource::stream(self : QuerySource, query? : Query, pending_bytes? : Int) -> MessageStream raise McapError

    Open a source-local message stream without Dataset ID remapping. The caller must cancel on early termination. Source ordering and limits follow Query.

    RangeReader

    pub struct RangeReader {
    source : ReadSource
    limits : Limits
    declarations : Log
    indexes : Array[ChunkIndex]
    cache : Array[CachedChunk]
    cache_budget : Int
    fallback_budget : Int
    data_end : UInt64
    cache_bytes : Int
    peak_cache_bytes : Int
    decoded_chunks : Int
    cache_hits : Int
    full_validation : Bool
    }

    Summary-checked selective reader. Unread data and DataEnd CRC are not validated.

    RangeReader::channels

    fn RangeReader::channels(self : RangeReader) -> Array[Channel]

    RangeReader::clear_cache

    fn RangeReader::clear_cache(self : RangeReader) -> Unit

    RangeReader::cursor

    fn RangeReader::cursor(self : RangeReader, query : Query, pending_bytes? : Int, span_bytes? : Int) -> FileCursor raise McapError

    RangeReader::declaration_log

    fn RangeReader::declaration_log(self : RangeReader) -> Log

    RangeReader::diagnostics

    fn RangeReader::diagnostics(self : RangeReader) -> ReadDiagnostics

    RangeReader::plan_field_query

    fn RangeReader::plan_field_query(self : RangeReader, index : FieldIndex, filter : JsonFilter, fingerprint : (ReadSource) -> String raise McapError, decoder_identity : String, decoder? : PayloadDecoder) -> FieldQueryPlan raise McapError

    Caller must use the same decoder semantics identified when building the index.

    RangeReader::query

    fn RangeReader::query(self : RangeReader, query : Query) -> Array[Message] raise McapError

    RangeReader::query_source

    fn RangeReader::query_source(self : RangeReader) -> QuerySource raise McapError

    RangeReader::validate_all

    fn RangeReader::validate_all(self : RangeReader) -> Unit raise McapError

    RangeRecord

    type RangeRecord

    ReadDiagnostics

    pub(all) struct ReadDiagnostics {
    read_calls : Int
    bytes_read : UInt64
    decoded_chunks : Int
    cache_hits : Int
    cache_bytes : Int
    peak_cache_bytes : Int
    indexed : Bool
    full_validation : Bool
    } derive(Eq,
    Debug
    )

    ReadSource

    pub struct ReadSource {
    size : UInt64
    read_at : (UInt64, Int) -> Bytes raise McapError
    calls : Int
    bytes_read : UInt64
    }

    The callback may short-read, but must return bytes from a stable source.

    ReadSource::read_exact

    fn ReadSource::read_exact(self : ReadSource, offset : UInt64, length : Int) -> Bytes raise McapError

    Retry short reads, rejecting zero progress, over-reads and out-of-bounds spans.

    ReadSource::size

    fn ReadSource::size(self : ReadSource) -> UInt64

    ReadSpan

    pub(all) struct ReadSpan {
    offset : UInt64
    length : Int
    } derive(Eq,
    Debug
    )

    Record

    pub(all) struct Record {
    opcode : Byte
    offset : Int
    body : Bytes
    } derive(Eq,
    Debug
    )

    Record::equal

    fn Record::equal(Record, Record) -> Bool

    Record::not_equal

    fn Record::not_equal(x : Record, y : Record) -> Bool

    Record::to_repr

    RecordingOptions

    pub(all) struct RecordingOptions {
    stream : StreamOptions
    segment_messages : Int
    segment_bytes_target : UInt64
    segment_span : UInt64
    segment_limit : Int
    sequences : SequencePolicy
    source_identity : String
    } derive(Eq,
    Debug
    )

    RecordingSession

    pub struct RecordingSession {
    template : Log
    config : Json
    options : RecordingOptions
    storage : RecordingStorage
    completed : Array[Json]
    next_ordinal : Int
    active : Int?
    writer : StreamWriter?
    next_sequence : Map[Int, UInt64]
    committed_next : Map[Int, UInt64]
    total : UInt64
    start_time : UInt64
    end_time : UInt64
    state : WriterState
    adopted : Int
    quarantined : Int
    }

    RecordingSession::append

    fn RecordingSession::append(self : RecordingSession, message : Message) -> Unit raise McapError

    RecordingSession::cancel

    fn RecordingSession::cancel(self : RecordingSession) -> Unit

    Leaves a reserved, possibly incomplete tail for inspection on reopen.

    RecordingSession::committed_messages

    fn RecordingSession::committed_messages(self : RecordingSession) -> UInt64

    RecordingSession::finish

    fn RecordingSession::finish(self : RecordingSession) -> Json raise McapError

    RecordingSession::info

    RecordingStorage

    pub struct RecordingStorage {
    create : (Int) -> WriteSink raise McapError
    open : (Int) -> ReadSource raise McapError
    exists : (Int) -> Bool raise McapError
    seal : (Int) -> Unit raise McapError
    save : (Bytes) -> Unit raise McapError
    quarantine : (Int) -> Unit raise McapError
    list : () -> Array[Int] raise McapError
    fingerprint : (ReadSource) -> String raise McapError
    }

    save/seal must return only after their durable atomic publication. create owns a new empty segment; quarantine preserves its bytes outside the set.

    RecoveredBlock

    pub(all) struct RecoveredBlock {
    start : UInt64
    end : UInt64
    messages : Int
    } derive(Eq,
    Debug
    )

    RecoveryReport

    pub(all) struct RecoveryReport {
    log : Log
    recovered_records : Int
    stop_offset : Int
    complete : Bool
    reason : String
    }

    Replay

    pub struct Replay {
    source : QuerySource
    query : Query
    budget : Int
    stream : MessageStream
    pending : Message?
    state : ReplayState
    position : UInt64
    wall : UInt64
    last_clock : UInt64?
    numerator : Int
    denominator : Int
    emitted : Int
    generation : Int
    previous : UInt64?
    }

    Replay::cancel

    fn Replay::cancel(self : Replay) -> Unit

    Replay::pause

    fn Replay::pause(self : Replay, now : UInt64) -> Unit raise McapError

    Replay::poll

    fn Replay::poll(self : Replay, now : UInt64) -> ReplayPoll raise McapError

    Replay::progress

    fn Replay::progress(self : Replay, now : UInt64) -> ReplayProgress raise McapError

    Replay::resume

    fn Replay::resume(self : Replay, now : UInt64) -> Unit raise McapError

    Replay::seek

    fn Replay::seek(self : Replay, time : UInt64, now : UInt64) -> Unit raise McapError

    Replay::set_rate

    fn Replay::set_rate(self : Replay, numerator : Int, denominator : Int, now : UInt64) -> Unit raise McapError

    Replay::step

    fn Replay::step(self : Replay, now : UInt64) -> Message? raise McapError

    ReplayPoll

    pub(all) enum ReplayPoll {
    ReplayWait(UInt64)
    ReplayIdle
    ReplayEvent(Message)
    ReplayEnd
    }

    ReplayProgress

    pub(all) struct ReplayProgress {
    state : ReplayState
    position : UInt64
    emitted : Int
    rate_numerator : Int
    rate_denominator : Int
    generation : Int
    }

    ReplayState

    pub(all) enum ReplayState {
    ReplayReady
    ReplayRunning
    ReplayPaused
    ReplayDone
    ReplayCancelled
    ReplayFailed
    } derive(Eq,
    Debug
    )

    Ros1Decoder

    pub struct Ros1Decoder {
    layout : CdrDecoder
    }

    Ros1Decoder::decode

    fn Ros1Decoder::decode(self : Ros1Decoder, bytes : Bytes) -> Json raise McapError

    Ros1Decoder::view

    fn Ros1Decoder::view(self : Ros1Decoder, bytes : Bytes, selection? : DecodeSelection) -> PayloadView raise McapError

    RunHead

    type RunHead

    RunInput

    type RunInput

    Schema

    pub(all) struct Schema {
    id : Int
    name : String
    encoding : String
    data : Bytes
    } derive(Eq,
    Debug
    )

    Schema::equal

    fn Schema::equal(Schema, Schema) -> Bool

    Schema::not_equal

    fn Schema::not_equal(x : Schema, y : Schema) -> Bool

    Schema::to_repr

    SchemaDecoder

    type SchemaDecoder

    SequencePolicy

    pub(all) enum SequencePolicy {
    PreserveSequence
    ContinueSequence
    } derive(Eq,
    Debug
    )

    SidecarCursor

    pub struct SidecarCursor {
    reader : SidecarReader
    original : Query
    allowed : Map[Int, Bool]
    query : Query
    state : FileCursorState
    position : Int
    remaining : Int
    offset_left : Int
    emitted : Int
    skipped : Int
    waiting : ReadSpan?
    ready : Message?
    }

    SidecarCursor::cancel

    fn SidecarCursor::cancel(self : SidecarCursor) -> Unit

    SidecarCursor::feed

    fn SidecarCursor::feed(self : SidecarCursor, bytes : Bytes) -> Unit raise McapError

    SidecarCursor::next

    fn SidecarCursor::next(self : SidecarCursor) -> Message? raise McapError

    SidecarCursor::next_page

    fn SidecarCursor::next_page(self : SidecarCursor, size : Int) -> Array[Message] raise McapError

    SidecarCursor::pending_bytes

    fn SidecarCursor::pending_bytes(self : SidecarCursor) -> Int

    SidecarCursor::poll

    fn SidecarCursor::poll(self : SidecarCursor) -> FilePoll raise McapError

    SidecarCursor::progress

    fn SidecarCursor::progress(self : SidecarCursor) -> Json

    SidecarCursor::seek

    fn SidecarCursor::seek(self : SidecarCursor, time : UInt64) -> Unit raise McapError

    SidecarCursor::write_ndjson

    fn SidecarCursor::write_ndjson(self : SidecarCursor, sink : WriteSink) -> Int raise McapError

    SidecarEntry

    type SidecarEntry

    SidecarReader

    pub struct SidecarReader {
    source : ReadSource
    declarations : Log
    entries : Array[SidecarEntry]
    positions : Array[Int]
    limits : Limits
    digest : String
    content_verified : Bool
    index_bytes : Int
    cache_budget : Int
    cached_offset : UInt64?
    cached_span : Int
    cached : Chunk?
    cached_bytes : Int
    }

    SidecarReader::channels

    fn SidecarReader::channels(self : SidecarReader) -> Array[Channel]

    SidecarReader::clear_cache

    fn SidecarReader::clear_cache(self : SidecarReader) -> Unit

    SidecarReader::cursor

    fn SidecarReader::cursor(self : SidecarReader, query : Query) -> SidecarCursor raise McapError

    SidecarReader::declaration_log

    fn SidecarReader::declaration_log(self : SidecarReader) -> Log

    SidecarReader::info

    fn SidecarReader::info(self : SidecarReader) -> Json

    SidecarReader::query_source

    fn SidecarReader::query_source(self : SidecarReader) -> QuerySource raise McapError

    SidecarReader::validate_entries

    fn SidecarReader::validate_entries(self : SidecarReader) -> Unit raise McapError

    SkippedBlock

    pub(all) struct SkippedBlock {
    start : UInt64
    end : UInt64
    reason : String
    } derive(Eq,
    Debug
    )

    SortRow

    type SortRow

    SortSession

    type SortSession

    SortStorage

    pub struct SortStorage {
    create : (Int) -> WriteSink raise McapError
    open : (Int) -> ReadSource raise McapError
    remove : (Int) -> Unit raise McapError
    finish : () -> Unit raise McapError
    }

    SourceScanEvent

    pub(all) enum SourceScanEvent {
    RecordStart(UInt64)
    RecordRead(UInt64, Byte, Int)
    RecordAccepted(UInt64, Byte, Int)
    DataChecksum(UInt64, UInt, UInt)
    DataValidated(UInt64, Bool)
    SummaryStart(UInt64)
    }

    Physical scan evidence. Accepted records can still fail the final data CRC.

    SourceTables

    type SourceTables

    SourceVisitor

    pub(all) struct SourceVisitor {
    schema : (Schema) -> Unit raise McapError
    channel : (Channel) -> Unit raise McapError
    message : (Message, MessageLocation) -> Unit raise McapError
    attachment : (Attachment) -> Unit raise McapError
    metadata : (Metadata) -> Unit raise McapError
    }

    StreamDecoder

    pub struct StreamDecoder {
    limits : Limits
    pending :
    Buffer

    expected : Int
    offset : Int
    count : Int
    failed : Bool
    finished : Bool
    }

    Incremental record framing. File magic/envelope checks belong to scan_file.

    StreamDecoder::buffered_bytes

    fn StreamDecoder::buffered_bytes(self : StreamDecoder) -> Int

    StreamDecoder::feed

    fn StreamDecoder::feed(self : StreamDecoder, bytes : Bytes) -> Array[Record] raise McapError

    StreamDecoder::finish

    fn StreamDecoder::finish(self : StreamDecoder) -> Unit raise McapError

    StreamDecoder::new

    fn StreamDecoder::new(limits? : Limits, base? : Int) -> StreamDecoder raise McapError

    StreamDecoder::record_count

    fn StreamDecoder::record_count(self : StreamDecoder) -> Int

    StreamOptions

    pub(all) struct StreamOptions {
    compression : Compression
    chunk_bytes : Int
    chunk_messages : Int
    chunk_span : UInt64
    summary_bytes : Int
    } derive(Eq,
    Debug
    )

    StreamWriter

    pub struct StreamWriter {
    sink : WriteSink
    options : StreamOptions
    limits : Limits
    declarations : Log
    pending : Array[Record]
    summary : Array[(Byte, Bytes)]
    counts : Map[Int, UInt64]
    state : WriterState
    crc : UInt
    records : Int
    buffered : Int
    peak_buffered : Int
    retained : Int
    messages : UInt64
    chunks : UInt
    attachments : UInt
    metadata : UInt
    start_time : UInt64
    end_time : UInt64
    chunk_start : UInt64
    chunk_end : UInt64
    }

    StreamWriter::add_attachment

    fn StreamWriter::add_attachment(self : StreamWriter, value : Attachment) -> Unit raise McapError

    StreamWriter::add_channel

    fn StreamWriter::add_channel(self : StreamWriter, channel : Channel) -> Unit raise McapError

    StreamWriter::add_metadata

    fn StreamWriter::add_metadata(self : StreamWriter, value : Metadata) -> Unit raise McapError

    StreamWriter::add_schema

    fn StreamWriter::add_schema(self : StreamWriter, schema : Schema) -> Unit raise McapError

    StreamWriter::append

    fn StreamWriter::append(self : StreamWriter, message : Message) -> Unit raise McapError

    StreamWriter::cancel

    fn StreamWriter::cancel(self : StreamWriter) -> Unit

    Leaves an unfinished output prefix; does not fabricate a successful footer.

    StreamWriter::finish

    fn StreamWriter::finish(self : StreamWriter) -> WriteProgress raise McapError

    StreamWriter::flush

    fn StreamWriter::flush(self : StreamWriter) -> Unit raise McapError

    Manual flush; automatic flush also occurs before byte/count/time thresholds.

    StreamWriter::progress

    fn StreamWriter::progress(self : StreamWriter) -> WriteProgress

    SummaryOffset

    pub(all) struct SummaryOffset {
    opcode : Byte
    start : UInt64
    length : UInt64
    }

    TemporalJoinCursor

    pub struct TemporalJoinCursor {
    options : JoinOptions
    left : Message?
    before : Message?
    after : Message?
    left_done : Bool
    right_done : Bool
    waiting : Int
    last_left : UInt64?
    last_right : UInt64?
    state : FileCursorState
    left_read : UInt64
    right_read : UInt64
    matched : UInt64
    unmatched : UInt64
    peak_bytes : Int
    }

    TemporalJoinCursor::cancel

    fn TemporalJoinCursor::cancel(self : TemporalJoinCursor) -> Unit

    TemporalJoinCursor::feed_left

    fn TemporalJoinCursor::feed_left(self : TemporalJoinCursor, message : Message?) -> Unit raise McapError

    TemporalJoinCursor::feed_right

    fn TemporalJoinCursor::feed_right(self : TemporalJoinCursor, message : Message?) -> Unit raise McapError

    TemporalJoinCursor::info

    TemporalJoinCursor::poll

    Ascending many-to-one left join. Equal-distance ties prefer the earlier timestamp; duplicate right timestamps prefer the first stable input record.

    TemporalJoinStream

    pub struct TemporalJoinStream {
    cursor : TemporalJoinCursor
    left : MessageStream
    right : MessageStream
    closed : Bool
    }

    TemporalJoinStream::cancel

    fn TemporalJoinStream::cancel(self : TemporalJoinStream) -> Unit

    TemporalJoinStream::info

    TemporalJoinStream::next

    TemporalMatch

    pub(all) struct TemporalMatch {
    left : Message
    right : Message?
    distance : UInt64?
    right_is_earlier : Bool
    } derive(Eq,
    Debug
    )

    TemporaryWorkspace

    type TemporaryWorkspace

    TopicStats

    pub(all) struct TopicStats {
    channel_id : Int
    topic : String
    message_count : Int
    payload_bytes : UInt64
    first_time : UInt64?
    last_time : UInt64?
    maximum_gap : UInt64
    median_gap : UInt64
    p95_gap : UInt64
    arrival_regressions : Int
    sequence_discontinuities : Int
    publish_after_log : Int
    frequency_hz : Double
    }

    TransformOptions

    pub(all) struct TransformOptions {
    query : Query
    topic_map : Array[(String, String)]
    sample_period : UInt64
    sample_origin : UInt64
    align_start : UInt64?
    attachment_names : Array[String]
    attachment_media_types : Array[String]
    attachment_start : UInt64
    attachment_end : UInt64
    metadata_names : Array[String]
    metadata_entries : Array[(String, String)]
    }

    TransformReport

    pub(all) struct TransformReport {
    input_messages : UInt64
    sampled_messages : Int
    output_messages : Int
    output_bytes : UInt64
    peak_temporary_bytes : UInt64
    }

    TriggerOptions

    pub(all) struct TriggerOptions {
    pre : UInt64
    post : UInt64
    buffer_bytes : Int
    buffer_messages : Int
    clip_bytes : Int
    clip_messages : Int
    event_limit : Int
    }

    TriggerRecorder

    pub struct TriggerRecorder {
    template : Log
    options : TriggerOptions
    emit : (Log, Json) -> Unit raise McapError
    ring : Array[Message?]
    head : Int
    buffered_bytes : Int
    first : UInt64?
    last : UInt64?
    capacity_evicted_through : UInt64?
    evicted : UInt64
    active : Array[Message]?
    active_bytes : Int
    start : UInt64
    deadline : UInt64
    pre_complete : Bool
    labels : Array[String]
    completed : Int
    state : WriterState
    delivering : Bool
    }

    TriggerRecorder::cancel

    fn TriggerRecorder::cancel(self : TriggerRecorder) -> Unit

    TriggerRecorder::feed

    fn TriggerRecorder::feed(self : TriggerRecorder, message : Message) -> Unit raise McapError

    TriggerRecorder::finish

    fn TriggerRecorder::finish(self : TriggerRecorder) -> Unit raise McapError

    TriggerRecorder::progress

    fn TriggerRecorder::progress(self : TriggerRecorder) -> Json

    TriggerRecorder::trigger

    fn TriggerRecorder::trigger(self : TriggerRecorder, label : String) -> Unit raise McapError

    Trigger at the latest fed message's log time. Repeated triggers extend a single window; output count and label count are independently bounded.

    WriteProgress

    pub(all) struct WriteProgress {
    state : WriterState
    bytes_written : UInt64
    messages : UInt64
    chunks : UInt
    buffered_bytes : Int
    peak_buffered_bytes : Int
    retained_summary_bytes : Int
    } derive(Eq,
    Debug
    )

    WriteSink

    pub struct WriteSink {
    write_all : (Bytes) -> Unit raise McapError
    position : UInt64
    }

    WriteSink::position

    fn WriteSink::position(self : WriteSink) -> UInt64

    WriteSink::write

    fn WriteSink::write(self : WriteSink, bytes : Bytes) -> Unit raise McapError

    Write raw bytes to an exclusively owned sink (for temporary run storage).

    WriterState

    pub(all) enum WriterState {
    WriterOpen
    WriterFinished
    WriterFailed
    WriterCancelled
    } derive(Eq,
    Debug
    )

    aggregate_external

    fn aggregate_external(cursor : DatasetCursor, channels : Array[Channel], storage : SortStorage, output : WriteSink, aggregate? : AggregateOptions, external? : ExternalOptions, cancelled? : () -> Bool) -> Json raise McapError

    Stream one exact aggregate JSON object per group; failures leave provisional output. Temporary storage is dedicated and always cleaned on return.

    aggregate_json

    fn aggregate_json(cursor : DatasetCursor, channels : Array[Channel], options? : AggregateOptions) -> Json raise McapError

    Consume ordered messages without retaining history; returns deterministic, bounded groups. Payload JSON is capped at 1 MiB/64 levels.

    aggregate_options

    fn aggregate_options() -> AggregateOptions

    all_query

    fn all_query() -> Query

    block_recovery_json

    fn block_recovery_json(report : BlockRecoveryReport) -> Json

    build_disk_sidecar

    fn build_disk_sidecar(source : ReadSource, storage : SortStorage, output : WriteSink, fingerprint : (ReadSource) -> String raise McapError, identity? : String, page_entries? : Int, options? : ExternalOptions, cancelled? : () -> Bool) -> Json raise McapError

    Write MIDX2 pages using external sorting. Storage is dedicated to this job. Output is provisional until success; caller owns atomic publication.

    build_field_index

    fn build_field_index(source : ReadSource, paths : Array[String], fingerprint : (ReadSource) -> String raise McapError, decoder_identity : String, decoder? : PayloadDecoder, limits? : Limits, max_zones? : Int, state_bytes? : Int, cancelled? : () -> Bool) -> FieldIndex raise McapError

    build_paged_field_index

    fn build_paged_field_index(source : ReadSource, sink : WriteSink, paths : Array[String], fingerprint : (ReadSource) -> String raise McapError, decoder_identity : String, page_bytes? : Int, page_zones? : Int, bloom? : Bool, limits? : Limits, cancelled? : () -> Bool) -> Json raise McapError

    Streaming physical-order build: one region and one fixed-size page in memory. Output must be staged by the caller, since source validation finishes last.

    build_query_index

    fn build_query_index(log : Log) -> QueryIndex raise McapError

    build_sidecar

    fn build_sidecar(source : ReadSource, fingerprint : (ReadSource) -> String raise McapError, identity? : String, limits? : Limits, index_bytes? : Int, declaration_bytes? : Int, cancelled? : () -> Bool) -> Bytes raise McapError

    calibrate_log

    fn calibrate_log(log : Log, mapping : ClockMap, same_publish_clock? : Bool) -> Log raise McapError

    Rewrite all log clocks, optionally publish/create clocks only when the caller asserts the same clock domain. Payload timestamps are deliberately untouched.

    cdr_decoder

    fn cdr_decoder(name : String, definition : Bytes, options? : CdrOptions) -> CdrDecoder raise McapError

    cdr_options

    fn cdr_options() -> CdrOptions

    check_match_quality

    fn check_match_quality(left : MessageStream, right : MessageStream, tolerance : UInt64, minimum_basis_points : Int) -> Json raise McapError

    Match availability over two ascending streams, retaining the existing join's bounded state. Basis points use integer arithmetic and include unmatched rows.

    check_quality

    fn check_quality(source : QuerySource, rules : Array[QualityRule], start : UInt64, end : UInt64, detail_limit? : Int, pending_bytes? : Int, cancelled? : () -> Bool) -> Json raise McapError

    Evaluate the entire explicit closed interval, without query pagination. Gap checks include leading/trailing silence. Sequence checks are per channel.

    checkpoint_progress

    fn checkpoint_progress(bytes : Bytes) -> Json raise McapError

    checkpoint_run_ids

    fn checkpoint_run_ids(bytes : Bytes) -> Array[Int] raise McapError

    Read owned run IDs for a storage adapter's orphan cleanup. No path strings.

    clock_map

    fn clock_map(anchors : Array[ClockAnchor], extrapolate? : Bool) -> ClockMap raise McapError

    Monotone, piecewise affine map. Integer interpolation rounds down. Anchors are supplied by the caller, never inferred from unrelated sensor values.

    codec_worker_handler

    fn codec_worker_handler(input_bytes? : Int, output_bytes? : Int, context_bytes? : Int) -> ((String) -> String) raise McapError

    Bounded pure MoonBit handler for a trusted host worker entry.

    compress_payload

    fn compress_payload(bytes : Bytes, compression : Compression) -> Bytes

    compute_file_statistics

    fn compute_file_statistics(log : Log, chunk_count : UInt) -> FileStatistics

    context_rule

    fn context_rule(topic : String) -> ContextRule

    copy_log

    fn copy_log(log : Log) -> Log

    Copy mutable input bytes and arrays so later caller mutations do not stale the index.

    crc32

    fn crc32(bytes : Bytes) -> UInt

    IEEE CRC32, reflected polynomial, initial and final XOR 0xffffffff.

    crc32_extend

    fn crc32_extend(previous : UInt, bytes : Bytes) -> UInt

    Accepts a previously finalized CRC, allowing independent streaming blocks.

    dataset_message_json

    fn dataset_message_json(record : DatasetMessage, channels : Array[Channel]) -> Json

    decode_attachment

    fn decode_attachment(bytes : Bytes, limits? : Limits) -> Attachment raise McapError

    decode_attachment_index

    fn decode_attachment_index(bytes : Bytes, limits? : Limits) -> AttachmentIndex raise McapError

    decode_channel

    fn decode_channel(bytes : Bytes, limits? : Limits) -> Channel raise McapError

    decode_chunk

    fn decode_chunk(bytes : Bytes, limits? : Limits) -> Chunk raise McapError

    decode_chunk_index

    fn decode_chunk_index(bytes : Bytes, limits? : Limits) -> ChunkIndex raise McapError

    decode_file_statistics

    fn decode_file_statistics(bytes : Bytes, limits? : Limits) -> FileStatistics raise McapError

    fn decode_footer(bytes : Bytes) -> Footer raise McapError

    decode_header

    fn decode_header(bytes : Bytes, limits? : Limits) -> Header raise McapError

    decode_hex

    fn decode_hex(value : String, max_bytes? : Int) -> Bytes raise McapError

    decode_message

    fn decode_message(bytes : Bytes, limits? : Limits) -> Message raise McapError

    decode_message_index

    fn decode_message_index(bytes : Bytes, limits? : Limits) -> MessageIndex raise McapError

    decode_metadata

    fn decode_metadata(bytes : Bytes, limits? : Limits) -> Metadata raise McapError

    decode_metadata_index

    fn decode_metadata_index(bytes : Bytes, limits? : Limits) -> MetadataIndex raise McapError

    decode_records

    fn decode_records(bytes : Bytes, limits? : Limits, base? : Int) -> Array[Record] raise McapError

    decode_schema

    fn decode_schema(bytes : Bytes, limits? : Limits) -> Schema raise McapError

    decode_selection

    fn decode_selection() -> DecodeSelection

    decode_summary_offset

    fn decode_summary_offset(bytes : Bytes) -> SummaryOffset raise McapError

    decompress_payload

    fn decompress_payload(bytes : Bytes, compression : String, expected_size : UInt64, limits? : Limits) -> Bytes raise McapError

    default_limits

    fn default_limits() -> Limits

    detect_events

    fn detect_events(source : QuerySource, rule : EventRule, start? : UInt64, end? : UInt64, event_limit? : Int, pending_bytes? : Int, cancelled? : () -> Bool) -> Array[EventSpan] raise McapError

    diagnose_file

    fn diagnose_file(bytes : Bytes, limits? : Limits) -> FileDiagnosis

    Full validation; precise offsets are supplied only when directly known.

    diagnose_source

    fn diagnose_source(source : ReadSource, limits? : Limits, summary_bytes? : Int, read_bytes? : Int, cancelled? : () -> Bool) -> Json raise McapError

    Strict bounded-record validation. Stops at the first failure; prefix parsing is not a proof of whole-file integrity. Metadata is validated before data.

    diagnosis_json

    fn diagnosis_json(report : FileDiagnosis) -> Json

    diff_options

    fn diff_options() -> DiffOptions

    diff_sources

    fn diff_sources(left : ReadSource, right : ReadSource, options? : DiffOptions) -> Json raise McapError

    Compare validated sources by exact timestamp buckets and message multisets. Ignores compression, physical ID values, indexes and Header.library.

    empty_log

    fn empty_log(profile? : String) -> Log

    encode_attachment

    fn encode_attachment(value : Attachment, checksums? : Bool) -> Bytes

    encode_attachment_index

    fn encode_attachment_index(value : AttachmentIndex) -> Bytes

    encode_channel

    fn encode_channel(value : Channel) -> Bytes raise McapError

    encode_chunk

    fn encode_chunk(records : Array[Record], start_time : UInt64, end_time : UInt64, checksums? : Bool, compression? : Compression) -> Bytes raise McapError

    encode_chunk_index

    fn encode_chunk_index(value : ChunkIndex) -> Bytes raise McapError

    encode_file_statistics

    fn encode_file_statistics(value : FileStatistics) -> Bytes raise McapError

    fn encode_footer(value : Footer) -> Bytes

    encode_header

    fn encode_header(value : Header) -> Bytes

    encode_hex

    fn encode_hex(bytes : Bytes) -> String

    Portable hexadecimal transport for binary fixtures and command output.

    encode_message

    fn encode_message(value : Message) -> Bytes raise McapError

    encode_message_index

    fn encode_message_index(value : MessageIndex) -> Bytes raise McapError

    encode_metadata

    fn encode_metadata(value : Metadata) -> Bytes raise McapError

    encode_metadata_index

    fn encode_metadata_index(value : MetadataIndex) -> Bytes

    encode_schema

    fn encode_schema(value : Schema) -> Bytes raise McapError

    encode_summary_offset

    fn encode_summary_offset(value : SummaryOffset) -> Bytes

    event_detector

    fn event_detector(rule : EventRule) -> EventDetector raise McapError

    event_rule

    fn event_rule(topic : String, enter : String) -> EventRule

    event_span_json

    fn event_span_json(event : EventSpan) -> Json

    event_windows

    fn event_windows(events : Array[EventSpan], pre : UInt64, post : UInt64, merge_gap? : UInt64) -> Array[EventWindow] raise McapError

    Overlapping/nearby closed windows merge; event evidence remains separate.

    expand_worker_chunk

    fn expand_worker_chunk(body : Bytes, limits? : Limits) -> Bytes raise McapError

    Pure MoonBit work unit: bounded decompression and CRC, preserving raw records.

    external_merge

    fn external_merge(sources : Array[ReadSource], storage : SortStorage, output : WriteSink, options? : ExternalOptions, cancelled? : () -> Bool, preserve_ids? : Bool) -> ExternalReport raise McapError

    Stable external sorting of full MCAP inputs; temporary storage is caller-selected.

    external_options

    fn external_options() -> ExternalOptions

    external_report_json

    fn external_report_json(report : ExternalReport) -> Json

    extract_context_clip

    fn extract_context_clip(source : QuerySource, start : UInt64, end : UInt64, topics? : Array[String], context? : Array[ContextRule], message_limit? : Int, byte_limit? : Int, pending_bytes? : Int, cancelled? : () -> Bool) -> ContextClip raise McapError

    Export a bounded, independent message clip. Explicit context rules select the last N messages per topic (not semantic state merging). Aux data is not fetched by QuerySource; the manifest states this instead of implying retention.

    find_attachments

    fn find_attachments(log : Log, name? : String, media_type? : String) -> Array[Attachment]

    frame

    fn frame(opcode : Byte, body : Bytes) -> Bytes raise McapError

    inspect_json

    fn inspect_json(log : Log) -> Json raise McapError

    inspect_recording_manifest

    fn inspect_recording_manifest(bytes : Bytes) -> Json raise McapError

    io_error

    fn io_error(message : String) -> McapError

    Build an error for host adapters without exposing internal error constructors.

    join_options

    fn join_options() -> JoinOptions

    json_filter

    fn json_filter(expression : String, select? : Array[String], invalid? : JsonInvalidPolicy, max_bytes? : Int, max_depth? : Int) -> JsonFilter raise McapError

    json_query_record

    fn json_query_record(record : DatasetMessage, channels : Array[Channel], filter : JsonFilter) -> Json raise McapError

    magic

    fn magic() -> Bytes

    memory_source

    fn memory_source(bytes : Bytes) -> ReadSource

    merge_logs

    fn merge_logs(logs : Array[Log], chronological? : Bool) -> Log raise McapError

    Deduplicate declarations by content, not source-local identifiers.

    message_stream

    fn message_stream(next : () -> Message? raise McapError, cancel : () -> Unit, pending_bytes? : () -> Int) -> MessageStream

    messages_csv

    fn messages_csv(messages : Array[Message], channels : Array[Channel]) -> String

    messages_json

    fn messages_json(messages : Array[Message], channels : Array[Channel]) -> Json

    new_builder

    fn new_builder(profile? : String, limits? : Limits) -> LogBuilder raise McapError

    new_read_source

    fn new_read_source(size : UInt64, read_at : (UInt64, Int) -> Bytes raise McapError) -> ReadSource

    new_stream_writer

    fn new_stream_writer(sink : WriteSink, profile? : String, options? : StreamOptions, limits? : Limits) -> StreamWriter raise McapError

    new_write_sink

    fn new_write_sink(write_all : (Bytes) -> Unit raise McapError) -> WriteSink

    Synchronous backpressure: callback must accept all bytes or raise.

    open_dataset

    fn open_dataset(sources : Array[QuerySource], limits? : Limits, declaration_bytes? : Int, max_sources? : Int) -> Dataset raise McapError

    open_disk_sidecar

    fn open_disk_sidecar(source : ReadSource, index : ReadSource, verification : IndexVerification, limits? : Limits, declaration_bytes? : Int, cache_pages? : Int, chunk_cache_bytes? : Int, index_bytes? : UInt64) -> DiskSidecarReader raise McapError

    open_field_index

    fn open_field_index(source : ReadSource, bytes : Bytes, fingerprint : (ReadSource) -> String raise McapError, decoder_identity : String, max_bytes? : Int, max_zones? : Int) -> FieldIndex raise McapError

    open_paged_field_header

    fn open_paged_field_header(source_size : UInt64, index_size : UInt64, bytes : Bytes, digest : String, decoder_identity : String) -> PagedFieldHeader raise McapError

    Header bytes consist of the 12-byte prefix followed by JSON and its CRC.

    open_reader

    fn open_reader(source : ReadSource, limits? : Limits, cache_bytes? : Int, summary_bytes? : Int, fallback_bytes? : Int) -> RangeReader raise McapError

    open_recording_session

    fn open_recording_session(template : Log, storage : RecordingStorage, manifest? : Bytes, options? : RecordingOptions) -> RecordingSession raise McapError

    open_sidecar

    fn open_sidecar(source : ReadSource, bytes : Bytes, verification : IndexVerification, limits? : Limits, index_bytes? : Int, cache_bytes? : Int) -> SidecarReader raise McapError

    parse_command

    fn parse_command(args : Array[String]) -> CommandOptions raise McapError

    parse_decimal

    fn parse_decimal(text : String) -> UInt64 raise McapError

    Strict unsigned decimal parsing, preserving all UInt64 timestamp bits.

    payload_decoder

    fn payload_decoder(decode : (Schema?, Channel, Bytes) -> Json raise McapError) -> PayloadDecoder

    protobuf_decoder

    fn protobuf_decoder(descriptor_set : Bytes, options? : ProtobufOptions) -> ProtobufDecoder raise McapError

    protobuf_json_options

    fn protobuf_json_options() -> ProtobufOptions

    protobuf_options

    fn protobuf_options() -> ProtobufOptions

    quality_rule

    fn quality_rule(topic : String) -> QualityRule

    query_file

    fn query_file(bytes : Bytes, query : Query, limits? : Limits) -> QueryResult raise McapError

    Query already-loaded bytes using MCAP chunk ranges and channel index maps. Structural validation still scans all records; this is not disk-lazy I/O.

    query_source

    fn query_source(declarations : Log, open : (Query, Int) -> MessageStream raise McapError, materialized? : Bool) -> QuerySource raise McapError

    Provider contract: stable snapshot, ordered messages, exact declarations, and truthful nonnegative pending payload accounting.

    read_diagnostics_json

    fn read_diagnostics_json(d : ReadDiagnostics) -> Json

    read_log

    fn read_log(bytes : Bytes, limits? : Limits) -> Log raise McapError

    rebuild_indexes

    fn rebuild_indexes(bytes : Bytes, compression? : Compression, messages_per_chunk? : Int, limits? : Limits) -> IndexRepair raise McapError

    Rebuild only from a complete, checked data section. Never salvage across damage.

    recording_options

    fn recording_options() -> RecordingOptions

    recording_storage

    fn recording_storage(create : (Int) -> WriteSink raise McapError, open : (Int) -> ReadSource raise McapError, exists : (Int) -> Bool raise McapError, seal : (Int) -> Unit raise McapError, save : (Bytes) -> Unit raise McapError, quarantine : (Int) -> Unit raise McapError, list : () -> Array[Int] raise McapError, fingerprint : (ReadSource) -> String raise McapError) -> RecordingStorage

    recover_blocks

    fn recover_blocks(source : ReadSource, sink : WriteSink, limits? : Limits, options? : StreamOptions, summary_bytes? : Int, on_block? : (BlockRecoveryEvidence) -> Unit raise McapError) -> BlockRecoveryReport raise McapError

    Explicit partial extraction, never a claim that all source records survived. Requires a nonzero verified summary CRC and a CRC on each accepted Chunk.

    recover_blocks_detailed

    fn recover_blocks_detailed(source : ReadSource, sink : WriteSink, limits? : Limits, options? : StreamOptions, summary_bytes? : Int, max_ranges? : Int) -> Json raise McapError

    recover_prefix

    fn recover_prefix(bytes : Bytes, limits? : Limits) -> RecoveryReport raise McapError

    Recover only a verified contiguous prefix. Never resynchronize through corrupt bytes.

    repair_prefix

    fn repair_prefix(bytes : Bytes, limits? : Limits) -> (Bytes, RecoveryReport) raise McapError

    Rewrites a recoverable prefix as a fresh valid sequential MCAP file.

    replay

    fn replay(source : QuerySource, query? : Query, rate_numerator? : Int, rate_denominator? : Int, pending_bytes? : Int) -> Replay raise McapError

    resumable_chunk_merge

    fn resumable_chunk_merge(sources : Array[ReadSource], parts_storage : SortStorage, merge_storage : SortStorage, output : WriteSink, fingerprint : (ReadSource) -> String raise McapError, save : (Bytes) -> Unit raise McapError, checkpoint? : Bytes, options? : ExternalOptions, max_parts? : Int, cancelled? : () -> Bool) -> Json raise McapError

    resumable_chunk_pipeline

    fn resumable_chunk_pipeline(sources : Array[ReadSource], storage : SortStorage, output : WriteSink, fingerprint : (ReadSource) -> String raise McapError, save : (Bytes) -> Unit raise McapError, checkpoint? : Bytes, options? : ExternalOptions, max_parts? : Int, cancelled? : () -> Bool) -> Json raise McapError

    resumable_merge

    fn resumable_merge(sources : Array[ReadSource], storage : SortStorage, output : WriteSink, fingerprint : (ReadSource) -> String raise McapError, save : (Bytes) -> Unit raise McapError, checkpoint? : Bytes, options? : ExternalOptions, cancelled? : () -> Bool) -> ExternalReport raise McapError

    Resume only durable ingestion/merge boundaries. Caller atomically saves and fsyncs checkpoints and runs, owns a job lock, and removes unreferenced runs. Final output may be retried; committed runs remain until caller cleanup.

    ros1_decoder

    fn ros1_decoder(name : String, definition : Bytes, options? : CdrOptions) -> Ros1Decoder raise McapError

    scan_file

    fn scan_file(bytes : Bytes, limits? : Limits) -> Array[Record] raise McapError

    Decode the complete file envelope. Unknown records are available to callers.

    scan_source

    fn scan_source(source : ReadSource, visitor : SourceVisitor, limits? : Limits, declaration_bytes? : Int, cancelled? : () -> Bool, trace? : (SourceScanEvent) -> Unit) -> UInt64 raise McapError

    Bounded full validation in physical order. Callbacks precede final CRC validation; callers must discard side effects if validation fails.

    shift_time

    fn shift_time(log : Log, magnitude : UInt64, backwards? : Bool) -> Log raise McapError

    Shift both clocks by a signed magnitude without signed-integer narrowing.

    slice_log

    fn slice_log(log : Log, query : Query) -> Log raise McapError

    Select messages while pruning unreachable declarations and retaining metadata.

    sort_storage

    fn sort_storage(create : (Int) -> WriteSink raise McapError, open : (Int) -> ReadSource raise McapError, remove : (Int) -> Unit raise McapError, finish? : () -> Unit raise McapError) -> SortStorage

    source_visitor

    fn source_visitor(schema? : (Schema) -> Unit raise McapError, channel? : (Channel) -> Unit raise McapError, message? : (Message, MessageLocation) -> Unit raise McapError, attachment? : (Attachment) -> Unit raise McapError, metadata? : (Metadata) -> Unit raise McapError) -> SourceVisitor

    standard_payload_decoder

    fn standard_payload_decoder(options? : ProtobufOptions, cdr? : CdrOptions, cache_bytes? : Int, max_schemas? : Int, selection? : DecodeSelection) -> PayloadDecoder raise McapError

    Schema identity includes descriptor bytes, not just a source-local schema ID.

    start_disk_sidecar_audit

    fn start_disk_sidecar_audit(source : ReadSource, index : ReadSource, verification : IndexVerification, limits? : Limits, declaration_bytes? : Int, cache_pages? : Int, chunk_cache_bytes? : Int, index_bytes? : UInt64) -> DiskSidecarAudit raise McapError

    Headers/source envelopes are read synchronously here; pages are fed by poll.

    stats_report

    fn stats_report(stats : LogStats) -> String

    Human-readable report without converting timestamps to floating-point numbers.

    stream_options

    fn stream_options() -> StreamOptions

    stream_transform

    fn stream_transform(source : ReadSource, storage : SortStorage, output : WriteSink, options : TransformOptions, external? : ExternalOptions, content? : JsonFilter, cancelled? : () -> Bool) -> TransformReport raise McapError

    Multi-pass bounded transformation. Sorting, selected messages and merge runs share the same physical temporary-space budget. No historical Message array.

    summarize

    fn summarize(log : Log) -> LogStats raise McapError

    temporal_join

    fn temporal_join(left : MessageStream, right : MessageStream, options? : JoinOptions) -> TemporalJoinStream raise McapError

    temporal_join_cursor

    fn temporal_join_cursor(options? : JoinOptions) -> TemporalJoinCursor raise McapError

    transform_log

    fn transform_log(log : Log, options : TransformOptions) -> Log raise McapError

    Filters and samples before pagination. Renames are simultaneous and retain channel IDs. Sampling selects the first physical message in each time bucket, separately per channel.

    transform_options

    fn transform_options() -> TransformOptions

    transform_report_json

    fn transform_report_json(report : TransformReport) -> Json

    trigger_options

    fn trigger_options() -> TriggerOptions

    trigger_recorder

    fn trigger_recorder(template : Log, emit : (Log, Json) -> Unit raise McapError, options? : TriggerOptions) -> TriggerRecorder raise McapError

    The callback is synchronous backpressure: feed/trigger/finish do not return until output is accepted. Buffers are volatile, unlike RecordingSession.

    validate_attachment_indexes

    fn validate_attachment_indexes(bytes : Bytes, limits? : Limits) -> Int raise McapError

    validate_file_indexes

    fn validate_file_indexes(bytes : Bytes, limits? : Limits) -> Array[ChunkIndex] raise McapError

    Check physical index pointers against record boundaries and chunk contents.

    validate_file_statistics

    fn validate_file_statistics(log : Log, value : FileStatistics, chunk_count : UInt) -> Unit raise McapError

    validate_log

    fn validate_log(log : Log, limits? : Limits) -> Unit raise McapError

    validate_message_index

    fn validate_message_index(index : MessageIndex, chunk : Chunk) -> Unit raise McapError

    Verify index tuples refer to actual Message records inside the same chunk.

    validate_metadata_indexes

    fn validate_metadata_indexes(bytes : Bytes, limits? : Limits) -> Int raise McapError

    validate_summary

    fn validate_summary(bytes : Bytes, limits? : Limits) -> Int raise McapError

    Check grouped summary records and every declared SummaryOffset span.

    verify_crc

    fn verify_crc(bytes : Bytes, expected : UInt, label? : String) -> Unit raise McapError

    worker_chunk_properties

    fn worker_chunk_properties(body : Bytes, limits? : Limits) -> (Int, String) raise McapError

    Validated response bound from the original Chunk envelope, before expansion.

    write_chunked

    fn write_chunked(log : Log, messages_per_chunk? : Int, checksums? : Bool, compression? : Compression, limits? : Limits) -> Bytes raise McapError

    write_indexed

    fn write_indexed(log : Log, messages_per_chunk? : Int, compression? : Compression, limits? : Limits) -> Bytes raise McapError

    Serialize MessageIndex, ChunkIndex, grouped summary and SummaryOffset records.

    write_log

    fn write_log(log : Log, checksums? : Bool, limits? : Limits) -> Bytes raise McapError

    Minimal interoperable layout; zero summary pointers mean a sequential file.