🥮 mooncakes.io

moon-avro

Pure MoonBit Apache Avro schema, datum codec, and Object Container File toolkit.

avro
serialization
ocf
codec
data
moon add yugonlian/moon-avro@0.3.0
Download zip
Author
Version
0.3.0
License
Apache-2.0
Repository
Last updated
3 hours ago
Downloads
22

Dependencies

README

#moon-avro

纯 MoonBit 的 Apache Avro 工具包，覆盖 Schema、通用 datum 二进制编解码、 Single-Object Encoding、schema-driven JSON 映射和 OCF 容器（null/raw deflate）。核心 API 不依赖文件系统，输入错误会以结构化错误返回，并提供 递归、集合和字节大小限制。

#已实现能力

  • Schema：primitive、record、enum、array、map、union、fixed；namespace/fullname、 aliases、命名引用、默认值校验、Parsing Canonical Form 和 CRC-64-AVRO fingerprint；
  • Binary：zigzag/varint、little-endian float/double、UTF-8、严格尾随数据检查、 byte offset 诊断与 CodecLimits；
  • Datum：所有核心 primitive 及 record/enum/array/map/union/fixed 递归编解码；
  • OCF：header metadata、sync marker、按记录数/字节目标自动多块写入、null 与 标准 raw DEFLATE，各块可独立解压并校验；
  • Single-Object Encoding：C3 01 marker、小端 CRC-64-AVRO fingerprint、 expected-schema 校验和可注入的 schema resolver；
  • JSON：Avro JSON 约定、union 单键表示、record aliases/defaults、精确 Int64；
  • CLI：schema inspect、encode、decode、ocf-info 和 roundtrip。

不包含 Avro RPC、Schema Registry、代码生成、Snappy 和 schema evolution reader。 OCF 当前 API 提供有序的内存 reader/writer 和有界多数据块 framing；大文件文件流 适配器不是本版本承诺。

#快速开始

moon check --target all --deny-warn
moon test --target all --deny-warn
moon run examples/basic
moon run examples/primitive
moon run examples/ocf
moon run cmd/moon_avro -- roundtrip
moon run cmd/moon_avro -- encode '"long"' '42'  # 输出 54
moon run cmd/moon_avro -- decode '"long"' '54' # 输出 42

在其他 MoonBit 项目中安装：

moon add yugonlian/moon-avro@0.3.0

然后导入根包或子包：

import { "yugonlian/moon-avro" @avro }

#验收与许可证

bash scripts/acceptance.sh 会更新注册表索引并执行格式、四目标 check/build/test、 示例和 CLI smoke；发布后可用 bash scripts/external-consumer-smoke.sh 验证独立消费者安装。 可用 bash scripts/benchmark.sh 100 获取本机趋势基线。 实现依据 Apache Avro Specification， 项目采用 Apache-2.0。

#
CodecError

source

Public facade for datum codec APIs.

#
CodecLimits

source

Public facade for datum codec APIs.

#
CompressionCodec

source

Public facade for Object Container File APIs.

#
Datum

source

Public facade for datum codec APIs.

#
Decoder

source

Public facade for datum codec APIs.

#
Encoder

source

Public facade for datum codec APIs.

#
Field

source

Public facade for schema APIs.

#
JsonError

source

Public facade for schema-driven Avro JSON datum conversion.

#
JsonLimits

source

Public facade for schema-driven Avro JSON datum conversion.

#
NamedDefinition

source

Public facade for schema APIs.

#
OcfBlockConfig

source

Public facade for Object Container File APIs.

#
OcfError

source

Public facade for Object Container File APIs.

#
OcfHeader

source

Public facade for Object Container File APIs.

#
OcfLimits

source

Public facade for Object Container File APIs.

#
OcfReader

source

Public facade for Object Container File APIs.

#
OcfWriter

source

Public facade for Object Container File APIs.

#
Primitive

source

Public facade for schema APIs.

#
Schema

source

Public facade for schema APIs.

#
SchemaError

source

Public facade for schema APIs.

#
SchemaKind

source

Public facade for schema APIs.

#
SingleObjectError

source

Public error type for Avro Single-Object Encoding operations.

#
decode

source

Decode one datum according to an Avro schema and reject trailing bytes.

#
decode_container

source

Decode an entire OCF file, checking block sizes, record counts, sync markers and all datum payloads against the schema carried by its header.

#
decode_from

source

Decode one datum from a caller-owned stream decoder, leaving later bytes available for the next datum.

#
decode_header

source

Decode only an OCF header. bytes_consumed is the first block offset.

#
decode_single_object

source

Decode one Single-Object Encoding value using an expected schema.

#
decode_single_object_resolved

source

Resolve a Single-Object Encoding writer schema by fingerprint and decode the enclosed datum.

#
default_block_config

source

A balanced default for ordinary in-memory files: at most 1,000 records or approximately 256 KiB of uncompressed payload per data block.

#
default_limits

source

Conservative defaults suitable for untrusted in-memory OCF input.

#
encode

source

Encode one datum according to an Avro schema.

#
encode_container

source

Encode one complete Avro Object Container File as a single data block. Use encode_container_blocks when output should be partitioned for bounded decompression and incremental interoperability.

#
encode_container_blocks

source

Encode one complete OCF file using independently compressed data blocks. Blocks are closed when either configured record count or target uncompressed byte size is reached. Record order is preserved.

#
encode_header

source

Serialize an OCF header (magic, metadata and 16-byte sync marker).

#
encode_into

source

Encode one datum into a caller-owned stream encoder.

#
encode_single_object

source

Encode one datum using the Avro Single-Object Encoding envelope.

#
from_json

source

Convert a JSON value to an Avro datum using the supplied schema.

#
is_ocf

source
fn is_ocf(data : Bytes) -> Bool

#
magic

source
fn magic() -> Bytes

Apache Avro Object Container File magic bytes.

#
parse

source

Parse an Avro JSON schema, resolve named references, and retain named declarations for later datum and OCF consumers.

#
parse_json

source

Parse JSON text into a datum without colliding with the schema parse API.

#
single_object_fingerprint

source
fn single_object_fingerprint(input : Bytes) -> UInt64 raise 
SingleObjectError

Extract the CRC-64-AVRO schema fingerprint from a single-object envelope.

#
stringify

source

Convert an Avro datum to compact JSON text according to its schema.

#
to_json

source

Convert an Avro datum to its schema-defined JSON value.

Source Files

Powered by MoonBit

Site sourceReport issuePackagesBuild queueSkillsStatistics

© 2026 mooncakes.io