🥮 mooncakes.io

    moonjournalkit

    Deterministic transactional write-ahead log framing and crash recovery for MoonBit.

    write-ahead-log
    crash-recovery
    journal
    crc32c
    storage-engine
    Download zip
    Author
    Version
    0.2.0
    License
    Apache-2.0
    Repository
    Last updated
    3 months ago
    Downloads
    44

    No dependencies

    #MoonJournalKit

    面向 MoonBit 的事务预写日志（WAL）格式、崩溃恢复与校验基础库。

    MoonJournalKit 不绑定文件系统、数据库或异步运行时。调用方负责持久化字节， 本库负责生成稳定记录、识别可信边界，并且只重放已经明确提交的事务。因此同一套 核心代码可以运行在 Wasm、Wasm GC、JavaScript 和 Native 后端。

    #主要能力

    • 24 字节固定头部的 MWAL v1 二进制记录格式
    • CRC32C（Castagnoli）逐记录完整性校验及增量接口
    • Begin / Put / Delete / Commit / Abort / Checkpoint 事务语义
    • 截断尾部、校验损坏、版本不支持、序列异常的精确区分
    • 只重放已提交事务的确定性恢复计划
    • 检查点裁剪、日志分段轮转、分段链连续性验证
    • 稳定 JSON 诊断结果，便于 CI 和运维系统接入
    • 逐字节断电切断和逐比特破坏的穷举验证

    #安装

    moon add Sxy11112/moonjournalkit

    #快速示例

    let records = [
      @journal.JournalRecord::new(Begin, 1U, 7U),
      @journal.JournalRecord::new(
        Put,
        2U,
        7U,
        payload=@journal.encode_put_payload(b"user/7", b"active"),
      ),
      @journal.JournalRecord::new(Commit, 3U, 7U),
    ]

    let bytes = @journal.encode_records(records)
    let scan = @journal.scan_journal(bytes)
    let plan = @journal.build_recovery_plan(scan.records)
    assert_eq(plan.actions.length(), 1)

    发生断电时，应先用 scan.valid_bytes 截取可信前缀，再执行恢复计划。 缺失 Commit 的事务会出现在 incomplete_transactions 中，但不会进入 actions。

    #可复现验证

    moon check --target all
moon test --target wasm
moon test --target wasm-gc
moon run cmd/main
moon run bench/main

    当前测试覆盖标准 CRC32C 向量、二进制往返、损坏和截断、事务交错、检查点、 分段轮转以及崩溃注入。CLI 演示会输出四组 JSON 证据。基准工作负载包含 10,000 个事务、30,000 条记录，并对一个 100 事务样本检查所有字节切断点。

    #设计边界

    本项目提供可嵌入的日志格式和恢复算法，不直接执行 fsync，也不替调用方决定 文件命名、目录布局、刷盘策略或并发模型。这样的边界既避免平台 API 污染，也让 浏览器持久化、对象存储、本地文件和自定义块设备能够共享同一恢复语义。

    格式细节见 docs/FORMAT.md，崩溃安全约束见 docs/SAFETY.md，社区项目差异见 docs/RELATED_WORK.md。

    许可证：Apache-2.0。

    CorruptionSweepReport

    source
    pub(all) struct CorruptionSweepReport {
      bits_checked : Int
      corruptions_detected : Int
      undetected : Int
      passed : Bool
    } derive(Eq, 
    Debug    )

    Aggregate evidence from flipping every individual bit in a byte stream.

    CorruptionSweepReport::to_json

    source
    fn CorruptionSweepReport::to_json(self : CorruptionSweepReport) -> String

    Stable machine-readable corruption verification evidence.

    CrashSweepReport

    source
    pub(all) struct CrashSweepReport {
      cuts_checked : Int
      exact_prefixes : Int
      unsafe_replays : Int
      passed : Bool
    } derive(Eq, 
    Debug    )

    Aggregate evidence from cutting a journal at every byte position.

    CrashSweepReport::to_json

    source
    fn CrashSweepReport::to_json(self : CrashSweepReport) -> String

    Stable machine-readable crash-cut verification evidence.

    DecodeResult

    source
    pub(all) struct DecodeResult {
      status : DecodeStatus
      record : JournalRecord?
      next_offset : Int
      message : String
    } derive(Eq, 
    Debug    )

    Result of decoding one record at a byte offset.

    DecodeStatus

    source
    pub(all) enum DecodeStatus {
      Decoded
      NeedMoreData
      Corrupt
      Unsupported
    } derive(Eq, 
    Debug    )

    Outcome category for bounded, non-throwing record decoding.

    JournalRecord

    source
    pub(all) struct JournalRecord {
      kind : RecordKind
      sequence : UInt
      transaction : UInt
      payload : Bytes
    } derive(Eq, 
    Debug    )

    One decoded journal record.

    JournalRecord::encoded_size

    source
    fn JournalRecord::encoded_size(self : JournalRecord) -> Int

    Encoded byte size for this record.

    JournalRecord::new

    source
    fn JournalRecord::new(kind : RecordKind, sequence : UInt, transaction : UInt, payload? : Bytes) -> JournalRecord

    Creates one journal record.

    Mutation

    source
    pub(all) struct Mutation {
      transaction : UInt
      sequence : UInt
      kind : MutationKind
      key : Bytes
      value : Bytes
    } derive(Eq, 
    Debug    )

    One key mutation associated with its transaction and source sequence.

    MutationDecode

    source
    pub(all) struct MutationDecode {
      valid : Bool
      mutation : Mutation?
      message : String
    } derive(Eq, 
    Debug    )

    Bounded result of decoding a put or delete payload.

    MutationKind

    source
    pub(all) enum MutationKind {
      PutValue
      DeleteKey
    } derive(Eq, 
    Debug    )

    A committed state change reconstructed from the journal.

    RecordKind

    source
    pub(all) enum RecordKind {
      Begin
      Put
      Delete
      Commit
      Abort
      Checkpoint
    } derive(Eq, 
    Debug    )

    Semantic record kinds supported by the transactional journal.

    RecoveryIssue

    source
    pub(all) struct RecoveryIssue {
      code : String
      sequence : UInt
      transaction : UInt
      message : String
    } derive(Eq, 
    Debug    )

    A semantic journal problem that does not compromise byte-level framing.

    RecoveryPlan

    source
    pub(all) struct RecoveryPlan {
      actions : Array[Mutation]
      committed_transactions : Int
      aborted_transactions : Int
      incomplete_transactions : Array[UInt]
      checkpoint_sequence : UInt
      issues : Array[RecoveryIssue]
      recoverable : Bool
    } derive(Eq, 
    Debug    )

    Deterministic replay plan produced from a trusted record prefix.

    RecoveryPlan::to_json

    source
    fn RecoveryPlan::to_json(self : RecoveryPlan) -> String

    Stable machine-readable recovery summary and diagnostics.

    ScanResult

    source
    pub(all) struct ScanResult {
      records : Array[JournalRecord]
      valid_bytes : Int
      total_bytes : Int
      stop : ScanStop
      fault_offset : Int
      message : String
    } derive(Eq, 
    Debug    )

    Result of scanning a byte stream up to its last trusted boundary.

    ScanResult::to_json

    source
    fn ScanResult::to_json(self : ScanResult) -> String

    Stable machine-readable scan summary.

    ScanResult::valid_prefix

    source
    fn ScanResult::valid_prefix(self : ScanResult, data : BytesView) -> Bytes

    Returns the trusted prefix suitable for crash-tail truncation.

    ScanStop

    source
    pub(all) enum ScanStop {
      CleanEnd
      TruncatedTail
      Corruption
      UnsupportedVersion
      SequenceViolation
      RecordLimit
    } derive(Eq, 
    Debug    )

    Why a journal scan stopped.

    SegmentMeta

    source
    pub(all) struct SegmentMeta {
      id : UInt
      first_sequence : UInt
      last_sequence : UInt
      record_count : Int
      byte_size : Int
    } derive(Eq, 
    Debug    )

    Immutable metadata for one planned journal segment.

    SegmentPlan

    source
    pub(all) struct SegmentPlan {
      segments : Array[SegmentMeta]
      total_records : Int
      total_bytes : Int
      valid : Bool
      message : String
    } derive(Eq, 
    Debug    )

    Result of deterministic journal rotation planning.

    StreamFeedResult

    source
    pub(all) struct StreamFeedResult {
      records : Array[JournalRecord]
      buffered_bytes : Int
      stop : ScanStop?
      message : String
    } derive(Eq, 
    Debug    )

    Result of one incremental decode operation.

    StreamJournalDecoder

    source
    pub(all) struct StreamJournalDecoder {
      strict_sequence : Bool
      max_payload : Int
      max_buffered : Int
      pending : Bytes
      previous_sequence : UInt
    } derive(Eq, 
    Debug    )

    A bounded incremental decoder for append-only journal byte streams.

    Feed arbitrary chunks from an I/O adapter. Complete records are emitted once, while a partial final record remains buffered for the next chunk. The core never opens files or browser storage itself.

    StreamJournalDecoder::buffered_bytes

    source
    fn StreamJournalDecoder::buffered_bytes(self : StreamJournalDecoder) -> Int

    StreamJournalDecoder::feed

    source
    fn StreamJournalDecoder::feed(self : StreamJournalDecoder, chunk : BytesView) -> StreamFeedResult

    Decodes complete records without retaining previously emitted records.

    StreamJournalDecoder::new

    source
    fn StreamJournalDecoder::new(strict_sequence? : Bool, max_payload? : Int, max_buffered? : Int) -> StreamJournalDecoder

    StreamJournalDecoder::reset

    source
    fn StreamJournalDecoder::reset(self : StreamJournalDecoder) -> Unit

    JOURNAL_HEADER_SIZE

    source
    let JOURNAL_HEADER_SIZE : Int

    Fixed header size in bytes.

    JOURNAL_VERSION

    source
    let JOURNAL_VERSION : Byte

    Fixed wire-format version.

    build_recovery_plan

    source
    fn build_recovery_plan(records : Array[JournalRecord], max_key? : Int) -> RecoveryPlan

    Builds a replay plan. Only mutations from explicitly committed transactions are returned; aborted and crash-incomplete transactions never leak through.

    crc32c

    source
    fn crc32c(data : BytesView) -> UInt

    Computes CRC32C (Castagnoli) with the reflected polynomial.

    crc32c_extend

    source
    fn crc32c_extend(state : UInt, data : BytesView) -> UInt

    Incrementally updates a CRC32C state before final xor.

    decode_mutation

    source
    fn decode_mutation(record : JournalRecord, max_key? : Int) -> MutationDecode

    Decodes a put or delete record without allocating before bounds checks pass.

    decode_record

    source
    fn decode_record(data : BytesView, offset? : Int, max_payload? : Int) -> DecodeResult

    Decodes one record without reading beyond available bytes.

    encode_checkpoint_payload

    source
    fn encode_checkpoint_payload(sequence : UInt) -> Bytes

    Encodes the highest sequence known to be durable in a checkpoint payload.

    encode_delete_payload

    source
    fn encode_delete_payload(key : BytesView) -> Bytes

    Encodes a delete payload as key length followed by key bytes.

    encode_put_payload

    source
    fn encode_put_payload(key : BytesView, value : BytesView) -> Bytes

    Encodes a put payload as key length, key bytes, then value bytes.

    encode_record

    source
    fn encode_record(record : JournalRecord) -> Bytes

    Encodes one checksummed record in the portable MWAL format.

    encode_records

    source
    fn encode_records(records : Array[JournalRecord]) -> Bytes

    Encodes records in input order into one journal byte stream.

    plan_segments

    source
    fn plan_segments(records : Array[JournalRecord], max_bytes : Int, max_records : Int, first_id? : UInt) -> SegmentPlan

    Plans segment rotation without splitting a record. A single oversized record receives its own segment so that forward progress remains deterministic.

    record_kind_name

    source
    fn record_kind_name(kind : RecordKind) -> String

    Stable lower-case name used by reports and diagnostics.

    recovery_segment_index

    source
    fn recovery_segment_index(segments : Array[SegmentMeta], checkpoint_sequence : UInt) -> Int

    Returns the first segment that may contain records newer than a checkpoint. A return value equal to segment count means no segment needs replay.

    scan_journal

    source
    fn scan_journal(data : BytesView, strict_sequence? : Bool, max_payload? : Int, max_records? : Int) -> ScanResult

    Scans records without accepting bytes after the first invalid boundary.

    sweep_crash_cuts

    source
    fn sweep_crash_cuts(records : Array[JournalRecord]) -> CrashSweepReport

    Exhaustively cuts the encoded journal after every byte and validates that scanning returns an exact record prefix and recovery never invents a commit.

    sweep_single_bit_corruption

    source
    fn sweep_single_bit_corruption(records : Array[JournalRecord]) -> CorruptionSweepReport

    Flips each bit independently and verifies that framing, checksums, or sequence validation reject every modified stream.

    validate_segment_chain

    source
    fn validate_segment_chain(segments : Array[SegmentMeta]) -> Bool

    Validates IDs, sequence continuity, counts, and non-empty segment metadata.

    Source Files

    Powered by MoonBit

    Site sourceReport issueModulesSkillsCLIBuild queueStatistics

    © 2026 mooncakes.io