moonchunk

    Content-defined chunking and incremental synchronization primitives for MoonBit

    chunking
    deduplication
    merkle
    sync
    storage
    Download zip
    Author
    Version
    0.1.0
    License
    Apache-2.0
    Last updated
    2 months ago
    Downloads
    33

    No dependencies

    #MoonChunk

    MoonChunk is a MoonBit-native content-defined chunking and incremental synchronization library. It turns a byte stream into stable, content-addressed chunks, builds a verifiable Merkle manifest, and produces a plan that reuses unchanged chunks between versions.

    Repository: https://github.com/xgzh1111/1234

    Module: xgzh1111/moonchunk

    #Why it exists

    Fixed-size blocks are simple, but inserting a few bytes near the beginning of a large file shifts every later block. A content-defined boundary follows the data instead of the absolute offset, so backups, build caches, edge replicas, and object stores can avoid retransmitting unchanged content.

    #Current capabilities

    • Configurable content-defined chunking with minimum, average, and maximum sizes.
    • Streaming chunker for fragmented input.
    • Dependency-free deterministic 256-bit content digest for addressing and corruption detection. It is not a cryptographic authentication primitive.
    • Canonical line-oriented Manifest format with root validation.
    • Merkle roots, inclusion proofs, and proof verification.
    • Manifest diffing and a reusable in-memory ChunkStore.
    • Restore and integrity reports that identify missing, corrupt, or incorrectly sized chunks.
    • Native executable example and MoonBit test suite.

    #Quick start

    Install the MoonBit toolchain, then run:

    moon check moon test --target js moon run cmd/main --target js

    Library usage:

    let input = "hello MoonChunk".to_bytes().to_array()
    let cfg = @moonchunk.ChunkerConfig::new(min_size=16, average_size=32, max_size=64)
    let chunks = @moonchunk.chunk_bytes(input, cfg~)
    let manifest = @moonchunk.manifest_from_chunks("hello.txt", chunks)
    let store = @moonchunk.ChunkStore::new()
    @moonchunk.store_chunks(store, chunks)
    let report = @moonchunk.verify_store(store, manifest)
    assert_true(report.passed())

    #What the example does

    The runnable example in cmd/main chunks a sample document, builds a manifest, stores the chunks in an in-memory store, and verifies that the stored content can be checked back against the manifest root.

    Expected output:

    MoonChunk demonstration source bytes: 250 chunks: 9 root: 7019a4e06126793bfd0a35c6654fbc33 valid: 9 chunks, 250 bytes

    #Build and test status

    • moon check passes
    • moon test --target js passes with 56/56 tests
    • moon run cmd/main --target js runs successfully
    • moon package --list produces the publishable package artifact
    • GitHub Actions runs check, test, and runnable example commands on every push

    On this Windows machine, MoonBit native runtime compilation currently fails in the toolchain C runtime (rand_s related compile error). The project remains fully verifiable through the JavaScript target locally and through CI on GitHub.

    #Design boundaries

    MoonChunk is a library and reference CLI, not a network protocol or a cloud storage service. Transport, authentication, encryption, and durable storage belong to the application embedding it. The public API is intentionally backend-neutral so the same model can be used from Native, JavaScript, or WebAssembly builds.

    #Acceptance evidence

    #Project layout

    FileResponsibility
    types.mbtPublic data model and synchronization result types
    digest.mbtDeterministic digest and Merkle primitives
    chunker.mbtBatch and streaming content-defined chunking
    manifest.mbtManifest construction and canonical encoding
    sync.mbtDiff plans, in-memory store, restore
    verify.mbtIntegrity reports and proof checks
    cmd/mainRunnable demonstration

    #License

    Apache-2.0. See LICENSE.

    BenchmarkInput

    pub(all) struct BenchmarkInput {
    name : String
    bytes : Array[Byte]
    description : String
    } derive(Show)
    Deterministic benchmark scenarios without wall-clock dependencies.

    The benchmark reports work in bytes and operations so it can be compared across Native, JavaScript, and WebAssembly targets in CI.

    BenchmarkInput::bytes

    fn BenchmarkInput::bytes(self : BenchmarkInput) -> Array[Byte]

    BenchmarkInput::description

    fn BenchmarkInput::description(self : BenchmarkInput) -> String

    BenchmarkInput::name

    fn BenchmarkInput::name(self : BenchmarkInput) -> String

    BenchmarkResult

    pub(all) struct BenchmarkResult {
    name : String
    input_bytes : Int
    chunks : Int
    digest_operations : Int
    manifest_bytes : Int
    restored_bytes : Int
    reused_bytes : Int
    uploaded_bytes : Int
    checksum : String
    } derive(Show)

    BenchmarkResult::checksum

    fn BenchmarkResult::checksum(self : BenchmarkResult) -> String

    BenchmarkResult::chunks

    fn BenchmarkResult::chunks(self : BenchmarkResult) -> Int

    BenchmarkResult::digest_ops

    fn BenchmarkResult::digest_ops(self : BenchmarkResult) -> Int

    BenchmarkResult::input

    fn BenchmarkResult::input(self : BenchmarkResult) -> Int

    BenchmarkResult::manifest_size

    fn BenchmarkResult::manifest_size(self : BenchmarkResult) -> Int

    BenchmarkResult::name

    fn BenchmarkResult::name(self : BenchmarkResult) -> String

    BenchmarkResult::restored

    fn BenchmarkResult::restored(self : BenchmarkResult) -> Int

    BenchmarkResult::reused

    fn BenchmarkResult::reused(self : BenchmarkResult) -> Int

    BenchmarkResult::uploaded

    fn BenchmarkResult::uploaded(self : BenchmarkResult) -> Int

    Chunk

    pub(all) struct Chunk {
    ordinal : Int
    span : Span
    digest : String
    data : Array[Byte]
    } derive(Show)
    A content-addressed block. The digest is represented as a stable hex string.

    Chunk::bytes

    fn Chunk::bytes(self : Chunk) -> Array[Byte]

    Chunk::id

    fn Chunk::id(self : Chunk) -> String

    Chunk::index

    fn Chunk::index(self : Chunk) -> Int

    Chunk::new

    fn Chunk::new(ordinal : Int, span : Span, data : Array[Byte]) -> Chunk

    Chunk::range

    fn Chunk::range(self : Chunk) -> Span

    Chunk::size

    fn Chunk::size(self : Chunk) -> Int

    ChunkProfile

    pub(all) struct ChunkProfile {
    total_bytes : Int
    chunk_count : Int
    mean_size : Int
    median_size : Int
    min_size : Int
    max_size : Int
    unique_count : Int
    duplicate_count : Int
    histogram : Array[Int]
    } derive(Show)
    Analysis helpers used by the CLI, examples, and benchmark reports.

    ChunkProfile::bytes

    fn ChunkProfile::bytes(self : ChunkProfile) -> Int

    ChunkProfile::count

    fn ChunkProfile::count(self : ChunkProfile) -> Int

    ChunkProfile::distribution

    fn ChunkProfile::distribution(self : ChunkProfile) -> Array[Int]

    ChunkProfile::duplicates

    fn ChunkProfile::duplicates(self : ChunkProfile) -> Int

    ChunkProfile::maximum

    fn ChunkProfile::maximum(self : ChunkProfile) -> Int

    ChunkProfile::mean

    fn ChunkProfile::mean(self : ChunkProfile) -> Int

    ChunkProfile::median

    fn ChunkProfile::median(self : ChunkProfile) -> Int

    ChunkProfile::minimum

    fn ChunkProfile::minimum(self : ChunkProfile) -> Int

    ChunkProfile::unique

    fn ChunkProfile::unique(self : ChunkProfile) -> Int

    ChunkRef

    pub(all) struct ChunkRef {
    position : Int
    digest : String
    size : Int
    } derive(Eq, Show)
    A reference to a stored chunk inside a manifest.

    ChunkRef::at

    fn ChunkRef::at(self : ChunkRef) -> Int

    ChunkRef::id

    fn ChunkRef::id(self : ChunkRef) -> String

    ChunkRef::length

    fn ChunkRef::length(self : ChunkRef) -> Int

    ChunkRef::new

    fn ChunkRef::new(position : Int, digest : String, size : Int) -> ChunkRef

    ChunkStore

    pub(all) struct ChunkStore {
    digests : Array[String]
    blocks : Array[Array[Byte]]
    inserted : Int
    hits : Int
    }

    ChunkStore::all_digests

    fn ChunkStore::all_digests(self : ChunkStore) -> Array[String]

    ChunkStore::clear

    fn ChunkStore::clear(self : ChunkStore) -> Unit

    ChunkStore::get

    fn ChunkStore::get(self : ChunkStore, digest : String) -> Array[Byte]?

    ChunkStore::has

    fn ChunkStore::has(self : ChunkStore, digest : String) -> Bool

    ChunkStore::hits

    fn ChunkStore::hits(self : ChunkStore) -> Int

    ChunkStore::insertions

    fn ChunkStore::insertions(self : ChunkStore) -> Int

    ChunkStore::len

    fn ChunkStore::len(self : ChunkStore) -> Int

    ChunkStore::new

    fn ChunkStore::new() -> ChunkStore

    ChunkStore::put

    fn ChunkStore::put(self : ChunkStore, chunk : Chunk) -> Bool

    ChunkStore::remove

    fn ChunkStore::remove(self : ChunkStore, digest : String) -> Bool

    ChunkerConfig

    pub(all) struct ChunkerConfig {
    min_size : Int
    average_size : Int
    max_size : Int
    seed : UInt
    } derive(Eq, Show)
    Parameters used by the rolling chunker.

    ChunkerConfig::average

    fn ChunkerConfig::average(self : ChunkerConfig) -> Int

    ChunkerConfig::maximum

    fn ChunkerConfig::maximum(self : ChunkerConfig) -> Int

    ChunkerConfig::minimum

    fn ChunkerConfig::minimum(self : ChunkerConfig) -> Int

    ChunkerConfig::new

    fn ChunkerConfig::new(min_size? : Int, average_size? : Int, max_size? : Int, seed? : UInt) -> ChunkerConfig

    ChunkerConfig::validate

    fn ChunkerConfig::validate(self : ChunkerConfig) -> Result[Unit, String]

    ChunkerConfig::with_seed

    fn ChunkerConfig::with_seed(self : ChunkerConfig, seed : UInt) -> ChunkerConfig

    ChunkerStats

    pub(all) struct ChunkerStats {
    input_bytes : Int
    chunk_count : Int
    smallest : Int
    largest : Int
    average : Int
    boundaries : Array[Int]
    } derive(Show)
    Content-defined chunking.

    ChunkerStats::boundary_offsets

    fn ChunkerStats::boundary_offsets(self : ChunkerStats) -> Array[Int]

    ChunkerStats::count

    fn ChunkerStats::count(self : ChunkerStats) -> Int

    ChunkerStats::empty

    fn ChunkerStats::empty() -> ChunkerStats

    ChunkerStats::input

    fn ChunkerStats::input(self : ChunkerStats) -> Int

    ChunkerStats::max_size

    fn ChunkerStats::max_size(self : ChunkerStats) -> Int

    ChunkerStats::mean

    fn ChunkerStats::mean(self : ChunkerStats) -> Int

    ChunkerStats::min_size

    fn ChunkerStats::min_size(self : ChunkerStats) -> Int

    Manifest

    pub(all) struct Manifest {
    name : String
    total_size : Int
    chunks : Array[ChunkRef]
    root : String
    format_version : Int
    } derive(Show)
    An immutable description of one version of a file.

    Manifest::count

    fn Manifest::count(self : Manifest) -> Int

    Manifest::filename

    fn Manifest::filename(self : Manifest) -> String

    Manifest::is_valid

    fn Manifest::is_valid(self : Manifest) -> Bool

    Manifest::new

    fn Manifest::new(name : String, chunks : Array[ChunkRef]) -> Manifest

    Manifest::references

    fn Manifest::references(self : Manifest) -> Array[ChunkRef]

    Manifest::root_digest

    fn Manifest::root_digest(self : Manifest) -> String

    Manifest::size

    fn Manifest::size(self : Manifest) -> Int

    Manifest::version

    fn Manifest::version(self : Manifest) -> Int

    Span

    pub(all) struct Span {
    start : Int
    end : Int
    } derive(Eq, Show)
    A half-open byte interval in the source stream.

    Span::is_empty

    fn Span::is_empty(self : Span) -> Bool

    Span::length

    fn Span::length(self : Span) -> Int

    Span::new

    fn Span::new(start : Int, end : Int) -> Span

    StorageReport

    pub(all) struct StorageReport {
    logical_bytes : Int
    physical_bytes : Int
    chunk_count : Int
    unique_chunks : Int
    deduplicated_bytes : Int
    manifest_bytes : Int
    } derive(Show)
    Inventory and accounting functions for storage dashboards.

    StorageReport::chunks

    fn StorageReport::chunks(self : StorageReport) -> Int

    StorageReport::logical

    fn StorageReport::logical(self : StorageReport) -> Int

    StorageReport::manifest

    fn StorageReport::manifest(self : StorageReport) -> Int

    StorageReport::physical

    fn StorageReport::physical(self : StorageReport) -> Int

    StorageReport::saved

    fn StorageReport::saved(self : StorageReport) -> Int

    StorageReport::unique

    fn StorageReport::unique(self : StorageReport) -> Int

    StreamingChunker

    pub(all) struct StreamingChunker {
    config : ChunkerConfig
    buffer : Array[Byte]
    total_seen : Int
    next_ordinal : Int
    rolling : UInt
    boundaries : Array[Int]
    }
    Incremental chunker for readers that arrive in arbitrary input fragments.

    StreamingChunker::bytes_seen

    fn StreamingChunker::bytes_seen(self : StreamingChunker) -> Int

    StreamingChunker::config

    StreamingChunker::finish

    Finish the stream and return the final partial chunk.

    StreamingChunker::finish_all

    fn StreamingChunker::finish_all(self : StreamingChunker) -> Array[Chunk]

    StreamingChunker::new

    StreamingChunker::pending_size

    fn StreamingChunker::pending_size(self : StreamingChunker) -> Int

    StreamingChunker::push

    fn StreamingChunker::push(self : StreamingChunker, fragment : Array[Byte]) -> Array[Chunk]
    Feed one fragment. It may return zero or more complete chunks.

    SyncOp

    pub(all) enum SyncOp {
    Reuse(digest~ : String, size~ : Int)
    Upload(digest~ : String, size~ : Int, source_index~ : Int)
    Remove(digest~ : String)
    } derive(Eq, Show)
    A single operation in an incremental synchronization plan.

    SyncPlan

    pub(all) struct SyncPlan {
    operations : Array[SyncOp]
    reused_bytes : Int
    uploaded_bytes : Int
    removed_chunks : Int
    } derive(Show)

    SyncPlan::new

    fn SyncPlan::new(operations : Array[SyncOp]) -> SyncPlan

    SyncPlan::operations

    fn SyncPlan::operations(self : SyncPlan) -> Array[SyncOp]

    SyncPlan::removed

    fn SyncPlan::removed(self : SyncPlan) -> Int

    SyncPlan::reused

    fn SyncPlan::reused(self : SyncPlan) -> Int

    SyncPlan::uploaded

    fn SyncPlan::uploaded(self : SyncPlan) -> Int

    VerificationReport

    pub(all) struct VerificationReport {
    result : VerifyResult
    checked_chunks : Int
    checked_bytes : Int
    warnings : Array[String]
    } derive(Show)
    Integrity checks and diagnostic reports.

    VerificationReport::bytes

    fn VerificationReport::bytes(self : VerificationReport) -> Int

    VerificationReport::checked

    fn VerificationReport::checked(self : VerificationReport) -> Int

    VerificationReport::passed

    fn VerificationReport::passed(self : VerificationReport) -> Bool

    VerificationReport::result

    VerificationReport::warnings

    fn VerificationReport::warnings(self : VerificationReport) -> Array[String]

    VerifyResult

    pub(all) enum VerifyResult {
    Valid
    InvalidManifest(reason~ : String)
    MissingChunk(digest~ : String, position~ : Int)
    CorruptChunk(expected~ : String, actual~ : String, position~ : Int)
    SizeMismatch(expected~ : Int, actual~ : Int)
    } derive(Eq, Show)
    A detailed verification result suitable for a CLI or a UI.

    VerifyResult::ok

    fn VerifyResult::ok(self : VerifyResult) -> Bool

    VersionComparison

    pub(all) struct VersionComparison {
    old_bytes : Int
    new_bytes : Int
    reused_bytes : Int
    uploaded_bytes : Int
    removed_chunks : Int
    reuse_ratio : Double
    upload_ratio : Double
    } derive(Show)

    VersionComparison::new_size

    fn VersionComparison::new_size(self : VersionComparison) -> Int

    VersionComparison::old_size

    fn VersionComparison::old_size(self : VersionComparison) -> Int

    VersionComparison::removed

    fn VersionComparison::removed(self : VersionComparison) -> Int

    VersionComparison::reuse_fraction

    fn VersionComparison::reuse_fraction(self : VersionComparison) -> Double

    VersionComparison::reused

    fn VersionComparison::reused(self : VersionComparison) -> Int

    VersionComparison::upload_fraction

    fn VersionComparison::upload_fraction(self : VersionComparison) -> Double

    VersionComparison::uploaded

    fn VersionComparison::uploaded(self : VersionComparison) -> Int

    add_storage_reports

    fn add_storage_reports(left : StorageReport, right : StorageReport) -> StorageReport

    apply_plan

    fn apply_plan(old_store : ChunkStore, new_store : ChunkStore, old_manifest : Manifest, new_manifest : Manifest) -> Result[Array[Byte], String]

    benchmark_all

    fn benchmark_all(config? : ChunkerConfig) -> Array[BenchmarkResult]

    benchmark_checksum_chain

    fn benchmark_checksum_chain(inputs : Array[BenchmarkInput]) -> String

    benchmark_deletion_overlap

    fn benchmark_deletion_overlap(config : ChunkerConfig) -> Int

    benchmark_inputs_are_deterministic

    fn benchmark_inputs_are_deterministic() -> Bool

    benchmark_insertion_overlap

    fn benchmark_insertion_overlap(config : ChunkerConfig) -> Int

    benchmark_mutation_overlap

    fn benchmark_mutation_overlap(config : ChunkerConfig) -> Int

    benchmark_report

    fn benchmark_report(results : Array[BenchmarkResult]) -> String

    benchmark_round_trip_ok

    fn benchmark_round_trip_ok(result : BenchmarkResult) -> Bool

    benchmark_scenarios

    fn benchmark_scenarios() -> Array[BenchmarkInput]

    benchmark_total_chunks

    fn benchmark_total_chunks(results : Array[BenchmarkResult]) -> Int

    benchmark_total_input

    fn benchmark_total_input(results : Array[BenchmarkResult]) -> Int

    benchmark_total_manifest_bytes

    fn benchmark_total_manifest_bytes(results : Array[BenchmarkResult]) -> Int

    boundary_gaps

    fn boundary_gaps(chunks : Array[Chunk]) -> Array[Int]

    boundary_jitter

    fn boundary_jitter(chunks : Array[Chunk]) -> Int

    chunk_boundaries

    fn chunk_boundaries(chunks : Array[Chunk]) -> Array[Int]

    chunk_bytes

    fn chunk_bytes(source : Array[Byte], cfg : ChunkerConfig) -> Array[Chunk]
    Split a complete byte array using content-defined boundaries.

    chunk_histogram

    fn chunk_histogram(chunks : Array[Chunk], buckets : Int) -> Array[Int]

    chunk_ids_with_positions

    fn chunk_ids_with_positions(chunks : Array[Chunk]) -> Array[(Int, String)]

    chunk_stats

    fn chunk_stats(chunks : Array[Chunk], input_bytes : Int) -> ChunkerStats

    chunk_total_size

    fn chunk_total_size(chunks : Array[Chunk]) -> Int

    collect_extra

    fn collect_extra(store : ChunkStore, manifest : Manifest) -> Array[String]

    collect_missing

    fn collect_missing(store : ChunkStore, manifest : Manifest) -> Array[ChunkRef]

    compare_boundaries

    fn compare_boundaries(left : Array[Chunk], right : Array[Chunk]) -> Int

    compare_versions

    fn compare_versions(old : Manifest, next : Manifest) -> VersionComparison

    comparison_report

    fn comparison_report(result : BenchmarkResult) -> String

    comparison_summary

    fn comparison_summary(comparison : VersionComparison) -> String

    content_overlap

    fn content_overlap(left : Array[Chunk], right : Array[Chunk]) -> Int

    decode_manifest

    fn decode_manifest(encoded : String) -> Result[Manifest, String]
    Parse the canonical format and validate the recomputed Merkle root.

    delete_region

    fn delete_region(input : BenchmarkInput, start : Int, length : Int) -> BenchmarkInput

    diff_manifests

    fn diff_manifests(old : Manifest, next : Manifest) -> SyncPlan

    digest_bytes

    fn digest_bytes(data : Array[Byte]) -> String

    digest_equal

    fn digest_equal(left : String, right : String) -> Bool

    digest_frequency

    fn digest_frequency(chunks : Array[Chunk]) -> Array[(String, Int)]

    digest_string

    fn digest_string(value : String) -> String

    duplicate_ratio

    fn duplicate_ratio(chunks : Array[Chunk]) -> Double

    empty_storage_report

    fn empty_storage_report() -> StorageReport

    encode_manifest

    fn encode_manifest(manifest : Manifest) -> String
    Canonical line-oriented encoding. It is intentionally simple to make the format inspectable in a hex editor and stable across MoonBit backends.

    estimated_savings

    fn estimated_savings(chunks : Array[Chunk]) -> Int

    estimated_storage

    fn estimated_storage(chunks : Array[Chunk]) -> Int

    format_histogram

    fn format_histogram(histogram : Array[Int]) -> String

    insert_region

    fn insert_region(input : BenchmarkInput, at : Int, inserted : Array[Byte]) -> BenchmarkInput

    inspect_storage

    fn inspect_storage(store : ChunkStore, manifest : Manifest) -> StorageReport

    inventory_digest

    fn inventory_digest(report : StorageReport) -> String

    inventory_equal

    fn inventory_equal(left : StorageReport, right : StorageReport) -> Bool

    manifest_chunk_sizes

    fn manifest_chunk_sizes(manifest : Manifest) -> Array[Int]

    manifest_contains

    fn manifest_contains(manifest : Manifest, digest : String) -> Bool

    manifest_digests

    fn manifest_digests(manifest : Manifest) -> Array[String]

    manifest_from_chunks

    fn manifest_from_chunks(name : String, chunks : Array[Chunk]) -> Manifest
    Manifest construction, canonical encoding and lookup helpers.

    manifest_from_digests

    fn manifest_from_digests(name : String, digests : Array[String], sizes : Array[Int]) -> Result[Manifest, String]

    manifest_has_all_chunks

    fn manifest_has_all_chunks(store : ChunkStore, manifest : Manifest) -> Bool

    manifest_has_no_extra_chunks

    fn manifest_has_no_extra_chunks(store : ChunkStore, manifest : Manifest) -> Bool

    manifest_lookup

    fn manifest_lookup(manifest : Manifest, digest : String) -> ChunkRef?

    manifest_position

    fn manifest_position(manifest : Manifest, digest : String) -> Int?

    manifest_prefix

    fn manifest_prefix(manifest : Manifest, count : Int) -> Manifest

    manifest_reorder

    fn manifest_reorder(manifest : Manifest, order : Array[Int]) -> Result[Manifest, String]

    manifest_same_content

    fn manifest_same_content(left : Manifest, right : Manifest) -> Bool

    manifest_suffix

    fn manifest_suffix(manifest : Manifest, start : Int) -> Manifest

    manifest_summary

    fn manifest_summary(manifest : Manifest) -> String

    manifest_validate_order

    fn manifest_validate_order(manifest : Manifest) -> Bool

    merkle_proof

    fn merkle_proof(digests : Array[String], index : Int) -> Array[(String, Bool)]

    merkle_root

    fn merkle_root(digests : Array[String]) -> String

    most_repeated

    fn most_repeated(chunks : Array[Chunk]) -> (String, Int)?

    mutate_region

    fn mutate_region(input : BenchmarkInput, start : Int, length : Int, seed : UInt) -> BenchmarkInput

    plan_removals

    fn plan_removals(plan : SyncPlan) -> Int

    plan_reuses

    fn plan_reuses(plan : SyncPlan) -> Int

    plan_uploads

    fn plan_uploads(plan : SyncPlan) -> Int

    profile_chunks

    fn profile_chunks(chunks : Array[Chunk], buckets? : Int) -> ChunkProfile

    profile_is_reasonable

    fn profile_is_reasonable(profile : ChunkProfile, config : ChunkerConfig) -> Bool

    profile_strings

    fn profile_strings(chunks : Array[Chunk], buckets : Int) -> String

    repetitive_input

    fn repetitive_input(name : String, size : Int, pattern : String) -> BenchmarkInput

    report_message

    fn report_message(report : VerificationReport) -> String

    restore_manifest

    fn restore_manifest(store : ChunkStore, manifest : Manifest) -> Result[Array[Byte], String]

    run_benchmark

    fn run_benchmark(input : BenchmarkInput, config : ChunkerConfig) -> BenchmarkResult

    run_comparison

    fn run_comparison(old : BenchmarkInput, next : BenchmarkInput, config : ChunkerConfig) -> BenchmarkResult

    size_percentile

    fn size_percentile(chunks : Array[Chunk], percentile : Int) -> Int

    storage_is_consistent

    fn storage_is_consistent(report : StorageReport) -> Bool

    storage_report_line

    fn storage_report_line(report : StorageReport) -> String

    storage_saving_ratio

    fn storage_saving_ratio(report : StorageReport) -> Double

    store_chunks

    fn store_chunks(store : ChunkStore, chunks : Array[Chunk]) -> Int

    store_logical_size

    fn store_logical_size(store : ChunkStore, manifest : Manifest) -> Int

    store_physical_size

    fn store_physical_size(store : ChunkStore, manifest : Manifest) -> Int

    sync_ratio

    fn sync_ratio(plan : SyncPlan, original_size : Int) -> Double

    sync_savings

    fn sync_savings(plan : SyncPlan, original_size : Int) -> Int

    synthetic_input

    fn synthetic_input(name : String, size : Int, seed : UInt) -> BenchmarkInput

    text_input

    fn text_input(name : String, paragraphs : Int) -> BenchmarkInput

    verify_bytes

    fn verify_bytes(data : Array[Byte], expected : String) -> Bool

    verify_manifest_root

    fn verify_manifest_root(manifest : Manifest) -> Bool

    verify_merkle_proof

    fn verify_merkle_proof(leaf : String, proof : Array[(String, Bool)], root : String) -> Bool

    verify_proof_for_manifest

    fn verify_proof_for_manifest(manifest : Manifest, position : Int) -> Bool

    verify_store

    fn verify_store(store : ChunkStore, manifest : Manifest) -> VerificationReport