moonchunk

Content-defined chunking and incremental synchronization primitives for MoonBit

chunking
deduplication
merkle
sync
storage
moon add xgzh1111/moonchunk@0.1.0
Download zip
Author
Version
0.1.0
License
Apache-2.0
Last updated
7 days ago
Downloads
2
README

#MoonChunk

MoonChunk is a MoonBit-native content-defined chunking and incremental synchronization library. It turns a byte stream into stable, content-addressed chunks, builds a verifiable Merkle manifest, and produces a plan that reuses unchanged chunks between versions.

Repository: https://github.com/xgzh1111/1234

Module: xgzh1111/moonchunk

#Why it exists

Fixed-size blocks are simple, but inserting a few bytes near the beginning of a large file shifts every later block. A content-defined boundary follows the data instead of the absolute offset, so backups, build caches, edge replicas, and object stores can avoid retransmitting unchanged content.

#Current capabilities

  • Configurable content-defined chunking with minimum, average, and maximum sizes.
  • Streaming chunker for fragmented input.
  • Dependency-free deterministic 256-bit content digest for addressing and corruption detection. It is not a cryptographic authentication primitive.
  • Canonical line-oriented Manifest format with root validation.
  • Merkle roots, inclusion proofs, and proof verification.
  • Manifest diffing and a reusable in-memory ChunkStore.
  • Restore and integrity reports that identify missing, corrupt, or incorrectly sized chunks.
  • Native executable example and MoonBit test suite.

#Quick start

Install the MoonBit toolchain, then run:

moon check moon test --target js moon run cmd/main --target js

Library usage:

let input = "hello MoonChunk".to_bytes().to_array()
let cfg = @moonchunk.ChunkerConfig::new(min_size=16, average_size=32, max_size=64)
let chunks = @moonchunk.chunk_bytes(input, cfg~)
let manifest = @moonchunk.manifest_from_chunks("hello.txt", chunks)
let store = @moonchunk.ChunkStore::new()
@moonchunk.store_chunks(store, chunks)
let report = @moonchunk.verify_store(store, manifest)
assert_true(report.passed())

#What the example does

The runnable example in cmd/main chunks a sample document, builds a manifest, stores the chunks in an in-memory store, and verifies that the stored content can be checked back against the manifest root.

Expected output:

MoonChunk demonstration source bytes: 250 chunks: 9 root: 7019a4e06126793bfd0a35c6654fbc33 valid: 9 chunks, 250 bytes

#Build and test status

  • moon check passes
  • moon test --target js passes with 56/56 tests
  • moon run cmd/main --target js runs successfully
  • moon package --list produces the publishable package artifact
  • GitHub Actions runs check, test, and runnable example commands on every push

On this Windows machine, MoonBit native runtime compilation currently fails in the toolchain C runtime (rand_s related compile error). The project remains fully verifiable through the JavaScript target locally and through CI on GitHub.

#Design boundaries

MoonChunk is a library and reference CLI, not a network protocol or a cloud storage service. Transport, authentication, encryption, and durable storage belong to the application embedding it. The public API is intentionally backend-neutral so the same model can be used from Native, JavaScript, or WebAssembly builds.

#Acceptance evidence

#Project layout

FileResponsibility
types.mbtPublic data model and synchronization result types
digest.mbtDeterministic digest and Merkle primitives
chunker.mbtBatch and streaming content-defined chunking
manifest.mbtManifest construction and canonical encoding
sync.mbtDiff plans, in-memory store, restore
verify.mbtIntegrity reports and proof checks
cmd/mainRunnable demonstration

#License

Apache-2.0. See LICENSE.

#
BenchmarkInput

pub(all) struct BenchmarkInput {
name : String
bytes : Array[Byte]
description : String
} derive(Show)
Deterministic benchmark scenarios without wall-clock dependencies.

The benchmark reports work in bytes and operations so it can be compared across Native, JavaScript, and WebAssembly targets in CI.

#
BenchmarkInput::bytes

fn BenchmarkInput::bytes(self : BenchmarkInput) -> Array[Byte]

#
BenchmarkInput::description

fn BenchmarkInput::description(self : BenchmarkInput) -> String

#
BenchmarkInput::name

fn BenchmarkInput::name(self : BenchmarkInput) -> String

#
BenchmarkResult

pub(all) struct BenchmarkResult {
name : String
input_bytes : Int
chunks : Int
digest_operations : Int
manifest_bytes : Int
restored_bytes : Int
reused_bytes : Int
uploaded_bytes : Int
checksum : String
} derive(Show)

#
BenchmarkResult::checksum

fn BenchmarkResult::checksum(self : BenchmarkResult) -> String

#
BenchmarkResult::chunks

fn BenchmarkResult::chunks(self : BenchmarkResult) -> Int

#
BenchmarkResult::digest_ops

fn BenchmarkResult::digest_ops(self : BenchmarkResult) -> Int

#
BenchmarkResult::input

fn BenchmarkResult::input(self : BenchmarkResult) -> Int

#
BenchmarkResult::manifest_size

fn BenchmarkResult::manifest_size(self : BenchmarkResult) -> Int

#
BenchmarkResult::name

fn BenchmarkResult::name(self : BenchmarkResult) -> String

#
BenchmarkResult::restored

fn BenchmarkResult::restored(self : BenchmarkResult) -> Int

#
BenchmarkResult::reused

fn BenchmarkResult::reused(self : BenchmarkResult) -> Int

#
BenchmarkResult::uploaded

fn BenchmarkResult::uploaded(self : BenchmarkResult) -> Int

#
Chunk

pub(all) struct Chunk {
ordinal : Int
span : Span
digest : String
data : Array[Byte]
} derive(Show)
A content-addressed block. The digest is represented as a stable hex string.

#
Chunk::bytes

fn Chunk::bytes(self : Chunk) -> Array[Byte]

#
Chunk::id

fn Chunk::id(self : Chunk) -> String

#
Chunk::index

fn Chunk::index(self : Chunk) -> Int

#
Chunk::new

fn Chunk::new(ordinal : Int, span : Span, data : Array[Byte]) -> Chunk

#
Chunk::range

fn Chunk::range(self : Chunk) -> Span

#
Chunk::size

fn Chunk::size(self : Chunk) -> Int

#
ChunkProfile

pub(all) struct ChunkProfile {
total_bytes : Int
chunk_count : Int
mean_size : Int
median_size : Int
min_size : Int
max_size : Int
unique_count : Int
duplicate_count : Int
histogram : Array[Int]
} derive(Show)
Analysis helpers used by the CLI, examples, and benchmark reports.

#
ChunkProfile::bytes

fn ChunkProfile::bytes(self : ChunkProfile) -> Int

#
ChunkProfile::count

fn ChunkProfile::count(self : ChunkProfile) -> Int

#
ChunkProfile::distribution

fn ChunkProfile::distribution(self : ChunkProfile) -> Array[Int]

#
ChunkProfile::duplicates

fn ChunkProfile::duplicates(self : ChunkProfile) -> Int

#
ChunkProfile::maximum

fn ChunkProfile::maximum(self : ChunkProfile) -> Int

#
ChunkProfile::mean

fn ChunkProfile::mean(self : ChunkProfile) -> Int

#
ChunkProfile::median

fn ChunkProfile::median(self : ChunkProfile) -> Int

#
ChunkProfile::minimum

fn ChunkProfile::minimum(self : ChunkProfile) -> Int

#
ChunkProfile::unique

fn ChunkProfile::unique(self : ChunkProfile) -> Int

#
ChunkRef

pub(all) struct ChunkRef {
position : Int
digest : String
size : Int
} derive(Eq, Show)
A reference to a stored chunk inside a manifest.

#
ChunkRef::at

fn ChunkRef::at(self : ChunkRef) -> Int

#
ChunkRef::id

fn ChunkRef::id(self : ChunkRef) -> String

#
ChunkRef::length

fn ChunkRef::length(self : ChunkRef) -> Int

#
ChunkRef::new

fn ChunkRef::new(position : Int, digest : String, size : Int) -> ChunkRef

#
ChunkStore

pub(all) struct ChunkStore {
digests : Array[String]
blocks : Array[Array[Byte]]
inserted : Int
hits : Int
}

#
ChunkStore::all_digests

fn ChunkStore::all_digests(self : ChunkStore) -> Array[String]

#
ChunkStore::clear

fn ChunkStore::clear(self : ChunkStore) -> Unit

#
ChunkStore::get

fn ChunkStore::get(self : ChunkStore, digest : String) -> Array[Byte]?

#
ChunkStore::has

fn ChunkStore::has(self : ChunkStore, digest : String) -> Bool

#
ChunkStore::hits

fn ChunkStore::hits(self : ChunkStore) -> Int

#
ChunkStore::insertions

fn ChunkStore::insertions(self : ChunkStore) -> Int

#
ChunkStore::len

fn ChunkStore::len(self : ChunkStore) -> Int

#
ChunkStore::new

fn ChunkStore::new() -> ChunkStore

#
ChunkStore::put

fn ChunkStore::put(self : ChunkStore, chunk : Chunk) -> Bool

#
ChunkStore::remove

fn ChunkStore::remove(self : ChunkStore, digest : String) -> Bool

#
ChunkerConfig

pub(all) struct ChunkerConfig {
min_size : Int
average_size : Int
max_size : Int
seed : UInt
} derive(Eq, Show)
Parameters used by the rolling chunker.

#
ChunkerConfig::average

fn ChunkerConfig::average(self : ChunkerConfig) -> Int

#
ChunkerConfig::maximum

fn ChunkerConfig::maximum(self : ChunkerConfig) -> Int

#
ChunkerConfig::minimum

fn ChunkerConfig::minimum(self : ChunkerConfig) -> Int

#
ChunkerConfig::new

fn ChunkerConfig::new(min_size? : Int, average_size? : Int, max_size? : Int, seed? : UInt) -> ChunkerConfig

#
ChunkerConfig::validate

fn ChunkerConfig::validate(self : ChunkerConfig) -> Result[Unit, String]

#
ChunkerConfig::with_seed

fn ChunkerConfig::with_seed(self : ChunkerConfig, seed : UInt) -> ChunkerConfig

#
ChunkerStats

pub(all) struct ChunkerStats {
input_bytes : Int
chunk_count : Int
smallest : Int
largest : Int
average : Int
boundaries : Array[Int]
} derive(Show)
Content-defined chunking.

#
ChunkerStats::boundary_offsets

fn ChunkerStats::boundary_offsets(self : ChunkerStats) -> Array[Int]

#
ChunkerStats::count

fn ChunkerStats::count(self : ChunkerStats) -> Int

#
ChunkerStats::empty

fn ChunkerStats::empty() -> ChunkerStats

#
ChunkerStats::input

fn ChunkerStats::input(self : ChunkerStats) -> Int

#
ChunkerStats::max_size

fn ChunkerStats::max_size(self : ChunkerStats) -> Int

#
ChunkerStats::mean

fn ChunkerStats::mean(self : ChunkerStats) -> Int

#
ChunkerStats::min_size

fn ChunkerStats::min_size(self : ChunkerStats) -> Int

#
Manifest

pub(all) struct Manifest {
name : String
total_size : Int
chunks : Array[ChunkRef]
root : String
format_version : Int
} derive(Show)
An immutable description of one version of a file.

#
Manifest::count

fn Manifest::count(self : Manifest) -> Int

#
Manifest::filename

fn Manifest::filename(self : Manifest) -> String

#
Manifest::is_valid

fn Manifest::is_valid(self : Manifest) -> Bool

#
Manifest::new

fn Manifest::new(name : String, chunks : Array[ChunkRef]) -> Manifest

#
Manifest::references

fn Manifest::references(self : Manifest) -> Array[ChunkRef]

#
Manifest::root_digest

fn Manifest::root_digest(self : Manifest) -> String

#
Manifest::size

fn Manifest::size(self : Manifest) -> Int

#
Manifest::version

fn Manifest::version(self : Manifest) -> Int

#
Span

pub(all) struct Span {
start : Int
end : Int
} derive(Eq, Show)
A half-open byte interval in the source stream.

#
Span::is_empty

fn Span::is_empty(self : Span) -> Bool

#
Span::length

fn Span::length(self : Span) -> Int

#
Span::new

fn Span::new(start : Int, end : Int) -> Span

#
StorageReport

pub(all) struct StorageReport {
logical_bytes : Int
physical_bytes : Int
chunk_count : Int
unique_chunks : Int
deduplicated_bytes : Int
manifest_bytes : Int
} derive(Show)
Inventory and accounting functions for storage dashboards.

#
StorageReport::chunks

fn StorageReport::chunks(self : StorageReport) -> Int

#
StorageReport::logical

fn StorageReport::logical(self : StorageReport) -> Int

#
StorageReport::manifest

fn StorageReport::manifest(self : StorageReport) -> Int

#
StorageReport::physical

fn StorageReport::physical(self : StorageReport) -> Int

#
StorageReport::saved

fn StorageReport::saved(self : StorageReport) -> Int

#
StorageReport::unique

fn StorageReport::unique(self : StorageReport) -> Int

#
StreamingChunker

pub(all) struct StreamingChunker {
config : ChunkerConfig
buffer : Array[Byte]
total_seen : Int
next_ordinal : Int
rolling : UInt
boundaries : Array[Int]
}
Incremental chunker for readers that arrive in arbitrary input fragments.

#
StreamingChunker::bytes_seen

fn StreamingChunker::bytes_seen(self : StreamingChunker) -> Int

#
StreamingChunker::config

#
StreamingChunker::finish

Finish the stream and return the final partial chunk.

#
StreamingChunker::finish_all

fn StreamingChunker::finish_all(self : StreamingChunker) -> Array[Chunk]

#
StreamingChunker::new

#
StreamingChunker::pending_size

fn StreamingChunker::pending_size(self : StreamingChunker) -> Int

#
StreamingChunker::push

fn StreamingChunker::push(self : StreamingChunker, fragment : Array[Byte]) -> Array[Chunk]
Feed one fragment. It may return zero or more complete chunks.

#
SyncOp

pub(all) enum SyncOp {
Reuse(digest~ : String, size~ : Int)
Upload(digest~ : String, size~ : Int, source_index~ : Int)
Remove(digest~ : String)
} derive(Eq, Show)
A single operation in an incremental synchronization plan.

#
SyncPlan

pub(all) struct SyncPlan {
operations : Array[SyncOp]
reused_bytes : Int
uploaded_bytes : Int
removed_chunks : Int
} derive(Show)

#
SyncPlan::new

fn SyncPlan::new(operations : Array[SyncOp]) -> SyncPlan

#
SyncPlan::operations

fn SyncPlan::operations(self : SyncPlan) -> Array[SyncOp]

#
SyncPlan::removed

fn SyncPlan::removed(self : SyncPlan) -> Int

#
SyncPlan::reused

fn SyncPlan::reused(self : SyncPlan) -> Int

#
SyncPlan::uploaded

fn SyncPlan::uploaded(self : SyncPlan) -> Int

#
VerificationReport

pub(all) struct VerificationReport {
result : VerifyResult
checked_chunks : Int
checked_bytes : Int
warnings : Array[String]
} derive(Show)
Integrity checks and diagnostic reports.

#
VerificationReport::bytes

fn VerificationReport::bytes(self : VerificationReport) -> Int

#
VerificationReport::checked

fn VerificationReport::checked(self : VerificationReport) -> Int

#
VerificationReport::passed

fn VerificationReport::passed(self : VerificationReport) -> Bool

#
VerificationReport::result

#
VerificationReport::warnings

fn VerificationReport::warnings(self : VerificationReport) -> Array[String]

#
VerifyResult

pub(all) enum VerifyResult {
Valid
InvalidManifest(reason~ : String)
MissingChunk(digest~ : String, position~ : Int)
CorruptChunk(expected~ : String, actual~ : String, position~ : Int)
SizeMismatch(expected~ : Int, actual~ : Int)
} derive(Eq, Show)
A detailed verification result suitable for a CLI or a UI.

#
VerifyResult::ok

fn VerifyResult::ok(self : VerifyResult) -> Bool

#
VersionComparison

pub(all) struct VersionComparison {
old_bytes : Int
new_bytes : Int
reused_bytes : Int
uploaded_bytes : Int
removed_chunks : Int
reuse_ratio : Double
upload_ratio : Double
} derive(Show)

#
VersionComparison::new_size

fn VersionComparison::new_size(self : VersionComparison) -> Int

#
VersionComparison::old_size

fn VersionComparison::old_size(self : VersionComparison) -> Int

#
VersionComparison::removed

fn VersionComparison::removed(self : VersionComparison) -> Int

#
VersionComparison::reuse_fraction

fn VersionComparison::reuse_fraction(self : VersionComparison) -> Double

#
VersionComparison::reused

fn VersionComparison::reused(self : VersionComparison) -> Int

#
VersionComparison::upload_fraction

fn VersionComparison::upload_fraction(self : VersionComparison) -> Double

#
VersionComparison::uploaded

fn VersionComparison::uploaded(self : VersionComparison) -> Int

#
add_storage_reports

fn add_storage_reports(left : StorageReport, right : StorageReport) -> StorageReport

#
apply_plan

fn apply_plan(old_store : ChunkStore, new_store : ChunkStore, old_manifest : Manifest, new_manifest : Manifest) -> Result[Array[Byte], String]

#
benchmark_all

fn benchmark_all(config? : ChunkerConfig) -> Array[BenchmarkResult]

#
benchmark_checksum_chain

fn benchmark_checksum_chain(inputs : Array[BenchmarkInput]) -> String

#
benchmark_deletion_overlap

fn benchmark_deletion_overlap(config : ChunkerConfig) -> Int

#
benchmark_inputs_are_deterministic

fn benchmark_inputs_are_deterministic() -> Bool

#
benchmark_insertion_overlap

fn benchmark_insertion_overlap(config : ChunkerConfig) -> Int

#
benchmark_mutation_overlap

fn benchmark_mutation_overlap(config : ChunkerConfig) -> Int

#
benchmark_report

fn benchmark_report(results : Array[BenchmarkResult]) -> String

#
benchmark_round_trip_ok

fn benchmark_round_trip_ok(result : BenchmarkResult) -> Bool

#
benchmark_scenarios

fn benchmark_scenarios() -> Array[BenchmarkInput]

#
benchmark_total_chunks

fn benchmark_total_chunks(results : Array[BenchmarkResult]) -> Int

#
benchmark_total_input

fn benchmark_total_input(results : Array[BenchmarkResult]) -> Int

#
benchmark_total_manifest_bytes

fn benchmark_total_manifest_bytes(results : Array[BenchmarkResult]) -> Int

#
boundary_gaps

fn boundary_gaps(chunks : Array[Chunk]) -> Array[Int]

#
boundary_jitter

fn boundary_jitter(chunks : Array[Chunk]) -> Int

#
chunk_boundaries

fn chunk_boundaries(chunks : Array[Chunk]) -> Array[Int]

#
chunk_bytes

fn chunk_bytes(source : Array[Byte], cfg : ChunkerConfig) -> Array[Chunk]
Split a complete byte array using content-defined boundaries.

#
chunk_histogram

fn chunk_histogram(chunks : Array[Chunk], buckets : Int) -> Array[Int]

#
chunk_ids_with_positions

fn chunk_ids_with_positions(chunks : Array[Chunk]) -> Array[(Int, String)]

#
chunk_stats

fn chunk_stats(chunks : Array[Chunk], input_bytes : Int) -> ChunkerStats

#
chunk_total_size

fn chunk_total_size(chunks : Array[Chunk]) -> Int

#
collect_extra

fn collect_extra(store : ChunkStore, manifest : Manifest) -> Array[String]

#
collect_missing

fn collect_missing(store : ChunkStore, manifest : Manifest) -> Array[ChunkRef]

#
compare_boundaries

fn compare_boundaries(left : Array[Chunk], right : Array[Chunk]) -> Int

#
compare_versions

fn compare_versions(old : Manifest, next : Manifest) -> VersionComparison

#
comparison_report

fn comparison_report(result : BenchmarkResult) -> String

#
comparison_summary

fn comparison_summary(comparison : VersionComparison) -> String

#
content_overlap

fn content_overlap(left : Array[Chunk], right : Array[Chunk]) -> Int

#
decode_manifest

fn decode_manifest(encoded : String) -> Result[Manifest, String]
Parse the canonical format and validate the recomputed Merkle root.

#
delete_region

fn delete_region(input : BenchmarkInput, start : Int, length : Int) -> BenchmarkInput

#
diff_manifests

fn diff_manifests(old : Manifest, next : Manifest) -> SyncPlan

#
digest_bytes

fn digest_bytes(data : Array[Byte]) -> String

#
digest_equal

fn digest_equal(left : String, right : String) -> Bool

#
digest_frequency

fn digest_frequency(chunks : Array[Chunk]) -> Array[(String, Int)]

#
digest_string

fn digest_string(value : String) -> String

#
duplicate_ratio

fn duplicate_ratio(chunks : Array[Chunk]) -> Double

#
empty_storage_report

fn empty_storage_report() -> StorageReport

#
encode_manifest

fn encode_manifest(manifest : Manifest) -> String
Canonical line-oriented encoding. It is intentionally simple to make the format inspectable in a hex editor and stable across MoonBit backends.

#
estimated_savings

fn estimated_savings(chunks : Array[Chunk]) -> Int

#
estimated_storage

fn estimated_storage(chunks : Array[Chunk]) -> Int

#
format_histogram

fn format_histogram(histogram : Array[Int]) -> String

#
insert_region

fn insert_region(input : BenchmarkInput, at : Int, inserted : Array[Byte]) -> BenchmarkInput

#
inspect_storage

fn inspect_storage(store : ChunkStore, manifest : Manifest) -> StorageReport

#
inventory_digest

fn inventory_digest(report : StorageReport) -> String

#
inventory_equal

fn inventory_equal(left : StorageReport, right : StorageReport) -> Bool

#
manifest_chunk_sizes

fn manifest_chunk_sizes(manifest : Manifest) -> Array[Int]

#
manifest_contains

fn manifest_contains(manifest : Manifest, digest : String) -> Bool

#
manifest_digests

fn manifest_digests(manifest : Manifest) -> Array[String]

#
manifest_from_chunks

fn manifest_from_chunks(name : String, chunks : Array[Chunk]) -> Manifest
Manifest construction, canonical encoding and lookup helpers.

#
manifest_from_digests

fn manifest_from_digests(name : String, digests : Array[String], sizes : Array[Int]) -> Result[Manifest, String]

#
manifest_has_all_chunks

fn manifest_has_all_chunks(store : ChunkStore, manifest : Manifest) -> Bool

#
manifest_has_no_extra_chunks

fn manifest_has_no_extra_chunks(store : ChunkStore, manifest : Manifest) -> Bool

#
manifest_lookup

fn manifest_lookup(manifest : Manifest, digest : String) -> ChunkRef?

#
manifest_position

fn manifest_position(manifest : Manifest, digest : String) -> Int?

#
manifest_prefix

fn manifest_prefix(manifest : Manifest, count : Int) -> Manifest

#
manifest_reorder

fn manifest_reorder(manifest : Manifest, order : Array[Int]) -> Result[Manifest, String]

#
manifest_same_content

fn manifest_same_content(left : Manifest, right : Manifest) -> Bool

#
manifest_suffix

fn manifest_suffix(manifest : Manifest, start : Int) -> Manifest

#
manifest_summary

fn manifest_summary(manifest : Manifest) -> String

#
manifest_validate_order

fn manifest_validate_order(manifest : Manifest) -> Bool

#
merkle_proof

fn merkle_proof(digests : Array[String], index : Int) -> Array[(String, Bool)]

#
merkle_root

fn merkle_root(digests : Array[String]) -> String

#
most_repeated

fn most_repeated(chunks : Array[Chunk]) -> (String, Int)?

#
mutate_region

fn mutate_region(input : BenchmarkInput, start : Int, length : Int, seed : UInt) -> BenchmarkInput

#
plan_removals

fn plan_removals(plan : SyncPlan) -> Int

#
plan_reuses

fn plan_reuses(plan : SyncPlan) -> Int

#
plan_uploads

fn plan_uploads(plan : SyncPlan) -> Int

#
profile_chunks

fn profile_chunks(chunks : Array[Chunk], buckets? : Int) -> ChunkProfile

#
profile_is_reasonable

fn profile_is_reasonable(profile : ChunkProfile, config : ChunkerConfig) -> Bool

#
profile_strings

fn profile_strings(chunks : Array[Chunk], buckets : Int) -> String

#
repetitive_input

fn repetitive_input(name : String, size : Int, pattern : String) -> BenchmarkInput

#
report_message

fn report_message(report : VerificationReport) -> String

#
restore_manifest

fn restore_manifest(store : ChunkStore, manifest : Manifest) -> Result[Array[Byte], String]

#
run_benchmark

fn run_benchmark(input : BenchmarkInput, config : ChunkerConfig) -> BenchmarkResult

#
run_comparison

fn run_comparison(old : BenchmarkInput, next : BenchmarkInput, config : ChunkerConfig) -> BenchmarkResult

#
size_percentile

fn size_percentile(chunks : Array[Chunk], percentile : Int) -> Int

#
storage_is_consistent

fn storage_is_consistent(report : StorageReport) -> Bool

#
storage_report_line

fn storage_report_line(report : StorageReport) -> String

#
storage_saving_ratio

fn storage_saving_ratio(report : StorageReport) -> Double

#
store_chunks

fn store_chunks(store : ChunkStore, chunks : Array[Chunk]) -> Int

#
store_logical_size

fn store_logical_size(store : ChunkStore, manifest : Manifest) -> Int

#
store_physical_size

fn store_physical_size(store : ChunkStore, manifest : Manifest) -> Int

#
sync_ratio

fn sync_ratio(plan : SyncPlan, original_size : Int) -> Double

#
sync_savings

fn sync_savings(plan : SyncPlan, original_size : Int) -> Int

#
synthetic_input

fn synthetic_input(name : String, size : Int, seed : UInt) -> BenchmarkInput

#
text_input

fn text_input(name : String, paragraphs : Int) -> BenchmarkInput

#
verify_bytes

fn verify_bytes(data : Array[Byte], expected : String) -> Bool

#
verify_manifest_root

fn verify_manifest_root(manifest : Manifest) -> Bool

#
verify_merkle_proof

fn verify_merkle_proof(leaf : String, proof : Array[(String, Bool)], root : String) -> Bool

#
verify_proof_for_manifest

fn verify_proof_for_manifest(manifest : Manifest, position : Int) -> Bool

#
verify_store

fn verify_store(store : ChunkStore, manifest : Manifest) -> VerificationReport