🥮 mooncakes.io

moonbench

A tiny MoonBit benchmark statistics toolkit for stopwatch-style measurements.

benchmark
stopwatch
statistics
moonbit
osc2026
moon add sa2360/moonbench@0.3.0
Download zip
Author
Version
0.3.0
License
MIT
Repository
Last updated
last month
Downloads
16
README

#moonbench

moonbench is a small MoonBit benchmark reporting toolkit for stopwatch-style measurements, external timing samples, and CI performance budgets.

It is designed for the MoonBit OSC2026 ecosystem package track: clear API boundary, runnable tests, a minimal CLI example, CI configuration, and an OSI-approved open-source license. It complements the official moon bench workflow instead of replacing it.

#Why

MoonBit projects often need lightweight timing summaries while developing parsers, algorithms, CLI tools, and examples. moonbench does not own the clock source. It accepts elapsed nanoseconds from your runtime, then provides stable aggregate calculations and compact report strings.

For quick demos and lightweight tooling, benchmark can also run a function repeatedly and measure it with the standard environment clock. For production tools, benchmark_with_clock accepts an injected Clock adapter so callers can use a monotonic platform clock or deterministic test clock. For CI smoke checks, Budget, meets_budget, and budget_report turn timing summaries into pass/fail reports. For release review, compare_to_baseline, within_regression_budget, and regression_report compare current results with a saved baseline. Imported samples can be cleaned with SampleFilter, filter_samples, and trim_samples before reporting.

See docs/moon-bench-comparison.md for the difference from official moon bench.

#Install

Add the package to an existing MoonBit project:

moon add sa2360/moonbench

Then import it from a package with an alias:

{
  "import": [
    {
      "path": "sa2360/moonbench",
      "alias": "bench"
    }
  ]
}

Use the package from MoonBit code:

let set = @bench.sample_set_from([1_000_000, 1_200_000, 1_100_000])
println(@bench.sample_set_json(set))

#API

  • empty() -> Summary
  • singleton(elapsed_ns : Int) -> Summary
  • sample_set() -> SampleSet
  • sample_set_from(samples : Array[Int]) -> SampleSet
  • SampleFilter
  • filter_samples(set : SampleSet, filter : SampleFilter) -> SampleSet
  • trim_samples(set : SampleSet, drop_each_side : Int) -> SampleSet
  • drop_warmup(set : SampleSet, warmup_count : Int) -> SampleSet
  • clamp_samples(set : SampleSet, min_ns : Int, max_ns : Int) -> SampleSet
  • Budget
  • Comparison
  • Quality
  • Clock
  • system_clock() -> Clock
  • measure(run : () -> Unit) -> Int
  • measure_with_clock(clock : Clock, run : () -> Unit) -> Int
  • benchmark(iterations : Int, run : () -> Unit) -> Summary
  • benchmark_with_clock(iterations : Int, clock : Clock, run : () -> Unit) -> SampleSet
  • has_samples(summary : Summary) -> Bool
  • Summary::add_sample(summary : Summary, elapsed_ns : Int) -> Summary
  • SampleSet::add_sample(set : SampleSet, elapsed_ns : Int) -> SampleSet
  • merge(left : Summary, right : Summary) -> Summary
  • percentile_ns(set : SampleSet, pct : Int) -> Int
  • variance_ns2(set : SampleSet) -> Double
  • std_dev_ns(set : SampleSet) -> Double
  • distribution(set : SampleSet) -> Distribution
  • spread_ns(summary : Summary) -> Int
  • relative_spread_ppm(summary : Summary) -> Int
  • std_dev_ppm(set : SampleSet) -> Int
  • quality(set : SampleSet, tolerance_ppm : Int) -> Quality
  • samples_per_second(summary : Summary) -> Int
  • is_stable(summary : Summary, tolerance_ppm : Int) -> Bool
  • summary_csv_header() -> String
  • summary_csv(summary : Summary) -> String
  • summary_json(summary : Summary) -> String
  • distribution_json(distribution : Distribution) -> String
  • sample_set_json(set : SampleSet) -> String
  • summary_markdown(summary : Summary) -> String
  • sample_set_markdown(set : SampleSet) -> String
  • quality_report(set : SampleSet, tolerance_ppm : Int) -> String
  • format_ns(ns : Int) -> String
  • describe(summary : Summary) -> String
  • meets_budget(summary : Summary, budget : Budget) -> Bool
  • budget_report(summary : Summary, budget : Budget) -> String
  • compare_to_baseline(current : Summary, baseline : Summary) -> Comparison
  • within_regression_budget(current : Summary, baseline : Summary, tolerance_ppm : Int) -> Bool
  • regression_report(current : Summary, baseline : Summary, tolerance_ppm : Int) -> String

#Example

let summary = @bench.benchmark(5, () => {
  let mut total = 0
  for i in 0..<20_000_000 {
    total = total + i % 17
  }
  ignore(total)
})

println(@bench.describe(summary))
println("spread=\{@bench.format_ns(@bench.spread_ns(summary))}")
println("stable(5%)=\{@bench.is_stable(summary, 50_000)}")

Example output, values depend on the machine:

count=5, mean=13 ms, min=11 ms, max=23 ms, throughput=72/s
spread=12 ms
stable(5%)=false

#Reports and Percentiles

let set = @bench.sample_set_from([100, 140, 160, 200])
let dist = @bench.distribution(set)

println(@bench.summary_csv_header())
println(@bench.summary_csv(set.summary))
println(@bench.sample_set_json(set))
println("p95=\{@bench.format_ns(dist.p95_ns)}")

This produces machine-readable output for CI systems, release notes, dashboards, or GitHub Actions artifacts. The raw sample set can come from moonbench, from another runtime clock, or from benchmark logs collected by a separate tool.

#Imported Samples, Cleanup, and Regression Checks

let imported = @bench.sample_set_from([0, 12_000_000, 12_400_000, 80_000_000])
let cleaned = @bench.filter_samples(
  imported,
  @bench.SampleFilter::{ min_ns: 1, max_ns: 50_000_000, drop_zero: true },
)
let baseline = @bench.sample_set_from([12_200_000, 12_200_000, 12_300_000])

println(@bench.sample_set_markdown(cleaned))
println(@bench.regression_report(cleaned.summary, baseline.summary, 80_000))

This path is useful when another benchmark runner already produced timing data. moonbench can focus on cleanup, summaries, release-note tables, and regression decisions without taking over the benchmark execution step.

#Data-Backed Example

The repository includes concrete run data in data/benchmark-runs.csv and notes in docs/benchmark-data.md. These files record local moon run cmd/main output and an imported-sample scenario, including summary CSV fields, p50/p95/p99 percentiles, and budget status.

#Platform Clock Adapter

let mut tick = 0
let fake_clock = @bench.Clock::{
  name: "deterministic",
  now_ns: () => {
    tick = tick + 10
    tick
  },
}

let set = @bench.benchmark_with_clock(3, fake_clock, () => ignore(1 + 1))
println(@bench.sample_set_json(set))

Applications can replace the fake clock with a monotonic native clock while keeping the same statistics and report layer.

#Run

moon check
moon build
moon fmt
moon info
moon test
moon run cmd/main

cmd/main uses system_clock(), which is backed by the portable env.now() clock. On platforms where that clock is millisecond-grained, very short samples may round to 0 ns. In that case the demo budget line can occasionally print fail even though the command exits successfully and the CSV/JSON reports are still valid. Use an injected monotonic Clock for stricter production budgets.

#OSC2026 Checklist

  • Public repository: https://github.com/sa2360/moonbit.
  • MoonBit as main language: source is in .mbt files.
  • README: this file explains motivation, API, example, and commands.
  • Runnable example: cmd/main.
  • Tests: moonbench_test.mbt.
  • CI: .github/workflows/ci.yml runs moon check, moon build, formatting, moon info, moon test, and the example.
  • License: MIT.
  • Mooncakes release: run moon login, then moon publish.

#
Budget

source
pub(all) struct Budget {
  max_mean_ns : Int
  max_spread_ppm : Int
} derive(Eq, 
Debug)

Simple performance budget for CI or documentation smoke checks.

#
Clock

source
pub(all) struct Clock {
  name : String
  now_ns : () -> Int
}

Clock adapter used by benchmark helpers.

Production tools can pass a platform-specific monotonic clock. Tests can pass a deterministic clock so examples do not depend on wall time.

#
Comparison

source
pub(all) struct Comparison {
  baseline_mean_ns : Int
  current_mean_ns : Int
  delta_ns : Int
  delta_ppm : Int
  faster : Bool
  slower : Bool
} derive(Eq, 
Debug)

Difference between a current summary and a baseline summary.

delta_ppm is measured against the baseline mean. Positive values are slower, negative values are faster.

#
Distribution

source
pub(all) struct Distribution {
  p50_ns : Int
  p90_ns : Int
  p95_ns : Int
  p99_ns : Int
  variance_ns2 : Double
  std_dev_ns : Double
} derive(
Debug)

Distribution statistics derived from all recorded samples.

#
Quality

source
pub(all) struct Quality {
  sample_count : Int
  spread_ppm : Int
  std_dev_ppm : Int
  stable : Bool
} derive(Eq, 
Debug)

Compact quality signal for a sample set.

The fields are expressed in parts per million so CI thresholds can stay integer based. 100_000ppm means 10 percent.

#
SampleFilter

source
pub(all) struct SampleFilter {
  min_ns : Int
  max_ns : Int
  drop_zero : Bool
} derive(Eq, 
Debug)

Sample cleanup settings for noisy clocks and imported benchmark logs.

max_ns <= 0 means "no upper bound". Set drop_zero when a coarse clock can occasionally round very short samples to 0 ns.

#
SampleSet

source
pub(all) struct SampleSet {
  samples : Array[Int]
  summary : Summary
} derive(Eq, 
Debug)

Raw benchmark samples plus the derived aggregate summary.

Use SampleSet when a caller needs percentiles, variance, CSV/JSON reports, or later export of the original elapsed samples.

#
SampleSet::add_sample

source
fn SampleSet::add_sample(set : SampleSet, elapsed_ns : Int) -> SampleSet

Add one elapsed sample and update aggregate statistics.

#
Summary

source
pub(all) struct Summary {
  count : Int
  total_ns : Int
  min_ns : Int
  max_ns : Int
  mean_ns : Int
} derive(Eq, 
Debug)

Aggregated timing statistics for stopwatch-style benchmark samples.

The package keeps the data model intentionally small: callers can feed elapsed nanoseconds from any clock source and use the derived summary in CLI tools, test reports, CI logs, or documentation examples.

#
Summary::add_sample

source
fn Summary::add_sample(summary : Summary, elapsed_ns : Int) -> Summary

Add one elapsed sample to an existing summary.

#
benchmark

source
fn benchmark(iterations : Int, run : () -> Unit) -> Summary

Execute run repeatedly and aggregate the measured elapsed times.

#
benchmark_with_clock

source
fn benchmark_with_clock(iterations : Int, clock : Clock, run : () -> Unit) -> SampleSet

Execute run repeatedly and keep every measured elapsed sample.

#
budget_report

source
fn budget_report(summary : Summary, budget : Budget) -> String

Produce a compact pass/fail budget report for CI logs.

#
clamp_samples

source
fn clamp_samples(set : SampleSet, min_ns : Int, max_ns : Int) -> SampleSet

Clamp samples into a min/max range while keeping the sample count.

This is useful when a dashboard should keep every iteration but cap known sensor, clock, or runner artifacts before reporting percentiles.

#
compare_to_baseline

source
fn compare_to_baseline(current : Summary, baseline : Summary) -> Comparison

Compare current benchmark results against a baseline summary.

#
describe

source
fn describe(summary : Summary) -> String

Produce a one-line report suitable for CLI output and CI logs.

#
distribution

source
fn distribution(set : SampleSet) -> Distribution

Compute common latency percentiles and variance for a sample set.

#
distribution_json

source
fn distribution_json(distribution : Distribution) -> String

Machine-readable JSON report for common distribution statistics.

#
drop_warmup

source
fn drop_warmup(set : SampleSet, warmup_count : Int) -> SampleSet

Drop the first N samples while preserving the original order of the rest.

This is intended for benchmark warm-up samples, for example the first run after loading a CLI, parser, or runtime cache.

#
empty

source
fn empty() -> Summary

Return an empty summary.

#
filter_samples

source
fn filter_samples(set : SampleSet, filter : SampleFilter) -> SampleSet

Keep only samples that match the cleanup filter.

This is useful when imported data contains warm-up artifacts, known outlier bounds, or 0 ns values caused by a coarse demo clock.

#
format_ns

source
fn format_ns(ns : Int) -> String

Format a nanosecond duration into a compact human-readable unit.

#
has_samples

source
fn has_samples(summary : Summary) -> Bool

Return true when the summary contains at least one sample.

#
is_stable

source
fn is_stable(summary : Summary, tolerance_ppm : Int) -> Bool

Return true when the min/max spread is inside a parts-per-million tolerance.

A tolerance of 100_000 means 10 percent.

#
measure

source
fn measure(run : () -> Unit) -> Int

Measure one execution of run and return elapsed nanoseconds.

This helper uses the standard environment clock, so it is portable and intended for lightweight tool output rather than laboratory-grade timing.

#
measure_with_clock

source
fn measure_with_clock(clock : Clock, run : () -> Unit) -> Int

Measure one execution of run with a caller-provided clock.

#
meets_budget

source
fn meets_budget(summary : Summary, budget : Budget) -> Bool

Return true when a summary is non-empty and fits the given performance budget.

#
merge

source
fn merge(left : Summary, right : Summary) -> Summary

Merge two summaries. Empty summaries are identity values.

#
percentile_ns

source
fn percentile_ns(set : SampleSet, pct : Int) -> Int

Return a percentile in nanoseconds using linear interpolation.

pct is clamped into [0, 100]. Empty sample sets return 0.

#
quality

source
fn quality(set : SampleSet, tolerance_ppm : Int) -> Quality

Summarize whether a sample set is stable enough for a tolerance threshold.

#
quality_report

source
fn quality_report(set : SampleSet, tolerance_ppm : Int) -> String

Human-readable quality report for CI logs and benchmark notes.

#
regression_report

source
fn regression_report(current : Summary, baseline : Summary, tolerance_ppm : Int) -> String

Produce a compact pass/fail regression report for release checks.

#
relative_spread_ppm

source
fn relative_spread_ppm(summary : Summary) -> Int

Min/max spread relative to the mean, expressed in parts per million.

#
sample_set

source
fn sample_set() -> SampleSet

Return an empty sample set.

#
sample_set_from

source
fn sample_set_from(samples : Array[Int]) -> SampleSet

Build a sample set from elapsed nanosecond values.

#
sample_set_json

source
fn sample_set_json(set : SampleSet) -> String

Machine-readable JSON report including summary and distribution fields.

#
sample_set_markdown

source
fn sample_set_markdown(set : SampleSet) -> String

Markdown report including summary and percentile fields.

#
samples_per_second

source
fn samples_per_second(summary : Summary) -> Int

Integer operations per second, using nanoseconds as the time base.

#
singleton

source
fn singleton(elapsed_ns : Int) -> Summary

Start a summary with one elapsed sample in nanoseconds.

#
spread_ns

source
fn spread_ns(summary : Summary) -> Int

Difference between the slowest and fastest samples in nanoseconds.

#
std_dev_ns

source
fn std_dev_ns(set : SampleSet) -> Double

Sample standard deviation in nanoseconds.

#
std_dev_ppm

source
fn std_dev_ppm(set : SampleSet) -> Int

Sample standard deviation relative to the mean, expressed in ppm.

#
summary_csv

source
fn summary_csv(summary : Summary) -> String

Machine-readable CSV row for a summary.

#
summary_csv_header

source
fn summary_csv_header() -> String

CSV header for summary rows.

#
summary_json

source
fn summary_json(summary : Summary) -> String

Machine-readable JSON report for a summary.

#
summary_markdown

source
fn summary_markdown(summary : Summary) -> String

Markdown table header and row for a summary.

#
system_clock

source
fn system_clock() -> Clock

Return the system clock adapter used by the default benchmark helpers.

env.now() currently exposes milliseconds, so the adapter converts the value to nanoseconds for a single public time unit.

#
trim_samples

source
fn trim_samples(set : SampleSet, drop_each_side : Int) -> SampleSet

Drop the same number of sorted samples from the low and high ends.

If the requested trim would remove every sample, an empty set is returned.

#
variance_ns2

source
fn variance_ns2(set : SampleSet) -> Double

Sample variance in square nanoseconds.

#
within_regression_budget

source
fn within_regression_budget(current : Summary, baseline : Summary, tolerance_ppm : Int) -> Bool

Return true when the current mean is not slower than the tolerated baseline.

Source Files

Powered by MoonBit

Site sourceReport issuePackagesBuild queueSkillsStatistics

© 2026 mooncakes.io