moon_metric_lab

A pure MoonBit machine learning evaluation metrics and experiment comparison framework supporting native, wasm-gc, and js backends.

machine-learning
metrics
evaluation
benchmark
data-science
moon add myc1234567/moon_metric_lab@0.1.3
Download zip
Version
0.1.3
License
Apache-2.0
Last updated
yesterday
Downloads
12
README

#MoonMetricLab ๐ŸŒ™๐Ÿ“Š

License: Apache-2.0 MoonBit Version Targets OSC 2026 GitLink Repo GitHub Mirror

MoonMetricLab is a pure MoonBit evaluation suite and benchmark comparison framework for machine learning models and experiments. It uses zero-copy views and explicit numerical validation to evaluate, compare, and visualize pipelines across supported MoonBit targets without third-party runtime dependencies.


#โœจ Key Highlights

  • ๐ŸŽฏ Complete Domain Coverage: Out-of-the-box support for Classification (binary, multiclass, ROC/PR curves), Regression (continuous and robust losses), Clustering (internal separation indices and external agreement metrics), and Ranking (information retrieval DCG/NDCG, MRR, MAP).
  • ๐Ÿš€ Zero-Copy Data Handling: Lightweight VectorView and MatrixView abstractions enable zero-copy slicing, windowing, and high-speed statistical operations over contiguous double and integer buffers.
  • ๐Ÿ›ก๏ธ Checked Numerical Stability boundaries: Strict division-by-zero protection (EPSILON guards), underflow/overflow prevention (log_cosh and log-loss clipping), and granular structured errors (raise @core.MetricError) instead of ungraceful panics.
  • ๐Ÿ“ˆ Benchmark Comparison & Report Engine: Automated candidate vs. baseline comparison reports computing absolute deltas, percentage shifts, and automatic improvement/degradation detection.
  • โœ… Release-Ready Experiment Workflow: Aggregate cross-fold metrics, rank models, enforce minimum/maximum/stability gates, and export deployment evidence as Markdown or CSV.
  • ๐ŸŽจ Multi-Format Visualization Suite: Built-in ASCII terminal bar charts, GitHub-flavored Markdown tables, self-contained SVG curve generators (to_svg_roc_curve), and dependency-free JSON/CSV exporters.


#๐Ÿ—๏ธ Architecture & Package Sitemap

MoonMetricLab follows a strictly modular, layered architecture:

myc1234567/moon_metric_lab โ”œโ”€โ”€ core # Base numerical primitives, zero-copy views, constants, and structured errors โ”œโ”€โ”€ classification # Binary & multi-class metrics, confusion matrix, threshold scanner, ROC/PR curves โ”œโ”€โ”€ regression # Continuous error metrics (MAE/MSE/RMSE/R2/MAPE) and robust losses (Huber/Pinball/Log-Cosh) โ”œโ”€โ”€ clustering # Internal (Silhouette, CH, DBI) and external (ARI, NMI, FMI) clustering evaluations โ”œโ”€โ”€ ranking # Information retrieval ranking metrics (DCG@K, NDCG@K, MRR, MAP) โ”œโ”€โ”€ report # Evaluation containers and automated candidate vs. baseline comparison engine โ”œโ”€โ”€ visualization # ASCII bar charts, Markdown tables, SVG curve plots, and JSON/CSV exporters โ”œโ”€โ”€ workflow # Cross-fold aggregation, model ranking, quality gates, and deployment reports โ”œโ”€โ”€ cmd/main # Standalone CLI showcase executable demonstrator โ””โ”€โ”€ examples # Runnable demo projects (binary_demo, comparison_demo)

PackageDescriptionKey APIs
coreNumerical foundation & errorsVectorView, MatrixView, validate_same_length, EPSILON
classificationClassification evaluationaccuracy, f1_score, log_loss, mcc, roc_curve, ConfusionMatrix
regressionContinuous regression metricsmae, mse, rmse, r2_score, mape, huber_loss, pinball_loss
clusteringClustering evaluationsilhouette_score, calinski_harabasz_score, adjusted_rand_index
rankingIR & recommendation metricsndcg_at_k, reciprocal_rank, mean_reciprocal_rank, average_precision
reportBenchmark reportingModelResult, compare_models, ComparisonReport
visualizationMulti-format renderingto_ascii_bar_chart, to_markdown_table, to_svg_roc_curve, to_csv_string
hypothesis_testingMulti-fold statistics, significance comparison and effect sizesaggregate_folds_mean, welch_t_test, paired_mean_difference, standard_error, cohens_d
workflowReproducible experiment-to-release workflowsummarize, compare, rank, evaluate_gate, deployment_report


#๐Ÿ“ฆ Installation & Dependency Integration

Add moon_metric_lab as a dependency in your moon.mod:

[deps] "myc1234567/moon_metric_lab" = "0.1.3"

Then, you can import any sub-package (e.g., core, classification, regression, hypothesis_testing, workflow) in your moon.pkg:

import { "myc1234567/moon_metric_lab/classification" "myc1234567/moon_metric_lab/hypothesis_testing" "myc1234567/moon_metric_lab/workflow" }

Or via the moon command-line tool:

moon add myc1234567/moon_metric_lab@0.1.3

After adding the dependency, import the package path you use in moon.pkg. The public API is described by the tracked pkg.generated.mbti files, so consumers can inspect signatures without reading implementation files.


#๐Ÿ’ก Quick Start & Usage Examples

#1. Binary & Multi-Class Classification

test "README classification demo" {
let y_true = [0, 0, 1, 1]
let y_pred = [0, 1, 1, 1]
let y_prob = [0.1, 0.6, 0.8, 0.9]

let acc = @classification.accuracy(y_true, y_pred)
inspect!(acc, content="0.75")

let f1 = @classification.f1_score(y_true, y_pred, pos_label=1)
inspect!(f1 > 0.79 && f1 < 0.81, content="true")

let roc = @classification.roc_curve(y_true, y_prob, pos_label=1)
inspect!(roc.auc, content="1")
}

#2. Regression & Robust Loss Functions

test "README regression demo" {
let y_true = [3.0, -0.5, 2.0, 7.0]
let y_pred = [2.5, 0.0, 2.0, 8.0]

let rmse_val = @regression.rmse(y_true, y_pred)
inspect!(rmse_val > 0.61 && rmse_val < 0.62, content="true")

let r2 = @regression.r2_score(y_true, y_pred)
inspect!(r2 > 0.94, content="true")

let huber = @regression.huber_loss(y_true, y_pred, delta=1.0)
inspect!(huber, content="0.1875")
}

#3. Clustering Separation & Agreement

test "README clustering demo" {
let x = [[1.0, 1.0], [1.2, 0.8], [5.0, 5.0], [5.2, 4.8]]
let labels = [0, 0, 1, 1]

let sil = @clustering.silhouette_score(x, labels)
inspect!(sil > 0.8, content="true")

let ari = @clustering.adjusted_rand_index([0, 0, 1, 1], [1, 1, 0, 0])
inspect!(ari, content="1")
}

#4. Ranking & Information Retrieval

test "README ranking demo" {
let y_rels = [3.0, 2.0, 3.0, 0.0, 1.0]
let y_scores = [0.9, 0.8, 0.7, 0.6, 0.5]

let ndcg = @ranking.ndcg_at_k(y_rels, y_scores, k=3)
inspect!(ndcg > 0.8, content="true")
}

#5. Benchmark Comparison & Visualization

test "README comparison demo" {
let baseline = @report.ModelResult::new("LogReg", "classification")
baseline.add_metric("accuracy", 0.80)
baseline.add_metric("log_loss", 0.45)

let candidate = @report.ModelResult::new("XGBoost", "classification")
candidate.add_metric("accuracy", 0.88)
candidate.add_metric("log_loss", 0.32)

let comp = @report.compare_models(baseline, candidate)
let table = comp.to_table_string()
inspect!(table.contains("IMPROVED"), content="true")
}

#6. Multi-fold significance and effect size

test "README significance demo" {
let before = @core.Vector::from_array([0.70, 0.75, 0.80, 0.85])
let after = @core.Vector::from_array([0.75, 0.80, 0.85, 0.90])
let difference = @hypothesis_testing.paired_mean_difference(before, after)
inspect!(difference > 0.049 && difference < 0.051, content="true")
let effect = @hypothesis_testing.cohens_d(before, after)
inspect!(effect < 0.0, content="true")
}

#7. Cross-fold evaluation to a deployment decision

The workflow package is the complete application-facing chain: record fold-level results, aggregate mean/stddev/min/max, compare a baseline with a candidate, apply release rules, and export review evidence. The fixture metadata is intentionally explicit so an external data run can be reproduced without redistributing the original UCI or LETOR files.

test "README release workflow demo" {
let candidate = @workflow.Experiment::new("calibrated-model", "classification")
candidate.add_metadata("dataset", "UCI Breast Cancer Wisconsin")
for i, auc in [0.94, 0.95, 0.93, 0.94] {
let fold = @workflow.FoldObservation::new(i + 1)
fold.add_metric("roc_auc", auc)
fold.add_metric("log_loss", 1.0 - auc)
candidate.add_fold(fold)
}
let summary = @workflow.summarize(candidate)
let gate = @workflow.evaluate_gate(summary, "production-v1", [
@workflow.GateRule::new("roc_auc").with_minimum(0.92),
@workflow.GateRule::new("log_loss").with_maximum(0.10),
])
inspect!(gate.is_passed(), content="true")
inspect!(@workflow.summary_to_markdown(summary).contains("roc_auc"), content="true")
}


#๐Ÿ› ๏ธ Tooling & Commands

Build and verify the whole project across targets using the official MoonBit CLI (moon):

# Check compilation across all targets (Wasm-GC, Wasm, JS, Native) moon check --target all # Run the complete test suite on every available backend moon test --target all --deny-warn # Generate updated package interfaces moon info # Verify that generated public interfaces are already up to date git diff --exit-code -- '**/pkg.generated.mbti' # Run the full CLI evaluation showcase moon run --target wasm-gc cmd/main

#Publishing to mooncakes.io

The published module name is myc1234567/moon_metric_lab, and the repository version must be bumped before each release. Run the same validation commands above, then execute moon publish from the module root. Verify the result with moon add myc1234567/moon_metric_lab@<version> in a clean consumer module. This repository keeps generated interfaces and the root LICENSE in the package so the published archive remains inspectable.

For dataset provenance, reproducible benchmark protocol, and the boundary matrix, see BENCHMARKS.md. The repository contains compact examples rather than redistributed copies of external datasets.


#๐Ÿ“„ License & OSC 2026 Originality Statement

This project is open-sourced under the Apache License 2.0. Developed for the MoonBit OSC 2026 Open Source Track, showcasing pure MoonBit architectural design, comprehensive documentation, and multi-backend portability.

#๐Ÿ›ก๏ธ Third-Party Source Boundary & Originality Declaration (็ฌฌไธ‰ๆ–นๆฅๆบ่พน็•ŒไธŽๅŽŸๅˆ›ไปฃ็ ๅฃฐๆ˜Ž)

  • Original implementation scope: The repository currently contains 4,400+ effective lines of MoonBit source (excluding generated interfaces and build artifacts), including the hypothesis_testing and workflow packages and their boundary/fixture tests. The implementation is authored in this repository; external dataset rights remain with their respective providers.
  • Mathematical principle boundary: The metrics implemented here (ROC AUC, Huber Loss, Silhouette, NDCG@K, Confusion Matrix, MAE/RMSE and others) follow standard mathematical definitions. The repository's data structures, error handling, numerical validation and multi-format rendering are implemented as MoonBit engineering in this project. See THIRD_PARTY_NOTICES.md for the source and license boundary.

Powered by MoonBit

Site sourceReport issuePackagesBuild queueSkillsStatistics

ยฉ 2026 mooncakes.io