#laya.mbt

#Status

• Per-stage numerics match laya-mlx in float32 to within 1e-5 relative at every stage — embeddings, each encoder layer, the decision head, the scorer. Final option logits agree to <1e-4 absolute.
• Tokenization is identical to Hugging Face tokenizers across 76 cases per checkpoint, including leading/repeated whitespace, CJK, Hangul, decomposed accents and ligatures.
• Rendered output is byte-identical to laya_mlx.Agent.predict's JSON for every fixture case, on both checkpoints and on both backends. The two backends are also compared to each other at full precision, since the rendered JSON rounds to four decimals.

#Requirements

• MoonBit toolchain with the native backend (developed against moon 0.1.20260915).
• A BLAS: Apple Accelerate on macOS (always present), OpenBLAS elsewhere (libopenblas-dev).
• Node, for build.js, which embeds the Metal kernels and picks the platform's linker flags.
• ~1.5 GB of RAM for the large checkpoint, ~0.8 GB for the multilingual one. The GPU backend adds nothing: on Apple silicon it addresses the same weights.

#Getting a checkpoint

mkdir -p models/laya-multilingual-mlx
cd models/laya-multilingual-mlx
BASE=https://huggingface.co/aac6fef/laya-multilingual-mlx/resolve/main
curl -sLO $BASE/model.safetensors
curl -sLO $BASE/rl_agent_config.json
mkdir -p encoder tokenizer
curl -sL $BASE/encoder/config.json      -o encoder/config.json
curl -sL $BASE/tokenizer/tokenizer.json -o tokenizer/tokenizer.json
curl -sL $BASE/tokenizer/tokenizer_config.json -o tokenizer/tokenizer_config.json

<directory>/model.safetensors
<directory>/rl_agent_config.json
<directory>/encoder/config.json
<directory>/tokenizer/tokenizer.json
<directory>/tokenizer/tokenizer_config.json

#Using the library

moon add majikxu/laya

///|
fn triage() -> Unit raise {
  // `backend` defaults to Metal, falling back to the CPU when there is no GPU.
  // Pass `backend=Cpu` to force it; `agent.device()` reports what you got.
  let agent = @laya.Agent::load("models/laya-multilingual-mlx")
  let state = "I was billed twice this month. Please refund the duplicate charge."
  let prediction = agent.predict(state, [
    (
      "department",
      @laya.Question::choice("Who should handle this?", [
        "billing", "technical", "sales",
      ]),
    ),
    (
      "urgency",
      @laya.Question::score("How urgent is this?", [
        "not urgent", "soon", "urgent", "critical",
      ]),
    ),
    ("refund", @laya.Question::noul("The customer is asking for a refund.")),
  ])
  for entry in prediction.answers {
    let (id, answer) = entry
    match answer.decision {
      Choice(label~, ..) => println("\{id}: \{label} (\{answer.confidence})")
      Score(value~, ..) => println("\{id}: \{value} (\{answer.confidence})")
      Noul(probability~) => println("\{id}: \{probability}")
    }
  }
}

@laya.Question::choice_described("What should the billing system do next?", [
  ("dunning", "send a payment reminder"),
  ("suspend", "suspend the workspace"),
  ("wait", "take no action yet"),
])

#Prompt format

[CLS] <type> question: <instructions> [SEP] [MASK] <option 0> [MASK] <option 1> … [SEP] <state> [SEP]

#Using the CLI

moon build --target native --release
./_build/native/release/build/cmd/laya/laya.exe \
  --model models/laya-multilingual-mlx \
  --state "I was billed twice this month. Please refund the duplicate charge." \
  --questions examples/questions.json

{
  "department": {
    "type": "choice",
    "instructions": "Who should handle this?",
    "criteria": ["billing", "technical", "sales"]
  },
  "refund": {
    "type": "noul",
    "instructions": "The customer is asking for a refund."
  }
}

#Performance

#Where the remaining time goes

• No batching. Each question is a separate forward pass, so a three-question call streams the weights three times. At short prompt lengths this workload is bound by exactly that: one multilingual pass reads 441 MB of float32 weights regardless of how many tokens it is processing. The reference runtime pads a batch of questions into one pass and masks the padding out of every attention row, which does not change results — src/model/forward.mbt notes where the mask would go back.
• float32 weights. Halving them to float16 would halve that 441 MB, and MPS multiplies float16 directly. This is what laya-mlx does by default, and it is the single biggest remaining lever on both backends.

#How it fits together

src/                      facade: Agent, Question, Answer, Prompt, Vocabulary
src/error/                LayaError, shared by every layer
src/tensor/               the numeric layer, both backends:
  laya_native.c             BLAS, float16 decoding, checkpoint reads
  ops.mbt                   CPU kernels (layer norm, GELU, RoPE, softmax)
  laya_metal.m              Metal device, buffers, dispatch, MPS matmul
  metal/laya_kernels.metal  GPU kernels, one per CPU kernel
src/safetensors/          safetensors header parsing and typed tensor reads
src/model/                config parsing, the weight arena, both forward passes
src/cmd/laya/             the CLI
ref/                      Python reference runtime and fixture generators

• GEMM, through cblas_sgemm. PyTorch stores Linear weights as [out, in], so the transpose is expressed to BLAS rather than materialised.
• The weight arena: one flat float32 allocation outside the GC heap, since ModernBERT-large expands to ~1.4 GB and there is no reason for the collector to walk it. MoonBit addresses it by element offset.
• Reading the checkpoint, with pread into a 256 KiB staging buffer.

#The Metal backend

• Matrix multiplication is MPSMatrixMultiplication, cached by shape: a pass issues around ninety multiplies but only a handful of distinct shapes, and building one is not free. Attention batches all heads into a single call using uniform strides.
• Weights are not copied to the GPU. The float32 arena is mapped at a page boundary specifically so newBufferWithBytesNoCopy: can wrap it; on Apple silicon the CPU and GPU address the same bytes. ModernBERT-large would otherwise need a second 1.4 GB.
• The token embedding stays on the CPU. It is the largest tensor in the checkpoint and is only ever indexed, so rows are read from the file and uploaded as a [len, hidden] block.
• The action head stays on the CPU. It is two multiplies against a single row, and its calibration features come from a softmax over the option logits — which would force a round trip anyway.
• GELU is hand-written. The Metal shading language has no erf, so the shader uses Abramowitz & Stegun 7.1.26 (max absolute error 1.5e-7) and a test compares it against the CPU path's libm erff across 4001 points.

#One question per forward pass

#Verifying against the reference

cd ref && uv sync                       # installs laya-mlx as the oracle
uv run python dump_fixtures.py ../models/laya-multilingual-mlx \
    ../fixtures/laya-multilingual.json --full
uv run python dump_fixtures.py ../models/laya-mlx ../fixtures/laya-en.json --full
uv run python dump_tokenizer_fixtures.py ../models/laya-multilingual-mlx/tokenizer \
    ../fixtures/tokenizer-multilingual.json
uv run python dump_tokenizer_fixtures.py ../models/laya-mlx/tokenizer \
    ../fixtures/tokenizer-en.json
cd .. && moon test --target native

#Two upstream tokenizer corrections

• Replace(" " → "▁") ahead of Metaspace (mmBERT). Hugging Face's Metaspace prepends its replacement character only when the text does not already start with it; the port tests for a literal space, so once the normalizer has rewritten a leading space the marker is prepended twice. " false: …" gains a spurious ▁, which matters because every option body is built with a leading space. Dropping the normalizer and letting Metaspace do the substitution itself matches Hugging Face exactly, and stays correct if the upstream test is widened.
• NFC (ModernBERT). The port decomposes precomposed Hangul syllables into jamo without recomposing them, so "테스트" tokenizes as raw jamo bytes. Latin composition is fine, but rather than special-case Hangul this defers to moonbit-community/normalization.

#What is not here

• Batched inference. One question per forward pass; see Performance.
• float16 compute. Checkpoints are decoded to float32 on both backends.
• GPU outside Apple platforms. No CUDA or Vulkan backend.
• The router. Upstream's Router picks a checkpoint per request. Load the Agent you want.
• Presets, email helpers, shortlisting. Upstream's convenience layers on top of predict are not ported.
• Quantization. bfloat16 payloads are recognised by the header parser but not yet decoded.

#Licence and attribution

#Appendix: option rendering

///|
test "options render as the model sees them" {
  let choice = @laya.Question::choice_described("Who should handle this?", [
    ("billing", "payments and invoices"),
    ("technical", "bugs and outages"),
  ])
  debug_inspect(
    choice.rendered_options(),
    content=(
      #|["billing: payments and invoices", "technical: bugs and outages"]
    ),
  )
  let score = @laya.Question::score("How urgent?", ["not urgent", "critical"])
  debug_inspect(
    score.rendered_options(),
    content=(
      #|["level 0: not urgent", "level 1: critical"]
    ),
  )
  let noul = @laya.Question::noul("The customer wants a refund.")
  debug_inspect(
    noul.rendered_options(),
    content=(
      #|[
      #|  "false: no, the statement does not hold",
      #|  "true: yes, the statement holds",
      #|]
    ),
  )
  inspect(noul.type_name(), content="noul")
  inspect(noul.type_index(), content="2")
}

#laya.mbt

#Status

• Per-stage numerics match laya-mlx in float32 to within 1e-5 relative at every stage — embeddings, each encoder layer, the decision head, the scorer. Final option logits agree to <1e-4 absolute.
• Tokenization is identical to Hugging Face tokenizers across 76 cases per checkpoint, including leading/repeated whitespace, CJK, Hangul, decomposed accents and ligatures.
• Rendered output is byte-identical to laya_mlx.Agent.predict's JSON for every fixture case, on both checkpoints and on both backends. The two backends are also compared to each other at full precision, since the rendered JSON rounds to four decimals.

#Requirements

• MoonBit toolchain with the native backend (developed against moon 0.1.20260915).
• A BLAS: Apple Accelerate on macOS (always present), OpenBLAS elsewhere (libopenblas-dev).
• Node, for build.js, which embeds the Metal kernels and picks the platform's linker flags.
• ~1.5 GB of RAM for the large checkpoint, ~0.8 GB for the multilingual one. The GPU backend adds nothing: on Apple silicon it addresses the same weights.

#Getting a checkpoint

mkdir -p models/laya-multilingual-mlx
cd models/laya-multilingual-mlx
BASE=https://huggingface.co/aac6fef/laya-multilingual-mlx/resolve/main
curl -sLO $BASE/model.safetensors
curl -sLO $BASE/rl_agent_config.json
mkdir -p encoder tokenizer
curl -sL $BASE/encoder/config.json      -o encoder/config.json
curl -sL $BASE/tokenizer/tokenizer.json -o tokenizer/tokenizer.json
curl -sL $BASE/tokenizer/tokenizer_config.json -o tokenizer/tokenizer_config.json

<directory>/model.safetensors
<directory>/rl_agent_config.json
<directory>/encoder/config.json
<directory>/tokenizer/tokenizer.json
<directory>/tokenizer/tokenizer_config.json

#Using the library

moon add majikxu/laya

///|
fn triage() -> Unit raise {
  // `backend` defaults to Metal, falling back to the CPU when there is no GPU.
  // Pass `backend=Cpu` to force it; `agent.device()` reports what you got.
  let agent = @laya.Agent::load("models/laya-multilingual-mlx")
  let state = "I was billed twice this month. Please refund the duplicate charge."
  let prediction = agent.predict(state, [
    (
      "department",
      @laya.Question::choice("Who should handle this?", [
        "billing", "technical", "sales",
      ]),
    ),
    (
      "urgency",
      @laya.Question::score("How urgent is this?", [
        "not urgent", "soon", "urgent", "critical",
      ]),
    ),
    ("refund", @laya.Question::noul("The customer is asking for a refund.")),
  ])
  for entry in prediction.answers {
    let (id, answer) = entry
    match answer.decision {
      Choice(label~, ..) => println("\{id}: \{label} (\{answer.confidence})")
      Score(value~, ..) => println("\{id}: \{value} (\{answer.confidence})")
      Noul(probability~) => println("\{id}: \{probability}")
    }
  }
}

@laya.Question::choice_described("What should the billing system do next?", [
  ("dunning", "send a payment reminder"),
  ("suspend", "suspend the workspace"),
  ("wait", "take no action yet"),
])

#Prompt format

[CLS] <type> question: <instructions> [SEP] [MASK] <option 0> [MASK] <option 1> … [SEP] <state> [SEP]

#Using the CLI

moon build --target native --release
./_build/native/release/build/cmd/laya/laya.exe \
  --model models/laya-multilingual-mlx \
  --state "I was billed twice this month. Please refund the duplicate charge." \
  --questions examples/questions.json

{
  "department": {
    "type": "choice",
    "instructions": "Who should handle this?",
    "criteria": ["billing", "technical", "sales"]
  },
  "refund": {
    "type": "noul",
    "instructions": "The customer is asking for a refund."
  }
}

#Performance

#Where the remaining time goes

• No batching. Each question is a separate forward pass, so a three-question call streams the weights three times. At short prompt lengths this workload is bound by exactly that: one multilingual pass reads 441 MB of float32 weights regardless of how many tokens it is processing. The reference runtime pads a batch of questions into one pass and masks the padding out of every attention row, which does not change results — src/model/forward.mbt notes where the mask would go back.
• float32 weights. Halving them to float16 would halve that 441 MB, and MPS multiplies float16 directly. This is what laya-mlx does by default, and it is the single biggest remaining lever on both backends.

#How it fits together

src/                      facade: Agent, Question, Answer, Prompt, Vocabulary
src/error/                LayaError, shared by every layer
src/tensor/               the numeric layer, both backends:
  laya_native.c             BLAS, float16 decoding, checkpoint reads
  ops.mbt                   CPU kernels (layer norm, GELU, RoPE, softmax)
  laya_metal.m              Metal device, buffers, dispatch, MPS matmul
  metal/laya_kernels.metal  GPU kernels, one per CPU kernel
src/safetensors/          safetensors header parsing and typed tensor reads
src/model/                config parsing, the weight arena, both forward passes
src/cmd/laya/             the CLI
ref/                      Python reference runtime and fixture generators

• GEMM, through cblas_sgemm. PyTorch stores Linear weights as [out, in], so the transpose is expressed to BLAS rather than materialised.
• The weight arena: one flat float32 allocation outside the GC heap, since ModernBERT-large expands to ~1.4 GB and there is no reason for the collector to walk it. MoonBit addresses it by element offset.
• Reading the checkpoint, with pread into a 256 KiB staging buffer.

#The Metal backend

• Matrix multiplication is MPSMatrixMultiplication, cached by shape: a pass issues around ninety multiplies but only a handful of distinct shapes, and building one is not free. Attention batches all heads into a single call using uniform strides.
• Weights are not copied to the GPU. The float32 arena is mapped at a page boundary specifically so newBufferWithBytesNoCopy: can wrap it; on Apple silicon the CPU and GPU address the same bytes. ModernBERT-large would otherwise need a second 1.4 GB.
• The token embedding stays on the CPU. It is the largest tensor in the checkpoint and is only ever indexed, so rows are read from the file and uploaded as a [len, hidden] block.
• The action head stays on the CPU. It is two multiplies against a single row, and its calibration features come from a softmax over the option logits — which would force a round trip anyway.
• GELU is hand-written. The Metal shading language has no erf, so the shader uses Abramowitz & Stegun 7.1.26 (max absolute error 1.5e-7) and a test compares it against the CPU path's libm erff across 4001 points.

#One question per forward pass

#Verifying against the reference

cd ref && uv sync                       # installs laya-mlx as the oracle
uv run python dump_fixtures.py ../models/laya-multilingual-mlx \
    ../fixtures/laya-multilingual.json --full
uv run python dump_fixtures.py ../models/laya-mlx ../fixtures/laya-en.json --full
uv run python dump_tokenizer_fixtures.py ../models/laya-multilingual-mlx/tokenizer \
    ../fixtures/tokenizer-multilingual.json
uv run python dump_tokenizer_fixtures.py ../models/laya-mlx/tokenizer \
    ../fixtures/tokenizer-en.json
cd .. && moon test --target native

#Two upstream tokenizer corrections

• Replace(" " → "▁") ahead of Metaspace (mmBERT). Hugging Face's Metaspace prepends its replacement character only when the text does not already start with it; the port tests for a literal space, so once the normalizer has rewritten a leading space the marker is prepended twice. " false: …" gains a spurious ▁, which matters because every option body is built with a leading space. Dropping the normalizer and letting Metaspace do the substitution itself matches Hugging Face exactly, and stays correct if the upstream test is widened.
• NFC (ModernBERT). The port decomposes precomposed Hangul syllables into jamo without recomposing them, so "테스트" tokenizes as raw jamo bytes. Latin composition is fine, but rather than special-case Hangul this defers to moonbit-community/normalization.

#What is not here

• Batched inference. One question per forward pass; see Performance.
• float16 compute. Checkpoints are decoded to float32 on both backends.
• GPU outside Apple platforms. No CUDA or Vulkan backend.
• The router. Upstream's Router picks a checkpoint per request. Load the Agent you want.
• Presets, email helpers, shortlisting. Upstream's convenience layers on top of predict are not ported.
• Quantization. bfloat16 payloads are recognised by the header parser but not yet decoded.

#Licence and attribution

#Appendix: option rendering

///|
test "options render as the model sees them" {
  let choice = @laya.Question::choice_described("Who should handle this?", [
    ("billing", "payments and invoices"),
    ("technical", "bugs and outages"),
  ])
  debug_inspect(
    choice.rendered_options(),
    content=(
      #|["billing: payments and invoices", "technical: bugs and outages"]
    ),
  )
  let score = @laya.Question::score("How urgent?", ["not urgent", "critical"])
  debug_inspect(
    score.rendered_options(),
    content=(
      #|["level 0: not urgent", "level 1: critical"]
    ),
  )
  let noul = @laya.Question::noul("The customer wants a refund.")
  debug_inspect(
    noul.rendered_options(),
    content=(
      #|[
      #|  "false: no, the statement does not hold",
      #|  "true: yes, the statement holds",
      #|]
    ),
  )
  inspect(noul.type_name(), content="noul")
  inspect(noul.type_index(), content="2")
}