🥮 mooncakes.io

moonjournalkit

Deterministic transactional write-ahead log framing and crash recovery for MoonBit.

write-ahead-log
crash-recovery
journal
crc32c
storage-engine
moon add Sxy11112/moonjournalkit@0.2.0
Download zip
Author
Version
0.2.0
License
Apache-2.0
Repository
Last updated
last month
Downloads
12
README

#MoonJournalKit

面向 MoonBit 的事务预写日志（WAL）格式、崩溃恢复与校验基础库。

MoonJournalKit 不绑定文件系统、数据库或异步运行时。调用方负责持久化字节， 本库负责生成稳定记录、识别可信边界，并且只重放已经明确提交的事务。因此同一套 核心代码可以运行在 Wasm、Wasm GC、JavaScript 和 Native 后端。

#主要能力

  • 24 字节固定头部的 MWAL v1 二进制记录格式
  • CRC32C（Castagnoli）逐记录完整性校验及增量接口
  • Begin / Put / Delete / Commit / Abort / Checkpoint 事务语义
  • 截断尾部、校验损坏、版本不支持、序列异常的精确区分
  • 只重放已提交事务的确定性恢复计划
  • 检查点裁剪、日志分段轮转、分段链连续性验证
  • 稳定 JSON 诊断结果，便于 CI 和运维系统接入
  • 逐字节断电切断和逐比特破坏的穷举验证

#安装

moon add Sxy11112/moonjournalkit

#快速示例

let records = [
  @journal.JournalRecord::new(Begin, 1U, 7U),
  @journal.JournalRecord::new(
    Put,
    2U,
    7U,
    payload=@journal.encode_put_payload(b"user/7", b"active"),
  ),
  @journal.JournalRecord::new(Commit, 3U, 7U),
]

let bytes = @journal.encode_records(records)
let scan = @journal.scan_journal(bytes)
let plan = @journal.build_recovery_plan(scan.records)
assert_eq(plan.actions.length(), 1)

发生断电时，应先用 scan.valid_bytes 截取可信前缀，再执行恢复计划。 缺失 Commit 的事务会出现在 incomplete_transactions 中，但不会进入 actions。

#可复现验证

moon check --target all
moon test --target wasm
moon test --target wasm-gc
moon run cmd/main
moon run bench/main

当前测试覆盖标准 CRC32C 向量、二进制往返、损坏和截断、事务交错、检查点、 分段轮转以及崩溃注入。CLI 演示会输出四组 JSON 证据。基准工作负载包含 10,000 个事务、30,000 条记录，并对一个 100 事务样本检查所有字节切断点。

#设计边界

本项目提供可嵌入的日志格式和恢复算法，不直接执行 fsync，也不替调用方决定 文件命名、目录布局、刷盘策略或并发模型。这样的边界既避免平台 API 污染，也让 浏览器持久化、对象存储、本地文件和自定义块设备能够共享同一恢复语义。

格式细节见 docs/FORMAT.md，崩溃安全约束见 docs/SAFETY.md，社区项目差异见 docs/RELATED_WORK.md。

许可证：Apache-2.0。

#
CorruptionSweepReport

source
pub(all) struct CorruptionSweepReport {
  bits_checked : Int
  corruptions_detected : Int
  undetected : Int
  passed : Bool
} derive(Eq, 
Debug)

Aggregate evidence from flipping every individual bit in a byte stream.

#
CorruptionSweepReport::to_json

source
fn CorruptionSweepReport::to_json(self : CorruptionSweepReport) -> String

Stable machine-readable corruption verification evidence.

#
CrashSweepReport

source
pub(all) struct CrashSweepReport {
  cuts_checked : Int
  exact_prefixes : Int
  unsafe_replays : Int
  passed : Bool
} derive(Eq, 
Debug)

Aggregate evidence from cutting a journal at every byte position.

#
CrashSweepReport::to_json

source
fn CrashSweepReport::to_json(self : CrashSweepReport) -> String

Stable machine-readable crash-cut verification evidence.

#
DecodeResult

source
pub(all) struct DecodeResult {
  status : DecodeStatus
  record : JournalRecord?
  next_offset : Int
  message : String
} derive(Eq, 
Debug)

Result of decoding one record at a byte offset.

#
DecodeStatus

source
pub(all) enum DecodeStatus {
  Decoded
  NeedMoreData
  Corrupt
  Unsupported
} derive(Eq, 
Debug)

Outcome category for bounded, non-throwing record decoding.

#
JournalRecord

source
pub(all) struct JournalRecord {
  kind : RecordKind
  sequence : UInt
  transaction : UInt
  payload : Bytes
} derive(Eq, 
Debug)

One decoded journal record.

#
JournalRecord::encoded_size

source
fn JournalRecord::encoded_size(self : JournalRecord) -> Int

Encoded byte size for this record.

#
JournalRecord::new

source
fn JournalRecord::new(kind : RecordKind, sequence : UInt, transaction : UInt, payload? : Bytes) -> JournalRecord

Creates one journal record.

#
Mutation

source
pub(all) struct Mutation {
  transaction : UInt
  sequence : UInt
  kind : MutationKind
  key : Bytes
  value : Bytes
} derive(Eq, 
Debug)

One key mutation associated with its transaction and source sequence.

#
MutationDecode

source
pub(all) struct MutationDecode {
  valid : Bool
  mutation : Mutation?
  message : String
} derive(Eq, 
Debug)

Bounded result of decoding a put or delete payload.

#
MutationKind

source
pub(all) enum MutationKind {
  PutValue
  DeleteKey
} derive(Eq, 
Debug)

A committed state change reconstructed from the journal.

#
RecordKind

source
pub(all) enum RecordKind {
  Begin
  Put
  Delete
  Commit
  Abort
  Checkpoint
} derive(Eq, 
Debug)

Semantic record kinds supported by the transactional journal.

#
RecoveryIssue

source
pub(all) struct RecoveryIssue {
  code : String
  sequence : UInt
  transaction : UInt
  message : String
} derive(Eq, 
Debug)

A semantic journal problem that does not compromise byte-level framing.

#
RecoveryPlan

source
pub(all) struct RecoveryPlan {
  actions : Array[Mutation]
  committed_transactions : Int
  aborted_transactions : Int
  incomplete_transactions : Array[UInt]
  checkpoint_sequence : UInt
  issues : Array[RecoveryIssue]
  recoverable : Bool
} derive(Eq, 
Debug)

Deterministic replay plan produced from a trusted record prefix.

#
RecoveryPlan::to_json

source
fn RecoveryPlan::to_json(self : RecoveryPlan) -> String

Stable machine-readable recovery summary and diagnostics.

#
ScanResult

source
pub(all) struct ScanResult {
  records : Array[JournalRecord]
  valid_bytes : Int
  total_bytes : Int
  stop : ScanStop
  fault_offset : Int
  message : String
} derive(Eq, 
Debug)

Result of scanning a byte stream up to its last trusted boundary.

#
ScanResult::to_json

source
fn ScanResult::to_json(self : ScanResult) -> String

Stable machine-readable scan summary.

#
ScanResult::valid_prefix

source
fn ScanResult::valid_prefix(self : ScanResult, data : BytesView) -> Bytes

Returns the trusted prefix suitable for crash-tail truncation.

#
ScanStop

source
pub(all) enum ScanStop {
  CleanEnd
  TruncatedTail
  Corruption
  UnsupportedVersion
  SequenceViolation
  RecordLimit
} derive(Eq, 
Debug)

Why a journal scan stopped.

#
SegmentMeta

source
pub(all) struct SegmentMeta {
  id : UInt
  first_sequence : UInt
  last_sequence : UInt
  record_count : Int
  byte_size : Int
} derive(Eq, 
Debug)

Immutable metadata for one planned journal segment.

#
SegmentPlan

source
pub(all) struct SegmentPlan {
  segments : Array[SegmentMeta]
  total_records : Int
  total_bytes : Int
  valid : Bool
  message : String
} derive(Eq, 
Debug)

Result of deterministic journal rotation planning.

#
StreamFeedResult

source
pub(all) struct StreamFeedResult {
  records : Array[JournalRecord]
  buffered_bytes : Int
  stop : ScanStop?
  message : String
} derive(Eq, 
Debug)

Result of one incremental decode operation.

#
StreamJournalDecoder

source
pub(all) struct StreamJournalDecoder {
  strict_sequence : Bool
  max_payload : Int
  max_buffered : Int
  pending : Bytes
  previous_sequence : UInt
} derive(Eq, 
Debug)

A bounded incremental decoder for append-only journal byte streams.

Feed arbitrary chunks from an I/O adapter. Complete records are emitted once, while a partial final record remains buffered for the next chunk. The core never opens files or browser storage itself.

#
StreamJournalDecoder::buffered_bytes

source
fn StreamJournalDecoder::buffered_bytes(self : StreamJournalDecoder) -> Int

#
StreamJournalDecoder::feed

source
fn StreamJournalDecoder::feed(self : StreamJournalDecoder, chunk : BytesView) -> StreamFeedResult

Decodes complete records without retaining previously emitted records.

#
StreamJournalDecoder::new

source
fn StreamJournalDecoder::new(strict_sequence? : Bool, max_payload? : Int, max_buffered? : Int) -> StreamJournalDecoder

#
StreamJournalDecoder::reset

source
fn StreamJournalDecoder::reset(self : StreamJournalDecoder) -> Unit

#
JOURNAL_HEADER_SIZE

source
let JOURNAL_HEADER_SIZE : Int

Fixed header size in bytes.

#
JOURNAL_VERSION

source
let JOURNAL_VERSION : Byte

Fixed wire-format version.

#
build_recovery_plan

source
fn build_recovery_plan(records : Array[JournalRecord], max_key? : Int) -> RecoveryPlan

Builds a replay plan. Only mutations from explicitly committed transactions are returned; aborted and crash-incomplete transactions never leak through.

#
crc32c

source
fn crc32c(data : BytesView) -> UInt

Computes CRC32C (Castagnoli) with the reflected polynomial.

#
crc32c_extend

source
fn crc32c_extend(state : UInt, data : BytesView) -> UInt

Incrementally updates a CRC32C state before final xor.

#
decode_mutation

source
fn decode_mutation(record : JournalRecord, max_key? : Int) -> MutationDecode

Decodes a put or delete record without allocating before bounds checks pass.

#
decode_record

source
fn decode_record(data : BytesView, offset? : Int, max_payload? : Int) -> DecodeResult

Decodes one record without reading beyond available bytes.

#
encode_checkpoint_payload

source
fn encode_checkpoint_payload(sequence : UInt) -> Bytes

Encodes the highest sequence known to be durable in a checkpoint payload.

#
encode_delete_payload

source
fn encode_delete_payload(key : BytesView) -> Bytes

Encodes a delete payload as key length followed by key bytes.

#
encode_put_payload

source
fn encode_put_payload(key : BytesView, value : BytesView) -> Bytes

Encodes a put payload as key length, key bytes, then value bytes.

#
encode_record

source
fn encode_record(record : JournalRecord) -> Bytes

Encodes one checksummed record in the portable MWAL format.

#
encode_records

source
fn encode_records(records : Array[JournalRecord]) -> Bytes

Encodes records in input order into one journal byte stream.

#
plan_segments

source
fn plan_segments(records : Array[JournalRecord], max_bytes : Int, max_records : Int, first_id? : UInt) -> SegmentPlan

Plans segment rotation without splitting a record. A single oversized record receives its own segment so that forward progress remains deterministic.

#
record_kind_name

source
fn record_kind_name(kind : RecordKind) -> String

Stable lower-case name used by reports and diagnostics.

#
recovery_segment_index

source
fn recovery_segment_index(segments : Array[SegmentMeta], checkpoint_sequence : UInt) -> Int

Returns the first segment that may contain records newer than a checkpoint. A return value equal to segment count means no segment needs replay.

#
scan_journal

source
fn scan_journal(data : BytesView, strict_sequence? : Bool, max_payload? : Int, max_records? : Int) -> ScanResult

Scans records without accepting bytes after the first invalid boundary.

#
sweep_crash_cuts

source
fn sweep_crash_cuts(records : Array[JournalRecord]) -> CrashSweepReport

Exhaustively cuts the encoded journal after every byte and validates that scanning returns an exact record prefix and recovery never invents a commit.

#
sweep_single_bit_corruption

source
fn sweep_single_bit_corruption(records : Array[JournalRecord]) -> CorruptionSweepReport

Flips each bit independently and verifies that framing, checksums, or sequence validation reject every modified stream.

#
validate_segment_chain

source
fn validate_segment_chain(segments : Array[SegmentMeta]) -> Bool

Validates IDs, sequence continuity, counts, and non-empty segment metadata.

Source Files

Powered by MoonBit

Site sourceReport issuePackagesBuild queueSkillsStatistics

© 2026 mooncakes.io