README- UCodecError
- UCodecLimits
- UCompressionCodec
- UDatum
- UDecoder
- UEncoder
- UField
- UJsonError
- UJsonLimits
- UNamedDefinition
- UOcfBlockConfig
- UOcfError
- UOcfHeader
- UOcfLimits
- UOcfReader
- UOcfWriter
- UPrimitive
- USchema
- USchemaError
- USchemaKind
- USingleObjectError
- Vdecode
- Vdecode_container
- Vdecode_from
- Vdecode_header
- Vdecode_single_object
- Vdecode_single_object_resolved
- Vdefault_block_config
- Vdefault_limits
- Vencode
- Vencode_container
- Vencode_container_blocks
- Vencode_header
- Vencode_into
- Vencode_single_object
- Vfrom_json
- Vis_ocf
- Vmagic
- Vparse
- Vparse_json
- Vsingle_object_fingerprint
- Vstringify
- Vto_json
moon-avro
Pure MoonBit Apache Avro schema, datum codec, and Object Container File toolkit.
avro
serialization
ocf
codec
data
Author
Version
0.3.0
License
Apache-2.0
Repository
Last updated
2 months ago
Downloads
211
#moon-avro
纯 MoonBit 的 Apache Avro 工具包，覆盖 Schema、通用 datum 二进制编解码、
Single-Object Encoding、schema-driven JSON 映射和 OCF 容器（null/raw
deflate）。核心 API 不依赖文件系统，输入错误会以结构化错误返回，并提供
递归、集合和字节大小限制。
#已实现能力
- Schema：primitive、record、enum、array、map、union、fixed；namespace/fullname、 aliases、命名引用、默认值校验、Parsing Canonical Form 和 CRC-64-AVRO fingerprint；
- Binary：zigzag/varint、little-endian float/double、UTF-8、严格尾随数据检查、 byte offset 诊断与 CodecLimits；
- Datum：所有核心 primitive 及 record/enum/array/map/union/fixed 递归编解码；
- OCF：header metadata、sync marker、按记录数/字节目标自动多块写入、null 与 标准 raw DEFLATE，各块可独立解压并校验；
- Single-Object Encoding：C3 01 marker、小端 CRC-64-AVRO fingerprint、 expected-schema 校验和可注入的 schema resolver；
- JSON：Avro JSON 约定、union 单键表示、record aliases/defaults、精确 Int64；
- CLI：schema inspect、encode、decode、ocf-info 和 roundtrip。
不包含 Avro RPC、Schema Registry、代码生成、Snappy 和 schema evolution reader。
OCF 当前 API 提供有序的内存 reader/writer 和有界多数据块 framing；大文件文件流
适配器不是本版本承诺。
#快速开始
moon check --target all --deny-warn
moon test --target all --deny-warn
moon run examples/basic
moon run examples/primitive
moon run examples/ocf
moon run cmd/moon_avro -- roundtrip
moon run cmd/moon_avro -- encode '"long"' '42' # 输出 54
moon run cmd/moon_avro -- decode '"long"' '54' # 输出 42在其他 MoonBit 项目中安装：
moon add yugonlian/moon-avro@0.3.0然后导入根包或子包：
import { "yugonlian/moon-avro" @avro }#验收与许可证
bash scripts/acceptance.sh 会更新注册表索引并执行格式、四目标 check/build/test、
示例和 CLI smoke；发布后可用 bash scripts/external-consumer-smoke.sh 验证独立消费者安装。
可用 bash scripts/benchmark.sh 100 获取本机趋势基线。
实现依据 Apache Avro Specification，
项目采用 Apache-2.0。
CodecError
Public facade for datum codec APIs.
CodecLimits
Public facade for datum codec APIs.
CompressionCodec
Public facade for Object Container File APIs.
JsonError
Public facade for schema-driven Avro JSON datum conversion.
JsonLimits
Public facade for schema-driven Avro JSON datum conversion.
NamedDefinition
Public facade for schema APIs.
OcfBlockConfig
Public facade for Object Container File APIs.
OcfError
Public facade for Object Container File APIs.
OcfHeader
Public facade for Object Container File APIs.
OcfLimits
Public facade for Object Container File APIs.
OcfReader
Public facade for Object Container File APIs.
OcfWriter
Public facade for Object Container File APIs.
SchemaError
Public facade for schema APIs.
SingleObjectError
Public error type for Avro Single-Object Encoding operations.
decode
Decode one datum according to an Avro schema and reject trailing bytes.
decode_container
Decode an entire OCF file, checking block sizes, record counts, sync markers
and all datum payloads against the schema carried by its header.
decode_from
Decode one datum from a caller-owned stream decoder, leaving later bytes
available for the next datum.
decode_header
Decode only an OCF header. bytes_consumed is the first block offset.
decode_single_object
fn decode_single_object(schema : Schema, input : Bytes, limits? : CodecLimits) -> Datum raise SingleObjectErrorDecode one Single-Object Encoding value using an expected schema.
decode_single_object_resolved
fn decode_single_object_resolved(input : Bytes, resolve : (UInt64) -> Schema?, limits? : CodecLimits) -> (Schema, Datum) raise SingleObjectErrorResolve a Single-Object Encoding writer schema by fingerprint and decode
the enclosed datum.
default_block_config
A balanced default for ordinary in-memory files: at most 1,000 records or
approximately 256 KiB of uncompressed payload per data block.
default_limits
Conservative defaults suitable for untrusted in-memory OCF input.
encode
Encode one datum according to an Avro schema.
encode_container
Encode one complete Avro Object Container File as a single data block.
Use encode_container_blocks when output should be partitioned for bounded
decompression and incremental interoperability.
encode_container_blocks
Encode one complete OCF file using independently compressed data blocks.
Blocks are closed when either configured record count or target
uncompressed byte size is reached. Record order is preserved.
encode_header
Serialize an OCF header (magic, metadata and 16-byte sync marker).
encode_into
Encode one datum into a caller-owned stream encoder.
encode_single_object
Encode one datum using the Avro Single-Object Encoding envelope.
from_json
Convert a JSON value to an Avro datum using the supplied schema.
parse
Parse an Avro JSON schema, resolve named references, and retain named
declarations for later datum and OCF consumers.
parse_json
Parse JSON text into a datum without colliding with the schema parse API.
single_object_fingerprint
Extract the CRC-64-AVRO schema fingerprint from a single-object envelope.
stringify
Convert an Avro datum to compact JSON text according to its schema.
Source Files
Install
Download zipPure MoonBit Apache Avro schema, datum codec, and Object Container File toolkit.
avroserializationocfcodecdata
Apache-2.0
Updated 2 months ago
211 downloads