🥮 mooncakes.io

    #UTF-8

    Encoding and decoding between strings and UTF-8 byte sequences.

    #Encoding

    Use encode to convert a string to UTF-8 bytes. Set bom=true to prepend the UTF-8 BOM (U+FEFF).

    For unpaired UTF-16 surrogates, the JavaScript backend replaces each one with U+FFFD, following TextEncoder; other backends panic. Well-formed strings encode identically on all backends.

    ///|
    test "encode" {
      let bytes = @utf8.encode("hi")
      inspect(bytes, content="b\"hi\"")
    }

    ///|
    test "encode_with_bom" {
      let bytes = @utf8.encode("hi", bom=true)
      inspect(bytes, content="b\"\\xef\\xbb\\xbfhi\"")
    }

    #Decoding

    Use decode to convert UTF-8 bytes back to a string. Raises Malformed on invalid UTF-8 sequences. Set ignore_bom=true to strip a leading BOM if present.

    ///|
    test "decode" {
      let bytes : Bytes = b"\x68\x69"
      let s = @utf8.decode(bytes)
      inspect(s, content="hi")
    }

    #Lossy Decoding

    Use decode_lossy to decode bytes that may contain invalid UTF-8, replacing invalid sequences with the Unicode replacement character (U+FFFD).

    ///|
    test "decode_lossy" {
      let bytes : Bytes = b"\x68\x80\x69"
      let s = @utf8.decode_lossy(bytes)
      inspect(s, content="h\u{FFFD}i")
    }

    Malformed

    source
    pub suberror Malformed {
      Malformed(BytesView)
    } derive(
    Debug    )

    Error type Malformed.

    decode

    source
    fn decode(bytes : BytesView, ignore_bom? : Bool) -> String raise Malformed

    Decode input bytes/text into structured output.

    decode_lossy

    source
    fn decode_lossy(bytes : BytesView, ignore_bom? : Bool) -> String

    References :
    • https://www.unicode.org/versions/Unicode16.0.0/core-spec/chapter-3/#G66453
    • https://www.unicode.org/versions/Unicode16.0.0/core-spec/chapter-5/#G40630

    encode

    source
    fn encode(str : StringView, bom? : Bool) -> Bytes

    Encodes a string into a UTF-8 byte array.

    Panics if the string contains an invalid surrogate pair.

    Source Files

    Powered by MoonBit

    Site sourceReport issuePackagesBuild queueSkillsStatistics

    © 2026 mooncakes.io