#bytes

This package provides utilities for working with sequences of bytes, offering both an owned representation (Bytes) and a slice representation (BytesView); both are immutable.

#Creating Bytes

You can create Bytes from various sources including arrays, fixed arrays, and iterators:

///|
test "bytes creation" {
  // Create from array of bytes
  let arr = [b'h', b'e', b'l', b'l', b'o']
  let bytes1 = Bytes::from_array(arr)
  inspect(
    bytes1,
    content=(
      #|b"hello"
    ),
  )

  // Create from fixed array
  let fixed = FixedArray::make(3, b'a')
  let bytes2 = Bytes::from_array(fixed)
  inspect(
    bytes2,
    content=(
      #|b"aaa"
    ),
  )

  // Create empty bytes
  let empty = (Default::default() : Bytes)
  inspect(
    empty,
    content=(
      #|b""
    ),
  )

  // Create from iterator
  let iter_bytes = Bytes::from_iter(arr.iter())
  inspect(
    iter_bytes,
    content=(
      #|b"hello"
    ),
  )
}

#Converting Between Formats

Bytes can be converted to and from different formats:

///|
test "bytes conversion" {
  let original : ReadOnlyArray[Byte] = [b'x', b'y', b'z']
  let bytes = Bytes::from_array(original)

  // Convert to array
  let array = bytes.to_array()
  debug_inspect(
    array,
    content=(
      #|[0x78, 0x79, 0x7a]
    ),
  )

  // Convert to fixed array
  let fixed = bytes.to_fixedarray()
  debug_inspect(
    fixed,
    content=(
      #|<FixedArray: [0x78, 0x79, 0x7a]>
    ),
  )

  // Convert to iterator and collect back
  let collected = bytes.iter().to_array()
  debug_inspect(
    collected,
    content=(
      #|[0x78, 0x79, 0x7a]
    ),
  )
}

#Working with Views

Use bytes.exact_view(start~, end~) to validate bounds, bytes.get_view(start~, end~) to return None for invalid bounds, or bytes.clamped_view(start~, end~) to clamp them. These methods are also available on BytesView. The old view and sub names are deprecated in favor of exact_view. For bounds already validated by the caller, unsafe_view(start~, end~) skips all bounds checks. It requires 0 <= start <= end <= length(); violating that precondition can cause undefined behavior.

Views provide a way to work with portions of bytes and interpret them as various numeric types:

For Bytes and BytesView, bytes[start:end] clamps each bound to the current length, treats negative bounds as zero, and returns an empty view for an inverted range. Omitted bounds select the corresponding end of the bytes or view. Use bytes.exact_view(start~, end~) to require valid bounds, or bytes.get_view(start~, end~) to return None for an invalid range.

///|
test "bytes view operations" {
  // Create bytes with numeric data
  let num_bytes = Bytes::from_array([0x12, 0x34, 0x56, 0x78])

  // Create a view
  let view = num_bytes[:]

  // Get individual bytes
  inspect(view[0], content="b'\\x12'")

  // Interpret as integers (big-endian)
  guard view is [i32be(x), ..] else {
    fail("Failed to match big-endian integer pattern")
  }
  inspect(x, content="305419896")

  // Interpret as integers (little-endian)
  guard view is [i32le(y), ..] else {
    fail("Failed to match little-endian integer pattern")
  }
  inspect(y, content="2018915346")

  // Create a sub-view
  let sub_view = view[1:3]
  inspect(sub_view.length(), content="2")
}

#Binary Data Interpretation

Views provide methods to interpret byte sequences as various numeric types in both little-endian and big-endian formats:

///|
test "numeric interpretation" {
  // Create test data
  let int64_bytes = Bytes::from_array([
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x42,
  ])
  guard int64_bytes is [i64be(x), ..] else {
    fail("Failed to match big-endian int64 pattern")
  }
  inspect(x, content="66")
  guard int64_bytes is [u64le(x), ..] else {
    fail("Failed to match little-endian uint64 pattern")
  }
  inspect(x, content="4755801206503243776")
}

#Concatenation and Comparison

Bytes can be concatenated and compared:

///|
test "bytes operations" {
  let b1 = Bytes::from_array([b'a', b'b'])
  let b2 = Bytes::from_array([b'c', b'd'])

  // Concatenation
  let combined = b1 + b2
  inspect(
    combined,
    content=(
      #|b"abcd"
    ),
  )

  // Comparison
  let same = Bytes::from_array([b'a', b'b'])
  let different = Bytes::from_array([b'x', b'y'])
  inspect(b1 == same, content="true")
  inspect(b1 == different, content="false")
  inspect(b1 < b2, content="true")
}

#Prefixes, Suffixes, and Chopping

You can check for prefixes and suffixes or remove them when present:

///|
test "bytes prefix/suffix" {
  let bytes = b"hello"
  inspect(bytes.has_prefix(b"he"), content="true")
  inspect(bytes.has_suffix(b"lo"), content="true")
  debug_inspect(
    bytes.strip_prefix(b"he"),
    content="Some(<BytesView: [0x6c, 0x6c, 0x6f]>)",
  )
  debug_inspect(
    bytes.strip_suffix(b"lo"),
    content="Some(<BytesView: [0x68, 0x65, 0x6c]>)",
  )
  debug_inspect(bytes.strip_prefix(b"zz"), content="None")
}

#How lexmatch Uses BytesRegex

With regex_backend = "runtime" in moon.pkg, a first-match lexmatch on Bytes or BytesView is compiled to BytesRegex searches through BytesRegex::internal_search. It works as Regex::internal_search does, which the string package's README describes, with byte offsets:

///|
test "internal_search offsets" {
  let regex = @bytes.BytesRegex::BytesRegex("(a)|(b)")
  guard regex.internal_search(b"xa") is Some(offsets) else { fail("no match") }
  debug_inspect(offsets, content="<FixedArray: [1, 2, 1, 2, -1, -1]>")
}

Use execute in hand-written code. internal_search is pub only for the code lexmatch compiles to: the next search with the same BytesRegex overwrites the array it returns, and -1 marks a group that did not participate.

View

source

using @moonbitlang/core/builtin { type BytesView as View }

Type View used by this package APIs.

BytesRegex

source

pub struct BytesRegex {
  // private fields
}

A compiled regular expression over bytes: the Bytes counterpart of @string.Regex, with its own byte-specialized parser and engine.

impl Add for BytesRegex

fn add(self : BytesRegex, other : BytesRegex) -> BytesRegex

source

Concatenates two regexes in sequence.

Example:

test {
  let regex = @bytes.BytesRegex("ab") + BytesRegex("\\xff")
  guard regex.execute(b"xab\xff") is Some(m) else { fail("Expected match") }
  inspect(m.content(), content="b\"ab\\xff\"")
}

impl BitOr for BytesRegex

fn lor(self : BytesRegex, other : BytesRegex) -> BytesRegex

source

Builds an alternation that matches either regex.

Example:

test {
  let regex = @bytes.BytesRegex("^GET$") | BytesRegex("^POST$")
  inspect(regex.execute(b"POST") is Some(_), content="true")
  inspect(regex.execute(b"PUT") is Some(_), content="false")
}

impl Debug for BytesRegex

BytesRegex::BytesRegex

source

fn BytesRegex::BytesRegex(pattern : StringView) -> BytesRegex raise

Compiles a regex pattern string into a BytesRegex.

The syntax is that of @string.Regex, read over bytes:

An atom is one byte, and . matches any byte, including newline.
\xHH is the byte 0xHH; it may appear in character classes and ranges, as in [\x80-\xff].
The pattern itself is ASCII: a non-ASCII character, \uXXXX and \u{X...} are errors. Write a byte above 0x7F as \xHH.
(?i: ... ), \b, \B and the POSIX classes such as [[:alpha:]] only know ASCII. Every byte above 0x7F is a non-word byte.
Negated classes such as [^a] match every other byte, including bytes above 0x7F.

Everything else, from quantifiers and groups to anchors and error positions, is as in @string.Regex. Positions are byte offsets.

Raises when pattern is not a valid bytes regex pattern.

Example:

test {
  let regex = @bytes.BytesRegex("\\xff+(?<rest>[[:digit:]]+)")
  guard regex.execute(b"a\xff\xff42") is Some(m) else { fail("Expected match") }
  inspect(m.content(), content="b\"\\xff\\xff42\"")
  inspect(m.named_group("rest").unwrap(), content="b\"42\"")
}

BytesRegex::add

source

fn BytesRegex::add(self : BytesRegex, other : BytesRegex) -> BytesRegex

BytesRegex::execute

source

fn BytesRegex::execute(self : BytesRegex, input : BytesView, last_index? : Int) -> MatchResult?

Search input for the leftmost-first match, as @string.Regex::execute does for strings.

Search starts at last_index (default 0), a byte offset that must satisfy 0 <= last_index <= input.length(). The returned match, when present, starts at or after that index.

last_index only controls where searching starts. It does not change anchor semantics: ^ still matches only the beginning of input and $ only its end, and \b looks at the byte before last_index.

Returns None when there is no match from last_index to the end of input.

Example:

test {
  let regex = @bytes.BytesRegex("[[:digit:]]+")
  let input = b"a12b34"
  guard regex.execute(input) is Some(first) else {
    fail("Expected first match")
  }
  inspect(first.content(), content="b\"12\"")
  let next = first.before().length() + first.content().length()
  guard regex.execute(input, last_index=next) is Some(second) else {
    fail("Expected second match")
  }
  inspect(second.content(), content="b\"34\"")
  let anchored = @bytes.BytesRegex("^ab$")
  inspect(anchored.execute(b"xaby", last_index=1) is Some(_), content="false")
}

BytesRegex::lor

source

fn BytesRegex::lor(self : BytesRegex, other : BytesRegex) -> BytesRegex

MatchResult

source

type MatchResult derive(@moonbitlang/core/debug.Debug
Debug)

Result of one successful match produced by BytesRegex::execute.

MatchResult::after

source

fn MatchResult::after(self : MatchResult) -> BytesView

Return match after view.

MatchResult::before

source

fn MatchResult::before(self : MatchResult) -> BytesView

Return match before view.

MatchResult::content

source

fn MatchResult::content(self : MatchResult) -> BytesView

Return match content view.

MatchResult::group

source

fn MatchResult::group(self : MatchResult, group_index : Int) -> BytesView?

Returns a capture group; group 0 is the entire match. Returns None for an out-of-range index or a group that did not participate.

MatchResult::named_group

source

fn MatchResult::named_group(self : MatchResult, name : String) -> BytesView?

Access capture named_group information.

Source Files

Install

Installed by default

corestandard library

moonbitlang

Apache-2.0

github.com/moonbitlang/core

Updated 21 hours ago

Site source Report issue Modules Skills Build queue Statistics

#bytes

#Creating Bytes

#Converting Between Formats

#Working with Views

#Binary Data Interpretation

#Concatenation and Comparison

#Prefixes, Suffixes, and Chopping

#How lexmatch Uses BytesRegex

#View

#BytesRegex

#BytesRegex::BytesRegex

#BytesRegex::add

#BytesRegex::execute

#BytesRegex::lor

#MatchResult

#MatchResult::after

#MatchResult::before

#MatchResult::content

#MatchResult::group

#MatchResult::named_group

Source Files

View

BytesRegex

BytesRegex::BytesRegex

BytesRegex::add

BytesRegex::execute

BytesRegex::lor

MatchResult

MatchResult::after

MatchResult::before

MatchResult::content

MatchResult::group

MatchResult::named_group