Add byte_order standard library package

New byte_order package with BigEndian and LittleEndian primitives for reading and writing multi-byte values in a specific byte order.
6 files changed +858 −0

Motivation

Closes ponylang/rfcs#112. Binary protocols and file formats require reading and writing integers in a specified byte order. Without a stdlib package, every project implements its own byte-swapping logic.

Read methods are generic over ReadSeq[U8], so they work with any byte sequence type. When the concrete type is Array[U8], iftype specialization selects an optimized path that uses native-width loads and conditional byte swaps. The generic path uses shift-and-OR.

File Map

▶ Release notes 1 file

Behavioral Changes

A new use "byte_order" import becomes available. The package provides two primitives that convert between native integers and their big-endian or little-endian byte representations.

Each primitive has 22 public methods: 10 read methods (U16, U32, U64, U128, I16, I32, I64, I128, F32, F64) that decode from a byte sequence, and 10 write methods that encode by appending bytes to an Array[U8]. Unsigned read methods (u16, u32, u64, u128) each have an iftype-specialized variant for Array[U8] that uses native-width loads and byte swaps instead of the generic shift-and-OR path. Signed reads delegate to unsigned, float reads delegate via bits()/from_bits().

All read methods are partial — they raise an error when the sequence has too few bytes at the given offset.

Public API Surface

New primitive BigEndian
Read and write multi-byte values in big-endian (network) byte order. Read methods are generic over any ReadSeq[U8]. When the concrete sequence is an Array[U8], an optimized path uses a single native-width load and a conditional byte swap.
  • fun u16[S: ReadSeq[U8] box](seq: S, offset: USize = 0): U16 ?
    Read a big-endian U16 from seq at offset.
  • fun u32[S: ReadSeq[U8] box](seq: S, offset: USize = 0): U32 ?
    Read a big-endian U32 from seq at offset.
  • fun u64[S: ReadSeq[U8] box](seq: S, offset: USize = 0): U64 ?
    Read a big-endian U64 from seq at offset.
  • fun u128[S: ReadSeq[U8] box](seq: S, offset: USize = 0): U128 ?
    Read a big-endian U128 from seq at offset.
  • fun i16[S: ReadSeq[U8] box](seq: S, offset: USize = 0): I16 ?
    Read a big-endian I16 from seq at offset.
  • fun i32[S: ReadSeq[U8] box](seq: S, offset: USize = 0): I32 ?
    Read a big-endian I32 from seq at offset.
  • fun i64[S: ReadSeq[U8] box](seq: S, offset: USize = 0): I64 ?
    Read a big-endian I64 from seq at offset.
  • fun i128[S: ReadSeq[U8] box](seq: S, offset: USize = 0): I128 ?
    Read a big-endian I128 from seq at offset.
  • fun f32[S: ReadSeq[U8] box](seq: S, offset: USize = 0): F32 ?
    Read a big-endian F32 from seq at offset.
  • fun f64[S: ReadSeq[U8] box](seq: S, offset: USize = 0): F64 ?
    Read a big-endian F64 from seq at offset.
  • fun push_u16(into: Array[U8] ref, value: U16)
    Append a U16 to into in big-endian byte order.
  • fun push_u32(into: Array[U8] ref, value: U32)
    Append a U32 to into in big-endian byte order.
  • fun push_u64(into: Array[U8] ref, value: U64)
    Append a U64 to into in big-endian byte order.
  • fun push_u128(into: Array[U8] ref, value: U128)
    Append a U128 to into in big-endian byte order.
  • fun push_i16(into: Array[U8] ref, value: I16)
    Append an I16 to into in big-endian byte order.
  • fun push_i32(into: Array[U8] ref, value: I32)
    Append an I32 to into in big-endian byte order.
  • fun push_i64(into: Array[U8] ref, value: I64)
    Append an I64 to into in big-endian byte order.
  • fun push_i128(into: Array[U8] ref, value: I128)
    Append an I128 to into in big-endian byte order.
  • fun push_f32(into: Array[U8] ref, value: F32)
    Append an F32 to into in big-endian byte order.
  • fun push_f64(into: Array[U8] ref, value: F64)
    Append an F64 to into in big-endian byte order.
New primitive LittleEndian
Read and write multi-byte values in little-endian byte order. Same method set as BigEndian; byte order is reversed.
  • fun u16[S: ReadSeq[U8] box](seq: S, offset: USize = 0): U16 ?
    Read a little-endian U16 from seq at offset.
  • fun u32[S: ReadSeq[U8] box](seq: S, offset: USize = 0): U32 ?
    Read a little-endian U32 from seq at offset.
  • fun u64[S: ReadSeq[U8] box](seq: S, offset: USize = 0): U64 ?
    Read a little-endian U64 from seq at offset.
  • fun u128[S: ReadSeq[U8] box](seq: S, offset: USize = 0): U128 ?
    Read a little-endian U128 from seq at offset.
  • fun i16[S: ReadSeq[U8] box](seq: S, offset: USize = 0): I16 ?
    Read a little-endian I16 from seq at offset.
  • fun i32[S: ReadSeq[U8] box](seq: S, offset: USize = 0): I32 ?
    Read a little-endian I32 from seq at offset.
  • fun i64[S: ReadSeq[U8] box](seq: S, offset: USize = 0): I64 ?
    Read a little-endian I64 from seq at offset.
  • fun i128[S: ReadSeq[U8] box](seq: S, offset: USize = 0): I128 ?
    Read a little-endian I128 from seq at offset.
  • fun f32[S: ReadSeq[U8] box](seq: S, offset: USize = 0): F32 ?
    Read a little-endian F32 from seq at offset.
  • fun f64[S: ReadSeq[U8] box](seq: S, offset: USize = 0): F64 ?
    Read a little-endian F64 from seq at offset.
  • fun push_u16(into: Array[U8] ref, value: U16)
    Append a U16 to into in little-endian byte order.
  • fun push_u32(into: Array[U8] ref, value: U32)
    Append a U32 to into in little-endian byte order.
  • fun push_u64(into: Array[U8] ref, value: U64)
    Append a U64 to into in little-endian byte order.
  • fun push_u128(into: Array[U8] ref, value: U128)
    Append a U128 to into in little-endian byte order.
  • fun push_i16(into: Array[U8] ref, value: I16)
    Append an I16 to into in little-endian byte order.
  • fun push_i32(into: Array[U8] ref, value: I32)
    Append an I32 to into in little-endian byte order.
  • fun push_i64(into: Array[U8] ref, value: I64)
    Append an I64 to into in little-endian byte order.
  • fun push_i128(into: Array[U8] ref, value: I128)
    Append an I128 to into in little-endian byte order.
  • fun push_f32(into: Array[U8] ref, value: F32)
    Append an F32 to into in little-endian byte order.
  • fun push_f64(into: Array[U8] ref, value: F64)
    Append an F64 to into in little-endian byte order.

Architecture

Both primitives are stateless and share the same structure. Read methods dispatch at compile time via iftype specialization:

Read path dispatch (compile-time) u16 / u32 / u64 / u128 S <: Array[U8] box other ReadSeq[U8] read_uN() + ifdef bswap shift-and-OR via apply() single load + 0–1 bswap instructions N byte reads + N−1 shifts + N−1 ORs Write path (no specialization needed) push_uN: shift + push bytes in order

Signed reads (i16–i128) delegate to the unsigned read and cast. Float reads (f32, f64) delegate via bits()/from_bits(). Write methods are the inverse: signed and float writes delegate to unsigned, which pushes individual bytes in the appropriate order.

The difference between BigEndian and LittleEndian is two-fold: the ifdef guard checks bigendian vs. littleendian in the Array path, and the shift-and-OR generic path reverses which end of the value the low-offset bytes map to.

Release Notes

Add byte_order standard library package

A new byte_order package provides BigEndian and LittleEndian primitives for converting multi-byte values to and from their wire representation.

Read methods decode from any ReadSeq[U8] source (with an optimized path for Array[U8]). Write methods append encoded bytes to an Array[U8]. Both primitives cover U16–U128, their signed counterparts, and F32/F64.

use "byte_order"

// Encode a U32 in network (big-endian) order
let buf = Array[U8]
BigEndian.push_u32(buf, 0xDEADBEEF)
// buf is now [0xDE; 0xAD; 0xBE; 0xEF]

// Decode it back
let value = BigEndian.u32[Array[U8] box](buf)?  // 0xDEADBEEF