Add binary_codec package
Codec[A] once and it handles both decoding bytes into A and encoding A back to bytes.
Motivation
Pony programs that speak binary protocols (TCP, UDP, file formats) need to decode incoming bytes and encode outgoing values. Without a library, every project writes its own ad-hoc parsing with manual offset arithmetic, byte-shifting, and error checking. This is repetitive, error-prone, and impossible to test in isolation.
The binary_codec package provides a composable, type-safe approach: small primitive codecs (U16BE, BitsU8(4), Bytes(16)) compose into larger structures via combinators (Seq4, Mapped, FlatZip, Discriminated). A single codec definition drives both directions, so parsing and serialization can't diverge.
The library is bit-native. Sub-byte fields (4-bit nibbles, 1-bit flags, 13-bit offsets) work the same way as byte-aligned fields. Errors are data: InsufficientData means "wait for more bytes," MalformedData means "this data is wrong." FrameDecoder uses this distinction to implement zero-copy TCP stream framing.
File Map
- codec.pony +90
- binary_codec.pony +60
- frame_notify.pony +15
- primitives.pony +513
- bits.pony +131
- bytes.pony +119
- seq.pony +281
- combinators.pony +219
- mapped_codec.pony +73
- discriminated.pony +120
- repetition.pony +101
- frame_decoder.pony +49
- _test.pony +841
- _bit_cursor.pony +78
- _bit_writer.pony +69
Public API Surface
Core trait and error types
val — immutable and shareable across actors. The bit offset lets sub-byte fields compose naturally.- fun decode(data: Array[U8] val, offset: USize): ((A, USize) | DecodeError)
- fun encode(value: A): (Array[U8] val | EncodeError)
- fun string(): String iso^
- new val create(message': String, bit_offset': USize = 0)
- fun message(): String
- fun bit_offset(): USize
- fun string(): String iso^
decode.- new val create(message': String)
- fun message(): String
- fun string(): String iso^
encode.Byte-aligned primitives
Bool.Bit-width codecs
BitsU8(4) reads a nibble.- new val create(count: U8)
- new val create(count: U8)
- new val create(count: U8)
- new val create(count: U8)
Byte and literal codecs
count bytes. Requires byte alignment. Returns a zero-copy trim view of the input array.- new val create(count: USize)
MalformedData if the bytes don't match. The decoded value is None.- new val create(expected: Array[U8] val)
Sequence combinators
- new val create(a: Codec[A], b: Codec[B])
- new val create(a: Codec[A], b: Codec[B], c: Codec[C])
- new val create(a: Codec[A], b: Codec[B], c: Codec[C], d: Codec[D])
- new val create(a: Codec[A], b: Codec[B], c: Codec[C], d: Codec[D], e: Codec[E])
- new val create(a: Codec[A], b: Codec[B], c: Codec[C], d: Codec[D], e: Codec[E], f: Codec[F])
Structural combinators
A and a domain type B. The forward function may fail (partial) to signal malformed data.- new val create(c: Codec[A], f: {(A): B ?} val, r: {(B): A} val)
V to a broader union Msg. Used internally by DiscriminatedBuilder.- new val create(codec: Codec[V], narrow: {(Msg): V ?} val)
- new val create(codec: Codec[A], literal: Literal)
- new val create(literal: Literal, codec: Codec[A])
A is available from the tuple so the same dispatch works in reverse.- new val create(ca: Codec[A], f: {(A): Codec[B]} val)
- new val create(size_codec: Codec[U16], body_codec: Codec[A])
None.- new val create(flag: Bool, codec: Codec[A])
- new val create(count: USize)
Repetition
- new val create(count_codec: Codec[U16], body_codec: Codec[A])
- new val create(codec: Codec[A])
Tagged union dispatch
case_of, then call build().- new ref create(disc_codec: Codec[D])
- fun ref case_of[V: Msg](disc: D, codec: Codec[V], narrow: {(Msg): V ?} val): DiscriminatedBuilder[D, Msg] ref
- fun ref build(): Discriminated[D, Msg]
TCP stream framing
iso chunks are concatenated with unconsumed remainder into a val buffer for zero-copy decoding. InsufficientData means "wait for more bytes"; MalformedData is reported via FrameNotify.- new ref create(codec: Codec[A])
- fun ref received(data: Array[U8] iso, notify: FrameNotify[A] ref)
- fun ref frame_received(value: A)
- fun ref frame_error(err: MalformedData)
Architecture
Primitive codecs (U16BE, BitsU8, Bytes, etc.) implement the Codec trait directly. Combinators compose codecs: Seq for tuples, Mapped for type mapping, FlatZip for dependent parsing, Discriminated for tagged unions, Many/Count for repetition. All combinators are themselves codecs, so they compose recursively.
FrameDecoder wraps any composed codec for TCP streaming, using the InsufficientData/MalformedData distinction to separate "wait for more" from "corrupt data." It notifies the application through the FrameNotify interface.
Release Notes
Add binary_codec package
A new standard library package for bidirectional binary parsing. Define a Codec[A] once — the same definition decodes bytes into a value and encodes the value back to bytes.
Sub-byte fields (4-bit nibbles, 1-bit flags, 13-bit offsets) work the same as byte-aligned fields. decode takes Array[U8] val so byte-slice codecs return zero-copy views via Array.trim.
Codecs are val and compose via combinators: Seq2–Seq6 sequence fields into tuples, Mapped maps between wire tuples and domain types, FlatZip threads a decoded value into a dependent codec, Many and Count handle repetition, and Discriminated dispatches on a tag value.
Decode errors are data: InsufficientData means the input is incomplete, MalformedData means the data is structurally wrong. FrameDecoder wraps any codec for TCP stream framing and uses this distinction to separate “not enough data yet” from “corrupt message.”
use "binary_codec"
class val UdpHeader
let src_port: U16
let dst_port: U16
let length: U16
let checksum: U16
new val create(s: U16, d: U16, l: U16, c: U16) =>
src_port = s; dst_port = d; length = l; checksum = c
actor Main
new create(env: Env) =>
let codec: Codec[UdpHeader] =
Mapped[(U16, U16, U16, U16), UdpHeader](
Seq4[U16, U16, U16, U16](U16BE, U16BE, U16BE, U16BE),
{(t) => UdpHeader(t._1, t._2, t._3, t._4)},
{(h) => (h.src_port, h.dst_port, h.length, h.checksum)})
let data: Array[U8] val =
[0x00; 0x50; 0x00; 0x51; 0x00; 0x08; 0x12; 0x34]
match codec.decode(data, 0)
| (let h: UdpHeader, let bits: USize) =>
env.out.print("src=" + h.src_port.string())
env.out.print("dst=" + h.dst_port.string())
| InsufficientData =>
env.out.print("need more data")
| let e: MalformedData =>
env.out.print("bad data: " + e.string())
end