Mix Transport Implementation Walk Through - Wire Format Foundation

libp2p_mix_transport/wire.nim defines the Protobuf envelope exchanged by two MixTransport endpoints. The file is not merely a collection of data types: it is the boundary that rejects malformed combinations of fields before transport state is changed, converts public SURBs through Mix’s canonical serialization API and calculates how much application data fits in one Sphinx packet.

Package users obtain these public types through the root libp2p_mix_transport.nim facade.

Service Identity and Sphinx Capacity

const
  MixTransportCodec* = "/libp2p/mix-transport/1.0.0"
  MixTransportVersion* = 3'u32

MixTransportCodec is the service name registered with MixProtocol. A final-hop Mix delivery carrying this codec is passed to MixTransport rather than interpreted by Mix as a connection protocol. MixTransportVersion is inside the opaque service payload and lets the transport reject an incompatible envelope.

let MaxTransportFrameBytes* = getMaxMessageSizeForCodec(MixTransportCodec, 0).expect(
    "MixTransportCodec framing leaves no room for a transport frame"
  )

This calculates the Sphinx payload space left after the Mix service codec. The second argument is zero because MixTransport does not use the embedded legacy SURB envelope. When MixTransport needs public SURBs, it serializes them in its own control frames.

Frame Kinds

FrameKind* {.pure.} = enum
  Connect = 1
  ConnectAck = 2
  OpenStream = 3
  StreamAck = 4
  Data = 5
  Ack = 6
  CloseStream = 7
  ResetStream = 8
  Disconnect = 9
  ResetSession = 10
  StreamReject = 11
  SurbSupply = 12
  SurbStatusProbe = 13
  SurbStatus = 14

The numeric values are explicit wire assignments. Existing values must not change if the enum is reordered, and a removed value must not be reused for another meaning.

Each kind has a defined responsibility:

Frames	Responsibility
Connect, ConnectAck	Establish and confirm a long-lived transport session
OpenStream, StreamAck, StreamReject	Open or reject one virtual application stream
Data, Ack	Carry ordered bytes and absolute receive-window state
SurbSupply	Carry consecutively numbered individual public SURBs
SurbStatusProbe, SurbStatus	Recover supply-state synchronization when the recipient’s ordinary SURB queue can be empty
CloseStream, ResetStream	Finish one stream after its preceding Data or abort it immediately
Disconnect, ResetSession	Finish an idle session or abort the session and all of its streams

The Common Envelope

MixTransportFrame* {.proto2.} = object
  version* {.fieldNumber: 1, required, pint.}: uint32
  sessionId* {.fieldNumber: 2, required, ext.}: PeerId
  kind* {.fieldNumber: 3, required, ext.}: FrameKind
  streamId* {.fieldNumber: 4, fixed.}: Opt[StreamId]
  sequence* {.fieldNumber: 5, fixed.}: Opt[SequenceNumber]
  payload* {.fieldNumber: 6.}: Opt[seq[byte]]
  codec* {.fieldNumber: 7.}: Opt[string]
  receiveBase* {.fieldNumber: 8, fixed.}: Opt[SequenceNumber]
  acknowledgementBitmap* {.fieldNumber: 9.}: Opt[seq[byte]]
  firstSurbSequence* {.fieldNumber: 10, fixed.}: Opt[SurbSupplySequence]
  surbSupplyReceiveBase* {.fieldNumber: 11, fixed.}: Opt[SurbSupplySequence]
  surbSupplyAcknowledgementBitmap* {.fieldNumber: 12.}: Opt[seq[byte]]
  surbSupplyLimit* {.fieldNumber: 13, fixed.}: Opt[SurbSupplySequence]
  surbs* {.fieldNumber: 14.}: seq[seq[byte]]
  rejectionReason* {.fieldNumber: 15.}: Opt[string]
  finalSequence* {.fieldNumber: 16, fixed.}: Opt[SequenceNumber]

Every frame carries version, sessionId and kind. sessionId is the initiator-generated session pseudonym, not the initiator’s authenticated network identity. On the recipient it becomes the peerId metadata of incoming virtual connections, but it is never passed to Switch.connect, Switch.dial or the ordinary peer store.

Opt distinguishes an absent field from a field carrying its numeric default. The semantic validator then relates presence to kind.

Field	Frame kind
streamId	OpenStream, StreamAck, StreamReject, Data, Ack, CloseStream, ResetStream
sequence, payload	Data
codec	OpenStream
receiveBase, acknowledgementBitmap	Ack
firstSurbSequence	SurbSupply, and Connect or initiator-originated OpenStream when numbered supply follows the two response SURBs
surbSupplyReceiveBase, surbSupplyAcknowledgementBitmap, surbSupplyLimit	complete snapshots on reverse frames, including stream and session teardown sent through SURBs
rejectionReason	optionally StreamReject
surbs	Connect, OpenStream, SurbSupply and SurbStatusProbe
finalSequence	CloseStream

Data and Absolute Acknowledgement State

A Data frame identifies one transport chunk:

MixTransportFrame(
  version: MixTransportVersion,
  sessionId: session.sessionId,
  kind: FrameKind.Data,
  streamId: Opt.some(stream.streamId),
  sequence: Opt.some(reservedSequence),
  payload: Opt.some(move(chunk)),
)

An Ack frame contains an absolute snapshot of the receiver:

MixTransportFrame(
  version: MixTransportVersion,
  sessionId: session.sessionId,
  kind: FrameKind.Ack,
  streamId: Opt.some(stream.streamId),
  receiveBase: Opt.some(snapshot.receiveBase),
  acknowledgementBitmap: Opt.some(snapshot.acknowledgementBitmap),
)

The constants tie the bitmap length directly to the receive window:

const
  ReceiveWindowChunks* = 256
  AckBitmapBytes* = ReceiveWindowChunks div 8
  MaxInflightChunks* = 64
 
static:
  doAssert ReceiveWindowChunks mod 8 == 0
  doAssert MaxInflightChunks <= ReceiveWindowChunks

Every sequence below receiveBase has entered the receiver’s ordered BufferStream. Bitmap bit i says whether sequence receiveBase + i is retained in the current receive window. The fixed 32-byte representation has the same size for a contiguous prefix, one isolated gap or alternating received and missing chunks.

The sender-side 64-chunk in-flight limit is separate from the 256-position receive window. It bounds retained retransmission state without changing what one ACK can describe.

Individual Public SURBs

The common frame carries every public SURB as a separate repeated byte field:

surbs* {.fieldNumber: 14.}: seq[seq[byte]]

The wire format does not record persistent redundancy groups. In Connect and initiator-originated OpenStream, position gives the first two SURBs immediate-response semantics and gives any remaining SURBs numbered session-supply semantics. A standalone SurbSupply frame contains only numbered session supply. The recipient stores each accepted numbered SURB in one session queue and selects several queue entries only when it forms a temporary redundancy batch for one reverse frame. Mix Transport SURB Replenishment Strategy explains why ordinary redundancy is a send-time operation rather than persistent wire state.

The sender converts each public SURB with serializeSurb. The receiver applies deserializeSurb independently to every repeated value. An invalid serialized SURB can therefore be discarded without rejecting valid SURBs carried by the same transport frame.

MixTransport does not reconstruct private SURB fields manually. Local encoding checks that each value has SurbSize, and both encoding and decoding bound the total number of repeated values. Inbound structural validation deliberately leaves individual deserialization to the frame handler so one malformed SURB does not invalidate the remaining supply.

Numbered SURB Supply and Absolute State

The initiator begins numbered supply in Connect and continues the same sequence in initiator-originated OpenStream and standalone SurbSupply frames. firstSurbSequence identifies the first numbered SURB, and each later supply item has the next consecutive sequence:

MixTransportFrame(
  version: MixTransportVersion,
  sessionId: session.sessionId,
  kind: FrameKind.SurbSupply,
  firstSurbSequence: Opt.some(firstSequence),
  surbs: encodedSurbs,
)

MaxSurbSupplyPerFrame is five. This is a packet-size bound rather than a persistent redundancy-group size: the recipient decodes and accepts every repeated SURB independently. The SurbSupply validator also checks that the complete consecutive range remains inside the valid supply sequence space.

The recipient reports supply receipt and replacement credit with one complete snapshot:

surbSupplyReceiveBase* {.fieldNumber: 11, fixed.}: Opt[SurbSupplySequence]
surbSupplyAcknowledgementBitmap* {.fieldNumber: 12.}: Opt[seq[byte]]
surbSupplyLimit* {.fieldNumber: 13, fixed.}: Opt[SurbSupplySequence]

surbSupplyReceiveBase is the first supply sequence whose receipt is not part of the contiguous acknowledged prefix. Bitmap bit i reports receipt of surbSupplyReceiveBase + i, allowing later sequences to be acknowledged across a gap. surbSupplyLimit is the exclusive absolute upper bound for new sequences the initiator may introduce.

All three snapshot fields must be present together. ConnectAck and SurbStatus require a snapshot. Recipient-originated StreamAck, StreamReject, Data and ACK frames also carry a snapshot when the transport sends them through the reverse path. Every snapshot is absolute, so duplicate or delayed copies do not grant supply capacity twice.

The supply bitmap is tied to its receive window in the same way as the Data acknowledgement bitmap:

const
  SurbSupplyWindow* = 256
  SurbSupplyAckBitmapBytes* = SurbSupplyWindow div 8
  MaxConnectSurbs* = 5
  MaxOpenStreamSurbs* = 4
  MaxSurbSupplyPerFrame* = 5

SurbStatusProbe carries fresh, unnumbered SURBs dedicated to one status response. SurbStatus returns the absolute snapshot through those SURBs. The dedicated response paths allow a recipient whose normal session queue is empty to report newly available credit.

Stream Rejection Is Diagnostic, Not Enumerated

StreamReject carries the rejected streamId and may carry a bounded string:

rejectionReason* {.fieldNumber: 15.}: Opt[string]

The recipient knows why protocol lookup or admission failed and truncates its reason to MaxStreamRejectionReasonBytes, currently 255. The decoder rejects an overlong value. An absent or empty reason remains a valid rejection; the initiator substitutes remote rejected the stream for an unknown reason.

Graceful Stream Closure Carries an Ordering Boundary

CloseStream carries the last Data sequence allocated by its sender:

finalSequence* {.fieldNumber: 16, fixed.}: Opt[SequenceNumber]

Mix delivery can reorder packets, so the receiving endpoint cannot treat arrival of CloseStream as proof that every earlier Data frame has arrived. The receiver records finalSequence and closes only after its receiveBase has advanced beyond that sequence. A stream that sent no Data uses final sequence 0; Data itself starts at sequence 1.

ResetStream has no ordering boundary because reset discards the stream immediately. Frame validation therefore requires finalSequence on CloseStream and forbids the field on every other frame kind. Mix Transport Implementation Walk Through - Remote Teardown follows these fields through the stream and session lifecycle.

Semantic Validation

Protobuf checks field encoding, but it cannot express that one field is mandatory for one frame kind and forbidden for every other kind. The local require template returns a Result error:

template require(condition: bool, message: string): untyped =
  if not condition:
    return err(message)

The compact equality checks both directions:

require frame.sequence.isSome == (frame.kind == FrameKind.Data),
  "sequence does not match the frame kind"
require frame.receiveBase.isSome == (frame.kind == FrameKind.Ack),
  "receiveBase does not match the frame kind"
require frame.finalSequence.isSome == (frame.kind == FrameKind.CloseStream),
  "finalSequence does not match the frame kind"
let carriesNumberedSurbSupply =
  frame.kind == FrameKind.SurbSupply or (
    frame.kind in {FrameKind.Connect, FrameKind.OpenStream} and
    frame.surbs.len > DefaultReplySurbRedundancy
  )
require frame.firstSurbSequence.isSome == carriesNumberedSurbSupply,
  "firstSurbSequence does not match the supplied SURBs"
require frame.surbSupplyReceiveBase.isSome == carriesSurbSupplyState and
  frame.surbSupplyAcknowledgementBitmap.isSome == carriesSurbSupplyState and
  frame.surbSupplyLimit.isSome == carriesSurbSupplyState,
  "SURB supply state is incomplete"

For example, Data without sequence is rejected, and ConnectAck carrying a Data sequence is also rejected. The same pattern covers payload, codec, Data ACK state, SURB supply state and rejection reason.

The bitmap has one exact legal size:

require frame.acknowledgementBitmap.isNone or
  frame.acknowledgementBitmap.get().len == AckBitmapBytes,
  "acknowledgement bitmap has the wrong size"

Kind-specific checks reject empty Data, a Connect without two response SURBs, frames above their SURB capacity, an empty OpenStream codec, an overflowing supply range, and a probe without a complete response batch. OpenStream permits zero SURBs or a dedicated response batch with an optional numbered suffix. After session lookup, the handler requires the former for a recipient-originated opening and the latter for an initiator-originated opening.

Encoding and Decoding

Encoding validates before Protobuf serialization and checks the final size afterward:

proc encode*(frame: MixTransportFrame): Result[seq[byte], string] =
  frame.validate().isOkOr:
    return err(error)
 
  let encoded = Protobuf.encode(frame)
  require encoded.len <= MaxTransportFrameBytes, "transport frame is too large"
  ok(encoded)

Decoding rejects oversized input before allocating protocol state, catches SerializationError, and runs the same semantic validator:

let frame =
  try:
    decodeFrame(data)
  except SerializationError as exc:
    return err("could not decode transport frame: " & exc.msg)
frame.validateFrame(requireValidSurbEncoding = false).isOkOr:
  return err(error)
ok(frame)

After decode returns ok, transport handlers may safely call required accessors such as frame.streamId.get() for a stream frame because validation already proved that the field is present. Inbound decoding validates the number of repeated SURBs but deliberately postpones validation of each serialized SURB value to the frame handler. That separation lets handleSurbSupply keep valid values when another value in the same frame is malformed.

Fixed Data Payload Capacity

StreamId, Data SequenceNumber, the Data ACK receive base and SurbSupplySequence use fixed-width Protobuf encoding. The session identifier is bounded to the 39-byte representation generated by PeerId.random. The transport can therefore calculate one maximum Data-frame overhead without encoding candidate frames.

The overhead includes the complete fixed-size SURB supply snapshot because recipient-originated Data carries that snapshot. Initiator-originated Data does not need the snapshot, but using the same MaxDataPayloadBytes in both directions keeps chunking independent of the sender’s session role.

MaxDataPayloadBytes subtracts this named overhead from MaxTransportFrameBytes. The final encoder still checks the complete serialized length, so the calculated bound and the actual Sphinx capacity are enforced independently. See Mix Transport Implementation Walk Through - Bounded Data Flow for the complete write path.