Protocols

Technical reference for OctoMY™'s wire protocols, packet formats, and communication layers.

Security Note

All established OctoMY sessions use XChaCha20-Poly1305 AEAD encryption with hybrid asymmetric key exchange (RSA-4096 or Ed25519 via the ISigningKey interface). The multimagic field at the start of every packet enables rapid packet-type dispatch before any crypto operations, providing efficient DoS protection while maintaining full encryption for authenticated traffic.

Protocol stack

Default ports

Packet structure

Basic packet format

Every packet begins with a SESSION_ID_TYPE (2-byte quint16) multimagic field. Reserved multimagic values identify protocol control packets; any value at or above MULTIMAGIC_LAST is interpreted as a remote session ID, making the packet courier data.

Protocol magic and version (handshake and data packets)

Handshake and data packets include a 4-byte protocol magic (OCTOMY_PROTOCOL_MAGIC = 0x0C701111) and a 4-byte protocol version (OCTOMY_PROTOCOL_VERSION_CURRENT = 1) immediately after the multimagic. The magic allows early rejection of non-OctoMY traffic; the version selects the QDataStream serialization format.

#define OCTOMY_PROTOCOL_MAGIC          (0x0C701111)
#define OCTOMY_PROTOCOL_VERSION_CURRENT (1)

Session ID

Each side of an established session chooses a local session ID (SESSION_ID_TYPE, 2 bytes). The remote session ID assigned by the peer is used as the multimagic value in courier data packets, so the receiver can look up the correct session in O(1).

Packet types (multimagic values)

OctoMY does not use TCP-style bit-flag headers. Instead, the first 2 bytes of every UDP datagram contain a multimagic value from the Multimagic enum that determines the packet type:

The receiver dispatches on static_cast<Multimagic>(multimagic) in a switch; values not matching any known control type fall through to the courier-data path.

Handshake protocol

Three-way handshake

Handshake packet content

All handshake bodies are hybrid-encrypted with the recipient's public key (see Encryption below). The encrypted payload contains the fields listed; the outer packet is [multimagic:2][encrypted blob].

SYN Packet (initiator -> responder, encrypted with responder's public key):

SYN-ACK Packet (responder -> initiator, encrypted with initiator's public key):

ACK Packet (initiator -> responder, encrypted with responder's public key):

Encryption

Hybrid asymmetric encryption (handshake)

Handshake packets use hybrid encryption: an asymmetric key exchange produces a one-time symmetric key, which is used for XChaCha20-Poly1305 AEAD encryption of the actual payload. The wire format depends on the recipient's key algorithm:

RSA hybrid (version byte 0x01):

[0x01][RSA-encrypted symKey : rsaKeyBytes][nonce : 24][AEAD ciphertext + tag : N + 16]

A random 32-byte symmetric key is generated, RSA-encrypted with the recipient's public key, and prepended. The payload is then AEAD-encrypted with that symmetric key.

Ed25519 hybrid (version byte 0x02):

[0x02][ephemeral Ed25519 pubkey : 32][nonce : 24][AEAD ciphertext + tag : N + 16]

An ephemeral Ed25519 keypair is generated. Both keys are converted to X25519. An ECDH key exchange with the recipient's X25519 public key produces the shared secret. The payload is AEAD-encrypted with that shared secret.

The version byte allows decrypt() to auto-detect the format and use the correct unwrapping path.

Session key derivation

After the three-way handshake completes, both sides derive a shared session key for AEAD encryption of all subsequent courier data:

sessionKey = SHA256(sharedSecret_or_keyMaterial || synNonce_BE || synAckNonce_BE)

The derivation path depends on the key algorithms in use:

Nonces are serialized big-endian before hashing.

Courier data encryption (AEAD)

Established sessions encrypt courier data using XChaCha20-Poly1305 with the derived session key:

[sessionID : 2][nonceCounter : 8][AEAD ciphertext + tag]

• sessionID — the remote session ID (doubles as the multimagic value)
• nonceCounter — monotonically increasing 64-bit counter, sent in the clear so the receiver can reconstruct the 24-byte AEAD nonce: [counter:8 big-endian][zeros:16]
• The AEAD ciphertext contains [courierID:4][payloadSize:2][courier payload]

Reliability system

OctoMY does not have per-packet sequence numbers, ack bitmasks, or a transport-level retransmission layer. Reliability is handled per-courier at the application level:

Flow control

FlowControl uses RTT-based send-rate adjustment rather than a sliding window or congestion-avoidance algorithm:

Transitions:

• Good -> Bad: If RTT exceeds 250 ms, drop to Bad mode immediately. A penalty timer doubles (up to 60 s) each time the channel drops out of Good before sustaining 10 s of good conditions.
• Bad -> Good: If RTT stays below 250 ms for longer than the current penalty time, upgrade to Good mode. After 10 s of sustained good conditions, the penalty timer halves (minimum 1 s).

Discovery protocol

LAN multicast discovery

Discovery packet format

LAN discovery uses its own simple binary format (not QDataStream). The magic is "OCTOMY" (6 ASCII bytes):

Total: 78 bytes.

Discovery packet types: MULTIMAGIC_LAN_ANNOUNCE (broadcast), MULTIMAGIC_LAN_ANNOUNCE_ACK (unicast response).

Identity exchange packets (MULTIMAGIC_LAN_IDENTITY_REQ, MULTIMAGIC_LAN_IDENTITY_RESP) use a larger variable-length format that includes the node's display name and public key in DER encoding.

NAT traversal

Hole punching

Punch protocol

1. Both nodes register with Zoo server
2. Zoo provides peer's external address
3. Both send simultaneous UDP packets
4. NAT creates mapping, allowing responses
5. Direct communication established

Blob transfer protocol

Large data transfer

For data larger than MTU (~1400 bytes):

Blob Header:

Chunk Packet:

Blob reassembly

Courier protocol

Courier ID allocation

Standard courier IDs

Heartbeat and keepalive

Keepalive mechanism

SessionCourier heartbeat

The SessionCourier sends a 16-byte heartbeat payload at ~1 Hz:

When the remote echoes back our timestamp, RTT is computed as now - echoedTimestamp. This drives FlowControl and connection-alive detection.

MULTIMAGIC_IDLE packets (multimagic only, no payload) serve as a lightweight keepalive for sessions that have no active couriers.

Error handling

Protocol errors

Connection errors

Security considerations

Protections

Best practices

• Use secure random for nonces and key material
• Validate all packet fields and sizes before processing
• Rate limit connection attempts
• Wipe intermediate key material after session key derivation
• Expire idle sessions to limit state accumulation