Communications

How communications are carried out

Subpages

Topics

  • communications
  • architecture

Communications

The main mode of node-to-node communication in OctoMY™ is carried over UDP. While other carriers such as Bluetooth® are supported for authentication and pairing purposes, UDP is the day-to-day workhorse.

Before selecting UDP, we considered many alternatives. For example, TCP was a strong contender. We ultimately decided to drop TCP because:

  1. TCP tries to pretend that network traffic is an assured linear unbroken stream of bytes. This has the benefit of being easy to understand and use, but:
    • This is far from how a network actually works, making TCP hard to implement effectively. It has taken decades of evolution for TCP to become as good as it is today, but it is still limited by this fallacy.
    • Some needs may actually be better met by not thinking about the network as a linear stream of bytes, so going the length to pretend that it is can get in the way.

CommsChannel

CommChannel is the main node-to-node communications API in OctoMY™, and it is a wrapper around the UDP code found in Qt5. It exploits the benefits of communications over UDP by modeling them closely in a way that hides their inherent complexities. This means, among other things, that the API embraces the UDP payload size of ~512 bytes as the largest continuous array of data that may be shifted at one time. CommsChannel may work over other carriers in the future, but this packet-centric view will not change.

NOTE: We have some conflicting requirements for the protocol:

Where is the conflict? For every mode/state you add, you save bytes going over the wire (because the other side will "remember" the state), while for every mode/state you store, you add an opportunity for failure resulting from missing state-carrying packets. Sending all state in every packet means no state is ever dropped, but it adds up to a lot of data.

To alleviate this conflict, we do the following:

We call this mechanism "sync," and every packet may request sync.

The CommChannel API works like this:

  1. Users of the API register Couriers that each is responsible for keeping the latest data fresh and ready for sending should an opportunity present itself. Couriers each tend to a certain type of packet with a certain priority and desired sending frequency. It is up to each courier to maintain its own state.
  2. CommChannel is in charge and decides the speed at which packets are sent and which couriers get their packets sent on each opportunity.
  3. CommChannel may at any time send non-payload data in each packet or even special-purpose network-only packets to sustain its operation. If there is no data to be sent by couriers, CommChannel may send no-op packets that facilitate the calculation of network characteristics such as round trip time. This is done transparently to the couriers (so you could say couriers are on a higher layer in the OSI model).
  4. CommChannel binds to a local address and port but does not really discriminate from where inbound traffic arrives. All packets are treated equally as all UDP packets inherently contain an identification of the source.
  5. Communication between control and agent is initiated when the user presses the "connect" buttons in the respective user interfaces, at which point agent will attempt to contact all trusted controls at their last known address until one or more answers are received. From then on, all connections that did not result in valid responses will be closed and only retried periodically.
  6. All agent-initiated communication will be broadcast to all active controls in parallel. Remote-initiated transfers will remain private.

Courier

The CommsChannel holds a list of registered Couriers and will try to keep them happy. Couriers may be active or inactive, only active couriers will receive data or get sending opportunities by the CommsChannel. There are many types of Couriers, each looking after different sets of data. Here is a list of important Couriers:

CourierMandate

The Courier interface allows couriers to expose a CourierMandate object to CommsChannel. This object reveals the following:

Multiplexing in Controls

The agent lives the easy life. The decision to be connected is simply an on-off switch in its user interface. The controls on the other hand have to juggle between connections for any agents that it will control. How is this managed?

We will disregard hub in this section and focus on remote. In remote, there exists one ClientWidget instance per agent that is communicating. That instance is responsible for adding and removing the appropriate couriers to CommsChannel dynamically as needed. The CommsChannel itself will simply look at the currently registered couriers and work with those:

Protocols

The comms protocol is the language of OctoMY™ in network communication.

The tamper protocol is an extra layer of protection beside the comms protocol to detect attempts to tamper with communications. It has validation checks that are not necessary by the comms protocol but add tells to the authenticity of the data.

NOTE: The implementation of the comms protocol has the highest priority, tamper protocol will start implementation as soon as we have MVP working.

Session

This is resolved in the following manner:

// Did handshake complete already?
//if(session->established()) {
// TODO: Handle this case:
/*
 When an initial packet is received after session was already established the following logic applies:
 + If packet timestamp was before session-established-and-confirmed timestamp, it is ignored
 + The packet is logged and the stream is examined for packets indicating the original session is still in effect.
 + If there were no sign of the initial session after a timeout of X seconds, the session is torn down, and the last of the session initial packet is answered to start a new hand shake
 + If one or more packets indicating that the session is still going, the initial packet and it's recepient is flagged as hacked and the packet is ignored. This will trigger a warning to the user in UI, and rules may be set up in plan to automatically shut down communication uppon such an event.
 + No matter if bandwidth management is in effect or not, valid initial packets should be processed at a rate at most once per 3 seconds.
 */
//} else {
//}

Handshake

Sessions are established through a handshake that is similar in concept to the 3-way TCP handshake. Both parts of the handshake (A & B) will need the other part's valid RSA pubkey and network address before the handshake may take place. This is exchanged through the pairing process that happens prior.

In the handshake, part A is the initiator:

  1. A sends SYN to B:
    • Hi B. Here is my
      • DESIRED SESSION-ID
      • NONCE
    • ENCODED WITH YOUR (B's) PUBKEY.
  2. B answers SYN-ACK to A:
    • Hi A. HERE IS MY
      • FULL-ID
      • DESIRED SESSION-ID
      • RETURN NONCE
      • NEW NONCE
    • ENCODED WITH YOUR (A's) PUBKEY back at you.
  3. A answers ACK to B:
    • Hi again B. HERE IS MY
      • FULL-ID
      • RETURN NONCE
    • WELL RECEIVED.

At this point, the session is established.

NOTE: For every step in this protocol, if any part is waiting for data from the other that does not arrive, it will attempt to resend its last message on a regular interval until it does (or some error condition or other state change occurs).

Node

Intrinsic parts of comms include the following:

Please note that the expensive and complex intrinsic features such as reliability and encryption of CommsChannel are invoked only when needed.

When the amount of data needed for intrinsic features is extensive, separate "intrinsic packets" will be sent, while other lesser-in-size intrinsic data such as counters will instead accompany each packet. Protocol dictates when such dedicated packets will be needed or not, and changes in this part of the protocol should not affect the higher-level courier interface.