Communications

How communications are carried out

Topics

  • communications
  • architecture

Communications

CommsCarrier

In node-to-node communications, we need a carrier mechanism that allows transfer of data between two endpoints.

In OctoMY™ the carrie mechanism is implemeted as a class with the following mandate: - Handle a single protocol, such as UDP - Connect to/disconnect from the other side - Transfer data to/from the other side - Adhere to desired opportunity interval (see CommsChannel below) - Provide statistics about the latest events such as transfer rates, error rates etc. - Give a good guess on wether we are connected or not based on the latest events - Provide robust error handling and error reporting

In OctoMY™ the most common carrier is UDP, but there are several other supported or planned carrier types. Here is a complete list: - UDP: SUPPORTED User Datagram Protocol is a low-latency, connectionless internet protocol. - Bluetooth®: PLANNED Bluetooth is a short range low-power wireless protocol. - NFC: PLANNED Near Field Communication is a short range wireless protocol that guarantees proximity to some extent. - Local: IN-DEVELOPMENT We use QLocalServer and QLocalSocket to communicate locally via non-network sockets. Useful when testing and when using virtual endpoints.

Before selecting UDP as the main workhorse protocol to use with OctoMY™, we considered TCP. We ultimately decided to drop TCP because:

CommsChannel

CommChannel is the main node-to-node communications API in OctoMY™. It depends on the CommsCarrier as a transport. It exploits the benefits of communications over low-latency connectionless carriers such as UDP by modeling them closely in a way that hides their inherent complexities. This means, among other things, that the API embraces the relatively short payload size of ~512 bytes that is common for such carriers, as the largest continuous array of data that may be shifted at one time.

NOTE: We have some conflicting requirements for the protocol:

Where is the conflict? For every mode/state you add, you save bytes going over the wire (because the other side will "remember" the state), while for every mode/state you store, you add an opportunity for failure resulting from missing state-carrying packets. Sending all state in every packet means no state is ever dropped, but it adds up to a lot of data.

To alleviate this conflict, we do the following:

We call this mechanism "sync," and every packet may request sync.

The CommChannel API works like this:

  1. Users of the API register Couriers that each is responsible for keeping the latest data fresh and ready for sending should an opportunity present itself. Couriers each tend to a certain type of packet with a certain priority and desired sending frequency. It is up to each courier to maintain its own state.
  2. CommChannel is in charge and decides the speed at which packets are sent and which couriers get their packets sent on each opportunity ("Oportunity Interval").
  3. CommChannel may at any time send non-payload data in each packet or even special-purpose network-only packets to sustain its operation. If there is no data to be sent by couriers, CommChannel may send no-op packets that facilitate the calculation of network characteristics such as round trip time. This is done transparently to the couriers (so you could say couriers are on a higher layer in the OSI model).
  4. CommChannel binds to a local address and port but does not really discriminate from where inbound traffic arrives. All packets are treated equally as all UDP packets inherently contain an identification of the source.
  5. Communication between control and agent is initiated when the user presses the "connect" buttons in the respective user interfaces, at which point agent will attempt to contact all trusted controls at their last known address until one or more answers are received. From then on, all connections that did not result in valid responses will be closed and only retried periodically.
  6. All agent-initiated communication will be broadcast to all active controls in parallel. Remote-initiated transfers will remain private.

Courier

The CommsChannel holds a list of registered Couriers and will try to keep them happy. Couriers may be active or inactive, only active couriers will receive data or get sending opportunities by the CommsChannel. There are many types of Couriers, each looking after different sets of data. Here is a list of important Couriers:

CourierMandate

The Courier interface allows couriers to expose a CourierMandate object to CommsChannel. This object reveals the following:

Multiplexing in Controls

The agent lives the easy life. The decision to be connected is simply an on-off switch in its user interface. The controls on the other hand have to juggle between connections for any agents that it will control. How is this managed?

We will disregard hub in this section and focus on remote. In remote, there exists one ClientWidget instance per agent that is communicating. That instance is responsible for adding and removing the appropriate couriers to CommsChannel dynamically as needed. The CommsChannel itself will simply look at the currently registered couriers and work with those:

Protocols

The comms protocol is the language of OctoMY™ in network communication.

The tamper protocol is an extra layer of protection beside the comms protocol to detect attempts to tamper with communications. It has validation checks that are not necessary by the comms protocol but add tells to the authenticity of the data.

NOTE: The implementation of the comms protocol has the highest priority, tamper protocol will start implementation as soon as we have MVP working.

Session

This is resolved in the following manner:

// Did handshake complete already?
//if(session->established()) {
// TODO: Handle this case:
/*
 When an initial packet is received after session was already established the following logic applies:
 + If packet timestamp was before session-established-and-confirmed timestamp, it is ignored
 + The packet is logged and the stream is examined for packets indicating the original session is still in effect.
 + If there were no sign of the initial session after a timeout of X seconds, the session is torn down, and the last of the session initial packet is answered to start a new hand shake
 + If one or more packets indicating that the session is still going, the initial packet and it's recepient is flagged as hacked and the packet is ignored. This will trigger a warning to the user in UI, and rules may be set up in plan to automatically shut down communication uppon such an event.
 + No matter if bandwidth management is in effect or not, valid initial packets should be processed at a rate at most once per 3 seconds.
 */
//} else {
//}

Handshake

Sessions are established through a handshake that is similar in concept to the 3-way TCP handshake. Both parts of the handshake (A & B) will need the other part's valid RSA pubkey and network address before the handshake may take place. This is exchanged through the pairing process that happens prior.

In the handshake, part A is the initiator:

  1. A sends SYN to B:
    • Hi B. Here is my
      • DESIRED SESSION-ID
      • NONCE
    • ENCODED WITH YOUR (B's) PUBKEY.
  2. B answers SYN-ACK to A:
    • Hi A. HERE IS MY
      • FULL-ID
      • DESIRED SESSION-ID
      • RETURN NONCE
      • NEW NONCE
    • ENCODED WITH YOUR (A's) PUBKEY back at you.
  3. A answers ACK to B:
    • Hi again B. HERE IS MY
      • FULL-ID
      • RETURN NONCE
    • WELL RECEIVED.

At this point, the session is established.

NOTE: For every step in this protocol, if any part is waiting for data from the other that does not arrive, it will attempt to resend its last message on a regular interval until it does (or some error condition or other state change occurs).

Discovery

The discovery and pairing for OctoMY™ nodes works as follows:

Discovery is the mechanism in which nodes seek out and establish secure connections with eachother. Pairing is the mechanism in which the user will assign a level of trust to already discovered nodes.

Discovery is done separately from pairing. This means that signatures that has been discovered previously will remain discovered regardless if the user has selected to trust them or not. In fact the user is free to trust/dis-trust/re-trust any of the discovered nodes as many times as she likes.

Removing discovery records is considered an advanced operation that users will not need to conduct unless they are in fact debugging the pairing protocol.

Discovery proceeds as follows:

  1. Physical proximity is established (in order of preference from security standpoint): a. NFC: by having NFC range physical proximity is implied b. Cam/QR: by having scanning QR with camera, physical proximity is implied c. Bluetooth: by having bluetooth range physical proximity is implied d. Zoo: posting expiring gps coordinates together with pairing signature e. LAN: Identify common gateway
  2. Pairing signatures and pubkeys are exchanged
  3. Pairing challenge/responses are exchanged to verify legitimity of signature/pubkeys
  4. Identicon for signatures is shown in list on each node for user to verify
  5. Optionally: multifactor authentications are conducted to improve security further
  6. The list of nodes is updated to show the "security level" of each node

Pairing proceeds as follows:

As soon as a node is discovered (secure communication is established), it's then up to the user to verify that the signatures match her expectations by inspecting the respective identity information closely.

Glossary:

Node

Intrinsic parts of comms include the following:

Please note that the expensive and complex intrinsic features such as reliability and encryption of CommsChannel are invoked only when needed.

When the amount of data needed for intrinsic features is extensive, separate "intrinsic packets" will be sent, while other lesser-in-size intrinsic data such as counters will instead accompany each packet. Protocol dictates when such dedicated packets will be needed or not, and changes in this part of the protocol should not affect the higher-level courier interface.

WIP: New comms architecture for protocols and addresses


Debug enabled