The Tether Messaging Protocol

Version 1.0

This Gist is the draft document for the Tether Messaging Protocol implemented in the Tether package for Dart.

Introduction

The Web is by definition a very connected platform. A device connected to the internet virtually has access to every other device on the web. There are multiple protocols that describe communications between these devices. For web pages, the dominating protocol is HTTP.

HTTP at the highest level is a standard for mapping resources and actions applied to those resources to a series of methods (GET, POST, PUT, PATCH, DELETE, OPTIONS and more) and URIs.

It was created to satisfy the needs for the early web. Sending documents back and forth, and potentially filling out some forms and so on. However, as the web grew, HTTP started to get used for not only sending documents or filled forms, but for sending structured data as well.

Since HTTP is based on a request/response cycle between a server and a client, the server had no direct connection to the client, other than at the time of the request, so clever people figured out that if a client requests a resource lazyly, not expecting a direct response, the server could delay the response until there were an update to be shared with the client.

This method is called "long polling", and together with other hacks and workarounds it started being used to bring realtime communication to the web.

Skip forward a few years, and the HTML5 technologies started taking shape. One of these projects were WebSockets, a bi-directional communications protocol that creates a persistent connection between the server and the client, like other protocols before it. However this was exactly what the web platform needed, because it removed the overhead of having to conform to a protocol that was made for a single request/response connection, and get a much quicker and simpler way of communicating freely between the client and the server.

WebSockets isn't a perfect replacement for HTTP though, because it doesn't use the same method-and-URI protocol as HTTP. It does only specify a low level "tunnel" between the two peers, where data can just fly through.

The Tether was the flagship feature of the Bridge Framework for Dart. It works on top of bidirectional protocols by adding an API to communicate between two devices.

To enable more devices to hook into the Tether communication, the protocol will be described in detail in this document, so that more implementations can be created.

Semantic versions

As seen below, each message sent from one Tether to the other must provide the protocol version it implements. The versions must follow the Semantic Versioning standard. That means that one Tether using protocol version 1.0 must be able to communicate without issues with a Tether using version 1.1. To enable future expansions, the protocol doesn't allow constraints on received messages.

For example, if a future version would introduce a new key on the Message interface, the older version cannot complain that it received an unrecognized field. It must only use the keys that its own specification describes.

Messages Protocol

A Message sent from one side of the connection to the other consists of a JSON packet containing the following keys:

protocol – The value will be a string containing a version identifier for the version of the protocol is being expected by the implementation. For version 1.0 this is "1.0".
key – The value will be a string containing the identifier of the message channel.
payload – The value can be any JSON structure.
returnKey – The value will be a string containing the identifier that must be used for the response of this message.
isError – The value will be a boolean value of whether or not the payload of this message is marked as an error.

Request/Response

As an example, I want to send a string message "Hello, World!" through the Tether on the channel "hello". Given I have a connected Tether with a session token of "x", I would have to do two things:

First, I must determine a return key. This key will define a single use channel that will only be used for the response of this message. Most likely, the implementation of this protocol will generate a unique identifier automatically. Let's say we use the return key "hello-back".

Next, we construct the Message:

{
  "protocol": "1.0",
  "key": "hello",
  "payload": "Hello, World!",
  "returnKey": "hello-back",
  "isError": false
}

Now we're ready to use the Anchor to fire off this message, and wait for a response message on the channel "hello-back". If everything went all right, we expect to get a response message with a payload of null. Like this:

{
  "protocol": "1.0",
  "key": "hello-back",
  "payload": null,
  "returnKey": null,
  "isError": false
}

However, if something went wrong on the other side, that Tether should respond with an error:

{
  "protocol": "1.0",
  "key": "hello-back",
  "payload": "We could not process the message you sent us. Sorry!",
  "returnKey": null,
  "isError": true
}

Notice that when the isError flag is set to true, the payload value can be of any type or structure, representing the error that occured. It is up to you to parse the message and determine how to respond.

Establishing connection

The Tether Protocol does not describe the low level communications protocol, it only goes on top of a working bi-directional system.

Once a connection has been established with the platform selected (e.g. WebSockets), Tether will kick in and perform its own handshaking procedure.

Anchors

Each side of a single connection is called an Anchor. An Anchor only lives for the duration of the connection. If the connection is lost, a new Anchor must be used to reconnect the Tether to the other side.

Master and Slave

The protocol demands that one side of the communication is responsible for managing the Session of the Tether. This Tether will be called the Master. Most likely the Master handles multiple connections and Sessions. The other side will be the Slave Tether.

In the context of a server communicating with a client web browser, the server will be the Master and the client the Slave.

Handshake

To establish a new session, the Master Tether must send a Message with the key of "__handshake" and a payload of any structure that will represent the Session of the Tether connection. The Slave Tether will accept the Session value and store it for authentication. If nothing went wrong, the Slave will respond with a payload of null.

Reconnecting

When an Anchor's connection is interrupted or closed, a notification must be sent to the Tether. It is then up to the Tether to provide a new Anchor (using some reconnect strategy). The Slave still has the pending Session object remaining in memory. When another connection between two Anchors has been established, a reconnect procedure will interfere with the handshake procedure, to let the Slave tell the Master what session token to use instead of the Master's allocated one.

First, the Master sends the handshake message. However, the Slave will not respond with null, but with the Session token that it still has. The master will evaluate and validate the reconnecting Session, and verify that the Session is being re-established by sending the same session object on the channel "__reconnect". If the Slave is satisfied with the response (sees that it is the same session as the one it is trying to reconnect) it responds to the "__reconnect" message by returning a null message on the return key.

Summary

Here is a low level example of a Session being set up and used:

Master (handshake):

{
  "protocol": "1.0",
  "key": "__handshake",
  "payload": "x",
  "returnKey": "e6VKpYjjYiLNKWKxvJC9",
  "isError": false
}

Slave (handshake):

{
  "protocol": "1.0",
  "key": "e6VKpYjjYiLNKWKxvJC9",
  "payload": null,
  "returnKey": null,
  "isError": false
}

Slave (tries to get an article resource with an ID of 1):

{
  "protocol": "1.0",
  "key": "getArticleById",
  "payload": 1,
  "returnKey": "YMx7TqRrMQBZUntyKh3N",
  "isError": false
}

Master (responds with an error that no article with that id exists):

{
  "protocol": "1.0",
  "key": "YMx7TqRrMQBZUntyKh3N",
  "payload": {
    "errorCode": 123,
    "message": "That article does not exist!"
  },
  "returnKey": null,
  "isError": true
}

Slave (tries to get the article of ID 2):

{
  "protocol": "1.0",
  "key": "getArticleById",
  "payload": 2,
  "returnKey": "DEGpDUDLnKqtAC72iiLr",
  "isError": false
}

Master (responds with the article object):

{
  "protocol": "1.0",
  "key": "DEGpDUDLnKqtAC72iiLr",
  "payload": {
    "articleId": 2,
    "title": "My Awesome Article",
    "content": "My awesome article body text."
  },
  "returnKey": null,
  "isError": false
}

Master (notifies Slave of a new event):

{
  "protocol": "1.0",
  "key": "newArticleWasPosted",
  "payload": {
    "articleId": 3,
    "title": "New Article",
    "content": "This one is awesome, too!"
  },
  "returnKey": "sVjMv2pEsQWA7ptdGCaE",
  "isError": false
}

Slave (responds with null to confirm that it received the message):

{
  "protocol": "1.0",
  "key": "sVjMv2pEsQWA7ptdGCaE",
  "payload": null,
  "returnKey": null,
  "isError": false
}

Here, the connection is interrupted in some way, but the Slave still has the session "x" in memory. So when a new connection is established, the Master tries to give the Slave a new Session, but instead the Slave uses the old Session to reconnect: