SatelliteLab: Protocol specification

Each satellite communicates via UDP with exactly two other nodes: the sun, which keeps track of the available satellites and assigns them to experiments, and, at any given time, exactly one planet. The satellite responds to any probe messages that are sent by the planet, and it periodically sends a heartbeat message to both the sun and the planet.

Each satellite maintains the following internal state:

• Sun address: The IP address and port number of the sun.
• Email and identifier: Unique identifier for the satellite.
• Sun password: Random eight-byte string that is used to identify messages from the sun. Changes every 120 seconds.
• Planet address: The IP address and port number of the current planet if one has been assigned; otherwise zero.
• Sun heartbeat interval: The number of seconds between two consecutive sun heartbeat messages.
• Planet heartbeat interval: The same for planet heartbeat messages.
• Planet password: An arbitrary string that is assigned by the sun and included in each planet heartbeat message.
• Bytes sent/received: Total number of bytes sent and received by the satellite since startup (incl. IP and UDP headers).
• Probe history: A sequence of up to 100 recent probe identifiers (last identifier first).
• Packet template: A sequence of bytes that is copied into the unused portion of each response message.

Satellites accept messages on a single UDP port. Any port may be used, but port 30000 should be chosen if available.

Sun heartbeat message

The satellite MUST send this message to the sun once in every sun heartbeat interval.

   uint32 magic                         // 0x5341544C 'SATL'
   uint16 softwareVersion               // MSB: 1=Unix, 2=Java, 3=MacOS, 4=Windows
   byte[8] sunPassword
   uint16 sunHeartbeatInterval
   uint16 planetHeartbeatInterval
   in_addr_t planetIP
   uint16 planetPort
   uint32 bytesSent
   uint32 bytesReceived
   char[] email
   byte zero1
   char[] identifier
   byte zero2
   byte[] planetPassword

Planet heartbeat message

Unless the planet IP is zero, the satellite MUST send this message to the planet once in every planet heartbeat interval.

   uint16 magic1                        // 0xFFFE
   uint32 magic2                        // 0x5341544C 'SATL'
   uint16 softwareVersion               // MSB: 1=Unix, 2=Java, 3=MacOS, 4=Windows
   byte[] planetPassword

Since heartbeat messages and response messages are sent to the same port on the planet, the heartbeat message carries the pattern 0xFFFE in the first two bytes, which can never appear in a valid response message.

Configuration message

The sun MAY send this message to the satellite in response to a sun heartbeat message. The satellite MUST ignore any configuration message whose source IP address and/or source port is different from the preconfigured values for the sun, or whose sunPassword field does not match the satellite's current sun password.

   uint32 magic                         // 0x5341544C 'SATL'
   uint16 softwareVersion
   byte[8] sunPassword                  // must match sunPassword from most recent heartbeat
   uint16 sunHeartbeatInterval          // in seconds; minimum is 1
   uint16 planetHeartbeatInterval       // in seconds; minimum is 1
   in_addr_t planetIP                   // zero to disable planet communication
   uint16 planetPort
   byte[] planetPassword                // up to 100 bytes; may be empty

When a satellite accepts a configuration message, it MUST perform the following three steps: It must 1) update its state with the specified values for the sun heartbeat interval, the planet heartbeat interval, the planet IP, the planetPort, and the planetPassword; it must 2) reset both heartbeat timers; and it must 3) immediately send both a sun heartbeat and a planet heartbeat message.

Probe and response messages

The planet MAY send messages of this type to the satellite at any time. The satellite MUST send a response if the source IP and port number of the probe match the current planet IP and planet port; otherwise it MUST ignore the message. Satellites MUST support probe and response packets of at least 9216 bytes (with IP fragmentation); they MAY support larger packets.

When a satellite receives a probe message, it performs the following steps:

1. Read a 16-bit value from the probe message (the probeID).
2. If the probeID is zero or the end of the packet has been reached, stop.
3. If the probeID is 0xFFFE, erase the probe history and copy the entire probe message to the packet template, then stop.
4. Otherwise, read another 16-bit value (the requestedSize). If the end of the packet has been reached, stop.
5. Check whether the probeID already appears in the probe history. If it does appear, continue with step 1.
6. Send a response packet with requestedSize bytes and the following contents:
   • If requestedSize is at least two, copy the first two bytes from the probe message.
   • If requestedSize is at least four, add the probeID.
   • If there is enough space left, add up to 20 entries from the probe history.
   • If there is still enough space left, add a 16-bit zero, followed by two 32-bit timestamps, indicating the wallclock time the probe packet was received and the time the response packet was sent (in microseconds).
   • Fill any remaining space with the corresponding bytes from the packet template.
7. If probeID is not 0xFFFF, prepend it to the probe history.
8. Continue with step 1.

Here is an example interaction between a planet and its satellite (the numbers represent 16-bit values):

  Planet:    FFFE 1234 1234 1234 1234 1234 1234 1234  // clear history; set template
  Satellite: (no response)
  Planet:    0001 0010 0000 4444 4444 4444            // ID 1; request 16-byte response
  Satellite: 0001 0001 0000 <rcvtime> <sndtime> 1234  // probe history is still empty
  Planet:    0001 0010                                // ID 1 again (duplicate)
  Satellite: (no response)                            // ID is already in history
  Planet:    0003 000C 0002 0008 0001 0014 0000       // IDs 3 and 2 are new
  Satellite: 0003 0003 0001 0000 <rcvtime>            // Response for ID 3 (12 bytes)
             0003 0002 0003 0001                      // Response for ID 2 (8 bytes)
  Planet:    FFFF 000A                                // Check history
  Satellite: FFFF FFFF 0002 0003 0001                 // Response
  Planet:    FFFF 0006                                // Check does not change history
  Satellite: FFFF FFFF 0002                           // Response

In the above example, the packet template simply consists of 1234 (upstream) and 4444 (downstream). SatelliteLab uses a more complicated template, which contains a plain-text message that describes the experiment. This is to avoid problems in case the packets are logged by a firewall or examined by an administrator.