DataLink protocol 1.0 (2008/9/12)

General concepts

DataLink operates to two different modes: query mode and streaming mode. In query mode a client submits commands to a server and the server responds with either status replies or data content. In streaming mode a server sends selected data packets to a client as they become available on the server with no specific per-packet interaction from the client. A DataLink connection is always in query mode by default, switching to streaming mode only when specifically requested by the client.

Client commands and server replies are encapsulated in DataLink packet structures. All command and reply exchanges are formulated using ASCII characters with the exception of a single byte length parameter (preheader) and packet data.

Various client commands and server responses contain time stamps represented as high-precision time values defined as Unix/POSIX epoch times in microseconds. These high-precision time values are abbreviated to hptime, hppkttime, hpdatastart or hpdataend.

DataLink packet structure

Packet structure: 'preheader' | 'header' [| 'data']

Note: Vertical bars (|) are used to separate the DataLink packet sections in this documentation, they are not included in the packets.

All DataLink packets include a preheader and header, some exchanges do not include a data portion.
The preheader is always a 3 byte sequence, the first two bytes are the sequence bytes 'D' and 'L' and the third is an unsigned 1-byte integer indicating the size of the header portion following the preheader.
The header is a maximum of 255 bytes in length. The header portion is always composed of ASCII characters.

Client commands

DataLink client commands go into the header section of a DataLink packet with any associated data in the data section.

The server will reply to most client commands with a status indicator of either OK or ERROR and include a integer value (meaning is command dependent) and an optional message.

The command description below includes the commands themselves in upper case letters with required arguments in lower case letters followed by a short description of the command.

ID

Syntax: ID clientid

Purpose: Send client identification and receive server identification

Client ID string is: programname:username:processid:architecture
Reply is header-only formulated as: ID DataLink <version> :: <capabilities>

POSITION SET

Syntax: POSITION SET pktid hppkttime

Purpose: Set the client read position in the server to a specific packet

As special cases pktid can be "EARLIEST" or "LATEST"
Reply is OK|ERROR with value as packet ID on success

POSITION AFTER

Syntax: POSITION AFTER hptime

Purpose: Set the client read position to first packet with data start time after hptime

Reply is OK|ERROR with value as packet ID on success

MATCH

Syntax: MATCH size|<match pattern of length size>

Purpose: Set the connection packet matching expression

The matching pattern is sent in the data section of the DataLink packet
Size in the header is the length of the match pattern in bytes
Reply is OK|ERROR with value as number currently matched streams

REJECT

Syntax: REJECT size|<reject pattern of length size>

Purpose: Set the connection packet rejecting expression

The rejection pattern is sent in the data section of the DataLink packet
Size in the header is the length of the reject pattern in bytes
Reply is OK|ERROR with value as number currently rejected streams

WRITE

Syntax: WRITE streamid hpdatastart hpdataend flags size|<packet data of length size>

Purpose: Send data packet to DataLink server

Flags: "N" = no acknowledgement requested, "A" = acknowledgement requested
Size in the header is the length of the packet data in bytes
Reply when acknowledgement requested is OK|ERROR with value of inserted packet ID

READ

Syntax: READ pktid

Purpose: Request data packet from DataLink server

Reply is either PACKET (DataLink data packet) or ERROR

STREAM

Syntax: STREAM

Purpose: Request start of streaming mode

Server sends all current and future data packets to client starting at the client read position using the matching and rejecting expressions.
No client commands are accepted in streaming mode except ID requests

ENDSTREAM

Syntax: ENDSTREAM

Purpose: Request end of streaming mode, connection returns to query mode

Server returns ENDSTREAM when streaming mode has ended

INFO

Syntax: INFO type|[match]

Purpose: Request an INFO packet from the server

INFO types include: STATUS, STREAMS and CONNECTIONS
A match expression can be supplied with CONNECTIONS, matches client IDs and addresses
Server returns INFO (DataLink INFO packet) or ERROR

Server responses

OK or ERROR

Syntax: OK value size|<status message>

Syntax: ERROR value size|<error message>

Purpose: Return status of a command

Integer value is command dependent
Optional message may be included in the data section
Size in the header is the length of the message in bytes

INFO

Syntax: INFO type size|<XML info data of length size>

Purpose: Return INFO data requested

All INFO data is returned as XML
The type will always be the same as that requested
Size in the header is the length of the XML in bytes

PACKET

Syntax: PACKET <streamid> <pktid> <hppackettime> <hpdatastart> <hpdataend> <size>|<packet data of length size>

Purpose: Send data packets to client

Size in the header is the length of the packet data in bytes

Each DataLink data packet includes the following header information:

Stream ID :: A string identifying a data stream in W_X_Y_Z/TYPE format
Packet ID :: An integer ID, unique within a DataLink server at any given time
Packet Time :: Time that a packet was accepted by the DataLink server
Data Start :: Start time associated with the data in the packet
Data End :: End time associated with the data in the packet
Data Size :: Length of the data section of the packet in bytes

The combination of packet ID and packet time uniquely identify a packet, using this combination is necessary as packet IDs will eventually be reused.

Example exchange

One of the most common client-server sequences would be a client connecting to a server, exchanging IDs, setting a stream matching pattern and collecting data in streaming mode. This is illustrated below (the preheaders of the DataLink packets have been omitted for clarity):

<client connects to server>
C: ID dalitool:rt:12501:SunOS-5.10
S: ID DataLink 2008.196 :: DLPROTO:1.0 PACKETSIZE:512
C: MATCH 9|IU_ANMO.*
S: OK 10 31|10 streams selected after match
C: STREAM
S: PACKET IU_ANMO_00_BH1/MSEED 3973329 1216097732215400 1216097361127600 1216097381127600 512|<packet data>
S: PACKET IU_ANMO_00_BH2/MSEED 3973330 1216097732875400 1216097362789500 1216097382789500 512|<packet data>
S: PACKET IU_ANMO_00_BHZ/MSEED 3973331 1216097732784300 1216097361789600 1216097381789600 512|<packet data>
... <more packets until either a connection shutdown or ENDSTREAM command>

where C is the client and S is the server.