iodine/doc/proto_00000800.txt

Detailed specification of protocol in version 00000800
======================================================

Note: work in progress!!

======================================================
1. DNS protocol
======================================================

Quick alphabetical index / register:
	0-9	Data packet
	A-F	Data packet
	I	IP address
	L	Login
	N	Downstream fragsize	(NS.topdomain A-type reply)
	O	Options
	P	Ping
	R	Downstream fragsize probe
	S	Switch upstream codec
	V	Version
	W				(WWW.topdomain A-type reply)
	Y	Downstream codec check
	Z	Upstream codec check


CMC = Cache Miss Counter, increased every time it is used

Version:
Client sends:
	First byte v or V
	Rest encoded with base32:
	4 bytes big endian protocol version
	CMC
Server replies:
	4 chars:
		VACK (version ok), followed by login challenge
		VNAK (version differs), followed by server protocol version
		VFUL (server has no free slots), followed by max users
	4 byte value: means login challenge/server protocol version/max users
	1 byte userid of the new user, or any byte if not VACK

Login:
Client sends:
	First byte l or L
	1 byte userid char (hex)
	Rest encoded with base32:
	1 byte flags: (least to most significant bits)
		0: connect to remote TCP port (data pipe/ProxyCommand mode)
		1: use non-localhost remote IP
		2: remote IP is IPv6
		3: use TCP-over-tun optimisation (drop extra packets)
		4: check forward connected status
		5-8: unused
	16 bytes MD5 hash of: (first 32 bytes of password) xor (8 repetitions of login challenge)
	2 bytes remote TCP port (big endian)
	(TCP port appears only when flags bit 0 is set)
	4-16 bytes remote IP address (4 bytes if IPv4 or 16 for IPv6)
	(IP address included only if flags bits 0, 1 are set)
	2 bytes CMC
Server replies:
	LNAK means either auth or options not accepted
	flag [-x.x.x.x-y.y.y.y-mtu-netmask]|[error message] means accepted
		(server ip, client ip, mtu, netmask bits)
	flag can be one of:
		I: Login success (followed by IP addresses in [...])
		C: TCP forward connected
		W: TCP forward connection waiting
		E: TCP connection error - followed by human readable message string
If the requested TCP forwarding options are not accepted by the server, the
response is simply LNAK.
If TCP forwarding is requested, server opens a connection to the specified host
and IP in the background and responds with a flag corresponding to the current
connection status.
The client repeats login requests with the check flag (bit 4) set to poll TCP
connection status, not resending the remote host address or setting any other
flags. Once the server responds with 'C' or 'E', the client either continues
the handshake or prints the error message and exits.
		

IP Request: (for where to try raw login)
Client sends:
	First byte i or I
	1 byte userid char (hex)
	CMC as 4 Base32 chars
Server replies
	BADIP if bad userid
	First byte I
	Then comes external IP address of iodined server
	as 4 bytes (IPv4) or 16 bytes (IPv6)

Upstream codec check / bounce:
Client sends:
	First byte z or Z
	Lots of data that should not be decoded
Server replies:
	The requested domain copied raw, in the lowest-grade downstream codec
	available for the request type.

Downstream codec check:
Client sends:
	First byte y or Y
	1 char, meaning downstream codec to use
	rest encoded in base32:
	1 byte check variant
	2 bytes CMC
	Possibly extra data, depending on check variant
Server sends:
	Data encoded with requested downstream codec; data content depending
	on check variant number.
	BADCODEC if requested downstream codec not available.
	BADLEN if check variant is not available, or problem with extra data.

	Downstream codec chars are same as in 'O' Option request, below.

	Check variants:
	1: Send encoded DOWNCODECCHECK1 string as defined in encoding.h

	(Other variants reserved; possibly variant that sends a decoded-encoded
	copy of Base32-encoded extra data in the request)

Switch codec:
Client sends:
	First byte s or S
	1 byte userid char (hex)
	rest encoded in base32:
	1 byte meaning number of bits per encoded byte in new codec:
		5: Base32   (a-z0-5)
		6: Base64   (a-zA-Z0-9+-)
		26: Base64u (a-zA-Z0-9_-)
		7: Base128  (a-zA-Z0-9\274-\375)
	2 bytes CMC
Server sends:
	Name of codec if accepted. After this all upstream data packets must
	be encoded with the new codec.
	BADCODEC if not accepted. Client must then revert to previous codec
	BADLEN if length of query is too short


Set Options:
Client sends:
	First byte o or O
	1 byte userid char (hex)
	rest encoded in base32:
	1 byte option flags and 2 bytes CMC:
	0        1 - 3
    +76543210+---+
    |0TSUVRCL|CMC|
    +--------+---+
Server sends:
	Full name of encoding type used if successful (case insensitive).
	BADCODEC if not accepted. Previous situation remains.
	BADLEN if number of options doesn't match length of query.
	All options affect only the requesting client.
Option flags:
	T: Downstream encoding Base32, for TXT/CNAME/A/MX (default)
	S: Downstream encoding Base64, for TXT/CNAME/A/MX
	U: Downstream encoding Base64u, for TXT/CNAME/A/MX
	V: Downstream encoding Base128, for TXT/CNAME/A/MX
	R: Downstream encoding Raw, for PRIVATE/TXT/NULL (assumed for
		PRIVATE and NULL)
	C: Downstream compression enabled (compressed before encoding)
	L: Lazy mode enabled, server will keep a number of requests waiting until
		data becomes available to send downstream or the requests time out.
		The timeout value for requests is controlled by the client.
		Applies only to data transfer; handshake is always answered immediately.
	If codec unsupported for request type, server will use Base32; note
	that server will answer any mix of request types that a client sends.
	Server may disregard the encoding options; client must always use the
	downstream encoding type indicated in every downstream DNS packet.


Probe downstream fragment size:
Client sends:
	First byte r or R
	1 byte userid char (hex)
	2 bytes big-endian fragsize encoded as 4 bytes base32
	Then follows a long random query which contents does not matter.
Server sends:
	Requested number of bytes as a response. The first two bytes contain
	the requested length. The third byte is 107 (0x6B). The fourth byte
	is a random value, and each following byte is incremented with 107.
	This is checked by the client to determine corruption.
	BADFRAG if requested length not accepted.

Set downstream fragment size:
Client sends:
	First byte n or N
	1 byte userid char (hex)
	Rest encoded with base32:
	2 bytes new downstream fragment size (big-endian)
	CMC
Server sends:
	2 bytes new downstream fragment size. After this all downstream
	payloads will be max (fragsize + 2) bytes long.
	BADFRAG if not accepted.


Upstream data header:
     76543 21076 54321076 54321076 5432
    +!----+!----+!----!--+--!----!+----+
    |0UUUU|UDCMC| Seq ID | Dn ACK |ACFL|
    +-----+-----+--------+--------+----+

Downstream data header:        |=> only if ping (P) flag set       |
    0        1        2        3        4        5        6 
    +--------+--------+76543210+--------+--------+--------+--------+
    | Seq ID | Up ACK |0EIPACFL|Dn Wsize|Up Wsize|DnWstart|UpWstart|
    +--------+--------+--------+--------+--------+--------+--------+

UUUU = Userid
L = Last fragment flag
A = ACK flag
F = First fragment flag
C = Compression enabled for downstream packet
P = ping flag: extra header present
I = responded to immediately (for RTT calculation) - downstream only
E = TCP Forward error (data following is text string reason)
UDCMC = Upstream Data CMC char (base36 [a-z0-9])

Up/Dn Wsize/Wstart = upstream/downstream window size/window start Seq ID 

Upstream data packet starts with 1 byte ASCII hex coded user byte; then
1 char data-CMC; then 4 bytes Base32 encoded header; then comes the payload
data, encoded with the chosen upstream codec.

Downstream data starts with 3 byte header, followed by data, which may be
compressed. If Ping flag is set, another 4 bytes are appended to the header,
containing upstream and downstream window sizes and window start sequence IDs.
The response does not need to contain data. If the server has no data to send,
the response will always include the ping header and the ping flag will be set.

If the TCP forward error (E) flag is set, the TCP connection at the server is
closed and the client sends EOF to stdout and exits.

In NULL and PRIVATE responses, downstream data is always raw. In all other
response types, downstream data is encoded (see Options above).
Encoding type is indicated by 1 prefix char (before the data header):
TXT:
	End result is always DNS-chopped (series of len-prefixed strings
	<=255 bytes)
	t or T: Base32	 encoded before chop, decoded after un-chop
	s or S: Base64	 encoded before chop, decoded after un-chop
	u or U: Base64u	 encoded before chop, decoded after un-chop
	v or V: Base128	 encoded before chop, decoded after un-chop
	r or R: Raw	 no encoding, only DNS-chop
SRV/MX/CNAME/A:
	h or H: Hostname encoded with Base32
	i or I: Hostname encoded with Base64
	j or J: Hostname encoded with Base64u
	k or K: Hostname encoded with Base128
SRV and MX may reply with multiple hostnames, each encoded separately. Each
has a 10-multiple priority, and encoding/decoding is done in strictly
increasing priority sequence 10, 20, 30, etc. without gaps. Note that some DNS
relays will shuffle the answer records in the response.


Ping:
Client sends:
	First byte p or P
	Second byte CMC
	1 byte userid char (hex)
	Rest encoded with Base32:
    0        1...7        8 - 9
    +--------+---+76543210+---+
    |Dn SeqID|...|00DWTANR|CMC|
    +--------+---+--------+---+
	1 byte Downstream seq ID ACK
	1 byte window size (upstream)
	1 byte window size (downstream)
	1 byte window start (upstream)
	1 byte window start (downstream)
	2 bytes big-endian server timeout in ms
	2 bytes big-endian downstream fragment ACK timeout in ms
	
	1 byte flags:
		D = disconnect remote TCP forward (client should then exit)
		W = update window frag timeout
		T = update server timeout
		A = is ACKing downstream frag
		N = is NACKing downstream frag (unused)
		R = response must contain ping header (data optional)
	2 bytes CMC

The server responses to Ping and Data packets are compatible, and are described
above (refer to downstream data header).

If R (respond) bit is set, the server responds immediately with a ping header.
The server must also adjust its window sizes to those provided by the ping.
If the T but is set, the server sets the user's DNS timeout to the value spec-
ified by the packet.

If the bit corresponding to changing a particular value (ie. window timeout) is
not set, the value should be random. (note: this is disabled in debug mode).

In lazy mode, unless the R flag is set, the server will hold the ping until it
times out or more data becomes available to send.


"Lazy-mode" operation
=====================

Client-server DNS traffic sequence has been reordered to provide increased
(interactive) performance and greatly reduced latency.

Idea taken from Lucas Nussbaum's slides (24th IFIP International Security
Conference, 2009) at http://www.loria.fr/~lnussbau/tuns.html. Current
implementation is original to iodine, no code or documentation from any other
project was consulted during development.

Server:
In lazy mode, except where otherwise specified, responses are sent using the
oldest pending query held in the server's buffer (QMEM). The server responds
to a stored pending query when the query times out, an upstream ACK is pending
(for that user), or the server has an excess of pending queries (more than the
user's downstream window size).

Upstream data fragments are ACK'd immediately to keep data flowing.

Upstream pings are answered immediately only when the Respond flag is set (see
ping header), in which case the response is to the same DNS query as the ping.
Immediate responses (<10ms old) to either ping or data requests are marked
and used to calculate the round-trip-time for the connection.

Client:
The client keeps track of all queries it sends, and maintains a minimum of
<downstream window size> pending queries to fill the server buffer.
Downstream data is always ACK'd immediately with a new request (either a ping
or data if available). The client sends excess requests (ie. already has enough
pending queries) for ACKs or for new data.


======================================================
2. Raw UDP protocol
======================================================

This protocol does not implement data windowing and does not guarantee data
delivery, however it is faster since the data is not encoded and transferred
on top of the DNS protocol. Full packets are compressed and sent when they
arrive on the tun device, and are processed immediately on the other side.

All Raw UDP protcol messages start with a 3 byte header: 0x10d19e
This is not the start of a valid DNS message so it is easy to identify.
The fourth byte contains the command (C) and the user id (U).

	 7654 3210
	+----+----+
	|CCCC|UUUU|
	+----+----+

Login message (command = 1):
The header is followed by a MD5 hash with the same password as in the DNS
login. The client starts the raw mode by sending this message, and uses
the login challenge +1, and the server responds using the login challenge -1.
After the login message has been exchanged, both the server and the client
switch to raw udp mode for the rest of the connection.

Data message (command = 2):
After the header comes the payload data, which is always compressed.

Ping message (command = 3):
Sent from client to server and back to keep session open. Has no payload.