tailscale/logtail/api.md

Tailscale Logs Service

The Tailscale Logs Service defines a REST interface for configuring, storing, retrieving, and processing log entries.

Overview

HTTP requests are received at the service base URL https://log.tailscale.io, and return JSON-encoded responses using standard HTTP response codes.

Authorization for the configuration and retrieval APIs is done with a secret API key passed as the HTTP basic auth username. Secret keys are generated via the web UI at base URL. An example of using basic auth with curl:

curl -u <log_api_key>: https://log.tailscale.io/collections

In the future, an HTTP header will allow using MessagePack instead of JSON.

Collections

Logs are organized into collections. Inside each collection is any number of instances.

A collection is a domain name. It is a grouping of related logs. As a guideline, create one collection per product using subdomains of your company's domain name. Collections must be registered with the logs service before any attempt is made to store logs.

Instances

Each collection is a set of instances. There is one instance per machine writing logs.

An instance has a name and a number. An instance has a private and public ID. The private ID is a 32-byte random number encoded as hex. The public ID is the SHA-256 hash of the private ID, encoded as hex.

The private ID is used to write logs. The only copy of the private ID should be on the machine sending logs. Ideally it is generated on the machine. Logs can be written as soon as a private ID is generated.

The public ID is used to read and adopt logs. It is designed to be sent to a service that also holds a logs service API key.

The tailscale logs service will store any logs for a short period of time. To enable logs retention, the log can be adopted using the public ID and a logs service API key. Once this is done, logs will be retained long-term (for the configured retention period).

Unadopted instance logs are stored temporarily to help with debugging: a misconfigured machine writing logs with a bad ID can be spotted by reading the logs. If a public ID is not adopted, storage is tightly capped and logs are deleted after 12 hours.

APIs

Storage

POST /c/<collection-name>/<private-ID> — send a log

The body of the request is JSON.

A single message is an object with properties:

{ }

The client may send any properties it wants in the JSON message, except for the logtail property which has special meaning. Inside the logtail object the client may only set the following properties:

• client_time in the format of RFC3339: "2006-01-02T15:04:05.999999999Z07:00"

A future version of the logs service API will also support:

• client_time_offset a integer of nanoseconds since the client was reset
• client_time_reset a boolean if set to true resets the time offset counter

On receipt by the server the client_time_offset is transformed into a client_time based on the server_time when the first (or client_time_reset) event was received.

If any other properties are set in the logtail object they are moved into the "error" field, the message is saved and a 4xx status code is returned.

A batch of messages is a JSON array filled with single message objects:

[ { }, { }, ... ]

If any of the array entries are not objects, the content is converted into a message with a "logtail": { "error": ...} property, saved, and a 4xx status code is returned.

Similarly any other request content not matching one of these formats is saved in a logtail error field, and a 4xx status code is returned.

An invalid collection name returns {"error": "invalid collection name"} along with a 403 status code.

Clients are encouraged to:

• POST as rapidly as possible (if not battery constrained). This minimizes both the time necessary to see logs in a log viewer and the chance of losing logs.
• Use HTTP/2 when streaming logs, as it does a much better job of maintaining a TLS connection to minimize overhead for subsequent posts.

A future version of logs service API will support sending requests with Content-Encoding: zstd.

Retrieval

GET /collections — query the set of collections and instances

Returns a JSON object listing all of the named collections.

The caller can query-encode the following fields:

• collection-name — limit the results to one collection

  {
    "collections": {
      "collection1.yourcompany.com": {
        "instances": {
          "<logid.PublicID>" :{
            "first-seen": "timestamp",
            "size": 4096
          },
          "<logid.PublicID>" :{
            "first-seen": "timestamp",
            "size": 512000,
            "orphan": true,
          }
        }
      }
    }
  }
  

GET /c/<collection_name> — query stored logs

The caller can query-encode the following fields:

• instances — zero or more log collection instances to limit results to
• time-start — the earliest log to include
• One of:
  • time-end — the latest log to include
  • max-count — maximum number of logs to return, allows paging
  • stream — boolean that keeps the response dangling, streaming in logs like tail -f. Incompatible with logtail-time-end.

In stream=false mode, the response is a single JSON object:

{
	// TODO: header fields
	"logs": [ {}, {}, ... ]
}

In stream=true mode, the response begins with a JSON header object similar to the storage format, and then is a sequence of JSON log objects, {...}, one per line. The server continues to send these until the client closes the connection.

Configuration

For organizations with a small number of instances writing logs, the Configuration API are best used by a trusted human operator, usually through a GUI. Organizations with many instances will need to automate the creation of tokens.

POST /collections — create or delete a collection

The caller must set the collection property and action=create or action=delete, either form encoded or JSON encoded. Its character set is restricted to the mundane: [a-zA-Z0-9-_.]+

Collection names are a global space. Typically they are a domain name.

POST /instances — adopt an instance into a collection

The caller must send the following properties, form encoded or JSON encoded:

• collection — a valid FQDN ([a-zA-Z0-9-_.]+)
• instances an instance public ID encoded as hex

The collection name must be claimed by a group the caller belongs to. The pair (collection-name, instance-public-ID) may or may not already have logs associated with it.

On failure, an error message is returned with a 4xx or 5xx status code:

{"error": "what went wrong"}