tailscale/cmd/tailscaled
Joe Tsai 7a77a2edf1
logtail: optimize JSON processing (#11671)
Changes made:

* Avoid "encoding/json" for JSON processing, and instead use
"github.com/go-json-experiment/json/jsontext".
Use jsontext.Value.IsValid for validation, which is much faster.
Use jsontext.AppendQuote instead of our own JSON escaping.

* In drainPending, use a different maxLen depending on lowMem.
In lowMem mode, it is better to perform multiple uploads
than it is to construct a large body that OOMs the process.

* In drainPending, if an error is encountered draining,
construct an error message in the logtail JSON format
rather than something that is invalid JSON.

* In appendTextOrJSONLocked, use jsontext.Decoder to check
whether the input is a valid JSON object. This is faster than
the previous approach of unmarshaling into map[string]any and
then re-marshaling that data structure.
This is especially beneficial for network flow logging,
which produces relatively large JSON objects.

* In appendTextOrJSONLocked, enforce maxSize on the input.
If too large, then we may end up in a situation where the logs
can never be uploaded because it exceeds the maximum body size
that the Tailscale logs service accepts.

* Use "tailscale.com/util/truncate" to properly truncate a string
on valid UTF-8 boundaries.

* In general, remove unnecessary spaces in JSON output.

Performance:

    name       old time/op    new time/op    delta
    WriteText     776ns ± 2%     596ns ± 1%   -23.24%  (p=0.000 n=10+10)
    WriteJSON     110µs ± 0%       9µs ± 0%   -91.77%  (p=0.000 n=8+8)

    name       old alloc/op   new alloc/op   delta
    WriteText      448B ± 0%        0B       -100.00%  (p=0.000 n=10+10)
    WriteJSON    37.9kB ± 0%     0.0kB ± 0%   -99.87%  (p=0.000 n=10+10)

    name       old allocs/op  new allocs/op  delta
    WriteText      1.00 ± 0%      0.00       -100.00%  (p=0.000 n=10+10)
    WriteJSON     1.08k ± 0%     0.00k ± 0%   -99.91%  (p=0.000 n=10+10)

For text payloads, this is 1.30x faster.
For JSON payloads, this is 12.2x faster.

Updates #cleanup
Updates tailscale/corp#18514

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2024-04-12 12:05:36 -07:00
..
childproc all: update copyright and license headers 2023-01-27 15:36:29 -08:00
debug.go net/netmon: make ChangeFunc's signature take new ChangeDelta, not bool 2023-08-23 10:42:14 -07:00
depaware.txt logtail: optimize JSON processing (#11671) 2024-04-12 12:05:36 -07:00
generate.go cmd/mkmanifest, cmd/tailscale, cmd/tailscaled: remove Windows arm32 resources from OSS 2023-03-01 15:45:12 -07:00
install_darwin.go all: update copyright and license headers 2023-01-27 15:36:29 -08:00
install_windows.go all: update copyright and license headers 2023-01-27 15:36:29 -08:00
manifest_windows_386.syso cmd/{tailscale,tailscaled}: embed manifest into Windows binaries 2022-11-21 18:15:51 -05:00
manifest_windows_amd64.syso cmd/{tailscale,tailscaled}: embed manifest into Windows binaries 2022-11-21 18:15:51 -05:00
manifest_windows_arm64.syso cmd/{tailscale,tailscaled}: embed manifest into Windows binaries 2022-11-21 18:15:51 -05:00
proxy.go all: update copyright and license headers 2023-01-27 15:36:29 -08:00
required_version.go all: require Go 1.21 2023-08-08 21:15:08 -07:00
sigpipe.go all: adjust some build tags for plan9 2023-08-24 15:42:35 -07:00
ssh.go all: update copyright and license headers 2023-01-27 15:36:29 -08:00
taildrop.go taildrop: remove breaking abstraction layers for apple (#10728) 2024-01-09 14:11:34 -06:00
tailscaled_bird.go all: update copyright and license headers 2023-01-27 15:36:29 -08:00
tailscaled_notwindows.go all: update copyright and license headers 2023-01-27 15:36:29 -08:00
tailscaled_test.go util/linuxfw: remove yet-unused code to fix linux/arm64 crash 2023-07-20 23:18:40 -07:00
tailscaled_windows.go tailscale: update tailfs file and package names (#11590) 2024-04-02 13:32:30 -07:00
tailscaled.defaults cmd/tailscaled: rename relaynode reference in defaults file comment 2020-03-13 14:38:04 -07:00
tailscaled.go cmd/tailscaled: move cleanup to an implicit action during startup 2024-04-09 12:44:08 -07:00
tailscaled.openrc tstest/integration/vms: use an in-process logcatcher (#2360) 2021-07-08 14:39:45 -04:00
tailscaled.service cmd/tailscaled: move cleanup to an implicit action during startup 2024-04-09 12:44:08 -07:00
windows-manifest.xml cmd/{tailscale,tailscaled}: embed manifest into Windows binaries 2022-11-21 18:15:51 -05:00
with_cli.go all: update copyright and license headers 2023-01-27 15:36:29 -08:00