Exposes some internal state of the sockstats package via the C2N and
PeerAPI endpoints, so that it can be used for debugging. For now this
includes the estimated radio on percentage and a second-by-second view
of the times the radio was active.
Also fixes another off-by-one error in the radio on percentage that
was leading to >100% values (if n seconds have passed since we started
to monitor, there may be n + 1 possible seconds where the radio could
have been on).
Updates tailscale/corp#9230
Signed-off-by: Mihai Parparita <mihai@tailscale.com>
When splitting the radio monitor usage array, we were splitting at now %
3600 to get values into chronological order. This caused the value for
the final second to be included at the beginning of the ordered slice
rather than the end. If there was activity during that final second, an
extra five seconds of high power usage would get recorded in some cases.
This could result in a final calculation of greater than 100% usage.
This corrects that by splitting values at (now+1 % 3600).
This also simplifies the percentage calculation by always rounding
values down, which is sufficient for our usage.
Signed-off-by: Will Norris <will@tailscale.com>
Using log.Printf may end up being printed out to the console, which
is not desirable. I noticed this when I was investigating some client
logs with `sockstats: trace "NetcheckClient" was overwritten by another`.
That turns to be harmless/expected (the netcheck client will fall back
to the DERP client in some cases, which does its own sockstats trace).
However, the log output could be visible to users if running the
`tailscale netcheck` CLI command, which would be needlessly confusing.
Updates tailscale/corp#9230
Signed-off-by: Mihai Parparita <mihai@tailscale.com>
We use it to gate code that depends on custom Go toolchain, but it's
currently only passed in the corp runners. Add a set on OSS so that we
can catch regressions earlier.
To specifically test sockstats this required adding a build tag to
explicitly enable them -- they're normally on for iOS, macOS and Android
only, and we don't run tests on those platforms normally.
Signed-off-by: Mihai Parparita <mihai@tailscale.com>
power state is very roughly approximated based on observed network
activity and AT&T's state transition timings for a typical 3G radio.
Updates tailscale/corp#9230
Updates #3363
Signed-off-by: Will Norris <will@tailscale.com>
Followup to #7518 to also export client metrics when the active interface
is cellular.
Updates tailscale/corp#9230
Updates #3363
Signed-off-by: Mihai Parparita <mihai@tailscale.com>
They're not needed for the sockstats logger, and they're somewhat
expensive to return (since they involve the creation of a map per
label). We now have a separate GetInterfaces() method that returns
them instead (which we can still use in the PeerAPI debug endpoint).
If changing sockstatlog to sample at 10,000 Hz (instead of the default
of 10Hz), the CPU usage would go up to 59% on a iPhone XS. Removing the
per-interface stats drops it to 20% (a no-op implementation of Get that
returns a fixed value is 16%).
Updates tailscale/corp#9230
Updates #3363
Signed-off-by: Mihai Parparita <mihai@tailscale.com>
Followup to #7499 to make validation a separate function (
GetWithValidation vs. Get). This way callers that don't need it don't
pay the cost of a syscall per active TCP socket.
Also clears the conn on close, so that we don't double-count the stats.
Also more consistently uses Go doc comments for the exported API of the
sockstats package.
Updates tailscale/corp#9230
Updates #3363
Signed-off-by: Mihai Parparita <mihai@tailscale.com>
Though not fine-grained enough to be useful for detailed analysis, we
might as well export that we gather as client metrics too, since we have
an upload/analysis pipeline for them.
clientmetric.Metric.Add is an atomic add, so it's pretty cheap to also
do per-packet.
Updates tailscale/corp#9230
Updates #3363
Signed-off-by: Mihai Parparita <mihai@tailscale.com>
withSockStats may be called before setLinkMonitor, in which case we
don't have a populated knownInterfaces map. Since we pre-populate the
per-interface counters at creation time, we would end up with an
empty map. To mitigate this, we do an on-demand request for the list of
interfaces.
This would most often happen with the logtail instrumentation, since we
initialize it very early on.
Updates tailscale/corp#9230
Signed-off-by: Mihai Parparita <mihai@tailscale.com>
We can use the TCP_CONNECTION_INFO getsockopt() on Darwin to get
OS-collected tx/rx bytes for TCP sockets. Since this API is not available
for UDP sockets (or on Linux/Android), we can't rely on it for actual
stats gathering.
However, we can use it to validate the stats that we collect ourselves
using read/write hooks, so that we can be more confident in them. We
do need additional hooks from the Go standard library (added in
tailscale/go#59) to be able to collect them.
Updates tailscale/corp#9230
Updates #3363
Signed-off-by: Mihai Parparita <mihai@tailscale.com>
Makes it cheaper/simpler to persist values, and encourages reuse of
labels as opposed to generating an arbitrary number.
Updates tailscale/corp#9230
Updates #3363
Signed-off-by: Mihai Parparita <mihai@tailscale.com>
Followup to #7177 to avoid adding extra dependencies to the CLI. We
instead declare an interface for the link monitor.
Signed-off-by: Mihai Parparita <mihai@tailscale.com>
Uses the hooks added by tailscale/go#45 to instrument the reads and
writes on the major code paths that do network I/O in the client. The
convention is to use "<package>.<type>:<label>" as the annotation for
the responsible code path.
Enabled on iOS, macOS and Android only, since mobile platforms are the
ones we're most interested in, and we are less sensitive to any
throughput degradation due to the per-I/O callback overhead (macOS is
also enabled for ease of testing during development).
For now just exposed as counters on a /v0/sockstats PeerAPI endpoint.
We also keep track of the current interface so that we can break out
the stats by interface.
Updates tailscale/corp#9230
Updates #3363
Signed-off-by: Mihai Parparita <mihai@tailscale.com>
