The Go zstd package is not friendly for stateless zstd compression.
Passing around multiple zstd.Encoder just for stateless compression
is a waste of memory since the memory is never freed and seldom
used if no compression operations are happening.
For performance, we pool the relevant Encoder/Decoder
with the specific options set.
Functionally, this package is a wrapper over the Go zstd package
with a more ergonomic API for stateless operations.
This package can be used to cleanup various pre-existing zstd.Encoder
pools or one-off handlers spread throughout our codebases.
Performance:
BenchmarkEncode/Best 1690 610926 ns/op 25.78 MB/s 1 B/op 0 allocs/op
zstd_test.go:137: memory: 50.336 MiB
zstd_test.go:138: ratio: 3.269x
BenchmarkEncode/Better 10000 100939 ns/op 156.04 MB/s 0 B/op 0 allocs/op
zstd_test.go:137: memory: 20.399 MiB
zstd_test.go:138: ratio: 3.131x
BenchmarkEncode/Default 15775 74976 ns/op 210.08 MB/s 105 B/op 0 allocs/op
zstd_test.go:137: memory: 1.586 MiB
zstd_test.go:138: ratio: 3.064x
BenchmarkEncode/Fastest 23222 53977 ns/op 291.81 MB/s 26 B/op 0 allocs/op
zstd_test.go:137: memory: 599.458 KiB
zstd_test.go:138: ratio: 2.898x
BenchmarkEncode/FastestLowMemory 23361 50789 ns/op 310.13 MB/s 15 B/op 0 allocs/op
zstd_test.go:137: memory: 334.458 KiB
zstd_test.go:138: ratio: 2.898x
BenchmarkEncode/FastestNoChecksum 23086 50253 ns/op 313.44 MB/s 26 B/op 0 allocs/op
zstd_test.go:137: memory: 599.458 KiB
zstd_test.go:138: ratio: 2.900x
BenchmarkDecode/Checksum 70794 17082 ns/op 300.96 MB/s 4 B/op 0 allocs/op
zstd_test.go:163: memory: 316.438 KiB
BenchmarkDecode/NoChecksum 74935 15990 ns/op 321.51 MB/s 4 B/op 0 allocs/op
zstd_test.go:163: memory: 316.438 KiB
BenchmarkDecode/LowMemory 71043 16739 ns/op 307.13 MB/s 0 B/op 0 allocs/op
zstd_test.go:163: memory: 79.347 KiB
We can see that the options are taking effect where compression ratio improves
with higher levels and compression speed diminishes.
We can also see that LowMemory takes effect where the pooled coder object
references less memory than other cases.
We can see that the pooling is taking effect as there are 0 amortized allocations.
Additional performance:
BenchmarkEncodeParallel/zstd-24 1857 619264 ns/op 1796 B/op 49 allocs/op
BenchmarkEncodeParallel/zstdframe-24 1954 532023 ns/op 4293 B/op 49 allocs/op
BenchmarkDecodeParallel/zstd-24 5288 197281 ns/op 2516 B/op 49 allocs/op
BenchmarkDecodeParallel/zstdframe-24 6441 196254 ns/op 2513 B/op 49 allocs/op
In concurrent usage, handling the pooling in this package
has a marginal benefit over the zstd package,
which relies on a Go channel as the pooling mechanism.
In particular, coders can be freed by the GC when not in use.
Coders can be shared throughout the program if they use this package
instead of multiple independent pools doing the same thing.
The allocations are unrelated to pooling as they're caused by the spawning of goroutines.
Updates #cleanup
Updates tailscale/corp#18514
Updates tailscale/corp#17653
Updates tailscale/corp#18005
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
This can be used to reload a value periodically, whether from disk or
another source, while handling jitter and graceful shutdown.
Updates tailscale/corp#1297
Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: Iee2b4385c9abae59805f642a7308837877cb5b3f
There are container environments such as GitHub codespaces that have
partial IPv6 support - routing support is enabled at the kernel level,
but lacking IPv6 filter support in the iptables module.
In the specific example of the codespaces environment, this also has
pre-existing legacy iptables rules in the IPv4 tables, as such the
nascent firewall mode detection will always pick iptables.
We would previously fault trying to install rules to the filter table,
this catches that condition earlier, and disables IPv6 support under
these conditions.
Updates #5621
Updates #11344
Updates #11354
Signed-off-by: James Tucker <james@tailscale.com>
Ensure that the latest DNATNonTailscaleTraffic rule
gets inserted on top of any pre-existing rules.
Updates tailscale/tailscale#11281
Signed-off-by: Irbe Krumina <irbe@tailscale.com>
Updates ENG-2133. Adds the ResetToDefaults visibility policy currently only available on macOS, so that the Windows client can read its value.
Signed-off-by: Andrea Gottardo <andrea@gottardo.me>
This package uses a count-min sketch and a heap to track the top K items
in a stream of data. Tracking a new item and adding a count to an
existing item both require no memory allocations and is at worst
O(log(k)) complexity.
Change-Id: I0553381be3fef2470897e2bd806d43396f2dbb36
Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
The new math/rand/v2 package includes an m-local global random number
generator that can not be reseeded by the user, which is suitable for
most uses without the RNG pools we have in a number of areas of the code
base.
The new API still does not have an allocation-free way of performing a
seeded operations, due to the long term compiler bug around interface
parameter escapes, and the Source interface.
This change introduces the two APIs that math/rand/v2 can not yet
replace efficiently: seeded Perm() and Shuffle() operations. This
implementation chooses to use the PCG random source from math/rand/v2,
as with sufficient compiler optimization, this source should boil down
to only two on-stack registers for random state under ideal conditions.
Updates #17243
Signed-off-by: James Tucker <james@tailscale.com>
Providing a hash.Block512 is an implementation detail of how deephash
works today, but providing an opaque type with mostly equivalent API
(i.e., HashUint8, HashBytes, etc. methods) is still sensible.
Thus, define a public Hasher type that exposes exactly the API
that an implementation of SelfHasher would want to call.
This gives us freedom to change the hashing algorithm of deephash
at some point in the future.
Also, this type is likely going to be called by types that are
going to memoize their own hash results, we additionally add
a HashSum method to simplify this use case.
Add documentation to SelfHasher on how a type might implement it.
Updates: corp#16409
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
expvarx.SafeFunc wraps an expvar.Func with a time limit. On reaching the
time limit, calls to Value return nil, and no new concurrent calls to
the underlying expvar.Func will be started until the call completes.
Updates tailscale/corp#16999
Signed-off-by: James Tucker <james@tailscale.com>
For use in corp, where we appear to have re-implemented this in a few
places with varying signatures.
Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: Id863a87e674f3caa87945519be8e09650e9c1d76
This is a useful primitive for asynchronous execution of ordered work I
want to use in another change.
Updates tailscale/corp#16833
Signed-off-by: James Tucker <james@tailscale.com>
Also perform minor cleanups on the ctxkey package itself.
Provide guidance on when to use ctxkey.Key[T] over ctxkey.New.
Also, allow for interface kinds because the value wrapping trick
also happens to fix edge cases with interfaces in Go.
Updates #cleanup
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
The lack of type-safety in context.WithValue leads to the common pattern
of defining of package-scoped type to ensure global uniqueness:
type fooKey struct{}
func withFoo(ctx context, v Foo) context.Context {
return context.WithValue(ctx, fooKey{}, v)
}
func fooValue(ctx context) Foo {
v, _ := ctx.Value(fooKey{}).(Foo)
return v
}
where usage becomes:
ctx = withFoo(ctx, foo)
foo := fooValue(ctx)
With many different context keys, this can be quite tedious.
Using generics, we can simplify this as:
var fooKey = ctxkey.New("mypkg.fooKey", Foo{})
where usage becomes:
ctx = fooKey.WithValue(ctx, foo)
foo := fooKey.Value(ctx)
See https://go.dev/issue/49189
Updates #cleanup
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Run `staticcheck` with `U1000` to find unused code. This cleans up about
a half of it. I'll do the other half separately to keep PRs manageable.
Updates #cleanup
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
The cmpx.Compare function (and associated interface) are now available
in the standard library as cmp.Compare. Remove our version of it and use
the version from the standard library.
Updates #cleanup
Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I4be3ac63d466c05eb7a0babb25cb0d41816fbd53
Ensure that if getOrCreateChain creates a new chain, it actually returns the created chain
Updates tailscale/tailscale#10399
Signed-off-by: Irbe Krumina <irbe@tailscale.com>
Previously, the "RunExitNode" policy merely controlled the visibility of
the "run as exit node" menu item, not the setting itself. This migrates
that setting to a preference option named "AdvertiseExitNode".
Updates ENG-2138
Change-Id: Ia6a125beb6b4563d380c6162637ce4088f1117a0
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
Updates ENG-2513. Renames client metrics keys used on Windows for consistency with Apple platforms.
Signed-off-by: Andrea Gottardo <andrea@tailscale.com>
This package allows caching arbitrary key/value pairs in-memory, along
with an interface implemented by the cache types.
Extracted from #7493
Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: Ic8ca820927c456721cf324a0c8f3882a57752cc9
Due to the Sparkle preference naming convention, macsys already has a
policy key named "ApplyUpdates" that merely shows or hides the menu
item that controls if auto updates are installed, rather than directly
controlling the setting.
For other platforms, we are going to use "InstallUpdates" instead
because it seemed better than the other options that were considered.
Updates ENG-2127
Updates tailscale/corp#16247
Change-Id: Ia6a125beb6b4563d380c6162637ce4088f1117a0
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
Previously, policies affected the default prefs for a new profile, but
that does not affect existing profiles. This change ensures that
policies are applied whenever preferences are loaded or changed, so a
CLI or GUI client that does not respect the policies will still be
overridden.
Exit node IP is dropped from this PR as it was implemented elsewhere
in #10172.
Fixestailscale/corp#15585
Change-Id: Ide4c3a4b00a64e43f506fa1fab70ef591407663f
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
* util/linuxfw, wgengine: allow ingress to magicsock UDP port on Linux
Updates #9084.
Currently, we have to tell users to manually open UDP ports on Linux when
certain firewalls (like ufw) are enabled. This change automates the process of
adding and updating those firewall rules as magicsock changes what port it
listens on.
Signed-off-by: Naman Sood <mail@nsood.in>
These keys were intended to match the Apple platforms, but accidentally
used the wrong name.
Updates ENG-2133
Change-Id: I9ed7a17919e34e2d8896a5c64efc4d0c0003166e
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
This PR is all about adding functionality that will enable the installer's
upgrade sequence to terminate processes belonging to the previous version,
and then subsequently restart instances belonging to the new version within
the session(s) corresponding to the processes that were killed.
There are multiple parts to this:
* We add support for the Restart Manager APIs, which allow us to query the
OS for a list of processes locking specific files;
* We add the RestartableProcess and RestartableProcesses types that query
additional information about the running processes that will allow us
to correctly restart them in the future. These types also provide the
ability to terminate the processes.
* We add the StartProcessInSession family of APIs that permit us to create
new processes within specific sessions. This is needed in order to
properly attach a new GUI process to the same RDP session and desktop that
its previously-terminated counterpart would have been running in.
* I tweaked the winutil token APIs again.
* A lot of this stuff is pretty hard to test without a very elaborate
harness, but I added a unit test for the most complicated part (though it
requires LocalSystem to run).
Updates https://github.com/tailscale/corp/issues/13998
Signed-off-by: Aaron Klotz <aaron@tailscale.com>
Adds policy keys ExitNodeID and ExitNodeIP.
Uses the policy keys to determine the exit node in preferences.
Fixestailscale/corp#15683
Signed-off-by: Claire Wang <claire@tailscale.com>
Marshal as a JSON list instead of a map. Because set elements are
`comparable` and not `cmp.Ordered`, we cannot easily sort the items
before marshaling.
Updates #cleanup
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
These policy keys are supported on Apple platforms in Swift code; in
order to support them on platforms using Go (e.g. Windows), they also
need to be recorded here.
This does not affect any code, it simply adds the constants for now.
Updates ENG-2240
Updates ENG-2127
Updates ENG-2133
Change-Id: I0aa9863a3641e5844479da3b162761452db1ef42
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
The Windows base registry key is already exported but the policy key was
not. util/osdiag currently replicates the string rather than the
preferred approach of reusing the constant.
Updates #cleanup
Change-Id: I6c1c45337896c744059b85643da2364fb3f232f2
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
This PR changes the internal getTokenInfo function to use generics.
I also removed our own implementations for obtaining a token's user
and primary group in favour of calling the ones now available in
x/sys/windows.
Furthermore, I added two new functions for working with tokens, logon
session IDs, and Terminal Services / RDP session IDs.
I modified our privilege enabling code to allow enabling of multiple
privileges via one single function call.
Finally, I added the ProcessImageName function and updated the code in
tailscaled_windows.go to use that instead of directly calling the
underlying API.
All of these changes will be utilized by subsequent PRs pertaining to
this issue.
Updates https://github.com/tailscale/corp/issues/13998
Signed-off-by: Aaron Klotz <aaron@tailscale.com>
This package is a wrapper for os/user that handles non-cgo builds,
gokrazy and user shells.
Updates tailscale/corp#15405
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
To safely request and drop privileges, runtime.Lock/UnlockOSThread and
windows.Impersonate/RevertToSelf should be called. Add these calls to
winutil.EnableCurrentThreadPrivilege so that callers don't need to worry
about it.
Updates https://github.com/tailscale/corp/issues/15488
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
(*Token).IsAdministrator is supposed to return true even when the user is
running with a UAC limited token. The idea is that, for the purposes of
this check, we don't care whether the user is *currently* running with
full Admin rights, we just want to know whether the user can
*potentially* do so.
We accomplish this by querying for the token's "linked token," which
should be the fully-elevated variant, and checking its group memberships.
We also switch ipn/ipnserver/(*Server).connIsLocalAdmin to use the elevation
check to preserve those semantics for tailscale serve; I want the
IsAdministrator check to be used for less sensitive things like toggling
auto-update on and off.
Fixes#10036
Signed-off-by: Aaron Klotz <aaron@tailscale.com>
Noticed both while re-reading this code.
Updates #cleanup
Change-Id: I3b70f1d5dc372853fa292ae1adbdee8cfc6a9a7b
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
In addition to the new policy keys for the new options, some
already-in-use but missing policy keys are also being added to
util/syspolicy.
Updates ENG-2133
Change-Id: Iad08ca47f839ea6a65f81b76b4f9ef21183ebdc6
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
We were previously using the netlink API to see if there are chains/rules that
already exist. This works fine in environments where there is either full
nftable support or no support at all. However, we have identified certain
environments which have partial nftable support and the only feasible way of
detecting such an environment is to try to create some of the chains that we
need.
This adds a check to create a dummy postrouting chain which is immediately
deleted. The goal of the check is to ensure we are able to use nftables and
that it won't error out later. This check is only done in the path where we
detected that the system has no preexisting nftable rules.
Updates #5621
Updates #8555
Updates #8762
Signed-off-by: Maisem Ali <maisem@tailscale.com>
These tests were broken at HEAD. CI currently does not run these
as root, will figure out how to do that in a followup.
Updates #5621
Updates #8555
Updates #8762
Signed-off-by: Maisem Ali <maisem@tailscale.com>
This commit changes the PostureChecking syspolicy key to be a
PreferenceOption(user-defined, always, never) instead of Bool.
This aligns better with the defaults implementation on macOS allowing
CLI arguments to be read when user-defined or no defaults is set.
Updates #tailscale/tailscale/5902
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
This adds support for parsing Range and Content-Range headers
according to RFC 7230. The package could be extended in the future
to handle other headers.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
This migrates containerboot to reuse the NetfilterRunner used
by tailscaled instead of manipulating iptables rule itself.
This has the added advantage of now working with nftables and
we can potentially drop the `iptables` command from the container
image in the future.
Updates #9310
Co-authored-by: Irbe Krumina <irbe@tailscale.com>
Signed-off-by: Maisem Ali <maisem@tailscale.com>
This allows using the fake runner in different packages
that need to manage filter rules.
Updates #cleanup
Signed-off-by: Maisem Ali <maisem@tailscale.com>
Just a refactor to consolidate the firewall detection logic in a single
package so that it can be reused in a later commit by containerboot.
Updates #9310
Signed-off-by: Maisem Ali <maisem@tailscale.com>
Add an explicit accept rule for input to the tun interface, as a mirror
to the explicit rule to accept output from the tun interface.
The rule matches any packet in to our tun interface and accepts it, and
the rule is positioned and prioritized such that it should be evaluated
prior to conventional ufw/iptables/nft rules.
Updates #391Fixes#7332
Updates #9084
Signed-off-by: James Tucker <james@tailscale.com>
Instead of just falling back to making a TCP query to an upstream DNS
server when the UDP query returns a truncated query, also start a TCP
query in parallel with the UDP query after a given race timeout. This
ensures that if the upstream DNS server does not reply over UDP (or if
the response packet is blocked, or there's an error), we can still make
queries if the server replies to TCP queries.
This also adds a new package, util/race, to contain the logic required for
racing two different functions and returning the first non-error answer.
Updates tailscale/corp#14809
Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I4311702016c1093b1beaa31b135da1def6d86316
Then use it in tailcfg which had it duplicated a couple times.
I think we have it a few other places too.
And use slices.Equal in wgengine/router too. (found while looking for callers)
Updates #cleanup
Change-Id: If5350eee9b3ef071882a3db29a305081e4cd9d23
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Add a more generalized package for getting policies.
Updates tailcale/corp#10967
Signed-off-by: Claire Wang <claire@tailscale.com>
Co-authored-by: Adrian Dewhurst <adrian@tailscale.com>
Appears to be a missing nil handling case. I looked back over other
usage of findRule and the others all have nil guards. findRule returns
nil when no rules are found matching the arguments.
Fixes#9553
Signed-off-by: James Tucker <james@tailscale.com>
Fixestailscale/corp#14747
Signed-off-by: David Anderson <danderson@tailscale.com>
Co-authored-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Signed-off-by: David Anderson <danderson@tailscale.com>
I didn't clean up the more idiomatic map[T]bool with true values, at
least yet. I just converted the relatively awkward struct{}-valued
maps.
Updates #cleanup
Change-Id: I758abebd2bb1f64bc7a9d0f25c32298f4679c14f
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
For use in tsweb debug handlers, so that we can easily inspect cache
and limiter state when troubleshooting.
Updates tailscale/corp#3601
Signed-off-by: David Anderson <danderson@tailscale.com>
pre-generics container/list is quite unpleasant to use, and the pointer
manipulation operations for an LRU are simple enough to implement directly
now that we have generic types.
With this change, the LRU uses a ring (aka circularly linked list) rather
than a simple doubly-linked list as its internals, because the ring makes
list manipulation edge cases more regular: the only remaining edge case is
the transition between 0 and 1 elements, rather than also having to deal
specially with manipulating the first and last members of the list.
While the primary purpose was improved readability of the code, as it
turns out removing the indirection through an interface box also speeds
up the LRU:
│ before.txt │ after.txt │
│ sec/op │ sec/op vs base │
LRU-32 67.05n ± 2% 59.73n ± 2% -10.90% (p=0.000 n=20)
│ before.txt │ after.txt │
│ B/op │ B/op vs base │
LRU-32 21.00 ± 0% 10.00 ± 0% -52.38% (p=0.000 n=20)
│ before.txt │ after.txt │
│ allocs/op │ allocs/op vs base │
LRU-32 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=20) ¹
Updates #cleanup
Signed-off-by: David Anderson <danderson@tailscale.com>
The benchmark simulates an LRU being queries with uniformly random
inputs, in a set that's too large for the LRU, which should stress
the eviction codepath.
Signed-off-by: David Anderson <danderson@tailscale.com>
We use it a number of places in different repos. Might as well make
one. Another use is coming.
Updates #cleanup
Change-Id: Ib7ce38de0db35af998171edee81ca875102349a4
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Replace %w verb with %v verb when logging errors.
Use %w only for wrapping errors with fmt.Errorf()
Fixes: #9213
Signed-off-by: Craig Rodrigues <rodrigc@crodrigues.org>
It's very common for OOM crashes on Windows to be caused by lack of page
file space (the NT kernel does not overcommit). Since Windows automatically
manages page file space by default, unless the machine is out of disk space,
this is typically caused by manual page file configurations that are too
small.
This patch obtains the current page file size, the amount of free page file
space, and also determines whether the page file is automatically or manually
managed.
Fixes#9090
Signed-off-by: Aaron Klotz <aaron@tailscale.com>
The Windows Security Center is a component that manages the registration of
security products on a Windows system. Only products that have obtained a
special cert from Microsoft may register themselves using the WSC API.
Practically speaking, most vendors do in fact sign up for the program as it
enhances their legitimacy.
From our perspective, this is useful because it gives us a high-signal
source of information to query for the security products installed on the
system. I've tied this query into the osdiag package and is run during
bugreports.
It uses COM bindings that were automatically generated by my prototype
metadata processor, however that program still has a few bugs, so I had
to make a few manual tweaks. I dropped those binding into an internal
package because (for the moment, at least) they are effectively
purpose-built for the osdiag use case.
We also update the wingoes dependency to pick up BSTR.
Fixes#10646
Signed-off-by: Aaron Klotz <aaron@tailscale.com>
In order for the installer to restart the GUI correctly post-upgrade, we
need the GUI to be able to register its restart preferences.
This PR adds API support for doing so. I'm adding it to OSS so that it
is available should we need to do any such registrations on OSS binaries
in the future.
Updates https://github.com/tailscale/corp/issues/13998
Signed-off-by: Aaron Klotz <aaron@tailscale.com>
And flesh it out and use idiomatic doc style ("whether" for bools)
and end in a period while there anyway.
Updates #cleanup
Change-Id: Ieb82f13969656e2340c3510e7b102dc8e6932611
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
I'd added a test case of deephash against a tailcfg.Node to make sure
it worked at all more than anything. We don't care what the exact
bytes are in this test, just that it doesn't fail. So adjust for that.
Then when we make changes to tailcfg.Node and types under it, we don't
need to keep adjusting this test.
Updates #cleanup
Change-Id: Ibf4fa42820aeab8f5292fe65f9f92ffdb0b4407b
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This commit tries to mimic the way iptables-nft work with the filewall rules. We
follow the convention of using tables like filter, nat and the conventional
chains, to make our nftables implementation work with ufw.
Updates: #391
Signed-off-by: KevinLiang10 <kevinliang@tailscale.com>
