While the previous logic was correct, it did not perform well.
Resuming is a dance between the client and server, where
1. the client requests hashes for a partial file,
2. the server then computes those hashes,
3. the client computes hashes locally and compares them.
4. goto 1 while the partial file still has data
While step 2 is running, the client is sitting idle.
While step 3 is running, the server is sitting idle.
By streaming over the block hash immediately after the server
computes it, the client can start checking the hash,
while the server works on the next hash (in a pipelined manner).
This performs dramatically better and also uses less memory
as we don't need to hold a list of hashes, but only need to
handle one hash at a time.
There are two detriments to this approach:
* The HTTP API relies on a JSON stream,
which is not a standard REST-like pattern.
However, since we implement both client and server,
this is fine.
* While the stream is on-going, we hold an open file handle
on the server side while the file is being hashed.
On really slow streams, this could hold a file open forever.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Co-authored-by: Rhea Ghosh <rhea@tailscale.com>
Minor fixes:
* The branch for listing or hashing partial files was inverted.
* The host for peerapi call needs to be real (rather than bogus).
* Handle remote peers that don't support resuming.
* Make resume failures non-fatal (since we can still continue).
This was tested locally, end-to-end system test is future work.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Co-authored-by: Rhea Ghosh <rhea@tailscale.com>
Two bug fixes:
1. when tailscale update is executed as root, `os.UserCacheDir` may
return an error because `$XDG_CACHE_HOME` and `$HOME` are not set;
fallback to `os.TempDir` in those cases
2. on some weird distros (like my EndeavourOS), `/usr/sbin` is just a
symlink to `/usr/bin`; when we resolve `tailscale` binary path from
`tailscaled`, allow `tailscaled` to be in either directory
Updates #755
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
File resumption requires keeping partial files around for some time,
but we must still eventually delete them if never resumed.
Thus, we implement asynchronous file deletion, which could
spawn a background goroutine to delete the files.
We also use the same mechanism for deleting files on Windows,
where a file can't be deleted if there is still an open file handle.
We can enqueue those with the asynchronous file deleter as well.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
The old code would always retain value `true` if it was set once, even
if you then change `prefs.AutoUpdate.Apply` to `false`.
Instead of using the previous value, use the default (envknob) value to
OR with.
Updates #755
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
This change:
* adds a partial files peerAPI endpoint to get a list of partial files
* adds a helper function to extract the basename of a file
* updates the peer put peerAPI endpoint
* updates the file put localapi endpoint to allow resume functionality
Updates #14772
Signed-off-by: Rhea Ghosh <rhea@tailscale.com>
The Sparkle-based update is not quite working yet. Make `NewUpdater`
return `ErrUnsupported` for it to avoid the proliferation of exceptions
up the stack.
Updates #755
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
This commit changes the PostureChecking syspolicy key to be a
PreferenceOption(user-defined, always, never) instead of Bool.
This aligns better with the defaults implementation on macOS allowing
CLI arguments to be read when user-defined or no defaults is set.
Updates #tailscale/tailscale/5902
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
We add the following API:
* type FileChecksums
* type Checksum
* func Manager.PartialFiles
* func Manager.HashPartialFile
* func ResumeReader
The Manager methods provide the ability to query for partial files
and retrieve a list of checksums for a given partial file.
The ResumeReader function is a helper that wraps an io.Reader
to discard content that is identical locally and remotely.
The FileChecksums type represents the checksums of a file
and is safe to JSON marshal and send over the wire.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Co-authored-by: Rhea Ghosh <rhea@tailscale.com>
We were eagerly doing a synchronous renewal of the cert while
trying to serve traffic. Instead of that, just do the cert
renewal in the background and continue serving traffic as long
as the cert is still valid.
This regressed in c1ecae13ab when
we introduced ARI support and were trying to make the experience
of `tailscale cert` better. However, that ended up regressing
the experience for tsnet as it would not always doing the renewal
synchronously.
Fixes#9783
Signed-off-by: Maisem Ali <maisem@tailscale.com>
Changes made:
* Move all HTTP related functionality from taildrop to ipnlocal.
* Add two arguments to taildrop.Manager.PutFile to specify
an opaque client ID and a resume offset (both unused for now).
* Cleanup the logic of taildrop.Manager.PutFile
to be easier to follow.
* Implement file conflict handling where duplicate files are renamed
(e.g., "IMG_1234.jpg" -> "IMG_1234 (2).jpg").
* Implement file de-duplication where "renaming" a partial file
simply deletes it if it already exists with the same contents.
* Detect conflicting active puts where a second concurrent put
results in an error.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Co-authored-by: Rhea Ghosh <rhea@tailscale.com>
These log paths were actually unexpected until the refactor in
fe95d81b43. This moves the logs
to the callsites where they are actually unexpected.
Fixes#9670
Signed-off-by: Maisem Ali <maisem@tailscale.com>
This commit adds support for getting serial numbers from SMBIOS
on Windows/Linux (and BSD) using go-smbios.
Updates #5902
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
Changes made:
* Unexport declarations specific to internal taildrop functionality.
* Document all exported functionality.
* Move TestRedactErr to the taildrop package.
* Rename and invert Handler.DirectFileDoFinalRename as AvoidFinalRename.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
This is being moved to taildrop, so clean it up to stop depending
on so much unreleated functionality by removing a dependency
on peerAPIHandler.
Updates tailscale/corp#14772
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Co-authored-by: Rhea Ghosh <rhea@tailscale.com>
Over time all taildrop functionality will be contained in the
taildrop package. This will include end to end unit tests. This is
simply the first smallest piece to move over.
There is no functionality change in this commit.
Updates tailscale/corp#14772
Signed-off-by: Rhea Ghosh <rhea@tailscale.com>
Co-authored-by: Joseph Tsai <joetsai@tailscale.com>
Advertise it on Android (it looks like it already works once advertised).
And both advertise & likely fix it on iOS. Yet untested.
Updates #9672
Change-Id: If3b7e97f011dea61e7e75aff23dcc178b6cf9123
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This optionally uploads MAC address(es) to control, then adds a
c2n handler so control can ask a node to send a WoL packet.
Updates #306
RELNOTE=now supports waking up peer nodes on your LAN via Wake-on-LAN packets
Change-Id: Ibea1275fcd2048dc61d7059039abfbaf1ad4f465
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Make the 'tailscale debug component-logs' command print the component names for
which extra logging can be turned on, for easier discoverability of debug
functions.
Updates #cleanup
Co-authored-by: Paul Scott <paul@tailscale.com>
Signed-off-by: Val <valerie@tailscale.com>
Thanks to @qur and @eric for debugging!
Fixes#6973
Change-Id: Ib2cf8f030cf595cc73dd061c72e78ac19f5fae5d
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Control sends ExitNodeDNSResolvers when configured for IsWireGuardOnly
nodes that are to be used as the default resolver with a lower
precedence than split DNS, and a lower precedence than "Override local
DNS", but otherwise before local DNS is used when the exit node is in
use.
Neither of the below changes were problematic, but appeared so alongside
a number of other client and external changes. See tailscale/corp#14809.
Reland ea9dd8fabc.
Reland d52ab181c3.
Updates #9377
Updates tailscale/corp#14809
Signed-off-by: James Tucker <james@tailscale.com>
This PR ensures zombie foregrounds are shutdown if a new
ServeConfig is created that wipes the ongoing foreground ones.
For example, "tailscale serve|funnel reset|off" should close
all open sessions.
Updates #8489
Signed-off-by: Marwan Sulaiman <marwan@tailscale.com>
For loading testing & profiling the cost of full netmap updates.
Updates #1909
Change-Id: I0afdf5de9967f8d95c7f81d5b531ed1c92c3208f
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This would've prevented #9470.
This used to pass, fails as of 9538e9f970, and passes again
once #9472 is in.
Updates #9470
Change-Id: Iab97666f7a318432fb3b6372a177ab50c55d4697
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
9538e9f970 broke LocalBackend.WhoIs
where you can no longer lookup yourself in WhoIs.
This occurs because the LocalBackend.peers map only contains peers.
If we fail to lookup a peer, double-check whether it is ourself.
Fixes#9470
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Co-authored-by: Rhea Ghosh <rhea@tailscale.com>
The initial implementation directly mirrored the behavior of Tailscale
exit nodes, where the WireGuard exit node DNS took precedence over other
configuration.
This adjusted implementation treats the WireGuard DNS
resolvers as a lower precedence default resolver than the tailnet
default resolver, and allows split DNS configuration as well.
This also adds test coverage to the existing DNS selection behavior with
respect to default resolvers and split DNS routes for Tailscale exit
nodes above cap 25. There may be some refinement to do in the logic in
those cases, as split DNS may not be working as we intend, though that
would be a pre-existing and separate issue.
Updates #9377
Signed-off-by: James Tucker <james@tailscale.com>
This PR plumbs through awareness of an IPv6 SNAT/masquerade address from the wire protocol
through to the low-level (tstun / wgengine). This PR is the first in two PRs for implementing
IPv6 NAT support to/from peers.
A subsequent PR will implement the data-plane changes to implement IPv6 NAT - this is just plumbing.
Signed-off-by: Tom DNetto <tom@tailscale.com>
Updates ENG-991
Like PeerCapMap, add a field to `tailcfg.Node` which provides
a map of Capability to raw JSON messages which are deferred to be
parsed later by the application code which cares about the specific
capabilities. This effectively allows us to prototype new behavior
without having to commit to a schema in tailcfg, and it also opens up
the possibilities to develop custom behavior in tsnet applications w/o
having to plumb through application specific data in the MapResponse.
Updates #4217
Signed-off-by: Maisem Ali <maisem@tailscale.com>
And convert all callers over to the methods that check SelfNode.
Now we don't have multiple ways to express things in tests (setting
fields on SelfNode vs NetworkMap, sometimes inconsistently) and don't
have multiple ways to check those two fields (often only checking one
or the other).
Updates #9443
Change-Id: I2d7ba1cf6556142d219fae2be6f484f528756e3c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
We have some flaky integration tests elsewhere that have no one place
to ask about the state of the world. This makes LocalBackend be that
place (as it's basically there anyway) but doesn't yet add the ForTest
accessor method.
This adds a LocalBackend.peers map[NodeID]NodeView that is
incrementally updated as mutations arrive. And then we start moving
away from using NetMap.Peers at runtime (UpdateStatus no longer uses
it now). And remove another copy of NodeView in the LocalBackend
nodeByAddr map. Change that to point into b.peers instead.
Future changes will then start streaming whole-node-granularity peer
change updates to WatchIPNBus clients, tracking statefully per client
what each has seen. This will get the GUI clients from receiving less
of a JSON storm of updates all the time.
Updates #1909
Change-Id: I14a976ca9f493bdf02ba7e6e05217363dcf422e5
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
