tailscale/ipn/ipnlocal
Joe Tsai 3f27087e9d
taildrop: switch hashing to be streaming based (#9861)
While the previous logic was correct, it did not perform well.
Resuming is a dance between the client and server, where
1. the client requests hashes for a partial file,
2. the server then computes those hashes,
3. the client computes hashes locally and compares them.
4. goto 1 while the partial file still has data

While step 2 is running, the client is sitting idle.
While step 3 is running, the server is sitting idle.

By streaming over the block hash immediately after the server
computes it, the client can start checking the hash,
while the server works on the next hash (in a pipelined manner).
This performs dramatically better and also uses less memory
as we don't need to hold a list of hashes, but only need to
handle one hash at a time.

There are two detriments to this approach:
* The HTTP API relies on a JSON stream,
  which is not a standard REST-like pattern.
  However, since we implement both client and server,
  this is fine.
* While the stream is on-going, we hold an open file handle
  on the server side while the file is being hashed.
  On really slow streams, this could hold a file open forever.

Updates tailscale/corp#14772

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Co-authored-by: Rhea Ghosh <rhea@tailscale.com>
2023-10-17 17:53:40 -07:00
..
testdata ipn/ipnlocal: fix the path for writing cert files (#7203) 2023-02-07 14:34:04 -08:00
breaktcp_darwin.go cmd/tailscale: add debug commands to break connections 2023-08-11 06:37:26 -07:00
breaktcp_linux.go cmd/tailscale: add debug commands to break connections 2023-08-11 06:37:26 -07:00
c2n_pprof.go tailcfg: move LogHeapPprof from Debug to c2n [capver 69] 2023-08-16 20:35:04 -07:00
c2n.go clientupdate: fix background install for linux tarballs (#9852) 2023-10-17 14:24:06 -07:00
cert_js.go ipn/ipnlocal: do unexpired cert renewals in the background 2023-10-12 16:02:45 -07:00
cert_test.go all: use Go 1.21 slices, maps instead of x/exp/{slices,maps} 2023-08-17 08:42:35 -07:00
cert.go ipn/ipnlocal: do unexpired cert renewals in the background 2023-10-12 16:02:45 -07:00
dnsconfig_test.go types/netmap: remove NetworkMap.{Addresses,MachineStatus} 2023-09-18 17:08:11 +01:00
expiry_test.go types/netmap, all: make NetworkMap.SelfNode a tailcfg.NodeView 2023-08-21 13:34:49 -07:00
expiry.go types/netmap, all: make NetworkMap.SelfNode a tailcfg.NodeView 2023-08-21 13:34:49 -07:00
local_test.go ipn/ipnlocal: plumb ExitNodeDNSResolvers for IsWireGuardOnly exit nodes 2023-09-27 19:47:38 -07:00
local.go taildrop: implement asynchronous file deletion (#9844) 2023-10-17 13:46:05 -07:00
loglines_test.go tsd: add package with System type to unify subsystem init, discovery 2023-05-04 14:21:59 -07:00
network-lock_test.go ipn/ipnlocal: add tailnet MagicDNS name to ipn.LoginProfile 2023-09-18 13:58:32 -04:00
network-lock.go ipn/ipnlocal: add tailnet MagicDNS name to ipn.LoginProfile 2023-09-18 13:58:32 -04:00
peerapi_h2c.go all: update copyright and license headers 2023-01-27 15:36:29 -08:00
peerapi_macios_ext.go all: update copyright and license headers 2023-01-27 15:36:29 -08:00
peerapi_test.go taildrop: implement asynchronous file deletion (#9844) 2023-10-17 13:46:05 -07:00
peerapi.go taildrop: switch hashing to be streaming based (#9861) 2023-10-17 17:53:40 -07:00
profiles_notwindows.go ipn/ipnlocal: fix profile duplication 2023-08-08 13:43:37 -06:00
profiles_test.go ipn/ipnlocal: add tailnet MagicDNS name to ipn.LoginProfile 2023-09-18 13:58:32 -04:00
profiles_windows.go ipn/ipnlocal: add more logging during profile migration 2023-06-01 21:00:55 -04:00
profiles.go winutil: refactor methods to get values from registry to also return (#9536) 2023-09-26 13:15:11 -04:00
serve_test.go ipn/ipnlocal: close foreground sessions on SetServeConfig 2023-09-26 00:29:50 +02:00
serve.go ipn/ipnlocal: do unexpired cert renewals in the background 2023-10-12 16:02:45 -07:00
ssh_stub.go all: update copyright and license headers 2023-01-27 15:36:29 -08:00
ssh_test.go ipn/ipnlocal: drop not required StateKey parameter 2023-01-30 17:58:55 -08:00
ssh.go all: use Go 1.21 slices, maps instead of x/exp/{slices,maps} 2023-08-17 08:42:35 -07:00
state_test.go ipn/ipnlocal: add tailnet MagicDNS name to ipn.LoginProfile 2023-09-18 13:58:32 -04:00