taildrop: add logic for resuming partial files (#9785)

We add the following API:
* type FileChecksums
* type Checksum
* func Manager.PartialFiles
* func Manager.HashPartialFile
* func ResumeReader

The Manager methods provide the ability to query for partial files
and retrieve a list of checksums for a given partial file.
The ResumeReader function is a helper that wraps an io.Reader
to discard content that is identical locally and remotely.
The FileChecksums type represents the checksums of a file
and is safe to JSON marshal and send over the wire.

Updates tailscale/corp#14772

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Co-authored-by: Rhea Ghosh <rhea@tailscale.com>
This commit is contained in:
Joe Tsai
2023-10-12 16:50:11 -07:00
committed by GitHub
parent 24f322bc43
commit b1867eb23f
7 changed files with 298 additions and 25 deletions

View File

@@ -22,7 +22,7 @@ type incomingFileKey struct {
}
type incomingFile struct {
clock tstime.Clock
clock tstime.DefaultClock
started time.Time
size int64 // or -1 if unknown; never 0
@@ -62,6 +62,7 @@ func (f *incomingFile) Write(p []byte) (n int, err error) {
// The baseName must be a base filename without any slashes.
// The length is the expected length of content to read from r,
// it may be negative to indicate that it is unknown.
// It returns the length of the entire file.
//
// If there is a failure reading from r, then the partial file is not deleted
// for some period of time. The [Manager.PartialFiles] and [Manager.HashPartialFile]
@@ -78,9 +79,9 @@ func (m *Manager) PutFile(id ClientID, baseName string, r io.Reader, offset, len
case distro.Get() == distro.Unraid && !m.DirectFileMode:
return 0, ErrNotAccessible
}
dstPath, ok := m.joinDir(baseName)
if !ok {
return 0, ErrInvalidFileName
dstPath, err := m.joinDir(baseName)
if err != nil {
return 0, err
}
redactAndLogError := func(action string, err error) error {