archiver: reduce memory usage for large files

FutureBlob now uses a Take() method as a more memory-efficient way to
retrieve the futures result. In addition, futures are now collected
while saving the file. As only a limited number of blobs can be queued
for uploading, for a large file nearly all FutureBlobs already have
their result ready, such that the FutureBlob object just consumes
memory.
This commit is contained in:
Michael Eischer
2022-05-22 15:14:25 +02:00
parent b817681a11
commit 4a10ebed15
5 changed files with 64 additions and 61 deletions

View File

@@ -34,7 +34,7 @@ func startFileSaver(ctx context.Context, t testing.TB) (*FileSaver, context.Cont
wg, ctx := errgroup.WithContext(ctx)
saveBlob := func(ctx context.Context, tpe restic.BlobType, buf *Buffer) FutureBlob {
ch := make(chan saveBlobResponse)
ch := make(chan SaveBlobResponse)
close(ch)
return FutureBlob{ch: ch}
}