Add version for 0.16.5

Generate CHANGELOG.md for 0.16.5
Prepare changelog for 0.16.5
2025-08-19 10:37:28 +00:00 · 2024-07-01 21:25:34 +02:00 · 2024-07-01 21:25:33 +02:00 · 2024-07-01 21:25:33 +02:00 · 2024-06-26 21:07:16 +02:00 · 2024-06-26 20:59:51 +02:00
57 changed files with 2312 additions and 1950 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
--- a/2
+++ b/2
@@ -1 +1 @@
-0.16.3
+0.16.5
--- a/changelog/0.16.4_2024-02-04/issue-4529
+++ b/changelog/0.16.4_2024-02-04/issue-4529
@@ -0,0 +1,18 @@
+Enhancement: Add extra verification of data integrity before upload
+
+Hardware issues, or a bug in restic or its dependencies, could previously cause
+corruption in the files restic created and stored in the repository. Detecting
+such corruption previously required explicitly running the `check --read-data`
+or `check --read-data-subset` commands.
+
+To further ensure data integrity, even in the case of hardware issues or
+software bugs, restic now performs additional verification of the files about
+to be uploaded to the repository.
+
+These extra checks will increase CPU usage during backups. They can therefore,
+if absolutely necessary, be disabled using the `--no-extra-verify` global
+option. Please note that this should be combined with more active checking
+using the previously mentioned check commands.
+
+https://github.com/restic/restic/issues/4529
+https://github.com/restic/restic/pull/4681
--- a/changelog/0.16.4_2024-02-04/issue-4677
+++ b/changelog/0.16.4_2024-02-04/issue-4677
@@ -0,0 +1,19 @@
+Bugfix: Downgrade zstd library to fix rare data corruption at max. compression
+
+In restic 0.16.3, backups where the compression level was set to `max` (using
+`--compression max`) could in rare and very specific circumstances result in
+data corruption due to a bug in the library used for compressing data. Restic
+0.16.1 and 0.16.2 were not affected.
+
+Restic now uses the previous version of the library used to compress data, the
+same version used by restic 0.16.2. Please note that the `auto` compression
+level (which restic uses by default) was never affected, and even if you used
+`max` compression, chances of being affected by this issue are small.
+
+To check a repository for any corruption, run `restic check --read-data`. This
+will download and verify the whole repository and can be used at any time to
+completely verify the integrity of a repository. If the `check` command detects
+anomalies, follow the suggested steps.
+
+https://github.com/restic/restic/issues/4677
+https://github.com/restic/restic/pull/4679
--- a/changelog/0.16.5_2024-07-01/issue-4873
+++ b/changelog/0.16.5_2024-07-01/issue-4873
@@ -0,0 +1,6 @@
+Enhancement: Update dependencies
+
+A few potentially vulnerable dependencies were updated.
+
+https://github.com/restic/restic/issues/4873
+https://github.com/restic/restic/pull/4878
--- a/changelog/0.16.5_2024-07-01/pull-4799
+++ b/changelog/0.16.5_2024-07-01/pull-4799
@@ -0,0 +1,5 @@
+Enhancement: Add option to force use of Azure CLI credential
+
+A new environment variable `AZURE_FORCE_CLI_CREDENTIAL=true` allows forcing the use of Azure CLI credential, ignoring other credentials like managed identity.
+
+https://github.com/restic/restic/pull/4799
--- a/cmd/restic/global.go
+++ b/cmd/restic/global.go
@@ -43,7 +43,7 @@ import (
 	"golang.org/x/term"
 )

-var version = "0.16.3"
+var version = "0.16.5"

 // TimeFormat is the format used for all timestamps printed by restic.
 const TimeFormat = "2006-01-02 15:04:05"
@@ -67,6 +67,7 @@ type GlobalOptions struct {
 	CleanupCache    bool
 	Compression     repository.CompressionMode
 	PackSize        uint
+	NoExtraVerify   bool

 	backend.TransportOptions
 	limiter.Limits
@@ -139,6 +140,7 @@ func init() {
 	f.BoolVar(&globalOptions.InsecureTLS, "insecure-tls", false, "skip TLS certificate verification when connecting to the repository (insecure)")
 	f.BoolVar(&globalOptions.CleanupCache, "cleanup-cache", false, "auto remove old cache directories")
 	f.Var(&globalOptions.Compression, "compression", "compression mode (only available for repository format version 2), one of (auto|off|max) (default: $RESTIC_COMPRESSION)")
+	f.BoolVar(&globalOptions.NoExtraVerify, "no-extra-verify", false, "skip additional verification of data before upload (see documentation)")
 	f.IntVar(&globalOptions.Limits.UploadKb, "limit-upload", 0, "limits uploads to a maximum `rate` in KiB/s. (default: unlimited)")
 	f.IntVar(&globalOptions.Limits.DownloadKb, "limit-download", 0, "limits downloads to a maximum `rate` in KiB/s. (default: unlimited)")
 	f.UintVar(&globalOptions.PackSize, "pack-size", 0, "set target pack `size` in MiB, created pack files may be larger (default: $RESTIC_PACK_SIZE)")
@@ -453,8 +455,9 @@ func OpenRepository(ctx context.Context, opts GlobalOptions) (*repository.Reposi
 	}

 	s, err := repository.New(be, repository.Options{
-		Compression: opts.Compression,
-		PackSize:    opts.PackSize * 1024 * 1024,
+		Compression:   opts.Compression,
+		PackSize:      opts.PackSize * 1024 * 1024,
+		NoExtraVerify: opts.NoExtraVerify,
 	})
 	if err != nil {
 		return nil, errors.Fatal(err.Error())
--- a/doc/030_preparing_a_new_repo.rst
+++ b/doc/030_preparing_a_new_repo.rst
@@ -548,9 +548,23 @@ For authentication export one of the following variables:
    # For SAS
    $ export AZURE_ACCOUNT_SAS=<SAS_TOKEN>

-Alternatively, if run on Azure, restic will automatically uses service accounts configured
+For authentication using ``az login`` ensure the user has
+the minimum permissions of the role assignment ``Storage Blob Data Contributor`` on Azure RBAC
+for the storage account.
+
+.. code-block:: console
+
+    $ az login
+
+Alternatively, if run on Azure, restic will automatically use service accounts configured
 via the standard environment variables or Workload / Managed Identities.

+To enforce the use of the Azure CLI credential when other credentials are present, set the following environment variable:
+
+.. code-block:: console
+
+    $ export AZURE_FORCE_CLI_CREDENTIAL=true
+
 Restic will by default use Azure's global domain ``core.windows.net`` as endpoint suffix.
 You can specify other suffixes as follows:

--- a/doc/040_backup.rst
+++ b/doc/040_backup.rst
@@ -597,6 +597,7 @@ environment variables. The following lists these environment variables:
    AZURE_ACCOUNT_KEY                   Account key for Azure
    AZURE_ACCOUNT_SAS                   Shared access signatures (SAS) for Azure
    AZURE_ENDPOINT_SUFFIX               Endpoint suffix for Azure Storage (default: core.windows.net)
+    AZURE_FORCE_CLI_CREDENTIAL          Force the use of Azure CLI credentials for authentication

    B2_ACCOUNT_ID                       Account ID or applicationKeyId for Backblaze B2
    B2_ACCOUNT_KEY                      Account Key or applicationKey for Backblaze B2
--- a/doc/047_tuning_backup_parameters.rst
+++ b/doc/047_tuning_backup_parameters.rst
@@ -60,6 +60,20 @@ only applied for the single run of restic. The option can also be set via the en
 variable ``RESTIC_COMPRESSION``.


+Data Verification
+=================
+
+To prevent the upload of corrupted data to the repository, which can happen due
+to hardware issues or software bugs, restic verifies that generated files can
+be decoded and contain the correct data beforehand. This increases the CPU usage
+during backups. If necessary, you can disable this verification using the
+``--no-extra-verify`` option of the ``backup`` command. However, in this case
+you should verify the repository integrity more actively using
+``restic check --read-data`` (or the similar ``--read-data-subset`` option).
+Otherwise, data corruption due to hardware issues or software bugs might go
+unnoticed.
+
+
 File Read Concurrency
 =====================

--- a/doc/bash-completion.sh
+++ b/doc/bash-completion.sh
@@ -488,6 +488,7 @@ _restic_backup()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -560,6 +561,7 @@ _restic_cache()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -624,6 +626,7 @@ _restic_cat()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -696,6 +699,7 @@ _restic_check()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -794,6 +798,7 @@ _restic_copy()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -860,6 +865,7 @@ _restic_diff()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -944,6 +950,7 @@ _restic_dump()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -1058,6 +1065,7 @@ _restic_find()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -1228,6 +1236,7 @@ _restic_forget()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -1312,6 +1321,7 @@ _restic_generate()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -1372,6 +1382,7 @@ _restic_help()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -1463,6 +1474,7 @@ _restic_init()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -1539,6 +1551,7 @@ _restic_key()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -1603,6 +1616,7 @@ _restic_list()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -1689,6 +1703,7 @@ _restic_ls()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -1757,6 +1772,7 @@ _restic_migrate()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -1849,6 +1865,7 @@ _restic_mount()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -1935,6 +1952,7 @@ _restic_prune()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -1999,6 +2017,7 @@ _restic_recover()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -2059,6 +2078,7 @@ _restic_repair_help()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -2126,6 +2146,7 @@ _restic_repair_index()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -2190,6 +2211,7 @@ _restic_repair_packs()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -2274,6 +2296,7 @@ _restic_repair_snapshots()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -2342,6 +2365,7 @@ _restic_repair()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -2450,6 +2474,7 @@ _restic_restore()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -2552,6 +2577,7 @@ _restic_rewrite()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -2620,6 +2646,7 @@ _restic_self-update()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -2712,6 +2739,7 @@ _restic_snapshots()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -2794,6 +2822,7 @@ _restic_stats()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -2884,6 +2913,7 @@ _restic_tag()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -2950,6 +2980,7 @@ _restic_unlock()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -3014,6 +3045,7 @@ _restic_version()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
@@ -3106,6 +3138,7 @@ _restic_root_command()
    flags+=("--limit-upload=")
    two_word_flags+=("--limit-upload")
    flags+=("--no-cache")
+    flags+=("--no-extra-verify")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("--option")
--- a/doc/man/restic-backup.1
+++ b/doc/man/restic-backup.1
@@ -171,6 +171,10 @@ Exit status is 3 if some source data could not be read (incomplete snapshot crea
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-cache.1
+++ b/doc/man/restic-cache.1
@@ -80,6 +80,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-cat.1
+++ b/doc/man/restic-cat.1
@@ -68,6 +68,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-check.1
+++ b/doc/man/restic-check.1
@@ -85,6 +85,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-copy.1
+++ b/doc/man/restic-copy.1
@@ -109,6 +109,10 @@ new destination repository using the "init" command.
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-diff.1
+++ b/doc/man/restic-diff.1
@@ -93,6 +93,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-dump.1
+++ b/doc/man/restic-dump.1
@@ -96,6 +96,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-find.1
+++ b/doc/man/restic-find.1
@@ -117,6 +117,10 @@ It can also be used to search for restic blobs or trees for troubleshooting.
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-forget.1
+++ b/doc/man/restic-forget.1
@@ -179,6 +179,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-generate.1
+++ b/doc/man/restic-generate.1
@@ -89,6 +89,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-init.1
+++ b/doc/man/restic-init.1
@@ -96,6 +96,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-key.1
+++ b/doc/man/restic-key.1
@@ -80,6 +80,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-list.1
+++ b/doc/man/restic-list.1
@@ -68,6 +68,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-ls.1
+++ b/doc/man/restic-ls.1
@@ -107,6 +107,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-migrate.1
+++ b/doc/man/restic-migrate.1
@@ -74,6 +74,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-mount.1
+++ b/doc/man/restic-mount.1
@@ -144,6 +144,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-prune.1
+++ b/doc/man/restic-prune.1
@@ -97,6 +97,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-recover.1
+++ b/doc/man/restic-recover.1
@@ -70,6 +70,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-repair-index.1
+++ b/doc/man/restic-repair-index.1
@@ -73,6 +73,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-repair-packs.1
+++ b/doc/man/restic-repair-packs.1
@@ -72,6 +72,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-repair-snapshots.1
+++ b/doc/man/restic-repair-snapshots.1
@@ -107,6 +107,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-repair.1
+++ b/doc/man/restic-repair.1
@@ -63,6 +63,10 @@ Repair the repository
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-restore.1
+++ b/doc/man/restic-restore.1
@@ -117,6 +117,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-rewrite.1
+++ b/doc/man/restic-rewrite.1
@@ -121,6 +121,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-self-update.1
+++ b/doc/man/restic-self-update.1
@@ -75,6 +75,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-snapshots.1
+++ b/doc/man/restic-snapshots.1
@@ -92,6 +92,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-stats.1
+++ b/doc/man/restic-stats.1
@@ -114,6 +114,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-tag.1
+++ b/doc/man/restic-tag.1
@@ -99,6 +99,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-unlock.1
+++ b/doc/man/restic-unlock.1
@@ -72,6 +72,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic-version.1
+++ b/doc/man/restic-version.1
@@ -69,6 +69,10 @@ Exit status is 0 if the command was successful, and non-zero if there was any er
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/doc/man/restic.1
+++ b/doc/man/restic.1
@@ -65,6 +65,10 @@ The full documentation can be found at https://restic.readthedocs.io/ .
 \fB--no-cache\fP[=false]
 	do not use a local cache

+.PP
+\fB--no-extra-verify\fP[=false]
+	skip additional verification of data before upload (see documentation)
+
 .PP
 \fB--no-lock\fP[=false]
 	do not lock the repository, this allows some operations on read-only repositories
--- a/go.mod
+++ b/go.mod
@@ -1,9 +1,9 @@
 module github.com/restic/restic

 require (
-	cloud.google.com/go/storage v1.34.0
-	github.com/Azure/azure-sdk-for-go/sdk/azcore v1.9.1
-	github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.4.0
+	cloud.google.com/go/storage v1.41.0
+	github.com/Azure/azure-sdk-for-go/sdk/azcore v1.11.1
+	github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.6.0
 	github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.2.0
 	github.com/Backblaze/blazer v0.6.1
 	github.com/anacrolix/fuse v0.2.0
@@ -25,56 +25,64 @@ require (
 	github.com/spf13/cobra v1.7.0
 	github.com/spf13/pflag v1.0.5
 	go.uber.org/automaxprocs v1.5.3
-	golang.org/x/crypto v0.17.0
-	golang.org/x/net v0.19.0
-	golang.org/x/oauth2 v0.15.0
-	golang.org/x/sync v0.5.0
-	golang.org/x/sys v0.15.0
-	golang.org/x/term v0.15.0
-	golang.org/x/text v0.14.0
+	golang.org/x/crypto v0.24.0
+	golang.org/x/net v0.26.0
+	golang.org/x/oauth2 v0.20.0
+	golang.org/x/sync v0.7.0
+	golang.org/x/sys v0.21.0
+	golang.org/x/term v0.21.0
+	golang.org/x/text v0.16.0
 	golang.org/x/time v0.5.0
-	google.golang.org/api v0.149.0
+	google.golang.org/api v0.182.0
 )

+replace github.com/klauspost/compress => github.com/klauspost/compress v1.17.2
+
 require (
-	cloud.google.com/go v0.110.9 // indirect
-	cloud.google.com/go/compute v1.23.1 // indirect
-	cloud.google.com/go/compute/metadata v0.2.3 // indirect
-	cloud.google.com/go/iam v1.1.3 // indirect
-	github.com/Azure/azure-sdk-for-go/sdk/internal v1.5.1 // indirect
-	github.com/AzureAD/microsoft-authentication-library-for-go v1.2.0 // indirect
+	cloud.google.com/go v0.114.0 // indirect
+	cloud.google.com/go/auth v0.4.2 // indirect
+	cloud.google.com/go/auth/oauth2adapt v0.2.2 // indirect
+	cloud.google.com/go/compute/metadata v0.3.0 // indirect
+	cloud.google.com/go/iam v1.1.8 // indirect
+	github.com/Azure/azure-sdk-for-go/sdk/internal v1.8.0 // indirect
+	github.com/AzureAD/microsoft-authentication-library-for-go v1.2.2 // indirect
 	github.com/cpuguy83/go-md2man/v2 v2.0.3 // indirect
 	github.com/dustin/go-humanize v1.0.1 // indirect
 	github.com/felixge/fgprof v0.9.3 // indirect
-	github.com/golang-jwt/jwt/v5 v5.0.0 // indirect
+	github.com/felixge/httpsnoop v1.0.4 // indirect
+	github.com/go-logr/logr v1.4.1 // indirect
+	github.com/go-logr/stdr v1.2.2 // indirect
+	github.com/golang-jwt/jwt/v5 v5.2.1 // indirect
 	github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
-	github.com/golang/protobuf v1.5.3 // indirect
+	github.com/golang/protobuf v1.5.4 // indirect
 	github.com/google/pprof v0.0.0-20230926050212-f7f687d19a98 // indirect
 	github.com/google/s2a-go v0.1.7 // indirect
-	github.com/google/uuid v1.5.0 // indirect
+	github.com/google/uuid v1.6.0 // indirect
 	github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
-	github.com/googleapis/gax-go/v2 v2.12.0 // indirect
+	github.com/googleapis/gax-go/v2 v2.12.4 // indirect
 	github.com/inconshreveable/mousetrap v1.1.0 // indirect
 	github.com/json-iterator/go v1.1.12 // indirect
 	github.com/klauspost/cpuid/v2 v2.2.6 // indirect
 	github.com/kr/fs v0.1.0 // indirect
-	github.com/kr/text v0.2.0 // indirect
 	github.com/kylelemons/godebug v1.1.0 // indirect
 	github.com/minio/md5-simd v1.1.2 // indirect
 	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
 	github.com/modern-go/reflect2 v1.0.2 // indirect
-	github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8 // indirect
+	github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
 	github.com/rs/xid v1.5.0 // indirect
 	github.com/russross/blackfriday/v2 v2.1.0 // indirect
 	github.com/sirupsen/logrus v1.9.3 // indirect
 	go.opencensus.io v0.24.0 // indirect
-	golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 // indirect
-	google.golang.org/appengine v1.6.8 // indirect
-	google.golang.org/genproto v0.0.0-20231016165738-49dd2c1f3d0b // indirect
-	google.golang.org/genproto/googleapis/api v0.0.0-20231016165738-49dd2c1f3d0b // indirect
-	google.golang.org/genproto/googleapis/rpc v0.0.0-20231016165738-49dd2c1f3d0b // indirect
-	google.golang.org/grpc v1.59.0 // indirect
-	google.golang.org/protobuf v1.31.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.49.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 // indirect
+	go.opentelemetry.io/otel v1.24.0 // indirect
+	go.opentelemetry.io/otel/metric v1.24.0 // indirect
+	go.opentelemetry.io/otel/trace v1.24.0 // indirect
+	google.golang.org/genproto v0.0.0-20240401170217-c3f982113cda // indirect
+	google.golang.org/genproto/googleapis/api v0.0.0-20240513163218-0867130af1f8 // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20240521202816-d264139d666e // indirect
+	google.golang.org/grpc v1.64.0 // indirect
+	google.golang.org/protobuf v1.34.1 // indirect
 	gopkg.in/ini.v1 v1.67.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 )
--- a/go.sum
+++ b/go.sum
@@ -1,25 +1,27 @@
 cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
-cloud.google.com/go v0.110.9 h1:e7ITSqGFFk4rbz/JFIqZh3G4VEHguhAL4BQcFlWtU68=
-cloud.google.com/go v0.110.9/go.mod h1:rpxevX/0Lqvlbc88b7Sc1SPNdyK1riNBTUU6JXhYNpM=
-cloud.google.com/go/compute v1.23.1 h1:V97tBoDaZHb6leicZ1G6DLK2BAaZLJ/7+9BB/En3hR0=
-cloud.google.com/go/compute v1.23.1/go.mod h1:CqB3xpmPKKt3OJpW2ndFIXnA9A4xAy/F3Xp1ixncW78=
-cloud.google.com/go/compute/metadata v0.2.3 h1:mg4jlk7mCAj6xXp9UJ4fjI9VUI5rubuGBW5aJ7UnBMY=
-cloud.google.com/go/compute/metadata v0.2.3/go.mod h1:VAV5nSsACxMJvgaAuX6Pk2AawlZn8kiOGuCv6gTkwuA=
-cloud.google.com/go/iam v1.1.3 h1:18tKG7DzydKWUnLjonWcJO6wjSCAtzh4GcRKlH/Hrzc=
-cloud.google.com/go/iam v1.1.3/go.mod h1:3khUlaBXfPKKe7huYgEpDn6FtgRyMEqbkvBxrQyY5SE=
-cloud.google.com/go/storage v1.34.0 h1:9KHBBTbaHPsNxO043SFmH3pMojjZiW+BFl9H41L7xjk=
-cloud.google.com/go/storage v1.34.0/go.mod h1:Eji+S0CCQebjsiXxyIvPItC3BN3zWsdJjWfHfoLblgY=
-github.com/Azure/azure-sdk-for-go/sdk/azcore v1.9.1 h1:lGlwhPtrX6EVml1hO0ivjkUxsSyl4dsiw9qcA1k/3IQ=
-github.com/Azure/azure-sdk-for-go/sdk/azcore v1.9.1/go.mod h1:RKUqNu35KJYcVG/fqTRqmuXJZYNhYkBrnC/hX7yGbTA=
-github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.4.0 h1:BMAjVKJM0U/CYF27gA0ZMmXGkOcvfFtD0oHVZ1TIPRI=
-github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.4.0/go.mod h1:1fXstnBMas5kzG+S3q8UoJcmyU6nUeunJcMDHcRYHhs=
-github.com/Azure/azure-sdk-for-go/sdk/internal v1.5.1 h1:6oNBlSdi1QqM1PNW7FPA6xOGA5UNsXnkaYZz9vdPGhA=
-github.com/Azure/azure-sdk-for-go/sdk/internal v1.5.1/go.mod h1:s4kgfzA0covAXNicZHDMN58jExvcng2mC/DepXiF1EI=
+cloud.google.com/go v0.114.0 h1:OIPFAdfrFDFO2ve2U7r/H5SwSbBzEdrBdE7xkgwc+kY=
+cloud.google.com/go v0.114.0/go.mod h1:ZV9La5YYxctro1HTPug5lXH/GefROyW8PPD4T8n9J8E=
+cloud.google.com/go/auth v0.4.2 h1:sb0eyLkhRtpq5jA+a8KWw0W70YcdVca7KJ8TM0AFYDg=
+cloud.google.com/go/auth v0.4.2/go.mod h1:Kqvlz1cf1sNA0D+sYJnkPQOP+JMHkuHeIgVmCRtZOLc=
+cloud.google.com/go/auth/oauth2adapt v0.2.2 h1:+TTV8aXpjeChS9M+aTtN/TjdQnzJvmzKFt//oWu7HX4=
+cloud.google.com/go/auth/oauth2adapt v0.2.2/go.mod h1:wcYjgpZI9+Yu7LyYBg4pqSiaRkfEK3GQcpb7C/uyF1Q=
+cloud.google.com/go/compute/metadata v0.3.0 h1:Tz+eQXMEqDIKRsmY3cHTL6FVaynIjX2QxYC4trgAKZc=
+cloud.google.com/go/compute/metadata v0.3.0/go.mod h1:zFmK7XCadkQkj6TtorcaGlCW1hT1fIilQDwofLpJ20k=
+cloud.google.com/go/iam v1.1.8 h1:r7umDwhj+BQyz0ScZMp4QrGXjSTI3ZINnpgU2nlB/K0=
+cloud.google.com/go/iam v1.1.8/go.mod h1:GvE6lyMmfxXauzNq8NbgJbeVQNspG+tcdL/W8QO1+zE=
+cloud.google.com/go/storage v1.41.0 h1:RusiwatSu6lHeEXe3kglxakAmAbfV+rhtPqA6i8RBx0=
+cloud.google.com/go/storage v1.41.0/go.mod h1:J1WCa/Z2FcgdEDuPUY8DxT5I+d9mFKsCepp5vR6Sq80=
+github.com/Azure/azure-sdk-for-go/sdk/azcore v1.11.1 h1:E+OJmp2tPvt1W+amx48v1eqbjDYsgN+RzP4q16yV5eM=
+github.com/Azure/azure-sdk-for-go/sdk/azcore v1.11.1/go.mod h1:a6xsAQUZg+VsS3TJ05SRp524Hs4pZ/AeFSr5ENf0Yjo=
+github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.6.0 h1:U2rTu3Ef+7w9FHKIAXM6ZyqF3UOWJZ12zIm8zECAFfg=
+github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.6.0/go.mod h1:9kIvujWAA58nmPmWB1m23fyWic1kYZMxD9CxaWn4Qpg=
+github.com/Azure/azure-sdk-for-go/sdk/internal v1.8.0 h1:jBQA3cKT4L2rWMpgE7Yt3Hwh2aUj8KXjIGLxjHeYNNo=
+github.com/Azure/azure-sdk-for-go/sdk/internal v1.8.0/go.mod h1:4OG6tQ9EOP/MT0NMjDlRzWoVFxfu9rN9B2X+tlSVktg=
 github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.2.0 h1:Ma67P/GGprNwsslzEH6+Kb8nybI8jpDTm4Wmzu2ReK8=
 github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.2.0 h1:gggzg0SUMs6SQbEw+3LoSsYf9YMjkupeAnHMX8O9mmY=
 github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.2.0/go.mod h1:+6KLcKIVgxoBDMqMO/Nvy7bZ9a0nbU3I1DtFQK3YvB4=
-github.com/AzureAD/microsoft-authentication-library-for-go v1.2.0 h1:hVeq+yCyUi+MsoO/CU95yqCIcdzra5ovzk8Q2BBpV2M=
-github.com/AzureAD/microsoft-authentication-library-for-go v1.2.0/go.mod h1:wP83P5OoQ5p6ip3ScPr0BAq0BvuPAvacpEuSzyouqAI=
+github.com/AzureAD/microsoft-authentication-library-for-go v1.2.2 h1:XHOnouVk1mxXfQidrMEnLlPk9UMeRtyBTnEFtxkV0kU=
+github.com/AzureAD/microsoft-authentication-library-for-go v1.2.2/go.mod h1:wP83P5OoQ5p6ip3ScPr0BAq0BvuPAvacpEuSzyouqAI=
 github.com/Backblaze/blazer v0.6.1 h1:xC9HyC7OcxRzzmtfRiikIEvq4HZYWjU6caFwX2EXw1s=
 github.com/Backblaze/blazer v0.6.1/go.mod h1:7/jrGx4O6OKOto6av+hLwelPR8rwZ+PLxQ5ZOiYAjwY=
 github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
@@ -39,11 +41,9 @@ github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGX
 github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
 github.com/cpuguy83/go-md2man/v2 v2.0.3 h1:qMCsGGgs+MAzDFyp9LpAe1Lqy/fY/qCovCm0qnXZOBM=
 github.com/cpuguy83/go-md2man/v2 v2.0.3/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
-github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
-github.com/dnaeon/go-vcr v1.2.0 h1:zHCHvJYTMh1N7xnV7zf1m1GPBF9Ad0Jk/whtQ1663qI=
 github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
 github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
 github.com/dvyukov/go-fuzz v0.0.0-20200318091601-be3528f3a813/go.mod h1:11Gm+ccJnvAhCNLlf5+cS9KjtbaD5I5zaZpFMsTHWTw=
@@ -56,10 +56,17 @@ github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1m
 github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
 github.com/felixge/fgprof v0.9.3 h1:VvyZxILNuCiUCSXtPtYmmtGvb65nqXh2QFWc0Wpf2/g=
 github.com/felixge/fgprof v0.9.3/go.mod h1:RdbpDgzqYVh/T9fPELJyV7EYJuHB55UTEULNun8eiPw=
+github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
+github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
+github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
+github.com/go-logr/logr v1.4.1 h1:pKouT5E8xu9zeFC39JXRDukb6JFQPXM5p5I91188VAQ=
+github.com/go-logr/logr v1.4.1/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
+github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
+github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
 github.com/go-ole/go-ole v1.3.0 h1:Dt6ye7+vXGIKZ7Xtk4s6/xVdGDQynvom7xCFEdWr6uE=
 github.com/go-ole/go-ole v1.3.0/go.mod h1:5LS6F96DhAwUc7C+1HLexzMXY1xGRSryjyPPKW6zv78=
-github.com/golang-jwt/jwt/v5 v5.0.0 h1:1n1XNM9hk7O9mnQoNBGolZvzebBQ7p93ULHRc28XJUE=
-github.com/golang-jwt/jwt/v5 v5.0.0/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
+github.com/golang-jwt/jwt/v5 v5.2.1 h1:OuVbFODueb089Lh128TAcimifWaLhJwVflnrgM17wHk=
+github.com/golang-jwt/jwt/v5 v5.2.1/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
 github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
 github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
 github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
@@ -74,33 +81,30 @@ github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:W
 github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
 github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
 github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
-github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
-github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
-github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg=
-github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
+github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
+github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
 github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
 github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
 github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
 github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
 github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
 github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
-github.com/google/martian/v3 v3.3.2 h1:IqNFLAmvJOgVlpdEBiQbDc2EwKW77amAycfTuWKdfvw=
+github.com/google/martian/v3 v3.3.3 h1:DIhPTQrbPkgs2yJYdXU/eNACCG5DVQjySNRNlflZ9Fc=
 github.com/google/pprof v0.0.0-20211214055906-6f57359322fd/go.mod h1:KgnwoLYCZ8IQu3XUZ8Nc/bM9CCZFOyjUNOSygVozoDg=
 github.com/google/pprof v0.0.0-20230926050212-f7f687d19a98 h1:pUa4ghanp6q4IJHwE9RwLgmVFfReJN+KbQ8ExNEUUoQ=
 github.com/google/pprof v0.0.0-20230926050212-f7f687d19a98/go.mod h1:czg5+yv1E0ZGTi6S6vVK1mke0fV+FaUhNGcd6VRS9Ik=
 github.com/google/s2a-go v0.1.7 h1:60BLSyTrOV4/haCDW4zb1guZItoSq8foHCXrAnjBo/o=
 github.com/google/s2a-go v0.1.7/go.mod h1:50CgR4k1jNlWBu4UfS4AcfhVe1r6pdZPygJ3R8F0Qdw=
 github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
-github.com/google/uuid v1.5.0 h1:1p67kYwdtXjb0gL0BPiP1Av9wiZPo5A8z2cWkTZ+eyU=
-github.com/google/uuid v1.5.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
+github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
 github.com/googleapis/enterprise-certificate-proxy v0.3.2 h1:Vie5ybvEvT75RniqhfFxPRy3Bf7vr3h0cechB90XaQs=
 github.com/googleapis/enterprise-certificate-proxy v0.3.2/go.mod h1:VLSiSSBs/ksPL8kq3OBOQ6WRI2QnaFynd1DCjZ62+V0=
-github.com/googleapis/gax-go/v2 v2.12.0 h1:A+gCJKdRfqXkr+BIRGtZLibNXf0m1f9E4HG56etFpas=
-github.com/googleapis/gax-go/v2 v2.12.0/go.mod h1:y+aIqrI5eb1YGMVJfuV3185Ts/D7qKpsEkdD5+I6QGU=
+github.com/googleapis/gax-go/v2 v2.12.4 h1:9gWcmF85Wvq4ryPFvGFaOgPIs1AQX0d0bcbGw4Z96qg=
+github.com/googleapis/gax-go/v2 v2.12.4/go.mod h1:KYEYLorsnIGDi/rPC8b5TdlB9kbKoFubselGIoBMCwI=
 github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
 github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
 github.com/ianlancetaylor/demangle v0.0.0-20210905161508-09a460cdf81d/go.mod h1:aYm2/VgdVmcIU8iMfdMvDMsRAQjcfZSKFby6HOFvi/w=
@@ -108,16 +112,15 @@ github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2
 github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
 github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
 github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
-github.com/klauspost/compress v1.17.4 h1:Ej5ixsIri7BrIjBkRZLTo6ghwrEtHFk7ijlczPW4fZ4=
-github.com/klauspost/compress v1.17.4/go.mod h1:/dCuZOvVtNoHsyb+cuJD3itjs3NbnF6KH9zAO4BDxPM=
+github.com/klauspost/compress v1.17.2 h1:RlWWUY/Dr4fL8qk9YG7DTZ7PDgME2V4csBXA8L/ixi4=
+github.com/klauspost/compress v1.17.2/go.mod h1:ntbaceVETuRiXiv4DpjP66DpAtAGkEQskQzEyD//IeE=
 github.com/klauspost/cpuid/v2 v2.0.1/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
 github.com/klauspost/cpuid/v2 v2.2.6 h1:ndNyv040zDGIDh8thGkXYjnFtiN02M1PVVF+JE/48xc=
 github.com/klauspost/cpuid/v2 v2.2.6/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws=
 github.com/kr/fs v0.1.0 h1:Jskdu9ieNAYnjxsi0LbQp1ulIKZV1LAFgK1tWhpZgl8=
 github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
-github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
+github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
 github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
-github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
 github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
 github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
 github.com/minio/md5-simd v1.1.2 h1:Gdi1DZK69+ZVMoNHRXJyNcxrMA4dSxoYHZSQbirFg34=
@@ -133,8 +136,8 @@ github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9G
 github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
 github.com/ncw/swift/v2 v2.0.2 h1:jx282pcAKFhmoZBSdMcCRFn9VWkoBIRsCpe+yZq7vEk=
 github.com/ncw/swift/v2 v2.0.2/go.mod h1:z0A9RVdYPjNjXVo2pDOPxZ4eu3oarO1P91fTItcb+Kg=
-github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8 h1:KoWmjvw+nsYOo29YJK9vDA65RGE3NrOnUtO7a+RF9HU=
-github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8/go.mod h1:HKlIX3XHQyzLZPlr7++PzdhaXEj94dEiJgZDTsxEqUI=
+github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c h1:+mdjkGKdHQG3305AYmdv1U2eRNDiU2ErMBj1gwrq8eQ=
+github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c/go.mod h1:7rwL4CYBLnjLxUqIJNnCWiEdr3bn6IUYi15bNlnbCCU=
 github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
 github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pkg/profile v1.7.0 h1:hnbDkaNWPCLMO9wGLdBFTIZvzDrDfBM2072E1S9gJkA=
@@ -150,6 +153,7 @@ github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:
 github.com/restic/chunker v0.4.0 h1:YUPYCUn70MYP7VO4yllypp2SjmsRhRJaad3xKu1QFRw=
 github.com/restic/chunker v0.4.0/go.mod h1:z0cH2BejpW636LXw0R/BGyv+Ey8+m9QGiOanDHItzyw=
 github.com/robertkrimen/godocdown v0.0.0-20130622164427-0bfa04905481/go.mod h1:C9WhFzY47SzYBIvzFqSvHIR6ROgDo4TtdTuRaOMjF/s=
+github.com/rogpeppe/go-internal v1.12.0 h1:exVL4IDcn6na9z1rAb56Vxr+CgyK3nn3O+epU5NdKM8=
 github.com/rs/xid v1.5.0 h1:mKX4bl4iPYJtEIxp6CYiUuLQ/8DYMoz0PUdtGgMFRVc=
 github.com/rs/xid v1.5.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg=
 github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
@@ -169,13 +173,24 @@ github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
 github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
 github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
-github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
+github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
 github.com/tv42/httpunix v0.0.0-20191220191345-2ba4b9c3382c h1:u6SKchux2yDvFQnDHS3lPnIRmfVJ5Sxy3ao2SIdysLQ=
 github.com/tv42/httpunix v0.0.0-20191220191345-2ba4b9c3382c/go.mod h1:hzIxponao9Kjc7aWznkXaL4U4TWaDSs8zcsY4Ka08nM=
 github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
 go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=
 go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=
+go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.49.0 h1:4Pp6oUg3+e/6M4C0A/3kJ2VYa++dsWVTtGgLVj5xtHg=
+go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.49.0/go.mod h1:Mjt1i1INqiaoZOMGR1RIUJN+i3ChKoFRqzrRQhlkbs0=
+go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 h1:jq9TW8u3so/bN+JPT166wjOI6/vQPF6Xe7nMNIltagk=
+go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0/go.mod h1:p8pYQP+m5XfbZm9fxtSKAbM6oIllS7s2AfxrChvc7iw=
+go.opentelemetry.io/otel v1.24.0 h1:0LAOdjNmQeSTzGBzduGe/rU4tZhMwL5rWgtp9Ku5Jfo=
+go.opentelemetry.io/otel v1.24.0/go.mod h1:W7b9Ozg4nkF5tWI5zsXkaKKDjdVjpD4oAt9Qi/MArHo=
+go.opentelemetry.io/otel/metric v1.24.0 h1:6EhoGWWK28x1fbpA4tYTOWBkPefTDQnb8WSGXlc88kI=
+go.opentelemetry.io/otel/metric v1.24.0/go.mod h1:VYhLe1rFfxuTXLgj4CBiyz+9WYBA8pNGJgDcSFRKBco=
+go.opentelemetry.io/otel/sdk v1.24.0 h1:YMPPDNymmQN3ZgczicBY3B6sf9n62Dlj9pWD3ucgoDw=
+go.opentelemetry.io/otel/trace v1.24.0 h1:CsKnnL4dUAr/0llH9FKuc698G04IrpWV0MQA/Y1YELI=
+go.opentelemetry.io/otel/trace v1.24.0/go.mod h1:HPc3Xr/cOApsBI154IU0OI0HJexz+aw5uPdbs3UCjNU=
 go.uber.org/automaxprocs v1.5.3 h1:kWazyxZUrS3Gs4qUpbwo5kEIMGe/DAvi5Z4tl2NW4j8=
 go.uber.org/automaxprocs v1.5.3/go.mod h1:eRbA25aqJrxAbsLO0xy5jVwPt7FQnRgjW+efnwa1WM0=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
@@ -183,8 +198,8 @@ golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8U
 golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
 golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
 golang.org/x/crypto v0.1.0/go.mod h1:RecgLatLF4+eUMCP1PoPZQb+cVrJcOPbHkTkbkB9sbw=
-golang.org/x/crypto v0.17.0 h1:r8bRNjWL3GshPW3gkd+RpvzWrZAwPS49OmTGZ/uhM4k=
-golang.org/x/crypto v0.17.0/go.mod h1:gCAAfMLgwOJRpTjQ2zCCt2OcSfYMTeZVSRtQlPC7Nq4=
+golang.org/x/crypto v0.24.0 h1:mnl8DM0o513X8fdIkmyFE/5hTYxbwYOjDS/+rK6qpRI=
+golang.org/x/crypto v0.24.0/go.mod h1:Z1PMYSOR5nyMcyAVAIQSKCDwalqy85Aqn1x3Ws4L5DM=
 golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
 golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
@@ -202,18 +217,18 @@ golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwY
 golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
 golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
 golang.org/x/net v0.1.0/go.mod h1:Cx3nUiGt4eDBEyega/BKRp+/AlGL8hYe7U9odMt2Cco=
-golang.org/x/net v0.19.0 h1:zTwKpTd2XuCqf8huc7Fo2iSy+4RHPd10s4KzeTnVr1c=
-golang.org/x/net v0.19.0/go.mod h1:CfAk/cbD4CthTvqiEl8NpboMuiuOYsAr/7NOjZJtv1U=
+golang.org/x/net v0.26.0 h1:soB7SVo0PWrY4vPW/+ay0jKDNScG2X9wFeYlXIvJsOQ=
+golang.org/x/net v0.26.0/go.mod h1:5YKkiSynbBIh3p6iOc/vibscux0x38BZDkn8sCUPxHE=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
-golang.org/x/oauth2 v0.15.0 h1:s8pnnxNVzjWyrvYdFUQq5llS1PX2zhPXmccZv99h7uQ=
-golang.org/x/oauth2 v0.15.0/go.mod h1:q48ptWNTY5XWf+JNten23lcvHpLJ0ZSxF5ttTHKVCAM=
+golang.org/x/oauth2 v0.20.0 h1:4mQdhULixXKP1rwYBW0vAijoXnkTG0BLCDRzfe1idMo=
+golang.org/x/oauth2 v0.20.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI=
 golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.5.0 h1:60k92dhOjHxJkrqnwsfl8KuaHbn/5dl0lUPUklKo3qE=
-golang.org/x/sync v0.5.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
+golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M=
+golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
 golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -221,7 +236,6 @@ golang.org/x/sys v0.0.0-20191210023423-ac6580df4449/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20210616045830-e2b7044e8c71/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20211007075335-d3039528d8ac/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20220408201424-a24fb2fb8a0f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
@@ -229,20 +243,19 @@ golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBc
 golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.15.0 h1:h48lPFYpsTvQJZF4EKyI4aLHaev3CxivZmv7yZig9pc=
-golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.org/x/sys v0.21.0 h1:rF+pYz3DAGSQAxAu1CbC7catZg4ebC4UIeIhKxBZvws=
+golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
 golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
 golang.org/x/term v0.1.0/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
-golang.org/x/term v0.15.0 h1:y/Oo/a/q3IXu26lQgl04j/gjuBDOBlx7X6Om1j2CPW4=
-golang.org/x/term v0.15.0/go.mod h1:BDl952bC7+uMoWR75FIrCDx79TPU9oHkTZ9yRbYOrX0=
+golang.org/x/term v0.21.0 h1:WVXCp+/EBEHOj53Rvu+7KiT/iElMrO8ACK16SMZ3jaA=
+golang.org/x/term v0.21.0/go.mod h1:ooXLefLobQVslOqselCNF4SxFAaoS6KujMbsGzSDmX0=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
-golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ=
 golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
-golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
-golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
+golang.org/x/text v0.16.0 h1:a94ExnEXNtEwYLGJSIUxnWoxoRz/ZcCsV63ROupILh4=
+golang.org/x/text v0.16.0/go.mod h1:GhwF1Be+LQoKShO3cGOHzqOgRrGaYc9AvblQOmPVHnI=
 golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk=
 golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
 golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
@@ -257,29 +270,26 @@ golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8T
 golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 h1:+cNy6SZtPcJQH3LJVLOSmiC7MMxXNOb3PU/VUEz+EhU=
-golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028/go.mod h1:NDW/Ps6MPRej6fsCIbMTohpP40sJ/P/vI1MoTEGwX90=
-google.golang.org/api v0.149.0 h1:b2CqT6kG+zqJIVKRQ3ELJVLN1PwHZ6DJ3dW8yl82rgY=
-google.golang.org/api v0.149.0/go.mod h1:Mwn1B7JTXrzXtnvmzQE2BD6bYZQ8DShKZDZbeN9I7qI=
+google.golang.org/api v0.182.0 h1:if5fPvudRQ78GeRx3RayIoiuV7modtErPIZC/T2bIvE=
+google.golang.org/api v0.182.0/go.mod h1:cGhjy4caqA5yXRzEhkHI8Y9mfyC2VLTlER2l08xaqtM=
 google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
 google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
-google.golang.org/appengine v1.6.8 h1:IhEN5q69dyKagZPYMSdIjS2HqprW324FRQZJcGqPAsM=
-google.golang.org/appengine v1.6.8/go.mod h1:1jJ3jBArFh5pcgW8gCtRJnepW8FzD1V44FJffLiz/Ds=
 google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
 google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
 google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
-google.golang.org/genproto v0.0.0-20231016165738-49dd2c1f3d0b h1:+YaDE2r2OG8t/z5qmsh7Y+XXwCbvadxxZ0YY6mTdrVA=
-google.golang.org/genproto v0.0.0-20231016165738-49dd2c1f3d0b/go.mod h1:CgAqfJo+Xmu0GwA0411Ht3OU3OntXwsGmrmjI8ioGXI=
-google.golang.org/genproto/googleapis/api v0.0.0-20231016165738-49dd2c1f3d0b h1:CIC2YMXmIhYw6evmhPxBKJ4fmLbOFtXQN/GV3XOZR8k=
-google.golang.org/genproto/googleapis/api v0.0.0-20231016165738-49dd2c1f3d0b/go.mod h1:IBQ646DjkDkvUIsVq/cc03FUFQ9wbZu7yE396YcL870=
-google.golang.org/genproto/googleapis/rpc v0.0.0-20231016165738-49dd2c1f3d0b h1:ZlWIi1wSK56/8hn4QcBp/j9M7Gt3U/3hZw3mC7vDICo=
-google.golang.org/genproto/googleapis/rpc v0.0.0-20231016165738-49dd2c1f3d0b/go.mod h1:swOH3j0KzcDDgGUWr+SNpyTen5YrXjS3eyPzFYKc6lc=
+google.golang.org/genproto v0.0.0-20240401170217-c3f982113cda h1:wu/KJm9KJwpfHWhkkZGohVC6KRrc1oJNr4jwtQMOQXw=
+google.golang.org/genproto v0.0.0-20240401170217-c3f982113cda/go.mod h1:g2LLCvCeCSir/JJSWosk19BR4NVxGqHUC6rxIRsd7Aw=
+google.golang.org/genproto/googleapis/api v0.0.0-20240513163218-0867130af1f8 h1:W5Xj/70xIA4x60O/IFyXivR5MGqblAb8R3w26pnD6No=
+google.golang.org/genproto/googleapis/api v0.0.0-20240513163218-0867130af1f8/go.mod h1:vPrPUTsDCYxXWjP7clS81mZ6/803D8K4iM9Ma27VKas=
+google.golang.org/genproto/googleapis/rpc v0.0.0-20240521202816-d264139d666e h1:Elxv5MwEkCI9f5SkoL6afed6NTdxaGoAo39eANBwHL8=
+google.golang.org/genproto/googleapis/rpc v0.0.0-20240521202816-d264139d666e/go.mod h1:EfXuqaE1J41VCDicxHzUDm+8rk+7ZdXzHV0IhO/I6s0=
 google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
 google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
 google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
 google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
 google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
-google.golang.org/grpc v1.59.0 h1:Z5Iec2pjwb+LEOqzpB2MR12/eKFhDPhuqW91O+4bwUk=
-google.golang.org/grpc v1.59.0/go.mod h1:aUPDwccQo6OTjy7Hct4AfBPD1GptF4fyUjIkQ9YtF98=
+google.golang.org/grpc v1.64.0 h1:KH3VH9y/MgNQg1dE7b3XfVK0GsPSIzJwdF617gUSbvY=
+google.golang.org/grpc v1.64.0/go.mod h1:oxjF8E3FBnjp+/gVFYdWacaLDx9na1aqy9oovLpxQYg=
 google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
 google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
 google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
@@ -289,15 +299,12 @@ google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2
 google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
 google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
 google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
-google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
-google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
-google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8=
-google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
+google.golang.org/protobuf v1.34.1 h1:9ddQBjfCyZPOHPUiPxpYESBLc+T8P3E+Vo4IbKZgFWg=
+google.golang.org/protobuf v1.34.1/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
-gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
+gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
 gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
 gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
-gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
 gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
--- a/helpers/verify-release-binaries.sh
+++ b/helpers/verify-release-binaries.sh
@@ -89,13 +89,14 @@ extract_docker() {
    restic_platform=$3
    out=restic_${restic_version}_linux_${restic_platform}.bz2

+    # requires at least docker 25.0
    docker image pull --platform "linux/${docker_platform}" ${image}:${restic_version} > /dev/null
    docker image save ${image}:${restic_version} -o docker.tar

    mkdir img
-    tar xvf docker.tar -C img --wildcards \*/layer.tar > /dev/null
+    tar xvf docker.tar -C img --wildcards blobs/sha256/\* > /dev/null
    rm docker.tar
-    for i in img/*/layer.tar; do
+    for i in img/blobs/sha256/*; do
        tar -xvf "$i" -C img usr/bin/restic 2> /dev/null 1>&2 || true
        if [[ -f img/usr/bin/restic ]]; then
            if [[ -f restic-docker ]]; then
--- a/internal/archiver/archiver_test.go
+++ b/internal/archiver/archiver_test.go
@@ -1879,7 +1879,7 @@ func TestArchiverContextCanceled(t *testing.T) {
 	})

 	// Ensure that the archiver itself reports the canceled context and not just the backend
-	repo := repository.TestRepositoryWithBackend(t, &noCancelBackend{mem.New()}, 0)
+	repo := repository.TestRepositoryWithBackend(t, &noCancelBackend{mem.New()}, 0, repository.Options{})

 	back := restictest.Chdir(t, tempdir)
 	defer back()
--- a/internal/backend/azure/azure.go
+++ b/internal/backend/azure/azure.go
@@ -102,10 +102,20 @@ func open(cfg Config, rt http.RoundTripper) (*Backend, error) {
 			return nil, errors.Wrap(err, "NewAccountSASClientFromEndpointToken")
 		}
 	} else {
-		debug.Log(" - using DefaultAzureCredential")
-		cred, err := azidentity.NewDefaultAzureCredential(nil)
-		if err != nil {
-			return nil, errors.Wrap(err, "NewDefaultAzureCredential")
+		var cred azcore.TokenCredential
+
+		if cfg.ForceCliCredential {
+			debug.Log(" - using AzureCLICredential")
+			cred, err = azidentity.NewAzureCLICredential(nil)
+			if err != nil {
+				return nil, errors.Wrap(err, "NewAzureCLICredential")
+			}
+		} else {
+			debug.Log(" - using DefaultAzureCredential")
+			cred, err = azidentity.NewDefaultAzureCredential(nil)
+			if err != nil {
+				return nil, errors.Wrap(err, "NewDefaultAzureCredential")
+			}
 		}

 		client, err = azContainer.NewClient(url, cred, opts)
--- a/internal/backend/azure/config.go
+++ b/internal/backend/azure/config.go
@@ -3,6 +3,7 @@ package azure
 import (
 	"os"
 	"path"
+	"strconv"
 	"strings"

 	"github.com/restic/restic/internal/errors"
@@ -13,12 +14,13 @@ import (
 // Config contains all configuration necessary to connect to an azure compatible
 // server.
 type Config struct {
-	AccountName    string
-	AccountSAS     options.SecretString
-	AccountKey     options.SecretString
-	EndpointSuffix string
-	Container      string
-	Prefix         string
+	AccountName        string
+	AccountSAS         options.SecretString
+	AccountKey         options.SecretString
+	ForceCliCredential bool
+	EndpointSuffix     string
+	Container          string
+	Prefix             string

 	Connections uint `option:"connections" help:"set a limit for the number of concurrent connections (default: 5)"`
 }
@@ -73,6 +75,11 @@ func (cfg *Config) ApplyEnvironment(prefix string) {
 		cfg.AccountSAS = options.NewSecretString(os.Getenv(prefix + "AZURE_ACCOUNT_SAS"))
 	}

+	var forceCliCred, err = strconv.ParseBool(os.Getenv(prefix + "AZURE_FORCE_CLI_CREDENTIAL"))
+	if err == nil {
+		cfg.ForceCliCredential = forceCliCred
+	}
+
 	if cfg.EndpointSuffix == "" {
 		cfg.EndpointSuffix = os.Getenv(prefix + "AZURE_ENDPOINT_SUFFIX")
 	}
--- a/internal/migrations/upgrade_repo_v2_test.go
+++ b/internal/migrations/upgrade_repo_v2_test.go
@@ -69,7 +69,7 @@ func TestUpgradeRepoV2Failure(t *testing.T) {
 		Backend:                   be,
 	}

-	repo := repository.TestRepositoryWithBackend(t, be, 1)
+	repo := repository.TestRepositoryWithBackend(t, be, 1, repository.Options{})
 	if repo.Config().Version != 1 {
 		t.Fatal("test repo has wrong version")
 	}
--- a/internal/pack/pack.go
+++ b/internal/pack/pack.go
@@ -1,6 +1,7 @@
 package pack

 import (
+	"bytes"
 	"context"
 	"encoding/binary"
 	"fmt"
@@ -74,7 +75,7 @@ func (p *Packer) Finalize() error {
 	p.m.Lock()
 	defer p.m.Unlock()

-	header, err := p.makeHeader()
+	header, err := makeHeader(p.blobs)
 	if err != nil {
 		return err
 	}
@@ -83,6 +84,12 @@ func (p *Packer) Finalize() error {
 	nonce := crypto.NewRandomNonce()
 	encryptedHeader = append(encryptedHeader, nonce...)
 	encryptedHeader = p.k.Seal(encryptedHeader, nonce, header, nil)
+	encryptedHeader = binary.LittleEndian.AppendUint32(encryptedHeader, uint32(len(encryptedHeader)))
+
+	if err := verifyHeader(p.k, encryptedHeader, p.blobs); err != nil {
+		//nolint:revive // ignore linter warnings about error message spelling
+		return fmt.Errorf("Detected data corruption while writing pack-file header: %w\nCorrupted data is either caused by hardware issues or software bugs. Please open an issue at https://github.com/restic/restic/issues/new/choose for further troubleshooting.", err)
+	}

 	// append the header
 	n, err := p.wr.Write(encryptedHeader)
@@ -90,18 +97,33 @@ func (p *Packer) Finalize() error {
 		return errors.Wrap(err, "Write")
 	}

-	hdrBytes := len(encryptedHeader)
-	if n != hdrBytes {
+	if n != len(encryptedHeader) {
 		return errors.New("wrong number of bytes written")
 	}
+	p.bytes += uint(len(encryptedHeader))

-	// write length
-	err = binary.Write(p.wr, binary.LittleEndian, uint32(hdrBytes))
+	return nil
+}
+
+func verifyHeader(k *crypto.Key, header []byte, expected []restic.Blob) error {
+	// do not offer a way to skip the pack header verification, as pack headers are usually small enough
+	// to not result in a significant performance impact
+
+	decoded, hdrSize, err := List(k, bytes.NewReader(header), int64(len(header)))
 	if err != nil {
-		return errors.Wrap(err, "binary.Write")
+		return fmt.Errorf("header decoding failed: %w", err)
+	}
+	if hdrSize != uint32(len(header)) {
+		return fmt.Errorf("unexpected header size %v instead of %v", hdrSize, len(header))
+	}
+	if len(decoded) != len(expected) {
+		return fmt.Errorf("pack header size mismatch")
+	}
+	for i := 0; i < len(decoded); i++ {
+		if decoded[i] != expected[i] {
+			return fmt.Errorf("pack header entry mismatch got %v instead of %v", decoded[i], expected[i])
+		}
 	}
-	p.bytes += uint(hdrBytes + binary.Size(uint32(0)))
-
 	return nil
 }

@@ -111,10 +133,10 @@ func (p *Packer) HeaderOverhead() int {
 }

 // makeHeader constructs the header for p.
-func (p *Packer) makeHeader() ([]byte, error) {
-	buf := make([]byte, 0, len(p.blobs)*int(entrySize))
+func makeHeader(blobs []restic.Blob) ([]byte, error) {
+	buf := make([]byte, 0, len(blobs)*int(entrySize))

-	for _, b := range p.blobs {
+	for _, b := range blobs {
 		switch {
 		case b.Type == restic.DataBlob && b.UncompressedLength == 0:
 			buf = append(buf, 0)
--- a/internal/pack/pack_internal_test.go
+++ b/internal/pack/pack_internal_test.go
@@ -4,6 +4,7 @@ import (
 	"bytes"
 	"encoding/binary"
 	"io"
+	"strings"
 	"testing"

 	"github.com/restic/restic/internal/crypto"
@@ -177,3 +178,60 @@ func TestReadRecords(t *testing.T) {
 		}
 	}
 }
+
+func TestUnpackedVerification(t *testing.T) {
+	// create random keys
+	k := crypto.NewRandomKey()
+	blobs := []restic.Blob{
+		{
+			BlobHandle:         restic.NewRandomBlobHandle(),
+			Length:             42,
+			Offset:             0,
+			UncompressedLength: 2 * 42,
+		},
+	}
+
+	type DamageType string
+	const (
+		damageData       DamageType = "data"
+		damageCiphertext DamageType = "ciphertext"
+		damageLength     DamageType = "length"
+	)
+
+	for _, test := range []struct {
+		damage DamageType
+		msg    string
+	}{
+		{"", ""},
+		{damageData, "pack header entry mismatch"},
+		{damageCiphertext, "ciphertext verification failed"},
+		{damageLength, "header decoding failed"},
+	} {
+		header, err := makeHeader(blobs)
+		rtest.OK(t, err)
+
+		if test.damage == damageData {
+			header[8] ^= 0x42
+		}
+
+		encryptedHeader := make([]byte, 0, crypto.CiphertextLength(len(header)))
+		nonce := crypto.NewRandomNonce()
+		encryptedHeader = append(encryptedHeader, nonce...)
+		encryptedHeader = k.Seal(encryptedHeader, nonce, header, nil)
+		encryptedHeader = binary.LittleEndian.AppendUint32(encryptedHeader, uint32(len(encryptedHeader)))
+
+		if test.damage == damageCiphertext {
+			encryptedHeader[8] ^= 0x42
+		}
+		if test.damage == damageLength {
+			encryptedHeader[len(encryptedHeader)-1] ^= 0x42
+		}
+
+		err = verifyHeader(k, encryptedHeader, blobs)
+		if test.msg == "" {
+			rtest.Assert(t, err == nil, "expected no error, got %v", err)
+		} else {
+			rtest.Assert(t, strings.Contains(err.Error(), test.msg), "expected error to contain %q, got %q", test.msg, err)
+		}
+	}
+}
--- a/internal/repository/fuzz_test.go
+++ b/internal/repository/fuzz_test.go
@@ -4,7 +4,6 @@ import (
 	"context"
 	"testing"

-	"github.com/restic/restic/internal/backend/mem"
 	"github.com/restic/restic/internal/restic"
 	"golang.org/x/sync/errgroup"
 )
@@ -19,7 +18,7 @@ func FuzzSaveLoadBlob(f *testing.F) {
 		}

 		id := restic.Hash(blob)
-		repo := TestRepositoryWithBackend(t, mem.New(), 2)
+		repo := TestRepositoryWithVersion(t, 2)

 		var wg errgroup.Group
 		repo.StartPackUploader(context.TODO(), &wg)
--- a/internal/repository/repack_test.go
+++ b/internal/repository/repack_test.go
@@ -346,7 +346,8 @@ func TestRepackWrongBlob(t *testing.T) {
 }

 func testRepackWrongBlob(t *testing.T, version uint) {
-	repo := repository.TestRepositoryWithVersion(t, version)
+	// disable verification to allow adding corrupted blobs to the repository
+	repo := repository.TestRepositoryWithBackend(t, nil, version, repository.Options{NoExtraVerify: true})

 	seed := time.Now().UnixNano()
 	rand.Seed(seed)
@@ -371,7 +372,8 @@ func TestRepackBlobFallback(t *testing.T) {
 }

 func testRepackBlobFallback(t *testing.T, version uint) {
-	repo := repository.TestRepositoryWithVersion(t, version)
+	// disable verification to allow adding corrupted blobs to the repository
+	repo := repository.TestRepositoryWithBackend(t, nil, version, repository.Options{NoExtraVerify: true})

 	seed := time.Now().UnixNano()
 	rand.Seed(seed)
--- a/internal/repository/repository.go
+++ b/internal/repository/repository.go
@@ -59,8 +59,9 @@ type Repository struct {
 }

 type Options struct {
-	Compression CompressionMode
-	PackSize    uint
+	Compression   CompressionMode
+	PackSize      uint
+	NoExtraVerify bool
 }

 // CompressionMode configures if data should be compressed.
@@ -423,6 +424,11 @@ func (r *Repository) saveAndEncrypt(ctx context.Context, t restic.BlobType, data
 	// encrypt blob
 	ciphertext = r.key.Seal(ciphertext, nonce, data, nil)

+	if err := r.verifyCiphertext(ciphertext, uncompressedLength, id); err != nil {
+		//nolint:revive // ignore linter warnings about error message spelling
+		return 0, fmt.Errorf("Detected data corruption while saving blob %v: %w\nCorrupted blobs are either caused by hardware issues or software bugs. Please open an issue at https://github.com/restic/restic/issues/new/choose for further troubleshooting.", id, err)
+	}
+
 	// find suitable packer and add blob
 	var pm *packerManager

@@ -438,6 +444,31 @@ func (r *Repository) saveAndEncrypt(ctx context.Context, t restic.BlobType, data
 	return pm.SaveBlob(ctx, t, id, ciphertext, uncompressedLength)
 }

+func (r *Repository) verifyCiphertext(buf []byte, uncompressedLength int, id restic.ID) error {
+	if r.opts.NoExtraVerify {
+		return nil
+	}
+
+	nonce, ciphertext := buf[:r.key.NonceSize()], buf[r.key.NonceSize():]
+	plaintext, err := r.key.Open(nil, nonce, ciphertext, nil)
+	if err != nil {
+		return fmt.Errorf("decryption failed: %w", err)
+	}
+	if uncompressedLength != 0 {
+		// DecodeAll will allocate a slice if it is not large enough since it
+		// knows the decompressed size (because we're using EncodeAll)
+		plaintext, err = r.getZstdDecoder().DecodeAll(plaintext, nil)
+		if err != nil {
+			return fmt.Errorf("decompression failed: %w", err)
+		}
+	}
+	if !restic.Hash(plaintext).Equal(id) {
+		return errors.New("hash mismatch")
+	}
+
+	return nil
+}
+
 func (r *Repository) compressUnpacked(p []byte) ([]byte, error) {
 	// compression is only available starting from version 2
 	if r.cfg.Version < 2 {
@@ -474,7 +505,8 @@ func (r *Repository) decompressUnpacked(p []byte) ([]byte, error) {

 // SaveUnpacked encrypts data and stores it in the backend. Returned is the
 // storage hash.
-func (r *Repository) SaveUnpacked(ctx context.Context, t restic.FileType, p []byte) (id restic.ID, err error) {
+func (r *Repository) SaveUnpacked(ctx context.Context, t restic.FileType, buf []byte) (id restic.ID, err error) {
+	p := buf
 	if t != restic.ConfigFile {
 		p, err = r.compressUnpacked(p)
 		if err != nil {
@@ -489,6 +521,11 @@ func (r *Repository) SaveUnpacked(ctx context.Context, t restic.FileType, p []by

 	ciphertext = r.key.Seal(ciphertext, nonce, p, nil)

+	if err := r.verifyUnpacked(ciphertext, t, buf); err != nil {
+		//nolint:revive // ignore linter warnings about error message spelling
+		return restic.ID{}, fmt.Errorf("Detected data corruption while saving file of type %v: %w\nCorrupted data is either caused by hardware issues or software bugs. Please open an issue at https://github.com/restic/restic/issues/new/choose for further troubleshooting.", t, err)
+	}
+
 	if t == restic.ConfigFile {
 		id = restic.ID{}
 	} else {
@@ -506,6 +543,29 @@ func (r *Repository) SaveUnpacked(ctx context.Context, t restic.FileType, p []by
 	return id, nil
 }

+func (r *Repository) verifyUnpacked(buf []byte, t restic.FileType, expected []byte) error {
+	if r.opts.NoExtraVerify {
+		return nil
+	}
+
+	nonce, ciphertext := buf[:r.key.NonceSize()], buf[r.key.NonceSize():]
+	plaintext, err := r.key.Open(nil, nonce, ciphertext, nil)
+	if err != nil {
+		return fmt.Errorf("decryption failed: %w", err)
+	}
+	if t != restic.ConfigFile {
+		plaintext, err = r.decompressUnpacked(plaintext)
+		if err != nil {
+			return fmt.Errorf("decompression failed: %w", err)
+		}
+	}
+
+	if !bytes.Equal(plaintext, expected) {
+		return errors.New("data mismatch")
+	}
+	return nil
+}
+
 // Flush saves all remaining packs and the index
 func (r *Repository) Flush(ctx context.Context) error {
 	if err := r.flushPacks(ctx); err != nil {
--- a/internal/repository/repository_internal_test.go
+++ b/internal/repository/repository_internal_test.go
@@ -3,8 +3,10 @@ package repository
 import (
 	"math/rand"
 	"sort"
+	"strings"
 	"testing"

+	"github.com/restic/restic/internal/crypto"
 	"github.com/restic/restic/internal/restic"
 	rtest "github.com/restic/restic/internal/test"
 )
@@ -72,3 +74,101 @@ func BenchmarkSortCachedPacksFirst(b *testing.B) {
 		sortCachedPacksFirst(cache, cpy[:])
 	}
 }
+
+func TestBlobVerification(t *testing.T) {
+	repo := TestRepository(t).(*Repository)
+
+	type DamageType string
+	const (
+		damageData       DamageType = "data"
+		damageCompressed DamageType = "compressed"
+		damageCiphertext DamageType = "ciphertext"
+	)
+
+	for _, test := range []struct {
+		damage DamageType
+		msg    string
+	}{
+		{"", ""},
+		{damageData, "hash mismatch"},
+		{damageCompressed, "decompression failed"},
+		{damageCiphertext, "ciphertext verification failed"},
+	} {
+		plaintext := rtest.Random(800, 1234)
+		id := restic.Hash(plaintext)
+		if test.damage == damageData {
+			plaintext[42] ^= 0x42
+		}
+
+		uncompressedLength := uint(len(plaintext))
+		plaintext = repo.getZstdEncoder().EncodeAll(plaintext, nil)
+
+		if test.damage == damageCompressed {
+			plaintext = plaintext[:len(plaintext)-8]
+		}
+
+		nonce := crypto.NewRandomNonce()
+		ciphertext := append([]byte{}, nonce...)
+		ciphertext = repo.Key().Seal(ciphertext, nonce, plaintext, nil)
+
+		if test.damage == damageCiphertext {
+			ciphertext[42] ^= 0x42
+		}
+
+		err := repo.verifyCiphertext(ciphertext, int(uncompressedLength), id)
+		if test.msg == "" {
+			rtest.Assert(t, err == nil, "expected no error, got %v", err)
+		} else {
+			rtest.Assert(t, strings.Contains(err.Error(), test.msg), "expected error to contain %q, got %q", test.msg, err)
+		}
+	}
+}
+
+func TestUnpackedVerification(t *testing.T) {
+	repo := TestRepository(t).(*Repository)
+
+	type DamageType string
+	const (
+		damageData       DamageType = "data"
+		damageCompressed DamageType = "compressed"
+		damageCiphertext DamageType = "ciphertext"
+	)
+
+	for _, test := range []struct {
+		damage DamageType
+		msg    string
+	}{
+		{"", ""},
+		{damageData, "data mismatch"},
+		{damageCompressed, "decompression failed"},
+		{damageCiphertext, "ciphertext verification failed"},
+	} {
+		plaintext := rtest.Random(800, 1234)
+		orig := append([]byte{}, plaintext...)
+		if test.damage == damageData {
+			plaintext[42] ^= 0x42
+		}
+
+		compressed := []byte{2}
+		compressed = repo.getZstdEncoder().EncodeAll(plaintext, compressed)
+
+		if test.damage == damageCompressed {
+			compressed = compressed[:len(compressed)-8]
+		}
+
+		nonce := crypto.NewRandomNonce()
+		ciphertext := append([]byte{}, nonce...)
+		ciphertext = repo.Key().Seal(ciphertext, nonce, compressed, nil)
+
+		if test.damage == damageCiphertext {
+			ciphertext[42] ^= 0x42
+		}
+
+		err := repo.verifyUnpacked(ciphertext, restic.IndexFile, orig)
+		if test.msg == "" {
+			rtest.Assert(t, err == nil, "expected no error, got %v", err)
+		} else {
+			rtest.Assert(t, strings.Contains(err.Error(), test.msg), "expected error to contain %q, got %q", test.msg, err)
+		}
+	}
+}
--- a/internal/repository/testing.go
+++ b/internal/repository/testing.go
@@ -43,7 +43,7 @@ const TestChunkerPol = chunker.Pol(0x3DA3358B4DC173)
 // TestRepositoryWithBackend returns a repository initialized with a test
 // password. If be is nil, an in-memory backend is used. A constant polynomial
 // is used for the chunker and low-security test parameters.
-func TestRepositoryWithBackend(t testing.TB, be restic.Backend, version uint) restic.Repository {
+func TestRepositoryWithBackend(t testing.TB, be restic.Backend, version uint, opts Options) restic.Repository {
 	t.Helper()
 	TestUseLowSecurityKDFParameters(t)
 	restic.TestDisableCheckPolynomial(t)
@@ -52,7 +52,7 @@ func TestRepositoryWithBackend(t testing.TB, be restic.Backend, version uint) re
 		be = TestBackend(t)
 	}

-	repo, err := New(be, Options{})
+	repo, err := New(be, opts)
 	if err != nil {
 		t.Fatalf("TestRepository(): new repo failed: %v", err)
 	}
@@ -78,6 +78,7 @@ func TestRepository(t testing.TB) restic.Repository {
 func TestRepositoryWithVersion(t testing.TB, version uint) restic.Repository {
 	t.Helper()
 	dir := os.Getenv("RESTIC_TEST_REPO")
+	opts := Options{}
 	if dir != "" {
 		_, err := os.Stat(dir)
 		if err != nil {
@@ -85,7 +86,7 @@ func TestRepositoryWithVersion(t testing.TB, version uint) restic.Repository {
 			if err != nil {
 				t.Fatalf("error creating local backend at %v: %v", dir, err)
 			}
-			return TestRepositoryWithBackend(t, be, version)
+			return TestRepositoryWithBackend(t, be, version, opts)
 		}

 		if err == nil {
@@ -93,7 +94,7 @@ func TestRepositoryWithVersion(t testing.TB, version uint) restic.Repository {
 		}
 	}

-	return TestRepositoryWithBackend(t, nil, version)
+	return TestRepositoryWithBackend(t, nil, version, opts)
 }

 // TestOpenLocal opens a local repository.
--- a/internal/restic/lock_test.go
+++ b/internal/restic/lock_test.go
@@ -65,7 +65,7 @@ func (be *failLockLoadingBackend) Load(ctx context.Context, h restic.Handle, len

 func TestMultipleLockFailure(t *testing.T) {
 	be := &failLockLoadingBackend{Backend: mem.New()}
-	repo := repository.TestRepositoryWithBackend(t, be, 0)
+	repo := repository.TestRepositoryWithBackend(t, be, 0, repository.Options{})
 	restic.TestSetLockTimeout(t, 5*time.Millisecond)

 	lock1, err := restic.NewLock(context.TODO(), repo)
Author	SHA1	Message	Date
Alexander Neumann	fe9f142b52	Add version for 0.16.5	2024-07-01 21:25:34 +02:00
Alexander Neumann	6ae760751a	Generate CHANGELOG.md for 0.16.5	2024-07-01 21:25:33 +02:00
Alexander Neumann	2fa1b42706	Prepare changelog for 0.16.5	2024-07-01 21:25:33 +02:00
Michael Eischer	ca04a88e65	Merge pull request #4879 from restic/backport-azure-cli-option Backport azure cli option	2024-06-26 21:07:16 +02:00
Michael Eischer	12e858b7af	azure: deduplicate cli and default credentials case	2024-06-26 20:59:51 +02:00
Maik Riechert	834f08fe2d	Azure: add option to force use of CLI credential	2024-06-26 20:59:51 +02:00
Michael Eischer	814ef4901f	Merge pull request #4878 from restic/patch-deps-bump Bump dependencies	2024-06-26 20:58:29 +02:00
Michael Eischer	84bc9432de	update release verification script for latest docker	2024-06-26 20:49:23 +02:00
Michael Eischer	e9d711422a	bump azure, golang and gcs dependencies	2024-06-26 20:49:23 +02:00
Alexander Neumann	0f9fa44de5	Set development version for 0.16.4	2024-02-04 19:50:56 +01:00
Alexander Neumann	3786536dc1	Add version for 0.16.4	2024-02-04 19:50:52 +01:00
Alexander Neumann	811be5984d	Update manpages and auto-completion	2024-02-04 19:50:51 +01:00
Alexander Neumann	b0ead75de5	Generate CHANGELOG.md for 0.16.4	2024-02-04 19:50:34 +01:00
Alexander Neumann	6cd2804bff	Prepare changelog for 0.16.4	2024-02-04 19:50:34 +01:00
Michael Eischer	a72c2b74f3	Apply changelog entry / documentation improvements from review	2024-02-04 19:10:06 +01:00
Michael Eischer	261b1455c7	add documentation for --no-extra-verify option	2024-02-04 19:10:06 +01:00
Michael Eischer	2a0bd2b637	rename `--no-verify-pack` to `--no-extra-verify`	2024-02-04 19:10:05 +01:00
Michael Eischer	4589da7eb9	add data verification changelog entry	2024-02-04 19:09:49 +01:00
Michael Eischer	75e72d826c	pack: verify integrity of pack file header	2024-02-04 19:09:49 +01:00
Michael Eischer	d8916bc3d9	repository: ask users to report corrupted data while saving blobs	2024-02-04 19:09:49 +01:00
Michael Eischer	dc11d012bb	Make --no-verify-pack globally available Verifying all blobs before upload comes with a notable performance impact. Allow users to skip it if necessary.	2024-02-04 19:09:49 +01:00
Michael Eischer	8ef5425351	repository: test verification of blobs/unpacked data	2024-02-04 19:09:46 +01:00
Michael Eischer	885431ec2b	repository: Allow skipping verification for tests Some tests have to explicitly create pack files with blobs that don't match their ID. For those blobs the builtin verification of the repository must be disabled.	2024-02-04 19:08:30 +01:00
Michael Eischer	cb85fb46dd	backup: verify unpacked files before upload	2024-02-04 19:07:48 +01:00
Michael Eischer	2f30c940b2	backup: verify blobs before upload This only covers the blobs themselves, the pack header is not verified so far. Unpacked files are also not covered by the integrity check.	2024-02-04 19:07:48 +01:00
Michael Eischer	0ea62b5ac6	repository: make repo.Options configurable for test repos	2024-02-04 19:07:46 +01:00
Michael Eischer	29e1caf825	add changelog draft for data corruption on max compression	2024-02-04 19:05:51 +01:00
Michael Eischer	0164f5310d	Downgrade klauspost/compress to fix data corruption at max. compression	2024-02-04 19:05:50 +01:00
Alexander Neumann	0ec9383ba2	Set development version for 0.16.3	2024-01-14 20:21:45 +01:00
@@ -1 +1 @@
 .16.3
 .16.5