Compare commits

..

6 Commits

Author SHA1 Message Date
Alexander Neumann
594f155eb6 Add version for 0.13.1 2022-04-10 10:58:55 +02:00
Alexander Neumann
90f1a9b5f5 Generate CHANGELOG.md for 0.13.1 2022-04-10 10:58:22 +02:00
Alexander Neumann
2ad3d50535 Prepare changelog for 0.13.1 2022-04-10 10:58:22 +02:00
Charlie Jiang
59fd21e30e Fix rclone (scoop shim) and sftp issue due to detached console on Windows 2022-04-10 10:56:55 +02:00
greatroar
c31f1e797b Cast unix.Statfs_t.Type to int64 when checking for btrfs
Fixes #3687. Uses the cast suggested by @MichaelEischer, except that the
contant isn't cast along, because it's untyped and will be converted by
the compiler as necessary.
2022-04-10 10:56:37 +02:00
Alexander Neumann
53ac0bfe85 Fix diff
Nodes in trees were always printed with a `+` in diff, regardless of
whether or not a dir was added or removed. Let's use the mode we were
passed in printDir().

Closes #3685
2022-04-10 10:56:19 +02:00
911 changed files with 30612 additions and 67918 deletions

12
.dockerignore Normal file
View File

@@ -0,0 +1,12 @@
# Folders
.git/
.github/
changelog/
doc/
docker/
helpers/
# Files
.gitignore
.golangci.yml
*.md

2
.gitattributes vendored
View File

@@ -1,2 +0,0 @@
# Workaround for https://github.com/golang/go/issues/52268.
**/testdata/fuzz/*/* eol=lf

View File

@@ -32,30 +32,23 @@ Output of `restic version`
--------------------------
What backend/service did you use to store the repository?
---------------------------------------------------------
Problem description / Steps to reproduce
----------------------------------------
How did you run restic exactly?
-------------------------------
<!--
This section should include at least:
* A description of the problem you are having with restic.
* The complete command line and any environment variables you used to
configure restic's backend access. Make sure to replace sensitive values!
* The output of the commands, what restic prints gives may give us much
information to diagnose the problem!
* The more time you spend describing an easy way to reproduce the behavior (if
this is possible), the easier it is for the project developers to fix it!
-->
What backend/server/service did you use to store the repository?
----------------------------------------------------------------
Expected behavior
-----------------
@@ -72,12 +65,22 @@ In this section, please try to concentrate on observations, so only describe
what you observed directly.
-->
Steps to reproduce the behavior
-------------------------------
<!--
The more time you spend describing an easy way to reproduce the behavior (if
this is possible), the easier it is for the project developers to fix it!
-->
Do you have any idea what may have caused this?
-----------------------------------------------
<!--
Did something noteworthy happen on your system, Internet connection, backend services, etc?
-->
Do you have an idea how to solve the issue?
-------------------------------------------
Did restic help you today? Did it make you happy in any way?

View File

@@ -28,15 +28,13 @@ Checklist
You do not need to check all the boxes below all at once. Feel free to take
your time and add more commits. If you're done and ready for review, please
check the last box. Enable a checkbox by replacing [ ] with [x].
Please always follow these steps:
- Read the [contribution guidelines](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#providing-patches).
- Enable [maintainer edits](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/allowing-changes-to-a-pull-request-branch-created-from-a-fork).
- Run `gofmt` on the code in all commits.
- Format all commit messages in the same style as [the other commits in the repository](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#git-commits).
-->
- [ ] I have read the [contribution guidelines](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#providing-patches).
- [ ] I have [enabled maintainer edits](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/allowing-changes-to-a-pull-request-branch-created-from-a-fork).
- [ ] I have added tests for all code changes.
- [ ] I have added documentation for relevant changes (in the manual).
- [ ] There's a new file in `changelog/unreleased/` that describes the changes for our users (see [template](https://github.com/restic/restic/blob/master/changelog/TEMPLATE)).
- [ ] I have run `gofmt` on the code in all commits.
- [ ] All commit messages are formatted in the same style as [the other commits in the repo](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#git-commits).
- [ ] I'm done! This pull request is ready for review.

View File

@@ -1,13 +0,0 @@
version: 2
updates:
# Dependencies listed in go.mod
- package-ecosystem: "gomod"
directory: "/" # Location of package manifests
schedule:
interval: "monthly"
# Dependencies listed in .github/workflows/*.yml
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "monthly"

View File

@@ -1,95 +0,0 @@
name: Create and publish a Docker image
on:
push:
tags:
- 'v*'
branches:
- 'master'
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
build-and-push-image:
if: github.repository == 'restic/restic'
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
outputs:
image: ${{ steps.image.outputs.image }}
digest: ${{ steps.build-and-push.outputs.digest }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Log in to the Container registry
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=sha
type=ref,event=branch
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Set up QEMU
uses: docker/setup-qemu-action@68827325e0b33c7199eb31dd4e31fbe9023e06e3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@2b51285047da1547ffb1b2203d8be4c0af6b1f20
- name: Ensure consistent binaries
run: |
echo "removing git directory for consistency with release binaries"
rm -rf .git
# remove VCS information from release builds, keep VCS for nightly builds on master
if: github.ref != 'refs/heads/master'
- name: Build and push Docker image
id: build-and-push
uses: docker/build-push-action@15560696de535e4014efeff63c48f16952e52dd1
with:
push: true
context: .
file: docker/Dockerfile.release
platforms: linux/386,linux/amd64,linux/arm,linux/arm64
pull: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
- name: Output image
id: image
run: |
# NOTE: Set the image as an output because the `env` context is not
# available to the inputs of a reusable workflow call.
image_name="${REGISTRY}/${IMAGE_NAME}"
echo "image=$image_name" >> "$GITHUB_OUTPUT"
provenance:
needs: [build-and-push-image]
permissions:
actions: read # for detecting the Github Actions environment.
id-token: write # for creating OIDC tokens for signing.
packages: write # for uploading attestations.
if: github.repository == 'restic/restic'
uses: slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@v2.1.0
with:
image: ${{ needs.build-and-push-image.outputs.image }}
digest: ${{ needs.build-and-push-image.outputs.digest }}
registry-username: ${{ github.actor }}
secrets:
registry-password: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -7,13 +7,9 @@ on:
# run tests for all pull requests
pull_request:
merge_group:
permissions:
contents: read
env:
latest_go: "1.24.x"
latest_go: "1.18.x"
GO111MODULE: on
jobs:
@@ -23,31 +19,47 @@ jobs:
# list of jobs to run:
include:
- job_name: Windows
go: 1.24.x
go: 1.18.x
os: windows-latest
install_verb: install
- job_name: macOS
go: 1.24.x
go: 1.18.x
os: macOS-latest
test_fuse: false
install_verb: install
- job_name: Linux
go: 1.24.x
go: 1.18.x
os: ubuntu-latest
test_cloud_backends: true
test_fuse: true
check_changelog: true
- job_name: Linux (race)
go: 1.24.x
os: ubuntu-latest
test_fuse: true
test_opts: "-race"
install_verb: install
- job_name: Linux
go: 1.23.x
go: 1.17.x
os: ubuntu-latest
test_fuse: true
install_verb: install
- job_name: Linux
go: 1.16.x
os: ubuntu-latest
test_fuse: true
install_verb: get
- job_name: Linux
go: 1.15.x
os: ubuntu-latest
test_fuse: true
install_verb: get
- job_name: Linux
go: 1.14.x
os: ubuntu-latest
test_fuse: true
install_verb: get
name: ${{ matrix.job_name }} Go ${{ matrix.go }}
runs-on: ${{ matrix.os }}
@@ -56,18 +68,15 @@ jobs:
GOPROXY: https://proxy.golang.org
steps:
- name: Check out code
uses: actions/checkout@v4
- name: Set up Go ${{ matrix.go }}
uses: actions/setup-go@v5
uses: actions/setup-go@v2
with:
go-version: ${{ matrix.go }}
- name: Get programs (Linux/macOS)
run: |
echo "build Go tools"
go install github.com/restic/rest-server/cmd/rest-server@master
go ${{ matrix.install_verb }} github.com/restic/rest-server/cmd/rest-server@latest
echo "install minio server"
mkdir $HOME/bin
@@ -89,7 +98,7 @@ jobs:
chmod 755 $HOME/bin/rclone
rm -rf rclone*
# add $HOME/bin to path ($GOBIN was already added to the path by setup-go@v3)
# add $HOME/bin to path ($GOBIN was already added to the path by setup-go@v2)
echo $HOME/bin >> $GITHUB_PATH
if: matrix.os == 'ubuntu-latest' || matrix.os == 'macOS-latest'
@@ -99,7 +108,7 @@ jobs:
$ProgressPreference = 'SilentlyContinue'
echo "build Go tools"
go install github.com/restic/rest-server/cmd/rest-server@master
go ${{ matrix.install_verb }} github.com/restic/rest-server/...
echo "install minio server"
mkdir $Env:USERPROFILE/bin
@@ -111,7 +120,7 @@ jobs:
unzip rclone.zip
copy rclone*/rclone.exe $Env:USERPROFILE/bin
# add $USERPROFILE/bin to path ($GOBIN was already added to the path by setup-go@v3)
# add $USERPROFILE/bin to path ($GOBIN was already added to the path by setup-go@v2)
echo $Env:USERPROFILE\bin >> $Env:GITHUB_PATH
echo "install tar"
@@ -132,23 +141,18 @@ jobs:
echo $Env:USERPROFILE\tar\bin >> $Env:GITHUB_PATH
if: matrix.os == 'windows-latest'
- name: Check out code
uses: actions/checkout@v2
- name: Build with build.go
run: |
go run build.go
- name: Minimal test
run: |
./restic init
./restic backup .
env:
RESTIC_REPOSITORY: ../testrepo
RESTIC_PASSWORD: password
- name: Run local Tests
env:
RESTIC_TEST_FUSE: ${{ matrix.test_fuse }}
run: |
go test -cover ${{matrix.test_opts}} ./...
go test -cover ./...
- name: Test cloud backends
env:
@@ -185,13 +189,11 @@ jobs:
# prepare credentials for Google Cloud Storage tests in a temp file
export GOOGLE_APPLICATION_CREDENTIALS=$(mktemp --tmpdir restic-gcs-auth-XXXXXXX)
echo $RESTIC_TEST_GS_APPLICATION_CREDENTIALS_B64 | base64 -d > $GOOGLE_APPLICATION_CREDENTIALS
go test -cover -parallel 5 -timeout 15m ./internal/backend/...
go test -cover -parallel 4 ./internal/backend/...
# only run cloud backend tests for pull requests from and pushes to our
# own repo, otherwise the secrets are not available
# Skip for Dependabot pull requests as these are run without secrets
# https://docs.github.com/en/code-security/dependabot/working-with-dependabot/automating-dependabot-with-github-actions#responding-to-events
if: ((github.repository == 'restic/restic' && github.event_name == 'push') || github.event.pull_request.head.repo.full_name == github.repository) && (github.actor != 'dependabot[bot]') && matrix.test_cloud_backends
if: (github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository) && matrix.test_cloud_backends
- name: Check changelog files with calens
run: |
@@ -204,57 +206,71 @@ jobs:
cross_compile:
strategy:
# ATTENTION: the list of architectures must be in sync with helpers/build-release-binaries/main.go!
matrix:
# run cross-compile in three batches parallel so the overall tests run faster
subset:
- "0/3"
- "1/3"
- "2/3"
# run cross-compile in two batches parallel so the overall tests run faster
targets:
- "linux/386 linux/amd64 linux/arm linux/arm64 linux/ppc64le linux/mips linux/mipsle linux/mips64 linux/mips64le linux/s390x \
openbsd/386 openbsd/amd64"
- "freebsd/386 freebsd/amd64 freebsd/arm \
aix/ppc64 \
darwin/amd64 darwin/arm64 \
netbsd/386 netbsd/amd64 \
windows/386 windows/amd64 \
solaris/amd64"
env:
GOPROXY: https://proxy.golang.org
runs-on: ubuntu-latest
name: Cross Compile for subset ${{ matrix.subset }}
name: Cross Compile for ${{ matrix.targets }}
steps:
- name: Check out code
uses: actions/checkout@v4
- name: Set up Go ${{ env.latest_go }}
uses: actions/setup-go@v5
uses: actions/setup-go@v2
with:
go-version: ${{ env.latest_go }}
- name: Cross-compile for subset ${{ matrix.subset }}
- name: Install gox
run: |
mkdir build-output build-output-debug
go run ./helpers/build-release-binaries/main.go -o build-output -s . --platform-subset ${{ matrix.subset }}
go run ./helpers/build-release-binaries/main.go -o build-output-debug -s . --platform-subset ${{ matrix.subset }} --tags debug
go install github.com/mitchellh/gox@latest
- name: Check out code
uses: actions/checkout@v2
- name: Cross-compile with gox for ${{ matrix.targets }}
env:
GOFLAGS: "-trimpath"
GOX_ARCHS: "${{ matrix.targets }}"
run: |
mkdir build-output
gox -parallel 2 -verbose -osarch "$GOX_ARCHS" -output "build-output/{{.Dir}}_{{.OS}}_{{.Arch}}" ./cmd/restic
gox -parallel 2 -verbose -osarch "$GOX_ARCHS" -tags debug -output "build-output/{{.Dir}}_{{.OS}}_{{.Arch}}_debug" ./cmd/restic
lint:
name: lint
runs-on: ubuntu-latest
permissions:
contents: read
# allow annotating code in the PR
checks: write
steps:
- name: Check out code
uses: actions/checkout@v4
- name: Set up Go ${{ env.latest_go }}
uses: actions/setup-go@v5
uses: actions/setup-go@v2
with:
go-version: ${{ env.latest_go }}
- name: Check out code
uses: actions/checkout@v2
- name: golangci-lint
uses: golangci/golangci-lint-action@v6
uses: golangci/golangci-lint-action@v2
with:
# Required: the version of golangci-lint is required and must be specified without patch version: we always use the latest patch version.
version: v1.64.8
version: v1.45
# Optional: show only new issues if it's a pull request. The default value is `false`.
only-new-issues: true
args: --verbose --timeout 5m
skip-go-installation: true
# only run golangci-lint for pull requests, otherwise ALL hints get
# reported. We need to slowly address all issues until we can enable
@@ -267,31 +283,16 @@ jobs:
go mod tidy
git diff --exit-code go.mod go.sum
analyze:
name: Analyze results
needs: [test, cross_compile, lint]
if: always()
permissions: # no need to access code
contents: none
runs-on: ubuntu-latest
steps:
- name: Decide whether the needed jobs succeeded or failed
uses: re-actors/alls-green@05ac9388f0aebcb5727afa17fcccfecd6f8ec5fe
with:
jobs: ${{ toJSON(needs) }}
docker:
name: docker
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v4
uses: actions/checkout@v2
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
uses: docker/metadata-action@v3
with:
# list of Docker images to use as base name for tags
images: |
@@ -307,14 +308,14 @@ jobs:
type=sha
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v1
- name: Build and push
id: docker_build
uses: docker/build-push-action@v6
uses: docker/build-push-action@v2
with:
push: false
context: .

2
.gitignore vendored
View File

@@ -1,5 +1,3 @@
/.idea
/restic
/restic.exe
/.vagrant
/.vscode

View File

@@ -10,10 +10,13 @@ linters:
# make sure all errors returned by functions are handled
- errcheck
# find unused code
- deadcode
# show how code can be simplified
- gosimple
# make sure code is formatted
# # make sure code is formatted
- gofmt
# examine code and report suspicious constructs, such as Printf calls whose
@@ -21,7 +24,7 @@ linters:
- govet
# make sure names and comments are used according to the conventions
- revive
- golint
# detect when assignments to existing variables are not used
- ineffassign
@@ -32,14 +35,15 @@ linters:
# find unused variables, functions, structs, types, etc.
- unused
# find unused struct fields
- structcheck
# find unused global variables
- varcheck
# parse and typecheck code
- typecheck
# ensure that http response bodies are closed
- bodyclose
- importas
issues:
# don't use the default exclude rules, this hides (among others) ignored
# errors from Close() calls
@@ -47,24 +51,7 @@ issues:
# list of things to not warn about
exclude:
# revive: do not warn about missing comments for exported stuff
- exported (function|method|var|type|const) .* should have comment or be unexported
# revive: ignore constants in all caps
# golint: do not warn about missing comments for exported stuff
- exported (function|method|var|type|const) `.*` should have comment or be unexported
# golint: ignore constants in all caps
- don't use ALL_CAPS in Go names; use CamelCase
# revive: lots of packages don't have such a comment
- "package-comments: should have a package comment"
# staticcheck: there's no easy way to replace these packages
- "SA1019: \"golang.org/x/crypto/poly1305\" is deprecated"
- "SA1019: \"golang.org/x/crypto/openpgp\" is deprecated"
- "redefines-builtin-id:"
exclude-rules:
# revive: ignore unused parameters in tests
- path: (_test\.go|testing\.go|backend/.*/tests\.go)
text: "unused-parameter:"
linters-settings:
importas:
alias:
- pkg: github.com/restic/restic/internal/test
alias: rtest

View File

@@ -1,22 +0,0 @@
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
version: 2
build:
os: ubuntu-22.04
tools:
python: "3.11"
# Build HTMLZip
formats:
- htmlzip
# Build documentation in the docs/ directory with Sphinx
sphinx:
configuration: doc/conf.py
# https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html
python:
install:
- requirements: doc/requirements.txt

File diff suppressed because it is too large Load Diff

View File

@@ -6,8 +6,7 @@ Ways to Help Out
Thank you for your contribution! Please **open an issue first** (or add a
comment to an existing issue) if you plan to work on any code or add a new
feature. This way, duplicate work is prevented and we can discuss your ideas
and design first. Small bugfixes are an exception to this rule, just open a
pull request in this case.
and design first.
There are several ways you can help us out. First of all code contributions and
bug fixes are most welcome. However even "minor" details as fixing spelling
@@ -49,8 +48,9 @@ environment was used and so on. Please tell us at least the following things:
Remember, the easier it is for us to reproduce the bug, the earlier it will be
corrected!
In addition, you can instruct restic to create a debug log by setting the
environment variable `DEBUG_LOG` to a file, e.g. like this:
In addition, you can compile restic with debug support by running
`go run build.go -tags debug` and instructing it to create a debug
log by setting the environment variable `DEBUG_LOG` to a file, e.g. like this:
$ export DEBUG_LOG=/tmp/restic-debug.log
$ restic backup ~/work
@@ -59,19 +59,6 @@ Please be aware that the debug log file will contain potentially sensitive
things like file and directory names, so please either redact it before
uploading it somewhere or post only the parts that are really relevant.
If restic gets stuck, please also include a stacktrace in the description.
On non-Windows systems, you can send a SIGQUIT signal to restic or press
`Ctrl-\` to achieve the same result. This causes restic to print a stacktrace
and then exit immediately. This will not damage your repository, however,
it might be necessary to manually clean up stale lock files using
`restic unlock`.
On Windows, please set the environment variable `RESTIC_DEBUG_STACKTRACE_SIGINT`
to `true` and press `Ctrl-C` to create a stacktrace.
If you think restic uses too much memory or a too large cache directory, then
please include the output of `restic stats --mode debug`.
Development Environment
=======================
@@ -79,8 +66,8 @@ Development Environment
The repository contains the code written for restic in the directories
`cmd/` and `internal/`.
Make sure you have the minimum required Go version installed. Clone the repo
(without having `$GOPATH` set) and `cd` into the directory:
Restic requires Go version 1.14 or later for compiling. Clone the repo (without
having `$GOPATH` set) and `cd` into the directory:
$ unset GOPATH
$ git clone https://github.com/restic/restic
@@ -90,42 +77,12 @@ Then use the `go` tool to build restic:
$ go build ./cmd/restic
$ ./restic version
restic 0.14.0-dev (compiled manually) compiled with go1.19 on linux/amd64
To create a debug build use:
$ go build -tags debug ./cmd/restic
restic 0.10.0-dev (compiled manually) compiled with go1.15.2 on linux/amd64
You can run all tests with the following command:
$ go test ./...
Performance and Memory Usage Issues
===================================
Debug builds of restic support the `--block-profile`, `--cpu-profile`,
`--mem-profile`, and `--trace-profile` options which collect performance data
that later on can be analyzed using the go tools:
$ restic --cpu-profile . [...]
$ go tool pprof -http localhost:12345 cpu.pprof
To analyze a trace profile use `go tool trace -http=localhost:12345 trace.out`.
As the memory usage of restic changes over time, it may be useful to capture a
snapshot of the current heap. This is possible using then `--listen-profile`
option. Then while restic runs you can query and afterwards analyze the heap statistics.
$ restic --listen-profile localhost:12345 [...]
$ curl http://localhost:12345/debug/pprof/heap -o heap.pprof
$ go tool pprof -http localhost:12345 heap.pprof
Further useful tools are setting the environment variable `GODEBUG=gctrace=1`,
which provides information about garbage collector runs. For a graphical variant
combine this with gcvis.
Providing Patches
=================

View File

@@ -10,7 +10,8 @@ For detailed usage and installation instructions check out the [documentation](h
You can ask questions in our [Discourse forum](https://forum.restic.net).
## Quick start
Quick start
-----------
Once you've [installed](https://restic.readthedocs.io/en/latest/020_installation.html) restic, start
off with creating a repository for your backups:
@@ -58,7 +59,7 @@ Therefore, restic supports the following backends for storing backups natively:
Restic is a program that does backups right and was designed with the
following principles in mind:
- **Easy**: Doing backups should be a frictionless process, otherwise
- **Easy:** Doing backups should be a frictionless process, otherwise
you might be tempted to skip it. Restic should be easy to configure
and use, so that, in the event of a data loss, you can just restore
it. Likewise, restoring data should not be complicated.
@@ -91,17 +92,20 @@ reproduce a byte identical version from the source code for that
release. Instructions on how to do that are contained in the
[builder repository](https://github.com/restic/builder).
## News
News
----
You can follow the restic project on Mastodon [@resticbackup](https://fosstodon.org/@restic) or subscribe to
You can follow the restic project on Twitter [@resticbackup](https://twitter.com/resticbackup) or by subscribing to
the [project blog](https://restic.net/blog/).
## License
License
-------
Restic is licensed under [BSD 2-Clause License](https://opensource.org/licenses/BSD-2-Clause). You can find the
complete text in [`LICENSE`](LICENSE).
complete text in [``LICENSE``](LICENSE).
## Sponsorship
Sponsorship
-----------
Backend integration tests for Google Cloud Storage and Microsoft Azure Blob
Storage are sponsored by [AppsCode](https://appscode.com)!

View File

@@ -1 +1 @@
0.18.0-dev
0.13.1

View File

@@ -3,8 +3,8 @@
// This program aims to make building Go programs for end users easier by just
// calling it with `go run`, without having to setup a GOPATH.
//
// This program checks for a minimum Go version. It will use Go modules for
// compilation. It builds the package configured as Main in the Config struct.
// This program needs Go >= 1.12. It'll use Go modules for compilation. It
// builds the package configured as Main in the Config struct.
// BSD 2-Clause License
//
@@ -43,6 +43,7 @@ package main
import (
"fmt"
"io"
"io/ioutil"
"os"
"os/exec"
"path/filepath"
@@ -53,14 +54,12 @@ import (
// config contains the configuration for the program to build.
var config = Config{
Name: "restic", // name of the program executable and directory
Namespace: "github.com/restic/restic", // subdir of GOPATH, e.g. "github.com/foo/bar"
Main: "./cmd/restic", // package name for the main package
// disable_grpc_modules is necessary to reduce the binary size since cloud.google.com/go/storage v1.44.0
// see https://github.com/googleapis/google-cloud-go/issues/11448
DefaultBuildTags: []string{"selfupdate", "disable_grpc_modules"}, // specify build tags which are always used
Tests: []string{"./..."}, // tests to run
MinVersion: GoVersion{Major: 1, Minor: 23, Patch: 0}, // minimum Go version supported
Name: "restic", // name of the program executable and directory
Namespace: "github.com/restic/restic", // subdir of GOPATH, e.g. "github.com/foo/bar"
Main: "./cmd/restic", // package name for the main package
DefaultBuildTags: []string{"selfupdate"}, // specify build tags which are always used
Tests: []string{"./..."}, // tests to run
MinVersion: GoVersion{Major: 1, Minor: 14, Patch: 0}, // minimum Go version supported
}
// Config configures the build.
@@ -180,7 +179,7 @@ func test(cwd string, env map[string]string, args ...string) error {
// getVersion returns the version string from the file VERSION in the current
// directory.
func getVersionFromFile() string {
buf, err := os.ReadFile("VERSION")
buf, err := ioutil.ReadFile("VERSION")
if err != nil {
verbosePrintf("error reading file VERSION: %v\n", err)
return ""
@@ -300,21 +299,19 @@ func (v GoVersion) AtLeast(other GoVersion) bool {
return true
}
if v.Major > other.Major {
return true
}
if v.Major < other.Major {
return false
}
if v.Minor > other.Minor {
return true
}
if v.Minor < other.Minor {
return false
}
return v.Patch >= other.Patch
if v.Patch < other.Patch {
return false
}
return true
}
func (v GoVersion) String() string {
@@ -322,8 +319,12 @@ func (v GoVersion) String() string {
}
func main() {
if !goVersion.AtLeast(GoVersion{1, 12, 0}) {
die("Go version (%v) is too old, restic requires Go >= 1.12\n", goVersion)
}
if !goVersion.AtLeast(config.MinVersion) {
fmt.Fprintf(os.Stderr, "Detected version %s is too old, restic requires at least %s\n", goVersion, config.MinVersion)
fmt.Fprintf(os.Stderr, "%s detected, this program requires at least %s\n", goVersion, config.MinVersion)
os.Exit(1)
}

View File

@@ -10,7 +10,7 @@ https://github.com/restic/restic/issues/2244
NOTE: This new implementation does not guarantee order in which blobs
are written to the target files and, for example, the last blob of a
file can be written to the file before any of the preceding file blobs.
file can be written to the file before any of the preceeding file blobs.
It is therefore possible to have gaps in the data written to the target
files if restore fails or interrupted by the user.

View File

@@ -1,6 +1,6 @@
Bugfix: Don't abort the stats command when data blobs are missing
Running the stats command in the blobs-per-file mode on a repository with
Runing the stats command in the blobs-per-file mode on a repository with
missing data blobs previously resulted in a crash.
https://github.com/restic/restic/pull/2668

View File

@@ -1,6 +1,6 @@
Bugfix: Mark repository files as read-only when using the local backend
Files stored in a local repository were marked as writable on the
Files stored in a local repository were marked as writeable on the
filesystem for non-Windows systems, which did not prevent accidental file
modifications outside of restic. In addition, the local backend did not work
with certain filesystems and network mounts which do not permit modifications

View File

@@ -5,7 +5,7 @@ another process using an exclusive lock through a filesystem snapshot. Restic
was unable to backup those files before. This update enables backing up these
files.
This needs to be enabled explicitly using the --use-fs-snapshot option of the
This needs to be enabled explicitely using the --use-fs-snapshot option of the
backup command.
https://github.com/restic/restic/issues/340

View File

@@ -1,7 +1,7 @@
Bugfix: Correct statistics for overlapping backup sources
Bugfix: Correct statistics for overlapping targets
A user reported that restic's statistics and progress information during backup
was not correctly calculated when the backup sources (files/dirs to save)
was not correctly calculated when the backup targets (files/dirs to save)
overlap. For example, consider a directory `foo` which contains (among others)
a file `foo/bar`. When `restic backup foo foo/bar` was run, restic counted the
size of the file `foo/bar` twice, so the completeness percentage as well as the

View File

@@ -2,7 +2,7 @@ Enhancement: Parallelize scan of snapshot content in `copy` and `prune`
The `copy` and `prune` commands used to traverse the directories of
snapshots one by one to find used data. This snapshot traversal is
now parallelized which can speed up this step several times.
now parallized which can speed up this step several times.
In addition the `check` command now reports how many snapshots have
already been processed.

View File

@@ -0,0 +1,7 @@
Bugfix: Fix the diff command
There was a bug in the `diff` command, it would always show files in a removed
directory as added. We've fixed that.
https://github.com/restic/restic/issues/3685
https://github.com/restic/restic/pull/3686

View File

@@ -0,0 +1,12 @@
Bugfix: Fix rclone (shimmed by Scoop) and sftp stopped working on Windows
In #3602 a fix was introduced to fix the problem that rclone prematurely exits
when Ctrl+C is pressed on Windows. The solution was to create the subprocess
with its console detached from the restic console. However, such solution
fails when using rclone install by scoop or using sftp with a passphrase-
protected private key. We've fixed that by using a different approach to prevent
Ctrl-C from passing down too early.
https://github.com/restic/restic/issues/3681
https://github.com/restic/restic/issues/3692
https://github.com/restic/restic/pull/3696

View File

@@ -1,9 +0,0 @@
Enhancement: Support pruning even when the disk is full
When running out of disk space it was no longer possible to add or remove
data from a repository. To help with recovering from such a deadlock, the
prune command now supports an `--unsafe-recover-no-free-space` option to
recover from these situations. Make sure to read the documentation first!
https://github.com/restic/restic/issues/1153
https://github.com/restic/restic/pull/3481

View File

@@ -1,8 +0,0 @@
Change: Support debug log creation in release builds
Creating a debug log was only possible in debug builds which required users to
manually build restic. We changed the release builds to allow creating debug
logs by simply setting the environment variable `DEBUG_LOG=logname.log`.
https://github.com/restic/restic/issues/1842
https://github.com/restic/restic/pull/3826

View File

@@ -1,28 +0,0 @@
Enhancement: Add compression support
We've added compression support to the restic repository format. To create a
repository using the new format run `init --repository-version 2`. Please note
that the repository cannot be read by restic versions prior to 0.14.0.
You can configure whether data is compressed with the option `--compression`. It
can be set to `auto` (the default, which will compress very fast), `max` (which
will trade backup speed and CPU usage for better compression), or `off` (which
disables compression). Each setting is only applied for the current run of restic
and does *not* apply to future runs. The option can also be set via the
environment variable `RESTIC_COMPRESSION`.
To upgrade in place run `migrate upgrade_repo_v2` followed by `prune`. See the
documentation for more details. The migration checks the repository integrity
and upgrades the repository format, but will not change any data. Afterwards,
prune will rewrite the metadata to make use of compression.
As an alternative you can use the `copy` command to migrate snapshots; First
create a new repository using
`init --repository-version 2 --copy-chunker-params --repo2 path/to/old/repo`,
and then use the `copy` command to copy all snapshots to the new repository.
https://github.com/restic/restic/issues/21
https://github.com/restic/restic/issues/3779
https://github.com/restic/restic/pull/3666
https://github.com/restic/restic/pull/3704
https://github.com/restic/restic/pull/3733

View File

@@ -1,18 +0,0 @@
Enhancement: Adaptive IO concurrency based on backend connections
Many commands used hard-coded limits for the number of concurrent operations.
This prevented speed improvements by increasing the number of connections used
by a backend.
These limits have now been replaced by using the configured number of backend
connections instead, which can be controlled using the
`-o <backend-name>.connections=5` option. Commands will then automatically
scale their parallelism accordingly.
To limit the number of CPU cores used by restic, you can set the environment
variable `GOMAXPROCS` accordingly. For example to use a single CPU core, use
`GOMAXPROCS=1`.
https://github.com/restic/restic/issues/2162
https://github.com/restic/restic/issues/1467
https://github.com/restic/restic/pull/3611

View File

@@ -1,8 +0,0 @@
Bugfix: Support `self-update` on Windows
Restic `self-update` would fail in situations where the operating system
locks running binaries, including Windows. The new behavior works around
this by renaming the running file and swapping the updated file in place.
https://github.com/restic/restic/issues/2248
https://github.com/restic/restic/pull/3675

View File

@@ -1,12 +0,0 @@
Enhancement: Allow pack size customization
Restic now uses a target pack size of 16 MiB by default. This can be customized
using the `--pack-size size` option. Supported pack sizes range between 4 and
128 MiB.
It is possible to migrate an existing repository to _larger_ pack files using
`prune --repack-small`. This will rewrite every pack file which is
significantly smaller than the target size.
https://github.com/restic/restic/issues/2291
https://github.com/restic/restic/pull/3731

View File

@@ -1,14 +0,0 @@
Enhancement: Allow use of SAS token to authenticate to Azure
Previously restic only supported AccountKeys to authenticate to Azure
storage accounts, which necessitates giving a significant amount of
access.
We added support for Azure SAS tokens which are a more fine-grained
and time-limited manner of granting access. Set the `AZURE_ACCOUNT_NAME`
and `AZURE_ACCOUNT_SAS` environment variables to use a SAS token for
authentication. Note that if `AZURE_ACCOUNT_KEY` is set, it will take
precedence.
https://github.com/restic/restic/issues/2295
https://github.com/restic/restic/pull/3661

View File

@@ -1,13 +0,0 @@
Enhancement: Improve backup speed with many small files
We have restructured the backup pipeline to continue reading files while all
upload connections are busy. This allows the backup to already prepare the next
data file such that the upload can continue as soon as a connection becomes
available. This can especially improve the backup performance for high latency
backends.
The upload concurrency is now controlled using the `-o <backend-name>.connections=5`
option.
https://github.com/restic/restic/issues/2696
https://github.com/restic/restic/pull/3489

View File

@@ -1,15 +0,0 @@
Enhancement: Make snapshot directory structure of `mount` command customizable
We've added the possibility to customize the snapshot directory structure of
the `mount` command using templates passed to the `--snapshot-template` option.
The formatting of snapshots' timestamps is now controlled using `--time-template`
and supports subdirectories to for example group snapshots by year. Please
see `restic help mount` for further details.
Characters in tag names which are not allowed in a filename are replaced by
underscores `_`. For example a tag `foo/bar` will result in a directory name
of `foo_bar`.
https://github.com/restic/restic/issues/2907
https://github.com/restic/restic/pull/2913
https://github.com/restic/restic/pull/3691

View File

@@ -1,12 +0,0 @@
Enhancement: Optimize handling of duplicate blobs in `prune`
Restic `prune` always used to repack all data files containing duplicate
blobs. This effectively removed all duplicates during prune. However, as a
consequence all these data files were repacked even if the unused repository
space threshold could be reached with less work.
This is now changed and `prune` works nice and fast even when there are lots
of duplicate blobs.
https://github.com/restic/restic/issues/3114
https://github.com/restic/restic/pull/3290

View File

@@ -1,13 +0,0 @@
Change: Deprecate `check --check-unused` and add further checks
Since restic 0.12.0, it is expected to still have unused blobs after running
`prune`. This made the `--check-unused` option of the `check` command rather
useless and tended to confuse users. This option has been deprecated and is
now ignored.
The `check` command now also warns if a repository is using either the legacy
S3 layout or mixed pack files with both tree and data blobs. The latter is
known to cause performance problems.
https://github.com/restic/restic/issues/3295
https://github.com/restic/restic/pull/3730

View File

@@ -1,14 +0,0 @@
Bugfix: List snapshots in backend at most once to resolve snapshot IDs
Many commands support specifying a list of snapshot IDs which are then used to
determine the snapshots to be processed by the command. To resolve snapshot IDs
or `latest`, and check that these exist, restic previously listed all snapshots
stored in the repository. Depending on the backend this could be a slow and/or
expensive operation.
Restic now lists the snapshots only once and remembers the result in order to
resolve all further snapshot IDs swiftly.
https://github.com/restic/restic/issues/3428
https://github.com/restic/restic/pull/3570
https://github.com/restic/restic/pull/3395

View File

@@ -1,14 +0,0 @@
Bugfix: Fix rare 'not found in repository' error for `copy` command
In rare cases `copy` (and other commands) would report that `LoadTree(...)`
returned an `id [...] not found in repository` error. This could be caused by
a backup or copy command running concurrently. The error was only temporary;
running the failed restic command a second time as a workaround did resolve the
error.
This issue has now been fixed by correcting the order in which restic reads data
from the repository. It is now guaranteed that restic only loads snapshots for
which all necessary data is already available.
https://github.com/restic/restic/issues/3432
https://github.com/restic/restic/pull/3570

View File

@@ -1,10 +0,0 @@
Enhancement: Improve handling of temporary files on Windows
In some cases restic failed to delete temporary files, causing the current
command to fail. This has now been fixed by ensuring that Windows automatically
deletes the file. In addition, temporary files are only written to disk when
necessary, reducing disk writes.
https://github.com/restic/restic/issues/3465
https://github.com/restic/restic/issues/1551
https://github.com/restic/restic/pull/3610

View File

@@ -1,7 +0,0 @@
Bugfix: The `diff` command incorrectly listed some files as added
There was a bug in the `diff` command, causing it to always show files in a
removed directory as added. This has now been fixed.
https://github.com/restic/restic/issues/3685
https://github.com/restic/restic/pull/3686

View File

@@ -1,13 +0,0 @@
Bugfix: Fix rclone (shimmed by Scoop) and sftp not working on Windows
In #3602 a fix was introduced to address the problem of `rclone` prematurely
exiting when Ctrl+C is pressed on Windows. The solution was to create the
subprocess with its console detached from the restic console.
However, this solution failed when using `rclone` installed by Scoop or using
`sftp` with a passphrase-protected private key. We've now fixed this by using
a different approach to prevent Ctrl-C from passing down too early.
https://github.com/restic/restic/issues/3681
https://github.com/restic/restic/issues/3692
https://github.com/restic/restic/pull/3696

View File

@@ -1,11 +0,0 @@
Enhancement: Validate exclude patterns before backing up
Exclude patterns provided via `--exclude`, `--iexclude`, `--exclude-file` or
`--iexclude-file` previously weren't validated. As a consequence, invalid
patterns resulted in files that were meant to be excluded being backed up.
Restic now validates all patterns before running the backup and aborts with
a fatal error if an invalid pattern is detected.
https://github.com/restic/restic/issues/3709
https://github.com/restic/restic/pull/3734

View File

@@ -1,13 +0,0 @@
Bugfix: Directory sync errors for repositories accessed via SMB
On Linux and macOS, accessing a repository via a SMB/CIFS mount resulted in
restic failing to save the lock file, yielding the following errors:
Save(<lock/071fe833f0>) returned error, retrying after 552.330144ms: sync /repo/locks: no such file or directory
Save(<lock/bf789d7343>) returned error, retrying after 552.330144ms: sync /repo/locks: invalid argument
This has now been fixed by ignoring the relevant error codes.
https://github.com/restic/restic/issues/3720
https://github.com/restic/restic/issues/3751
https://github.com/restic/restic/pull/3752

View File

@@ -1,8 +0,0 @@
Bugfix: The `stats` command miscalculated restore size for multiple snapshots
Since restic 0.10.0 the restore size calculated by the `stats` command for
multiple snapshots was too low. The hardlink detection was accidentally applied
across multiple snapshots and thus ignored many files. This has now been fixed.
https://github.com/restic/restic/issues/3736
https://github.com/restic/restic/pull/3740

View File

@@ -1,8 +0,0 @@
Enhancement: Improve SFTP repository initialization over slow links
The `init` command, when used on an SFTP backend, now sends multiple `mkdir`
commands to the backend concurrently. This reduces the waiting times when
creating a repository over a very slow connection.
https://github.com/restic/restic/issues/3837
https://github.com/restic/restic/pull/3840

View File

@@ -1,9 +0,0 @@
Bugfix: Yield error on invalid policy to `forget`
The `forget` command previously silently ignored invalid/unsupported
units in the duration options, such as e.g. `--keep-within-daily 2w`.
Specifying an invalid/unsupported duration unit now results in an error.
https://github.com/restic/restic/issues/3861
https://github.com/restic/restic/pull/3862

View File

@@ -1,21 +0,0 @@
Enhancement: Use config file permissions to control file group access
Previously files in a local/SFTP repository would always end up with very
restrictive access permissions, allowing access only to the owner. This
prevented a number of valid use-cases involving groups and ACLs.
We now use the permissions of the config file in the repository to decide
whether group access should be given to newly created repository files or
not. We arrange for repository files to be created group readable exactly
when the repository config file is group readable.
To opt-in to group readable repositories, a simple `chmod -R g+r` or
equivalent on the config file can be used. For repositories that should
be writable by group members a tad more setup is required, see the docs.
Posix ACLs can also be used now that the group permissions being forced to
zero no longer masks the effect of ACL entries.
https://github.com/restic/restic/issues/2351
https://github.com/restic/restic/pull/3419
https://forum.restic.net/t/1391

View File

@@ -1,9 +0,0 @@
Enhancement: Allow limiting IO concurrency for local and SFTP backend
Restic did not support limiting the IO concurrency / number of connections for
accessing repositories stored using the local or SFTP backends. The number of
connections is now limited as for other backends, and can be configured via the
`-o local.connections=2` and `-o sftp.connections=5` options. This ensures that
restic does not overwhelm the backend with concurrent IO operations.
https://github.com/restic/restic/pull/3475

View File

@@ -1,13 +0,0 @@
Enhancement: Stream data in `check` and `prune` commands
The commands `check --read-data` and `prune` previously downloaded data files
into temporary files which could end up being written to disk. This could cause
a large amount of data being written to disk.
The pack files are now instead streamed, which removes the need for temporary
files. Please note that *uploads* during `backup` and `prune` still require
temporary files.
https://github.com/restic/restic/pull/3484
https://github.com/restic/restic/issues/3710
https://github.com/restic/restic/pull/3717

View File

@@ -1,10 +0,0 @@
Enhancement: Improve speed of `copy` command
The `copy` command could require a long time to copy snapshots for non-local
backends. This has been improved to provide a throughput comparable to the
`restore` command.
Additionally, `copy` now displays a progress bar.
https://github.com/restic/restic/issues/2923
https://github.com/restic/restic/pull/3513

View File

@@ -1,8 +0,0 @@
Change: Update dependencies and require Go 1.15 or newer
We've updated most dependencies. Since some libraries require newer language
features we're dropping support for Go 1.14, which means that restic now
requires at least Go 1.15 to build.
https://github.com/restic/restic/issues/3680
https://github.com/restic/restic/issues/3883

View File

@@ -1,7 +0,0 @@
Bugfix: Print "wrong password" to stderr instead of stdout
If an invalid password was entered, the error message was printed on stdout and
not on stderr as intended. This has now been fixed.
https://github.com/restic/restic/pull/3716
https://forum.restic.net/t/4965

View File

@@ -1,8 +0,0 @@
Enhancement: Display full IDs in `check` warnings
When running commands to inspect or repair a damaged repository, it is often
necessary to supply the full IDs of objects stored in the repository.
The output of `check` now includes full IDs instead of their shortened variant.
https://github.com/restic/restic/pull/3729

View File

@@ -1,19 +0,0 @@
Change: Replace `--repo2` option used by `init`/`copy` with `--from-repo`
The `init` and `copy` commands can read data from another repository.
However, confusingly `--repo2` referred to the repository *from* which the
`init` command copies parameters, but for the `copy` command `--repo2`
referred to the copy *destination*.
We've introduced a new option, `--from-repo`, which always refers to the
source repository for both commands. The old parameter names have been
deprecated but still work. To create a new repository and copy all snapshots
to it, the commands are now as follows:
```
restic -r /srv/restic-repo-copy init --from-repo /srv/restic-repo --copy-chunker-params
restic -r /srv/restic-repo-copy copy --from-repo /srv/restic-repo
```
https://github.com/restic/restic/pull/3742
https://forum.restic.net/t/5017

View File

@@ -1,14 +0,0 @@
Bugfix: Correctly rebuild index for legacy repositories
After running `rebuild-index` on a legacy repository containing mixed pack
files (that is, pack files which store both metadata and file data), `check`
printed warnings like `pack 12345678 contained in several indexes: ...`.
This warning was not critical, but has now nonetheless been fixed by properly
handling mixed pack files while rebuilding the index.
Running `prune` for such legacy repositories will also fix the warning by
reorganizing the pack files which caused it.
https://github.com/restic/restic/pull/3772
https://github.com/restic/restic/pull/3884
https://forum.restic.net/t/5044/13

View File

@@ -1,7 +0,0 @@
Enhancement: Optimize memory usage for directories with many files
Backing up a directory with hundreds of thousands or more files caused restic
to require large amounts of memory. We've now optimized the `backup` command
such that it requires up to 30% less memory.
https://github.com/restic/restic/pull/3773

View File

@@ -1,11 +0,0 @@
Bugfix: Limit number of key files tested while opening a repository
Previously, restic tested the password against every key in the repository
when opening a repository. The more keys there were in the repository, the
slower this operation became.
Restic now tests the password against up to 20 key files in the repository.
Alternatively, you can use the `--key-hint=<key ID>` option to specify a
specific key file to use instead.
https://github.com/restic/restic/pull/3776

View File

@@ -1,11 +0,0 @@
Enhancement: Validate include/exclude patterns before restoring
Patterns provided to `restore` via `--exclude`, `--iexclude`, `--include`
and `--iinclude` weren't validated before running the restore. Invalid
patterns would result in error messages being printed repeatedly, and
possibly unwanted files being restored.
Restic now validates all patterns before running the restore, and aborts
with a fatal error if an invalid pattern is detected.
https://github.com/restic/restic/pull/3819

View File

@@ -1,8 +0,0 @@
Enhancement: Implement `rewrite` command
Restic now has a `rewrite` command which allows to rewrite existing snapshots
to remove unwanted files.
https://github.com/restic/restic/issues/14
https://github.com/restic/restic/pull/2731
https://github.com/restic/restic/pull/4079

View File

@@ -1,15 +0,0 @@
Enhancement: Inform about successful retries after errors
When a recoverable error is encountered, restic shows a warning message saying
that it's retrying, e.g.:
`Save(<data/956b9ced99>) returned error, retrying after 357.131936ms: ...`
This message can be confusing in that it never clearly states whether the retry
is successful or not. This has now been fixed such that restic follows up with
a message confirming a successful retry, e.g.:
`Save(<data/956b9ced99>) operation successful after 1 retries`
https://github.com/restic/restic/issues/1734
https://github.com/restic/restic/pull/2661

View File

@@ -1,12 +0,0 @@
Enhancement: Improve handling of directories with duplicate entries
If for some reason a directory contains a duplicate entry, the `backup` command
would previously fail with a `node "path/to/file" already present` or `nodes
are not ordered got "path/to/file", last "path/to/file"` error.
The error handling has been improved to only report a warning in this case. Make
sure to check that the filesystem in question is not damaged if you see this!
https://github.com/restic/restic/issues/1866
https://github.com/restic/restic/issues/3937
https://github.com/restic/restic/pull/3880

View File

@@ -1,10 +0,0 @@
Bugfix: Make `mount` return exit code 0 after receiving Ctrl-C / SIGINT
To stop the `mount` command, a user has to press Ctrl-C or send a SIGINT
signal to restic. This used to cause restic to exit with a non-zero exit code.
The exit code has now been changed to zero as the above is the expected way
to stop the `mount` command and should therefore be considered successful.
https://github.com/restic/restic/issues/2015
https://github.com/restic/restic/pull/3894

View File

@@ -1,19 +0,0 @@
Enhancement: Support B2 API keys restricted to hiding but not deleting files
When the B2 backend does not have the necessary permissions to permanently
delete files, it now automatically falls back to hiding files. This allows
using restic with an application key which is not allowed to delete files.
This can prevent an attacker from deleting backups with such an API key.
To use this feature create an application key without the `deleteFiles`
capability. It is recommended to restrict the key to just one bucket.
For example using the `b2` command line tool:
`b2 create-key --bucket <bucketName> <keyName> listBuckets,readFiles,writeFiles,listFiles`
Alternatively, you can use the S3 backend to access B2, as described
in the documentation. In this mode, files are also only hidden instead
of being deleted permanently.
https://github.com/restic/restic/issues/2134
https://github.com/restic/restic/pull/2398

View File

@@ -1,11 +0,0 @@
Enhancement: Make `init` open only one connection for the SFTP backend
The `init` command using the SFTP backend used to connect twice to the
repository. This could be inconvenient if the user must enter a password,
or cause `init` to fail if the server does not correctly close the first SFTP
connection.
This has now been fixed by reusing the first/initial SFTP connection opened.
https://github.com/restic/restic/issues/2152
https://github.com/restic/restic/pull/3882

View File

@@ -1,13 +0,0 @@
Enhancement: Handle cache corruption on disk and in downloads
In rare situations, like for example after a system crash, the data stored
in the cache might be corrupted. This could cause restic to fail and required
manually deleting the cache.
Restic now automatically removes broken data from the cache, allowing it
to recover from such a situation without user intervention. In addition,
restic retries downloads which return corrupt data in order to also handle
temporary download problems.
https://github.com/restic/restic/issues/2533
https://github.com/restic/restic/pull/3521

View File

@@ -1,17 +0,0 @@
Bugfix: Don't read password from stdin for `backup --stdin`
The `backup` command when used with `--stdin` previously tried to read first
the password, then the data to be backed up from standard input. This meant
it would often confuse part of the data for the password.
From now on, it will instead exit with the message `Fatal: cannot read both
password and data from stdin` unless the password is passed in some other
way (such as `--restic-password-file`, `RESTIC_PASSWORD`, etc).
To enter the password interactively a password command has to be used. For
example on Linux, `mysqldump somedatabase | restic backup --stdin
--password-command='sh -c "systemd-ask-password < /dev/tty"'` securely reads
the password from the terminal.
https://github.com/restic/restic/issues/2591
https://github.com/restic/restic/pull/4011

View File

@@ -1,9 +0,0 @@
Enhancement: Support restoring symbolic links on Windows
The `restore` command now supports restoring symbolic links on Windows. Because
of Windows specific restrictions this is only possible when running restic with
the `SeCreateSymbolicLinkPrivilege` privilege or as an administrator.
https://github.com/restic/restic/issues/1078
https://github.com/restic/restic/issues/2699
https://github.com/restic/restic/pull/2875

View File

@@ -1,20 +0,0 @@
Enhancement: Stricter repository lock handling
Previously, restic commands kept running even if they failed to refresh their
locks in time. This could be a problem e.g. in case the client system running
a backup entered the standby power mode while the backup was still in progress
(which would prevent the client from refreshing its lock), and after a short
delay another host successfully runs `unlock` and `prune` on the repository,
which would remove all data added by the in-progress backup. If the backup
client later continues its backup, even though its lock had expired in the
meantime, this would lead to an incomplete snapshot.
To address this, lock handling is now much stricter. Commands requiring a lock
are canceled if the lock is not refreshed successfully in time. In addition,
if a lock file is not readable restic will not allow starting a command. It may
be necessary to remove invalid lock files manually or use `unlock --remove-all`.
Please make sure that no other restic processes are running concurrently before
doing this, however.
https://github.com/restic/restic/issues/2715
https://github.com/restic/restic/pull/3569

View File

@@ -1,9 +0,0 @@
Change: Include full snapshot ID in JSON output of `backup`
We have changed the JSON output of the backup command to include the full
snapshot ID instead of just a shortened version, as the latter can be ambiguous
in some rare cases. To derive the short ID, please truncate the full ID down to
eight characters.
https://github.com/restic/restic/issues/2724
https://github.com/restic/restic/pull/3993

View File

@@ -1,8 +0,0 @@
Enhancement: Add support for `credential_process` to S3 backend
Restic now uses a newer library for the S3 backend, which adds support for the
`credential_process` option in the AWS credential configuration.
https://github.com/restic/restic/issues/3029
https://github.com/restic/restic/issues/4034
https://github.com/restic/restic/pull/4025

View File

@@ -1,8 +0,0 @@
Enhancement: Make `mount` command support macOS using macFUSE 4.x
Restic now uses a different FUSE library for mounting snapshots and making them
available as a FUSE filesystem using the `mount` command. This adds support for
macFUSE 4.x which can be used to make this work on recent macOS versions.
https://github.com/restic/restic/issues/3096
https://github.com/restic/restic/pull/4024

View File

@@ -1,7 +0,0 @@
Enhancement: Support JSON output for the `init` command
The `init` command used to ignore the `--json` option, but now outputs a JSON
message if the repository was created successfully.
https://github.com/restic/restic/issues/3124
https://github.com/restic/restic/pull/3132

View File

@@ -1,14 +0,0 @@
Bugfix: Delete files on Backblaze B2 more reliably
Restic used to only delete the latest version of files stored in B2. In most
cases this worked well as there was only a single version of the file. However,
due to retries while uploading it is possible for multiple file versions to be
stored at B2. This could lead to various problems for files that should have
been deleted but still existed.
The implementation has now been changed to delete all versions of files, which
doubles the amount of Class B transactions necessary to delete files, but
assures that no file versions are left behind.
https://github.com/restic/restic/issues/3161
https://github.com/restic/restic/pull/3885

View File

@@ -1,12 +0,0 @@
Bugfix: Make SFTP backend report no space left on device
Backing up to an SFTP backend would spew repeated SSH_FX_FAILURE messages when
the remote disk was full. Restic now reports "sftp: no space left on device"
and exits immediately when it detects this condition.
A fix for this issue was implemented in restic 0.12.1, but unfortunately the
fix itself contained a bug that prevented it from taking effect.
https://github.com/restic/restic/issues/3336
https://github.com/restic/restic/pull/3345
https://github.com/restic/restic/pull/4075

View File

@@ -1,10 +0,0 @@
Bugfix: Improve handling of interrupted syscalls in `mount` command
Accessing restic's FUSE mount could result in "input/output" errors when using
programs in which syscalls can be interrupted. This is for example the case for
Go programs. This has now been fixed by improved error handling of interrupted
syscalls.
https://github.com/restic/restic/issues/3567
https://github.com/restic/restic/issues/3694
https://github.com/restic/restic/pull/3875

View File

@@ -1,7 +0,0 @@
Bugfix: Fix stuck `copy` command when `-o <backend>.connections=1`
When running the `copy` command with `-o <backend>.connections=1` the
command would be infinitely stuck. This has now been fixed.
https://github.com/restic/restic/issues/3897
https://github.com/restic/restic/pull/3898

View File

@@ -1,9 +0,0 @@
Bugfix: Correct prune statistics for partially compressed repositories
In a partially compressed repository, one data blob can exist both in an
uncompressed and a compressed version. This caused the `prune` statistics to
become inaccurate and e.g. report a too high value for the unused size, such
as "unused size after prune: 16777215.991 TiB". This has now been fixed.
https://github.com/restic/restic/issues/3918
https://github.com/restic/restic/pull/3980

View File

@@ -1,11 +0,0 @@
Change: Make `unlock` display message only when locks were actually removed
The `unlock` command used to print the "successfully removed locks" message
whenever it was run, regardless of lock files having being removed or not.
This has now been changed such that it only prints the message if any lock
files were actually removed. In addition, it also reports the number of
removed lock files.
https://github.com/restic/restic/issues/3929
https://github.com/restic/restic/pull/3935

View File

@@ -1,15 +0,0 @@
Enhancement: Improve handling of ErrDot errors in rclone and sftp backends
Since Go 1.19, restic can no longer implicitly run relative executables which
are found in the current directory (e.g. `rclone` if found in `.`). This is a
security feature of Go to prevent against running unintended and possibly
harmful executables.
The error message for this was just "cannot run executable found relative to
current directory". This has now been improved to yield a more specific error
message, informing the user how to explicitly allow running the executable
using the `-o rclone.program` and `-o sftp.command` extended options with `./`.
https://github.com/restic/restic/issues/3932
https://pkg.go.dev/os/exec#hdr-Executables_in_the_current_directory
https://go.dev/blog/path-security

View File

@@ -1,8 +0,0 @@
Bugfix: Make `backup` no longer hang on Solaris when seeing a FIFO file
The `backup` command used to hang on Solaris whenever it encountered a FIFO
file (named pipe), due to a bug in the handling of extended attributes. This
bug has now been fixed.
https://github.com/restic/restic/issues/4003
https://github.com/restic/restic/pull/4053

View File

@@ -1,8 +0,0 @@
Bugfix: Support ExFAT-formatted local backends on macOS Ventura
ExFAT-formatted disks could not be used as local backends starting from macOS
Ventura. Restic commands would fail with an "inappropriate ioctl for device"
error. This has now been fixed.
https://github.com/restic/restic/issues/4016
https://github.com/restic/restic/pull/4021

View File

@@ -1,11 +0,0 @@
Change: Don't print skipped snapshots by default in `copy` command
The `copy` command used to print each snapshot that was skipped because it
already existed in the target repository. The amount of this output could
practically bury the list of snapshots that were actually copied.
From now on, the skipped snapshots are by default not printed at all, but
this can be re-enabled by increasing the verbosity level of the command.
https://github.com/restic/restic/issues/4033
https://github.com/restic/restic/pull/4066

View File

@@ -1,10 +0,0 @@
Bugfix: Make `init` ignore "Access Denied" errors when creating S3 buckets
In restic 0.9.0 through 0.13.0, the `init` command ignored some permission
errors from S3 backends when trying to check for bucket existence, so that
manually created buckets with custom permissions could be used for backups.
This feature became broken in 0.14.0, but has now been restored again.
https://github.com/restic/restic/issues/4085
https://github.com/restic/restic/pull/4086

View File

@@ -1,10 +0,0 @@
Bugfix: Don't generate negative UIDs and GIDs in tar files from `dump`
When using a 32-bit build of restic, the `dump` command could in some cases
create tar files containing negative UIDs and GIDs, which cannot be read by
GNU tar. This corner case especially applies to backups from stdin on Windows.
This is now fixed such that `dump` creates valid tar files in these cases too.
https://github.com/restic/restic/issues/4103
https://github.com/restic/restic/pull/4104

View File

@@ -1,17 +0,0 @@
Enhancement: Restore files with long runs of zeros as sparse files
When using `restore --sparse`, the restorer may now write files containing long
runs of zeros as sparse files (also called files with holes), where the zeros
are not actually written to disk.
How much space is saved by writing sparse files depends on the operating
system, file system and the distribution of zeros in the file.
During backup restic still reads the whole file including sparse regions, but
with optimized processing speed of sparse regions.
https://github.com/restic/restic/issues/79
https://github.com/restic/restic/issues/3903
https://github.com/restic/restic/pull/2601
https://github.com/restic/restic/pull/3854
https://forum.restic.net/t/sparse-file-support/1264

View File

@@ -1,7 +0,0 @@
Enhancement: Make backup file read concurrency configurable
The `backup` command now supports a `--read-concurrency` option which allows
tuning restic for very fast storage like NVMe disks by controlling the number
of concurrent file reads during the backup process.
https://github.com/restic/restic/pull/2750

View File

@@ -1,8 +0,0 @@
Bugfix: Make `restore` replace existing symlinks
When restoring a symlink, restic used to report an error if the target path
already existed. This has now been fixed such that the potentially existing
target path is first removed before the symlink is restored.
https://github.com/restic/restic/issues/2578
https://github.com/restic/restic/pull/3780

View File

@@ -1,6 +0,0 @@
Enhancement: Optimize prune memory usage
The `prune` command needs large amounts of memory in order to determine what to
keep and what to remove. This is now optimized to use up to 30% less memory.
https://github.com/restic/restic/pull/3899

View File

@@ -1,6 +0,0 @@
Enhancement: Improve speed of parent snapshot detection in `backup` command
Backing up a large number of files using `--files-from-verbatim` or `--files-from-raw`
options could require a long time to find the parent snapshot. This has been improved.
https://github.com/restic/restic/pull/3905

View File

@@ -1,12 +0,0 @@
Enhancement: Add compression statistics to the `stats` command
When executed with `--mode raw-data` on a repository that supports compression,
the `stats` command now calculates and displays, for the selected repository or
snapshots: the uncompressed size of the data; the compression progress
(percentage of data that has been compressed); the compression ratio of the
compressed data; the total space saving.
It also takes into account both the compressed and uncompressed data if the
repository is only partially compressed.
https://github.com/restic/restic/pull/3915

View File

@@ -1,6 +0,0 @@
Enhancement: Provide command completion for PowerShell
Restic already provided generation of completion files for bash, fish and zsh.
Now powershell is supported, too.
https://github.com/restic/restic/pull/3925/files

View File

@@ -1,10 +0,0 @@
Enhancement: Allow `backup` file tree scanner to be disabled
The `backup` command walks the file tree in a separate scanner process to find
the total size and file/directory count, and uses this to provide an ETA. This
can slow down backups, especially of network filesystems.
The command now has a new option `--no-scan` which can be used to disable this
scanning in order to speed up backups when needed.
https://github.com/restic/restic/pull/3931

View File

@@ -1,9 +0,0 @@
Enhancement: Ignore additional/unknown files in repository
If a restic repository had additional files in it (not created by restic),
commands like `find` and `restore` could become confused and fail with an
`multiple IDs with prefix "12345678" found` error. These commands now
ignore such additional files.
https://github.com/restic/restic/pull/3943
https://forum.restic.net/t/which-protocol-should-i-choose-for-remote-linux-backups/5446/17

View File

@@ -1,7 +0,0 @@
Bugfix: Make `ls` return exit code 1 if snapshot cannot be loaded
The `ls` command used to show a warning and return exit code 0 when failing
to load a snapshot. This has now been fixed such that it instead returns exit
code 1 (still showing a warning).
https://github.com/restic/restic/pull/3951

View File

@@ -1,9 +0,0 @@
Enhancement: Improve `backup` performance for small files
When backing up small files restic was slower than it could be. In particular
this affected backups using maximum compression.
This has been fixed by reworking the internal parallelism of the backup
command, making it back up small files around two times faster.
https://github.com/restic/restic/pull/3955

View File

@@ -1,7 +0,0 @@
Change: Update dependencies and require Go 1.18 or newer
Most dependencies have been updated. Since some libraries require newer language
features, support for Go 1.15-1.17 has been dropped, which means that restic now
requires at least Go 1.18 to build.
https://github.com/restic/restic/pull/4041

View File

@@ -1,11 +0,0 @@
Bugfix: Make `self-update` enabled by default only in release builds
The `self-update` command was previously included by default in all builds of
restic as opposed to only in official release builds, even if the `selfupdate`
tag was not explicitly enabled when building.
This has now been corrected, and the `self-update` command is only available
if restic was built with `-tags selfupdate` (as done for official release
builds by `build.go`).
https://github.com/restic/restic/pull/4100

View File

@@ -1,10 +0,0 @@
Bugfix: Remove `b2_download_file_by_name: 404` warning from B2 backend
In some cases the B2 backend could print `b2_download_file_by_name: 404: :
b2.b2err` warnings. These are only debug messages and can be safely ignored.
Restic now uses an updated library for accessing B2, which removes the warning.
https://github.com/restic/restic/issues/3750
https://github.com/restic/restic/issues/4144
https://github.com/restic/restic/pull/4146

View File

@@ -1,7 +0,0 @@
Bugfix: Make `prune --quiet` not print progress bar
A regression in restic 0.15.0 caused `prune --quiet` to show a progress bar
while deciding how to process each pack files. This has now been fixed.
https://github.com/restic/restic/issues/4147
https://github.com/restic/restic/pull/4153

View File

@@ -1,19 +0,0 @@
Enhancement: Ignore empty lock files
With restic 0.15.0 the checks for stale locks became much stricter than before.
In particular, empty or unreadable locks were no longer silently ignored. This
made restic to complain with `Load(<lock/1234567812>, 0, 0) returned error,
retrying after 552.330144ms: load(<lock/1234567812>): invalid data returned`
and fail in the end.
The error message is now clarified and the implementation changed to ignore
empty lock files which are sometimes created as the result of a failed uploads
on some backends.
Please note that unreadable lock files still have to cleaned up manually. To do
so, you can run `restic unlock --remove-all` which removes all existing lock
files. But first make sure that no other restic process is currently using the
repository.
https://github.com/restic/restic/issues/4143
https://github.com/restic/restic/pull/4152

View File

@@ -1,13 +0,0 @@
Bugfix: Make `self-update --output` work with new filename on Windows
Since restic 0.14.0 the `self-update` command did not work when a custom output
filename was specified via the `--output` option. This has now been fixed.
As a workaround, either use an older restic version to run the self-update or
create an empty file with the output filename before updating e.g. using CMD:
`type nul > new-file.exe`
`restic self-update --output new-file.exe`
https://github.com/restic/restic/pull/4163
https://forum.restic.net/t/self-update-windows-started-failing-after-release-of-0-15/5836

View File

@@ -1,6 +0,0 @@
Bugfix: Add missing ETA in `backup` progress bar
A regression in restic 0.15.0 caused the ETA to be missing from the progress
bar displayed by the `backup` command. This has now been fixed.
https://github.com/restic/restic/pull/4167

Some files were not shown because too many files have changed in this diff Show More