Compare commits

..

4 Commits

Author SHA1 Message Date
Alexander Neumann
0912a8db07 Update Gopkg.lock 2018-01-15 12:37:42 +01:00
Alexander Neumann
eefeb387d9 WIP: WebDAV server 2018-01-15 11:44:06 +01:00
Alexander Neumann
c7d789ab04 Add entry to changelog 2018-01-14 14:24:14 +01:00
Alexander Neumann
92918ef1b6 fuse/mount: Add option for snapshot template 2018-01-14 14:22:08 +01:00
6268 changed files with 3481336 additions and 146575 deletions

View File

@@ -1,27 +1,63 @@
<!--
NOTE: Not filling out the issue template needs a good reason, otherwise it may
take a lot longer to find the problem! Please take the time to help us
debugging the problem by collecting information, even if it seems irrelevant to
you. Thanks!
Welcome! If you have a question or are unsure if you should open an issue,
please use the forum instead!
If you have a question, the forum at https://discourse.restic.net is a better place.
Please do not create issues for usage or documentation questions! We're using
the GitHub issue tracker mainly for tracking bugs and feature requests.
-->
https://forum.restic.net
## Output of `restic version`
The forum is a better place for questions about restic or general suggestions
and topics, e.g. usage or documentation questions! This issue tracker is mainly
for tracking bugs and feature requests directly relating to the development of
the software itself, rather than the project.
Thanks for understanding, and for contributing to the project!
## How did you run restic exactly?
<!--
This section should include at least:
* The complete command line and any environment variables you used to
configure restic's backend access. Make sure to replace sensitive values!
* The output of the commands, what restic prints gives may give us much
information to diagnose the problem!
-->
Output of `restic version`
--------------------------
## What backend/server/service did you use to store the repository?
## Expected behavior
<!--
Please add the version of restic you're currently using here, this helps us
later to see what has changed in restic when we revisit this issue after some
time.
Describe what you'd like restic to do differently.
-->
Describe the issue
------------------
## Actual behavior
<!--
In this section, please try to concentrate on observations, so only describe
what you observed directly.
-->
## Steps to reproduce the behavior
<!--
The more time you spend describing an easy way to reproduce the behavior (if
this is possible), the easier it is for the project developers to fix it!
-->
## Do you have any idea what may have caused this?
## Do you have an idea how to solve the issue?
## Did restic help you or made you happy in any way?
<!--
Answering this question is not required, but if you have anything positive to share, please do so here!
Sometimes we get tired of reading bug reports all day and a little positive end note does wonders.
Idea by Joey Hess, https://joeyh.name/blog/entry/two_holiday_stories/
-->

View File

@@ -1,93 +0,0 @@
---
name: Bug report
about: Report a problem with restic to help us resolve it and improve
---
<!--
Welcome! - We kindly ask that you:
1. Fill out the issue template below - not doing so needs a good reason.
2. Use the forum if you have a question rather than a bug or feature request.
The forum is at: https://forum.restic.net
NOTE: Not filling out the issue template needs a good reason, as otherwise it
may take a lot longer to find the problem, not to mention it can take up a lot
more time which can otherwise be spent on development. Please also take the
time to help us debug the issue by collecting relevant information, even if
it doesn't seem to be relevant to you. Thanks!
The forum is a better place for questions about restic or general suggestions
and topics, e.g. usage or documentation questions! This issue tracker is mainly
for tracking bugs and feature requests directly relating to the development of
the software itself, rather than the project.
Thanks for understanding, and for contributing to the project!
-->
Output of `restic version`
--------------------------
How did you run restic exactly?
-------------------------------
<!--
This section should include at least:
* The complete command line and any environment variables you used to
configure restic's backend access. Make sure to replace sensitive values!
* The output of the commands, what restic prints gives may give us much
information to diagnose the problem!
-->
What backend/server/service did you use to store the repository?
----------------------------------------------------------------
Expected behavior
-----------------
<!--
Describe what you'd like restic to do differently.
-->
Actual behavior
---------------
<!--
In this section, please try to concentrate on observations, so only describe
what you observed directly.
-->
Steps to reproduce the behavior
-------------------------------
<!--
The more time you spend describing an easy way to reproduce the behavior (if
this is possible), the easier it is for the project developers to fix it!
-->
Do you have any idea what may have caused this?
-----------------------------------------------
Do you have an idea how to solve the issue?
-------------------------------------------
Did restic help you or made you happy in any way?
-------------------------------------------------
<!--
Answering this question is not required, but if you have anything positive to share, please do so here!
Sometimes we get tired of reading bug reports all day and a little positive end note does wonders.
Idea by Joey Hess, https://joeyh.name/blog/entry/two_holiday_stories/
-->

View File

@@ -1,57 +0,0 @@
---
name: Feature request
about: Suggest a new feature or enhancement for restic
---
<!--
Welcome! - We kindly ask that you:
1. Fill out the issue template below - not doing so needs a good reason.
2. Use the forum if you have a question rather than a bug or feature request.
The forum is at: https://forum.restic.net
The forum is a better place for questions about restic or general suggestions
and topics, e.g. usage or documentation questions! This issue tracker is mainly
for tracking bugs and feature requests directly relating to the development of
the software itself, rather than the project.
Thanks for understanding, and for contributing to the project!
-->
Output of `restic version`
--------------------------
<!--
Please add the version of restic you're currently using here, this helps us
later to see what has changed in restic when we revisit this issue after some
time.
-->
What should restic do differently? Which functionality do you think we should add?
----------------------------------------------------------------------------------
<!--
Please describe the feature you'd like us to add here.
-->
What are you trying to do?
--------------------------
<!--
This section should contain a brief description what you're trying to do, which
would be possible after implementing the new feature.
-->
Did restic help you or made you happy in any way?
-------------------------------------------------
<!--
Answering this question is not required, but if you have anything positive to share, please do so here!
Sometimes we get tired of reading bug reports all day and a little positive end note does wonders.
Idea by Joey Hess, https://joeyh.name/blog/entry/two_holiday_stories/
-->

View File

@@ -1,5 +1,3 @@
<!--
Thank you very much for contributing code or documentation to restic! Please
fill out the following questions to make it easier for us to review your
@@ -10,27 +8,24 @@ your time and add more commits. If you're done and ready for review, please
check the last box.
-->
What is the purpose of this change? What does it change?
--------------------------------------------------------
### What is the purpose of this change? What does it change?
<!--
Describe the changes here, as detailed as needed.
-->
Was the change discussed in an issue or in the forum before?
------------------------------------------------------------
### Was the change discussed in an issue or in the forum before?
<!--
Link issues and relevant forum posts here.
-->
Checklist
---------
### Checklist
- [ ] I have read the [Contribution Guidelines](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#providing-patches)
- [ ] I have added tests for all changes in this PR
- [ ] I have added documentation for the changes (in the manual)
- [ ] There's a new file in `changelog/unreleased/` that describes the changes for our users (template [here](https://github.com/restic/restic/blob/master/changelog/TEMPLATE))
- [ ] There's a new file in a subdir of `changelog/x.y.z` that describe the changes for our users (template [here](https://github.com/restic/restic/blob/master/changelog/changelog-entry.tmpl))
- [ ] I have run `gofmt` on the code in all commits
- [ ] All commit messages are formatted in the same style as [the other commits in the repo](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#git-commits)
- [ ] I'm done, this Pull Request is ready for review

1
.gitignore vendored
View File

@@ -1,2 +1,3 @@
/restic
/.vagrant
/doc/_build

View File

@@ -1,40 +1,30 @@
language: go
sudo: false
go:
- 1.8.x
- 1.9.x
os:
- linux
- osx
env:
matrix:
RESTIC_TEST_FUSE=0
matrix:
exclude:
- os: osx
go: 1.8.x
- os: linux
go: 1.9.x
include:
- os: linux
go: "1.9.x"
env: RESTIC_TEST_FUSE=0 RESTIC_TEST_CLOUD_BACKENDS=0 RESTIC_BUILD_SOLARIS=0
cache:
directories:
- $HOME/.cache/go-build
- $HOME/gopath/pkg/mod
- os: linux
go: "1.10.x"
env: RESTIC_TEST_FUSE=0 RESTIC_TEST_CLOUD_BACKENDS=0
cache:
directories:
- $HOME/.cache/go-build
- $HOME/gopath/pkg/mod
# only run fuse and cloud backends tests on Travis for the latest Go on Linux
- os: linux
go: "1.11.x"
go: 1.9.x
sudo: true
cache:
directories:
- $HOME/.cache/go-build
- $HOME/gopath/pkg/mod
- os: osx
go: "1.11.x"
env: RESTIC_TEST_FUSE=0 RESTIC_TEST_CLOUD_BACKENDS=0
cache:
directories:
- $HOME/Library/Caches/go-build
- $HOME/gopath/pkg/mod
env:
RESTIC_TEST_FUSE=1
branches:
only:
@@ -58,4 +48,4 @@ script:
- go run run_integration_tests.go
after_success:
- test -r all.cov && bash <(curl -s https://codecov.io/bash) -f all.cov
- bash <(curl -s https://codecov.io/bash) -f all.cov

View File

@@ -1,863 +1,3 @@
Changelog for restic 0.9.3 (2018-10-13)
=======================================
The following sections list the changes in restic 0.9.3 relevant to
restic users. The changes are ordered by importance.
Summary
-------
* Fix #1935: Remove truncated files from cache
* Fix #1978: Do not return an error when the scanner is faster than backup
* Enh #1766: Restore: suppress lchown errors when not running as root
* Enh #1909: Reject files/dirs by name first
* Enh #1940: Add directory filter to ls command
* Enh #1967: Use `--host` everywhere
* Enh #2028: Display size of cache directories
* Enh #1777: Improve the `find` command
* Enh #1876: Display reason why forget keeps snapshots
* Enh #1891: Accept glob in paths loaded via --files-from
* Enh #1920: Vendor dependencies with Go 1.11 Modules
* Enh #1949: Add new command `self-update`
* Enh #1953: Ls: Add JSON output support for restic ls cmd
Details
-------
* Bugfix #1935: Remove truncated files from cache
When a file in the local cache is truncated, and restic tries to access data beyond the end of the
(cached) file, it used to return an error "EOF". This is now fixed, such truncated files are
removed and the data is fetched directly from the backend.
https://github.com/restic/restic/issues/1935
* Bugfix #1978: Do not return an error when the scanner is faster than backup
When restic makes a backup, there's a background task called "scanner" which collects
information on how many files and directories are to be saved, in order to display progress
information to the user. When the backup finishes faster than the scanner, it is aborted
because the result is not needed any more. This logic contained a bug, where quitting the
scanner process was treated as an error, and caused restic to print an unhelpful error message
("context canceled").
https://github.com/restic/restic/issues/1978
https://github.com/restic/restic/pull/1991
* Enhancement #1766: Restore: suppress lchown errors when not running as root
Like "cp" and "rsync" do, restic now only reports errors for changing the ownership of files
during restore if it is run as root, on non-Windows operating systems. On Windows, the error
is reported as usual.
https://github.com/restic/restic/issues/1766
* Enhancement #1909: Reject files/dirs by name first
The current scanner/archiver code had an architectural limitation: it always ran the
`lstat()` system call on all files and directories before a decision to include/exclude the
file/dir was made. This lead to a lot of unnecessary system calls for items that could have been
rejected by their name or path only.
We've changed the archiver/scanner implementation so that it now first rejects by name/path,
and only runs the system call on the remaining items. This reduces the number of `lstat()`
system calls a lot (depending on the exclude settings).
https://github.com/restic/restic/issues/1909
https://github.com/restic/restic/pull/1912
* Enhancement #1940: Add directory filter to ls command
The ls command can now be filtered by directories, so that only files in the given directories
will be shown. If the --recursive flag is specified, then ls will traverse subfolders and list
their files as well.
It used to be possible to specify multiple snapshots, but that has been replaced by only one
snapshot and the possibility of specifying multiple directories.
Specifying directories constrains the walk, which can significantly speed up the listing.
https://github.com/restic/restic/issues/1940
https://github.com/restic/restic/pull/1941
* Enhancement #1967: Use `--host` everywhere
We now use the flag `--host` for all commands which need a host name, using `--hostname` (e.g.
for `restic backup`) still works, but will print a deprecation warning. Also, add the short
option `-H` where possible.
https://github.com/restic/restic/issues/1967
* Enhancement #2028: Display size of cache directories
The `cache` command now by default shows the size of the individual cache directories. It can be
disabled with `--no-size`.
https://github.com/restic/restic/issues/2028
https://github.com/restic/restic/pull/2033
* Enhancement #1777: Improve the `find` command
We've updated the `find` command to support multiple patterns.
`restic find` is now able to list the snapshots containing a specific tree or blob, or even the
snapshots that contain blobs belonging to a given pack. A list of IDs can be given, as long as they
all have the same type.
The command `find` can also display the pack IDs the blobs belong to, if the `--show-pack-id`
flag is provided.
https://github.com/restic/restic/issues/1777
https://github.com/restic/restic/pull/1780
* Enhancement #1876: Display reason why forget keeps snapshots
We've added a column to the list of snapshots `forget` keeps which details the reasons to keep a
particuliar snapshot. This makes debugging policies for forget much easier. Please remember
to always try things out with `--dry-run`!
https://github.com/restic/restic/pull/1876
* Enhancement #1891: Accept glob in paths loaded via --files-from
Before that, behaviour was different if paths were appended to command line or from a file,
because wild card characters were expanded by shell if appended to command line, but not
expanded if loaded from file.
https://github.com/restic/restic/issues/1891
* Enhancement #1920: Vendor dependencies with Go 1.11 Modules
Until now, we've used `dep` for managing dependencies, we've now switch to using Go modules.
For users this does not change much, only if you want to compile restic without downloading
anything with Go 1.11, then you need to run: `go build -mod=vendor build.go`
https://github.com/restic/restic/pull/1920
* Enhancement #1949: Add new command `self-update`
We have added a new command called `self-update` which downloads the latest released version
of restic from GitHub and replaces the current binary with it. It does not rely on any external
program (so it'll work everywhere), but still verifies the GPG signature using the embedded
GPG public key.
By default, the `self-update` command is hidden behind the `selfupdate` built tag, which is
only set when restic is built using `build.go` (including official releases). The reason for
this is that downstream distributions will then not include the command by default, so users
are encouraged to use the platform-specific distribution mechanism.
https://github.com/restic/restic/pull/1949
* Enhancement #1953: Ls: Add JSON output support for restic ls cmd
We've implemented listing files in the repository with JSON as output, just pass `--json` as an
option to `restic ls`. This makes the output of the command machine readable.
https://github.com/restic/restic/pull/1953
Changelog for restic 0.9.2 (2018-08-06)
=======================================
The following sections list the changes in restic 0.9.2 relevant to
restic users. The changes are ordered by importance.
Summary
-------
* Fix #1854: Allow saving files/dirs on different fs with `--one-file-system`
* Fix #1870: Fix restore with --include
* Fix #1880: Use `--cache-dir` argument for `check` command
* Fix #1893: Return error when exclude file cannot be read
* Fix #1861: Fix case-insensitive search with restic find
* Enh #1906: Add support for B2 application keys
* Enh #874: Add stats command to get information about a repository
* Enh #1772: Add restore --verify to verify restored file content
* Enh #1853: Add JSON output support to `restic key list`
* Enh #1477: S3 backend: accept AWS_SESSION_TOKEN
* Enh #1901: Update the Backblaze B2 library
Details
-------
* Bugfix #1854: Allow saving files/dirs on different fs with `--one-file-system`
Restic now allows saving files/dirs on a different file system in a subdir correctly even when
`--one-file-system` is specified.
The first thing the restic archiver code does is to build a tree of the target
files/directories. If it detects that a parent directory is already included (e.g. `restic
backup /foo /foo/bar/baz`), it'll ignore the latter argument.
Without `--one-file-system`, that's perfectly valid: If `/foo` is to be archived, it will
include `/foo/bar/baz`. But with `--one-file-system`, `/foo/bar/baz` may reside on a
different file system, so it won't be included with `/foo`.
https://github.com/restic/restic/issues/1854
https://github.com/restic/restic/pull/1855
* Bugfix #1870: Fix restore with --include
We fixed a bug which prevented restic to restore files with an include filter.
https://github.com/restic/restic/issues/1870
https://github.com/restic/restic/pull/1900
* Bugfix #1880: Use `--cache-dir` argument for `check` command
`check` command now uses a temporary sub-directory of the specified directory if set using the
`--cache-dir` argument. If not set, the cache directory is created in the default temporary
directory as before. In either case a temporary cache is used to ensure the actual repository is
checked (rather than a local copy).
The `--cache-dir` argument was not used by the `check` command, instead a cache directory was
created in the temporary directory.
https://github.com/restic/restic/issues/1880
* Bugfix #1893: Return error when exclude file cannot be read
A bug was found: when multiple exclude files were passed to restic and one of them could not be
read, an error was printed and restic continued, ignoring even the existing exclude files.
Now, an error message is printed and restic aborts when an exclude file cannot be read.
https://github.com/restic/restic/issues/1893
* Bugfix #1861: Fix case-insensitive search with restic find
We've fixed the behavior for `restic find -i PATTERN`, which was broken in v0.9.1.
https://github.com/restic/restic/pull/1861
* Enhancement #1906: Add support for B2 application keys
Restic can now use so-called "application keys" which can be created in the B2 dashboard and
were only introduced recently. In contrast to the "master key", such keys can be restricted to a
specific bucket and/or path.
https://github.com/restic/restic/issues/1906
https://github.com/restic/restic/pull/1914
* Enhancement #874: Add stats command to get information about a repository
https://github.com/restic/restic/issues/874
https://github.com/restic/restic/pull/1729
* Enhancement #1772: Add restore --verify to verify restored file content
Restore will print error message if restored file content does not match expected SHA256
checksum
https://github.com/restic/restic/pull/1772
* Enhancement #1853: Add JSON output support to `restic key list`
This PR enables users to get the output of `restic key list` in JSON in addition to the existing
table format.
https://github.com/restic/restic/pull/1853
* Enhancement #1477: S3 backend: accept AWS_SESSION_TOKEN
Before, it was not possible to use s3 backend with AWS temporary security credentials(with
AWS_SESSION_TOKEN). This change gives higher priority to credentials.EnvAWS credentials
provider.
https://github.com/restic/restic/issues/1477
https://github.com/restic/restic/pull/1479
https://github.com/restic/restic/pull/1647
* Enhancement #1901: Update the Backblaze B2 library
We've updated the library we're using for accessing the Backblaze B2 service to 0.5.0 to
include support for upcoming so-called "application keys". With this feature, you can create
access credentials for B2 which are restricted to e.g. a single bucket or even a sub-directory
of a bucket.
https://github.com/restic/restic/pull/1901
https://github.com/kurin/blazer
Changelog for restic 0.9.1 (2018-06-10)
=======================================
The following sections list the changes in restic 0.9.1 relevant to
restic users. The changes are ordered by importance.
Summary
-------
* Fix #1801: Add limiting bandwidth to the rclone backend
* Fix #1822: Allow uploading large files to MS Azure
* Fix #1825: Correct `find` to not skip snapshots
* Fix #1833: Fix caching files on error
* Fix #1834: Resolve deadlock
Details
-------
* Bugfix #1801: Add limiting bandwidth to the rclone backend
The rclone backend did not respect `--limit-upload` or `--limit-download`. Oftentimes it's
not necessary to use this, as the limiting in rclone itself should be used because it gives much
better results, but in case a remote instance of rclone is used (e.g. called via ssh), it is still
relevant to limit the bandwidth from restic to rclone.
https://github.com/restic/restic/issues/1801
* Bugfix #1822: Allow uploading large files to MS Azure
Sometimes, restic creates files to be uploaded to the repository which are quite large, e.g.
when saving directories with many entries or very large files. The MS Azure API does not allow
uploading files larger that 256MiB directly, rather restic needs to upload them in blocks of
100MiB. This is now implemented.
https://github.com/restic/restic/issues/1822
* Bugfix #1825: Correct `find` to not skip snapshots
Under certain circumstances, the `find` command was found to skip snapshots containing
directories with files to look for when the directories haven't been modified at all, and were
already printed as part of a different snapshot. This is now corrected.
In addition, we've switched to our own matching/pattern implementation, so now things like
`restic find "/home/user/foo/**/main.go"` are possible.
https://github.com/restic/restic/issues/1825
https://github.com/restic/restic/issues/1823
* Bugfix #1833: Fix caching files on error
During `check` it may happen that different threads access the same file in the backend, which
is then downloaded into the cache only once. When that fails, only the thread which is
responsible for downloading the file signals the correct error. The other threads just assume
that the file has been downloaded successfully and then get an error when they try to access the
cached file.
https://github.com/restic/restic/issues/1833
* Bugfix #1834: Resolve deadlock
When the "scanning" process restic runs to find out how much data there is does not finish before
the backup itself is done, restic stops doing anything. This is resolved now.
https://github.com/restic/restic/issues/1834
https://github.com/restic/restic/pull/1835
Changelog for restic 0.9.0 (2018-05-21)
=======================================
The following sections list the changes in restic 0.9.0 relevant to
restic users. The changes are ordered by importance.
Summary
-------
* Fix #1608: Respect time stamp for new backup when reading from stdin
* Fix #1652: Ignore/remove invalid lock files
* Fix #1730: Ignore sockets for restore
* Fix #1684: Fix backend tests for rest-server
* Fix #1745: Correctly parse the argument to --tls-client-cert
* Enh #1433: Support UTF-16 encoding and process Byte Order Mark
* Enh #1561: Allow using rclone to access other services
* Enh #1665: Improve cache handling for `restic check`
* Enh #1721: Add `cache` command to list cache dirs
* Enh #1758: Allow saving OneDrive folders in Windows
* Enh #549: Rework archiver code
* Enh #1552: Use Google Application Default credentials
* Enh #1477: Accept AWS_SESSION_TOKEN for the s3 backend
* Enh #1648: Ignore AWS permission denied error when creating a repository
* Enh #1649: Add illumos/Solaris support
* Enh #1709: Improve messages `restic check` prints
* Enh #827: Add --new-password-file flag for non-interactive password changes
* Enh #1735: Allow keeping a time range of snaphots
* Enh #1782: Use default AWS credentials chain for S3 backend
Details
-------
* Bugfix #1608: Respect time stamp for new backup when reading from stdin
When reading backups from stdin (via `restic backup --stdin`), restic now uses the time stamp
for the new backup passed in `--time`.
https://github.com/restic/restic/issues/1608
https://github.com/restic/restic/pull/1703
* Bugfix #1652: Ignore/remove invalid lock files
This corrects a bug introduced recently: When an invalid lock file in the repo is encountered
(e.g. if the file is empty), the code used to ignore that, but now returns the error. Now, invalid
files are ignored for the normal lock check, and removed when `restic unlock --remove-all` is
run.
https://github.com/restic/restic/issues/1652
https://github.com/restic/restic/pull/1653
* Bugfix #1730: Ignore sockets for restore
We've received a report and correct the behavior in which the restore code aborted restoring a
directory when a socket was encountered. Unix domain socket files cannot be restored (they are
created on the fly once a process starts listening). The error handling was corrected, and in
addition we're now ignoring sockets during restore.
https://github.com/restic/restic/issues/1730
https://github.com/restic/restic/pull/1731
* Bugfix #1684: Fix backend tests for rest-server
The REST server for restic now requires an explicit parameter (`--no-auth`) if no
authentication should be allowed. This is fixed in the tests.
https://github.com/restic/restic/pull/1684
* Bugfix #1745: Correctly parse the argument to --tls-client-cert
Previously, the --tls-client-cert method attempt to read ARGV[1] (hardcoded) instead of the
argument that was passed to it. This has been corrected.
https://github.com/restic/restic/issues/1745
https://github.com/restic/restic/pull/1746
* Enhancement #1433: Support UTF-16 encoding and process Byte Order Mark
On Windows, text editors commonly leave a Byte Order Mark at the beginning of the file to define
which encoding is used (oftentimes UTF-16). We've added code to support processing the BOMs in
text files, like the exclude files, the password file and the file passed via `--files-from`.
This does not apply to any file being saved in a backup, those are not touched and archived as they
are.
https://github.com/restic/restic/issues/1433
https://github.com/restic/restic/issues/1738
https://github.com/restic/restic/pull/1748
* Enhancement #1561: Allow using rclone to access other services
We've added the ability to use rclone to store backup data on all backends that it supports. This
was done in collaboration with Nick, the author of rclone. You can now use it to first configure a
service, then restic manages the rest (starting and stopping rclone). For details, please see
the manual.
https://github.com/restic/restic/issues/1561
https://github.com/restic/restic/pull/1657
https://rclone.org
* Enhancement #1665: Improve cache handling for `restic check`
For safety reasons, restic does not use a local metadata cache for the `restic check` command,
so that data is loaded from the repository and restic can check it's in good condition. When the
cache is disabled, restic will fetch each tiny blob needed for checking the integrity using a
separate backend request. For non-local backends, that will take a long time, and depending on
the backend (e.g. B2) may also be much more expensive.
This PR adds a few commits which will change the behavior as follows:
* When `restic check` is called without any additional parameters, it will build a new cache in a
temporary directory, which is removed at the end of the check. This way, we'll get readahead for
metadata files (so restic will fetch the whole file when the first blob from the file is
requested), but all data is freshly fetched from the storage backend. This is the default
behavior and will work for almost all users.
* When `restic check` is called with `--with-cache`, the default on-disc cache is used. This
behavior hasn't changed since the cache was introduced.
* When `--no-cache` is specified, restic falls back to the old behavior, and read all tiny blobs
in separate requests.
https://github.com/restic/restic/issues/1665
https://github.com/restic/restic/issues/1694
https://github.com/restic/restic/pull/1696
* Enhancement #1721: Add `cache` command to list cache dirs
The command `cache` was added, it allows listing restic's cache directoriers together with
the last usage. It also allows removing old cache dirs without having to access a repo, via
`restic cache --cleanup`
https://github.com/restic/restic/issues/1721
https://github.com/restic/restic/pull/1749
* Enhancement #1758: Allow saving OneDrive folders in Windows
Restic now contains a bugfix to two libraries, which allows saving OneDrive folders in
Windows. In order to use the newer versions of the libraries, the minimal version required to
compile restic is now Go 1.9.
https://github.com/restic/restic/issues/1758
https://github.com/restic/restic/pull/1765
* Enhancement #549: Rework archiver code
The core archiver code and the complementary code for the `backup` command was rewritten
completely. This resolves very annoying issues such as 549. The first backup with this release
of restic will likely result in all files being re-read locally, so it will take a lot longer. The
next backup after that will be fast again.
Basically, with the old code, restic took the last path component of each to-be-saved file or
directory as the top-level file/directory within the snapshot. This meant that when called as
`restic backup /home/user/foo`, the snapshot would contain the files in the directory
`/home/user/foo` as `/foo`.
This is not the case any more with the new archiver code. Now, restic works very similar to what
`tar` does: When restic is called with an absolute path to save, then it'll preserve the
directory structure within the snapshot. For the example above, the snapshot would contain
the files in the directory within `/home/user/foo` in the snapshot. For relative
directories, it only preserves the relative path components. So `restic backup user/foo`
will save the files as `/user/foo` in the snapshot.
While we were at it, the status display and notification system was completely rewritten. By
default, restic now shows which files are currently read (unless `--quiet` is specified) in a
multi-line status display.
The `backup` command also gained a new option: `--verbose`. It can be specified once (which
prints a bit more detail what restic is doing) or twice (which prints a line for each
file/directory restic encountered, together with some statistics).
Another issue that was resolved is the new code only reads two files at most. The old code would
read way too many files in parallel, thereby slowing down the backup process on spinning discs a
lot.
https://github.com/restic/restic/issues/549
https://github.com/restic/restic/issues/1286
https://github.com/restic/restic/issues/446
https://github.com/restic/restic/issues/1344
https://github.com/restic/restic/issues/1416
https://github.com/restic/restic/issues/1456
https://github.com/restic/restic/issues/1145
https://github.com/restic/restic/issues/1160
https://github.com/restic/restic/pull/1494
* Enhancement #1552: Use Google Application Default credentials
Google provide libraries to generate appropriate credentials with various fallback
sources. This change uses the library to generate our GCS client, which allows us to make use of
these extra methods.
This should be backward compatible with previous restic behaviour while adding the
additional capabilities to auth from Google's internal metadata endpoints. For users
running restic in GCP this can make authentication far easier than it was before.
https://github.com/restic/restic/pull/1552
https://developers.google.com/identity/protocols/application-default-credentials
* Enhancement #1477: Accept AWS_SESSION_TOKEN for the s3 backend
Before, it was not possible to use s3 backend with AWS temporary security credentials(with
AWS_SESSION_TOKEN). This change gives higher priority to credentials.EnvAWS credentials
provider.
https://github.com/restic/restic/issues/1477
https://github.com/restic/restic/pull/1479
https://github.com/restic/restic/pull/1647
* Enhancement #1648: Ignore AWS permission denied error when creating a repository
It's not possible to use s3 backend scoped to a subdirectory(with specific permissions).
Restic doesn't try to create repository in a subdirectory, when 'bucket exists' of parent
directory check fails due to permission issues.
https://github.com/restic/restic/pull/1648
* Enhancement #1649: Add illumos/Solaris support
https://github.com/restic/restic/pull/1649
* Enhancement #1709: Improve messages `restic check` prints
Some messages `restic check` prints are not really errors, so from now on restic does not treat
them as errors any more and exits cleanly.
https://github.com/restic/restic/pull/1709
https://forum.restic.net/t/what-is-the-standard-procedure-to-follow-if-a-backup-or-restore-is-interrupted/571/2
* Enhancement #827: Add --new-password-file flag for non-interactive password changes
This makes it possible to change a repository password without being prompted.
https://github.com/restic/restic/issues/827
https://github.com/restic/restic/pull/1720
https://forum.restic.net/t/changing-repo-password-without-prompt/591
* Enhancement #1735: Allow keeping a time range of snaphots
We've added the `--keep-within` option to the `forget` command. It instructs restic to keep
all snapshots within the given duration since the newest snapshot. For example, running
`restic forget --keep-within 5m7d` will keep all snapshots which have been made in the five
months and seven days since the latest snapshot.
https://github.com/restic/restic/pull/1735
* Enhancement #1782: Use default AWS credentials chain for S3 backend
Adds support for file credentials to the S3 backend (e.g. ~/.aws/credentials), and reorders
the credentials chain for the S3 backend to match AWS's standard, which is static credentials,
env vars, credentials file, and finally remote.
https://github.com/restic/restic/pull/1782
Changelog for restic 0.8.3 (2018-02-26)
=======================================
The following sections list the changes in restic 0.8.3 relevant to
restic users. The changes are ordered by importance.
Summary
-------
* Fix #1633: Fixed unexpected 'pack file cannot be listed' error
* Fix #1641: Ignore files with invalid names in the repo
* Fix #1638: Handle errors listing files in the backend
* Enh #1497: Add --read-data-subset flag to check command
* Enh #1560: Retry all repository file download errors
* Enh #1623: Don't check for presence of files in the backend before writing
* Enh #1634: Upgrade B2 client library, reduce HTTP requests
Details
-------
* Bugfix #1633: Fixed unexpected 'pack file cannot be listed' error
Due to a regression introduced in 0.8.2, the `rebuild-index` and `prune` commands failed to
read pack files with size of 587, 588, 589 or 590 bytes.
https://github.com/restic/restic/issues/1633
https://github.com/restic/restic/pull/1635
* Bugfix #1641: Ignore files with invalid names in the repo
The release 0.8.2 introduced a bug: when restic encounters files in the repo which do not have a
valid name, it tries to load a file with a name of lots of zeroes instead of ignoring it. This is now
resolved, invalid file names are just ignored.
https://github.com/restic/restic/issues/1641
https://github.com/restic/restic/pull/1643
https://forum.restic.net/t/help-fixing-repo-no-such-file/485/3
* Bugfix #1638: Handle errors listing files in the backend
A user reported in the forum that restic completes a backup although a concurrent `prune`
operation was running. A few error messages were printed, but the backup was attempted and
completed successfully. No error code was returned.
This should not happen: The repository is exclusively locked during `prune`, so when `restic
backup` is run in parallel, it should abort and return an error code instead.
It was found that the bug was in the code introduced only recently, which retries a List()
operation on the backend should that fail. It is now corrected.
https://github.com/restic/restic/pull/1638
https://forum.restic.net/t/restic-backup-returns-0-exit-code-when-already-locked/484
* Enhancement #1497: Add --read-data-subset flag to check command
This change introduces ability to check integrity of a subset of repository data packs. This
can be used to spread integrity check of larger repositories over a period of time.
https://github.com/restic/restic/issues/1497
https://github.com/restic/restic/pull/1556
* Enhancement #1560: Retry all repository file download errors
Restic will now retry failed downloads, similar to other operations.
https://github.com/restic/restic/pull/1560
* Enhancement #1623: Don't check for presence of files in the backend before writing
Before, all backend implementations were required to return an error if the file that is to be
written already exists in the backend. For most backends, that means making a request (e.g. via
HTTP) and returning an error when the file already exists.
This is not accurate, the file could have been created between the HTTP request testing for it,
and when writing starts, so we've relaxed this requeriment, which saves one additional HTTP
request per newly added file.
https://github.com/restic/restic/pull/1623
* Enhancement #1634: Upgrade B2 client library, reduce HTTP requests
We've upgraded the B2 client library restic uses to access BackBlaze B2. This reduces the
number of HTTP requests needed to upload a new file from two to one, which should improve
throughput to B2.
https://github.com/restic/restic/pull/1634
Changelog for restic 0.8.2 (2018-02-17)
=======================================
The following sections list the changes in restic 0.8.2 relevant to
restic users. The changes are ordered by importance.
Summary
-------
* Fix #1506: Limit bandwith at the http.RoundTripper for HTTP based backends
* Fix #1512: Restore directory permissions as the last step
* Fix #1528: Correctly create missing subdirs in data/
* Fix #1590: Strip spaces for lines read via --files-from
* Fix #1589: Complete intermediate index upload
* Fix #1594: Google Cloud Storage: Use generic HTTP transport
* Fix #1595: Backup: Remove bandwidth display
* Enh #1522: Add support for TLS client certificate authentication
* Enh #1541: Reduce number of remote requests during repository check
* Enh #1567: Reduce number of backend requests for rebuild-index and prune
* Enh #1507: Only reload snapshots once per minute for fuse mount
* Enh #1538: Reduce memory allocations for querying the index
* Enh #1549: Speed up querying across indices and scanning existing files
* Enh #1554: Fuse/mount: Correctly handle EOF, add template option
* Enh #1564: Don't terminate ssh on SIGINT
* Enh #1579: Retry Backend.List() in case of errors
* Enh #1584: Limit index file size
Details
-------
* Bugfix #1506: Limit bandwith at the http.RoundTripper for HTTP based backends
https://github.com/restic/restic/issues/1506
https://github.com/restic/restic/pull/1511
* Bugfix #1512: Restore directory permissions as the last step
This change allows restoring into directories that were not writable during backup. Before,
restic created the directory, set the read-only mode and then failed to create files in the
directory. This change now restores the directory (with its permissions) as the very last
step.
https://github.com/restic/restic/issues/1512
https://github.com/restic/restic/pull/1536
* Bugfix #1528: Correctly create missing subdirs in data/
https://github.com/restic/restic/issues/1528
https://github.com/restic/restic/pull/1529
* Bugfix #1590: Strip spaces for lines read via --files-from
Leading and trailing spaces in lines read via `--files-from` are now stripped, so it behaves
the same as with lines read via `--exclude-file`.
https://github.com/restic/restic/issues/1590
https://github.com/restic/restic/pull/1613
* Bugfix #1589: Complete intermediate index upload
After a user posted a comprehensive report of what he observed, we were able to find a bug and
correct it: During backup, restic uploads so-called "intermediate" index files. When the
backup finishes during a transfer of such an intermediate index, the upload is cancelled, but
the backup is finished without an error. This leads to an inconsistent state, where the
snapshot references data that is contained in the repo, but is not referenced in any index.
The situation can be resolved by building a new index with `rebuild-index`, but looks very
confusing at first. Since all the data got uploaded to the repo successfully, there was no risk
of data loss, just minor inconvenience for our users.
https://github.com/restic/restic/pull/1589
https://forum.restic.net/t/error-loading-tree-check-prune-and-forget-gives-error-b2-backend/406
* Bugfix #1594: Google Cloud Storage: Use generic HTTP transport
It was discovered that the Google Cloud Storage backend did not use the generic HTTP transport,
so things such as bandwidth limiting with `--limit-upload` did not work. This is resolved now.
https://github.com/restic/restic/pull/1594
* Bugfix #1595: Backup: Remove bandwidth display
This commit removes the bandwidth displayed during backup process. It is misleading and
seldomly correct, because it's neither the "read bandwidth" (only for the very first backup)
nor the "upload bandwidth". Many users are confused about (and rightly so), c.f. #1581, #1033,
#1591
We'll eventually replace this display with something more relevant when the new archiver code
is ready.
https://github.com/restic/restic/pull/1595
* Enhancement #1522: Add support for TLS client certificate authentication
Support has been added for using a TLS client certificate for authentication to HTTP based
backend. A file containing the PEM encoded private key and certificate can be set using the
`--tls-client-cert` option.
https://github.com/restic/restic/issues/1522
https://github.com/restic/restic/pull/1524
* Enhancement #1541: Reduce number of remote requests during repository check
This change eliminates redundant remote repository calls and significantly improves
repository check time.
https://github.com/restic/restic/issues/1541
https://github.com/restic/restic/pull/1548
* Enhancement #1567: Reduce number of backend requests for rebuild-index and prune
We've found a way to reduce then number of backend requests for the `rebuild-index` and `prune`
operations. This significantly speeds up the operations for high-latency backends.
https://github.com/restic/restic/issues/1567
https://github.com/restic/restic/pull/1574
https://github.com/restic/restic/pull/1575
* Enhancement #1507: Only reload snapshots once per minute for fuse mount
https://github.com/restic/restic/pull/1507
* Enhancement #1538: Reduce memory allocations for querying the index
This change reduces the internal memory allocations when the index data structures in memory
are queried if a blob (part of a file) already exists in the repo. It should speed up backup a bit,
and maybe even reduce RAM usage.
https://github.com/restic/restic/pull/1538
* Enhancement #1549: Speed up querying across indices and scanning existing files
This change increases the whenever a blob (part of a file) is searched for in a restic
repository. This will reduce cpu usage some when backing up files already backed up by restic.
Cpu usage is further decreased when scanning files.
https://github.com/restic/restic/pull/1549
* Enhancement #1554: Fuse/mount: Correctly handle EOF, add template option
We've added the `--snapshot-template` string, which can be used to specify a template for a
snapshot directory. In addition, accessing data after the end of a file via the fuse mount is now
handled correctly.
https://github.com/restic/restic/pull/1554
* Enhancement #1564: Don't terminate ssh on SIGINT
We've reworked the code which runs the `ssh` login for the sftp backend so that it can prompt for a
password (if needed) but does not exit when the user presses CTRL+C (SIGINT) e.g. during
backup. This allows restic to properly shut down when it receives SIGINT and remove the lock
file from the repo, afterwards exiting the `ssh` process.
https://github.com/restic/restic/pull/1564
https://github.com/restic/restic/pull/1588
* Enhancement #1579: Retry Backend.List() in case of errors
https://github.com/restic/restic/pull/1579
* Enhancement #1584: Limit index file size
Before, restic would create a single new index file on `prune` or `rebuild-index`, this may
lead to memory problems when this huge index is created and loaded again. We're now limiting the
size of the index file, and split newly created index files into several smaller ones. This
allows restic to be more memory-efficient.
https://github.com/restic/restic/issues/1412
https://github.com/restic/restic/issues/979
https://github.com/restic/restic/issues/526
https://github.com/restic/restic/pull/1584
Changelog for restic 0.8.1 (2017-12-27)
=======================================
@@ -1187,7 +327,7 @@ Summary
* Enh #1061: Add Dockerfile and official Docker image
* Enh #1126: Use the standard Go git repository layout, use `dep` for vendoring
* Enh #1134: Add support for storing backups on Google Cloud Storage
* Enh #1144: Properly report errors when reading files with exclude patterns
* Enh #1144: Properly report errors when reading files with exclude patterns.
* Enh #1149: Add support for storing backups on Microsoft Azure Blob Storage
* Enh #1196: Add `--group-by` to `forget` command for flexible grouping
* Enh #1203: Print stats on all BSD systems when SIGINFO (ctrl+t) is received
@@ -1275,7 +415,7 @@ Details
https://github.com/restic/restic/pull/1134
https://github.com/restic/restic/pull/1052
* Enhancement #1144: Properly report errors when reading files with exclude patterns
* Enhancement #1144: Properly report errors when reading files with exclude patterns.
https://github.com/restic/restic/pull/1144
@@ -1295,7 +435,7 @@ Details
* Enhancement #1203: Print stats on all BSD systems when SIGINFO (ctrl+t) is received
https://github.com/restic/restic/pull/1203
https://github.com/restic/restic/pull/1082#issuecomment-326279920
https://github.com/restic/restic/pull/1082
* Enhancement #1205: Allow specifying time/date for a backup with `--time`
@@ -1335,12 +475,12 @@ Details
* Enhancement #1055: Create subdirs below `data/` for local/sftp backends
The local and sftp backends now create the subdirs below `data/` on open/init. This way, restic
makes sure that they always exist. This is connected to an issue for the sftp server.
makes sure that they always exist. This is connected to an issue for the sftp server:
https://github.com/restic/restic/issues/1055
https://github.com/restic/rest-server/pull/11#issuecomment-309879710
https://github.com/restic/restic/pull/1077
https://github.com/restic/restic/pull/1105
https://github.com/restic/rest-server/pull/11#issuecomment-309879710
* Enhancement #1067: Allow loading credentials for s3 from IAM
@@ -1352,7 +492,7 @@ Details
* Enhancement #1073: Add `migrate` cmd to migrate from `s3legacy` to `default` layout
The `migrate` command for changing the `s3legacy` layout to the `default` layout for s3
The `migrate` command for chaning the `s3legacy` layout to the `default` layout for s3
backends has been improved: It can now be restarted with `restic migrate --force s3_layout`
and automatically retries operations on error.

View File

@@ -46,15 +46,12 @@ Remember, the easier it is for us to reproduce the bug, the earlier it will be
corrected!
In addition, you can compile restic with debug support by running
`go run -mod=vendor build.go -tags debug` and instructing it to create a debug
log by setting the environment variable `DEBUG_LOG` to a file, e.g. like this:
`go run build.go -tags debug` and instructing it to create a debug log by
setting the environment variable `DEBUG_LOG` to a file, e.g. like this:
$ export DEBUG_LOG=/tmp/restic-debug.log
$ restic backup ~/work
For Go < 1.11, you need to remove the `-mod=vendor` option from the build
command.
Please be aware that the debug log file will contain potentially sensitive
things like file and directory names, so please either redact it before
uploading it somewhere or post only the parts that are really relevant.
@@ -63,37 +60,9 @@ uploading it somewhere or post only the parts that are really relevant.
Development Environment
=======================
The repository contains several sets of directories with code: `cmd/` and
`internal/` contain the code written for restic, whereas `vendor/` contains
copies of libraries restic depends on. The libraries are managed with the
command `go mod vendor`.
Go >= 1.11
----------
For Go version 1.11 or later, you should clone the repo (without having
`$GOPATH` set) and `cd` into the directory:
$ unset GOPATH
$ git clone https://github.com/restic/restic
$ cd restic
Then use the `go` tool to build restic:
$ go build ./cmd/restic
$ ./restic version
restic 0.9.2-dev (compiled manually) compiled with go1.11 on linux/amd64
You can run all tests with the following command:
$ go test ./...
Go < 1.11
---------
In order to compile restic with Go before 1.11, it needs to be checked out at
the right path within a `GOPATH`. The concept of a `GOPATH` is explained in
["How to write Go code"](https://golang.org/doc/code.html).
In order to compile restic with the `go` tool directly, it needs to be checked
out at the right path within a `GOPATH`. The concept of a `GOPATH` is explained
in ["How to write Go code"](https://golang.org/doc/code.html).
If you do not have a directory with Go code yet, executing the following
instructions in your shell will create one for you and check out the restic
@@ -114,7 +83,12 @@ You can then build restic as follows:
The following commands can be used to run all the tests:
$ go test ./...
$ go test ./cmd/... ./internal/...
The repository contains two sets of directories with code: `cmd/` and
`internal/` contain the code written for restic, whereas `vendor/` contains
copies of libraries restic depends on. The libraries are managed with the
[`dep`](https://github.com/golang/dep) tool.
Providing Patches
=================
@@ -146,8 +120,7 @@ down to the following steps:
6. You will receive comments on your code and the feature or bug that they
address. Maybe you need to rework some minor things, in this case push new
commits to the branch you created for the pull request (or amend the
existing commit, use common sense to decide which is better), they will be
commits to the branch you created for the pull request, they will be
automatically added to the pull request.
7. If your pull request changes anything that users should be aware of (a
@@ -167,14 +140,13 @@ run
gofmt -w **/*.go
in the project root directory before committing. For each Pull Request, the
formatting is tested with `gofmt` for the latest stable version of Go.
Installing the script `fmt-check` from https://github.com/edsrzf/gofmt-git-hook
locally as a pre-commit hook checks formatting before committing automatically,
just copy this script to `.git/hooks/pre-commit`.
in the project root directory before committing. Installing the script
`fmt-check` from https://github.com/edsrzf/gofmt-git-hook locally as a
pre-commit hook checks formatting before committing automatically, just copy
this script to `.git/hooks/pre-commit`.
For each pull request, several different systems run the integration tests on
Linux, macOS and Windows. We won't merge any code that does not pass all tests
Linux, OS X and Windows. We won't merge any code that does not pass all tests
for all systems, so when a tests fails, try to find out what's wrong and fix
it. If you need help on this, please leave a comment in the pull request, and
we'll be glad to assist. Having a PR with failing integration tests is nothing
@@ -191,7 +163,7 @@ history and triaging bugs much easier.
Git commit messages have a very terse summary in the first line of the commit
message, followed by an empty line, followed by a more verbose description or a
List of changed things. For examples, please refer to the excellent [How to
Write a Git Commit Message](https://chris.beams.io/posts/git-commit/).
Write a Git Commit Message](http://chris.beams.io/posts/git-commit/).
If you change/add multiple different things that aren't related at all, try to
make several smaller commits. This is much easier to review. Using `git add -p`

View File

@@ -1,27 +0,0 @@
# restic project governance
## Overview
The restic project uses a governance model commonly described as Benevolent
Dictator For Life (BDFL). This document outlines our understanding of what this
means. It is derived from the [i3 window manager project
governance](https://raw.githubusercontent.com/i3/i3/next/.github/GOVERNANCE.md).
## Roles
* user: anyone who interacts with the restic project
* core contributor: a handful of people who have contributed significantly to
the project by any means (issue triage, support, documentation, code, etc.).
Core contributors are recognizable via GitHubs "Member" badge.
* Benevolent Dictator For Life (BDFL): a single individual who makes decisions
when consensus cannot be reached. restic's current BDFL is [@fd0](https://github.com/fd0).
## Decision making process
In general, we try to reach consensus in discussions. In case consensus cannot
be reached, the BDFL makes a decision.
## Contribution process
The contribution process is described in a separate document called
[CONTRIBUTING](CONTRIBUTING.md).

219
Gopkg.lock generated Normal file
View File

@@ -0,0 +1,219 @@
# This file is autogenerated, do not edit; changes may be undone by the next 'dep ensure'.
[[projects]]
branch = "master"
name = "bazil.org/fuse"
packages = [".","fs","fuseutil"]
revision = "371fbbdaa8987b715bdd21d6adc4c9b20155f748"
[[projects]]
name = "cloud.google.com/go"
packages = ["compute/metadata"]
revision = "2d3a6656c17a60b0815b7e06ab0be04eacb6e613"
version = "v0.16.0"
[[projects]]
name = "github.com/Azure/azure-sdk-for-go"
packages = ["storage"]
revision = "7692b0cef22674113fcf71cc17ac3ccc1a7fef48"
version = "v11.2.2-beta"
[[projects]]
name = "github.com/Azure/go-autorest"
packages = ["autorest","autorest/adal","autorest/azure","autorest/date"]
revision = "c67b24a8e30d876542a85022ebbdecf0e5a935e8"
version = "v9.4.1"
[[projects]]
name = "github.com/cenkalti/backoff"
packages = ["."]
revision = "61153c768f31ee5f130071d08fc82b85208528de"
version = "v1.1.0"
[[projects]]
name = "github.com/cpuguy83/go-md2man"
packages = ["md2man"]
revision = "1d903dcb749992f3741d744c0f8376b4bd7eb3e1"
version = "v1.0.7"
[[projects]]
name = "github.com/dgrijalva/jwt-go"
packages = ["."]
revision = "dbeaa9332f19a944acb5736b4456cfcc02140e29"
version = "v3.1.0"
[[projects]]
branch = "master"
name = "github.com/dustin/go-humanize"
packages = ["."]
revision = "bb3d318650d48840a39aa21a027c6630e198e626"
[[projects]]
name = "github.com/elithrar/simple-scrypt"
packages = ["."]
revision = "2325946f714c95de4a6088202c402fbdfa64163b"
version = "v1.2.0"
[[projects]]
name = "github.com/go-ini/ini"
packages = ["."]
revision = "32e4c1e6bc4e7d0d8451aa6b75200d19e37a536a"
version = "v1.32.0"
[[projects]]
branch = "master"
name = "github.com/golang/protobuf"
packages = ["proto"]
revision = "1e59b77b52bf8e4b449a57e6f79f21226d571845"
[[projects]]
name = "github.com/inconshreveable/mousetrap"
packages = ["."]
revision = "76626ae9c91c4f2a10f34cad8ce83ea42c93bb75"
version = "v1.0"
[[projects]]
branch = "master"
name = "github.com/juju/ratelimit"
packages = ["."]
revision = "59fac5042749a5afb9af70e813da1dd5474f0167"
[[projects]]
branch = "master"
name = "github.com/kr/fs"
packages = ["."]
revision = "2788f0dbd16903de03cb8186e5c7d97b69ad387b"
[[projects]]
name = "github.com/kurin/blazer"
packages = ["b2","base","internal/b2types","internal/blog"]
revision = "e269a1a17bb6aec278c06a57cb7e8f8d0d333e04"
version = "v0.2.1"
[[projects]]
branch = "master"
name = "github.com/minio/go-homedir"
packages = ["."]
revision = "21304a94172ae3a09dee2cd86a12fb6f842138c7"
[[projects]]
name = "github.com/minio/minio-go"
packages = [".","pkg/credentials","pkg/encrypt","pkg/policy","pkg/s3signer","pkg/s3utils","pkg/set"]
revision = "57a8ae886b49af6eb0d2c27c2d007ed2f71e1da5"
version = "4.0.3"
[[projects]]
branch = "master"
name = "github.com/ncw/swift"
packages = ["."]
revision = "c95c6e5c2d1a3d37fc44c8c6dc9e231c7500667d"
[[projects]]
name = "github.com/pkg/errors"
packages = ["."]
revision = "645ef00459ed84a119197bfb8d8205042c6df63d"
version = "v0.8.0"
[[projects]]
name = "github.com/pkg/profile"
packages = ["."]
revision = "5b67d428864e92711fcbd2f8629456121a56d91f"
version = "v1.2.1"
[[projects]]
name = "github.com/pkg/sftp"
packages = ["."]
revision = "98203f5a8333288eb3163b7c667d4260fe1333e9"
version = "1.0.0"
[[projects]]
name = "github.com/pkg/xattr"
packages = ["."]
revision = "23c75e3f6c1d8b13b3dd905b011a7f38a06044b7"
version = "v0.2.1"
[[projects]]
name = "github.com/restic/chunker"
packages = ["."]
revision = "db83917be3b88cc307464b7d8a221c173e34a0db"
version = "v0.2.0"
[[projects]]
name = "github.com/russross/blackfriday"
packages = ["."]
revision = "4048872b16cc0fc2c5fd9eacf0ed2c2fedaa0c8c"
version = "v1.5"
[[projects]]
name = "github.com/satori/uuid"
packages = ["."]
revision = "879c5887cd475cd7864858769793b2ceb0d44feb"
version = "v1.1.0"
[[projects]]
name = "github.com/sirupsen/logrus"
packages = ["."]
revision = "f006c2ac4710855cf0f916dd6b77acf6b048dc6e"
version = "v1.0.3"
[[projects]]
name = "github.com/spf13/cobra"
packages = [".","doc"]
revision = "7b2c5ac9fc04fc5efafb60700713d4fa609b777b"
version = "v0.0.1"
[[projects]]
name = "github.com/spf13/pflag"
packages = ["."]
revision = "e57e3eeb33f795204c1ca35f56c44f83227c6e66"
version = "v1.0.0"
[[projects]]
branch = "master"
name = "golang.org/x/crypto"
packages = ["curve25519","ed25519","ed25519/internal/edwards25519","pbkdf2","poly1305","scrypt","ssh","ssh/terminal"]
revision = "94eea52f7b742c7cbe0b03b22f0c4c8631ece122"
[[projects]]
branch = "master"
name = "golang.org/x/net"
packages = ["context","context/ctxhttp","webdav","webdav/internal/xml"]
revision = "a8b9294777976932365dabb6640cf1468d95c70f"
[[projects]]
branch = "master"
name = "golang.org/x/oauth2"
packages = [".","google","internal","jws","jwt"]
revision = "f95fa95eaa936d9d87489b15d1d18b97c1ba9c28"
[[projects]]
branch = "master"
name = "golang.org/x/sys"
packages = ["unix","windows"]
revision = "8b4580aae2a0dd0c231a45d3ccb8434ff533b840"
[[projects]]
branch = "master"
name = "google.golang.org/api"
packages = ["gensupport","googleapi","googleapi/internal/uritemplates","storage/v1"]
revision = "3a1d936b7575b82197a1fea0632218dd07b1e65c"
[[projects]]
name = "google.golang.org/appengine"
packages = [".","internal","internal/app_identity","internal/base","internal/datastore","internal/log","internal/modules","internal/remote_api","internal/urlfetch","urlfetch"]
revision = "150dc57a1b433e64154302bdc40b6bb8aefa313a"
version = "v1.0.0"
[[projects]]
branch = "v2"
name = "gopkg.in/yaml.v2"
packages = ["."]
revision = "287cf08546ab5e7e37d55a84f7ed3fd1db036de5"
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
inputs-digest = "b01eeeb2be041c7cd11f9ee50324ef456ac1e1cd0720408c6d72f88f92f09320"
solver-name = "gps-cdcl"
solver-version = 1

21
Gopkg.toml Normal file
View File

@@ -0,0 +1,21 @@
# Gopkg.toml example
#
# Refer to https://github.com/golang/dep/blob/master/docs/Gopkg.toml.md
# for detailed Gopkg.toml documentation.
#
# required = ["github.com/user/thing/cmd/thing"]
# ignored = ["github.com/user/project/pkgX", "bitbucket.org/user/project/pkgA/pkgY"]
#
# [[constraint]]
# name = "github.com/user/project"
# version = "1.0.0"
#
# [[constraint]]
# name = "github.com/user/project2"
# branch = "dev"
# source = "github.com/myfork/project2"
#
# [[override]]
# name = "github.com/x/y"
# version = "2.4.0"

View File

@@ -3,7 +3,7 @@
all: restic
restic:
go run -mod=vendor build.go || go run build.go
go run build.go
clean:
rm -f restic

View File

@@ -1,4 +1,4 @@
|Documentation| |Build Status| |Build status| |Report Card| |Say Thanks| |TestCoverage| |Reviewed by Hound|
|Documentation| |Build Status| |Build status| |Report Card| |Say Thanks| |TestCoverage|
Introduction
------------
@@ -29,7 +29,7 @@ and add some data:
.. code-block:: console
$ restic --repo /tmp/backup backup ~/work
$ restic -r /tmp/backup backup ~/work
enter password for repository:
scan [/home/user/work]
scanned 764 directories, 1816 files in 0:00
@@ -57,7 +57,6 @@ Therefore, restic supports the following backends for storing backups natively:
- `BackBlaze B2 <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#backblaze-b2>`__
- `Microsoft Azure Blob Storage <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#microsoft-azure-blob-storage>`__
- `Google Cloud Storage <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#google-cloud-storage>`__
- And many other services via the `rclone <https://rclone.org>`__ `Backend <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#other-services-via-rclone>`__
Design Principles
-----------------
@@ -111,18 +110,10 @@ License
Restic is licensed under `BSD 2-Clause License <https://opensource.org/licenses/BSD-2-Clause>`__. You can find the
complete text in ``LICENSE``.
Sponsorship
-----------
Backend integration tests for Google Cloud Storage and Microsoft Azure Blob
Storage are sponsored by `AppsCode <https://appscode.com>`__!
|AppsCode|
.. |Documentation| image:: https://readthedocs.org/projects/restic/badge/?version=latest
:target: https://restic.readthedocs.io/en/latest/?badge=latest
.. |Build Status| image:: https://travis-ci.com/restic/restic.svg?branch=master
:target: https://travis-ci.com/restic/restic
.. |Build Status| image:: https://travis-ci.org/restic/restic.svg?branch=master
:target: https://travis-ci.org/restic/restic
.. |Build status| image:: https://ci.appveyor.com/api/projects/status/nuy4lfbgfbytw92q/branch/master?svg=true
:target: https://ci.appveyor.com/project/fd0/restic/branch/master
.. |Report Card| image:: https://goreportcard.com/badge/github.com/restic/restic
@@ -131,7 +122,3 @@ Storage are sponsored by `AppsCode <https://appscode.com>`__!
:target: https://saythanks.io/to/restic
.. |TestCoverage| image:: https://codecov.io/gh/restic/restic/branch/master/graph/badge.svg
:target: https://codecov.io/gh/restic/restic
.. |AppsCode| image:: https://cdn.appscode.com/images/logo/appscode/ac-logo-color.png
:target: https://appscode.com
.. |Reviewed by Hound| image:: https://img.shields.io/badge/Reviewed_by-Hound-8E64B0.svg
:target: https://houndci.com

View File

@@ -1 +1 @@
0.9.3
0.8.1

View File

@@ -7,9 +7,6 @@ branches:
only:
- master
cache:
- '%LocalAppData%\go-build'
init:
- ps: >-
$app = Get-WmiObject -Class Win32_Product -Filter "Vendor = 'http://golang.org'"
@@ -20,8 +17,8 @@ init:
install:
- rmdir c:\go /s /q
- appveyor DownloadFile https://dl.google.com/go/go1.11.windows-amd64.msi
- msiexec /i go1.11.windows-amd64.msi /q
- appveyor DownloadFile https://storage.googleapis.com/golang/go1.9.windows-amd64.msi
- msiexec /i go1.9.windows-amd64.msi /q
- go version
- go env
- appveyor DownloadFile http://sourceforge.netcologne.de/project/gnuwin32/tar/1.13-1/tar-1.13-1-bin.zip -FileName tar.zip
@@ -29,4 +26,4 @@ install:
- set PATH=bin/;%PATH%
build_script:
- go run -mod=vendor run_integration_tests.go
- go run run_integration_tests.go

410
build.go
View File

@@ -1,48 +1,4 @@
// Description
//
// This program aims to make building Go programs for end users easier by just
// calling it with `go run`, without having to setup a GOPATH.
//
// For Go < 1.11, it'll create a new GOPATH in a temporary directory, then run
// `go build` on the package configured as Main in the Config struct.
//
// For Go >= 1.11 if the file go.mod is present, it'll use Go modules and not
// setup a GOPATH. It builds the package configured as Main in the Config
// struct with `go build -mod=vendor` to use the vendored dependencies.
// The variable GOPROXY is set to `off` so that no network calls are made. All
// files are copied to a temporary directory before `go build` is called within
// that directory.
// BSD 2-Clause License
//
// Copyright (c) 2016-2018, Alexander Neumann <alexander@bumpern.de>
// All rights reserved.
//
// This file has been copied from the repository at:
// https://github.com/fd0/build-go
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are met:
//
// * Redistributions of source code must retain the above copyright notice, this
// list of conditions and the following disclaimer.
//
// * Redistributions in binary form must reproduce the above copyright notice,
// this list of conditions and the following disclaimer in the documentation
// and/or other materials provided with the distribution.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
// AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
// DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
// FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
// DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
// SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
// CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
// OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
// +build ignore_build_go
// +build ignore
package main
@@ -52,44 +8,69 @@ import (
"io/ioutil"
"os"
"os/exec"
"path"
"path/filepath"
"runtime"
"strconv"
"strings"
)
// config contains the configuration for the program to build.
var config = Config{
Name: "restic", // name of the program executable and directory
Namespace: "github.com/restic/restic", // subdir of GOPATH, e.g. "github.com/foo/bar"
Main: "./cmd/restic", // package name for the main package
DefaultBuildTags: []string{"selfupdate"}, // specify build tags which are always used
Tests: []string{"./..."}, // tests to run
MinVersion: GoVersion{Major: 1, Minor: 9, Patch: 0}, // minimum Go version supported
}
// Config configures the build.
type Config struct {
Name string
Namespace string
Main string
DefaultBuildTags []string
Tests []string
MinVersion GoVersion
}
var (
verbose bool
keepGopath bool
runTests bool
enableCGO bool
enablePIE bool
goVersion = ParseGoVersion(runtime.Version())
)
// copy all Go files in src to dst, creating directories on the fly, so calling
var config = struct {
Name string
Namespace string
Main string
Tests []string
MinVersion GoVersion
}{
Name: "restic", // name of the program executable and directory
Namespace: "github.com/restic/restic", // subdir of GOPATH, e.g. "github.com/foo/bar"
Main: "github.com/restic/restic/cmd/restic", // package name for the main package
Tests: []string{ // tests to run
"github.com/restic/restic/internal/...",
"github.com/restic/restic/cmd/..."},
MinVersion: GoVersion{Major: 1, Minor: 8, Patch: 0}, // minimum Go version supported
}
// specialDir returns true if the file begins with a special character ('.' or '_').
func specialDir(name string) bool {
if name == "." {
return false
}
base := filepath.Base(name)
if base == "vendor" || base[0] == '_' || base[0] == '.' {
return true
}
return false
}
// excludePath returns true if the file should not be copied to the new GOPATH.
func excludePath(name string) bool {
ext := path.Ext(name)
if ext == ".go" || ext == ".s" || ext == ".h" {
return false
}
parentDir := filepath.Base(filepath.Dir(name))
if parentDir == "testdata" {
return false
}
return true
}
// updateGopath builds a valid GOPATH at dst, with all Go files in src/ copied
// to dst/prefix/, so calling
//
// copy("/tmp/gopath/src/github.com/restic/restic", "/home/u/restic")
// updateGopath("/tmp/gopath", "/home/u/restic", "github.com/restic/restic")
//
// with "/home/u/restic" containing the file "foo.go" yields the following tree
// at "/tmp/gopath":
@@ -100,15 +81,19 @@ var (
// └── restic
// └── restic
// └── foo.go
func copy(dst, src string) error {
verbosePrintf("copy contents of %v to %v\n", src, dst)
func updateGopath(dst, src, prefix string) error {
verbosePrintf("copy contents of %v to %v\n", src, filepath.Join(dst, prefix))
return filepath.Walk(src, func(name string, fi os.FileInfo, err error) error {
if name == src {
return err
}
if name == ".git" {
return filepath.SkipDir
if specialDir(name) {
if fi.IsDir() {
return filepath.SkipDir
}
return nil
}
if err != nil {
@@ -119,13 +104,17 @@ func copy(dst, src string) error {
return nil
}
if excludePath(name) {
return nil
}
intermediatePath, err := filepath.Rel(src, name)
if err != nil {
return err
}
fileSrc := filepath.Join(src, intermediatePath)
fileDst := filepath.Join(dst, intermediatePath)
fileDst := filepath.Join(dst, "src", prefix, intermediatePath)
return copyFile(fileDst, fileSrc)
})
@@ -140,15 +129,6 @@ func directoryExists(dirname string) bool {
return stat.IsDir()
}
func fileExists(filename string) bool {
stat, err := os.Stat(filename)
if err != nil && os.IsNotExist(err) {
return false
}
return stat.Mode().IsRegular()
}
// copyFile creates dst from src, preserving file attributes and timestamps.
func copyFile(dst, src string) error {
fi, err := os.Stat(src)
@@ -168,34 +148,30 @@ func copyFile(dst, src string) error {
fdst, err := os.Create(dst)
if err != nil {
_ = fsrc.Close()
return err
}
_, err = io.Copy(fdst, fsrc)
if err != nil {
_ = fsrc.Close()
_ = fdst.Close()
if _, err = io.Copy(fdst, fsrc); err != nil {
return err
}
err = fdst.Close()
if err != nil {
_ = fsrc.Close()
return err
if err == nil {
err = fsrc.Close()
}
err = fsrc.Close()
if err != nil {
return err
if err == nil {
err = fdst.Close()
}
err = os.Chmod(dst, fi.Mode())
if err != nil {
return err
if err == nil {
err = os.Chmod(dst, fi.Mode())
}
return os.Chtimes(dst, fi.ModTime(), fi.ModTime())
if err == nil {
err = os.Chtimes(dst, fi.ModTime(), fi.ModTime())
}
return nil
}
// die prints the message with fmt.Fprintf() to stderr and exits with an error
@@ -211,15 +187,12 @@ func showUsage(output io.Writer) {
fmt.Fprintf(output, "OPTIONS:\n")
fmt.Fprintf(output, " -v --verbose output more messages\n")
fmt.Fprintf(output, " -t --tags specify additional build tags\n")
fmt.Fprintf(output, " -k --keep-tempdir do not remove the temporary directory after build\n")
fmt.Fprintf(output, " -k --keep-gopath do not remove the GOPATH after build\n")
fmt.Fprintf(output, " -T --test run tests\n")
fmt.Fprintf(output, " -o --output set output file name\n")
fmt.Fprintf(output, " --enable-cgo use CGO to link against libc\n")
fmt.Fprintf(output, " --enable-pie use PIE buildmode\n")
fmt.Fprintf(output, " --goos value set GOOS for cross-compilation\n")
fmt.Fprintf(output, " --goarch value set GOARCH for cross-compilation\n")
fmt.Fprintf(output, " --goarm value set GOARM for cross-compilation\n")
fmt.Fprintf(output, " --tempdir dir use a specific directory for compilation\n")
}
func verbosePrintf(message string, args ...interface{}) {
@@ -230,20 +203,11 @@ func verbosePrintf(message string, args ...interface{}) {
fmt.Printf("build: "+message, args...)
}
// cleanEnv returns a clean environment with GOPATH, GOBIN and GO111MODULE
// removed (if present).
// cleanEnv returns a clean environment with GOPATH and GOBIN removed (if
// present).
func cleanEnv() (env []string) {
removeKeys := map[string]struct{}{
"GOPATH": struct{}{},
"GOBIN": struct{}{},
"GO111MODULE": struct{}{},
}
for _, v := range os.Environ() {
data := strings.SplitN(v, "=", 2)
name := data[0]
if _, ok := removeKeys[name]; ok {
if strings.HasPrefix(v, "GOPATH=") || strings.HasPrefix(v, "GOBIN=") {
continue
}
@@ -254,28 +218,13 @@ func cleanEnv() (env []string) {
}
// build runs "go build args..." with GOPATH set to gopath.
func build(cwd string, env map[string]string, args ...string) error {
func build(cwd, goos, goarch, gopath string, args ...string) error {
a := []string{"build"}
if goVersion.AtLeast(GoVersion{1, 10, 0}) {
verbosePrintf("Go version is at least 1.10, using new syntax for -gcflags\n")
// use new prefix
a = append(a, "-asmflags", fmt.Sprintf("all=-trimpath=%s", cwd))
a = append(a, "-gcflags", fmt.Sprintf("all=-trimpath=%s", cwd))
} else {
a = append(a, "-asmflags", fmt.Sprintf("-trimpath=%s", cwd))
a = append(a, "-gcflags", fmt.Sprintf("-trimpath=%s", cwd))
}
if enablePIE {
a = append(a, "-buildmode=pie")
}
a = append(a, "-asmflags", fmt.Sprintf("-trimpath=%s", gopath))
a = append(a, "-gcflags", fmt.Sprintf("-trimpath=%s", gopath))
a = append(a, args...)
cmd := exec.Command("go", a...)
cmd.Env = append(cleanEnv(), "GOPROXY=off")
for k, v := range env {
cmd.Env = append(cmd.Env, k+"="+v)
}
cmd.Env = append(cleanEnv(), "GOPATH="+gopath, "GOARCH="+goarch, "GOOS="+goos)
if !enableCGO {
cmd.Env = append(cmd.Env, "CGO_ENABLED=0")
}
@@ -283,30 +232,20 @@ func build(cwd string, env map[string]string, args ...string) error {
cmd.Dir = cwd
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
verbosePrintf("chdir %q\n", cwd)
verbosePrintf("go %q\n", a)
verbosePrintf("go %s\n", args)
return cmd.Run()
}
// test runs "go test args..." with GOPATH set to gopath.
func test(cwd string, env map[string]string, args ...string) error {
args = append([]string{"test", "-count", "1"}, args...)
func test(cwd, gopath string, args ...string) error {
args = append([]string{"test"}, args...)
cmd := exec.Command("go", args...)
cmd.Env = append(cleanEnv(), "GOPROXY=off")
for k, v := range env {
cmd.Env = append(cmd.Env, k+"="+v)
}
if !enableCGO {
cmd.Env = append(cmd.Env, "CGO_ENABLED=0")
}
cmd.Env = append(cleanEnv(), "GOPATH="+gopath)
cmd.Dir = cwd
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
verbosePrintf("chdir %q\n", cwd)
verbosePrintf("go %q\n", args)
verbosePrintf("go %s\n", args)
return cmd.Run()
}
@@ -389,39 +328,30 @@ func ParseGoVersion(s string) (v GoVersion) {
s = s[2:]
data := strings.Split(s, ".")
if len(data) < 2 || len(data) > 3 {
// invalid version
return GoVersion{}
if len(data) != 3 {
return
}
var err error
v.Major, err = strconv.Atoi(data[0])
major, err := strconv.Atoi(data[0])
if err != nil {
return GoVersion{}
return
}
// try to parse the minor version while removing an eventual suffix (like
// "rc2" or so)
for s := data[1]; s != ""; s = s[:len(s)-1] {
v.Minor, err = strconv.Atoi(s)
if err == nil {
break
}
minor, err := strconv.Atoi(data[1])
if err != nil {
return
}
if v.Minor == 0 {
// no minor version found
return GoVersion{}
patch, err := strconv.Atoi(data[2])
if err != nil {
return
}
if len(data) >= 3 {
v.Patch, err = strconv.Atoi(data[2])
if err != nil {
return GoVersion{}
}
v = GoVersion{
Major: major,
Minor: minor,
Patch: patch,
}
return
}
@@ -454,24 +384,19 @@ func (v GoVersion) String() string {
}
func main() {
if !goVersion.AtLeast(config.MinVersion) {
fmt.Fprintf(os.Stderr, "%s detected, this program requires at least %s\n", goVersion, config.MinVersion)
ver := ParseGoVersion(runtime.Version())
if !ver.AtLeast(config.MinVersion) {
fmt.Fprintf(os.Stderr, "%s detected, this program requires at least %s\n", ver, config.MinVersion)
os.Exit(1)
}
buildTags := config.DefaultBuildTags
buildTags := []string{}
skipNext := false
params := os.Args[1:]
goEnv := map[string]string{}
buildEnv := map[string]string{
"GOOS": runtime.GOOS,
"GOARCH": runtime.GOARCH,
"GOARM": "",
}
tempdir := ""
targetGOOS := runtime.GOOS
targetGOARCH := runtime.GOARCH
var outputFilename string
@@ -491,28 +416,20 @@ func main() {
die("-t given but no tag specified")
}
skipNext = true
buildTags = append(buildTags, strings.Split(params[i+1], " ")...)
buildTags = strings.Split(params[i+1], " ")
case "-o", "--output":
skipNext = true
outputFilename = params[i+1]
case "--tempdir":
skipNext = true
tempdir = params[i+1]
case "-T", "--test":
runTests = true
case "--enable-cgo":
enableCGO = true
case "--enable-pie":
enablePIE = true
case "--goos":
skipNext = true
buildEnv["GOOS"] = params[i+1]
targetGOOS = params[i+1]
case "--goarch":
skipNext = true
buildEnv["GOARCH"] = params[i+1]
case "--goarm":
skipNext = true
buildEnv["GOARM"] = params[i+1]
targetGOARCH = params[i+1]
case "-h":
showUsage(os.Stdout)
return
@@ -523,7 +440,10 @@ func main() {
}
}
verbosePrintf("detected Go version %v\n", goVersion)
if len(buildTags) == 0 {
verbosePrintf("adding build-tag release\n")
buildTags = []string{"release"}
}
for i := range buildTags {
buildTags[i] = strings.TrimSpace(buildTags[i])
@@ -536,16 +456,48 @@ func main() {
die("Getwd(): %v\n", err)
}
gopath, err := ioutil.TempDir("", fmt.Sprintf("%v-build-", config.Name))
if err != nil {
die("TempDir(): %v\n", err)
}
verbosePrintf("create GOPATH at %v\n", gopath)
if err = updateGopath(gopath, root, config.Namespace); err != nil {
die("copying files from %v/src to %v/src failed: %v\n", root, gopath, err)
}
vendor := filepath.Join(root, "vendor")
if directoryExists(vendor) {
if err = updateGopath(gopath, vendor, filepath.Join(config.Namespace, "vendor")); err != nil {
die("copying files from %v to %v failed: %v\n", root, gopath, err)
}
}
defer func() {
if !keepGopath {
verbosePrintf("remove %v\n", gopath)
if err = os.RemoveAll(gopath); err != nil {
die("remove GOPATH at %s failed: %v\n", err)
}
} else {
verbosePrintf("leaving temporary GOPATH at %v\n", gopath)
}
}()
if outputFilename == "" {
outputFilename = config.Name
if buildEnv["GOOS"] == "windows" {
if targetGOOS == "windows" {
outputFilename += ".exe"
}
}
cwd, err := os.Getwd()
if err != nil {
die("Getwd() returned %v\n", err)
}
output := outputFilename
if !filepath.IsAbs(output) {
output = filepath.Join(root, output)
output = filepath.Join(cwd, output)
}
version := getVersion()
@@ -556,65 +508,13 @@ func main() {
ldflags := "-s -w " + constants.LDFlags()
verbosePrintf("ldflags: %s\n", ldflags)
var (
buildArgs []string
testArgs []string
)
mainPackage := config.Main
if strings.HasPrefix(mainPackage, config.Namespace) {
mainPackage = strings.Replace(mainPackage, config.Namespace, "./", 1)
}
buildTarget := filepath.FromSlash(mainPackage)
buildCWD := ""
if goVersion.AtLeast(GoVersion{1, 11, 0}) && fileExists("go.mod") {
verbosePrintf("Go >= 1.11 and 'go.mod' found, building with modules\n")
buildCWD = root
buildArgs = append(buildArgs, "-mod=vendor")
testArgs = append(testArgs, "-mod=vendor")
} else {
if tempdir == "" {
tempdir, err = ioutil.TempDir("", fmt.Sprintf("%v-build-", config.Name))
if err != nil {
die("TempDir(): %v\n", err)
}
}
verbosePrintf("Go < 1.11 or 'go.mod' not found, create GOPATH at %v\n", tempdir)
targetdir := filepath.Join(tempdir, "src", filepath.FromSlash(config.Namespace))
if err = copy(targetdir, root); err != nil {
die("copying files from %v to %v/src failed: %v\n", root, tempdir, err)
}
defer func() {
if !keepGopath {
verbosePrintf("remove %v\n", tempdir)
if err = os.RemoveAll(tempdir); err != nil {
die("remove GOPATH at %s failed: %v\n", tempdir, err)
}
} else {
verbosePrintf("leaving temporary GOPATH at %v\n", tempdir)
}
}()
buildCWD = targetdir
goEnv["GOPATH"] = tempdir
buildEnv["GOPATH"] = tempdir
}
verbosePrintf("environment:\n go: %v\n build: %v\n", goEnv, buildEnv)
buildArgs = append(buildArgs,
args := []string{
"-tags", strings.Join(buildTags, " "),
"-ldflags", ldflags,
"-o", output, buildTarget,
)
"-o", output, config.Main,
}
err = build(buildCWD, buildEnv, buildArgs...)
err = build(filepath.Join(gopath, "src"), targetGOOS, targetGOARCH, gopath, args...)
if err != nil {
die("build failed: %v\n", err)
}
@@ -622,9 +522,7 @@ func main() {
if runTests {
verbosePrintf("running tests\n")
testArgs = append(testArgs, config.Tests...)
err = test(buildCWD, goEnv, testArgs...)
err = test(cwd, gopath, config.Tests...)
if err != nil {
die("running tests failed: %v\n", err)
}

View File

@@ -2,9 +2,9 @@ Enhancement: Create subdirs below `data/` for local/sftp backends
The local and sftp backends now create the subdirs below `data/` on
open/init. This way, restic makes sure that they always exist. This is
connected to an issue for the sftp server.
connected to an issue for the sftp server:
https://github.com/restic/restic/issues/1055
https://github.com/restic/rest-server/pull/11#issuecomment-309879710
https://github.com/restic/restic/issues/1055
https://github.com/restic/restic/pull/1077
https://github.com/restic/restic/pull/1105

View File

@@ -1,6 +1,6 @@
Enhancement: Add `migrate` cmd to migrate from `s3legacy` to `default` layout
The `migrate` command for changing the `s3legacy` layout to the `default` layout
The `migrate` command for chaning the `s3legacy` layout to the `default` layout
for s3 backends has been improved: It can now be restarted with `restic migrate
--force s3_layout` and automatically retries operations on error.

View File

@@ -1,3 +1,3 @@
Enhancement: Properly report errors when reading files with exclude patterns
Enhancement: Properly report errors when reading files with exclude patterns.
https://github.com/restic/restic/pull/1144

View File

@@ -1,8 +0,0 @@
Enhancement: Add support for TLS client certificate authentication
Support has been added for using a TLS client certificate for authentication to
HTTP based backend. A file containing the PEM encoded private key and
certificate can be set using the `--tls-client-cert` option.
https://github.com/restic/restic/issues/1522
https://github.com/restic/restic/pull/1524

View File

@@ -1,7 +0,0 @@
Enhancement: Reduce number of remote requests during repository check
This change eliminates redundant remote repository calls and significantly
improves repository check time.
https://github.com/restic/restic/issues/1541
https://github.com/restic/restic/pull/1548

View File

@@ -1,9 +0,0 @@
Enhancement: Reduce number of backend requests for rebuild-index and prune
We've found a way to reduce then number of backend requests for the
`rebuild-index` and `prune` operations. This significantly speeds up the
operations for high-latency backends.
https://github.com/restic/restic/issues/1567
https://github.com/restic/restic/pull/1574
https://github.com/restic/restic/pull/1575

View File

@@ -1,7 +0,0 @@
Bugfix: Strip spaces for lines read via --files-from
Leading and trailing spaces in lines read via `--files-from` are now stripped,
so it behaves the same as with lines read via `--exclude-file`.
https://github.com/restic/restic/issues/1590
https://github.com/restic/restic/pull/1613

View File

@@ -1,7 +0,0 @@
Enhancement: Speed up querying across indices and scanning existing files
This change increases the whenever a blob (part of a file) is searched for in a
restic repository. This will reduce cpu usage some when backing up files already
backed up by restic. Cpu usage is further decreased when scanning files.
https://github.com/restic/restic/pull/1549

View File

@@ -1,10 +0,0 @@
Enhancement: Don't terminate ssh on SIGINT
We've reworked the code which runs the `ssh` login for the sftp backend so that
it can prompt for a password (if needed) but does not exit when the user
presses CTRL+C (SIGINT) e.g. during backup. This allows restic to properly shut
down when it receives SIGINT and remove the lock file from the repo, afterwards
exiting the `ssh` process.
https://github.com/restic/restic/pull/1564
https://github.com/restic/restic/pull/1588

View File

@@ -1,3 +0,0 @@
Enhancement: Retry Backend.List() in case of errors
https://github.com/restic/restic/pull/1579

View File

@@ -1,12 +0,0 @@
Enhancement: Limit index file size
Before, restic would create a single new index file on `prune` or
`rebuild-index`, this may lead to memory problems when this huge index is
created and loaded again. We're now limiting the size of the index file, and
split newly created index files into several smaller ones. This allows restic
to be more memory-efficient.
https://github.com/restic/restic/pull/1584
https://github.com/restic/restic/issues/1412
https://github.com/restic/restic/issues/979
https://github.com/restic/restic/issues/526

View File

@@ -1,17 +0,0 @@
Bugfix: Complete intermediate index upload
After a user posted a comprehensive report of what he observed, we were able to
find a bug and correct it: During backup, restic uploads so-called
"intermediate" index files. When the backup finishes during a transfer of such
an intermediate index, the upload is cancelled, but the backup is finished
without an error. This leads to an inconsistent state, where the snapshot
references data that is contained in the repo, but is not referenced in any
index.
The situation can be resolved by building a new index with `rebuild-index`, but
looks very confusing at first. Since all the data got uploaded to the repo
successfully, there was no risk of data loss, just minor inconvenience for our
users.
https://github.com/restic/restic/pull/1589
https://forum.restic.net/t/error-loading-tree-check-prune-and-forget-gives-error-b2-backend/406

View File

@@ -1,7 +0,0 @@
Bugfix: Google Cloud Storage: Use generic HTTP transport
It was discovered that the Google Cloud Storage backend did not use the generic
HTTP transport, so things such as bandwidth limiting with `--limit-upload` did
not work. This is resolved now.
https://github.com/restic/restic/pull/1594

View File

@@ -1,11 +0,0 @@
Bugfix: backup: Remove bandwidth display
This commit removes the bandwidth displayed during backup process. It is
misleading and seldomly correct, because it's neither the "read
bandwidth" (only for the very first backup) nor the "upload bandwidth".
Many users are confused about (and rightly so), c.f. #1581, #1033, #1591
We'll eventually replace this display with something more relevant when
the new archiver code is ready.
https://github.com/restic/restic/pull/1595

View File

@@ -1,8 +0,0 @@
Enhancement: Add --read-data-subset flag to check command
This change introduces ability to check integrity of a subset of repository
data packs. This can be used to spread integrity check of larger repositories
over a period of time.
https://github.com/restic/restic/issues/1497
https://github.com/restic/restic/pull/1556

Some files were not shown because too many files have changed in this diff Show More