Compare commits

...

1431 Commits

Author SHA1 Message Date
Alexander Neumann
beb208e159 Add VERSION for 0.9.2 2018-08-06 21:28:59 +02:00
Alexander Neumann
c221d662d0 Update manpages and auto-completion 2018-08-06 21:28:59 +02:00
Alexander Neumann
143597d445 Generate CHANGELOG.md for 0.9.2 2018-08-06 21:28:48 +02:00
Alexander Neumann
16ca837763 Create changelog/ release subdir for 0.9.2 2018-08-06 21:28:16 +02:00
Alexander Neumann
ce7fb166b3 Merge pull request #1927 from prosoitos/minor-fixes_examples
minor-fixes_examples
2018-08-05 12:27:03 +02:00
Marie-Helene Burle
9de51d04ec minor-fixes_examples
- update link to latest restic release to install restic for `restic` user
- correct typo in path to this app
2018-08-05 02:15:17 -07:00
Alexander Neumann
dc39773cd2 Merge pull request #1924 from cuspat96/master
Clarify B2 Application Keys once again
2018-08-03 21:15:16 +02:00
cuspat96
30fa305c07 Clarify B2 Application Keys once again
This commit fixes documentation about Application Keys in Backblaze B2.
2018-08-02 21:29:29 +02:00
Alexander Neumann
686f24b578 doc: Clarify B2 application keys 2018-08-02 21:14:05 +02:00
Alexander Neumann
247d2b7215 Merge pull request #1921 from salbertson/patch-1
Add a "Reviewed by Hound" badge
2018-08-02 20:03:43 +02:00
Alexander Neumann
017cd113d3 Merge pull request #1922 from salbertson/patch-2
Use https when linking to chris.beams.io
2018-08-02 20:03:40 +02:00
Scott Albertson
f744c2553e Use https when linking to chris.beams.io
Why not link to How to [Write a Git Commit Message](https://chris.beams.io/posts/git-commit/) using HTTPS, it's going to redirect anyway.
2018-08-01 14:59:06 -07:00
Alexander Neumann
56cd6bd495 Merge pull request #1919 from restic/update-deps
Update dependencies
2018-08-01 23:56:55 +02:00
Alexander Neumann
bff635bc5f Update dependencies, enable pruning for vendor/
So, `dep` got an nice new feature to remove tests and non-go files from
`vendor/`, and this brings the size of the vendor directory from ~300MiB
down to ~20MiB. We don that now.
2018-08-01 21:32:15 +02:00
Alexander Neumann
3422c1ca83 Merge pull request #1729 from mholt/stats
Implement `restic stats` command to get more info about a repository
2018-07-31 23:24:36 +02:00
Matthew Holt
f6b2731aa5 stats: Add manual doc, improve -h doc
Also rename files-by-content to files-by-contents, once and for all
2018-07-31 22:54:10 +02:00
Scott Albertson
3eb5b45b41 Add a "Reviewed by Hound" badge 2018-07-31 13:53:24 -07:00
Alexander Neumann
01aacf41b5 Merge pull request #1915 from mlissner/patch-2
Adds warning re performance of prune
2018-07-31 22:42:20 +02:00
Mike Lissner
2caf8edc55 Add warning of the performance of prune
I went pretty loud with this, but I think the performance is bad enough
that it's really worth highlighting, especially since it locks the index
during the prune.
2018-07-31 22:41:40 +02:00
Alexander Neumann
3151978f58 Fix changelog type 2018-07-31 21:57:27 +02:00
Alexander Neumann
ab4ef432ff Add entry to changelog 2018-07-31 21:29:47 +02:00
Alexander Neumann
be4f54b603 Merge pull request #1913 from restic/restic-password-stdin-message
Print message for password being read from stdin
2018-07-31 21:28:12 +02:00
Alexander Neumann
7260110c27 Merge pull request #1914 from restic/update-blazer
Add support for B2 application keys
2018-07-31 21:27:50 +02:00
Alexander Neumann
2437f11af7 Update github.com/kurin/blazer to 0.5.1
This adds support for B2 application keys.
2018-07-31 20:51:36 +02:00
Alexander Neumann
57873502f8 Add note about B2 application keys to the docs 2018-07-31 20:49:54 +02:00
Alexander Neumann
3678ec9ad8 Print message for password being read from stdin
Closes #1911
2018-07-31 20:21:18 +02:00
Alexander Neumann
a717e9e6f7 Improve message for number of bytes newly added 2018-07-31 19:08:43 +02:00
Alexander Neumann
12c797700e make statsWalkSnapshot return a function 2018-07-27 21:44:59 +02:00
Matthew Holt
daca9d6815 Consolidate mode flags; use new Walk function 2018-07-27 21:27:40 +02:00
Matthew Holt
930602a444 Update comment now that question was answered 2018-07-27 21:27:39 +02:00
Matthew Holt
acb05e7855 Fix filepath uniqueness bug for blobs-per-file mode 2018-07-27 21:27:39 +02:00
Matthew Holt
a7b95d716a Implement four counting modes 2018-07-27 21:27:39 +02:00
Matthew Holt
925b542eb0 Count unique files by blob sequence rather than tree ID 2018-07-27 21:27:39 +02:00
Matthew Holt
f7659bd8b0 stats: Initial implementation of stats command 2018-07-27 21:27:39 +02:00
Alexander Neumann
8c124a2b75 Merge pull request #1902 from mlissner/patch-1
b2 bucket names need to be unique
2018-07-23 22:58:42 +02:00
Mike Lissner
d3ad63a4ec b2 bucket names need to be unique
Adds a small warning indicating that b2 bucket names need to be unique. It's an easy mistake to make, and it's surprising to get the following error if you're not accustomed to the way B2 works:

    Fatal: create repository at b2:postgres failed: NewBucket: b2_create_bucket: 400: Bucket name is already in use
2018-07-23 11:48:59 -07:00
Alexander Neumann
271c50cf5c Add entry to changelog 2018-07-23 20:15:55 +02:00
Alexander Neumann
1aeb193fd9 Merge pull request #1900 from restic/fix-1870
restorer: Add test for restore with include filter
2018-07-23 20:15:50 +02:00
Alexander Neumann
f715bef82f Merge pull request #1899 from garrmcnu/check-cache-dir
check: Use --cache-dir argument
2018-07-22 21:03:52 +02:00
Alexander Neumann
4fc00d4120 Merge pull request #1901 from restic/update-blazer
Update github.com/kurin/blazer
2018-07-22 20:59:52 +02:00
Garry McNulty
7603ab7ac1 check: Update --cache-dir argument handling based on code review comments
The temporary cache directory is created in the specified directory, or
if not specified in the default temporary directory.
2018-07-22 18:24:11 +01:00
Alexander Neumann
36fa1f8c20 Merge pull request #1894 from restic/fix1893
Return error when exclude file cannot be read
2018-07-22 14:34:27 +02:00
Alexander Neumann
445fb23b6d Rework issue templates for Bug reports and Features 2018-07-22 14:26:23 +02:00
Alexander Neumann
5f79b4cb6c Update issue template again 2018-07-22 14:21:08 +02:00
Alexander Neumann
8e15b59347 Use underline style markup for issue/PR templates 2018-07-22 14:17:53 +02:00
Alexander Neumann
6e2e957332 Add entry to changelog 2018-07-22 14:16:08 +02:00
Alexander Neumann
7ffc03ff8f Update github.com/kurin/blazer to 0.5.0
This includes support for the upcoming B2 application keys feature.
2018-07-22 14:12:02 +02:00
Alexander Neumann
44924ba043 restorer: Fix traverseTree
traverseTree() was meant to call enterDir() whenever a directory is
selected for restore, either explicitly or implicitly (=contains a file
which is to be restored). After restoring a file, leaveDir() is called
in reverse order for all intermediate directories so that the metadata
can be restored.

When a directory is selected implicitly, the metadata for it is
restored. This is different from the previous restorer behavior, which
created implicitly selected intermediate directories with permissions
0700 (only user can read/write it).

This commit changes the behavior back to the old one. Only a directory
is explicitly selected for restore, enterDir()/leaveDir() are called for
it. Otherwise, only visitNode() is called, so visitNode() needs to make
sure the parent directory exists. If the directory is explicitly
included, leaveDir() will then restore the metadata correctly.

When we decide to change the behavior (restore metadata for all
intermediate directories, even if selected implicitly), we should do
that in the selection functions, not here.

This finally resolves #1870
2018-07-21 23:24:40 +02:00
Alexander Neumann
ce19f26948 restorer: Add tests for traverseTree 2018-07-21 23:24:40 +02:00
Alexander Neumann
74016d5981 restorer: Fix return of saveSnapshot 2018-07-21 23:24:40 +02:00
Alexander Neumann
57636a4573 restorer: Run tests in the same package 2018-07-21 23:24:40 +02:00
Alexander Neumann
4f6d2502f7 restorer: Add test for restore with include filter 2018-07-21 23:24:40 +02:00
Garry McNulty
f1f69bc648 check: Use --cache-dir argument
Closes #1880
2018-07-20 20:51:20 +01:00
Alexander Neumann
d7551d7b0c Add entry to changelog 2018-07-18 21:41:20 +02:00
Alexander Neumann
fb74de6360 Return an error when exclude files cannot be read 2018-07-18 21:39:07 +02:00
Alexander Neumann
67535e00a8 Merge pull request #1889 from ProactiveServices/patch-3
doc: Minor grammar, RST syntax fixes
2018-07-18 21:22:10 +02:00
Alexander Neumann
19592285eb Merge pull request #1888 from ProactiveServices/patch-2
doc: Minor grammar fixes
2018-07-18 21:21:52 +02:00
Alexander Neumann
f64862722a Merge pull request #1887 from restic/disable-error-size
checker: Disable size check for now
2018-07-18 21:19:54 +02:00
Adam Piggott
254239c2a9 doc: Minor grammar, RST syntax fixes
Fix unescaped backslash
Fix wording of Windows installation
2018-07-18 02:28:23 +01:00
Adam Piggott
cce1a1f768 doc: Minor grammar fixes 2018-07-18 02:25:31 +01:00
Alexander Neumann
754482fe6c checker: Disable size check for now 2018-07-15 21:52:38 +02:00
Alexander Neumann
73153dbd3f Merge pull request #1885 from restic/create-restore-target
restore: Make sure the target directory exists
2018-07-15 16:28:25 +02:00
Alexander Neumann
92421ec47f restore: Make sure target directory exists 2018-07-15 16:02:04 +02:00
Alexander Neumann
9acc9243ba Add test for not-existing top-level dir and top-level file 2018-07-15 16:00:26 +02:00
Alexander Neumann
df64998649 Merge pull request #1882 from duzvik/aws-credentials-priority
Change AWS credentials priority, to accept AWS_SESSION_TOKEN
2018-07-14 20:48:42 +02:00
Alexander Neumann
64d27eed86 doc: Improve dump to stdout
Closes #1884
2018-07-14 20:45:52 +02:00
Alexander Neumann
abb18a830c Fix test 2018-07-14 11:51:34 +02:00
denis.uzvik
1e42f4f300 S3 backend: accept AWS_SESSION_TOKEN 2018-07-12 16:18:19 +03:00
Alexander Neumann
bd742ddb69 cache: Don't recreate CACHEDIR.TAG 2018-07-08 12:05:12 +02:00
Alexander Neumann
b511f4dce2 Improve help message for check 2018-07-05 22:19:08 +02:00
Alexander Neumann
7961740dcc Fix link 2018-07-05 21:03:40 +02:00
Alexander Neumann
dc3032c360 Mention that AppsCode is sponsoring backend tests 2018-07-05 21:01:57 +02:00
Alexander Neumann
44fb2a860f Merge pull request #1861 from McKael/fix_find_ignore_case
Fix find -i (case-insensitive search)
2018-06-24 21:08:03 +02:00
Mikael Berthe
fbf8073dfc Fix find -i (case-insensitive search) 2018-06-24 19:35:11 +02:00
Alexander Neumann
7ddf91b65c Merge pull request #1772 from ifedorenko/restore-verify
Restore verify
2018-06-23 14:50:29 +02:00
Alexander Neumann
8dae2de2ce Merge pull request #1858 from restic/fix-1857
Allow excluding files with $ via --exclude-file
2018-06-22 21:22:23 +02:00
Alexander Neumann
03a0377410 Merge pull request #1856 from stkw0/master
Ignore ExcludeOtherFs if Stdin is true
2018-06-22 20:49:34 +02:00
Alexander Neumann
025ec9dff5 Allow excluding files with $ via --exclude-file
Previously it wasn't possbile to exclude files with a literal dollar
sign (`$`) via exclude files, now users can write `$$` for that.
2018-06-22 20:46:04 +02:00
David Roman
2384c1cee7 Ignore ExcludeOtherFs if Stdin is true
Closes: #1807
2018-06-21 23:12:38 +02:00
Alexander Neumann
bb2ad76833 Merge pull request #1855 from restic/fix-1854
Allows saving files/dirs on different file systems together with `--one-file-system`.
2018-06-21 20:48:33 +02:00
Alexander Neumann
30cfd13328 Add changelog 2018-06-20 23:05:09 +02:00
Alexander Neumann
9ffc26883a archiver: Unroll tree 2018-06-20 22:56:41 +02:00
Alexander Neumann
83c51db903 fs: Add helper functions ReadDir/ReadDirNames 2018-06-20 22:53:53 +02:00
Alexander Neumann
d30d5d4473 Merge pull request #1853 from skriss/list-keys-json-output
add JSON output support for restic key list cmd
2018-06-19 21:28:45 +02:00
Steve Kriss
5088905502 add JSON output support for restic key list cmd
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-18 15:26:26 -07:00
Alexander Neumann
ae72b438b0 doc: Fix typo 2018-06-18 22:19:01 +02:00
Alexander Neumann
ddf2065ce2 doc: Split references out into smaller files
Closes #1852
2018-06-18 22:17:48 +02:00
Alexander Neumann
228a970540 doc: Add paragraph about finding path in file 2018-06-17 20:49:11 +02:00
Alexander Neumann
c7a8086c19 Merge pull request #1851 from restic/fix-1849
walker: Don't ignore empty trees by default
2018-06-17 12:25:33 +02:00
Alexander Neumann
c2c06ae2c9 walker: Don't ignore empty trees by default
Closes #1849
2018-06-17 09:49:03 +02:00
Alexander Neumann
1824168aa3 Merge pull request #1848 from iquiw/netbsd-support
Support NetBSD without fuse
2018-06-16 17:12:20 +02:00
Iku Iwasa
350761f1ba Add cross-compilation for netbsd/386 and netbsd/amd64 2018-06-16 21:05:26 +09:00
Alexander Neumann
3231945a85 Merge pull request #1846 from cbergmann/patch-1
Clarify live status display
2018-06-16 11:19:58 +02:00
Iku Iwasa
f080142137 Support NetBSD without fuse 2018-06-16 15:55:04 +09:00
Clemens Bergmann
ff785924de Clarify live status display
When I backup one of my filesystems which has a lot of Hard Links (Backup directory of burp) the live status shows me 4.5 TB but it only takes up 1.2 TB of space in the repository. This is confusing because my repo is on S3 and I feared a huge Bill. This change should clarify this.
2018-06-15 12:41:48 +02:00
Alexander Neumann
393a7266c9 Merge pull request #1845 from dimejo/patch-2
doc: enhance FAQ about AV software on Windows
2018-06-14 20:48:15 +02:00
Alex JOST
cb8d2d3df5 doc: enhance FAQ about AV software on Windows
As discussed in #1840, realtime protection of AV software can cause serious performance degradation on Windows OS.
2018-06-14 19:39:04 +02:00
Alexander Neumann
a884ce1566 Merge pull request #1841 from jernst/master
Added documentation about what happens if you run out of space during a backup
2018-06-14 19:13:31 +02:00
Alexander Neumann
5ae8316c24 Merge pull request #1844 from gsanchietti/patch-1
Documentation: minor RST syntax fixes
2018-06-14 19:12:57 +02:00
Giacomo Sanchietti
85eca1b5e9 Documentation: minor RST syntax fixes
- Fix bad link syntax
- Fix compilation warning
2018-06-14 15:01:49 +02:00
Johannes Ernst
a1536f38fa Fixed typo and grammar per feedback on PR. 2018-06-13 00:24:42 +00:00
Johannes Ernst
888f52afd1 Added documentation about what happens if you run out of space
during a backup, based on conversation on the forum:
https://forum.restic.net/t/limited-backup-destination-space/733/6
2018-06-11 21:20:08 +00:00
Igor Fedorenko
e206680947 restore: New --verify flag to verify restored files content
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-06-11 07:57:22 -04:00
Igor Fedorenko
5fa6dc53cb Refactor: introduced restorer tree visitor
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-06-11 07:57:22 -04:00
Igor Fedorenko
26be094f28 Refactor: moved restorer to separate package
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-06-11 07:57:22 -04:00
Alexander Neumann
e4c0d77bdd Add VERSION for 0.9.1 2018-06-10 11:31:03 +02:00
Alexander Neumann
1dd655dad2 Generate CHANGELOG.md for 0.9.1 2018-06-10 11:30:53 +02:00
Alexander Neumann
581d0984fe Fix changelog entry 2018-06-10 11:29:57 +02:00
Alexander Neumann
e62add84bc Move changelog files for 0.9.1 2018-06-10 11:22:32 +02:00
Alexander Neumann
63779c1eb4 Merge pull request #1839 from restic/fix-find
Fix find, do not skip some snapshots
2018-06-10 10:08:47 +02:00
Alexander Neumann
c204382ea9 Revert "Fix integration tests on Windows"
This reverts commit 33dbd0ba5c.
2018-06-10 00:01:28 +02:00
Alexander Neumann
321efec60c Fix integration tests on Windows 2018-06-10 00:00:22 +02:00
Alexander Neumann
33dbd0ba5c Fix integration tests on Windows 2018-06-09 23:58:44 +02:00
Alexander Neumann
9a73869c27 Update docs for RHEL/CentOS 2018-06-09 23:41:40 +02:00
Alexander Neumann
8f26fe271c ls: Use walker for ls 2018-06-09 23:35:20 +02:00
Alexander Neumann
251335f124 Add entry to changelog 2018-06-09 23:35:20 +02:00
Alexander Neumann
081743d0a5 find: Use walker.Walk 2018-06-09 23:35:20 +02:00
Alexander Neumann
3a86f4852b Add walker for trees in the repo 2018-06-09 23:35:20 +02:00
Alexander Neumann
14aead94b3 filter: Allow double wildcard in ChildMatch 2018-06-09 23:18:13 +02:00
Alexander Neumann
ce01ca30d6 find: Correct tree pruning optimization
The `find` command will now take care to only mark trees as "not found"
when the pattern couldn't be found within any subtree.

Closes #1825, #1823
2018-06-09 18:59:13 +02:00
Alexander Neumann
e2d347a698 find: Use OS independent slash-based format 2018-06-09 18:58:13 +02:00
Alexander Neumann
42ebb0a0a6 backup: Parse timestamp earlier 2018-06-09 18:21:12 +02:00
Alexander Neumann
419acad3c3 Merge pull request #1837 from restic/fix-1833
cache: Ensure failed downloads are retried
2018-06-09 18:20:21 +02:00
Alexander Neumann
810b5ea076 Add entry to changelog 2018-06-09 17:55:51 +02:00
Alexander Neumann
fc5439a37a cache: Ensure failed downloads are retried
This fixes #1833, which consists of two different bugs:

 * The `defer` in `cacheFile()` may remove a channel from the
   `inProgress` map although it is not responsible for downloading the
   file

 * If the download fails, goroutines waiting for the file to be cached
   assumed that the file was there, there was no way to signal the
   error.
2018-06-09 17:50:56 +02:00
Alexander Neumann
48aab8bd65 Merge pull request #1836 from restic/update-blazer
Update github.com/kurin/blazer
2018-06-09 14:31:12 +02:00
Alexander Neumann
6fbcd1694b Add entry to changelog 2018-06-09 14:31:02 +02:00
Alexander Neumann
494fe2a8b5 Merge pull request #1835 from restic/fix-1834
backup: Fix deadlock
2018-06-09 14:28:16 +02:00
Alexander Neumann
f761068f4e Update github.com/kurin/blazer 2018-06-09 12:32:18 +02:00
Alexander Neumann
c44e808aa5 backup: Fix deadlock
When the archiver is faster than the scanner, restic deadlocks. This
commit adds a `finished` channel to the struct in `ui/backup.go` so that
scanner results are ignored when the archiver is already finished.

Closes #1834
2018-06-09 12:15:19 +02:00
Alexander Neumann
ab37c6095a Merge pull request #1821 from michaelkoetter/fix-1795
#1795 use unix.IoctlGetWinsize to get terminal size
2018-06-07 20:20:06 +02:00
Michael Kötter
d6fd94e49d Don't run Solaris build for go1.9 2018-06-04 15:04:50 +02:00
Michael Kötter
53040a2e34 add "solaris/amd64" to cross-compile archs 2018-06-04 12:51:34 +02:00
Alexander Neumann
cfc19b4582 Merge pull request #1828 from restic/handle-s3-list-errors
s3: Pass list errors up to the caller
2018-06-02 10:34:49 +02:00
Alexander Neumann
141fabdd09 s3: Pass list errors up to the caller 2018-06-01 22:15:23 +02:00
Alexander Neumann
d49ca42771 Merge pull request #1827 from restic/azure-large-files
azure: Support uploading large files
2018-06-01 18:37:26 +02:00
Alexander Neumann
f6fded729d Add entry to changelog 2018-06-01 14:52:16 +02:00
Alexander Neumann
465700595c azure: Support uploading large files
Closes #1822
2018-06-01 14:52:16 +02:00
Alexander Neumann
0fcd9d6926 Merge pull request #1824 from rfjakob/ssh_command_exited
sftp: persist "ssh command exited" error
2018-05-31 21:26:39 +02:00
Jakob Unterwurzacher
dd3b9910ee sftp: persist "ssh command exited" error
If our ssh process has died, not only the next, but all subsequent
calls to clientError() should indicate the error.

restic output when the ssh process is killed with "kill -9":

  Save(<data/afb68adbf9>) returned error, retrying after 253.661803ms: Write: failed to send packet header: write |1: file already closed
  Save(<data/afb68adbf9>) returned error, retrying after 580.752212ms: ssh command exited: signal: killed
  Save(<data/afb68adbf9>) returned error, retrying after 790.150468ms: ssh command exited: signal: killed
  Save(<data/afb68adbf9>) returned error, retrying after 1.769595051s: ssh command exited: signal: killed
  [...]
  error in cleanup handler: ssh command exited: signal: killed

Before this patch:

  Save(<data/de698d934f>) returned error, retrying after 252.84163ms: Write: failed to send packet header: write |1: file already closed
  Save(<data/de698d934f>) returned error, retrying after 660.236963ms: OpenFile: failed to send packet header: write |1: file already closed
  Save(<data/de698d934f>) returned error, retrying after 568.049909ms: OpenFile: failed to send packet header: write |1: file already closed
  Save(<data/de698d934f>) returned error, retrying after 2.428813824s: OpenFile: failed to send packet header: write |1: file already closed
  [...]
  error in cleanup handler: failed to send packet header: write |1: file already closed
2018-05-30 19:28:14 +02:00
Alexander Neumann
185b60c22b Document project governance 2018-05-28 22:29:06 +02:00
Michael Kötter
589c23dc23 #1795 use unix.IoctlGetWinsize to get terminal size 2018-05-27 23:44:48 +02:00
Alexander Neumann
0183fea926 Merge pull request #1820 from restic/fix-1803
termstatus: Fix panic for non-terminal runs
2018-05-27 13:08:25 +02:00
Alexander Neumann
7d9642523b termstatus: Fix panic for non-terminal runs
Closes #1803
2018-05-27 12:52:01 +02:00
Alexander Neumann
4bf07a74a0 Merge pull request #1806 from mholt/patch-1
doc: Clarify multiple forget policies get ORed
2018-05-26 11:03:22 +02:00
Alexander Neumann
2a976d795f b2: Remove extra error check 2018-05-26 10:12:30 +02:00
Alexander Neumann
1892b314f8 Merge pull request #1815 from restic/update-blazer
Update github.com/kurin/blazer
2018-05-25 20:46:35 +02:00
Alexander Neumann
b7bed406b9 Update github.com/kurin/blazer 2018-05-25 20:26:26 +02:00
Matt Holt
ee4202f7c3 doc: Clarify multiple forget policies get ORed 2018-05-23 17:28:02 -06:00
Alexander Neumann
4cd28713b6 Merge pull request #1802 from restic/rclone-add-limits
rclone: Add limiting bandwidth to the rclone backend
2018-05-22 21:19:52 +02:00
Alexander Neumann
e3fe87f269 Remove superseded feature from the CHANGELOG
For a discussion please see https://github.com/restic/restic/issues/1796
2018-05-22 20:57:07 +02:00
Alexander Neumann
a02698fcdd Add entry to changelog 2018-05-22 20:48:29 +02:00
Alexander Neumann
bfd923e81e rclone: Respect bandwith limits 2018-05-22 20:48:17 +02:00
Alexander Neumann
20bfed5985 Update build.go 2018-05-21 20:31:19 +02:00
Alexander Neumann
e40191942d Add VERSION for 0.9.0 2018-05-21 15:57:32 +02:00
Alexander Neumann
abd34ab76f Update manpages and auto-completion 2018-05-21 15:57:32 +02:00
Alexander Neumann
4b43a269ee Generate CHANGELOG.md for 0.9.0 2018-05-21 15:57:22 +02:00
Alexander Neumann
e2b7dc6528 Move changelog files for the 0.9.0 release 2018-05-21 15:57:02 +02:00
Alexander Neumann
d2431b667f Merge pull request #1794 from restic/fix-1789
Update docs, clarify --cacert
2018-05-21 12:35:06 +02:00
Alexander Neumann
b70fdf61c4 Merge pull request #1791 from restic/fix-1790
archiver: Fix backup from stdin
2018-05-21 12:07:55 +02:00
Alexander Neumann
e6f25c4811 Update docs, clarify --cacert
Closes #1789
2018-05-21 12:06:30 +02:00
Alexander Neumann
adb682bc43 archiver: Don't open files with O_NONBLOCK
This is not necessary any more, we're doing an lstat() before opening
an item, so we already known it's a file and not a pipe.
2018-05-20 16:11:51 +02:00
Alexander Neumann
1e9744c9a4 archiver: Refuse to save an empty snapshot 2018-05-20 16:11:51 +02:00
Alexander Neumann
9a02f17cc2 archiver: Add tests for Save() for fs.Reader 2018-05-20 16:11:36 +02:00
Alexander Neumann
c284712cae Merge pull request #1784 from restic/improve-error-list
list: Improve error message
2018-05-18 21:53:09 +02:00
Alexander Neumann
2dbdf381b2 Merge pull request #1782 from skriss/add-s3-file-creds
Add S3 file creds and reorder creds chain
2018-05-18 21:52:54 +02:00
Alexander Neumann
a1a49ce211 Merge pull request #1787 from tyll/patch-1
Use more descriptive long option
2018-05-18 21:51:24 +02:00
Till Maas
3252e4200c Use more descriptive long option
-r could also mean recursive so use --repo to be clear.
2018-05-18 19:25:56 +02:00
Alexander Neumann
8d9d218d1c list: Improve error message
Before:

    $ restic list
    Fatal: type not specified

After:

    $ restic list
    Fatal: type not specified, usage: list [blobs|packs|index|snapshots|keys|locks]

Closes #1783
2018-05-17 19:41:56 +02:00
Steve Kriss
0785fbd418 changelog entry
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-17 10:09:38 -07:00
Steve Kriss
b358dd369b S3: rearrange credentials chain to be standard
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-16 16:49:33 -07:00
Steve Kriss
d67b9a32c6 S3: add file credentials to chain
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-16 16:35:14 -07:00
Alexander Neumann
ecfe59235e Merge pull request #1781 from McKael/fix_restick_backup_rejectByDevice
Fix restic backup --one-file-system /
2018-05-16 21:58:58 +02:00
Mikael Berthe
a868a30f4d Fix restic backup --one-file-system /
This patch should fix the following panic when trying to backup the
root filesystem with thre --one-file-system flag:

    % restic backup --one-file-system /
    (...)
    panic: item /, device id 2082 not found, allowedDevs: map[/:2082]
2018-05-16 13:44:14 +02:00
Alexander Neumann
347a645450 Fix double error message 2018-05-15 11:03:33 +02:00
Alexander Neumann
9f5565b0fc Merge pull request #1735 from mholt/forget-max-age
forget: Add --max-age policy to set hard cutoff for removing snapshots
2018-05-14 21:18:06 +02:00
Alexander Neumann
fd979ab4c5 Merge pull request #1779 from restic/improve-error
Improve error message for readdir/readdirnames
2018-05-14 21:17:57 +02:00
Alexander Neumann
375868edcf Add documentation 2018-05-13 12:54:23 +02:00
Alexander Neumann
060d8b57e0 Restructure TestApplyPolicy 2018-05-13 12:48:42 +02:00
Alexander Neumann
cc627e832b Add custom Duration type 2018-05-13 12:48:42 +02:00
Alexander Neumann
5a0f0e3faa Add support for keeping a range of snapshots 2018-05-13 12:48:42 +02:00
Matthew Holt
b52f2aa9a4 forget: Add policy to keep snapshots before a date 2018-05-13 12:48:10 +02:00
Alexander Neumann
60ea2435be Improve error message for readdir/readdirnames
As mentioned in the forum[1], restic does not include the dir name when
readdir/readdirnames fails.

[1] https://forum.restic.net/t/readdirnames-readdirent-no-such-file-or-directory/653
2018-05-13 10:34:50 +02:00
Alexander Neumann
159badf5ba Merge pull request #1778 from restic/fix-1771
archiver: Improve error handling
2018-05-13 00:13:54 +02:00
Alexander Neumann
903a3a31dc Merge pull request #1776 from restic/fix-1775
Always use cleaned path for excludes
2018-05-13 00:13:52 +02:00
Alexander Neumann
548227e6df Merge pull request #1773 from ExoUNX/patch-1
Fix small typo
2018-05-13 00:13:49 +02:00
Alexander Neumann
cd03275005 Merge pull request #1774 from ExoUNX/patch-2
Fix formatting
2018-05-13 00:13:46 +02:00
Alexander Neumann
e43c9202a6 archiver: Make sure backend error is passed up 2018-05-12 23:55:59 +02:00
Alexander Neumann
c5e75d1c98 archiver: Add test for early abort on unhandled error 2018-05-12 23:55:59 +02:00
Alexander Neumann
526956af35 archiver: Read files/dirs in order 2018-05-12 23:55:54 +02:00
Alexander Neumann
256104111d archiver: Clarify names 2018-05-12 23:55:54 +02:00
Alexander Neumann
21c83b1725 archiver: Add high-level documentation 2018-05-12 23:55:54 +02:00
Alexander Neumann
581c62ee72 archiver: Improve error handling
This commit changes how the worker goroutines for saving e.g. blobs
interact. Before, it was possible to get stuck sending an instruction to
archive a file or dir when no worker goroutines were available any more.
This commit introduces a `done` channel for each of the worker pools,
which is set to the channel returned by `tomb.Dying()`, so it is closed
when the first worker returned an error.
2018-05-12 23:55:54 +02:00
Alexander Neumann
ef7747313d backup: Use absolute paths for allowed devices 2018-05-11 21:32:44 +02:00
Alexander Neumann
18d4ac2fd9 backup: Always use cleaned path for excludes 2018-05-11 21:32:41 +02:00
Alexander Neumann
9180e2c48a Remove unneeded file excludes 2018-05-11 21:26:10 +02:00
Gaige Lama
a63989afcd Ran gofmt on backend.go
I ran gofmt on backend.go, this appears to valid edit.
2018-05-11 11:07:16 -04:00
Gaige Lama
d3c0bd6d0e Fix small typo
"explicitely" should be "explicitly"
2018-05-11 10:55:12 -04:00
Alexander Neumann
fcfa6f0355 build: Add option to enable PIE build mode 2018-05-11 09:50:10 +02:00
Alexander Neumann
580f90d745 Merge pull request #1770 from restic/handle-blob-save-errors
archiver: Correct error handling
2018-05-10 22:36:00 +02:00
Alexander Neumann
c7b624ba0d Merge pull request #1769 from McKael/cat_snapshot_stacktrace
cat snapshot: Do not display a stack trace with invalid IDs
2018-05-10 21:41:04 +02:00
Alexander Neumann
ca4af43c03 archiver: Return low-level errors
This commit changes the archiver so that low-level errors saving data to
the repo are returned to the caller (instead of being handled by the
error callback function). This correctly bubbles up errors like a full
temp file system and makes restic abort early and makes all other worker
goroutines exit.
2018-05-10 21:30:09 +02:00
Alexander Neumann
1f2463f42e archiver: Return correct error 2018-05-10 20:48:00 +02:00
Alexander Neumann
157c854d04 backup: Remove errored files from status display 2018-05-10 20:48:00 +02:00
Mikael Berthe
ffc276a603 cat: Do not display a stack trace
Don't show a stack trace when "restic cat snapshot" is invoked with
invalid/nonexistent IDs.
2018-05-10 19:22:46 +02:00
Alexander Neumann
e42b7db008 Merge pull request #1767 from restic/update-blazer
Update github.com/kurin/blazer
2018-05-08 22:48:54 +02:00
Alexander Neumann
024148cac9 Update github.com/kurin/blazer 2018-05-08 22:33:55 +02:00
Alexander Neumann
8943741a0b Fix code block 2018-05-08 20:36:58 +02:00
Alexander Neumann
95c5517c35 Merge pull request #1765 from restic/update-deps
Update golang.org/x/{sys,crypto}
2018-05-08 19:34:30 +02:00
Alexander Neumann
06179a7e81 Change URL for Travis 2018-05-08 19:13:39 +02:00
Alexander Neumann
cf1fb50f9c Merge pull request #1764 from restic/fix-display
Fix display
2018-05-07 23:30:50 +02:00
Alexander Neumann
6793300850 Raise minimum Go version to 1.9
The golang.org/sys/crypto library uses math/bits, which was only
introduced in Go 1.9.
2018-05-07 23:28:48 +02:00
Alexander Neumann
2cbdfbf652 Update golang.org/x/{sys,crypto} 2018-05-07 22:47:39 +02:00
Alexander Neumann
b2208bb9c2 Rework termstatus
This now keeps the cursor at the first column of the first status line
so that messages printed to stdout or stderr by some other part of the
progarm will still be visible. The message will overwrite the status
lines, but those are easily reprinted on the next status update.
2018-05-07 21:40:07 +02:00
Alexander Neumann
4c25495d68 backup: Hide percent until total size is known 2018-05-06 20:20:25 +02:00
Alexander Neumann
abdd59ea1b Merge pull request #1757 from restic/fix-debug-1755
Fix panic for nil-pointer dereference
2018-05-03 21:11:37 +02:00
Alexander Neumann
05ca903d48 Mention that first backup will take longer 2018-05-03 20:57:30 +02:00
Alexander Neumann
fd77646f8b Fix panic for nil-pointer dereference
Closes #1755
2018-05-03 20:49:30 +02:00
Alexander Neumann
2a67258867 Merge pull request #1754 from restic/fix-quiet
backup: Disable status output for --quiet
2018-05-02 21:42:17 +02:00
Alexander Neumann
fca4fe4459 backup: Disable status output for --quiet 2018-05-02 21:24:18 +02:00
Alexander Neumann
26757ae2e5 Merge pull request #1750 from restic/archiver-fix-small-things
backup: Tune user interface a bit
2018-05-01 23:15:54 +02:00
Alexander Neumann
9d6890a236 Merge pull request #1751 from restic/fix-archiver
archiver: Use lstat before open/fstat
2018-05-01 23:15:32 +02:00
Alexander Neumann
2218ecd049 archiver: Use lstat before open/fstat
The previous code tried to be as efficient as possible and only do a
single open() on an item to save, and then fstat() on the fd to find out
what the item is (file, dir, other). For normal files, it would then
start reading the data without opening the file again, so it could not
be exchanged for e.g. a symlink.

This behavior starts the watchdog on my machine when /dev is saved
with restic, and after a few seconds, the machine reboots.

This commit reverts the behavior to the strategy the old archiver code
used: run lstat(), then decide what to do. For normal files, open the
file and then run fstat() on the fd to verify it's still a normal file,
then start reading the data.

The downside is that for normal files we now do two stat() calls
(lstat+fstat) instead of only one. On the upside, this does not start
the watchdog. :)
2018-05-01 23:05:50 +02:00
Alexander Neumann
d0974c155d backup: Tune user interface a bit
Make non-verbose mode a bit more verbose.
2018-05-01 22:18:32 +02:00
Alexander Neumann
8026e6fdfb Merge pull request #1749 from restic/add-cache-cmd
Add 'cache' command to list and cleanup cache dirs
2018-05-01 21:25:52 +02:00
Alexander Neumann
01f9662614 Merge pull request #1748 from restic/detect-bom
Respect Encoding and Byte Order Mark when reading text files
2018-05-01 21:25:46 +02:00
Alexander Neumann
f928aeec34 Add 'cache' command 2018-05-01 16:27:44 +02:00
Alexander Neumann
f77bc0fae8 Use textfile.Read() to read files
This converts the text to UTF-8 and removes a Byte Order Mark.
2018-05-01 16:25:53 +02:00
Alexander Neumann
eb6650b201 Add textfile 2018-05-01 14:38:59 +02:00
Alexander Neumann
bc68d55e94 fs: Add TestTempFile 2018-05-01 14:38:41 +02:00
Alexander Neumann
ecbbd851a1 Merge pull request #1744 from restic/fix-parent-detect
backup: Clean target paths before finding parent
2018-05-01 11:13:19 +02:00
Alexander Neumann
336719b058 Merge pull request #1746 from brycied00d/fix_tls-client-cert_argument_parsing
http backend: Parse the correct argument when loading --tls-client-cert
2018-05-01 10:22:10 +02:00
Bryce Chidester
e9f1721678 http backend: Parse the correct argument when loading --tls-client-cert
Previously, the function read from ARGV[1] (hardcoded) rather than the
value passed to it, the command-line argument as it exists in globalOptions.

Resolves #1745
2018-04-30 15:21:09 -07:00
Alexander Neumann
64d98945a6 backup: Clean target paths before finding parent
This resolves an issue described in the forum where restic could not
find a parent snapshot if the target path ends with a slash:
https://forum.restic.net/t/new-archiver-code-please-test/623/23
2018-04-30 22:03:11 +02:00
Alexander Neumann
84f82dae1a Correct typo
Closes #1743
2018-04-30 21:41:30 +02:00
Alexander Neumann
6bfd9f833b Merge pull request #1742 from tbm/typos
Fix typos in section on rclone
2018-04-30 19:21:19 +02:00
Martin Michlmayr
bb1a22d1e6 Fix typos in section on rclone 2018-04-30 18:32:10 +02:00
Alexander Neumann
438719f269 Merge pull request #1741 from restic/improve-archiver
Improve archiver, fix hang on fifos
2018-04-30 16:00:10 +02:00
Alexander Neumann
c83c03ed63 archiver: Fix blocking on pipes 2018-04-30 15:34:58 +02:00
Alexander Neumann
19b9c881ca fs: Add O_NONBLOCK 2018-04-30 15:34:58 +02:00
Alexander Neumann
4e34325035 archiver: Process dirs concurrently 2018-04-30 15:13:28 +02:00
Alexander Neumann
78bd591c7c archiver: Improve buffer pool 2018-04-30 15:13:28 +02:00
Alexander Neumann
39ac12f6ea archiver: Correct comment 2018-04-30 14:19:07 +02:00
Alexander Neumann
400730afca archiver: Improve memory usage, tune buffer pool 2018-04-30 14:19:07 +02:00
Alexander Neumann
d80e108b03 backup: Clear status lines on finish 2018-04-30 14:19:07 +02:00
Alexander Neumann
846c2b6869 backup: Fix ETA calculation for >= 100% 2018-04-30 14:19:07 +02:00
Alexander Neumann
d8bbe5dc84 Print repository ID after opening 2018-04-30 14:19:07 +02:00
Alexander Neumann
d926b9fd80 Add profile build flag 2018-04-30 14:19:04 +02:00
Alexander Neumann
4ba8d40282 Add block profile option 2018-04-30 14:18:45 +02:00
Alexander Neumann
4fb1401266 Fix --cacert help text 2018-04-29 13:17:33 +02:00
Alexander Neumann
6d4c40f8d0 Add section about file read concurrency 2018-04-28 22:39:56 +02:00
Alexander Neumann
56e394ac33 Merge pull request #1494 from restic/new-archiver
New archiver code
2018-04-28 22:24:39 +02:00
Alexander Neumann
c3cc5d7cee Update docs 2018-04-28 22:08:11 +02:00
Alexander Neumann
6b12b92339 Add entry to changelog 2018-04-28 22:08:11 +02:00
Alexander Neumann
16c314ab7f termstatus: Don't print status if in background 2018-04-28 22:08:11 +02:00
Alexander Neumann
1449d7dc29 Remove background checking code 2018-04-28 22:08:11 +02:00
Alexander Neumann
0e78ac92d8 Use new archiver code for backup 2018-04-28 22:08:11 +02:00
Alexander Neumann
c703d21d55 Vendor gopkg.in/tomb.v2 2018-04-27 21:42:15 +02:00
Alexander Neumann
1af96fc6dd Add termstatus 2018-04-27 21:42:15 +02:00
Alexander Neumann
9fac2ca832 Add flags to set verbosity 2018-04-25 14:42:45 +02:00
Alexander Neumann
a5c0cf2324 Add workaround for symlinked temp dir on darwin
Chdir to the tempdir, then use os.Getwd() to get the name that
filepath.Abs() uses (and stores in the Snapshot).
2018-04-25 14:42:45 +02:00
Alexander Neumann
38926d8576 Use new archiver code in tests 2018-04-25 14:42:45 +02:00
Alexander Neumann
f279731168 Add new archiver code 2018-04-25 14:42:45 +02:00
Alexander Neumann
76b616451f Remove unneeded code 2018-04-23 21:40:33 +02:00
Alexander Neumann
fd12a3af20 Remove old archiver code 2018-04-23 21:40:33 +02:00
Alexander Neumann
3cd92efdcf Vendor github.com/mattn/go-isatty 2018-04-22 11:37:05 +02:00
Alexander Neumann
b804279fe8 integration tests: Don't print anything to stdout 2018-04-22 11:37:05 +02:00
Alexander Neumann
a56b8fad87 repository: Improve buffer pooling 2018-04-22 11:37:05 +02:00
Alexander Neumann
4c00efd4bf Vendor go-cmp 2018-04-22 11:37:05 +02:00
Alexander Neumann
b6f98bdb02 node: Fill minimal info 2018-04-22 11:37:05 +02:00
Alexander Neumann
c4b2486b7c fs: Add interface and FS implementations
This adds two implementations of the new `FS` interface: One for the local
file system (`Local`) and one for a single file read from an
`io.Reader` (`Reader`).
2018-04-22 11:37:05 +02:00
Alexander Neumann
83ca08245b checker: Check metadata size and blob sizes 2018-04-22 11:37:05 +02:00
Alexander Neumann
a069467e72 ls: Improve output 2018-04-22 11:37:05 +02:00
Alexander Neumann
6a7c23d2ae tree: Add convenience functions 2018-04-22 11:37:05 +02:00
Alexander Neumann
cc847a3d6d tree: Improve error for pre-existing node 2018-04-22 11:37:05 +02:00
Alexander Neumann
baebf45e2e FindLatestSnapshot: Use absolute paths 2018-04-22 11:37:05 +02:00
Alexander Neumann
fa4f438bc1 snapshot: Do not modify slice of paths 2018-04-22 11:37:05 +02:00
Alexander Neumann
4e0b2a8e3a snapshot: correct error handling for filepath.Abs 2018-04-22 11:37:05 +02:00
Alexander Neumann
0532f08048 Add test.Helper, also works with Go 1.8 2018-04-22 11:37:05 +02:00
Alexander Neumann
a472868e06 fs: Add TestChdir() 2018-04-22 11:37:05 +02:00
Alexander Neumann
e4fdc5eb76 fs: Add IsRegularFile() 2018-04-22 11:37:05 +02:00
Alexander Neumann
09365cc4ea Add --trace-profile 2018-04-22 11:37:05 +02:00
Eri Bastos
2aa6b49651 Return exit code 130 when SIGINT is received 2018-04-20 21:09:50 +02:00
Alexander Neumann
7877797c7e Merge pull request #1720 from mholt/new-password-flag
key: Add --new-password flag for non-interactive password changes
2018-04-20 14:52:41 +02:00
Matthew Holt
1a26355dbe Add changelog file 2018-04-20 14:29:39 +02:00
Matthew Holt
c5829e9ffc key: Add flag for non-interactive password changes 2018-04-20 14:29:39 +02:00
Alexander Neumann
b5b246edd5 Add entry to changelog 2018-04-20 14:23:37 +02:00
Alexander Neumann
ee5e14d536 Merge pull request #1731 from restic/fix-1730
Do not restore sockets, correct error handling
2018-04-20 14:21:01 +02:00
Alexander Neumann
09bd924710 Do not restore sockets, correct error handling
Closes #1730
2018-04-20 13:53:11 +02:00
Alexander Neumann
a9c2e84ccd Merge pull request #1718 from ardichoke/patch-1
Update Autocompletion  Generation Documentation
2018-04-13 22:23:37 +02:00
Ryan DeShone
a144b81c4a Update Autocompletion Generation Documentation
The autocompletion command has been changed to generate. Update documentation to reflect this change.
2018-04-12 11:50:26 -04:00
Alexander Neumann
3c453a4217 Merge pull request #1715 from rawtaz/patch-1
Add "Including and Exluding Files" heading in docs
2018-04-11 21:27:22 +02:00
rawtaz
1e2f23d77a Add "Including and Exluding Files" heading in docs
Adds "Including and Exluding Files" heading in the backup section in the docs.

[ci skip]
2018-04-11 21:14:26 +02:00
Alexander Neumann
2c76e724ab Merge pull request #1712 from restic/list-ignore-non-existing-dirs
Improve handling non-existing dirs in List()
2018-04-10 21:54:00 +02:00
Alexander Neumann
577faa7570 local/sftp: Handling non-existing dirs in List() 2018-04-10 21:35:30 +02:00
Alexander Neumann
6a34e0d10f Merge pull request #1709 from restic/improve-check-errors
Improve error message for orphaned pack files
2018-04-07 13:38:41 +02:00
Alexander Neumann
b08f21cdc6 Add entry to changelog 2018-04-07 13:05:44 +02:00
Alexander Neumann
1c1fede399 Improve error message for orphaned pack files 2018-04-07 10:07:54 +02:00
Alexander Neumann
63a0913e6e Merge pull request #1705 from alirazeen/doc-add-scripting
Add scripting section to docs
2018-04-07 09:47:17 +02:00
Ali Razeen
325957443e Add scripting section to docs 2018-04-04 10:43:20 -04:00
Alexander Neumann
4b2d3b15a2 Add entry to changelog 2018-04-03 20:42:06 +02:00
Alexander Neumann
4e2a87c920 Merge pull request #1703 from ebastos/issue1608
Fixed issue #1608 - Use --time argument properly
2018-04-03 20:40:41 +02:00
Eri Bastos
901e1b129c Fixed issue #1608 - Use --time argument properly
Backups via stdin will now handle --time argument and pass it down as
expected
2018-04-03 14:40:42 -03:00
Alexander Neumann
4478d633e2 Merge pull request #1702 from restic/update-simple-scrypt
Update github.com/elithrar/simple-scrypt
2018-04-02 20:07:18 +02:00
Alexander Neumann
92f516b1d4 Update github.com/elithrar/simple-scrypt
For details see #1697
2018-04-02 19:48:25 +02:00
Alexander Neumann
03193e6d92 Fix changelog entries 2018-04-02 12:42:48 +02:00
Alexander Neumann
01fe719aff check: Make sure temp cache dir is removed 2018-04-01 18:09:53 +02:00
Alexander Neumann
2c964df3e2 Merge pull request #1699 from restic/fix-incremental-backup-test
Slightly increment size for TestIncrementalBackup
2018-04-01 14:33:12 +02:00
Alexander Neumann
8919125b0b Merge pull request #1696 from restic/fix-check-cache
check: Improve cache handling
2018-04-01 14:33:00 +02:00
Alexander Neumann
1f5137aa70 Add entry to CHANGELOG 2018-04-01 13:59:27 +02:00
Alexander Neumann
a95eb33616 check: Use cache in temporary directory if possible
Closes #1694
2018-04-01 13:59:27 +02:00
Alexander Neumann
e68a7fea8a check: Allow filling the cache during check
Closes #1665
2018-04-01 13:59:27 +02:00
Alexander Neumann
2e7ec717c1 repository: Move cache preparation into function 2018-04-01 13:59:27 +02:00
Alexander Neumann
22d5061df2 Merge pull request #1698 from restic/reduce-backend-tests-travis
Travis: Skip cloud backend tests for most Go versions
2018-04-01 13:58:38 +02:00
Alexander Neumann
4544a77172 Slightly increment size for TestIncrementalBackup
This should make the test more reliable, it should hit the accidental
"repo is has grown too much" way less often.
2018-04-01 13:49:42 +02:00
Alexander Neumann
b3a073e066 Travis: Skip cloud backend tests for most Go versions 2018-04-01 13:13:38 +02:00
Alexander Neumann
b077a1227b Merge pull request #1657 from restic/rclone-backend
Rclone backend
2018-04-01 10:56:10 +02:00
Alexander Neumann
3f48e0e0f4 Add extra options to rclone
For details see https://github.com/restic/restic/pull/1657#issuecomment-377707486
2018-04-01 10:34:30 +02:00
Alexander Neumann
86f4b03730 Remove unneeded byte counters 2018-04-01 10:18:38 +02:00
Alexander Neumann
c43c94776b rclone: Make concurrent connections configurable 2018-04-01 10:18:38 +02:00
Alexander Neumann
0b776e63e7 backend/rclone: Request random file name
When `/` is requested, rclone returns the list of all files in the
remote, which is not what we want (and it can take quite some time).
2018-04-01 10:18:38 +02:00
Alexander Neumann
360ff1806a doc: Fix instructions for rclone backend 2018-04-01 10:18:38 +02:00
Alexander Neumann
1beeb7d0dd doc/REST: Make documentation match reality 2018-04-01 10:18:38 +02:00
Alexander Neumann
e978b36713 doc: Add hint how to debug rclone 2018-04-01 10:18:38 +02:00
Alexander Neumann
737d93860a Extend first timeout to 60 seconds. 2018-04-01 10:18:38 +02:00
Alexander Neumann
011217e4bf backend/rclone: Improve documentation and README 2018-04-01 10:18:38 +02:00
Alexander Neumann
362d5afec4 Add entry to changelog 2018-04-01 10:18:38 +02:00
Alexander Neumann
4172fcd167 doc: Add rclone backend 2018-04-01 10:18:38 +02:00
Alexander Neumann
518bf4e5f6 doc: Correct verbatim text in the manual 2018-04-01 10:18:38 +02:00
Alexander Neumann
17312d3a98 backend/rest: Ensure base URL ends with slash
This makes it easier for rclone.
2018-04-01 10:18:38 +02:00
Alexander Neumann
4d5c7a8749 backend/rclone: Make sure rclone terminates 2018-04-01 10:18:38 +02:00
Alexander Neumann
fc0295016a Address code review comments 2018-04-01 10:18:38 +02:00
Alexander Neumann
99b62c11b8 backend/rclone: Stop rclone in case of errors 2018-04-01 10:18:38 +02:00
Alexander Neumann
6d9a029e09 backend/rclone: Prefix all error messages 2018-04-01 10:18:38 +02:00
Alexander Neumann
20352886f3 Update Gopkg.lock 2018-04-01 10:18:38 +02:00
Alexander Neumann
3622b60c13 CI: Check that rclone backend test isn't skipped 2018-04-01 10:16:31 +02:00
Alexander Neumann
065fe1e54f backend/rclone: Skip test if binary is unavailable 2018-04-01 10:16:31 +02:00
Alexander Neumann
4dc0f24b38 backend/tests: Drain reader before returning error 2018-04-01 10:16:31 +02:00
Alexander Neumann
fe99340e40 Add rclone backend 2018-04-01 10:16:31 +02:00
Alexander Neumann
e377759c81 rest: Export Backend struct 2018-04-01 10:16:31 +02:00
Alexander Neumann
61f6db25f4 CI: install rclone 2018-04-01 10:16:31 +02:00
Alexander Neumann
cabbbd2b14 backend/rest: Export Content-Types 2018-04-01 10:16:31 +02:00
Alexander Neumann
cf4cf94418 Move backend/sftp.StartForeground to backend/ 2018-04-01 10:16:31 +02:00
Alexander Neumann
34f27edc03 Refactor SplitShellStrings 2018-04-01 10:16:31 +02:00
Alexander Neumann
345b6c4694 Move backend/sftp.SplitShellArgs to backend/ 2018-04-01 10:16:31 +02:00
Alexander Neumann
e4a39e02d2 Merge pull request #1695 from xulongwu4/patch-2
Add instructions for installation on Solus
2018-03-31 09:14:25 +02:00
xulongwu4
432e167255 Add instructions for installation on Solus 2018-03-30 17:36:29 -04:00
Alexander Neumann
594256bfa4 Merge pull request #1693 from restic/update-deps
Update dependencies
2018-03-30 17:19:08 +02:00
Alexander Neumann
0fcb1e6b7a Merge pull request #1692 from restic/print-forget-policy
forget: Print policy
2018-03-30 17:19:05 +02:00
Alexander Neumann
38795c66c9 Update vendored library gopkg.in/yaml.v2 2018-03-30 12:53:13 +02:00
Alexander Neumann
c0960f538f Update vendored library google.golang.org/api 2018-03-30 12:51:18 +02:00
Alexander Neumann
5b6568875c Update vendored library golang.org/x/text 2018-03-30 12:50:04 +02:00
Alexander Neumann
d8dd79eb0b Update vendored library golang.org/x/sys 2018-03-30 12:48:49 +02:00
Alexander Neumann
2bdeb645b9 Update vendored library golang.org/x/sync 2018-03-30 12:47:30 +02:00
Alexander Neumann
9f2ffa3e50 Update vendored library golang.org/x/oauth2 2018-03-30 12:46:26 +02:00
Alexander Neumann
d4bab5c133 Update vendored library golang.org/x/net 2018-03-30 12:45:07 +02:00
Alexander Neumann
3473d73d0c Update vendored library github.com/spf13/cobra 2018-03-30 12:43:03 +02:00
Alexander Neumann
917cc542c9 Update vendored library github.com/sirupsen/logrus 2018-03-30 12:41:46 +02:00
Alexander Neumann
a9cf5d482a Update vendored library github.com/russross/blackfriday 2018-03-30 12:40:05 +02:00
Alexander Neumann
75946e7c58 Update vendored library github.com/pkg/xattr 2018-03-30 12:38:37 +02:00
Alexander Neumann
19035e977b Update vendored library github.com/pkg/sftp 2018-03-30 12:37:16 +02:00
Alexander Neumann
d9ba9279e0 Update vendored library github.com/ncw/swift 2018-03-30 12:35:13 +02:00
Alexander Neumann
31e156c666 Update vendored library github.com/minio/minio-go 2018-03-30 12:33:40 +02:00
Alexander Neumann
7e6fff324c Update vendored library golang.org/x/crypto 2018-03-30 12:26:26 +02:00
Alexander Neumann
e94d2da890 Update vendored library github.com/golang/protobuf 2018-03-30 11:52:18 +02:00
Alexander Neumann
874b3dbbd9 Update vendored library github.com/go-ini/ini 2018-03-30 11:51:01 +02:00
Alexander Neumann
0d01c27c9e Update vendored library github.com/dgrijalva/jwt-go 2018-03-30 11:49:18 +02:00
Alexander Neumann
30110fcfc2 Update vendored library github.com/cpuguy83/go-md2man 2018-03-30 11:48:16 +02:00
Alexander Neumann
673f0bbd6c Update vendored library github.com/cenkalti/backoff 2018-03-30 11:45:27 +02:00
Alexander Neumann
5a77b2ab49 Update vendored library github.com/Azure/azure-sdk-for-go 2018-03-30 11:42:11 +02:00
Alexander Neumann
a951e7b126 Update vendored library cloud.google.com/go 2018-03-30 11:41:12 +02:00
Alexander Neumann
d3f9c8b362 forget: Print policy 2018-03-30 10:24:26 +02:00
Alexander Neumann
a4ff591165 Update URL to template 2018-03-25 11:55:37 +02:00
Alexander Neumann
49dd70c771 Merge pull request #1686 from gtrafimenkov/minor-spelling-fixes
Correct spelling mistakes
2018-03-25 11:54:53 +02:00
Gennady Trafimenkov
a64f24029b Correct spelling mistakes 2018-03-25 00:42:33 +03:00
Alexander Neumann
0886738d24 Add entry to CHANGELOG 2018-03-24 18:40:49 +01:00
Alexander Neumann
9fc38803e0 Merge pull request #1684 from restic/fix-rest-tests
Fix rest-server tests
2018-03-24 18:39:44 +01:00
Alexander Neumann
e5c929b793 Fix rest-server tests
Since today, the rest-server needs to be explicitly told (via
`--no-auth`) that authentication is not necessary.
2018-03-24 18:06:21 +01:00
Alexander Neumann
0e0fee9c8f Update changelog template 2018-03-24 17:31:21 +01:00
Alexander Neumann
26769a39eb Merge pull request #1679 from rawtaz/version-oneliner
Make version output one line.
2018-03-23 21:14:37 +01:00
Leo R. Lundgren
923be90906 Make version output one line. 2018-03-21 21:49:03 +01:00
Alexander Neumann
84a22eac92 Merge pull request #1675 from oliver/doc-fixes
Minor doc fixes/improvements
2018-03-21 20:54:51 +01:00
Alexander Neumann
6eb1be0be4 Fix changelog files 2018-03-21 20:53:11 +01:00
Alexander Neumann
f31bbcf1a9 CI: Return error when calens fails 2018-03-21 20:53:01 +01:00
Alexander Neumann
5d09fca6a2 Merge pull request #1676 from bowensong/quiet-skip-scan
Skip archiver.Scan before backup when --quiet is set
2018-03-21 20:45:34 +01:00
Bowen Song
34671d7c9b Skip archiver.Scan before backup when --quiet is set 2018-03-20 20:44:10 +00:00
Alexander Neumann
4a524da736 FAQ: Add suggestion on how to solve sftp error 2018-03-18 21:39:11 +01:00
Oliver Gerlich
e361cc3807 040_backup.rst: add note regarding paths in --files-from argument 2018-03-18 20:51:04 +01:00
Alexander Neumann
3cd8a7bc96 Fix small things left open by #1552 2018-03-18 19:54:12 +01:00
Alexander Neumann
8206f85d2e Merge pull request #1552 from lawrencejones/use-auto-auth
Automatically load Google auth
2018-03-18 19:53:30 +01:00
Alexander Neumann
7022144e0f Merge pull request #1673 from rawtaz/doc-precompiled
doc: Add more info about using pre-compiled builds.
2018-03-18 19:50:25 +01:00
Oliver Gerlich
1bee3e01fa 040_backup.rst: fix typo 2018-03-18 17:06:21 +01:00
Oliver Gerlich
624a2d8305 040_backup.rst: quote wildcard characters
This doesn't really matter for --exclude in most cases, but it avoids
spreading bad ideas.
2018-03-18 17:05:53 +01:00
Lawrence Jones
57c6233982 dep ensure 2018-03-16 10:31:30 +00:00
Leo R. Lundgren
c161aba084 doc: Add more info about using pre-compiled builds. 2018-03-15 21:23:07 +01:00
Alexander Neumann
0279fd7212 Merge pull request #1669 from restic/make-tests-faster
Reduce test runtime
2018-03-13 19:50:34 +01:00
Alexander Neumann
dedf17f5e8 Merge pull request #1650 from copart/patch-1
Added copr repositories for RHEL/Centos/Fedora
2018-03-11 21:52:18 +01:00
Alexander Neumann
817890794d Merge pull request #1668 from restic/fix-1663
Return the first password/key which works
2018-03-11 21:51:54 +01:00
Alexander Neumann
b9ada91054 Reduce data for TestCreateSnapshot 2018-03-11 21:42:39 +01:00
Alexander Neumann
dfb6d0fced Reduce data for TestIncrementalBackup 2018-03-11 21:17:27 +01:00
Alexander Neumann
c6c1dccc53 Reduce data set for TestRestore 2018-03-11 21:10:37 +01:00
Alexander Neumann
279566bafe Reduce dataset for integration tests 2018-03-11 21:07:47 +01:00
Alexander Neumann
c67a8452f7 Disable polynomial check for chunker for tests 2018-03-11 20:59:40 +01:00
Alexander Neumann
5253ef218c Remove TestParallelSaveWithDuplication 2018-03-11 19:49:48 +01:00
Alexander Neumann
0923976909 Remove TestArchiverDuplication 2018-03-11 19:44:25 +01:00
Lawrence Jones
492baf991f Update docs and add changelog entry: Google auth
Add documentation around using default Google application credentials,
along with a changelog extra that describes the feature and the
potential impact on existing restic uses (read: none).
2018-03-11 17:12:30 +00:00
Lawrence Jones
0dfdc11ed9 Automatically load Google auth
This change removes the hardcoded Google auth mechanism for the GCS
backend, instead using Google's provided client library to discover and
generate credential material.

Google recommend that client libraries use their common auth mechanism
in order to authorise requests against Google services. Doing so means
you automatically support various types of authentication, from the
standard GOOGLE_APPLICATION_CREDENTIALS environment variable to making
use of Google's metadata API if running within Google Container Engine.
2018-03-11 17:11:25 +00:00
Alexander Neumann
54c6837ec4 Merge pull request #1651 from qbit/obsd_inst
doc: Add install instructions for OpenBSD
2018-03-11 14:49:40 +01:00
Alexander Neumann
e085713b35 Return the first password/key which works
Closes #1663
2018-03-11 14:12:21 +01:00
Alexander Neumann
e77d8c64a7 Merge pull request #1661 from restic/fix-rest-content-length
rest: Really set Content-Length HTTP header
2018-03-10 20:34:30 +01:00
Alexander Neumann
a410fa16a1 Merge pull request #1667 from restic/improve-error-config
Return error message for config decryption failure
2018-03-10 20:34:25 +01:00
Alexander Neumann
b3e1089cf9 Return error message for config decryption failure
See #1663
2018-03-09 21:05:35 +01:00
Alexander Neumann
7f8e269891 Merge pull request #1662 from ebastos/version_password
Skip checking for password file issue #1632
2018-03-09 20:42:32 +01:00
Alexander Neumann
fcc9ce81ba rest: Really set Content-Length HTTP header 2018-03-09 20:21:34 +01:00
Eri Bastos
b9d643358a Skip checking for password file existence if command is 'version' - Issue #1632 2018-03-08 17:55:03 -04:00
Alexander Neumann
ab5ef600a2 Merge pull request #1660 from ncw/rest-fix
backend/rest: check HTTP error response for List
2018-03-08 21:55:58 +01:00
Nick Craig-Wood
04c4033695 backend/rest: check HTTP error response for List
Before this change restic would attempt to JSON decode the error
message resulting in confusing `Decode: invalid character 'B' looking
for beginning of value` messages.  Afterwards it will return `List
failed, server response: 400 Bad Request (400)`
2018-03-08 10:22:43 +00:00
Alexander Neumann
de37b68baa Move all unreleased changelog entries 2018-03-05 21:31:52 +01:00
Alexander Neumann
bdc206d440 Remove unneeded mkdir 2018-03-05 21:17:57 +01:00
Alexander Neumann
efe2e792b3 Correct changelog entries 2018-03-05 21:17:52 +01:00
Alexander Neumann
6f3c23eba7 Merge pull request #1653 from restic/fix-1652
lock: Ignore invalid lock file
2018-03-05 20:49:03 +01:00
Alexander Neumann
4b34bc3210 Reformat changelog entry 2018-03-05 20:40:40 +01:00
Alexander Neumann
6ed9100aa1 Add version number hint for the changelog entry 2018-03-05 20:34:07 +01:00
Alexander Neumann
c63b02d0f1 Move changelog entries to unreleased version 2018-03-05 20:32:10 +01:00
Alexander Neumann
d0205ec889 Add entry to changelog 2018-03-05 20:22:45 +01:00
Alexander Neumann
d8dcbc89d1 lock: Ignore invalid lock file
This commit fixes a bug introduced in
e9ea268847: When an invalid lock is
encountered (e.g. if the file is empty), the code used to ignore that,
but now returns the error.

Now, invalid files are ignored for the normal lock check, and removed
when `restic unlock --remove-all` is run.

Closes #1652
2018-03-05 20:19:57 +01:00
Alexander Neumann
be0a5b7f06 Merge pull request #1649 from jasperla/solaris
Minimal set of patches to get restic working on Solaris
2018-03-05 20:00:17 +01:00
Aaron Bieber
24ce08e122 doc: Add install instructions for OpenBSD 2018-03-05 06:31:57 -07:00
copart
864eaeab7c Added copr repositories for RHEL/Centos/Fedora 2018-03-04 15:50:30 -05:00
Jasper Lievisse Adriaanse
96311d1a2b Add support for illumos/Solaris
This does come without xattr/fuse support at this point.

NB: not hooking up the integration tests as restic won't compile without
    cgo with Go < 1.10.
2018-03-04 20:11:29 +00:00
Alexander Neumann
da77f4a2e2 Merge pull request #1647 from duzvik/aws-session-token
Change priority of AWS credential providers to accept AWS_SESSION_TOKEN
2018-03-04 20:54:56 +01:00
denis.uzvik
6bb1bcce03 Change priority of AWS credential providers to accept AWS_SESSION_TOKEN 2018-03-04 19:58:27 +02:00
Alexander Neumann
6edf28d1e1 Merge pull request #1639 from restic/improve-backend-save
backend: Improve/Cleanup Save()
2018-03-04 13:35:50 +01:00
Alexander Neumann
929afc63d5 Use int64 for the length in the RewindReader 2018-03-04 10:40:42 +01:00
Alexander Neumann
99f7fd74e3 backend: Improve Save()
As mentioned in issue [#1560](https://github.com/restic/restic/pull/1560#issuecomment-364689346)
this changes the signature for `backend.Save()`. It now takes a
parameter of interface type `RewindReader`, so that the backend
implementations or our `RetryBackend` middleware can reset the reader to
the beginning and then retry an upload operation.

The `RewindReader` interface also provides a `Length()` method, which is
used in the backend to get the size of the data to be saved. This
removes several ugly hacks we had to do to pull the size back out of the
`io.Reader` passed to `Save()` before. In the `s3` and `rest` backend
this is actively used.
2018-03-03 15:49:44 +01:00
Alexander Neumann
58306bfabb Merge pull request #1648 from duzvik/s3-bucketexists
Ignore s3 AccessDenied error, during creation of repository
2018-03-02 22:38:33 +01:00
denis.uzvik
f6890210aa Add entry to changelog 2018-03-02 11:06:06 +02:00
denis.uzvik
5873ab4031 Ignore s3 AccessDenied error, during creation of repository 2018-03-02 10:47:20 +02:00
Alexander Neumann
ab7a3a803d Update build.go from github.com/fd0/build-go 2018-02-28 21:19:28 +01:00
Alexander Neumann
1e868933c5 build.go: Allow specifying the temp dir to use 2018-02-27 21:56:42 +01:00
Alexander Neumann
21f67a0a13 Fix linebreak in GitHub changelog template 2018-02-26 21:41:37 +01:00
Alexander Neumann
272ccec7e1 Add VERSION for 0.8.3 2018-02-26 21:32:16 +01:00
Alexander Neumann
68bf1509bd Update manpages and auto-completion 2018-02-26 21:32:16 +01:00
Alexander Neumann
cfccd67600 Generate CHANGELOG.md for 0.8.3 2018-02-26 21:32:07 +01:00
Alexander Neumann
bc461d32e0 Add release date for 0.8.3 2018-02-26 21:31:50 +01:00
Alexander Neumann
ee4bfdf954 changelog: Fix spelling 2018-02-26 21:27:32 +01:00
Alexander Neumann
3037894f62 Add entry to changelog 2018-02-26 21:20:41 +01:00
Alexander Neumann
89075bdf6d Merge pull request #1643 from restic/fix-1641
Ignore files in the repo with invalid names
2018-02-26 21:18:05 +01:00
Alexander Neumann
c323f73bf9 Ignore files in the repo with invalid names
Closes #1641
2018-02-26 20:53:38 +01:00
Alexander Neumann
aef5e03731 Merge pull request #1638 from restic/fix-list-retry
backend/retry: return worker function error and abort
2018-02-25 21:20:08 +01:00
Alexander Neumann
fc1f74d32d Merge pull request #1640 from restic/fix-1637
mount: Ignore non-existing locks
2018-02-25 14:00:04 +01:00
Alexander Neumann
7d59df1ab8 mount: Ignore non-existing locks
Closes #1637
2018-02-25 13:11:03 +01:00
Alexander Neumann
2866f3f31c Add pull request to changelog entry 2018-02-24 14:53:46 +01:00
Alexander Neumann
dc1154c8ad Merge pull request #1556 from ifedorenko/check-subset
Add --read-data-subset flag to check command
2018-02-24 14:53:20 +01:00
Alexander Neumann
35a816e8ab Add entry to changelog 2018-02-24 13:34:42 +01:00
Alexander Neumann
93210614f4 backend/retry: return worker function error and abort
This is a bug fix: Before, when the worker function fn in List() of the
RetryBackend returned an error, the operation is retried with the next
file. This is not consistent with the documentation, the intention was
that when fn returns an error, this is passed on to the caller and the
List() operation is aborted. Only errors happening on the underlying
backend are retried.

The error leads to restic ignoring exclusive locks that are present in
the repo, so it may happen that a new backup is written which references
data that is going to be removed by a concurrently running `prune`
operation.

The bug was reported by a user here:
https://forum.restic.net/t/restic-backup-returns-0-exit-code-when-already-locked/484
2018-02-24 13:26:13 +01:00
Alexander Neumann
dfd37afee2 Merge pull request #1636 from kurin/pack-header
Refactor the eager-header reads for readability.
2018-02-23 17:36:45 +01:00
Toby Burress
08a5281bd4 Incorporate PR review comments. 2018-02-22 17:37:10 +00:00
Toby Burress
cdb48a8970 Add tests for the eager-header refactor. 2018-02-22 01:14:04 +00:00
Toby Burress
4fd5f0b8a9 Refactor the eager-header reads for readability.
This pulls the header reads into a function that works in terms of the
number of records requested.  This preserves the existing logic of
initially reading 15 records and then falling back if that fails.

In the event of a header with more than 15 records, it will read all
records, including the already-seen final 15 records.
2018-02-22 00:45:40 +00:00
Alexander Neumann
92ad6bf74f Add pull request to changelog 2018-02-21 19:52:16 +01:00
Alexander Neumann
2c7dd3edf4 Merge pull request #1635 from ifedorenko/1633-negative-load-offset
Fixed unexpected 'pack file cannot be listed' error
2018-02-21 19:51:38 +01:00
Igor Fedorenko
19e7803ac6 Fixed unexpected 'pack file cannot be listed' error
Fixes #1633

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-02-20 21:28:57 -05:00
Alexander Neumann
9f0605766c Add entry to changelog 2018-02-20 22:10:52 +01:00
Alexander Neumann
1a5d7a9965 Merge pull request #1634 from restic/update-blazer
Update github.com/kurin/blazer to 0.3.0
2018-02-20 22:01:30 +01:00
Alexander Neumann
296769355d Update github.com/kurin/blazer to 0.3.0
This commit will reduce the number of HTTP requests per file uploaded
from two to one.
2018-02-20 21:01:21 +01:00
Igor Fedorenko
07d080830e Add --read-data-subset flag to check command
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-02-18 23:31:27 -05:00
Alexander Neumann
c99eabfb37 Merge pull request #1625 from restic/update-blazer
Update B2 client library (github.com/kurin/blazer)
2018-02-18 19:18:52 +01:00
Alexander Neumann
842fe43590 Update github.com/kurin/blazer to 0.2.2 2018-02-18 14:53:23 +01:00
Alexander Neumann
be02008025 Merge pull request #1611 from qbit/master
On OpenBSD only root can set sticky bit
2018-02-18 12:57:54 +01:00
Alexander Neumann
29da86b473 Merge pull request #1623 from restic/backend-relax-restrictions
backend: Relax requirement for new files
2018-02-18 12:56:52 +01:00
Alexander Neumann
bad7215696 Add entry to CHANGELOG 2018-02-18 12:04:44 +01:00
Alexander Neumann
881ff5e554 Move changelog file, improve text 2018-02-18 11:51:45 +01:00
Alexander Neumann
86b7fd0335 Merge pull request #1624 from rawtaz/patch-1
Attempt to make issue template a bit clearer
2018-02-18 11:50:11 +01:00
rawtaz
70209d7d1d End both list items with a dot.
Changed my mind after checking other lists in the project's files, ending with a dot seems to be preferred here, and I like that better too.
2018-02-17 23:47:40 +01:00
rawtaz
f07552161c Remove potentially excessive dot
Some people like list items to not end with a comma or dot, some like it when they do. To keep things like and coherent I removed them in this case.
2018-02-17 23:40:30 +01:00
rawtaz
856f3a9135 Add forum URL back 2018-02-17 23:36:47 +01:00
Alexander Neumann
49e9bcadb7 Merge pull request #1560 from ifedorenko/1559-load-error-handling
Retry all repository file download errors
2018-02-17 23:25:28 +01:00
rawtaz
1b8823ef2e Attempt to make issue template a bit clearer 2018-02-17 23:19:58 +01:00
Alexander Neumann
b5062959c8 backend: Relax requirement for new files
Before, all backend implementations were required to return an error if
the file that is to be written already exists in the backend. For most
backends, that means making a request (e.g. via HTTP) and returning an
error when the file already exists.

This is not accurate, the file could have been created between the HTTP
request testing for it, and when writing starts. In addition, apart from
the `config` file in the repo, all other file names have pseudo-random
names with a very very low probability of a collision. And even if a
file name is written again, the way the restic repo is structured this
just means that the same content is placed there again. Which is not a
problem, just not very efficient.

So, this commit relaxes the requirement to return an error when the file
in the backend already exists, which allows reducing the number of API
requests and thereby the latency for remote backends.
2018-02-17 22:39:18 +01:00
Alexander Neumann
2fb4d44a4d Add VERSION for 0.8.2 2018-02-17 20:27:28 +01:00
Alexander Neumann
3c08f956a3 Update manpages and auto-completion 2018-02-17 20:27:28 +01:00
Alexander Neumann
1463a7aaaa Generate CHANGELOG.md for 0.8.2 2018-02-17 20:27:17 +01:00
Alexander Neumann
76b8e080c3 changelog: Add release date for 0.8.2 2018-02-17 20:26:43 +01:00
Alexander Neumann
a9f9243afc scripts/release: Fix usage for restic/builder
The restic/builder Docker container now has a slightly different usage.
2018-02-17 19:10:32 +01:00
Alexander Neumann
34bf9de339 Fix Go versions for Travis 2018-02-17 18:48:15 +01:00
Alexander Neumann
597f6be4e4 Disable CI tests on Go 1.8 on OS X 2018-02-17 18:39:16 +01:00
Alexander Neumann
aeeec0ff94 Update Go version for CI tests 2018-02-17 18:37:37 +01:00
Igor Fedorenko
ab040d8811 Introduced repository.DownloadAndHash helper
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-02-16 21:13:11 -05:00
Igor Fedorenko
d58ae43317 Reworked Backend.Load API to retry errors during ongoing download
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-02-16 21:12:14 -05:00
Alexander Neumann
b723094739 scripts/release: Check changelog subdir with date 2018-02-16 23:13:36 +01:00
Alexander Neumann
84dcc3c396 Update changelog/ for new version of calens
We've reworked calens[1] a bit, the changelog/releases file is not
needed any more. Insteady, the release date is stored in the dir name
which contains the changelog entries.

[1] https://github.com/restic/calens
2018-02-16 23:05:58 +01:00
Alexander Neumann
21747bc219 Remove punctuation from changelog entry title 2018-02-16 22:49:24 +01:00
Alexander Neumann
28b00891d2 scripts/release: Check Go version in container 2018-02-16 22:35:53 +01:00
Alexander Neumann
b4e00c324f scripts/release: Check release date first 2018-02-16 21:52:34 +01:00
Alexander Neumann
4eb9df63cf Merge pull request #1616 from rawtaz/fix-backup-doc
Restructure and complement exclude options doc
2018-02-14 22:11:14 +01:00
Alexander Neumann
fde674d214 Merge pull request #1615 from rawtaz/fix-forget-doc
Fix #1519 and formatting in `forget` doc
2018-02-14 22:11:10 +01:00
rawtaz
fbb08c14de Note that multiple --exclude-file may be used. 2018-02-14 22:05:52 +01:00
Leo R. Lundgren
8b95b48bae Fix grammar and formatting. 2018-02-14 02:59:47 +01:00
Leo R. Lundgren
71a4ccea71 Document refusal to process empty forget policy. 2018-02-14 02:59:41 +01:00
Leo R. Lundgren
5a9c7c51e6 Fix --keep-* options list formatting. 2018-02-14 02:59:33 +01:00
rawtaz
5abd133e26 Restructure and complement exclude options doc 2018-02-14 02:03:04 +01:00
Alexander Neumann
2debb5c0e0 Merge pull request #1613 from restic/strip-space-files-from
Strip spaces from items read via --files-from
2018-02-11 21:35:23 +01:00
Alexander Neumann
54c0794cf3 Merge pull request #1603 from jfparis/master
Add support for GOARM parameter when cross compiling
2018-02-11 21:03:29 +01:00
Alexander Neumann
cff3f3dc0d Add entry to CHANGELOG 2018-02-11 21:00:38 +01:00
Alexander Neumann
c5575c7ed9 Strip spaces from items read via --files-from
In #1590, it was mentioned that while lines read from exclude files via
`--exclude-file` have leading and trailing spaces stripped, this is not
the case for lines read via `--files-from`. This commit fixes that,
spaces are always stripped.
2018-02-11 20:56:11 +01:00
Aaron Bieber
99d88ad297 Disable the 'testSticky' test on OpenBSD. Only root can set sticky. 2018-02-11 07:46:31 -07:00
Alexander Neumann
7f0d964a05 Update build.go 2018-02-11 11:34:03 +01:00
Alexander Neumann
514f1b8917 Relax timeout backend test 2018-02-10 12:53:38 +01:00
Alexander Neumann
a7a99be460 Correct forum host name in issue template
Closes #1609
2018-02-10 11:13:22 +01:00
Alexander Neumann
e17f65b23a Update logo 2018-02-05 20:48:48 +01:00
Jean-François Paris
e86fb5eedd Add support for GOARM parameter when cross compiling
Go can cross compile to older ARM architectures (e.g. v5). In order to
do so one needs to use the GOARM parameter as documented in
https://github.com/golang/go/wiki/GoArm
2018-02-04 14:15:12 +00:00
Alexander Neumann
c8cb9a9509 Merge pull request #1579 from ifedorenko/RetryBackend-List
Implement RetryBackend.List()
2018-01-30 21:39:21 +01:00
Igor Fedorenko
aa333f4d49 Implement RetryBackend.List()
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-29 22:14:12 -05:00
Alexander Neumann
c40002246d Merge pull request #1598 from Phaeilo/s3-tutorial
Update AWS S3 tutorial, fixing #1593
2018-01-29 21:52:32 +01:00
Philip Huppert
77a6054d64 Update AWS S3 tutorial, fixing #1593 2018-01-29 20:47:27 +01:00
Alexander Neumann
94ec55ff2c Merge pull request #1595 from restic/remove-read-bandwidth
Remove archiver progress "data processed" bandwith
2018-01-28 10:15:53 +01:00
Alexander Neumann
2369da158f Merge pull request #1592 from ncw/helpful-tests
Make backend tests more helpful
2018-01-28 10:09:35 +01:00
Alexander Neumann
5379950c64 Add entry to changelog 2018-01-28 09:36:55 +01:00
Alexander Neumann
cbfa516575 Remove archiver progress "data processed" bandwith
This commit removes the bandwidth displayed during backup process. It is
misleading and seldomly correct, because it's neither the "read
bandwidth" (only for the very first backup) nor the "upload bandwidth".
Many users are confused about (and rightly so), c.f. #1581, #1033, #1591

We'll eventually replace this display with something more relevant when
 #1494 is done.
2018-01-28 09:30:26 +01:00
Nick Craig-Wood
fb62da1748 Make backend tests more helpful
* In TestList check that backend is empty first
  * Improve error message in TestBackend
2018-01-27 21:36:35 +00:00
Alexander Neumann
b1b1f6e04f Merge pull request #1594 from restic/gcs-use-http-transport
GS: Use generic http transport
2018-01-27 22:16:49 +01:00
Alexander Neumann
05f826a4fc Update Gopkg.lock 2018-01-27 21:48:41 +01:00
Alexander Neumann
649afb1198 Add entry to changelog 2018-01-27 20:16:51 +01:00
Alexander Neumann
5dc8d3588d GS: Use generic http transport
During the development of #1524 I discovered that the Google Cloud
Storage backend did not yet use the HTTP transport, so things such as
bandwidth limiting did not work. This commit does the necessary magic to
make the GS library use our HTTP transport.
2018-01-27 20:12:34 +01:00
Alexander Neumann
9d2aa0a71c Merge pull request #1524 from brycied00d/https_client_certificate
Support for TLS client certificate authentication
2018-01-27 20:12:09 +01:00
Alexander Neumann
c34db983d8 Read TLS client cert and key from the same file 2018-01-27 14:02:01 +01:00
Bryce Chidester
e805b968b1 Support for TLS client certificate authentication
This adds --tls-client-cert and --tls-client-key parameters and enables use
of that certificate/key pair when connecting to https servers.
2018-01-27 13:18:22 +01:00
Alexander Neumann
e706f1a8d1 Merge pull request #1589 from restic/fix-intermediate-index-upload
archiver: Fix intermediate index upload
2018-01-27 12:22:11 +01:00
Alexander Neumann
4219bfbcc9 Add entry to changelog 2018-01-26 22:05:53 +01:00
Alexander Neumann
f99c95c766 archiver: Fix intermediate index upload
A user discovered[1] that when the backup finishes during the upload of
an intermediate index, the upload is cancelled and the index never fully
saved, but the snapshot is saved and the backup finalizes without an
error. This lead to a situation where a snapshot references data that is
contained in the repo, but not referenced in any index, leading to
strange error messages.

This commit uses a dedicated context to signal the intermediate index
uploading routine to terminate after the last index has been uploaded.
This way, an upload running when the backup finishes is completed before
the routine terminates and the snapshot is saved.

[1] https://forum.restic.net/t/error-loading-tree-check-prune-and-forget-gives-error-b2-backend/406
2018-01-26 22:01:07 +01:00
Alexander Neumann
cccb2fc7e7 Merge pull request #1583 from restic/close-open-backend-files
Close backend files in case of errors
2018-01-26 21:57:28 +01:00
Alexander Neumann
ff3de66ddf Merge pull request #1582 from restic/optimize-debug-log
Optimize debug logs
2018-01-26 21:57:18 +01:00
Alexander Neumann
fb0d01ef1f Add pull request URL to changelog 2018-01-26 21:56:59 +01:00
Alexander Neumann
e11a183578 Merge pull request #1588 from restic/fix-sftp-without-tty
sftp: Allow running ssh without a tty
2018-01-26 21:56:41 +01:00
Alexander Neumann
ee6732d2c6 Add entry to changelog 2018-01-26 21:56:29 +01:00
Alexander Neumann
4b2f2b542d Merge pull request #1584 from restic/limit-index-file-size
Limit index file size
2018-01-26 21:53:12 +01:00
Alexander Neumann
6eb2d76435 index: Lower parallel load to 4 2018-01-26 21:10:38 +01:00
Alexander Neumann
f63c7048c7 index: Limit index files to 3000 pack files 2018-01-26 21:07:16 +01:00
Alexander Neumann
ca3c566a99 index: Check structure after saving a new index 2018-01-26 21:07:16 +01:00
Alexander Neumann
7719cf88d9 b2: Check timeout 2018-01-26 21:07:05 +01:00
Alexander Neumann
00e905ebe6 sftp: Allow running ssh without a tty 2018-01-26 19:21:14 +01:00
Alexander Neumann
164acceb55 Appveyor: Update Go version and download URL 2018-01-25 21:09:23 +01:00
Alexander Neumann
909d9273cc Close backend files in case of errors 2018-01-25 21:05:57 +01:00
Alexander Neumann
d62a664bb7 Merge pull request #1580 from ifedorenko/tests-errors-cause
Use errors.Cause in backend TestListCancel
2018-01-25 20:55:43 +01:00
Alexander Neumann
663c57ab4d debug: Remove manual Str() call Log() 2018-01-25 20:49:41 +01:00
Alexander Neumann
ed99f53786 debug: Auto-shorten IDs passed as parameters 2018-01-25 20:49:26 +01:00
Igor Fedorenko
abc4027083 Use errors.Cause in backend TestListCancel
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-25 08:53:50 -05:00
Alexander Neumann
d6212ee2d9 Fix entry to changelog 2018-01-24 20:54:51 +01:00
Alexander Neumann
9c55e8d69c Merge pull request #1549 from MJDSys/more_index_lookup_avoids
More optimizations to avoid calling Index.Lookup()
2018-01-24 20:53:30 +01:00
Alexander Neumann
1c9fca6d14 Combine changelog files for #1574 and #1575 2018-01-24 20:07:22 +01:00
Alexander Neumann
be205563be Merge pull request #1575 from ifedorenko/1567_optimize-repository-ListPack
Optimize Repository.ListPack()
2018-01-24 20:01:15 +01:00
Alexander Neumann
fc6c341b26 Merge pull request #1574 from ifedorenko/1567_optimize-pack-readHeader
Optimize pack readHeader() implementation
2018-01-24 19:54:14 +01:00
Alexander Neumann
f86c141b83 Merge pull request #1577 from EdwardBetts/patch-1
Correct a spelling mistake.
2018-01-24 19:44:49 +01:00
Alexander Neumann
a6d4888d48 Merge pull request #1573 from restic/update-dps
Update dependencies
2018-01-24 19:43:32 +01:00
Alexander Neumann
44550a88a0 Merge pull request #1571 from restic/rework-backend-list
Rework backend list
2018-01-24 19:43:07 +01:00
Edward Betts
b5c23695c9 Correct a spelling mistake. 2018-01-24 12:09:41 +00:00
Igor Fedorenko
0084e42cb6 Optimize Repository.ListPack()
Use pack file size returned by Backend.List() to avoid extra per-pack
Backend.Stat() requests

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-23 22:39:51 -05:00
Matthew Dawson
fe33c05a20 debug/log: Add benchmarks for calling the logging function
Add some benchmarks for calling Log, both with a static string
along with calling the ID.Str and ID.String functions.
2018-01-23 22:30:00 -05:00
Matthew Dawson
3789e55e20 repostiory/index: Remove logging from Lookup function.
The logging in these functions double the time they take to execute.
However, it is only really useful on failures, which are better
reported by the calling functions.

benchmark                                            old ns/op     new ns/op     delta
BenchmarkMasterIndexLookupSingleIndex-6              897           395           -55.96%
BenchmarkMasterIndexLookupMultipleIndex-6            2001          1090          -45.53%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       492           215           -56.30%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     1649          912           -44.69%

benchmark                                            old allocs     new allocs     delta
BenchmarkMasterIndexLookupSingleIndex-6              9              1              -88.89%
BenchmarkMasterIndexLookupMultipleIndex-6            19             1              -94.74%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       6              0              -100.00%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     16             0              -100.00%

benchmark                                            old bytes     new bytes     delta
BenchmarkMasterIndexLookupSingleIndex-6              160           96            -40.00%
BenchmarkMasterIndexLookupMultipleIndex-6            240           96            -60.00%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       48            0             -100.00%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     128           0             -100.00%
2018-01-23 22:28:38 -05:00
Matthew Dawson
4cec7e236a Add Changelog 2018-01-23 22:28:32 -05:00
Matthew Dawson
3a16148447 archiver/archiver: Use Index.Has() instead of Index.Lookup() in isKnownBlob
Index.Has() is a faster then Index.Lookup() for checking if a blob exists
in the index.  As the returned data is never used, this avoids a ton
of allocations.
2018-01-23 22:26:10 -05:00
Matthew Dawson
df2c03a6a4 repository/master_index: Optimize Index.Lookup()
When looking up a blob in the master index, with several
indexes present in the master index, a significant amount of time
is spent generating errors for each failed lookup.  However, these
errors are often used to check if a blob is present, but the contents
are not inspected making the overhead of the error not useful.

Instead, change Index.Lookup (and Index.LookupSize) to instead return
a boolean denoting if the blob was found instead of an error.  Also change
all the calls to these functions to handle the new function signature.

benchmark                                            old ns/op     new ns/op     delta
BenchmarkMasterIndexLookupSingleIndex-6              820           897           +9.39%
BenchmarkMasterIndexLookupMultipleIndex-6            12821         2001          -84.39%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       5378          492           -90.85%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     17026         1649          -90.31%

benchmark                                            old allocs     new allocs     delta
BenchmarkMasterIndexLookupSingleIndex-6              9              9              +0.00%
BenchmarkMasterIndexLookupMultipleIndex-6            59             19             -67.80%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       22             6              -72.73%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     72             16             -77.78%

benchmark                                            old bytes     new bytes     delta
BenchmarkMasterIndexLookupSingleIndex-6              160           160           +0.00%
BenchmarkMasterIndexLookupMultipleIndex-6            3200          240           -92.50%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       1232          48            -96.10%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     4272          128           -97.00%
2018-01-23 22:25:56 -05:00
Matthew Dawson
ebce4b2581 repository/index: Speed up benchmarks and tests
When setting up the index used for benchmarking, use math/rand instead of
crypto/rand since the generated ids don't need to be evenly distributed,
and not be secure against guessing.  As such, use a different random id
function (only available during tests) that uses math/rand instead.
2018-01-23 22:21:05 -05:00
Igor Fedorenko
953f3d55ee Optimize pack readHeader() implementation
Load pack header length and 15 header entries with single backend
request. This eliminates separate header Load() request for most pack
files and significantly improves index.New() performance.

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-23 22:02:25 -05:00
Alexander Neumann
7e6bfdae79 backend/rest: Implement REST API v2 2018-01-23 23:15:26 +01:00
Alexander Neumann
0f4cbea27d Document the REST v2 protocol 2018-01-23 23:15:26 +01:00
Alexander Neumann
59782e347c repository: Fix repack test
This reduces the chance of duplicate blobs, otherwise the tests fail
(make the contents of a blob depend on a pseudo-random number instead of
the size, sizes may be duplicate).
2018-01-23 23:14:05 +01:00
Alexander Neumann
e835abeceb backend/test: Reliably trigger timeout error 2018-01-23 23:14:05 +01:00
Alexander Neumann
8e812b7ac0 errors: Make Cause() unwrap *url.Error 2018-01-23 23:14:05 +01:00
Alexander Neumann
685ce719ad Add comments for List() 2018-01-23 23:14:05 +01:00
Alexander Neumann
2b39f9f4b2 Update dependencies
Among others, this updates minio-go, so that the new "eu-west-3" zone
for AWS is supported.
2018-01-23 19:40:42 +01:00
Alexander Neumann
b63de7c798 script/release: Tag versioned docker image 2018-01-21 21:41:14 +01:00
Alexander Neumann
b0c6e53241 Fix calls to repo/backend.List() everywhere 2018-01-21 21:15:09 +01:00
Alexander Neumann
e9ea268847 Change List() implementation for all backends 2018-01-21 21:15:09 +01:00
Alexander Neumann
dd91b13ff3 Vendor golang.org/x/sync 2018-01-21 18:35:37 +01:00
Alexander Neumann
c4e9d5d11e backend: Add tests for new List() function 2018-01-21 18:35:37 +01:00
Alexander Neumann
52230b8f07 backend: Rework List()
For a discussion see #1567
2018-01-21 18:35:37 +01:00
Alexander Neumann
a3d43a92b3 Merge pull request #1569 from restic/ext-rest-backend-test
REST: refactor backend tests
2018-01-21 18:02:09 +01:00
Alexander Neumann
2130897ce0 rest: Add test for external server 2018-01-20 10:25:47 +01:00
Alexander Neumann
67da240068 rest: Refactor backend tests 2018-01-20 10:25:37 +01:00
Alexander Neumann
1046eabf95 rest: Remove unneeded tempdir 2018-01-20 10:13:04 +01:00
Alexander Neumann
663e8284b2 init: Correct text (backend vs. repository)
Closes #1565
2018-01-20 09:51:49 +01:00
Alexander Neumann
5a43ca1ccb Add entry to CHANGELOG 2018-01-20 09:50:31 +01:00
Alexander Neumann
200415e0a1 Merge pull request #1564 from restic/sftp-password-prompt
sftp: Prompt for password, don't terminate on SIGINT
2018-01-20 09:48:17 +01:00
Alexander Neumann
ea50687e8b Merge pull request #1568 from martinseener/patch-1
Added hint where to place restic.exe on windows
2018-01-20 09:47:50 +01:00
Alexander Neumann
7335a2492f Add PR to changelog 2018-01-19 13:05:55 +01:00
Alexander Neumann
bcb6881ffb Merge pull request #1548 from ifedorenko/checker-backend-Test
checker: Optimize checker.Packs()
2018-01-19 13:05:35 +01:00
Alexander Neumann
37df829b3d CONTRIBUTING: clarify wording 2018-01-19 13:00:49 +01:00
Martin
39f1ba1897 Added hint where to place restic.exe on windows
I've added a hint for pre-compiled binaries on windows where to place them for easier use in the shell or in scripts without the hassle for absolute paths.
2018-01-19 12:24:09 +01:00
Igor Fedorenko
231076fa4a checker: Optimize checker.Packs()
Use result of single repository.List() to find both missing and
orphaned data packs. For 500GB repository this eliminates ~100K
repository.Test() calls and improves check time by >30M in my
environment (~45min before this change and ~7min after).

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-18 20:50:39 -05:00
Alexander Neumann
0bdb131521 Remove SuspendSignalHandler 2018-01-17 23:14:47 +01:00
Alexander Neumann
c31a5e7e5c Add argument to Skipf() 2018-01-17 23:14:37 +01:00
Alexander Neumann
05958caf6e sftp: Prompt for password, don't terminate on SIGINT
This is a follow-up on fb9729fdb9, which
runs the `ssh` in its own process group and selects that process group
as the foreground group. After the sftp connection is established,
restic switches back to the previous foreground process group.

This allows `ssh` to prompt for the password, but it won't receive
the interrupt signal (SIGINT, ^C) later on, because it is not in the
foreground process group any more, allowing a clean tear down.
2018-01-17 23:02:47 +01:00
Alexander Neumann
87e31799f2 Merge pull request #1483 from dstosberg/master
document how to create a full backup without running restic as root
2018-01-16 17:21:55 +01:00
Alexander Neumann
35528506a6 Merge pull request #1554 from restic/fix-988
fuse: Correct behavior for reading after EOF, add snapshot template string
2018-01-16 17:17:55 +01:00
Alexander Neumann
2544978aaf Add entry to changelog 2018-01-15 14:47:17 +01:00
Alexander Neumann
97565d68a2 fuse/mount: Add option for snapshot template 2018-01-15 14:47:17 +01:00
Alexander Neumann
f49f5c5903 fuse: Don't return an error for EOF 2018-01-13 10:14:10 +01:00
Alexander Neumann
d89f2e5226 fuse: Remove debug log message 2018-01-13 10:10:07 +01:00
Alexander Neumann
02f4f5dc66 Update build.go, correct version detection 2018-01-12 22:06:42 +01:00
Alexander Neumann
5723636b35 Add entry for changelog 2018-01-09 21:57:07 +01:00
Alexander Neumann
5632ca4f44 Merge pull request #1538 from MJDSys/make_lookup_internal
repository/index: Avoid creating an error when checking if an id is in the index.
2018-01-09 21:55:37 +01:00
Matthew Dawson
539599d1f1 repository/index: Optimize index.Has()
When backing up several million files (>14M tested here) with few changes,
a large amount of time is spent failing to find an id in an index and creating
an error to signify this.  Since this is checked using the Has method,
which doesn't use this error, this time creating the error is wasted.

Instead, directly check if the given id and type are present in the index.
This also avoids reporting all the packs containing this blob, further
reducing cpu usage.
2018-01-08 21:46:17 +01:00
Alexander Neumann
d77a326bb0 Add benchmark for Index.Has() 2018-01-08 21:10:30 +01:00
Dennis Stosberg
60a7a9d019 manual: document how to create a full backup without running restic as root 2018-01-08 19:59:15 +01:00
Alexander Neumann
99f0fce673 Merge pull request #1536 from restic/fix-restorer-permissions
restore: Fix directory permissions
2018-01-07 17:18:29 +01:00
Alexander Neumann
b708cfee3f Add entry to changelog 2018-01-07 15:16:16 +01:00
Alexander Neumann
b8620429e2 restorer: Set directory mode as last step
Closes #1512
2018-01-07 15:13:24 +01:00
Alexander Neumann
aab414b368 Add test case for #1512 2018-01-07 14:50:47 +01:00
Alexander Neumann
e120b17abd Update PR template 2018-01-07 11:53:30 +01:00
Alexander Neumann
5f43e17918 Merge pull request #1535 from ifedorenko/check-retry
fixed restic-check does not retry backend.Test failures
2018-01-07 09:43:49 +01:00
Alexander Neumann
489eef5a6f Merge pull request #1534 from restic/improve-error-restore
restore: Improve error message for more than one ID
2018-01-07 09:40:06 +01:00
Igor Fedorenko
8c550ca011 fixed restic-check does not retry backend.Test failures
added missing RetryBackend.Test implementation

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-06 23:22:35 -05:00
Alexander Neumann
032621289b errors: Add "Fatal" prefix for Fatalf() 2018-01-06 23:27:47 +01:00
Alexander Neumann
92316a9853 restore: Improve error message for more than one ID 2018-01-06 22:44:18 +01:00
Alexander Neumann
180741609e Merge pull request #1530 from juliangehring/patch-1
Fix code block formatting in FAQs
2018-01-06 18:46:14 +01:00
Julian Gehring
70250762f3 Fix code block formatting in FAQs
Fixes the formatting of a code block in the FAQs and makes it render properly.
2018-01-05 23:02:03 +00:00
Alexander Neumann
39ba17a2d6 Add entry to CHANGELOG 2018-01-05 19:29:02 +01:00
Alexander Neumann
cfe8c8c9cd Merge pull request #1529 from restic/fix-1528
Fix creating missing dirs in local/sftp repo
2018-01-05 19:28:17 +01:00
Alexander Neumann
b45fc89512 local/sftp: Create repo dirs on demand in Save() 2018-01-05 17:51:09 +01:00
Alexander Neumann
aabc0ccaa7 fs: Make HasPathPrefix work with relative paths 2018-01-05 17:38:52 +01:00
Alexander Neumann
2bc4d200d4 Merge pull request #1518 from restic/fix-1317-again
forget: Also run prune when only IDs are forgotten
2018-01-02 20:39:12 +01:00
Alexander Neumann
0247fe01c0 Update homepage URL in README 2018-01-01 21:50:44 +01:00
Alexander Neumann
c912b38bf0 forget: Also run prune when only IDs are forgotten 2018-01-01 21:27:40 +01:00
Alexander Neumann
559946c58a Add linebreak to VERSION generation 2017-12-30 09:55:56 +01:00
Alexander Neumann
a99637c613 Merge pull request #1511 from restic/improve-http-limiting
Improve limiting HTTP based backends
2017-12-29 20:57:02 +01:00
Alexander Neumann
36501dda73 Add changelog issue 2017-12-29 12:51:43 +01:00
Alexander Neumann
18ecd9df30 Improve limiting HTTP based backends 2017-12-29 12:43:49 +01:00
Alexander Neumann
c686dd0448 Merge pull request #1507 from restic/fix-fuse-list-timeout
fuse: Only reload list of snapshots once per minute
2017-12-28 14:06:59 +01:00
Alexander Neumann
6d91d468e7 Add entry to releases 2017-12-28 13:20:19 +01:00
Alexander Neumann
5f9b5b0219 Add entry to changelog 2017-12-28 13:20:03 +01:00
Alexander Neumann
3f7d85360a fuse: Only reload list of snapshots once per minute 2017-12-28 13:18:27 +01:00
Alexander Neumann
8d8456590c Update release script, add check for changelog/releases 2017-12-27 23:32:11 +01:00
Alexander Neumann
85f9f3e290 Fix changelog template for GitHub 2017-12-27 23:26:19 +01:00
Alexander Neumann
9e1180a29b Fix changelog IDs 2017-12-27 23:22:35 +01:00
Alexander Neumann
e17c1096a0 Add date for 0.8.1 2017-12-27 22:42:15 +01:00
Alexander Neumann
546d6f36b2 Add VERSION for 0.8.1 2017-12-27 22:31:03 +01:00
Alexander Neumann
6ecd14d780 Update manpages and auto-completion 2017-12-27 22:31:03 +01:00
Alexander Neumann
f6ed7dc013 Generate CHANGELOG.md for 0.8.1 2017-12-27 22:30:57 +01:00
Alexander Neumann
e290f2591e Merge pull request #1504 from restic/changelog-generator
Replace manual CHANGELOG with generated one
2017-12-27 22:00:15 +01:00
Alexander Neumann
75f90ca303 Run calens on Travis 2017-12-27 21:21:21 +01:00
Alexander Neumann
ca1430184f Replace CHANGELOG.md with generated version 2017-12-27 21:07:26 +01:00
Alexander Neumann
a297ab9d7c Add PrimaryURL to github template 2017-12-27 19:30:04 +01:00
Alexander Neumann
f078525d98 Add changelog entries 2017-12-27 19:22:06 +01:00
Alexander Neumann
e03cc81a9a Add changelog generation to release script 2017-12-27 19:21:55 +01:00
Alexander Neumann
af27f1dde5 Merge pull request #1503 from gliptak/patch-2
Correct golint warning on return
2017-12-27 12:52:31 +01:00
Gábor Lipták
32505c3916 Correct golint warning on return 2017-12-26 17:09:41 -05:00
Alexander Neumann
9a8d5a1bff Merge pull request #1499 from ametzler/improve-docs-exclude-396
doc: Improvements for --exclude
2017-12-24 21:10:28 +01:00
Alexander Neumann
740ee787c1 Merge pull request #1501 from gliptak/patch-1
Correct typos
2017-12-24 21:10:08 +01:00
Gábor Lipták
2eba0bfeec Correct typos 2017-12-24 12:06:52 -05:00
Andreas Metzler
d780ec4bce doc: Improvements for --exclude
Describe exclude pattern handling, adding multiple examples.

Closes #396
2017-12-24 15:39:18 +01:00
Alexander Neumann
6b564d21b3 Merge pull request #1493 from restic/retry-delete
backend: Retry deletes
2017-12-23 11:53:09 +01:00
Alexander Neumann
6c2b2a58ad backend: Retry deletes 2017-12-22 22:41:28 +01:00
Alexander Neumann
b80b68dcb3 doc: Correct setence 2017-12-22 17:59:52 +01:00
Alexander Neumann
29c92ca415 Merge pull request #1491 from restic/improve-b2-config
b2: Warn when account ID or key is empty
2017-12-20 18:06:58 +01:00
Alexander Neumann
bc04ce8e6b Merge pull request #1482 from restic/fix-1478
backend: Only return top-level files for most dirs
2017-12-20 18:06:47 +01:00
Alexander Neumann
6b6b75fa4a Merge pull request #1488 from da2x/patch-1
Doc: Specify `key passwd` usage
2017-12-19 21:19:38 +01:00
Alexander Neumann
84e493beba b2: Warn when account ID or key is empty
Closes #1490
2017-12-19 21:12:38 +01:00
Daniel Aleksandersen
323376efa2 Doc: Specify key passwd usage
Specify that `key passwd` is used to change a password.
2017-12-19 04:53:31 +01:00
Alexander Neumann
e353b00501 Merge pull request #1481 from restic/add-stdin-path
backup: Reject directories in filename for --stdin
2017-12-14 19:36:11 +01:00
Alexander Neumann
2510d770ab Add question about positive things to issue template 2017-12-14 19:22:23 +01:00
Alexander Neumann
7d8765a937 backend: Only return top-level files for most dirs
Fixes #1478
2017-12-14 19:14:16 +01:00
Alexander Neumann
81a04656c5 fuse: Cleanup node name 2017-12-13 20:21:18 +01:00
Alexander Neumann
2f26fb8834 Only print cache warning for terminals 2017-12-13 19:58:10 +01:00
Alexander Neumann
d3ebe1311f backup: Reject filenames with paths read from stdin 2017-12-13 19:55:53 +01:00
Alexander Neumann
42a8c19aae Merge pull request #1476 from harshavardhana/fix
Since upgrade to minio-go 4.0 remove workaround
2017-12-09 11:45:29 +01:00
Harshavardhana
27ccea6371 Since upgrade to minio-go 4.0 remove workaround
We added previously a code to fix the issue of chaining
credentials, we do not need this anymore since the
upstream minio-go already has this relevant change.
2017-12-09 02:01:42 -08:00
Alexander Neumann
4f46b4f393 Merge pull request #1475 from restic/update-minio-go
Update to minio-go v4
2017-12-09 10:12:05 +01:00
Alexander Neumann
221e741537 Fix broken link in README 2017-12-09 10:04:39 +01:00
Alexander Neumann
8b3b7bc5ef s3: Use context 2017-12-08 22:04:55 +01:00
Alexander Neumann
934ae1b559 Update to minio-go 4 2017-12-08 21:52:50 +01:00
Alexander Neumann
0e7e3cb714 Update minio-go 2017-12-08 20:45:59 +01:00
Alexander Neumann
95b6e4e9e9 Add entry to CHANGELOG 2017-12-08 20:29:50 +01:00
Alexander Neumann
3a5e040b7e Merge pull request #1439 from armhold/propagate-context
replace ad-hoc context.TODO() with gopts.ctx
2017-12-08 20:27:36 +01:00
Alexander Neumann
28c826868b Merge pull request #1471 from restic/fix-diff
Fix diff
2017-12-06 19:47:17 +01:00
George Armhold
1695c8ed55 use global context for check, debug, dump, find, forget, init, key,
list, mount, tag, unlock commands

gh-1434
2017-12-06 07:02:55 -05:00
George Armhold
366622f09a use global context for cat and ls
gh-1434
2017-12-06 05:48:39 -05:00
George Armhold
0dc31c03e1 remove check for context.Canceled
gh-1434
2017-12-06 05:38:29 -05:00
Alexander Neumann
0405e67f8b Change diff output slightly 2017-12-05 22:26:56 +01:00
Alexander Neumann
df350e1f6e Improve diff 2017-12-05 22:05:22 +01:00
Alexander Neumann
06cb3f7058 Merge pull request #1469 from zertrin/patch-1
Update usage help output in doc/manual_rest.rst
2017-12-04 21:36:16 +01:00
Alexander Neumann
56b884be17 Merge pull request #1468 from restic/update-deps
Update dependencies
2017-12-04 21:34:52 +01:00
Alexander Neumann
a25d280f3e Merge pull request #1462 from restic/add-diff
Add diff command
2017-12-04 21:34:44 +01:00
zertrin
2253a73837 Update usage help output in doc/manual_rest.rst
The usage help in the documentation was outdated relative to current version (0.8.0) with some commands outdated and some missing.
2017-12-04 22:57:00 +08:00
Alexander Neumann
946c8399e2 Update dependenciess
Exclude minio-go for now (pin to 3.x.y).
2017-12-03 21:22:33 +01:00
Alexander Neumann
9d0f13c4c0 Prominently mention SIGPIPE issue in CHANGELOG 2017-12-03 19:12:07 +01:00
Alexander Neumann
eb9e2bc79a manual: Document diff command 2017-12-03 18:33:35 +01:00
Alexander Neumann
0722c44ba1 Add entry to CHANGELOG 2017-12-03 18:33:35 +01:00
Alexander Neumann
2424012d75 Add 'diff' command 2017-12-03 18:33:35 +01:00
Alexander Neumann
82ded35706 node: Correctly compare times 2017-12-03 18:33:35 +01:00
Alexander Neumann
69fcb604c8 Merge pull request #1465 from restic/fuse-handle-others
fuse: Handle/format all node types correctly
2017-12-03 18:25:49 +01:00
Alexander Neumann
88607fc625 Correctly format all node types 2017-12-03 17:38:55 +01:00
Alexander Neumann
7092af6329 fuse: Handle sockets/fifos/devs correctly
Closes #1463
2017-12-03 17:25:00 +01:00
Alexander Neumann
23d7d91597 Merge pull request #1464 from restic/reenable-cache
Reenable cache
2017-12-03 16:15:13 +01:00
Alexander Neumann
ad82781743 Reenable cache
In 6341c7d72c, the cache was accidentally
disabled due to a bug, this commit reenables the cache.
2017-12-03 15:52:57 +01:00
George Armhold
20d78ab0d9 add context to repo.Flush() call in test code
gh-1434
2017-12-03 07:32:50 -05:00
George Armhold
be24237063 make retry code context-aware.
detect cancellation in backend, so that retry code does not keep trying
once user has hit ^c

gh-1434
2017-12-03 07:22:14 -05:00
George Armhold
d886cb5c27 replace ad-hoc context.TODO() with gopts.ctx, so that cancellation
can properly trickle down from cmd_*.

gh-1434
2017-12-03 07:22:14 -05:00
Alexander Neumann
63bb1933e5 Merge pull request #1461 from ljani/patch-1
Fix a typo
2017-12-03 09:51:08 +01:00
Jani
81e6a9d0d0 Fix a typo 2017-12-03 10:33:46 +02:00
Alexander Neumann
5d4110d2a7 Add entry to CHANGELOG 2017-12-01 21:52:07 +01:00
Alexander Neumann
0cedb3ac9f Merge pull request #1459 from restic/improve-s3-do-spaces
Improve s3 backend for DigitalOcean Spaces
2017-12-01 21:51:30 +01:00
Alexander Neumann
0b44c629f2 retry: Remove file after failed save 2017-11-30 22:05:14 +01:00
Alexander Neumann
2579fe6b7b Ignore SIGPIPE
Handling SIGPIPE made restic abort when a TCP connection was reset by a
server. This happened on DigitalOcean Spaces, which uses the s3 backend.
2017-11-30 21:23:43 +01:00
Alexander Neumann
812ce4bfc4 Add entry to CHANGELOG 2017-11-29 18:44:03 +01:00
Alexander Neumann
410efe0694 Merge pull request #1454 from restic/fix-cache-dir
Fix cache dir detection
2017-11-29 18:43:07 +01:00
Alexander Neumann
b2d944d5cb Merge pull request #1452 from restic/add-with-atime
backup: Do not save access time by default
2017-11-29 18:42:45 +01:00
Alexander Neumann
b846c3915c Add entry to CHANGELOG 2017-11-28 21:41:38 +01:00
Alexander Neumann
ffbc68aa2e Document access time handling 2017-11-28 21:36:20 +01:00
Alexander Neumann
eddb8549ef backup: By default, do not save the access time
This can be re-enabled with `--with-atime`.
2017-11-28 21:31:35 +01:00
Alexander Neumann
bb44855078 Add pull request template 2017-11-28 19:40:27 +01:00
Alexander Neumann
2567026ccb Merge pull request #1447 from fawick/fixMultipleRejectIfPresent
Fix multiple rejectIfPresent cancelling each other
2017-11-28 19:20:29 +01:00
Alexander Neumann
0cc8fc6f18 Merge pull request #1431 from n0npax/master
fix #1411
2017-11-27 21:54:06 +01:00
Alexander Neumann
cc81b916a6 Add entry to CHANGELOG 2017-11-27 21:40:13 +01:00
Fabian Wickborn
27fadd2c6e Document approach for multiple reject-if-present test 2017-11-27 21:38:15 +01:00
Alexander Neumann
dc38265b54 Merge pull request #1436 from restic/remove-old-cache
Remove old cache directories
2017-11-27 21:37:05 +01:00
Fabian Wickborn
1ea518d5ef cmd/restic: Use a dedicated cache for each rejectIfPresent 2017-11-27 17:33:53 +01:00
Fabian Wickborn
901cd5edef cmd/restic: Add test for rejectIfPresent bug
All RejectFuncs returned by rejectIfPresent share the same rejection
cache and hence might cancel each other out.
2017-11-27 17:26:19 +01:00
Alexander Neumann
e1fd47765b Fix scripts/release.go 2017-11-26 19:59:21 +01:00
Alexander Neumann
c02923fbfc Add VERSION for 0.8.0 2017-11-26 19:46:16 +01:00
Alexander Neumann
7c5ce83044 Update manpages and auto-completion 2017-11-26 19:46:16 +01:00
Alexander Neumann
37e2e9a844 Add version to CHANGELOG 2017-11-26 19:46:01 +01:00
Alexander Neumann
26e5db1849 Merge pull request #1446 from restic/fix-relative-restore
restore: Fix restore to relative path
2017-11-26 19:03:34 +01:00
Alexander Neumann
a2766ffe0c CI: return directly after an error occurred 2017-11-26 18:50:40 +01:00
Alexander Neumann
0f5e38609f restore: Fix restore to relative path 2017-11-26 18:36:48 +01:00
Alexander Neumann
f178cbf93d Merge pull request #1445 from restic/clean-node-name
restorer: Clean node names
2017-11-26 17:34:05 +01:00
Alexander Neumann
c8096ca8d2 Add entry to CHANGELOG 2017-11-26 15:31:09 +01:00
Alexander Neumann
27d29b9853 restorer: Make sure node names are clean 2017-11-26 15:28:45 +01:00
Alexander Neumann
8a171731ba restorer: Add tests for invalid node names 2017-11-26 15:28:37 +01:00
Alexander Neumann
abde9e2fba doc: Add --cacert to REST section 2017-11-26 10:09:54 +01:00
Alexander Neumann
6a4a328bbc Merge pull request #1443 from restic/fix-init-rest-cacert
rest: Use client for creating the repository
2017-11-25 22:12:18 +01:00
Alexander Neumann
8253fadc96 doc: Fix typo 2017-11-25 22:11:47 +01:00
Alexander Neumann
134abbd82b rest: Use client for creating the repository
Before, creating a new repo via REST would use the defaut HTTP client,
which is not a problem unless the server uses HTTPS and a TLS
certificate which isn't signed by a CA in the system's CA store. In this
case, all commands work except the 'init' command, which fails with a
message like "invalid certificate".
2017-11-25 20:56:40 +01:00
Alexander Neumann
fe557b022a Add entry to CHANGELOG 2017-11-25 12:12:08 +01:00
Alexander Neumann
cd8226130a Improve issue template 2017-11-25 11:43:44 +01:00
Alexander Neumann
1ebf0e8de8 Merge pull request #1437 from restic/fix-1292
s3: Document and remove default prefix
2017-11-25 11:34:26 +01:00
Alexander Neumann
37ea764000 cache: Add more documentation 2017-11-25 11:28:59 +01:00
Alexander Neumann
0fdb9a6129 Merge pull request #1426 from pmkane/master
fix the signal name for progress reports in the docs
2017-11-24 21:58:11 +01:00
Alexander Neumann
47b326b7b5 Merge pull request #1423 from harshavardhana/creds
Fix chaining of credentials for minio-go
2017-11-24 21:57:52 +01:00
Alexander Neumann
e2cf6eb434 Merge pull request #1428 from stephengroat/patch-1
use default brew formula
2017-11-24 21:38:04 +01:00
Alexander Neumann
f79698dcdd Merge pull request #1410 from armhold/deadlock2
unify behavior for max http connections across backends
2017-11-24 21:32:56 +01:00
Alexander Neumann
35a5307db3 Merge pull request #1415 from armhold/signals
also handle SIGPIPE in cleanup routines
2017-11-24 21:28:08 +01:00
Alexander Neumann
6341c7d72c cache: Add option to remove old cache dirs 2017-11-24 20:53:28 +01:00
Alexander Neumann
f4bab789b8 cache: Simplify cache dir creation 2017-11-24 20:53:26 +01:00
Alexander Neumann
fa893ee477 cache: Add detection code for old cache dirs 2017-11-24 20:53:26 +01:00
Alexander Neumann
014cec06f1 Add entry to CHANGELOG 2017-11-21 21:33:09 +01:00
Alexander Neumann
431ab5aa6a manual: Add hint about old default prefix 2017-11-21 21:33:09 +01:00
Alexander Neumann
262b0cd9d4 s3: Remove default prefix "/restic" 2017-11-21 21:33:09 +01:00
Alexander Neumann
e83ec17e95 s3: Correct comment 2017-11-20 22:21:39 +01:00
Alexander Neumann
ea593fca1b cache: Correctly return dir for Windows/darwin 2017-11-20 06:11:18 +01:00
Alexander Neumann
fe1f151ae1 cache: Return error during default dir detection 2017-11-20 06:10:42 +01:00
n0npax
b12bba4e2a fix #1411
replace panic during index save with fatal error
2017-11-19 11:40:47 +01:00
Stephen
e2005e02bb use default brew formula 2017-11-18 08:02:41 -08:00
Harshavardhana
41c8c946ba Fix chaining of credentials for minio-go
chaining failed because chaining provider
was only looking for subsequent credentials
provider after an error. Writer a new
chaining provider which proceeds to fetch
new credentials also under situations where
providers do not return but instead return
no keys at all.

Fixes https://github.com/restic/restic/issues/1422
2017-11-18 02:51:12 -08:00
Patrick Kane
fe08686558 fix the signal name for progress reports in the docs 2017-11-16 19:17:25 -08:00
George Armhold
0ed2401711 exit 1 if received signal is other than SIGINT
send cleanup msg to stderr, not stdout
gh-1413
2017-11-09 07:16:01 -05:00
Alexander Neumann
06bd606d85 Merge pull request #1414 from bket/1307
Fix test failing on OpenBSD #1307
2017-11-08 18:54:44 +01:00
George Armhold
c347431907 also handle SIGPIPE in cleanup routines
fixes gh-1413: restic fails to cleanup locks when bash pipeline fails
2017-11-05 07:14:27 -05:00
Björn Ketelaars
f63d7048f9 Fix test failing on OpenBSD #1307
Is seems that #1307 is similar to #1087, which describes a comparable
observation on Apple's new filesystem. #1389 Has been committed and
fixes the problem on Darwin.

Although I'm not sure if the root cause of the issue is the same the
solution is similar for OpenBSD, and leverages #1389.
2017-11-05 07:27:58 +01:00
Alexander Neumann
f39f7c76dd Merge pull request #1406 from michaeldorner/master
Minor fixes on README and LICENSE
2017-11-04 09:58:21 +01:00
George Armhold
0268d0e7d6 swift backend: limit http concurrency in Save(), Stat(), Test(), Remove(),
List().

move comment regarding problematic List() backend api (it's s3's ListObjects
that has a problem, NOT swift's ObjectsWalk).

As per discussion in PR #1399.
2017-11-02 18:29:32 -04:00
George Armhold
8515d093e0 swift backend: fix premature release of semaphore in Load() & document
concurrency issue in List().

refactor wrapReader from b2 -> semaphore so it can be used elsewhere.

As per discussion in PR #1399.
2017-11-02 12:38:17 -04:00
Michael Dorner
fe3f326d8d Update LICENSE
- improved formatting of license information (BSD 2-clase) so Github can automatically detect it.
2017-11-02 11:39:49 +01:00
Michael Dorner
8170db40c7 Update README.rst
- added Codecov badge
- added link to OSI for the license
2017-11-02 11:36:25 +01:00
George Armhold
99ac0da4bc s3 backend: limit http concurrency in Save(), Stat(), Test(), Remove()
NB: List() is NOT currently limited, as it would cause deadlock due to
be.client.ListObjects() implementation.

as per discussion in PR #1399
2017-11-01 09:40:54 -04:00
Alexander Neumann
7e2c93420f Merge pull request #1397 from restic/crypto-aead
crypto: Make crypto.Key implement cipher.AEAD
2017-11-01 13:21:10 +01:00
Alexander Neumann
6d46824fb0 Pass in a nil buffer to Open() 2017-11-01 10:30:40 +01:00
Alexander Neumann
bb435b39d9 crypto: Rework Seal/Open to use sliceForAppend 2017-11-01 10:30:40 +01:00
Alexander Neumann
2a67d7a6c2 crypto: Correct test function names 2017-11-01 10:30:40 +01:00
Alexander Neumann
ba43c8bab5 crypto: Fix nonce test, make it faster 2017-11-01 10:30:40 +01:00
Alexander Neumann
931e6ed2ac Use Seal/Open everywhere 2017-11-01 10:30:40 +01:00
Alexander Neumann
a5f0e9ab65 Remove custom Encrypt/Decrypt methods 2017-11-01 10:30:40 +01:00
Alexander Neumann
6fc133ad6a Run tests on Seal/Open 2017-11-01 10:30:40 +01:00
Alexander Neumann
e1b80859f2 Make crypto.Key implement cipher.AEAD 2017-11-01 10:30:39 +01:00
George Armhold
d069ee31b2 GS backend: limit http concurrency in Save(), Stat(), Test(), Remove(), List()
as per discussion in PR #1399
2017-10-31 08:01:43 -04:00
George Armhold
981752ade0 Azure backend: limit http concurrency in Stat(), Test(), Remove()
as per discussion in PR #1399
2017-10-31 07:32:30 -04:00
Alexander Neumann
d01d07fc0a Merge pull request #1400 from armhold/deadlock2
log unexpected errs from b2 ListCurrentObject()
2017-10-30 18:39:33 +01:00
Alexander Neumann
526aaca6f5 Merge pull request #1389 from wojas/fix-modtime-tests-on-apfs
Fix modtime tests on APFS: allow 1us difference
2017-10-29 14:20:40 +01:00
George Armhold
2f8147af59 log unexpected errs from b2 ListCurrentObject()
gh-1385
2017-10-29 08:53:39 -04:00
Konrad Wojas
f3016a9096 Darwin test fix: allow 1μs timestamp difference
On Darwin, allow a 1μs difference in restored timestamps, because
macOS <10.13 cannot restore with nanosecond precision and the current
version of Go (1.9.2) does not yet support the new syscall required
for this. (#1087 #1389)
2017-10-29 20:48:07 +08:00
Alexander Neumann
f854a41ba9 Merge pull request #1399 from armhold/deadlock2
prevent deadlock in List() for B2 when b2.connections=1
2017-10-29 09:26:46 +01:00
Alexander Neumann
ca3cadef5e Merge pull request #1398 from armhold/sftp-readdir
sftp ReadDir: add path to return error messages (gh-1323)
2017-10-29 09:26:42 +01:00
George Armhold
3304b0fcf0 prevent deadlock in List() for B2 when b2.connections=1
This is a fix for the following situation (gh-1188):

List() grabs a semaphore token upon entry, starts a goroutine, and
does not release the token until the routine exits (via a defer).

The goroutine iterates over the results from ListCurrentObjects(),
sending them one at a time to a channel, where they are ultimately
processed by be.Load().

Since be.Load() also needs a token, this will result in deadlock if
b2.connections=1.

This fix changes List() so that the token is only held during the call
to ListCurrentObjects().
2017-10-28 18:46:47 -04:00
George Armhold
d8938e259a sftp ReadDir: add path to return error messages (gh-1323)
fix missing "Close" string in debug log fmt
2017-10-28 14:16:27 -04:00
Alexander Neumann
53a554c89d Merge pull request #1395 from restic/small-crypto-improvements
Small improvements
2017-10-28 12:57:18 +02:00
Alexander Neumann
e71db01230 Improve crypto test 2017-10-28 12:09:33 +02:00
Alexander Neumann
178e946fc7 Rename KDFParams -> Params 2017-10-28 10:28:29 +02:00
Alexander Neumann
f3bff12939 Merge pull request #1394 from zcalusic/master
Use lowercase in messages from check/tag commands, too
2017-10-28 09:56:29 +02:00
Alexander Neumann
7a99418dc5 Merge pull request #1393 from armhold/lint-errcheck
detect errors from fs.Walk() in local backend List()
2017-10-28 09:56:11 +02:00
Alexander Neumann
c71ba466ea Merge pull request #1391 from armhold/b2-listmax
pass in defaultListMaxItems to b2Backend constructor
2017-10-28 09:54:57 +02:00
Alexander Neumann
8ce5d35543 Merge pull request #1390 from armhold/lint
small cleanup
2017-10-28 09:54:35 +02:00
Zlatko Čalušić
134f834c60 Use lowercase in messages from check/tag commands, too 2017-10-27 21:06:34 +02:00
George Armhold
8a37c07295 send errors from fs.Walk() to debug log
clarify non-err returns from Walk where err is already proved to be nil
2017-10-27 08:41:17 -04:00
George Armhold
bd0ada7842 go fmt 2017-10-26 16:37:11 -04:00
George Armhold
eea96f652d go fmt 2017-10-26 16:22:10 -04:00
George Armhold
38c3061df7 pass in defaultListMaxItems to b2Backend constructor
gh-1385
2017-10-26 14:22:16 -04:00
George Armhold
f5fa602482 detect and return error from file Close() in Node.createFileAt()
gh-1385
2017-10-26 13:53:31 -04:00
Alexander Neumann
e44ac55f63 Merge pull request #1387 from wojas/snapshots-filter-last
Add --last flag to snapshots command
2017-10-26 19:08:49 +02:00
Alexander Neumann
f1cfb73a8b Merge pull request #1384 from wojas/reduce-ticker-refresh
Reduce ticker refresh rate with RESTIC_PROGRESS_FPS env
2017-10-26 19:02:59 +02:00
Konrad Wojas
5b96885c6d Control progress rate with RESTIC_PROGRESS_FPS env
Add a RESTIC_PROGRESS_FPS environment variable to limit the interval
at which the progress indicator updates (allowed values: 1-60).

The default rate of 60 FPS can cause high terminal CPU load on some
systems, like iTerm2 on macOS with font anti-aliasing enabled.

Usage:

    RESTIC_PROGRESS_FPS=1 restic ...
    RESTIC_PROGRESS_FPS=60 restic ...
2017-10-26 14:46:56 +08:00
Konrad Wojas
c5da90a5b7 Add --last flag to snapshots command
Add --last flag to snapshots command to only show the last entry for any
(hostname, paths) combination.

This makes it easier to check when various paths were last backed up.
2017-10-26 14:02:29 +08:00
George Armhold
bcdebfb84e small cleanup:
- be explicit when discarding returned errors from .Close(), etc.
- remove named return values from funcs when naked return not used
- fix some "err" shadowing when redeclaration not needed
2017-10-25 12:03:55 -04:00
Alexander Neumann
359b273649 Merge pull request #1381 from tbm/broken-link
Remove Markdown syntax from documentation
2017-10-24 20:38:40 +02:00
Martin Michlmayr
2e2c8dc620 Remove Markdown syntax from documentation
Convert a link from Markdown to RST syntax.  This link must
have been missed during the recent conversion.
2017-10-24 14:40:33 +01:00
Alexander Neumann
8d37b723ca Upadte vendored dependencies 2017-10-22 10:07:36 +02:00
Alexander Neumann
315b7f282f Remove explicit version constraints 2017-10-22 10:07:23 +02:00
Alexander Neumann
a3f8e9dfa7 Add entry to CHANGELOG 2017-10-21 12:22:50 +02:00
Alexander Neumann
982810f7cc Merge pull request #1368 from TobyLL/master
Ignore comments (lines starting with #) in the --files-from file
2017-10-21 12:21:11 +02:00
Alexander Neumann
90b96d19cd Merge pull request #1365 from felix9/fix_1068
Fix failure to detect some legacy s3 repos
2017-10-21 12:19:58 +02:00
Alexander Neumann
6a52bb6f54 Merge pull request #1374 from restic/update-blazer
Update github.com/kurin/blazer
2017-10-21 11:26:49 +02:00
Alexander Neumann
cacaa4393f Merge pull request #1373 from restic/check-gopkg-lock
CI: Check the vendor dir and Gopkg.lock
2017-10-21 11:14:30 +02:00
Alexander Neumann
d63ab4e9a4 Merge pull request #1358 from prattmic/chunk_size
gs: add option to set chunk size
2017-10-21 11:13:48 +02:00
Alexander Neumann
ca6daec8dd CI: Check the vendor dir and Gopkg.lock 2017-10-21 10:50:38 +02:00
Alexander Neumann
c87f2420a6 Update github.com/kurin/blazer 2017-10-21 10:30:39 +02:00
Alexander Neumann
f5bbbc52f4 Update issue template 2017-10-21 10:17:58 +02:00
TobyLL
9e3dde8ec7 Fix whitespace 2017-10-19 15:52:06 +01:00
TobyLL
9dba182e51 Ignore comments (lines starting with #) in the --files-from file 2017-10-19 15:48:22 +01:00
Felix Lee
944fc857eb Fix failure to detect some legacy s3 repos
Sometimes s3 listobjects for a directory includes an entry for that
directory. The restic s3 backend doesn't expect that and returns
an error.

Symptom is:
  ReadDir: invalid key name restic/key/, removing prefix
     restic/key/ yielded empty string

I'm not sure when s3 does that; I'm unable to reproduce it myself.

But in any case, it seems correct to ignore that when it happens.

Fixes #1068
2017-10-18 13:45:31 -07:00
Alexander Neumann
7507a658ac Merge pull request #1362 from restic/fix-backend-retry
Fix Save() backend retry
2017-10-18 07:45:56 +02:00
Michael Pratt
9fa4f5eb6b gs: disable resumable uploads
By default, the GCS Go packages have an internal "chunk size" of 8MB,
used for blob uploads.

Media().Do() will buffer a full 8MB from the io.Reader (or less if EOF
is reached) then write that full 8MB to the network all at once.

This behavior does not play nicely with --limit-upload, which only
limits the Reader passed to Media. While the long-term average upload
rate will be correctly limited, the actual network bandwidth will be
very spikey.

e.g., if an 8MB/s connection is limited to 1MB/s, Media().Do() will
spend 8s reading from the rate-limited reader (performing no network
requests), then 1s writing to the network at 8MB/s.

This is bad for network connections hurt by full-speed uploads,
particularly when writing 8MB will take several seconds.

Disable resumable uploads entirely by setting the chunk size to zero.
This causes the io.Reader to be passed further down the request stack,
where there is less (but still some) buffering.

My connection is around 1.5MB/s up, with nominal ~15ms ping times to
8.8.8.8.

Without this change, --limit-upload 1024 results in several seconds of
~200ms ping times (uploading), followed by several seconds of ~15ms ping
times (reading from rate-limited reader). A bandwidth monitor reports
this as several seconds of ~1.5MB/s followed by several seconds of
0.0MB/s.

With this change, --limit-upload 1024 results in ~20ms ping times and
the bandwidth monitor reports a constant ~1MB/s.

I've elected to make this change unconditional of --limit-upload because
the resumable uploads shouldn't be providing much benefit anyways, as
restic already uploads mostly small blobs and already has a retry
mechanism.

--limit-download is not affected by this problem, as Get().Download()
returns the real http.Response.Body without any internal buffering.

Updates #1216
2017-10-17 21:12:04 -07:00
Alexander Neumann
ce4d71d626 backend: Add partial read failure to error backend 2017-10-17 22:11:38 +02:00
Alexander Neumann
8e2ef3f38b cache: Store first, then store in the backend
Store the file in the backend, then rewind the reader and store it
in the cache afterwards.
2017-10-17 22:01:20 +02:00
Alexander Neumann
8dc952775e backend: Correctly retry Save() calls
Make sure the given reader is an io.Seeker and rewind it properly each
time.
2017-10-17 21:46:38 +02:00
Alexander Neumann
99b6163e27 Add 'dump' to manual 2017-10-16 20:24:47 +02:00
Alexander Neumann
beaf55f1fc dump: Allow absolute paths 2017-10-16 20:22:01 +02:00
Alexander Neumann
980bb9059f Add entry to changelog 2017-10-16 20:15:19 +02:00
Alexander Neumann
0e7281eb71 Merge pull request #1346 from fawick/cmdCatFile
Add dump sub-command for dumping files to stdout
2017-10-16 20:13:44 +02:00
Alexander Neumann
0b6133d7b5 Merge pull request #1360 from mungomat/fuse_bugfix
fuse: bugfix: ignore empty tags
2017-10-16 20:07:05 +02:00
Tobias Klein
b57ca64275 fuse: bugfix: ignore empty tags 2017-10-16 16:22:08 +02:00
Alexander Neumann
faadbd734b Add entry to CHANGELOG 2017-10-14 20:29:47 +02:00
Alexander Neumann
88b0a93409 Merge pull request #1353 from restic/backend-retry
Retry failed backend requests
2017-10-14 20:28:57 +02:00
Alexander Neumann
4a995105a9 sftp: Fix Delete() 2017-10-14 16:08:15 +02:00
Alexander Neumann
7fe496f983 Ensure TestDelete runs last 2017-10-14 16:04:29 +02:00
Alexander Neumann
e56370eb5b Remove Deleter interface 2017-10-14 16:04:29 +02:00
Alexander Neumann
b8af7f63a0 backend test: Always remove files for TestList 2017-10-14 15:56:25 +02:00
Alexander Neumann
3eea555155 Add dependency cenkalti/backoff 2017-10-14 15:56:25 +02:00
Alexander Neumann
897c923cc9 Retry failed backend requests 2017-10-14 15:56:25 +02:00
Alexander Neumann
67193e3deb Improve error message when creating lock failed 2017-10-14 15:56:25 +02:00
Alexander Neumann
0e722efb09 backend: Add Delete() to restic.Backend interface 2017-10-14 15:56:25 +02:00
Alexander Neumann
3736f33ebf Merge pull request #1352 from tyll/manual_links
Fix links to manual
2017-10-14 14:54:16 +02:00
Fabian Wickborn
d1d9c3f9d7 Renamed cmd_catfile.go to cmd_dump.go 2017-10-14 13:55:21 +02:00
Fabian Wickborn
cd5cbe0910 Rename debug dump related variable and run function 2017-10-14 13:55:00 +02:00
Fabian Wickborn
814e992c0b Rename subcommand catfile to dump 2017-10-14 11:34:04 +02:00
Till Maas
660fe78735 Fix links to manual 2017-10-13 20:01:19 +02:00
Fabian Wickborn
87d084e18c Add subcommand dump 2017-10-12 20:24:34 +02:00
Alexander Neumann
9ce2a73fc5 Add issue and PR to the changelog entry 2017-10-12 20:20:06 +02:00
Alexander Neumann
f2314b26ba Move 'dump' to 'debug dump' 2017-10-12 20:18:45 +02:00
Alexander Neumann
74dcf41f25 Disable 'dump' for non-debug builds 2017-10-12 20:14:48 +02:00
Alexander Neumann
b6ba30186f Merge pull request #1336 from rmdashrf/rate-limit-backup
Add basic rate limiting to backup
2017-10-12 20:08:55 +02:00
rmdashrf
32637a0328 Basic rate limiting implementation.
Added `--limit-upload` and `--limit-download` flags to rate limit
backups and restores.
2017-10-11 20:01:20 -07:00
Alexander Neumann
0addd90e14 Merge pull request #1345 from harshavardhana/creds
Refactor credentials management to support multiple mechanisms.
2017-10-11 21:01:19 +02:00
Alexander Neumann
1b5ee5b10a Merge pull request #1343 from mungomat/fuse_speedup
fuse mount: speedup
2017-10-11 20:55:52 +02:00
Harshavardhana
042adeb5d0 Refactor credentials management to support multiple mechanisms.
This PR adds the ability of chaining the credentials provider,
such that restic as a tool attempts to honor credentials from
multiple different ways.

Currently supported mechanisms are

 - static (user-provided)
 - IAM profile (only valid inside configured ec2 instances)
 - Standard AWS envs (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
 - Standard Minio envs (MINIO_ACCESS_KEY, MINIO_SECRET_KEY)

Refer https://github.com/restic/restic/issues/1341
2017-10-09 12:51:39 -07:00
Tobias Klein
7e4ce0dacc fuse mount: speedup 2017-10-09 10:26:56 +02:00
Alexander Neumann
8ceb22fe8a Add entry to CHANGELOG 2017-10-08 09:48:22 +02:00
Alexander Neumann
c5553ec855 Merge pull request #1276 from fawick/supply_ca_cert
Add REST backend option to use CA root certificate
2017-10-08 09:47:23 +02:00
Alexander Neumann
bb3ed54291 Merge pull request #1337 from mungomat/fuse_updateRepo
fuse: mount and backup in parallel (#1330)
2017-10-08 09:45:28 +02:00
Alexander Neumann
513ba3b6f7 Merge pull request #1340 from jniggemann/fix-664
doc: Add info on docker container
2017-10-08 09:44:34 +02:00
Alexander Neumann
17d688afef Merge pull request #1339 from felix9/fix_1251
fixes #1251, race when writing indexes
2017-10-07 16:36:16 +02:00
Jan Niggemann
d81eee26b3 doc: Add info on docker container
fixes #664
2017-10-07 16:03:08 +02:00
Felix Lee
cc5ada63a4 fixes #1251, race when writing indexes 2017-10-07 05:11:42 -07:00
Alexander Neumann
88fb60e0b5 Improve issue template 2017-10-07 12:16:46 +02:00
Tobias Klein
02200acad0 fuse: mount and backup in parallel (#1330) 2017-10-07 11:48:48 +02:00
Alexander Neumann
1a2d190bdb Merge pull request #1334 from felix9/backup_tags
backup should use latest parent regardless of tags
2017-10-07 10:45:02 +02:00
Alexander Neumann
2db4ff168a Merge pull request #1327 from fawick/fix-1294
Create missing lock dir when saving lock
2017-10-07 10:40:29 +02:00
Alexander Neumann
a77c8cc5d2 Add entry to CHANGELOG 2017-10-07 10:23:56 +02:00
Alexander Neumann
b8866c1fe4 Merge pull request #1326 from fawick/rejectionCache
Cache evaluated directories in isExcludedByFile
2017-10-07 09:57:54 +02:00
Fabian Wickborn
f0f17db847 Cache evaluated directories in isExcludedByFile
Fixes #1271.
2017-10-06 22:11:22 +02:00
Alexander Neumann
a5c003acb0 Merge pull request #1333 from restic/fix-1328
Do not cache invalid/truncated files
2017-10-05 21:36:52 +02:00
Felix Lee
7b44fd0f9d fix #1143
Backup was choosing a parent snapshot that had the same tags, which
makes backup unnecessarily slow when there are newer snapshots with
different tags.

There's no reason parent has to have the same tags.

This change makes backup choose the newest snapshot instead.
2017-10-05 11:48:26 -07:00
Alexander Neumann
cebee0b8fa cache: Refuse to cache truncated files 2017-10-05 20:40:02 +02:00
Alexander Neumann
d886bc6c48 Ignore invalid index files, print warning 2017-10-05 20:39:53 +02:00
Alexander Neumann
d81adcfaa5 cache: Add file name to error message 2017-10-05 19:30:56 +02:00
Fabian Wickborn
6da9bfbbce Create missing lock dir when saving lock 2017-10-05 00:07:48 +02:00
Fabian Wickborn
69a6e622d0 Add REST backend option to use CA root certificate
Closes #1114.
2017-10-04 22:14:10 +02:00
Alexander Neumann
1dcfd64028 Merge pull request #1325 from antonlindstrom/lookup-group
Add group name in fillUser
2017-10-04 21:32:46 +02:00
Alexander Neumann
5d1c1f721e Merge pull request #1324 from antonlindstrom/password-feedback
Add password successful feedback
2017-10-04 21:31:24 +02:00
Alexander Neumann
fbc8bbf305 Merge pull request #1321 from damienstanton/master
fix broken installation link in README
2017-10-04 21:02:33 +02:00
Alexander Neumann
cdef55bb88 Merge pull request #1312 from hgfischer/issue-1119
nit: please remove dot imports #1119
2017-10-04 21:01:40 +02:00
Anton Lindstrom
26df48b2aa Add group name in fillUser
This does a lookup of the group name from the GID and adds it to the
Node.
2017-10-04 20:11:54 +02:00
Anton Lindstrom
a7baea0522 Output password successful on terminal stdout
This removes the conditions that checks if the password is supplied
through environment variable or file and outputs password is successful
on terminal and when --quiet is not supplied.
2017-10-04 14:55:04 +02:00
Anton Lindstrom
55e6003749 Add password successful feedback
This adds some feedback when entering the password on the command line.
When the password is entered and supplied through stdin (and stdout is a
terminal) then the a message saying `password is correct` if correct is
printed.
2017-10-04 13:45:05 +02:00
damienstanton
846acd5d4c use https 2017-10-03 14:28:17 -04:00
Herbert
43f8145858 Revert "Add .vscode to gitignore"
This reverts commit 1b5242b4f9.
2017-10-03 20:21:54 +02:00
damienstanton
79759928f6 fix broken installation link in README 2017-10-03 13:28:33 -04:00
Alexander Neumann
eb59d28154 Add entry to CHANGELOG 2017-10-03 18:54:07 +02:00
Alexander Neumann
79f63a2e74 Merge pull request #1320 from restic/add-snapshots-json-short-id
snapshots: Add short ID to JSON output
2017-10-03 18:53:17 +02:00
Alexander Neumann
6a62254048 Merge pull request #1299 from mungomat/fuse_addSnapshotIDs
fuse: Add a snapshot IDs directory (#1102)
2017-10-03 18:53:05 +02:00
Alexander Neumann
657a1d75af snapshots: Add short ID to JSON output
We're using the short ID in all output to users, so it should also be
included in the JSON output of the `snapshots` command.
2017-10-03 15:24:09 +02:00
Alexander Neumann
2694def56a Add entry to CHANGELOG 2017-10-03 14:52:10 +02:00
Alexander Neumann
7843341da3 Merge pull request #1319 from antonlindstrom/check-ok-output
Add explicit OK output to check command
2017-10-03 14:51:31 +02:00
Anton Lindstrom
d46314648e Add explicit OK output to check command
This adds additional output to the check command when no errors were
found. It means that when all checks have been completed, the following
output is displayed:

	No errors were found

The output is added to make sure that it is easier to understand that no
errors were found.

Full example output:

	Create exclusive lock for repository
	Load indexes
	Check all packs
	Check snapshots, trees and blobs
	No errors were found
2017-10-03 12:57:51 +02:00
Alexander Neumann
e45011af57 Merge pull request #1304 from jannickfahlbusch/showAmountOfSnapshots
Show the number of snapshots
2017-10-03 12:41:47 +02:00
Alexander Neumann
fb09884893 Merge pull request #1311 from felix9/dump-no-trees
"dump" command no longer has a "trees" option
2017-10-03 12:40:24 +02:00
Alexander Neumann
d1eecafa63 Merge pull request #1314 from jannickfahlbusch/fixTypo
Fix typo in comment
2017-10-03 12:39:24 +02:00
Alexander Neumann
b47d991f56 Merge pull request #1316 from restic/prune-wrong-data
prune: Warn about wrong plaintext blob ID
2017-10-03 12:39:04 +02:00
Alexander Neumann
553ea812a7 Add entry to CHANGELOG 2017-10-03 12:38:00 +02:00
Alexander Neumann
216e374310 Merge pull request #1315 from prattmic/cache_doc
Clarify cache location documentation
2017-10-03 12:36:53 +02:00
Alexander Neumann
034b0b8040 forget: Run prune for manually forgotten snapshots 2017-10-03 11:56:13 +02:00
Michael Pratt
5ab9e12b46 Clarify cache location documentation
PR #1287 changed the default cache location on darwin and windows.
Update the changelog and manual to reflect the new behavior.

Since the cache hasn't been included in an official release yet, I've
just changed the main cache changelog entry.

Fixes #1309
2017-10-03 11:44:09 +02:00
Alexander Neumann
abe6e0d22d Merge pull request #1300 from jniggemann/refactor-manual
doc: Refactors the documentation
2017-10-03 11:41:00 +02:00
Jan Niggemann
f5b550191c doc: Refactors the documentation
This commit refactors the documentation according to my proposal in #1273
and the discussion I had with fd0 on IRC.

The bits from the manual that I could not immediately put into the new
structure are contained in manual_rest.rst Anything else is still there,
nothing has been deleted.

I changed the heading markup to follow the convention used in Python’s
Style Guide for documentation, this convention is explained in a comment
at the top of every file.

I also added a paragraph on installing restic on Debian.
2017-10-03 11:21:53 +02:00
Jannick Fahlbusch
3dcacb3730 Fix typo in comment 2017-10-02 18:19:22 +02:00
Herbert
033589a66b Use rtest on these as well to keep codebase consistent 2017-10-02 17:48:08 +02:00
Jannick Fahlbusch
e46a647c45 Just print the number of snapshots 2017-10-02 17:05:59 +02:00
Alexander Neumann
f26492fc2d prune: Warn about wrong plaintext blob ID 2017-10-02 16:27:08 +02:00
Herbert
3473c3f7b6 Remove all dot-imports 2017-10-02 15:06:39 +02:00
Herbert
1b5242b4f9 Add .vscode to gitignore 2017-10-02 15:06:23 +02:00
Felix Lee
6d897def1b also fix man page 2017-10-01 18:33:41 -07:00
Jannick Fahlbusch
2f10e25738 Show the amount of snapshots 2017-10-01 17:14:54 +02:00
Felix Lee
555bd257bd dump command no longer has 'trees' option 2017-10-01 08:04:52 -07:00
Alexander Neumann
3afd974dea Add entry to CHANGELOG 2017-10-01 10:35:15 +02:00
Alexander Neumann
f4120c9d45 Merge pull request #1301 from restic/update-blazer
Update vendored dependencies
2017-10-01 10:33:54 +02:00
Alexander Neumann
61cb1cc6f8 Update vendored dependencies
This includes github.com/kurin/blazer 0.2.0, which resolves #1291
2017-10-01 10:13:39 +02:00
Tobias Klein
49bc1d0b3b fuse: Add a snapshot IDs directory (#1102) 2017-09-30 16:22:14 +02:00
Alexander Neumann
ba23d24dd1 Merge pull request #1298 from restic/fix-1288
sftp: Handle/Document tilde character
2017-09-30 12:28:09 +02:00
Alexander Neumann
556a63de19 sftp: Return error when path starts with a tilde (~) 2017-09-30 10:34:23 +02:00
Alexander Neumann
fae3c4d437 faq: Add entry about Synology NAS' sftp path 2017-09-30 10:30:21 +02:00
Alexander Neumann
89c2ed2a1c manual: Document that sftp does not expand tilde 2017-09-30 10:24:28 +02:00
Alexander Neumann
23f1cb06d6 Fix FAQ 2017-09-30 10:21:56 +02:00
Alexander Neumann
ac92e2dd2d Add to CHANGELOG entry 2017-09-29 21:45:37 +02:00
Alexander Neumann
bf58425351 Merge pull request #1287 from prattmic/cachedir
cache: OS-specific cache directories
2017-09-29 21:42:34 +02:00
Alexander Neumann
a3dc0ab398 Merge pull request #1295 from bachp/patch-2
Fix wrong quotes for command in installation.
2017-09-29 21:41:13 +02:00
Pascal Bach
224ebdb8b9 Fix wrong quotes for command in installation. 2017-09-29 10:08:24 +02:00
Alexander Neumann
cf80d295f3 Merge pull request #1285 from bachp/patch-1
Add instruction on how to install restic using Nix
2017-09-28 22:03:19 +02:00
Michael Pratt
2133869127 cache: OS-specific cache directories
Windows, and to a lesser extent OS X, don't conform to XDG and have
their own preferred locations for caches.

On Windows, use %LOCALAPPDATA%/restic (i.e., ~/AppData/Local/restic). I
can't find authoritative documentation from Microsoft recommending
specifically which of %APPDATA%, %LOCALAPPDATA%, and %TEMP% should be
used for caches, but %LOCALAPPDATA% is where browsers store their
caches, so it seems like a good fit.

On OS X, use ~/Library/Caches/restic, which is recommended by the Apple
documentation. They do suggest using the application "bundle identifier"
as the base folder name, but restic doesn't have one, so I just used
"restic".
2017-09-27 21:16:22 -07:00
Pascal Bach
97330ac621 Add instruction on how to install restic using Nix 2017-09-27 22:41:20 +02:00
Alexander Neumann
1ee1559506 Add release script 2017-09-26 22:14:58 +02:00
Alexander Neumann
eccc336319 Add entry to CHANGELOG 2017-09-26 14:18:37 +02:00
Alexander Neumann
7fe657ec71 Merge pull request #1282 from restic/rework-autogeneration
Rework generation of manpages and completion files
2017-09-26 14:16:41 +02:00
Alexander Neumann
77c07bfd19 Remove integration test for manpage 2017-09-26 13:16:55 +02:00
Alexander Neumann
4de938d97a Update manpages and auto-completion 2017-09-26 13:16:08 +02:00
Alexander Neumann
dad1c87afe Rework generation of manpages and completion files
This commit removes the `manpages` and `autocomplet` commands and
replaces them with the more generic `generate` command. Also, zsh
completion file support was added.
2017-09-26 13:12:12 +02:00
Alexander Neumann
801dbb6d03 Merge pull request #1281 from prattmic/gcs_perms
gs: allow backend creation without storage.buckets.get
2017-09-26 09:38:33 +02:00
Michael Pratt
fa0be82da8 gs: allow backend creation without storage.buckets.get
If the service account used with restic does not have the
storage.buckets.get permission (in the "Storage Admin" role), Create
cannot use Get to determine if the bucket is accessible.

Rather than always trying to create the bucket on Get error, gracefully
fall back to assuming the bucket is accessible. If it is, restic init
will complete successfully. If it is not, it will fail on a later call.

Here is what init looks like now in different cases.

Service account without "Storage Admin":

Bucket exists and is accessible (this is the case that didn't work
before):

$ ./restic init -r gs:this-bucket-does-exist:/
enter password for new backend:
enter password again:
created restic backend c02e2edb67 at gs:this-bucket-does-exist:/

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.

Bucket exists but is not accessible:

$ ./restic init -r gs:this-bucket-does-exist:/
enter password for new backend:
enter password again:
create key in backend at gs:this-bucket-does-exist:/ failed:
service.Objects.Insert: googleapi: Error 403:
my-service-account@myproject.iam.gserviceaccount.com does not have
storage.objects.create access to object this-bucket-exists/keys/0fa714e695c8ecd58cb467cdeb04d36f3b710f883496a90f23cae0315daf0b93., forbidden

Bucket does not exist:

$ ./restic init -r gs:this-bucket-does-not-exist:/
create backend at gs:this-bucket-does-not-exist:/ failed:
service.Buckets.Insert: googleapi: Error 403:
my-service-account@myproject.iam.gserviceaccount.com does not have storage.buckets.create access to bucket this-bucket-does-not-exist., forbidden

Service account with "Storage Admin":

Bucket exists and is accessible: Same

Bucket exists but is not accessible: Same. Previously this would fail
when Create tried to create the bucket. Now it fails when trying to
create the keys.

Bucket does not exist:

$ ./restic init -r gs:this-bucket-does-not-exist:/
enter password for new backend:
enter password again:
created restic backend c3c48b481d at gs:this-bucket-does-not-exist:/

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.
2017-09-25 22:25:51 -07:00
Alexander Neumann
7e8bc8d362 Merge pull request #1279 from restic/fix-eof-error
cache: Synchronize downloading
2017-09-25 16:39:20 +02:00
Alexander Neumann
0bb2a8e0d0 cache: Synchronize downloading
This commit adds code to synchronize downloading files to the cache.
Before, requests that came in for files currently downloading would fail
because the file was not completed in the cache. Now, the code waits
until the download is completed.

Closes #1278
2017-09-25 15:58:20 +02:00
Alexander Neumann
2e72b57f2f Correct debug message 2017-09-25 14:35:37 +02:00
Alexander Neumann
bff1039e3a Add entry to CHANGELOG 2017-09-25 13:17:44 +02:00
Alexander Neumann
5a999cb77f Merge pull request #1040 from restic/add-cache
Add local cache
2017-09-25 13:13:07 +02:00
Alexander Neumann
3a2539e0ac doc: Update manpages 2017-09-24 23:13:04 +02:00
Alexander Neumann
e262f35d0a cache: Auto-remove invalid files 2017-09-24 23:11:47 +02:00
Alexander Neumann
176bfa6529 backend: Improve ReadAt 2017-09-24 23:11:23 +02:00
Alexander Neumann
240c4cf2fd cache: In case of an error, fall back backend 2017-09-24 23:11:23 +02:00
Alexander Neumann
db5ec5d876 repo: Automatically cache tree-only pack files 2017-09-24 23:11:23 +02:00
Alexander Neumann
e1dfaf5d87 cache: Allow proactive caching of tree packs
This commit adds a function to the cache which can decide to proactively
load the complete pack file and store it in the cache. This is helpful
for pack files containing only tree blobs, as it is likely that the same
file is accessed again in the future.
2017-09-24 23:11:23 +02:00
Alexander Neumann
5436154f0d cache: Add PerformReadahead 2017-09-24 23:11:23 +02:00
Alexander Neumann
809e218d20 cache: Improve debug logs 2017-09-24 23:11:23 +02:00
Alexander Neumann
1eaad6cebb index: Add TreePacks() 2017-09-24 21:54:53 +02:00
Alexander Neumann
56fccecd06 prune: Repack mixed pack files 2017-09-24 21:54:53 +02:00
Alexander Neumann
3890a947ca Clear data files in cache 2017-09-24 21:54:53 +02:00
Alexander Neumann
e299272378 repo: Try cached pack files first 2017-09-24 21:54:53 +02:00
Alexander Neumann
70248bd05a repo: Clear indexes 2017-09-24 21:54:53 +02:00
Alexander Neumann
7a5fde8f5a repository: Save pack files for trees in cache 2017-09-24 21:54:53 +02:00
Alexander Neumann
62ba9f1950 check: Disable cache by default 2017-09-24 21:54:53 +02:00
Alexander Neumann
610b676444 Automatically exclude current restic cache 2017-09-24 21:54:53 +02:00
Alexander Neumann
58699e3c90 Write CACHEDIR.TAG to cache base directory 2017-09-24 21:54:53 +02:00
Alexander Neumann
9be24a1c9f Add cache
This commits adds rudimentary support for a cache directory, enabled by
default. The cache directory is created if it does not exist. The cache
is used if there's anything in it, newly created snapshot and index
files are written to the cache automatically.
2017-09-24 21:54:53 +02:00
Alexander Neumann
5ace41471e Merge pull request #1277 from prattmic/gcs_cleanup
Document GCS permission requirements
2017-09-24 20:49:02 +02:00
Michael Pratt
3b2106ed30 gs: document required permissions
In the manual, state which standard roles the service account must
have to work correctly, as well as the specific permissions required,
for creating even more specific custom roles.
2017-09-24 11:25:57 -07:00
Michael Pratt
5f4f997126 gs: minor comment cleanups
* Remove a reference to S3.
* Config can only be used for GCS, not other "gcs compatibile servers".
* Make comments complete sentences.
2017-09-24 10:10:56 -07:00
Alexander Neumann
49d397a419 Merge pull request #1275 from fawick/sort_snapshots
Always sort snapshots lists ascending by timestamp
2017-09-24 16:32:40 +02:00
Fabian Wickborn
ea1ab96749 Always sort snapshots lists ascending by timestamp
Fixes #1219.
2017-09-24 13:01:13 +02:00
Alexander Neumann
24c62e719a Add entry to CHANGELOG 2017-09-23 22:15:10 +02:00
Alexander Neumann
9c6b7f688e Merge pull request #1270 from restic/sftp-allow-password-prompt
sftp: Allow password entry
2017-09-23 22:13:04 +02:00
Alexander Neumann
d41dce5ecb Merge pull request #1272 from jniggemann/doc-faq-add-prio
doc: FAQ: Add info on IO and CPU prioritization
2017-09-23 20:08:17 +02:00
Jan Niggemann
52a3eafede doc: FAQ: Add info on IO and CPU prioritization 2017-09-23 19:32:07 +02:00
Alexander Neumann
55dfc85159 manual: Add hint for RESTIC_PASSWORD_FILE 2017-09-23 19:16:07 +02:00
Alexander Neumann
a7a478a19e doc: Correct FAQ 2017-09-23 19:15:21 +02:00
Alexander Neumann
2080afd9de Merge pull request #1259 from jniggemann/doc-add-restic-check-advice
adds advice to run restic check regularly
2017-09-23 14:07:55 +02:00
Alexander Neumann
9aa136b982 Merge pull request #1260 from jniggemann/doc-add-info-on-special-items
Doc add info on special items
2017-09-23 14:07:20 +02:00
Alexander Neumann
3a191f37cb Add entry to CHANGELOG 2017-09-23 14:05:55 +02:00
Alexander Neumann
429106340f Merge pull request #1267 from harshavardhana/possible-fix-memory
Implement Size() and Len() to know the optimal size.
2017-09-23 14:04:15 +02:00
Alexander Neumann
530c73b457 Merge pull request #1269 from mrzv/forget-compact
Add --compact option to forget
2017-09-23 14:02:34 +02:00
Alexander Neumann
fb9729fdb9 sftp: Allow password entry
This was a bit tricky: We start the ssh binary, but we want it to ignore
SIGINT. In contrast, restic itself should process SIGINT and clean up
properly. Before, we used `setsid()` to give the ssh process its own
process group, but that means it cannot prompt the user for a password
because the tty is gone.

So, now we're passing in two functions that ignore SIGINT just before
the ssh process is started and re-install it after start.
2017-09-23 11:43:33 +02:00
Alexander Neumann
45a09c76ff Allow suspending SIGINT handler 2017-09-23 11:12:56 +02:00
Dmitriy Morozov
efd65a1b65 Update manpage for forget 2017-09-22 16:35:58 -07:00
Dmitriy Morozov
ae60188eb9 Add --compact option to forget 2017-09-22 16:32:59 -07:00
Jan Niggemann
3b904525d9 manual: Add info on special items (device files) 2017-09-22 22:58:26 +02:00
Jan Niggemann
1e31f5202f manual: Add info on special items
Add info about handling of symlinks and bind-mounts.

Closes: #1014
2017-09-22 22:51:31 +02:00
Jan Niggemann
f12d41138a Add advice to run check regularly 2017-09-22 22:27:10 +02:00
Harshavardhana
98369f6a5d Implement Size() and Len() to know the optimal size. 2017-09-22 12:09:17 -07:00
Alexander Neumann
8f9bf1995b Merge pull request #1265 from restic/improve-packers
Improve packers, prepare for cache
2017-09-22 16:16:10 +02:00
Alexander Neumann
e7de3b5f9d Merge pull request #1266 from JaCoB1123/document_hostname_option
Add note about rescan to hostname flag (fixes #1221)
2017-09-22 15:40:24 +02:00
Alexander Neumann
3541d06d07 repo: Split packers for tree and data
The code now bundles tree blobs and data blobs into different pack
files, so we'll end up with pack files that either only contain data or
trees. This is in preparation to adding a cache (#1040), because
tree-only pack files can easily be cached later on.
2017-09-22 15:36:47 +02:00
Alexander Neumann
db0e3cd772 repo: Remove packer limits
This commit simplifies finding a packer: The first open packer is taken,
and the upper limit for the pack file is removed.
2017-09-22 15:36:47 +02:00
Alexander Neumann
d3fee08f9a Merge pull request #1263 from restic/cleanups
Small cleanups
2017-09-22 15:36:13 +02:00
Jan Bader
727fb6eabe Add note about rescan to hostname flag (fixes #1221) 2017-09-22 14:29:04 +02:00
Alexander Neumann
d610c60991 repo: Remove unused sync.Pool 2017-09-22 12:37:10 +02:00
Alexander Neumann
3f6e11d26e Allow sorting nodes in trees 2017-09-22 12:37:05 +02:00
Alexander Neumann
a4577769ae Merge pull request #1262 from mafgh/bytes
fix duplicate bytes in prune output
2017-09-22 11:59:10 +02:00
Stefan Völkel
7f927d4774 fix duplicate bytes in prune output 2017-09-22 10:07:24 +02:00
Alexander Neumann
e091673f8f Merge pull request #1258 from restic/sftp-remove-stat
local/sftp: Remove unneeded stat() call
2017-09-21 22:37:12 +02:00
Alexander Neumann
9842eff887 local/sftp: Remove unneeded stat() call 2017-09-21 21:47:03 +02:00
Alexander Neumann
c40b3d3983 Add entry to CHANGELOG 2017-09-21 20:34:17 +02:00
Alexander Neumann
ac5eefdee4 Merge pull request #1249 from mungomat/fuse_symlink
fuse: added symlink 'latest' to snapshots-dir
2017-09-21 20:33:10 +02:00
Alexander Neumann
bf508643a5 Merge pull request #1257 from restic/update-deps
Update vendored dependencies
2017-09-21 20:19:25 +02:00
Alexander Neumann
02fc16e97d Update vendored dependencies 2017-09-21 17:48:45 +02:00
Tobias Klein
1a83a739dc fuse: added symlink 'latest' to snapshots-dir 2017-09-21 16:41:20 +02:00
Alexander Neumann
81473f4538 Add VERSION file for 0.7.3 2017-09-20 20:50:07 +02:00
Alexander Neumann
e1a847e4d1 Add new version to CHANGELOG 2017-09-20 20:49:55 +02:00
Alexander Neumann
0f426c3795 Merge pull request #1254 from jniggemann/doc_metadata
documents metadata handling, fixes #647
2017-09-20 20:24:57 +02:00
Alexander Neumann
6df3d169b8 Add entry to CHANGELOG 2017-09-20 11:05:35 +02:00
Alexander Neumann
5479daa6d4 Merge pull request #1247 from restic/fix-1246
Fix backend List()
2017-09-20 11:04:05 +02:00
Jan Niggemann
397fec0152 documents metadata handling, fixes #647 2017-09-19 15:49:54 +02:00
Alexander Neumann
d7e644272f prune: Add plausibility check 2017-09-19 10:50:07 +02:00
Alexander Neumann
e91749bbb0 Merge pull request #1245 from anarcat/faq
add explanation of restic automation
2017-09-19 10:46:55 +02:00
Antoine Beaupré
bcd1e45ba7 fix typo, add note about file permissions 2017-09-18 08:55:18 -04:00
Alexander Neumann
4c6b626db6 backend: Improve TestList 2017-09-18 13:18:42 +02:00
Alexander Neumann
835ba16c27 b2: Add pagination for List() 2017-09-18 12:13:35 +02:00
Alexander Neumann
3b6a580b32 backend: Make pagination for List configurable 2017-09-18 12:01:54 +02:00
Alexander Neumann
01c486d486 Merge pull request #1250 from dvrkps/patch-1
travis: update go versions
2017-09-17 20:59:35 +02:00
Alexander Neumann
6342a08a16 Merge pull request #1248 from mungomat/fuse_typo
fuse: typo
2017-09-17 20:57:42 +02:00
Davor Kapsa
94c8ee11f8 travis: update go versions 2017-09-17 19:02:22 +02:00
Tobias Klein
9b38980ed9 fuse: typo 2017-09-17 17:39:28 +02:00
Alexander Neumann
649c536250 backend: Improve test for pagination in list 2017-09-17 11:36:45 +02:00
Alexander Neumann
dd49e2b12d Azure: Fix List(), use pagination marker 2017-09-17 11:32:05 +02:00
Alexander Neumann
f61dab1774 backend: Add test for List() 2017-09-17 11:09:16 +02:00
Alexander Neumann
40edf00182 gs: implement pagination 2017-09-17 11:08:51 +02:00
Alexander Neumann
c35518a865 Azure/GS: Remove ReadDir() 2017-09-17 11:05:30 +02:00
Antoine Beaupré
7a0b4428e3 add explanation of restic automation
every time i look at restic, i block on this and figured it may be useful for others
2017-09-16 10:17:36 -04:00
Alexander Neumann
c784a15aaa Merge pull request #1244 from restic/fix-swift-backend-tests
Ignore "not exist" errors for swift backend tests
2017-09-16 14:54:32 +02:00
Alexander Neumann
ce180de9b8 Merge pull request #1243 from restic/improve-error-reporting
Improve error reporting
2017-09-16 14:54:30 +02:00
Alexander Neumann
fca9a523e9 Merge pull request #1241 from restic/fix-timestamp-check
Use .Equal() instead of == for time.Time
2017-09-16 14:54:26 +02:00
Alexander Neumann
8a3889be11 Merge pull request #1240 from restic/config-autocomplete-dir
Correct bash completion file path
2017-09-16 14:54:20 +02:00
Alexander Neumann
2a1633621b Ignore "not exist" errors for swift backend tests 2017-09-16 13:59:55 +02:00
Alexander Neumann
e2deeceb1b Update manpage 2017-09-16 11:29:37 +02:00
Alexander Neumann
d4e994de7b Improve error reporting
This will print the error (including a stack trace) if available before
exiting.
2017-09-16 10:55:13 +02:00
Alexander Neumann
a60e751217 Use .Equal() instead of == for time.Time
Closes #1238
2017-09-15 20:57:35 +02:00
Alexander Neumann
81c5d8a968 Correct bash completion file path 2017-09-15 20:45:16 +02:00
Alexander Neumann
5b1e4df177 Add version to CHANGELOG 2017-09-13 17:14:43 +02:00
Alexander Neumann
4d80744cbb Add VERSION file for 0.7.2 2017-09-13 17:14:26 +02:00
Alexander Neumann
e243d4b7ee Merge pull request #1236 from restic/update-deps
Update dependencies
2017-09-13 14:51:07 +02:00
Alexander Neumann
dce35fcb00 Merge pull request #1232 from prattmic/patch-1
doc: remove broken link
2017-09-13 14:18:46 +02:00
Alexander Neumann
e45a21b0b6 Merge pull request #1231 from restic/fix-local-datadir-create
local: Fix creating data dirs
2017-09-13 14:15:30 +02:00
Alexander Neumann
fda563d606 Update dependencies 2017-09-13 14:09:48 +02:00
Alexander Neumann
f3b49987f8 Add entry to CHANGELOG 2017-09-13 14:04:55 +02:00
Alexander Neumann
c8c01a5cae Merge pull request #1223 from mrzv/snapshots-compact
Add --compact option to snapshots
2017-09-13 14:02:31 +02:00
Michael Pratt
f7ece90129 doc: remove broken link
The link to SFTP is broken, remove it.

I don't even bother to put a new link, since SFTP is literally the next section.
2017-09-12 21:02:34 -07:00
Alexander Neumann
0f25ef9498 Merge pull request #1230 from shayneholmes/update-short-help
Update style in short help commands
2017-09-11 22:25:31 +02:00
Alexander Neumann
5bf2228596 local: Fix creating data dirs 2017-09-11 21:48:25 +02:00
Alexander Neumann
227b01395f local: Add test for open non-existing dir 2017-09-11 21:34:26 +02:00
Shayne Holmes
9f52fe1a10 Update manpages
This is a programmatic change; just ran `restic man-page`
2017-09-11 12:05:51 -07:00
Shayne Holmes
affc6c3390 Correct manpage command 2017-09-11 11:21:44 -07:00
Alexander Neumann
951a34dcbf Add entry to CHANGELOG 2017-09-11 19:02:36 +02:00
Alexander Neumann
36eaa22ed0 Merge pull request #1205 from mungomat/backup_time
Backup time
2017-09-11 19:01:41 +02:00
Alexander Neumann
62df316356 Merge pull request #1194 from mungomat/bugfix_bucker_always
fix: bucker
2017-09-11 19:01:23 +02:00
Shayne Holmes
00797fdd85 Synchronize manual with help output
Two commands in the output weren't mentioned in the manual.
2017-09-11 09:33:19 -07:00
Shayne Holmes
9eb39cef05 Capitalize short help commands
Unify existing Cobra help command, and git-help's style.
2017-09-11 09:32:44 -07:00
Shayne Holmes
ee6150f67c Change short help messages to imperative voice
Unify the output of `restic help`.
2017-09-11 09:26:13 -07:00
Alexander Neumann
9fa909ccd6 Update golden files 2017-09-11 17:52:22 +02:00
Alexander Neumann
b1af544b1d Merge pull request #1224 from restic/improve-exclude-caches
Allow multiple exclude-if-present
2017-09-11 17:31:24 +02:00
Dmitriy Morozov
7d5b17ac72 Update man page for snapshots 2017-09-10 15:28:06 -07:00
Dmitriy Morozov
7a221f2473 Run changes through gofmt 2017-09-10 15:09:28 -07:00
Dmitriy Morozov
bdbe956c5c Add --compact option to snapshots
With --compact, snapshots doesn't list directories and puts all tags on a single
line. This way each snapshot takes up exactly one line.
2017-09-10 13:06:43 -07:00
Alexander Neumann
8e5b1e6f2f Add --group-by to manual 2017-09-10 21:00:51 +02:00
Alexander Neumann
257a454515 Add entry to CHANGELOG 2017-09-10 20:55:02 +02:00
Alexander Neumann
b6aeea425b Merge pull request #1196 from mungomat/forget_groupByTagsOnly
forget: group-by-tags-only
2017-09-10 20:52:15 +02:00
Alexander Neumann
c8e05d1f2a Add entry to CHANGELOG 2017-09-10 20:29:08 +02:00
Alexander Neumann
a8aa4eb06c Rename parameter filename -> path 2017-09-10 20:28:21 +02:00
Alexander Neumann
c1a02cc081 Merge pull request #1228 from restic/fix-1204-3
Always use long name for keys
2017-09-10 19:39:23 +02:00
Alexander Neumann
e66adc42da Always use long name for keys
Otherwise the code panics if a file with a short name is tried.
2017-09-10 15:35:10 +02:00
Alexander Neumann
89938bc21c Update manual pages 2017-09-10 15:33:20 +02:00
Alexander Neumann
0b2947dedb Add test for rejectByPattern 2017-09-10 15:31:58 +02:00
Alexander Neumann
47ddd34266 Improve test 2017-09-10 15:20:41 +02:00
Alexander Neumann
2fdca5d310 Improve debug message 2017-09-10 15:16:05 +02:00
Alexander Neumann
e5d4e33509 Improve error message if no targets specified 2017-09-10 15:14:11 +02:00
Alexander Neumann
e117f613af Move device test into new RejectFunc 2017-09-10 15:13:40 +02:00
Alexander Neumann
0dfdf02885 Rework pattern excludes 2017-09-10 14:34:28 +02:00
Alexander Neumann
4a0129fc2b Rename excludeByFile -> rejectIfPresent 2017-09-10 14:25:58 +02:00
Alexander Neumann
a9c705009c Move reject functions to new file 2017-09-10 14:25:25 +02:00
Alexander Neumann
d937ad8cf6 Rename FilenameCheck to RejectFunc
We already have the opposite: pipe.SelectFunc(item string, fi
os.FileInfo) bool, so RejectFunc is a good name.
2017-09-10 14:21:51 +02:00
Alexander Neumann
1a08a8219f Merge pull request #1227 from restic/fix-1204-2
Handle invalid key file
2017-09-10 14:12:51 +02:00
Tobias Klein
9924c311c9 added test cases 2017-09-10 12:23:28 +02:00
Alexander Neumann
e846e14965 Ignore files with invalid name in the repo 2017-09-10 11:00:07 +02:00
Alexander Neumann
36e70228f2 Handle invalid key file 2017-09-10 10:55:01 +02:00
Tobias Klein
a677f1139a removed unnacessary line 2017-09-10 10:41:07 +02:00
Alexander Neumann
6f8eba9c28 Merge pull request #1222 from damiencourousse/manual-typo-fix
manual: typo fix
2017-09-09 21:40:00 +02:00
Alexander Neumann
c22c582546 Allow multiple exclude-if-present 2017-09-09 21:24:29 +02:00
Alexander Neumann
ea75509d6e Print warning for non-existing items 2017-09-09 21:12:41 +02:00
Tobias Klein
ed30bd7b76 gofmt 2017-09-09 18:19:19 +02:00
Damien Couroussé
7090c5ceeb manual: typo fix 2017-09-09 16:58:17 +02:00
Tobias Klein
bee09c1a0f test 2017-09-09 16:33:51 +02:00
Tobias Klein
8f9ef4402b error in case of unknown grouping option 2017-09-09 15:55:37 +02:00
Tobias Klein
f26c0cb70f testcase updated 2017-09-09 15:33:12 +02:00
Tobias Klein
81d7ecba2b manual updated 2017-09-09 13:26:35 +02:00
Tobias Klein
087c3fe1dc tests updated 2017-09-09 13:26:35 +02:00
Tobias Klein
43ff971dfd new sub-option for backup: time
New option to specify the timestamp for a backup
2017-09-09 13:26:35 +02:00
Alexander Neumann
5c75a98053 Merge pull request #1220 from restic/fix-1204
Fix panic when file name is too short
2017-09-09 11:49:23 +02:00
Alexander Neumann
7ce47402fb Merge pull request #1170 from fawick/exclude_caches
Add option to exclude contents of cache directories
2017-09-09 10:56:12 +02:00
Alexander Neumann
1e48141648 Fix panic when file name is too short
Closes #1204
2017-09-09 10:50:32 +02:00
Fabian Wickborn
dbda892542 Add option to exclude directories with a tagfile
The option is named --exclude-if-present and accepts a parameter
filename[:content]. Directories are excluded and their contents is not
backed up if they contain a file with the specified name and,
optionally, that starts with the specified content. The tagfile itself
is never excluded.

There is also a shortcut --exclude-caches that works in the same way as
the likewise-named option of tar(1): Directories are recognized as cache
if they contain a file named "CACHEDIR.TAG.

Closes #317.
2017-09-09 09:57:42 +02:00
Alexander Neumann
b46774be21 Merge pull request #1214 from ricardoseriani/fix-manual.rst-key-remove
Fix manual.rst to use key remove instead of key rm
2017-09-07 21:16:46 +02:00
Tobias Klein
1073bfba37 flexible grouping option for the forget-command 2017-09-06 20:14:18 +02:00
Alexander Neumann
5dfb4d1195 Merge pull request #1209 from restic/handle-colliding-names
Resolve name collisions
2017-09-05 22:33:32 +02:00
Alexander Neumann
0a2219c5f7 Travis: Reduce workload by removing Go tip 2017-09-05 22:08:23 +02:00
Alexander Neumann
ff3149831e Merge pull request #1210 from prattmic/glob_test
filter: document recursive wildcards
2017-09-05 21:50:27 +02:00
Alexander Neumann
c935d0558c Add entry to CHANGELOG 2017-09-05 21:48:13 +02:00
Alexander Neumann
83eb075e3a Resolve name collisions
At the moment when two items to be saved have the same directory name,
restic only saves the first one to the repo. Let's say we have a
structure like this:

    dir1
    └── subdir
        └── file
    dir2
    └── subdir
        └── file

When restic is run on `dir1/subdir` and `dir2/subdir`, it will only save
the first `subdir`:

    $ restic backup dir1/subdir dir2/subdir
    [...]

    $ restic ls -l latest
    drwxr-xr-x  1000   100      0 2017-08-27 20:56:39 /subdir
    -rw-r--r--  1000   100     17 2017-08-27 20:56:39 /subdir/file

That's obviously a bad thing, caused by an early decision to strip the
full path to the files/dirs to save and only leave the last directory.

This commit partly resolves this by handling colliding names and
resolving the conflicts. Restic will now append a counter to the file
(`-123`) until the conflict is resolved. So in the example above, we'll
end up with the following structure:

    $ restic ls -l latest
    drwxr-xr-x  1000   100      0 2017-08-27 20:56:39 /subdir
    -rw-r--r--  1000   100     17 2017-08-27 20:56:39 /subdir/file
    drwxr-xr-x  1000   100      0 2017-08-27 20:56:46 /subdir-1
    -rw-r--r--  1000   100     17 2017-08-27 20:56:46 /subdir-1/file

This partly addresses #549 and closes #1179.

At first I thought that the obvious correction would be to archive the
full path. But it turns out that collisions may still occur: Suppose you
have a file named `foo` in the current directory, and the parent directory
also contains a file `foo`. Archiving these with restic also causes a
collision, since restic strips the `../` from the first file:

    $ restic backup ../foo foo

This also happens with `tar`, which does not handle the collision and
will happily archive two files called `foo`.

So, the best way forward is to handle name collisions and archive the
whole path. The latter will be tackled in a separate PR.
2017-09-05 21:47:02 +02:00
Ricardo Seriani
204c2bf09c Fix manual to use key remove instead of key rm 2017-09-05 16:22:42 -03:00
Alexander Neumann
2444522243 Add test for colliding names 2017-09-05 21:10:02 +02:00
Michael Pratt
92eb1cbffd filter: document recursive wildcards
Match/ChildMatch accept a ** pattern which is not noted in the doc
string, nor do any of the docs or tests specify whether the match is
greedy (i.e., can 'foo/**/bar' match paths with additional intermediate
bar directories?).

Add a note to the doc string and add test cases for greedy matches.
2017-09-04 14:38:48 -07:00
Alexander Neumann
8c40ae5a03 Add entry to CHANGELOG 2017-09-04 21:58:33 +02:00
Alexander Neumann
fa2ee78a5c Merge pull request #1044 from lloeki/982-improve-restore
Improve restore
2017-09-04 21:51:12 +02:00
Tobias Klein
e4a5cdc5bc forget: group-by-tags-only 2017-09-03 17:11:25 +02:00
Tobias Klein
2d73a273af saving a variable 2017-09-03 17:09:55 +02:00
Tobias Klein
761af08889 fix: bucker
bucker "always" does not return a unique id in case of exact same timestamps
2017-09-03 17:09:55 +02:00
Alexander Neumann
0ee1650f82 Merge pull request #1191 from prattmic/profile
debug: properly handle interrupted profiles
2017-09-03 09:49:46 +02:00
Alexander Neumann
0e647417f3 Add entry to CHANGELOG 2017-09-03 09:49:37 +02:00
Alexander Neumann
d1bf5a4882 Merge pull request #1203 from myfreeweb/master
Handle SIGINFO on all supported platforms
2017-09-03 09:44:24 +02:00
Alexander Neumann
b8414b240c Add entry to CHANGELOG 2017-09-03 09:44:15 +02:00
Greg V
3fbdd12b04 Handle SIGINFO on all supported platforms
Not just darwin
2017-09-02 22:06:31 +03:00
Alexander Neumann
a3f6bf3e5a Merge pull request #1202 from restic/fix-manpages
Fix manpages, update Go version
2017-09-02 10:11:43 +02:00
Alexander Neumann
3a5805db50 Update Go versions for CI 2017-09-02 09:29:02 +02:00
Alexander Neumann
de8c64e767 Use deterministic date for man pages 2017-09-02 09:27:11 +02:00
Alexander Neumann
73d6b15095 Merge pull request #1201 from gjmf/patch-1
Fixed word-o. ("Package", not "packet".)
2017-09-01 21:21:18 +02:00
Alexander Neumann
5d396b9302 Merge pull request #1200 from molivier/patch-1
Update manual.rst
2017-09-01 21:21:16 +02:00
Graham Freeman
61d2519111 Fixed word-o. ("Package", not "packet".)
Fixed a word-o. homebrew is a package manager, not a packet manager. :)
2017-08-31 13:34:53 -07:00
Matthieu OLIVIER
e61c94a846 Update manual.rst
`key remove` becomes `key rm`.
2017-08-31 18:26:04 +02:00
Alexander Neumann
7ed0f61f3f Merge pull request #1189 from FiloSottile/patch-2
doc/design: fix keys.data MAC format description
2017-08-29 20:44:31 +02:00
Alexander Neumann
85055d1c68 Merge pull request #1187 from FiloSottile/patch-1
internal/crypto: small simplifications
2017-08-29 20:43:15 +02:00
Michael Pratt
e4c469c149 debug: properly handle interrupted profiles
By default (i.e., without profile.NoShutdownHook), profile.Start listens
for SIGINT and will stop the profile and call os.Exit(0).

restic already listens for SIGINT and runs its own cleanup handlers
before calling os.Exit(0).

As is, these handlers are racing when an interrupt occurs, and in my
experience, restic tends to win the race, resulting in an unusable
profile.

Eliminate the race and properly stop profiles on interrupt by disabling
package profile's signal handler and instead stop the profile in a
restic cleanup handler.
2017-08-28 22:03:26 -07:00
Filippo Valsorda
9940e8d9f1 internal/crypto: small simplifications
* append operates on len, not cap (not a bug since len is set to cap above, but let's avoid the confusion)
* no need to extend ciphertext again to cap after we made it big enough
* make consistent use of ciphertext[:ivSize] vs iv[:]
* make all input problems errors and impossible/catastrophic cases panics
2017-08-29 00:30:06 +02:00
Filippo Valsorda
3dccca1f27 doc/design: fix keys.data MAC format description
"not including the last 32 byte" was wrong, should have been 16 bytes. But the whole description is redundant anyway.
2017-08-29 00:22:11 +02:00
Alexander Neumann
22e96a37f8 Merge pull request #1184 from prattmic/docs
Doc cleanup and mention restore include/exclude
2017-08-28 21:14:48 +02:00
Alexander Neumann
48b1ab5aaf Merge pull request #1182 from restic/fix-1167
local: do not create dirs below data/ for non-existing dir
2017-08-28 21:13:24 +02:00
Alexander Neumann
0230fa188f Add entry to CHANGELOG 2017-08-28 21:13:14 +02:00
Alexander Neumann
4118ce876e Merge pull request #1185 from prattmic/gcs_panic
gs: fix nil dereference
2017-08-28 21:11:30 +02:00
Michael Pratt
9537bc561d gs: fix nil dereference
info can be nil if err != nil, resulting in a nil dereference while
logging:

$ # GCS config
$ ./restic init
debug enabled
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x935947]

goroutine 1 [running]:
github.com/restic/restic/internal/backend/gs.(*Backend).Save(0xc420012690, 0xe84e80, 0xc420010448, 0xb57149, 0x3, 0xc4203fc140, 0x40, 0xe7be40, 0xc4201d8f90, 0xa0, ...)
	src/github.com/restic/restic/internal/backend/gs/gs.go:226 +0x6d7
github.com/restic/restic/internal/repository.AddKey(0xe84e80, 0xc420010448, 0xc4202f0360, 0xc42000a1b0, 0x4, 0x0, 0xa55b60, 0xc4203043e0, 0xa55420)
	src/github.com/restic/restic/internal/repository/key.go:235 +0x4a1
github.com/restic/restic/internal/repository.createMasterKey(0xc4202f0360, 0xc42000a1b0, 0x4, 0xa55420, 0xc420304370, 0x6a6070)
	src/github.com/restic/restic/internal/repository/key.go:62 +0x60
github.com/restic/restic/internal/repository.(*Repository).init(0xc4202f0360, 0xe84e80, 0xc420010448, 0xc42000a1b0, 0x4, 0x1, 0xc42030a440, 0x40, 0x32a4573d3d9eb5, 0x0, ...)
	src/github.com/restic/restic/internal/repository/repository.go:403 +0x5d
github.com/restic/restic/internal/repository.(*Repository).Init(0xc4202f0360, 0xe84e80, 0xc420010448, 0xc42000a1b0, 0x4, 0xe84e40, 0xc42004ad80)
	src/github.com/restic/restic/internal/repository/repository.go:397 +0x12c
main.runInit(0xc420018072, 0x16, 0x0, 0x0, 0x0, 0xe84e40, 0xc42004ad80, 0xc42000a1b0, 0x4, 0xe7dac0, ...)
	src/github.com/restic/restic/cmd/restic/cmd_init.go:47 +0x2a4
main.glob..func9(0xeb5000, 0xedad70, 0x0, 0x0, 0x0, 0x0)
	src/github.com/restic/restic/cmd/restic/cmd_init.go:20 +0x8e
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).execute(0xeb5000, 0xedad70, 0x0, 0x0, 0xeb5000, 0xedad70)
	src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:649 +0x457
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xeb3e00, 0xc420011650, 0xa55b60, 0xc420011660)
	src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:728 +0x339
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).Execute(0xeb3e00, 0x25, 0xc4201a7eb8)
	src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:687 +0x2b
main.main()
	src/github.com/restic/restic/cmd/restic/main.go:72 +0x268

(The error was likely because I had just enabled the GCS API. Subsequent
runs were fine.)
2017-08-27 21:36:04 -07:00
Michael Pratt
ae43c47ca8 doc: add mention of restore --exclude/--include
There is a lot more detail that could be added here, but it is worth
getting things off the ground with at least a mention that it is
possible to restore individual files.

Updates #396
2017-08-27 18:36:00 -07:00
Michael Pratt
2fa4060991 doc: fix inconsistencies
* Replace references to ~/shared/work/web which should be ~/web.

* Replace references to ~/tmp which should be /tmp.

* Restore /home/art to /tmp/restore-art instead of /tmp/restore-work,
  which is clearly a copy/paste from the command above.
2017-08-27 18:35:01 -07:00
Alexander Neumann
f9a934759f sftp: Improve error handling for non-existing dir 2017-08-27 20:53:04 +02:00
Alexander Neumann
3686b1ffe5 local: Create directories below data/ if it exists 2017-08-27 20:52:58 +02:00
Alexander Neumann
ea017a49c3 local: Add test for #1167
It was discovered that restic creates directories when a non-existing
directory is specified as a local repository.
2017-08-27 20:38:46 +02:00
Alexander Neumann
3559f9c776 Merge pull request #1174 from pwaring/patch-1
Update minimum Go version
2017-08-25 21:26:13 +02:00
Paul Waring
637f57ca71 Update minimum Go version
Minimum version is now 1.8 according to build.go (from latest master, cloned a few minutes ago):

```
paul@voga:~/third-dev/restic$ go run build.go 
Go version go1.7.4 detected, restic requires at least Go 1.8
exit status 1
```
2017-08-24 19:51:11 +01:00
Alexander Neumann
4e60156b45 Add entry to CHANGELOG 2017-08-18 19:50:32 +02:00
Alexander Neumann
af9946b098 Merge pull request #1164 from ricardoseriani/fix-key-remove-command
Change key rm command to key remove
2017-08-18 19:49:50 +02:00
Ricardo Seriani
b7d4b0f821 Update man pages 2017-08-17 16:44:28 -03:00
Ricardo Seriani
62ed776a8c Change key rm command to key remove
Change key rm command to key remove, to follow manual and other commands
2017-08-17 11:03:26 -03:00
Loic Nageleisen
f880ff21aa Fixing restore with excluded
An exclude filter is basically a 'wildcard but foo', so even if a
childMayMatch, other children of a dir may not, therefore childMayMatch
does not matter, but we should not go down unless the dir is selected
for restore.
2017-08-16 15:25:02 +02:00
Loic Nageleisen
4a36993c19 Smarter filter when children won't match
This improves restore performance by several orders of magniture by not
going through the whole tree recursively when we can anticipate that no
match will ever occur.
2017-08-16 15:25:02 +02:00
Alexander Neumann
d87b2f189d Merge pull request #1157 from ceh/defer-file-close-after-err-check
internal: check error before deferring file Close()
2017-08-13 19:50:05 +02:00
Alexander Neumann
f9a097a8c0 Merge pull request #1158 from ceh/fix-contributing-typo
Fix contribution typo
2017-08-13 19:49:03 +02:00
Alexander Neumann
d43358b6dd Correct URL to forum 2017-08-13 19:47:54 +02:00
Alexander Neumann
8058f196e1 Merge pull request #1156 from dimejo/patch-1
Readme: Correct link formatting
2017-08-13 19:47:31 +02:00
Emil Hessman
e13e6f34d2 Fix contribution typo 2017-08-13 19:35:53 +02:00
Emil Hessman
c2ff7150aa internal: check error before deferring file Close()
If there is an error, file will be `nil`. We should check the returned error before deferring file `Close()`.
2017-08-13 19:28:13 +02:00
dimejo
a899621930 Readme: Correct link formatting 2017-08-13 15:07:40 +02:00
Alexander Neumann
a0966e1d1d Update README 2017-08-11 21:38:18 +02:00
Alexander Neumann
e2464382ed Update issue template 2017-08-11 21:36:10 +02:00
Alexander Neumann
095bc79dc3 Correct README #2 2017-08-09 21:58:20 +02:00
Alexander Neumann
1fd3c2488e Correct README 2017-08-09 21:57:55 +02:00
Alexander Neumann
2ee8485886 Update README 2017-08-09 21:56:41 +02:00
Alexander Neumann
b67c178672 Merge pull request #1149 from restic/azure-support
Add Azure blob storage as backend
2017-08-09 21:30:35 +02:00
Alexander Neumann
7ac4f0a525 Merge pull request #1134 from restic/gcs-support
Add backend for Google Cloud Storage (GCS)
2017-08-09 21:28:36 +02:00
Alexander Neumann
c4613c51d1 Add note about Go 1.8 2017-08-09 20:33:30 +02:00
Alexander Neumann
77bf17076b Add entry to CHANGELOG 2017-08-09 20:23:29 +02:00
Alexander Neumann
8dd6beba15 gs: Add section to the manual 2017-08-09 20:22:07 +02:00
Alexander Neumann
a345386967 Add a section to the CHANGELOG 2017-08-09 20:15:08 +02:00
Alexander Neumann
bdd43bd430 Add a section to the manual 2017-08-09 20:13:34 +02:00
Alexander Neumann
1716501598 CI: Make sure the GCS backend tests run on Travis 2017-08-06 21:47:56 +02:00
Alexander Neumann
d9a5b9178e gs: Rework path initialization 2017-08-06 21:47:56 +02:00
Alexander Neumann
8ca6a9a240 Vendor dependencies for GCS 2017-08-06 21:47:56 +02:00
Dipta Das
ba75a3884c Add Google Cloud Storage as backend
Environment variables:
GOOGLE_PROJECT_ID=gcp-project-id
GOOGLE_APPLICATION_CREDENTIALS=path-to-json-file

Environment variables for test:
RESTIC_TEST_GS_PROJECT_ID=gcp-project-id
RESTIC_TEST_GS_APPLICATION_CREDENTIALS=path-to-json-file
RESTIC_TEST_GS_REPOSITORY=gs:us-central1/test-bucket

Init repository:
$ restic -r gs🪣/[prefix] init
2017-08-06 21:47:55 +02:00
Alexander Neumann
d91d89eef6 azure: Create container if it does not exist 2017-08-06 21:47:04 +02:00
Alexander Neumann
a726c91116 azure: Rework path initialization 2017-08-06 21:47:04 +02:00
Alexander Neumann
d00fe95f10 Upgrade min Go version to 1.8 2017-08-06 21:47:04 +02:00
Alexander Neumann
072b7a014e azure: User internal errors package 2017-08-06 21:47:04 +02:00
Alexander Neumann
618ce115d7 Azure: Use default HTTP transport 2017-08-06 21:47:04 +02:00
Alexander Neumann
d973aa82fe Vendor dependencies for azure backend 2017-08-06 21:47:04 +02:00
Dipta Das
3a85b6b7c6 Add Azure Blob Storage as backend
Environment variables:
AZURE_ACCOUNT_NAME=storage-account-name
AZURE_ACCOUNT_KEY=storage-account-key

Environment variables for test:
RESTIC_TEST_AZURE_ACCOUNT_NAME=storage-account-name
RESTIC_TEST_AZURE_ACCOUNT_KEY=storage-account-key
RESTIC_TEST_AZURE_REPOSITORY=azure:restic-test-container

Init repository:
$ restic -r azure:container-name:/prefix/dir init
2017-08-06 21:47:04 +02:00
Alexander Neumann
2c22ff175c Merge pull request #1150 from restic/removed-date-from-manpage
Fix manpage generation
2017-08-06 21:46:44 +02:00
Alexander Neumann
6bc43a4198 manpage: Remove auto gen tag from man page 2017-08-06 21:31:01 +02:00
Alexander Neumann
e348b3deeb manpage: Do not panic when no command is given 2017-08-06 21:01:49 +02:00
Alexander Neumann
6724b9a583 Merge pull request #1148 from restic/update-simple-scrypt
Lock simple-scrypt library to master branch
2017-08-05 20:04:33 +02:00
Alexander Neumann
41c35b2218 Lock simple-scrypt library to master branch
The master branch includes a fix for i386, otherwise the calibration
panics. See https://github.com/restic/restic/issues/676 for details.
2017-08-05 19:24:56 +02:00
Alexander Neumann
4477d76f03 Merge pull request #1147 from restic/add-manpage
Add new command 'manpage'
2017-08-05 12:39:42 +02:00
Alexander Neumann
14f5f6235a Add entry to CHANGELOG 2017-08-05 12:05:53 +02:00
Alexander Neumann
739350fd8e backup: Do not print hostname in help text
This is necessary so that the manpage generation is deterministic and we
can test if the man pages are up to date when the CI tests run.
2017-08-05 12:05:53 +02:00
Alexander Neumann
14ed97102b Add instructions for developers 2017-08-05 12:05:53 +02:00
Alexander Neumann
db389058fa Add generated manual pages 2017-08-05 12:05:53 +02:00
Alexander Neumann
b557d04007 CI: Fix test for forbidden packages 2017-08-05 12:05:53 +02:00
Alexander Neumann
52c5da997b Add CI test for manpages 2017-08-05 12:05:53 +02:00
Alexander Neumann
57d198f99a Vendor dependencies for manpage generation 2017-08-05 11:08:49 +02:00
Alexander Neumann
a3ab17b470 Add 'manpage' command to generate manual pages 2017-08-05 10:57:01 +02:00
Alexander Neumann
9bf3141893 Add entry to CHANGELOG 2017-08-01 22:02:08 +02:00
Alexander Neumann
d35eb6a0c3 Merge pull request #1144 from wjkohnen/close-exclude-files
Close exclude files and check errors
2017-08-01 22:01:43 +02:00
Johannes Kohnen
37aad2e3aa Close exclude files and check errors 2017-08-01 17:34:27 +00:00
Alexander Neumann
efc5d0699a Merge pull request #1139 from donat-b/pwfile-doesnt-exist
Error message in case PasswordFile is missing
2017-07-27 15:41:50 +02:00
donat
893bc9f777 Error message in case PasswordFile is missing 2017-07-27 14:23:08 +03:00
Alexander Neumann
61b8729ef9 Merge pull request #1138 from stapelberg/patch-1
fix typo: explicitely → explicitly
2017-07-27 09:01:47 +02:00
Michael Stapelberg
b89d3cc4d0 fix typo: explicitely → explicitly 2017-07-27 08:24:53 +02:00
Alexander Neumann
e8cc11ea34 Add entry to CHANGELOG 2017-07-26 22:29:39 +02:00
Alexander Neumann
2e804511ca Merge pull request #1133 from middelink/fix-1132
Force restic to ask the password when adding a key.
2017-07-26 22:11:37 +02:00
Alexander Neumann
b6790c491b Merge pull request #1131 from restic/test-cleanups
Rework withTestEnvironment
2017-07-26 22:06:06 +02:00
Pauline Middelink
c95e2b009e Simpify cmd_backup and cmd_init now we have the password in gopts. 2017-07-24 23:32:55 +02:00
Pauline Middelink
d5615a67c8 Refactor password resolving.
Instead of determining the password lazily during ReadPassword(), do so now in
cobra.PersistentPreRunE() so we can store the result in the globalOptions and
reuse/override when applicable without having to worry about the environment
or flag options interfering.
2017-07-24 23:05:37 +02:00
Pauline Middelink
d9b9bbd4a8 Force restic to ask the password when adding a key.
As `restic key add` uses the same `ReadPasswordTwice()` as the
rest of restic, it is sensitive to the environment variable
RESTIC_PASSWORD or --password-file= override.

When asking for the new key, temporary remove these 2 overrides, forcing
the password to be asked.
2017-07-24 22:00:44 +02:00
Alexander Neumann
d780b1eede Rework withTestEnvironment
Switch from a function passed as a parameter to a cleanup function,
which is also executed when the test function panics, so no temporary
directories are left behind.
2017-07-24 21:32:34 +02:00
Alexander Neumann
608adf15a3 Merge pull request #1130 from middelink/fix-fuse-test
Fuse testing leaves test mountpoint around.
2017-07-24 21:26:05 +02:00
Pauline Middelink
1717391f6c Fuse testing leaves test mountpoint around. Move it under the testing tree which is removed after each test. 2017-07-24 20:33:39 +02:00
Alexander Neumann
2e6e9ff6f8 Merge pull request #1129 from restic/move-restic-pkg
Move restic package to internal/restic
2017-07-24 19:19:28 +02:00
Alexander Neumann
23c903074c Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
Alexander Neumann
94030a12cf Add entry to CHANGELOG 2017-07-23 16:54:56 +02:00
Alexander Neumann
f63d7de9da Merge pull request #1126 from restic/switch-to-default-go-git-layout
Use idiomatic default Go git repo layout
2017-07-23 16:54:53 +02:00
Alexander Neumann
13ee6792df Add remark about GOPATH on forked repos 2017-07-23 16:36:13 +02:00
Alexander Neumann
6302444f34 Remove linebreak from Errorf() format string 2017-07-23 15:51:44 +02:00
Alexander Neumann
61c5e4b54a Fix glyphcheck 2017-07-23 15:51:39 +02:00
Alexander Neumann
d6118871be Update other files 2017-07-23 14:40:05 +02:00
Alexander Neumann
94b27e8933 Fix paths for tests 2017-07-23 14:39:57 +02:00
Alexander Neumann
05500dc5f8 Update documentation 2017-07-23 14:32:31 +02:00
Alexander Neumann
c5a72971fe Remove Vagrantfile 2017-07-23 14:30:33 +02:00
Alexander Neumann
5bc6486e3b Update docs 2017-07-23 14:25:39 +02:00
Alexander Neumann
59e18bce0a Fix build.go 2017-07-23 14:25:39 +02:00
Alexander Neumann
898c5b6df5 Fix integration tests 2017-07-23 14:25:39 +02:00
Alexander Neumann
9cd422791a Update build.go 2017-07-23 14:25:38 +02:00
Alexander Neumann
91edebf1fe Vendor dependencies with dep 2017-07-23 14:25:38 +02:00
Alexander Neumann
df8a5792f1 Remove Dockerfile 2017-07-23 14:25:38 +02:00
Alexander Neumann
cda7b417cd Remove envrc 2017-07-23 14:25:38 +02:00
Alexander Neumann
d2ac35af26 Remove vendor 2017-07-23 14:25:37 +02:00
Alexander Neumann
6caeff2408 Run goimports 2017-07-23 14:21:03 +02:00
Alexander Neumann
83d1a46526 Moves files 2017-07-23 14:19:13 +02:00
Alexander Neumann
d1bd160b0a Merge pull request #1061 from bclermont/docker-image
add docker image
2017-07-22 11:52:22 +02:00
Alexander Neumann
bc88a8bb03 Add entry to CHANGELOG 2017-07-22 11:52:16 +02:00
Alexander Neumann
04cfb984ae Add VERSION file for 0.7.1 2017-07-22 11:04:32 +02:00
Alexander Neumann
02a245941a Adapt CHANGELOG for 0.7.1 2017-07-22 11:03:44 +02:00
Alexander Neumann
7fb1352aa1 Merge pull request #1124 from restic/use-minio-300
Set minio-go to v3.0.0
2017-07-22 11:01:57 +02:00
Alexander Neumann
4c555bad2e Set minio-go to v3.0.0 2017-07-22 10:19:52 +02:00
Alexander Neumann
75c789bab4 Merge pull request #1122 from restic/swift-remove-range-test
swift: Remove check for byte range
2017-07-21 23:05:05 +02:00
Alexander Neumann
626d020e62 swift: Remove check for byte range
Closes #1084
Closes #1094
2017-07-21 20:45:25 +02:00
Alexander Neumann
3830117735 Merge pull request #1121 from restic/fix-swift-test
swift: Increase backend test delay for removed file
2017-07-21 20:37:10 +02:00
Alexander Neumann
042cee8e36 Merge pull request #1117 from donat-b/password-file-env
Set default value for password-file flag from env
2017-07-21 19:47:49 +02:00
Alexander Neumann
03cc5b47e9 appveyor: Update Go version to 1.8.3 2017-07-21 19:42:34 +02:00
Alexander Neumann
46fa45942e swift: Increase backend test delay for removed file 2017-07-21 19:42:34 +02:00
Alexander Neumann
0cb4104aa7 Fix Go report card URLs (thanks @tyll) 2017-07-20 22:31:41 +02:00
donat
f2bbc5fbc4 Set default value for password-file flag from env
Allows defining password file path as RESTIC_PASSWORD_FILE=/foo
2017-07-20 10:47:02 +03:00
Alexander Neumann
16340ce811 Merge pull request #1090 from middelink/fix-1081
Update HasTags() and HasPaths() to follow #1081 feature request
2017-07-19 17:11:18 +02:00
Alexander Neumann
2cf8153f4a Add entry to CHANGELOG 2017-07-19 17:09:02 +02:00
Alexander Neumann
2f00287e45 Merge pull request #1112 from restic/fix-chmod-not-supported
Ignore error for Chmod() on FS that don't support it
2017-07-19 17:05:18 +02:00
Alexander Neumann
0de17f64e9 Add entry to CHANGELOG 2017-07-19 17:02:53 +02:00
Alexander Neumann
c30838878f Merge pull request #1115 from restic/fix-prune-index
prune: Fix newly created index
2017-07-19 17:01:11 +02:00
Alexander Neumann
bd31281f1e prune: Fix newly created index
This fixes a bug introduced in c79fb6fcdd
where the index file after a prune contains pack files that do not exist
any more. Restic will detect this during backup and abort with an error
message until the user runs `prune` or `rebuild-index` again.
2017-07-18 23:10:30 +02:00
Alexander Neumann
7fc54ed98e Improve test case for prune 2017-07-18 23:07:29 +02:00
Alexander Neumann
0abdcedcab Ignore error for Chmod() on FS that don't support it
See #1079
2017-07-18 21:47:30 +02:00
Alexander Neumann
6c05353086 Add entry to CHANGELOG 2017-07-17 22:02:06 +02:00
Alexander Neumann
e7575bf380 Merge pull request #1108 from restic/update-deps
Update vendored deps
2017-07-17 22:01:52 +02:00
Alexander Neumann
89ace85903 s3: Use streaming API and remove workarounds 2017-07-17 20:43:53 +02:00
Alexander Neumann
68a91d66b7 s3: Use new API for CopyObject 2017-07-17 20:43:45 +02:00
Alexander Neumann
724b5bf4fe Update minio-go 2017-07-17 20:19:04 +02:00
Alexander Neumann
d6da9211bc Update vendored deps (except minio-go) 2017-07-17 20:00:44 +02:00
Alexander Neumann
f45abac27f Merge pull request #1107 from bclermont/fix-s3-panic
Fix S3 panic on Invalid configuration
2017-07-17 18:50:46 +02:00
Bruno Clermont
00b9a1d87d evaluate open error 2017-07-17 11:33:19 +03:00
Alexander Neumann
20b835b5a4 Improve help text 2017-07-16 21:41:13 +02:00
Alexander Neumann
7bb1a474df Add entry to CHANGELOG 2017-07-16 21:41:13 +02:00
Alexander Neumann
750ee35dbf Add more examples to the manual 2017-07-16 21:40:53 +02:00
Alexander Neumann
fda5e1f543 Adress code review comments 2017-07-16 21:40:53 +02:00
Alexander Neumann
78d090aea5 Implement TagList and TagLists as pflag.Value 2017-07-16 21:40:53 +02:00
Alexander Neumann
7362569cf5 Use TagLists for all commands 2017-07-16 21:40:53 +02:00
Alexander Neumann
f5b1c7e5f1 Add TagList 2017-07-16 21:40:53 +02:00
Pauline Middelink
c554cdac4c Update HasTags() and HasPaths() to follow #1081 idea
Replace all but 3 occurences of StringSliceVar to StringArrayVar. This will
prevent the flag parser to interpretate the given values as CSV string.

Both --tag, --keep-tag and --path can be given multiple times, the command will
match snapshots matching ANY of the tags/paths. Only when a value is given which
contains a comma separated list of tags/paths, ALL elements need to match.
2017-07-16 21:40:53 +02:00
Alexander Neumann
41b624ea1b Merge pull request #1105 from restic/improve-sftp-open
sftp: Improve check for data subdirs
2017-07-16 15:41:13 +02:00
Alexander Neumann
7cdcaadcf5 Add entry to CHANGELOG 2017-07-16 15:11:26 +02:00
Alexander Neumann
4ad33d3c3b sftp: Improve check for data subdirs 2017-07-16 15:10:06 +02:00
Alexander Neumann
2778ac21de Merge pull request #1103 from tobya/docsupdate
Update links to design.md to design.rst
2017-07-16 10:39:53 +02:00
Toby Allen
cb3cd57926 Update links to design.md to design.rst 2017-07-15 19:35:45 +01:00
Alexander Neumann
ba6815d413 Merge pull request #1100 from fawick/master
Allow absolute target path in build.go
2017-07-15 10:11:45 +02:00
Fabian Wickborn
52004cdde8 Allow absolute target path in build.go
Fixes #1099.
2017-07-14 11:54:46 +02:00
Alexander Neumann
1d2045cb61 Test error for os.PathError
See https://github.com/restic/restic/issues/1079#issuecomment-315177469
for details.
2017-07-13 21:29:29 +02:00
Alexander Neumann
357e2e404a Merge pull request #1080 from restic/fix-1079
local: Ignore ENOTSUP error for chmod
2017-07-13 20:12:47 +02:00
Alexander Neumann
38e5640cda Add CHANGELOG entry 2017-07-09 21:43:05 +02:00
Alexander Neumann
c4c731bd9a Merge pull request #1082 from Habbie/siginfo
support SIGINFO on Darwin
2017-07-09 21:41:29 +02:00
Alexander Neumann
04d27acd60 Add entry to CHANGELOG 2017-07-05 20:54:37 +02:00
Alexander Neumann
80f0303b21 Merge pull request #1086 from kamsz/iam
Add support for IAM instance profile
2017-07-05 20:53:31 +02:00
Kamil Szczygieł
d651d9b427 more verbose debug 2017-07-05 19:21:57 +02:00
Kamil Szczygieł
3b2648bd5e iam instance profile 2017-07-05 16:19:25 +02:00
Peter van Dijk
73cc11f000 support SIGINFO on Darwin 2017-07-03 20:39:42 +02:00
Alexander Neumann
637de0149c Add entry to CHANGELOG 2017-07-03 19:49:18 +02:00
Alexander Neumann
855575e5a7 Merge pull request #1077 from restic/create-subdirs
local/sftp: Auto-create subdirs of data/ on init/open
2017-07-03 19:47:58 +02:00
Alexander Neumann
ed2999a163 Merge pull request #1075 from restic/migrate-s3-continue
s3: Improve migration to new layout
2017-07-03 19:47:22 +02:00
Alexander Neumann
a18c16e19e local: Ignore ENOTSUP error for chmod
Closes: #1079
2017-07-03 19:45:56 +02:00
Alexander Neumann
9032ab2eec local/sftp: Create dirs on open() 2017-07-02 19:35:45 +02:00
Alexander Neumann
03f66b8d74 Create subdirs below data/ 2017-07-02 19:35:45 +02:00
Alexander Neumann
8c30ae7c65 Add entry to CHANGELOG 2017-07-02 11:21:05 +02:00
Alexander Neumann
453c9c9199 s3 migrate layout: Retry on errors 2017-07-02 11:15:20 +02:00
Alexander Neumann
993e370f92 s3 migrate layout: Ignore already renamed files 2017-07-02 10:47:50 +02:00
Alexander Neumann
2bcd3a3acc s3 migrate layout: Rename key files last 2017-07-02 10:47:20 +02:00
Alexander Neumann
c54c632ca1 s3 migrate layout: Force old layout for rename 2017-07-02 10:47:03 +02:00
Alexander Neumann
28a4a35625 Allow migrate to run althoug check failed 2017-07-02 10:29:41 +02:00
Alexander Neumann
e7577d7bb4 Add stub to CHANGELOG 2017-07-01 15:11:36 +02:00
Alexander Neumann
27ea0623d7 Add VERSION file for 0.7.0 2017-07-01 14:12:07 +02:00
Alexander Neumann
390e2bbddc Merge pull request #1070 from restic/warn-unsupported-repo-type
Return an error for invalid backend schemes
2017-06-30 22:15:17 +02:00
Alexander Neumann
b50fc08f39 Add entry to CHANGELOG 2017-06-30 22:15:00 +02:00
Alexander Neumann
b2ce7e8d84 Return an error for invalid backend schemes
Closes #1021
2017-06-30 21:28:39 +02:00
Alexander Neumann
2b1c6d3cf8 Merge pull request #1066 from restic/update-minio-go
Update minio-go
2017-06-30 20:40:43 +02:00
Alexander Neumann
c658305a1b Correct path for rest-server 2017-06-27 21:19:48 +02:00
Alexander Neumann
63235d8f94 Update minio-go 2017-06-26 22:06:57 +02:00
Bruno Clermont
d702227af0 install fuse and ca-certificates 2017-06-23 10:38:19 +02:00
Bruno Clermont
b7251dbea5 add docker image 2017-06-23 10:28:18 +02:00
Alexander Neumann
144b7f3386 doc: Correct path in manual 2017-06-22 19:54:55 +02:00
Alexander Neumann
9583dc820f Merge pull request #1051 from restic/refactor-crypto
crypto: Make Encrypt/Decrypt a method of *Key
2017-06-21 19:26:11 +02:00
Alexander Neumann
a03076f2d8 Merge pull request #1056 from restic/fix-1053
prune: Delete invalid/incomplete pack files
2017-06-21 19:25:55 +02:00
Alexander Neumann
d76fa22b4b prune: Delete invalid/incomplete pack files
Closes #1053
2017-06-20 22:53:49 +02:00
Alexander Neumann
f960831f10 crypto: Make Encrypt/Decrypt a method of *Key 2017-06-20 22:14:51 +02:00
Alexander Neumann
b0fb95dfc9 backend tests: Use delayedRemove() 2017-06-19 20:02:49 +02:00
Alexander Neumann
bca9566849 Merge pull request #1050 from restic/extend-fuse-mount
fuse: Add more directories
2017-06-19 19:52:45 +02:00
Alexander Neumann
8760de42fe Merge pull request #1046 from restic/s3-split-open
s3: Split Create() from Open()
2017-06-19 19:52:40 +02:00
Alexander Neumann
2c02efd1fe fuse: Reduce code duplication, add MetaDir 2017-06-18 21:32:07 +02:00
Alexander Neumann
4b4a63ed44 fuse: Add tags dir 2017-06-18 21:32:07 +02:00
Alexander Neumann
64f434eca4 fuse: Add hosts dir 2017-06-18 21:32:07 +02:00
Alexander Neumann
f4e85a53e7 fuse: Add '.' and '..' entries to all directories 2017-06-18 21:32:07 +02:00
Alexander Neumann
f8176a74ec fuse: Rename DirSnapshots -> SnapshotsDir 2017-06-18 21:32:07 +02:00
Alexander Neumann
e60a96a71a swift: Increase delete timeout to 20s 2017-06-18 21:31:48 +02:00
Alexander Neumann
216e2607ca Add entry to CHANGELOG 2017-06-18 21:18:11 +02:00
Alexander Neumann
53f8026018 Merge pull request #1048 from restic/cleanup-fuse-mount
Cleanup/fix fuse mount
2017-06-18 18:41:02 +02:00
Alexander Neumann
de92ce7a88 Merge pull request #1049 from restic/fix-backend-tests-delayed-remove
backend tests: Add configurable delay for delayed remove
2017-06-18 18:31:38 +02:00
Alexander Neumann
eb8041b943 backend tests: Add configurable delay for delayed remove 2017-06-18 17:36:57 +02:00
Alexander Neumann
9c6e9bcf33 fuse: Add build tags for unsupported OS 2017-06-18 17:02:07 +02:00
Alexander Neumann
154816ffd0 fuse: Fix file test 2017-06-18 16:29:00 +02:00
Alexander Neumann
c86e425df6 fuse: Fix file inode 2017-06-18 16:28:55 +02:00
Alexander Neumann
3883c7a190 fuse: Fix blob length cache 2017-06-18 16:28:39 +02:00
Alexander Neumann
a66760d86d fuse: Fix inode handling 2017-06-18 15:11:32 +02:00
Alexander Neumann
52752659c1 fuse: Rewrite fuse implementation 2017-06-18 14:59:44 +02:00
Alexander Neumann
f676c0c41b index: Add Each() to MasterIndex 2017-06-18 14:52:14 +02:00
Alexander Neumann
f31e993f09 fuse: Reenable integration tests 2017-06-18 14:23:35 +02:00
Alexander Neumann
56f610e548 fuse: Remove struct SnapshotWithId 2017-06-18 14:11:33 +02:00
Alexander Neumann
052a6a0acc Move snapshot filter function to restic package 2017-06-18 13:18:12 +02:00
Alexander Neumann
77037e33c9 Move snapshot finding functions to new file 2017-06-18 13:06:52 +02:00
Alexander Neumann
5a34799554 Move Snapshots struct and policy to other files 2017-06-18 13:05:47 +02:00
Alexander Neumann
47282abfa4 fuse: Use Mutex instead of RWMutex 2017-06-17 23:00:38 +02:00
Alexander Neumann
c9cc724b31 s3: Split Create() from Open() 2017-06-17 22:15:58 +02:00
Alexander Neumann
0d3674245b Merge pull request #1043 from restic/fix-gcs
s3: Fix GCS
2017-06-17 10:35:10 +02:00
Alexander Neumann
82b21cdf4a Merge pull request #1027 from restic/s3-set-retry
s3: Allow setting the number of retries for minio-go
2017-06-17 10:34:36 +02:00
Alexander Neumann
c4592f577a Merge pull request #1036 from restic/prune-remove-invalid-files
prune: Remove invalid files
2017-06-16 22:52:44 +02:00
Alexander Neumann
3cd851e578 Update github.com/minio/minio-go 2017-06-16 22:29:40 +02:00
Alexander Neumann
e074833a7d Merge pull request #1045 from restic/prune-fix-progress
prune: Fix progress information
2017-06-16 20:21:55 +02:00
Alexander Neumann
c5f1a83cb4 prune: Fix progress information 2017-06-16 19:03:26 +02:00
Alexander Neumann
1baaa778ee Add entry to CHANGELOG 2017-06-16 12:27:44 +02:00
Alexander Neumann
6a948d5afd s3: Fix backend for Google Cloud Storage 2017-06-16 11:25:06 +02:00
Alexander Neumann
ea66ae0811 s3: Fix IsNotExist() 2017-06-16 10:54:46 +02:00
Alexander Neumann
bf8a155fb1 Update github.com/minio/minio-go 2017-06-16 10:53:38 +02:00
Alexander Neumann
4ae59bef96 prune: Remove invalid files
Closes #1029
2017-06-15 20:56:22 +02:00
Alexander Neumann
eadf5dcb2d Merge pull request #1038 from restic/s3-prevent-close
Improve GCS support
2017-06-15 20:54:52 +02:00
Alexander Neumann
91a24e8229 Merge pull request #1035 from restic/fix-1032
prune: Remove files as the last step
2017-06-15 20:22:42 +02:00
Alexander Neumann
e3c979a7a4 Merge pull request #1034 from restic/fix-1030
prune: Fix status string for narrow terminals
2017-06-15 20:22:33 +02:00
Alexander Neumann
05365706c0 backend/tests: Correct error message and delayed remove 2017-06-15 20:05:35 +02:00
Alexander Neumann
bbca31b661 test/s3: Retry connection to Minio server 2017-06-15 19:51:55 +02:00
Alexander Neumann
eb7fc12e01 backend tests: Delay listing for swift backend 2017-06-15 19:41:07 +02:00
Alexander Neumann
98ae7b1210 s3: Save config in backend 2017-06-15 16:41:09 +02:00
Alexander Neumann
51877cecf7 s3: Prevent closing of the reader for GCS 2017-06-15 16:39:42 +02:00
Alexander Neumann
9053b2000b s3: Delete ignores error if the object doesn't exist 2017-06-15 16:27:19 +02:00
Alexander Neumann
dd6ce5f9d8 Remove backend.Closer, use ioutil.NopCloser() instead 2017-06-15 15:58:23 +02:00
Alexander Neumann
9a8301fc74 prune: Fix status string for narrow terminals
Closes #1030
2017-06-15 15:41:07 +02:00
Alexander Neumann
aabe2a0a30 Merge pull request #1002 from restic/test-codecov
Remove codecov config file
2017-06-15 15:09:50 +02:00
Alexander Neumann
c79fb6fcdd prune: Delete repacked files as the very last step 2017-06-15 14:46:50 +02:00
Alexander Neumann
af9ba3be91 backend: Add IsNotExist 2017-06-15 13:40:27 +02:00
Alexander Neumann
6f24d038f8 prune: Only remove data after index has been uploaded
Closes #1032
2017-06-15 13:12:46 +02:00
Alexander Neumann
cf65893c4b s3: Allow setting the number of retries for minio-go
https://github.com/restic/restic/issues/1013#issuecomment-307883970
2017-06-12 21:09:37 +02:00
Alexander Neumann
bd7d5a429f Merge pull request #1025 from restic/fix-1013
s3: Switch back to high-level API for upload
2017-06-12 19:58:12 +02:00
Alexander Neumann
7b54f6e642 Add entry to CHANGELOG 2017-06-12 19:56:50 +02:00
Alexander Neumann
422c0dfb5e s3: Exit test loop for minio server on success 2017-06-11 20:49:56 +02:00
Alexander Neumann
73b296918b s3: Reorder debug messages
This way the semaphore token acquisition can be observed in the debug
log.
2017-06-11 20:49:53 +02:00
Alexander Neumann
907c201693 debug: Add version number to debug log 2017-06-11 20:48:46 +02:00
Alexander Neumann
58de8bf392 swift/rest: Reduce number of connections 2017-06-11 20:48:46 +02:00
Alexander Neumann
a89a7a783a s3: Correct comment on the connections option 2017-06-11 20:48:46 +02:00
Alexander Neumann
c422010597 s3: Fix test 2017-06-11 20:48:46 +02:00
Alexander Neumann
08e1d9ffad s3: Switch back to high-level API, limit connections 2017-06-11 20:48:42 +02:00
Alexander Neumann
a4e8dc3371 s3: Improve error message in debug log 2017-06-11 11:22:25 +02:00
Alexander Neumann
19da56a6ea debug: Add log before panic() 2017-06-11 11:22:25 +02:00
Alexander Neumann
d3c06c39f9 debug: Fix EOF detection in HTTP transport 2017-06-11 11:22:25 +02:00
Alexander Neumann
6301620428 s3: Add more debug logs 2017-06-11 11:22:25 +02:00
Alexander Neumann
a6f157f346 Merge pull request #1024 from restic/remove-unused
Remove unused code/variables
2017-06-11 11:18:02 +02:00
Alexander Neumann
8d4417ec92 Remove unused code/variables 2017-06-11 09:29:53 +02:00
Alexander Neumann
0b55be2581 prune: Fix debug log 2017-06-10 22:16:42 +02:00
Alexander Neumann
88a59fd0ca options: Handle uint 2017-06-10 21:07:10 +02:00
Alexander Neumann
539674614b Merge pull request #1019 from restic/fix-1017
ls: Print names with percent correctly
2017-06-10 12:43:46 +02:00
Alexander Neumann
9d1b9157d4 ls: Print names with percent correctly
Closes #1017
2017-06-10 12:17:21 +02:00
Alexander Neumann
5f449045d2 Merge pull request #1003 from fwilhe/contributing-md-link
Fix relative link to CONTRIBUTING.md
2017-06-09 20:56:21 +02:00
Alexander Neumann
3e4d236751 Merge pull request #1010 from restic/update-minio-go
Update github.com/minio/minio-go
2017-06-09 20:55:49 +02:00
Alexander Neumann
4fe6593fbe Merge pull request #1011 from restic/fix-1009
pack: Handle small files
2017-06-09 20:53:52 +02:00
Florian Wilhelm
635633379a Fix link to CONTRIBUTING.md 2017-06-09 00:36:31 +02:00
Alexander Neumann
48fecd791d pack: Handle more invalid header cases 2017-06-08 21:04:07 +02:00
Alexander Neumann
a325a20fb4 s3: Increase wait time for minio server 2017-06-08 20:50:56 +02:00
Alexander Neumann
1f0916b01b Merge pull request #1004 from restic/add-migrate-s3
Add 'migrate' command, change s3 layout
2017-06-08 20:48:27 +02:00
Alexander Neumann
eb767ab15f pack: Handle small files 2017-06-08 20:40:12 +02:00
Alexander Neumann
92c0aa3854 Merge pull request #998 from restic/fix-820
fuse: Add cache for blob sizes
2017-06-08 20:21:26 +02:00
Alexander Neumann
a61016cb55 Update github.com/minio/minio-go 2017-06-08 19:40:06 +02:00
Alexander Neumann
eb7ddd6e11 Add entry to CHANGELOG 2017-06-08 19:21:52 +02:00
Alexander Neumann
ff3d2e42f4 migrate: Be a bit more verbose 2017-06-08 19:19:45 +02:00
Alexander Neumann
1aab123b6c Merge pull request #1008 from chaquotay/patch-1
Fixing tiny typo
2017-06-08 19:04:14 +02:00
Stephan Müller
d11f8d294f Fixing tiny typo 2017-06-08 13:27:22 +02:00
Alexander Neumann
04ded881f6 s3: Change the default layout to "default" 2017-06-07 23:08:20 +02:00
Alexander Neumann
4f9bf5312b Add migrate
This commits adds a 'migrate' command and a migration to move s3
repositories from the 's3legacy' to the 'default' layout.
2017-06-07 23:08:02 +02:00
Alexander Neumann
7cf8f59987 layout: Add String() and Name() 2017-06-07 21:59:41 +02:00
Alexander Neumann
b8b5c8e8c9 s3: Rename struct to Backend 2017-06-07 21:59:01 +02:00
Alexander Neumann
a46baf7685 s3: Remove cache 2017-06-07 20:51:45 +02:00
Alexander Neumann
f2a51aa37c Add entry to CHANGELOG 2017-06-07 20:51:08 +02:00
Alexander Neumann
233eaf8ee9 fuse: Improve semantics of the blob size cache
Wrap it in a struct and add a Lookup() function to make clear that it
is only queried, not changed, so we don't have any race conditions.
2017-06-07 20:04:58 +02:00
Alexander Neumann
067be2c551 fuse: Add cache for blob sizes
Closes: #820
2017-06-07 20:04:15 +02:00
Alexander Neumann
550e1feaec Merge pull request #999 from restic/backend-use-semaphore
backends: Use new semaphore
2017-06-07 19:48:32 +02:00
Alexander Neumann
f90ce23f30 Merge pull request #994 from restic/add-context
Add context.Context to the backend
2017-06-07 19:11:56 +02:00
Alexander Neumann
29f8f8fe68 Update github.com/kurin/blazer
Reduces cost-intensive list_files API calls.
2017-06-07 19:10:05 +02:00
Alexander Neumann
48c1e7b00d Fix location tests 2017-06-06 21:12:38 +02:00
Alexander Neumann
2175ccedd2 Remove codecov config file 2017-06-06 21:02:19 +02:00
Alexander Neumann
d4e74f20aa Add context to dump command 2017-06-06 00:37:25 +02:00
Alexander Neumann
aa5bc39311 swift: Use semaphore 2017-06-06 00:33:25 +02:00
Alexander Neumann
46049b4236 rest: Use semaphore 2017-06-06 00:26:29 +02:00
Alexander Neumann
683ebef6c6 s3: Use semaphore 2017-06-06 00:17:39 +02:00
Alexander Neumann
5010e95c23 Add error handling to semaphore 2017-06-06 00:17:21 +02:00
Alexander Neumann
46b7a270a6 Add context parameters to tests 2017-06-05 23:56:59 +02:00
Alexander Neumann
cf497c2728 Add context to restic packages 2017-06-04 14:35:14 +02:00
Alexander Neumann
16fcd07110 Add a Context to the backend 2017-06-04 14:02:44 +02:00
Alexander Neumann
a9a2798910 Merge pull request #993 from restic/improve-search-performance
Improve find
2017-06-04 12:44:29 +02:00
Alexander Neumann
9cd664caa3 Add entry to CHANGELOG 2017-06-04 11:50:38 +02:00
Alexander Neumann
a90e0c6595 find: Check trees only once 2017-06-04 11:42:40 +02:00
Alexander Neumann
7b5efaf7b0 find: Move functions to struct 2017-06-04 11:38:46 +02:00
Alexander Neumann
3b7ca4ac35 find: Improve debug log 2017-06-04 11:22:56 +02:00
Alexander Neumann
40a61b82ce Merge pull request #978 from restic/add-backblaze-backend
Add Backblaze B2 backend
2017-06-03 14:54:04 +02:00
Alexander Neumann
028f43299a Merge pull request #975 from restic/add-swift-backend
Add swift backend
2017-06-03 14:52:47 +02:00
Alexander Neumann
3a4727f0f5 Add entry to CHANGELOG.md 2017-06-03 14:28:29 +02:00
Alexander Neumann
fec89f95fb Improve swift backend 2017-06-03 14:28:18 +02:00
Bartłomiej Święcki
5681d41f76 Implement OpenStack swift backend
This commit implements support for OpenStack swift
storage server, tested on OVH public cloud storage.

Special thanks to jayme-github <tuxnet@gmail.com>
who helped with the implementation.
2017-06-03 14:26:29 +02:00
Alexander Neumann
efd61d97ef Vendor github.com/ncw/swift 2017-06-03 14:25:57 +02:00
Alexander Neumann
3ed56f2192 Add entry to CHANGELOG.md 2017-06-03 14:24:59 +02:00
Alexander Neumann
122462b9b1 Add Backblaze B2 backend
This is based on prior work by Joe Turgeon <arithmetric@gmail.com>
@arithmetric.
2017-06-03 14:24:59 +02:00
Alexander Neumann
2217b9277e Vendor github.com/kurin/blazer 2017-06-03 14:24:59 +02:00
Alexander Neumann
b5e0e3631b Addd nev version section 2017-06-03 14:10:28 +02:00
Alexander Neumann
be68e43871 Fix link 2017-06-02 22:08:04 +02:00
Alexander Neumann
f6034c0882 Merge pull request #990 from tmcarr/fix_readme_links
Fix the links in the readme to render in RST
2017-06-02 21:59:41 +02:00
Travis Carr
f693781bf0 Fix the links in the readme to render right. 2017-06-02 12:32:13 -07:00
2066 changed files with 399161 additions and 110761 deletions

1
.envrc
View File

@@ -1 +0,0 @@
GOPATH=$PWD:$PWD/vendor

View File

@@ -1,25 +0,0 @@
<!--
NOTE: Not filling out the issue template needs a good reason, otherwise the
issue may be closed instantly! Please take the time to help us debugging the
problem by collecting information, even if it seems irrelevant to you. Thanks!
-->
## Output of `restic version`
## How did you start restic exactly? (Include the complete command line)
## What backend/server/service did you use?
## Expected behavior
## Actual behavior
## Steps to reproduce the behavior
## Do you have any idea what may have caused this?

93
.github/ISSUE_TEMPLATE/Bug.md vendored Normal file
View File

@@ -0,0 +1,93 @@
---
name: Bug report
about: Report a problem with restic to help us resolve it and improve
---
<!--
Welcome! - We kindly ask that you:
1. Fill out the issue template below - not doing so needs a good reason.
2. Use the forum if you have a question rather than a bug or feature request.
The forum is at: https://forum.restic.net
NOTE: Not filling out the issue template needs a good reason, as otherwise it
may take a lot longer to find the problem, not to mention it can take up a lot
more time which can otherwise be spent on development. Please also take the
time to help us debug the issue by collecting relevant information, even if
it doesn't seem to be relevant to you. Thanks!
The forum is a better place for questions about restic or general suggestions
and topics, e.g. usage or documentation questions! This issue tracker is mainly
for tracking bugs and feature requests directly relating to the development of
the software itself, rather than the project.
Thanks for understanding, and for contributing to the project!
-->
Output of `restic version`
--------------------------
How did you run restic exactly?
-------------------------------
<!--
This section should include at least:
* The complete command line and any environment variables you used to
configure restic's backend access. Make sure to replace sensitive values!
* The output of the commands, what restic prints gives may give us much
information to diagnose the problem!
-->
What backend/server/service did you use to store the repository?
----------------------------------------------------------------
Expected behavior
-----------------
<!--
Describe what you'd like restic to do differently.
-->
Actual behavior
---------------
<!--
In this section, please try to concentrate on observations, so only describe
what you observed directly.
-->
Steps to reproduce the behavior
-------------------------------
<!--
The more time you spend describing an easy way to reproduce the behavior (if
this is possible), the easier it is for the project developers to fix it!
-->
Do you have any idea what may have caused this?
-----------------------------------------------
Do you have an idea how to solve the issue?
-------------------------------------------
Did restic help you or made you happy in any way?
-------------------------------------------------
<!--
Answering this question is not required, but if you have anything positive to share, please do so here!
Sometimes we get tired of reading bug reports all day and a little positive end note does wonders.
Idea by Joey Hess, https://joeyh.name/blog/entry/two_holiday_stories/
-->

57
.github/ISSUE_TEMPLATE/Feature.md vendored Normal file
View File

@@ -0,0 +1,57 @@
---
name: Feature request
about: Suggest a new feature or enhancement for restic
---
<!--
Welcome! - We kindly ask that you:
1. Fill out the issue template below - not doing so needs a good reason.
2. Use the forum if you have a question rather than a bug or feature request.
The forum is at: https://forum.restic.net
The forum is a better place for questions about restic or general suggestions
and topics, e.g. usage or documentation questions! This issue tracker is mainly
for tracking bugs and feature requests directly relating to the development of
the software itself, rather than the project.
Thanks for understanding, and for contributing to the project!
-->
Output of `restic version`
--------------------------
<!--
Please add the version of restic you're currently using here, this helps us
later to see what has changed in restic when we revisit this issue after some
time.
-->
What should restic do differently? Which functionality do you think we should add?
----------------------------------------------------------------------------------
<!--
Please describe the feature you'd like us to add here.
-->
What are you trying to do?
--------------------------
<!--
This section should contain a brief description what you're trying to do, which
would be possible after implementing the new feature.
-->
Did restic help you or made you happy in any way?
-------------------------------------------------
<!--
Answering this question is not required, but if you have anything positive to share, please do so here!
Sometimes we get tired of reading bug reports all day and a little positive end note does wonders.
Idea by Joey Hess, https://joeyh.name/blog/entry/two_holiday_stories/
-->

36
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@@ -0,0 +1,36 @@
<!--
Thank you very much for contributing code or documentation to restic! Please
fill out the following questions to make it easier for us to review your
changes.
You do not need to check all the boxes below all at once, feel free to take
your time and add more commits. If you're done and ready for review, please
check the last box.
-->
What is the purpose of this change? What does it change?
--------------------------------------------------------
<!--
Describe the changes here, as detailed as needed.
-->
Was the change discussed in an issue or in the forum before?
------------------------------------------------------------
<!--
Link issues and relevant forum posts here.
-->
Checklist
---------
- [ ] I have read the [Contribution Guidelines](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#providing-patches)
- [ ] I have added tests for all changes in this PR
- [ ] I have added documentation for the changes (in the manual)
- [ ] There's a new file in `changelog/unreleased/` that describes the changes for our users (template [here](https://github.com/restic/restic/blob/master/changelog/TEMPLATE))
- [ ] I have run `gofmt` on the code in all commits
- [ ] All commit messages are formatted in the same style as [the other commits in the repo](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#git-commits)
- [ ] I'm done, this Pull Request is ready for review

3
.gitignore vendored
View File

@@ -1,6 +1,3 @@
/pkg
/bin
/restic
/.vagrant
/vendor/pkg
/doc/_build

View File

@@ -1,35 +1,20 @@
language: go
sudo: false
go:
- 1.7.6
- 1.8.3
- tip
os:
- linux
- osx
env:
matrix:
RESTIC_TEST_FUSE=0
matrix:
exclude:
- os: osx
go: 1.7.6
- os: osx
go: tip
- os: linux
go: 1.8.3
include:
- os: linux
go: 1.8.3
go: "1.9.x"
env: RESTIC_TEST_FUSE=0 RESTIC_TEST_CLOUD_BACKENDS=0 RESTIC_BUILD_SOLARIS=0
# only run fuse and cloud backends tests on Travis for the latest Go on Linux
- os: linux
go: "1.10.x"
sudo: true
env:
RESTIC_TEST_FUSE=1
allow_failures:
- go: tip
- os: osx
go: "1.10.x"
env: RESTIC_TEST_FUSE=0 RESTIC_TEST_CLOUD_BACKENDS=0
branches:
only:

File diff suppressed because it is too large Load Diff

View File

@@ -60,50 +60,35 @@ uploading it somewhere or post only the parts that are really relevant.
Development Environment
=======================
For development you need the build tool [`gb`](https://getgb.io), it can be
installed by running the following command:
In order to compile restic with the `go` tool directly, it needs to be checked
out at the right path within a `GOPATH`. The concept of a `GOPATH` is explained
in ["How to write Go code"](https://golang.org/doc/code.html).
$ go get github.com/constabulary/gb/...
The repository contains two directories with code: `src/` contains the code
written for restic, whereas `vendor/` contains copies of libraries restic
depends on. The libraries are managed with the `gb vendor` command.
Just clone the repository, `cd` to it and run `gb build` to build the binary:
If you do not have a directory with Go code yet, executing the following
instructions in your shell will create one for you and check out the restic
repo:
$ export GOPATH="$HOME/go"
$ mkdir -p "$GOPATH/src/github.com/restic"
$ cd "$GOPATH/src/github.com/restic"
$ git clone https://github.com/restic/restic
$ cd restic
$ gb build
[...]
$ bin/restic version
You can then build restic as follows:
$ go build ./cmd/restic
$ ./restic version
restic compiled manually
compiled at unknown time with go1.7
compiled with go1.8.3 on linux/amd64
The following commands can be used to run all the tests:
$ gb test
ok github.com/restic/restic 8.174s
[...]
$ go test ./cmd/... ./internal/...
If you want to run your tests on Linux, OpenBSD or FreeBSD, you can use
[vagrant](https://www.vagrantup.com/) with the provided `Vagrantfile` to
quickly set up VMs and run the tests, e.g.:
$ vagrant up freebsd
[...]
$ vagrant ssh freebsd -c 'cd restic/restic; go test -v ./...'
[...]
The default `go` tool can also be used by setting the environment variable
`GOPATH` to the following value while being in the top level directory in the
git repository:
$ export GOPATH=$PWD:$PWD/vendor
The file `.envrc` allows automatic `GOPATH` configuration with
[direnv](https://direnv.net/), inspect the file and then allow automatic
configuration by running `direnv allow`.
The repository contains two sets of directories with code: `cmd/` and
`internal/` contain the code written for restic, whereas `vendor/` contains
copies of libraries restic depends on. The libraries are managed with the
[`dep`](https://github.com/golang/dep) tool.
Providing Patches
=================
@@ -122,7 +107,8 @@ down to the following steps:
2. Clone the repository locally and create a new branch. If you are working on
the code itself, please set up the development environment as described in
the previous section.
the previous section. Especially take care to place your forked repository
at the correct path (`src/github.com/restic/restic`) within your `GOPATH`.
3. Then commit your changes as fine grained as possible, as smaller patches,
that handle one and only one issue are easier to discuss and merge.
@@ -134,7 +120,8 @@ down to the following steps:
6. You will receive comments on your code and the feature or bug that they
address. Maybe you need to rework some minor things, in this case push new
commits to the branch you created for the pull request, they will be
commits to the branch you created for the pull request (or amend the
existing commit, use common sense to decide which is better), they will be
automatically added to the pull request.
7. If your pull request changes anything that users should be aware of (a
@@ -144,7 +131,7 @@ down to the following steps:
would I need to be aware of with this change.
8. Once your code looks good and passes all the tests, we'll merge it. Thanks
a low for your contribution!
a lot for your contribution!
Please provide the patches for each bug or feature in a separate branch and
open up a pull request for each.
@@ -170,14 +157,14 @@ what the tests are there for.
Git Commits
-----------
I would be good if you could follow the same general style regarding Git
It would be good if you could follow the same general style regarding Git
commits as the rest of the project, this makes reviewing code, browsing the
history and triaging bugs much easier.
Git commit messages have a very terse summary in the first line of the commit
message, followed by an empty line, followed by a more verbose description or a
List of changed things. For examples, please refer to the excellent [How to
Write a Git Commit Message](http://chris.beams.io/posts/git-commit/).
Write a Git Commit Message](https://chris.beams.io/posts/git-commit/).
If you change/add multiple different things that aren't related at all, try to
make several smaller commits. This is much easier to review. Using `git add -p`

View File

@@ -1,57 +0,0 @@
# This Dockerfiles configures a container that is similar to the Travis CI
# environment and can be used to run tests locally.
#
# build the image:
# docker build -t restic/test .
#
# run all tests and cross-compile restic:
# docker run --rm -v $PWD:/home/travis/restic restic/test go run run_integration_tests.go -minio minio
#
# run interactively:
# docker run --interactive --tty --rm -v $PWD:/home/travis/restic restic/test /bin/bash
#
# run a subset of tests:
# docker run --rm -v $PWD:/home/travis/restic restic/test gb test -v ./backend
#
# build the image for an older version of Go:
# docker build --build-arg GOVERSION=1.6.4 -t restic/test:go1.6.4 .
FROM ubuntu:14.04
ARG GOVERSION=1.8.3
ARG GOARCH=amd64
# install dependencies
RUN apt-get update
RUN apt-get install -y --no-install-recommends ca-certificates wget git build-essential openssh-server
# add and configure user
ENV HOME /home/travis
RUN useradd -m -d $HOME -s /bin/bash travis
# run everything below as user travis
USER travis
WORKDIR $HOME
# download and install Go
RUN wget -q -O /tmp/go.tar.gz https://storage.googleapis.com/golang/go${GOVERSION}.linux-${GOARCH}.tar.gz
RUN tar xf /tmp/go.tar.gz && rm -f /tmp/go.tar.gz
ENV GOROOT $HOME/go
ENV GOPATH $HOME/gopath
ENV PATH $PATH:$GOROOT/bin:$GOPATH/bin:$HOME/bin
RUN mkdir -p $HOME/restic
# pre-install tools, this speeds up running the tests itself
RUN go get github.com/constabulary/gb/...
RUN go get golang.org/x/tools/cmd/cover
RUN go get github.com/mitchellh/gox
RUN go get github.com/pierrre/gotestcover
RUN mkdir $HOME/bin \
&& wget -q -O $HOME/bin/minio https://dl.minio.io/server/minio/release/linux-${GOARCH}/minio \
&& chmod +x $HOME/bin/minio
# set TRAVIS_BUILD_DIR for integration script
ENV TRAVIS_BUILD_DIR $HOME/restic
WORKDIR $HOME/restic

27
GOVERNANCE.md Normal file
View File

@@ -0,0 +1,27 @@
# restic project governance
## Overview
The restic project uses a governance model commonly described as Benevolent
Dictator For Life (BDFL). This document outlines our understanding of what this
means. It is derived from the [i3 window manager project
governance](https://raw.githubusercontent.com/i3/i3/next/.github/GOVERNANCE.md).
## Roles
* user: anyone who interacts with the restic project
* core contributor: a handful of people who have contributed significantly to
the project by any means (issue triage, support, documentation, code, etc.).
Core contributors are recognizable via GitHubs "Member" badge.
* Benevolent Dictator For Life (BDFL): a single individual who makes decisions
when consensus cannot be reached. restic's current BDFL is [@fd0](https://github.com/fd0).
## Decision making process
In general, we try to reach consensus in discussions. In case consensus cannot
be reached, the BDFL makes a decision.
## Contribution process
The contribution process is described in a separate document called
[CONTRIBUTING](CONTRIBUTING.md).

474
Gopkg.lock generated Normal file
View File

@@ -0,0 +1,474 @@
# This file is autogenerated, do not edit; changes may be undone by the next 'dep ensure'.
[[projects]]
branch = "master"
digest = "1:94e9caf404409a2990cfd22aca37d758494c098eff3e2c37fda1abed862e74dd"
name = "bazil.org/fuse"
packages = [
".",
"fs",
"fuseutil",
]
pruneopts = "UT"
revision = "65cc252bf6691cb3c7014bcb2c8dc29de91e3a7e"
[[projects]]
digest = "1:5c3894b2aa4d6bead0ceeea6831b305d62879c871780e7b76296ded1b004bc57"
name = "cloud.google.com/go"
packages = ["compute/metadata"]
pruneopts = "UT"
revision = "aad3f485ee528456e0768f20397b4d9dd941e755"
version = "v0.25.0"
[[projects]]
digest = "1:46ea9487304f4b3c787f54483ecb13a338d686dcd670db0ab1a112ed0ae2128e"
name = "github.com/Azure/azure-sdk-for-go"
packages = [
"storage",
"version",
]
pruneopts = "UT"
revision = "4e8cbbfb1aeab140cd0fa97fd16b64ee18c3ca6a"
version = "v19.1.0"
[[projects]]
digest = "1:27d0cd1a78fc836f7c0f07749d029a5f7895c84ad066187b08b70e9d1830098e"
name = "github.com/Azure/go-autorest"
packages = [
"autorest",
"autorest/adal",
"autorest/azure",
"autorest/date",
"logger",
"version",
]
pruneopts = "UT"
revision = "dd94e014aaf16d1df746762e392aa201c1b4c461"
version = "v10.15.0"
[[projects]]
digest = "1:2209584c0f7c9b68c23374e659357ab546e1b70eec2761f03280f69a8fd23d77"
name = "github.com/cenkalti/backoff"
packages = ["."]
pruneopts = "UT"
revision = "2ea60e5f094469f9e65adb9cd103795b73ae743e"
version = "v2.0.0"
[[projects]]
digest = "1:7cb4fdca4c251b3ef8027c90ea35f70c7b661a593b9eeae34753c65499098bb1"
name = "github.com/cpuguy83/go-md2man"
packages = ["md2man"]
pruneopts = "UT"
revision = "20f5889cbdc3c73dbd2862796665e7c465ade7d1"
version = "v1.0.8"
[[projects]]
digest = "1:76dc72490af7174349349838f2fe118996381b31ea83243812a97e5a0fd5ed55"
name = "github.com/dgrijalva/jwt-go"
packages = ["."]
pruneopts = "UT"
revision = "06ea1031745cb8b3dab3f6a236daf2b0aa468b7e"
version = "v3.2.0"
[[projects]]
branch = "master"
digest = "1:6f9339c912bbdda81302633ad7e99a28dfa5a639c864061f1929510a9a64aa74"
name = "github.com/dustin/go-humanize"
packages = ["."]
pruneopts = "UT"
revision = "9f541cc9db5d55bce703bd99987c9d5cb8eea45e"
[[projects]]
digest = "1:c7edfbb6320d6a93240d663dc52bca92bed4c116abe54c35679eec4e7cc2bd77"
name = "github.com/elithrar/simple-scrypt"
packages = ["."]
pruneopts = "UT"
revision = "d150773194090feb6c897805a7bcea8d49544e2c"
version = "v1.3.0"
[[projects]]
digest = "1:fe8a03a8222d5b913f256972933d26d24ad7c8286692a42943bc01633cc8fce3"
name = "github.com/go-ini/ini"
packages = ["."]
pruneopts = "UT"
revision = "358ee7663966325963d4e8b2e1fbd570c5195153"
version = "v1.38.1"
[[projects]]
digest = "1:15042ad3498153684d09f393bbaec6b216c8eec6d61f63dff711de7d64ed8861"
name = "github.com/golang/protobuf"
packages = ["proto"]
pruneopts = "UT"
revision = "b4deda0973fb4c70b50d226b1af49f3da59f5265"
version = "v1.1.0"
[[projects]]
digest = "1:2e3c336fc7fde5c984d2841455a658a6d626450b1754a854b3b32e7a8f49a07a"
name = "github.com/google/go-cmp"
packages = [
"cmp",
"cmp/internal/diff",
"cmp/internal/function",
"cmp/internal/value",
]
pruneopts = "UT"
revision = "3af367b6b30c263d47e8895973edcca9a49cf029"
version = "v0.2.0"
[[projects]]
digest = "1:870d441fe217b8e689d7949fef6e43efbc787e50f200cb1e70dbca9204a1d6be"
name = "github.com/inconshreveable/mousetrap"
packages = ["."]
pruneopts = "UT"
revision = "76626ae9c91c4f2a10f34cad8ce83ea42c93bb75"
version = "v1.0"
[[projects]]
digest = "1:190ff84d9b2ed6589088f178cba8edb4b8ecb334df4572421fb016be1ac20463"
name = "github.com/juju/ratelimit"
packages = ["."]
pruneopts = "UT"
revision = "59fac5042749a5afb9af70e813da1dd5474f0167"
version = "1.0.1"
[[projects]]
digest = "1:9cedee824c21326bd26950bd9e1ffe9dc4e7ca03dc8634d0e6f954ee6a383172"
name = "github.com/kr/fs"
packages = ["."]
pruneopts = "UT"
revision = "1455def202f6e05b95cc7bfc7e8ae67ae5141eba"
version = "v0.1.0"
[[projects]]
digest = "1:1faa76bd9bffce9c25eaca0597afb67bd05a21ae57fe4378154ce8855ef163d1"
name = "github.com/kurin/blazer"
packages = [
"b2",
"base",
"internal/b2assets",
"internal/b2types",
"internal/blog",
"x/window",
]
pruneopts = "UT"
revision = "caf65aa76491dc533bac68ad3243ce72fa4e0a0a"
version = "v0.5.1"
[[projects]]
digest = "1:4e878df5f4e9fd625bf9c9aac77ef7cbfa4a74c01265505527c23470c0e40300"
name = "github.com/marstr/guid"
packages = ["."]
pruneopts = "UT"
revision = "8bd9a64bf37eb297b492a4101fb28e80ac0b290f"
version = "v1.1.0"
[[projects]]
digest = "1:d4d17353dbd05cb52a2a52b7fe1771883b682806f68db442b436294926bbfafb"
name = "github.com/mattn/go-isatty"
packages = ["."]
pruneopts = "UT"
revision = "0360b2af4f38e8d38c7fce2a9f4e702702d73a39"
version = "v0.0.3"
[[projects]]
digest = "1:95c73c666919be2843b955eafc83f58c136312b74f79c703152f4c4a95fd64dc"
name = "github.com/minio/minio-go"
packages = [
".",
"pkg/credentials",
"pkg/encrypt",
"pkg/s3signer",
"pkg/s3utils",
"pkg/set",
]
pruneopts = "UT"
revision = "70799fe8dae6ecfb6c7d7e9e048fce27f23a1992"
version = "v6.0.5"
[[projects]]
branch = "master"
digest = "1:8eb17c2ec4df79193ae65b621cd1c0c4697db3bc317fe6afdc76d7f2746abd05"
name = "github.com/mitchellh/go-homedir"
packages = ["."]
pruneopts = "UT"
revision = "3864e76763d94a6df2f9960b16a20a33da9f9a66"
[[projects]]
digest = "1:928de5172dd3563964d1b88a4ee3775cf72e16f1efabb482ab6d0e0bab86ee69"
name = "github.com/ncw/swift"
packages = ["."]
pruneopts = "UT"
revision = "b2a7479cf26fa841ff90dd932d0221cb5c50782d"
version = "v1.0.39"
[[projects]]
digest = "1:40e195917a951a8bf867cd05de2a46aaf1806c50cf92eebf4c16f78cd196f747"
name = "github.com/pkg/errors"
packages = ["."]
pruneopts = "UT"
revision = "645ef00459ed84a119197bfb8d8205042c6df63d"
version = "v0.8.0"
[[projects]]
digest = "1:cfa0d7741863a0e1d30e0ccdd4b48a96a471cdb47892303de8b92c3713af3e77"
name = "github.com/pkg/profile"
packages = ["."]
pruneopts = "UT"
revision = "5b67d428864e92711fcbd2f8629456121a56d91f"
version = "v1.2.1"
[[projects]]
digest = "1:23ed92ba5d90a2dfe817f3895027ccef796e79c30be5125d48e17afdcc395d73"
name = "github.com/pkg/sftp"
packages = ["."]
pruneopts = "UT"
revision = "57673e38ea946592a59c26592b7e6fbda646975b"
version = "v1.8.0"
[[projects]]
digest = "1:0d67664e93e366f072ac9672feea29bfc63c9f90f005e9e8a0df1954153f5a14"
name = "github.com/pkg/xattr"
packages = ["."]
pruneopts = "UT"
revision = "ae385d07bb53f092fcc7daaf738d8513df084931"
version = "v0.3.1"
[[projects]]
digest = "1:13ecc4000f49cf0aa3ee56fffcc93119c8edffacfff08674c80d2757d8c33a83"
name = "github.com/restic/chunker"
packages = ["."]
pruneopts = "UT"
revision = "db83917be3b88cc307464b7d8a221c173e34a0db"
version = "v0.2.0"
[[projects]]
digest = "1:8bc629776d035c003c7814d4369521afe67fdb8efc4b5f66540d29343b98cf23"
name = "github.com/russross/blackfriday"
packages = ["."]
pruneopts = "UT"
revision = "55d61fa8aa702f59229e6cff85793c22e580eaf5"
version = "v1.5.1"
[[projects]]
digest = "1:274f67cb6fed9588ea2521ecdac05a6d62a8c51c074c1fccc6a49a40ba80e925"
name = "github.com/satori/go.uuid"
packages = ["."]
pruneopts = "UT"
revision = "f58768cc1a7a7e77a3bd49e98cdd21419399b6a3"
version = "v1.2.0"
[[projects]]
digest = "1:d867dfa6751c8d7a435821ad3b736310c2ed68945d05b50fb9d23aee0540c8cc"
name = "github.com/sirupsen/logrus"
packages = ["."]
pruneopts = "UT"
revision = "3e01752db0189b9157070a0e1668a620f9a85da2"
version = "v1.0.6"
[[projects]]
digest = "1:e01b05ba901239c783dfe56450bcde607fc858908529868259c9a8765dc176d0"
name = "github.com/spf13/cobra"
packages = [
".",
"doc",
]
pruneopts = "UT"
revision = "ef82de70bb3f60c65fb8eebacbb2d122ef517385"
version = "v0.0.3"
[[projects]]
digest = "1:9424f440bba8f7508b69414634aef3b2b3a877e522d8a4624692412805407bb7"
name = "github.com/spf13/pflag"
packages = ["."]
pruneopts = "UT"
revision = "583c0c0531f06d5278b7d917446061adc344b5cd"
version = "v1.0.1"
[[projects]]
branch = "master"
digest = "1:eefb1f49ec07e71206d4c9ea1a3e634cad331c2180733e4121b8ae39e8e92ecb"
name = "golang.org/x/crypto"
packages = [
"argon2",
"blake2b",
"curve25519",
"ed25519",
"ed25519/internal/edwards25519",
"internal/chacha20",
"internal/subtle",
"pbkdf2",
"poly1305",
"scrypt",
"ssh",
"ssh/terminal",
]
pruneopts = "UT"
revision = "c126467f60eb25f8f27e5a981f32a87e3965053f"
[[projects]]
branch = "master"
digest = "1:8356aa7bdcb10a210b814b64ff76d61de7c36ac4cb6263de3af5e3e2e546956d"
name = "golang.org/x/net"
packages = [
"context",
"context/ctxhttp",
"http/httpguts",
"http2",
"http2/hpack",
"idna",
]
pruneopts = "UT"
revision = "32f9bdbd7df18e8641d215e7ea68be88b971feb0"
[[projects]]
branch = "master"
digest = "1:bea0314c10bd362ab623af4880d853b5bad3b63d0ab9945c47e461b8d04203ed"
name = "golang.org/x/oauth2"
packages = [
".",
"google",
"internal",
"jws",
"jwt",
]
pruneopts = "UT"
revision = "3d292e4d0cdc3a0113e6d207bb137145ef1de42f"
[[projects]]
branch = "master"
digest = "1:39ebcc2b11457b703ae9ee2e8cca0f68df21969c6102cb3b705f76cca0ea0239"
name = "golang.org/x/sync"
packages = ["errgroup"]
pruneopts = "UT"
revision = "1d60e4601c6fd243af51cc01ddf169918a5407ca"
[[projects]]
branch = "master"
digest = "1:a220a85c72a6cb7339c412cb2b117019a7fd94007cdfffb3b5b1d058227a2bf8"
name = "golang.org/x/sys"
packages = [
"cpu",
"unix",
"windows",
]
pruneopts = "UT"
revision = "bd9dbc187b6e1dacfdd2722a87e83093c2d7bd6e"
[[projects]]
digest = "1:e5a8511f063c38c51ab9ab80e718e9149f692652aeb4e393a8c020dd1bf38ca2"
name = "golang.org/x/text"
packages = [
"collate",
"collate/build",
"encoding",
"encoding/internal",
"encoding/internal/identifier",
"encoding/unicode",
"internal/colltab",
"internal/gen",
"internal/tag",
"internal/triegen",
"internal/ucd",
"internal/utf8internal",
"language",
"runes",
"secure/bidirule",
"transform",
"unicode/bidi",
"unicode/cldr",
"unicode/norm",
"unicode/rangetable",
]
pruneopts = "UT"
revision = "f21a4dfb5e38f5895301dc265a8def02365cc3d0"
version = "v0.3.0"
[[projects]]
branch = "master"
digest = "1:fb983acae7bd9c3ed9aadc1b1241d9e559ed21dbf84c17a0dda663ca169ccd69"
name = "google.golang.org/api"
packages = [
"gensupport",
"googleapi",
"googleapi/internal/uritemplates",
"storage/v1",
]
pruneopts = "UT"
revision = "31ca0e01cd791f07750cb23fc99327721f753290"
[[projects]]
digest = "1:c8907869850adaa8bd7631887948d0684f3787d0912f1c01ab72581a6c34432e"
name = "google.golang.org/appengine"
packages = [
".",
"internal",
"internal/app_identity",
"internal/base",
"internal/datastore",
"internal/log",
"internal/modules",
"internal/remote_api",
"internal/urlfetch",
"urlfetch",
]
pruneopts = "UT"
revision = "b1f26356af11148e710935ed1ac8a7f5702c7612"
version = "v1.1.0"
[[projects]]
branch = "v2"
digest = "1:5bb148b78468350091db2ffbb2370f35cc6dcd74d9378a31b1c7b86ff7528f08"
name = "gopkg.in/tomb.v2"
packages = ["."]
pruneopts = "UT"
revision = "d5d1b5820637886def9eef33e03a27a9f166942c"
[[projects]]
digest = "1:342378ac4dcb378a5448dd723f0784ae519383532f5e70ade24132c4c8693202"
name = "gopkg.in/yaml.v2"
packages = ["."]
pruneopts = "UT"
revision = "5420a8b6744d3b0345ab293f6fcba19c978f1183"
version = "v2.2.1"
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
input-imports = [
"bazil.org/fuse",
"bazil.org/fuse/fs",
"github.com/Azure/azure-sdk-for-go/storage",
"github.com/cenkalti/backoff",
"github.com/elithrar/simple-scrypt",
"github.com/google/go-cmp/cmp",
"github.com/juju/ratelimit",
"github.com/kurin/blazer/b2",
"github.com/mattn/go-isatty",
"github.com/minio/minio-go",
"github.com/minio/minio-go/pkg/credentials",
"github.com/ncw/swift",
"github.com/pkg/errors",
"github.com/pkg/profile",
"github.com/pkg/sftp",
"github.com/pkg/xattr",
"github.com/restic/chunker",
"github.com/spf13/cobra",
"github.com/spf13/cobra/doc",
"github.com/spf13/pflag",
"golang.org/x/crypto/poly1305",
"golang.org/x/crypto/scrypt",
"golang.org/x/crypto/ssh/terminal",
"golang.org/x/net/context",
"golang.org/x/net/context/ctxhttp",
"golang.org/x/net/http2",
"golang.org/x/oauth2/google",
"golang.org/x/sync/errgroup",
"golang.org/x/sys/unix",
"golang.org/x/text/encoding/unicode",
"google.golang.org/api/googleapi",
"google.golang.org/api/storage/v1",
"gopkg.in/tomb.v2",
]
solver-name = "gps-cdcl"
solver-version = 1

25
Gopkg.toml Normal file
View File

@@ -0,0 +1,25 @@
# Gopkg.toml example
#
# Refer to https://github.com/golang/dep/blob/master/docs/Gopkg.toml.md
# for detailed Gopkg.toml documentation.
#
# required = ["github.com/user/thing/cmd/thing"]
# ignored = ["github.com/user/project/pkgX", "bitbucket.org/user/project/pkgA/pkgY"]
#
# [[constraint]]
# name = "github.com/user/project"
# version = "1.0.0"
#
# [[constraint]]
# name = "github.com/user/project2"
# branch = "dev"
# source = "github.com/myfork/project2"
#
# [[override]]
# name = "github.com/x/y"
# version = "2.4.0"
[prune]
unused-packages = true
go-tests = true

18
LICENSE
View File

@@ -1,19 +1,21 @@
BSD 2-Clause License
Copyright (c) 2014, Alexander Neumann <alexander@bumpern.de>
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR

View File

@@ -6,7 +6,8 @@ restic:
go run build.go
clean:
rm -rf restic
rm -f restic
test:
go test ./...
go test ./cmd/... ./internal/...

View File

@@ -1,4 +1,4 @@
|Documentation| |Build Status| |Build status| |Report Card| |Say Thanks|
|Documentation| |Build Status| |Build status| |Report Card| |Say Thanks| |TestCoverage| |Reviewed by Hound|
Introduction
------------
@@ -7,11 +7,13 @@ restic is a backup program that is fast, efficient and secure.
For detailed usage and installation instructions check out the `documentation <https://restic.readthedocs.io/en/latest>`__.
You can ask questions in our `Discourse forum <https://forum.restic.net>`__.
Quick start
-----------
Once you've `installed
<https://restic.readthedocs.io/en/latest/installation.html>`__ restic, start
<https://restic.readthedocs.io/en/latest/020_installation.html>`__ restic, start
off with creating a repository for your backups:
.. code-block:: console
@@ -27,7 +29,7 @@ and add some data:
.. code-block:: console
$ restic -r /tmp/backup backup ~/work
$ restic --repo /tmp/backup backup ~/work
enter password for repository:
scan [/home/user/work]
scanned 764 directories, 1816 files in 0:00
@@ -35,7 +37,27 @@ and add some data:
duration: 0:29, 54.47MiB/s
snapshot 40dc1520 saved
For more options check out the `manual guide <https://restic.readthedocs.io/en/latest/manual.html>`__.
Next you can either use ``restic restore`` to restore files or use ``restic
mount`` to mount the repository via fuse and browse the files from previous
snapshots.
For more options check out the `online documentation <https://restic.readthedocs.io/en/latest/>`__.
Backends
--------
Saving a backup on the same machine is nice but not a real backup strategy.
Therefore, restic supports the following backends for storing backups natively:
- `Local directory <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#local>`__
- `sftp server (via SSH) <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#sftp>`__
- `HTTP REST server <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#rest-server>`__ (`protocol <doc/100_references.rst#rest-backend>`__ `rest-server <https://github.com/restic/rest-server>`__)
- `AWS S3 <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#amazon-s3>`__ (either from Amazon or using the `Minio <https://minio.io>`__ server)
- `OpenStack Swift <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#openstack-swift>`__
- `BackBlaze B2 <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#backblaze-b2>`__
- `Microsoft Azure Blob Storage <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#microsoft-azure-blob-storage>`__
- `Google Cloud Storage <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#google-cloud-storage>`__
- And many other services via the `rclone <https://rclone.org>`__ `Backend <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#other-services-via-rclone>`__
Design Principles
-----------------
@@ -72,30 +94,44 @@ Reproducible Builds
-------------------
The binaries released with each restic version starting at 0.6.1 are
[reproducible](https://reproducible-builds.org/), which means that you can
`reproducible <https://reproducible-builds.org/>`__, which means that you can
easily reproduce a byte identical version from the source code for that
release. Instructions on how to do that are contained in the
[build repository](https://github.com/restic/build).
`builder repository <https://github.com/restic/builder>`__.
News
----
You can follow the restic project on Twitter `@resticbackup <https://twitter.com/resticbackup>`__ or by subscribing to
the `development blog <https://restic.github.io/blog/>`__.
the `development blog <https://restic.net/blog/>`__.
License
-------
Restic is licensed under "BSD 2-Clause License". You can find the
Restic is licensed under `BSD 2-Clause License <https://opensource.org/licenses/BSD-2-Clause>`__. You can find the
complete text in ``LICENSE``.
Sponsorship
-----------
Backend integration tests for Google Cloud Storage and Microsoft Azure Blob
Storage are sponsored by `AppsCode <https://appscode.com>`__!
|AppsCode|
.. |Documentation| image:: https://readthedocs.org/projects/restic/badge/?version=latest
:target: https://restic.readthedocs.io/en/latest/?badge=latest
.. |Build Status| image:: https://travis-ci.org/restic/restic.svg?branch=master
:target: https://travis-ci.org/restic/restic
.. |Build Status| image:: https://travis-ci.com/restic/restic.svg?branch=master
:target: https://travis-ci.com/restic/restic
.. |Build status| image:: https://ci.appveyor.com/api/projects/status/nuy4lfbgfbytw92q/branch/master?svg=true
:target: https://ci.appveyor.com/project/fd0/restic/branch/master
.. |Report Card| image:: http://goreportcard.com/badge/github.com/restic/restic
:target: http://goreportcard.com/report/github.com/restic/restic
.. |Report Card| image:: https://goreportcard.com/badge/github.com/restic/restic
:target: https://goreportcard.com/report/github.com/restic/restic
.. |Say Thanks| image:: https://img.shields.io/badge/Say%20Thanks-!-1EAEDB.svg
:target: https://saythanks.io/to/restic
.. |TestCoverage| image:: https://codecov.io/gh/restic/restic/branch/master/graph/badge.svg
:target: https://codecov.io/gh/restic/restic
.. |AppsCode| image:: https://cdn.appscode.com/images/logo/appscode/ac-logo-color.png
:target: https://appscode.com
.. |Reviewed by Hound| image:: https://img.shields.io/badge/Reviewed_by-Hound-8E64B0.svg
:target: https://houndci.com

View File

@@ -1 +1 @@
0.6.1
0.9.2

124
Vagrantfile vendored
View File

@@ -1,124 +0,0 @@
# -*- mode: ruby -*-
# vi: set ft=ruby :
GO_VERSION = '1.7'
def packages_freebsd
return <<-EOF
pkg install -y git
pkg install -y curl
echo 'fuse_load="YES"' >> /boot/loader.conf
echo 'vfs.usermount=1' >> /etc/sysctl.conf
kldload fuse
sysctl vfs.usermount=1
pw groupmod operator -M vagrant
EOF
end
def packages_openbsd
return <<-EOF
. ~/.profile
pkg_add git curl bash gtar--
ln -sf /usr/local/bin/gtar /usr/local/bin/tar
EOF
end
def packages_linux
return <<-EOF
apt-get update
apt-get install -y git curl
EOF
end
def packages_darwin
return <<-EOF
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew cask install osxfuse
EOF
end
def install_gimme
return <<-EOF
rm -rf /opt/gimme
mkdir -p /opt/gimme || true
git clone https://github.com/meatballhat/gimme /opt/gimme
perl -p -i -e 's,/bin/bash,/usr/bin/env bash,' /opt/gimme/gimme
ln -sf /opt/gimme/gimme /usr/bin/gimme
EOF
end
def prepare_user(boxname)
return <<-EOF
mkdir -p ~/go/src
export PATH=/usr/local/bin:$PATH
gimme #{GO_VERSION} >> ~/.profile
echo export 'GOPATH=/vagrant/go' >> ~/.profile
echo export 'PATH=$GOPATH/bin:/usr/local/bin:$PATH' >> ~/.profile
. ~/.profile
go get golang.org/x/tools/cmd/cover
go get github.com/constabulary/gb/...
echo
echo "Run:"
echo " vagrant rsync #{boxname}"
echo " vagrant ssh #{boxname} -c 'cd /vagrant; gb build && gb test'"
EOF
end
def fix_perms
return <<-EOF
chown -R vagrant /vagrant
EOF
end
# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure(2) do |config|
# use rsync to copy content to the folder
config.vm.synced_folder ".", "/vagrant", :type => "rsync"
# fix permissions on synced folder
config.vm.provision "fix perms", :type => :shell, :inline => fix_perms
config.vm.define "linux" do |b|
b.vm.box = "ubuntu/trusty64"
b.vm.provision "packages linux", :type => :shell, :inline => packages_linux
b.vm.provision "install gimme", :type => :shell, :inline => install_gimme
b.vm.provision "prepare user", :type => :shell, :privileged => false, :inline => prepare_user("linux")
# fix network card
config.vm.provider "virtualbox" do |v|
v.customize ["modifyvm", :id, "--nictype1", "virtio"]
end
end
config.vm.define "freebsd" do |b|
b.vm.box = "geoffgarside/freebsd-10.1"
b.vm.provision "packages freebsd", :type => :shell, :inline => packages_freebsd
b.vm.provision "install gimme", :type => :shell, :inline => install_gimme
b.vm.provision "prepare user", :type => :shell, :privileged => false, :inline => prepare_user("freebsd")
end
config.vm.define "openbsd" do |b|
b.vm.box = "tmatilai/openbsd-5.6"
b.vm.provision "packages openbsd", :type => :shell, :inline => packages_openbsd
b.vm.provision "install gimme", :type => :shell, :inline => install_gimme
b.vm.provision "prepare user", :type => :shell, :privileged => false, :inline => prepare_user("openbsd")
end
config.vm.define "darwin" do |b|
#b.vm.box = "jhcook/osx-yosemite-10.10"
b.vm.box = "jhcook/yosemite-clitools"
b.vm.provision "packages darwin", :type => :shell, :privileged => false, :inline => packages_darwin
b.vm.provision "install gimme", :type => :shell, :inline => install_gimme
b.vm.provision "prepare user", :type => :shell, :privileged => false, :inline => prepare_user("darwin")
end
end

View File

@@ -17,8 +17,8 @@ init:
install:
- rmdir c:\go /s /q
- appveyor DownloadFile https://storage.googleapis.com/golang/go1.8.1.windows-amd64.msi
- msiexec /i go1.8.1.windows-amd64.msi /q
- appveyor DownloadFile https://dl.google.com/go/go1.10.windows-amd64.msi
- msiexec /i go1.10.windows-amd64.msi /q
- go version
- go env
- appveyor DownloadFile http://sourceforge.netcologne.de/project/gnuwin32/tar/1.13-1/tar-1.13-1-bin.zip -FileName tar.zip

238
build.go
View File

@@ -1,4 +1,33 @@
// +build ignore
// BSD 2-Clause License
//
// Copyright (c) 2016-2018, Alexander Neumann <alexander@bumpern.de>
// All rights reserved.
//
// This file has been copied from the repository at:
// https://github.com/fd0/build-go
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are met:
//
// * Redistributions of source code must retain the above copyright notice, this
// list of conditions and the following disclaimer.
//
// * Redistributions in binary form must reproduce the above copyright notice,
// this list of conditions and the following disclaimer in the documentation
// and/or other materials provided with the distribution.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
// AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
// DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
// FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
// DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
// SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
// CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
// OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
// +build ignore_build_go
package main
@@ -11,28 +40,39 @@ import (
"path"
"path/filepath"
"runtime"
"strconv"
"strings"
)
// config contains the configuration for the program to build.
var config = Config{
Name: "restic", // name of the program executable and directory
Namespace: "github.com/restic/restic", // subdir of GOPATH, e.g. "github.com/foo/bar"
Main: "github.com/restic/restic/cmd/restic", // package name for the main package
Tests: []string{ // tests to run
"github.com/restic/restic/internal/...",
"github.com/restic/restic/cmd/...",
},
MinVersion: GoVersion{Major: 1, Minor: 9, Patch: 0}, // minimum Go version supported
}
// Config configures the build.
type Config struct {
Name string
Namespace string
Main string
Tests []string
MinVersion GoVersion
}
var (
verbose bool
keepGopath bool
runTests bool
enableCGO bool
enablePIE bool
)
var config = struct {
Name string
Namespace string
Main string
Tests []string
}{
Name: "restic", // name of the program executable and directory
Namespace: "", // subdir of GOPATH, e.g. "github.com/foo/bar"
Main: "cmds/restic", // package name for the main package
Tests: []string{"restic/...", "cmds/..."}, // tests to run
}
// specialDir returns true if the file begins with a special character ('.' or '_').
func specialDir(name string) bool {
if name == "." {
@@ -77,7 +117,12 @@ func excludePath(name string) bool {
// └── restic
// └── foo.go
func updateGopath(dst, src, prefix string) error {
verbosePrintf("copy contents of %v to %v\n", src, filepath.Join(dst, prefix))
return filepath.Walk(src, func(name string, fi os.FileInfo, err error) error {
if name == src {
return err
}
if specialDir(name) {
if fi.IsDir() {
return filepath.SkipDir
@@ -86,6 +131,10 @@ func updateGopath(dst, src, prefix string) error {
return nil
}
if err != nil {
return err
}
if fi.IsDir() {
return nil
}
@@ -126,7 +175,6 @@ func copyFile(dst, src string) error {
if err != nil {
return err
}
defer fsrc.Close()
if err = os.MkdirAll(filepath.Dir(dst), 0755); err != nil {
fmt.Printf("MkdirAll(%v)\n", filepath.Dir(dst))
@@ -137,17 +185,28 @@ func copyFile(dst, src string) error {
if err != nil {
return err
}
defer fdst.Close()
_, err = io.Copy(fdst, fsrc)
if _, err = io.Copy(fdst, fsrc); err != nil {
return err
}
if err == nil {
err = fsrc.Close()
}
if err == nil {
err = fdst.Close()
}
if err == nil {
err = os.Chmod(dst, fi.Mode())
}
if err == nil {
err = os.Chtimes(dst, fi.ModTime(), fi.ModTime())
}
return err
return nil
}
// die prints the message with fmt.Fprintf() to stderr and exits with an error
@@ -167,8 +226,11 @@ func showUsage(output io.Writer) {
fmt.Fprintf(output, " -T --test run tests\n")
fmt.Fprintf(output, " -o --output set output file name\n")
fmt.Fprintf(output, " --enable-cgo use CGO to link against libc\n")
fmt.Fprintf(output, " --enable-pie use PIE buildmode\n")
fmt.Fprintf(output, " --goos value set GOOS for cross-compilation\n")
fmt.Fprintf(output, " --goarch value set GOARCH for cross-compilation\n")
fmt.Fprintf(output, " --goarm value set GOARM for cross-compilation\n")
fmt.Fprintf(output, " --tempdir dir use a specific directory for compilation\n")
}
func verbosePrintf(message string, args ...interface{}) {
@@ -194,13 +256,28 @@ func cleanEnv() (env []string) {
}
// build runs "go build args..." with GOPATH set to gopath.
func build(cwd, goos, goarch, gopath string, args ...string) error {
func build(cwd string, ver GoVersion, goos, goarch, goarm, gopath string, args ...string) error {
a := []string{"build"}
a = append(a, "-asmflags", fmt.Sprintf("-trimpath=%s", gopath))
a = append(a, "-gcflags", fmt.Sprintf("-trimpath=%s", gopath))
if ver.AtLeast(GoVersion{1, 10, 0}) {
verbosePrintf("Go version is at least 1.10, using new syntax for -gcflags\n")
// use new prefix
a = append(a, "-asmflags", fmt.Sprintf("all=-trimpath=%s", gopath))
a = append(a, "-gcflags", fmt.Sprintf("all=-trimpath=%s", gopath))
} else {
a = append(a, "-asmflags", fmt.Sprintf("-trimpath=%s", gopath))
a = append(a, "-gcflags", fmt.Sprintf("-trimpath=%s", gopath))
}
if enablePIE {
a = append(a, "-buildmode=pie")
}
a = append(a, args...)
cmd := exec.Command("go", a...)
cmd.Env = append(cleanEnv(), "GOPATH="+gopath, "GOARCH="+goarch, "GOOS="+goos)
if goarm != "" {
cmd.Env = append(cmd.Env, "GOARM="+goarm)
}
if !enableCGO {
cmd.Env = append(cmd.Env, "CGO_ENABLED=0")
}
@@ -208,7 +285,7 @@ func build(cwd, goos, goarch, gopath string, args ...string) error {
cmd.Dir = cwd
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
verbosePrintf("go %s\n", args)
verbosePrintf("go %s\n", a)
return cmd.Run()
}
@@ -289,10 +366,89 @@ func (cs Constants) LDFlags() string {
return strings.Join(l, " ")
}
// GoVersion is the version of Go used to compile the project.
type GoVersion struct {
Major int
Minor int
Patch int
}
// ParseGoVersion parses the Go version s. If s cannot be parsed, the returned GoVersion is null.
func ParseGoVersion(s string) (v GoVersion) {
if !strings.HasPrefix(s, "go") {
return
}
s = s[2:]
data := strings.Split(s, ".")
if len(data) < 2 || len(data) > 3 {
// invalid version
return GoVersion{}
}
var err error
v.Major, err = strconv.Atoi(data[0])
if err != nil {
return GoVersion{}
}
// try to parse the minor version while removing an eventual suffix (like
// "rc2" or so)
for s := data[1]; s != ""; s = s[:len(s)-1] {
v.Minor, err = strconv.Atoi(s)
if err == nil {
break
}
}
if v.Minor == 0 {
// no minor version found
return GoVersion{}
}
if len(data) >= 3 {
v.Patch, err = strconv.Atoi(data[2])
if err != nil {
return GoVersion{}
}
}
return
}
// AtLeast returns true if v is at least as new as other. If v is empty, true is returned.
func (v GoVersion) AtLeast(other GoVersion) bool {
var empty GoVersion
// the empty version satisfies all versions
if v == empty {
return true
}
if v.Major < other.Major {
return false
}
if v.Minor < other.Minor {
return false
}
if v.Patch < other.Patch {
return false
}
return true
}
func (v GoVersion) String() string {
return fmt.Sprintf("Go %d.%d.%d", v.Major, v.Minor, v.Patch)
}
func main() {
ver := runtime.Version()
if strings.HasPrefix(ver, "go1") && ver < "go1.7" {
fmt.Fprintf(os.Stderr, "Go version %s detected, restic requires at least Go 1.7\n", ver)
ver := ParseGoVersion(runtime.Version())
if !ver.AtLeast(config.MinVersion) {
fmt.Fprintf(os.Stderr, "%s detected, this program requires at least %s\n", ver, config.MinVersion)
os.Exit(1)
}
@@ -303,6 +459,9 @@ func main() {
targetGOOS := runtime.GOOS
targetGOARCH := runtime.GOARCH
targetGOARM := ""
gopath := ""
var outputFilename string
@@ -326,16 +485,24 @@ func main() {
case "-o", "--output":
skipNext = true
outputFilename = params[i+1]
case "--tempdir":
skipNext = true
gopath = params[i+1]
case "-T", "--test":
runTests = true
case "--enable-cgo":
enableCGO = true
case "--enable-pie":
enablePIE = true
case "--goos":
skipNext = true
targetGOOS = params[i+1]
case "--goarch":
skipNext = true
targetGOARCH = params[i+1]
case "--goarm":
skipNext = true
targetGOARM = params[i+1]
case "-h":
showUsage(os.Stdout)
return
@@ -346,6 +513,8 @@ func main() {
}
}
verbosePrintf("detected Go version %v\n", ver)
if len(buildTags) == 0 {
verbosePrintf("adding build-tag release\n")
buildTags = []string{"release"}
@@ -362,19 +531,21 @@ func main() {
die("Getwd(): %v\n", err)
}
gopath, err := ioutil.TempDir("", fmt.Sprintf("%v-build-", config.Name))
if err != nil {
die("TempDir(): %v\n", err)
if gopath == "" {
gopath, err = ioutil.TempDir("", fmt.Sprintf("%v-build-", config.Name))
if err != nil {
die("TempDir(): %v\n", err)
}
}
verbosePrintf("create GOPATH at %v\n", gopath)
if err = updateGopath(gopath, filepath.Join(root, "src"), config.Namespace); err != nil {
if err = updateGopath(gopath, root, config.Namespace); err != nil {
die("copying files from %v/src to %v/src failed: %v\n", root, gopath, err)
}
vendor := filepath.Join(root, "vendor", "src")
vendor := filepath.Join(root, "vendor")
if directoryExists(vendor) {
if err = updateGopath(gopath, vendor, ""); err != nil {
if err = updateGopath(gopath, vendor, filepath.Join(config.Namespace, "vendor")); err != nil {
die("copying files from %v to %v failed: %v\n", root, gopath, err)
}
}
@@ -401,7 +572,10 @@ func main() {
if err != nil {
die("Getwd() returned %v\n", err)
}
output := filepath.Join(cwd, outputFilename)
output := outputFilename
if !filepath.IsAbs(output) {
output = filepath.Join(cwd, output)
}
version := getVersion()
constants := Constants{}
@@ -417,7 +591,7 @@ func main() {
"-o", output, config.Main,
}
err = build(filepath.Join(gopath, "src"), targetGOOS, targetGOARCH, gopath, args...)
err = build(filepath.Join(gopath, "src"), ver, targetGOOS, targetGOARCH, targetGOARM, gopath, args...)
if err != nil {
die("build failed: %v\n", err)
}

View File

@@ -0,0 +1,8 @@
Enhancement: Make `forget` consistent
The `forget` command was corrected to be more consistent in which snapshots are
to be forgotten. It is possible that the new code removes more snapshots than
before, so please review what would be deleted by using the `--dry-run` option.
https://github.com/restic/restic/pull/957
https://github.com/restic/restic/issues/953

View File

@@ -0,0 +1,11 @@
Enhancement: Unify repository layout for all backends
Up to now the s3 backend used a special repository layout. We've decided to
unify the repository layout and implemented the default layout also for the s3
backend. For creating a new repository on s3 with the default layout, use
`restic -o s3.layout=default init`. For further commands the option is not
necessary any more, restic will automatically detect the correct layout to use.
A future version will switch to the default layout for new repositories.
https://github.com/restic/restic/pull/966
https://github.com/restic/restic/issues/965

View File

@@ -0,0 +1,15 @@
Enhancement: Improve memory and runtime for the s3 backend
We've updated the library used for accessing s3, switched to using a lower
level API and added caching for some requests. This lead to a decrease in
memory usage and a great speedup. In addition, we added benchmark functions for
all backends, so we can track improvements over time. The Continuous
Integration test service we're using (Travis) now runs the s3 backend tests not
only against a Minio server, but also against the Amazon s3 live service, so we
should be notified of any regressions much sooner.
https://github.com/restic/restic/pull/962
https://github.com/restic/restic/pull/960
https://github.com/restic/restic/pull/946
https://github.com/restic/restic/pull/938
https://github.com/restic/restic/pull/883

View File

@@ -0,0 +1,8 @@
Enhancement: Allow multiple parallel idle HTTP connections
Backends based on HTTP now allow several idle connections in parallel. This
is especially important for the REST backend, which (when used with a local
server) may create a lot connections and exhaust available ports quickly.
https://github.com/restic/restic/issues/985
https://github.com/restic/restic/pull/986

View File

@@ -0,0 +1,6 @@
Enhancement: Remove temporary path from binary in `build.go`
The `build.go` now strips the temporary directory used for compilation from
the binary. This is the first step in enabling reproducible builds.
https://github.com/restic/restic/pull/981

View File

@@ -0,0 +1,7 @@
Enhancement: Remove regular status reports
Regular status report: We've removed the status report that was printed
every 10 seconds when restic is run non-interactively. You can still force
reporting the current status by sending a `USR1` signal to the process.
https://github.com/restic/restic/pull/974

View File

@@ -0,0 +1,9 @@
Bugfix: Switch back to using the high-level minio-go API for s3
For the s3 backend we're back to using the high-level API the s3 client library
for uploading data, a few users reported dropped connections (which the library
will automatically retry now).
https://github.com/restic/restic/issues/1013
https://github.com/restic/restic/issues/1023
https://github.com/restic/restic/pull/1025

View File

@@ -0,0 +1,7 @@
Enhancement: Detect invalid backend name and print error
restic now tries to detect when an invalid/unknown backend is used and
returns an error message.
https://github.com/restic/restic/issues/1021
https://github.com/restic/restic/pull/1070

View File

@@ -0,0 +1,8 @@
Enhancement: Remove invalid pack files when `prune` is run
The `prune` command has been improved and will now remove invalid pack files,
for example files that have not been uploaded completely because a backup was
interrupted.
https://github.com/restic/restic/issues/1029
https://github.com/restic/restic/pull/1036

View File

@@ -0,0 +1,4 @@
Enhancement: Add Backblaze B2 backend
https://github.com/restic/restic/issues/512
https://github.com/restic/restic/pull/978

View File

@@ -0,0 +1,9 @@
Enhancement: Add dirs `tags` and `hosts` to fuse mount
The fuse mount now has two more directories: `tags` contains a subdir for
each tag, which in turn contains only the snapshots that have this tag. The
subdir `hosts` contains a subdir for each host that has a snapshot, and the
subdir contains the snapshots for that host.
https://github.com/restic/restic/issues/636
https://github.com/restic/restic/pull/1050

View File

@@ -0,0 +1,8 @@
Bugfix: Switch to `default` repo layout for the s3 backend
The default layout for the s3 backend is now `default` (instead of `s3legacy`).
Also, there's a new `migrate` command to convert an existing repo, it can be
run like this: `restic migrate s3_layout`
https://github.com/restic/restic/issues/965
https://github.com/restic/restic/pull/1004

View File

@@ -0,0 +1,8 @@
Enhancement: Improve performance of the `find` command
Improved performance for the `find` command: Restic recognizes paths it has
already checked for the files in question, so the number of backend requests
is reduced a lot.
https://github.com/restic/restic/issues/989
https://github.com/restic/restic/pull/993

View File

@@ -0,0 +1,4 @@
Enhancement: Add new backend for OpenStack Swift
https://github.com/restic/restic/pull/975
https://github.com/restic/restic/pull/648

View File

@@ -0,0 +1,5 @@
Enhancement: Improve performance of the fuse mount
Listing directories which contain large files now is significantly faster.
https://github.com/restic/restic/pull/998

View File

@@ -0,0 +1,10 @@
Enhancement: Create subdirs below `data/` for local/sftp backends
The local and sftp backends now create the subdirs below `data/` on
open/init. This way, restic makes sure that they always exist. This is
connected to an issue for the sftp server.
https://github.com/restic/restic/issues/1055
https://github.com/restic/rest-server/pull/11#issuecomment-309879710
https://github.com/restic/restic/pull/1077
https://github.com/restic/restic/pull/1105

View File

@@ -0,0 +1,8 @@
Enhancement: Allow loading credentials for s3 from IAM
When no S3 credentials are specified in the environment variables, restic
now tries to load credentials from an IAM instance profile when the s3
backend is used.
https://github.com/restic/restic/issues/1067
https://github.com/restic/restic/pull/1086

View File

@@ -0,0 +1,8 @@
Enhancement: Add `migrate` cmd to migrate from `s3legacy` to `default` layout
The `migrate` command for changing the `s3legacy` layout to the `default` layout
for s3 backends has been improved: It can now be restarted with `restic migrate
--force s3_layout` and automatically retries operations on error.
https://github.com/restic/restic/issues/1073
https://github.com/restic/restic/pull/1075

View File

@@ -0,0 +1,4 @@
Enhancement: Clarify semantic for `--tasg` for the `forget` command
https://github.com/restic/restic/issues/1081
https://github.com/restic/restic/pull/1090

View File

@@ -0,0 +1,5 @@
Enhancement: Ignore chmod() errors on filesystems which do not support it
https://github.com/restic/restic/pull/1080
https://github.com/restic/restic/pull/1112

View File

@@ -0,0 +1,3 @@
Enhancement: Print stats on SIGINFO on Darwin and FreeBSD (ctrl+t)
https://github.com/restic/restic/pull/1082

View File

@@ -0,0 +1,9 @@
Bugfix: Fix `prune`, only include existing files in indexes
A bug was found (and corrected) in the index rebuilding after prune, which led
to indexes which include blobs that were not present in the repo any more.
There were already checks in place which detected this situation and aborted
with an error message. A new run of either `prune` or `rebuild-index` corrected
the index files. This is now fixed and a test has been added to detect this.
https://github.com/restic/restic/pull/1115

View File

@@ -0,0 +1,8 @@
Enhancement: Make `key` command always prompt for a password
The `key` command now prompts for a password even if the original password
to access a repo has been specified via the `RESTIC_PASSWORD` environment
variable or a password file.
https://github.com/restic/restic/issues/1132
https://github.com/restic/restic/pull/1133

View File

@@ -0,0 +1,8 @@
Bugfix: Do not create a local repo unless `init` is used
When a restic command other than `init` is used with a local repository and the
repository directory does not exist, restic creates the directory structure.
That's an error, only the `init` command should create the dir.
https://github.com/restic/restic/issues/1167
https://github.com/restic/restic/pull/1182

View File

@@ -0,0 +1,4 @@
Enhancement: Resolve name conflicts, append a counter
https://github.com/restic/restic/issues/1179
https://github.com/restic/restic/pull/1209

View File

@@ -0,0 +1,7 @@
Enhancement: Add `--compact` to `snapshots` command
The option `--compact` was added to the `snapshots` command to get a better
overview of the snapshots in a repo. It limits each snapshot to a single line.
https://github.com/restic/restic/issues/1218
https://github.com/restic/restic/pull/1223

View File

@@ -0,0 +1,10 @@
Enhancement: Add `--exclude-caches` and `--exclude-if-present`
A new option `--exclude-caches` was added that allows excluding cache
directories (that are tagged as such). This is a special case of a more generic
option `--exclude-if-present` which excludes a directory if a file with a
specific name (and contents) is present.
https://github.com/restic/restic/issues/317
https://github.com/restic/restic/pull/1170
https://github.com/restic/restic/pull/1224

View File

@@ -0,0 +1,4 @@
Enhancement: Automatically generate man pages for all restic commands
https://github.com/restic/restic/issues/697
https://github.com/restic/restic/pull/1147

View File

@@ -0,0 +1,3 @@
Enhancement: Improve `restore`, do not traverse/load excluded directories
https://github.com/restic/restic/pull/1044

View File

@@ -0,0 +1,3 @@
Enhancement: Add Dockerfile and official Docker image
https://github.com/restic/restic/pull/1061

View File

@@ -0,0 +1,7 @@
Enhancement: Use the standard Go git repository layout, use `dep` for vendoring
The git repository layout was changed to resemble the layout typically used in
Go projects, we're not using `gb` for building restic any more and vendoring
the dependencies is now taken care of by `dep`.
https://github.com/restic/restic/pull/1126

View File

@@ -0,0 +1,5 @@
Enhancement: Add support for storing backups on Google Cloud Storage
https://github.com/restic/restic/pull/1134
https://github.com/restic/restic/pull/1052
https://github.com/restic/restic/issues/211

View File

@@ -0,0 +1,3 @@
Enhancement: Properly report errors when reading files with exclude patterns
https://github.com/restic/restic/pull/1144

View File

@@ -0,0 +1,8 @@
Enhancement: Add support for storing backups on Microsoft Azure Blob Storage
The library we're using to access the service requires Go 1.8, so restic now
needs at least Go 1.8.
https://github.com/restic/restic/pull/1149
https://github.com/restic/restic/pull/1059
https://github.com/restic/restic/issues/609

View File

@@ -0,0 +1,3 @@
Bugfix: Make the `key remove` command behave as documented
https://github.com/restic/restic/pull/1164

View File

@@ -0,0 +1,8 @@
Bugfix: Make sure to write profiling files on interrupt
Since a few releases restic had the ability to write profiling files for memory
and CPU usage when `debug` is enabled. It was discovered that when restic is
interrupted (ctrl+c is pressed), the proper shutdown hook is not run. This is
now corrected.
https://github.com/restic/restic/pull/1191

View File

@@ -0,0 +1,3 @@
Enhancement: Add `--group-by` to `forget` command for flexible grouping
https://github.com/restic/restic/pull/1196

View File

@@ -0,0 +1,5 @@
Enhancement: Print stats on all BSD systems when SIGINFO (ctrl+t) is received
https://github.com/restic/restic/pull/1203
https://github.com/restic/restic/pull/1082#issuecomment-326279920

View File

@@ -0,0 +1,3 @@
Enhancement: Allow specifying time/date for a backup with `--time`
https://github.com/restic/restic/pull/1205

View File

@@ -0,0 +1,9 @@
Bugfix: List all files stored in Google Cloud Storage
For large backups stored in Google Cloud Storage, the `prune` command fails
because listing only returns the first 1000 files. This has been corrected, no
data is lost in the process. In addition, a plausibility check was added to
`prune`.
https://github.com/restic/restic/issues/1246
https://github.com/restic/restic/pull/1247

View File

@@ -0,0 +1,9 @@
Enhancement: Add subdirectory `ids` to fuse mount
The fuse mount now has an `ids` subdirectory which contains the snapshots below
their (short) IDs.
https://github.com/restic/restic/issues/1102
https://github.com/restic/restic/pull/1299
https://github.com/restic/restic/pull/1320

View File

@@ -0,0 +1,10 @@
Enhancement: Add `--cacert` to specify TLS certificates to check against
We've added the `--cacert` option which can be used to pass one (or more) CA
certificates to restic. These are used in addition to the system CA
certificates to verify HTTPS certificates (e.g. for the REST backend).
https://github.com/restic/restic/issues/1114
https://github.com/restic/restic/pull/1276

View File

@@ -0,0 +1,9 @@
Enhancement: Add upload/download limiting
We've added support for rate limiting through `--limit-upload` and
`--limit-download` flags.
https://github.com/restic/restic/issues/1216
https://github.com/restic/restic/pull/1336
https://github.com/restic/restic/pull/1358

View File

@@ -0,0 +1,7 @@
Bugfix: Re-enable workaround for S3 backend
We've re-enabled a workaround for `minio-go` (the library we're using to
access s3 backends), this reduces memory usage.
https://github.com/restic/restic/issues/1256
https://github.com/restic/restic/pull/1267

View File

@@ -0,0 +1,6 @@
Enhancement: Cache results for excludes for `backup`
The `backup` command now caches the result of excludes for a directory.
https://github.com/restic/restic/issues/1271
https://github.com/restic/restic/pull/1326

View File

@@ -0,0 +1,9 @@
Enhancement: Add `generate` command, replaces `manpage` and `autocomplete`
The `generate` command has been added, which replaces the now removed
commands `manpage` and `autocomplete`. This release of restic contains the
most recent manpages in `doc/man` and the auto-completion files for bash and
zsh in `doc/bash-completion.sh` and `doc/zsh-completion.zsh`
https://github.com/restic/restic/issues/1274
https://github.com/restic/restic/pull/1282

View File

@@ -0,0 +1,8 @@
Bugfix: Reuse backend TCP connections to BackBlaze B2
A bug was discovered in the library we're using to access Backblaze, it now
reuses already established TCP connections which should be a lot faster and
not cause network failures any more.
https://github.com/restic/restic/issues/1291
https://github.com/restic/restic/pull/1301

View File

@@ -0,0 +1,7 @@
Enhancement: Allow comments in files read from via `--file-from`
When the list of files/dirs to be saved is read from a file with
`--files-from`, comment lines (starting with `#`) are now ignored.
https://github.com/restic/restic/issues/1367
https://github.com/restic/restic/pull/1368

View File

@@ -0,0 +1,18 @@
Security: Prevent writing outside the target directory during restore
A vulnerability was found in the restic restorer, which allowed attackers in
special circumstances to restore files to a location outside of the target
directory. Due to the circumstances we estimate this to be a low-risk
vulnerability, but urge all users to upgrade to the latest version of restic.
Exploiting the vulnerability requires a Linux/Unix system which saves backups
via restic and a Windows systems which restores files from the repo. In
addition, the attackers need to be able to create create files with arbitrary
names which are then saved to the restic repo. For example, by creating a file
named "..\test.txt" (which is a perfectly legal filename on Linux) and
restoring a snapshot containing this file on Windows, it would be written to
the parent of the target directory.
We'd like to thank Tyler Spivey for reporting this responsibly!
https://github.com/restic/restic/pull/1445

View File

@@ -0,0 +1,9 @@
Enhancement: sftp backend prompts for password
The sftp backend now prompts for the password if a password is necessary for
login.
https://github.com/restic/restic/issues/448
https://github.com/restic/restic/pull/1270

View File

@@ -0,0 +1,7 @@
Enhancement: Add `dump` command
We've added the `dump` command which prints a file from a snapshot to
stdout. This can e.g. be used to restore files read with `backup --stdin`.
https://github.com/restic/restic/issues/510
https://github.com/restic/restic/pull/1346

View File

@@ -0,0 +1,23 @@
Enhancement: Add local metadata cache
We've added a local cache for metadata so that restic doesn't need to load
all metadata (snapshots, indexes, ...) from the repo each time it starts. By
default the cache is active, but there's a new global option `--no-cache`
that can be used to disable the cache. By deafult, the cache a standard
cache folder for the OS, which can be overridden with `--cache-dir`. The
cache will automatically populate, indexes and snapshots are saved as they
are loaded. Cache directories for repos that haven't been used recently can
automatically be removed by restic with the `--cleanup-cache` option.
A related change was to by default create pack files in the repo that contain
either data or metadata, not both mixed together. This allows easy caching of
only the metadata files. The next run of `restic prune` will untangle mixed
files automatically.
https://github.com/restic/restic/pull/1040
https://github.com/restic/restic/issues/29
https://github.com/restic/restic/issues/738
https://github.com/restic/restic/issues/282
https://github.com/restic/restic/pull/1287
https://github.com/restic/restic/pull/1436
https://github.com/restic/restic/pull/1265

View File

@@ -0,0 +1,6 @@
Enhancement: Add `latest` symlink in fuse mount
The directory structure in the fuse mount now exposes a symlink `latest`
which points to the latest snapshot in that particular directory.
https://github.com/restic/restic/pull/1249

View File

@@ -0,0 +1,6 @@
Enhancement: Add `--compact` to `forget` command
The option `--compact` was added to the `forget` command to provide the same
compact view as the `snapshots` command.
https://github.com/restic/restic/pull/1269

View File

@@ -0,0 +1,7 @@
Enhancement: Google Cloud Storage backend needs less permissions
The Google Cloud Storage backend no longer requires the service account to
have the `storage.buckets.get` permission ("Storage Admin" role) in `restic
init` if the bucket already exists.
https://github.com/restic/restic/pull/1281

View File

@@ -0,0 +1,7 @@
Bugfix: Run prune when `forget --prune` is called with just snapshot IDs
A bug in the `forget` command caused `prune` not to be run when `--prune` was
specified without a policy, e.g. when only snapshot IDs that should be
forgotten are listed manually.
https://github.com/restic/restic/pull/1317

View File

@@ -0,0 +1,8 @@
Enhancement: Make `check` print `no errors found` explicitly
The `check` command now explicetly prints `No errors were found` when no errors
could be found.
https://github.com/restic/restic/pull/1319
https://github.com/restic/restic/issues/1303

View File

@@ -0,0 +1,3 @@
Enhancement: Retry failed backend requests
https://github.com/restic/restic/pull/1353

View File

@@ -0,0 +1,10 @@
Bugfix: Remove implicit path `/restic` for the s3 backend
The s3 backend used the subdir `restic` within a bucket if no explicit path
after the bucket name was specified. Since this version, restic does not use
this default path any more. If you created a repo on s3 in a bucket without
specifying a path within the bucket, you need to add `/restic` at the end of
the repository specification to access your repo: `s3:s3.amazonaws.com/bucket/restic`
https://github.com/restic/restic/pull/1437
https://github.com/restic/restic/issues/1292

View File

@@ -0,0 +1,4 @@
Bugfix: Improve s3 backend with DigitalOcean Spaces
https://github.com/restic/restic/issues/1457
https://github.com/restic/restic/pull/1459

View File

@@ -0,0 +1,9 @@
Enhancement: Add code to detect old cache directories
We've added code to detect old cache directories of repositories that
haven't been used in a long time, restic now prints a note when it detects
that such dirs exist. Also, the option `--cleanup-cache` was added to
automatically remove such directories. That's not a problem because the
cache will be rebuild once a repo is accessed again.
https://github.com/restic/restic/pull/1436

View File

@@ -0,0 +1,6 @@
Enhancement: Improve cancellation logic
The cancellation logic was improved, restic can now shut down cleanly when
requested to do so (e.g. via ctrl+c).
https://github.com/restic/restic/pull/1439

View File

@@ -0,0 +1,9 @@
Change: Do not save atime by default
By default, the access time for files and dirs is not saved any more. It is
not possible to reliably disable updating the access time during a backup,
so for the next backup the access time is different again. This means a lot
of metadata is saved. If you want to save the access time anyway, pass
`--with-atime` to the `backup` command.
https://github.com/restic/restic/pull/1452

View File

@@ -0,0 +1,6 @@
Bugfix: Correct cache dir location for Windows and Darwin
The cache directory on Windows and Darwin was not correct, instead the
directory `.cache` was used.
https://github.com/restic/restic/pull/1454

View File

@@ -0,0 +1,9 @@
Bugfix: Disable handling SIGPIPE
We've disabled handling SIGPIPE again. Turns out, writing to broken TCP
connections also raised SIGPIPE, so restic exits on the first write to a
broken connection. Instead, restic should retry the request.
https://github.com/restic/restic/pull/1459
https://github.com/restic/restic/issues/1457
https://github.com/restic/restic/issues/1466

View File

@@ -0,0 +1,8 @@
Enhancement: Add the `diff` command
The command `diff` was added, it allows comparing two snapshots and listing
all differences.
https://github.com/restic/restic/issues/11
https://github.com/restic/restic/issues/1460
https://github.com/restic/restic/pull/1462

View File

@@ -0,0 +1,4 @@
Bugfix: Limit bandwith at the http.RoundTripper for HTTP based backends
https://github.com/restic/restic/issues/1506
https://github.com/restic/restic/pull/1511

View File

@@ -0,0 +1,9 @@
Bugfix: Restore directory permissions as the last step
This change allows restoring into directories that were not writable during
backup. Before, restic created the directory, set the read-only mode and then
failed to create files in the directory. This change now restores the directory
(with its permissions) as the very last step.
https://github.com/restic/restic/issues/1512
https://github.com/restic/restic/pull/1536

View File

@@ -0,0 +1,8 @@
Enhancement: Add support for TLS client certificate authentication
Support has been added for using a TLS client certificate for authentication to
HTTP based backend. A file containing the PEM encoded private key and
certificate can be set using the `--tls-client-cert` option.
https://github.com/restic/restic/issues/1522
https://github.com/restic/restic/pull/1524

View File

@@ -0,0 +1,4 @@
Bugfix: Correctly create missing subdirs in data/
https://github.com/restic/restic/issues/1528
https://github.com/restic/restic/pull/1529

View File

@@ -0,0 +1,7 @@
Enhancement: Reduce number of remote requests during repository check
This change eliminates redundant remote repository calls and significantly
improves repository check time.
https://github.com/restic/restic/issues/1541
https://github.com/restic/restic/pull/1548

View File

@@ -0,0 +1,9 @@
Enhancement: Reduce number of backend requests for rebuild-index and prune
We've found a way to reduce then number of backend requests for the
`rebuild-index` and `prune` operations. This significantly speeds up the
operations for high-latency backends.
https://github.com/restic/restic/issues/1567
https://github.com/restic/restic/pull/1574
https://github.com/restic/restic/pull/1575

View File

@@ -0,0 +1,7 @@
Bugfix: Strip spaces for lines read via --files-from
Leading and trailing spaces in lines read via `--files-from` are now stripped,
so it behaves the same as with lines read via `--exclude-file`.
https://github.com/restic/restic/issues/1590
https://github.com/restic/restic/pull/1613

View File

@@ -0,0 +1,3 @@
Enhancement: Only reload snapshots once per minute for fuse mount
https://github.com/restic/restic/pull/1507

View File

@@ -0,0 +1,7 @@
Enhancement: Reduce memory allocations for querying the index
This change reduces the internal memory allocations when the index data
structures in memory are queried if a blob (part of a file) already exists in
the repo. It should speed up backup a bit, and maybe even reduce RAM usage.
https://github.com/restic/restic/pull/1538

View File

@@ -0,0 +1,7 @@
Enhancement: Speed up querying across indices and scanning existing files
This change increases the whenever a blob (part of a file) is searched for in a
restic repository. This will reduce cpu usage some when backing up files already
backed up by restic. Cpu usage is further decreased when scanning files.
https://github.com/restic/restic/pull/1549

View File

@@ -0,0 +1,7 @@
Enhancement: fuse/mount: Correctly handle EOF, add template option
We've added the `--snapshot-template` string, which can be used to specify a
template for a snapshot directory. In addition, accessing data after the end of
a file via the fuse mount is now handled correctly.
https://github.com/restic/restic/pull/1554

View File

@@ -0,0 +1,10 @@
Enhancement: Don't terminate ssh on SIGINT
We've reworked the code which runs the `ssh` login for the sftp backend so that
it can prompt for a password (if needed) but does not exit when the user
presses CTRL+C (SIGINT) e.g. during backup. This allows restic to properly shut
down when it receives SIGINT and remove the lock file from the repo, afterwards
exiting the `ssh` process.
https://github.com/restic/restic/pull/1564
https://github.com/restic/restic/pull/1588

View File

@@ -0,0 +1,3 @@
Enhancement: Retry Backend.List() in case of errors
https://github.com/restic/restic/pull/1579

View File

@@ -0,0 +1,12 @@
Enhancement: Limit index file size
Before, restic would create a single new index file on `prune` or
`rebuild-index`, this may lead to memory problems when this huge index is
created and loaded again. We're now limiting the size of the index file, and
split newly created index files into several smaller ones. This allows restic
to be more memory-efficient.
https://github.com/restic/restic/pull/1584
https://github.com/restic/restic/issues/1412
https://github.com/restic/restic/issues/979
https://github.com/restic/restic/issues/526

Some files were not shown because too many files have changed in this diff Show More