From 851e85b42671d08b367cdcbf5ce44a114cc76367 Mon Sep 17 00:00:00 2001 From: Thomas Waldmann Date: Sun, 5 Nov 2017 23:09:14 +0100 Subject: [PATCH 1/3] build_usage --- docs/usage/create.rst.inc | 8 +++-- docs/usage/delete.rst.inc | 7 +++- docs/usage/diff.rst.inc | 71 ++++++++++++++++--------------------- docs/usage/help.rst.inc | 6 ++-- docs/usage/list.rst.inc | 9 ----- docs/usage/prune.rst.inc | 7 +++- docs/usage/recreate.rst.inc | 13 ++++++- docs/usage/upgrade.rst.inc | 14 ++++---- 8 files changed, 71 insertions(+), 64 deletions(-) diff --git a/docs/usage/create.rst.inc b/docs/usage/create.rst.inc index 1ef6c23b5b..1b6d22167d 100644 --- a/docs/usage/create.rst.inc +++ b/docs/usage/create.rst.inc @@ -212,6 +212,11 @@ The ``--progress`` option shows (from left to right) Original, Compressed and De (O, C and D, respectively), then the Number of files (N) processed so far, followed by the currently processed path. +When using ``--stats``, you will get some statistics about how much data was +added - the "This Archive" deduplicated size there is most interesting as that is +how much your repository will grow. +Please note that the "All archives" stats refer to the state after creation. + See the output of the "borg help patterns" command for more help on exclude patterns. See the output of the "borg help placeholders" command for more help on placeholders. @@ -231,9 +236,6 @@ only include the objects specified by ``--exclude-if-present`` in your backup, and not include any other contents of the containing folder, this can be enabled through using the ``--keep-exclude-tags`` option. -Borg respects the nodump flag. Files flagged nodump will be marked as excluded (x) -in ``--list`` output. - Item flags ++++++++++ diff --git a/docs/usage/delete.rst.inc b/docs/usage/delete.rst.inc index 93a3ec3cb8..5951c71a27 100644 --- a/docs/usage/delete.rst.inc +++ b/docs/usage/delete.rst.inc @@ -85,4 +85,9 @@ Description This command deletes an archive from the repository or the complete repository. Disk space is reclaimed accordingly. If you delete the complete repository, the -local cache for it (if any) is also deleted. \ No newline at end of file +local cache for it (if any) is also deleted. + +When using ``--stats``, you will get some statistics about how much data was +deleted - the "Deleted data" deduplicated size there is most interesting as +that is how much your repository will shrink. +Please note that the "All archives" stats refer to the state after deletion. \ No newline at end of file diff --git a/docs/usage/diff.rst.inc b/docs/usage/diff.rst.inc index ef5b831a60..5074a09939 100644 --- a/docs/usage/diff.rst.inc +++ b/docs/usage/diff.rst.inc @@ -12,43 +12,37 @@ borg diff .. class:: borg-options-table - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | **positional arguments** | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``REPO_ARCHIVE1`` | repository location and ARCHIVE1 name | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``ARCHIVE2`` | ARCHIVE2 name (no repository location allowed) | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``PATH`` | paths of items inside the archives to compare; patterns are supported | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | **optional arguments** | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--numeric-owner`` | only consider numeric user and group identifiers | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--same-chunker-params`` | Override check of chunker parameters. | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--sort`` | Sort the output lines by file path. | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | .. class:: borg-common-opt-ref | - | | - | :ref:`common_options` | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | **Exclusion options** | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--pattern PATTERN`` | experimental: include/exclude paths matching PATTERN | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--exclude-caches`` | exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html) | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--exclude-if-present NAME`` | exclude directories that are tagged by containing a filesystem object with the given NAME | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--keep-exclude-tags``, ``--keep-tag-files`` | if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | **positional arguments** | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``REPO_ARCHIVE1`` | repository location and ARCHIVE1 name | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``ARCHIVE2`` | ARCHIVE2 name (no repository location allowed) | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``PATH`` | paths of items inside the archives to compare; patterns are supported | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | **optional arguments** | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--numeric-owner`` | only consider numeric user and group identifiers | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--same-chunker-params`` | Override check of chunker parameters. | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--sort`` | Sort the output lines by file path. | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | .. class:: borg-common-opt-ref | + | | + | :ref:`common_options` | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | **Exclusion options** | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--pattern PATTERN`` | experimental: include/exclude paths matching PATTERN | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ .. raw:: html @@ -82,9 +76,6 @@ borg diff --exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line --pattern PATTERN experimental: include/exclude paths matching PATTERN --patterns-from PATTERNFILE experimental: read include/exclude patterns from PATTERNFILE, one per line - --exclude-caches exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html) - --exclude-if-present NAME exclude directories that are tagged by containing a filesystem object with the given NAME - --keep-exclude-tags, --keep-tag-files if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive Description diff --git a/docs/usage/help.rst.inc b/docs/usage/help.rst.inc index 152e01b540..c5fb6a71e5 100644 --- a/docs/usage/help.rst.inc +++ b/docs/usage/help.rst.inc @@ -124,10 +124,12 @@ Examples:: may specify the backup roots (starting points) and patterns for inclusion/exclusion. A root path starts with the prefix `R`, followed by a path (a plain path, not a file pattern). An include rule starts with the prefix +, an exclude rule starts - with the prefix -, both followed by a pattern. + with the prefix -, an exclude-norecurse rule starts with !, all followed by a pattern. Inclusion patterns are useful to include paths that are contained in an excluded path. The first matching pattern is used so if an include pattern matches before - an exclude pattern, the file is backed up. + an exclude pattern, the file is backed up. If an exclude-norecurse pattern matches + a directory, it won't recurse into it and won't discover any potential matches for + include rules below that directory. Note that the default pattern style for ``--pattern`` and ``--patterns-from`` is shell style (`sh:`), so those patterns behave similar to rsync include/exclude diff --git a/docs/usage/list.rst.inc b/docs/usage/list.rst.inc index e6b8988840..6cf097665c 100644 --- a/docs/usage/list.rst.inc +++ b/docs/usage/list.rst.inc @@ -55,12 +55,6 @@ borg list +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--exclude-caches`` | exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html) | - +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--exclude-if-present NAME`` | exclude directories that are tagged by containing a filesystem object with the given NAME | - +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--keep-exclude-tags``, ``--keep-tag-files`` | if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive | - +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ .. raw:: html @@ -101,9 +95,6 @@ borg list --exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line --pattern PATTERN experimental: include/exclude paths matching PATTERN --patterns-from PATTERNFILE experimental: read include/exclude patterns from PATTERNFILE, one per line - --exclude-caches exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html) - --exclude-if-present NAME exclude directories that are tagged by containing a filesystem object with the given NAME - --keep-exclude-tags, --keep-tag-files if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive Description diff --git a/docs/usage/prune.rst.inc b/docs/usage/prune.rst.inc index f98fbab847..81c2d32f94 100644 --- a/docs/usage/prune.rst.inc +++ b/docs/usage/prune.rst.inc @@ -134,4 +134,9 @@ negative number of archives to keep means that there is no limit. The ``--keep-last N`` option is doing the same as ``--keep-secondly N`` (and it will keep the last N archives under the assumption that you do not create more than one -backup archive in the same second). \ No newline at end of file +backup archive in the same second). + +When using ``--stats``, you will get some statistics about how much data was +deleted - the "Deleted data" deduplicated size there is most interesting as +that is how much your repository will shrink. +Please note that the "All archives" stats refer to the state after pruning. \ No newline at end of file diff --git a/docs/usage/recreate.rst.inc b/docs/usage/recreate.rst.inc index ef258e6941..b3071ab3da 100644 --- a/docs/usage/recreate.rst.inc +++ b/docs/usage/recreate.rst.inc @@ -150,4 +150,15 @@ With ``--target`` the original archive is not replaced, instead a new archive is When rechunking space usage can be substantial, expect at least the entire deduplicated size of the archives using the previous chunker params. When recompressing expect approx. (throughput / checkpoint-interval) in space usage, -assuming all chunks are recompressed. \ No newline at end of file +assuming all chunks are recompressed. + +If you recently ran borg check --repair and it had to fix lost chunks with all-zero +replacement chunks, please first run another backup for the same data and re-run +borg check --repair afterwards to heal any archives that had lost chunks which are +still generated from the input data. + +Important: running borg recreate to re-chunk will remove the chunks_healthy +metadata of all items with replacement chunks, so healing will not be possible +any more after re-chunking (it is also unlikely it would ever work: due to the +change of chunking parameters, the missing chunk likely will never be seen again +even if you still have the data that produced it). \ No newline at end of file diff --git a/docs/usage/upgrade.rst.inc b/docs/usage/upgrade.rst.inc index eafa4362ae..47a240dd06 100644 --- a/docs/usage/upgrade.rst.inc +++ b/docs/usage/upgrade.rst.inc @@ -130,13 +130,13 @@ make sure the cache files are also removed: borg delete borg -Unless ``--inplace`` is specified, the upgrade process first -creates a backup copy of the repository, in -REPOSITORY.before-upgrade-DATETIME, using hardlinks. This takes -longer than in place upgrades, but is much safer and gives -progress information (as opposed to ``cp -al``). Once you are -satisfied with the conversion, you can safely destroy the -backup copy. +Unless ``--inplace`` is specified, the upgrade process first creates a backup +copy of the repository, in REPOSITORY.before-upgrade-DATETIME, using hardlinks. +This requires that the repository and its parent directory reside on same +filesystem so the hardlink copy can work. +This takes longer than in place upgrades, but is much safer and gives +progress information (as opposed to ``cp -al``). Once you are satisfied +with the conversion, you can safely destroy the backup copy. WARNING: Running the upgrade in place will make the current copy unusable with older version, with no way of going back From 3097c059ffbe8cffcad4b3b2ffdd56b27de321b0 Mon Sep 17 00:00:00 2001 From: Thomas Waldmann Date: Sun, 5 Nov 2017 23:09:39 +0100 Subject: [PATCH 2/3] build_man --- docs/man/borg-benchmark-crud.1 | 2 +- docs/man/borg-benchmark.1 | 2 +- docs/man/borg-break-lock.1 | 2 +- docs/man/borg-change-passphrase.1 | 2 +- docs/man/borg-check.1 | 2 +- docs/man/borg-common.1 | 2 +- docs/man/borg-compression.1 | 2 +- docs/man/borg-create.1 | 10 ++++++---- docs/man/borg-delete.1 | 7 ++++++- docs/man/borg-diff.1 | 11 +---------- docs/man/borg-export-tar.1 | 2 +- docs/man/borg-extract.1 | 2 +- docs/man/borg-info.1 | 2 +- docs/man/borg-init.1 | 2 +- docs/man/borg-key-change-passphrase.1 | 2 +- docs/man/borg-key-export.1 | 2 +- docs/man/borg-key-import.1 | 2 +- docs/man/borg-key-migrate-to-repokey.1 | 2 +- docs/man/borg-key.1 | 2 +- docs/man/borg-list.1 | 11 +---------- docs/man/borg-mount.1 | 2 +- docs/man/borg-patterns.1 | 8 +++++--- docs/man/borg-placeholders.1 | 2 +- docs/man/borg-prune.1 | 9 +++++++-- docs/man/borg-recreate.1 | 13 ++++++++++++- docs/man/borg-rename.1 | 2 +- docs/man/borg-serve.1 | 2 +- docs/man/borg-umount.1 | 2 +- docs/man/borg-upgrade.1 | 16 ++++++++-------- docs/man/borg-with-lock.1 | 2 +- docs/man/borg.1 | 3 +++ 31 files changed, 71 insertions(+), 61 deletions(-) diff --git a/docs/man/borg-benchmark-crud.1 b/docs/man/borg-benchmark-crud.1 index c9f1334229..d5fc412f44 100644 --- a/docs/man/borg-benchmark-crud.1 +++ b/docs/man/borg-benchmark-crud.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-BENCHMARK-CRUD 1 "2017-10-22" "" "borg backup tool" +.TH BORG-BENCHMARK-CRUD 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-benchmark-crud \- Benchmark Create, Read, Update, Delete for archives. . diff --git a/docs/man/borg-benchmark.1 b/docs/man/borg-benchmark.1 index 1feed69bed..7e0e1a192d 100644 --- a/docs/man/borg-benchmark.1 +++ b/docs/man/borg-benchmark.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-BENCHMARK 1 "2017-10-22" "" "borg backup tool" +.TH BORG-BENCHMARK 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-benchmark \- benchmark command . diff --git a/docs/man/borg-break-lock.1 b/docs/man/borg-break-lock.1 index 8f85ade5d5..7bf3d1d7a4 100644 --- a/docs/man/borg-break-lock.1 +++ b/docs/man/borg-break-lock.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-BREAK-LOCK 1 "2017-10-22" "" "borg backup tool" +.TH BORG-BREAK-LOCK 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-break-lock \- Break the repository lock (e.g. in case it was left by a dead borg. . diff --git a/docs/man/borg-change-passphrase.1 b/docs/man/borg-change-passphrase.1 index a377dfaba7..5089fdb61a 100644 --- a/docs/man/borg-change-passphrase.1 +++ b/docs/man/borg-change-passphrase.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-CHANGE-PASSPHRASE 1 "2017-10-22" "" "borg backup tool" +.TH BORG-CHANGE-PASSPHRASE 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-change-passphrase \- Change repository key file passphrase . diff --git a/docs/man/borg-check.1 b/docs/man/borg-check.1 index e0eadf5ab6..50282c8e9f 100644 --- a/docs/man/borg-check.1 +++ b/docs/man/borg-check.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-CHECK 1 "2017-10-22" "" "borg backup tool" +.TH BORG-CHECK 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-check \- Check repository consistency . diff --git a/docs/man/borg-common.1 b/docs/man/borg-common.1 index 91e0f7ab71..9ccb7d96f1 100644 --- a/docs/man/borg-common.1 +++ b/docs/man/borg-common.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-COMMON 1 "2017-10-22" "" "borg backup tool" +.TH BORG-COMMON 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-common \- Common options of Borg commands . diff --git a/docs/man/borg-compression.1 b/docs/man/borg-compression.1 index c4a444a3a1..0a504ea5d7 100644 --- a/docs/man/borg-compression.1 +++ b/docs/man/borg-compression.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-COMPRESSION 1 "2017-10-22" "" "borg backup tool" +.TH BORG-COMPRESSION 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-compression \- Details regarding compression . diff --git a/docs/man/borg-create.1 b/docs/man/borg-create.1 index 0c47dc6c99..2405c1cc56 100644 --- a/docs/man/borg-create.1 +++ b/docs/man/borg-create.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-CREATE 1 "2017-10-22" "" "borg backup tool" +.TH BORG-CREATE 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-create \- Create new archive . @@ -108,6 +108,11 @@ The \fB\-\-progress\fP option shows (from left to right) Original, Compressed an (O, C and D, respectively), then the Number of files (N) processed so far, followed by the currently processed path. .sp +When using \fB\-\-stats\fP, you will get some statistics about how much data was +added \- the "This Archive" deduplicated size there is most interesting as that is +how much your repository will grow. +Please note that the "All archives" stats refer to the state after creation. +.sp See the output of the "borg help patterns" command for more help on exclude patterns. See the output of the "borg help placeholders" command for more help on placeholders. .SH OPTIONS @@ -306,9 +311,6 @@ all of its contents will be omitted from the backup. If, however, you wish to only include the objects specified by \fB\-\-exclude\-if\-present\fP in your backup, and not include any other contents of the containing folder, this can be enabled through using the \fB\-\-keep\-exclude\-tags\fP option. -.sp -Borg respects the nodump flag. Files flagged nodump will be marked as excluded (x) -in \fB\-\-list\fP output. .SS Item flags .sp \fB\-\-list\fP outputs a list of all files, directories and other diff --git a/docs/man/borg-delete.1 b/docs/man/borg-delete.1 index 013ab288a5..f76ab2aa17 100644 --- a/docs/man/borg-delete.1 +++ b/docs/man/borg-delete.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-DELETE 1 "2017-10-22" "" "borg backup tool" +.TH BORG-DELETE 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-delete \- Delete an existing repository or archives . @@ -38,6 +38,11 @@ borg [common options] delete [options] [TARGET] [ARCHIVE...] This command deletes an archive from the repository or the complete repository. Disk space is reclaimed accordingly. If you delete the complete repository, the local cache for it (if any) is also deleted. +.sp +When using \fB\-\-stats\fP, you will get some statistics about how much data was +deleted \- the "Deleted data" deduplicated size there is most interesting as +that is how much your repository will shrink. +Please note that the "All archives" stats refer to the state after deletion. .SH OPTIONS .sp See \fIborg\-common(1)\fP for common options of Borg commands. diff --git a/docs/man/borg-diff.1 b/docs/man/borg-diff.1 index 9a96832443..cab36023a9 100644 --- a/docs/man/borg-diff.1 +++ b/docs/man/borg-diff.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-DIFF 1 "2017-10-22" "" "borg backup tool" +.TH BORG-DIFF 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-diff \- Diff contents of two archives . @@ -92,15 +92,6 @@ experimental: include/exclude paths matching PATTERN .TP .BI \-\-patterns\-from \ PATTERNFILE experimental: read include/exclude patterns from PATTERNFILE, one per line -.TP -.B \-\-exclude\-caches -exclude directories that contain a CACHEDIR.TAG file (\fI\%http://www.brynosaurus.com/cachedir/spec.html\fP) -.TP -.BI \-\-exclude\-if\-present \ NAME -exclude directories that are tagged by containing a filesystem object with the given NAME -.TP -.B \-\-keep\-exclude\-tags\fP,\fB \-\-keep\-tag\-files -if tag objects are specified with \fB\-\-exclude\-if\-present\fP, don\(aqt omit the tag objects themselves from the backup archive .UNINDENT .SH EXAMPLES .INDENT 0.0 diff --git a/docs/man/borg-export-tar.1 b/docs/man/borg-export-tar.1 index 3691b3c3b9..9d3c0f7d4e 100644 --- a/docs/man/borg-export-tar.1 +++ b/docs/man/borg-export-tar.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-EXPORT-TAR 1 "2017-10-22" "" "borg backup tool" +.TH BORG-EXPORT-TAR 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-export-tar \- Export archive contents as a tarball . diff --git a/docs/man/borg-extract.1 b/docs/man/borg-extract.1 index e17adddf5a..76b090dc70 100644 --- a/docs/man/borg-extract.1 +++ b/docs/man/borg-extract.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-EXTRACT 1 "2017-10-22" "" "borg backup tool" +.TH BORG-EXTRACT 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-extract \- Extract archive contents . diff --git a/docs/man/borg-info.1 b/docs/man/borg-info.1 index 0c9cce9566..ea075ff48a 100644 --- a/docs/man/borg-info.1 +++ b/docs/man/borg-info.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-INFO 1 "2017-10-22" "" "borg backup tool" +.TH BORG-INFO 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-info \- Show archive details such as disk space used . diff --git a/docs/man/borg-init.1 b/docs/man/borg-init.1 index ccf81dbfd7..0fb5417d91 100644 --- a/docs/man/borg-init.1 +++ b/docs/man/borg-init.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-INIT 1 "2017-10-22" "" "borg backup tool" +.TH BORG-INIT 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-init \- Initialize an empty repository . diff --git a/docs/man/borg-key-change-passphrase.1 b/docs/man/borg-key-change-passphrase.1 index f76707ddb0..3894588452 100644 --- a/docs/man/borg-key-change-passphrase.1 +++ b/docs/man/borg-key-change-passphrase.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-KEY-CHANGE-PASSPHRASE 1 "2017-10-22" "" "borg backup tool" +.TH BORG-KEY-CHANGE-PASSPHRASE 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-key-change-passphrase \- Change repository key file passphrase . diff --git a/docs/man/borg-key-export.1 b/docs/man/borg-key-export.1 index 6cea595f24..31f877966f 100644 --- a/docs/man/borg-key-export.1 +++ b/docs/man/borg-key-export.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-KEY-EXPORT 1 "2017-10-22" "" "borg backup tool" +.TH BORG-KEY-EXPORT 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-key-export \- Export the repository key for backup . diff --git a/docs/man/borg-key-import.1 b/docs/man/borg-key-import.1 index ee08b97954..20fa7ceb13 100644 --- a/docs/man/borg-key-import.1 +++ b/docs/man/borg-key-import.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-KEY-IMPORT 1 "2017-10-22" "" "borg backup tool" +.TH BORG-KEY-IMPORT 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-key-import \- Import the repository key from backup . diff --git a/docs/man/borg-key-migrate-to-repokey.1 b/docs/man/borg-key-migrate-to-repokey.1 index 154fcfe12d..7edaa023d7 100644 --- a/docs/man/borg-key-migrate-to-repokey.1 +++ b/docs/man/borg-key-migrate-to-repokey.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-KEY-MIGRATE-TO-REPOKEY 1 "2017-10-22" "" "borg backup tool" +.TH BORG-KEY-MIGRATE-TO-REPOKEY 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-key-migrate-to-repokey \- Migrate passphrase -> repokey . diff --git a/docs/man/borg-key.1 b/docs/man/borg-key.1 index 32b4d701d8..a449f50742 100644 --- a/docs/man/borg-key.1 +++ b/docs/man/borg-key.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-KEY 1 "2017-10-22" "" "borg backup tool" +.TH BORG-KEY 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-key \- Manage a keyfile or repokey of a repository . diff --git a/docs/man/borg-list.1 b/docs/man/borg-list.1 index 409a74b23f..eb8761b8ef 100644 --- a/docs/man/borg-list.1 +++ b/docs/man/borg-list.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-LIST 1 "2017-10-22" "" "borg backup tool" +.TH BORG-LIST 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-list \- List archive or repository contents . @@ -97,15 +97,6 @@ experimental: include/exclude paths matching PATTERN .TP .BI \-\-patterns\-from \ PATTERNFILE experimental: read include/exclude patterns from PATTERNFILE, one per line -.TP -.B \-\-exclude\-caches -exclude directories that contain a CACHEDIR.TAG file (\fI\%http://www.brynosaurus.com/cachedir/spec.html\fP) -.TP -.BI \-\-exclude\-if\-present \ NAME -exclude directories that are tagged by containing a filesystem object with the given NAME -.TP -.B \-\-keep\-exclude\-tags\fP,\fB \-\-keep\-tag\-files -if tag objects are specified with \fB\-\-exclude\-if\-present\fP, don\(aqt omit the tag objects themselves from the backup archive .UNINDENT .SH EXAMPLES .INDENT 0.0 diff --git a/docs/man/borg-mount.1 b/docs/man/borg-mount.1 index 1bc624f40d..97ab864b55 100644 --- a/docs/man/borg-mount.1 +++ b/docs/man/borg-mount.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-MOUNT 1 "2017-10-22" "" "borg backup tool" +.TH BORG-MOUNT 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-mount \- Mount archive or an entire repository as a FUSE filesystem . diff --git a/docs/man/borg-patterns.1 b/docs/man/borg-patterns.1 index 531ad5556c..0377cc6ea9 100644 --- a/docs/man/borg-patterns.1 +++ b/docs/man/borg-patterns.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-PATTERNS 1 "2017-10-22" "" "borg backup tool" +.TH BORG-PATTERNS 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-patterns \- Details regarding patterns . @@ -160,10 +160,12 @@ with the experimental \fB\-\-pattern\fP and \fB\-\-patterns\-from\fP options. Us may specify the backup roots (starting points) and patterns for inclusion/exclusion. A root path starts with the prefix \fIR\fP, followed by a path (a plain path, not a file pattern). An include rule starts with the prefix +, an exclude rule starts -with the prefix \-, both followed by a pattern. +with the prefix \-, an exclude\-norecurse rule starts with !, all followed by a pattern. Inclusion patterns are useful to include paths that are contained in an excluded path. The first matching pattern is used so if an include pattern matches before -an exclude pattern, the file is backed up. +an exclude pattern, the file is backed up. If an exclude\-norecurse pattern matches +a directory, it won\(aqt recurse into it and won\(aqt discover any potential matches for +include rules below that directory. .sp Note that the default pattern style for \fB\-\-pattern\fP and \fB\-\-patterns\-from\fP is shell style (\fIsh:\fP), so those patterns behave similar to rsync include/exclude diff --git a/docs/man/borg-placeholders.1 b/docs/man/borg-placeholders.1 index 31b5fb355d..e3491fd722 100644 --- a/docs/man/borg-placeholders.1 +++ b/docs/man/borg-placeholders.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-PLACEHOLDERS 1 "2017-10-22" "" "borg backup tool" +.TH BORG-PLACEHOLDERS 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-placeholders \- Details regarding placeholders . diff --git a/docs/man/borg-prune.1 b/docs/man/borg-prune.1 index e2ffdb2540..58e35ff169 100644 --- a/docs/man/borg-prune.1 +++ b/docs/man/borg-prune.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-PRUNE 1 "2017-10-22" "" "borg backup tool" +.TH BORG-PRUNE 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-prune \- Prune repository archives according to specified rules . @@ -73,6 +73,11 @@ negative number of archives to keep means that there is no limit. The \fB\-\-keep\-last N\fP option is doing the same as \fB\-\-keep\-secondly N\fP (and it will keep the last N archives under the assumption that you do not create more than one backup archive in the same second). +.sp +When using \fB\-\-stats\fP, you will get some statistics about how much data was +deleted \- the "Deleted data" deduplicated size there is most interesting as +that is how much your repository will shrink. +Please note that the "All archives" stats refer to the state after pruning. .SH OPTIONS .sp See \fIborg\-common(1)\fP for common options of Borg commands. @@ -171,7 +176,7 @@ $ borg prune \-v \-\-list \-\-keep\-within=10d \-\-keep\-weekly=4 \-\-keep\-mont .UNINDENT .sp There is also a visualized prune example in \fBdocs/misc/prune\-example.txt\fP: -.IP "System Message: ERROR/3 (docs/virtmanpage.rst:, line 140)" +.IP "System Message: ERROR/3 (docs/virtmanpage.rst:, line 145)" Unknown directive type "highlight". .INDENT 0.0 .INDENT 3.5 diff --git a/docs/man/borg-recreate.1 b/docs/man/borg-recreate.1 index 0768cd7485..4169e6f842 100644 --- a/docs/man/borg-recreate.1 +++ b/docs/man/borg-recreate.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-RECREATE 1 "2017-10-22" "" "borg backup tool" +.TH BORG-RECREATE 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-recreate \- Re-create archives . @@ -71,6 +71,17 @@ When rechunking space usage can be substantial, expect at least the entire deduplicated size of the archives using the previous chunker params. When recompressing expect approx. (throughput / checkpoint\-interval) in space usage, assuming all chunks are recompressed. +.sp +If you recently ran borg check \-\-repair and it had to fix lost chunks with all\-zero +replacement chunks, please first run another backup for the same data and re\-run +borg check \-\-repair afterwards to heal any archives that had lost chunks which are +still generated from the input data. +.sp +Important: running borg recreate to re\-chunk will remove the chunks_healthy +metadata of all items with replacement chunks, so healing will not be possible +any more after re\-chunking (it is also unlikely it would ever work: due to the +change of chunking parameters, the missing chunk likely will never be seen again +even if you still have the data that produced it). .SH OPTIONS .sp See \fIborg\-common(1)\fP for common options of Borg commands. diff --git a/docs/man/borg-rename.1 b/docs/man/borg-rename.1 index c628299f18..55fcd86c88 100644 --- a/docs/man/borg-rename.1 +++ b/docs/man/borg-rename.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-RENAME 1 "2017-10-22" "" "borg backup tool" +.TH BORG-RENAME 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-rename \- Rename an existing archive . diff --git a/docs/man/borg-serve.1 b/docs/man/borg-serve.1 index 3a567ee840..0aa10d97ea 100644 --- a/docs/man/borg-serve.1 +++ b/docs/man/borg-serve.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-SERVE 1 "2017-10-22" "" "borg backup tool" +.TH BORG-SERVE 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-serve \- Start in server mode. This command is usually not used manually. . diff --git a/docs/man/borg-umount.1 b/docs/man/borg-umount.1 index 932a407500..8bae2dff5e 100644 --- a/docs/man/borg-umount.1 +++ b/docs/man/borg-umount.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-UMOUNT 1 "2017-10-22" "" "borg backup tool" +.TH BORG-UMOUNT 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-umount \- un-mount the FUSE filesystem . diff --git a/docs/man/borg-upgrade.1 b/docs/man/borg-upgrade.1 index 0d5ef5905b..abe4dbf188 100644 --- a/docs/man/borg-upgrade.1 +++ b/docs/man/borg-upgrade.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-UPGRADE 1 "2017-10-22" "" "borg backup tool" +.TH BORG-UPGRADE 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-upgrade \- upgrade a repository from a previous version . @@ -104,13 +104,13 @@ borg delete borg .UNINDENT .UNINDENT .sp -Unless \fB\-\-inplace\fP is specified, the upgrade process first -creates a backup copy of the repository, in -REPOSITORY.before\-upgrade\-DATETIME, using hardlinks. This takes -longer than in place upgrades, but is much safer and gives -progress information (as opposed to \fBcp \-al\fP). Once you are -satisfied with the conversion, you can safely destroy the -backup copy. +Unless \fB\-\-inplace\fP is specified, the upgrade process first creates a backup +copy of the repository, in REPOSITORY.before\-upgrade\-DATETIME, using hardlinks. +This requires that the repository and its parent directory reside on same +filesystem so the hardlink copy can work. +This takes longer than in place upgrades, but is much safer and gives +progress information (as opposed to \fBcp \-al\fP). Once you are satisfied +with the conversion, you can safely destroy the backup copy. .sp WARNING: Running the upgrade in place will make the current copy unusable with older version, with no way of going back diff --git a/docs/man/borg-with-lock.1 b/docs/man/borg-with-lock.1 index 27321d56a8..9f7951f4bd 100644 --- a/docs/man/borg-with-lock.1 +++ b/docs/man/borg-with-lock.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH BORG-WITH-LOCK 1 "2017-10-22" "" "borg backup tool" +.TH BORG-WITH-LOCK 1 "2017-11-05" "" "borg backup tool" .SH NAME borg-with-lock \- run a user specified command with the repository lock held . diff --git a/docs/man/borg.1 b/docs/man/borg.1 index 9f3da1e24a..703bdc4b5d 100644 --- a/docs/man/borg.1 +++ b/docs/man/borg.1 @@ -432,6 +432,9 @@ security relevant data. .B BORG_CACHE_DIR Default to \(aq~/.cache/borg\(aq. This directory contains the local cache and might need a lot of space for dealing with big repositories). +.TP +.B BORG_CONFIG_DIR +Default to \(aq~/.config/borg\(aq. This directory contains the whole config directories. .UNINDENT .TP .B Building: From ce7936b608bcc8199d4a6cee7ef1e3c950ff4a03 Mon Sep 17 00:00:00 2001 From: Thomas Waldmann Date: Sun, 5 Nov 2017 23:52:15 +0100 Subject: [PATCH 3/3] vagrant: use https pypi url for wheezy http url stopped working --- Vagrantfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Vagrantfile b/Vagrantfile index 2eb9a462d8..10f100f628 100644 --- a/Vagrantfile +++ b/Vagrantfile @@ -31,7 +31,7 @@ def packages_debianoid(user) # this way it works on older dists (like ubuntu 12.04) also: # for python 3.2 on ubuntu 12.04 we need pip<8 and virtualenv<14 as # newer versions are not compatible with py 3.2 any more. - easy_install3 'pip<8.0' + easy_install3 -i https://pypi.python.org/simple/ 'pip<8.0' pip3 install 'virtualenv<14.0' EOF end