--exclude-if-present .nobackup option got lost when upgrading

[ThomasWaldmann]

Description

Before upgrade:

2023-11-30 02:14:54,486 - vorta.borg.borg_job - INFO - Running command /.../bin/borg create --list --progress --info --log-json --json --filter=AM -C lz4 --exclude-from /var/folders/.../T/tmpd_okqjz0 --exclude-if-present .nobackup borg@server:repo::archive /Users/myuser

After upgrade:

2023-11-30 16:29:40,929 - vorta.borg.borg_job - INFO - Running command /.../bin/borg create --list --progress --info --log-json --json --filter=AM -C lz4 --exclude-from /var/folders/.../T/tmpdxecfkpe borg@server:repo::archive /Users/myuser

--exclude-if-present .nobackup just got lost (I did not remove it manually) and I noticed it because that backup contained files that were previously excluded by that.

Environment

• OS: macOS Ventura
• Vorta version: 0.9.1beta3 (after upgrade - iirc it was beta2 before)
• Installed from: .dmg binary
• Borg version: 1.2.6

[ThomasWaldmann]

Can be fixed manually by adding --exclude-if-present .nobackup in Schedule -> Shell commands -> Extra arguments for borg create.

[ThomasWaldmann]

@diivi is this related to / caused by your exclude gui changes?

[diivi]

Yeah, I think the exclude-if-present option was commented out with this PR - https://github.com/borgbase/vorta/pull/1846/files#diff-2559f308260b3597027f7b0650bc98f2783fbb1ed513a244a3ccb4756256d11cL180

[m3nu]

Yes, we didn't have time to do this properly and my theory is that the feature wasn't used widely. So I'd like to see if it was used much and then maybe redo it with a new dialog.

[m3nu]

Put this in the changelog to avoid confusion.

https://github.com/borgbase/vorta/releases/tag/v0.9.1-beta3

[StorageB]

This appears to be an issue in the official 0.9.1 release.

I noticed Vorta running much longer than normal this morning and saw that it was backing up files that were previously excluded, and the .nobackup exclusion was no longer there.

[m3nu]

This option isn't implemented in the new exclusion dialog yet. See the release logs for breaking changes and a simple workaround:

https://github.com/borgbase/vorta/releases/tag/v0.9.1-beta3

[dietmarw]

Just my vote to bring this back since it just almost borked ;-) my backups and I heavily rely on the possibility of specifying a .nobackup option. It is one of many features that make it easy to fine-tune a backup without going through Vorta's settings. So please bring this back again. I have the workaround as pointed out in the release notes (now that I found them) in place though.

[dietmarw]

Maybe I can elaborate on how this missing feature from the new version can "bork" a backup in order to create some understanding of the severity of dropping an existing feature in a 0.x release. One of my backup targets is OneDrive which gets mounted under ~/OneDrive. Now my backup is configured to backup ~/ and in order to not backup the backup target for OneDrive itself I have placed ~/OneDrive/.nobackup file. After the update to the 0.9.x version of Vorta it now created an infinite backup loop where the target gets backed up to itself. So it is not just a minor inconvenience but something that caused a major disruption in my case.

[real-yfprojects]

We need an idea (mockup) how reincorporate this feature into the GUI before we can start implementing anything.

[StorageB]

My thought is to add this to the Manage Excluded Items popup window, and rename that window to "Manage Excluded Items" instead of "Add patterns to exclude." A new tab could be added for this, and the Preview tab could also report what exclude-if-present files are included.

By default, maybe the .nobackup item is listed under the new Folders tab, but not checked. If it is not desirable from a programming standpoint to have multiple exclude-if-present files, then instead of a list under the Folders tab, perhaps it could just be a single checkbox option "Exclude folder from the backup if a file named .nobackup is present in that folder" (or similar wording to what was previously used).

[NovaViper]

+1 for adding this feature back. I was just about to do a backup and suddenly a bunch of folders I specifically didn't want to get backed up (some extremely large Github repos I cloned for testing), almost took my raspberry pi server down because it was so large 😓

[shivansh02]

Working on bringing back the feature. Came up with a draft UI as per suggestions from @m3nu. It ended up looking exactly like the one suggested by @StorageB above. Would love to hear everyone's inputs on this.

[StorageB]

This is great!! I appreciate the work on this.

My thoughts on the UI:

• The Preview tab should be last (and it should show the active exclude if present items). I think a more intuitive tab order would be: Presets, Custom, Raw, Exclude If Present, Preview
• The wording "Directories that contain the following filesystem objects...." is similar to how Borg describes this function in their documentation, but I don't think it's intuitive for a new user. Something like "Folders that contain the following files will be excluded from backups" may be easier to understand? Version 0.8.11 before removing the feature simply said "Exclude If Present (exclude folders with these files)"
• Maybe capitalize all the words in the tab name Exclude If Present for a cleaner look

[shivansh02]

Resolved in PR #2016! Thank you for your patience and feedback. If you encounter issues with this, please open a new issue.

[dmlary]

Note, this is a major bug, not minor. It caused all of my backups to exceed their quotas when the vorta auto-upgrader suggested applying the update.

You need a very, very, very large warning in the update dialog if exclusion files are configured because the failure case can be gigs of incomplete backups.