Several files are backed up with each snapshot #1437
Labels
Milestone
Comments
|
Moin @Elbchillharmonie , thanks for reporting. Can you please confirm that the backups are duplicated files and not hardlinks? You can use If it is a duplicate please compare their content via |
|
Moin @buhtz thanks for your response. I just checked some examples in the last two snapshots, they have different inodes: Only the link count seems to be different. Diff shows no difference at all. |
|
The issue might be a duplicate of #988 as you stated. Currently there is no easy or quick way to fix it because we still don't know why it happens. It is hard to reproduce and investigate. You still tried to use For our investigation can you give us some more information's about how the source and destination are linked together?
|
|
Oh sorry, it seems it didn't execute with
(Do you want to copy your SSH public key to the remote machine to enable passwordless login?) If I click "No", a password prompt follows and I'm in a loop. I can only cancel, but then the rsync argument isn't saved. If I click "yes", I'm in the same loop. I can only cancel, but then the rsync argument isn't saved. Is this another bug? I tried to do it in the config file directly, but it keeps on overwriting it. How can I pass the message to bit? Edit: Forgot to answer the other questions:
|
I'm not sure if I understand that. Is passwordless login activated? I'm also not sure about the Please have a look into the log file of your last snapshot. You will find it in |
|
Sorry no. 2, I used the wrong field (ssh prefix). I'll try again. I've only one big log The second entry, I can't find, but it's in the bit GUI log viewer:
|
That From my understand If the destination would be mounted via NFS or Samba there are a lot of ways to influence permissions. But when using just SSH I have no idea what it could be. |
|
Now I've several errors How can I tell backintime that it shall start a completely new snapshot and do not reuse "new_snapshot"? I'm only using the SSH profile and let backintime do the rest. I'm not aware of any adjustments. |
|
Can you start |
|
Here we go: |
|
I can't see the errors you described. The output looks OK to me. |
|
They started a bit later and are also in the log: There's a lot more... |
|
I restarted my computer and after that, backintime automatically deleted "new-snapshot" and the error messages disappeared. However, it still took several hours and backed up approx. 500 GB of files. The issue remains. |
|
Somehow the permission is modified. That looks like a fact. But it will give us a good hint to further investigation. |
|
The applied The question is: How did this happen? Should we recognize this when generating the
(Oversimplified) So With Should we issue a clear warning in the BiT GUI if a user changes a profile from |
I was asking this myself. The manpage of rsync tells the latest argument wins. I need to sit down to understand the rest of the topic. It is hard for me to get that into my head. 😅 |
|
...continued...: The backintime/common/snapshots.py Line 1124 in 66d248f calling Lines 571 to 592 in 66d248f So We should fix this... Perhaps we could add a "no perms" expert option checkbox and recognize conflicting expert options... |
|
Do you mean this has an effect on the behavior/bug? Here is the part of the rsyn manpage explaining the priority of arguments. |
Looks indeed like the last option wins (even though the example does not directly mention what happens if directly conflicting arguments are used but only if another argument implicitly sets more [here: conflicting] arguments).
I'd say: No (for this issue)... |
Which photo software are you using? Maybe it updates meta data "in place" when viewing the photos ("last viewed")... Can you discover any pattern in the duplicated backup files...? |
|
Hi again, Sorry for the late feedback, but I only found time to dig deeper into this now. First, I'm using digikam as my photo management software. The option "Write to item and XMP Sidecar" is activated. I assume that every time I write a tag to a photo or any similar operation, the file is considered as updated and backintime would add it to the snapshot. However, this does not explain why a lot of files are backed up every time as I do not change the photos every week or month. The good side of this option is that I can see by checking the date of the side car file (xmp file) when I made the last modification and this does not match with the frequent backups. I analysed the inodes of the snapshots and I can see two patterns:
One thing that caught my eyes is that all changing inodes have "-rwxr-x---" as permissions, while the others have "-rw-r--r--". Could that be an issue? Please find the output of "ln -ali" for several example files below: My version history of bt and rsync: |
|
Thanks for your patience and comprehensive analysis!
I'm quite sure this is the root of the problem. It's the same bug/misbehavior as in #994. We're working on it, but it's a tricky one, I'm afraid … |
|
I found that commenting out the "--executability" in tools.py (about at line 899 in the 1.4.1 code) plus addition of the "--no-perms" option solved my problem with the excess copies. I can live with that, but is of course is no clean solution. There seems to be an incomplete implementation of a boolean "no_perms" value internally, that could do what the name implies, but it is
|
|
Sounds like a hot trail ... 🔥 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I've the issue that backintime backs up several files with each snapshot, especially photos. As photos are the vast majority of my backup, it's probably more than 90% which are backed up again and again although I did not touch these files. The speed is also very low and thus needs hours. The transfer is from my laptop (ext4) to my openmediavault NAS (ext4) with 1Gbit LAN. My NAS regularly runs out of space because of this. However, I think it's not a full backup (my last snapshot hat a size of 1.3 TB, my laptop has approx. 1.4 TB). The snapshot compare function makes backintime not responding.
I don't know whether this is related to issue #988. I tried to run it with "--no-perms --no-group --no-owner", but I've the impression that even more files are being backed up with these settings.
I'm using backintime-git from AUR:
The text was updated successfully, but these errors were encountered: