Remove APScheduler and dateutils dependencies

[m3nu]

My second shot at replacing APScheduler. Thanks to the work by @bastiencyr, it got simpler because the scheduler doesn't need its own thread.

• Remove APScheduler dependency and use QTimer directly
• Option to catch up with missed backups when machine was off or hibernated
• Remove Apply button on Schedule tab
• Add weekly interval option
• Add Python 3.10 support

Fixes #862, fixes #263, fixes #1083, fixes #1088, replaces #551 (PR), replaces #908 (PR)

[bastiencyr]

Hello @m3nu ,
Oh, multiple queues arrived at a good timing.
I have a train at 17h, so will have time to review 😊

[bastiencyr]

The cancel button doesnt work, even if I launch manually a backup. I tried to understand. THe method cancel_all_jobs is not called (you can set DEBUG to true).

[m3nu]

Also noticed that we don't really use many features from python-datutils. So might as well toss that too.

Also noticed that we never removed old deps from Flatpak. Hope that's ok for Flatpak, @Hofer-Julian .

[m3nu]

The cancel button doesnt work, even if I launch manually a backup. I tried to understand. The method cancel_all_jobs is not called (you can set DEBUG to true).

Thanks. Checking this out now.

[m3nu]

The cancel button doesnt work, even if I launch manually a backup. I tried to understand. The method cancel_all_jobs is not called (you can set DEBUG to true).

This is fixed and pushed.

[bastiencyr]

Cancel button works now <33
If I missed 2 backup on two diffferent profiles (and 2 different repos), so only one is launched if I start Vorta.
Step :

• Add schedule backup every minutes for 2 different profiles on 2 different repos.
• Quit Vorta
• Wait 2 minutes (to be sure)
• Relaunch Vorta, Only one backup is started

EDIT:
Its possible that 2 backups are started but in anyway, it is started for the same repo.

[m3nu]

Investigating the 2-profile issue now. I should point out that I'm using the previous backup time including seconds. So the backup may start at 20:15:58, which could be confusing because we show the "Next Time" just as minutes.

Should I drop seconds from the next scheduled job?

[m3nu]

Its possible that 2 backups are started but in anyway, it is started for the same repo.

No, there were 2 bugs in your observation:

1. A race condition where the Borg version wasn't set yet, when the first backup started. So it failed in some cases.
2. Another bug in the workflow, where the catch up was scheduled and then it would schedule the same job again almost at the same time.

[bastiencyr]

Investigating the 2-profile issue now. I should point out that I'm using the previous backup time including seconds. So the backup may start at 20:15:58, which could be confusing because we show the "Next Time" just as minutes.

Should I drop seconds from the next scheduled job?

Yes i notice that. It is confusing, particularly if you test scheduler as a user for the first time. If you can, you should drop secs

[bastiencyr]

Is the new scheduler capable of restarting a backup in case of power failure on the next boot ?

[Hofer-Julian]

Also noticed that we don't really use many features from python-datutils. So might as well toss that too.

Also noticed that we never removed old deps from Flatpak. Hope that's ok for Flatpak, @Hofer-Julian .

Yeah, that part sucks a bit at the moment. You use this flatpak folder only for local development, there is a duplicated one on the flathub repo. Since I seem to be the only one who used that we might want to remove it here until we find another use case again.

[m3nu]

Is the new scheduler capable of restarting a backup in case of power failure on the next boot ?

It's only looking at the event log time to determine the last backup for a profile. This log is saved when the job starts. So if power fails, it wouldn't catch up with this missed backup.

To solve this, we would need to log the at job at the end instead.

Relevant line: https://github.com/borgbase/vorta/pull/1086/files#diff-5aac1bfe6bba70f21fbb7569b769d9a92ff1ecc74e4cf1322bd4894f3f068c1fR209

EDIT: Just noticed that the return code is only added after the job is complete. So I could filter by only adding successful jobs. This would make it catch up with jobs that started, but never finished.

[bastiencyr]

Is the new scheduler capable of restarting a backup in case of power failure on the next boot ?

It's only looking at the event log time to determine the last backup for a profile. This log is saved when the job starts. So if power fails, it wouldn't catch up with this missed backup.

To solve this, we would need to log the at job at the end instead.

Relevant line: https://github.com/borgbase/vorta/pull/1086/files#diff-5aac1bfe6bba70f21fbb7569b769d9a92ff1ecc74e4cf1322bd4894f3f068c1fR209

EDIT: Just noticed that the return code is only added after the job is complete. So I could filter by only adding successful jobs. This would make it catch up with jobs that started, but never finished.

Ok it was just to known because during multiple queues implementation, I planned to save running Job in db and remove them at the end of their execution.

I will test more Vorta today with the new scheduler ;)

[m3nu]

I added a new field end_time to event logs and use that for scheduling. This is because the scheduler can be a bit early and if the backup takes a long time.

Next issue I though of:

How can the scheduler skip profiles that already have a backup running in the background? Would your queue take care of that? E.g. what happens if my backup takes 1h and I schedule them for every 5min? Is this something we need to consider?

[m3nu]

Apart from the issue with long-running backups jobs, I'm happy with the current implementation. Will wait for your guidance on this last question.

[bastiencyr]

I thought about the same thing and the problem is that vorta works at profile level and jobs manager at repository level just like borg. Its easy to limit at only one backup per repos but it is not what the vorta user expect (a repo can be use by 2 different is profile).

I can add a control at profile level in the jobs manager. In any case, I think it is the role of the jobs managet to skip a backup. So it can wait another PR

(Currently, if a user run a backup every 5 mn on a repo, it will be enqueue and run later even if the backup takes 1 hour)

[m3nu]

(Currently, if a user run a backup every 5 mn on a repo, it will be enqueue and run later even if the backup takes 1 hour)

Would be good if you can set it to skip same repos. It's fine if it's at the repo level instead of profile level. That makes even more sense from Borg's point of view.

So I'll merge this on Tuesday or so and then you can adjust the job scheduler to skip duplicates.

Thanks for the feedback and reviewing!