[Bug?]: Tagging torrents for cleanup after the criteria is met doesn't delete the files.

[daviddanko]

Describe the Bug

Tagging a torrent with a criteria for cleanup which is already met (for example max_seeding_time=1m, ratio=0.01,... ) doesn't remove the files from the disk, only the .torrent from the client.

So when I start downloading something and I plan to delete it after let's say it reaches ratio 2.0, but I have the scheduler run only every 24 hour, if I already reached a ratio 2+, and THEN scheduler tags it with that rule, the torrent is deleted, however the file stays on the disk.

Not sure if that's intended though.

Config

# This is an example configuration file that documents all the options.
# It will need to be modified for your specific use case.
# Please refer to the link below for more details on how to set up the configuration file
# https://github.com/StuffAnThings/qbit_manage/wiki/Config-Setup

commands:
  # The commands defined below will IGNORE any commands used in command line and docker env variables.
  dry_run: false
  cross_seed: false
  recheck: false
  cat_update: false
  tag_update: true
  rem_unregistered: true
  tag_tracker_error: false
  rem_orphaned: false
  tag_nohardlinks: false
  share_limits: true
  skip_qb_version_check: false
  skip_cleanup: false

qbt:
  # qBittorrent parameters
  host: 
  user:
  pass:

settings:
  force_auto_tmm: false # Will force qBittorrent to enable Automatic Torrent Management for each torrent.
  tracker_error_tag: issue # Will set the tag of any torrents that do not have a working tracker.
  nohardlinks_tag: noHL # Will set the tag of any torrents with no hardlinks.
  share_limits_tag: seed_limit # Will add this tag when applying share limits to provide an easy way to filter torrents by share limit group/priority for each torrent
  share_limits_min_seeding_time_tag: MinSeedTimeNotReached # Tag to be added to torrents that have not yet reached the minimum seeding time
  share_limits_min_num_seeds_tag: MinSeedsNotMet # Tag to be added to torrents that have not yet reached the minimum number of seeds
  share_limits_last_active_tag: LastActiveLimitNotReached # Tag to be added to torrents that have not yet reached the last active limit
  cross_seed_tag: cross-seed # Will set the tag of any torrents that are added by cross-seed command
  cat_filter_completed: false # Filters for completed torrents only when running cat_update command
  share_limits_filter_completed: false # Filters for completed torrents only when running share_limits command
  tag_nohardlinks_filter_completed: true # Filters for completed torrents only when running tag_nohardlinks command
  cat_update_all: true # Checks and udpates all torrent categories if set to True when running cat_update command, otherwise only update torrents that are uncategorized

directory:
  root_dir: /media/downloads/
  cross_seed:
  recycle_bin: '' #/media/downloads/.RecycleBin
  torrents_dir: '' #/config/qbittorrent/qBittorrent/BT_backup
  orphaned_dir: '' #/media/downloads/orphaned_data
  remote_dir: ''

cat:
  movie-radarr: /media/movies/
  show-sonarr: /media/shows/

cat_change:

tracker:
  ncore:
    tag: nCore
  other:
    tag: other

nohardlinks:

share_limits:
  nCore:
    priority: 1
    include_any_tags:
    - nCore
    max_seeding_time: 30d
    cleanup: true
  default:
    priority: 2
    max_seeding_time: 30d
    max_ratio: 2
    cleanup: true

recyclebin:
  enabled: false
  empty_after_x_days:
  save_torrents: false
  split_by_category: false

orphaned:
  empty_after_x_days: 10
  exclude_patterns:

apprise:

notifiarr:

webhooks:
  error:
  run_start:
  run_end:
  function:
    cross_seed:
    recheck:
    cat_update:
    tag_update:
    rem_unregistered:
    tag_tracker_error:
    rem_orphaned:
    tag_nohardlinks:
    share_limits:
    cleanup_dirs:

Logs

https://gist.github.com/daviddanko/44093e58e9bcef8ef65e830d4ba92a54

Screenshots

No response

Installation

Docker

Version Number

4.1.7

What branch are you on?

master

[bakerboy448]

Not a bug; nothing indicates this is a bug...docs clearly indicate it's not a bug

WARNING!! Setting this as true will remove and delete contents of any torrents that satisfies the share limits (max time OR max ratio) It will also delete the torrent's data if and only if no other torrents are using the same folder/files.

[daviddanko]

Alright, what about this case?
Let's say I have set share limits max_ratio=1.0 with a cleanup=true and have qbit_manage running every 4 hours.
At 08:00 the script runs.
At 08:10 I download two movies, movie A and movie B.
At 11:50 movie A has a ratio of 0.95 and movie B a ratio 1.05.
At 12:00 qbit_manage runs and tags both of them with the same rule.
Movie A gets tagged, it doesn't meet the criteria (has a ratio of 0.95), so torrent + files are not deleted. Correct.
Movie B, however, meets the criteria (has a ratio of 1.05), but because it wasn't tagged beforehand, only the torrent gets deleted, files stay on the disk.
At 12:10 movie A reaches the ratio of 1.00 and both the torrent and files are deleted, which is correct, however, the files of movie B are still on the disk and will stay there forever.
Wouldn't it make sense if movie B's files would also be deleted?
I have applied the same exact ruleset for both movies, they basically reach the same ratio (which in this case is the only criteria for deletion), but because one torrent reached it 10 minutes earlier, then it stays on the disk?

[bobokun]

The only time files will not be deleted if multiple torrents are referencing the same movie.
So if you have Movie A [Tracker 1] and Movie A [Tracker 2], and Movie B [Tracker 1].

If Movie A[Tracker 1] Meets the share limits, only the torrent will be deleted and not the files (Because the Movie A [Tracker 2] is still referencing the files)
Once Movie A [Tracker 2] meets share limits then both the torrent and files will be deleted.
If Movie B [Tracker 1] meets share limits then the torrent and files will be deleted because no other torrent is referencing movie B. Hope that helps.

[daviddanko]

I'm aware of that but that’s not what’s happening. Each and every one of my torrents reference only one movie. It’s about when is the torrent tagged. I’ve tested it a couple times and that’s my conclusion at least.

[daviddanko]

Hi bobokun, have you had the chance to check this? I've run some tests today as well and I am pretty sure that's what's happening.

Every torrent of mine references only one file. No files are referenced by two torrents. When the torrent is tagged with a rule which is already met, the files are never deleted. I have to tag the torrent before the rule is met for it to delete the files + torrent.

Just download something and don't tag it yet. Let it seed for a ratio of 0.02 or more. Then tag it with a rule which deletes the torrent when a ratio reaches 0.01. And check if the files are deleted.

[bobokun]

Try setting the remote_dir to be the same as root_dir instead of leaving it ''

[bobokun]

What are you tagging the torrent in order to trigger the rule to delete?

[daviddanko]

Try setting the remote_dir to be the same as root_dir instead of leaving it ''

I did it and downloaded something. Same behavior.

You can see that the ratio, which is one of the requirements for deletion, is already met before tagging. So the content files were not deleted. If I had the max_ratio set to 3.00, I would have tagged it while it has only a ratio a 2.24, as seen below, and the files would've been deleted.

|========= Cleaning up torrents that have reached ratio/seed limit for default. Priority 2 ==========|
| Torrent Name: xxxxxxxxxxxxxxxxxx.2024.1080p.AMZN.WEBRip.1600MB.DD5.1.x264-Gala |
| Tracker: http://tracker.opentrackr.org:1337 |
| Ratio vs Max Ratio: 2.24 >= 2.00 |
| Cleanup: True [Meets Share Limits] |
| Deleted .torrent but NOT content files. |
| Scheduled Mode: Running every 1 hour. |
|====================================================================================================|

What are you tagging the torrent in order to trigger the rule to delete?

I'm afraid I don't understand the question.

[daviddanko]

Hi bobokun, have you been able to test this?

[Dark3clipse]

I seem to have the same issue. My torrents files don't get deleted even though I don't reference the same files in other torrents.

|    Torrent Name: Jonestown.Terror.in.the.Jungle.S01.1080p.AMZN.WEB-DL.DDP2.0.H.264-playWEB         |
|         Tracker: http://nebulance.io:4400                                                          |
|         Seeding Time vs Max Seed Time: 6 days, 19:09:14 >= 5 days, 0:00:00                         |
|         Cleanup: True [Meets Share Limits]                                                         |
|         Deleted .torrent but NOT content files.   

This happens for all my torrents.

[bakerboy448]

@Dark3clipse Sharing logs that show a torrent is removed but not content itself is completely and utterly useless....

as stated earlier - content is only deleted if no other torrents are referring to that data....that proof has not been provided

[Dark3clipse]

@bakerboy448 I don't understand why other torrents could be refering to the same data. I'm not using cross seed, it's from a private tracker, and it has its own content folder.

I've looked at the source code and I noticed there are multiple locations where this line is printed. I will try to make these prints unique and build a custom image, to see which of the print lines is triggering for me.

[Dark3clipse]

I changed the source code as follows:

if os.path.exists(torrent["content_path"].replace(self.root_dir, self.remote_dir)):
                    # Checks if any of the original torrents are working
                    if self.qbt.has_cross_seed(torrent) and ("" in t_msg or 2 in t_status):
                        self.stats_deleted += 1
                        attr["torrents_deleted_and_contents"] = False
                        t_deleted.add(t_name)
                        if not self.config.dry_run:
                            self.qbt.tor_delete_recycle(torrent, attr)
                        body += logger.print_line(
                            logger.insert_space("Deleted .torrent but NOT content files. Reason: is cross-seed", 8),
                            self.config.loglevel,
                        )
                    else:
                        self.stats_deleted_contents += 1
                        attr["torrents_deleted_and_contents"] = True
                        t_deleted_and_contents.add(t_name)
                        if not self.config.dry_run:
                            self.qbt.tor_delete_recycle(torrent, attr)
                        body += logger.print_line(
                            logger.insert_space("Deleted .torrent AND content files.", 8), self.config.loglevel
                        )
                else:
                    self.stats_deleted += 1
                    attr["torrents_deleted_and_contents"] = False
                    t_deleted.add(t_name)
                    if not self.config.dry_run:
                        self.qbt.tor_delete_recycle(torrent, attr)
                    body += logger.print_line(
                        logger.insert_space("Deleted .torrent but NOT content files. Reason: path does not exist [path="+torrent["content_path"].replace(self.root_dir, self.remote_dir)+"].", 8), self.config.loglevel
                    )

Now my tracing is as follows:

|    Torrent Name: Jonestown.Terror.in.the.Jungle.S01.1080p.AMZN.WEB-DL.DDP2.0.H.264-playWEB         |
|         Tracker: http://nebulance.io:4400                                                          |
|         Seeding Time vs Max Seed Time: 6 days, 20:39:42 >= 5 days, 0:00:00                         |
|         Cleanup: True [Meets Share Limits]                                                         |
|         Deleted .torrent but NOT content files. Reason: path does not exist [path=/mnt/user/data/torrents/autobrr/Jonestown.Terror.in.the.Jungle.S01.1080p.AMZN.WEB-DL.DDP2.0.H.264-playWEB]. |

Which is indeed not a correct path from the perspective of the running container. So this appears to be a configuration problem on my end, sorry for bumping the issue. Unsetting remote_dir solved it for me.

[bakerboy448]

PRing that logging tweak likely won't hurt :)

[bakerboy448]

@daviddanko can you pull develop with the enhanced logging and re-run see if easier to debug now

[daviddanko]

bakerboy448, thanks for still looking into this. I tried and I am getting the same error as Dark3clipse.

|====================== Updating Share Limits for [Group default] [Priority 2] ======================|
|    Torrent Name: XXXXXXXXXXXXX S04E01 1080p HEVC x265-MeGusta                                           |
|         Tracker: http://tracker.opentrackr.org:1337                                                |
|         Added Tag: seed_limit_2.default                                                            |
|     Share Limit: Max Ratio = 2.0, Max Seed Time = 0:01:00                                          |
|========= Cleaning up torrents that have reached ratio/seed limit for default. Priority 2 ==========|
|    Torrent Name: The boys S04E01 1080p HEVC x265-MeGusta                                           |
|         Tracker: http://tracker.opentrackr.org:1337                                                |
|         Seeding Time vs Max Seed Time: 0:01:36 >= 0:01:00                                          |
|         Cleanup: True [Meets Share Limits]                                                         |
|         Deleted .torrent but NOT content files. Reason: path does not exist [path=/media/downloads/XXXXXXXXXXXXX.S04E01.1080p.HEVC.x265-MeGusta[EZTVx.to].mkv]. | 

I used the following rule:

default:
    priority: 2
    max_seeding_time: 1m
    max_ratio: 2
    cleanup: true

I downloaded that torrent, run qbit_manage 1.5 mins AFTER finishing the torrent and it showed the above error. The torrent was deleted, but not the files.

However, that being said, I deleted the files and redownloaded THE SAME TORRENT, tagged it qbit_manage IMMEDIATELY after adding it to qbittorrent and when the 1 minute passed the files were now gone. So the path does in fact exist and I can see it in the container:

2da58dc92ac2:/media/downloads$ ls -la
total 632815
drwxrwx--- 21 1000 1000        22 Aug 28 09:25  .
drwxrwx---  5 1000 1000         5 Aug 26 11:00  ..
-rwxrwx---  1 1000 1000 665214048 Aug 28 09:26 'XXXXXXXXXXXXX.S04E01.1080p.HEVC.x265-MeGusta[EZTVx.to].mkv'

I have no idea why, but for me it's the time of the torrent tagging which has the impact on whether the files will be deleted or not.

Tried it with remote_dir and without.

Here's my config:

commands:
  # The commands defined below will IGNORE any commands used in command line and docker env variables.
  dry_run: false
  cross_seed: false
  recheck: false
  cat_update: false
  tag_update: true
  rem_unregistered: true
  tag_tracker_error: false
  rem_orphaned: false
  tag_nohardlinks: false
  share_limits: true
  skip_qb_version_check: false
  skip_cleanup: false

qbt:
  host: https://qbit......./
  user:
  pass:

settings:
  force_auto_tmm: false # Will force qBittorrent to enable Automatic Torrent Management for each torrent.
  tracker_error_tag: issue # Will set the tag of any torrents that do not have a working tracker.
  nohardlinks_tag: noHL # Will set the tag of any torrents with no hardlinks.
  share_limits_tag: seed_limit # Will add this tag when applying share limits to provide an easy way to filter torrents by share limit group/priority for each torrent
  share_limits_min_seeding_time_tag: MinSeedTimeNotReached # Tag to be added to torrents that have not yet reached the minimum seeding time
  share_limits_min_num_seeds_tag: MinSeedsNotMet # Tag to be added to torrents that have not yet reached the minimum number of seeds
  share_limits_last_active_tag: LastActiveLimitNotReached # Tag to be added to torrents that have not yet reached the last active limit
  cross_seed_tag: cross-seed # Will set the tag of any torrents that are added by cross-seed command
  cat_filter_completed: false # Filters for completed torrents only when running cat_update command
  share_limits_filter_completed: false # Filters for completed torrents only when running share_limits command
  tag_nohardlinks_filter_completed: true # Filters for completed torrents only when running tag_nohardlinks command
  cat_update_all: true # Checks and udpates all torrent categories if set to True when running cat_update command, otherwise only update torrents that are uncategorized
  force_auto_tmm_ignore_tags: []

directory:
  root_dir: /media/downloads/
  cross_seed: ''
  recycle_bin: '' #/media/downloads/.RecycleBin
  torrents_dir: '' #/config/qbittorrent/qBittorrent/BT_backup
  orphaned_dir: '' #/media/downloads/orphaned_data
  remote_dir: ''

cat:
  movie-radarr: /media/movies/
  show-sonarr: /media/shows/

cat_change:

tracker:
  ncore:
    tag: nCore
  other:
    tag: other

nohardlinks:

share_limits:
  nCore:
    priority: 1
    include_any_tags:
    - nCore
    max_seeding_time: 30d
    cleanup: true
  default:
    priority: 2
    max_seeding_time: 1m
    max_ratio: 2
    cleanup: true

recyclebin:
  enabled: false
  empty_after_x_days:
  save_torrents: false
  split_by_category: false

orphaned:
  empty_after_x_days: 10
  exclude_patterns:

apprise:

notifiarr:

webhooks:
  error:
  run_start:
  run_end:
  function:
    cross_seed:
    recheck:
    cat_update:
    tag_update:
    rem_unregistered:
    tag_tracker_error:
    rem_orphaned:
    tag_nohardlinks:
    share_limits:
    cleanup_dirs:

Also adding the compose in case something is iffy there:

qbittorrent:
   image: lscr.io/linuxserver/qbittorrent:latest
   container_name: qbittorrent
   environment:
     - PUID=${PUID}
     - PGID=${PGID}
     - TZ=${TIMEZONE}
     - WEBUI_PORT=${QBIT_WEBUI_PORT}
     - DOCKER_MODS=ghcr.io/gabe565/linuxserver-mod-vuetorrent
   volumes:
     - ${QBIT_CONFIG_FOLDER}/qbittorrent:/config
     - /mnt/nas/media:/media
   ports:
     - ${QBIT_WEBUI_PORT}:${QBIT_WEBUI_PORT}
     - ${QBIT_PORT}:6881
     - ${QBIT_PORT}:6881/udp
   networks:
     - qbit
   restart: unless-stopped

 qbm:
   container_name: qbm
   image: ghcr.io/stuffanthings/qbit_manage:develop
   volumes:
     - ${QBIT_CONFIG_FOLDER}/qbm:/config
     - /mnt/nas/media:/media:rw
     - ${QBIT_CONFIG_FOLDER}/qbittorrent:/qbittorrent/:ro
   environment:
     - PUID=${PUID}
     - PGID=${PGID}
     - TZ=${TIMEZONE}
   #  - QBT_STARTUP_DELAY=60
     - QBT_SCHEDULE=60
     - QBT_CONFIG=config.yml
     - QBT_LOGFILE=activity.log
     - QBT_LOG_LEVEL=DEBUG
   networks:
     - qbit
   restart: unless-stopped

[daviddanko]

Simply speaking, if I want everything (torrent + files) to be deleted, I have to tag the torrent before the conditions are met.

So in this case I want to delete the torrent + files after 1 minute of seeding. If qbit_manage tags it before the seed time reaches 1 minute, everything is fine. But if I tag it after 2 mins of seed time, the files will stay on the disk.

Dark3clipse, would you mind testing this, please?

[daviddanko]

In case someone has the same issue in the future, you should use the
image: ghcr.io/hotio/qbitmanage
That fixed it for me.

[bakerboy448]

using a different image would not materially change the underlying qbitmanage code

[daviddanko]

using a different image would not materially change the underlying qbitmanage code

No one said it would, but that was the recommendation from the dev (bobokun) on the discord channel, and I can confirm, after thorough testing, that it fixed my issue.