Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add queue management script with rotating capability and category support #66

Closed
chros73 opened this issue Jul 16, 2016 · 0 comments
Closed

Comments

@chros73
Copy link
Owner

chros73 commented Jul 16, 2016

Add queue management script with rotating (deleting data) capability and category support. (Original idea by Stefano)
Run it from cron in every 5 minutes.

This is how the setup works: it's completely based on moving data AND meta files all around the place.

  • there's a .downloading dir (with subdirs = categories) (watch-dir) with which rtorrent will start downloading new torrents
  • when a download is finished rtorrent moves their data (from incomplete dir) AND meta files (from one of the subdirs of .downloading dir) into their final place! The .downloading dir gets empty all the time in this way.
  • the .queue dir (with subdirs = categories) holds the meta files that should be moved into one of the subdirs of .downloading dir to be able to start them by rtorrent
  • there are 3 predefined categories/dirs for rotating torrents: rotating, unsafe categories and .delqueue dir (the latter should contain only symlinks to deletable metafiles)
  • there's also a unsafe_data custom variable in rtorrent (values: [0|1|2]) that makes torrents deletable (value 2 means that the torrent is in delete-queue)

It contains 2 functionality:

  • downloading queue manager
  • disk free space manager that will delete data to make free space for the newcomers (this is included in the downloading queue manager)

A. Downloading queue manager: moves meta files from one of the subdirs of .queue dir into one of the subdirs of .downloading dir (so rtorrent can start to download them)

  • its top priority is that it shouldn't break rtorrent in any case
  • checks how many meta files are in the .downloading dir: means what the number of active downloads is currently and moves the appropriate number of meta files from one of the subdirs of .queue dir
  • size of downloading queue and order of processing is configurable
  • doesn't move meta file(s) if:
    • rtorrent stopped running
    • there's not enough disk space for the queueable torrents (even after trying to make more)
    • target data dir/file (in incomplete dir) OR meta file exists (in one of the subdirs of .downloading dir), in this case move meta file into .duplicated dir and append current time to its filename
  • it runs disk free space manager every time (except for when rtorrent is stopped)
  • it can send report about all the action that was made (some of them are configurable)

Note:

  • if a meta file is placed directly into one of the subdirs of .downloading dir then downloading will start immediately (hash-checkable data/meta should be used in this way)
  • the categories directory structure has to be created 4 times in this way, in: /mnt/Torrents, .queue, .downloading, .completed dirs

B. Disk free space manager: deletes meta files from the predefined rotating, unsafe category dirs and from .delqueue dir (so rtorrent can delete their data)

  • rtorrent is configured to only delete their data if their unsafe_data custom variable is set to [1|2] !
  • processing order is the following:
    • deletes meta files from .delqueue dir in reverse order (oldest first)
    • deletes meta files from the predefined rotating, unsafe category dirs in reverse order (oldest first), handling them altogether
  • enabling this part of the script is configurable (disabled by default for safety reasons with AUTOROTATETORRENTS=false)
  • doesn't delete meta file(s) if:
    • this functionality is disabled
    • there's enough disk space
    • there's nothing to be deleted from the defined rotating, unsafe categories and .delqueue dir
  • it should run by the downloading queue manager
  • amount of reserved disk space is configurable
  • it should wait for 15 seconds after successful deletion for letting rtorrent to finish deleting data
  • it can send report about all the deleted torrents (configurable)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant