[Question] Dynamically change number of concurrent jobs

[dipterix]

Hi,
Thanks for making this great package. I'm writing code to run multiple registries at the same time. Some tasks (collections of jobs) finish prior to the others. I was wondering if is there any way to release/balance/cancel resources dynamically

For example

• Task 1: job 1,2,3, with concurrent jobs = 2
• Task 2: job 1,2,3,4,5,...,100 with concurrent jobs = 10
• Task 3: job 1-1000

If Task 1 finish earlier than the Task 2, is there a way I can increase the max concurrent jobs to 20 for task 2 and cancel task 3 for those which is still submitting?

One way I come up with is we could monitor certain files within the registry folder, if those files exist, then dynamically change the registry settings for jobs that haven't been submitted yet and then remove those files