Enhancement requests

[COshmyan]

Hi!
First of all, I'd like to thank for this great project - it is really useful!

At the same time, I'd like to share my troubles also and ask some enhancement requests.
At the moment we use a cpu-only version 1.2.1 of Task Spooler.

1. I'd like to monitor the current size of queue. It's possible to use the "-R" option to receive the number of jobs that are currently running; however, there is no a simple similar option to obtain the number of jobs that are currently waiting in the queue (or totally: sum of running and queued jobs, i.e. current queue size).

2. I'd like to have possibility to add job to the queue only if this job still has not been added.

Use case 1
Some jobs are scheduled to be placed into a queue via CRON (for example, every 10 minutes). However, sometimes it could be occurred that this job is running longer then usual (for example, 15-20 minutes). In this situation the CRON daemon will place this job to the queue twice or thrice that is not needed.
In this case I need to place job to the queue only if it's absent in this queue in either the "running" or "queued" states.

Use case 2
Job is placed to a queue by some external signal. Job should process some new data, and a signal has meaning: "some new data has arrived and ready for processing". However, sometimes new data could arrive just during the job is running. In this case this job should be re-started again when it completes; so it should be placed to the queue again. At the same time, if this job is queued already, it's not necessary to place it to this queue more that once.
In this case I need to place job to the queue only if it's absent in this queue in "queued" state only (even if it's present in "running" state).

3. Probably, the usage of labels could be extended. At least, it could be useful to check a state of some specific jobs identifying them by label instead of job ID.

4. All these tasks could be solved using some scripts obtaining the list of jobs with all their attributes (state, label, command line, etc.). However, the current version shortens some long fields (at least, label and command lines) according to the screen size; even if the command "ts -l" is used in scripts and STDOUT is not a terminal but redirected to the file or pipe. It causes to problem: script should initially obtain the list of jobs, and then to loop by this list using a job ID for every job and use some options like "-i", "-s", "-p" or "-a" to obtain a details for a specific job. Besides an additional work, it's impossible to obtain a consistent information, as during this loop the state is changing. Is it possible, please, to have possibility to obtain the full queue list (independently of a label and command sizes) for processing in scripts? It could be acceptable if this list has some well known format (for example, CSV or TAB-separated fields).

[justanhduc]

Hi @COshmyan. Thanks for suggesting interesting new features. Before actually implementing something, I need to clarify my understanding a bit.

1. A prototype of this feature actually has existed a long time ago for the GPU version (Please see here). In the end, I was satisfied with some workaround, so this feature has never made it to user's UI. However, if there are interests, surely I can make a UI for it. Could you please confirm this is the feature you want (for CPU of course)?
2. and 3. Surely this looks quite useful and we need to discuss further about its behavior here. How can ts distinguish jobs? Maybe you want to do it by labels? Then does it mean these options should be available only for jobs with labels, and throw some error message if jobs do not have? Do you have any UI in mind? Maybe --queued_once and --run_once?
3. Do you mean there should be a feature like ts -l but without shortening? I can make something like dumping the job list in CSV format to stdout like you suggested.

Finally, would you like to send a PR for any propositions here?

[COshmyan]

Hi @justanhduc , thank you for a reply!

I'm not sure I understand what the PR does mean. Is it Pull Request? What exactly should I perform?

For me the most actual is my enhancement request number 4.
Yes, I'd like to have something like ts -l but without any shortening, acceptable for processing in scripts.
For example, it's not necessary to make a running time be more human-readable (substituting seconds, minutes or hours), it's enough to have a row values only (second or milliseconds, what they are?).
It is not our case, but in theory - a command line could contain almost any character inside, including a 'comma' (,) and 'quote' (") characters. So, it's necessary to define: how to process them in the CSV output (just forbid? screen by backslashes?).

Regarding feature request number 1: yes, there is an interest to have some possibility to obtain a current queue size (i.e. both: currently running and still queued). It could be quite acceptable to have a single command-line parameter to return 2 numbers together. I'm feeling it should not be very difficult to implement.

Regarding feature request 2 and 3: yes, I mean to use a labels for a job identification. Unfortunately, at the moment we have another troubles with label usage (see a neighboring issue).
However, these feature requests are much less actual, especially if the feature request number 4 is implemented.

[justanhduc]

Hi @COshmyan. Then let's do 4 as it covers all you need. ALso I would not be too bothered by complex commands. As stated (somewhere) in the readme, ts is not a command parser so if the command is complicated users better put it in a script. Can you suggest a character to act as the delimiter?

Yes PR means pull request. Basically you can contribute codes to this repo and your name will show up as a contributor of this project.

[COshmyan]

Hi @justanhduc ,

Can you suggest a character to act as the delimiter?

My proposal is the following:

• use a TAB character as a field separator;
• if the TAB character is present in the original command line, replace it by the 'space' character in the output.

I believe that there is no an ideal universal solution (for every use-case); but this approach could mininize a side effects.

Thank you for your time! :)

[justanhduc]

Hi @COshmyan. I'm so sorry for the belated update, but finally I have made this feature available in the cpu-only branch. You can get a job list in plain text by ts --plain. Be sure to check it out and let me know if something goes wrong!

[COshmyan]

Great news, thanks!
We will test it.