Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: -async #205

Open
potash opened this issue Nov 9, 2015 · 0 comments
Open

Feature request: -async #205

potash opened this issue Nov 9, 2015 · 0 comments

Comments

@potash
Copy link

potash commented Nov 9, 2015

There is a disparity between the doc and the implementation on asynchronous job execution. The doc describes an async option whereas the implementation is the command line option --jobs.

In my pipeline I have some steps that are bigger than others. For the smaller steps I want to run drake --jobs=16 but the big steps will run out of memory or fight for CPU (because the steps are themselves multi-threaded).

The docs describe the "async" option and I think it would be very simple and effective to simply allow "-async" to turn off async execution for a step. In that situation I would expect drake to run that step alone and not in parallel with other steps. In order to do so it may have to wait until previous steps are done.

(There are more complicated variations on this where we might have step options like jobs:4, indicating that this step needs/wants 4 job slots, etc. Another possibility would be for drake could make a $JOBS environment variable available to the step representing how many of the command line arg --jobs slots are currently available. But I think a simple -async would be enough for many use cases)

Thoughts? Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant