Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tune] Support extending Async Hyperband runs past max_t #14235

Closed
ANarayan opened this issue Feb 21, 2021 · 0 comments · Fixed by #24222
Closed

[tune] Support extending Async Hyperband runs past max_t #14235

ANarayan opened this issue Feb 21, 2021 · 0 comments · Fixed by #24222
Assignees
Labels
enhancement Request for new feature and/or capability P1 Issue that should be fixed within a few weeks tune Tune-related issues

Comments

@ANarayan
Copy link

Would it please be possible to extend the async hyperband scheduler implementation to have the best n trials train to completion beyond max_t and stop based on some other stopping criteria (i.e. TrialPlateauStopper)? Currently, the scheduler stops training a given trial when it is the last one left in its bracket. Hence, it wont actually return a fully trained model that has minimized or maximized the metric in question.

A similar feature was implemented for the BOHB/Hyperband scheduler here.

Having access to the fully trained models and knowledge of their final performance metrics would be very useful for benchmarking and comparative analysis tasks.

@ANarayan ANarayan added the enhancement Request for new feature and/or capability label Feb 21, 2021
@richardliaw richardliaw added the tune Tune-related issues label Feb 22, 2021
@richardliaw richardliaw assigned richardliaw and unassigned ANarayan Feb 22, 2021
@richardliaw richardliaw added the P1 Issue that should be fixed within a few weeks label Feb 22, 2021
krfricke added a commit that referenced this issue Apr 27, 2022
…er max_t (#24222)

Adds a new flag `stop_last_trials` to AsyncHyperband that allows the last trials of each bracket to continue training after `max_t`. This feature existed for synchronous hyperband before, and the extension had been requested in #14235.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Request for new feature and/or capability P1 Issue that should be fixed within a few weeks tune Tune-related issues
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants