-
Notifications
You must be signed in to change notification settings - Fork 583
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Identify the action items for adapting custom training in HPO #467
Comments
I created an HPOJob with the following CRD
and this is the log of one of the underlying trainingjob https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logsV2:log-groups/log-group/$252Faws$252Fsagemaker$252FTrainingJobs/log-events/9c6aeeed28af4dd48d0ff14af57ec168-005-1db03577$252Falgo-1-1597092293 In which it can be easily seen that SageMaker still uses the same way to pass hyperparameters to the underlying training job
This just confirms that the training job underlying a hpo job also uses the same interface as that of a standalone training job. Ok, now we will have to figure out how to pass them to user's python function... are hyperparameter inputs? if so we will have to download and manipulate the inputs?.What SageMaker does is that it will put a summarized map of hyperparameters and values to the path |
I found a potential problem when doing the experiment: |
* Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * Fix build break Signed-off-by: Haytham Abuelfutuh <[email protected]> * Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> Signed-off-by: Flyte-Bot <[email protected]> Signed-off-by: Haytham Abuelfutuh <[email protected]> Co-authored-by: flyte-bot <[email protected]> Co-authored-by: Haytham Abuelfutuh <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
…ion for child task execution (flyteorg#467)
* Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * Fix build break Signed-off-by: Haytham Abuelfutuh <[email protected]> * Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> Signed-off-by: Flyte-Bot <[email protected]> Signed-off-by: Haytham Abuelfutuh <[email protected]> Co-authored-by: flyte-bot <[email protected]> Co-authored-by: Haytham Abuelfutuh <[email protected]>
Hello 👋, This issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will close the issue if we detect no activity in the next 7 days. Thank you for your contribution and understanding! 🙏 |
Hello 👋, This issue has been inactive for over 9 months and hasn't received any updates since it was marked as stale. We'll be closing this issue for now, but if you believe this issue is still relevant, please feel free to reopen it. Thank you for your contribution and understanding! 🙏 |
Signed-off-by: troychiu <[email protected]>
Signed-off-by: troychiu <[email protected]>
Signed-off-by: troychiu <[email protected]>
Signed-off-by: troychiu <[email protected]>
Hello 👋, this issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will engage on it to decide if it is still applicable. |
HyperparameterTuningJobTask
should copy thetimeout
value from its underlyingCustomTrainingJobTask
outputs
of the underlyingCustomTrainingJobTask
should be embedded into the output interface ofHyperparameterTuningJobTask
inputs.pb
The text was updated successfully, but these errors were encountered: