Adding proto definitions for supporting SageMaker TrainingJob (built-in algorithms) and HyperparameterTuningJob #66
Conversation
|
|
||
| package flyte.plugins.sagemaker; | ||
|
|
||
| option go_package = "github.com/kumare3/awsflyteplugins/gen/pb-go/proto"; |
There was a problem hiding this comment.
Hmmm I think you should follow the regular path here... of other plugins...
option go_package = "github.com/lyft/flyteidl/gen/pb-go/flyteidl/plugins";
There was a problem hiding this comment.
Since this has a lot of messages, can you actually add one extra level of nesting?
option go_package = "github.com/lyft/flyteidl/gen/pb-go/flyteidl/plugins/sagemaker";
If this works, can you also remove the SageMaker prefix of the messages below?
There was a problem hiding this comment.
sure let me give it a try
| HPOJobObjective Objective = 2; | ||
| int64 MaxNumberOfTrainingJobs = 3; | ||
| int64 MaxParallelTrainingJobs = 4; | ||
| ParameterRanges ParameterRanges = 5; |
There was a problem hiding this comment.
Is this really a "config"? not an input of the job?
There was a problem hiding this comment.
This was trying to follow the CRD definition of the amazon-sagemaker-operator-for-k8s. But I think what you suggested here makes more sense.
| ParameterRanges ParameterRanges = 5; | ||
| } | ||
|
|
||
| message SagemakerHPOJob { |
There was a problem hiding this comment.
I presume you meant for this to go into the custom field of the task?
| int64 max_wait_time_in_seconds = 2; | ||
| } | ||
|
|
||
| message VpcConfig { |
There was a problem hiding this comment.
I think we should leave this out. This should be a backend configuration IMO
|
|
||
| message TrainingJob { | ||
| string region = 1; | ||
| string role_arn = 2; |
There was a problem hiding this comment.
I think this should be the same role that we are using for the workflow?
There was a problem hiding this comment.
Right. That means we do not need to define this as a part of the message?
protos/flyteidl/plugins/sagemaker/hyperparameter_tuning_job.proto
Outdated
Show resolved
Hide resolved
|
Lets just mostly remove the |
…in algorithms) and HyperparameterTuningJob (#66) * adding sagemaker protos Co-authored-by: Haytham AbuelFutuh <[email protected]>
TL;DR
This PR adds the essential proto messages for the support of SageMaker TrainingJob and HyperparameterTuningJob.
Type
Are all requirements met?
Complete description
This PR adds the essential proto messages for the support of SageMaker TrainingJob and HyperparameterTuning.
The major messages added include
HyperparameterTuningJob,TrainingJob, andParameterRanges.The additions allow users to:
Tracking Issue
flyteorg/flyte#255
Follow-up issue
flyteorg/flyte#431