Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SageMaker on Flyte: TrainingJob for training with built-in algorithms and basic HPOJob support [Alpha] #120

Merged
merged 88 commits into from
Jul 31, 2020
Merged

SageMaker on Flyte: TrainingJob for training with built-in algorithms and basic HPOJob support [Alpha] #120

merged 88 commits into from
Jul 31, 2020

Commits on Jun 3, 2020

  1. adding trainingjob model and sagemaker task

    @bnsblue
    bnsblue committed Jun 3, 2020
    Configuration menu
    Copy the full SHA
    f80f9e7 View commit details
    Browse the repository at this point in the history

Commits on Jun 4, 2020

  1. adding models for sagemaker proto messages

    @bnsblue
    bnsblue committed Jun 4, 2020
    Configuration menu
    Copy the full SHA
    db31d1c View commit details
    Browse the repository at this point in the history

  2. add new line at eof

    @bnsblue
    bnsblue committed Jun 4, 2020
    Configuration menu
    Copy the full SHA
    55420b8 View commit details
    Browse the repository at this point in the history

Commits on Jun 9, 2020

  1. adding common trainingjob task

    @bnsblue
    bnsblue committed Jun 9, 2020
    Configuration menu
    Copy the full SHA
    0edef9c View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2020

  1. redo flytekit changes to comply with new interface and proto definition

    @bnsblue
    bnsblue committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    0e98520 View commit details
    Browse the repository at this point in the history

  2. Fix a logic bug in training job model. Adding SdkSimpleTrainingJobTas… 

    …k type
    @bnsblue
    bnsblue committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    5fa5d40 View commit details
    Browse the repository at this point in the history

  3. Add a comment

    @bnsblue
    bnsblue committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    d588e3e View commit details
    Browse the repository at this point in the history

  4. Add SdkSimpleHPOJobTask

    @bnsblue
    bnsblue committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    54c5d2e View commit details
    Browse the repository at this point in the history

  5. Remove the embedding of the underlying trainingjob's output from the … 

    …hpojob's interface
    @bnsblue
    bnsblue committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    27a9ba4 View commit details
    Browse the repository at this point in the history

  6. fix a typo

    @bnsblue
    bnsblue committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    e40e033 View commit details
    Browse the repository at this point in the history

  7. add new line at eof

    @bnsblue
    bnsblue committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    56ba534 View commit details
    Browse the repository at this point in the history

  8. adding custom training job sdk type

    @bnsblue
    bnsblue committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    a38657d View commit details
    Browse the repository at this point in the history

Commits on Jun 23, 2020

  1. add code for tranlating an enum in hpo_job model; fix hpo_job_task sd… 

    …k task
    @bnsblue
    bnsblue committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    a081fa0 View commit details
    Browse the repository at this point in the history

  2. missing a colon

    @bnsblue
    bnsblue committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    e38674c View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2020

  1. add the missing input stopping_condition for training job tasks

    @bnsblue
    bnsblue committed Jun 24, 2020
    Configuration menu
    Copy the full SHA
    18172ea View commit details
    Browse the repository at this point in the history

  2. bump flyteidl version

    @bnsblue
    bnsblue committed Jun 24, 2020
    Configuration menu
    Copy the full SHA
    f08be91 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2020

  1. bump to a beta version

    @bnsblue
    bnsblue committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    1df9b63 View commit details
    Browse the repository at this point in the history

  2. merge with master and bump version accordingly

    @bnsblue
    bnsblue committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    635f19c View commit details
    Browse the repository at this point in the history

  3. fixing unit tests

    @bnsblue
    bnsblue committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    7e8ff64 View commit details
    Browse the repository at this point in the history

  4. fixing unit tests

    @bnsblue
    bnsblue committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    643708a View commit details
    Browse the repository at this point in the history

  5. replacing interface types

    @bnsblue
    bnsblue committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    fcb3dee View commit details
    Browse the repository at this point in the history

  6. change

    Yee Hing Tong committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    669ff39 View commit details
    Browse the repository at this point in the history

  7. fixed training job unit test

    @bnsblue
    bnsblue committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    186e977 View commit details
    Browse the repository at this point in the history

  8. fix hpo job task interface and hide task type from users

    @bnsblue
    bnsblue committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    f65d97f View commit details
    Browse the repository at this point in the history

  9. Merge branch 'add-sagemaker-trainingjob-hpojob' of github.com:lyft/fl… 

    …ytekit into add-sagemaker-trainingjob-hpojob
    @bnsblue
    bnsblue committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    babefc1 View commit details
    Browse the repository at this point in the history

  10. fix hpo job task interface

    @bnsblue
    bnsblue committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    f0d37cd View commit details
    Browse the repository at this point in the history

  11. fix hpo models

    @bnsblue
    bnsblue committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    20b9809 View commit details
    Browse the repository at this point in the history

  12. fix serialization of the underlying trainingjob of a hpo job

    @bnsblue
    bnsblue committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    008cf38 View commit details
    Browse the repository at this point in the history

  13. Expose training job as a parameter

    Ketan Umare committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    2f274d6 View commit details
    Browse the repository at this point in the history

  14. Working!

    Ketan Umare committed Jun 25, 2020
    Configuration menu
    Copy the full SHA
    ac15490 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2020

  1. replacing hyphens with underscores

    @bnsblue
    bnsblue committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    a7d4cdd View commit details
    Browse the repository at this point in the history

  2. updated

    Ketan Umare committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    43a42a0 View commit details
    Browse the repository at this point in the history

  3. bug fix

    Ketan Umare committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    5204e9d View commit details
    Browse the repository at this point in the history

  4. Sagemaker nb

    Ketan Umare committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    aff9055 View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2020

  1. Sagemaker HPO

    Ketan Umare committed Jun 29, 2020
    Configuration menu
    Copy the full SHA
    4f3a3c1 View commit details
    Browse the repository at this point in the history

  2. remove .demo directory

    Ketan Umare committed Jun 29, 2020
    Configuration menu
    Copy the full SHA
    edfd2a6 View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2020

  1. Merge branch 'master' into add-sagemaker-trainingjob-hpojob

    Ketan Umare committed Jun 30, 2020
    Configuration menu
    Copy the full SHA
    b0bbca2 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into add-sagemaker-trainingjob-hpojob

    Ketan Umare committed Jun 30, 2020
    Configuration menu
    Copy the full SHA
    ba40a9b View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2020

  1. register and launch standalone trainingjob task

    @bnsblue
    bnsblue committed Jul 9, 2020
    Configuration menu
    Copy the full SHA
    345e057 View commit details
    Browse the repository at this point in the history

  2. Merge

    @EngHabu
    EngHabu committed Jul 9, 2020
    Configuration menu
    Copy the full SHA
    9d5f243 View commit details
    Browse the repository at this point in the history

Commits on Jul 14, 2020

  1. Complete the examples in sagemaker-hpo notebook and add text descript… 

    …ions
    @bnsblue
    bnsblue committed Jul 14, 2020
    Configuration menu
    Copy the full SHA
    5ba0483 View commit details
    Browse the repository at this point in the history

  2. update notebook

    @bnsblue
    bnsblue committed Jul 14, 2020
    Configuration menu
    Copy the full SHA
    acab1a9 View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2020

  1. all hands demo notebook added

    @bnsblue
    bnsblue committed Jul 15, 2020
    Configuration menu
    Copy the full SHA
    70a4954 View commit details
    Browse the repository at this point in the history

  2. update a notebook

    @bnsblue
    bnsblue committed Jul 15, 2020
    Configuration menu
    Copy the full SHA
    7f28f26 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2020

  1. update the demo notebook

    @bnsblue
    bnsblue committed Jul 16, 2020
    Configuration menu
    Copy the full SHA
    aaa2056 View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2020

  1. adding unit test for SdkSimpleHPOJobTask

    @bnsblue
    bnsblue committed Jul 21, 2020
    Configuration menu
    Copy the full SHA
    044f7b5 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2020

  1. failing the unit test

    @bnsblue
    bnsblue committed Jul 22, 2020
    Configuration menu
    Copy the full SHA
    e45ccf7 View commit details
    Browse the repository at this point in the history

  2. failing the unit test

    @bnsblue
    bnsblue committed Jul 22, 2020
    Configuration menu
    Copy the full SHA
    1aa3a7c View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2020

  1. wip for custom training job

    @bnsblue
    bnsblue committed Jul 23, 2020
    Configuration menu
    Copy the full SHA
    4e8f923 View commit details
    Browse the repository at this point in the history

  2. Revert "wip for custom training job" 

    This reverts commit 4e8f923.
    @bnsblue
    bnsblue committed Jul 23, 2020
    Configuration menu
    Copy the full SHA
    e82e242 View commit details
    Browse the repository at this point in the history

  3. Revert "failing the unit test" 

    This reverts commit 1aa3a7c.
    @bnsblue
    bnsblue committed Jul 23, 2020
    Configuration menu
    Copy the full SHA
    c725a1d View commit details
    Browse the repository at this point in the history

  4. Revert "failing the unit test" 

    This reverts commit e45ccf7.
    @bnsblue
    bnsblue committed Jul 23, 2020
    Configuration menu
    Copy the full SHA
    dcc33cd View commit details
    Browse the repository at this point in the history

  5. fixing unit tests

    @bnsblue
    bnsblue committed Jul 23, 2020
    Configuration menu
    Copy the full SHA
    9982aa7 View commit details
    Browse the repository at this point in the history

  6. bump minor version

    @bnsblue
    bnsblue committed Jul 23, 2020
    Configuration menu
    Copy the full SHA
    70472ac View commit details
    Browse the repository at this point in the history

  7. preventing installing numpy==1.19.0 which introduces a breaking chang… 

    …e for unit tests
    @bnsblue
    bnsblue committed Jul 23, 2020
    Configuration menu
    Copy the full SHA
    a081557 View commit details
    Browse the repository at this point in the history

  8. fix semver

    @bnsblue
    bnsblue committed Jul 23, 2020
    Configuration menu
    Copy the full SHA
    6f5003f View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2020

  1. Merge

    @EngHabu
    EngHabu committed Jul 24, 2020
    Configuration menu
    Copy the full SHA
    adcc3b5 View commit details
    Browse the repository at this point in the history

  2. make changes corresponding to flyteidl changes (renaming hpo to hyper… 

    …parameter tuning)
    @bnsblue
    bnsblue committed Jul 24, 2020
    Configuration menu
    Copy the full SHA
    a463f3f View commit details
    Browse the repository at this point in the history

  3. bump beta version

    @bnsblue
    bnsblue committed Jul 24, 2020
    Configuration menu
    Copy the full SHA
    58b1d5e View commit details
    Browse the repository at this point in the history

  4. Merge branch 'add-sagemaker-trainingjob-hpojob' of github.com:lyft/fl… 

    …ytekit into add-sagemaker-trainingjob-hpojob
    @bnsblue
    bnsblue committed Jul 24, 2020
    Configuration menu
    Copy the full SHA
    c32045b View commit details
    Browse the repository at this point in the history

  5. Delete config.yaml

    @EngHabu
    EngHabu authored Jul 24, 2020
    Configuration menu
    Copy the full SHA
    6d14fe6 View commit details
    Browse the repository at this point in the history

  6. sagemaker-hpo notebook update

    @bnsblue
    bnsblue committed Jul 24, 2020
    Configuration menu
    Copy the full SHA
    6f29de6 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'add-sagemaker-trainingjob-hpojob' of github.com:lyft/fl… 

    …ytekit into add-sagemaker-trainingjob-hpojob
    @bnsblue
    bnsblue committed Jul 24, 2020
    Configuration menu
    Copy the full SHA
    f659a47 View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2020

  1. make changes to reflect changes in flyteidl

    @bnsblue
    bnsblue committed Jul 27, 2020
    Configuration menu
    Copy the full SHA
    06c254e View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2020

  1. make task name consistent

    @bnsblue
    bnsblue committed Jul 28, 2020
    Configuration menu
    Copy the full SHA
    da9fc56 View commit details
    Browse the repository at this point in the history

  2. add missing properties for hyperparameter models

    @bnsblue
    bnsblue committed Jul 28, 2020
    Configuration menu
    Copy the full SHA
    cd70862 View commit details
    Browse the repository at this point in the history

  3. add missing type hints and remove unused imports

    @bnsblue
    bnsblue committed Jul 28, 2020
    Configuration menu
    Copy the full SHA
    998aac0 View commit details
    Browse the repository at this point in the history

  4. remove unused sdk sagemaker dir

    @bnsblue
    bnsblue committed Jul 28, 2020
    Configuration menu
    Copy the full SHA
    585d5ab View commit details
    Browse the repository at this point in the history

  5. remove unused test file

    @bnsblue
    bnsblue committed Jul 28, 2020
    Configuration menu
    Copy the full SHA
    f77d7f1 View commit details
    Browse the repository at this point in the history

  6. revert numpy semver

    @bnsblue
    bnsblue committed Jul 28, 2020
    Configuration menu
    Copy the full SHA
    5b912bb View commit details
    Browse the repository at this point in the history

  7. removing notebooks

    @bnsblue
    bnsblue committed Jul 28, 2020
    Configuration menu
    Copy the full SHA
    02800df View commit details
    Browse the repository at this point in the history

  8. remove type hints for self because CI is using python 3.6.3 while __f… 

    …uture__.annotations requires python 3.7
    @bnsblue
    bnsblue committed Jul 28, 2020
    Configuration menu
    Copy the full SHA
    16867fb View commit details
    Browse the repository at this point in the history

  9. complete docstrings for hpo job task

    @bnsblue
    bnsblue committed Jul 28, 2020
    Configuration menu
    Copy the full SHA
    7c6461e View commit details
    Browse the repository at this point in the history

  10. merging with master and resolve conflict

    @bnsblue
    bnsblue committed Jul 28, 2020
    Configuration menu
    Copy the full SHA
    5429d3d View commit details
    Browse the repository at this point in the history

  11. fix unit test

    @bnsblue
    bnsblue committed Jul 28, 2020
    Configuration menu
    Copy the full SHA
    e6345ae View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2020

  1. adding input_file_type (wip)

    @bnsblue
    bnsblue committed Jul 29, 2020
    Configuration menu
    Copy the full SHA
    d4008d8 View commit details
    Browse the repository at this point in the history

  2. add input file type support

    @bnsblue
    bnsblue committed Jul 29, 2020
    Configuration menu
    Copy the full SHA
    af64b52 View commit details
    Browse the repository at this point in the history

  3. add docs

    @bnsblue
    bnsblue committed Jul 29, 2020
    Configuration menu
    Copy the full SHA
    363beff View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2020

  1. reflecting the renamed type and field

    @bnsblue
    bnsblue committed Jul 30, 2020
    Configuration menu
    Copy the full SHA
    21f3a5d View commit details
    Browse the repository at this point in the history

  2. reflecting remove of libsvm content type

    @bnsblue
    bnsblue committed Jul 30, 2020
    Configuration menu
    Copy the full SHA
    8b0c1f4 View commit details
    Browse the repository at this point in the history

  3. reflecting remove of libsvm content type

    @bnsblue
    bnsblue committed Jul 30, 2020
    Configuration menu
    Copy the full SHA
    d09f37f View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2020

  1. Give metric_definitions a None as the default value because built-in … 

    …algorithm does not allow custom metrics
    @bnsblue
    bnsblue committed Jul 31, 2020
    Configuration menu
    Copy the full SHA
    76b2f0f View commit details
    Browse the repository at this point in the history

  2. nix a print statement

    @bnsblue
    bnsblue committed Jul 31, 2020
    Configuration menu
    Copy the full SHA
    0eee0b3 View commit details
    Browse the repository at this point in the history

  3. nix custom training job for the current release

    @bnsblue
    bnsblue committed Jul 31, 2020
    Configuration menu
    Copy the full SHA
    84a6972 View commit details
    Browse the repository at this point in the history

  4. rename SdkSimpleTrainingJobTask to SdkBuiltinAlgorithmTrainingJobTask

    @bnsblue
    bnsblue committed Jul 31, 2020
    Configuration menu
    Copy the full SHA
    4cd76d3 View commit details
    Browse the repository at this point in the history

  5. merge with master and bump minor version

    @bnsblue
    bnsblue committed Jul 31, 2020
    Configuration menu
    Copy the full SHA
    050f50c View commit details
    Browse the repository at this point in the history

  6. revert setup.py dependency

    @bnsblue
    bnsblue committed Jul 31, 2020
    Configuration menu
    Copy the full SHA
    62e6af6 View commit details
    Browse the repository at this point in the history

  7. add back existing notebooks

    @bnsblue
    bnsblue committed Jul 31, 2020
    Configuration menu
    Copy the full SHA
    7dbac57 View commit details
    Browse the repository at this point in the history