Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Backend][Plugin] Add support for submitting Custom TrainingJob (framework mode and bring-you-own-container mode) to SageMaker from within Flyte #431

Closed
2 of 6 tasks
bnsblue opened this issue Jul 23, 2020 · 1 comment
Closed
2 of 6 tasks

[Backend][Plugin] Add support for submitting Custom TrainingJob (framework mode and bring-you-own-container mode) to SageMaker from within Flyte #431

bnsblue opened this issue Jul 23, 2020 · 1 comment
Labels
enhancement New feature or request propeller Issues related to flyte propeller
Milestone
0.11.0

Comments

@bnsblue
Copy link
Contributor

bnsblue commented Jul 23, 2020

Why would this plugin be helpful to the Flyte community
This enhancement of the plugin aims to allow the users to leverage the SageMaker Training functionality more comprehensively by enabling them to write custom code or even submit custom-built container to SageMaker.

Type of Plugin

  • Web Service (e.g. AWS Sagemaker, GCP DataFlow, Qubole etc...)
  • Kubernetes Operator (e.g. TfOperator, SparkOperator, FlinkK8sOperator, etc...)
  • Customized Plugin using native kubernetes constructs
  • Other

Can you help us with the implementation?

  • Yes
  • No

Additional context
...
@bnsblue bnsblue added the untriaged This issues has not yet been looked at by the Maintainers label Jul 23, 2020
@bnsblue bnsblue mentioned this issue Jul 23, 2020
Merged
8 tasks
@bnsblue bnsblue added enhancement New feature or request propeller Issues related to flyte propeller and removed untriaged This issues has not yet been looked at by the Maintainers labels Jul 24, 2020
This was referenced Jul 24, 2020
Merged
Merged
@kumare3
Copy link
Contributor

kumare3 commented Feb 16, 2021

This is available with Flyte already, marking it to be resovled for 0.11.0

@kumare3 kumare3 added this to the 0.11.0 milestone Feb 16, 2021
@kumare3 kumare3 closed this as completed Feb 16, 2021
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Dec 6, 2022
@pmahindrakar-oss
Store job entry ID for fixed rate scheduler and also fixes cron sched…
32161db 
…ule for missing key (flyteorg#431)

* Store job entry ID for fixed rate scheduler

Signed-off-by: Prafulla Mahindrakar <[email protected]>

* Add kickoff time input arg only if the user set it in the workflow

Signed-off-by: Prafulla Mahindrakar <[email protected]>

* Added unit test for no kick off time arg

Signed-off-by: Prafulla Mahindrakar <[email protected]>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Dec 20, 2022
@SandraGH5
Update task_cache.py (flyteorg#431)
f65d318
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Aug 21, 2023
@pmahindrakar-oss
Store job entry ID for fixed rate scheduler and also fixes cron sched…
4080d3f 
…ule for missing key (flyteorg#431)

* Store job entry ID for fixed rate scheduler

Signed-off-by: Prafulla Mahindrakar <[email protected]>

* Add kickoff time input arg only if the user set it in the workflow

Signed-off-by: Prafulla Mahindrakar <[email protected]>

* Added unit test for no kick off time arg

Signed-off-by: Prafulla Mahindrakar <[email protected]>
eapolinario pushed a commit that referenced this issue Sep 8, 2023
@katrogan
Correct comment for array job max parallelism (#431)
fb13157 
Signed-off-by: Katrina Rogan <[email protected]>
eapolinario pushed a commit that referenced this issue Sep 13, 2023
@katrogan @eapolinario
Correct comment for array job max parallelism (#431)
224c594 
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
eapolinario pushed a commit that referenced this issue Sep 26, 2023
@wild-endeavor @honnix
@pingsutw @katrogan @jeevb @hamersaw
GPU Type (#419)
7bd98a9 
* add field

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Jeev B <[email protected]>

* Pass task execution metadata from agent (#422)

* Pass task execution metadata from agent

Signed-off-by: Hongxin Liang <[email protected]>

* Add doc

Signed-off-by: Hongxin Liang <[email protected]>

* Update protos/flyteidl/admin/agent.proto

Co-authored-by: Kevin Su <[email protected]>
Signed-off-by: Honnix <[email protected]>

* Regenerate

---------

Signed-off-by: Hongxin Liang <[email protected]>
Signed-off-by: Honnix <[email protected]>
Co-authored-by: Kevin Su <[email protected]>
Signed-off-by: Jeev B <[email protected]>

* Add tags to execution spec (#414)

* add tags to execution spec

Signed-off-by: Kevin Su <[email protected]>

* add tags to execution spec

Signed-off-by: Kevin Su <[email protected]>

* add comment

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Jeev B <[email protected]>

* Correct comment for array job max parallelism (#431)

Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Jeev B <[email protected]>

* Add the scalar to the operand (#427)

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Jeev B <[email protected]>

* add selector

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Jeev B <[email protected]>

* move selectors from container to task metadata

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Jeev B <[email protected]>

* drop only_preferred

Signed-off-by: Jeev B <[email protected]>

* Updating boilerplate to lock golangci-lint version (#435)

Signed-off-by: Daniel Rammer <[email protected]>
Signed-off-by: Jeev B <[email protected]>

* add unpartitioned selector

Signed-off-by: Jeev B <[email protected]>

* refactor

Signed-off-by: Jeev B <[email protected]>

* refactor

Signed-off-by: Jeev B <[email protected]>

* fix oneof names

Signed-off-by: Jeev B <[email protected]>

* add build.os for read the docs

Signed-off-by: Jeev B <[email protected]>

---------

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Jeev B <[email protected]>
Signed-off-by: Hongxin Liang <[email protected]>
Signed-off-by: Honnix <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Daniel Rammer <[email protected]>
Co-authored-by: Honnix <[email protected]>
Co-authored-by: Kevin Su <[email protected]>
Co-authored-by: Kevin Su <[email protected]>
Co-authored-by: Katrina Rogan <[email protected]>
Co-authored-by: Jeev B <[email protected]>
Co-authored-by: Dan Rammer <[email protected]>
pvditt pushed a commit that referenced this issue Dec 29, 2023
@katrogan
Correct comment for array job max parallelism (#431)
c1417ee 
Signed-off-by: Katrina Rogan <[email protected]>
pvditt pushed a commit that referenced this issue Dec 29, 2023
@wild-endeavor @honnix
@pingsutw @katrogan @jeevb @hamersaw
GPU Type (#419)
b92acad 
* add field

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Jeev B <[email protected]>

* Pass task execution metadata from agent (#422)

* Pass task execution metadata from agent

Signed-off-by: Hongxin Liang <[email protected]>

* Add doc

Signed-off-by: Hongxin Liang <[email protected]>

* Update protos/flyteidl/admin/agent.proto

Co-authored-by: Kevin Su <[email protected]>
Signed-off-by: Honnix <[email protected]>

* Regenerate

---------

Signed-off-by: Hongxin Liang <[email protected]>
Signed-off-by: Honnix <[email protected]>
Co-authored-by: Kevin Su <[email protected]>
Signed-off-by: Jeev B <[email protected]>

* Add tags to execution spec (#414)

* add tags to execution spec

Signed-off-by: Kevin Su <[email protected]>

* add tags to execution spec

Signed-off-by: Kevin Su <[email protected]>

* add comment

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Jeev B <[email protected]>

* Correct comment for array job max parallelism (#431)

Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Jeev B <[email protected]>

* Add the scalar to the operand (#427)

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Jeev B <[email protected]>

* add selector

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Jeev B <[email protected]>

* move selectors from container to task metadata

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Jeev B <[email protected]>

* drop only_preferred

Signed-off-by: Jeev B <[email protected]>

* Updating boilerplate to lock golangci-lint version (#435)

Signed-off-by: Daniel Rammer <[email protected]>
Signed-off-by: Jeev B <[email protected]>

* add unpartitioned selector

Signed-off-by: Jeev B <[email protected]>

* refactor

Signed-off-by: Jeev B <[email protected]>

* refactor

Signed-off-by: Jeev B <[email protected]>

* fix oneof names

Signed-off-by: Jeev B <[email protected]>

* add build.os for read the docs

Signed-off-by: Jeev B <[email protected]>

---------

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Jeev B <[email protected]>
Signed-off-by: Hongxin Liang <[email protected]>
Signed-off-by: Honnix <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Daniel Rammer <[email protected]>
Co-authored-by: Honnix <[email protected]>
Co-authored-by: Kevin Su <[email protected]>
Co-authored-by: Kevin Su <[email protected]>
Co-authored-by: Katrina Rogan <[email protected]>
Co-authored-by: Jeev B <[email protected]>
Co-authored-by: Dan Rammer <[email protected]>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Apr 30, 2024
@Future-Outlier
force flag (flyteorg#431)
1152c5f 
Signed-off-by: Future Outlier <[email protected]>
Co-authored-by: Future Outlier <[email protected]>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Apr 30, 2024
@Future-Outlier
force flag (flyteorg#431)
7f80f02 
Signed-off-by: Future Outlier <[email protected]>
Co-authored-by: Future Outlier <[email protected]>
austin362667 pushed a commit to austin362667/flyte that referenced this issue May 7, 2024
@Future-Outlier @austin362667
force flag (flyteorg#431)
69cf3c2 
Signed-off-by: Future Outlier <[email protected]>
Co-authored-by: Future Outlier <[email protected]>
robert-ulbrich-mercedes-benz pushed a commit to robert-ulbrich-mercedes-benz/flyte that referenced this issue Jul 2, 2024
@Future-Outlier @robert-ulbrich-mercedes-benz
force flag (flyteorg#431)
f19a880 
Signed-off-by: Future Outlier <[email protected]>
Co-authored-by: Future Outlier <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request propeller Issues related to flyte propeller
Projects
None yet
Milestone
0.11.0
Development

No branches or pull requests
2 participants
@bnsblue @kumare3