-
Notifications
You must be signed in to change notification settings - Fork 657
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature] Get*ExecutionData should optionally return data inline #419
Labels
enhancement
New feature or request
Comments
katrogan
added
enhancement
New feature or request
untriaged
This issues has not yet been looked at by the Maintainers
labels
Jul 20, 2020
This was referenced Jul 22, 2020
katrogan
removed
the
untriaged
This issues has not yet been looked at by the Maintainers
label
Aug 12, 2020
This was referenced Aug 19, 2020
eapolinario
pushed a commit
to eapolinario/flyte
that referenced
this issue
Dec 6, 2022
* change label selector op Signed-off-by: Lisa <[email protected]> Signed-off-by: aeioulisa <[email protected]> * lint error Signed-off-by: Lisa <[email protected]> Signed-off-by: aeioulisa <[email protected]> * test error Signed-off-by: Lisa <[email protected]> Signed-off-by: aeioulisa <[email protected]> * revert para name and keep old function Signed-off-by: Lisa <[email protected]> Signed-off-by: aeioulisa <[email protected]> * remove strconv.Itoa Signed-off-by: aeioulisa <[email protected]> * two lebal Signed-off-by: aeioulisa <[email protected]> * two lebal Signed-off-by: aeioulisa <[email protected]> * reset config and add split function Signed-off-by: aeioulisa <[email protected]>
eapolinario
pushed a commit
to eapolinario/flyte
that referenced
this issue
Dec 6, 2022
* add flytescheduler to the release archives Signed-off-by: Haytham Abuelfutuh <[email protected]> * Update flytepropeller version Signed-off-by: Haytham Abuelfutuh <[email protected]>
eapolinario
pushed a commit
to eapolinario/flyte
that referenced
this issue
Dec 20, 2022
Embedding a video to outline the Pytorch training process for single-node single GPUs.
eapolinario
pushed a commit
to eapolinario/flyte
that referenced
this issue
Dec 20, 2022
* feat: webpack config split * chore: import react from cdn * chore: update readmes * chore: add script to start prod build locally Signed-off-by: Carina Ursu <[email protected]>
eapolinario
pushed a commit
to eapolinario/flyte
that referenced
this issue
Dec 20, 2022
# [0.55.0](http://github.com/lyft/flyteconsole/compare/v0.54.2...v0.55.0) (2022-04-22) ### Features * webpack config changes ([flyteorg#419](http://github.com/lyft/flyteconsole/issues/419)) ([f93e94f](http://github.com/lyft/flyteconsole/commit/f93e94fd5b04c8da65b0bcd97ce1097451d83b28))
eapolinario
pushed a commit
to eapolinario/flyte
that referenced
this issue
Aug 9, 2023
* change label selector op Signed-off-by: Lisa <[email protected]> Signed-off-by: aeioulisa <[email protected]> * lint error Signed-off-by: Lisa <[email protected]> Signed-off-by: aeioulisa <[email protected]> * test error Signed-off-by: Lisa <[email protected]> Signed-off-by: aeioulisa <[email protected]> * revert para name and keep old function Signed-off-by: Lisa <[email protected]> Signed-off-by: aeioulisa <[email protected]> * remove strconv.Itoa Signed-off-by: aeioulisa <[email protected]> * two lebal Signed-off-by: aeioulisa <[email protected]> * two lebal Signed-off-by: aeioulisa <[email protected]> * reset config and add split function Signed-off-by: aeioulisa <[email protected]>
eapolinario
pushed a commit
to eapolinario/flyte
that referenced
this issue
Aug 21, 2023
* add flytescheduler to the release archives Signed-off-by: Haytham Abuelfutuh <[email protected]> * Update flytepropeller version Signed-off-by: Haytham Abuelfutuh <[email protected]>
eapolinario
pushed a commit
that referenced
this issue
Sep 26, 2023
* add field Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Jeev B <[email protected]> * Pass task execution metadata from agent (#422) * Pass task execution metadata from agent Signed-off-by: Hongxin Liang <[email protected]> * Add doc Signed-off-by: Hongxin Liang <[email protected]> * Update protos/flyteidl/admin/agent.proto Co-authored-by: Kevin Su <[email protected]> Signed-off-by: Honnix <[email protected]> * Regenerate --------- Signed-off-by: Hongxin Liang <[email protected]> Signed-off-by: Honnix <[email protected]> Co-authored-by: Kevin Su <[email protected]> Signed-off-by: Jeev B <[email protected]> * Add tags to execution spec (#414) * add tags to execution spec Signed-off-by: Kevin Su <[email protected]> * add tags to execution spec Signed-off-by: Kevin Su <[email protected]> * add comment Signed-off-by: Kevin Su <[email protected]> --------- Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Jeev B <[email protected]> * Correct comment for array job max parallelism (#431) Signed-off-by: Katrina Rogan <[email protected]> Signed-off-by: Jeev B <[email protected]> * Add the scalar to the operand (#427) Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Jeev B <[email protected]> * add selector Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Jeev B <[email protected]> * move selectors from container to task metadata Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Jeev B <[email protected]> * drop only_preferred Signed-off-by: Jeev B <[email protected]> * Updating boilerplate to lock golangci-lint version (#435) Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Jeev B <[email protected]> * add unpartitioned selector Signed-off-by: Jeev B <[email protected]> * refactor Signed-off-by: Jeev B <[email protected]> * refactor Signed-off-by: Jeev B <[email protected]> * fix oneof names Signed-off-by: Jeev B <[email protected]> * add build.os for read the docs Signed-off-by: Jeev B <[email protected]> --------- Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Jeev B <[email protected]> Signed-off-by: Hongxin Liang <[email protected]> Signed-off-by: Honnix <[email protected]> Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Katrina Rogan <[email protected]> Signed-off-by: Daniel Rammer <[email protected]> Co-authored-by: Honnix <[email protected]> Co-authored-by: Kevin Su <[email protected]> Co-authored-by: Kevin Su <[email protected]> Co-authored-by: Katrina Rogan <[email protected]> Co-authored-by: Jeev B <[email protected]> Co-authored-by: Dan Rammer <[email protected]>
eapolinario
pushed a commit
that referenced
this issue
Sep 26, 2023
* Revert "GPU Type (#419)" This reverts commit 7bd98a9. Signed-off-by: Jeev B <[email protected]> * Restore .readthedocs.yml Signed-off-by: Jeev B <[email protected]> --------- Signed-off-by: Jeev B <[email protected]>
pvditt
pushed a commit
that referenced
this issue
Dec 29, 2023
* add field Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Jeev B <[email protected]> * Pass task execution metadata from agent (#422) * Pass task execution metadata from agent Signed-off-by: Hongxin Liang <[email protected]> * Add doc Signed-off-by: Hongxin Liang <[email protected]> * Update protos/flyteidl/admin/agent.proto Co-authored-by: Kevin Su <[email protected]> Signed-off-by: Honnix <[email protected]> * Regenerate --------- Signed-off-by: Hongxin Liang <[email protected]> Signed-off-by: Honnix <[email protected]> Co-authored-by: Kevin Su <[email protected]> Signed-off-by: Jeev B <[email protected]> * Add tags to execution spec (#414) * add tags to execution spec Signed-off-by: Kevin Su <[email protected]> * add tags to execution spec Signed-off-by: Kevin Su <[email protected]> * add comment Signed-off-by: Kevin Su <[email protected]> --------- Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Jeev B <[email protected]> * Correct comment for array job max parallelism (#431) Signed-off-by: Katrina Rogan <[email protected]> Signed-off-by: Jeev B <[email protected]> * Add the scalar to the operand (#427) Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Jeev B <[email protected]> * add selector Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Jeev B <[email protected]> * move selectors from container to task metadata Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Jeev B <[email protected]> * drop only_preferred Signed-off-by: Jeev B <[email protected]> * Updating boilerplate to lock golangci-lint version (#435) Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Jeev B <[email protected]> * add unpartitioned selector Signed-off-by: Jeev B <[email protected]> * refactor Signed-off-by: Jeev B <[email protected]> * refactor Signed-off-by: Jeev B <[email protected]> * fix oneof names Signed-off-by: Jeev B <[email protected]> * add build.os for read the docs Signed-off-by: Jeev B <[email protected]> --------- Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Jeev B <[email protected]> Signed-off-by: Hongxin Liang <[email protected]> Signed-off-by: Honnix <[email protected]> Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Katrina Rogan <[email protected]> Signed-off-by: Daniel Rammer <[email protected]> Co-authored-by: Honnix <[email protected]> Co-authored-by: Kevin Su <[email protected]> Co-authored-by: Kevin Su <[email protected]> Co-authored-by: Katrina Rogan <[email protected]> Co-authored-by: Jeev B <[email protected]> Co-authored-by: Dan Rammer <[email protected]>
pvditt
pushed a commit
that referenced
this issue
Dec 29, 2023
* Revert "GPU Type (#419)" This reverts commit b92acad. Signed-off-by: Jeev B <[email protected]> * Restore .readthedocs.yml Signed-off-by: Jeev B <[email protected]> --------- Signed-off-by: Jeev B <[email protected]>
eapolinario
pushed a commit
to eapolinario/flyte
that referenced
this issue
Apr 30, 2024
Signed-off-by: Future Outlier <[email protected]> Co-authored-by: Future Outlier <[email protected]>
eapolinario
pushed a commit
to eapolinario/flyte
that referenced
this issue
Apr 30, 2024
…rg#423) * WIP. Marked places where an acknowledgement before an update is needed. Signed-off-by: Kamal Eybov <[email protected]> * (1) Added error handling for methods fetching matchable attributes when attributes do not exist. (2) Added fetching data that is needed for diffing during updates. Signed-off-by: Kamal Eybov <[email protected]> * Diff and ask for ack. Signed-off-by: Kamal Eybov <[email protected]> * Fixed some of the TODOs. Signed-off-by: Kamal Eybov <[email protected]> * Cleaned up error handling. Signed-off-by: Kamal Eybov <[email protected]> * Updated tests. Signed-off-by: Kamal Eybov <[email protected]> * More tests. Signed-off-by: Kamal Eybov <[email protected]> * Fix case for No string to no (flyteorg#419) Signed-off-by: Future Outlier <[email protected]> Co-authored-by: Future Outlier <[email protected]> Signed-off-by: Kamal Eybov <[email protected]> * Replaced diffing implementation. Signed-off-by: Kamal Eybov <[email protected]> * Addressed pull request comments. Signed-off-by: Kamal Eybov <[email protected]> * Fixed linter errors. Signed-off-by: Kamal Eybov <[email protected]> --------- Signed-off-by: Kamal Eybov <[email protected]> Signed-off-by: Future Outlier <[email protected]> Co-authored-by: Future-Outlier <[email protected]> Co-authored-by: Future Outlier <[email protected]>
eapolinario
pushed a commit
to eapolinario/flyte
that referenced
this issue
Apr 30, 2024
Signed-off-by: Future Outlier <[email protected]> Co-authored-by: Future Outlier <[email protected]>
eapolinario
pushed a commit
to eapolinario/flyte
that referenced
this issue
Apr 30, 2024
…rg#423) * WIP. Marked places where an acknowledgement before an update is needed. Signed-off-by: Kamal Eybov <[email protected]> * (1) Added error handling for methods fetching matchable attributes when attributes do not exist. (2) Added fetching data that is needed for diffing during updates. Signed-off-by: Kamal Eybov <[email protected]> * Diff and ask for ack. Signed-off-by: Kamal Eybov <[email protected]> * Fixed some of the TODOs. Signed-off-by: Kamal Eybov <[email protected]> * Cleaned up error handling. Signed-off-by: Kamal Eybov <[email protected]> * Updated tests. Signed-off-by: Kamal Eybov <[email protected]> * More tests. Signed-off-by: Kamal Eybov <[email protected]> * Fix case for No string to no (flyteorg#419) Signed-off-by: Future Outlier <[email protected]> Co-authored-by: Future Outlier <[email protected]> Signed-off-by: Kamal Eybov <[email protected]> * Replaced diffing implementation. Signed-off-by: Kamal Eybov <[email protected]> * Addressed pull request comments. Signed-off-by: Kamal Eybov <[email protected]> * Fixed linter errors. Signed-off-by: Kamal Eybov <[email protected]> --------- Signed-off-by: Kamal Eybov <[email protected]> Signed-off-by: Future Outlier <[email protected]> Co-authored-by: Future-Outlier <[email protected]> Co-authored-by: Future Outlier <[email protected]>
austin362667
pushed a commit
to austin362667/flyte
that referenced
this issue
May 7, 2024
Signed-off-by: Future Outlier <[email protected]> Co-authored-by: Future Outlier <[email protected]>
austin362667
pushed a commit
to austin362667/flyte
that referenced
this issue
May 7, 2024
…rg#423) * WIP. Marked places where an acknowledgement before an update is needed. Signed-off-by: Kamal Eybov <[email protected]> * (1) Added error handling for methods fetching matchable attributes when attributes do not exist. (2) Added fetching data that is needed for diffing during updates. Signed-off-by: Kamal Eybov <[email protected]> * Diff and ask for ack. Signed-off-by: Kamal Eybov <[email protected]> * Fixed some of the TODOs. Signed-off-by: Kamal Eybov <[email protected]> * Cleaned up error handling. Signed-off-by: Kamal Eybov <[email protected]> * Updated tests. Signed-off-by: Kamal Eybov <[email protected]> * More tests. Signed-off-by: Kamal Eybov <[email protected]> * Fix case for No string to no (flyteorg#419) Signed-off-by: Future Outlier <[email protected]> Co-authored-by: Future Outlier <[email protected]> Signed-off-by: Kamal Eybov <[email protected]> * Replaced diffing implementation. Signed-off-by: Kamal Eybov <[email protected]> * Addressed pull request comments. Signed-off-by: Kamal Eybov <[email protected]> * Fixed linter errors. Signed-off-by: Kamal Eybov <[email protected]> --------- Signed-off-by: Kamal Eybov <[email protected]> Signed-off-by: Future Outlier <[email protected]> Co-authored-by: Future-Outlier <[email protected]> Co-authored-by: Future Outlier <[email protected]>
robert-ulbrich-mercedes-benz
pushed a commit
to robert-ulbrich-mercedes-benz/flyte
that referenced
this issue
Jul 2, 2024
Signed-off-by: Future Outlier <[email protected]> Co-authored-by: Future Outlier <[email protected]>
robert-ulbrich-mercedes-benz
pushed a commit
to robert-ulbrich-mercedes-benz/flyte
that referenced
this issue
Jul 2, 2024
…rg#423) * WIP. Marked places where an acknowledgement before an update is needed. Signed-off-by: Kamal Eybov <[email protected]> * (1) Added error handling for methods fetching matchable attributes when attributes do not exist. (2) Added fetching data that is needed for diffing during updates. Signed-off-by: Kamal Eybov <[email protected]> * Diff and ask for ack. Signed-off-by: Kamal Eybov <[email protected]> * Fixed some of the TODOs. Signed-off-by: Kamal Eybov <[email protected]> * Cleaned up error handling. Signed-off-by: Kamal Eybov <[email protected]> * Updated tests. Signed-off-by: Kamal Eybov <[email protected]> * More tests. Signed-off-by: Kamal Eybov <[email protected]> * Fix case for No string to no (flyteorg#419) Signed-off-by: Future Outlier <[email protected]> Co-authored-by: Future Outlier <[email protected]> Signed-off-by: Kamal Eybov <[email protected]> * Replaced diffing implementation. Signed-off-by: Kamal Eybov <[email protected]> * Addressed pull request comments. Signed-off-by: Kamal Eybov <[email protected]> * Fixed linter errors. Signed-off-by: Kamal Eybov <[email protected]> --------- Signed-off-by: Kamal Eybov <[email protected]> Signed-off-by: Future Outlier <[email protected]> Co-authored-by: Future-Outlier <[email protected]> Co-authored-by: Future Outlier <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Motivation: Why do you think this is important?
The current model of fetching signed URL references to execution data [inputs + outputs] works fine in a hosted AWS environment but doesn't actually successfully return a usable link in local/sandbox mode. This is because "the url returned to flytekit is a presigned url, with the host name which is kubernetes cluster local - like - http://minio.flyte.svc.cluster.local:9000" (@kumare3) and makes for a frustrating and broken experience when standing up flyte locally.
Goal: What should the final outcome look like, ideally?
Ideally the Get*ExecutionData endpoints [Workflow, Node, Task] should allow callers to specify whether they want populated inputs & outputs in addition to signed URL blobs pointing to where the same data is stored.
The endpoints should remain separate from Get*Execution as is, but offer more customizability.
Describe alternatives you've considered
Oneof responses that include either the URL blob or the populated inputs. However URL blobs are cheap to generate and lightweight to pass around so we can keep them as is.
Optionally the inputs & outputs can be returned inline in the Get*Execution responses but this risks bloating them.
Flyte component
[Optional] Propose: Link/Inline
Additional context
First raised by @kumare3
Is this a blocker for you to adopt Flyte
N/A
The text was updated successfully, but these errors were encountered: