Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kubeflow 1.4 Upgrade: CWA decision with Jupyter-api #1302

Closed
wg102 opened this issue Aug 10, 2022 · 3 comments
Closed

Kubeflow 1.4 Upgrade: CWA decision with Jupyter-api #1302

wg102 opened this issue Aug 10, 2022 · 3 comments
Assignees
Labels
size/S ~1 day

Comments

@wg102
Copy link
Contributor

wg102 commented Aug 10, 2022

This ticket is to track the review and decision if we keep 2 repos, merge into the kubeflow repo, or do another decision abotu Jupyter-apis.

Here are some of the informations:

List of all the commits we have on Jupyter-apis:

#1242

Why we made the decision like that in the past:

too much work? lack of GO knowledge?
Zach / Will just did it in a night? (no idea the process they used)
while we were floundering around...

How What was it done last time:

Unfortunately there is not steps as to HOW is was done.
front end: Will StatCan/jupyter-apis@dfa205f
backend: Will and zach StatCan/jupyter-apis@863a996

Why we want to change it.

  • The Backend needs to stay ours, so at most we would either create a new folder for our backend,
    or keep it in a separate repository.
  • It would be easier to get the front end changes? Currently we just don't have what they do?

What it would impact or not.

  • WE have golang they have python, everytime we need update we need their shit, and Idk how. Same if we ever want to upstream.
    This has nothing to do with the argument, could just have a backend-go folder.
  • Frontend. It hsould be the same, but we customized it a lot. Including our own volume table. Ignoring tensorboard etc.
    2 repos,
    It's hard do a rebase or a compare
    Having 1 repo would make it easier, but then again, our customizing is harder every time to merge.

Unless we want to push upstream might be not worth it.

It might be a 'do we want to upstream' choice?

@wg102 wg102 mentioned this issue Aug 10, 2022
14 tasks
@wg102 wg102 self-assigned this Aug 10, 2022
@wg102 wg102 added the size/S ~1 day label Aug 10, 2022
@wg102
Copy link
Contributor Author

wg102 commented Aug 16, 2022

We discussed this issue at the elaboration session on August 11th 2022.

Many things were talked about, including why we have originally gone from Python to Golang, and if wanting the custom backend code was still the case. This was related to previous investigations into tickets, and the python language still has the issue of efficacity when handling the amount of profiles that we do.

The decision was to keep it in two repositories, but to make sure to document the HOW to upgrade the jupyter-api repo by reaching out to Will and possibly Zach and know how they managed to create the previous PRs.

If we do want to upstream the golang code, it will have to be in back into the main kubeflow repo.

We have a LOT of customization on the jupyter-api, including but not limited to:

  • Validation messages for our own concerns
  • Hardcoded elements
  • Volume table that we decided to keep ours since we would need to disable multiple elements of theirs
  • Kubecost as part of jupyter-api

We also discussed about Tensorboard, since it's a new element of CWA. But it was said that maybe the user could install it on their own if need be, instead of being a platform thing.

TL/DN: We will not be merging the repos anytime soon and we will add to the jupyter-api readme how to rebase the code.

@Jose-Matsuda
Copy link
Contributor

Jose-Matsuda commented Aug 16, 2022

Yup at that elaboration we came to a consensus that keeping them separate was fine.

Something I raised up was that I don't believe we should be attempting to pull jupyter-apis into kubeflow unless we want to make a concentrated effort to get that go backend upstream. At least with jupyter-apis in my head then I can easily see a clear separation.

As stated, we have a lot of customizations in our frontend that regardless if we move everything to kubeflow or keep it in jupyter-apis there is bit of a headache each time we need to re-base.
(will edit this comment if any more details arise)

@wg102
Copy link
Contributor Author

wg102 commented Aug 25, 2022

Note: forgot to mention but we also talked that Tensorboard might be not needed. And that users could install it themselves if need be.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/S ~1 day
Projects
None yet
Development

No branches or pull requests

2 participants