Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MinIO & MLFlow configuration to notebooks in form of configuration #58

Open
Barteus opened this issue Feb 9, 2022 · 2 comments
Open
Labels
enhancement New feature or request

Comments

@Barteus
Copy link

Barteus commented Feb 9, 2022

We need a way to pass the MinIO & MLFlow configurations to the notebooks (2 separate configuration values in dropdown). Best if similar to the KFP configuration. #49

Expected
Minio configuration in notebook creation screen section Configurations. Additionally added to the documentation.
image

Workaround 1
Add cell in notebook:

os.environ['MLFLOW_TRACKING_URI'] = "http://mlflow-server.kubeflow.svc.cluster.local:5000"
os.environ["MLFLOW_S3_ENDPOINT_URL"] = "http://minio.kubeflow.svc.cluster.local:9000"
os.environ["AWS_ACCESS_KEY_ID"] = "minio"
os.environ["AWS_SECRET_ACCESS_KEY"] = "minio123"

Workaround 2
Add Pod Defaults with the above configuration to the user namespace, which will be shown in the jupyter notebooks creation screen.

@ca-scribner ca-scribner added the enhancement New feature or request label Feb 15, 2022
@ca-scribner
Copy link
Contributor

If we solve this via PodDefault, this depends on us having a way to inject those PodDefault objects into user namespaces (see jira tasks KF-220/KF-228)

@ca-scribner
Copy link
Contributor

A potential issue with this is that if the PodDefaults directly inject credentials, those credentials would be in the pod's spec as environment variable literals (eg: you can read the environment variable's content from kubectl describe pod ___). To obscure this, I think we'd need PodDefaults that inject environment variables as secret references (similar to how we pass creds to our workload pods like in Minio), and then we'd also need a way of injecting Secrets with the actual creds into each namespace.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants