Size of gcr.io/kubeflow/tensorflow-notebook-*

[flx42]

From the README:

We also ship standard docker images that you can use for training Tensorflow models with Jupyter.

gcr.io/kubeflow/tensorflow-notebook-cpu
gcr.io/kubeflow/tensorflow-notebook-gpu

[...] Note that GPU-based image is several gigabytes in size and may take a few minutes to localize.

("localize"?)

They are both large:

$ docker images gcr.io/kubeflow/tensorflow-notebook-gpu:latest
REPOSITORY                                TAG                 IMAGE ID            CREATED             SIZE
gcr.io/kubeflow/tensorflow-notebook-gpu   latest              e68d36c67064        2 weeks ago         7.11GB

$ docker images gcr.io/kubeflow/tensorflow-notebook-cpu:latest
REPOSITORY                                TAG                 IMAGE ID            CREATED             SIZE
gcr.io/kubeflow/tensorflow-notebook-cpu   latest              9cb2a6008740        2 weeks ago         5.17GB

Are the Dockerfiles public for these images? I can probably do a quick PR to improve the size.

You might be interested to look at the improvements I did in the devel-gpu Dockerfile for TensorFlow:
tensorflow/tensorflow#15355

Also, it would be helpful if you could chime in on this RFE:
tensorflow/tensorflow#15284
Maybe we can have a single image with Jupyter+TensorFlow+TensorBoard? That would shrink the other TensorFlow images that are shipped today (e.g. gpu and devel-gpu).

[pineking]

I also think the size can be reduced. BTW: is it possible to push the images to dockerhub instead of gcr.io?

[jlewi]

@vishh Are we just using the tensorflow Docker images? I don't see any Dcokerfiles for these notebook images inside google/kubeflow.

[flx42]

No it's not the same, if you do docker history --no-trunc gcr.io/kubeflow/tensorflow-notebook-gpu, you can see it's different from https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/docker/Dockerfile.devel-gpu or https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/docker/Dockerfile.gpu

[jlewi]

We'd like to support a bunch of different frameworks e.g.

• TensorFlow
• xgboost
• scikits

Some questions:

• Should we provide one fat image with all these libraries or have multiple images?
• Are there existing, curated images that we can reuse as opposed to building our own?

[aronchick]

Was there something in particular that was an issue with GCR.io?

…

On Mon, Dec 18, 2017 at 9:21 AM Jeremy Lewi ***@***.***> wrote: We'd like to support a bunch of different frameworks e.g. - TensorFlow - xgboost - scikits Some questions: - Should we provide one fat image with all these libraries or have multiple images? - Are there existing, curated images that we can reuse as opposed to building our own? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#37 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADIdcz5ZT3V0JVJi_GKLtIAalLjMac4ks5tBp8OgaJpZM4RE31c> .

[yuvipanda]

@aronchick afaict gcr.io doesn't provide a human friendly URL to pass to people, which I've always found annoying for public images.

@jlewi there's some at http://github.com/jupyter/docker-stacks/ (and PRs welcome!) that do get a fair amount of usage.

[flx42]

I think it makes sense to have one "fat" image, it it allows us to keep the other images lean.
This image could target being a development environment for data scientists: Jupyter, TensorFlow, TensorBoard and usual python dependencies.

[jlewi]

This is the source for our existing Docker images
https://github.com/GoogleCloudPlatform/container-engine-accelerators/tree/master/example/tensorflow-notebook-image

So everything is public and we should probably move them into Kubeflow.

Using an NVIDIA as the base image for our GPU images makes sense to me.

/cc @flx42

[flx42]

Ok, let's discuss about the size again when it's on this repo.
But I believe it still makes sense to check with the TensorFlow team if a single common Jupyter image can be created.