Support ARM/multi-arch images for Kubeapps containers

[obeyler]

Does a version of Kubeapps already exist for arm processor ?
I've done a Kubernetes cluster with raspberry pi, and as I think kubeapps is a wonderfull product I want to put it on my personal cluster :-)
Thanks
Olivier

[prydonius]

Hey @obeyler, we aren't yet building images for ARM, but given Kubeapps is mostly Go I don't think this would be too difficult. It's a nice idea, I can't say it's high on our priority list, though. If you would like to help port Kubeapps to ARM, that would be really amazing!

[arnaubennassar]

@obeyler I will give it a try!

[arapulido]

@arnaubennassar were you able to give this a try?

[arnaubennassar]

@arapulido I started but then I got some new components for my cluster and I've been focused on changing the network and storage. Once I've the cluster working again I will keep trying.

So far I was trying to build a multi arch, multi stage docker image to be able to build the containers for any architecture and being able to do so entirely with docker. Something similar to this. I tried to build the containers directly on the Raspberry as described in the documentation, but the process didn't finish apparently due to not having enough memory.

My goal is to be able to do some CI/CD magic to keep updated versions for ARM. But pleas keep in mind that I'm learning kubernetes and CI/CD tech in my free time, so I don't expect to have this any time soon.

If you think there is a better approach for this, or better suited for the flow of the project let me know and I will try to do my best to make this land in the future to this repo :D

[prydonius]

@arnaubennassar thanks for the update, that's completely fine and we appreciate the efforts! I don't really have any experience with building multi-arch or ARM containers, so will happily defer to you on that ;). If there is any way we can help with this, though, let us know.

Happy to keep this issue open in the meantime.

[fr34k8]

@prydonius @arnaubennassar @obeyler
is there any progress here ?

[andresmgot]

Hi @fr34k8, not really. As @prydonius pointed out we don't have experience building ARM containers so we are accepting community contributions here :)

[arnaubennassar]

Nope sorry, I no longer host an ARM cluster, so I won't work on this

[fr34k8]

@andresmgot @arnaubennassar did anybody of you start any code to achieve the goal ?

[andresmgot]

not in my side

[arnaubennassar]

Not really. If you want to start, this may be helpful: https://docs.docker.com/buildx/working-with-buildx/#build-multi-platform-images There's good chances that the images used in this project will cross compile using buildx.

[gabriel-duque]

hey ! Any news on this ? I host a k3s cluster on some armv8 boards and would love to deploy kubeapps to ease it's management! I can try to work on this using buildx as @arnaubennassar suggested !

[batiati]

With the new multicluster feature you can deploy kubeapps on top of a regular x86 cluster and manage your k3s arm cluster from there.

[gabriel-duque]

Thanks for the info, I didn't know I could do that. Still, I would like to deploy kubeapps on the k3s cluster itself :)

[absoludity]

We haven't yet prioritized building arm/multi-arch images, sorry. As @batiati mentioned, the only current way would be to install Kubeapps on a regular cluster supporting multiple clusters (but that work is not yet complete - you can see how it will work in https://www.youtube.com/watch?v=KIoW4zZDtdY ).

[gabriel-duque]

@absoludity thanks for the video. I looked into this and it seems like this would not be a lot of work but sort of a hassle because the deploying kubeapps using helm on arm boards would imply building a bunch of bitnami's Docker images for arm too.

[fr34k8]

@zuh0 if I can help in any way, let me know.

[absoludity]

@absoludity thanks for the video. I looked into this and it seems like this would not be a lot of work but sort of a hassle because the deploying kubeapps using helm on arm boards would imply building a bunch of bitnami's Docker images for arm too.

The idea mentioned above was that you wouldn't need to deploy kubeapps on arm boards at all - it'd be running on a different x86 cluster, but able to deploy to other clusters, including your arm one. In the video demo, there are no Kubeapps components installed on the additional cluster. The only requirement is that the additional cluster's k8s api server be setup with oidc pointing at dex running on the initial cluster. But yes, I can't say whether it's worth trying out (and can say multicluster support won't be available for a couple of weeks yet).

[gabriel-duque]

@absoludity I understood what you were saying.

But I would like to avoid setting up a x86 cluster. I am self-hosting my cluster using k3s on some raspberry pi and the whole point was to run everything on armv8.

[gabriel-duque]

@fr34k8 In order for this to be possible, here are the Docker images which would need to be built for armv8:

• kubeapps/apprepository-controller
• kubeapps/assetsvc
• kubeapps/asset-syncer
• kubeapps/dashboard
• kubeapps/kubeops
• kubeapps/tiller-proxy (for people using Helm 2)
• bitnami/kubectl
• bitnami/nginx
• bitnami/oauth2-proxy
• bitnami/node (used by some of kubeapps Dockerfiles)

I will try to open issues on the bitnami repos to build multi-architecture images. If the maintainers seem positive about this I will try to do this but from what I saw on the kubeapps images, it should be rather straightforward as the Dockerfiles use multi-architecture base images like golang or ubuntu.

The complicated work is going to be having a multi-architecture bitnami/minideb image which is a dependency of bitnami/node and others.

[absoludity]

@absoludity I understood what you were saying.

But I would like to avoid setting up a x86 cluster. I am self-hosting my cluster using k3s on some raspberry pi and the whole point was to run everything on armv8.

Sorry, I understood "this" in

@absoludity thanks for the video. I looked into this and it seems like this would not be a lot of work but sort of a hassle because the deploying kubeapps using helm on arm boards would imply building a bunch of bitnami's Docker images for arm too.

to be referring to the multicluster setup, but I see you meant actually doing the arm setup :)

[amoravek]

I think the ARM(v8) support would be really helpful since the architecture support is growing globally. Our company develops small Kubernetes clusters based on the ARM64 platform (see http://www.edgeclusters.com/ if curious) - and we would really appreciate the possibility to run kubeapps by simply using helm install... I've tried to build it from sources recently, but find it too difficult to use since it depends on too many not-arm-enabled images. I hope it will change soon. Is there any progress?

[andresmgot]

hi @amoravek, not that I am aware of. I am not sure if @zuh0 got more information from the bitnami team regarding ARM since they are the ones that at the end of the day, build all the containers, including our dependencies.

[amoravek]

If offer my help with building and testing.

[andresmgot]

The Bitnami team plans to officially support it (for the Bitnami open source catalog, including Kubeapps) but there is no ETA yet.

[ddelange]

@andresmgot could you link a reference to that announcement? I'm currently browsing for a quick-fix to run bitnami-docker-owncloud on arm64, and I'm considering resorting to the slightly outdated https://hub.docker.com/r/thenoots/arm-bitnami-minideb (closest I got to finding the corresponding repo for that image was https://github.com/radicand/minideb-extras-base-arm/commit/9ef34b94d57c4b1375a541419d2ff76a4a36ef37).

[andresmgot]

It's still early stages so there is no link I can send you yet (that image in the link you posted is not related to Bitnami).

A related question, in which platform are you interested? Raspberries, cloud ARM instances, the new Mac?

[ddelange]

I see, all good. Yes the image is of unknown source, so not very keen on using it.

Raspberries, cloud ARM instances, the new Mac?

You picked three very good reasons to prioritise arm bitnami images! For me Raspberry is the one at the moment, but the cheaper AWS ARM instances (and in a few years the ARM Mac...) are definitely on my radar as well!

[ddelange]

Just found a draft PR to add multi arch support from two months ago:

bitnami/minideb#90

Doesn't seem like the author intends to continue the work on making it run in CI, but its a start :)

[ppbaena]

This issue had no activity for a long time; in order to focus on the current roadmap, we are closing this issue. Should anyone be interested in contributing or have any further questions, please reopen it.

Particularly, this ARM issue is being tracked at bitnami/charts#7305. Note that our official images are being built by Bitnami; therefore, any question regarding the Helm chart or the container images should be asked in the Bitnami repository.

Thanks for your comprehension

[samip5]

Just to add to this that all bitnami images are to be expected to not have multi-arch support anytime soon, so the best option is to migrate away from their images.

[ppbaena]

Reopen to continue work started on #5208

[antgamdia]

It seems our colleagues from Bitnami have added support for ARM images! More information here.

EDIT: I've been told there are some temporary issues when releasing some of the Kubeapps assets (eg. pinniped-proxy), if you find some issues when using the Kubeapps chart in ARM, it probably is because of theses issues. Sorry for the inconvenience!