[Question][Help] how to run pods as Docker-containers?

[davidpelfree]

Hi.

I am using Docker Desktop v2.2.0.3 (42716) on Windows 10.

It works great out of the box!
It creates a single Docker container per cluster, with containerd and other k8s stuff inside.

However, I wonder, is there a way to run pods using Docker instead of containerd which runs inside docker-container?

Rational: I am trying to configure a development machine on a laptop, and enable easy understanding and visualization of the deployment. For example, I develop Java based server app, and there's a DB running as well as part of the stack. Now I want my server app to be available on main Docker as a container (i.e. to appear in Docker Desktop Dashboard) and to expose 2 ports: main web port (8080) and debug port (5005).
It indeed works nicely with Okteto, however they use a lot of "magic" behind the scenes: port forwarding and file sync service. I prefer raw Docker port forwarding and host-volume-mounting.

I tried running k3d with Docker option: k3d create -x '--docker' but got:

F0216 13:51:34.330022       6 server.go:269] failed to run Kubelet: failed to create kubelet: failed to get docker version: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

So I tried running k3s manually like this: docker run --name k3d-k3s-default-server -d -v //var/run/docker.sock:/var/run/docker.sock rancher/k3s:v1.17.2-k3s1 server --docker but got this error from kubelet:

F0216 13:53:44.446664       1 kubelet.go:1413] error configuring root directory: failed to bind-mount /var/lib/kubelet: operation not permitted

I understand the move inside k3s from docker runtime to containerd runtime, and it's a perfect solution for IoT and edge, however, in my use case, which is k3d, there is already docker engine, so I want to use only it, without containerd.

Is my use case clear?
Can you please help and decide if the errors above are bugs? Maybe this use case is related to other existing issues?

Maybe related to #113

Thanks.

[iwilltry42]

Hi there @davidpelfree , thanks for opening this issue.
You have a special use case there... is it not sufficient for you to expose your app via ingress or nodePort/hostPort within k3d/k3s to access it from the outside?

The --docker mode is not officially supported by k3d, however, there have been some efforts to do so in the past.
You can find a docker-compose file to get what you tried already (add the --docker option though) in the k3s repo: https://github.com/rancher/k3s/blob/master/docker-compose.yml .
The error you saw when doing it manually comes from not having the tempfs mounts.

#113 surely is related and I guess we should close this issue in favor of #113.

The cluster creation works with k3d create -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker:/var/lib/docker -x '--docker' and you'll see the pods being created on your local docker daemon, but still the stuff will fail because of several non-existent paths (e.g under /proc).
I guess then it's better to use a k3s-dind image (just like in #113) and mount the docker socket there (that could work if you only want to be able to see your app there).

[iwilltry42]

Closing this in favor of #113