How to access docker api on a mac #11397

tibcoplord · 2021-09-01T18:13:12Z

/kind feature

Description

I've been following https://github.com/containers/podman/blob/main/docs/tutorials/mac_experimental.md on my mac, specifically -

podman machine init --cpus 6 --disk-size 200
podman machine start
podman build ...
podman run ...

Actually works pretty well :-) However next I want to use a docker client ( specifically the fabric8 maven plugin to build images, but we can test with docker command ), but I see -

$ docker ps
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

I read at https://github.com/containers/podman/blob/main/docs/source/markdown/podman-system-service.1.md that there is an option to provide the API service, but this fails with -

$ podman system service
Error: unrecognized command `podman system service`
Try 'podman system --help' for more information.

Steps to reproduce the issue:

On a mac with podman machine running, run podman system service

Describe the results you received:

Error: unrecognized command podman system service

Describe the results you expected:

Able to connect to the socket from a docker client (outside the machine)

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

$ podman version
Client:
Version:      3.3.0
API Version:  3.3.0
Go Version:   go1.16.6
Built:        Fri Aug 20 18:47:23 2021
OS/Arch:      darwin/amd64

Server:
Version:      3.3.0
API Version:  3.3.0
Go Version:   go1.16.6
Built:        Fri Aug 20 20:36:14 2021
OS/Arch:      linux/amd64

Output of podman info --debug:

$ podman info --debug
host:
  arch: amd64
  buildahVersion: 1.22.3
  cgroupControllers: []
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.29-2.fc34.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.29, commit: '
  cpus: 6
  distribution:
    distribution: fedora
    version: "34"
  eventLogger: journald
  hostname: localhost
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.13.12-200.fc34.x86_64
  linkmode: dynamic
  memFree: 2224381952
  memTotal: 25198764032
  ociRuntime:
    name: crun
    package: crun-0.21-1.fc34.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 0.21
      commit: c4c3cdf2ce408ed44a9e027c618473e6485c635b
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.9-1.fc34.x86_64
    version: |-
      slirp4netns version 1.1.8+dev
      commit: 6dc0186e020232ae1a6fcc1f7afbc3ea02fd3876
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.0
  swapFree: 0
  swapTotal: 0
  uptime: 4h 50m 50.8s (Approximately 0.17 days)
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 1
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/core/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 18
  runRoot: /run/user/1000/containers
  volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
  APIVersion: 3.3.0
  Built: 1629488174
  BuiltTime: Fri Aug 20 19:36:14 2021
  GitCommit: ""
  GoVersion: go1.16.6
  OsArch: linux/amd64
  Version: 3.3.0

Additional environment details (AWS, VirtualBox, physical, etc.):

mac client with podman machine running.

I did wonder if the technique I need is to somehow tunnel the socket running inside the podman machine to outside ... but I couldn't find this in the docs.

The text was updated successfully, but these errors were encountered:

mheon · 2021-09-01T18:47:59Z

@baude @jwhonce PTAL

rhatdan · 2021-09-01T19:12:55Z

I think you need to specify DOCKER_HOST environment variable for this, but this is where my knowledge ends.

matejvasek · 2021-09-01T19:16:19Z

As Dan mentioned export DOCKER_HOST="/run/user/1000/podman/podman.sock" should do the trick. It works for instance with docker CLI and official Go client library.

matejvasek · 2021-09-01T19:18:55Z

However it still my happen that some application wrongly assume that socket is always /var/run/docker.sock.

tibcoplord · 2021-09-01T19:19:23Z

On the mac I see -

$ podman ps
CONTAINER ID  IMAGE                    COMMAND               CREATED      STATUS          PORTS       NAMES
5b8fccfa9106  localhost/jenkins:1.0.0  ./start-jenkins.s...  5 hours ago  Up 5 hours ago              jenkins

$ DOCKER_HOST="/run/user/1000/podman/podman.sock" docker ps
Cannot connect to the Docker daemon at tcp://localhost:2375/run/user/1000/podman/podman.sock. Is the docker daemon running?

Indeed, /run/user/1000/podman/podman.sock doesn't exist on the mac, only inside the podman machine.

matejvasek · 2021-09-01T19:20:32Z

oh my bad, you need to specify protocol too export DOCKER_HOST="unix:///run/user/1000/podman/podman.sock".

tibcoplord · 2021-09-01T19:22:21Z

$ DOCKER_HOST="unix:///run/user/1000/podman/podman.sock" docker ps
Cannot connect to the Docker daemon at unix:///run/user/1000/podman/podman.sock. Is the docker daemon running?

I'm not seeing how the socket would work if its inside the podman machine.

baude · 2021-09-01T19:26:14Z

it can be made to work i think, the problem is a little more sophisticated than the network connection I am afraid. i'm looking into casually, but the first issue is the presence of moby on the os.

tibcoplord · 2021-09-01T19:27:41Z

Many thanks :-)

matejvasek · 2021-09-01T19:30:15Z

@baude isn't podman on mac actually podman-remote?

afbjorklund · 2021-09-01T19:31:11Z

We ended up symlinking inside the VM, since the location of the unix socket is hardcoded (when using ssh: address) :

export DOCKER_HOST=$CONTAINER_HOST

ssh host connection is not valid: extra path after the host: "/run/podman/podman.sock"

So had to remove the /run/podman/podman.sock from the URL, and then link /var/run/docker.sock to it (in the VM)

To work in CoreOS, you first have to systemctl stop docker.socket. And then remove the old docker.sock.

$ podman machine ssh
Connecting to vm podman-machine-default. To close connection, use `~.` or `exit`
Fedora CoreOS 34.20210821.1.1
Tracker: https://github.com/coreos/fedora-coreos-tracker
Discuss: https://discussion.fedoraproject.org/c/server/coreos/

[core@localhost ~]$ sudo systemctl stop docker.socket
[core@localhost ~]$ sudo rm /var/run/docker.sock
[core@localhost ~]$ sudo ln -s /run/podman/podman.sock /var/run/docker.sock
[core@localhost ~]$ sudo docker version
Client:
 Version:           20.10.7
 API version:       1.40
 Go version:        go1.16.5
 Git commit:        f0df350
 Built:             Fri Jul 16 12:30:15 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: linux/amd64/fedora-34
 Podman Engine:
  Version:          3.3.0
  APIVersion:       3.3.0
  Arch:             amd64
  BuildTime:        2021-08-20T19:36:14Z
  Experimental:     true
  GitCommit:        
  GoVersion:        go1.16.6
  KernelVersion:    5.13.12-200.fc34.x86_64
  MinAPIVersion:    3.1.0
  Os:               linux
 Engine:
  Version:          3.3.0
  API version:      1.40 (minimum version 1.24)
  Go version:       go1.16.6
  Git commit:       
  Built:            Fri Aug 20 19:36:14 2021
  OS/Arch:          linux/amd64
  Experimental:     true
[core@localhost ~]$ exit

Now it will work with the URL from podman system connection ls, as long as the path is removed.

$ podman system connection ls
Name                         Identity                                  URI
podman-machine-default*      /home/anders/.ssh/podman-machine-default  ssh://core@localhost:45271/run/user/1000/podman/podman.sock
podman-machine-default-root  /home/anders/.ssh/podman-machine-default  ssh://root@localhost:45271/run/podman/podman.sock

CONTAINER_HOST=ssh://root@localhost:45271/run/podman/podman.sock podman --remote version
DOCKER_HOST=ssh://root@localhost:45271 docker version

And similar for the rootless socket. (located in the VM at /run/user/1000/podman/podman.sock)

EDIT: You also need to ssh-add the identity, since there is no CONTAINER_SSHKEY in Docker...

matejvasek · 2021-09-01T19:32:28Z

ssh host connection is not valid: extra path after the host: "/run/podman/podman.sock"

yeah docker doesn't support unix socket over ssh it seems, but I can be worked around using ssh tunnel

afbjorklund · 2021-09-01T19:33:46Z

It does support the socket, it just hardcodes the location. So I guess it doesn't support giving a path.

And apparently CoreOS still ships with Docker, so it will actually start the real dockerd instead...

matejvasek · 2021-09-01T19:35:25Z

what I mean you can have something like ssh://user@host as DOCKER_HOST but not something like ssh://use@host/some/path.sock

matejvasek · 2021-09-01T19:37:28Z

ssh -nNT -L/tmp/podman.sock:/run/user/1000/podman/podman.sock ssh://user@host
or
ssh -nNT -L/run/user/1000/podman/podman.sock:/tmp/podman.sock ssh://user@host
not sure which one, probably the first one, I always mix it up.
Then you can set export DOCKER_HOST=unix:///tmp/podman.sock

matejvasek · 2021-09-01T19:38:32Z

The command above basically tunnels remote unix socket to you local machine.

matejvasek · 2021-09-01T19:39:44Z

what I mean you can have something like ssh://user@host as DOCKER_HOST but not something like ssh://use@host/some/path.sock

At least this seems to be true for standard Go client library.

podman is better at this since CONTAINER_HOST can be ssh://use@host/some/path.sock

matejvasek · 2021-09-01T19:40:58Z

I played with this when I tried to connect to podman running in CRC OpenShift VM.

matejvasek · 2021-09-01T19:42:30Z

@tibcoplord

I'm not seeing how the socket would work if its inside the podman machine.

Ah I missed that it's VM via ssh. Then you probably need to workaround this using ssh tunnel.

matejvasek · 2021-09-01T20:01:12Z

@afbjorklund symlink solution looks better that using ssh tunnel, I myself used tunnel in past because my VM didn't allow me create symlinks.

afbjorklund · 2021-09-01T20:08:57Z

@afbjorklund symlink solution looks better that using ssh tunnel, I myself used tunnel in past because my VM didn't allow me create symlinks.

We had to use the tunnel hack for containerd/buildkitd, they are even worse than Docker when it comes to remoting.

https://minikube.sigs.k8s.io/docs/handbook/pushing/

For Podman it is mostly that the name of binary changes, once we get past the whole Varlink business in ancient history.

(and maybe --remote won't be needed anymore in 4.0)

tibcoplord · 2021-09-01T20:16:34Z

Ah ha :-) So following the sym link approach I got to -

$ export DOCKER_HOST=ssh://root@localhost:57586 
$ docker version
Client:
 Cloud integration: 1.0.17
 Version:           20.10.8
 API version:       1.40
 Go version:        go1.16.6
 Git commit:        3967b7d
 Built:             Fri Jul 30 19:55:20 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: linux/amd64/fedora-34
 Podman Engine:
  Version:          3.3.0
  APIVersion:       3.3.0
  Arch:             amd64
  BuildTime:        2021-08-20T19:36:14Z
  Experimental:     true
  GitCommit:        
  GoVersion:        go1.16.6
  KernelVersion:    5.13.12-200.fc34.x86_64
  MinAPIVersion:    3.1.0
  Os:               linux
 Engine:
  Version:          3.3.0
  API version:      1.40 (minimum version 1.24)
  Go version:       go1.16.6
  Git commit:       
  Built:            Fri Aug 20 19:36:14 2021
  OS/Arch:          linux/amd64
  Experimental:     true

But then in my next use-case, fabric8 maven plugin -

[INFO] --- fabric8-maven-plugin:4.4.1:build (build container image) @ test-datasink ---
[INFO] F8: Running in Kubernetes mode
[INFO] F8: Building Container image with Docker in Kubernetes mode
[ERROR] F8: Cannot create docker access object  [ssh protocol is not supported]

I'll try the tunnel approach.

matejvasek · 2021-09-01T20:18:07Z

ssh protocol is not supported

seems like flaw of the java lib

afbjorklund · 2021-09-01T20:21:49Z

@tibcoplord

Well, if docker client doesn't support ssh and podman server doesn't support tcp - I guess you are left to tunneling unix over ssh.

I think Docker has some way of presenting a unix socket from the VM at /var/run/docker.sock, but not sure how it works:

https://web.archive.org/web/20190623234611/https://docs.docker.com/docker-for-mac/docker-toolbox/

Some kind of virtual sockets.

https://github.com/linuxkit/virtsock

matejvasek · 2021-09-01T20:23:54Z

I believe podman can be set to use plain TCP.

matejvasek · 2021-09-01T20:24:26Z

e.g. podman system service --time=0 tcp:0.0.0.0:1234 --log-level=debug

matejvasek · 2021-09-01T20:25:21Z

but I don't know how the podman service process is started in VM, it my require changes to some systemd config files maybe?

matejvasek · 2021-09-01T20:29:50Z

and even if it didn't support TCP you still could use socat 😄 , not much better that tunneling

@afbjorklund what problems did you encountered with tunneling?

afbjorklund · 2021-09-01T20:32:50Z

@afbjorklund what problems did you encountered with tunneling?

It's a bunch of arcane commands, instead of an URL in a variable.

So it's hard to explain to the end users, that just want to "run Docker".

It probably doesn't work on Windows either, but then again what does...

rhatdan · 2021-10-31T09:55:50Z

@ashley-cui ?

github-actions · 2021-12-01T00:04:34Z

A friendly reminder that this issue had no activity for 30 days.

drpebcak · 2021-12-01T16:26:26Z

This would be really nice to have!

rhatdan · 2021-12-01T20:34:58Z

Agreed.

github-actions · 2022-01-01T00:04:51Z

A friendly reminder that this issue had no activity for 30 days.

github-actions · 2022-02-02T00:05:03Z

A friendly reminder that this issue had no activity for 30 days.

dangthanh360 · 2022-07-08T07:06:50Z

thanks @tibcoplord, it works well on my MacOS BigSur.

Well, with -

$ ssh -nNT -L/tmp/podman.sock:/run/user/1000/podman/podman.sock -i ~/.ssh/podman-machine-default ssh://core@localhost:64724 &
$ export DOCKER_HOST=unix:///tmp/podman.sock

The fabric8 build worked :)

[INFO] --- fabric8-maven-plugin:4.4.1:build (build container image) @ sbrt-base ---
[INFO] F8: Running in Kubernetes mode
[INFO] F8: Building Container image with Docker in Kubernetes mode
ba2c2d4a4d0c: Download complete 
7a497d63e726: Download complete 
[INFO] F8: Pulled docker.io/library/almalinux:8.4 in 12 seconds 
...

Need to find the best set of commands for the two options.

I use this command to get the port number
PODMAN_PORT=$(podman system connection ls --format="{{.URI}}" | grep "core" | grep -o -E '[0-9]+' | head -n 1)

Then run these commands to create tunnel over SSH

ssh -i ~/.ssh/podman-machine-default -nNT -L '/tmp/podman.sock:/run/user/1000/podman/podman.sock' ssh://core@localhost:$PODMAN_PORT
export DOCKER_HOST=unix:///tmp/podman.sock

dangthanh360 · 2022-07-08T07:10:57Z

thanks @n0rig, the simplest way as you mentioned:

For each shell, let's export the podman based SSH socket for the docker cli to use
export DOCKER_HOST="cat ~/.config/containers/containers.conf | grep -ioE "ssh://root@localhost:[0-9]+""

I use
export DOCKER_HOST="`cat ~/.config/containers/containers.conf | grep -ioE "ssh://core@localhost:[0-9]+"`"

debu999 · 2022-11-20T10:19:56Z

please note the following actually helped to make it work. Thanks for your help. You all rock.

user is per system so is port

alias docker=podman or sudo ln -sfv /usr/local/bin/podman /usr/local/bin/docker
podman system connection ls -- <-- This is to get port and user details in my case its user=501 and port=49523
Name URI Identity Default
podman-machine-default ssh://core@localhost:49523/run/user/501/podman/podman.sock /Users/wwww/.ssh/podman-machine-default true
podman-machine-default-root ssh://root@localhost:49523/run/podman/podman.sock /Users/wwww/.ssh/podman-machine-default false
ssh -fnNT -L/tmp/podman.sock:/run/user/501/podman/podman.sock -i ~/.ssh/podman-machine-default ssh://core@localhost:49523 -o StreamLocalBindUnlink=yes
export DOCKER_HOST='unix:///tmp/podman.sock'

Key Takeaway Commands to execute to make it work

ssh -fnNT -L/tmp/podman.sock:/run/user/ZZZ/podman/podman.sock -i ~/.ssh/podman-machine-default ssh://core@localhost:YYYYY -o StreamLocalBindUnlink=yes
export DOCKER_HOST='unix:///tmp/podman.sock'

davidhsingyuchen · 2022-11-28T21:58:45Z

The socket address is actually printed in the output of podman machine start:

You can still connect Docker API clients by setting DOCKER_HOST using the
following command in your terminal session:

	export DOCKER_HOST='unix:///Users/davidhyc/.local/share/containers/podman/machine/podman-machine-default/podman.sock'

Maybe we just need to make that info more accessible (e.g., printed by some command)? system connection ls is printing something else:

➜  ~ podman system connection ls
Name                         URI                                                         Identity                                     Default
podman-machine-default       ssh://core@localhost:63594/run/user/504/podman/podman.sock  /Users/davidhyc/.ssh/podman-machine-default  true
podman-machine-default-root  ssh://root@localhost:63594/run/podman/podman.sock           /Users/davidhyc/.ssh/podman-machine-default  false

rhatdan · 2022-11-30T20:54:00Z

Would you expect that same line to print out again there, if you have a podman machine running?

davidhsingyuchen · 2022-12-01T02:42:14Z

Would you expect that same line to print out again there, if you have a podman machine running?

Thanks for the response. I think that would work. Or anywhere under the system subcommand that makes sense .

rhatdan · 2022-12-03T13:01:51Z

Please open a new issue on this.

openshift-ci bot added the kind/feature Categorizes issue or PR as related to a new feature. label Sep 1, 2021

rhatdan added the machine label Oct 31, 2021

github-actions bot added the stale-issue label Dec 1, 2021

rhatdan added 4.0 macos MacOS (OSX) related and removed stale-issue labels Dec 1, 2021

github-actions bot added the stale-issue label Jan 1, 2022

rhatdan removed the stale-issue label Jan 2, 2022

matejvasek mentioned this issue Jan 28, 2022

Running docker-compose with podman as a backend on MacOS is failing #13069

Closed

jkremser mentioned this issue Jan 31, 2022

[FEATURE] Podman support k3d-io/k3d#84

Closed

github-actions bot added the stale-issue label Feb 2, 2022

rhatdan removed the stale-issue label Feb 2, 2022

praveenkumar mentioned this issue Feb 9, 2022

[RFE] Access docker api on a mac crc-org/crc#2995

Closed

n1hility mentioned this issue Feb 11, 2022

Mac API forwarding using a privileged docker socket claim helper #13075

Merged

openshift-merge-robot closed this as completed in #13075 Feb 16, 2022

davidhsingyuchen mentioned this issue Dec 4, 2022

a command that prints out podman socket address #16725

Closed

github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 8, 2023

github-actions bot locked as resolved and limited conversation to collaborators Sep 8, 2023

How to access docker api on a mac #11397

How to access docker api on a mac #11397

Comments

tibcoplord commented Sep 1, 2021 • edited Loading

mheon commented Sep 1, 2021

rhatdan commented Sep 1, 2021

matejvasek commented Sep 1, 2021 • edited Loading

matejvasek commented Sep 1, 2021

tibcoplord commented Sep 1, 2021

matejvasek commented Sep 1, 2021 • edited Loading

tibcoplord commented Sep 1, 2021

baude commented Sep 1, 2021

tibcoplord commented Sep 1, 2021

matejvasek commented Sep 1, 2021

afbjorklund commented Sep 1, 2021 • edited Loading

matejvasek commented Sep 1, 2021

afbjorklund commented Sep 1, 2021 • edited Loading

matejvasek commented Sep 1, 2021

matejvasek commented Sep 1, 2021 • edited Loading

matejvasek commented Sep 1, 2021

matejvasek commented Sep 1, 2021

matejvasek commented Sep 1, 2021

matejvasek commented Sep 1, 2021

matejvasek commented Sep 1, 2021

afbjorklund commented Sep 1, 2021 • edited Loading

tibcoplord commented Sep 1, 2021

matejvasek commented Sep 1, 2021 • edited Loading

afbjorklund commented Sep 1, 2021 • edited Loading

matejvasek commented Sep 1, 2021

matejvasek commented Sep 1, 2021 • edited Loading

matejvasek commented Sep 1, 2021 • edited Loading

matejvasek commented Sep 1, 2021

afbjorklund commented Sep 1, 2021 • edited Loading

rhatdan commented Oct 31, 2021

github-actions bot commented Dec 1, 2021

drpebcak commented Dec 1, 2021

rhatdan commented Dec 1, 2021

github-actions bot commented Jan 1, 2022

github-actions bot commented Feb 2, 2022

dangthanh360 commented Jul 8, 2022 • edited Loading

dangthanh360 commented Jul 8, 2022

debu999 commented Nov 20, 2022 • edited Loading

user is per system so is port

Key Takeaway Commands to execute to make it work

davidhsingyuchen commented Nov 28, 2022 • edited Loading

rhatdan commented Nov 30, 2022

davidhsingyuchen commented Dec 1, 2022

rhatdan commented Dec 3, 2022

tibcoplord commented Sep 1, 2021 •

edited

Loading

matejvasek commented Sep 1, 2021 •

edited

Loading

matejvasek commented Sep 1, 2021 •

edited

Loading

afbjorklund commented Sep 1, 2021 •

edited

Loading

afbjorklund commented Sep 1, 2021 •

edited

Loading

matejvasek commented Sep 1, 2021 •

edited

Loading

afbjorklund commented Sep 1, 2021 •

edited

Loading

matejvasek commented Sep 1, 2021 •

edited

Loading

afbjorklund commented Sep 1, 2021 •

edited

Loading

matejvasek commented Sep 1, 2021 •

edited

Loading

matejvasek commented Sep 1, 2021 •

edited

Loading

afbjorklund commented Sep 1, 2021 •

edited

Loading

dangthanh360 commented Jul 8, 2022 •

edited

Loading

debu999 commented Nov 20, 2022 •

edited

Loading

davidhsingyuchen commented Nov 28, 2022 •

edited

Loading