Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mac API forwarding using a privileged docker socket claim helper #13075

Merged
merged 3 commits into from
Feb 16, 2022

Conversation

n1hility
Copy link
Member

@n1hility n1hility commented Jan 29, 2022

This PR provides a comparable low-touch experience to the current windows API forwarding approach by introducing a privileged helper that runs under launchd.

Fixes #11462
Fixes #11397
Fixes #11389
Fixes #11422

Additional changes in this PR

  • podman machine can now be configured to run as rootful or rootless by passing --rootful=true|false to init
  • Added new podman machine set command to change this after init (and in the future other settings that can be changed)
  • Switched Windows to rootless default to match mac default
  • podman machine start now advises how to switch to rootful
  • Windows is now updated to pull podman4 from the copr repo like on mac

The reasons for this approach are:

  1. docker.sock is immediately under /var/run which is not writable to users
  2. /var/run is backed by tmpfs, leading to repeated authentication + reinstallation cycles after reboot
  3. dockerd aggressively reclaims /var/run/docker.sock, forcing it more frequently even after a stable boot

Using a privileged helper allows podman machine to request relinking to a fixed predetermined location as needed. It executes as a nowait inetd service, so is only briefly active when a switch is required. The control channel is a unix socket maintained by launchd that is only accessible to the podman machine user. Both the link location and target are fixed by installed configuration, and not influenced by socket input, which is restricted to a simple trigger action. Multiple machine instances are handled by having an intermediate link which is managed by podman machine without involving the helper, since the link is located in the user data directory.

Before taking control of the socket link, podman machine first checks if it's in use. If so, as is the case in the Windows impl, the system falls back to a machine local socket and prints instructions on how to set DOCKER_HOST. The same occurs if the helper is not available.

There is a one-time action that podman machine start will prompt the user to run. We could have podman machine init run it automatically via osacript using osx auth processes (biometric auth works for example). However, I assume that everyone would prefer a brew formula action, plus the one sudo launch is straightforward.

The referenced installation action sudo podman-mac-helper install copies the helper into a dedicated fs location and verifies the fs path is sufficiently hardened since the typical brew managed locations are too loose for system processes (user-owned with lots of links so easily swappable).

Here is what it looks like pre helper installation:

Starting machine "podman-machine-default"
INFO[0000] waiting for clients...                       
INFO[0000] new connection from  to /var/folders/5p/1gyddrwn4cj6w3ccfy8bj4kr0000gs/T/podman/qemu_podman-machine-default.sock 
Waiting for VM ...
INFO[0013] Socket forward established: /Users/jason/.local/share/containers/podman/machine/podman-machine-default/podman.sock to /run/user/1000/podman/podman.sock 

This machine is currently configured in rootless mode. If your containers
require root permissions (e.g. ports < 1024), or if you run into compatibility
issues with non-podman clients, you can switch using the following command: 

	podman machine set --rootful

API forwarding listening on: /Users/jason/.local/share/containers/podman/machine/podman-machine-default/podman.sock

The system helper service is not installed; the default Docker API socket
address can't be used by podman. If you would like to install it run the
following command:

	sudo /Users/jason/devel/podman/bin/darwin/podman-mac-helper install

You can still connect Docker API clients by setting DOCKER_HOST using the
following command in your terminal session:

	export DOCKER_HOST='unix:///Users/jason/.local/share/containers/podman/machine/podman-machine-default/podman.sock'

Machine "podman-machine-default" started successfully

After installing:

Starting machine "podman-machine-default"
INFO[0000] waiting for clients...                       
INFO[0000] new connection from  to /var/folders/5p/1gyddrwn4cj6w3ccfy8bj4kr0000gs/T/podman/qemu_podman-machine-default.sock 
Waiting for VM ...
INFO[0013] Socket forward established: /Users/jason/.local/share/containers/podman/machine/podman-machine-default/podman.sock to /run/user/1000/podman/podman.sock 

This machine is currently configured in rootless mode. If your containers
require root permissions (e.g. ports < 1024), or if you run into compatibility
issues with non-podman clients, you can switch using the following command: 

	podman machine set --rootful

API forwarding listening on: /var/run/docker.sock
Docker API clients default to this address. You do not need to set DOCKER_HOST.

Machine "podman-machine-default" started successfully

If something like dockerd is using the socket:

Starting machine "podman-machine-default"
INFO[0000] waiting for clients...                       
INFO[0000] new connection from  to /var/folders/5p/1gyddrwn4cj6w3ccfy8bj4kr0000gs/T/podman/qemu_podman-machine-default.sock 
Waiting for VM ...
INFO[0012] Socket forward established: /Users/jason/.local/share/containers/podman/machine/podman-machine-default/podman.sock to /run/podman/podman.sock 
API forwarding listening on: /Users/jason/.local/share/containers/podman/machine/podman-machine-default/podman.sock

Another process was listening on the default Docker API socket address.
You can still connect Docker API clients by setting DOCKER HOST using the
following command in your terminal session:

	export DOCKER_HOST='unix:///Users/jason/.local/share/containers/podman/machine/podman-machine-default/podman.sock'

Machine "podman-machine-default" started successfully

In action with API clients:


jason@Tripel podman (main *) $ docker version
Client:
 Cloud integration: 1.0.17
 Version:           20.10.8
 API version:       1.40
 Go version:        go1.16.6
 Git commit:        3967b7d
 Built:             Fri Jul 30 19:55:20 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: linux/amd64/fedora-35
 Podman Engine:
  Version:          4.0.0-rc5
  APIVersion:       4.0.0-rc5
  Arch:             amd64
  BuildTime:        2022-02-12T07:26:48-06:00
  Experimental:     false
  GitCommit:        
  GoVersion:        go1.16.13
  KernelVersion:    5.15.18-200.fc35.x86_64
  MinAPIVersion:    4.0.0-rc5
  Os:               linux

[NO NEW TESTS NEEDED]

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 29, 2022
@n1hility
Copy link
Member Author

@baude sorry I didn't get this in last night had a hectic Friday Hopefully I didn't disrupt your weekend plans too much in this area. Ill keep an eye on chat and here if you need me.

@n1hility n1hility force-pushed the mac-forward-helper branch 2 times, most recently from afdd830 to f00c108 Compare January 29, 2022 22:51
@rhatdan rhatdan added the 4.1 label Feb 1, 2022
@mheon
Copy link
Member

mheon commented Feb 7, 2022

@baude @ashley-cui PTAL

@n1hility
Copy link
Member Author

n1hility commented Feb 9, 2022

i had a few cycles in the evening and addressed a number of loose ends. I'll submit an update tomorrow

@n1hility n1hility marked this pull request as ready for review February 9, 2022 22:21
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 9, 2022
@n1hility
Copy link
Member Author

n1hility commented Feb 9, 2022

Ok this is finally pushed with an update to address TODOs and squashed

@n1hility n1hility changed the title Initial implementation of mac forwarding using a privileged docker socket claim helper Mac API forwarding using a privileged docker socket claim helper Feb 10, 2022
@baude
Copy link
Member

baude commented Feb 10, 2022

/approve

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 10, 2022
@n1hility
Copy link
Member Author

/hold more commits coming

@openshift-ci openshift-ci bot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Feb 15, 2022
@openshift-ci openshift-ci bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 16, 2022
@n1hility n1hility force-pushed the mac-forward-helper branch 5 times, most recently from 4f483be to 963bb4a Compare February 16, 2022 09:14
…t=t|f

Switch default to rootless for mac and windows

Signed-off-by: Jason T. Greene <[email protected]>
Signed-off-by: Jason T. Greene <[email protected]>
@rhatdan
Copy link
Member

rhatdan commented Feb 16, 2022

LGTM
/approve
@baude PTAL
@containers/podman-maintainers PTAL

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Feb 16, 2022

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: baude, n1hility, rhatdan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@n1hility
Copy link
Member Author

only pending item I have (other than changes from review) is to double check machine on linux, will do that ASAP and lift the hold I had on this

@n1hility
Copy link
Member Author

/remove-hold

My local testing of machine on linux shows no regressions

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 16, 2022
@mheon
Copy link
Member

mheon commented Feb 16, 2022

Changes LGTM but I'd like @baude to do the final merge

@baude
Copy link
Member

baude commented Feb 16, 2022

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Feb 16, 2022
@openshift-merge-robot openshift-merge-robot merged commit f918a94 into containers:main Feb 16, 2022
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 21, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
6 participants