cgroupControllers difference between remote and on vm(podman on mac) when trying to run rootless kind

[HoogWater]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description
When trying to run kind with rootless podman the following error occurs

ERROR: failed to create cluster: running kind with rootless provider requires setting systemd property "Delegate=yes", see https://kind.sigs.k8s.io/docs/user/rootless/

Steps to reproduce the issue:

1. kind create cluster

Describe the results you received:
ERROR: failed to create cluster: running kind with rootless provider requires setting systemd property "Delegate=yes", see https://kind.sigs.k8s.io/docs/user/rootless/

Describe the results you expected:
a running kind cluster

Additional information you deem important (e.g. issue happens only occasionally):
This seems to be a problem with the cgroupcontrollers.

there is a difference in output of "podman info" when running it remote(on mac) or on the podman vm

Remote:

  cgroupControllers:
  - memory
  - pids

In the VM(after changing Delegate to yes in the vm, prior to that it was "cpu io memory pids"):

  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - misc

Output of podman version:

Client:       Podman Engine
Version:      4.0.2
API Version:  4.0.2
Go Version:   go1.17.8

Built:      Wed Mar  2 15:04:36 2022
OS/Arch:    darwin/amd64

Server:       Podman Engine
Version:      4.0.2
API Version:  4.0.2
Go Version:   go1.16.14

Built:      Thu Mar  3 15:56:56 2022
OS/Arch:    linux/amd64

Output of podman info --debug:

(paste your output here)

Package info (e.g. output of rpm -q podman or apt list podman):

podman on mac via homebrew

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):
podman 4.0.2 on MacOS

[HoogWater]

see kubernetes-sigs/kind#2684 for the issue for the kind project

[Luap99]

@giuseppe @flouthoc PTAL

[rhatdan]

Are you sure you are not comparing podman --remote using rootless and podman --remote=false in rootfull mode?

[HoogWater]

Are you sure you are not comparing podman --remote using rootless and podman --remote=false in rootfull mode?

i am sure
the first is the output of "podman info" on the mac
the second is when ssh'ed into the vm and running a "podman info"

[rhatdan]

Sure, I just want to make sure that the other parts of info agree.

$ podman info | grep rootless
rootless: true
$ podman --remote info | grep rootless
rootless: true

[ncdc]

Can confirm rootless: true for both those commands

[ncdc]

$ podman machine ssh

$ podman info | grep -A6 cgroupControllers
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2

$ podman --remote info | grep -A6 cgroupControllers
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.0-2.fc35.x86_64

[rhatdan]

And both are podman 4.0.2 correct?

[ncdc]

Yes:

podman version
Client:       Podman Engine
Version:      4.0.2
API Version:  4.0.2
Go Version:   go1.17.8

Built:      Wed Mar  2 09:04:36 2022
OS/Arch:    darwin/amd64

Server:       Podman Engine
Version:      4.0.2
API Version:  4.0.2
Go Version:   go1.16.14

Built:      Thu Mar  3 09:56:56 2022
OS/Arch:    linux/amd64

[rhatdan]

@giuseppe ideas?

[giuseppe]

opened a PR: #13716

now I get:

$ bin/podman-remote  info | head -10
host:
  arch: amd64
  buildahVersion: 1.24.1
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2

If you'd like to play with the fix, you can just ssh into the machine, copy /usr/lib/systemd/user/podman.service under /etc/systemd/user/ and edit it there to add Delegate=yes under [Service]

[ncdc]

Tried the manual fix, rebooted the podman vm, and things are looking good - thanks!

[rhatdan]

@ncdc Can you add a LGTM to the PR.

[ncdc]

@rhatdan done!