Podman pod YAML with --security-opt seccomp

[nikAizuddin]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind feature

Description

How do I create a pod YAML file that is equivalent to --security-opt seccomp=profile.json? For example the following situation:

$ git clone https://github.com/saferwall/saferwall.git
$ cd saferwall
$ podman run -it --rm --security-opt seccomp=./build/data/seccomp.json -v ./test/multiav/clean/eicar.com:/opt/eicar.com:ro,z docker.io/saferwall/windefender bash -c "cd /opt/windows-defender && ./mpclient /opt/eicar.com"

I have to use seccomp. Otherwise, the ./mpclient command will fail.

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

podman version 2.1.1

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.16.1
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.21-2.fc32.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.21, commit: 81d18b6c3ffc266abdef7ca94c1450e669a6a388'
  cpus: 4
  distribution:
    distribution: fedora
    version: "32"
  eventLogger: journald
  hostname: saferwall-box
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.6.6-300.fc32.x86_64
  linkmode: dynamic
  memFree: 687673344
  memTotal: 8339902464
  ociRuntime:
    name: crun
    package: crun-0.15-5.fc32.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 0.15
      commit: 56ca95e61639510c7dbd39ff512f80f626404969
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1000/podman/podman.sock
  rootless: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.4-1.fc32.x86_64
    version: |-
      slirp4netns version 1.1.4
      commit: b66ffa8e262507e37fca689822d23430f3357fe8
      libslirp: 4.3.1
      SLIRP_CONFIG_VERSION_MAX: 2
  swapFree: 0
  swapTotal: 0
  uptime: 13h 1m 26.37s (Approximately 0.54 days)
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - registry.centos.org
  - docker.io
store:
  configFile: /home/vagrant/.config/containers/storage.conf
  containerStore:
    number: 10
    paused: 0
    running: 10
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-1.2.0-1.fc32.x86_64
      Version: |-
        fusermount3 version: 3.9.1
        fuse-overlayfs: version 1.1.0
        FUSE library version 3.9.1
        using FUSE kernel interface version 7.31
  graphRoot: /home/vagrant/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 136
  runRoot: /run/user/1000/containers
  volumePath: /home/vagrant/.local/share/containers/storage/volumes
version:
  APIVersion: 2.0.0
  Built: 1601494271
  BuiltTime: Wed Sep 30 19:31:11 2020
  GitCommit: ""
  GoVersion: go1.14.9
  OsArch: linux/amd64
  Version: 2.1.1

Package info (e.g. output of rpm -q podman or apt list podman):

podman-2.1.1-7.fc32.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

No

Additional environment details (AWS, VirtualBox, physical, etc.):

I'm running Fedora 32 in Vagrant box with libvirt provider.

[mheon]

@haircommander PTAL

[haircommander]

It is not currently supported, would you have any interest in trying to implement it @nikAizuddin ?

We'd need to add a flag to generate kube similar to play kube: --seccomp-profile-path which tells generate kube the relative directory the seccomp.json will be in. Then, the user could actually use podman play kube --seccomp-profile-path=$DIR to run the generated yaml

[nikAizuddin]

Wait, I didn't noticed podman play kube has --seccomp-profile-root option. I'll try with this option first.

[nikAizuddin]

I create multiav-pod.yaml as follow:

apiVersion: v1
kind: Pod
metadata:
  name: multiav-pod
spec:
  restartPolicy: Never
  securityContext:
    seccompProfile:
      type: Localhost
      localhostProfile: seccomp.json
  containers:
  - name: windefender
    image: docker.io/saferwall/gowindefender:0.0.1
    ports:
    - containerPort: 50051
      hostPort: 50051
      hostIP: 0.0.0.0
      protocol: tcp

Then, create pod and container with:

$ podman play kube --seccomp-profile-root /home/vagrant/Documents/saferwall/build/data multiav-pod.yaml

The seccomp.json is ignored by podman play kube:

$ podman exec -it multiav-pod-windefender bash
$ cd /opt/windows-defender/
$ ./mpclient -h
Aborted (core dumped)

Here YAML file generated from $ podman generate kube multiav-pod command:

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2020-10-27T04:49:37Z"
  labels:
    app: multiav-pod
  name: multiav-pod
spec:
  containers:
  - command:
    - /bin/windefenderscanner
    env:
    - name: PATH
      value: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    - name: TERM
      value: xterm
    - name: HOSTNAME
      value: multiav-pod
    - name: WINDOWS_DEFENDER_UPDATE
      value: https://go.microsoft.com/fwlink/?LinkID=121721&arch=x86
    - name: WINDOWS_DEFENDER_LOADLIBRARY
      value: https://codeload.github.com/taviso/loadlibrary/zip/master
    - name: WINDOWS_DEFENDER_INSTALL_DIR
      value: /opt/windows-defender
    - name: WINDOWS_DEFENDER_TMP
      value: /tmp/windows-defender
    - name: WINDOWS_DEFENDER_DB_UPDATE_DATE
      value: /av_db_update_date.txt
    image: docker.io/saferwall/gowindefender:0.0.1
    name: multiav-pod-windefender
    ports:
    - containerPort: 50051
      hostIP: 0.0.0.0
      hostPort: 50051
      protocol: TCP
    resources: {}
    securityContext:
      allowPrivilegeEscalation: true
      capabilities: {}
      privileged: false
      readOnlyRootFilesystem: false
      runAsGroup: 102
      runAsUser: 101
      seLinuxOptions: {}
    workingDir: /
  restartPolicy: Never
status: {}
---
metadata:
  creationTimestamp: null
spec: {}
status:
  loadBalancer: {}

[nikAizuddin]

@haircommander I just tried with --seccomp-profile-root as above and the seccomp.json seems ignored. So, command such as podman play kube --seccomp-profile-root=/var/lib/kubelet/seccomp/ pod.yaml is not yet supported?

[haircommander]

you need to specify the file name in the pod annotations:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    - seccomp.security.alpha.kubernetes.io/pod: localhost/seccomp.json
  creationTimestamp: "2020-10-27T04:49:37Z"
  labels:
    app: multiav-pod
  name: multiav-pod
spec:
  containers:
  - command:
    - /bin/windefenderscanner
    env:
    - name: PATH
      value: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    - name: TERM
      value: xterm
    - name: HOSTNAME
      value: multiav-pod
    - name: WINDOWS_DEFENDER_UPDATE
      value: https://go.microsoft.com/fwlink/?LinkID=121721&arch=x86
    - name: WINDOWS_DEFENDER_LOADLIBRARY
      value: https://codeload.github.com/taviso/loadlibrary/zip/master
    - name: WINDOWS_DEFENDER_INSTALL_DIR
      value: /opt/windows-defender
    - name: WINDOWS_DEFENDER_TMP
      value: /tmp/windows-defender
    - name: WINDOWS_DEFENDER_DB_UPDATE_DATE
      value: /av_db_update_date.txt
    image: docker.io/saferwall/gowindefender:0.0.1
    name: multiav-pod-windefender
    ports:
    - containerPort: 50051
      hostIP: 0.0.0.0
      hostPort: 50051
      protocol: TCP
    resources: {}
    securityContext:
      allowPrivilegeEscalation: true
      capabilities: {}
      privileged: false
      readOnlyRootFilesystem: false
      runAsGroup: 102
      runAsUser: 101
      seLinuxOptions: {}
    workingDir: /
  restartPolicy: Never
status: {}
---
metadata:
  creationTimestamp: null
spec: {}
status:
  loadBalancer: {}

[rhatdan]

@nikAizuddin were you able to get this to work?
@haircommander should we make this easier for podman play kube? Specifying this directory to tell where seccomp.json files are located seems a little forced. Should we search, /usr/share/containers, /etc/containers, "./", as well as /var/lib/kubelet/seccomp/?

[haircommander]

The issue is we're trying for kube compatibility, but also use expressiveness. It may be confusing if we silently find a seccomp.json in /etc/containers, and kube fails to find such a file because it only looks in /var/lib/kubelet/seccomp/. Having the flag makes this behaviour explicit

[rhatdan]

But if I have the file in /var/lib/kubelet/seccomp/ then it would work?

What does podman generate kube do with this information?

[nikAizuddin]

@nikAizuddin were you able to get this to work?

I haven't tried yet. At the moment I manually create a pod and then add windefender container with --security-opt into the pod.

you need to specify the file name in the pod annotations

I'll try this solution by tomorrow and I'll report back.

[rhatdan]

@nikAizuddin What were the results?

[nikAizuddin]

Hi, sorry for delay. I just got time to test it today. Unfortunately, I cannot execute YAML with annotations. I got the following error:

Error: unable to read YAML "multiav-pod.yaml" as Kube Pod: error unmarshaling JSON: json: cannot unmarshal array into Go struct field ObjectMeta.metadata.annotations of type map[string]string

I can execute the YAML file if I remove annotations.

[nikAizuddin]

But if I change from list into dict, it works:

 kind: Pod
 metadata:
   annotations:
-    - seccomp.security.alpha.kubernetes.io/pod: localhost/seccomp.json
+    seccomp.security.alpha.kubernetes.io/pod: localhost/seccomp.json
   name: multiav-pod
 spec:
   restartPolicy: OnFailur

[nikAizuddin]

I haven't tried yet. At the moment I manually create a pod and then add windefender container with --security-opt into the pod.

I'm going to replace this existing workaround with multiav-pod.yaml and I'll report back.

[nikAizuddin]

Alright, I have replaced the workaround with YAML file and tested working. This the rendered version of the YAML file:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    seccomp.security.alpha.kubernetes.io/pod: localhost/seccomp.json
  name: multiav-pod
spec:
  restartPolicy: OnFailure
  containers:
  - name: clamav
    image: localhost/saferwall/goclamav:0.0.2
    ports:
    - containerPort: 10004
      hostPort: 10004
      hostIP: 0.0.0.0
      protocol: tcp
    env:
    - name: LISTEN_PORT
      value: 10004
    volumeMounts:
    - mountPath: /samples:rw,z
      name: samples
    - mountPath: /etc/clamav/clamd.conf:ro,z
      name: clamav-conf
  - name: comodo
    image: localhost/saferwall/gocomodo:0.0.2
    ports:
    - containerPort: 10005
      hostPort: 10005
      hostIP: 0.0.0.0
      protocol: tcp
    env:
    - name: LISTEN_PORT
      value: 10005
    volumeMounts:
    - mountPath: /samples:rw,z
      name: samples
  - name: sophos
    image: localhost/saferwall/gosophos:0.0.2
    ports:
    - containerPort: 10011
      hostPort: 10011
      hostIP: 0.0.0.0
      protocol: tcp
    env:
    - name: LISTEN_PORT
      value: 10011
    volumeMounts:
    - mountPath: /samples:rw,z
      name: samples
  - name: windefender
    image: localhost/saferwall/gowindefender:0.0.1
    ports:
    - containerPort: 10014
      hostPort: 10014
      hostIP: 0.0.0.0
      protocol: tcp
    env:
    - name: LISTEN_PORT
      value: 10014
    volumeMounts:
    - mountPath: /samples:rw,z
      name: samples
  volumes:
  - name: samples
    hostPath:
      path: ./samples
  - name: clamav-conf
    hostPath:
      path: ./clamd.conf

I have one more question. Is it possible to apply seccomp profile only for windefender container? Because from the YAML file above, it seems the seccomp profile are applied to all containers.

[fpoirotte]

I have one more question. Is it possible to apply seccomp profile only for windefender container? Because from the YAML file above, it seems the seccomp profile are applied to all containers.

I know this is an old thread, but I think others might look for the same answer.

seccomp.security.alpha.kubernetes.io/pod indeed applies to every container in the pod.

To apply the profile to a single container, use container.seccomp.security.alpha.kubernetes.io/<container> instead (replace <container> with the container name).
E.g. container.seccomp.security.alpha.kubernetes.io/windefender: localhost/seccomp.json

If both annotations are specified, the container-oriented one takes precedence.