Runc 1.1.4 container fails with `permission denied: unknown` when relying on capabilities #3715

Kern-- · 2023-01-30T21:45:23Z

After upgrading from runc 1.1.3 to runc 1.1.4 we started to see a particular container fail with an error like:

FATA[0000] failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec /entrypoint.sh: permission denied: unknown

The same container runs successfully with runc 1.1.3.

Upon inspection of the container, we discoverd that the entrypoint's permissions were set to 744, but it was owned by root instead of the container's user. We were able to fix the issue by correctly chowning the entrypoint script, but the upgrade caused an outage, so I'm reporting it here as a bug.

Root Cause
I traced the cause down to this PR #3522

In that PR, a new check was introduced to exit early if the entrypoint isn't executable. It does so by

Attempting to use faccessat2(2) to check the effective permissions if available (requires kernel 5.8+ and libseccomp 2.4.0+)
Falling back to access(2) to check the real users permission if faccessat2(2) isn't available.

We were hitting case 2. The problem is that this check is more restrictive than execve(2) because access(2) removes capabilities when checking for permission.

Docker and containerd have a set of default capabilities that they add to the runtime spec. Both include CAP_DAC_OVERRIDE in this set. See containerd's list, docker's list.
The capabilities man page says this about CAP_DAC_OVERRIDE

Bypass file read, write, and execute permission checks. (DAC is an abbreviation of "discretionary access control".)

So in runc 1.1.3, execve worked because runc has the CAP_DAC_OVERRIDE capability to ignore the fact that the user doesn't actually have execute permission. In runc 1.1.4, the addtional access(2) check striped capabilities when checking permissions and caused runc to exit with permission denied: unknown.

Conditions for this bug

kernel < 5.8 or libseccomp < 2.4.0
An entry point script where the owner has execute permission, but the container user has readonly permission
CAP_DAC_OVERRIDE or similar capabilities set (which is the default for both containerd and docker)

Reproduction

entrypoint.sh

#!/bin/sh

echo "Hello, World!"

Dockerfile

FROM alpine:latest

COPY ./entrypoint.sh /entrypoint.sh
RUN  chown root:root /entrypoint.sh\
      && chmod 744 /entrypoint.sh

RUN adduser --disabled-password testuser
USER testuser
ENTRYPOINT ["/entrypoint.sh"]

When run with runc 1.1.3:

$ sudo nerdctl run --rm -it --net=none runc114error:latest
Hello, World!

When run with runc 1.1.4:

$ sudo nerdctl run --rm -it --net=none runc114error:latest
FATA[0000] failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec /entrypoint.sh: permission denied: unknown

The text was updated successfully, but these errors were encountered:

This is a test case for issue reported as opencontainers#3715. In short, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. Signed-off-by: Kir Kolyshkin <[email protected]>

kolyshkin · 2023-02-15T02:29:10Z

A quick heads up.

Looking into this for the past few days, the fix is to take into account whether CAP_DAC_OVERRIDE is set, which is sort of trivial, but for Go 1.20, we will see this very error from exec.LookPath thanks to https://go-review.googlesource.com/c/go/+/414824 (which made its way into Go 1.20).

@AkihiroSuda suggested reverting #3522 for 1.1.5, but it is not enough because in Go 1.20 this won't work.

So, this needs to be fixed in Go 1.20 as well, and I am working on it. Again, the fix is sort of trivial, but the test case is not.

kolyshkin · 2023-02-16T02:16:02Z

Bug reported to golang: golang/go#58552

The fix (to golang) is here: https://go.dev/cl/468735

CL 416115 added using faccessat2(2) from syscall.Faccessat on Linux (which is the only true way to implement AT_EACCESS flag handing), if available. If not available, it uses some heuristics to mimic the kernel behavior, mostly taken from glibc (see CL 126415). Next, CL 414824 added using the above call (via unix.Eaccess) to exec.LookPath in order to check if the binary can really be executed. As a result, in a very specific scenario, described below, syscall.Faccessat (and thus exec.LookPath) mistakenly tells that the binary can not be executed, while in reality it can be. This makes this bug a regression in Go 1.20. This scenario involves all these conditions: - no faccessat2 support available (i.e. either Linux kernel < 5.8, or a seccomp set up to disable faccessat2); - the current user is not root (i.e. geteuid() != 0); - CAP_DAC_OVERRIDE capability is set for the current process; - the file to be executed does not have executable permission bit set for either the current EUID or EGID; - the file to be executed have at least one executable bit set. Unfortunately, this set of conditions was observed in the wild -- a container run as a non-root user with the binary file owned by root with executable permission set for a user only [1]. Essentially it means it is not as rare as it may seem. Now, CAP_DAC_OVERRIDE essentially makes the kernel bypass most of the checks, so execve(2) and friends work the same was as for root user, i.e. if at least one executable bit it set, the permission to execute is granted (see generic_permission() function in the Linux kernel). Modify the code to check for CAP_DAC_OVERRIDE and mimic the kernel behavior for permission checks. [1] opencontainers/runc#3715 Fixes #58552. Change-Id: I82a7e757ab3fd3d0193690a65c3b48fee46ff067 Reviewed-on: https://go-review.googlesource.com/c/go/+/468735 Reviewed-by: Damien Neil <[email protected]> TryBot-Result: Gopher Robot <[email protected]> Run-TryBot: Ian Lance Taylor <[email protected]> Auto-Submit: Ian Lance Taylor <[email protected]> Reviewed-by: Ian Lance Taylor <[email protected]>

CL 126516 added support for flags argument, implemented in the same way as glibc does (it tries to guess what the kernel would do). CL 246537 added using faccess2(2) Linux syscall which supports the flags directly. For older kernels, though, the syscall is not available, and the code uses glibc-like fallback. There is one very specific scenario in which the fallback fails. The scenario involves all these conditions: - no faccessat2 support available (i.e. either Linux kernel < 5.8, or a seccomp set up to disable faccessat2); - the current user is not root (i.e. geteuid() != 0); - CAP_DAC_OVERRIDE capability is set for the current process; - the file to be executed does not have executable permission bit set for either the current EUID or EGID; - the file to be executed have at least one executable bit set. Unfortunately, this set of conditions was observed in the wild -- a container run as a non-root user with the binary file owned by root with executable permission set for a user only [1]. Essentially it means it is not as rare as it may seem. Now, CAP_DAC_OVERRIDE essentially makes the kernel bypass most of the checks, so execve(2) and friends work the same was as for root user, i.e. if at least one executable bit it set, the permission to execute is granted (see generic_permission() function in the Linux kernel). Modify the code to check for CAP_DAC_OVERRIDE and mimic the kernel behavior for permission checks. This is essentially the same fix as CL 468735 for Go syscall package. Tested on CentOS 7 with the repro similar to the one from [2]. [1] opencontainers/runc#3715 [2] golang/go#58552 (comment) Change-Id: I726b6acab6a6e6d0358ef98e6a582b405c347614 Signed-off-by: Kir Kolyshkin <[email protected]> Reviewed-on: https://go-review.googlesource.com/c/sys/+/468877 Reviewed-by: Ian Lance Taylor <[email protected]> Run-TryBot: Ian Lance Taylor <[email protected]> TryBot-Result: Gopher Robot <[email protected]> Reviewed-by: Bryan Mills <[email protected]> Auto-Submit: Ian Lance Taylor <[email protected]>

kolyshkin · 2023-02-21T20:43:43Z

The fix to Golang has landed into the master branch; backport to Go 1.20.x is initiated: golang/go#58624

CL 416115 added using faccessat2(2) from syscall.Faccessat on Linux (which is the only true way to implement AT_EACCESS flag handing), if available. If not available, it uses some heuristics to mimic the kernel behavior, mostly taken from glibc (see CL 126415). Next, CL 414824 added using the above call (via unix.Eaccess) to exec.LookPath in order to check if the binary can really be executed. As a result, in a very specific scenario, described below, syscall.Faccessat (and thus exec.LookPath) mistakenly tells that the binary can not be executed, while in reality it can be. This makes this bug a regression in Go 1.20. This scenario involves all these conditions: - no faccessat2 support available (i.e. either Linux kernel < 5.8, or a seccomp set up to disable faccessat2); - the current user is not root (i.e. geteuid() != 0); - CAP_DAC_OVERRIDE capability is set for the current process; - the file to be executed does not have executable permission bit set for either the current EUID or EGID; - the file to be executed have at least one executable bit set. Unfortunately, this set of conditions was observed in the wild -- a container run as a non-root user with the binary file owned by root with executable permission set for a user only [1]. Essentially it means it is not as rare as it may seem. Now, CAP_DAC_OVERRIDE essentially makes the kernel bypass most of the checks, so execve(2) and friends work the same was as for root user, i.e. if at least one executable bit it set, the permission to execute is granted (see generic_permission() function in the Linux kernel). Modify the code to check for CAP_DAC_OVERRIDE and mimic the kernel behavior for permission checks. [1] opencontainers/runc#3715 Fixes golang#58552. Change-Id: I82a7e757ab3fd3d0193690a65c3b48fee46ff067 Reviewed-on: https://go-review.googlesource.com/c/go/+/468735 Reviewed-by: Damien Neil <[email protected]> TryBot-Result: Gopher Robot <[email protected]> Run-TryBot: Ian Lance Taylor <[email protected]> Auto-Submit: Ian Lance Taylor <[email protected]> Reviewed-by: Ian Lance Taylor <[email protected]>

This is a test case for issue reported as opencontainers#3715. In short, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. Since the upstream golang is also broken (see [1]), let's skip this test for Go 1.20 and 1.20.1 (and hope it'll be fixed in Go 1.20.2 as per [2]. [1] https://go.dev/issue/58552 [2] https://go-review.googlesource.com/c/go/+/469956 Signed-off-by: Kir Kolyshkin <[email protected]>

Since this commit was made, a few things happened: - a similar functionality appeared in go 1.20 [1] - a bug in runc was found [2], which also affects go [3] - the bug was fixed in go 1.21 [4] and 1.20.2 [5] - a similar fix was made to x/sys/unix.Faccessat [6] Revert commit 957d97b so we can fix the bug [2] when go > 1.21.1 is used. Note that this will reintroduce the older bug [7] when the older go version is used, but since this is a minor bug which will be fixed once everyone switches to a recent go version, let's keep things simple and not introduce any complex code here. [1] https://go-review.googlesource.com/c/go/+/414824 [2] opencontainers#3715 [3] https://go.dev/issue/58552 [4] https://go-review.googlesource.com/c/go/+/468735 [5] https://go-review.googlesource.com/c/go/+/469956 [6] https://go-review.googlesource.com/c/sys/+/468877 [7] opencontainers#3520 Signed-off-by: Kir Kolyshkin <[email protected]>

kolyshkin · 2023-02-28T00:14:01Z

OK this is how we're fixing this: #3753

Since this commit was made, a few things happened: - a similar functionality appeared in go 1.20 [1], so the issue mentioned in the comment (being removed) is no longer true; - a bug in runc was found [2], which also affects go [3]; - the bug was fixed in go 1.21 [4] and 1.20.2 [5]; - a similar fix was made to x/sys/unix.Faccessat [6]. Revert commit 957d97b so we can fix the bug [2] when go > 1.21.1 is used. Note that this will reintroduce the older bug [7] when the older go version is used, but since this is a minor bug which will be fixed once everyone switches to a recent go version, let's keep things simple and not introduce any complex code here. [1] https://go-review.googlesource.com/c/go/+/414824 [2] opencontainers#3715 [3] https://go.dev/issue/58552 [4] https://go-review.googlesource.com/c/go/+/468735 [5] https://go-review.googlesource.com/c/go/+/469956 [6] https://go-review.googlesource.com/c/sys/+/468877 [7] opencontainers#3520 Signed-off-by: Kir Kolyshkin <[email protected]>

This is a test case for issue reported as opencontainers#3715. In short, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. Since the upstream golang is also broken (see [1]), let's skip this test for Go 1.20 and 1.20.1 (and hope it'll be fixed in Go 1.20.2 as per [2]. [1] https://go.dev/issue/58552 [2] https://go-review.googlesource.com/c/go/+/469956 Signed-off-by: Kir Kolyshkin <[email protected]>

kolyshkin · 2023-02-28T01:27:16Z

OK, what I guess we'll do is

for Go 1.20.x, do not use own system.Eaccess (introduced in Fix error from runc run on noexec fs #3522)
for older versions, use unix.Faccessat from a future version of x/sys/unix (with https://go-review.googlesource.com/c/sys/+/468877)
add a test case.

This is a test case for issue reported as opencontainers#3715. In short, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. Since the upstream golang is also broken (see [1]), let's skip this test for Go 1.20 and 1.20.1 (and hope it'll be fixed in Go 1.20.2 as per [2]. [1] https://go.dev/issue/58552 [2] https://go-review.googlesource.com/c/go/+/469956 Signed-off-by: Kir Kolyshkin <[email protected]>

…E on Linux CL 416115 added using faccessat2(2) from syscall.Faccessat on Linux (which is the only true way to implement AT_EACCESS flag handing), if available. If not available, it uses some heuristics to mimic the kernel behavior, mostly taken from glibc (see CL 126415). Next, CL 414824 added using the above call (via unix.Eaccess) to exec.LookPath in order to check if the binary can really be executed. As a result, in a very specific scenario, described below, syscall.Faccessat (and thus exec.LookPath) mistakenly tells that the binary can not be executed, while in reality it can be. This makes this bug a regression in Go 1.20. This scenario involves all these conditions: - no faccessat2 support available (i.e. either Linux kernel < 5.8, or a seccomp set up to disable faccessat2); - the current user is not root (i.e. geteuid() != 0); - CAP_DAC_OVERRIDE capability is set for the current process; - the file to be executed does not have executable permission bit set for either the current EUID or EGID; - the file to be executed have at least one executable bit set. Unfortunately, this set of conditions was observed in the wild -- a container run as a non-root user with the binary file owned by root with executable permission set for a user only [1]. Essentially it means it is not as rare as it may seem. Now, CAP_DAC_OVERRIDE essentially makes the kernel bypass most of the checks, so execve(2) and friends work the same was as for root user, i.e. if at least one executable bit it set, the permission to execute is granted (see generic_permission() function in the Linux kernel). Modify the code to check for CAP_DAC_OVERRIDE and mimic the kernel behavior for permission checks. [1] opencontainers/runc#3715 For #58552. Fixes #58624. Change-Id: I82a7e757ab3fd3d0193690a65c3b48fee46ff067 Reviewed-on: https://go-review.googlesource.com/c/go/+/468735 Reviewed-by: Damien Neil <[email protected]> TryBot-Result: Gopher Robot <[email protected]> Run-TryBot: Ian Lance Taylor <[email protected]> Auto-Submit: Ian Lance Taylor <[email protected]> Reviewed-by: Ian Lance Taylor <[email protected]> (cherry picked from commit 031401a) Reviewed-on: https://go-review.googlesource.com/c/go/+/469956 Auto-Submit: Dmitri Shuralyov <[email protected]> Run-TryBot: Than McIntosh <[email protected]> Reviewed-by: Than McIntosh <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]>

…E on Linux CL 416115 added using faccessat2(2) from syscall.Faccessat on Linux (which is the only true way to implement AT_EACCESS flag handing), if available. If not available, it uses some heuristics to mimic the kernel behavior, mostly taken from glibc (see CL 126415). Next, CL 414824 added using the above call (via unix.Eaccess) to exec.LookPath in order to check if the binary can really be executed. As a result, in a very specific scenario, described below, syscall.Faccessat (and thus exec.LookPath) mistakenly tells that the binary can not be executed, while in reality it can be. This makes this bug a regression in Go 1.20. This scenario involves all these conditions: - no faccessat2 support available (i.e. either Linux kernel < 5.8, or a seccomp set up to disable faccessat2); - the current user is not root (i.e. geteuid() != 0); - CAP_DAC_OVERRIDE capability is set for the current process; - the file to be executed does not have executable permission bit set for either the current EUID or EGID; - the file to be executed have at least one executable bit set. Unfortunately, this set of conditions was observed in the wild -- a container run as a non-root user with the binary file owned by root with executable permission set for a user only [1]. Essentially it means it is not as rare as it may seem. Now, CAP_DAC_OVERRIDE essentially makes the kernel bypass most of the checks, so execve(2) and friends work the same was as for root user, i.e. if at least one executable bit it set, the permission to execute is granted (see generic_permission() function in the Linux kernel). Modify the code to check for CAP_DAC_OVERRIDE and mimic the kernel behavior for permission checks. [1] opencontainers/runc#3715 For golang#58552. Fixes golang#58624. Change-Id: I82a7e757ab3fd3d0193690a65c3b48fee46ff067 Reviewed-on: https://go-review.googlesource.com/c/go/+/468735 Reviewed-by: Damien Neil <[email protected]> TryBot-Result: Gopher Robot <[email protected]> Run-TryBot: Ian Lance Taylor <[email protected]> Auto-Submit: Ian Lance Taylor <[email protected]> Reviewed-by: Ian Lance Taylor <[email protected]> (cherry picked from commit 031401a) Reviewed-on: https://go-review.googlesource.com/c/go/+/469956 Auto-Submit: Dmitri Shuralyov <[email protected]> Run-TryBot: Than McIntosh <[email protected]> Reviewed-by: Than McIntosh <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]>

Since commit 957d97b was made to fix issue [7], a few things happened: - a similar functionality appeared in go 1.20 [1], so the issue mentioned in the comment (being removed) is no longer true; - a bug in runc was found [2], which also affects go [3]; - the bug was fixed in go 1.21 [4] and 1.20.2 [5]; - a similar fix was made to x/sys/unix.Faccessat [6]. The essense of [2] is, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. To fix this [2] without reintroducing the older bug [7]: - drop own Eaccess implementation; - use the one from x/sys/unix for Go 1.19 (depends on [6]); - do not use anything when Go 1.20+ is used. NOTE it is virtually impossible to fix the bug [2] when Go 1.20 or Go 1.20.1 is used because of [3]. A test case is added by a separate commit. Fixes: opencontainers#3715. [1] https://go-review.googlesource.com/c/go/+/414824 [2] opencontainers#3715 [3] https://go.dev/issue/58552 [4] https://go-review.googlesource.com/c/go/+/468735 [5] https://go-review.googlesource.com/c/go/+/469956 [6] https://go-review.googlesource.com/c/sys/+/468877 [7] opencontainers#3520 Signed-off-by: Kir Kolyshkin <[email protected]>

This is a test case for issue reported as opencontainers#3715. In short, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. Since the upstream golang is also broken (see [1]), let's skip this test for Go 1.20 and 1.20.1 (as it's fixed in Go 1.20.2 as per [2]). [1] https://go.dev/issue/58552 [2] https://go-review.googlesource.com/c/go/+/469956 Signed-off-by: Kir Kolyshkin <[email protected]>

Since commit 957d97b was made to fix issue [7], a few things happened: - a similar functionality appeared in go 1.20 [1], so the issue mentioned in the comment (being removed) is no longer true; - a bug in runc was found [2], which also affects go [3]; - the bug was fixed in go 1.21 [4] and 1.20.2 [5]; - a similar fix was made to x/sys/unix.Faccessat [6]. The essense of [2] is, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. To fix this [2] without reintroducing the older bug [7]: - drop own Eaccess implementation; - use the one from x/sys/unix for Go 1.19 (depends on [6]); - do not use anything when Go 1.20+ is used. NOTE it is virtually impossible to fix the bug [2] when Go 1.20 or Go 1.20.1 is used because of [3]. A test case is added by a separate commit. Fixes: opencontainers#3715. [1] https://go-review.googlesource.com/c/go/+/414824 [2] opencontainers#3715 [3] https://go.dev/issue/58552 [4] https://go-review.googlesource.com/c/go/+/468735 [5] https://go-review.googlesource.com/c/go/+/469956 [6] https://go-review.googlesource.com/c/sys/+/468877 [7] opencontainers#3520 Signed-off-by: Kir Kolyshkin <[email protected]>

This is a test case for issue reported as opencontainers#3715. In short, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. Since the upstream golang is also broken (see [1]), let's skip this test for Go 1.20 and 1.20.1 (as it's fixed in Go 1.20.2 as per [2]). [1] https://go.dev/issue/58552 [2] https://go-review.googlesource.com/c/go/+/469956 Signed-off-by: Kir Kolyshkin <[email protected]>

Since commit 957d97b was made to fix issue [7], a few things happened: - a similar functionality appeared in go 1.20 [1], so the issue mentioned in the comment (being removed) is no longer true; - a bug in runc was found [2], which also affects go [3]; - the bug was fixed in go 1.21 [4] and 1.20.2 [5]; - a similar fix was made to x/sys/unix.Faccessat [6]. The essense of [2] is, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. To fix this [2] without reintroducing the older bug [7]: - drop own Eaccess implementation; - use the one from x/sys/unix for Go 1.19 (depends on [6]); - do not use anything when Go 1.20+ is used. NOTE it is virtually impossible to fix the bug [2] when Go 1.20 or Go 1.20.1 is used because of [3]. A test case is added by a separate commit. Fixes: opencontainers#3715. [1] https://go-review.googlesource.com/c/go/+/414824 [2] opencontainers#3715 [3] https://go.dev/issue/58552 [4] https://go-review.googlesource.com/c/go/+/468735 [5] https://go-review.googlesource.com/c/go/+/469956 [6] https://go-review.googlesource.com/c/sys/+/468877 [7] opencontainers#3520 Signed-off-by: Kir Kolyshkin <[email protected]>

This is a test case for issue reported as opencontainers#3715. In short, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. Since the upstream golang is also broken (see [1]), let's skip this test for Go 1.20 and 1.20.1 (as it's fixed in Go 1.20.2 as per [2]). [1] https://go.dev/issue/58552 [2] https://go-review.googlesource.com/c/go/+/469956 Signed-off-by: Kir Kolyshkin <[email protected]>

kolyshkin · 2023-03-17T16:59:45Z

Fixed by #3753 (which needs to be merged and backported to 1.1)

Since commit 957d97b was made to fix issue [7], a few things happened: - a similar functionality appeared in go 1.20 [1], so the issue mentioned in the comment (being removed) is no longer true; - a bug in runc was found [2], which also affects go [3]; - the bug was fixed in go 1.21 [4] and 1.20.2 [5]; - a similar fix was made to x/sys/unix.Faccessat [6]. The essense of [2] is, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. To fix this [2] without reintroducing the older bug [7]: - drop own Eaccess implementation; - use the one from x/sys/unix for Go 1.19 (depends on [6]); - do not use anything when Go 1.20+ is used. NOTE it is virtually impossible to fix the bug [2] when Go 1.20 or Go 1.20.1 is used because of [3]. A test case is added by a separate commit. Fixes: opencontainers#3715. [1] https://go-review.googlesource.com/c/go/+/414824 [2] opencontainers#3715 [3] https://go.dev/issue/58552 [4] https://go-review.googlesource.com/c/go/+/468735 [5] https://go-review.googlesource.com/c/go/+/469956 [6] https://go-review.googlesource.com/c/sys/+/468877 [7] opencontainers#3520 Signed-off-by: Kir Kolyshkin <[email protected]>

This is a test case for issue reported as opencontainers#3715. In short, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. Note that since the upstream golang is also broken (see [1]), this test will fail for Go 1.20 and 1.20.1 (fix is in Go 1.20.2 as per [2]). [1] https://go.dev/issue/58552 [2] https://go-review.googlesource.com/c/go/+/469956 Signed-off-by: Kir Kolyshkin <[email protected]>

kolyshkin · 2023-03-28T16:30:55Z

Not fixed until we do a backport to release-1.1 (in progress).

1. bump golang.org/x/sys to v0.6.0; 2. bump golang.org/x/net to v0.8.0 3. require Go 1.17. Newer x/sys is needed to fix [1]. Go 1.17 is needed because x/sys/unix is using unsafe.Slice which requires go1.17 or later. [1] opencontainers#3715 Signed-off-by: Kir Kolyshkin <[email protected]>

kolyshkin · 2023-03-28T17:06:15Z

As this is a rather rare bug, and we want to release 1.1.5 ASAP due to a couple of CVEs (see #3789), let's fix this issue after 1.1.5 is out.

1. bump golang.org/x/sys to v0.6.0; 2. bump golang.org/x/net to v0.8.0 3. require Go 1.17. Newer x/sys is needed to fix [1]. Go 1.17 is needed because x/sys/unix is using unsafe.Slice which requires go1.17 or later. [1] opencontainers#3715 Signed-off-by: Kir Kolyshkin <[email protected]>

1. bump golang.org/x/sys to v0.6.0; 2. bump golang.org/x/net to v0.8.0 3. require Go 1.17. Newer x/sys is needed to fix [1]. Go 1.17 is needed because x/sys/unix is using unsafe.Slice which requires go1.17 or later. This reuses parts of main commit a0f8847. [1] opencontainers#3715 Signed-off-by: Kir Kolyshkin <[email protected]>

Since commit 957d97b was made to fix issue [7], a few things happened: - a similar functionality appeared in go 1.20 [1], so the issue mentioned in the comment (being removed) is no longer true; - a bug in runc was found [2], which also affects go [3]; - the bug was fixed in go 1.21 [4] and 1.20.2 [5]; - a similar fix was made to x/sys/unix.Faccessat [6]. The essense of [2] is, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. To fix this [2] without reintroducing the older bug [7]: - drop own Eaccess implementation; - use the one from x/sys/unix for Go 1.19 (depends on [6]); - do not use anything when Go 1.20+ is used. NOTE it is virtually impossible to fix the bug [2] when Go 1.20 or Go 1.20.1 is used because of [3]. A test case is added by a separate commit. Fixes: opencontainers#3715. [1] https://go-review.googlesource.com/c/go/+/414824 [2] opencontainers#3715 [3] https://go.dev/issue/58552 [4] https://go-review.googlesource.com/c/go/+/468735 [5] https://go-review.googlesource.com/c/go/+/469956 [6] https://go-review.googlesource.com/c/sys/+/468877 [7] opencontainers#3520 Signed-off-by: Kir Kolyshkin <[email protected]> (cherry picked from commit 8491d33) Signed-off-by: Kir Kolyshkin <[email protected]>

This is a test case for issue reported as opencontainers#3715. In short, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. Note that since the upstream golang is also broken (see [1]), this test will fail for Go 1.20 and 1.20.1 (fix is in Go 1.20.2 as per [2]). [1] https://go.dev/issue/58552 [2] https://go-review.googlesource.com/c/go/+/469956 Signed-off-by: Kir Kolyshkin <[email protected]> (cherry picked from commit 8293ef2) Signed-off-by: Kir Kolyshkin <[email protected]>

Since commit 957d97b was made to fix issue [7], a few things happened: - a similar functionality appeared in go 1.20 [1], so the issue mentioned in the comment (being removed) is no longer true; - a bug in runc was found [2], which also affects go [3]; - the bug was fixed in go 1.21 [4] and 1.20.2 [5]; - a similar fix was made to x/sys/unix.Faccessat [6]. The essense of [2] is, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. To fix this [2] without reintroducing the older bug [7]: - drop own Eaccess implementation; - use the one from x/sys/unix for Go 1.19 (depends on [6]); - do not use anything when Go 1.20+ is used. NOTE it is virtually impossible to fix the bug [2] when Go 1.20 or Go 1.20.1 is used because of [3]. A test case is added by a separate commit. Fixes: opencontainers#3715. [1] https://go-review.googlesource.com/c/go/+/414824 [2] opencontainers#3715 [3] https://go.dev/issue/58552 [4] https://go-review.googlesource.com/c/go/+/468735 [5] https://go-review.googlesource.com/c/go/+/469956 [6] https://go-review.googlesource.com/c/sys/+/468877 [7] opencontainers#3520 Signed-off-by: Kir Kolyshkin <[email protected]> (cherry picked from commit 8491d33) Signed-off-by: Kir Kolyshkin <[email protected]>

This is a test case for issue reported as opencontainers#3715. In short, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. Note that since the upstream golang is also broken (see [1]), this test will fail for Go 1.20 and 1.20.1 (fix is in Go 1.20.2 as per [2]). [1] https://go.dev/issue/58552 [2] https://go-review.googlesource.com/c/go/+/469956 Signed-off-by: Kir Kolyshkin <[email protected]> (cherry picked from commit 8293ef2) Signed-off-by: Kir Kolyshkin <[email protected]>

kolyshkin · 2023-04-07T01:40:57Z

Fixed in 1.1 by #3817

Since commit e2100c8 was made to fix issue [7], a few things happened: - a similar functionality appeared in go 1.20 [1], so the issue mentioned in the comment (being removed) is no longer true; - a bug in runc was found [2], which also affects go [3]; - the bug was fixed in go 1.21 [4] and 1.20.2 [5]; - a similar fix was made to x/sys/unix.Faccessat [6]. The essense of [2] is, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. To fix this [2] without reintroducing the older bug [7]: - drop own Eaccess implementation; - use the one from x/sys/unix for Go 1.19 (depends on [6]); - do not use anything when Go 1.20+ is used. NOTE it is virtually impossible to fix the bug [2] when Go 1.20 or Go 1.20.1 is used because of [3]. A test case is added by a separate commit. Fixes: #3715. [1] https://go-review.googlesource.com/c/go/+/414824 [2] opencontainers/runc#3715 [3] https://go.dev/issue/58552 [4] https://go-review.googlesource.com/c/go/+/468735 [5] https://go-review.googlesource.com/c/go/+/469956 [6] https://go-review.googlesource.com/c/sys/+/468877 [7] opencontainers/runc#3520 Signed-off-by: Kir Kolyshkin <[email protected]>

1. bump golang.org/x/sys to v0.6.0; 2. bump golang.org/x/net to v0.8.0 3. require Go 1.17. Newer x/sys is needed to fix [1]. Go 1.17 is needed because x/sys/unix is using unsafe.Slice which requires go1.17 or later. This reuses parts of main commit a0f8847. [1] opencontainers/runc#3715 Signed-off-by: Kir Kolyshkin <[email protected]>

Since commit 957d97b was made to fix issue [7], a few things happened: - a similar functionality appeared in go 1.20 [1], so the issue mentioned in the comment (being removed) is no longer true; - a bug in runc was found [2], which also affects go [3]; - the bug was fixed in go 1.21 [4] and 1.20.2 [5]; - a similar fix was made to x/sys/unix.Faccessat [6]. The essense of [2] is, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. To fix this [2] without reintroducing the older bug [7]: - drop own Eaccess implementation; - use the one from x/sys/unix for Go 1.19 (depends on [6]); - do not use anything when Go 1.20+ is used. NOTE it is virtually impossible to fix the bug [2] when Go 1.20 or Go 1.20.1 is used because of [3]. A test case is added by a separate commit. Fixes: #3715. [1] https://go-review.googlesource.com/c/go/+/414824 [2] opencontainers/runc#3715 [3] https://go.dev/issue/58552 [4] https://go-review.googlesource.com/c/go/+/468735 [5] https://go-review.googlesource.com/c/go/+/469956 [6] https://go-review.googlesource.com/c/sys/+/468877 [7] opencontainers/runc#3520 Signed-off-by: Kir Kolyshkin <[email protected]> (cherry picked from commit 8491d33) Signed-off-by: Kir Kolyshkin <[email protected]>

1. bump golang.org/x/sys to v0.6.0; 2. bump golang.org/x/net to v0.8.0 3. require Go 1.17. Newer x/sys is needed to fix [1]. Go 1.17 is needed because x/sys/unix is using unsafe.Slice which requires go1.17 or later. This reuses parts of main commit 03e2e4429f04c418a81. [1] opencontainers/runc#3715 Signed-off-by: Kir Kolyshkin <[email protected]>

Since commit a32275ab054d34d46683 was made to fix issue [7], a few things happened: - a similar functionality appeared in go 1.20 [1], so the issue mentioned in the comment (being removed) is no longer true; - a bug in runc was found [2], which also affects go [3]; - the bug was fixed in go 1.21 [4] and 1.20.2 [5]; - a similar fix was made to x/sys/unix.Faccessat [6]. The essense of [2] is, even if a (non-root) user that the container is run as does not have execute permission bit set for the executable, it should still work in case runc has the CAP_DAC_OVERRIDE capability set. To fix this [2] without reintroducing the older bug [7]: - drop own Eaccess implementation; - use the one from x/sys/unix for Go 1.19 (depends on [6]); - do not use anything when Go 1.20+ is used. NOTE it is virtually impossible to fix the bug [2] when Go 1.20 or Go 1.20.1 is used because of [3]. A test case is added by a separate commit. Fixes: #3715. [1] https://go-review.googlesource.com/c/go/+/414824 [2] opencontainers/runc#3715 [3] https://go.dev/issue/58552 [4] https://go-review.googlesource.com/c/go/+/468735 [5] https://go-review.googlesource.com/c/go/+/469956 [6] https://go-review.googlesource.com/c/sys/+/468877 [7] opencontainers/runc#3520 Signed-off-by: Kir Kolyshkin <[email protected]> (cherry picked from commit b7fff20cb91cc48e3995a86fedfa10d7baa17c4c) Signed-off-by: Kir Kolyshkin <[email protected]>

AkihiroSuda added kind/bug regression labels Jan 31, 2023

AkihiroSuda mentioned this issue Feb 8, 2023

Fix error from runc run on noexec fs #3522

Merged

kolyshkin added this to the 1.1.5 milestone Feb 9, 2023

kolyshkin mentioned this issue Feb 28, 2023

Fix runc run "permission denied" when rootless #3753

Merged

AkihiroSuda closed this as completed in #3753 Mar 28, 2023

kolyshkin reopened this Mar 28, 2023

kolyshkin mentioned this issue Mar 28, 2023

[1.1] refresh ci #3791

Merged

kolyshkin modified the milestones: 1.1.5, 1.1.6 Mar 28, 2023

kolyshkin mentioned this issue Mar 28, 2023

Release v1.1.5 (Fix CVE-2023-27561, CVE-2023-25809, CVE-2023-28642) #3789

Closed

kolyshkin mentioned this issue Apr 6, 2023

[1.1] Fix runc run "permission denied" when rootless #3817

Merged

kolyshkin closed this as completed Apr 7, 2023

kolyshkin mentioned this issue Apr 7, 2023

Release runc 1.1.6 #3818

Merged

cyphar mentioned this issue Jul 5, 2023

If we use a .sh or .py(-rwxr-xr--.) has defined #!/bin/sh or #!/usr/bin/python3 as the pid=1 of the container, starting the container with a normal user without permissions will report an error "permission denied" #3926

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Runc 1.1.4 container fails with `permission denied: unknown` when relying on capabilities #3715

Runc 1.1.4 container fails with `permission denied: unknown` when relying on capabilities #3715

Kern-- commented Jan 30, 2023

kolyshkin commented Feb 15, 2023

kolyshkin commented Feb 16, 2023

kolyshkin commented Feb 21, 2023

kolyshkin commented Feb 28, 2023

kolyshkin commented Feb 28, 2023

kolyshkin commented Mar 17, 2023

kolyshkin commented Mar 28, 2023

kolyshkin commented Mar 28, 2023

kolyshkin commented Apr 7, 2023

Runc 1.1.4 container fails with permission denied: unknown when relying on capabilities #3715

Runc 1.1.4 container fails with permission denied: unknown when relying on capabilities #3715

Comments

Kern-- commented Jan 30, 2023

kolyshkin commented Feb 15, 2023

kolyshkin commented Feb 16, 2023

kolyshkin commented Feb 21, 2023

kolyshkin commented Feb 28, 2023

kolyshkin commented Feb 28, 2023

kolyshkin commented Mar 17, 2023

kolyshkin commented Mar 28, 2023

kolyshkin commented Mar 28, 2023

kolyshkin commented Apr 7, 2023

Runc 1.1.4 container fails with `permission denied: unknown` when relying on capabilities #3715

Runc 1.1.4 container fails with `permission denied: unknown` when relying on capabilities #3715