Permissions issue with VirtioFS

[abentele]

• I have tried with the latest version of Docker Desktop
• I have tried disabling enabled experimental features
• I have uploaded Diagnostics
• Diagnostics ID:

Summary / Steps to reproduce

I started a gitlab container with docker compose.
This container has a volume:

<some-local-folder>:/var/opt/gitlab:cached

what I did in the running container (note the commands access a sub-folder of the volume):

chmod 2770 /var/opt/gitlab/git-data/repositories
stat --printf='%04a' $(readlink -f /var/opt/gitlab/git-data/repositories) | grep -o '....$'

Expected behavior

I would expect the permissions of this folder are as set by the chmod command before:

Therefore the output of the stat command should be:

2770

Actual behavior

output of the stat command with setting "gRPC FUSE" of Docker Desktop (this is ok):

2770

output of the stat command with setting "VirtioFS" of Docker Desktop (this is wrong!):

0770

Btw.: gitlab fails on startup because of this issue.

Information

The problem should be reproducible with any other container.
The problem is since I switched my settings to VirtioFS. With setting gRPC FUSE it works.

• macOS Version: 13.0.1
• Intel chip or Apple chip: Intel
• Docker Desktop Version: 4.15.0 (93002)

Output of /Applications/Docker.app/Contents/MacOS/com.docker.diagnose check

Starting diagnostics

[PASS] DD0027: is there available disk space on the host?
[PASS] DD0028: is there available VM disk space?
[PASS] DD0018: does the host support virtualization?
[PASS] DD0001: is the application running?
[PASS] DD0017: can a VM be started?
[PASS] DD0016: is the LinuxKit VM running?
[PASS] DD0011: are the LinuxKit services running?
[PASS] DD0004: is the Docker engine running?
[PASS] DD0015: are the binary symlinks installed?
[PASS] DD0031: does the Docker API work?
[PASS] DD0013: is the $PATH ok?
[PASS] DD0003: is the Docker CLI working?
[PASS] DD0014: are the backend processes running?
[PASS] DD0007: is the backend responding?
[PASS] DD0008: is the native API responding?
[PASS] DD0009: is the vpnkit API responding?
[PASS] DD0010: is the Docker API proxy responding?
[PASS] DD0012: is the VM networking working?
[SKIP] DD0030: is the image access management authorized?
[FAIL] DD0019: is the com.docker.vmnetd process responding? failed to ping vmnetd with error: failed to connect to /var/run/com.docker.vmnetd.sock: is vmnetd running?: dial unix /var/run/com.docker.vmnetd.sock: connect: no such file or directory
[PASS] DD0033: does the host have Internet access?
[PASS] DD0018: does the host support virtualization?
[PASS] DD0001: is the application running?
[PASS] DD0017: can a VM be started?
[PASS] DD0016: is the LinuxKit VM running?
[PASS] DD0011: are the LinuxKit services running?
[PASS] DD0004: is the Docker engine running?
[PASS] DD0015: are the binary symlinks installed?
[PASS] DD0031: does the Docker API work?
[PASS] DD0032: do Docker networks overlap with host IPs?

Please investigate the following 1 issue:

1 : The test: is the com.docker.vmnetd process responding?
    Failed with: failed to ping vmnetd with error: failed to connect to /var/run/com.docker.vmnetd.sock: is vmnetd running?: dial unix /var/run/com.docker.vmnetd.sock: connect: no such file or directory

The com.docker.vmnetd process is needed to create symlinks for CLIs in your path.

Steps to reproduce the behavior

See above.

[niccolomineo]

Still testing this, but might be related to what the OP is experiencing: since updating to 4.15, when I git pull the files are set to 755 instead of 644. My umask seems to be defaulting to what I expect (0022). I don't seem to have any problems with the other virtualization modes available.

[thaJeztah]

Thanks for reporting; I just hit this issue myself; I'm discussing with the team working on VirtioFS, and they're looking into it.

[fasmat]

Same here. For some reason all files inside my devcontainer are now marked as executable after a git pull.

[bramru]

Same here, rolling back to 4.14.1 solves the problem

[thaJeztah]

Short update; I see a patch is being worked on to fix this (but not finished yet); from the description, it looks like there may be a bug on Apple's side, but we're looking at a workaround until that's fixed.

[markeissler]

I can also confirm this issue. It's really creating havoc while working in a VSCode devcontainer. Files randomly reset their permissions from 0644 to 0755 in the container. Then if try to run a git status in the host OS (macOS Monterey) git thinks ALL of the files have changed (or a ton of them) and a git diff indicates that the permissions have changed (from 0755 to 0644). This might be impacting my ability to rebase as well but not sure.

[markeissler]

Switched back to "gRPC FUSE" for now. Seems like a good enough work around until a fix is in place.

[fredericdalleau]

Hey there, this build no longer sets the executable bit on your volume files :
amd64: https://desktop-stage.docker.com/mac/main/amd64/94201/Docker.dmg
arm64: https://desktop-stage.docker.com/mac/main/arm64/94201/Docker.dmg
Let us know if any issue occurs, Thanks!

[fredericdalleau]

@abentele note that this executable bit issue is different for the issue you're facing. Investigation continues.

[oinopion]

@fredericdalleau my team experienced the same problem with permissions in dev container on 4.15. The build that you've linked resolves the problem for us 👍

Is this likely to be released soon?

[fredericdalleau]

@oinopion thanks for feedback, it should land in our next release early next year

[umbrellait-danil-rodin]

@fredericdalleau Thanks a lot for this Docker build!

[niccolomineo]

4.16 preview works like a charm on my system, thank you for that. 👍

[mat007]

Closing this issue because a fix has been released in Docker Desktop 4.16. See the release notes for more details.

[pirog]

Sadly we are still having issues with 4.16.1 over at https://github.com/lando/lando which is unfortunate because our users definitely crave being able to use VirtioFS.

4.16.1 produces breaking behavior because it does not mount our boot up entrypoint scripts with the same permissions they have on the host.

This is what the perms look like for our scripts. Is this intentional?

total 544
     0 drwxr-xr-x  134 501      dialout       4288 Nov 18 19:53 .
     4 drwxr-xr-x    1 root     root          4096 Jan 13 18:15 ..
     8 -rwxr-xr-x    1 501      dialout       6148 Oct 26  2021 .DS_Store
     4 -------r--    1 501      dialout       1248 May  6  2020 999-proxy-certs.sh
     4 -------r--    1 501      dialout        811 Jan 13 18:15 acquia-clone.sh
     4 -------r--    1 501      dialout        242 Jan 13 18:15 acquia-config-symlink.sh
     4 -------r--    1 501      dialout        322 Jan 13 18:15 acquia-generate-key.sh
     4 -------r--    1 501      dialout       1188 Mar 17  2021 acquia-get-remote-url.sh
     4 -------r--    1 501      dialout       3152 Jan 13 18:15 acquia-pull.sh
     4 -------r--    1 501      dialout       2087 Jan 13 18:15 acquia-push.sh
     4 -------r--    1 501      dialout        509 Mar 13  2021 acquia-settings.inc
     8 -------r--    1 501      dialout       4319 Jan 13 18:15 add-cert.sh
     4 -------r--    1 501      dialout       2920 Sep 15  2021 agent.sh
     4 -------r--    1 501      dialout       1456 Jan 13 18:15 auth.sh
...

[jalaziz]

Can confirm issues with 4.16.1 as well.

We're building protobuf files on a host-mounted volume which results in temp files with no permissions (which ultimately leads to a proto build failure):

---------- 1 jalaziz staff     0 Jan 15 02:10 sednOBSOT

[fredericdalleau]

@pirog, can you share what command you used to get this result?

[pirog]

@fredericdalleau sadly i cannot!

I reset Docker Desktop to factory defaults and rebooted my computer and now it seems to work as expected. At least for me. Does that make any sense to you?

[fredericdalleau]

You may need to reenable virtiofs as it not default. If it happens again, you can try on the host : xattr -l <filename> and see there is any output.
If so you can clear it with xattr -c <filename>, and set the permissions again. (and share the output here)

[pirog]

@fredericdalleau good to know.

We have some users reporting the same initial no-perms behavior so i'm going to walk through it with a few of them and hopefully i'll be able to track things down a bit better for you all.

[xEtherealx]

@fredericdalleau sorry I wasn't super clear -- the build commands need to be run in the host (within the mounted directory) and the first creates the image, the second executes the build within the container. Does that resolve your issue?

[fredericdalleau]

@xEtherealx the first command succeeds, the second fails with that error. make V=1 bmc2837-config also fails with the same error. Is batocera-bcm2837_defconfig in the right place?

[fredericdalleau]

@thomas-schwieters we tried an arm mac running mac OS Ventura and it worked. An Intel mac running macOS 12.5.1 reproduced the error. What OS architecture and version do you run?

[thomas-schwieters]

that is really strange. I am running a Mac mini M1 2020 with Ventura 13.2 (Docker Engine v20.10.22).
It's really strange, because the error appears always for me. When I delete the folder and try again from scratch, it's the same issue.
I also have a MacBook Air M1, I can try to reproduce on that machine.

[thomas-schwieters]

I just tried from my MacBook. Same issue, see screenshot.

[thomas-schwieters]

Hmm, on my MacBook Air it seems to be a different error message. But anyhow, it's related to VirtioFS, because as soon as I switch to gRPC Fuse, the command works

[nulllpoint]

docker desktop for mac 4.17.0 (99724), the problem is still.
In addition, 'mysql/mysql-server' has the same problem. At present, only grpc-fuse can be used.

[aguckenber-chwy]

Still seeing this issue

[Ch4s3r]

I also got some permission problems when running Kibana and Elastic with docker-compose:

FATAL  Error: EACCES: permission denied, open 'config/certs/ca/ca.crt'

Switching off containerd helped in my case.
Also tested some scenarios and it showed that whenever I activated containerd it was going to fail with permission problems.

grpc fuse + containerd      = failure
virtiofs  + containerd      = failure

grpc fuse + no containerd   = works
virtiofs  + no containerd   = works

Maybe this also applies to others here?

[anthonymich01]

Docker Desktop 4.23.0 (120376) on Intel MacOS Ventura 13.5.1

Use Virtualization framework making Segment fault Error 139 for React on Rails - shakapacker

VirtioFS + Virtualization framework       => Fail
gRPC     + Virtualization framework       => Fail
gRPC     + Uncheck Virtualization frmwrk  => Succeed

[niccolomineo]

Docker Desktop 4.23.0 (120376) on Intel MacOS Ventura 13.5.1

Use Virtualization framework making Segment fault Error 139 for React on Rails - shakapacker

VirtioFS + Virtualization framework       => Fail
gRPC     + Virtualization framework       => Fail
gRPC     + Uncheck Virtualization frmwrk  => Succeed

I just got the heads-up of the upgrade and now I am scared of going further.

[rubysolo]

I think I'm running into this issue as well -- Desktop 4.23.0 on MacOS Sonoma 14.0 (was also happening on Ventura) on M1. I run a container that bind-mounts my ~/.kube directory and uses the k8s Go SDK to switch contexts. This process creates a temporary lockfile to serialize updates to the config file.

When VirtioFS is enabled, it appears to be creating the lockfile with no/invalid permissions bits:

ls -l output from inside the container:

-????????? ? ?    ?        ?            ? config.lock

ls -l output from the host system:

---------- 1 [myuser] [mygroup] 0 Sep 14 21:13 config.lock

Then something (likely removal of the lockfile) fails with a permission denied error.

When the file sharing implementation is switched to gRPC FUSE, it works correctly.

[kalevitan]

Still seeing this permissions issue only when VirtioFS is enabled when trying to add node as a composer dependency to my Lando project. Node is added by pantheon-se/node-composer.

An error occurred while extracting (/app/vendor/node-v14.18.1-linux-arm64.tar.gz) to /app/vendor/node-v14.18.1-linux-arm64: tar: bin/npx: Cannot open: Permission denied
  tar: bin/npm: Cannot open: Permission denied
  tar: Exiting with failure status due to previous errors

[llimllib]

An easy way to reproduce is to try and run gitlab under virtio:

sudo docker run --detach \
  --hostname gitlab.example.com \
  --publish 443:443 --publish 80:80 --publish 22:22 \
  --name gitlab \
  --restart always \
  --volume $HOME/gitlab/config:/etc/gitlab \
  --volume $HOME/gitlab/logs:/var/log/gitlab \
  --volume $HOME/gitlab/data:/var/opt/gitlab \
  --shm-size 256m \
  gitlab/gitlab-ee:latest

After the setup process completes, you'll find postgres logs in $HOME/gitlab/logs/postgresql/ with the error:

2023-10-25_15:06:54.26177 LOG:  starting PostgreSQL 13.11 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, 64-bit
2023-10-25_15:06:54.26436 LOG:  could not set permissions of file "/var/opt/gitlab/postgresql/.s.PGSQL.5432": Invalid argument
2023-10-25_15:06:54.26451 WARNING:  could not create Unix-domain socket in directory "/var/opt/gitlab/postgresql"
2023-10-25_15:06:54.26461 FATAL:  could not create any Unix-domain sockets

[renkosteenbeek]

Using OrbStack does solve these issues for me. It's much faster then Docker Desktop too. http://orbstack.dev

[nulllpoint]

Using OrbStack does solve these issues for me. It's much faster then Docker Desktop too. http://orbstack.dev

I just tried it.
OrbStack is just a launcher, which has nothing to do with virtirfs......

[mcabreb]

Hello there, it has been more than a year with this bug. From release 4.14 to 4.27. gRPC FUSE is much slower than VirtioFS... Any news on the issue?

[carlosfrodriguez]

This is also reproducible with Docker Mailserver https://docker-mailserver.github.io/docker-mailserver/latest/

Apple Silicon Mac, Docker Desktop 4.29.0

[bkline]

@thaJeztah

I'm discussing with the team working on VirtioFS, and they're looking into it.

That was back in 2022. Are they still looking into it? Or have they decided this isn't a very important bug, and moved on to other things?

[miamilabs]

@thaJeztah

I'm discussing with the team working on VirtioFS, and they're looking into it.

That was back in 2022. Are they still looking into it? Or have they decided this isn't a very important bug, and moved on to other things?

imagine having mac minis as amazing servers which consume almost no power. But docker still hangs years back.
Just seen this issue, will be fixed provably in 2034, hosted mode took years after all.