Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to start virtiofsd error wtih lxd 5.21.1 LTS #13325

Closed
aludman opened this issue Apr 12, 2024 · 2 comments · Fixed by #13332
Closed

Failed to start virtiofsd error wtih lxd 5.21.1 LTS #13325

aludman opened this issue Apr 12, 2024 · 2 comments · Fixed by #13332
Assignees
Labels
Bug Confirmed to be a bug
Milestone

Comments

@aludman
Copy link

aludman commented Apr 12, 2024

Required information

  • Distribution: Ubuntu
  • Distribution version: 20.04.6 LTS
  • The output of "snap list --all lxd core20 core22 core24 snapd":
Name    Version       Rev    Tracking       Publisher   Notes
core20  20240227      2264   latest/stable  canonical✓  base
core20  20240111      2182   latest/stable  canonical✓  base,disabled
core22  20231123      1033   latest/stable  canonical✓  base,disabled
core22  20240111      1122   latest/stable  canonical✓  base
lxd     5.20-f3dd836  27049  5.20/stable    canonical✓  held
snapd   2.61.2        21184  latest/stable  canonical✓  snapd
snapd   2.61.1        20671  latest/stable  canonical✓  snapd,disabled
  • The output of "lxc info":
    Note that this is from after we have reinstalled 5.20 since I do not want to go back and break ths system just so I can get this information. The problem is very easily reproduced using the steps further below.

lxcinfo.txt

Issue description

We're unable start / exec to our lxd vms after an automatic refresh of lxd snap from 5.20 to 5.21.1.

The error we get is the following:

Error: Failed to start device "SHARED_DIR": Failed to setup virtiofsd for device "SHARED_DIR": Failed to start virtiofsd: Unable to start process: fork/exec /snap/lxd/28156/bin/virtiofsd: invalid argument

I was expecting the system to work as before after the automatic update.

Reverting back to 5.20 using sudo snap revert lxd resulted in lxd not working at all, so we ended up uninstalling lxd completely and reinstalling the 5.20 version and recreating everything from scratch, which resolved our issue.

Steps to reproduce

The problem is reproducible on our system by following these basic steps:

sudo snap install lxd
lxd init --minimal
lxc init ubuntu:22.04 --vm test
lxc config set test limits.cpu=8 limits.memory=4GiB
echo -en "both $(id -u) 0" | lxc config set test raw.idmap -
lxc config device add test SHARED_DIR disk source="/tmp" path="/mnt/tmp"
lxc start test
lxc exec test -- /bin/bash

The key step is the lxc config set test raw.idmap step. If I skip that then the instance starts up just fine.
The lxc config set test limits.cpu=8 limits.memory=4GiB step is probably not important but it was part our steps, so I've included it.

@tomponline tomponline added the Bug Confirmed to be a bug label Apr 13, 2024
@tomponline tomponline added this to the lxd-6.1 milestone Apr 13, 2024
mihalicyn added a commit to mihalicyn/lxd that referenced this issue Apr 15, 2024
…mapping only if it's required

Fix diskAddRootUserNSEntry logic to correcly handle case when
we have not-groupped UID/GID mapping like this:
[{true false 1000 0 1} {false true 1000 0 1}]

because right now it assumes that mapping should be like
[{true true 1000 0 1}]

which is logically equivalent.

This become noticable after my change in canonical#12718, before that
this issue in logic was reproducible only if user will do something like this:
  cat << EOF
uid $(id -u) 1000000
gid $(id -g) 1000000
EOF
  ) | lxc config set idmap raw.idmap -
which is an extremely rare usecase.

Fixes: canonical#13325
Signed-off-by: Alexander Mikhalitsyn <[email protected]>
@mihalicyn
Copy link
Member

Hi @aludman !

Reverting back to 5.20 using sudo snap revert lxd resulted in lxd not working at all, so we ended up uninstalling lxd completely and reinstalling the 5.20 version and recreating everything from scratch, which resolved our issue.

Ugh, this is really painful. Sorry about that.

I have created a PR with the fix for this problem #13332

tomponline pushed a commit to tomponline/lxd that referenced this issue Apr 16, 2024
…mapping only if it's required

Fix diskAddRootUserNSEntry logic to correcly handle case when
we have not-groupped UID/GID mapping like this:
[{true false 1000 0 1} {false true 1000 0 1}]

because right now it assumes that mapping should be like
[{true true 1000 0 1}]

which is logically equivalent.

This become noticable after my change in canonical#12718, before that
this issue in logic was reproducible only if user will do something like this:
  cat << EOF
uid $(id -u) 1000000
gid $(id -g) 1000000
EOF
  ) | lxc config set idmap raw.idmap -
which is an extremely rare usecase.

Fixes: canonical#13325
Signed-off-by: Alexander Mikhalitsyn <[email protected]>
@tomponline
Copy link
Member

This is now cherry-picked into the 5.21/stable snap channel.

tomponline pushed a commit to tomponline/lxd that referenced this issue Apr 17, 2024
…mapping only if it's required

Fix diskAddRootUserNSEntry logic to correcly handle case when
we have not-groupped UID/GID mapping like this:
[{true false 1000 0 1} {false true 1000 0 1}]

because right now it assumes that mapping should be like
[{true true 1000 0 1}]

which is logically equivalent.

This become noticable after my change in canonical#12718, before that
this issue in logic was reproducible only if user will do something like this:
  cat << EOF
uid $(id -u) 1000000
gid $(id -g) 1000000
EOF
  ) | lxc config set idmap raw.idmap -
which is an extremely rare usecase.

Fixes: canonical#13325
Signed-off-by: Alexander Mikhalitsyn <[email protected]>
tomponline pushed a commit to tomponline/lxd that referenced this issue Apr 17, 2024
…mapping only if it's required

Fix diskAddRootUserNSEntry logic to correcly handle case when
we have not-groupped UID/GID mapping like this:
[{true false 1000 0 1} {false true 1000 0 1}]

because right now it assumes that mapping should be like
[{true true 1000 0 1}]

which is logically equivalent.

This become noticable after my change in canonical#12718, before that
this issue in logic was reproducible only if user will do something like this:
  cat << EOF
uid $(id -u) 1000000
gid $(id -g) 1000000
EOF
  ) | lxc config set idmap raw.idmap -
which is an extremely rare usecase.

Fixes: canonical#13325
Signed-off-by: Alexander Mikhalitsyn <[email protected]>
hamistao pushed a commit to hamistao/lxd that referenced this issue Apr 18, 2024
…mapping only if it's required

Fix diskAddRootUserNSEntry logic to correcly handle case when
we have not-groupped UID/GID mapping like this:
[{true false 1000 0 1} {false true 1000 0 1}]

because right now it assumes that mapping should be like
[{true true 1000 0 1}]

which is logically equivalent.

This become noticable after my change in canonical#12718, before that
this issue in logic was reproducible only if user will do something like this:
  cat << EOF
uid $(id -u) 1000000
gid $(id -g) 1000000
EOF
  ) | lxc config set idmap raw.idmap -
which is an extremely rare usecase.

Fixes: canonical#13325
Signed-off-by: Alexander Mikhalitsyn <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Confirmed to be a bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants