Linux 6.9+ idmapping abilities are required but aren't supported on system

[KosmX]

Required information

• Distribution: Arch Linux
• Distribution version: 2024-05-19T21:22:13Z (the time when opening this issue)
• The output of "incus info" or if that fails:

config: {}
api_extensions:
- storage_zfs_remove_snapshots
- container_host_shutdown_timeout
- container_stop_priority
- container_syscall_filtering
- auth_pki
- container_last_used_at
- etag
- patch
- usb_devices
- https_allowed_credentials
- image_compression_algorithm
- directory_manipulation
- container_cpu_time
- storage_zfs_use_refquota
- storage_lvm_mount_options
- network
- profile_usedby
- container_push
- container_exec_recording
- certificate_update
- container_exec_signal_handling
- gpu_devices
- container_image_properties
- migration_progress
- id_map
- network_firewall_filtering
- network_routes
- storage
- file_delete
- file_append
- network_dhcp_expiry
- storage_lvm_vg_rename
- storage_lvm_thinpool_rename
- network_vlan
- image_create_aliases
- container_stateless_copy
- container_only_migration
- storage_zfs_clone_copy
- unix_device_rename
- storage_lvm_use_thinpool
- storage_rsync_bwlimit
- network_vxlan_interface
- storage_btrfs_mount_options
- entity_description
- image_force_refresh
- storage_lvm_lv_resizing
- id_map_base
- file_symlinks
- container_push_target
- network_vlan_physical
- storage_images_delete
- container_edit_metadata
- container_snapshot_stateful_migration
- storage_driver_ceph
- storage_ceph_user_name
- resource_limits
- storage_volatile_initial_source
- storage_ceph_force_osd_reuse
- storage_block_filesystem_btrfs
- resources
- kernel_limits
- storage_api_volume_rename
- network_sriov
- console
- restrict_dev_incus
- migration_pre_copy
- infiniband
- dev_incus_events
- proxy
- network_dhcp_gateway
- file_get_symlink
- network_leases
- unix_device_hotplug
- storage_api_local_volume_handling
- operation_description
- clustering
- event_lifecycle
- storage_api_remote_volume_handling
- nvidia_runtime
- container_mount_propagation
- container_backup
- dev_incus_images
- container_local_cross_pool_handling
- proxy_unix
- proxy_udp
- clustering_join
- proxy_tcp_udp_multi_port_handling
- network_state
- proxy_unix_dac_properties
- container_protection_delete
- unix_priv_drop
- pprof_http
- proxy_haproxy_protocol
- network_hwaddr
- proxy_nat
- network_nat_order
- container_full
- backup_compression
- nvidia_runtime_config
- storage_api_volume_snapshots
- storage_unmapped
- projects
- network_vxlan_ttl
- container_incremental_copy
- usb_optional_vendorid
- snapshot_scheduling
- snapshot_schedule_aliases
- container_copy_project
- clustering_server_address
- clustering_image_replication
- container_protection_shift
- snapshot_expiry
- container_backup_override_pool
- snapshot_expiry_creation
- network_leases_location
- resources_cpu_socket
- resources_gpu
- resources_numa
- kernel_features
- id_map_current
- event_location
- storage_api_remote_volume_snapshots
- network_nat_address
- container_nic_routes
- cluster_internal_copy
- seccomp_notify
- lxc_features
- container_nic_ipvlan
- network_vlan_sriov
- storage_cephfs
- container_nic_ipfilter
- resources_v2
- container_exec_user_group_cwd
- container_syscall_intercept
- container_disk_shift
- storage_shifted
- resources_infiniband
- daemon_storage
- instances
- image_types
- resources_disk_sata
- clustering_roles
- images_expiry
- resources_network_firmware
- backup_compression_algorithm
- ceph_data_pool_name
- container_syscall_intercept_mount
- compression_squashfs
- container_raw_mount
- container_nic_routed
- container_syscall_intercept_mount_fuse
- container_disk_ceph
- virtual-machines
- image_profiles
- clustering_architecture
- resources_disk_id
- storage_lvm_stripes
- vm_boot_priority
- unix_hotplug_devices
- api_filtering
- instance_nic_network
- clustering_sizing
- firewall_driver
- projects_limits
- container_syscall_intercept_hugetlbfs
- limits_hugepages
- container_nic_routed_gateway
- projects_restrictions
- custom_volume_snapshot_expiry
- volume_snapshot_scheduling
- trust_ca_certificates
- snapshot_disk_usage
- clustering_edit_roles
- container_nic_routed_host_address
- container_nic_ipvlan_gateway
- resources_usb_pci
- resources_cpu_threads_numa
- resources_cpu_core_die
- api_os
- container_nic_routed_host_table
- container_nic_ipvlan_host_table
- container_nic_ipvlan_mode
- resources_system
- images_push_relay
- network_dns_search
- container_nic_routed_limits
- instance_nic_bridged_vlan
- network_state_bond_bridge
- usedby_consistency
- custom_block_volumes
- clustering_failure_domains
- resources_gpu_mdev
- console_vga_type
- projects_limits_disk
- network_type_macvlan
- network_type_sriov
- container_syscall_intercept_bpf_devices
- network_type_ovn
- projects_networks
- projects_networks_restricted_uplinks
- custom_volume_backup
- backup_override_name
- storage_rsync_compression
- network_type_physical
- network_ovn_external_subnets
- network_ovn_nat
- network_ovn_external_routes_remove
- tpm_device_type
- storage_zfs_clone_copy_rebase
- gpu_mdev
- resources_pci_iommu
- resources_network_usb
- resources_disk_address
- network_physical_ovn_ingress_mode
- network_ovn_dhcp
- network_physical_routes_anycast
- projects_limits_instances
- network_state_vlan
- instance_nic_bridged_port_isolation
- instance_bulk_state_change
- network_gvrp
- instance_pool_move
- gpu_sriov
- pci_device_type
- storage_volume_state
- network_acl
- migration_stateful
- disk_state_quota
- storage_ceph_features
- projects_compression
- projects_images_remote_cache_expiry
- certificate_project
- network_ovn_acl
- projects_images_auto_update
- projects_restricted_cluster_target
- images_default_architecture
- network_ovn_acl_defaults
- gpu_mig
- project_usage
- network_bridge_acl
- warnings
- projects_restricted_backups_and_snapshots
- clustering_join_token
- clustering_description
- server_trusted_proxy
- clustering_update_cert
- storage_api_project
- server_instance_driver_operational
- server_supported_storage_drivers
- event_lifecycle_requestor_address
- resources_gpu_usb
- clustering_evacuation
- network_ovn_nat_address
- network_bgp
- network_forward
- custom_volume_refresh
- network_counters_errors_dropped
- metrics
- image_source_project
- clustering_config
- network_peer
- linux_sysctl
- network_dns
- ovn_nic_acceleration
- certificate_self_renewal
- instance_project_move
- storage_volume_project_move
- cloud_init
- network_dns_nat
- database_leader
- instance_all_projects
- clustering_groups
- ceph_rbd_du
- instance_get_full
- qemu_metrics
- gpu_mig_uuid
- event_project
- clustering_evacuation_live
- instance_allow_inconsistent_copy
- network_state_ovn
- storage_volume_api_filtering
- image_restrictions
- storage_zfs_export
- network_dns_records
- storage_zfs_reserve_space
- network_acl_log
- storage_zfs_blocksize
- metrics_cpu_seconds
- instance_snapshot_never
- certificate_token
- instance_nic_routed_neighbor_probe
- event_hub
- agent_nic_config
- projects_restricted_intercept
- metrics_authentication
- images_target_project
- images_all_projects
- cluster_migration_inconsistent_copy
- cluster_ovn_chassis
- container_syscall_intercept_sched_setscheduler
- storage_lvm_thinpool_metadata_size
- storage_volume_state_total
- instance_file_head
- instances_nic_host_name
- image_copy_profile
- container_syscall_intercept_sysinfo
- clustering_evacuation_mode
- resources_pci_vpd
- qemu_raw_conf
- storage_cephfs_fscache
- network_load_balancer
- vsock_api
- instance_ready_state
- network_bgp_holdtime
- storage_volumes_all_projects
- metrics_memory_oom_total
- storage_buckets
- storage_buckets_create_credentials
- metrics_cpu_effective_total
- projects_networks_restricted_access
- storage_buckets_local
- loki
- acme
- internal_metrics
- cluster_join_token_expiry
- remote_token_expiry
- init_preseed
- storage_volumes_created_at
- cpu_hotplug
- projects_networks_zones
- network_txqueuelen
- cluster_member_state
- instances_placement_scriptlet
- storage_pool_source_wipe
- zfs_block_mode
- instance_generation_id
- disk_io_cache
- amd_sev
- storage_pool_loop_resize
- migration_vm_live
- ovn_nic_nesting
- oidc
- network_ovn_l3only
- ovn_nic_acceleration_vdpa
- cluster_healing
- instances_state_total
- auth_user
- security_csm
- instances_rebuild
- numa_cpu_placement
- custom_volume_iso
- network_allocations
- zfs_delegate
- storage_api_remote_volume_snapshot_copy
- operations_get_query_all_projects
- metadata_configuration
- syslog_socket
- event_lifecycle_name_and_project
- instances_nic_limits_priority
- disk_initial_volume_configuration
- operation_wait
- image_restriction_privileged
- cluster_internal_custom_volume_copy
- disk_io_bus
- storage_cephfs_create_missing
- instance_move_config
- ovn_ssl_config
- certificate_description
- disk_io_bus_virtio_blk
- loki_config_instance
- instance_create_start
- clustering_evacuation_stop_options
- boot_host_shutdown_action
- agent_config_drive
- network_state_ovn_lr
- image_template_permissions
- storage_bucket_backup
- storage_lvm_cluster
- shared_custom_block_volumes
- auth_tls_jwt
- oidc_claim
- device_usb_serial
- numa_cpu_balanced
- image_restriction_nesting
- network_integrations
- instance_memory_swap_bytes
- network_bridge_external_create
- network_zones_all_projects
- storage_zfs_vdev
- container_migration_stateful
- profiles_all_projects
- instances_scriptlet_get_instances
- instances_scriptlet_get_cluster_members
- network_acl_stateless
- instance_state_started_at
api_status: stable
api_version: "1.0"
auth: trusted
public: false
auth_methods:
- tls
auth_user_name: root
auth_user_method: unix
environment:
 addresses: []
 architectures:
 - x86_64
 - i686
 certificate: |
   -----BEGIN CERTIFICATE-----
   MIICCDCCAY2gAwIBAgIRAJXX3ZoU8p3Ydtc2tXcrXnowCgYIKoZIzj0EAwMwNDEZ
   MBcGA1UEChMQTGludXggQ29udGFpbmVyczEXMBUGA1UEAwwOcm9vdEBhcmNobGlu
   dXgwHhcNMjQwNTE5MjI0OTQ1WhcNMzQwNTE3MjI0OTQ1WjA0MRkwFwYDVQQKExBM
   aW51eCBDb250YWluZXJzMRcwFQYDVQQDDA5yb290QGFyY2hsaW51eDB2MBAGByqG
   SM49AgEGBSuBBAAiA2IABBRa2itJ/saCAwuz9RMvFgyRPi269v+1ha8uVrnDnTNn
   D1pn2UeKd7ELoQbh+7Kr9FzBsLSd40m9QqprJ9Cv0xPGeZPd1zxzHfhV7o0HA9lX
   dKsjc4gebbdyZuJZM/fs+qNjMGEwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQMMAoG
   CCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwLAYDVR0RBCUwI4IJYXJjaGxpbnV4hwR/
   AAABhxAAAAAAAAAAAAAAAAAAAAABMAoGCCqGSM49BAMDA2kAMGYCMQDrUJ9R+eun
   /6EROOzhWiNwT+FP33HKKOM2yFbodSvs9H+Jw95OsJmkISXB1x3QwwsCMQCRXk8X
   u5QeGYEHR4C559QXi73s5Ekglt9Bg28/1a0vU0R2ShfsLgx0fe6sld+DURY=
   -----END CERTIFICATE-----
 certificate_fingerprint: afef8965778a8b9b543f4f19d278b298bbb733acb8bc31c723d86089cb12e0e1
 driver: qemu | lxc
 driver_version: 9.0.0 | 6.0.0
 firewall: nftables
 kernel: Linux
 kernel_architecture: x86_64
 kernel_features:
   idmapped_mounts: "true"
   netnsid_getifaddrs: "true"
   seccomp_listener: "true"
   seccomp_listener_continue: "true"
   uevent_injection: "true"
   unpriv_binfmt: "true"
   unpriv_fscaps: "true"
 kernel_version: 6.9.1-zen1-1-zen
 lxc_features:
   cgroup2: "true"
   core_scheduling: "true"
   devpts_fd: "true"
   idmapped_mounts_v2: "true"
   mount_injection_file: "true"
   network_gateway_device_route: "true"
   network_ipvlan: "true"
   network_l2proxy: "true"
   network_phys_macvlan_mtu: "true"
   network_veth_router: "true"
   pidfd: "true"
   seccomp_allow_deny_syntax: "true"
   seccomp_notify: "true"
   seccomp_proxy_send_notify_fd: "true"
 os_name: Arch Linux
 os_version: ""
 project: default
 server: incus
 server_clustered: false
 server_event_mode: full-mesh
 server_name: archlinux
 server_pid: 2848
 server_version: "6.1"
 storage: btrfs
 storage_version: 6.8.1
 storage_supported_drivers:
 - name: dir
   version: "1"
   remote: false
 - name: lvm
   version: 2.03.23(2) (2023-11-21) / 1.02.197 (2023-11-21) / 4.48.0
   remote: false
 - name: lvmcluster
   version: 2.03.23(2) (2023-11-21) / 1.02.197 (2023-11-21) / 4.48.0
   remote: true
 - name: btrfs
   version: 6.8.1
   remote: false

Issue description

Can't mound shifted filesystem.
It works on the same install if using earlier kernel than linux 6.9.
It might be a bug in the kernel, I don't know.

Steps to reproduce

# install and init incus with defaults
sudo incus launch images:ubuntu/22.04 demo                                   
# Launching demo
sudo incus stop demo
sudo incus config device add demo sharedir disk source=/home/kosmx path=/mnt shift=true
# Device sharedir added to demo
sudo incus start demo
# Error: Failed to setup device mount "sharedir": idmapping abilities are required but aren't supported on system
# Try `incus info --show-log demo` for more info

Information to attach

• Any relevant kernel output (dmesg) no relevant info
• Container log (incus info NAME --show-log)

Name: demo
Status: STOPPED
Type: container
Architecture: x86_64
Created: 2024/05/19 22:57 UTC
Last Used: 2024/05/19 22:57 UTC

Log:

yes, there is nothing

• Output of the client with --debug

DEBUG  [2024-05-19T23:01:35Z] Connecting to a local Incus over a Unix socket 
DEBUG  [2024-05-19T23:01:35Z] Sending request to Incus                      etag= method=GET url="http://unix.socket/1.0"
DEBUG  [2024-05-19T23:01:35Z] Got response struct from Incus               
DEBUG  [2024-05-19T23:01:35Z] 
   {
   	"config": {},
   	"api_extensions": [
   		"storage_zfs_remove_snapshots",
   		"container_host_shutdown_timeout",
   		"container_stop_priority",
   		"container_syscall_filtering",
   		"auth_pki",
   		"container_last_used_at",
   		"etag",
   		"patch",
   		"usb_devices",
   		"https_allowed_credentials",
   		"image_compression_algorithm",
   		"directory_manipulation",
   		"container_cpu_time",
   		"storage_zfs_use_refquota",
   		"storage_lvm_mount_options",
   		"network",
   		"profile_usedby",
   		"container_push",
   		"container_exec_recording",
   		"certificate_update",
   		"container_exec_signal_handling",
   		"gpu_devices",
   		"container_image_properties",
   		"migration_progress",
   		"id_map",
   		"network_firewall_filtering",
   		"network_routes",
   		"storage",
   		"file_delete",
   		"file_append",
   		"network_dhcp_expiry",
   		"storage_lvm_vg_rename",
   		"storage_lvm_thinpool_rename",
   		"network_vlan",
   		"image_create_aliases",
   		"container_stateless_copy",
   		"container_only_migration",
   		"storage_zfs_clone_copy",
   		"unix_device_rename",
   		"storage_lvm_use_thinpool",
   		"storage_rsync_bwlimit",
   		"network_vxlan_interface",
   		"storage_btrfs_mount_options",
   		"entity_description",
   		"image_force_refresh",
   		"storage_lvm_lv_resizing",
   		"id_map_base",
   		"file_symlinks",
   		"container_push_target",
   		"network_vlan_physical",
   		"storage_images_delete",
   		"container_edit_metadata",
   		"container_snapshot_stateful_migration",
   		"storage_driver_ceph",
   		"storage_ceph_user_name",
   		"resource_limits",
   		"storage_volatile_initial_source",
   		"storage_ceph_force_osd_reuse",
   		"storage_block_filesystem_btrfs",
   		"resources",
   		"kernel_limits",
   		"storage_api_volume_rename",
   		"network_sriov",
   		"console",
   		"restrict_dev_incus",
   		"migration_pre_copy",
   		"infiniband",
   		"dev_incus_events",
   		"proxy",
   		"network_dhcp_gateway",
   		"file_get_symlink",
   		"network_leases",
   		"unix_device_hotplug",
   		"storage_api_local_volume_handling",
   		"operation_description",
   		"clustering",
   		"event_lifecycle",
   		"storage_api_remote_volume_handling",
   		"nvidia_runtime",
   		"container_mount_propagation",
   		"container_backup",
   		"dev_incus_images",
   		"container_local_cross_pool_handling",
   		"proxy_unix",
   		"proxy_udp",
   		"clustering_join",
   		"proxy_tcp_udp_multi_port_handling",
   		"network_state",
   		"proxy_unix_dac_properties",
   		"container_protection_delete",
   		"unix_priv_drop",
   		"pprof_http",
   		"proxy_haproxy_protocol",
   		"network_hwaddr",
   		"proxy_nat",
   		"network_nat_order",
   		"container_full",
   		"backup_compression",
   		"nvidia_runtime_config",
   		"storage_api_volume_snapshots",
   		"storage_unmapped",
   		"projects",
   		"network_vxlan_ttl",
   		"container_incremental_copy",
   		"usb_optional_vendorid",
   		"snapshot_scheduling",
   		"snapshot_schedule_aliases",
   		"container_copy_project",
   		"clustering_server_address",
   		"clustering_image_replication",
   		"container_protection_shift",
   		"snapshot_expiry",
   		"container_backup_override_pool",
   		"snapshot_expiry_creation",
   		"network_leases_location",
   		"resources_cpu_socket",
   		"resources_gpu",
   		"resources_numa",
   		"kernel_features",
   		"id_map_current",
   		"event_location",
   		"storage_api_remote_volume_snapshots",
   		"network_nat_address",
   		"container_nic_routes",
   		"cluster_internal_copy",
   		"seccomp_notify",
   		"lxc_features",
   		"container_nic_ipvlan",
   		"network_vlan_sriov",
   		"storage_cephfs",
   		"container_nic_ipfilter",
   		"resources_v2",
   		"container_exec_user_group_cwd",
   		"container_syscall_intercept",
   		"container_disk_shift",
   		"storage_shifted",
   		"resources_infiniband",
   		"daemon_storage",
   		"instances",
   		"image_types",
   		"resources_disk_sata",
   		"clustering_roles",
   		"images_expiry",
   		"resources_network_firmware",
   		"backup_compression_algorithm",
   		"ceph_data_pool_name",
   		"container_syscall_intercept_mount",
   		"compression_squashfs",
   		"container_raw_mount",
   		"container_nic_routed",
   		"container_syscall_intercept_mount_fuse",
   		"container_disk_ceph",
   		"virtual-machines",
   		"image_profiles",
   		"clustering_architecture",
   		"resources_disk_id",
   		"storage_lvm_stripes",
   		"vm_boot_priority",
   		"unix_hotplug_devices",
   		"api_filtering",
   		"instance_nic_network",
   		"clustering_sizing",
   		"firewall_driver",
   		"projects_limits",
   		"container_syscall_intercept_hugetlbfs",
   		"limits_hugepages",
   		"container_nic_routed_gateway",
   		"projects_restrictions",
   		"custom_volume_snapshot_expiry",
   		"volume_snapshot_scheduling",
   		"trust_ca_certificates",
   		"snapshot_disk_usage",
   		"clustering_edit_roles",
   		"container_nic_routed_host_address",
   		"container_nic_ipvlan_gateway",
   		"resources_usb_pci",
   		"resources_cpu_threads_numa",
   		"resources_cpu_core_die",
   		"api_os",
   		"container_nic_routed_host_table",
   		"container_nic_ipvlan_host_table",
   		"container_nic_ipvlan_mode",
   		"resources_system",
   		"images_push_relay",
   		"network_dns_search",
   		"container_nic_routed_limits",
   		"instance_nic_bridged_vlan",
   		"network_state_bond_bridge",
   		"usedby_consistency",
   		"custom_block_volumes",
   		"clustering_failure_domains",
   		"resources_gpu_mdev",
   		"console_vga_type",
   		"projects_limits_disk",
   		"network_type_macvlan",
   		"network_type_sriov",
   		"container_syscall_intercept_bpf_devices",
   		"network_type_ovn",
   		"projects_networks",
   		"projects_networks_restricted_uplinks",
   		"custom_volume_backup",
   		"backup_override_name",
   		"storage_rsync_compression",
   		"network_type_physical",
   		"network_ovn_external_subnets",
   		"network_ovn_nat",
   		"network_ovn_external_routes_remove",
   		"tpm_device_type",
   		"storage_zfs_clone_copy_rebase",
   		"gpu_mdev",
   		"resources_pci_iommu",
   		"resources_network_usb",
   		"resources_disk_address",
   		"network_physical_ovn_ingress_mode",
   		"network_ovn_dhcp",
   		"network_physical_routes_anycast",
   		"projects_limits_instances",
   		"network_state_vlan",
   		"instance_nic_bridged_port_isolation",
   		"instance_bulk_state_change",
   		"network_gvrp",
   		"instance_pool_move",
   		"gpu_sriov",
   		"pci_device_type",
   		"storage_volume_state",
   		"network_acl",
   		"migration_stateful",
   		"disk_state_quota",
   		"storage_ceph_features",
   		"projects_compression",
   		"projects_images_remote_cache_expiry",
   		"certificate_project",
   		"network_ovn_acl",
   		"projects_images_auto_update",
   		"projects_restricted_cluster_target",
   		"images_default_architecture",
   		"network_ovn_acl_defaults",
   		"gpu_mig",
   		"project_usage",
   		"network_bridge_acl",
   		"warnings",
   		"projects_restricted_backups_and_snapshots",
   		"clustering_join_token",
   		"clustering_description",
   		"server_trusted_proxy",
   		"clustering_update_cert",
   		"storage_api_project",
   		"server_instance_driver_operational",
   		"server_supported_storage_drivers",
   		"event_lifecycle_requestor_address",
   		"resources_gpu_usb",
   		"clustering_evacuation",
   		"network_ovn_nat_address",
   		"network_bgp",
   		"network_forward",
   		"custom_volume_refresh",
   		"network_counters_errors_dropped",
   		"metrics",
   		"image_source_project",
   		"clustering_config",
   		"network_peer",
   		"linux_sysctl",
   		"network_dns",
   		"ovn_nic_acceleration",
   		"certificate_self_renewal",
   		"instance_project_move",
   		"storage_volume_project_move",
   		"cloud_init",
   		"network_dns_nat",
   		"database_leader",
   		"instance_all_projects",
   		"clustering_groups",
   		"ceph_rbd_du",
   		"instance_get_full",
   		"qemu_metrics",
   		"gpu_mig_uuid",
   		"event_project",
   		"clustering_evacuation_live",
   		"instance_allow_inconsistent_copy",
   		"network_state_ovn",
   		"storage_volume_api_filtering",
   		"image_restrictions",
   		"storage_zfs_export",
   		"network_dns_records",
   		"storage_zfs_reserve_space",
   		"network_acl_log",
   		"storage_zfs_blocksize",
   		"metrics_cpu_seconds",
   		"instance_snapshot_never",
   		"certificate_token",
   		"instance_nic_routed_neighbor_probe",
   		"event_hub",
   		"agent_nic_config",
   		"projects_restricted_intercept",
   		"metrics_authentication",
   		"images_target_project",
   		"images_all_projects",
   		"cluster_migration_inconsistent_copy",
   		"cluster_ovn_chassis",
   		"container_syscall_intercept_sched_setscheduler",
   		"storage_lvm_thinpool_metadata_size",
   		"storage_volume_state_total",
   		"instance_file_head",
   		"instances_nic_host_name",
   		"image_copy_profile",
   		"container_syscall_intercept_sysinfo",
   		"clustering_evacuation_mode",
   		"resources_pci_vpd",
   		"qemu_raw_conf",
   		"storage_cephfs_fscache",
   		"network_load_balancer",
   		"vsock_api",
   		"instance_ready_state",
   		"network_bgp_holdtime",
   		"storage_volumes_all_projects",
   		"metrics_memory_oom_total",
   		"storage_buckets",
   		"storage_buckets_create_credentials",
   		"metrics_cpu_effective_total",
   		"projects_networks_restricted_access",
   		"storage_buckets_local",
   		"loki",
   		"acme",
   		"internal_metrics",
   		"cluster_join_token_expiry",
   		"remote_token_expiry",
   		"init_preseed",
   		"storage_volumes_created_at",
   		"cpu_hotplug",
   		"projects_networks_zones",
   		"network_txqueuelen",
   		"cluster_member_state",
   		"instances_placement_scriptlet",
   		"storage_pool_source_wipe",
   		"zfs_block_mode",
   		"instance_generation_id",
   		"disk_io_cache",
   		"amd_sev",
   		"storage_pool_loop_resize",
   		"migration_vm_live",
   		"ovn_nic_nesting",
   		"oidc",
   		"network_ovn_l3only",
   		"ovn_nic_acceleration_vdpa",
   		"cluster_healing",
   		"instances_state_total",
   		"auth_user",
   		"security_csm",
   		"instances_rebuild",
   		"numa_cpu_placement",
   		"custom_volume_iso",
   		"network_allocations",
   		"zfs_delegate",
   		"storage_api_remote_volume_snapshot_copy",
   		"operations_get_query_all_projects",
   		"metadata_configuration",
   		"syslog_socket",
   		"event_lifecycle_name_and_project",
   		"instances_nic_limits_priority",
   		"disk_initial_volume_configuration",
   		"operation_wait",
   		"image_restriction_privileged",
   		"cluster_internal_custom_volume_copy",
   		"disk_io_bus",
   		"storage_cephfs_create_missing",
   		"instance_move_config",
   		"ovn_ssl_config",
   		"certificate_description",
   		"disk_io_bus_virtio_blk",
   		"loki_config_instance",
   		"instance_create_start",
   		"clustering_evacuation_stop_options",
   		"boot_host_shutdown_action",
   		"agent_config_drive",
   		"network_state_ovn_lr",
   		"image_template_permissions",
   		"storage_bucket_backup",
   		"storage_lvm_cluster",
   		"shared_custom_block_volumes",
   		"auth_tls_jwt",
   		"oidc_claim",
   		"device_usb_serial",
   		"numa_cpu_balanced",
   		"image_restriction_nesting",
   		"network_integrations",
   		"instance_memory_swap_bytes",
   		"network_bridge_external_create",
   		"network_zones_all_projects",
   		"storage_zfs_vdev",
   		"container_migration_stateful",
   		"profiles_all_projects",
   		"instances_scriptlet_get_instances",
   		"instances_scriptlet_get_cluster_members",
   		"network_acl_stateless",
   		"instance_state_started_at"
   	],
   	"api_status": "stable",
   	"api_version": "1.0",
   	"auth": "trusted",
   	"public": false,
   	"auth_methods": [
   		"tls"
   	],
   	"auth_user_name": "root",
   	"auth_user_method": "unix",
   	"environment": {
   		"addresses": [],
   		"architectures": [
   			"x86_64",
   			"i686"
   		],
   		"certificate": "-----BEGIN CERTIFICATE-----\nMIICCDCCAY2gAwIBAgIRAJXX3ZoU8p3Ydtc2tXcrXnowCgYIKoZIzj0EAwMwNDEZ\nMBcGA1UEChMQTGludXggQ29udGFpbmVyczEXMBUGA1UEAwwOcm9vdEBhcmNobGlu\ndXgwHhcNMjQwNTE5MjI0OTQ1WhcNMzQwNTE3MjI0OTQ1WjA0MRkwFwYDVQQKExBM\naW51eCBDb250YWluZXJzMRcwFQYDVQQDDA5yb290QGFyY2hsaW51eDB2MBAGByqG\nSM49AgEGBSuBBAAiA2IABBRa2itJ/saCAwuz9RMvFgyRPi269v+1ha8uVrnDnTNn\nD1pn2UeKd7ELoQbh+7Kr9FzBsLSd40m9QqprJ9Cv0xPGeZPd1zxzHfhV7o0HA9lX\ndKsjc4gebbdyZuJZM/fs+qNjMGEwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQMMAoG\nCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwLAYDVR0RBCUwI4IJYXJjaGxpbnV4hwR/\nAAABhxAAAAAAAAAAAAAAAAAAAAABMAoGCCqGSM49BAMDA2kAMGYCMQDrUJ9R+eun\n/6EROOzhWiNwT+FP33HKKOM2yFbodSvs9H+Jw95OsJmkISXB1x3QwwsCMQCRXk8X\nu5QeGYEHR4C559QXi73s5Ekglt9Bg28/1a0vU0R2ShfsLgx0fe6sld+DURY=\n-----END CERTIFICATE-----\n",
   		"certificate_fingerprint": "afef8965778a8b9b543f4f19d278b298bbb733acb8bc31c723d86089cb12e0e1",
   		"driver": "qemu | lxc",
   		"driver_version": "9.0.0 | 6.0.0",
   		"firewall": "nftables",
   		"kernel": "Linux",
   		"kernel_architecture": "x86_64",
   		"kernel_features": {
   			"idmapped_mounts": "true",
   			"netnsid_getifaddrs": "true",
   			"seccomp_listener": "true",
   			"seccomp_listener_continue": "true",
   			"uevent_injection": "true",
   			"unpriv_binfmt": "true",
   			"unpriv_fscaps": "true"
   		},
   		"kernel_version": "6.9.1-zen1-1-zen",
   		"lxc_features": {
   			"cgroup2": "true",
   			"core_scheduling": "true",
   			"devpts_fd": "true",
   			"idmapped_mounts_v2": "true",
   			"mount_injection_file": "true",
   			"network_gateway_device_route": "true",
   			"network_ipvlan": "true",
   			"network_l2proxy": "true",
   			"network_phys_macvlan_mtu": "true",
   			"network_veth_router": "true",
   			"pidfd": "true",
   			"seccomp_allow_deny_syntax": "true",
   			"seccomp_notify": "true",
   			"seccomp_proxy_send_notify_fd": "true"
   		},
   		"os_name": "Arch Linux",
   		"os_version": "",
   		"project": "default",
   		"server": "incus",
   		"server_clustered": false,
   		"server_event_mode": "full-mesh",
   		"server_name": "archlinux",
   		"server_pid": 2848,
   		"server_version": "6.1",
   		"storage": "btrfs",
   		"storage_version": "6.8.1",
   		"storage_supported_drivers": [
   			{
   				"Name": "dir",
   				"Version": "1",
   				"Remote": false
   			},
   			{
   				"Name": "lvm",
   				"Version": "2.03.23(2) (2023-11-21) / 1.02.197 (2023-11-21) / 4.48.0",
   				"Remote": false
   			},
   			{
   				"Name": "lvmcluster",
   				"Version": "2.03.23(2) (2023-11-21) / 1.02.197 (2023-11-21) / 4.48.0",
   				"Remote": true
   			},
   			{
   				"Name": "btrfs",
   				"Version": "6.8.1",
   				"Remote": false
   			}
   		]
   	}
   } 
DEBUG  [2024-05-19T23:01:35Z] Sending request to Incus                      etag= method=GET url="http://unix.socket/1.0/instances/demo"
DEBUG  [2024-05-19T23:01:35Z] Got response struct from Incus               
DEBUG  [2024-05-19T23:01:35Z] 
   {
   	"architecture": "x86_64",
   	"config": {
   		"image.architecture": "amd64",
   		"image.description": "Ubuntu jammy amd64 (20240519_07:42)",
   		"image.os": "Ubuntu",
   		"image.release": "jammy",
   		"image.serial": "20240519_07:42",
   		"image.type": "squashfs",
   		"image.variant": "default",
   		"volatile.base_image": "8f6c33eda6bc95be1a7205b743ed0a7b4fd67368488f3e7d6022d70d40b00bf3",
   		"volatile.cloud-init.instance-id": "6c0f15fa-96d1-4631-8e2d-6c93cda56268",
   		"volatile.eth0.hwaddr": "00:16:3e:10:af:3f",
   		"volatile.idmap.base": "0",
   		"volatile.idmap.current": "[{\"Isuid\":true,\"Isgid\":false,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000},{\"Isuid\":false,\"Isgid\":true,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000}]",
   		"volatile.idmap.next": "[{\"Isuid\":true,\"Isgid\":false,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000},{\"Isuid\":false,\"Isgid\":true,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000}]",
   		"volatile.last_state.idmap": "[{\"Isuid\":true,\"Isgid\":false,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000},{\"Isuid\":false,\"Isgid\":true,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000}]",
   		"volatile.last_state.power": "STOPPED",
   		"volatile.last_state.ready": "false",
   		"volatile.uuid": "16667b01-c34a-4f3b-8a86-d9638abf7105",
   		"volatile.uuid.generation": "16667b01-c34a-4f3b-8a86-d9638abf7105"
   	},
   	"devices": {
   		"sharedir": {
   			"path": "/mnt",
   			"shift": "true",
   			"source": "/home/kosmx",
   			"type": "disk"
   		}
   	},
   	"ephemeral": false,
   	"profiles": [
   		"default"
   	],
   	"stateful": false,
   	"description": "",
   	"created_at": "2024-05-19T22:57:35.369689384Z",
   	"expanded_config": {
   		"image.architecture": "amd64",
   		"image.description": "Ubuntu jammy amd64 (20240519_07:42)",
   		"image.os": "Ubuntu",
   		"image.release": "jammy",
   		"image.serial": "20240519_07:42",
   		"image.type": "squashfs",
   		"image.variant": "default",
   		"volatile.base_image": "8f6c33eda6bc95be1a7205b743ed0a7b4fd67368488f3e7d6022d70d40b00bf3",
   		"volatile.cloud-init.instance-id": "6c0f15fa-96d1-4631-8e2d-6c93cda56268",
   		"volatile.eth0.hwaddr": "00:16:3e:10:af:3f",
   		"volatile.idmap.base": "0",
   		"volatile.idmap.current": "[{\"Isuid\":true,\"Isgid\":false,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000},{\"Isuid\":false,\"Isgid\":true,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000}]",
   		"volatile.idmap.next": "[{\"Isuid\":true,\"Isgid\":false,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000},{\"Isuid\":false,\"Isgid\":true,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000}]",
   		"volatile.last_state.idmap": "[{\"Isuid\":true,\"Isgid\":false,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000},{\"Isuid\":false,\"Isgid\":true,\"Hostid\":1000000,\"Nsid\":0,\"Maprange\":1000000000}]",
   		"volatile.last_state.power": "STOPPED",
   		"volatile.last_state.ready": "false",
   		"volatile.uuid": "16667b01-c34a-4f3b-8a86-d9638abf7105",
   		"volatile.uuid.generation": "16667b01-c34a-4f3b-8a86-d9638abf7105"
   	},
   	"expanded_devices": {
   		"eth0": {
   			"name": "eth0",
   			"network": "incusbr0",
   			"type": "nic"
   		},
   		"root": {
   			"path": "/",
   			"pool": "default",
   			"type": "disk"
   		},
   		"sharedir": {
   			"path": "/mnt",
   			"shift": "true",
   			"source": "/home/kosmx",
   			"type": "disk"
   		}
   	},
   	"name": "demo",
   	"status": "Stopped",
   	"status_code": 102,
   	"last_used_at": "2024-05-19T22:57:36.217622434Z",
   	"location": "none",
   	"type": "container",
   	"project": "default"
   } 
DEBUG  [2024-05-19T23:01:35Z] Connected to the websocket: ws://unix.socket/1.0/events 
DEBUG  [2024-05-19T23:01:35Z] Sending request to Incus                      etag= method=PUT url="http://unix.socket/1.0/instances/demo/state"
DEBUG  [2024-05-19T23:01:35Z] 
   {
   	"action": "start",
   	"timeout": 0,
   	"force": false,
   	"stateful": false
   } 
DEBUG  [2024-05-19T23:01:35Z] Got operation from Incus                     
DEBUG  [2024-05-19T23:01:35Z] 
   {
   	"id": "f11c1849-921e-43ce-9b05-2cb610ba376e",
   	"class": "task",
   	"description": "Starting instance",
   	"created_at": "2024-05-19T23:01:35.512280975Z",
   	"updated_at": "2024-05-19T23:01:35.512280975Z",
   	"status": "Running",
   	"status_code": 103,
   	"resources": {
   		"instances": [
   			"/1.0/instances/demo"
   		]
   	},
   	"metadata": null,
   	"may_cancel": false,
   	"err": "",
   	"location": "none"
   } 
DEBUG  [2024-05-19T23:01:35Z] Sending request to Incus                      etag= method=GET url="http://unix.socket/1.0/operations/f11c1849-921e-43ce-9b05-2cb610ba376e"
DEBUG  [2024-05-19T23:01:35Z] Got response struct from Incus               
DEBUG  [2024-05-19T23:01:35Z] 
   {
   	"id": "f11c1849-921e-43ce-9b05-2cb610ba376e",
   	"class": "task",
   	"description": "Starting instance",
   	"created_at": "2024-05-19T23:01:35.512280975Z",
   	"updated_at": "2024-05-19T23:01:35.512280975Z",
   	"status": "Running",
   	"status_code": 103,
   	"resources": {
   		"instances": [
   			"/1.0/instances/demo"
   		]
   	},
   	"metadata": null,
   	"may_cancel": false,
   	"err": "",
   	"location": "none"
   } 
Error: Failed to setup device mount "sharedir": idmapping abilities are required but aren't supported on system
Try `incus info --show-log demo` for more info

• Output of the daemon with --debug (alternatively output of incus monitor --pretty while reproducing the issue)

DEBUG  [2024-05-19T23:02:23Z] Event listener server handler started         id=1b1e41bd-f9b0-47fc-b98a-438015d98eb0 local=/var/lib/incus/unix.socket remote=@
DEBUG  [2024-05-19T23:02:32Z] Handling API request                          ip=@ method=GET protocol=unix url=/1.0 username=root
DEBUG  [2024-05-19T23:02:32Z] Handling API request                          ip=@ method=GET protocol=unix url=/1.0/instances/demo username=root
DEBUG  [2024-05-19T23:02:32Z] Handling API request                          ip=@ method=GET protocol=unix url=/1.0/events username=root
DEBUG  [2024-05-19T23:02:32Z] Event listener server handler started         id=8bdc24cd-1b0d-49d7-988f-17923cd2f74b local=/var/lib/incus/unix.socket remote=@
DEBUG  [2024-05-19T23:02:32Z] Handling API request                          ip=@ method=PUT protocol=unix url=/1.0/instances/demo/state username=root
DEBUG  [2024-05-19T23:02:32Z] Start started                                 instance=demo instanceType=container project=default stateful=false
DEBUG  [2024-05-19T23:02:32Z] Started operation                             class=task description="Starting instance" operation=c4e0fa48-14e5-4ef5-a4e4-08336b6af966 project=default
DEBUG  [2024-05-19T23:02:32Z] New operation                                 class=task description="Starting instance" operation=c4e0fa48-14e5-4ef5-a4e4-08336b6af966 project=default
INFO   [2024-05-19T23:02:32Z] ID: c4e0fa48-14e5-4ef5-a4e4-08336b6af966, Class: task, Description: Starting instance  CreatedAt="2024-05-19 23:02:32.083670784 +0000 UTC" Err= Location=none MayCancel=false Metadata="map[]" Resources="map[instances:[/1.0/instances/demo]]" Status=Pending StatusCode=Pending UpdatedAt="2024-05-19 23:02:32.083670784 +0000 UTC"
INFO   [2024-05-19T23:02:32Z] ID: c4e0fa48-14e5-4ef5-a4e4-08336b6af966, Class: task, Description: Starting instance  CreatedAt="2024-05-19 23:02:32.083670784 +0000 UTC" Err= Location=none MayCancel=false Metadata="map[]" Resources="map[instances:[/1.0/instances/demo]]" Status=Running StatusCode=Running UpdatedAt="2024-05-19 23:02:32.083670784 +0000 UTC"
INFO   [2024-05-19T23:02:32Z] Starting instance                             action=start created="2024-05-19 22:57:35.369689384 +0000 UTC" ephemeral=false instance=demo instanceType=container project=default stateful=false used="2024-05-19 22:57:36.217622434 +0000 UTC"
DEBUG  [2024-05-19T23:02:32Z] Instance operation lock created               action=start instance=demo project=default reusable=false
DEBUG  [2024-05-19T23:02:32Z] Handling API request                          ip=@ method=GET protocol=unix url=/1.0/operations/c4e0fa48-14e5-4ef5-a4e4-08336b6af966 username=root
DEBUG  [2024-05-19T23:02:32Z] MountInstance started                         driver=btrfs instance=demo pool=default project=default
DEBUG  [2024-05-19T23:02:32Z] MountInstance finished                        driver=btrfs instance=demo pool=default project=default
DEBUG  [2024-05-19T23:02:32Z] Starting device                               device=eth0 instance=demo instanceType=container project=default type=nic
DEBUG  [2024-05-19T23:02:32Z] Starting device                               device=root instance=demo instanceType=container project=default type=disk
DEBUG  [2024-05-19T23:02:32Z] Starting device                               device=sharedir instance=demo instanceType=container project=default type=disk
DEBUG  [2024-05-19T23:02:32Z] Stopping device                               device=sharedir instance=demo instanceType=container project=default type=disk
DEBUG  [2024-05-19T23:02:32Z] Stopping device                               device=eth0 instance=demo instanceType=container project=default type=nic
DEBUG  [2024-05-19T23:02:32Z] Stopping device                               device=root instance=demo instanceType=container project=default type=disk
DEBUG  [2024-05-19T23:02:32Z] UnmountInstance started                       driver=btrfs instance=demo pool=default project=default
DEBUG  [2024-05-19T23:02:32Z] Instance operation lock finished              action=start err="Failed to setup device mount \"sharedir\": idmapping abilities are required but aren't supported on system" instance=demo project=default reusable=false
DEBUG  [2024-05-19T23:02:32Z] UnmountInstance finished                      driver=btrfs instance=demo pool=default project=default
INFO   [2024-05-19T23:02:32Z] ID: c4e0fa48-14e5-4ef5-a4e4-08336b6af966, Class: task, Description: Starting instance  CreatedAt="2024-05-19 23:02:32.083670784 +0000 UTC" Err="Failed to setup device mount \"sharedir\": idmapping abilities are required but aren't supported on system" Location=none MayCancel=false Metadata="map[]" Resources="map[instances:[/1.0/instances/demo]]" Status=Failure StatusCode=Failure UpdatedAt="2024-05-19 23:02:32.083670784 +0000 UTC"
DEBUG  [2024-05-19T23:02:32Z] Start finished                                instance=demo instanceType=container project=default stateful=false
DEBUG  [2024-05-19T23:02:32Z] Failure for operation                         class=task description="Starting instance" err="Failed to setup device mount \"sharedir\": idmapping abilities are required but aren't supported on system" operation=c4e0fa48-14e5-4ef5-a4e4-08336b6af966 project=default
DEBUG  [2024-05-19T23:02:32Z] Event listener server handler stopped         listener=8bdc24cd-1b0d-49d7-988f-17923cd2f74b local=/var/lib/incus/unix.socket remote=@

[KosmX]

The whole demo (logs) were done with btrfs host filesystem. (this might be relevant.)
The demo was in a VM, but I'm experiencing the same issue on a bare metal linux.

[qupfer]

same here
Not sure if related: incus info --show-log demo does show an empty Log (empty after Log:, general information are shown)

[KosmX]

@qupfer did you try to mount an idmapped filesystem?
if yes, then probably you're experiencing the same issue.

[qupfer]

@qupfer did you try to mount an idmapped filesystem?

that’s that I mean with „same here“. Since 6.9 this error appears to my container with mounted filesystems and shift=true.

Maybe also interesting: after upgrading to 6.9, the ipv4 adress of running containers was not shown anymore.

[stgraber]

We've tracked down the problematic kernel change and are working on a workaround.