spiderpool only supports sriov-network-operator for sriov?

[haiwu]

What help do you need？

Does spiderpool only support sriov-network-operator for sriov? sriov-network-operator is hard coded to only support redhat os and intel cpu.

How does spiderpool handle allocating ip for sriov for kubevirt vm? (which does not support ipam for sriov itself, and we have to put IP info in its cloud-init section of its vm yaml spec as a workaround)

[cyclinder]

Does spiderpool only support sriov-network-operator for sriov?

At the moment we only see sriov-network-operator as the only solution for sriov.

sriov-network-operator is hard coded to only support redhat os and intel CPU.

sriov-network-operator can also run on ubuntu and other os, which has nothing to do with the CPU vendor.

How does spiderpool handle allocating ip for sriov for kubevirt vm? (which does not support ipam for sriov itself, and we have to put IP info in its cloud-init section of its vm yaml spec as a workaround)

Can you explain this case in detail? Or provide some context so I can understand better, thanks.

[haiwu]

@cyclinder : sriov-network-operator hard codes intel cpu processor stuff here: https://github.com/k8snetworkplumbingwg/sriov-network-operator/blob/68b6c020fbba1596ddd3f2e4a70cfcdb034b7e28/pkg/consts/constants.go#L125, for amd, that should be amd_iommu instead.
It also hard codes tools like ostree/grubby here: https://github.com/k8snetworkplumbingwg/sriov-network-operator/blob/68b6c020fbba1596ddd3f2e4a70cfcdb034b7e28/bindata/scripts/enable-kargs.sh#L10, these are only available on redhat os.

My understanding is that https://github.com/k8snetworkplumbingwg/sriov-network-device-plugin works better on non-redhat os.

My tests show that Kubevirt VM wit sriov can't get IP from ipam, such vm could only get IP using cloudInitNoCloud.networkData.ethernets.eth0.addresses..

[cyclinder]

Hi @haiwu, We have many Ubuntu os to run sriov-operator, and we don't see any limitations, It works well so far. Do you hit any issues?

The sriov-operator simply helps users better utilize the sriov-cni and other CNI plugins. It also assists in installing the sriov-network-plugin. We can fully utilize it directly without going through the sriov-operator.

Reading https://kubevirt.io/user-guide/network/interfaces_and_networks/#sriov, Can you confirm that IPAM cannot be configured for sriovNetwork?

[haiwu]

@cyclinder : Yes, it was errorring out with latest sriov-network-operator release, it kept complaining about grubby not found.

I could not get static IPAM to work with sriov device plugin. That got completely ignored by KubeVirt VM with sriov. The only way to assign an IP to a KubeVirt VM with sriov is via cloudinit.

Tested ok with Kubevirt vm with sriov with: https://kubevirt.io/user-guide/user_workloads/hook-sidecar/ using preCloudInitIso hook to inject IP to the VM.

[cyclinder]

, it was errorring out with latest sriov-network-operator release, it kept complaining about grubby not found.

Are you using the v1.4.0? https://github.com/k8snetworkplumbingwg/sriov-network-operator/releases/tag/v1.4.0, Currently, Spiderpool is using v1.3.0, We haven't test it for v1.4.0 now, We'll check that next month.

I could not get static IPAM to work with sriov device plugin. That got completely ignored by KubeVirt VM with sriov. The only way to assign an IP to a KubeVirt VM with sriov is via cloudinit.

so I think this is more of a kubevirt issue? can you open an issue for this to kubevirt?

[haiwu]

I think so, although its code here about grubby is from 2 years ago: https://github.com/k8snetworkplumbingwg/sriov-network-operator/blob/2b02ba1fe885224f47a6bcdae95cd8a5f0a5c6e8/bindata/scripts/enable-kargs.sh#L20.

Just noticed upstream there trying to address this intel_iommu issue 2 hours ago: k8snetworkplumbingwg/sriov-network-operator#796. Think they still miss something..

I think this is by design for Kubevirt. Just found the comment on this: kubevirt/kubevirt#10358:
"You cannot have IPAM on SR-IOV VF/s when used with VM/s.".