Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sriov network operator fails on vanilla k8s #218

Closed
adrianchiris opened this issue Dec 19, 2021 · 3 comments · Fixed by #219
Closed

sriov network operator fails on vanilla k8s #218

adrianchiris opened this issue Dec 19, 2021 · 3 comments · Fixed by #219
Labels
bug Something isn't working

Comments

@adrianchiris
Copy link
Collaborator 

Entering sleep... (success)
Entering sleep... (success)
I1218 22:22:44.635369    6970 start.go:133] starting node writer
I1218 22:22:44.657919    6970 start.go:153] Running on platform: Baremetal
I1218 22:22:44.657961    6970 writer.go:44] Run(): start writer
I1218 22:22:44.657977    6970 writer.go:47] Run(): once
I1218 22:22:44.731918    6970 utils.go:589] getLinkType(): Device 0000:06:00.0
W1218 22:22:44.732426    6970 utils.go:119] DiscoverSriovDevices(): unable to get device mode 0000:06:00.0 "no such device"
I1218 22:22:44.734002    6970 utils.go:589] getLinkType(): Device 0000:06:00.1
W1218 22:22:44.734321    6970 utils.go:119] DiscoverSriovDevices(): unable to get device mode 0000:06:00.1 "no such device"
I1218 22:22:44.753685    6970 writer.go:132] setNodeStateStatus(): syncStatus: , lastSyncError: 
I1218 22:22:44.764414    6970 writer.go:170] writeCheckpointFile(): try to decode the checkpoint file
I1218 22:22:44.764817    6970 start.go:159] Starting SriovNetworkConfigDaemon
I1218 22:22:44.764932    6970 writer.go:44] Run(): start writer
I1218 22:22:44.764982    6970 daemon.go:228] Run(): start daemon
I1218 22:22:44.867503    6970 utils.go:510] LoadKernelModule(): try to load kernel module tun
I1218 22:22:44.945165    6970 daemon.go:418] Set log verbose level to: 2
I1218 22:22:50.043012    6970 daemon.go:295] Starting workers
I1218 22:22:50.043060    6970 daemon.go:298] Started workers
I1218 22:22:50.043107    6970 daemon.go:338] worker queue size: 1
I1218 22:22:50.043135    6970 daemon.go:340] get item: 1
I1218 22:22:50.043151    6970 daemon.go:430] nodeStateSyncHandler(): new generation is 1
I1218 22:22:50.049006    6970 daemon.go:938] registerPlugins(): [mellanox_plugin]
I1218 22:22:50.049106    6970 daemon.go:648] loadVendorPlugins(): try to load plugin mellanox_plugin
I1218 22:22:50.049121    6970 plugin.go:38] loadPlugin(): load plugin from /plugins/mellanox_plugin.so
I1218 22:22:50.049181    6970 writer.go:61] Run(): refresh trigger
I1218 22:22:50.049221    6970 writer.go:80] pollNicStatus()
I1218 22:22:50.049246    6970 utils.go:52] DiscoverSriovDevices
I1218 22:22:50.088161    6970 daemon.go:648] loadVendorPlugins(): try to load plugin k8s_plugin
I1218 22:22:50.088219    6970 plugin.go:38] loadPlugin(): load plugin from /plugins/k8s_plugin.so
I1218 22:22:50.121789    6970 utils.go:406] getNetdevMTU(): get MTU for device 0000:06:00.0
I1218 22:22:50.121902    6970 utils.go:401] tryGetInterfaceName(): name is enp6s0f0
I1218 22:22:50.122021    6970 utils.go:401] tryGetInterfaceName(): name is enp6s0f0
I1218 22:22:50.122033    6970 utils.go:427] getNetDevMac(): get Mac for device enp6s0f0
I1218 22:22:50.122078    6970 utils.go:439] getNetDevLinkSpeed(): get LinkSpeed for device enp6s0f0
I1218 22:22:50.123041    6970 utils.go:589] getLinkType(): Device 0000:06:00.0
I1218 22:22:50.123424    6970 utils.go:642] GetNicSriovMode(): device 0000:06:00.0
W1218 22:22:50.123538    6970 utils.go:119] DiscoverSriovDevices(): unable to get device mode 0000:06:00.0 "no such device"
I1218 22:22:50.123659    6970 utils.go:406] getNetdevMTU(): get MTU for device 0000:06:00.1
I1218 22:22:50.123749    6970 utils.go:401] tryGetInterfaceName(): name is enp6s0f1
I1218 22:22:50.123886    6970 utils.go:401] tryGetInterfaceName(): name is enp6s0f1
I1218 22:22:50.123898    6970 utils.go:427] getNetDevMac(): get Mac for device enp6s0f1
I1218 22:22:50.123936    6970 utils.go:439] getNetDevLinkSpeed(): get LinkSpeed for device enp6s0f1
I1218 22:22:50.124593    6970 utils.go:589] getLinkType(): Device 0000:06:00.1
I1218 22:22:50.124759    6970 utils.go:642] GetNicSriovMode(): device 0000:06:00.1
W1218 22:22:50.124863    6970 utils.go:119] DiscoverSriovDevices(): unable to get device mode 0000:06:00.1 "no such device"
I1218 22:22:50.131286    6970 writer.go:132] setNodeStateStatus(): syncStatus: InProgress, lastSyncError: 
E1218 22:22:50.144373    6970 runtime.go:78] Observed a panic: &fs.PathError{Op:"open", Path:"bindata/manifests/switchdev-config/switchdev-units/switchdev-configuration.yaml", Err:0x2} (open bindata/manifests/switchdev-config/switchdev-units/switchdev-configuration.yaml: no such file or directory)
goroutine 141 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic(0x2053720, 0xc0005b21b0)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:74 +0x95
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:48 +0x86
panic(0x2053720, 0xc0005b21b0)
	/usr/local/go/src/runtime/panic.go:965 +0x1c5
github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/plugins/k8s.init.0()
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/plugins/k8s/k8s_plugin.go:92 +0x151
plugin.open(0xc001136000, 0x13, 0x223fa7a, 0x21, 0xc000891900)
	/usr/local/go/src/plugin/plugin_dlopen.go:101 +0xc1e
plugin.Open(...)
	/usr/local/go/src/plugin/plugin.go:32
github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon.loadPlugin(0xc001136000, 0x16, 0x224bf6a, 0x2a, 0xc0008919c8, 0x1)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon/plugin.go:39 +0xe5
github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon.(*Daemon).loadVendorPlugins(0xc000a76340, 0xc000683b00, 0xc00013a030, 0xc00004203a)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon/daemon.go:649 +0x25d
github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon.(*Daemon).nodeStateSyncHandler(0xc000a76340, 0x1, 0xc0013100d0, 0xc000258090)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon/daemon.go:459 +0x10ed
github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon.(*Daemon).processNextWorkItem.func1(0xc000a76340, 0x1f99580, 0x2f91408, 0x0, 0x0)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon/daemon.go:361 +0xdf
github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon.(*Daemon).processNextWorkItem(0xc000a76340, 0x203000)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon/daemon.go:377 +0x169
github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon.(*Daemon).runWorker(...)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon/daemon.go:322
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc0003e5430)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc0003e5430, 0x24524a0, 0xc000507b90, 0x1, 0xc0005ac120)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156 +0x9b
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0003e5430, 0x3b9aca00, 0x0, 0xc00051e601, 0xc0005ac120)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.Until(0xc0003e5430, 0x3b9aca00, 0xc0005ac120)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90 +0x4d
created by github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon.(*Daemon).Run
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon/daemon.go:297 +0xb05
panic: open bindata/manifests/switchdev-config/switchdev-units/switchdev-configuration.yaml: no such file or directory [recovered]
	panic: open bindata/manifests/switchdev-config/switchdev-units/switchdev-configuration.yaml: no such file or directory

goroutine 141 [running]:
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:55 +0x109
panic(0x2053720, 0xc0005b21b0)
	/usr/local/go/src/runtime/panic.go:965 +0x1c5
github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/plugins/k8s.init.0()
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/plugins/k8s/k8s_plugin.go:92 +0x151
plugin.open(0xc001136000, 0x13, 0x223fa7a, 0x21, 0xc000891900)
	/usr/local/go/src/plugin/plugin_dlopen.go:101 +0xc1e
plugin.Open(...)
	/usr/local/go/src/plugin/plugin.go:32
github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon.loadPlugin(0xc001136000, 0x16, 0x224bf6a, 0x2a, 0xc0008919c8, 0x1)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon/plugin.go:39 +0xe5
github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon.(*Daemon).loadVendorPlugins(0xc000a76340, 0xc000683b00, 0xc00013a030, 0xc00004203a)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon/daemon.go:649 +0x25d
github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon.(*Daemon).nodeStateSyncHandler(0xc000a76340, 0x1, 0xc0013100d0, 0xc000258090)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon/daemon.go:459 +0x10ed
github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon.(*Daemon).processNextWorkItem.func1(0xc000a76340, 0x1f99580, 0x2f91408, 0x0, 0x0)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon/daemon.go:361 +0xdf
github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon.(*Daemon).processNextWorkItem(0xc000a76340, 0x203000)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon/daemon.go:377 +0x169
github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon.(*Daemon).runWorker(...)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon/daemon.go:322
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc0003e5430)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc0003e5430, 0x24524a0, 0xc000507b90, 0x1, 0xc0005ac120)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156 +0x9b
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0003e5430, 0x3b9aca00, 0x0, 0xc00051e601, 0xc0005ac120)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.Until(0xc0003e5430, 0x3b9aca00, 0xc0005ac120)
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90 +0x4d
created by github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon.(*Daemon).Run
	/go/src/github.com/k8snetworkplumbingwg/sriov-network-operator/pkg/daemon/daemon.go:297 +0xb05
@adrianchiris adrianchiris added the bug Something isn't working label Dec 19, 2021
@adrianchiris
Copy link
Collaborator Author

@zshi-redhat this seems to be related to #202

@adrianchiris adrianchiris mentioned this issue Dec 19, 2021
Merged
@zshi-redhat zshi-redhat mentioned this issue Dec 20, 2021
Merged
@zshi-redhat
Copy link
Collaborator

@zshi-redhat this seems to be related to #202

Yes, you're right, I didn't update the service profiles in k8s plugin.
I created a PR #219 for this issue.

@adrianchiris
Copy link
Collaborator Author

Thx @zshi-redhat will review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add switchdev-after-networkmanager service file to k8s plugin zshi-redhat/sriov-network-operator-1
2 participants
@zshi-redhat @adrianchiris