Antrea ovs crashes everyday at 00:0x

[ColonelBundy]

Describe the bug
antrea-ovs container seems to be crashing everyday in an error state.

To Reproduce
Not clear other than running antrea and letting the log file roll over at 00:00 UTC

Actual behavior
Antrea crahes presumably due to these error messages:

E0301 00:08:42.759037 1 proxier.go:316] "Error when removing flows of stale Endpoints for Service" err="message is canceled because of disconnection from the Switch" ServicePortName="kube-system/antrea-agent:"
W0301 00:08:42.354091 1 ofSwitch.go:223] Received ERROR message from switch 00:00:e6:6e:9d:ef:5a:4f. Err: EOF
E0301 00:08:42.353959 1 stream.go:153] "InboundError" err="EOF"
2024-03-01T00:08:42.343629747+00:00 error: failed to rename /var/log/openvswitch/ovs-vswitchd.log to /var/log/openvswitch/ovs-vswitchd.log.1: Permission denied

I suspect the log file rolls over and keeps X amount of logs on disk, which it seems like it does not have permissions to do.
All agents crash at the same time so dropped packets has been observed.

Versions:
Openshift 4.12-4.14
antrea/antrea-ubi image is used and it seems all Anrea versions are affected (1.7.1 and onwards at least)

Additional context
We are managing antrea through https://github.com/vmware/antrea-operator-for-kubernetes

[tnqn]

@xliuxu I remember you handled the issue before. Have this been fixed? If yes, can you share which version includes the fix?

[xliuxu]

I can confirm this issue with the latest UBI image. It is caused by the mismatch user when performing logrotate. We should use root user instead of openvswitch in our scenario. I will create a PR to fix this issue.

https://github.com/openvswitch/ovs/blob/master/rhel/openvswitch-fedora.spec.in#L323