eBPF Manager Tech Preview - Add limitations of running INFW with eBPF Manager

[Billy99]

Which section(s) is the issue in?

A list of the limitations of using eBPF Manager (bpfman) with Ingress Node Firewall needs to be added somewhere in the documents, either in the eBPF Manager portion of the docs, or with the INFW section that show how to use with eBPF Manager.

What needs fixing?

We need to add a list of the limitations of using bpfman with INFW somewhere. Indicate that with bpfman still in Tech Preview, there are some nuances to using it. @msherif1234 has a better list, but here are a few that come to mind:

• TCX is currently not supported in bpfman, which INFW uses as a backup if XDP is not supported. So INFW won't work in this release (will be available next release) with bpfman on systems that don't support XDP (i.e. ROSA).
• When deployed with bpfman operator, the INFW daemonset will remain in the "container creating" state until the rules are applied. Everything is working fine, the INFW daemonset pods are just waiting for the eBPF maps to be created and volume mounted into the pods. It can't do anything until the maps are created anyway. This is a security feature which avoids mounting the eBPF maps on the host, which in turn avoids having to grant INFW pods access to the host filesystem.
• Ideally when INFW is deployed with bpfman operator, the daemonset pods would run as unprivileged. There is an issue with perf event arrays (still being investigated) that is preventing access without privileged mode.