Disable kube-rbac-proxy from prometheus-exporter-operator controller-manager

[slopezz]

kube-rbac-proxy is used by default on operator-sdk to protect operator metrics path, in case you don't want anyone in the cluster but only k8s authenticated resources can access to them.

But there are 2 current issues with kube-rbac-proxy:

1. Upgrade from kube-rbac-proxy:v0.5.0 to latest kube-rbac-proxy:v0.8.0 Ansible operator-sdk v1.5.0 with updated kube-rbac-proxy:v0.8.0 fails to run with permission denied operator-framework/operator-sdk#4684 causes error on OpenShift 4.6+:

Error: container create failed: time="2021-03-19T15:51:12Z" level=error msg="container_linux.go:366: starting container     process caused: chdir to cwd (\"/home/nonroot\") set in config.json failed: permission denied"

• The current solution is to use a different proxy image on OpenShift 4.6+ openshift4/ose-kube-rbac-proxy:v4.7.0 (which works OK), but this image is behind registry.redhat.io registry which requires authenticated if not using Openshift (it doesn't work directly on vanilla k8s), so you need to maintain 2 different bundles with different proxy images if you want to run the operator on both OpenShift or K8s, which makes maintenance more complex.

2. In addition, OpenShift User Workload Monitoring (OpenShift official monitoring stack) does not support ServiceMonitor with bearerTokenFile field Prometheus ServiceMonitor failing to scrape operator metrics served though kube-proxy HTTPS 8443 port operator-framework/operator-sdk#4764 (comment) (which is needed to scrape metrics behind kube-rbac-proxy) , so it seems there is no way of having operator metrics with auth if using OCP UWM.

For that reason, taking into account that operator metrics are not that important to have them with forced auth, I have disabled kube-rbac-proxy container (making a few changes to make that work with a new patch, and leaving default proxy yamls there, in case they want to be enabled easily in the future), so anyone once inside the cluster could check operator metrics without any problem on both OCP and k8s (even if using OCP UWM).

[slopezz]

/kind feature
/priority important-soon
/label size/xs
/assign

[roivaz]

/lgtm

[3scale-robot]

LGTM label has been added.

Git tree hash: 81832963d343714fa6597e71a85f79fc36b6afa0

[raelga]

/lgtm

[raelga]

/shrug

[slopezz]

/approve

[3scale-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: slopezz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [slopezz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment