Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow namespace scoped operator #719

Merged
merged 10 commits into from
Feb 29, 2024
Merged

Allow namespace scoped operator #719

merged 10 commits into from
Feb 29, 2024

Conversation

spilchen
Copy link
Collaborator

This change allows for deploying the operator at the namespace scope, only for Helm deployments. When deployed this way, the operator will watch only custom resources within its deployment namespace. Multiple namespace deployments are possible.

By default, the webhook is deployed with the operator. However, with multiple namespace scoped deployments in a cluster, the webhook can only be deployed once.

There are two ways to handle this:

  • Set webhook.enable to false for any subsequent operator deployment
  • Deploy only the webhook (controllers.enable set to false) and deploy each operator with the webhook disabled (webhook.enable set to false)

A new Helm chart parameter, controllers.scope, has been added to control the scope of the operator, with valid values of "cluster" or "namespace" and a default of "cluster."

The e2e-leg-2 has been converted to deploy the operator with namespace scope.

All command-line options used by the operator have been replaced with environment variables read from a ConfigMap. This change was made when adding the namespace scope and controllers disable options, as this method is more extensible.

This change removes most of the logging Helm chart parameters, leaving only logging.level. Those using the logging Helm chart parameters can set appropriate entries in the ConfigMap to retain the old behavior.

Finally, the 'make run' option has been removed. The operator can now only be run by running a container within Kubernetes, as it was too difficult to set up a ConfigMap and environment variables to run the operator.

@spilchen spilchen self-assigned this Feb 27, 2024
@spilchen spilchen changed the title Namespace scoped operator deployment Allow namespace scoped operator Feb 27, 2024
Copy link
Collaborator

@roypaulin roypaulin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First round. I am still looking at other files.

pkg/opcfg/config.go Outdated Show resolved Hide resolved
pkg/opcfg/config.go Outdated Show resolved Hide resolved
pkg/opcfg/config.go Outdated Show resolved Hide resolved
pkg/opcfg/config.go Outdated Show resolved Hide resolved
pkg/opcfg/config.go Outdated Show resolved Hide resolved
pkg/opcfg/config.go Outdated Show resolved Hide resolved
pkg/opcfg/config.go Outdated Show resolved Hide resolved
pkg/controllers/vdb/verticadb_controller.go Show resolved Hide resolved
pkg/vadmin/at.go Outdated
@@ -45,11 +46,11 @@ func (a *Admintools) execAdmintools(ctx context.Context, initiatorPod types.Name
// Dump relevant contents of the admintools.conf before and after the
// admintools calls. We do this for PD purposes to see what changes occurred
// in the file.
if a.DevMode {
if opcfg.GetIsDebugLoggingEnabled() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We were doing this in dev mode so why in debug mode now?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We used devMode as a proxy for extra debugging. I think its clearer if we use the log level. I should have done this initially.


# 21. Change change ClusterRoles/ClusterRoleBindings for the manager to be
# Roles/RoleBindings if the operator is scoped to a single namespace.
for f in $TEMPLATE_DIR/verticadb-operator-manager-clusterrolebinding-crb.yaml \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where do you check if the operator is namespace-scoped?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are three places:

  • in helm-charts/verticadb-operator/templates/_helpers.tpl, we check to know if a ClusterRole/ClusterRoleBinding should really be a Role/RoleBinding. See the "vdb-op.roleKind" and "vdb-op.roleBindingKing" defines.
  • in PodSecurityReconciler, with a namespace scoped operator we don't have permissions to read the annotations of a namespace anymore.
  • in pkg/opcfg/config, we use the scope to know what namespace the operator needs to watch

@spilchen spilchen merged commit 04390c6 into main Feb 29, 2024
30 checks passed
@spilchen spilchen deleted the spilchen/webhook-decouple branch February 29, 2024 01:18
spilchen pushed a commit that referenced this pull request Mar 1, 2024
The helm chart parameter `prometheus.expose` controls if the Prometheus
metrics in the operator are exposed. The old default was
`ExposeWithProxyAuth`. However, this required a
ClusterRole/ClusterRoleBinding to be created. This can be onerous,
especially in environments where creation of those is prohibited except
for cluster admins. The new default is `Disable` to make it easier to
deploy in a more restricted environment.

This also adds back logging helm chart parameters that were removed in
#719. Instead, those parameters have been deprecated and will be removed
in a future version.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants