Skip to content
This repository has been archived by the owner on Jun 26, 2023. It is now read-only.

HNC: Avoid restart HNC after adding a new CRD #488

Closed
sophieliu15 opened this issue Mar 4, 2020 · 6 comments · Fixed by #514
Closed

HNC: Avoid restart HNC after adding a new CRD #488

sophieliu15 opened this issue Mar 4, 2020 · 6 comments · Fixed by #514
Assignees
@sophieliu15

Comments

@sophieliu15
Copy link
Contributor

sophieliu15 commented Mar 4, 2020 •
edited
Loading

Right now when HNC is running and we add a new CRD and hope to config the new type, there are two problems:

  • Reconciliation of ConfigReconciler will not be triggered automatically. We need to change config object to trigger reconciliation.
  • Even after manually triggering the reconciliation from ConfigReconciler, ObjectReconciler will not recognize the CRD and as a result corresponding object reconciler cannot be created.

In order to create the object reconciler for the new type, we need to kill the manager pod to restart HNC. We should either avoid restarting HNC after adding a new CRD, or prevent users from adding a new CRD after starting HNC.
@sophieliu15
Copy link
Contributor Author

/assign @sophieliu15

@sophieliu15
Copy link
Contributor Author

Discussion here: #476 (comment)

@sophieliu15 sophieliu15 mentioned this issue Mar 4, 2020
Merged
@sophieliu15
Copy link
Contributor Author

The first problem can be resolved by letting ConfigReconciler watch for changes in crds. I will address this problem after resolving the second problem (see below).

The cause of the second problem is that the controller manager's client's RESTMapper is initialized at startup and never updates when adding new CRDs to the apiserver. Therefore, the client is only aware of CRDs installed before starting the controller manager. This issue was address in kubernetes-sigs/controller-runtime#321 and was fixed in kubernetes-sigs/controller-runtime#544. However, after updating controller-runtime from current version (v0.2.2) to the latest version (v0.5.0), the object reconciler could be created successfully even when the CRD is not installed. It then received an error complaining the CRD was not found when the Reconcile method was called. I haven't found a way to check CRD before creating the reconciler or figured out the reason why the reconciler creation could succeed after the update. See discussion about this issue in kubernetes-sigs/controller-runtime#840. Working code can be found here.

@sophieliu15
Copy link
Contributor Author

Add @adrianludwin and @yiqigao217 in the loop.

@adrianludwin
Copy link
Contributor

Nice digging! If controller-runtime doesn't reject it, we'll just have to use the Discovery API to figure out if a type exists ourselves before we create the object reconciler. I have no idea how to do that but I suspect it's not too terribly difficult.

sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 6, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
14345b9 
Reason for upgrade:
The new version uses [DynamicRESTMapper](https://github.com/kubernetes-sigs/controller-runtime/blob/master/pkg/client/apiutil/dynamicrestmapper.go) as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR:

- Upgrade go version in `go.mod` and `Dockerfile` to 1.13. The [errors.As](https://golang.org/pkg/errors/#As) in [dynamicrestmapper.go](https://github.com/kubernetes-sigs/controller-runtime/blob/bfc982769615817ee15db8a543662652d900d27b/pkg/client/apiutil/dynamicrestmapper.go#L48) requires go 1.13.
- A higher version for k8s.io/cli-runtime and k8s.io/client-go are required after upgrading controller-runtime to v0.5.0
- Version changes of other packages in `go.mod` are updated automatically.
- serializer.DirectCodecFactory was renamed to [serializer.WithoutConversionCodecFactory](https://godoc.org/k8s.io/apimachinery/pkg/runtime/serializer#WithoutConversionCodecFactory) after k8s.io/apimachinery 1.16 (see [here](kubernetes/apimachinery@ed8af17), and [here](kubernetes/apimachinery@4fac835))
- Default [LeaderElectionID](https://github.com/kubernetes-sigs/controller-runtime/blob/bfc982769615817ee15db8a543662652d900d27b/pkg/leaderelection/leader_election.go#L46) in controller-runtime manager is removed after kubernetes-sigs/controller-runtime#446.
- [NewlineReporter](https://godoc.org/sigs.k8s.io/controller-runtime/pkg/envtest/printer) is moved from `sigs.k8s.io/controller-runtime/pkg/envtest/` to `https://godoc.org/sigs.k8s.io/controller-runtime/pkg/envtest/printer` by [this](kubernetes-sigs/controller-runtime@748f55d#diff-42de1d59fbe8f8b90154f01dd01f5748) commit.
- In controller-runtime v0.2.2, if a resource does not exist, a
controller cannot be created successfully. After update
controller-runtime to v0.5.0, a controller can be created without error.
However, when the `Reconcile` method is triggered, there will be an
error complaining the resource does not exist. Therefore, we will
explicitly check if a resource exists before creating the corresponding
object reconciler in `createObjectReconciler` in `hnc_config.go` (see
details in kubernetes-sigs/controller-runtime#840)

Tested:

- Unit tests.
- Went through [demo script](https://docs.google.com/document/d/1tKQgtMSf0wfT3NOGQx9ExUQ-B8UkkdVZB6m4o3Zqn64/edit#) to make sure HNC behaves as expected on a GKE cluster.
- Manually test if the PR solves the restart problem as described in kubernetes-retired#488 with following workflow:
     - Install HNC
     - Install a new CRD
     - Config the new type in `config` singleton

Before this PR, corresponding object reconciler for the new type will not be created unless we restart HNC. After the change, corresponding object reconciler can be created and it reconciles objects of the new type as expected without restarting HNC.

This partly solve:  kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 6, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
f7d1a30 
Reason for upgrade:
The new version uses [DynamicRESTMapper](https://github.com/kubernetes-sigs/controller-runtime/blob/master/pkg/client/apiutil/dynamicrestmapper.go) as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR: see PR description

Tested:

- Unit tests.
- Went through [demo script](https://docs.google.com/document/d/1tKQgtMSf0wfT3NOGQx9ExUQ-B8UkkdVZB6m4o3Zqn64/edit#) to make sure HNC behaves as expected on a GKE cluster.
- Manually test if the PR solves the restart problem as described in kubernetes-retired#488 with following workflow:
     - Install HNC
     - Install a new CRD
     - Config the new type in `config` singleton

Before this PR, corresponding object reconciler for the new type will not be created unless we restart HNC. After the change, corresponding object reconciler can be created and it reconciles objects of the new type as expected without restarting HNC.

This partly solve:  kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 6, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
3b8d658 
Reason for upgrade:
The new version uses DynamicRESTMapper as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR: see PR description

Tested:

- Unit tests.
- Went through demo script to make sure HNC behaves as expected on a GKE cluster.
- Manually test if the PR solves the restart problem as described in kubernetes-retired#488 with following workflow:
     - Install HNC
     - Install a new CRD
     - Config the new type in `config` singleton

Before this PR, corresponding object reconciler for the new type will not be created unless we restart HNC. After the change, corresponding object reconciler can be created and it reconciles objects of the new type as expected without restarting HNC.

This partly solve:  kubernetes-retired#488
@sophieliu15 sophieliu15 mentioned this issue Mar 6, 2020
Merged
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 6, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
12f3961 
Reason for upgrade:
The new version uses DynamicRESTMapper as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR: see PR description

Tested:

- Unit tests.
- Went through demo script to make sure HNC behaves as expected on a GKE cluster.
- Manually test if the PR solves the restart problem as described in kubernetes-retired#488 with following workflow:
     - Install HNC
     - Install a new CRD
     - Config the new type in `config` singleton

Before this PR, corresponding object reconciler for the new type will not be created unless we restart HNC. After the change, corresponding object reconciler can be created and it reconciles objects of the new type as expected without restarting HNC.

This partly solve:  kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 6, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
806014b 
Reason for upgrade:
The new version uses DynamicRESTMapper as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR: see PR description

Tested:

- Unit tests.
- Went through demo script to make sure HNC behaves as expected on a GKE cluster.
- Manually test if the PR solves the restart problem as described in kubernetes-retired#488 with following workflow:
     - Install HNC
     - Install a new CRD
     - Config the new type in `config` singleton

Before this PR, corresponding object reconciler for the new type will not be created unless we restart HNC. After the change, corresponding object reconciler can be created and it reconciles objects of the new type as expected without restarting HNC.

This partly solve:  kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 6, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
673b588 
Reason for upgrade:
The new version uses DynamicRESTMapper as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR: see PR description

Tested:

- Unit tests.
- Went through demo script to make sure HNC behaves as expected on a GKE cluster.
- Manually test if the PR solves the restart problem as described in kubernetes-retired#488 with following workflow:
     - Install HNC
     - Install a new CRD
     - Config the new type in `config` singleton

Before this PR, corresponding object reconciler for the new type will not be created unless we restart HNC. After the change, corresponding object reconciler can be created and it reconciles objects of the new type as expected without restarting HNC.

This partly solve:  kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 7, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
7fc7304 
Reason for upgrade:
The new version uses DynamicRESTMapper as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR: see PR description

Tested:

- Unit tests.
- Went through demo script to make sure HNC behaves as expected on a GKE cluster.
- Manually test if the PR solves the restart problem as described in kubernetes-retired#488 with following workflow:
     - Install HNC
     - Install a new CRD
     - Config the new type in `config` singleton

Before this PR, corresponding object reconciler for the new type will not be created unless we restart HNC. After the change, corresponding object reconciler can be created and it reconciles objects of the new type as expected without restarting HNC.

This partly solve:  kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 7, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
804c6b9 
Reason for upgrade:
The new version uses DynamicRESTMapper as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR: see PR description

Tested:

- Unit tests.
- Went through demo script to make sure HNC behaves as expected on a GKE cluster.
- Manually test if the PR solves the restart problem as described in kubernetes-retired#488 with following workflow:
     - Install HNC
     - Install a new CRD
     - Config the new type in `config` singleton

Before this PR, corresponding object reconciler for the new type will not be created unless we restart HNC. After the change, corresponding object reconciler can be created and it reconciles objects of the new type as expected without restarting HNC.

This partly solve:  kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 7, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
e01d127 
Reason for upgrade:
The new version uses DynamicRESTMapper as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR: see PR description

Tested:

- Unit tests.
- Went through demo script to make sure HNC behaves as expected on a GKE cluster.
- Manually test if the PR solves the restart problem as described in kubernetes-retired#488 with following workflow:
     - Install HNC
     - Install a new CRD
     - Config the new type in `config` singleton

Before this PR, corresponding object reconciler for the new type will not be created unless we restart HNC. After the change, corresponding object reconciler can be created and it reconciles objects of the new type as expected without restarting HNC.

This partly solve:  kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 7, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
7ea29e2 
Reason for upgrade:
The new version uses DynamicRESTMapper as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR: see PR description

Tested:

- Unit tests.
- Went through demo script to make sure HNC behaves as expected on a GKE cluster.
- Manually test if the PR solves the restart problem as described in kubernetes-retired#488 with following workflow:
     - Install HNC
     - Install a new CRD
     - Config the new type in `config` singleton

Before this PR, corresponding object reconciler for the new type will not be created unless we restart HNC. After the change, corresponding object reconciler can be created and it reconciles objects of the new type as expected without restarting HNC.

This partly solve:  kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 9, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
17e2ee1 
Reason for upgrade:
The new version uses DynamicRESTMapper as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR: see PR description

Tested:

- Unit tests.
- Went through demo script to make sure HNC behaves as expected on a GKE cluster.
- Manually test if the PR solves the restart problem as described in kubernetes-retired#488 with following workflow:
     - Install HNC
     - Install a new CRD
     - Config the new type in `config` singleton

Before this PR, corresponding object reconciler for the new type will not be created unless we restart HNC. After the change, corresponding object reconciler can be created and it reconciles objects of the new type as expected without restarting HNC.

This partly solve:  kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 9, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
d036ebb 
Reason for upgrade:
The new version uses DynamicRESTMapper as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR: see PR description

Tested:

- Unit tests.
- Went through demo script to make sure HNC behaves as expected on a GKE cluster.
- Manually test if the PR solves the restart problem as described in kubernetes-retired#488 with following workflow:
     - Install HNC
     - Install a new CRD
     - Config the new type in `config` singleton

Before this PR, corresponding object reconciler for the new type will not be created unless we restart HNC. After the change, corresponding object reconciler can be created and it reconciles objects of the new type as expected without restarting HNC.

This partly solve:  kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 9, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
4f87cf6 
Reason for upgrade:
The new version uses DynamicRESTMapper as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR: see PR description

Tested:

- Unit tests.
- Went through demo script to make sure HNC behaves as expected on a GKE cluster.
- Manually test if the PR solves the restart problem as described in kubernetes-retired#488 with following workflow:
     - Install HNC
     - Install a new CRD
     - Config the new type in `config` singleton

Before this PR, corresponding object reconciler for the new type will not be created unless we restart HNC. After the change, corresponding object reconciler can be created and it reconciles objects of the new type as expected without restarting HNC.

This partly solve:  kubernetes-retired#488
@sophieliu15
Copy link
Contributor Author

sophieliu15 commented Mar 9, 2020 •
edited
Loading

From discussion in kubernetes-sigs/controller-runtime#840 following are expected when a GVK does not exist after upgrading to controller-runtime v0.5.0:

  • controller can be created successfully
  • controller and manager will both terminate

We will have 3 PRs to address this issue:

sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 9, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
66b1901 
Reason for upgrade:
The new version uses DynamicRESTMapper as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR: see PR description

Tested:

- Unit tests.
- Went through demo script to make sure HNC behaves as expected on a GKE cluster.
- Manually test if the PR solves the restart problem as described in kubernetes-retired#488 with following workflow:
     - Install HNC
     - Install a new CRD
     - Config the new type in `config` singleton

Before this PR, corresponding object reconciler for the new type will not be created unless we restart HNC. After the change, corresponding object reconciler can be created and it reconciles objects of the new type as expected without restarting HNC.

This partly solve:  kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 9, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
ff27d39 
Reason for upgrade:
The new version uses DynamicRESTMapper as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR: see PR description

Tested:

- Unit tests.
- Went through demo script to make sure HNC behaves as expected on a GKE cluster.
- Manually test if the PR solves the restart problem as described in kubernetes-retired#488 with following workflow:
     - Install HNC
     - Install a new CRD
     - Config the new type in `config` singleton

Before this PR, corresponding object reconciler for the new type will not be created unless we restart HNC. After the change, corresponding object reconciler can be created and it reconciles objects of the new type as expected without restarting HNC.

This partly solve:  kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 9, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
28e54a5 
Reason for upgrade:
The new version uses DynamicRESTMapper as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR: see PR description

Tested:

- Unit tests.
- Went through demo script to make sure HNC behaves as expected on a GKE cluster.
- Manually test if the PR solves the restart problem as described in kubernetes-retired#488 with following workflow:
     - Install HNC
     - Install a new CRD
     - Config the new type in `config` singleton

Before this PR, corresponding object reconciler for the new type will not be created unless we restart HNC. After the change, corresponding object reconciler can be created and it reconciles objects of the new type as expected without restarting HNC.

This partly solve:  kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 9, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
99e1b7e 
Reason for upgrade:
The new version uses DynamicRESTMapper as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR: see PR description

Tested:

- Unit tests.
- Went through demo script to make sure HNC behaves as expected on a GKE cluster.
- Manually test if the PR solves the restart problem as described in kubernetes-retired#488 with following workflow:
     - Install HNC
     - Install a new CRD
     - Config the new type in `config` singleton

Before this PR, corresponding object reconciler for the new type will not be created unless we restart HNC. After the change, corresponding object reconciler can be created and it reconciles objects of the new type as expected without restarting HNC.

This partly solve:  kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 9, 2020
@sophieliu15
Upgrade controller-runtime from v0.2.2 to v0.5.0.
3e7c4ba 
Reason for upgrade:
The new version uses DynamicRESTMapper as default `RESTMapper` for controller runtime manager. `DynamicRESTMapper` will "reload the delegated `meta.RESTMapper` on a cache miss" (see kubernetes-sigs/controller-runtime#554), which can solve problem that we need to restart HNC after adding a new CRD to create corresponding object reconciler when using controller-runtime v0.2.2 (see details in kubernetes-retired#488).

Incompatibility issues addressed in this PR: see PR description

Tested:

- Unit tests.
- Went through demo script to make sure HNC behaves as expected on a GKE cluster.

This partly solve:  kubernetes-retired#488
This was referenced Mar 10, 2020
Merged
Merged
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 11, 2020
@sophieliu15
Make ConfigReconciler watches for custom resources
709c3ce 
This PR makes ConfigReconciler watches for custom resources and triggers reconciliation of the config singleton when any custom resource is created/updated.

Tested: unit tests, GKE cluster

closes kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 11, 2020
@sophieliu15
Make ConfigReconciler watches for custom resources
62ce684 
This PR makes ConfigReconciler watches for custom resources and triggers reconciliation of the config singleton when any custom resource is created/updated.

Tested: unit tests, GKE cluster

part of kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 11, 2020
@sophieliu15
Make ConfigReconciler watches for custom resources
acf202e 
This PR makes ConfigReconciler watches for custom resources and triggers reconciliation of the config singleton when any custom resource is created/updated.

Tested: unit tests, GKE cluster

part of kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 12, 2020
@sophieliu15
Make ConfigReconciler watches for custom resources
61906f3 
This PR makes ConfigReconciler watches for custom resources and triggers reconciliation of the config singleton when any custom resource is created/updated.

Tested: unit tests, GKE cluster

part of kubernetes-retired#488
sophieliu15 added a commit to sophieliu15/multi-tenancy that referenced this issue Mar 12, 2020
@sophieliu15
Make ConfigReconciler watches for custom resources
e459641 
This PR makes ConfigReconciler watches for custom resources and triggers reconciliation of the config singleton when any custom resource is created/updated.

Tested: unit tests, GKE cluster

part of kubernetes-retired#488
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@sophieliu15 sophieliu15

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Make ConfigReconciler watches for custom resources sophieliu15/multi-tenancy
2 participants
@adrianludwin @sophieliu15