Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revise TidbMonitor controller #1500

Merged
merged 20 commits into from
Jan 15, 2020
Merged

Conversation

Yisaer
Copy link
Contributor

@Yisaer Yisaer commented Jan 7, 2020

What problem does this PR solve?

  1. fix the bug for the log level of the Prometheus Spec in TidbMonitor which would cause TidbMonitor Failed to start. TidbMonitor: should not specify log level when prometheus.logLevel is empty #1482

  2. Add default namespace for theClusterRef if it's empty in TidbMonitor Spec. no default for .spec.clusterRef.namespace of TiDB monitor #1480

  3. Add events for TidbMonitor Controller for a more user-friendly experience. clearer message when TidbMonitor failed syncing because of the absence of referenced cluster #1481

Does this PR introduce a user-facing change?:

Adding default ClusterRef namespace for TidbMonitor as same as it deployed and fix the bug when TidbMonitor's Pod can't be created when Spec.PrometheusSpec.logLevel is missing.

@Yisaer Yisaer added area/monitor monitoring area/controller status/WIP Issue/PR is being worked on labels Jan 7, 2020
@Yisaer
Copy link
Contributor Author

Yisaer commented Jan 8, 2020

As #1445 add e2e test for TidbMonitor, we could wait until #1445 are merged.

@Yisaer Yisaer removed the status/WIP Issue/PR is being worked on label Jan 9, 2020
@Yisaer Yisaer requested a review from aylei January 9, 2020 09:02
@aylei
Copy link
Contributor

aylei commented Jan 9, 2020

please add a release note

@aylei
Copy link
Contributor

aylei commented Jan 15, 2020

/run-all-test

@aylei
Copy link
Contributor

aylei commented Jan 15, 2020

/run-all-tests

@Yisaer Yisaer requested a review from cofyc January 15, 2020 05:18
@Yisaer
Copy link
Contributor Author

Yisaer commented Jan 15, 2020

@aylei @cofyc PTAL

pkg/monitor/monitor/monitor_manager.go Outdated Show resolved Hide resolved
return err
}
mm.recorder.Event(monitor, corev1.EventTypeNormal, SuccessSync, fmt.Sprintf("Sync TidbMonitor[%s/%s] Service Success", monitor.Namespace, monitor.Name))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest not recording successful sync to reduce verbosity, also, note that there is a global event rate-limit of tidb-controller-manager

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated, all successful sync wouldn't be recorded.

Copy link
Contributor

@cofyc cofyc Jan 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the good practice is to record a successful event once the operation is done successfully and skip if nothing needs to do because the actual state matches the desired state
in this case, the events of objects will be like:

  • optional warning events about failures
  • one successful event

Copy link
Contributor

@cofyc cofyc Jan 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this pattern can be applied to the most scenarios that the desired state is often only changed by the reconciliation controller, e.g. object specs managed by our controller
we can improve it in the future

pkg/monitor/monitor/monitor_manager.go Outdated Show resolved Hide resolved
pkg/monitor/monitor/monitor_manager.go Outdated Show resolved Hide resolved
@@ -182,3 +209,13 @@ func (mm *MonitorManager) syncTidbMonitorRbac(monitor *v1alpha1.TidbMonitor) (*c

return sa, nil
}

// If the namespace in ClusterRef is empty, we would set the TidbMonitor's namespace in the default
func setDefaultNamespaceForClusterRef(tm *v1alpha1.TidbMonitor) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do not mutate spec in controller, spec is user intent and by definition, is owned by user

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated.

Copy link
Contributor

@cofyc cofyc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, can we add some basic tests for functions in pkg/monitor/monitor? No tests yet right now.

@@ -729,3 +734,10 @@ func getMonitorPVC(monitor *v1alpha1.TidbMonitor) *core.PersistentVolumeClaim {
},
}
}

// If the namespace in ClusterRef is empty, we would set the TidbMonitor's namespace in the default
func setDefaultNamespaceForClusterRef(model *MonitorConfigModel, tm *v1alpha1.TidbMonitor) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we don't need to create a new function for this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated.

@Yisaer
Copy link
Contributor Author

Yisaer commented Jan 15, 2020

By the way, can we add some basic tests for functions in pkg/monitor/monitor? No tests yet right now.

Sure, and issued in #1554.

@Yisaer Yisaer requested review from cofyc and aylei January 15, 2020 06:33
Copy link
Contributor

@aylei aylei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@aylei
Copy link
Contributor

aylei commented Jan 15, 2020

BTW, it is recommended to separate PR to logic complete unit, so that each feature/fix could be independently rollbacked or cherry-picked, also, the release note will be easier to write.

@Yisaer
Copy link
Contributor Author

Yisaer commented Jan 15, 2020

/merge

@sre-bot
Copy link
Contributor

sre-bot commented Jan 15, 2020

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented Jan 15, 2020

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented Jan 15, 2020

@Yisaer merge failed.

@Yisaer
Copy link
Contributor Author

Yisaer commented Jan 15, 2020

/run-e2e-in-kind

@Yisaer Yisaer merged commit d168502 into pingcap:master Jan 15, 2020
@sre-bot
Copy link
Contributor

sre-bot commented Jan 15, 2020

cherry pick to release-1.1 in PR #1557

cofyc pushed a commit that referenced this pull request Jan 15, 2020
* fix log bug

* add default namespace for clusterRef

* fix error log which don't requeue

* add event recorder

* Update pkg/monitor/monitor/monitor_manager.go

Co-Authored-By: Aylei <[email protected]>

* fix by conment

* Update monitor_manager.go

* Update util.go

Co-authored-by: Song Gao <[email protected]>
Co-authored-by: Aylei <[email protected]>
@Yisaer Yisaer deleted the fix_tidbmonitor_bug branch February 26, 2020 04:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants