[feat] health check for katib-controller

[anencore94]

What this PR does / why we need it:

• Need to check readiness/liveness when deploying katib-controller
  • The main things that we need to check is whether the webhook server is started well.

I've referenced the health check way from other components:

• kubeflow/pipelines's apiserver : https://github.com/kubeflow/pipelines/blob/master/manifests/kustomize/base/pipeline/ml-pipeline-apiserver-deployment.yaml#L84-L122
• mysql : https://github.com/kubeflow/katib/blob/master/manifests/v1beta1/components/mysql/mysql.yaml#L42-L67

So in my opinion, the liveness check with just ping is ok, but for readiness check, I'm not sure do we need more detailed way to check it or not.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #1930

Checklist:

• Docs included if any changes are user facing

[coveralls]

Coverage decreased (-0.1%) to 73.431% when pulling e7b328f on anencore94:feature/controller-healthz into 96ab64b on kubeflow:master.

[tenzen-y]

Thanks for your contribution! @anencore94
I left a few comments.

[tenzen-y]

Also, can you update this doc?

[anencore94]

Also, can you update this doc?

Thanks for checking carefully and sorry for the late reponse @tenzen-y :) I've updated it!

[tenzen-y]

@anencore94 Thanks for the updates!
/lgtm

[anencore94]

Could you re-run the failed action and re-review if you don't mind? @johnugeorge

[johnugeorge]

/approve

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: anencore94, johnugeorge

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [johnugeorge]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment