charts: backup: flexible credentials #1248
Conversation
allow retrieving credentials from the environment on GCP, stream the backup upload
|
We have the backup controller which makes it much easier to do with backup and restore. Better to add this feature to the controller. /cc @onlymellb @DanielZhangQD |
|
yes, the backup controller has all these same issues. If we are going to have the chart in this repo, this PR should be accepted. Otherwise we should delete the chart from the repo. |
|
I suggest though that we may want to keep a chart for mydumper if we move the controller to the new br. |
|
We aren't going to remove mydumper backup from the backup controller even if we have integrated the new BR. |
There was a problem hiding this comment.
I suggest though that we may want to keep a chart for mydumper if we move the controller to the new br.
The backup controller does not use this chart.
It seems that no need to keep this chart after the backup controller introduced? @onlymellb
| @@ -94,6 +106,7 @@ s3: {} | |||
| # region: "" | |||
| # bucket: "" | |||
| # secretName is the name of the secret which stores s3 object store access key and secret key | |||
| # This is not necessary on AWS. Instead you should be able to get the credentials from the EKS service IAM role. | |||
There was a problem hiding this comment.
Then how to use it in the job container?
There was a problem hiding this comment.
The AWS SDK will find the credentials, it just needs to run with a service account. You have to be using EKS or properly running something like iam-authenticator in the k8s cluster.
There was a problem hiding this comment.
We just need to attach the service account, no need to install the SDK or call any command to run with it?
There was a problem hiding this comment.
Does iam-authenticator need to be packaged in the Docker image?
There was a problem hiding this comment.
yes, these changes are tested as working. Just set the service account.
On GCP the association with the k8s service account to the cloud service account is done with
kubectl -n operations annotate serviceaccount ${svcacct} iam.gke.io/gcp-service-account=${svcacct}@${PROJECT}.iam.gserviceaccount.com
On EKS I believe one just needs to attach a policy. If not using EKS you run an authenticator program for the entire cluster. https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-technical-overview.html
In order to be compatible with the old version of tidb-operator, I think it can't be deleted in a short time. |
|
/run-e2e-test |
|
/run-e2e-test |
|
This PR was also tested on AWS now. |
allow retrieving credentials from metadata on AWS and GCP
on GCP, stream the backup upload
What problem does this PR solve?
The current backup chart does not allow the recommended approach to getting credentials on AWS and GCP
What is changed and how does it work?
Don't require a k8s secret for credentials
Check List
Code changes
Side effects
Related changes
Does this PR introduce a user-facing change?: