Fix Terraform error caused by unavailable metrics.k8s.io/v1beta1

[NimJay]

Background

• This fixed terraform apply errs on kubectl wait #1813.
• Even though terraform apply would successfully deploy a functional Online Boutique, the Terraform output had errors:

│ Error running command 'kubectl wait --for=condition=ready pods --all -n default --timeout=-1s': exit status 1. Output: + kubectl wait
│ --for=condition=ready pods --all -n default --timeout=-1s
│ E0605 13:26:36.342439    2173 memcache.go:287] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the
│ request

• kubectl wait ... pods ... is failing because the metrics.k8s.io/v1beta1 resource is not ready.
• See more logs in this comment.
• According to this StackOverflow post and my personal testing, this error happens on new clusters (just created) — during the first few minutes.
• In the first few minutes after cluster creation, running kubectl get apiservices outputs:

NAME                               SERVICE                      AVAILABLE                  AGE
...
v1beta1.metrics.k8s.io             kube-system/metrics-server   False (MissingEndpoints)   39s
...

• After a few minutes, the output is:

NAME                               SERVICE                      AVAILABLE   AGE
...
v1beta1.metrics.k8s.io             kube-system/metrics-server   True        83m
...

• A solution is to wait until the v1beta1.metrics.k8s.io resource is ready before kubectl wait ... pods ....

Change Summary

• Before kubectl wait-ing for Pods, the Terraform now kubectl waits for the v1beta1.metrics.k8s.io API service.
• We also add a timeout of 20 minutes (each) to both kubectl wait commands.

Testing Procedure

• I've already tested terraform apply with this change, and it worked!
• Also, once this is merged, we can check the DeployStack build run against the main branch (to see if terraform apply works):
  

Additional info

• Running kubectl get apiservice v1beta1.metrics.k8s.io -o yaml during the first few minutes after cluster creation outputs:

apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
...
  name: v1beta1.metrics.k8s.io
...
status:
  conditions:
  - lastTransitionTime: "2023-06-05T13:25:13Z"
    message: endpoints for service/metrics-server in "kube-system" have no addresses
      with port name ""
    reason: MissingEndpoints
    status: "False"
    type: Available

[NimJay]

The use of <<-EOT is documented here.

[NimJay]

The previously used --timeout of -1s refers to 1 week. Related docs here.

[bourgeoisor]

LGTM!