GPUs: Check for existing NVIDIA container toolkit install

[spowelljr]

Fixes #17405

Before:

$ minikube start --gpus all
...

$ minikube stop
...

$ minikube start
😄  minikube v1.31.2 on Debian rodete
✨  Using the docker driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
🔄  Restarting existing docker container for "minikube" ...
❗  Using GPUs with the Docker driver is experimental, if you experience any issues please report them at: https://github.com/kubernetes/minikube/issues/new/choose
🛠️   Installing the NVIDIA Container Toolkit...

❌  Exiting due to RUNTIME_ENABLE: Failed to enable container runtime: failed installing the NVIDIA Container Toolkit: /bin/bash -c "curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg": Process exited with status 2
stdout:

stderr:
gpg: cannot open '/dev/tty': No such device or address
curl: (23) Failed writing body


╭───────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                           │
│    😿  If the above advice does not help, please let us know:                             │
│    👉  https://github.com/kubernetes/minikube/issues/new/choose                           │
│                                                                                           │
│    Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │
│                                                                                           │
╰───────────────────────────────────────────────────────────────────────────────────────────╯

After:

$ minikube start --gpus all
...

$ minikube stop
...

$ minikube start
😄  minikube v1.31.2 on Debian rodete
✨  Using the docker driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
🔄  Restarting existing docker container for "minikube" ...
❗  Using GPUs with the Docker driver is experimental, if you experience any issues please report them at: https://github.com/kubernetes/minikube/issues/new/choose
🐳  Preparing Kubernetes v1.28.3 on Docker 24.0.6 ...
🔗  Configuring bridge CNI (Container Networking Interface) ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
    ▪ Using image nvcr.io/nvidia/k8s-device-plugin:v0.14.1
🌟  Enabled addons: nvidia-device-plugin, storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

[medyagh]

/ok-to-test

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: medyagh, spowelljr

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [medyagh,spowelljr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[minikube-pr-bot]

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 17488) |
+----------------+----------+---------------------+
| minikube start | 50.2s    | 50.8s               |
| enable ingress | 27.3s    | 26.3s               |
+----------------+----------+---------------------+

Times for minikube start: 51.9s 50.2s 48.9s 49.2s 50.7s
Times for minikube (PR 17488) start: 50.8s 52.2s 50.1s 49.5s 51.4s

Times for minikube ingress: 27.2s 27.1s 27.6s 27.1s 27.7s
Times for minikube (PR 17488) ingress: 27.2s 27.1s 27.1s 25.6s 24.7s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 17488) |
+----------------+----------+---------------------+
| minikube start | 24.0s    | 22.5s               |
| enable ingress | 19.9s    | 20.7s               |
+----------------+----------+---------------------+

Times for minikube start: 21.6s 24.4s 24.6s 24.7s 24.4s
Times for minikube (PR 17488) start: 24.5s 23.8s 21.4s 20.8s 22.0s

Times for minikube ingress: 20.8s 20.8s 18.3s 17.8s 21.8s
Times for minikube (PR 17488) ingress: 18.8s 23.3s 18.8s 20.8s 21.8s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 17488) |
+----------------+----------+---------------------+
| minikube start | 20.3s    | 22.6s               |
| enable ingress | 26.3s    | 27.5s               |
+----------------+----------+---------------------+

Times for minikube start: 20.2s 20.5s 20.4s 20.4s 19.8s
Times for minikube (PR 17488) start: 23.6s 23.2s 23.7s 21.8s 20.7s

Times for minikube ingress: 19.3s 31.3s 31.3s 31.3s 18.3s
Times for minikube (PR 17488) ingress: 18.3s 19.3s 31.3s 47.3s 21.3s

[minikube-pr-bot]

These are the flake rates of all failed tests.

Environment	Failed Tests	Flake Rate (%)
Docker_Linux_containerd_arm64	TestScheduledStopUnix (gopogh)	0.00 (chart)
Docker_macOS	TestBinaryMirror (gopogh)	0.00 (chart)
Docker_macOS	TestDownloadOnlyKic (gopogh)	0.00 (chart)
Docker_macOS	TestFunctional/serial/MinikubeKubectlCmd (gopogh)	0.00 (chart)
Docker_macOS	TestFunctional/serial/MinikubeKubectlCmdDirectly (gopogh)	0.00 (chart)
Docker_Windows	TestFunctional/parallel/License (gopogh)	0.00 (chart)
Docker_Windows	TestPause/serial/SecondStartNoReconfiguration (gopogh)	0.00 (chart)
QEMU_macOS	TestErrorSpam/setup (gopogh)	0.00 (chart)
KVM_Linux	TestStoppedBinaryUpgrade/Upgrade (gopogh)	1.43 (chart)
Docker_Linux_crio	TestAddons/parallel/Headlamp (gopogh)	2.90 (chart)
Hyperkit_macOS	TestCertOptions (gopogh)	5.77 (chart)
Hyperkit_macOS	TestStartStop/group/newest-cni/serial/EnableAddonWhileActive (gopogh)	5.77 (chart)
Hyperkit_macOS	TestStartStop/group/newest-cni/serial/FirstStart (gopogh)	5.77 (chart)
Docker_macOS	TestDownloadOnly/v1.28.3/json-events (gopogh)	16.67 (chart)
Docker_macOS	TestDownloadOnly/v1.28.3/kubectl (gopogh)	16.67 (chart)
Docker_macOS	TestMultiNode/serial/DeployApp2Nodes (gopogh)	28.57 (chart)
Docker_macOS	TestMultiNode/serial/PingHostFrom2Pods (gopogh)	28.57 (chart)
Hyperkit_macOS	TestBinaryMirror (gopogh)	38.46 (chart)
Hyperkit_macOS	TestDownloadOnly/v1.28.3/json-events (gopogh)	39.22 (chart)
Hyperkit_macOS	TestDownloadOnly/v1.28.3/kubectl (gopogh)	39.22 (chart)
Hyperkit_macOS	TestFunctional/serial/MinikubeKubectlCmd (gopogh)	40.38 (chart)
Hyperkit_macOS	TestFunctional/serial/MinikubeKubectlCmdDirectly (gopogh)	40.38 (chart)
Hyperkit_macOS	TestMultiNode/serial/DeployApp2Nodes (gopogh)	44.23 (chart)
Hyperkit_macOS	TestMultiNode/serial/PingHostFrom2Pods (gopogh)	44.23 (chart)

To see the flake rates of all tests by environment, click here.