KubeCluster hangs if it fails to start dask-scheduler

[svetlin-mladenov]

What happened:
KubeCluster creation just hangs and waits indefinitely.

What you expected to happen:
An error to be reported

Minimal Complete Verifiable Example:

from dask_kubernetes import KubeCluster, make_pod_spec

pod_spec = make_pod_spec(image='ubuntu:latest')
with KubeCluster(pod_spec) as cluster:
    cluster.scale(5)

Anything else we need to know?:
I was trying to start a KubeCluster with an image that was broken and was missing the required dependencies. KubeCluster says Creating scheduler pod on cluster. This may take some time. and just hangs there. Initially I thought it was just pulling the image. After 30 mins I inspected the pod and discovered the issue. The current behavior is just bad user experience. I think an error message should be reported in this case which is going to ease debugging such cases especially for new users.

Environment:

• Dask version: 2022.01.0
• Python version: 3.9
• Operating System: Linux
• Install method (conda, pip, source): conda

[jacobtomlinson]

Thanks for raising this @svetlin-mladenov. We are currently putting all our effort into #392 which will ultimately replace KubeCluster, so it is unlikely we will dig into bugs here for now.

If someone else has time to pick this up that would be great.

[jacobtomlinson]

The classic KubeCluster was removed in #890. All users will need to migrate to the Dask Operator. Closing.