Multiple issues related to the runpod backend

[peterschmidt85]

• Uses the hardcoded Docker image name (runpod/pytorch:2.1.1-py3.10-cuda12.1.1-devel-ubuntu22.04 instead of dstack's Docker image)
• The runpod backend always uses the Docker image's default entrypoint. In that case no configuration will work if the Docker image's default entrypoint isn't bash or sh.
• Uses the hardcoded Docker args (instead of the default Docker commands used in all other backend)
• Uses the 22 SSH port on the container (instead of 10022 as other backends do)
• Doesn't support registry_auth
• If a Docker image is large, the runpod bakend fails with f"Wait instance {instance_id} timeout" and proceed to trying another offer – without terminating the pod that is being created. Ths leads to creating multiple pods instead of failing the job.
• If there is an error when running the command, the runpod backend fails

dstack/src/dstack/_internal/core/backends/runpod/compute.py", line 95, in run_job
  for port in pod["runtime"]["ports"]:
TypeError: 'NoneType' object is not iterable

and proceed to trying another offer – without terminating the pod that is being created. Ths leads to creating multiple pods instead of failing the job.

[peterschmidt85]

@jvstme In theory, we could detect if the image has a non-default entrypoint automatically, skip the runpod backend in that case, and show a warning. Would that be easy to implement?

[jvstme]

@peterschmidt85, I think it shouldn't be difficult. See example of detecting the image entrypoint