[Misc]: cannot run docker image 0.5.0 with cuda version == 12.1

[kratorado]

Anything you want to discuss about vllm.

I have ubuntu-22.04, cuda==12.1, nvidia driver == 530, gpu==V100

when running newer version of vllm (after 0.4.3) , it does not work. According to the doc, vllm is built with cuda 12.1, so I think it should not require cuda>=12.4.

Error log

docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy'
nvidia-container-cli: requirement error: unsatisfied condition: cuda>=12.4, please update your driver to a newer version, or use an earlier cuda container: unknown.

[mohit-zangoh]

+1

[kratorado]

found the related docker commands
https://hub.docker.com/layers/vllm/vllm-openai/v0.4.3/images/sha256-d55c38328bfda7b049dd6c3695d60fff8d88a63b19db058791879dc6bf3188af?context=explore

what new features require cuda>=12.4 ?

[youkaichao]

we use 12.4 because the old versions of base container will be deleted soon, according to nv policy https://gitlab.com/nvidia/container-images/cuda/-/blob/master/doc/support-policy.md .

[pseudotensor]

It seems even v0.4.3 is now built with cuda 12.4 required.

[kratorado]

we use 12.4 because the old versions of base container will be deleted soon, according to nv policy https://gitlab.com/nvidia/container-images/cuda/-/blob/master/doc/support-policy.md .

I see. Thanks.

[kratorado]

It seems even v0.4.3 is now built with cuda 12.4 required.

Yes. But the document didn't update with that. https://docs.vllm.ai/en/v0.4.3/getting_started/installation.html

[youkaichao]

hopefully this can be fixed when i fix #5646

[kratorado]

hopefully this can be fixed when i fix #5646

great!