[Error (docker)]: response from daemon: Unknown runtime specified nvidia AND could not select device driver "" with capabilities: [[gpu]].

[Luxcium]

Docker Error

I am unable to troubleshoot this issue can you let me know what information could be helpful to help me ???

docker: Error response from daemon: Unknown runtime specified nvidia.

❯ REPO=ghcr.io/rapidsai/node
VERSIONS="21.12.00-runtime-node16.10.0-cudagl11.4.2-ubuntu20.04"

# Be sure to pass either the `--runtime=nvidia` or `--gpus` flag!
docker run --rm \
    --runtime=nvidia \
    -e "DISPLAY=$DISPLAY" \
    -v "/etc/fonts:/etc/fonts:ro" \
    -v "/tmp/.X11-unix:/tmp/.X11-unix:rw" \
    -v "/usr/share/fonts:/usr/share/fonts:ro" \
    -v "/usr/share/icons:/usr/share/icons:ro" \
    $REPO:$VERSIONS-demo-amd64 \
    npx @rapidsai/demo-graph
docker: Error response from daemon: Unknown runtime specified nvidia.
See 'docker run --help'.
❯ echo $DISPLAY
:0

docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

❯ REPO=ghcr.io/rapidsai/node
VERSIONS="21.12.00-runtime-node16.10.0-cuda11.4.2-ubuntu20.04"

# Be sure to pass either the `--runtime=nvidia` or `--gpus` flag!
docker run --rm --gpus=0 $REPO:$VERSIONS-cudf-amd64 \
    -p "const {Series, DataFrame} = require('@rapidsai/cudf');\
        new DataFrame({ a: Series.new([0, 1, 2]) }).toString()"
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

[AjayThorve]

Hey @Luxcium do you have nvidia-docker2 installed on your system?

Might be related to that!

If you do, may be this discussion might help

[Luxcium]

I think Fedora team hates people using NVIDIA or NVIDIA team hates people using Fedora

[Luxcium]

Hey @Luxcium do you have nvidia-docker2 installed on your system?

Might be related to that!

If you do, may be this discussion might help

Thanks @AjayThorve
do you know if I can get it except from https://rpms.if-not-true-then-false.com/inttf.repo (link to the blog post)

I use Fedora release 34 (Thirty Four) as shown in the hidden post above ...

[Luxcium]

I am doing it then...

[Luxcium]

Using nvidia-docker2

I have a new error message now

nvidia-container-cli: container error: cgroup subsystem devices not found: unknown

❯ REPO=ghcr.io/rapidsai/node
VERSIONS="21.12.00-runtime-node16.10.0-cudagl11.4.2-ubuntu20.04"

docker run --rm --runtime=nvidia -e "DISPLAY=$DISPLAY" -v "/etc/fonts:/etc/fonts:ro" \
              -v "/tmp/.X11-unix:/tmp/.X11-unix:rw" -v "/usr/share/fonts:/usr/share/fonts:ro" \ 
              -v "/usr/share/icons:/usr/share/icons:ro" $REPO:$VERSIONS-demo-amd64 npx @rapidsai/demo-graph
docker: Error response from daemon: 
OCI runtime create failed: 
container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: 
Running hook #1:: error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: 
container error: cgroup subsystem devices not found: unknown.

❯ REPO=ghcr.io/rapidsai/node
VERSIONS="21.12.00-runtime-node16.10.0-cuda11.4.2-ubuntu20.04"

docker run --rm --gpus=0 $REPO:$VERSIONS-cudf-amd64 -p \
        "const {Series, DataFrame} = require('@rapidsai/cudf');\
        new DataFrame({ a: Series.new([0, 1, 2]) }).toString()"
docker: Error response from daemon: OCI runtime create failed: container_linux.go:380: 
starting container process caused: process_linux.go:545: container init caused: 
Running hook #0:: error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: 
container error: cgroup subsystem devices not found: unknown.

[Luxcium]

after one hour of googling and trying to find a solution I must admit that I will wait to see if someone could help me here I was looking into the container error: cgroup subsystem devices not found: unknown but maybe I am starting to be blind to solution if you know the solution just let me know or please ask me more details about my system or configuration

[trxcllnt]

@Luxcium not entirely sure what you've tried, but generally the 3 things you will need (in addition to the driver) are:

• docker-ce
• nvidia-container-toolkit
• nvidia-container-runtime

I know it's possible to use GPUs in docker in RHEL, because we publish RHEL (Centos) images for the core RAPIDS libraries. Let me know if it still doesn't work after installing the above. I don't have a box with Centos right now, but I could put it on one of my spare machines to test if I need to.

[klueska]

Please see my comment here about the error of container error: cgroup subsystem devices not found: unknown regarding the lack of cgroupv2 support.

[trxcllnt]

@Luxcium does this work for you? NVIDIA/nvidia-docker#706 (comment)