Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker step fails in object_detection benchmark #619

Closed
mahmoodn opened this issue Feb 1, 2023 · 2 comments
Closed

Docker step fails in object_detection benchmark #619

mahmoodn opened this issue Feb 1, 2023 · 2 comments

Comments

@mahmoodn
Copy link

mahmoodn commented Feb 1, 2023

Apparently, the object_detection fails at the docker step. Although I have CUDA-11.6 on the host machine, it says no CUDA found, too.

$ which nvcc
/usr/local/cuda-11.6/bin/nvcc


$ nvidia-docker build . -t mlperf/object_detection
Sending build context to Docker daemon  4.998MB
Step 1/9 : FROM pytorch/pytorch:1.10.0-cuda11.3-cudnn8-devel
1.10.0-cuda11.3-cudnn8-devel: Pulling from pytorch/pytorch
e4ca327ec0e7: Pull complete 
b99d76492afe: Pull complete 
b52f6fb756a5: Pull complete 
5a09baa528d6: Pull complete 
2df930949a05: Pull complete 
934eb401e46c: Pull complete 
3244eb9db036: Pull complete 
e2e27029eb8e: Pull complete 
bb65579dd223: Pull complete 
3f69be5194e3: Pull complete 
6d6c50c6f288: Pull complete 
8cc3f36aa546: Pull complete 
cd184e261431: Pull complete 
Digest: sha256:913e6689c5958b187e65561e528ec6c3ce8a02deedcdd38cb50c9cab301907bb
Status: Downloaded newer image for pytorch/pytorch:1.10.0-cuda11.3-cudnn8-devel
 ---> 46961cbf2ac7
Step 2/9 : RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
 ---> Running in 2f483c00f70d
Removing intermediate container 2f483c00f70d
 ---> 3aa86539e1ba
Step 3/9 : RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub
 ---> Running in ffa05f188c8d
Warning: apt-key output should not be parsed (stdout is not a terminal)
Executing: /tmp/apt-key-gpghome.upYdn7brFv/gpg.1.sh --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub
gpg: requesting key from 'https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub'
gpg: key A4B469963BF863CC: public key "cudatools <[email protected]>" imported
gpg: Total number processed: 1
gpg:               imported: 1
Removing intermediate container ffa05f188c8d
 ---> 2872f994af9a
Step 4/9 : RUN apt-get update -y  && apt-get install -y apt-utils                        libglib2.0-0=2.56.1-2ubuntu1                        libsm6=2:1.2.2-1                        libxext6=2:1.3.3-1                        libxrender-dev=1:0.9.10-1
 ---> Running in 99f5b00fe166
Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Get:3 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  InRelease [1581 B]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [83.3 kB]
Get:6 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1578 kB]
Get:7 http://archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [186 kB]
Get:8 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [3161 kB]
Get:9 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [11.3 MB]
Ign:10 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  InRelease
Get:11 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  Packages [1124 kB]
Get:12 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [22.9 kB]
Get:13 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [1389 kB]
Get:14 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  Release [564 B]
Get:15 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  Release.gpg [833 B]
Get:16 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  Packages [73.8 kB]
Get:17 http://archive.ubuntu.com/ubuntu bionic/restricted amd64 Packages [13.5 kB]
Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages [1344 kB]
Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [3581 kB]
Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [30.8 kB]
Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [2353 kB]
Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [1430 kB]
Get:23 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [64.0 kB]
Get:24 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [20.5 kB]
Fetched 28.2 MB in 3s (9635 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  libapt-inst2.0 libbsd0 libglib2.0-data libice6 libicu60
  libpthread-stubs0-dev libx11-6 libx11-data libx11-dev libx11-doc libxau-dev
  libxau6 libxcb1 libxcb1-dev libxdmcp-dev libxdmcp6 libxml2 libxrender1
  shared-mime-info x11-common x11proto-core-dev x11proto-dev xdg-user-dirs
  xorg-sgml-doctools xtrans-dev
Suggested packages:
  libxcb-doc
The following NEW packages will be installed:
  apt-utils libapt-inst2.0 libbsd0 libglib2.0-0 libglib2.0-data libice6
  libicu60 libpthread-stubs0-dev libsm6 libx11-6 libx11-data libx11-dev
  libx11-doc libxau-dev libxau6 libxcb1 libxcb1-dev libxdmcp-dev libxdmcp6
  libxext6 libxml2 libxrender-dev libxrender1 shared-mime-info x11-common
  x11proto-core-dev x11proto-dev xdg-user-dirs xorg-sgml-doctools xtrans-dev
0 upgraded, 30 newly installed, 0 to remove and 81 not upgraded.
Need to get 14.7 MB of archives.
After this operation, 62.0 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxau6 amd64 1:1.0.8-1ubuntu1 [7556 B]
Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libbsd0 amd64 0.8.7-1ubuntu0.1 [41.6 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 libxdmcp6 amd64 1:1.1.2-3 [10.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxcb1 amd64 1.13-2~ubuntu18.04 [45.5 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libx11-data all 2:1.6.4-3ubuntu0.4 [114 kB]
Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libx11-6 amd64 2:1.6.4-3ubuntu0.4 [572 kB]
Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 libxext6 amd64 2:1.3.3-1 [29.4 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 x11-common all 1:7.7+19ubuntu7.1 [22.5 kB]
Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libice6 amd64 2:1.0.9-2ubuntu0.18.04.1 [40.7 kB]
Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsm6 amd64 2:1.2.2-1 [15.8 kB]
Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libapt-inst2.0 amd64 1.6.14 [57.3 kB]
Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 apt-utils amd64 1.6.14 [209 kB]
Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 libglib2.0-0 amd64 2.56.1-2ubuntu1 [1167 kB]
Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libglib2.0-data all 2.56.4-0ubuntu0.18.04.9 [4728 B]
Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libicu60 amd64 60.2-3ubuntu3.2 [8050 kB]
Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxml2 amd64 2.9.4+dfsg1-6.1ubuntu1.8 [664 kB]
Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 shared-mime-info amd64 1.9-2 [426 kB]
Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 xdg-user-dirs amd64 0.17-1ubuntu1 [48.0 kB]
Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 libpthread-stubs0-dev amd64 0.3-4 [4068 B]
Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 xorg-sgml-doctools all 1:1.11-1 [12.9 kB]
Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 x11proto-dev all 2018.4-4 [251 kB]
Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 x11proto-core-dev all 2018.4-4 [2620 B]
Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxau-dev amd64 1:1.0.8-1ubuntu1 [9476 B]
Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 libxdmcp-dev amd64 1:1.1.2-3 [25.1 kB]
Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 xtrans-dev all 1.3.5-1 [70.5 kB]
Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxcb1-dev amd64 1.13-2~ubuntu18.04 [80.0 kB]
Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libx11-dev amd64 2:1.6.4-3ubuntu0.4 [641 kB]
Get:28 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libx11-doc all 2:1.6.4-3ubuntu0.4 [2065 kB]
Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 libxrender1 amd64 1:0.9.10-1 [18.7 kB]
Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 libxrender-dev amd64 1:0.9.10-1 [24.9 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 14.7 MB in 1s (11.0 MB/s)
Selecting previously unselected package libxau6:amd64.
(Reading database ... 12598 files and directories currently installed.)
Preparing to unpack .../00-libxau6_1%3a1.0.8-1ubuntu1_amd64.deb ...
Unpacking libxau6:amd64 (1:1.0.8-1ubuntu1) ...
Selecting previously unselected package libbsd0:amd64.
Preparing to unpack .../01-libbsd0_0.8.7-1ubuntu0.1_amd64.deb ...
Unpacking libbsd0:amd64 (0.8.7-1ubuntu0.1) ...
Selecting previously unselected package libxdmcp6:amd64.
Preparing to unpack .../02-libxdmcp6_1%3a1.1.2-3_amd64.deb ...
Unpacking libxdmcp6:amd64 (1:1.1.2-3) ...
Selecting previously unselected package libxcb1:amd64.
Preparing to unpack .../03-libxcb1_1.13-2~ubuntu18.04_amd64.deb ...
Unpacking libxcb1:amd64 (1.13-2~ubuntu18.04) ...
Selecting previously unselected package libx11-data.
Preparing to unpack .../04-libx11-data_2%3a1.6.4-3ubuntu0.4_all.deb ...
Unpacking libx11-data (2:1.6.4-3ubuntu0.4) ...
Selecting previously unselected package libx11-6:amd64.
Preparing to unpack .../05-libx11-6_2%3a1.6.4-3ubuntu0.4_amd64.deb ...
Unpacking libx11-6:amd64 (2:1.6.4-3ubuntu0.4) ...
Selecting previously unselected package libxext6:amd64.
Preparing to unpack .../06-libxext6_2%3a1.3.3-1_amd64.deb ...
Unpacking libxext6:amd64 (2:1.3.3-1) ...
Selecting previously unselected package x11-common.
Preparing to unpack .../07-x11-common_1%3a7.7+19ubuntu7.1_all.deb ...
dpkg-query: no packages found matching nux-tools
Unpacking x11-common (1:7.7+19ubuntu7.1) ...
Selecting previously unselected package libice6:amd64.
Preparing to unpack .../08-libice6_2%3a1.0.9-2ubuntu0.18.04.1_amd64.deb ...
Unpacking libice6:amd64 (2:1.0.9-2ubuntu0.18.04.1) ...
Selecting previously unselected package libsm6:amd64.
Preparing to unpack .../09-libsm6_2%3a1.2.2-1_amd64.deb ...
Unpacking libsm6:amd64 (2:1.2.2-1) ...
Selecting previously unselected package libapt-inst2.0:amd64.
Preparing to unpack .../10-libapt-inst2.0_1.6.14_amd64.deb ...
Unpacking libapt-inst2.0:amd64 (1.6.14) ...
Selecting previously unselected package apt-utils.
Preparing to unpack .../11-apt-utils_1.6.14_amd64.deb ...
Unpacking apt-utils (1.6.14) ...
Selecting previously unselected package libglib2.0-0:amd64.
Preparing to unpack .../12-libglib2.0-0_2.56.1-2ubuntu1_amd64.deb ...
Unpacking libglib2.0-0:amd64 (2.56.1-2ubuntu1) ...
Selecting previously unselected package libglib2.0-data.
Preparing to unpack .../13-libglib2.0-data_2.56.4-0ubuntu0.18.04.9_all.deb ...
Unpacking libglib2.0-data (2.56.4-0ubuntu0.18.04.9) ...
Selecting previously unselected package libicu60:amd64.
Preparing to unpack .../14-libicu60_60.2-3ubuntu3.2_amd64.deb ...
Unpacking libicu60:amd64 (60.2-3ubuntu3.2) ...
Selecting previously unselected package libxml2:amd64.
Preparing to unpack .../15-libxml2_2.9.4+dfsg1-6.1ubuntu1.8_amd64.deb ...
Unpacking libxml2:amd64 (2.9.4+dfsg1-6.1ubuntu1.8) ...
Selecting previously unselected package shared-mime-info.
Preparing to unpack .../16-shared-mime-info_1.9-2_amd64.deb ...
Unpacking shared-mime-info (1.9-2) ...
Selecting previously unselected package xdg-user-dirs.
Preparing to unpack .../17-xdg-user-dirs_0.17-1ubuntu1_amd64.deb ...
Unpacking xdg-user-dirs (0.17-1ubuntu1) ...
Selecting previously unselected package libpthread-stubs0-dev:amd64.
Preparing to unpack .../18-libpthread-stubs0-dev_0.3-4_amd64.deb ...
Unpacking libpthread-stubs0-dev:amd64 (0.3-4) ...
Selecting previously unselected package xorg-sgml-doctools.
Preparing to unpack .../19-xorg-sgml-doctools_1%3a1.11-1_all.deb ...
Unpacking xorg-sgml-doctools (1:1.11-1) ...
Selecting previously unselected package x11proto-dev.
Preparing to unpack .../20-x11proto-dev_2018.4-4_all.deb ...
Unpacking x11proto-dev (2018.4-4) ...
Selecting previously unselected package x11proto-core-dev.
Preparing to unpack .../21-x11proto-core-dev_2018.4-4_all.deb ...
Unpacking x11proto-core-dev (2018.4-4) ...
Selecting previously unselected package libxau-dev:amd64.
Preparing to unpack .../22-libxau-dev_1%3a1.0.8-1ubuntu1_amd64.deb ...
Unpacking libxau-dev:amd64 (1:1.0.8-1ubuntu1) ...
Selecting previously unselected package libxdmcp-dev:amd64.
Preparing to unpack .../23-libxdmcp-dev_1%3a1.1.2-3_amd64.deb ...
Unpacking libxdmcp-dev:amd64 (1:1.1.2-3) ...
Selecting previously unselected package xtrans-dev.
Preparing to unpack .../24-xtrans-dev_1.3.5-1_all.deb ...
Unpacking xtrans-dev (1.3.5-1) ...
Selecting previously unselected package libxcb1-dev:amd64.
Preparing to unpack .../25-libxcb1-dev_1.13-2~ubuntu18.04_amd64.deb ...
Unpacking libxcb1-dev:amd64 (1.13-2~ubuntu18.04) ...
Selecting previously unselected package libx11-dev:amd64.
Preparing to unpack .../26-libx11-dev_2%3a1.6.4-3ubuntu0.4_amd64.deb ...
Unpacking libx11-dev:amd64 (2:1.6.4-3ubuntu0.4) ...
Selecting previously unselected package libx11-doc.
Preparing to unpack .../27-libx11-doc_2%3a1.6.4-3ubuntu0.4_all.deb ...
Unpacking libx11-doc (2:1.6.4-3ubuntu0.4) ...
Selecting previously unselected package libxrender1:amd64.
Preparing to unpack .../28-libxrender1_1%3a0.9.10-1_amd64.deb ...
Unpacking libxrender1:amd64 (1:0.9.10-1) ...
Selecting previously unselected package libxrender-dev:amd64.
Preparing to unpack .../29-libxrender-dev_1%3a0.9.10-1_amd64.deb ...
Unpacking libxrender-dev:amd64 (1:0.9.10-1) ...
Setting up libapt-inst2.0:amd64 (1.6.14) ...
Setting up libicu60:amd64 (60.2-3ubuntu3.2) ...
Setting up libpthread-stubs0-dev:amd64 (0.3-4) ...
Setting up apt-utils (1.6.14) ...
Setting up libglib2.0-0:amd64 (2.56.1-2ubuntu1) ...
No schema files found: doing nothing.
Setting up xorg-sgml-doctools (1:1.11-1) ...
Setting up libbsd0:amd64 (0.8.7-1ubuntu0.1) ...
Setting up libxml2:amd64 (2.9.4+dfsg1-6.1ubuntu1.8) ...
Setting up x11proto-dev (2018.4-4) ...
Setting up xtrans-dev (1.3.5-1) ...
Setting up libglib2.0-data (2.56.4-0ubuntu0.18.04.9) ...
Setting up libx11-doc (2:1.6.4-3ubuntu0.4) ...
Setting up shared-mime-info (1.9-2) ...
Setting up libxdmcp6:amd64 (1:1.1.2-3) ...
Setting up x11-common (1:7.7+19ubuntu7.1) ...
update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Setting up xdg-user-dirs (0.17-1ubuntu1) ...
Setting up libx11-data (2:1.6.4-3ubuntu0.4) ...
Setting up libxau6:amd64 (1:1.0.8-1ubuntu1) ...
Setting up x11proto-core-dev (2018.4-4) ...
Setting up libxau-dev:amd64 (1:1.0.8-1ubuntu1) ...
Setting up libxdmcp-dev:amd64 (1:1.1.2-3) ...
Setting up libice6:amd64 (2:1.0.9-2ubuntu0.18.04.1) ...
Setting up libxcb1:amd64 (1.13-2~ubuntu18.04) ...
Setting up libsm6:amd64 (2:1.2.2-1) ...
Setting up libx11-6:amd64 (2:1.6.4-3ubuntu0.4) ...
Setting up libxrender1:amd64 (1:0.9.10-1) ...
Setting up libxcb1-dev:amd64 (1.13-2~ubuntu18.04) ...
Setting up libx11-dev:amd64 (2:1.6.4-3ubuntu0.4) ...
Setting up libxext6:amd64 (2:1.3.3-1) ...
Setting up libxrender-dev:amd64 (1:0.9.10-1) ...
Processing triggers for libc-bin (2.27-3ubuntu1.4) ...
Removing intermediate container 99f5b00fe166
 ---> 5831626c56af
Step 5/9 : RUN pip install ninja==1.8.2.post2                 yacs==0.1.5                 cython==0.29.5                 matplotlib==3.0.2                 opencv-python==4.0.0.21                 mlperf_compliance==0.0.10                 torchvision==0.2.2                 pycocotools==2.0.2
 ---> Running in 5d09184533cd
Collecting ninja==1.8.2.post2
  Downloading ninja-1.8.2.post2-cp37-cp37m-manylinux1_x86_64.whl (98 kB)
Collecting yacs==0.1.5
  Downloading yacs-0.1.5-py3-none-any.whl (9.2 kB)
Collecting cython==0.29.5
  Downloading Cython-0.29.5-cp37-cp37m-manylinux1_x86_64.whl (2.1 MB)
Collecting matplotlib==3.0.2
  Downloading matplotlib-3.0.2-cp37-cp37m-manylinux1_x86_64.whl (12.9 MB)
Collecting opencv-python==4.0.0.21
  Downloading opencv_python-4.0.0.21-cp37-cp37m-manylinux1_x86_64.whl (25.4 MB)
Collecting mlperf_compliance==0.0.10
  Downloading mlperf_compliance-0.0.10-py3-none-any.whl (24 kB)
Collecting torchvision==0.2.2
  Downloading torchvision-0.2.2-py2.py3-none-any.whl (64 kB)
Collecting pycocotools==2.0.2
  Downloading pycocotools-2.0.2.tar.gz (23 kB)
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1
  Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
Collecting kiwisolver>=1.0.1
  Downloading kiwisolver-1.4.4-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.1 MB)
Requirement already satisfied: numpy>=1.10.0 in /opt/conda/lib/python3.7/site-packages (from matplotlib==3.0.2) (1.21.2)
Collecting cycler>=0.10
  Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting python-dateutil>=2.1
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Requirement already satisfied: setuptools>=18.0 in /opt/conda/lib/python3.7/site-packages (from pycocotools==2.0.2) (58.0.4)
Collecting tqdm==4.19.9
  Downloading tqdm-4.19.9-py2.py3-none-any.whl (52 kB)
Requirement already satisfied: torch in /opt/conda/lib/python3.7/site-packages (from torchvision==0.2.2) (1.10.0)
Requirement already satisfied: pillow>=4.1.1 in /opt/conda/lib/python3.7/site-packages (from torchvision==0.2.2) (8.4.0)
Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from torchvision==0.2.2) (1.16.0)
Requirement already satisfied: PyYAML in /opt/conda/lib/python3.7/site-packages (from yacs==0.1.5) (5.4.1)
Requirement already satisfied: typing-extensions in /opt/conda/lib/python3.7/site-packages (from kiwisolver>=1.0.1->matplotlib==3.0.2) (3.10.0.2)
WARNING: The candidate selected for download or install is a yanked version: 'torchvision' candidate (version 0.2.2 at https://files.pythonhosted.org/packages/ce/a1/66d72a2fe580a9f0fcbaaa5b976911fbbde9dce9b330ba12791997b856e9/torchvision-0.2.2-py2.py3-none-any.whl#sha256=686cda0037bc904a35593a1394eaf7337edd141b78dc2eadc4667abb6fd569b5 (from https://pypi.org/simple/torchvision/))
Reason for being yanked: So that users won't accidentally install this when using python 3.11
Building wheels for collected packages: pycocotools
  Building wheel for pycocotools (setup.py): started
  Building wheel for pycocotools (setup.py): finished with status 'done'
  Created wheel for pycocotools: filename=pycocotools-2.0.2-cp37-cp37m-linux_x86_64.whl size=273198 sha256=f3e2dbaf8ec85e21e1aa9844af2add7ca0866a0d03bfbd316a50328ed8940523
  Stored in directory: /root/.cache/pip/wheels/bc/cf/1b/e95c99c5f9d1648be3f500ca55e7ce55f24818b0f48336adaf
Successfully built pycocotools
Installing collected packages: python-dateutil, pyparsing, kiwisolver, cycler, tqdm, matplotlib, cython, yacs, torchvision, pycocotools, opencv-python, ninja, mlperf-compliance
  Attempting uninstall: tqdm
    Found existing installation: tqdm 4.61.2
    Uninstalling tqdm-4.61.2:
      Successfully uninstalled tqdm-4.61.2
  Attempting uninstall: torchvision
    Found existing installation: torchvision 0.11.0
    Uninstalling torchvision-0.11.0:
      Successfully uninstalled torchvision-0.11.0
Successfully installed cycler-0.11.0 cython-0.29.5 kiwisolver-1.4.4 matplotlib-3.0.2 mlperf-compliance-0.0.10 ninja-1.8.2.post2 opencv-python-4.0.0.21 pycocotools-2.0.2 pyparsing-3.0.9 python-dateutil-2.8.2 torchvision-0.2.2 tqdm-4.19.9 yacs-0.1.5
Removing intermediate container 5d09184533cd
 ---> 9caab9e9d18b
Step 6/9 : RUN pip install --no-cache-dir https://github.com/mlperf/logging/archive/9ea0afa.zip
 ---> Running in f682451add64
Collecting https://github.com/mlperf/logging/archive/9ea0afa.zip
  Downloading https://github.com/mlperf/logging/archive/9ea0afa.zip
Building wheels for collected packages: mlperf-logging
  Building wheel for mlperf-logging (setup.py): started
  Building wheel for mlperf-logging (setup.py): finished with status 'done'
  Created wheel for mlperf-logging: filename=mlperf_logging-0.0.0-py3-none-any.whl size=50376 sha256=6cb17664258d4e18301a40677dc1aa3d45a9afb1b362c609f8c67f266057e8be
  Stored in directory: /tmp/pip-ephem-wheel-cache-mr_4xux8/wheels/dd/1c/8d/f4516d946ad6f0b7e761d90d62443e1a0e6615f96187bbdf71
Successfully built mlperf-logging
Installing collected packages: mlperf-logging
Successfully installed mlperf-logging-0.0.0
Removing intermediate container f682451add64
 ---> baffa176e484
Step 7/9 : WORKDIR /workspace/object_detection
 ---> Running in 3e60f9f29b5f
Removing intermediate container 3e60f9f29b5f
 ---> b0bc4735937e
Step 8/9 : COPY . .
 ---> e382af16df90
Step 9/9 : RUN cd pytorch  && rm -rf build/  && python setup.py clean build develop --user
 ---> Running in f448d2cccd4f
No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
running clean
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark
copying maskrcnn_benchmark/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/utils
copying maskrcnn_benchmark/utils/imports.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/utils
copying maskrcnn_benchmark/utils/env.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/utils
copying maskrcnn_benchmark/utils/metric_logger.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/utils
copying maskrcnn_benchmark/utils/cv2_util.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/utils
copying maskrcnn_benchmark/utils/comm.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/utils
copying maskrcnn_benchmark/utils/miscellaneous.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/utils
copying maskrcnn_benchmark/utils/collect_env.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/utils
copying maskrcnn_benchmark/utils/registry.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/utils
copying maskrcnn_benchmark/utils/logger.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/utils
copying maskrcnn_benchmark/utils/c2_model_loading.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/utils
copying maskrcnn_benchmark/utils/model_serialization.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/utils
copying maskrcnn_benchmark/utils/mlperf_logger.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/utils
copying maskrcnn_benchmark/utils/model_zoo.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/utils
copying maskrcnn_benchmark/utils/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/utils
copying maskrcnn_benchmark/utils/checkpoint.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/utils
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/structures
copying maskrcnn_benchmark/structures/segmentation_mask.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/structures
copying maskrcnn_benchmark/structures/image_list.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/structures
copying maskrcnn_benchmark/structures/boxlist_ops.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/structures
copying maskrcnn_benchmark/structures/bounding_box.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/structures
copying maskrcnn_benchmark/structures/keypoint.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/structures
copying maskrcnn_benchmark/structures/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/structures
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling
copying maskrcnn_benchmark/modeling/utils.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling
copying maskrcnn_benchmark/modeling/make_layers.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling
copying maskrcnn_benchmark/modeling/box_coder.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling
copying maskrcnn_benchmark/modeling/matcher.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling
copying maskrcnn_benchmark/modeling/registry.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling
copying maskrcnn_benchmark/modeling/balanced_positive_negative_sampler.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling
copying maskrcnn_benchmark/modeling/poolers.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling
copying maskrcnn_benchmark/modeling/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data
copying maskrcnn_benchmark/data/build.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data
copying maskrcnn_benchmark/data/collate_batch.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data
copying maskrcnn_benchmark/data/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/config
copying maskrcnn_benchmark/config/defaults.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/config
copying maskrcnn_benchmark/config/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/config
copying maskrcnn_benchmark/config/paths_catalog.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/config
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/engine
copying maskrcnn_benchmark/engine/trainer.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/engine
copying maskrcnn_benchmark/engine/inference.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/engine
copying maskrcnn_benchmark/engine/tester.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/engine
copying maskrcnn_benchmark/engine/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/engine
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/solver
copying maskrcnn_benchmark/solver/lr_scheduler.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/solver
copying maskrcnn_benchmark/solver/build.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/solver
copying maskrcnn_benchmark/solver/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/solver
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/layers
copying maskrcnn_benchmark/layers/roi_pool.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/layers
copying maskrcnn_benchmark/layers/misc.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/layers
copying maskrcnn_benchmark/layers/sigmoid_focal_loss.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/layers
copying maskrcnn_benchmark/layers/nms.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/layers
copying maskrcnn_benchmark/layers/smooth_l1_loss.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/layers
copying maskrcnn_benchmark/layers/_utils.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/layers
copying maskrcnn_benchmark/layers/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/layers
copying maskrcnn_benchmark/layers/roi_align.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/layers
copying maskrcnn_benchmark/layers/batch_norm.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/layers
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/detector
copying maskrcnn_benchmark/modeling/detector/detectors.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/detector
copying maskrcnn_benchmark/modeling/detector/generalized_rcnn.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/detector
copying maskrcnn_benchmark/modeling/detector/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/detector
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads
copying maskrcnn_benchmark/modeling/roi_heads/roi_heads.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads
copying maskrcnn_benchmark/modeling/roi_heads/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/backbone
copying maskrcnn_benchmark/modeling/backbone/resnet.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/backbone
copying maskrcnn_benchmark/modeling/backbone/backbone.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/backbone
copying maskrcnn_benchmark/modeling/backbone/fpn.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/backbone
copying maskrcnn_benchmark/modeling/backbone/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/backbone
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/rpn
copying maskrcnn_benchmark/modeling/rpn/utils.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/rpn
copying maskrcnn_benchmark/modeling/rpn/loss.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/rpn
copying maskrcnn_benchmark/modeling/rpn/inference.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/rpn
copying maskrcnn_benchmark/modeling/rpn/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/rpn
copying maskrcnn_benchmark/modeling/rpn/anchor_generator.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/rpn
copying maskrcnn_benchmark/modeling/rpn/rpn.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/rpn
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/keypoint_head
copying maskrcnn_benchmark/modeling/roi_heads/keypoint_head/loss.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/keypoint_head
copying maskrcnn_benchmark/modeling/roi_heads/keypoint_head/inference.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/keypoint_head
copying maskrcnn_benchmark/modeling/roi_heads/keypoint_head/keypoint_head.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/keypoint_head
copying maskrcnn_benchmark/modeling/roi_heads/keypoint_head/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/keypoint_head
copying maskrcnn_benchmark/modeling/roi_heads/keypoint_head/roi_keypoint_predictors.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/keypoint_head
copying maskrcnn_benchmark/modeling/roi_heads/keypoint_head/roi_keypoint_feature_extractors.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/keypoint_head
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/mask_head
copying maskrcnn_benchmark/modeling/roi_heads/mask_head/roi_mask_predictors.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/mask_head
copying maskrcnn_benchmark/modeling/roi_heads/mask_head/loss.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/mask_head
copying maskrcnn_benchmark/modeling/roi_heads/mask_head/inference.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/mask_head
copying maskrcnn_benchmark/modeling/roi_heads/mask_head/mask_head.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/mask_head
copying maskrcnn_benchmark/modeling/roi_heads/mask_head/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/mask_head
copying maskrcnn_benchmark/modeling/roi_heads/mask_head/roi_mask_feature_extractors.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/mask_head
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/box_head
copying maskrcnn_benchmark/modeling/roi_heads/box_head/loss.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/box_head
copying maskrcnn_benchmark/modeling/roi_heads/box_head/inference.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/box_head
copying maskrcnn_benchmark/modeling/roi_heads/box_head/roi_box_feature_extractors.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/box_head
copying maskrcnn_benchmark/modeling/roi_heads/box_head/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/box_head
copying maskrcnn_benchmark/modeling/roi_heads/box_head/box_head.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/box_head
copying maskrcnn_benchmark/modeling/roi_heads/box_head/roi_box_predictors.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/roi_heads/box_head
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/rpn/retinanet
copying maskrcnn_benchmark/modeling/rpn/retinanet/retinanet.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/rpn/retinanet
copying maskrcnn_benchmark/modeling/rpn/retinanet/loss.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/rpn/retinanet
copying maskrcnn_benchmark/modeling/rpn/retinanet/inference.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/rpn/retinanet
copying maskrcnn_benchmark/modeling/rpn/retinanet/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/modeling/rpn/retinanet
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/samplers
copying maskrcnn_benchmark/data/samplers/grouped_batch_sampler.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/samplers
copying maskrcnn_benchmark/data/samplers/iteration_based_batch_sampler.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/samplers
copying maskrcnn_benchmark/data/samplers/distributed.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/samplers
copying maskrcnn_benchmark/data/samplers/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/samplers
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/transforms
copying maskrcnn_benchmark/data/transforms/build.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/transforms
copying maskrcnn_benchmark/data/transforms/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/transforms
copying maskrcnn_benchmark/data/transforms/transforms.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/transforms
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/datasets
copying maskrcnn_benchmark/data/datasets/voc.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/datasets
copying maskrcnn_benchmark/data/datasets/coco.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/datasets
copying maskrcnn_benchmark/data/datasets/list_dataset.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/datasets
copying maskrcnn_benchmark/data/datasets/concat_dataset.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/datasets
copying maskrcnn_benchmark/data/datasets/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/datasets
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/datasets/evaluation
copying maskrcnn_benchmark/data/datasets/evaluation/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/datasets/evaluation
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/datasets/evaluation/voc
copying maskrcnn_benchmark/data/datasets/evaluation/voc/voc_eval.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/datasets/evaluation/voc
copying maskrcnn_benchmark/data/datasets/evaluation/voc/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/datasets/evaluation/voc
creating build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/datasets/evaluation/coco
copying maskrcnn_benchmark/data/datasets/evaluation/coco/coco_eval.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/datasets/evaluation/coco
copying maskrcnn_benchmark/data/datasets/evaluation/coco/__init__.py -> build/lib.linux-x86_64-3.7/maskrcnn_benchmark/data/datasets/evaluation/coco
running build_ext
building 'maskrcnn_benchmark._C' extension
creating /workspace/object_detection/pytorch/build/temp.linux-x86_64-3.7
creating /workspace/object_detection/pytorch/build/temp.linux-x86_64-3.7/workspace
creating /workspace/object_detection/pytorch/build/temp.linux-x86_64-3.7/workspace/object_detection
creating /workspace/object_detection/pytorch/build/temp.linux-x86_64-3.7/workspace/object_detection/pytorch
creating /workspace/object_detection/pytorch/build/temp.linux-x86_64-3.7/workspace/object_detection/pytorch/maskrcnn_benchmark
creating /workspace/object_detection/pytorch/build/temp.linux-x86_64-3.7/workspace/object_detection/pytorch/maskrcnn_benchmark/csrc
creating /workspace/object_detection/pytorch/build/temp.linux-x86_64-3.7/workspace/object_detection/pytorch/maskrcnn_benchmark/csrc/cpu
creating /workspace/object_detection/pytorch/build/temp.linux-x86_64-3.7/workspace/object_detection/pytorch/maskrcnn_benchmark/csrc/cuda
Traceback (most recent call last):
  File "setup.py", line 81, in <module>
    cmdclass={"build_ext": torch.utils.cpp_extension.BuildExtension},
  File "/opt/conda/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/opt/conda/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/opt/conda/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/opt/conda/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.7/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/opt/conda/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/opt/conda/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/opt/conda/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/opt/conda/lib/python3.7/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 735, in build_extensions
    build_ext.build_extensions(self)
  File "/opt/conda/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 194, in build_extensions
    self.build_extension(ext)
  File "/opt/conda/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
    _build_ext.build_extension(self, ext)
  File "/opt/conda/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
    depends=ext.depends)
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 551, in unix_wrap_ninja_compile
    cuda_post_cflags = unix_cuda_flags(cuda_post_cflags)
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 450, in unix_cuda_flags
    cflags + _get_cuda_arch_flags(cflags))
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1606, in _get_cuda_arch_flags
    arch_list[-1] += '+PTX'
IndexError: list index out of range
The command '/bin/sh -c cd pytorch  && rm -rf build/  && python setup.py clean build develop --user' returned a non-zero code: 1
@Zijie-Tian
Copy link

Same Problem. This works for me. According to this issue of Pytorch cuda does not install. I change Dockerfile

WORKDIR /workspace/object_detection
COPY . .
RUN cd pytorch \
 && rm -rf build/ \
 && TORCH_CUDA_ARCH_LIST="3.5;5.0;6.0;6.1;7.0;7.5;8.0;8.6+PTX" python setup.py clean build develop --user

You need to check you cuda version, Choose you own TORCH_CUDA_ARCH_LIST, According to this script:

CUDA_VERSION=$(/usr/local/cuda/bin/nvcc --version | sed -n 's/^.*release \([0-9]\+\.[0-9]\+\).*$/\1/p')
if [[ ${CUDA_VERSION} == 9.0* ]]; then
    export TORCH_CUDA_ARCH_LIST="3.5;5.0;6.0;7.0+PTX"
elif [[ ${CUDA_VERSION} == 9.2* ]]; then
    export TORCH_CUDA_ARCH_LIST="3.5;5.0;6.0;6.1;7.0+PTX"
elif [[ ${CUDA_VERSION} == 10.* ]]; then
    export TORCH_CUDA_ARCH_LIST="3.5;5.0;6.0;6.1;7.0;7.5+PTX"
elif [[ ${CUDA_VERSION} == 11.0* ]]; then
    export TORCH_CUDA_ARCH_LIST="3.5;5.0;6.0;6.1;7.0;7.5;8.0+PTX"
elif [[ ${CUDA_VERSION} == 11.* ]]; then
    export TORCH_CUDA_ARCH_LIST="3.5;5.0;6.0;6.1;7.0;7.5;8.0;8.6+PTX"
else
    echo "unsupported cuda version."
    exit 1
fi

@mahmoodn
Copy link
Author

Works fine. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants