OSError: undefined symbol: _ZN2at4_ops10zeros...

[jessienab]

Hello,
I attempt to start a transcription using a cloud compute provider, but I encounter an error I can't find any information about online...

Genuinely unsure how to proceed. I followed install instructions (near) exactly per README:

wget https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-amd64-static.tar.xz
tar xf ffmpeg-git-amd64-static.tar.xz
sudo mv ffmpeg*/*ffmpeg* /usr/local/bin/

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
chmod +x Miniconda3-latest-Linux-x86_64.sh
./Miniconda3-latest-Linux-x86_64.sh

conda create --name whisperx python=3.10 && conda activate whisperx

conda install pytorch==2.0.0 torchaudio==2.0.0 pytorch-cuda=11.8 -c pytorch -c nvidia && pip install git+https://github.com/m-bain/whisperx.git

Input audiofile is 2 channel FLAC 44.1khz/16-bit or WAV pcm_s16le.

Thank you!

$ whisperx strike.wav 
Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/envs/whisperx/bin/whisperx", line 5, in <module>
    from whisperx.transcribe import cli
  File "/home/ubuntu/miniconda3/envs/whisperx/lib/python3.10/site-packages/whisperx/__init__.py", line 1, in <module>
    from .transcribe import load_model
  File "/home/ubuntu/miniconda3/envs/whisperx/lib/python3.10/site-packages/whisperx/transcribe.py", line 9, in <module>
    from .alignment import align, load_align_model
  File "/home/ubuntu/miniconda3/envs/whisperx/lib/python3.10/site-packages/whisperx/alignment.py", line 11, in <module>
    import torchaudio
  File "/home/ubuntu/miniconda3/envs/whisperx/lib/python3.10/site-packages/torchaudio/__init__.py", line 1, in <module>
    from torchaudio import (  # noqa: F401
  File "/home/ubuntu/miniconda3/envs/whisperx/lib/python3.10/site-packages/torchaudio/_extension/__init__.py", line 43, in <module>
    _load_lib("libtorchaudio")
  File "/home/ubuntu/miniconda3/envs/whisperx/lib/python3.10/site-packages/torchaudio/_extension/utils.py", line 61, in _load_lib
    torch.ops.load_library(path)
  File "/home/ubuntu/miniconda3/envs/whisperx/lib/python3.10/site-packages/torch/_ops.py", line 1295, in load_library
    ctypes.CDLL(path)
  File "/home/ubuntu/miniconda3/envs/whisperx/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /home/ubuntu/miniconda3/envs/whisperx/lib/python3.10/site-packages/torchaudio/lib/libtorchaudio.so: undefined symbol: _ZN2at4_ops10zeros_like4callERKNS_6TensorEN3c108optionalINS5_10ScalarTypeEEENS6_INS5_6LayoutEEENS6_INS5_6DeviceEEENS6_IbEENS6_INS5_12MemoryFormatEEE

$ uname -a
Linux hostname 5.15.0-92-generic #102~20.04.1-Ubuntu SMP Mon Jan 15 13:09:14 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

$ nvidia-smi
Fri Aug  9 13:58:44 2024       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.183.06             Driver Version: 535.183.06   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA L40                     On  | 00000000:00:06.0 Off |                    0 |
| N/A   35C    P8              23W / 300W |      0MiB / 46068MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|  No running processes found                                                           |
+---------------------------------------------------------------------------------------+

$ apt list installed

libnvidia-cfg1-535/unknown,now 535.183.06-0ubuntu1 amd64 [installed,automatic]
libnvidia-common-535/unknown,now 535.183.06-0ubuntu1 all [installed,automatic]
libnvidia-compute-535/unknown,now 535.183.06-0ubuntu1 amd64 [installed,automatic]
libnvidia-decode-535/unknown,now 535.183.06-0ubuntu1 amd64 [installed,automatic]
libnvidia-encode-535/unknown,now 535.183.06-0ubuntu1 amd64 [installed,automatic]
libnvidia-extra-535/unknown,now 535.183.06-0ubuntu1 amd64 [installed,automatic]
libnvidia-fbc1-535/unknown,now 535.183.06-0ubuntu1 amd64 [installed,automatic]
libnvidia-gl-535/unknown,now 535.183.06-0ubuntu1 amd64 [installed,automatic]
libpython3-stdlib/focal,now 3.8.2-0ubuntu2 amd64 [installed,automatic]
libpython3.8-minimal/focal-updates,focal-security,now 3.8.10-0ubuntu1~20.04.11 amd64 [installed,automatic]
libpython3.8-stdlib/focal-updates,focal-security,now 3.8.10-0ubuntu1~20.04.11 amd64 [installed,automatic]
libpython3.8/focal-updates,focal-security,now 3.8.10-0ubuntu1~20.04.11 amd64 [installed,automatic]
nvidia-compute-utils-535/unknown,now 535.183.06-0ubuntu1 amd64 [installed,automatic]
nvidia-dkms-535/unknown,now 535.183.06-0ubuntu1 amd64 [installed,automatic]
nvidia-driver-535/unknown,now 535.183.06-0ubuntu1 amd64 [installed]
nvidia-kernel-common-535/unknown,now 535.183.06-0ubuntu1 amd64 [installed,automatic]
nvidia-kernel-source-535/unknown,now 535.183.06-0ubuntu1 amd64 [installed,automatic]
nvidia-utils-535/unknown,now 535.183.06-0ubuntu1 amd64 [installed,automatic]
python-apt-common/focal-updates,now 2.0.1ubuntu0.20.04.1 all [installed,automatic]
python3-apport/focal-updates,now 2.20.11-0ubuntu27.27 all [installed,automatic]
python3-apt/focal-updates,now 2.0.1ubuntu0.20.04.1 amd64 [installed,automatic]
python3-attr/focal,now 19.3.0-2 all [installed,automatic]
python3-automat/focal,now 0.8.0-1ubuntu1 all [installed,automatic]
python3-blinker/focal,now 1.4+dfsg1-0.3ubuntu1 all [installed,automatic]
python3-certifi/focal,now 2019.11.28-1 all [installed,automatic]
python3-cffi-backend/focal,now 1.14.0-1build1 amd64 [installed,automatic]
python3-chardet/focal,now 3.0.4-4build1 all [installed,automatic]
python3-click/focal,now 7.0-3 all [installed,automatic]
python3-colorama/focal,now 0.4.3-1build1 all [installed,automatic]
python3-commandnotfound/focal-updates,now 20.04.6 all [installed,automatic]
python3-configobj/focal,now 5.0.6-4 all [installed,automatic]
python3-constantly/focal,now 15.1.0-1build1 all [installed,automatic]
python3-cryptography/focal-updates,focal-security,now 2.8-3ubuntu0.3 amd64 [installed,automatic]
python3-dbus/focal,now 1.2.16-1build1 amd64 [installed,automatic]
python3-debconf/focal,now 1.5.73 all [installed,automatic]
python3-debian/focal-updates,now 0.1.36ubuntu1.1 all [installed,automatic]
python3-distro-info/focal-updates,now 0.23ubuntu1.1 all [installed,automatic]
python3-distro/focal,now 1.4.0-1 all [installed,automatic]
python3-distupgrade/focal-updates,now 1:20.04.41 all [installed,automatic]
python3-distutils/focal-updates,focal-security,now 3.8.10-0ubuntu1~20.04 all [installed]
python3-entrypoints/focal,now 0.3-2ubuntu1 all [installed,automatic]
python3-gdbm/focal-updates,focal-security,now 3.8.10-0ubuntu1~20.04 amd64 [installed,automatic]
python3-gi/focal,now 3.36.0-1 amd64 [installed,automatic]
python3-gpg/focal-updates,now 1.13.1-7ubuntu2.2 amd64 [installed,automatic]
python3-hamcrest/focal,now 1.9.0-3 all [installed,automatic]
python3-httplib2/focal,now 0.14.0-1ubuntu1 all [installed,automatic]
python3-hyperlink/focal,now 19.0.0-1 all [installed,automatic]
python3-idna/focal-updates,focal-security,now 2.8-1ubuntu0.1 all [installed,automatic]
python3-importlib-metadata/focal,now 1.5.0-1 all [installed]
python3-incremental/focal,now 16.10.1-3.2 all [installed,automatic]
python3-jinja2/focal-updates,focal-security,now 2.10.1-2ubuntu0.3 all [installed]
python3-json-pointer/focal,now 2.0-0ubuntu1 all [installed]
python3-jsonpatch/focal,now 1.23-3 all [installed]
python3-jsonschema/focal,now 3.2.0-0ubuntu2 all [installed]
python3-jwt/focal-updates,focal-security,now 1.7.1-2ubuntu2.1 all [installed,automatic]
python3-keyring/focal,now 18.0.1-2ubuntu1 all [installed,automatic]
python3-launchpadlib/focal,now 1.10.13-1 all [installed,automatic]
python3-lazr.restfulclient/focal,now 0.14.2-2build1 all [installed,automatic]
python3-lazr.uri/focal,now 1.0.3-4build1 all [installed,automatic]
python3-lib2to3/focal-updates,focal-security,now 3.8.10-0ubuntu1~20.04 all [installed]
python3-magic/focal,now 2:0.4.15-3 all [installed,automatic]
python3-markupsafe/focal,now 1.1.0-1build2 amd64 [installed]
python3-minimal/focal,now 3.8.2-0ubuntu2 amd64 [installed,automatic]
python3-more-itertools/focal,now 4.2.0-1build1 all [installed]
python3-nacl/focal,now 1.3.0-5 amd64 [installed]
python3-netifaces/focal,now 0.10.4-1ubuntu4 amd64 [installed,automatic]
python3-newt/focal,now 0.52.21-4ubuntu2 amd64 [installed,automatic]
python3-oauthlib/focal,now 3.1.0-1ubuntu2 all [installed,automatic]
python3-openssl/focal,now 19.0.0-1build1 all [installed,automatic]
python3-pexpect/focal,now 4.6.0-1build1 all [installed,automatic]
python3-pkg-resources/focal-updates,focal-security,now 45.2.0-1ubuntu0.1 all [installed,automatic]
python3-problem-report/focal-updates,now 2.20.11-0ubuntu27.27 all [installed,automatic]
python3-ptyprocess/focal,now 0.6.0-1ubuntu1 all [installed,automatic]
python3-pyasn1-modules/focal,now 0.2.1-0.2build1 all [installed,automatic]
python3-pyasn1/focal,now 0.4.2-3build1 all [installed,automatic]
python3-pymacaroons/focal,now 0.13.0-3 all [installed]
python3-pyrsistent/focal,now 0.15.5-1build1 amd64 [installed]
python3-requests-unixsocket/focal,now 0.2.0-2 all [installed,automatic]
python3-requests/focal-updates,focal-security,now 2.22.0-2ubuntu1.1 all [installed,automatic]
python3-secretstorage/focal,now 2.3.1-2ubuntu1 all [installed,automatic]
python3-serial/focal,now 3.4-5.1 all [installed]
python3-service-identity/focal,now 18.1.0-5build1 all [installed,automatic]
python3-setuptools/focal-updates,focal-security,now 45.2.0-1ubuntu0.1 all [installed]
python3-simplejson/focal,now 3.16.0-2ubuntu2 amd64 [installed,automatic]
python3-six/focal,now 1.14.0-2 all [installed,automatic]
python3-software-properties/focal-updates,now 0.99.9.12 all [installed,automatic]
python3-systemd/focal,now 234-3build2 amd64 [installed,automatic]
python3-twisted-bin/focal-updates,focal-security,now 18.9.0-11ubuntu0.20.04.3 amd64 [installed,automatic]
python3-twisted/focal-updates,focal-security,now 18.9.0-11ubuntu0.20.04.3 all [installed,automatic]
python3-unidiff/focal,now 0.5.5-2 all [installed,automatic]
python3-update-manager/focal-updates,now 1:20.04.10.21 all [installed,automatic]
python3-urllib3/focal-updates,focal-security,now 1.25.8-2ubuntu0.3 all [installed,automatic]
python3-wadllib/focal,now 1.3.3-3build1 all [installed,automatic]
python3-xdg/focal,now 0.26-1ubuntu1 all [installed,automatic]
python3-yaml/focal-updates,focal-security,now 5.3.1-1ubuntu0.1 amd64 [installed,automatic]
python3-zipp/focal-updates,focal-security,now 1.0.0-1ubuntu0.1 all [installed]
python3-zope.interface/focal,now 4.7.1-1 amd64 [installed,automatic]
python3.8-minimal/focal-updates,focal-security,now 3.8.10-0ubuntu1~20.04.11 amd64 [installed,automatic]
python3.8/focal-updates,focal-security,now 3.8.10-0ubuntu1~20.04.11 amd64 [installed,automatic]
python3/focal,now 3.8.2-0ubuntu2 amd64 [installed,automatic]
xserver-xorg-video-nvidia-535/unknown,now 535.183.06-0ubuntu1 amd64 [installed,automatic]

[levnikolaevich]

Have the same problem. I've made the steps to install whisperx exactly from the repository guide

Ubuntu 20.04, wsl2

[GuyPaddock]

It seems like perhaps pytorch and torchaudio need to be updated to work with the latest Nvidia integration. There are a slew of other issues out there like this as of February:

AbdBarho/stable-diffusion-webui-docker#649

Unfortunately, I don't get much better results when updating to newer versions:

• 2.1.0: RuntimeError: parallel_for failed: cudaErrorNoKernelImageForDevice: no kernel image is available for execution on the device
• 2.2.0: RuntimeError: parallel_for failed: cudaErrorNoKernelImageForDevice: no kernel image is available for execution on the device
• 2.3.0: RuntimeError: parallel_for failed: cudaErrorNoKernelImageForDevice: no kernel image is available for execution on the device
• 2.4.0: Could not load library libcudnn_ops_infer.so.8. Error: libcudnn_ops_infer.so.8: cannot open shared object file: No such file or directory

[GuyPaddock]

Looks like the errors I was seeing with 2.1.0 through 2.3.0 are covered by #794 and the workaround there works for me when using this environment.yaml:

name: whisperX
channels:
  - pytorch
  - nvidia
  - defaults
dependencies:
  - python=3.10
  - pip
  - pytorch==2.3.1
  - torchaudio==2.3.1
  - pytorch-cuda=11.8
  - torchvision==0.18.1
  - pip:
    - git+https://github.com/m-bain/whisperx.git

[GUUser91]

I want to give a heads up to those not using conda and using a standard virtual python environment. Install these

pip install nvidia-cudnn-cu12==8.9.7.29

pip install ctranslate2

Find the path of the libcudnn file

sudo find / -type f -iname '*libcudnn_ops_infer.so.8*'

export it to the terminal, it should look something like this

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/user/whisperx/lib/python3.11/site-packages/nvidia/cudnn/lib/

Now you it should work

 whisperx --model large-v3 --language en audio.wav

[totomtornado]

I am running the cpu install on linux... I encountered this same error... searched around a lot.... I was able to get things running by using:

pip install -U torch torchaudio

NOTE: I also needed to add a flag to my command when I called whisperx...

--compute_type int8

this was detailed #348