OSError: libcusparse.so.11: cannot open shared object file: No such file or directory

[turmeric-blend]

📚 Installation

---

Error occurs when I run from torch_geometric.data import Data

OSError Traceback (most recent call last)
in
1 import torch
----> 2 from torch_geometric.data import Data

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_geometric/init.py in
1 from .debug import is_debug_enabled, debug, set_debug
----> 2 import torch_geometric.nn
3 import torch_geometric.data
4 import torch_geometric.datasets
5 import torch_geometric.transforms

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_geometric/nn/init.py in
1 from .meta import MetaLayer
----> 2 from .data_parallel import DataParallel
3 from .reshape import Reshape
4 from .conv import * # noqa
5 from .norm import * # noqa

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_geometric/nn/data_parallel.py in
3
4 import torch
----> 5 from torch_geometric.data import Batch
6
7

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_geometric/data/init.py in
----> 1 from .data import Data
2 from .batch import Batch
3 from .dataset import Dataset
4 from .in_memory_dataset import InMemoryDataset
5 from .dataloader import DataLoader, DataListLoader, DenseDataLoader

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_geometric/data/data.py in
6 import torch
7 import torch_geometric
----> 8 from torch_sparse import coalesce, SparseTensor
9 from torch_geometric.utils import (contains_isolated_nodes,
10 contains_self_loops, is_undirected)

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_sparse/init.py in
11 ]:
12 torch.ops.load_library(importlib.machinery.PathFinder().find_spec(
---> 13 library, [osp.dirname(file)]).origin)
14
15 if torch.cuda.is_available() and torch.version.cuda: # pragma: no cover

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch/_ops.py in load_library(self, path)
103 # static (global) initialization code in order to register custom
104 # operators with the JIT.
--> 105 ctypes.CDLL(path)
106 self.loaded_libraries.add(path)
107

~/anaconda3/envs/pytorch_env/lib/python3.7/ctypes/init.py in init(self, name, mode, handle, use_errno, use_last_error)
362
363 if handle is None:
--> 364 self._handle = _dlopen(self._name, mode)
365 else:
366 self._handle = handle

OSError: libcusparse.so.11: cannot open shared object file: No such file or directory

Environment

• OS: Ubuntu 18.04
• Python version: 3.7.6
• PyTorch version: 1.7.1
• CUDA/cuDNN version: 11.0
• GCC version: 7.5.0
• How did you try to install PyTorch Geometric and its extensions (wheel, source): Binaries
• Any other relevant information: followed this guide Installation via Binaries

Checklist

• [y] I followed the installation guide.
• I cannot find my error message in the FAQ.
• I set up CUDA correctly and can compile CUDA code via nvcc. <--- not sure how to do this, but I have been running pytorch on GPU fine all this while
• I do have multiple CUDA versions on my machine. <--- not sure how to check but when I run python -c "import torch; print(torch.__version__)" returns 1.7.1+cu110 and python -c "import torch; print(torch.version.cuda)" returns 11.0

Additional context

Installed via:

pip install --no-index torch-scatter -f https://pytorch-geometric.com/whl/torch-1.7.0+cu110.html
pip install --no-index torch-sparse -f https://pytorch-geometric.com/whl/torch-1.7.0+cu110.html
pip install --no-index torch-cluster -f https://pytorch-geometric.com/whl/torch-1.7.0+cu110.html
pip install --no-index torch-spline-conv -f https://pytorch-geometric.com/whl/torch-1.7.0+cu110.html
pip install torch-geometric

[rusty1s]

You can try to add the minconda CUDA libs installed by PyTorch to $LD_LIBRARY_PATH, e.g.:

export LD_LIBRARY_PATH="your_miniconda_path/lib:$LD_LIBRARY_PATH"

This folder should contain libcusparse.so.11.

[zilangch]

I have write it "export PATH=/usr/local/cuda-11.0/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH" in ~/.bashrc, but it doesn't work, it's so weird.

However, I can use geometric in terminal, but can't pycharm

[rusty1s]

I'm not a PyCharm user, so I can't give you any advice on this one, but I guess PyCharm has some kind of editor to set environment variables.

[domilay]

I have write it "export PATH=/usr/local/cuda-11.0/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH" in ~/.bashrc, but it doesn't work, it's so weird.

However, I can use geometric in terminal, but can't pycharm

Same problem, work well in terminal but not in pycharm...

[DeVriesMatt]

I have the same problem, however it is not working in terminal either

[rusty1s]

Please make sure that the folder containing libcusparse.so.11 is linked in LD_LIBRARY_PATH.

[DeVriesMatt]

@rusty1s I did this but still was not working.
I have just got it to work through downgrading the CUDA version for both PyTorch and the PyTorch Geometric dependencies from 11.1 to 10.2

[kt66nf]

I had the exact same issue with torch 1.9.0 and cu 11.1 as mentioned in OP. I did what @DeVriesMatt did and it seems to work perfectly fine now.

[Yonggie]

I got the same problem with python3.8 torch_geometric1.9.0.

[Yonggie]

I've checked my virtual environment path your_miniconda_path/lib only to find out there's actually no file named libcusparse.so...
Does that mean the torch-sparse is illedly installed?

[Yonggie]

My other virtual envs containing torch-geometric 1.8.0 does have libcusparse.so in lib, but geometric1.9.0+cu111 doesn't.
though they are installed by me following exactly the same as documentation installation.

[Yonggie]

I had the exact same issue with torch 1.9.0 and cu 11.1 as mentioned in OP. I did what @DeVriesMatt did and it seems to work perfectly fine now.

I did downgrading too, solved my problem.

[tchayintr]

I had similar issue with OP.
I was set conda environment with python 3.8.x. pytorch 1.8.1 with cuda 11.1, and pyg 1.7.2.
It works totally fine with cuda 11.1 but when I upgraded pytorch 1.8.1 to 1.9.0 along with compatible torch-geometric by using pip and try to import some modules e.g., GCNConv it will show the error as mentioned in OP:
OSError: libcusparse.so.11: cannot open shared object file: No such file or directory.

I checked environment's lib, there actually is libcusparse.so.11 but it seems pyg could not find it somehow after upgrading.
I linked the folder containing libcusparse.so.11 to LD_LIBRARY_PATH, however, it doesn't work for me.

So I tried to create new conda-environment with python 3.9.6 and install pytorch 1.9.0 with cuda11.1 plus compatible pyg by using conda.
I am able to execute pyg's lib without any error so far.
I could not provide an accurate reason about this issue but hope this information can help someone somehow.

[Zhang-Zhiyuan-zzy]

You can try to add the minconda CUDA libs installed by PyTorch to $LD_LIBRARY_PATH, e.g.:

export LD_LIBRARY_PATH="your_miniconda_path/lib:$LD_LIBRARY_PATH"

This folder should contain libcusparse.so.11.

@rusty1s I find that my miniconda/lib folder do not contain the "libcusparse.so.11". How can I install it?

[rusty1s]

If you are inside a custom miniconda environment, you may also find them inside miniconda3/envs/{env_name}/lib. Can you check? Note that you need to run conda install cudatoolkit=... -c pytorch first.

[kunal-bhadra]

10.2

Still works today, thanks for the info!

[tyang816]

I had similar issue with OP. I was set conda environment with python 3.8.x. pytorch 1.8.1 with cuda 11.1, and pyg 1.7.2. It works totally fine with cuda 11.1 but when I upgraded pytorch 1.8.1 to 1.9.0 along with compatible torch-geometric by using pip and try to import some modules e.g., GCNConv it will show the error as mentioned in OP: OSError: libcusparse.so.11: cannot open shared object file: No such file or directory.

I checked environment's lib, there actually is libcusparse.so.11 but it seems pyg could not find it somehow after upgrading. I linked the folder containing libcusparse.so.11 to LD_LIBRARY_PATH, however, it doesn't work for me.

So I tried to create new conda-environment with python 3.9.6 and install pytorch 1.9.0 with cuda11.1 plus compatible pyg by using conda. I am able to execute pyg's lib without any error so far. I could not provide an accurate reason about this issue but hope this information can help someone somehow.

Thanks! conda install is useful.

[smiles724]

I have write it "export PATH=/usr/local/cuda-11.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH" in ~/.bashrc, but it doesn't work, it's so weird.

However, I can use geometric in terminal, but can't pycharm

I met a similar case. I can run it in the terminal but not in the jupyter notebook

[rusty1s]

can you add the path inside the jupyter notebook as well?

[MirMurtazaa]

can you add the path inside the jupyter notebook as well?

I added the path in Jupyter Notebook, but did not work. Any suggestions? Thanks

[MirMurtazaa]

I have write it "export PATH=/usr/local/cuda-11.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH" in ~/.bashrc, but it doesn't work, it's so weird.
However, I can use geometric in terminal, but can't pycharm

I met a similar case. I can run it in the terminal but not in the jupyter notebook

I solved this by:

Steps:

1. Open the .bashrc file in your home directory (for example, /home/your-user-name/.bashrc) in a text editor.
2. Add export "export LD_LIBRARY_PATH="your_minicondaORanaconda_path/lib:$LD_LIBRARY_PATH"" to the last line of the file
3. Save the .bashrc file.
4. Restart(maybe)

[chinggg]

For those who cannot find libcusparse.so in anywhere and don't have root permission to install it easily, check out #392 (comment) to install CUDA toolkit without root.

[HelloWorldLTY]

Hi, I meet this problem again.
libcusparse.so.11: cannot open shared object file: No such file or directory

I tried to use conda install to install pyg. Does anyone successfully do that? Thanks.

[rusty1s]

What is your PyTorch version and how did you install it? Do you have a local CUDA version installed?

[xuhongzuo]

I have write it "export PATH=/usr/local/cuda-11.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH" in ~/.bashrc, but it doesn't work, it's so weird.

However, I can use geometric in terminal, but can't pycharm

@smiles724 @zilangch @domilay
I found a solution in Pycharm. It can be solved by manually adding LIBRARY_PATH in the run/debug configurations.

[gorkamunoz]

I had similar issue with OP. I was set conda environment with python 3.8.x. pytorch 1.8.1 with cuda 11.1, and pyg 1.7.2. It works totally fine with cuda 11.1 but when I upgraded pytorch 1.8.1 to 1.9.0 along with compatible torch-geometric by using pip and try to import some modules e.g., GCNConv it will show the error as mentioned in OP: OSError: libcusparse.so.11: cannot open shared object file: No such file or directory.

I checked environment's lib, there actually is libcusparse.so.11 but it seems pyg could not find it somehow after upgrading. I linked the folder containing libcusparse.so.11 to LD_LIBRARY_PATH, however, it doesn't work for me.

So I tried to create new conda-environment with python 3.9.6 and install pytorch 1.9.0 with cuda11.1 plus compatible pyg by using conda. I am able to execute pyg's lib without any error so far. I could not provide an accurate reason about this issue but hope this information can help someone somehow.

Using conda install also solved my problem. In my case I was facing the same problem but with CUDA toolkit 11.6.

[zhuwangjulia]

I have write it "export PATH=/usr/local/cuda-11.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH" in ~/.bashrc, but it doesn't work, it's so weird.

However, I can use geometric in terminal, but can't pycharm

if the above method does not work, you can try to add the following code in your main.py
import sys
sys.path.append('/home/name/.conda/envs/envs_name/')
Also the path should contain the file libcusparse.so.11 .

[kietbg0079]

I have write it "export PATH=/usr/local/cuda-11.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH" in ~/.bashrc, but it doesn't work, it's so weird.
However, I can use geometric in terminal, but can't pycharm

I met a similar case. I can run it in the terminal but not in the jupyter notebook

I solved this by:

Steps:

1. Open the .bashrc file in your home directory (for example, /home/your-user-name/.bashrc) in a text editor.
2. Add export "export LD_LIBRARY_PATH="your_minicondaORanaconda_path/lib:$LD_LIBRARY_PATH"" to the last line of the file
3. Save the .bashrc file.
4. Restart(maybe)

I met the same issue, it can run in the terminal but failed when run in jupyter. But this solution doesn't work for me. P

[FancyRay]

I have write it "export PATH=/usr/local/cuda-11.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH" in ~/.bashrc, but it doesn't work, it's so weird.
However, I can use geometric in terminal, but can't pycharm

@smiles724 @zilangch @domilay I found a solution in Pycharm. It can be solved by manually adding LIBRARY_PATH in the run/debug configurations.

Hi, thanks for your solution. But i wonder wether i have to manually edit the run/debug configurations everytime i run a new python script. I try to add the environment variables in setting-Build,Excution,Deployment-Console in pycharm, but it did not work. Did you find any solution for this?

[harshap-ai]

Check if nvcc --version works. If not, do sudo apt install nvidia-cuda-tookit.

Even if you had lubcusparse.so.11 in your lib directory prior to this step, it maybe outdated/broken. Doing the above install worked for me.

[AllenZ01]

I've also come across this issue and have solved it. Here are my findings:

Solution:

Make sure the versions of your CUDA, CUDA toolkit, Pytorch, torch_sparse&scatter&... satisfies:

torch_sparse&scatter&... == Pytorch == CUDA toolkit <= CUDA

for those who are confused with these:

1 Check your CUDA version

use nvidia-smi to check your CUDA version

2. Check your CUDA toolkit version

First you need to know whether you are using cudatoolkit from a virtual environment or from system environment. Note this is not equal to whether you are running your code in a virtual environment or system environment. Actually you can run your code in a virtual environment and using cudatoolkit from system environment. A simple way to tell is: if you are using conda and followed the instructions in https://pytorch.org/get-started/previous-versions/ to install Pytorch & cudatoolkit, e.g. conda install pytorch==1.6.0 torchvision==0.7.0 cudatoolkit=10.2 -c pytorch, then you are using cudatoolkit from your conda virtual environment. Else you are using cudatoolkit from your system environment.

2.1 if you are using cudatoolkit from a virtual environment(conda)

use conda list after you activate the corresponding environment, and check the version of cudatoolkit.
note: cudatoolkit should be installed together with Pytorch. You should use commands in https://pytorch.org/get-started/previous-versions/ to install Pytorch & cudatoolkit

2.2 if you are using cudatoolkit from system environment

You need to download cudatoolkit from https://developer.nvidia.com/cuda-toolkit-archive and install it manually. You can install multiple versions of cudatoolkit in your system environment. But you need to specify the version (corresponding to your code) before you run your codes. This Article might help:
https://towardsdatascience.com/managing-multiple-cuda-versions-on-a-single-machine-a-comprehensive-guide-97db1b22acdc
use nvcc -V to check what version of cudatoolkit you are using

3. Pytorch version

You should follow the instructions in https://pytorch.org/get-started/previous-versions/ to install Pytorch. If you are not using conda, make sure your Pytorch version == cudatoolkit version.
e.g. if your cudatoolkit version==10.1, then you should use torch==x.x.x+cu101.

4. PyG Dependencies version

You should follow the instructions in https://pytorch-geometric.readthedocs.io/en/latest/install/installation.html to install PyG and its dependencies (torch_scatter, torch_sparse,...), the versions of which depend on the version of Pytorch (https://data.pyg.org/whl/).

Hope this will help.