Segmentation Fault with PyTorch

[Stannislav]

xgboost leads to a segmentation fault when torch is imported before xgboost. Tested on py37/macOS

To reproduce

Prepare the environment

virtualenv venv -p37
. venv/bin/activate
pip install torch 'xgboost[scikit-learn]'

Versions:

$ python -VV
Python 3.7.10 (default, Jun  9 2021, 14:25:02) 
[Clang 12.0.5 (clang-1205.0.22.9)]

$ pip freeze | grep -E 'torch|xgboost|scikit-learn'
scikit-learn==1.0.1
torch==1.10.1
xgboost==1.5.1

Minimal script:

# segfault_example.py
import numpy as np
import torch
from xgboost import XGBClassifier


model = XGBClassifier(objective="multi:softmax", num_class=3)
x = np.random.rand(3, 10)
y = np.arange(3)
model.fit(x, y)

run

$ python segfault_example.py
<unrelated warnings>
[1]    27331 segmentation fault  segfault_example.py

Note

If torch is imported after xgboost then the segfault goes away. (Try swappign the imports in the script above.)

System

• macOS Big Sur - Version 11.6.2
• python 3.7.10
• scikit-learn==1.0.1
• torch==1.10.1
• xgboost==1.5.1

[trivialfis]

Is it related to #7039 ?

[Stannislav]

Is it related to #7039 ?

Good point. I just downgraded to libomp==11.1.0 as per #7039 and the error went away.

I guess torch does something with libomp as well? Not sure why the error goes away if torch is imported after xgboost.

[trivialfis]

Not sure why the error goes away if torch is imported after xgboost

I don't know, would be curious to take a look into libomp if the issue is on Linux ..

Closing as it's duplicated to #7039