Runtime conflicts between libsbml wheels and other packages

[cdiener]

Hi,

we noticed that some other wheels can break libsbml behavior. For instance on Python 3.9 with liSBML=5.19.0:

In [1]: import symengine

In [2]: import libsbml

In [3]: doc = libsbml.readSBML("iJO1366.xml.gz")

In [4]: doc.getModel().getReaction("R_LDH_D").getSBOTermID()
Out[4]: 'SBO:'

Removing the symengine import will change the output to:

Python 3.9.0 (default, Nov 15 2020, 14:28:56) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.21.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import libsbml

In [2]: doc = libsbml.readSBML("iJO1366.xml.gz")

In [3]: doc.getModel().getReaction("R_LDH_D").getSBOTermID()
Out[3]: 'SBO:0000375'

Could be related to libc or llvm issues.

I also noticed that you are using a fairly old manylinux version for Python 3.9 (see the table at https://github.com/pypa/manylinux).

[Midnighter]

Some more background in opencobra/cobrapy#1056. This is the issue I mentioned at HARMONY @fbergmann.

[fbergmann]

I've tried it with manylinux2010, but am actually getting crashes immediately when importing libsbml after symengine. @cdiener or @Midnighter, could you try whether this issue occurs as well, when libSBML is installed via conda-forge?

https://github.com/conda-forge/python-libsbml-feedstock

thanks

[Midnighter]

From conda-forge it indeed works for me.

Python 3.9.2 | packaged by conda-forge | (default, Feb 21 2021, 05:02:46)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import symengine
>>> import libsbml
>>> doc = libsbml.readSBML("iJO1366.xml.gz")
>>> doc.getModel().getReaction("R_LDH_D").getSBOTermID()
'SBO:0000375'

channels:
  - conda-forge
  - bioconda
  - defaults
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=1_gnu
  - ca-certificates=2020.12.5=ha878542_0
  - certifi=2020.12.5=py39hf3d152e_1
  - gmp=6.2.1=h58526e2_0
  - ld_impl_linux-64=2.35.1=hea4e1c9_2
  - libffi=3.3=h58526e2_2
  - libflint=2.7.1=h6307760_nontl_1
  - libgcc-ng=9.3.0=h2828fa1_18
  - libgomp=9.3.0=h2828fa1_18
  - libstdcxx-ng=9.3.0=h6de172a_18
  - mpc=1.1.0=h04dde30_1009
  - mpfr=4.0.2=he80fd80_1
  - ncurses=6.2=h58526e2_4
  - openssl=1.1.1j=h7f98852_0
  - pip=21.0.1=pyhd8ed1ab_0
  - python=3.9.2=hffdb5ce_0_cpython
  - python-libsbml=5.19.0=py39he80948d_1
  - python-symengine=0.7.0=py39h68068d8_0
  - python_abi=3.9=1_cp39
  - readline=8.0=he28a2e2_2
  - setuptools=49.6.0=py39hf3d152e_3
  - sqlite=3.35.2=h74cdb3f_0
  - symengine=0.7.0=hf1fd44d_1
  - tk=8.6.10=h21135ba_1
  - tzdata=2021a=he74cb21_0
  - wheel=0.36.2=pyhd3deb0d_0
  - xz=5.2.5=h516909a_1
  - zlib=1.2.11=h516909a_1010

[cdiener]

This is weird. I would think it works with conda since that guaranteed the same version in libc etc. So symbols won't conflict. But this should be the same case if both use the same manylinux image. Maybe it installed the 2014 version for symengine? We only observed the error on Python 3.9 so it could be an issue with the manylinux2014 image specifically.

[fbergmann]

I also ran the manylinux2014 container and built it there. There pip would install the manylinux2010 version of symengine. But the crash is also happening.

[cdiener]

Okay, thanks for checking!

[Midnighter]

As far as I can tell, there are 72 symbols with the same name available from the libsbml and symengine shared libraries. I've dumped outputs and a quick notebook looking at them in this gist.

[cdiener]

This is a shot in the dark but could this be a bug? I looked at https://github.com/sbmlteam/libsbml/blob/947560213b8f22fe6c64919b6d1abb1a018b7d64/src/sbml/SBO.cpp. And saw the following:

bool
SBO::checkTerm (const std::string& sboTerm)  // <- signature requires string
{
  string::size_type size = sboTerm.size();
  bool              okay = (size == 11);

  char sbo[4]    = {83, 66, 79, 58};
  unsigned int n = 0;

  while (okay && n < 4)
  {
    okay = (sboTerm[n] == sbo[n]);
    n++;
  }

  for (n = 4; okay && n < size; ++n) okay = isdigit(sboTerm[n]);

  return okay;
}

...

string
SBO::intToString (int sboTerm)
{
  string result = "";

  if ( checkTerm(sboTerm) )  // <- see here, sboTerm is an int
  {
    ostringstream stream;
    stream << "SBO:";
    stream << setw(7) << setfill('0') << sboTerm;
    result = stream.str();
  }

  return result;
}

Maybe there is some undefined behavior involved because of the implicit cast?

[fbergmann]

there is an overload with int too:

bool
SBO::checkTerm (int sboTerm)
{
  return (sboTerm >= 0 && sboTerm <= 9999999);
}

the issue is more likely related to one of the library linking statically against the c++ library. and some static string members being freed.

[fbergmann]

As indicated on that project, it compiles with with -static-libgcc -static-libstdcxx, that could cause the issue as described here:

conda-forge/conda-forge.github.io#388

[cdiener]

This has been fixed in symengine 0.7.1 now. Thanks for your help with this!

Python 3.9.2 (default, Feb 28 2021, 17:03:44) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.21.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import symengine

In [2]: import libsbml

In [3]: doc = libsbml.readSBML("iJO1366.xml")

In [4]: doc.getModel().getReaction("R_LDH_D").getSBOTermID()
Out[4]: 'SBO:0000375'