Wrong zlib library might be loaded

[jdemeyer]

After the zlib upgrade, I cannot no longer run Sage under plain Python:

./sage --python
Python 2.7.13 (default, May 31 2017, 17:36:06) 
[GCC 4.9.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sage.all
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/sage/sage-git/local/lib/python2.7/site-packages/sage/all.py", line 98, in <module>
    from sage.rings.all      import *
  File "/opt/sage/sage-git/local/lib/python2.7/site-packages/sage/rings/all.py", line 54, in <module>
    from .number_field.all import *
  File "/opt/sage/sage-git/local/lib/python2.7/site-packages/sage/rings/number_field/all.py", line 9, in <module>
    from .totallyreal import enumerate_totallyreal_fields_prim
  File "sage/rings/number_field/totallyreal_data.pxd", line 12, in init sage.rings.number_field.totallyreal (/opt/sage/sage-git/src/build/cythonized/sage/rings/number_field/totallyreal.c:12702)
  File "sage/rings/number_field/totallyreal_data.pyx", line 41, in init sage.rings.number_field.totallyreal_data (/opt/sage/sage-git/src/build/cythonized/sage/rings/number_field/totallyreal_data.c:13440)
  File "/opt/sage/sage-git/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring_constructor.py", line 462, in PolynomialRing
    R = _single_variate(base_ring, name, sparse, implementation)
  File "/opt/sage/sage-git/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring_constructor.py", line 540, in _single_variate
    R = m.PolynomialRing_integral_domain(base_ring, name, sparse, implementation)
  File "/opt/sage/sage-git/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring.py", line 1603, in __init__
    sparse=sparse, element_class=element_class, category=category)
  File "/opt/sage/sage-git/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring.py", line 1474, in __init__
    sparse=sparse, element_class=element_class, category=category)
  File "/opt/sage/sage-git/local/lib/python2.7/site-packages/sage/rings/polynomial/polynomial_ring.py", line 290, in __init__
    from sage.matrix.matrix_space import MatrixSpace
  File "/opt/sage/sage-git/local/lib/python2.7/site-packages/sage/matrix/matrix_space.py", line 48, in <module>
    from . import matrix_modn_sparse
  File "sage/matrix/matrix_integer_dense.pxd", line 10, in init sage.matrix.matrix_modn_sparse (/opt/sage/sage-git/src/build/cythonized/sage/matrix/matrix_modn_sparse.c:15164)
  File "sage/matrix/matrix_mod2_dense.pxd", line 4, in init sage.matrix.matrix_integer_dense (/opt/sage/sage-git/src/build/cythonized/sage/matrix/matrix_integer_dense.c:56343)
ImportError: /lib64/libz.so.1: version `ZLIB_1.2.9' not found (required by /opt/sage/sage-git/local/lib/libpng16.so.16)

This is because there is some kind of race condition: certain system libraries (OpenSSL for example) link against /lib64/libz.so.1 while Sage components link against the Sage-built libz.

In turns out that the first libz library which is loaded "wins": that one will be used by anything linking against libz. When starting IPython, the correct libz library is loaded because some IPython component uses libz. When running plain Python, this is not the case and the wrong libz is loaded. The easiest fix is to explicitly import libz when Sage starts up.

Component: build

Author: Jeroen Demeyer

Branch/Commit: d59afe3

Reviewer: Erik Bray

Issue created by migration from https://trac.sagemath.org/ticket/23122

[vbraun]

comment:3

Who is requiring the system libz? Maybe run with LD_DEBUG=files sage -python

[jdemeyer]

comment:4

Replying to @vbraun:

Who is requiring the system libz?

$ ldd local/lib/python2.7/lib-dynload/_ssl.so
        linux-vdso.so.1 (0x00007fffaa1ff000)
        libssl.so.1.0.0 => /usr/lib64/libssl.so.1.0.0 (0x00007ff017e96000)
        libcrypto.so.1.0.0 => /usr/lib64/libcrypto.so.1.0.0 (0x00007ff017a52000)
        libpython2.7.so.1.0 => /opt/sage/sage-git/local/lib/libpython2.7.so.1.0 (0x00007ff01762c000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff017410000)
        libc.so.6 => /lib64/libc.so.6 (0x00007ff017076000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007ff016e72000)
        libz.so.1 => /lib64/libz.so.1 (0x00007ff016c5c000)
        libutil.so.1 => /lib64/libutil.so.1 (0x00007ff016a58000)
        libm.so.6 => /lib64/libm.so.6 (0x00007ff016753000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff01832a000)

[jdemeyer]

Branch: u/jdemeyer/wrong_zlib_library_might_be_loaded

[jdemeyer]

New commits:

d59afe3

Load the correct zlib library

[jdemeyer]

Commit: d59afe3

[embray]

Reviewer: Erik Bray

[embray]

comment:7

I think this is fine. I've run into this problem before, for example, with libxml.

[vbraun]

comment:8

Hmm strange, I get

$ ldd local/lib/python2.7/lib-dynload/_ssl.so
	linux-vdso.so.1 (0x00007ffd963e3000)
	libssl.so.10 => /lib64/libssl.so.10 (0x00007f9162479000)
	libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f9162018000)
	libpython2.7.so.1.0 => /home/vbraun/Code/sage.git/local/lib/libpython2.7.so.1.0 (0x00007f9161bfd000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f91619df000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f9161619000)
	libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f91613ca000)
	libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f91610e4000)
	libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f9160ee0000)
	libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f9160caf000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f9160aab000)
	libz.so.1 => /home/vbraun/Code/sage.git/local/lib/libz.so.1 (0x00007f9160891000)
	libc.so.6 => /lib/libc.so.6 (0xf71dc000)
	libkrb5support.so.0 => /lib/libkrb5support.so.0 (0xf71cd000)
	libkeyutils.so.1 => /lib/libkeyutils.so.1 (0xf71c6000)
	libpthread.so.0 => /lib/libpthread.so.0 (0xf71a9000)
	/lib/ld-linux.so.2 (0x56570000)
	libselinux.so.1 => /lib/libselinux.so.1 (0xf7182000)
	libpcre.so.1 => /lib/libpcre.so.1 (0xf710b000)

[vbraun]

Changed branch from u/jdemeyer/wrong_zlib_library_might_be_loaded to d59afe3