pear install: error in malloc_consolidate

[mokraemer]

Description

pear install raises error on shutdown in conjunction with ZTS:
pear -c pearrc install --nodeps --packagingroot /root/php-pear-Calendar/BUILDROOT/php-pear-Calendar-0.5.5-10.mga9.x86_64 /root/php-pear-Calendar/SOURCES/Calendar-0.5.5.tgz

shows the followin error in strace:
malloc_consolidate(): unaligned fastbin chunk detected

#0 0x00007f6d9bac5bec in __pthread_kill_implementation () from /lib64/libc.so.6
#1 0x00007f6d9ba792f2 in raise () from /lib64/libc.so.6
#2 0x00007f6d9ba65457 in abort () from /lib64/libc.so.6
#3 0x00007f6d9baba788 in __libc_message () from /lib64/libc.so.6
#4 0x00007f6d9bace98a in malloc_printerr () from /lib64/libc.so.6
#5 0x00007f6d9bacf41c in malloc_consolidate () from /lib64/libc.so.6
#6 0x00007f6d9bad0800 in _int_free () from /lib64/libc.so.6
#7 0x00007f6d9bad2a4b in free () from /lib64/libc.so.6
#8 0x00000000008856e5 in root_buffer_dtor (gc_globals=0x27244f0) at /usr/src/debug/php-8.1.3-2.mga9.x86_64/Zend/zend_gc.c:425
#9 0x0000000000784154 in tsrm_shutdown () at /usr/src/debug/php-8.1.3-2.mga9.x86_64/TSRM/TSRM.c:194
#10 0x00000000006402b7 in main (argc=31, argv=0x2722da0) at /usr/src/debug/php-8.1.3-2.mga9.x86_64/sapi/cli/php_cli.c:1388

This is a new issue in PHP 8.1 with 8.0.x no error is raised.

PHP Version

php 8.1.3

Operating System

Mageia 8

[arnaud-lb]

Hello

I was not able to reproduce with php 8.1.4 zts on ubuntu 20 with the following command:

pear install Calendar-0.5.5

Do you have any extensions installed ? What is the output of php -m ?

[mokraemer]

pear command is:

exec /usr/bin/php -n -C \
        -d extension=filter \
        -d extension=openssl \
        -d include_path=/usr/share/pear \
        -d date.timezone=UTC \
        -d output_buffering=1 \
        -d variables_order=EGPCS \
        -d register_argc_argv="On" \
        -d open_basedir="" \
        -d auto_prepend_file="" \
        -d auto_append_file=""  \
        /usr/share/pear/pearcmd.php "$@"

so no comfig is used.

php -n -m
[PHP Modules]
Core
date
hash
json
libxml
pcre
Reflection
SimpleXML
SPL
standard
xml

gcc version 12.0.1
glibc-2.35

php -i|grep "Configure Command"
Configure Command =>  './configure'  '--enable-fpm' '--with-fpm-user=apache' '--with-fpm-group=apache' '--with-fpm-systemd' '--enable-cli' '--with-apxs2=/usr/bin/apxs' '--enable-zts' '--with-pic' '--build=x86_64-mageia-linux-gnu' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--localstatedir=/var/lib' '--mandir=/usr/share/man' '--enable-shared=yes' '--enable-static=no' '--disable-debug' '--enable-bcmath=shared' '--enable-calendar=shared' '--enable-ctype=shared' '--enable-dba=shared' '--enable-dom=shared,/usr' '--enable-exif=shared' '--enable-fileinfo=shared' '--enable-filter=shared' '--enable-ftp=shared' '--enable-gd=shared' '--enable-intl=shared' '--enable-mbregex' '--enable-mbstring=shared,/usr' '--enable-mysqlnd=shared,/usr/bin/mysql_config' '--enable-opcache=shared' '--enable-pcntl=shared' '--enable-pdo=shared,/usr' '--enable-phar=shared' '--enable-phpdbg' '--enable-phpdbg-webhelper' '--enable-posix=shared' '--enable-session=shared,/usr' '--enable-shmop=shared,/usr' '--enable-simplexml' '--enable-soap=shared,/usr' '--enable-sockets=shared,/usr' '--enable-sysvmsg=shared,/usr' '--enable-sysvsem=shared,/usr' '--enable-sysvshm=shared,/usr' '--enable-tokenizer=shared,/usr' '--enable-xmlreader=shared,/usr' '--enable-xmlwriter=shared,/usr' '--with-zip=shared' '--with-bz2=shared,/usr' '--with-cdb' '--with-config-file-path=/etc' '--with-config-file-scan-dir=/etc/php.d' '--with-curl=shared,/usr' '--with-db4' '--with-enchant=shared,/usr' '--with-freetype=/usr' '--with-gdbm' '--with-external-gd=shared,/usr' '--with-gettext=shared,/usr' '--with-gmp=shared,/usr' '--with-iconv=shared' '--with-imap=shared,/usr' '--with-imap-ssl=/usr' '--with-jpeg=/usr' '--with-ldap-sasl' '--with-ldap=shared,/usr' '--with-libdir=lib64' '--with-mhash=shared,/usr' '--with-mysqli=shared,mysqlnd' '--with-mysql-sock=/var/lib/mysql/mysql.sock' '--with-openssl-dir=/usr' '--with-openssl=shared,/usr' '--without-pear' '--with-external-pcre=/usr' '--with-pcre-jit' '--with-pdo-dblib=shared,/usr' '--with-pdo-firebird=shared,/usr/lib64/firebird' '--with-pdo-mysql=shared,mysqlnd' '--with-pdo-odbc=shared,unixODBC,/usr' '--with-pdo-pgsql=shared,/usr' '--with-pdo-sqlite=shared,/usr' '--with-pgsql=shared,/usr' '--with-readline=shared,/usr' '--with-snmp=shared,/usr' '--with-sodium=shared,/usr' '--with-sqlite3=shared,/usr' '--with-system-ciphers' '--with-tidy=shared,/usr' '--with-unixODBC=shared,/usr' '--with-webp=/usr' '--with-xpm=/usr/X11R6' '--with-xsl=shared,/usr' '--with-zlib=shared,/usr' '--with-zlib-dir=/usr' 'build_alias=x86_64-mageia-linux-gnu' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig' 'ENCHANT_CFLAGS=-I/usr/include/enchant-2 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -pthread ' 'ENCHANT_LIBS=-lenchant-2 '

[arnaud-lb]

I tried reproducing with the same configure command, with no luck. Here is the docker file I used:

FROM ubuntu:20.04

ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y git gcc autoconf make bison re2c pkg-config libsystemd-dev libxml2-dev libssl-dev libpcre2-dev zlib1g-dev libbz2-dev libcurl4-openssl-dev libsqlite3-dev wget
RUN git clone --depth 1 --single-branch --branch PHP-8.1 https://github.com/php/php-src
RUN cd php-src && \
    ./buildconf && \
    './configure'  '--enable-fpm' '--with-fpm-user=apache' '--with-fpm-group=apache' '--with-fpm-systemd' '--enable-cli' '--enable-zts' '--with-pic' '--build=x86_64-mageia-linux-gnu' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--localstatedir=/var/lib' '--mandir=/usr/share/man' '--enable-shared=yes' '--enable-static=no' '--disable-debug'     '--enable-filter=shared'      '--enable-mbregex'       '--enable-phpdbg' '--enable-phpdbg-webhelper'    '--enable-simplexml'           '--with-cdb' '--with-config-file-path=/etc' '--with-config-file-scan-dir=/etc/php.d'  '--with-freetype=/usr'     '--with-imap-ssl=/usr' '--with-jpeg=/usr' '--with-ldap-sasl'  '--with-libdir=lib64'   '--with-mysql-sock=/var/lib/mysql/mysql.sock' '--with-openssl=shared' '--with-openssl-dir=/usr'  '--with-pear' '--with-external-pcre=/usr' '--with-pcre-jit'            '--with-system-ciphers'   '--with-webp=/usr' '--with-xpm=/usr/X11R6'   '--with-zlib-dir=/usr' 'build_alias=x86_64-mageia-linux-gnu' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig' 'ENCHANT_CFLAGS=-I/usr/include/enchant-2 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -pthread ' 'ENCHANT_LIBS=-lenchant-2 ' && \
    make -j16 && \
    make install
RUN echo 'extension=openssl.so' >> /etc/php.ini

(I've just removed some shared extensions from the configure command.)

[mokraemer]

this is strange - it even happens with my latest backport of php 8.1.5:
https://ftp-stud.hs-esslingen.de/pub/Mirrors/Mageia/distrib/8/x86_64/media/core/backports_testing/
where php-8.0.18 installs without problems.

Do you have a negative return code of pear-install?
I'm not sure why this fails during destruction, but at least the write operation on the filedescriptor fails (since it was already closed). I'm not sure if this is "real" crash reason.

What version of glibc are you using?
I didn't see a problem during regular php runs - and php test cases run w/o problem.

I've just checked glibc, but I don't see when fastbins are used and when normal bin used during malloc. And this test is only active on "fastbins".

[arnaud-lb]

Here is a Dockerfile based on mageia 8 and your backport packages:

FROM mageia:8

RUN urpmi.addmedia BackportsTesting --update https://ftp-stud.hs-esslingen.de/pub/Mirrors/Mageia/distrib/8/x86_64/media/core/backports_testing/
RUN urpmi.update BackportsTesting
RUN urpmi --auto php-cli php-filter php-pear
RUN /usr/bin/php -n -C \
        -d extension=filter \
        -d extension=openssl \
        -d include_path=/usr/share/pear \
        -d date.timezone=UTC \
        -d output_buffering=1 \
        -d variables_order=EGPCS \
        -d register_argc_argv="On" \
        -d open_basedir="" \
        -d auto_prepend_file="" \
        -d auto_append_file=""  \
        /usr/share/pear/pearcmd.php install Calendar-0.5.5 && echo Success

Here is the output of docker build -t mageia . :

Step 5/5 : RUN /usr/bin/php -n -C         -d extension=filter         -d extension=openssl         -d include_path=/usr/share/pear         -d date.timezone=UTC         -d output_buffering=1         -d variables_order=EGPCS         -d register_argc_argv="On"         -d open_basedir=""         -d auto_prepend_file=""         -d auto_append_file=""          /usr/share/pear/pearcmd.php install Calendar-0.5.5 && echo "Success"
 ---> Running in 7d58acac3917
WARNING: channel "pear.php.net" has updated its protocols, use "pear channel-update pear.php.net" to update
Did not download optional dependencies: pear/Date, use --alldeps to download automatically
pear/Calendar can optionally use package "pear/Date"
downloading Calendar-0.5.5.tar ...
Starting to download Calendar-0.5.5.tar (Unknown size)
.............................................................................................................done: 544,256 bytes
install ok: channel://pear.php.net/Calendar-0.5.5
Success

Unfortunately, the pear command returns successfully.

Could you try to adapt this dockerfile to match your environment ?

[mokraemer]

I can confirm this one working. But

cd /tmp
wget http://pear.php.net/get/Calendar-0.5.5.tgz
/usr/bin/php -n -C\
         -d extension=filter\
         -d extension=openssl\
         -d include_path=/usr/share/pear\
         -d date.timezone=UTC\
         -d output_buffering=1\
         -d variables_order=EGPCS\
         -d register_argc_argv="On"\
         -d open_basedir=""\
         -d auto_prepend_file=""\
         -d auto_append_file=""\
          /usr/share/pear/pearcmd.php install  /tmp/Calendar-0.5.5.tgz
Did not download optional dependencies: pear/Date, use --alldeps to download automatically
pear/Calendar can optionally use package "pear/Date"
install ok: channel://pear.php.net/Calendar-0.5.5
Abgebrochen (Speicherabzug geschrieben)

raises an exception (core dump).

[arnaud-lb]

Thank you, I can reproduce the issue now :)

Running the command with USE_ZEND_ALLOC=0 valgrind ... gives the following hint:

==957== Invalid read of size 1
==957==    at 0x8A0E8C: UnknownInlinedFun (zend_string.h:329)
==957==    by 0x8A0E8C: free_ini_entry (zend_ini.c:78)
==957==    by 0x832E3C: zend_hash_destroy (zend_hash.c:1591)
==957==    by 0x8A13DC: zend_ini_dtor (zend_ini.c:111)
==957==    by 0x81D073: executor_globals_dtor (zend.c:808)
==957==    by 0x7A2846: ts_free_id (TSRM.c:541)
==957==    by 0x81F2CD: zend_shutdown (zend.c:1138)
==957==    by 0x7A6158: UnknownInlinedFun (main.c:2413)
==957==    by 0x7A6158: php_module_shutdown (main.c:2390)
==957==    by 0x64310D: main (php_cli.c:1382)
==957==  Address 0x65c12c4 is 4 bytes inside a block of size 48 free'd
==957==    at 0x48378E9: free (vg_replace_malloc.c:538)
==957==    by 0x832E3C: zend_hash_destroy (zend_hash.c:1591)
==957==    by 0x7A50C5: php_request_shutdown (main.c:1871)
==957==    by 0x9256ED: do_cli (php_cli.c:1135)
==957==    by 0x6430EB: main (php_cli.c:1367)
==957==  Block was alloc'd at
==957==    at 0x4836751: malloc (vg_replace_malloc.c:307)
==957==    by 0x7E4BFC: __zend_malloc (zend_alloc.c:3056)
==957==    by 0x8B7403: UnknownInlinedFun (zend_string.h:150)
==957==    by 0x8B7403: UnknownInlinedFun (zend_string.h:172)
==957==    by 0x8B7403: zend_string_init_interned_request (zend_string.c:313)
==957==    by 0x8A1751: zend_register_ini_entries (zend_ini.c:219)
==957==    by 0x6668860: ???
==957==    by 0x824399: UnknownInlinedFun (zend_API.c:2199)
==957==    by 0x824399: zend_startup_module_ex (zend_API.c:2152)
==957==    by 0x7314A7: php_load_extension (dl.c:225)
==957==    by 0x73167E: php_dl (dl.c:268)
==957==    by 0x731729: zif_dl (dl.c:61)
==957==    by 0x89928E: UnknownInlinedFun (zend_vm_execute.h:1297)
==957==    by 0x89928E: execute_ex (zend_vm_execute.h:55404)
==957==    by 0x89F85E: zend_execute (zend_vm_execute.h:59771)
==957==    by 0x81FB1A: zend_execute_scripts (zend.c:1792)

The extension in zif_dl is zlib.so

php_request_shutdown (main.c:1871) is this line: https://github.com/php/php-src/blob/php-8.1.5/main/main.c#L1871

The crash does not happen with GC disabled, or if zlib is loaded with -dextension=zlib

[arnaud-lb]

I will try to dig into this near Friday (unless someones does it first).

Updated Dockerfile with debuginfo and tools:

FROM mageia:8

RUN urpmi.addmedia BackportsTesting --update https://ftp-stud.hs-esslingen.de/pub/Mirrors/Mageia/distrib/8/x86_64/media/core/backports_testing/
RUN urpmi.addmedia BackportsTestingDebug --update https://ftp-stud.hs-esslingen.de/pub/Mirrors/Mageia/distrib/8/x86_64/media/debug/core/backports_testing/
RUN urpmi.addmedia CoreDebug https://ftp-stud.hs-esslingen.de/pub/Mirrors/Mageia/distrib/8/x86_64/media/debug/core/release/
RUN urpmi --auto php-cli php-filter php-pear wget gdb valgrind php-devel
RUN urpmi --auto $(rpm -qa | grep php-|grep -v debug|grep -v devel|grep -v php-ini|grep -v pear|cut -d- -f1,2 | while read p; do echo "$p-debuginfo"; done)
RUN urpmi --auto --replacepkgs locales
RUN cd /tmp
RUN wget http://pear.php.net/get/Calendar-0.5.5.tgz

[mokraemer]

I'm glad it is now reproduceable. I'm wondering why this only happens, if the file is given via command line and not, if it is downloaded by php itself. Strange - but some bugs are ;)

[arnaud-lb]

I dug into this bug again today: The problem happens because pear is loading the zlib extension dynamically with dl(). This causes the extension to allocate resources with a storage whose lifespan is shorter than the resources themselves. The extension creates interned strings for ini entries during a phase when truly permanent interned strings are read only, so the interned strings are created with a request scope, which is shorter than the ini entries in ZTS builds.

A workaround for this problem is to load the zlib extension earlier by using the extension directive.

Currently the manual for dl() has a note stating that the dl() function is currently not supported in ZTS builds.

[nikic]

@arnaud-lb I believe

php-src/Zend/zend_API.c

Line 2973 in 733023b

zend_unregister_ini_entries(module->module_number);

should be unregistering ini entries for temporary modules during request shudown. Any idea why that doesn't happen?

[arnaud-lb]

This removes entries from registered_zend_ini_directives, but ini entries for temporary modules are registered in EG(ini_entries).

So that would be the actual issue. A possible fix would be that temporary modules unregister from EG(ini_entries)

[mokraemer]

I've added the zlib extension to the list of modules at startup for pear. It works. But I leave this one open, in case you found sth. to worry about. Since this can happen in any other case, it might be worth try to omit this error.

[arnaud-lb]

Thank you for confirming this. A fix is ongoing at #8435

[arnaud-lb]

Fixed in #8435

Thank you @mokraemer

[HeenaBansal2009]

@nikic @arnaud-lb , is this fix available in php 7.4 ?
I didn't find anything in changeling for php 7.4?
I'll appreciate your response on this.

[arnaud-lb]

@HeenaBansal2009 this has been fixed in 8.1, but not in 7.4 because this version is not supported anymore (see https://www.php.net/supported-versions.php).

[HeenaBansal2009]

@HeenaBansal2009 this has been fixed in 8.1, but not in 7.4 because this version is not supported anymore (see https://www.php.net/supported-versions.php).

Thanks @arnaud-lb for quick response. Is this fix not available in 8.0 as well.? I updated my image to latest version 8..0.30 but still being this issue.Please confirm.

[arnaud-lb]

8.0 is not supported anymore either (except for security issues)