FS#27736 - [glibc] 2.15-1 causes apache-2.2.21-1 to segfault

Attached to Project: Arch Linux
Opened by Slavi Pantaleev (s.pantaleev) - Tuesday, 27 December 2011, 15:35 GMT
Last edited by Allan McRae (Allan) - Friday, 30 December 2011, 20:25 GMT
Task Type Bug Report
Category Packages: Testing
Status Closed
Assigned To Allan McRae (Allan)
Architecture x86_64
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 7
Private No

Details

Description:
httpd segfaults every time it tries to start, ever since glibc was upgraded from 2.14 to 2.15.

Rebuilding apache while running glibc 2.15 has no effect.
Downgrading to glibc-2.14.1-4 fixes the problem.
This task depends upon

Closed by  Allan McRae (Allan)
Friday, 30 December 2011, 20:25 GMT
Reason for closing:  Fixed
Additional comments about closing:  glibc-2.15-3
Comment by Ionut Biru (wonder) - Tuesday, 27 December 2011, 15:49 GMT
that's simple to debug, compared with nvidia.

can you rebuild apache with debug symbols and get a proper debug stack?

https://wiki.archlinux.org/index.php/Debug_-_Getting_Traces#One_package_settings_only
Comment by Slavi Pantaleev (s.pantaleev) - Tuesday, 27 December 2011, 16:34 GMT
Does that help?

#0 0x0000000000005446 in ?? ()
#1 0x00007fffebddb095 in floor () from /lib/libm.so.6
#2 0x00007ffff7de805c in _dl_relocate_object () from /lib/ld-linux-x86-64.so.2
#3 0x00007ffff7dee526 in dl_open_worker () from /lib/ld-linux-x86-64.so.2
#4 0x00007ffff7dea286 in _dl_catch_error () from /lib/ld-linux-x86-64.so.2
#5 0x00007ffff7dede6a in _dl_open () from /lib/ld-linux-x86-64.so.2
#6 0x00007ffff66fef26 in ?? () from /lib/libdl.so.2
#7 0x00007ffff7dea286 in _dl_catch_error () from /lib/ld-linux-x86-64.so.2
#8 0x00007ffff66ff4cf in ?? () from /lib/libdl.so.2
#9 0x00007ffff66fefc1 in dlopen () from /lib/libdl.so.2
#10 0x00007ffff7742e5b in apr_dso_load () from /usr/lib/libapr-1.so.0
#11 0x0000000000446261 in load_module (cmd=0x7fffffffde30, dummy=<optimized out>, modname=<optimized out>, filename=<optimized out>)
at /home/user/Desktop/apache/src/httpd-2.2.21/modules/mappers/mod_so.c:238
#12 0x0000000000435669 in invoke_cmd (cmd=0x452c20, parms=0x7fffffffde30, mconfig=0x7fffffffdd88, args=0x6a5331 "")
at /home/user/Desktop/apache/src/httpd-2.2.21/server/config.c:800
#13 0x0000000000436e26 in execute_now (parent=<optimized out>, sub_tree=0x7fffffffdd88, ptemp=<optimized out>, p=<optimized out>, parms=0x7fffffffde30,
args=0x6a5313 "php5_module modules/libphp5.so", cmd_line=0x6e5d70 "LoadModule") at /home/user/Desktop/apache/src/httpd-2.2.21/server/config.c:1441
#14 ap_build_config_sub (p=<optimized out>, temp_pool=<optimized out>, l=<optimized out>, parms=0x7fffffffde30, current=0x7fffffffdde0, curr_parent=0x7fffffffdde8,
conftree=0x660810) at /home/user/Desktop/apache/src/httpd-2.2.21/server/config.c:1012
#15 0x0000000000437367 in ap_build_config (parms=0x7fffffffde30, p=0x66a138, temp_pool=0x69e2d8, conftree=0x660810)
at /home/user/Desktop/apache/src/httpd-2.2.21/server/config.c:1224
#16 0x0000000000437875 in process_resource_config_nofnmatch (s=0x6943a8, fname=0x695d38 "/etc/httpd/conf/httpd.conf", conftree=0x660810, p=0x66a138, ptemp=0x69e2d8, depth=0)
at /home/user/Desktop/apache/src/httpd-2.2.21/server/config.c:1634
#17 0x0000000000437cce in ap_process_resource_config (s=0x6943a8, fname=0x695d38 "/etc/httpd/conf/httpd.conf", conftree=0x660810, p=0x66a138, ptemp=0x69e2d8)
at /home/user/Desktop/apache/src/httpd-2.2.21/server/config.c:1666
#18 0x0000000000438965 in ap_read_config (process=<optimized out>, ptemp=0x69e2d8, filename=0x4496b4 "conf/httpd.conf", conftree=0x660810)
at /home/user/Desktop/apache/src/httpd-2.2.21/server/config.c:2026
#19 0x000000000042398a in main (argc=1, argv=0x7fffffffe378) at /home/user/Desktop/apache/src/httpd-2.2.21/server/main.c:632
Comment by Allan McRae (Allan) - Tuesday, 27 December 2011, 21:48 GMT
It looks like the crash is caused when loading some module. Can you try limiting it down to a particular module?
Comment by Slavi Pantaleev (s.pantaleev) - Tuesday, 27 December 2011, 23:45 GMT
Commenting `LoadModule php5_module modules/libphp5.so` in httpd.conf allows it to start.
I've tried letting the php module to load with some of its extensions commented - it didn't make a difference.

Rebuilding php with debug symbols while running glibc 2.15 also didn't fix the problem and didn't show anything more in gdb. The freshly built php module is unusable when downgrading to glibc 2.14 and refuses to load in apache with the following error message:
Cannot load /etc/httpd/modules/libphp5.so into server: /lib/libc.so.6: version `GLIBC_2.15' not found (required by /etc/httpd/modules/libphp5.so)

Another problem I see with glibc 2.15 is that the quakelive plugin for firefox freezes when trying to enter a game.
Comment by Allan McRae (Allan) - Wednesday, 28 December 2011, 08:11 GMT
Can you confirm this is still an issue with glibc-2.15-2?
Comment by Allan McRae (Allan) - Wednesday, 28 December 2011, 09:46 GMT
Also, can you tell me what processor the affected computer has?
Comment by Slavi Pantaleev (s.pantaleev) - Wednesday, 28 December 2011, 09:55 GMT
The problem continues on glibc-2.15-2.
The CPU is: Intel(R) Core(TM)2 Duo CPU T7500
Comment by Jan de Groot (JGC) - Wednesday, 28 December 2011, 15:00 GMT
Please recompile apr with debug symbols. Note that I added the testsuite in SVN trunk (not in repo), so you might trigger a crash there too.

So far I haven't been able to reproduce this issue on two Core2duo systems (E8400 and T6570).
Comment by Slavi Pantaleev (s.pantaleev) - Wednesday, 28 December 2011, 15:33 GMT
I'm now using glibc-2.15.2, apache and apr custom-built with debug symbols and vanilla php from [extra].
The problem is still there with nothing new in the trace.

All tests from the apr testsuite passed.
Comment by Allan McRae (Allan) - Wednesday, 28 December 2011, 20:58 GMT
This is a glibc issue in the x86_64 optimizations of floor and ceil. I have the commit that causes this isolated and am trying to fix...
Comment by Allan McRae (Allan) - Wednesday, 28 December 2011, 23:03 GMT Comment by Vladimir (_v_l) - Thursday, 29 December 2011, 00:27 GMT
[subversion task was marked as duplication of this one]

Allan McRae (Allan):
> Please test this package:
Same here:
$ svn --version
[segfault]

System: Archlinux x86_64, Intel Core i5-2410M.

I checked the difference between gdb output from 2.15-2 and 2.15-2.3 (diff -Naur), except the addresses the fail function now is sin. I attach new gdb output.

P.S. I don't understand how glibc is working but previously 'ceil' failed now it is 'sin' so something with math :).

P.P.S. I also tried to compile subversion from source (abs then makepkg in copy of subversion/ but before I switch off apr in configuration because I don't have and don't want to install apache) with glibc-2.15-1 and it failed as from repo.
Comment by Victor Hooi (victorhooi) - Thursday, 29 December 2011, 00:53 GMT
Hi,

I can confirm that I also get a segfault with Subversion 1.7.2-1 (as per https://bugs.archlinux.org/task/27743).

My system:

[victorhooi@myhost djangoProjects]$ uname -a
Linux myhost 3.1.6-1-ARCH #1 SMP PREEMPT Thu Dec 22 09:11:48 CET 2011 x86_64 Intel(R) Xeon(R) CPU 5130 @ 2.00GHz GenuineIntel GNU/Linux

Cheers,
Victor
Comment by Allan McRae (Allan) - Thursday, 29 December 2011, 06:33 GMT Comment by Vladimir (_v_l) - Thursday, 29 December 2011, 08:13 GMT
It even worse!
After installing 2.15-2.4 gdb segfaults as well as svn (hopefully I can install 2.15-2, pacman is not segfault).

Nevertheless, I run both gdb and svn with valgrind (glibc 2.15-2.4)
$ valgrind svn --version > svn-version-valgrind.log
$ valgrind gdb --version > gdb-version-valgrind.log

Also I made report from valgrind of 'svn --version' for 2.15-2.

All reports from valgrind are attached.
Comment by Slavi Pantaleev (s.pantaleev) - Thursday, 29 December 2011, 08:24 GMT
So far I haven't been able to get subversion to crash for me with any of the glibc packages.

My apache problem seems to be resolved by glibc-2.15-2.3.
Both apache and gdb segfault when using the newer glibc-2-15-2.4
Comment by Allan McRae (Allan) - Thursday, 29 December 2011, 08:44 GMT
Thanks for letting me know about -2.3 fixing apache... at least I know how to fix that then.

I now have access to a computer showing this bug so I should (hopefully) be able to track it down now
Comment by Vladimir (_v_l) - Thursday, 29 December 2011, 08:58 GMT
2Slavi Pantaleev (s.pantaleev):
May be I did it in wrong way?

I usually run urxvt[^1] and tmux[^1] in it.

I did:
1. sudo pacman -Su && sudo pacman -Syy
2. svn --version
-> segfault
3. sudo pacman -U [2.15-2.3]
4. svn --version
-> segfault
in the same tmux window and session.

^1 Both packages I compile on my system with customized PKGBUILDs.
Comment by Slavi Pantaleev (s.pantaleev) - Thursday, 29 December 2011, 09:03 GMT
I'm not using urxvt or tmux, neither have I built subversion manually - I installed it from [extra].
Comment by Allan McRae (Allan) - Friday, 30 December 2011, 08:19 GMT
Should be "fixed" in glibc-2.15-3...
Comment by Slavi Pantaleev (s.pantaleev) - Friday, 30 December 2011, 09:06 GMT
It seems to be - works fine for me.
Comment by Vladimir (_v_l) - Friday, 30 December 2011, 10:48 GMT
Subversion works with glibc-2.15-3.

Loading...