Arch Linux

Please read this before reporting a bug:
https://wiki.archlinux.org/title/Bug_reporting_guidelines

Do NOT report bugs when a package is just outdated, or it is in the AUR. Use the 'flag out of date' link on the package page, or the Mailing List.

REPEAT: Do NOT report bugs for outdated packages!
Tasklist

FS#19182 - [sed] unexpected behaviour

Attached to Project: Arch Linux
Opened by Martin Sandsmark (sandsmark) - Tuesday, 20 April 2010, 15:14 GMT
Last edited by Allan McRae (Allan) - Friday, 21 May 2010, 12:04 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Allan McRae (Allan)
Architecture x86_64
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Confirmed on two separate machines running ArchLinux x86_64, the following command leads to unexpected behaviour in sed:
echo -en "libgestures_plugin.la \\ \nlibnetsync_plugin.la \\ \nlibhotkeys_plugin.la\n" | sed -n -e 's/^.*lib\([^ ]*\)_plugin\.la.*/\1/p'

It leads to sed trying to allocate 18446744073709551611 bytes of memory.

Rebuilding sed with -O0 (after a suggestion from the sed maintainer) seemingly fixed it, so I guess it is a bug in GCC.

The bug is kind of unfortunate, as the bootstrap script in VLC triggers it.

And a backtrace:
Here's a backtrace:
#0 ck_realloc (ptr=0x640fc0, size=18446744073709551611) at utils.c:468
#1 0x0000000000405451 in resize_line (lb=0x60fa00, len=18446744073709551611) at execute.c:232
#2 0x0000000000405573 in str_append (to=0x60fa00, string=0x6401b4 "hotkeys_plugin.laa \\\\c unimotion.h", length=18446744073709551611) at execute.c:247
#3 0x0000000000405cc0 in str_append_modified (string=0x6401b4 "hotkeys_plugin.laa \\\\c unimotion.h", length=18446744073709551611, type=REPL_ASIS, to=<value optimized out>) at execute.c:331
#4 0x000000000040687a in append_replacement (vec=<value optimized out>, input=<value optimized out>) at execute.c:1162
#5 do_subst (vec=<value optimized out>, input=<value optimized out>) at execute.c:1240
#6 execute_program (vec=<value optimized out>, input=<value optimized out>) at execute.c:1624
#7 0x0000000000407574 in process_files (the_program=0x619030, argv=<value optimized out>) at execute.c:1855
#8 0x00000000004026dc in main (argc=6, argv=0x7fffffffe338) at sed.c:349
This task depends upon

Closed by  Allan McRae (Allan)
Friday, 21 May 2010, 12:04 GMT
Reason for closing:  Works for me
Comment by Martin Sandsmark (sandsmark) - Tuesday, 20 April 2010, 15:51 GMT
It seems like the re_search() call in sed/regexp.c:252 (or therearound) sets regarray->end[1] to -1, for some reason or another.
Comment by Martin Sandsmark (sandsmark) - Tuesday, 20 April 2010, 16:26 GMT
So it should probably be reassigned to glibc, fwiw.
Comment by Allan McRae (Allan) - Wednesday, 21 April 2010, 00:18 GMT
What version of sed are you using? I can not replicate with 4.2.1-2
Comment by Martin Sandsmark (sandsmark) - Wednesday, 21 April 2010, 13:51 GMT
glibc 2.11.1-2 and sed 4.2.1-2.
Comment by Martin Sandsmark (sandsmark) - Wednesday, 21 April 2010, 13:53 GMT
The stuff about recompiling with -O0 seems to be wrong, though.
But now I can reproduce it consistently.

Are you on x86_64?
Comment by Allan McRae (Allan) - Wednesday, 21 April 2010, 14:06 GMT
Tested using an x86_64 chroot on i686 system (with x86_64 kernel). That should be able to replicate...

What do you mean that you can consistently reproduce it? The expression in the bug report always crashes, or sed regularly crashes on any expression?
Comment by Martin Sandsmark (sandsmark) - Wednesday, 21 April 2010, 14:11 GMT
«echo -en "libgestures_plugin.la \\ \nlibnetsync_plugin.la \\ \nlibhotkeys_plugin.la\n" | sed -n -e 's/^.*lib\([^ ]*\)_plugin\.la.*/\1/p'» generates the following output every time I run it:

gestures
netsync
sed: couldn't re-allocate memory
Comment by Allan McRae (Allan) - Wednesday, 21 April 2010, 23:52 GMT
I have just ask 4 other people to test this out on their machines and there were no issues...

Try removing the sed and glibc packages from your pacman cache and redownloading and installing them.
Comment by Martin Sandsmark (sandsmark) - Thursday, 22 April 2010, 00:24 GMT
How much memory was available on these machines? And were all of them x86_64?

Cleaning my pacman cache and re-downloading/installing sed and glibc didn't work (no surprise, the chance of an accidental checksum collision in a package is about the same that all the oxygen molecules in the room clump up in a corner so I suffocate...:-D).
Comment by Allan McRae (Allan) - Thursday, 22 April 2010, 00:33 GMT
All of them were x86_64 and none of them had the 16EB of ram that your error message indicates it is trying to allocate!

As far as I can see, this is specific to your (two!) machines. Are these both running stock Arch packages or have you rebuilt some? When did these errors start and can you correlate that with any particular update?
Comment by Martin Sandsmark (sandsmark) - Thursday, 22 April 2010, 13:12 GMT
I only discovered the behaviour when trying to build VLC from git, and the build script failed.

The reason I asked about the amount of memory is that both the machines I've been able to reproduce it on have 4GB of memory, while I've been unable to reproduce it on my file server with only 512MB (which is also x86_64).

I only use stock arch packages, except for a couple of applications from AUR. I can get you a list of AUR packages that are installed on both machines.
Comment by Martin Sandsmark (sandsmark) - Monday, 26 April 2010, 01:21 GMT
I'm sorry it took so long, I completely forgot about it, but here's the output of `pacman -Qm`:
aspell-nb 0.50.1-2
bin32-wine 1.1.43-1
dropbox 0.7.110-2
ksshaskpass 0.5.3-1
lib32-flac 1.2.1-2
lib32-jack 0.118.0-2
lib32-lcms 1.18-3
lib32-libldap 2.4.21-2
lib32-libogg 1.1.4-1
lib32-libsasl 2.1.23-3
lib32-libsndfile 1.0.21-1
lib32-libvorbis 1.3.1-1
packer 20100420-1
ttf-droid 20103103-1
virtualbox_bin 3.1.6-1
Comment by Allan McRae (Allan) - Monday, 26 April 2010, 01:36 GMT
None of these will cause the issue.

I have now asked dozens of people to try and replicate and none have. Until someone else has issues, the only conclusion is that something is wrong with your system. Although I have no idea what...
Comment by Martin Sandsmark (sandsmark) - Monday, 26 April 2010, 17:50 GMT
I now managed to reproduce it on a third machine, my old laptop with only 2GB of RAM.

Trying to track down this bug inside the glibc source tree is not something that sounds like a fun way to spend an evening, though. :/
Comment by Allan McRae (Allan) - Monday, 26 April 2010, 22:26 GMT
I'm finding this really, really, really weird. You can replicate on three computers but no-one else I have asked can... Something is going on here but I am unsure what.
Comment by Martin Sandsmark (sandsmark) - Wednesday, 05 May 2010, 20:23 GMT
I don't know if it is related, but when trying to turn this into a test-case, I only managed to make it segfault (works fine on the debian login server I tested on, though, seems arch-specific):
http://sprunge.us/ABXc?c

#0 re_compile_internal (preg=0x7fffffffe180, pattern=0x4007c5 "^.*lib\\([^ ]*\\)_plugin\\.la.*", length=28, syntax=0) at regcomp.c:762
#1 0x00007ffff7b3943d in __re_compile_pattern (pattern=<value optimized out>, length=<value optimized out>, bufp=<value optimized out>) at regcomp.c:229
#2 0x0000000000400621 in main (argc=1, argv=0x7fffffffe2f8) at test.c:19
Comment by Martin Sandsmark (sandsmark) - Wednesday, 05 May 2010, 20:52 GMT
never mind, I just forgot to initialize the right stuff in `pattern`...
Comment by Allan McRae (Allan) - Friday, 21 May 2010, 12:03 GMT
Been a month with no replicate. Closing.

Loading...