FS#13592 - Strip shared libraries by default

Attached to Project: Pacman
Opened by Bruno Tsubouchi Yporti (yportilog) - Saturday, 28 February 2009, 08:25 GMT
Last edited by Allan McRae (Allan) - Monday, 15 March 2010, 00:37 GMT
Task Type Feature Request
Category makepkg
Status Closed
Assigned To Allan McRae (Allan)
Architecture All
Severity Low
Priority Normal
Reported Version 3.2.2
Due in Version 3.4.0
Due Date Undecided
Percent Complete 100%
Votes 4
Private No

Details

Strip all symbols from shared libraries (.so) by default could be save a lot of disk space and maybe improve performance(?).

Is there any special reason to not remove all symbols of libraries? Several distribuitions, like Debian and Gentoo, uses this by defualt whithout problems.

The difference in size of libraries is quite significant in some cases:

4,4M /usr/lib/libgtk-x11-2.0.so.0.1400.7 (default)
3,6M /usr/lib/libgtk-x11-2.0.so.0.1400.7 (fully stripped)

Thanks.
This task depends upon

Closed by  Allan McRae (Allan)
Monday, 15 March 2010, 00:37 GMT
Reason for closing:  Implemented
Additional comments about closing:  http://projects.archlinux.org/pacman.git /commit/?id=5fe41df8a9eaac288433a54e216f 96b1fe729c01
Comment by Xavier (shining) - Saturday, 28 February 2009, 08:32 GMT
makepkg already does that :
http://projects.archlinux.org/?p=pacman.git;a=blob;f=scripts/makepkg.sh.in;h=52e80d17e296ab22ad1503ca3ae690d4d516e829;hb=HEAD#l830

What is the output of : file -biz /usr/lib/libgtk-x11-2.0.so.0.1400.7 ?
Comment by Bruno Tsubouchi Yporti (yportilog) - Saturday, 28 February 2009, 09:00 GMT
Xavier, the output is right. But makepkg uses strip -S. Using strip whithout parameters results in the difference above.

Look this:

/usr/lib/libgtk-x11-2.0.so.0.1400.7: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, not stripped (Default)
/usr/lib/libgtk-x11-2.0.so.0.1400.7: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, stripped (after strip)

Comment by Bruno Tsubouchi Yporti (yportilog) - Saturday, 28 February 2009, 09:08 GMT
Interesting results in other distributions:

Gentoo x86 stable

/usr/lib/libgtk-x11-2.0.so.0.1200.11: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, stripped

Debian 5.0 stable

libgtk-x11-2.0.so.0.1200.11: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, stripped
Comment by Xavier (shining) - Saturday, 28 February 2009, 14:44 GMT
I would like to know what we lose here. I asked google, I only found this link for now :
http://www.linuxfromscratch.org/pipermail/lfs-dev/2001-May/014565.html

So if we do change something, we should only do it on shared libs and not static libs.
Comment by Bruno Tsubouchi Yporti (yportilog) - Saturday, 28 February 2009, 15:08 GMT
I agree. I read the script for strip used in Gentoo and it only use strip in binaries (like archlinux) and in shared libs. Static libs use 'strip -g' (the same of strip -S of new makepkg).


Regards.
Comment by Allan McRae (Allan) - Sunday, 01 March 2009, 06:12 GMT
Coming from a LFS background, I have always used "--strip-all" on binaries and "--strip-debug" on libraries. I did not realise you could strip shared libs more. Has anyone run the additional striping on the .so in their whole system to see if all stays well?

References I found:
http://www.happy-monkey.net/pocket/ch03.html#d0e705 (not an authoritative source...)
"Using --strip-unneeded shrinks the file size, but leaves the symbols needed for relocation intact which is something that shared libraries need to function properly."
Comment by Bruno Tsubouchi Yporti (yportilog) - Sunday, 01 March 2009, 12:05 GMT
Allan, I built several systems (all core and some extra packages) with strip in shared libraries and works fine. I had problems with only strip all in static libs. All systems built with strip only in binaries and shared libraries works fine.

Like I said, several stable distributions use this strip mode in shared libraries without problems. I think this will not cause problems on Archlinux.

Regards.
Comment by Allan McRae (Allan) - Friday, 06 March 2009, 21:56 GMT
Gentoo (http://sources.gentoo.org/viewcvs.py/gentoo-src/portage/bin/prepstrip?rev=1.29&view=markup):

binaries: strip <file>
static libraries: strip -g <file>
shared libraries: strip --strip-unneeded <file> (by default)

The --strip-unneeded flag is not portable to BSD (http://www.gsp.com/cgi-bin/man.cgi?section=1&topic=strip) and probably not on OS X given that does not even have --strip-debug. So this would need to be done in configure.ac.

I am happy for this change to be made. Patches welcome.





Comment by Gerardo Exequiel Pozzi (djgera) - Sunday, 15 March 2009, 10:34 GMT
strip-all for binaries (but maybe more secure strip-unneeded, i don't know a program that break with strip-all)
strip-debug for static libraries (don't use strip-unneeded, will be destroy static libs when linking)
strip-unneeded for shared libraries

The only exception here is glibc, don't use strip-unneeded because can cause problems when use pthreads, but maybe this was a old historic problem when glibc uses LinuxThreads, now with NPTL don't know the status. I think that is safe to do only a strip-debug for glibc libs, and don't strip the dynamic linker/loader ld-linux.

But i personally prefer to don't strip glibc at all.

My two cents.
Comment by Allan McRae (Allan) - Monday, 16 March 2009, 07:50 GMT
Good point. Both gentoo and slackware manually strip glibc and only use strip-debug for its libraries, not strip unneeded, to stop this pthreads issue. We have an option to disable stripping allowing this to be done manually.

From the man and info pages, I can not actually figure out what level of stripping does "strip <file>" do? I would be happy to change this to strip-all for binaries. I used that in my LFS builds and everything was fine.
Comment by Gerardo Exequiel Pozzi (djgera) - Monday, 16 March 2009, 18:49 GMT
strip without arguments is --strip-all, i see it in the objcopy.c (strip and objcopy are the same program)
Comment by Allan McRae (Allan) - Monday, 16 March 2009, 22:10 GMT
OK then. You came to the same conclusions as me on how files should be stripped. I now maintain glibc so once this is implemented I can manually strip the files.
Comment by Robson Roberto Souza Peixoto (robsonpeixoto) - Monday, 13 April 2009, 10:33 GMT
My sugestion is:
I think the sprit-all can be the default. If the developer put something like this: options=(strip_carefully).

Normal strip:
cd $pkgdir
find . | xargs file | grep "executable" | grep ELF | cut -f 1 -d : | xargs strip --strip-unneeded 2> /dev/null
find . | xargs file | grep "shared object" | grep ELF | cut -f 1 -d : | xargs strip --strip-unneeded 2> /dev/null
cd -

strip_carefully:
cd $pkgdir
find . | xargs file | grep "executable" | grep ELF | cut -f 1 -d : | xargs strip --strip-unneeded 2> /dev/null
find . | xargs file | grep "shared object" | grep ELF | cut -f 1 -d : | xargs strip --strip-debug 2> /dev/null
cd -

ref:
ftp://ftp.slackware.com/pub/slackware/slackware-current/source/l/glibc/glibc.SlackBuild
ftp://ftp.slackware.com/pub/slackware/slackware-current/source/d/nasm/nasm.SlackBuild
ftp://ftp.slackware.com/pub/slackware/slackware-current/source/d/gcc/gcc.SlackBuild
Comment by xduugu (xduugu) - Friday, 06 November 2009, 14:52 GMT
What's the current status on that?
Comment by Allan McRae (Allan) - Friday, 06 November 2009, 15:05 GMT
This has not been implemented.

This is how I want it implemented:
Three new variables in makepkg.conf STRIP_BINARIES, STRIP_STATIC, STRIP_SHARED. Their default values should be "--strip-all", "--strip-debug", "--strip-unneeded" respectively.

In configure.ac, in the "Host-dependant definitions" section, for darwin these values should be set to "", "-S", "-S" (as currently in makepkg) as I do not think the long options work there. cygwin should work with the default values, as should BSD strip, but it is worth confirming this and adding overrides as necessary.
Comment by Gavin Bisesi (Daenyth) - Saturday, 21 November 2009, 18:14 GMT
I like Allan's suggestion above.
Comment by xduugu (xduugu) - Wednesday, 20 January 2010, 14:10 GMT

Loading...