Pacman

Historical bug tracker for the Pacman package manager.

The pacman bug tracker has moved to gitlab:
https://gitlab.archlinux.org/pacman/pacman/-/issues

This tracker remains open for interaction with historical bugs during the transition period. Any new bugs reports will be closed without further action.
Tasklist

FS#28487 - [makepkg] Please use x86 filter with .xz compression

Attached to Project: Pacman
Opened by Marti (intgr) - Friday, 17 February 2012, 14:13 GMT
Last edited by Allan McRae (Allan) - Sunday, 22 April 2012, 10:01 GMT
Task Type Feature Request
Category System
Status Closed
Assigned To No-one
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 2
Private No

Details

Description: The LZMA2 (.xz) compression format supports normalization filters for some data types, which improve compression efficiency. One such filter is the x86 branch-call-jump filter, which converts relative jump addresses to absolute addresses (both i386 and x86_64). This operation is designed to be always reversible, so there's no risk of compressing non-executable data with this flag.

Since most Arch packages contian executables, this translates into improved compression ratios. In the worst case -- no executable code -- it can increase the file size slightly, but that increase is usually very small.

All that's needed to enable this is to call xz with two extra arguments: --x86 --lzma2 (in this order)

I did a small test to compare the compression ratios and speed. Eclipse here is a worst-case package that contains no x86 code, which increased 0.01% in size. chromium decreased by 4.7%, firefox 2.9%, libreoffice-common by 6.3% and libreoffice-writer by 12.9% (!)

-rw-r--r-- 1 marti marti 25742360 Feb 17 14:11 chromium-17.0.963.56-1-x86_64.pkg.tar.xz
-rw-r--r-- 1 marti marti 24520560 Feb 17 14:11 chromium-17.0.963.56-1-x86_64.pkg.tar.xz.x86
-rw-r--r-- 1 marti marti 177386192 Feb 17 15:16 eclipse-3.7.1-2-x86_64.pkg.tar.xz
-rw-r--r-- 1 marti marti 177402980 Feb 17 15:16 eclipse-3.7.1-2-x86_64.pkg.tar.xz.x86
-rw-r--r-- 1 marti marti 13895816 Feb 17 14:11 firefox-10.0.1-1-x86_64.pkg.tar.xz
-rw-r--r-- 1 marti marti 13498660 Feb 17 14:11 firefox-10.0.1-1-x86_64.pkg.tar.xz.x86
-rw-r--r-- 1 marti marti 63470324 Feb 17 14:14 libreoffice-common-3.5.0-1-x86_64.pkg.tar.xz
-rw-r--r-- 1 marti marti 59469704 Feb 17 14:14 libreoffice-common-3.5.0-1-x86_64.pkg.tar.xz.x86
-rw-r--r-- 1 marti marti 1832272 Feb 17 14:14 libreoffice-writer-3.5.0-1-x86_64.pkg.tar.xz
-rw-r--r-- 1 marti marti 1596652 Feb 17 14:14 libreoffice-writer-3.5.0-1-x86_64.pkg.tar.xz.x86

The timings show a similar pattern. Packages with x86 code now compress faster, though the difference is less significant. In the worst case, Eclipse compresses 1.3% slower.

time xz --keep --x86 --lzma2 -S .xz.x86 chromium-17.0.963.56-1-x86_64.pkg.tar
52,03s user 0,20s system 99% cpu 52,301 total
time xz --keep --lzma2 -S .xz chromium-17.0.963.56-1-x86_64.pkg.tar
53,04s user 0,16s system 99% cpu 53,231 total
time xz --keep --x86 --lzma2 -S .xz.x86 firefox-10.0.1-1-x86_64.pkg.tar
20,39s user 0,12s system 99% cpu 20,603 total
time xz --keep --lzma2 -S .xz firefox-10.0.1-1-x86_64.pkg.tar
20,54s user 0,06s system 99% cpu 20,646 total
time xz --keep --x86 --lzma2 -S .xz.x86 libreoffice-common-3.5.0-1-x86_64.pkg.tar
113,03s user 0,30s system 99% cpu 1:53,75 total
time xz --keep --lzma2 -S .xz libreoffice-common-3.5.0-1-x86_64.pkg.tar
115,90s user 0,33s system 99% cpu 1:57,02 total
time xz --keep --x86 --lzma2 -S .xz.x86 libreoffice-writer-3.5.0-1-x86_64.pkg.tar
3,43s user 0,02s system 99% cpu 3,478 total
time xz --keep --lzma2 -S .xz libreoffice-writer-3.5.0-1-x86_64.pkg.tar
3,45s user 0,03s system 99% cpu 3,483 total
time xz --keep --x86 --lzma2 -S .xz.x86 eclipse-3.7.1-2-x86_64.pkg.tar
78,05s user 0,72s system 99% cpu 1:19,35 total
time xz --keep --lzma2 -S .xz eclipse-3.7.1-2-x86_64.pkg.tar
77,04s user 0,50s system 98% cpu 1:18,56 total
This task depends upon

Closed by  Allan McRae (Allan)
Sunday, 22 April 2012, 10:01 GMT
Reason for closing:  Won't implement
Additional comments about closing:  See second comment for explanation.
Comment by Pierre Schmitz (Pierre) - Friday, 17 February 2012, 14:31 GMT
In theory we know when a package might contain binaries by checking if the packages arch is "any" or not. But we need to evaluate if the benefit is worth the effort.
Comment by Allan McRae (Allan) - Sunday, 22 April 2012, 10:01 GMT
This is not being added as default to makepkg as pacman runs on many other architectures... In particular arm and ppc.

However, in 4.1.0, there is an option to override the compression command in makepkg.conf.

Loading...