FS#25875 - [linux] change from CONFIG_KERNEL_XZ to CONFIG_KERNEL_GZIP

Attached to Project: Arch Linux
Opened by Tom Gundersen (tomegun) - Saturday, 03 September 2011, 11:35 GMT
Last edited by Tobias Powalowski (tpowa) - Wednesday, 05 October 2011, 08:00 GMT
Task Type Feature Request
Category Packages: Core
Status Closed
Assigned To Tobias Powalowski (tpowa)
Thomas Bächler (brain0)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

This was discussed in the past, but no firm conclusion was reached as far as I understand, so I'm putting it out here again.

The main concern when choosing the compression algorithm for the kernel image should be boot speed (imho). One could also consider the size of the image, but as the difference in size between the different algorithms is less than one megabyte (I don't have the exact number, but this is certainly an upper bound), I don't think it is worth considering.

The slower the hdd speed, the more a machine will benefit from a high compression ratio (as the time it takes to load the kernel will be disk read + decompression). However, even on the machine with the slowest disk I could get my hands on (~35MB/s buffered reads) using gzip is significantly faster than using xz.

As was pointed out to me by Thomas, the situation might be different on machines with buggy BIOS'es (where int13h disk access is extremely slow) or when using PXE boot. If the majority of BIOS'es are known to be buggy and it is known that XZ is the better choice in these cases, I guess it should stay as it is. However, if we don't know, then I don't think it makes sense to optimize for this case.
This task depends upon

Closed by  Tobias Powalowski (tpowa)
Wednesday, 05 October 2011, 08:00 GMT
Reason for closing:  Fixed
Additional comments about closing:  3.0.6-1
Comment by Rémy Oudompheng (remyoudompheng) - Saturday, 03 September 2011, 12:37 GMT
What is the boot time gain you are expecting from that change? My measurement with perf says that unxz on the xz part of the kernel uses 750M instructions, while gunzip would use 250M instructions. Even on a very old machine, I cannot expect that to make a great difference in boot time.
Comment by Tom Gundersen (tomegun) - Saturday, 03 September 2011, 14:24 GMT
When I said "significantly faster", I'm talking about the relative speed (three times faster sounds believable).

As to real time, my estimate is that the savings will be in the hundreds of milliseconds, but I don't know the best way to get the proper numbers. By analogy, I compared the initrd times and there I saved about about 150 ms when switching from xz to gzip. My initrd is about half the size of my kernel, so I guess a couple of hundred milliseconds will not be too far off the mark.

Obviously 200 ms (or whatever it is) will not be noticeable in isolation, but we need to start somewhere. The reason I'm suggesting the change is that gzip seems to be strictly better than xz (unless there is something I'm missing). Is there any reason to stick with xz?

I have been googling this to find out why people might want to choose the different options, but I have not found any compelling arguments either way that apply to us.
Comment by Gaetan Bisson (vesath) - Saturday, 03 September 2011, 21:18 GMT
I'm with Tom here: the three-fold increase in decompression time isn't worth the 1MB saved...
In fact, I would expect it even slows things down for PXE on LAN (10MB/s ==> 100ms transfer time saved).
My new laptop has a SSD and decompressing the kernel is visibly (a few hundred milliseconds) the slowest part of the post-bootloader startup process.
Comment by Vincent Cappe (vcap) - Sunday, 04 September 2011, 13:48 GMT
size measurements with a kernel prepared with linux-3.0.4-1 PKGBUILD, i686:

CONFIG_KERNEL_GZIP : 2882624
CONFIG_KERNEL_BZIP2 : 2932720
CONFIG_KERNEL_LZMA : 2341616
CONFIG_KERNEL_XZ : 2350128
CONFIG_KERNEL_LZO : 3337136

<---- script to run from the kernel sources root --->
#! /bin/bash
set -eu

comp_methods=( CONFIG_KERNEL_{GZIP,BZIP2,LZMA,XZ,LZO} )

# disable all
for comp_method in ${comp_methods[@]}; do
./scripts/config --disable $comp_method
done

# enable one at a time
for comp_method in ${comp_methods[@]}; do
./scripts/config --enable $comp_method
#diff -u ../../config .config | less

make bzImage >/dev/null 2>&1
size=$(stat -c %s ./arch/x86/boot/bzImage)
printf '%-20s:%8d\n' $comp_method $size

./scripts/config --disable $comp_method
done
Comment by Jan de Groot (JGC) - Monday, 05 September 2011, 19:02 GMT
The size difference isn't shocking. As for size vs speed difference: note that low-memory systems will have trouble with XZ compression. I've seen this on old Compaq Deskpro EN/SFF systems with only 64MB memory, they can't PXE boot when the initramfs is compressed with XZ, as there's not enough memory for that.
Comment by Rémy Oudompheng (remyoudompheng) - Tuesday, 06 September 2011, 05:19 GMT
Makes a good point to me.
Comment by Dan McGee (toofishes) - Thursday, 08 September 2011, 18:36 GMT
+1 for gzip by default- I'm not even sure where this 1MB difference number is coming from given the kernel image itself is only 2.3MB right now anyway. Vincent's numbers show a 520K difference in size (22% bigger). We can live with a 2.8MB kernel given our modules directory is 51MB.

XZ makes sense for our packages; not so much as a low-level efficient compression method for time/memory/space-saved, where gzip still seems to be the winner.
Comment by Tom Gundersen (tomegun) - Thursday, 08 September 2011, 21:30 GMT
@toofishes: the "1 megabyte" was my fault, I gave the upper bound: "the difference in size between the different algorithms is less than one megabyte". I didn't check carefully, as anything less than one megabyte is insignificant when all we are talking about is diskspace. Thanks to Vincent for providing the hard numbers.
Comment by Tobias Powalowski (tpowa) - Friday, 09 September 2011, 13:11 GMT
Fixed on svn.

Loading...