FS#66649 - [mkinitcpio] reconsider compression option

Attached to Project: Arch Linux
Opened by T.J. Townsend (blakkheim) - Thursday, 14 May 2020, 16:03 GMT
Last edited by Giancarlo Razzolini (grazzolini) - Wednesday, 17 February 2021, 12:06 GMT
Task Type Feature Request
Category Packages: Core
Status Closed
Assigned To Dave Reisner (falconindy)
Giancarlo Razzolini (grazzolini)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description:
The default /etc/mkinitcpio.conf uses a single-threaded gzip implementation when rebuilding the initramfs. This could be improved by doing one of two things:

1) Switching to a multithreaded xz compression. Even with the lowest amount of compression, xz creates a smaller file than gzip's mid-range default. It can also use all CPU cores, making (re-)generation of the file faster. The memory overhead when using -0 is quite small.

2) Switching to the "cat" method of using no compression. The generated file is very small: ~60MB currently. The compressed versions are ~21MB (gzip -6) and ~18MB (xz -0). All three are quite small by today's standards and we might not want to bother with compressing them at all. Copying 60MB is likely a near-instant operation, even on the worst SD cards.

As a proposal, I would change the default "COMPRESSION=" line to either "cat" or "xz"

If using xz, additionally defaulting to "COMPRESSION_OPTIONS=(-0 -T 0)" for speed.
This task depends upon

Closed by  Giancarlo Razzolini (grazzolini)
Wednesday, 17 February 2021, 12:06 GMT
Reason for closing:  Won't implement
Comment by Giancarlo Razzolini (grazzolini) - Thursday, 14 May 2020, 17:00 GMT
xz is not reproducible when using multithread, as far as I know. And using cat is a big no from me.
Comment by Giancarlo Razzolini (grazzolini) - Thursday, 14 May 2020, 17:19 GMT
If and when the kernel adds support for initrd compressed with zstd, I'll switch to zstd by default, most likely. zstd *is* reproducible, even on multithread.
Comment by Dave Reisner (falconindy) - Thursday, 14 May 2020, 18:50 GMT
We've stuck with gzip because historically it's been the best tradeoff between disk space and read/write speed. If you don't like it, I'd encourage you to the pick the compression algo that suits your purposes. We're not going to default to xz because of the non-reproducibility, as Giancarlo points out. zstd is a better default once it's available as an option.
Comment by T.J. Townsend (blakkheim) - Sunday, 18 October 2020, 20:03 GMT
Is it possible to re-open this discussion with 5.9 now?
Comment by loqs (loqs) - Sunday, 18 October 2020, 20:25 GMT
@mysta should the decision not be deferred until linux-lts supports zstd?
Comment by Giancarlo Razzolini (grazzolini) - Monday, 19 October 2020, 15:00 GMT
I'll add zstd support to mkinitcpio, but of course it won't work on -lts. It will be up to the user to figure this out, best we can do is add on the man page which kernel gets zstd.
Comment by T.J. Townsend (blakkheim) - Wednesday, 17 February 2021, 03:31 GMT
https://git.archlinux.org/mkinitcpio.git/commit/?id=2f4a2b592fb98aab1a0e4f3135d4592219b7c25c
Comment by Giancarlo Razzolini (grazzolini) - Wednesday, 17 February 2021, 12:06 GMT
I did some testing for this, it turns out xz is reproducible even with -T0, but zstd is faster both to compress and decompress. Also, xz -0 and zstd -3 (the default) both yield images smaller than gzip. Hence why zstd is the new default.

Loading...