Release Engineering

This project is intented for all release related issues (isos, installer, etc), under the umbrella of the ArchLinux Release Engineers

FS#63683 - [archiso] reproducible arch iso image

Attached to Project: Release Engineering
Opened by Jelle van der Waa (jelly) - Saturday, 07 September 2019, 12:08 GMT
Last edited by David Runge (dvzrv) - Wednesday, 04 November 2020, 11:48 GMT
Task Type Bug Report
Category ArchISO
Status Closed
Assigned To David Runge (dvzrv)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No


For the reproducible builds effort within Arch, we also want to make our iso reproducible. A bunch of patches have been posted on the mailing list. This ticket should provide an overview of the issues:

- mirrorlist is not reproducible
offending line:
curl -o ${work_dir}/x86_64/airootfs/etc/pacman.d/mirrorlist ''

fixed this by just copying the system mirrorlist for now; we should
probably just use pacman-mirrorlist

- install.txt generation is not stable over time
offending line:
lynx -dump -nolist '' >> ${work_dir}/x86_64/airootfs/root/install.txt

fixed this by removing it for now; can we just version it in archiso
monthly or something?

- various instances of `cp` which do not copy timestamps
think this has been fixed now by tweaking initramfs

- work/x86_64/airootfs/var/log/pacman.log
different timestamps inside the file itself; hack this out by deleting it
do we really need this in the release archiso?

- install dates etc; can we replace INSTALLDATE in package desc with a sed macro (just set all to e.g. 1, or the ISO date with 0 time)
--- releng/work/x86_64/airootfs/var/lib/pacman/local/licenses-20181104-1/desc 2019-09-04 15:01:15.244798335 +0100
+++ releng2/work/x86_64/airootfs/var/lib/pacman/local/licenses-20181104-1/desc 2019-09-04 15:04:59.893817680 +0100
@@ -20,7 +20,7 @@


- there are a few gzip commands with no -n
syslinux modalias.gz; fixed now

- ldconfig aux-cache
pretty sure we can just delete this, hackfixed by deleting for now

- mkinitcpio does not create reproducible initramfs
mkinitcpio: Add --reproducible flag #1
adds an additional dependency on cpio
for now I have a local pkg with --reproducible hardcoded on
this is because the mkinitcpio hooks trigger from linux.preset
where to change this?

- systemd journal catalog is not reproducible
journal: Make the output of --update-catalog deterministic #13482
for now I have a local pkg with a systemd-git build, seems to work

- java cacerts
the trust tool responds to SOURCE_DATE_EPOCH so just do a final run there

- efiboot.img
the fat filesystem has a partition ID, set that to 0
the timestamps within the fs need to be 1'd or SOURCE_DATE_EPOCH'd
This task depends upon

Closed by  David Runge (dvzrv)
Wednesday, 04 November 2020, 11:48 GMT
Reason for closing:  Deferred
Additional comments about closing:  Closing in favor of rchiso/-/issues/68
Comment by Daniel Edgecumbe (esotericnonsense) - Saturday, 07 September 2019, 12:13 GMT
FWIW you can check out a patchset that fixes some of these issues on the arch-releng ML

and I have a branch here

(esotericnonsense/20190906repro is the patchset, reproducible is a WIP branch)
Comment by Jelle van der Waa (jelly) - Saturday, 07 September 2019, 12:17 GMT
Some more notes about downloading things:

- mirrorlist, some people argue that running reflector once network connection has been established.

- wiki, as an alternative to downloading the install.txt file, maybe we can put arch-wiki-lite with a link? On the archiso. The downside is that arch-wiki-docs is big due to HTML and arch-wiki-lite isn't automatically generated we should take that up with keenerd.

- EFI images, these as far as I can see are always downloaded from master. Maybe we can actually build and package these images in our repo?
Comment by Daniel Edgecumbe (esotericnonsense) - Saturday, 07 September 2019, 18:12 GMT
Quick comment on EFI images:

I think that longer term it may make sense to build them, but in the short term we can probably link to a specific revision rather than master. That way the verification won't fail.
Comment by Jelle van der Waa (jelly) - Sunday, 09 February 2020, 19:14 GMT
Another requirement for reproducible arch iso is to keep a BUILDINFO file with the installed packages and versions otherwise it would be impossible to reproduce.
Comment by David Runge (dvzrv) - Monday, 03 August 2020, 15:23 GMT
Many of the problems have been fixed already. I will move the remaining ones as separate issues to our gitlab and eventually close this ticket.

@jelly: Thanks for researching all these things!

@esotericnonsense: We will open up the gitlab instance to the public soonish. Your contributions are very much welcome there! Thanks so much for looking into all of these issues already and sorry for not merging stuff earlier (we kinda moved things to gitlab and started working on other - more pressing - issues which turned out to change large amounts of the code base already).