Arch Linux

Please read this before reporting a bug:

Do NOT report bugs when a package is just outdated, or it is in the AUR. Use the 'flag out of date' link on the package page, or the Mailing List.

REPEAT: Do NOT report bugs for outdated packages!

FS#43407 - [devtools] Reproducible builds

Attached to Project: Arch Linux
Opened by Pam Noell (pamnoell) - Sunday, 11 January 2015, 10:46 GMT
Last edited by Eli Schwartz (eschwartz) - Wednesday, 03 January 2018, 18:14 GMT
Task Type Feature Request
Category Packages: Extra
Status Closed
Assigned To Eli Schwartz (eschwartz)
Architecture All
Severity Low
Priority Normal
Reported Version 4.2.0
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 14
Private No


No amount of source code eyeballing can protect from a Trusting Trust attack on the build tools. If the build computer of an Arch Linux developer gets compromised, it would be very hard for the community to detect a backdoor in the binaries built and signed by that developer. Thus to target almost all Arch Linux users it is enough to successfully attack just one of the package maintainers of the core repository. The attack might not be trivial to pull off but the payoff would be enormous.

Have Arch Linux developers considered how to eliminate these single points of failure in the security of the distribution?

Some free software organizations, such as Tor, Debian and Red Hat, have responded to this threat by utilizing reproducible builds. The idea is to control the build process to such extent that no matter which computer builds the source, the binaries produced are identical.

On 2014-12-27 Mike Perry and Seth Schoen gave a talk on this topic at 31C3: . The Debian wiki has info on the Debian implementation: . Red Hat blogged about their ideas in 2013: .

I wasn't sure whether to file this request for pacman or release engineering as it touches on both.
This task depends upon

Closed by  Eli Schwartz (eschwartz)
Wednesday, 03 January 2018, 18:14 GMT
Reason for closing:  Fixed
Additional comments about closing: ommit/?id=eab5aba9b027a7689acaf2382a04ff 69b5b8771e
Comment by Allan McRae (Allan) - Sunday, 11 January 2015, 11:07 GMT
I think this is an interesting idea - maybe slightly harder on a rolling release distribution. These are steps I think we need:

1) create a list of packages installed in the buildchroot when the package was installed. The can be done in devtools. The a user can (at least theoretically) recreate the chroot, although this is difficult in Arch as we do not store old packages long. Ideally the file with the chroot state would be supplied alongside our package in the repos.

2) Identify a list of packages that are not reproducibly buildable and locate the cause. Does the package need fixed or makepkg? This would make a good community project. Just build a package twice using a fresh copy of the same chroot (devtools can do this) and run a diff. The .PKGINFO file in the package records the build time, but that difference is fine.

We have had some great success with community projects lately, so I look forward to someone taking on this one.

Comment by Doug Newgard (Scimmia) - Sunday, 11 January 2015, 16:13 GMT
Allan, for #1, there is another feature request for this as well:

For #2, we already have someone building packages over and over (, maybe talk to him about doing diffs?
Comment by ilf (ilf) - Saturday, 16 May 2015, 17:01 GMT
I think everyone who watched the talk wants to do this. The Debian people are offering help and support. They invite intetested people to come by #debian-reproducible on and ask questions. There might also be resources to to arch builds on also sais:

> Join #debian-reproducible on OFTC or send us an email to get support for making sure your packages build reproducibly too. Also, we care about free software in general, so if you are an upstream developer or working on another distribution, we'd love to hear from you!
Comment by Benny (mail2benny) - Wednesday, 16 September 2015, 11:28 GMT Comment by François Guerraz (kubrick) - Saturday, 26 March 2016, 17:51 GMT

This is going to seem a little bit off-topic but it's not:
One of my main problems with Arch is the lack of debug packages in the repositories. When a program crashes, most of the time it's hard to reproduce and you end up with a core dump that may very well contain some very useful debugging information, but it's lost forever as the debug symbols were stripped.
I have found many places where Arch users have asked for debug packages to be distributed but this has always been rejected because of the size of the mirrors, bandwidth, databases, and all sorts of apparently insurmountable problems.

So I was wondering, wouldn't that be the solution? If we builds are reproducible and we have a crash, couldn't we rebuild the package, extract and install the symbols in /var/lib/debug and then we can get a backtrace from this precious core dump?

Unless the idea is flawed (hit me!) I think that would be an elegant solution to this long-standing problem.
Comment by Eli Schwartz (eschwartz) - Wednesday, 03 January 2018, 18:13 GMT
Implemented in pacman-git via  FS#50086 

Devtools now supports automatically triggering this via