FS#70767 - [releng] Distributed UEFI image's integrity is not properly verified

Attached to Project: Release Engineering
Opened by Spyros Seimenis (sespiros) - Sunday, 09 May 2021, 18:58 GMT
Last edited by David Runge (dvzrv) - Friday, 18 February 2022, 17:39 GMT
Task Type Bug Report
Category ArchISO
Status Closed
Assigned To David Runge (dvzrv)
Architecture All
Severity High
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

The distributed UEFI image doesn't properly verify the integrity of all the downloaded files as it is claimed in https://archlinux.org/releng/netboot/. While the bug is actually located in ipxe-netboot AUR package, I filed the bug here since I verified that it is being used for the image generation and directly affects the distributed ipxe.efi image.

ipxe-netboot uses the iPXE's chain command to download an installation script

https://aur.archlinux.org/cgit/aur.git/tree/arch.ipxe?h=ipxe-netboot
#!ipxe
ifconf
ntp pool.ntp.org
chain https://ipxe.archlinux.org/releng/netboot/archlinux.ipxe || shell

images' signature verification is indeed happening inside the downloaded installation script, using iPXE's imgverify which verifies the downloaded images and their signatures against the pem certificates included in the ipxe-netboot package.

However that script's integrity is not properly verified. Considering that integrity verification with PGP signatures protects users from scenarios under which the distribution servers get compromised, a malicious user could just modify the archlinux.ipxe installation script to remove the image verification checks and distribute tampered images.

I verified that this package is being used for the generation of the distributed image by running strings in the latest ipxe.efi image

$ strings ipxe.176077d7fccb.efi | grep -B2 chain
#!ipxe
ifconf
chain https://ipxe.archlinux.org/releng/netboot/archlinux.ipxe || shell
This task depends upon

Closed by  David Runge (dvzrv)
Friday, 18 February 2022, 17:39 GMT
Reason for closing:  Deferred
Additional comments about closing:  Closing in favor of https://gitlab.archlinux.org/archlinux/r eleng/-/issues/9 as the Release Engineering project on flyspray will be closed.
Comment by David Runge (dvzrv) - Sunday, 09 May 2021, 20:30 GMT
@sespiros: Thanks for the report!

I'm not 100% sure I understand what you mean though.

The efi file you are referring to is currently a hardcoded static asset of archweb [1].
The script that is being targeted by it, is defined in a template of archweb [2].

What do you mean by "that script's integrity is not properly verified"?
How would you envision an integrity check in the context of the IPXE environment, that is being run?

[1] https://github.com/archlinux/archweb/tree/master/sitestatic/netboot
[2] https://github.com/archlinux/archweb/blob/master/templates/releng/archlinux.ipxe
Comment by Spyros Seimenis (sespiros) - Monday, 10 May 2021, 01:58 GMT
@dvzrv: I am referring to the archlinux.ipxe (https://ipxe.archlinux.org/releng/netboot/archlinux.ipxe) installation script which is downloaded with the chain command. There is no signature verification happening for that file.

The generated ipxe image file's integrity is indeed verified properly as described in https://github.com/archlinux/archweb#updating-ipxe-image

However the text in https://archlinux.org/releng/netboot/ indicates that the integrity of all downloaded files is verified as well. That means that the actual image files that the ipxe image will download upon booting are verified and the code attempts to do it by the following steps:

a) 2 pem formatted certificates are included in the AUR package (that generates the original image) to be used as trusted root certificates as per https://ipxe.org/crypto (https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=ipxe-netboot#n57)

b) arch.ipxe (https://aur.archlinux.org/cgit/aur.git/tree/arch.ipxe?h=ipxe-netboot) downloads the archlinux.ipxe script without verifying its integrity.

c) Only then archlinux.ipxe (after booting and running in iPXE shell) uses imgverify in several places to verify the integrity of the images coming from the various mirrors against the trusted root certificates specified in step a.

Based on the above, the threat model seems to assume that the integrity of all images are verified as long as the PGP key is not compromised. If a malicious actor gains control of the archlinux.org servers or github repo that assumption is violated.

My original example in more detail would be a malicious actor who gains access to the archlinux.ipxe file (by either compromising archlinux.org or the github repo) without having access to a code signing key, would be able to modify the archlinux.ipxe to add mirrors of his own, remove verification checks etc. Any netboot image would then download that modified script without verifying its integrity and execute it. That would give the attacker arbitrary command execution without him having access to the private key that signs the archlinux releases.

To answer your second question, I would envision the integrity check in one of the following ways:
1) Have archlinux.ipxe inside the ipxe-netboot package instead of the arch.ipxe downloader script.
2) If archlinux.ipxe has to be part of the archweb (in order to frequently update it for example without rebuilding new images), a signature would have to be created and uploaded in archweb too. Then the arch.ipxe downloader script should be updated to something like the following (untested):

#!ipxe

...
# allow only trusted images
imgtrust

chain https://ipxe.archlinux.org/releng/netboot/archlinux.ipxe
imgverify archlinux.ipxe https://ipxe.archlinux.org/releng/netboot/archlinux.ipxe.sig || goto failed_verify
imgargs archlinux.ipxe || shell

...
Comment by David Runge (dvzrv) - Monday, 10 May 2021, 07:28 GMT
@sespiros: Thanks for the detailed example! Your initial report led me to believe you meant guarding the script with a PGP signature (which is what you obviously didn't mean! :D), which had me running in the wrong direction.

Indeed, the trust chain can be improved in this scenario.
As a matter of fact I'm currently preparing an ipxe package [1] that we can ship to Arch Linux users and also use as (non-static) artifacts for archweb for external users (the ipxe script is still the same currently).
Meanwhile archiso is gaining the feature of exporting (and optionally codesigning) the netboot artifacts directly [2].
This happens in the context of automating our releases [3].

I'll have to evaluate whether it is possible for us to provide a signed ipxe script to archweb (the current one is a template, due to all mirror and release specific information) by generating it in a pipeline alongside the IPXE related artifacts.

In case you are interested in helping out, feel free to reach out in #archlinux-releng on freenode.net

[1] https://gitlab.archlinux.org/dvzrv/pkgbuilds/-/tree/master/ipxe
[2] https://gitlab.archlinux.org/archlinux/archiso/-/merge_requests/170
[3] https://gitlab.archlinux.org/archlinux/releng/-/issues
Comment by David Runge (dvzrv) - Monday, 10 May 2021, 08:52 GMT
I have added https://gitlab.archlinux.org/archlinux/releng/-/issues/9 to investigate this, once we have all the pieces together to start working on the releng release pipelines.
Comment by Spyros Seimenis (sespiros) - Monday, 10 May 2021, 10:07 GMT
@dvzrv: Thanks for the fast response, the context/information regarding the whole process and the effort for producing these images. I'm glad to be of help!

Loading...