Release Engineering

This project is intented for all release related issues (isos, installer, etc), under the umbrella of the ArchLinux Release Engineers
Tasklist

FS#50188 - [archiso] PXE booting from EFI fails

Attached to Project: Release Engineering
Opened by Peter Wu (Lekensteyn) - Thursday, 28 July 2016, 18:10 GMT
Last edited by Gerardo Exequiel Pozzi (djgera) - Wednesday, 20 March 2019, 14:42 GMT
Task Type Feature Request
Category ArchISO
Status Assigned   Reopened
Assigned To Gerardo Exequiel Pozzi (djgera)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 0%
Votes 9
Private No

Details

Description:
The current lpxelinux.0 and .c32 files from syslinux only work for BIOS systems[1] and fails to boot a UEFI system.
Booting efistub (arch/boot/x86_64/vmlinuz) directly fails because the initrd cannot be located.
Main problem is that the EFI version of syslinux and the related ldlinux and .c32 modules are missing.

Had to do some tricks to get a working boot:
cp -a /usr/lib/syslinux/efi64 boot/syslinux/efi64 && cd $_
# make sure that ldlinux.e64 is loaded at the first attempt from arch/ (see below)
mv ldlinux.e64 ../../../
# Clone cfg files to load .c32 files from efi64/ instead of loading 32-bit .c32 for BIOS
for i in ../*.cfg; do sed 's,/syslinux/,&efi64/,' "$i" > "${i#*/}"; done
sed '1i PATH boot/syslinux/efi64/' archiso.cfg -i
# ensure that splash file and other files are reachable
ln -s ../{hdt/,memdisk,splash.png} ./

For some reason, if ldlinux.e64 is not the first loaded file (e.g. you put it in arch/boot/syslinux/) no further files are being retrieved:
---8<---
sent /tmp/archiso/arch/boot/syslinux/efi64/syslinux.efi to 192.168.2.61
file /tmp/archiso/arch/ldlinux.e64 not found
file /tmp/archiso/arch//ldlinux.e64 not found
file /tmp/archiso/arch//boot/isolinux/ldlinux.e64 not found
file /tmp/archiso/arch//isolinux/ldlinux.e64 not found
file /tmp/archiso/arch//boot/syslinuxldlinux.e64 not found
sent /tmp/archiso/arch//boot/syslinux/ldlinux.e64 to 192.168.2.61
error 0 No error, file close received from 192.168.2.61
failed sending /tmp/archiso/arch//boot/syslinux/ldlinux.e64 to 192.168.2.61
---8<---
putting ldlinux.e64 in arch/ successfully continues to the bootloader.

Tested on an ASUSTeK COMPUTER INC. X550ZE/X550ZE, BIOS 206 09/02/2015
Same configuration (but additionally using pxelinux.0 instead of lpxelinux.0) also boots an older Clevo B7130 (not using EFI).

dnsmasq.conf
---8<---
interface=br0
bind-interfaces

# DHCP
dhcp-range=192.168.2.60,192.168.2.64,12h

# For EFI (7 = ia64, 6 = ia32)
dhcp-match=set:efi-ia64,option:client-arch,7

#dhcp-boot=/arch/boot/syslinux/lpxelinux.0
# HACK because lpxelinux.0 cannot seem to load ldlinux.c32
dhcp-boot=/arch/boot/syslinux/pxelinux.0
dhcp-option-force=209,boot/syslinux/archiso.cfg
dhcp-option-force=210,/arch/
dhcp-option-force=66,192.168.2.1
enable-tftp
tftp-root=/tmp/archiso

dhcp-boot=tag:efi-ia64,/arch/boot/syslinux/efi64/syslinux.efi
dhcp-option-force=tag:efi-ia64,209,boot/syslinux/efi64/archiso.cfg
---8<---


Package versions:
* syslinux 6.03-6
* remastered archlinux-2016.07.01-dual.iso

[1]: http://www.syslinux.org/wiki/index.php?title=PXELINUX#UEFI
This task depends upon

Comment by Gerardo Exequiel Pozzi (djgera) - Thursday, 28 July 2016, 21:18 GMT
  • Field changed: Attached to Project (Arch Linux → Release Engineering)
Not a bug. Sure, PXE EFI is not currently supported. I am glad to merge a patch if you provide it. Thanks.
Comment by Andrew Shark (ashark) - Wednesday, 20 March 2019, 14:42 GMT
Actually not solved.
Comment by Gerardo Exequiel Pozzi (djgera) - Wednesday, 20 March 2019, 14:44 GMT
Looks like now syslinux.efi can also be used to work via PXE-EFI. Just use it instead of lpxelinux.0
Comment by Andrew Shark (ashark) - Wednesday, 20 March 2019, 17:26 GMT
So the change we are going to make goes to https://git.archlinux.org/archiso.git/tree/configs/releng/build.sh
I have several questions:
What are considerations while choosing one or another bootloader for arch iso?
I have found out that grub is also able to be booted via pxe.
systemd-boot currently is not bootable via pxe, but could be added by developers (see https://github.com/systemd/systemd/issues/11850).
So what should be "ideal" selection of bootloaders?

For now, I can suggest to add syslinux uefi pxe bootloader, but then should we replace systemd-boot with it?

This page explains how to handle multi-firmware installation of syslinux: https://wiki.syslinux.org/wiki/index.php?title=PXELINUX-Multi-Arch
As we need to use c32 modules, we cannot use Common config no COM32. Also, because this is going to iso file system, we cannot use symlinks there (we do not use extensions for this?), so we cannot use Distinct directory symlink path and Distinct directory symlink file.

So we have to choose between Common directory distinct config and Distinct directory common kernel path. I like Distinct directory common kernel path. What is your opinion?
Comment by Gerardo Exequiel Pozzi (djgera) - Wednesday, 20 March 2019, 22:31 GMT
1) archiso uses syslinux, neither systemd-boot or grub.
2) the only change in build.sh is to copy syslinux.efi to syslinux dir and adjust the README.altbootmethods, to add note about that.
Comment by Andrew Shark (ashark) - Wednesday, 20 March 2019, 22:32 GMT
> 1) archiso uses syslinux, neither systemd-boot or grub.
No, I mean in uefi mode. It uses systemd-boot
> 2) no changes are needed in build.sh, just the README..altbootmethods.
I mean pathes where to install syslinux files and cfg files of course needs to be changed.
I want to make a separate directories isoroot/arch/boot/syslinux/{bios,efi64}/* instead of isoroot/arch/boot/syslinux/*. And I only wanted to ask where to place cfg files better:
a) in isoroot/arch/boot/syslinux/ for both modes, or
b) in isoroot/arch/boot/syslinux/bios/ for bios and in isoroot/arch/boot/syslinux/efi64/ for uefi mode.
I like b variant more than a variant.
Comment by Gerardo Exequiel Pozzi (djgera) - Wednesday, 20 March 2019, 22:44 GMT
sorry I mixed things (I use syslinux for my custom archiso). Ignore me.

Yes a separate directory for EFI is needed for syslinux, but i want to get feedback if syslinux.efi works before doing changes.
Comment by Andrew Shark (ashark) - Wednesday, 20 March 2019, 22:54 GMT
I have experimented a bit with syslinux.efi, it is able to boot, but I need to test if other things are still working (such as booting from CD) after we move bios directory.
So to be finally clear, we are going b variant, which is "Distinct directory common kernel path", ok?
I wanted to know this before making many changes in pathes and taking many tests.
Comment by Gerardo Exequiel Pozzi (djgera) - Thursday, 21 March 2019, 14:35 GMT
Just guessing... the "best thing" to do without altering much things is putting inside EFI/syslinux all files releated for syslinux (binaries + config files [just the PXE parts]), and adjust dnsmasq paths. Should work...
Comment by Gerardo Exequiel Pozzi (djgera) - Thursday, 21 March 2019, 15:11 GMT
This is the first idea... (untested)
Comment by Andrew Shark (ashark) - Friday, 29 March 2019, 05:25 GMT
Yes, you have suggested really good place for it.

At line 185 cp ${work_dir}/x86_64/airootfs/usr/lib/syslinux/bios/syslinux.efi ${work_dir}/iso/EFI/syslinux you should use efi64 folder, not bios folder.
Missing ldlinux.e64 file, add cp ${work_dir}/x86_64/airootfs/usr/lib/syslinux/efi64/ldlinux.e64 ${work_dir}/iso/EFI/syslinux
Add " (UEFI PXE)" to title "Arch Linux" in ${work_dir}/iso/EFI/syslinux/archiso_head.cfg
Splash png is missing and commented in config
Messed up % symbols in configs/releng/pxelinux/archiso_pxe.cfg so final iso contains unreplaced %INSTAL_DIR/%, %INSTAL_DIR/
Also missed "iso" word here: cp ${script_path}/syslinux_efi64/splash.png ${work_dir}/EFI/syslinux

And what I can already say:
Boot existing OS hangs a machine. In archwiki there is link to bug saying that currently chain.c32 only supports bios bootloader chainloading, but my machine hanged even when I enabled legacy support.
Hardware Information (HDT) entry fails the same as in bios mode (see  FS#62235 ), but file libgpl.c32 was sent correctly.
Reboot entry is not working (system hangs), most likely syslinux's bug, so we can grey it out for now with MENU DISABLE.
Power Off entry we possibly should remove, because poweroff.c32 is missing in efi64 folder of syslinux

It's a shame that syslinux is so buggy... We should not replace systemd-boot with it, because it do not support chainloading of other efi executables (like efi shell).

Then we need to test pathes, mine are not working now.
Because bios configs are relative to (iso_root)/arch (it is done that way now, we should make them relative to bios syslinux directory for configs to work and remove /arch/boot from there), they can use downway directories. But uefi configs will be relative to (iso_root)/EFI/syslinux and they need to use kernel (that is residing in another branch), pathes needs to go upway. But we cannot use ../ there, because dnsmasq denies permission for such pathes. I have found another solution - using ::. This way it is working.
Comment by Andrew Shark (ashark) - Sunday, 07 April 2019, 12:05 GMT
Hi, Gerardo! I have created a patch for this.
What changed:
1) Added pxelinux uefi configs
2) Added release version to syslinux menu title
3) Used syslinux dir as cwd in configs
What tested:
Generated iso booted normally in uefi and in legacy modes.
What not changed:
Booting in uefi usb and legacy usb mode was not touched.

Please review, then commit it to repo and then I will edit ArchWiki PXE article to correspond new pathes.
Comment by Andrew Shark (ashark) - Sunday, 07 April 2019, 13:33 GMT
In build.sh forgot to add ldlinux.e64, added some ";" symbols in sed; in EFI/syslinux/archiso_pxe.cfg replaced %INSTALL_DIR% for NBD method.
Attached another patch.
By the way, thanks for changing cwd to syslinux, now dnsmasq log is much more clear. There is no clutter about not found modules, all files are requested at correct path in the first attempt. This is good.
Comment by Gerardo Exequiel Pozzi (djgera) - Sunday, 07 April 2019, 16:28 GMT
Many thanks for your work!.

Some things to correct:
1) There is no concept of release version in Arch Linux, adding such thing to the boot menu is not a good idea (this was discussed long long time ago)
2) Please do not use absolute paths in boot config files (at least at build time) /arch

Some things to check:
I see that you touch many paths. I remember with the releng-team, to make syslinux works unified with all posible boot scenarios was a great challenge. This should be tested under:
(a) boot-cd (ELTORITO)
(b) usb-hybrid (AKA "dd")
(c) usb-copy (cp -r arch ... ; extlinux -i ...)
(d) pxe-http / pxe-nfs / pxe-nbd
Comment by Andrew Shark (ashark) - Sunday, 07 April 2019, 18:29 GMT
1) Are you absolutely sure? What if think about it not as a release version, but iso file version? For me it looks useful in title. Can you please share a link to that old discussion?
2) Which exactly files do you mean? The only file which uses path starting with "/arch" explicitly is isolinux/isolinux.cfg, but it was already using it before my patch. Or do you mean absolute pathes in archiso_pxe.cfg files that looks like "::/arch"?

Also found another mistake: in EFI/syslinux/archiso_pxe.cfg and in arch/boot/syslinux/archiso_pxe.cfg in arch64_nfs I forgot to write "::/arch" before "boot/x86_64/vmlinuz". Attached 0004 patch fixing that.
Lets make a final decision of 1 and 2, then I will make a global tests (a,b,c,d in all modes).
Comment by Gerardo Exequiel Pozzi (djgera) - Monday, 08 April 2019, 03:58 GMT
1) its really old, maybe around... 2009-2010... (indeed ISO was tagged with version, like you want now) In any case please split the patch in "simple" things. (What I want to say, the change for showing iso version is unrelated with the PXE changes.)
2) All files, yes. Nope isolinux.cfg is using %INSTALL_DIR%, because is needed, there is no other way to do. In general the idea is to avoid absolute paths at all, this allow to the user without touching much things, ready to move/rename "/arch" to another place and always work. This was one of design choice.
Comment by Andrew Shark (ashark) - Monday, 08 April 2019, 04:27 GMT
1) You mean I need to attach several patch files? Then how should I make next patches after first: diff between origin/HEAD and second_patch or diff between first_patch and second_patch (sorry if silly question, did not done that before)?
2) Yeah, I mean in isolinux.cfg "/%INSTALL_DIR%" becomes "/arch" is explicitly absolute. I did not touch that. Now what about absolute pathes in pxe configs. As I told in previous message: "we cannot use ../ there, because dnsmasq denies permission for such pathes. I have found another solution - using ::. This way it is working.". Is there a way to allow "../" for dnsmasq tftp pathes?
Anyway, these entries "::/" are only appeared in pxe-related configs (archiso_pxe.cfg both legacy and uefi) and in tail of EFI/syslinux/archiso_tail.cfg for hdt: APPEND modules_alias=::/arch/boot/syslinux/hdt/modalias.gz pciids=::/arch/boot/syslinux/hdt/pciids.gz
Syslinux_efi64 is not booted in usb or cd mode now, so users are not affected (hdt_efi64 does not exist now, but maybe could be activated via systemd-boot if possible).
I was preparing working configuration for pxe booting like described in PXE article. It is easy to follow and get things working out of the box. But if users want to add arch to their existing pxe server, they will edit pxe configs anyway (which i described in article). More of that, these files would not be used at all, but just be a reference for their own configs resided in their location.
So I see no problem leaving "::" there. What do you think?
Comment by Andrew Shark (ashark) - Thursday, 11 April 2019, 09:52 GMT
> Is there a way to allow "../" for dnsmasq tftp pathes?
I have found an answer. From man dnsmasq:
--tftp-root=<directory>[,<interface>]
Look for files to transfer using TFTP relative to the given directory. When this is set, TFTP paths which include ".." are rejected, to stop clients getting outside the specified root.
...
It is not recommended to run dnsmasq as root with TFTP enabled, and certainly not without specifying --tftp-root. Doing so can expose any world-readable file on the server to any host on the net.

So, technically possible, but highly insecure. And doesn't worth it, keeping in mind that I said in previous comment.

Are you ok with my patch, so could I begin a global test?
Comment by Gerardo Exequiel Pozzi (djgera) - Sunday, 14 April 2019, 14:53 GMT
I am really busy these weeks to review and to answer all things. For (1) yes one patch after last in logic sequence. Not a big one that changes lots of things.
Comment by Andrew Shark (ashark) - Sunday, 14 April 2019, 19:25 GMT
I have splitted my changes to several patches. Please review.
Especially check that configs/releng/syslinux/syslinux.cfg with its APPEND ../../ instruction. I am not sure it is correct. I guess it is needed for legacy booting in extlinux mode.
Comment by Gerardo Exequiel Pozzi (djgera) - Sunday, 14 April 2019, 23:22 GMT
Nice. Please use %INSTALL_DIR% not /arch/ directly, then parse it at built time.
Cosmetic: call BIOS boot, not legacy.
Comment by Andrew Shark (ashark) - Monday, 15 April 2019, 07:51 GMT
Replaced "arch" to "%INSTALL_DIR%" in all pathes in releng/syslinux/archiso_pxe.cfg, releng/syslinux_efi64/archiso_{pxe,tail}.cfg
Replaced "Legacy" to "BIOS" in releng/syslinux/archiso_head.cfg
Comment by Andrew Shark (ashark) - Friday, 19 April 2019, 12:52 GMT
I was right about configs/releng/syslinux/syslinux.cfg: we must remove that "APPEND ../../" line.
I have made a global test, here are results:
(a) ElTorito bios: all ok
(a) ElTorito uefi: all ok
(b) usb-hybrid bios:
poweroff - APM not present
Boot existing - not tested, because I do not have any legacy boot os installed, but should work
All other things - ok
(b) usb-hybrid uefi: all ok
(c) usb-copy bios:
poweroff - APM not present
Boot existing - not tested, because I do not have any legacy boot os installed, but should work
All othere things - ok
(c) usb-copy uefi: all ok
(d) pxe bios:
pxe-methods - tested previously, all ok
(d) pxe uefi:
HDT - libgpl.c32 failed, see FS#59987
reboot - just hangs, not reboot (upstream bug?)
pxe-methods - tested previously, all ok

So what about EFI/syslinux/archiso_tail.cfg? Should we remove it? It do not contain any working entry now:
boot existing - useless, because currently it cannot boot efi executables
memtest - does not work (because of previous reason?). By the way, we can add uefi variant of memtest86 to systemd-boot loader
hdt - currently fails
reboot - hangs system instead of reboot
poweroff - no such module for efi64, so commented out
Comment by Gerardo Exequiel Pozzi (djgera) - Saturday, 20 April 2019, 18:50 GMT
Nice, thanks for the tests.

Yes, *personally* I never like the reboot / poweroff options. The "poweroff" with much more reason, because harddisk does not poweroff in right way (sync, heads-off, power-off). And reboot is a bit redundant, I guess, (ie CTRL+ALT+DEL).
About ¿hdt? I never used this for real purposes, other than.. Yes seems to run.
The memtest this is nice, but the EFI version, I do not know about license, indeed if my memory does not fail, there is a ticket here about this.
The "existing", is like playing with a rulette, some times can work others not, depending on the hardware and maybe will boot the unexpected things on multiboot system.
Comment by Andrew Shark (ashark) - Saturday, 20 April 2019, 19:15 GMT
This is out of scope of this ticket, but I agree, maybe let's remove that poweroff and reboot entries completely (from legacy and uefi)? I have googled quickly about APM, as I understand it is an old technology, probably replaced by ACPI.
I used hdt to quickly determine supported vesa resolutions on a machine without OS installed, and also to check pci devices with their associated modules. Probably it is useful for somebody, I mean people may use Arch Linux Live usb as a repair/diagnostic tool.
Here in aur there is memtest86-efi package. And also memtest86+-pxe-git - NBP variant. Looks like memtest86-efi is GPL.
Also I am not sure that SERIAL instruction is needed in archiso_head.cfg.
Comment by Andrew Shark (ashark) - Thursday, 23 May 2019, 17:34 GMT
Hi. What is your decision about this?

Loading...