FS#77780 - [systemd] boot fails, cannot find PARTUUID (systemd-boot with BTRFS-Subvolumes)
Attached to Project:
Arch Linux
Opened by Peter Weber (hoschi) - Wednesday, 08 March 2023, 14:10 GMT
Last edited by Toolybird (Toolybird) - Thursday, 09 March 2023, 19:59 GMT
Opened by Peter Weber (hoschi) - Wednesday, 08 March 2023, 14:10 GMT
Last edited by Toolybird (Toolybird) - Thursday, 09 March 2023, 19:59 GMT
|
Details
Description:
Hello! Today I've upgraded from 253.1-1 to 253.1-2 which rendered my system unbootable. Upon boot systemd failed to mount the BTRFS subvolume of the root directory. The PARTUUID is not the one usually found in /etc/fstab but with blkid (look for PARTUUID). Please take into account that the system was recently installed with archinstall, using BTRFS-Subvolumes with systemd-boot. With both I'm not familiar. Maybe I did something stupid? These were the default settings presented by archinstall. Usually I'm installing manually with EXT4 and GRUB2. Additional info: * package version(s): systemd-253.1-2, systemd-libs-253.1-2, systemd-sysvcompat-253.1-2 * config and/or log files etc.: [2023-03-08T13:41:11+0100] [PACMAN] Running 'pacman -Syyu' [2023-03-08T13:41:11+0100] [PACMAN] synchronizing package lists [2023-03-08T13:41:12+0100] [PACMAN] starting full system upgrade [2023-03-08T13:41:43+0100] [ALPM] transaction started [2023-03-08T13:41:43+0100] [ALPM] upgraded systemd-libs (253.1-1 -> 253.1-2) [2023-03-08T13:41:44+0100] [ALPM] upgraded systemd (253.1-1 -> 253.1-2) [2023-03-08T13:41:45+0100] [ALPM] upgraded libedit (20210910_3.1-2 -> 20221030_3.1-1) [2023-03-08T13:41:45+0100] [ALPM] upgraded sdl2 (2.26.3-1 -> 2.26.4-1) [2023-03-08T13:41:45+0100] [ALPM] upgraded gtk-update-icon-cache (1:4.8.3-3 -> 1:4.10.0-3) [2023-03-08T13:41:45+0100] [ALPM] upgraded chromium (110.0.5481.177-1 -> 111.0.5563.64-1) [2023-03-08T13:41:45+0100] [ALPM] upgraded libwpe (1.14.1-1 -> 1.14.1-2) [2023-03-08T13:41:45+0100] [ALPM] upgraded gnome-photos (1:43.0-2 -> 1:44.0-1) [2023-03-08T13:41:45+0100] [ALPM] upgraded gtk4 (1:4.8.3-3 -> 1:4.10.0-3) [2023-03-08T13:41:45+0100] [ALPM] upgraded gtk4-demos (1:4.8.3-3 -> 1:4.10.0-3) [2023-03-08T13:41:46+0100] [ALPM] upgraded gtk4-docs (1:4.8.3-3 -> 1:4.10.0-3) [2023-03-08T13:41:46+0100] [ALPM] upgraded inetutils (2.3-1 -> 2.4-1) [2023-03-08T13:41:46+0100] [ALPM] upgraded libbpf (1.0.1-1 -> 1.1.0-1) [2023-03-08T13:41:46+0100] [ALPM] upgraded lmdb (0.9.30-2 -> 0.9.30-3) [2023-03-08T13:41:46+0100] [ALPM] upgraded phodav (3.0-1 -> 3.0-2) [2023-03-08T13:41:46+0100] [ALPM] upgraded python-urllib3 (1.26.12-1 -> 1.26.13-1) [2023-03-08T13:41:46+0100] [ALPM] installed spice-protocol (0.14.4-1) [2023-03-08T13:41:46+0100] [ALPM] upgraded spice-gtk (0.42-1 -> 0.42-3) [2023-03-08T13:41:46+0100] [ALPM] upgraded systemd-sysvcompat (253.1-1 -> 253.1-2) * link to upstream bug report, if any: none Preconditions: * root filesystem is btrfs with subvolumes * bootloader is systemd-boot The archinstall-script will give you this setup as default on an UEFI-System. Steps to reproduce: 1. pacman -Syu 2. reboot Workaround/Recovery: The command arch-chroot didn't worked for me. And the usual chroot also not because the subvolumes of BTRFS makes things more complicated. I roughly followed[1] this instructions. Basically look first at hour /etc/fstab and then: # mount /dev/${REPLACEDEVICE} /mnt/ -t btrfs -o subvol=/@ # mount /dev/${REPLACEDEVICE} /mnt/home -t btrfs -o subvol=/@honme # mount /dev/${REPLACEDEVICE} /mnt//var/cache/pacman/pkg -t btrfs -o subvol=/@pkg # mount /dev/${REPLACEDEVICE} /mnt/var/log -t btrfs -o subvol=/@log # mount /dev/${REPLACEDEVICE} /mnt/.snapshots -t btrfs -o subvol=/@.snapshots # mount /dev/${OTHERDEVICE} /mnt/boot # now usual stuff # mount --bind /dev /mnt/dev # mount -t proc /proc /mnt/proc # mount -t sysfs /sys /mnt/sys # mount -t tmpfs tmpfs /mnt/run # chroot /mnt # cd /var/cache/pacmang/pkg # pacman -U ... # exit # umount # I skipped... # reboot Thank you [1] https://fedoramagazine.org/os-chroot-101-covering-btrfs-subvolumes/ |
This task depends upon
Closed by Toolybird (Toolybird)
Thursday, 09 March 2023, 19:59 GMT
Reason for closing: None
Additional comments about closing: Reporter says "mkinitcpio fails due to hardware problem, corrupted initramfs. nothing to do with systemd!"
Thursday, 09 March 2023, 19:59 GMT
Reason for closing: None
Additional comments about closing: Reporter says "mkinitcpio fails due to hardware problem, corrupted initramfs. nothing to do with systemd!"
Not from me but timing matches.
Archinstall didn't setup an XBOOTLDR here:
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
zram0 254:0 0 4G 0 disk [SWAP]
nvme0n1 259:0 0 476.9G 0 disk
├─nvme0n1p1 259:1 0 511M 0 part /boot
└─nvme0n1p2 259:2 0 476.4G 0 part /var/log
/var/cache/pacman/pkg
/home
/.snapshots
/
https://bbs.archlinux.org/viewtopic.php?id=284062
// edit
Probably same user as on Stackexchange.
https://bugs.archlinux.org/task/77715
btw. same for the other affect user
[2023-03-08T13:41:47+0100] [ALPM-SCRIPTLET] ==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
[2023-03-08T13:41:47+0100] [ALPM-SCRIPTLET] -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
[2023-03-08T13:41:48+0100] [ALPM-SCRIPTLET] ==> Starting build: 6.2.2-arch1-1
[2023-03-08T13:41:48+0100] [ALPM-SCRIPTLET] -> Running build hook: [base]
[2023-03-08T13:41:48+0100] [ALPM-SCRIPTLET] -> Running build hook: [udev]
[2023-03-08T13:41:48+0100] [ALPM-SCRIPTLET] -> Running build hook: [autodetect]
[2023-03-08T13:41:48+0100] [ALPM-SCRIPTLET] sort: cannot read: '/sys/devices/pci0000:00/0000:00:08.1/0000:07:00.4/usb3/3-1/3-1.2/3-1.2.2/3-1.2.2:1.2/ep_04/uevent': No such file or directory
[2023-03-08T13:41:48+0100] [ALPM-SCRIPTLET] modprobe: ERROR: missing parameters. See -h.
[2023-03-08T13:41:48+0100] [ALPM-SCRIPTLET] -> Running build hook: [keyboard]
[2023-03-08T13:41:48+0100] [ALPM-SCRIPTLET] -> Running build hook: [keymap]
[2023-03-08T13:41:48+0100] [ALPM-SCRIPTLET] -> Running build hook: [modconf]
[2023-03-08T13:41:48+0100] [ALPM-SCRIPTLET] -> Running build hook: [block]
[2023-03-08T13:41:48+0100] [ALPM-SCRIPTLET] -> Running build hook: [filesystems]
[2023-03-08T13:41:48+0100] [ALPM-SCRIPTLET] -> Running build hook: [fsck]
[2023-03-08T13:41:49+0100] [ALPM-SCRIPTLET] ==> Generating module dependencies
[2023-03-08T13:41:49+0100] [ALPM-SCRIPTLET] ==> Creating zstd-compressed initcpio image: /boot/initramfs-linux.img
[2023-03-08T13:41:49+0100] [ALPM-SCRIPTLET] ==> Image generation successful
[2023-03-08T13:41:49+0100] [ALPM-SCRIPTLET] ==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'fallback'
[2023-03-08T13:41:49+0100] [ALPM-SCRIPTLET] -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-fallback.img -S autodetect
[2023-03-08T13:41:49+0100] [ALPM-SCRIPTLET] ==> Starting build: 6.2.2-arch1-1
[2023-03-08T13:41:49+0100] [ALPM-SCRIPTLET] -> Running build hook: [base]
[2023-03-08T13:41:49+0100] [ALPM-SCRIPTLET] -> Running build hook: [udev]
[2023-03-08T13:41:49+0100] [ALPM-SCRIPTLET] -> Running build hook: [keyboard]
[2023-03-08T13:41:49+0100] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: xhci_pci
[2023-03-08T13:41:50+0100] [ALPM-SCRIPTLET] -> Running build hook: [keymap]
[2023-03-08T13:41:50+0100] [ALPM-SCRIPTLET] -> Running build hook: [modconf]
[2023-03-08T13:41:50+0100] [ALPM-SCRIPTLET] -> Running build hook: [block]
[2023-03-08T13:41:51+0100] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: aic94xx
[2023-03-08T13:41:51+0100] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: bfa
[2023-03-08T13:41:52+0100] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: qed
[2023-03-08T13:41:52+0100] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: qla1280
[2023-03-08T13:41:52+0100] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: qla2xxx
[2023-03-08T13:41:52+0100] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: wd719x
[2023-03-08T13:41:53+0100] [ALPM-SCRIPTLET] -> Running build hook: [filesystems]
[2023-03-08T13:41:54+0100] [ALPM-SCRIPTLET] -> Running build hook: [fsck]
[2023-03-08T13:41:55+0100] [ALPM-SCRIPTLET] ==> Generating module dependencies
[2023-03-08T13:41:55+0100] [ALPM-SCRIPTLET] ==> Creating zstd-compressed initcpio image: /boot/initramfs-linux-fallback.img
[2023-03-08T13:41:56+0100] [ALPM-SCRIPTLET] ==> Image generation successful
This looks like a faulty USB-Device. Can this caused some side-effect?
-> Running build hook: [autodetect]
sort: stat failed: '/sys/devices/pci0000:00/0000:00:08.1/0000:07:00.4/usb4/4-1/4-1.2/4-1.2.4/4-1.2.4:1.1/ep_81/uevent': No such file or directory
modprobe: ERROR: missing parameters. See -h.
Same error. Different USB-Device?
It looks like mkinitcpio falls over some weird USB-Device which is provided through my docking-station.
To chroot into my broken system I had to attach a USB-Thumbdrive, therefore I undocked my laptop and the image creation succeeded.
Maybe mkiniticpio can be improved?
So it either fails 'hard' when an error of this kind occurs or handles such issues elegantly?
My UUIDs for my disks/partitions did not match the same UUIDs I had in my fstab or systemd-boot entries. I could not boot and was punted to an initramfs shell with the same error as in the first image attached to the OP.
My fix was to inspect my disk and partitions' UUIDs, and then edit /etc/fstab and my systemd-boot entries with the correct UUID values.
Either my boot config was updated with incorrect UUIDs, or the disks/partitions were given new UUIDs for some reason.
I was using dracut and not mkinitcpio, but I believe I might have had it installed at the time.
It does seem like an odd failure mode. If you can repro and have the time, mkinitcpio bugs can be reported here [1].
[1] https://gitlab.archlinux.org/archlinux/mkinitcpio/mkinitcpio