FS#54052 - mkinitcpio crash prevents subsequent boot

Attached to Project: Arch Linux
Opened by Liam (networkimprov) - Sunday, 14 May 2017, 18:40 GMT
Last edited by Dave Reisner (falconindy) - Monday, 15 May 2017, 17:55 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To No-one
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

When doing pacman -Syu after a long hiatus (kernel 4.7 to 4.10)...

Error:
:: Running post-transaction hooks...
( 1/14) Updating linux initcpios
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
-> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
==> Starting build: 4.10.13-1-ARCH
-> Running build hook: [base]
-> Running build hook: [udev]
-> Running build hook: [autodetect]
modprobe: ERROR: missing parameters. See -h.

This is accompanied by a kernel panic which I no longer have the output for.

On subsequent boot attempts:
ERROR: Unable to find root device 'LABEL=root'.

However according to grub, that is the correct volume label.
This task depends upon

Closed by  Dave Reisner (falconindy)
Monday, 15 May 2017, 17:55 GMT
Reason for closing:  None
Additional comments about closing:  Nothing to do here if the reporter can't reproduce the bug.
Comment by Dave Reisner (falconindy) - Sunday, 14 May 2017, 18:45 GMT
What are you suggesting as a fix? The interesting problem here is what caused mkinitcpio to invoke modprobe incorrectly -- not that a failing run of mkinitcpio produces an unbootable system.
Comment by Liam (networkimprov) - Sunday, 14 May 2017, 19:52 GMT
A failed run of mkinitcpio should not hose the system!

Suggestions on diagnosing mkinitcpio?
Comment by Dave Reisner (falconindy) - Sunday, 14 May 2017, 20:13 GMT
That's easier said than done. You upgraded your kernel, replacing and old image with a new one, and deleting the old/associated initramfs with it. It's up to you, the user, to ensure that a new initramfs is generally successfully. If you don't like that, install a second kernel (e.g. linux-lts), or propose a fix to  FS#16702 .

'bash -x mkinitcpio' would be a good place to start for diagnosing if you're able to reproduce the error.
Comment by Liam (networkimprov) - Sunday, 14 May 2017, 20:28 GMT
If I changed the mkinitcpio config, then it's on me if it doesn't work. But I am using the default config.

I just did a system upgrade from even older install (stacklet image for VMWare from 2014) and the initramfs wasn't updated...?
Comment by Dave Reisner (falconindy) - Sunday, 14 May 2017, 20:38 GMT
> If I changed the mkinitcpio config, then it's on me if it doesn't work. But I am using the default config.
I'm asking for your help in reproducing the error and providing debugging information.

> I just did a system upgrade from even older install (stacklet image for VMWare from 2014) and the initramfs wasn't updated...?
And that makes sense to me -- it's the sort of esoteric problem you get when you try to update an Arch Linux install from 3+ years ago. pacman from that age didn't support hooks, and the initramfs is only supported via hooks nowadays.
Comment by Liam (networkimprov) - Sunday, 14 May 2017, 20:40 GMT
And altho the initramfs wasn't updated, it still fails to boot with same error.

I see these before the LABEL=root error:

[0.09...][Firmware Bug]: CPU1: APIC id mismatch. Firmware: 1 APIC: 2
Warning: /lib/modules/4.10.13-1-ARCH/modules.devname not found - ignoring
Comment by Liam (networkimprov) - Sunday, 14 May 2017, 22:25 GMT
Well the mkinitcpio failure doesn't recur after system update of the old stacklet image. I tried:

bash -x -c 'mkinitcpio -p linux -k 4.10.13-1-ARCH'

That works, and now boot works fine.
Comment by Liam (networkimprov) - Monday, 15 May 2017, 01:05 GMT
OK, my proposal is that kernel updates be tested against several prior kernel releases, covering at least two LTS versions.
Comment by Dave Reisner (falconindy) - Monday, 15 May 2017, 01:17 GMT
There's only so long I'm willing to support between updates. That isn't just true of mkinitcpio -- that's true for all of Arch.

> OK, my proposal is that kernel updates be tested against several prior kernel releases, covering at least two LTS versions.
I don't think you understand the problems here, then.
Comment by Liam (networkimprov) - Monday, 15 May 2017, 01:53 GMT
I last updated to a v4.7 release, around 8 months ago. Is that unreasonably long?

Testing kernel install against multiple prior releases could be done against the x.y.0 version for each. I didn't mean to suggest testing against x.y.* :-)

Has a transaction encompassing the kernel & initramfs been considered?
Comment by Doug Newgard (Scimmia) - Monday, 15 May 2017, 16:00 GMT
Ok, I think this has gone on long enough. Either debug the original failure or there's not much to talk about here.
Comment by Liam (networkimprov) - Monday, 15 May 2017, 17:45 GMT
I couldn't reproduce it when updating from an older kernel; I suspect it was due to a kernel bug in 4.7. However any issue that stops mkinitcpio renders the system unbootable. This is fixable:

Let the kernel package install a flag, e.g. /boot/outdated-initramfs.
Let mkinitcpio remove the flag on successful completion.
Let grub skip the initramfs if the flag is present.
Comment by Dave Reisner (falconindy) - Monday, 15 May 2017, 17:54 GMT
> I suspect it was due to a kernel bug in 4.7.
> Let grub skip the initramfs if the flag is present.
You have a fundamental misunderstanding as to how the boot process works.

Loading...