FS#44938 - [systemd] initrd is not regenerated after systemd/udev update

Attached to Project: Arch Linux
Opened by David Ryskalczyk (david_rysk) - Monday, 11 May 2015, 23:13 GMT
Last edited by Dave Reisner (falconindy) - Sunday, 27 November 2016, 23:01 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Dave Reisner (falconindy)
Tom Gundersen (tomegun)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 7
Private No


After a systemd update (which also updates udev), the initrd is not regenerated, leading to a version mismatch between the udev in the initrd and the udev and systemd on the rootfs. This causes at least one strange issue with btrfs:
systemd/udev 218 to 219: btrfs raid5/6 volumes specified in /etc/fstab refuse to automount with the following error:

Job dev-disk-by\x2dlabel-bigarray.device/start timed out.
Timed out waiting for device dev-disk-by\x2dlabel-bigarray.device.
Dependency failed for /mnt/bigarray.
Job mnt-bigarray.mount/start failed with result 'dependency'.
Job dev-disk-by\x2dlabel-bigarray.device/start failed with result 'timeout'.

Attempting a manual mount of this volume using the mount command causes systemd to immediately unmount it since the unit is failed. Removing the volume from fstab and running systemctl daemon-reload allows manual mounting.
After running mkinitcpio -p linux and rebooting (which updates the udev version on the initrd), the problem disappears. I was able to reproduce this through multiple downgrade/upgrade cycles.

However, after several reboots (at least 3-5) the problem strangely went away. Is is unclear why, but the problem was reproducible before this.

Chances are this problem isn't very visible because if systemd and the kernel are upgraded at the same time, mkinitcpio is automatically run.

Additional info:
package versions: old: systemd-218-2 (and related); new: systemd-219-6 (and related)
fstab line: LABEL=bigarray /mnt/bigarray btrfs autodefrag,compress=lzo,relatime,nofail 0 0

Steps to reproduce:
1. Set up a btrfs raid5 or raid6 array included in /etc/fstab.
2. Downgrade to systemd-218. Regenerate the initrd and reboot.
3. Observe that the btrfs array is mounting automatically, as it should be.
4. Use pacman to update the system to systemd-219 or newer.
5. Reboot and observe that the volume is no longer automatically mounting.
6. Run mkinitcpio to regenerate the initrd and reboot.
7. Observe that the volume is now correctly mounting.
This task depends upon

Closed by  Dave Reisner (falconindy)
Sunday, 27 November 2016, 23:01 GMT
Reason for closing:  Fixed
Additional comments about closing:  covered by /usr/share/libalpm/hooks/99-linux.hook
Comment by Doug Newgard (Scimmia) - Tuesday, 12 May 2015, 15:13 GMT
I've seen this before (can't find where), but the problems were always theoretical. Interesting that there is an actual problem.
Comment by David Ryskalczyk (david_rysk) - Tuesday, 12 May 2015, 15:15 GMT
Yep, two or three others in the #btrfs IRC channel were complaining about the same issue. I'm uncertain whether regenerating initrd would fix this for them (the systemd error messages are *extremely* unhelpful here but were identical to the ones I was seeing), and I have been unable to track them down yet. I really dislike the nondeterministic aspects of this bug — especially with it going away after several reboots and now being difficult (or nearly impossible) to reproduce.

Maybe systemd should check to make sure there isn't such a mismatch?
Comment by patrick (potomac) - Tuesday, 23 June 2015, 16:13 GMT
same problem today for the upgrade of systemd 221-1 ( 2015-06-22 ), there is no kernel update at this time, this scenario brings the bug of a wrong version of systemd displayed at screen on boot


Comment by James Harvey (jamespharvey20) - Wednesday, 24 June 2015, 18:37 GMT
There was also a problem with not automatically running mkinitcpio when the udev runtime data moved from /dev/.udev/ to /run/udev. https://lists.archlinux.org/pipermail/arch-general/2011-April/019457.html

My vote (not that I have one) is to add automatically running mkinitcpio