I believe this fix may be related to  FS#49071 , as the mdadm_udev hook will then only return once the arrays are assembled, in time for the lvm2 hook to start using the lv metadata.

The bug is still present and therefore booting from an degraded md-raid device fails.

The way I understand it, the package build file "mdadm_udev_install" misses the following build rules:

add_systemd_unit "mdadm-last-resort@.service"
add_systemd_unit "mdadm-last-resort@.timer"

This makes sure that the required systemd units are copied to the initramfs. They are triggered by 64-md-raid-assembly.rules as described above.
No additional fixes should be required.

eaut, the fix you suggest works in a systemd init but not in a busybox init because systemd timer functionality is not available.

The fix I suggested is independent of the init system. It is less complicated than it may seem from the attached files. In fact, there is a one-line install patch and a few-line hook. The hook sleeps for 10 seconds if degraded arrays are detected and then retries to assemble the arrays and starts them even if some are still degraded.


--- /usr/lib/initcpio/install/mdadm_udev
+++ /etc/initcpio/install/mdadm_udev
@@ -12,8 +12,6 @@
add_binary "/usr/bin/mdadm"
add_file "/usr/lib/udev/rules.d/63-md-raid-arrays.rules"
add_file "/usr/lib/udev/rules.d/64-md-raid-assembly.rules"
+
+ add_runscript
}

help() {

@Daan: I did not have busybox init in mind - why are you preferring it over systemd init?
Hopefully all the currently open initrd bugs with LVM, MD, etc. will be history after the anticipated Dracut switch.

@Daan: I did not have busybox init in mind - why are you preferring it over systemd init?
Hopefully all the currently open initrd bugs with LVM, MD, etc. will be history after the anticipated Dracut switch.

@drrossum thank you very much!

Here is my test case:

==0==) Build raid1 array

==1==) Add 1 line to /usr/lib/initcpio/install/mdadm_udev
--- /usr/lib/initcpio/install/mdadm_udev.orig 2020-03-15 16:37:07.967859694 +0300
+++ /usr/lib/initcpio/install/mdadm_udev 2020-03-15 16:38:18.824530798 +0300
@@ -12,6 +12,8 @@
add_binary "/usr/bin/mdadm"
add_file "/usr/lib/udev/rules.d/63-md-raid-arrays.rules"
add_file "/usr/lib/udev/rules.d/64-md-raid-assembly.rules"
+
+ add_runscript
}

help() {

==2==) Add new file /usr/lib/initcpio/hooks/mdadm_udev (mdadm_udev.hook from bug report)

==3==) mkinitcpio -p linux

==4==) shutdown -h now

==5==) Remove one hard drive and turn on the computer

It's started!

==6==) Shutdown computer, attach disk back and start the computer again

==7==) Attach disk back to raid
mdadm --manage --add /dev/md126 /dev/sdb2

==8==) Wait for sync to be complete
watch 'cat /proc/mdstat'

Since mdadm 4.1-1 and mdadm hook has been replaced by mdadm_udev hook, I didn't yet faced a situation where I needed to reboot a server with a degraded root array.

Tobias, did you check if booting on a degraded array is now possible with mdadm_udev? This could close this one, and, if not it's a real issue.

Sébastien, "add_systemd_unit" is still missing in mdadm_udev_install.
See my comment from Sunday, 01 September 2019.

I've just run into the same issue when using qemu and -hda test1.qcow2 and -hdb test2.qcow2 .

After install, removing -hda test1.qcow2 causes the system to fail booting.

-- bug wrangling day --
bug is still present, easy fix is described in comment from Sunday, 01 September 2019

Please note the perceived shortcomings of that suggestion, as described in the comment from 2019-10-18.

In short: the proposed workaround seems to depend on optional hooks in the initramfs, so it may not work for all users.

Why are the hooks optional? They are already provided by the package!
See pacman -Ql mdadm | grep mdadm-last-resort
The only dependency for this fix is systemd - I would guess a solution for 99% of all systems.

Well, the user has the option to choose between systemd and busybox as a basis for their initcpio. If the user opts for busybox then the added systemd timer will never by executed.

A direct quote from the ArchWiki documentation (https://wiki.archlinux.org/index.php/Mkinitcpio#Using_RAID)
"Note: mdadm is deprecated. If using it you will see ==> WARNING: Hook 'mdadm' is deprecated. Replace it with 'mdadm_udev' in your config when doing an upgrade."

No, this is about the init system: base+udev vs base+systemd hooks

Anyway, the real problem is that we have bug fixes for both systemd and busybox for many months now and nobody integrates them.
In case of systemd init it is a simple two line oversight that needs to be fixed.

Is this still an issue with 4.2?

Guys, Im sorry, but I am very noob, but I really need this issue to be fixed. "This can be fixed using custom install and hook files for the mdadm_udev hook, see attached." - please tell how can this may be done. Googled, but not found. Thanks in advance!

It is still an issue with mdadm 4.2-2.

What happened here, was a classical case of trying to fix too much in one go.

The mdadm udev rules themselves are already targeted at systemd, which is obvious since they rely on the timer. The fix proposed by the op using an additional hook may work in the generic case, but the one by eaut is actually correct for a system using systemd. The more generic fix replaces a broken part of the systemd way with a workaround, but leaves the broken part in place, making for an awkward solution.

The correct solution for a system using busybox would be to change the udev rules themselves such that they do not rely on systemd. That would of course be a little more complicated than anything already proposed here.

I suggest doing two things. First, fix the obvious bug. Then deal with the more generic problem separately.
* apply eaut's fix in this context to solve the problem for systemd users
* open a new ticket to fix the rules for systems not running systemd but busybox init in the mkinitcpio. This can be done either by any of the following:
1. enhancing the udev rules that they are agnostic to sysdemd/busybox
2. creating a separate set of udev rules for this case
3. creating a separate mkinitcpio hook (mdadm_busybox) which fixes the udev rules by running the hook script provided by the op, but does not make the regular mdadm+systemd case awkward.

Opening a new ticket also should increase the chance to get anything done quickly for the busybox side, because all relevant information can be presented in a well sorted and concise manner.

The original description of this ticket does not mention busybox and therefore the fix applied for this ticket should be the obvious one, adding the missing timer to the initcpio.

Thank you, Alexander, for this valuable input. I agree with your assessment.

For the busybox case, I would vote for solution 3: a separate mkinitcpio hook that starts the arrays if they are still degraded after the timeout.

Hi, I spent several hours about solving this problem before I found this thread. Please can you include this fix directly in the mdadm package? I think it will make life easier for more users. Thanks!

I'm also affected by this issue. I have /dev/md0 is boot and /dev/md1 is cryptdevice. It drops to rescue shell and there I see that md0 is started but md1 is not started.

This is an automated comment as this bug is open for more then 2 years. Please reply if you still experience this bug otherwise this issue will be closed after 1 month.