Arch Linux

Scenario:

partitionable raid-5 array created with mdadm, i.e. mdadm -C /dev/md/d0 -l5 -n4 -x1 -e1 /dev/sd{a,b,c,d,e}4 -z8388608 --auto=p6

On the array we create 2 partitions - nevermind the size/id/etc., just be sure they're created. Then we update mdadm.conf, automatically or not, to:

ARRAY /dev/md/d0 UUID=<valid uid>
CREATE auto=p6 mode=0660 group=disk owner=root

Next, to see what is happening on udev side, let's start udevd manually, with debug options (--debug-trace --verbose).

Under standard arch's udev rules, and fully updated arch64 (inc. mdadm 2.6.2-1), following happens, starting with clean, unassembled array:

1) mdadm -A /dev/md/d0

No rules in arch for md_d* case at all - so only standard /dev/md_d0 is created, whereas mdadm tries to create md/d0 and symlink it as /dev/md_d0. Unless of course we explicitely prohibit mdadm from creating nodes and/or symlinks.

No big deal yet, but...

2) mdadm -S /dev/md/d0

"add" uevent for every existing partition is generated; looking at udevd trace, /dev/md_d0p1 and /dev/md_d0p2 are created, replacing already existing symlinks created by mdadm in step 1) (as per udev rules). Looks like partition related uevents happen later, and assembling array in step 1) causes only md_d0 uevent.

3) mdadm -A /dev/md/d0

Here - "remove" uevent for every created partition; causing in most cases removal of symlinks created by mdadm. Similar remark as above.


It seems that if any partition / mdadm operation happens between 1) and 2), or 2) and 3), respective "add" and "remove" will happen then.


On the related subject - as I've modified a bit my setup, I've added simple raid-all hook combining raid and raid-partitions functionality (and relying only on mdadm).


Workarounds / solutions:


General approach is to stop mdadm and udev getting in each others way.

For example - as I decided to use subdirectory approach - md/N md/dNpM, I've added to mdadm.conf:

CREATE owner=root group=disk symlinks=no auto=p2

Important part is symlinks part, which will stop mdadm from creating them (my initcpio included mdadm.conf by default now).

On the udev front I've replaced:

KERNEL=="md[0-9]*", NAME="md%n", SYMLINK+="md/%n"

with

KERNEL=="md[0-9]*", NAME="%k"
KERNEL=="md_d[0-9]*", NAME="%k"

(both rules can be removed actually, as it's the default behaviour)

This way, mdadm and udev can coexist peacefully. The only thing to remember, is that udev will wait a bit with partition uevents (some next mdadm use, or some partition related operation).


Other possible approaches - stop mdadm from touching devices at all (i.e. use auto=no in mdadm.conf) and write proper udev rules, remembering that disk device node would have to be created by some other means (echo "add" to uevent, udevtrigger, mknod, ...)


Or the other way around - stop udev from making any nodes (i.e. NAME="", or even ignore_device), and rely on mdadm only.


[Originally I posted it on forums here: http://bbs.archlinux.org/viewtopic.php?id=35657 but this is the updated version. There're also udev traces there.]