Arch Linux

Maybe this will help:
http://cgit.freedesktop.org/systemd/systemd/commit/?id=185abfc3d6b4e8f804a3f7216cd8b0459593af87

I filed an upstream report:

https://bugs.freedesktop.org/show_bug.cgi?id=90662

Darek: I've just tried systemd-git 220.24.g185abfc (http://cgit.freedesktop.org/systemd/systemd/commit/?id=185abfc3d6b4e8f804a3f7216cd8b0459593af87), but this version gives exactly the same behaviour. Please note that you have to rebuild your initrd by using mkinitcpio after installing the new version, otherwise you'll end up with the old systemd version in the initrd and won't be able to reproduce the problem.

Btw, are we certain this is a bug in systemd and not one in cryptsetup? After all, cryptsetup seems to be the only thing triggering this bug so far.

This is what I did:
1) install the official archlinux cryptsetup 1.6.6 and systemd 220 -> re-generated initramfs image -> saw the bug.
2) I rebooted to rescue cd, installed systemd 219, regenerated initramfs image, rebooted to system.
3) Then I updated cryptsetup 1.6.7 by editing the pkgbuild file, compiled systemd 220 against it, regenerated initramfs image, rebooted and saw bug again.
4) repeated step 2.

So, if it is a cryptsetup bug, it's not fixed in 1.6.7

I can confirm that the system does not boot anymore. I get the following error before I can enter the password.
inotify_add_watch(7,/dev/dm-0,10) failed: Bad file descriptor
inotify_add_watch(7,/dev/dm-1,10) failed: Bad file descriptor
My system is up to date and I have updated the initrd.

Commit 185abfc3 [0] should fix the inotify_add_watch() error.
Pascal E. (hardfalcon) reported that it does not fix the LUKS issue.

[0] http://cgit.freedesktop.org/systemd/systemd/commit/?id=185abfc3

I think I am also affected by this issue. How can I regenerate a new kernel image from within Mac or Ubuntu? And why on earth doesn't any of the systemd people test their own updates on an hdd-encrypted computer?

So now the bug report I created on this issue was closed by Evangelos Foutras (foutrelis) as duplicate and I am still unable to boot my machine.

Can anyone of you please point me to a tutorial on the rescue cd where I can downgrade systemd and regenerate my initramfs image?

Boot a archlinux installation CD then:
cryptsetup luksOpen /dev/sda3 root (replace sda3 with whatever your encrypted partition is)
mount /dev/mapper/root /mnt
arch-chroot /mnt
mount /boot
pacman -Sy core/systemd core/systemd-sysvcompat core/libsystemd
mkinitcpio -p linux
sync
umount /boot
exit
umount /mnt
reboot

Thank you Hussam for pointing me into the right direction.
I had already tried the cryptsetup commands with the current arch live cd, but I get the following error when trying to use it: http://i.imgur.com/RBoQNOf.jpg

Okay, apparently the arch iso cannot be burnt using a windows machine, I'll try again with another computer and report back. Sorry for spamming, but I did not expect for systemd to interfere with my in such a way.. :-(

your screenshot shows errors reading your cd/dvd. you may want to verify it was correctly burned.
If you have a usb disk, try dd instead:
https://wiki.archlinux.org/index.php/USB_flash_installation_media#Using_dd

Benjamin: Just off the top of my head:

1. Boot the machine using a (correctly burned) Archlinux CD. Login as root, set the correct keymap using the "loadkeys" command and configure your network connection if you don't have DHCP in your network.

2. Decrypt and mount the encrypted rootfs on your harddisk:
# cryptsetup luksOpen /dev/sda3 luks-sda3
# mount /dev/mapper/luks-sda3 /mnt

3. chroot into the installation on your harddisk:
# arch-chroot /mnt /bin/bash

4. mount the /boot partition:
# mount /boot

5. Reinstall the systemd packages from core instead of testing:
# pacman -Sy --asdeps core/libsystemd core/systemd core/systemd-sysvcompat

6. Rebuild the initrd for your installed kernel(s):
# mkinitcpio -p linux
# mkinitcpio -p linux-lts
# mkinitcpio -p linux-grsec

7. Exit the chroot and reboot:
# exit
# reboot

'pacman -S core/systemd core/libsystemd core/systemd-sysvcompat' should be enough for downgrading the systemd packages.

Please do NOT specify --asdeps as that will alter the install reason of explicitly installed packages (systemd and systemd-sysvcompat).

This issue should be fixed in systemd 220-2; a couple of confirmations would be nice.

Okay, again I want to thank you all for your kind help. I've gotten it to work now after some time of tinkering. Plus, I've found out the hard way that I have testing enabled. I must have done that a couple of years ago as I first installed arch and compiled several "exotic" libraries.

Pascal and Hussam, your writeups really helped. Unfortunately the situation on my Thinkpad W530 was a bit complicated:
- The first recovery CD (Arch 2015/05) was burnt using the Win7 burning tool, resulting in this error with the "hybrid image" according to people on IRC: http://i.imgur.com/RBoQNOf.jpg
- The second recovery CD was burnt using an OSS tool on Windows called Infrarecorder, and the cd gave the same error when booting
- The third recovery attempt was using a bootable USB stick created on a Mac, which booted...
- .. until it hung at "triggering uevents" ( http://i.imgur.com/JfGcmt6.jpg ); an old and nearly forgotten issue that pops up from time to time with my laptop.
- The acpi / noapic kernel parameters allowed me to finally boot the rescue cd, but the root shell was unusable because my Thinkpad's keyboard did not work (???)
- When plugging in an USB keyboard, the recovery shell from the arch live usb stick was finally usable and I could downgrade systemd

I know it this is a string of unlucky issues and my fixing attempts were at first a bit.. clumsy, but I just wanted to inform you about the reason for all my requests and the annoyance I caused you.

Now everything works like a charm again, and I made sure to remove the testing repos from my pacman.conf.

Thanks again and keep up the good work.

systemd 220-2 fixes the issue for me

Yes systemd 220-2 fixes the issue for me.