FS#16735 - [cryptsetup] LUKS encrypted volumes not opened during init
Attached to Project:
Arch Linux
Opened by Clemens Lutz (tkdfighter) - Monday, 19 October 2009, 07:30 GMT
Last edited by Paul Mattal (paul) - Monday, 08 March 2010, 03:57 GMT
Opened by Clemens Lutz (tkdfighter) - Monday, 19 October 2009, 07:30 GMT
Last edited by Paul Mattal (paul) - Monday, 08 March 2010, 03:57 GMT
|
Details
Description:
LUKS encrypted volumes aren't opened during init. The root volume comes up fine, all others give error message: "No key available with this passphrase". The problems started for me after updating on October 16, the most recent update as of now (19.10.2009) doesn't resolve the issue. There's already a discussion on the forums here: http://bbs.archlinux.org/viewtopic.php?id=81795 Additional info: * package version(s) * config and/or log files etc. Fully updated as of now, i686 Steps to reproduce: On a system using LUKS: boot or run "cryptsetup luksOpen -d /etc/keys/home.key /dev/sda4 home" |
This task depends upon
Closed by Paul Mattal (paul)
Monday, 08 March 2010, 03:57 GMT
Reason for closing: No response
Additional comments about closing: Seems to be working now, no response from original bug filer.
Monday, 08 March 2010, 03:57 GMT
Reason for closing: No response
Additional comments about closing: Seems to be working now, no response from original bug filer.
Some more info about my setup (I should have included this right away, sorry):
Kernel 2.6.31.4-1, using ext3 with the standard arch settings. Also, I'm using a keyfile on usb-stick as described in the wiki and keyfiles on the rootfs for /home and swap.
cat pacman.log | grep "upgraded cryptsetup"
[2009-07-14 19:56] upgraded cryptsetup (1.0.6-2 -> 1.0.6-3)
[2009-08-27 01:25] upgraded cryptsetup (1.0.6-3 -> 1.0.7-1)
Is it ok that I reported this as "critical"? I saw some Firefox related issues rated "high", so I thought this would be a step higher.
Can you build a cryptsetup 1.1.0rc2 package and confirm that this fixes the problem? Also, this problem seems to occur only on some volumes and not on others. Can you provide info about which packages you updated prior to the first failure?
[2009-10-15 18:09] synchronizing package lists
[2009-10-15 18:10] starting full system upgrade
[2009-10-15 19:33] removed bluez-gnome (1.8-5)
[2009-10-15 19:33] upgraded glib2 (2.20.4-1 -> 2.22.2-1)
[2009-10-15 19:33] upgraded atk (1.26.0-1 -> 1.28.0-1)
[2009-10-15 19:33] installed eggdbus (0.5-1)
[2009-10-15 19:33] installed polkit (0.94-1)
[2009-10-15 19:33] upgraded consolekit (0.3.0-5 -> 0.3.1-1)
[2009-10-15 19:33] upgraded pango (1.24.5-1 -> 1.26.0-1)
[2009-10-15 19:33] upgraded gtk2 (2.16.5-1 -> 2.18.2-1)
[2009-10-15 19:33] upgraded gconf (2.26.2-3 -> 2.28.0-1)
[2009-10-15 19:33] upgraded libsigc++2.0 (2.2.3-1 -> 2.2.4.2-1)
[2009-10-15 19:33] upgraded glibmm (2.20.1-1 -> 2.22.1-1)
[2009-10-15 19:33] installed libunique (1.1.2-1)
[2009-10-15 19:33] upgraded libsoup (2.26.3-1 -> 2.28.0-1)
[2009-10-15 19:33] upgraded gnome-keyring (2.26.3-1 -> 2.28.0-1)
[2009-10-15 19:33] installed libsoup-gnome (2.28.0-1)
[2009-10-15 19:33] upgraded libgphoto2 (2.4.6-2 -> 2.4.6-3)
[2009-10-15 19:33] installed parted (1.8.8-3)
[2009-10-15 19:33] installed libatasmart (0.14-1)
[2009-10-15 19:33] installed devicekit-disks (007-1)
[2009-10-15 19:34] installed gnome-disk-utility (2.28.0-1)
[2009-10-15 19:34] installed gvfs (1.4.0-1)
[2009-10-15 19:34] installed obexd (0.17-1)
[2009-10-15 19:34] upgraded gnome-bluetooth (0.12.0-1 -> 2.28.1-1)
[2009-10-15 19:34] upgraded gnome-common (2.26.0-2 -> 2.28.0-1)
[2009-10-15 19:34] upgraded gnome-icon-theme (2.26.0-1 -> 2.28.0-1)
[2009-10-15 19:34] upgraded pygobject (2.18.0-1 -> 2.20.0-1)
[2009-10-15 19:34] upgraded pygtk (2.14.1-4 -> 2.16.0-1)
[2009-10-15 19:34] upgraded gnome-menus (2.26.2-1 -> 2.28.0.1-1)
[2009-10-15 19:34] upgraded libbonobo (2.24.1-1 -> 2.24.2-1)
[2009-10-15 19:34] upgraded gnome-vfs (2.24.1-2 -> 2.24.2-1)
[2009-10-15 19:34] upgraded libgnome (2.26.0-2 -> 2.28.0-1)
[2009-10-15 19:34] upgraded libbonoboui (2.24.1-1 -> 2.24.2-1)
[2009-10-15 19:34] upgraded libgnomeui (2.24.1-1 -> 2.24.2-1)
[2009-10-15 19:34] upgraded gnome-python (2.26.1-1 -> 2.28.0-1)
[2009-10-15 19:34] upgraded libxklavier (3.9-2 -> 4.0-1)
[2009-10-15 19:35] upgraded kdebase-workspace (4.3.2-1 -> 4.3.2-2)
[2009-10-15 19:35] upgraded kdelibs (4.3.2-3 -> 4.3.2-4)
[2009-10-15 19:35] upgraded libwnck (2.26.2-1 -> 2.28.0-1)
[2009-10-15 19:35] upgraded openssh (5.2p1-1 -> 5.3p1-1)
[2009-10-15 19:35] upgraded poppler (0.10.7-2 -> 0.12.0-1)
[2009-10-15 19:35] upgraded poppler-qt (0.10.7-1 -> 0.12.0-1)
[2009-10-15 19:35] upgraded pyqt (4.6.0-1 -> 4.6.0-3)
[2009-10-15 19:35] upgraded texlive-bin (2009.4-2 -> 2009.4-3)
[2009-10-15 19:35] upgraded vte (0.20.5-1 -> 0.22.2-1)
[2009-10-15 22:06] synchronizing package lists
[2009-10-15 22:06] starting full system upgrade
[2009-10-16 00:20] installed netbeans (6.7.1-1)
[2009-10-16 15:32] synchronizing package lists
[2009-10-16 15:33] starting full system upgrade
[2009-10-16 16:28] synchronizing package lists
[2009-10-16 16:28] starting full system upgrade
[2009-10-16 16:28] removed kdebluetooth (4.0.3-2)
[2009-10-16 16:28] upgraded bluez (4.54-1 -> 4.56-1)
[2009-10-16 16:28] upgraded curl (7.19.6-1 -> 7.19.6-2)
[2009-10-16 16:28] upgraded dhcpcd (5.1.0-1 -> 5.1.1-1)
[2009-10-16 16:28] upgraded iptables (1.4.4-1 -> 1.4.5-1)
[2009-10-16 16:28] installed kbluetooth (4.0.4RC1-1)
[2009-10-16 16:28] upgraded openobex (1.4-1 -> 1.5-1)
[2009-10-16 16:28] upgraded obexd (0.17-1 -> 0.18-1)
[2009-10-16 17:52] :: In order to use p7zip with mc:
[2009-10-16 17:52] :: Add u7z to /usr/share/mc/extfs/extfs.ini
[2009-10-16 17:52] :: and add the following to /usr/share/mc/mc.ext:
[2009-10-16 17:52] ::
[2009-10-16 17:52] :: regex/\.(7z|7Z)$
[2009-10-16 17:52] :: View=%view{ascii} 7za l %f
[2009-10-16 17:52] :: Open=%cd %p#u7z
[2009-10-16 17:52] installed p7zip (9.04-1)
[2009-10-16 17:52] installed moonlight (1.0.1-5)
[2009-10-19 08:59] synchronizing package lists
[2009-10-19 09:01] synchronizing package lists
[2009-10-19 09:02] starting full system upgrade
[2009-10-19 09:04] removed foomatic-db-hpijs (20090625-1)
[2009-10-19 09:04] upgraded amarok (2.2.0-1 -> 2.2.0-2)
[2009-10-19 09:04] upgraded libcups (1.3.11-1 -> 1.4.1-1)
[2009-10-19 09:04] upgraded cups (1.3.11-1 -> 1.4.1-1)
[2009-10-19 09:04] upgraded dialog (1.1_20080819-2 -> 1.1_20080819-3)
[2009-10-19 09:04] upgraded foomatic-db (4.0_20090625-1 -> 4.0.3_20090904-2)
[2009-10-19 09:04] upgraded ghostscript (8.70-1 -> 8.70-2)
[2009-10-19 09:04] upgraded foomatic-filters (4.0_20090625-1 -> 4.0.3_20090904-2)
[2009-10-19 09:04] upgraded foomatic-db-engine (4.0_20090625-1 -> 4.0.3_20090904-2)
[2009-10-19 09:04] upgraded gzip (1.3.12-6 -> 1.3.13-1)
[2009-10-19 09:04]
[2009-10-19 09:04] NOTE
[2009-10-19 09:04] ----
[2009-10-19 09:04] # If you want to use this driver with sane:
[2009-10-19 09:04] # echo "hpaio" >> /etc/sane.d/dll.conf
[2009-10-19 09:04]
[2009-10-19 09:04]
[2009-10-19 09:04] UPGRADING
[2009-10-19 09:04] ----
[2009-10-19 09:04] # This version no longer uses an init script. You should remove hplip
[2009-10-19 09:04] # from the /etc/rc.conf daemon list.
[2009-10-19 09:04]
[2009-10-19 09:04] upgraded hplip (3.9.6b-2 -> 3.9.8-2)
[2009-10-19 09:04] upgraded imagemagick (6.5.6.1-1 -> 6.5.6.10-1)
[2009-10-19 09:05] WARNING: /boot appears to be a seperate partition but is not mounted
[2009-10-19 09:05] This is most likely not what you want. Please mount your /boot
[2009-10-19 09:05] partition and reinstall the kernel unless you are sure this is OK
[2009-10-19 09:05] >>> Updating module dependencies. Please wait ...
[2009-10-19 09:05] >>> MKINITCPIO SETUP
[2009-10-19 09:05] >>> ----------------
[2009-10-19 09:05] >>> If you use LVM2, Encrypted root or software RAID,
[2009-10-19 09:05] >>> Ensure you enable support in /etc/mkinitcpio.conf .
[2009-10-19 09:05] >>> More information about mkinitcpio setup can be found here:
[2009-10-19 09:05] >>> http://wiki.archlinux.org/index.php/Mkinitcpio
[2009-10-19 09:05]
[2009-10-19 09:05] >>> Generating initial ramdisk, using mkinitcpio. Please wait...
[2009-10-19 09:05] ==> Building image "default"
[2009-10-19 09:05] ==> Running command: /sbin/mkinitcpio -k 2.6.31-ARCH -c /etc/mkinitcpio.conf -g /boot/kernel26.img
[2009-10-19 09:05] :: Begin build
[2009-10-19 09:05] :: Parsing hook [base]
[2009-10-19 09:05] :: Parsing hook [udev]
[2009-10-19 09:05] :: Parsing hook [autodetect]
[2009-10-19 09:05] :: Parsing hook [pata]
[2009-10-19 09:05] :: Parsing hook [scsi]
[2009-10-19 09:05] :: Parsing hook [sata]
[2009-10-19 09:05] :: Parsing hook [keymap]
[2009-10-19 09:05] :: Parsing hook [usb]
[2009-10-19 09:05] :: Parsing hook [encrypt]
[2009-10-19 09:05] :: Parsing hook [filesystems]
[2009-10-19 09:05] :: Generating module dependencies
[2009-10-19 09:05] :: Generating image '/boot/kernel26.img'...SUCCESS
[2009-10-19 09:05] ==> SUCCESS
[2009-10-19 09:05] ==> Building image "fallback"
[2009-10-19 09:05] ==> Running command: /sbin/mkinitcpio -k 2.6.31-ARCH -c /etc/mkinitcpio.conf -g /boot/kernel26-fallback.img -S autodetect
[2009-10-19 09:05] :: Begin build
[2009-10-19 09:05] :: Parsing hook [base]
[2009-10-19 09:05] :: Parsing hook [udev]
[2009-10-19 09:05] :: Parsing hook [pata]
[2009-10-19 09:05] :: Parsing hook [scsi]
[2009-10-19 09:05] :: Parsing hook [sata]
[2009-10-19 09:05] :: Parsing hook [keymap]
[2009-10-19 09:05] :: Parsing hook [usb]
[2009-10-19 09:05] :: Parsing hook [encrypt]
[2009-10-19 09:06] :: Parsing hook [filesystems]
[2009-10-19 09:06] :: Generating module dependencies
[2009-10-19 09:06] :: Generating image '/boot/kernel26-fallback.img'...SUCCESS
[2009-10-19 09:06] ==> SUCCESS
[2009-10-19 09:06] upgraded kernel26 (2.6.31.3-1 -> 2.6.31.4-1)
[2009-10-19 09:06] upgraded lirc-utils (0.8.5-1 -> 0.8.6-1)
[2009-10-19 09:06] upgraded mlocate (0.22.1-1 -> 0.22.2-1)
[2009-10-19 09:06] upgraded spidermonkey (1.7.0-2 -> 1.7.0-3)
Before these I'm sure I had rebooted, going from kernel 2.6.30.6-1 -> 2.6.31.3-1 without problems. I only noticed the issue this morning as I had not used my laptop over the weekend.
Later this evening I'll try to build and install the RC and report back.
Also, as you can see below, some of these packages were pulled in because I installed gnome-2.28(No previous gnome-install), more specifically I'm looking at eggdbus and polkit.
[2009-10-13 03:07] synchronizing package lists
[2009-10-13 03:07] starting full system upgrade
[2009-10-13 03:10] upgraded glib2 (2.20.4-1 -> 2.22.2-1)
[2009-10-13 03:10] upgraded atk (1.26.0-1 -> 1.28.0-1)
[2009-10-13 03:10] installed eggdbus (0.5-1)
[2009-10-13 03:10] installed polkit (0.94-1)
[2009-10-13 03:10] upgraded consolekit (0.3.0-5 -> 0.3.1-1)
[2009-10-13 03:10] upgraded libsigc++2.0 (2.2.3-1 -> 2.2.4.2-1)
[2009-10-13 03:10] upgraded glibmm (2.20.1-1 -> 2.22.1-1)
[2009-10-13 03:10] upgraded gnome-doc-utils (0.16.1-1 -> 0.18.0-1)
[2009-10-13 03:10] upgraded pango (1.24.5-1 -> 1.26.0-1)
[2009-10-13 03:10] upgraded gtk2 (2.16.5-1 -> 2.18.2-1)
[2009-10-13 03:10] upgraded gnome-icon-theme (2.26.0-1 -> 2.28.0-1)
[2009-10-13 03:10] upgraded gtk-engines (2.18.2-1 -> 2.18.4-1)
[2009-10-13 03:10] upgraded pangomm (2.24.0-1 -> 2.26.0-1)
[2009-10-13 03:10] upgraded gtkmm (2.16.0-1 -> 2.18.2-1)
[2009-10-13 03:10] upgraded kbd (1.15-2 -> 1.15.1-1)
[2009-10-13 03:10] upgraded pango-perl (1.220-1 -> 1.221-1)
[2009-10-13 03:10] upgraded poppler (0.10.7-2 -> 0.12.0-1)
[2009-10-13 03:10] upgraded poppler-glib (0.10.7-1 -> 0.12.0-1)
[2009-10-13 03:10] upgraded pygobject (2.18.0-1 -> 2.20.0-1)
[2009-10-13 03:10] upgraded pygtk (2.14.1-4 -> 2.16.0-1)
[2009-10-13 03:10] upgraded vte (0.20.5-1 -> 0.22.2-1)
BUT: it now spits out following error-message:
Enter passphrase for /dev/sdc2:
device-mapper: remove ioctl failed: Device or resource busy
Key slot 1 unlocked.
BUT: it now spits out following error-message:
Enter passphrase for /dev/sdc2:
device-mapper: remove ioctl failed: Device or resource busy
Key slot 1 unlocked.
The device-mapper error is weird, do you have any /dev/mapper/crpytsetup-temporary-* files? Maybe we can adjust our udev rules so this doesn't happen anymore.
I've also noticed that after entering my pass-phrase my system is hit by extreme 'lag'. The console (scrolling that single line of output) gets really slow, and the music I had playing in the background got all distorted. Sometimes the 'device-mapper: remove ioctl failed: Device or resource busy' gets echoes out twice before "Key slot 1 unlocked".
However, we still don't know why 1.0.7 fails. I'm happy that this is fixed with 1.1.0rc2 though.
cryptsetup 1.1.0rc2 should stop printing that error message and 1.0.7 should work again.
EDIT: Forgot to mention: I thought I also had devicekit installed, but I haven't. This never happens in initramfs too, and I only use cryptsetup there.
EDIT2: Make sure the file name starts with 00 and ends with .rules (like 00-ignore-temp-cryptsetup.rules), otherwise it might not be executed or be executed too late.
that file you posted is the only change I made, I'm still using cryptsetup-1.0.7. Rebooted twice, works! Many thanks!
I will incorporate this into the cryptsetup package I guess. Thanks for the patience.
Patience? I think that was more on your part as dev than mine as a user :)
Thanks again!
This caused bugs in old cryptsetup versions, which were worked around by running "udevadm settle" inside cryptsetup in 1.0.6. In 1.0.7 some more clever error handling was introduced, which started to fail now due to something devicekit did. The rule prevents udev from informing devicekit (or any other program) about the device. I should have added this years ago I guess.
This is on x86-64.
Recently, I started regularly getting the following error message when opening encrypted volumes:
Buffer I/O error on device dm-x, logical block X
So I installed the new udev rule, generated my new kernel.img and performed a "udevadm trigger" which activates the new rule. (No reboot yet)
The "Buffer I/O" errors stopped, and everything was fine.
A few days later, I rebooted (or tried to).
The reboot failed, because /dev/mapper/root did not get generated. The only solution was to chroot into my encrypted installation using a Live-CD, remove the new udev rule and run mkinitcpio. The system now boots perfectly again, but the "Buffer I/O" errors are back.
I am confused though how this rule could break your setup.
- It's an old, slow system (Pentium III, 700MHz)
- Arch runs on a 4GB SD-Card (in an IDE / SD adapter)
1) Remove the custom udev rule file I posted here
2) Install latest extra/devicekit-disks, testing/udev, testing/device-mapper, testing/lvm2 (in case you have lvm) and testing/initscripts
3) Reboot
4) Confirm that all issues are solved
$ ls -l /dev/mapper/
total 0
crw-rw---- 1 root root 10, 60 2009-11-10 11:54 control
lrwxrwxrwx 1 root root 7 2009-11-10 11:54 root -> ../dm-0
lrwxrwxrwx 1 root root 7 2009-11-10 11:54 swap -> ../dm-1
lrwxrwxrwx 1 root root 7 2009-11-10 11:54 temporary-cryptsetup-1223 -> ../dm-1
$ dd if=/dev/zero of=abx.bin count=123456
$ losetup /dev/loop0 abx.bin
$ cryptsetup luksFormat /dev/loop0
$ cryptsetup luksOpen /dev/loop0 abx
$ ls -l /dev/mapper/
total 0
lrwxrwxrwx 1 root root 7 2009-11-10 13:36 abx -> ../dm-2
crw-rw---- 1 root root 10, 60 2009-11-10 13:33 control
lrwxrwxrwx 1 root root 7 2009-11-10 13:33 root -> ../dm-0
lrwxrwxrwx 1 root root 7 2009-11-10 13:33 swap -> ../dm-1
lrwxrwxrwx 1 root root 7 2009-11-10 13:33 temporary-cryptsetup-1244 -> ../dm-1
lrwxrwxrwx 1 root root 7 2009-11-10 13:36 temporary-cryptsetup-1893 -> ../dm-2
My pacman.log tail:
[2009-11-10 11:52] upgraded device-mapper (2.02.54-2 -> 2.02.54-2)
[2009-11-10 11:52] upgraded udev (146-2 -> 146-3)
[2009-11-10 11:52] upgraded devicekit-disks (009-1 -> 009-1)
[2009-11-10 11:52] upgraded initscripts (2009.08-1 -> 2009.11-1)
[2009-11-10 11:52] upgraded lvm2 (2.02.53-1 -> 2.02.54-2)
Additionally, a clean shutdown is not always possible since this change ( / is busy) causing a fsck.ext2 to be required at the next boot.
$ cryptsetup luksClose abx
$ losetup -d /dev/loop0
Can you temporarily uninstall (pacman -Rd) devicekit to see if the issue is related to that? You should exit gnome before doing that though :).
Command failed: /dev/mapper/temporary-cryptsetup-1345 open failed: No such file or directory
When the modem is not plugged during boot everything is okay.
except that in my case all three temporary-cryptsetup-xxxx symlinks point to ../dm-2
I also tried to boot with USB flash drive - same issue as with modem,
however the modem does _not_ appear as storage device, so I am not sure why the modem triggers the bug.
home -> ../dm-2
work -> ../dm-3
temporary-cryptsetup-1357 -> ../dm-3
temporary-cryptsetup-1420 -> ../dm-4
temporary-cryptsetup-1478 -> ../dm-4
Note that according to "ok" from initscripts output 'work' was unlocked,
but temporary file for it was not removed,
other two temporary files are for 'media' and 'archive' partitions,
and only those 2 files were mentioned in error messages.
The file udevadm.txt contains the results of the test with "udevadm monitor --env" when performing:
$ losetup /dev/loop0 abx.bin ; cryptsetup luksOpen /dev/loop0 abx
"Error locking luks device: timeout (10s)
waiting for cleartext device to be removed"
And the following when 'safely removing the device'(without unmounting it beforehand):
Failed to eject media; one or more volumes on the media are busy.
Checking /dev/mapper reveals:
lrwxrwxrwx 1 root root 7 2009-11-19 18:46 temporary-cryptsetup-4473 -> ../dm-0
However nautilus indicates that the partition has been locked again. Mounting/Unlocking it again will work, also unmounting will work(no errors) but that 'temporary'-link stays.
It should be available in Testing soon, in the meantime you can build your own package, here's the PKGBUILD.
However, when I delete that file I get the following message when booting:
Unlocking encrypted volumes: swap.. device-mapper: remove ioctl failed: Device or resource busy
and in dmesg:
device-mapper: ioctl: unable to remove open device temporary-cryptsetup-1249
John, did you try 1.1.0rc3 as suggested by Roman?
[2009-12-07 08:53] upgraded cryptsetup (1.0.7-1 -> 1.1.0rc3-1)
and device-mapper from testing
$pacman -Q device-mapper
device-mapper 2.02.56-1
I still get something about removing temporary-cryptsetup-* when unlocking / with rc3 cryptsetup package, but everything seems to be fine (I have 5 encrypted partitions besides / and they all unlock correctly).
I know you're doing some mkinitcpio changes and there's been some new udev releases,
are any of these still holding the update to the latest cryptsetup?
Please report how the situation is for you now with these versions.
However the 'ioctl-error/warning' + an extreme performance hit on my system during unlock are back.
This is for an external usb-drive. Also that gnome error "Error locking luks device: timeout (10s) waiting for cleartext device to be removed" when unmounting is still present (and has always been) but seems to be irrelevant.
I know the warnings are there, I am just interested if anything actually fails now.
What do you think?
Might as well be cryptsetup and not pmount.
Also, which change brought back the /dev/mapper/... listings in df output? Anyway, thanks for that, I couldn't stand the /dev/dm-X stuff.
If you do, please write in by March 6 (bug day), and if there's no confirmations by then, I propose we close this as working now.