Arch Linux

Please read this before reporting a bug:
https://wiki.archlinux.org/index.php/Reporting_Bug_Guidelines

Do NOT report bugs when a package is just outdated, or it is in the AUR. Use the 'flag out of date' link on the package page, or the Mailing List.

REPEAT: Do NOT report bugs for outdated packages!
Tasklist

FS#70272 - Upgrade to systemd 248-1 completely breakes LUKS using systemd-cryptsetup

Attached to Project: Arch Linux
Opened by Nikita (Slenderchat) - Saturday, 03 April 2021, 00:36 GMT
Last edited by Andreas Radke (AndyRTR) - Saturday, 03 April 2021, 07:33 GMT
Task Type Bug Report
Category Packages: Core
Status Assigned
Assigned To Christian Hesse (eworm)
Architecture x86_64
Severity Critical
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 0%
Votes 8
Private No

Details

Description:
Suddenly after systemd upgrade my server and laptop, which is LUKS encrypted and uses systemd-cryptsetup with keys in initramfs and root filesystem to auto-unlock, started to boot to emergency shell (photo of output will be attached). I started to investigate the problem, and find out that culprit was systemd-cryptsetup@root unit. Logs for that unit contained this error: "Failed to load key file "root.key": Argument list too long". After i extracted keys from initramfs file and chrooted to my broken systems, I decided to downgrade systemd to 247.4-2. After it was done and after initcpios was updated, both systems booted succesfully. This is the only change which was done and it helped, so the culprit is systemd 248-1. Seems to be, that FS#70264 is related and has the same culprit.

Steps to reproduce:
1. Setup system with LUKS full disk encryption which uses systemd-cryptsetup to auto-unlock partitions on boot with keyfiles in initcpios;
2. Make sure that you've replaced udev hook with systemd hook in mkinitcpio.conf, added sd-encrypt hook, and included /etc/cryptsetup-keys.d/root.key in FILES array;
3. Make sure, that you use systemd >= 248;
4. Try to boot.

P.S. Unfortunately I only captured output of server, which has "quiet" in kernel parameters. All other information was gained on laptop by removing "quiet" option, and by including /etc/shadow in initcpio's to get emergency shell access.
This task depends upon

Comment by xgjmibzr (xgjmibzr) - Saturday, 03 April 2021, 02:12 GMT
I had a similar bug, but on a secondary disk rather than root.
Error was "Apr 02 17:37:32 arch-server systemd-cryptsetup[247]: Failed to activate with key file '/etc/keys/[UUID].key': Invalid argument."

Downgraded to 247.4-2 and boots without error.
Comment by Robert (robson) - Saturday, 03 April 2021, 11:32 GMT
And did you install libfido2, which is an optional dependency for systemd 248?
Comment by Frederick Zhang (FrederickZh) - Saturday, 03 April 2021, 12:07 GMT
Similar issue here. While my main disk is opened via SSH hence unaffected, other additional ones all failed with

> Failed to activate with key file '/etc/crypttab.key': Invalid argument

I filed a ticket upstream: https://github.com/systemd/systemd/issues/19193
Comment by Vince (Heronymous) - Saturday, 03 April 2021, 17:59 GMT
I wanted to throw my hat in and confirm that I also have the same issue with systemd 248-1. Downgrading to systemd 247.3-1 fixed my issue (and only this downgrade was needed).
Comment by Vince (Heronymous) - Saturday, 03 April 2021, 19:39 GMT
I wanted to throw my hat in and confirm that I also have the same issue with systemd 248-1. Downgrading to systemd 247.3-1 fixed my issue (and only this downgrade was needed).
Comment by Nikita (Slenderchat) - Saturday, 03 April 2021, 21:27 GMT
@Robert Read the description for libfido2, it’s needed only for usb devices.
Comment by Robert (robson) - Saturday, 03 April 2021, 21:40 GMT
@Nikita Well, take a look at this https://i.postimg.cc/QMpWx4wz/screen.png
Comment by xgjmibzr (xgjmibzr) - Saturday, 03 April 2021, 22:14 GMT
@Robert, take a look at this https://developers.yubico.com/libfido2/

> libfido2 provides library functionality and command-line tools to communicate with a FIDO device over USB, and to verify attestation and assertion signatures.
Comment by Kawai Tomato (kawaitomato) - Sunday, 04 April 2021, 05:17 GMT
I got a similar problem with my computer Hasee Z6-CT5NA. I use LVM on LUKS to encrypt my disk, and use systemd-boot as boot manager. After upgrade systemd from version 247 to version 248-3(I haven't upgraded for a few days), I couldn't boot my system. The screenshot: https://www.kawaitomato.com/files/problem.jpg
Comment by MMH (mmh) - Monday, 05 April 2021, 22:32 GMT
I get "Failed to load key file 'disk.key': Argument list too long" like the author but I don't see any log messages related to "Invalid argument".
Are those two different issues?
Comment by Aaron Maslen (microsoftenator) - Tuesday, 06 April 2021, 04:00 GMT
@mhh: I'd guess that's a separate issue.

The bug appears to be in systemd-cryptsetup, specifically the "Invalid Argument" error message appears to originate in the code that reads the keyfile from the disk - https://github.com/systemd/systemd/blob/34fde9f898f63096262d95c61d75db85dabe6fe4/src/cryptsetup/cryptsetup.c#L1222

My guess would be that the bug is triggered by the crypt_activate_by_passphrase function, but I don't know the codebase well enough to attempt to debug further than this.

I can confirm that this issue also occurs for me with systemd version 248 and that downgrading to 247.4-2 works.
Comment by Kevin (kp7) - Wednesday, 07 April 2021, 09:08 GMT
I have encountered the same bug.
My system is configured to unlock the encrypted /home partition (which is on a secondary drive) automatically with a keyfile - /etc/cryptsetup-keys.d/home.key.
After upgrading to systemd-248, the system boots in emergency mode. However, I am able to correctly mount my home partition using both the said keyfile and the passphrase.
The issue is not present anymore after downgrading to systemd-247
Comment by Simon Paul (smnpl) - Sunday, 11 April 2021, 18:24 GMT
Same here as Kevin. A couple of devices that should get unlocked by a keyfile. Broke with 248 and works with 247.
Right now I downgraded, but that is very unfortunate for a systems core component.

Is this already filed as bug at the systemd github repo?

Loading...