FS#44658 - [systemd] immediately unmounts entries not configured in fstab

Attached to Project: Arch Linux
Opened by Andrea Barisani (lcars-ip) - Monday, 20 April 2015, 09:28 GMT
Last edited by Dave Reisner (falconindy) - Sunday, 03 May 2015, 20:45 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Thomas Bächler (brain0)
Dave Reisner (falconindy)
Tom Gundersen (tomegun)
Architecture All
Severity Critical
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 11
Private No

Details

Description: systemd immediately unmounts entries not configured in fstab

The recent systemd update (219-5) seems to be affected by this bug:

https://bugzilla.redhat.com/show_bug.cgi?id=1196452

This prevents manually mounted ramdisks and/or encrypted partition to be correctly mounted, as they get unmounted right away by systemd.

Relevant log example for encrypted partition (managed via cryptmount):

cryptmount[756]: mount of "encdisk" by user succeeded
systemd[168]: Unit mnt-encdisk.mount is bound to inactive unit. Stopping, too.
systemd[168]: Unmounting /mnt/encdisk...
systemd[1]: Unit mnt-encdisk.mount is bound to inactive unit. Stopping, too.
systemd[1]: Unmounting /mnt/encdisk...
systemd[1]: Unmounted /mnt/encdisk.
umount[800]: umount: /mnt/encdisk: not mounted
systemd[168]: mnt-encdisk.mount mount process exited, code=exited status=1
systemd[168]: Unmounted /mnt/encdisk.
systemd[168]: Unit mnt-encdisk.mount entered failed state.
This task depends upon

Closed by  Dave Reisner (falconindy)
Sunday, 03 May 2015, 20:45 GMT
Reason for closing:  Fixed
Additional comments about closing:  mostly fixed by systemd-219-6. Remaining issues are addressed upstream.
Comment by Fredrik (vEX) - Monday, 20 April 2015, 19:24 GMT
It seems like plymouth is required to be able to enter the passphrase for encrypted disks.

apr 20 21:02:57 odin systemd[444]: Failed at step EXEC spawning /bin/plymouth: No such file or directory
-- Subject: Process /bin/plymouth could not be executed

Debian bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=779612
Ubuntu bug report: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1432265

I'm so glad that I have only encrypted my home partition so I could downgrade all systemd packages (systemd, libsystemd, systemd-sysvcompat) and have a working system again.
Comment by Rick Klement (tybalt89) - Monday, 20 April 2015, 19:59 GMT
This fails for me doing a loop mount of an iso image file.

Configuring fstab did not help.
Comment by Ramon Buldó (kirek) - Tuesday, 21 April 2015, 03:06 GMT
Had a similar issue when mounting squashfs files as loop device, they where immediately unmounted, or not mounted at all.
This patch solved it http://cgit.freedesktop.org/systemd/systemd/commit/src/core/mount.c?id=628c89cc68ab96fce2de7ebba5933725d147aecc
Comment by l3iggs (l3iggs) - Tuesday, 21 April 2015, 21:28 GMT
This is nasty. 219 should be rolled back immediately because of this. By the way, I'm not manually mounting a ramdisk and/or an encrypted partition and my mounts are getting clobbered by this bug.
Comment by Dave Reisner (falconindy) - Wednesday, 22 April 2015, 01:10 GMT
Please try systemd-219-6 from [testing].
Comment by M. Greyson Christoforo (greyltc) - Wednesday, 22 April 2015, 08:28 GMT
Using systemd-219-6 solves the issue for me.
EDIT: Still major issues. See my new comment below.
Comment by Andrea Barisani (lcars-ip) - Wednesday, 22 April 2015, 09:03 GMT
Just upgraded to testing/systemd-219-6 and I can mount just fine now, no other issues witnessed so far. Thanks!
Comment by M. Greyson Christoforo (greyltc) - Wednesday, 22 April 2015, 09:33 GMT
There are still major issues here (systemd-219-6). Nested unmounting is broken (only tested with btrfs subvolumes, see below). systemd seems to try to unmount parent and other related child mounts when it should not.

Steps to reproduce:
# mount /dev/sda -o subvol=root /mnt
# cd /mnt
# mount /dev/sda -o subvol=A /mnt/nestedMountA
# mount /dev/sda -o subvol=B /mnt/nestedMountB
# umount /mnt/nestedMountA

Now watch as your log is filled with a never-ending stream of messages from systemd and umount (because systemd is furiously trying to unmount your current work directory for some unknown reason):
...
Apr 22 11:12:50 rack umount[11465]: umount: /mnt: target is busy
Apr 22 11:12:50 rack systemd[1]: Failed unmounting /mnt.
...
It appears as though manually unmounting /mnt/nestedMountA causes systemd to attempt to unmount the whole tree, including /mnt and /mnt/nestedMountB.

systemd breaking mount/unmount is really quite bad. Please put 219 back into [testing] until this is ironed out.
Comment by Dave Reisner (falconindy) - Wednesday, 22 April 2015, 12:43 GMT
Please report this upstream.
Comment by M. Greyson Christoforo (greyltc) - Wednesday, 22 April 2015, 13:37 GMT
Reported upstream: https://bugs.freedesktop.org/show_bug.cgi?id=89383#c5

In the mean time, while upstream gets their shit together on this, something should be done to get it out of Arch.
Comment by Rick Klement (tybalt89) - Wednesday, 22 April 2015, 16:16 GMT
I agree with greyitc. Fix Arch now. Perhaps systemd has gotten too complicated for the systemd folks :(

Let's not wait...
Comment by Bastien Traverse (Neitsab) - Thursday, 23 April 2015, 09:50 GMT
Just updated a home server which is full disk encrypted with LUKS and remotely unlocked via dropbear_initrd_encrypt. I am locked out of it with the following error messages:

- on client trying to ssh in dropbear: "Device /dev/disk/by-id/wwn-<redacted_id>-part3 doesn't exist or access denied."
- on server (old laptop repurposed as server so it has a screen):

(dropbear initialization sequence)
Starting dropbear
[123] Apr 23 09:43:56 Running in background
(try to connect remotely)
Pubkey auth succeeded for 'root' with key XXX from 192.XXX
syslogin_perform_logout: logout(pts/0) returned an error: No such file or directory
Exit (root) Disconnect received
ERROR: device '/dev/mapper/lvm-archroot' not found. Skipping fsck.
ERROR: Unable to find rot device '/dev/mapper/lvm-archroot'
You are being dropped to a recovery shell

If I don't try to login via SSH there is a 15 seconds delay between [123] line and the first ERROR; there is no mean whatsoever to unlock dropbear locally as used to be the case ("enter passphrase for /dev/disk/by-id/wwn-<redacted_id>-part").

I suggest withdrawing affected versions of systemd-219 to testing until this is fixed upstream.
Comment by Rick Klement (tybalt89) - Friday, 24 April 2015, 14:16 GMT
I just tried the new systemd-219-6 in core.

It took seven attempts before it succeeded. It is still a critical problem.
Comment by Rick Klement (tybalt89) - Wednesday, 29 April 2015, 20:17 GMT
I upgraded to linux 4.0.1-1 and my problems with the loopback mount have gone away. hmmm...
Comment by Fredrik (vEX) - Friday, 01 May 2015, 06:45 GMT
Just gave it a new try, updated all packages and rebooted. I was able to enter the passphrase for one of the encrypted drives before it failed and asked for the root password to enter maintenance mode.

maj 01 08:33:54 odin systemd[1]: proc-bus-usb.mount mount process exited, code=exited status=32
maj 01 08:33:54 odin systemd[1]: Failed to mount /proc/bus/usb.
maj 01 08:33:54 odin systemd[1]: Dependency failed for Local File Systems.
maj 01 08:33:54 odin systemd[1]: Job local-fs.target/start failed with result 'dependency'.
maj 01 08:33:54 odin systemd[1]: Triggering OnFailure= dependencies of local-fs.target.
maj 01 08:33:54 odin systemd[1]: Unit proc-bus-usb.mount entered failed state.
maj 01 08:33:54 odin systemd[1]: Stopped Getty on tty1.
maj 01 08:33:54 odin systemd[1]: Stopped target Graphical Interface.
maj 01 08:33:54 odin systemd[1]: Stopped SLiM Simple Login Manager.
maj 01 08:33:54 odin systemd[1]: Stopped target Multi-User System.
maj 01 08:33:54 odin systemd[1]: Stopped Permit User Sessions.

and a bunch of more lines before entering emergency mode

maj 01 08:33:54 odin systemd[1]: Reached target Emergency Mode.
maj 01 08:33:54 odin systemd[1]: Starting Emergency Mode.

I have an entry in fstab for /proc/bus/usb used by VirtualBox:

# VirtualBox USB-support
none /proc/bus/usb usbfs auto,busgid=108,busmode=0775,devgid=108,devmode=0664 0 0

This is probably some old remnant that isn't needed anymore. I will comment it out and give it another try and report back.

- EDIT -
Yep, removing that line and using the latest systemd works just fine now!
Comment by Bastien Traverse (Neitsab) - Sunday, 03 May 2015, 16:20 GMT
No problem with booting a full disk encrypted *laptop* after the last complete upgrade, however on a **remotely unlocked** headless machine the problem persists and makes it unbootable. After downgrading {systemd,libsystemd,systemd-sysvcompat} to 218-2 and regenarating initramfs I could successfully boot the machine.

Using latest linux-4.0.4-1 or linux-lts-3.14.43-1 and systemd-219-6 gave the same result on the remotely unlocked machine.

Edit: update description of problematic use case
Comment by smradlev (smradlev) - Sunday, 03 May 2015, 19:42 GMT
I also confirm that I no longer have problems booting with LVM on LUKS with linux 4.0.1-1 and systemd 219-6.

Loading...