FS#59266 - [lvm2] Dependency failed for File System Check with 2.02.179-1

Attached to Project: Arch Linux
Opened by Pryka (Pryka) - Sunday, 08 July 2018, 07:13 GMT
Last edited by Christian Hesse (eworm) - Friday, 20 July 2018, 14:55 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Christian Hesse (eworm)
Architecture x86_64
Severity Critical
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 68
Private No

Details

Description: There is an issue on boot with mounting partitions on the new LVM2 in testing:

lip 04 17:32:40 Iluvatar systemd[1]: systemd-fsck@dev-disk-by\x2duuid-990c5d4a\x2d7415\x2d4e34\x2db200\x2d451aab41469c.service: Bound to unit dev-disk-by\x2duuid-990c5d4a\x2d7415\x2d4e34\x2db200\x2d451aab41469c.device, but unit isn't active.
lip 04 17:32:40 Iluvatar systemd[1]: Dependency failed for File System Check on /dev/disk/by-uuid/990c5d4a-7415-4e34-b200-451aab41469c.
lip 04 17:32:40 Iluvatar systemd[1]: Dependency failed for /mnt/backup.
lip 04 17:32:40 Iluvatar systemd[1]: Dependency failed for Local File Systems.
lip 04 17:32:40 Iluvatar systemd[1]: local-fs.target: Job local-fs.target/start failed with result 'dependency'.
lip 04 17:32:40 Iluvatar systemd[1]: local-fs.target: Triggering OnFailure= dependencies.
lip 04 17:32:40 Iluvatar systemd[1]: mnt-backup.mount: Job mnt-backup.mount/start failed with result 'dependency'.
lip 04 17:32:40 Iluvatar systemd[1]: systemd-fsck@dev-disk-by\x2duuid-990c5d4a\x2d7415\x2d4e34\x2db200\x2d451aab41469c.service: Job systemd-fsck@dev-disk-by\x2duuid-990c5d4a\x2d7415\x2d4e34\x2db200\x2d451aab41469c.service/start failed with result 'dependency'.

Full boot log - https://pastebin.com/3bz07QrB

I'm not the only one affected, we have a topic regarding this on arch forums - https://bbs.archlinux.org/viewtopic.php?id=238554

This should be reported upstream also?

Additional info:
* lvm2 2.02.179-1 - testing.

Steps to reproduce:
Start or reboot of entire system.

PS. Downgrading LVM2 fix the issue.
This task depends upon

Closed by  Christian Hesse (eworm)
Friday, 20 July 2018, 14:55 GMT
Reason for closing:  Fixed
Additional comments about closing:  lvm2 2.02.180-1
Comment by yanlin (godspeed) - Wednesday, 11 July 2018, 03:45 GMT
I encountered the same problem

The problem still exist even i set pass=0 in fstab
Comment by Aaron Barany (akb825) - Wednesday, 11 July 2018, 04:49 GMT
Unfortunately this made it through to stable, and I encountered this same issue after updating today. Downgrading back to lvm2-2.02.177-5 worked around the problem. I have attached my fstab in case it depends on the configuration. The error occurred on my backup drive (/mnt/Backup), which is an external USB drive.
   fstab (1.4 KiB)
Comment by V.S. (vence) - Wednesday, 11 July 2018, 05:28 GMT
The same problem after the system is updated today. Fixed by downgrading lvm2 to 2.02.177-5
Comment by Jozef Babjak (cronin) - Wednesday, 11 July 2018, 06:21 GMT
I'm confirming the problem. The workaround with lvm2 package downgrade works, too.
Comment by Oscar Garcia (ogarcia) - Wednesday, 11 July 2018, 08:05 GMT
Same problem here. I attach my fstab too. In my case is with /var/cache/pacman/pkg and /home/.storage, but seems occur randomly.
   fstab (1 KiB)
Comment by Andy (andydecleyre) - Wednesday, 11 July 2018, 15:24 GMT
Thank you, same problem here. The pertinent drive is set to mount to a child of my regular user's home, with:

ext4 defaults,rw,noatime 0 1

An identical drive that mounts, with the same options, to /, does not exhibit the issue.
Comment by Andrea Amorosi (AndreaA) - Wednesday, 11 July 2018, 17:35 GMT
I have the same issue.
Boot stops in an emergency console but using ctrl-D it continues and X seems to work correctly.
Downgrading lvm2 to 2.02.177-5 solves the issue and the boot process does not stop.
Comment by John Schroeder (POINTS) - Wednesday, 11 July 2018, 20:20 GMT
I have the same issue. Downgrading lvm2 to 2.02.177-5 caused the issue to go away. Someone mentioned in a forum post that there is a new pacnew file in the config files for lvm2 but I don't believe this is the case between the two versions.

I have /home mounted to a separate drive. I believe that this is why I am affected by this issue.
Comment by Pryka (Pryka) - Wednesday, 11 July 2018, 20:25 GMT
Looks like it mainly(or only) effects secondary/external drives.
Comment by Tomas Psika (lavoisier) - Wednesday, 11 July 2018, 20:41 GMT
Same problem.
I have LVM2 PV on primary drive (/dev/sda11). And have secondary SSD drive /dev/sdb. Error occured on /dev/sdb1.
Fixed by downgrading lvm2 to 2.02.177-5.
Comment by Steve (Roken) - Wednesday, 11 July 2018, 20:41 GMT
No, it also affects remote drives (NFS). Whilst not wanting to repeat. Ctrl-D to continue boot and sudo mount -a (NFS setup in fstab) then mounts the remote drives, too.
Comment by Maciej Sobkowski (maciejjo) - Wednesday, 11 July 2018, 20:47 GMT
I don't even have LVM set up on my disks and when lvm2 is not downgraded mounting of partitions from my secondary drive fails.
Comment by loqs (loqs) - Wednesday, 11 July 2018, 21:29 GMT
@maciejjo if you mask lvm2-monitor.service and upgrade lvm2 does the issue continue?
Comment by John Schroeder (POINTS) - Wednesday, 11 July 2018, 21:51 GMT
@loqs masking lvm2-monitor.service and then upgrading lvm2 does not encounter issue. I have a similar setup where I don't believe I have LVM setup but I do mount /home on a secondary drive.
Comment by laurent85 (laurent85) - Wednesday, 11 July 2018, 22:03 GMT
Adding mount options 'noauto,x-systemd.automount' in fstab as a workaround works for partitions you don't need at boot time.
See arch forums https://bbs.archlinux.org/viewtopic.php?id=238554
Other option suggested kernel parameter fsck.mode=skip did not work for me.
Comment by Nick Stefanov (mozo) - Thursday, 12 July 2018, 09:29 GMT
Same problem here. 'noauto,x-systemd.automount' workaround is working for now.
Comment by Pryka (Pryka) - Thursday, 12 July 2018, 11:27 GMT
Anyway I don't use lvm2 either. It's just there.
Comment by Krister Bäckman (ixevix) - Thursday, 12 July 2018, 11:59 GMT
Same problem. The 'noauto,x-systemd.automount' workaround didn't seem to work however I'm using an encrypted /home partition. After dropping to rescue mode and opening it with cryptsetup and mounting it I can get access to it normally. Reading through the forum post that was linked here the problem seems to be in systemd. Edit: Downgrading fixes the problem.
Comment by Oskar (aragonix) - Thursday, 12 July 2018, 13:15 GMT
Same problem. Downgrading lvm2 to 2.02.177-5 fixed it for me.
Comment by Andy (andydecleyre) - Thursday, 12 July 2018, 21:06 GMT
For those of us not using lvm, why is the service running at all?
Comment by Pryka (Pryka) - Thursday, 12 July 2018, 21:19 GMT
Probably because lvm2 is on base group and no one(including me) bothered to turn off related systemd services and they are active directly after lvm2 installation.
Comment by Doug Newgard (Scimmia) - Friday, 13 July 2018, 02:35 GMT
  • Field changed: Severity (Medium → Critical)
This is a serious issue on a remote system, so setting it to Critical.
Comment by jam (jam) - Friday, 13 July 2018, 15:32 GMT
Related discussion at https://github.com/systemd/systemd/issues/1741
Comment by loqs (loqs) - Friday, 13 July 2018, 17:17 GMT
@jam and in general does anyone affected have in their logs systemd unmounting anything before Dependency failed....?
Which would indicate it is https://github.com/systemd/systemd/issues/1741
Comment by Andrea Amorosi (AndreaA) - Saturday, 14 July 2018, 10:20 GMT
This is my boot log
   boot_lvm2.log (172.9 KiB)
Comment by loqs (loqs) - Saturday, 14 July 2018, 10:54 GMT
Could someone affected please try https://bbs.archlinux.org/viewtopic.php?pid=1797052#p1797052
Comment by Andrea Amorosi (AndreaA) - Saturday, 14 July 2018, 13:20 GMT
This is the output of the bisect process:

57bb46c5e7f8d6af1737af5ed1acae8abc37cded is the first bad commit
commit 57bb46c5e7f8d6af1737af5ed1acae8abc37cded
Author: David Teigland <teigland@redhat.com>
Date: Thu May 3 17:12:07 2018 -0500

filter: use bcache for filter reads

Filters are still applied before any device reading or
the label scan, but any filter checks that want to read
the device are skipped and the device is flagged.

After bcache is populated, but before lvm looks for
devices (i.e. before label scan), the filters are
reapplied to the devices that were flagged above.
The filters will then find the data they need in
bcache.

:040000 040000 721531d5dfb3c46b9eb6778d168ab4924b8e5b17 43153836af55b5de10d7717006056b4b7c547007 M lib
:040000 040000 2f55881a01e084feb23bcd53e183507e24dd8af1 c25495fb0704d96a8b802e6bc48f6a935547cd52 M tools
Comment by loqs (loqs) - Saturday, 14 July 2018, 13:25 GMT
@eworm the bisect does not seem to match https://github.com/systemd/systemd/issues/1741 / https://github.com/systemd/systemd/issues/8596 / https://bugzilla.redhat.com/show_bug.cgi?id=1494014
Which was the assumed upstream issue. Do you think a new bug report should be opened with lvm2 for this issue?
Comment by Lucjan B (lucck) - Sunday, 15 July 2018, 14:19 GMT
I have similar error without LVM2 using raw devices. Arch is unable to boot randomly, with filesystem dependecy failed.

My fstab bellow:

# <file system> <dir> <type> <options> <dump> <pass>
PARTLABEL=uefi_boot /boot/efi vfat rw,noatime,discard 0 2
PARTLABEL=linux_root / ext4 defaults,noatime,discard 0 1
PARTLABEL=linux_home /home ext4 defaults,noatime,discard 0 2
tmpfs /tmp tmpfs nodev,nosuid 0 0
Comment by stargazer (bernie) - Sunday, 15 July 2018, 18:27 GMT
I have a similar problem with mount and bind in fstab, the option bind (also with option force) is broken here for mounting a hdd as home partition:

/home is not mounted or gets unmounted. Using "mount -a" after bootup is working.

# fstab:
UUID=<dev-UUID> /mnt/extra ext4 rw,defaults 0 2
# mount bind
/mnt/extra/home /home none defaults,bind 0 0

Downgrade to 2.02.177-5 helps : device-mapper + lvm2
Comment by stargazer (bernie) - Sunday, 15 July 2018, 18:29 GMT
The device/hdd is also without LVM.
Comment by Jeff Sapire (Jeffus) - Monday, 16 July 2018, 14:58 GMT
Downgrade lvm2 "solves" the problem
Comment by Heorhii Valakhanovich (tormoz) - Monday, 16 July 2018, 16:26 GMT
Oh. This even breaks system without lvm volumes. I had to downgrade lvm to solve "/boot not mounted" issue and I have no lvm volumes.
Comment by loqs (loqs) - Monday, 16 July 2018, 19:45 GMT
Can anyone else affected confirm AndreaA's bisection result that 39ce38eb8868e367fa796ddb12b1020647d14c63 is the last good commit
and 57bb46c5e7f8d6af1737af5ed1acae8abc37cded is the first bad commit?
Comment by loqs (loqs) - Monday, 16 July 2018, 20:49 GMT
Please stop with the me too posts or repeating already mentioned observations such as it affects systems not using lvm2.
Comment by loqs (loqs) - Monday, 16 July 2018, 20:58 GMT
@iyanmv https://bugs.archlinux.org/task/59266#comment171211 or https://bbs.archlinux.org/viewtopic.php?pid=1797357#p1797357 for all the known solutions so far.
I would rather new information is not lost in repetition of already noted information.
Edit:
r418 is the last good commit to test r419 is the first bad commit you only need to install the lvm2 package not the device-mapper package as well
(application/x-gzip)    lvm2-2.02.177.r418.g39ce38eb8... (4 KiB)
(application/x-gzip)    lvm2-2.02.177.r419.g57bb46c5e... (4 KiB)
Comment by circleface (circleface) - Tuesday, 17 July 2018, 02:50 GMT
@loqs I can confirm the above. 57bb46c5e7f8d6af1737af5ed1acae8abc37cded is the first commit that has the error at startup.
Comment by Peter B (kinoe) - Tuesday, 17 July 2018, 06:39 GMT
updating kernel from 4.17.4-1 to 4.17.5-1 seems to resolve the issue!
Comment by Christian Neubauer (bauner) - Tuesday, 17 July 2018, 08:16 GMT
@Peter B did you mean the kernel update from today 4.17.5-1 -> 4.17.6-1 ??
The kernel update did not fix the problem for me
Comment by loqs (loqs) - Tuesday, 17 July 2018, 09:19 GMT
Thank you circleface and AndreaA as you both found the same commit to be responsible I would report the issue to upstream lvm2 as a new bug http://www.sourceware.org/lvm2/
Comment by Christian Hesse (eworm) - Tuesday, 17 July 2018, 22:50 GMT
So far I could not reproduce this on any of my own systems.
Anybody affected please open an upstream bug report. Thanks!

Sadly the found commit is quite huge and can not be reverted easily.

Did anybody try git master to test if the issue has been solved already?
Comment by patrick (potomac) - Tuesday, 17 July 2018, 23:37 GMT
@Christian : how many hard drives do you have on your system ?
it seems that the bug will be triggered if the PC has several hard drives and/or several type of partitions (ext4, ntfs),

those who have a laptop with only one hard drive may not see the bug
Comment by loqs (loqs) - Wednesday, 18 July 2018, 00:29 GMT
PKGBUILD for lvm2 git master needed changes because both lvmetad and applib support has been removed,
also the current commit needed configure to be regenerated with autoreconf so it would not fail from missing lvmedtad
This is only build tested.
(application/x-gzip)    lvm2-2.02.178.rc1.r275.g63ec4... (4 KiB)
Comment by Christian Hesse (eworm) - Thursday, 19 July 2018, 19:24 GMT
Anybody wants to test version 2.02.180?
Comment by Matthias Schabhüttl (msschabh) - Thursday, 19 July 2018, 19:31 GMT
@Christian official release? or testing?
Comment by loqs (loqs) - Thursday, 19 July 2018, 19:40 GMT
2.02.180 = lvm2 2.02.180-1 currently in testing
Comment by Heorhii Valakhanovich (tormoz) - Thursday, 19 July 2018, 19:49 GMT
Got several successful boots with 2.02.180-1 from testing. Looks like bug was fixed.

Even my old "less stable" /etc/fstab (with label-based mounting and enabled fsck) works
Comment by loqs (loqs) - Thursday, 19 July 2018, 20:32 GMT
Perhaps https://sourceware.org/git/?p=lvm2.git;a=commit;h=a30e6222799409ab6e6151683c95eb13f4abaefb was the fix then.
Comment by circleface (circleface) - Thursday, 19 July 2018, 21:53 GMT
lvm2 2.02.180-1 from testing seems to work fine.
Comment by Simon (Giggi) - Friday, 20 July 2018, 11:57 GMT
Tested the package lvm2 2.02.180-1 from testing; It works!

Loading...