FS#35811 - [networkmanager][systemd] race condition at shutdown with mounted NFS drive

Attached to Project: Arch Linux
Opened by Stefan Förster (HotblackDesiato) - Sunday, 16 June 2013, 06:59 GMT
Last edited by Dave Reisner (falconindy) - Tuesday, 24 December 2013, 21:06 GMT
Task Type Bug Report
Category System
Status Closed
Assigned To Jan Alexander Steffens (heftig)
Dave Reisner (falconindy)
Tom Gundersen (tomegun)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 3
Private No



I recently recently switched my three Arch Linux machines to systemd. All three machines mount an NFS drive (/media/data) within my LAN on startup. Two of the machines are notebooks. That's why I use NetworkManager to manage network access. The third machine is a desktop. On this machine I use netcfg for networking.

At shutdown, I get on both notebooks the following race condition:

[** ] A stop job is running for /media/dataNetworkManager[1716]: <error> [1371362286.799529] [nm-dbus-manager.c:278] nm_dbus_manager_init_dbus(): Could not get the system bus.
Make sure the message bus daemon is running! Message: failed to connect to socket /run/dbus/system_bus_socket: Verbindungsaufbau abgelehnt

("Verbindungsaufbau abgelehnt" means "connection refused".)

I get hundreds of the same error messages, only the number in brackets [] changes, with the "[*** ]" at the beginning of each line in red. It seems that some processes can't handle the unmounting of the NFS drive properly and run into a race condition.

This error appeared first when I switched to systemd and is 100% reproducible.

When I unmount the drive /media/data manually before shutdown, the shutdown is flawless.

On the desktop where I use netcfg instead of NetworkManager, the shutdown is also flawlees, with mounted NFS drive.

Thus it seems that the shutdown procedure of NetworkManager is buggy.

Additional info:
The machines are up-to-date (June 16):

extra/network-manager-applet [installed]
extra/networkmanager [installed]
community/networkmanager-dispatcher-netfs 1.0-2 [installed]
core/systemd 204-3 [installed]
core/systemd-sysvcompat 204-3 (base) [installed]
core/nfs-utils 1.2.8-6 [installed]

Steps to reproduce:

Connect to a network via NetworkManager and mount a NFS drive. Try to shutdown the machine w/o unmounting the NFS drive first.
This task depends upon

Closed by  Dave Reisner (falconindy)
Tuesday, 24 December 2013, 21:06 GMT
Reason for closing:  Duplicate
Additional comments about closing:   FS#35671 
Comment by Stefan Förster (HotblackDesiato) - Sunday, 16 June 2013, 07:03 GMT
picture of the error messages
Comment by Aaron Barany (akb825) - Thursday, 04 July 2013, 20:09 GMT
I am also experiencing this problem. I only occasionally get the error spew, but it always hangs on shutdown.

The problem appears to be a cycle in the shutdown process, which systemd will break by removing one of the services. I have attached the output of journalctl during shutdown to show what it is doing: it is breaking the cycles by removing the NFS mount stop services. This causes the shutdown to hang since it ends up shutting down the network before something else tries to unmount the NFS drives.

My guess is when it spews dbus errors it's because of a different service being removed to break the cycle.
Comment by Jan Alexander Steffens (heftig) - Monday, 08 July 2013, 09:41 GMT
What does your fstab look like? local-fs.target shouldn't depend on a network mount, that's what remote-fs.target is for.
Comment by Stefan Förster (HotblackDesiato) - Tuesday, 09 July 2013, 00:00 GMT
Here is the fstab of one of my three Arch Linux machines. They are all identical iro NFS drives.

# /etc/fstab: static file system information
# <file system> <dir> <type> <options> <dump> <pass>
tmpfs /tmp tmpfs nodev,nosuid 0 0
UUID=1722928c-e30e-4b79-96b8-ad3c12324fd0 swap swap defaults 0 0
UUID=605efaf2-a19b-479d-8d0f-e5e02fcf8368 /boot ext4 defaults,data=writeback,noatime 0 1
UUID=72695d97-55f4-4493-92d6-086e005b0eea / btrfs defaults,ssd,ssd_spread,noatime,compress 0 1 /media/data nfs rw,suid,dev,exec,auto,user,sync,_netdev,rsize=32768,wsize=32768,hard,intr,timeo=4,retrans=5 0 0

/dev/sr0 /media/cdrom auto noauto,user 0 0
/dev/sr0 /media/dvd auto noauto,user 0 0
Comment by Aaron Barany (akb825) - Tuesday, 09 July 2013, 08:17 GMT
I have attached my fstab.
   fstab (1.1 KiB)
Comment by Jiri Kraml (hellsheep) - Wednesday, 10 July 2013, 06:24 GMT
I had the same problem when mounting an NFS share via KDE's dolphin.
adding "x-systemd.automount" in /etc/fstab solved it.
excerpt from my fstab: /network/zw106 nfs defaults,noauto,user,intr,_netdev,x-systemd.automount 0 0

The option is supposed to work like autofs - mounting the folder on demand, so perhaps the user option is not necessary, but I was too lazy to try it out

EDIT: this option breaks most KDE applications when the server is not available
Comment by Stefan Förster (HotblackDesiato) - Thursday, 11 July 2013, 23:04 GMT
Jiri, thanks for the tip! I tried it on one of the affected systems (fluxbox, no KDE) and it works. I'll use"x-systemd.automount" on all my system for further tests,
Comment by Balló György (City-busz) - Friday, 16 August 2013, 23:46 GMT
The x-systemd.automount option doesn't solve the problem for me. I tried various mount options for NFS, but none of them work. The shutdown hangs for minutes many times.
Comment by Dave Reisner (falconindy) - Tuesday, 24 December 2013, 21:06 GMT
I suspect this is probably related to  FS#35671