FS#50663 - [nfs-utils] rpc-gssd.service: Inconsistency detected by ld.so: dl-close.c: 811

Attached to Project: Arch Linux
Opened by Maxime de Roucy (akira86) - Monday, 05 September 2016, 16:00 GMT
Last edited by Andreas Radke (AndyRTR) - Monday, 04 September 2017, 18:45 GMT
Task Type Bug Report
Category Upstream Bugs
Status Closed
Assigned To Andreas Radke (AndyRTR)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Description:
rpc-gssd.service fail to start.

max@max-laptop % journalctl -b -u rpc-gssd.service
-- Logs begin at mer. 2016-08-17 13:34:10 CEST, end at lun. 2016-09-05 17:52:12 CEST. --
sept. 05 17:48:04 max-laptop systemd[1]: Starting RPC security service for NFS client and server...
sept. 05 17:48:04 max-laptop rpc.gssd[529]: Inconsistency detected by ld.so: dl-close.c: 811: _dl_close: Assertion `map->l_init_called' failed!
sept. 05 17:48:04 max-laptop systemd[1]: rpc-gssd.service: Control process exited, code=exited status=127
sept. 05 17:48:04 max-laptop systemd[1]: Failed to start RPC security service for NFS client and server.
sept. 05 17:48:04 max-laptop systemd[1]: rpc-gssd.service: Unit entered failed state.
sept. 05 17:48:04 max-laptop systemd[1]: rpc-gssd.service: Failed with result 'exit-code'.

/usr/sbin/rpc.gssd start without problem when run in console (sudo /usr/sbin/rpc.gssd)

Additional info:
* nfs-utils 1.3.4-1
* the problem appear when I upgrade from nfs-utils 1.3.3-3 to 1.3.4-1

max@max-laptop % sudo systemctl cat rpc-gssd.service
# /usr/lib/systemd/system/rpc-gssd.service
[Unit]
Description=RPC security service for NFS client and server
DefaultDependencies=no
Conflicts=umount.target
Requires=var-lib-nfs-rpc_pipefs.mount
After=var-lib-nfs-rpc_pipefs.mount

ConditionPathExists=/etc/krb5.keytab

PartOf=nfs-utils.service

Wants=nfs-config.service
After=nfs-config.service

[Service]
EnvironmentFile=-/run/sysconfig/nfs-utils

Type=forking
ExecStart=/usr/sbin/rpc.gssd $GSSDARGS

# /etc/systemd/system/rpc-gssd.service.d/override.conf
[Unit]
Requires=network-online.target
After=network-online.target

Steps to reproduce:

systemctl start rpc-gssd.service
This task depends upon

Closed by  Andreas Radke (AndyRTR)
Monday, 04 September 2017, 18:45 GMT
Reason for closing:  Fixed
Comment by Andreas Radke (AndyRTR) - Monday, 05 September 2016, 17:28 GMT
root@server64:/root # journalctl -u rpc-gssd.service
-- No entries --

root@server64:/root # systemctl status rpc-gssd
● rpc-gssd.service - RPC security service for NFS client and server
Loaded: loaded (/usr/lib/systemd/system/rpc-gssd.service; static; vendor preset: disabled)
Active: inactive (dead)
Condition: start condition failed at Mo 2016-09-05 19:22:37 CEST; 5min ago
└─ ConditionPathExists=/etc/krb5.keytab was not met

krb5 auth isn't used here. My nfs4 share works well here.
Comment by Maxime de Roucy (akira86) - Tuesday, 06 September 2016, 11:54 GMT
Yes, my NFS server use Kerberos to auth user, so I need rpc.gssd to mount NFS shares on the client (archlinux).
You don't need to use kerberos to reproduce the problem ; simply try to start rpc-gssd.service.
Comment by Andreas Radke (AndyRTR) - Tuesday, 06 September 2016, 15:22 GMT
root@laptop64:/root # systemctl restart rpc-gssd.service
root@laptop64:/root #

Seems to work here. I can't reproduce your error. Is your system/mirror fully up to date? Is this i686 or x86_64?

You can try recent a git version to see if it has been fixed and feel free to git bisect from the last working version.
Comment by Maxime de Roucy (akira86) - Tuesday, 06 September 2016, 17:11 GMT
I have this issue on another archlinux install. both up to date. x86_64.
I will try to dig on this issue if I have some time.
Comment by Terje (digitus2001) - Friday, 09 September 2016, 15:32 GMT
I am also having this problem on all of my arch machines, including arm and x86_64.

What I do not understand is why rpc-gssd fails consistently and repeatedly if run under systemd, but if I start manually (as root) on commandline, it works perfectly. If I do it manually, everything starts working (accessing nfs, etc) as I would have expected...

Ideas?
Comment by Maxime de Roucy (akira86) - Friday, 09 September 2016, 20:40 GMT
I think I found the bug.

I think it's the same as this one:
https://bugzilla.redhat.com/show_bug.cgi?id=1189856

Which lead us to a glibc bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1264556

glibc bug which isn't fixed yet (from what I found).

From 1264556: "This looks like dlclose racing against exit"
Currently rpc.gssd is forking himself… that's why "exit" is called.
By using rpc.gssd option -f (foreground) we avoid the "exit" call.

So to bypass the problem I modified the "Type" and "ExecStart" options of rpc-gssd.service :

###### (I didn't found the syntax for code block)
max@max-laptop % sudo systemctl cat rpc-gssd.service
# /usr/lib/systemd/system/rpc-gssd.service
[Unit]
Description=RPC security service for NFS client and server
DefaultDependencies=no
Conflicts=umount.target
Requires=var-lib-nfs-rpc_pipefs.mount
After=var-lib-nfs-rpc_pipefs.mount

ConditionPathExists=/etc/krb5.keytab

PartOf=nfs-utils.service

Wants=nfs-config.service
After=nfs-config.service

[Service]
EnvironmentFile=-/run/sysconfig/nfs-utils

Type=forking
ExecStart=/usr/sbin/rpc.gssd $GSSDARGS

# /etc/systemd/system/rpc-gssd.service.d/override.conf
[Unit]
Requires=network-online.target
After=network-online.target

[Service]
Type=simple
ExecStart=
ExecStart=/usr/sbin/rpc.gssd -f
max@max-laptop % sudo systemctl show -p Type,ExecStart rpc-gssd.service
Type=simple
ExecStart={ path=/usr/sbin/rpc.gssd ; argv[]=/usr/sbin/rpc.gssd -f ; …
######

It works.
Comment by Allan McRae (Allan) - Saturday, 10 September 2016, 22:50 GMT
Actual glibc bug for tracking:
https://sourceware.org/bugzilla/show_bug.cgi?id=14697

Looks like this is a situation that needs upstream to fix. Not much I can do here.
Comment by Maxime de Roucy (akira86) - Sunday, 11 September 2016, 11:36 GMT
https://sourceware.org/bugzilla/show_bug.cgi?id=14697 :
Reported: 2012-10-10 18:04 UTC by Rich Felker
Modified: 2014-10-06 17:51 UTC (History)
Assignee: Not yet assigned to anyone

whoa… at this point I would say "it's not a bug… it's a feature"

I created a bug in the nfs-utils bug tracker to ask them to use the rpc.gssd "-f" option and "Type=simple" by default in their rpc-gssd.service file:
https://bugzilla.linux-nfs.org/show_bug.cgi?id=301
Comment by Allan McRae (Allan) - Saturday, 11 March 2017, 04:12 GMT
Nothing I can do at this stage from the glibc end - it looks a complicated fix... Leaving open in case nfs-utils packages want to make the suggested change in the systemd file.
Comment by Andreas Radke (AndyRTR) - Monday, 04 September 2017, 15:13 GMT
Fedora people report it working now. How's it working in Arch with recent glibc?
Comment by Maxime de Roucy (akira86) - Monday, 04 September 2017, 18:26 GMT
I just test and it works.

This issue seems to be fixed (don't know how…).
I think we can close it.

Loading...