FS#55570 - [filesystem]: nsswitch.conf with systemd for users breaks e.g. SSHD

Attached to Project: Arch Linux
Opened by Tobias Hunger (hunger) - Saturday, 09 September 2017, 18:24 GMT
Last edited by Sébastien Luttringer (seblu) - Sunday, 10 December 2017, 16:04 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To Sébastien Luttringer (seblu)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description:

I did a fresh install of Arch with the new filesystem package that comes with a new nsswitch.conf including the following lines:

passwd: compat mymachines systemd
group: compat mymachines systemd

This results in systemd-sysusers *NOT* adding the user nobody to /etc/password anymore (the user is defined in /usr/lib/sysusers.d/basic.conf). This is not a problem since e.g. 'id' still lists the user (due to systemd being listed in passwd).

*BUT* e.g. running "/usr/bin/sshd -D" coredumps. Adding nobody to /etc/passwd fixes this crash.

Additional info:
* filesystem 2017.09-3
* openssh 7.5p1-2 (with /etc/ssh/sshd_config from the package)
* glibc 2.26-3

Steps to reproduce:
* Install arch:-)
This task depends upon

Closed by  Sébastien Luttringer (seblu)
Sunday, 10 December 2017, 16:04 GMT
Reason for closing:  Not a bug
Comment by Tobias Hunger (hunger) - Saturday, 09 September 2017, 18:32 GMT
Sorry... s/systemd being listed in passwd/systemd being listed in nsswitch.conf/
Comment by loqs (loqs) - Saturday, 09 September 2017, 19:11 GMT Comment by Tobias Hunger (hunger) - Saturday, 09 September 2017, 19:17 GMT
Same if using "files" instead of "compat", so  FS#54592  does not apply.
Comment by loqs (loqs) - Saturday, 09 September 2017, 20:48 GMT
So the nobody user was failed to be created by the pacman systemd-sysusers.hook on system installation.
systemd-sysusers.service should have tried to create the service on boot provided ConditionNeedsUpdate=/etc was met
Does the system's journal record systemd-sysusers ever being run and if so did the service fail and if so with what error?
filesystem 2017.09-3 is from testing did you pacstrap the new installation with packages straight from testing or update the packages to testing later?
Comment by Tobias Hunger (hunger) - Sunday, 10 September 2017, 07:47 GMT
I do pacstrap without testing, then add testing to the installation and run pacman -Syu.

I see systemd-sysusers being run during installation. It does create groups/users defined in /usr/lib/sysusers.d as normal: It creates all users it considers to be missing on the system. "nobody" is handled already provided by the "systemd" entry in nsswitch.conf's passwd line, so "nobody" does exist for systemd-sysusers and is *not* added to /etc/password.

The crash of SSHD is probably a bug in glibc: From what I understand glibc does something before calling SSHD's main() and that crashes without "nobody" in /etc/passwd. Since "nobody" is visible to the system as a whole (e.g. "id nobody" works), that lookup should probably need to check more than /etc/passwd for the nobody user.
Comment by Tobias Hunger (hunger) - Sunday, 10 September 2017, 07:50 GMT
The crash seems similar to what is described in  FS#52129 , which was closed without a real idea what might have caused it:-)
Comment by loqs (loqs) - Sunday, 10 September 2017, 23:26 GMT
https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/filesystem&id=20928f58767d34ed6711befd6255f6a0b1706ae8
Is the commit that removes the user nobody from /etc/passwd.
So the first affected release should be 2017.08-1 so if pacstrap then upgrade to testing should not have triggered the issue.
However the issue will certainly appear should 2017.09-3 be moved out of testing.
Using systemd-sysusers with the the current /etc/nsswitch.conf as you already noted will not add the nobody user/group
Adding the user/group back by either changing /etc/nsswitch.conf or reverting to the old method of supplying an /etc/password with the nobody user and using useradd for systems with the user missing
would stop the issue from occurring but upstream openssh should probably be made aware of the issue as well.
Comment by loqs (loqs) - Monday, 11 September 2017, 00:20 GMT
I suspect sshd is failing https://github.com/openssh/openssh-portable/blob/ec9d22cc251cc5acfe7b2bcef9cc7a1fe0e949d8/sshd.c#L1648
because privsep_pw->pw_passwd as returned by libnss_systemd.so.2 is read only ( speculation need investigation )
modifying crash.c from  FS#52129  to check if pwd->pw_passwd is "*" and only bzero the memory if it is not avoids the crash.
   crash.c (0.4 KiB)
Comment by loqs (loqs) - Monday, 11 September 2017, 00:54 GMT Comment by loqs (loqs) - Monday, 11 September 2017, 21:45 GMT
http://pubs.opengroup.org/onlinepubs/9699919799/
The application shall not modify the structure to which the return value points, nor any storage areas pointed to by pointers within the structure. The returned pointer, and pointers within the structure, might be invalidated or the structure or the storage areas might be overwritten by a subsequent call to getpwent(), getpwnam(), or getpwuid(). The returned pointer, and pointers within the structure, might also be invalidated if the calling thread is terminated.
So openssh seems to be ignoring POSIX.1-2008 and modifying storage areas pointed to by pointers within the structure. While libnss_systemd.so.2 seems to be allowed by the standard to return a read only result.
Comment by loqs (loqs) - Monday, 11 September 2017, 22:04 GMT
As  FS#55589  is covering the failure to add the nobody user to /etc/passwd this issue should probably be reassigned against openssh as an upstream issue.
Comment by loqs (loqs) - Monday, 11 September 2017, 23:13 GMT
https://github.com/openssh/openssh-portable/commit/c7431341911e911993776c357ca57d1fb7ed1b23
15 year old commit appears to be the original source of writing to pw->pw_passwd
Comment by Sébastien Luttringer (seblu) - Monday, 11 September 2017, 23:29 GMT
Nice digging! I imagine he did that to improve security by removing the password from memory.
Comment by loqs (loqs) - Wednesday, 13 September 2017, 16:21 GMT

Loading...