FS#46490 - [systemd] Custom NOFILE limits not effective after upgrade to systemd-226

Attached to Project: Arch Linux
Opened by Tassilo Horn (tsdh) - Wednesday, 30 September 2015, 06:27 GMT
Last edited by Dave Reisner (falconindy) - Friday, 19 February 2016, 14:34 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Thomas Bächler (brain0)
Dave Reisner (falconindy)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 5
Private No

Details

Description:

After having upgraded from systemd 225 to 226 (now 226-3) last week, I got "Too many open file" errors when working on some LaTeX document which includes thousands of other files. A simpler test case is

printf 'exec &%d>/dev/null\n' {3..9999} | sh

which requires at least a nofile limit of 10000 in order not to signal "Too many open files" errors.

The reason is that my custom NOFILE limits I set in /etc/security/limits.d/98-my-limits.conf containing

horn soft nofile 500000
horn hard nofile 900000

where horn is my user name don't become effective anymore:

% ulimit -a
-t: cpu time (seconds) unlimited
-f: file size (blocks) unlimited
-d: data seg size (kbytes) unlimited
-s: stack size (kbytes) 8192
-c: core file size (blocks) 0
-m: resident set size (kbytes) unlimited
-u: processes 63409
-n: file descriptors 1024
-l: locked-in-memory size (kbytes) 1024
-v: address space (kbytes) unlimited
-x: file locks unlimited
-i: pending signals 63409
-q: bytes in POSIX msg queues 819200
-e: max nice 20
-r: max rt priority 0
-N 15: unlimited

% ulimit -Hn
4096

% ulimit -Sn
1024

In theory, PAM should be responsible for setting the limits for users, so I added the debug option to the PAM services requiring pam_limits.so:

% grep pam_limits /etc/pam.d/*
/etc/pam.d/crond:session required pam_limits.so debug
/etc/pam.d/system-auth:session required pam_limits.so debug
/etc/pam.d/system-services:session required pam_limits.so debug

With that, journalctl actually reveals that my nofile limits seem to be actually applied when I login:

Sep 30 08:06:49 thinkpad-t440p systemd[1]: Created slice user-1000.slice.
Sep 30 08:06:49 thinkpad-t440p systemd[1]: Starting User Manager for UID 1000...
Sep 30 08:06:49 thinkpad-t440p systemd-logind[462]: New session c2 of user horn.
Sep 30 08:06:49 thinkpad-t440p systemd[1]: Started Session c2 of user horn.
Sep 30 08:06:49 thinkpad-t440p systemd[740]: pam_limits(systemd-user:session): reading settings from '/etc/security/limits.conf'
Sep 30 08:06:49 thinkpad-t440p systemd[740]: pam_limits(systemd-user:session): process_limit: processing - rtprio 0 for DEFAULT
Sep 30 08:06:49 thinkpad-t440p systemd[740]: pam_limits(systemd-user:session): process_limit: processing - nice 0 for DEFAULT
Sep 30 08:06:49 thinkpad-t440p systemd[740]: pam_limits(systemd-user:session): checking if horn is in group audio
Sep 30 08:06:49 thinkpad-t440p systemd[740]: pam_limits(systemd-user:session): checking if horn is in group audio
Sep 30 08:06:49 thinkpad-t440p systemd[740]: pam_limits(systemd-user:session): checking if horn is in group audio
Sep 30 08:06:49 thinkpad-t440p systemd[740]: pam_limits(systemd-user:session): reading settings from '/etc/security/limits.d/10-gcr.conf'
Sep 30 08:06:49 thinkpad-t440p systemd[740]: pam_limits(systemd-user:session): checking if horn is in group users
Sep 30 08:06:49 thinkpad-t440p systemd[740]: pam_limits(systemd-user:session): process_limit: processing - memlock 1024 for GROUP
Sep 30 08:06:49 thinkpad-t440p systemd[740]: pam_limits(systemd-user:session): reading settings from '/etc/security/limits.d/98-my-limits.conf'
Sep 30 08:06:49 thinkpad-t440p systemd[740]: pam_limits(systemd-user:session): process_limit: processing soft nofile 500000 for USER
Sep 30 08:06:49 thinkpad-t440p systemd[740]: pam_limits(systemd-user:session): process_limit: processing hard nofile 900000 for USER
Sep 30 08:06:49 thinkpad-t440p systemd[740]: pam_limits(systemd-user:session): reading settings from '/etc/security/limits.d/99-audio.conf'
Sep 30 08:06:49 thinkpad-t440p systemd[740]: pam_limits(systemd-user:session): checking if horn is in group audio
Sep 30 08:06:49 thinkpad-t440p systemd[740]: pam_limits(systemd-user:session): checking if horn is in group audio
Sep 30 08:06:49 thinkpad-t440p systemd[740]: pam_unix(systemd-user:session): session opened for user horn by (uid=0)

However, even though the log says they were set, uname still shows the 1024/4096 soft/hard limits. So either PAM's setting did not work for some reason or something else lowers the limits again after PAM has had its turn.

As a workaround, I now set DefaultLimitNOFILE=500000 in /etc/systemd/system.conf which is obviously now what I want to do but at least it lets me work again on the computer.

Additional info:
* package version(s)
* config and/or log files etc.


Steps to reproduce:
1. Set non-default increased nofile limits in /etc/security/limits.conf or /etc/security/limits.d/NN-your-limits.conf for your user.
2. Reboot and check if the custom limits are effective.
This task depends upon

Closed by  Dave Reisner (falconindy)
Friday, 19 February 2016, 14:34 GMT
Reason for closing:  Upstream
Comment by Tassilo Horn (tsdh) - Monday, 19 October 2015, 05:45 GMT
The issue is still present in core/systemd 227-1.
Comment by George (Vash63) - Tuesday, 01 December 2015, 01:55 GMT
I can confirm this bug is happening to me also. I need higher than the 1024 default for Unity engine games and would rather have a user-specific solution here.
Comment by BoBeR182 (BoBeR182) - Friday, 22 January 2016, 18:43 GMT Comment by Tassilo Horn (tsdh) - Friday, 22 January 2016, 19:20 GMT
That upstream thread is not about this issue.
Comment by Cameron Matheson (cmatheson) - Wednesday, 03 February 2016, 06:52 GMT
Additional info:

logging in at the console or ssh *does* respect the limits set in limits.conf.

logging in with gdm though limits my user to 1024 (can't tell where that limit comes from).

Relevant log lines:

systemd[1968]: pam_systemd(systemd-user:session): pam-systemd initializing
gdm-password][1910]: pam_systemd(gdm-password:session): Reply from logind: id=c2 object_path=/org/freedesktop/login1/session/c2 runtime_path=/run/user/NNNN session_fd=14 seat=seat0 vtnr=2 original_uid=NNNN
/usr/lib/gdm/gdm-x-session[2047]: X.Org X Server 1.18.0

I *think* gdm-x-session must be at fault because the other login methods look the same.
Comment by Cameron Matheson (cmatheson) - Wednesday, 03 February 2016, 07:46 GMT
OK, I'm back:

I've been checking the nofile limit on various processes using prlimit. Anything under gdm-x-session actually does have the correct limits (this includes gnome-session-binary and anything that is autostarted by gnome). *However*, anything I start in the gnome session does *not* have the correct limits. Those process are not children of the gdm-x-session/gnome-shell though. They are children of my user systemd session (systemd --user). I'm not sure what starts that thing (its parent PID is 1). It has the bad limits.

I'm not sure if the problem is that the systemd user session has the wrong limits, or that everything is getting started as a child of the systemd session.
Comment by Tassilo Horn (tsdh) - Friday, 19 February 2016, 01:56 GMT
  • Field changed: Percent Complete (100% → 0%)
The issue is NOT solved by systemd 229. However, I can confirm cmatheson's comments. On a Linux console, the NOFILE limits I configured in /etc/security/limits.d/98-my-limits.conf are active. However, apps started from within GNOME (which in turn was started by GDM) have again NOFILE limits of 1024.
Comment by Dave Reisner (falconindy) - Friday, 19 February 2016, 02:51 GMT
Then isn't this a problem of GDM or your pam setup?
Comment by Tassilo Horn (tsdh) - Friday, 19 February 2016, 06:28 GMT
Dave, you might very well be correct in that it's not systemd's fault. However, I don't have any special GDM or PAM configurations. From what I can see in /etc/pam.d, the cron, system-auth, and system-services require pam_limits.so (which I guess would apply the limits from /etc/security/limits.d/). I grepped all files installed for gdm for 1024 (the limit set for apps in my GNOME session started by GDM) but couldn't find a match.
Comment by Dave Reisner (falconindy) - Friday, 19 February 2016, 14:34 GMT
1024 isn't any default from systemd either. If you're still convinced there's still some systemd bug, please report it upstream.

Loading...