FS#47303 - [lxc] [systemd] service file does not specify TasksMax which leads to quick exhaustion of pids.
Attached to Project:
Community Packages
Opened by Jakub Sztandera (Kubuxu) - Tuesday, 08 December 2015, 22:00 GMT
Last edited by Sergej Pupykin (sergej) - Wednesday, 09 March 2016, 19:31 GMT
Opened by Jakub Sztandera (Kubuxu) - Tuesday, 08 December 2015, 22:00 GMT
Last edited by Sergej Pupykin (sergej) - Wednesday, 09 March 2016, 19:31 GMT
|
Details
Description:
If LXC is run from systemd service then it runs in its cgroup. Default limit of 512 pids if too low for most use-cases. Fix is to assing: [Service] TasksMax=infinity in service files. Additional info: * systemd 228-3 * lxc 1:1.1.5-1 Steps to reproduce: 1. Start container. 2. Start many threads: for i in {1..1000}; do sleep 1000& done; 3. Look and quirks in the console: bash: fork: retry: No child processes bash: fork: retry: No child processes bash: fork: retry: No child processes bash: fork: retry: No child processes bash: fork: Resource temporarily unavailable bash: let: detached=: syntax error: operand expected (error token is "=") bash: fork: retry: No child processes bash: fork: retry: No child processes bash: fork: retry: No child processes bash: fork: retry: No child processes bash: fork: Resource temporarily unavailable bash: let: detached+=: syntax error: operand expected (error token is "+=") bash: fork: retry: No child processes bash: fork: retry: No child processes |
This task depends upon
https://github.com/lxc/lxc/blob/master/src/lxc/cgroup.c#L198
FS#47662and http://unix.stackexchange.com/q/253903/59955.I reported the problem on the forum https://bbs.archlinux.org/viewtopic.php?id=207255 and then at
FS#47787but was asked to report it here instead.I'm wondering if the default value for logind.conf's UserTaskMax of 4096 is not too small.
I propose to up it a bit, I don't know what is the perfect value, but I think that even for a normal user that is also a developer, for example, you can run into this limit quite rapidly. I myself experienced it during development when running tests on Java code that were using many threads.
Also, I don't see the reason for limiting too much this value except in multi-user settings, and it would make sense that the default behaviour is 1) for a one-user installation, which I think is one of the main use of archlinux, and 2) closer to the system's behaviour before this problem appeared (before linux 4.3 apparently).
In that case, something like 32000 could be a good value since it is close to the default max number of user processes according to ulimit (-u).
I guess the systemd files of the Arch Linux package need to be adjusted.
I don’t know what the thing about the “init.scope” is supposed to mean. At least for docker containers, it looks like the systemd task limit is applied correctly.
This should be tagged under linux kernel?
Kubuxu: I don't know how lxc and docker relate to each other, as far as I know docker uses lxc. At least docker doesn't seem to have the problem you describe.
The thing is: by default the limit is 512 because there is a bug in systemd that was fixed but not released (see https://github.com/systemd/systemd/pull/1987).
As for the default when this bug is fixed (which is 4096), I agree it should either be configured by myself or fixed upstream.
I tried to create another issue (
FS#47787) but it was refused to me even though I think it does not make sense to report these problems here.FS#47787in order not to pollute this issue with a different problem.Upstream is working on it.
1. The systemd service file provided with the Arch lxc package should have TasksMax increased. This is what this issue is about.
2. The TasksMax attribute does not work properly for lxc, because lxc containers are inserted into the init.scope cgroup in addition to their designated cgroup. This is being dealt with in lxc upstream: https://github.com/lxc/lxc/issues/713
3. The UserTasksMax default limit of 4096 is too low for some use cases. This is being dealt with in systemd upstream and is being increased to 12288: https://github.com/systemd/systemd/issues/2388. It is additionally being discussed in
FS#47787.4. The systemd service file provided with the Arch docker package should have TasksMax increased. In docker upstream the limit has been increased in the example service files: https://github.com/docker/docker/pull/19391. I have opened a separate bug report
FS#47872to increase the value in the Arch package.Other issues were fixed or upstream as I understand