FS#58892 - [systemd] nspawn --keep-unit failed to attach to systemd cgroup

Attached to Project: Arch Linux
Opened by bartus (bartoszek) - Wednesday, 06 June 2018, 16:43 GMT
Last edited by freswa (frederik) - Thursday, 05 March 2020, 21:53 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Dave Reisner (falconindy)
Christian Hesse (eworm)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 9
Private No

Details

Description:
When building packages within chroot using arch-nspawn occasionally build renders systemd nspawn container unusable.
Every consecutive invocation of systemd-nspawn crashes with this type of error:

Failed to attach 8185 to compat systemd cgroup /user.slice/user-1000.slice/session-c1.scope/payload: No such file or directory
Failed to attach 8157 to compat systemd cgroup /user.slice/user-1000.slice/session-c1.scope/supervisor: No such file or directory
Failed to chown() cgroup /sys/fs/cgroup/systemd/user.slice/user-1000.slice/session-c1.scope/payload: No such file or directory

Steps to reproduce:
$ aursync -c -C /tmp/ -T ananicy-git
// first time works fine
$ aursync -c -C /tmp/ -T ananicy-git
// this time it crashes

Add1. The problem was mentioned on mailing list: https://lists.archlinux.org/pipermail/arch-dev-public/2018-January/029105.html
Add2. `payload` and `supervisor` directorys are located in /sys/fs/cgroups/undefinded instead of /sys/fs/cgroups/systemd ...
$ tree /sys/fs/cgroup
.
├── blkio
...
├── cpu -> cpu,cpuacct
├── cpuacct -> cpu,cpuacct
├── cpu,cpuacct
...
├── cpuset
...
├── devices
...
├── freezer
...
├── hugetlb
...
├── memory
...
├── net_cls -> net_cls,net_prio
├── net_cls,net_prio
...
├── net_prio -> net_cls,net_prio
├── perf_event
...
├── pids
...
├── rdma
...
├── systemd
│   ├── cgroup.clone_children
│   ├── cgroup.procs
│   ├── cgroup.sane_behavior
│   ├── init.scope
...
│   ├── notify_on_release
│   ├── release_agent
│   ├── system.slice
...
│   ├── tasks
│   └── user.slice
│   ├── cgroup.clone_children
│   ├── cgroup.procs
│   ├── notify_on_release
│   ├── tasks
│   └── user-1000.slice
│   ├── cgroup.clone_children
│   ├── cgroup.procs
│   ├── notify_on_release
│   ├── session-c1.scope
│   │   ├── cgroup.clone_children
│   │   ├── cgroup.procs
│   │   ├── notify_on_release
│   │   └── tasks
│   ├── tasks
│   └── user@1000.service
...
└── unified
├── cgroup.controllers
├── cgroup.max.depth
├── cgroup.max.descendants
├── cgroup.procs
├── cgroup.stat
├── cgroup.subtree_control
├── cgroup.threads
├── init.scope
│   ├── cgroup.controllers
│   ├── cgroup.events
│   ├── cgroup.max.depth
│   ├── cgroup.max.descendants
│   ├── cgroup.procs
│   ├── cgroup.stat
│   ├── cgroup.subtree_control
│   ├── cgroup.threads
│   ├── cgroup.type
│   └── cpu.stat
├── system.slice
...
└── user.slice
├── cgroup.controllers
├── cgroup.events
├── cgroup.max.depth
├── cgroup.max.descendants
├── cgroup.procs
├── cgroup.stat
├── cgroup.subtree_control
├── cgroup.threads
├── cgroup.type
├── cpu.stat
└── user-1000.slice
├── cgroup.controllers
├── cgroup.events
├── cgroup.max.depth
├── cgroup.max.descendants
├── cgroup.procs
├── cgroup.stat
├── cgroup.subtree_control
├── cgroup.threads
├── cgroup.type
├── cpu.stat
├── session-c1.scope
│   ├── cgroup.controllers
│   ├── cgroup.events
│   ├── cgroup.max.depth
│   ├── cgroup.max.descendants
│   ├── cgroup.procs
│   ├── cgroup.stat
│   ├── cgroup.subtree_control
│   ├── cgroup.threads
│   ├── cgroup.type
│   ├── cpu.stat
│   ├── payload
│   │   ├── cgroup.controllers
│   │   ├── cgroup.events
│   │   ├── cgroup.max.depth
│   │   ├── cgroup.max.descendants
│   │   ├── cgroup.procs
│   │   ├── cgroup.stat
│   │   ├── cgroup.subtree_control
│   │   ├── cgroup.threads
│   │   ├── cgroup.type
│   │   └── cpu.stat
│   └── supervisor
│   ├── cgroup.controllers
│   ├── cgroup.events
│   ├── cgroup.max.depth
│   ├── cgroup.max.descendants
│   ├── cgroup.procs
│   ├── cgroup.stat
│   ├── cgroup.subtree_control
│   ├── cgroup.threads
│   ├── cgroup.type
│   └── cpu.stat
└── user@1000.service
...
588 directories, 8918 files
This task depends upon

Closed by  freswa (frederik)
Thursday, 05 March 2020, 21:53 GMT
Reason for closing:  Fixed
Additional comments about closing:  Reporter: I haven't encounter this problem since ~1yer - think it was resolved with `systemd` update.
Comment by bartus (bartoszek) - Wednesday, 06 June 2018, 16:52 GMT
Add3. When nspawn is working those directroys does not exist nither under /sys/fs/cgroup/unified/user.slice/user-1000.slice/session-c2.scope nor /sys/fs/cgroup/systemd/user.slice/user-1000.slice/session-c2.scope.
Comment by Alexander F. Rødseth (xyproto) - Thursday, 08 November 2018, 15:46 GMT
I also get similar errors when attempting to build packages with "extra-x86_64-build":

Failed to attach 5172 to compat systemd cgroup /user.slice/user-1000.slice/session-1.scope/payload: No such file or directory
Failed to attach 5151 to compat systemd cgroup /user.slice/user-1000.slice/session-1.scope/supervisor: No such file or directory
Failed to chown() cgroup /sys/fs/cgroup/systemd/user.slice/user-1000.slice/session-1.scope/payload: No such file or directory
==> ERROR: Aborting...

This worked a couple of days ago, but has stopped working on two separate systems. I have tried re-logging in, and also rebooted both.
Comment by Alexander F. Rødseth (xyproto) - Thursday, 08 November 2018, 17:13 GMT
Removing "--keep-unit" from /usr/bin/arch-nspawn in combination with chmod a+rwX /var/lib/archbuild/extra-x86_64 made "sudo extra-x86_64-build" work again. (I am not recommending a+rwX on a general basis).

Thanks to sangy for the tip about removing "--keep-unit".
Comment by Eli Schwartz (eschwartz) - Thursday, 10 January 2019, 00:48 GMT
So from a devtools perspective this basically seems to happen because of  FS#55082 , right? Is --keep-unit even supposed to be used like this?
Comment by Jamin Collins (jamincollins) - Thursday, 07 March 2019, 15:51 GMT
Any update on this? The bug/issue is still present in current Arch devtools 20180531-4.

Loading...