FS#70316 - Upgrade to ostree 2021.1-1 breaks rootless docker

Attached to Project: Arch Linux
Opened by D B (abcalphabet) - Tuesday, 06 April 2021, 15:52 GMT
Last edited by Andreas Radke (AndyRTR) - Wednesday, 07 April 2021, 05:19 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To No-one
Architecture x86_64
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description:

After upgrading to ostree 2021.1-1 and trying to start a rootless-docker container with vfs as the storage-system, it fails with the following error message:

Cannot start service consul: OCI runtime create failed:
container_linux.go:367: starting container process caused:
process_linux.go:495: container init caused:
rootfs_linux.go:60: mounting "cgroup" to rootfs at "/sys/fs/cgroup" caused: invalid argument: unknown


Once ostree is downgraded to 2020.8-1 this error disappears and everything works as normal.


Additional info:

OS: Arch Linux x86_64
Kernel: 5.11.11-arch1-1
docker: 1:20.10.5-1
docker-rootless-extras-bin: 20.10.3-1

Issue in ostree git: https://github.com/ostreedev/ostree/issues/2322

Steps to reproduce:
1. install docker and rootless docker
2. set storage-driver for docker daemon to vfs
3. attempt to run a docker container (example was consul)
This task depends upon

Closed by  Andreas Radke (AndyRTR)
Wednesday, 07 April 2021, 05:19 GMT
Reason for closing:  None
Additional comments about closing:  Resolved in comments
Comment by Doug Newgard (Scimmia) - Tuesday, 06 April 2021, 16:15 GMT
systemd 248 switched to cgroups v2. That's probably what you're seeing, in combination with the ostree update.
Comment by D B (abcalphabet) - Tuesday, 06 April 2021, 17:09 GMT
That does indeed seem to have been the issue.

I was able to resolve the issue by adding "systemd.unified_cgroup_hierarchy=1" to my linux cmdline and rebuilding my docker images.
Comment by Doug Newgard (Scimmia) - Tuesday, 06 April 2021, 17:12 GMT
That doesn't explain why 2020.8 works with cgroups v2 and 2021.1 doesn't.
Comment by D B (abcalphabet) - Tuesday, 06 April 2021, 17:24 GMT
Yes, pardon me, I forgot to add that I also set "native.cgroupdriver=systemd" in the docker "exec-opts". There seems to be some interaction with the way docker handles it's cgroups/namespaces when specifying this option.

Under the above configs I was able to run containers 2021.1 without problems as well.

Loading...