FS#55984 - [systemd] 235.0-1 prevents to startx. Failed to change ownership of session keyring: Disk quota exc

Attached to Project: Arch Linux
Opened by dreem (dreem) - Saturday, 14 October 2017, 08:21 GMT
Last edited by Sven-Hendrik Haase (Svenstaro) - Thursday, 03 March 2022, 13:13 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Dave Reisner (falconindy)
Christian Hesse (eworm)
Architecture x86_64
Severity High
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description:
Hi,
it reminds me of the problem i reported some time ago:
https://bugs.archlinux.org/task/53268



So after upgrading systemd to 235.0-1 and rebooting the machine, starting sddm, and authorizing user - desktop environment (particulary plasma) is unable to start showing "Could not sync environment to dbus." error window.


journalctl said:

Oct 14 09:31:38 xaos sddm[576]: Greeter starting...
Oct 14 09:31:38 xaos sddm[576]: Adding cookie to "/var/run/sddm/{d56b4e0b-37d7-48be-ac77-e5f3a1c2c879}"
Oct 14 09:31:38 xaos sddm-helper[662]: [PAM] Starting...
Oct 14 09:31:38 xaos sddm-helper[662]: [PAM] Authenticating...
Oct 14 09:31:38 xaos sddm-helper[662]: [PAM] returning.
Oct 14 09:31:38 xaos sddm-helper[662]: pam_unix(sddm-greeter:session): session opened for user sddm by (uid=0)
Oct 14 09:31:38 xaos systemd[1]: Created slice User Slice of sddm.
Oct 14 09:31:38 xaos systemd[1]: Starting User Manager for UID 994...
Oct 14 09:31:38 xaos systemd-logind[519]: New session c1 of user sddm.
Oct 14 09:31:38 xaos systemd[664]: user@994.service: Failed to change ownership of session keyring: Disk quota exceeded
Oct 14 09:31:38 xaos systemd[664]: user@994.service: Failed to set up kernel keyring: Disk quota exceeded
Oct 14 09:31:38 xaos systemd[664]: user@994.service: Failed at step KEYRING spawning /usr/lib/systemd/systemd: Disk quota exceeded
Oct 14 09:31:38 xaos systemd[1]: Started Session c1 of user sddm.
Oct 14 09:31:38 xaos systemd[1]: user@994.service: Failed with result 'protocol'.
Oct 14 09:31:38 xaos systemd[1]: Failed to start User Manager for UID 994.
Oct 14 09:31:38 xaos sddm-helper[662]: pam_systemd(sddm-greeter:session): Failed to create session: Start job for unit user@994.service failed with 'failed'
Oct 14 09:31:38 xaos sddm[576]: Greeter session started successfully


Downgrading systemd packages to 234.11-9 solved the problem.

But even then on older systemd I can see something wrong I think:
[root@xaos ~]# keyctl session
Joined session keyring: 565121823
[root@xaos ~]# keyctl chown 565121823 1002
keyctl_chown: Disk quota exceeded

so it may be more general one...

BTW I checked this commands after reading this:
https://github.com/systemd/systemd/issues/6281

I would appriciate your help


Additional info:
* package version(s)
systemd 235.0-1
* config and/or log files etc.
journalctl -b: https://hastebin.com/mujarehiwe.coffeescript

Steps to reproduce:
1. update systemd to 235.0-1
2. reboot
This task depends upon

Closed by  Sven-Hendrik Haase (Svenstaro)
Thursday, 03 March 2022, 13:13 GMT
Reason for closing:  No response
Additional comments about closing:  2022-02-27: A task closure has been requested. Reason for request: No response in years. Assuming fixed upstream.
Comment by dreem (dreem) - Tuesday, 17 October 2017, 20:52 GMT
for a seccond i was hopping that 235.8 would fix it, but no, still the same behaviour.

I decided to go back to working systemd 234 and check keyctl problem closer:

[root@xaos ~]# keyctl chown 252000957 1002
keyctl_chown: Disk quota exceeded
[root@xaos ~]# strace keyctl chown 252000957 1002
execve("/usr/bin/keyctl", ["keyctl", "chown", "252000957", "1002"], 0x7ffe79f2a0d8 /* 17 vars */) = 0
brk(NULL) = 0xae7000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=379402, ...}) = 0
mmap(NULL, 379402, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f95fd031000
close(3) = 0
openat(AT_FDCWD, "/usr/lib/libkeyutils.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\26\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=14616, ...}) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f95fd02f000
mmap(NULL, 2109712, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f95fcc66000
mprotect(0x7f95fcc69000, 2093056, PROT_NONE) = 0
mmap(0x7f95fce68000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f95fce68000
close(3) = 0
openat(AT_FDCWD, "/usr/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\20\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=2065840, ...}) = 0
mmap(NULL, 3893456, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f95fc8af000
mprotect(0x7f95fca5d000, 2093056, PROT_NONE) = 0
mmap(0x7f95fcc5c000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1ad000) = 0x7f95fcc5c000
mmap(0x7f95fcc62000, 14544, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f95fcc62000
close(3) = 0
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f95fd02c000
arch_prctl(ARCH_SET_FS, 0x7f95fd02c740) = 0
mprotect(0x7f95fcc5c000, 16384, PROT_READ) = 0
mprotect(0x7f95fce68000, 4096, PROT_READ) = 0
mprotect(0x605000, 4096, PROT_READ) = 0
mprotect(0x7f95fd08e000, 4096, PROT_READ) = 0
munmap(0x7f95fd031000, 379402) = 0
keyctl(KEYCTL_CHOWN, 252000957, 1002, -1) = -1 EDQUOT (Disk quota exceeded)
dup(2) = 3
fcntl(3, F_GETFL) = 0x8002 (flags O_RDWR|O_LARGEFILE)
brk(NULL) = 0xae7000
brk(0xb08000) = 0xb08000
fstat(3, {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 3), ...}) = 0
write(3, "keyctl_chown: Disk quota exceede"..., 34keyctl_chown: Disk quota exceeded
) = 34
close(3) = 0
exit_group(1) = ?
+++ exited with 1 +++


I don;t see anything particuallary interesting here, could it be btrfs problem? I'm using btrfs with snapshots, i removed snapshot but it also didn't help with 235 nor keyctl
Also:
btrfs qgroup show -erF /
ERROR: can't list qgroups: quotas not enabled

Comment by Dave Reisner (falconindy) - Tuesday, 17 October 2017, 20:54 GMT
The quota referred to is coming from the keyctl syscall as your strace shows:

keyctl(KEYCTL_CHOWN, 252000957, 1002, -1) = -1 EDQUOT (Disk quota exceeded)

All keyrings have a max size in order to avoid an unprivileged user DoS'ing the kernel.
Comment by dreem (dreem) - Tuesday, 17 October 2017, 21:06 GMT
I red somwhere it was a (more common) problem before big_key kernel option, but arch is using it as default.

Dave what do you suggest? should I look for a way of removing/destroing some keys?

Not sure if its helpful (or is the session key of my user with id 1002 should be empty):
[root@xaos ~]# cat /proc/keys
01c6995b I--Q--- 1 perm 1f3f0000 0 65534 keyring _uid_ses.0: 1
05035782 I------ 1 perm 1f0f0000 0 0 keyring .secondary_trusted_keys: 1
090d73d9 I--Q--- 191 perm 3f030000 1002 1002 keyring _ses: empty
0e0ff256 I------ 2 perm 1f0b0000 0 0 keyring .builtin_trusted_keys: empty
192c9bfc I------ 1 perm 1f0b0000 0 0 keyring .blacklist: empty
2f953c2e I--Q--- 2 perm 1f3f0000 0 65534 keyring _uid.0: empty
[root@xaos ~]# cat /proc/key
keys key-users
[root@xaos ~]# cat /proc/key-users
0: 8 7/7 2/1000000 22/25000000
1002: 2 2/2 2/1 10/20000
Comment by Sean Lingham (Cxpher) - Friday, 06 July 2018, 15:38 GMT
Can confirm this was reproduced on my system. Downgrading systemd resolved it.
Comment by Dave Reisner (falconindy) - Friday, 06 July 2018, 16:51 GMT
You're replying 8 months later and all you can say is +1? What version was affected? What version did you downgrade to? There have been 4 major version releases and 2 dozen new Arch packages since this report was filed.
Comment by Sean Lingham (Cxpher) - Saturday, 07 July 2018, 05:13 GMT
Apologies for now giving more information earlier.

239.0-2 is the version that's affected.
238.133-4 is the version i downgraded to.
All other packages are upgraded to latest available.

Basically, if i upgrade to 239.0-2, X fails to boot but i can still use a tty to login. Which is how i used Pacman to downgrade.

This is the only package i downgraded to get past the issue.
Comment by loqs (loqs) - Saturday, 07 July 2018, 11:06 GMT
@Cxpher are you sure you are not experiencing https://bbs.archlinux.org/viewtopic.php?id=238602 ?
Comment by Sean Lingham (Cxpher) - Saturday, 07 July 2018, 12:27 GMT
Not too sure.

I didn't downgrade libsystemd though. That's still @ 239.0-2
Comment by Dave Reisner (falconindy) - Saturday, 07 July 2018, 13:37 GMT
> Not too sure.
This bug report has a specific error in journalctl: Failed at step KEYRING spawning /usr/lib/systemd/systemd: Disk quota exceeded

Do you see that error? I suspect not.
Comment by Dave Reisner (falconindy) - Sunday, 08 December 2019, 11:48 GMT
Is this still relevant with systemd v244?

Loading...