FS#57041 - [filesystem] UIDs and GIDs auto-assigned via sysusers.d cause issues with systemd, cups
Attached to Project:
Arch Linux
Opened by Ken Milmore (octree) - Wednesday, 10 January 2018, 19:23 GMT
Last edited by Doug Newgard (Scimmia) - Saturday, 20 January 2018, 16:55 GMT
Opened by Ken Milmore (octree) - Wednesday, 10 January 2018, 19:23 GMT
Last edited by Doug Newgard (Scimmia) - Saturday, 20 January 2018, 16:55 GMT
|
Details
Description:
The systemd package has switched to defining core user and group IDs via the files in /usr/lib/sysusers.d. Many of these UIDs/GIDs are auto-assigned so the exact UID/GID numbers on any individual system are fluid. This has the potential to cause problems with any package which contains files or directories with non-root UIDs or GIDs. I have found some specific instances: The systemd package itself contains the directory /var/log/journal with GID 993: Presumably this is the (arbitrary) GID of systemd-journal from the machine the package was built on. However on my system this group happens to have the old GID of 190. Luckily this mistake gets corrected soon after the directory is unpacked, because there is also an entry in tmpfiles.d which sets the group by name in the post-install hooks. However, it causes spurious warnings to be generated when checking package integrity using pacman -Qkk (which relies on UIDs/GIDs from the .MTREE file). Another problem seems to be the cups package which contains files with GID 988 which are presumably from the lp group on the build system. These GIDs never get corrected on install - if you have the old lp GID of 7 or some other value then they will be installed incorrectly. I am reporting that problem here because the lp group is defined by /lib/sysusers.d/basic.conf which is part of the systemd package. The above issues can be easily corrected. But I would suggest that in the short term, to avoid a lot of breakage with other packages it might be better to edit the files in /lib/sysusers.d so they assign well-defined ID numbers to the core users and groups (following the old table here https://wiki.archlinux.org/index.php/DeveloperWiki:UID_/_GID_Database). Using fixed IDs is probably desirable for core packages anyway. In the longer term, it is worth observing that pacman's internal reliance on numeric UIDs and GIDs (as opposed to user and group names) does not sit well with the new dynamic assignment scheme. Basically, any UID/GID other than 0 applied to files in a package is likely to become problematic in future. Additional info: * package version(s) systemd 236.0-2 cups 2.2.6-1 * config and/or log files etc. n/a Steps to reproduce: **** systemd-journal issue: **** Note arbitrary GID 993 # getent group systemd-journal systemd-journal:x:190: # zcat /var/lib/pacman/local/systemd-*/mtree | grep -e 'uid=[1-9][0-9]*' -e 'gid=[1-9][0-9]*' ./usr/share/polkit-1/rules.d time=1513605745.748008429 mode=750 gid=102 type=dir ./var/log/journal time=1513605747.298001867 mode=2755 gid=993 type=dir # pacman -Qkkv systemd warning: systemd: /var/log/journal (GID mismatch) **** cups/lp issue: **** Note arbitrary GID 988 # getent group lp lp:x:7:daemon # mv /etc/cups /etc/cups.XXX # pacman -Syu cups # ls -l /etc/cups -rw-r--r-- 1 root 988 0 Nov 1 15:37 classes.conf -rw-r----- 1 root 988 6346 Nov 1 15:37 cupsd.conf -rw-r----- 1 root 988 6346 Nov 1 15:37 cupsd.conf.default -rw-r----- 1 root 988 2947 Nov 1 15:37 cups-files.conf -rw-r----- 1 root 988 2947 Nov 1 15:37 cups-files.conf.default drwxr-xr-x 2 root 988 4096 Nov 1 15:37 ppd -rw-r--r-- 1 root 988 0 Nov 1 15:37 printers.conf -rw-r----- 1 root 988 142 Nov 1 15:37 snmp.conf -rw-r----- 1 root 988 142 Nov 1 15:37 snmp.conf.default drwx------ 2 root 988 4096 Nov 1 15:37 ssl -rw-r--r-- 1 root 988 0 Nov 1 15:37 subscriptions.conf |
This task depends upon
Closed by Doug Newgard (Scimmia)
Saturday, 20 January 2018, 16:55 GMT
Reason for closing: Duplicate
Additional comments about closing: FS#56818
Saturday, 20 January 2018, 16:55 GMT
Reason for closing: Duplicate
Additional comments about closing:
FS#56818.However, presumably to avoid clashing with the systemd package they have not redefined any groups which are already set up by the systemd package (see in particular /usr/lib/sysusers.d/{basic.conf,systemd.conf}).
Unfortunately most of the standard groups defined by the systemd package (which include lp, wheel, kmem, utmp etc) do *not* have numeric GIDs specified in the conf files. That seems dangerously inconsistent.
FS#56662?FS#56662, it is essentially the same issue, although I'm surprised to see it is has been flagged low priority and that the potential security implications don't seem to have been considered in any detail. This was a very dangerous change to push through without prior thought IMHO.# zgrep -w -e 'uid=[1-9][0-9]*' -e 'gid=[1-9][0-9]*' /var/lib/pacman/local/*/mtree
Looking at the output on my system, I see that libutempter installs /usr/lib/utempter/utempter with gid 20=utmp, but utmp is defined dynamically in sysusers.d, so that's another potential problem.
I should also elaborate a bit more on the possible security implications with dynamically assigned UIDs/GIDs. This applies in particular to the systemd issue with /var/log/journal, but also with any other file stored in a package which is assigned a dynamically assigned UID/GID. Consider the following:
1. Packager builds package on a machine where someuser has auto-assigned uid X, and chowns some files in the package. The uid X is then encoded in the package tarball by makepkg.
2. User installs package on a machine where someuser happens to have a different auto-assigned uid Y. uid X may be already in use on this machine for a webserver service or whatever.
3. Pacman unpacks the package tarball into the filesystem, giving the relevant files the original uid X.
4. Now the webserver with uid X can, for a time, access files it isn't supposed to.
5. Eventually the install scripts and pacman hooks are run, and settings like tmpfiles.d are (hopefully) applied to patch up the files to the local uid Y.
Now, the preferred way to set permissions post-installation seems to be through /usr/lib/tmpfiles.d (systemd does this with /var/log/journal, which is why it ends up with the correct local gid). However, the tmpfiles.d processing is done once from a hook at the end of installing a batch of packages. If the batch is a large one, the time between pacman creating the file with the wrong gid and the gid being set properly can be significant, a minute or more. This is quite a potential security hole.
One might infer from all this that after transitioning to dynamically-assigned UIDs/GIDs, it is dangerous for any package tarball to have files with any ownership other than root:root - Even if post-install scripts fix the permissions up securely, there will always be a possibility of creating a window where another unknown user gets temporary access to the files during installation.
That being said, files really should not be being modified by tmpfiles.d, part of the goal of using tmpfiles.d is that we want to avoid ever installing files/directories with the wrong ownership (mostly directories).
If files with specific content need to be installed with specific ownership, we are still AFAIK using static uids. Certainly regarding the TODO list which I am going through to migrate to sysusers/tmpfiles, I am keeping static uids unless I can leave all packaged files/directories permanently owned by root...
(It is another thing entirely to have no packaged non-root-owned directory, and then create that directory using a tmpfiles snippet.)
I'm not *advocating* setting all file ownerships to root:root and fixing this in tmpfiles.d (or post-install scripts), I'm saying that with dynamic uids/gids you have little other choice (save for adding new pacman features or some such).
Unless I'm missing something, once the static uid/gid lists are fixed up (and people who have installed new systems with the dynamic ones fix their passwd and group files), the packages can all be rebuilt and the systemd, cups and utempter issues should all go away.
Dynamic uids/gids are great for things like wireshark where the dumpcap binary is set sgid, so anyone in the wireshark group can dump network traffic and additionally for transient things like bumbleebee -- I don't need to know or care what gid is assigned to the bumblebee service since it's used purely for the sake of local access that won't carry over to any other system.
This impacts so many things things, the biggest one that comes to mind is permissions preservation for file syncing/archiving. Removing the users and wheel groups is a serious oversight IMHO, but I plea to you to _at least_ grandfather in the entries that have been the same statically defined values for over a decade now!
https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/filesystem&id=45cda7ae27f309237eefa7b294e0224849fd55d4
diff --git a/trunk/passwd b/trunk/passwd
new file mode 100644
index 00000000000..f886802bfc5
--- /dev/null
+++ b/trunk/passwd
@@ -0,0 +1,6 @@
+root:x:0:0:root:/root:/bin/bash
+bin:x:1:1:bin:/bin:/bin/false
+daemon:x:2:2:daemon:/sbin:/bin/false
+mail:x:8:12:mail:/var/spool/mail:/bin/false
+ftp:x:14:11:ftp:/home/ftp:/bin/false
+nobody:x:99:99:nobody:/:/bin/false
diff --git a/trunk/group b/trunk/group
new file mode 100644
index 00000000000..c5856efd0e0
--- /dev/null
+++ b/trunk/group
@@ -0,0 +1,26 @@
+root::0:root
+bin::1:root,bin,daemon
+daemon::2:root,bin,daemon
+sys::3:root,bin
+adm::4:root,daemon
+tty::5:
+disk::6:root
+lp::7:daemon
+mem::8:
+kmem::9:
+wheel::10:root
+ftp::11:
+mail::12:
+uucp::14:
+log::19:root
+smmsp::25:
+games::50:
+network:x:90:
+video:x:91:
+audio::92:
+optical::93:
+floppy:x:94:
+storage:x:95:
+power:x:98:
+nobody::99:
+users::100:
That way, you would get to keep the static uids/gids without having to preseed then into /etc/passwd and /etc/group (which is what the filesystem package maintainers seem to want to get away from).
AFAICT some sort of bun fight seems to be going on between the systemd and filesystem guys which is preventing them from reaching an agreement.
For the time being, the systemd package effectively controls all those "traditional" uids/gids and is forcing them to be dynamically assigned to different values on every system.
I currently have Arch installed on my personal workstation, my personal Macbook Air, an Intel NUC I use as a general sandbox environment, a HP tower at work, my work Macbook Pro, my work Dell XPS, and most recently an Acer Predator I just purchased the other night I was attempting to install Arch on before getting side-tracked with this issue. And please don't misread my comments here as a personal mission to ensure some sort of self-entitled behavior. I greatly appreciate all the work that goes into this distro, actively been an Arch user since 2004 (originally started on Slackware back in '96) as well as maintaining a handful of AUR packages.
There's definitely a few ways to go about progressing forward while still maintaining compatibility, I'm just a little upset with such a critical component being altered without any heads up, no changes to reflect the uid/gid table mapping in the developer wiki I referenced in my initial post, no mention of it in the install.txt file in the root of the usb image, and at the end of the day what's it all for anyway. To save a few kilobytes of space or something?
Throughout the years I've never had a major issue or failure with any of my systems, so HUGE kudos to all the devs that make this distro a reality. In fact this is the first time I've chimed in on an issue like this, Arch has always been amazingly on-point with picking the right technologies (ie: moving to systemd), the documentation in the wiki, the community of people and last but certainly not least the AUR is simply one of the strongest communities I've ever seen or been a part of. And the most amazing part about all of that is -- it all just works! Most distros can't pin down that feat even when they custom bake everything in a nice packaged bundle, essentially forcing your hand to do things their way (upstart, ufw, etc) in an environment of their choosing (Unity).
I'm sure a consensus can be reached in regards to the best (ie: simplistic) way to at least give users an option to maintain some sort of legacy when needed. I haven't versed myself in the new sysusers thing systemd introduced, but would a preset be applicable here? Like maybe in addition to the default systemd preset (/usr/lib/systemd/system-preset/99-default.preset), create a new preset called /usr/lib/systemd/system-preset/90-legacy.preset or something and mention it in the install.txt file? I'm sure this isn't the best example, just spitballing my $0.02USD.
Thanks everyone.
https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/filesystem&id=20928f58767d34ed6711befd6255f6a0b1706ae8
I can see that this was done with the intention of migrating to /lib/sysusers.d eventually, but it was premature.
Also, I don't buy the argument that systemd upstream has broken everything: Arch likes to keep its packaging vanilla, but sometimes a bit of integration is necessary. They don't have to ship every config file from upstream without any modifications whatsoever.
For example systemd hardcodes nobody uid to 65534 while Arch sets it to 99 which broke things, see https://bugs.archlinux.org/task/56828 . Recently many packages like cups,kvm,qemu were broken due to systemd upstream changes to default uid/gid. Status quo isn't the answer here because it's falling apart with every systemd release.
Please don't misconstrue my comments as a rant against the devs; I realise they have a lot to deal with. But there is the appearance of a disconnect here when something which causes so many problems has been left uncorrected for several weeks after it was noticed.
As @falconindy pointed out, the gids which are relevant to this bug (lp, utmp, systemd-journal etc) have all been defined in sysusers.d by the systemd package since way back. Looking at the archive I think they were introduced around version 215 - that's back in 2014.
So most of these gids have *not* gone wrong because of a very recent change to systemd. However, since they were recently removed from /etc/groups in the filesystem package, we now fall back on the settings in /lib/sysusers.d where the groups are set up for automatic allocation - and have been for some time.
It seems to me there are only two possible ways around this:
EITHER (1) reinstate the passwd and group files in the filesystem package for the time being, until a more permanent fix is found.
OR (2) patch the conf files installed into /lib/sysusers.d by the systemd package so that they specify static uids and gids - most of the offending groups can be found in basic.conf.
A variant of (2) would be to omit the /lib/sysusers.d files (or at least those which don't define systemd-specific groups) from the systemd package and put them instead in filesystem, or some other package where they can be centrally managed in future.
FS#56662only other thing was in https://bugs.archlinux.org/task/57041#comment165372 octree notesI see that libutempter installs /usr/lib/utempter/utempter with gid 20=utmp, but utmp is defined dynamically in sysusers.d, so that's another potential problem.
So I asked octree to open a separate bug report to be opened for that.
FS#56662. The discussion here is clearly being viewed as unproductive.@loqs, I have not raised an issue with libutempter: I only mentioned it as another example of the breakage caused by the changes to filesystem - there are probably several other packages in a similar position. Please raise an issue yourself if you think it useful.
Best regards.
FS#56662seems to be outside the scope of this ticket as well. The complaint here is the UID/GID ownership of files in systemd and cups. It's taken a MAJOR left turn in the comments.