FS#60724 - [systemd] update to 239.300-1 crashes session

Attached to Project: Arch Linux
Opened by Norbert Pfeiler (npfeiler) - Tuesday, 06 November 2018, 22:56 GMT
Last edited by freswa (frederik) - Thursday, 13 August 2020, 11:40 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Dave Reisner (falconindy)
Christian Hesse (eworm)
Architecture All
Severity Critical
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 13
Private No

Details

My sessions just crashed during a, or rather, because of a, system upgrade and i believe the reason to be the systemd package.

The journal states:
systemd[1]: Reexecuting.

And everything running got SIGTERM’ed.

TTYs were not available for some time after that
>systemd-logind[828]: Failed to start autovt@tty2.service: Transport endpoint is not connected
, but eventually lightdm successfully restarted itself.

relevant pacman log excerpt:
[2018-11-06 23:28] [PACMAN] Running 'pacman -S -y --config /etc/pacman.conf --'
[2018-11-06 23:28] [PACMAN] synchronizing package lists
[2018-11-06 23:28] [PACMAN] Running 'pacman -S --ignore -u --config /etc/pacman.conf -- libsystemd systemd systemd-sysvcompat brltty gdm harfbuzz harfbuzz-icu libgdm openmpi usbmuxd firefox-developer-edition python-sphinx lib32-harfbuzz lib32-systemd'
[2018-11-06 23:28] [PACMAN] starting full system upgrade
[2018-11-06 23:28] [ALPM] transaction started
[2018-11-06 23:28] [ALPM] upgraded libsystemd (239.2-1 -> 239.300-1)
[2018-11-06 23:28] [ALPM] warning: /etc/systemd/system.conf installed as /etc/systemd/system.conf.pacnew
[2018-11-06 23:28] [ALPM] upgraded systemd (239.2-1 -> 239.300-1)
[2018-11-06 23:28] [ALPM] upgraded systemd-sysvcompat (239.2-1 -> 239.300-1)
[2018-11-06 23:28] [ALPM] upgraded brltty (5.6-5 -> 5.6-6)
[2018-11-06 23:28] [ALPM] upgraded libgdm (3.30.1-1 -> 3.30.2-1)
[2018-11-06 23:28] [ALPM] upgraded harfbuzz (2.1.0-1 -> 2.1.1-1)
[2018-11-06 23:28] [ALPM] upgraded usbmuxd (1.1.0+28+g46bdf3e-1 -> 1.1.0+48+g1cc8b34-1)
[2018-11-06 23:28] [ALPM] upgraded harfbuzz-icu (2.1.0-1 -> 2.1.1-1)
[2018-11-06 23:28] [ALPM] upgraded gdm (3.30.1-1 -> 3.30.2-1)
[2018-11-06 23:28] [ALPM] upgraded openmpi (3.1.2-1 -> 3.1.3-1)
[2018-11-06 23:28] [ALPM] transaction interrupted
This task depends upon

Closed by  freswa (frederik)
Thursday, 13 August 2020, 11:40 GMT
Reason for closing:  No response
Additional comments about closing:  This is stalled for a long time and systemd has been updated since. Feel free to fill a re-open request if this occurs again. Thank you
Comment by Norbert Pfeiler (npfeiler) - Tuesday, 06 November 2018, 23:06 GMT
journal attached

removed names and some calendar entries as this is a work pc and i’m not the only user
Comment by Mathias Anselmann (matse) - Wednesday, 07 November 2018, 13:39 GMT
Had the same problem on my laptop, couldn't switch to another TTY, GDM didn't restart itself.
Could just reboot with REISUB.

On another computer (no laptop), update didn't make any problems.

Some output of the relevalt journalctl part:

gdm: Child process -726 was already dead
gdm: GdmLocalDisplayFactory: Failed to issue method call: connection closed
systemd-logind: Session 2 logged out. Waiting for processes to exit.
systemd-logind: Failed to abandon session scope, ignoring: Transport endpoint is not connected
systemd-logind: Failed to start autovt@tty3.service: Transport endpoint is not connected
Comment by Dan (Mr.D) - Wednesday, 07 November 2018, 17:05 GMT
Same problem here. I lost my internet connection and chromium crashed during the upgrade. Trying to restart network manager manager said "service not found". Thankfully restarting the computer with "shutdown -r now" seems to have made the problem go away.
Comment by Daniél Lecoq (GalaxyCurse) - Wednesday, 07 November 2018, 20:17 GMT
Same problem, but I don't think it was the systemd update, I got a message about libgdm failing for some reason. I managed to get my system back by deleting libgdm in var/lib/pacman and reinstall it, then pacman -S --force gdm
I had to use --force since pacman failed with message about gdm already existing.
Comment by Damian Nowak (Nowaker) - Wednesday, 07 November 2018, 22:24 GMT
People are discussing this on Reddit - a lot of people seem to be affected. https://www.reddit.com/r/archlinux/comments/9uyoql/arch_update_warning
Comment by Julian Phillips (chr0mag) - Thursday, 08 November 2018, 03:48 GMT
Got hit by this as well. I got kicked out of my gnome session during the upgrade. (Firefox core dumped as well.) Trying to log back in just got me kicked out again. Things seem fine after a reboot.

Relevant pacman.log output:
[2018-11-07 19:02] [ALPM] transaction started
[2018-11-07 19:02] [ALPM] upgraded libsystemd (239.2-1 -> 239.300-1)
[2018-11-07 19:02] [ALPM] upgraded systemd (239.2-1 -> 239.300-1)
[2018-11-07 19:02] [ALPM] upgraded libgdm (3.30.1-1 -> 3.30.2-1)
[2018-11-07 19:02] [ALPM] upgraded gdm (3.30.1-1 -> 3.30.2-1)
[2018-11-07 19:02] [ALPM] upgraded gtkmm3 (3.22.2+2+g3002ed86-1 -> 3.22.3-1)
[2018-11-07 19:02] [ALPM] transaction interrupted

Can provide complete journal output and FF core dump if that's helpful.
Comment by Ben Edelman (namlede) - Saturday, 10 November 2018, 21:27 GMT
Same thing happened to me.

pacman.log output:
[2018-11-10 12:11] [ALPM] transaction started
[2018-11-10 12:11] [ALPM] upgraded tzdata (2018e-2 -> 2018g-1)
[2018-11-10 12:11] [ALPM] upgraded glibc (2.28-4 -> 2.28-5)
[2018-11-10 12:11] [ALPM-SCRIPTLET] Generating locales...
[2018-11-10 12:11] [ALPM-SCRIPTLET] en_US.UTF-8... done
[2018-11-10 12:12] [ALPM-SCRIPTLET] en_US.UTF-8... done
[2018-11-10 12:12] [ALPM-SCRIPTLET] Generation complete.
[2018-11-10 12:12] [ALPM] upgraded a52dec (0.7.4-9 -> 0.7.4-10)
[2018-11-10 12:12] [ALPM] upgraded libffi (3.2.1-2 -> 3.2.1-3)
[2018-11-10 12:12] [ALPM] upgraded libutil-linux (2.32.1-2 -> 2.33-2)
[2018-11-10 12:12] [ALPM] upgraded glib2 (2.58.1-1 -> 2.58.1+67+g17519e039-1)
[2018-11-10 12:12] [ALPM] upgraded libcap (2.25-1 -> 2.25-2)
[2018-11-10 12:12] [ALPM] upgraded libgcrypt (1.8.3-1 -> 1.8.4-1)
[2018-11-10 12:12] [ALPM] upgraded libsystemd (239.2-1 -> 239.300-1)
[2018-11-10 12:12] [ALPM] upgraded device-mapper (2.02.181-1 -> 2.02.182-1)
[2018-11-10 12:12] [ALPM] upgraded cryptsetup (2.0.4-1 -> 2.0.5-1)
[2018-11-10 12:12] [ALPM] upgraded libmnl (1.0.4-1 -> 1.0.4-2)
[2018-11-10 12:12] [ALPM] upgraded iptables (1:1.6.2-3 -> 1:1.8.0-1)
[2018-11-10 12:12] [ALPM] upgraded cracklib (2.9.6-1 -> 2.9.6-3)
[2018-11-10 12:12] [ALPM] upgraded gdbm (1.18-1 -> 1.18.1-1)
[2018-11-10 12:12] [ALPM] upgraded kbd (2.0.4-1 -> 2.0.4-2)
[2018-11-10 12:12] [ALPM] upgraded gmp (6.1.2-1 -> 6.1.2-2)
[2018-11-10 12:12] [ALPM] upgraded util-linux (2.32.1-2 -> 2.33-2)
[2018-11-10 12:12] [ALPM] upgraded systemd (239.2-1 -> 239.300-1)

(my session crashed at this time, but there isn't a 'transaction interrupted' log entry)

The following may be unrelated (I hadn't done a system upgrade in a few weeks, so there were lots of packages to upgrade), but upon restarting my system GNOME Shell wasn't working. Specifically, when I log in to a GNOME session, I see a mouse on a black screen; guake is running I can open other graphical applications from there. Also, the lightdm login screen shows a new user 'dynamic user'. I am up and running with XFCE now, but I'm mentioning all this in case it's related to the systemd problem.
Comment by Julian Phillips (chr0mag) - Monday, 12 November 2018, 00:51 GMT
I've since upgraded 3 headless instances to systemd 239.300-1 without issue. I only encountered this issue on my Laptop so it does seem more likely that gdm/libgdm is the source of the problem.

On a related note, I noticed I was missing a systemd user account after the interrupted upgrade so I reinstalled systemd/libsystemd which fixed that particular issue. I reinstalled all packages from the failed upgrade transaction just to be safe.
Comment by Ales Kvapil (aleskva) - Tuesday, 13 November 2018, 21:17 GMT
Here is my log from that day:

[2018-11-07 07:04] [PAMAC] synchronizing package lists
[2018-11-07 07:04] [ALPM] transaction started
[2018-11-07 07:04] [ALPM] upgraded libsystemd (239.2-1 -> 239.300-1)
[2018-11-07 07:04] [ALPM] upgraded systemd (239.2-1 -> 239.300-1)
[2018-11-07 07:05] [ALPM] upgraded brltty (5.6-5 -> 5.6-6)
[2018-11-07 07:05] [ALPM] upgraded harfbuzz (2.1.0-1 -> 2.1.1-1)
[2018-11-07 07:05] [ALPM] upgraded ffmpeg (1:4.0.3-1 -> 1:4.1-1)
[2018-11-07 07:05] [ALPM] upgraded libgdm (3.30.1-1 -> 3.30.2-1)
[2018-11-07 07:05] [ALPM] upgraded usbmuxd (1.1.0+28+g46bdf3e-1 -> 1.1.0+48+g1cc8b34-1)
[2018-11-07 07:05] [ALPM] upgraded harfbuzz-icu (2.1.0-1 -> 2.1.1-1)
*** crash, following is my try to reinstall/downgrade gdm, because gdm was failing to be launched ***
[2018-11-07 07:07] [PACMAN] Running 'pacman -U /var/cache/pacman/pkg/gdm-3.30.2-1-x86_64.pkg.tar.xz'
[2018-11-07 07:09] [PACMAN] Running 'pacman -R gdm'
[2018-11-07 07:09] [ALPM] transaction started
[2018-11-07 07:09] [ALPM] removed gdm (3.30.2-1)
[2018-11-07 07:09] [ALPM] transaction completed
[2018-11-07 07:11] [PACMAN] Running 'pacman -U /var/cache/pacman/pkg/gdm-3.30.1-1-x86_64.pkg.tar.xz'
[2018-11-07 07:11] [ALPM] transaction started
[2018-11-07 07:11] [ALPM] installed gdm (3.30.1-1)
[2018-11-07 07:11] [ALPM] transaction completed
[2018-11-07 07:11] [ALPM] running 'gtk-update-icon-cache.hook'...
[2018-11-07 07:11] [ALPM] running 'systemd-daemon-reload.hook'...
[2018-11-07 07:11] [ALPM] running 'systemd-sysusers.hook'...
[2018-11-07 07:11] [ALPM] running 'systemd-udev-reload.hook'...
[2018-11-07 07:11] [ALPM] running 'systemd-update.hook'...
*** downgrade helped, following is the update after reboot ***
[2018-11-07 07:17] [PAMAC] synchronizing package lists
[2018-11-07 07:17] [ALPM] transaction started
[2018-11-07 07:18] [ALPM] upgraded python-pyparsing (2.2.2-1 -> 2.3.0-1)
[2018-11-07 07:18] [ALPM] upgraded python-urllib3 (1.24-1 -> 1.24.1-1)
[2018-11-07 07:18] [ALPM] upgraded python-sphinx (1.8.0-1 -> 1.8.1-2)
[2018-11-07 07:18] [ALPM] upgraded python2-pyparsing (2.2.2-1 -> 2.3.0-1)
[2018-11-07 07:18] [ALPM] upgraded python2-urllib3 (1.24-1 -> 1.24.1-1)
[2018-11-07 07:18] [ALPM] upgraded seabios (1.11.0-1 -> 1.11.0-3)
[2018-11-07 07:18] [ALPM] upgraded systemd-sysvcompat (239.2-1 -> 239.300-1)
[2018-11-07 07:18] [ALPM] transaction completed
[2018-11-07 07:18] [ALPM] running 'systemd-update.hook'...

What seems suspicious to me is: a) systemd-sysvcompat was still waiting for the upgrade when the crash happened b) the reinstallation of gdm didn't help at all, only the downgrade c) some time after this incident I upgraded gdm with no issue like this

Loading...