Arch Linux

Please read this before reporting a bug:

Do NOT report bugs when a package is just outdated, or it is in the AUR. Use the 'flag out of date' link on the package page, or the Mailing List.

REPEAT: Do NOT report bugs for outdated packages!

FS#75609 - System froze after updating to linux-5.19.1

Attached to Project: Arch Linux
Opened by Adam Beavan (ajbeavan) - Monday, 15 August 2022, 08:34 GMT
Last edited by Toolybird (Toolybird) - Tuesday, 16 August 2022, 03:34 GMT
Task Type Bug Report
Category Kernel
Status Waiting on Response
Assigned To No-one
Architecture All
Severity Critical
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 0%
Votes 7
Private No


Description:Updated to linux-5.19.1 unable to boot the system. Just froze at the point where GRUB tries to load a kernel image, had to rollback to previous version.

Additional info:
* package version(s)
* config and/or log files etc.
* link to upstream bug report, if any

Steps to reproduce:
This task depends upon

Comment by Toolybird (Toolybird) - Tuesday, 16 August 2022, 03:34 GMT
"It doesn't work" style bug reports are completely useless. If you're not prepared to put some effort in and provide a decent bug report, you're better off going through support channels first (forum/IRC/etc). Please re-read the bug guidelines [1]

Usual advice for kernel regressions is:
- provide logs if possible
- perform a bisection
- at least provide your hardware details?
- read this [2]

Ball is in your court.

Comment by Adam Beavan (ajbeavan) - Tuesday, 16 August 2022, 06:35 GMT
I will repeat 'No kernel image was loaded' which means no journal file would have been written too, I have provided all the information I have. If you know something different then please enlighten me.
Comment by michael buckley (mokchira) - Tuesday, 16 August 2022, 17:43 GMT
Hi, first time poster on here so please go easy :)
I am also running into a hang on boot after running
pacman -Syu
yesterday. So a full system upgrade, which bumped my kernel to 5.19.1 from 5.18.9.

Happy to provide any logs that I can, but I'm not sure how I would provide them since the I can only access the filesystem of the machine via a boot + arch-chroot from a USB stick with the arch installation on it, and networking doesn't seem to be working (on a wifi).

I can say that during boot the last message before the hang that I can see is something like "Reached Target System Time Set" at the default log level and quiet turned off.

Hardware is an Intel i9-9900K with an Nvidia RTX 2080. It is an Alienware 51m Laptop from 2020.

Happy to provide any other information if I can. I am in the process of doing a kernel downgrade to try and get things working again.
Comment by Christian (Watnuss) - Wednesday, 17 August 2022, 10:14 GMT
I also cannot boot into this kernel (tested on a Lenovo Thinkpad T14s and X395). Going back to any 5.18 kernel resolved the issue (whatever was the last in my local pacman pkg cache). I am happy to supply more information and testing stuff but I don't know how.

I am using syslinux as bootloader. In my case it doesn't freeze but it will just reset the machine and therefore I am in an endless boot loop.
Comment by Geoff (perseus) - Wednesday, 17 August 2022, 12:22 GMT
Two reasonably detailed Forum threads deal with this. I started one of them.

With one possible outlier (Grub) they have in common the issue that, regardless of hardware, syslinux (whatever syslinux.cfg may contain), is incapable of booting this kernel. Attempts always end in bootloops. At least that is true for all who have reported the issue. I have not seen any reports of success.

Downgrading - typically to 5.18.16 always fixes the problem.
Comment by Adam Beavan (ajbeavan) - Saturday, 20 August 2022, 16:14 GMT
Have just updated to the latest linux-5.19.2 after rolling back to 5.18.9, still have the same problem where the system fails to load the kernel image,could it possibly be a GRUB issue. have gone back to 15.8.9.
Comment by loqs (loqs) - Saturday, 20 August 2022, 23:02 GMT
Related upstream bug report [1]. Both syslinux and GRUB are reported affected.

Comment by Toolybird (Toolybird) - Sunday, 21 August 2022, 05:03 GMT
@loqs, outstanding support in the forum! The bad commit is AMD related. It's still unclear if bootloader is a factor. Do you think the folks on Intel CPU's are experiencing a different bug?

@ajbeavan, we're still waiting for your hardware details.
Comment by loqs (loqs) - Sunday, 21 August 2022, 05:28 GMT
@toolybird I do not know if Intel CPU's can experience this issue. Could an affected Intel user would please test the patch or the built kernel with it applied from [1] and report back.

Comment by Adam Beavan (ajbeavan) - Sunday, 21 August 2022, 07:51 GMT
As requested the hardware details of the machine, which is a PCSpecialist Vyper Series Laptop
CPU : Intel(R) Core i7-10875
RAM : 16GB
Motherboard : Standard / GM7MP7P(TongFang)
Hard Drive: SSD 500GB Samsung 970
Graphics : IntelCometLake-H GT2[UHD] + nVidia GeForce RTX 2070 Mobile
Comment by loqs (loqs) - Sunday, 21 August 2022, 18:19 GMT Comment by Adam Beavan (ajbeavan) - Sunday, 28 August 2022, 20:47 GMT
I have not applied the kernel patch was sort of hoping the issue may have been resolved with future updates. But just installed the latest 5.19.4 and I am still getting the same system hang issues.
Comment by loqs (loqs) - Sunday, 28 August 2022, 21:03 GMT
The patch will be in linux-mainline 6.0-rc3 available prebuilt from [1] probably available tomorrow if you you do not want to apply the patch yourself or use a package I built.
That patch will make its way to a future stable release.
Queued for 5.19.6 [2]

Comment by loqs (loqs) - Friday, 02 September 2022, 00:10 GMT
Is the issue resolved with linux 5.19.6.arch1-1?
Comment by Christian (Watnuss) - Friday, 02 September 2022, 07:46 GMT
Thanks for pinging the issue. Yes, it is resolved with 5.19.6-arch1-1 =)
Comment by Adam Beavan (ajbeavan) - Saturday, 03 September 2022, 19:59 GMT
Hi have just upgraded to 5.19.6, and I am still getting the system froze problem though is is now intermittent and will sometimes progress to the login stage before hanging. Will look for clues in the journal file...
Comment by Adam Beavan (ajbeavan) - Sunday, 04 September 2022, 06:30 GMT
Looking through the journal file, i am getting the following errors:
FAT-fs (sdd): unable to read boot sector to mark fs as dirty
Sep 03 22:25:45 chippy kernel: sd 3:0:0:0: [sdc] No Caching mode page found
Sep 03 22:25:45 chippy kernel: sd 3:0:0:0: [sdc] Assuming drive cache: write through
Comment by Adam Beavan (ajbeavan) - Monday, 05 September 2022, 19:19 GMT
This has not been fixed, the system still hangs but nothing is being written to the journal file, though I feel it is intel related.
Comment by loqs (loqs) - Monday, 05 September 2022, 20:01 GMT
@ajbeavan then bisect between 5.18 and 5.19 to find the cause, you may need to cherry-pick 4b1c742407571eff58b6de9881889f7ca7c4b4dc if your system was also affected by that issue.
Comment by Adam Beavan (ajbeavan) - Tuesday, 06 September 2022, 17:44 GMT
Latest kernel 5.19.7 seems to be working OK, no freeing at the moment.
Comment by Adam Beavan (ajbeavan) - Tuesday, 06 September 2022, 20:40 GMT
sorry spoke to soon system frozen, very frustrating.
Comment by Jay Wolff (jaywolff5) - Tuesday, 13 September 2022, 14:08 GMT
I've been lurking, following this thread and a few others for a few weeks now hoping there would be a resolution by now but I'm having similar issues still as well.

Ever since the 5.19 kernel update my machine has crashed at least once a day. It's usually random crashes. However, I can also reproduce it crashing by just undocking my laptop, or if I just leave it on and step away from the computer for over an hour, I'll come back and its completely unresponsive, can't even jump into another tty. While I usually am docked I have had crashes while not connected to the dock.

My System info:
Host: 20L70025US ThinkPad T480s
OS: Arch Linux x86_64
CPU: Intel i7-8650U (8) @ 4.200GHz
GPU: Intel UHD Graphics 620
Kernel: 5.19.7-arch1-1
Mesa version: 22.1.7-1
Xserver version (if applicable): xorg-server 21.1.4-1
Desktop manager and compositor: i3-gaps 4.20.1-2
Dock: ThinkPad USB-C Dock Gen2 (LDC-G2)

I've attached the full journal log with `journalctl -b -1 -x` from boot to crash, where the system crashed after I undocked it.
   crash.log (275.8 KiB)
Comment by loqs (loqs) - Tuesday, 13 September 2022, 17:21 GMT
@jaywolff5 please try linux 5.19.8.arch1-1
Comment by Jay Wolff (jaywolff5) - Tuesday, 13 September 2022, 19:37 GMT
Can confirm after the kernel update to 5.19.8-arch1-1 I can no longer reproduce a system crash by un-docking/re-docking the laptop. Sweet. Will also follow up on stability after a day or two of normal usage. Thanks for the heads up on the latest update @loqs
Comment by Adam Beavan (ajbeavan) - Wednesday, 14 September 2022, 16:25 GMT
Updated to the latest kernel and still getting random freezing,if I run libvirt and pci passthrough to the NVIDIA card then it will certainly freeze and require a number of power cycles to get a login prompt. Looks like I am going back to 5.18 again....
Comment by Jay Wolff (jaywolff5) - Friday, 16 September 2022, 13:37 GMT
Very happy to report that my system has been stable since the 5.19.8 update, haven't had a single crash since.