FS#72045 - [linux] linux 5.14.1.arch1-1 kernel panic when shutting down

Attached to Project: Arch Linux
Opened by Andrew Stephen (andrewstephengames) - Monday, 06 September 2021, 11:50 GMT
Last edited by Jan Alexander Steffens (heftig) - Thursday, 09 September 2021, 17:45 GMT
Task Type Bug Report
Category Kernel
Status Closed
Assigned To Jan Alexander Steffens (heftig)
Architecture All
Severity High
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description:
Shutting down triggers kernel panic after tty1 is closed. Rebooting does not cause the kernel panic.

Additional info:
package: testing/linux 5.14.1.arch1-1
CPU: Intel Pentium G620
RAM: 4GB DDR3 1066Mhz
MB: HP 1497
GPU: Intel HD Graphics 2000

Log: in the file attached

Steps to reproduce:
Poweroff computer either in tty or in X
This task depends upon

Closed by  Jan Alexander Steffens (heftig)
Thursday, 09 September 2021, 17:45 GMT
Reason for closing:  Fixed
Additional comments about closing:  5.14.2.arch1-2
Comment by Andrew Stephen (andrewstephengames) - Monday, 06 September 2021, 11:57 GMT
The file didn't get attached, this is the kernel panic:
   20210906_004954.jpg (1.82 MiB)
Comment by John (graysky) - Monday, 06 September 2021, 17:47 GMT
I can confirm a reproducible panic on reboot with an Intel Haswell CPU. I do not see anything recorded in the journal as it successfully hit the 'System Reboot' target and journaling stopped.
Comment by loqs (loqs) - Monday, 06 September 2021, 20:50 GMT
Have you tried reverting cb011044e34c293e139570ce5c01aed66a34345c? Symptoms are different but the commit is acknowledged to be broken.

https://lore.kernel.org/linux-watchdog/20210803165108.4154cd52%40endymion/
Comment by Andrew Stephen (andrewstephengames) - Tuesday, 07 September 2021, 06:22 GMT
Reverting the commit does seem to fix the issue.
Comment by axt (axt) - Tuesday, 07 September 2021, 13:06 GMT
I have to confirm this problem, kernel panic on shutdown with kernel 5.14.1-arch1-1 (Ivy Bridge i3, 8 GiB). Up to and including 5.13.13-arch1-1 w/o problems.
Comment by John (graysky) - Tuesday, 07 September 2021, 13:43 GMT
I applied the revert to 5.14.1, but found the panic still occurred.
Comment by loqs (loqs) - Tuesday, 07 September 2021, 17:42 GMT
@graysky can you bisect to find the cause?
Comment by John (graysky) - Tuesday, 07 September 2021, 17:57 GMT
Love to but limited time for the next few weeks. Doubtful I will have an opportunity.
Comment by Peter Buxton (somercet) - Wednesday, 08 September 2021, 13:34 GMT
I have a Core i5 560M, Arrandale/Westmere-Nehalem. For me, the problem came up when suspending and restarting.

I need to install the kernel (and a bunch of the tool chain), but I will try to isolate the fault.
Comment by John (graysky) - Wednesday, 08 September 2021, 14:48 GMT
I just built 5.14.2 and was able to reboot cleanly/no panic. Others?
Comment by Jan Alexander Steffens (heftig) - Thursday, 09 September 2021, 00:09 GMT
I've added a patch to 5.14.2.arch1 that might be relevant here.
Comment by Peter Buxton (somercet) - Thursday, 09 September 2021, 00:12 GMT
Yeah, this patch failed.

https://lkml.org/lkml/2021/7/26/349

I then installed 5.14.2.arch1-1 and the system is still crashing on reboot. The screen is not large enough to show the whole panic msg and this laptop is too slow to build and rebuild the kernel. :-(
Comment by Herbert J. Skuhra (zapata) - Thursday, 09 September 2021, 06:09 GMT
With kernel 5.14.2.arch1-1 my laptop is still crashing on shutdown/reboot. Is this crash logged anywhere? Otherwise I could send a photo.
Comment by John (graysky) - Thursday, 09 September 2021, 08:16 GMT
@zapata - See my comment above, "I do not see anything recorded in the journal as it successfully hit the 'System Reboot' target and journaling stopped." That was true on my Haswell CPU. As I said, when I built and booted into my 5.14.2-1 with this PKGBUILD[1], I do NOT see the bug (5 successful reboots then stopped testing).

When I tried the official 5.14.2-arch1-1 kernel, I DO see the bug.

Note that my PKGBUILD was built before hefteg updated the official one so it currently lacks his customization, including:

0002-Bluetooth-Move-shutdown-callback-before-flushing-tx-.patch
0003-watchdog-iTCO_wdt-Fix-detection-of-SMI-off-case.patch
0004-Arch-Linux-kernel-v5.14.2-arch1.patch

Also, I used a slightly older config and I do not build with the DEBUG_INFO=y stuff, so those are differences too.
When I have more time today, I will update mine with these patches and the new config option and confirm functionality.

1. https://github.com/graysky2/linux-optimized

If you want to just download a pre-compiled kernel based on my 5.14.2 to see if that works for you, they are hosted on my [repo-ck]. See the wiki: https://wiki.archlinux.org/title/Unofficial_user_repositories/Repo-ck
Comment by Jan Alexander Steffens (heftig) - Thursday, 09 September 2021, 09:42 GMT
It must be CONFIG_WATCHDOG_HRTIMER_PRETIMEOUT=y triggering this crash.
Comment by Herbert J. Skuhra (zapata) - Thursday, 09 September 2021, 17:13 GMT
5.14.2-arch1-2 works for me. Thanks.
Comment by Andrew Stephen (andrewstephengames) - Thursday, 09 September 2021, 17:22 GMT
5.14.2 also fixed the problem for me. Thanks very much!
Comment by John (graysky) - Thursday, 09 September 2021, 17:30 GMT
@heftig - 5.14.2-arch1-2 does not trigger the bug on my Haswell system.

Loading...