FS#76922 - [linux] Kernel 6.1 causes the system to freeze on haswell laptop

Attached to Project: Arch Linux
Opened by Sergio Callegari (callegar) - Friday, 23 December 2022, 20:13 GMT
Last edited by Jan Alexander Steffens (heftig) - Wednesday, 18 January 2023, 21:54 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Jan Alexander Steffens (heftig)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 4
Private No

Details

Description:

Kernels 6.1.0 and 6.1.1 are KO on a haswell laptop (CPU: quad core Intel Core i7-4750HQ with integrated graphics). Systems regularly freezes a few seconds after launching the desktop environment.

Note that you need to actually launch the desktop env (up to the `sddm` screen things are fine, if you do not login there but you switch to a text console the system works fine) to see the issue.

I see the issue using a KDE plasma desktop (with compositing and opengl acceleration). Wayland or X11 both show the issue.

This is certainly a kernel regression because the same system works just fine with kernels up to the 6.0.x series.

The freeze is not restricted to the DE, but is a full system lock: the machine stops responding to ping and not even the Sysrq magic (that I keep enabled) works.

The freeze seems to come with data loss and corruption of filesystem if data is in transit. System does not appear to be able to log anything on persistent storage about the problem.

I have reported the issue upstream at https://lore.kernel.org/regressions/38cd1c38-b469-f25d-369e-57877865fdbb%40gmail.com/T/#t

Also posting here just in case this turned out to be specific to arch and derivatives (don't think so) and to check if someone on arch can reproduce.




Additional info:
* package version(s)
* config and/or log files etc.
* link to upstream bug report, if any

Steps to reproduce:
This task depends upon

Closed by  Jan Alexander Steffens (heftig)
Wednesday, 18 January 2023, 21:54 GMT
Reason for closing:  Fixed
Additional comments about closing:  6.1.7.arch1-1
Comment by loqs (loqs) - Friday, 23 December 2022, 22:06 GMT
$ git bisect start
status: waiting for both good and bad commits
$ git bisect bad v6.1
status: waiting for good commit(s), bad commit known
$ git bisect good v6.0
Bisecting: 7514 revisions left to test after this (roughly 13 steps)
[33e591dee915832c618cf68bb1058c8e7d296128] Merge tag 'phy-for-6.1' of git://git.kernel.org/pub/scm/linux/kernel/git/phy/linux-phy

https://drive.google.com/file/d/1GnjhKe-7r34GKwAkWxtDWjWP1OAjICB5/view?usp=share_link linux-6.0-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1zlWGu1x24xbSiqw3q2uJwUu0aiLZjxio/view?usp=share_link linux-headers-6.0-1-x86_64.pkg.tar.zst

https://drive.google.com/file/d/1kS7oHaIDYpHHkogJDJf_fwR7QvCn7faJ/view?usp=share_link linux-6.1-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1hsDiYp9gFiHa8Z0w3-BsJIjb6GQqNuTA/view?usp=share_link linux-headers-6.1-1-x86_64.pkg.tar.zst

https://drive.google.com/file/d/12nS4h0WAJVh9VRXs2z0nALA53D3Rnoo4/view?usp=share_link linux-6.0.r7594.g33e591dee915-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1YVkwO-Q8_TO0yHR4fJN-7I-8c7a_iVyt/view?usp=share_link linux-headers-6.0.r7594.g33e591dee915-1-x86_64.pkg.tar.zst
Comment by Sergio Callegari (callegar) - Saturday, 24 December 2022, 08:08 GMT
Am I meant to try linux-6.0.r7594.g33e591dee915-1-x86_64.pkg.tar.zst and say whether it is good or bad? If so, please let me know and thanks for building it...
Comment by loqs (loqs) - Saturday, 24 December 2022, 11:11 GMT
Please try all three. This ensures there is a known good and bad as well as the first bisection result.
Comment by Sergio Callegari (callegar) - Sunday, 25 December 2022, 11:33 GMT
Bracketed:

linux-6.0-1-x86_64.pkg.tar.zst is good (in fact, my usual kernel is 6.0.15, also good).

linux-6.0.r7594.g33e591dee915-1-x86_64.pkg.tar.zst is bad
Comment by loqs (loqs) - Sunday, 25 December 2022, 14:24 GMT
git bisect bad
Bisecting: 4354 revisions left to test after this (roughly 12 steps)
[a47e60729d9624e931f988709ab76e043e2ee8b9] Merge tag 'backlight-next-6.1' of git://git.kernel.org/pub/scm/linux/kernel/git/lee/backlight

https://drive.google.com/file/d/1doDi61mOL3flwTC5pTQLY9wbKVd85zH1/view?usp=share_link linux-6.0.r3239.ga47e60729d96-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1OtWHrGP9EbXiMExgRVFQvulDFT7tSuYe/view?usp=share_link linux-headers-6.0.r3239.ga47e60729d96-1-x86_64.pkg.tar.zst
Comment by Sergio Callegari (callegar) - Monday, 26 December 2022, 09:41 GMT
linux-6.0.r3239.ga47e60729d96-1-x86_64.pkg.tar.zst is bad
Comment by loqs (loqs) - Monday, 26 December 2022, 10:37 GMT
git bisect bad
Bisecting: 1573 revisions left to test after this (roughly 11 steps)
[915b96c52763e2988e6368b538b487a7138b8fa4] Merge tag 'wireless-next-2022-09-30' of git://git.kernel.org/pub/scm/linux/kernel/git/wireless/wireless-next

https://drive.google.com/file/d/1ngfZDVcNbh7BMHhrx82Cvs-cugBww4QW/view?usp=share_link linux-6.0rc7.r1795.g915b96c52763-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1ds-cDkG7qY4yiK8QU5H27cPMVtGGgPFy/view?usp=share_link linux-headers-6.0rc7.r1795.g915b96c52763-1-x86_64.pkg.tar.zst
Comment by Sergio Callegari (callegar) - Monday, 26 December 2022, 11:45 GMT
linux-6.0rc7.r1795.g915b96c52763-1-x86_64.pkg.tar.zst is bad
Comment by Sergio Callegari (callegar) - Monday, 26 December 2022, 12:53 GMT
linux-6.0rc7.r1795.g915b96c52763-1-x86_64.pkg.tar.zst is bad
Comment by loqs (loqs) - Monday, 26 December 2022, 13:17 GMT
git bisect bad
Bisecting: 837 revisions left to test after this (roughly 10 steps)
[2c119d9982b1aba54a2eca59c2455cd09f3bc749] net: dsa: microchip: add the support for set_ageing_time

https://drive.google.com/file/d/1TFAxIDutYYaldVr-yd9WDCXi9ZBYcSrd/view?usp=share_link linux-6.0rc4.r953.g2c119d9982b1-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1OHSuUw2xhZje8xBG_J8ORZXOB5XvtLH5/view?usp=share_link linux-headers-6.0rc4.r953.g2c119d9982b1-1-x86_64.pkg.tar.zst
Comment by Sergio Callegari (callegar) - Monday, 26 December 2022, 13:35 GMT
linux-6.0rc4.r953.g2c119d9982b1-1-x86_64.pkg.tar.zst is good
Comment by loqs (loqs) - Monday, 26 December 2022, 14:02 GMT
git bisect good
Bisecting: 418 revisions left to test after this (roughly 9 steps)
[454b20e19322e6a9375cbaad68fff3c93bd27716] net: ethernet: mtk_eth_soc: fix usage of foe_entry_size

https://drive.google.com/file/d/1ZhJNTQD2w9I7taNqKkmIkHoexbk7h56e/view?usp=share_link linux-6.0rc6.r1412.g454b20e19322-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1HVnQKfBfu0wmCABTPj7zHXSJYiWbckpn/view?usp=share_link linux-headers-6.0rc6.r1412.g454b20e19322-1-x86_64.pkg.tar.zst
Comment by Sergio Callegari (callegar) - Monday, 26 December 2022, 15:16 GMT
linux-6.0rc6.r1412.g454b20e19322-1-x86_64.pkg.tar.zst is good
Comment by Leonardo Costa (leo95) - Monday, 26 December 2022, 15:30 GMT
This also affects Broadwell Mobile(5200U) and I have been using 6.0.12 as a workaround
Comment by loqs (loqs) - Monday, 26 December 2022, 16:40 GMT
git bisect good
Bisecting: 198 revisions left to test after this (roughly 8 steps)
[6cf5e9066dd3332cf4c77ea95a116f70e7f9acf7] Merge ath-next from git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git

https://drive.google.com/file/d/1r_BnHzEG2tRchJYCs7snLjUhuYV4ywb6/view?usp=share_link linux-6.0rc3.r858.g6cf5e9066dd3-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1k9q7fbKa9Abg0TT6QMThs0_Uh5OxHqmq/view?usp=share_link linux-headers-6.0rc3.r858.g6cf5e9066dd3-1-x86_64.pkg.tar.zst
Comment by Sergio Callegari (callegar) - Monday, 26 December 2022, 17:56 GMT
linux-6.0rc3.r858.g6cf5e9066dd3-1-x86_64.pkg.tar.zst is bad

Please wait before generating a new kernel. Must retest linux-6.0rc6.r1412.g454b20e19322-1-x86_64.pkg.tar.zst. Did not freeze and worked regularly for more than one hour, but did not shut down well.

Good to hear that I am not the only one experiencing the issue.

@leo95 probably I will not be able to do enough tests to complete. My office is shutting down and I won't be allowed to take this machine out. Will probably reopen on 31st Dec. Can you take over the bisection? First make sure that my bracketing of the issue works for you too.
Comment by Sergio Callegari (callegar) - Tuesday, 27 December 2022, 11:07 GMT
Retested `linux-6.0rc6.r1412.g454b20e19322-1-x86_64.pkg.tar.zst` that was indeed good.

Please generate another kernel.

Looks like I am going to have another day for testing.
Comment by loqs (loqs) - Tuesday, 27 December 2022, 11:35 GMT
git bisect bad
Bisecting: 116 revisions left to test after this (roughly 7 steps)
[2e405cff04ecd633285cc2051699383d535b3912] wifi: rtw89: coex: update coexistence to 6.3.0

https://drive.google.com/file/d/1RX4YkNme7UupV1PYzoIJwpWm3KoWIMcs/view?usp=share_link linux-6.0rc3.r741.g2e405cff04ec-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1TH4FBHWJSgTLZF1lbSJ4nsS_o_mgPCM5/view?usp=share_link linux-headers-6.0rc3.r741.g2e405cff04ec-1-x86_64.pkg.tar.zst
Comment by Sergio Callegari (callegar) - Tuesday, 27 December 2022, 12:22 GMT
linux-6.0rc3.r741.g2e405cff04ec-1-x86_64.pkg.tar.zst is bad
Comment by loqs (loqs) - Tuesday, 27 December 2022, 13:01 GMT
git bisect bad
Bisecting: 51 revisions left to test after this (roughly 6 steps)
[e963a19c64ac0d2f8785d36a27391abd91ac77aa] wifi: rtl8xxxu: gen2: Fix mistake in path B IQ calibration

https://drive.google.com/file/d/1m99lUnCZ27wlBSJ9CIViQEdsOU2HGlAp/view?usp=share_link linux-6.0rc3.r689.ge963a19c64ac-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1VsRuDdUoLHOYpmOx3biDKqEcJ0LLg-PS/view?usp=share_link linux-headers-6.0rc3.r689.ge963a19c64ac-1-x86_64.pkg.tar.zst
Edit:
Does applying https://patchwork.kernel.org/project/linux-wireless/patch/20221217085624.52077-1-nbd%40nbd.name/ resolve the issue?
https://drive.google.com/file/d/1LMwmrY5VRYXTazOlCTVzhFB0WRe8cDRY/view?usp=share_link linux-6.1.1.arch1-1.2-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1ayP1ZhBMY_bu7_q6bXDL8O8yGxGiVZxR/view?usp=share_link linux-headers-6.1.1.arch1-1.2-x86_64.pkg.tar.zst
Comment by Sergio Callegari (callegar) - Tuesday, 27 December 2022, 13:26 GMT
linux-6.0rc3.r689.ge963a19c64ac-1-x86_64.pkg.tar.zst is bad
Comment by loqs (loqs) - Tuesday, 27 December 2022, 13:28 GMT
Please try the kernel from edit to my last post.
Comment by Joao Fonseca (knaick) - Tuesday, 27 December 2022, 13:43 GMT
I have the same PC as @leo95. But cannot test this fix before the beginning of the year. Will keep you updated after that @callegar
Comment by Sergio Callegari (callegar) - Tuesday, 27 December 2022, 14:22 GMT
That's it: kernel linux-6.1.1.arch1-1.2-x86_64.pkg.tar.zst from your last edit is OK!!!

Weird I was not getting the crash and call trace as in the post https://patchwork.kernel.org/project/linux-wireless/patch/20221217085624.52077-1-nbd%40nbd.name/
Comment by Toolybird (Toolybird) - Tuesday, 27 December 2022, 21:50 GMT
Awesome work @loqs, yet again! Thanks for all your efforts.
Comment by Toolybird (Toolybird) - Thursday, 05 January 2023, 20:49 GMT
Dupes  FS#77000   FS#77011 
Comment by loqs (loqs) - Thursday, 05 January 2023, 20:51 GMT Comment by i0f (I0F) - Wednesday, 18 January 2023, 12:24 GMT
Hi, the patch seems to be accepted. Is there a chance to get it into the arch kernel?
Comment by Jan Alexander Steffens (heftig) - Wednesday, 18 January 2023, 21:54 GMT
Patch added to 6.1.7.arch1-1

Loading...