FS#73387 - [linux-firmware] Intel AX210 iwlwifi broken on kernel and firmware upgrade

Attached to Project: Arch Linux
Opened by Alexander Kissinger (kissy) - Monday, 17 January 2022, 08:33 GMT
Last edited by Jelle van der Waa (jelly) - Thursday, 14 September 2023, 17:52 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Laurent Carlier (lordheavy)
Architecture All
Severity High
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 12
Private No

Details

Description:

Upon upgrading to kernel 5.16.0 and linux-firmware, version 20211216.f682ecb-1 iwlwifi seems
to crash loading the latest ucode.

[ 418.714212] Intel(R) Wireless WiFi driver for Linux
[ 418.732635] iwlwifi 0000:ae:00.0: api flags index 2 larger than supported by driver
[ 418.732649] iwlwifi 0000:ae:00.0: TLV_FW_FSEQ_VERSION: FSEQ Version: 0.0.2.34
[ 418.732924] iwlwifi 0000:ae:00.0: loaded firmware version 67.8f59b80b.0 ty-a0-gf-a0-67.ucode op_mode iwlmvm
[ 418.752983] iwlwifi 0000:ae:00.0: Detected Intel(R) Wi-Fi 6 AX210 160MHz, REV=0x420
[ 418.760145] thermal thermal_zone6: failed to read out thermal zone (-61)
[ 418.760155] iwlwifi 0000:ae:00.0: WRT: Failed to set DRAM buffer for alloc id 1, ret=-1
[ 418.760157] iwlwifi 0000:ae:00.0: WRT: Failed to set DRAM buffer for alloc id 2, ret=-1
[ 418.760158] iwlwifi 0000:ae:00.0: WRT: Failed to set DRAM buffer for alloc id 3, ret=-1
[ 419.784846] iwlwifi 0000:ae:00.0: SecBoot CPU1 Status: 0x75a7, CPU2 Status: 0xb03
[ 419.784896] iwlwifi 0000:ae:00.0: UMAC PC: 0x8047f8e0
[ 419.784919] iwlwifi 0000:ae:00.0: LMAC PC: 0x0
[ 419.784924] iwlwifi 0000:ae:00.0: WRT: Collecting data: ini trigger 13 fired (delay=0ms).
[ 419.785096] iwlwifi 0000:ae:00.0: Loaded firmware version: 67.8f59b80b.0 ty-a0-gf-a0-67.ucode
[ 419.785101] iwlwifi 0000:ae:00.0: 0x00000000 | ADVANCED_SYSASSERT
[ 419.785106] iwlwifi 0000:ae:00.0: 0x00000000 | trm_hw_status0
[ 419.785109] iwlwifi 0000:ae:00.0: 0x00000000 | trm_hw_status1
[ 419.785112] iwlwifi 0000:ae:00.0: 0x00000000 | branchlink2

Forcing iwlwifi to load:
iwlwifi-ty-a0-gf-a0-63.ucode

by renaming/removing:
iwlwifi-ty-a0-gf-a0-67
iwlwifi-ty-a0-gf-a0-66

in /lib/firmware seems to work.

[ 639.727284] Intel(R) Wireless WiFi driver for Linux
[ 639.747131] iwlwifi 0000:ae:00.0: Direct firmware load for iwlwifi-ty-a0-gf-a0-67.ucode failed with error -2
[ 639.747163] iwlwifi 0000:ae:00.0: Direct firmware load for iwlwifi-ty-a0-gf-a0-66.ucode failed with error -2
[ 639.747179] iwlwifi 0000:ae:00.0: Direct firmware load for iwlwifi-ty-a0-gf-a0-65.ucode failed with error -2
[ 639.747193] iwlwifi 0000:ae:00.0: Direct firmware load for iwlwifi-ty-a0-gf-a0-64.ucode failed with error -2
[ 639.752129] iwlwifi 0000:ae:00.0: api flags index 2 larger than supported by driver
[ 639.752146] iwlwifi 0000:ae:00.0: TLV_FW_FSEQ_VERSION: FSEQ Version: 0.0.2.25
[ 639.752452] iwlwifi 0000:ae:00.0: loaded firmware version 63.c04f3485.0 ty-a0-gf-a0-63.ucode op_mode iwlmvm
[ 639.770851] iwlwifi 0000:ae:00.0: Detected Intel(R) Wi-Fi 6 AX210 160MHz, REV=0x420
[ 639.778086] thermal thermal_zone6: failed to read out thermal zone (-61)
[ 639.932368] iwlwifi 0000:ae:00.0: loaded PNVM version 4b50f925
[ 639.944152] iwlwifi 0000:ae:00.0: Detected RF GF, rfid=0x10d000
[ 640.014414] iwlwifi 0000:ae:00.0: base HW address: f4:7b:09:99:60:f6
[ 640.036907] iwlwifi 0000:ae:00.0 wlp174s0: renamed from wlan0

Additional info:
* package version(s)
kernel 5.16.0-arch1-1
linux-firmware 20211216.f682ecb-1

* config and/or log files etc.
* link to upstream bug report, if any

Steps to reproduce:

Have the following wifi card

ae:00.0 Network controller: Intel Corporation Wi-Fi 6 AX210/AX211/AX411 160MHz (rev 1a)
Subsystem: Intel Corporation Wi-Fi 6 AX210 160MHz
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 18
IOMMU group: 19
Region 0: Memory at 88400000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [c8] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [40] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <512ns, L1 unlimited
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr+ NoSnoop+ FLReset-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L1, Exit Latency L1 <8us
ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM+ AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s (ok), Width x1 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range B, TimeoutDis+ NROPrPrP- LTR+
10BitTagComp- 10BitTagReq- OBFF Via WAKE#, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 16ms to 55ms, TimeoutDis- LTR+ OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [80] MSI-X: Enable+ Count=16 Masked-
Vector table: BAR=0 offset=00002000
PBA: BAR=0 offset=00003000
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [14c v1] Latency Tolerance Reporting
Max snoop latency: 3145728ns
Max no snoop latency: 3145728ns
Capabilities: [154 v1] L1 PM Substates
L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
PortCommonModeRestoreTime=30us PortTPowerOnTime=18us
L1SubCtl1: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+
T_CommonMode=0us LTR1.2_Threshold=65536ns
L1SubCtl2: T_PwrOn=44us
Kernel driver in use: iwlwifi
Kernel modules: iwlwifi

Upgrade kernel and linux-firmware to affected versions. Observe the crash in dmesg mentioned above.
This task depends upon

Closed by  Jelle van der Waa (jelly)
Thursday, 14 September 2023, 17:52 GMT
Reason for closing:  Deferred
Additional comments about closing:  Old kernel, please retry with the latest
Comment by Tobias Powalowski (tpowa) - Tuesday, 18 January 2022, 21:15 GMT
Please try latest package from teseting.
Comment by Alexander Kissinger (kissy) - Tuesday, 18 January 2022, 23:19 GMT
Installed testing and still fails to load. I've attached the dmesg log. Here are the new files:

# ls /lib/firmware/iwlwifi-ty-a0-gf-a0*
/lib/firmware/iwlwifi-ty-a0-gf-a0-59.ucode /lib/firmware/iwlwifi-ty-a0-gf-a0-63.ucode /lib/firmware/iwlwifi-ty-a0-gf-a0-67.ucode /lib/firmware/iwlwifi-ty-a0-gf-a0.pnvm
/lib/firmware/iwlwifi-ty-a0-gf-a0-62.ucode /lib/firmware/iwlwifi-ty-a0-gf-a0-66.ucode /lib/firmware/iwlwifi-ty-a0-gf-a0-68.ucode
   wifi.log (16.2 KiB)
Comment by Hexhu (hexhu) - Wednesday, 02 February 2022, 09:01 GMT
Just FYI AX200 works again after upgrading to linux 5.16.4.arch1-1 and linux-firmware 20220119.0c6a7b3-2, has been broken as described in https://bugs.archlinux.org/task/73159.

Not sure if your issue with AX210 is also resolved. Can you try the new packages?
Comment by Térence Clastres (Terence) - Monday, 07 February 2022, 11:27 GMT
I'm on 5.16.5 with an AX210 and it's not fixed for me. Removing 67/66 firmware files as suggested by kissy also works for me.
Comment by Corey Hinshaw (electrickite) - Wednesday, 09 February 2022, 21:30 GMT
Observed the same issue today after updating to 5.16.8 with an AX210:

Linux 5.16.8-zen1-1-zen #1 ZEN SMP PREEMPT Tue, 08 Feb 2022 21:21:15 +0000 x86_64 GNU/Linux
linux-firmware 20220119.0c6a7b3-2
Detected Intel(R) Wi-Fi 6 AX210 160MHz, REV=0x420
Comment by Lennart (LeAlex27) - Wednesday, 09 February 2022, 21:37 GMT
I'm using linux-lts 5.15.22-1 with linux-firmware 20220119.0c6a7b3-2 and ran into the same problem.
Deleting 66/67 helped. Instead, firmware ty-a0-gf-a0-63.ucode was loaded, which works.
Comment by Alexander Kissinger (kissy) - Wednesday, 16 February 2022, 01:36 GMT
Updated and the issue persists.

linux 5.16.8.arch1-1
linux-firmware 20220119.0c6a7b3-2
Comment by Heitor de Bittencourt (heitorpb) - Tuesday, 01 March 2022, 14:44 GMT
I can confirm the issue. Just updated the kernel to 5.16.11-arch1-1, linux-firmware to 20220209.6342082-1 and the wifi card stopped working.

Removing /lib/firmware/iwlwifi-ty-a0-gf-a0-6{6,7,8} solved the problem.
Comment by Grzegorz (reset) - Wednesday, 02 March 2022, 08:17 GMT
new firmware versions for ax210 available
Comment by Jun (jun) - Sunday, 06 March 2022, 08:52 GMT
I think I might have experienced the same issue. Sometimes wifi is detected after boot but most of the time it doesn't.

I also have noticed that network-online.target took 2 minutes to start.

Following heitorpb, removing /lib/firmware/iwlwifi-ty-a0-gf-a0-6{6,7,8} works. And network-online.target only takes a second to start.
Comment by Magnus (DeArchDev) - Sunday, 06 March 2022, 14:34 GMT
Link to upstream bug report(s) :
Regarding kernel failing to load/read thermal zone : https://bugzilla.kernel.org/show_bug.cgi?id=201761,
Regarding ucode crash : https://bugzilla.kernel.org/show_bug.cgi?id=211019
Comment by Alice Addison (rlmineing_dead) - Saturday, 26 March 2022, 15:47 GMT
interestingly, I decided to boot into windows 11 and faced the same issue, on windows I had to rollback the driver to a previous version. It seems intel fucked something up on their end and they screwed up all platforms
Comment by Alice Addison (rlmineing_dead) - Saturday, 26 March 2022, 16:07 GMT
I can confirm Jun's fix worked for my on my AX210
Comment by Jeoshua Collins (jeoshua) - Sunday, 27 March 2022, 00:38 GMT
No fixes I could find around the internet worked for me. I decided to look at my custom compiled kernel and found that the iwlmvm and mac80211 kernel modules were not compiled in. The same went for several other kernels I looked at, such as xanmod. After turning these modules on in my kernel config and recompiling, the card works.

For those still experiencing this issue after trying many things, you might try a custom kernel that you've ensured the following modules are being compiled in:
cfg80211
mac80211
iwlmvm
iwlwifi
libarc4
Comment by Steve McClain (sdmcclain) - Monday, 28 March 2022, 01:01 GMT
Confirming that this issue still persists.

Wifi Adapter: Intel Corporation Wi-Fi 6 AX210 160MHz
Kernel: 5.16.16-arch1-1
linux-firmware: 20220309.cd01f85-1

Following the advice in this thread, I have been able to restore WiFi by using version 63 of the firmware:

~ ls /lib/firmware/iwlwifi-ty-a0-gf-a0*
/lib/firmware/iwlwifi-ty-a0-gf-a0-59.ucode.xz
/lib/firmware/iwlwifi-ty-a0-gf-a0-62.ucode.xz
/lib/firmware/iwlwifi-ty-a0-gf-a0-63.ucode.xz
/lib/firmware/iwlwifi-ty-a0-gf-a0-66.ucode.xz.bak
/lib/firmware/iwlwifi-ty-a0-gf-a0-67.ucode.xz.bak
/lib/firmware/iwlwifi-ty-a0-gf-a0-68.ucode.xz.bak
/lib/firmware/iwlwifi-ty-a0-gf-a0-71.ucode.xz.bak
/lib/firmware/iwlwifi-ty-a0-gf-a0.pnvm.xz

~ sudo dmesg| grep iwlwifi
[ 2.279117] iwlwifi 0000:aa:00.0: enabling device (0000 -> 0002)
[ 2.285822] iwlwifi 0000:aa:00.0: Direct firmware load for iwlwifi-ty-a0-gf-a0-67.ucode failed with error -2
[ 2.286026] iwlwifi 0000:aa:00.0: Direct firmware load for iwlwifi-ty-a0-gf-a0-66.ucode failed with error -2
[ 2.286066] iwlwifi 0000:aa:00.0: Direct firmware load for iwlwifi-ty-a0-gf-a0-65.ucode failed with error -2
[ 2.286105] iwlwifi 0000:aa:00.0: Direct firmware load for iwlwifi-ty-a0-gf-a0-64.ucode failed with error -2
[ 2.325558] iwlwifi 0000:aa:00.0: api flags index 2 larger than supported by driver
[ 2.325573] iwlwifi 0000:aa:00.0: TLV_FW_FSEQ_VERSION: FSEQ Version: 0.0.2.25
[ 2.325786] iwlwifi 0000:aa:00.0: loaded firmware version 63.c04f3485.0 ty-a0-gf-a0-63.ucode op_mode iwlmvm
[ 2.502175] iwlwifi 0000:aa:00.0: Detected Intel(R) Wi-Fi 6 AX210 160MHz, REV=0x420
[ 2.656589] iwlwifi 0000:aa:00.0: loaded PNVM version 05a8dfca
[ 2.667632] iwlwifi 0000:aa:00.0: Detected RF GF, rfid=0x10d000
[ 2.737624] iwlwifi 0000:aa:00.0: base HW address: f4:7b:09:9b:ab:db
[ 3.457796] iwlwifi 0000:aa:00.0: Got NSS = 4 - trimming to 2

Loading...