FS#64018 - After upgrading to kernel 5.3 module e1000e not loaded - no network

Attached to Project: Arch Linux
Opened by Peter Sulyok (peter_s) - Friday, 04 October 2019, 17:23 GMT
Last edited by Antonio Rojas (arojas) - Thursday, 30 January 2020, 13:25 GMT
Task Type Bug Report
Category Kernel
Status Closed
Assigned To No-one
Architecture x86_64
Severity High
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 4
Private No

Details

After upgrading to kernel 5.3.1-arch1-1-ARCH (from 5.2.x latest) my static ethernet configuration is not working anymore:

systemd[1]: Failed to start Static ethernet connection.

and root cause is that the e1000e module is not loaded:

Oct 04 17:25:57 gianta kernel: e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
Oct 04 17:25:57 gianta kernel: e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
Oct 04 17:25:57 gianta kernel: e1000e 0000:00:1f.6: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
Oct 04 17:25:57 gianta kernel: e1000e 0000:00:1f.6 0000:00:1f.6 (uninitialized): registered PHC clock
Oct 04 17:25:57 gianta kernel: e1000e 0000:00:1f.6 eth0: (PCI Express:2.5GT/s:Width x1) 70:85:c2:a4:d0:16
Oct 04 17:25:57 gianta kernel: e1000e 0000:00:1f.6 eth0: Intel(R) PRO/1000 Network Connection
Oct 04 17:25:57 gianta kernel: e1000e 0000:00:1f.6 eth0: MAC: 13, PHY: 12, PBA No: FFFFFF-0FF
Oct 04 17:25:57 gianta kernel: e1000e 0000:00:1f.6 eno1: renamed from eth0
Oct 04 17:26:04 gianta kernel: e1000e: eno1 NIC Link is Down
Oct 04 17:26:04 gianta kernel: e1000e: eno1 NIC Link is Up 1000 Mbps Half Duplex, Flow Control: None

The link is down and the static network configuration is not working. If I downgrade to kernel linux-lts (4.19) everything is working fine again.

This is a AsRock Z390 Taichi motherboard with Intel i9-9900K CPU
The network chip is Intel i219-V (all other network components are disabled for me).

I thought there will be a quick kernel upgrade on 5.3 but it is not coming.
This task depends upon

Closed by  Antonio Rojas (arojas)
Thursday, 30 January 2020, 13:25 GMT
Reason for closing:  Fixed
Comment by loqs (loqs) - Friday, 04 October 2019, 18:15 GMT
https://bugzilla.kernel.org/show_bug.cgi?id=205047 ?
Comment by Peter Sulyok (peter_s) - Saturday, 05 October 2019, 14:33 GMT
Thanks for reporting this to kernel.org.
Comment by loqs (loqs) - Saturday, 05 October 2019, 18:18 GMT
Please test the attached package for a DKMS build of the e1000e with the commit identified in the kernel bug report reverted to confirm it is the same issue.
(application/x-gzip)    e1000e-dkms-5.3.1-1.src.tar.gz (1.1 KiB)
Comment by Robin KERDILES (Dr-Shadow) - Tuesday, 08 October 2019, 15:40 GMT
Same problem on Asrock Z370 Taichi motherboard with i7-8700k CPU and linux kernel 5.3.5.

The dkms package fixes the eno1 interface.
Comment by loqs (loqs) - Tuesday, 08 October 2019, 21:00 GMT
Thank you for testing. Please also test this PKGBUILD for the e1000e out-of-tree driver version 3.6.0 and report if that works here https://bugzilla.kernel.org/show_bug.cgi?id=205047
(application/x-gzip)    e1000e-dkms-3.6.0-1.src.tar.gz (1 KiB)
Comment by Peter Sulyok (peter_s) - Thursday, 10 October 2019, 19:45 GMT
Some test results:
- e1000e-dkms-5.3.1-1.src.tar.gz WORKS for me (I installed that on kernel 5.3.5-arch1-1-ARCH)
- e1000e-dkms-3.6.0-1.src.tar.gz does not work at all, same issue, link is down

Thanks for the fixes.
Comment by loqs (loqs) - Thursday, 10 October 2019, 22:03 GMT
New version to test using patch from http://patchwork.ozlabs.org/patch/1172931/
Needed to cherry-pick ee2e80c194628a0ba5cd0a5c37b46fa136e11004 from 5.4 for the patch to apply.
(application/x-gzip)    e1000e-dkms-5.3.1-2.src.tar.gz (1.2 KiB)
Comment by Peter Sulyok (peter_s) - Friday, 11 October 2019, 16:33 GMT
It works on kernel 5.3.5-arch1-1-ARCH.
Comment by Robin KERDILES (Dr-Shadow) - Friday, 11 October 2019, 16:39 GMT
What works on kernel 5.3.5 ? the bundled e1000e driver or a dkms package quoted above ?
Comment by loqs (loqs) - Friday, 11 October 2019, 16:40 GMT
@peter_s can you please report that to the upstream bug report https://bugzilla.kernel.org/show_bug.cgi?id=205047 including the output of dmesg.
Comment by Peter Sulyok (peter_s) - Friday, 11 October 2019, 17:58 GMT
Oops, something went wrong with the installation of e1000e-dkms-5.3.1-2. It is not working indeed as it stated here https://bugzilla.kernel.org/show_bug.cgi?id=205047
The latest working patch is e1000e-dkms-5.3.1-1 for me on kernel 5.3.5-arch1-1-ARCH.
Comment by Robin KERDILES (Dr-Shadow) - Friday, 11 October 2019, 18:02 GMT
Thank you for your feedback Peter. I won't test this dkms package but I will follow this thread and the other one on bugzilla.
If needed I can test newer dkms packages.
Comment by loqs (loqs) - Friday, 11 October 2019, 18:24 GMT
What was the build error for e1000e-dkms-5.3.1-2?
Comment by Peter Sulyok (peter_s) - Friday, 11 October 2019, 19:11 GMT
It was built properly, just the installation failed because of the multiple kernels (I have both linux and linux-lts installed).
Comment by loqs (loqs) - Sunday, 03 November 2019, 21:11 GMT
New version to test using the suggestion from https://bugzilla.kernel.org/show_bug.cgi?id=205047#c24
(application/x-gzip)    e1000e-dkms-5.3.8-1.src.tar.gz (1.5 KiB)
Comment by Peter Sulyok (peter_s) - Sunday, 03 November 2019, 22:25 GMT
Not works for me :(

Nov 03 23:09:54 gianta kernel: e1000e: loading out-of-tree module taints kernel.
Nov 03 23:09:54 gianta kernel: e1000e: module verification failed: signature and/or required key missing - tainting kernel
Nov 03 23:09:54 gianta kernel: e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-l
Nov 03 23:09:54 gianta kernel: e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
Nov 03 23:09:54 gianta kernel: e1000e 0000:00:1f.6: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
Nov 03 23:09:54 gianta kernel: e1000e 0000:00:1f.6 0000:00:1f.6 (uninitialized): registered PHC clock
Nov 03 23:09:54 gianta kernel: e1000e 0000:00:1f.6 eth0: (PCI Express:2.5GT/s:Width x1) 70:85:c2:a4:d0:16
Nov 03 23:09:54 gianta kernel: e1000e 0000:00:1f.6 eth0: Intel(R) PRO/1000 Network Connection
Nov 03 23:09:54 gianta kernel: e1000e 0000:00:1f.6 eth0: MAC: 13, PHY: 12, PBA No: FFFFFF-0FF
Nov 03 23:09:54 gianta kernel: e1000e 0000:00:1f.6 eno1: renamed from eth0
Nov 03 23:10:01 gianta kernel: e1000e: eno1 NIC Link is Down

Thanks for the update.
Comment by loqs (loqs) - Sunday, 03 November 2019, 22:33 GMT
Please post on the kernel bug report that it failed.
Comment by loqs (loqs) - Tuesday, 19 November 2019, 17:50 GMT
New version to test using the suggestion from https://bugzilla.kernel.org/show_bug.cgi?id=205047#c28
This reverts https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=ee2e80c194628a0ba5cd0a5c37b46fa136e11004
(application/x-gzip)    e1000e-dkms-5.3.11-1.src.tar.... (1.1 KiB)
Comment by Peter Sulyok (peter_s) - Tuesday, 19 November 2019, 22:01 GMT
Unfortunately is not working either. Will be reported on https://bugzilla.kernel.org/show_bug.cgi?id=205047#c28 as well.

[ 4.195139] e1000e: loading out-of-tree module taints kernel.
[ 4.215400] e1000e: module verification failed: signature and/or required key missing - tainting kernel
[ 4.218393] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-l
[ 4.218394] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[ 4.218581] e1000e 0000:00:1f.6: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
[ 4.475106] e1000e 0000:00:1f.6 0000:00:1f.6 (uninitialized): registered PHC clock
[ 4.543593] e1000e 0000:00:1f.6 eth0: (PCI Express:2.5GT/s:Width x1) 70:85:c2:a4:d0:16
[ 4.543596] e1000e 0000:00:1f.6 eth0: Intel(R) PRO/1000 Network Connection
[ 4.543677] e1000e 0000:00:1f.6 eth0: MAC: 13, PHY: 12, PBA No: FFFFFF-0FF
[ 4.604465] e1000e 0000:00:1f.6 eno1: renamed from eth0
[ 11.407189] e1000e: eno1 NIC Link is Down
[ 12.435905] e1000e: eno1 NIC Link is Up 1000 Mbps Half Duplex, Flow Control: None
Comment by loqs (loqs) - Wednesday, 20 November 2019, 11:09 GMT
I reverted the wrong commit in the last test.
How I turned 59653e6497d16f7ac1d9db088f3959f57ee8c3db into ee2e80c194628a0ba5cd0a5c37b46fa136e11004 I have no idea.
Updated version with the correct commit reverted.
(application/x-gzip)    e1000e-dkms-5.3.11-2.src.tar.... (1.1 KiB)
Comment by Peter Sulyok (peter_s) - Wednesday, 20 November 2019, 19:05 GMT
Finally it is working :)
Thanks for the support.

[ 4.156613] e1000e: loading out-of-tree module taints kernel.
[ 4.162353] e1000e: module verification failed: signature and/or required key missing - tainting kernel
[ 4.196559] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-l
[ 4.196560] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[ 4.196767] e1000e 0000:00:1f.6: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
[ 4.437055] e1000e 0000:00:1f.6 0000:00:1f.6 (uninitialized): registered PHC clock
[ 4.503870] e1000e 0000:00:1f.6 eth0: (PCI Express:2.5GT/s:Width x1) 70:85:c2:a4:d0:16
[ 4.503875] e1000e 0000:00:1f.6 eth0: Intel(R) PRO/1000 Network Connection
[ 4.503956] e1000e 0000:00:1f.6 eth0: MAC: 13, PHY: 12, PBA No: FFFFFF-0FF
[ 4.584850] e1000e 0000:00:1f.6 eno1: renamed from eth0
[ 11.243194] e1000e: eno1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None

I will report that on https://bugzilla.kernel.org/show_bug.cgi?id=205047#c28 too.
Comment by Peter Sulyok (peter_s) - Wednesday, 27 November 2019, 22:32 GMT
Are you planning to integrate the this fix into kernel 5.3 or it will be only fixed in 5.4?
Comment by loqs (loqs) - Wednesday, 27 November 2019, 23:59 GMT
As 5.4.arch1-1 is in testing I doubt there will be any more 5.3 releases.
Could you test the 5.4 release. 59653e6497d16f7ac1d9db088f3959f57ee8c3db does not appear to have been reverted.
Comment by Peter Sulyok (peter_s) - Thursday, 28 November 2019, 15:00 GMT
I just tested kernel "testing/linux 5.4-arch-1" and e1000e driver is NOT working indeed.
You were right.
Comment by Peter Sulyok (peter_s) - Monday, 02 December 2019, 14:37 GMT
5.4.1-arch1-1 tested and it is not working either.
Comment by loqs (loqs) - Monday, 02 December 2019, 17:29 GMT
59653e6497d16f7ac1d9db088f3959f57ee8c3db does not revert cleanly on 5.4.
Included patch is my attempt to revert the commit on 5.4. This version will not work with 5.3.
Does the older 5.3 based version work with 5.4?
(application/x-gzip)    e1000e-dkms-5.4.1-1.src.tar.gz (2.7 KiB)
Comment by Peter Sulyok (peter_s) - Tuesday, 03 December 2019, 18:16 GMT
The dkms driver you provided (e1000e-dkms-5.4.1-1.src.tar.gz) is working fine on kernel 5.4.1-arch1-1. Thanks.
Maybe we can ping the thread on kernel.org about their plans.
Comment by Robin KERDILES (Dr-Shadow) - Wednesday, 11 December 2019, 13:53 GMT
This issue doesn't affect my setup anymore since some specific kernel 5.3 to 5.4.2
I think this could be related to some kind of firmware bug because I did a lot of changes since then.
(Bios / Intel ME update with bios patch to disable intel ME, intel microcode update and UEFI secure boot enable)
I don't use WLAN/bluetooth at all on my motherboard.
Comment by Richard Tippl (tippl) - Saturday, 04 January 2020, 22:37 GMT
This issue still affects my setup. I am using DKMS package found on the forums which reverts the def4ec6dce393e2136b62a05712f35a7fa5f5e56 commit.

Removing the package on my system (kernel 5.4.7) results in exactly the same issue i have had since the beginning. The driver starts to connect to wired network, fails and this continues in a loop for some time till it finally connects (sometimes it does not and i have to reboot).

Observed on 2 of my computers. How can the network driver be this broken for the past 3 months.
Comment by Peter Sulyok (peter_s) - Saturday, 04 January 2020, 22:49 GMT
I have the same situation here and I think that the bug has not been fixed yet.
Maybe in 5.5?
Comment by Peter Sulyok (peter_s) - Saturday, 04 January 2020, 23:02 GMT
Just did a ping here https://bugzilla.kernel.org/show_bug.cgi?id=205047 as well.
Let us see the reactions.
Comment by Peter Sulyok (peter_s) - Tuesday, 07 January 2020, 21:13 GMT
Hi logs, can you please check if the latest suggested patch here

https://bugzilla.kernel.org/show_bug.cgi?id=205047#c49

is the same you created here in the latest dkms package? Thanks.
Comment by loqs (loqs) - Tuesday, 07 January 2020, 21:31 GMT
Yes it is the same. Both revert 59653e6497d16f7ac1d9db088f3959f57ee8c3db.
Comment by Richard Tippl (tippl) - Saturday, 25 January 2020, 00:20 GMT
Kernel 5.4.14 includes patch reverting 59653e6497d16f7ac1d9db088f3959f57ee8c3db (Revert "e1000e: Make watchdog use delayed work").

I can confirm this fixes the issue on my machine. Can anyone else confirm the issue being fixed?
Comment by Ariel Rosenfeld (lord_rel) - Monday, 27 January 2020, 14:05 GMT
can confirm its fixed for me with kernel 5.4.14-arch1-1 on an Asus Z170 Pro gaming
Comment by Peter Sulyok (peter_s) - Tuesday, 28 January 2020, 16:17 GMT
Hi logs, it is working now on kernel 5.4.15 (without any additional dkms modul).
What happened? Is the fix coming from the kernel directly or it was local fix in ArchLinux?
Thanks for everybody's support here.
Comment by loqs (loqs) - Tuesday, 28 January 2020, 16:52 GMT
Arch applied the commit locally for 5.4.14-arch1 [1] and v5.4.15-arch1 [2]
For 5.5 onward the patch is applied upstream.

[1] https://git.archlinux.org/linux.git/log/?h=v5.4.14-arch1
[2] https://git.archlinux.org/linux.git/log/?h=v5.4.15-arch1

Loading...