Arch Linux

Please read this before reporting a bug:
https://wiki.archlinux.org/title/Bug_reporting_guidelines

Do NOT report bugs when a package is just outdated, or it is in the AUR. Use the 'flag out of date' link on the package page, or the Mailing List.

REPEAT: Do NOT report bugs for outdated packages!
Tasklist

FS#75685 - Unable to load modules built with DKMS with kernel 5.19.3-arch1-1 and gcc 12.2.0-1

Attached to Project: Arch Linux
Opened by Matt M. (9numbernine9) - Wednesday, 24 August 2022, 12:22 GMT
Last edited by Jan Alexander Steffens (heftig) - Wednesday, 24 August 2022, 12:43 GMT
Task Type Bug Report
Category Kernel
Status Closed
Assigned To Tobias Powalowski (tpowa)
Jan Alexander Steffens (heftig)
David Runge (dvzrv)
freswa (frederik)
Architecture x86_64
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Description:

Greetings! Firstly, I apologize in advance if this is not strictly a kernel bug - I'm unsure if this is a kernel issue, a GCC issue, or an Arch packaging issue.

Since upgrading to kernel 5.19.3-arch1-1 and GCC 12.2.0-1, I have been unable to load _any_ modules that are built using DKMS. As an example, I have built the xone_gip module but attempting to modprobe/insmod the module returns the following:

modprobe: ERROR: could not insert 'xone_gip': Operation not permitted

If I look in dmesg, I can see the following whenever I try to load the module:

[ 47.372403] BPF: [112255] Invalid kind:19
[ 47.372405] failed to validate module [xone_gip] BTF: -22

After some digging, it seems like support for BTF/BPF is something that was added to GCC in the 12.x series. For fun I tried dumping the symbols from one of the compiled modules using objdump -x xone-gip.ko (after decompressing it) and noticed the following section:

30 .BTF 0000dc6e 0000000000000000 0000000000000000 00005e4c 2**0

So it seems to me that now GCC is adding that .BTF section into the compiled modules, and the kernel doesn't like it. Stripping out all the BTF sections with this shell script seems to allow the modules to load successfully:

cd /usr/lib/modules/$(uname -r)/updates/dkms
for file in *.zst
do
cp "$file" "$file.bak"
unzstd -f "$file"
base=$(basename "$file" .zst)
objcopy --remove-section .BTF "$base"
zstd -f "$base"
rm -f "$base"
done

Once stripped thusly, loading the modules works fine.

I have tested this with the following DKMS modules:
- xone_gip
- hid-xpadneo
- openrazer

In all scenarios, the behaviour is the same: without running the stripping script above, trying to load the module only yields the "Invalid kind" and "BTF" errors.

Additional info:
* package version(s)
linux-5.19.3-arch1-1, gcc-12.2.0-1
* config and/or log files etc.
* link to upstream bug report, if any

Steps to reproduce:
- Install linux-5.19.3-arch1-1 and gcc-12.2.0-1
- Reboot
- Build a module using DKMS
- Attempt to modprobe it and receive error similar to "modprobe: ERROR: could not insert 'hid_xpadneo': Invalid argument"
- Observe "Invalid kind:19" and "BTF: -22" error messages in dmesg output

Related discussions:
- https://old.reddit.com/r/archlinux/comments/ww9fa2/problems_with_dkms_modules_and_kernel_519x_gcc_122/ (I am the original author of this post)
- https://bbs.archlinux.org/viewtopic.php?pid=2053487
This task depends upon

Closed by  Jan Alexander Steffens (heftig)
Wednesday, 24 August 2022, 12:43 GMT
Reason for closing:  Fixed
Additional comments about closing:  pahole 1:1.23-1
Comment by Jan Alexander Steffens (heftig) - Wednesday, 24 August 2022, 12:34 GMT
Does downgrading pahole to 1.23-1 work?
Comment by Doug Newgard (Scimmia) - Wednesday, 24 August 2022, 12:34 GMT
The full error that we've been seeing:

[ 4.421228] BPF: [112814] ENUM rpm_status
[ 4.421944] BPF: size=4 vlen=5
[ 4.422640] BPF:
[ 4.423317] BPF: Invalid btf_info kind_flag
[ 4.424014] BPF:
[ 4.424689] failed to validate module [nvidia] BTF: -22

Had one user where it was happening on one system but not the other, both fully up to date. Haven't been able to find a common thread on the affected systems that could be causing this.
Comment by Matt M. (9numbernine9) - Wednesday, 24 August 2022, 12:40 GMT
@heftig No, downgrading doesn't seem to help (I admit I might be doing something wrong however). Downgraded pahole, dkms uninstall a module, dkms install the same module, try modprobing the new module - no change.


EDIT - My bad. Didn't clean things correctly. Downgrading pahole works fine!
Comment by Jan Alexander Steffens (heftig) - Wednesday, 24 August 2022, 12:41 GMT
Can reproduce. Fixed by downgrading pahole.
Comment by Jan Alexander Steffens (heftig) - Wednesday, 24 August 2022, 12:42 GMT
I've downgraded pahole then reinstalled the module package I was testing (acpi_call-dkms).

Loading...