FS#79633 - [btrfs-progs] 6.5-1 broken on certain older CPUs

Attached to Project: Arch Linux
Opened by Alexander (alexander.k) - Sunday, 10 September 2023, 11:09 GMT
Last edited by Toolybird (Toolybird) - Thursday, 14 September 2023, 22:19 GMT
Task Type Bug Report
Category Upstream Bugs
Status Closed
Assigned To Tobias Powalowski (tpowa)
Sébastien Luttringer (seblu)
Architecture x86_64
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description:
Illegal instruction: Upgrading from btrfs-progs-6.3.3 to btrfs-progs-6.5 broke "send/recieve" and "check" functionality

Additional info:
* package version(s)
- btrfs-progs-6.3.3-1-x86_64
- btrfs-progs-6.5-1-x86_64

* link to upstream bug report, if any
- https://github.com/kdave/btrfs-progs/issues/676

Steps to reproduce:

btrfs check /dev/sda1

Opening filesystem to check...
Illegal instruction

Journal:

Sep 10 13:20:24 pc kernel: traps: btrfs[1491] trap invalid opcode ip:55a7d4989eb0 sp:7ffd40934f60 error:0 in btrfs[55a7d48b5000+d6000]
Sep 10 13:20:24 pc systemd[1]: Created slice Slice /system/systemd-coredump.
Sep 10 13:20:24 pc systemd[1]: Started Process Core Dump (PID 1492/UID 0).
Sep 10 13:20:24 pc systemd-coredump[1493]: Resource limits disable core dumping for process 1491 (btrfs).
Sep 10 13:20:24 pc systemd-coredump[1493]: [🡕] Process 1491 (btrfs) of user 0 dumped core.
Sep 10 13:20:24 pc sudo[1489]: pam_unix(sudo:session): session closed for user root
Sep 10 13:20:24 pc systemd[1]: systemd-coredump@0-1492-0.service: Deactivated successfully.

uname -a

Linux pc 6.1.52-1-lts #1 SMP PREEMPT_DYNAMIC Thu, 07 Sep 2023 05:17:41 +0000 x86_64 GNU/Linux
Arch Linux

PS: Downgrading back to the v6.3.3 fixes the problem


This task depends upon

Closed by  Toolybird (Toolybird)
Thursday, 14 September 2023, 22:19 GMT
Reason for closing:  Fixed
Additional comments about closing:  btrfs-progs 6.5.1-1
Comment by Toolybird (Toolybird) - Sunday, 10 September 2023, 21:40 GMT
> trap invalid opcode

Interesting...I wonder where the invalid x86 CPU instruction is coming from? Which CPU is in this system? There are no other reports about this. Could you please post a backtrace that includes debugging symbols [1]? It's usually as simple as this (must have gdb installed):

$ coredumpctl gdb (then answer y when it asks "Enable debuginfod for this session?")
(gdb) set logging enabled
(gdb) bt (or bt full)

Then post gdb.txt

[1] https://wiki.archlinux.org/title/Debugging/Getting_traces
Comment by Alexander (alexander.k) - Monday, 11 September 2023, 08:50 GMT
Unfortunately I am getting error: Coredump entry has no core attached (neither internally in the journal nor externally on disk).

My CPU is Intel i7 860 [ https://ark.intel.com/content/www/us/en/ark/products/41316/intel-core-i7860-processor-8m-cache-2-80-ghz.html ]

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 36 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Vendor ID: GenuineIntel
Model name: Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz
CPU family: 6
Model: 30
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1
Stepping: 5
BogoMIPS: 8029.78
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl
xtopology nonstop_tsc cpuid aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ep
t vpid dtherm flush_l1d


---

└─⮞ sudo btrfs fi show /dev/sda1
Illegal instruction

└─⮞ sudo coredumpctl gdb
PID: 9578 (btrfs)
UID: 0 (root)
GID: 0 (root)
Signal: 4 (ILL)
Timestamp: Mon 2023-09-11 11:33:06 MSK (4s ago)
Command Line: btrfs fi show /dev/sda1
Executable: /usr/bin/btrfs
Control Group: /user.slice/user-1000.slice/user@1000.service/app.slice/app-org.kde.konsole-15a5159788464f5d9f56b7680d213a31.scope
Unit: user@1000.service
User Unit: app-org.kde.konsole-15a5159788464f5d9f56b7680d213a31.scope
Slice: user-1000.slice
Owner UID: 1000 (user)
Boot ID: 64f73ab27c21490582937e5bf367a9f4
Machine ID: ab52ac6061004474bc26b74cc6c41065
Hostname: pc
Storage: none
Message: Process 9578 (btrfs) of user 0 dumped core.

Coredump entry has no core attached (neither internally in the journal nor externally on disk).
Comment by Alexander (alexander.k) - Monday, 11 September 2023, 14:01 GMT
"btrfs recieve" produces following gdb file

Core was generated by `btrfs receive -q /run/media/sdb2/backup/2023-09-11--16:53:14'.
Program terminated with signal SIGILL, Illegal instruction.
#0 0x000055a6c4dd8eb0 in crc_pcl ()
(gdb) set logging enabled
Copying output to gdb.txt.
Copying debug output to gdb.txt.
(gdb) bt full
#0 0x000055a6c4dd8eb0 in crc_pcl ()
No symbol table info available.
#1 0x0000000000001040 in ?? ()
No symbol table info available.
#2 0x000055a6c506d9b0 in ?? ()
No symbol table info available.
#3 0x000000000000000f in ?? ()
No symbol table info available.
#4 0x000055a6c4d56ed4 in btrfs_read_and_process_send_stream ()
No symbol table info available.
#5 0x000055a6c4d7e539 in ?? ()
No symbol table info available.
#6 0x000055a6c4d04179 in main ()
No symbol table info available.
(gdb) quit
   gdb.txt (0.7 KiB)
Comment by Alexander (alexander.k) - Monday, 11 September 2023, 15:08 GMT
duplicate deleted
   gdb.txt (0.7 KiB)
Comment by Toolybird (Toolybird) - Tuesday, 12 September 2023, 02:09 GMT
In the upstream ticket, it seems that a bug has been acknowledged (thanks also to some excellent input from @loqs). With your CPU, I would just stay on btrfs-progs 6.3.3 until 6.5.1 is released (and lands in the repos).
Comment by Alexander (alexander.k) - Tuesday, 12 September 2023, 13:17 GMT
I'll do that, thanks
Comment by loqs (loqs) - Wednesday, 13 September 2023, 11:56 GMT
Please try the attached btrfs-progs that has the proposed fixed [1] from [2] applied as well as the fix for  FS#79654 .

[1] https://github.com/kdave/btrfs-progs/commit/03f41ac508d2c47fdfef08f0fd27ab154f7dcfd8
[2] https://github.com/kdave/btrfs-progs/issues/676
Comment by Alexander (alexander.k) - Wednesday, 13 September 2023, 12:45 GMT
Works great!
Comment by Tobias Powalowski (tpowa) - Wednesday, 13 September 2023, 14:06 GMT
Thanks guys for investigating. 6.5-2 should work now.
Comment by Leonidas Spyropoulos (inglor) - Thursday, 14 September 2023, 06:14 GMT
official 6.5.1 is out.
Comment by Tobias Powalowski (tpowa) - Thursday, 14 September 2023, 07:39 GMT
6.5.1 is in [testing] now.
Comment by Alexander (alexander.k) - Thursday, 14 September 2023, 15:36 GMT
Official 6.5.1 [testing]: My CPU test passed as well

Loading...