Arch Linux

Please read this before reporting a bug:
https://wiki.archlinux.org/title/Bug_reporting_guidelines

Do NOT report bugs when a package is just outdated, or it is in the AUR. Use the 'flag out of date' link on the package page, or the Mailing List.

REPEAT: Do NOT report bugs for outdated packages!
Tasklist

FS#43704 - [linux-lts] 3.14.31-1 breaks btrfs

Attached to Project: Arch Linux
Opened by Jan (DosAmp) - Thursday, 05 February 2015, 09:53 GMT
Last edited by Andreas Radke (AndyRTR) - Saturday, 14 February 2015, 21:11 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Andreas Radke (AndyRTR)
Architecture All
Severity Critical
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 7
Private No

Details

After updating to linux-lts 3.14.31-1, initcpio (both default and fallback, confirmed to contain btrfs.ko) fails to mount a btrfs-formatted root filesystem:

Booting the kernel.
starting version 218
mount: unknown filesystem type 'btrfs'
...
[rootfs /]# modprobe btrfs
modprobe: ERROR: could not insert 'btrfs': Unknown symbol in module, or unknown parameter (see dmesg)

dmesg doesn't contain any information relevant to btrfs at this point. Downgrading to linux-lts-3.14.30-1 corrects the issue.
This task depends upon

Closed by  Andreas Radke (AndyRTR)
Saturday, 14 February 2015, 21:11 GMT
Reason for closing:  Fixed
Additional comments about closing:  3.14.33-1
Comment by fax (fax) - Thursday, 05 February 2015, 21:16 GMT
I believe this is a more general issue with the current version of the LTS kernel. I am getting this upon boot:

kernel: Linux version 3.14.31-1-lts (builduser@andyrtr) (gcc version 4.9.2 20141224 (prerelease) (GCC) ) #1 SMP Fri Jan 30 15:1
...
systemd[1]: Unit mnt-backup.mount entered failed state.
mount[450]: mount: unknown filesystem type 'xfs'
mount[535]: mount: unknown filesystem type 'xfs'
systemd[1]: var-lib-backuppc.mount mount process exited, code=exited status=32
systemd[1]: Failed to mount /var/lib/backuppc.

When I comment out the xfs filesystems in /etc/fstab, the boot proceeds but fails to bring up X11:

kernel: nvidia: module license 'NVIDIA' taints kernel.
kernel: Disabling lock debugging due to kernel taint
kernel: nvidia: disagrees about version of symbol node_data
kernel: nvidia: Unknown symbol node_data (err -22)
Comment by Andreas Radke (AndyRTR) - Friday, 06 February 2015, 05:53 GMT
Sounds like an upstream regression. There are some "mm" related commits in https://www.kernel.org/pub/linux/kernel/v3.0/ChangeLog-3.14.31 / http://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/log/?h=linux-3.14.y and there's one mentioning several file systems:

http://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?h=linux-3.14.y&id=35dbe179fe2af754b0ac92c629435c39bd95681c

Build a kernel with this commit reverted (patch -R) and check if it boots again. Look also out for upstream regression bugs in the kernel tracker.
No problem here for me with ext4.
Comment by Claire Farron (clfarron4) - Friday, 06 February 2015, 10:49 GMT
Hi, I'm going to follow this as well, because someone complained about not being to access xfs volumes with the latest 3.10.x kernel.

Yes, I know it's not supported officially, but I'd still like to follow it.
Comment by Andreas Radke (AndyRTR) - Friday, 06 February 2015, 17:02 GMT
I suggest someone affected to git bisect your problem. I'm going to build the new 3.14.32 release later tonight but from the changelog there's nothing related to this.
Comment by James Bunton (delx) - Sunday, 08 February 2015, 20:29 GMT
I haven't had time to git-bisect this, but I did find a work-around to load xfs or btrfs for me.
# modprobe crc32c
# modprobe xfs

I'm using ext4 to boot, so this is good enough for me. But if you need this to work as a root fs then you should be able to add this to MODULES in /etc/mkinitcpio.conf.

I hope this helps track down the problem.
Comment by fax (fax) - Monday, 09 February 2015, 21:41 GMT
I just tried but "modprobe xfs" in the rescue console fails for me, saying something along the lines of "Unknown symbol in module, or unknown parameter (see dmesg)". The same goes for "modprobe libcrc32c", which seems to be a dependency of xfs. I couldn't find any log entries relating to that modprobe with journalctl and dmesg.

I also tried adding nvidia and xfs to the MODULES section in mkinitcpio.conf and regenerated initramfs with "pacman -S linux-lts" but that didn't help either.
Comment by Philip Müller (philm) - Monday, 09 February 2015, 22:51 GMT
We found a solution upstream. To quote Mathias Krause:

v3.14.31:crypto/crc32c.c is missing the MODULE_ALIAS_CRYPTO("crc32c"). That's probably because crypto/crc32c.c was renamed to crypto/crc32c_generic.c in commit 06e5a1f29819759392239669beb2cad27059c8ec and therefore fell through the cracks when backporting commit 5d26a105b5a73e5635eae0629b42fa0a90e07b7b.

So the affected kernels (all that backported the "crypto-" prefix patches) need this additional patch.
Comment by Philip Müller (philm) - Monday, 09 February 2015, 22:52 GMT
@fax: a quick workaround is to add crc32c to your MODULES line.
Comment by Jan (DosAmp) - Tuesday, 10 February 2015, 09:24 GMT
I can confirm adding crc32c to MODULES is a workaround for booting from btrfs for 3.14.31 and 3.14.32 as well.
Comment by Andreas Radke (AndyRTR) - Tuesday, 10 February 2015, 16:16 GMT
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?h=linux-3.14.y&id=31c8446aacd8595a9de61610b2625bcf84e1b7aa

Looks like that commit includes your suggested patch - so I assume this is no solution.

diff --git a/crypto/crc32.c b/crypto/crc32.c
index 9d1c415..187ded2 100644
--- a/crypto/crc32.c
+++ b/crypto/crc32.c
@@ -156,3 +156,4 @@ module_exit(crc32_mod_fini);
MODULE_AUTHOR("Alexander Boyko <alexander_boyko@xyratex.com>");
MODULE_DESCRIPTION("CRC32 calculations wrapper for lib/crc32");
MODULE_LICENSE("GPL");
+MODULE_ALIAS_CRYPTO("crc32");

Isn't that all also part of stable recent kernel 3.18/19 ? Why do they keep booting? Do we have an issue with out mkinitcpio?
Comment by fax (fax) - Tuesday, 10 February 2015, 22:19 GMT
Well, I can confirm that the LTS kernel boots for me with MODULES="crc32c xfs" in mkinitcpio.conf. Also the nvidia module loads fine now but I guess that's because of the nvidia-lts package update that came in today. FWIW, my mkinitcpio configuration looks like this:

$ cat /etc/mkinitcpio.d/linux-lts.preset
...
ALL_config="/etc/mkinitcpio-lts.conf"
ALL_kver="/boot/vmlinuz-linux-lts"
...

$ cat /etc/mkinitcpio-lts.conf
...
MODULES="crc32c xfs"
...

$ cat /etc/mkinitcpio.conf
...
MODULES=""
...

I had some very odd issues booting both the LTS kernel and the regular kernel at first. lvm was failing to find some logical volumes:

lvmetad[375]: Cannot lock lockfile [/run/lvmetad.pid], error was [Resource temporarily unavailable]
systemd[1]: Job dev-mapper-backup2\x2dbackup.device/start timed out.
systemd[1]: Timed out waiting for device dev-mapper-backup2\x2dbackup.device.
... and a few more like these ...

It worked fine for other volumes however and the issue went away after a few reboots. And the volumes had different filesystems, ext4 and xfs. Maybe there was some LVM lockfile lingering that finally got removed after a successful boot?
Comment by Philip Müller (philm) - Wednesday, 11 February 2015, 17:07 GMT Comment by Andreas Radke (AndyRTR) - Friday, 13 February 2015, 17:27 GMT
new kernel is in testing. give it a try.
Comment by Jan (DosAmp) - Saturday, 14 February 2015, 16:45 GMT
Both XFS and btrfs look fine to me: https://bpaste.net/show/f585dd5ea416

Loading...