FS#49380 - [mkinitcpio] cannot boot: missing crc32 driver

Attached to Project: Arch Linux
Opened by Alois Nespor (anespor) - Wednesday, 18 May 2016, 13:47 GMT
Last edited by Dave Reisner (falconindy) - Saturday, 11 March 2017, 18:16 GMT
Task Type Bug Report
Category Arch Projects
Status Closed
Assigned To Thomas Bächler (brain0)
Evangelos Foutras (foutrelis)
Dave Reisner (falconindy)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 7
Private No

Details

Description:
after upgrade to ZEN kernel 4.6 from [testing], i cannot boot up, because i become error:

F2FS fs: cannot load 'crc32' drivers


Steps to reproduce:

upgrade to kernel linux-zen-4.6-1 und use F2FS (f2fs-tools 1.6.1-1) as filesystem on root (sda2)


https://bbs.archlinux.org/viewtopic.php?id=210673
This task depends upon

Closed by  Dave Reisner (falconindy)
Saturday, 11 March 2017, 18:16 GMT
Reason for closing:  Fixed
Additional comments about closing:  https://git.archlinux.org/mkinitcpio.git /commit/?id=5249350c80a6158ebcf278e780f4 2c6abc3643f1
Comment by Samantha McVey (samcv) - Thursday, 09 June 2016, 19:27 GMT
 FS#49636  is a duplicate of this bug
Comment by Thomas Kowaliczek (LinuxDonald) - Saturday, 11 June 2016, 11:46 GMT
I have the same problem
Comment by Samantha McVey (samcv) - Sunday, 26 June 2016, 08:27 GMT
Can somebody post their /etc/mkinitcpio.conf file? Reading the thread it looks like people have seen this on 4.6-zen and mainline (4.6rc1).
Comment by Alois Nespor (anespor) - Sunday, 26 June 2016, 08:56 GMT
from Archwiki (F2FS):
The driver included in the kernel 4.6 or higher uses Crypto API for computing CRC32. Add crc32_generic and crc32-pclmul to the MODULES array in /etc/mkinitpcio.conf and regenerate the initramfs if the driver fails to load a CRC32 module at boot.


Edit: crc32_generic and crc32-pclmul in the MODULES array be enough if is only standard partitions used.
Comment by Samantha McVey (samcv) - Sunday, 26 June 2016, 08:59 GMT
Yes I read that in the thread. This is my hooks:
HOOKS="base udev resume autodetect modconf block encrypt lvm2 filesystems keyboard fsck"
I think lvm2 may depend on crc32 which is why I didn't see this problem. Encrypt also may include the crc32 crypto automatically. My setup is LUKS > LVM > F2FS.
Comment by Dave Reisner (falconindy) - Sunday, 26 June 2016, 13:19 GMT
lvm2 does not depend on crc32c.

$ modtree -nl dm-mod dm-snapshot dm-mirror dm-cache dm-cache-mq dm-thin-pool | awk '!a[$1]++ { print $1 }'
dm_mod
dm_snapshot
dm_bufio
dm_mirror
dm_log
dm_region_hash
dm_cache
libcrc32c
dm_bio_prison
dm_persistent_data
dm_cache_smq
dm_thin_pool

(modtree is https://github.com/falconindy/modtree)

libcrc32c has a softdep of crc32c, but mkinitcpio does no parsing of softdeps. That's probably what needs fixing here.

edit: erm, and we probably also need to add an f2fs quirk for libcrc32c.
Comment by Dave Reisner (falconindy) - Saturday, 02 July 2016, 14:54 GMT
This is fixed pending a release of mkinitcpio:

https://git.archlinux.org/mkinitcpio.git/commit/?id=08762d8401ff8e01a3c7a1558e69d5c30c983d20

(along with improved softdep handling)
Comment by Zachary Cook (Zeik) - Friday, 11 November 2016, 03:12 GMT
  • Field changed: Percent Complete (100% → 0%)
crc32 is needed, not crc32c

Still cannot boot without crc32-pclmul or crc32_generic, getting the same cannot load crc32 driver error.

mkinitcpio version 21-1

See dracut commit that determined the same thing:
https://github.com/dracutdevs/dracut/commit/34a42f9f49d4ad8897c4890af5b9a455e1335c66

Seems like everyone that boots on f2fs must still have the modules included explicitly and so haven't noticed that the fix was wrong?
Comment by Dave Reisner (falconindy) - Friday, 11 November 2016, 11:55 GMT
Please file a bug upstream -- "depends" in modules is a real thing. f2fs should depend on crc32 instead of making distros guess at the requirements.
Comment by Zachary Cook (Zeik) - Saturday, 12 November 2016, 00:03 GMT
Okay, got it working by adding MODULE_SOFTDEP("pre: crypto-crc32"); to the end of fs/f2fs/super.c and recompiling the module, now to get that upstream.

There was a similar patch here: http://www.spinics.net/lists/linux-fsdevel/msg98073.html but it seems to have never went upstream, I wonder why? This patch suggests crc32c is used for something too (fs encryption?).

I also tested reverting the quirk (https://git.archlinux.org/mkinitcpio.git/commit/?id=08762d8401ff8e01a3c7a1558e69d5c30c983d20) that was added in mkinitcpio 20-1 for f2fs and it boots just fine without the libcrc32c modules, although these might be used by fs encryption and so can stay, or maybe should be turned into a softdep too? (I guess all of those quirks should really be softdeps in upstream right?)
Comment by Zachary Cook (Zeik) - Saturday, 12 November 2016, 00:40 GMT
reported upstream in kernel bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=187471
Comment by Zachary Cook (Zeik) - Monday, 28 November 2016, 05:28 GMT
  • Field changed: Percent Complete (100% → 0%)
Fix doesn't work here, sorry, it needs to be changed to
add_module "crypto-crc32"

I tested that and made sure, so hopefully this time the bug will stay dead.

Edit: removed ? from end (and tested again!), as crypto-crc32 aliases both modules according to modinfo, and the wildcard wasn't working it seems.
Comment by Dave Reisner (falconindy) - Monday, 28 November 2016, 13:04 GMT
That makes no sense to me. A few things:

- The ? is not a wildcard -- it's an indication to mkinitcpio that the module_add call shouldn't fail if the module doesn't exist (e.g. because it's compiled in, rather than being a module)
- The crc32 and crypto-crc32 aliases resolve to the *same* *modules*. Why does one work for you but the other doesn't?
- dracut continues to use crc32 and not crypto-crc32. Why is it good enough for dracut but not for mkinitcpio? They use very similar alias resolution techniques (kmod).

Please explain more about why this "doesn't work". I don't buy your argument that this "needs* to be crypto-crc32.
Comment by Zachary Cook (Zeik) - Monday, 28 November 2016, 18:12 GMT
Ah, that makes sense, I was just guessing at how the ? worked.

I have no clue why crc32? Doesn't work, just that the resulting img doesn't contain the crc32_generic and crc32-pclmul modules when it is used, but does if crypto-crc32? is used. Is there an easy way to run this through a debugger to see what is going on? It doesn't make sense to me either.
Comment by Zachary Cook (Zeik) - Monday, 28 November 2016, 18:43 GMT
I used set -x and set +x to get some info on what is going on in the script, looks like when it gets to crc32 it skips because it is already included? I do see crc32-pclmul in the autodetect module list, but I dont see it being added.

+ firmware=()
+ local target= module= softdeps= deps= field= value= firmware
+ local ign_errors=0 found=0
+ [[ 4.8.11-1-ARCH == none ]]
+ [[ crc32? = *\? ]]
+ ign_errors=1
+ set -- crc32
+ target=crc32
+ target=crc32
+ (( _addedmodules[crc32] ))
+ return
++ (( 0 ))
+ set +x
Comment by Zachary Cook (Zeik) - Monday, 28 November 2016, 19:16 GMT
Okay, tested booting again, it seems crc32 is built-in to kernel, but it doesn't boot with the same cannot load crc32 driver error. So that is where the problem is, it needs the modules, but detects crc32 being already included
Comment by Zachary Cook (Zeik) - Monday, 28 November 2016, 20:02 GMT
Looks like the kernel had a problem with 2 different modules having the same name, but had to leave the alias, so crc32 seems like it will not work as it is ambiguous, dracut gets away with it because they seem to just add the module without checking for a built-in, so kmod does the right thing, mkinitcpio just doesn't give kmod a chance and skips including it when it sees the built-in.

See this message and follow-ups:
http://www.spinics.net/lists/linux-crypto/msg18316.html
Comment by Dave Reisner (falconindy) - Friday, 10 March 2017, 12:20 GMT Comment by Zachary Cook (Zeik) - Friday, 10 March 2017, 16:06 GMT
I installed mkinitcpio-git and did a boot test, that definitely fixed it, thanks.

Loading...