FS#59265 - [dkms] Faster install remove for multiple modules

Attached to Project: Arch Linux
Opened by Chris Severance (severach) - Sunday, 08 July 2018, 06:54 GMT
Last edited by Sébastien Luttringer (seblu) - Tuesday, 09 November 2021, 10:16 GMT
Task Type Feature Request
Category Packages: Extra
Status Closed
Assigned To Sébastien Luttringer (seblu)
Architecture x86_64
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Dkms is very slow to install and remove many modules. I removed some modules manually and I noticed why. It runs depmod after every install or remove.

Each dkms install/remove should specify --no-depmod and depmod should be run once at the end, preferably once at the end of both remove and install.
This task depends upon

Closed by  Sébastien Luttringer (seblu)
Tuesday, 09 November 2021, 10:16 GMT
Reason for closing:  Fixed
Additional comments about closing:  dkms 2.8.1-4
Comment by Sébastien Luttringer (seblu) - Friday, 26 October 2018, 11:48 GMT
Yes could be a improvement. A patch is welcome.
Comment by Sébastien Luttringer (seblu) - Wednesday, 27 May 2020, 16:32 GMT
It's in dkms-2.8.1-4. Please test.
Comment by Chris Severance (severach) - Thursday, 28 May 2020, 05:24 GMT
This started as a sorted patch but then I found some bugs created by --no-depmod.

# Bug fixes and features
# 1. Sort modules, randomly reversing for BUILD_DEPENDS
# 2. Omit --no-depmod for a dependency of a to be installed module
# 3. Omit --no-depmod when only one module is updated
# 4. install ignores -D
Comment by Chris Severance (severach) - Thursday, 28 May 2020, 05:56 GMT
When I upgrade kernel to same version I get option -D and remove depmod is suppressed as expected. When I upgrade or downgrade kernel to different version I do not get option -D and remove depmod is run. Pacman is running remove hook when it should be running upgrade hook. Hook files look good.

https://www.archlinux.org/pacman/alpm-hooks.5.html

>Installations are considered an upgrade if the package or file is already present on the system regardless of whether the new package version is actually greater than the currently installed version.
Comment by Sébastien Luttringer (seblu) - Thursday, 28 May 2020, 13:03 GMT
Ok, I got the one about -D and kernel upgrades. That's because matched files are different from one kernel version to another.

What are the others issues you found; Could you be more specific to each one?
What issue is fixed by randomizing the module list?
Comment by Chris Severance (severach) - Thursday, 28 May 2020, 19:53 GMT
# Bug fixes and features
# 1. Sort modules, randomly reversing for BUILD_DEPENDS

Random order is annoying. Many of my modules fail to compile on the latest kernel. Determining which are failing is easier if the order is sorted in some way. Always sorting the same way will mask problems that had been revealed by random order.

We randomly reverse to catch missing BUILD_DEPENDS. If we always sort in ascending order as requested in  FS#53669 , the need for a BUILD_DEPENDS between ZFS and SPL wouldn't have caused the cascade of bug reports that lead to this problem being solved. It also tests the install loop to ensure that dependencies are always sequenced correctly. I added fake BUILD_DEPENDS to two of my nine kernel packages and one run took 3 rounds to get them all compiled. Sorting by module name makes the rounds obvious.

# 2. Omit --no-depmod for a dependency of a to be installed module

ZPL depends on SPL.

So far as I know, for ZPL to compile the SPL symbols must be depmod. --no-depmod prevents this and the ZPL compile would always fail. We could build a solver like systemd to minimize the number of depmod run but deps are rare so I took the easy way. Any module that is a dependency of another module skips --no-depmod. An install sequence with a lot of dependencies might get a few unnecessary depmod, no worse than where we started.

All my test dependencies were fake and SPL is long gone so I know that dependency compile order is correct but I don't know how to test to see if depmod is necessary for dependencies.

# 3. Omit --no-depmod when only one module is updated

No reason to --no-depmod only to depmod later. Let dkms install do the work.

I like having the variation with and without --no-depmod. People will notice, ask questions, which will lead to bug fixes and better dkms packages.

# 4. install ignores -D

Install must always run depmod whether or not -D is erronously specified. -D only applies to remove. usage() text should be amended. Just because you control the hooks and script and won't ever put -D on install doesn't mean this bug should be allowed. Having the disable code in install suggests there is a reason for it, and there is not.

***

NeedsTargets and Arch DKMS hooks have nothing to do with it. Pacman calls the remove hook when it should call the upgrade hook. Pacman might call remove when changing from dkms to dkms-git provides=dkms but it should call upgrade when upgrading from Linux 4.9.224 to 4.9.225 exactly as it calls upgrade when upgrading from Linux 4.9.225 to 4.9.225.

Pacman calls the remove hook when upgrading from Linux 4.9.224 to 4.9.225. The help says that's an upgrade. Allanbrokeit didn't notice because up till now there were no upgrade and remove hooks with a visible difference.

Loading...