FS#62886 - libblockdev plugins' dependencies should be optional

Attached to Project: Arch Linux
Opened by Arsenii (ettavolt) - Thursday, 13 June 2019, 06:36 GMT
Last edited by Balló György (City-busz) - Tuesday, 06 June 2023, 11:19 GMT
Task Type Feature Request
Category Packages: Extra
Status Closed
Assigned To Felix Yan (felixonmars)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 8
Private No

Details

Description:
Currently libblockdev includes all possible plugins and requires their dependencies. However, it doesn't crash if these dependencies are missing and still provides information about about 'native' block devices. Including entire slew of FS tools just because a user wants to have Gnome's Nautilus (nautilus - gvfs - udisk2 - libblockdev) doesn't seem to be an Arch way.
I've conducted next tests:
1.1. Remove btrfs-progs dosfstools dmraid gptfdisk lvm2 mdadm ndctl parted volume_key xfsprogs.
1.2. Restart udisk2.
1.3. Start Gnome Disks and observe disks and volumes. It shows everything.
2.1. Reinstall dosfstools (I thought that because of ESP init may have wanted fsck.vfat, but that doesn't seem to be the case).
2.2. Rebuild initramfs.
2.3. Reboot.
2.4. Open Gnome Disks and observe disks and volumes. It shows everything.
Note: I only use GPT and FAT32 (ESP) of the 'FS' libblockdev dependencies. Whenever users add LVM2/RAID/etc into their system they'll have respective dependency anyway.

Making these dependencies optional will reduce system size and also help avoid problems like shutdown lockups due to LVM2. At least for those not using it.

Additional info:
* libblockdev 2.21-1, udisks 2.8.2-1
* Look how, e.g. Ubuntu is packaging their libblockdev: https://packages.ubuntu.com/source/disco/libblockdev https://packages.ubuntu.com/disco/libblockdev2
This task depends upon

Closed by  Balló György (City-busz)
Tuesday, 06 June 2023, 11:19 GMT
Reason for closing:  Implemented
Additional comments about closing:  libblockdev 2.28-4
Comment by Eli Schwartz (eschwartz) - Thursday, 13 June 2019, 11:44 GMT
The problem is not Nautilus, it is udisks2 entirely:  FS#54329  and  FS#54338 

Are you now claiming that these bug reports are incorrect, or more likely that udisks2 has changed things?
Comment by Arsenii (ettavolt) - Thursday, 13 June 2019, 11:58 GMT
I didn't try without libbytesize (it doesn't seem to be a 'plugin'), only without volume_key and gptfdisk. So, it seems, udisks2 is more robust now.
BTW, Ubuntu lists 'required' libblockdev plugins for their udisks2 package: https://packages.ubuntu.com/disco/udisks2.
Comment by Arsenii (ettavolt) - Wednesday, 07 August 2019, 19:26 GMT
When I've tried to mount an exFAT drive, I've noticed (because it didn't mount), that udisks outputs some errors due to missing volume_key and parted packages immediately at start. So, probably, these should be dependencies anyway.
Comment by Arsenii (ettavolt) - Thursday, 10 September 2020, 11:56 GMT
Retested with
btrfs-progs dosfstools dmraid gptfdisk lvm2 mdadm ndctl parted volume_key xfsprogs
removed: errors from crypto, fs, part modules on restart of the service, again fixed by reinstalling
parted volume_key
So, at least
btrfs-progs dosfstools dmraid gptfdisk lvm2 mdadm ndctl xfsprogs
could be removed from hard dependencies.
Comment by Balló György (City-busz) - Thursday, 02 February 2023, 17:07 GMT
The current version of udisks2 does not crash when a plugin fails to load, so these dependencies should be really optional. I analyzed the dependencies:

Core dependencies, linked by libblockdev.so:
depends=(glib2 systemd-libs kmod)

Base metapackage depends on these packages implicitly, making them optional would be useless:
depends+=(device-mapper e2fsprogs util-linux)

The FS plugin is somewhat essential, no filesystem can be mounted without it. linked by libbd_fs.so:
depends+=(parted)

The libbytesize package is really small, and used by multiple plugins:
depends+=(libbytesize)

These were the essentials, all other dependencies can be optional:
optdepends=(
'btrfs-progs: BTRFS plugin'
'dmraid: DeviceMapper plugin'
'dosfstools: VFAT filesystem support'
'gptfdisk: Part plugin'
'libyaml: VDO plugin'
'lvm2: LVM plugin'
'mdadm: MD RAID plugin'
'multipath-tools: Mpath plugin'
'ndctl: NVDIMM plugin'
'ntfs-3g: NTFS filesystem support'
'volume_key: Crypto plugin'
'xfsprogs: XFS filesystem support'
)

So the final depends/makedepends are:
depends=('device-mapper' 'e2fsprogs' 'glib2' 'kmod' 'libbytesize' 'parted' 'systemd-libs' 'util-linux')
makedepends=('autoconf-archive' 'dmraid' 'gobject-introspection' 'libyaml' 'ndctl' 'python' 'systemd' 'volume_key')

Note that the following optional dependencies are in AUR:
optdepends+=(
'bcache-tools: KernelBlockDevices plugin'
'mpathconf: Mpath plugin'
'vdo: VDO plugin'
)

If these packages are not pulled in from AUR, the following cofigure options can be used to disable these plugins (the KernelBlockDevices and VDO plugins and dmraid support are deprecated anyway):
--without-dmraid --without-kbd --without-bcache --without-mpath --without-vdo
Comment by Chih-Hsuan Yen (yan12125) - Sunday, 19 February 2023, 09:00 GMT
Upstream changed missing libblockdev plugins from errors into warnings in udisks 2.7.3 [1], while such issues are fatal again in git-master [2], so tricks above fail. For example, after backporting commit [2] and `pacman -Rdd volume_key`, starting udisksd (on a system with encrypted partitions) fails with:

Feb 19 16:56:19 udisksd[14241]: failed to load module crypto: libvolume_key.so.1: cannot open shared object file: No such file or directory
Feb 19 16:56:19 udisksd[14241]: Failed to load the 'crypto' libblockdev plugin
Feb 19 16:56:19 kernel: traps: udisksd[14241] trap int3 ip:7f4e66784ec1 sp:7ffd5c4a5df0 error:0 in libglib-2.0.so.0.7400.5[7f4e66741000+93000]

[1] https://github.com/storaged-project/udisks/commit/e15d8b93228493032e875dcbc73eab0d3c09421c
[2] https://github.com/storaged-project/udisks/commit/ff5fddb16a97d8f1b8475a219566c2ee95f3c349
Comment by loqs (loqs) - Sunday, 19 February 2023, 12:13 GMT
@yan12125 what if you remove /usr/lib/libbd_crypto.so.2.0.0 to simulate it being split packaged? Does that still produce the error?
Comment by Chih-Hsuan Yen (yan12125) - Sunday, 19 February 2023, 13:04 GMT
> what if you remove /usr/lib/libbd_crypto.so.2.0.0 to simulate it being split packaged? Does that still produce the error?

An interesting question! Here is the bad news: a similar crash in udisksd with a backported commit

Feb 19 21:00:14 udisksd[15513]: failed to load module crypto: libbd_crypto.so.2: cannot open shared object file: No such file or directory
Feb 19 21:00:14 udisksd[15513]: Failed to load the 'crypto' libblockdev plugin
Feb 19 21:00:14 kernel: traps: udisksd[15513] trap int3 ip:7f2f5bb58ec1 sp:7fff7f4821c0 error:0 in libglib-2.0.so.0.7400.5[7f2f5bb15000+93000]

Apparently udisksd hard-codes a list of libblockdev plugins to load: https://github.com/storaged-project/udisks/blob/8493c56b7ae0dcd23dc68a57bf09d8f8f4d72e04/src/udisksdaemon.c#L323-L329.
Comment by loqs (loqs) - Sunday, 19 February 2023, 13:30 GMT
After that news are the plugins that could be split out without changing udisks depends:
libbd_btrfs (btrfs-progs)
libbd_kbd ()
libbd_lvm (lvm2)
libbd_mpath (multipath-tools)
libbd_vdo (libyaml)
Edit:
Attached split package with hopefully City-busz dependency updates and allowing for udisks2 depends.
Comment by Balló György (City-busz) - Wednesday, 01 March 2023, 01:05 GMT
@yan12125: Thanks for investigating this. In this case 'gptfdisk', 'mdadm' and 'volume_key' must be in depends=(), but the rest can be still optional.
Comment by loqs (loqs) - Friday, 03 March 2023, 15:26 GMT
Updated diff for libblockdev, changed file-system utilities to optdepends and added exfatprogs f2fs-tools nilfs-utils ntfs-3g udftools
Drop gptfdisk [1]
Note about PKGBUILD.diff.udisks2 it also includes a fix for  FS#76345 

[1] https://github.com/storaged-project/libblockdev/pull/569
Comment by Balló György (City-busz) - Friday, 03 March 2023, 15:41 GMT
@loqs: the gptfdisk dependency was removed from the development branch, but not from the 2.x branch which is used for stable releases:
https://github.com/storaged-project/libblockdev/blob/2.x-branch/src/plugins/part.c#L148

@loqs: f2fs-tools, nilfs-utils and udftools are added only to the development branch:
https://github.com/storaged-project/libblockdev/tree/master/src/plugins/fs
https://github.com/storaged-project/libblockdev/tree/2.x-branch/src/plugins/fs
Comment by loqs (loqs) - Friday, 03 March 2023, 15:49 GMT
@City-busz thank you for the review. Please see updated diff, attached.
Comment by Balló György (City-busz) - Saturday, 03 June 2023, 21:42 GMT
I pushed libblockdev 2.28-4 and udisks2 2.9.4-3 with reduced dependencies to [extra-testing]. Please check it.
Comment by loqs (loqs) - Saturday, 03 June 2023, 22:32 GMT
> I pushed libblockdev 2.28-4 and udisks2 2.9.4-3 with reduced dependencies to [extra-testing]. Please check it.
No issues related to the dependency changes. ( Unrelated issues  FS#76345  )
Comment by Arsenii (ettavolt) - Monday, 05 June 2023, 01:34 GMT
Good for me as well.
I've uninstalled xfsprogs and lvm2. Had to keep ndctl, as it is a hard requirement of qemu-system-x86.
Comment by Balló György (City-busz) - Tuesday, 06 June 2023, 11:19 GMT
Okay, I'm closing this task, as many dependencies are optional now.

@loqs: If you still think that the package splitting is important, then open another task, and then the current maintainer (felixonmars) decide if he wants to apply it or not. Personally I don't think that package splitting is important in this case.

Loading...