FS#66489 - [libvirt] massive overlinking

Attached to Project: Community Packages
Opened by Toolybird (Toolybird) - Friday, 01 May 2020, 05:03 GMT
Last edited by freswa (frederik) - Wednesday, 14 October 2020, 20:44 GMT
Task Type Bug Report
Category Packages
Status Closed
Assigned To Christian Rebischke (Shibumi)
Robin Broda (coderobe)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

This PKGBUILD is in rather poor shape. The amount of overlinking as reported by namcap is simply horrendous:

namcap libvirt-5.10.0-3-x86_64.pkg.tar.zst | grep "Unused shared library" | wc -l
895

Most of this can be attributed to libtool brain damage and can be solved with a single line as seen elsewhere in Arch - the libtool hack! Insert this after `./configure' and before `make'

sed -i -e 's/ -shared / -Wl,-O1,--as-needed\0/g' libtool

This makes namcap much happier and as a bonus will even allow split packages à la qemu. Upstream is moving towards meson so this problem will go away eventually.

Now, there are some other questionable things that should be addressed:

1. export LDFLAGS=-lX11

Why? There is absolutely no logical reason for this. No other distro does it. It seems crazy.

2. lxc makedep

Really? There is zero connection between libvirt containers and the "other" lxc.

3. netcf dep

This thing does nothing. Arch has even received criticism from upstream. It needs to go.

https://www.redhat.com/archives/virt-tools-list/2018-October/msg00049.html

4. Numerous other little things which are too tedious to list.

For reference I'm attaching my WIP which is current as at 6.3.0-rc1 (nudge, nudge!). No complaints from namcap re deps. I've analysed the deps quite thoroughly but feel free to poke holes. AFAICT this fixes most known libvirt bugs in the Arch bug tracker. Credit to @loqs for the tmpfiles.d idea.
   PKGBUILD (6.2 KiB)
This task depends upon

Closed by  freswa (frederik)
Wednesday, 14 October 2020, 20:44 GMT
Reason for closing:  Fixed
Additional comments about closing:  libvirt-6.8.0-1 in [community-testing]
Comment by loqs (loqs) - Friday, 01 May 2020, 11:55 GMT
@toolybird please see the updated solution I proposed for  FS#63152  https://bugs.archlinux.org/task/63152#comment187885
Edit:
My proposed changes for libvirt can be found at https://github.com/loqs/packages/commits/wip
Comment by Toolybird (Toolybird) - Saturday, 02 May 2020, 23:34 GMT
> My proposed changes for libvirt

@loqs, good stuff. We both have been scratching the same itch :)

It's a shame Arch has seemingly dropped the ball here. 3 releases behind.. (almost 4), a mounting bug toll. A great thing about Arch is the ABS. Don't like the distro pkg? Build it yourself!

> please see the updated solution I proposed for  FS#63152 

hmm, not sure I agree here. Seems to go further than necessary. There is no real need to change the user/group at this stage IMHO (unless we want to match other distros).
Comment by loqs (loqs) - Sunday, 03 May 2020, 00:43 GMT
My motivation for the move away from the user nobody is based on https://bugs.archlinux.org/task/56828#comment175949 and it does align with upstream's spec file.
The spec file is also where I found the GUID reproducibility fix.
I think after the changes most of the opened libvirt issues can be closed except  FS#64789  did you drop lxc support because of that?
Without lxc I believe you can also drop fuse2 when I was looking through the source I only found it used by lxc.
Comment by Christian Rebischke (Shibumi) - Sunday, 03 May 2020, 12:44 GMT
Hi loqs and toolybird.

Yep, I can whole wholeheartedly confirm your complains.
Let me be honest with you guys, I've just adopted the package, because I thought it shouldn't be orphaned + I needed it for my vagrant images.
I would actually prefer to have a maintainer for it, who takes more care than me about the package + is actually using the package.

@loqs
You've contributed a lot on this package in the last months, would you like to become a TU? Do you have any AUR packages I could checkout? I would be glad to sponsor you.
This package needs a maintainer who does more than bumping a version number and applying patches to keep it running.

Comment by Christian Rebischke (Shibumi) - Sunday, 03 May 2020, 13:17 GMT
@toolybird,

I had a look on your PKGBUILD. Do you have one for the current stable? We don't package release candidates, sorry.
Comment by Toolybird (Toolybird) - Monday, 04 May 2020, 07:02 GMT
> did you drop lxc support because of that?

@loqs, AFAICT I didn't drop any functionality. I only got rid of the bogus makedep on the "lxc" pkg.

Libvirt lxc containers are supposed to be self sufficient and not rely on the "lxc" pkg. Arch wiki suggests same.

Frankly, I've never tried to use Libvirt lxc... but I'll now definitely have a crack.

@Shibumi, thanks for your honest appraisal of the situation.
Comment by Toolybird (Toolybird) - Wednesday, 06 May 2020, 02:45 GMT
> I've never tried to use Libvirt lxc

I just tried the simple "Hello World" example from [1] and it worked fine.

> Do you have one for the current stable?

6.3.0 stable is now released. Seems to run well in my limited testing. Have attached my updated PKGBUILD which is fully self-contained so no additional config files needed. I don't expect Arch to apply this as it's quite a diversion from current Arch with the split packages etc. But you can test it out if you like :)

Note: It's always a good idea to update libvirt-python to the same version when updating libvirt.

[1]: https://www.berrange.com/posts/2011/09/27/getting-started-with-lxc-using-libvirt/
Comment by Robin Broda (coderobe) - Wednesday, 06 May 2020, 11:26 GMT
I really like the approach. We've decided to merge this to our repo and push a build to -testing, which i'll do in a few. Thanks a lot, it's a huge improvement over the current state of things.
Comment by Eli Schwartz (eschwartz) - Wednesday, 06 May 2020, 13:05 GMT
+ install -Dv -t "$pkgdir/usr/lib/libvirt/storage-backend" "$pkgdir/../libvirt_storage_backend_iscsi-direct.so"

WTF, this is using the parent directory of $pkgdir as temporary storage and you have no idea what that is or does without reading makepkg implementation details.
Comment by Toolybird (Toolybird) - Thursday, 07 May 2020, 01:28 GMT
> Thanks a lot

No probs :)

> WTF

You're right, it's not ideal. Will try and improve it...

> update libvirt-python to the same version when updating libvirt

I should have re-phrased the above. There is risk of virt-manager breakage if libvirt and libvirt-python get out of sync. They should always be updated in lockstep IMHO. libvirt-python should be a straight pkgver bump, (except upstream forgot the sig file for 6.3.0) @coderobe, what do you think?
Comment by Eli Schwartz (eschwartz) - Thursday, 07 May 2020, 01:42 GMT
Typically you'd just use "$srcdir/split-backends" or whatever.
Comment by Eli Schwartz (eschwartz) - Thursday, 07 May 2020, 01:45 GMT
Random question: why does the libiscsi backend need to be split? Its dependency is rather tiny. If backends need to be split to prevent load errors (presumably libvirt fails to gracefully ignore plugins which cannot be dlopened, so it is necessary to ensure they are not even on disk?) then I'd just make it hard-depend on libiscsi and have one less package. It's 0.5 MB.
Comment by Robin Broda (coderobe) - Thursday, 07 May 2020, 01:52 GMT
> There is risk of virt-manager breakage if libvirt and libvirt-python get out of sync

noted

> upstream forgot the sig file for libvirt-python 6.3.0

i'm not dropping signatures, downgrading sig checks should never happen - upstream needs to release a signed release if this is the case
fwiw, virt-manager still works (or at least i haven't hit a failure case), so i don't see it as critical enough to even consider dropping signature checks
Comment by Toolybird (Toolybird) - Thursday, 07 May 2020, 01:57 GMT
> Random question: why does the libiscsi backend need to be split?

It doesn't really. I simply followed qemu here. You're right that it's small, but I figured 1 less dep for typical desktop usage is a good thing.
Comment by Robin Broda (coderobe) - Thursday, 07 May 2020, 01:59 GMT
Might as well keep it this way in line with the other backends.
Comment by Eli Schwartz (eschwartz) - Thursday, 07 May 2020, 02:09 GMT
Well, there are 7 other backends which aren't split :p because they don't have any other dependencies at all. Reducing the number of split packages reduces cognitive overhead and for cases of minimal (0.5 MB) gain, I wouldn't split them. But it's your call.

Anyway, this might be worth mentioning to upstream as there really should be no difference in a plugin architecture between a plugin that doesn't exist, and one that does, but cannot be dlopened. If fixed, splitting would longer be needed at all.

I see the ceph optdepends is totally removed. Is this intentional? Is ceph-libs on its own, the only thing needed for the *-rbd backend?
Comment by Toolybird (Toolybird) - Wednesday, 02 September 2020, 20:18 GMT
Latest libvirt-6.7.0 has switched to meson. I've adapted the PKGBUILD and tested it. There are some issues:

- if using `--buildtype plain' (as per arch-meson wrapper script), meson does not add -Wall to compile args which causes massive warnings. Easily fixed by adding -Wall to CFLAGS. Some discussion here [1]

- the current ZFS hack no longer works due to being Autoconf specific. We will need to either a) patch the source or b) use sed to implement a gross hack, or c) drop ZFS support. I've opted for b) (careful, you might puke!). Debian solution here [2]

- tarballs are now signed by a new maintainer. Arch will need to verify the chain of trust

The good news is the libtool hack is no longer needed.

I've fixed up some other little nits and attached a diff. Please take a look. If you decide to drop ZFS support, change the `-D storage_zfs=enabled' to disabled and delete the hacks.

[1]. https://www.redhat.com/archives/libvirt-users/2020-September/date.html
[2]. https://www.redhat.com/archives/libvirt-users/2020-September/msg00003.html
Comment by freswa (frederik) - Sunday, 11 October 2020, 22:47 GMT
This is on hold due to a new gpg key used for signing the archive starting from v6.6 without any chain of trust between the old and the new key.

Loading...