FS#65145 - [intel-media-sdk] Please consider splitting out libmfx.so

Attached to Project: Community Packages
Opened by Thomas Lübking (luebking) - Tuesday, 14 January 2020, 19:08 GMT
Last edited by Maxime Gauduin (Alucryd) - Wednesday, 29 January 2020, 10:09 GMT
Task Type Feature Request
Category Packages
Status Closed
Assigned To Daniel Bermond (Bermond)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 10
Private No

Details

See https://bbs.archlinux.org/viewtopic.php?pid=1882702 for context.

tl;dr - omnipresent ffpmeg's libav* link libmfx.so, pulling this package and in turn some more dependencies w/o interest for ffmpeg users not relying on intels VAAPI (but AMD or nvidia or software if they don't have a supported chip), more than tripling the footprint of ffmpeg.

Apparently dlopen'ing the library is not an option and/or rejected by the ffmpeg maintainer, but feel free to try punting over there.

Thanks for reading until here.
This task depends upon

Closed by  Maxime Gauduin (Alucryd)
Wednesday, 29 January 2020, 10:09 GMT
Reason for closing:  Implemented
Additional comments about closing:  19.4.0-4
Comment by Maciej Stanczew (stanczew) - Wednesday, 15 January 2020, 18:59 GMT
There is a libmfx package in community [1], with `provides=(libmfx.so)`.

Installing it (instead of intel-media-sdk) results in the following warnings when running ffmpeg (or anything that links to libav*):
ffmpeg: /usr/lib/libmfx.so.1: no version information available (required by /usr/lib/libavfilter.so.7)
ffmpeg: /usr/lib/libmfx.so.1: no version information available (required by /usr/lib/libavcodec.so.58)
ffmpeg: /usr/lib/libmfx.so.1: no version information available (required by /usr/lib/libavcodec.so.58)
ffmpeg: /usr/lib/libmfx.so.1: no version information available (required by /usr/lib/libavutil.so.56)

Other than that, ffmpeg seems to be working fine -- at least on AMD and NVIDIA systems. I can play videos using mpv without any issues.

[1] https://www.archlinux.org/packages/community/x86_64/libmfx/
Comment by Thomas Lübking (luebking) - Wednesday, 15 January 2020, 20:17 GMT
The library in that package seems dated, so it's no general substitute (also the provisions currently don't match)

But actually the existence of the package (thanks for the heads up) makes the requested split all the more reasonable, because it will collide with this one here (not on a filepath level, but the library resolution will become somewhat ambigious)

Interestingly, its maintainership is split between the maintainers of intel-media-sdk and ffmpeg… ;-)
Comment by Daniel Bermond (Bermond) - Saturday, 18 January 2020, 20:50 GMT
I have discussed this internally with the ffmpeg package maintainer.

From the intel-media-sdk packaging point of view, it does not make any sense to split it into another package just to provide libmfx.so, as the package itself does not need to be splitted. This would not conform with the Arch Linux KISS principle and will add extra work for packaging. Please allow me to remember that here at Arch Linux we do not provide -devel and lib- packages as other distributions do, but we ship everything into a single package to conform with the KISS principle, at the "cost" of higher package sizes.

Having said that, splitting the package would not be a reasonable implementation for intel-media-sdk, and I understand that unfortunately this may frustrate some people.

So, I'm sorry guys, but intel-media-sdk will not be splitted, at least for now.
Comment by Eli Schwartz (eschwartz) - Sunday, 19 January 2020, 07:38 GMT
I would just like to point out, that this is a garbage reason. KISS and "we do not just split out devel and lib packages like other distros do" flies in the face of *practicalities*, like:

- linux (a 73MB package more or less everyone needs) and linux-headers (a 112.18MB package very few people need)
- boost-libs
- ceph-libs
- gcc-libs
- ghc-libs
- llvm-libs
- mariadb-libs
- ocaml-compiler-libs
- postgresql-libs
- systemd-libs

When it's suspected many or even most users do not need the entirety of a package, and splitting it out can dramatically reduce the size of packages that need to be downloaded and installed, then it is eminently reasonable to create a split package.

Ultimately it is up to the maintainer to decide how to handle things, but please don't throw policy at people and say you're not allowed to do it.
Comment by Eli Schwartz (eschwartz) - Sunday, 19 January 2020, 07:43 GMT
> Having said that, splitting the package would not be a reasonable implementation for intel-media-sdk

This sounds like you may be indicating that for reasons internal to the package, it would be impossible to provide this because it would be broken. But the initial close message said it wasn't "needed", and that it isn't KISS.

So, which is it? If the former, then this can be closed as "WONTFIX: it would be broken to split this". If the latter, then I believe closing this was a hasty decision.
Comment by Eli Schwartz (eschwartz) - Sunday, 19 January 2020, 07:50 GMT
I would also like to know, if it is so silly to split out libmfx, why is there a standalone libmfx package at all? And for that matter... why is there a standalone package either way? Is there some reason its one user (emby-server) can't use the version of the libmfx dispatcher library provided by intel-media-sdk?

If it *can* use it, and the fact that intel-media-sdk "provides" libmfx implies such, this indicates that the initial premise was right, and libmfx is fully standalone, so it should be *provided* fully standalone due to being far less... weighty. For much the same reason as the previously mentioned list of -libs packages which archlinux does in fact provide.
Comment by Daniel Bermond (Bermond) - Sunday, 19 January 2020, 15:22 GMT
@eschwartz

All the lib packages that you have mentioned have a significant amount of files and/or have a significant size. While in this case it would be a single 50kb lib and IMHO a split would not look reasonable to me.

I have not mean the package would be broken.

I have not said that it would be silly to split out libmfx. You're are telling this.

I'm not "throwing" anything on anyone here. I'm just following well established principles after discussing the case with the related developer and reaching agreement with him.

Regarding libmfx itself, it needs a bit of history. Back in time, Intel provided libmfx as part of a huge file in their developer website, in a place where you needed to register to be able to see the download link. This huge file was bundled with a lot of other things from Intel. In this time, there was an effort to provide a lighter version of libmfx with a different build system at GitHub by a guy named lu-zero, for the purpose of easing the testing of the Intel Quick Sync Video support in Libav (this is the content of package libmfx in our repos). After some time, Intel came and released their media-sdk at GitHub, making lu-zero's libmfx to be not needed anymore. lu-zero's libmfx is non-official (is not from Intel) and tends to be outdated as it's always following Intel. Intel is the official developer of their media-sdk and that's what should be used when needing to deal with Intel Quick Sync Video. I have already suggested the emby-server packager (he is the same one as ffmpeg package) about using intel-media-sdk in replacement of lu-zero's libmfx due to these exposed reasons, and as you can see ffmpeg has already started to use it. When/if lu-zero's libmfx is not needed by any package anymore I think we can drop it and stay only with intel-media-sdk.
Comment by Eli Schwartz (eschwartz) - Sunday, 19 January 2020, 15:48 GMT
If packages "could" use a standalone libmfx to satisfy shared linker dependencies and then dispatch to 64.45 MB worth of intel-media-driver, intel-gmmlib, and the vast bulk of intel-media-sdk only for people who have the relevant hardware (I *think* libmfx looks like something which dlopens plugins as needed?) then that is indeed a significant size!

With this context, I feel justified in saying there is no "well established principles". Instead, there is an ambiguity and a place where the maintainer can and should exercise their judgment, just as all those *-lib packages exercised their judgment (and surely other packages exercised their judgment in the opposite direction, but I have no immediately available stats on that, so my examples are unfortunately one-sided. I acknowledge this).

If you still don't feel splitting out the package is worth it, then okay. But I just wanted to make sure that this bug report wasn't being incorrectly closed purely on the grounds that "it is established practice to not split packages", because that's simply not true.

> I have not mean the package would be broken.

Then in that case, if we did in fact have a split package for libmfx vs. the rest of intel-media-sdk, the practical outcome would be most ffmpeg users being able to use ffmpeg just fine, while removing 65MB of packages from their systems which they don't need, and a small handful of people manually installing intel-media-sdk for enhanced performance on their hardware? Including the 33MB /usr/lib/dri/iHD_drv_video.so?

...

Thank you for the context behind libmfx. I agree it would make sense to consolidate on one libmfx representation.
Comment by Thomas Lübking (luebking) - Sunday, 19 January 2020, 15:59 GMT
> in this case it would be a single 50kb lib and IMHO a split would not look reasonable to me.

To be sure: your objection is not that the libmfx package would be "too small"™, is it?
Comment by Maxime Gauduin (Alucryd) - Tuesday, 21 January 2020, 07:17 GMT
@Eli Yes the main reason was KISS, not that the package would be broken. Some of us actually feel stronger about it, but we're open to valid arguments, and this is one of them. The other reason was the intel media SDK is a bit more complicated than the legacy libmfx and splitting the SDK needed more consideration. After discussing with Daniel, it seems to be okay to just split libmfx and replace our legacy package with it, so we'll go ahead and do just that.
Comment by Eli Schwartz (eschwartz) - Tuesday, 21 January 2020, 14:46 GMT
Daniel, Maxime,

Thanks for clarifying, and for the consideration. I'm happy to see there is a way to allow everyone to be reasonably happy. :D

I appreciate you bearing with me while I argued in favor of this change.

Loading...