FS#79254 - [yt-dlp] "optional" dependencies aren't actually optional

Attached to Project: Arch Linux
Opened by RuRo (ruro) - Sunday, 30 July 2023, 16:13 GMT
Last edited by Antonio Rojas (arojas) - Tuesday, 01 August 2023, 19:43 GMT
Task Type General Gripe
Category Packages: Extra
Status Closed
Assigned To Jaroslav Lichtblau (Dragonlord)
Antonio Rojas (arojas)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Currently, the upstream setup.py script declares all dependencies listed in requirements.txt as strict (non-optional) dependencies.

- https://github.com/yt-dlp/yt-dlp/blob/2023.07.06/setup.py#L146
- https://github.com/yt-dlp/yt-dlp/blob/2023.07.06/requirements.txt

In reality, these dependencies are effectively optional, but the maintainers "recommend to install all dependencies" and are (ab)using install_requires to specify the "default" dependencies (as a workaround for setuptools lacking support for "default" extras).

- https://github.com/yt-dlp/yt-dlp/pull/5482#issuecomment-1308087564

This means that the python dist-info metadata erroneously lists these packages as required, while the PKGBUILD lists them as optdepends.

According to the following issue, downstream distributors should either
1) patch setup.py or requirements.txt when building the wheel from source
OR
2) require mutagen, pycryptodomex, websockets, brotli/brotlicffi and certifi as strict (non-optional) dependencies

- https://github.com/yt-dlp/yt-dlp/issues/6484

Assuming, that we don't want to add these packages as strict (non-optional) dependencies, we should probably do something along the lines of:

sed '/mutagen/d;/pycryptodomex/d;/websockets/d;/brotli/d' -i requirements.txt
This task depends upon

Closed by  Antonio Rojas (arojas)
Tuesday, 01 August 2023, 19:43 GMT
Reason for closing:  Won't fix
Comment by Toolybird (Toolybird) - Sunday, 30 July 2023, 21:37 GMT
This issue seems pretty minor...

> In reality, these dependencies are effectively optional

That's the important part for an Arch pkg.
Comment by Antonio Rojas (arojas) - Sunday, 30 July 2023, 22:12 GMT
what is the actual issue here?
Comment by RuRo (ruro) - Monday, 31 July 2023, 07:16 GMT
Yeah, it's a minor issue (hence the "General Gripe" category), but the fix for it is also pretty simple.

The issue is that the python metadata and pacman metadata disagree about whether these dependencies are optional. The installed package itself THINKS that these dependencies are REQUIRED, but pacman doesn't install them by default.

> grep 'Requires-Dist' /usr/lib/python*/site-packages/yt_dlp*.dist-info/METADATA
Requires-Dist: mutagen
Requires-Dist: pycryptodomex
Requires-Dist: websockets
Requires-Dist: certifi
Requires-Dist: brotlicffi ; platform_python_implementation != "CPython"
Requires-Dist: brotli ; platform_python_implementation == "CPython"

So, for example, `pip check` reports this package instal as broken due to missing dependencies:

> pip check
yt-dlp 2023.7.6 requires brotli, which is not installed.
yt-dlp 2023.7.6 requires mutagen, which is not installed.
yt-dlp 2023.7.6 requires pycryptodomex, which is not installed.
Comment by Antonio Rojas (arojas) - Tuesday, 01 August 2023, 19:43 GMT
I don't think any of the possible solutions to this is worth it (be it the patching effort or the additional dependencies). As a distribution, we make our own decisions regarding package dependencies.

Loading...