FS#78413 - [python] Ship EXTERNALLY-MANAGED file (PEP 668)

Attached to Project: Arch Linux
Opened by Phil Schaf (flying-sheep) - Friday, 05 May 2023, 10:47 GMT
Last edited by Felix Yan (felixonmars) - Monday, 05 June 2023, 10:38 GMT
Task Type Feature Request
Category Packages: Core
Status Closed
Assigned To Felix Yan (felixonmars)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 2
Private No

Details

PEP 668 (https://peps.python.org/pep-0668/) says:

> Software distributors who have a non-Python-specific package manager that manages libraries in the sys.path of their Python package should, in general, ship a EXTERNALLY-MANAGED file in their standard library directory.

*This definitely applies to Arch Linux*

The file is an INI file containing error message(s) in potentially multiple languages (See the PEP for specifics)

Currently, having the file (unless circumvented via CLI switch or configuration) prevents `pip`
1. from installing packages into the pacman managed site-packages directory (which is nothing but great) but also
2. from installing packages using `pip install --user` into `~/.local/lib/python*/site-packages`, which is ultimately good but might confuse users at first.

The existence of `pipx` and virtual environments means that better options exist for all use cases covered by `pip install --user`, but users could be confused as to why this stopped working.

I think we should therefore mention all these use cases and maybe the way to circumvent the behavior in the error message:
- People are supposed to use `pacman` to manage system packages (`/usr/lib/python*/site-packages`)
- People are supposed to use `python -m venv` or `python -m virtualenv` to create virtual environments for developing Python projects.
- People can choose to use `pipx` for user-wide installation of Python command line tools.
- If they really want, people can use `pip --break-system-packages` or the corresponding config file entry or env variable to circumvent this behavior.

In summary, I see no downside to doing this:
- Upstream wants us to do it
- There are blessed ways to do things that are encouraged by the change
- There are flexible ways to restore old behavior for users resistant to change or on a deadline
- There is a way to inform users of those ways (the configurable error message)
This task depends upon

Closed by  Felix Yan (felixonmars)
Monday, 05 June 2023, 10:38 GMT
Reason for closing:  Implemented
Additional comments about closing:  3.11.3-2
Comment by Toolybird (Toolybird) - Friday, 05 May 2023, 20:56 GMT Comment by Phil Schaf (flying-sheep) - Monday, 05 June 2023, 08:03 GMT
There’s now an AUR package implementing this functionality. You could use its `EXTERNALLY-MANAGED` file as inspiration:

https://aur.archlinux.org/packages/python-externally-managed
Comment by Felix Yan (felixonmars) - Monday, 05 June 2023, 10:37 GMT
Thanks, that is indeed very helpful.

Loading...