Arch Linux

Please read this before reporting a bug:
https://wiki.archlinux.org/title/Bug_reporting_guidelines

Do NOT report bugs when a package is just outdated, or it is in the AUR. Use the 'flag out of date' link on the package page, or the Mailing List.

REPEAT: Do NOT report bugs for outdated packages!
Tasklist

FS#61552 - [python-pyqt5] .pyc files untracked by pacman

Attached to Project: Arch Linux
Opened by Chris Billington (chrisjbillington) - Sunday, 27 January 2019, 21:50 GMT
Last edited by Antonio Rojas (arojas) - Saturday, 02 February 2019, 14:25 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To Antonio Rojas (arojas)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description:

The python-pyqt5 package includes .py files that are not compiled to python bytecode (.pyc files). In some circumstances they can be created after installing the package, but are untracked by pacman:

$ sudo lostfiles | grep PyQt5
/usr/lib/python2.7/site-packages/PyQt5/__init__.pyc
/usr/lib/python3.7/site-packages/PyQt5/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/__pycache__/__init__.cpython-37.pyc
/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/compiler.cpython-37.pyc
/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/indenter.cpython-37.pyc
/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/__init__.cpython-37.pyc
/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/misc.cpython-37.pyc
/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/proxy_metaclass.cpython-37.pyc
/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/qobjectcreator.cpython-37.pyc
/usr/lib/python3.7/site-packages/PyQt5/uic/Compiler/__pycache__/qtproxies.cpython-37.pyc
/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__/ascii_upper.cpython-37.pyc
/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__/as_string.cpython-37.pyc
/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__/__init__.cpython-37.pyc
/usr/lib/python3.7/site-packages/PyQt5/uic/port_v3/__pycache__/proxy_base.cpython-37.pyc
/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__
/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/exceptions.cpython-37.pyc
/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/icon_cache.cpython-37.pyc
/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/__init__.cpython-37.pyc
/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/objcreator.cpython-37.pyc
/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/properties.cpython-37.pyc
/usr/lib/python3.7/site-packages/PyQt5/uic/__pycache__/uiparser.cpython-37.pyc

This is undesirable for two reasons.

1. If a program running as root imports python-pyqt5, the python modules will be compiled, and the resulting .pyc files saved under in /usr/lib/python3.7/site-packages. These files are not tracked by pacman and so remain after PyQt is uninstalled, and may cause issues if PyQt5 is updated. I know this sort of thing is not supposed to cause issues since Python checks that the bytecode is not out of date compared to the source file, but corner cases still can come up in the case of for example, a .py file being deleted, but the .pyc file still being present. The module will still be importable even though it was supposed to have been removed by the package. Here is an example of a similar thing causing issues with the conda packages:

https://github.com/conda/conda/issues/5290

2. If the user never runs pyqt5 programs as root, then any pyqt5 programs they do run will start a little slower owing to the lack of precompiled Python bytecode. (Though as soon as they run one as root, subsequent runs of programs not as root will be able to use the precompiled bytecode.

Additional info:
* 5.11.3-2

Steps to reproduce:
sudo pacman -S python-pyqt5
sudo python -c 'import PyQt5'
lostfiles | grep PyQt5

The solution is to explicitly precompile all the python files so that they are available and tracked by pacman. A patch to the PKGBUILD that adds these lines is attached. Note that it also deletes a submodule from each of the python versions' packages - this is because both bundle python2 and python3 versions of the module, and which is imported is determined at runtime by checking the Python version. I did this because otherwise the compileall command emits syntax errors about finding python3 source whilst it is running in python 2.

See also this similar issue in hplip:

https://bugs.archlinux.org/task/61329
This task depends upon

Closed by  Antonio Rojas (arojas)
Saturday, 02 February 2019, 14:25 GMT
Reason for closing:  Fixed
Additional comments about closing:  pyqt5 5.11.3-4
Comment by Chris Billington (chrisjbillington) - Monday, 28 January 2019, 22:00 GMT
  • Field changed: Percent Complete (100% → 0%)
I noticed that you didn't include the lines that produce optimised byte code. The Arch Python package guidelines suggest optimised bytecode should be included too for Python packages, would you consider including them too? Otherwise running python with 'sudo Python -O' would create untracked files still. Apologies for not explaining the purpose of these lines in the bug report.

Loading...