FS#30812 - [python2] _EXEC_PREFIX in sysconfig.py is assigned an invalid path when python 2.7.3 i

Attached to Project: Arch Linux
Opened by Edmond Burnett (Phoric) - Monday, 23 July 2012, 06:53 GMT
Last edited by Evangelos Foutras (foutrelis) - Tuesday, 11 March 2014, 16:05 GMT
Task Type Bug Report
Category Upstream Bugs
Status Closed
Assigned To Angel Velasquez (angvp)
Evangelos Foutras (foutrelis)
Felix Yan (felixonmars)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 14
Private No

Details

Description:

Python breaks with an IOError when invoked from some applications such as uWSGI and GUFW, and when these applications are run as a daemon from a script. In these conditions, a variable named _EXEC_PREFIX, in /usr/lib/python2.7/sysconfig.py is getting assigned an incorrect or empty value for the path to /usr.


Additional info:
* extra/python2 2.7.3
* Forum thread: https://bbs.archlinux.org/viewtopic.php?id=145364
* uwsgi log output w/Python Traceback:

*** Starting uWSGI 1.2.4 (64bit) on [Sat Jul 21 01:31:17 2012] ***
compiled with version: 4.7.1 on 19 July 2012 00:08:12
detected number of CPU cores: 4
current working directory: /
writing pidfile to /var/run/uwsgi.pid
detected binary path: /usr/bin/uwsgi
*** WARNING: you are running uWSGI without its master process manager ***
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
uwsgi socket 0 bound to TCP address 127.0.0.1:3031 fd 3
Python version: 2.7.3 (default, Apr 24 2012, 00:00:54) [GCC 4.7.0 20120414 (prerelease)]
Traceback (most recent call last):
File "/lib/python2.7/site.py", line 563, in <module>
main()
File "/lib/python2.7/site.py", line 545, in main
known_paths = addusersitepackages(known_paths)
File "/lib/python2.7/site.py", line 278, in addusersitepackages
user_site = getusersitepackages()
File "/lib/python2.7/site.py", line 253, in getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
File "/lib/python2.7/site.py", line 243, in getuserbase
USER_BASE = get_config_var('userbase')
File "/lib/python2.7/sysconfig.py", line 521, in get_config_var
return get_config_vars().get(name)
File "/lib/python2.7/sysconfig.py", line 420, in get_config_vars
_init_posix(_CONFIG_VARS)
File "/lib/python2.7/sysconfig.py", line 299, in _init_posix
raise IOError(msg)
IOError: invalid Python installation: unable to open //include/python2.7/pyconfig.h (No such file or directory)



(Note the double forward-slash ('//') on the last line. The path should read /usr/include/python2.7/pyconfig.h)


Steps to reproduce:

1. Install python2 from extra.
2. Install uWSGI from the AUR or using "pip2 install uwsgi".
3. Configure an rc.d startup script to daemonize a uWSGI/python2 web app.
4. View uwsgi log output
This task depends upon

Closed by  Evangelos Foutras (foutrelis)
Tuesday, 11 March 2014, 16:05 GMT
Reason for closing:  Fixed
Additional comments about closing:  pyconfig.h is no longer parsed and thus this bug doesn't occur anymore.
Comment by Mahoru (Mahoru) - Tuesday, 24 July 2012, 11:43 GMT
This problem happends to when adding a rule to ufw with aur kcm-ufw
Comment by Jean-Patrick Simard (jpsimard) - Saturday, 28 July 2012, 13:17 GMT
The same problem is affecting /usr/bin/hp-firmware and hp-check-plugin, both raising "IOError: invalid Python installation: unable to open //include/python2.7/pyconfig.h (No such file or directory)".

This prevents my hp p1005 printer from having its firmware uploaded, and therefore prevents it to print at all.
Comment by Jelle van der Waa (jelly) - Sunday, 29 July 2012, 09:54 GMT
The HP programs work perfect for me, they launch. I guess this is more a AUR program problem from my viewpoint. But since I don't know about _EXEC_PREFIX, I assigned it to our python maintainer.
Comment by Stéphane Gaudreault (stephane) - Sunday, 29 July 2012, 13:20 GMT
In /usr/lib/python2.7/sysconfig.py the variable is initialized with
_EXEC_PREFIX = os.path.normpath(sys.exec_prefix)

On my machine, this return the correct path

$ python2
Python 2.7.3 (default, Apr 24 2012, 00:00:54)
[GCC 4.7.0 20120414 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, sys
>>> os.path.normpath(sys.exec_prefix)
'/usr'
>>>
Comment by Jean-Patrick Simard (jpsimard) - Sunday, 29 July 2012, 13:52 GMT
I just tried the same python commands, and my system does respond in the same way, printing out '/usr'. It remains that it doesn't work with hp-firmware or hp-plugin and I'm not alone (https://bbs.archlinux.org/viewtopic.php?pid=1137090#p1137090). The problem is contemporary to the /lib to /usr/lib thing, I suspect that there is a relation there. Not that I actually know how or why the problem exists, let alone how to fix it should my assumptions be true.
Comment by Jelle van der Waa (jelly) - Sunday, 29 July 2012, 14:20 GMT
This all worksforme:
[jelle@P8][~]% /usr/bin/uwsgi --daemonize /var/log/uwsgid.log --ini /etc/uwsgi/uwsgi.ini --pidfile /var/run/uwsgid.pid
[uWSGI] getting INI configuration from /etc/uwsgi/uwsgi.ini

My _EXEC_PREFIX is also the same as stephane's.

So please show your hp/python version
Comment by Jean-Patrick Simard (jpsimard) - Sunday, 29 July 2012, 14:44 GMT
The packages I have installed as pacman shows them...

python2 2.7.3-2

hplip 3.12.6-3

hplip-plugin 3.12.6-3 (that's from the AUR)

I'm on a 64bit install.
Comment by Edmond Burnett (Phoric) - Sunday, 29 July 2012, 22:50 GMT
I can also confirm that Python behaves as expected when invoked directly:

python2 -c "import os,sys; print os.path.normpath(sys.exec_prefix)"

returns: /usr

The error still occurs for me when uWSGI is invoked from an initscript, on both my workstation and a VM. I am not installing uWSGI from AUR, rather from pip. I don't have an HP printer so I can't test that method.
Comment by Gerardo Exequiel Pozzi (djgera) - Saturday, 11 August 2012, 19:18 GMT
Running a simple test_path.py:

--
#!/usr/bin/env python2

import os, sys
print os.path.normpath(sys.exec_prefix)
--

from udev rule with:
--
RUN+="/bin/sh -c '/root/test_path.py >> /tmp/path_python'"
--

gives "/" but if you change shebang "#!/usr/bin/env python2" to "#!/usr/bin/python2" gives the correct path "/usr"

This is at least happens for hp-firmware.
Comment by Stéphane Gaudreault (stephane) - Wednesday, 15 August 2012, 23:33 GMT
From what I understand the environment variables provided to an udev rule are scarce and so using "#!/usr/bin/env python" might not work. For example, PATH is not even defined in this context.
I think it is a better practice to use a shebang line that explicitly references the system version of Python.
Comment by Evangelos Foutras (foutrelis) - Sunday, 26 August 2012, 04:11 GMT
The issue with uwsgi can be replicated like this:

1) Install uwsgi from the AUR
2) Change your location to the root directory (/) and run 'uwsgi --master'
3) Receive delicious error about not being able to open //include/python2.7/pyconfig.h
4) (Optional) Notice how running the above command from e.g. your home directory won't trigger the error from (3)
Comment by Evangelos Foutras (foutrelis) - Sunday, 26 August 2012, 04:48 GMT
Taking uwsgi out of the picture:

If I copy /usr/bin/python2.7 to /python2.7 and try to execute that, I get "unable to open //include/python2.7/pyconfig.h" as before.

However, if I copy /usr/bin/python2.7 to say /mnt/python2.7, it'll run just fine.

strace reveals that in the first case the /lib/python2.7/ path is used, whereas in the second case the path includes the /usr prefix (i.e. /usr/lib/python2.7/).

[Captain Obvious] I believe /lib being a symlink to /usr/lib is of relevance.
Comment by Evangelos Foutras (foutrelis) - Wednesday, 19 September 2012, 19:59 GMT
Filed a bug report upstream after researching the issue a bit more:

http://bugs.python.org/issue15976
Comment by Felix Yan (felixonmars) - Saturday, 15 February 2014, 15:59 GMT
@foutrelis
I cannot reproduce the issue now, neither with uwsgi as OP said, or by copying python2.7 to / as you tried, python always loads successfully.

Was it already fixed?
Comment by Evangelos Foutras (foutrelis) - Saturday, 15 February 2014, 16:50 GMT
It appears that Python no longer parses pyconfig.h so this issue doesn't occur anymore. :)

Upstream issue that changed this: http://bugs.python.org/issue13150

Loading...