FS#40713 - [python-pip] pip does not use /etc/ssl/certs/ca-certificates.crt

Attached to Project: Arch Linux
Opened by Quentin (Quentin) - Thursday, 05 June 2014, 15:48 GMT
Last edited by Eli Schwartz (eschwartz) - Sunday, 03 June 2018, 04:35 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To Felix Yan (felixonmars)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 6
Private No

Details

python-pip 1.5.6-1 does not use the system SSL certificates. This is an issue in many workplaces where an SSL proxy uses a self-signed certificate that needs to be added to the system.

Here is how you could solve the issue. pip bundles the requests Python library to download files. This library includes its own certificate bundle (located at usr/lib/python3.4/site-packages/pip/_vendor/requests/cacert.pem) which is used by default. However, it is not designed to be used in distributions and [usr/lib/python3.4/site-packages/pip/_vendor/requests/certs.py](https://github.com/kennethreitz/requests/blob/40a060cf579a7f0996c92eed578148e526d9416f/requests/certs.py) is an easy way to point to the correct bundle, that is, /etc/ssl/certs/ca-certificates.crt.
This task depends upon

Closed by  Eli Schwartz (eschwartz)
Sunday, 03 June 2018, 04:35 GMT
Reason for closing:  Fixed
Additional comments about closing:  python-pip 10.0.1-1
Comment by Dan McGee (toofishes) - Sunday, 24 May 2015, 14:29 GMT
If we use the PIP_NO_VENDOR_FOR_DOWNSTREAM option when building, we could use the system requests library which has already been patched in this fashion. https://github.com/pypa/pip/blob/develop/setup.py#L13
Comment by Eli Schwartz (eschwartz) - Sunday, 06 August 2017, 23:55 GMT
Given that we still do not de-vendor requests, maybe the time has come to patch this? :)
Comment by Felix Yan (felixonmars) - Monday, 07 August 2017, 05:23 GMT
I did that quite some versions ago, so I guess only PIP_NO_VENDOR_FOR_DOWNSTREAM is needed :)
Comment by Eli Schwartz (eschwartz) - Wednesday, 25 April 2018, 16:11 GMT
Any update on this? It should be as simple as adding relevant dependencies to the PKGBUILD, then deleting everything in pip-$pkgver/_vendor/ except for __init__.py

(PIP_NO_VENDOR_FOR_DOWNSTREAM is obsolete since https://github.com/pypa/pip/commit/2912910b44e7f1123fc014be3c377fa8ee08c44d in June 2015.)
Comment by Eli Schwartz (eschwartz) - Wednesday, 16 May 2018, 21:59 GMT
So I've been looking into this, and the following PKGBUILD works.

I needed to submit this pull request: https://github.com/pypa/pip/pull/5418
I also added the cachecontrol, progress, and pytoml modules for python2/3 to [community], and I intend to add distlib as well but the testsuite is failing: https://bitbucket.org/pypa/distlib/issues/107/test-suite-failures

So distlib is still vendored, but otherwise I'm now running the latest pip 10.x with devendored everything, more reliably respecting requests' ca-certs fix, and as a side benefit python{,2}-pip are both 5 MB smaller due to deduplication.
Comment by Eli Schwartz (eschwartz) - Friday, 18 May 2018, 03:52 GMT
distlib is now in community, and this PKGBUILD builds against it.

https://pkgbuild.com/~eschwartz/python-pip-10.0.1-1-any.pkg.tar.xz
https://pkgbuild.com/~eschwartz/python-pip-10.0.1-1-any.pkg.tar.xz.sig
https://pkgbuild.com/~eschwartz/python2-pip-10.0.1-1-any.pkg.tar.xz
https://pkgbuild.com/~eschwartz/python2-pip-10.0.1-1-any.pkg.tar.xz.sig

pip is now fully de-vendored.

EDIT: now with manpages.
   PKGBUILD (2.9 KiB)
Comment by Felix Yan (felixonmars) - Saturday, 19 May 2018, 15:12 GMT
python-pip-10.0.1-1 has been pushed to [testing]. Thanks!
Comment by Eli Schwartz (eschwartz) - Friday, 25 May 2018, 01:34 GMT
So, this is issueful because of https://github.com/pypa/pip/issues/5429 and  FS#58670 

The root of the problem seems to be setuptools, which does borky things with the pkg_resources vendoring code. There's one proposed fix there, but upstream hasn't responded in a couple days, so it might make more sense to use something like this logic to simply revert *all* import paths in setuptools (at least) to their native, unvendored versions: https://paste.xinu.at/EiIvv/

This would let us completely circumvent the whole vendor-everything-then-run-weird-proxies issue, at the cost of, well, patching stuff. It seems to work well, though -- ironically, better than using the supported method.

Loading...