FS#71344 - [salt] pkg_resources.DistributionNotFound: The 'contextvars' distribution was not found and is requi

Attached to Project: Community Packages
Opened by Imran Iqbal (myii) - Wednesday, 23 June 2021, 21:02 GMT
Last edited by Morten Linderud (Foxboron) - Monday, 28 June 2021, 19:54 GMT
Task Type Bug Report
Category Packages
Status Closed
Assigned To Johannes Löthberg (demize)
Morten Linderud (Foxboron)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Description:

We build Docker images with Salt pre-installed but we hit a problem with the latest version released (`3003.1`). The package installs fine but trying to use Salt results in this traceback:

```
# salt -V
Traceback (most recent call last):
File "/usr/sbin/salt", line 33, in <module>
sys.exit(load_entry_point('salt==3003.1', 'console_scripts', 'salt')())
File "/usr/lib/python3.9/site-packages/salt/scripts.py", line 533, in salt_main
import salt.cli.salt
File "/usr/lib/python3.9/site-packages/salt/cli/salt.py", line 6, in <module>
import salt.utils.job
File "/usr/lib/python3.9/site-packages/salt/utils/job.py", line 12, in <module>
import salt.minion
File "/usr/lib/python3.9/site-packages/salt/minion.py", line 23, in <module>
import salt.beacons
File "/usr/lib/python3.9/site-packages/salt/beacons/__init__.py", line 9, in <module>
import salt.loader
File "/usr/lib/python3.9/site-packages/salt/loader.py", line 26, in <module>
import salt.config
File "/usr/lib/python3.9/site-packages/salt/config/__init__.py", line 16, in <module>
import salt.syspaths
File "/usr/lib/python3.9/site-packages/salt/syspaths.py", line 21, in <module>
import salt.utils.platform
File "/usr/lib/python3.9/site-packages/salt/utils/platform.py", line 10, in <module>
from salt.utils.decorators import memoize as real_memoize
File "/usr/lib/python3.9/site-packages/salt/utils/decorators/__init__.py", line 19, in <module>
import salt.utils.args
File "/usr/lib/python3.9/site-packages/salt/utils/args.py", line 15, in <module>
import salt.utils.versions
File "/usr/lib/python3.9/site-packages/salt/utils/versions.py", line 30, in <module>
import salt.version
File "/usr/lib/python3.9/site-packages/salt/version.py", line 9, in <module>
import salt.utils.entrypoints
File "/usr/lib/python3.9/site-packages/salt/utils/entrypoints.py", line 36, in <module>
import pkg_resources
File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3243, in <module>
def _initialize_master_working_set():
File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3226, in _call_aside
f(*args, **kwargs)
File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3255, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 568, in _build_master
ws.require(__requires__)
File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 886, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 772, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'contextvars' distribution was not found and is required by salt

```

Spoke to one of the Salt employees and was told:

> either contextvars needs packaged, or they need to patch it to not require contextvars
> contextvars is only needed for salt-ssh to older platforms.

We are able to manually resolve this by installing `contextvars` with `pip`:

```
# pip install contextvars
Collecting contextvars
Downloading contextvars-2.4.tar.gz (9.6 kB)
Collecting immutables>=0.9
Downloading immutables-0.15-cp39-cp39-manylinux1_x86_64.whl (102 kB)
|████████████████████████████████| 102 kB 4.5 MB/s
Using legacy 'setup.py install' for contextvars, since package 'wheel' is not installed.
Installing collected packages: immutables, contextvars
Running setup.py install for contextvars ... done
Successfully installed contextvars-2.4 immutables-0.15

[root@47bdaa6d7ef2 /]# salt -V
Salt Version:
Salt: 3003.1

Dependency Versions:
cffi: 1.14.5
cherrypy: Not Installed
dateutil: Not Installed
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.0.1
libgit2: Not Installed
M2Crypto: 0.37.1
Mako: Not Installed
msgpack: 1.0.2
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: 2.20
pycrypto: Not Installed
pycryptodome: 3.10.1
pygit2: Not Installed
Python: 3.9.5 (default, May 24 2021, 12:50:35)
python-gnupg: Not Installed
PyYAML: 5.4.1
PyZMQ: 22.0.3
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.4

System Versions:
dist: arch
locale: utf-8
machine: x86_64
release: 5.4.0-74-generic
system: Linux
version: Arch Linux
```
This task depends upon

Closed by  Morten Linderud (Foxboron)
Monday, 28 June 2021, 19:54 GMT
Reason for closing:  Fixed
Additional comments about closing:  3003.1-2
Comment by loqs (loqs) - Wednesday, 23 June 2021, 21:47 GMT
What if you rebuild the package with the requirement on contextvars removed? Please see attached diff.
   PKGBUILD.diff (0.5 KiB)
Comment by Morten Linderud (Foxboron) - Wednesday, 23 June 2021, 22:00 GMT
What does "to older platforms" mean here? Is it fine to patch it out or should I package it up? what's the potential loss?
Comment by loqs (loqs) - Wednesday, 23 June 2021, 22:34 GMT
contextvars was added to the python library in 3.7 [1][2][3]

So I believe it is available but the requirement is not satisfied.

[1] https://docs.python.org/3/library/contextvars.html
[2] https://pypi.org/project/contextvars/
[3] https://www.python.org/dev/peps/pep-0567/
Edit:
See also
https://github.com/saltstack/salt/issues/59942
https://github.com/saltstack/salt/commit/27b402a5ed5925c392d8baf43a0ba62db139bae7
https://docs.saltproject.io/en/latest/topics/development/modules/ssh_wrapper.html
Comment by Nigel Kukard (nkukard) - Friday, 25 June 2021, 02:15 GMT
The easiest fix is probably the change for prepare().

This removes contextvars from requires.txt and results in salt not trying to find it in sites-packages, contextvars is already present in python 3.9 as loqs said.

I"m +1 for the removal of contextvars in prepare(), nice easy fix.
Comment by Eli Schwartz (eschwartz) - Friday, 25 June 2021, 02:53 GMT
The fallout would be, if you use salt to ssh into a machine running older versions of python then those salt minions will be broken because they are using python 3.6 or something and thus don't have this import.

Apparently if it's a site-packages dependency, then salt will include it in the code sent over the network to run remotely.
Comment by Imran Iqbal (myii) - Friday, 25 June 2021, 18:02 GMT
Apologies, I missed some of the first comments because my e-mail notifications got disturbed at the time.

I'd suggest removing `contextvars`. I'm told that this patch is likely to be proposed to the main Salt repo soon:

```
diff -Nar -u0 a/requirements/base.txt b/requirements/base.txt
--- a/requirements/base.txt 2021-06-09 14:25:03.000000000 +0000
+++ b/requirements/base.txt 2021-06-23 18:36:51.108332870 +0000
@@ -7 +6,0 @@
-contextvars
```

Which refers to this file: https://github.com/saltstack/salt/blob/master/requirements/base.txt.

I was also told:

> you can apply that patch to anything py3.7 or newer, it just won't work with salt-ssh targets with less than 3.7.

As has already been mentioned above in other comments.

Loading...