FS#78571 - [python-urwid] using git source caused unintended breaking changes

Attached to Project: Arch Linux
Opened by Levi Zim (kxxt) - Monday, 22 May 2023, 15:45 GMT
Last edited by David Runge (dvzrv) - Sunday, 03 September 2023, 12:56 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To David Runge (dvzrv)
Levente Polyak (anthraxx)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Description:

The PKGBUILD is using git repo source at commit abc098c3400220b51aeb6fec14175e1bfa1f8bfb, which contains breaking changes that makes it incompatible with upstream version 2.1.2.

The breaking change is https://github.com/urwid/urwid/commit/d1710f0983e86dc5ec06efec6e94f3b6d204bbb3

To see how this would break packages in arch linux repo, build todoman with extra-x86_64-build yields the following error in check():

=================================== FAILURES ===================================
______________________________ test_ctrl_c_clears ______________________________

default_formatter = <todoman.formatters.DefaultFormatter object at 0x7f414c3b3210>
todo_factory = <function todo_factory.<locals>.inner at 0x7f414c372a20>

def test_ctrl_c_clears(default_formatter, todo_factory):
todo = todo_factory()
editor = TodoEditor(todo, [todo.list], default_formatter)

# Simulate that ctrl+c gets pressed, since we can't *really* do that
# trivially inside unit tests.
> with mock.patch(
"urwid.main_loop.MainLoop.run", side_effect=KeyboardInterrupt
), mock.patch(
"urwid.main_loop.MainLoop.stop",
) as mocked_stop:

tests/test_ui.py:147:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3.11/unittest/mock.py:1421: in __enter__
self.target = self.getter()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

name = 'urwid.main_loop.MainLoop'

def resolve_name(name):
"""
Resolve a name to an object.

It is expected that `name` will be a string in one of the following
formats, where W is shorthand for a valid Python identifier and dot stands
for a literal period in these pseudo-regexes:

W(.W)*
W(.W)*:(W(.W)*)?

The first form is intended for backward compatibility only. It assumes that
some part of the dotted name is a package, and the rest is an object
somewhere within that package, possibly nested inside other objects.
Because the place where the package stops and the object hierarchy starts
can't be inferred by inspection, repeated attempts to import must be done
with this form.

In the second form, the caller makes the division point clear through the
provision of a single colon: the dotted name to the left of the colon is a
package to be imported, and the dotted name to the right is the object
hierarchy within that package. Only one import is needed in this form. If
it ends with the colon, then a module object is returned.

The function will return an object (which might be a module), or raise one
of the following exceptions:

ValueError - if `name` isn't in a recognised format
ImportError - if an import failed when it shouldn't have
AttributeError - if a failure occurred when traversing the object hierarchy
within the imported package to get to the desired object.
"""
global _NAME_PATTERN
if _NAME_PATTERN is None:
# Lazy import to speedup Python startup time
import re
dotted_words = r'(?!\d)(\w+)(\.(?!\d)(\w+))*'
_NAME_PATTERN = re.compile(f'^(?P<pkg>{dotted_words})'
f'(?P<cln>:(?P<obj>{dotted_words})?)?$',
re.UNICODE)

m = _NAME_PATTERN.match(name)
if not m:
raise ValueError(f'invalid format: {name!r}')
gd = m.groupdict()
if gd.get('cln'):
# there is a colon - a one-step import is all that's needed
mod = importlib.import_module(gd['pkg'])
parts = gd.get('obj')
parts = parts.split('.') if parts else []
else:
# no colon - have to iterate to find the package boundary
parts = name.split('.')
modname = parts.pop(0)
# first part *must* be a module/package.
mod = importlib.import_module(modname)
while parts:
p = parts[0]
s = f'{modname}.{p}'
try:
mod = importlib.import_module(s)
parts.pop(0)
modname = s
except ImportError:
break
# if we reach this point, mod is the module, already imported, and
# parts is the list of parts in the object hierarchy to be traversed, or
# an empty list if just the module is wanted.
result = mod
for p in parts:
> result = getattr(result, p)
E AttributeError: module 'urwid' has no attribute 'main_loop'

/usr/lib/python3.11/pkgutil.py:715: AttributeError
=============================== warnings summary ===============================
../../../../usr/lib/python3.11/site-packages/pkg_resources/__init__.py:121
/usr/lib/python3.11/site-packages/pkg_resources/__init__.py:121: DeprecationWarning: pkg_resources is deprecatedas an API
warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)

../../../../usr/lib/python3.11/site-packages/pkg_resources/__init__.py:2870
../../../../usr/lib/python3.11/site-packages/pkg_resources/__init__.py:2870
../../../../usr/lib/python3.11/site-packages/pkg_resources/__init__.py:2870
../../../../usr/lib/python3.11/site-packages/pkg_resources/__init__.py:2870
/usr/lib/python3.11/site-packages/pkg_resources/__init__.py:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('sphinxcontrib')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)

tests/test_cli.py: 10 warnings
tests/test_ui.py: 27 warnings
/usr/lib/python3.11/site-packages/urwid/wimp.py:267: PendingDeprecationWarning: only for backwards compatibility.You may also use the new standard container property `focus_position` to get the focus.
self._w.focus_col = 0

tests/test_cli.py: 7 warnings
tests/test_ui.py: 12 warnings
/usr/lib/python3.11/site-packages/urwid/signals.py:168: DeprecationWarning: Don't use user_arg argument, use user_args instead.
warnings.warn(

../../../../usr/lib/python3.11/site-packages/_pytest/cacheprovider.py:433
/usr/lib/python3.11/site-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: could not create cache path /dev/.pytest_cache/v/cache/nodeids
config.cache.set("cache/nodeids", sorted(self.cached_nodeids))

../../../../usr/lib/python3.11/site-packages/_pytest/cacheprovider.py:387
/usr/lib/python3.11/site-packages/_pytest/cacheprovider.py:387: PytestCacheWarning: could not create cache path /dev/.pytest_cache/v/cache/lastfailed
config.cache.set("cache/lastfailed", self.lastfailed)

../../../../usr/lib/python3.11/site-packages/_pytest/stepwise.py:56
/usr/lib/python3.11/site-packages/_pytest/stepwise.py:56: PytestCacheWarning: could not create cache path /dev/.pytest_cache/v/cache/stepwise
session.config.cache.set(STEPWISE_CACHE_DIR, [])

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED ../../../../dev/tests/test_ui.py::test_ctrl_c_clears - AttributeError:...
======= 1 failed, 190 passed, 5 skipped, 3 xfailed, 64 warnings in 5.41s =======
==> ERROR: A failure occurred in check().
This task depends upon

Closed by  David Runge (dvzrv)
Sunday, 03 September 2023, 12:56 GMT
Reason for closing:  Fixed
Additional comments about closing:  Fixed with python-urwid 2.1.2.r63.g3c21a42-1 and/or todoman 4.3.1-1
Comment by loqs (loqs) - Monday, 22 May 2023, 18:04 GMT Comment by David Runge (dvzrv) - Monday, 22 May 2023, 21:47 GMT
FWIW, the python-urwid upstream was *very* responsive and up for fixing issues caused for other projects.
We can probably leverage this current activity by reporting both to todoman (to let them know about potential breakage) and urwid (to get fixes for todoman).
Comment by loqs (loqs) - Monday, 22 May 2023, 21:58 GMT
The required change to fix the test appears very minimal. See attachment.
Comment by Levi Zim (kxxt) - Tuesday, 23 May 2023, 00:44 GMT
I understand that the required change to fix the test appears very minimal.
But from a user's perspective, we probably shouldn't let breaking changes happen to python-urwid.
Or at least a polyfill is needed to fix the breaking changes so that python-urwid 2.1.2.* stays compatible with upstream 2.1.2 tag.
Comment by Levi Zim (kxxt) - Tuesday, 23 May 2023, 01:45 GMT
> Have you contacted either upstream about the incompatibility?

No, since this is an issue about the package in the arch repo. Upstream didn't make a release for that breaking change.

> Or at least a polyfill is needed to fix the breaking changes so that python-urwid 2.1.2.* stays compatible with upstream 2.1.2 tag.

I attached a polyfill fix.
Comment by Levi Zim (kxxt) - Tuesday, 23 May 2023, 01:46 GMT
attachments.
Comment by David Runge (dvzrv) - Tuesday, 23 May 2023, 10:15 GMT
> No, since this is an issue about the package in the arch repo. Upstream didn't make a release for that breaking change.

They also didn't make a release for Python 3.11 (or for anything >= 3.9 for that matter), yet here we are three years later.

Will have a look at the attached fixes and bump to a later version later.

Loading...