FS#73002 - [python] broken subinterpreters

Attached to Project: Arch Linux
Opened by John (graysky) - Monday, 13 December 2021, 19:40 GMT
Last edited by Toolybird (Toolybird) - Tuesday, 30 May 2023, 08:06 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Felix Yan (felixonmars)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 17
Private No

Details

Seems as though cpython is broken when working with subinterpreters. The problematic change could be this[1] affecting import.c. Reverting this change and rebuilding python fixes the issues on my system with some scripts that import sqlite. Attached a revert patch but I am NOT expert with python by any means.

The bug is seen within several Kodi plugins including the YouTube[2] plugin and the IMDB Trailers plugin. Others in the bbs have reported similar issues affecting other python code[3].

Additional info:
* package version(s) 3.10.1-1

1. https://github.com/python/cpython/commit/d0d29655ffc43d426ad68542d8de8304f7f1346a
2. https://github.com/anxdpanic/plugin.video.youtube/issues/255
3. https://bbs.archlinux.org/viewtopic.php?id=272121
   0001-Revert-bpo-44050-Extensi... (3.5 KiB)
This task depends upon

Closed by  Toolybird (Toolybird)
Tuesday, 30 May 2023, 08:06 GMT
Reason for closing:  Fixed
Additional comments about closing:  python 3.10.4-2
Comment by Jonas Witschel (diabonas) - Monday, 13 December 2021, 19:53 GMT
Possibly related:  FS#72979 
Comment by Geert Hendrickx (ghen) - Tuesday, 14 December 2021, 07:40 GMT
Also  FS#73004 
Comment by John (graysky) - Tuesday, 14 December 2021, 10:58 GMT
More or less mirrored this report upstream: https://bugs.python.org/issue46070
Comment by Geert Hendrickx (ghen) - Tuesday, 14 December 2021, 11:28 GMT
I confirm this patch fixes my Kodi issue  FS#73004 .
Comment by John (graysky) - Thursday, 16 December 2021, 11:23 GMT
For those affected by this bug, you can downgrade to python-3.10.0-1 which is fine.
Comment by Peter Weber (hoschi) - Tuesday, 21 December 2021, 17:14 GMT
Ignore this post.
Comment by Aaron Barany (akb825) - Thursday, 30 December 2021, 23:00 GMT
Just to add another voice to this, Kodi is completely unusable with this bug as it crashes on startup. Based on other posts it seems to be plugins, but they are very common ones. Applying the patch resolved the crash.
Comment by Geert Hendrickx (ghen) - Wednesday, 05 January 2022, 18:19 GMT
I've been running with this patch for 3 weeks now with no issues. Can it be considered for inclusion in Arch's python package? (while the upstream bug report seems to be mixing up with another, unrelated issue)
Comment by Michael Fuchs (mukl) - Thursday, 06 January 2022, 22:44 GMT
The patch also works for me on Arch ARM on a raspberry pi 4 with this PKGBUILD: https://gist.github.com/fuchsmich/1ccf4773243cf8a4774ab483505dc99f

(couldnt get the check()s working).
Comment by John (graysky) - Friday, 07 January 2022, 09:54 GMT
Turns out that a simple downgrade to 3.0.0-2 does not fix the IMDB plugin but, as pointed out by idovitz[1], the attached patch "fixes" both Kodi's imdb and youtube plugins.

1. https://bbs.archlinux.org/viewtopic.php?pid=2012681#p2012681
   0001-comment-register_adapter... (0.9 KiB)
Comment by Geert Hendrickx (ghen) - Friday, 07 January 2022, 11:02 GMT
This looks like a completely different fix than the OP one, or is it essentially the same solution?
Comment by Geert Hendrickx (ghen) - Sunday, 16 January 2022, 14:53 GMT
python 3.10.2-1 seems to be working for me, strangely because it contains neither of the patches proposed here?
Comment by John (graysky) - Sunday, 16 January 2022, 16:19 GMT
python 3.10.2-1 is working for me as well under the same test cases that fail with 3.10.1-2. Perhaps something upstream fixed it despite the upstream report still being open? I honestly got lost trying to follow it.
Comment by loqs (loqs) - Sunday, 16 January 2022, 19:33 GMT
Could it have been fixed by [1]?

[1] https://github.com/python/cpython/commit/72c260cf0c71eb01eb13100b751e9d5007d00b70
Comment by Aaron Barany (akb825) - Friday, 13 May 2022, 20:46 GMT
This was fixed in 3.10.2 but broke again in 3.10.4, though the behavior was a little better. In the case of kodi, accessing addons like Twitch and YouTube would always crash before, but with the newer regression they crash the *second* time you open them.

See the following issues:
https://github.com/python/cpython/issues/92036
https://github.com/xbmc/xbmc/issues/21243

This is the fix on the 3.10 branch:
https://github.com/python/cpython/commit/178a238f25ab8aff7689d7a09d66dc1583ecd6cb

According to the Python release schedule, this should be available in version 3.10.5 in early June, though it would be nice to have the fix in sooner for the Arch package.

Note to anybody building Python from source to apply the patch: apparently there's a bug in the python-setuptools package in Arch (which is rather out of date) that causes building Python to fail if it's installed. To work around the issue, set the following environment variable before building: SETUPTOOLS_USE_DISTUTILS=stdlib (see https://github.com/pypa/setuptools/issues/3007)
Comment by John (graysky) - Friday, 13 May 2022, 21:09 GMT
@Aaron - Thanks for the detailed reply. I found the IMDb plugin for Kodi would crash as you are describing using python-3.10.4-1; downgrading to python-3.10.2-1 fixed it.

I can confirm that applying the commit you mentioned fixes the problem. I agree that it would be nice to have this applied to our 3.10.4 package.
Comment by Felix Yan (felixonmars) - Saturday, 14 May 2022, 08:34 GMT
Patch applied in [testing]/python 3.10.4-2.

Loading...