FS#40718 - [gprename] does not work with new perl 5.20

Attached to Project: Community Packages
Opened by Stefan Husmann (stefanhusmann) - Thursday, 05 June 2014, 19:23 GMT
Last edited by Florian Pritz (bluewind) - Monday, 22 September 2014, 10:39 GMT
Task Type Bug Report
Category Packages
Status Closed
Assigned To Jaroslav Lichtblau (Dragonlord)
Florian Pritz (bluewind)
Architecture All
Severity High
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 4
Private No

Details

Description: Running gprename gives

Invalid version format (non-numeric data) at /usr/lib/perl5/core_perl/Encode.pm line 12.
BEGIN failed--compilation aborted at /usr/lib/perl5/core_perl/Encode.pm line 12.
Compilation failed in require at /usr/bin/gprename line 52.
BEGIN failed--compilation aborted at /usr/bin/gprename line 52.

Additional info:
* package version(s) perl 5.20.0-2, gprename 2.7-3


* config and/or log files etc.
A simple rebuild of gprename did not change anything here.

Steps to reproduce: Run gprename.
This task depends upon

Closed by  Florian Pritz (bluewind)
Monday, 22 September 2014, 10:39 GMT
Reason for closing:  Fixed
Additional comments about closing:  perl 5.20.1-1
Comment by Florian Pritz (bluewind) - Saturday, 07 June 2014, 08:06 GMT
Upstream issue: http://search.cpan.org/~rjbs/perl-5.20.0/pod/perldelta.pod#Locale_decimal_point_character_no_longer_leaks_outside_of_use%C2%A0locale_scope

Edit: Actually it's an issue in Encode which is part of perl core. Funny. Adding use locale; there fixes it.

Edit: Strangely enough `LC_ALL=de_DE.utf8 perl -e 'use Encode qw(decode encode);'` can't reproduce the issue. Not sure what's going on.
Comment by Michael Kogan (Photon) - Sunday, 15 June 2014, 15:41 GMT
It's not related to gprename, happens also in Shutter (Copy.pm) and Gmusicbrowser (Encode.pm). Maybe the misleading title can be edited to reflect this.
Comment by Michael Kogan (Photon) - Sunday, 15 June 2014, 16:17 GMT
Bug reports for Copy.pm and Encode.pm sent upstream.

edit: Bug reports:

https://rt.cpan.org/Public/Bug/Display.html?id=96467 for Encode.pm
https://rt.perl.org/Public/Bug/Display.html?id=122105 for Copy.pm

Related forum thread: https://bbs.archlinux.org/viewtopic.php?pid=1422917#p1422917
Comment by Michael Kogan (Photon) - Monday, 16 June 2014, 19:52 GMT
As discussed in the comments to the second bug above (especially https://rt.perl.org/Public/Bug/Display.html?id=122105#txn-1297477 and below) there is a patch that needs testing by Arch users. If anybody has a playground Arch install in a Virtual Box already, please have a look. I won't be able to do any testing till weekend at earliest.
Comment by Stefan Husmann (stefanhusmann) - Tuesday, 17 June 2014, 05:26 GMT
I did not manage to apply the patch to the perl code, but it is simply adding a line to vutil.c, and I used sed for doing so. That way, at least for gprename the issue went away.
   PKGBUILD (7.6 KiB)
Comment by Michael Kogan (Photon) - Tuesday, 17 June 2014, 06:25 GMT
Stefan, I tried to build your PKGBUILD, however khw.patch is not available in the official package, is it your patch? With this patch removed, it fails after a pretty long time with:

../ext/Pod-Html/t/cache.t (Wstat: 256 Tests: 10 Failed: 1)
Failed test: 8
Non-zero exit status: 1
Files=2405, Tests=721294, 1000 wallclock secs (79.16 usr 7.69 sys + 586.07 cusr 47.23 csys = 720.15 CPU)
Result: FAIL
makefile:839: recipe for target 'test_harness' failed
make: *** [test_harness] Error 90
Comment by Stefan Husmann (stefanhusmann) - Wednesday, 18 June 2014, 23:51 GMT
Sorry, khw.patch is not needed. I missed to remove it from the PKGBUILD. I found it in the link you posted above. And you need to do

makepkg --nocheck

because the check function fails.
Comment by Justin Davis (juster) - Thursday, 19 June 2014, 00:01 GMT
The earlier suspected causes of this bug are not the true causes. The cause is actually deeper in the perl core, inside of the "version" module, which is integrated with the code for perl and not just a core module.

The bug we are seeing was supposedly fixed by an earlier version of the "version" CPAN distribution. The error was introduced around version 0.9904 of "version" on 09/02/13 and was supposedly fixed in version 0.00907 on 01/15/14. The version module also exists in the perl core, where the C code is copied into vutil.c so it is even more closely integrated than most core modules. A similar bug was reported and supposedly fixed. I am trying to understand exactly why it isn't working.
https://rt.cpan.org/Public/Bug/Display.html?id=91987

If you use the previous patch, please be aware that it reintroduces a bug similar to one that was fixed in 2011.
https://rt.perl.org/Ticket/Display.html?id=105784

Comment by Florian Pritz (bluewind) - Thursday, 19 June 2014, 08:36 GMT
I think this post by Karl explains the issue best: https://rt.perl.org/Ticket/Display.html?id=121930#txn-1296317

Also the bug can be reproduced by running `LC_ALL=de_DE.utf8 perl -e'use Gtk2 -init; use 5.8.0'`

If you use en_US locale or don't load gtk it will not crash. Best solution would probably be to take a look at gtk and have it set the locale in a way perl knows about. That's assuming it doesn't, but after having read that explanation (and the remaining thread which talks about gtk3 doing just that) I think that's probably the case.
Comment by Justin Davis (juster) - Thursday, 19 June 2014, 11:48 GMT
I've read that bug report, with the post from Karl, which is referenced by our forum thread. He claims that perl's version code in vutil.c cannot just use setlocale, instead of the perl-internal locale switching code because of performance issues. Later he mentions that Gtk2 is to blame and we should all rally against their locale changing.

The first claim is simply untrue as far as I can tell. Previous versions of vutil.c used setlocale prior to 1/15/14. See version of the "version" module prior to 0.9907. Here is a diff between 0.9906 and 0.9907. To see for yourself search for STORE_NUMERIC_LOCAL_SET_STANDARD in vutil.c:
https://metacpan.org/diff/file?target=JPEACOCK/version-0.9907/&source=JPEACOCK/version-0.9906/#vutil/vutil.c

The second claim seems to be based on the idea that perl checks the active locale. The locale exists in two senses, the one set internally by perl, by perl code for example which uses "use locale" and the locale currently set for the process by glibc's setlocale. Perl only checks to see if it changed the locale earlier, and doesn't actually call setlocale to retrieve the current active glibc locale. This is theoretically faster but seems more error prone.

Blaming Gtk2 seems counterproductive when reverting the "version" distribution will solve our immediate problem. The version parsing code that parses the "use 5.8.0" needs to revert to properly setting the numeric locale to C before parsing. Just to reiterate, this is exactly what it used to do. This is what it tries to do.

edit: I toned down the tone of the comment. I think because I just woke up I was a little grumpy.
Comment by Michael Kogan (Photon) - Thursday, 19 June 2014, 12:48 GMT
Whatever is the best solution, I think that it would be nice if Arch could use the patch linked above to provide a workaround for the time till the problem is solved in a good way. As far as I can see, right now for every Arch user all applications using perl and GTK are broken, if the system locale has a comma as decimal point, which I'd call an exceptional state. This should be dealt with as fast as possible, discussions on a proper solution can still follow later.
Comment by Justin Davis (juster) - Thursday, 19 June 2014, 12:59 GMT
If you use the above patch please be aware that it does not restore the locale properly, or do any error checking, etc. This was not intended as a long term solution. The patch was intended to diagnose the problem. I'm writing a patch right now that reverts version's vutil.c (in perl) like I suggested.
Comment by Justin Davis (juster) - Thursday, 19 June 2014, 15:01 GMT
Attached is the patch which is also available as a gist.
https://gist.github.com/juster/bfb1f50ab0c624c4e4fd

In order to call setlocale directly, to duplicate the bug, I created a rough XS module and simple test script. I didn't want to install Gtk2 solely for this. If anyone is in a similar situation they can use the following XS module. Be sure to read the README if you do.
https://github.com/juster/perl-setlocale

edit: By the way, the patch will cause the porting/customized test to fail in perl's test suite. According to the error message, this is because the SHA1 for vutil.c was changed by changing the file contents.
Comment by Florian Pritz (bluewind) - Thursday, 19 June 2014, 19:22 GMT
Thanks for the patch, I've applied it to perl 5.20.0-5 in [testing].
Comment by Michael Kogan (Photon) - Friday, 20 June 2014, 05:44 GMT
Thanks, 5.20.0-5 fixes the problem for me (in Shutter and Gmusicbrowser).
Comment by Stefan Husmann (stefanhusmann) - Saturday, 21 June 2014, 12:07 GMT
Thanks, gprename also works again with perl 5.20.0-5.
Comment by Tobias Veit (nicebloom) - Wednesday, 25 June 2014, 07:05 GMT
New perl also fixes issues with pacmanager from aur.
Comment by Stefan Husmann (stefanhusmann) - Friday, 01 August 2014, 22:53 GMT
I'd say, this bug can be closed then, couldn't it?
Comment by Stefan Husmann (stefanhusmann) - Saturday, 02 August 2014, 12:00 GMT
sorry, double post
Comment by Florian Pritz (bluewind) - Saturday, 02 August 2014, 19:56 GMT
I'll keep this open at least until perl 5.22 is released to remind me to look at this again.
Comment by Florian Pritz (bluewind) - Wednesday, 17 September 2014, 10:03 GMT
I've pushed perl 5.20.1-1 to [testing], looks like the patch isn't needed any more and at least gprename works for me. Please test and report any issues.

Loading...