FS#58881 - [boost] numpy shipped with boost produce python modules with apparently wrong endianness

Attached to Project: Arch Linux
Opened by luca penasa (luke_penn) - Tuesday, 05 June 2018, 15:23 GMT
Last edited by Evangelos Foutras (foutrelis) - Thursday, 12 July 2018, 00:08 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To Levente Polyak (anthraxx)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 3
Private No

Details

Description:
This is to report the same problem originally reported here https://github.com/boostorg/python/issues/209
The same problem affect e.g. ubuntu the same way as arch (tested in a container), thus it might be related with the boost library itself

Additional info:
* package version(s) ANY I've been tried with many different versions of boost compiling them with makepkg. all of them suffer of the same problem

when compiling a python module calling np::initialize(); the following error is obtained:

RuntimeError: FATAL: module compiled as little endian, but detected different endianness at runtime
ImportError: numpy.core.umath failed to import
Traceback (most recent call last):
File "<string>", line 1, in <module>
SystemError: initialization of hello raised unreported exception

In [2]:

Steps to reproduce:
this can be reproduced following these steps (dev tools, boost, cmake and git must be installed):

git clone https://github.com/luca-penasa/boost-numpy-minimal-bug.git bnmb.git
cd bnmb.git
mkdir build
cd build
cmake ..
make

python3 -c "import hello"

I can confirm this code produce a valid result when linking directly to boost python and boost numpy compiled from scratch from https://github.com/boostorg/python

I can provide steps to reproduce this in a docker if needed.
This task depends upon

Closed by  Evangelos Foutras (foutrelis)
Thursday, 12 July 2018, 00:08 GMT
Reason for closing:  Fixed
Additional comments about closing:  Fixed upstream and the patch has been added to boost 1.67.0-5.
Comment by Doug Newgard (Scimmia) - Wednesday, 06 June 2018, 15:17 GMT
It was built on an x86_64 machine for x86_64 machines. If it somehow built things with the wrong endianness, there's a problem with the build system.

Edit: or actually reading better, there's a problem with their detection. x86_64 is little endian, so unless you're on a different architecture...
Comment by luca penasa (luke_penn) - Wednesday, 06 June 2018, 15:22 GMT
hi Doug, yes I guess they have some problems with the detection...
have you tries the same code? can you confirm this is not only my problem?

I'll try to report to boost if that is the case.

Thanks
luca
Comment by Evangelos Foutras (foutrelis) - Thursday, 07 June 2018, 01:08 GMT
Can confirm it's an issue on Arch. Maybe it has to do with how we build boost_python{,3} and boost_numpy{,3}.

Regarding reproducing the same issue on Ubuntu [1], I believe it would have worked after installing python3-numpy. (So it's not the same problem.)

[1] https://github.com/boostorg/python/issues/209#issuecomment-394721776
Comment by Filip Matzner (floop) - Wednesday, 13 June 2018, 09:05 GMT
I can confirm the issue. Downgrading boost and boost-libs to 1.66.0-1 (and its dependency icu to 60.2-1) from the archives fixes the issue for me (but naturally breaks many other things).
Comment by Evangelos Foutras (foutrelis) - Wednesday, 13 June 2018, 09:24 GMT
Interesting, 1.66.0-2 works but 1.66.0-3 doesn't. The latter is the first boost package compiled with GCC 8.

To be investigated further...
Comment by Filip Matzner (floop) - Wednesday, 20 June 2018, 13:53 GMT
Hi, is there any progress on this? Is there anything I can do to help? I confirm the 1.66.0-2 is the last working version. For those who suffer from this, this is a workaround:
pacman -U https://archive.archlinux.org/packages/b/boost/boost-1.66.0-2-x86_64.pkg.tar.xz https://archive.archlinux.org/packages/b/boost-libs/boost-libs-1.66.0-2-x86_64.pkg.tar.xz
Comment by Doug Newgard (Scimmia) - Wednesday, 20 June 2018, 13:58 GMT
Which will break everything on your system that uses boost. Don't do that.
Comment by Petr Bělohlávek (petrbel) - Monday, 09 July 2018, 10:20 GMT
Same issue on my desktop. Any progress with the issue? How can I help? Downgrading/upgrading boost all the time is so annoying. Thanks!

Loading...