FS#67816 - [python-pytorch-cuda] SSE4 instructions are strictly required to use the package

Attached to Project: Community Packages
Opened by Maxim Terpilowski (maximtrp) - Thursday, 03 September 2020, 08:29 GMT
Last edited by Sven-Hendrik Haase (Svenstaro) - Thursday, 22 October 2020, 18:49 GMT
Task Type Bug Report
Category Packages
Status Closed
Assigned To Sven-Hendrik Haase (Svenstaro)
Konstantin Gizdov (kgizdov)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description:
Currently, the non-optimized packages "python-pytorch" and "python-pytorch-cuda" require AVX/SSE4 instructions. Thus, they cannot be used with old CPUs (unlike the non-optimized "python-tensorflow" and "python-tensorflow-cuda" packages). Even the simplest Python code results in error:
```
import torch
torch.cuda.is_available()
```

Additional info:
* package version: 1.6.0-2

Possible solution:
Disable AVX/SSE4 instructions to improve compatibility. There is an optimized package "python-pytorch-opt" that can be installed by users to take advantage of AVX/SSE4 instructions.

Instructions can be disabled with the following compile flags:
DISABLE_AVX2
DISABLE_AVX512F
DISABLE_FMA4
DISABLE_SSE4
This task depends upon

Closed by  Sven-Hendrik Haase (Svenstaro)
Thursday, 22 October 2020, 18:49 GMT
Reason for closing:  Upstream
Additional comments about closing:  Fixed upstream and next release (soon) should have this included.
Comment by Sven-Hendrik Haase (Svenstaro) - Monday, 07 September 2020, 03:20 GMT
I added a patch and some other stuff to make sure to force x86_64. Can you try 1.6.0-3?
Comment by Maxim Terpilowski (maximtrp) - Monday, 07 September 2020, 18:21 GMT
Thank you!
I have tried 1.6.0-4, and it is again exiting with an error.

Here is the log from gdb python -c "import torch":
https://pastebin.com/RuXGLdLE

Also, a line from dmesg:
[ 837.474115] traps: ipython[755] trap invalid opcode ip:7f8cbb16dac4 sp:7ffcf3b8b750 error:0 in libtorch_cpu.so[7f8cba7af000+5119000]

Output of cat /proc/cpuinfo:
https://pastebin.com/EgPSEGS0

cmake log on my target machine:
https://pastebin.com/knmdYRcA

cmake error log:
https://pastebin.com/gSwbEtUd
Comment by Maxim Terpilowski (maximtrp) - Wednesday, 09 September 2020, 07:31 GMT
Tried to build python-pytorch myself using the source files (https://github.com/archlinux/svntogit-community/tree/packages/python-pytorch/trunk).
Got this error: https://dpaste.com/8U4UG9UU6

Is it a pytorch bug? It seems package building was not thoroughly tested on a non-SSE4 CPU. Should I report it in their github repo?
Comment by Maxim Terpilowski (maximtrp) - Wednesday, 09 September 2020, 07:32 GMT
I have also noticed two flags that may need to be disabled:
-DUSE_AVX
-DUSE_AVX2
Comment by Sven-Hendrik Haase (Svenstaro) - Wednesday, 09 September 2020, 08:36 GMT
Oof well can you somehow try to trick the build into submission? You get much more immediate feedback than I do. Perhaps also report this upstream as an issue because frankly their build is broken at this point if it incorrectly detects your processor.
Comment by Maxim Terpilowski (maximtrp) - Thursday, 22 October 2020, 18:28 GMT
I have opened an issue in PyTorch GitHub repo: https://github.com/pytorch/pytorch/issues/45894. It should be fixed in the next version, I guess.

I have noticed one more thing: currently, PKGBUILD file (https://github.com/archlinux/svntogit-community/blob/packages/python-pytorch/trunk/PKGBUILD) does not include 3.7 and 5.0 compute capabilities, though they are just deprecated (still not discontinued):
```
export TORCH_CUDA_ARCH_LIST="5.2;5.3;6.0;6.0+PTX;6.1;6.1+PTX;6.2;6.2+PTX;7.0;7.0+PTX;7.2;7.2+PTX;7.5;7.5+PTX;8.0"
```

PyTorch PyPi wheels are built with support of 3.7 and 5.0 (building script: https://github.com/pytorch/builder/blob/master/manywheel/build.sh), so I think it would be more consistent to include these CUDA ARCHs, too:
```
export TORCH_CUDA_ARCH_LIST="3.7;5.0;5.2;5.3;6.0;6.0+PTX;6.1;6.1+PTX;6.2;6.2+PTX;7.0;7.0+PTX;7.2;7.2+PTX;7.5;7.5+PTX;8.0"
```
Comment by Sven-Hendrik Haase (Svenstaro) - Thursday, 22 October 2020, 18:48 GMT
Thanks for following up on this. I'll close this as upstream then. If it re-occurs in the next release, reopen this.

As for the arch's, if you feel strongly about this, open an issue. However, I'd prefer not to build deprecated archs as it just bloats the package and well, they are deprecated.

Loading...