FS#54706 - [julia] Uses single-threaded BLAS instead of multi-threaded OpenBLAS

Attached to Project: Community Packages
Opened by Ted Krovetz (krovetz) - Wednesday, 05 July 2017, 14:36 GMT
Last edited by Alexander F. Rødseth (xyproto) - Wednesday, 15 November 2017, 11:27 GMT
Task Type Feature Request
Category Packages
Status Closed
Assigned To Alexander F. Rødseth (xyproto)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 3
Private No

Details

Description:

Julia is designed to be a high-performance scientific programming language. One of the ways it achieves high-performance is through the use of high-performance libraries. Currently Julia in Arch is using the "blas" package for linear algebra operations, which is single threaded. When asked, a "Steward" at JuliaLang.org recommended using OpenBLAS instead, which is used by them in their builds and provides multi-threaded linear-algebra subroutines.

https://discourse.julialang.org/t/multithreaded-libraries/
This task depends upon

Closed by  Alexander F. Rødseth (xyproto)
Wednesday, 15 November 2017, 11:27 GMT
Reason for closing:  Fixed
Comment by Antonio Rojas (arojas) - Wednesday, 05 July 2017, 14:55 GMT
openblas, atlas-lapack and other blas implementations are available in AUR. You can install any of them as a drop-in replacement for blas/lapack, no changes are needed to the Julia package
Comment by Ted Krovetz (krovetz) - Thursday, 06 July 2017, 15:47 GMT
The people over at julialang.org believe that simply substituting the openblas-lapack package for the blas package isn't a solution: "The parts that are unlikely to work immediately are the blas-vendor dependent functions, primarily changing the number of threads from the julia api."

Ultimately, the best way forward is to promote openblas-lapack out of AUR and use it, otherwise, julia will be either slow or broken on Arch.

See https://discourse.julialang.org/t/multithreaded-libraries/ for more information.
Comment by Alexander F. Rødseth (xyproto) - Monday, 10 July 2017, 16:14 GMT
julia is currently an orphan. Assigning to last packager. Will look into openblas-lapack and see if it's feasible to move it.
Comment by Antonio Rojas (arojas) - Monday, 10 July 2017, 16:21 GMT
I've already given my opinion about this. IMO the binary repos are not the place for optimized blas implementations - that defeats their purpose, which is being optimized for the user's CPU. There are already some alternative julia packages in AUR with additional options enabled, nothing prevents anybody from uploading a julia-openblas package.
Comment by Milan Bouchet-Valat (nalimilan) - Monday, 10 July 2017, 17:01 GMT
I don't really understand that position. OpenBLAS comes with multiple CPU-specific kernels which are loaded at runtime depending on the machine's CPU. For example, Fedora ships OpenBLAS in the official repositories and it works very well.

Moreover, Julia upstream recommends OpenBLAS. If OpenBLAS cannot be included in default repos, the package would better be moved to AUR so that users do not incorrectly believe that Julia is slow. This can really hurt Julia's reputation.
Comment by Oliver Blanthorn (bovine3dom) - Thursday, 27 July 2017, 20:40 GMT
> so that users do not incorrectly believe that Julia is slow. This can really hurt Julia's reputation.

For what it's worth, I just spent quite a few hours trying to understand why Julia was so much slower (~50x in some cases) on all my Linux machines than on my Windows machines. I finally figured it out by comparing which BLAS libraries were being used on the machines, using a pretty arcane command from within Julia.

Could openblas be added as a recommended optional package? Or some sort of helpful warning to users post install?
Comment by Colin Caine (colin) - Thursday, 31 August 2017, 16:17 GMT
@trontonic Did you look into moving openblas-lapack or is this indefinitely postponed in line with @arojas's comments?
Comment by Alexander F. Rødseth (xyproto) - Saturday, 02 September 2017, 19:41 GMT
Adopted and updated the Julia package to use openblas that comes with the Julia source tarball.

Please re-open if there are further issues with this.

Loading...