Please read this before reporting a bug:
https://wiki.archlinux.org/title/Bug_reporting_guidelines
Do NOT report bugs when a package is just outdated, or it is in the AUR. Use the 'flag out of date' link on the package page, or the Mailing List.
REPEAT: Do NOT report bugs for outdated packages!
https://wiki.archlinux.org/title/Bug_reporting_guidelines
Do NOT report bugs when a package is just outdated, or it is in the AUR. Use the 'flag out of date' link on the package page, or the Mailing List.
REPEAT: Do NOT report bugs for outdated packages!
FS#60227 - [julia] Build configuration breaks GPU packages
Attached to Project:
Community Packages
Opened by Valentin Churavy (wallnuss) - Friday, 28 September 2018, 03:29 GMT
Last edited by Alexander F. Rødseth (xyproto) - Thursday, 27 December 2018, 20:53 GMT
Opened by Valentin Churavy (wallnuss) - Friday, 28 September 2018, 03:29 GMT
Last edited by Alexander F. Rødseth (xyproto) - Thursday, 27 December 2018, 20:53 GMT
|
DetailsDescription:
As noted in https://bugs.archlinux.org/task/57387 Julia depends on a custom-build LLVM, since we have to carry bug-fixes that are discovered after a LLVM release has been made. We work towards upstreaming those as fast and often as we can, but LLVM development is fast and almost with every upgrade we encounter new bugs. As the testing done on our LLVM configuration is different from the general LLVM release I would caution against carrying our patches for a general purpose LLVM package. The current set of patches against 6.0.1 can be found here https://github.com/staticfloat/LLVMBuilder/tree/master/patches/llvm_patches which is a list I maintain. Currently the PKGBUILD for Julia disables building the custom Julia in a shared-library configuration. `override USE_LLVM_SHLIB = 0`, which will statically link Julia against LLVM, but breaks the GPU ecosystem in Julia since that has to load LLVM.so through https://github.com/maleadt/LLVM.jl. There have been issues in the past with multiple version of LLVM being loaded into the system, but since then we have started to use symbol versioning to namespace the custom LLVM Julia requires so that is doesn't interfere with an already loaded LLVM. So hopefully shipping a custom LLVM shared library should NOT cause any problems. Additional info: * Affects Julia 1.0.0 Steps to reproduce: julia> import Pkg; Pkg.add("LLVM"); Pkg.test("LLVM") |
This task depends upon
Closed by Alexander F. Rødseth (xyproto)
Thursday, 27 December 2018, 20:53 GMT
Reason for closing: Fixed
Thursday, 27 December 2018, 20:53 GMT
Reason for closing: Fixed
And building this statically is a supported, expected configuration according to upstream makefiles, so maybe this should be clarified there first. I'm not sure I see the logic in bloating the package size *even more*, by defeating the purpose of shared libraries by shipping libraries in the shared library *fileformat* (which is larger than statically linking what you need) but not actually, like, *sharing* them. (Which we would love to do. By linking to the system libLLVM.so, which, by the way, is currently in the process of being upgraded to llvm 7.)
The expected, rational approach to handling this, by the way, is to explicitly export the symbols you want to provide
If julia *expects* to export llvm as *public ABI* to the package ecosystem then this needs to be clarified... major time. Kind of like the current mess with exporting libgfortran as *public ABI*, which apparently means yet more package ecosystem software is completely and utterly broken outside of cherry-picked prebuilt binaries.
...
I'm not understanding the problem anyway, since as far as I can tell the ecosystem package in question is perfectly happy with loading a second libLLVM.so and as you point out, the vendored one uses namespaced symbols.
> And building this statically is a supported, expected configuration according to upstream makefiles, so maybe this should be clarified there first.
Sure, it is a supported configuration as far as upstream Julia is concerned, but the LLVM.jl package (which is unrelated to Julia itself) has a different requirement. I guess that, since the (again, external) GPU stack relies on LLVM.jl, the build option could be removed, but that's besides the point of this discussion.
> I'm not understanding the problem anyway, since as far as I can tell the ecosystem package in question is perfectly happy with loading a second libLLVM.so and as you point out, the vendored one uses namespaced symbols.
LLVM.jl is used to interface with the LLVM library as loaded by Julia, to e.g. replace parts of the compiler for the purpose of emitting GPU compatible code. That means we are using global state of the library (LLVM context, uniqued types, etc), and cannot use a second copy for that.
The situation might improve in the future, and there's a couple of paths forward, but as it stands right now the USE_LLVM_SHLIB=0 build option breaks use of LLVM.jl and consequently CUDAnative.jl/CuArrays.jl/...