Community Packages

Please read this before reporting a bug:
https://wiki.archlinux.org/title/Bug_reporting_guidelines

Do NOT report bugs when a package is just outdated, or it is in the AUR. Use the 'flag out of date' link on the package page, or the Mailing List.

REPEAT: Do NOT report bugs for outdated packages!
Tasklist

FS#73422 - [julia] Building against system LLVM causes issues with certain packages

Attached to Project: Community Packages
Opened by Paul Melis (paulmelis) - Wednesday, 19 January 2022, 20:34 GMT
Last edited by Antonio Rojas (arojas) - Thursday, 20 January 2022, 08:58 GMT
Task Type Bug Report
Category Packages
Status Closed
Assigned To No-one
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Description:

Julia packages can contain "artifacts", which are basically precompiled native shared libs. Popular packages, such as Flux, depend on a number of packages using artifacts, for example LLVM and CUDA. In case of the LLVM julia package, this indirectly links against libLLVM-12jl.so (through its used of LLVMExtra_jll), which isn't available in the Arch julia package. Hence, although it is possible to install, say, the Flux julia package, not all functionality is available. In fact, precompilation of several packages (LLVM, CUDA, GPUCompiler) clearly fails, which will cause other packages depending on them to fail in unexpected ways.

Additional info:
* julia 2:1.7.1-4

Steps to reproduce:

[code]
melis@juggle 21:17:/tmp/doh$ julia --project=.
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.7.1 (2021-12-22)
_/ |\__'_|_|_|\__'_| |
|__/ |

(doh) pkg> add LLVM
Updating registry at `~/.julia/registries/General.toml`
Resolving package versions...
Updating `/tmp/doh/Project.toml`
[929cbde3] + LLVM v4.7.1
Updating `/tmp/doh/Manifest.toml`
[fa961155] + CEnum v0.4.1
[692b3bcd] + JLLWrappers v1.4.0
[929cbde3] + LLVM v4.7.1
[21216c6a] + Preferences v1.2.3
[dad2f222] + LLVMExtra_jll v0.0.13+1
[0dad84c5] + ArgTools
[56f22d72] + Artifacts
[2a0f44e3] + Base64
[ade2ca70] + Dates
[f43a241f] + Downloads
[b77e0a4c] + InteractiveUtils
[b27032c2] + LibCURL
[76f85450] + LibGit2
[8f399da3] + Libdl
[56ddb016] + Logging
[d6f4376e] + Markdown
[ca575930] + NetworkOptions
[44cfe95a] + Pkg
[de0858da] + Printf
[3fa0cd96] + REPL
[9a3f8284] + Random
[ea8e919c] + SHA
[9e88b42a] + Serialization
[6462fe0b] + Sockets
[fa267f1f] + TOML
[a4e569a6] + Tar
[cf7118a7] + UUIDs
[4ec0a83e] + Unicode
[deac9b47] + LibCURL_jll
[29816b5a] + LibSSH2_jll
[c8ffd9c3] + MbedTLS_jll
[14a3606d] + MozillaCACerts_jll
[83775a58] + Zlib_jll
[8e850ede] + nghttp2_jll
[3f19e933] + p7zip_jll
Precompiling project...
✗ LLVM
0 dependencies successfully precompiled in 1 seconds (7 already precompiled)
1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package

(doh) pkg> precompile
Precompiling project...
✗ LLVM
0 dependencies successfully precompiled in 1 seconds (7 already precompiled)

ERROR: The following 1 direct dependency failed to precompile:

LLVM [929cbde3-209d-540e-8aea-75f648917ca0]

Failed to precompile LLVM [929cbde3-209d-540e-8aea-75f648917ca0] to /home/melis/.julia/compiled/v1.7/LLVM/jl_IpaLUW.
ERROR: LoadError: InitError: could not load library "/home/melis/.julia/artifacts/f1d0060dd1a7698898f46626e143e03a442af46f/lib/libLLVMExtra-12.so"
libLLVM-12jl.so: cannot open shared object file: No such file or directory
Stacktrace:
[1] macro expansion
@ ~/.julia/packages/JLLWrappers/RYztT/src/products/library_generators.jl:54 [inlined]
[2] __init__()
@ LLVMExtra_jll ~/.julia/packages/LLVMExtra_jll/pXQ5F/src/wrappers/x86_64-linux-gnu-cxx11-julia_version+1.7.0.jl:8
[3] top-level scope (repeats 2 times)
@ none:1
during initialization of module LLVMExtra_jll
in expression starting at /home/melis/.julia/packages/LLVM/vQ98J/src/LLVM.jl:1

(doh) pkg>
[/code]

Focusing on the LLVMExtra_jll julia package:

[code]
melis@juggle 21:33:/tmp/doh$ julia --project=.
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.7.1 (2021-12-22)
_/ |\__'_|_|_|\__'_| |
|__/ |

julia> using LLVMExtra_jll
│ Package LLVMExtra_jll not found, but a package named LLVMExtra_jll is available from a registry.
│ Install package?
│ (doh) pkg> add LLVMExtra_jll
└ (y/n) [y]: y
Updating registry at `~/.julia/registries/General.toml`
Resolving package versions...
Updating `/tmp/doh/Project.toml`
[dad2f222] + LLVMExtra_jll v0.0.13+1
No Changes to `/tmp/doh/Manifest.toml`
ERROR: InitError: could not load library "/home/melis/.julia/artifacts/f1d0060dd1a7698898f46626e143e03a442af46f/lib/libLLVMExtra-12.so"
libLLVM-12jl.so: cannot open shared object file: No such file or directory
Stacktrace:
[1] macro expansion
@ ~/.julia/packages/JLLWrappers/RYztT/src/products/library_generators.jl:54 [inlined]
[2] __init__()
@ LLVMExtra_jll ~/.julia/packages/LLVMExtra_jll/pXQ5F/src/wrappers/x86_64-linux-gnu-cxx11-julia_version+1.7.0.jl:8
during initialization of module LLVMExtra_jll
[/code]

Checking what's up with the reported shared lib:
[code]
melis@juggle 21:17:/tmp/doh$ ls -l /home/melis/.julia/artifacts/f1d0060dd1a7698898f46626e143e03a442af46f/lib/libLLVMExtra-12.so
-r-xr-xr-x 1 melis users 5626088 Jan 19 21:06 /home/melis/.julia/artifacts/f1d0060dd1a7698898f46626e143e03a442af46f/lib/libLLVMExtra-12.so

melis@juggle 21:17:/tmp/doh$ readelf -d /home/melis/.julia/artifacts/f1d0060dd1a7698898f46626e143e03a442af46f/lib/libLLVMExtra-12.so

Dynamic section at offset 0x557000 contains 30 entries:
Tag Type Name/Value
0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN]
0x0000000000000001 (NEEDED) Shared library: [libLLVM-12jl.so]
0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000e (SONAME) Library soname: [libLLVMExtra-12.so]
0x000000000000000c (INIT) 0x7000
0x000000000000000d (FINI) 0x11664
0x0000000000000019 (INIT_ARRAY) 0x15b50
0x000000000000001b (INIT_ARRAYSZ) 8 (bytes)
0x0000000000000004 (HASH) 0x19b08
0x000000006ffffef5 (GNU_HASH) 0x920
0x0000000000000005 (STRTAB) 0x1b000
0x0000000000000006 (SYMTAB) 0xbf8
0x000000000000000a (STRSZ) 10469 (bytes)
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000003 (PLTGOT) 0x16000
0x0000000000000002 (PLTRELSZ) 3240 (bytes)
0x0000000000000014 (PLTREL) RELA
0x0000000000000017 (JMPREL) 0x5640
0x0000000000000007 (RELA) 0x4fb0
0x0000000000000008 (RELASZ) 1680 (bytes)
0x0000000000000009 (RELAENT) 24 (bytes)
0x000000006ffffffb (FLAGS_1) Flags: NODELETE
0x000000006ffffffe (VERNEED) 0x4ee0
0x000000006fffffff (VERNEEDNUM) 4
0x000000006ffffff0 (VERSYM) 0x4cde
0x000000006ffffff9 (RELACOUNT) 25
0x0000000000000000 (NULL) 0x0
[/code]

The issue is that the libLLVM-12jl.so used by LLVMExtra_jll is not available. In the official Julia distribution it *is* included and is one of the libraries used by the `julia` binary.


This task depends upon

Closed by  Antonio Rojas (arojas)
Thursday, 20 January 2022, 08:58 GMT
Reason for closing:  Won't fix
Comment by Antonio Rojas (arojas) - Wednesday, 19 January 2022, 21:20 GMT
Yes, this is a known side effect of building against system libs. But building the package using bundled libs would be pointless as it would make the package the same as the official binary. The whole point of packaging julia on the Arch binary repos is to take advantage of system libraries. Please use the julia-bin AUR package instead if you need packages that rely on using bundled libs.
Comment by Paul Melis (paulmelis) - Thursday, 20 January 2022, 08:46 GMT
Strange situation then. It would be good to show a note during 'pacman -S julia' that the Julia installation is effectively crippled and it's best to use either the AUR package or download the official binaries. Silently not being able to correctly use packages like Flux or CUDA is just mind-boggling.
Comment by Antonio Rojas (arojas) - Thursday, 20 January 2022, 08:58 GMT
It is already noted on the wiki https://wiki.archlinux.org/title/Julia, feel free to expand if you think it's needed.

Loading...