FS#65540 - [julia] signal (11): Segmentation fault

Attached to Project: Community Packages
Opened by Zorn (zsoerenm) - Tuesday, 18 February 2020, 12:26 GMT
Last edited by Antonio Rojas (arojas) - Saturday, 18 April 2020, 20:25 GMT
Task Type Bug Report
Category Packages
Status Closed
Assigned To Jan de Groot (JGC)
Lukas Fleischer (lfleischer)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 9
Private No

Details

Description:

If you update packages through the julia package manager, you'll get a segmentation fault:

```
(v1.3) pkg> up
Updating registry at `~/.julia/registries/General`
Updating git-repo `https://github.com/JuliaRegistries/General.git`

signal (11): Segmentation fault
in expression starting at none:0
unknown function (ip: 0x7f0301d7be73)
http_parser_execute at /usr/bin/../lib/libhttp_parser.so.2.9 (unknown line)
unknown function (ip: 0x7f0301d7d390)
unknown function (ip: 0x7f0301d7f770)
unknown function (ip: 0x7f0301d81b58)
unknown function (ip: 0x7f0301d7ff91)
git_remote_fetch at /usr/bin/../lib/libgit2.so (unknown line)
unknown function (ip: 0x7f0340bdacb2)
#fetch at ./none:0 [inlined]
#fetch#115 at /build/julia/src/julia-1.3.1/usr/share/julia/stdlib/v1.3/LibGit2/src/LibGit2.jl:290
#fetch at ./none:0
#fetch#5 at /build/julia/src/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/GitTools.jl:156
```


Additional info:
* package version: v1.3.1
* config and/or log files etc.
* It is also mentioned here: https://discourse.julialang.org/t/pkg-crashes/34750/4

Steps to reproduce:

Go to the package manager with ]
and type `up` to update.
This task depends upon

Closed by  Antonio Rojas (arojas)
Saturday, 18 April 2020, 20:25 GMT
Reason for closing:  Fixed
Additional comments about closing:  julia 1.4.1-2
Comment by Eli Schwartz (eschwartz) - Tuesday, 18 February 2020, 14:05 GMT
In the duplicate  FS#65542  it is mentioned that the crash was bisected to an update of:

http-parser 2.9.2 -> 2.9.3
Comment by Jonas Witschel (diabonas) - Tuesday, 18 February 2020, 18:10 GMT
It's probably enough to rebuild libgit2 against the new http-parser version, at least that was the case for community/tang: version 2.9.3 of http-parser broke ABI compatibility in commit https://github.com/nodejs/http-parser/commit/7d5c99d09f6743b055d53fc3f642746d9801479b by changing the http_parser struct.
Comment by Alexey Stukalov (alyst) - Wednesday, 19 February 2020, 12:49 GMT
Rebuilding julia-mkl fails for me during precompile statements generation. Here's the excerpt from the debug output:
```
...
julia> Pkg.add("Test") # adding an stdlib doesn't require internet access
Cloning default registries into `/tmp/jl_2HmbMp`
Cloning registry from "https://github.com/JuliaRegistries/General.git"

signal (11): Segmentation fault
in expression starting at REPL[19]:1
unknown function (ip: 0x7f8f80d00e73)
http_parser_execute at /usr/lib/libhttp_parser.so.2.9 (unknown line)
unknown function (ip: 0x7f8f80d02390)
unknown function (ip: 0x7f8f80d04770)
unknown function (ip: 0x7f8f80d06b58)
unknown function (ip: 0x7f8f80d04f91)
git_remote_fetch at /.../aur/julia-mkl/src/julia-1.3.1/usr/bin/../lib/julia/libgit2.so (unknown line)
git_clone at /.../aur/julia-mkl/src/julia-1.3.1/usr/bin/../lib/julia/libgit2.so (unknown line)

...

```
Comment by Jonas Witschel (diabonas) - Thursday, 20 February 2020, 09:03 GMT
Sorry, my comment above should have read "It's probably enough to rebuild libgit2" instead of "It's probably enough to rebuild julia" - julia doesn't directly depend on http-parser, but via libgit2.
Comment by Lukas Fleischer (lfleischer) - Thursday, 20 February 2020, 10:54 GMT
The libgit2 rebuild is currently in [staging].
Comment by Zorn (zsoerenm) - Wednesday, 04 March 2020, 07:27 GMT
With the newest libgit2 version installed I get the following error now:

```
(v1.3) pkg> up
Updating registry at `~/.julia/registries/General`
Updating git-repo `https://github.com/JuliaRegistries/General.git`
┌ Warning: Some registries failed to update:
│ — /home/zorn/.julia/registries/General — failed to fetch from repo
└ @ Pkg.Types /build/julia/src/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:1199
Updating git-repo `https://github.com/Evizero/UnicodePlots.jl.git`
ERROR: failed to fetch from https://github.com/Evizero/UnicodePlots.jl.git, error: GitError(Code:ERROR, Class:Invalid, invalid version 0 on git_proxy_options)

```
Comment by bob (getzze) - Wednesday, 04 March 2020, 12:46 GMT
Could be because some functions were renamed:
https://github.com/libgit2/libgit2/blob/master/docs/changelog.md

Maybe try to add these lines in prepare() PKGBUILD of julia:

cd stdlib/LibGit2/src
sed -i "s/:git_cred/:git_credential/g" callbacks.jl

and rebuild (it takes hours on my computer, so I could not try it yet).
Comment by Jakob Schöttl (nurfz) - Friday, 06 March 2020, 14:11 GMT
Works again for me, after glib was upgraded:

[2020-03-01T19:37:32+0100] [ALPM] upgraded glib2 (2.62.5-1 -> 2.64.0-1)
[2020-03-01T19:37:32+0100] [ALPM] upgraded glib2-docs (2.62.5-1 -> 2.64.0-1)
[2020-03-01T19:37:32+0100] [ALPM] upgraded lib32-glib2 (2.62.5-1 -> 2.64.0-1)
Comment by Zorn (zsoerenm) - Friday, 06 March 2020, 15:52 GMT
I can not confirm that. I have glib2 v2.64.0-1 installed but I get the error above (failed to fetch from repo).
Comment by Marcel Dijkstra (MDykstra) - Monday, 09 March 2020, 11:45 GMT
Similar to Zorn removing and adding `General` registry gives me `git_proxy_options` error:
```
pkg> repository remove General
pkg> repository add General
Cloning registry from "https://github.com/JuliaRegistries/General.git"
ERROR: failed to clone from https://github.com/JuliaRegistries/General.git, error: GitError(Code:ERROR, Class:Invalid, invalid version 0 on git_proxy_options)
```
Using: libgit2 1:0.99.0-2, (glib2 2.64.0-1), julia 2:1.3.1-1
The function `git_proxy_options` is part of libgit2.

Comment by Wil Thomason (wbthomason) - Thursday, 12 March 2020, 00:02 GMT
Apologies if this was already confirmed, but: I applied bob's suggested PKGBUILD edit and rebuilt Julia, but still encounter the error that Zorn and Marcel mention. My versions of Julia, glib2, lib32-glib2, and libgit2 all match those of Jakob and Marcel.

Incidentally, I also get a test failure if I allow check() to run (LinearAlgebra/lu reports a failure), but I don't think this is related.
Comment by Jonas Witschel (diabonas) - Thursday, 12 March 2020, 10:05 GMT
Julia appears to be incompatible with the new libgit2 0.99.0 for some reason. According to https://github.com/JuliaLang/julia/issues/33111#issuecomment-596537771 this should be fixed in the next version of Julia: "You can use the soon-to-be-released Julia 1.4 with JULIA_PKG_SERVER=pkg.julialang.org and it won't use libgit2 at all for installing registered packages."

Upstream only supports the version of libgit2 bundled with Julia, so an official patch for Julia 1.3 and libgit2 0.99.0 is unlikely: https://github.com/JuliaLang/julia/issues/35043
Comment by Bart Janssens (bart_janssens) - Sunday, 22 March 2020, 14:21 GMT
This still happens on Julia 1.4, using a clean depot and no startup file, after a complete system update:

(@v1.4) pkg> add Unitful
Cloning default registries into `~/tmp/cleandepot`
Cloning registry from "https://github.com/JuliaRegistries/General.git"
ERROR: failed to clone from https://github.com/JuliaRegistries/General.git, error: GitError(Code:ERROR, Class:Invalid, invalid version 0 on git_proxy_options)
Comment by Zorn (zsoerenm) - Monday, 23 March 2020, 08:15 GMT
I can confirm this. Julia v1.4 still throws GitError(Code:ERROR, Class:Invalid, invalid version 0 on git_proxy_options)
Comment by Jonas Witschel (diabonas) - Monday, 23 March 2020, 11:14 GMT
I found the issue in Julia and submitted a fix upstream: https://github.com/JuliaLang/julia/pull/35232 Rebuilding the Arch Linux package with the patch from this PR fixes the issue for me.

As a temporary workaround until the package is fixed, calling Julia as "JULIA_PKG_SERVER=pkg.julialang.org julia" (cf. https://bugs.archlinux.org/task/65540#comment187329) should allow you to download and update packages as well. This doesn't fix the LibGit2 issue, but avoids using this library for downloading packages altogether.
Comment by bob (getzze) - Monday, 23 March 2020, 11:20 GMT
With julia 1.4, calling"JULIA_PKG_SERVER=pkg.julialang.org julia" does not fix the issue for me, as stated in previous comments.
Can someone confirm that applying the patch solves the issue?
Comment by Jose Marino (oniram) - Tuesday, 24 March 2020, 16:05 GMT
@bob Try deleting "~/.julia/registries/General" and then run "JULIA_PKG_SERVER=pkg.julialang.org julia".

Quoted from the julia issue: "If you already have the registry as a git clone it will keep using it as a git clone instead of using the Pkg server, thats why you have to delete it."
Comment by bob (getzze) - Wednesday, 25 March 2020, 10:52 GMT
I have a lot of unregistered packages and development packages, these still cannot be updated using "JULIA_PKG_SERVER=pkg.julialang.org" and it makes the whole update fail, so it's unusable for me unfortunately.
I hope the patch can be incorporated.
Comment by Marcel Dijkstra (MDykstra) - Wednesday, 25 March 2020, 22:32 GMT
The package server `pkg.julialang.org` seems independent from the list of available packages.
The `General` registry (list of packages) is maintained on GitHub. So `JULIA_PKG_SERVER` does
not solve the bug, pulling registries from GitHub.
Comment by Marcel Dijkstra (MDykstra) - Friday, 17 April 2020, 14:43 GMT
This task is not fixed! I still get "get_proxy_options()" symbol missing in libGit; trying to add "General" registry to Julia. Optionally the package server can be changed with JULIA_PKG_SERVER. However Julia registries (indexing packages) are pulled from GitHub and require dependency on Git.
Comment by Jonas Witschel (diabonas) - Saturday, 18 April 2020, 14:37 GMT
Marcel, I cannot reproduce this with a fresh Julia installation (non-existent ~/.julia directory) when running the command "using Pkg; Pkg.update()". Which versions of Julia and libgit2 are you using (output of "pacman -Q julia libgit2"), and where exactly are you seeing this error? I don't think the the message you are describing could have been produced by Julia itself at all...
Comment by Marcel Dijkstra (MDykstra) - Saturday, 18 April 2020, 18:18 GMT
At "julia>" prompt press "]" to go into "@v1.4 pkg>" prompt, type "help".
Refresh the General repository; "pkg> registry remove General" and "pkg> registry add General"
Comment by Antonio Rojas (arojas) - Saturday, 18 April 2020, 18:47 GMT
Please post the output of "pacman -Q julia libgit2" as diabonas asked
Comment by Marcel Dijkstra (MDykstra) - Saturday, 18 April 2020, 20:22 GMT
Using julia 2:1.4.0-1, libgit2 1:0.99.0-3.
Comment by Antonio Rojas (arojas) - Saturday, 18 April 2020, 20:25 GMT
Please update your system before requesting to reopen bugs

Loading...