FS#69829 - [cuda] Automatic library version detect.

Attached to Project: Community Packages
Opened by bartus (bartus) - Sunday, 28 February 2021, 17:29 GMT
Last edited by Sven-Hendrik Haase (Svenstaro) - Wednesday, 03 March 2021, 12:18 GMT
Task Type Bug Report
Category Packages
Status Closed
Assigned To Sven-Hendrik Haase (Svenstaro)
Felix Yan (felixonmars)
Konstantin Gizdov (kgizdov)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Is there any limitation of `makepkg` preventing use of automatic version for `libcu{blas,solver,parser},libcudart`.

https://git.archlinux.org/pacman.git/tree/scripts/makepkg.sh.in#n524

Current scheme of using `$pkgver` prevents PKGBUILDS depending on `libcudart` to pin cuda libary version, as probed one differs from `$pkgver` of cuda package.

Example:

$expac -S "%P" cuda|grep -oP "libcudart.*?\s"

libcudart.so=11.2.1

$expac -S "%D" popsift-libs|grep -oP "libcudart.*?$"

libcudart.so=11.0-64

popsift-libs: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=popsift#n63


This task depends upon

Closed by  Sven-Hendrik Haase (Svenstaro)
Wednesday, 03 March 2021, 12:18 GMT
Reason for closing:  Fixed
Comment by bartus (bartus) - Sunday, 28 February 2021, 17:33 GMT
Without automatic library version detection we're back at square on in task #69629 having to use `expac` to probe lib version...
Comment by Sven-Hendrik Haase (Svenstaro) - Sunday, 28 February 2021, 23:15 GMT
I don't think it's possible as makepkg will only check the default locations for libs and cuda lives in a non-default location by design.
Comment by bartus (bartus) - Monday, 01 March 2021, 07:07 GMT
@Svenstro: Think you could be wrong here, I've tested it with `extra-x86_64-build -c -r /tmp/cuda` and it works just fine.

Dropped all `=$pkgver` from `provides array:
```
tar xf cuda-11.2.1-2-x86_64.pkg.tar.zst .PKGINFO|grep provides.*lib .PKGINFO
provides = libcudart.so=11.0-64
provides = libcublas.so=11-64
provides = libcublas.so=11-64
provides = libcusolver.so=11-64
provides = libcusolver.so=11-64
provides = libcusparse.so=11-64
provides = libcusparse.so=11-64
```

libs `provides` array should be moved inside `package_cuda()` as they are missing in `package_cuda-tools`, resulting in some warnings:

```
==> Creating package "cuda-tools"...
-> Generating .PKGINFO file...
==> WARNING: Cannot find library listed in 'provides': libcudart.so
==> WARNING: Cannot find library listed in 'provides': libcublas.so
==> WARNING: Cannot find library listed in 'provides': libcusolver.so
==> WARNING: Cannot find library listed in 'provides': libcusparse.so
```

Current PKGBUILD gave ambiguous result when installing `libcudart.so`
```
pacman -S libcudart.so
:: There are 2 providers available for libcudart.so:
:: Repository community
1) cuda 2) cuda-tools
```

Also there's some missing dependencies and a broken symlink to license.

```
$namcap cuda-11.2.1-2-x86_64.pkg.tar.zst|grep cuda\ E:
cuda E: Symlink (usr/share/licenses/cuda/EULA.pdf) points to non-existing /opt/cuda/doc/pdf/EULA.pdf
cuda E: Dependency glu detected and not included (libraries ['usr/lib/libGLU.so.1'] needed in files ['opt/cuda/extras/demo_suite/randomFog'])
cuda E: Dependency ncurses5-compat-libs detected and not included (libraries ['usr/lib/libncursesw.so.5'] needed in files ['opt/cuda/bin/cuda-gdb'])
cuda E: Dependency freeglut detected and not included (libraries ['usr/lib/libglut.so.3'] needed in files ['opt/cuda/extras/demo_suite/randomFog']
```

Think `extras` and `samples` belongs to `cuda-tools` instead of core `cuda` package.

I've made a sample implementation of mentioned issues: http://ix.io/2RhQ/diff
Comment by Konstantin Gizdov (kgizdov) - Monday, 01 March 2021, 11:35 GMT
I've had a look at your PKGBUILD.diff and have implemented most of the things. Thanks for the suggestions. However, we cannot move the extras to cuda-tools as they are required by libcupti.so, which is part of CUDA. Fixing that will necessitate the creation of another package, cuda-extras, that would sit in the middle of everything and that's just more trouble than it's worth. Please have a look at testing/cuda-11.2.1-4 and see if that addresses your issues.
Comment by Eli Schwartz (eschwartz) - Monday, 01 March 2021, 12:47 GMT
I don't understand the original motivation of this provides at all. If you only provide libfoo.sp=$pkgver then there is ZERO utility in it, since anyone depending on a specific $pkgver can just depend on "cuda=11.2.1" to precisely the same effect.

Regardless of whether makepkg *can* handle automatic version detection of libcudart.so, this is very much absolutely "automatic version detection or bust". makepkg DOES NOT ALLOW you to play hinky games with the "libfoo.so style provides" version value. Any and all hinky games will ALWAYS result in the "libfoo.so style depends" being calculated differently and producing unresolvable pacman dependencies that prevent the resulting package from being installed under ANY circumstances.

If you don't think it will work correctly, remove it entirely.

If you do think it will work correctly, *do it correctly*.
Comment by bartus (bartus) - Monday, 01 March 2021, 16:52 GMT
@kgizdov Works great, thanks (๑˘︶˘๑)

Loading...