Arch Linux

Please read this before reporting a bug:
https://wiki.archlinux.org/title/Bug_reporting_guidelines

Do NOT report bugs when a package is just outdated, or it is in the AUR. Use the 'flag out of date' link on the package page, or the Mailing List.

REPEAT: Do NOT report bugs for outdated packages!
Tasklist

FS#78930 - [texlive-texmf]: makepkg --printsrcinfo output incomplete

Attached to Project: Arch Linux
Opened by Christian Cornelssen (ccorn) - Friday, 30 June 2023, 05:31 GMT
Last edited by Antonio Rojas (arojas) - Sunday, 16 July 2023, 13:53 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To Antonio Rojas (arojas)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description:

In the package directory of texlive-texmf,
`makepkg --printsrcinfo` outputs no attributes (pkgdesc, depends, optdepends, conflicts, provides, replaces, groups) for the collection-based subpackages.
That is bad for tools which use --printsrcinfo to extract such information.
I happen to use such a tool, which is why I am affected.

The reason is probably that makepkg uses `grep_function` for such info, which is a `declare -f` piped to a `grep`:

https://gitlab.archlinux.org/pacman/pacman/-/blob/master/scripts/libmakepkg/util/pkgbuild.sh.in#L33

The problem is NOT the `eval` in the PKGBUILD, but the fact that the variables of interest (pkgdesc, depends, optdepends, conflicts, provides, replaces, groups) are not assigned explicitly within the body template of those package_* functions, but indirectly within _package. Therefore `grep_function` misses those settings.

Suggestion:

Instead of `if [[ $1 == basic ]]; then optdepends+=(...) ...` within _package, do
`_optdepends[basic]="'...' '...'"` globally, using associative arrays.
Then, in the collections eval loop, add "optdepends=(_${optdepends[$_coll]})" to the function body template. Likewise for the other package variables.

Additional info:
* texlive-texmf 2023.66594-15, makepkg as of pacman 6.0.2-7

Steps to reproduce:

makepkg --printsrcinfo
This task depends upon

Closed by  Antonio Rojas (arojas)
Sunday, 16 July 2023, 13:53 GMT
Reason for closing:  Won't implement
Comment by Christian Cornelssen (ccorn) - Friday, 30 June 2023, 06:25 GMT
Complication:

In my previous suggestion, `_pkgdesc[$_coll]` would contain the package description.
Which would be generated dynamically in `prepare()`.

However, some tools do split builds, separating prepare runs from packaging runs.
Which means that prepare() should not be used to set variables needed later.

New suggestion: Extend the functionality of `list-collections` to set those package variables statically.
Or even better, make that script expand the function definitions such that the resulting `PKGBUILD` does not contain any `eval`. For such runs, a read-only `PKGBUILD.in` would be helpful to guard against mishaps.
Comment by Christian Cornelssen (ccorn) - Friday, 30 June 2023, 06:29 GMT
I just noticed that the splitting scenario has already been taken care of, by using files for dynamically generated metadata. Good work.
Comment by Toolybird (Toolybird) - Friday, 30 June 2023, 08:23 GMT
`--printsrcinfo' doesn't cope well with complex PKGBUILDs. It's a makepkg limitation, so probably not much can be done here.. Out of curiosity, what is your use case for the srcinfo?
Comment by Christian Cornelssen (ccorn) - Friday, 30 June 2023, 15:52 GMT
I have attached a patch with a script `make-pkgbuild.sh` that generates an eval-free `PKGBUILD` from a `PKGBUILD.in`. This makes `list-collections` obsolete.

If you want, you can provide arguments to the script:

1. Path to the tlpdb. Default: `tlpkg/texlive.tlpdb` (from the SVN checkout)
2. Output filename. Default: `PKGBUILD`
3. Template filename. Default: `$2.in`

The code parts of the previous `prepare()` and `_package()` that create metadata have been moved to `make-pkgbuild.sh`.
The remaining code deals mostly with putting files and is kept in `PKGBUILD.in`.

The template syntax for `PKGBUILD.in` is simple:
Only lines with the following markers are replaced:

@DEFINE_COLLECTIONS@
@PACKAGE_FUNCTIONS@

These should occur once and only once and in the given order.

Now `makepkg --printsrcinfo` gives the right output.
However, it needs almost 24 minutes to do so.
But improving makepkg performance seems out of scope here.

I have also added a check for reserved names.
For example, consider the package "luatex" in "collection-basic" vs "collection-luatex".
Both would want `provides+=(texlive-luatex)`.
In this case, we prevent "texlive-basic" from providing "texlive-luatex".
I have added the extra names we use (or declare replaced) to the set of reserved names:

bin doc meta texmf core science langextra

The script also reports trivial clashes (package name = name of the collection it is in) for:

context latex metapost pstricks xetex

I have verified that

- `makepkg --printsrcinfo` now works (albeit SLOW: 24m)
- Resulting package set is the same
- package file lists and package metadata (up to permutations) are unchanged except:

* pkgrel, build timestamps, and exact package sizes of course,
* metadata lists are unsorted and may be permuted
* (my build of) texlive-langeuropean rel=15 had corrupted provides
(contained parts of the longdesc); the new build (from this patch) is OK

With the following shell commands:

ver=2023.66594 rel0=15 rel1=16 # rel=15 is from git main branch
test $(ls *-$ver-$rel0-*.pkg.tar.zst | wc -l) = $(ls *-$ver-$rel1-*.pkg.tar.zst | wc -l) || echo "Package count mismatch"
for p in *-$ver-$rel0-*.pkg.tar.zst; do q=${p/-$rel0-/-$rel1-}; diff -u --label "$p" --label "$q" <(pacman -Qip "$p"; pacman -Qlqp "$p") <(pacman -Qip "$q"; pacman -Qlqp "$q"); done | less

I would have expected the build of texlive-basic rel=15 to wrongly provide texlive-luatex, but it does not.
Comment by Christian Cornelssen (ccorn) - Friday, 30 June 2023, 17:20 GMT
Another patch on top of the previous one:

Also simplify the (manually written) package_texlive-meta() such that `makepkg --printsrcinfo` can analyze it.

Tricks used:

_nonlang=(${_collections[*]/#lang*})
depends+=(${_nonlang[@]/#/texlive-})

The first right-hand side filters out the lang* entries from ${_collections[*]}.
Only works for unquoted expansions; else the lang* entries would just get replaced with empty strings.

The second right-hand side prepends `texlive-` to each entry.
Modern bash can do such things.
Comment by Christian Cornelssen (ccorn) - Friday, 30 June 2023, 19:21 GMT
Another minor patch on top: Clarifying comments.

Should be fine now. You might want to consider additional tweaks:

- In `PKGBUILD.in`, move `@PACKAGE_FUNCTIONS@` to the very end.
Currently it is placed where the `eval` was.
(Makes diffs easier to check.)
Comment by Antonio Rojas (arojas) - Sunday, 16 July 2023, 07:34 GMT
Thanks for the patch, but this is adding too much complexity to solve an issue whose severity is not clear to me at all. Using dynamically generated arrays in PKGBUILDs is quite common (firefox-i18n, thunderbird, r, gap, hunspell-es just to name a few) and the fact that --printsrcinfo can't cope with it is something that should be solved in makepkg instead of forbidding packagers to use a rather useful resource.
Additionally, there's the fact that this outputs an almost 5K LOC PKGBUILD and it increases compilation time (as the tlpkg database now needs to be parsed at PKGBUILD generation and at build time).
Comment by Christian Cornelssen (ccorn) - Sunday, 16 July 2023, 13:42 GMT
I can live with that decision.
I am going to make a variant of the `make-pkgbuild.sh` to directly extract the info I need instead of putting that info in `PKGBUILD` and then waiting an eon for `makepkg --printsrcinfo` to process it.

Loading...