FS#58186 - Add rmlint like step to makepkg and/or namcap

Attached to Project: Pacman
Opened by Anadon Aeroheart (anadon) - Tuesday, 10 April 2018, 12:25 GMT
Last edited by Allan McRae (Allan) - Tuesday, 19 June 2018, 10:47 GMT
Task Type Feature Request
Category Packages: Testing
Status Closed
Assigned To No-one
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

rmlint can save space by deduplicating files, and catch some problems with empty directories and files which should probably be done in all packaging anyways. It'd be nice if it was integrated more into the build system.


This task depends upon

Closed by  Allan McRae (Allan)
Tuesday, 19 June 2018, 10:47 GMT
Reason for closing:  Won't implement
Additional comments about closing:  Can be implemented as libmakepkg extension.
Comment by Eli Schwartz (eschwartz) - Tuesday, 10 April 2018, 13:57 GMT
We already have options=(!emptydirs) which uses find (so no extra dependencies) to remove empty directories. I wouldn't want to remove empty files, though, since those are *very* often not mistakes, e.g. __init__.py

As for duplicate files, this seems like something which should be fixed upstream, not in packaging. Notice how we do not even enable emptydirs by default, because some packages create those deliberately, e.g. subdirectories of /etc which are owned by some UID which corresponds to a daemon user.
Comment by Anadon Aeroheart (anadon) - Tuesday, 10 April 2018, 14:02 GMT
That works for good upstream sources, but I just went through a slew of biologists trying to be programmers and it was bad. Inconsistent semi-recursive file copy backups bad. I'm not as familiar with the other stuff you're getting at, but the sources I just packaged need this kind of treatment.
Comment by Eli Schwartz (eschwartz) - Tuesday, 10 April 2018, 14:49 GMT
Hmm, well, rmlint could definitely be done by hand in package(), but I'm not sure we should add it into makepkg itself... at least when we cannot guarantee rmlint is available. I guess we could guard it by checking if the executable is available, and expect people to install rmlint in makedepends if they set the option in the PKGBUILD itself for problematic packages.

This *definitely* will not be enabled-by-default. It could cause "interesting" things to happen with e.g. files that have their abspath consulted for whatever reason.

Since current versions of makepkg are implemented as a library, it is easy to extend it (this is partly why it was refactored this way). Would you be interested in trying to implement a new tidy_install() dropin by creating a new file in /usr/share/makepkg/tidy/, which implements rmlint deduplication? This could be installed separately from makepkg, but if Allan likes it enough could be merged into the main codebase.
Comment by Anadon Aeroheart (anadon) - Tuesday, 10 April 2018, 15:06 GMT
Possibly. Right now, I have other Lovecraftian code I'm trying to make workable. Given that this could help prevent 80GB compressed, over 500GB uncompressed packages though I certainly have the impetus to. Where would I start to read up on how to make such an addition correctly?
Comment by Eli Schwartz (eschwartz) - Tuesday, 10 April 2018, 15:28 GMT
Oh, basically just look how the other tidy routines in that directory work.

libmakepkg is pretty simple, we source /usr/share/makepkg/*.sh and each of those files, e.g. /usr/share/makepkg/tidy.sh, might itself source the corresponding /usr/share/makepkg/tidy/*.sh

So we can add files tidy/*.sh which define new functions and add their names to the arrays "tidy_remove" and "tidy_modify", then each element of both those arrays is run as a function in tidy_install() (from tidy.sh itself) when makepkg's core invokes it in between run_package (which runs the package() function from the PKGBUILD) and create_package (which writes metadata then tar's everything up).

Similar logic is used to make lint_{package,pkgbuild}.sh extensible (and lint_config.sh in one of my pending patchsets).

Also see https://aur.archlinux.org/packages/makepkg-optimize/ which does some similar work.

Loading...