FS#63092 - [devtools] Skip building packages using makechrootpkg if already built

Attached to Project: Arch Linux
Opened by Siddhartha Sahu (sdh) - Wednesday, 03 July 2019, 23:21 GMT
Last edited by Kristian (klausenbusk) - Saturday, 03 June 2023, 18:07 GMT
Task Type Bug Report
Category Arch Projects
Status Closed
Assigned To Levente Polyak (anthraxx)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description:
makepkg stops building a package if it finds a pre-built package of the same version in PKGDEST. This is especially useful for vcs packages as I can run makepkg regularly and get a build only if there are new commits.

However, makechrootpkg does not seem to exhibit this behavior. Reading the script, I found that makechrootpkg's internal PKGDEST is defined separately as /pkgdest and later packages are copied to externally configured PKGDEST.

I could get the above desired behavior by changing the internal PKGDEST to /startdir. Can this (or a better technique) be added to the official repo?

Additional info:
* package version: 20190329-1
* I asked this on the forums before but got no responses: https://bbs.archlinux.org/viewtopic.php?id=247166

Steps to reproduce:
* Start PKGBUILD in a folder which already has the package built.
* Expect build to fail with message "A package has already been built. (use -f to overwrite)"
* Instead, the build starts from scratch everytime.
* Update PKGDEST in /usr/bin/makechrootpkg to "/startdir"
* Builds now behave as expected.
This task depends upon

Closed by  Kristian (klausenbusk)
Saturday, 03 June 2023, 18:07 GMT
Reason for closing:  Upstream
Additional comments about closing:  Please report upstream if this is still relevant: https://gitlab.archlinux.org/archlinux/d evtools.
Comment by Eli Schwartz (eschwartz) - Wednesday, 03 July 2019, 23:39 GMT
I believe this is intentional, because the standard use case of makechrootpkg is to iterate over scratch builds until we get it right. Your use case is pretty specific to git packages, anyway, which we don't support in the official repos.

Workaround that doesn't involve initializing a chroot just to test whether the package is already built:

makepkg -dd --nobuild # hopefully no prepare function runs commands that don't exist yet
makepkg --packagelist | IFS= while read -r line; do
if [[ -f $line ]]; then
echo "there is already a built package, aborting..."
exit 13
fi
done
extra-x86_64-build
Comment by Levente Polyak (anthraxx) - Wednesday, 03 July 2019, 23:42 GMT
im much against such a behavior for exactly the reasons pointed out by Eli, iterate over scratch builds, which is a far more common use case than the desire to skip potentially the very same package.
Comment by Eli Schwartz (eschwartz) - Wednesday, 03 July 2019, 23:51 GMT
Note that /startdir is definitely wrong, only PKGDEST should be used for your proposal.

Also note that makechrootpkg will atomically update the pkgfile after checking that all packages in a split package succeeded. Depending on bare makepkg for this (and using --force) means a partially failed build will make the previous successful build that you tried to overwrite, unusable, without replacing it with a new set of packages. Admittedly this is behavior intrinsic to makepkg itself...
Comment by Siddhartha Sahu (sdh) - Wednesday, 03 July 2019, 23:58 GMT
Thanks for the workaround. I can live with that I guess.

> Note that /startdir is definitely wrong, only PKGDEST should be used for your proposal.
The PKGBUILD is placed inside /startdir, so setting PKGDEST=/startdir is what allows makepkg to check existing packages. I didn't find a way to do otherwise.

> Your use case is pretty specific to git packages
I figured that skipping builds for an already built package could be a general feature. But it seems not, so that's fine.

Thanks for the prompt responses.
Comment by Siddhartha Sahu (sdh) - Thursday, 04 July 2019, 00:07 GMT
I'll note that having it inside makechrootpkg is more streamlined because it reuses the same makepkg settings.
With the workaround, I'll have to remember to use the same makepkg config.
Comment by Eli Schwartz (eschwartz) - Thursday, 04 July 2019, 00:30 GMT
Setting PKGDEST to /startdir will not work, because that is just $PWD for the PKGBUILD. Instead you would need to handle PKGDEST the same way we are already handling srcdest -- with a separate directory, that can be bind mounted as either SRCDEST (if defined outside the chroot) or $PWD (if not defined outside the chroot).
Comment by Eli Schwartz (eschwartz) - Thursday, 04 July 2019, 00:36 GMT
If this were to be implemented in makechrootpkg I think it would probably be better to be opt in, not opt out. A possibly better way to preserve atomicity would be to find all files in PKGDEST that begin with members of "${pkgname[@]}" and `touch` the corresponding files in the chroot. Alternatively just mount the PKGDEST and assume the user does not care about that.

Loading...