FS#35050 - [makepkg] support for incremental builds with VCS sources
Attached to Project:
Pacman
Opened by Lukas Jirkovsky (6xx) - Wednesday, 01 May 2013, 17:45 GMT
Last edited by Allan McRae (Allan) - Monday, 17 November 2014, 07:19 GMT
Opened by Lukas Jirkovsky (6xx) - Wednesday, 01 May 2013, 17:45 GMT
Last edited by Allan McRae (Allan) - Monday, 17 November 2014, 07:19 GMT
|
Details
It would be great if there was a support for incremental
builds when using VCS sources.
Right now, when building a package, the $srcdir seems to be removed and then updated repository content from the $SRCDEST is then exported to the $srcdir. This means that whenever package is built using makepkg, everything is rebuilt. Because the packages with VCS sources are likely to be used by people who follow development closely and thus they are likely to be build the package very often, it would make sense to (optionally?) just update the $srcdir content, so the incremental build works as this can save quite a few hours. |
This task depends upon
Closed by Allan McRae (Allan)
Monday, 17 November 2014, 07:19 GMT
Reason for closing: Implemented
Additional comments about closing: In git
Monday, 17 November 2014, 07:19 GMT
Reason for closing: Implemented
Additional comments about closing: In git
1) update git sources (use git pull origin <some branch> in the $srcdir repo)
1a) possibly run some cleanup on the repo to remove stale built files (git clean or make clean, maybe?)
2) run makepkg -e (will use the existing source tree)
But honestly, this shouldn't be too much of an issue. The only part of the whole 'clone to srcdir' bit that actually takes any time is the checkout of the files. If you have $SRCDEST and $srcdir on the same filesystem, then git will use hardlinking (see the git-clone(1) man page). You shouldn't really notices much of a change, honestly, unless you're using a repo that's >400M or so. And even that won't be too much of a hassle. I currently build the linux-mainline package using a git source, and that is the only one I could think of taking more than a couple of seconds to update, clone, and start building.
Allan: I'm not sure what you mean. Do you mean skip making package and just use the "make install" or something similar? That would not be very clean (and I don't want to put my system into risk of some problem caused by problem in the buildsystem).
Before pacman 4.1 all I needed to do for the incremental build to work was to omit the removal of the build directory. Now that the makepkg handles the VCS sources (which I really like), it would be nice to have an option that basically does the steps mentioned by KaiSforza (ie. update sources in $srcdir and build the package without cleaning the srcdir). I think this would be a nice addition for developers.
mkdir $srcdir/build
cd $srcdir/build
../$pkgname-$pkgver/configure ....
_svnname=blender
source=("src/${_svnname}::svn+https://svn.blender.org/svnroot/bf-blender/trunk/blender")
noextract=("src/${_svnname}")
This way the SVN repository is cloned into the same location as originally and `noextract` prevents makepkg to destroy it (currently makepkg deletes `src/blender` and then tries to copy `src/blender` to `src/blender`, which obviously fails). Then the repository can be used as before, all timestamps are preserved and there are no unnecessary copies around.
However this works only with SVN, Git repositories are cloned as "bare" and they cannot be used this way, as there is no working copy to work with. In my opinion the easiest way to support this is to have some option (e.g. `!bare`) which will clone the repository with working copy.
On the other hand this will work only if the repository isn't touched in any way (no in-source building, patching etc.), as subsequent updates might fail with conflicts (and there is no way to clean up the repository before updating, as `prepare()` is called after that).
https://mailman.archlinux.org/pipermail/pacman-dev/2013-September/017874.html
Any comments are welcome. If everything is fine, I would like to extend this functionality to other VCSs, too. I'm also attaching a patch that can be applied directly against /usr/bin/makepkg from pacman 4.1.2 to make this easier to test.