FS#7816 - get source from svn/Subversion URLs
Attached to Project:
Pacman
Opened by Devin J. Pohly (djpohly) - Tuesday, 14 August 2007, 18:20 GMT
Last edited by Allan McRae (Allan) - Friday, 10 August 2012, 04:55 GMT
Opened by Devin J. Pohly (djpohly) - Tuesday, 14 August 2007, 18:20 GMT
Last edited by Allan McRae (Allan) - Friday, 10 August 2012, 04:55 GMT
|
Details
Summary and Info:
Since some projects don't provide tarballs, I'd like to see makepkg have the ability to build a package directly from the latest Subversion repository. (Other VCSs could be added too: Git and CVS come readily to mind.) This might be done just by adding a Subversion URL to the source array, or perhaps by creating another array for source repositories. Steps to Reproduce: 1. Add a Subversion repository to the source array in PKGBUILD, or perhaps to a new repository array. 2. Run makepkg. Actual Results: 3. It doesn't know what the heck you're trying to do. Expected Results: 3. It does know what the heck you're trying to do. 4. It checks out the source or updates an existing repository in the src directory. I wield some decent bash-fu and would be happy to implement this with a bit of guidance. Other Thoughts: * SVN-over-HTTP URLs can't be readily recognized, so we couldn't put them verbatim into the source array. Either we could prefix all Subversion sources with 'svn:', or we could have a separate svnsource array that is handled specially. * We would need to allow the PKGBUILD to specify the local directory name for the repository, so that if more than one is checked out, we don't have a conflicting "trunk" directory. Again, we could append ':local/path/or/name' to the URL in the source array, or we could allow for this in a separate array as above. * Repositories would need to be exempt from the MD5 check. * It would be cool if we could get a revision number (a la svnversion) as part of the pkgver or pkgrel. This could be complicated, though, if there is more than one repository checked out. |
This task depends upon
Closed by Allan McRae (Allan)
Friday, 10 August 2012, 04:55 GMT
Reason for closing: Implemented
Additional comments about closing: https://projects.archlinux.org/pacman.gi t/commit/?id=024bc44a
Friday, 10 August 2012, 04:55 GMT
Reason for closing: Implemented
Additional comments about closing: https://projects.archlinux.org/pacman.gi t/commit/?id=024bc44a
http://aur.archlinux.org/packages.php?do_Details=1&ID=9272
http://wiki.archlinux.org/index.php/Arch_CVS/SVN_PKGBUILD_guidelines#versionpkg_-_a_makepkg_wrapper_for_CVS.2FSVN_builds
I still feel like the current way of doing repo builds is a bit of a hack. Downloading the source doesn't seem like it should be a responsibility of the build() function, and I always found that confusing when trying to use the makepkg -o and -e options to create a custom build. For example, unless you modified the PKGBUILD, you couldn't get makepkg (or, from a brief perusal, versionpkg) to download the sources /and/ the repository without building a package, even though this is what I'd expect makepkg -o to do. It also seems kludgy to put the VCS package in the makedepends array if it's not really required for the build itself.
In addition, if the checkout process is handled by makepkg and not build(), it would be much easier to standardize how it is done, a la the wikipage linked above. Not to mention it would just be a really slick feature. :)
I hesitate to hardcode into makepkg different SCM-fetching schemes. However, this sounds like a great thing to maybe have in /usr/lib/pacman/makepkg/ or something. Source could look something like this:
source () {
/usr/lib/pacman/makepkg/fetchsvn <url> <any other parameters>
}
Thoughts? This would keep complexity out of makepkg while allowing there to be a standard set of scripts, one for each scm, to get the source.
Having a separate script for each SCM makes good sense. I don't see this needing the flexibility of an entire source() function, unless there are some other issues with source-fetching that would require that detailed level of control. But you know better than I would...
What would be great is if each SCM's script was installed with its package, and makepkg then automatically knew it could use that SCM. Sort of a conf.d or profile.d-like directory that is sourced by makepkg, that uses some kind of "hooks" to add functionality.
How about this: an entry in the source array that specifies the SCM to use. Something that is clearly not a URL, e.g. "@svn". Makepkg extracts the name of the SCM into $netfile, uses case to match @*, and calls the corresponding function via fetch_${netfile:1}. Parameters could either be specified with the source (e.g. "@svn,URL,dir-to-checkout-into,user,pass") and split apart, or they could be put in SCM-specific variables (svn_source=(URL1 URL2); svn_user=(user1 user2); svn_pass=(secret1 secret2); ...).
There could be a sourcedepends array for the needed package, or it could just be determined by the success/failure of finding the fetch_xxx function.
http://projects.archlinux.org/git/?p=pacman.git;a=commit;h=8a9c83dd4bffff575a21207248e7acaae5a0d6f9
It looks to me like the commit just changes the version number if a VCS source was used... which is a good thing, just not what I had in mind in this RFE.
Think about it, how do we specify the revision in a svn URL? we need custom syntax, and the parsing of it, etc etc. Then once we set this precedent we'd have to support other SCMs too... it's dirty
I prefer the idea of more variables to custom URLs, as I was pondering in comment 4. For example:
sourcedepends=('subversion')
_subversion_source=('http://example.com/svn/repo/' 'http://example.net/svn/repo2')
_subversion_revision=('HEAD' '2006-12-31')
_subversion_user=('guest' 'guest')
_subversion_pass=('' 'guest')
Then, as you mentioned above, the scripts /usr/lib/pacman/makepkg/<whatever> (which would be installed with the SCM) could do the actual work. We'd validate sourcedepends to make sure the script would be there, then run it during the source-fetching phase of makepkg.
a) You can't pin them down. The idea is to have a single PKGBUILD to handle many different revisions of the source code; it defeats the point if we'd have to change the md5sums array every time a source file changes.
b) The SCM client is (or should be) already doing its own integrity check. It's not a straight downloader like Wget or cURL.
c) It would probably be complicated and not fun to program.
How to tell the difference between files that need checked and those that don't will depend on how the download/checkout is implemented, I guess.
if [ "$file" !~ /^git/ ];
elif [ "$file !` /^svn/ ];
And so on for every handler... which would be fugly as hell. I suppose we could define an array of scm:// things and loop through that for each $file... which would also suck. I don't see any good way to do it... If bash supported hashes then it wouldn't be so hard, but I really have no clue.
case "$file" in
ftp:* | http:*)
checksum_stuff
;;
*)
no_checksum_stuff
;;
esac
http://gitorious.org/~ciprian.craciun/pacman/ciprian-craciun-patches
In summary:
* added a script: /usr/bin/git-dlagent that takes exactly two arguments: URL and output.
* updated /etc/makepkg.conf to include the following URL's: git://, git+file://, git+http://, git+https://, git+ssh://, git+rsync;
* a Git url looks-like: git+git://projects.archlinux.org/pacman.git#master##pacman.tar where:
* what is before the first `#` is the URL used to (maybe) clone the Git repository.
* what is after the first `#` and before `##` is the refspec (head / tag / etc.) to export;
* what is after `##` (optional this part is) it can be used to make the file-part of the URL look like a tar file;
How git-dlagent it works:
* in case git+file:// is given `git --git-dir=... archive` is done;
* else it tries to do `git archive --remote=...`; (usually this doesn't work unless the repository is git and the barkend is enabled);
* but anyway it fallbacks to `git clone ...`, and then a local git archive;
Any comments?
P.S.: I've tested the script directly with my installed pacman, and only after that I've thrown the script in the repository. But I think it should work.
(My approach just uses the Git / SVN / etc tools to create a .tar or .zip file, and leave the rest up to makepkg. I've also patched makepkg to ignore those hashes that are equal to `--`.)