AUR web interface

Tasklist

FS#2643 - cvs access to PKGBUILDs

Attached to Project: AUR web interface
Opened by maciek (demonus) - Saturday, 23 April 2005, 12:22 GMT
Last edited by Paul Mattal (paul) - Friday, 10 June 2005, 11:18 GMT
Task Type Feature Request
Category Backend
Status Closed
Assigned To Paul Mattal (paul)
Architecture All
Severity Very Low
Priority Normal
Reported Version 1.0.1
Due in Version Undecided
Due Date Undecided
Percent Complete 20%
Votes 0
Private No

Details

this shouldn't be too much work, accessing aur repository through cvs so that PKGBUILDs could be merged to current ABS tree would be very useful
This task depends upon

Closed by  Paul Mattal (paul)
Friday, 29 July 2005, 04:01 GMT
Reason for closing:  Implemented
Additional comments about closing:  We've implemented CVS access to PKGBUILDs via ABS for community, and decided not to do so for unsupported for now, due to the dangers that could pose.
Comment by Simo Leone (neotuli) - Saturday, 23 April 2005, 15:34 GMT
You mean use CVS just so you can download the packages, not for version control, and no write-access? That's feasable. We might be able to do it with rsync, webdav, or cvsup.
If you're talking about adding write access and using it for version control and so forth, there's a lot involved and we haven't seen enough demand for such features yet.

Let me know, in more detail, what you're looking for.
Comment by maciek (demonus) - Saturday, 23 April 2005, 20:22 GMT
just plain downloading of PKGBUILDs with cvsup, no write-access, so that you could add proper supfile for ABS and fetch the current tree of AUR
Comment by Paul Mattal (paul) - Sunday, 24 April 2005, 01:28 GMT
I've got an email in to Judd on this to add support to the cvsup server. I'll let you know what he says.
Comment by Paul Mattal (paul) - Monday, 25 April 2005, 11:47 GMT
Quoth the Judd:

"Yep, it's on my list for the 2.9.6 pacman release."

So we'll see that soon in due course. I'll leave this open until we do!

- P
Comment by Aaron Griffin (phrakture) - Thursday, 28 April 2005, 17:13 GMT
yeah - it's not too hard... the supfile would be a direct copy of any of the others... the problem is that the aur tag isn't setup as a cvsup collection... so abs won't get it (yet) - it shouldn't be that hard to make it work... just one change on the server, right?
Comment by Paul Mattal (paul) - Thursday, 28 April 2005, 20:08 GMT
Yep, it should be very easy. I'm sure Judd will get to it soon.
Comment by maciek (demonus) - Thursday, 28 April 2005, 20:22 GMT
glad to hear it, one step further to getting pkgbuilds collection equvalent to freebsd ports (at least probably some time in the future)
Comment by Aaron Griffin (phrakture) - Friday, 29 April 2005, 15:09 GMT
AUR + srcpac = godlike

I think allowing the cvsup should be a much higher priority - it will hype AUR alot more if someone can sync abs and then run "srcpac -S lighttpd" to build aur packages...
Comment by Aaron Griffin (phrakture) - Friday, 29 April 2005, 19:45 GMT
Anyway, for now - I did this so I can get a list of everything in AUR... it's not perfect, but it's a solution for now:

http://bbs.archlinux.org/viewtopic.php?t=12037
Comment by Simo Leone (neotuli) - Monday, 02 May 2005, 22:44 GMT
If it becomes as easy as running "srcpac", this is most certainly a BAD thing. Innocent, clueless, users, could get duped into installing and doing god knows what (as root nonetheless), and never even see the PKGBUILD.
I think we need to leave things in a way such that the user has to at least LOOK at the PKGBUILD before going and compiling/installing it, though I also do very well see the convenience in being able to run srcpac for AUR packages.
Just thought I'd point this out, there's probably a way to do this and achieve a happy medium, though I'm thinking that it would become stupidly complicated.
IMHO, if we make it this easy, we need to at least put really big red signs all over the place.
Comment by Paul Mattal (paul) - Tuesday, 03 May 2005, 02:46 GMT
I'm not totally convinced that simple == bad. Not to say that I don't see your point.. I think we need a good long discussion on this topic with lots of people involved. More than just in this bug.

As a minimum, the "community" packages should be accessible via srcpac/abs/cvsup. I think that's a no brainer.

If we did include unsupported, it might even be worth having srcpac bark a warning if you're building something from unsupported. This would require a special case mod to srcpac, but it's probably worth it.

This sort of topic is at the very heart of the delicate balance that is the AUR.. security vs. ease of use. It's the kind of thing we've got to tackle head on and think creatively about. Thanks for starting the discussion!
Comment by Jason Chu (jason) - Tuesday, 03 May 2005, 04:47 GMT
I saw somewhere a script that downloaded all the PKGBUILDs from unsupported.

Being someone who has worked on AUR, srcpac, namcap, and the TUR (the original user repos) I like to think I've had a little bit of experience.

Let's see if I can list off my responses:
1) Putting unsupported PKGBUILDs in cvs (so that you can actually use cvsup) seems sort of wasteful. Why not use the bash script?

2) pacman is going to do what srcpac does in 2.9.6?

3) the community repo's tagging is the same as current and extra, so cvsup will work with it no problem. Currently, unsupported isn't in anything remotely like a cvs repo. I had mentioned making abs a wrapper to support svn, cvsup, and other storage methods, but it was poo-pooed by some people because it wasn't "simple". That's why "community" packages are stored in cvs instead of svn. I still disagree with that assertion.

4) srcpac won't work with just an abs tree. It needs pacman's dependency and upgrade logic. Without a proper package repo srcpac will never realize any of those packages need updating.

5) I don't think users should be able to blindly install/update anything uploaded to unsupported. That's why it's unsupported. If they want to be able to use srcpac and pacman with it, they should become a TU and maintain it in community.

6) I don't see why people haven't written a supfile for community yet. I'm sure that some people have (this sort of thing used to happen all the time on irc, but I have noticed a much higher signal to noise ratio in our community).
Comment by Simo Leone (neotuli) - Saturday, 07 May 2005, 14:30 GMT
as to point #4: It is not difficult to modify gensync to just forget about the package file and just generate the parts that are relevant (deps, descriptions, versions). Hell, I just did it, commented out maybe a dozen lines, works like a charm.
In other words, it certainly wouldn't be difficult to make srcpac work properly with unsupported...

5: yes, i'm with Jason
Comment by Paul Mattal (paul) - Saturday, 07 May 2005, 14:38 GMT
re #4: Interesting. So are we worried about this? I'm inclined to let people do that mod on their own if they like. I mean, we can't stop people from actually trying to do foolish things. But I still think it's worth making the PKGBUILDs from unsupported available via cvsup so that people can get the whole bunch of them easily and peruse them on their own machine.

re #5: Me too. It's taken a little while for me to think the whole thing through, but I definitely agree with you and Jason. The alternative is just too dangerous.
Comment by Jason Chu (jason) - Saturday, 07 May 2005, 15:41 GMT
It's not gensync that srcpac has problems with. srcpac -Sb will be able to build the package originally, but if the PKGBUILD is upgraded it will never pick up on that and never rebuild it. It's only good for install (assuming you mod it to not need pacman for that case), but not a long term system maintenance.
Comment by Simo Leone (neotuli) - Saturday, 07 May 2005, 18:56 GMT
I just tried out the upgrading, works fine. *shrug*

I'm not sure this is a good idea, but I've made the right hacks to the right scripts to get srcpac working with unsupported, with the help of a web scraper someone made and posted to the forums.

These scripts are attached for the daring and curious, have fun. (there's a better description inside the tarball)
Comment by Jason Chu (jason) - Monday, 09 May 2005, 14:03 GMT
So, you can srcpac -Sb a package. Then update the PKGBUILD's version and srcpac -Su and have it upgrade?

For this to work, you'd have to go through the whole /var/lib/srcpac, match up all the from-source packages with unsupported, verify that their versions are newer, make sure their depends, provides, conflicts, etc are all valid and then mark them for building.

If all you got working was a second srcpac -Sb brings the package up to date, that's not exactly good enough. If there's no notification that packages need updating, everyone will just be using older versions of unsupported.
Comment by Simo Leone (neotuli) - Monday, 09 May 2005, 23:33 GMT
Well the way i set these up are for them to work outside of the rest of the package tree for now (so pacman doesn't go all crazy complaining about upgrades that may or may not really exist), however pacman knows that they are installed. Upgrading seemed to work if you re-synced the unsupported pkgbuilds and re-gensync'ed.
Comment by Jason Chu (jason) - Tuesday, 10 May 2005, 00:27 GMT
We are talking about unsupported here, right? How are you generating a repo without the related package files (md5sums, for example)?

How exactly does this "work outside of the rest of the package tree"? Can you describe your setup a little better?
Comment by Paul Mattal (paul) - Friday, 10 June 2005, 05:41 GMT
To awaken this horrible thread again, the following recap. Correct me if I end up wrong:

1) There's no reason not to make an ABS collection for community. That's also pretty straightforward to do.

2) It sounds like srcpac won't work properly with only an ABS collection without a real repo.

3) There's a bash script out there that lets you fetch the whole set of unsupported files to your local hard drive.

4) cvsup supports sharing things as collections that aren't in cvs. Oddly.

THEREFORE, I suggest:

1) Let's create a community collection for ABS.

2) Let's create an unsupported collection for ABS. Effectively, people can do this now with the bash script, it's just slightly less convenient. Realistically, we're not going to stop people from fetching the pile of AUR unsupported PKGBUILDs and putting them on their machine.

3) Both community and unsupported should be commented out in the default ABS configuration file, and there should be large comments in place to disclaim turning them on.

4) Let's MAKE SURE that srcpac doesn't work with unsupported. Let's actually CAUSE it not to work with unsupported if it happens to. This way we keep it from being casually misused.

Thoughts?
Comment by Jason Chu (jason) - Friday, 10 June 2005, 06:33 GMT
Everything looks good up until the #4 suggestion. I don't want to break srcpac in some subtle way to make it sort of not work with unsupported. Unless a package with the same name exists in extra (or some other actual repo), it won't ever build a package from unsupported.
Comment by Paul Mattal (paul) - Friday, 10 June 2005, 12:21 GMT
Sure, I guess I didn't actually mean to break it. I just meant that it might make sense to specially handle "unsupported" so that it just doesn't work. A safety check. I realize it's not a pure solution, but it would guarantee the safety we desire.

I'm still wrestling with the whole notion of not allowing package names that are already in current/extra, and am becoming more fond of the idea.

- P
Comment by Simo Leone (neotuli) - Friday, 10 June 2005, 15:25 GMT
The no dupes thing is do-able, I think. If we cross check with the main database for the same entry during a new submission...
Oh and I guess we'd have to do some manual queries to clean up the dupes that already exist up until that point.

Loading...