FS#2803 - tarballs should be rebuilt

Attached to Project: AUR web interface
Opened by Dusty Phillips (Dusty) - Sunday, 05 June 2005, 17:21 GMT
Last edited by Simo Leone (neotuli) - Saturday, 11 June 2005, 04:53 GMT
Task Type Feature Request
Category Backend
Status Closed
Assigned To Simo Leone (neotuli)
Architecture All
Severity Medium
Priority Normal
Reported Version 1.0.3
Due in Version 1.2
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

When a user uploads a tarball, it may have several formats and names. AUR should automatically untar the tarball and recreate it, ensuring that the tarball that is linked from the package's page has a consistent name (ie: proj.tar or proj-version.tar). It should also have a consistent directory structure (ie: the PKGBUILD and other files should be inside a directory with the name of the proj and not just inside the tarball). Finally, it should ensure that only files needed by the PKGBUILD (ie: those in the source array that do not have external URLs and NOT accidentally uploaded binaries) are included in the tarball.

This will make it easier for people wishing to write scripts to interface with the AUR (ie: me!). It will also make it a bit easier for users downloading the tarballs because they will consistently know what to expect. Also, it can reduce download size by removing unnecessary files.

Dusty
This task depends upon

Closed by  Simo Leone (neotuli)
Saturday, 21 January 2006, 18:56 GMT
Reason for closing:  Fixed
Additional comments about closing:  finally.
Comment by Simo Leone (neotuli) - Monday, 06 June 2005, 14:42 GMT
This sounds reasonable, believe it or not, I was considering it.
Comment by Simo Leone (neotuli) - Monday, 06 June 2005, 20:11 GMT
One thing that needs to get discussed is whether or not we want the server having to do the work of re-tarring it. I don't think there will be any problems, but I could see how there might be an objection.
Comment by Paul Mattal (paul) - Friday, 10 June 2005, 04:05 GMT
How about for now just looking at the contents of the tar and rejecting ones that aren't in the expected form? We could use tar -t to look at the contents first and process those. This would also allow us to spot other features of the tarball that might be nonstandard.

Also, while we're fixing this, we should make sure that we chmod the entire untarred tree u+w. Some people have uploaded stuff where that's not the case, and it makes the system very unhappy when it comes time to delete the untarred package.
Comment by Simo Leone (neotuli) - Friday, 10 June 2005, 15:33 GMT
I asked Judd about re-tarring, he said it would be fine resource-wise. We could definitly enforce a stricter format on the user side, but we have to handle ~700 legacy packages now too.
Another thing we should check for are pkg/ and src/ dirs, even I uploaded one like that by accident.
Comment by Dusty Phillips (Dusty) - Friday, 10 June 2005, 15:44 GMT
instead of checking for pkg/ and src/ directories, why not do this:

* untar the existing package
* confirm existence of PKGBUILD
* check src array of PKGBUILD for files that are local (no URLs)
* include only the PKGBUILD and local files in the src array
* retar the existing package in the correct directory structure

Dusty
Comment by Paul Mattal (paul) - Friday, 19 August 2005, 17:26 GMT
Re-opened per request from dibble
Comment by Simo Leone (neotuli) - Sunday, 11 September 2005, 03:26 GMT
We need to go back and manually fix those that were preexisting, right?
Comment by Simo Leone (neotuli) - Saturday, 21 January 2006, 18:56 GMT
Ok this is absolutely, positively, fixed. I hacked together some scripts to cut the number of manually fixed packages down to under 200, and proceeded to fix all legacy packages. Also, I made some changes to the links in the AUR so that they always point to "standard" tarballs.

So in short, everything should be swell now.

Loading...