Arch Linux

Assigning to a handful of opinionated people :)

So being opinionated... I think we would be better getting deltas working. The support is basically complete in pacman (I think a script is needed to clear unneeded deltas from the repo db but I have seen a version or two floating around). Of course, xdelta3 is patched for xz support, so these are not exclusive options.

I have not looked at the delta implementation yet but I expect that deltas increase the traffic between mirrors and the required disk space because of the additional delta files and only decrease the traffic between the end-user and its mirror. On the other hand, the xz compression would decrease disk usage and all kind of traffic and it is even less complex. Would it not make sense to switch to xz compression and use delta patches for these smaller packages then?

I think Allan should open his own feature request for this as this is completely unrelated. :P

For xz: Using this compression decreases upload/download time and disk space significantly. I have added support for xz to our scripts some time ago and it should just work. Pacman itself doesn't need to know about it thanks to libarchive. The problem is the migration step. We have two possibilities:

1) Recompress every package with xz and adjust makepkg.conf to use xz by default. This way we wont need to change any scripts and it should just work. Of course this will be quite insane because recompression of the whole repo takes a long time and mirros would have to resync every single package.

2) update our scripts to be able to use both compression methos at the same time and set xz as default. This way we would migrate slowly with every new package. The downside is that our scripts need to have some voodoo added which makes them even more complex. I have had a look at this and it's not as simple as it sounds. So: patches are welcome.

Why would we need to have support for both formats? Can we not just declare packages are in xz format from a given time and release a new devtools/db-scripts on that day. If really needed, the db-scripts could probably sit in a separate folder for a while to allow some sort of transition until people update devtools and their makepkg.conf. pacman will handle repos with mixtures of file compression just fine.

Yeah, there definitely is zero voodoo required on the pacman side...coding for the future is something we do try to keep in mind, and as every package entry in the database has the filename of the package, we could name them whatever the heck we want and pacman will find them. If dbscripts didn't have this in mind, then I'm with Allan- there isn't really a need to support multiple formats, and even if there was it can't be that complex to do.

The only place db-scripts would need looking at is in the section that greps the PKGINFO file

Good point Allan. I didn't really think about it, but accepting only xz for new packages might help here. But there are still scipts left which would have to deal with both. For example the cleanup-scripts move and remove scripts and probably more.

Actually, lets try to get a review of the dbscripts in. We should try to use the filename from the DB entry whenever possible and perhaps use PKG-VER-REL-ARCH.* for the file scanning pattern. Does anyone see a problem with that?

I don't really see a problem with that, Aaron.

To clarify, the only place where mixed-extension stuff is not supported is within db-scripts and devtools? That seems like the place to target this IMO.

So, do we have a volunteer to take the lead and make all necessary changes to db-scripts & devtools?

(unassigned myself as I am not interested in doing this)

Nor am I, not to be a Debbie Downer.