FS#14229 - Pacman needs delta compressed package files for minor upgrades
Attached to Project:
Pacman
Opened by Ajay (ashyanbhog) - Tuesday, 14 April 2009, 11:17 GMT
Last edited by Allan McRae (Allan) - Saturday, 06 March 2010, 11:24 GMT
Opened by Ajay (ashyanbhog) - Tuesday, 14 April 2009, 11:17 GMT
Last edited by Allan McRae (Allan) - Saturday, 06 March 2010, 11:24 GMT
|
Details
Summary and Info: Pacman has absolutely no support for
performing minor upgrades through delta compressed packages.
As a result, whole packages are downloaded even for minor
revisions, where the binary diff between two packages could
be less than 0.0001%
A sample execution procedure for pacman once this feature is implemented could be something like this, 1. User runs pacman -Syu with the "new" delta compressed packages enabled 2. Pacman updates repo info, and checks local cache for packages already on hard disk (pacman already has this feature, to resume partially downloads) 3. Pacman downloads packages for major upgrades. For minor upgrades, pacman downloads delta compressed packages when previous version of package is available in local cache. 4. Pacman combines delta compressed packages and previous versions to generate the new package versions 5. Pacman performs integrity check and proceeds to perfoom upgrade. On server side, package maintainers can generate the delta compressed packages. Repo mirroring software may need to be updated to support these features This can save a lot of bandwidth for repo hosters. The same applies on user side as well, though the saving is more pronounced in terms of time. For many users with low bandwidth speeds, like me, this will be a big boon Over the past few days, mysql has seen minor updates repeatedly, and it is frustrating to the entire 25MB package being downloaded for what is probably a revision that is only the size a few KBs, I'm not a developer, so sorry for the long write up and any mistakes. To the best of my knowledge, Zlib supports this feature, Pacman developers, please consider incorporating this featue |
This task depends upon
Closed by Allan McRae (Allan)
Saturday, 06 March 2010, 11:24 GMT
Reason for closing: Implemented
Additional comments about closing: Everything is implemented for use of deltas
Saturday, 06 March 2010, 11:24 GMT
Reason for closing: Implemented
Additional comments about closing: Everything is implemented for use of deltas
However, there is still a lot of work to do on server side. Currently you have to generate deltas between two packages using a create-xdelta script.
Then you need to add these deltas to the database using repo-add.
But this process in Arch is automated, arch developers use dbscript tools to upload packages and have them automatically added to the database.
These tools could be updated to support deltas, but it requires work, and there are several pending issues :
* do we always add deltas to the db, even when they are not interesting (nearly same size of the new package)
* when and how do we clean old deltas
There is still a lot of work to do in that area, and I am not sure there is anyone interested and motivated to do it :)
It looks like it is much easier and transparent to switch to lmza/xz compression, and it is still interesting, even if the size reduction is not as big.