FS#56128 - {mkinitcpio} patch for --rsyncable option

Attached to Project: Arch Linux
Opened by Charles Duffy (charles-dyfis-net) - Thursday, 26 October 2017, 17:43 GMT
Last edited by Dave Reisner (falconindy) - Sunday, 08 December 2019, 11:25 GMT
Task Type Feature Request
Category Arch Projects
Status Closed
Assigned To Dave Reisner (falconindy)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Presently, running consecutive cpio builds with the same inputs, the resulting outputs can be almost entirely different. This means that using a storage system such as `rdiff-backup`, a binary-delta-based update distribution mechanism such as `casync`, or the like will effectively need to store both old and new builds in full, no matter how small the actual change between them.

With the attached patch, consecutive builds which previously had ~27.5MB of deltas are reduced to only 120KB of data transmitted by rsync to sync them (at a cost of somewhat worse compression and lost timestamps within the cpio archive, hence keeping this option off-by-default).
This task depends upon

Closed by  Dave Reisner (falconindy)
Sunday, 08 December 2019, 11:25 GMT
Reason for closing:  Implemented
Additional comments about closing:  But also sort of wontfix? went in a different direction to get to the same end result.
Comment by Eli Schwartz (eschwartz) - Thursday, 26 October 2017, 18:30 GMT
  • Field changed: Summary ([PATCH] mkinitcpio --rsyncable → {mkinitcpio} patch for --rsyncable option)
  • Field changed: Status (Unconfirmed → Assigned)
  • Field changed: Category (Packages: Core → Arch Projects)
  • Task assigned to Dave Reisner (falconindy)
You're probably best off sending this to the arch-projects mailing list for discussion: https://lists.archlinux.org/listinfo/arch-projects

Kind of awkward to do patch review here, but...

Why does it matter whether rsyncable is used, for purposes of sorting the filelist passed on to cpio? AFAIK there is no downside to doing it either way... it simplifies the logic.
Comment by Charles Duffy (charles-dyfis-net) - Thursday, 26 October 2017, 19:39 GMT
Good point. Patch attached (which --autosquash should merge) that makes the filename sort nonconditional, and also adds -n to the set of gzip options in use when --rsyncable is specified.

Will be starting an arch-projects thread momentarily. No position taken on whether to keep this ticket open in the interim. (BTW, might not hurt to mention the arch-projects list in the HACKING or README files going forward).
Comment by Charles Duffy (charles-dyfis-net) - Friday, 27 October 2017, 16:58 GMT Comment by Dave Reisner (falconindy) - Sunday, 08 December 2019, 11:24 GMT
mkinitcpio should be generating reproducible archives these days.

Loading...