FS#69932 - pacman downloads the database on every sync when using IPFS

Attached to Project: Pacman
Opened by Ruben Kelevra (RubenKelevra) - Wednesday, 10 March 2021, 09:43 GMT
Last edited by Allan McRae (Allan) - Friday, 26 November 2021, 21:21 GMT
Task Type Bug Report
Category Backend/Core
Status Closed
Assigned To No-one
Architecture All
Severity Very Low
Priority Normal
Reported Version git
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Summary and Info:
I'm using a mirror that uses long etags, like 'bafk2bzacecyxibvkow7n2grvjdhc4zslsdekmavv3dvcu6dwi5akrd2hhobeg' which are static unless there are changes to the database files. But pacman seems either having trouble handling the long etag or just uses last-modified timestamps - which isn't working for this application.

Steps to Reproduce:
- Use `https://ipfs.io/ipns/x86-64.archlinux.pkg.pacman.store/$repo` as mirror
- Run `pacman -Sy` twice


Pacman will download the database files regardless of changes.
This task depends upon

Closed by  Allan McRae (Allan)
Friday, 26 November 2021, 21:21 GMT
Reason for closing:  Upstream
Additional comments about closing:  
https://github.com/ipfs/go-ipfs/issues/7 968
Comment by Allan McRae (Allan) - Wednesday, 10 March 2021, 10:23 GMT
Looks like ipfs reports the current time as the files "last-modified" time:

curl -s -v -X HEAD https://ipfs.io/ipns/x86-64.archlinux.pkg.pacman.store/core/core.db 2>&1 | grep last


As far as I am concerned, this is correct behaviour by pacman.
Comment by Ruben Kelevra (RubenKelevra) - Wednesday, 10 March 2021, 11:05 GMT
Agreed that a last-modified tag isn't correct in this case, since ipfs isn't able to know when a ressource has changed. So 'now' is as wrong as every other date.

Why is pacman not just storing the etag and sending it via 'If-None-Match' for the update request? If the etag hasn't changed it'll get a 304 back. This avoids that different mirrors might have wrong clocks etc.
Comment by Ruben Kelevra (RubenKelevra) - Wednesday, 10 March 2021, 11:18 GMT
I've created a ticket for the IMHO wrong tag in the response header:

https://github.com/ipfs/go-ipfs/issues/7968
Comment by Allan McRae (Allan) - Wednesday, 10 March 2021, 11:35 GMT
I can not see an option for libcurl to retrieve tags. It seems curl's ability to get etags was added in 2019 and added directly to the app and not libcurl.
Comment by Eli Schwartz (eschwartz) - Monday, 05 April 2021, 23:27 GMT
How would etags even work? pacman would need to... store these in some sort of state file? What do etags mean for a mirror network, if say mirrors happen to independently add etags that aren't consistent?
Comment by Ruben Kelevra (RubenKelevra) - Friday, 26 November 2021, 12:33 GMT
Well, they could be stored into a xattr. And sure, they could just be a SHA256 sum on regular servers or something like that? This could confirm the same file version even through pacman switched servers.

Loading...