Thanks yitz: changed=$(date -d "$(wget -S --spider "$url" 2>&1 | awk '$1 == "Last-Modified:" {for (i = 2; i <= NF; i++ ) printf("%s ", $i)}')" +%s)

curl -z also.

It is too bad that pkgfile -u can't just use rsync, like abs does.
How often do the file lists really change? Not much, I'd wager.

Usually it's done by a cron job a few times a day I think. It would be changed with the same frequency as db updates really...

Can I use rsync to fetch from http?

rsync cannot use HTTP.
It would need an rsync server somewhere, like abs uses.

As for updates to the data, I meant that the data itself does not change
very frequently. Yes, the lists may be fetched one or more times per day,
but for the most part, each fetch is going to grab a lot of data that
did not change since the last fetch. That's why rsync would be a win.
Have a look at the timestamps on
/var/cache/pkgtools/lists/*/*/files

True. The mirrors would have to provide rsync though.

pkgfile should also extract only what's needed as rebase does