FS#43760 - [paccache] 4.2.0 does not remove packages when the architecture has changed

Attached to Project: Pacman
Opened by Moabit (Moabit) - Tuesday, 10 February 2015, 00:07 GMT
Last edited by Allan McRae (Allan) - Tuesday, 11 October 2016, 10:30 GMT
Task Type Bug Report
Category Scripts & Tools
Status Closed
Assigned To Dave Reisner (falconindy)
Architecture All
Severity Medium
Priority Normal
Reported Version 4.2.0
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Summary and Info:
paccache doesn't understand packages with changed architectures. It doesn't include these packages in either a default search, nor the `-u` search.

Steps to Reproduce:
In /var/cache/pacman/pkg, there are five shutter packages. Originally, this was installed as a `-x86_64` package, but was upgraded to an `-any` package.

shutter-0.91-2-x86_64.pkg.tar.xz
shutter-0.92-1-x86_64.pkg.tar.xz
shutter-0.92-2-any.pkg.tar.xz
shutter-0.93-1-any.pkg.tar.xz
shutter-0.93.1-1-any.pkg.tar.xz

If I run `paccache -dvvv`, it says `no candidate packages found for pruning`. Presumably this is because there are three (or less) `-any` packages. It doesn't include the `-x86_64` packages in this analysis. However, if I run `paccache -dvvvuk0`, the `-x86_64` packages aren't included either.

These packages should be included in either search, probably the first one, as this would allow users to revert to previous versions, ignoring architecture changes in packages.
This task depends upon

Closed by  Allan McRae (Allan)
Tuesday, 11 October 2016, 10:30 GMT
Reason for closing:  Won't fix
Additional comments about closing:  the contrib directory has been removed
Comment by Moabit (Moabit) - Tuesday, 11 October 2016, 10:57 GMT
@Allan I'm not sure why this was closed? I'm not sure what the contrib directory is, nor how it being removed relates to this bug. FWIW I can still replicate this bug.
Comment by Allan McRae (Allan) - Tuesday, 11 October 2016, 11:07 GMT
paccache has been removed from the pacman codebase.
Comment by Moabit (Moabit) - Wednesday, 12 October 2016, 01:01 GMT
Oh, ok! Are these scripts going to be housed elsewhere?
Comment by Daniel M. Capella (polyzen) - Thursday, 13 October 2016, 19:06 GMT
Moabit, pacman-contrib [] will be available in the repos by the next pacman release. pacman-contrib-git is in the AUR, for those using pacman-git.

[]: https://git.archlinux.org/pacman-contrib.git
Comment by Moabit (Moabit) - Friday, 14 October 2016, 01:37 GMT
Thanks Daniel. Since this bug report was closed, should a new one be opened against pacman-contrib, as opposed to this being migrated?
Comment by Dave Reisner (falconindy) - Friday, 14 October 2016, 02:02 GMT
Are you sure this is still reproducible? Your example seems to check out for me:

$ paccache -c ~/tmp/cache -dvvuk0
==> Candidate packages:
/home/noclaf/tmp/cache/shutter-0.91-2-x86_64.pkg.tar.xz
/home/noclaf/tmp/cache/shutter-0.92-1-x86_64.pkg.tar.xz
/home/noclaf/tmp/cache/shutter-0.92-2-any.pkg.tar.xz
/home/noclaf/tmp/cache/shutter-0.93-1-any.pkg.tar.xz
/home/noclaf/tmp/cache/shutter-0.93.1-1-any.pkg.tar.xz

==> finished dry run: 5 candidates (disk space saved: 0 B)
Comment by Dave Reisner (falconindy) - Friday, 14 October 2016, 02:09 GMT
The one thing that might be confusing here is that paccache prunes by architecture, not by package. The specific case of a package changing architecture from something to 'any' seems like a bit of an extreme edge case. What if there were also shutter-i686 packages present?
Comment by Moabit (Moabit) - Friday, 14 October 2016, 02:53 GMT
I can still confirm it here.

$ paccache -dvvvuk0
==> no candidate packages found for pruning

Are your results because you don't actually have shutter installed? Even if this bug didn't exist, your output of *-any packages should not be there if you specify -u.

Yes, I presumed that paccache should prune by package. From a user perspective, I'd want the oldest shutter packages removed; I don't really care about their architecture. If they are older than the previous three installs, then they should be gone.

Regardless, I think the behaviour of paccache is inconsistent. I have an *-any package installed at the moment. On one hand, paccache considers the *-x86_64 packages to be "installed", because they don't show up in -uk0. However, if I try and remove all but the most recent three, by using paccache with no flags, these old but "installed" packages aren't counted, and are never removed.

An analogy would be, say, shutter being replaced (and hence conflicting) with a hypothetical shutter-next-gen. paccache -uk0 should remove the old shutter. If paccache (for some reason) considered shutter and shutter-next-gen as equivalent, it should just keep a total of the three most-recent versions of both packages combined.

As far as it being an extreme edge case, I had a look at /var/cache/pacman/pkg , and I have six packages that have changed architecture in the past, and still have 5-6 versions cached, instead of 3.

kaccounts-providers
foomatic-db-nonfree
foomatic-db-engine
foomatic-db
breeze-icons
shutter

I'm not really sure what you mean by having shutter-i686 packages present. Only one shutter can be installed, right, even if other architecture version are present?
Comment by Dave Reisner (falconindy) - Friday, 14 October 2016, 12:17 GMT
Ah sure, the difference is that I don't have shutter installed.

To expand on my point about "what if there's i686 packages installed", consider the following supposed cachedir contents:

shutter-0.91-2-x86_64.pkg.tar.xz
shutter-0.91-2-i686.pkg.tar.xz
shutter-0.92-1-x86_64.pkg.tar.xz
shutter-0.92-1-i686.pkg.tar.xz
shutter-0.92-2-any.pkg.tar.xz
shutter-0.93-1-any.pkg.tar.xz
shutter-0.93.1-1-any.pkg.tar.xz

Assuming paccache would then prune based on package and not architecture, if I ask paccache to leave the oldest 3 versions, what would you expect the remaining packages to be? It seems clear that the "any" packages would be pruned, but then you're left with a coin toss -- do you delete shutter-0.92-1-i686 or shutter-0.92-1-x86_64? Pruning by architecture avoids this problem entirely.

> I have six packages that have changed architecture in the past
You're missing a dimension here. In the past what? Month? Year?
Comment by Moabit (Moabit) - Monday, 17 October 2016, 01:33 GMT
I think you may have mixed up oldest and newest? Paccache doesn't leave the *oldest* three versions, but the *newest* three versions. However, I understand your point. I'm imagine that having both x86_64 and i686 versions cached would be very rare, but I guess it'd be possible if users migrated a system. If this did occur, then I'd personally code paccache to keep the last three *versions*, i.e. just look at version numbers, and if there are multiple architectures, then keep all of them. Over time, this would prune back down to three files.

I'll clarify my "not an extreme edge case" comments. I have paccache scheduled to run regularly, keeping the latest three versions. However, when some packages changed architecture, I have the six most recent versions instead. I keep old versions so that I can revert if necessary, but I'm probably not going to revert to a package that is six versions old. Even worse, since shutter is presumably only *-any now, these versions will advance, while paccache continues to keep the old *-x86_64 versions. In the future, I'll have cached the three most recent versions as expected (say versions 30, 31, 32), but also (say) version 11, 12 and 13.

I don't think this situation is a rare case, because this change of architecture has occurred previously for six packages that I have installed, as listed in my previous post. These packages were also not pruned by paccache down to three files, and each has 5-6 versions cached instead.
Comment by Dave Reisner (falconindy) - Monday, 17 October 2016, 21:24 GMT
Having mixed binaries in your cache is common for anyone who needs to maintain i686 chroots, e.g. anyone who builds packages for Arch.

$ x86=(/pkg/bin/*x86_64*); i686=(/pkg/bin/*i686*); echo "${#x86[@]} ${#i686[@]}"
2692 1039

I'm still interested in the timeframe in which you had this many packages change architecture.
Comment by Moabit (Moabit) - Monday, 17 October 2016, 22:08 GMT
Ah good point.

Yes, sorry I misunderstood your query. I've gone through the packages with >3 cached, and looked at the dates. I've listed the date of the latest-cached previous architecture, and the date of the earliest-cached current architecture.

kaccounts-providers 2015-11-10 2016-08-21 any -> x86_64
foomatic-db-nonfree 2015-03-08 2016-08-27 x86_64 -> any
foomatic-db-engine 2015-03-08 2015-11-09 x86_64 -> any
foomatic-db-engine 2016-02-14 2016-04-18 any -> x86_64
foomatic-db 2015-03-08 2016-08-27 x86_64 -> any
breeze-icons 2015-11-10 2016-09-14 x86_64 -> any
shutter 2014-08-11 2014-08-19 x86_64 -> any

(foomatic-db-engine has changed architecture twice.)

Loading...