Description:

The issue [1] is that a couple of recently updated x86_64 packages were moved to the cleanup directory. The old packages were left untouched.

Eric and I discussed [2] the possible reasons for this behavior and we agreed that we may be dealing with a race condition, as ftpdir-cleanup doesn't lock the database while it's working.

Some further observations:

- Only x86_64 packages were affected.
- The cleanup email [3] was sent at 18:09. The packages were added to the database at 18:06. These timestamps are very close.
- The packages that were added at the same time (18:06) were: apache, slim, twisted. apache wasn't affected, but the old package wasn't removed either.

What I believe happened:

- ftpdir-cleanup starts processing the extra repository for the x86_64 architecture. It extracts the database in a temporary location and starts going through the list of packages.
- Before the cleanup script finishes and after it has processed apache (which is very high alphabetically, compared to the other two which are near the bottom), the three new packages are copied to the ftp directory (and the database gets updated).
- It verifies that the old packages are present (because it's consulting the old database) and marks the new packages are out of date.

A possible solution is to add locking to the cleanup script. Granted that package additions using db-update take very little time, and to ensure that the cleanup script runs when it's supposed to, it could wait a little and retry a fixed amount of times (3-5?) if it cannot acquire a database lock immediately.

Thoughts or opinions?

----
[1] http://bbs.archlinux.org/viewtopic.php?id=84312
[2] See attached file.
[3] http://mailman.archlinux.org/mailman/private/arch-notifications/2009-November/001065.html