FS#12148 - pacman exits with no output when there is a database error

Attached to Project: Pacman
Opened by Federico Asara (Eons) - Sunday, 16 November 2008, 22:05 GMT
Last edited by Xavier (shining) - Sunday, 31 May 2009, 15:46 GMT
Task Type Bug Report
Category Output
Status Closed
Assigned To Dan McGee (toofishes)
Architecture All
Severity Low
Priority Normal
Reported Version 3.2.1
Due in Version 3.2.2
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Summary and Info:
Pacman quits without printing anything if a db file in /var/lib/pacman/sync/ supposed to be a directory is a regular file. In my case, it was the unstable repository's directory.

Steps to Reproduce:
Don't really know. I addded some repositories, like unstable, to pacman.conf. Then I executed pacman -Sy and.. no output.
I "solved" this by deleting /var/lib/pacman/sync/unstable .
If you really want to reproduce this bug, do:
rm -r /var/lib/pacman/sync/$repo
touch /var/lib/pacman/sync/$repo
where $repo is a directory name.
I don't know how it happened.

Output with and without --debug:
[root@localhost ~]# pacman -Sy
[root@localhost ~]# pacman -Sy --debug
debug: config: attempting to read file /etc/pacman.conf
debug: config: new section 'options'
debug: config: HoldPkg: pacman
debug: config: HoldPkg: glibc
debug: config: SyncFirst: pacman
debug: config: new section 'testing'
debug: setlibpaths() called
debug: option 'cachedir' = /var/cache/pacman/pkg/
debug: registering sync database 'testing'
debug: opening database 'testing'
debug: opening database from path '/var/lib/pacman/sync/testing/'
debug: adding new server URL to database 'testing': ftp://mi.mirror.garr.it/mirrors/archlinux/testing/os/i686
debug: config: new section 'core'
debug: registering sync database 'core'
debug: opening database 'core'
debug: opening database from path '/var/lib/pacman/sync/core/'
debug: adding new server URL to database 'core': ftp://mi.mirror.garr.it/mirrors/archlinux/core/os/i686
debug: config: new section 'extra'
debug: registering sync database 'extra'
debug: opening database 'extra'
debug: opening database from path '/var/lib/pacman/sync/extra/'
debug: adding new server URL to database 'extra': ftp://mi.mirror.garr.it/mirrors/archlinux/extra/os/i686
debug: config: new section 'community'
debug: registering sync database 'community'
debug: opening database 'community'
debug: opening database from path '/var/lib/pacman/sync/community/'
debug: adding new server URL to database 'community': ftp://mi.mirror.garr.it/mirrors/archlinux/community/os/i686
debug: config: new section 'kdemod-core'
debug: registering sync database 'kdemod-core'
debug: opening database 'kdemod-core'
debug: opening database from path '/var/lib/pacman/sync/kdemod-core/'
debug: adding new server URL to database 'kdemod-core': http://kdemod.iskrembilen.com/repo/core/i686
debug: config: new section 'kdemod-extragear'
debug: registering sync database 'kdemod-extragear'
debug: opening database 'kdemod-extragear'
debug: opening database from path '/var/lib/pacman/sync/kdemod-extragear/'
debug: adding new server URL to database 'kdemod-extragear': http://kdemod.iskrembilen.com/repo/extragear/i686
debug: config: new section 'archlinuxfr'
debug: registering sync database 'archlinuxfr'
debug: opening database 'archlinuxfr'
debug: opening database from path '/var/lib/pacman/sync/archlinuxfr/'
debug: adding new server URL to database 'archlinuxfr': http://repo.archlinux.fr/i686
debug: config: new section 'unstable'
debug: registering sync database 'unstable'
debug: database dir '/var/lib/pacman/sync/unstable' does not exist, creating it
debug: opening database 'unstable'
debug: opening database from path '/var/lib/pacman/sync/unstable/'
debug: returning error 10 from _alpm_db_open : could not open database
debug: returning error 10 from _alpm_db_register_sync : could not open database
debug: returning error 12 from alpm_db_setserver : database not initialized
debug: closing database 'testing'
debug: unregistering database 'testing'
debug: closing database 'core'
debug: unregistering database 'core'
debug: closing database 'extra'
debug: unregistering database 'extra'
debug: closing database 'community'
debug: unregistering database 'community'
debug: closing database 'kdemod-core'
debug: unregistering database 'kdemod-core'
debug: closing database 'kdemod-extragear'
debug: unregistering database 'kdemod-extragear'
debug: closing database 'archlinuxfr'
debug: unregistering database 'archlinuxfr'
This task depends upon

Closed by  Xavier (shining)
Sunday, 31 May 2009, 15:46 GMT
Reason for closing:  Fixed
Additional comments about closing:  The original request was already fixed in 3.2.2, the "unstable" problem is fixed in git by commit 472e51b975b9cb8fe1f67c03ff72bbc067fa7f01 .
Comment by Xavier (shining) - Monday, 17 November 2008, 09:30 GMT
Just FYI, I remember this issue being mentioned before : unstable was a file instead of a directory.
This is very weird, and I have no clue how it is possible. But it is not a totally isolated issue.
Comment by Nagy Gabor (combo) - Monday, 17 November 2008, 15:43 GMT
I will create a patch for handling the error returned by alpm_db_register_sync in the front-end.
Comment by Dan McGee (toofishes) - Tuesday, 18 November 2008, 02:27 GMT
  • Field changed: Status (Unconfirmed → Assigned)
  • Field changed: Due in Version (Undecided → 3.2.2)
  • Task assigned to Dan McGee (toofishes)
Patch is currently under review on the ML for this issue.
Comment by Nathan Wayde (kumyco) - Friday, 09 January 2009, 12:22 GMT
getting same issue in Pacman v3.2.2 - libalpm v3.1.1

i'm not sure if this is an issue with pacman itself or not
because the database file appears corrupted(or just empty) ftp://ftp.archlinux.org/unstable/os/i686/unstable.db.tar.gz
though i'd assume pacman should just end up with the origin(delta)
Comment by Dan McGee (toofishes) - Friday, 09 January 2009, 12:22 GMT
Any ideas from the original patch creator?
Comment by Nathan Wayde (kumyco) - Friday, 09 January 2009, 13:26 GMT
i looked at an strace of the failed update

open("/var/lib/pacman/sync/unstable", O_WRONLY|O_CREAT|O_EXCL|O_LARGEFILE, 0755) = -1 EEXIST (File exists)
lstat64("/var/lib/pacman/sync/unstable", {st_mode=S_IFDIR|0755, st_size=1024, ...}) = 0
rmdir("/var/lib/pacman/sync/unstable") = 0
open("/var/lib/pacman/sync/unstable", O_WRONLY|O_CREAT|O_EXCL|O_LARGEFILE, 0755) = 8
umask(022) = 0
close(8) = 0
umask(022) = 022
close(7) = 0
brk(0x867a000) = 0x867a000
unlink("/var/lib/pacman/unstable.db.tar.gz") = 0
open("/var/lib/pacman/sync/unstable/.lastupdate", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = -1 ENOTDIR (Not a directory)

i think that is relevant
following the rmdir(), shouldn't there be an mkdir() before the open()?
or maybe no rmdir of the tree root
as the directory was re-created earlier

access("/var/lib/pacman/sync/unstable", F_OK) = -1 ENOENT (No such file or directory)
mkdir("/var/lib/pacman/sync/unstable", 0755) = 0
umask(022) = 0
open("/var/lib/pacman/sync/unstable/", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 4
Comment by Nagy Gabor (combo) - Friday, 09 January 2009, 13:51 GMT
Due to empty unstable.db.tar.gz the first -Sy creates an unstable regular file (not dir), and pacman doesn't detect this bug. (It works as if unstable database were empty, which is in fact true. :-) At least it turned out, when pacman creates regular file instead of dir (ref.: first comment of Xav).

The second -Sy attempt ends up with "Could not register ..." error. (Because the unstable file is sitting in place of unstable dir).

So we have 2 bugs here:
1. Empty unstable.db.tar.gz mess things up.
2. Pacman doesn't report unsuccessful -Sy
Comment by Nathan Wayde (kumyco) - Friday, 09 January 2009, 14:05 GMT
someone might wanna take a look at `int _alpm_db_populate(pmdb_t *db)'
it returns `0' if readdir fails, the only caller in 'libalpm/'
`int _alpm_db_load_pkgcache(pmdb_t *db)' checks for `-1'
(failure) which is returned for other failure within the loop
maybe initialize %count to -1
then following the loop, check if it's -1, return -1
otherwise increment %count by 1
and continue the sorting ...
Comment by Nagy Gabor (combo) - Friday, 09 January 2009, 14:21 GMT
This is not our main bug, but I found the following in _alpm_db_register_sync() (and local):

if(stat(path, &buf) != 0 || !S_ISDIR(buf.st_mode)) {
_alpm_log(PM_LOG_DEBUG, "database dir '%s' does not exist, creating it\n", path);
}
alpm_makepath(path);
...

If the (database) path exists, but not a dir, we cannot create the path, so db_register will stop with an error, and our front-end could not even *register* the database, so -Sy cannot solve the user's database issue. The only question is, if we have a non-dir path here, should we remove it or not? (Removing things without user confirmation may be dangerous.)
Comment by Nagy Gabor (combo) - Friday, 09 January 2009, 14:40 GMT
I think, I found the problem.

_alpm_db_update() simply calls _alpm_unpack() with the db-path param without first creating it. Probably if the db.tar.gz file is not empty, libarchive does this job, but if we have an empty archive, it doesn't.
Comment by Nagy Gabor (combo) - Friday, 09 January 2009, 15:19 GMT
No, I was wrong, forget my last comment.
Comment by Nagy Gabor (combo) - Friday, 09 January 2009, 16:43 GMT
Something really strange happening here. Since I am not a libarchive expert, I attach my patch here. This maybe a libarchive bug.

Briefly: When you call _alpm_unpack() on unstable.db.tar.gz from _alpm_db_update(), our "unstable" empty dir still exists (checked). (Created by db_register). Now during extraction we overwrite(?!) the destination dir to a regular file, when extracting "./" from the archive. So my patch skips the extraction of "./", no new failing pactests. Clearly, this reasoning makes not much sense (why don't we overwrite the destination dir in the usual case, "./" is never the first entry?), but the attached patch works...
Comment by Nagy Gabor (combo) - Friday, 09 January 2009, 16:54 GMT
Testing my patched pacman with --debug, I noticed, that during core.db.tar.gz extraction there is no "debug: skipping: ./" line; during extraction of unstable.db.tar.gz we have that line.
Comment by Nagy Gabor (combo) - Saturday, 10 January 2009, 13:20 GMT
I realized, that there is a bug in my patch. Namely, when fn != NULL, we should indicate "not found" with !=0 retval. But this is not important here.
Comment by Xavier (shining) - Sunday, 18 January 2009, 12:36 GMT
It seems really weird to have ./ in an archive, this is normally never the case.
So because of that, I don't know if we should really add a special check for it.

I was wondering if we could change the way prefix is dealt with. What if we just changed the working directly to prefix (assuming it was created. or maybe we could create it otherwise), and then extract files normally. Maybe the extraction of ./ would not hurt anything if we did that.
Comment by Xavier (shining) - Sunday, 18 January 2009, 13:06 GMT
Here is a patch for the above suggestion. I don't really understand, but I just followed my intuition and it worked :)
Now the unstable directory will stay empty after the extraction, just like when we use bsdtar manually.

However, if an unstable file was previously created because of this bug, it still needs to be removed manually.
And in this case, the "cannot register db" error is quite annoying and is preventing every operation. But this is a separate problem.
Comment by Andi Clemens (TheGrudge) - Sunday, 31 May 2009, 11:07 GMT
I just discovered this problem today. I added unstable and after that pacman is broken:
error: could not register 'unstable' database (could not open database)

I can't do anything but delete the unstable folder again.
The unstable download was only 1kb, which I found very strange, and then the problem occurred.
Comment by Andi Clemens (TheGrudge) - Sunday, 31 May 2009, 11:08 GMT
Oh forgot to say that unstable is a file, not a folder in my case.
Comment by Nathan Wayde (kumyco) - Sunday, 31 May 2009, 11:15 GMT
fyu, the [unstable] was dropped last year (which is why the file is empty).
Comment by Nagy Gabor (combo) - Sunday, 31 May 2009, 12:52 GMT
TheGrudge: This bug has been fixed in git, so this won't happen with pacman 3.3.

Loading...