FS#22896 - "returning error 1 from local_db_populate : out of memory!" on i686 netbook

Attached to Project: Pacman
Opened by h (Cdh) - Monday, 14 February 2011, 21:52 GMT
Last edited by Allan McRae (Allan) - Thursday, 17 February 2011, 11:01 GMT
Task Type Bug Report
Category Backend/Core
Status Closed
Assigned To No-one
Architecture i686
Severity High
Priority Normal
Reported Version git
Due in Version 3.5.0
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Summary and Info:

On 64 bit everything still works fine but on my 32 bit netbook I got a problem with installing or upgrading packages.

"pacman -Suu --debug" ("pacman -S" too) gives
http://aur.pastebin.com/1AEGeR4K

I am pretty sure that I am not "out of memory"...

Here the culprit got introduced:
http://mailman.archlinux.org/pipermail/pacman-dev/2011-February/012402.html

I don't know the code of pacman but there seems to be another reason that _alpm_pkghash_create(est_count * 2) returns NULL. Before that commit (03.02.2011) it worked fine, after the commit (08.02.2011) it doesn't, though I have not checked yet if _alpm_pkghash_create returned NULL before.
This task depends upon

Closed by  Allan McRae (Allan)
Thursday, 17 February 2011, 11:01 GMT
Reason for closing:  Fixed
Additional comments about closing:  git commit d1cc1ef6
Comment by Allan McRae (Allan) - Monday, 14 February 2011, 22:03 GMT
The relevant error is:
error: database larger than maximum size
(which needs a newline added at the end...)

There seems to be something wrong with the estimation of local db size instead of a out-of-memory error. This would only be noticed after the commit you pointed out as previously it would have just returned a zero size hash table that would have been rehashed as needed.

Can you add:
_alpm_log(PM_LOG_ERROR, "requested size: %zd\n", size);
after the previous error (line 74, pkghash.c) so we can see what size is being requested.

Comment by Allan McRae (Allan) - Monday, 14 February 2011, 22:10 GMT
Also, post the output of "ls /var/lib/pacman/local/ | wc -l"
Comment by h (Cdh) - Monday, 14 February 2011, 22:40 GMT
---snip---
debug: checking for package upgrades
debug: loading package cache for repository 'local'
error: database larger than maximum sizeerror: requested size: -2
debug: returning error 1 from local_db_populate : out of memory!
---snip---

chris@chriseee ~ % ls /var/lib/pacman/local | wc -l
1362
Comment by Allan McRae (Allan) - Monday, 14 February 2011, 22:45 GMT
What filesystem is you /var/lib/pacman/local on? Anything out of the ordinary like a loopback?
Comment by h (Cdh) - Monday, 14 February 2011, 22:49 GMT
It's btrfs with compression and space_cache.

On my 64 bit machine it's ext4... That's another difference, yes.
Comment by Dan McGee (toofishes) - Monday, 14 February 2011, 23:20 GMT
-2 comes from (0 - 2), where we are seeing the stat buffer st_nlink field be set to zero rather than the "right" amount. Interesting.

Can you post the output of `ls -l /var/lib/pacman/` as well as `stat /var/lib/pacman/local/`?

total 72
drwxr-xr-x 1171 root root 65536 Feb 11 09:23 local
drwxr-xr-x 2 root root 4096 Feb 11 09:16 sync
File: `/var/lib/pacman/local/'
Size: 65536 Blocks: 136 IO Block: 4096 directory
Device: 901h/2305d Inode: 1917424 Links: 1171
Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2011-01-09 10:39:40.000000000 -0600
Modify: 2011-02-11 09:23:35.000000000 -0600
Change: 2011-02-11 09:23:35.000000000 -0600
Birth: -

Comment by Dan McGee (toofishes) - Monday, 14 February 2011, 23:24 GMT
Well found the answer we were (not) looking for: http://kerneltrap.org/mailarchive/linux-btrfs/2010/1/23/6723483/thread
Comment by Dan McGee (toofishes) - Monday, 14 February 2011, 23:50 GMT
Give this a try, it should make this code a bit more robust.
   alternate-count-method.patch (1.5 KiB)
Comment by h (Cdh) - Tuesday, 15 February 2011, 00:05 GMT
Yes, looks good.
At least pacman -Syu and -U worked fine now.

And don't forget the missing newline... :)
Comment by Dan McGee (toofishes) - Tuesday, 15 February 2011, 22:39 GMT
I have no idea what you are alluding to with the "missing newline" comment.
Comment by h (Cdh) - Tuesday, 15 February 2011, 22:45 GMT
Sorry, not relevant to the bug. Therefore the smiley.

The first comment, referring to the line in the original debug output:

error: database larger than maximum sizedebug: returning error 1 from local_db_populate : out of memory!

should rather be

error: database larger than maximum size
debug: returning error 1 from local_db_populate : out of memory!
Comment by Dan McGee (toofishes) - Tuesday, 15 February 2011, 22:47 GMT
Aha- it is relevant enough, I'll patch that when I fix this.

Loading...