FS#8155 - pacman crashes in 64MB environment

Attached to Project: Pacman
Opened by Tobias Powalowski (tpowa) - Friday, 28 September 2007, 16:40 GMT
Last edited by Dan McGee (toofishes) - Thursday, 04 September 2008, 15:18 GMT
Task Type Bug Report
Category General
Status Closed
Assigned To Aaron Griffin (phrakture)
Dan McGee (toofishes)
Architecture All
Severity Medium
Priority Normal
Reported Version 3.0.6
Due in Version 3.2.0
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

While trying to install with 64MB RAM.
pacman installation of packages is just stopping in the middle of installation and exits.
dmesg bumps up a huge amount of memory stuff.

Adding additional swap to RAM doesn't help either.
I don't know if pacman just has bad memory usage at this point.
Installing packages one after the other is working so something in memory usage seems to trigger this.
Perhaps you ahve an idea why it doesn't work.
I used pacman-3.0.6.
This task depends upon

Closed by  Dan McGee (toofishes)
Thursday, 04 September 2008, 15:18 GMT
Reason for closing:  Fixed
Additional comments about closing:  Works for us, doesn't seem to be an issue anymore with proper environment setup.
Comment by Aaron Griffin (phrakture) - Friday, 28 September 2007, 16:47 GMT
Could you provide us with:

a) the actual output you're talking about (dmesg)
b) the --debug output
c) a way to replicate this bug

Thanks
Comment by Tobias Powalowski (tpowa) - Friday, 28 September 2007, 17:10 GMT
boot with mem=64M and try to install all core packages to a root directory
Comment by Aaron Griffin (phrakture) - Friday, 28 September 2007, 17:14 GMT
Thanks, adding it to my list of things to look into this weekend.
Comment by Tobias Powalowski (tpowa) - Friday, 28 September 2007, 17:19 GMT
additional info isntall environmnt size is 20MB + 8 mb ramdisk + 1.7 mb kernel.
so 30 megs for pacman running available.
Comment by Dan McGee (toofishes) - Thursday, 04 October 2007, 03:47 GMT
When I was testing this scenario with pacman 3.1, I never saw it using more than 23 or so MB of memory. There could have been some changes since the 3.0.X split that account for this, but I'm not sure. I haven't tested it in a restricted memory environment however.
Comment by Tobias Powalowski (tpowa) - Thursday, 04 October 2007, 05:43 GMT
i think it depends on how many packages are installed at once.
Comment by Gerhard Brauer (GerBra) - Sunday, 07 October 2007, 14:24 GMT
For your info, I've done at the moment a test installation on a VM with only 48MB RAM. Install with the new ISO and lowmem initrd works like a charm.
Also never have a pacman problem, neither during installation nor when installing for ex. xorg-server and it's dependencies.
During this operation the system never use swap space. Maybe only a problem if one have low memory (X running, etc) during pacman usage.
Comment by Tobias Powalowski (tpowa) - Sunday, 07 October 2007, 19:43 GMT
how many packages did you install?
Comment by Gerhard Brauer (GerBra) - Sunday, 07 October 2007, 21:11 GMT
Sorry, at the moment i could not say it exactly, but
- during installation base+support (0 byte swap)
- xorg-server, xterm (and deps) (0 byte swap)
- xfce4-group (and deps) (about 25MB Swap, xorg+icewm running)
Maybe about 15-20 packages per run.
If it make a sense i could try tomorrow for ex. a complete install of kde.
Comment by Tobias Powalowski (tpowa) - Monday, 08 October 2007, 05:05 GMT
i tested only with all packages selected at once.
Comment by Gerhard Brauer (GerBra) - Monday, 08 October 2007, 11:51 GMT
I've tested now with:
pacman -Sv kde gnome (would work dep check etc., i don't do the real install)

Then something one would never do in real life:
pacman -Sl extra | cut -d " " -f1 | xargs pacman -Sv
and
pacman -Sl extra | cut -d " " -f1 | xargs pacman -Sdfv
So pacman check for dependencies conflicts and inter-conflicts for about ~2300 packages.
Cause there ARE conflicts, above never got installed.
But i think these are the actions were pacman needs most resources and oom-killer would come.
I've never hat more swap than ~25MB, pacman itself uses about 10-15MB VIRT.
Comment by Tobias Powalowski (tpowa) - Monday, 08 October 2007, 13:28 GMT
the oom kill doesnt happen during intitialisation it comes while installing packages, suddenly
Comment by Aaron Griffin (phrakture) - Monday, 08 October 2007, 17:12 GMT
tpowa: if you want our tests to be the same, can you provide us with the EXACT list of packages? I was under the assumption the installer simply did a "pacman -S base" because that makes more sense to me - if it does not, can you provide us with a list?
Comment by Tobias Powalowski (tpowa) - Tuesday, 09 October 2007, 05:10 GMT
it installs everything from packages.txt, because i selected all packages.
it also happens with quickinst so it's nothing setup specific.
quickinst only installs base
Comment by Aaron Griffin (phrakture) - Tuesday, 09 October 2007, 15:37 GMT
tpowa, can you try with pacman.static? I'm curious if this is loaded libraries causing the excess memory overhead.
Comment by Tobias Powalowski (tpowa) - Saturday, 20 October 2007, 15:18 GMT
pacman.static also fails, i used no swap.
Comment by Dan McGee (toofishes) - Saturday, 19 January 2008, 02:44 GMT
This should be fixed by our master (3.2 development) branch, as we just made a lot of changes related to backend memory allocation (static to dynamic) as well reducing duplication of entire pmpkg_t structs in memory (90% of these dups were unnecessary).

tpowa- if i give you a pacman.static from our development tree, could you test it out? It should have no problem even installing all of [core] (base, base-devel, etc. all at once).
Comment by Dan McGee (toofishes) - Thursday, 24 January 2008, 04:23 GMT
I think we can safely say this should be fixed in 3.2. I blogged about it, actually, as massif changed a bit and I wanted to rediscuss it.

http://toofishes.net/blog/valgrind-330-and-new-massif/
Comment by Aaron Griffin (phrakture) - Thursday, 24 January 2008, 16:49 GMT
Woot, go Dan!
Comment by Tobias Powalowski (tpowa) - Thursday, 24 January 2008, 18:10 GMT
sounds pretty good, dan when will it be ready or is it backported to 3.1 branch?
new isos should come finally with 2.6.24 kernel and with this fix it would be so great.

greetings
tpowa
Comment by Dan McGee (toofishes) - Thursday, 24 January 2008, 19:22 GMT
Definitely NOT backportable. This is a huge change that needs a lot more testing to ensure we don't have any memory allocation issues.

Speaking of new ISOs, did you ever see the archiso scripts that Simo and I got working quite nicely?
http://archlinux.org/pipermail/arch-dev-public/2007-October/002616.html
http://projects.archlinux.org/git/?p=archiso.git;a=summary
Comment by Xavier (shining) - Thursday, 24 July 2008, 09:21 GMT
Maybe this can be closed now that archiso is in use, pacman 3.2 is close to release, and the next official iso in preparation might have it.
Comment by Gerhard Brauer (GerBra) - Thursday, 24 July 2008, 10:00 GMT
From my view: close it.
Maybe i/we could make some tests with lowmem machines with new iso's and then reopen or file a new report if this become important.
Comment by Gerhard Brauer (GerBra) - Wednesday, 30 July 2008, 21:52 GMT
Sorry for warming up this thread, but i have found some interesting news about this task.
I have had the need for testing again such a enviroment (VM with 64 MB RAM) and with our current install-iso.

By kernelparameter we use 75% from RAM for ramdisk/unionfs later as root-fs(/). With 64 MB RAM this leads to a
Root with araound 54MB.
Package selection and installation give error during copying (BTW: also with error the installer says: Coping/installing successfully at the end - sounds like a installer bug, i investigate this later)

Next try i used a ramdisk size from 90%. This fails also. The reason is easy - the base package files (and other selected by the user) are copied to /tmp (cause /var/cache/pacman/pkg isn't availabe).
But base occupies alone ~110 MB in /tmp.
And this is the problem - / is getting full.

So after a little thinking i found a solution.
After partitioning and before package selection/installing:
mkdir /mnt/tmp
chmod 1777 /mnt/tmp
mount -o bind /mnt/tmp /tmp

So the package installer has enough space for saving downloaded or copied packages and the installation goes
on without errors. The system and installer itself needs during the whole installation process not more than
~15-17MB.
So this i a chance to install Arch now on machines with very little RAM. I don't know if this method were also possible with the on-topic 2007 ISO ("tpowas"), but with the new ISO it is definitely no problem anymore.

If you agree this could be a hint in the installation guide.
Comment by Aaron Griffin (phrakture) - Wednesday, 30 July 2008, 21:57 GMT
Well, I think the issue here is that the cache directory needs to be created on the host system before installing. Can you try this:

After partitioning try: mkdir -p /mnt/var/cache/pacman/pkg/
(Note: this may be /media and not /mnt, I can't remember)

The install may then use that dir as cache (and cache downloaded packages on the new disk. That's the way it SHOULD work
Comment by Dan McGee (toofishes) - Wednesday, 30 July 2008, 22:22 GMT
For the next installer, this patch should help. It might apply to the current installer too:
http://code.neotuli.net/gitweb/?p=installer.git;a=commitdiff;h=b677b3e44624608e684de6fb4a5a9b2531ac1257;hp=6cd6408df9686534e8c9a8f05d77e4eec2ba32ab

The new pacman (3.2.0) will definitely be better in the memory usage department as it does everything dynamically, which cuts memory usage by a good 50% on most operations over the 3.1.x series.
Comment by Gerhard Brauer (GerBra) - Wednesday, 30 July 2008, 22:44 GMT
Sorry for warming up this thread, but i have found some interesting news about this task.
I have had the need for testing again such a enviroment (VM with 64 MB RAM) and with our current install-iso.

By kernelparameter we use 75% from RAM for ramdisk/unionfs later as root-fs(/). With 64 MB RAM this leads to a
Root with araound 54MB.
Package selection and installation give error during copying (BTW: also with error the installer says: Coping/installing successfully at the end - sounds like a installer bug, i investigate this later)

Next try i used a ramdisk size from 90%. This fails also. The reason is easy - the base package files (and other selected by the user) are copied to /tmp (cause /var/cache/pacman/pkg isn't availabe).
But base occupies alone ~110 MB in /tmp.
And this is the problem - / is getting full.

So after a little thinking i found a solution.
After partitioning and before package selection/installing:
mkdir /mnt/tmp
chmod 1777 /mnt/tmp
mount -o bind /mnt/tmp /tmp

So the package installer has enough space for saving downloaded or copied packages and the installation goes
on without errors. The system and installer itself needs during the whole installation process not more than
~15-17MB.
So this i a chance to install Arch now on machines with very little RAM. I don't know if this method were also possible with the on-topic 2007 ISO ("tpowas"), but with the new ISO it is definitely no problem anymore.

If you agree this could be a hint in the installation guide.
Comment by Gerhard Brauer (GerBra) - Wednesday, 30 July 2008, 23:02 GMT
Oh, uups posted my comment double? Sorry ;-) I hunted a beasty fly and must have pushed some keys on keyboard....

@Aaron:
mkdir -p /mnt/var/cache/pacman/pkg/ doesn't solved it. Still says: couldn't create package cache, using /tmp instead.

It's this in the current installer i think:
-----------
if [ "$delpkg" = "yes" ]; then

- # set up a symlink to fool pacman so it doesn't copy all the packages to the cache

- ln -sf /src/core/pkg /var/cache/pacman/pkg >/dev/null 2>&1

- else
-------------
I used the option "delete packages", so the symlink is created to an ro-only filesystem. If choosing "keep packages" this symlink goes to /mnt/var/.......

But as Dan mentioned: This part is modified in new installer.

Comment by Gavin Bisesi (Daenyth) - Monday, 11 August 2008, 17:46 GMT
This bug is marked as due for 3.2.0
Is it fixed?
Comment by Xavier (shining) - Monday, 11 August 2008, 18:19 GMT
What Dan said on 23 July :
"I'd like to push this to Simo and others to actually do some testing
once we do get 3.2 released, so for now leave it open."

Loading...