FS#11284 - [squashfs-tools] Hang with mksquashfs and large filesystem

Attached to Project: Arch Linux
Opened by Denis Martinez (denis) - Friday, 22 August 2008, 12:39 GMT
Last edited by Paul Mattal (paul) - Saturday, 06 February 2010, 23:12 GMT
Task Type Bug Report
Category Upstream Bugs
Status Closed
Assigned To Aaron Griffin (phrakture)
Architecture All
Severity Low
Priority Normal
Reported Version None
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Description:
I have setup a chrooted Arch install in a directory, and I want to compress it to squashfs. The directory is ~4GB. So, I ran mksquashfs, with no additional parameters:
# mksquashfs file.sqfs squash/
Then the program initializes, the percentage bar advances and gets stuck at ~80%. There is no more activity on any of my CPU cores.

Additional info:
Version 3.3-1


Ubuntu mentions a patch on launchpad, but I haven't had time to try it yet:
(the bug is possibly related to multithreading)
https://bugs.launchpad.net/ubuntu/+source/squashfs/+bug/222700

I will try to apply the patch ant post more details later.
This task depends upon

Closed by  Paul Mattal (paul)
Saturday, 06 February 2010, 23:12 GMT
Reason for closing:  Fixed
Comment by Denis Martinez (denis) - Friday, 22 August 2008, 16:18 GMT
I retried with the patch applied and it works
Comment by Gerardo Exequiel Pozzi (djgera) - Saturday, 06 June 2009, 17:20 GMT
status with latest 4.0-1?
Comment by Tobias Powalowski (tpowa) - Monday, 15 June 2009, 20:27 GMT
i don't use squashfs at all, aaron please have a look at it.
Comment by christopher rogers (godane) - Saturday, 03 October 2009, 17:38 GMT
I believe this bug should be fix in 4.0. This is only cause this bug was in squashfs tools 3.3. It would have been fix in 3.4 release based on the patch date.
Comment by christopher rogers (godane) - Saturday, 03 October 2009, 17:44 GMT
Sorry about double posting. I didn't see the date of the last ubuntu bug message. But here is a patch that may fix it:
http://squashfs.cvs.sourceforge.net/viewvc/squashfs/squashfs/squashfs-tools/mksquashfs.c?r1=1.84&r2=1.85

I think we can remove the mksquashfs name change since that part of the patch will not work or matters for the fix.
Comment by christopher rogers (godane) - Saturday, 03 October 2009, 17:58 GMT
Never mind. That patch is in squashfs 4.0. So maybe its all fix like i thought it was in the first post.
Comment by Aaron Griffin (phrakture) - Monday, 05 October 2009, 21:31 GMT
Hmm, @godane do you know if this works now? I don't have the time to generate a 4GB dir to test this right now...
Comment by Dario Freddi (drf) - Sunday, 08 November 2009, 23:50 GMT
I can confirm this using unsquashfs as well. Unsquashing a modest 170MB squashfs takes up 424MB of RAM with valgrind telling me that only 4 of them are useful. Moreover, any option supplied "like -p 1" results in a floating point error. I'd say there is something wrong
Comment by Paul Mattal (paul) - Tuesday, 05 January 2010, 04:30 GMT
I was able to create a 3.3GB squashfs using 4.0-1 on i686. I found the syntax I needed was backwards from the above (mksquashfs /sourcedir my_new.sqfs) but otherwise worked fine.

I recommend closing this bug as fixed on February bug day, unless godane or drf report in before then that some behavior is still broken for them with 4.0-1.

I agree that the memory usage is sometimes large, but it does not grow boundlessly, nor seem to vary directly with image size. Maybe the squashfs compression algorithm grabs more RAM if it looks like you have a lot, in the hopes of increasing your compression ratio?

Loading...