Arch Linux

It assume that /tmp if mounted is tmpfs (or ramfs). I think for this case is needed to add/replace as test:

findmnt -k -t notmpfs,noramfs /tmp

Gerardo is right. The assumption was that it would be mounted as tmpfs. This is obviously not necessarily the case, so we should either check if it is tmpfs or just revert to deleting everything.

That said, I would like to add that the proper fix is to mount tmp as tmpfs (and convert whatever partition you used to have it on into swap space). Is there a use case where this is not the best solution (I would be very interested to know)?

I'm surprised that nowadays the general assumption is that /tmp should be mounted as tmpfs. Does the Arch Installer default to creating /tmp as tmpfs, and have other distributions started doing the same?

I agree that a check is the best idea, especially since many older systems probably mount /tmp on a storage device.

Computationally intensive applications, especially graphic design and photography software often store several gigabytes of information on the /tmp partition. Gimp stores a scratch disk in /tmp, and I recently finished a hugin panorama requiring approximately 12GB of /tmp scratch space.

I did some brief research and the default tmpfs partition can grow to half the available memory after which data will be swapped out.

However, these same applications often use considerable memory, and their /tmp scratch spaces are specifically designed to accommodate any data that will not fit into memory. I suspect that reducing the amount of memory - which is already at a premium on 32-bit systems, at a max of 3GB - will reduce performance and might even hasten thrashing.

On the other hand, the maximum size of the tmpfs partitions could be set to a small value such as 128MB, after which writing to /tmp becomes synonymous to writing to swap space. Which makes it necessary to compare features, performance and efficency of using tmpfs to write to a swap partition versus writing to a storage device using standard file-systems (ext/xfs/jfs/btfs...). In both cases the kernel is responsible for managing the disk, so tmpfs presents no benefits to databases. Aside from the spurious performance benefits gained by assuming (possibly incorrectly) the often-accessed and small locks and pipes residing in /tmp will remain in memory and not be swapped out, I see no reason to prefer one approach over the other.

Off the top of my head I can think of a spurious security issue. Does swap storage (hence tmpfs) properly implement delete, or will the data simply reside in swap space till overwritten?

Also, having /tmp as a traditional partition allows imposing immutable quotas, whereas tmpfs shares swap space with memory. For example:
As a partition, /tmp space can be limited to the maximum size of the partition
As tmpfs, /tmp space becomes tmpfs size + swap size. There is no way of preventing /tmp from accessing swap space reserved for memory.

Granted, I don't think this last point is of any importance.

Thanks for your response.

We do not assume that tmp is tmpfs. It was just a mistake I made when adding the check to think that there were no other options than "no mountpoint" and "mounted as tmpfs". I'm not sure what other distro's do, but I believe it is common to mount /tmp as tmpfs.

A few clarifications about tmpfs:

* size= denotes the total size of the partition, and the partition will not grow beyond this. E.g.:
# mount -t tmpfs none -o size=1m /mnt/test
# cp /boot/kernel26.img /mnt/test
cp: writing `test/kernel26.img': No space left on device
cp: failed to extend `test/kernel26.img': No space left on device

* creating a tmpfs partition will not mean that you reserve memory for it. Only the data you write to it will take up space in mem/swap. I.e., if you were to convert your partition to swap space, you will have gained some swap space (whatever is not actually being used by /tmp) and not have lost any memory.

* swap does not "delete" your data, but then I don't think any of the common file systems do either. What you can do is to use an encrypted swap partition (with a randomly created key at each boot). This will make the swap meaningless once the power is switched off (without the need to clear the data).

* the big benefit of using tmpfs compared to a regular file system is that the kernel knows that you don't care if it saves the data to disk. A normal file system will write data to disk both in case it needs some more memory (in this case the tmpfs will write the data to swap), but also a normal file system will write data to disk at regular intervals to make sure it is not lost in case of a crash. As an example, imagine you have some gigs of data in memory that reside on /tmp when you shutdown your machine. In case of /tmp being on tmpfs all this data is simply ignored. In case /tmp is on a regular file system all the data has to be written to disk before the computer will shut down (and on boot it is all deleted again).

There are cases in which using tmpfs is not ideal (e.g., it does not yet support user quotas), but I do not think this is relevant in your case.

I will in any case add a fix to this issue for the next release.

Instead of checking if /tmp is a mount point (assuming it's mounted as tmpfs) and upon that decide to delete or not its content, considering that we're talking about directories and files deletion (meaning that checking for mount point isn't relevant) isn't better to have a boolean variable in /etc/rc.conf like CLEANTMP?
By doing so if it's set to "yes" in init script you perform rm's stuff else you left /tmp as it is (if the user stores data for temporary usage, which doesn't mean with a lifetime of a single reboot, he could clean it by himself or in /etc/rc.local or with cron or whatever he wants).

@ff0000.it: As much as possible I don't want to add any more parameters to rc.conf. Probably the best thing to do in this case is to delete /tmp unconditionally, there was no improvement by adding the conditional (a similar conditional was needed for /var/{run,lock}, but not for /tmp).

@tomegun: i understand your will to keep /etc/rc.conf simple as much as possible, but (my opinion) is a good behaviour that a distro delete files and directory without letting a user choosing about? And for choosing i don't mean editing a bash script ;-)
I've got another one: supposing that a user is working with his temporary data in /tmp (yes, a traditional filesystem ;-)) and the o.s. crashes, then come a reboot and the (unstoppable?) deletion of /tmp content that leads the user to loose all of he was working on; should be better considering to clean up /tmp during the shutdown (/etc/rc.shutdown)?

The semantics of /tmp is well defined (e.g. <http://www.pathname.com/fhs/2.2/fhs-3.15.html>).

If we start supporting other usages than what it is designed for we will have a maintenance nightmare.

By putting something in /tmp or /var/tmp you say that as soon as the program that created the files close, you no longer need them, and the system is allowed to delete them.

The difference between /tmp and /var/tmp is that stuff in /tmp will never be used twice whereas stuff in /var/tmp could be used twice, but if it is lost it is not important (e.g. a cache that could be regenerated).

Other directories like /run, /var/run and /var/lock have similar semantics, and if you put stuff in them it can and will get deleted behind your back.




If you use /tmp in such a way that having it on tmpfs is not the best solution, then you probably should not be using /tmp. However, I could be wrong, so if you have a specific use-case I'd be interested to know.

@tomegun: i get it! Thanks for that link :-)
I don't use /tmp in any specific case, mine was just an hypothesis and now that i red fhs it's gone forever ;-)

@ff0000.it: ok :-)

@tomegun: I know you said that you'll fix it, and I'm very happy to hear that, but I did want to present to you one case where tmpfs is not as suitable: big files at the outer edge of virtual memory usage.

For instance, rdiff-backup will use /tmp at times, and when you're backing up, say, VirtualBox images, /tmp can fill up right quickly. It's one thing to have a dedicated filesystem fill up, but it's quite another to have tmpfs start filling RAM and swap, especially when you may have one or more preexisting memory-hungry processes.

The difference is that in the first instance, rdiff-backup dies with an out-of-space error (easy to diagnose), while in the second either rdiff-backup dies with either an out-of-space or an out-of-memory error (potentially difficult to diagnose) -or- some other, unrelated, and otherwise harmless process dies with an out-of-memory error (unacceptable behavior).

@dhaines: thanks for the feedback, I'm still interested to hear more use-cases like this.

I'm not sure I understand your scenario though. Imagine you have a 2G (just a random number) disk where /tmp is mounted, you also have a 2G swap disk and 2G ram (so you will get ENOMEM if you use more than 4G of virtual memory).

Now mount /tmp as tmpfs with size=2G instead, and use the underlying disk as swap (giving you a total of 4G of swap, and ENOMEM if you use more than 6G of virtual memory).

If your process fills more than 2G into /tmp it will get ENOSP. The only thing that will be different is that the stuff you put on /tmp could use up your swap space, but it could at most use 2G more swap than before, and that's what you added, so you are no more likely to get ENOMEM (for any process) than before.


After looking around a bit it looks like the only problem you might hit is if you have any particular file being larger than 256GB (on 64bit, 2TB on 32bit), the tmpfs mount itself can be as big as 16EB (on 64 bit, 16TB on 32bit), so that should hopefully not be a problem.

If you are in a situation where you need larger than 256GB files, transparent hugepages might help, but I'm not up to speed THP development and how far they have come. Is this your case?

Sorry for my ignorance, but isn't it a bad idea to have /tmp as tmpfs in the first place? What about crash/accident recovery?

There is no real advantage, in terms of performance, of having /tmp in RAM on a desktop system. On a build server, it is true that compiling can benefit from the speedup provided by tmpfs, but one could just remount /tmp by hand in such cases...

@lisaev:

1) tmpfs provide the semantics that /tmp guarantee (look at the above link to FHS, not that we are following it religiously, but in this case it makes a lot of sense). If your program crashes it should assume that whatever was in /tmp is lost forever (even if the OS itself does not crash). Any important data (such as backups/snapshots) should be stored elsewhere.

2) using tmpfs is particularly useful on laptops where you want to conserve power. Since there are no requirements to preserve data in case of a crash, data will be written to disk much less frequently (only when low on RAM), so the disk will spin up less often, and it will free up io bandwidth for other processes.

In general, using a backing fs that provide stricter guarantees (like persistence) than what your directory requires, will obviously come at a cost.

@tomegun:

You make a very good and valid point.

I would only say in response that it may be desired that your swap partition(s) not be large enough to accommodate large files on /tmp, but rather that the user may desire a large ext4/reiserfs/jfs/other_general-purpose_filesystem partition for usage on /tmp, if nothing else for the ability to remount it and use it as something other than /tmp. You can't do that with swap without reformatting.

Really, all of this is academic at best, as you've already said that you'll update rc.sysinit to wipe /tmp in any event. Essentially, it boils down to user preference. I think I might try out your tmpfs solution with the size= option. I'll report back if I notice any difference in operation.

Thanks for the enlightening discussion.

Well, I didn't mean that the data from /tmp should be reused by an application, but rather by a user trying to understand why the system keeps crashing (booting from a livecd, for instance)... So, /tmp should be cleaned, but only if everything goes well.

I thought that these considerations motivated erasing /tmp/* in rc.sysinit, as opposed to rc.shutdown. I don't know about other modern systems, but at least RHELs 5.x (client and server alike) do not clean /tmp at all.

@lisaev: I still don't think that is a useful thing to do (e.g., logs sholud be in /var/log). There might of course be cases where the contents of /tmp might be useful for debugging, but I really don't think it is worth it.

I'm not sure about RHEL, but fedora 15 (so I guess also eventually RHEL) deletes files from /tmp both at boot and when they have not been touched for 10 days.

@dhaines: I don't thinke there is such a big difference between doing "rm -rf * .*" or "mkfs.XXX" on a device, especially as I assume it is a rare occurrence.

Thanks to everyone who took part in the discussion, it has been very useful. I think the best solution which should accommodate everyone (but which will be most convenient for what should be the standard setup) is the following (I'll propose this to arch-projects and see if they agree before implementing it):

1) recommend everyone to put /tmp on tmpfs (and tell them to possibly add a swap partition and set a size= accordingly)
2) never touch /tmp at all in rc.sysinit, which is the right thing to do for tmpfs (the fewer "rm -rf *" I see in code that I maintain and that is ran on lots of machines, the happier I am)
3) advice people who choose to use something different than tmpfs to add "rm -rf /tmp/* /tmp/.*" to their rc.local, or in some other way maintain the contents of /tmp as they wish (I believe there was a tmpwatch package in AUR, maybe it is still there...).

Tom, if you want /tmp to be in RAM by default, it's OK with me (however, I would personally use memory for something more useful). The only thing left to do then is to put a condition on those <mkdir -p /tmp/.{ICE,X11}-unix> lines, which currently produce an error on boot.

Nah, I was convinced by falconindy that making this change is not worth the effort. I'll just reinstate the old behavior.

@lisaev: this was the misconception I was trying to correct. By having /tmp on tmpfs you are not using any more memory. Not a single byte.

@tomegun: after this discussion, a variable to clear /tmp in /etc/rc.conf should be so ugly at all? This is my last try, after that I dismiss... I promise! ;-D
@ALL: however, this give and take has been useful, thanks! :-)