FS#18157 - [filesystem] symlink /var/{run,lock} to /run/{,lock}
Attached to Project:
Arch Linux
Opened by Thomas Bächler (brain0) - Thursday, 04 February 2010, 13:05 GMT
Last edited by Tom Gundersen (tomegun) - Tuesday, 13 December 2011, 00:05 GMT
Opened by Thomas Bächler (brain0) - Thursday, 04 February 2010, 13:05 GMT
Last edited by Tom Gundersen (tomegun) - Tuesday, 13 December 2011, 00:05 GMT
|
Details
Many things changed since this request was first opened, the
original problems are solved elsewhere, but there are still
reasons to implement this:
The new tmpfs-mounted /run folder allows programs like udev, lvm and mdadm to keep runtime data from initrd until shutdown. We can move all runtime data to these folders by simply creating symlinks in /var. In addition to consolidating all the runtime data in one place, this will remove needless writes to disk, as well as remove the need to cleanup the directories on boot. Implementing this is still blocked by packages that (mistakenly) rely on data in /var/{run,lock} being persistent after reboot (as described below). Original feature request: ======================== In another bug report, I got the impression that mounting tmpfs for /var/run and /var/lock very early in the initialization process is a good idea and solves some stupid problems when running applications while /var is still readonly. Problems: 1) People might have /var on a separate partition from / 2) Some programs don't work if their /var/{run,lock}/whatever folder is missing - I would consider these bugs, but we need to think about it. We would also have to stop putting /var/{lock,run}/* files or directories into packages. On my system: $ grep -e ^var/run -e ^var/lock /var/lib/pacman/local/*/files | cut -d: -f2|sort -u var/lock/ var/lock/sane/ var/run/ var/run/bftpd/ var/run/ConsoleKit/ var/run/dbus/ var/run/hald/ var/run/hald/hald-local/ var/run/hald/hald-runner/ var/run/httpd/ var/run/lirc/ var/run/mpd/ var/run/mysqld/ var/run/network/ var/run/network/interfaces/ var/run/network/profiles/ var/run/openntpd/ var/run/PolicyKit/ var/run/samba/ I assigned a few people to this who might have an opinion and useful input here, feel free to remove yourself if you don't care. |
This task depends upon
Closed by Tom Gundersen (tomegun)
Tuesday, 13 December 2011, 00:05 GMT
Reason for closing: Implemented
Additional comments about closing: This is now in initscripts in git, please give it a try and reports problems to the ML.
Tuesday, 13 December 2011, 00:05 GMT
Reason for closing: Implemented
Additional comments about closing: This is now in initscripts in git, please give it a try and reports problems to the ML.
From this comment "solves some stupid problems when running applications while /var is still readonly", I get the impression that these programs are wanting to write something in /var/{lock,run}. How would it be handled when these files disappear when the tmpfs is removed and the real /var is mounted rw?
From the directories above, some don't come with initscripts, which means we'll have to hack the software ourselves. Examples of this are PolicyKit and ConsoleKit, both are daemons launched from dbus.
@JGC: From what I remember, Debian is planning to do this in their next major release, so at least one other distro is "crazy enough". IMO, software should create necessary /var/{run,lock}/whatever directories when needed, if it doesn't it can be considered a bug. Having these folders on tmpfs doesn't sound too crazy to me anyway. Tell me, how else would you solve
FS#18153? It wants to write PID files and/or lock files very early in the boot process, at a time when access to certain file systems might be impossible because they are on LVM.What about adding this as an optional "experimental" feature to rc.sysinit/rc.conf and letting crazy people like me test it and see how it goes?
A workaround for the problem of separated /var patition could be to mount the early tmpfs somewhere else for example /dev/shm and use a bind mount at boot stage and in fstab. I believe that should work for both separated /var partitions and /var on root.
Or just use the the tmpfs on boot time and when the partitions are mounted copy the stuff to the harddisk. I know that isn't a wounderful solution. :-D
As for dmeventd, it's a logging daemon that records devicemapper events to syslog. Is it really needed to run it that early on bootup?
Regardless of /var/run is tmpfs or not:
* Making rc-scripts compatible with empty /var/run (variable run-time data), makes more robust and elegant.
* Makes rc-scripts more FHS-complaint.
* pacman does not handle empty dirs at all (I refer to "/var/run/program-name")
If /var/run is tmpfs:
* Makes unnecessary the cleanup at rc.sysinit
* Some netbooks users, can benefit from this change.
If /var/lock is tmpfs:
* Since is (chmod 1777) mounting as tmpfs can control space usage.
Some rc-scripts are currently compatible with empty /var/run, these automatically create the necessary directories inside (for example hald)
I am talking about /var/run because the usage is much more than /var/lock, but this also applies to /var/lock.
http://www.pathname.com/fhs/pub/fhs-2.3.html#VARRUNRUNTIMEVARIABLEDATA
http://www.pathname.com/fhs/pub/fhs-2.3.html#VARLOCKLOCKFILES
One potential gotcha I had to work around: the umount command in /etc/rc.shutdown doesn't unmount tmpfs. You could change the switches to make it do so. Or, if you've mounted /var/{run,lock} on top of a separate /var partition, you'll have to move them again so that /var can be umounted.
FS#18153, which is blocked by this, get serious if you have mirrored or snapshot logical volumes, as the boot time heavily increases.Here is a description of how Fedora will implement it: <http://www.spinics.net/lists/fedora-devel/msg146048.html>. They intend to use a systemd feature to work around bugs in packages (packages that are not able to create the needed directories themselves). However, we could easily also implement this workaround independently of systemd (it is described in the email). If there is interest in this, I'd be happy to help with the implementation.
It should be noted that, to the best of my knowledge, Fedora (and probably others) will not mount /var/run before /var. I believe the longterm goal is to support a read-only root, in that case / (ro) and /var (rw) will be on separate partitions and /var will be mounted before /var/run (tmpfs).
I'm not using LVM, but I'm pretty certain
FS#18153can be solved without moving mounts around.FS#18153is that /var might be a logical volume (over LVM), starting LVM with snapshots or mirroring requires rw access to /var/{lock,run} (as per current default configuration), hence we need /var/{lock,run} before /var is mounted.Having tmpfs for /var/{run,lock} will help, but I cannot see a way to avoid moving mounts around because of this issue.
I think it is worthwhile to support /var/run and /var/lock being tmpfs. However, I think we should follow the example of udev/dracut/systemd/... and implement it as follows:
1) Make /run and /run/lock tmpfs and mount them early in rc.sysinit and/or in initrd.
2) Make /var/run and /var/lock symlinks to the above directories.
3) Implement <http://0pointer.de/public/systemd-man/tmpfiles.d.html> to recreate the required directories/files on boot.
The last point is the biggest blocker as it would require us to create a daemon (which we do not have the manpower to do I think), or use someone else's (we could easily use the systemd one, but that would be a bit strange packaging-wise). Secondly, every package that relies on files in /var/run and /var/lock would need to come with a config file to create the relevant files/folders. This we should get for free at some point in the future as systemd is requiring this anyway.
What do you think?
I guess we should discuss this further on the mailinglist.
There would need to be some tool to create the needed dirs with the right permissions on boot for the programs that cannot do it themselves, and there would have to be some configuration files to indicate that this tool should create a certain directory. Since many packages probably will come with files on the tmpfiles.d format soon anyway, I figured it would make sense to reuse that standard rather than to make up a new one.
It should be fairly simple to make such a tool (we would only need to implement a minimal subset of the above standard, enough to cover our needs), so if anyone is interested in this I think it is definitely doable and I'd be happy to help. The tool should be able to work both when the dirs are mounted on a persistent device as well as on tmpfs, and it should be able to replace the relevant snippet in rc.sysinit.
What do you think?
Okay, I see :-)
But are there really any programs, that haven't an rc.d file an need folders or files created?
And it shouldn't be that hard to create an small sh script that can create that dirs and files based on some small conf-files.
As you say this should be very easy. Just make a script that parses some conf files, reads off the directories and creates them with the correct permissions and ownerships.
The standard I linked to above is really very simple. We would have to go through the files in /etc/tmpfiles.d/*.conf which have space separated lines of the format:
d /var/run/screens 1777 root root 10d
I suggest we ignore the last entry (Age) and directives of type "x" (possibly also "r" and "R"), but except for that implement the standard.
Are you interested in working on this? Any patches sent to the mailinglist would be greatly appreciated :-)
A further benefit of doing this is that if /var/run and /var/lock are tmpfs, we can move (bind-mount) them to /run and /run/lock as some upstream projects have started doing, and we simplify our boot/shutdown process quite a bit.
community/bind-geodns-9.4.1-3
community/cherokee-1.2.2-2
community/collectd-4.10.3-1
community/couchdb-1.0.2-1
community/courier-imap-4.8.1-1
community/courier-mta-0.65.3-1
community/dspam-3.9.0-3
community/fail2ban-0.8.4-2
community/fcron-3.0.6-4
community/freeradius-2.1.10-4
community/inn-2.5.2-4
community/ircservices-5.1.23-1
community/jabberd14-1.6.1.1-8
community/lastfmsubmitd-1.0.6-4
community/libinfinity-0.4.2-1
community/libvirt-0.8.8-1
community/minbif-1.0.4-1
community/miredo-1.2.3-1
community/murmur-1.2.3-1
community/nginx-0.8.54-2
community/openntpd-3.9p1-13
community/p3scan-2.3.2-4
community/privoxy-3.0.17-1
community/prosody-0.7.0-4
community/tinyproxy-1.8.2-2
community/ultimate-ircd-3.0.2-2
community/wdm-1.28-4
community/xl2tpd-1.2.8-1
core/dbus-core-1.4.1-1
coreystem-2010.12-1/files
core/netcfg-2.5.4-1
extra/apache-2.2.17-1
extra/avahi-0.6.29-1
extra/bftpd-3.1-1
extra/bind-9.8.0-1
extra/bitlbee-3.0.2-1
extra/consolekit-0.4.3-2
extra/cups-1.4.6-1
extra/cyrus-sasl-2.1.23-4
extra/dansguardian-2.10.1.1-2
extra/dovecot-2.0.11-2
extra/fetchmail-6.3.19-1
extra/gdm-2.32.1-1
extra/hal-0.5.14-6
extra/iptraf-3.0.0-2
extra/laptop-mode-tools-1.57-1
extra/lighttpd-1.4.28-4
extra/lxdm-0.3.0-1
extra/munin-1.4.5-2
extra/munin-node-1.4.5-2
extra/networkmanager-0.8.3-0.20110113
extra/php-fpm-5.3.6-1
extra/proftpd-1.3.3d-2
extra/samba-3.5.8-2
extra/udisks-1.0.2-2
extra/wesnoth-1.8.5-3
Full list of packages with /var/lock:
community/ejabberd-2.1.6-1
core/dmraid-1.0.0.rc16+CVS-2
extra/hylafax-6.0.4-1
extra/sane-1.0.22-1
See some patches required for [extra]:
FS#13204FS#13211FS#13225FS#13226FS#13227FS#13228FS#13229FS#13230FS#13231FS#13232FS#13233FS#13234FS#13235. Some was reject and others are currently implemented.I did nothing for [community] because two years ago nobody interested in implementing this.
Some notes about few packages can be interested:arch-general@archlinux.org/msg03581.html"> http://www.mail-archive.com/arch-general@archlinux.org/msg03581.html
dbus-core-1.4.1-1
dmraid-1.0.0.rc16+CVS-2
netcfg-2.5.4-1
apache-2.2.17-1
bftpd-3.1-1
bind-9.8.0-1
bitlbee-3.0.2-1
consolekit-0.4.3-2
cups-1.4.6-1
cyrus-sasl-2.1.23-4
dovecot-2.0.11-2
fetchmail-6.3.19-1
gdm-2.32.1-1
hal-0.5.14-6
iptraf-3.0.0-2
laptop-mode-tools-1.57-1
lighttpd-1.4.28-4
lxdm-0.3.0-1
munin-1.4.5-2
munin-node-1.4.5-2
networkmanager-0.8.3-0.20110113
php-fpm-5.3.6-1
samba-3.5.8-2
sane-1.0.22-1
udisks-1.0.2-2
wesnoth-1.8.5-3
The only correction needed is the makefile. Remove line 9, "$(DESTDIR)/var/run/network/{interfaces,profiles} \"
Is there a better place for this report?
If anyone want to try it out, create the symlinks and apply this patch to initscripts:
<https://github.com/teg/initscripts-arch/commit/b688326d1d1f1b1903ff90c0d60ed441157ecdb1>.
I updated the bug description to indicate the current state of affairs.
What do you guys think?
If you have technical reasons why my proposal is not good, I'd be interested to heard them, but problems you have had with your sound card hardly counts.
And I just doubt that the ideas and implementations of someone whose software is not working for everybody (I'm not talking about a few bugs) shall be called and used as a standard at least not without discussing this across all distros and the major upstream projects.
Lennart is not a standardization board, he's just one developer.
Allow me to clarify: I'm only referring to the specification (the manpage), not any particular implementation of it. What I meant by calling it "a standard" is that it is an implementation independent specification that was developed as a collaboration and agreed upon by many (if not most) of the relevant parties. I did not mean to imply that it has anything to do with standardization bodies.
The specification I linked to is being implemented by Fedora, SuSE and Debian at least, and the systemd team that developed that document consists of people from all those distro's (and several others, including Arch).
If the systemd project succeeds in pushing these "tmpfiles" into the relevant upstream projects, then Arch would benefit from implementing the same specification so we would not have to maintain these things ourselves, but they would come bundled with upstream packages. If the systemd project fails in this goal, then we have not lost anything, as we would anyway have to implement something along these lines.
Some example files are shipped with initscripts, and the script /usr/lib/initscripts/arch-tmpfiles contains comments explaining how the different entries work.