FS#32191 - [systemd] system.journal corruption

Attached to Project: Arch Linux
Opened by Jason (Meyithi) - Tuesday, 23 October 2012, 23:54 GMT
Last edited by Dave Reisner (falconindy) - Monday, 10 December 2012, 15:46 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Dave Reisner (falconindy)
Tom Gundersen (tomegun)
Architecture x86_64
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 11
Private No

Details

Description: My system.journal is deemed to be corrupt upon every reboot and is renamed and replaced.

Additional info:
* package version(s): 194-4
* config and/or log files etc.

dmesg @ http://www.meyithi.com/dmesg.log
df -Th @ http://www.meyithi.com/df.log
journald.conf @ http://www.meyithi.com/journald.conf
mkinitcpio.conf @ http://www.meyithi.com/mkinitcpio.conf
fstab @ http://www.meyithi.com/fstab.log


Steps to reproduce: Reboot.

I added the shutdown hook to mkinitcpio and regenerated my images according to the thread @ https://bbs.archlinux.org/viewtopic.php?id=150442 but it hasn't made any difference it seems. I do have /var on it's own partition (reiserfs) and the hdd it is on is fine as are the other partitions and the data within them.

There doesn't seem to be any ill effects, I wasn't aware it was a problem until I stumbled on the above forum post, journalctl still seems to work and the only real damage is the bloat in /var/log/journal
This task depends upon

Closed by  Dave Reisner (falconindy)
Monday, 10 December 2012, 15:46 GMT
Reason for closing:  Upstream
Additional comments about closing:  Nothing to do here. You're free to use the shutdown hook which should resolve this, but it ultimately needs to be fixed in the journal itself.
Comment by Jason (Meyithi) - Tuesday, 23 October 2012, 23:58 GMT
Relevant bit of dmesg - [ 1.933672] systemd-journald[238]: File /var/log/journal/a83c09bbec854118b18e67bb8cb4bc20/system.journal corrupted or uncleanly shut down, renaming and replacing.

journalctl of a halt - http://www.meyithi.com/pubdock/journalctl_halt_log - seems I can't get /home and /var to unmount cleanly even with shutdown hook
Comment by Dave Reisner (falconindy) - Wednesday, 24 October 2012, 12:00 GMT
/var will be unmounted with the shutdown hook -- you will _not_ see it as part of any log.
Comment by Pierre Schmitz (Pierre) - Wednesday, 24 October 2012, 14:05 GMT
I see this on different systems and I am not sure if it is related to shutdown. E.g. in dmesg I found this:

[Mon Oct 15 10:59:38 2012] systemd-journald[154]: File /var/log/journal/c68aff1abad02ac2a514383f00006eb2/system.journal corrupted or uncleanly shut down, renaming and replacing.
[Mon Oct 15 11:00:03 2012] systemd-journald[154]: File /var/log/journal/c68aff1abad02ac2a514383f00006eb2/user-985.journal corrupted or uncleanly shut down, renaming and replacing.
[Mon Oct 15 11:33:39 2012] systemd-journald[154]: File /var/log/journal/c68aff1abad02ac2a514383f00006eb2/user-1000.journal corrupted or uncleanly shut down, renaming and replacing.

There was no reboot in between. But the first entry is always printed right after boot.
Comment by Jason (Meyithi) - Thursday, 25 October 2012, 13:05 GMT
I've accidentally solved the issue on my box, I compiled ext4 and reiserfs into the kernel and generated a small custom initramfs, no shutdown hook.

new dmesg @ http://www.meyithi.com/pubdock/dmesg2.log
new mkinitcpio.conf @ http://www.meyithi.com/pubdock/mkinitcpio2.conf

No journal corruption in 5 test reboots as of yet - admittedly the journal corruption *fix* was a side effect of general system optimisation and as to why it's fixed I don't know, I'm just happy it is.
Comment by Jason (Meyithi) - Thursday, 01 November 2012, 20:40 GMT
EDIT/ Nevermind, problem returned for a bit with 3.6.4-1 but fixed itself somehow after a few reboots...
Comment by jl (vwyodajl) - Saturday, 10 November 2012, 16:20 GMT
I have the same issue. Regardless of the shutdown hook being in mkinitcpio.conf. I have had the shutdown hook in my configs before the switch to systemd. I just tried a couple reboots with out it being in there and no difference on either linux or linux-lts kernels. I also have the var issue not mounting and unmounting cleanly. If I can provide more information or any logs let me know and I will post them. I have compression set to bzip2 if that makes any difference in mkinitcpio.conf. No other changes in the conf file.

[ 22.199471] systemd[1]: Mounting /var...
[ 22.200188] systemd[1]: var.mount: Directory /var to mount over is not empty, mounting anyway. (To see the over-mounted files, please manually mount the underlying file system to a secondary location.)
[ 22.850888] systemd[1]: Mounted /var.
[ 30.331409] systemd-journald[156]: File /var/log/journal/6ed89c31ae5c383a0442b8f400000c47/system.journal corrupted or uncleanly shut down, renaming and replacing.

Loading...