FS#56749 - [linux-hardened] 4.14.6.a-1 freezes hard during rsync copy of large (>2GB) files over NFS connection

Attached to Project: Arch Linux
Opened by Gary S. Martin (GaryScottMartin) - Sunday, 17 December 2017, 04:33 GMT
Last edited by Levente Polyak (anthraxx) - Friday, 14 September 2018, 11:37 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To Levente Polyak (anthraxx)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description:
While using rsync to copy files from a local NTFS formatted drive to my NAS over NFS, the system repeatedly froze while moving ISOs and large video files (>2GB). It never froze during the copy of a smaller file. Once frozen the system was unresponsive to mouse or keyboard input. Even the kernel keyboard shortcuts were unresponsive. The only way to recover the system was a hard shoutdown and reboot. The kernel keyboard shortcuts (ALT - SysRq - [e/i/b]) were enabled and worked as expected when the system was not frozen. No freezes occured are rebooting to the (not hardened) linux 4.14.5-1 kernel, even though more than ten large files were copied up to the NAS.

Additional info:
* linux kernel: linux-hardened 4.14.6.a-1
* The last occurence was at 1948 local (based on time displayed by system post freeze). The journalctl messages immediately preceeding and following that time were:

Dec 16 19:44:42 TehachapiMtn kdeinit5[17862]: kf5.kio.kio_file: "Couldn't preserve group for '/net/nas01/mnt/md0/Public/documents/TSP-21-G.pdf'"
-- Reboot --
Dec 16 19:56:01 TehachapiMtn kernel: Linux version 4.14.5-1-ARCH (builduser@heftig-19247) (gcc version 7.2.1 20171128 (GCC)) #1 SMP PREEMPT Sun Dec 10 14:50:30 UTC 2017

Steps to reproduce:
sudo ionice -c3 rsync -avz '/wdata/Users/Public/Public Videos/' /net/nas01/mnt/md0/Public/media/video

This appears to be an ongoing issue with the hardened kernel and may be an upstream issue.

Reference:  FS#54847  - [linux-hardened] system freeze on copy and rsync
This task depends upon

Closed by  Levente Polyak (anthraxx)
Friday, 14 September 2018, 11:37 GMT
Reason for closing:  No response
Comment by Gary S. Martin (GaryScottMartin) - Sunday, 17 December 2017, 04:37 GMT
Sorry, I just realized this went into the Community Packages Project rather than the Arch Linux Project.
Comment by Levente Polyak (anthraxx) - Sunday, 17 December 2017, 09:58 GMT
please try linux 4.14.6-1 this seems like a more general issue then packaging related to hardening
Comment by Gary S. Martin (GaryScottMartin) - Monday, 18 December 2017, 00:40 GMT
It definitely seems to be related to the linux-hardened 4.14.6.a-1 kernel. I upgraded my system to linux 4.14.6-1, and booted that kernel. I deleted 5 large files from the NAS to force my script (attached) to rsync them back up to the NAS. Linux 4.14.6-1 not only copied all five of those files without a problem but completed the remainder of the sync activity which included quite a few cell phone video files that were large enough to cause linux-hardened 4.14.6.a-1 to freeze the system. Just to ensure that there was nothing special about the particular files that I chose, I deleted them from the NAS again and rebooted with linux-hardened 4.14.6.a-1 and executed my sync script again. Linux-hardened 4.14.6.a-1 succesfully copied the first file (2.9 GB), but froze at about 667 MB into the second file (3.1 GB). I rebooted with the linux 4.14.6-1 kernel and launched the script again. It completed without errors again.
Comment by Gary S. Martin (GaryScottMartin) - Monday, 18 December 2017, 19:45 GMT
I am beginning to think that this may be a hardware or filesystem related issue. I have now seen a freeze when not copying a large file, although it did follow closely after the cutting and pasting of a 4.4 GB ISO from my ext4 ~/Download folder to the NTFS formatted partition.

I recently (two weeks ago) moved my /root partition to an SSD. I am using btrfs as the file system for the /root partition. When I try to balance the file system it errors out leaving the following in the syslog:

[ +0.061365] BTRFS warning (device sda1): csum failed root -9 ino 270 off 24629248 csum 0xdaf9872a expected csum 0xc25e6565 mirror 1
[ +0.007077] BTRFS warning (device sda1): csum failed root -9 ino 270 off 24629248 csum 0xdaf9872a expected csum 0xc25e6565 mirror 1
[

However, btrfs check reports no errors:

[gary@TehachapiMtn ~]$ sudo btrfs check --force /dev/sda1
[sudo] password for gary:
WARNING: filesystem mounted, continuing because of --force
Checking filesystem on /dev/sda1
UUID: 798dc3ce-fb0c-43f0-b752-087d35e2e90e
checking extents
checking free space cache
checking fs roots
checking csums
checking root refs
found 42964029440 bytes used, no error found
total csum bytes: 41012668
total tree bytes: 870072320
total fs tree bytes: 799784960
total extent tree bytes: 20955136
btree space waste bytes: 203434234
file data blocks allocated: 1505120923648
referenced 36112232448


Comment by Levente Polyak (anthraxx) - Wednesday, 04 July 2018, 22:37 GMT
Still not convinced this is really a linux-hardened related issue itself but located somewhere else, does it still happen for you?

Loading...