FS#50022 - [coreutils] dd doesn't exit when device full

Attached to Project: Arch Linux
Opened by Eivind (mokkurkalve) - Monday, 11 July 2016, 17:51 GMT
Last edited by Dave Reisner (falconindy) - Tuesday, 12 July 2016, 20:33 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To No-one
Architecture x86_64
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description:
Trying to zero out a usb device, running `sudo dd if=/dev/zero of=/dev/sdc bs=4096' then,
when device is full dd gives an expected error notice of no more space on unit but it does
not exit. AFAIK or remember, previously when I've done this, dd gives the error message then exits.
Now I cannot even terminate the process by `sudo kill -9 $PID'. Only when physically removing the
usb device, dd exits. I reckon this is not expected behaviour...?

Additional info:
* package version(s)
8.25-2
* config and/or log files etc.

Steps to reproduce:
Described above.
This task depends upon

Closed by  Dave Reisner (falconindy)
Tuesday, 12 July 2016, 20:33 GMT
Reason for closing:  Not a bug
Additional comments about closing:  Nothing wrong here in userspace -- just evidence of buffering.
Comment by Dave Reisner (falconindy) - Monday, 11 July 2016, 18:08 GMT
If the process doesn't even exit when you send a SIGKILL, then it's not a problem which exists in userspace. My guess is the "freeze" is due to flushing the bytes to the device, and dd is waiting for this to complete. It really is trying to exit, but it's waiting for the kernel.
Comment by Eivind (mokkurkalve) - Monday, 11 July 2016, 18:28 GMT
You're probably right. Did it again, then issued a `sudo sync'. This also hung until I physically removed the device.
Don't know how to proceed from here.... Kernel:
uname -a
Linux audhumla 4.6.3-1-ARCH #1 SMP PREEMPT Fri Jun 24 21:19:13 CEST 2016 x86_64 GNU/Linux
Comment by Eivind (mokkurkalve) - Monday, 11 July 2016, 18:36 GMT
Well, found something in /var/log/errors.log:
kernel: blk_update_request: I/O error, dev sdc, sector 13102920
kernel: Buffer I/O error on dev sdc, logical block 1637865, lost async page write
kernel: Buffer I/O error on dev sdc, logical block 1637866, lost async page write
kernel: Buffer I/O error on dev sdc, logical block 1637867, lost async page write
(....)
Comment by Dave Reisner (falconindy) - Monday, 11 July 2016, 18:45 GMT
Is it possible those are from when you physically pulled the device from the machine? Not sure why /var/log/errors.log would be populated....
Comment by Eivind (mokkurkalve) - Monday, 11 July 2016, 18:49 GMT
That's possible. Actually it's probably that....
Comment by Eivind (mokkurkalve) - Monday, 11 July 2016, 18:54 GMT
Anyway I think the device got poperly zero'd out. It's a 8GB memorystick. `lsblk -S'
sdc 7:0:0:0 disk SanDisk Cruzer Fit 1.27 usb
But having to yank it to get dd to exit feels a bit akward....
Comment by Dave Reisner (falconindy) - Tuesday, 12 July 2016, 12:04 GMT
The point is that if you wait long enough, dd will exit on its own. I don't think there's any bug in coreutils here.
Comment by Eivind (mokkurkalve) - Tuesday, 12 July 2016, 13:17 GMT
Dunno. I've waited for like half an hour after the device was filled and error message regarding that displayed before I yanked the device.
No sign of exit from dd. But I agree whatever caused it it's likely not a bug in coreutils, so this bug can probably be closed.
I'll use dd on an OpenBSD box instead.
Comment by AK (Andreaskem) - Tuesday, 12 July 2016, 16:37 GMT
For a cheap USB drive, half an hour to fill 8 GiB is not extraordinary. Some of them are really slow when writing [1]. 4 MiB/s for 1800 s would be 7200 MiB. In fact, I think the errors in your log files indicate that the kernel was still writing data when you pulled the drive.

FYI: You should be able to check the progress by sending SIGUSR1 to the dd process.

http://usbspeed.nirsoft.net/?pdesc=SanDisk+Cruzer+Fit&vname=SanDisk+Corp.&vid=1921&pid=21873
Comment by Eivind (mokkurkalve) - Tuesday, 12 July 2016, 18:13 GMT
You misunderstand. I wrote "half an hour AFTER the device was filled". I did not state the time of writing before, which was indeed three quarters of an hour. The period is after the error message clearly stated that device is full. And then (another half an hour later), after yanking out device the message from dd is that megabytes according to size of device are indeed written. The half hour I describe occurred when writing was finished, and is also a hang. This have not happened me on Arch earlier, when I did similar things, and did not happen now when I took same two devices and did the same on an OpenBSD box (where the writing itself was slower, because it's a slow box).
Comment by Dave Reisner (falconindy) - Tuesday, 12 July 2016, 18:25 GMT
> You misunderstand. I wrote "half an hour AFTER the device was filled"
Sorry, but the misunderstanding is on your side. The device *appears* full, but the data hasn't been physically written yet -- it's still buffered. Had it *actually* been full, the kernel wouldn't have complained when you yanked the device from the port (and dd would have exited).

You can probably change this behavior by forcing dd not to buffer (see the 'sync' option).
Comment by Eivind (mokkurkalve) - Tuesday, 12 July 2016, 18:31 GMT
I'll test that. Will report back.
Comment by Eivind (mokkurkalve) - Tuesday, 12 July 2016, 20:16 GMT
Seems you're right about that. Used commandline this time:
# dd if=/dev/zero of=/dev/sdc bs=4096 status=progress iflag=sync oflag=sync
It took like half an hour longer and this time dd returned after reporting device full.
I have never experienced this before, even having used dd quite a bit for different stuff. Have gnu dd changed in the use of buffer? The async buffer between read and write here in normal operation must be quite large...?
Comment by Dave Reisner (falconindy) - Tuesday, 12 July 2016, 20:33 GMT
This is pretty typical behavior of Linux. It doesn't really have anything to do with dd, other than the fact that it's writing to a raw device rather than some sort of filesystem.

Loading...