FS#8725 - pacman reports negative download rate

Attached to Project: Pacman
Opened by Petr Slansky (psl) - Wednesday, 21 November 2007, 09:12 GMT
Last edited by Dan McGee (toofishes) - Sunday, 24 August 2008, 00:15 GMT
Task Type Bug Report
Category Output
Status Closed
Assigned To Xavier (shining)
Dan McGee (toofishes)
Architecture All
Severity Low
Priority Normal
Reported Version 3.1.4
Due in Version 3.2.1
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description: pacman reports negative download rate


Additional info:
* package version(s)
* config and/or log files etc.

Notice that download rate reported is negatove, like -265.1K/s; notice huge number of hours, 1193048; this download took less than a minute in the real... Non initialised variable??

Steps to reproduce:

$ sudo pacman -Syu
:: Synchronizing package databases...
core is up to date
extra 315.4K -265.1K/s 1193046:28:15 [#####################] 100%
community 328.2K -461.1K/s 1193046:28:15 [#####################] 100%
:: Starting full system upgrade...
local database is up to date


$ pacman --version

.--. Pacman v3.0.6 - libalpm v1.0.0
/ _.-' .-. .-. .-. Copyright (C) 2002-2007 Judd Vinet <jvinet@zeroflux.org>
\ '-. '-' '-' '-'
'--'

$ date
Tue Nov 20 13:47:18 CET 2007
This task depends upon

Closed by  Dan McGee (toofishes)
Sunday, 24 August 2008, 00:15 GMT
Reason for closing:  Fixed
Additional comments about closing:  Commit e760c4f4784c7e7b59717cd10c56dac04f175b73
Comment by Petr Slansky (psl) - Wednesday, 21 November 2007, 09:26 GMT
I found a source of this problem. Time was not correct on my machine. Correct time after reboot is something like this:

$ date
Wed Nov 21 10:20:54 CET 2007

NTP server is used to synchronize time at my machine, for some reason it didn't updated time correctly and I missed it. I noticed only negative download rate in pacman report... It is possible that NTP system was broken during huge package update yesterday (serious core update).
Comment by Roman Kyrylych (Romashka) - Friday, 23 November 2007, 09:18 GMT
@ Aaron & Dan: shouldn't pacman handle those numbers ok even with broken system time?
Comment by Dan McGee (toofishes) - Friday, 23 November 2007, 15:02 GMT
As long as you didn't change your system time during the download itself, a wrong system time shouldn't do anything as we just do differences between gettimeofday() calls.

Taking a quick look at the code, i don't see any uninitialized variables. If anything this is some kind of weird overflow error... Note that 1193046 is much larger than the largest unsigned int value of 65535.
Comment by Petr Slansky (psl) - Saturday, 24 November 2007, 19:50 GMT
I promiss, I didn't changed time manualy. I have NTP daemon running on my PC. NTP server is OK, other computers (running Ubuntu) were not affected. I think I noticed problem with negative download rate before and I reported it when I saw the issue for second time. My PC has correct time now (NTP works), and pacman doesn't report negative download rates anymore. It is possible that problem was created by core update (tzdata package was updated, etc). I rebooted my system when I noticed wrong time and it is ok now. BTW, I use ARCH for i686.

From your notice it looks as my timer was going backwards; is it possible? Timer running in oposite direction will explain behaviour I observed...
Comment by Aaron Griffin (phrakture) - Monday, 26 November 2007, 17:25 GMT
Are you using a proxy for downloads? I've seen this with some proxies - really off the wall values.
Comment by Petr Slansky (psl) - Monday, 26 November 2007, 17:42 GMT
Yes, I use transparent http proxy; IPcop (www.ipcop.org).
Comment by Dan McGee (toofishes) - Monday, 26 November 2007, 17:52 GMT
I bet the filesize the proxy is returning is bogus, causing all sorts of weird math problems, which would eventually affect the time value shown as well as the download rate. Not much we can do about this.

(the culprit is likely ust.size during the while loop of the download)
Comment by Xavier (shining) - Monday, 11 August 2008, 22:29 GMT
I have that issue too at work, where a proxy is used. The strange thing is that the problem does not happen with all mirrors.
The problem happens with at least one of the french mirrors.
Comment by Dan McGee (toofishes) - Tuesday, 12 August 2008, 00:47 GMT
Any debug outputs or inspections of some of the size values we are getting would be great.
Comment by Xavier (shining) - Tuesday, 12 August 2008, 08:30 GMT
It appears we are not getting any value. The Content-Length is apparently just missing from the answer.

I enabled libdownload debug and verbose mode (in pacman), and installed libdownload with debug enabled too.
Then I ran pacman -Syy, with just the core repo enabled.
With the mir1 french server, all is fine.
With the mir2 french server, it breaks.
Comment by Xavier (shining) - Wednesday, 13 August 2008, 11:26 GMT
Firstly, using http with that mir2 server works just fine. Only using ftp is weird.
And secondly, there can be all kind of scenarios depending on which file is downloaded. It is really odd.
Sometimes, there is just one progress bar, with unkown length (sync of core)
Some other times, the progress bar is split in two, both with unknown lengh (sync of community)
Other times, only one progress bar with known length (sync of extra)
Other times, the first progress bar with unknown length, the second with known (download of wireshark).

The funniest thing is that this looks totally random, but it is not. It is 100% reproducible, as can be seen in the logs I attached. First instance normal, second with verbose libdownload. Maybe it just depends on the size of the file downloaded, I don't know.

And finally, it looks like wget and curl are not able to get the content length either.
Comment by Xavier (shining) - Wednesday, 13 August 2008, 11:31 GMT
Anyway, I feel like I am wasting my time here.
If the issue is some kind of weird incompatibilities between some weird http proxies and some weird ftp servers, then this is not our problem, so let's just close this bug.
Comment by Dan McGee (toofishes) - Wednesday, 20 August 2008, 00:29 GMT
If we have no content length, can we suppress the progress bar or something like that? I don't know, this definitely isn't high priority to me though.
Comment by Xavier (shining) - Wednesday, 20 August 2008, 21:59 GMT
I think this should do it, but I still need to test it when I get a chance...
Comment by Xavier (shining) - Thursday, 21 August 2008, 10:01 GMT
So I tested it and it works perfectly. I am happy with that solution, it is also a very trivial patch :)
Just one thing, I was not sure about the value of file_total in this case. I just checked <= 0. But after testing, the value is -1.
And it is documented in man libdownload as well :
"If the size could not be obtained from the server, the size field is set to -1."

So I will update my patch with a stricter test.

Loading...