FS#16134 - 302 and depency problem of http://www.archlinux.org/packages/

Attached to Project: Arch Linux
Opened by Liu Dongiao (liudongmiao) - Friday, 11 September 2009, 06:09 GMT
Last edited by Jan de Groot (JGC) - Sunday, 20 September 2009, 12:27 GMT
Task Type Bug Report
Category Web Sites
Status Closed
Assigned To Dusty Phillips (Dusty)
Jan de Groot (JGC)
Pierre Schmitz (Pierre)
Aaron Griffin (phrakture)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 3
Private No

Details

*Description:*

When we visit www.archlinux.org/packages/[search], for example, www.archlinux.org/packages/firefox with firefox
*sometimes* firefox doesnt follow the information of web, but try to save a file.
the content of saved file is like this:

0300 0000 0000 0000 0000 (ten bytes)
HTTP/1.1 302 FOUND
Date: Fri, 11 Sep 2009 04:51:04 GMT
Server: Apache
Content-Length: 0
Location: http://www.archlinux.org/packages/?arch=&repo=&q=firefox
Vary: User-Agent,Accept-Encoding
Content-Encoding: gzip
Keep-Alive: timeout=2, max=499
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8
0d0a 0d0a 1f8b 0800 0000 0000 0003 0200 0000 ffff
0300 0000 0000 0000 0000 (ten bytes)

but other browser(tested with opera, konqueror, arora) or wget goes well: (example in wget)
-> HEAD, http://www.archlinux.org/packages/firefox
<- 301, http://www.archlinux.org/packages/firefox/
-> HEAD, http://www.archlinux.org/packages/firefox/
<- 302, http://www.archlinux.org/packages/?arch=&repo=&q=firefox
-> HEAD, http://www.archlinux.org/packages/?arch=&repo=&q=firefox
<- 200
-> GET, http://www.archlinux.org/packages/?arch=&repo=&q=firefox
<- 200
note: wget uses HEAD, then GET, but other browser like konqueror uses GET

Another Problem of www.archlinux.org/packages
vim in extra depends on vi in testing, actually, vim in extra should depend vi in core
And i suggest that
packages in core, extra, community follow the depencies in core, extra, community, never follow testing or kde-unstable

1. testing search depencies in testing, never other //should core, extra, community?
2. core search depencies in core, never other.
3. extra search depencies in core, extra, never other
4. community search depencies in core, extra, community, never other

Additional info:
firefox: 3.5.2-1

Steps to reproduce:
302: http://www.archlinux.org/packages/firefox
depency: http://www.archlinux.org/packages/extra/i686/vim/
   rMuM_lCm.bin.part (0.3 KiB)
This task depends upon

Closed by  Jan de Groot (JGC)
Sunday, 20 September 2009, 12:27 GMT
Reason for closing:  Fixed
Additional comments about closing:  Patched mod_fastcgi with the patch named in the last comment. Thanks for pointing this out. Also enabled mod_deflate on the https dev vhost again, as this was disabled to fix the same problem there.
Comment by Pierre Schmitz (Pierre) - Friday, 11 September 2009, 11:09 GMT
Could you open a separate report for your second issue? This seem totally unrelated to the first problem.
Comment by Henning Garus (garns) - Sunday, 13 September 2009, 20:43 GMT
looks like this is triggered by any search which results in only one package, but only most of the time (at least for me).

When searching for all arches this only happens with any packages:

phpmyadmin,phpmya ... phpm
perl-uri,perl-ur
modular initramfs image

When limiting the search to specific arch and repo this behaviour is triggered much more often:

i686/core/{bash,dash,kernel26-lts} most package names will do


Since Dusty reported "works fine" and I got the correct result very seldomly, is there some kind of load balancing going on?
Comment by Dusty Phillips (Dusty) - Sunday, 13 September 2009, 21:19 GMT
Its baffling, but I have yet to get incorrect search results. I'm using an up-to-date stock Arch Linux firefox (unbranded Shiretoko).

There's no load balancing, but there is django level caching (memcached). This problem has started since we switched the server over to FCGI vs modpython, but that doesn't explain the issue.

Is it possible that during the apache reconfigure, some kind of compression got turned on that is wrapping the HTTP request and making it look like binary to firefox? Why are other browsers interpreting it correctly?
Comment by Ionut Biru (wonder) - Sunday, 13 September 2009, 21:25 GMT
i had this problem and i discuss it with eric couples of days ago. since then, i didn't encounter it anymore.
Comment by Henning Garus (garns) - Sunday, 13 September 2009, 23:24 GMT
Could be related to compression, when I change network.http.accept-encoding in about:config from gzip,deflate to just deflate, firefox stops coming up with these binaries, the site looks somewhat shitty though (as in raw html, without css).

I am running up-to-date Arch firefox as well, I tried with the official build from mozilla and default config, same problem. Midori and firefox on windows work lika a charm.
Comment by Liu Dongiao (liudongmiao) - Monday, 14 September 2009, 00:58 GMT
@pierre, should i split this issue to two? but i think it's the same issue.
@garns, er, if i change HEADER from deflate to gzip,deflate, a more funny question occurs..
As the result is compressed, the size should be smaller than orignal html. but, in response, its the same with non-compress html.
Take example as http://www.archlinux.org/packages/firefox
the `deflate' size is 12204, ang `gzip' size is 2273. but in the response to `gzip'-wget, the Content-Length is 12205, not 2273.
Then, wget will always try to get 12204 bytes of html, and will fail permanently.

in attachments, i should wgetrc and the response information.
(application/x-gzip)    wgetrc_n_log.tar.gz (0.9 KiB)
Comment by Aaron Griffin (phrakture) - Wednesday, 16 September 2009, 22:54 GMT
Hmmm so it comes through without CSS on the failure cases? Could it be related to our CSS file somehow?

As a quick survey, what browsers are you using?
Comment by Ionut Biru (wonder) - Thursday, 17 September 2009, 14:27 GMT
today i've clicked on http://archlinux.org/download and that same issue appeared.
but adding / at the end is making the page load.
Comment by Henning Garus (garns) - Thursday, 17 September 2009, 19:48 GMT
@liudongmiao: I don't really understand what you are saying, which header did you change?

@Aaron:
There are several problematic cases, which seem to be related:

1. When I try to access the package search by url, like http://archlinux.org/packages/<SEARCHTERM&gt;, firefox starts downloading a binary similar to the one shown in the report. With a header that says:

302 FOUND
Location: http://www.archlinux.org/packages/?arch=&repo=&q=<SEARCHTERM&gt;

same behaviour with http://archlinux.org/packages/search/<SEARCHTERM&gt;. Behaviour seems the same for all search terms.

2. When I enter a search term which identifies a specific package into the search box on the website, i.e. drupal, phpmya, i686/core/dash, I get a binary file with a header which says 200 OK.

3. When I try to access a specific package by url without a trailing slash: http://archlinux.org/core/i686/bash I get a binary file with a header similar to case 2. With trailing slash I get the correct site.

4. An URL like http://archlinux.org/packages/foo/bar/baz also results in a binary file, with a header that says 404 NOT FOUND.

When I deactivate usage of gzip encoding in about:config firefox shows the correct site in all cases, but without CSS.

I took a look using livehttpheaders and it seems that firefox fails when retrieving a redirected site. The redirection is successful, but upon recieving the site I get presented the binary instead. I can reproduce this pretty consistently (firefox on windows in virturalbox behaves correctly, sometimes) with different versions of firefox (3.5.2-arch, 3.5.3, 3.0.14, 3.7a1, 3.0.8) on i686 and x86_64 and across several distros (Arch,Ubuntu 8.04 and 9.04, zenwalk) and operating systems (opensolaris,win XP).

Other browsers: I tested midori and ie, which both show correct behaviour.
Comment by Aaron Griffin (phrakture) - Thursday, 17 September 2009, 21:53 GMT
Didn't we also update to a newer Django when we did this? Would anyone be able to check django's bug tracker for something similar?
Comment by Henning Garus (garns) - Thursday, 17 September 2009, 23:54 GMT
Just a quick note: setting network.http.keep-alive to false leads to correct behaviour on firefox.
Comment by Aaron Griffin (phrakture) - Friday, 18 September 2009, 00:40 GMT
Interesting. That works for me as well. Seems like this is related to FGCI + HTTP keep-alives ? Really weird
Comment by Liu Dongiao (liudongmiao) - Friday, 18 September 2009, 01:12 GMT
@garns: http header `Accept-Encoding` for WGET, when i change it from `defalte` to `gzip, deflate`
@garns: en, network.http.keep-alive works for me too.

Again, Can bugs.archlinux.org support Markdown?
Comment by Aaron Griffin (phrakture) - Friday, 18 September 2009, 03:49 GMT
It supports dokuwiki markup, but we can't enable that as it breaks far too many existing bug reports
Comment by Roman Kyrylych (Romashka) - Friday, 18 September 2009, 07:34 GMT
After reading this bugreport I'm now even more confident that firefox is crap.
I cannot reproduce the bug with firefox on windows, but probably because I'm behind a corporate proxy.
Comment by Henning Garus (garns) - Friday, 18 September 2009, 16:25 GMT
I have done some digging and our problem looks pretty much like the one described in [1]. You can actually see the 10 spurious bytes in the bin.part file from the original report. I have set up a server in virtualbox and the problem goes away when I upgrade mod_fastcgi to SNAP-0811090952, which contains the patch from [2]. A mod_fastcgi release seems to lie in a rather distant future (see [3]), so I am unsure about the best way to fix this. It looks like Ubuntu is going to take just the patch from [2].

Do you want me to report a new bug against mod_fastcgi?

[1]: https://bugs.launchpad.net/ubuntu/+source/libapache-mod-fastcgi/+bug/381384
[2]: http://article.gmane.org/gmane.comp.web.fastcgi.devel/2613
[3]: http://article.gmane.org/gmane.comp.web.fastcgi.devel/2819

Loading...