FS#36159 - [calibre] web server broken with python2-cherrypy 3.2.3-1

Attached to Project: Community Packages
Opened by Alfredo Amatriain (amatriain) - Monday, 15 July 2013, 12:00 GMT
Last edited by Balló György (City-busz) - Monday, 06 January 2014, 08:37 GMT
Task Type Bug Report
Category Upstream Bugs
Status Closed
Assigned To Jelle van der Waa (jelly)
Daniel Wallace (gtmanfred)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 3
Private No

Details

After the python2-cherrypy update from 3.2.2-2 to 3.2.3-1, the Calibre web server stopped working. Trying to load the page returns an HTTP 500 error page generated by cherrypy.

Downgrading python2-cherrypy to 3.2.2-2 the server works again.

This happens at least with calibre-0.9.38-1 and calibre-0.9.38-1.

Attached the error log of Calibre when trying to access the web server. It seems to be some problem when trying to do digest authentication.

Steps to reproduce:

- Open the Calibre web server config from Preferences -> Sharing over the net
- Set username "calibre" and set any non-blank password
- Click on Start Server button
- Open the calibre web server in a broser (e.g. http://localhost:8270)

Expected result: the calibre web server interface should open. Instead an HTTP 500 error page is displayed.
This task depends upon

Closed by  Balló György (City-busz)
Monday, 06 January 2014, 08:37 GMT
Reason for closing:  Upstream
Comment by Alfredo Amatriain (amatriain) - Monday, 15 July 2013, 12:02 GMT
Sorry, wrong calibre versions.

Calibre versions in which I've tested that the error happens:

calibre-0.9.38
calibre-0.9.39
Comment by Jelle van der Waa (jelly) - Sunday, 21 July 2013, 14:21 GMT
What's the status with 0.9.40?
Comment by Alfredo Amatriain (amatriain) - Sunday, 21 July 2013, 15:03 GMT
Error still happening with calibre 0.9.40 and python2-cherrypy 3.2.3
Comment by Alfredo Amatriain (amatriain) - Sunday, 21 July 2013, 15:13 GMT
Update about this: I've tried removing the password from the calibre web server. After restarting, it works as expected (server can be accessed correctly without authentication prompt).

When setting the old password again and restarting, the bug reappears: after the authentication prompt, an HTTP 500 error page generated by cherrypy is served, instead of the calibre page.

I'll make some more tests, maybe there's some character in the password that cherrypy can't handle? I haven't changed this password in years and it never gave me trouble before.

As a workaround I could leave authentication passwordless in the Calibre config and put Calibre behind an Apache reverse proxy with digest authentication enabled. But I still think there is a bug in the way recent cherrypy versions handle authentication.
Comment by Alfredo Amatriain (amatriain) - Sunday, 21 July 2013, 15:47 GMT
Definitely leaving the calibre server unprotected (blank password) but running it behind an Apache reverse proxy with digest authentication enabled to restrict access works fine.

However it seems setting any password in the calibre server configuration causes it to fail. Tried a simple password like "abc" to rule out any problems with charset handling and it still fails.
Comment by Lan (lan) - Friday, 02 August 2013, 16:44 GMT
Passwords still don't work in calibre-0.9.41.
Comment by Jelle van der Waa (jelly) - Friday, 16 August 2013, 19:31 GMT
Yup I get the same error with 0.9.44, did you inform upstream?
Comment by Lan (lan) - Friday, 16 August 2013, 19:34 GMT
I tried to open a bug report just earlier today.
Unfortunately they do not support distro provided packages...

https://bugs.launchpad.net/calibre/+bug/1213138
Comment by Jelle van der Waa (jelly) - Friday, 16 August 2013, 19:39 GMT
The attached patch might fix your problem, but I haven't tried to build it with that patch.
Comment by Alfredo Amatriain (amatriain) - Wednesday, 21 August 2013, 10:34 GMT
I just installed the official calibre binary from their page, as upstream suggests, in a clean Ubuntu 13.04 virtual machine:

http://calibre-ebook.com/download_linux

Last version is 0.9.44, the same as in the arch repos.

I've found that 0.9.44 from their binary installer in the VM works correctly. However 0.9.44-1 from the arch repos in my arch machine has this bug.

If I have time later I will install the official binary in an arch VM to see what happens, but right now it looks like a packaging problem.
Comment by Lan (lan) - Wednesday, 21 August 2013, 10:42 GMT
Which version of CherryPy is in the official binary package?
If it is 3.2.2, then it is not surprising it works - it worked in Arch too.
Comment by Alfredo Amatriain (amatriain) - Wednesday, 21 August 2013, 11:07 GMT
I'm not familiar enough with python to know for sure the CherryPy version, but a grep in the source code folder calibre/src/cherrypy gives several lines like this:

version = "CherryPy/3.2.2"

so yes, I suppose the official calibre installer comes with cherrypy 3.2.2

Maybe we should report it in the cherrypy bugtracker?

Otherwise if it's an API change from 3.2.2 to 3.2.3 but the calibre dev insists on only supporting his installer, which comes with 3.2.2, I'm not sure what can we do.
Comment by Will Szumski (blackbeard) - Sunday, 25 August 2013, 13:31 GMT
Like jelle suggested the problem is the unicode strings in users_dict. Should this be fixed in cherrypy, ie:

cherrypy/lib/auth_digest.py: (please ignore : doesn't work for python 3)

from cherrypy._cpcompat import md5, ntob, tonative
md5_hex = lambda s: md5(ntob(tonative(s,encoding='UTF-8'))).hexdigest()

or in the calling code (calibre/library/server/utils.py):

see attached snippet.

Comment by Jelle van der Waa (jelly) - Sunday, 25 August 2013, 14:22 GMT
I'm now not sure if cherrypy or calibre should fix this bug. But I could propose it upstream
Comment by Will Szumski (blackbeard) - Sunday, 25 August 2013, 16:06 GMT
Please ignore my previous proposal for the change to cherrypy as it would break python-3 code - tonative() needs to passed the correct encoding scheme.
Comment by Balló György (City-busz) - Wednesday, 25 December 2013, 11:53 GMT
Could you still reproduce the problem?
Comment by Lan (lan) - Wednesday, 25 December 2013, 12:08 GMT
Yes, it is still present with calibre 1.16.0 and python2-cherrypy 3.2.4.
Comment by Balló György (City-busz) - Wednesday, 25 December 2013, 12:20 GMT
Then please report it to the upstream developers of CherryPy:
https://bitbucket.org/cherrypy/cherrypy/issues

Loading...