FS#21650 - [wget] [curl] do not work with HTTPS when using Turkish locale

Attached to Project: Arch Linux
Opened by Alper Kanat (T-u-N-i-X) - Tuesday, 09 November 2010, 07:41 GMT
Last edited by Angel Velasquez (angvp) - Saturday, 19 February 2011, 22:44 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To Pierre Schmitz (Pierre)
Architecture All
Severity High
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

I was filling a bug report about that wget and curl wasn't working with HTTPS at all and trying to prepare console outputs to paste into this bug report. Then I just realized that in fact it was a problem with Turkish locale that I use with my home user.

$ locale
LANG=tr_TR.UTF-8
LC_CTYPE="tr_TR.UTF-8"
LC_NUMERIC="tr_TR.UTF-8"
LC_TIME="tr_TR.UTF-8"
LC_COLLATE="tr_TR.UTF-8"
LC_MONETARY="tr_TR.UTF-8"
LC_MESSAGES="tr_TR.UTF-8"
LC_PAPER="tr_TR.UTF-8"
LC_NAME="tr_TR.UTF-8"
LC_ADDRESS="tr_TR.UTF-8"
LC_TELEPHONE="tr_TR.UTF-8"
LC_MEASUREMENT="tr_TR.UTF-8"
LC_IDENTIFICATION="tr_TR.UTF-8"
LC_ALL=tr_TR.UTF-8

$ wget https://aur.archlinux.org/packages/yaourt/yaourt.tar.gz
--2010-11-09 09:43:32-- https://aur.archlinux.org/packages/yaourt/yaourt.tar.gz
aur.archlinux.org çözümleniyor... 208.92.232.29
aur.archlinux.org[208.92.232.29]:443 bağlanılıyor... bağlantı kuruldu.
HATA: cannot verify aur.archlinux.org's certificate, issued by `/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 2 Primary Intermediate Server CA':
Self-signed certificate encountered.
aur.archlinux.org adresine güvenliği gözardı ederek bağlanmak için `--no-check-certificate' seçeneğini kullanın.

$ export LANG=C
$ export LC_ALL=C
$ wget https://aur.archlinux.org/packages/yaourt/yaourt.tar.gz
--2010-11-09 09:44:04-- https://aur.archlinux.org/packages/yaourt/yaourt.tar.gz
Resolving aur.archlinux.org... 208.92.232.29
Connecting to aur.archlinux.org|208.92.232.29|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 866 [application/x-tgz]
Saving to: `yaourt.tar.gz'

100%[=========================================================================================================================================>] 866 --.-K/s in 0s

2010-11-09 09:44:06 (19.8 MB/s) - `yaourt.tar.gz' saved [866/866]
This task depends upon

Closed by  Angel Velasquez (angvp)
Saturday, 19 February 2011, 22:44 GMT
Reason for closing:  Fixed
Additional comments about closing:  Fixed on [testing]
Comment by Allan McRae (Allan) - Tuesday, 09 November 2010, 13:49 GMT
@Pierre: is this a ca-certificates issue as it is not specific to wget or curl?
Comment by Pierre Schmitz (Pierre) - Wednesday, 10 November 2010, 11:48 GMT
I was able to reproduce it with curl and wget. Doing a little strace revealed:

on tr /etc/ssl/certs/a9535edf.0 is opened which does not exist
on other locales /etc/ssl/certs/ae8153b9.0 is opened
Comment by Alper Kanat (T-u-N-i-X) - Wednesday, 10 November 2010, 12:34 GMT
@Pierre: with the?

I've already ran update-ca-certificates --fresh a few times. It doesn't seem to solve the problem.

This might be related with the classical case conversion problem of Turkish language:

ı -> I
i -> İ

But I'm not sure if the problem is related with this or not.
Comment by Pierre Schmitz (Pierre) - Wednesday, 10 November 2010, 13:59 GMT
Sorry, ignore that last sentence; somehow missed to delete it (did it now). Yes, update-ca-certificates does not work. I have to say that I am not sure where the bug might be located.
Comment by Allan McRae (Allan) - Monday, 15 November 2010, 05:49 GMT
Long shot... have you tried to rebuild wget?
Comment by Alper Kanat (T-u-N-i-X) - Wednesday, 17 November 2010, 18:18 GMT
I guess I haven't.. (if i remember correctly, i tried recompiling from aur but something went wrong and the package was orphaned afaik)
Comment by Allan McRae (Allan) - Friday, 03 December 2010, 14:13 GMT
Unassigning me as this does not appear to be a wget issue but rather somewhere down the dependency chain...
Comment by Timothy Redaelli (tredaelli) - Friday, 03 December 2010, 20:35 GMT
It's a multi-byte problem or something similar.

A good workaround, for now, can be to put "export LC_CTYPE=C" in ~/.bashrc
Comment by Dave Reisner (falconindy) - Saturday, 19 February 2011, 05:21 GMT
This boils down to out of date and/or missing ca-certs. curl includes a perl script (lib/mk-cert-bundleF.pl) which downloads and converts a whole bunch of mozilla certs to a ca-bundle.crt file. Pointing wget or curl at this file via their respective ca-cert options fixes this bug. Not the ideal solution, so I kept digging...

Found this in OpenSuSe factory:
http://rpmfind.net/linux/RPM/opensuse/factory/noarch/ca-certificates-mozilla-1.70-2.1.noarch.html

And this from Fedora Rawhide:
http://rpmfind.net/linux/RPM/fedora/devel/rawhide/src/ca-certificates-2011.70-2.fc15.src.html

I've yet to test the sources they pull, but either one has far newer sources than our 2009 ca-cert package taken from debian. I'll look into this tomorrow. Also, as a note to myself or anyone else who pokes around at this, we should be careful that we don't conflict with 21719... or maybe the newer cert bundle will just fix it in the process.

edit: uhh, ignore the attached log files i was going to initially include but aren't necessary (and can't delete).
Comment by Dave Reisner (falconindy) - Saturday, 19 February 2011, 16:11 GMT
Attached are revised PKGBUILDs for wget and curl so that they point at the .crt bundle included with our ca-certificates package. We should deal with getting a new source for ca-certificates separately. Note that I've forced ca-certificates as a hard depend instead of an optdepend of wget.

Loading...