FS#72186 - Stricter return code checks in curl 7.79.0 cause download errors on some sites

Attached to Project: Community Packages
Opened by Jan Kohnert (jankoh) - Monday, 20 September 2021, 21:04 GMT
Last edited by Christian Hesse (eworm) - Tuesday, 21 September 2021, 07:27 GMT
Task Type Bug Report
Category Packages
Status Closed
Assigned To Christian Hesse (eworm)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description:
After the recent update to curl 7.79.0 one of my our packages (eccodes) fails to download when trying to do a makepkg. The old version of curl as well as wget or chrome download the package without error.

The old curl versions output is:

jankoh@kohni-mobil ~/projects/eccodes $ curl -v https://confluence.ecmwf.int/
download/attachments/45757960/eccodes-2.23.0-Source.tar.gz?api=v2
* Trying 136.156.180.232:443...
* Connected to confluence.ecmwf.int (136.156.180.232) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: jurisdictionC=GB; businessCategory=Non-Commercial Entity;
serialNumber=ISBN: 9780101820028; C=GB; ST=West Berkshire; L=Reading;
O=European Cen
tre for Medium-Range Weather Forecasts; CN=confluence.ecmwf.int
* start date: Mar 29 16:16:45 2021 GMT
* expire date: Mar 29 16:26:00 2022 GMT
* subjectAltName: host "confluence.ecmwf.int" matched cert's
"confluence.ecmwf.int"
* issuer: C=NL; O=QuoVadis Trustlink B.V.; CN=QuoVadis Europe EV SSL CA G1
* SSL certificate verify ok.
> GET /download/attachments/45757960/eccodes-2.23.0-Source.tar.gz?api=v2 HTTP/
1.1
> Host: confluence.ecmwf.int
> User-Agent: curl/7.78.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 200
< Date: Sun, 19 Sep 2021 19:48:11 GMT
< Server: Apache
< Cache-Control: no-cache, must-revalidate
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
< X-Confluence-Request-Time: 1632080871142
< Set-Cookie: JSESSIONID=659BDBFF1F03F2DC7848A7BEECF3CCC3; Path=/; HttpOnly
< Last-Modified: Mon, 19 Jan 1970 20:46:07 GMT
< Accept-Ranges: bytes
< X-Content-Type-Options: nosniff
< Content-Disposition: inline; filename="eccodes-2.23.0-Source.tar.gz"
< Content-Type: application/x-gzip;charset=UTF-8
< Content-Length: 12037258
< Strict-Transport-Security: max-age=15552000
<
Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.
* Failure writing output to destination
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, close notify (256):
jankoh@kohni-mobil ~/projects/eccodes $

The output of the new version is:

jankoh@kohni-mobil ~/projects/eccodes $ curl -v -i https://
confluence.ecmwf.int/download/attachments/45757960/eccodes-2.23.0-
Source.tar.gz?api=v2
* Trying 136.156.180.232:443...
* Connected to confluence.ecmwf.int (136.156.180.232) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: jurisdictionC=GB; businessCategory=Non-Commercial Entity;
serialNumber=ISBN: 9780101820028; C=GB; ST=West Berkshire; L=Reading;
O=European Centre for Medium-Range Weather Forecasts; CN=confluence.ecmwf.int
* start date: Mar 29 16:16:45 2021 GMT
* expire date: Mar 29 16:26:00 2022 GMT
* subjectAltName: host "confluence.ecmwf.int" matched cert's
"confluence.ecmwf.int"
* issuer: C=NL; O=QuoVadis Trustlink B.V.; CN=QuoVadis Europe EV SSL CA G1
* SSL certificate verify ok.
> GET /download/attachments/45757960/eccodes-2.23.0-Source.tar.gz?api=v2 HTTP/
1.1
> Host: confluence.ecmwf.int
> User-Agent: curl/7.79.0
> Accept: */*
>
* Unsupported response code in HTTP response
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, close notify (256):
curl: (1) Unsupported response code in HTTP response
jankoh@kohni-mobil ~/projects/eccodes $

So obviously the webpage issues a wrong return code (a second 200) and should be fixed, but it is surely not the only page issuing incorrect HTTP return codes.

Infos on the change can be found on https://daniel.haxx.se/blog/2021/09/15/curl-7-79-0-secure-local-cookies/

> HTTP 1.1: disallow >3-digit response codes
> The HTTP protocol is defined to only allow three-digit numbers and
now curl enforces that check stricter. This was in part made to align
behavior when curl is built to use hyper.

While I surely second that stricter type checking is a good thing, it might be an option enable or disable that behaviour via an option.

Probably this needs to be filed upstream, too.

Steps to reproduce:
* git clone https://aur.archlinux.org/eccodes.git
* cd eccodes
* makepkg -s
This task depends upon

Closed by  Christian Hesse (eworm)
Tuesday, 21 September 2021, 07:27 GMT
Reason for closing:  Fixed
Additional comments about closing:  curl 7.79.0-4
Comment by Christian Hesse (eworm) - Tuesday, 21 September 2021, 07:14 GMT
I do not think this is something we should handle downstream.

You should notify the admins of confluence.ecmwf.int and hope they fix it.

As this was an intended upstream change... Open an issue there if you think this should change.
Comment by Christian Hesse (eworm) - Tuesday, 21 September 2021, 07:25 GMT Comment by Christian Hesse (eworm) - Tuesday, 21 September 2021, 07:27 GMT
Should be fixed with curl 7.79.0-4...

Loading...