FS#67865 - Download progress not shown when using Cloudflare mirrors

Attached to Project: Pacman
Opened by Yuval Adam (yuvadm) - Thursday, 10 September 2020, 10:53 GMT
Last edited by Allan McRae (Allan) - Thursday, 15 December 2022, 00:53 GMT
Task Type Bug Report
Category Arch Projects
Status Closed
Assigned To No-one
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Description:

Cloudflare have recently launched a new mirror service - https://cloudflaremirrors.com/

To the best of my understanding they simply mirror the existing mirrors and serve them from their CDN.

When using their mirror service, pacman doesn't output any package download progress. This is most likely due to missing Content-Length headers in the response.

At the very least, it would be better to gracefully degrade and at least show which packages are *currently* being downloaded, even if the specific package progress is missing.

Additional info:
* pacman 5.2.2

Steps to reproduce:
1. Add Server = https://cloudflaremirrors.com/archlinux/$repo/os/$arch to your mirrorlist
2. pacman -Syu
This task depends upon

Closed by  Allan McRae (Allan)
Thursday, 15 December 2022, 00:53 GMT
Reason for closing:  Won't fix
Comment by Eli Schwartz (eschwartz) - Thursday, 10 September 2020, 11:10 GMT
  • Field changed: Attached to Project (Arch Linux → Pacman)
Moving to dedicated pacman project.
Comment by Eli Schwartz (eschwartz) - Thursday, 10 September 2020, 11:13 GMT
Their mirror sucks.
error: failed retrieving file '.......' from cloudflaremirrors.com : HTTP server doesn't seem to support byte ranges. Cannot resume.


pacman 5.2.2 *does* show download progress, but pacman from git doesn't. pacman from git also doesn't emit errors for the failing byte range.
Comment by Eli Schwartz (eschwartz) - Thursday, 10 September 2020, 11:22 GMT
Actually what. pacman, both from git and stable, only shows progress *iff* it tried to continue a previous download (and failed).

pacman from git will still show this, though:

<foo>-x86_64.pkg.tar.zst downloading...

Then reports the average at the end.

pacman stable merely quietly shows nothing for them.
Comment by Yuval Adam (yuvadm) - Thursday, 10 September 2020, 11:24 GMT
Yep that's similar to the behavior I'm seeing. Regardless of how shitty the CF quasi-mirror is, seems pacman can still handle this edge case better.
Comment by Eli Schwartz (eschwartz) - Thursday, 10 September 2020, 11:51 GMT
Yes. Cloudflare is only good at one thing -- breaking the correct functioning of the internet -- but they do provide a valuable role in fuzzing our programs by throwing junk at them and seeing what breaks.

This is definitely broken somehow. Though, the worst breakage -- the completely silent failure to report anything -- is incidentally fixed due to the parallel download rewrite.
Comment by Michael Kogan (Photon) - Wednesday, 03 November 2021, 09:19 GMT
Also happens with https://mirror.osbeck.com/archlinux/$repo/os/$arch for me. But at least it only fails to show the progress of specific packages, it still shows which package is currently being downloaded.
Comment by Daurnimator (daurnimator) - Monday, 20 June 2022, 08:34 GMT
This appears to be fixed now:

```
Total Download Size: 24.07 MiB

:: Proceed with download? [Y/n]
:: Retrieving packages...
^Ceph-15.2.14-6-x86_64 18.3 MiB 14.2 MiB/s 00:00 [----------------------------------------------co o o o o ] 75%
Interrupt signal received

0620 182047 daurnimator@daurn-z170 ~ exit=130 $ sudo pacman -Sw ceph
resolving dependencies...

Packages (2) ceph-libs-15.2.14-6 ceph-15.2.14-6

Total Download Size: 2.51 MiB

:: Proceed with download? [Y/n]
:: Retrieving packages...
^Ceph-15.2.14-6-x86_64 1378.4 KiB 1587 KiB/s 00:00 [--------------------------------C o o o o o o o o o ] 53%
Interrupt signal received

0620 182057 daurnimator@daurn-z170 ~ exit=130 $ sudo pacman -Sw ceph
resolving dependencies...

Packages (2) ceph-libs-15.2.14-6 ceph-15.2.14-6

Total Download Size: 1.03 MiB

:: Proceed with download? [Y/n]
:: Retrieving packages...
^Ceph-15.2.14-6-x86_64 398.5 KiB 518 KiB/s 00:01 [----------------------Co o o o o o o o o o o o o ] 37%
Interrupt signal received
```
Comment by Daurnimator (daurnimator) - Wednesday, 22 June 2022, 12:18 GMT
  • Field changed: Percent Complete (100% → 0%)
Turns out this might not be fixed.
On the same computer as my previous comment, I ran `pacman -Syuw` today and got:

error: failed retrieving file 'ceph-15.2.14-6-x86_64.pkg.tar.zst' from cloudflaremirrors.com : HTTP server doesn't seem to support byte ranges. Cannot resume.
Comment by Allan McRae (Allan) - Thursday, 15 December 2022, 00:53 GMT
Closing - not fixing issues with mirrors that are crappy, and pacman should now show which files are being downloaded.

Loading...