Summary and Info:

Pacman fails to download a file from secondary mirrors when the primary mirror returns 404 not found.

This bug can only be reproduced when the primary mirror is out of sync, specifically: when the primary mirror lacks a requested file. In this case, I suspect that the following occurs:

1) pacman requests file X from mirror 1
2) mirror 1 returns 404 not found
3) pacman creates a file named X with a length of zero bytes
4) pacman prepares to request file X from mirror 2
5) pacman sees that file X already exists, so
6) pacman decides to request a byte range. (pacman does this even though file X has length zero.)
7) mirror 2 does not support byte ranges, so mirror 2 returns an error
8) repeat steps 4 through 7 for mirrors 3 through N

----

The fix would be either: (a) prevent pacman from sending byte-range requests when file X already exists but has length zero, or (b) prevent pacman from creating zero length files on 404 errors, or (c, unlikely) require mirrors to support byte-range requests.

Below is the output from when the problem occurred. Eventually, I assume that mirrors.evowise.com was updated to contain the requested file, and the error stopped occurring.

The runtime environment for the below was the archlinux-2021.06.01-x86_64.iso installation environment.

# pacman -r /mnt/userland3 -Sy --noconfirm pacman

:: Synchronizing package databases...
core downloading...
extra downloading...
community downloading...
resolving dependencies...
looking for conflicting packages...

Packages (66) acl-2.3.1-1 archlinux-keyring-20210110-1 attr-2.5.1-1 audit-3.0.1-1 bash-5.1.008-1 bzip2-1.0.8-4 ca-certificates-20210529-1 ca-certificates-mozilla-3.66-1 ca-certificates-utils-20210529-1 coreutils-8.32-1 curl-7.77.0-1 e2fsprogs-1.46.2-1 expat-2.4.1-1 filesystem-2021.05.31-1 findutils-4.8.0-1 gcc-libs-11.1.0-1 glib2-2.68.2-1 glibc-2.33-5 gmp-6.2.1-1 gnupg-2.2.27-1 gnutls-3.7.2-1 gpgme-1.15.1-1 iana-etc-20210202-1 keyutils-1.6.3-1 krb5-1.19.1-1 libarchive-3.5.1-1 libassuan-2.5.5-1 libcap-2.50-1 libcap-ng-0.8.2-1 libffi-3.3-4 libgcrypt-1.9.3-1 libgpg-error-1.42-1 libidn2-2.3.1-1 libksba-1.4.0-2 libldap-2.4.58-3 libnghttp2-1.43.0-1 libp11-kit-0.23.22-1 libpsl-0.21.1-1 libsasl-2.1.27-3 libsecret-0.20.4-1 libssh2-1.9.0-3 libtasn1-4.17.0-1 libtirpc-1.3.2-1 libunistring-0.9.10-3 libxcrypt-4.4.20-1 linux-api-headers-5.12.3-1 lz4-1:1.9.3-2 ncurses-6.2-2 nettle-3.7.3-1 npth-1.6-3 openssl-1.1.1.k-1 p11-kit-0.23.22-1 pacman-mirrorlist-20210509-1 pam-1.5.1-1 pambase-20200721.1-2 pcre-8.44-1 pinentry-1.1.1-1 readline-8.1.001-1 sqlite-3.35.5-1 systemd-libs-248.3-2 tzdata-2021a-1 util-linux-libs-2.37-1 xz-5.2.5-1 zlib-1:1.2.11-4 zstd-1.5.0-1 pacman-6.0.0-2

Total Download Size: 22.94 MiB
Total Installed Size: 312.36 MiB

:: Proceed with installation? [Y/n]
:: Retrieving packages...
pambase-20200721.1-2-any downloading...
error: failed retrieving file 'pambase-20200721.1-2-any.pkg.tar.zst' from mirrors.evowise.com : The requested URL returned error: 404
error: failed retrieving file 'pambase-20200721.1-2-any.pkg.tar.zst' from mirror.rackspace.com : HTTP server doesn't seem to support byte ranges. Cannot resume.
error: failed retrieving file 'pambase-20200721.1-2-any.pkg.tar.zst' from mirror.rackspace.com : HTTP server doesn't seem to support byte ranges. Cannot resume.
error: failed retrieving file 'pambase-20200721.1-2-any.pkg.tar.zst' from mirror.aarnet.edu.au : HTTP server doesn't seem to support byte ranges. Cannot resume.
error: failed retrieving file 'pambase-20200721.1-2-any.pkg.tar.zst' from archlinux.mirror.digitalpacific.com.au : HTTP server doesn't seem to support byte ranges. Cannot resume.
error: failed retrieving file 'pambase-20200721.1-2-any.pkg.tar.zst' from archlinux.mirror.digitalpacific.com.au : HTTP server doesn't seem to support byte ranges. Cannot resume.
error: failed retrieving file 'pambase-20200721.1-2-any.pkg.tar.zst' from ftp.iinet.net.au : HTTP server doesn't seem to support byte ranges. Cannot resume.
error: failed retrieving file 'pambase-20200721.1-2-any.pkg.tar.zst' from mirror.internode.on.net : HTTP server doesn't seem to support byte ranges. Cannot resume.
error: failed retrieving file 'pambase-20200721.1-2-any.pkg.tar.zst' from mirror.launtel.net.au : HTTP server doesn't seem to support byte ranges. Cannot resume.
error: failed retrieving file 'pambase-20200721.1-2-any.pkg.tar.zst' from mirror.launtel.net.au : HTTP server doesn't seem to support byte ranges. Cannot resume.
error: failed retrieving file 'pambase-20200721.1-2-any.pkg.tar.zst.sig' from arch.lucassymons.net : The requested URL returned error: 404
warning: failed to retrieve some files
error: failed to commit transaction (failed to retrieve some files)
Errors occurred, no packages were upgraded.

Steps to Reproduce:

Apparently, this bug can only be reproduced when the primary mirror lacks a file that pacman requests and the secondary mirrors fail to support byte-range requests.

The bug happened every time I ran the following commands, which are excerpted from a script I wrote:

1) pacman-key --init
2) pacman-key --populate archlinux
3) mkdir -p /mnt/userland3/etc/pacman.d
4) mkdir -p /mnt/userland3/var/lib/pacman/
5) cp /etc/resolv.conf /mnt/userland3/etc/
6) cp -a /etc/pacman.d/gnupg /mnt/userland3/etc/pacman.d/
7) mirror='Server = https://mirror.sfo12.us.leaseweb.net/archlinux/$repo/os/$arch';
8) echo "$mirror" >> /mnt/userland3/etc/pacman.d/mirrorlist
9) pacman -r /mnt/userland3 -Sy --noconfirm pacman

I suspect command 5-8 are extraneous. 1 and 2 may also be extraneous, depending on your runtime environment.

I suspect the relevant program logic is in the file pacman/lib/libalpm/dload.c. I am not a curl API expert, so I cannot say for sure.