FS#73529 - [pacman] Fails to retrieve signature after slow parallel pkg download on poor connection

Attached to Project: Arch Linux
Opened by Oliver Ford (OJFord) - Thursday, 27 January 2022, 15:05 GMT
Last edited by Toolybird (Toolybird) - Wednesday, 20 September 2023, 23:41 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Levente Polyak (anthraxx)
Architecture x86_64
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 2
Private No

Details

Description:
Since enabling parallel downloads, I've noticed quite often while on a slow connection that while the packages themselves eventually download fine (on the order of say 100KiB/s), pacman then abruptly exits that signature download was too slow. For example:

```
shellcheck-0.8.0-9-x86_64 3.2 MiB 219 KiB/s 00:15 [########################################################] 100%
clojure-1.10.3.1069-1-any 16.3 MiB 521 KiB/s 00:32 [########################################################] 100%
Total (2/2) 19.4 MiB 621 KiB/s 00:32 [########################################################] 100%
error: failed retrieving file 'shellcheck-0.8.0-9-x86_64.pkg.tar.zst.sig' from archlinux.thaller.ws : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds
warning: failed to retrieve some files
error: failed to commit transaction (unexpected error)
Errors occurred, no packages were upgraded.
```

When I then re-run it, it's almost immediate, since however slow the connection is it doesn't take long to acquire the .sig file, and the package already exists.


Additional info:
* package version(s): Pacman v6.0.1 - libalpm v13.0.1
* config and/or log files etc.: default, appended with:
```ini
[options]
Color
ParallelDownloads = 5
```
* link to upstream bug report, if any: N/A

I'm fairly confident I experienced this prior to v6; I don't recall it experiencing it without parallel downloads, but I added that option around the same time as I began sometimes using a slower connection.

Steps to reproduce:
* Use slow connection
* `pacman -Syu`
This task depends upon

Closed by  Toolybird (Toolybird)
Wednesday, 20 September 2023, 23:41 GMT
Reason for closing:  Upstream
Additional comments about closing:  Please refer to comments from PM.
Comment by gay (pidor) - Sunday, 30 January 2022, 00:01 GMT
Maybe you could use XFerCommand in your pacman.conf to specify an outside program (such as wget) to handle the transfers and use the --timeout=seconds option of wget to set the timeout. Like

XferCommand = /usr/bin/wget --timeout=20 --tries=3 -c -O %o %u

Take into account that I haven't tried this and have never tried this. I just ripped the example out of the comments in my pacman.conf and replaced the --passive-ftp option with the --timeout and --tries options. Hope it helps!
Comment by Oliver Ford (OJFord) - Sunday, 30 January 2022, 22:09 GMT
Thanks, that indeed works (I left the timeout & tries at defaults of read-timeout=900 & tries=20), but the resulting output is much messier.

I'm sure this is a bug, since it happens every time and 'less than 1B/s in the last 10s' just doesn't seem right, slow as it is. It happens every time, and every time works immediately on retry (such that all it has to do is download and check the small signature).

At a guess, it seems like the file is requested and then not marked complete or checked until the package downloads finish - which since it's so small makes it look very slow (<1B/s) if the larger packages themselves are a bit slow (500KiB/s).

I can't see why it would be true *every time I update* that the signature file is transferred at <1B/s while the package is hundreds of thousands times faster (though still slow), and this only to be true when the package download isn't faster. For it to depend on the package dl speed like this, it seems there's something off with the measurement.

I can also now confirm it only happens with `ParallelDownloads` enabled. Tested off, worked first time; back on (and when new updates available):
```
linux-headers-5.16.4.arch1-1-x86_64 25.7 MiB 681 KiB/s 00:39 [########################################################] 100%
linux-5.16.4.arch1-1-x86_64 155.7 MiB 1079 KiB/s 02:28 [########################################################] 100%
Total (2/2) 181.4 MiB 1257 KiB/s 02:28 [########################################################] 100%
error: failed retrieving file 'linux-headers-5.16.4.arch1-1-x86_64.pkg.tar.zst.sig' from archlinux.thaller.ws : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds
warning: failed to retrieve some files
error: failed to commit transaction (unexpected error)
Errors occurred, no packages were upgraded.
```

As always, works and proceeds to install 'immediately' the next time it's run.
Comment by Levente Polyak (anthraxx) - Saturday, 02 September 2023, 12:44 GMT
Please report this in the project's bug tracker: https://gitlab.archlinux.org/pacman/pacman/-/issues/

Loading...