Pacman

Welcome to the Pacman bug tracker. Please search the current bugs and feature requests before filing a new one! Use advanced search and select "Search in Comments".

* Please select the correct category and version.
* Write a descriptive summary, background info, and provide a reproducible test case whenever possible.
Tasklist

FS#6389 - Pacman handling of mirrors/repos

Attached to Project: Pacman
Opened by Scott H (stonecrest) - Saturday, 10 February 2007, 03:31 GMT
Last edited by Dan McGee (toofishes) - Tuesday, 06 November 2007, 05:52 GMT
Task Type Feature Request
Category General
Status Closed
Assigned To Aaron Griffin (phrakture)
Architecture All
Severity Low
Priority Normal
Reported Version 0.7.2 Gimmick
Due in Version 3.1.0
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

20:23 stonecrest| pacman should change the way it handles mirrors so that you type one mirror to be used for various repos. that would also prevent users from using different mirrors and getting inconsistent packages.
20:24 stonecrest| s/various/multiple
20:24 phrakture| http://img.4chan.org/b/src/1171077297949.jpg
20:24 stonecrest| plus it would mean i don't have to open four configs every time i re-arrange mirrors.
20:24 phrakture| stonecrest: good idea actually - can you bug report that?
20:25 stonecrest| sure

I'm sure there are a lot of ways this could be structured in the pacman.conf.
This task depends upon

This task blocks these from closing
 FS#8109 - Pacman 3.1 Release Roadmap 
Closed by  Dan McGee (toofishes)
Tuesday, 06 November 2007, 05:52 GMT
Reason for closing:  Implemented
Additional comments about closing:  $repo expansion implemented in GIT releases, now a single mirrorlist. Additional servers can be specified with other Include lines or Server lines.
Comment by Dan McGee (toofishes) - Saturday, 10 February 2007, 17:14 GMT
I've thought of this before too, always wondered why it wasn't that way. I do know a few things:
1) Some mirrors don't mirror community and/or unstable, only current and extra.
2) Testing is never mirrored.

Perhaps something like a "default mirror" could be specified, and then overrides if necessary?
Comment by Roman Kyrylych (Romashka) - Thursday, 29 March 2007, 20:48 GMT
> 1) Some mirrors don't mirror community and/or unstable, only current and extra.

This should be changed IMO.

> 2) Testing is never mirrored.

Wrong. :P
It's mirrored on some mirrors.

And now when there's trottling on archlinux.org, I don't think we could have "default mirror".
Lets just encourage/ask mirror maintainers to sync all repos.
Comment by Dan McGee (toofishes) - Friday, 30 March 2007, 18:19 GMT
Roman, mirroring all mirrors does not make sense in some cases. Suppose someone has a cluster of 50 Arch machines and has a private rsync server set up. They've realized they only use a lot of packages from current, but not that many from extra or community. However, they clearly want to use their own mirror before public mirrors. This kind of situation needs to be accounted for.
Comment by Roman Kyrylych (Romashka) - Friday, 30 March 2007, 22:36 GMT
Thanks for pointing this out, Dan.

Then, I think it's better to leave thigs as they are now. Either rankmirrors can be used for sorting mirrorlists or write/copy/paste Server=... lines for every repo in pacman.conf (I do this sometime instead of editing mirrorlists).

However I thought more about "default mirror", and think it could be written like
DefaultServer = http://something.com/mirror/%repo%/os/i686
Repos = current extra community
And pacman could replace %repo% with reponame to get real URLs for each repository.

If there are
[current]
Server = ...
# Include = /etc/pacman.d/current
then it overrides DefaultServer & Repos of course.

I don't know yet if I'll prefer this additional functionality though. :-/
Comment by Scott H (stonecrest) - Saturday, 31 March 2007, 17:37 GMT
No, it absolutely should not remain how it is. There are two reasons this needs to change. The most important is to prevent the user from having different mirrors for different repos, which can screw up -Syu's. Secondly, it's a pain to make the same changes to 4 files.

I also don't understand the point of DefaultServer. Which of these two points above does it solve?

The logic used by DefaultServer should apply to all servers. That is:

Repos = (testing current extra community)

Server = http://something.com/mirror/%repo%/os/i686
Server = http://blah.com/mirror/%repo%/os/i686
Server = http://foo.com/mirror/%repo%/os/i686
Server = http://mirror.com/mirror/%repo%/os/i686

And users should probably still be able to use the current approach for mirrors that don't have all of the repos specified in Repos=():

Server = http://something.com/mirror/{current extra community}/os/i686

Syntax isn't so important, of course, but resolving both issues above is.
Comment by Loic Nageleisen (lloeki) - Friday, 20 April 2007, 11:03 GMT
the latter seems very similar to debian syntax, like:
http://something.com/mirror current extra community

the idea of a 'mirror pool' and a 'subscribed repos' list is interesting, because it solves some repetitiveness in the above (e.g you just want to add or remove testing punctually):

Repos = (testing current extra community)
Server = http://something.com/mirror
Server = http://somethingelse.com/mirror

then, what if some mirror pool does not have testing nor community? grouping! like:

[arch]
Repos = (testing current extra community)
Server = http://something.com/mirror
Server = http://somethingelse.com/mirror

[privatenet]
Repos = (current extra)
Server = http://somethingaroundthere/mirror
Server = http://somethingelsearound/mirror

[local]
Repos = (whatever)
Server = http://somethinglocal/mirror

and so on.

"/os/i686" part being somehow 'automatic', e.g from current architecture. maybe mandatory to have such a dir structure in any repo, so that people don't f**k up and go use a 686 repo for 64bit. and not much of a constraint, since it's constant for one given machine, plus it allows using the same mirror file regardless of the arch (and thus easily distribute this file across 30 i686 + 50 x64 machines)
Comment by Aaron Griffin (phrakture) - Friday, 18 May 2007, 05:10 GMT
Adding simplistic experimental support for '$repo' expansion to git

It's not what everyone was talking about here, but it's easy and cleaner.

Here's a usage example:

#/etc/pacman.d/mirrors
Server = http://foo.com/$repo/os/i686
Server = ftp://bar.net/$repo/os/i686

#/etc/pacman.conf
[current]
Include = /etc/pacman.d/mirrors

[extra]
Include = /etc/pacman.d/mirrors

If a given mirror doesn't have a certain repo, we can deal with that issue later, but it'd just mean additional files, or separate listings beyond the 'mirrors' file.

Comment by Scott H (stonecrest) - Friday, 18 May 2007, 16:17 GMT
I haven't tested it, but I like it :)
Comment by Aaron Griffin (phrakture) - Friday, 18 May 2007, 16:34 GMT
I can see some minor usage problems here, with, for instance, mirrors that don't have community, but it's easy enough to keep the existing files with only mirrors that don't have everything, so that each section looks something like:

[current]
Include = /etc/pacman.d/current
Include = /etc/pacman.d/mirrors
Comment by Scott H (stonecrest) - Friday, 18 May 2007, 16:43 GMT
Why are there mirrors that aren't providing every repo? It's a bad idea for any user to use multiple mirrors for official repos because of the possibility that the mirrors won't sync together and the user will run into problems. Part of the idea here, IMO, was to prevent (or discourage) users from doing this. I wonder if maybe pacman should (could?) generate some sort of warning. I have a feeling that pacman doesn't know the difference between an "official" repo and an unofficial repo though, which would presumably make this impossible. Maybe just a warning note in the default pacman.conf?
Comment by Aaron Griffin (phrakture) - Friday, 18 May 2007, 16:49 GMT
Yeah, it's probably ideal that all repos are mirrored. I think, at this moment, a lot of that is size constrain. For instance, the Oregon mirror cactus got running (I think) doesn't mirror community because they only had X gigs to supply.

I think some more (prolonged) package cleanup would help here too.
Comment by Roman Kyrylych (Romashka) - Friday, 18 May 2007, 17:13 GMT
Yeah, I'm strongly against not mirroring all repos/isos/archs.
I think one ftp will be cleared we need to release an announcement to mirror maintainers asking them to use the script from Local Mirror wiki (which will be updated to reflect FTP changes).
Comment by Xavier (shining) - Tuesday, 19 June 2007, 11:12 GMT
Any news about this ?
I find it odd to have the same mirror list duplicated 6 times.
Besides, it's already in one place in pacman source tree : etc/pacman.d/mirrorlist.in
Is it really necessary to generate 6 nearly identical files from it ?
Comment by Glenn Matthys (RedShift) - Sunday, 19 August 2007, 17:55 GMT
Any user who understands a package manager should know that he shouldn't mix mirrors. I don't see why you can't use different mirrors for current and extra: what if you have your own repository where some packages from for example extra are built slightly different (other configure switches to say one thing).

Loading...