Release Engineering

This project is intented for all release related issues (isos, installer, etc), under the umbrella of the ArchLinux Release Engineers
Tasklist

FS#12944 - Pacman mirrorlist installer edits

Attached to Project: Release Engineering
Opened by Greg (dolby) - Saturday, 24 January 2009, 21:11 GMT
Last edited by Dieter Plaetinck (Dieter_be) - Saturday, 31 January 2009, 19:02 GMT
Task Type Bug Report
Category ArchISO
Status Closed
Assigned To Aaron Griffin (phrakture)
Gerhard Brauer (GerBra)
Dieter Plaetinck (Dieter_be)
Architecture All
Severity High
Priority Normal
Reported Version 2009.01-beta
Due in Version 2009.02
Due Date Undecided
Percent Complete 100%
Votes 2
Private No

Details

The installer adds an entry on top of pacman mirrrorlist containing the mirror used for installation.
BUT the entry is the following: eg. Server = ftp://ftp.archlinux.org/core/os/i686
If you leave the mirrorlist file without altering it during configuration of /etc files in the ISO, when you boot to your newly installed system, you always get an error message about not being able to retieve db, or packages in extra or other than core repos from the mirror you used during installation as this is entry is on top of the file.
I wonder if the entry used by the installer could change to eg. Server = ftp://ftp.archlinux.org/$repo/os/i686
This task depends upon

Closed by  Dieter Plaetinck (Dieter_be)
Saturday, 31 January 2009, 19:02 GMT
Reason for closing:  Fixed
Comment by Greg (dolby) - Sunday, 25 January 2009, 02:31 GMT
i would also like to add that if the first mirrorlist entry fails to download the db, or package, the next one is archlinux.org.
so not only you get many 1 error message for each failed download from the used in installation mirror, but more load on the archlinux.org ftp :P
Comment by Dan McGee (toofishes) - Sunday, 25 January 2009, 05:47 GMT
Where do we use [core] explicitly in the mirrorlist? This commit seems to show otherwise:

http://projects.archlinux.org/?p=archiso.git;a=commitdiff;h=7b2dea215ed5a1a088c5cc5f40358094906a3487

That means the installer is probably the broken thing and needs fixing?
Comment by Dieter Plaetinck (Dieter_be) - Sunday, 25 January 2009, 09:27 GMT
Assigning this to Tobias and myself, to check archlinux-installer and aif, respectively.
I find it odd that only one person reported problems with this though ...
Comment by Greg (dolby) - Sunday, 25 January 2009, 16:14 GMT
The 2008.06 IIRC asked if you want to use the mirror you used in installation.
That resulted in many commented out entries and only 1 active, the mirror you used during installation on bottom.
The current one, adds the mirror you used in installation entry eg. Server = ftp://ftp.archlinux.org/core/os/i686 and not $repo on top
and has all other entries active, not commented. That why you get the above described behaviour.
Just for the record this is all with this iso: http://users.archlinux.de/~gerbra/iso/test-usb/archlinux-2009.01-1-ftp-i686.img
Comment by Dieter Plaetinck (Dieter_be) - Sunday, 25 January 2009, 18:58 GMT
I can confirm that with archlinux-installer 2009.01 the topmost mirror in /etc/pacman.d/mirrorlist in ftp mode becomes the user-selected mirror, with $repo replaced by "core"
Also the default proposed mirror at the DIALOG --inputbox "Enter the full URL to core repo." question, uses core instead of $repo.

Can any archlinux-installer maintainer fix this?
I'm not sure i'll have time to fix aif in time for the release, but aif is alpha anyway..
Comment by Dieter Plaetinck (Dieter_be) - Sunday, 25 January 2009, 20:55 GMT
Hey I just found out this seems to be the intended behaviour, though I don't know why. (maybe it's only needed during the installation and maybe it's forgotten to swap core/$repo back when writing to the mirrorlist file.)
See http://projects.archlinux.org/?p=installer.git;a=blob_plain;f=setup;hb=HEAD

" # Form the full URL for our mirror by grepping for the server name in
# our mirrorlist and pulling the full URL out. Substitute 'core' in
# for the repository name, and ensure that if it was listed twice we
# only return one line for the mirror.
SYNC_URL=$(egrep -o "${_server}.*" "${MIRRORLIST}" | sed 's/\$repo/core/g' | head -n1)"


And a while later:


" # /etc/pacman.d/mirrorlist
# add installer-selected mirror to the top of the mirrorlist
if [ "$MODE" = "ftp" -a "${SYNC_URL}" != "" ]; then
awk "BEGIN { printf(\"# Mirror used during installation\nServer = "${SYNC_URL}"\n\n\") } 1 " "${DESTDIR}/etc/pacman.d/mirrorlist" > /tmp/inst-mirrorlist
mv /tmp/inst-mirrorlist "${DESTDIR}/etc/pacman.d/mirrorlist"
fi
"

At this point we probably should use ${SYNC_URL/core/\$repo} instead of $SYNC_URL ?



Comment by Greg (dolby) - Sunday, 25 January 2009, 22:30 GMT
"Hey I just found out this seems to be the intended behaviour, though I don't know why"
Maybe the change was done because users are expected to edit this file. After all it wasnt part of the files youre supposed to edit during the installation before (IIRC).
But IMO, a) there shouldnt be an entry using explicit repo names in the file in the first place, but always use $repo
b) With the latest changes to get a decent mirrorlist you are expected to comment out many repository lines, or move the mirror you want to use manually on top.
Comment by Gerhard Brauer (GerBra) - Monday, 26 January 2009, 14:07 GMT
I've made a patch for the installer which will fix this problem. Posted on arch-releng, but i attach it also here.
Comment by Dieter Plaetinck (Dieter_be) - Monday, 26 January 2009, 18:04 GMT
Actually both Gerhard and mine solutions are not 100% waterproof: it will give unexpected behavior if the hostname (or any part of the path before the repo name) contains the word 'core'. But it will do for now... I'll implement a clean longterm solution in aif after 2009.01 release.
Comment by Dieter Plaetinck (Dieter_be) - Monday, 26 January 2009, 19:15 GMT
Aaron, please integrate Gerhard/mine fix and close this :-)
Comment by Gerhard Brauer (GerBra) - Monday, 26 January 2009, 19:17 GMT
100% waterproofed is my pink rubber-boat <g>. But after sending the patch i thought about the same and have looked at our mirrorlists. Actually we will not have a problem with only replacing "core" ;-) But the patch could also sed core/os to $repo/os if we want...
Comment by Dieter Plaetinck (Dieter_be) - Monday, 26 January 2009, 19:57 GMT
in aif i'll do it the other way around: I'll just use the url containing $repo and substitute it by core or whatever if need to.
Comment by Greg (dolby) - Monday, 26 January 2009, 20:47 GMT
The 2008.06 ISOs as i said before worked flawlessly ragarding that. I described the behaviour above. You might wanna take a look on those too.
Comment by Evangelos Foutras (foutrelis) - Monday, 26 January 2009, 21:35 GMT
Well, during the installation, SYNC_URL is used by pacman in just one place. It makes sense to replace '$repo' at that point, and keep SYNC_URL intact.

I also modified the step where you specify a custom mirror so that it uses CARCH from /etc/makepkg.conf instead of hardcoding i686 in it (which is incorrect on x86_64 Live CDs).
Comment by Jordy van Wolferen (jordz) - Monday, 26 January 2009, 22:26 GMT
The backups made by the installer are because of the default settings in /etc/virc.
Would be nice if it was set to "set nobackup" by default.
Comment by Greg (dolby) - Tuesday, 27 January 2009, 03:45 GMT
+1 on what jordz said. I think that starting with 7.2 and beyond vi(m) doesnt contain nobackup in its default configuration file.
That creates backup files in a clean installation if you use vi to edit the files during installation.
Vi would have to be rebuilt adding nobackup to the default config.
Comment by Aaron Griffin (phrakture) - Tuesday, 27 January 2009, 04:01 GMT
a) One bug per report please
b) I don't have a big problem with the backup files. At the very least, your system still has a pristine copy of the original.
Comment by Aaron Griffin (phrakture) - Tuesday, 27 January 2009, 04:05 GMT Comment by Dieter Plaetinck (Dieter_be) - Thursday, 29 January 2009, 16:54 GMT
please redirect all vim backup file thoughts to  FS#12995 
Comment by Dieter Plaetinck (Dieter_be) - Saturday, 31 January 2009, 19:01 GMT
This is fixed in beta3 and later.

Loading...