FS#17298 - [mkinitcpio] remove klibc dependency

Attached to Project: Arch Linux
Opened by Thomas Bächler (brain0) - Friday, 27 November 2009, 10:42 GMT
Last edited by Thomas Bächler (brain0) - Sunday, 14 February 2010, 13:37 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Tobias Powalowski (tpowa)
Aaron Griffin (phrakture)
Thomas Bächler (brain0)
Architecture All
Severity Medium
Priority High
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

klibc is mostly unmaintained and doesn't build properly with newer kernels. Fixing it all the time is a waste of time. Also, first attempts with uclibc yielded different, but as annoying problems. So I decided to use glibc in initramfs. What needs to be done is this:

1) Configure and build busybox - this is mostly done, it just needs to be moved to glibc instead of uclibc: http://repos.archlinux.org/wsvn/packages/mkinitcpio-busybox/trunk/
2) Remove klibc'isms from mkinitcpio and make it work with the normal tools.
3) Make sure latest udev and other non-busybox tools get proper hooks, make sure they all work.
4) Move away from kinit, use busybox's switch_root instead.
5) Add the missing stuff, like NFS support, that gets lost in the removal of kinit.

1) to 4) is the work of an afternoon, no idea about 5).
This task depends upon

Closed by  Thomas Bächler (brain0)
Sunday, 14 February 2010, 13:37 GMT
Reason for closing:  Implemented
Comment by Tobias Powalowski (tpowa) - Friday, 27 November 2009, 10:55 GMT
6) move pcmcia hook to pcmciautils
7) replace raid hook and use mdadm hook instead
8) remove firmware hook, modules should have proper firmware entries, if not it's a bug!
9) remove dsdt hook, we don't support the dsdt patch anymore.
Comment by Sven-Hendrik Haase (Svenstaro) - Friday, 04 December 2009, 14:14 GMT
Why isn't busybox in [community] suitable? It is already compiled against glibc, is it not?
Comment by Thomas Bächler (brain0) - Friday, 04 December 2009, 14:22 GMT
No idea, I didn't even know that community/busybox existed. It might be too big for our needs, or things that we need might be missing in the configuration, but I didn't check that yet.
Comment by Sven-Hendrik Haase (Svenstaro) - Friday, 04 December 2009, 14:43 GMT
I'm interested in doing some work to get this bug sorted. Can you guys push your current progress to http://repos.archlinux.org/wsvn/packages/mkinitcpio-busybox/trunk/ or whatever repo you use for development on this end so I can get started?
Comment by Thomas Bächler (brain0) - Friday, 04 December 2009, 15:06 GMT
There is probably not much to do on busybox. The svn trunk you referenced is probably working as it is, except that you need to tell the config to use the normal gcc and compile with -Os. Then remove the uclibc-dependencies and build the package.

You could start with mkinitcpio.git then:
a) Replace the shebang of /init from #!/bin/sh to #!/bin/busybox sh
b) Make the first command in /init "/bin/busybox -s" (then everything will be populated with the proper tools by creating symlinks to busybox)
c) Adjust the /lib/initcpio/install files to not install klibc stuff, but rather busybox and glibc (glibc should be autodetected with mkinitcpio's "BINARIES" stuff I guess)
d) Replace the call to "kinit" in /init with some mount-and-switch_root code (switch_root is part of busybox)
e) Do some magic with the udev hook to install the usual udev to the initramfs

After that, it might already boot in a standard setup. Maybe I can get to this myself this weekend, but I have doubts this will happen now. What I outlined here is basically a start for steps 2-4 from the abover bug report.
Comment by Sven-Hendrik Haase (Svenstaro) - Friday, 18 December 2009, 18:46 GMT
I have now tried and failed multiple times attempting this. I do not have enough knowledge of the whole process and current implementation implications for the hooks. For example, why use sysfs instead of /proc? /proc gives us autodetection for mount. I'll probably just be patient about this now and wait till you get to do it. Sorry I couldn't help :(.
Comment by christopher rogers (godane) - Wednesday, 13 January 2010, 08:30 GMT
i got the new kill-klibc code working. I did have to change the init script. You see /bin/busybox --install -s doesn't make soft link /bin/mount or sleep. It only does the applets.

I did this instead:
for i in $(/bin/busybox --help | /bin/busybox grep -v Copyright | /bin/busybox grep , | /bin/busybox tr , " "); do
/bin/busybox ln -s /bin/busybox /bin/$i
done

This does mean that anything that would be in /sbin folder will be in /bin folder but it works. I hope this helps.
Comment by Thomas Bächler (brain0) - Wednesday, 13 January 2010, 09:14 GMT
I cannot reproduce this. /bin/mount and /bin/sleep are created fine here by --install -s. The version as it is in mkinitcpio.git right now booted fine on virtualbox for me. I only tested i686, because virtualbox won't create a 64 bit machine without VT support in the host CPU.
Comment by Thomas Bächler (brain0) - Sunday, 24 January 2010, 16:57 GMT
So, now something can be tested. I recommend using it in conjunction with testing/udev only:

[kill-klibc]
Server = http://dev.archlinux.org/~thomas/kill-klibc/i686/

or

[kill-klibc]
Server = http://dev.archlinux.org/~thomas/kill-klibc/x86_64/

The standard hooks work, so do keymap, lvm2, encrypt. What does NOT work is raid/mdadm/dmraid and root on NFS. I would appreciate any input and patches w.r.t. those, as well as any testing. The v86d package also needs to be fixed, but that is trivial.

device-mapper, lvm2 and cryptsetup have a weird "1.local" pkgrel so they won't collide with the ones from the repo, sorry if that bothers anyone.

If you upgrade to this repositories, your existing mkinitcpio will be replaced and all klibc packages will be gone too. So, if you try it, please save your existing initramfs to a different filename as a fallback, and do not perform kernel upgrades until you either confirmed that it works, or downgraded to the klibc-based mkinitcpio.
Comment by Gerardo Exequiel Pozzi (djgera) - Thursday, 04 February 2010, 20:29 GMT
The package "which" must be a dep of new mkinitcpio (used at /lib/initcpio/functions), otherwise will fail on install pacman -S base ...

...
/lib/initcpio/functions: line 175: which: command not found
...
Comment by Gerardo Exequiel Pozzi (djgera) - Thursday, 04 February 2010, 21:45 GMT
losetup is missing and is needed by archiso, can be added to "base" hook or archiso should add it?
Comment by Thomas Bächler (brain0) - Thursday, 04 February 2010, 23:33 GMT
1) We could add which as a dependency, or just use type -p.
2) We can add losetup to mkinitcpio-busybox, as commited for -2 here: http://repos.archlinux.org/wsvn/packages/?compare[]=%2Fmkinitcpio-busybox%2Ftrunk%2F%4067221&compare[]=%2Fmkinitcpio-busybox%2Ftrunk%2F%4062602&comparesubmit=Compare+Revisions&op=comp
Comment by Gerardo Exequiel Pozzi (djgera) - Friday, 05 February 2010, 04:15 GMT
* Add -fno-strict-aliasing for CFLAGS (lots of "warning: dereferencing type-punned pointer will break strict-aliasing rules") maybe wrong code can be generated, so avoid this optimization, as happends some time ago with some packages (for example mysql).
* Use build() and package() avoiding usage of fakeroot for build.
* Update checksum for current config in trunk.
Comment by Gerardo Exequiel Pozzi (djgera) - Friday, 05 February 2010, 04:58 GMT
can enable "openvt" (CONFIG_OPENVT) command? With this can execute "openvt -- /bin/sh -i" to get a shell with job control. Since is not posible to get a job control with the default /dev/console. So can use ^C and avoid reboots when some "bad" command is used. ;)
Comment by Thomas Bächler (brain0) - Friday, 05 February 2010, 08:35 GMT
Oh, I didn't see the attachment at first, so I wrote an almost identical patch myself. Thanks anyway, I uploaded mkinitcpio-busybox-1.15.3-3 to kill-klibc.
Comment by Thomas Bächler (brain0) - Friday, 05 February 2010, 08:36 GMT
Hm, I didn't know about the openvt command, I'll have a look at it. Will 'less' also work then (it fails now and does what 'cat' does, 'vi' works though)?
Comment by Gerardo Exequiel Pozzi (djgera) - Friday, 05 February 2010, 21:01 GMT
Exactly that is the idea, cat and less works :) I don't know about 'vi', never used much time. What is not working? Seem to work on default console, I enabled it on mkinitcpio-busybox to test.
Comment by Thomas Bächler (brain0) - Friday, 05 February 2010, 21:11 GMT
vi does work fine, that's the weird thing. The openvt thing sounds nice, I'll have a look.
Comment by Gerardo Exequiel Pozzi (djgera) - Friday, 05 February 2010, 21:36 GMT
Oh OK. Note that my idea of using openvt is for using from command line, because if you try to replace in init script to launch the shell, the script will continue the normal execution.
Comment by Gerardo Exequiel Pozzi (djgera) - Friday, 05 February 2010, 21:42 GMT
ignore my last comment openvt from busybox also support -w -s so, using it at init looks very good.

tested on archiso hook

PS1="ramfs$ " openvt -w -s -- /bin/sh -i

will switch to tty1 automatically, then wehn shell exits , will switch back ;)
Comment by Thomas Bächler (brain0) - Friday, 05 February 2010, 22:42 GMT
Wait, will this switch terminals and thus make the previous output invisible? This will be counter-productive as you won't see the previous output in case of an emergency.
Comment by Gerardo Exequiel Pozzi (djgera) - Friday, 05 February 2010, 23:00 GMT
True. But not at all, you can back with alt+f2 (of course history buffer is lost) so:

* You can remove -s then (to avoid auto-switch), add a small message like "press alt+f1 to go to rescue shell" or something like this.
* This should be used only when no "console=" parameter is used, (for example if you boot from serial console, I use a serial console (stdin) very frequently in KVM)
* Keep things like now, but keep openvt in initramfs to take the advantages.
Comment by Thomas Bächler (brain0) - Friday, 05 February 2010, 23:11 GMT
I'll go with the third option, mkinitcpio-busybox 1.15.3-4 (trunk) has openvt enabled.
Comment by Gerardo Exequiel Pozzi (djgera) - Friday, 05 February 2010, 23:58 GMT
Thanks, good choice.

Another note, maybe is redundant or not, because there are no nss services at this stage, and no delays are experimented in udev, but maybe can add "--resolve-names=never" when launch udevd. I say this because the some of rules files contains GROUP="something", and maybe udev try to resolve them, with this avoid lookups. Another option is removing GROUP from rules.

Comment by Thomas Bächler (brain0) - Sunday, 07 February 2010, 21:09 GMT
We use unchanged udev rules from the normal system, so the first option sounds better.
Comment by Gerardo Exequiel Pozzi (djgera) - Tuesday, 09 February 2010, 02:00 GMT
[PATCH] Fix path when looking at /sys/block for rootdev

otherwise will access to /sys/block//sys/block/sda/sda1/dev (for example)
Comment by Thomas Bächler (brain0) - Tuesday, 09 February 2010, 08:15 GMT
Oops, I never got to test this code path and it will probably be a corner case. Applied.
Comment by Gerardo Exequiel Pozzi (djgera) - Wednesday, 10 February 2010, 22:48 GMT
[PATCH] Removed/changed messages/comments about klibc/kinit

Also remove comment about "modload" in conf.
Comment by Thomas Bächler (brain0) - Wednesday, 10 February 2010, 23:04 GMT
Applied.
Comment by Gerardo Exequiel Pozzi (djgera) - Friday, 12 February 2010, 00:41 GMT
Seems that nobody was affected in all this time, anyway...

[PATCH] Fix module name in usb/usbinput install hook

There is no module named sl1811-hcd, must be sl811-hcd

Loading...