Arch Linux

I could confirm this bugreport.
Using my laptop with an USB keyboard i could use this during cups menu, but not on crypt-Hook passphrase input.
Using instead my laptop keyboard i could unlock the crypt partitions, boot goes on and after that i could use also the usb keyboard to login on tty-console.

Seems that the usb/usbinput hooks not load the necessary modules. (I found the workaraound above). But loading only hid and usbhid in MODULES is not enough, only with usbcore (this should normally be loaded during mkinitcpio -M) and the uhci/ehci modules this workaround works.

Confirmed for x86_64.
Temporary workaround seems to be to either load the modules mentioned in the bug report or to attach the keyboard with an USB-to-PS/2 adapter.

Sounds strange to me; hid,usbcore are deps of usbhid. Aren't those modules within the generated image?

I need to see the filelist of an image generated with an empty MODULES="" line (only the HOOKS as mentioned in the original report). To get a filelist, run "zcat /boot/kernel26.img | cpio -t > filelist".

From what I can see, all the necessary modules should be added by the usbinput hook and loaded by udev at runtime. What confuses me is the order of modules here: "uhci_hcd ehci_hcd" - in recent versions of filesystem and mkinitcpio, we ensured that ehci is always loaded before uhci (which is apparently required from the kernel end). Does it still work when you change the order in your MODULES line to load ehci first?

So here's my filelist with 2.6.28.1 compiled, the HOOKS like above and empty MODULES.
BTW: The workaround with enhancing the MODULES seems to not to work for me. My MODULES were so far:
MODULES="pata_acpi ahci pata_jmicron ata_generic scsi_mod ata_piix"
Adding those for the workaround didn't change here anything.

Oh, I'm using x86_64, too, (I forgot that to mention)

That is weird, uhci and ohci are missing, so USB 1.1 devices (like keyboards) will not work. I'll have to check the autodetect, usbinput and usb hooks out, one of them is apparently broken.

EDIT: It's the autodetect hook, mkinitcpio -M does not list uhci or ohci here either.
EDIT2: And I know how to fix the problem now, expect a new mkinitcpio package tomorrow.

Here my filelist's. Laptop is a Thinkpad T22 (this has USB 1.1). System is i686

filelist.bad is with this settings in mkinitcpio.conf (my normal settings):
MODULES="ata_generic ata_piix nls_cp437 vfat"
HOOKS="base udev autodetect usb usbinput pata keymap encrypt filesystems"

filelist.work This work with a USB keyboard i use for this tests.
MODULES="ata_generic ata_piix nls_cp437 vfat hid usbhid usbcore uhci_hcd ehci_hcd"
HOOKS="base udev autodetect usb usbinput pata keymap encrypt filesystems"

Diffing the filelists i got these:
--- filelist.bad 2009-01-21 00:30:08.000000000 +0100
+++ filelist.work 2009-01-21 00:24:25.000000000 +0100
@@ -15,6 +15,16 @@
/lib/modules/2.6.28-ARCH/kernel/fs/fat
/lib/modules/2.6.28-ARCH/kernel/fs/fat/fat.ko
/lib/modules/2.6.28-ARCH/kernel/fs/fat/vfat.ko
+/lib/modules/2.6.28-ARCH/kernel/drivers/hid
+/lib/modules/2.6.28-ARCH/kernel/drivers/hid/hid.ko
+/lib/modules/2.6.28-ARCH/kernel/drivers/usb
+/lib/modules/2.6.28-ARCH/kernel/drivers/usb/core
+/lib/modules/2.6.28-ARCH/kernel/drivers/usb/core/usbcore.ko
+/lib/modules/2.6.28-ARCH/kernel/drivers/hid/usbhid
+/lib/modules/2.6.28-ARCH/kernel/drivers/hid/usbhid/usbhid.ko
+/lib/modules/2.6.28-ARCH/kernel/drivers/usb/host
+/lib/modules/2.6.28-ARCH/kernel/drivers/usb/host/uhci-hcd.ko
+/lib/modules/2.6.28-ARCH/kernel/drivers/usb/host/ehci-hcd.ko
/etc
/etc/udev

Exactly the needed modules are not in the not-working kernel26.img.
And cause no usb module (also usbcore is not included from usb/usbinput hook) i see that i could also not access my usb-stick during initrd phase (encrypt).

Random note: because using cpio can end up in disaster (overwriting files on your system), bsdtar works better for getting file lists.

bsdtar -tf /boot/kernel26.img

The attached patch for /lib/initcpio/functions should fix this. It depends on our udev package, just an FYI.

Please test

This patch solves the problem for me. mkinitcpio -M detects now the usb-host modules and usbinput-Hook loads usbhid if a USB keyboard is attached during mkinitcpio - but only then.

That is indeed the desired behaviour. I will see to it that we get this commit out to the repositories before the ISO making, but not the other commits Aaron made (at least two of them promised to break mkinitcpio instantly and I don't have the time to properly fix those until the ISO is out).

Sorry, the patch doesn't solved it for my situation. Normally i have no usb keyboards on my laptop, only for this bugreport i use one.
With the patch mkinitcpio -M now detects uhci_hcd, usbcore and usbhid (i only checked these before i above sayed: fixed)
But the usb/usbinput hooks don't add these detected modules to the generated initrd image!
So i have still no (usb)keyboard when i have to enter the passphrase. But it seems that there is on my side only a probelm with usb 1.1 hsot controllers.

On my desktop pc USB 2.0 (normally i don't need usb keyboard support during initrd) all wen't fine - also without the patch.
Activating usbinput hook i have all needed modudles in initrd image.

The patch doesn't work for me (but the workaround didn't either... ).
Is the usb entry in HOOKS required? I only have usbinput there.
Or is it a problem, that I compile the kernel while chrooted from GRML64? (Don't have a PS/2 keyboard here for testing.)

Little testing to confuses still more ;-)

I've modified /lib/initcpio/install/usbinput to get some debug values.
usbinput hook:
install ()
{
MOD=" $(checked_modules "/usb/host" ) "
echo $MOD > /tmp/usbinput
MODULES=" $(checked_modules "/usb/host" | grep -ve "_cs" -e "sl1811-hcd" -e

MODULES=$(echo ${MODULES}) #trim whitespace
if [ -n "${MODULES}" ]; then
MODULES="${MODULES} usbhid"
echo "with hid!"
echo $MODULES > /tmp/usbinput-hid
fi
BINARIES=""
FILES=""
SCRIPT=""
}


PC: USB 2.0 without patch (USB Keyboard inserted)
-------------------------
HOOKS="base udev autodetect pata scsi sata keymap usbinput filesystems"

mkinitcpio -M | grep -e usb -e hcd
usblp
usbcore
ehci-hcd

mkinitcpio -g /boot/kernel26.img.test
:: Begin build
:: Parsing hook [base]
:: Parsing hook [udev]
:: Parsing hook [autodetect]
:: Parsing hook [pata]
:: Parsing hook [scsi]
:: Parsing hook [sata]
:: Parsing hook [keymap]
:: Parsing hook [usbinput]
with hid!
:: Parsing hook [filesystems]
:: Generating module dependencies
:: Generating image '/boot/kernel26.img.test'...SUCCESS

bsdtar -tf /boot/kernel26.img.test |grep usb
/etc/modprobe.d/usb-load-ehci-first
/lib/modules/2.6.28-ARCH/kernel/drivers/usb
/lib/modules/2.6.28-ARCH/kernel/drivers/usb/core
/lib/modules/2.6.28-ARCH/kernel/drivers/usb/core/usbcore.ko
/lib/modules/2.6.28-ARCH/kernel/drivers/usb/host
/lib/modules/2.6.28-ARCH/kernel/drivers/usb/host/ehci-hcd.ko
/lib/modules/2.6.28-ARCH/kernel/drivers/hid/usbhid
/lib/modules/2.6.28-ARCH/kernel/drivers/hid/usbhid/usbhid.ko

Seems all went fine with this PC. I use usbinput, so i have the
needed modules in initrd.

Doing the same without usbinput hook:
bsdtar -tf /boot/kernel26.img.test |grep usb
/etc/modprobe.d/usb-load-ehci-first

On my desktop i would say: I'm not affected from Bug  FS#12867 
All neccassary modules are in the initrd image.


Laptop: USB 1.1 (with USB keyboard plugged in)
=======================================
HOOKS="base udev autodetect usb usbinput pata keymap encrypt filesystems"

without Patch:
mkinitcpio -M | grep -e usb -e hcd
usbcore

mkinitcpio -g /boot/kernel26.img.test
:: Begin build
:: Parsing hook [base]
:: Parsing hook [udev]
:: Parsing hook [autodetect]
:: Parsing hook [usb]
:: Parsing hook [usbinput]
:: Parsing hook [pata]
:: Parsing hook [keymap]
:: Parsing hook [encrypt]
:: Parsing hook [filesystems]
:: Generating module dependencies
:: Generating image '/boot/kernel26.img.test'...SUCCESS

bsdtar -tf /boot/kernel26.img.test |grep usb
/etc/udev/rules.d/8-usbstick.rules
/etc/modprobe.d/usb-load-ehci-first
/hooks/usb

So without the patch no usb support in initrd, and so no usb keyboard.


with Patch:
mkinitcpio -M | grep -e usb -e hcd
uhci_hcd
usbcore
usbhid

mkinitcpio -g /boot/kernel26.img.test
:: Begin build
:: Parsing hook [base]
:: Parsing hook [udev]
:: Parsing hook [autodetect]
:: Parsing hook [usb]
:: Parsing hook [usbinput]
:: Parsing hook [pata]
:: Parsing hook [keymap]
:: Parsing hook [encrypt]
:: Parsing hook [filesystems]
:: Generating module dependencies
:: Generating image '/boot/kernel26.img.test'...SUCCESS

bsdtar -tf /boot/kernel26.img.test |grep usb
/etc/udev/rules.d/8-usbstick.rules
/etc/modprobe.d/usb-load-ehci-first
/hooks/usb

Altough that with the patch mkinitcpio autodetect the needed modules, they
don't get inserted from usb/usbinput hook in the initrd.
Seems the hooks have problem to work correctly with USB1.1 (uhci_HCD)
On this Laptop i could still not use usb keyboard to enter passphrases.

@gerbra: your output is only confusing here, I don't get your point at all. As soon as the required uhci/ohci modules are in the initrd, they will be autoloaded by udev - and as soon as a keyboard is connected, the keyboard driver will be loaded. The only thing I can think of is that our install rule breaks this. Try adding ehci_hcd to the image as well and see if this still breaks. I hope I can test this with my old laptop which has no ehci either, but I didn't boot it in almost two years.

@lobotomisator: I suspected as much, your problem is deeper:
hub 3-2:1.0: unable to enumerate USB device on port 3
hub 3-2:1.0: Cannot enable port 3. Maybe the USB cable is bad?
This suggests there is a USB controller loaded, but the device cannot be added. No idea where to go from here.
EDIT: Can we get some more output from your boot screen?

My LUKS key is stored on a USB-Stick. After upgrading, the USB-Stick was not recognized and so the LUKS key could not be read. Luckily (very luckily) I had defined a plain text keyphrase for LUKS slot 2 and so I was able to enter this via the PS2 keyboard. Anyone who has not provided this "backdoor" for themselves faces a very difficult task.

Same problem for you, the autodetect hook didn't see your host controllers. The patch Aaron posted above should help you as well.

Please see mkinitcpio 0.5.22 in testing (soon) for a fix

Well, none of the solutions works for me.
I installed testing/mkinitcpio-0.5.22-1 (but had the patch even installed before) and then compiled the kernel again.
At the moment the keyboard and mouse even have no power at password requestion. Only on event triggering (pressing a button on mouse/keyboard) they have power for half a second. I don't get the "unable to enumerate" messages anymore, the usb support seems to be disabled completely. A look at dmesg shows, that usb devices are detected at line 518, but password requestion is already at line 472, I guess.

My mkinitcpio.conf is:
MODULES="pata_acpi ahci pata_jmicron ata_generic scsi_mod ata_piix"
HOOKS="base udev autodetect pata scsi sata usbinput keymap encrypt filesystems"
(respectively HOOKS="base udev autodetect pata scsi sata usb usbinput keymap encrypt filesystems", that had no effect)


$ bsdtar -tf /boot/kernel26.img | grep usb
/etc/modprobe.d/usb-load-ehci-first
/hooks/usb

Which else information could be interesting for you?

You *did* regenerate your mkinitcpio image, right?

Yes, that does the job, doesn't it:
/sbin/mkinitcpio -k 2.6.28-ARCH -c /etc/mkinitcpio.conf -g /boot/kernel26.img
I also tried reinstalling the kernel26 package.

Can you run that mkinitcpio command with -v (note: without -g runs as a dry-run):
/sbin/mkinitcpio -k 2.6.28-ARCH -c /etc/mkinitcpio.conf -v > output.txt

And attach the output file here?

Surprisingly few unter [usbinput]...

(I have the USB keyboard and a PS/2 keyboard plugged in at the moment)

What about the output of:
$ for a in $(find /sys/devices/ -name modalias -exec cat {} \;); do resolve-modalias /lib/modules/2.6.28-ARCH/modules.alias $a; done

Here it is.
(Tried to contact you with your jabberid, but it seem to be out of date - could be easier for those debugging?)

so uhci and ehci are in there. Are you 100% positive you're using the newest mkinitcpio? This works perfectly fine for me:

$ . /lib/initcpio/functions
$ KERNELVERSION=2.6.28-ARCH
$ auto_modules
....
usbcore usbhid uhci_hcd usbcore
usbcore uhci_hcd usbcore
usbcore uhci_hcd usbcore
usbcore hci_usb hci_usb hci_usb hci_usb ehci_hcd usbcore
....

You should have similar output. Are you using a custom kernel or the stock kernel?

PS fixed my jabber ID

After playing around with this, I'm confident this is - and _ mismatch stuff.

There are a few solutions, here's one:
http://dev.archlinux.org/~aaron/underscore-and-dash-crap.patch

Only a small post this time ;-)

Patching for - vs. _ makes no difference on my side (i need only uhci_hcd, which gets not included in initrd).
But maybe we should think about the reason for this report: this problem is first since 2.6.28.
Our mkinitcpio tools/functions/hooks work well before this kernel. What have changed in this version on the usb stuff that breaks our process of creating initrd images (I personally have not idea ATM).

Gerhard, please go through and provide all the info I asked lobotomisator for.

The kernel has no bearing on whether modules are included in the ramfs.

The original solution to the usb2/usb1.1 bug was to add a special file for modprobe which would load ehci first, and that is what caused this issue.
The initial fix (use resolve-modalias) didn't fix 100% of this (- vs _ changes). So I need to know what ELSE could be broken.

Laptop, USB1.1
mkinitcpio -v -g /boot/kernel26.img.test -> output.txt
for a in $(find /sys/devices.... -> find.txt
auto_modules -> auto-modules.txt

And this from my desktop pc(USB2.0). Here it seems that all would work right.
mkinitcpio -v |grep -e usb -e hcd -> pc-ok.txt

Are you sure you have the latest mkinitcpio from testing and the changes from the dash/underscore patch applied? Considering you HAVE uhci_hcd in the list from the first 2 commands, I'd be surprised if there is a different reason why it's failing.

Got me...

I've installed now the testing version and this works (with the dash/underscore patch).
Before i've made the changes (the resolve-modalias thing and dash/underscore) by hand and there i must have made a misktake....
So silly....

Ok, so - does this new patch actually solve the problem?

Solved for me with the new mkinitcpio + underscore-and-dash-crap.patch. Thanks.

Then lobotomisator is the only one remaining and I believe in a kernel bug here.

lobotomisator contacted me over jabber - this fixes the issue somewhat - his keyboard gets power, but apparently doesn't work. I think this issue is 90% solved here, he may still have a kernel bug

Should be fixed by 0.5.23 in testing. Please confirm

0.5.23 works fine for me.

As I'm using a Macbook 1,1, I originally thought my problem was part of this Apple-specific bug report: http://bugs.archlinux.org/task/12815 but it turns out that the LUKS bug more accurately maps to my problem. I've been watching task 12867 over the past couple days, but had not applied any of the patches or work-arounds described above. Arch devs sure work fast! :) I waited until today to try mkinitcpio 0.5.23 with kernel26 2.6.28. Rebooted, and still no input after I hit Enter at the GRUB screen. I'm willing to paste some log info, just let me know what would be helpful, and where it can be found.

Everything that appears on your screen between udev is started and the LUKS prompt will be useful. You did regenerate your initramfs, right? Also the filelist of your initramfs (bsdtar -tf /boot/kernel26.img) will be useful.

Found out something interesting: I have a remote for the pc, that's working with an ir-usb-cable and is detected as keyboard - and this works at my boot. My Cherry Stream Evolution usb keyboard for some reason still doesn't. I had this ir-usb-device all the time plugged, so I tried compiling/booting/etc without it and it made no difference.

But I had a closer look on my dmesg again (see attachment) and I wondered a bit. (I don't have much knowledge about the inner processes, so that all are guessings from me, don't blame me if I say something wrong)
There are the interesting events in this order:

line 547:
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.14.0-ioctl (2008-04-23) initialised: dm-devel@redhat.com
=== here comes the pw request ===

line 552:
usb 4-1: new low speed USB device using uhci_hcd and address 2
usb 4-1: configuration #1 chosen from 1 choice
usbcore: registered new interface driver hiddev
input: Cypress Keypad&Remote Control as /class/input/input2
generic-usb 0003:04B4:0102.0001: input,hidraw0: USB HID v1.00 Keyboard [Cypress Keypad&Remote Control] on usb-0000:00:1a.1-1/input0
=== my remote ir usb keyboard device is detected, loaded and working (the console output also comes after the pw question) ===

line 585:
SGI XFS with ACLs, security attributes, realtime, large block/inode numbers, no debug enabled
SGI XFS Quota Management subsystem
=== pw was correct and fs is loaded here ===

line 633 & 635 (!):
input: HID 046a:0023 as /class/input/input9
cherry 0003:046A:0023.0003: input,hidraw5: USB HID v1.11 Keyboard [HID 046a:0023] on usb-0000:00:1d.0-1/input0
=== usb keyboard is loaded here first, almost after the complete boot sequence ===

Does that information sound useful for you?
Is it "bad" that the driver says "cherry" and not "generic-usb"?
I hope this brings us forward.

PS:
I forgot to say that it of course pauses before line 585 until I entered the pw.

And an addition:

If I unplug and plug in my mouse on boot it looks like that:
usb 5-2.4: USB disconnect, address 4
usb 5-2.4: new low speed USB device using uhci_hcd and address 9
usb 5-2.4: configuration #1 chosen from 1 choice
input: Razer Razer Diamondback 3G as /class/input/input7
generic-usb 0003:1532:000D.000A: input,hidraw2: USB HID v1.10 Mouse [Razer Razer Diamondback 3G] on usb-0000:00:1a.2-2.4/input0

And with my keyboard just like that:
usb 5-2.3: USB disconnect, address 3
usb 5-2.3: new low speed USB device using uhci_hcd and address 8
usb 5-2.3: configuration #1 chosen from 1 choice
=== and long long after pw entry then sometime ===
cherry 0003:046A:0023.0008: input,hidraw5: USB HID v1.11 Keyboard [HID 046a:0023] on usb-0000:00:1a.2-2.3/input0

Is there a special keyboard driver needed, which is stored on the crypted hdd and for some reason still not included in the initrd?

Yeah, please post the output of "lsmod" when the system is running. It looks like it's using usbhid though, but better check.

Suspicious: hid_cherry
:)

Well, finally I got a workaround! Adding hid_cherry to the MODULES in mkinitcpio.conf did the job!
But the final bug is not dead yet...

BTW: I can truly hardly remember that with <=2.6.27 the usb power was off for a short time on boot.
Between GRUB and password entry all usb devices go down for 2-3 seconds (good noticable e.g. at the glowing mousewheel).
I think this behaviour is new, I would have noticed when my external harddisk drives spun down on every boot in the past!
Do you have these issues too? Did you have them earlier?

Okay, maybe we have to add all hid-* modules to the usbhid hook, and let it autodetect which ones you have.

About the power down, that was always the case for me (Logitech mouse that lights up red when USB is on): Up to grub, the hid boot protocol would keep them alive and they would shut down when linux loads (not the usb devices themselves though, only the hid components).

I think this bug is solved now, after we decided what's up with the hid-* modules.

@Thomas: Yeah, looks like we need to expand the usbhid hook a bit to include more modules.

are those modules autoloaded if they exist?
i ahve a cherry keybord here and i don't need this module at all

For lobotomisator, it was autoloaded when needed.

Aaron, I'll add all those hid-* modules as auto_modules, so they will only be added if a matching keyboard is connected, or if the fallback image is built.

Can you try
http://projects.archlinux.org/?p=mkinitcpio.git;a=commitdiff;h=8182def24483ea7894e0153fb56668555ebf2656

When you add this line to /lib/initcpio/install/usbinput and remove hid-cherry from your mkinitcpio.conf, hid-cherry.ko should still be added to your image (assuming the keyboard is connected to your computer while generating the image). It will always be added to your fallback image.

This issue isn't solved for me, too.
I do have the latest and greatest mkinitcpio 0.5.23-1 and of course I did regenerate my initramfs.

Still there is no option to use the USB Keyboard (neither normal nor fallback-image).
My modell is a Microsoft Wireless Comfort Keyboard 1.0A (shame on me).

I attached the following information:

Output of "mkinitcpio -v -p kernel26": mkinit.log
Contents of /boot/kernel26.img: kernel26cont.log
Contents of /boot/kernel26fallback.img: kernel26fallbackcont.log

please try with thomas fix and regenerate your initramfs after adding the line to usbinput.

Interesting. The hid modules are in there, and all should be fine. I imagine you're missing a module in the same way lobotomisator was.

Could you give us the output of lsmod on a live system? Also, perhaps try applying the patch Thomas linked above and regenerating the image?

Well, Thomas' fix works for the fallback image, but my normal kernel26.img still doesn't include the cherry-module.

lobotomisator: Please post 'find /sys/devices/ -name modalias -exec cat {} \;' and 'mkinitcpio -M'.

Ok, heres another bunch of outputs ;)

I applied Thomas fix and regenerated my initramfs but it didn't work out which wasn't that surprising since a quick diff over the contents of the normal initramfs shows that they both contain exactly the same files.
On the other side I see the same effect as lobotomisator, the fallback-image now works and contains the hid_microsoft module.

I also attached the output of lsmod when having the keyboard plugged in via ps/2 respectively usb. Diffing the files shows that I obviously need the module hid_microsoft.

Please provide the same information I just requested from lobo. Make sure the keyboard is plugged via USB while running the commands.

Weird. I wonder how this isn't getting autodetected.

Here, if it helps

Strange. The cherry module is not listed in there at all.

Note the only modalias supported is: hid:b0003v0000046Ap00000023
Same with the hid-microsoft module.

Looks like we don't get modalias lines for hid devices...

Interim solution: add ALL hid modules in the usbinput hook. They're 284K total. No big deal

Here are the requested outputs.
I'm pretty confused why these modules don't get autodetected. It worked like a charm for all the previous kernel versions I used with arch. So maybe we're trying to workaround a kernel/udev-bug? On the other hand I guess Arch users aren't the only ones who tried to decrypt their HDDs with kernel 2.6.28 so there would have been a lot more fuss about it.
Striking logic, isn't it? ;)

BTW: It's impressive how fast you guys react and work on this stuff!

I don't think udev is loading these modules. They only export aliases with a hid: prefix, and non of your modaliases list hid: aliases.

My guess is that usbhid loads these modules, or something of the sort.

@Thomas: For now, lets just add all hid-* modules in the usbhid hook. This will solve the problem and adds very little overhead.

In my searching, it looks like they're still figuring out how to do the hid stuff in sysfs. I've seen mentioned of adding an "hid bus" to the sysfs info.

I think these modules are new in .28, and usbhid loads them. I might get some more info if I had a machine available where I could look at sysfs more closely.

Thomas, sorry it took me awhile to get back to you. I gave kernel26-2.6.28-3-i686 a go today, but the no keyborad input problem still persists. See attached screenshot.

so whats the status of this?

We still need to change mkinitcpio to add all HID modules and skip autodetection on them, otherwise we should be fine.

shouldn't thuis be fixed with latest mkinitcpio?