FS#30293 - Xen VPS. Unable to find root device '/dev/xvd*"

Attached to Project: Arch Linux
Opened by Rasmus Edgar (ashren) - Thursday, 14 June 2012, 13:20 GMT
Last edited by Dave Reisner (falconindy) - Thursday, 14 June 2012, 20:59 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Dave Reisner (falconindy)
Architecture x86_64
Severity High
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Description:
VPS running on Xen. (linode)
LVM2 used only for a separate /usr, /var and /tmp.

After upgrading to kernel 3.3.8-1 and systemd-tools 185-1 the root disk /dev/xvda is not found during boot.

Booting into to recovery, chrooting the system and regenerating with mkinitcpio -p linux does not help.

Tried downgrading kernel, systemd-tools and regenerating initramfs, does no difference.

The solution listed on the forums where reinstalling udev (systemd-tools) and regenerating initramfs does not help.

My mkinitcpio.conf hooks are: "base udev fsck autodetect pata scsi sata filesystems shutdown usr"

Mkinitcpio generates images without errors.

Steps to reproduce:
Upgrade a Xen virtual host. Reboot.

Here is the boot output:
Booting 'Arch Linux [/boot/vmlinuz-linux]'

root (hd0)
Filesystem type is ext2fs, using whole disk
kernel /boot/vmlinuz-linux root=/dev/xvda ro init=/bin/systemd
initrd /boot/initramfs-linux.img

close blk: backend at /local/domain/0/backend/vbd/340/51712
close blk: backend at /local/domain/0/backend/vbd/340/51728
close blk: backend at /local/domain/0/backend/vbd/340/51744
[903783.017385] i8042: No controller found
[903783.058838] drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
:: running early hook [udev]
:: running hook [udev]
:: Triggering uevents...
Waiting 10 seconds for device /dev/xvda ...
ERROR: device '/dev/xvda' not found. Skipping fsck.
ERROR: Unable to find root device '/dev/xvda'.
You are being dropped to a recovery shell
This task depends upon

Closed by  Dave Reisner (falconindy)
Thursday, 14 June 2012, 20:59 GMT
Reason for closing:  Fixed
Additional comments about closing:  mkinitcpio-0.9.2-2
Comment by Dave Reisner (falconindy) - Thursday, 14 June 2012, 13:25 GMT
Please post the actual initramfs image and the full config.
Comment by Dave Reisner (falconindy) - Thursday, 14 June 2012, 13:33 GMT
Semi-related: you're going to have problems running the usr hook without the LVM hook included if your /usr is on LVM.
Comment by Rasmus Edgar (ashren) - Thursday, 14 June 2012, 13:42 GMT
Initramfs attached.

The actual mkinitcpio config:
[root@finnix boot]# grep -v ^# /etc/mkinitcpio.conf
MODULES="xen-blkfront xen-fbfront xenfs xen-netfront xen-kbdfront"

BINARIES=""

FILES=""

HOOKS="base udev fsck autodetect pata scsi sata filesystems shutdown usr"

I removed the LVM2 hook because the documentation only states that it is needed for LVM on root. I will add it again. It is unrelated to the problem though, as I've removed the hook in my troubleshooting efforts.

I also need to add that the init=/bin/systemd also was added after installing systemd, to see if it made any difference.
Comment by Dave Reisner (falconindy) - Thursday, 14 June 2012, 13:56 GMT
The legit bug here is already fixed in testing. You will still have a non-useful initramfs without the LVM hook.
Comment by Isaac (isaacwhanson) - Thursday, 14 June 2012, 19:20 GMT
I do not believe this is related to LVM or /usr.

I am having the same problem, but not using LVM or a separate /usr partition. My root partition device is not found and I am dropped to a recovery shell (just like above).

The problem seems to be that the modules listed in mkinitcpio.conf are not being auto-loaded. If I 'modprobe xen-blkfront' in the recovery shell, I can exit the shell and the system boots normally.

MODULES="xen-blkfront xen-netfront xen-kbdfront"
HOOKS="base udev autodetect filesystems fsck shutdown"

------------------------

Filesystem type is ext2fs, using whole disk
kernel /boot/vmlinuz-linux root=/dev/xvda xencons=tty console=tty1 console=hvc0
nosep nodevfs ramdisk_size=32768 ip_conntrack.hashsize=8192 nf_conntrack.hashs
ize=8192 ro devtmpfs.mount=1
initrd /boot/initramfs-linux.img

close blk: backend at /local/domain/0/backend/vbd/1055/51712
close blk: backend at /local/domain/0/backend/vbd/1055/51728
close blk: backend at /local/domain/0/backend/vbd/1055/51744
[2583526.042977] i8042: No controller found
[2583526.084911] drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
:: running early hook [udev]
:: running hook [udev]
:: Triggering uevents...
Waiting 10 seconds for device /dev/xvda ...
ERROR: device '/dev/xvda' not found. Skipping fsck.
ERROR: Unable to find root device '/dev/xvda'.
You are being dropped to a recovery shell
Type 'exit' to try and continue booting
sh: can't access tty; job control turned off
[rootfs /]# modprobe xen-blkfront
[rootfs /]# exit
[rootfs /]# Trying to continue (this will most likely fail) ...
:: mounting '/dev/xvda' on real root
:: running cleanup hook [shutdown]
:: running cleanup hook [udev]
INIT: version 2.88 booting

> Arch Linux
Comment by Dave Reisner (falconindy) - Thursday, 14 June 2012, 19:42 GMT
No, LVM/usr is not related. I pointed out that it was a separate issue that the reporter needs to deal with himself.

Again, the legit bug here is already fixed in testing.

http://projects.archlinux.org/mkinitcpio.git/commit/?id=7cf7ac4eb2c29a2b1aa748dae7658da5fbbc3a18
Comment by Isaac (isaacwhanson) - Thursday, 14 June 2012, 19:52 GMT
ah, I was confused. Thank you!
Comment by Rasmus Edgar (ashren) - Thursday, 14 June 2012, 20:55 GMT
Yep, upgrading from testing fixed it.

This task can be closed.

Was this bug discussed on the mailing list?
Comment by Dave Reisner (falconindy) - Thursday, 14 June 2012, 20:58 GMT
Not on any mailing list I'm aware of.

Loading...