FS#46505 - [linux] should be built w CONFIG_USB_SERIAL_CONSOLE=y to use usb serial converters on boot

Attached to Project: Arch Linux
Opened by Niccolò Belli (darkbasic) - Thursday, 01 October 2015, 07:57 GMT
Last edited by Jan Alexander Steffens (heftig) - Friday, 26 October 2018, 18:58 GMT
Task Type Feature Request
Category Packages: Core
Status Closed
Assigned To Tobias Powalowski (tpowa)
Jan Alexander Steffens (heftig)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 3
Private No

Details

If you enable this option theoretically users should be able to add "usb_common usbcore usbserial pl2303" to MODULES= in mkinitcpio.conf to log early boot messages with a usb serial converter.
Please see this topic: https://bbs.archlinux.org/viewtopic.php?pid=1566230
This task depends upon

Closed by  Jan Alexander Steffens (heftig)
Friday, 26 October 2018, 18:58 GMT
Reason for closing:  Fixed
Additional comments about closing:  4.19.arch1-1
Comment by Tobias Powalowski (tpowa) - Thursday, 10 December 2015, 07:19 GMT
It's hidden by EXPERIMENTAL and we don't add those options.
Comment by Niccolò Belli (darkbasic) - Wednesday, 26 September 2018, 20:53 GMT
This isn't experimental since 2.6.26: https://cateee.net/lkddb/web-lkddb/USB_SERIAL_CONSOLE.html

Also CONFIG_USB_SERIAL_CONSOLE=n prevents debugging kernel crashes (and you DEFINITELY want your users to report bugs).
Comment by Eli Schwartz (eschwartz) - Wednesday, 26 September 2018, 20:58 GMT
In v2.6.25-4627-g1b75dc4de94e: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1b75dc4de94e4e11ab22e284fc0853e21d1ac07a

USB: Remove EXPERIMENTAL designation from USB serial/ Kconfig entries
Since nothing under the USB serial/ directory seems to be obviously
experimental, remove the EXPERIMENTAL dependency from all of those
Kconfig entries.

...

Seems like maybe it was never intended to be experimental at all?
Comment by Niccolò Belli (darkbasic) - Wednesday, 26 September 2018, 21:41 GMT
Looks like so indeed, good catch.
Comment by Aaron G (disco) - Thursday, 18 October 2018, 14:27 GMT
By making this change, VFIO is no longer able to preempt USB Host devices without having to unload the driver. Can you please reassess whether this (and other downstream impact) are actually justified here? The technical debt this created should not be ignored.
Comment by Jan Alexander Steffens (heftig) - Thursday, 18 October 2018, 14:37 GMT
But that's something you can fix without compiling a new kernel, no?
Comment by Aaron G (disco) - Thursday, 18 October 2018, 21:21 GMT
Yes, I could now forever maintain my own kernel here on something I've had working for the past 2 years.

Though perhaps given that the function of vfio is to preempt drivers binding to devices, it ought to just be in the kernel already?

Comment by Jan Alexander Steffens (heftig) - Thursday, 18 October 2018, 21:22 GMT
Is it enough to build in vfio?
Comment by Aaron G (disco) - Thursday, 18 October 2018, 21:26 GMT
$ modinfo vfio | grep dep
softdep: post: vfio_iommu_type1 vfio_iommu_spapr_tce
depends:
$ modinfo vfio-pci | grep dep
depends: vfio,irqbypass,vfio_virqfd

I guess it would have to be all the vfio- mods
Comment by Tom Yan (tom.ty89) - Monday, 22 October 2018, 17:32 GMT Comment by Tom Yan (tom.ty89) - Monday, 22 October 2018, 17:42 GMT
And yes, +1 to revert https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/linux&id=f9c4f2b0e05d19ebc5f4b63f69894ea9c037b903, it's pretty much insane. Does CONFIG_USB_SERIAL_CONSOLE=y (or CONFIG_USB_SERIAL_CONSOLE=m?) requires any of the m to y changes anyway? Not seeing any sense in this.
Comment by Jan Alexander Steffens (heftig) - Monday, 22 October 2018, 18:06 GMT
Yes, for the serial console to work it must be compiled in without modules. It's not a tristate option.
Comment by Jensen McKenzie (your_doomsday) - Monday, 22 October 2018, 21:23 GMT
Seeing how many regressions it causes maybe it's (serial console) not worth it?

I checked for other distros:

Debian doesn't have CONFIG_USB_SERIAL_CONSOLE=y

Fedora has CONFIG_USB_SERIAL_CONSOLE=y

None of them have CONFIG_VFIO=y (only m)
Comment by Jan Alexander Steffens (heftig) - Monday, 22 October 2018, 21:25 GMT
Why can't you use vfio parameters in the kernel command line?
Comment by Jensen McKenzie (your_doomsday) - Monday, 22 October 2018, 21:41 GMT Comment by Aaron G (disco) - Monday, 22 October 2018, 22:20 GMT
Obviously a usb serial device won't output anything until the driver is loaded, but why would having it be a module block it completely? Won't it start outputting as soon as the module is loaded? Is this just about catching the very beginning of the kernel's console output, or does it really not work at all after being loaded as a module?


My thinking on VFIO in the kernel or not:

Leave as module:

* Most people likely don't use this
* What's next? There's a reason the kernel is modular; let's not bloat it
* In-line with other distributions

Change to static compile:

* VFIO is special in it's function. Its not a physical device, its a meta driver for binding to devices. PCI_STUB was marked y!
* Its not that big
* It would greatly simplify the configuration of VFIO if people didn't have to generate special initramfs images to use it.

I'm sure there are things I don't know though, anyone have other arguments for not putting it in the kernel as 'y'?


Comment by Jensen McKenzie (your_doomsday) - Monday, 22 October 2018, 23:47 GMT
@disco

CONFIG_USB_SERIAL_CONSOLE can't be module. It's either Y or N:
https://cateee.net/lkddb/web-lkddb/USB_SERIAL_CONSOLE.html
Comment by Aaron G (disco) - Tuesday, 23 October 2018, 00:01 GMT
@your_doomsday Ahhh, okay then =)

Is this true? Is there a parallel thread on this somewhere we missed?

https://www.reddit.com/r/VFIO/comments/9qizja/archlinux_since_41816_vfiopci_is_compiledin_no/
Comment by Tom Yan (tom.ty89) - Tuesday, 23 October 2018, 02:35 GMT
Ah, I thought you can't pass any IDs to it once vfio-pci is builtin. I guess I am fine with it being a builtin then.

I still don't think making all the USB host and hid drivers builtin a good idea though. Sometimes the "disability" could be quite important in case of a bug or such, unless again there's a way to "blacklist" builtin driver that I am not aware of.

While CONFIG_USB_SERIAL_CONSOLE can only be y or n, the db site you guys having been quoting ONLY states that it requires CONFIG_USB_SERIAL to be y. So does the feature not practically work without all the m to y changes in the commit or what?
Comment by Kasza Peter (mr.schyte) - Friday, 26 October 2018, 07:55 GMT
I've tried specifying the vfio-pci.ids parameters on the kernel command line, but the radeon driver still claims my device first. I've reverted back the kernel for now.

Is there a way to blacklist a pci device with the radeon driver? I cannot blacklist the whole module since I have another AMD card with a different model id.
Comment by Aaron G (disco) - Friday, 26 October 2018, 08:05 GMT
@mr.schyte

You need a script like the one described here:
https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#Script_variants

Since its not longer a module, you need to put it elsewhere in the initramfs by using HOOKS.

I haven't had time to update my system, but I suggested the config and had someone confirm that it worked in this reddit post:
https://www.reddit.com/r/VFIO/comments/9qizja/archlinux_since_41816_vfiopci_is_compiledin_no/e8ccujm/

Be sure to scroll down to see that the install script needed some lines for the binary dependencies.

Once I've had time to do this myself I'll see if I can get it added to the wiki...
Comment by Kasza Peter (mr.schyte) - Friday, 26 October 2018, 08:54 GMT
@disco
Thanks for the tip. I've tried it, but unfortunately it didn't help. The device is still bound to the radeon driver :\

# grep '^HOOKS' /etc/mkinitcpio.conf
HOOKS=(base udev vfio autodetect modconf block filesystems keyboard fsck)

# cat /etc/initcpio/hooks/vfio
#!/usr/bin/ash

run_hook() {
echo vfio-pci > /sys/bus/pci/devices/0000:09:00.0/driver_override
echo vfio-pci > /sys/bus/pci/devices/0000:09:00.1/driver_override
}

# lspci -knn -d 1002:683d
08:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde XT [Radeon HD 7770/8760 / R7 250X] [1002:683d]
Subsystem: Gigabyte Technology Co., Ltd Cape Verde XT [Radeon HD 7770/8760 / R7 250X] [1458:2283]
Kernel driver in use: radeon
Kernel modules: radeon, amdgpu
Comment by Aaron G (disco) - Friday, 26 October 2018, 08:57 GMT
You got the /etc/initcpio/install/vfio file also?
Comment by Aaron G (disco) - Friday, 26 October 2018, 08:59 GMT
@mr.schyte:

Oh wait, you're overriding the devices with address 09.00.0/1,

But your lspci shows the device at 08:00.0. Is that the boot vga then or did you override the wrong one maybe?
Comment by Kasza Peter (mr.schyte) - Friday, 26 October 2018, 09:06 GMT
I've attached the wrong output for lspci. 00:09 is the device which I want to pass through:

% lspci -nnk -d 1002:67b0
09:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii XT / Grenada XT [Radeon R9 290X/390X] [1002:67b0]
Subsystem: PC Partner Limited / Sapphire Technology R9 290X Tri-X OC [174b:e285]
Kernel modules: radeon, amdgpu

# cat /etc/initcpio/install/vfio
#!/bin/bash

build() {
add_runscript
}

help() {
cat <<EOF
Configure vfio
EOF
}
Comment by Aaron G (disco) - Friday, 26 October 2018, 09:07 GMT
You need to scroll down in that reddit post and see where I fix the install script (the add binaries I mentioned)
Comment by Kasza Peter (mr.schyte) - Friday, 26 October 2018, 09:10 GMT
I've changed the hook so that it doesn't need those extra binaries (see above). echo is a shell builtin so this should be fine.
Comment by Jan Alexander Steffens (heftig) - Friday, 26 October 2018, 09:13 GMT
I think I'm going to revert most of the changes for v4.19 and keep only the USB core (without HCI drivers) built-in. That should solve the vfio problems.
Comment by Aaron G (disco) - Friday, 26 October 2018, 09:14 GMT
Oh you did at that, sorry. Hrmmm..

How were you doing it before vfio and usb got compiled in?
Comment by Kasza Peter (mr.schyte) - Friday, 26 October 2018, 09:18 GMT
I've just had the following in /etc/mkinitcpio.conf:
MODULES=(vfio vfio_iommu_type1 vfio_pci vfio_virqfd)

and set the following in /etc/modprobe.d/vfio.conf:
options vfio-pci ids=1002:67b0,1002:aac8

This works like a charm with the previous kernel. I've also wrote a small SUID binary (https://github.com/mrschyte/vfioctl) which rebinds a USB hub and my soundcard to vfio when I start the VM.

Unfortunately I cannot rebind the VGA since that results in a kernel panic.
Comment by Aaron G (disco) - Friday, 26 October 2018, 09:45 GMT
I think I know what's wrong here. They need to move VFIO before VGA in the kernel config. It does them in the order they're in the config file. PCI_STUB is earlier... VFIO should probably sit somewhere near where that is.
Comment by Kasza Peter (mr.schyte) - Friday, 26 October 2018, 13:03 GMT
Hmm. I switched to systemd-boot from using straight efibootmgr and vfio-pci now binds to the device first. For some reason the kernel did not pick up the vfio-pci parameters when I added them using efibootmgr. (The params did show in "efibootmgr -v", but they were missing from /proc/cmdline)
Comment by Aaron G (disco) - Friday, 26 October 2018, 13:14 GMT
Ahh... Yea I use systemd-boot also.

Are you saying it wasn't picking up kernel command line params? Or you're saying now the initramfs scripts work?
Comment by Kasza Peter (mr.schyte) - Friday, 26 October 2018, 13:41 GMT
The kernel didn't pick up the vfio-pci.ids=XXXX parameter, so vfio didn't even try to bind to the device...

After switching to systemd-boot, I can now see the vfio-pci.ids parameter in /proc/cmdline

I've also removed the vfio initramfs script; turns out it wasn't needed at all.

Loading...