FS#38869 - [linux] package refactoring for -debug split

Attached to Project: Arch Linux
Opened by Steven Noonan (neunon) - Tuesday, 11 February 2014, 07:13 GMT
Last edited by Allan McRae (Allan) - Wednesday, 12 February 2014, 06:52 GMT
Task Type Feature Request
Category Packages: Core
Status Closed
Assigned To Tobias Powalowski (tpowa)
Thomas Bächler (brain0)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Earlier this week I submitted a patch for handling *.ko files when using options=('strip' 'debug') in a kernel PKGBUILD. Per Allan McRae's request, I'm submitting a feature request to implement split debug packages for the Arch kernel.

One negative side effect with strip+debug in the current Arch kernel PKGBUILD is that 'vmlinux' needs to stay unstripped in order for module builds to work, i.e.:

> MODPOST 1 modules
> FATAL: vmlinux has no symtab?
> /usr/src/linux-3.13.2-1-ec2/scripts/Makefile.modpost:90: recipe for
> target '__modpost' failed
> make[1]: *** [__modpost] Error 1

Note that even with the corresponding -debug package installed, the module build won't succeed.

So ideally 'vmlinux' will remain unstripped while the kernel modules do get separate debug symbols. I'm not sure I understand why vmlinux is in the main 'linux' package in the first place. I suspect it would be better placed in the linux-headers package, since it's needed for module builds and needs to remain unstripped.

My eventual solution to getting a functionally correct split debug kernel PKGBUILD was:

1. Disable module compression step to allow for splitting debug symbols.
2. Move vmlinux into the linux-headers package.
3. Set options=('!strip') for linux-headers, but options=('strip' 'debug') for the main kernel package.

Patch follows (untested against this PKGBUILD, just provided for demo purposes):

diff --git a/linux/trunk/PKGBUILD b/linux/trunk/PKGBUILD
index 68f4b04..b671789 100644
--- a/linux/trunk/PKGBUILD
+++ b/linux/trunk/PKGBUILD
@@ -11,7 +11,7 @@ arch=('i686' 'x86_64')
url="http://www.kernel.org/"
license=('GPL2')
makedepends=('xmlto' 'docbook-xsl' 'kmod' 'inetutils' 'bc')
-options=('!strip')
+options=('strip' 'debug')
source=("http://www.kernel.org/pub/linux/kernel/v3.x/${_srcname}.tar.xz"
"http://www.kernel.org/pub/linux/kernel/v3.x/patch-${pkgver}.xz"
# the main kernel config files
@@ -169,8 +169,6 @@ _package() {
rm -f "${pkgdir}"/lib/modules/${_kernver}/{source,build}
# remove the firmware
rm -rf "${pkgdir}/lib/firmware"
- # gzip -9 all modules to save 100MB of space
- find "${pkgdir}" -name '*.ko' -exec gzip -9 {} \;
# make room for external modules
ln -s "../extramodules-${_basekernel}${_kernelname:--ARCH}" "${pkgdir}/lib/modules/${_kernver}/extramodules"
# add real version for building modules and running depmod from post_install/upgrade
@@ -183,9 +181,6 @@ _package() {
# move module tree /lib -> /usr/lib
mkdir -p "${pkgdir}/usr"
mv "${pkgdir}/lib" "${pkgdir}/usr/"
-
- # add vmlinux
- install -D -m644 vmlinux "${pkgdir}/usr/lib/modules/${_kernver}/build/vmlinux"
}

_package-headers() {
@@ -193,10 +188,13 @@ _package-headers() {
provides=("kernel26${_kernelname}-headers=${pkgver}")
conflicts=("kernel26${_kernelname}-headers")
replaces=("kernel26${_kernelname}-headers")
+ options=('!strip')

install -dm755 "${pkgdir}/usr/lib/modules/${_kernver}"

cd "${srcdir}/${_srcname}"
+ install -D -m644 vmlinux \
+ "${pkgdir}/usr/lib/modules/${_kernver}/build/vmlinux"
install -D -m644 Makefile \
"${pkgdir}/usr/lib/modules/${_kernver}/build/Makefile"
install -D -m644 kernel/Makefile \
This task depends upon

Closed by  Allan McRae (Allan)
Wednesday, 12 February 2014, 06:52 GMT
Reason for closing:  Won't implement
Comment by Dave Reisner (falconindy) - Tuesday, 11 February 2014, 14:03 GMT
I'm pretty sure Allan was only nudging you to request pushing vmlinux into the headers package. Arch repos are a ways away from supporting debug packages.
Comment by Steven Noonan (neunon) - Tuesday, 11 February 2014, 15:36 GMT
Either way.
Comment by Thomas Bächler (brain0) - Tuesday, 11 February 2014, 21:48 GMT
Supporting either the strip or debug option for the linux package makes no sense whatsoever. The only ELF file included is vmlinux, which is only useful for booting with Xen or profiling with oprofile IIRC. You cannot attach gdb to the kernel, so debug packages make no sense either. I am at a complete loss what you are trying to achieve by splitting debug symbols out of .ko files. The kernel has no way of reading those debug symbols separately.
Comment by Steven Noonan (neunon) - Tuesday, 11 February 2014, 22:33 GMT
First off, vmlinux isn't needed for Xen at all, but you're correct that it is needed for oprofile.

Your assertion that debug packages don't make sense is strange to me. Let's compare to another distribution. Fedora packages their kernel this way:

First, a Fedora kernel enables CONFIG_DEBUG_INFO:

[root@localhost ~]# grep DEBUG_INFO /boot/config*
CONFIG_DEBUG_INFO=y
# CONFIG_DEBUG_INFO_REDUCED is not set

But the modules do not have debug information in the Fedora 'kernel' package. They split the debug info exactly as I've specified, into a separate package 'kernel-debuginfo':

[root@localhost ~]# rpm -ql kernel-debuginfo
/usr/lib/debug
/usr/lib/debug/.build-id
/usr/lib/debug/.build-id/00
/usr/lib/debug/.build-id/00/2f59fc7aeaa258a0d6f71785d199301db1aa48
/usr/lib/debug/.build-id/00/2f59fc7aeaa258a0d6f71785d199301db1aa48.debug
[...]
/usr/lib/debug/lib/modules/3.12.10-300.fc20.x86_64/extra/drivers/char/tpm/tpm_atmel.ko.debug
/usr/lib/debug/lib/modules/3.12.10-300.fc20.x86_64/extra/drivers/char/tpm/tpm_infineon.ko.debug
/usr/lib/debug/lib/modules/3.12.10-300.fc20.x86_64/extra/drivers/char/tpm/tpm_nsc.ko.debug
/usr/lib/debug/lib/modules/3.12.10-300.fc20.x86_64/extra/drivers/char/tpm/tpm_tis.ko.debug
/usr/lib/debug/lib/modules/3.12.10-300.fc20.x86_64/extra/drivers/input
/usr/lib/debug/lib/modules/3.12.10-300.fc20.x86_64/extra/drivers/input/joydev.ko.debug
/usr/lib/debug/lib/modules/3.12.10-300.fc20.x86_64/extra/drivers/input/joystick
/usr/lib/debug/lib/modules/3.12.10-300.fc20.x86_64/extra/drivers/input/joystick/a3d.ko.debug
[...]

Their 'kernel-debuginfo' package also contains an untouched vmlinux, but Pacman doesn't give us the freedom to put it in 'linux-debug', so stripping the 'linux' package contents, using '!strip' on 'linux-headers', and adding vmlinux to 'linux-headers' seems to make the most sense.

Rereading what you've said, I suppose you're asserting that the CFLAGS changes of options=('debug') are worthless here, and I totally agree (CONFIG_DEBUG_INFO handles that in the kernel build process). Note however that there is a different reason to enable 'debug': the canonical way with Pacman to get a split debug symbols package is to enable 'strip' and 'debug' at the same time.

Note that many tools fully comprehend split debug kernels (e.g. kgdb, perf, oprofile, etc).
Comment by Steven Noonan (neunon) - Tuesday, 11 February 2014, 22:42 GMT
Apparently systemtap also likes split debuginfo.
Comment by Allan McRae (Allan) - Tuesday, 11 February 2014, 23:16 GMT
Our repos do not handle debug packages yet (apparently that is being fixed...). Once that happens, I would like to get systemtap into our repos, which involves having CONFIG_DEBUG_INFO and CONFIG_KPROBES enabled. Anyway, that is for another day.

The point is, many tools are designed with split debug info in mind - even for kernel modules. We made sure that the file layout of the split debug info was optimal for gdb/valgrind and also other tools that do not use build-id information.
Comment by Thomas Bächler (brain0) - Tuesday, 11 February 2014, 23:58 GMT
I still don't see how splitting debug symbols makes sense when we have CONFIG_DEBUG_INFO disabled and don't plan to enable it. After all, you can separate the debug info from modules, but not from the kernel. In addition, I won't drop module compression, the difference in installed size is too significant.

About the vmlinux file. I don't care where it is. Since we compress the kernel with gzip again (instead of lzma), it's unneeded by our Xen installation nowadays. I don't use oprofile either, so I have no use for the file.
Comment by Allan McRae (Allan) - Wednesday, 12 February 2014, 06:52 GMT
I'm closing this. Until Arch does ship separate debug packages the vmlinux file is fine where it is.

For people building their own kernel package with debug information available, I'd suggest putting vmlinux in linux-debug by:
1) mv $pkgdir/{linux,linux-debug}/usr/lib/modules/${pkgver}-${pkgrel}-ARCH/build/vmlinux
2) use options=('debug') in the package_linux() function.

Loading...