Arch Linux

Please read this before reporting a bug:
https://wiki.archlinux.org/index.php/Reporting_Bug_Guidelines

Do NOT report bugs when a package is just outdated, or it is in Unsupported. Use the 'flag out of date' link on the package page, or the Mailing List.

REPEAT: Do NOT report bugs for outdated packages!
Tasklist

FS#38755 - {dbscripts} add support for debug symbol package

Attached to Project: Arch Linux
Opened by Phil Schaf (flying-sheep) - Saturday, 01 February 2014, 09:44 GMT
Last edited by Eli Schwartz (eschwartz) - Sunday, 18 February 2018, 04:33 GMT
Task Type Feature Request
Category Arch Projects
Status Assigned
Assigned To Pierre Schmitz (Pierre)
Eli Schwartz (eschwartz)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 0%
Votes 27
Private No

Details

As of  FS#10975 , makepkg supports the creation of debug symbol packages.

but in  FS#20921 , it is mentioned, that the server dbscripts lack support, which is why there are no debug packages in the repos.

i couldn't find a report about that, so i’m creting this in order to discuss/track the state of the feature.
This task depends upon

Comment by Phil Schaf (flying-sheep) - Saturday, 05 April 2014, 09:42 GMT
any progress on this? i get really shitty crashes in amarok now and i do not want to compile every program that ever crashes. (i’m currently finishing my master’s thesis, and while i have time to let dr. konqui send crash dumps to the KDE bug tracker, i don’t have time to set up build environments for a program that crashes, build and install it, reproduce the crash there, …)

it would be really nice of you to help users and developers alike by providing debug packages.
Comment by Phil Schaf (flying-sheep) - Wednesday, 15 October 2014, 12:17 GMT
i now have time to work on this.

the repo is https://projects.archlinux.org/dbscripts.git, right?

what’s needed to support debug symbol packages? how is this different from split packages? or isn’t it really, but it would be better to special-case debug-packages instead of making everything into split packages?
Comment by Phil Schaf (flying-sheep) - Thursday, 16 April 2015, 11:18 GMT
can someone give me any hint on what is needed to fix this?

or do you not want your users to help file bugs to upstream packages?
Comment by Kevin (kjslag) - Friday, 27 November 2015, 19:04 GMT
Allan explained what is needed in a forum post (https://bbs.archlinux.org/viewtopic.php?pid=1544964#p1544964):
1) devtools: alter to build debug packages by default and sign/upload them to our server with the other packages
2) dbscripts: recognise a debug package and add it to the foo-debug repo when the main package is being added to foo.
Comment by dx (dx) - Monday, 20 March 2017, 03:17 GMT
Some time ago these patches were posted by Allan McRae as a starting point:

https://lists.archlinux.org/pipermail/arch-projects/2015-August/004300.html
https://lists.archlinux.org/pipermail/arch-projects/2015-August/004301.html

I had a shot at fixing them - didn't get anywhere due to time constraints but I'm pushing what I have and documenting a bit so it's not forgotten.

https://github.com/dequis/arch-devtools/commits/debug
https://github.com/dequis/arch-dbscripts/commits/debug

This is *not* complete, and I'm not even sure it's correct. The devtools patch just had a syntax error, and that works as far as I can see, but dbscripts still needs a lot of work.

The devtools patch can be tested by building something with "extra-x86_64-build" followed by "extrapkg -s localhost" (extrapkg is an alias of commitpkg, and this rsyncs to localhost:~/staging )

dbscripts is a fairly obscure piece of the arch build system, only used by people who have commit access to the svn, but luckily none of that is needed because it includes a completely self-contained test suite which creates tiny svn repositories with a handful of packages, it's pretty cool. That was my main fear when getting in, that this might only be understood by arch devs, but I'm not so worried about that.

One of the problems with debug packages as opposed to split packages is that they are generated implicitly based on the existence of strippable binaries and the right combination of makepkg flags, instead of being announced beforehand as part of the PKGBUILD variables. I'm pretty sure all of this is run when built packages already exist, so in theory every place that reads PKGBUILD can be replaced with reading PKGINFO.

The other main problem is just putting the packages in the right places. If a function receives two package files for 'extra' and one of them is a debug package, should it have additional logic to put the debug package in 'extra-debug', or should it blame the caller for passing it packages for the wrong repos? Plenty of places where that decision should be made, and maybe different answers are more convenient in different places.

Might be a problem, but not sure: there are some packages in AUR that don't strictly fit the definition of "debug package" but their names end in -debug. For example clang-debug, flashplayer-standalone-debug or python-flask-debug. Should this sort of thing be forbidden? Should the code be written so that it never needs to blindly append a -debug to a package? That might not be possible. It would be a problem if you were packaging python-flask (which has no binaries) and it sent python-flask-debug to the debug repo instead of community.
Comment by Phil Schaf (flying-sheep) - Tuesday, 21 March 2017, 08:45 GMT
first of all: thank you so much for helping here. i would, too, but i have no idea how to get a testing environment set up (i’d assume that’d be some virtual machine hosting a repo server?)

hmm, the project name for python-flask-debug is “Flask-Debug”. We lowercase it anyway, so s/-/_/ wouldn’t be a prohibitive requirement IMHO. flashplayer-standalone-debug would be more correctly named “flashplayer-standalone-debugger” (see the download links on its webpage). finally i’m not sure about the purpose of the llvm-debug base-package (containing llvm-debug, clang-debug, …) but if it’s just clang/llvm plus debug symbols, it could be easily replaced with {clang,llvm,…}-debug packages containing only the debug symbols and depending on {clang,llvm,…}, respectively

another alternative would be using -dbg as suffix.
Comment by Alex Maystrenko (technic93) - Friday, 26 May 2017, 07:43 GMT
Hi, thank you for working on this feature, that I really would like to have in Arch linux. So I can contribute with testing.
Comment by marc boocha (marcthe12) - Thursday, 09 November 2017, 05:40 GMT
A another solution is to have a debug repo. Debug symbols are good for bug reports so it could be an opt in thing. And also testing repo must have debug package for obvious reasons
Comment by Cedric Bellegarde (gnumdk) - Saturday, 24 March 2018, 18:10 GMT
You also need to add support for dwz (and package it in archlinux).

Actual webkit2gtk-debug size => Around 4gio
On Fedora => 1 gio thanks to dwz support in RPM.
Comment by Brian G. (gissf1) - Friday, 16 November 2018, 04:23 GMT
From my understanding, there are 2 possible approaches to packaging debug information:

Method 1 - The one seemingly favored above, with code started, is creating 2 packages:
(1) a package with stripped binaries, as we do now for general distribution from the mirrors
(2) a package in a "*-debug" repo containing unstripped binaries with debug information embedded.

Method 2 - Similarly, creating 2 packages:
(1) a package with splitdebug binaries, for general distribution from the mirrors
(2) a package in a "*-debug" repo containing just the splitdebug symbols.

In method 1 the "*-debug" repo binaries become drop-in replacements for the stripped binaries, and are seemingly simple to generate. We just need to package once before stripping (the debug version), then again after (the stripped version).

But if we used debug repos like that, wouldn't the debug repo packages conflict with the non-debug versions? I see 2 potential issues happening:
(a) conflicting files in the packages - since the debug and non-debug packages contain the same filenames, this may also break dependencies from other packages.
(b) pacman may select packages from the enabled debug repo (or the non-debug repo) against the user's goals. The user may want debug versions of those packages, or not, and pacman may do the opposite. One can, of course, force a particular repo when installing like "pacman -S targetrepo/package", but what about in the default case?

Method 2 has additional benefits that the "debug packages" (containing only the split symbols) do not need to conflict with their non-debug counterparts, and, if anything, the "debug packages" can depend on the standard package via typical Arch package dependency methods. Another benefit to this is that the debug packages are smaller, since they can avoid repackaging the binaries themselves. In my experience, the size of the stripped/split binaries is generally significantly less than the size of the debug symbols, so the size aspect is likely a smaller benefit.

From another perspective, if we had splitdebug packages already made with the stripped binaries and the split out debug files in one package, a user could choose to skip installing debug symbols by simply using the "NoExtract" option in their /etc/pacman.conf. Obviously, this would cause an larger package download size, but it would work.

So building off that "skipping files" approach, we could build the package once with splitdebug enabled, then just package the debug symbols (by the filesystem path to the symbols themselves) independent of the rest of the package. So basically, the packaging tool would skip the path we used for symbols (so /usr/lib64/debug/* or similar) when packaging the standard binary package, then package only the /usr/lib64/debug/* files as the debug package.

Am I missing something as to why Method 1 is preferred?



Regarding the package naming conflicts, we could use a special separator on package names; maybe a double dash separator ("bash--debug") or underscores ("bash__debug") or something like that, but I imagine special cases in package names are not ideal either, so maybe we should use "-debugsymbols" or another non-conflicting suffix instead?



Regarding the "how to build a test environment" comment by Phil (flying-sheep), I think detailing this out would be a good starting point to getting people to provide more patches. If nobody but core Arch developers ever use the tools, it will be very hard to get new people in here with enough experience to help develop the tools.

I haven't tried, but I think dx provided enough detail to test the devtools tool (I have some idea what I would need to do to test it). For dbscripts, on the other hand, I'm still not sure what to do. Looking at the repo, I think it may be as simple as checking it out into some local path, running the test suite, and seeing what the resulting packages look like. Is this correct?



As a bit of background and a slight rant: I'm coming from a Gentoo background, but I run Arch on my smaller/older systems because they don't have the necessary power to rebuild everything from sources. I've ran across the need to debug programs several times on my Arch systems, where debug information is simply not available without doing the exact thing that made me avoid Gentoo and choose Arch to begin with. I realize that even makepkg support for splitdebug is rather new, so I'm not expecting it to manifest tomorrow, but hopefully we can make some progress sooner than 4 years from now. I don't have the time to dig into developing this functionality right now, but I may attempt it in the near future if I get some helpful feedback.

Thanks!
Comment by Eli Schwartz (eschwartz) - Friday, 16 November 2018, 05:02 GMT
makepkg has builtin support for "foo-debug" packages, as the initial feature request states. They contain only debugging symbols, and you still need the "foo" package for the actual binaries themselves. This is not new, it was implemented in 2013, but it's one of the lesser-known features. As such, there is absolutely zero possibility that we will agree to implement #1.

This feature request is intended to track the progress for implementing #2, adding the necessary tooling to track these packages and add them to [community-debug] style repositories.

Among other things, this requires:

- handling package bases which appear in multiple repositories
- handling the special case of debug packages, which are not in the ${pkgname[@]} array
- handling packages which may or may not exist, and removing the old version of a package that no longer exists

The last case would be made significantly easier by using git, which supports tags, so add in a migration from subversion to git...

Also see some documentation on existing work and brainstorming: https://wiki.archlinux.org/index.php/User:Gbs/dbscripts-status

Loading...