FS#69869 - [linux] PKGBUILD - support git bisect

Attached to Project: Arch Linux
Opened by James (thx1138) - Wednesday, 03 March 2021, 14:39 GMT
Last edited by Buggy McBugFace (bugbot) - Saturday, 25 November 2023, 20:14 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Tobias Powalowski (tpowa)
Jan Alexander Steffens (heftig)
David Runge (dvzrv)
Levente Polyak (anthraxx)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 3
Private No

Details

In the linux PKGBUILD, at:
```
# remove build and source links
rm "$modulesdir"/{source,build}
```
instead, this needs to be:
```
# remove build and source links
rm "$modulesdir"/{source,build} || true
```
to allow successive `git bisect` builds to complete the install stage.
This task depends upon

Closed by  Buggy McBugFace (bugbot)
Saturday, 25 November 2023, 20:14 GMT
Reason for closing:  Moved
Additional comments about closing:  https://gitlab.archlinux.org/archlinux/p ackaging/packages/linux/issues/8
Comment by Jan Alexander Steffens (heftig) - Thursday, 04 March 2021, 18:28 GMT
This looks wrong. It shouldn't be an issue because makepkg cleans the pkgdir before starting the build.
Comment by James (thx1138) - Friday, 05 March 2021, 02:22 GMT
Is the pkgdir cleaned when running `makepkg -ef`?

man PKGBUILD

"pkgdir ... This variable should only be used in the package() function."

Does the PKGBUILD documentation need to be changed?


```
_package-headers() {
...
local builddir="$pkgdir/usr/lib/modules/$(<version)/build"
...
mkdir -p "$pkgdir/usr/src"
ln -sr "$builddir" "$pkgdir/usr/src/$pkgbase"
}
```

```
_package-docs() {
...
local builddir="$pkgdir/usr/lib/modules/$(<version)/build"
...
mkdir -p "$pkgdir/usr/share/doc"
ln -sr "$builddir/Documentation" "$pkgdir/usr/share/doc/$pkgbase"
}
```
Comment by loqs (loqs) - Friday, 05 March 2021, 15:00 GMT
Yes makepkg cleans $pkgdir when `makepkg -ef` is used.

The following maps the packaging functions to match $pkgbase so $pkgdir is only being used in the package function. As it is a split package producing three packages it has three package functions.

pkgname=("$pkgbase" "$pkgbase-headers" "$pkgbase-docs")
for _p in "${pkgname[@]}"; do
eval "package_$_p() {
$(declare -f "_package${_p#$pkgbase}")
_package${_p#$pkgbase}
}"
done

You might make the changes in the attached diff if you were bisecting mainline:
change git used to pick up tags
pkgver function
disable building and packaging docs
disable saving scmversion in prepare so kernel can update version internally
Edit:
Missed from the diff disable the call to make htmldocs
Comment by James (thx1138) - Saturday, 06 March 2021, 01:17 GMT
Ok - thanks for that. I also commented-out the `make htmldocs` in the build function. That is actually a quite time-consuming process. And, for what I was searching, I didn't need the headers package either.

Hmm - I tried running git bisect again, without the `|| true`, to see what was happening.

So, `makepkg -e`, remember, does *not* run the prepare() function. And, in this PKGBUILD, the prepare() function writes a file named "version".
```
make -s kernelrelease > version
echo "Prepared $pkgbase version $(<version)"
```

This is the initial starting point for the git bisect. Of course, in this particular instance, the file "version" contains the text "5.11.2-arch1-1". This file's content is now invariant, simply because it is not written again in the build() function or in the package() function. But also remember, the idea here is to run git bisect, and the actual "kernelrelease" could change to anything, depending upon the mix of files chosen by git bisect.

So, in this PKGBUILD, the package() function has one idea about "modulesdir", based upon the previously saved file "version",
```
local kernver="$(<version)"
local modulesdir="$pkgdir/usr/lib/modules/$kernver"
```

and then, the subsequent linux "make ... modules_install" has its own idea about the kernel version, based upon the actual version being built under git bisect. In this case, this produces the following result:

```
$ ll /sdc/build/linux/pkg/linux/usr/lib/modules/*
/sdc/build/linux/pkg/linux/usr/lib/modules/5.11.2-arch1-1:
total 8904
drwxr-xr-x 1 james james 28 Mar 5 12:10 .
drwxr-xr-x 1 james james 50 Mar 5 12:10 ..
-rw-r--r-- 1 james james 6 Mar 5 12:10 pkgbase
-rw-r--r-- 1 james james 9113568 Mar 5 12:10 vmlinuz

/sdc/build/linux/pkg/linux/usr/lib/modules/5.9.0-rc5-1:
total 5680
drwxr-xr-x 1 james james 464 Mar 5 12:12 .
drwxr-xr-x 1 james james 50 Mar 5 12:10 ..
lrwxrwxrwx 1 james james 36 Mar 5 12:10 build -> /sdc/build/linux/src/archlinux-linux
drwxr-xr-x 1 james james 100 Mar 5 12:11 kernel
-rw-r--r-- 1 james james 1364437 Mar 5 12:12 modules.alias
-rw-r--r-- 1 james james 1343736 Mar 5 12:12 modules.alias.bin
-rw-r--r-- 1 james james 8202 Mar 5 12:10 modules.builtin
-rw-r--r-- 1 james james 0 Mar 5 12:12 modules.builtin.alias.bin
-rw-r--r-- 1 james james 10465 Mar 5 12:12 modules.builtin.bin
-rw-r--r-- 1 james james 61360 Mar 5 12:10 modules.builtin.modinfo
-rw-r--r-- 1 james james 650909 Mar 5 12:12 modules.dep
-rw-r--r-- 1 james james 880089 Mar 5 12:12 modules.dep.bin
-rw-r--r-- 1 james james 455 Mar 5 12:12 modules.devname
-rw-r--r-- 1 james james 207156 Mar 5 12:10 modules.order
-rw-r--r-- 1 james james 1017 Mar 5 12:12 modules.softdep
-rw-r--r-- 1 james james 561695 Mar 5 12:12 modules.symbols
-rw-r--r-- 1 james james 688897 Mar 5 12:12 modules.symbols.bin
lrwxrwxrwx 1 james james 36 Mar 5 12:10 source -> /sdc/build/linux/src/archlinux-linux
```

And, this gives:
```
makepkg -efs
...
INSTALL virt/lib/irqbypass.ko
DEPMOD 5.9.0-rc5-1
rm: cannot remove '/sdc/build/linux/pkg/linux/usr/lib/modules/5.11.2-arch1-1/source': No such file or directory
rm: cannot remove '/sdc/build/linux/pkg/linux/usr/lib/modules/5.11.2-arch1-1/build': No such file or directory
==> ERROR: A failure occurred in package_linux().
Aborting...
```

In my case, no dkms modules were being built during testing with git bisect, so the left-over build and source links were of no consequence. And, luckily, I didn't need the vmlinuz and pkgbase files, then missing in the resulting package. Still, it would be nice to get this cleaned-up.

It may be simplest to re-run `make -s kernelrelease` in the package() function, to properly place the vmlinux and pkgbase files, and to remove the build and source links.
Comment by loqs (loqs) - Saturday, 06 March 2021, 02:58 GMT
What if you move the two lines from prepare() to build()? See attachment.
Comment by James (thx1138) - Saturday, 06 March 2021, 03:58 GMT
Yes, I suppose that something like that should work, presuming that `make -s kernelrelease` does the right thing when running before `make all` actually runs. In the message, reported to the user, the verb should be changed from "Prepared" to "Building". Or, `make all` could run first, and the subsequent message could be changed to say "Built ...".

It would be courteous to add comments before part of `makedepends=()`, `pkgname=()`, and `make htmldocs`, to the effect "Comment-out for git bisect" or "Use instead for git bisect".

I don't know that it is strictly necessary to use the kernel.org git repository instead of the archlinux.org git repository, as suggested in the first PKGBUILD.diff. I don't know how they differ, but I didn't have any problems there, simply using the default git.archlinux.org.

Also, after running a complete series of git bisect, /usr/lib/modules/ could end-up with the cruft of several different version directories, but I don't think that is a concern that the PKGBUILD need address.

Jan will decide what changes to make.
Comment by loqs (loqs) - Saturday, 06 March 2021, 04:20 GMT
https://git.archlinux.org/linux.git lacks upstream tags so I could not use v5.11 in the PKGBUILD for the initial build or pass it to git bisect.
Comment by James (thx1138) - Saturday, 06 March 2021, 04:29 GMT
It would be useful to keep your alternate kernel.org source in the PKGBUILD as a comment. But still, I get a seemingly complete list of tags from `git tag`, for the Arch Linux versions. For instance, for 5.11, `git tag` shows:
```
...
v5.11-arch1
v5.11-arch2
v5.11.1-arch1
v5.11.2-arch1
...
```
Were you needing something different? Do I misunderstand what you wanted?
Comment by Constantine (Hi-Angel) - Saturday, 10 April 2021, 10:31 GMT
> I get a seemingly complete list of tags from `git tag`, for the Arch Linux versions. For instance, for 5.11, `git tag` shows: […]

As someone who did bisection this week, I'd point out that it's not actually complete, which I think was the loqs' point. For one, the tags list is missing `rc` releases, and most often bugs are introduced in rc1 or rc2.

With that said, adding another remote after initial dir was cloned is not a big deal, just enter `src/archlinux-linux` and do a `git remote add torvalds https://github.com/torvalds/linux && git fetch torvalds`.
Comment by James (thx1138) - Saturday, 01 May 2021, 18:02 GMT
...
DEPMOD 5.11.0-1
rm: cannot remove '/sdc/build12/linux/pkg/linux/usr/lib/modules/5.12.0-arch1-1/source': No such file or directory
rm: cannot remove '/sdc/build12/linux/pkg/linux/usr/lib/modules/5.12.0-arch1-1/build': No such file or directory
==> ERROR: A failure occurred in package_linux().
Aborting...

Still annoying in v5.12-arch1, when I forget.

For git bisect, the simplest revision seems to be just repeating `make -s kernelrelease > version` at the beginning of the build() function, after `cd $_srcname`. There is no reason to remove this from the prepare() function.

`git describe` cannot be used for pkgver(), which can give:

$ git describe
fatal: No tags can describe 'f40ddce88593482919761f74910f42f4b84c004b'.
Try --always, or create some tags.

The use of the "pkgdir" variable still violates the rule given in "man 5 PKGBUILD".
Comment by Buggy McBugFace (bugbot) - Tuesday, 08 August 2023, 19:11 GMT
This is an automated comment as this bug is open for more then 2 years. Please reply if you still experience this bug otherwise this issue will be closed after 1 month.

Loading...