FS#50280 - makepkg: Add feature to run prepare() when using --noextract

Attached to Project: Pacman
Opened by Christoph Haag (haagch) - Saturday, 06 August 2016, 11:13 GMT
Task Type Feature Request
Category makepkg
Status Unconfirmed
Assigned To No-one
Architecture All
Severity Low
Priority Normal
Reported Version 5.0.1
Due in Version Undecided
Due Date Undecided
Percent Complete 0%
Votes 1
Private No

Details

I may not be using makepkg as intended here, but here is an example:

You have the linux-git package directory and in src/linux you play around a bit. Add remotes. Check out specific branches.
Then you want to compile it and have incremental builds working. Problem is: prepare() does some very useful stuff for when your playing around caused to switch to another version like setting CONFIG_LOCALVERSION and running make config again.

Some time ago makepkg -e would run prepare() and everything was fine but currently there seems to be no way to have makepkg NOT extract the sources BUT run prepare().

I personally would like the old behavior back and people who want --noextract without running prepare() could still use --noextract --noprepare together.
This task depends upon

Comment by Allan McRae (Allan) - Saturday, 06 August 2016, 11:16 GMT
Why not set CONFIG_LOCALVERSION and running make config in build()?
Comment by mirh (mirh) - Wednesday, 25 July 2018, 13:24 GMT
Indeed, I was having the same question (for as much as use case is light years different)
https://git.archlinux.org/pacman.git/tree/scripts/makepkg.sh.in#n1651

Why no*extract* is killing everything short of build()?
It should just wrap extract_sources imo.

If then somebody wants to skip "re-checking sources" (aka "using existing tree").. What is there that --skipinteg and --noprepare don't already accomplish?
Comment by Eli Schwartz (eschwartz) - Wednesday, 25 July 2018, 15:22 GMT
Because prepare() is meant for doing things that touch up the extracted sources in ways which should have happened in the source itself.

It's meant for ensuring that --noextract is idempotent, if you run makepkg --noextract multiple times e.g. to continue a build which you stopped with CTRL+C.

Hence Allan's question, what is the motivation for moving things which belong in build() into prepare() then claiming makepkg should always no matter what run prepare() -- which defeats the purpose of having such a function IMO.
Comment by mirh (mirh) - Wednesday, 25 July 2018, 17:27 GMT
Yes, absolutely, but sometimes needed sources don't all come in a single one neat archive.
And I, for one, didn't want them all cluttered together inside the root of srcdir.

Ergo I skip bsdtar (even because it's not able to recognize alone the "improper" lzma compression used by a file I have) and handle the thing myself.
Continuing a build I stopped is exactly the use case I have.. and it seems stupid having to use noextract= directive for every single file, for just this situation.

I'm not sure who claimed prepare should *always* run then. I just said --noprepare exist exactly for that.
And since extraction happens before and outside of it, it didn't feel right to have them intertwined.
Comment by mirh (mirh) - Saturday, 28 July 2018, 21:37 GMT
I just now noticed [the functions called by] extract_sources to be *also* responsible for copying stuff from startdir to srcdir.
So.. I think I have understood the reasoning behind this naming choice.

Similarly to somebody wanting to save time by skipping prepare() though, I'm not sure why doing the same with extraction would be any less legit.
What about something similar to SOURCEONLY double setting, but for NOEXTRACT?
Comment by Eli Schwartz (eschwartz) - Sunday, 29 July 2018, 04:26 GMT
So, your PKGBUILD literally doesn't work unless you use --noprepare?

That seems... unwise.

I know some packages which use noextract=("${source[@]%%::*}") for this precise reason.

...

I don't generally consider --noprepare to be advised, it is sort of advanced usage and, I guess, useful for cases where you want to skip downstream patching/fixing for edge-case reasons. It definitely should not be something you use by default even for one specific PKGBUILD.
Comment by mirh (mirh) - Sunday, 29 July 2018, 11:18 GMT
It actually very much requires prepare to run.
It's just that when I'm testing the thing and I'm tinkering with very sizeable archives, it starts to become a PITA to wait minutes.

I guess like that noextract oneliner would already do it for me then, thanks.

Though it still feel like some "option"'s still left missing - see OP point.
I'm not really knowledgeable on kernel internals.. but even arch's official linux package does that thing in prepare, for example.
Comment by Eli Schwartz (eschwartz) - Sunday, 29 July 2018, 14:37 GMT
Ah, okay, it wasn't entirely clear what you're trying to do so I'll take your word for it. :)

Happy to help on the noextract front.

As for the kernel thing, everything it does there is something that should be done exactly once in an idempotent manner before building, so I think it makes sense. I'm not sure why modifying the extracted sources further should affect this, TBH, e.g. CONFIG_LOCALVERSION should not need to be changed and make config is something I'd expect to see happening manually while outside of makepkg. The use case of completely switching the development branch e.g. between stable/mainline seems sort of outside the domain of makepkg, not sure we should be handling this in a generic manner... I don't know, maybe it makes sense at least for that use case (linux-git) to do this in build() instead.

One thing to note: It's been proposed before, IIRC, that our different --no* flags should have options to re-enable these features, the obvious use case would be that by default prepare() is *not* (and should not be) run during --noextract, but one could explicitly opt in to that behavior "for $reasons" using --prepare or re-enable options baked into makechrootpkg by default, or similar.
Comment by Christoph Haag (haagch) - Sunday, 29 July 2018, 15:32 GMT
Sorry, I haven't really replied. My example is just one of many possible ones.

The general issue is about more fine grained control over what functions makepkg runs. I have no problems with --noextract implying a default setting of --noprepare, as you say, a switch --prepare/--run-prepare or whatever would be perfectly fine for me.

There's also other areas where I sometimes wish there was more fine grained control, like "extracting" sources from a git source, but not fetching the remote is not possible. So if you want to build a git package while the remote is down, you always have to wait for the connection timeout. (Again, just an example, no need to really discuss it).
Comment by Eli Schwartz (eschwartz) - Sunday, 29 July 2018, 15:46 GMT
In fact, --holdver allows you to skip fetching new commits/changesets from the remote

Loading...