Release Engineering

Tasklist

FS#48382 - [archiso] build script not working since v24 update

Attached to Project: Release Engineering
Opened by Vincent Blazutti (VinZ) - Tuesday, 01 March 2016, 10:44 GMT
Last edited by Gerardo Exequiel Pozzi (djgera) - Saturday, 12 March 2016, 04:28 GMT
Task Type Bug Report
Category ArchISO
Status Closed
Assigned To Pierre Schmitz (Pierre)
Thomas Bächler (brain0)
Gerardo Exequiel Pozzi (djgera)
Architecture All
Severity High
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description:
Since the v24 update of archiso (adding gpg) the build script fails when entering make_setup_mkinitcpio().

Error :

./build.sh: line 83: gpg_key: unbound variable

Additional info:
Archiso v24

Steps to reproduce:
Try to build the releng config.
This task depends upon

Closed by  Gerardo Exequiel Pozzi (djgera)
Saturday, 12 March 2016, 04:28 GMT
Reason for closing:  Fixed
Additional comments about closing:  v25
Comment by Thomas Bächler (brain0) - Tuesday, 01 March 2016, 19:03 GMT
I thought I had caught those. The attached patch should help.
Comment by Q (compwiz3688) - Tuesday, 01 March 2016, 19:39 GMT
It seems that the conditions were not thoroughly checked when -g is not provided in both build.sh and /sbin/mkarchiso scripts:
1. build.sh did not initialize the $gpg_key variable. When coupled with set -u, this causes the error message you mentioned.
2. mkarchiso uses an "error if false" method to detect whether $gpg_key has data. When coupled with set -e, this causes the script to stop immediately when $gpg_key is not set.

Fix for #1: Add the line "gpg_key=" (without quotes) just after "out_dir=out" (near the top of the script).
Fix for #2: Since you are not using GPG signing anyway, just comment out the line starting with "[[ $gpg_key ]];". The actual fix is to use the if then block found in the build.sh.

Alternatively, if you still have v23 of archiso, downgrade to that instead until they have fixed these bugs for us non-gpg-signers.

To the developer: please see the attached for the diff.


EDIT: Whoops. I guess I took too long to post. My comment about #2 still stands though :(
Comment by Thomas Bächler (brain0) - Tuesday, 01 March 2016, 21:14 GMT
Those patches look good.
Comment by Gerardo Exequiel Pozzi (djgera) - Tuesday, 01 March 2016, 21:53 GMT
Yes, patches are right. I will merge them and release v25 in next hours.

PS: I am happy if you send in git format (git am). Thanks
Comment by David Marceau (archfan777) - Wednesday, 02 March 2016, 00:01 GMT
I applied the build.sh-v24.patch and the mkarchiso-v24.patch.
I had a few tweaks within the customize_airootfs.sh that functioned successfully on 20 previous other iterations with the earlier version of archiso.
I adapted to the situation and created a gpg key for use to build the iso.
After following the basic recipe to build an archiso, as root I usually run:
./build.sh -v
But this time as root, I ran using the suggested new convention:
su -c "GNUPGHOME=/home/david/.gnupg ./build.sh -v -g 3B2212D6"
Please note 3B2212D6 is my gpg id and not a secret/passphrase. In fact the build.sh never prompted me for a gpg passphrase during the process.
Shouldn't it ask me for the gpg passphrase associated with the provided gpg key id(3B2212D6)?

The archiso usually builds a dual-32bit(i686)/64-bit(x86_64) iso.
After the squashfs for i686, it gave this output:
[mkarchiso] INFO: Creating checksum file for self-test...Done!
[mkarchiso] INFO: Creating signature file...
gpg: WARNING: unsafe ownership on homedir '/home/david/.gnupg'
gpg: using "3B2212D6" as default secret key for signing
gpg: signing failed: Permission denied
gpg: signing failed: Permission denied

Then it brought be back to the root prompt #.
No iso file was generated.

I'm going to give it another try directly as root without su:
GNUPGHOME=/home/david/.gnupg ./build.sh -v -g 3B2212D6



Two questions:
-how do I successfully add my key 3B2212D6 to gpg-agent?
-how do I ensure gpg-agent is started before running the build.sh command?
gpg-connect-agent reloadagent /bye
tells me connection to agent established, but I never configured /home/david/.gnupg/gpg-agent.conf
in fact it doesn't exist.

I will try following command with version archiso-25 and let you know:
GNUPGHOME=/home/david/.gnupg ./build.sh -v -g 3B2212D6
Creating SquashFS
Done!
Creating checksum file for self-test
Done!
Creating signature file...
gpg: WARNING: unsafe ownership on homedir '/home/david/.gnupg'
gpg:using "3B2212D6" as default secret key for signing
gpg: signing failed: Operation cancelled
gpg: signing failed: Operation cancelled

How can I prevent that from happening? It is definitely related to the gpg-agent since I have not used it with archiso before v24/v25. I want to conform and use it but the gpg-agent setup instructions to do so need clarification.


It seems the following must be run as user david. It will not succeed with user root:
1)login as david
2)cd /home/david/archlive/
3)do your customized setup
4)run the build.sh as david through su:
su -c "GNUPGHOME=/home/david/.gnupg /home/david/archlive/build.sh -v -g 3B2212D6"
Comment by Gerardo Exequiel Pozzi (djgera) - Wednesday, 02 March 2016, 03:01 GMT
Patches added to git repo.

@david: You need to added your key to gpg-agent before build ;)
Comment by Gerardo Exequiel Pozzi (djgera) - Wednesday, 02 March 2016, 13:43 GMT
v25 is on [extra] please test.
Comment by Thomas Bächler (brain0) - Wednesday, 02 March 2016, 18:50 GMT
David, sadly, gpg-agent needs to be started as the user that owns the keyring BEFORE the build. Otherwise, things will fail.

As for those patches, I am pretty sure I fixed exactly those issues during testing, but it seems I forgot to pull the fixes from my VM to my main machine. Sorry about that.
Comment by David Marceau (archfan777) - Thursday, 10 March 2016, 14:08 GMT
  • Field changed: Percent Complete (100% → 0%)
Did not succeed when building iso with the provided gpg key.
Comment by Gerardo Exequiel Pozzi (djgera) - Thursday, 10 March 2016, 14:12 GMT
@David: Then your are doing something wrong. Looks like "su" is executed as "root", not as your "david".
Comment by Gerardo Exequiel Pozzi (djgera) - Thursday, 10 March 2016, 14:51 GMT
Indeed, for example I am building archiso inside Konsole from KDE, gpg-agent is not running at all, but is automagically launched on-demand (as root) and a dialog appears to enter the passphrase (pinentry-gtk), and all is good. If instead gpg-agent is already running as your user, no on-demand loading, passphrase is asked with (in my case pinentry-qt), and again all is good.
Comment by David Marceau (archfan777) - Thursday, 10 March 2016, 18:50 GMT
[david@archona-t-00300 ~] 2016-03-10 13:56:24 EST -0500
$ gpg --list-keys
/home/david/.gnupg/pubring.kbx
------------------------------
pub rsa2048/3B2212D6 2016-03-01 [SC]
uid [ultimate] David Marceau <davidmarceau@storagequest.com>
sub rsa2048/9342A3F9 2016-03-01 [E]

# uname -a
Linux archona-t-00300 4.4.3-1-ARCH #1 SMP PREEMPT Fri Feb 26 15:09:29 CET 2016 x86_64 GNU/Linux

# pacman -Ss archiso
extra/archiso 25-1 [installed]
Tools for creating Arch Linux live and install iso images

It seems the following must be run as user david. It will not succeed with user root:
1)login as david
2)cd /home/david/archlive/
3)do your customized setup
4)run the build.sh as david through su:
su -c "GNUPGHOME=/home/david/.gnupg /home/david/archlive/build.sh -v -g 3B2212D6"

I did do these 4 steps and it still failed. I have provided the output.
Comment by Gerardo Exequiel Pozzi (djgera) - Thursday, 10 March 2016, 23:19 GMT
you did not specified where are running. only in console?

I guess if you run the same command (1)... will fail, in that case, make things more easy, and leave "su" apart from the game (2) ;)

(1) $ su -c "GNUPGHOME=~david/.gnupg gpg --detach-sign --default-key 3B2212D6 testfile.txt"
(2) # GNUPGHOME=~david/.gnupg gpg --detach-sign --default-key 3B2212D6 testfile.txt

there is no need to preload gpg-agent, since is loaded on-demand, but all these things depends on enviroment vars.
Comment by David Marceau (archfan777) - Friday, 11 March 2016, 17:14 GMT
When creating your gpg key for the very first time,

Do not use:
gpg --gen-key
BECAUSE DOING SO DOES NOT CREATE A SECRET KEY.
When I did:
gpg --list-secret-keys
IT RETURNS EMPTY

Instead use:
gpg2 --full-gen-key
THIS ONE RETURNS A NON-EMPTY LIST OF SECRET KEYS.

Use the settings as detailed here for the creation only.
You may also edit the key to create a signing key only, but it's optional.
https://alexcabal.com/creating-the-perfect-gpg-keypair/


$ su -c 'GNUPGHOME=/home/david/.gnupg gpg -v -v --detach-sign --default-key 3D06072F testfile.txt'
Password:
gpg: WARNING: unsafe ownership on homedir '/home/david/.gnupg'
gpg: using PGP trust model
gpg: key 3D06072F: accepted as trusted key
gpg: using "3D06072F" as default secret key for signing
gpg: using subkey C49FA489 instead of primary key 3D06072F
File 'testfile.txt.sig' exists. Overwrite? (y/N) y
gpg: writing to 'testfile.txt.sig'
gpg: RSA/SHA256 signature from: "C49FA489 David Marceau <davidmarceau@storagequest.com>"

Event with the newer key that has a secret key associated with it, it is still failing when invoked with:
[david@archona-t-00300 ~/archlive] 2016-03-11 12:29:47 EST -0500
$ su -c 'GNUPGHOME=/home/david/.gnupg /home/david/archlive/build.sh -v -g 3D06072F'
[david@archona-t-00300 ~/archlive] 2016-03-11 12:53:42 EST -0500


gpg: WARNING: unsafe permissions on homedir '/tmp/mkinitcpio.Y2WHUM/root/gpg'
gpg: keybox '/tmp/mkinitcpio.Y2WHUM/root/gpg/pubring.kbx' created
gpg: /tmp/mkinitcpio.Y2WHUM/root/gpg/trustdb.gpg: trustdb created
gpg: key 3D06072F: public key "David Marceau <davidmarceau@storagequest.com>" imported
gpg: Total number processed: 1
gpg: imported: 1

[mkarchiso] INFO: Done!
[mkarchiso] INFO: Creating checksum file for self-test...
[mkarchiso] INFO: Done!
[mkarchiso] INFO: Creating signature file...
gpg: WARNING: unsafe ownership on homedir '/home/david/.gnupg'
gpg: using "3D06072F" as default secret key for signing
gpg: signing failed: Permission denied
gpg: signing failed: Permission denied



I have secret keys for that ID. Why does it say permission denied? One other thing to note. I am not using any desktop. Just a console. The gpg-agent got invoked successfully when I did gpg for the detach-sign suggestion above with the very same gpg key id.

$ gpg --list-secret-keys
/home/david/.gnupg/pubring.kbx
------------------------------
sec rsa4096/3D06072F 2016-03-11 [SC]
uid [ultimate] David Marceau <davidmarceau@storagequest.com>
ssb rsa4096/8C481D0C 2016-03-11 [E]
ssb rsa4096/C49FA489 2016-03-11 [S]
Comment by Gerardo Exequiel Pozzi (djgera) - Friday, 11 March 2016, 21:09 GMT
wow! crazy, please avoid using "su", and use root directly. (login as root, not su, not sudo, just root on plain console), this should work
Comment by David Marceau (archfan777) - Friday, 11 March 2016, 23:18 GMT
What command exactly do I need to run to make build.sh run successfuly with archiso-25 with a gnupg key?
I have tried:
1)as root:
/root/archlive/build.sh -v
didn't work permission denied.
2)as root:
/root/archlive/build.sh -v -g 3D06072F
didn't work permission denied.
3)as david:
/home/david/archlive/build.sh -v -g 3D06072F
but be root and gave me build.sh usage help
4)as david:
su -c 'bash -x /home/david/archlive/build.sh -v -g 3D06072F'
didn't work permission denied.


This asserts I have archiso installed:
$ pacman -Ss archiso
extra/archiso 25-1 [installed]
Tools for creating Arch Linux live and install iso images

As I stated before, I do want to use archiso-25, but for me the only archiso that works is archiso-23 as root with no gpgkey.
If I currently want to succeed at all with archiso I need to downgrade back to archiso-23.

I actually exported this in my /root/.bashrc and /home/david/.bashrc along with the actual script.

Here is my shortest test scenario to duplicate the error in less than 5 minutes:
This script is called testminimal with pgpsign:
+ killall -r gpg-agent
+ su -c 'rm -rf /home/david/archlive'
+ mkdir -p /home/david/archlive
+ cp -r /usr/share/archiso/configs/releng/airootfs /usr/share/archiso/configs/releng/build.sh /usr/share/archiso/configs/releng/efiboot /usr/share/archiso/configs/releng/isolinux /usr/share/archiso/configs/releng/mkinitcpio.conf /usr/share/archiso/configs/releng/packages.both /usr/share/archiso/configs/releng/packages.i686 /usr/share/archiso/configs/releng/packages.x86_64 /usr/share/archiso/configs/releng/pacman.conf /usr/share/archiso/configs/releng/syslinux /home/david/archlive
+ cd /home/david/archlive/
+ export GNUPGHOME=/home/david/.gnupg
+ GNUPGHOME=/home/david/.gnupg
+ su -c 'bash -x /home/david/archlive/build.sh -v -g 3D06072F'
+ set +x


Here are the commands as executed by mkarchiso on my box:
$ cat mkarchisoDebug.log
+ export LANG=C
+ LANG=C
+ app_name=mkarchiso
++ uname -m
+ arch=i686
+ pkg_list=
+ run_cmd=
+ quiet=y
+ pacman_conf=/etc/pacman.conf
++ date +%Y%m
+ export iso_label=ARCH_201603
+ iso_label=ARCH_201603
+ iso_publisher='Arch Linux <http://www.archlinux.org>'
+ iso_application='Arch Linux Live/Rescue CD'
+ install_dir=arch
+ work_dir=work
+ out_dir=out
+ sfs_mode=sfs
+ sfs_comp=xz
+ gpg_key=
+ [[ 0 -ne 0 ]]
+ getopts p:r:C:L:P:A:D:w:o:s:c:g:vh arg
+ case "${arg}" in
+ quiet=n
+ getopts p:r:C:L:P:A:D:w:o:s:c:g:vh arg
+ case "${arg}" in
+ work_dir=work
+ getopts p:r:C:L:P:A:D:w:o:s:c:g:vh arg
+ case "${arg}" in
+ install_dir=arch
+ getopts p:r:C:L:P:A:D:w:o:s:c:g:vh arg
+ case "${arg}" in
+ gpg_key=3D06072F
+ getopts p:r:C:L:P:A:D:w:o:s:c:g:vh arg
+ shift 7
+ [[ 1 -lt 1 ]]
+ command_name=prepare
+ case "${command_name}" in
+ command_prepare
+ _show_config prepare
+ local _mode=prepare
+ echo
+ _msg_info 'Configuration settings'
+ local '_msg=Configuration settings'
+ echo '[mkarchiso] INFO: Configuration settings'
+ _msg_info ' Command: prepare'
+ local '_msg= Command: prepare'
+ echo '[mkarchiso] INFO: Command: prepare'
+ _msg_info ' Architecture: i686'
+ local '_msg= Architecture: i686'
+ echo '[mkarchiso] INFO: Architecture: i686'
+ _msg_info ' Working directory: work'
+ local '_msg= Working directory: work'
+ echo '[mkarchiso] INFO: Working directory: work'
+ _msg_info ' Installation directory: arch'
+ local '_msg= Installation directory: arch'
+ echo '[mkarchiso] INFO: Installation directory: arch'
+ case "${_mode}" in
+ echo
+ _cleanup
+ _msg_info 'Cleaning up what we can on airootfs...'
+ local '_msg=Cleaning up what we can on airootfs...'
+ echo '[mkarchiso] INFO: Cleaning up what we can on airootfs...'
+ [[ -d work/airootfs/boot ]]
+ find work/airootfs/boot -type f -name '*.img' -delete
+ [[ -d work/airootfs/boot ]]
+ find work/airootfs/boot -type f -name 'vmlinuz*' -delete
+ [[ -d work/airootfs/var/lib/pacman ]]
+ find work/airootfs/var/lib/pacman -maxdepth 1 -type f -delete
+ [[ -d work/airootfs/var/lib/pacman/sync ]]
+ find work/airootfs/var/lib/pacman/sync -delete
+ [[ -d work/airootfs/var/cache/pacman/pkg ]]
+ find work/airootfs/var/cache/pacman/pkg -type f -delete
+ [[ -d work/airootfs/var/log ]]
+ find work/airootfs/var/log -type f -delete
+ [[ -d work/airootfs/var/tmp ]]
+ find work/airootfs/var/tmp -mindepth 1 -delete
+ find work '(' -name '*.pacnew' -o -name '*.pacsave' -o -name '*.pacorig' ')' -delete
+ _msg_info 'Done!'
+ local '_msg=Done!'
+ echo '[mkarchiso] INFO: Done!'
+ [[ sfs == \s\f\s ]]
+ _mkairootfs_sfs
+ [[ ! -e work/airootfs ]]
+ mkdir -p work/iso/arch/i686
+ _msg_info 'Creating SquashFS image, this may take some time...'
+ local '_msg=Creating SquashFS image, this may take some time...'
+ echo '[mkarchiso] INFO: Creating SquashFS image, this may take some time...'
+ [[ n = \y ]]
+ mksquashfs work/airootfs work/iso/arch/i686/airootfs.sfs -noappend -comp xz -no-progress
+ _msg_info 'Done!'
+ local '_msg=Done!'
+ echo '[mkarchiso] INFO: Done!'
+ _mkchecksum
+ _msg_info 'Creating checksum file for self-test...'
+ local '_msg=Creating checksum file for self-test...'
+ echo '[mkarchiso] INFO: Creating checksum file for self-test...'
+ cd work/iso/arch/i686
+ md5sum airootfs.sfs
+ cd /home/david/archlive
+ _msg_info 'Done!'
+ local '_msg=Done!'
+ echo '[mkarchiso] INFO: Done!'
+ [[ -n 3D06072F ]]
+ _mksignature
+ _msg_info 'Creating signature file...'
+ local '_msg=Creating signature file...'
+ echo '[mkarchiso] INFO: Creating signature file...'
+ cd work/iso/arch/i686
+ gpg --detach-sign --default-key 3D06072F airootfs.sfs


The mkarchiso gives this output:
gpg: signing failed: Permission denied
gpg: signing failed: Permission denied


BUT WHEN I RUN THESE COMMANDS DIRECTLY, I READ DIFFERENT ERROR MESSAGES:
cd /home/david/archlive/work/iso/arch/i686/

As user david WITHOUT su -c:
gpg --detach-sign --default-key 3D06072F airootfs.sfs
gpg: using "3D06072F" as default secret key for signing
gpg: can't create 'airootfs.sfs.sig': Permission denied
gpg: signing failed: Permission denied
[david@archona-t-00300 ~/testgnupg] 2016-03-11 17:42:42 EST -0500

As user david WITH su -c:
su -c 'gpg -v -v --detach-sign --default-key 3D06072F airootfs.sfs'
Password:
gpg: WARNING: unsafe ownership on homedir '/home/david/.gnupg'
gpg: using PGP trust model
gpg: key 3D06072F: accepted as trusted key
gpg: using "3D06072F" as default secret key for signing
gpg: using subkey C49FA489 instead of primary key 3D06072F
gpg: writing to 'airootfs.sfs.sig'
gpg: RSA/SHA256 signature from: "C49FA489 David Marceau <davidmarceau@storagequest.com>"
[david@archona-t-00300 ~/testgnupg] 2016-03-11 17:42:48 EST -0500

So I am perplexed.
I am user david, I su the build.sh script which effectively makes me root.
but the gpg fails because it doesn't have permission sign a file(airootfs.sfs) owned by root. gpg also seems to not have permission to create a new file in a directory owned by root. gpg would be happier in a directory owned by user david. Archiso tells me the error, but in no way provides me any suggestion as to rectify it. THIS IS THE BUG AND IT STILL PERSISTS.

Yet as user david, I su the gpg sign the exact same file directly and I succeed.
export GNUPGHOME=/home/david/.gnupg
For both scenarios it was important to have GNUPGHOME set in advance.

UPDATE:
You said to "just copy the pgpkey to the root profile and be happy."

I did:
rm -rf /root/.gnupg/
cp -Rf /home/david/.gnupg/ /root/
export GNUPGHOME=/root/.gnupg
cd /root/archlive
./build.sh -v -g 3D06072F
SUCCESS!!! Thank you. I'm happy.
Comment by Gerardo Exequiel Pozzi (djgera) - Saturday, 12 March 2016, 01:48 GMT
Looks like you are mixing old scripts. Just build "releng" profile provided by archiso package, not your own custom profile...

All you need to do from console (tty1) is as root (not sudo su -, not su -, not sudo, not su, or any kind of combinations) and no gpg-agent should be running as the user "david", gpg-agent will be executed on-demand and then, passphrase is asked via pinentry ncurses interface.
# GNUPGHOME=/home/david/.gnupg /usr/share/archiso/configs/releng/build.sh -g 3D06072F

or just copy the pgpkey to the root profile and be happy.

PS: In any way, I suggest to use forums to support request, you have something broken and forums is a better place for this. reading large messages mixing messages and code is not funny.

Loading...