FS#75855 - [qemu-user-static] "chrooting" into arm container fails with bash not found error

Attached to Project: Arch Linux
Opened by solsTiCe (zebul666) - Thursday, 08 September 2022, 23:27 GMT
Last edited by David Runge (dvzrv) - Saturday, 01 October 2022, 11:34 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To David Runge (dvzrv)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Whatever the method (arch-chroot, systemd-nspawn), trying to launch an arm container, ends up with an error about /bin/bash not found error:

# systemd-nspawn -M eupheme --bind-ro /etc/resolv.conf -D raspios-bullseye-armhf-lite-eupheme/
Spawning container eupheme on /var/lib/machines/raspios-bullseye-armhf-lite-eupheme.
Press ^] three times within 1s to kill container.
execv(/bin/bash, /bin/bash, /bin/sh) failed: No such file or directory
Container eupheme failed with error code 1.

It used to work fine before the new qemu-user-static package in [extra]

Nothing as changed (that seems to me relevant) apart the binfmt-qemu-static package that I deleted and the new qemu-user-static
This task depends upon

Closed by  David Runge (dvzrv)
Saturday, 01 October 2022, 11:34 GMT
Reason for closing:  Fixed
Additional comments about closing:  Fixed with qemu 7.1.0-6 (only one provider of qemu-user-binfmt-provider may be installed at a time).
Comment by solsTiCe (zebul666) - Thursday, 08 September 2022, 23:50 GMT
When I switch back to the AUR package qemu-user-static-7.1.0-1 and binfmt-qemu-static-20220907, it works fine

Using the linux-zen kernel, if that matters
Comment by Tobias Powalowski (tpowa) - Saturday, 10 September 2022, 07:49 GMT
Just a guess, is your binfmt.d setup correct?
My archboot arm containers work fine with qemu-user-static.
Comment by David Runge (dvzrv) - Saturday, 10 September 2022, 09:04 GMT
@zebul666: Thanks for the ticket!

Do you see any significant difference between your previous (AUR based) binfmt.d setup and the current one?
Maybe there is something we can improve?
Comment by solsTiCe (zebul666) - Saturday, 10 September 2022, 09:44 GMT
I don't know much about that. I just use the package from AUR and voilà. Never thought about it.

So I looked at the difference, and the last flag field seems different (like C instead of CF). So I changed them and restarted systemd-bindfmt.service. But it changed nothing. May be, I need to reinstall the binfmt or reboot ?

But if it works for tpowa...
I don't have time to look more into that right now
Comment by Tobias Powalowski (tpowa) - Saturday, 10 September 2022, 10:41 GMT
You need to restart your system to see an effect. My experience systemd caches this somehow.
Ah and for completeness my setup uses aarch64 static.
Comment by David Runge (dvzrv) - Saturday, 10 September 2022, 11:30 GMT
According to the kernel documentation [1] maybe it could make sense to add P (preserve argv[0]).
Using the C (credentials) flag looks rather dangerous tbh.

The /usr/lib/binfmt.d/qemu-*-static.conf files currently only use the F flag (fix binary).

@zebul666: Can you please try whether using FP as flags works for you (you will have to reboot after changing the file(s))?


[1] https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html
Comment by Sebastian Gsänger (hein09) - Saturday, 10 September 2022, 17:54 GMT
I've had the same or a similar issue, for me it was because the binfmt_misc mechanism insisted on using the non-static binaries.

Both qemu-user and qemu-user-static now provide binfmt configuration on their own.
While that's a good thing in itself, it doesn't make sense to have the config files in parallel as only one set can be picked up by the kernel.

The only way i could enforce using the qemu-user-static binaries via binfmt_misc was uninstalling qemu-user (and the qemu-emulator-full and qemu-full packages with it).

Would it make sense to have the packages being alternatives to each other?
Comment by solsTiCe (zebul666) - Saturday, 10 September 2022, 20:15 GMT
Yes. this is it.

If I forcefully remove qemu-user (with pacman -R -dd qemu-user), and only use qemu-user-static (from [extra]) and restart systemd-binfmt.service, this works!

I don't know why this used to work with the package from the AUR (qemu-user-static+binfmt-qemu-static) and qemu-user, while, now, you only can get it to work with only qemu-user-static (from [extra]) installed but not with qemu-user installed too.

Comment by Sven-Hendrik Haase (Svenstaro) - Monday, 19 September 2022, 12:51 GMT
I found exactly the same problem on my system and eventually doing what zebul666 suggested worked indeed.
Comment by David Runge (dvzrv) - Friday, 30 September 2022, 08:44 GMT
I think I'll split out the binary format rules into qemu-user-binfmt and qemu-user-static-binfmt.
Both will provide and conflict qemu-user-binfmt-provider, so that only one of them can be installed.
Comment by David Runge (dvzrv) - Friday, 30 September 2022, 09:30 GMT
Please check whether qemu 7.1.0-6 in [testing] fixes your issue.
Comment by David Runge (dvzrv) - Saturday, 01 October 2022, 11:33 GMT
According to @Svenstaro, 7.1.0-6 in [testing] works and fixes this issue.

Loading...